mcp: run tool calls in worker threads - #312
Merged
Merged
Conversation
bhcopeland
force-pushed
the
mcp-offload-tool-calls
branch
4 times, most recently
from
September 3, 2026 10:35
704d043 to
759b821
Compare
Member
|
rebase |
bhcopeland
force-pushed
the
mcp-offload-tool-calls
branch
from
September 17, 2026 08:57
759b821 to
a1f0044
Compare
Member
Author
|
Rebased |
Member
|
rebase, this conflicts with your previous PR |
FastMCP awaits an async tool but calls a sync one inline on the event loop. Every tool here is sync and makes a blocking request, so calls were serialised across all sessions: measured against a running server, a trivial list_trees took 58.66s instead of 0.14s while one slow call was in flight. Register an async wrapper that hands the call to a worker thread. The dashboard tools funnel through one registration site and inherit this; the Maestro tools are wrapped individually, so a new one needs the decorator adding. Concurrency also makes the per-call stdout redirect unsafe, as it mutates a global that overlapping calls can restore out of order, so that becomes a single redirect for the life of the server, entered inside the stdio transport once it has taken stdout for the protocol writer. anyio is imported where it is used, since kcidev.main imports every subcommand at startup and anyio comes only with the mcp extra. Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
bhcopeland
force-pushed
the
mcp-offload-tool-calls
branch
from
September 17, 2026 09:28
a1f0044 to
326ed9c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
FastMCP awaits an async tool but calls a sync one inline on the event loop. Every tool here is sync and makes a blocking request, so calls were serialised across all sessions: measured against a running server, a trivial list_trees took 58.66s instead of 0.14s while one slow call was in flight.
Register an async wrapper that hands the call to a worker thread. The dashboard tools funnel through one registration site and inherit this; the Maestro tools are wrapped individually, so a new one needs the decorator adding.
Concurrency also makes the per-call stdout redirect unsafe, as it mutates a global that overlapping calls can restore out of order, so that becomes a single redirect for the life of the server, entered inside the stdio transport once it has taken stdout for the protocol writer. anyio is imported where it is used, since kcidev.main imports every subcommand at startup and anyio comes only with the mcp extra.