Skip to content

MCP Tools in Drills: drill-check-resync-readiness, drill_validate-for-execution, drill_resource_add-or-update, drill_run_mark-complete - #3370

Open
dynamicdhx wants to merge 3 commits into
mainfrom
users/dhruvbharuka/drillresync-readiness-check
Open

MCP Tools in Drills: drill-check-resync-readiness, drill_validate-for-execution, drill_resource_add-or-update, drill_run_mark-complete#3370
dynamicdhx wants to merge 3 commits into
mainfrom
users/dhruvbharuka/drillresync-readiness-check

Conversation

@dynamicdhx

Copy link
Copy Markdown
Contributor

What does this PR do?

[Provide a clear, concise description of the changes]

[Add additional context, screenshots, or information that helps reviewers]

GitHub issue number?

[Link to the GitHub issue this PR addresses]

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Created a changelog entry if the change falls among the following: new feature, bug fix, UI/UX update, breaking change, or updated dependencies. Follow the changelog entry guide
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes running the script ./eng/scripts/Process-PackageReadMe.ps1. See Package README
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For tools with new names, including new tools or renamed tools, update consolidated-tools.json
    • For renamed tools, follow the Tool Rename Checklist and tag the PR with the breaking-change label
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated command list in servers/Azure.Mcp.Server/docs/azmcp-commands.md
    • Ran ./eng/scripts/Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
    • Updated test prompts in servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.

@dynamicdhx

dynamicdhx commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

Tool Description Evaluator Results

Test 1

Expected Tool: resilience_drill_check-resync-readiness
Prompt: Check whether resilience drill <drill_name> in service group <service_group> is ready to resync

Results

Rank Score Tool Status
1 0.768303 resilience_drill_check-resync-readiness EXPECTED
2 0.606843 resilience_recoveryplan_checkreadiness
3 0.583951 resilience_drill_get
4 0.491690 resilience_drill_resource_get
5 0.472516 resilience_drill_run_get

Test 2

Expected Tool: resilience_drill_check-resync-readiness
Prompt: Run a resync readiness check for resilience drill <drill_name> in service group <service_group>

Results

Rank Score Tool Status
1 0.798056 resilience_drill_check-resync-readiness EXPECTED
2 0.606457 resilience_drill_get
3 0.555740 resilience_recoveryplan_checkreadiness
4 0.533586 resilience_drill_run_get
5 0.509845 resilience_drill_resource_get

Test 3

Expected Tool: resilience_drill_validate-for-execution
Prompt: Validate resilience drill <drill_name> in service group <service_group> for execution from source location <source_location>

Results

Rank Score Tool Status
1 0.679885 resilience_drill_validate-for-execution EXPECTED
2 0.622777 resilience_drill_get
3 0.602468 resilience_drill_check-resync-readiness
4 0.566701 resilience_drill_resource_get
5 0.525479 resilience_drill_resource_add-or-update

Test 4

Expected Tool: resilience_drill_validate-for-execution
Prompt: Preflight resilience drill <drill_name> in service group <service_group> to confirm it is ready to run from source locations <source_locations>

Results

Rank Score Tool Status
1 0.645025 resilience_drill_validate-for-execution EXPECTED
2 0.603578 resilience_drill_check-resync-readiness
3 0.546839 resilience_drill_get
4 0.533188 resilience_recoveryplan_checkreadiness
5 0.501034 resilience_drill_resource_get

Test 5

Expected Tool: resilience_drill_resource_add-or-update
Prompt: Add resource <resource_id> to resilience drill <drill_name> in service group <service_group> with a fault duration of <fault_duration_minutes> minutes

Results

Rank Score Tool Status
1 0.730711 resilience_drill_resource_add-or-update EXPECTED
2 0.527114 resilience_drill_get
3 0.516581 resilience_drill_resource_get
4 0.444488 resilience_drill_check-resync-readiness
5 0.437515 resilience_drill_validate-for-execution

Test 6

Expected Tool: resilience_drill_resource_add-or-update
Prompt: Update or exclude the resources of resilience drill <drill_name> in service group <service_group>

Results

Rank Score Tool Status
1 0.641648 resilience_drill_resource_add-or-update EXPECTED
2 0.601242 resilience_drill_get
3 0.601011 resilience_drill_resource_get
4 0.536848 resilience_recoveryplan_resource_update
5 0.496929 resilience_drill_run_resource_get

Test 7

Expected Tool: resilience_drill_run_mark-complete
Prompt: Mark the FaultInjection stage of drill run <drill_run_name> for drill <drill_name> in service group <service_group> as complete

Results

Rank Score Tool Status
1 0.672075 resilience_drill_run_mark-complete EXPECTED
2 0.450026 resilience_drill_run_get
3 0.411140 resilience_drill_resource_add-or-update
4 0.393840 resilience_drill_check-resync-readiness
5 0.377590 resilience_drill_run_resource_get

Test 8

Expected Tool: resilience_drill_run_mark-complete
Prompt: Complete the fault injection stage of drill run <drill_run_name> for drill <drill_name> in service group <service_group> so the drill run can proceed

Results

Rank Score Tool Status
1 0.630925 resilience_drill_run_mark-complete EXPECTED
2 0.503606 resilience_drill_run_get
3 0.470063 resilience_drill_check-resync-readiness
4 0.461755 resilience_drill_resource_add-or-update
5 0.445459 resilience_drill_get

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new Azure Resilience Management drill operation to start a resync readiness check, wiring it through the tool command surface, service layer, JSON serialization (AOT), and documentation and tests so it is available through azmcp and the Azure MCP Server tool catalog.

Changes:

  • Added resilience drill check-resync-readiness command (and mapped tool resilience_drill_check-resync-readiness) plus service implementation and result model.
  • Added unit and tool-level tests for validation, success, and RequestFailedException message sanitization, and updated recorded-test assets tag.
  • Updated Azure MCP Server docs and metadata: consolidated-tools mapping, azmcp-commands.md, e2eTestPrompts.md, and a changelog entry.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/ResilienceManagementCommandTests.cs Adds an end-to-end tool invocation test for the new drill resync readiness check.
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/Drills/DrillCheckResyncReadinessCommandTests.cs Adds unit tests covering required option validation, happy path, invalid path segments, and RequestFailedException sanitization.
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/assets.json Updates the recorded test assets tag to the new snapshot.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/ResilienceManagementService.cs Implements CheckDrillResyncReadinessAsync using the ARM drill resource operation.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/IResilienceManagementService.cs Adds the new service contract method.
tools/Azure.Mcp.Tools.ResilienceManagement/src/ResilienceManagementSetup.cs Registers the new command and adds it to the drill command group.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Options/Drills/DrillCheckResyncReadinessOption.cs Introduces the options POCO for the new command.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Models/DrillResyncReadinessResult.cs Adds a result model for operation ID and completion state.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/ResilienceManagementJsonContext.cs Registers new result types for source-generated JSON serialization.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/Drills/DrillCheckResyncReadinessCommand.cs Adds the new tool command, validation, and error message shaping.
servers/Azure.Mcp.Server/src/Resources/consolidated-tools.json Maps the new drill readiness check tool into the consolidated “readiness” tool and updates its description.
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md Adds prompt entries for the new tool.
servers/Azure.Mcp.Server/docs/azmcp-commands.md Adds the new CLI command documentation block.
servers/Azure.Mcp.Server/changelog-entries/dhruvbharuka-drill-check-resync-readiness.yml Adds a changelog entry for the new tool.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +131 to +135
var drills = new CommandGroup("drill", "Resilience drill operations - Commands for listing, getting, and checking resync readiness of resilience drills for an Azure service group.");
resilienceManagement.AddSubGroup(drills);

drills.AddCommand<DrillGetCommand>(serviceProvider);
drills.AddCommand<DrillCheckResyncReadinessCommand>(serviceProvider);
Comment on lines +14 to +18
[CommandMetadata(
Id = "9136d433-50df-4e08-bc6a-660881d39421",
Name = "check-resync-readiness",
Title = "Check Resilience Drill Resync Readiness",
Description = "Checks whether a resilience drill is ready to resync in a service group. Starts a resync and readiness check that resyncs the drill's configuration and evaluates whether the drill is ready, then returns the operation ID for the started check. Use this to run a resync readiness check and confirm drill readiness before running the drill.",
…ecord all four drill tool live tests

Clubs the drill resource add-or-update and drill run mark-complete MCP tools into this PR alongside resync-readiness and validate-for-execution. Updates recordings tag to Azure.Mcp.Tools.ResilienceManagement.Tests_d8b926b9e3 covering all four recorded integration tests.
@dynamicdhx dynamicdhx changed the title MCP Tools in Drills: Drill Resync Readiness Check MCP Tools in Drills: drill-check-resync-readiness, drill_validate-for-execution, drill_resource_add-or-update, drill_run_mark-complete Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants