Repository navigation
Open
Labels
needs-live-provider-verificationParked pending human verification with real cloud provider credentialsParked pending human verification with real cloud provider credentials
Description
Why
The current probe layer (src/main/probes/) only knows Anthropic. The catalog has 18 entries but probeProvider() returns error: No probe implemented for "<id>" for any provider whose probeKind is aws, gcp, or azure.
Bedrock and Vertex don't expose a cheap "validate this credential" HTTP endpoint the way OpenAI/Anthropic do — there's no /v1/models you can hit. The standard check is to call a tiny STS-shaped API and parse the response. Three axes:
- Bedrock — call
bedrock:ListFoundationModels(orbedrock:ListCustomModels) with the user's AWS credentials (default chain). 403/expired token ≠ working. - Vertex — call
aiplatform.googleapis.com/v1/projects/<project>/locations/global/publishers/google/modelswith the configured ADC + project + region. TheenvKeys[]for Vertex is empty today so we need to surface a "Project + Region required" step in the wizard before this can be wired. - Azure —
GET <endpoint>/openai/deployments?api-version=2024-02-01with the user'sAZURE_OPENAI_API_KEY. Or just probe the deployment the user names.
Scope
- Wire
awsprobe kind: uses SDK's default credential provider chain; minimal IAM sigv4 againstbedrock-runtimeorbedrockcontrol plane; returnsvalid | errorwith the AWS error code surfaced indetail. - Wire
gcpprobe kind: takes aprojectandregionfrom wizard config; uses@google-cloud/aiplatformor just an authenticated RESTGET; requiresgcloud auth application-default loginupstream. - Wire
azureprobe kind (andazure-openai): simpleGETagainst the deployment list with the configured key. - Vault entry for Vertex needs to carry
{project, region}alongside the credential. Either widenSecretWriteOptionsor surface project/region as separate UI fields in the Keys step. - Update
PROVIDER_CATALOGnotes for cloud_creds rows to indicate "wizard will ask for project/region first". - Renderer Keys step: when user picks Vertex, ask for project id + region before "Save"; persist into vault alongside the credential (or as a sibling secret).
- App + CLI both pick up the same probes via the shared
runProbe()dispatcher.
Acceptance criteria
-
runProbe({ providerId: 'bedrock' })returns{ valid: true, status: 'ok' }against a working AWS account with Bedrock access. - Same returns
{ valid: false, status: 'invalid', detail: 'AWS error: ...' }for an unconfigured credential chain. - Vertex probe requires a saved project + region; errors sensibly without one.
- Azure probe works against an actual deployment.
- All three appear in the
Validatebutton in the renderer without code changes beyond the probe files. - Tests (or fixture-mocked HTTP) for at least the error paths.
-
npm run typecheck && npm run lint && npm run buildclean.
References
- ADR-0003 (
docs/adr/0003-key-validity-and-expiry-probing.md) - AWS SigV4 reference: https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html
- Vertex
aiplatform.googleapis.com: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/rest - Azure OpenAI REST: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference
Metadata
Metadata
Assignees
Labels
needs-live-provider-verificationParked pending human verification with real cloud provider credentialsParked pending human verification with real cloud provider credentials
Activity
johnnyhuy commentedon Nov 8, 2026
Split for review size: the probe layer (
src/main/probes/aws/gcp/azure kinds, ProbeContext project/region, catalog notes, mocked-HTTP tests) is being worked as #47 on branchagent/issue-47-cloud-probes. The vault storage for Vertex {project, region} and the renderer Keys step fields stay on this issue for a later run. CLI parity incli/src/lib/probes.tswill be a separate PR (independent release flow).johnnyhuy commentedon Nov 8, 2026
Progress from the work loop - the probe layer landed in #48 (closes #47), and the vault and renderer slice for Vertex project and region is up in #50 (closes #49). Still open on this issue - CLI probe parity in cli/src/lib/probes.ts as a separate CLI PR, and live-key verification against real Bedrock and Vertex accounts.
johnnyhuy commentedon Nov 8, 2026
CLI probe parity landed in #52 (the CLI half of this issue, split out as #51). The app probe layer from #48, the vault wiring from #50 and the CLI dispatch now cover Bedrock, Vertex and Azure with mocked HTTP tests on both sides. What remains here needs live provider accounts: a Bedrock probe against a working AWS account, a Vertex probe against a real GCP project, and an Azure probe against a real deployment. Parking the live verification with this note rather than closing, since no agent run can supply those credentials.
johnnyhuy commentedon Nov 12, 2026
The scheduled run rechecked origin/main at 93c2a42. The implementation slices are already merged in #48, #50 and #52. The remaining live-provider acceptance checks need Johnny or a reviewer with working cloud accounts. This run made no live provider calls and used no real credentials.
Added
needs-live-provider-verificationso this issue is explicitly parked and stays open. Existing implementation limits still apply as recorded in #48 and #52: Bedrock reads environment credentials rather than the full AWS SDK default chain, and Vertex takes an access token rather than loading ADC itself. The checks below do not establish live-provider compatibility.Both repository gates passed against the exact origin/main commit.
mise run ci:app- exit 0; catalog check, typecheck, lint, all unit suites and the main/preload/renderer builds passed.mise run ci:cli- exit 0; typecheck, 25 tests with 0 failures, and the Linux x64 compiled binary build passed.git diff --check- exit 0; worktree clean after both gates.Representative proof from the app gate, using mocked HTTP only:
The CLI gate passed
aws mocked 200 returns ok and sends a SigV4 signed request,gcp mocked 200 returns ok with project and region in the path,azure missing endpoint returns error, mocked 200 validates, and the 16 discovery-cache fixture cases.No user-facing change, new commit or PR was needed in this run. The dependency install reported 23 audit findings (12 moderate, 9 high, 2 critical); no dependency versions were changed.
Sweep completed with 0 merged PRs, 0 closed PRs, 0 remote branches deleted and 4 leftover local agent branches deleted after confirming their work had merged. Release PR #46 is excluded from the sweep. The schedule remains active because the exit condition requires a clear PR list and #46 is still open.
johnnyhuy commentedon Nov 13, 2026
Scheduled run reverified origin/main at 93c2a42 on 13 November 2026. No user-facing change or new PR was needed. #16 and #17 are closed; this issue remains explicitly parked with needs-live-provider-verification.
Both repository gates passed with exit 0.
mise run ci:apppassed catalog freshness, typecheck, lint, all unit suites and main/preload/renderer builds.mise run ci:clipassed typecheck, 25 tests with 0 failures and the Linux x64 compiled binary build.git diff --checkpassed and the worktree remained clean.Representative proof from the app gate:
CLI proof includes
aws mocked 200 returns ok and sends a SigV4 signed request,gcp mocked 200 returns ok with project and region in the path,azure missing endpoint returns error, mocked 200 validatesand all 16 discovery-cache fixture tests:No real cloud credentials were used and no live provider calls were made. Johnny or a reviewer with working cloud accounts still needs to perform the acceptance checks. Existing credential-source limits remain as recorded in #48 and #52.
Sweep completed with 0 merged, 0 closed and 0 deleted branches. Only release PR #46 remains open and was excluded from the sweep. The schedule stays active because the exit condition requires a clear PR list.
The app dependency install still reports 23 audit findings (12 moderate, 9 high, 2 critical). No dependencies were changed.
johnnyhuy commentedon Nov 13, 2026
Reverified origin/main at 93c2a42 on 13 November 2026. No user-facing change or new PR was needed. The implementation is already merged in #48, #50 and #52; #16 and #17 are closed.
Both repository gates passed with exit 0.
mise run ci:apppassed catalog freshness, typecheck, lint, all unit suites and the main/preload/renderer builds.mise run ci:clipassed typecheck, all 25 tests and the Linux x64 compiled binary build.git diff --checkpassed; the worktree is clean.Representative app proof from mocked HTTP fixtures:
CLI proof includes
aws mocked 200 returns ok and sends a SigV4 signed request,gcp mocked 200 returns ok with project and region in the path,azure missing endpoint returns error, mocked 200 validatesand all 16 discovery-cache fixture tests.No real provider credentials or live provider calls were used. This issue stays parked with
needs-live-provider-verificationuntil Johnny or a reviewer can perform the live acceptance checks. Existing credential-source limits remain as recorded in #48 and #52.Sweep completed with 0 merged PRs, 0 closed PRs and 0 deleted branches. Only release PR #46 remains open and was excluded from the sweep. The schedule stays active because the exit condition requires a clear PR list. No release commands were run.
The app dependency install still reports 23 audit findings (12 moderate, 9 high, 2 critical). No dependencies were changed.
johnnyhuy commentedon Nov 13, 2026
Scheduled run verified origin/main at
93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 13 November 2026. No user-facing change or new PR was needed. #16 and #17 are closed; #15 stays parked withneeds-live-provider-verificationfor Johnny or a reviewer with cloud accounts.Both repository gates passed with exit 0.
mise run ci:apppassed catalog freshness, typecheck, lint, all unit suites and the main/preload/renderer builds.mise run ci:clipassed typecheck, 25 tests and the Linux x64 compiled binary build.git diff --checkpassed and the worktree is clean.Representative proof from the app gate, using mocked responses only:
CLI proof includes
aws mocked 200 returns ok and sends a SigV4 signed request,gcp mocked 200 returns ok with project and region in the path,azure missing endpoint returns error, mocked 200 validates, and the discovery-cache fixture tests.No real provider credentials or live provider calls were used. Credential-source limitations remain as previously recorded: Bedrock reads environment credentials rather than the full SDK default chain; Vertex takes an access token rather than loading ADC itself.
Sweep completed with 0 merged PRs, 0 closed PRs and 0 deleted branches. Only release PR #46 remains open and was excluded. The schedule stays active because the exit condition requires a clear PR list. No release commands were run.
The app install reported the existing 23 dependency audit findings (12 moderate, 9 high, 2 critical). No dependency versions were changed.
johnnyhuy commentedon Nov 13, 2026
Reverified origin/main at
93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 13 November 2026. No user-facing change or new PR was needed. This issue remains parked pending live-provider checks by Johnny or a reviewer with working cloud accounts.Both repository gates passed with exit 0.
mise run ci:apppassed catalog freshness, typecheck, lint, all unit suites and main/preload/renderer builds.mise run ci:clipassed typecheck, all 25 tests and the Linux x64 compiled binary build.git diff --checkpassed and the worktree is clean.Representative app proof from mocked HTTP fixtures:
CLI proof includes
aws mocked 200 returns ok and sends a SigV4 signed request,gcp mocked 200 returns ok with project and region in the path,azure missing endpoint returns error, mocked 200 validatesand all 16 discovery-cache fixture cases.No real credentials or live provider calls were used. The previously documented credential-source limits remain. Implementation PRs #48, #50 and #52 are merged.
Sweep completed with 0 merged PRs, 0 closed PRs and 2 stale local agent branches deleted after confirming they were ancestors of origin/main. Release PR #46 remains excluded from the sweep. The schedule remains active because the exit condition requires a clear PR list. No release commands were run.
The app dependency install reported 23 audit findings (12 moderate, 9 high, 2 critical). No dependencies were changed.
johnnyhuy commentedon Nov 13, 2026
Reverified
origin/mainat93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 13 November 2026. No user-facing change or new PR was needed.mise run ci:apppassed with exit 0, including catalog freshness, typecheck, lint, all unit suites and the app builds.mise run ci:clipassed with exit 0, including typecheck, 25 tests, 0 failures and the Linux x64 binary build.git diff --checkpassed and the worktree stayed clean.Mocked provider proof from the gates:
No live provider calls or real credentials were used. This issue remains parked for a reviewer with working cloud accounts; the previously documented credential-source limitations remain.
Sweep completed with 0 merged, 0 closed and 0 branches deleted. Release PR #46 remains excluded. The schedule stays active because the pause condition requires a clear PR list. No release commands were run.
johnnyhuy commentedon Nov 13, 2026
Reverified origin/main at
93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 14 November 2026. No user-facing change, commit or new PR was needed; the implementation slices are already merged.mise run ci:apppassed with exit 0, covering catalog freshness, typecheck, lint, all unit suites and the main/preload/renderer builds.mise run ci:clipassed with exit 0, covering typecheck, 25 tests with 0 failures and the Linux x64 compiled binary build.git diff --checkpassed; the worktree stayed clean.Representative proof from mocked provider tests:
No real credentials or live provider calls were used. #15 remains parked with
needs-live-provider-verificationfor Johnny or a reviewer with cloud accounts. The existing credential-source limits remain: Bedrock reads environment credentials rather than the full SDK default chain; Vertex takes an access token rather than loading ADC itself.Sweep completed with 0 merged PRs, 0 closed PRs and 0 branches deleted. Release PR #46 remains excluded; the schedule stays active because the pause condition requires a clear PR list. No release commands were run. The app dependency install reported the existing 23 audit findings; no dependencies were changed.
johnnyhuy commentedon Nov 13, 2026
Verified origin/main at
93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 14 November 2026. No user-facing change, commit or PR was needed; #16 and #17 are closed and this issue remains parked withneeds-live-provider-verification.Both repository gates passed with exit 0.
mise run ci:apppassed catalog freshness, typecheck, lint, all unit suites and main/preload/renderer builds.mise run ci:clipassed typecheck, all 25 tests with 0 failures and the Linux x64 compiled binary build.git diff --checkpassed and the worktree stayed clean.Representative mocked-response proof from this run:
No live provider calls or real credentials were used. Johnny or a reviewer with cloud accounts still needs to perform the live acceptance checks. The existing credential-source limitations remain as recorded in #48 and #52.
Sweep completed with 0 merged PRs, 0 closed PRs and 1 stale local agent branch deleted after confirming it was an ancestor of origin/main. Release PR #46 remains excluded. The schedule stays active because the pause condition requires a clear PR list. No release commands were run. The app dependency install reported the existing 23 audit findings; no dependencies were changed.
johnnyhuy commentedon Nov 14, 2026
Reverified origin/main at
93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 14 November 2026. No user-facing change, commit or new PR was needed. #16 and #17 are closed; this issue stays parked withneeds-live-provider-verificationfor Johnny or a reviewer with cloud accounts.Both repository gates passed with exit 0.
mise run ci:apppassed catalog freshness, typecheck, lint, all unit suites and main/preload/renderer builds.mise run ci:clipassed typecheck, all 25 tests with 0 failures and the Linux x64 compiled binary build.git diff --checkpassed; the worktree is clean.Representative proof from mocked responses only:
No real credentials or live provider calls were used. Existing credential-source limitations remain as recorded in #48 and #52. Live acceptance checks still need a reviewer with working accounts.
Sweep completed with 0 merged PRs, 0 closed PRs and 1 stale local agent branch deleted after confirming it was an ancestor of origin/main. Release PR #46 remains excluded. The schedule stays active because the pause condition requires a clear PR list. No release commands were run.
johnnyhuy commentedon Nov 14, 2026
Reverified origin/main at 93c2a42 on 14 November 2026. Both repository gates passed with exit 0.
mise run ci:apppassed catalog freshness, typecheck, lint, all unit suites and the app builds.mise run ci:clipassed typecheck, 25 tests with 0 failures, and the Linux x64 compiled binary build.git diff --checkpassed; the worktree is clean.Mocked-response proof includes
aws mocked 200 returns ok,aws Authorization uses SigV4 credential scope,aws mocked 403 returns invalid,gcp missing config makes no HTTP request,gcp mocked 200 returns ok,gcp request path carries project and region,azure mocked 200 returns ok, andazure request sends the api-key header. All cloud probe, probe context and discovery cache suites passed. No real credentials or live provider calls were used.No user-facing change, commit or new PR was needed. This issue stays parked with needs-live-provider-verification for Johnny or a reviewer with working cloud accounts. Existing credential-source limitations remain as recorded in #48 and #52.
Sweep completed with 0 merged PRs, 0 closed PRs and 0 remote branches deleted. Removed the temporary local agent branch created during setup. Release PR #46 remains excluded. The schedule stays active because the pause condition requires a clear PR list. No release commands were run. The app install reported the existing 23 dependency audit findings; no dependencies were changed.
johnnyhuy commentedon Nov 14, 2026
Reverified origin/main at
93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 14 November 2026. No user-facing change, commit or new PR was needed. #16 and #17 are closed; #15 remains parked withneeds-live-provider-verification.Both repository gates passed with exit 0.
mise run ci:apppassed catalog freshness, typecheck, lint, all unit suites and main/preload/renderer builds.mise run ci:clipassed typecheck, all 25 tests and the Linux x64 compiled binary build.git diff --checkpassed; the worktree stayed clean.Representative proof from this run, using mocked HTTP fixtures only:
No live provider calls or real credentials were used. Johnny or a reviewer with working cloud accounts still needs to perform live acceptance checks. The credential-source limitations recorded in #48 and #52 remain.
Sweep completed with 0 merged PRs, 0 closed PRs and 0 branches deleted. Release PR #46 remains excluded. The schedule stays active because the pause condition requires a clear PR list. No release commands were run. The app dependency install reported the existing 23 audit findings; no dependencies were changed.
johnnyhuy commentedon Nov 14, 2026
Reverified
origin/mainat93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 14 November 2026. No user-facing change, commit or new PR was needed. #16 and #17 are closed. This issue remains parked withneeds-live-provider-verificationpending Johnny or a reviewer with working cloud accounts.Both repository gates passed with exit 0.
mise run ci:apppassed catalog freshness, typecheck, lint, all unit suites and the main/preload/renderer builds.mise run ci:clipassed typecheck, all 25 tests and the Linux x64 compiled binary build.git diff --checkpassed and the worktree stayed clean.Proof from this run, using mocked HTTP responses only:
No real provider credentials or live provider calls were used. The credential-source limits recorded in #48 and #52 remain.
Sweep completed with 0 merged PRs, 0 closed PRs and 0 deleted branches. Release PR #46 remains excluded from the sweep. The schedule stays active because the exit condition requires a clear PR list. No release commands were run.
johnnyhuy commentedon Nov 14, 2026
Reverified origin/main at
93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 15 November 2026. No user-facing change, commit or new PR was needed. #16 and #17 are closed; #15 remains parked withneeds-live-provider-verificationfor Johnny or a reviewer with working cloud accounts.Both repository gates passed with exit 0.
mise run ci:apppassed catalog freshness, typecheck, lint, all unit suites and main/preload/renderer builds.mise run ci:clipassed typecheck, all 25 tests and the Linux x64 compiled binary build.git diff --checkpassed and the worktree stayed clean.Proof from this run, using mocked HTTP responses only:
No real provider credentials or live provider calls were used. The credential-source limits recorded in #48 and #52 remain. Live compatibility is still unverified.
Sweep completed with 0 merged PRs, 0 closed PRs and 0 deleted branches. Only release PR #46 remains open and was excluded. The schedule stays active because the exit condition requires a clear PR list. No release commands were run. The dependency install reported the existing 23 audit findings; no dependency versions were changed.
johnnyhuy commentedon Nov 14, 2026
Reverified origin/main at
93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 15 November 2026. No user-facing change or new PR was needed.mise run ci:apppassed with exit 0, including all unit suites and the app builds.mise run ci:clipassed with exit 0, including 25 tests, 0 failures and the Linux x64 binary build.git diff --checkpassed and the worktree is clean.Mocked HTTP proof included
aws mocked 200 returns ok,aws Authorization uses SigV4 credential scope,gcp missing config makes no HTTP request,gcp request path carries project and region, andazure request sends the api-key header. No real credentials or live provider calls were used.This issue remains parked for a reviewer with cloud accounts. Sweep completed with 0 merged, 0 closed and 0 branches deleted. Only excluded release PR #46 remains open, so the schedule pause condition is not met. No release commands were run.
johnnyhuy commentedon Nov 15, 2026
Reverified origin/main at
93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 15 November 2026. No user-facing change, commit or new PR was needed. #16 and #17 are closed; this issue remains parked withneeds-live-provider-verificationfor Johnny or a reviewer with working cloud accounts.mise run ci:apppassed with exit 0, including catalog freshness, typecheck, lint, all unit suites and the main/preload/renderer builds.mise run ci:clipassed with exit 0, including typecheck, 25 tests with 0 failures, and the Linux x64 compiled binary build.git diff --checkpassed; the worktree is clean.Proof from mocked HTTP fixtures only:
No real credentials or live provider calls were used. Live compatibility remains unverified. The credential-source limits recorded in #48 and #52 remain.
Sweep completed with 0 merged PRs, 0 closed PRs and 0 deleted branches. Only excluded release PR #46 remains open, so the schedule pause condition is not met. No release commands were run. The dependency install reported the existing 23 audit findings; no dependency versions were changed.
johnnyhuy commentedon Nov 15, 2026
Reverified origin/main at
93c2a420c5c1e63f687e64f1cf0681b14b7dfaf2on 15 November 2026. No user-facing change or new PR was needed.mise run ci:apppassed with exit 0, including catalog freshness, typecheck, lint, all unit suites and the main/preload/renderer builds.mise run ci:clipassed with exit 0, including 25 tests, 0 failures and the Linux x64 binary build.git diff --checkpassed and the worktree is clean.Mocked HTTP proof included
aws mocked 200 returns ok,aws Authorization uses SigV4 credential scope,gcp missing config makes no HTTP request,gcp request path carries project and regionandazure request sends the api-key header. No live provider calls or real credentials were used. Live compatibility still needs a reviewer with working cloud accounts; this issue remains parked withneeds-live-provider-verification.Sweep completed with 0 merged, 0 closed and 0 branches deleted. Only excluded release PR #46 remains open, so the schedule pause condition is unmet. No release commands were run. The dependency install reported the existing 23 audit findings; no dependencies were changed.