Add portable exact-run SCIM validation reports (depends on authentication)

Microsoft.SCIM.LogicAppValidationTemplate/README.md

@@ -25,9 +25,11 @@ The authentication readiness policy is **AND across selected methods**: every de
 
 For OAuth-only or federation-only runs, set `scimBearerToken` to `""`. The shared token workflow acquires access tokens, and each test suite refreshes its cache before expiry. There is no static-token fallback. Sending an OAuth access token in an HTTP Bearer header does not mean the endpoint supports caller-supplied static tokens. Entra provisioning connectivity is configured and validated separately with the supported method.
 
+At startup the agent creates a unique version/timestamp/run-ID subfolder directly under its starting current directory. Every run, including setup-blocked runs, exports `validation-run-details.json`, a Markdown summary, and optionally a portable archive using [Export-ScimValidationReport.ps1](StandardLogicApp/Export-ScimValidationReport.ps1), with versioned shareable copies. These contain exact-run timing, source/deployed version, deployment provenance, environment/operator context, and sanitized diagnostics with explicit evidence gaps. Tokens and credentials remain protected; unexposed model/runtime settings are not invented. Review reports before sharing because organizational identifiers and test-user data may remain.
+
 See the [Standard setup walkthrough](StandardLogicApp/SetupLogicApp-Standard-Agent.md) and [test overview](StandardLogicApp/SCIM-Validation-Test-Overview.md) for the current behavior. The generic legacy test descriptions below do not override the Standard authentication contract.
 
-The [authentication change guide](StandardLogicApp/Authentication-Changes.md) explains capability selection, token transport, workflow limits, and verification boundaries. Portable reporting is a separate follow-up change.
+The [authentication change guide](StandardLogicApp/Authentication-Changes.md) describes the prerequisite authentication change. The [validation-reporting guide](StandardLogicApp/Validation-Reporting-Changes.md) describes this dependent reporting change, evidence scope, provenance, data transformations, and limitations.
 
 ---
 

Microsoft.SCIM.LogicAppValidationTemplate/StandardLogicApp/Orchestrator_Parameters.json

@@ -58,7 +58,7 @@
       "description": "Overall bundle and agent version, copied from the same-bundle VERSION file."
     },
     "type": "String",
-    "value": "14.0"
+    "value": "15.0"
   },
   "attributeTransforms": {
     "metadata": {

Microsoft.SCIM.LogicAppValidationTemplate/StandardLogicApp/SCIM-Validation-Test-Overview.md

@@ -177,17 +177,29 @@ This applies to: Create_Group, Update_Group, Group_Update_Add_Member, Group_Upda
 | Group tests `FAILED` (no Group schema) | **Failed** | Must add Group support to SCIM endpoint |
 | Any mandatory test `FAILED` | **Failed** | Must fix failing test(s) before submission |
 
-## Authentication regression checks
+## Shareable evidence and regression checks
 
-Run `.\Test-Authentication.ps1` after changing authentication behavior.
-The suite evaluates the actual workflow expressions and direct-request
-token-cache wiring. Live federation still requires a federation-capable
-environment; offline exchange parity is not a live authentication result.
+The [canonical agent](scim-onboarding.agent.md) creates a version/timestamp/local-ID subfolder under its starting current directory before validation. It records VERSION and source hashes, selected methods, and setup-step timings. Its report step uses [Export-ScimValidationReport.ps1](Export-ScimValidationReport.ps1) to produce `validation-run-details.json`, `validation-report.md`, and an optional portable archive for the **exact** run ID, plus versioned shareable copies.
 
-See [Authentication-Changes.md](Authentication-Changes.md) for the design
-and the boundary between authentication readiness and functional readiness.
+Evidence includes UTC workflow/test/action/repetition timings, nested failures and retries, sanitized HTTP requests/responses, source/deployment provenance, and environment/operator identifiers. Protected token payloads remain excluded. Unavailable runtime/model information and missing evidence are explicitly reported, never guessed. Review the artifacts before sharing: redaction removes credentials, not all organizational identifiers or test-user data.
+
+Generate the final report after the exact Orchestrator run and its child workflows have reached terminal states. A report collected while the run is active is an incomplete snapshot, not a final readiness result. Detailed evidence collection can take longer than a short Azure CLI access-token lifetime; collection failures must remain visible until the missing evidence has been retrieved successfully.
+
+To resume an interrupted collection, stop or verify completion of the previous process and rerun the same exact-run invocation without `-ResumeReportPath`; the matching sanitized checkpoint is loaded automatically. To import a prior final report explicitly, add `-ResumeReportPath "$WorkDirectory\validation-run-details.json"`. The exporter reuses successful sanitized terminal payloads, refreshes metadata, and retries missing evidence. Run only one collector per work directory. Resuming does not turn missing evidence into a pass. Until-loop history uses the container timing/counts and supported nested action repetitions rather than an unsupported loop repetition endpoint.
+
+Run the dependency-free PowerShell regressions after changes:
+
+```powershell
+.\Test-Authentication.ps1
+.\Test-ScimValidationReport.ps1
+```
+
+The authentication suite evaluates the actual workflow expressions, including OAuth-only, WIF-only, alternative-method failures, legacy inference, non-authentication failures, and every direct SCIM request's cache/transport wiring. Live WIF testing still requires a WIF-capable environment; offline exchange parity is not a live authentication result.
+
+For reporting scope, provenance, data transformations, and root-cause analysis, see [Validation reporting changes](Validation-Reporting-Changes.md). The prerequisite authentication behavior is documented separately in [Authentication changes](Authentication-Changes.md).
 
 ---
+
 ## How to Run
 
 ### Method A: AI Agent (Automated, 30–60 min)

Microsoft.SCIM.LogicAppValidationTemplate/StandardLogicApp/SetupLogicApp-Standard-Agent.md

@@ -434,19 +434,39 @@ The agent clears undeclared credentials and reports those tests as
 declared method must be supplied or reported as blocked; they are not
 silently treated as an unsupported method.
 
-### Authentication readiness
+### Authentication readiness and run evidence
 
 Every selected authentication method must pass its own test (AND).
-Selecting both OAuth and WIF requires both to pass; a selected failure,
-skip, or missing result blocks authentication readiness. An undeclared
-method is Not applicable and issues no authentication requests.
-Other mandatory functional failures remain independent readiness gates.
-
-Use one coherent bundle: seven workflows, the complete parameters file,
-the canonical agent, authentication tests, and [VERSION](VERSION).
-Keep `LogicAppVersion` synchronized with VERSION. This change does not
-require a portable report exporter; reporting is a separate follow-up.
-See [Authentication-Changes.md](Authentication-Changes.md).
+Selecting both OAuth and WIF requires both to pass; one passing method
+cannot hide a selected failure, skipped result, or missing result.
+Other mandatory functional failures still prevent readiness.
+For example, a WIF-only integration with a passing federation test and
+no OAuth endpoint is authentication-ready; OAuth is Not applicable.
+
+Use one coherent agent/template bundle: seven workflows, the complete
+parameters file, `VERSION`, and the report exporter from the same source revision.
+Do not replace locally updated templates with older downloads.
+
+The [VERSION](VERSION) file is the sole overall and agent version.
+Bump it for each bundle change and keep `LogicAppVersion` synchronized.
+Before asking for inputs, the agent creates a unique subfolder under
+its starting current directory, for example
+`scimrun-v15.0-20261016T170000Z-a1b2c3d4`. The short name includes the
+version, UTC start time, and local run ID; metadata records full Azure
+identifiers. Final shareable filenames also include the exact Azure
+run ID. Stable helper filenames remain available within that folder.
+
+For every outcome, including setup blockers, the agent exports a
+sanitized **`validation-run-details.json`** and human-readable report in
+the run work directory, with an optional portable archive. These include
+the exact run ID, source/deployed version, source/deployment hashes, selected authentication
+methods, timings, environment/operator identifiers, failure diagnostics,
+and explicit evidence gaps. Token payloads remain protected. Unexposed
+model, reasoning, and context settings are recorded as unavailable.
+Review the artifacts before sharing because organizational identifiers
+and test-user data may remain. See the
+[canonical agent's report step](scim-onboarding.agent.md#step-6g-export-a-shareable-run-report-required-for-every-outcome).
+
 ### Question 4: Azure Subscription Selection
 
 > “Which Azure subscription would you like to use?”

Microsoft.SCIM.LogicAppValidationTemplate/StandardLogicApp/Test-Authentication.ps1

@@ -282,6 +282,28 @@ Assert-Equal $authenticationTests.Count 2 'The existing report inventory retains
 foreach ($test in $authenticationTests) {
     Assert-Equal $test.testCategory 'Declared/StrictSuccess' 'Each declared authentication test is individually strict'
 }
+$agentText = Get-Content -LiteralPath "$root\scim-onboarding.agent.md" -Raw
+$startup = [regex]::Match($agentText, '(?s)```powershell\r?\n(?<code>.*?)\r?\n```')
+Assert-Equal $startup.Success $true 'Canonical agent contains executable startup-folder instructions'
+$testDirectory = Join-Path ([IO.Path]::GetTempPath()) ('scim-startup-' + [guid]::NewGuid().ToString('N'))
+$null = New-Item -ItemType Directory -Path $testDirectory -ErrorAction Stop
+$createdDirectory = $null
+Push-Location -LiteralPath $testDirectory
+try {
+    $createdDirectory = & {
+        $sourceWorkflowDirectory = $root
+        & ([scriptblock]::Create($startup.Groups['code'].Value + "`n`$workDirectory"))
+    }
+    Assert-Equal (Split-Path -Parent $createdDirectory) $testDirectory 'Run folder is a direct child of the starting current directory'
+    $version = (Get-Content -LiteralPath "$root\VERSION" -Raw).Trim()
+    Assert-Equal ((Split-Path -Leaf $createdDirectory) -match ('^scimrun-v' + [regex]::Escape($version) + '-\d{8}T\d{6}Z-[a-f0-9]{8}$')) $true 'Run folder contains the actual VERSION, UTC start and unique local run ID'
+} finally {
+    Pop-Location
+    if ($createdDirectory -and (Test-Path -LiteralPath $createdDirectory)) {
+        Remove-Item -LiteralPath $createdDirectory -ErrorAction Stop
+    }
+    Remove-Item -LiteralPath $testDirectory -ErrorAction Stop
+}
 if (-not $orchestrator.definition.actions.Evaluate_Test_Results.inputs.overallLogicAppResult.Contains("not(equals(coalesce(body('Get_SCIMTests_Output_Content')?['authenticationModelResult'],'FAILED'),'PASSED'))")) {
     throw 'Orchestrator must reject missing or non-passing authentication readiness.'
 }