From b84daf54cbfe8172d655f8294a485fa581a2702b Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Tue, 22 Sep 2026 16:04:19 -0300 Subject: [PATCH 01/23] Add XML documentation validation to OneBranch builds Adds a preflight for the XML documentation cross-references that feed dotnet/sqlclient-api-docs, so malformed documentation IDs fail our own build rather than surfacing as xref-not-found warnings on an API Docs pull request. This commit exposes the problems; it does not fix them. The findings it reports are left in place so they can be reviewed and fixed separately. Validation ---------- eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 reports, as errors: documentation-ID syntax defects (C# aliases in signatures, empty parentheses on parameterless members, embedded whitespace, array T: UIDs), unknown namespace roots, cross-references the compiler could not bind, malformed files, stale allowlist entries, documentation that should exist but does not, and lib/ versus ref/ documentation trimming defects in a package. Resolution findings that a target-framework-conditional or cross-assembly member can legitimately trigger are warnings. No network access is required: these defects are decidable from documentation-ID syntax and the local member index, so the published Learn xref map is not needed. It stays available via -ExternalXrefMapPath. It runs at three points: snippet sources before the build, generated documentation after it, and the assembled packages during package validation. Only the last shows what a consumer actually receives. Expectations come from the project ---------------------------------- GenerateDocumentationFile in the csproj is the single declaration, so nothing is restated in the pipeline and the two cannot drift. A project that generates documentation must produce it; one that does not is reported as information naming the reason, and documentation appearing there is reported as a warning. Snippet validation follows the same principle: only the snippets a project's sources reference are validated. Project files are parsed as XML rather than searched as text, because a comment naming the property would otherwise read as setting it. Gating ------ One failOnValidationError parameter governs the localization, XML documentation, and package validation steps. When false, findings are reported as warnings and the build continues; malformed inputs and a validator that fails to run still fail the step. The official pipeline sets it true and the non-official pipeline exposes it at queue time, with no intermediate template declaring a default. Steps reporting findings mark themselves SucceededWithIssues, because task.logissue alone leaves the task result untouched and a step carrying warnings would otherwise render as a clean success. breakOnSdlError is renamed failOnSdlError to match. Microsoft.SqlServer.Server -------------------------- Enables documentation generation. Its public types already carry complete documentation comments, producing 63 documented members with no warnings, but they were compiled away so consumers got no IntelliSense. Set unconditionally: net46 is type-forwards only and gets an essentially empty file, but a TargetFramework condition would leave the project without a single unambiguous answer about whether it produces documentation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 752bc4acf00cbe8fbadfe953ed2b7950a7034e75) --- .../documentation.instructions.md | 47 + .../onebranch-pipeline-design.instructions.md | 20 +- .../onebranch/jobs/build-buildproj-job.yml | 51 + .../onebranch/jobs/validate-packages-job.yml | 29 + .../onebranch/scripts/tests/README.md | 10 +- .../tests/pipeline-invocation.Tests.ps1 | 141 ++ .../tests/validate-localization.Tests.ps1 | 45 + .../scripts/tests/validate-packages.Tests.ps1 | 38 +- .../scripts/tests/validate-xml-docs.Tests.ps1 | 900 +++++++++++++ .../scripts/validate-localization.ps1 | 30 +- .../onebranch/scripts/validate-packages.ps1 | 24 +- .../onebranch/scripts/validate-xml-docs.ps1 | 1147 +++++++++++++++++ .../onebranch/sqlclient-non-official.yml | 46 +- .../onebranch/sqlclient-official.yml | 33 +- .../onebranch/stages/build-stages.yml | 24 + .../steps/validate-localization-step.yml | 34 +- .../steps/validate-packages-step.yml | 45 +- .../steps/validate-xml-docs-step.yml | 123 ++ .../Microsoft.SqlServer.Server.csproj | 14 + 19 files changed, 2746 insertions(+), 55 deletions(-) create mode 100644 eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 create mode 100644 eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 create mode 100644 eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 create mode 100644 eng/pipelines/onebranch/steps/validate-xml-docs-step.yml diff --git a/.github/instructions/documentation.instructions.md b/.github/instructions/documentation.instructions.md index 57f547ee6d..49313c6ee9 100644 --- a/.github/instructions/documentation.instructions.md +++ b/.github/instructions/documentation.instructions.md @@ -140,6 +140,53 @@ public override void Open() | `` | Additional details | | `` | Related members | +### Cross-References (`cref`) + +Our XML documentation is ingested into [dotnet/sqlclient-api-docs](https://github.com/dotnet/sqlclient-api-docs), where Open Publishing resolves every `cref` against the Learn xref map. A malformed documentation ID cannot resolve and produces an `xref-not-found` warning on the API Docs pull request, long after the change left this repository. + +A `cref` may be written unqualified (``), in which case the compiler binds it from the surrounding source. Once you write an explicit `T:`/`M:`/`P:`/`F:`/`E:`/`N:` prefix, the compiler passes the value through verbatim and no longer checks it, so the rules below are yours to get right. + +| Rule | Wrong | Right | +|------|-------|-------| +| Use CLR type names, not C# aliases | `M:...GetSchema(string)` | `M:...GetSchema(System.String)` | +| Omit parentheses on a parameterless member | `M:...GetSchema()` | `M:...GetSchema` | +| Never include whitespace | `M:...Add(System.String, System.String)` | `M:...Add(System.String,System.String)` | +| `T:` names a type, never an array | `T:System.Byte[]` | `T:System.Byte` array | +| Match the prefix to the member kind | `M:...SqlCommand.CommandTimeout` | `P:...SqlCommand.CommandTimeout` | +| Generic arguments use braces | `T:...List` | `T:...List{System.String}` | + +Array, pointer and by-reference markers are legal *inside* a member signature (`M:...Decrypt(System.Byte[])`); they are only invalid as the whole target of a `T:` reference. + +### Validating Cross-References Locally + +`eng/pipelines/onebranch/scripts/validate-xml-docs.ps1` enforces the rules above. It runs in the OneBranch build jobs against snippet sources, generated documentation, and the assembled packages, so run it before pushing documentation changes: + +```powershell +./eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 -SnippetsDirectory ./doc/snippets +``` + +To also resolve references against the members the build actually emitted, which additionally catches wrong-kind prefixes and cross-references the compiler failed to bind: + +```powershell +dotnet build ./src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj -c Release +./eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 -DocumentationPath ./artifacts/Microsoft.Data.SqlClient.ref +``` + +Validation is offline by design; it needs no network access and no xref map download. + +### Trimmed vs. Full Documentation + +The driver package ships **two** XML documentation files per target framework, and they are deliberately different: + +| Package folder | Content | Consumer | +|----------------|---------|----------| +| `lib//` | Full documentation, including `` and `` | IntelliSense | +| `ref//` | Trimmed by `tools/intellisense/TrimDocs.ps1`, which strips `` and `` | Reference assemblies | + +Remarks and examples render poorly in Visual Studio tooltips, which is why `ref/` is trimmed. The two files must never be the same: if `lib/` is sourced from the trimmed artifact, IntelliSense silently loses every remark and example. That regression shipped in 7.1.0, so the packaged-documentation gate now fails the build when a package's `lib/` XML is trimmed, its `ref/` XML is not, or the two are byte-identical. + +When changing the `` mappings in `Microsoft.Data.SqlClient.nuspec`, keep `lib/` pointed at the implementation artifact and `ref/` at the reference artifact. + ### Writing Style - Use third person ("Opens a connection" not "Open a connection") - Be concise but complete diff --git a/.github/instructions/onebranch-pipeline-design.instructions.md b/.github/instructions/onebranch-pipeline-design.instructions.md index 02717a3a13..bfbada84f9 100644 --- a/.github/instructions/onebranch-pipeline-design.instructions.md +++ b/.github/instructions/onebranch-pipeline-design.instructions.md @@ -26,9 +26,23 @@ Respect this graph when modifying build stages: 5. `Microsoft.Data.SqlClient.Extensions.Azure` — depends on Abstractions + Logging 6. `Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider` — depends on SqlClient + Abstractions + Logging -## Localization Validation +## Validation -The SqlClient build job runs `steps/validate-localization-step.yml` before building the driver. Validation always fails the build for missing or obsolete keys, empty localized values whose English value is non-empty, and untranslated resources. Approved identical translations are listed by culture and resource key in `.config/LocalizationValidationAllowlist.json`. +Validation runs in three places and shares one gating switch. + +- **Localization** — `steps/validate-localization-step.yml`, in the SqlClient build job before the driver is built. Reports missing or obsolete keys, empty localized values whose English value is non-empty, and untranslated resources. Approved identical translations are listed by culture and resource key in `.config/LocalizationValidationAllowlist.json`. +- **XML documentation** — `steps/validate-xml-docs-step.yml`, three times per run: snippet sources before the build, generated documentation after it, and the assembled packages in `package_validation`. Reports malformed documentation IDs, unresolved cross-references, and `lib/` vs `ref/` documentation-trimming defects. +- **Packages** — `steps/validate-packages-step.yml`, in `package_validation`. Runs `tools/PackageValidator` across the whole drop so its cross-package version and dependency rules apply. + +### `failOnValidationError` + +All three honour the `failOnValidationError` queue-time parameter, which defaults to `true`. + +When `false`, findings are logged as warnings and the build continues. This exists because the gates span jobs that depend on one another: a hard failure in an early gate aborts the build stage, so the later gates never run and a single run cannot show the full picture. Turning it off lets one run exercise every gate at once, which is how a newly added gate or a known backlog of findings is assessed. + +Report-only affects *findings* only. Malformed or missing inputs, and a validator that fails to run, still fail the step — neither produced findings worth reporting. + +The parameter is exposed at queue time only by `sqlclient-non-official.yml`. `sqlclient-official.yml` hardcodes it to `true` at its `build-stages.yml` call site, so an official run cannot be started with validation downgraded. Stages, jobs and steps declare it with no default and simply honour what they are given — the policy lives in one place rather than being re-asserted at every level. ## Build Stages @@ -146,7 +160,7 @@ Variable groups: ## SDL and Compliance - TSA: enabled only in official pipeline; disabled in non-official to avoid spurious alerts -- ApiScan: enabled in both; `break` follows the `breakOnSdlError` parameter +- ApiScan: enabled in both; `break` follows the `failOnSdlError` parameter - Each package is registered with APIScan under its own name/version pair, so the `globalSdl.apiscan` blocks deliberately omit `softwareName`/`versionNumber`. `build-buildproj-job.yml` is the single place they are set, via `ob_sdl_apiscan_softwareName` (the package's `packageFullName`) and `ob_sdl_apiscan_versionNumber` (the `apiScanSoftwareVersion` parameter) - `compute-versions.ps1` derives APIScan registration versions as major.minor from the effective canonical package versions and publishes them as stage outputs. A package name/version pair must still be registered with APIScan before releasing a new major.minor. Consume these as runtime `$(...)` references so values such as `1.0` remain strings rather than being coerced to numbers by template expressions - Jobs that produce no assemblies (symbol publishing, signed-package validation, version computation) set `ob_sdl_apiscan_enabled: false` rather than reporting a name/version diff --git a/eng/pipelines/onebranch/jobs/build-buildproj-job.yml b/eng/pipelines/onebranch/jobs/build-buildproj-job.yml index 6576627a6a..1846992988 100644 --- a/eng/pipelines/onebranch/jobs/build-buildproj-job.yml +++ b/eng/pipelines/onebranch/jobs/build-buildproj-job.yml @@ -29,6 +29,16 @@ parameters: - name: shouldSignPackage type: boolean + # When true, validation findings fail this job. When false, they are reported as warnings and + # the job continues. + - name: failOnValidationError + type: boolean + + # Project file for this package. Documentation expectations are read from it rather than + # restated here, so the project and the build cannot disagree. + - name: projectPath + type: string + # Signing Parameters ----------------------------------------------------- # @TODO: Signing Parameters Object @@ -141,6 +151,28 @@ jobs: # building so missing or untranslated strings fail every SqlClient build. - ${{ if eq(parameters.packageShortName, 'SqlClient') }}: - template: /eng/pipelines/onebranch/steps/validate-localization-step.yml@self + parameters: + failOnValidationError: ${{ parameters.failOnValidationError }} + + # Validate the documentation snippet sources. This requires no build, so a malformed + # cross-reference is reported in seconds against the file and line a developer edits, rather + # than surfacing much later as an xref-not-found warning during API docs ingestion. + # + # Restricted to the packages whose sources incorporate doc/snippets content. + - ${{ if or(eq(parameters.packageShortName, 'SqlClient'), eq(parameters.packageShortName, 'SqlServer')) }}: + - template: /eng/pipelines/onebranch/steps/validate-xml-docs-step.yml@self + parameters: + displayName: 'Validate XML documentation sources' + snippetsDirectory: '$(Build.SourcesDirectory)/doc/snippets' + documentationPath: '' + packagesPath: '' + extractPath: '' + reportPath: '$(JOB_OUTPUT)/validation/xml-docs-sources.json' + failOn: + - error + failOnValidationError: ${{ parameters.failOnValidationError }} + projectPath: '${{ parameters.projectPath }}' + projectSearchRoot: '' - ${{ each package in parameters.dependencies }}: # Build the dependency version arguments passed to the build/pack/analysis steps. The @@ -193,6 +225,25 @@ jobs: versionPropertySuffix: ${{ parameters.versionPropertySuffix }} packageVersion: ${{ parameters.packageVersion }} + # Validate the XML documentation the compiler emitted. Compared with the snippet sources, + # this artifact additionally reveals cross-references the compiler could not bind (it rewrites + # those with a "!:" prefix) and whether a reference to our own API matches a member that was + # actually emitted. It runs for every package: documentation is shared through , so + # a defect can surface in an assembly whose own sources contain no cross-references at all. + - template: /eng/pipelines/onebranch/steps/validate-xml-docs-step.yml@self + parameters: + displayName: 'Validate generated XML documentation' + snippetsDirectory: '' + documentationPath: '$(BUILD_OUTPUT)/${{ parameters.packageFullName }}' + packagesPath: '' + extractPath: '' + reportPath: '$(JOB_OUTPUT)/validation/xml-docs-generated.json' + failOn: + - error + failOnValidationError: ${{ parameters.failOnValidationError }} + projectPath: '${{ parameters.projectPath }}' + projectSearchRoot: '' + - ${{ if eq(parameters.shouldSignPackage, true) }}: # ESRP sign the DLLs. - template: /eng/pipelines/onebranch/steps/esrp-dll-signing-step.yml@self diff --git a/eng/pipelines/onebranch/jobs/validate-packages-job.yml b/eng/pipelines/onebranch/jobs/validate-packages-job.yml index f1501c7d1d..3fd6cf929c 100644 --- a/eng/pipelines/onebranch/jobs/validate-packages-job.yml +++ b/eng/pipelines/onebranch/jobs/validate-packages-job.yml @@ -62,6 +62,11 @@ parameters: - name: isOfficial type: boolean + # When true, validation findings fail this job. When false, they are reported as warnings and + # the job continues. + - name: failOnValidationError + type: boolean + jobs: - job: validate_packages displayName: 'Validate Packages' @@ -98,6 +103,11 @@ jobs: - name: extractRoot value: '$(Pipeline.Workspace)/validate-extract' + # Expansion root for the packaged XML documentation gate. Kept distinct from extractRoot so + # the two expansions cannot overwrite or stale each other. + - name: xmlDocsExtractRoot + value: '$(Pipeline.Workspace)/validate-xml-docs-extract' + steps: - template: /eng/pipelines/onebranch/steps/script-output-environment-variables-step.yml@self @@ -154,6 +164,7 @@ jobs: reportPath: '$(JOB_OUTPUT)/validation/package-validation.json' sqlClientPackageVersion: '${{ parameters.sqlClientPackageVersion }}' sqlClientFileVersion: '${{ parameters.sqlClientFileVersion }}' + failOnValidationError: ${{ parameters.failOnValidationError }} # Omitted when SqlServer is not built: its package is absent from the drop, and the # validator rejects an expectation with an empty value. ${{ if eq(parameters.buildSqlServer, true) }}: @@ -182,6 +193,24 @@ jobs: - delay-signed - unsigned + # Validate the XML documentation as a consumer receives it. Only the assembled package shows + # which XML actually landed in each lib/ and ref/ target framework, a mapping that has + # silently changed before without any earlier check noticing. + - template: /eng/pipelines/onebranch/steps/validate-xml-docs-step.yml@self + parameters: + displayName: 'Validate packaged XML documentation' + snippetsDirectory: '' + documentationPath: '' + packagesPath: '$(packagesRoot)' + extractPath: '$(xmlDocsExtractRoot)' + reportPath: '$(JOB_OUTPUT)/validation/xml-docs-packages.json' + failOn: + - error + failOnValidationError: ${{ parameters.failOnValidationError }} + # The drop spans every package, so expectations come from every project rather than one. + projectPath: '' + projectSearchRoot: '$(REPO_ROOT)/src' + # Signature verification, official builds only. PackageValidator reports strong-name and # NuGet signature *presence* cross-platform; these steps additionally verify that the # signatures are trusted, which requires the Windows trust store. diff --git a/eng/pipelines/onebranch/scripts/tests/README.md b/eng/pipelines/onebranch/scripts/tests/README.md index 593c7768d6..783bb5aee0 100644 --- a/eng/pipelines/onebranch/scripts/tests/README.md +++ b/eng/pipelines/onebranch/scripts/tests/README.md @@ -32,13 +32,15 @@ Invoke-Pester ./publish-symbols.Tests.ps1 -Output Detailed | Area | What's tested | | --------------------- | ---------------------------------------------------------------- | | Version computation | Canonical output parsing, effective package selection, target version composition, and failures | -| Localization validation | Missing, obsolete, or empty strings, English-value matches, and culture-specific allowlisting | +| Localization validation | Missing, obsolete, or empty strings, English-value matches, culture-specific allowlisting, and report-only mode | | Parameter validation | Empty strings rejected for all mandatory parameters | | URL construction | Base URL, register URL, request URL built from parameters | | Request bodies | Registration body, default publish flags, flag overrides | | Error handling | Token failure, registration failure, publish failure, status failure — all verify expanded URI in error message | | Status validation | Detects Failed/Cancelled results, respects PublishToInternal/PublishToPublic flags, passes on Succeeded/Pending | -| Package validation | Wildcard vs per-id version expectations, SqlServer omitted when unbuilt, gate tokens, report written before gating, exit-code handling | +| Package validation | Wildcard vs per-id version expectations, SqlServer omitted when unbuilt, gate tokens, report written before gating, exit-code handling, report-only mode suppressing the gate but not a broken validator | +| XML docs validation | Every documentation-ID defect form reported by the API Docs build (array `T:` UIDs, empty parentheses, C# aliases, embedded whitespace, misspelled namespace roots) plus valid controls, compiler-unresolved `!:` crefs, local UID and wrong-prefix resolution, package expansion, allowlisting and staleness, report-only mode | +| XML docs lib/ref layout | Full `lib/` XML paired with trimmed `ref/` XML accepted; trimmed `lib/`, untrimmed `ref/`, and byte-identical `lib`/`ref` rejected; per-target-framework isolation; packages without a `ref/` folder ignored | | Package signatures | Every package and symbol package verified, all failures reported before throwing | | Assembly signatures | Package expansion, native binaries under `runtimes/` included, stale expansions replaced, all unsigned assemblies reported | @@ -51,3 +53,7 @@ Invoke-Pester ./publish-symbols.Tests.ps1 -Output Detailed it is absent. Only the signature lookup is substituted; package expansion and reporting run for real against packages built in the test's temporary directory. - Tests validate scripts in the parent directory relative to this directory. +- Validation scripts accept `-ReportOnly`, which downgrades findings to warnings. The pipelines + drive it from the `failOnValidationError` parameter (inverted). Report-only suppresses + *findings* only: malformed or missing inputs, and a validator that fails to run, still fail the + step, because neither produced findings worth reporting. diff --git a/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 new file mode 100644 index 0000000000..d36b1af222 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 @@ -0,0 +1,141 @@ +<# +.SYNOPSIS + Pester tests for the OneBranch validation step templates and their script invocation form. + +.DESCRIPTION + The PowerShell task dot-sources a filePath script under -Command rather than running it with + -File. The two parse switch arguments differently: -Command requires a bare token or an + explicit $true, and rejects the -Switch:Value form that -File accepts. A step template that + renders -ReportOnly:True therefore fails at argument binding before the script runs at all. + + These tests pin both halves of that contract: the templates must not emit a -Switch:Value form, + and the scripts must accept a bare switch when dot-sourced the way the task does. +#> + +BeforeAll { + $script:repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..' '..' '..' '..' '..')).Path + $script:stepsPath = Join-Path $script:repoRoot 'eng/pipelines/onebranch/steps' + $script:scriptsPath = Join-Path $script:repoRoot 'eng/pipelines/onebranch/scripts' + + # Runs a command line the way the PowerShell task does: a generated wrapper, dot-sourced by + # pwsh -Command. Returns the exit code. + function Invoke-AsPipelineTask { + param([Parameter(Mandatory)][string]$CommandLine) + + $wrapper = Join-Path $TestDrive ([guid]::NewGuid().ToString('n') + '.ps1') + Set-Content -LiteralPath $wrapper -Value $CommandLine -Encoding utf8 + + $pwshPath = (Get-Process -Id $PID).Path + & $pwshPath -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted ` + -Command ". '$wrapper'" *> $null + return $LASTEXITCODE + } + + function New-LocalizationResources { + param([switch]$WithFindings) + + $path = Join-Path $TestDrive ([guid]::NewGuid().ToString('n')) + New-Item -ItemType Directory -Path $path | Out-Null + 'Hello' | + Set-Content -LiteralPath (Join-Path $path 'Strings.resx') -Encoding utf8 + $french = if ($WithFindings) { 'Hello' } else { 'Bonjour' } + "$french" | + Set-Content -LiteralPath (Join-Path $path 'Strings.fr.resx') -Encoding utf8 + return $path + } +} + +Describe 'Validation step templates' { + It 'never passes a switch using the -Switch:Value form' -ForEach @( + @{ Template = 'validate-xml-docs-step.yml' } + @{ Template = 'validate-localization-step.yml' } + @{ Template = 'validate-packages-step.yml' } + ) { + # -Switch:Value binds as a string under -Command and fails before the script runs. + $content = Get-Content -LiteralPath (Join-Path $script:stepsPath $Template) -Raw + $content | Should -Not -Match '-\w+:\$?\{\{' + $content | Should -Not -Match '-ReportOnly:' + } + + It 'appends -ReportOnly as a bare token' -ForEach @( + @{ Template = 'validate-xml-docs-step.yml' } + @{ Template = 'validate-localization-step.yml' } + @{ Template = 'validate-packages-step.yml' } + ) { + $content = Get-Content -LiteralPath (Join-Path $script:stepsPath $Template) -Raw + $content | Should -Match "\+= ' -ReportOnly'" + } + + It 'builds one argument list rather than repeating it per branch' -ForEach @( + @{ Template = 'validate-xml-docs-step.yml' } + @{ Template = 'validate-localization-step.yml' } + @{ Template = 'validate-packages-step.yml' } + ) { + # A duplicated arguments block drifts silently when only one copy is updated, so the + # validation script's arguments must come from a single composed variable. Counting + # 'arguments:' keys would be wrong here: a template may invoke other tasks that carry + # their own unrelated arguments. + $content = Get-Content -LiteralPath (Join-Path $script:stepsPath $Template) -Raw + $content | Should -Match 'task\.setvariable variable=\w+Arguments' + $content | Should -Match '(?m)^\s*arguments:\s*\$\(\w+Arguments\)\s*$' + } + + It 'rejects an unexpected failOnValidationError value rather than assuming report-only' -ForEach @( + @{ Template = 'validate-xml-docs-step.yml' } + @{ Template = 'validate-localization-step.yml' } + @{ Template = 'validate-packages-step.yml' } + ) { + # Defaulting an unrecognised value to report-only would silently disable gating. + $content = Get-Content -LiteralPath (Join-Path $script:stepsPath $Template) -Raw + $content | Should -Match 'Unexpected failOnValidationError value' + } +} + +Describe 'Validation scripts under the pipeline invocation form' { + It 'accepts a bare -ReportOnly switch when dot-sourced: validate-xml-docs.ps1' { + $target = Join-Path $script:scriptsPath 'validate-xml-docs.ps1' + $snippets = Join-Path $TestDrive ([guid]::NewGuid().ToString('n')) + New-Item -ItemType Directory -Path $snippets | Out-Null + '' | + Set-Content -LiteralPath (Join-Path $snippets 'Sample.xml') -Encoding utf8 + $report = Join-Path $TestDrive ([guid]::NewGuid().ToString('n') + '.json') + + $line = ". '$target' -ReportPath `"$report`" -FailOn `"error`" " + + "-SnippetsDirectory `"$snippets`" -DocumentationPath `"`" -PackagesPath `"`" " + + "-ExtractPath `"`" -ReportOnly" + + Invoke-AsPipelineTask -CommandLine $line | Should -Be 0 + Test-Path -LiteralPath $report | Should -BeTrue + } + + It 'still fails when dot-sourced without -ReportOnly: validate-xml-docs.ps1' { + $target = Join-Path $script:scriptsPath 'validate-xml-docs.ps1' + $snippets = Join-Path $TestDrive ([guid]::NewGuid().ToString('n')) + New-Item -ItemType Directory -Path $snippets | Out-Null + '' | + Set-Content -LiteralPath (Join-Path $snippets 'Sample.xml') -Encoding utf8 + $report = Join-Path $TestDrive ([guid]::NewGuid().ToString('n') + '.json') + + $line = ". '$target' -ReportPath `"$report`" -FailOn `"error`" " + + "-SnippetsDirectory `"$snippets`" -DocumentationPath `"`" -PackagesPath `"`" " + + "-ExtractPath `"`"" + + Invoke-AsPipelineTask -CommandLine $line | Should -Not -Be 0 + } + + It 'accepts a bare -ReportOnly switch when dot-sourced: validate-localization.ps1' { + $target = Join-Path $script:scriptsPath 'validate-localization.ps1' + $resources = New-LocalizationResources -WithFindings + + Invoke-AsPipelineTask -CommandLine ". '$target' -ResourcesDirectory `"$resources`" -ReportOnly" | + Should -Be 0 + } + + It 'still fails when dot-sourced without -ReportOnly: validate-localization.ps1' { + $target = Join-Path $script:scriptsPath 'validate-localization.ps1' + $resources = New-LocalizationResources -WithFindings + + Invoke-AsPipelineTask -CommandLine ". '$target' -ResourcesDirectory `"$resources`"" | + Should -Not -Be 0 + } +} diff --git a/eng/pipelines/onebranch/scripts/tests/validate-localization.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-localization.Tests.ps1 index 5f27b883d3..4b1903279d 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-localization.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-localization.Tests.ps1 @@ -188,3 +188,48 @@ Describe 'validate-localization.ps1' { Should -Throw '*does not have a non-empty English value*' } } + +Describe 'validate-localization.ps1 Report-Only Mode' { + It 'warns instead of failing when localized files have findings' { + $resources = New-ResourcesDirectory + Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello'; Farewell = 'Goodbye' } + Set-ResourceFile (Join-Path $resources 'Strings.de.resx') @{ Greeting = 'Hallo' } + + $output = & $scriptPath -ResourcesDirectory $resources -ReportOnly *>&1 | Out-String + + $output | Should -Match 'report-only mode' + $output | Should -Match '##vso\[task.logissue type=warning\]' + # Without this the step renders as a clean success despite reporting warnings. + $output | Should -Match '##vso\[task\.complete result=SucceededWithIssues;\]' + $output | Should -Not -Match '##vso\[task.logissue type=error\]' + } + + It 'does not claim the validation passed when findings were suppressed' { + $resources = New-ResourcesDirectory + Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' } + Set-ResourceFile (Join-Path $resources 'Strings.de.resx') @{ Greeting = 'Hello' } + + $output = & $scriptPath -ResourcesDirectory $resources -ReportOnly *>&1 | Out-String + + $output | Should -Not -Match 'validation passed' + $output | Should -Match 'validation examined' + } + + It 'still fails on malformed inputs, which produce no findings to report' { + $resources = New-ResourcesDirectory + Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' } + + { & $scriptPath -ResourcesDirectory $resources -ReportOnly } | + Should -Throw '*No localized Strings.*.resx files were found*' + } + + It 'reports success normally when there are no findings' { + $resources = New-ResourcesDirectory + Set-ResourceFile (Join-Path $resources 'Strings.resx') @{ Greeting = 'Hello' } + Set-ResourceFile (Join-Path $resources 'Strings.fr.resx') @{ Greeting = 'Bonjour' } + + $output = & $scriptPath -ResourcesDirectory $resources -ReportOnly *>&1 | Out-String + + $output | Should -Match 'validation passed' + } +} diff --git a/eng/pipelines/onebranch/scripts/tests/validate-packages.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-packages.Tests.ps1 index 626dd2dca9..38d4579d78 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-packages.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-packages.Tests.ps1 @@ -25,7 +25,8 @@ BeforeAll { [string]$SqlClientFileVersion = '7.1.0.26238', [string]$SqlServerPackageVersion = '', [string]$SqlServerFileVersion = '', - [string[]]$FailOn = @('error') + [string[]]$FailOn = @('error'), + [switch]$ReportOnly ) & $scriptPath ` @@ -37,6 +38,7 @@ BeforeAll { -SqlServerPackageVersion $SqlServerPackageVersion ` -SqlServerFileVersion $SqlServerFileVersion ` -FailOn $FailOn ` + -ReportOnly:$ReportOnly ` -DotnetPath 'dotnet' *>&1 | Out-String } @@ -163,6 +165,40 @@ Describe 'validate-packages.ps1 Exit Codes' { } } +Describe 'validate-packages.ps1 Report-Only Mode' { + It 'warns instead of failing when a gate is tripped' { + Set-DotnetMock -GateExitCode 2 + + $output = Invoke-ValidatePackages -ReportOnly + $output | Should -Match 'report-only mode' + $output | Should -Match '##vso\[task.logissue type=warning\]' + # Without this the step renders as a clean success despite reporting warnings. + $output | Should -Match '##vso\[task\.complete result=SucceededWithIssues;\]' + } + + It 'still fails when the validator itself fails' { + # Report-only suppresses findings, not a broken tool: a validator that could not run + # produced nothing to report, so downgrading it would hide a real breakage. + Set-DotnetMock -GateExitCode 1 + + { Invoke-ValidatePackages -ReportOnly } | Should -Throw '*exited unexpectedly with code 1*' + } + + It 'still fails when the reporting run fails' { + Set-DotnetMock -ReportExitCode 1 + + { Invoke-ValidatePackages -ReportOnly } | + Should -Throw '*failed while writing the JSON report (exit code 1)*' + } + + It 'does not alter behaviour when there are no findings' { + Set-DotnetMock -GateExitCode 0 + + $output = Invoke-ValidatePackages -ReportOnly + $output | Should -Match 'Package validation passed' + } +} + Describe 'validate-packages.ps1 Error Handling' { BeforeEach { Set-DotnetMock diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 new file mode 100644 index 0000000000..02b1737015 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -0,0 +1,900 @@ +<# +.SYNOPSIS + Pester tests for validate-xml-docs.ps1. + +.DESCRIPTION + Every cross-reference form that produced an xref-not-found warning against + dotnet/sqlclient-api-docs PR 99 is covered here, alongside the valid controls those rules must + not flag. The controls matter as much as the failures: the rules run over every cref in + doc/snippets, so a rule that over-reports would block the build on correct documentation. +#> + +BeforeAll { + $scriptPath = Join-Path $PSScriptRoot '..' 'validate-xml-docs.ps1' + + function New-TestDirectory { + $path = Join-Path $TestDrive ([guid]::NewGuid().ToString('n')) + New-Item -ItemType Directory -Path $path | Out-Null + return $path + } + + <# + Writes a snippet file shaped like doc/snippets: a tree whose members carry + the cref-bearing elements the compiler pulls in via . + #> + function New-SnippetDirectory { + param([string[]]$Crefs = @(), [string]$RawBody) + + $path = New-TestDirectory + $body = if ($PSBoundParameters.ContainsKey('RawBody')) { + $RawBody + } + else { + ($Crefs | ForEach-Object { " " }) -join "`n" + } + + @" + + + + Sample. +$body + + + +"@ | Set-Content -LiteralPath (Join-Path $path 'Sample.xml') -Encoding utf8 + + return $path + } + + <# + Writes a compiler-shaped XML documentation file: with the + documented members that the local UID index is built from. + #> + function New-DocumentationDirectory { + param( + [string[]]$Members = @('T:Microsoft.Data.SqlClient.SqlConnection'), + [string[]]$Crefs = @() + ) + + $path = New-TestDirectory + $memberXml = ($Members | ForEach-Object { " Doc." }) -join "`n" + $crefXml = ($Crefs | ForEach-Object { " " }) -join "`n" + + @" + + Microsoft.Data.SqlClient + +$memberXml + + Sample. +$crefXml + + + +"@ | Set-Content -LiteralPath (Join-Path $path 'Microsoft.Data.SqlClient.xml') -Encoding utf8 + + return $path + } + + function Get-Report { + param([Parameter(Mandatory)][string]$Path) + return Get-Content -LiteralPath $Path -Raw | ConvertFrom-Json + } + + <# + Writes a project file, and optionally a source file whose documentation comment pulls in a + snippet, so the script's expectation derivation has something real to read. + #> + function New-Project { + param( + [switch]$GeneratesDocumentation, + [switch]$ThenDisables, + [string]$Comment, + [string]$SnippetReference + ) + + $directory = New-TestDirectory + $projectPath = Join-Path $directory 'Sample.csproj' + + $properties = '' + if ($GeneratesDocumentation) { + $properties += ' true' + [Environment]::NewLine + } + if ($ThenDisables) { + $properties += ' false' + [Environment]::NewLine + } + + $commentXml = if ([string]::IsNullOrEmpty($Comment)) { '' } else { " " } + + @" + +$commentXml + +$properties + +"@ | Set-Content -LiteralPath $projectPath -Encoding utf8 + + if (-not [string]::IsNullOrEmpty($SnippetReference)) { + # The include path is relative to the source file that declares it, matching how the + # compiler resolves it. + $relative = [System.IO.Path]::GetRelativePath($directory, $SnippetReference) -replace '\\', '/' + @" +/// +public class Sample { } +"@ | Set-Content -LiteralPath (Join-Path $directory 'Sample.cs') -Encoding utf8 + } + + return $projectPath + } +} + +Describe 'validate-xml-docs.ps1' { + + Context 'PR 99 failure forms' { + + It 'rejects a T: cref naming an array' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte[]') + + { & $scriptPath -SnippetsDirectory $snippets } | + Should -Throw '*XML documentation validation failed with 1 issue*' + } + + It 'suggests the element type when a T: cref names an array' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte[]') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'invalid-docid' + $finding.Message | Should -BeLike '*T:System.Byte*array*' + } + + It 'rejects a parameterless method cref written with empty parentheses' { + $snippets = New-SnippetDirectory -Crefs @('M:Microsoft.Data.SqlClient.SqlConnection.GetSchema()') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'invalid-docid' + $finding.Message | Should -BeLike "*use 'M:Microsoft.Data.SqlClient.SqlConnection.GetSchema'*" + } + + It 'rejects a C# alias in a method signature' { + $snippets = New-SnippetDirectory -Crefs @('M:Microsoft.Data.SqlClient.SqlConnection.GetSchema(string)') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'invalid-docid' + $finding.Message | Should -BeLike "*C# alias 'string'*" + } + + It 'reports both the whitespace and the alias in a signature carrying both' { + $snippets = New-SnippetDirectory -Crefs @('M:Microsoft.Data.SqlClient.SqlConnection.GetSchema(string, string[])') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 2 + ($findings.Message -join ' ') | Should -BeLike '*whitespace*' + ($findings.Message -join ' ') | Should -BeLike "*C# alias 'string'*" + } + + It 'reports one alias finding when a signature repeats the same alias' { + $snippets = New-SnippetDirectory -Crefs @('M:Sample.Type.Method(string,string[])') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings | Where-Object { $_.Message -like '*alias*' }) + $findings.Count | Should -Be 1 + } + + It 'rejects whitespace in an otherwise correct signature' { + $snippets = New-SnippetDirectory -Crefs @('M:Microsoft.Data.SqlClient.SqlParameterCollection.Add(System.String, System.String)') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Message | Should -BeLike '*Add(System.String,System.String)*' + } + + It 'rejects a misspelled namespace root' { + $snippets = New-SnippetDirectory -Crefs @('P:Microssoft.Data.SqlClient.SqlCommand.EnableOptimizedParameterBinding') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'unknown-namespace-root' + $finding.Message | Should -BeLike "*'Microssoft'*" + } + } + + Context 'valid controls' { + + It 'accepts the corrected forms of every PR 99 defect' { + $snippets = New-SnippetDirectory -Crefs @( + 'T:System.Byte', + 'M:Microsoft.Data.SqlClient.SqlConnection.GetSchema', + 'M:Microsoft.Data.SqlClient.SqlConnection.GetSchema(System.String)', + 'M:Microsoft.Data.SqlClient.SqlConnection.GetSchema(System.String,System.String[])', + 'M:Microsoft.Data.SqlClient.SqlParameterCollection.Add(System.String,System.String)', + 'P:Microsoft.Data.SqlClient.SqlCommand.EnableOptimizedParameterBinding') + + { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw + } + + It 'accepts arrays, by-reference and pointer markers inside a member signature' { + $snippets = New-SnippetDirectory -Crefs @( + 'M:Microsoft.Data.SqlClient.Sample.Decrypt(System.Byte[])', + 'M:Microsoft.Data.SqlClient.Sample.TryGet(System.Int32@)', + 'M:Microsoft.Data.SqlClient.Sample.Raw(System.Byte*)') + + { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw + } + + It 'accepts generic types whose arguments contain commas' { + $snippets = New-SnippetDirectory -Crefs @( + 'M:Microsoft.Data.SqlClient.Sample.Map(System.Collections.Generic.Dictionary{System.String,System.Int32})', + 'T:System.Collections.Generic.IReadOnlyList`1') + + { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw + } + + It 'accepts a conversion operator documentation ID' { + $snippets = New-SnippetDirectory -Crefs @( + 'M:Microsoft.Data.SqlTypes.SqlJson.op_Explicit(System.String)~Microsoft.Data.SqlTypes.SqlJson') + + { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw + } + + It 'does not fail on an unprefixed cref, which the compiler binds from source' { + $snippets = New-SnippetDirectory -Crefs @('SqlJson', 'string', 'System.Text.Json.JsonDocument') + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -SnippetsDirectory $snippets -ReportPath $report } | Should -Not -Throw + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 3 + $findings.Severity | Should -Not -Contain 'error' + } + + It 'accepts a namespace cref' { + $snippets = New-SnippetDirectory -Crefs @('N:Microsoft.Data.SqlClient') + + { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw + } + } + + Context 'documentation mode' { + + It 'rejects a cref the compiler could not bind' { + $docs = New-DocumentationDirectory -Crefs @('!:Microsoft.Data.SqlClient.Missing') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'unresolved-cref' + } + + It 'resolves a local cref against the members the build emitted' { + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('T:Microsoft.Data.SqlClient.SqlConnection') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report -ReportOnly + + @((Get-Report -Path $report).Findings) | Should -BeNullOrEmpty + } + + It 'reports a local cref with no matching emitted member as a warning, not an error' { + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('T:Microsoft.Data.SqlClient.DoesNotExist') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'missing-local-uid' + $finding.Severity | Should -Be 'warning' + } + + It 'fails on a missing local UID when that category is named in -FailOn' { + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('T:Microsoft.Data.SqlClient.DoesNotExist') + + { & $scriptPath -DocumentationPath $docs -FailOn error, missing-local-uid } | + Should -Throw '*failed with 1 issue*' + } + + It 'names the expected prefix when a cref uses the wrong member kind' { + $docs = New-DocumentationDirectory ` + -Members @('P:Microsoft.Data.SqlClient.SqlCommand.CommandTimeout') ` + -Crefs @('M:Microsoft.Data.SqlClient.SqlCommand.CommandTimeout') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'mismatched-docid-prefix' + $finding.Severity | Should -Be 'warning' + $finding.Message | Should -BeLike "*was emitted as 'P:'*" + } + + It 'does not resolve a namespace cref against the member index' { + # The compiler never emits a entry for a namespace, so an N: cref must not be + # treated as an unresolved reference. + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('N:Microsoft.Data.SqlClient') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report + + @((Get-Report -Path $report).Findings) | Should -BeNullOrEmpty + } + + It 'does not attempt local resolution in source mode' { + $snippets = New-SnippetDirectory -Crefs @('T:Microsoft.Data.SqlClient.AnythingAtAll') + + { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw + } + + It 'ignores non-documentation XML found in a scanned directory' { + $path = New-TestDirectory + '' | Set-Content -LiteralPath (Join-Path $path 'build.xml') -Encoding utf8 + '' | + Set-Content -LiteralPath (Join-Path $path 'Microsoft.Data.SqlClient.xml') -Encoding utf8 + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $path -ReportPath $report + + (Get-Report -Path $report).DocumentationFiles | Should -Be 1 + } + + It 'validates the XML documentation inside a package' { + $staging = New-TestDirectory + $libPath = Join-Path $staging 'lib/net8.0' + New-Item -ItemType Directory -Path $libPath -Force | Out-Null + '' | + Set-Content -LiteralPath (Join-Path $libPath 'Microsoft.Data.SqlClient.xml') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory( + $staging, (Join-Path $packages 'Microsoft.Data.SqlClient.7.1.0.nupkg')) + + { & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) } | + Should -Throw '*failed with 1 issue*' + } + } + + Context 'package lib/ref documentation layout' { + + BeforeAll { + # A realistic pair: the implementation XML carries narrative documentation, and the + # reference XML is the same content after TrimDocs.ps1 has removed it. + function New-LayoutPackage { + param( + [Parameter(Mandatory)][hashtable]$Files, + [string]$PackageName = 'Microsoft.Data.SqlClient.7.1.0.nupkg' + ) + + $staging = New-TestDirectory + foreach ($entry in $Files.GetEnumerator()) { + $target = Join-Path $staging $entry.Key + New-Item -ItemType Directory -Path (Split-Path -Parent $target) -Force | Out-Null + Set-Content -LiteralPath $target -Value $entry.Value -Encoding utf8 + } + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory($staging, (Join-Path $packages $PackageName)) + return $packages + } + + $script:FullXml = 'A' + + 'S.' + + 'Detail.x' + + $script:TrimmedXml = 'A' + + 'S.' + + '' + } + + It 'accepts a full lib/ XML paired with a trimmed ref/ XML' { + $packages = New-LayoutPackage -Files @{ + 'lib/net8.0/Microsoft.Data.SqlClient.xml' = $script:FullXml + 'ref/net8.0/Microsoft.Data.SqlClient.xml' = $script:TrimmedXml + } + + { & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) } | Should -Not -Throw + } + + It 'rejects a lib/ XML that has been trimmed' { + $packages = New-LayoutPackage -Files @{ + 'lib/net8.0/Microsoft.Data.SqlClient.xml' = $script:TrimmedXml + 'ref/net8.0/Microsoft.Data.SqlClient.xml' = $script:TrimmedXml + } + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) -ReportPath $report -ReportOnly + + $categories = @((Get-Report -Path $report).Findings.Category) + $categories | Should -Contain 'lib-documentation-trimmed' + } + + It 'rejects lib/ and ref/ XML that are byte-identical' { + # The 7.1.0 regression: the nuspec mapped the trimmed reference artifact into both + # targets, so IntelliSense silently lost every remark and example. + $packages = New-LayoutPackage -Files @{ + 'lib/net8.0/Microsoft.Data.SqlClient.xml' = $script:TrimmedXml + 'ref/net8.0/Microsoft.Data.SqlClient.xml' = $script:TrimmedXml + } + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) -ReportPath $report -ReportOnly + + $categories = @((Get-Report -Path $report).Findings.Category) + $categories | Should -Contain 'lib-ref-documentation-identical' + } + + It 'rejects a ref/ XML that was never trimmed' { + $packages = New-LayoutPackage -Files @{ + 'lib/net8.0/Microsoft.Data.SqlClient.xml' = $script:FullXml + 'ref/net8.0/Microsoft.Data.SqlClient.xml' = ($script:FullXml -replace 'Detail\.', 'Other.') + } + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) -ReportPath $report -ReportOnly + + $categories = @((Get-Report -Path $report).Findings.Category) + $categories | Should -Contain 'ref-documentation-untrimmed' + } + + It 'checks each target framework independently' { + # Mirrors the real package, where net462 maps correctly but the modern frameworks do not. + $packages = New-LayoutPackage -Files @{ + 'lib/net462/Microsoft.Data.SqlClient.xml' = $script:FullXml + 'ref/net462/Microsoft.Data.SqlClient.xml' = $script:TrimmedXml + 'lib/net8.0/Microsoft.Data.SqlClient.xml' = $script:TrimmedXml + 'ref/net8.0/Microsoft.Data.SqlClient.xml' = $script:TrimmedXml + } + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings | Where-Object { $_.Category -like '*documentation*' }) + $findings.Count | Should -Be 2 + @($findings | Where-Object { $_.Path -like '*net462*' }) | Should -BeNullOrEmpty + @($findings | Where-Object { $_.Path -like '*net8.0*' }).Count | Should -Be 2 + } + + It 'ignores a package that has no ref/ folder' { + # Most packages in the family ship lib/ only, and their documentation is never trimmed, + # so there is nothing to compare and nothing to report. + $packages = New-LayoutPackage ` + -PackageName 'Microsoft.Data.SqlClient.Internal.Logging.7.1.0.nupkg' ` + -Files @{ 'lib/net8.0/Microsoft.Data.SqlClient.Internal.Logging.xml' = $script:TrimmedXml } + + { & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) } | Should -Not -Throw + } + + It 'reports missing documentation as an error when the project generates it' { + # Documentation vanishing from a project that declares it is the regression this check + # exists to catch, so it must not be downgraded to a warning. + $project = New-Project -GeneratesDocumentation + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath (New-TestDirectory) -ProjectPath $project ` + -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'missing-documentation' + $finding.Severity | Should -Be 'error' + } + + It 'fails when a project that generates documentation produced none' { + $project = New-Project -GeneratesDocumentation + + { & $scriptPath -DocumentationPath (New-TestDirectory) -ProjectPath $project } | + Should -Throw '*failed with 1 issue*' + } + + It 'reports why nothing was checked when the project generates no documentation' { + # A project that deliberately emits none must pass, but must still say so rather than + # leaving a silently empty run. + $project = New-Project + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -DocumentationPath (New-TestDirectory) -ProjectPath $project -ReportPath $report } | + Should -Not -Throw + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'documentation-not-expected' + $finding.Severity | Should -Be 'info' + $finding.Message | Should -BeLike '*does not set GenerateDocumentationFile*' + } + + It 'reports documentation found for a project that declares none' { + # The project and the build disagree; the documentation is still validated. + $project = New-Project + $docs = New-DocumentationDirectory -Members @('T:Microsoft.Data.SqlClient.SqlConnection') + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -DocumentationPath $docs -ProjectPath $project -ReportPath $report } | + Should -Not -Throw + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'unexpected-documentation' + $finding.Severity | Should -Be 'warning' + } + + It 'does not treat a comment mentioning the property as setting it' { + # The project file is parsed as XML: a comment naming GenerateDocumentationFile, such + # as one recording why it is absent, must not be read as enabling it. + $project = New-Project -Comment 'GenerateDocumentationFile is deliberately not set.' + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath (New-TestDirectory) -ProjectPath $project ` + -ReportPath $report -ReportOnly + + @((Get-Report -Path $report).Findings.Category) | Should -Be 'documentation-not-expected' + } + + It 'honours the last assignment when the property is set more than once' { + $project = New-Project -GeneratesDocumentation -ThenDisables + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath (New-TestDirectory) -ProjectPath $project ` + -ReportPath $report -ReportOnly + + @((Get-Report -Path $report).Findings.Category) | Should -Be 'documentation-not-expected' + } + + It 'validates only the snippets the project references' { + $snippets = New-TestDirectory + '' | + Set-Content -LiteralPath (Join-Path $snippets 'Used.xml') -Encoding utf8 + '' | + Set-Content -LiteralPath (Join-Path $snippets 'Other.xml') -Encoding utf8 + + $project = New-Project -SnippetReference (Join-Path $snippets 'Used.xml') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ProjectPath $project ` + -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings | Where-Object { $_.Category -eq 'invalid-docid' }) + $findings.Count | Should -Be 1 + $findings[0].Cref | Should -Be 'T:System.Byte[]' + } + + It 'reports when a project references no snippets' { + $snippets = New-TestDirectory + 'x' | + Set-Content -LiteralPath (Join-Path $snippets 'S.xml') -Encoding utf8 + $project = New-Project + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ProjectPath $project ` + -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'documentation-not-expected' + $finding.Message | Should -BeLike '*references no documentation snippets*' + } + + It 'requires every documented assembly in a package to ship its documentation' { + # The regression this replaces the old blanket check with: a package that drops its + # XML file leaves consumers with no IntelliSense, and nothing else would notice. + $projects = New-TestDirectory + $projectDir = Join-Path $projects 'Shipped' + New-Item -ItemType Directory -Path $projectDir | Out-Null + @' + + + Contoso.Shipped + true + + +'@ | Set-Content -LiteralPath (Join-Path $projectDir 'Shipped.csproj') -Encoding utf8 + + $staging = New-TestDirectory + $libPath = Join-Path $staging 'lib/net8.0' + New-Item -ItemType Directory -Path $libPath -Force | Out-Null + 'binary' | Set-Content -LiteralPath (Join-Path $libPath 'Contoso.Shipped.dll') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory( + $staging, (Join-Path $packages 'Contoso.Shipped.1.0.0.nupkg')) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) ` + -ProjectSearchRoot $projects -ReportPath $report -ReportOnly + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Category -eq 'missing-documentation' }) | Select-Object -First 1 + $finding | Should -Not -BeNullOrEmpty + $finding.Severity | Should -Be 'error' + $finding.Message | Should -BeLike '*Contoso.Shipped.xml*' + } + + It 'accepts a package that ships documentation beside its assembly' { + $projects = New-TestDirectory + $projectDir = Join-Path $projects 'Shipped' + New-Item -ItemType Directory -Path $projectDir | Out-Null + @' + + + Contoso.Shipped + true + + +'@ | Set-Content -LiteralPath (Join-Path $projectDir 'Shipped.csproj') -Encoding utf8 + + $staging = New-TestDirectory + $libPath = Join-Path $staging 'lib/net8.0' + New-Item -ItemType Directory -Path $libPath -Force | Out-Null + 'binary' | Set-Content -LiteralPath (Join-Path $libPath 'Contoso.Shipped.dll') -Encoding utf8 + 'W.' | + Set-Content -LiteralPath (Join-Path $libPath 'Contoso.Shipped.xml') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory( + $staging, (Join-Path $packages 'Contoso.Shipped.1.0.0.nupkg')) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) ` + -ProjectSearchRoot $projects -ReportPath $report -ReportOnly + + @((Get-Report -Path $report).Findings | Where-Object { $_.Category -eq 'missing-documentation' }) | + Should -BeNullOrEmpty + } + + It 'does not require documentation for an assembly whose project generates none' { + $projects = New-TestDirectory + $projectDir = Join-Path $projects 'Plain' + New-Item -ItemType Directory -Path $projectDir | Out-Null + @' + + + Contoso.Plain + + +'@ | Set-Content -LiteralPath (Join-Path $projectDir 'Plain.csproj') -Encoding utf8 + + $staging = New-TestDirectory + $libPath = Join-Path $staging 'lib/net8.0' + New-Item -ItemType Directory -Path $libPath -Force | Out-Null + 'binary' | Set-Content -LiteralPath (Join-Path $libPath 'Contoso.Plain.dll') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory( + $staging, (Join-Path $packages 'Contoso.Plain.1.0.0.nupkg')) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) ` + -ProjectSearchRoot $projects -ReportPath $report -ReportOnly + + @((Get-Report -Path $report).Findings | Where-Object { $_.Category -eq 'missing-documentation' }) | + Should -BeNullOrEmpty + } + + It 'fails when the supplied project file does not exist' { + { & $scriptPath -DocumentationPath (New-TestDirectory) ` + -ProjectPath (Join-Path $TestDrive 'missing.csproj') } | + Should -Throw '*was not found*' + } + + It 'does not report a directory holding only non-documentation XML as unexpected' { + $path = New-TestDirectory + '' | + Set-Content -LiteralPath (Join-Path $path 'build.xml') -Encoding utf8 + $project = New-Project + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -DocumentationPath $path -ProjectPath $project -ReportPath $report } | + Should -Not -Throw + + @((Get-Report -Path $report).Findings.Category) | Should -Be 'documentation-not-expected' + } + + It 'does not report missing documentation when the files were malformed' { + # malformed-xml already names the cause; a second finding would misdirect. + $path = New-TestDirectory + '' | Set-Content -LiteralPath (Join-Path $path 'Broken.xml') -Encoding utf8 + $project = New-Project -GeneratesDocumentation + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $path -ProjectPath $project -ReportPath $report -ReportOnly + + $categories = @((Get-Report -Path $report).Findings.Category) + $categories | Should -Contain 'malformed-xml' + $categories | Should -Not -Contain 'missing-documentation' + } + + It 'does not apply layout rules outside package mode' { + $docs = New-TestDirectory + $libPath = Join-Path $docs 'lib/net8.0' + New-Item -ItemType Directory -Path $libPath -Force | Out-Null + Set-Content -LiteralPath (Join-Path $libPath 'Microsoft.Data.SqlClient.xml') ` + -Value $script:TrimmedXml -Encoding utf8 + + { & $scriptPath -DocumentationPath $docs } | Should -Not -Throw + } + } + + Context 'allowlist' { + + It 'exempts an allowlisted cref' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte[]') + $allowlist = Join-Path (New-TestDirectory) 'allowlist.json' + '{ "IgnoredCrefs": [ "T:System.Byte[]" ] }' | Set-Content -LiteralPath $allowlist -Encoding utf8 + + { & $scriptPath -SnippetsDirectory $snippets -AllowlistPath $allowlist } | Should -Not -Throw + } + + It 'fails on an allowlisted cref that no longer appears' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte') + $allowlist = Join-Path (New-TestDirectory) 'allowlist.json' + '{ "IgnoredCrefs": [ "T:System.Byte[]" ] }' | Set-Content -LiteralPath $allowlist -Encoding utf8 + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -SnippetsDirectory $snippets -AllowlistPath $allowlist -ReportPath $report } | + Should -Throw '*failed with 1 issue*' + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'stale-allowlist-entry' + $finding.Message | Should -BeLike '*stale*' + } + + It 'honours a configured namespace root' { + $snippets = New-SnippetDirectory -Crefs @('T:Contoso.Widgets.Widget') + $allowlist = Join-Path (New-TestDirectory) 'allowlist.json' + '{ "AllowedNamespaceRoots": [ "Microsoft", "System", "Contoso" ] }' | + Set-Content -LiteralPath $allowlist -Encoding utf8 + + { & $scriptPath -SnippetsDirectory $snippets -AllowlistPath $allowlist } | Should -Not -Throw + } + } + + Context 'reporting and gating' { + + It 'writes the report even when validation fails' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte[]') + $report = Join-Path (New-TestDirectory) 'nested' 'report.json' + + { & $scriptPath -SnippetsDirectory $snippets -ReportPath $report } | Should -Throw + + Test-Path -LiteralPath $report | Should -BeTrue + (Get-Report -Path $report).CountsByCategory.'invalid-docid' | Should -Be 1 + } + + It 'records the file and line of each finding' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte[]') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Path | Should -BeLike '*Sample.xml' + $finding.LineNumber | Should -BeGreaterThan 0 + $finding.Member | Should -Be 'SampleType' + } + + It 'marks the task succeeded-with-issues when findings are reported' { + # task.logissue records an issue but leaves the task result alone, so without this the + # step renders as a clean success despite reporting warnings. + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte[]') + + $output = & $scriptPath -SnippetsDirectory $snippets -ReportOnly *>&1 | Out-String + + $output | Should -Match '##vso\[task\.complete result=SucceededWithIssues;\]' + } + + It 'marks the task succeeded-with-issues for non-gating findings in a gating run' { + # missing-local-uid is a warning, so the run passes, but it must still be visible. + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('T:Microsoft.Data.SqlClient.DoesNotExist') + + $output = & $scriptPath -DocumentationPath $docs *>&1 | Out-String + + $output | Should -Match '##vso\[task\.complete result=SucceededWithIssues;\]' + } + + It 'does not mark the task succeeded-with-issues when there are no findings' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte') + + $output = & $scriptPath -SnippetsDirectory $snippets *>&1 | Out-String + + $output | Should -Not -Match 'task\.complete' + $output | Should -Match 'validation passed' + } + + It 'does not fail in report-only mode' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte[]') + + { & $scriptPath -SnippetsDirectory $snippets -ReportOnly } | Should -Not -Throw + } + + It 'accepts -FailOn as a single comma-separated argument' { + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('T:Microsoft.Data.SqlClient.DoesNotExist') + + { & $scriptPath -DocumentationPath $docs -FailOn 'error,missing-local-uid' } | + Should -Throw '*failed with 1 issue*' + } + + It 'rejects an unknown -FailOn token' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte') + + { & $scriptPath -SnippetsDirectory $snippets -FailOn 'nonsense' } | + Should -Throw "*Unknown -FailOn token 'nonsense'*" + } + + It 'reports every finding in one run rather than stopping at the first' { + $snippets = New-SnippetDirectory -Crefs @( + 'T:System.Byte[]', + 'T:System.Char[]', + 'M:Microsoft.Data.SqlClient.SqlConnection.GetSchema()') + + { & $scriptPath -SnippetsDirectory $snippets } | + Should -Throw '*failed with 3 issues*' + } + } + + Context 'malformed input' { + + It 'reports a file that is not well-formed XML' { + $path = New-TestDirectory + '' | Set-Content -LiteralPath (Join-Path $path 'Broken.xml') -Encoding utf8 + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -SnippetsDirectory $path -ReportPath $report } | Should -Throw + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'malformed-xml' + } + + It 'continues past a malformed file to validate the rest' { + $path = New-TestDirectory + '' | Set-Content -LiteralPath (Join-Path $path 'Broken.xml') -Encoding utf8 + '' | + Set-Content -LiteralPath (Join-Path $path 'Good.xml') -Encoding utf8 + + { & $scriptPath -SnippetsDirectory $path } | Should -Throw '*failed with 2 issues*' + } + + It 'fails when no input paths are supplied' { + { & $scriptPath } | Should -Throw '*No input was supplied*' + } + + It 'fails when a supplied path does not exist' { + { & $scriptPath -SnippetsDirectory (Join-Path $TestDrive 'nope') } | + Should -Throw '*was not found*' + } + + It 'fails when -PackagesPath is supplied without -ExtractPath' { + { & $scriptPath -PackagesPath (New-TestDirectory) } | + Should -Throw '*requires -ExtractPath*' + } + } +} diff --git a/eng/pipelines/onebranch/scripts/validate-localization.ps1 b/eng/pipelines/onebranch/scripts/validate-localization.ps1 index 1922c78c5b..6cc9b59099 100644 --- a/eng/pipelines/onebranch/scripts/validate-localization.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-localization.ps1 @@ -7,6 +7,10 @@ .PARAMETER AllowlistPath Optional JSON file containing approved English-value matches grouped by localized filename. + +.PARAMETER ReportOnly + Report validation findings as warnings without failing. Malformed or missing inputs still fail, + because a run that could not examine the resources has produced no result to report. #> # Licensed to the .NET Foundation under one or more agreements. @@ -19,7 +23,9 @@ param( [ValidateNotNullOrEmpty()] [string]$ResourcesDirectory, - [string]$AllowlistPath + [string]$AllowlistPath, + + [switch]$ReportOnly ) Set-StrictMode -Version Latest @@ -169,12 +175,28 @@ foreach ($localizedFile in $localizedFiles) { } if ($failures.Count -gt 0) { + $issueType = if ($ReportOnly) { 'warning' } else { 'error' } foreach ($failure in $failures) { - Write-Host "##vso[task.logissue type=error]$failure" + Write-Host "##vso[task.logissue type=$issueType]$failure" } + $errorNoun = if ($failures.Count -eq 1) { 'error' } else { 'errors' } - throw "Localization validation failed with $($failures.Count) $errorNoun. Review the preceding errors." + if (-not $ReportOnly) { + throw "Localization validation failed with $($failures.Count) $errorNoun. Review the preceding errors." + } + + Write-Host "##vso[task.logissue type=warning]Localization validation found $($failures.Count) $errorNoun but is running in report-only mode, so the build is not failed." + + # task.logissue attaches an issue to the timeline record but leaves the task result untouched, + # so a step reporting only warnings would still render as a clean success. + Write-Host '##vso[task.complete result=SucceededWithIssues;]' } $fileNoun = if ($localizedFiles.Count -eq 1) { 'file' } else { 'files' } -Write-Host "Localization validation passed for $($localizedFiles.Count) localized $fileNoun. Resource keys checked: $($englishStrings.Count); approved English-value matches allowlisted: $allowedMatchCount." +$summary = "$($localizedFiles.Count) localized $fileNoun. Resource keys checked: $($englishStrings.Count); approved English-value matches allowlisted: $allowedMatchCount." +if ($failures.Count -eq 0) { + Write-Host "Localization validation passed for $summary" +} +else { + Write-Host "Localization validation examined $summary" +} diff --git a/eng/pipelines/onebranch/scripts/validate-packages.ps1 b/eng/pipelines/onebranch/scripts/validate-packages.ps1 index af5149c9f5..fed5f3f5ce 100644 --- a/eng/pipelines/onebranch/scripts/validate-packages.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-packages.ps1 @@ -59,6 +59,11 @@ dotnet executable to invoke. Defaults to the dotnet command resolved from PATH. This parameter primarily supports isolated testing. +.PARAMETER ReportOnly + Report gate findings as warnings without failing. This suppresses only the --fail-on gate: a + run in which PackageValidator itself failed still fails the step, because it produced no + trustworthy findings to report. + .EXAMPLE ./validate-packages.ps1 ` -ValidatorPath ./PackageValidator.dll ` @@ -115,7 +120,10 @@ param( [Parameter(HelpMessage = "dotnet executable to invoke.")] [ValidateNotNullOrEmpty()] - [string]$DotnetPath = "dotnet" + [string]$DotnetPath = "dotnet", + + [Parameter(HelpMessage = "Report gate findings as warnings without failing the build.")] + [switch]$ReportOnly ) Set-StrictMode -Version Latest @@ -136,6 +144,7 @@ Write-Host "SqlClientFileVersion: ${SqlClientFileVersion}" Write-Host "SqlServerPackageVersion: ${SqlServerPackageVersion}" Write-Host "SqlServerFileVersion: ${SqlServerFileVersion}" Write-Host "FailOn: $($failOnTokens -join ', ')" +Write-Host "ReportOnly: ${ReportOnly}" Write-Host "====================================" if (-not (Test-Path -LiteralPath $ValidatorPath)) { @@ -206,7 +215,18 @@ if ($exitCode -eq 0) { Write-Host "Package validation passed." } elseif ($exitCode -eq 2) { - throw "Package validation failed: one or more findings matched the gate ($($failOnTokens -join ', '))." + $message = "Package validation failed: one or more findings matched the gate ($($failOnTokens -join ', '))." + if (-not $ReportOnly) { + throw $message + } + + # Report-only suppresses a tripped gate, but not a validator that failed to run: the latter + # produced no trustworthy findings, so there is nothing to downgrade to a warning. + Write-Host "##vso[task.logissue type=warning]${message} Running in report-only mode, so the build is not failed." + + # task.logissue attaches an issue to the timeline record but leaves the task result untouched, + # so a step reporting only warnings would still render as a clean success. + Write-Host '##vso[task.complete result=SucceededWithIssues;]' } else { throw "PackageValidator exited unexpectedly with code ${exitCode}." diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 new file mode 100644 index 0000000000..0f3ea33cdd --- /dev/null +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -0,0 +1,1147 @@ +<# +.SYNOPSIS + Validates the XML documentation cross-references that feed dotnet/sqlclient-api-docs. + +.DESCRIPTION + Open Publishing resolves every // cref in our XML documentation + against the Learn xref map. A cref whose documentation ID is malformed cannot resolve, and the + resulting xref-not-found warnings only surface after an API Docs pull request has already been + opened. This script is the local preflight for that round trip. + + Validation runs in two modes, and a single invocation may use either or both: + + Source mode (-SnippetsDirectory) reads the doc/snippets files that the compiler pulls into + the generated XML via . It needs no build, so it gates in seconds and reports the + file and line a developer actually edits. + + Documentation mode (-DocumentationPath) reads generated or packaged XML documentation. It + sees what actually ships, so it additionally catches crefs the compiler failed to bind (which + it rewrites to a "!:" prefix) and can resolve our own UIDs against the members the build + really emitted. + + The rules are deliberately offline. Every xref-not-found warning reported against + dotnet/sqlclient-api-docs PR 99 is a documentation-ID syntax defect or a namespace typo, both + of which are detectable without consulting the Learn xref service. Avoiding that service keeps + the check usable where outbound network access is unavailable, and avoids downloading the + published Learn .NET xref map, which is roughly 338 MB. Resolution against that map is + optional, via -ExternalXrefMapPath. + + Findings are collected rather than thrown one at a time, so a single run reports every defect. + + Package mode additionally checks how documentation was mapped into the package. The driver + ships two XML documentation files per target framework and they are required to differ: lib/ + carries the full text, while ref/ has and stripped, because those render + badly in Visual Studio IntelliSense. The two have been silently collapsed before -- in 7.1.0 + the modern lib/ XML was byte-identical to the trimmed ref/ XML, so IntelliSense lost every + remark and example. The lib/ref pairs are therefore compared against each other rather than + against an absolute expectation, which keeps the rule meaningful for packages that legitimately + have no ref/ folder at all. + +.PARAMETER SnippetsDirectory + One or more directories scanned recursively for documentation snippet .xml files. + +.PARAMETER DocumentationPath + One or more generated XML documentation files, or directories scanned recursively for them. + Only files carrying a / structure are treated as XML documentation; anything else + found in a scanned directory is ignored, so a build output tree may be passed wholesale. + +.PARAMETER PackagesPath + Optional directory scanned recursively for .nupkg files. Each is expanded beneath -ExtractPath + and the XML documentation inside it is validated, which is what a consumer actually receives. + Requires -ExtractPath. + +.PARAMETER ExtractPath + Directory that -PackagesPath expands into. Existing expansions are replaced so a rerun cannot + validate stale content. + +.PARAMETER AllowlistPath + Optional JSON file carrying approved exceptions and namespace configuration. Recognized keys: + + AllowedNamespaceRoots Leading identifiers a prefixed cref may use. Default: Microsoft, + System, Interop. Interop is included because the implementation + assembly documents its internal P/Invoke types, which never reach + the published documentation but do appear in the lib/ XML. + LocalNamespacePrefixes Namespaces owned by this repository, which must resolve against the + documentation being validated. Default: Microsoft.Data, Microsoft.SqlServer. + IgnoredCrefs Exact cref values to exempt from all cref rules. + + Ignored crefs that no longer appear are reported as stale, so an obsolete exception cannot + silently weaken future validation. + +.PARAMETER ExternalXrefMapPath + Optional path to a downloaded Learn .NET xref map (.xrefmap.json). When supplied, crefs outside + the local namespaces are additionally resolved against it. The published map is roughly 338 MB + and must be downloaded separately, so this is intended for local investigation rather than + routine use. + +.PARAMETER ReportPath + Optional path of a JSON report to write. Parent directories are created as needed. The report + is always written before gating, so it exists even when validation fails. + +.PARAMETER FailOn + Finding severities and/or categories that fail the build. Severities are error, warning and + info; categories are listed in the table below. Defaults to error. + + Accepts either an array or a single comma-separated string, because an Azure Pipelines task + argument line collapses to one token and PowerShell's -File mode does not split it. + +.PARAMETER ReportOnly + Report findings without failing, overriding -FailOn. Used to shake the gate out on a pipeline + before its findings are fixed. The official pipeline never sets this. + +.PARAMETER ProjectSearchRoot + Directory scanned recursively for project files, used with -PackagesPath to check that every + assembly whose project sets GenerateDocumentationFile ships its XML documentation beside it in + lib/ and ref/. Derived from the projects rather than a list, so it stays correct as packages + are added or change. + +.PARAMETER ProjectPath + Project file whose documentation expectations apply to the supplied inputs. The project is the + declaration, so nothing needs restating in the caller: + + GenerateDocumentationFile=true XML documentation must be produced; its absence is an error. + otherwise No XML documentation is expected. Its absence is reported as + information naming the reason, and its presence is reported + as a warning, so the project and the build cannot disagree + silently. + + When the project's sources reference documentation snippets, only the snippet files they + reference are validated; a project referencing none is reported as information. + +.OUTPUTS + Findings are categorized as: + + malformed-xml error File is not well-formed XML. + unresolved-cref error Compiler could not bind the cref and emitted a "!:" prefix. + invalid-docid error Documentation ID violates the documentation-ID grammar. + unknown-namespace-root error Leading identifier is not an allowed namespace root. + stale-allowlist-entry error Allowlisted cref no longer appears in any validated file. + lib-documentation-trimmed + error A package's lib/ XML has no remarks or examples, so + IntelliSense would show only summaries. + ref-documentation-untrimmed + error A package's ref/ XML still carries remarks or examples. + lib-ref-documentation-identical + error A package's lib/ and ref/ XML are byte-identical, so the + nuspec mapped one artifact into both targets. + missing-local-uid warning Cref names this repository but no such member was emitted. + Warning rather than error because a member may legitimately + be absent from the documentation under validation: it may be + conditional on another target framework, or defined in a + sibling assembly that this run did not include. + mismatched-docid-prefix warning Cref names a real member but with the wrong kind prefix, + such as M: on a property or T: on a member. + missing-external-uid warning Cref is absent from the supplied Learn xref map. + unprefixed-cref info Cref carries no "T:"/"M:"/... prefix. Legal; the compiler + binds it. Reported for visibility only. + missing-documentation error Documentation that should exist does not. A project sets + GenerateDocumentationFile but produced none, or a package + ships an assembly without its documentation file. + unexpected-documentation + warning Documentation was found for a project that does not set + GenerateDocumentationFile. The project and the build + disagree; the documentation is still validated. + +.EXAMPLE + ./validate-xml-docs.ps1 -SnippetsDirectory ./doc/snippets + +.EXAMPLE + ./validate-xml-docs.ps1 ` + -DocumentationPath ./artifacts/bin ` + -ReportPath ./out/xml-docs-validation.json ` + -FailOn error,missing-local-uid +#> + +# Licensed to the .NET Foundation under one or more agreements. +# The .NET Foundation licenses this file to you under the MIT license. +# See the LICENSE file in the project root for more information. + +[CmdletBinding()] +param( + [string[]]$SnippetsDirectory, + + [string[]]$DocumentationPath, + + [string]$PackagesPath, + + [string]$ExtractPath, + + [string]$AllowlistPath, + + [string]$ExternalXrefMapPath, + + [string]$ReportPath, + + [string[]]$FailOn = @('error'), + + [switch]$ReportOnly, + + [string]$ProjectPath, + + [string]$ProjectSearchRoot +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' + +# Severity of each finding category. Only the categories named here may be produced, and -FailOn +# gates on either a category or the severity it maps to. +$script:CategorySeverities = [ordered]@{ + 'malformed-xml' = 'error' + 'unresolved-cref' = 'error' + 'invalid-docid' = 'error' + 'unknown-namespace-root' = 'error' + 'stale-allowlist-entry' = 'error' + 'lib-documentation-trimmed' = 'error' + 'ref-documentation-untrimmed' = 'error' + 'lib-ref-documentation-identical' = 'error' + 'missing-local-uid' = 'warning' + 'mismatched-docid-prefix' = 'warning' + 'missing-documentation' = 'error' + 'unexpected-documentation' = 'warning' + 'documentation-not-expected' = 'info' + 'missing-external-uid' = 'warning' + 'unprefixed-cref' = 'info' +} + +# C# keyword aliases. A documentation ID names CLR types, so an alias in a cref is always a defect +# even though it reads correctly in source. +$script:CSharpAliases = [System.Collections.Generic.HashSet[string]]::new( + [string[]]@( + 'bool', 'byte', 'char', 'decimal', 'double', 'float', 'int', 'long', 'nint', 'nuint', + 'object', 'sbyte', 'short', 'string', 'uint', 'ulong', 'ushort', 'void' + ), + [System.StringComparer]::Ordinal) + +# Documentation ID prefixes defined by the C# specification, plus "!" which the compiler emits for +# a cref it could not bind. +$script:KnownDocIdPrefixes = [System.Collections.Generic.HashSet[string]]::new( + [string[]]@('N', 'T', 'F', 'P', 'M', 'E', '!'), + [System.StringComparer]::Ordinal) + +$script:Findings = [System.Collections.Generic.List[object]]::new() + +function Add-Finding { + param( + [Parameter(Mandatory)][string]$Category, + [Parameter(Mandatory)][string]$Message, + [string]$Path, + [int]$LineNumber, + [string]$Cref, + [string]$Member + ) + + if (-not $script:CategorySeverities.Contains($Category)) { + throw "Internal error: unknown finding category '$Category'." + } + + $script:Findings.Add([pscustomobject]@{ + Category = $Category + Severity = $script:CategorySeverities[$Category] + Message = $Message + Path = $Path + LineNumber = $LineNumber + Cref = $Cref + Member = $Member + }) +} + +<# + Splits a documentation ID argument list on commas that sit outside any nesting. Generic + arguments use braces in a documentation ID (List{System.String}), so a naive split on comma + would tear nested generics apart and report phantom parameters. +#> +function Split-DocIdArguments { + param([Parameter(Mandatory)][AllowEmptyString()][string]$Arguments) + + $parts = [System.Collections.Generic.List[string]]::new() + $depth = 0 + $start = 0 + for ($index = 0; $index -lt $Arguments.Length; $index++) { + switch ($Arguments[$index]) { + '{' { $depth++ } + '(' { $depth++ } + '[' { $depth++ } + '}' { $depth-- } + ')' { $depth-- } + ']' { $depth-- } + ',' { + if ($depth -eq 0) { + $parts.Add($Arguments.Substring($start, $index - $start)) + $start = $index + 1 + } + } + } + } + $parts.Add($Arguments.Substring($start)) + + return $parts +} + +<# + Reduces a documentation ID parameter to the bare type identifier so it can be compared against + the C# alias set: array, pointer and by-reference markers are stripped, as are generic + arguments, which are validated separately as parameters in their own right. +#> +function Get-DocIdCoreTypeName { + param([Parameter(Mandatory)][AllowEmptyString()][string]$TypeName) + + $core = $TypeName.Trim() + $braceIndex = $core.IndexOf('{') + if ($braceIndex -ge 0) { + $core = $core.Substring(0, $braceIndex) + } + + return $core.TrimEnd('[', ']', '@', '*', '&') +} + +<# + Applies the documentation-ID grammar to one cref and records every violation it carries. + + Ordering matters: a cref is reported against the most specific rule that explains it, and + reporting stops there. A cref such as "T:string" is an alias defect, not an unknown namespace + root, and emitting both would send a reader chasing the wrong fix. +#> +function Test-Cref { + param( + [Parameter(Mandatory)][string]$Cref, + [Parameter(Mandatory)][hashtable]$Context + ) + + $trimmed = $Cref.Trim() + if ([string]::IsNullOrEmpty($trimmed)) { + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message 'Cref is empty.' + return + } + + # An unprefixed cref is bound by the compiler from the surrounding source context, so its + # documentation ID is generated rather than authored and none of the grammar rules below apply. + if ($trimmed.Length -lt 2 -or $trimmed[1] -ne ':') { + Add-Finding @Context -Category 'unprefixed-cref' -Cref $Cref -Message ( + "Cref '$trimmed' has no documentation-ID prefix. The compiler binds it from source " + + 'context, so it is not validated here.') + return + } + + $prefix = [string]$trimmed[0] + $body = $trimmed.Substring(2) + + if (-not $script:KnownDocIdPrefixes.Contains($prefix)) { + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' uses unknown documentation-ID prefix '${prefix}:'. Expected one of " + + 'N:, T:, F:, P:, M: or E:.') + return + } + + if ($prefix -eq '!') { + Add-Finding @Context -Category 'unresolved-cref' -Cref $Cref -Message ( + "Cref '$body' could not be bound by the compiler, which emitted it as '!:'. It will " + + 'never resolve in the published documentation.') + return + } + + if ([string]::IsNullOrWhiteSpace($body)) { + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' has a '${prefix}:' prefix but no member identifier.") + return + } + + # A documentation ID is a single token. Whitespace anywhere inside it, most commonly a space + # after a comma in a signature copied from C# source, prevents the xref from resolving. + # Validation continues against the whitespace-stripped form, because such a cref is usually + # pasted from source and carries alias defects too, and reporting only the whitespace would + # send the author back for a second round. + if ($body -match '\s') { + $body = ($body -replace '\s', '') + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' contains whitespace. Documentation IDs contain no whitespace; use " + + "'${prefix}:$body'.") + } + + $signatureStart = $body.IndexOf('(') + $namePart = if ($signatureStart -ge 0) { $body.Substring(0, $signatureStart) } else { $body } + + if ($signatureStart -ge 0) { + if (-not $body.Contains(')')) { + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' has an unterminated parameter list.") + return + } + + # The conversion-operator return marker (~) trails the parameter list, so the argument text + # ends at the last ')' rather than at the end of the body. + $signatureEnd = $body.LastIndexOf(')') + $arguments = $body.Substring($signatureStart + 1, $signatureEnd - $signatureStart - 1) + + # A parameterless method's documentation ID is written without parentheses. Emitting "()" + # produces a UID that matches nothing. + if ([string]::IsNullOrWhiteSpace($arguments)) { + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' declares an empty parameter list. A parameterless member omits " + + "the parentheses entirely; use '${prefix}:$namePart'.") + return + } + + # A signature may repeat the same alias (string and string[] both reduce to string), so + # report the distinct offenders once rather than once per parameter. + $aliases = [System.Collections.Generic.List[string]]::new() + foreach ($argument in (Split-DocIdArguments -Arguments $arguments)) { + $core = Get-DocIdCoreTypeName -TypeName $argument + if ($script:CSharpAliases.Contains($core) -and -not $aliases.Contains($core)) { + $aliases.Add($core) + } + } + if ($aliases.Count -gt 0) { + $quoted = ($aliases | ForEach-Object { "'$_'" }) -join ', ' + $noun = if ($aliases.Count -eq 1) { 'the C# alias' } else { 'the C# aliases' } + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' uses $noun $quoted in its parameter list. Documentation IDs name " + + 'CLR types, so use the full type name instead.') + return + } + } + + # An array, pointer or by-reference construction is not a named type, so it has no type page + # and no UID in the Learn xref map. Only a T: cref can make this mistake; the same suffixes are + # legal inside a member signature. + if ($prefix -eq 'T' -and $body -match '(\[\]|\*|@|&)$') { + $element = $body -replace '(\[\]|\*|@|&)+$', '' + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' names a constructed type, which has no documentation page. " + + "Reference the element type instead, for example array.") + return + } + + if ($script:CSharpAliases.Contains((Get-DocIdCoreTypeName -TypeName $namePart))) { + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' uses the C# alias '$namePart'. Documentation IDs name CLR types, so " + + 'use the full type name instead.') + return + } + + # The leading identifier catches misspelled namespaces, which are otherwise indistinguishable + # from a valid reference to a type this build does not contain. + $root = ($namePart -split '[.`{]', 2)[0] + if (-not $script:AllowedNamespaceRoots.Contains($root)) { + $allowed = ($script:AllowedNamespaceRoots | Sort-Object) -join ', ' + Add-Finding @Context -Category 'unknown-namespace-root' -Cref $Cref -Message ( + "Cref '$trimmed' starts with unknown namespace root '$root'. Allowed roots: $allowed. " + + 'Check for a misspelled namespace.') + return + } + + # Anything this repository owns must be present in the documentation under validation. This can + # only be judged when generated documentation was supplied; source mode has no member list. + # + # Namespaces are excluded because the compiler never emits a entry for one, so every + # N: cref would otherwise be reported as missing. + if ($null -ne $script:LocalUids -and $prefix -ne 'N') { + $isLocal = $false + foreach ($localPrefix in $script:LocalNamespacePrefixes) { + if ($namePart -eq $localPrefix -or $namePart.StartsWith("$localPrefix.", [System.StringComparison]::Ordinal)) { + $isLocal = $true + break + } + } + + if ($isLocal) { + if (-not $script:LocalUids.Contains($trimmed)) { + # The same member under a different prefix is the common case here: a cref written + # as M: for a property, or T: for a member, names something real but produces a UID + # that matches nothing. Say which prefix was expected rather than reporting a bare + # lookup failure. + if ($script:LocalUidsByBody.ContainsKey($namePart) -or $script:LocalUidsByBody.ContainsKey($body)) { + $key = if ($script:LocalUidsByBody.ContainsKey($body)) { $body } else { $namePart } + $actual = ($script:LocalUidsByBody[$key] | Sort-Object) -join ', ' + Add-Finding @Context -Category 'mismatched-docid-prefix' -Cref $Cref -Message ( + "Cref '$trimmed' uses prefix '${prefix}:', but '$key' was emitted as " + + "'$actual'. Use the prefix matching the member kind.") + } + else { + Add-Finding @Context -Category 'missing-local-uid' -Cref $Cref -Message ( + "Cref '$trimmed' names this repository but no matching documented member " + + 'was emitted by the build. The reference will not resolve.') + } + } + return + } + } + + if ($null -ne $script:ExternalUids -and -not $script:ExternalUids.Contains($body)) { + Add-Finding @Context -Category 'missing-external-uid' -Cref $Cref -Message ( + "Cref '$trimmed' was not found in the supplied Learn xref map.") + } +} + +<# + Reads GenerateDocumentationFile from a project file. + + The project is parsed as XML rather than searched as text, because a comment mentioning the + property would otherwise be read as setting it. Only the last assignment is honoured, matching + MSBuild's last-one-wins evaluation within a file. +#> +function Test-ProjectGeneratesDocumentation { + param([Parameter(Mandatory)][string]$Path) + + $document = [System.Xml.Linq.XDocument]::Load($Path) + $value = $null + foreach ($element in $document.Descendants()) { + if ($element.Name.LocalName -eq 'GenerateDocumentationFile') { + $value = $element.Value.Trim() + } + } + + return $value -eq 'true' +} + +<# + Returns the documentation files referenced by a project's sources. + + Snippets reach the compiler through in a doc comment, so the references + live in the source files rather than in the project file. Paths are relative to the file that + declares them. Every include is returned; the caller narrows them to the snippet directory it + was given, so no assumption is made about where snippets live. +#> +function Get-ReferencedSnippetFile { + param([Parameter(Mandatory)][string]$ProjectDirectory) + + $pattern = [regex]"include\s+file\s*=\s*'([^']+)'" + $referenced = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::OrdinalIgnoreCase) + + foreach ($source in Get-ChildItem -LiteralPath $ProjectDirectory -Filter '*.cs' -File -Recurse -ErrorAction SilentlyContinue) { + foreach ($match in $pattern.Matches([System.IO.File]::ReadAllText($source.FullName))) { + $candidate = Join-Path $source.DirectoryName $match.Groups[1].Value + try { + $resolved = [System.IO.Path]::GetFullPath($candidate) + } + catch { + continue + } + [void]$referenced.Add($resolved) + } + } + + # Comma prevents PowerShell from unrolling the set on return, which would yield $null for an + # empty set and a bare string for a single entry. + return , $referenced +} + +<# + Returns the assembly names whose projects generate XML documentation. + + AssemblyName falls back to the project file name, matching MSBuild. Two projects may share an + assembly name, such as an implementation and its reference assembly, so the result is a set. +#> +function Get-DocumentedAssemblyName { + param([Parameter(Mandatory)][string]$SearchRoot) + + $names = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::OrdinalIgnoreCase) + foreach ($project in Get-ChildItem -LiteralPath $SearchRoot -Filter '*.csproj' -File -Recurse -ErrorAction SilentlyContinue) { + if (-not (Test-ProjectGeneratesDocumentation -Path $project.FullName)) { + continue + } + + $document = [System.Xml.Linq.XDocument]::Load($project.FullName) + $assemblyName = $null + foreach ($element in $document.Descendants()) { + if ($element.Name.LocalName -eq 'AssemblyName') { + $assemblyName = $element.Value.Trim() + } + } + + if ([string]::IsNullOrWhiteSpace($assemblyName)) { + $assemblyName = [System.IO.Path]::GetFileNameWithoutExtension($project.FullName) + } + + [void]$names.Add($assemblyName) + } + + return , $names +} + +function Resolve-InputPaths { + param( + [string[]]$Paths, + [Parameter(Mandatory)][string]$Description + ) + + $resolved = [System.Collections.Generic.List[string]]::new() + foreach ($path in @($Paths)) { + if ([string]::IsNullOrWhiteSpace($path)) { + continue + } + if (-not (Test-Path -LiteralPath $path)) { + throw "$Description path '$path' was not found." + } + + $item = Get-Item -LiteralPath $path + if ($item.PSIsContainer) { + foreach ($file in Get-ChildItem -LiteralPath $item.FullName -Filter '*.xml' -File -Recurse) { + $resolved.Add($file.FullName) + } + } + else { + $resolved.Add($item.FullName) + } + } + + return ($resolved | Sort-Object -Unique) +} + +# Load configuration ------------------------------------------------------------------------- + +$script:AllowedNamespaceRoots = [System.Collections.Generic.HashSet[string]]::new( + [string[]]@('Microsoft', 'System', 'Interop'), [System.StringComparer]::Ordinal) +$script:LocalNamespacePrefixes = @('Microsoft.Data', 'Microsoft.SqlServer') +$ignoredCrefs = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) +$observedIgnoredCrefs = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) + +if (-not [string]::IsNullOrWhiteSpace($AllowlistPath)) { + if (-not (Test-Path -LiteralPath $AllowlistPath -PathType Leaf)) { + throw "XML documentation allowlist file '$AllowlistPath' was not found." + } + + $configuration = Get-Content -LiteralPath $AllowlistPath -Raw | ConvertFrom-Json + + $rootsProperty = $configuration.PSObject.Properties['AllowedNamespaceRoots'] + if ($null -ne $rootsProperty) { + $script:AllowedNamespaceRoots = [System.Collections.Generic.HashSet[string]]::new( + [string[]]@($rootsProperty.Value), [System.StringComparer]::Ordinal) + } + + $localProperty = $configuration.PSObject.Properties['LocalNamespacePrefixes'] + if ($null -ne $localProperty) { + $script:LocalNamespacePrefixes = @($localProperty.Value) + } + + $ignoredProperty = $configuration.PSObject.Properties['IgnoredCrefs'] + if ($null -ne $ignoredProperty) { + foreach ($cref in @($ignoredProperty.Value)) { + if ([string]::IsNullOrWhiteSpace($cref)) { + throw "XML documentation allowlist file '$AllowlistPath' contains an empty ignored cref." + } + if (-not $ignoredCrefs.Add($cref)) { + throw "XML documentation allowlist file '$AllowlistPath' contains duplicate ignored cref '$cref'." + } + } + } +} + +# Gather inputs ------------------------------------------------------------------------------ + +$snippetFiles = @(Resolve-InputPaths -Paths $SnippetsDirectory -Description 'Snippets') + +# The project file is the declaration for what documentation should exist, so nothing needs +# restating by the caller. +$documentationExpected = $false +$projectDescription = '' +$referencedSnippets = $null +$documentedAssemblies = $null + +if (-not [string]::IsNullOrWhiteSpace($ProjectSearchRoot)) { + if (-not (Test-Path -LiteralPath $ProjectSearchRoot)) { + throw "Project search root '$ProjectSearchRoot' was not found." + } + + $documentedAssemblies = Get-DocumentedAssemblyName -SearchRoot $ProjectSearchRoot + Write-Host ("Assemblies whose projects generate XML documentation: " + + "$(($documentedAssemblies | Sort-Object) -join ', ').") +} + +if (-not [string]::IsNullOrWhiteSpace($ProjectPath)) { + if (-not (Test-Path -LiteralPath $ProjectPath -PathType Leaf)) { + throw "Project file '$ProjectPath' was not found." + } + + $projectFile = (Resolve-Path -LiteralPath $ProjectPath).Path + $projectDescription = [System.IO.Path]::GetFileName($projectFile) + $documentationExpected = Test-ProjectGeneratesDocumentation -Path $projectFile + $referencedSnippets = Get-ReferencedSnippetFile -ProjectDirectory ([System.IO.Path]::GetDirectoryName($projectFile)) + + Write-Host ("Project '$projectDescription': GenerateDocumentationFile=$documentationExpected; " + + "documentation snippets referenced: $($referencedSnippets.Count).") + + # Validate only the snippets this project pulls in, rather than every snippet in the tree, so a + # finding is attributed to a build that actually consumes it. + if ($snippetFiles.Count -gt 0) { + $snippetFiles = @($snippetFiles | Where-Object { $referencedSnippets.Contains($_) }) + } +} +$documentationRoots = [System.Collections.Generic.List[string]]::new() + +# Expanded package directory -> package file name, used by the lib/ref layout checks below. +$script:ExpandedPackageRoots = [ordered]@{} +foreach ($path in @($DocumentationPath)) { + if (-not [string]::IsNullOrWhiteSpace($path)) { + $documentationRoots.Add($path) + } +} + +# Expand any packages so the XML a consumer actually receives is validated, not only the build +# output it was assembled from. +if (-not [string]::IsNullOrWhiteSpace($PackagesPath)) { + if ([string]::IsNullOrWhiteSpace($ExtractPath)) { + throw '-PackagesPath requires -ExtractPath.' + } + if (-not (Test-Path -LiteralPath $PackagesPath)) { + throw "Packages path '$PackagesPath' was not found." + } + + $packages = @(Get-ChildItem -Path $PackagesPath -Recurse -File -Filter *.nupkg -ErrorAction SilentlyContinue) + if ($packages.Count -eq 0) { + throw "No .nupkg files were found under '$PackagesPath'." + } + + New-Item -ItemType Directory -Force -Path $ExtractPath | Out-Null + + Add-Type -AssemblyName System.IO.Compression.FileSystem + foreach ($package in $packages) { + $destination = Join-Path $ExtractPath $package.BaseName + if (Test-Path -LiteralPath $destination) { + Remove-Item -LiteralPath $destination -Recurse -Force + } + + Write-Host "Expanding $($package.Name)" + [System.IO.Compression.ZipFile]::ExtractToDirectory($package.FullName, $destination) + $script:ExpandedPackageRoots[$destination] = $package.Name + } + + $documentationRoots.Add((Resolve-Path -LiteralPath $ExtractPath).Path) +} + +$documentationCandidates = @(Resolve-InputPaths -Paths $documentationRoots -Description 'Documentation') + +# Distinguish "nothing was asked for", which is a caller error, from "what was asked for held no +# documentation", which is reported below as a finding. +$snippetsRequested = @(@($SnippetsDirectory) | Where-Object { -not [string]::IsNullOrWhiteSpace($_) }).Count -gt 0 +$documentationRequested = $documentationRoots.Count -gt 0 + +if (-not $snippetsRequested -and -not $documentationRequested) { + throw 'No input was supplied. Supply -SnippetsDirectory, -DocumentationPath, -PackagesPath, or a combination.' +} + +# Parse every file up front so that a malformed file is reported as a finding rather than aborting +# the run, and so the local UID index is complete before any cref is resolved. +$documents = [System.Collections.Generic.List[object]]::new() +$malformedByKind = @{ snippet = 0; documentation = 0 } +foreach ($entry in @( + @{ Files = $snippetFiles; Kind = 'snippet' }, + @{ Files = $documentationCandidates; Kind = 'documentation' })) { + + foreach ($file in $entry.Files) { + try { + $document = [System.Xml.Linq.XDocument]::Load($file, [System.Xml.Linq.LoadOptions]::SetLineInfo) + } + catch { + Add-Finding -Category 'malformed-xml' -Path $file -Message ( + "File is not well-formed XML: $($_.Exception.Message)") + $malformedByKind[$entry.Kind]++ + continue + } + + # A scanned directory may hold far more .xml files than documentation. Recognize + # documentation by its shape and silently skip everything else, so a whole + # output directory can be supplied without pre-filtering it. + if ($entry.Kind -eq 'documentation') { + if ($null -eq $document.Root -or + $document.Root.Name.LocalName -ne 'doc' -or + $null -eq $document.Root.Element('members')) { + continue + } + } + + $documents.Add([pscustomobject]@{ + Path = $file + Kind = $entry.Kind + Document = $document + RemarksCount = @($document.Descendants('remarks')).Count + ExampleCount = @($document.Descendants('example')).Count + }) + } +} + +$documentationDocuments = @($documents | Where-Object { $_.Kind -eq 'documentation' }) + +# Whether documentation must exist is the caller's declaration, not an inference. Asserting it both +# ways is what keeps the declaration honest: documentation vanishing from a package that should +# have it is an error, and documentation appearing in one that should not have it means the +# declaration is stale. Without that, a package emitting nothing and a package that silently +# stopped emitting look identical. +# +# Files that were found but failed to parse are excluded: malformed-xml already names the problem, +# and adding a "nothing was found" finding on top of it would point at the wrong cause. +$snippetDocumentCount = @($documents | Where-Object { $_.Kind -eq 'snippet' }).Count + +if ($snippetsRequested -and $snippetDocumentCount -eq 0 -and $malformedByKind['snippet'] -eq 0) { + if ($null -ne $referencedSnippets) { + Add-Finding -Category 'documentation-not-expected' -Message ( + "$projectDescription references no documentation snippets, so none were validated.") + } + elseif ($documentationExpected) { + Add-Finding -Category 'missing-documentation' -Message ( + 'No documentation snippet files were found under the supplied -SnippetsDirectory.') + } +} + +if ($documentationRequested -and $malformedByKind['documentation'] -eq 0) { + $hasProject = -not [string]::IsNullOrEmpty($projectDescription) + + if ($documentationExpected -and $documentationDocuments.Count -eq 0) { + $reason = if ($hasProject) { + "$projectDescription sets GenerateDocumentationFile" + } + else { + 'documentation was required' + } + Add-Finding -Category 'missing-documentation' -Message ( + "No XML documentation was found under the supplied -DocumentationPath or " + + "-PackagesPath, but $reason. Documentation that should exist is missing.") + } + # The remaining cases contradict a project's declaration, so they are only meaningful when a + # project was supplied. Without one there is nothing for the result to disagree with. + elseif ($hasProject -and -not $documentationExpected -and $documentationDocuments.Count -eq 0) { + # Stated rather than passed over in silence, so a run shows why nothing was checked. + Add-Finding -Category 'documentation-not-expected' -Message ( + "No XML documentation was found, and none is expected: $projectDescription does not " + + 'set GenerateDocumentationFile.') + } + elseif ($hasProject -and -not $documentationExpected -and $documentationDocuments.Count -gt 0) { + # The project and the build disagree; the documentation is still validated. + Add-Finding -Category 'unexpected-documentation' -Message ( + "$($documentationDocuments.Count) XML documentation file(s) were found although " + + "$projectDescription does not set GenerateDocumentationFile.") + } +} + +# Build the local UID index from the documented members the build emitted. Source mode alone leaves +# this null, which disables local resolution rather than reporting every cref as unresolvable. +$script:LocalUids = $null +$script:LocalUidsByBody = [System.Collections.Generic.Dictionary[string, System.Collections.Generic.HashSet[string]]]::new( + [System.StringComparer]::Ordinal) +if ($documentationDocuments.Count -gt 0) { + $script:LocalUids = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) + foreach ($entry in $documentationDocuments) { + foreach ($member in $entry.Document.Root.Element('members').Elements('member')) { + $name = $member.Attribute('name') + if ($null -eq $name -or [string]::IsNullOrWhiteSpace($name.Value)) { + continue + } + + $uid = $name.Value.Trim() + [void]$script:LocalUids.Add($uid) + + # Index the identifier without its prefix so a cref carrying the wrong prefix can be + # told apart from one naming a member that was never emitted at all. + if ($uid.Length -gt 2 -and $uid[1] -eq ':') { + $body = $uid.Substring(2) + if (-not $script:LocalUidsByBody.ContainsKey($body)) { + $script:LocalUidsByBody[$body] = [System.Collections.Generic.HashSet[string]]::new( + [System.StringComparer]::Ordinal) + } + [void]$script:LocalUidsByBody[$body].Add("$($uid[0]):") + } + } + } +} + +# Load the external Learn xref map when one was supplied. It is large, so only the UID column is +# retained. +$script:ExternalUids = $null +if (-not [string]::IsNullOrWhiteSpace($ExternalXrefMapPath)) { + if (-not (Test-Path -LiteralPath $ExternalXrefMapPath -PathType Leaf)) { + throw "External xref map '$ExternalXrefMapPath' was not found." + } + + Write-Host "Loading external xref map '$ExternalXrefMapPath'. This may take several minutes." + $map = Get-Content -LiteralPath $ExternalXrefMapPath -Raw | ConvertFrom-Json + $script:ExternalUids = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) + foreach ($reference in @($map.references)) { + if ($null -ne $reference.uid) { + [void]$script:ExternalUids.Add([string]$reference.uid) + } + } + Write-Host "Loaded $($script:ExternalUids.Count) external UIDs." +} + +# Validate ----------------------------------------------------------------------------------- + +$crefCount = 0 +foreach ($entry in $documents) { + foreach ($element in $entry.Document.Descendants()) { + $crefAttribute = $element.Attribute('cref') + if ($null -eq $crefAttribute) { + continue + } + + $crefCount++ + + if ($ignoredCrefs.Contains($crefAttribute.Value)) { + [void]$observedIgnoredCrefs.Add($crefAttribute.Value) + continue + } + + # Name the nearest enclosing documented member so a finding in a large generated file can + # be traced back to the API it documents. + $member = '' + $ancestor = $element + while ($null -ne $ancestor) { + if ($ancestor.Name.LocalName -eq 'member') { + $nameAttribute = $ancestor.Attribute('name') + if ($null -ne $nameAttribute) { + $member = $nameAttribute.Value + } + break + } + if ($ancestor.Name.LocalName -eq 'members') { + $nameAttribute = $ancestor.Attribute('name') + if ($null -ne $nameAttribute) { + $member = $nameAttribute.Value + } + break + } + $ancestor = $ancestor.Parent + } + + $lineInfo = [System.Xml.IXmlLineInfo]$crefAttribute + $context = @{ + Path = $entry.Path + LineNumber = if ($lineInfo.HasLineInfo()) { $lineInfo.LineNumber } else { 0 } + Member = $member + } + + Test-Cref -Cref $crefAttribute.Value -Context $context + } +} + +# Package layout ------------------------------------------------------------------------------ + +# The driver ships a full XML documentation file under lib/ and a trimmed one under ref/. Compare +# the two per target framework rather than testing either against an absolute expectation, so the +# rule stays correct for packages that have no ref/ folder and needs no list of which packages do. +if ($script:ExpandedPackageRoots.Count -gt 0) { + $documentationByPath = @{} + foreach ($entry in $documentationDocuments) { + $documentationByPath[$entry.Path] = $entry + } + + # An assembly whose project generates documentation must carry it into the package. Checked + # here rather than from the build output because only the package shows what a consumer + # receives, and a package can drop a file the build produced. + if ($null -ne $documentedAssemblies) { + foreach ($packageRoot in $script:ExpandedPackageRoots.Keys) { + $packageName = $script:ExpandedPackageRoots[$packageRoot] + + foreach ($folder in @('lib', 'ref')) { + $folderPath = Join-Path $packageRoot $folder + if (-not (Test-Path -LiteralPath $folderPath -PathType Container)) { + continue + } + + foreach ($frameworkDirectory in Get-ChildItem -LiteralPath $folderPath -Directory) { + # Not recursive: satellite resource assemblies sit in culture subdirectories + # and carry no documentation of their own. + foreach ($assembly in Get-ChildItem -LiteralPath $frameworkDirectory.FullName -Filter '*.dll' -File) { + $assemblyName = [System.IO.Path]::GetFileNameWithoutExtension($assembly.Name) + if (-not $documentedAssemblies.Contains($assemblyName)) { + continue + } + + $expectedXml = [System.IO.Path]::ChangeExtension($assembly.FullName, '.xml') + if (-not (Test-Path -LiteralPath $expectedXml -PathType Leaf)) { + Add-Finding -Category 'missing-documentation' -Path $assembly.FullName -Message ( + "$packageName ships $folder/$($frameworkDirectory.Name)/$($assembly.Name) " + + "without $assemblyName.xml, although its project generates XML " + + 'documentation. Consumers of this package get no IntelliSense text.') + } + } + } + } + } + } + + foreach ($packageRoot in $script:ExpandedPackageRoots.Keys) { + $packageName = $script:ExpandedPackageRoots[$packageRoot] + $rootFull = (Resolve-Path -LiteralPath $packageRoot).Path + + # Index the documentation this package contains by folder kind, target framework and file + # name, so lib/net8.0/X.xml can be matched with ref/net8.0/X.xml. + $byKind = @{ 'lib' = @{}; 'ref' = @{} } + foreach ($path in $documentationByPath.Keys) { + if (-not $path.StartsWith($rootFull, [System.StringComparison]::OrdinalIgnoreCase)) { + continue + } + + $relative = $path.Substring($rootFull.Length).TrimStart([char]'/', [char]'\') + $segments = $relative -split '[/\\]' + if ($segments.Count -lt 3) { + continue + } + + $kind = $segments[0].ToLowerInvariant() + if (-not $byKind.ContainsKey($kind)) { + continue + } + + $byKind[$kind]["$($segments[1])/$($segments[-1])"] = $path + } + + foreach ($key in ($byKind['lib'].Keys | Sort-Object)) { + if (-not $byKind['ref'].ContainsKey($key)) { + continue + } + + $libPath = $byKind['lib'][$key] + $refPath = $byKind['ref'][$key] + $lib = $documentationByPath[$libPath] + $ref = $documentationByPath[$refPath] + + $libHasNarrative = ($lib.RemarksCount + $lib.ExampleCount) -gt 0 + $refHasNarrative = ($ref.RemarksCount + $ref.ExampleCount) -gt 0 + + if (-not $libHasNarrative) { + Add-Finding -Category 'lib-documentation-trimmed' -Path $libPath -Message ( + "$packageName lib/$key contains no or elements, so " + + 'IntelliSense would show summaries only. The lib/ documentation must be the ' + + 'full implementation XML; only ref/ is trimmed.') + } + + if ($refHasNarrative) { + Add-Finding -Category 'ref-documentation-untrimmed' -Path $refPath -Message ( + "$packageName ref/$key still contains $($ref.RemarksCount) and " + + "$($ref.ExampleCount) elements. It must be trimmed by " + + 'tools/intellisense/TrimDocs.ps1.') + } + + # Identical content means a single artifact was used for both targets, which defeats the + # purpose of shipping two files. Reported separately so the finding names the cause + # rather than only its symptom. + $libHash = (Get-FileHash -LiteralPath $libPath -Algorithm SHA256).Hash + $refHash = (Get-FileHash -LiteralPath $refPath -Algorithm SHA256).Hash + if ($libHash -eq $refHash) { + Add-Finding -Category 'lib-ref-documentation-identical' -Path $libPath -Message ( + "$packageName lib/$key and ref/$key are byte-identical. The nuspec must map " + + 'the implementation XML to lib/ and the trimmed reference XML to ref/.') + } + } + } +} + +# An exception that no longer matches anything must be removed, otherwise a reintroduced defect +# would be silently suppressed by an obsolete entry. +foreach ($cref in ($ignoredCrefs | Sort-Object)) { + if (-not $observedIgnoredCrefs.Contains($cref)) { + Add-Finding -Category 'stale-allowlist-entry' -Path $AllowlistPath -Cref $cref -Message ( + "Allowlisted cref '$cref' no longer appears in any validated file. Remove the stale " + + 'entry from the allowlist.') + } +} + +# Report ------------------------------------------------------------------------------------- + +$findings = @($script:Findings | Sort-Object Path, LineNumber, Category, Cref) + +$countsByCategory = [ordered]@{} +foreach ($category in $script:CategorySeverities.Keys) { + $countsByCategory[$category] = @($findings | Where-Object { $_.Category -eq $category }).Count +} + +# The report is written before gating so it survives a failing run. +if (-not [string]::IsNullOrWhiteSpace($ReportPath)) { + $reportDirectory = Split-Path -Parent $ReportPath + if (-not [string]::IsNullOrWhiteSpace($reportDirectory) -and -not (Test-Path -LiteralPath $reportDirectory)) { + New-Item -ItemType Directory -Path $reportDirectory -Force | Out-Null + } + + [pscustomobject]@{ + FilesValidated = $documents.Count + SnippetFiles = @($documents | Where-Object { $_.Kind -eq 'snippet' }).Count + DocumentationFiles = $documentationDocuments.Count + CrefsValidated = $crefCount + LocalUids = if ($null -ne $script:LocalUids) { $script:LocalUids.Count } else { 0 } + CountsByCategory = $countsByCategory + Findings = $findings + } | ConvertTo-Json -Depth 6 | Set-Content -LiteralPath $ReportPath -Encoding utf8 + + Write-Host "XML documentation validation report written to '$ReportPath'." +} + +# Gate --------------------------------------------------------------------------------------- + +$gateTokens = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::OrdinalIgnoreCase) +foreach ($token in @($FailOn)) { + foreach ($part in ([string]$token).Split(',')) { + $trimmed = $part.Trim() + if (-not [string]::IsNullOrEmpty($trimmed)) { + [void]$gateTokens.Add($trimmed) + } + } +} + +foreach ($token in $gateTokens) { + if (-not $script:CategorySeverities.Contains($token) -and + $token -notin @('error', 'warning', 'info')) { + $categories = ($script:CategorySeverities.Keys | Sort-Object) -join ', ' + throw "Unknown -FailOn token '$token'. Expected a severity (error, warning, info) or a category ($categories)." + } +} + +$gatingFindings = @($findings | Where-Object { + $gateTokens.Contains($_.Category) -or $gateTokens.Contains($_.Severity) + }) + +foreach ($finding in $findings) { + $isGating = (-not $ReportOnly) -and + ($gateTokens.Contains($finding.Category) -or $gateTokens.Contains($finding.Severity)) + $issueType = if ($isGating) { 'error' } else { 'warning' } + + $location = if ([string]::IsNullOrWhiteSpace($finding.Path)) { + '' + } + else { + ";sourcepath=$($finding.Path);linenumber=$($finding.LineNumber);columnnumber=1" + } + + $detail = if ([string]::IsNullOrWhiteSpace($finding.Member)) { + $finding.Message + } + else { + "$($finding.Message) (in $($finding.Member))" + } + + Write-Host "##vso[task.logissue type=$issueType$location]$($finding.Category): $detail" +} + +$summary = "XML documentation validation examined $crefCount cref(s) across $($documents.Count) file(s)." +foreach ($category in $countsByCategory.Keys) { + if ($countsByCategory[$category] -gt 0) { + $summary += " $category=$($countsByCategory[$category]);" + } +} +Write-Host $summary + +# task.logissue attaches an issue to the timeline record but leaves the task result untouched, so +# a step reporting only warnings would still render as a clean success. Setting the result marks it +# as succeeded-with-issues, which is what makes the warnings visible without failing the build. +function Set-SucceededWithIssues { + Write-Host '##vso[task.complete result=SucceededWithIssues;]' +} + +if ($ReportOnly) { + if ($findings.Count -gt 0) { + Write-Host "##vso[task.logissue type=warning]XML documentation validation found $($findings.Count) issue(s) but is running in report-only mode, so the build is not failed. $($gatingFindings.Count) of them would fail a gating run." + Set-SucceededWithIssues + } + return +} + +if ($gatingFindings.Count -gt 0) { + $noun = if ($gatingFindings.Count -eq 1) { 'issue' } else { 'issues' } + throw "XML documentation validation failed with $($gatingFindings.Count) $noun. Review the preceding errors." +} + +if ($findings.Count -gt 0) { + # Nothing here fails the build, but non-gating findings were still reported. + Set-SucceededWithIssues +} + +Write-Host 'XML documentation validation passed.' diff --git a/eng/pipelines/onebranch/sqlclient-non-official.yml b/eng/pipelines/onebranch/sqlclient-non-official.yml index 88231ed15b..4cd5feae6b 100644 --- a/eng/pipelines/onebranch/sqlclient-non-official.yml +++ b/eng/pipelines/onebranch/sqlclient-non-official.yml @@ -17,8 +17,8 @@ parameters: # When true, any SDL errors will break the build. When false, SDL errors will be logged but # will not break the build. TSA bug filing is always disabled in the non-official pipeline # (see the tsa block in globalSdl). - - name: breakOnSdlError - displayName: Break on SDL error + - name: failOnSdlError + displayName: Fail on SDL errors type: boolean default: true @@ -28,6 +28,19 @@ parameters: type: boolean default: false + # When true, validation findings fail the build. When false, they are reported as warnings and + # the build continues. + # + # Validation covers XML documentation cross-references, localized resources, and the produced + # NuGet packages. Those gates run at several points across the build, so a hard failure at the + # first one prevents the rest from running. Set this false to let a single run reach every gate + # and report all findings at once, which is the practical way to assess a newly added gate or a + # backlog of known findings. + - name: failOnValidationError + displayName: Fail on validation errors + type: boolean + default: true + # True to publish symbols to private and public servers. - name: publishSymbols displayName: Publish symbols @@ -161,7 +174,7 @@ extends: # # https://eng.ms/docs/products/onebranch/securitycompliancegovernanceandpolicies/sdlforcontainerizedworkflows/customizesdlforcontainerbuilds - # Snapshot of the SDL analyzer findings that pre-existed the breakOnSdlError rollout, so + # Snapshot of the SDL analyzer findings that pre-existed the failOnSdlError rollout, so # builds only break on NEW findings. Generated from the SDL analysis artifacts of a full # non-official run and kept under .config/ alongside the other SDL tool configs # (CredScan/PoliCheck/TSA). @@ -180,8 +193,8 @@ extends: # variable. enabled: true - # APIScan errors break the build when breakOnSdlError is set. - break: ${{ parameters.breakOnSdlError }} + # APIScan errors break the build when failOnSdlError is set. + break: ${{ parameters.failOnSdlError }} # Use pre-release mode for non-official pipelines. modeType: prerelease @@ -196,7 +209,7 @@ extends: armory: enabled: true - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} asyncSdl: # Disabling this as it complicates the build process with minimal gain @@ -204,7 +217,7 @@ extends: binskim: enabled: true - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} codeql: # CodeQL 3000 is configured under the `compiled` key; `enabled` is a sub-property of @@ -221,17 +234,17 @@ extends: # Only useful for repos with ECMAscript - which we do not have. enabled: false # Break value is wired preemptively so it takes effect if eslint is ever enabled. - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} policheck: enabled: true - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} exclusionsFile: '$(REPO_ROOT)\.config\PolicheckExclusions.xml' psscriptanalyzer: # Static analysis of the repository's PowerShell scripts (e.g. under eng/). enabled: true - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} roslyn: # Enabling Roslyn SDL analysis here requires that our .NET builds _produce_ Roslyn findings. @@ -242,7 +255,7 @@ extends: # may log processing errors (for example, Post Analysis's SDL artifact report) even though # Roslyn collection and Guardian policy ingestion succeed. enabled: true - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} publishLogs: enabled: true @@ -265,20 +278,20 @@ extends: # SDL scans and iterate on fixes before they land. We do not want it filing (or # updating) bug work items -- that would create churn and duplicate the bugs owned by # the official pipeline, where TSA is hardcoded enabled. Here, findings surface - # directly in the build instead, gated by breakOnSdlError. + # directly in the build instead, gated by failOnSdlError. # # Interaction with other SDL tool defaults: # OneBranch derives each SDL tool's DEFAULT `break` value from whether TSA is enabled -- # per the OneBranch "Customize SDL" docs, most tools default to `break: false` when TSA # is enabled and `break: true` when it is disabled. We do NOT rely on that implicit - # coupling: every tool above sets `break: ${{ parameters.breakOnSdlError }}` explicitly, + # coupling: every tool above sets `break: ${{ parameters.failOnSdlError }}` explicitly, # which overrides the TSA-derived default. We have hit bugs in OneBranch's implicit # break-altering logic in the past, so both `tsa.enabled` and each tool's `break` are # set explicitly to keep the behaviour deterministic. # - # Otherwise independent of breakOnSdlError: - # Aside from that default-flipping, TSA and breakOnSdlError are orthogonal. Disabling - # TSA here does not by itself force breaking -- breakOnSdlError is what controls whether + # Otherwise independent of failOnSdlError: + # Aside from that default-flipping, TSA and failOnSdlError are orthogonal. Disabling + # TSA here does not by itself force breaking -- failOnSdlError is what controls whether # findings fail the build. enabled: false # Keep this in sync with Official even though TSA is disabled here. @@ -300,6 +313,7 @@ extends: parameters: isOfficial: false # This is a non-official pipeline. buildSqlServer: ${{ parameters.buildSqlServer }} + failOnValidationError: ${{ parameters.failOnValidationError }} abstractionsArtifactsName: '${{ variables.abstractionsArtifactsName }}' akvProviderArtifactsName: '${{ variables.akvProviderArtifactsName }}' diff --git a/eng/pipelines/onebranch/sqlclient-official.yml b/eng/pipelines/onebranch/sqlclient-official.yml index b8521fc0d7..9b13d21337 100644 --- a/eng/pipelines/onebranch/sqlclient-official.yml +++ b/eng/pipelines/onebranch/sqlclient-official.yml @@ -26,8 +26,8 @@ parameters: # When true, any SDL errors will break the build. When false, SDL errors will be logged but # will not break the build. TSA bug filing is always enabled in the official pipeline and is # independent of this parameter (see the tsa block in globalSdl). - - name: breakOnSdlError - displayName: Break on SDL error + - name: failOnSdlError + displayName: Fail on SDL errors type: boolean default: true @@ -176,7 +176,7 @@ extends: # # https://eng.ms/docs/products/onebranch/securitycompliancegovernanceandpolicies/sdlforcontainerizedworkflows/customizesdlforcontainerbuilds - # Snapshot of the SDL analyzer findings that pre-existed the breakOnSdlError rollout, so + # Snapshot of the SDL analyzer findings that pre-existed the failOnSdlError rollout, so # builds only break on NEW findings. Generated from the SDL analysis artifacts of a full # non-official run and kept under .config/ alongside the other SDL tool configs # (CredScan/PoliCheck/TSA). @@ -195,8 +195,8 @@ extends: # variable. enabled: true - # APIScan errors break the build when breakOnSdlError is set. - break: ${{ parameters.breakOnSdlError }} + # APIScan errors break the build when failOnSdlError is set. + break: ${{ parameters.failOnSdlError }} # Use release mode for official pipelines. modeType: release @@ -211,7 +211,7 @@ extends: armory: enabled: true - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} asyncSdl: # Disabling this as it complicates the build process with minimal gain @@ -219,7 +219,7 @@ extends: binskim: enabled: true - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} codeql: # CodeQL 3000 is configured under the `compiled` key; `enabled` is a sub-property of @@ -236,17 +236,17 @@ extends: # Only useful for repos with ECMAscript - which we do not have. enabled: false # Break value is wired preemptively so it takes effect if eslint is ever enabled. - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} policheck: enabled: true - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} exclusionsFile: '$(REPO_ROOT)\.config\PolicheckExclusions.xml' psscriptanalyzer: # Static analysis of the repository's PowerShell scripts (e.g. under eng/). enabled: true - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} roslyn: # Enabling Roslyn SDL analysis here requires that our .NET builds _produce_ Roslyn findings. @@ -257,7 +257,7 @@ extends: # may log processing errors (for example, Post Analysis's SDL artifact report) even though # Roslyn collection and Guardian policy ingestion succeed. enabled: true - break: ${{ parameters.breakOnSdlError }} + break: ${{ parameters.failOnSdlError }} publishLogs: enabled: true @@ -280,15 +280,15 @@ extends: # OneBranch derives each SDL tool's DEFAULT `break` value from whether TSA is enabled -- # per the OneBranch "Customize SDL" docs, most tools default to `break: false` when TSA # is enabled and `break: true` when it is disabled. We do NOT rely on that implicit - # coupling: every tool above sets `break: ${{ parameters.breakOnSdlError }}` explicitly, + # coupling: every tool above sets `break: ${{ parameters.failOnSdlError }}` explicitly, # which overrides the TSA-derived default. We have hit bugs in OneBranch's implicit # break-altering logic in the past, so both `tsa.enabled` and each tool's `break` are # set explicitly to keep the behaviour deterministic. # - # Otherwise independent of breakOnSdlError: - # Aside from that default-flipping, TSA and breakOnSdlError are orthogonal. TSA files + # Otherwise independent of failOnSdlError: + # Aside from that default-flipping, TSA and failOnSdlError are orthogonal. TSA files # bugs from the complete finding set (the TSAUpload step runs after all analyzers have - # produced results), while breakOnSdlError separately controls whether those same + # produced results), while failOnSdlError separately controls whether those same # findings also fail the build. Enabling TSA here does not suppress breaking, and # breaking does not prevent bug filing within a job. enabled: true @@ -310,6 +310,9 @@ extends: parameters: isOfficial: true # This is an official pipeline. buildSqlServer: ${{ parameters.buildSqlServer }} + # Validation findings always fail an official build. Set here rather than offered as a + # queue-time parameter, so a run cannot be started with validation downgraded. + failOnValidationError: true abstractionsArtifactsName: '${{ variables.abstractionsArtifactsName }}' akvProviderArtifactsName: '${{ variables.akvProviderArtifactsName }}' diff --git a/eng/pipelines/onebranch/stages/build-stages.yml b/eng/pipelines/onebranch/stages/build-stages.yml index 34f88bd9b3..78971cc1e7 100644 --- a/eng/pipelines/onebranch/stages/build-stages.yml +++ b/eng/pipelines/onebranch/stages/build-stages.yml @@ -22,6 +22,17 @@ parameters: - name: isOfficial type: boolean + # When true, validation findings fail the build. When false, they are reported as warnings and + # the build continues. + # + # The stages defined here depend on one another, so a hard failure in an early validation gate + # skips every later stage along with its own gates. Setting this false lets one run reach every + # gate and surface all findings at once. + # + # No default: the invoking pipeline states its intent. + - name: failOnValidationError + type: boolean + # ── Build parameters ─────────────────────────────────────────────────── # Whether to build Microsoft.SqlServer.Server. The SqlClient family is always built; SqlServer @@ -101,6 +112,8 @@ stages: apiScanPdbPath: '$(REPO_ROOT)/apiScan/Microsoft.Data.SqlClient.Internal.Logging/pdbs' apiScanSoftwareVersion: '$(sqlClientApiScanVersion)' shouldSignPackage: ${{ parameters.isOfficial }} + failOnValidationError: ${{ parameters.failOnValidationError }} + projectPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient.Internal/Logging/src/Logging.csproj' signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' @@ -124,6 +137,8 @@ stages: apiScanPdbPath: '$(REPO_ROOT)/apiScan/Microsoft.SqlServer.Server/pdbs' apiScanSoftwareVersion: '$(sqlServerApiScanVersion)' shouldSignPackage: ${{ parameters.isOfficial }} + failOnValidationError: ${{ parameters.failOnValidationError }} + projectPath: '$(REPO_ROOT)/src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj' signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' @@ -165,6 +180,8 @@ stages: apiScanPdbPath: '$(REPO_ROOT)/apiScan/Microsoft.Data.SqlClient.Extensions.Abstractions/pdbs' apiScanSoftwareVersion: '$(sqlClientApiScanVersion)' shouldSignPackage: ${{ parameters.isOfficial }} + failOnValidationError: ${{ parameters.failOnValidationError }} + projectPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient.Extensions/Abstractions/src/Abstractions.csproj' signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' @@ -211,6 +228,8 @@ stages: apiScanPdbPath: '$(REPO_ROOT)/apiScan/Microsoft.Data.SqlClient/pdbs' apiScanSoftwareVersion: '$(sqlClientApiScanVersion)' shouldSignPackage: ${{ parameters.isOfficial }} + failOnValidationError: ${{ parameters.failOnValidationError }} + projectPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj' signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' @@ -249,6 +268,8 @@ stages: apiScanPdbPath: '$(REPO_ROOT)/apiScan/Microsoft.Data.SqlClient.Extensions.Azure/pdbs' apiScanSoftwareVersion: '$(sqlClientApiScanVersion)' shouldSignPackage: ${{ parameters.isOfficial }} + failOnValidationError: ${{ parameters.failOnValidationError }} + projectPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient.Extensions/Azure/src/Azure.csproj' signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' @@ -296,6 +317,8 @@ stages: apiScanPdbPath: '$(REPO_ROOT)/apiScan/Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider/pdbs' apiScanSoftwareVersion: '$(sqlClientApiScanVersion)' shouldSignPackage: ${{ parameters.isOfficial }} + failOnValidationError: ${{ parameters.failOnValidationError }} + projectPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider/src/Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider.csproj' signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' @@ -372,3 +395,4 @@ stages: buildSqlServer: ${{ parameters.buildSqlServer }} isOfficial: ${{ parameters.isOfficial }} + failOnValidationError: ${{ parameters.failOnValidationError }} diff --git a/eng/pipelines/onebranch/steps/validate-localization-step.yml b/eng/pipelines/onebranch/steps/validate-localization-step.yml index 8cb590f6d0..7036eaff55 100644 --- a/eng/pipelines/onebranch/steps/validate-localization-step.yml +++ b/eng/pipelines/onebranch/steps/validate-localization-step.yml @@ -4,13 +4,41 @@ # See the LICENSE file in the project root for more information. # ################################################################################# +parameters: + # When true, localization findings fail this step. When false, they are reported as warnings and + # the step succeeds. + - name: failOnValidationError + type: boolean + steps: + # Compose the argument line first so the optional switch can be appended without repeating the + # whole list. The variable is set immediately before the task that reads it. + # + # The switch is appended as a bare token because the PowerShell task dot-sources the script under + # -Command, where a -Switch:Value form binds the value as a string and is rejected. + # + # Report-only is enabled only by the value False (the comparison is case-insensitive). Any value + # other than True or False is rejected rather than assumed, so an unrecognised value cannot + # silently downgrade validation to warnings. + - pwsh: | + $arguments = '-ResourcesDirectory "$(Build.SourcesDirectory)/src/Microsoft.Data.SqlClient/src/Resources"' + + ' -AllowlistPath "$(Build.SourcesDirectory)/.config/LocalizationValidationAllowlist.json"' + + $failOnValidationError = '${{ parameters.failOnValidationError }}' + if ($failOnValidationError -eq 'False') { + $arguments += ' -ReportOnly' + } + elseif ($failOnValidationError -ne 'True') { + throw "Unexpected failOnValidationError value '$failOnValidationError'." + } + + Write-Host "##vso[task.setvariable variable=validateLocalizationArguments]$arguments" + displayName: 'Compose localization validation arguments' + - task: PowerShell@2 displayName: 'Validate localized resources' inputs: targetType: filePath pwsh: true filePath: $(Build.SourcesDirectory)/eng/pipelines/onebranch/scripts/validate-localization.ps1 - arguments: >- - -ResourcesDirectory "$(Build.SourcesDirectory)/src/Microsoft.Data.SqlClient/src/Resources" - -AllowlistPath "$(Build.SourcesDirectory)/.config/LocalizationValidationAllowlist.json" + arguments: $(validateLocalizationArguments) diff --git a/eng/pipelines/onebranch/steps/validate-packages-step.yml b/eng/pipelines/onebranch/steps/validate-packages-step.yml index 173df4d7ed..e85ec2e118 100644 --- a/eng/pipelines/onebranch/steps/validate-packages-step.yml +++ b/eng/pipelines/onebranch/steps/validate-packages-step.yml @@ -53,6 +53,11 @@ parameters: default: - error + # When true, gate findings fail this step. When false, they are reported as warnings and the + # step succeeds. + - name: failOnValidationError + type: boolean + steps: - task: DotNetCoreCLI@2 displayName: 'build.proj - BuildPackageValidator' @@ -63,18 +68,40 @@ steps: -t:BuildPackageValidator -p:Configuration=Release + # Compose the argument line first so the optional switch can be appended without repeating the + # whole list. The variable is set immediately before the task that reads it. + # + # The switch is appended as a bare token because the PowerShell task dot-sources the script under + # -Command, where a -Switch:Value form binds the value as a string and is rejected. + # + # Report-only is enabled only by the value False (the comparison is case-insensitive). Any value + # other than True or False is rejected rather than assumed, so an unrecognised value cannot + # silently downgrade validation to warnings. + - pwsh: | + $arguments = '-ValidatorPath "$(REPO_ROOT)/tools/PackageValidator/src/bin/Release/net10.0/PackageValidator.dll"' + + ' -PackagesPath "${{ parameters.packagesPath }}"' + + ' -ReportPath "${{ parameters.reportPath }}"' + + ' -SqlClientPackageVersion "${{ parameters.sqlClientPackageVersion }}"' + + ' -SqlClientFileVersion "${{ parameters.sqlClientFileVersion }}"' + + ' -SqlServerPackageVersion "${{ parameters.sqlServerPackageVersion }}"' + + ' -SqlServerFileVersion "${{ parameters.sqlServerFileVersion }}"' + + ' -FailOn "${{ join(',', parameters.failOn) }}"' + + $failOnValidationError = '${{ parameters.failOnValidationError }}' + if ($failOnValidationError -eq 'False') { + $arguments += ' -ReportOnly' + } + elseif ($failOnValidationError -ne 'True') { + throw "Unexpected failOnValidationError value '$failOnValidationError'." + } + + Write-Host "##vso[task.setvariable variable=validatePackagesArguments]$arguments" + displayName: 'Compose package validation arguments' + - task: PowerShell@2 displayName: 'Validate NuGet packages' inputs: targetType: filePath pwsh: true filePath: $(REPO_ROOT)/eng/pipelines/onebranch/scripts/validate-packages.ps1 - arguments: >- - -ValidatorPath "$(REPO_ROOT)/tools/PackageValidator/src/bin/Release/net10.0/PackageValidator.dll" - -PackagesPath "${{ parameters.packagesPath }}" - -ReportPath "${{ parameters.reportPath }}" - -SqlClientPackageVersion "${{ parameters.sqlClientPackageVersion }}" - -SqlClientFileVersion "${{ parameters.sqlClientFileVersion }}" - -SqlServerPackageVersion "${{ parameters.sqlServerPackageVersion }}" - -SqlServerFileVersion "${{ parameters.sqlServerFileVersion }}" - -FailOn "${{ join(',', parameters.failOn) }}" + arguments: $(validatePackagesArguments) diff --git a/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml b/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml new file mode 100644 index 0000000000..d593c44763 --- /dev/null +++ b/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml @@ -0,0 +1,123 @@ +################################################################################# +# Licensed to the .NET Foundation under one or more agreements. # +# The .NET Foundation licenses this file to you under the MIT license. # +# See the LICENSE file in the project root for more information. # +################################################################################# + +# Runs eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 over XML documentation cross-references. +# +# Open Publishing resolves every cref in our XML documentation against the Learn xref map when the +# API docs are ingested into dotnet/sqlclient-api-docs. A malformed documentation ID cannot +# resolve, and the resulting xref-not-found warnings only appear after an API Docs pull request has +# been opened -- a slow round trip through another repository. This step moves that feedback into +# our own build. +# +# Three kinds of input may be validated, in any combination, so one template can serve more than +# one gate. Each is selected by a non-empty value and skipped by an empty one: +# +# snippetsDirectory documentation snippet sources +# documentationPath generated XML documentation +# packagesPath XML documentation inside .nupkg files +# +# Validation requires no network access: the defects it detects are documentation-ID syntax errors +# and namespace typos, which are decidable without consulting the Learn xref service. The +# published Learn .NET xref map is roughly 338 MB, so avoiding it also keeps the step fast. + +parameters: + # Step name shown in the pipeline log. Several gates use this template in one run, so each + # supplies a name describing what it covers. + - name: displayName + type: string + + # Directory of documentation snippet sources, scanned recursively. Pass '' to skip source mode. + - name: snippetsDirectory + type: string + + # Generated XML documentation file or directory, scanned recursively. Non-documentation XML in a + # scanned directory is ignored, so a whole build output tree may be passed. Pass '' to skip. + - name: documentationPath + type: string + + # Directory scanned recursively for .nupkg files whose XML documentation is validated. Requires + # extractPath. Pass '' to skip package mode. + - name: packagesPath + type: string + + # Directory that packagesPath expands into. Pass '' when packagesPath is ''. + - name: extractPath + type: string + + # Path of the JSON report to write. Written before gating, so it survives a failing run. + - name: reportPath + type: string + + # Finding severities and/or categories that fail the build. + # + # The error severity covers malformed-xml, unresolved-cref, invalid-docid, unknown-namespace-root, + # stale-allowlist-entry, and the three package layout categories (lib-documentation-trimmed, + # ref-documentation-untrimmed, lib-ref-documentation-identical). + # + # It deliberately does not cover the two resolution categories, which are warnings: + # missing-local-uid fires when a cref names this repository but no matching member was emitted, + # which a target-framework-conditional or cross-assembly member can trigger legitimately; and + # mismatched-docid-prefix depends on the same index. Name either explicitly once a run has + # shown it to be clean. + - name: failOn + type: object + + # When true, findings fail this step. When false, they are reported as warnings and the step + # succeeds. + - name: failOnValidationError + type: boolean + + # Project file whose GenerateDocumentationFile setting determines whether XML documentation must + # exist, and whose sources determine which documentation snippets are validated. Pass '' when + # the inputs span more than one project. + - name: projectPath + type: string + + # Directory scanned for project files, used with packagesPath to check that every assembly whose + # project generates XML documentation ships it. Pass '' when not validating packages. + - name: projectSearchRoot + type: string + +steps: + # Compose the argument line first so the optional switch can be appended without repeating the + # whole list. The variable is set immediately before the task that reads it, so repeated uses of + # this template within one job each read their own value. + # + # Unused inputs are passed empty rather than omitted, keeping one argument list for every mode. + # The switch is appended as a bare token because the PowerShell task dot-sources the script under + # -Command, where a -Switch:Value form binds the value as a string and is rejected. + # + # Report-only is enabled only by the value False (the comparison is case-insensitive). Any value + # other than True or False is rejected rather than assumed, so an unrecognised value cannot + # silently downgrade validation to warnings. + - pwsh: | + $arguments = '-ReportPath "${{ parameters.reportPath }}"' + + ' -FailOn "${{ join(',', parameters.failOn) }}"' + + ' -SnippetsDirectory "${{ parameters.snippetsDirectory }}"' + + ' -DocumentationPath "${{ parameters.documentationPath }}"' + + ' -PackagesPath "${{ parameters.packagesPath }}"' + + ' -ExtractPath "${{ parameters.extractPath }}"' + + ' -ProjectPath "${{ parameters.projectPath }}"' + + ' -ProjectSearchRoot "${{ parameters.projectSearchRoot }}"' + + $failOnValidationError = '${{ parameters.failOnValidationError }}' + if ($failOnValidationError -eq 'False') { + $arguments += ' -ReportOnly' + } + elseif ($failOnValidationError -ne 'True') { + throw "Unexpected failOnValidationError value '$failOnValidationError'." + } + + Write-Host "##vso[task.setvariable variable=validateXmlDocsArguments]$arguments" + displayName: 'Compose arguments - ${{ parameters.displayName }}' + + - task: PowerShell@2 + displayName: '${{ parameters.displayName }}' + inputs: + targetType: filePath + pwsh: true + filePath: $(REPO_ROOT)/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 + arguments: $(validateXmlDocsArguments) diff --git a/src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj b/src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj index a995fa13b9..f3a8da6200 100644 --- a/src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj +++ b/src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj @@ -5,6 +5,20 @@ net46;netstandard2.0 + + + + true + + @@ -104,13 +104,13 @@ } - The event returns this output: + The event returns this output: Event Arguments: (command=Microsoft.Data.SqlClient.SqlCommand commandType=2 status=0) - The event returns this output: + The event returns this output: Event Arguments: (command=Microsoft.Data.SqlClient.SqlCommand commandType=2 recordsAffected=1 row=System.Data.DataRow[37] status=0) diff --git a/doc/snippets/Microsoft.Data.SqlClient/SqlRowUpdatingEventArgs.xml b/doc/snippets/Microsoft.Data.SqlClient/SqlRowUpdatingEventArgs.xml index ee545db905..b8bd2ce87c 100644 --- a/doc/snippets/Microsoft.Data.SqlClient/SqlRowUpdatingEventArgs.xml +++ b/doc/snippets/Microsoft.Data.SqlClient/SqlRowUpdatingEventArgs.xml @@ -102,13 +102,13 @@ } - The event returns this output: + The event returns this output: event args: (command=Microsoft.Data.SqlClient.SQLCommand commandType=2 status=0) - The event returns this output: + The event returns this output: event args: (command=Microsoft.Data.SqlClient.SQLCommand commandType=2 recordsAffected=1 row=System.Data.DataRow[37] status=0) diff --git a/doc/snippets/Microsoft.SqlServer.Server/SqlFunctionAttribute.xml b/doc/snippets/Microsoft.SqlServer.Server/SqlFunctionAttribute.xml index 00900d7b9d..47b6a57154 100644 --- a/doc/snippets/Microsoft.SqlServer.Server/SqlFunctionAttribute.xml +++ b/doc/snippets/Microsoft.SqlServer.Server/SqlFunctionAttribute.xml @@ -176,7 +176,7 @@ Indicates whether the function requires access to data stored in the system catalogs or virtual system tables of SQL Server. - : Does not access system data. : Only reads system data. + : Does not access system data. : Only reads system data. The default is . diff --git a/doc/snippets/Microsoft.SqlServer.Server/SqlUserDefinedTypeAttribute.xml b/doc/snippets/Microsoft.SqlServer.Server/SqlUserDefinedTypeAttribute.xml index be808f7878..bd2728e0a1 100644 --- a/doc/snippets/Microsoft.SqlServer.Server/SqlUserDefinedTypeAttribute.xml +++ b/doc/snippets/Microsoft.SqlServer.Server/SqlUserDefinedTypeAttribute.xml @@ -44,7 +44,7 @@ A required attribute on a user-defined type (UDT), used to confirm that the given type is a UDT and to indicate the storage format of the UDT. - The following example specifies that the of the user-defined type is and the is 8000 bytes. + The following example specifies that the of the user-defined type is and the is 8000 bytes. diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 0f3ea33cdd..fc11e96a98 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -1000,8 +1000,8 @@ if ($script:ExpandedPackageRoots.Count -gt 0) { if (-not $libHasNarrative) { Add-Finding -Category 'lib-documentation-trimmed' -Path $libPath -Message ( - "$packageName lib/$key contains no or elements, so " + - 'IntelliSense would show summaries only. The lib/ documentation must be the ' + + "$packageName lib/$key contains no or elements. The lib/ " + + 'documentation is the source the API docs pipeline consumes, so it must be the ' + 'full implementation XML; only ref/ is trimmed.') } diff --git a/src/Directory.Build.props b/src/Directory.Build.props index d9b9a7cdc8..6151f5a121 100644 --- a/src/Directory.Build.props +++ b/src/Directory.Build.props @@ -253,4 +253,17 @@ 14 + + + + + diff --git a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj index 43aca3f1c1..e2b14cb4a8 100644 --- a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj +++ b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj @@ -64,12 +64,31 @@ Text="The 'pwsh' dotnet local tool is not available. Run 'dotnet tool restore' from the repository root before building." /> - + + + dotnet tool run pwsh -- @@ -84,6 +103,11 @@ + + + diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.nuspec b/src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.nuspec index 9a0cf9e885..7222ea4673 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.nuspec +++ b/src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.nuspec @@ -136,6 +136,29 @@ + @@ -158,7 +181,7 @@ - + @@ -178,7 +201,7 @@ - + @@ -198,7 +221,10 @@ - + + @@ -221,7 +247,17 @@ - + From 969ca01112c8a13f098b442ccabda150bd53f674 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Wed, 23 Sep 2026 13:06:52 -0300 Subject: [PATCH 03/23] Split unresolved local references by public API surface The generated-documentation and packaged-documentation gates reported 187 missing-local-uid warnings, none of which were actionable. They marked three steps as succeeded-with-issues and buried a real defect. Two populations were mixed together. A reference to an internal type that carries no documentation of its own reaches no reader, because internal members are never published. A reference from a public member does reach a reader, and becomes an unresolved reference on the published page. A reference assembly contains the public API and nothing else, so the documentation in a package's ref/ folder is exactly that surface. Severity now follows the member holding the reference: from a public member it is an error, and otherwise it is informational. Where no reference documentation is available the surface is unknown and nothing is escalated, which is the case for the per-assembly gates. On the net8.0 package this separates 19 findings from 60. The separation exposed a namespace written with a type prefix in SqlDataReader.xml, which would have produced an unresolved reference on a published page. Corrected to N:. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit ed8aed69be156ebcbc3c170b519e57540c891eed) --- .../SqlDataReader.xml | 2 +- .../scripts/tests/validate-xml-docs.Tests.ps1 | 76 +++++++++++++++- .../onebranch/scripts/validate-xml-docs.ps1 | 86 +++++++++++++++++-- 3 files changed, 151 insertions(+), 13 deletions(-) diff --git a/doc/snippets/Microsoft.Data.SqlClient/SqlDataReader.xml b/doc/snippets/Microsoft.Data.SqlClient/SqlDataReader.xml index 10c67a4890..6c4c223df6 100644 --- a/doc/snippets/Microsoft.Data.SqlClient/SqlDataReader.xml +++ b/doc/snippets/Microsoft.Data.SqlClient/SqlDataReader.xml @@ -183,7 +183,7 @@ The following example creates a , a The value of the specified column. - Not supported for . + Not supported for . The specified cast is not valid. diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index 02b1737015..bca41b30d6 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -297,7 +297,9 @@ Describe 'validate-xml-docs.ps1' { @((Get-Report -Path $report).Findings) | Should -BeNullOrEmpty } - It 'reports a local cref with no matching emitted member as a warning, not an error' { + It 'reports a local cref with no matching emitted member as information, not an error' { + # Without reference documentation the public API surface is unknown, and such a + # reference may resolve in another target framework or a sibling assembly. $docs = New-DocumentationDirectory ` -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` -Crefs @('T:Microsoft.Data.SqlClient.DoesNotExist') @@ -307,7 +309,7 @@ Describe 'validate-xml-docs.ps1' { $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 $finding.Category | Should -Be 'missing-local-uid' - $finding.Severity | Should -Be 'warning' + $finding.Severity | Should -Be 'info' } It 'fails on a missing local UID when that category is named in -FailOn' { @@ -729,6 +731,74 @@ Describe 'validate-xml-docs.ps1' { $categories | Should -Not -Contain 'missing-documentation' } + It 'reports an unresolved cref from a public member as an error' { + # Severity follows the member holding the reference: a public member's page is + # published, so the unresolved reference becomes visible. + $staging = New-TestDirectory + New-Item -ItemType Directory -Path (Join-Path $staging 'lib/net8.0') -Force | Out-Null + New-Item -ItemType Directory -Path (Join-Path $staging 'ref/net8.0') -Force | Out-Null + + # The public member appears in ref/; the reference it makes resolves nowhere. + 'W.' + + '' | + Set-Content -LiteralPath (Join-Path $staging 'lib/net8.0/Microsoft.Data.SqlClient.xml') -Encoding utf8 + 'W.' | + Set-Content -LiteralPath (Join-Path $staging 'ref/net8.0/Microsoft.Data.SqlClient.xml') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory($staging, (Join-Path $packages 'P.1.0.0.nupkg')) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) ` + -ReportPath $report -ReportOnly + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Cref -eq 'T:Microsoft.Data.SqlClient.Missing' })[0] + $finding.Category | Should -Be 'missing-public-uid' + $finding.Severity | Should -Be 'error' + } + + It 'reports the same cref from a non-public member as information' { + $staging = New-TestDirectory + New-Item -ItemType Directory -Path (Join-Path $staging 'lib/net8.0') -Force | Out-Null + New-Item -ItemType Directory -Path (Join-Path $staging 'ref/net8.0') -Force | Out-Null + + # The holder is absent from ref/, so it is not part of the public API surface. + 'I.' + + '' | + Set-Content -LiteralPath (Join-Path $staging 'lib/net8.0/Microsoft.Data.SqlClient.xml') -Encoding utf8 + 'W.' | + Set-Content -LiteralPath (Join-Path $staging 'ref/net8.0/Microsoft.Data.SqlClient.xml') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory($staging, (Join-Path $packages 'P.1.0.0.nupkg')) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) ` + -ReportPath $report -ReportOnly + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Cref -eq 'T:Microsoft.Data.SqlClient.Missing' })[0] + $finding.Category | Should -Be 'missing-local-uid' + $finding.Severity | Should -Be 'info' + } + + It 'does not classify as public when no reference documentation identifies the surface' { + # Without a ref/ folder the public API surface is unknown, so nothing is escalated. + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('T:Microsoft.Data.SqlClient.DoesNotExist') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'missing-local-uid' + $finding.Severity | Should -Be 'info' + } + It 'does not apply layout rules outside package mode' { $docs = New-TestDirectory $libPath = Join-Path $docs 'lib/net8.0' @@ -809,7 +879,7 @@ Describe 'validate-xml-docs.ps1' { } It 'marks the task succeeded-with-issues for non-gating findings in a gating run' { - # missing-local-uid is a warning, so the run passes, but it must still be visible. + # missing-local-uid is informational, so the run passes, but it must still be visible. $docs = New-DocumentationDirectory ` -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` -Crefs @('T:Microsoft.Data.SqlClient.DoesNotExist') diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index fc11e96a98..67a4a666a2 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -124,11 +124,17 @@ lib-ref-documentation-identical error A package's lib/ and ref/ XML are byte-identical, so the nuspec mapped one artifact into both targets. - missing-local-uid warning Cref names this repository but no such member was emitted. - Warning rather than error because a member may legitimately - be absent from the documentation under validation: it may be - conditional on another target framework, or defined in a - sibling assembly that this run did not include. + missing-public-uid error Cref names this repository but no such member was emitted, + and it is referenced from a public API member, so the + published page carries an unresolved reference. Reported + only when reference documentation identifies which members + are public. + missing-local-uid info As above, but referenced from a member that is not public, + or from any member when the public API surface is unknown. + Not an error because such a member is never published, and + because a reference may legitimately resolve elsewhere: to + another target framework, or to a sibling assembly that this + run did not include. mismatched-docid-prefix warning Cref names a real member but with the wrong kind prefix, such as M: on a property or T: on a member. missing-external-uid warning Cref is absent from the supplied Learn xref map. @@ -195,7 +201,8 @@ $script:CategorySeverities = [ordered]@{ 'lib-documentation-trimmed' = 'error' 'ref-documentation-untrimmed' = 'error' 'lib-ref-documentation-identical' = 'error' - 'missing-local-uid' = 'warning' + 'missing-public-uid' = 'error' + 'missing-local-uid' = 'info' 'mismatched-docid-prefix' = 'warning' 'missing-documentation' = 'error' 'unexpected-documentation' = 'warning' @@ -458,9 +465,25 @@ function Test-Cref { "'$actual'. Use the prefix matching the member kind.") } else { - Add-Finding @Context -Category 'missing-local-uid' -Cref $Cref -Message ( - "Cref '$trimmed' names this repository but no matching documented member " + - 'was emitted by the build. The reference will not resolve.') + # Severity follows the member that holds the reference, not the reference + # itself: a public member's page is published, so a reference it cannot resolve + # becomes a visible xref-not-found. An internal member is never published, so + # the same reference reaches no reader. + $containingMemberIsPublic = $null -ne $script:PublicUids -and + -not [string]::IsNullOrEmpty($Context.Member) -and + $script:PublicUids.Contains($Context.Member) + + if ($containingMemberIsPublic) { + Add-Finding @Context -Category 'missing-public-uid' -Cref $Cref -Message ( + "Cref '$trimmed' names this repository but no matching documented " + + 'member was emitted by the build. It is referenced from a public API ' + + 'member, so the published page will carry an unresolved reference.') + } + else { + Add-Finding @Context -Category 'missing-local-uid' -Cref $Cref -Message ( + "Cref '$trimmed' names this repository but no matching documented " + + 'member was emitted by the build.') + } } } return @@ -559,6 +582,33 @@ function Get-DocumentedAssemblyName { return , $names } +<# + Reports whether a documentation file sits in the ref/ folder of an expanded package. + + Only a package has this structure, so this is how reference documentation is recognized without + the caller naming it. The path is compared against the expanded package roots rather than + searched for a "ref" segment anywhere, so an unrelated directory called ref cannot be mistaken + for one. +#> +function Test-IsReferenceDocumentationPath { + param([Parameter(Mandatory)][string]$Path) + + foreach ($packageRoot in $script:ExpandedPackageRoots.Keys) { + $rootFull = (Resolve-Path -LiteralPath $packageRoot).Path + if (-not $Path.StartsWith($rootFull, [System.StringComparison]::OrdinalIgnoreCase)) { + continue + } + + $relative = $Path.Substring($rootFull.Length).TrimStart([char]'/', [char]'\') + $segments = $relative -split '[/\\]' + if ($segments.Count -ge 2 -and $segments[0] -eq 'ref') { + return $true + } + } + + return $false +} + function Resolve-InputPaths { param( [string[]]$Paths, @@ -754,6 +804,10 @@ foreach ($entry in @( Path = $file Kind = $entry.Kind Document = $document + # Documentation shipped in a package's ref/ folder describes the reference + # assembly, whose members are exactly the public API surface. + IsReferenceDocumentation = ($entry.Kind -eq 'documentation') -and + (Test-IsReferenceDocumentationPath -Path $file) RemarksCount = @($document.Descendants('remarks')).Count ExampleCount = @($document.Descendants('example')).Count }) @@ -818,6 +872,12 @@ if ($documentationRequested -and $malformedByKind['documentation'] -eq 0) { $script:LocalUids = $null $script:LocalUidsByBody = [System.Collections.Generic.Dictionary[string, System.Collections.Generic.HashSet[string]]]::new( [System.StringComparer]::Ordinal) + +# Members that form the published API surface. A reference assembly contains the public API and +# nothing else, so its documentation is exactly that set. Left null when no reference documentation +# was supplied, which disables the public/internal distinction rather than guessing at it. +$script:PublicUids = $null + if ($documentationDocuments.Count -gt 0) { $script:LocalUids = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) foreach ($entry in $documentationDocuments) { @@ -830,6 +890,14 @@ if ($documentationDocuments.Count -gt 0) { $uid = $name.Value.Trim() [void]$script:LocalUids.Add($uid) + if ($entry.IsReferenceDocumentation) { + if ($null -eq $script:PublicUids) { + $script:PublicUids = [System.Collections.Generic.HashSet[string]]::new( + [System.StringComparer]::Ordinal) + } + [void]$script:PublicUids.Add($uid) + } + # Index the identifier without its prefix so a cref carrying the wrong prefix can be # told apart from one naming a member that was never emitted at all. if ($uid.Length -gt 2 -and $uid[1] -eq ':') { From 13689d5261c54c5761da2707785022588150025b Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Wed, 23 Sep 2026 15:04:38 -0300 Subject: [PATCH 04/23] Keep informational findings out of the step result Demoting missing-local-uid to informational changed its severity but not how it was reported: every non-gating finding was written as a warning, and any finding at all marked the step as succeeded-with-issues. A step reporting nothing but informational findings therefore still drew attention to itself. task.logissue has no informational level, so informational findings are now written as plain output and only warnings and errors mark the step. Also corrects a cref naming a type that does not exist. The attribute is Microsoft.SqlServer.Server.SqlUserDefinedTypeAttribute, which eight other snippets already reference correctly; the ninth named a namespace that contains no such type. It is referenced from the public SqlMetaData constructors, so the published page would have carried an unresolved reference. This was the only finding the packaged-documentation gate reported against the public API surface. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 4761b5afc27360c6f4b61bce40716417f02b2102) --- .../SqlMetaData.xml | 2 +- .../scripts/tests/validate-xml-docs.Tests.ps1 | 19 +++++++++++++--- .../onebranch/scripts/validate-xml-docs.ps1 | 22 ++++++++++++++----- 3 files changed, 34 insertions(+), 9 deletions(-) diff --git a/doc/snippets/Microsoft.Data.SqlClient.Server/SqlMetaData.xml b/doc/snippets/Microsoft.Data.SqlClient.Server/SqlMetaData.xml index 492e85e46b..c0de5d25c8 100644 --- a/doc/snippets/Microsoft.Data.SqlClient.Server/SqlMetaData.xml +++ b/doc/snippets/Microsoft.Data.SqlClient.Server/SqlMetaData.xml @@ -395,7 +395,7 @@ The is . - A that is not allowed was passed to the constructor as , or points to a type that does not have declared. + A that is not allowed was passed to the constructor as , or points to a type that does not have declared. diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index bca41b30d6..c82755626c 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -879,16 +879,29 @@ Describe 'validate-xml-docs.ps1' { } It 'marks the task succeeded-with-issues for non-gating findings in a gating run' { - # missing-local-uid is informational, so the run passes, but it must still be visible. + # mismatched-docid-prefix is a warning, so the run passes, but it must still be visible. $docs = New-DocumentationDirectory ` - -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` - -Crefs @('T:Microsoft.Data.SqlClient.DoesNotExist') + -Members @('P:Microsoft.Data.SqlClient.SqlCommand.CommandTimeout') ` + -Crefs @('M:Microsoft.Data.SqlClient.SqlCommand.CommandTimeout') $output = & $scriptPath -DocumentationPath $docs *>&1 | Out-String + $output | Should -Match 'mismatched-docid-prefix' $output | Should -Match '##vso\[task\.complete result=SucceededWithIssues;\]' } + It 'does not mark the task succeeded-with-issues for informational findings alone' { + # Informational findings describe things that are correct as they stand, so marking on + # them would leave every run permanently flagged. + $snippets = New-SnippetDirectory -Crefs @('SqlJson', 'string') + + $output = & $scriptPath -SnippetsDirectory $snippets *>&1 | Out-String + + $output | Should -Match 'unprefixed-cref' + $output | Should -Not -Match 'task\.complete' + $output | Should -Not -Match '##vso\[task\.logissue' + } + It 'does not mark the task succeeded-with-issues when there are no findings' { $snippets = New-SnippetDirectory -Crefs @('T:System.Byte') diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 67a4a666a2..3d1ad5ebcc 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -1157,10 +1157,14 @@ $gatingFindings = @($findings | Where-Object { $gateTokens.Contains($_.Category) -or $gateTokens.Contains($_.Severity) }) +# Findings that warrant drawing attention to the step. Informational findings are reported in the +# log and the JSON report but do not mark the step, because they describe things that are correct +# as they stand and would otherwise leave every run permanently marked. +$notableFindings = @($findings | Where-Object { $_.Severity -ne 'info' }) + foreach ($finding in $findings) { $isGating = (-not $ReportOnly) -and ($gateTokens.Contains($finding.Category) -or $gateTokens.Contains($finding.Severity)) - $issueType = if ($isGating) { 'error' } else { 'warning' } $location = if ([string]::IsNullOrWhiteSpace($finding.Path)) { '' @@ -1176,6 +1180,14 @@ foreach ($finding in $findings) { "$($finding.Message) (in $($finding.Member))" } + # task.logissue has no informational level, so an info finding is written as plain output + # rather than being promoted to a warning it does not deserve. + if ($finding.Severity -eq 'info' -and -not $isGating) { + Write-Host "$($finding.Category): $detail" + continue + } + + $issueType = if ($isGating) { 'error' } else { 'warning' } Write-Host "##vso[task.logissue type=$issueType$location]$($finding.Category): $detail" } @@ -1195,8 +1207,8 @@ function Set-SucceededWithIssues { } if ($ReportOnly) { - if ($findings.Count -gt 0) { - Write-Host "##vso[task.logissue type=warning]XML documentation validation found $($findings.Count) issue(s) but is running in report-only mode, so the build is not failed. $($gatingFindings.Count) of them would fail a gating run." + if ($notableFindings.Count -gt 0) { + Write-Host "##vso[task.logissue type=warning]XML documentation validation found $($notableFindings.Count) issue(s) but is running in report-only mode, so the build is not failed. $($gatingFindings.Count) of them would fail a gating run." Set-SucceededWithIssues } return @@ -1207,8 +1219,8 @@ if ($gatingFindings.Count -gt 0) { throw "XML documentation validation failed with $($gatingFindings.Count) $noun. Review the preceding errors." } -if ($findings.Count -gt 0) { - # Nothing here fails the build, but non-gating findings were still reported. +if ($notableFindings.Count -gt 0) { + # Nothing here fails the build, but findings above informational level were reported. Set-SucceededWithIssues } From ef48f5c24cd6b42fe84b3676b5077bde7e5e6493 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 08:10:27 -0300 Subject: [PATCH 05/23] Address PR review feedback Resolves the unresolved review threads on #4730, and the three items the reviewer recorded in the review body rather than as threads. Validator --------- The extraction root is cleared wholesale before packages are expanded. Removing only the destinations of the current packages left expansions from an earlier invocation in place, and the whole root is scanned afterwards, so stale XML was validated and its members entered the local UID index. Alias detection recurses into generic arguments. The check reduced a parameter to its core type, which discards the arguments, so an alias nested inside one such as List{string} went unreported. The local UID lookup compares the normalized identifier. The whitespace rule rewrites the cref body, but the lookup still used the original, so a cref whose only defect was whitespace also reported a prefix mismatch it did not have. Package validation no longer propagates PackageValidator's exit code. The script decides what that code means and reports accordingly, but the task re-propagated the stale value and failed report-only runs. Documentation build ------------------- The trim no longer rewrites its own input. An isolated analyzer build redirects bin while deliberately sharing obj, so rewriting the intermediate file left the shared copy trimmed, and a later real build could publish trimmed text as the full text or omit the untrimmed copy entirely. The compiler's output now stays authoritative, the trimmed result is a separate intermediate that the output copy is pointed at, and the untrimmed copy is keyed on the output path. Verified across a clean build, two incremental rebuilds, and a real build following an isolated one, for every target framework. Documentation ------------- Corrects the consumer table in documentation.instructions.md, which had lib and ref the wrong way round: lib carries the full text for the API docs pipeline and ref is trimmed for IntelliSense. Corrects three crefs: an Add overload that does not exist, DataAccessKind where the documented property takes SystemDataAccessKind, and prose calling the SqlClientFactory.Instance field a property. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 5ccb6cdb296f0352233acdd99a07a1f9b3c75040) --- .../documentation.instructions.md | 6 +- .../SqlClientFactory.xml | 2 +- .../SqlParameterCollection.xml | 2 +- .../SqlFunctionAttribute.xml | 2 +- .../onebranch/scripts/validate-xml-docs.ps1 | 58 ++++++++++++++-- .../steps/validate-packages-step.yml | 5 ++ .../ref/Microsoft.Data.SqlClient.csproj | 67 +++++++++++++------ 7 files changed, 110 insertions(+), 32 deletions(-) diff --git a/.github/instructions/documentation.instructions.md b/.github/instructions/documentation.instructions.md index 49313c6ee9..f663c2e503 100644 --- a/.github/instructions/documentation.instructions.md +++ b/.github/instructions/documentation.instructions.md @@ -180,10 +180,10 @@ The driver package ships **two** XML documentation files per target framework, a | Package folder | Content | Consumer | |----------------|---------|----------| -| `lib//` | Full documentation, including `` and `` | IntelliSense | -| `ref//` | Trimmed by `tools/intellisense/TrimDocs.ps1`, which strips `` and `` | Reference assemblies | +| `lib//` | Full documentation, including `` and `` | The .NET API docs pipeline, which builds the Learn pages | +| `ref//` | Trimmed by `tools/intellisense/TrimDocs.ps1`, which strips `` and `` | Visual Studio IntelliSense | -Remarks and examples render poorly in Visual Studio tooltips, which is why `ref/` is trimmed. The two files must never be the same: if `lib/` is sourced from the trimmed artifact, IntelliSense silently loses every remark and example. That regression shipped in 7.1.0, so the packaged-documentation gate now fails the build when a package's `lib/` XML is trimmed, its `ref/` XML is not, or the two are byte-identical. +Remarks and examples render poorly in Visual Studio tooltips, which is why the `ref/` copy is trimmed. The two files must never be the same: if `lib/` is sourced from the trimmed artifact, the published Learn pages silently lose every remark and example. That regression shipped in 7.1.0, so the packaged-documentation gate now fails the build when a package's `lib/` XML is trimmed, its `ref/` XML is not, or the two are byte-identical. When changing the `` mappings in `Microsoft.Data.SqlClient.nuspec`, keep `lib/` pointed at the implementation artifact and `ref/` at the reference artifact. diff --git a/doc/snippets/Microsoft.Data.SqlClient/SqlClientFactory.xml b/doc/snippets/Microsoft.Data.SqlClient/SqlClientFactory.xml index 8a2347c0fb..473003728c 100644 --- a/doc/snippets/Microsoft.Data.SqlClient/SqlClientFactory.xml +++ b/doc/snippets/Microsoft.Data.SqlClient/SqlClientFactory.xml @@ -278,7 +278,7 @@ - The following code fragment uses the property to retrieve a instance, and then return a strongly typed instance: + The following code fragment uses the field to retrieve a instance, and then return a strongly typed instance: SqlClientFactory newFactory = SqlClientFactory.Instance; diff --git a/doc/snippets/Microsoft.Data.SqlClient/SqlParameterCollection.xml b/doc/snippets/Microsoft.Data.SqlClient/SqlParameterCollection.xml index 25a9fb5649..e78da3365e 100644 --- a/doc/snippets/Microsoft.Data.SqlClient/SqlParameterCollection.xml +++ b/doc/snippets/Microsoft.Data.SqlClient/SqlParameterCollection.xml @@ -159,7 +159,7 @@ parameters.Add("@pname", Convert.ToInt32(0)); - If you do not perform this conversion, the compiler assumes that you are trying to call the overload. + If you do not perform this conversion, the compiler assumes that you are trying to call the overload. diff --git a/doc/snippets/Microsoft.SqlServer.Server/SqlFunctionAttribute.xml b/doc/snippets/Microsoft.SqlServer.Server/SqlFunctionAttribute.xml index 47b6a57154..18e8fc7608 100644 --- a/doc/snippets/Microsoft.SqlServer.Server/SqlFunctionAttribute.xml +++ b/doc/snippets/Microsoft.SqlServer.Server/SqlFunctionAttribute.xml @@ -176,7 +176,7 @@ Indicates whether the function requires access to data stored in the system catalogs or virtual system tables of SQL Server. - : Does not access system data. : Only reads system data. + : Does not access system data. : Only reads system data. The default is . diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 3d1ad5ebcc..fa21ed57dd 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -393,9 +393,10 @@ function Test-Cref { # report the distinct offenders once rather than once per parameter. $aliases = [System.Collections.Generic.List[string]]::new() foreach ($argument in (Split-DocIdArguments -Arguments $arguments)) { - $core = Get-DocIdCoreTypeName -TypeName $argument - if ($script:CSharpAliases.Contains($core) -and -not $aliases.Contains($core)) { - $aliases.Add($core) + foreach ($alias in (Get-DocIdAlias -TypeName $argument)) { + if (-not $aliases.Contains($alias)) { + $aliases.Add($alias) + } } } if ($aliases.Count -gt 0) { @@ -452,7 +453,11 @@ function Test-Cref { } if ($isLocal) { - if (-not $script:LocalUids.Contains($trimmed)) { + # Compared against the normalized UID rather than the original: the whitespace rule + # above rewrote $body, so a cref whose only defect is whitespace still resolves here + # and is reported once, for the whitespace, instead of also as a prefix mismatch. + $normalized = "${prefix}:$body" + if (-not $script:LocalUids.Contains($normalized)) { # The same member under a different prefix is the common case here: a cref written # as M: for a property, or T: for a member, names something real but produces a UID # that matches nothing. Say which prefix was expected rather than reporting a bare @@ -609,6 +614,41 @@ function Test-IsReferenceDocumentationPath { return $false } +<# + Returns every C# alias used anywhere in a documentation ID parameter. + + Generic arguments are inspected recursively rather than discarded, because an alias nested + inside one, as in List{string}, is just as wrong as an alias at the top level and would + otherwise pass unreported. +#> +function Get-DocIdAlias { + param([Parameter(Mandatory)][AllowEmptyString()][string]$TypeName) + + $found = [System.Collections.Generic.List[string]]::new() + + $core = Get-DocIdCoreTypeName -TypeName $TypeName + if ($script:CSharpAliases.Contains($core)) { + $found.Add($core) + } + + # Recurse into the generic argument list, which Get-DocIdCoreTypeName deliberately drops. + $trimmedType = $TypeName.Trim() + $braceIndex = $trimmedType.IndexOf('{') + if ($braceIndex -ge 0) { + $closeIndex = $trimmedType.LastIndexOf('}') + if ($closeIndex -gt $braceIndex) { + $inner = $trimmedType.Substring($braceIndex + 1, $closeIndex - $braceIndex - 1) + foreach ($argument in (Split-DocIdArguments -Arguments $inner)) { + foreach ($alias in (Get-DocIdAlias -TypeName $argument)) { + $found.Add($alias) + } + } + } + } + + return , $found +} + function Resolve-InputPaths { param( [string[]]$Paths, @@ -742,14 +782,18 @@ if (-not [string]::IsNullOrWhiteSpace($PackagesPath)) { throw "No .nupkg files were found under '$PackagesPath'." } + # Cleared wholesale rather than per package. Removing only the destinations for the current + # packages would leave expansions from an earlier invocation in place, and the whole extraction + # root is scanned below, so that stale XML would be validated and its members added to the + # local UID index. + if (Test-Path -LiteralPath $ExtractPath) { + Remove-Item -LiteralPath $ExtractPath -Recurse -Force + } New-Item -ItemType Directory -Force -Path $ExtractPath | Out-Null Add-Type -AssemblyName System.IO.Compression.FileSystem foreach ($package in $packages) { $destination = Join-Path $ExtractPath $package.BaseName - if (Test-Path -LiteralPath $destination) { - Remove-Item -LiteralPath $destination -Recurse -Force - } Write-Host "Expanding $($package.Name)" [System.IO.Compression.ZipFile]::ExtractToDirectory($package.FullName, $destination) diff --git a/eng/pipelines/onebranch/steps/validate-packages-step.yml b/eng/pipelines/onebranch/steps/validate-packages-step.yml index e85ec2e118..e66756368f 100644 --- a/eng/pipelines/onebranch/steps/validate-packages-step.yml +++ b/eng/pipelines/onebranch/steps/validate-packages-step.yml @@ -105,3 +105,8 @@ steps: pwsh: true filePath: $(REPO_ROOT)/eng/pipelines/onebranch/scripts/validate-packages.ps1 arguments: $(validatePackagesArguments) + # The script inspects PackageValidator's exit codes itself and decides what they mean: a + # tripped gate is a warning in report-only mode and a throw otherwise. $LASTEXITCODE still + # holds the tool's code afterwards, which the task would otherwise re-propagate and fail a + # report-only run. + ignoreLASTEXITCODE: true diff --git a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj index e2b14cb4a8..848098b248 100644 --- a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj +++ b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj @@ -65,36 +65,30 @@ - + dotnet tool run pwsh -- -NonInteractive -ExecutionPolicy Unrestricted - -Command "$(RepoRoot)tools\intellisense\TrimDocs.ps1 -inputFile '$(DocumentationFile)' -outputFile '$(DocumentationFile)'" + -Command "$(RepoRoot)tools\intellisense\TrimDocs.ps1 -inputFile '$(DocumentationFile)' -outputFile '$(IntermediateOutputPath)trimmed/$(TargetName).xml'" @@ -103,11 +97,46 @@ + + + + - - + + + + + + + + + + From 659cf1a854c2c8b8672f8135abf717987daba30c Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 08:31:50 -0300 Subject: [PATCH 06/23] Reject angle-bracket generics and multidimensional array type references Two gaps in the documentation-ID grammar, both reported in review. Angle brackets were never rejected. A documentation ID writes generic arguments in braces, so a cref such as T:System.Collections.Generic.List is malformed, but it carries an allowed namespace root and resolves against no local index in source mode, so it passed the preflight and would still produce the unresolved published reference this validation exists to prevent. Angle brackets are now rejected wherever they appear, with the braced form suggested. The constructed-type rule recognized only the [] suffix. The multidimensional forms [,] and the documentation-ID spelling [0:,0:] denote constructed types just as [] does, so a T: reference to one passed. Any bracketed array suffix is now matched. The same suffixes remain legal as parameter types, where they name a real signature rather than a type page. Adds regression cases for both, including controls for the valid braced generic and for a multidimensional array used as a parameter type. Reverting either rule fails three of them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 22c283161528e2fc448debb96ac92488e48aec87) --- .../scripts/tests/validate-xml-docs.Tests.ps1 | 49 ++++++++++++++++++- .../onebranch/scripts/validate-xml-docs.ps1 | 18 ++++++- 2 files changed, 64 insertions(+), 3 deletions(-) diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index c82755626c..20e50eea39 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -241,7 +241,54 @@ Describe 'validate-xml-docs.ps1' { { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw } - It 'accepts generic types whose arguments contain commas' { + It 'rejects angle brackets in a generic type reference' { + # Angle brackets are C# source syntax; a documentation ID writes generic arguments in + # braces. Without this the cref has an allowed namespace root and passes the source + # gate, then fails to resolve once published. + $snippets = New-SnippetDirectory -Crefs @('T:System.Collections.Generic.List<System.String>') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'invalid-docid' + $finding.Message | Should -BeLike '*braces*List{System.String}*' + } + + It 'rejects angle brackets inside a member signature' { + $snippets = New-SnippetDirectory -Crefs @( + 'M:Microsoft.Data.SqlClient.Sample.Use(System.Collections.Generic.List<System.String>)') + + { & $scriptPath -SnippetsDirectory $snippets } | + Should -Throw '*failed with 1 issue*' + } + + It 'rejects a multidimensional array type reference' -ForEach @( + @{ Cref = 'T:System.Int32[,]' } + @{ Cref = 'T:System.Int32[,,]' } + @{ Cref = 'T:System.Int32[0:,0:]' } + ) { + # [] was recognized but the multidimensional forms were not, so an invalid T: array + # cref passed. + $snippets = New-SnippetDirectory -Crefs @($Cref) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'invalid-docid' + $finding.Message | Should -BeLike '*constructed type*' + } + + It 'accepts a multidimensional array inside a member signature' { + # The same suffix is legal as a parameter type; only a T: reference to it is wrong. + $snippets = New-SnippetDirectory -Crefs @( + 'M:Microsoft.Data.SqlClient.Sample.Grid(System.Int32[0:,0:])') + + { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw + } + + It 'accepts generic arguments written in braces' { $snippets = New-SnippetDirectory -Crefs @( 'M:Microsoft.Data.SqlClient.Sample.Map(System.Collections.Generic.Dictionary{System.String,System.Int32})', 'T:System.Collections.Generic.IReadOnlyList`1') diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index fa21ed57dd..252edf176d 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -365,6 +365,17 @@ function Test-Cref { "'${prefix}:$body'.") } + # A documentation ID writes generic arguments in braces, as List{System.String}. Angle brackets + # are C# source syntax and never appear in a documentation ID, so they are rejected wherever + # they occur rather than only in a signature. + if ($body -match '[<>]') { + $suggestion = ($body -replace '<', '{') -replace '>', '}' + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' uses angle brackets for its generic arguments. Documentation IDs use " + + "braces; use '${prefix}:$suggestion'.") + return + } + $signatureStart = $body.IndexOf('(') $namePart = if ($signatureStart -ge 0) { $body.Substring(0, $signatureStart) } else { $body } @@ -412,8 +423,11 @@ function Test-Cref { # An array, pointer or by-reference construction is not a named type, so it has no type page # and no UID in the Learn xref map. Only a T: cref can make this mistake; the same suffixes are # legal inside a member signature. - if ($prefix -eq 'T' -and $body -match '(\[\]|\*|@|&)$') { - $element = $body -replace '(\[\]|\*|@|&)+$', '' + # + # The array suffix is matched in any of its forms: [] for one dimension, [,] for more, and the + # documentation-ID spelling [0:,0:] that records lower bounds. + if ($prefix -eq 'T' -and $body -match '(\[[\d:,]*\]|\*|@|&)$') { + $element = $body -replace '(\[[\d:,]*\]|\*|@|&)+$', '' Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( "Cref '$trimmed' names a constructed type, which has no documentation page. " + "Reference the element type instead, for example array.") From 6494db3049620ecd85eaa70853cea53d7fbd8f63 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 08:42:38 -0300 Subject: [PATCH 07/23] Keep malformed XML fatal in report-only mode Report-only downgrades findings so a run can surface everything at once, but it was also downgrading malformed-xml. A file that could not be parsed was never examined, so none of its cross-references were checked and suppressing it reported an all-clear for content nobody read. A malformed packaged XML file could therefore pass a non-official run. This also contradicted the stated contract, which the pipeline guidance and the sibling validation scripts already follow: report-only covers findings, while unreadable or missing input fails outright because it produced no result to downgrade. Unreadable files are now reported and fail in either mode, with a message naming how many could not be read. The report is still written first, so the findings remain available. Ordinary findings continue to be downgraded. Adds a case for the contract and one confirming report-only still works for readable files, and updates the two existing cases that asserted the old message. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 8b9d8326d97ce61226d64018eb9b400dc583490c) --- .../scripts/tests/validate-xml-docs.Tests.ps1 | 31 +++++++++++++++++-- .../onebranch/scripts/validate-xml-docs.ps1 | 18 ++++++++++- 2 files changed, 46 insertions(+), 3 deletions(-) diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index 20e50eea39..74a42358a3 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -771,7 +771,9 @@ Describe 'validate-xml-docs.ps1' { $project = New-Project -GeneratesDocumentation $report = Join-Path (New-TestDirectory) 'report.json' - & $scriptPath -DocumentationPath $path -ProjectPath $project -ReportPath $report -ReportOnly + # Throws because the file could not be read; the report is still written beforehand. + { & $scriptPath -DocumentationPath $path -ProjectPath $project -ReportPath $report -ReportOnly } | + Should -Throw '*could not read 1 file*' $categories = @((Get-Report -Path $report).Findings.Category) $categories | Should -Contain 'malformed-xml' @@ -1004,13 +1006,38 @@ Describe 'validate-xml-docs.ps1' { $finding.Category | Should -Be 'malformed-xml' } + It 'fails on malformed XML even in report-only mode' { + # Report-only downgrades findings, but a file that could not be parsed was never + # examined, so suppressing it would report an all-clear for content nobody read. + $path = New-TestDirectory + '' | Set-Content -LiteralPath (Join-Path $path 'Broken.xml') -Encoding utf8 + + { & $scriptPath -SnippetsDirectory $path -ReportOnly } | + Should -Throw '*could not read 1 file*' + } + + It 'still downgrades ordinary findings in report-only mode alongside readable files' { + # The malformed-XML rule must not make report-only useless for everything else. + $snippets = New-SnippetDirectory -Crefs @('T:System.Byte[]') + + { & $scriptPath -SnippetsDirectory $snippets -ReportOnly } | Should -Not -Throw + } + It 'continues past a malformed file to validate the rest' { $path = New-TestDirectory '' | Set-Content -LiteralPath (Join-Path $path 'Broken.xml') -Encoding utf8 '' | Set-Content -LiteralPath (Join-Path $path 'Good.xml') -Encoding utf8 + $report = Join-Path (New-TestDirectory) 'report.json' - { & $scriptPath -SnippetsDirectory $path } | Should -Throw '*failed with 2 issues*' + # The unreadable file decides the failure message, but the readable one is still + # validated and its finding recorded. + { & $scriptPath -SnippetsDirectory $path -ReportPath $report } | + Should -Throw '*could not read 1 file*' + + $categories = @((Get-Report -Path $report).Findings.Category) + $categories | Should -Contain 'malformed-xml' + $categories | Should -Contain 'invalid-docid' } It 'fails when no input paths are supplied' { diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 252edf176d..c557c38ee2 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -89,6 +89,9 @@ Report findings without failing, overriding -FailOn. Used to shake the gate out on a pipeline before its findings are fixed. The official pipeline never sets this. + Does not cover a file that could not be parsed. Such a file was never examined, so suppressing + it would report an all-clear for content nobody read. + .PARAMETER ProjectSearchRoot Directory scanned recursively for project files, used with -PackagesPath to check that every assembly whose project sets GenerateDocumentationFile ships its XML documentation beside it in @@ -111,7 +114,10 @@ .OUTPUTS Findings are categorized as: - malformed-xml error File is not well-formed XML. + malformed-xml error File is not well-formed XML. Fails the step even in + report-only mode: the file was never examined, so none of + its cross-references were checked and there is no result + to downgrade. unresolved-cref error Compiler could not bind the cref and emitted a "!:" prefix. invalid-docid error Documentation ID violates the documentation-ID grammar. unknown-namespace-root error Leading identifier is not an allowed namespace root. @@ -1264,6 +1270,16 @@ function Set-SucceededWithIssues { Write-Host '##vso[task.complete result=SucceededWithIssues;]' } +# A file that could not be parsed was never examined, so none of its cross-references were checked. +# Reporting that as a suppressible finding would let a report-only run give an all-clear for +# content nobody read, so it fails regardless of mode. This mirrors a missing input path, and the +# sibling validation scripts, which also fail outright rather than reporting. +$unreadableFindings = @($findings | Where-Object { $_.Category -eq 'malformed-xml' }) +if ($unreadableFindings.Count -gt 0) { + $noun = if ($unreadableFindings.Count -eq 1) { 'file' } else { 'files' } + throw "XML documentation validation could not read $($unreadableFindings.Count) $noun. Review the preceding errors." +} + if ($ReportOnly) { if ($notableFindings.Count -gt 0) { Write-Host "##vso[task.logissue type=warning]XML documentation validation found $($notableFindings.Count) issue(s) but is running in report-only mode, so the build is not failed. $($gatingFindings.Count) of them would fail a gating run." From 7ea9f7f20b54a6d0f697d27c18a50e3d1e6c0eaa Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 08:55:50 -0300 Subject: [PATCH 08/23] Apply the public API classification to wrong-kind prefixes, and track sibling doc directories Two findings, both in code that predates the previous round. A wrong-kind prefix was always a warning, even where the reference documentation proves the containing member is public. Every pipeline invocation gates on error, so a new M: reference to a public property would stay unresolved on the published page yet pass validation. It is classified the same way as an unresolvable reference now, and for the same reason: the wrong prefix produces a UID that matches nothing, so from a public member it leaves an unresolved reference. Non-public and unknown-surface cases keep their lower severity. The classification lived inline in one of the two branches that report an unresolvable reference, which is how the branches came to disagree. It is now a single function both call. The compile inputs covered only the repository-level doc/snippets tree. The Abstractions and Azure projects include documentation from a doc directory beside their own src directory, so editing one of those files left CoreCompile up to date and published stale documentation. Both locations are now declared. Verified on Abstractions with a control: without the addition an edit to its documentation is silently dropped from the build output, and with it the edit is picked up. Escalating wrong-kind prefixes introduces no new errors against the current packages, the earlier prefix corrections having already cleared the public ones. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 10a6c2e07030f5aef215c26d947858827d50a130) --- .../scripts/tests/validate-xml-docs.Tests.ps1 | 61 +++++++++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 56 ++++++++++++----- src/Directory.Build.props | 13 ++-- 3 files changed, 112 insertions(+), 18 deletions(-) diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index 74a42358a3..66fbf58346 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -368,6 +368,67 @@ Describe 'validate-xml-docs.ps1' { Should -Throw '*failed with 1 issue*' } + It 'reports a wrong-kind prefix from a public member as an error' { + # Same classification as an unresolvable reference, and for the same reason: the wrong + # prefix produces a UID that matches nothing, so a public member's page carries an + # unresolved reference. + $staging = New-TestDirectory + New-Item -ItemType Directory -Path (Join-Path $staging 'lib/net8.0') -Force | Out-Null + New-Item -ItemType Directory -Path (Join-Path $staging 'ref/net8.0') -Force | Out-Null + + '' + + 'S.' + + 'W.' + + '' | + Set-Content -LiteralPath (Join-Path $staging 'lib/net8.0/Microsoft.Data.SqlClient.xml') -Encoding utf8 + '' + + 'S.' + + 'W.' + + '' | + Set-Content -LiteralPath (Join-Path $staging 'ref/net8.0/Microsoft.Data.SqlClient.xml') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory($staging, (Join-Path $packages 'P.1.0.0.nupkg')) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) ` + -ReportPath $report -ReportOnly + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Category -like 'mismatched*' })[0] + $finding.Category | Should -Be 'mismatched-public-docid-prefix' + $finding.Severity | Should -Be 'error' + } + + It 'reports a wrong-kind prefix from a non-public member as a warning' { + $staging = New-TestDirectory + New-Item -ItemType Directory -Path (Join-Path $staging 'lib/net8.0') -Force | Out-Null + New-Item -ItemType Directory -Path (Join-Path $staging 'ref/net8.0') -Force | Out-Null + + # The holder is absent from ref/, so it is not part of the public API surface. + '' + + 'S.' + + 'I.' + + '' | + Set-Content -LiteralPath (Join-Path $staging 'lib/net8.0/Microsoft.Data.SqlClient.xml') -Encoding utf8 + 'S.' | + Set-Content -LiteralPath (Join-Path $staging 'ref/net8.0/Microsoft.Data.SqlClient.xml') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory($staging, (Join-Path $packages 'P.1.0.0.nupkg')) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) ` + -ReportPath $report -ReportOnly + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Category -like 'mismatched*' })[0] + $finding.Category | Should -Be 'mismatched-docid-prefix' + $finding.Severity | Should -Be 'warning' + } + It 'names the expected prefix when a cref uses the wrong member kind' { $docs = New-DocumentationDirectory ` -Members @('P:Microsoft.Data.SqlClient.SqlCommand.CommandTimeout') ` diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index c557c38ee2..714815696c 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -141,8 +141,14 @@ because a reference may legitimately resolve elsewhere: to another target framework, or to a sibling assembly that this run did not include. - mismatched-docid-prefix warning Cref names a real member but with the wrong kind prefix, - such as M: on a property or T: on a member. + mismatched-public-docid-prefix + error Cref names a real member but with the wrong kind prefix, + such as M: on a property, and is referenced from a public + API member. The wrong prefix produces a UID that matches + nothing, so the published page carries an unresolved + reference. + mismatched-docid-prefix warning As above, but referenced from a member that is not public, + or from any member when the public API surface is unknown. missing-external-uid warning Cref is absent from the supplied Learn xref map. unprefixed-cref info Cref carries no "T:"/"M:"/... prefix. Legal; the compiler binds it. Reported for visibility only. @@ -208,6 +214,7 @@ $script:CategorySeverities = [ordered]@{ 'ref-documentation-untrimmed' = 'error' 'lib-ref-documentation-identical' = 'error' 'missing-public-uid' = 'error' + 'mismatched-public-docid-prefix' = 'error' 'missing-local-uid' = 'info' 'mismatched-docid-prefix' = 'warning' 'missing-documentation' = 'error' @@ -485,20 +492,21 @@ function Test-Cref { if ($script:LocalUidsByBody.ContainsKey($namePart) -or $script:LocalUidsByBody.ContainsKey($body)) { $key = if ($script:LocalUidsByBody.ContainsKey($body)) { $body } else { $namePart } $actual = ($script:LocalUidsByBody[$key] | Sort-Object) -join ', ' - Add-Finding @Context -Category 'mismatched-docid-prefix' -Cref $Cref -Message ( - "Cref '$trimmed' uses prefix '${prefix}:', but '$key' was emitted as " + - "'$actual'. Use the prefix matching the member kind.") + $message = "Cref '$trimmed' uses prefix '${prefix}:', but '$key' was emitted " + + "as '$actual'. Use the prefix matching the member kind." + + # Classified the same way as an unresolvable reference below, and for the same + # reason: the wrong prefix produces a UID that matches nothing, so from a + # public member it leaves an unresolved reference on the published page. + if (Test-ContainingMemberIsPublic -Context $Context) { + Add-Finding @Context -Category 'mismatched-public-docid-prefix' -Cref $Cref -Message $message + } + else { + Add-Finding @Context -Category 'mismatched-docid-prefix' -Cref $Cref -Message $message + } } else { - # Severity follows the member that holds the reference, not the reference - # itself: a public member's page is published, so a reference it cannot resolve - # becomes a visible xref-not-found. An internal member is never published, so - # the same reference reaches no reader. - $containingMemberIsPublic = $null -ne $script:PublicUids -and - -not [string]::IsNullOrEmpty($Context.Member) -and - $script:PublicUids.Contains($Context.Member) - - if ($containingMemberIsPublic) { + if (Test-ContainingMemberIsPublic -Context $Context) { Add-Finding @Context -Category 'missing-public-uid' -Cref $Cref -Message ( "Cref '$trimmed' names this repository but no matching documented " + 'member was emitted by the build. It is referenced from a public API ' + @@ -669,6 +677,26 @@ function Get-DocIdAlias { return , $found } +<# + Reports whether the member holding a cref is part of the published API surface. + + Severity follows the member that holds a reference, not the reference itself: a public member's + page is published, so a reference it cannot resolve becomes a visible xref-not-found. An + internal member is never published, so the same reference reaches no reader. Every rule that + reports an unresolvable reference uses this, so they cannot classify the same situation + differently. + + Answers false when no reference documentation identified the surface, which leaves such a + finding at its lower severity rather than escalating on a guess. +#> +function Test-ContainingMemberIsPublic { + param([Parameter(Mandatory)][hashtable]$Context) + + return $null -ne $script:PublicUids -and + -not [string]::IsNullOrEmpty($Context.Member) -and + $script:PublicUids.Contains($Context.Member) +} + function Resolve-InputPaths { param( [string[]]$Paths, diff --git a/src/Directory.Build.props b/src/Directory.Build.props index 6151f5a121..d1655a6068 100644 --- a/src/Directory.Build.props +++ b/src/Directory.Build.props @@ -255,15 +255,20 @@ + From 6a6b5df321849eb021386a0eff85fb965062aea4 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 09:05:42 -0300 Subject: [PATCH 09/23] Scope documentation compile inputs and fix a Format cref The CustomAdditionalCompileInputs wildcards were declared in src/Directory.Build.props, which every project under src/ inherits. That gave all 31 projects, 23 of them test projects that contain no references, a dependency on the whole doc/snippets tree, so touching any snippet invalidated CoreCompile solution-wide. Move the item group to src/Directory.Build.targets, imported after the project body has run, and guard it with an IncludesDocumentationSnippets property that the five projects actually using set for themselves. Those five continue to rebuild when a snippet changes; the other 26 no longer see the inputs at all. Also correct a cref in SqlUserDefinedTypeAttribute.xml. The sentence lists the attribute's properties, alongside MaxByteSize which is already written as P:, so Format there is the property rather than the enum type that shares its name. The other references to the enum in that file and its siblings are genuinely the type and are left alone. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit e7a9f5ad14b207307da0a8d6f3caa776c792cd34) --- .../SqlUserDefinedTypeAttribute.xml | 2 +- src/Directory.Build.props | 18 ---------------- src/Directory.Build.targets | 21 +++++++++++++++++++ .../Abstractions/src/Abstractions.csproj | 2 ++ .../Azure/src/Azure.csproj | 2 ++ .../ref/Microsoft.Data.SqlClient.csproj | 2 ++ .../src/Microsoft.Data.SqlClient.csproj | 2 ++ .../Microsoft.SqlServer.Server.csproj | 4 ++++ 8 files changed, 34 insertions(+), 19 deletions(-) diff --git a/doc/snippets/Microsoft.SqlServer.Server/SqlUserDefinedTypeAttribute.xml b/doc/snippets/Microsoft.SqlServer.Server/SqlUserDefinedTypeAttribute.xml index bd2728e0a1..957838fb14 100644 --- a/doc/snippets/Microsoft.SqlServer.Server/SqlUserDefinedTypeAttribute.xml +++ b/doc/snippets/Microsoft.SqlServer.Server/SqlUserDefinedTypeAttribute.xml @@ -44,7 +44,7 @@ A required attribute on a user-defined type (UDT), used to confirm that the given type is a UDT and to indicate the storage format of the UDT. - The following example specifies that the of the user-defined type is and the is 8000 bytes. + The following example specifies that the of the user-defined type is and the is 8000 bytes. diff --git a/src/Directory.Build.props b/src/Directory.Build.props index d1655a6068..d9b9a7cdc8 100644 --- a/src/Directory.Build.props +++ b/src/Directory.Build.props @@ -253,22 +253,4 @@ 14 - - - - - - diff --git a/src/Directory.Build.targets b/src/Directory.Build.targets index 553de3ef2c..4b486817c4 100644 --- a/src/Directory.Build.targets +++ b/src/Directory.Build.targets @@ -18,4 +18,25 @@ + + + + + diff --git a/src/Microsoft.Data.SqlClient.Extensions/Abstractions/src/Abstractions.csproj b/src/Microsoft.Data.SqlClient.Extensions/Abstractions/src/Abstractions.csproj index e8b3c55ba0..c047e5a45e 100644 --- a/src/Microsoft.Data.SqlClient.Extensions/Abstractions/src/Abstractions.csproj +++ b/src/Microsoft.Data.SqlClient.Extensions/Abstractions/src/Abstractions.csproj @@ -8,6 +8,8 @@ true + + true enable enable Microsoft.Data.SqlClient diff --git a/src/Microsoft.Data.SqlClient.Extensions/Azure/src/Azure.csproj b/src/Microsoft.Data.SqlClient.Extensions/Azure/src/Azure.csproj index da0a4cb4c7..788db0c952 100644 --- a/src/Microsoft.Data.SqlClient.Extensions/Azure/src/Azure.csproj +++ b/src/Microsoft.Data.SqlClient.Extensions/Azure/src/Azure.csproj @@ -8,6 +8,8 @@ true + + true enable enable Microsoft.Data.SqlClient diff --git a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj index 848098b248..56bc354e15 100644 --- a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj +++ b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj @@ -8,6 +8,8 @@ true + + true diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj b/src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj index c81f18031b..79ebda9c20 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj +++ b/src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj @@ -13,6 +13,8 @@ true true + + true diff --git a/src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj b/src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj index f3a8da6200..68bd9beab2 100644 --- a/src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj +++ b/src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj @@ -14,9 +14,13 @@ Deliberately unconditional. The net46 target consists of type forwarders, so its documentation file has no members in it, but keeping the setting free of a TargetFramework condition means the project gives one unambiguous answer to whether it produces documentation. + + IncludesDocumentationSnippets opts this project into the compile inputs declared in + src/Directory.Build.targets, so editing one of those snippets rebuilds the documentation file. --> true + true From a1b4f81465740aa59e103e7f5ad23789ad30a9c9 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 09:16:17 -0300 Subject: [PATCH 10/23] Detect nested aliases, invalidate on tool change, correct severity docs Three fixes from review. The alias check for type crefs compared only the outer type name, which a generic argument never reaches: T:List{string} passed because List is a perfectly ordinary type. Use Get-DocIdAlias, which already recurses, so an alias nested at any depth is reported, and cover the form with tests. This also catches an alias in the declaring type of a member cref, whose name part is examined by the same code. The documentation trimming target listed only the generated XML as an input, so editing TrimDocs.ps1 left the trimmed output considered up to date and kept text produced by the old rules until someone built clean. Add the script to Inputs through a property, and spell its path with forward slashes because the same property is passed to pwsh, where a backslash is an escape character rather than a separator on non-Windows hosts. The failOn documentation described a severity mapping that no longer held: it omitted missing-documentation and the two public resolution categories from the error list, and called missing-local-uid a warning when it is informational. Since the pipelines gate on error, that understated the default gate. Restate it in terms of the public/non-public split that decides severity, and pin the whole table with a test, as nothing else tied the prose to the code. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 7aa2ddfc636b35e7e66f3a2dc0c0cd0962639c37) --- .../scripts/tests/validate-xml-docs.Tests.ps1 | 93 +++++++++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 18 +++- .../steps/validate-xml-docs-step.yml | 21 +++-- .../ref/Microsoft.Data.SqlClient.csproj | 12 ++- 4 files changed, 132 insertions(+), 12 deletions(-) diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index 66fbf58346..acab647f00 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -195,6 +195,49 @@ Describe 'validate-xml-docs.ps1' { $findings.Count | Should -Be 1 } + It 'rejects a C# alias nested in a generic type cref' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Collections.Generic.List{string}') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Category | Should -Be 'invalid-docid' + $findings[0].Message | Should -BeLike "*C# alias 'string'*" + } + + It 'reports each distinct alias nested in a generic type cref' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Collections.Generic.Dictionary{string,int}') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Message | Should -BeLike "*C# aliases 'string', 'int'*" + } + + It 'accepts a generic type cref whose arguments are fully qualified' { + $snippets = New-SnippetDirectory -Crefs @('T:System.Collections.Generic.List{System.String}') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + @((Get-Report -Path $report).Findings).Count | Should -Be 0 + } + + It 'rejects a C# alias nested in the declaring type of a member cref' { + $snippets = New-SnippetDirectory -Crefs @('M:System.Collections.Generic.List{string}.Add(System.String)') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Message | Should -BeLike "*C# alias 'string'*" + } + It 'rejects whitespace in an otherwise correct signature' { $snippets = New-SnippetDirectory -Crefs @('M:Microsoft.Data.SqlClient.SqlParameterCollection.Add(System.String, System.String)') $report = Join-Path (New-TestDirectory) 'report.json' @@ -1054,6 +1097,56 @@ Describe 'validate-xml-docs.ps1' { } } + <# + Pins the severity of every category. + + The step template documents which categories the default 'error' gate covers, and nothing + else ties that prose to this table, so a severity changed here would silently leave the + documented contract wrong. Failing this test is the prompt to update + eng/pipelines/onebranch/steps/validate-xml-docs-step.yml alongside the change. + #> + Context 'severity contract' { + + It 'assigns the documented severity to every category' { + $ast = [System.Management.Automation.Language.Parser]::ParseFile( + $scriptPath, [ref]$null, [ref]$null) + $assignment = $ast.Find({ + param($node) + $node -is [System.Management.Automation.Language.AssignmentStatementAst] -and + $node.Left.Extent.Text -eq '$script:CategorySeverities' + }, $true) + $assignment | Should -Not -BeNullOrEmpty + + $actual = [scriptblock]::Create($assignment.Right.Extent.Text).Invoke()[0] + + $expected = [ordered]@{ + 'malformed-xml' = 'error' + 'unresolved-cref' = 'error' + 'invalid-docid' = 'error' + 'unknown-namespace-root' = 'error' + 'stale-allowlist-entry' = 'error' + 'lib-documentation-trimmed' = 'error' + 'ref-documentation-untrimmed' = 'error' + 'lib-ref-documentation-identical' = 'error' + 'missing-public-uid' = 'error' + 'mismatched-public-docid-prefix' = 'error' + 'missing-documentation' = 'error' + 'missing-local-uid' = 'info' + 'mismatched-docid-prefix' = 'warning' + 'unexpected-documentation' = 'warning' + 'documentation-not-expected' = 'info' + 'missing-external-uid' = 'warning' + 'unprefixed-cref' = 'info' + } + + ($actual.Keys | Sort-Object) -join ',' | + Should -Be (($expected.Keys | Sort-Object) -join ',') + foreach ($category in $expected.Keys) { + $actual[$category] | Should -Be $expected[$category] -Because "$category is documented as $($expected[$category])" + } + } + } + Context 'malformed input' { It 'reports a file that is not well-formed XML' { diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 714815696c..318ced5fd2 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -447,10 +447,22 @@ function Test-Cref { return } - if ($script:CSharpAliases.Contains((Get-DocIdCoreTypeName -TypeName $namePart))) { + # Aliases are gathered recursively rather than from the outer name alone, because a generic + # argument is itself a type reference: T:List{string} is as wrong as T:string, and the outer + # name of the former is a perfectly ordinary type. One cref can carry several distinct + # aliases, so they are reported together instead of one finding per argument. + $typeAliases = [System.Collections.Generic.List[string]]::new() + foreach ($alias in (Get-DocIdAlias -TypeName $namePart)) { + if (-not $typeAliases.Contains($alias)) { + $typeAliases.Add($alias) + } + } + if ($typeAliases.Count -gt 0) { + $quotedTypeAliases = ($typeAliases | ForEach-Object { "'$_'" }) -join ', ' + $aliasNoun = if ($typeAliases.Count -eq 1) { 'the C# alias' } else { 'the C# aliases' } Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( - "Cref '$trimmed' uses the C# alias '$namePart'. Documentation IDs name CLR types, so " + - 'use the full type name instead.') + "Cref '$trimmed' uses $aliasNoun $quotedTypeAliases. Documentation IDs name CLR " + + 'types, so use the full type name instead.') return } diff --git a/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml b/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml index d593c44763..d7a197c883 100644 --- a/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml +++ b/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml @@ -53,15 +53,22 @@ parameters: # Finding severities and/or categories that fail the build. # + # Severity follows the visibility of the member that holds a finding, because only a public + # member's page is published, where an unresolvable reference renders as xref-not-found. Each + # resolution rule therefore has a public form that is an error and a non-public form that is not: + # missing-public-uid against missing-local-uid, and mismatched-public-docid-prefix against + # mismatched-docid-prefix. A finding takes the lower form when no reference documentation + # identified the surface, so an unknown surface is never escalated on a guess. + # # The error severity covers malformed-xml, unresolved-cref, invalid-docid, unknown-namespace-root, - # stale-allowlist-entry, and the three package layout categories (lib-documentation-trimmed, - # ref-documentation-untrimmed, lib-ref-documentation-identical). + # stale-allowlist-entry, missing-documentation, the two public resolution categories above, and + # the three package layout categories (lib-documentation-trimmed, ref-documentation-untrimmed, + # lib-ref-documentation-identical). # - # It deliberately does not cover the two resolution categories, which are warnings: - # missing-local-uid fires when a cref names this repository but no matching member was emitted, - # which a target-framework-conditional or cross-assembly member can trigger legitimately; and - # mismatched-docid-prefix depends on the same index. Name either explicitly once a run has - # shown it to be clean. + # The warning severity covers mismatched-docid-prefix, unexpected-documentation and + # missing-external-uid. missing-local-uid, documentation-not-expected and unprefixed-cref are + # informational and never mark the step. Name any category explicitly to gate on it regardless + # of its severity. - name: failOn type: object diff --git a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj index 56bc354e15..ae5d0e5a25 100644 --- a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj +++ b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj @@ -54,6 +54,14 @@ them in the xml files, though, because those are the source for the docs site, which needs to have the full documentation. --> + + + $(RepoRoot)tools/intellisense/TrimDocs.ps1 + + @@ -90,7 +98,7 @@ dotnet tool run pwsh -- -NonInteractive -ExecutionPolicy Unrestricted - -Command "$(RepoRoot)tools\intellisense\TrimDocs.ps1 -inputFile '$(DocumentationFile)' -outputFile '$(IntermediateOutputPath)trimmed/$(TargetName).xml'" + -Command "$(TrimDocsScript) -inputFile '$(DocumentationFile)' -outputFile '$(IntermediateOutputPath)trimmed/$(TargetName).xml'" From fac0e1a96a190e1e01df6d106426ebc4d560ee5f Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 09:25:29 -0300 Subject: [PATCH 11/23] Report a reversed parameter list instead of abandoning the run A cref whose closing parenthesis precedes its opening one, such as M:Microsoft.Data.SqlClient.Sample.Bad)(, satisfied the test for a ')' being present anywhere in the body and then asked Substring for a negative length. The exception escaped, so the run ended without writing the JSON report, including under -ReportOnly where the report is the entire point of the mode. Compare the two positions instead of testing for the character. LastIndexOf answers -1 when ')' is absent, which is below every valid opening position, so one comparison covers both an unterminated list and a reversed one. Neither form was covered, which is how the arithmetic beside them went unnoticed; both are now tested, and the reversed case asserts the report exists rather than only the finding. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 0be0c4591ca9ab8a148f5f7647cbd5fff9f01570) --- .../scripts/tests/validate-xml-docs.Tests.ps1 | 28 +++++++++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 14 +++++++--- 2 files changed, 38 insertions(+), 4 deletions(-) diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index acab647f00..d7876cec7f 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -162,6 +162,34 @@ Describe 'validate-xml-docs.ps1' { $finding.Message | Should -BeLike "*use 'M:Microsoft.Data.SqlClient.SqlConnection.GetSchema'*" } + It 'rejects a signature whose parameter list is never closed' { + $snippets = New-SnippetDirectory -Crefs @('M:Microsoft.Data.SqlClient.SqlConnection.GetSchema(System.String') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'invalid-docid' + $finding.Message | Should -BeLike '*unterminated parameter list*' + } + + <# + The closing parenthesis precedes the opening one, so the argument arithmetic would ask + Substring for a negative length. An exception there ends the run before the report is + written, so this asserts the report exists rather than only the finding. + #> + It 'rejects a signature whose closing parenthesis precedes the opening one' { + $snippets = New-SnippetDirectory -Crefs @('M:Microsoft.Data.SqlClient.SqlConnection.GetSchema)(') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $report | Should -Exist + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'invalid-docid' + $finding.Message | Should -BeLike '*unterminated parameter list*' + } + It 'rejects a C# alias in a method signature' { $snippets = New-SnippetDirectory -Crefs @('M:Microsoft.Data.SqlClient.SqlConnection.GetSchema(string)') $report = Join-Path (New-TestDirectory) 'report.json' diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 318ced5fd2..3f46c35fb7 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -393,15 +393,21 @@ function Test-Cref { $namePart = if ($signatureStart -ge 0) { $body.Substring(0, $signatureStart) } else { $body } if ($signatureStart -ge 0) { - if (-not $body.Contains(')')) { + # The conversion-operator return marker (~) trails the parameter list, so the argument text + # ends at the last ')' rather than at the end of the body. + $signatureEnd = $body.LastIndexOf(')') + + # Catches a missing ')' and one that precedes the '(', which is not a parameter list at + # all. LastIndexOf answers -1 when the character is absent, which is below every valid + # opening position, so both forms fail this comparison. Reaching the arithmetic below with + # either would ask Substring for a negative length, and the resulting exception would + # abandon the run without writing the report that report-only mode exists to produce. + if ($signatureEnd -lt $signatureStart) { Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( "Cref '$trimmed' has an unterminated parameter list.") return } - # The conversion-operator return marker (~) trails the parameter list, so the argument text - # ends at the last ')' rather than at the end of the body. - $signatureEnd = $body.LastIndexOf(')') $arguments = $body.Substring($signatureStart + 1, $signatureEnd - $signatureStart - 1) # A parameterless method's documentation ID is written without parentheses. Emitting "()" From 8f6e19fadaa7f964285b0671f3905701f58a5154 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 10:23:11 -0300 Subject: [PATCH 12/23] Resolve namespace crefs against the namespaces members occupy A cref naming a namespace was exempt from local resolution, because the compiler emits no entry for a namespace and every N: cref would otherwise have been reported as missing. The exemption also excused a misspelling: N:Microsoft.Data.SqlClinet carries an allowed namespace root, and no pipeline invocation supplies an external xref map, so it passed every gate and still became an unresolved reference on Learn. Derive the set of namespaces from the members that were emitted, taking every leading portion of each identifier, and resolve N: crefs against it. A leading portion may also name a type containing a nested one, which is harmless here: the member index is consulted first, so a namespace prefix on a type is reported as the prefix mismatch it is rather than being admitted as a namespace. The unresolved message is chosen by prefix, as saying that no matching member was emitted explains nothing about a cref that never named a member. Verified against the emitted documentation for Microsoft.Data.SqlClient: the three real N: crefs still resolve and the finding count is unchanged at 70. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit cf79cfdb87f92caed4c523057ca929e9e7404dac) --- .../scripts/tests/validate-xml-docs.Tests.ps1 | 45 ++++++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 51 ++++++++++++++++--- 2 files changed, 88 insertions(+), 8 deletions(-) diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index d7876cec7f..6abbfc43a7 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -415,6 +415,51 @@ Describe 'validate-xml-docs.ps1' { @((Get-Report -Path $report).Findings) | Should -BeNullOrEmpty } + <# + A namespace has no entry of its own, so these three cover the set derived + from the members that do: a namespace they occupy resolves, a misspelling of it does + not, and a type named with N: is still reported for its prefix rather than being + admitted by that set. + #> + It 'resolves a namespace that the emitted members occupy' { + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('N:Microsoft.Data.SqlClient') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report -ReportOnly + + @((Get-Report -Path $report).Findings) | Should -BeNullOrEmpty + } + + It 'reports a misspelled local namespace' { + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('N:Microsoft.Data.SqlClinet') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Category | Should -Be 'missing-local-uid' + $findings[0].Message | Should -BeLike '*namespace this repository does not contain*' + } + + It 'reports a type named with the namespace prefix as a prefix mismatch' { + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('N:Microsoft.Data.SqlClient.SqlConnection') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Category | Should -Be 'mismatched-docid-prefix' + $findings[0].Message | Should -BeLike "*was emitted as 'T:'*" + } + It 'reports a local cref with no matching emitted member as information, not an error' { # Without reference documentation the public API surface is unknown, and such a # reference may resolve in another target framework or a sibling assembly. diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 3f46c35fb7..5b1f8d9100 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -485,10 +485,7 @@ function Test-Cref { # Anything this repository owns must be present in the documentation under validation. This can # only be judged when generated documentation was supplied; source mode has no member list. - # - # Namespaces are excluded because the compiler never emits a entry for one, so every - # N: cref would otherwise be reported as missing. - if ($null -ne $script:LocalUids -and $prefix -ne 'N') { + if ($null -ne $script:LocalUids) { $isLocal = $false foreach ($localPrefix in $script:LocalNamespacePrefixes) { if ($namePart -eq $localPrefix -or $namePart.StartsWith("$localPrefix.", [System.StringComparison]::Ordinal)) { @@ -523,17 +520,34 @@ function Test-Cref { Add-Finding @Context -Category 'mismatched-docid-prefix' -Cref $Cref -Message $message } } + elseif ($prefix -eq 'N' -and $script:LocalNamespaces.Contains($body)) { + # A namespace has no entry of its own, so it resolves against the + # namespaces derived from the members that were emitted. Reached only after + # the member index above has ruled out the body naming a type, so an N: cref + # written for a type is still reported as a prefix mismatch. + return + } else { + # A namespace is described by the members occupying it rather than by an entry + # of its own, so it needs its own wording: naming a member that was not emitted + # says nothing about a cref that never named a member. + $subject = if ($prefix -eq 'N') { + "names a namespace this repository does not contain, as no documented " + + 'member occupies it' + } + else { + 'names this repository but no matching documented member was emitted by ' + + 'the build' + } + if (Test-ContainingMemberIsPublic -Context $Context) { Add-Finding @Context -Category 'missing-public-uid' -Cref $Cref -Message ( - "Cref '$trimmed' names this repository but no matching documented " + - 'member was emitted by the build. It is referenced from a public API ' + + "Cref '$trimmed' $subject. It is referenced from a public API " + 'member, so the published page will carry an unresolved reference.') } else { Add-Finding @Context -Category 'missing-local-uid' -Cref $Cref -Message ( - "Cref '$trimmed' names this repository but no matching documented " + - 'member was emitted by the build.') + "Cref '$trimmed' $subject.") } } } @@ -988,8 +1002,14 @@ $script:LocalUidsByBody = [System.Collections.Generic.Dictionary[string, System. # was supplied, which disables the public/internal distinction rather than guessing at it. $script:PublicUids = $null +# Namespaces that the documented members occupy. The compiler emits no entry for a +# namespace, so an N: cref has nothing to resolve against unless the set is derived from the +# members that were emitted. Left null alongside LocalUids when no documentation was supplied. +$script:LocalNamespaces = $null + if ($documentationDocuments.Count -gt 0) { $script:LocalUids = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) + $script:LocalNamespaces = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) foreach ($entry in $documentationDocuments) { foreach ($member in $entry.Document.Root.Element('members').Elements('member')) { $name = $member.Attribute('name') @@ -1017,6 +1037,21 @@ if ($documentationDocuments.Count -gt 0) { [System.StringComparer]::Ordinal) } [void]$script:LocalUidsByBody[$body].Add("$($uid[0]):") + + # Every leading portion of an identifier names somewhere this repository really + # has: a namespace, or a type containing a nested one. Both are recorded, because + # a cref naming a type is recognized by the index above before the namespace set + # is consulted, so admitting a type name here cannot hide a wrong prefix. + $identifier = $body + $parenthesis = $identifier.IndexOf('(') + if ($parenthesis -ge 0) { + $identifier = $identifier.Substring(0, $parenthesis) + } + + $segments = $identifier.Split('.') + for ($index = 1; $index -lt $segments.Length; $index++) { + [void]$script:LocalNamespaces.Add(($segments[0..($index - 1)] -join '.')) + } } } } From c315a527ac43b7504a0c23daadb17616fae5a052 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 10:49:06 -0300 Subject: [PATCH 13/23] Separate a missing overload from a mistyped prefix, and name the right consumer Looking an identifier up without its signature is what lets a cref written as M: for a property be reported as a prefix mismatch rather than a bare lookup failure. The same fallback also matched a cref naming an overload that does not exist, because it shares its identifier, and therefore its prefix, with the overload that does: a cref for Run(System.String) against an emitted Run produced "uses prefix 'M:', but ... was emitted as 'M:'", advising a replacement of the prefix with itself and hiding that the build emitted no such member. Take the fallback only when the emitted prefixes differ from the one written. The header and the category table also named IntelliSense as what the 7.1.0 collapse harmed. IntelliSense reads the trimmed ref/ copy by design and was unaffected; lib/ feeds the pipeline that builds the Learn pages, and those are what lost every remark and example. Name that consumer, matching the finding messages and the package layout contract in documentation.instructions.md. The missing-documentation message named IntelliSense too, though it covers an assembly shipping no XML in either folder, so it now speaks of documentation text generally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 52a2ca5ee8abafd7aec9422b843b248d70805cf2) --- .../scripts/tests/validate-xml-docs.Tests.ps1 | 32 +++++++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 35 ++++++++++++++----- 2 files changed, 58 insertions(+), 9 deletions(-) diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index 6abbfc43a7..6b83a5fdca 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -559,6 +559,38 @@ Describe 'validate-xml-docs.ps1' { $finding.Message | Should -BeLike "*was emitted as 'P:'*" } + <# + An overload that was never emitted shares its prefix with the overload that was, so + matching on the identifier alone would advise replacing a prefix with itself. It is a + member this build does not contain, not a mistyped prefix. + #> + It 'reports an overload that was not emitted as a missing member' { + $docs = New-DocumentationDirectory ` + -Members @('M:Microsoft.Data.SqlClient.SqlCommand.ExecuteReader') ` + -Crefs @('M:Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(System.String)') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Category | Should -Be 'missing-local-uid' + } + + It 'still names the expected prefix when an overload carries the wrong member kind' { + $docs = New-DocumentationDirectory ` + -Members @('P:Microsoft.Data.SqlClient.SqlCommand.CommandTimeout') ` + -Crefs @('M:Microsoft.Data.SqlClient.SqlCommand.CommandTimeout(System.String)') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Category | Should -Be 'mismatched-docid-prefix' + $findings[0].Message | Should -BeLike "*was emitted as 'P:'*" + } + It 'does not resolve a namespace cref against the member index' { # The compiler never emits a entry for a namespace, so an N: cref must not be # treated as an unresolved reference. diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 5b1f8d9100..1b8d0c4347 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -30,10 +30,11 @@ Package mode additionally checks how documentation was mapped into the package. The driver ships two XML documentation files per target framework and they are required to differ: lib/ - carries the full text, while ref/ has and stripped, because those render - badly in Visual Studio IntelliSense. The two have been silently collapsed before -- in 7.1.0 - the modern lib/ XML was byte-identical to the trimmed ref/ XML, so IntelliSense lost every - remark and example. The lib/ref pairs are therefore compared against each other rather than + carries the full text for the .NET API docs pipeline that builds the Learn pages, while ref/ + has and stripped, because those render badly in Visual Studio IntelliSense, + which reads the ref/ copy. The two have been silently collapsed before -- in 7.1.0 the modern + lib/ XML was byte-identical to the trimmed ref/ XML, so the Learn pages lost every remark and + example. The lib/ref pairs are therefore compared against each other rather than against an absolute expectation, which keeps the rule meaningful for packages that legitimately have no ref/ folder at all. @@ -123,8 +124,8 @@ unknown-namespace-root error Leading identifier is not an allowed namespace root. stale-allowlist-entry error Allowlisted cref no longer appears in any validated file. lib-documentation-trimmed - error A package's lib/ XML has no remarks or examples, so - IntelliSense would show only summaries. + error A package's lib/ XML has no remarks or examples, so the + Learn API pages built from it would show only summaries. ref-documentation-untrimmed error A package's ref/ XML still carries remarks or examples. lib-ref-documentation-identical @@ -504,8 +505,23 @@ function Test-Cref { # as M: for a property, or T: for a member, names something real but produces a UID # that matches nothing. Say which prefix was expected rather than reporting a bare # lookup failure. - if ($script:LocalUidsByBody.ContainsKey($namePart) -or $script:LocalUidsByBody.ContainsKey($body)) { - $key = if ($script:LocalUidsByBody.ContainsKey($body)) { $body } else { $namePart } + # + # The identifier is looked up with its signature first and without it second. The + # second form explains a prefix mismatch only when the prefixes actually emitted + # differ from the one written, because a cref naming an overload that does not + # exist shares its prefix with the overload that does. Reporting that as a + # mismatch would advise replacing a prefix with itself and would hide a member + # this build never emitted. + $key = $null + if ($script:LocalUidsByBody.ContainsKey($body)) { + $key = $body + } + elseif ($script:LocalUidsByBody.ContainsKey($namePart) -and + -not $script:LocalUidsByBody[$namePart].Contains("${prefix}:")) { + $key = $namePart + } + + if ($null -ne $key) { $actual = ($script:LocalUidsByBody[$key] | Sort-Object) -join ', ' $message = "Cref '$trimmed' uses prefix '${prefix}:', but '$key' was emitted " + "as '$actual'. Use the prefix matching the member kind." @@ -1164,7 +1180,8 @@ if ($script:ExpandedPackageRoots.Count -gt 0) { Add-Finding -Category 'missing-documentation' -Path $assembly.FullName -Message ( "$packageName ships $folder/$($frameworkDirectory.Name)/$($assembly.Name) " + "without $assemblyName.xml, although its project generates XML " + - 'documentation. Consumers of this package get no IntelliSense text.') + 'documentation. Consumers of this package get no documentation ' + + 'text for it.') } } } From ca1034ade2cd2234b2558e75dcf4a991ff21c70d Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 11:01:52 -0300 Subject: [PATCH 14/23] Reject unbalanced braces around generic arguments The generic-syntax rule rejected angle brackets but said nothing about braces that do not pair up, so T:System.Collections.Generic.List{System.String passed with no findings despite naming nothing. Such a cref is read from the first brace to the last, so a missing delimiter quietly produces a different argument list than the text shows, or none at all, and the identifier reaches Learn unresolvable. Count the delimiters before the cref is taken apart, since the extraction that follows assumes they pair up. A closing brace with nothing open ends the scan, as nothing later can balance it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit f737219d767e4d6b77c989a8f7c1e86cc4097eae) --- .../scripts/tests/validate-xml-docs.Tests.ps1 | 29 +++++++++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 27 +++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index 6b83a5fdca..7c6688d5a8 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -334,6 +334,35 @@ Describe 'validate-xml-docs.ps1' { Should -Throw '*failed with 1 issue*' } + It 'rejects unbalanced generic argument braces' -ForEach @( + @{ Cref = 'T:System.Collections.Generic.List{System.String' } + @{ Cref = 'T:System.Collections.Generic.List}System.String{' } + @{ Cref = 'M:Microsoft.Data.SqlClient.Sample.Use(System.Collections.Generic.List{System.String)' } + ) { + $snippets = New-SnippetDirectory -Crefs @($Cref) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Category | Should -Be 'invalid-docid' + $findings[0].Message | Should -BeLike '*unbalanced braces*' + } + + It 'accepts balanced generic argument braces' -ForEach @( + @{ Cref = 'T:System.Collections.Generic.List{System.String}' } + @{ Cref = 'T:System.Collections.Generic.Dictionary{System.String,System.Collections.Generic.List{System.Int32}}' } + @{ Cref = 'M:Microsoft.Data.SqlClient.Sample.Use(System.Collections.Generic.List{System.String})' } + ) { + $snippets = New-SnippetDirectory -Crefs @($Cref) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + @((Get-Report -Path $report).Findings) | Should -BeNullOrEmpty + } + It 'rejects a multidimensional array type reference' -ForEach @( @{ Cref = 'T:System.Int32[,]' } @{ Cref = 'T:System.Int32[,,]' } diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 1b8d0c4347..51cc528e75 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -390,6 +390,33 @@ function Test-Cref { return } + # Those braces must pair up. An unmatched or misnested one leaves an identifier that names + # nothing, and it would otherwise survive: the generic argument list is read from the first + # brace to the last, so a missing delimiter silently yields a different set of arguments than + # the text suggests, or none at all. + $depth = 0 + foreach ($character in $body.ToCharArray()) { + if ($character -eq '{') { + $depth++ + } + elseif ($character -eq '}') { + $depth-- + + # A closing brace with nothing open cannot be balanced by anything later, and leaving + # the count negative reports it below. + if ($depth -lt 0) { + break + } + } + } + + if ($depth -ne 0) { + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' has unbalanced braces around its generic arguments. Documentation " + + 'IDs pair every { with a later }.') + return + } + $signatureStart = $body.IndexOf('(') $namePart = if ($signatureStart -ge 0) { $body.Substring(0, $signatureStart) } else { $body } From d4a79549cd4fcc44c258e70fa32d0e7c70d164ec Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 11:27:26 -0300 Subject: [PATCH 15/23] Validate the return type of a conversion operator Alias scanning covered the parameter list, and the name taken for the later checks ends at the '(', so the return type of a conversion operator was read by neither: M:System.Foo.op_Implicit(System.Int32)~string passed with no findings even though a documentation ID names CLR types throughout. Scan the return type with the parameters, and describe the finding as the signature rather than the parameter list, since the alias may now be in either. Nothing else may follow a parameter list, so text there that is not a return marker, or a marker naming no type, is reported rather than dropped. Both forms reached none of the surrounding checks, which read the name before the '(' and the arguments within it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 439575d0e6805c74cdc3f1f2229e14a850d8cbb9) --- .../scripts/tests/validate-xml-docs.Tests.ps1 | 65 +++++++++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 39 ++++++++++- 2 files changed, 101 insertions(+), 3 deletions(-) diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index 7c6688d5a8..97a4ed589d 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -403,6 +403,71 @@ Describe 'validate-xml-docs.ps1' { { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw } + <# + A conversion operator's return type sits after the parameter list, which is where the + alias scan stops and where the name taken from before the '(' has already ended. These + cover that the return type is scanned like any other part of the signature. + #> + It 'rejects a C# alias in a conversion operator return type' -ForEach @( + @{ Cref = 'M:Microsoft.Data.SqlTypes.SqlJson.op_Implicit(System.Int32)~string' } + @{ Cref = 'M:Microsoft.Data.SqlTypes.SqlJson.op_Explicit(System.Int32)~System.Collections.Generic.List{int}' } + ) { + $snippets = New-SnippetDirectory -Crefs @($Cref) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Category | Should -Be 'invalid-docid' + $findings[0].Message | Should -BeLike '*in its signature*' + } + + It 'reports the parameter and return aliases of one signature together' { + $snippets = New-SnippetDirectory -Crefs @( + 'M:Microsoft.Data.SqlTypes.SqlJson.op_Implicit(int)~string') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Message | Should -BeLike "*C# aliases 'int', 'string'*" + } + + It 'accepts a fully qualified generic conversion operator return type' { + $snippets = New-SnippetDirectory -Crefs @( + 'M:Microsoft.Data.SqlTypes.SqlJson.op_Explicit(System.Int32)~System.Collections.Generic.List{System.Int32}') + + { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw + } + + It 'rejects text after a parameter list that is not a return marker' { + $snippets = New-SnippetDirectory -Crefs @( + 'M:Microsoft.Data.SqlTypes.SqlJson.Parse(System.String)garbage') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Category | Should -Be 'invalid-docid' + $findings[0].Message | Should -BeLike '*unexpected text after its parameter list*' + } + + It 'rejects a return marker that names no type' { + $snippets = New-SnippetDirectory -Crefs @( + 'M:Microsoft.Data.SqlTypes.SqlJson.op_Implicit(System.String)~') + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -SnippetsDirectory $snippets -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Category | Should -Be 'invalid-docid' + $findings[0].Message | Should -BeLike '*names no return type*' + } + It 'does not fail on an unprefixed cref, which the compiler binds from source' { $snippets = New-SnippetDirectory -Crefs @('SqlJson', 'string', 'System.Text.Json.JsonDocument') $report = Join-Path (New-TestDirectory) 'report.json' diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 51cc528e75..def26cea73 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -447,10 +447,43 @@ function Test-Cref { return } + # Only a conversion operator may carry anything after its parameter list, written as the + # return marker ~ followed by a type. Anything else there is outside the grammar, and would + # otherwise go unexamined: the checks below read the name before the '(' and the arguments + # within it, so text beyond the ')' belongs to neither. + $returnType = $null + $trailing = $body.Substring($signatureEnd + 1) + if ($trailing.Length -gt 0) { + if ($trailing[0] -ne '~') { + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' has unexpected text after its parameter list. Only a " + + "conversion operator's return marker, written as '~' followed by a type, may " + + 'follow it.') + return + } + + $returnType = $trailing.Substring(1) + if ($returnType.Length -eq 0) { + Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( + "Cref '$trimmed' ends with a conversion operator return marker but names no " + + 'return type.') + return + } + } + # A signature may repeat the same alias (string and string[] both reduce to string), so - # report the distinct offenders once rather than once per parameter. - $aliases = [System.Collections.Generic.List[string]]::new() + # report the distinct offenders once rather than once per parameter. A conversion + # operator's return type is part of its signature, so it is scanned with the parameters. + $signatureTypes = [System.Collections.Generic.List[string]]::new() foreach ($argument in (Split-DocIdArguments -Arguments $arguments)) { + $signatureTypes.Add($argument) + } + if ($null -ne $returnType) { + $signatureTypes.Add($returnType) + } + + $aliases = [System.Collections.Generic.List[string]]::new() + foreach ($argument in $signatureTypes) { foreach ($alias in (Get-DocIdAlias -TypeName $argument)) { if (-not $aliases.Contains($alias)) { $aliases.Add($alias) @@ -461,7 +494,7 @@ function Test-Cref { $quoted = ($aliases | ForEach-Object { "'$_'" }) -join ', ' $noun = if ($aliases.Count -eq 1) { 'the C# alias' } else { 'the C# aliases' } Add-Finding @Context -Category 'invalid-docid' -Cref $Cref -Message ( - "Cref '$trimmed' uses $noun $quoted in its parameter list. Documentation IDs name " + + "Cref '$trimmed' uses $noun $quoted in its signature. Documentation IDs name " + 'CLR types, so use the full type name instead.') return } From 2ccec9215e52609e609c3c5ce3961dbb76653cdb Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 19:37:28 -0300 Subject: [PATCH 16/23] Resolve references into packages that were not built this run A NuGet package carries only its own XML documentation, so a cref from the SqlClient package into Microsoft.SqlServer.Server resolves against the SqlServer package in the drop. When SqlServer is not built there is no such package, and the packaged documentation gate reported 24 missing-public-uid errors for T:Microsoft.SqlServer.Server.SqlUserDefinedTypeAttribute -- a reference a consumer resolves perfectly well against the published package the build depends on. Build output does not have the problem, because a dependency's documentation file is copied next to the assembly that consumed it. Only the packaged gate sees a package in isolation, which is why this surfaced there alone. Add -DependencyDocumentationPath, whose documents are indexed for resolution but never themselves validated: a defect in an already-published package cannot be fixed by the run that reports it, and the crefs it contains point back into the version of this repository it shipped against. They are also kept out of the public API surface, so a member cannot count as public here on the strength of where it sits in a package this run did not produce. restore-package-documentation.ps1 collects that documentation from the exact version depended upon, restored through the repository's NuGet.config so it comes from the same governed feed as every other restore. The version is required to be exact, since a range would leave undetermined which version answered a reference. The job supplies it only when SqlServer was not built; when it was, resolution against a published copy would mask a reference the build itself broke. Verified against the packages from the failing run: the gate reproduces 29330 crefs across 13 files with 24 errors, and passes with 0 once the dependency documentation is supplied, every other count unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../onebranch/jobs/validate-packages-job.yml | 32 +++ .../scripts/restore-package-documentation.ps1 | 140 ++++++++++++ .../onebranch/scripts/tests/README.md | 2 + .../restore-package-documentation.Tests.ps1 | 215 ++++++++++++++++++ .../scripts/tests/validate-xml-docs.Tests.ps1 | 150 ++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 78 +++++-- .../steps/validate-xml-docs-step.yml | 10 + 7 files changed, 612 insertions(+), 15 deletions(-) create mode 100644 eng/pipelines/onebranch/scripts/restore-package-documentation.ps1 create mode 100644 eng/pipelines/onebranch/scripts/tests/restore-package-documentation.Tests.ps1 diff --git a/eng/pipelines/onebranch/jobs/validate-packages-job.yml b/eng/pipelines/onebranch/jobs/validate-packages-job.yml index 3fd6cf929c..c3891369e1 100644 --- a/eng/pipelines/onebranch/jobs/validate-packages-job.yml +++ b/eng/pipelines/onebranch/jobs/validate-packages-job.yml @@ -108,6 +108,12 @@ jobs: - name: xmlDocsExtractRoot value: '$(Pipeline.Workspace)/validate-xml-docs-extract' + # Documentation of packages this run depends on but did not build, collected so that the + # packaged XML documentation gate can resolve references into them. Unused when every + # referenced package was built this run. + - name: xmlDocsDependencyRoot + value: '$(Pipeline.Workspace)/validate-xml-docs-dependencies' + steps: - template: /eng/pipelines/onebranch/steps/script-output-environment-variables-step.yml@self @@ -196,6 +202,26 @@ jobs: # Validate the XML documentation as a consumer receives it. Only the assembled package shows # which XML actually landed in each lib/ and ref/ target framework, a mapping that has # silently changed before without any earlier check noticing. + # + # A package carries only its own documentation, so a cref from the SqlClient package into + # Microsoft.SqlServer.Server resolves against the SqlServer package in the drop. When + # SqlServer was not built there is no such package, and those references would be reported as + # unresolvable even though a consumer resolves them perfectly well against the published + # package the build depends on. Collect that package's documentation so the gate judges the + # same thing a consumer would. + - ${{ if eq(parameters.buildSqlServer, false) }}: + - task: PowerShell@2 + displayName: 'Collect dependency XML documentation - SqlServer' + inputs: + targetType: filePath + pwsh: true + filePath: $(REPO_ROOT)/eng/pipelines/onebranch/scripts/restore-package-documentation.ps1 + arguments: >- + -PackageId "Microsoft.SqlServer.Server" + -Version "${{ parameters.sqlServerPackageVersion }}" + -DestinationPath "$(xmlDocsDependencyRoot)" + -ConfigFile "$(REPO_ROOT)/NuGet.config" + - template: /eng/pipelines/onebranch/steps/validate-xml-docs-step.yml@self parameters: displayName: 'Validate packaged XML documentation' @@ -203,6 +229,12 @@ jobs: documentationPath: '' packagesPath: '$(packagesRoot)' extractPath: '$(xmlDocsExtractRoot)' + # Empty when SqlServer was built this run: its package is then in the drop, and + # resolution against a published copy would mask a reference the build itself broke. + ${{ if eq(parameters.buildSqlServer, true) }}: + dependencyDocumentationPath: '' + ${{ else }}: + dependencyDocumentationPath: '$(xmlDocsDependencyRoot)' reportPath: '$(JOB_OUTPUT)/validation/xml-docs-packages.json' failOn: - error diff --git a/eng/pipelines/onebranch/scripts/restore-package-documentation.ps1 b/eng/pipelines/onebranch/scripts/restore-package-documentation.ps1 new file mode 100644 index 0000000000..86b7e65d49 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/restore-package-documentation.ps1 @@ -0,0 +1,140 @@ +################################################################################# +# Licensed to the .NET Foundation under one or more agreements. # +# The .NET Foundation licenses this file to you under the MIT license. # +# See the LICENSE file in the project root for more information. # +################################################################################# + +<# +.SYNOPSIS + Restores a published package and collects the XML documentation it ships. + +.DESCRIPTION + A NuGet package carries only its own XML documentation. A cref from one package into a sibling + package therefore resolves against nothing when that sibling was not built in the same run, + which is the state the pipeline is in whenever it depends on a published package instead of + building it. + + This collects the documentation of that published dependency so the packaged documentation gate + can resolve against it, using the very version the packages under validation depend on. + + The restore is deliberately driven through the repository's NuGet.config so the package comes + from the same governed feed every other restore in the build uses, rather than from an + arbitrary source. + +.PARAMETER PackageId + Package to restore, for example Microsoft.SqlServer.Server. + +.PARAMETER Version + Exact version to restore. A floating or range notation is rejected: the point of this step is + to resolve against one known version, and a range would leave the version that answered a + reference undetermined. + +.PARAMETER DestinationPath + Directory the documentation is collected into, one subdirectory per target framework. Replaced + if it already exists, so a rerun cannot mix versions. + +.PARAMETER ConfigFile + NuGet.config governing the restore. Relative sources inside it resolve against its own + directory, so it may live outside the working directory. + +.PARAMETER TargetFramework + Target framework of the throwaway project used to drive the restore. Only affects which + dependency graph NuGet walks; documentation is collected from every framework the package + ships, not just this one. +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory)][string]$PackageId, + + [Parameter(Mandatory)][string]$Version, + + [Parameter(Mandatory)][string]$DestinationPath, + + [string]$ConfigFile, + + [string]$TargetFramework = 'netstandard2.0' +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' + +if ($Version -match '[\*\[\]\(\),]') { + throw "Version '$Version' is not an exact version. Supply the single version to resolve against." +} + +if (-not [string]::IsNullOrWhiteSpace($ConfigFile) -and -not (Test-Path -LiteralPath $ConfigFile -PathType Leaf)) { + throw "NuGet configuration file '$ConfigFile' was not found." +} + +# Built beneath the destination rather than in TEMP so that everything this step produces is +# removed together, and a rerun cannot read a package folder left behind by an earlier version. +if (Test-Path -LiteralPath $DestinationPath) { + Remove-Item -LiteralPath $DestinationPath -Recurse -Force +} +New-Item -ItemType Directory -Force -Path $DestinationPath | Out-Null + +$workingPath = Join-Path $DestinationPath '.restore' +New-Item -ItemType Directory -Force -Path $workingPath | Out-Null + +$packagesPath = Join-Path $workingPath 'packages' +$projectPath = Join-Path $workingPath 'DependencyDocumentation.csproj' + +# ManagePackageVersionsCentrally is disabled explicitly because this project carries its own +# version: were it to pick up a Directory.Packages.props, a version here would be an error. +@" + + + $TargetFramework + false + + + + + +"@ | Set-Content -LiteralPath $projectPath -Encoding utf8 + +$arguments = @('restore', $projectPath, '--packages', $packagesPath) +if (-not [string]::IsNullOrWhiteSpace($ConfigFile)) { + $arguments += @('--configfile', $ConfigFile) +} + +Write-Host "Restoring $PackageId $Version to collect its XML documentation." +& dotnet @arguments +if ($LASTEXITCODE -ne 0) { + throw "Restore of $PackageId $Version failed with exit code $LASTEXITCODE." +} + +# NuGet lowercases the identifier and version on disk, so the folder is located by search rather +# than by assuming a casing convention. +$packageRoot = Get-ChildItem -LiteralPath $packagesPath -Directory -ErrorAction SilentlyContinue | + Where-Object { $_.Name -eq $PackageId.ToLowerInvariant() } | + Select-Object -First 1 + +if ($null -eq $packageRoot) { + throw "Restore reported success but no folder for $PackageId was found under '$packagesPath'." +} + +# Only the package's own documentation is collected. Its dependencies were restored too, and +# indexing their members would resolve references against packages this build does not depend on. +$documentation = @(Get-ChildItem -LiteralPath $packageRoot.FullName -Recurse -File -Filter "$PackageId.xml") + +if ($documentation.Count -eq 0) { + throw "$PackageId $Version ships no XML documentation, so there is nothing to resolve against." +} + +foreach ($file in $documentation) { + # Keyed by the containing framework folder so that two frameworks shipping the same file name + # cannot overwrite one another. + $frameworkName = $file.Directory.Name + $frameworkPath = Join-Path $DestinationPath $frameworkName + New-Item -ItemType Directory -Force -Path $frameworkPath | Out-Null + Copy-Item -LiteralPath $file.FullName -Destination (Join-Path $frameworkPath $file.Name) -Force +} + +# The restore tree is large and is not an input to anything downstream; only the collected +# documentation is. Removing it also keeps the scan that follows from walking the dependencies. +Remove-Item -LiteralPath $workingPath -Recurse -Force + +Write-Host ("Collected $($documentation.Count) XML documentation file(s) for $PackageId $Version " + + "into '$DestinationPath'.") diff --git a/eng/pipelines/onebranch/scripts/tests/README.md b/eng/pipelines/onebranch/scripts/tests/README.md index 783bb5aee0..dae2979060 100644 --- a/eng/pipelines/onebranch/scripts/tests/README.md +++ b/eng/pipelines/onebranch/scripts/tests/README.md @@ -41,6 +41,8 @@ Invoke-Pester ./publish-symbols.Tests.ps1 -Output Detailed | Package validation | Wildcard vs per-id version expectations, SqlServer omitted when unbuilt, gate tokens, report written before gating, exit-code handling, report-only mode suppressing the gate but not a broken validator | | XML docs validation | Every documentation-ID defect form reported by the API Docs build (array `T:` UIDs, empty parentheses, C# aliases, embedded whitespace, misspelled namespace roots) plus valid controls, compiler-unresolved `!:` crefs, local UID and wrong-prefix resolution, package expansion, allowlisting and staleness, report-only mode | | XML docs lib/ref layout | Full `lib/` XML paired with trimmed `ref/` XML accepted; trimmed `lib/`, untrimmed `ref/`, and byte-identical `lib`/`ref` rejected; per-target-framework isolation; packages without a `ref/` folder ignored | +| XML docs dependencies | References into a package that was not built this run resolve against the published dependency's documentation; that documentation is not itself validated, does not establish the public API surface, and does not enable resolution when nothing is under validation | +| Dependency documentation restore | Exact version pinned, central package management not inherited, documentation collected per target framework, restore tree removed, stale destination replaced, and failures reported for a failed restore, a missing package folder, or a package shipping no documentation | | Package signatures | Every package and symbol package verified, all failures reported before throwing | | Assembly signatures | Package expansion, native binaries under `runtimes/` included, stale expansions replaced, all unsigned assemblies reported | diff --git a/eng/pipelines/onebranch/scripts/tests/restore-package-documentation.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/restore-package-documentation.Tests.ps1 new file mode 100644 index 0000000000..29b7aef2ec --- /dev/null +++ b/eng/pipelines/onebranch/scripts/tests/restore-package-documentation.Tests.ps1 @@ -0,0 +1,215 @@ +<# +.SYNOPSIS + Pester tests for restore-package-documentation.ps1. + +.DESCRIPTION + The script exists so the packaged XML documentation gate can resolve references into a package + this run depends on but did not build. What matters is that it collects exactly that package's + documentation, at exactly the requested version, and leaves nothing behind that a later run + could mistake for the current one. + + dotnet is mocked throughout, so no network access or feed credentials are required. The mock + writes the package layout NuGet would have produced, which is what the collection logic reads. +#> + +BeforeAll { + $script:repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..' '..' '..' '..' '..')).Path + $script:scriptPath = Join-Path $script:repoRoot 'eng/pipelines/onebranch/scripts/restore-package-documentation.ps1' + + function New-TestDirectory { + $path = Join-Path $TestDrive ([guid]::NewGuid().ToString('n')) + New-Item -ItemType Directory -Path $path | Out-Null + return $path + } + + # Locates --packages in the argument list the script built, so the mock writes where the script + # will look rather than where the test guessed it would. + function Get-PackagesPath { + param([Parameter(Mandatory)][string[]]$Arguments) + + $index = [array]::IndexOf($Arguments, '--packages') + if ($index -lt 0) { + throw 'The script did not pass --packages to dotnet.' + } + return $Arguments[$index + 1] + } + + function New-RestoredPackage { + param( + [Parameter(Mandatory)][string]$PackagesPath, + [Parameter(Mandatory)][string]$PackageId, + [Parameter(Mandatory)][string]$Version, + [string[]]$Frameworks = @('netstandard2.0', 'net46'), + [switch]$WithoutDocumentation + ) + + # NuGet lowercases both identifier and version on disk. + $root = Join-Path $PackagesPath ($PackageId.ToLowerInvariant() + '/' + $Version.ToLowerInvariant()) + foreach ($framework in $Frameworks) { + $libPath = Join-Path $root "lib/$framework" + New-Item -ItemType Directory -Path $libPath -Force | Out-Null + Set-Content -LiteralPath (Join-Path $libPath "$PackageId.dll") -Value 'binary' -Encoding utf8 + if (-not $WithoutDocumentation) { + Set-Content -LiteralPath (Join-Path $libPath "$PackageId.xml") ` + -Value "" ` + -Encoding utf8 + } + } + } +} + +Describe 'restore-package-documentation.ps1' { + + Context 'argument validation' { + + It 'rejects a version that is not exact' -ForEach @( + @{ Version = '1.*' } + @{ Version = '[1.0.0,2.0.0)' } + @{ Version = '1.0.0,2.0.0' } + ) { + # A range would leave undetermined which version answered a reference, which is the one + # thing this step exists to pin down. + { & $script:scriptPath -PackageId 'Contoso.Widget' -Version $Version ` + -DestinationPath (New-TestDirectory) } | + Should -Throw '*is not an exact version*' + } + + It 'rejects a NuGet configuration file that does not exist' { + { & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' ` + -DestinationPath (New-TestDirectory) ` + -ConfigFile (Join-Path (New-TestDirectory) 'absent.config') } | + Should -Throw '*was not found*' + } + } + + Context 'collection' { + + BeforeEach { + Mock dotnet { + $packagesPath = Get-PackagesPath -Arguments $args + New-RestoredPackage -PackagesPath $packagesPath -PackageId 'Contoso.Widget' -Version '1.0.0' + $global:LASTEXITCODE = 0 + } + } + + It 'collects the documentation of every framework the package ships' { + $destination = New-TestDirectory + + & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' -DestinationPath $destination + + Test-Path -LiteralPath (Join-Path $destination 'netstandard2.0/Contoso.Widget.xml') | Should -BeTrue + Test-Path -LiteralPath (Join-Path $destination 'net46/Contoso.Widget.xml') | Should -BeTrue + } + + It 'keeps each framework separate so identically named files cannot collide' { + $destination = New-TestDirectory + + & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' -DestinationPath $destination + + @(Get-ChildItem -LiteralPath $destination -Recurse -File -Filter '*.xml').Count | Should -Be 2 + } + + It 'collects documentation only, not the assemblies beside it' { + $destination = New-TestDirectory + + & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' -DestinationPath $destination + + @(Get-ChildItem -LiteralPath $destination -Recurse -File -Filter '*.dll') | Should -BeNullOrEmpty + } + + It 'removes the restore tree, leaving only the collected documentation' { + # The restore tree holds the dependencies too; leaving it would let the scan that + # follows index members from packages this build does not depend on. + $destination = New-TestDirectory + + & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' -DestinationPath $destination + + Test-Path -LiteralPath (Join-Path $destination '.restore') | Should -BeFalse + } + + It 'replaces an existing destination rather than mixing versions into it' { + $destination = New-TestDirectory + New-Item -ItemType Directory -Path (Join-Path $destination 'netstandard2.0') -Force | Out-Null + Set-Content -LiteralPath (Join-Path $destination 'netstandard2.0/Stale.xml') ` + -Value '' -Encoding utf8 + + & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' -DestinationPath $destination + + Test-Path -LiteralPath (Join-Path $destination 'netstandard2.0/Stale.xml') | Should -BeFalse + } + + It 'requests the exact version rather than a minimum' { + # A bare version is a minimum in NuGet, which would silently resolve to a newer package + # than the one the build depends on. Captured to a file inside the mock, because the + # project is deleted with the restore tree before the script returns, and a mock runs + # in its own scope so a variable would not survive either. + $capture = Join-Path (New-TestDirectory) 'captured.csproj' + $env:RESTORE_DOCS_TEST_CAPTURE = $capture + Mock dotnet { + Copy-Item -LiteralPath $args[1] -Destination $env:RESTORE_DOCS_TEST_CAPTURE -Force + New-RestoredPackage -PackagesPath (Get-PackagesPath -Arguments $args) ` + -PackageId 'Contoso.Widget' -Version '1.0.0' + $global:LASTEXITCODE = 0 + } + + try { + & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' -DestinationPath (New-TestDirectory) + + $projectContent = Get-Content -LiteralPath $capture -Raw + $projectContent | Should -Match '"Contoso\.Widget" Version="\[1\.0\.0\]"' + # Carrying its own version is an error under a Directory.Packages.props that + # happens to sit above wherever the destination was placed. + $projectContent | Should -Match 'false' + } + finally { + Remove-Item Env:\RESTORE_DOCS_TEST_CAPTURE -ErrorAction SilentlyContinue + } + } + + It 'passes the configuration file through when one is supplied' { + $config = Join-Path (New-TestDirectory) 'NuGet.config' + Set-Content -LiteralPath $config -Value '' -Encoding utf8 + + & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' ` + -DestinationPath (New-TestDirectory) -ConfigFile $config + + Should -Invoke dotnet -Times 1 -Exactly -ParameterFilter { + $args -contains '--configfile' -and $args -contains $config + } + } + } + + Context 'failure' { + + It 'throws when the restore fails' { + Mock dotnet { $global:LASTEXITCODE = 1 } + + { & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' ` + -DestinationPath (New-TestDirectory) } | + Should -Throw '*failed with exit code 1*' + } + + It 'throws when the restore succeeds but produces no package folder' { + Mock dotnet { $global:LASTEXITCODE = 0 } + + { & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' ` + -DestinationPath (New-TestDirectory) } | + Should -Throw '*no folder for Contoso.Widget was found*' + } + + It 'throws when the package ships no XML documentation' { + # Silence here would produce an empty directory and a gate that resolves nothing, + # reporting the very references the step was added to resolve. + Mock dotnet { + $packagesPath = Get-PackagesPath -Arguments $args + New-RestoredPackage -PackagesPath $packagesPath -PackageId 'Contoso.Widget' ` + -Version '1.0.0' -WithoutDocumentation + $global:LASTEXITCODE = 0 + } + + { & $script:scriptPath -PackageId 'Contoso.Widget' -Version '1.0.0' ` + -DestinationPath (New-TestDirectory) } | + Should -Throw '*ships no XML documentation*' + } + } +} diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index 97a4ed589d..aac27d9df0 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -1162,6 +1162,156 @@ Describe 'validate-xml-docs.ps1' { } } + Context 'dependency documentation' { + + BeforeAll { + <# + Builds the shape the packaged gate hits when a sibling package was not built this + run: a public member referencing a type that lives in another package of this + repository. Written as a package so the ref/ folder establishes the public API + surface, which is what makes the unresolved reference an error rather than + information. + #> + function New-PackageReferencingSibling { + $staging = New-TestDirectory + New-Item -ItemType Directory -Path (Join-Path $staging 'lib/net8.0') -Force | Out-Null + New-Item -ItemType Directory -Path (Join-Path $staging 'ref/net8.0') -Force | Out-Null + + 'W.' + + 'R.' | + Set-Content -LiteralPath (Join-Path $staging 'lib/net8.0/Microsoft.Data.SqlClient.xml') -Encoding utf8 + 'W.' + + '' | + Set-Content -LiteralPath (Join-Path $staging 'ref/net8.0/Microsoft.Data.SqlClient.xml') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory($staging, (Join-Path $packages 'P.1.0.0.nupkg')) + return $packages + } + + function New-SiblingDocumentation { + param([string]$Member = 'T:Microsoft.SqlServer.Server.Sibling') + + $path = New-TestDirectory + "S." | + Set-Content -LiteralPath (Join-Path $path 'Microsoft.SqlServer.Server.xml') -Encoding utf8 + return $path + } + } + + It 'reports a cref into a package that was not built without dependency documentation' { + # Establishes the defect the parameter exists to address, so the test below is shown to + # be suppressing a real finding rather than passing for an unrelated reason. + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath (New-PackageReferencingSibling) -ExtractPath (New-TestDirectory) ` + -ReportPath $report -ReportOnly + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Cref -eq 'T:Microsoft.SqlServer.Server.Sibling' })[0] + $finding.Category | Should -Be 'missing-public-uid' + } + + It 'resolves that cref against the dependency documentation' { + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -PackagesPath (New-PackageReferencingSibling) -ExtractPath (New-TestDirectory) ` + -DependencyDocumentationPath (New-SiblingDocumentation) -ReportPath $report } | + Should -Not -Throw + + @((Get-Report -Path $report).Findings.Cref) | Should -Not -Contain 'T:Microsoft.SqlServer.Server.Sibling' + } + + It 'still reports a cref the dependency documentation does not contain' { + # The parameter supplies members to resolve against; it does not exempt the namespace. + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath (New-PackageReferencingSibling) -ExtractPath (New-TestDirectory) ` + -DependencyDocumentationPath (New-SiblingDocumentation -Member 'T:Microsoft.SqlServer.Server.Other') ` + -ReportPath $report -ReportOnly + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Cref -eq 'T:Microsoft.SqlServer.Server.Sibling' })[0] + $finding.Category | Should -Be 'missing-public-uid' + } + + It 'does not validate the dependency documentation itself' { + # A defect in an already-published package cannot be fixed by the run that reports it. + $dependency = New-TestDirectory + 'S.' + + '' | + Set-Content -LiteralPath (Join-Path $dependency 'Microsoft.SqlServer.Server.xml') -Encoding utf8 + $report = Join-Path (New-TestDirectory) 'report.json' + + # T:System.Byte[] is an invalid-docid, and would be an error had the file been validated. + { & $scriptPath -PackagesPath (New-PackageReferencingSibling) -ExtractPath (New-TestDirectory) ` + -DependencyDocumentationPath $dependency -ReportPath $report } | Should -Not -Throw + + $result = Get-Report -Path $report + @($result.Findings.Cref) | Should -Not -Contain 'T:System.Byte[]' + $result.DependencyDocumentationFiles | Should -Be 1 + # Counted separately, so the summary still describes only what was validated. + $result.FilesValidated | Should -Be 2 + } + + It 'does not let dependency documentation establish the public API surface' { + # Public classification must describe this build's surface. A member is public because + # this run emitted it into ref/, never because a dependency package did. + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('T:Microsoft.Data.SqlClient.DoesNotExist') + $dependency = New-TestDirectory + New-Item -ItemType Directory -Path (Join-Path $dependency 'ref/net8.0') -Force | Out-Null + 'S.' | + Set-Content -LiteralPath (Join-Path $dependency 'ref/net8.0/Other.xml') -Encoding utf8 + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -DependencyDocumentationPath $dependency -ReportPath $report + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Cref -eq 'T:Microsoft.Data.SqlClient.DoesNotExist' })[0] + $finding.Category | Should -Be 'missing-local-uid' + $finding.Severity | Should -Be 'info' + } + + It 'resolves a namespace cref against the dependency documentation' { + # A namespace has no member entry of its own, so it resolves against the namespaces the + # indexed members occupy -- which must include those the dependencies contribute. + $docs = New-DocumentationDirectory ` + -Members @('T:Microsoft.Data.SqlClient.SqlConnection') ` + -Crefs @('N:Microsoft.SqlServer.Server') + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -DocumentationPath $docs ` + -DependencyDocumentationPath (New-SiblingDocumentation) -ReportPath $report } | + Should -Not -Throw + + @((Get-Report -Path $report).Findings.Cref) | Should -Not -Contain 'N:Microsoft.SqlServer.Server' + } + + It 'fails when the dependency documentation path does not exist' { + $docs = New-DocumentationDirectory + + { & $scriptPath -DocumentationPath $docs ` + -DependencyDocumentationPath (Join-Path (New-TestDirectory) 'absent') } | + Should -Throw '*Dependency documentation path*was not found*' + } + + It 'does not resolve against dependencies when nothing is under validation' { + # An index built from dependencies alone would describe members this build never + # emitted, so supplying them cannot by itself enable local resolution. + $snippets = New-SnippetDirectory -Crefs @('T:Microsoft.SqlServer.Server.Sibling') + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -SnippetsDirectory $snippets ` + -DependencyDocumentationPath (New-SiblingDocumentation) -ReportPath $report } | + Should -Not -Throw + + (Get-Report -Path $report).DocumentationFiles | Should -Be 0 + } + } + Context 'allowlist' { It 'exempts an allowlisted cref' { diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index def26cea73..fdd9fb2036 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -51,6 +51,19 @@ and the XML documentation inside it is validated, which is what a consumer actually receives. Requires -ExtractPath. +.PARAMETER DependencyDocumentationPath + Optional XML documentation files, or directories scanned recursively for them, supplied only so + that references resolve against them. Their own contents are never validated. + + A package carries only its own documentation, so a cref from one package into a sibling package + resolves against nothing whenever that sibling was not built in the same run -- the state a + pipeline is in when it depends on the published version of a package instead of building it. + Build output does not have this problem, because a dependency's documentation file is copied + next to the assembly that consumed it. + + Supply the documentation from the dependency version the packages under validation actually + declare, so that what resolves here is what a consumer will resolve against. + .PARAMETER ExtractPath Directory that -PackagesPath expands into. Existing expansions are replaced so a rerun cannot validate stale content. @@ -181,6 +194,8 @@ param( [string[]]$DocumentationPath, + [string[]]$DependencyDocumentationPath, + [string]$PackagesPath, [string]$ExtractPath, @@ -961,6 +976,16 @@ if (-not [string]::IsNullOrWhiteSpace($PackagesPath)) { $documentationCandidates = @(Resolve-InputPaths -Paths $documentationRoots -Description 'Documentation') +# Documentation supplied purely to resolve references against, never itself validated. A package +# carries only its own documentation, so a cref from one package into a sibling package resolves +# against nothing when that sibling was not built this run. Build output does not have the problem, +# because a dependency's documentation file is copied next to the assembly that consumed it. +# +# Resolution only, because these members belong to an already-published package: a defect found in +# them could not be fixed by the run that reports it, and the crefs they contain point back into +# the version of this repository they shipped against rather than the one being built. +$dependencyCandidates = @(Resolve-InputPaths -Paths $DependencyDocumentationPath -Description 'Dependency documentation') + # Distinguish "nothing was asked for", which is a caller error, from "what was asked for held no # documentation", which is reported below as a finding. $snippetsRequested = @(@($SnippetsDirectory) | Where-Object { -not [string]::IsNullOrWhiteSpace($_) }).Count -gt 0 @@ -973,10 +998,12 @@ if (-not $snippetsRequested -and -not $documentationRequested) { # Parse every file up front so that a malformed file is reported as a finding rather than aborting # the run, and so the local UID index is complete before any cref is resolved. $documents = [System.Collections.Generic.List[object]]::new() -$malformedByKind = @{ snippet = 0; documentation = 0 } +$dependencyDocuments = [System.Collections.Generic.List[object]]::new() +$malformedByKind = @{ snippet = 0; documentation = 0; dependency = 0 } foreach ($entry in @( @{ Files = $snippetFiles; Kind = 'snippet' }, - @{ Files = $documentationCandidates; Kind = 'documentation' })) { + @{ Files = $documentationCandidates; Kind = 'documentation' }, + @{ Files = $dependencyCandidates; Kind = 'dependency' })) { foreach ($file in $entry.Files) { try { @@ -992,7 +1019,7 @@ foreach ($entry in @( # A scanned directory may hold far more .xml files than documentation. Recognize # documentation by its shape and silently skip everything else, so a whole # output directory can be supplied without pre-filtering it. - if ($entry.Kind -eq 'documentation') { + if ($entry.Kind -ne 'snippet') { if ($null -eq $document.Root -or $document.Root.Name.LocalName -ne 'doc' -or $null -eq $document.Root.Element('members')) { @@ -1000,22 +1027,38 @@ foreach ($entry in @( } } - $documents.Add([pscustomobject]@{ - Path = $file - Kind = $entry.Kind - Document = $document - # Documentation shipped in a package's ref/ folder describes the reference - # assembly, whose members are exactly the public API surface. - IsReferenceDocumentation = ($entry.Kind -eq 'documentation') -and - (Test-IsReferenceDocumentationPath -Path $file) - RemarksCount = @($document.Descendants('remarks')).Count - ExampleCount = @($document.Descendants('example')).Count - }) + $record = [pscustomobject]@{ + Path = $file + Kind = $entry.Kind + Document = $document + # Documentation shipped in a package's ref/ folder describes the reference + # assembly, whose members are exactly the public API surface. + # + # Never set for a dependency: the public API surface being judged is this build's, and + # admitting another package's would let a member count as public here on the strength + # of where it sits in a package this run did not produce. + IsReferenceDocumentation = ($entry.Kind -eq 'documentation') -and + (Test-IsReferenceDocumentationPath -Path $file) + RemarksCount = @($document.Descendants('remarks')).Count + ExampleCount = @($document.Descendants('example')).Count + } + + if ($entry.Kind -eq 'dependency') { + $dependencyDocuments.Add($record) + } + else { + $documents.Add($record) + } } } $documentationDocuments = @($documents | Where-Object { $_.Kind -eq 'documentation' }) +if ($dependencyDocuments.Count -gt 0) { + Write-Host ("Resolving against $($dependencyDocuments.Count) dependency documentation file(s), " + + 'which are not themselves validated.') +} + # Whether documentation must exist is the caller's declaration, not an inference. Asserting it both # ways is what keeps the declaration honest: documentation vanishing from a package that should # have it is an error, and documentation appearing in one that should not have it means the @@ -1086,7 +1129,11 @@ $script:LocalNamespaces = $null if ($documentationDocuments.Count -gt 0) { $script:LocalUids = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) $script:LocalNamespaces = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) - foreach ($entry in $documentationDocuments) { + # Dependency documentation is indexed alongside, so a cref into a package this run did not + # build resolves. Gated on documentation being present rather than on the dependencies: with + # nothing under validation there is no cref to resolve, and an index built from dependencies + # alone would describe members this build never emitted. + foreach ($entry in @($documentationDocuments) + @($dependencyDocuments)) { foreach ($member in $entry.Document.Root.Element('members').Elements('member')) { $name = $member.Attribute('name') if ($null -eq $name -or [string]::IsNullOrWhiteSpace($name.Value)) { @@ -1346,6 +1393,7 @@ if (-not [string]::IsNullOrWhiteSpace($ReportPath)) { FilesValidated = $documents.Count SnippetFiles = @($documents | Where-Object { $_.Kind -eq 'snippet' }).Count DocumentationFiles = $documentationDocuments.Count + DependencyDocumentationFiles = $dependencyDocuments.Count CrefsValidated = $crefCount LocalUids = if ($null -ne $script:LocalUids) { $script:LocalUids.Count } else { 0 } CountsByCategory = $countsByCategory diff --git a/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml b/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml index d7a197c883..46d79f24fd 100644 --- a/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml +++ b/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml @@ -47,6 +47,15 @@ parameters: - name: extractPath type: string + # XML documentation supplied only so that references resolve against it; it is never itself + # validated. A package carries only its own documentation, so a cref from one package into a + # sibling package resolves against nothing when that sibling was not built this run. Pass the + # documentation of the dependency version the packages under validation declare. Pass '' when + # every referenced package was built in the same run. + - name: dependencyDocumentationPath + type: string + default: '' + # Path of the JSON report to write. Written before gating, so it survives a failing run. - name: reportPath type: string @@ -107,6 +116,7 @@ steps: ' -DocumentationPath "${{ parameters.documentationPath }}"' + ' -PackagesPath "${{ parameters.packagesPath }}"' + ' -ExtractPath "${{ parameters.extractPath }}"' + + ' -DependencyDocumentationPath "${{ parameters.dependencyDocumentationPath }}"' + ' -ProjectPath "${{ parameters.projectPath }}"' + ' -ProjectSearchRoot "${{ parameters.projectSearchRoot }}"' From 1d558204ae33103e2dac28efcb53af2ddae3a756 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 20:31:22 -0300 Subject: [PATCH 17/23] Detect enum field remarks the documentation build discards The API docs build for 7.1 reported ECMA2Yaml_Enum_NoRemarks against RegisteredApplication: on an enum field is ignored, so only the summary is rendered. The note explaining why sqlpackage reports its own identifier rather than the Data-Tier Application Framework's reached no page, and nothing in our own build said so. Fold that text into the field's summary, where it publishes. Add an enum-field-remarks rule so the next one is caught here rather than in another repository's validation run. Nothing in a snippet or in generated documentation says which types are enums, so the enum declarations in source are read for the answer. Their member names are read too, because a snippet identifies a member only by its element name and may hold more than one block for the same type: documents the type for another platform, and asking whether the name is a declared member is what tells it apart from a field without guessing from the name. In generated documentation the rule is confined to the public API surface. The implementation assembly documents its own P/Invoke enums -- Interop.Windows .NtDll.CreateOptions and the like -- whose remarks are discarded just the same, but which reach no published page for anything to be lost from. Where no reference documentation identifies the surface the rule stays silent, matching how every other rule here treats an unknown surface. Verified against the packages from the failing run: the rule reports the one public field, once per target framework, and none of the internal ones. The other finding on that run, an xref-not-found for a misspelled 'Microssoft.Data.SqlClient.SqlCommand.EnableOptimizedParameterBinding', needs no change here. Our snippet was corrected by #2889 and the ingestion under review is removing the stale text; the shipped 7.1.0 package contains no occurrence. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../RegisteredApplication.xml | 7 +- .../onebranch/scripts/tests/README.md | 1 + .../scripts/tests/validate-xml-docs.Tests.ps1 | 220 ++++++++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 215 +++++++++++++++++ 4 files changed, 438 insertions(+), 5 deletions(-) diff --git a/doc/snippets/Microsoft.Data.SqlClient/RegisteredApplication.xml b/doc/snippets/Microsoft.Data.SqlClient/RegisteredApplication.xml index 58dc683dcb..bf9609fdb3 100644 --- a/doc/snippets/Microsoft.Data.SqlClient/RegisteredApplication.xml +++ b/doc/snippets/Microsoft.Data.SqlClient/RegisteredApplication.xml @@ -125,15 +125,12 @@ - The sqlpackage command-line tool. + The sqlpackage command-line tool. It is built on the Data-Tier Application Framework, but reports its + own identifier so that command-line use can be told apart from other callers of that framework. 12 - - sqlpackage is built on the Data-Tier Application Framework, but reports its own identifier so that - command-line use can be told apart from other callers of that framework. - diff --git a/eng/pipelines/onebranch/scripts/tests/README.md b/eng/pipelines/onebranch/scripts/tests/README.md index dae2979060..9abc2b46b5 100644 --- a/eng/pipelines/onebranch/scripts/tests/README.md +++ b/eng/pipelines/onebranch/scripts/tests/README.md @@ -42,6 +42,7 @@ Invoke-Pester ./publish-symbols.Tests.ps1 -Output Detailed | XML docs validation | Every documentation-ID defect form reported by the API Docs build (array `T:` UIDs, empty parentheses, C# aliases, embedded whitespace, misspelled namespace roots) plus valid controls, compiler-unresolved `!:` crefs, local UID and wrong-prefix resolution, package expansion, allowlisting and staleness, report-only mode | | XML docs lib/ref layout | Full `lib/` XML paired with trimmed `ref/` XML accepted; trimmed `lib/`, untrimmed `ref/`, and byte-identical `lib`/`ref` rejected; per-target-framework isolation; packages without a `ref/` folder ignored | | XML docs dependencies | References into a package that was not built this run resolve against the published dependency's documentation; that documentation is not itself validated, does not establish the public API surface, and does not enable resolution when nothing is under validation | +| XML docs enum remarks | `` on an enum field reported, since the documentation build discards it; type blocks and platform-variant type blocks accepted; members of non-enum types accepted; enum members read from source through attributes, initializers and comments; in generated documentation only public fields reported, and none when the public API surface is unknown | | Dependency documentation restore | Exact version pinned, central package management not inherited, documentation collected per target framework, restore tree removed, stale destination replaced, and failures reported for a failed restore, a missing package folder, or a package shipping no documentation | | Package signatures | Every package and symbol package verified, all failures reported before throwing | | Assembly signatures | Package expansion, native binaries under `runtimes/` included, stale expansions replaced, all unsigned assemblies reported | diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index aac27d9df0..f5a0bee7bd 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -1479,6 +1479,7 @@ Describe 'validate-xml-docs.ps1' { 'lib-ref-documentation-identical' = 'error' 'missing-public-uid' = 'error' 'mismatched-public-docid-prefix' = 'error' + 'enum-field-remarks' = 'error' 'missing-documentation' = 'error' 'missing-local-uid' = 'info' 'mismatched-docid-prefix' = 'warning' @@ -1496,6 +1497,225 @@ Describe 'validate-xml-docs.ps1' { } } + Context 'enum field remarks' { + + BeforeAll { + <# + Writes a source tree declaring an enum, which is the only thing that says which + documented members are enum fields. + #> + function New-EnumSource { + param( + [string]$TypeName = 'Widget', + [string[]]$Members = @('First', 'Second'), + [string]$Extra = '' + ) + + $path = New-TestDirectory + $body = ($Members | ForEach-Object { " $_," }) -join "`n" + @" +namespace Contoso +{ + public enum $TypeName + { +$body + } +$Extra +} +"@ | Set-Content -LiteralPath (Join-Path $path 'Widget.cs') -Encoding utf8 + + # A project file, because the enum search root follows the project the caller names. + '' + + 'true' + + '' | + Set-Content -LiteralPath (Join-Path $path 'Sample.csproj') -Encoding utf8 + + return $path + } + + <# + Writes a snippet in the shape the repository uses: one block per member, nested + under a element naming the type. + #> + function New-EnumSnippet { + param( + [Parameter(Mandatory)][string]$Body, + [string]$TypeName = 'Widget' + ) + + $path = New-TestDirectory + "$Body" | + Set-Content -LiteralPath (Join-Path $path "$TypeName.xml") -Encoding utf8 + return $path + } + } + + It 'reports remarks on an enum field' { + $source = New-EnumSource + $snippets = New-EnumSnippet -Body ( + 'W.' + + 'F.Discarded.') + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -SnippetsDirectory $snippets -ProjectSearchRoot $source -ReportPath $report } | + Should -Throw + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Category -eq 'enum-field-remarks' })[0] + $finding.Severity | Should -Be 'error' + $finding.Member | Should -Be 'Widget.First' + } + + It 'accepts remarks on the block documenting the type itself' { + # A type's remarks are rendered; only a field's are discarded. + $source = New-EnumSource + $snippets = New-EnumSnippet -Body 'W.Kept.' + + { & $scriptPath -SnippetsDirectory $snippets -ProjectSearchRoot $source } | Should -Not -Throw + } + + It 'accepts remarks on a platform-variant block documenting the type' { + # WidgetNetfx documents the type for another platform, and is not a declared member, so + # it must not be mistaken for a field merely because its name differs from the type's. + $source = New-EnumSource + $snippets = New-EnumSnippet -Body ( + 'W.' + + 'W.Kept.') + + { & $scriptPath -SnippetsDirectory $snippets -ProjectSearchRoot $source } | Should -Not -Throw + } + + It 'accepts remarks on a member of a type that is not an enum' { + $source = New-EnumSource + $snippets = New-EnumSnippet -TypeName 'Gadget' -Body ( + 'G.' + + 'F.Kept.') + + { & $scriptPath -SnippetsDirectory $snippets -ProjectSearchRoot $source } | Should -Not -Throw + } + + It 'reports remarks on a public enum field in generated documentation' { + # A documentation ID states the member kind, so the field is recognized there too. + $source = New-EnumSource + $staging = New-TestDirectory + New-Item -ItemType Directory -Path (Join-Path $staging 'lib/net8.0') -Force | Out-Null + New-Item -ItemType Directory -Path (Join-Path $staging 'ref/net8.0') -Force | Out-Null + 'F.' + + 'Discarded.' | + Set-Content -LiteralPath (Join-Path $staging 'lib/net8.0/Contoso.xml') -Encoding utf8 + 'F.' | + Set-Content -LiteralPath (Join-Path $staging 'ref/net8.0/Contoso.xml') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory($staging, (Join-Path $packages 'P.1.0.0.nupkg')) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) ` + -ProjectSearchRoot $source -ReportPath $report -ReportOnly + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Category -eq 'enum-field-remarks' })[0] + $finding.Member | Should -Be 'F:Contoso.Widget.First' + } + + It 'ignores an internal enum field in generated documentation' { + # The implementation assembly documents its own P/Invoke enums. They reach no published + # page, so nothing is lost by remarks the build discards. + $source = New-EnumSource + $staging = New-TestDirectory + New-Item -ItemType Directory -Path (Join-Path $staging 'lib/net8.0') -Force | Out-Null + New-Item -ItemType Directory -Path (Join-Path $staging 'ref/net8.0') -Force | Out-Null + 'F.' + + 'Internal.' | + Set-Content -LiteralPath (Join-Path $staging 'lib/net8.0/Contoso.xml') -Encoding utf8 + # The field is absent from ref/, so it is not part of the public API surface. + 'W.' | + Set-Content -LiteralPath (Join-Path $staging 'ref/net8.0/Contoso.xml') -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::CreateFromDirectory($staging, (Join-Path $packages 'P.1.0.0.nupkg')) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) ` + -ProjectSearchRoot $source -ReportPath $report -ReportOnly + + @((Get-Report -Path $report).Findings.Category) | Should -Not -Contain 'enum-field-remarks' + } + + It 'does not report a generated field when the public API surface is unknown' { + # Without reference documentation nothing says which members publish, and the rule is + # not escalated on a guess. + $source = New-EnumSource + $docs = New-TestDirectory + 'F.' + + 'Unknown.' | + Set-Content -LiteralPath (Join-Path $docs 'Contoso.xml') -Encoding utf8 + + { & $scriptPath -DocumentationPath $docs -ProjectSearchRoot $source } | Should -Not -Throw + } + + It 'accepts remarks on a property of a class in generated documentation' { + $source = New-EnumSource + $docs = New-TestDirectory + 'F.' + + 'Kept.' | + Set-Content -LiteralPath (Join-Path $docs 'Contoso.xml') -Encoding utf8 + + { & $scriptPath -DocumentationPath $docs -ProjectSearchRoot $source } | Should -Not -Throw + } + + It 'does not apply the rule when no source tree identifies the enums' { + # Without declarations there is nothing to distinguish a field from a type, and + # guessing would report the very blocks that are legitimate. + $snippets = New-EnumSnippet -Body ( + 'W.' + + 'F.Unknown.') + + { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw + } + + It 'ignores an enum member that is only mentioned in a comment' { + $source = New-EnumSource -Members @('First') -Extra @' + // Second, is commented out and is not a member. +'@ + $snippets = New-EnumSnippet -Body ( + 'W.' + + 'S.Kept.') + + { & $scriptPath -SnippetsDirectory $snippets -ProjectSearchRoot $source } | Should -Not -Throw + } + + It 'reads every member of an enum whose entries carry attributes and values' { + $source = New-TestDirectory + @' +namespace Contoso +{ + public enum Widget + { + [System.ComponentModel.Description("a, b")] + First = 1, + Second = (2 + 3), + Third, + } +} +'@ | Set-Content -LiteralPath (Join-Path $source 'Widget.cs') -Encoding utf8 + + $snippets = New-EnumSnippet -Body ( + 'W.' + + 'T.Discarded.') + $report = Join-Path (New-TestDirectory) 'report.json' + + # Third is the last entry and follows one carrying a comma inside an attribute, so it is + # only found when entries are split on the commas that actually separate members. + { & $scriptPath -SnippetsDirectory $snippets -ProjectSearchRoot $source -ReportPath $report } | + Should -Throw + + @((Get-Report -Path $report).Findings.Category) | Should -Contain 'enum-field-remarks' + } + } + Context 'malformed input' { It 'reports a file that is not well-formed XML' { diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index fdd9fb2036..1c5851a2ff 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -163,6 +163,15 @@ reference. mismatched-docid-prefix warning As above, but referenced from a member that is not public, or from any member when the public API surface is unknown. + enum-field-remarks error An enum field carries , which the documentation + build discards, rendering only the summary. The text + reaches no page and nothing else reports its loss, so it is + folded into the field's instead. Fields are + identified from the enum declarations in source, so a + platform-variant block documenting the type is unaffected. + In generated documentation only public fields are reported, + since the implementation assembly documents internal + P/Invoke enums that reach no published page. missing-external-uid warning Cref is absent from the supplied Learn xref map. unprefixed-cref info Cref carries no "T:"/"M:"/... prefix. Legal; the compiler binds it. Reported for visibility only. @@ -231,6 +240,7 @@ $script:CategorySeverities = [ordered]@{ 'lib-ref-documentation-identical' = 'error' 'missing-public-uid' = 'error' 'mismatched-public-docid-prefix' = 'error' + 'enum-field-remarks' = 'error' 'missing-local-uid' = 'info' 'mismatched-docid-prefix' = 'warning' 'missing-documentation' = 'error' @@ -673,6 +683,113 @@ function Test-ProjectGeneratesDocumentation { return $value -eq 'true' } +<# + Returns the enum types declared in a source tree, each mapped to the names of its members. + + The documentation build silently discards on an enum field, so recognizing those + fields is what makes that loss detectable. Neither a snippet nor generated documentation says + which types are enums, so the declarations themselves are the only authority. + + Member names are collected as well as type names because a snippet identifies a member only by + its element name, and a snippet may hold more than one block for the same type -- a platform + variant such as documents the type, not a field. Asking + whether the name is a declared member separates the two without guessing from the name. +#> +function Get-EnumDeclaration { + param([Parameter(Mandatory)][string]$SearchRoot) + + # Anchored at the start of a line and allowing only attributes and modifiers before the + # keyword, so the word "enum" inside an identifier, a comment or a string is not mistaken for a + # declaration. + $declaration = [regex]::new( + '^\s*(?:\[[^\]]*\]\s*)*(?:(?:public|internal|protected|private|static|partial|new)\s+)*' + + 'enum\s+([A-Za-z_][A-Za-z0-9_]*)', + [System.Text.RegularExpressions.RegexOptions]::Multiline) + + # Comments are removed before the body is read so that a commented-out entry, or an identifier + # inside a doc comment, cannot be taken for a member. + $blockComment = [regex]::new('/\*[\s\S]*?\*/') + $lineComment = [regex]::new('//[^\r\n]*') + + $declarations = [System.Collections.Generic.Dictionary[string, System.Collections.Generic.HashSet[string]]]::new( + [System.StringComparer]::Ordinal) + + foreach ($source in Get-ChildItem -LiteralPath $SearchRoot -Filter '*.cs' -File -Recurse -ErrorAction SilentlyContinue) { + $text = [System.IO.File]::ReadAllText($source.FullName) + + foreach ($match in $declaration.Matches($text)) { + $name = $match.Groups[1].Value + + $open = $text.IndexOf('{', $match.Index + $match.Length) + if ($open -lt 0) { + continue + } + + # Tracked rather than assumed: an attribute or an initializer inside the body may carry + # braces of its own, and stopping at the first one would truncate the member list. + $depth = 0 + $close = -1 + for ($index = $open; $index -lt $text.Length; $index++) { + if ($text[$index] -eq '{') { + $depth++ + } + elseif ($text[$index] -eq '}') { + $depth-- + if ($depth -eq 0) { + $close = $index + break + } + } + } + + if ($close -lt 0) { + continue + } + + $body = $text.Substring($open + 1, $close - $open - 1) + $body = $blockComment.Replace($body, ' ') + $body = $lineComment.Replace($body, ' ') + + if (-not $declarations.ContainsKey($name)) { + $declarations[$name] = [System.Collections.Generic.HashSet[string]]::new( + [System.StringComparer]::Ordinal) + } + + # Split on commas that separate members, ignoring those inside an attribute or an + # initializer expression. + $depth = 0 + $start = 0 + $entries = [System.Collections.Generic.List[string]]::new() + for ($index = 0; $index -lt $body.Length; $index++) { + switch ($body[$index]) { + '[' { $depth++ } + '(' { $depth++ } + ']' { $depth-- } + ')' { $depth-- } + ',' { + if ($depth -eq 0) { + $entries.Add($body.Substring($start, $index - $start)) + $start = $index + 1 + } + } + } + } + $entries.Add($body.Substring($start)) + + foreach ($entry in $entries) { + # An entry is an optional attribute list followed by the member name, which is + # where the identifier is taken from; anything after '=' is its value. + $member = [regex]::Match($entry, '^\s*(?:\[[^\]]*\]\s*)*([A-Za-z_][A-Za-z0-9_]*)') + if ($member.Success) { + [void]$declarations[$name].Add($member.Groups[1].Value) + } + } + } + } + + return $declarations +} + <# Returns the documentation files referenced by a project's sources. @@ -928,6 +1045,26 @@ if (-not [string]::IsNullOrWhiteSpace($ProjectPath)) { $snippetFiles = @($snippetFiles | Where-Object { $referencedSnippets.Contains($_) }) } } + +# Enum declarations, used to detect documentation the build will discard. Taken from whichever +# source tree the caller identified: the project's own directory when one project is in view, and +# the search root when the inputs span several, as they do for a whole package drop. Left null when +# neither was supplied, which disables the rule rather than guessing at the declarations. +$script:EnumDeclarations = $null +$enumSearchRoot = if (-not [string]::IsNullOrWhiteSpace($ProjectPath)) { + [System.IO.Path]::GetDirectoryName((Resolve-Path -LiteralPath $ProjectPath).Path) +} +elseif (-not [string]::IsNullOrWhiteSpace($ProjectSearchRoot)) { + (Resolve-Path -LiteralPath $ProjectSearchRoot).Path +} +else { + $null +} + +if ($null -ne $enumSearchRoot) { + $script:EnumDeclarations = Get-EnumDeclaration -SearchRoot $enumSearchRoot + Write-Host "Enum types declared under '$enumSearchRoot': $($script:EnumDeclarations.Count)." +} $documentationRoots = [System.Collections.Generic.List[string]]::new() # Expanded package directory -> package file name, used by the lib/ref layout checks below. @@ -1249,6 +1386,84 @@ foreach ($entry in $documents) { } } +# The documentation build discards on an enum field, rendering only the summary. Text +# written there is therefore lost without trace: it neither appears on the published page nor is +# reported by anything the author sees. Fold it into the field's instead. +if ($null -ne $script:EnumDeclarations -and $script:EnumDeclarations.Count -gt 0) { + foreach ($entry in $documents) { + foreach ($element in $entry.Document.Descendants()) { + if ($element.Name.LocalName -ne 'remarks') { + continue + } + + # A snippet nests member elements under , naming each member by + # its element name; generated documentation names each member by a documentation ID. + # Both identify a field of an enum, so both are recognized. + $owner = $null + + $parent = $element.Parent + while ($null -ne $parent) { + if ($parent.Name.LocalName -eq 'member') { + # A documentation ID states the member kind outright, so an F: prefix on a + # member of a declared enum settles it without consulting the member list. + # + # Restricted to the public API surface, because generated documentation also + # describes internal types -- the implementation assembly documents its own + # P/Invoke enums -- and no text is lost from a page that is never published. + # Left unreported when the surface is unknown rather than escalated on a guess, + # matching how every other resolution rule here treats an unknown surface. + $nameAttribute = $parent.Attribute('name') + if ($null -ne $nameAttribute -and + $nameAttribute.Value.StartsWith('F:', [System.StringComparison]::Ordinal) -and + $null -ne $script:PublicUids -and + $script:PublicUids.Contains($nameAttribute.Value)) { + + $identifier = $nameAttribute.Value.Substring(2) + $lastDot = $identifier.LastIndexOf('.') + if ($lastDot -gt 0) { + $containing = $identifier.Substring(0, $lastDot) + $typeName = $containing.Substring($containing.LastIndexOf('.') + 1) + if ($script:EnumDeclarations.ContainsKey($typeName)) { + $owner = $nameAttribute.Value + } + } + } + break + } + + if ($parent.Name.LocalName -eq 'members') { + break + } + + $grandparent = $parent.Parent + if ($null -ne $grandparent -and $grandparent.Name.LocalName -eq 'members') { + $nameAttribute = $grandparent.Attribute('name') + if ($null -ne $nameAttribute -and + $script:EnumDeclarations.ContainsKey($nameAttribute.Value) -and + $script:EnumDeclarations[$nameAttribute.Value].Contains($parent.Name.LocalName)) { + + $owner = "$($nameAttribute.Value).$($parent.Name.LocalName)" + } + break + } + + $parent = $grandparent + } + + if ($null -eq $owner) { + continue + } + + $lineInfo = [System.Xml.IXmlLineInfo]$element + $lineNumber = if ($lineInfo.HasLineInfo()) { $lineInfo.LineNumber } else { 0 } + Add-Finding -Category 'enum-field-remarks' -Path $entry.Path ` + -LineNumber $lineNumber -Member $owner -Message ( + "Enum field '$owner' carries , which the documentation build discards. " + + 'Fold the text into the summary so it reaches the published page.') + } + } +} + # Package layout ------------------------------------------------------------------------------ # The driver ships a full XML documentation file under lib/ and a trimmed one under ref/. Compare From e719f759c68285a94a66105311fcda3be902895f Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Thu, 24 Sep 2026 20:48:51 -0300 Subject: [PATCH 18/23] Restore documentation lost to includes that matched nothing A documentation comment pulls its text in with , and when the path matches nothing the compiler does not fail. It copies the unresolved element into the output with a comment beside it and moves on, so the member ships with no documentation whatsoever and the only record is in a generated file nobody reads. Nine public members of the reference assembly were in that state: SqlCommand.BeginExecuteXmlReader(AsyncCallback, object) SqlBulkCopyColumnOrderHint..ctor(string, SortOrder) SqlBulkCopyColumnOrderHintCollection.Add(string, SortOrder) SspiAuthenticationParameters..ctor and its five properties Two causes. Fifteen include paths named a block whose second word differed in case from the one the snippet declares -- AsyncCallbackAndstateObject against AsyncCallbackAndStateObject, and the same slip in the bulk copy mapping constructors, SqlClientPermission and EndExecuteReader. XPath compares attributes case-sensitively, so each matched nothing. The snippets are consistent, so the paths were corrected to them. Separately, the SspiAuthenticationParameters paths nested each member inside the type's own block when the two are siblings, and omitted the trailing /*. This also explains the xref-not-found the API docs build reported for 'Microssoft.Data.SqlClient.SqlCommand.EnableOptimizedParameterBinding'. Our snippet was corrected by #2889, but BeginExecuteXmlReader's include had already stopped matching, so the member contributed nothing for the ingestion to write over. mdoc merges rather than deletes, so the pre-#2889 text stayed in the API docs repository, typo and all. The two occurrences under BeginExecuteReader, whose includes still worked, were refreshed and corrected in the same run -- which is why one of the three survived. It is also why the rendered page looks healthy: an unresolved reference renders as plain text among neighbours that resolve, in the exception list of a single overload. Add an unresolved-include rule so the next one is caught in our own build. The signal needs no XPath simulation: an surviving into generated documentation is itself proof that the member lost everything. All 2712 include directives in the tree now resolve against their snippets, and the reference assembly builds with none left unresolved. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../onebranch/scripts/tests/README.md | 1 + .../scripts/tests/validate-xml-docs.Tests.ps1 | 58 +++++++++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 49 ++++++++++++++++ .../ref/Microsoft.Data.SqlClient.cs | 18 +++--- .../SSPI/SspiAuthenticationParameters.cs | 12 ++-- .../SqlClient/SqlBulkCopyColumnMapping.cs | 6 +- .../SqlBulkCopyColumnMappingCollection.cs | 8 +-- .../SqlClient/SqlBulkCopyColumnOrderHint.cs | 2 +- .../SqlBulkCopyColumnOrderHintCollection.cs | 2 +- .../SqlClient/SqlClientPermission.netfx.cs | 4 +- .../Data/SqlClient/SqlCommand.Reader.cs | 6 +- .../Data/SqlClient/SqlCommand.Xml.cs | 2 +- 12 files changed, 138 insertions(+), 30 deletions(-) diff --git a/eng/pipelines/onebranch/scripts/tests/README.md b/eng/pipelines/onebranch/scripts/tests/README.md index 9abc2b46b5..54c6fa6cb4 100644 --- a/eng/pipelines/onebranch/scripts/tests/README.md +++ b/eng/pipelines/onebranch/scripts/tests/README.md @@ -43,6 +43,7 @@ Invoke-Pester ./publish-symbols.Tests.ps1 -Output Detailed | XML docs lib/ref layout | Full `lib/` XML paired with trimmed `ref/` XML accepted; trimmed `lib/`, untrimmed `ref/`, and byte-identical `lib`/`ref` rejected; per-target-framework isolation; packages without a `ref/` folder ignored | | XML docs dependencies | References into a package that was not built this run resolve against the published dependency's documentation; that documentation is not itself validated, does not establish the public API surface, and does not enable resolution when nothing is under validation | | XML docs enum remarks | `` on an enum field reported, since the documentation build discards it; type blocks and platform-variant type blocks accepted; members of non-enum types accepted; enum members read from source through attributes, initializers and comments; in generated documentation only public fields reported, and none when the public API surface is unknown | +| XML docs unresolved includes | An `` surviving into generated documentation reported, since the compiler leaves it in place and the member ships undocumented; the requested path echoed back; every occurrence reported; includes inside snippets ignored | | Dependency documentation restore | Exact version pinned, central package management not inherited, documentation collected per target framework, restore tree removed, stale destination replaced, and failures reported for a failed restore, a missing package folder, or a package shipping no documentation | | Package signatures | Every package and symbol package verified, all failures reported before throwing | | Assembly signatures | Package expansion, native binaries under `runtimes/` included, stale expansions replaced, all unsigned assemblies reported | diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index f5a0bee7bd..db6cd993d5 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -1480,6 +1480,7 @@ Describe 'validate-xml-docs.ps1' { 'missing-public-uid' = 'error' 'mismatched-public-docid-prefix' = 'error' 'enum-field-remarks' = 'error' + 'unresolved-include' = 'error' 'missing-documentation' = 'error' 'missing-local-uid' = 'info' 'mismatched-docid-prefix' = 'warning' @@ -1716,6 +1717,63 @@ namespace Contoso } } + Context 'unresolved include' { + + It 'reports an include the compiler could not resolve' { + # The compiler copies an unmatched include into its output rather than failing, so the + # member ships with no documentation and nothing else says so. + $docs = New-TestDirectory + '' + + '' + + '' + + '' | + Set-Content -LiteralPath (Join-Path $docs 'Microsoft.Data.SqlClient.xml') -Encoding utf8 + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -DocumentationPath $docs -ReportPath $report } | Should -Throw + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Category -eq 'unresolved-include' })[0] + $finding.Severity | Should -Be 'error' + $finding.Member | Should -Be 'M:Microsoft.Data.SqlClient.SqlCommand.BeginExecuteXmlReader(System.AsyncCallback,System.Object)' + # The path is quoted back, because the mismatch is usually a letter's case within it. + $finding.Message | Should -BeLike '*AsyncCallbackAndstateObject*' + } + + It 'accepts documentation whose includes all resolved' { + $docs = New-DocumentationDirectory -Members @('T:Microsoft.Data.SqlClient.SqlConnection') + + { & $scriptPath -DocumentationPath $docs } | Should -Not -Throw + } + + It 'does not report an include inside a snippet' { + # A snippet is the include's target, not its consumer; the compiler never reads one + # looking for includes, so an element there says nothing about a member losing its + # documentation. + $snippets = New-TestDirectory + 'S.' + + '' | + Set-Content -LiteralPath (Join-Path $snippets 'SqlCommand.xml') -Encoding utf8 + + { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw + } + + It 'reports every unresolved include rather than only the first' { + $docs = New-TestDirectory + '' + + '' + + '' + + '' | + Set-Content -LiteralPath (Join-Path $docs 'Microsoft.Data.SqlClient.xml') -Encoding utf8 + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report -ReportOnly + + @((Get-Report -Path $report).Findings | + Where-Object { $_.Category -eq 'unresolved-include' }).Count | Should -Be 2 + } + } + Context 'malformed input' { It 'reports a file that is not well-formed XML' { diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 1c5851a2ff..8d9718bd33 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -172,6 +172,13 @@ In generated documentation only public fields are reported, since the implementation assembly documents internal P/Invoke enums that reach no published page. + unresolved-include error An survived into generated documentation, which + means the compiler matched nothing at that path and the + member has no documentation at all. The compiler does not + fail on this; it leaves the element in place with a comment + beside it, so the loss is otherwise silent. Usually the + name attribute in the path differs in case from the one the + snippet declares. missing-external-uid warning Cref is absent from the supplied Learn xref map. unprefixed-cref info Cref carries no "T:"/"M:"/... prefix. Legal; the compiler binds it. Reported for visibility only. @@ -241,6 +248,7 @@ $script:CategorySeverities = [ordered]@{ 'missing-public-uid' = 'error' 'mismatched-public-docid-prefix' = 'error' 'enum-field-remarks' = 'error' + 'unresolved-include' = 'error' 'missing-local-uid' = 'info' 'mismatched-docid-prefix' = 'warning' 'missing-documentation' = 'error' @@ -1386,6 +1394,47 @@ foreach ($entry in $documents) { } } +# A documentation comment pulls its text in with , and the compiler resolves it at compile +# time. When the path matches nothing the compiler does not fail: it copies the unresolved +# element into the output and moves on, so the member ships with no documentation at all +# and the only trace is a comment in a generated file nobody reads. A surviving is +# therefore proof that a member has lost its entire documentation. +foreach ($entry in $documents) { + if ($entry.Kind -ne 'documentation') { + continue + } + + foreach ($element in $entry.Document.Descendants()) { + if ($element.Name.LocalName -ne 'include') { + continue + } + + $member = '' + $ancestor = $element.Parent + while ($null -ne $ancestor) { + if ($ancestor.Name.LocalName -eq 'member') { + $nameAttribute = $ancestor.Attribute('name') + if ($null -ne $nameAttribute) { + $member = $nameAttribute.Value + } + break + } + $ancestor = $ancestor.Parent + } + + $pathAttribute = $element.Attribute('path') + $requested = if ($null -ne $pathAttribute) { $pathAttribute.Value } else { '(no path)' } + + $lineInfo = [System.Xml.IXmlLineInfo]$element + $lineNumber = if ($lineInfo.HasLineInfo()) { $lineInfo.LineNumber } else { 0 } + Add-Finding -Category 'unresolved-include' -Path $entry.Path ` + -LineNumber $lineNumber -Member $member -Message ( + "Documentation for '$member' was not included: the compiler found nothing at " + + "'$requested', so the member has no documentation. Check the path against the " + + 'snippet, including the case of any name attribute.') + } +} + # The documentation build discards on an enum field, rendering only the summary. Text # written there is therefore lost without trace: it neither appears on the published page nor is # reported by anything the author sees. Fold it into the field's instead. diff --git a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.cs b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.cs index 5a8ff5c0ae..18f31f9aa6 100644 --- a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.cs +++ b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.cs @@ -468,7 +468,7 @@ public void Remove(Microsoft.Data.SqlClient.SqlBulkCopyColumnMapping value) { } /// public sealed class SqlBulkCopyColumnOrderHint { - /// + /// public SqlBulkCopyColumnOrderHint(string column, SortOrder sortOrder) { } /// public string Column { get { throw null; } set { } } @@ -483,7 +483,7 @@ public sealed class SqlBulkCopyColumnOrderHintCollection : System.Collections.Co public SqlBulkCopyColumnOrderHint this[int index] { get { throw null; } } /// public SqlBulkCopyColumnOrderHint Add(SqlBulkCopyColumnOrderHint columnOrderHint) { throw null; } - /// + /// public SqlBulkCopyColumnOrderHint Add(string column, SortOrder sortOrder) { throw null; } /// public new void Clear() { } @@ -876,7 +876,7 @@ public event System.Data.StatementCompletedEventHandler StatementCompleted { add [System.Security.Permissions.HostProtectionAttribute(System.Security.Permissions.SecurityAction.LinkDemand, ExternalThreading = true)] #endif public System.IAsyncResult BeginExecuteXmlReader() { throw null; } - /// + /// #if NETFRAMEWORK [System.Security.Permissions.HostProtectionAttribute(System.Security.Permissions.SecurityAction.LinkDemand, ExternalThreading = true)] #endif @@ -2409,7 +2409,7 @@ protected SspiContextProvider() { } /// public sealed class SspiAuthenticationParameters { - /// + /// public SspiAuthenticationParameters( string serverName, string resource, @@ -2417,18 +2417,18 @@ public SspiAuthenticationParameters( string databaseName = null, string password = null){} - /// + /// public string Resource { get { throw null; } } - /// + /// public string ServerName { get { throw null; } } - /// + /// public string UserId { get { throw null; } } - /// + /// public string DatabaseName { get { throw null; } } - /// + /// public string Password { get { throw null; } } } diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SSPI/SspiAuthenticationParameters.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SSPI/SspiAuthenticationParameters.cs index 38d96766ba..3d41cb770c 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SSPI/SspiAuthenticationParameters.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SSPI/SspiAuthenticationParameters.cs @@ -9,7 +9,7 @@ namespace Microsoft.Data.SqlClient /// public sealed class SspiAuthenticationParameters { - /// + /// public SspiAuthenticationParameters( string serverName, string resource, @@ -24,19 +24,19 @@ public SspiAuthenticationParameters( Password = password; } - /// + /// public string Resource { get; } - /// + /// public string ServerName { get; } - /// + /// public string? UserId { get; } - /// + /// public string? DatabaseName { get; } - /// + /// public string? Password { get; } } } diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnMapping.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnMapping.cs index 295fa56d0e..e725165121 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnMapping.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnMapping.cs @@ -110,7 +110,7 @@ public SqlBulkCopyColumnMapping() _internalSourceColumnOrdinal = -1; } - /// + /// public SqlBulkCopyColumnMapping(string sourceColumn, string destinationColumn) { SourceColumn = sourceColumn; @@ -124,14 +124,14 @@ public SqlBulkCopyColumnMapping(int sourceColumnOrdinal, string destinationColum DestinationColumn = destinationColumn; } - /// + /// public SqlBulkCopyColumnMapping(string sourceColumn, int destinationOrdinal) { SourceColumn = sourceColumn; DestinationOrdinal = destinationOrdinal; } - /// + /// public SqlBulkCopyColumnMapping(int sourceColumnOrdinal, int destinationOrdinal) { SourceOrdinal = sourceColumnOrdinal; diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnMappingCollection.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnMappingCollection.cs index f7bfd723e5..56dbac7fda 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnMappingCollection.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnMappingCollection.cs @@ -44,28 +44,28 @@ public SqlBulkCopyColumnMapping Add(SqlBulkCopyColumnMapping bulkCopyColumnMappi return bulkCopyColumnMapping; } - /// + /// public SqlBulkCopyColumnMapping Add(string sourceColumn, string destinationColumn) { AssertWriteAccess(); return Add(new SqlBulkCopyColumnMapping(sourceColumn, destinationColumn)); } - /// + /// public SqlBulkCopyColumnMapping Add(int sourceColumnIndex, string destinationColumn) { AssertWriteAccess(); return Add(new SqlBulkCopyColumnMapping(sourceColumnIndex, destinationColumn)); } - /// + /// public SqlBulkCopyColumnMapping Add(string sourceColumn, int destinationColumnIndex) { AssertWriteAccess(); return Add(new SqlBulkCopyColumnMapping(sourceColumn, destinationColumnIndex)); } - /// + /// public SqlBulkCopyColumnMapping Add(int sourceColumnIndex, int destinationColumnIndex) { AssertWriteAccess(); diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHint.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHint.cs index 2329df4140..b9d050d5a1 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHint.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHint.cs @@ -9,7 +9,7 @@ namespace Microsoft.Data.SqlClient /// public sealed class SqlBulkCopyColumnOrderHint { - /// + /// public SqlBulkCopyColumnOrderHint(string column, SortOrder sortOrder) { Column = column; diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHintCollection.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHintCollection.cs index 82fda1ad7d..8ed311a4cd 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHintCollection.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHintCollection.cs @@ -33,7 +33,7 @@ public SqlBulkCopyColumnOrderHint Add(SqlBulkCopyColumnOrderHint columnOrderHint return columnOrderHint; } - /// + /// public SqlBulkCopyColumnOrderHint Add(string column, SortOrder sortOrder) => Add(new SqlBulkCopyColumnOrderHint(column, sortOrder)); /// diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlClientPermission.netfx.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlClientPermission.netfx.cs index 0a670ad97f..23ab06b7ab 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlClientPermission.netfx.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlClientPermission.netfx.cs @@ -47,7 +47,7 @@ public SqlClientPermission(PermissionState state) : base(state) { } - /// + /// [Obsolete("SqlClientPermission(PermissionState state, Boolean allowBlankPassword) has been deprecated. Use the SqlClientPermission(PermissionState.None) constructor. http://go.microsoft.com/fwlink/?linkid=14202", true)] // MDAC 86034 public SqlClientPermission(PermissionState state, bool allowBlankPassword) : this(state) { @@ -107,7 +107,7 @@ private bool _IsUnrestricted #region Public/Internal Methods - /// + /// public override void Add(string connectionString, string restrictions, KeyRestrictionBehavior behavior) { DbConnectionString constr = new DbConnectionString(connectionString, restrictions, behavior, SqlConnectionOptions.KeywordMap); diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.Reader.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.Reader.cs index ec4d7709da..2b5f568c60 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.Reader.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.Reader.cs @@ -32,14 +32,14 @@ public sealed partial class SqlCommand public IAsyncResult BeginExecuteReader() => BeginExecuteReader(callback: null, stateObject: null, CommandBehavior.Default); - /// + /// #if NETFRAMEWORK [HostProtection(ExternalThreading = true)] #endif public IAsyncResult BeginExecuteReader(AsyncCallback callback, object stateObject) => BeginExecuteReader(callback, stateObject, CommandBehavior.Default); - /// + /// #if NETFRAMEWORK [HostProtection(ExternalThreading = true)] #endif @@ -60,7 +60,7 @@ public IAsyncResult BeginExecuteReader(AsyncCallback callback, object stateObjec public IAsyncResult BeginExecuteReader(CommandBehavior behavior) => BeginExecuteReader(callback: null, stateObject: null, behavior); - /// + /// public SqlDataReader EndExecuteReader(IAsyncResult asyncResult) { try diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.Xml.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.Xml.cs index d74bda54d6..e17aaf79e0 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.Xml.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.Xml.cs @@ -35,7 +35,7 @@ public sealed partial class SqlCommand public IAsyncResult BeginExecuteXmlReader() => BeginExecuteXmlReader(callback: null, stateObject: null); - /// + /// #if NETFRAMEWORK [HostProtection(ExternalThreading = true)] #endif From 6a2916dc626d538bd574dd1e01b3dd5efcc9df16 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Fri, 25 Sep 2026 08:03:19 -0300 Subject: [PATCH 19/23] Address review feedback on the XML documentation validation Three findings from PR #4752 and PR #4730. The ref project's comment gave the wrong reason for requiring forward slashes: PowerShell escapes with a backtick, not a backslash, and "C:\temp\new\test" prints literally. The constraint is real but it is that a backslash is not a path separator on a non-Windows host. documentation-not-expected was in CategorySeverities, was emitted in two places and was named in the step template, but was missing from .OUTPUTS, so the help did not match the categories the script can produce. A package root was compared with StartsWith and no trailing separator, which lets one root prefix-match a sibling whose name merely extends it. The segment checks downstream discarded the realistic cases, so this was latent rather than live; bounding the comparison at a directory boundary makes it independent of them. The accompanying test fails without the guard: a package named Contoso.Widgetref has its lib/ read as reference documentation against root Contoso.Widget, which defines the public API surface and so escalates an unresolved cref from info to error. 252 Pester tests pass, the snippets gate passes over 4687 crefs in 99 files, and the reference assembly builds clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../scripts/tests/validate-xml-docs.Tests.ps1 | 40 +++++++++++++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 18 ++++++++- .../ref/Microsoft.Data.SqlClient.csproj | 4 +- 3 files changed, 58 insertions(+), 4 deletions(-) diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index db6cd993d5..2a72dead71 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -766,6 +766,46 @@ Describe 'validate-xml-docs.ps1' { '' } + It 'keeps packages apart when one root name extends another' { + # A root compared without a trailing separator prefix-matches any sibling whose name + # merely extends it. Contoso.Widgetref extends Contoso.Widget, so its lib/ path reads + # as "ref/lib/net8.0/..." relative to the shorter root, and the leading segment makes + # it look like reference documentation -- which is what defines the public API + # surface. A cref that fails to resolve from there is then reported as an error + # against a public member rather than as information. + $staging = New-TestDirectory + $unresolved = 'A' + + 'H.' + + '' + + $widget = Join-Path $staging 'Contoso.Widget' + New-Item -ItemType Directory -Path (Join-Path $widget 'lib/net8.0') -Force | Out-Null + Set-Content -LiteralPath (Join-Path $widget 'lib/net8.0/Contoso.xml') ` + -Value $script:FullXml -Encoding utf8 + + # No ref/ folder of its own, so nothing here legitimately defines a public surface. + $widgetref = Join-Path $staging 'Contoso.Widgetref' + New-Item -ItemType Directory -Path (Join-Path $widgetref 'lib/net8.0') -Force | Out-Null + Set-Content -LiteralPath (Join-Path $widgetref 'lib/net8.0/Other.xml') ` + -Value $unresolved -Encoding utf8 + + $packages = New-TestDirectory + Add-Type -AssemblyName System.IO.Compression.FileSystem + foreach ($name in 'Contoso.Widget', 'Contoso.Widgetref') { + [System.IO.Compression.ZipFile]::CreateFromDirectory( + (Join-Path $staging $name), (Join-Path $packages "$name.nupkg")) + } + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -PackagesPath $packages -ExtractPath (New-TestDirectory) ` + -ReportPath $report -ReportOnly + + $finding = @((Get-Report -Path $report).Findings | + Where-Object { $_.Cref -eq 'T:Microsoft.Data.SqlClient.Absent' })[0] + $finding.Category | Should -Be 'missing-local-uid' + $finding.Severity | Should -Be 'info' + } + It 'accepts a full lib/ XML paired with a trimmed ref/ XML' { $packages = New-LayoutPackage -Files @{ 'lib/net8.0/Microsoft.Data.SqlClient.xml' = $script:FullXml diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index 8d9718bd33..c54b8f7890 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -189,6 +189,12 @@ warning Documentation was found for a project that does not set GenerateDocumentationFile. The project and the build disagree; the documentation is still validated. + documentation-not-expected + info No documentation was found, and none was expected: the + project does not set GenerateDocumentationFile, or it + references no documentation snippets. Stated rather than + passed over in silence, so a run shows why nothing was + checked. .EXAMPLE ./validate-xml-docs.ps1 -SnippetsDirectory ./doc/snippets @@ -875,7 +881,12 @@ function Test-IsReferenceDocumentationPath { param([Parameter(Mandatory)][string]$Path) foreach ($packageRoot in $script:ExpandedPackageRoots.Keys) { - $rootFull = (Resolve-Path -LiteralPath $packageRoot).Path + # Compared with a trailing separator so that one package root cannot prefix-match another + # whose name merely extends it, such as Microsoft.Data.SqlClient against + # Microsoft.Data.SqlClient.Extensions. The segment check below would discard such a path + # anyway, but bounding the comparison at a directory boundary makes that independent of it. + $rootFull = (Resolve-Path -LiteralPath $packageRoot).Path.TrimEnd([char]'/', [char]'\') + + [System.IO.Path]::DirectorySeparatorChar if (-not $Path.StartsWith($rootFull, [System.StringComparison]::OrdinalIgnoreCase)) { continue } @@ -1562,7 +1573,10 @@ if ($script:ExpandedPackageRoots.Count -gt 0) { foreach ($packageRoot in $script:ExpandedPackageRoots.Keys) { $packageName = $script:ExpandedPackageRoots[$packageRoot] - $rootFull = (Resolve-Path -LiteralPath $packageRoot).Path + # Trailing separator for the same reason as in Test-IsReferenceDocumentationPath: it keeps + # one package root from prefix-matching another whose name extends it. + $rootFull = (Resolve-Path -LiteralPath $packageRoot).Path.TrimEnd([char]'/', [char]'\') + + [System.IO.Path]::DirectorySeparatorChar # Index the documentation this package contains by folder kind, target framework and file # name, so lib/net8.0/X.xml can be matched with ref/net8.0/X.xml. diff --git a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj index ae5d0e5a25..e7b09420be 100644 --- a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj +++ b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj @@ -55,8 +55,8 @@ have the full documentation. --> $(RepoRoot)tools/intellisense/TrimDocs.ps1 From 16a45eb9b0e3af1c85a0899c2227be63f4e67f3d Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Fri, 25 Sep 2026 08:37:16 -0300 Subject: [PATCH 20/23] Fix documentation gaps found in Learn previews Add a reusable Learn preview audit procedure, detect over-broad XML include paths, and repair documentation that Open Publishing silently discarded. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../documentation.instructions.md | 48 ++++++++++ .../Microsoft.Data.SqlTypes/SqlJson.xml | 24 ++--- .../onebranch/scripts/tests/README.md | 1 + .../scripts/tests/validate-xml-docs.Tests.ps1 | 40 +++++++++ .../onebranch/scripts/validate-xml-docs.ps1 | 88 +++++++++++++++++++ .../steps/validate-xml-docs-step.yml | 4 +- ...eDirectoryAuthenticationProviderOptions.cs | 2 +- .../SqlClientConnectionCloseBefore.cs | 2 +- .../Data/SqlClient/OnChangedEventHandler.cs | 3 +- .../SqlBulkCopyColumnOrderHintCollection.cs | 6 ++ .../src/Microsoft/Data/SqlDbTypeExtensions.cs | 4 +- 11 files changed, 200 insertions(+), 22 deletions(-) diff --git a/.github/instructions/documentation.instructions.md b/.github/instructions/documentation.instructions.md index f663c2e503..5284da7788 100644 --- a/.github/instructions/documentation.instructions.md +++ b/.github/instructions/documentation.instructions.md @@ -174,6 +174,54 @@ dotnet build ./src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj Validation is offline by design; it needs no network access and no xref map download. +### Validating Learn Preview Output + +Local validation proves that XML is well formed and that references resolve, but it does not prove that Open Publishing rendered every documentation element. After an API docs ingestion PR is available in [dotnet/sqlclient-api-docs](https://github.com/dotnet/sqlclient-api-docs), compare the Learn previews with the source XML before approving the update. + +1. Find the latest **Learn Build status** comment and record the commit it validated. Do not use preview links from an older comment. +2. Enumerate every changed API XML file in the PR. The bot comment lists only the first 25 files, including framework indexes and package metadata, so its table is not the complete API-page list: + + ```bash + gh api repos/dotnet/sqlclient-api-docs/pulls//files --paginate \ + --jq '.[] | select(.filename | test("/xml/.+/.+\\.xml$")) | .filename' + ``` + +3. Open every API type preview. Use the `FullName` from the file's `` element as the lowercase API slug, preserve the `branch=pr-en-us-` query, and select the matching view: + - Main provider: `sqlclient-dotnet-core-` + - Azure Key Vault provider: `akvprovider-dotnet-core-` +4. Follow the preview's own links to every changed member or overload. Do not derive explicit-interface or operator URLs by string replacement; Learn uses special slugs for some members. Enum fields intentionally have no standalone pages and must be checked in the type's fields table. +5. Compare the rendered content through the entire publication path: + - Local snippet or source XML + - The `` path on the public declaration + - Generated or packaged XML documentation + - The API docs PR XML + - The rendered type and member previews +6. Check summaries, remarks, examples, parameters, returns or values, exceptions, overload descriptions, code samples, and xrefs. A healthy type landing page is not proof that each overload page is complete. +7. Classify expected renderer transformations before reporting a discrepancy: + - Learn adds display signatures to xrefs, such as `GetSchema()`. + - Markdown tables become separate cells and included snippets become rendered code. + - `To be added.` placeholders are suppressed. + - `` on enum fields are discarded; move required text into ``. +8. Treat content present in a snippet but absent from generated XML as a source or build-wiring problem. Common causes include an `` XPath that matches nothing or too much, documentation attached only to a ref declaration, and `lib/` packaging from trimmed `ref/` XML. + +The review site requires Microsoft authentication. On a corp-joined Windows device running WSL, the Linux browser may not have the required session. Launch a separate Windows Edge profile with Chrome DevTools Protocol enabled so Edge can use seamless Entra SSO: + +```bash +EDGE="/mnt/c/Program Files (x86)/Microsoft/Edge/Application/msedge.exe" +"$EDGE" --remote-debugging-port=9222 --remote-allow-origins=* \ + --user-data-dir=C:\\Temp\\edge-cdp-profile \ + --no-first-run --no-default-browser-check about:blank +``` + +Drive the browser from the Windows side because the Windows firewall can block WSL-to-Windows access to the debugging port: + +```bash +/mnt/c/Windows/System32/curl.exe -s http://localhost:9222/json/version +/mnt/c/Windows/System32/curl.exe -s http://localhost:9222/json +``` + +Never automate credentials or copy authentication tokens. Navigate the authenticated browser through CDP and capture the rendered article text or DOM for comparison. Keep crawl output outside the repository. + ### Trimmed vs. Full Documentation The driver package ships **two** XML documentation files per target framework, and they are deliberately different: diff --git a/doc/snippets/Microsoft.Data.SqlTypes/SqlJson.xml b/doc/snippets/Microsoft.Data.SqlTypes/SqlJson.xml index fa02584fb3..6e0b7fc396 100644 --- a/doc/snippets/Microsoft.Data.SqlTypes/SqlJson.xml +++ b/doc/snippets/Microsoft.Data.SqlTypes/SqlJson.xml @@ -19,11 +19,9 @@ The serialized JSON string to use, or null. - - - If the given string is not valid JSON. - - + + If the given string is not valid JSON. + @@ -34,11 +32,9 @@ The document to use, or null. - - - If the given document has been disposed of. - - + + If the given document has been disposed of. + @@ -54,11 +50,9 @@ Gets the serialized JSON string of this instance. - - - If the JSON value is null. - - + + If the JSON value is null. + diff --git a/eng/pipelines/onebranch/scripts/tests/README.md b/eng/pipelines/onebranch/scripts/tests/README.md index 54c6fa6cb4..49d92f429c 100644 --- a/eng/pipelines/onebranch/scripts/tests/README.md +++ b/eng/pipelines/onebranch/scripts/tests/README.md @@ -44,6 +44,7 @@ Invoke-Pester ./publish-symbols.Tests.ps1 -Output Detailed | XML docs dependencies | References into a package that was not built this run resolve against the published dependency's documentation; that documentation is not itself validated, does not establish the public API surface, and does not enable resolution when nothing is under validation | | XML docs enum remarks | `` on an enum field reported, since the documentation build discards it; type blocks and platform-variant type blocks accepted; members of non-enum types accepted; enum members read from source through attributes, initializers and comments; in generated documentation only public fields reported, and none when the public API surface is unknown | | XML docs unresolved includes | An `` surviving into generated documentation reported, since the compiler leaves it in place and the member ships undocumented; the requested path echoed back; every occurrence reported; includes inside snippets ignored | +| XML docs unexpected elements | Member containers emitted by an over-broad `` reported; compiler-supported top-level documentation elements accepted | | Dependency documentation restore | Exact version pinned, central package management not inherited, documentation collected per target framework, restore tree removed, stale destination replaced, and failures reported for a failed restore, a missing package folder, or a package shipping no documentation | | Package signatures | Every package and symbol package verified, all failures reported before throwing | | Assembly signatures | Package expansion, native binaries under `runtimes/` included, stale expansions replaced, all unsigned assemblies reported | diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index 2a72dead71..19ab5ff330 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -1521,6 +1521,7 @@ Describe 'validate-xml-docs.ps1' { 'mismatched-public-docid-prefix' = 'error' 'enum-field-remarks' = 'error' 'unresolved-include' = 'error' + 'unexpected-documentation-element' = 'error' 'missing-documentation' = 'error' 'missing-local-uid' = 'info' 'mismatched-docid-prefix' = 'warning' @@ -1814,6 +1815,45 @@ namespace Contoso } } + Context 'unexpected documentation elements' { + + It 'reports member containers emitted by an over-broad include' { + # Selecting every child of copies the type and property containers into the + # compiler output instead of their summary and remarks, so Learn receives no type docs. + $docs = New-TestDirectory + '' + + '' + + 'Options.' + + 'Setting.' + + '' | + Set-Content -LiteralPath (Join-Path $docs 'Microsoft.Data.SqlClient.xml') -Encoding utf8 + $report = Join-Path (New-TestDirectory) 'report.json' + + { & $scriptPath -DocumentationPath $docs -ReportPath $report } | Should -Throw + + $findings = @((Get-Report -Path $report).Findings | + Where-Object { $_.Category -eq 'unexpected-documentation-element' }) + $findings.Count | Should -Be 2 + $findings.Member | Should -Contain 'T:Microsoft.Data.SqlClient.Options' + ($findings.Message -join "`n") | Should -BeLike '**' + ($findings.Message -join "`n") | Should -BeLike '**' + } + + It 'accepts compiler-supported top-level documentation elements' { + $docs = New-TestDirectory + '' + + 'Runs.Details.Value.' + + 'Result.Failure.' + + 'Failure conditions.Example.' + + 'Article.' + + 'Internal source note.Implementation note.' + + '' | + Set-Content -LiteralPath (Join-Path $docs 'Microsoft.Data.SqlClient.xml') -Encoding utf8 + + { & $scriptPath -DocumentationPath $docs } | Should -Not -Throw + } + } + Context 'malformed input' { It 'reports a file that is not well-formed XML' { diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index c54b8f7890..c982e64c5d 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -179,6 +179,12 @@ beside it, so the loss is otherwise silent. Usually the name attribute in the path differs in case from the one the snippet declares. + unexpected-documentation-element + error A generated member contains an unrecognized top-level + container around standard documentation elements. This + usually means an path matched a member container + rather than that member's summary, remarks and other + documentation elements. missing-external-uid warning Cref is absent from the supplied Learn xref map. unprefixed-cref info Cref carries no "T:"/"M:"/... prefix. Legal; the compiler binds it. Reported for visibility only. @@ -255,6 +261,7 @@ $script:CategorySeverities = [ordered]@{ 'mismatched-public-docid-prefix' = 'error' 'enum-field-remarks' = 'error' 'unresolved-include' = 'error' + 'unexpected-documentation-element' = 'error' 'missing-local-uid' = 'info' 'mismatched-docid-prefix' = 'warning' 'missing-documentation' = 'error' @@ -1446,6 +1453,87 @@ foreach ($entry in $documents) { } } +# An include path can match too much as well as nothing. Selecting every child of , for +# example, copies the type and property containers into one compiler-generated rather than +# copying the type's summary and remarks. The compiler preserves those unknown elements without an +# error, but Open Publishing ignores them and produces an undocumented API page. +$supportedMemberElements = [System.Collections.Generic.HashSet[string]]::new( + [string[]]@( + 'a', + 'altmember', + 'block', + 'br', + 'c', + 'code', + 'description', + 'devdoc', + 'example', + 'exception', + 'exclude', + 'filterpriority', + 'format', + 'include', + 'inheritdoc', + 'item', + 'list', + 'listheader', + 'note', + 'overloads', + 'param', + 'paramref', + 'para', + 'permission', + 'preliminary', + 'remarks', + 'returns', + 'see', + 'seealso', + 'summary', + 'term', + 'threadsafety', + 'throws', + 'typeparam', + 'typeparamref', + 'value' + ), + [System.StringComparer]::OrdinalIgnoreCase) + +foreach ($entry in $documents) { + if ($entry.Kind -ne 'documentation') { + continue + } + + foreach ($memberElement in $entry.Document.Descendants('member')) { + $nameAttribute = $memberElement.Attribute('name') + $member = if ($null -ne $nameAttribute) { $nameAttribute.Value } else { '' } + + foreach ($element in $memberElement.Elements()) { + if ($supportedMemberElements.Contains($element.Name.LocalName)) { + continue + } + + # XML documentation is extensible, and this repository uses metadata elements such as + # , , and . The silent-loss signature is narrower: an unknown + # element wrapping standard documentation elements copied from a snippet member. + $nestedDocumentationElement = $element.Elements() | + Where-Object { $supportedMemberElements.Contains($_.Name.LocalName) } | + Select-Object -First 1 + if ($null -eq $nestedDocumentationElement) { + continue + } + + $lineInfo = [System.Xml.IXmlLineInfo]$element + $lineNumber = if ($lineInfo.HasLineInfo()) { $lineInfo.LineNumber } else { 0 } + Add-Finding -Category 'unexpected-documentation-element' -Path $entry.Path ` + -LineNumber $lineNumber -Member $member -Message ( + "Documentation for '$member' contains unexpected top-level container " + + "'<$($element.Name.LocalName)>' around '<$($nestedDocumentationElement.Name.LocalName)>'. " + + 'Check whether an path selected a member container instead of that ' + + "member's documentation elements.") + } + } +} + # The documentation build discards on an enum field, rendering only the summary. Text # written there is therefore lost without trace: it neither appears on the published page nor is # reported by anything the author sees. Fold it into the field's instead. diff --git a/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml b/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml index 46d79f24fd..879aaadad1 100644 --- a/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml +++ b/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml @@ -72,7 +72,9 @@ parameters: # The error severity covers malformed-xml, unresolved-cref, invalid-docid, unknown-namespace-root, # stale-allowlist-entry, missing-documentation, the two public resolution categories above, and # the three package layout categories (lib-documentation-trimmed, ref-documentation-untrimmed, - # lib-ref-documentation-identical). + # lib-ref-documentation-identical). It also covers documentation content that the compiler or + # Learn would silently discard: enum-field-remarks, unresolved-include and + # unexpected-documentation-element. # # The warning severity covers mismatched-docid-prefix, unexpected-documentation and # missing-external-uid. missing-local-uid, documentation-not-expected and unprefixed-cref are diff --git a/src/Microsoft.Data.SqlClient.Extensions/Azure/src/ActiveDirectoryAuthenticationProviderOptions.cs b/src/Microsoft.Data.SqlClient.Extensions/Azure/src/ActiveDirectoryAuthenticationProviderOptions.cs index 1573eb7f55..2097b859d6 100644 --- a/src/Microsoft.Data.SqlClient.Extensions/Azure/src/ActiveDirectoryAuthenticationProviderOptions.cs +++ b/src/Microsoft.Data.SqlClient.Extensions/Azure/src/ActiveDirectoryAuthenticationProviderOptions.cs @@ -6,7 +6,7 @@ namespace Microsoft.Data.SqlClient { - /// + /// public sealed class ActiveDirectoryAuthenticationProviderOptions { /// diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/Diagnostics/SqlClientConnectionCloseBefore.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/Diagnostics/SqlClientConnectionCloseBefore.cs index c891e8baaf..05d69383fe 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/Diagnostics/SqlClientConnectionCloseBefore.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/Diagnostics/SqlClientConnectionCloseBefore.cs @@ -11,7 +11,7 @@ namespace Microsoft.Data.SqlClient.Diagnostics /// public sealed class SqlClientConnectionCloseBefore : IReadOnlyList> { - /// + /// public const string Name = "Microsoft.Data.SqlClient.WriteConnectionCloseBefore"; internal SqlClientConnectionCloseBefore( diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/OnChangedEventHandler.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/OnChangedEventHandler.cs index 25b54a06f4..30b426b7d6 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/OnChangedEventHandler.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/OnChangedEventHandler.cs @@ -4,7 +4,6 @@ namespace Microsoft.Data.SqlClient { - /// + /// public delegate void OnChangeEventHandler(object sender, SqlNotificationEventArgs e); } - diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHintCollection.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHintCollection.cs index 8ed311a4cd..8d4923f4a5 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHintCollection.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopyColumnOrderHintCollection.cs @@ -36,6 +36,9 @@ public SqlBulkCopyColumnOrderHint Add(SqlBulkCopyColumnOrderHint columnOrderHint /// public SqlBulkCopyColumnOrderHint Add(string column, SortOrder sortOrder) => Add(new SqlBulkCopyColumnOrderHint(column, sortOrder)); + /// + public new void Clear() => base.Clear(); + /// /// Invoked before the collection is cleared using Clear(). Unregisters each order hint. /// @@ -83,6 +86,9 @@ public void Remove(SqlBulkCopyColumnOrderHint columnOrderHint) List.Remove(columnOrderHint); } + /// + public new void RemoveAt(int index) => base.RemoveAt(index); + /// /// Invoked before the order hint is removed using Remove() or RemoveAt(). Unregisters the order hint. /// diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlDbTypeExtensions.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlDbTypeExtensions.cs index 0b6780f900..fe38e98bc9 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlDbTypeExtensions.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlDbTypeExtensions.cs @@ -9,13 +9,13 @@ namespace Microsoft.Data /// public static class SqlDbTypeExtensions { - /// + /// #if NET9_0_OR_GREATER public const SqlDbType Json = SqlDbType.Json; #else public const SqlDbType Json = (SqlDbType)35; #endif - /// + /// #if NET10_0_OR_GREATER public const SqlDbType Vector = SqlDbType.Vector; #else From 95bca3305a705ea30123a9d0739ed73c224a18f4 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Fri, 25 Sep 2026 09:26:37 -0300 Subject: [PATCH 21/23] Address PR #4752 review feedback Enable XML snippet source validation for the Abstractions and Azure package jobs, and restrict the documented CDP browser origin. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../documentation.instructions.md | 5 +++- .../onebranch-pipeline-design.instructions.md | 2 +- .../onebranch/jobs/build-buildproj-job.yml | 12 ++++++-- .../onebranch/scripts/tests/README.md | 1 + .../tests/pipeline-invocation.Tests.ps1 | 28 +++++++++++++++++++ .../onebranch/stages/build-stages.yml | 4 +++ 6 files changed, 47 insertions(+), 5 deletions(-) diff --git a/.github/instructions/documentation.instructions.md b/.github/instructions/documentation.instructions.md index 5284da7788..79b5974d0f 100644 --- a/.github/instructions/documentation.instructions.md +++ b/.github/instructions/documentation.instructions.md @@ -208,11 +208,14 @@ The review site requires Microsoft authentication. On a corp-joined Windows devi ```bash EDGE="/mnt/c/Program Files (x86)/Microsoft/Edge/Application/msedge.exe" -"$EDGE" --remote-debugging-port=9222 --remote-allow-origins=* \ +"$EDGE" --remote-debugging-port=9222 --remote-allow-origins=http://localhost:9222 \ --user-data-dir=C:\\Temp\\edge-cdp-profile \ --no-first-run --no-default-browser-check about:blank ``` +Omit `--remote-allow-origins` when the CDP client sends no `Origin` header. Otherwise, allow only +the exact origin used by that client; never use a wildcard with an authenticated browser profile. + Drive the browser from the Windows side because the Windows firewall can block WSL-to-Windows access to the debugging port: ```bash diff --git a/.github/instructions/onebranch-pipeline-design.instructions.md b/.github/instructions/onebranch-pipeline-design.instructions.md index bfbada84f9..d2618b1e79 100644 --- a/.github/instructions/onebranch-pipeline-design.instructions.md +++ b/.github/instructions/onebranch-pipeline-design.instructions.md @@ -31,7 +31,7 @@ Respect this graph when modifying build stages: Validation runs in three places and shares one gating switch. - **Localization** — `steps/validate-localization-step.yml`, in the SqlClient build job before the driver is built. Reports missing or obsolete keys, empty localized values whose English value is non-empty, and untranslated resources. Approved identical translations are listed by culture and resource key in `.config/LocalizationValidationAllowlist.json`. -- **XML documentation** — `steps/validate-xml-docs-step.yml`, three times per run: snippet sources before the build, generated documentation after it, and the assembled packages in `package_validation`. Reports malformed documentation IDs, unresolved cross-references, and `lib/` vs `ref/` documentation-trimming defects. +- **XML documentation** — `steps/validate-xml-docs-step.yml`, in three modes: snippet sources before each snippet-consuming project is built (SqlClient, SqlServer, Abstractions, and Azure), generated documentation after documented projects are built, and assembled packages in `package_validation`. Reports malformed documentation IDs, unresolved cross-references, and `lib/` vs `ref/` documentation-trimming defects. - **Packages** — `steps/validate-packages-step.yml`, in `package_validation`. Runs `tools/PackageValidator` across the whole drop so its cross-package version and dependency rules apply. ### `failOnValidationError` diff --git a/eng/pipelines/onebranch/jobs/build-buildproj-job.yml b/eng/pipelines/onebranch/jobs/build-buildproj-job.yml index 1846992988..a716bacfd7 100644 --- a/eng/pipelines/onebranch/jobs/build-buildproj-job.yml +++ b/eng/pipelines/onebranch/jobs/build-buildproj-job.yml @@ -39,6 +39,12 @@ parameters: - name: projectPath type: string + # Directory containing the documentation snippets referenced by this project. An empty path + # disables source validation for projects that do not consume snippets. + - name: documentationSnippetsPath + type: string + default: '' + # Signing Parameters ----------------------------------------------------- # @TODO: Signing Parameters Object @@ -158,12 +164,12 @@ jobs: # cross-reference is reported in seconds against the file and line a developer edits, rather # than surfacing much later as an xref-not-found warning during API docs ingestion. # - # Restricted to the packages whose sources incorporate doc/snippets content. - - ${{ if or(eq(parameters.packageShortName, 'SqlClient'), eq(parameters.packageShortName, 'SqlServer')) }}: + # Enabled only for projects whose callers identify a documentation snippet directory. + - ${{ if ne(parameters.documentationSnippetsPath, '') }}: - template: /eng/pipelines/onebranch/steps/validate-xml-docs-step.yml@self parameters: displayName: 'Validate XML documentation sources' - snippetsDirectory: '$(Build.SourcesDirectory)/doc/snippets' + snippetsDirectory: '${{ parameters.documentationSnippetsPath }}' documentationPath: '' packagesPath: '' extractPath: '' diff --git a/eng/pipelines/onebranch/scripts/tests/README.md b/eng/pipelines/onebranch/scripts/tests/README.md index 49d92f429c..e43b324978 100644 --- a/eng/pipelines/onebranch/scripts/tests/README.md +++ b/eng/pipelines/onebranch/scripts/tests/README.md @@ -45,6 +45,7 @@ Invoke-Pester ./publish-symbols.Tests.ps1 -Output Detailed | XML docs enum remarks | `` on an enum field reported, since the documentation build discards it; type blocks and platform-variant type blocks accepted; members of non-enum types accepted; enum members read from source through attributes, initializers and comments; in generated documentation only public fields reported, and none when the public API surface is unknown | | XML docs unresolved includes | An `` surviving into generated documentation reported, since the compiler leaves it in place and the member ships undocumented; the requested path echoed back; every occurrence reported; includes inside snippets ignored | | XML docs unexpected elements | Member containers emitted by an over-broad `` reported; compiler-supported top-level documentation elements accepted | +| XML docs pipeline invocation | Source validation enabled by each job's configured snippet path; SqlClient, SqlServer, Abstractions, and Azure snippet directories wired and present | | Dependency documentation restore | Exact version pinned, central package management not inherited, documentation collected per target framework, restore tree removed, stale destination replaced, and failures reported for a failed restore, a missing package folder, or a package shipping no documentation | | Package signatures | Every package and symbol package verified, all failures reported before throwing | | Assembly signatures | Package expansion, native binaries under `runtimes/` included, stale expansions replaced, all unsigned assemblies reported | diff --git a/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 index d36b1af222..3407b94f90 100644 --- a/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 @@ -14,6 +14,8 @@ BeforeAll { $script:repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..' '..' '..' '..' '..')).Path + $script:jobsPath = Join-Path $script:repoRoot 'eng/pipelines/onebranch/jobs' + $script:stagesPath = Join-Path $script:repoRoot 'eng/pipelines/onebranch/stages' $script:stepsPath = Join-Path $script:repoRoot 'eng/pipelines/onebranch/steps' $script:scriptsPath = Join-Path $script:repoRoot 'eng/pipelines/onebranch/scripts' @@ -89,6 +91,32 @@ Describe 'Validation step templates' { $content = Get-Content -LiteralPath (Join-Path $script:stepsPath $Template) -Raw $content | Should -Match 'Unexpected failOnValidationError value' } + + It 'enables source validation from the configured snippet path' { + $content = Get-Content -LiteralPath (Join-Path $script:jobsPath 'build-buildproj-job.yml') -Raw + + $content | Should -Match "\$\{\{ if ne\(parameters\.documentationSnippetsPath, ''\) \}\}" + $content | Should -Match "snippetsDirectory: '\$\{\{ parameters\.documentationSnippetsPath \}\}'" + } + + It 'configures every snippet-consuming project with an existing directory' { + $content = Get-Content -LiteralPath (Join-Path $script:stagesPath 'build-stages.yml') -Raw + $expectedPaths = @( + 'doc/snippets' + 'src/Microsoft.Data.SqlClient.Extensions/Abstractions/doc' + 'src/Microsoft.Data.SqlClient.Extensions/Azure/doc' + ) + $matches = [regex]::Matches( + $content, + "documentationSnippetsPath: '\`$\(REPO_ROOT\)/([^']+)'") + $configuredPaths = @($matches | ForEach-Object { $_.Groups[1].Value }) + + $configuredPaths.Count | Should -Be 4 + foreach ($path in $expectedPaths) { + $configuredPaths | Should -Contain $path + Test-Path -LiteralPath (Join-Path $script:repoRoot $path) | Should -BeTrue + } + } } Describe 'Validation scripts under the pipeline invocation form' { diff --git a/eng/pipelines/onebranch/stages/build-stages.yml b/eng/pipelines/onebranch/stages/build-stages.yml index 78971cc1e7..74ee4d416c 100644 --- a/eng/pipelines/onebranch/stages/build-stages.yml +++ b/eng/pipelines/onebranch/stages/build-stages.yml @@ -139,6 +139,7 @@ stages: shouldSignPackage: ${{ parameters.isOfficial }} failOnValidationError: ${{ parameters.failOnValidationError }} projectPath: '$(REPO_ROOT)/src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj' + documentationSnippetsPath: '$(REPO_ROOT)/doc/snippets' signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' @@ -182,6 +183,7 @@ stages: shouldSignPackage: ${{ parameters.isOfficial }} failOnValidationError: ${{ parameters.failOnValidationError }} projectPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient.Extensions/Abstractions/src/Abstractions.csproj' + documentationSnippetsPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient.Extensions/Abstractions/doc' signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' @@ -230,6 +232,7 @@ stages: shouldSignPackage: ${{ parameters.isOfficial }} failOnValidationError: ${{ parameters.failOnValidationError }} projectPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj' + documentationSnippetsPath: '$(REPO_ROOT)/doc/snippets' signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' @@ -270,6 +273,7 @@ stages: shouldSignPackage: ${{ parameters.isOfficial }} failOnValidationError: ${{ parameters.failOnValidationError }} projectPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient.Extensions/Azure/src/Azure.csproj' + documentationSnippetsPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient.Extensions/Azure/doc' signingAppRegistrationClientId: '${{ parameters.signingAppRegistrationClientId }}' signingAppRegistrationTenantId: '${{ parameters.signingAppRegistrationTenantId }}' signingAuthAkvName: '${{ parameters.signingAuthAkvName }}' From 46932e73b93b4a771a5e9b20bc5fa1079e86bb7f Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Fri, 25 Sep 2026 11:07:46 -0300 Subject: [PATCH 22/23] Validate generated reference XML documentation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../documentation.instructions.md | 5 ++- .../onebranch-pipeline-design.instructions.md | 2 +- .../onebranch/jobs/build-buildproj-job.yml | 18 ++++++++ .../onebranch/scripts/tests/README.md | 4 +- .../tests/pipeline-invocation.Tests.ps1 | 23 ++++++++++ .../scripts/tests/validate-xml-docs.Tests.ps1 | 45 +++++++++++++++++++ .../Microsoft.Data.SqlClient.Diagnostics.cs | 2 +- .../ref/Microsoft.Data.SqlClient.cs | 2 +- .../ref/Microsoft.Data.cs | 5 +-- 9 files changed, 97 insertions(+), 9 deletions(-) diff --git a/.github/instructions/documentation.instructions.md b/.github/instructions/documentation.instructions.md index 79b5974d0f..4619c4e14d 100644 --- a/.github/instructions/documentation.instructions.md +++ b/.github/instructions/documentation.instructions.md @@ -169,9 +169,12 @@ To also resolve references against the members the build actually emitted, which ```powershell dotnet build ./src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj -c Release -./eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 -DocumentationPath ./artifacts/Microsoft.Data.SqlClient.ref +./eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 -DocumentationPath ./artifacts/Microsoft.Data.SqlClient.ref/Project-Release ``` +Use the configuration-specific directory so stale outputs from another reference mode or build +configuration are not included in the result. + Validation is offline by design; it needs no network access and no xref map download. ### Validating Learn Preview Output diff --git a/.github/instructions/onebranch-pipeline-design.instructions.md b/.github/instructions/onebranch-pipeline-design.instructions.md index d2618b1e79..c0a2b716cf 100644 --- a/.github/instructions/onebranch-pipeline-design.instructions.md +++ b/.github/instructions/onebranch-pipeline-design.instructions.md @@ -31,7 +31,7 @@ Respect this graph when modifying build stages: Validation runs in three places and shares one gating switch. - **Localization** — `steps/validate-localization-step.yml`, in the SqlClient build job before the driver is built. Reports missing or obsolete keys, empty localized values whose English value is non-empty, and untranslated resources. Approved identical translations are listed by culture and resource key in `.config/LocalizationValidationAllowlist.json`. -- **XML documentation** — `steps/validate-xml-docs-step.yml`, in three modes: snippet sources before each snippet-consuming project is built (SqlClient, SqlServer, Abstractions, and Azure), generated documentation after documented projects are built, and assembled packages in `package_validation`. Reports malformed documentation IDs, unresolved cross-references, and `lib/` vs `ref/` documentation-trimming defects. +- **XML documentation** — `steps/validate-xml-docs-step.yml`, in three modes: snippet sources before each snippet-consuming project is built (SqlClient, SqlServer, Abstractions, and Azure), generated documentation after documented projects are built (including the separate SqlClient reference-assembly output), and assembled packages in `package_validation`. Reports malformed documentation IDs, unresolved cross-references, and `lib/` vs `ref/` documentation-trimming defects. - **Packages** — `steps/validate-packages-step.yml`, in `package_validation`. Runs `tools/PackageValidator` across the whole drop so its cross-package version and dependency rules apply. ### `failOnValidationError` diff --git a/eng/pipelines/onebranch/jobs/build-buildproj-job.yml b/eng/pipelines/onebranch/jobs/build-buildproj-job.yml index a716bacfd7..cda5dbdc32 100644 --- a/eng/pipelines/onebranch/jobs/build-buildproj-job.yml +++ b/eng/pipelines/onebranch/jobs/build-buildproj-job.yml @@ -250,6 +250,24 @@ jobs: projectPath: '${{ parameters.projectPath }}' projectSearchRoot: '' + # SqlClient builds its reference assembly into a sibling output tree. Validate it separately + # before packing so malformed ref documentation is attributed to the source build rather than + # surfacing only after the assembled package is extracted in package_validation. + - ${{ if eq(parameters.packageShortName, 'SqlClient') }}: + - template: /eng/pipelines/onebranch/steps/validate-xml-docs-step.yml@self + parameters: + displayName: 'Validate generated reference XML documentation' + snippetsDirectory: '' + documentationPath: '$(BUILD_OUTPUT)/Microsoft.Data.SqlClient.ref/Package-Release' + packagesPath: '' + extractPath: '' + reportPath: '$(JOB_OUTPUT)/validation/xml-docs-generated-ref.json' + failOn: + - error + failOnValidationError: ${{ parameters.failOnValidationError }} + projectPath: '$(REPO_ROOT)/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj' + projectSearchRoot: '' + - ${{ if eq(parameters.shouldSignPackage, true) }}: # ESRP sign the DLLs. - template: /eng/pipelines/onebranch/steps/esrp-dll-signing-step.yml@self diff --git a/eng/pipelines/onebranch/scripts/tests/README.md b/eng/pipelines/onebranch/scripts/tests/README.md index e43b324978..c3952580b0 100644 --- a/eng/pipelines/onebranch/scripts/tests/README.md +++ b/eng/pipelines/onebranch/scripts/tests/README.md @@ -44,8 +44,8 @@ Invoke-Pester ./publish-symbols.Tests.ps1 -Output Detailed | XML docs dependencies | References into a package that was not built this run resolve against the published dependency's documentation; that documentation is not itself validated, does not establish the public API surface, and does not enable resolution when nothing is under validation | | XML docs enum remarks | `` on an enum field reported, since the documentation build discards it; type blocks and platform-variant type blocks accepted; members of non-enum types accepted; enum members read from source through attributes, initializers and comments; in generated documentation only public fields reported, and none when the public API surface is unknown | | XML docs unresolved includes | An `` surviving into generated documentation reported, since the compiler leaves it in place and the member ships undocumented; the requested path echoed back; every occurrence reported; includes inside snippets ignored | -| XML docs unexpected elements | Member containers emitted by an over-broad `` reported; compiler-supported top-level documentation elements accepted | -| XML docs pipeline invocation | Source validation enabled by each job's configured snippet path; SqlClient, SqlServer, Abstractions, and Azure snippet directories wired and present | +| XML docs unexpected elements | Member containers emitted by an over-broad `` reported, including an end-to-end compiler expansion; compiler-supported top-level documentation elements accepted | +| XML docs pipeline invocation | Source validation enabled by each job's configured snippet path; SqlClient, SqlServer, Abstractions, and Azure snippet directories wired and present; generated SqlClient reference documentation validated after build and before packing | | Dependency documentation restore | Exact version pinned, central package management not inherited, documentation collected per target framework, restore tree removed, stale destination replaced, and failures reported for a failed restore, a missing package folder, or a package shipping no documentation | | Package signatures | Every package and symbol package verified, all failures reported before throwing | | Assembly signatures | Package expansion, native binaries under `runtimes/` included, stale expansions replaced, all unsigned assemblies reported | diff --git a/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 index 3407b94f90..bc5bfcdf27 100644 --- a/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/pipeline-invocation.Tests.ps1 @@ -117,6 +117,29 @@ Describe 'Validation step templates' { Test-Path -LiteralPath (Join-Path $script:repoRoot $path) | Should -BeTrue } } + + It 'validates generated SqlClient reference documentation before packing' { + $content = Get-Content -LiteralPath (Join-Path $script:jobsPath 'build-buildproj-job.yml') -Raw + + $content | Should -Match "\$\{\{ if eq\(parameters\.packageShortName, 'SqlClient'\) \}\}" + $content | Should -Match "displayName: 'Validate generated reference XML documentation'" + $content | Should -Match "documentationPath: '\`$\(BUILD_OUTPUT\)/Microsoft\.Data\.SqlClient\.ref/Package-Release'" + $content | Should -Match "projectPath: '\`$\(REPO_ROOT\)/src/Microsoft\.Data\.SqlClient/ref/Microsoft\.Data\.SqlClient\.csproj'" + + $buildIndex = $content.IndexOf( + '/eng/pipelines/onebranch/steps/build-buildproj-step.yml@self', + [System.StringComparison]::Ordinal) + $referenceValidationIndex = $content.IndexOf( + "displayName: 'Validate generated reference XML documentation'", + [System.StringComparison]::Ordinal) + $packIndex = $content.IndexOf( + '/eng/pipelines/onebranch/steps/pack-buildproj-step.yml@self', + [System.StringComparison]::Ordinal) + + $buildIndex | Should -BeGreaterOrEqual 0 + $referenceValidationIndex | Should -BeGreaterThan $buildIndex + $packIndex | Should -BeGreaterThan $referenceValidationIndex + } } Describe 'Validation scripts under the pipeline invocation form' { diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index 19ab5ff330..fa72dfe910 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -1817,6 +1817,51 @@ namespace Contoso Context 'unexpected documentation elements' { + It 'rejects an over-broad include after the compiler expands it' { + $projectDirectory = New-TestDirectory + $projectPath = Join-Path $projectDirectory 'ReferenceDocs.csproj' + $snippetPath = Join-Path $projectDirectory 'ReferenceDocs.xml' + + @' + + + net8.0 + true + + +'@ | Set-Content -LiteralPath $projectPath -Encoding utf8 + + @' + + + + Sample type. + + + +'@ | Set-Content -LiteralPath $snippetPath -Encoding utf8 + + @' +/// +public class Sample { } +'@ | Set-Content -LiteralPath (Join-Path $projectDirectory 'Sample.cs') -Encoding utf8 + + & dotnet build $projectPath --configuration Release --nologo --ignore-failed-sources | + Out-Null + $LASTEXITCODE | Should -Be 0 + + $documentationPath = Join-Path $projectDirectory 'bin/Release/net8.0' + $report = Join-Path $projectDirectory 'report.json' + + { & $scriptPath -DocumentationPath $documentationPath -ReportPath $report } | + Should -Throw '*XML documentation validation failed with 1 issue*' + + $finding = (Get-Report -Path $report).Findings | Select-Object -First 1 + $finding.Category | Should -Be 'unexpected-documentation-element' + $finding.Member | Should -Be 'T:Sample' + $finding.Message | Should -BeLike '**' + } + It 'reports member containers emitted by an over-broad include' { # Selecting every child of copies the type and property containers into the # compiler output instead of their summary and remarks, so Learn receives no type docs. diff --git a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.Diagnostics.cs b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.Diagnostics.cs index edbda75d54..af234f58cd 100644 --- a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.Diagnostics.cs +++ b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.Diagnostics.cs @@ -119,7 +119,7 @@ public sealed class SqlClientConnectionCloseAfter : System.Collections.Generic.I /// public sealed class SqlClientConnectionCloseBefore : System.Collections.Generic.IReadOnlyList> { - /// + /// public const string Name = "Microsoft.Data.SqlClient.WriteConnectionCloseBefore"; /// public System.Guid OperationId => throw null; diff --git a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.cs b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.cs index 18f31f9aa6..7b8a60147f 100644 --- a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.cs +++ b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.cs @@ -13,7 +13,7 @@ public enum ApplicationIntent ReadWrite = 0 } -/// +/// public delegate void OnChangeEventHandler(object sender, Microsoft.Data.SqlClient.SqlNotificationEventArgs e); /// diff --git a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.cs b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.cs index 20aea8935f..c9966b3739 100644 --- a/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.cs +++ b/src/Microsoft.Data.SqlClient/ref/Microsoft.Data.cs @@ -21,10 +21,9 @@ private OperationAbortedException(System.Runtime.Serialization.SerializationInfo /// public static class SqlDbTypeExtensions { - /// + /// public const System.Data.SqlDbType Json = (System.Data.SqlDbType)35; - /// + /// public const System.Data.SqlDbType Vector = (System.Data.SqlDbType)36; } - From cc243909b753fb77f34dcf25f89efe16ed83b390 Mon Sep 17 00:00:00 2001 From: Paul Medynski <31868385+paulmedynski@users.noreply.github.com> Date: Fri, 25 Sep 2026 14:54:00 -0300 Subject: [PATCH 23/23] Validate inline API documentation xrefs Detect bare inline xrefs to parameterized methods before Open Publishing reports them, and correct the affected SqlBulkCopy documentation links. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../documentation.instructions.md | 17 ++++ .../SqlBulkCopyColumnOrderHintCollection.xml | 6 +- .../scripts/tests/validate-xml-docs.Tests.ps1 | 56 ++++++++++++- .../onebranch/scripts/validate-xml-docs.ps1 | 81 ++++++++++++++++++- .../steps/validate-xml-docs-step.yml | 9 ++- 5 files changed, 159 insertions(+), 10 deletions(-) diff --git a/.github/instructions/documentation.instructions.md b/.github/instructions/documentation.instructions.md index 4619c4e14d..38d6956f33 100644 --- a/.github/instructions/documentation.instructions.md +++ b/.github/instructions/documentation.instructions.md @@ -157,6 +157,23 @@ A `cref` may be written unqualified (``), in whi Array, pointer and by-reference markers are legal *inside* a member signature (`M:...Decrypt(System.Byte[])`); they are only invalid as the whole target of a `T:` reference. +Markdown inside `` uses `` tokens rather than `cref` +attributes. The compiler copies these tokens verbatim and cannot validate them. A parameterized +method has no bare UID, even when it has only one overload, so link to its overload page with the +URL-encoded wildcard `%2A`: + +```xml + + + + + +``` + +Types, properties, fields, events, and parameterless methods may continue to use their exact bare +UIDs. Generated-document validation indexes the emitted members and rejects a bare inline xref when +that index proves the target is a parameterized method. + ### Validating Cross-References Locally `eng/pipelines/onebranch/scripts/validate-xml-docs.ps1` enforces the rules above. It runs in the OneBranch build jobs against snippet sources, generated documentation, and the assembled packages, so run it before pushing documentation changes: diff --git a/doc/snippets/Microsoft.Data.SqlClient/SqlBulkCopyColumnOrderHintCollection.xml b/doc/snippets/Microsoft.Data.SqlClient/SqlBulkCopyColumnOrderHintCollection.xml index b94d1f2aeb..7b5e422c0e 100644 --- a/doc/snippets/Microsoft.Data.SqlClient/SqlBulkCopyColumnOrderHintCollection.xml +++ b/doc/snippets/Microsoft.Data.SqlClient/SqlBulkCopyColumnOrderHintCollection.xml @@ -101,7 +101,7 @@ Transact-SQL `INSERT … SELECT` statement to copy the data. method is most commonly used when you use a single instance to process more than one bulk copy operation. If you create column order hints for one bulk copy operation, you must clear the after the method and before processing the next bulk copy. +The method is most commonly used when you use a single instance to process more than one bulk copy operation. If you create column order hints for one bulk copy operation, you must clear the after the method and before processing the next bulk copy. Performing several bulk copies using the same instance will usually be more efficient from a performance point of view than using a separate for each operation. @@ -233,8 +233,8 @@ This code is provided to demonstrate the syntax for using **SqlBulkCopy** only. method is most commonly used when you use a single instance to process more than one bulk copy operation. If you create column order hints for one bulk copy operation, you must clear the after the method and before processing the next bulk copy. -You can clear the entire collection by using the method, or remove hints individually using the method or the method. +The method is most commonly used when you use a single instance to process more than one bulk copy operation. If you create column order hints for one bulk copy operation, you must clear the after the method and before processing the next bulk copy. +You can clear the entire collection by using the method, or remove hints individually using the method or the method. Performing several bulk copies using the same instance will usually be more efficient from a performance point of view than using a separate for each operation. diff --git a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 index fa72dfe910..d7cd88ec5a 100644 --- a/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 +++ b/eng/pipelines/onebranch/scripts/tests/validate-xml-docs.Tests.ps1 @@ -54,12 +54,20 @@ $body function New-DocumentationDirectory { param( [string[]]$Members = @('T:Microsoft.Data.SqlClient.SqlConnection'), - [string[]]$Crefs = @() + [string[]]$Crefs = @(), + [string[]]$InlineXrefs = @() ) $path = New-TestDirectory $memberXml = ($Members | ForEach-Object { " Doc." }) -join "`n" $crefXml = ($Crefs | ForEach-Object { " " }) -join "`n" + $inlineXrefXml = if ($InlineXrefs.Count -eq 0) { + '' + } + else { + $tokens = $InlineXrefs | ForEach-Object { "" } + "`n " + } @" @@ -68,7 +76,7 @@ $body $memberXml Sample. -$crefXml +$crefXml$inlineXrefXml @@ -704,6 +712,49 @@ Describe 'validate-xml-docs.ps1' { { & $scriptPath -SnippetsDirectory $snippets } | Should -Not -Throw } + It 'rejects a bare inline xref to a parameterized method' { + $target = 'Microsoft.Data.SqlClient.SqlBulkCopy.WriteToServer' + $docs = New-DocumentationDirectory ` + -Members @("M:$target(System.Data.DataTable)") ` + -InlineXrefs @($target) + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report -ReportOnly + + $findings = @((Get-Report -Path $report).Findings) + $findings.Count | Should -Be 1 + $findings[0].Category | Should -Be 'bare-parameterized-method-xref' + $findings[0].Message | Should -BeLike "**" + } + + It 'accepts an overload-family inline xref to a parameterized method' { + $target = 'Microsoft.Data.SqlClient.SqlBulkCopy.WriteToServer' + $docs = New-DocumentationDirectory ` + -Members @("M:$target(System.Data.DataTable)") ` + -InlineXrefs @("$target%2A") + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report + + @((Get-Report -Path $report).Findings) | Should -BeNullOrEmpty + } + + It 'accepts exact inline xrefs to types, properties and parameterless methods' { + $targets = @( + 'Microsoft.Data.SqlClient.SqlConnection', + 'Microsoft.Data.SqlClient.SqlConnection.ConnectionString', + 'Microsoft.Data.SqlClient.SqlBulkCopyColumnOrderHintCollection.Clear') + $docs = New-DocumentationDirectory ` + -Members @("T:$($targets[0])", "P:$($targets[1])", "M:$($targets[2])") ` + -InlineXrefs $targets + $report = Join-Path (New-TestDirectory) 'report.json' + + & $scriptPath -DocumentationPath $docs -ReportPath $report + + @((Get-Report -Path $report).Findings) | Should -BeNullOrEmpty + (Get-Report -Path $report).InlineXrefsValidated | Should -Be 3 + } + It 'ignores non-documentation XML found in a scanned directory' { $path = New-TestDirectory '' | Set-Content -LiteralPath (Join-Path $path 'build.xml') -Encoding utf8 @@ -1512,6 +1563,7 @@ Describe 'validate-xml-docs.ps1' { 'malformed-xml' = 'error' 'unresolved-cref' = 'error' 'invalid-docid' = 'error' + 'bare-parameterized-method-xref' = 'error' 'unknown-namespace-root' = 'error' 'stale-allowlist-entry' = 'error' 'lib-documentation-trimmed' = 'error' diff --git a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 index c982e64c5d..b668e612f0 100644 --- a/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 +++ b/eng/pipelines/onebranch/scripts/validate-xml-docs.ps1 @@ -134,6 +134,11 @@ to downgrade. unresolved-cref error Compiler could not bind the cref and emitted a "!:" prefix. invalid-docid error Documentation ID violates the documentation-ID grammar. + bare-parameterized-method-xref + error A Markdown token names a parameterized method + without a signature or the encoded overload wildcard %2A. + Open Publishing treats the bare name as an exact UID, which + does not exist for a parameterized method. unknown-namespace-root error Leading identifier is not an allowed namespace root. stale-allowlist-entry error Allowlisted cref no longer appears in any validated file. lib-documentation-trimmed @@ -252,6 +257,7 @@ $script:CategorySeverities = [ordered]@{ 'malformed-xml' = 'error' 'unresolved-cref' = 'error' 'invalid-docid' = 'error' + 'bare-parameterized-method-xref' = 'error' 'unknown-namespace-root' = 'error' 'stale-allowlist-entry' = 'error' 'lib-documentation-trimmed' = 'error' @@ -1278,6 +1284,7 @@ if ($documentationRequested -and $malformedByKind['documentation'] -eq 0) { $script:LocalUids = $null $script:LocalUidsByBody = [System.Collections.Generic.Dictionary[string, System.Collections.Generic.HashSet[string]]]::new( [System.StringComparer]::Ordinal) +$script:ParameterizedMethodNames = $null # Members that form the published API surface. A reference assembly contains the public API and # nothing else, so its documentation is exactly that set. Left null when no reference documentation @@ -1292,6 +1299,8 @@ $script:LocalNamespaces = $null if ($documentationDocuments.Count -gt 0) { $script:LocalUids = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) $script:LocalNamespaces = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::Ordinal) + $script:ParameterizedMethodNames = [System.Collections.Generic.HashSet[string]]::new( + [System.StringComparer]::Ordinal) # Dependency documentation is indexed alongside, so a cref into a package this run did not # build resolves. Gated on documentation being present rather than on the dependencies: with # nothing under validation there is no cref to resolve, and an index built from dependencies @@ -1318,6 +1327,13 @@ if ($documentationDocuments.Count -gt 0) { # told apart from one naming a member that was never emitted at all. if ($uid.Length -gt 2 -and $uid[1] -eq ':') { $body = $uid.Substring(2) + if ($uid[0] -eq 'M') { + $parameterList = $body.IndexOf('(') + if ($parameterList -gt 0) { + [void]$script:ParameterizedMethodNames.Add($body.Substring(0, $parameterList)) + } + } + if (-not $script:LocalUidsByBody.ContainsKey($body)) { $script:LocalUidsByBody[$body] = [System.Collections.Generic.HashSet[string]]::new( [System.StringComparer]::Ordinal) @@ -1412,6 +1428,68 @@ foreach ($entry in $documents) { } } +# Markdown documentation embedded in CDATA uses tokens rather than XML elements with +# cref attributes. The compiler copies these tokens verbatim, so it cannot bind or validate them. +# Open Publishing treats a bare method name as an exact UID. Parameterized methods have signatures +# in their UIDs, so their bare names resolve only when written as an overload-family reference +# ending in the URL-encoded wildcard %2A. +$inlineXrefCount = 0 +foreach ($entry in $documents) { + foreach ($textNode in $entry.Document.DescendantNodes()) { + if ($textNode -isnot [System.Xml.Linq.XText]) { + continue + } + + foreach ($match in [regex]::Matches($textNode.Value, '\s]+)>')) { + $inlineXrefCount++ + $target = $match.Groups[1].Value + + if ($ignoredCrefs.Contains($target)) { + [void]$observedIgnoredCrefs.Add($target) + continue + } + + # Source mode has no emitted member index, so it cannot distinguish a parameterless + # method, property or type from a parameterized method. Generated documentation mode + # provides that authority and is where this rule runs. + if ($null -eq $script:ParameterizedMethodNames -or + $target.EndsWith('%2A', [System.StringComparison]::OrdinalIgnoreCase) -or + $script:LocalUidsByBody.ContainsKey($target) -or + -not $script:ParameterizedMethodNames.Contains($target)) { + continue + } + + $member = '' + $ancestor = $textNode.Parent + while ($null -ne $ancestor) { + if ($ancestor.Name.LocalName -eq 'member') { + $nameAttribute = $ancestor.Attribute('name') + if ($null -ne $nameAttribute) { + $member = $nameAttribute.Value + } + break + } + if ($ancestor.Name.LocalName -eq 'members') { + $nameAttribute = $ancestor.Attribute('name') + if ($null -ne $nameAttribute) { + $member = $nameAttribute.Value + } + break + } + $ancestor = $ancestor.Parent + } + + $lineInfo = [System.Xml.IXmlLineInfo]$textNode + Add-Finding -Category 'bare-parameterized-method-xref' -Path $entry.Path ` + -LineNumber $(if ($lineInfo.HasLineInfo()) { $lineInfo.LineNumber } else { 0 }) ` + -Member $member -Cref $target -Message ( + "Inline xref '$target' names a parameterized method without a signature. " + + "Open Publishing resolves it as an exact UID, which does not exist. Use " + + "'' to link to the method's overload page.") + } + } +} + # A documentation comment pulls its text in with , and the compiler resolves it at compile # time. When the path matches nothing the compiler does not fail: it copies the unresolved # element into the output and moves on, so the member ships with no documentation at all @@ -1761,6 +1839,7 @@ if (-not [string]::IsNullOrWhiteSpace($ReportPath)) { DocumentationFiles = $documentationDocuments.Count DependencyDocumentationFiles = $dependencyDocuments.Count CrefsValidated = $crefCount + InlineXrefsValidated = $inlineXrefCount LocalUids = if ($null -ne $script:LocalUids) { $script:LocalUids.Count } else { 0 } CountsByCategory = $countsByCategory Findings = $findings @@ -1827,7 +1906,7 @@ foreach ($finding in $findings) { Write-Host "##vso[task.logissue type=$issueType$location]$($finding.Category): $detail" } -$summary = "XML documentation validation examined $crefCount cref(s) across $($documents.Count) file(s)." +$summary = "XML documentation validation examined $crefCount cref(s) and $inlineXrefCount inline xref(s) across $($documents.Count) file(s)." foreach ($category in $countsByCategory.Keys) { if ($countsByCategory[$category] -gt 0) { $summary += " $category=$($countsByCategory[$category]);" diff --git a/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml b/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml index 879aaadad1..590485caf2 100644 --- a/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml +++ b/eng/pipelines/onebranch/steps/validate-xml-docs-step.yml @@ -69,10 +69,11 @@ parameters: # mismatched-docid-prefix. A finding takes the lower form when no reference documentation # identified the surface, so an unknown surface is never escalated on a guess. # - # The error severity covers malformed-xml, unresolved-cref, invalid-docid, unknown-namespace-root, - # stale-allowlist-entry, missing-documentation, the two public resolution categories above, and - # the three package layout categories (lib-documentation-trimmed, ref-documentation-untrimmed, - # lib-ref-documentation-identical). It also covers documentation content that the compiler or + # The error severity covers malformed-xml, unresolved-cref, invalid-docid, + # bare-parameterized-method-xref, unknown-namespace-root, stale-allowlist-entry, + # missing-documentation, the two public resolution categories above, and the three package + # layout categories (lib-documentation-trimmed, ref-documentation-untrimmed, + # lib-ref-documentation-identical). It also covers documentation content that the compiler or # Learn would silently discard: enum-field-remarks, unresolved-include and # unexpected-documentation-element. #