Skip to content

Ad test consolidation - #2734

Draft
soulemike wants to merge 93 commits into
maester365:mainfrom
soulemike:ad-test-consolidation
Draft

soulemike wants to merge 93 commits into
maester365:mainfrom
soulemike:ad-test-consolidation

Conversation

@soulemike

@soulemike soulemike commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Important

Draft: this is the first AD consolidation and correctness batch. Another set of AD test updates and live AD testing will follow. This is not a request to merge during the Maester 3.0 code freeze.

📑 Description

Consolidate overlapping Active Directory count/detail checks and correct misleading assessment outcomes, incomplete-data handling, collection gaps, and remediation guidance.

Changes

  • Consolidate 50 redundant checks across GPO/GPOREP/GPOL, schema, optional features, RootDSE, DACL, computer, group membership, SPN and trust families. Keep existing surviving IDs to minimize file churn; 226 AD checks remain.
  • Separate Investigate inventories from actual pass/fail baselines. GPO age uses configurable last-modification age (five calendar years by default), with both creation and modification dates in the details.
  • Correct Recycle Bin status, configured versus effective tombstone lifetime, Windows LAPS schema readiness, DHCP authorization entries, and AD activation-object collection.
  • Cover sMSA, gMSA and dMSA delegation through the separate managed-account collection. Keep device percentages device-only and use verified DC inventory evidence rather than trusting primary groups 516/521.
  • Resolve direct, nested and primary-group Domain Admin membership for SPN assessment; fix account/SPN counts and remove silent group-count truncation.
  • Reject incomplete GPO version and password scans, including unreadable or malformed SYSVOL XML, without disclosing credential values. Declare Windows gating where permission parsing requires it.
  • Make KDS assessment visibility-aware: zero visible keys is Investigate unless complete inventory read/list access has been independently confirmed using KeyInventoryReadAccessConfirmed (default false). Do not recommend provisioning another key from an unconfirmed partial inventory.
  • Regenerate legacy-tag mappings from an explicit native-replacement registry and refresh affected guides and regression tests.

Compatibility and scope

  • Historical inclusion and exclusion tags map to the surviving merged check. An old exclusion can therefore suppress a broader control: -ExcludeTag AD-GPOREP-19 now excludes Critical AD-GPOREP-17, including both password checks. Upgrade guidance calls out the need to audit exclusions.
  • Failed Get-MtADGpoState metadata/report collection is represented by null fields, not an empty inventory. Custom tests must distinguish these cases. No new public command is exported.
  • Managed service accounts remain separate from device inventory but are included in delegation assessment.
  • Changes are split across 23 function-scoped commits, each affecting fewer than 25 files. No tenant writes or cache bypasses were added.

✅ Checks

  • My pull request adheres to the code style of this project.
  • My code requires changes to the documentation.
  • I have updated the documentation as required.
  • The build and unit tests pass after running /powershell/tests/pester.ps1 locally.

Validation

  • Final full module suite: 8,244 tests, zero failures, including repository lint, script analysis and native attribute checks.
  • Focused remediation regression/lint checks: 126 tests, zero failures.
  • Module build and built-output validation passed.
  • Native metadata: 226 AD checks, zero invalid.
  • SYSVOL malformed/unreadable XML transport regressions passed for both mocked Windows and Unix adapters.

ℹ️ Additional Information

Pending before ready-for-review

  • Apply the next planned AD test-update batch.
  • Run live AD validation, including visibility-restricted readers, real DC/RODC identity, managed-account delegation and SYSVOL access-failure paths.
  • Review the combined changes after the additional batch and live testing.

TRUST-06 remains Investigate: safe LDAP does not provide a reliable last successful trust-validation timestamp. It does not substitute whenChanged or retrieve protected trust authentication secrets. KDS visibility confirmation is an operator-attested prerequisite, not an automatic effective-permission probe.

Offline passing tests are not evidence of live tenant behavior. No live AD validation or successful post-remediation Opus approval is claimed. The external Anthropic attribute-review workflow has not been run locally.

@coderabbitai

coderabbitai Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Attribute review

Attribute review: 42 tests need review, more than the limit of 25 for one run. Split the change, or ask a maintainer to review the attributes and apply the attribute-review-override label.

Snozz-Al and others added 24 commits November 8, 2026 20:03
@julien-ly

Copy link
Copy Markdown
Contributor

In tests/ad/replication/Test.AD-FEAT-01.ps1 (line 70 at bf1709d), the enabled scopes are joined with <br>. The report shows raw HTML as text (Markdown.tsx escapes it), so a feature with more than one enabled scope gets a literal <br> between its DNs, the same problem this PR removes from AD-OU-01. Since DNs contain commas, ; would keep them readable in one cell; it's the separator #2742 uses for the Conditional Access references in MT.1196, MT.1197 and MT.1265. The current AD-FEAT-01 tests use one scope per feature, so they don't hit this; the Should -Not -Match '<br>' assertion used for AD-OU-01 in ADSiteOuBatch.Tests.ps1 could cover AD-FEAT-01 with a two-scope feature.

@earbona23 earbona23 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the largest change open on the repository right now — 563 files, net about 10,600 lines removed — so I measured its shape before reading any of it, and there is one number I think needs answering before it merges.

Disclosure first: this PR rewrites tests/ad/domain/Test.AD-FOR-03.ps1, which is the subject of my own open #2699. I am an interested party on that one file, and I have put that part last and framed it as a decision for a maintainer rather than a request.

What it does, measured

AD check files under tests/ad:      270 on main  ->  180 here
check IDs removed:                   90
check IDs added:                      0
Maester.LegacyTags.json:             +559 / -194
Maester.LegacyIds.json:              untouched

Maester.LegacyIds.json being untouched is correct, and I checked rather than assumed: it contains zero AD-* IDs, on main and here, so removing AD checks is not a trigger for Export-MtLegacyIdTable.ps1 and LegacyIds.Tests.ps1 has nothing to say about it. I had this down as a likely problem and it is not one.

The crosswalk is the right idea, and it covers a quarter of the removals

Where you do it, you do it properly. Test.AD-FOR-03.md:

AD-CFG-01 was consolidated into this check. […] Former AD-CFG-01 is covered here. The single primary Feature remains directory-configuration; forest context is linked through AD-FOR-01 and AD-FOR-04 rather than a second feature.

and the same crosswalk repeated in the result text itself, where an operator reading a report will see it. That makes a consolidation auditable, which is exactly what a 90-check removal needs.

But of the 90 removed IDs, only 23 are mentioned anywhere under tests/ad/ in this branch. 67 are removed with no note in any surviving check, including:

AD-CFG-14  AD-CFG-16  AD-CFG-17  AD-CFG-18  AD-CFG-19  AD-CFG-24
AD-COMP-09 AD-COMP-10 AD-DACL-04 AD-DACL-06 AD-DACL-08 AD-DACL-10
AD-DACL-12 AD-DACL-16 AD-DACL-18 AD-DC-06   AD-DC-08   AD-DCD-02   …

To be precise about what that does and does not mean: it is a statement about auditability, not about lost coverage. I spot-checked AD-DACL-08 ("DACL ACE distribution per identity should be investigated") and the branch still carries GenericAll / WriteDacl / WriteOwner logic in two files, so DACL coverage plainly survives in some form — I just cannot tell from the diff whether that particular inventory did. Neither can anyone else, and that is the problem with 67 of them: a reviewer cannot distinguish "folded into the family above" from "dropped as redundant" without re-deriving each one.

You already have the mechanism. Extending the crosswalk line to the other 67 — even as a single table in one place rather than per check — turns a 563-file diff from an act of faith into something a maintainer can approve in an afternoon. It is also what a user upgrading needs: an ID that disappears without a trace is a silent gap in whatever they had pinned to it.

Coordination: this conflicts with five other open PRs

#2734 <-> #2783  conflict (3 files)      #2734 <-> #2785  conflict (1 file)
#2734 <-> #2784  conflict (1 file)       #2734 <-> #2786  conflict (2 files)
#2734 <-> #2699  conflict (Test.AD-FOR-03.ps1)

And those four also conflict with each other (matrix on #2783). Five open PRs in tests/ad that pairwise collide is a scheduling decision for a maintainer, not something any one author can fix — but this PR is the one that should land first or last, never in the middle, because it is the one that moves 563 files.

The AD-FOR-03 collision is a decision, not a rebase

Two open PRs make opposite choices about the same check:

  • this one: "It always reports Investigate when that inventory is available; advisory comparison does not create a retention pass/fail threshold."
  • my #2699: $testResult = $null -ne $tombstoneLifetime — a pass only when the forest actually returned a lifetime, and an explicit "not set" row when it did not, which is what #2468 was about for this check.

Both are defensible. They are not compatible, and whichever merges second silently replaces the other's model with no conflict marker in the result anyone reads. @merill has reviewed mine and asked for a narrower change, so there is a maintainer view in flight already.

I am not asking you to adopt mine. I am asking that this one file be settled deliberately — if the consolidation's "always Investigate" is the intended end state for AD-FOR-03, say so and I will close #2699 and move the $null handling into whatever shape this check ends up with. What I would rather avoid is finding out which model shipped by reading the merge order.

One thing I filed an hour ago that this PR touches

#2796: no check under tests/ad/ escapes directory data in its result markdown — 0 of 270, with 77 putting a name-like attribute into a table row. This PR rewrites most of that surface, so it is either the cheapest moment to fix the class or the moment that makes the sweep conflict with everything. Your call which, but worth knowing it exists before the rebase.

@FiraSenax

Copy link
Copy Markdown
Contributor

Heads-up: #2812 builds on this PR’s certificate inventory/parser to add offline NTAuth CA certificate-path validation. It is blocked on this PR merging and will be rebased afterward. No additional changes are requested here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants