Skip to content

Complete shadow reporter parity coverage - #5992

Merged
Sean Larkin (TheLarkInn) merged 28 commits into
mainfrom
copilot/reporter-r3c-shadow-parity
Sep 11, 2026
Merged

Complete shadow reporter parity coverage#5992
Sean Larkin (TheLarkInn) merged 28 commits into
mainfrom
copilot/reporter-r3c-shadow-parity

Conversation

@TheLarkInn

@TheLarkInn Sean Larkin (TheLarkInn) commented Aug 28, 2026

Copy link
Copy Markdown
Member

Part of #5976

Stack

#5985 copilot/reporter-r1a-package-wiring
  -> #5986 copilot/reporter-r1b-bootstrap-generation
    -> #5987 copilot/reporter-r2a-experiment-config
      -> #5989 copilot/reporter-r2b-frontend-host-controls
        -> #5988 copilot/reporter-r3a-session-sink
          -> #5991 copilot/reporter-r3b-shadow-events
            -> this PR #5992 copilot/reporter-r3c-shadow-parity

Complete merge order: #5985 -> #5986 -> #5987 -> #5989 -> #5988 -> #5991 -> #5992.

R6 #5993 is a parallel fork from #5989 and is not part of this child chain.

Summary

  • add exhaustive shadow parity coverage for event ordering, correlation, trusted source identity, and privacy-safe telemetry projection
  • prove derived exit status matches success, warning-only success, operation failure, logical cancellation, configuration errors, and internal errors
  • verify project x phase operation identities remain distinct and stable across repeated watch-style iterations
  • prove stdout, stderr, and legacy StreamCollator rendering remain byte-identical while shadow reporting emits no raw output
  • cover no-sink/legacy frontend behavior by verifying scoped producers remain absent unless the optional engine sink is supplied
  • fix the shadow exit adapter so callers can preserve the canonical logical-cancellation outcome without changing authoritative process exit behavior

Independent stack review

  • Expose scoped RushSession reporter producers #5988 is narrowly scoped to the optional frontend-to-engine sink and trusted Rush/action/plugin producer facades; no identity spoofing, privacy leak, output change, or compatibility defect was found
  • Emit shadow Rush lifecycle events #5991 keeps legacy rendering and process exit authoritative, does not route operation chunks, and correctly correlates emitted operation diagnostics with legacy errors
  • the missing cancellation derivation and exhaustive parity matrix are addressed here rather than expanding either parent slice
  • telemetry assertions intentionally verify the Harden reporter telemetry privacy projection #5990 privacy boundary without duplicating its reporter-package implementation

Validation

  • focused Rush-lib reporter parity tests: 41 passed
  • rush test --only @rushstack/rush-reporter --only @microsoft/rush-lib --only @microsoft/rush --only @rushstack/rush-sdk (8 operations passed)
  • rush build --to @microsoft/rush --to @rushstack/rush-sdk (52 operations passed)
  • rush check
  • rush change --verify --no-fetch

No-output guarantee

No sink is created unless explicitly supplied. The shadow operation adapter emits lifecycle/status/diagnostic events only, never externalOutput; focused tests compare stdout, stderr, and the complete legacy StreamCollator rendering byte-for-byte with and without shadow reporting.

Non-goals

  • no reporter selection, CLI flags, repository defaults, or automatic activation
  • no StreamCollator replacement or R5 raw operation/output routing
  • no duplication of Harden reporter telemetry privacy projection #5990 telemetry-subscriber privacy hardening
  • no changes to legacy terminal/logger APIs or authoritative process exit behavior
  • no child-to-parent merges while stack ancestors remain open

@TheLarkInn

Copy link
Copy Markdown
Member Author

Combined deep review of current head 22089a0957.

No critical, high, or medium actionable findings. No useful low follow-ons.

@TheLarkInn
Sean Larkin (TheLarkInn) force-pushed the copilot/reporter-r3c-shadow-parity branch from 22089a0 to 7a751fc Compare August 28, 2026 17:13
@TheLarkInn

Copy link
Copy Markdown
Member Author

Restacked onto exact #5991 head 2f9ac7d90e365143c0b33da2e8dd9782fda99e35. New #5992 head: 7a751fc03f0a089f0279e8fb50000733a02fc47e. The PR is one R3C commit, four files, CLEAN, mergeable, and auto-merge remains disabled. The R3C parity additions were preserved while overlapping sharded and watch behavior remains owned by #5991. Combined stack validation passed.

@TheLarkInn

Copy link
Copy Markdown
Member Author

Final combined review of 7a751fc03f found no critical, high, medium, or useful low issues. The restack preserves parity, cancellation, ordering, privacy, exit authority, and byte-identical visible output. This head is review-clean.

@TheLarkInn
Sean Larkin (TheLarkInn) force-pushed the copilot/reporter-r3c-shadow-parity branch from 7a751fc to 2e9b7bb Compare September 3, 2026 17:25
@TheLarkInn
Sean Larkin (TheLarkInn) force-pushed the copilot/reporter-r3c-shadow-parity branch from 2e9b7bb to 7f92aaf Compare September 3, 2026 18:11
@TheLarkInn
Sean Larkin (TheLarkInn) force-pushed the copilot/reporter-r3c-shadow-parity branch from 7f92aaf to dcfb93c Compare September 3, 2026 18:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Production cancellation state is not propagated, and output parity does not verify stream identity.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 1 Medium severity · 1 Low severity

New issues introduced by this change (2)
Severity Finding
Medium severity libraries/​rush-lib/​src/​pluginFramework/​RushSession.ts — The cancellation override is not connected to a production cancellation path.…
Low severity libraries/​rush-lib/​src/​logic/​operations/​test/​OperationGraphEventSink.test.tsgetAllOutput() concatenates and ANSI-normalizes only chunk text, discarding TerminalChunkKind.…
What changed in this PR

Adds shadow reporter parity validation across lifecycle events, exit status, telemetry privacy, operation identity, and legacy output.

Changes:

  • Extends derived exit-status resolution with cancellation options.
  • Adds lifecycle, telemetry, watch-iteration, and output parity tests.
  • Adds the Rush patch change record.
File Description
RushSession.ts Extends shadow exit-status resolution.
RushSession.test.ts Adds lifecycle, privacy, and status coverage.
OperationGraphEventSink.test.ts Adds operation identity and output parity tests.
Change JSON Records the Rush patch change.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread libraries/rush-lib/src/pluginFramework/RushSession.ts
Comment thread libraries/rush-lib/src/logic/operations/test/OperationGraphEventSink.test.ts Outdated
Document both the typed event sink and the frontend-assigned sessionId in the cross-version handoff without changing its shape.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Share separated-value recognition with stripping so valueless controls cannot consume legacy flags, and use debug only as the unrequested primary file log-level default.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Retain selected-action cancellation for subsequent shadow observations while preserving legacy process and telemetry results. Exercise a native watch session and compare raw terminal chunks and per-stream bytes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Keep immutable errors intact, capture original pre-execution parser failures without changing legacy rendering, and observe final configured operation silence.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Consume --verbose only for known actions that do not define it and parse repository opt-in value controls only when they are not command-owned. Preserve native aliases, declared custom values, unresolved plugin namespaces, and pass-through arguments.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Preserve both original slice heads and combine their additive lifecycle/parity documentation. This stages the ready upper layers while the foundation update is finalized.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Keep the original R3A producer channel and documentation fix while carrying both frontend-owned value and flag stripping lists from the final foundation parent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Preserve the R3B owning error-correlation, parse-diagnostic, and configured-registration fixes while inheriting final command ownership and channel documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Carry final foundation controls without changing the owning cancellation-observer and raw terminal parity fixes or their old-head ancestry.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Preserve the exact published R2B slice and review corrections while reconciling native private members and replacing unbranded parser test objects with real execution paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Share separated-value recognition with stripping so valueless controls cannot consume legacy flags, and use debug only as the unrequested primary file log-level default.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Consume --verbose only for known actions that do not define it and parse repository opt-in value controls only when they are not command-owned. Preserve native aliases, declared custom values, unresolved plugin namespaces, and pass-through arguments.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Retain the exact scoped producer API and WeakMap-backed plugin facades while preserving native-private parser/plugin members and real launch-boundary coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Document both the typed event sink and the frontend-assigned sessionId in the cross-version handoff without changing its shape.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Preserve published early-failure, late-telemetry and operation-callback corrections; reconcile native lifecycle fields and telemetry references, with real branded parser regression coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Keep immutable errors intact, capture original pre-execution parser failures without changing legacy rendering, and observe final configured operation silence.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Preserve the published R3C slice and review fixes while incorporating the authoritative server-rebased immediate parent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Resolve physical cwd at parser entry so native Windows short names and directory aliases match Git repository paths. Keep real watch cancellation coverage and add symlink/junction regressions without mocking input analysis or watcher behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Native Node 24 and 26 validation showed that generic realpathSync and FileSystem.getRealPath retain 8.3 names. Use the existing native-realpath pattern to resolve the physical directory before configuration discovery.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Explicit pipes prevent Git line-ending notices from being mirrored onto the parent test stderr. Real setup failures still throw with the original captured error text. Reproduced the actual Rush production gate with process-local core.autocrlf=true and core.safecrlf=warn: unchanged tests exited with warnings before the fix and passed cleanly after it, without changing CI warning policy or watch assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Resolve both expected and actual link locations using native-backed realpath before comparing them. Add a real directory-alias regression that still rejects wrong and missing targets, and run it alongside the unchanged npm and Yarn integration workflows. Reproduced the previous lexical mismatch before the fix; the regression and complete suite pass under an invocation-owned aliased temporary root. No production code, dependency versions, or CI gates changed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Base automatically changed from copilot/reporter-r3b-shadow-events to main September 11, 2026 00:29
The newly landed parent tree is byte-identical to the already integrated parent. Preserve the entire reviewed slice tree and both parent histories while correcting ancestry after the protected stack-prefix squash landing. For R3C, exclude the 300 lines of duplicated old graph tests introduced by automatic merge; the final source, test, dependency and API trees are exactly unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
@TheLarkInn
Sean Larkin (TheLarkInn) merged commit a5a7fef into main Sep 11, 2026
10 checks passed
@github-project-automation github-project-automation Bot moved this from Needs triage to Closed in Bug Triage Sep 11, 2026
@TheLarkInn
Sean Larkin (TheLarkInn) deleted the copilot/reporter-r3c-shadow-parity branch September 11, 2026 05:30
Sean Larkin (TheLarkInn) added a commit that referenced this pull request Sep 11, 2026
Land the four remaining reviewed Reporter core slices (#5992, #5996, #5997, #5998) together after exact-head approval, zero unresolved discussions, and all six current CI runs with 36 complete platform workflows were verified. Preserve opt-in behavior and existing quality gates.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Closed

Development

Successfully merging this pull request may close these issues.

3 participants