Skip to content

Add managed LiveRamp RampID integration - #1054

Open
prk-Jr wants to merge 63 commits into
mainfrom
issue-355-liveramp-integration
Open

prk-Jr wants to merge 63 commits into
mainfrom
issue-355-liveramp-integration

Conversation

@prk-Jr

@prk-Jr prk-Jr commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add managed LiveRamp RampID support to Prebid so operators can configure IdentityLink centrally while preserving publisher User ID modules.
  • Carry opaque liveramp.com EIDs through the existing auction, consent, edge-cookie, and EC/KV paths, with generated-artifact coverage for exact default TCF behavior.
  • Keep ATS Direct audience segments and server-side identity resolution out of scope; live validation remains credential-gated.

Status

Code complete; live LiveRamp validation pending IABTechLab/uid2-optout#385.

Changes

File Change
.cargo/config.toml Add host-target clippy aliases for the CLI and OpenRTB codegen crates.
CLAUDE.md Include the new host-target aliases in the documented CI gates.
crates/trusted-server-cli/src/prebid_bundle.rs Reject LiveRamp bundle configuration that explicitly omits identityLinkIdSystem.
crates/trusted-server-core/src/consent/mod.rs Cover source-agnostic EID consent gating with multiple identity sources.
crates/trusted-server-core/src/ec/prebid_eids.rs Verify opaque LiveRamp envelopes survive bounded edge-cookie persistence.
crates/trusted-server-core/src/integrations/prebid.rs Add typed LiveRamp configuration, validation, defaults, and browser injection.
crates/trusted-server-js/lib/build-prebid-external.mjs Bundle IdentityLink and tcfControl with generated Prebid artifacts.
crates/trusted-server-js/lib/src/integrations/prebid/index.ts Install and protect the operator-managed IdentityLink entry while preserving publisher entries.
crates/trusted-server-js/lib/test/build-prebid-external.test.mjs Cover generated bundle metadata and explicit IdentityLink selection.
crates/trusted-server-js/lib/test/integrations/prebid/index.test.ts Cover ownership, queue ordering, API guards, EID transport, privacy, and idempotence.
crates/trusted-server-js/lib/test/integrations/prebid/user_id_modules.test.ts Cover the LiveRamp module mapping and default preset.
crates/trusted-server-js/lib/test/prebid-artifact-integration.test.mjs Exercise the real bundle and shim, including partial userSync updates and auction EID output.
crates/trusted-server-js/lib/test/prebid-consent-enforcement.test.mjs Verify Purpose 1, Purposes 3/4, and LiveRamp vendor consent independently against generated artifacts.
docs/guide/configuration.md Document LiveRamp settings, defaults, and validation constraints.
docs/guide/integrations/prebid.md Document setup, ownership limits, exact TCF defaults, network/storage effects, failure modes, and live validation.
docs/superpowers/plans/2026-08-21-liveramp-integration.md Record the reviewed implementation, correction, and verification plan.
docs/superpowers/specs/2026-08-21-liveramp-integration-design.md Specify architecture, consent behavior, security boundaries, tests, and excluded ATS Direct scope.
trusted-server.example.toml Add an example managed LiveRamp configuration.

Closes

Closes #355

Test plan

  • cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin
  • cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare && cargo clippy-cloudflare-wasm && cargo clippy-spin-native && cargo clippy-spin-wasm && cargo clippy-cli && cargo clippy-codegen
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run
  • JS build: cd crates/trusted-server-js/lib && node build-all.mjs
  • JS lint/format: cd crates/trusted-server-js/lib && npm run lint && npm run format
  • CLI tests: ./scripts/test-cli.sh
  • Docs format: cd docs && npm run format
  • WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing via fastly compute serve
  • Other: generated Prebid artifact tests cover managed configuration and the independent TCF matrix.

Hardening note

This change does not add config-derived regular-expression or pattern compilation. Invalid enabled LiveRamp configuration is rejected by typed validation and covered by Rust and CLI regression tests.

Checklist

  • Changes follow CLAUDE.md conventions.
  • No new unwrap() calls were introduced in production code.
  • No direct stdout logging was introduced in production code.
  • New behavior has automated coverage.
  • No secrets, credentials, live Placement IDs, or RampID envelope values are committed.

Acceptance boundary

RampID identity envelopes traverse the existing Prebid auction path; ATS Direct audience segments are not passed by this implementation.

@prk-Jr prk-Jr self-assigned this Aug 21, 2026
prk-Jr and others added 8 commits August 21, 2026 16:57
The bundle imported consentManagementTcf, consentManagementGpp and
consentManagementUsp, which only retrieve the consent signal. The activity
controls that act on it live in Prebid's tcfControl module, which was not
bundled, so a denied purpose changed nothing.

Reproduced with the real bundle and shim in JSDOM against a CMP stub denying
Purpose 1 and vendor 97: identityLink still issued a credentialed GET to
LiveRamp's envelope endpoint, still wrote idl_env and its bookkeeping cookies,
and the envelope still reached the ts-eids cookie. The run was identical to the
consent-granted one. Server-side gating stops EID forwarding only, not the
browser's collection and storage.

Import tcfControl alongside the consent collectors and lock the behavior with a
regression test that evaluates both production artifacts: denied purposes must
produce no vendor call and no storage, granted purposes must still resolve.

Note the scope: tcfControl enforces for every bidder and User ID module in the
bundle, not only LiveRamp. Regenerating a bundle is a manual step, so this
reaches a publisher only when `ts prebid bundle` is re-run.
`[integrations.prebid.liveramp]` combined with a pinned
`bundle.user_id_modules` list that lacks identityLinkIdSystem is valid config
that builds a bundle unable to serve it. The managed identityLink entry the
server injects has no submodule to drive, and the only signal is a browser-side
diagnostic.

Fail in `ts prebid bundle`, which already reads both keys from the same file. An
absent list still takes the generator's default preset, which includes the
module.
Three fixes to the setConfig and mergeConfig guards:

Skip seeding when getConfig is missing. `hasPrebidJsApi` only requires
registerBidAdapter, so getConfig can be absent. The previous fallback read an
empty entry list and then seeded the managed entry alone, silently dropping
every publisher module already configured. Log and leave the wrappers installed
instead, so a later publisher userIds call still gets the managed entry.

Build the managed entry per call. Prebid retains what it receives as
`submodule.config` for the life of the page, so one shared instance let any
mutation there leak into every later normalization.

Keep the normalization guard but report the error. Publisher configuration is
arbitrary page data and a throwing accessor must not break the publisher's own
setConfig call, so the guard stays; it now logs what failed instead of
discarding it. The redundant hasUserIdsPath check is dropped since
withManagedLiveRampUserId already performs it.
merge_auction_eids_deduplicates_liveramp_and_prefers_resolved_metadata was
merge_auction_eids_prefers_server_resolved_metadata_on_conflict with the source
string changed. merge_auction_eids is source-agnostic, so it added no coverage.

gate_eids_by_consent is all-or-nothing across sources. Rename its test and give
it a second source so the name no longer implies per-source gating that does not
exist.
The inline closure that mapped PrebidLiveRampConfig into its camel-cased
injection struct pushed rustfmt into indenting the whole payload literal an
extra level. A From implementation keeps the mapping next to the struct it
builds and restores the surrounding formatting.
The degraded-behavior table said only that no EID is forwarded when consent is
absent, which read as though nothing happened. Separate the TCF case, where
tcfControl now blocks resolution outright, from the US-state case, where the
server's forwarding gate is what applies.

Add the browser-side footprint the previous text omitted: the credentialed
request the page makes to LiveRamp's envelope endpoint, that Trusted Server does
not proxy it, the connect-src allowance a strict CSP needs, and every storage
entry IdentityLink writes. Note that the submodule prefers an ATS library
already on the page, and that this is therefore not a first-party replacement
for LiveRamp recognition.

Also record that the environment override cannot introduce the subsection, that
`ts prebid bundle` rejects a module list without identityLinkIdSystem, that
regenerating a bundle now enforces TCF for every bidder and User ID module, and
that KV persistence needs a matching [[ec.partners]] entry. Correct the design
document's consent section, which assumed the consent-management modules were
themselves the enforcement point, and flag the remaining GPP gap.
prk-Jr added 7 commits August 24, 2026 14:05
CI lints trusted-server-cli and trusted-server-openrtb-codegen explicitly
because no adapter-scoped alias covers them, but both invocations pin
x86_64-unknown-linux-gnu. There was no command a developer could run locally to
reproduce them, and CLAUDE.md's CI gate list named only the six adapter aliases,
so running the documented gates locally passed while CI failed on a
clippy::doc_markdown error in the CLI.

Add clippy-cli and clippy-codegen. Both omit --target so they build for the
host, and record them in the gate list.
CodeQL flagged js/incomplete-url-substring-sanitization on both assertions.
These are not sanitizers — the URLs come from the test's own fetch spy — so the
alert is not a vulnerability, but the substring test is still too loose for an
assertion: it would also match an unrelated host carrying api.rlcdn.com in its
name or query string, letting the granted-consent case count the wrong request.

Compare the parsed hostname instead. Verified the test is still non-vacuous:
dropping the tcfControl import fails two of its three cases.
@prk-Jr
prk-Jr marked this pull request as ready for review September 7, 2026 10:55
prk-Jr added a commit that referenced this pull request Sep 7, 2026

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Requesting changes for two consent-enforcement regressions and one managed User ID validation issue.

Comment thread crates/trusted-server-js/lib/src/integrations/prebid/index.ts
Comment thread crates/trusted-server-js/lib/src/integrations/prebid/index.ts Outdated
Comment thread crates/trusted-server-cli/src/prebid_bundle.rs
Main split the Prebid configuration into a browser-only
`PrebidIntegrationConfig` and a test-only `LegacyPrebidServerConfig`, so
move `managed_user_ids` onto the live browser config and map it through
the legacy conversion. Left on the legacy struct the field would have
compiled but never reached a running service.

Also un-gate the `HashSet` import now that managed User ID name
validation runs in production builds, and update the script-breakout
assertion for main's stronger JSON escaping, which escapes every `<`
rather than only `</script>`.
TCF activation read `window.__tcfapi` once, but managed User ID seeding
ran regardless. A CMP that installed itself after the deferred shim left
managed modules seeded with Prebid's GDPR handler disabled, and the
module fired its vendor request with no TCF parameters. No later
reconfiguration can recall that request, so hold managed entries out of
every configuration Prebid sees until CMP discovery concludes: at once
when `__tcfapi` is already callable, on a watched late assignment, or at
the first `requestBids`, since a conforming CMP installs its stub before
vendor tags request bids.

Prebid 10.26 reads `consentManagement` as
`config.gdpr || config.usp || config.gpp ? config.gdpr : config`, so a
legacy top-level TCF configuration carries no namespace key. Recognizing
only an own `gdpr` property let the shim append `gdpr: { cmpApi: "iab" }`
to such an object, which flipped Prebid to the namespaced shape and
discarded the publisher's static consent. Mirror Prebid's own rule in
both ownership checks. Retirement then needs a matching path: a
`mergeConfig` deep merge carries `gdpr: { enabled: false }` forward and
would disable the TCF module entirely, so remove the namespace instead of
leaving it disabled.

`ts prebid bundle` accepted each managed name independently even when
several resolved to one module. The registry maps both `sharedId` and
`pubCommonId` to `sharedIdSystem`; Prebid registers one submodule for a
module's name and its aliases and selects the first matching entry, so
the second managed configuration was silently ignored. Reject duplicate
resolved module names before generation.

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Reviewed d269ff8f9e3c78e7f024f29cd349739723279f51 against 705a69505bf6bf1eb5b498a88606d1473fdc6ea1. The bundle validation and focused tests pass, but the production Rust injection path drops the new configuration, and the first-auction CMP cutoff still permits managed identity resolution before a late CMP can enforce denied consent.

Comment thread crates/trusted-server-core/src/integrations/prebid.rs
Comment thread crates/trusted-server-js/lib/src/integrations/prebid/index.ts Outdated
ChristianPavilonis added a commit that referenced this pull request Sep 10, 2026

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Reviewed 66da8064d81f3ba9da969eb048857933743234c8. The late-CMP changes handle delayed API discovery, but the automatic TCF policy still fails open when a callable CMP stub does not return consent before Prebid's timeout.

Comment thread crates/trusted-server-js/lib/src/integrations/prebid/index.ts
@ChristianPavilonis
ChristianPavilonis dismissed their stale review September 10, 2026 17:47

Reclassified as a broader consent-timeout policy decision rather than an unconditional merge blocker; superseded by a follow-up discussion review.

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@prk-Jr the changes in prebid/index.ts seem to have privacy/demand implications that I'm not sure about so the following is worth calling out and perhaps needs product decision.

Consent-timeout policy worth discussing

I dismissed my earlier change request because this is better treated as a product/privacy policy decision than an unconditional implementation defect. The reproduction still stands: when __tcfapi is callable but never returns consent, Prebid's default 10-second timeout produces null consent with gdprApplies: false; the managed IdentityLink module can then call api.rlcdn.com without TCF parameters and write its identity cookies.

There are several defensible policies:

1. Leave the current behavior unchanged

Pros

  • Preserves Prebid's standard timeout semantics.
  • Avoids suppressing demand or analytics during a CMP outage.
  • Requires no additional lifecycle/state handling.

Cons

  • A stalled CMP is treated like GDPR does not apply, even though the callable API indicates that a CMP was expected.
  • An operator-managed identity module may contact its vendor and write storage without an affirmative jurisdiction or consent result.
  • This failure mode should be explicitly documented so operators understand the compliance tradeoff.

2. Set defaultGdprScope: true for the automatic TCF configuration

Pros

  • Small, direct change that fails closed after the timeout.
  • Responsive CMP behavior is unchanged: gdprApplies: false still permits activity, while valid grants and denials are enforced normally.
  • Prevents managed identity resolution when consent state is unavailable.

Cons

  • The blast radius is broader than managed User IDs. tcfControl can also block client-side bidders, analytics, EID transmission, user data, and other controlled Prebid activity.
  • A broken CMP could therefore reduce or eliminate monetization, including for non-EU users if the CMP never reports gdprApplies: false.
  • The page remains restricted until a later CMP callback updates the consent handler.

3. Defer only managed User ID seeding until a terminal CMP result

Seed managed entries only after gdprApplies: false, eventStatus: 'tcloaded', or eventStatus: 'useractioncomplete'.

Pros

  • Gives operator-managed IDs a fail-closed guarantee without changing Prebid's timeout policy for publisher bidders and analytics.
  • Keeps the monetization impact limited to the feature being introduced here.
  • A later valid CMP event can still activate the managed entries.

Cons

  • Requires more state coordination than the one-line scope change, especially for synchronous callbacks, late publisher ownership, listener retirement, and recovery after the first auction.
  • Managed IDs remain unavailable for the page lifetime if the CMP never reaches a terminal state.
  • Needs generated-artifact coverage for timeout, late recovery, denial, and gdprApplies: false.

4. Make the timeout posture operator-configurable

For example, expose a managed-ID consent policy selecting Prebid-default, global fail-closed, or managed-only deferral behavior.

Pros

  • Makes the compliance/revenue decision explicit for each deployment.
  • Supports operators with different jurisdiction and CMP guarantees.

Cons

  • Expands configuration and documentation substantially for a narrow feature.
  • A permissive default can still surprise operators; a restrictive default can still surprise them operationally.
  • Adds more combinations that require browser-artifact testing.

My preference would be option 3 if managed identities need a stronger guarantee without changing the rest of the publisher's Prebid failure behavior. Leaving the current behavior can also be intentional, but the timeout semantics and resulting vendor/storage activity should then be documented explicitly.

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

High-quality implementation of vendor-neutral managed Prebid User ID modules with LiveRamp RampID as the worked example. The fail-closed CMP-discovery machinery is carefully built, and the test depth (real-bundle TCF purpose/vendor matrix, ownership-transfer races, hostile-proxy edge cases, fail-closed CLI registry validation) is unusually strong. One correctness gap in the ownership invariant drives the verdict; the remaining findings are non-blocking.

3 of the 5 inline comments below carry a one-click GitHub suggestion (all scratch-verified against this head: cargo fmt + all six adapter clippy aliases + all four adapter test aliases + parity suite for the Rust changes; prettier + full vitest suite (967/967) + node build-all.mjs for the TypeScript change; verified in isolation and as a batch). Use Commit suggestion or Add suggestion to batch to apply them. The other two comments are observations with the reasoning in prose.

Blocking

🔧 wrench

  • Case-variant publisher entry silently defeats managed User ID ownership — see inline at crates/trusted-server-js/lib/src/integrations/prebid/index.ts:292

Non-blocking

♻️ refactor

  • Duplicate-name validation should use Prebid's case-insensitive matching rule — see inline at crates/trusted-server-core/src/integrations/prebid.rs:284

🤔 thinking

  • Real RampID envelopes vs the 512-byte EC value cap — see inline at crates/trusted-server-core/src/ec/prebid_eids.rs:804
  • watchForLateTcfApi accessor is never dismantled on the publisher-config seeding path — see inline at crates/trusted-server-js/lib/src/integrations/prebid/index.ts:1740

⛏ nitpick

  • Garbled doc phrase "untrimmed-free ASCII token" — see inline at crates/trusted-server-core/src/integrations/prebid.rs:227

CI Status

  • browser integration tests: PASS
  • integration tests: PASS
  • integration tests (Fastly EC lifecycle): PASS
  • cargo check/build/test (spin native + wasm32-wasip1): PASS
  • vitest: PASS
  • CodeQL: PASS
  • cargo test (ts CLI, native): PASS
  • cargo test (axum native): PASS
  • Analyze (javascript-typescript): PASS
  • format-typescript: PASS (required)
  • format-docs: PASS (required)
  • cargo check (cloudflare native + wasm32-unknown-unknown): PASS
  • Analyze (rust): PASS
  • cargo test: PASS (required)
  • cargo test (cross-adapter parity): PASS
  • cargo fmt: PASS (required)
  • prepare integration artifacts: PASS
  • Analyze (actions): PASS

Comment thread crates/trusted-server-js/lib/src/integrations/prebid/index.ts Outdated
Comment thread crates/trusted-server-core/src/integrations/prebid.rs Outdated
Comment thread crates/trusted-server-core/src/ec/prebid_eids.rs
Comment thread crates/trusted-server-js/lib/src/integrations/prebid/index.ts Outdated
Comment thread crates/trusted-server-core/src/integrations/prebid.rs Outdated
Managed Prebid User ID ownership compared entry names for exact
equality, but Prebid resolves a `userSync.userIds` entry to a submodule
on either its name or its alias, case-insensitively, and then takes the
first matching entry. Retained publisher entries sit ahead of managed
ones, so a publisher `IdentityLink` or `pubCommonId` entry silently won
over a managed `identityLink` or `sharedId`. Ownership now filters on
the module's full config-name set through `userIdConfigNameAliases`,
core rejects names that collide case-insensitively, and the `ts prebid
bundle` registry lookup resolves case variants instead of reporting
them as unregistered.

Managed seeding also treated a callable `window.__tcfapi` as a consent
decision. Prebid's GDPR handler times out after ten seconds and then
proceeds with null consent and `gdprApplies: false`, which `tcfControl`
cannot distinguish from a user outside GDPR scope, so a stalled CMP let
a managed module call its vendor and write identity storage with no
jurisdiction result behind it. Seeding now waits for a terminal CMP
result: `gdprApplies: false`, `tcloaded`, or `useractioncomplete`. A
refusal keeps entries deferred but reopens the wait, and a subscription
retired before the CMP's first event is removed on that event, since
TCF delivers the listener id only with a callback. Publisher-owned
`consentManagement` keeps Prebid's own timeout semantics untouched.

Document the settled-result policy, the alias-aware ownership rule, and
the 512-byte EID value cap that drops an oversized RampID envelope from
both the `/auction` payload and EC persistence.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest changes address my consent-timeout concern by deferring managed User ID seeding until the CMP returns a terminal result, while preserving publisher-owned Prebid timeout behavior. The generated-bundle regression covers the original stalled-CMP reproduction, and the current checks pass.

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

All five findings from the previous review are resolved at this head, each with tests: alias-aware case-insensitive ownership in the shim, case-insensitive duplicate validation in core and the CLI registry lookup, the watcher restorer captured and retired on every seeding path, the envelope-cap documentation (with the corrected /auction-path premise), and the doc wording. The additional fail-closed handling of a callable-but-silent CMP closes a real gap beyond the original findings and is reproduced against the real bundle.

The two comments below are non-blocking observations for follow-up; neither affects the verdict.

Non-blocking

🌱 seedling

  • Alias-set collisions between two managed entries still pass runtime validation — see inline at crates/trusted-server-js/lib/src/integrations/prebid/index.ts:297

🤔 thinking

  • An orphaned terminal-consent subscription has no path back — see inline at crates/trusted-server-js/lib/src/integrations/prebid/index.ts:1862

CI Status

All 19 checks pass at this head, including the required set (cargo fmt, cargo test, format-typescript, format-docs) and the non-required gates (vitest, cross-adapter parity, browser and EC-lifecycle integration tests, CodeQL analyses).

Comment thread crates/trusted-server-js/lib/src/integrations/prebid/index.ts
Comment thread crates/trusted-server-js/lib/src/integrations/prebid/index.ts Outdated
Prevent alias collisions from silently discarding operator settings,
and recover consent waits stranded on replaced silent CMP stubs.

Keep listener cleanup bound to its subscribing API so delayed callbacks
cannot remove another CMP consumer. Verify aliases against actual Prebid
registrations and cover consent recovery and cleanup regressions.
Keep AGENTS.md as the source of truth with CLAUDE.md pointing to it.
Carry forward the CLI and codegen lint requirements from this branch.
…e-355

# Conflicts:
#	crates/trusted-server-js/lib/src/integrations/prebid/index.ts
#	crates/trusted-server-js/lib/test/prebid-artifact-integration.test.mjs
The bound tracks the compact shim's normal output. Merging main's first
impression work into the LiveRamp branch adds both features' code to the
shim, taking it from roughly 30 KB to roughly 40 KB. Raise the bound to
41 KB, keeping the same small headroom over actual output that both
sides used, so material growth still has to be re-justified here while a
value-import of Prebid would still fail loudly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate and document LiveRamp integration

6 participants