Skip to content

Add SDK-owned blob purge service contract and shared task integration - #805

Draft
wangbill (YunchuWang) wants to merge 13 commits into
mainfrom
yunchuwang-df-blob-purge-sdk-adapters
Draft

wangbill (YunchuWang) wants to merge 13 commits into
mainfrom
yunchuwang-df-blob-purge-sdk-adapters

Conversation

@YunchuWang

@YunchuWang wangbill (YunchuWang) commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

Summary

What changed?

  • Add SDK-owned Microsoft.Azure.DurableTask.LargePayloadPurge.Abstractions, an independently versioned package containing the service capability and activity transport interfaces, with a source reference to the SDK Client project.
  • Keep the canonical tombstone/result records and disposition enum in Microsoft.DurableTask.Client; no model copies, moves or type forwarders.
  • Make GetLargePayloadTombstonesActivity and ReportLargePayloadPurgeResultsActivity publicly constructible with a task-hub-bound ILargePayloadPurgeClient.
  • Put ILargePayloadPurgeClient in the contracts assembly, preserving its namespace and signatures. The Blob implementation references the contracts package, so transport consumers do not need a reference to the Blob implementation.
  • Adapt the standalone worker's existing keyed gRPC transport to that interface, preserving authentication, reconnection, deadlines, status handling and opaque correlation tokens.
  • Add contract API/model-identity/signing tests and integrate the new package with the existing solution, signing, SBOM, packing and gated publication process.
  • Document the optional .NET isolated Functions integration and the separate service contract.

Why is this change needed?

  • The optional Microsoft.Azure.Functions.Worker.Extensions.DurableTask.AzureBlobPayloads package invokes the shared purge tasks through ordinary function bindings. Public construction and the narrow transport interface let it reuse the SDK implementation with a bound local-host bridge and worker-owned blob storage.
  • The companion Functions extension and provider supply function wrappers, client binding and the authenticated service bridge. This PR changes only the shared SDK integration surface.
  • Maintaining the optional service contract beside its canonical SDK models lets repository builds use a ProjectReference rather than requiring an externally published SDK model package to bootstrap compilation.

Issues / work items


Project checklist

  • Release notes are not required for the next release
    • Otherwise: Notes added to release_notes.md
  • Backport is not required
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • All required tests have been added/updated (unit tests, E2E tests)
  • Breaking change?
    • If yes:
      • Impact: N/A
      • Migration guidance: N/A

No breaking change for released SDK packages - the constructors and interfaces are additive relative to main.
The constants remain internal. The original bootstrap, orchestration, deletion, task names, payloads, retries
and continue-as-new behavior are unchanged.

The package preserves assembly name DurableTask.LargePayloadPurge.Abstractions, SDK signing key and planned
0.1.0 version. The service capability remains in DurableTask.LargePayloadPurge with its three required
deadline/cancellation signatures. The activity transport remains in Microsoft.DurableTask.AzureBlobPayloads
with its original two signatures and optional cancellation-token defaults. Canonical SDK model identity and
all runtime implementations are unchanged. No protobuf changes are included.

Breaking Change (unreleased prototype binaries only)

  • Type: binary
  • Impact: ILargePayloadPurgeClient moves from the unreleased Blob implementation assembly to the contracts assembly; no duplicate definition or type forwarder remains.
  • Migration: Rebuild prototype consumers against the updated contracts and Blob packages. Source namespace/signatures are unchanged. Consumers of the earlier differently signed service-contract prototype also require a rebuild.

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • No
  • Yes, AI helped write parts of this PR (e.g., GitHub Copilot)
  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot
  • AI-assisted areas/files: Contract project, tests and release integration; activity construction, transport adapter, standalone registration and documentation.
  • What you changed after AI output: No human changes recorded yet.

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

These first-person attestations remain for the human author.


Testing

Automated tests

  • Result: Passed - 7 updated contract tests and 230 extension tests, zero failures or skips. Before the move, the new export/dependency assertions failed as expected (2 failed, 5 passed); all 7 passed after the move.
  • dotnet test test\LargePayloadPurge.Abstractions.Tests\LargePayloadPurge.Abstractions.Tests.csproj --no-restore --verbosity quiet
  • dotnet test test\Extensions\AzureBlobPayloads.Tests\AzureBlobPayloads.Tests.csproj --no-restore --verbosity quiet
  • Local Release packing passed for the netstandard2.0 contract and its SDK dependency closure, including the extension's netstandard2.0, net6.0, net8.0 and net10.0 targets. Existing analyzer warnings remain.
  • The actual contracts package exports exactly IOrchestrationServiceLargePayloadPurgeClient and ILargePayloadPurgeClient, references the canonical Client models, and carries the SDK strong-name token.
  • Package manifests and assembly identities were inspected. Contracts have no Blob, storage, worker or gRPC implementation dependency. All four Blob target assets reference the contracts assembly and neither define nor forward the transport interface. Core has no reverse dependency.
  • Both original license texts were verified byte-for-byte in the package, with the expression Apache-2.0 AND MIT.
  • Both signing pipeline definitions parse, and the new public publication selector matches exactly one approval-gated job while excluding symbol packages. Hosted signing/publication was not executed locally.
  • Shared-task coverage executes actual Get/Delete/Report activities through the SDK shims and Core replay executor, checking argument envelopes, token correlation, structured failures, retry timers, external events and continue-as-new. These tests are not a Functions host/worker end-to-end run.
  • Existing standalone control, named-worker transport/reconnection, deadline, versioning and filter tests remain covered.

Manual validation (only if runtime/behavior changed)

  • Environment (OS, .NET version, components): Windows and .NET SDK 10.0.401 for SDK tests; a local Functions host and actual isolated worker for companion integration.
  • Historical companion integration results, recorded before the contract ownership and signing-key migration:
    1. The optional package's compiled functions executed Get/Delete/Report; all four eligible owned v2 blob targets were physically absent afterward.
    2. Enablement remained task-hub-scoped. A strict-version business orchestration failed while the empty-version purge runner stayed Running.
    3. A base-extension-only application had no purge functions or runner; explicit disable followed by restart did not re-enable cleanup.
  • Current SDK boundary validation consists of the 7 contract tests, 230 extension tests, source-based package builds, and package ABI/dependency-graph checks. The physical SQL/blob end-to-end run above has not been repeated for the new contract identity.
  • Evidence (optional): SDK TRX/build logs are retained locally. Historical companion results are separately owned and were not independently rerun in this SDK session. They establish the earlier reported local cases, not a cloud deployment, whole-container cleanup guarantee, or a fresh physical end-to-end result for this migration.

Notes for reviewers

  • The new package's planned initial version is 0.1.0, independent of the repository-wide SDK version. The service interface retains its Apache-2.0 notice and the activity transport interface retains its MIT notice. The package includes both original texts and declares Apache-2.0 AND MIT; no source was relicensed.
  • The dependency direction is Blob implementation -> contracts -> SDK Client -> SDK Abstractions -> Core. Contracts are not BCL-only, but have no Blob/storage/worker/gRPC implementation dependency. Core does not depend on the contracts or SDK Client.
  • Publish compatible Client/Abstractions packages containing the purge models before the new package. Published Client 1.26.0 lacks those models. Source builds do not need an external Client-version override; local package versions used for validation are not release claims.
  • Both activity constructor paths are used: the public typed transport path by the Functions integration, and the internal generated-client path by standalone registration and existing tests.
  • The purge transport, bound Functions client and payload store must target the same task hub. The integration owns their credentials, lifetime and reconnection policy.
  • The original SetLargePayloadAutoPurgeAsync remains authoritative: enable sets the backend flag, deduplicates the fixed unversioned runner, verifies identity/Running status and sends the batch event; disable only writes the setting.
  • Adoption requires compatible companion Functions/provider packages and the shared SDK release. No validation-only package versions or machine-local references are committed.

Reuse the existing purge tasks and bootstrap sequence with a task-hub-bound transport interface. Preserve standalone behavior and prove DTFx shim execution, failure handling, replay, and per-hub setup.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reuse the existing native gRPC client for setup instead of adding an alternate bootstrap overload. Remove the unused setting facade and client shim tests while retaining actual DTFx purge task execution coverage.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Document normal function indexing, language-worker task execution, and the bound local-host transport. Preserve the existing SDK implementation and explicit bootstrap without claiming Functions end-to-end validation.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Clarify normal compiled-library Function indexing and opt-in package registration without a metadata transformer or generator. Preserve the base extension behavior and unchanged shared SDK tasks.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Describe explicit registration, normal function metadata and bindings without naming an unreleased package or prescribing its metadata implementation. Leave shared task APIs and bootstrap unchanged.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Describe the optional feature package's normal build-time Function discovery and generated invocation. Preserve shared SDK task APIs, explicit bootstrap behavior, and standalone operation.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@YunchuWang wangbill (YunchuWang) changed the title Expose blob auto-purge integration APIs for alternate .NET hosts Expose shared blob auto-purge tasks for .NET isolated Functions Sep 23, 2026
Restore the constants file to its upstream scope after verifying final consumers use only the public activities and transport facade. Trim integration documentation to the current isolated Functions path.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve the interface package identity and canonical SDK Client model references with an independent version. Add focused API and signing tests, Apache license attribution, solution discovery, and existing release signing coverage.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add the preserved package ID to the existing independent NuGet publication jobs, retaining the manual approval gate and symbol exclusion.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@YunchuWang wangbill (YunchuWang) changed the title Expose shared blob auto-purge tasks for .NET isolated Functions Add SDK-owned blob purge service contract and shared task integration Sep 24, 2026
Explain the interface-only netstandard2.0 target alongside the matching Client model library. Use a typed xUnit assertion before validating activity deadlines.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve both interface APIs and SDK model identities while removing the Blob implementation dependency from transport consumers. Add ownership and dependency regression coverage and package the existing Apache and MIT notices together.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use an extension-bearing package path so NuGet places the linked MIT license at the documented package root rather than treating it as a directory.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the repository license linked without duplicating it and document the actual NuGet directory layout.

Copilot-Session: 883b4cbd-e93c-4d4e-8cb8-ffcf69525cfa

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant