Add large-payload blob auto-purge with a persistent orchestration - #758

Merged

wangbill (YunchuWang) merged 81 commits into

mainfrom

yunchuwang-wangbill-blob-payload-autopurge-sdk

Sep 21, 2026

Commits on Jul 13, 2026

Add opt-in blob payload auto-purge job to AzureBlobPayloads

wangbill (YunchuWang)
and
Copilot
committed
306d19f

Commits on Jul 14, 2026

Address PR #758 review feedback: naming, self-heal, poison-ack, starter simplification

wangbill (YunchuWang)
and
Copilot
committed
60f6637
Refine BlobPurgeJobStarter: pre-check bridge status before rescheduling

wangbill (YunchuWang)
and
Copilot
committed
149c63a
Classify RequestFailedException 400 as permanent in DeleteExternalBlobActivity

wangbill (YunchuWang)
and
Copilot
committed
4d52005
Reuse shared PayloadStore and register purge starter conditionally on AutoPurge

wangbill (YunchuWang)
and
Copilot
committed
780d743
Register fallback PayloadStore in shared Core for both client and worker

wangbill (YunchuWang)
and
Copilot
committed
3a2215c
Merge branch 'main' into yunchuwang-wangbill-blob-payload-autopurge-sdk
wangbill (YunchuWang)
authored
3fbf061
Stop self-registering PayloadStore on the client; consume the shared store

wangbill (YunchuWang)
and
Copilot
committed
47651dc

Commits on Jul 15, 2026

Commits on Jul 30, 2026

Register PayloadStore in the client builder extension (symmetry with worker; fixes client-only DI failure)

wangbill (YunchuWang)
and
Copilot
committed
50ae944

Commits on Jul 31, 2026

Merge branch 'main' into yunchuwang-wangbill-blob-payload-autopurge-sdk

wangbill (YunchuWang)
and
Copilot
committed
a680442
Resolve v2 tokens in DeleteAsync and discard payloads in unreachable accounts

wangbill (YunchuWang)
and
Copilot
committed
a5ed298
Align unreachable-account log, exception wording and v2 delete test names with review spec

wangbill (YunchuWang)
and
Copilot
committed
fff06b0
Gate blob auto-purge on v2 tokens and deleting stores

wangbill (YunchuWang)
and
Copilot
committed
6e4f5d0
Fix auto-purge starter registration and client resolution

wangbill (YunchuWang)
and
Copilot
committed
65e9cbb
Back off on zero-ack purge cycles and document TokenPrefixV1

wangbill (YunchuWang)
and
Copilot
committed
8f436df

Commits on Aug 2, 2026

docs(AzureBlobPayloads): reframe v1-token handling as a defensive guard

wangbill (YunchuWang)
and
Copilot
committed
81284e0

Commits on Aug 4, 2026

Commits on Aug 11, 2026

Reshape large-payload auto-purge to the finalized design

wangbill (YunchuWang)
and
Copilot
committed
1fcdb10
Clear three new-code warnings in the auto-purge files

wangbill (YunchuWang)
and
Copilot
committed
2026d71
Pin the no-inbound-enum invariant the numeric purge casts rely on

wangbill (YunchuWang)
and
Copilot
committed
e25e724
Clear the two remaining PR-introduced style warnings

wangbill (YunchuWang)
and
Copilot
committed
7bf5da8
Narrow the purge reason enum from 11 values to 7

wangbill (YunchuWang)
and
Copilot
committed
de218c8
Re-sync purge reason comments from canonical contract

wangbill (YunchuWang)
and
Copilot
committed
02b957c
Remove purge reason and storage error code from the contract

wangbill (YunchuWang)
and
Copilot
committed
e432d25

Commits on Aug 12, 2026

Dedupe the purge bridge in the backend instead of checking first

wangbill (YunchuWang)
and
Copilot
committed
1ac131f
Dedupe the purge bridge on Pending and Running only

wangbill (YunchuWang)
and
Copilot
committed
ea0b4c7
Add Stop operation and make the purge batch size configurable at runtime

wangbill (YunchuWang)
and
Copilot
committed
3c5c7ba

Commits on Aug 13, 2026

Read the purge job state before signalling Stop

wangbill (YunchuWang)
and
Copilot
committed
7d409ab
Self-heal the blob purge job by re-signalling Run on every host start

wangbill (YunchuWang)
and
Copilot
committed
7e0d0ed
Make LastModifiedAt mean a real change, and drop internal backend symbol names

wangbill (YunchuWang)
and
Copilot
committed
720e028
Drop an uncorrelatable backend column name from a proto comment

wangbill (YunchuWang)
and
Copilot
committed
2b389fa
Reconcile the blob auto-purge job periodically instead of once per host start

wangbill (YunchuWang)
and
Copilot
committed
3dd2b14
Correct comments that claimed the purge starter defers client construction

wangbill (YunchuWang)
and
Copilot
committed
3a1b673

Commits on Aug 14, 2026

Enable entity support when externalized payloads are configured

wangbill (YunchuWang)
and
Copilot
committed
4cc0baa

Commits on Aug 15, 2026

Purge externalized blobs over the worker's own gRPC transport

wangbill (YunchuWang)
and
Copilot
committed
b8e581b
Collapse purge client abstraction into direct sidecar client injection

wangbill (YunchuWang)
and
Copilot
committed
51a0f1c

Commits on Aug 16, 2026

Disable blob auto-purge on backends missing the purge RPCs

wangbill (YunchuWang)
and
Copilot
committed
00cbb68
Batch blob-purge deletes into chunks and bound each delete's wall-clock time

wangbill (YunchuWang)
and
Copilot
committed
d146c50

Commits on Aug 17, 2026

Commits on Aug 18, 2026

Correct stale purge-disposition docs left by the F1 quarantine change

wangbill (YunchuWang)
and
Copilot
committed
ce3b547
Revert F1: unowned blobs report Deleted again, not Quarantined

wangbill (YunchuWang)
and
Copilot
committed
7669cf9

Commits on Aug 27, 2026

Adopt the DTS large-payload purge service

wangbill (YunchuWang)
and
Copilot
committed
e2ea41b
Merge origin/main into the auto-purge branch

wangbill (YunchuWang)
and
Copilot
committed
adb1585
Gate work-item intake on a confirmed auto-purge setting

wangbill (YunchuWang)
and
Copilot
committed
45867b1
Follow the worker's transport for purge activity calls

wangbill (YunchuWang)
and
Copilot
committed
2190e3c
Correct the tombstone-token claims and two stale connect-setup docs

wangbill (YunchuWang)
and
Copilot
committed
4bb7c75
Report an established work-item connection only once one exists

wangbill (YunchuWang)
and
Copilot
committed
5ccd78e

Commits on Aug 28, 2026

Make large-payload auto-purge an explicit client-owned operation

wangbill (YunchuWang)
and
Copilot
committed
86e39f9
Move the auto-purge client seam into the Internal namespace

wangbill (YunchuWang)
and
Copilot
committed
ec28a77
Correct stale auto-purge recovery semantics in comments and messages

wangbill (YunchuWang)
and
Copilot
committed
954bd26

Commits on Sep 3, 2026

Treat a declined tombstone fetch as an empty cycle, not a failure

wangbill (YunchuWang)
and
Copilot
committed
bc8aa31
Drop two claims from the declined-fetch log that are not always true

wangbill (YunchuWang)
and
Copilot
committed
fbd55d6

Commits on Sep 14, 2026

Commits on Sep 15, 2026

Commits on Sep 16, 2026

Clean up old terminal auto-purge runners when enabling

wangbill (YunchuWang)
and
Copilot
committed
6cf6d56
Replace auto-purge entity with an eternal orchestration

wangbill (YunchuWang)
and
Copilot
committed
e3fce5d
Let the backend deduplicate auto-purge runner starts

wangbill (YunchuWang)
and
Copilot
committed
8ff4746

Commits on Sep 17, 2026

Refresh protobuf provenance using the upstream sync script

wangbill (YunchuWang)
and
Copilot
committed
c2c6f83
Simplify purge cycle state and durable wait helper

wangbill (YunchuWang)
and
Copilot
committed
88c3dbd
Wait for purge configuration directly after unsupported failures

wangbill (YunchuWang)
and
Copilot
committed
b34f040
Keep internal auto-purge names in explicit worker filters
wangbill (YunchuWang)
committed
fbc1fd3
Retry uncertain blob HEAD 404 responses during auto-purge

wangbill (YunchuWang)
and
Copilot
committed
c18dea6
Use Where when selecting missing auto-purge activity filters

wangbill (YunchuWang)
and
Copilot
committed
742346e
Isolate auto-purge transports by named worker

wangbill (YunchuWang)
and
Copilot
committed
bb582d8
Keep internal auto-purge tasks independent of business versioning

wangbill (YunchuWang)
and
Copilot
committed
611f654
Clarify purge test cleanup and intentional experimental coverage

wangbill (YunchuWang)
and
Copilot
committed
a0dd9ac