Add opt-in guard for duplicate awaited sub-orchestration IDs

[wangbill (YunchuWang)]

Summary

Related to Azure/azure-functions-durable-extension#1947. This is a Core prerequisite, not automatic resolution of the Functions issue.

• Add TaskHubWorker.FailOnDuplicateSubOrchestrationInstanceIds, defaulting to false, configured before StartAsync.
• Validate the entire decision batch at the shared dispatcher boundary after execute/resume and before decision history or outbound messages are created. A new awaited child start conflicting with another pending awaited child fails its parent orchestration with actionable, non-retriable DuplicateSubOrchestrationInstanceId details. Discard the whole conflicting batch, including unrelated activities, timers, events, and children.
• Own the derived pending-child index in OrchestrationRuntimeState. Lazily reduce cold history to surviving task-ID/instance-ID pairs before creating pending nodes, then maintain it incrementally through accepted AddEvent calls. Proposed actions have a separate temporary batch map and never poison the accepted-history index if rejected, unsent, split, or retried.
• Preserve ordinal instance-ID comparison, exact completion/failure task correlation, legacy duplicate histories, sequential/retry reuse, replay/resume, suspension, ContinueAsNew, rewind, and restore. Release index dictionary capacity when a concurrent fan-out drains.
• Correct the three added test files to the canonical lowercase test/... Git paths and use an explicit non-null pattern guard for failure-notification assertions.

This avoids faulting only one task in Task.WhenAll while another child remains pending, and covers middleware that returns raw orchestrator actions without using the in-process context.

Compatibility and lifecycle contract

• Default off: taking the code alone does not enable the guard. Its failure is terminal for the offending orchestration, not a catchable per-call duplicate-ID exception. Earlier scheduled batches are not cancelled.
• No historic stranded-parent repair, cross-parent/global uniqueness guarantee, fire-and-forget guarantee, ID renaming, result coalescing, or provider/schema change.
• Keep the existing concrete Events list and serialized payload shape. Custom history rewriters should construct a fresh OrchestrationRuntimeState, or call InvalidateSubOrchestrationInstanceIdIndex() after direct list edits or accepted indexed identity/correlation/fire-and-forget changes. Arbitrary external mutations are not detected automatically. The hook clears only derived tracking, not other runtime metadata.
• Warm checks on a reused state do not rescan history or copy all pending entries. Cold reconstruction remains O(H); ordinary uncached replay still processes history. Retaining the pending index has upfront maintenance/memory costs.
• No dependency/version bumps. Azure Functions must separately adopt a released Core package and expose/enable the option; this PR does not ship Functions configuration support.

Local validation

Coverage	Result
Core net8.0: validator/dispatcher plus context, retry interceptor, and middleware tests	80 passed
Same focused Core selection on net48	68 passed
AzureStorage net8.0 against dedicated local Azurite 3.37.0, project-referenced Core/AzureStorage	12 passed

Assertions inspect actual outbound messages and persisted history, not only child invocations. Coverage includes whole-batch suppression, message splitting, proposal isolation, duplicate old completions, legacy linked-node removal/overwrite, no repeated history scan, new-event maintenance, default-off/startup capture, mutation invalidation, unchanged Objects/Auto/All JSON, stream restore, ContinueAsNew, actual rewind, suspension, failed checkpoint/abandon/retry, parent failure propagation, and sequential/retry reuse.

AzureStorage event-gated tests cover same-batch and later-episode conflicts without releasing the held child, plus distinct/case-sensitive/automatic IDs and sequential reuse after success/failure, with extended sessions off/on. Rejected starts are absent from persisted history/instances and queues. Only unique local task hubs were used; owned emulators were stopped.

Verified the exact canonical Git paths and actual MSBuild Compile items on Windows and on an exported Git tree in a case-sensitive Linux filesystem. Linux Compile-item evaluation succeeded using existing managed SDK binaries; a full Linux build/test run was not performed.

Independent local .NET isolated Functions confirmation used the unchanged official isolated worker, the final locally patched host Core DLL, and temporary test-only host-extension option wiring. Six guard-on cases passed (same-batch/cross-episode rejection, distinct/automatic IDs, success/failure reuse). The same corrected host binaries with the guard off reproduced both original duplicate-ID hangs, retaining one valid completion and a different missing task correlation after a probe replay. This is local causal evidence, not published Functions support or a CI claim.

Synthetic helper performance

Matched old helper (2e9b8ef) versus indexed implementation; optimized Release/net8, seven-repetition medians, prebuilt inputs, shared host. These are helper CPU-bound elapsed time and managed allocations, not production end-to-end percentages.

Workload	Old lookup	New cold lookup	New warm lookup
Completed sequential history, 100k events + one child	1.749 ms / 512 B	1.815 ms / 552 B	~0.13 us / 256 B
Completed historical fanout, 100k events + one child	2.720 ms / 4.07 MB	2.828 ms / 4.07 MB	~0.14 us / 256 B
Still-pending fanout, 10k children + one child	0.694 ms / 1.88 MB	1.383 ms / 3.31 MB	~0.15 us / 256 B

Progressive reused-state runs of 1k/2k/4k sequential children, including accepted create and completion event/index maintenance, took 0.386/0.742/1.444 ms versus 17.32/68.83/276.57 ms for the old helper plus the same event bookkeeping. This is approximately linear incremental helper work, not a claim that all orchestration execution becomes linear.

The costs are not free: validating and accepting a fresh 10k-child fanout took about 1.877 ms versus 0.940 ms before. New acceptance/index maintenance allocated 2.36 MB; still-pending cold loads also construct retained nodes. Disabled/no-action paths allocated 0 B. Cold initialization and upfront acceptance costs are reported separately rather than hidden by warm-only numbers.

Core command (repeat with -f net48):

dotnet test .\test\DurableTask.Core.Tests\DurableTask.Core.Tests.csproj -f net8.0 --no-restore --filter "FullyQualifiedName~SubOrchestrationInstanceIdValidatorTests|FullyQualifiedName~DuplicateSubOrchestrationDispatcherTests|FullyQualifiedName~TaskOrchestrationContextTests|FullyQualifiedName~RetryInterceptorTests|FullyQualifiedName~DispatcherMiddlewareTests"

AzureStorage selector with an explicitly local DurableTaskTestStorageConnectionString:

dotnet test .\test\DurableTask.AzureStorage.Tests\DurableTask.AzureStorage.Tests.csproj -f net8.0 --no-restore --filter "FullyQualifiedName~DurableTask.AzureStorage.Tests.DuplicateSubOrchestrationInstanceIdTests"

[Copilot]

🟡 Changes recommended

Move the three test files into the active test/ projects so they are compiled and executed.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds an opt-in Core guard against duplicate IDs among concurrently awaited sub-orchestrations.

Changes:

• Adds worker configuration and dispatcher preflight validation.
• Emits non-retriable parent failures for conflicting batches.
• Documents behavior and adds regression tests.

File summaries

File	Summary
Test/DurableTask.Core.Tests/SubOrchestrationInstanceIdValidatorTests.cs	Validator tests; currently outside the active lowercase test/ project.
Test/DurableTask.Core.Tests/DuplicateSubOrchestrationDispatcherTests.cs	Dispatcher tests; currently outside the active lowercase test/ project.
Test/DurableTask.AzureStorage.Tests/DuplicateSubOrchestrationInstanceIdTests.cs	Azure Storage tests; currently outside the active lowercase test/ project.
src/DurableTask.Core/TaskOrchestrationDispatcher.cs	Validates complete decision batches before dispatch.
src/DurableTask.Core/TaskHubWorker.cs	Adds the default-off opt-in setting.
src/DurableTask.Core/SubOrchestrationInstanceIdValidator.cs	Detects conflicting pending child IDs.
docs/features/sub-orchestrations.md	Documents configuration, behavior, and limitations.

Review details

• Files reviewed: 7/7 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

The new tests reside under uppercase Test/ and are not compiled by the active lowercase test/ projects on case-sensitive systems.

Review details

Suppressed comments (3)

Test/DurableTask.Core.Tests/SubOrchestrationInstanceIdValidatorTests.cs:25

• This test is under Test/, but the solution references test/DurableTask.Core.Tests/DurableTask.Core.Tests.csproj (DurableTask.sln:26), and that SDK project does not link sources from the uppercase tree. On case-sensitive build agents this file is therefore never compiled or discovered, leaving the validator behavior untested. Move it into the lowercase test/DurableTask.Core.Tests/ project directory.

    [TestClass]
    public class SubOrchestrationInstanceIdValidatorTests

Test/DurableTask.Core.Tests/DuplicateSubOrchestrationDispatcherTests.cs:31

• This test is under Test/, while DurableTask.sln:26 builds the project from lowercase test/DurableTask.Core.Tests/; that project has no source link to this uppercase directory. Consequently these dispatcher regressions are skipped on case-sensitive systems. Move the file into test/DurableTask.Core.Tests/.

    [TestClass]
    public class DuplicateSubOrchestrationDispatcherTests

Test/DurableTask.AzureStorage.Tests/DuplicateSubOrchestrationInstanceIdTests.cs:30

• This file is outside the active Azure Storage test project: DurableTask.sln:42 points to lowercase test/DurableTask.AzureStorage.Tests/, whose project does not include sources from Test/. On case-sensitive CI none of these provider-level scenarios execute. Move the file into the lowercase test project directory.

    [TestClass]
    public class DuplicateSubOrchestrationInstanceIdTests

• Files reviewed: 7/7 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

The changes affect shared dispatcher semantics and cached runtime-state behavior across providers, warranting final human review.

Review details

• Files reviewed: 9/9 changed files
• Comments generated: 0 new
• Review effort level: Balanced