[rbev]

Summary

Use two ingestion writers where storage supports concurrent batches, split the default batch size across those writers, and wait up to 100 ms for partial batches to fill. Keep transport concurrency defaults unchanged.

Why

A batch sized to the full receive concurrency can occupy every receive slot while other writers have no messages to process. Dividing the concurrency across writers allows multiple full batches to be written concurrently. Two writers divide both existing transport defaults, 10 and 32, evenly.

The timeout lets partial batches collect more messages before storage writes begin. It trades latency for fuller batches as testing found that a zero timeout would rarely allow a full batch to be written decreasing total throughput.

Behavior

• Default to two writers for storage that supports concurrent batches, down from four. Storage that does not support concurrent batches still uses one writer.
• Calculate an unconfigured batch size as receive concurrency divided by the resolved writer count, using integer division and a minimum of one.
• Default the batch timeout to 100 ms instead of zero. Full batches do not wait; partial batches can wait up to 100 ms.
• Preserve explicitly configured batch sizes and timeouts, including a zero timeout. Preserve configured writer counts where storage supports them.
• Update the ingestion documentation, settings comments, unit tests, and settings approval files.

Transport concurrency remains 10 for error ingestion and 32 for audit ingestion, except SQL Server and PostgreSQL transports, where audit ingestion remains at 10.

Configuration	Receive concurrency	Writers	Default batch size
Error ingestion, SQL Server or PostgreSQL storage	10	2	5
Error ingestion, RavenDB storage	10	1	10
Audit ingestion, concurrent storage and a transport default of 32	32	2	16
Audit ingestion, concurrent storage and SQL Server or PostgreSQL transport	10	2	5

[warwickschroeder]

A potential concern is that existing installs change behaviour on upgrade with these new defaults.

[warwickschroeder]

I think this ??= 10 may have been protecting the connection pool. SqlServer transport holds a pooled connection for the whole receive (ProcessWithNativeTransaction), and our ingestion receives don't return until their batch is written, so 100 concurrency uses all 100 connections in the default pool. Any dispatch inside the batch that needs a fresh connection (e.g. audit control messages) then waits for the pool timeout. The error instance also runs four receivers on one pool, which 4 × 10 used to fit. The testing tool defaults to RabbitMQ, so it doesn't cover this. Have these new defaults been tested with all transports?

[warwickschroeder]

Npgsql also defaults to a pool of 100, so the SQL Server pool concern applies here for both the primary and audit endpoints.

[rbev]

A potential concern is that existing installs change behaviour on upgrade with these new defaults.

Updated PR to only change ingestion batching config and leave the transport defaults alone. If existing installs have increased these values the default batch size will scale appropriately.

Also reduced the default writer threads to 2, which will be more likely to evenly divide into the transport parallelism value - avoiding writers being starved under heavy ingestion load.