Skip to content

chore: the RabbitMQ Kamelets can scale, retry and dead-letter, and the sink can set the content type (#3170) - #3238

Merged
davsclaus merged 1 commit into
apache:mainfrom
davsclaus:ci-issue-3170-rabbitmq
Nov 18, 2026
Merged

davsclaus merged 1 commit into
apache:mainfrom
davsclaus:ci-issue-3170-rabbitmq

Conversation

@davsclaus

Copy link
Copy Markdown
Contributor

Part of #3170: spring-rabbitmq-source can run several consumers in parallel, set the prefetch count and the acknowledge mode, choose the exchange type, tune the retries and set up a dead letter queue; spring-rabbitmq-sink can send a message with no body and set the content type.

The options

spring-rabbitmq-source, all added to the endpoint URI with optional placeholders:

Option Template mapping What it does
exchangeType exchangeType={{?exchangeType}} The type of the exchange that autoDeclare creates: direct, fanout, headers or topic. With topic, routingKey is a pattern (orders.*, orders.#).
concurrentConsumers concurrentConsumers={{?concurrentConsumers}} How many consumers receive from the queue in parallel, each on its own channel.
prefetchCount prefetchCount={{?prefetchCount}} How many messages RabbitMQ sends ahead to each consumer (250 when unset).
acknowledgeMode acknowledgeMode={{?acknowledgeMode}} AUTO (as today) or NONE: removed from the queue on delivery, so a failing message is lost.
maximumRetryAttempts maximumRetryAttempts={{?maximumRetryAttempts}} How many times a failing message is tried again after the first try (5 when unset).
retryDelay retryDelay={{?retryDelay}} Milliseconds between the tries (1000 when unset).
deadLetterExchange deadLetterExchange={{?deadLetterExchange}} With autoDeclare, creates this exchange and sets it as the dead letter exchange of the queue.
deadLetterRoutingKey deadLetterRoutingKey={{?deadLetterRoutingKey}} The routing key a failed message is dead-lettered with, and that binds deadLetterQueue.
deadLetterQueue deadLetterQueue={{?deadLetterQueue}} With autoDeclare, creates this queue and binds it to deadLetterExchange.
deadLetterExchangeType deadLetterExchangeType={{?deadLetterExchangeType}} The type of the dead letter exchange (direct when unset).

spring-rabbitmq-sink:

Option Template mapping What it does
allowNullBody allowNullBody={{?allowNullBody}} in the endpoint URI Sends a message with no body as an empty message, instead of failing.
contentType a choice on '{{?contentType}}' != '' that sets the header CamelSpringRabbitmqContentType to {{?contentType}} The content type of every message, such as application/json.
  • Defaults: all options are optional and have no default, as in chore: mail-sink can use STARTTLS (#3170) #3174. When they are unset, the optional placeholders drop them and the endpoint URIs are the same as on main; the choice of the sink takes no branch.
  • Content type today: the sink had no option for it. The component sends a String body as text/plain and anything else as application/octet-stream, unless the header CamelSpringRabbitmqContentType says otherwise. contentType sets that header, so JSON text can go out as application/json. It is not a URI option of the component, hence the header.
  • Patterns: the dead letter names take ^[A-Za-z0-9_.:-]+$; contentType takes a media type with optional parameters.
  • Retry count: maximumRetryAttempts counts the retries, not the tries, with the Spring AMQP that Camel 4.24 uses: unset, a message is tried 6 times; 3 gives 4 tries, 0 gives 1. The Good to know of the source said "tried 5 times"; it now says 6. (The Camel option description, "the number of attempts includes the initial try", is no longer right; that is for apache/camel.)
  • Dead letters need autoDeclare: the component sets the dead letter exchange only on the queues it declares. With autoDeclare false, the options do nothing (verified), and the descriptions say to set the dead letter exchange on the queue in RabbitMQ, for example with a policy. deadLetterQueue without deadLetterRoutingKey fails to start ('routingKey' must be non-null), also with a fanout dead letter exchange; the description says so.
  • Left out:
    • acknowledgeMode: MANUAL: a Kamelet has no way to acknowledge, so every processed message stays unacknowledged (verified: unacked 1 after processing). The enum offers AUTO and NONE only.
    • rejectAndDontRequeue: with false the message is not requeued, as the component docs suggest; after the retries, Spring AMQP logs Message(s) dropped on recovery and acknowledges it, so it is lost, also when the queue has a dead letter exchange. The option would only turn the dead letter exchange off.
    • maxConcurrentConsumers: only used by the SMLC listener container; the Kamelet uses the default DMLC, where concurrentConsumers is a fixed count. Adding it would also need messageListenerContainerType.
  • Trust store (Fix #3210: spring-rabbitmq Kamelets check the broker certificate and host name with amqps, and take a trust store #3211): the connection factory beans and the trust store options are not changed; SpringRabbitmqKameletTlsTest passes.
  • Descriptions: the Good to know of the source now covers exchangeType, the start failure when the exchange or queue exists with other settings, the retry count, the dead letter options and parallel processing. The Input of the sink names contentType and allowNullBody, and no longer says that an "empty body" fails: an empty string is sent fine; only a message with no body fails.
  • Partial pages: the source page has new "Parallel processing" and "Retries and dead letters" sections with an example; the sink page has "Content type" and a usage example.

Verification

Tested with Camel JBang 4.24.0-SNAPSHOT against rabbitmq:management (RabbitMQ 4.4.1), reading the broker state from the management API.

Case Result
Options unset, both Kamelets, main and this branch camel get endpoint shows identical spring-rabbitmq URIs; the message arrives with CamelSpringRabbitmqContentType=text/plain as before
Unset, management API one consumer on the queue, prefetch 250
concurrentConsumers: 3, prefetchCount: 7 URI has concurrentConsumers=3&prefetchCount=7; /api/consumers: 3 consumers on the queue, on channels (1), (2), (3) of one connection, each with prefetch 7
concurrentConsumers: 3, 6 messages, 2 s of work each 3 start at the same moment on 3 threads, the next 3 two seconds later; all done in 4 s
maximumRetryAttempts: 3, retryDelay: 500, deadLetterExchange, deadLetterRoutingKey: failed, deadLetterQueue, failing route 4 tries, 500 ms apart; the queue has x-dead-letter-exchange and x-dead-letter-routing-key: failed; the dead letter queue holds the message, routing key failed, with x-death (reason rejected, queue and original routing key)
Retry options unset, failing route 6 tries, 1 s apart
maximumRetryAttempts: 1 / 0 2 tries / 1 try, then dead-lettered
deadLetterExchangeType: fanout Dead letter exchange created as fanout; message reaches the dead letter queue
deadLetterExchange only, no routing key or queue; a queue bound to it by hand with the original key The failed message keeps its routing key work and reaches that queue
deadLetterQueue without deadLetterRoutingKey Fails to start: 'routingKey' must be non-null
Dead letter options with autoDeclare: false, queue created by hand No dead letter exchange is created; the failed message is dropped
acknowledgeMode: NONE, failing route with dead letter options /api/consumers shows ack_required: false; 3 tries, then the message is gone, and the dead letter queue stays empty
acknowledgeMode: MANUAL (not offered) The processed message stays unacked 1
rejectAndDontRequeue: false (not offered) 2 tries, then Message(s) dropped on recovery; not requeued, not dead-lettered
exchangeType: topic, two sources bound with orders.* and orders.#; messages with keys orders.new, orders.eu.new, invoices.new Exchange created as topic; orders.* got orders.new only, orders.# got orders.new and orders.eu.new, nobody got invoices.new
exchangeType: topic on an existing direct exchange Fails to start: PRECONDITION_FAILED - inequivalent arg 'type'
Queue existing with dead letter key failed, started with other Fails to start: PRECONDITION_FAILED - inequivalent arg 'x-dead-letter-routing-key'
Sink, null body, allowNullBody: true Sent: 0 bytes, content type application/octet-stream
Sink, null body, allowNullBody: false; and on main Fails: Cannot send message as message body is null, and option allowNullBody is false.
Sink, empty string body, options unset Sent: 0 bytes, text/plain
Sink, contentType: "application/json; charset=utf-8", text body Content type application/json; charset=utf-8
Sink, bytes, contentType unset application/octet-stream
Sink, contentType: application/xml, message header saying text/plain application/xml
Sink, null body, allowNullBody: true, contentType: application/json 0 bytes, application/json
The two partial examples together (webhook-source → sink, source with a step that fails) curl POST of JSON; 4 consumers with prefetch 10; 3 tries 2 s apart; the message in orders.dlq with content type application/json and x-death

./mvnw clean verify -DskipTests and then ./mvnw clean verify passed from the root (including SpringRabbitmqKameletTlsTest and KameletsCatalogTest). Before building, the library copies of both Kamelets were refreshed with ./mvnw -pl library/camel-kamelets resources:copy-resources@copy-resource-one. The build regenerated kamelets-index.json.

Claude Code on behalf of Claus Ibsen

🤖 Generated with Claude Code

…e sink can set the content type (apache#3170)

spring-rabbitmq-source takes new optional options:

- exchangeType: the type of the exchange that autoDeclare creates
  (direct, fanout, headers, topic); with topic, routingKey is a pattern.
- concurrentConsumers and prefetchCount: how many consumers receive in
  parallel, each on its own channel, and how many messages RabbitMQ
  sends ahead to each.
- acknowledgeMode: AUTO or NONE. MANUAL is left out: a Kamelet cannot
  acknowledge, so the messages would stay unacknowledged.
- maximumRetryAttempts and retryDelay: how many times a failing message
  is tried again, and how long apart.
- deadLetterExchange, deadLetterRoutingKey, deadLetterQueue and
  deadLetterExchangeType: with autoDeclare, create a dead letter
  exchange and queue and set them on the queue, so that a message that
  still fails after the retries moves to the dead letter queue.

spring-rabbitmq-sink takes allowNullBody, to send a message with no
body as an empty message, and contentType, to send every message with
a content type such as application/json instead of text/plain or
application/octet-stream.

maximumRetryAttempts counts the retries, not the tries: a message is
tried 6 times when it is not set, so the description of the source,
which said 5, is corrected. rejectAndDontRequeue and
maxConcurrentConsumers are left out: with false, the first drops the
message after the retries instead of requeuing it, and the second works
only with the SMLC container, which the Kamelet does not use.

With the new options unset, the endpoint URIs are the same as before.
The connection factory and the trust store options are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@davsclaus
davsclaus merged commit 456f3e6 into apache:main Nov 18, 2026
6 checks passed
@davsclaus
davsclaus deleted the ci-issue-3170-rabbitmq branch November 18, 2026 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant