Repository navigation
chore: azure-servicebus-source can process messages in parallel and dead-letter failures, the sink can send bytes and delay messages (#3170) - #3239
Merged
Conversation
…ead-letter failures, the sink can send bytes and delay messages (apache#3170) azure-servicebus-source takes maxConcurrentCalls, how many messages are processed at the same time (with sessions, for each session), and enableDeadLettering, which moves a message whose processing fails to the dead-letter queue straight away, with the exception as the reason, instead of giving it back until the maximum delivery count. azure-servicebus-sink takes binary, to send the body as bytes rather than as text, and scheduledEnqueueDelay, a number of seconds Service Bus holds each message before readers can receive it. camel-azure-servicebus has a scheduledEnqueueTime endpoint option, but it is a fixed OffsetDateTime that cannot be set from a string (no type converter), and it is read only by the scheduleMessages operation. So, when scheduledEnqueueDelay is set, the template sets the CamelAzureServiceBusProducerOperation and CamelAzureServiceBusScheduledEnqueueTime headers to now plus the delay for the send, and removes them afterwards, so the message leaves the Kamelet unchanged. The time is computed with the bean language, so the sink now declares camel:bean. All the new options are optional placeholders without a default: when they are unset, the endpoint URIs are identical to before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #3170:
azure-servicebus-sourcecan process several messages at the same time, and move a message whose processing fails to the dead-letter queue straight away.azure-servicebus-sinkcan send the body as bytes, and have Service Bus hold each message for a delay.The options
maxConcurrentCallsmaxConcurrentCalls: "{{?maxConcurrentCalls}}"enableDeadLetteringenableDeadLettering: "{{?enableDeadLettering}}"truedead-letters a message whose processing fails, instead of abandoning it.binarybinary: "{{?binary}}"truesends the body as bytes instead of text.scheduledEnqueueDelaychoicesets theCamelAzureServiceBusProducerOperationheader toscheduleMessages, andCamelAzureServiceBusScheduledEnqueueTimeto${bean:type:java.time.OffsetDateTime?method=now()}plus{{?scheduledEnqueueDelay}}seconds, before the send; a secondchoiceremoves both headers after itdefault, as in chore: mail-sink can use STARTTLS (#3170) #3174. When they are unset, the optional placeholders drop them and thechoicedoes nothing, so the endpoint URIs are identical tomain's.maxConcurrentCalls:sessionEnabled, it counts for each session. The messages of one session are then processed at the same time too, and lose their order. The descriptions say so.enableDeadLettering:java.lang.IllegalStateException: …) and the stack trace as its error description. The component does this.PEEK_LOCK.subQueue: the component abandons the message there, as before.binary:NullPointerException: 'content' cannot be null). The descriptions say to turn it into JSON first.scheduledEnqueueDelayis a delay, not the component'sscheduledEnqueueTime. The component has ascheduledEnqueueTimeendpoint option, but it cannot be used from a Kamelet:OffsetDateTime, and Camel has no type converter from a String.scheduledEnqueueTime=2026-11-18T23:00:00+01:00fails withNo type converter available to convert from type: java.lang.String to the required type: java.time.OffsetDateTime.scheduleMessagesoperation. The Kamelet usessendMessages, which ignores it.CamelAzureServiceBusScheduledEnqueueTimeheader. It also sets the operation header for that send, so the endpoint staysproducerOperation=sendMessages, and no second sender client is created.CamelAzureServiceBusProducerOperation=scheduleMessagesback, which would schedule its next send to Service Bus as well.${bean:type:…}and an OGNL call), so the sink now declarescamel:bean. The catalog SBOM was regenerated by the build.Verification
Tested with Camel JBang 4.24.0-SNAPSHOT against the Azure Service Bus emulator (
mcr.microsoft.com/azure-messaging/servicebus-emulator, withmcr.microsoft.com/mssql/server:2022-latest). It was configured throughConfig.jsonwith:MaxDeliveryCount: 10;RequiresSession: true;Processing was failed on purpose with
throwException. Received bytes were compared through base64.mainand this branchcamel get endpoint --json: identical URIs (azure-servicebus://plainq?connectionString=…&credentialType=CONNECTION_STRING&producerOperation=sendMessages&serviceBusType=queue, and…&serviceBusReceiveMode=PEEK_LOCK&serviceBusType=queue); message sent and receivedbinary=true(stillproducerOperation=sendMessages). Source endpoint hasenableDeadLettering=true&maxConcurrentCalls=4. Message received 1 s later (scheduledEnqueueDelay: 1)maxConcurrentCallsunsetmaxConcurrentCalls: 5sessionEnabled: true,maxConcurrentCalls: 3, 1 s stepenableDeadLetteringunsetMaxDeliveryCountExceededenableDeadLettering: truesubQueue: DEAD_LETTER_QUEUE): D-1 with reasonjava.lang.IllegalStateException: processing failed on purposeand the stack trace as descriptionenableDeadLettering: trueon a session queue, S-2 failsenableDeadLettering: trueon a topic subscription, T-1 failsenableDeadLettering: truewithRECEIVE_AND_DELETE, R-1 failsenableDeadLettering: truewithsubQueue: DEAD_LETTER_QUEUE, processing failsbinaryunsetefbfbd)binary: truefile:route sending a 1 KiB binary file,binary: true(the partial's example)cmp)binary: true, textblåbærgrødblåbærgrød(UTF-8)binary: true, Map bodyNullPointerException: 'content' cannot be nullscheduledEnqueueDelay: 20, 2 messages, reader runningCamelAzureServiceBusProducerOperationandCamelAzureServiceBusScheduledEnqueueTimeare emptyscheduledEnqueueDelay: 10withsessionId: Bto a session queuescheduledEnqueueDelayset, template without the removal stepop=scheduleMessagesand the scheduled time after the sink, which is why the step is thereCamelAzureServiceBusScheduledEnqueueTimeheader (OffsetDateTime, now + 15 s)sendMessagesNot verified:
maxConcurrentCallsabove 1 combined withmaxConcurrentSessionsabove 1, for the total number of messages in parallel.maxConcurrentCallsunset on a queue without sessions. The description says that the next message may then start before the previous one has finished. This follows from the consumer code, which is the same as for sessions, where chore: the Azure Service Bus Kamelets work with session-enabled queues, and the source can read a dead-letter queue (#3170) #3207 observed it../mvnw clean verify -DskipTestsand then./mvnw clean verifypassed from the root:./mvnw -pl library/camel-kamelets resources:copy-resources@copy-resource-one.KameletsCatalogTestneeded no change.kamelets-index.jsonand the catalog SBOM (camel-beanfor the sink).Claude Code on behalf of Claus Ibsen
🤖 Generated with Claude Code