Skip to content

chore: aws-bedrock-agent-runtime-sink can retrieve passages and ask an agent, and the Redshift Kamelets can size their connection pool (#3170) - #3232

Merged
davsclaus merged 1 commit into
apache:mainfrom
davsclaus:ci-issue-3170-aws-bedrock-redshift
Nov 18, 2026
Merged

davsclaus merged 1 commit into
apache:mainfrom
davsclaus:ci-issue-3170-aws-bedrock-redshift

Conversation

@davsclaus

Copy link
Copy Markdown
Contributor

Part of #3170: aws-bedrock-agent-runtime-sink can find passages in a knowledge base without writing an answer, and ask a Bedrock agent. aws-redshift-source and aws-redshift-sink can size their connection pool. aws-bedrock-text-sink is left unchanged; see "Left out".

The options

aws-bedrock-agent-runtime-sink:

Option Template mapping What it does
operation operation: "{{operation}}", default: retrieveAndGenerate, enum retrieveAndGenerate, retrieve, invokeAgent retrieveAndGenerate is today's behaviour. retrieve gives the passages found, without an answer. invokeAgent sends the question to a Bedrock agent.
agentId agentId: "{{?agentId}}", pattern: ^[0-9A-Za-z]{10}$ The agent, for invokeAgent.
agentAliasId agentAliasId: "{{?agentAliasId}}", same pattern The alias of the agent, such as TSTALIASID, for invokeAgent.
numberOfResults A choice sets the CamelAwsBedrockAgentRuntimeNumberOfResults header to {{?numberOfResults}}, only when the option is set The most passages for retrieve.
searchType The same, with the CamelAwsBedrockAgentRuntimeSearchType header; enum HYBRID, SEMANTIC The kind of search for retrieve.
  • Headers, not endpoint options: the component reads the number of results and the search type only from these two headers. The template sets each header only when its option is set, with the '{{?x}}' != '' check that ceph-sink uses. When neither is set, the request has an empty vectorSearchConfiguration, as before.
  • operation has a default: today the template hard-codes operation: retrieveAndGenerate. Without a default, an unset option would leave the endpoint with no operation, and every message would fail.
  • required: modelId and knowledgeBaseId are no longer required, and their placeholders are now optional. invokeAgent uses neither of them, and retrieve does not use modelId. Their descriptions say which operation needs them. When they are set, the endpoint URI is the same as before.
  • Descriptions:
    • The summary names the three operations.
    • "Output" says what the body is for each operation. For retrieve, it is a List of AWS SDK objects. The CamelAwsBedrockAgentRuntimeRetrievedResults header holds the same List, and it is declared under dataTypes.
    • "Good to know" says which options each operation needs. It says that the passages from retrieve become {} when turned into JSON, so ${body[0].content.text} is the way to read them. It also says that invokeAgent without a session ID starts a new session for each message.
  • Partial page: it has a new "Operations" section with a table, and examples for retrieve and invokeAgent.

aws-redshift-source and aws-redshift-sink, on the commons-dbcp2 BasicDataSource bean, next to connectionProperties from #3181. The names and wording are the same as in #3215 for the other SQL Kamelets:

Option Bean mapping What it does
maxTotal maxTotal: "{{?maxTotal}}" The most connections open at once. 8 when not set.
maxIdle maxIdle: "{{?maxIdle}}" The most unused connections kept open. 8 when not set.
maxWaitMillis maxWaitMillis: "{{?maxWaitMillis}}" How long a message waits for a free connection before it fails. Unlimited when not set.
  • On the source, maxTotal warns that the source keeps one connection while it sends the rows of a poll, and consumedQuery needs a second one.
  • Both partial pages have a "Connection Pool" section.

Left out

  • aws-bedrock-text-sink, operation: converse: the converse operation of camel-aws-bedrock reads the conversation only from the CamelAwsBedrockConverseMessages header. That header must be a List of AWS SDK Message objects, and in POJO mode the body must be a ConverseRequest object. A Kamelet user cannot build either without code, and the component does not turn a text body into a message.
    • With operation: converse on a copy of the Kamelet, every message failed with Converse operation requires messages in header CamelAwsBedrockConverseMessages, and no request reached the server.
    • It needs a Camel change first: converse, and converseStream, would take a text body as the user message when the header is not set. Then the Kamelet can offer converse, where the body is the prompt, the same for every model.
  • guardrailIdentifier, guardrailVersion: the component applies them only to converse, converseStream and applyGuardrail, not to invokeTextModel. They come together with converse.
  • streamOutputMode: it only applies to the streaming operations. A sink gives one result message either way: with complete it is the same text, and with chunks it is a List of pieces of the text. A sink has no use for it.
  • The other agent-runtime operations (invokeFlow, invokeInlineAgent) and the other agent options (enableTrace, sessionId, the agent streamOutputMode) were not part of the gap list.

Verification

Tested with Camel JBang 4.24.0-SNAPSHOT.

Bedrock was not called. Bedrock has no local emulator. uriEndpointOverride pointed at a local Python HTTP server, which recorded each request (path and JSON body) and returned canned responses in the shape of Bedrock's. For invokeAgent, the response was in AWS event-stream encoding, with two chunks and a citation. The test shows what the Kamelet sends and how it reads a response. It does not show what Bedrock does with the request, for example that it rejects a retrieveAndGenerate request without a model.

Redshift: a PostgreSQL 17 container stood in for the database, reached with the Amazon Redshift JDBC driver 2.3.0 that the Kamelets use. Two things were needed for that:

  • PostgreSQL ran with md5 passwords, because the driver does not do SCRAM.
  • A small TCP proxy dropped the startup parameters that PostgreSQL does not know, such as driver_version, and passed everything else on unchanged.

The proxy also counted the open connections. connectionProperties: ssl=false turned off TLS, which the container did not offer.

aws-bedrock-agent-runtime-sink:

Case Result
Options unset, main and this branch, camel get endpoint Identical URIs, operation=retrieveAndGenerate. Identical POST /retrieveAndGenerate requests, with an empty vectorSearchConfiguration. Body Canned RAG answer, session header set.
operation: retrieve, numberOfResults: 3, searchType: HYBRID POST /knowledgebases/AB12CD34EF/retrieve with "vectorSearchConfiguration": {"numberOfResults": 3, "overrideSearchType": "HYBRID"}. The body is a List of 3 KnowledgeBaseRetrievalResult. ${body[0].content.text} = Passage 1, ${body[0].score} = 0.9. The ...RetrievedResults header has 3 entries.
The same body turned into JSON (marshal: json) [{},{},{}]
operation: retrieve, no numberOfResults or searchType "vectorSearchConfiguration": {}: no header was set
operation: retrieve, no knowledgeBaseId Fails before any request: retrieve operation requires knowledgeBaseId in the endpoint configuration
operation: invokeAgent, agentId: KX7Q2M9WZB, agentAliasId: TSTALIASID, no modelId and no knowledgeBaseId POST /agents/KX7Q2M9WZB/agentAliases/TSTALIASID/sessions/<random UUID>/text with inputText. The body is Canned agent answer, as a String. The session header holds the new UUID, and the citations header has 1 entry.
The same with the CamelAwsBedrockAgentRuntimeSessionId: my-session-42 header The request goes to .../sessions/my-session-42/text, and the header keeps my-session-42
operation: invokeAgent, no agentId Fails before any request: invokeAgent operation requires agentId in configuration or header CamelAwsBedrockAgentRuntimeAgentId
Default operation without modelId The request is sent without modelArn. Real Bedrock was not tried.

aws-bedrock-text-sink, not changed, on a scratch copy of the Kamelet:

Case Result
operation: converse, guardrailIdentifier, guardrailVersion Fails before any request: Converse operation requires messages in header CamelAwsBedrockConverseMessages

aws-redshift-source and aws-redshift-sink:

Case Result
Options unset, main and this branch, camel get endpoint, source and sink Identical
Source reads 5 rows, consumedQuery marks them, sink inserts a row (main and this branch) 5 rows sent and marked, 1 row inserted, CamelSqlUpdateCount 1
Sink, pool unset, 4 messages at once, query sleeps 2 s All 4 OK, 4 connections open
Sink, maxTotal: 1, maxWaitMillis: 1000, the same 4 messages 3 fail with Failed to obtain JDBC Connection, and 1 is OK
Sink, pool unset, 4 messages that finish 1 s apart 4 connections stay open until the integration stops
Sink, maxTotal: 4, maxIdle: 1, the same 4 messages After each message finishes, a connection is closed: 3, 2, 1 open. The last stays open until the integration stops.
Source, maxTotal: 1, maxWaitMillis: 2000, consumedQuery Each row is sent, then consumedQuery gives up with Timeout waiting for idle object, maxWaitDuration=PT1.99998S. No row is marked.
Source, maxTotal: 1, no maxWaitMillis, consumedQuery Blocks after the first row until the integration stops
Source, maxTotal: 2, maxWaitMillis: 2000, consumedQuery All 5 rows sent and marked

./mvnw clean verify -DskipTests and then ./mvnw clean verify passed. No test counts needed changing.

Claude Code on behalf of Claus Ibsen

🤖 Generated with Claude Code

…n agent, and the Redshift Kamelets can size their connection pool (apache#3170)

aws-bedrock-agent-runtime-sink takes `operation`: `retrieveAndGenerate`
(the default, as before), `retrieve` and `invokeAgent`. For `retrieve`,
`numberOfResults` and `searchType` set the vector search of the request,
through the headers the component reads, and only when they are set.
For `invokeAgent`, `agentId` and `agentAliasId` go on the endpoint.
`modelId` and `knowledgeBaseId` are no longer required, as `invokeAgent`
uses neither and `retrieve` no model; they are now optional placeholders.
With the new options unset, the endpoint URI is the same as before.

aws-redshift-source and aws-redshift-sink take `maxTotal`, `maxIdle` and
`maxWaitMillis` for the commons-dbcp2 data source, with the names and
wording of the other SQL Kamelets.

aws-bedrock-text-sink is unchanged: the converse operation reads the
conversation only from a header holding AWS SDK objects, which a Kamelet
user cannot set, and the guardrail options apply only to converse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@davsclaus
davsclaus merged commit 486f487 into apache:main Nov 18, 2026
6 checks passed
@davsclaus
davsclaus deleted the ci-issue-3170-aws-bedrock-redshift branch November 18, 2026 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant