Skip to content

CAMEL-23326: exclude transitive catalog-dependent modules from incremental testing - #30263

Open
gnodet wants to merge 2 commits into
apache:mainfrom
gnodet:fix/camel-23326-exclusion-list
Open

gnodet wants to merge 2 commits into
apache:mainfrom
gnodet:fix/camel-23326-exclusion-list

Conversation

@gnodet

@gnodet gnodet commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add 5 missing artifact IDs to EXCLUSION_LIST in incremental-build.sh so they are not spuriously pulled into targeted test runs when unrelated modules change.

Root Cause

The incremental build script uses -amd (also-make-dependents) when computing which modules to test after a change. Any module that depends on camel-catalog (directly or transitively) is pulled into the reactor. Five modules were missing from EXCLUSION_LIST and were being tested on every camel-catalog-touching PR:

  • camel-jbang-mcp
  • camel-jbang-plugin-validate
  • camel-launcher-container
  • camel-yaml-dsl-validator
  • camel-yaml-dsl-validator-maven-plugin

Fix

Add the five artifact IDs to EXCLUSION_LIST on line 51 of .github/actions/incremental-build/incremental-build.sh, grouped after the existing camel-launcher entry.

The existing filterExclusions() function (line 83) already handles the "this module IS the changed module" case: when any of these modules is itself the primary changed module, its exclusion entry is automatically lifted, so direct changes to these modules are still tested normally.

Add manual-it-mapping.txt entries for all 5 new modules, pointing to camel-jbang-it. When a contributor modifies one of these modules, CI will now post an advisory to run:

mvn verify -f dsl/camel-jbang/camel-jbang-it -Djbang-it-test

Add incremental-build-filter-test.sh with 7 unit tests for filterExclusions(): path-based match, colon-style :artifactId match, multiple matches, all-lifted, empty build_pl, and prefix non-match edge case. The script is picked up automatically by the pr-ci-scripts-validation.yml CI workflow.

Note: camel-jbang-plugin-route-parser was removed from the codebase (CAMEL-25587) and is not added.

Tradeoff

With -amd, !:module removes the module from the reactor (not even compiled). These 5 modules also stop being tested for their direct upstreams (not just camel-catalog). This is consistent with how camel-jbang-core and camel-yaml-dsl are already handled. The manual-it-mapping.txt entries advise contributors to run the integration tests manually when they touch these modules.

Fixes CAMEL-23326


Hermes Agent (Claude Sonnet 4.6) on behalf of Guillaume Nodet

…ental testing

Five modules were missing from EXCLUSION_LIST in incremental-build.sh and
were being pulled into the test reactor via -amd whenever camel-catalog
or its dependents changed:

  - camel-jbang-mcp
  - camel-jbang-plugin-validate
  - camel-launcher-container
  - camel-yaml-dsl-validator
  - camel-yaml-dsl-validator-maven-plugin

The existing filterExclusions() already handles the "primary changed
module" case: when any of these modules is itself the changed module,
its exclusion entry is automatically lifted so the test runs normally.

Note: camel-jbang-plugin-route-parser was removed from the codebase
(CAMEL-25587) and is not added.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@gnodet gnodet added the task label Nov 17, 2026
@gnodet gnodet added this to the 4.25.0 milestone Nov 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@gnodet
gnodet marked this pull request as ready for review November 17, 2026 21:15

@davsclaus davsclaus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this cuts real noise: these five come in through camel-catalog on almost every component PR. I checked that the directory names match the artifact IDs (so filterExclusions lifts an entry when the module itself changed) and that exclusions outside the -amd reactor are dropped before Maven sees them.

One tradeoff that should be explicit: with -amd, !:module removes the module from the reactor (not even compiled), and the entry is only lifted when that module is itself a primary change. So these modules also stop being tested for their direct upstreams, not just camel-catalog:

  • camel-yaml-dsl-validator (30 test classes): changes to camel-yaml-dsl, camel-yaml-io, camel-xml-io, camel-core-languages, camel-kamelet-main-support.
  • camel-jbang-mcp (44 test classes): changes to camel-chaos, camel-rest-openapi, camel-yaml-io, camel-xml-io, camel-jbang-core.
  • camel-jbang-plugin-validate and camel-yaml-dsl-validator-maven-plugin: a change to camel-yaml-dsl-validator itself no longer compiles or tests its two direct consumers.

That fits how camel-jbang-core and camel-yaml-dsl are already handled, so I'm fine with the direction, but could we at least add manual-it-mapping.txt entries (e.g. camel-yaml-dsl-validator -> camel-jbang-plugin-validate, camel-yaml-io/camel-yaml-dsl -> camel-yaml-dsl-validator) so CI tells the author which tests to run, or note the tradeoff in CI-ARCHITECTURE.md? A small committed test for filterExclusions next to pr_diff_test.sh would also be welcome (the description mentions a local one).

Reviewed with Claude Code on behalf of davsclaus. This review was generated by an AI agent and may contain inaccuracies; please verify all suggestions before applying.

@oscerd oscerd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right maintenance of the exclusion list. The CI step builds the components without the catalog-dependent and tooling modules, and the list had fallen behind the modules added since: this adds camel-jbang-mcp, camel-jbang-plugin-validate, camel-launcher-container, camel-yaml-dsl-validator and camel-yaml-dsl-validator-maven-plugin, which are exactly the jbang, launcher and validator modules that belong with the ones already excluded. Keeping the list current is what stops those modules from being pulled transitively into a step that is not meant to build them. Small and correct.

Claude Code on behalf of oscerd

…Exclusions test

- Add manual-it-mapping.txt entries for the 5 newly excluded modules
  (camel-jbang-mcp, camel-jbang-plugin-validate, camel-launcher-container,
  camel-yaml-dsl-validator, camel-yaml-dsl-validator-maven-plugin), all
  pointing to camel-jbang-it, so contributors are advised to run the
  integration tests when modifying these modules.
- Add incremental-build-filter-test.sh with 7 unit tests for filterExclusions:
  covers path-based match, colon-style :artifactId match, multiple matches,
  all-lifted, empty build_pl, and prefix non-match edge case.

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The 5 new EXCLUSION_LIST entries are correct and match the 5 manual-it-mapping.txt additions 1:1. All of @davsclaus's review feedback is addressed: the mapping entries are in place and the new incremental-build-filter-test.sh covers 7 cases including the prefix non-match edge case that validates the comma-sentinel matching logic.

The awk+eval extraction approach (instead of source) is the right design choice since incremental-build.sh has executable code outside functions that would trigger side effects on direct source.

This review was generated by an AI agent, Hermès on behalf of @gnodet.

@davsclaus davsclaus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the mapping entries, the tradeoff section and the tests.

One problem: incremental-build-filter-test.sh is not run by CI. pr-ci-scripts-validation.yml runs for test in *_test.sh, and this file name has -test.sh (hyphen), so the "PR CI scripts validation" run on d2feb36 passed without it. Renaming it to incremental_build_filter_test.sh, like pr_diff_test.sh and reactor_timing_test.sh, makes the description's "picked up automatically" true.

On the mapping: the new entries fire when one of the five modules is itself changed, and then its exclusion is lifted and its own tests already run. The gap from my earlier review is the other direction: a change to camel-yaml-dsl / camel-yaml-io no longer builds camel-yaml-dsl-validator, and a change to camel-yaml-dsl-validator no longer builds camel-jbang-plugin-validate and camel-yaml-dsl-validator-maven-plugin. Entries keyed on those upstreams (e.g. camel-yaml-dsl-validator:dsl/camel-yaml-dsl/camel-yaml-dsl-validator-maven-plugin:mvn verify -f ...) would cover it. With the tradeoff written down in the description, I'm fine leaving that as is if you prefer; the rename is the part I'd like before merging.

Reviewed with Claude Code on behalf of davsclaus. This review was generated by an AI agent and may contain inaccuracies; please verify all suggestions before applying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants