C++: Move build-mode-none extractor into codeql by paldepind · Pull Request #22688 · github/codeql

paldepind · 2026-09-28T12:55:54Z

Move the C/C++ build-mode-none Rust extractor into cpp/extractor/bmn so it can be built, tested, and maintained in the public repository.

This PR:

imports the existing extractor sources separately from subsequent adaptations;
adds a dedicated generated Rust dependency universe;
exposes the extractor and unit tests through Bazel;
adds Cargo/Bazel CI, lint integration, and developer documentation.

Validation:

Cargo formatting and Clippy pass;
all 133 Rust unit tests pass;
the Bazel target and test graph resolve successfully.

A dependent github/semmle-code PR will switch C++ packaging to this target and remove the migrated implementation.

Add the existing Rust implementation, manifests, documentation, and test fixtures verbatim so the later build integration remains separately reviewable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add a dedicated crate-universe vendoring target and update script for the build-mode-none extractor. Expose its Cargo manifests to Bazel and keep the standalone Cargo workspace separate from the repository-wide workspace. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Check in the Bazel crate-universe output produced by update_cpp_bmn_deps.sh so the build-mode-none extractor can consume its pinned Cargo dependencies without regenerating them during builds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add a public Bazel target for the Rust build-mode-none extractor, expose its generated crate universe through the standalone module, and resolve test resources through Bazel's TEST_SRCDIR runfiles root. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The updated Rust toolchain introduced additional Clippy lints. This fixes the lints.

Run Cargo and Bazel checks when the build-mode-none extractor changes, integrate its Rust lint into pre-commit, and document the standalone development workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot review overview

🟡 Changes recommended

Extraction failure propagation, overlay invalidation, source discovery, and dependency-installation issues can produce incomplete or misleading extraction results.

Review effort: Balanced
Findings: 2 · 3

Open (5)

Report extraction failure when commands cannot launch or succeed · New
Handle deleted headers when computing overlay dependency changes · New
Deduplicate package installation attempts before invoking installer · New
Allow whitespace between # and include in preprocessing directives · New
Recognize uppercase C and C++ source file extensions · New

What changed in this PR

Moves the C/C++ build-mode-none Rust extractor into the public CodeQL repository and integrates it with existing build, test, lint, and CI infrastructure.

Changes:

Imports the extractor, unit tests, fixtures, and developer documentation.
Adds Cargo and Bazel targets with a dedicated generated Rust dependency universe.
Adds CI and repository-wide lint/dependency-update integration.

File	Description
`.github/workflows/cpp-bmn.yml`	Adds Cargo and Bazel CI.
`.pre-commit-config.yaml`	Registers BMN linting.
`MODULE.bazel`	Registers BMN Rust repositories.
`lint`	Runs the BMN linter.
`cpp/extractor/bmn/.gitignore`	Ignores Cargo artifacts.
`cpp/extractor/bmn/BUILD.bazel`	Defines binary and test targets.
`cpp/extractor/bmn/Cargo.lock`	Locks Rust dependencies.
`cpp/extractor/bmn/Cargo.toml`	Defines the Rust package.
`cpp/extractor/bmn/FILE_FILTERING.md`	Documents file filtering.
`cpp/extractor/bmn/README.md`	Documents development workflows.
`cpp/extractor/bmn/lint`	Runs formatting and Clippy checks.
`cpp/extractor/bmn/src/compiler.rs`	Discovers and represents compilers.
`cpp/extractor/bmn/src/dependency_installation.rs`	Installs inferred packages.
`cpp/extractor/bmn/src/directory_tree.rs`	Models project files.
`cpp/extractor/bmn/src/environment.rs`	Reads extraction environment settings.
`cpp/extractor/bmn/src/extraction_command_runner.rs`	Executes extraction commands.
`cpp/extractor/bmn/src/file_filter.rs`	Implements source filtering.
`cpp/extractor/bmn/src/gnu_compiler_default_include_finder.rs`	Discovers compiler include paths.
`cpp/extractor/bmn/src/include_finder.rs`	Resolves include dependencies.
`cpp/extractor/bmn/src/include_scanner.rs`	Parses include directives.
`cpp/extractor/bmn/src/logger.rs`	Configures logging.
`cpp/extractor/bmn/src/main.rs`	Orchestrates extraction.
`cpp/extractor/bmn/src/overlays.rs`	Selects overlay dependencies.
`cpp/extractor/bmn/src/package_database.rs`	Queries package metadata.
`cpp/extractor/bmn/src/path_utils.rs`	Provides path utilities.
`cpp/extractor/bmn/src/project_definitions.rs`	Discovers project sources.
`cpp/extractor/bmn/src/telemetry.rs`	Emits extraction telemetry.
`cpp/extractor/bmn/src/test_utils.rs`	Provides test helpers.
`cpp/extractor/bmn/src/timing.rs`	Records operation timings.
`cpp/extractor/bmn/tests/resources/**`	Adds 24 extractor test fixtures.
`misc/bazel/3rdparty/BUILD.bazel`	Defines dependency generation.
`misc/bazel/3rdparty/cpp_bmn_deps_extension.bzl`	Exposes generated repositories.
`misc/bazel/3rdparty/cpp_bmn_deps/BUILD.bazel`	Configures crate vendoring.
`misc/bazel/3rdparty/cpp_bmn_deps/alias_rules.bzl`	Defines generated aliases.
`misc/bazel/3rdparty/cpp_bmn_deps/crates.bzl`	Registers vendored crates.
`misc/bazel/3rdparty/cpp_bmn_deps/defs.bzl`	Defines generated dependencies.
`misc/bazel/3rdparty/cpp_bmn_deps/BUILD.*.bazel`	Adds 163 generated crate targets.
`misc/bazel/3rdparty/update_cargo_deps.sh`	Includes BMN dependency updates.
`misc/bazel/3rdparty/update_cpp_bmn_deps.sh`	Regenerates BMN dependencies.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+        .attributes(extraction_stats.get_telemetry_value()),
+    );
+
+    Ok(())


+        let has_changed_header = result
+            .resolved_headers
+            .iter()
+            .any(|header| changed_files.contains(header));


+    let mut already_installed_packages = HashSet::new();
+
+    // Try to install packages for each missing include
+    let (installed_packages, failed_packages) = inferred_dependencies
+        .filter_map(|package_name| {
+            (!already_installed_packages.contains(&package_name)).then(|| {
+                let result = package_installer.install_package(&package_name);
+                if let Err(ref e) = result {
+                    info!("Package {package_name:?} was not installed due to error {e:?}.");
+                    Err(InstallationFailure {
+                        package_name,
+                        installation_error: e.to_string(),
+                    })
+                } else {
+                    debug!("Installed package {package_name:?}.");
+                    already_installed_packages.insert(package_name.clone());
+                    Ok(package_name)
+                }
+            })
+        })
+        .partition_result();


+static SYSTEM_INCLUDE_RE: LazyLock<Regex> = LazyLock::new(|| {
+    Regex::new(r#"^\s*#include\s+<([^">]+)>"#).expect("Failed to compile system include regex")
+});
+static LOCAL_INCLUDE_RE: LazyLock<Regex> = LazyLock::new(|| {
+    Regex::new(r#"^\s*#include\s+"([^">]+)""#).expect("Failed to compile local include regex")
+});


+fn get_extension_info(ext: &str) -> Option<(Language, bool)> {
+    match ext {
+        "c" => Some((Language::C, false)),
+        "h" => Some((Language::C, true)),
+        "cpp" | "cc" | "cxx" | "c++" => Some((Language::Cpp, false)),
+        "hpp" | "hxx" | "hh" | "h++" | "inc" => Some((Language::Cpp, true)),
+        _ => None,
+    }
+}


Regenerate the checked-in crate universe with the new cpp_bmn vendor target instead of adapting the former semmle-code output. This includes the generated build-script packages and the root vendor repository required by bazel mod tidy. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 87b09ca6-1af2-45eb-ab29-894f5128e483

paldepind and others added 6 commits September 28, 2026 14:25

C++: Import build mode none extractor sources

137b64f

Add the existing Rust implementation, manifests, documentation, and test fixtures verbatim so the later build integration remains separately reviewable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

C++: Fix BMN Rust lints

858c6f3

The updated Rust toolchain introduced additional Clippy lints. This fixes the lints.

C++: Add BMN CI and developer guidance

a1df8c2

Run Cargo and Bazel checks when the build-mode-none extractor changes, integrate its Rust lint into pre-commit, and document the standalone development workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI balanced review requested due to automatic review settings September 28, 2026 12:55

paldepind requested review from a team as code owners September 28, 2026 12:55

Copilot started reviewing on behalf of paldepind September 28, 2026 12:56 View session

paldepind marked this pull request as draft September 28, 2026 12:58

Copilot AI reviewed Sep 28, 2026

View reviewed changes

github-actions Bot added C++ documentation labels Sep 28, 2026

Navigation Menu

C++: Move build-mode-none extractor into codeql - #22688

C++: Move build-mode-none extractor into codeql#22688
paldepind wants to merge 7 commits into
mainfrom
cpp/bmn-extractor-move

paldepind commented Sep 28, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

paldepind commented Sep 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants