Skip to content

feat(tools): provision tools from declared environment selections - #198

Merged
lurtz merged 7 commits into
eclipse-score:mainfrom
etas-contrib:refactor/tool-environment-selections
Nov 5, 2026
Merged

lurtz merged 7 commits into
eclipse-score:mainfrom
etas-contrib:refactor/tool-environment-selections

Conversation

@AlexanderLanin

@AlexanderLanin AlexanderLanin commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Why

Adding or removing a tool currently means following several installation scripts and separate version files. Maintainers need one place to see which tools each environment provides, and to change that selection without rewriting installation steps.

For developers, basedpyright and pytest become available to all DevContainer users alongside pre-commit, rather than being installed into one user's home directory. They also gain Bazel targets using the same package versions. Invalid selections, missing declared dependencies and unsupported platform artifacts are reported before installation starts.

What changes

  • Tool selection: tools/environments.json lists the directly managed tools for the two existing container features and this repository's own setup.
  • Package definitions: tools/providers.json holds Python/npm packages, APT policies and special setup metadata. Native release versions and checksums stay in the existing lockfiles; Bazel reads Python definitions from the same JSON source.
  • Installation: each setup requests its selection with one installer call. The installer manages temporary build packages, tool setup, cleanup and required privileges, while npm retains the development user's environment. Tool-specific setup lives beside the installer.
  • Cleanup and fixes: remove the replaced YAML definitions and yq bootstrap, and repair config/state parent ownership without changing existing private permissions.

Existing tool versions and APT policies are retained. Distribution-selected packages still follow Ubuntu; the change does not make those packages exactly pinned.

Part of #190; builds on #196.


Possible split if this is too large

If this PR is too large to review as one change, we could split it into the following independently usable steps. Extractions #200 and #201 are ready for review, each based directly on main and independent of the other. This PR retains the complete implementation until the extractions land.

PR Scope Result
1. Fix directory ownership (#200) Repair config/state parent ownership while preserving private permissions, with regression coverage. An independent bugfix.
2. Consolidate package definitions (#201) Introduce providers.json and the Bazel adapter; replace the Python/YAML definitions and yq. Adapt existing installers to read the new metadata and expose the additional Python Bazel targets. Shared definitions, with existing installation flows largely retained.
3. Migrate repository setup Add environment selection, validation and native/Python/npm installation with privilege handling; migrate post_create_command.sh. The new model is used by this repository's own setup.
4. Migrate container features Add APT, temporary build dependencies and declared setup scripts; migrate both features, including system-wide Python tools, and remove their inline installation logic. The complete behavior proposed by this PR.

Steps 2 → 3 → 4 build on each other. Step 1 is independent and should land before step 4. If step 4 is still too large, it can be split further by bazel-feature and s-core-local. Tests and documentation would accompany their respective changes; intermediate steps would keep one source for each version rather than introducing duplicate definitions.

Copilot AI balanced review requested due to automatic review settings November 3, 2026 12:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

CodeQL invokes tar -I zstd without declaring a zstd dependency, allowing validation to pass and extraction to fail on images without zstd; add "requires": ["zstd"] to its provider definition.

1 open issue
What changed in this PR

This PR consolidates tool selections for SCORE’s existing development environments, separating caller membership from version metadata and installation logic. It builds on the shared catalog reader without adding a new image.

Changes:

  • Declare complete environment selections and apply them through validated build, runtime and cleanup phases.
  • Consolidate provider metadata and Python pins, replacing feature YAML files and the downloaded yq parser.
  • Expose basedpyright and pytest system-wide and through Bazel, with updated tests and documentation.
File Description
tools/​tests/​environment_test.py Adds selection validation and installer integration coverage.
tools/​tests/​catalog_test.py Updates expected Python catalog membership.
tools/​README.md Adds basedpyright and pytest to the tool table.
tools/​providers.json Defines package policies and special setup metadata.
tools/​lockfiles/​python_tools.bzl Adds shared basedpyright and pytest pins.
tools/​internal/​sync_readme.py Validates selections during documentation generation.
tools/​internal/​README.md Documents selections, provider ownership and installation phases.
tools/​internal/​devcontainer/​load_feature_versions.sh Replaces yq downloads with the Python metadata bridge.
tools/​internal/​devcontainer/​install.py Adds environment installation and metadata query commands.
tools/​internal/​catalog.py Adds provider loading and selection validation.
tools/​environments.json Declares tool membership for the three existing callers.
tools/​BUILD.bazel Exposes basedpyright and pytest targets.
src/​s-core-devcontainer/​.devcontainer/​s-core-local/​versions.yaml Removes superseded version metadata.
src/​s-core-devcontainer/​.devcontainer/​s-core-local/​tests/​test_default.sh Uses shared metadata and strips Debian epochs in version checks.
src/​s-core-devcontainer/​.devcontainer/​s-core-local/​install.sh Applies selections and gates special setup by membership.
src/​s-core-devcontainer/​.devcontainer/​bazel-feature/​versions.yaml Removes superseded Bazel feature metadata.
src/​s-core-devcontainer/​.devcontainer/​bazel-feature/​tests/​test_default.sh Reads versions through the shared metadata bridge.
src/​s-core-devcontainer/​.devcontainer/​bazel-feature/​install.sh Applies the Bazel selection and gates special setup.
REUSE.toml Adds license coverage for the new JSON files.
.pre-commit-config.yaml Watches environment and provider metadata changes.
.github/​renovate.json5 Removes obsolete YAML version matchers.
.devcontainer/​post_create_command.sh Applies repository selections while preserving user-level npm setup.

Review effort: Balanced

Comment thread tools/providers.json Resolved
Comment thread tools/internal/devcontainer/install.py Resolved
Comment thread tools/providers.json Resolved
Comment thread tools/providers.json Resolved
Comment thread tools/providers.json Resolved
Comment thread tools/providers.json Resolved
Comment thread tools/providers.json Resolved
Copilot AI balanced review requested due to automatic review settings November 3, 2026 13:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the tool-selection and provisioning refactor.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 3, 2026 19:58

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The ownership fix uses install -d, which resets existing private directories to mode 0755 and can expose configuration or state files to other users.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Medium severity Preserve existing directory permissions during ownership fix

src/​s-core-devcontainer/​.devcontainer/​s-core-local/​fix_config_volume_ownership.sh:43

install -d applies its default mode 0755 even to existing directories. If a user has made .config or .local private with mode 0700, this ownership fix makes it traversable by other users and can expose readable configuration/state files that relied on the parent's permissions. Create missing parents with mkdir -p, then change ownership non-recursively so existing access permissions are preserved.

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 4, 2026 10:44

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The installer omits JAVA_HOME setup for non-root APT-only OpenJDK selections and unnecessarily requires sudo and modifies system packages for native-only s-core-local installs.

2 open issues

Review effort: Balanced

Comment thread tools/internal/devcontainer/install.py Resolved
Comment thread tools/internal/devcontainer/install.py Resolved
Comment thread tools/internal/README.md Resolved
Comment thread tools/lockfiles/python_tools.bzl Resolved
Comment thread tools/BUILD.bazel Resolved
@AlexanderLanin AlexanderLanin changed the title refactor(tools): declare complete environment selections feat(tools): provision tools from declared environment selections Nov 4, 2026
Copilot AI balanced review requested due to automatic review settings November 4, 2026 12:17

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the tool-catalog consolidation.

1 open issue
1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 4, 2026 12:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The privileged provisioning lifecycle and shared Python environments require real amd64/ARM64 container validation and generated-catalog resolution through Bazel, which unavailable runtime results and mocked privilege tests cannot establish.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 4, 2026 12:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the tool-catalog consolidation.

No open issues 🎉

Review effort: Balanced

Comment thread tools/README.md Resolved
@lurtz
lurtz added this pull request to the merge queue Nov 5, 2026
Merged via the queue into eclipse-score:main with commit c40e0bf Nov 5, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants