Repository navigation
Description
User Story
As a maintainer porting e2e/rust tests into features and drivers suites, I want those suites to read OPENSHELL_TEST_UNSUPPORTED, so a test that fails on one environment gets a tracked unsupported.yaml entry instead of being left off that CI row.
Description
#5755 added unsupported.yaml for conformance only. Features and drivers tests still run through the #[tokio::test] wrappers that #[testkit::test] generates under cfg(test). The standard test harness decides "ignored" at compile time, so it can't skip a test per environment.
testkit::main() already answers the libtest interface that cargo test and nextest use. A test target with harness = false builds without cfg(test), so the wrappers drop out and the target runs through testkit::main(), as conformance does. This is a narrower first part of #5765 step 2: no [[bin]] per suite and no file moves.
Changes
- Each test target under
tests/suites/features/*andtests/suites/drivers/*(9 on main) gets a[[test]]entry withharness = falseand a name that is unique across suites, plusfn main() -> ExitCode { testkit::main() }. A unique name is needed because the crate name is the first part of an entry:docker-updatesandkubernetes-updatesboth havetests/update.rs. tests/suites/conformance/unsupported.yamlmoves totests/suites/unsupported.yaml, and the script that reads it loses itsconformance-name.- The CI check (
tasks/scripts/check-conformance-program.sh):- accepts environments from every family's rows in
e2e-suites.yml; - validates the entries for every workspace with
cargo nextest list; - rejects entries whose crate exists in no suite (the follow-up listed in test(testkit): run every testkit suite as one program, and let nextest run the prebuilt program #5765);
- fails if a testkit test target lacks
harness = false.
- accepts environments from every family's rows in
- The "Marking a test unsupported in CI" docs move from
tests/suites/conformance/README.mdto one shared place.
Context
- Part of test(e2e): decide which e2e/rust tests to port and in what order #5297. Narrows test(testkit): run every testkit suite as one program, and let nextest run the prebuilt program #5765 step 2; the rest of test(testkit): run every testkit suite as one program, and let nextest run the prebuilt program #5765 (prebuilt programs, nextest metadata) stays there.
- In-flight ports adding suites (test(e2e): port Docker and Podman driver feature tests from e2e/rust #5695, test(e2e): port gateway-configuration tests that need no restart from e2e/rust to feature suites #5729, test(e2e): port corporate-proxy tests from e2e/rust to features/corporate-proxy #5730, test(e2e): port OIDC authorization tests from e2e/rust to features/oidc-auth #5731) follow the pattern once this lands, or this PR converts them if they land first.
Definition of Done
- Every features and drivers test target runs through
testkit::main();cargo nextest listshows the same tests as before. - A test listed in
unsupported.yamlfor an environment shows as skipped with its issue on that environment's row, in a features suite and in conformance. - The CI check covers every suite and rejects unknown crates, unknown environments and test targets without
harness = false. - Docs describe one
unsupported.yamlfor all suites.
Activity