test: rework `--print-config current` tests and add `#[nightly_only_test]` annotation by ytmimi · Pull Request #7240 · rust-lang/rustfmt

ytmimi · 2026-11-10T01:55:52Z

These tests were changed to use file!() instead of .. Using file!() will ensure that rustfmt's rustfmt.toml is loaded even when running in r-l/r's CI.

As a result, we now need to use #[nightly_only_test] since unstable option's are set in rustfmt's rustfmt.toml.

Although this doesn't fully address the --print-config current issue (see #7241), preventing the wrong rustfmt.toml from loading should help us prevent another case like rust-lang/rust#165713

Those tests were introduced in #7196, and synced to r-l/r in rust-lang/rust#165378

I did not use an LLM to create a change in this PR.
I used an LLM to create a change in this PR, and I have explained below how it was used.

r? @jieyouxu

cc: @matthewhughes934

jieyouxu · 2026-11-10T02:07:53Z

    assert_eq!(exit_code, 0);
 }

+// Issue with rust-lang/rust CI. See <https://github.com/rust-lang/rust/pull/165713>


Question: isn't this just release channel things? I.e. --color=never is unstable so we cannot use them on the non-nightly {beta, stable} channels? And so it would certainly fail on the beta backport PR. (To be fair, #[rustfmt_only_ci_test] is fine to unblock the beta backport, it just seems unnecessarily strong to gate it on this repo's CI only.)

Should this be #[nightly_only_test]? We don't actually need to gate this on rustfmt CI only right, it's not trying to do in-source-directory modifications that's not allowed on rust-lang/rust CI's read-only checkout directory?

View changes since the review

Let me double check locally if #[nightly_only_test] will work for this. If so I'd prefer to do that instead. Thanks for calling that out.

rustfmt/config_proc_macro/src/lib.rs

Lines 41 to 44 in 85f6ad5

pub fn nightly_only_test(_args: TokenStream, input: TokenStream) -> TokenStream {

// if CFG_RELEASE_CHANNEL is not set we default to nightly, hence why the default is true

if option_env!("CFG_RELEASE_CHANNEL").map_or(true, |c| c == "nightly" || c == "dev") {

input

Yeah, I believe that should work, bootstrap sets these in r-l/r CI. But yeah please double-check just in case...

FWIW, the warnings came from unstable features in rust's own rustfmt.toml, not anything the test itself is doing. rust-lang/rust#165713 (comment)

(and warnings become test failures when it asserts stderr to be empty)

Oh I see, they're picking up "collateral". That's not ideal 🤔 Right, it's because r-l/r has

https://github.com/rust-lang/rust/blob/ac8310ba643a3796cbe77d2e06944d7c8513999f/rustfmt.toml#L1-L7

But yeah, I think nightly gating should still work.

jieyouxu

Thanks, you can r=me if you prefer to gate it on this repo's CI only. I wanted to ask if this just needs to be nightly gated in case it's just release channel logistics.

View changes since this review

ytmimi · 2026-11-10T03:02:48Z

For reference this was the error in r-l/r CI:

       Running tests/rustfmt/main.rs (obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/build/rustfmt-nightly/ef09887b5eebe2e7/out/rustfmt-ef09887b5eebe2e7)
  
  running 20 tests
  test config_path_errors_when_invalid_file_given ... ok
  test config_path_searches_home_dir_when_dir_path_no_resolve ... ok
  test config_path_searches_config_dir_when_dir_path_no_resolve ... ok
  test config_path_walks_parent_directories_with_dir_name ... ok
  test mod_resolution_error_multiple_candidate_files ... ok
  test mod_resolution_error_relative_module_not_found ... ok
  test mod_resolution_error_path_attribute_does_not_exist ... ok
  test print_config_errors_minimal_with_no_path ... ok
  test print_config_default ... ok
  test mod_resolution_error_sibling_module_not_found ... ok
  test print_config_merges_multiple_overrides_from_args ... FAILED
  test print_config_errors_on_unknown_option ... ok
  test print_config_writes_to_file ... ignored
  test print_config_errors_minimal_without_input_file ... ok
  test rustfmt_allow_not_a_dir_errors ... ok
  test print_config_merges_override_from_args ... FAILED
  test rustfmt_emits_error_on_line_overflow_true ... ok
  test rustfmt_error_improvement_regarding_invalid_toml ... ok
  test rustfmt_emits_error_when_control_brace_style_is_always_next_line ... ok
  test dont_emit_ICE ... ok
  
  failures:
  
  ---- print_config_merges_multiple_overrides_from_args stdout ----
  
  thread 'print_config_merges_multiple_overrides_from_args' (53523) panicked at src/tools/rustfmt/tests/rustfmt/main.rs:171:5:
  assertion `left == right` failed
    left: "Warning: can't set `imports_granularity = Module`, unstable features are only available in nightly channel.\nWarning: can't set `group_imports = StdExternalCrate`, unstable features are only available in nightly channel.\nWarning: can't set `ignore = IgnoreList { path_set: {\"/tests/crashes/\", \"library/backtrace\", \"/vendor/\", \"/tests/rustdoc-ui/\", \"src/tools/miri\", \"src/tools/rustc-perf\", \"/tests/pretty/\", \"/tests/rustdoc-html/\", \"library/compiler-builtins\", \"library/portable-simd\", \"/tests/codegen-llvm/simd-intrinsic/\", \"/tests/ui/\", \"src/doc/embedded-book\", \"compiler/rustc_codegen_cranelift/scripts\", \"src/tools/cargo\", \"src/tools/rustfmt\", \"/tests/incremental/\", \"compiler/rustc_codegen_gcc/src/intrinsic/archs.rs\", \"library/stdarch\", \"/build/\", \"src/doc/reference\", \"/*-build/\", \"src/doc/rust-by-example\", \"src/doc/nomicon\", \"/tests/debuginfo/\", \"/build-*/\", \"/tests/rustdoc-gui/\", \"src/tools/clippy\", \"compiler/rustc_codegen_gcc/tests\", \"/tests/run-make/export\", \"/tests/ui-fulldeps/\", \"src/tools/enzyme\", \"src/doc/book\", \"src/llvm-project\", \"src/doc/rustc-dev-guide\", \"src/doc/edition-guide\", \"src/gcc\", \"compiler/rustc_codegen_gcc/example\", \"src/tools/rust-analyzer\"}, rustfmt_toml_path: \"\" }`, unstable features are only available in nightly channel.\n"
   right: ""
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  ---- print_config_merges_multiple_overrides_from_args stdout end ----
  ---- print_config_merges_override_from_args stdout ----
  
  thread 'print_config_merges_override_from_args' (53525) panicked at src/tools/rustfmt/tests/rustfmt/main.rs:149:5:
  assertion `left == right` failed
    left: "Warning: can't set `imports_granularity = Module`, unstable features are only available in nightly channel.\nWarning: can't set `group_imports = StdExternalCrate`, unstable features are only available in nightly channel.\nWarning: can't set `ignore = IgnoreList { path_set: {\"/vendor/\", \"compiler/rustc_codegen_gcc/tests\", \"/tests/ui-fulldeps/\", \"compiler/rustc_codegen_gcc/example\", \"compiler/rustc_codegen_cranelift/scripts\", \"src/doc/rust-by-example\", \"src/tools/miri\", \"/tests/pretty/\", \"/tests/crashes/\", \"src/doc/reference\", \"src/tools/cargo\", \"library/portable-simd\", \"/*-build/\", \"/build-*/\", \"/tests/rustdoc-gui/\", \"src/doc/book\", \"src/tools/rust-analyzer\", \"/tests/incremental/\", \"src/tools/rustc-perf\", \"src/llvm-project\", \"/tests/ui/\", \"/tests/run-make/export\", \"compiler/rustc_codegen_gcc/src/intrinsic/archs.rs\", \"src/doc/nomicon\", \"src/tools/enzyme\", \"library/stdarch\", \"src/doc/embedded-book\", \"/build/\", \"/tests/codegen-llvm/simd-intrinsic/\", \"src/doc/edition-guide\", \"library/compiler-builtins\", \"src/tools/clippy\", \"/tests/rustdoc-html/\", \"src/gcc\", \"src/doc/rustc-dev-guide\", \"/tests/debuginfo/\", \"/tests/rustdoc-ui/\", \"library/backtrace\", \"src/tools/rustfmt\"}, rustfmt_toml_path: \"\" }`, unstable features are only available in nightly channel.\n"
   right: ""
  ---- print_config_merges_override_from_args stdout end ----
  
  failures:
      print_config_merges_multiple_overrides_from_args
      print_config_merges_override_from_args
  
  test result: FAILED. 17 passed; 2 failed; 1 ignored; 0 measured; 0 filtered out; finished in 154.58ms
  
  error: test failed, to rerun pass `--test rustfmt`
  Bootstrap failed while executing `test --stage 2 src/tools/rustfmt`
  Currently active steps:
  test::Rustfmt { compilers: RustcPrivateCompilers { build_compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target_compiler: Compiler { stage: 2, host: x86_64-unknown-linux-gnu, forced_compiler: false } } } at src/bootstrap/src/core/build_steps/test.rs:612
  Build completed unsuccessfully in 0:01:40
    local time: Mon Nov  9 22:34:34 UTC 2026
    network time: Mon, 09 Nov 2026 22:34:34 GMT

ytmimi · 2026-11-10T04:02:30Z

For a while I couldn't get r-l/r's CI issue to reproduce even when running tests with CFG_RELEASE_CHANNEL=stabel or CFG_RELEASE_CHANNEL=beta.

I wanted to understand this a little more so I did some digging. I think this is what's going on:

As currently written, the tests pass . as the path, and that'll get canonicalize to the root of the rustfmt repo when we run tests locally or in CI (maybe in r-l/r CI too?).

rustfmt/tests/rustfmt/main.rs

Lines 138 to 179 in 85f6ad5

    
           #[test] 
        
           fn print_config_merges_override_from_args() { 
        
               let args = &[ 
        
                   "--print-config", 
        
                   "current", 
        
                   ".", 
        
                   "--config=color=Never,edition=2018", 
        
               ]; 
        
               let (stdout, stderr, exit_code) = rustfmt(args); 
        
               assert_eq!(stderr, ""); 
        
               assert!( 
        
                   stdout.contains(r#"color = "Never""#) && stdout.contains(r#"edition = "2018""#), 
        
                   "expected overrides in stdout, got: {stdout}", 
        
               ); 
        
               assert_eq!(exit_code, 0); 
        
           } 
        
           #[test] 
        
           fn print_config_merges_multiple_overrides_from_args() { 
        
               let args = &[ 
        
                   "--print-config", 
        
                   "current", 
        
                   ".", 
        
                   "--config", 
        
                   "color=never,edition=2018", 
        
                   "--config", 
        
                   "color=always,format_strings=true", 
        
               ]; 
        
               let (stdout, stderr, exit_code) = rustfmt(args); 
        
               assert_eq!(stderr, ""); 
        
               assert!( 
        
                   stdout.contains(r#"color = "Always""#) 
        
                       && stdout.contains(r#"edition = "2018""#) 
        
                       && stdout.contains("format_strings = true"), 
        
                   "expected overrides in stdout, got: {stdout}", 
        
               ); 
        
               assert_eq!(exit_code, 0); 
        
           }

The Operation::ConfigOutputCurrent code path expects a file, not a directory, and unconditionally passes the parent to load_config, but . is already a directory so in our CI the repo's parent dir is passed to load_config. Since there's no rustfmt.toml in any ancestors the default config gets loaded, but in r-l/r's CI we end up loading the rustfmt.toml at the root of the r-l/r repo.

We'd actually run into the same issue that we saw in the r-l/r CI for the stable and beta channels since rustfmt's rustfmt.toml uses overflow_delimited_expr, error_on_line_overflow, and error_on_unformatted, which are all unstable.

rustfmt/src/bin/main.rs

Lines 284 to 297 in 85f6ad5

    
           Operation::ConfigOutputCurrent { path } => { 
        
               let path = match path { 
        
                   Some(path) => path, 
        
                   None => return Err(format_err!("PATH required for `--print-config current`")), 
        
               }; 
        
               let file = PathBuf::from(path); 
        
               let file = file.canonicalize().unwrap_or(file); 
        
               let (config, _) = load_config(Some(file.parent().unwrap()), Some(options))?; 
        
               let toml = config.all_options().to_toml()?; 
        
               io::stdout().write_all(toml.as_bytes())?; 
        
               Ok(0)

ytmimi · 2026-11-10T04:16:19Z

Created a follow up issue: #7241

ytmimi · 2026-11-10T04:33:07Z

@jieyouxu do you want to take another look at this? I've completely reworked it and now #[nightly_only_test] is required.

jieyouxu

Looks good, thanks

View changes since this review

…st] annotation These tests were changed to use `file!()` instead of `.`. Using `file!()` will ensure that rustfmt's `rustfmt.toml` is loaded even when running in r-l/r's CI. As a result, we now need to use `#[nightly_only_test]` since unstable option's are set in rustfmt's `rustfmt.toml`. Although this doesn't fully address the `--print-config current` issue, preventing the wrong `rustfmt.toml` from loading should help us prevent another case like rust-lang/rust 165713.

rustbot added the S-waiting-on-review Status: awaiting review from the assignee but also interested parties. label Nov 10, 2026

rustbot assigned jieyouxu Nov 10, 2026

ytmimi mentioned this pull request Nov 10, 2026

[beta] prepare Rust 1.101.0-beta rust-lang/rust#165713

Merged

jieyouxu reviewed Nov 10, 2026

View reviewed changes

jieyouxu approved these changes Nov 10, 2026

View reviewed changes

ytmimi mentioned this pull request Nov 10, 2026

Both --print-config default and --print-config current expect file paths, but accept directory paths #7241

Open

ytmimi force-pushed the prevent_tests_from_running_in_rust_lang_rust_ci branch from c2a8315 to 80e20de Compare November 10, 2026 04:29

jieyouxu approved these changes Nov 10, 2026

View reviewed changes

ytmimi force-pushed the prevent_tests_from_running_in_rust_lang_rust_ci branch from 80e20de to bb918c2 Compare November 10, 2026 05:01

ytmimi changed the title ~~fix: prevent some tests from running in r-l/r CI~~ test: rework --print-config current tests and add #[nightly_only_test] annotation Nov 10, 2026

ytmimi changed the title ~~test: rework --print-config current tests and add #[nightly_only_test] annotation~~ test: rework --print-config current tests and add #[nightly_only_test] annotation Nov 10, 2026

ytmimi added this pull request to the merge queue Nov 10, 2026

Merged via the queue into main with commit 0acb472 Nov 10, 2026
33 checks passed

rustbot added release-notes Needs an associated changelog entry and removed S-waiting-on-review Status: awaiting review from the assignee but also interested parties. labels Nov 10, 2026

jieyouxu deleted the prevent_tests_from_running_in_rust_lang_rust_ci branch November 12, 2026 13:29

Navigation Menu

Repository navigation

test: rework `--print-config current` tests and add `#[nightly_only_test]` annotation - #7240

test: rework `--print-config current` tests and add `#[nightly_only_test]` annotation#7240
ytmimi merged 1 commit into
mainfrom
prevent_tests_from_running_in_rust_lang_rust_ci

ytmimi commented Nov 10, 2026 •

edited

Loading

Uh oh!

jieyouxu Nov 10, 2026 •

edited by rustbot

Loading

Uh oh!

ytmimi Nov 10, 2026

Uh oh!

jieyouxu Nov 10, 2026

Uh oh!

cuviper Nov 10, 2026 •

edited

Loading

Uh oh!

jieyouxu Nov 10, 2026 •

edited

Loading

Uh oh!

jieyouxu left a comment •

edited by rustbot

Loading

Uh oh!

ytmimi commented Nov 10, 2026

Uh oh!

ytmimi commented Nov 10, 2026

Uh oh!

ytmimi commented Nov 10, 2026

Uh oh!

ytmimi commented Nov 10, 2026

Uh oh!

jieyouxu left a comment •

edited by rustbot

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	pub fn nightly_only_test(_args: TokenStream, input: TokenStream) -> TokenStream {
	// if CFG_RELEASE_CHANNEL is not set we default to nightly, hence why the default is true
	if option_env!("CFG_RELEASE_CHANNEL").map_or(true, \|c\| c == "nightly" \|\| c == "dev") {
	input

Conversation

ytmimi commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jieyouxu Nov 10, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ytmimi Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

jieyouxu Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

cuviper Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ytmimi commented Nov 10, 2026

Uh oh!

ytmimi commented Nov 10, 2026

Uh oh!

ytmimi commented Nov 10, 2026

Uh oh!

ytmimi commented Nov 10, 2026

Uh oh!

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ytmimi commented Nov 10, 2026 •

edited

Loading

jieyouxu Nov 10, 2026 •

edited by rustbot

Loading

cuviper Nov 10, 2026 •

edited

Loading

jieyouxu Nov 10, 2026 •

edited

Loading

jieyouxu left a comment •

edited by rustbot

Loading

jieyouxu left a comment •

edited by rustbot

Loading