Skip to content

fix(cli): read job ids given without a dependency type as afterany - #43

Merged
Yannick-Dayer merged 4 commits into
mainfrom
fix-repeat-dependency
Oct 1, 2026
Merged

Yannick-Dayer merged 4 commits into
mainfrom
fix-repeat-dependency

Conversation

@anjos

@anjos anjos commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Problem

sbatch reads a bare job id in --dependency as afterany, but it rejects a bare list of ids (5:6: 5 is not a dependency type). gridtk submit --repeat N extends the dependency of each new job by appending the previous local id, so it built such lists:

command 2nd job 3rd job
--repeat 3 <id1> <id1>:<id2> (rejected)
--dependency 5 --repeat 3 (reported by @Yannick-Dayer) 5:<id1> (rejected) 5:<id1>:<id2> (rejected)

Plain --dependency 5:6 was also passed through as is, and sbatch rejected it with an unclear error.

Fix

add_default_dep_type() in tools.py runs once, before submission. Every dependency spec that starts with a job id gets afterany: in front: 5 becomes afterany:5, and 5:6 becomes afterany:5:6. It keeps the , and ? separators, and leaves typed specs (afterok:5, singleton) unchanged. --repeat therefore always extends a spec that has a type. Without --dependency, the chain starts with afterany:, so a checkpoint-resuming chain keeps going when a job reaches its time limit.

The help text of --dependency and --repeat, the README and the CHANGELOG now describe the default.

Tests

  • Unit tests for add_default_dep_type(): bare, list, +time, typed, ,, ? and singleton specs.
  • test_submit_repeat_after_dependency: --dependency 1 --repeat 3 passes afterany:<g1>, then afterany:<g1>:<g2>, then afterany:<g1>:<g2>:<g3>. With afterok:1, the type is kept. The bare-id case fails on main.
  • The existing assertions in test_submit_with_dependencies now expect the explicit type.

Relation to #42

The code merges cleanly with #42, and the test suite passes with both applied. Both PRs add a ### Bug Fixes section to CHANGELOG.md, so whichever merges second needs a one-line conflict fix there.

…s given

From the third job on, --repeat without --dependency passed
--dependency <id1>:<id2> to sbatch, which rejects it.  sbatch reads a bare
job id as afterany, so make that type explicit.
@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/gridtk
  cli.py
  tools.py
Project Total  

This report was generated by python-coverage-comment-action

@Yannick-Dayer

Copy link
Copy Markdown
Member

The issue is also present with the --dependency option. If the user passes gridtk submit --dependency 5 --repeat 3, sbatch will still get invalid --dependency values: 5:6, 5:6:7, and 5:6:7:8(translated to Slurm IDs), with an unclear error reported.

It might be enough to add a note in the CLI option's help message to prepend afterany: to --dependency when multiple IDs are passed or when --repeat is involved.

--dependency 5 --repeat 3 built 5:6, then 5:6:7, which sbatch rejects.
sbatch reads a bare job id as afterany, so give every dependency spec that
starts with a job id that type explicitly, before --repeat extends it.
@anjos anjos changed the title fix(cli): chain --repeat jobs with afterany when no dependency type is given fix(cli): read job ids given without a dependency type as afterany Sep 30, 2026
@anjos

anjos commented Sep 30, 2026

Copy link
Copy Markdown
Member Author

Thanks @Yannick-Dayer, good catch: --dependency 5 --repeat 3 did build 5:<id1>, which sbatch rejects. Rather than a note in the help text, the CLI now handles it (2f3dfba): job ids given without a dependency type get afterany:, which is what sbatch already does for a single id. So 5 becomes afterany:5 and 5:6 becomes afterany:5:6, and --repeat always extends a spec that has a type. Typed specs such as afterok:5 are unchanged. There's a CLI test for your exact case.

Yannick-Dayer
Yannick-Dayer previously approved these changes Oct 1, 2026

@Yannick-Dayer Yannick-Dayer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for fixing the earlier condition and adding the tests.

I have only a minor tweak suggestion to the README.

Comment thread README.md Outdated
Co-authored-by: Yannick Dayer <60428834+Yannick-Dayer@users.noreply.github.com>
@anjos

anjos commented Oct 1, 2026

Copy link
Copy Markdown
Member Author

I applied the suggested changes! Thanks.

@Yannick-Dayer Yannick-Dayer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍🏻

@Yannick-Dayer
Yannick-Dayer merged commit b65787e into main Oct 1, 2026
10 checks passed
@Yannick-Dayer
Yannick-Dayer deleted the fix-repeat-dependency branch October 1, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants