Skip to content

eval-under-loop: options for image allocation, direct I/O, mkfs and mount - #17

Open
yarikoptic-gitmate wants to merge 3 commits into
masterfrom
claude/blissful-allen-xi3hhk
Open

yarikoptic-gitmate wants to merge 3 commits into
masterfrom
claude/blissful-allen-xi3hhk

Conversation

@yarikoptic-gitmate

Copy link
Copy Markdown

Makes the loop backend usable for large (up to TB) images, and adds hooks for tuning the filesystem under test. Today the image is always dd'd from /dev/zero, which means writing every byte up front.

New options (each with an EVAL_UNDER_LOOP_* env var)

Option What it does
--alloc zero|sparse|fallocate zero is the default and keeps the old dd behaviour, so existing CI cells are unchanged. sparse uses truncate: instant, allocates nothing. fallocate calls fallocate(2) directly: it reserves the blocks without writing them. Where the host can't do it (NFS < 4.2, ramfs) it fails, instead of glibc's posix_fallocate quietly writing every block.
--direct-io Attaches with losetup --direct-io=on, so image I/O isn't cached twice. If the host fs lacks O_DIRECT, current kernels refuse to attach; the backend now says that is why. Older kernels attach buffered instead, and the backend warns.
--mkfs-opts "..." Extra mkfs.<fs> arguments, split on whitespace (e.g. btrfs -m single -K). There is no quoting.
--mount-opts a,b Extra mount -o options. For vfat-like filesystems they are appended after uid=/gid=.

Also:

  • The log shows the image's apparent vs allocated size, so you can tell a sparse image from a full one.
  • --size is now checked to be a positive integer.
  • With ro, the post-mount chown no longer aborts the run; it warns instead.

Docs: GOTCHAS.md's loop settings table now lists the allocation and loop-I/O settings, plus the btrfs minimum size of 109 MiB. The README has a large sparse btrfs example.

Behaviour for existing callers

With no new options, the run does the same thing as before. mount -o defaults is equivalent to a bare mount; a real ext4 run still gives rw,relatime. The only new output is one I: image on host: … log line. bin/ci/run-under.sh passes none of the new options.

Review and testing

  • master is merged in, so the branch is up to date with master's NFS --mount-opts/--export-opts. That merge had no conflicts.
  • Two independent reviewers, one test-driving and one reading the code, went through the first version. Their findings are fixed in the second commit:
    • the direct-I/O and fallocate claims in the docs were wrong (tmpfs has had O_DIRECT since Linux 6.6, and vfat implements fallocate by zeroing in the kernel);
    • ro mounts broke the post-mount chown;
    • newlines in --mkfs-opts silently dropped the rest of the string;
    • --size wasn't validated;
    • some help text still said the image is always dd'd;
    • the new options had no parse tests.
  • Run for real as root on an ext4 host:
    • all three --alloc modes;
    • --direct-io (DIO=1), and on a ramfs host (clear error, clean teardown);
    • --mkfs-opts and --mount-opts, including invalid values, ro, and an embedded newline.
    • Teardown left no loop devices, mounts or image files behind in every case, failures included.
  • bin/ci/shellcheck.sh is clean. bats tests/ passes 34/34, including 4 new unprivileged loop tests.
  • Not tested: btrfs, vfat, xfs and NFS could not be mounted in the test container (its kernel lacks them). mkfs.btrfs itself was run: it rejects 100 MB (minimum is 114294784 bytes) and accepts 120 MB.

Not in this PR (follow-ups)

  • A Loop btrfs matrix row. The git-annex target's loop-size-mb: 100 is below btrfs's minimum and would need raising.
  • Nesting backends, e.g. eval-under nfs --no-root-squash -- eval-under loop --fs btrfs … to put the image on NFS. This already composes through TMPDIR, but it needs --no-root-squash, and turning it into a matrix row would need an over: field in evals/matrix.yaml.

🤖 Generated with Claude Code

https://claude.ai/code/session_0135xivQQnx5bWMzWD91J5TR


Generated by Claude Code

…ount

--alloc zero|sparse|fallocate picks how the backing image gets its
size. zero (dd from /dev/zero) stays the default so CI cells keep
measuring the same thing; sparse (truncate) and fallocate are instant,
which is what makes TB-sized images practical. fallocate uses
fallocate(2) directly, so a host without it (NFS < 4.2) fails loudly
instead of glibc's posix_fallocate silently writing every block.

--direct-io attaches with losetup --direct-io=on, so image I/O is not
cached twice, and warns if the kernel kept the device buffered.
--mkfs-opts / --mount-opts pass extra arguments through (e.g. btrfs
-m single, discard=async). The image's apparent vs allocated size is
logged, so a sparse image is visible as such.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0135xivQQnx5bWMzWD91J5TR
- --direct-io on a host fs without O_DIRECT: current kernels refuse to
  attach rather than fall back, so say why losetup failed instead of
  exiting with only its EINVAL. Help/GOTCHAS no longer claim tmpfs
  lacks O_DIRECT (it has it since 6.6; ramfs is the example now).
- --alloc fallocate: vfat implements it by zeroing in the kernel, so it
  is not an example of failing; list NFS < 4.2, ramfs, ext4 w/o extents.
- --mount-opts ro no longer aborts on the post-mount chown; warns.
- --mkfs-opts folds newlines into spaces (read -a stopped at the first)
  and documents that there is no quoting.
- --size is validated as a positive integer up front, so the three
  alloc modes cannot read a bad value three different ways.
- Help/README/header no longer say the image is always dd'd; README
  gets a large sparse btrfs example; default columns say (unset) like
  the nfs backend.
- bats: help/parse test for the new options (incl. value consumption),
  missing-value and --size validation tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0135xivQQnx5bWMzWD91J5TR

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants