Repository navigation
+ Reiser4 multi-level trees and metadata-preserving rebuilds, verified by reiser4progs - #445
Merged
Merged
Conversation
Wide namespaces outgrew the single-leaf writer, and rebuilds discarded native object metadata and directory structure. Pack bounded leaves and twig extents beneath as many pointer levels as needed, traverse the complete native tree, and carry object identity, Unix fields and opaque stat-data tails through staged edits, defrag, shrink and rebuild. Reserve every tree node during wipe and cache traversal for large streaming rebuilds. Sourcing rung 2: no licence-compatible managed Reiser4 implementation was identified. Derived node40/stat40/cde40/extent40 layout and synthetic tests are recorded in docs/REISER4-NATIVE-PROFILE.md; GPL reiser4progs 1.2.2 is an external oracle, not copied or converted source. Validation: Reiser4-focused run passed 95 tests with one obsolete sentinel skipped. fsck.reiser4 accepted nested 1-, 100- and 5000-file images; external mutation tests check Unix metadata changes, add/remove and defrag.
…gin profile
Every Reiser4 edit, defrag, shrink and relayout gave the volume a new mkfs id. A
volume made by mkfs.reiser4 with a non-default profile (-o hash=tea_hash,
formatting=tails, fibration=lexic_fibre) came out of any edit failing
fsck.reiser4 ("The Plugin Set slot N of the root directory ... does not match the
backed up value"), and with the tea hash a long name the edit added could not be
found by debugfs.reiser4.
The rebuild writes the captured mkfs prefix again with only UUID and label patched
in: the mkfs id was drawn at random, the format40 tail policy and the backup
record's copy of the root plugin set came from the default template, while the root
stat data (with the source's real plugin set) was carried over, and names were keyed
with r5 regardless of the volume's hash.
The mkfs id is now carried like UUID and label. Before staging anything, the
source's fixed blocks are compared with what the writer would write back: master,
format40 (magic, tail policy, flags, version), the journal header and footer
(must be blank - a kernel journal is not replayed), the status block (must say
consistent) and the backup record outside its identity fields. Any difference is
NotSupportedException with the image untouched; such volumes are still read.
Tests: the mkfs id survives add, remove and defrag; a patched tail policy, backup
hash slot, status block or journal header is refused by add, remove, defrag and
relayout with the image byte-identical, and shrink copies it through; the tree
grows 2 -> 3 levels past one twig of extents and shrinks back on removal; the
last file of a directory leaves an empty directory; names of 23, 24 and 255
characters round-trip. Reiser4ProfileExternalTests (WSL-gated) checks against
reiser4progs 1.2.2: an mkfs.reiser4 volume after adding nested files, long names
and an empty directory is consistent with UUID, label, mkfs id, policies and root
stat unchanged; the three non-default profiles are refused byte-identical; 12,000
files make a consistent height-4 tree; a tree fsck --build-fs rewrote (lost+found,
object 0xffff, uid 1000) is read and edited keeping its key and owner; a large-time
stat extension survives add and defrag (sdext_lt still shown). File bytes are taken
from the blocks debugfs.reiser4 -i names, since debugfs -k prints file contents
through printf.
Sourcing rung 2: reiser4progs (GPL) as black-box oracle only; the fixed-block
fields were located by diffing mkfs.reiser4 output across profiles.
UpdateMetadata wrote whatever link count and stat-data bytes it was handed. Extension bytes the mask does not declare make the stat data a fatal corruption to fsck.reiser4 (stat40_check_struct), yet passed our own verification, which compares only what it wrote; a link count other than the namespace's is silently wrong (fsck --check does not flag a regular file's nlink, so nothing catches it). The link count follows the namespace (one per name, a directory one more per sub-directory) and the rebuild already recomputes it for directories; extension bytes past the Unix fields are carried as found. UpdateMetadata now changes only mode, UID, GID and the three timestamps, and refuses a different link count, mask or extension tail with NotSupportedException, the volume unchanged. Sourcing rung 2: stat40 layout as in docs/REISER4-NATIVE-PROFILE.md; reiser4progs used only as an oracle.
The writer carried any stat-data bytes a rebuild or caller handed it. A body with four bytes its mask does not declare - the shape the round-trip test used to prove "opaque extension retention" - is, to fsck.reiser4, a fatal corruption (stat40_check_struct, "Fatal corruptions were found"), while our reader, which reads only the first 44 bytes, accepted it back. Extensions follow the 16-bit mask in bit order, each with a length fixed by its plugin or by a leading slot count. The writer now walks the mask before using a carried body: light-weight, Unix, large times, flags, plugin set and heir set must exactly fill the body; any other extension (symlink, crypto, a chained mask) or a leftover byte is NotSupportedException. A rebuild of a volume carrying such a stat data is therefore refused before its source is touched. The round-trip test now carries a real large-time extension instead of junk; new tests refuse trailing bytes, the symlink, crypto and chained-mask bits, and a volume whose stat data declares a large-time extension it does not hold (image byte-identical). Sourcing rung 2: extension ids and sizes (sdext_lw/unix/lt/flags/plug) are constants read from reiser4progs 1.2.2 headers; fsck.reiser4 was the oracle for the corruption verdict.
…eiser4progs evidence The package README row still described the root-only single-leaf profile, the bullet claimed "opaque stat-data extensions" survive rebuilds (a body with undeclared bytes is a fatal corruption to fsck.reiser4) and linked the profile notes relatively, which the package-readme generator rejects because nuget.org renders package READMEs. The README row and bullet and docs/REISER4-NATIVE-PROFILE.md now say what was verified: trees up to height 4, nested and empty directories, identity (UUID, label, mkfs id, object IDs, declared stat-data extensions) kept through edits, defrag in place only for a two-node tree, refusal of other plugin profiles, non-blank journals, unclean status and unparsable stat data, and no kernel mount (the WSL kernel has no Reiser4 driver). The profile notes carry the oracle table. REFERENCE.md regenerated.
Hawkynt
force-pushed
the
feat/reiser4-multileaf-metadata
branch
from
October 9, 2026 09:46
450bcca to
fc185b6
Compare
Hawkynt
changed the title
Hawkynt
marked this pull request as ready for review
October 9, 2026 09:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #409. Reiser4 writing was limited to one leaf and a flat namespace, and rebuilds lost object metadata. This writes and reads multi-level node40 trees (leaves, twigs with the extents, as many internal levels as needed), keeps nested and empty directories, and carries volume identity and object metadata through every staged edit, defrag, shrink and relayout.
Changes
UpdateMetadatastages typed changes. Every native tree node is reserved during wipe.mkfs.reiser4 -o hash=tea_hash/formatting=tails/fibration=lexic_fibrevolume wrote the default template's tail policy and plugin-set backup over a root that kept the real plugin set. fsck reportedPlugin Set slot N ... does not match the backed up value, and with tea_hash a long name could not be looked up. A rebuild is now refused (NotSupportedException, image byte-identical) when the master, format40, journal (must be blank), status (must be consistent) or backup block differs from what the writer writes back. Such volumes are still read.UpdateMetadatachanges only mode, UID, GID and times. It refuses link-count and extension changes.docs/REISER4-NATIVE-PROFILE.md(profile, refusal table, evidence table), XML summaries, REFERENCE regenerated.Oracle evidence (reiser4progs 1.2.2, WSL)
The WSL kernel 6.6.87.2 has no Reiser4 driver, so nothing was mounted. Volumes written by the tools themselves come from
mkfs.reiser4andfsck.reiser4 --build-fs. File bytes were taken from the blocksdebugfs.reiser4 -inames, becausedebugfs -kpasses file contents toprintfas the format string and is not a byte oracle.fsck --checkconsistent; height 3, 38 twigs / 113 leaves, 3,040 objects; 316 files byte-identical via-i+ block copydebugfs -i/-kbefore and afterfsck --build-fsadded lost+found (oid 0xffff, uid 1000)mkfs.reiser4volume + nested files, long names, empty dirmkfs.reiser4with tea_hash / tails / lexic_fibresdext_ltstill shownReiser4ProfileExternalTests(new) andReiser4MutationExternalTestsreplay these checks wherever reiser4progs is installed (WSL-gated, advisory tier). Local run: 167 Reiser4 tests, plus the platform-compression guard and maintenance-capability tests, passed. 2 skipped: the Linux-only native debugfs test and the obsolete sentinel.Limits
These are refused, not guessed: 4096-byte format40/node40 with large keys and the mkfs default plugin profile only; ASCII names; tail/ctail bodies, xattr items, symlinks, sparse or shared files, and other stat-data extensions. Defrag moves blocks in place only for a two-node tree. Larger trees go through a verified staged rebuild.