Import the TLV libraries from mutask with history by mfield4 · Pull Request #327 · google/libhoth

mfield4 · 2026-11-09T19:43:27Z

Part 2 of 3 (#324, #327, #328).

This PR imports the TLV and protobuf libraries from mutask with their commit history intact, gets them building in libhoth, and moves them to shared/. They stay no_std since shared/ is used by both firmware and host tools.

Please merge with "Rebase and merge" (not squash) to keep the imported commit history. Intermediate commits won't build on their own, only the PR head.

What to review

Commits 1-31 are imported history from mutask with original authors, dates, and commit messages preserved. They were already reviewed in mutask, so there's no need to review them line by line.

Commits 32-36 are new in libhoth:

crypto: Decouple sha256_sw from mutask: replaces mutask_subtle::as_mut_bytes() with a small const helper (the only logic change).
tlv: Build in libhoth: updates one dependency label in golden/BUILD.bazel; the rest of the diff is MODULE.bazel.lock.
Rename mutask crates and proto packages to openprot: renames mutask_* / mutask.* to openprot_* / openprot.*, mostly in regenerated *_tlv.rs files.
Format the imported protos with clang-format: whitespace and wrapping only; generated TLV code doesn't change.
Move TLV libraries to shared/: moves the directories and updates Bazel labels, import paths, and descriptor names.

History import

The imported commits come from mutask 3a5aa8d, filtered to contrib/{proto,aligned,tlv,openprot,crypto,fingerprints,golden} with empty commits dropped. Two commits (Add SHAKE256 software lib and contrib/crypto: Add mldsa_sw Rust wrapper for ML-DSA-44) were left out because they depend on a vendored PQC library being reviewed separately. The filter script also scrubs a few internal names from files and four commit messages, strips Change-Id: and Bug: footers, and adds Apache-2.0 headers to .rs, .proto, .bzl, and BUILD.bazel files (including in tlv/codegen.rs, so checked-in generated code still matches the generator at every commit).

Googlers with mutask access can reproduce the import and check that commits 1-31 match byte-for-byte (files, authors, dates, and messages) using contrib/libhoth_graft/ in mutask:

contrib/libhoth_graft/graft.py --out /tmp/libhoth-graft
cd /tmp/libhoth-graft
git fetch https://github.com/google/libhoth host-move:host-move mutask-graft:mutask-graft
<mutask>/contrib/libhoth_graft/verify.py host-move..mutask-graft
# expect: 31 imported (graft #1..#31 of 37), 0 mismatched, 5 not imported
<mutask>/contrib/libhoth_graft/audit.py host-move..mutask-graft

korran · 2026-11-12T22:21:12Z

@@ -32,6 +30,13 @@ static K: [u32; 64] = [
 const BLOCK_SIZE_BYTES: usize = 64;
 const BLOCK_SIZE_WORDS: usize = BLOCK_SIZE_BYTES / 4;

+/// Sets byte `i` of `block`, viewed as native-endian bytes.
+const fn set_block_byte(block: &mut [u32; BLOCK_SIZE_WORDS], i: usize, byte: u8) {


Remove this function. You can accomplish the same thing by using transmut_mut!():

let block_bytes: &mut [u8; BLOCK_SIZE_BYTES] = transmute_mut!(&mut self.block); block_bytes[5] = 42;

korran · 2026-11-12T22:40:21Z

+    }
+
+    let mut out = String::new();
+    out.push_str("// Copyright 2026 Google LLC\n");


I don't think we should hard-code this. Can we add a attr to the build rule (and flag to the binary), something like file_prefix=":apache2_header.txt"

This reorders the entire hash to be little-endian.

The old libraries weren't taking advantage of the crate_name attribute.

Unfortunately, crate_universe repo names are global, so this change makes collisions less likely.

This makes it easier for other repos to reference the exact same crates as mutask.

This provides a macro for comparing arbitrary text data to a golden file, and allowing a person running a test to automatically set the golden data to the "actual" data by setting a flag when running the unit test.

include_bytes!() is relative to the current source file, not the crate root, so use that when computing the full path with UPDATE_GOLDEN=1.

This can be used to assert golden-data for the generated rust code for a distro.

These clippy warnings are introduced in a new version of the compiler.

Encode/decode TLV structs to/from byte buffers

Will be used for proto -> TLV encoding

Adds: - TLV Code generator: Generate TLV struct from proto files - TLV Converter: Encode TLV annotated proto to TLV buffer

Mostly generated by Gemini from the implementation, but I checked it for correctness.

contrib/tlv depends on U64Aligned4 to optimize load/stores in generated code. This adds an unnecessary dependency on mutask types to contrib/tlv. Move the type to it's own contrib/aligned crate. Side benefit of having a nice spot to add other aligned types if it comes up.

TlvItem::header and TlvItem::payload are moved into a new DST type, TlvContainer. This makes it possible to get a reference to all the bytes of a TLV item with `item.container.as_bytes()`, which is useful when verifying signatures.

TLV codegen currently naively regenerates rust bindings for all imported proto dependencies, causing code duplication. Add a deps field to bazel rule that will import tlv crates in generated rust code.

Tag TLV update targets with "tlv_update" and query them in fix.sh to automatically regenerate checked-in TLV structs.

This was causing `ci.sh` to fail on my machine.

Add new trait + impls to calculate Sha256 digest on public key materials.

With opt=s, LLVM often makes poor inlining decisions that can double the size of the generated machine code. Force it to always inline these simple functions.

Also, improve the rustdoc for this type.

When tlv_codegen is invoked from an external repository (such as a downstream firmware repo), plain string labels like "//contrib/aligned" resolve relative to the caller's repository rather than @mutask. Wrap them in Label() so they always resolve relative to @mutask.

Define ChipFamily enum and ChipId TLV message (tag "C_ID") in contrib/openprot/proto/chipid.proto.

sha256_sw used mutask_subtle::as_mut_bytes() to write single bytes into the u32 block from const fns. Write them through a small const helper instead, so the crate only depends on zerocopy and stays no_std.

- golden: take argh from //third_party instead of @crates_mutask. - MODULE.bazel.lock: record protobuf's crate resolution, now that Rust protobuf is built.

The TLV libraries now live outside mutask. Rename the mutask_* crates to openprot_* and the mutask.* proto packages to openprot.*, and regenerate the checked-in TLV code.

libhoth's CI checks .proto files with clang-format. These are whitespace and wrapping changes only, and the generated TLV code is unchanged.

shared/ holds code that is shared between host tools and firmware. Update the Bazel labels, proto import paths and descriptor names that depend on the location, and regenerate the checked-in TLV code.

mfield4 mentioned this pull request Nov 9, 2026

Import tlvtool from mutask with history #328

Open

mfield4 force-pushed the mutask-graft branch from 34616bf to 47d3113 Compare November 9, 2026 19:59

mfield4 changed the base branch from main to host-move November 9, 2026 20:09

mfield4 added this pull request to stack #329 November 9, 2026 20:09

mfield4 force-pushed the mutask-graft branch from 47d3113 to 22af3be Compare November 10, 2026 00:12

mfield4 removed this pull request from stack #329 November 10, 2026 00:13

mfield4 added this pull request to stack #332 November 10, 2026 00:14

mfield4 removed this pull request from stack #332 November 10, 2026 00:35

mfield4 added this pull request to stack #333 November 10, 2026 00:37

mfield4 changed the title ~~Mutask graft~~ Import the TLV libraries from mutask with history Nov 10, 2026

mfield4 marked this pull request as ready for review November 10, 2026 00:42

mfield4 requested a review from korran November 10, 2026 00:52

mfield4 force-pushed the mutask-graft branch 2 times, most recently from 0be927a to fcea675 Compare November 12, 2026 00:59

korran reviewed Nov 12, 2026

View reviewed changes

mfield4 force-pushed the mutask-graft branch from fcea675 to 0fe3be3 Compare November 13, 2026 19:05

Base automatically changed from host-move to main November 13, 2026 19:28

mfield4 force-pushed the mutask-graft branch from 0fe3be3 to 403f84f Compare November 13, 2026 19:28

korran added 11 commits November 13, 2026 11:45

contrib: Add const sha256 implementation.

94497b0

sha256: Remove unsafe.

6a254a1

Make sha256 no_std :)

22fb608

sha256_sw: Replace finalize_words() with finalize_words_le().

388310e

This reorders the entire hash to be little-endian.

Make rust_lib() names consistent.

c8664e5

The old libraries weren't taking advantage of the crate_name attribute.

Rename @crates repo to @crates_mutask.

e21d267

Unfortunately, crate_universe repo names are global, so this change makes collisions less likely.

Add aliases to third-party crates under third_party/.

0ecf042

This makes it easier for other repos to reference the exact same crates as mutask.

Add "golden" library.

0f491a6

This provides a macro for comparing arbitrary text data to a golden file, and allowing a person running a test to automatically set the golden data to the "actual" data by setting a flag when running the unit test.

golden: Support source files in subdirs.

d5d10d9

include_bytes!() is relative to the current source file, not the crate root, so use that when computing the full path with UPDATE_GOLDEN=1.

Add edition = "2024" to all crates.

a7cfc30

Add mutask_codegen_test() rule.

b7ae04d

This can be used to assert golden-data for the generated rust code for a distro.

korran and others added 25 commits November 13, 2026 11:46

Fix new clippy warnings.

b2f5be7

These clippy warnings are introduced in a new version of the compiler.

TLV: Add library

1b7d936

Encode/decode TLV structs to/from byte buffers

Protobuf reader and reflection lib

7712c8f

Will be used for proto -> TLV encoding

TLV: Protobuf to TLV

df0d584

Adds: - TLV Code generator: Generate TLV struct from proto files - TLV Converter: Encode TLV annotated proto to TLV buffer

TLV codegen: Use #[no_std]

a0ea900

Document the TLV format.

7ca58bc

Mostly generated by Gemini from the implementation, but I checked it for correctness.

TLV codegen: Add extra_data support

2752d7c

TLV: Introduce TlvContainer type.

7db8d7c

TlvItem::header and TlvItem::payload are moved into a new DST type, TlvContainer. This makes it possible to get a reference to all the bytes of a TLV item with `item.container.as_bytes()`, which is useful when verifying signatures.

Make sha256() function usable at compile time.

95aabc6

Add deps to TLV codegen

f4a7e21

TLV codegen currently naively regenerates rust bindings for all imported proto dependencies, causing code duplication. Add a deps field to bazel rule that will import tlv crates in generated rust code.

fix.sh: Add TLV regeneration

7c56955

Tag TLV update targets with "tlv_update" and query them in fix.sh to automatically regenerate checked-in TLV structs.

Crypto: Add TLV proto and generated structs

9761195

KeyRotation: Add TLV proto and generated structs

2e859db

bazel: fix deprecated $(location) warning

42e612c

This was causing `ci.sh` to fail on my machine.

Key Rotation : Add fingerprint calculation for crypto types

929fb8d

Add new trait + impls to calculate Sha256 digest on public key materials.

tlv: Add inline(always) strategically.

6e6f3f3

With opt=s, LLVM often makes poor inlining decisions that can double the size of the generated machine code. Force it to always inline these simple functions.

Add add_copy() function to TlvBuilder.

14452c0

Also, improve the rustdoc for this type.

openprot/proto: Add chipid.proto and generated chipid_tlv.rs

e8ccd5b

Define ChipFamily enum and ChipId TLV message (tag "C_ID") in contrib/openprot/proto/chipid.proto.

crypto: Decouple sha256_sw from mutask

fdd2578

sha256_sw used mutask_subtle::as_mut_bytes() to write single bytes into the u32 block from const fns. Write them through a small const helper instead, so the crate only depends on zerocopy and stays no_std.

tlv: Build in libhoth

0618a88

- golden: take argh from //third_party instead of @crates_mutask. - MODULE.bazel.lock: record protobuf's crate resolution, now that Rust protobuf is built.

Rename mutask crates and proto packages to openprot

bdf2c21

The TLV libraries now live outside mutask. Rename the mutask_* crates to openprot_* and the mutask.* proto packages to openprot.*, and regenerate the checked-in TLV code.

Format the imported protos with clang-format

9bcd16c

libhoth's CI checks .proto files with clang-format. These are whitespace and wrapping changes only, and the generated TLV code is unchanged.

Move TLV libraries to shared/

67e89dc

shared/ holds code that is shared between host tools and firmware. Update the Bazel labels, proto import paths and descriptor names that depend on the location, and regenerate the checked-in TLV code.

mfield4 force-pushed the mutask-graft branch from 403f84f to 67e89dc Compare November 13, 2026 19:46

Navigation Menu

Repository navigation

Import the TLV libraries from mutask with history - #327

Import the TLV libraries from mutask with history#327
mfield4 wants to merge 36 commits into
mainfrom
mutask-graft

mfield4 commented Nov 9, 2026 •

edited

Loading

Uh oh!

korran Nov 12, 2026

Uh oh!

korran Nov 12, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

mfield4 commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What to review

History import

Uh oh!

korran Nov 12, 2026

Choose a reason for hiding this comment

Uh oh!

korran Nov 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mfield4 commented Nov 9, 2026 •

edited

Loading