[zhangyue19921010]

Problem

Java’s no-argument Dataset.describeIndices() fails when a __lance_frag_reuse system index is present because it attempts to resolve a scalar index plugin, causing Spark SHOW INDEXES to fail.

Solution

Return {} for system index details in Rust while preserving their names, types, and metadata. Add regression coverage for frag-reuse, MemWAL, and Java’s index description APIs.

Result

describeIndices() and describeIndexSummaries() work after compaction with defer_index_remap=true, while regular index behavior remains unchanged.

Validation

Merged current main (cbed18eff) into the branch with no conflicts. Local validation on the merge commit:

• cargo fmt --all and cargo fmt --all -- --check
• cargo clippy --profile ci --locked --all --tests --benches -- -D warnings
• Focused lance tests: remapper not created without remappable indices, MemWAL system-index description, unknown scalar plugin still errors, and mixed-layout segment descriptions (3 cases)
• ./mvnw test -Dtest=DatasetTest#testDescribeIndicesAfterDeferredCompaction+testDescribeIndicesByName with a rebuilt JNI library (2 passed)
• ./mvnw spotless:check
• JNI cargo clippy --locked --tests --manifest-path ./lance-jni/Cargo.toml -- -D warnings

Limits: only the focused Rust and Java suites above were run; hosted CI is not awaited in this update. The JNI crate is a standalone workspace without the repository ci Cargo profile, so its lint ran in the dev profile.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[github-actions]

Important

Format specification vote

This PR modifies the Lance format specification, so it requires 3 binding +1 votes from PMC members (excluding the proposer), at least one of them on the latest commit, and a minimum 72-hour voting period, weekends excluded, before it can merge. Vote by approving this PR (+1) or requesting changes (−1, a veto). See the voting process.

Approvals carry over across pushes, so a rebase or a typo fix does not send everyone back to re-vote. Whoever approves the latest commit is vouching that nothing substantive has changed since the earlier approvals; if something has, ask for fresh votes.

Status: ❌ Blocked — 0 of 3 required approvals

Approvals	none (0/3)
Latest commit approved by	none — one PMC member must approve the latest commit
Vetoes	none
Voting period	ends Thu 2026-11-19 00:00 UTC (Wed 16:00 PST)

_{Updated automatically by the format-spec vote gate, which re-checks every 15 minutes — just voted? Re-check now (press Run workflow; leave the input blank to re-check every open format PR). A PMC member may apply the format-waived label to waive the vote for a trivial edit (typo, wording, formatting).}

[lance-gatekeeper]

✅ Gate recommendation: approve.

The shared Rust fix preserves system-index identity and metadata without requiring scalar plugins. Focused Rust and Java regressions pass after the main merge, including both Java description APIs, ordinary-index details, and unknown-plugin errors.

[Xuanwo]

LGTM. Hosted checks on the accepted head are green and the candidate merge onto current main is clean.