Skip to content

Spec: Add HLL sketch blob type to Puffin spec - #18513

Open
ebyhr wants to merge 1 commit into
apache:mainfrom
ebyhr:ebi/spec-hll
Open

ebyhr wants to merge 1 commit into
apache:mainfrom
ebyhr:ebi/spec-hll

Conversation

@ebyhr

@ebyhr ebyhr commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Theta sketch computation for NDV column statistics is slow in some workloads, causing a 10-20% insert slowdown in Trino.

That PR adds an HLL option but has to store it as an engine-specific blob type, because the Puffin spec has no standard HLL type. This adds apache-datasketches-hll-v1, a compact Apache DataSketches HLL sketch with an ndv property, so engines can share HLL-based NDV statistics.

@github-actions github-actions Bot added the Specification Issues that may introduce spec changes. label Oct 15, 2026
@deniskuzZ

Copy link
Copy Markdown
Member

sorry for bringing this here, but maybe someone could help with similar request: #8202

@pvary

pvary commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Spec changes need to go through the dev list.
Maybe it would be useful to raise the question on the Community Sync as well.
If the implementation is easy maybe even create the 2 PRs: spec change and java impl.

@simhadri-g

Copy link
Copy Markdown
Member

sorry for bringing this here, but maybe someone could help with similar request: #8202

I agree, adding kll in the puffin spec would be helpful.

Can someone please helpout with this request ?

@zhjwpku zhjwpku left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Nov 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Specification Issues that may introduce spec changes. stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants