Skip to content

Add TinybirdApi.sample_datasource() for bounded S3/GCS/DynamoDB sampling - #24

Open
sandshoes wants to merge 1 commit into
mainfrom
tommy/sample-datasource-api
Open

sandshoes wants to merge 1 commit into
mainfrom
tommy/sample-datasource-api

Conversation

@sandshoes

Copy link
Copy Markdown
Contributor

Summary

  • Adds TinybirdApi.sample_datasource(), starting a bounded sample import job for S3/GCS/DynamoDB connected data sources via POST /v0/datasources/{name}/sample, matching tb datasource sample.
  • For blob storage connectors, max_files bounds how many files are imported (default 1).
  • For DynamoDB, the sample is bounded by rows or max_bytes (mutually exclusive), or full_export triggers a full PITR export instead of a bounded scan.

This is pre-existing in-progress work (not one of the PROTM-2255 sub-tickets) that was sitting uncommitted; shipping it separately so it doesn't get tangled with the parity-gap PRs.

Checklist

  • CI is green (lint, typecheck, test, secrets)
  • pre-commit run --all-files passes locally
  • Tests were added or updated when behavior changed
  • Public API / typing changes were reviewed
  • Documentation was updated (README.md / CONTRIBUTING.md) if needed
  • Breaking changes are clearly documented
  • CHANGELOG.md was updated when user-facing behavior changed

Mirrors `tb datasource sample`: starts a bounded sample import job for
blob storage (max_files) or DynamoDB (rows/max_bytes, or a full PITR
export via full_export) connected data sources.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant