Skip to content

Add dbt-run and dbt-version facets validation - #257

Closed
pawelmarut wants to merge 1 commit into
mainfrom
feature/dbt-validation
Closed

pawelmarut wants to merge 1 commit into
mainfrom
feature/dbt-validation

Conversation

@pawelmarut

Copy link
Copy Markdown
Collaborator

No description provided.

@pawelmarut
pawelmarut requested a review from tnazarew January 29, 2026 14:21
Signed-off-by: Pawel Marut <pawel.marut@xebia.com>
@pawelmarut
pawelmarut force-pushed the feature/dbt-validation branch from 8e62d25 to 2dddf25 Compare January 29, 2026 15:23
tnazarew added a commit that referenced this pull request Nov 4, 2026
Load dbt-version-run-facet.json and dbt-run-run-facet.json from the common
package so the dbt_version and dbt_run facets are schema-validated instead
of skipped. They stay in the skip list as a fallback for OpenLineage
releases before 1.40, whose schemas live outside the src/ layout that the
spec collection copies. dbt_model stays skipped, it has no published schema.

Folds in #257.

Co-authored-by: Pawel Marut <pawel.marut@xebia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: tnazarew <tomasz.nazarewicz@getindata.com>
@tnazarew

tnazarew commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

folded into #337, closing

@tnazarew tnazarew closed this Nov 4, 2026
@tnazarew

tnazarew commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

Thanks for this! The facet-validation change is in #337, which also had to touch is_custom_facet (it adds a skip for dbt_model, which has no published schema). dbt_version and dbt_run are now validated against their schemas there, with Co-authored-by credit to you.

One difference: dbt-version-run and dbt-run-run stay in the skip list as a fallback. It's only used when no schema is loaded, and for OpenLineage < 1.40 the schemas aren't collected (they're outside integration/common/src), so removing them would fail those releases. I left out the producer/dbt/runner/openlineage.yml change since CI doesn't read that file.

tnazarew added a commit that referenced this pull request Nov 4, 2026
* Skip schema validation of the dbt_model dataset facet

openlineage-dbt emits dbt_model (DbtModelDatasetFacet) with _schemaURL
.../integration/common/openlineage/schema/dbt-model-dataset-facet.json,
but that file is not published at any OpenLineage tag or on main; the
facet exists only in facets.py. Syntax validation therefore fails with
"facet type dbt_model not recognized". Treat it like dbt_version and
dbt_run until upstream publishes the schema.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: tnazarew <tomasz.nazarewicz@getindata.com>

* dbt: validate dbt_run and dbt_version facets against their schemas

Load dbt-version-run-facet.json and dbt-run-run-facet.json from the common
package so the dbt_version and dbt_run facets are schema-validated instead
of skipped. They stay in the skip list as a fallback for OpenLineage
releases before 1.40, whose schemas live outside the src/ layout that the
spec collection copies. dbt_model stays skipped, it has no published schema.

Folds in #257.

Co-authored-by: Pawel Marut <pawel.marut@xebia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: tnazarew <tomasz.nazarewicz@getindata.com>

---------

Signed-off-by: tnazarew <tomasz.nazarewicz@getindata.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Pawel Marut <pawel.marut@xebia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants