[fmercadop]

Adds a warning to the @task.datacoves_dbt decorator docs and an explicit note on the service connections page stating that the Schema field is required when a service connection is used with @task.datacoves_dbt. Leaving it empty causes a dbt profile validation error at runtime (Credentials in profile "...", target "..." invalid: None is not of type 'string'), and it's easy to miss since the Airflow Snowflake provider docs list schema as optional for the Airflow connection itself.

[noel]

@fmercadop what does this mean? e.g. why not just override in the decorator itself?

[fmercadop]

@noel I may be misreading your question, so let me check. I think you're asking about the Profile name row in the overrides table: why would we point people to a DATACOVES__DBT_PROFILE env var instead of overriding it in the decorator itself, as we do with target and overrides?

If that's the question: there's no profile-name param on @task.datacoves_dbt. That row (and the env={"DATACOVES__DBT_PROFILE": ...} example further down) was already in the docs on main. I only reordered the sentence into a table.

Would you like me to remove that row from the overrides table, so we only document target and overrides? If you were asking about something else, let me know which part and I'll clarify.

[noel]

I think we should also say to override things via the decorator. I am not sure if that works at run time, in that case they can use env = ..... but I dont know if dbt would pick that up.
I think R&D was trying to do something like that. maybe talk to Ian bec there was something he changes that may allow this. Basically I think they wanted to pass a parameter when they go to run the DAG which would replace the schema for example

[fmercadop]

Thanks @noel. I talked to Ian, checked the decorator code, and tested it in My Airflow.

Overriding in the decorator: yes, that works. target and overrides are templated fields since 6.0. I tested these three ways to set the schema, and all of them work:

1. Fixed value, same on every run:

@task.datacoves_dbt(
    connection_id="main",
    overrides={"schema": "my_schema"},
)

2. Chosen when you run the DAG manually. Add a schema param to the DAG with a default value:
   • When the DAG runs on its schedule, it uses the default.
   • When you trigger it manually from the Airflow UI, Airflow asks for the schema, pre-filled with the default, and you can change it before starting the run.

@dag(params={"schema": "my_default_schema"})
def my_dbt_dag():
    @task.datacoves_dbt(
        connection_id="main",
        overrides={"schema": "{{ params.schema }}"},
    )
    def run_dbt():
        return "dbt build"

3. Per environment, using an Airflow Variable, so the same DAG uses one schema in dev and another in prod:

@task.datacoves_dbt(
    connection_id="main",
    overrides={"schema": "{{ var.value.dbt_schema }}"},
)

The env= approach: you were right to doubt it. When connection_id is used, the generated profile's name is read from the worker's environment, which comes from the Profile name setting. The task's env is only passed to the shell that runs dbt, after the profile file has already been written. So env={"DATACOVES__DBT_PROFILE": ...} doesn't rename the generated profile.

Proposal for this PR:

• Document the three options above for overrides (and that target is templated too).
• Remove the profile-name row and the env= line from the overrides example, and note that the profile name comes from the Profile name setting (and must match profile: in dbt_project.yml).

Sound good?

[fmercadop]

Hi @noel, I pushed the doc changes we discussed. Summary:

Overrides

• Removed the DATACOVES__DBT_PROFILE row and env= line, since passing it via env doesn't rename the generated profile. The page now says the profile name comes from the Profile name setting and must match profile: in dbt_project.yml.

New "Setting values dynamically" section

• Explains that target and overrides support Airflow templating, using schema as the example.
• Covers the 4 options I tested: fixed value, params (chosen at trigger), Airflow Variable, and an env var resolved by dbt.
• For the env var option, it notes that {% raw %} and append_env=True are required.