Dedup @task sidecar entries by component identity (v0.1.14)

README.md

@@ -457,6 +457,10 @@ Notes:
 
 A minimal graph uses `@pipeline` for the graph and `@task` for local Python components. `@task` functions are not executed at compile time; the compiler records call sites, emits a sibling `<output>.components.yaml` with `local_from_python` entries, and rewrites task component refs to that sidecar. Hydrate later regenerates the same component YAML from the Python source.
 
+Sidecar entries are deduplicated by the **generated component**, not by function name. The dedup key is the module-qualified function identity — a logical module namespace derived from the project-relative source layout (`package_a/tasks.py` -> `package_a.tasks`, `pkg/__init__.py` -> `pkg`), plus `__qualname__` and the function name — together with every generation-affecting option: image (explicit or resolved `image_id`), `mode`, `resolve_root`, `dependencies_from`, the `unwrap` schema, and the source file. Runtime `__module__` is deliberately not used, because pipeline scripts are imported under throwaway UUID module names.
+
+Call sites sharing that whole identity collapse to one entry keyed by the readable hyphenated function name (`run_dbt` -> `run-dbt`). Otherwise every colliding variant — a shared helper re-decorated with different task-level options, or `package_a.tasks.run` alongside `package_b.tasks.run` — is emitted as `run-dbt--<hash>`, where `<hash>` is a content digest of the canonical identity; no variant keeps the unsuffixed name, and each task ref points at its own. Digests are anchored at the project source directory, so fragment names are unchanged by relocating the project, compiling into a different output directory, or reordering the pipeline's calls, and they never embed image, path, or credential-bearing values.
+
 ```python
 from cloud_pipelines import components
 from tangle_cli.python_pipeline import In, Out, pipeline, task

examples/python_pipeline/dedup_image_variants/__init__.py

@@ -0,0 +1,7 @@
+"""Package marker so the shared helper has a package-qualified module identity.
+
+With this file present the compiler derives the logical module namespace
+``dedup_image_variants.shared_dbt`` from the source layout (it walks up while
+``__init__.py`` exists). Without it the helper would still work, but its module
+identity would be the bare ``shared_dbt``.
+"""

examples/python_pipeline/dedup_image_variants/pipeline.py

@@ -0,0 +1,86 @@
+"""Manual-test example: per-variant @task sidecar dedup for ONE shared function.
+
+The SAME module-qualified function -- ``dedup_image_variants.shared_dbt.run_dbt``
+-- is re-decorated three times: twice with distinct task-level images, and once
+more with an image identical to the first. Before the dedup fix the sidecar was
+keyed by function NAME, so all three collapsed into the first entry and every
+task silently resolved to the first call site's image.
+
+Compile from the repository root with::
+
+    uv run tangle sdk pipelines compile \
+      examples/python_pipeline/dedup_image_variants/pipeline.py \
+      --pipeline dedup_image_variants_pipeline \
+      --output /tmp/tangle-dedup-demo/pipeline.yaml
+
+Then inspect the two generated artifacts::
+
+    cat /tmp/tangle-dedup-demo/pipeline.components.yaml
+    cat /tmp/tangle-dedup-demo/pipeline.yaml
+
+Expected assertions
+-------------------
+1. ``pipeline.components.yaml`` has EXACTLY TWO entries, both hashed:
+   ``run-dbt--<10 hex chars>``. Neither keeps the bare ``run-dbt`` name --
+   once a base collides, every variant is suffixed, so there is no arbitrary
+   "first variant wins".
+2. The two entries differ ONLY in ``local_from_python.image``
+   (``python:3.12-slim`` vs ``python:3.12``); both carry
+   ``function: run_dbt`` and the same ``file: ./shared_dbt.py``.
+3. Each graph task's ``componentRef.url`` is
+   ``resolve://./pipeline.components.yaml#run-dbt--<hash>`` pointing at the
+   fragment whose image matches that task's decorator:
+   ``daily_orders`` and ``hourly_sessions`` -> the SLIM entry (they share one
+   identity and therefore one fragment -- repeated identical calls still
+   dedup), ``backfill_orders`` -> the FAT entry.
+4. No raw image, registry, tag, digest, or source-path text appears in any
+   fragment name: the suffix is a SHA-256 prefix over the canonical identity,
+   so ``python``, ``slim``, ``3.12``, ``shared_dbt`` and ``.py`` must NOT be
+   substrings of any sidecar key.
+5. Fragment names are stable: recompiling to a different ``--output``
+   directory, or moving this example tree elsewhere, produces the SAME two
+   fragment names (identity is anchored at the pipeline source directory, not
+   the output directory and not an absolute machine path).
+
+Observed output on this revision (the digests are derived only from
+project-relative values, so they reproduce on any checkout). The sidecar lists
+colliding variants in sorted fragment order, which is why the fat entry appears
+first::
+
+    run-dbt--573d8d33bd   image: python:3.12
+    run-dbt--e61a70231d   image: python:3.12-slim
+
+    daily_orders     -> resolve://./pipeline.components.yaml#run-dbt--e61a70231d
+    hourly_sessions  -> resolve://./pipeline.components.yaml#run-dbt--e61a70231d
+    backfill_orders  -> resolve://./pipeline.components.yaml#run-dbt--573d8d33bd
+
+Note on ``file:``: sidecar paths are relative to the OUTPUT directory, so
+compiling into ``/tmp`` (outside the source tree) writes a long ``../../..``
+path to ``shared_dbt.py``. That is expected and is exactly the point of
+assertion 5 -- the emitted path tracks the output directory while the fragment
+NAMES do not. Compile next to the script if you want short paths.
+"""
+
+from tangle_cli.python_pipeline import Out, pipeline, task
+
+from shared_dbt import run_dbt
+
+# Three decorations of ONE function. ``dbt_slim`` and ``dbt_slim_again`` are
+# byte-identical configurations, so they share a single generated component;
+# ``dbt_fat`` differs in image, so it is a genuinely different component.
+dbt_slim = task(image="python:3.12-slim")(run_dbt)
+dbt_slim_again = task(image="python:3.12-slim")(run_dbt)
+dbt_fat = task(image="python:3.12")(run_dbt)
+
+
+@pipeline("Dedup image variants demo")
+def dedup_image_variants_pipeline() -> Out[str]:
+    # Two call sites, one image -> ONE sidecar entry, two task refs to it.
+    daily = dbt_slim.named("daily_orders")(model="orders_daily")
+    hourly = dbt_slim_again.named("hourly_sessions")(model="sessions_hourly")
+
+    # Same function, different image -> its OWN sidecar entry and task ref.
+    backfill = dbt_fat.named("backfill_orders")(model=daily, target="backfill")
+
+    print(hourly)
+    return backfill

examples/python_pipeline/dedup_image_variants/shared_dbt.py

@@ -0,0 +1,16 @@
+"""One shared dbt-style helper, imported and re-decorated by ``pipeline.py``.
+
+This module exists so the example exercises the REAL shape of the bug: a single
+module-qualified function (``dedup_image_variants.shared_dbt.run_dbt``) reached
+from several call sites with different task-level images, rather than two
+separate functions that happen to look alike.
+"""
+
+
+def run_dbt(model: str, target: str = "prod") -> str:
+    """Run one dbt model.
+
+    Metadata:
+        Name: Run Dbt Model
+    """
+    return f"{model}@{target}"

packages/tangle-cli/src/tangle_cli/__init__.py

@@ -14,6 +14,6 @@
 try:
     __version__ = metadata_version("tangle-cli")
 except PackageNotFoundError:
-    __version__ = "0.1.13"
+    __version__ = "0.1.14"
 
 __all__ = ["TangleDynamicDiscoveryClient", "__version__"]