duckdb.experimental.spark fails to import without typing_extensions, which is not a runtime dependency #653

What happens?

duckdb.experimental.spark imports typing_extensions at module import time, but typing-extensions is only listed in the stubdeps and test dependency groups in pyproject.toml, not as a runtime dependency and not in the all extra. In a clean environment with just duckdb and numpy (the dependency the all extra comment names for the spark module), importing the spark API fails.

The three imports are:

• duckdb/experimental/spark/_globals.py:36 — from typing_extensions import Self
• duckdb/experimental/spark/sql/types.py:14 — from typing_extensions import Self
• duckdb/experimental/spark/_typing.py:22 — from typing_extensions import Protocol, Self

Since the package requires Python >= 3.11, typing.Self and typing.Protocol are available from the standard library, so these can simply be from typing import Self / from typing import Protocol, Self. duckdb/polars_io.py and duckdb/experimental/spark/sql/_typing.py already guard their typing_extensions usage. The failure is masked when duckdb[all] is installed because typing_extensions arrives transitively through pandas/pyarrow.

Happy to send a PR if the from typing import ... change is the preferred fix.

To Reproduce

python -m venv v && . v/bin/activate
pip install --pre duckdb numpy
python -c "from duckdb.experimental.spark.sql import SparkSession"

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File ".../site-packages/duckdb/experimental/spark/__init__.py", line ...
  File ".../site-packages/duckdb/experimental/spark/_globals.py", line 36, in <module>
    from typing_extensions import Self
ModuleNotFoundError: No module named 'typing_extensions'

Same result on the 1.5.6 release.

OS:

macOS 15 (aarch64)

DuckDB Package Version:

2.0.0.dev2610011535 (nightly); also 1.5.6

Python Version:

3.12

Full Name:

Joaquin Hui Gomez

Affiliation:

Independent contributor

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a nightly build

Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

• Yes, I have