Skip to content

fix(provider): honour driver_name when a connection string omits the DBAPI - #2429

Open
C1-BA-B1-F3 wants to merge 3 commits into
geopython:masterfrom
C1-BA-B1-F3:fix/psycopg-driver-from-conn-str
Open

C1-BA-B1-F3 wants to merge 3 commits into
geopython:masterfrom
C1-BA-B1-F3:fix/psycopg-driver-from-conn-str

Conversation

@C1-BA-B1-F3

Copy link
Copy Markdown
Contributor

Summary

A provider configured with a bare connection string:

data: postgresql://postgres:secret@localhost/test

resolves to SQLAlchemy's default DBAPI for the backend — psycopg (v3) for PostgreSQL. Deployments that only install psycopg2 (per requirements-provider.txt) then fail at engine creation:

sqlalchemy/dialects/postgresql/psycopg.py in import_dbapi
ModuleNotFoundError: No module named 'psycopg'

even though PostgreSQLProvider.__init__ pins driver_name='postgresql+psycopg2'. The pin is silently ignored whenever data is a string, because store_db_parameters routes it to self.db_conn and get_engine uses the URL verbatim.

The CI failure in tests/provider/test_postgresql_provider (and test_api_connection_rfc3986 in the manager suite, which also builds its connection string with a bare postgresql:// URL) is exactly this.

Fix

In get_engine, when the caller's URL has no +driver suffix and the provider's driver_name does, rewrite the URL's drivername to the pinned one. Explicitly-pinned URLs (postgresql+psycopg://...) are untouched.

Test plan

  • Verified with mocked create_engine:
    • postgresql://… + pin postgresql+psycopg2 → engine URL becomes postgresql+psycopg2
    • postgresql+psycopg://… → untouched
    • no conn_str → built from driver_name as before
  • test_api_connection_rfc3986 now reaches the real connection attempt (psycopg2.OperationalError: connection refused locally) instead of failing at import psycopg, confirming the v2 driver is selected.

Fixes #2424

PostgreSQL and MongoDB managers ignored the limit and offset
parameters, returning all jobs regardless of the query string.

Apply offset before limit in both backends, matching the existing
TinyDB behaviour.

Fixes geopython#2426
…ints

The tilematrixsets and tilematrixset HTML handlers passed api.tpl_config
as the template configuration, but that dict carries the full server
config (including 'server') and lacks the 'path' key that
render_j2_template expects. The lookup therefore always fell back to
the default templates, ignoring any custom theme.

Pass api.config['server']['templates'] instead, matching every other
render_j2_template call site.

Fixes geopython#2422
…DBAPI

A provider configured with a bare connection string (e.g.
"postgresql://user@host/db") resolved to SQLAlchemy's default DBAPI
for that backend — psycopg (v3) for PostgreSQL. Deployments that only
install psycopg2 then fail with 'ModuleNotFoundError: No module named
psycopg' even though the provider pins 'postgresql+psycopg2'.

When the caller's URL has no '+driver' suffix and the provider's
driver_name does, rewrite the URL's drivername to the pinned one.

Fixes geopython#2424
@C1-BA-B1-F3

Copy link
Copy Markdown
Contributor Author

The Build failure is in test_describe_collections (assert 9 == 10) and test_get_collection_edr_query (HTTP 500), the same two pre-existing upstream regressions that also fail on the current master branch and on my other PRs (#2427, #2428). They exercise /collections and EDR queries; this PR only touches pygeoapi/provider/sql.py (the get_engine helper). Happy to rebase once upstream fixes the CI environment.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI is failing, because 'psycopg' is not installed

1 participant