Conversation
The STAC API search endpoint was ignoring the collections query parameter, returning items from all configured STAC collections regardless of filtering. This is because the search function iterated over every configured stac-collection resource and queried its provider without constraining by the requested collection IDs. Fix this by parsing the collections parameter (from GET query string or POST JSON body) and injecting a CQL-text filter on the provider's id_field before delegating to get_collection_items. For example, collections=conus404-daily becomes filter="id = 'conus404-daily'" which the SQL provider translates into a WHERE clause against the database. The original request args are saved before filter injection and restored afterward so that paging links reflect the user's original query parameters rather than the internal CQL translation.
The current PostgreSQL provider puts all columns into `properties`, which is correct for a GeoJSON Feature, but not a STAC Item, which needs properties like bbox, collection, stac_version, assets, and links to be top level. Thus, we introduce a new custom STAC PostgreSQL provider which inherits all the essentials from the PostgreSQL provider, while adding the custom serialization necessary for proper STAC construction.
Previously, the STAC items were rendered without a self link. If the source ingest provided relative links to resources, they would be published directly via the STAC API, causing downstream issues in pystac client: > STACError: Relative path ../collection.json encountered without owner "self" link set. This now converts all published links to absolute URLs, which pystac client can use to navigate around the catalog.
Adds conversion from the STAC conventions of
[{field, direction}] in the POST body to pygeoapi's
sortby string. This allows POST based clients, like
pystac-client, to sort. Previously only GET sorting
was supported.
Previously the ids filter was ignored. Now we translate it to a CQL `id in (...)` expression. CQL escaping is extracted into a `_cql_in` helper.
Add filter / filter-lang to the POST query intercept so cql2-text filters are preserved. cql2-json object filter support will arrive in a future commit.
These advertise filter and sort capabilities. Notably, pystac-client defaults to cql2-json for filtering, which is currently ignored. That will be properly implemented in the near future.
pygeoapi will use cql2-json filtering if given in
the body. If we receive a filter object in the search
query, it replaces the full request body to activate
the cql2-json parsing path.
This now adds native support for the conventional
catalog.search(filter={}) expression of pystac-client.
Notably, this does not combine with collections / ids filters,
which are injected as cql2-text filters, thus overriding the
cql2-json body. That is a rare combination, we can revisit
this in the future if it becomes a common problem.
Also add relevant conformance classes.
Previously the collections filter was erroneously filtering on the STAC item id field. This was leftover from when the search endpoint was searching collections. Now it correctly filters the collections field, as defined in the configuration of the provider.
Translates intersects GeoJSON into a cql2-json S_INTERSECTS filter, AND combined with any other explicit filters.
Introduce a `mode` provider key so a single provider class can serve both STAC Items and Collections from their respective tables, rather than duplicating the reshape and scoping machinery across two near-identical classes. The single-table limit is a property of the inherited PostgreSQLProvider (one reflected model per instance), not of pygeoapi itself, so `mode` selects the reshape per configured resource: `items` (default) keeps today's behaviour byte-for-byte; `collections` is a recognised, planned mode that fails fast until the collections-search work builds it out. No item behaviour changes. Config must now name STACSQLProvider and may set `mode: items` explicitly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reshape stac_collections rows into valid STAC Collections: lift the STAC top-level fields and JSONB extension blobs to the top level and drop the derived query-acceleration columns (geometry, the datetime bounds, created/updated). Collections mode also adds the two collection-search filters the items path lacked: datetime becomes an interval overlap against the start_datetime/end_datetime columns (NULL bounds treated as open), and a free-text q term becomes a case-insensitive ILIKE over the title/description columns, threaded through query() as a sentinel property so the parent's filter machinery is reused. Items mode behaviour is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the STAC API Collections resource and the collection-search
extension on top of the collections-mode provider:
- GET /stac-api/collections -- listing + search (bbox, datetime,
limit, sortby, filter, and q free-text), reusing the OGC feature
item machinery and reshaping the result into a Collections doc.
- GET /stac-api/collections/{collectionId} -- a single Collection.
- GET /stac-api/collections/{collectionId}/items -- items in one
collection (delegates to search, scoped to the collection).
- GET /stac-api/collections/{collectionId}/items/{itemId} -- a
single Item, so item self links resolve (previously 404).
Navigation links are rewritten to absolute STAC API links
(_rewrite_collection_links mirrors _rewrite_item_links). The
landing page advertises the collections + collection-search
conformance classes and a data link. Routes are wired into both
the Flask and Starlette apps.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
rajadain
force-pushed
the
tt/usgs-stac-collections-search
branch
from
September 17, 2026 15:44
9ea7626 to
81a318c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Serves the USGS Water Mission Area STAC catalog through pygeoapi as a spec-compliant STAC API.
The generic pygeoapi Postgres provider flattens every column into GeoJSON
properties, which is fine for Features but produces invalid STAC (buried assets, doubly-nestedproperties, no top-levelcollection/bbox). This PR adds a provider that reshapes rows correctly, then fixes the STAC/searchendpoint up to spec sopystac-clientcan actually drive it end-to-end, and finally extends the same provider to serve/collections(the collection-search extension) using the same reshape/scoping machinery.Custom provider (
pygeoapi/provider/stac_sql.py)STACSQLProvidersubclasses the existingPostgreSQLProviderand overrides only the row → document reshape and the query-filter seam, reusing the parent's engine, reflection, and paging. Amodekey (items|collections) picks which STAC document type the configured table holds and which reshape runs, since a single provider instance is still bound to one table.itemsmode also supports scoping a resource to onecollectionvalue, so onestac_itemstable can back many single-collection pygeoapi resources.collectionsmode adds two additional filters:datetimeas an interval overlap againststart_datetime/end_datetime(NULL = open), and a free-textqterm as anILIKEover title/description, threaded throughquery().Ultimately this Custom Provider may live in the USGS STAC repo. See stac!262. We add it here to get it onto the USGS Staging site.
Item search (
pygeoapi/api/stac.py)A series of fixes bring
/stac-api/searchin line with the needs ofpystac-client. STAC's POST search body doesn't map 1:1 onto pygeoapi's GET query-param model, so the POST handler translates each STAC convention into the args pygeoapi understands:collections/idsbecome CQLIN (...)predicates against the resource's configured id/collection fields,sortby's{field, direction}objects become pygeoapi's+/-fieldtokens,intersectsbecomes acql2-jsonS_INTERSECTSterm, and an explicitfilter(cql2-text or cql2-json) is preserved and AND-combined with the above. Filter/sort/CQL2 conformance classes are advertised to match.Separately, STAC Items were carrying source-catalog links (e.g.
../collection.json) straight through to API responses, which brokepystac-client's navigation the moment it followed aselflink; item responses now get their nav links (self/root/parent/collection) rewritten to absolute API URLs while portable links (e.g.cite-as) pass through untouched.A new
_stac_api_resources(api, mode)helper (added alongside the collections work) letssearch()fan out only overitems-mode resources, oncecollections-mode resources also exist.Collections search (
pygeoapi/api/stac.py,flask_app.py,starlette_app.py)With the provider able to serve
collectionsmode, this adds the API:GET /stac-api/collections(listing + collection-search: bbox, datetime, limit, sortby, filter, q),GET /stac-api/collections/{id},GET /stac-api/collections/{id}/items, andGET /stac-api/collections/{id}/items/{itemId}. The collections listing deliberately reuses the same OGC feature-item machinery as item search (get_collection_items) rather than a new query path, then reshapes the result into a STAC Collections document, same pattern as Item search.items/{id}closes a gap the link-rewrite work surfaced: itemselflinks now resolve to a real 200 instead of 404, since until now nothing served a single item by collection + id. Link rewriting for collections (_rewrite_collection_links) mirrors the item version.Routes are wired into both the Flask and Starlette apps; the deployed container serves Starlette.