Skip to content

standgen: call GDAM's public URL with the flow-matching model - #734

Merged
amarcozzi merged 1 commit into
mainfrom
gdam-flow-matching
Oct 16, 2026
Merged

amarcozzi merged 1 commit into
mainfrom
gdam-flow-matching

Conversation

@amarcozzi

Copy link
Copy Markdown
Contributor

Moves standgen's GDAM calls to the public URL and to GDAM's flow-matching model.

Changes

  • GDAM_API_URL (code default and standgen.yml) → https://api.fastfuels.silvxlabs.com/gdam. The handler appends /predict/batch, so requests go to https://api.fastfuels.silvxlabs.com/gdam/predict/batch.
  • Every /predict/batch request names {"name": "flow_matching", "version": "v2_west.1", "tree_observation": "overhead", "seed": ...}. The version is pinned so a change to GDAM's default can't change our output.
  • Seeds: each partition sends batch_seed(inventory_id, partition) (first 4 bytes of sha256). Reprocessing an inventory gives the same partitions and seeds, so it reproduces the values. Different partitions get different seeds, so batches don't share noise. The inventory doc has no allometry provenance field, so the model/version is only logged, not stored.
  • Retries: 429 and 502/503/504 are now retried along with transport errors (same GDAM_MAX_ATTEMPTS), honouring Retry-After up to 30 s and otherwise backing off exponentially. 500 and other 4xx stay terminal, and the error message now includes GDAM's response body.
  • Heights ≤ 0 now fail up front with NON_POSITIVE_HEIGHT. The flow model returns 400 for them, and uploads allow height >= 0.
  • The GDAM endpoint description no longer says a masked autoencoder fills the fields, and it now says height must be above zero.
  • Tests: unit tests for the model body, seeds per partition and across reruns, status retries, Retry-After, and non-positive heights. The live contract test now also checks that the pinned model_version answers with the seed we sent and that the same seed returns the same draws.

Measurements (public URL, synthetic 20,000-tree batch over ~0.5 km² near Missoula)

wall
autoencoder (old behaviour) 2.4 s
flow_matching, first request (cold) 23.6 s
flow_matching, warm 9.4 to 10.8 s
flow_matching, 2,000 trees 1.6 to 1.9 s

Repeated requests with the same seed returned identical DIA/CR/SPCD. GDAM_BATCH_SIZE (20k) and GDAM_REQUEST_TIMEOUT_S (120 s) stay as they are: 20k is well under GDAM's 50k cap and the timeout, and larger batches cost less per tree. Note that standgen's Cloud Run timeout is 300 s for the whole inventory, and flow batches are about 4× slower than autoencoder batches. A dense 16 km² domain (around 30 or more batches) could get close to that limit.

Behaviour change

Imputed dbh / crown_ratio / species values change for every new GDAM inventory. They are now draws from a distribution instead of point predictions. The API contract (fields, status codes) is unchanged.

Merge order

This PR and silvxlabs/FastFuels-IaC (standgen GDAM_API_URL) don't depend on each other. I verified that the old URL serves the flow model at /predict/batch, and that the public URL serves the autoencoder when model is omitted. Either can merge first. Until both are merged, the standgen deploy (--update-env-vars) and Terraform set different URLs, and both URLs work.

Point GDAM_API_URL (code default and standgen.yml) at
https://api.fastfuels.silvxlabs.com/gdam. Request flow_matching v2_west.1
with tree_observation "overhead", pinned so a GDAM default change can't
change our output. Each partition sends a seed derived from the inventory
id and partition number, so reprocessing reproduces the draws and batches
don't share noise.

Retry 429 and 502/503/504 with backoff, honouring Retry-After (capped at
30s). Reject non-positive heights up front (the flow model 400s on them).
Update the GDAM endpoint description, which named the autoencoder.
@amarcozzi
amarcozzi merged commit cc8650f into main Oct 16, 2026
1 check passed
@amarcozzi
amarcozzi deleted the gdam-flow-matching branch October 30, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant