Repository navigation
standgen: call GDAM's public URL with the flow-matching model - #734
Merged
Merged
Conversation
Point GDAM_API_URL (code default and standgen.yml) at https://api.fastfuels.silvxlabs.com/gdam. Request flow_matching v2_west.1 with tree_observation "overhead", pinned so a GDAM default change can't change our output. Each partition sends a seed derived from the inventory id and partition number, so reprocessing reproduces the draws and batches don't share noise. Retry 429 and 502/503/504 with backoff, honouring Retry-After (capped at 30s). Reject non-positive heights up front (the flow model 400s on them). Update the GDAM endpoint description, which named the autoencoder.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves standgen's GDAM calls to the public URL and to GDAM's flow-matching model.
Changes
GDAM_API_URL(code default andstandgen.yml) →https://api.fastfuels.silvxlabs.com/gdam. The handler appends/predict/batch, so requests go tohttps://api.fastfuels.silvxlabs.com/gdam/predict/batch./predict/batchrequest names{"name": "flow_matching", "version": "v2_west.1", "tree_observation": "overhead", "seed": ...}. The version is pinned so a change to GDAM's default can't change our output.batch_seed(inventory_id, partition)(first 4 bytes of sha256). Reprocessing an inventory gives the same partitions and seeds, so it reproduces the values. Different partitions get different seeds, so batches don't share noise. The inventory doc has no allometry provenance field, so the model/version is only logged, not stored.GDAM_MAX_ATTEMPTS), honouringRetry-Afterup to 30 s and otherwise backing off exponentially. 500 and other 4xx stay terminal, and the error message now includes GDAM's response body.NON_POSITIVE_HEIGHT. The flow model returns 400 for them, and uploads allowheight >= 0.model_versionanswers with the seed we sent and that the same seed returns the same draws.Measurements (public URL, synthetic 20,000-tree batch over ~0.5 km² near Missoula)
Repeated requests with the same seed returned identical DIA/CR/SPCD.
GDAM_BATCH_SIZE(20k) andGDAM_REQUEST_TIMEOUT_S(120 s) stay as they are: 20k is well under GDAM's 50k cap and the timeout, and larger batches cost less per tree. Note that standgen's Cloud Run timeout is 300 s for the whole inventory, and flow batches are about 4× slower than autoencoder batches. A dense 16 km² domain (around 30 or more batches) could get close to that limit.Behaviour change
Imputed dbh / crown_ratio / species values change for every new GDAM inventory. They are now draws from a distribution instead of point predictions. The API contract (fields, status codes) is unchanged.
Merge order
This PR and silvxlabs/FastFuels-IaC (standgen
GDAM_API_URL) don't depend on each other. I verified that the old URL serves the flow model at/predict/batch, and that the public URL serves the autoencoder whenmodelis omitted. Either can merge first. Until both are merged, the standgen deploy (--update-env-vars) and Terraform set different URLs, and both URLs work.