Validate Cloud Function deploy body after template rendering - #70531
Conversation
potiuk
left a comment
There was a problem hiding this comment.
Thanks — correct move. body is a template field, so GcpBodyFieldValidator in __init__ was validating the un-rendered value: a templated body could never pass validation, and a literal bad one broke Dag parsing instead of failing the task. Constructing the validator and the ZipPathPreprocessor in execute and running _validate_inputs() there is the right shape.
test_templated_body_deploys_after_rendering is a good addition — asserting create_new_function.assert_called_once() after a rendered body proves the happy path still works, not just that the failure path raises.
One robustness point inline about an attribute that no longer exists until execute runs.
Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting
8319802 to
3280b64
Compare
3280b64 to
05b9690
Compare
05b9690 to
1a79c1e
Compare
Rebased onto current main. Move body/location validation and zip preprocessing into execute so Jinja-templated body is checked after render.
CloudFunctionDeployFunctionOperator only assigned self.zip_path_preprocessor inside execute(), so between construction and the first execute() the attribute didn't exist at all and any access raised AttributeError - affecting subclasses, tests, and anything introspecting the operator between parse and run. Initialize it to None in __init__, like the sibling self._field_validator, and build the real ZipPathPreprocessor in execute() as a local variable that both _validate_inputs() and should_upload_function() use, so mypy can verify there is no Optional dereference.
test_templated_body_deploys_after_rendering passed unmodified on pre-PR main: with a templated string body and no zip_path, every ZipPathPreprocessor check is an "x in self.body" membership test, and `in` on a str is a substring test that just returns False, so the old __init__-time preprocessing never raised and the test asserted nothing that distinguished old from new behavior. Replace it with two tests that fail on the pre-PR code and pass on this PR: - test_templated_body_with_zip_path_uploads_after_rendering: with a templated body and a zip_path, the pre-PR code preprocessed the un-rendered string at __init__ time, so upload_function was silently left False and the zip was never uploaded - a wrong result, not a crash. Confirmed by running this test against functions.py from before this PR (c85aff6^): it fails with "Expected 'upload_function_zip' to be called once. Called 0 times." - test_templated_location_rendered_empty_raises: the pre-PR code checked `location` truthiness once, at __init__ time, against the truthy "{{ ... }}" string, and never re-checked it in execute, so an empty rendered location silently proceeded to deploy. Confirmed by running this test against the pre-PR code: it fails with "DID NOT RAISE AirflowException". Both tests pass against this PR's functions.py, which validates location and preprocesses the zip path after template rendering.
1a79c1e to
ea9dd20
Compare
Moving GcpBodyFieldValidator construction into execute() was one of the three render-order fixes here, but it was the only one left without a test. sourceRepositoryUrl is the single api_version-gated spec in CLOUD_FUNCTION_VALIDATION, so it is the only available lever to prove the validator now sees the rendered value rather than the Jinja string.
|
Edit: few corrections I made after review (drafted by Claude Code & Codex).
|
Part of the template-field validation burn-down tracked in #70296.
CloudFunctionDeployFunctionOperatorlistsbody,location, andapi_versionintemplate_fields, but__init__validated the body, ranZipPathPreprocessor.preprocess_body()(which both validates and mutates the body), and constructedGcpBodyFieldValidatorpinned to the un-renderedapi_version. A fully templatedbodycrashed withAttributeErrorat parse time inside the preprocessor. All of this now runs at the start ofexecute()against the rendered values.The missing-
location/bodytruthiness checks and the zip-path exclusivity rules depend on rendered values (an expression rendering to an empty string must count as missing), so per the discussion in #70505 these are genuine value reads, not provision checks.Tests: converted the four construction-time raise tests to execute-time, and added
test_templated_body_deploys_after_rendering, which constructs the operator with a templatedbody(previously a parse-time crash) and deploys once the field holds the rendered value — it fails against the previous implementation. The class is removed from the exemption list and thevalidate-operators-initcheck passes locally.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Fable 5) following the guidelines