Skip to content

fix(event_handler): harden multipart and JSON body validation - #8626

Merged
leandrodamascena merged 3 commits into
aws-powertools:developfrom
MTirnova:fix/openapi-validation-body-parsing
Oct 22, 2026
Merged

leandrodamascena merged 3 commits into
aws-powertools:developfrom
MTirnova:fix/openapi-validation-body-parsing

Conversation

@MTirnova

Copy link
Copy Markdown
Contributor

Issue number: closes #8578

Summary

Changes

All three problems are in middlewares/openapi_validation.py.

1. Multipart errors escaping as 500/502

  • A missing boundary now raises RequestUnsupportedContentType, so it returns 415 (missing_boundary) instead of a bare ValueError.
  • Removed the except ValueError: raise in _parse_multipart_data. It also re-raised UnicodeDecodeError and binascii.Error (both are ValueError subclasses), so they escaped as 500s. They now become the existing 422 multipart_invalid error.
  • Non-ASCII text fields in non-base64 bodies are decoded correctly instead of crashing. I kept the existing latin-1 round trip, because test_file_upload_non_base64_binary_content relies on it to keep raw file bytes intact. The body falls back to UTF-8 when it contains characters outside latin-1 (e.g. ş, €). Text fields and part headers are decoded as UTF-8, with a latin-1 fallback for bytes that came from the latin-1 round trip (e.g. café). If you'd rather switch fully to UTF-8 as suggested in the issue, I'm happy to change it, but that would change the bytes that non-base64 binary uploads get.

2. JSON arrays accepted for object/scalar body parameters

The single-element list unwrapping in _normalize_field_value is now only applied to form and multipart bodies (_request_body_to_args(..., is_form_body=...)). A JSON array sent to item: Item or Annotated[str, Body()] now returns 422. Query/header handling is unchanged.

Note for reviewers: this changes the JSON case of test_body_scalar_lists_keep_existing_normalization (added in #8509, where body normalization was intentionally left unchanged). I renamed it to test_body_scalar_lists_normalization and parametrized the expectation: JSON now returns 422, and form data still returns the first value.

3. filename= before name=

_extract_header_param now matches the parameter only at the start of a ;-separated segment, so looking up name no longer matches inside filename="...".

Tests

New functional tests cover the missing boundary (415), an invalid base64 body (422), non-ASCII text in non-base64 bodies (latin-1 and non-latin-1 text), UTF-8 text in base64 bodies, filename= before name=, and JSON arrays for model and scalar body parameters. 8 of the 9 new tests fail on develop; the base64 UTF-8 one already passed and guards that path.

tests/functional/event_handler and tests/unit/event_handler show the same failures before and after this change. The only failures locally come from optional dependencies missing in my environment (fastjsonschema). ruff and mypy are clean.

User experience

Before:

  • Multipart text field café sent without base64: UnicodeDecodeError → 502
  • Multipart without boundary: unhandled ValueError
  • [{"name": "a", "price": 1}, {"name": "b", "price": 2}] sent to def create(item: Item): 200 with only the first item
  • Content-Disposition: form-data; filename="a.txt"; name="file": 422 missing field

After:

  • café (and ş, €) is received correctly
  • Missing boundary: 415; other multipart parsing errors: 422
  • JSON array for an object or scalar parameter: 422
  • filename= before name= is parsed correctly

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

- Return 415 when multipart/form-data has no boundary, and 422 for other
  multipart parsing errors (invalid base64, undecodable text) instead of
  letting them escape as 500/502.
- Decode non-ASCII text fields in non-base64 multipart bodies correctly.
- Only unwrap single-element lists for form and multipart bodies, so a JSON
  array sent to an object or scalar body parameter returns 422.
- Parse Content-Disposition parameters so filename= before name= works.
@MTirnova
MTirnova requested a review from a team as a code owner October 18, 2026 19:34
@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 18, 2026
@leandrodamascena leandrodamascena added this to the 3.38.0 milestone Oct 19, 2026

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @MTirnova, this is careful work and almost all of it checks out: I ran the suites and deployed it behind a real REST API, and café, ş€, the missing boundary, bad base64, filename= before name=, the JSON array cases and list[Item] all behave. Binary uploads with binary media types configured come through byte for byte.

One case needs a guard though. Without binary media types, API Gateway decodes the body as UTF-8 and replaces invalid bytes with U+FFFD before we ever see it. On develop that blew up in encode("latin-1") with a 502, which was accidental but at least loud. With the UTF-8 fallback we now return 200 with a corrupted file (a 2 KB random upload came back as 3730 bytes with a different md5). Could you make the fallback raise a 422 multipart_invalid when the body contains "�", with a message pointing at binary media types? ş€ keeps working since it has no replacement character, and mangled binaries fail clearly instead of silently. Everything else can stay exactly as is.

Without binary media types, API Gateway decodes the body as UTF-8 and
replaces invalid bytes with U+FFFD. The UTF-8 fallback then passed a
corrupted file on with a 200. Return 422 multipart_invalid instead, with a
message pointing at Binary Media Types.
@MTirnova

Copy link
Copy Markdown
Contributor Author

Good catch, thanks for testing it behind a real REST API! The UTF-8 fallback now raises a 422 multipart_invalid when the body contains "\ufffd", with a message pointing at Binary Media Types. ş€ still goes through. I added a test with a JPEG header decoded the way API Gateway does it without binary media types. It returned 200 before and gets a 422 now.

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @MTirnova, quick turnaround again. I redeployed behind a real REST API without binary media types: the 2 KB binary upload now gets a clear 422 instead of a corrupted 200, and café and ş€ still come through. The JPEG header test mirrors what API Gateway actually sends, nice.

Approving now. This goes out in 3.38.0, so it will sit here a bit before I merge. No need to rebase or touch anything, I'll take care of it and you'll get the credit in the release notes. Thank you!

@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecov Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.93%. Comparing base (781f60e) to head (139649c).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #8626   +/-   ##
========================================
  Coverage    96.93%   96.93%           
========================================
  Files          318      318           
  Lines        16058    16067    +9     
  Branches      1430     1430           
========================================
+ Hits         15566    15575    +9     
  Misses         349      349           
  Partials       143      143           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@leandrodamascena
leandrodamascena merged commit ab83696 into aws-powertools:develop Oct 22, 2026
14 checks passed
@boring-cyborg

boring-cyborg Bot commented Oct 22, 2026

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request and thank you for helping improve everyone's experience!

@powertools-for-aws-oss-automation

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request and thank you for helping improve everyone's experience!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

event_handlers size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: body validation crashes on non-ASCII multipart data and accepts a JSON array for an object parameter

2 participants