Skip to content

perf(parser): load event models on demand - #8519

Open
leandrodamascena wants to merge 1 commit into
developfrom
fix/parser-models-lazy-imports-8515
Open

leandrodamascena wants to merge 1 commit into
developfrom
fix/parser-models-lazy-imports-8515

Conversation

@leandrodamascena

@leandrodamascena leandrodamascena commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Closes #8515

Summary

Changes

Importing a single parser model currently loads all 27 model modules and builds schemas for unrelated event types. Resolve model exports and submodule attributes on first access, then cache the original objects.

Retains the existing __all__, imports for static type checking, and attribute discovery through dir(). Adds tests in fresh Python processes for import paths, model dependencies, public exports, class identity, serialization, and concurrent access.

User experience

Existing imports such as from aws_lambda_powertools.utilities.parser.models import SqsModel keep their syntax and validation behavior while loading only the requested module and its dependencies.

Measured before and after this change:

Import Median before Median after Parser model modules before/after
SqsModel from parser.models 294 ms 14 ms 27 / 1
PartialItemFailureResponse with Pydantic already loaded 395 ms 190 ms 27 / 5

Each case ran in 20 fresh interpreters in the official Lambda Python 3.13 ARM64 image, with 512 MiB and 0.29 CPU. Both source copies were precompiled and used the same dependencies. Pydantic and an application model were initialized before timing. These are Docker import measurements, not deployed Lambda cold-start measurements; timings vary under CPU throttling.

Compatibility

Direct namespace inspection now sees only resolved exports. Code that reads an unloaded model from vars(models) or models.__dict__, or supplies that dictionary to Pydantic for forward-reference resolution, can fail until the model is imported. inspect.getmembers_static() is also affected. Explicit imports and getattr(models, name) resolve the model normally.

This PR addresses the parser model initializer. The separate Batch, data class, and Tracer changes in #8515 remain follow-up work.

Validation

  • 24 new subprocess scenarios; 21 reproduce unnecessary imports on the base commit.
  • Parser and Batch suites: 277 passed on each of Python 3.10 and 3.14.
  • parser.models: 100% line and branch coverage on Python 3.14.
  • Compared all 109 distinct public exports and 27 exposed submodules against the base.
  • Ruff, mypy, Bandit, Xenon, and applicable pre-commit hooks passed.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

@leandrodamascena
leandrodamascena requested a review from a team as a code owner October 5, 2026 14:48
@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Oct 5, 2026
@boring-cyborg boring-cyborg Bot added the tests label Oct 5, 2026
@sonarqubecloud

sonarqubecloud Bot commented Oct 5, 2026

Copy link
Copy Markdown

@codecov

codecov Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.92%. Comparing base (556478f) to head (e21b920).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #8519      +/-   ##
===========================================
- Coverage    96.92%   96.92%   -0.01%     
===========================================
  Files          317      317              
  Lines        15911    15899      -12     
  Branches      1398     1400       +2     
===========================================
- Hits         15422    15410      -12     
  Misses         353      353              
  Partials       136      136              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tech debt: eager imports in parser.models, data_classes and batch add 200–400 ms to cold starts

1 participant