Skip to content

test: release purged pcapkit generations in a long unittest leg (#2750) - #2752

Merged
JarryShaw merged 1 commit into
mainfrom
test/2750-release-purged-generations
Nov 3, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
test/2750-release-purged-generations

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • test — tests only

Description of your pull request and other information

In a long plain-unittest leg, purged copies of pcapkit stayed reachable and piled up: 13 live generations by the end of protocols (rest). Every negative issubclass(C, Mapping) walks all of them, so modules ran slower the later they came.

  • typing's subscription caches (Protocol[...], Optional[...] from get_type_hints) held the purged classes. purge_modules and restore_modules now run typing._cleanups, as CPython's own test runner does, and skip it where it is absent.
  • PCAPNGTruncatedFileTests._swept kept its failures' tracebacks for the rest of the process. A class cleanup now drops it.
  • New TypingCachesReleasedTests fails without the _support.py change.

Same leg, base vs fixed: peak RSS 1730 → 1016 MiB, elapsed 1319 → 828 s, end-of-leg generations 13 → 5. Nothing in pcapkit/ changes.

Closes #2750

- typing caches every alias it subscribes, Protocol[...] at class creation
  and Optional[...] from get_type_hints on an Info annotation, and no purge
  cleared it. Each cached alias kept a purged generation reachable, and its
  saved excepthook another, so live generations, the heap and the cost of a
  negative Mapping subclass check grew from module to module. purge_modules
  and restore_modules now run typing._cleanups beside _reset_abc_caches;
  TypingCachesReleasedTests pins both and fails without the change.
- PCAPNGTruncatedFileTests kept its sweep, whose failures hold their
  tracebacks and so the class's import, for the rest of the process. It is
  dropped by a class cleanup.

`util/run_unittest_leg.py protocols --exclude protocols/internet`: peak RSS
1730 -> 1016 MiB, 1319 -> 828 s, 13 -> 5 live generations at the end; the
touched modules and the tests/project guards pass under plain unittest.
@JarryShaw JarryShaw added test Pull requests that add or correct tests (test: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Nov 3, 2026
@JarryShaw JarryShaw moved this to In review in PyPCAPKit Nov 3, 2026
@github-actions github-actions Bot added this to the PyPCAPKit v1.6 milestone Nov 3, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO at f93af82ef, from a Sonnet review.

  • typing._cleanups holds only typing's own lru_cache.cache_clears, which is what CPython's regrtest clears between modules. It exists on CPython 3.7, 3.8 and 3.14 and on PyPy 3.10. The getattr(..., ()) makes a runtime without it a no-op. No test relies on alias identity, and re-created aliases compare equal.
  • addClassCleanup is on the 3.8 floor and runs under unittest and pytest. _swept is recomputed when it is None.
  • TypingCachesReleasedTests fails with only the _support.py hunk reverted, and it checks that the alias was cached, so it cannot pass vacuously.
  • Half 1 of the leg: 419 s, peak RSS 814 MiB on a loaded host. Project tests and Lint pass. The 3.15 job's time decides ci(unit-tests): the Python 3.15 job's p90 is past half its cap (CI timing HARD H1) #2751.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Nov 3, 2026
@JarryShaw
JarryShaw merged commit 05e1137 into main Nov 3, 2026
33 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Nov 3, 2026
@JarryShaw
JarryShaw deleted the test/2750-release-purged-generations branch November 3, 2026 21:02
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Nov 3, 2026
JarryShaw added a commit that referenced this pull request Nov 4, 2026
- Breaking: each protocol is a package of protocol, schema and data modules,
  the pcapkit.protocols.schema and .data trees and their aggregate exports
  removed with no alias, and a 1.5.x data model pickle fails to load (#2753),
  a new entry with the 1.5.1 imports and pickle measured.
- 1.6.0's pcapkit.protocols role targets point at the package modules; the
  aggregator export claims of unreleased entries, the NoPayload and FrameRecord
  pickle claims and the TLSKeyLogEntries alias follow the layout.
- CI: the LATEST tier's own job and the Coverage merge (#2754) and the
  ordering leg's halved rest of tests/protocols (21ec0a6, #2751), into the
  CI-legs entry; the package ruling into the conventions entry.
- No entry for the test-only #2752.
- The merge keeps #2483's class names in 1.5.0's conflicting roles under
  #2753's package paths.
- CHANGELOG.md regenerated; util/changelog_md.py --check passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test Pull requests that add or correct tests (test: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

test: a long single-process unittest leg slows as it runs (heap growth across modules)

1 participant