Repository navigation
refactor(protocols)!: group each protocol's protocol, schema and data into one package #2534
Description
Owner ruling on #2529 (2026-11-01), paraphrased: replace the three parallel trees (protocols/<layer>/<proto>.py, protocols/schema/<layer>/<proto>.py, protocols/data/<layer>/<proto>.py) with one package per protocol, protocols/<layer>/<proto>/{protocol,schema,data}.py.
Settled design:
- The package takes the protocol's name; its modules take their role; each module holds the class named after the package (refactor(protocols): a protocol class takes its module's name (igmpv1.IGMPv1, httpv1.HTTPv1) #2529's rule, restated per package).
- Shared pieces of a family (IGMP v1/v2 share one schema and data model) live in the family base's package, and members import them.
- Framework pieces stay in a core location at the top level:
protocols.protocol,protocols.schemaandprotocols.data. - No re-exports: the old paths and the
protocols.schema/protocols.dataaggregations go, under refactor(corekit)!: move the byte-order field helpers into corekit (#1711) #1715/docs: #719-style prose sweep after the #1090–#1102 defect wave #1103. Breaking, with!andversionremovedwhere a name shipped. misc/null.pybecomesno_payload(owner, refactor(protocols): a protocol class takes its module's name (igmpv1.IGMPv1, httpv1.HTTPv1) #2529 Q1). It rides along here, because the codemod rewrites the same import paths in 64 files.- Delivered as one codemod change after the current wave lands, piloted on one family first to measure import cycles and the docs build.
Pilot: the VLAN family (vlan, c_tag, s_tag), dispatched now as a local experiment, not a PR.
Blocked: the full move waits for the open PRs that touch pcapkit/protocols (#2528 #2527 #2523 #2522 #2521 #2519 #2509 #2483) and the streams in flight. It unblocks when those have merged or closed.
Activity
JarryShaw commentedon Nov 1, 2026
Pilot done on the VLAN family (local
f52b9808d, not for push). After the codemod and four kinds of hand edit, all 588 modules import cleanly, 36 test modules pass, and a nitpicky docs build adds no warnings. A dry run over the whole tree touches 56 packages, makes 142 file moves, rewrites 699 files and about 10k dotted references. The codemod is kept, ready for the real run. Decisions needed first:protocol.pyimports Raw's and NoPayload's schema/data at the top, and in the new layout those cycle back throughmiscintoprotocol.py. Options: (a) defer those imports; (b) keep Raw/NoPayload at the top level as framework; (c) make the layer__init__files lazy. Recommendation: (a), measured in the full run.schema=/data=fallback in__init_subclass__reads the aggregators that are going away. Only L2TP relies on it. Options: look in the class's own package for siblingschema/datamodules, or make the keywords mandatory. Recommendation: sibling lookup.__init__export its class, so thatprotocols.link.vlan.VLANstill resolves? refactor!: remove every surviving re-export of a moved name or module #1744's D3 allows a package__init__to aggregate its own submodules. Recommendation: yes, which also makes most imports non-breaking.ipv6_optionschema/data, used by HOPOPT and IPv6_Opts, goes into the family base's package:ipv6_ext'sschema/data, per your ruling 1. Recommendation: that.schema/schema.py→protocols/schema.py,data/data.pyanddata/protocol.py→protocols/data.py, and the layer bases stay at<layer>/<layer>.py. Recommendation: as listed.~cross-reference targets, which leaves the rendered text unchanged and keeps the links resolving. Recommendation: yes... module::, rather than a directory per package. Recommendation: keep the URLs.JarryShaw commentedon Nov 1, 2026
JarryShaw commentedon Nov 1, 2026
Owner rulings on the pilot, paraphrased:
protocol.pydefers its Raw/NoPayload schema and data imports.__init_subclass__findsschema/datain the class's own package, and an explicitschema=/data=class keyword still overrides it.__init__exports its class, soprotocols.link.vlan.VLANstill resolves.ipv6_optionschema and data go intoipv6_ext's package.protocols/schema.pyandprotocols/data.py; layer bases stay at<layer>/<layer>.py.~targets rewritten.Back to
blocked: the codemod runs once no open PR or in-flight stream touchespcapkit/protocols.JarryShaw commentedon Nov 2, 2026
A worker is bringing the pilot's codemod up to date for every family and dry-running it on current
main(all test legs and the docs build), so the final run needs no hand edits. The PR still waits for a tree where no open PR editspcapkit/protocols/, after #2594. Moved fromblockedtowip.3 remaining items
JarryShaw commentedon Nov 2, 2026
JarryShaw commentedon Nov 2, 2026
The owner took the recommendation on all three, paraphrased:
sslpackage, where a family's shared pieces live. That breaks thessl.protocol→tls.datacycle with no lazy export.~) references to moved paths are rewritten too, so the docs build stays clean.__init__exports its class only. Other shipped names live at their new module paths, withversionremovedat the old ones.The codemod worker updates
codemod-v2.pyand re-runs the dry run. The final run still waits for a quiet protocols tree, after #2692. This settles the questions, soneeds: decisioncomes off.JarryShaw commentedon Nov 3, 2026
The codemod is updated for all three answers and dry-run again, committed locally as
4bf328a45. No hand edits remain.ssl/data.py.tls/data.pykeepsTLS, and every package now imports eagerly, with no cycle. None of these shipped, so they get noversionremoved. A layout test fails if a base ever reads a member's schema or data module.~or not. This adds no new docs warnings outside the unfolded 1.6.0 lines.Every leg passes, apart from the CI-script failures
mainhas too. The final run takes about 50–60 minutes. It dry-runs clean on currentmainand on #2682, #2691 and #2692. #2704 needs a one-line fix-up to its new test once it lands. The run itself waits until #2692 and the other protocol PRs have merged.JarryShaw commentedon Nov 3, 2026
The serial window is open. #2483 is the only open PR, and it touches only the changelog. The final codemod run is underway on main
f45c82b5d, with no hand edits and the three rulings above in force. After it lands: #2490 (member reorder), then #2730 (lazy protocol packages).JarryShaw commentedon Nov 3, 2026
The PR is #2753 (
e9de6cc5e), the final codemod run: one command, no hand edits. A cross-review is running.refactor(protocols)!: group each protocol into one package (#2534)
refactor(protocols)!: group each protocol into one package (#2534)
refactor(protocols)!: group each protocol into one package (#2534)
refactor(protocols)!: group each protocol's protocol, schema and data…