Repository navigation
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## v2-architecture-import-linter #4191 +/- ##
================================================================
Coverage ? 88.29%
================================================================
Files ? 293
Lines ? 17510
Branches ? 2876
================================================================
Hits ? 15460
Misses ? 1337
Partials ? 713 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
scripts/rules/tests/test_v2_layers.py specifies a quality guard over every .py file under services/v2 and fastapi_routes/v2: - each file is assigned a layer by its name or by rules/v2-layer-manifest.toml, and an unassigned file fails; - route modules import no SQLAlchemy, SQLModel, connector, session or v2 database module and call no SQL builder or session method; - services build no SQL, call no data-access session method and import no FastAPI or Starlette; - connectors don't open, commit, roll back or close a session. Calls count only on a receiver named like a session (session.add, not set.add), and SQL builders only when imported from SQLAlchemy or SQLModel. Imports under TYPE_CHECKING count. Proof fixture trees for the four rules include a route that imports a connector and a file with no layer. These fail until scripts/rules/v2_layers.py exists. Refs #4190 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add the v2-layers quality guard (scripts/rules/v2_layers.py, standard library only), run by scripts/run_quality_guards.py in the quality-guards job. - It discovers every .py file under services/v2 and fastapi_routes/v2 and takes its layer from its file name, or from rules/v2-layer-manifest.toml for the 9 files with other names. A file with neither fails: new rule api.v2.files-assigned-to-layers. - api.v2.routes-no-sessions-or-sql is ratcheted. Its baseline lists the 4 route modules that import sqlalchemy.exc to catch SQLAlchemyError. - api.v2.services-no-sql and api.v2.connectors-never-commit are enforced, with no findings today. - Session calls count only on a receiver named like a session, and SQL builders and session factories only when imported from SQLAlchemy, SQLModel or the v2 database module. Imports under TYPE_CHECKING count. Each rule has proof fixture trees, and v2-code-organization.md cites each rule id. The test fix resolves "from .. import database_session" to the resource's own session module, as Python does. Fixes #4190 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Validators, transformations, types, process-state modules and package __init__ files touch no session: new rule api.v2.pure-modules-no-session-calls, with proof fixtures. A database_connectors/__init__.py counts as a connector. - The repository's own session factories count: the v1 ones in policyengine_api.data.orm, build_v2_session_factory, the engines, and a database_session container's transaction() in a route. - Services import nothing from SQLAlchemy or SQLModel but the Session type and sqlalchemy.exc, so and_, or_, func and col count, as does a select() on a model's __table__. - A name that only contains "session" (sessions_router, session_ids, request.session) is not a session. - Each occurrence of a finding needs its own baseline entry, and a duplicated entry is stale. - The manifest gives only the layers of the file's directory, never "package", and every entry needs a reason. Refs #4190 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New rule api.v2.pure-modules-no-session-calls: validators, transformations, types, process-state modules and package __init__ files call no session method, SQL builder or session factory. A database_connectors/__init__.py is checked as a connector. - Session factories are any call into policyengine_api.data.orm or policyengine_api.data.v2.database, or SQLAlchemy's Session, sessionmaker and create_engine. Routes and connectors may not import the v1 ORM module, and a route may not call a database_session container's read() or transaction(). - Services import nothing from SQLAlchemy or SQLModel but the Session type and sqlalchemy.exc, so and_, or_, func and col are caught, and a builder called on a model's __table__ counts. - A session receiver is named session or <name>_session, and request.session and http_session are not sessions, so sessions_router and session_ids are not flagged. - Each occurrence of a finding needs its own baseline line, and a duplicated line is reported as no longer applying. - The manifest may give only the layers of the file's directory, and every entry needs a reason. - v2-code-organization.md has a "File layers" section, which the files rule's rationale now links to. The real code still has exactly the 4 tolerated sqlalchemy.exc imports. Refs #4190 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #4190 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Routes and the pure layers may call no session method at all, so get_one, bulk_save_objects, connection and expunge count; services' and connectors' lists gain get_one, the bulk methods, reset and invalidate. - A manifest file in a pure or HTTP layer gets that layer's import and call checks, including session factories. - A database_session.py below a resource package has no layer. - Every call into SQLAlchemy or SQLModel but its exception classes counts, so "import sqlalchemy.exc" doesn't open sqlalchemy.and_. - request.session on any request object, and client or requests sessions, are not database sessions. - Only a builder called on <Model>.__table__ counts, once. Refs #4190 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Routes and the pure layers may call no method on a session, so a method missing from a list can't slip through. Services' and connectors' lists gain get_one, the bulk methods, connection, expunge, reset and invalidate. - The pure layers also get import checks (nothing from SQLAlchemy or SQLModel but the Session type; no connector, session, v1 ORM or v2 database module), so a file the manifest assigns to one is checked as fully as a named one. - A database_session.py below a resource package has no layer, so it can't escape the session-module rule. - Every call into SQLAlchemy or SQLModel counts in routes, services and the pure layers, except to its exception classes, so "import sqlalchemy.exc" no longer opens sqlalchemy.and_. - A request's .session and client or requests sessions are not database sessions; only <Model>.__table__.<builder>() counts as a table builder. - The pure rule's scope leaves out the connector package __init__ files, which are checked as connectors. The docs list what the guard recognises and what it doesn't. Refs #4190 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
force-pushed
the
v2-architecture-layer-check
branch
from
November 4, 2026 16:57
80e0f9d to
d18175c
Compare
- An injected session factory opens a session: session_factory.begin() and self._session_factory() count in connectors and routes, and so does a database_session container's transaction() in a connector. - New rule api.v2.session-modules-no-data-access: a database_session module opens, commits, rolls back and closes sessions, but adds, gets, merges or executes nothing and builds no SQL. Proof fixtures included. - A request's .session is recognised by its owner (req, websocket, *_request), so request_context.session and request_session are database sessions. Refs #4190 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Injected session factories open sessions: session_factory(), self._session_factory() and session_factory.begin() count in routes, connectors and the pure layers, and a connector may not call a database_session container's read() or transaction(). - New rule api.v2.session-modules-no-data-access: database_session modules call no data-access session method or SQL builder and import no connector. Today's modules only open, commit, roll back and close. - A request's .session is recognised by its owner (req, request, websocket, ws, *_request), so request_context.session and request_session are database sessions, and requests_session is not. - The docs say services may open sessions, which their rule doesn't forbid. The real code still has exactly the 4 tolerated sqlalchemy.exc imports. Refs #4190 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4190
Summary
Adds the
v2-layersquality guard, which gives every API v2 route and service file a layer and checks each layer's imports and calls. It turns threedoc-onlyv2 rules into checked ones and adds three rules. Step 2.6 of the agent-proof code-quality plan (its AST part). Stacked on #4184 (Import Linter), which builds on #4179 (merged).What the guard checks
scripts/rules/v2_layers.pyuses the standard library only, like the rest of the rules tool.scripts/run_quality_guards.pyruns it in thequality-guardsjob ofpr.yml.api.v2.files-assigned-to-layers(new).pyfile underpolicyengine_api/services/v2andpolicyengine_api/fastapi_routes/v2gets a layer from its file name or fromrules/v2-layer-manifest.toml. A file with neither fails.api.v2.routes-no-sessions-or-sql(wasdoc-only)fastapi_routes/v2imports SQLAlchemy, SQLModel, a connector, adatabase_sessionmodule, the v1 ORM modulepolicyengine_api.data.ormorpolicyengine_api.data.v2.database. None calls a SQLAlchemy or SQLModel function (except its exception classes), any method on a session (including adatabase_sessioncontainer'sread()ortransaction()) or a session factory.api.v2.services-no-sql(wasdoc-only)services.pymodules import nothing from SQLAlchemy or SQLModel but theSessiontype (for annotations) andsqlalchemy.exc(to raise typed failures). They call no SQLAlchemy or SQLModel function but its exception classes and session factories, which catchesand_,or_,funcandcoleven throughimport sqlalchemy.exc. Services may open sessions, which the rule doesn't forbid. They also call no data-access session method (exec,execute,add,get,get_one, the bulk methods,connection,deleteand so on), and import no FastAPI or Starlette.api.v2.connectors-never-commit(wasdoc-only)database_connectors/__init__.py, don't callcommit,rollback,close,begin,begin_nested,resetorinvalidateon a session, open none (no session factory, injectedsession_factory, ordatabase_sessioncontainerread()/transaction()), and import no v1 ORM, v2 database ordatabase_sessionmodule.api.v2.session-modules-no-data-access(new)database_session.pymodules sit directly in their resource package and call no data-access session method (add,get,merge,exec,executeand so on) or SQL builder, and import no connector. They open, commit, roll back and close sessions.api.v2.pure-modules-no-session-calls(new)__init__files:Sessiontype;The fourth
doc-onlyrule,api.v2.connectors-no-cross-calls, became ratcheted in #4184 through Import Linter's independence contract.Layers by file name:
services/v2/<resource>/:services.py,validators.py,transformations.py,types.pyanddatabase_session.py. Adatabase_session.pymust sit directly in its resource package, whereapi.v2.session-modules-no-queries(pytest) also checks it.services/v2/<resource>/database_connectors/:<creates|reads|updates|deletes>[_name].py.fastapi_routes/v2/:routes.pyand<name>_routes.py, plusrequest_models.pyandresponse_models.py.__init__.pyanywhere.Manifest (9 files with other names). It lives under the code-owned
rules/directory, and each entry gives a layer and a reason:http-support:fastapi_routes/v2/errors.py,user_auth.py,user_resources.py, andmetadata/{common,query_dependencies,response_cache,search_file_response}.py;http-model:households/document_models.py;process-state:services/v2/metadata/selected_catalogs.py.The manifest may give a file under
services/v2onlyservice,validator,transformation,typesorprocess-state, and a file underfastapi_routes/v2onlyroute,http-modelorhttp-support, so an entry can't switch a file's checks off: each of those layers has the full checks of its layer. Every entry needs a reason. The guard also fails on an entry for a missing file, or for a file whose name already gives it a layer.Attribute-aware calls.
sessionor<name>_session. The.sessionof a request or websocket (Starlette's session data:request,req,websocket,wsor*_request, also asself.request) and HTTP or client sessions (http_session,requests_session,_client_session) don't count;request_sessionandrequest_context.sessiondo. Sosession.add,self._session.execanddatabase_session.transaction()count;set.add,cache.get,sessions_router.getandsession_ids.adddon't. Routes and the pure layers may call no method on a session at all, so a method left off a list can't slip through there.<Model>.__table__.<builder>(). Connectors may build SQL. A session factory counts for any call intopolicyengine_api.data.ormorpolicyengine_api.data.v2.database(get_v1_session_factory,build_v2_session_factory, the engines, …), for SQLAlchemy'sSession,sessionmakerandcreate_engine, and for an injected*session_factory(session_factory(),self._session_factory(),session_factory.begin()). A local helper namedselectdoesn't count.from package import submoduleare resolved. Imports made only underTYPE_CHECKINGcount (decided 2026-11-04).Tolerated (baseline
rules/baselines/v2-layers-routes.txt, 4 entries, one per occurrence).countries/routes.py,households/routes.py,policies/routes.pyanduser_resources.pyimportsqlalchemy.excto catchSQLAlchemyError. I counted these as violations rather than allowing exception imports in routes; allowing them would mean changing the rule's statement. The fix is for the services to raise typed failures. A new finding fails, and so does a baseline entry that no longer occurs.Limits.
sessionin it isn't recognised.TABLE = Policy.__table__) isn't recognised.Import Linter (#4184) covers the import side transitively.
Open pull requests that add v2 files
I scanned each open PR's head with this guard. Once this merges, each PR must give every new file under
services/v2orfastapi_routes/v2a layer, by its name or a manifest entry. The baseline can't grow, so each PR must also clear its own new findings.master, 14 not counting__init__.py): every file has a layer.fastapi_routes/v2/reports/routes.pyimportssqlalchemy.exc, which failsapi.v2.routes-no-sessions-or-sql: the service should raise typed failures for the route to map.master, 24 not counting__init__.py):fastapi_routes/v2/reports/household_response_models.pymatches no layer name. Rename it (for example intoresponse_models.py) or add it torules/v2-layer-manifest.tomlashttp-model.reports/routes.pyfinding.__init__.py):services/v2/household_calculations/simulation_client.pymatches no layer name. Give it a manifest entry with a layer and a reason (its checks then follow that layer), or move the client out ofservices/v2.fastapi_routes/v2/household_calculations/routes.pyimportssqlalchemy.exc, the same finding as Serve economy reports at /v2/reports and save v2 report and simulation IDs #4108's.No other open PR adds files under these directories.
Docs and registry
v2-code-organization.mdhas a new "File layers" section on the assignment and how calls are recognised. It cites each rule id where it states the rule.docs/generated/rules.mdandAGENTS.mdare regenerated.rules/fixtures/<rule id>/, including a route that imports a connector and a file with no layer (shaped after Calculate stored households through /v2/household-calculations on the v2 household functions #4092'ssimulation_client.py).CI and environment variables
pythonand no project dependencies. Its proofs run in the existing "Test the rules registry tool" step.git ls-files, through the file-length guard's helper, to list tracked files.Testing
Tests first, in 4ba6e71, which failed because
scripts/rules/v2_layers.pydidn't exist yet. The review-round tests failed before their fixes: 5328792 before 834b127, d0a49ed before d18175c, and f7a58d2 before 5af955d.scripts/rules/tests/test_v2_layers.py(36 tests):expected_findingsand each good tree none;set.addnot counted);TYPE_CHECKINGcounted;addandflushbut not end a transaction;services/v2has its calls checked, and a connector package__init__counts as a connector;import sqlalchemy.exc;__table__builders;from .. import database_session(not...) names the resource's session module.imports sqlalchemyforimport sqlalchemy as sa.sqlmodel.selectimport. (The services test'sand_,or_,func.lowerandcolcall findings were added in the failing commit d0a49ed.)After the change:
python scripts/run_quality_guards.pypasses, includingv2-layers;python -m scripts.rules guard --base-ref origin/masterpasses;ruff format --check .,ruff check .andmypypass.🤖 Generated with Claude Code