Assign every v2 route and service file to a layer and enforce the v2 layer rules

AGENTS.md

@@ -87,11 +87,8 @@ To add or change a rule, read `docs/engineering/skills/rules-registry.md`.
 
 ### API v2 layer boundaries
 
-- Checked: `api.v2.connectors-mutations-no-reads`, `api.v2.connectors-no-cross-calls`, `api.v2.connectors-reads-no-mutations`, `api.v2.crud-checks-cover-every-resource`, `api.v2.import-layers`, `api.v2.pure-modules-no-database`, `api.v2.routes-import-no-database-access`, `api.v2.session-modules-no-queries`, `api.v2.types-transformations-no-validators`, `api.v2.user-services-no-sql`, `api.v2.validators-transformations-no-db`
+- Checked: `api.v2.connectors-mutations-no-reads`, `api.v2.connectors-never-commit`, `api.v2.connectors-no-cross-calls`, `api.v2.connectors-reads-no-mutations`, `api.v2.crud-checks-cover-every-resource`, `api.v2.files-assigned-to-layers`, `api.v2.import-layers`, `api.v2.pure-modules-no-database`, `api.v2.pure-modules-no-session-calls`, `api.v2.routes-import-no-database-access`, `api.v2.routes-no-sessions-or-sql`, `api.v2.services-no-sql`, `api.v2.session-modules-no-data-access`, `api.v2.session-modules-no-queries`, `api.v2.types-transformations-no-validators`, `api.v2.user-services-no-sql`, `api.v2.validators-transformations-no-db`
 - `api.routes.shared-query-parameters`: Routes that accept query parameters define them through the shared typed query models, never with a second manual parser. (doc-only, not checked: follow it yourself)
-- `api.v2.connectors-never-commit`: v2 connectors receive a session from the service and never open, commit, roll back or close it. (doc-only, not checked: follow it yourself)
-- `api.v2.routes-no-sessions-or-sql`: v2 route modules never open database sessions or construct SQL; they call one service method and map typed failures to HTTP. (doc-only, not checked: follow it yourself)
-- `api.v2.services-no-sql`: v2 services sequence work but never construct SQL expressions, call Session.exec, or define HTTP response models. (doc-only, not checked: follow it yourself)
 
 ### Runtime persistence and route boundaries
 

changelog.d/4190.added.md

@@ -0,0 +1 @@
+Check that every API v2 route and service file has a layer, and that v2 route modules open no session and build no SQL, services construct no SQL, connectors never open, commit, roll back or close a session, and validators, transformations and types make no session calls, in a new v2-layers quality guard.

docs/engineering/skills/v2-code-organization.md

@@ -1,8 +1,41 @@
 # API v2 Code Organization
 
 Organize API v2 code first by resource and then by one explicit technical
-responsibility. Route modules must not open database sessions or construct SQL.
-Service modules must sequence work but must not construct or execute SQL.
+responsibility. Route modules must not open database sessions or construct SQL
+(rule `api.v2.routes-no-sessions-or-sql`). Service modules must sequence work
+but must not construct or execute SQL (rule `api.v2.services-no-sql`).
+
+
+## File layers
+
+Every `.py` file under `policyengine_api/services/v2` and
+`policyengine_api/fastapi_routes/v2` has a layer, which the `v2-layers` quality
+guard (`scripts/rules/v2_layers.py`) takes from the file's name: `services.py`,
+`validators.py`, `transformations.py`, `types.py`, `database_session.py`,
+`database_connectors/<creates|reads|updates|deletes>[_name].py`, `routes.py`,
+`<name>_routes.py`, `request_models.py`, `response_models.py` or `__init__.py`.
+A file with another name needs an entry, with its layer and a reason, in
+`rules/v2-layer-manifest.toml`, or the guard fails (rule
+`api.v2.files-assigned-to-layers`). The manifest can give a file under
+`services/v2` the layer `service`, `validator`, `transformation`, `types` or
+`process-state`, and a file under `fastapi_routes/v2` the layer `route`,
+`http-model` or `http-support`; each layer follows that layer's rules below.
+A `database_session.py` sits directly in its resource package.
+
+The guard counts a method call as a session call when its receiver is named
+`session` or `<something>_session`, but not a request's `.session` or an HTTP
+or client session (`http_session`, `requests_session`). Route modules and the
+pure layers may call no method on a session at all. In routes, services and
+the pure layers a call into SQLAlchemy or SQLModel counts, except to its
+exception classes and, in services, to a session factory; a builder called on
+a model's `__table__` counts too. A session factory is any call into
+`policyengine_api.data.orm` or `policyengine_api.data.v2.database`,
+SQLAlchemy's `Session`, `sessionmaker` or `create_engine`, or an injected
+`*session_factory` or its `begin()`; only services and `database_session`
+modules may use one. The guard doesn't
+recognise a session bound to another name, an alias of a model's table, or a
+name re-exported through another module. A ratcheted rule's baseline lists one
+line per tolerated occurrence.
 
 ## HTTP adapters
 
@@ -40,7 +73,13 @@ individual resource paths.
 
 Request models describe HTTP bodies. Response models describe public response
 response structures and OpenAPI output. Route functions handle HTTP-only conditions,
-invoke one service method, and convert typed failures to HTTP responses.
+invoke one service method, and convert typed failures to HTTP responses. No module
+under `fastapi_routes/v2` imports SQLAlchemy, SQLModel, a connector, a
+`database_session` module, the v1 ORM module `policyengine_api.data.orm` or
+`policyengine_api.data.v2.database`, or calls a SQLAlchemy or SQLModel
+function, a session method or a session factory (rule
+`api.v2.routes-no-sessions-or-sql`; today's `sqlalchemy.exc` imports are in its
+baseline).
 
 ## Resource service packages
 
@@ -86,14 +125,20 @@ or more connector functions. A service may expose an entrypoint that accepts an
 existing session when several resources must change atomically in one caller-
 owned transaction.
 
-Do not construct SQL expressions, call `Session.exec`, or define HTTP response
-models in this module.
+Do not construct SQL expressions, call `Session.exec` or another data-access
+session method, or define HTTP response models in this module (rule
+`api.v2.services-no-sql`, which also forbids importing FastAPI or Starlette).
 
 ### `validators.py`
 
 Define functions that inspect already-available values and either return a
 validated value or raise a typed exception. Validators must not load records,
 open sessions, execute SQL, mutate database rows, or construct HTTP responses.
+Validators, transformations, types, process-state modules and package
+`__init__` files import nothing from SQLAlchemy or SQLModel but the `Session`
+type, import no connector or session module, and call no session method,
+SQLAlchemy or SQLModel function or session factory (rule
+`api.v2.pure-modules-no-session-calls`).
 
 When validation depends on stored state, a database connector loads the
 required rows and the service passes those rows to a validator. For example,
@@ -129,7 +174,9 @@ baseline).
 Define the resource's database-session lifetime container. It may open and
 close sessions, begin transactions, commit, or roll back. It must not construct
 SQL, choose records, validate business rules, transform records into response
-types, or determine multi-step operation order.
+types, or determine multi-step operation order. It calls no data-access
+session method (`add`, `get`, `merge`, `exec`, `execute` and so on) and
+imports no connector (rule `api.v2.session-modules-no-data-access`).
 
 Composition code should construct this container and inject it into the
 service. A request-scoped read service may wrap one already-open session; a
@@ -147,7 +194,7 @@ ORM rows belongs in this package. Organize connector files by CRUD operation:
 - `deletes.py` removes rows.
 
 Connector functions receive a session from the service. They do not open,
-commit, roll back, or close it. They do not call route functions, perform
+commit, roll back, or close it (rule `api.v2.connectors-never-commit`). They do not call route functions, perform
 request-level or business validation, or convert rows into public response
 types. Connector modules do not call one another to sequence a workflow; the
 service makes that ordering explicit. They never import one another (rule

docs/generated/rules.md

@@ -8,11 +8,11 @@ Every rule that governs `policyengine-api` is listed here with how it is enforce
 
 | Tier | enforced | ratcheted | advisory | doc-only | Total |
 | --- | ---: | ---: | ---: | ---: | ---: |
-| mechanical | 63 | 3 | 1 | 11 | 78 |
+| mechanical | 68 | 4 | 1 | 8 | 81 |
 | metric | 1 | 0 | 0 | 0 | 1 |
 | judgment | 0 | 0 | 0 | 1 | 1 |
 
-**Blocking:** 67 of 80 rules.
+**Blocking:** 73 of 83 rules.
 
 ## Gaps
 
@@ -27,9 +27,6 @@ Every rule that governs `policyengine-api` is listed here with how it is enforce
 - `api.process.makefile-phony`: Command-only Makefile targets are declared in .PHONY, and .PHONY stays in sync when targets are renamed or removed.
 - `api.process.rule-machinery-code-owned`: Changes to the rule machinery (rules/, scripts/rules/, scripts/guards/, scripts/run_quality_guards.py and CODEOWNERS) need approval from a core developer other than the author.
 - `api.routes.shared-query-parameters`: Routes that accept query parameters define them through the shared typed query models, never with a second manual parser.
-- `api.v2.connectors-never-commit`: v2 connectors receive a session from the service and never open, commit, roll back or close it.
-- `api.v2.routes-no-sessions-or-sql`: v2 route modules never open database sessions or construct SQL; they call one service method and map typed failures to HTTP.
-- `api.v2.services-no-sql`: v2 services sequence work but never construct SQL expressions, call Session.exec, or define HTTP response models.
 
 ### Advisory (checked but not blocking)
 
@@ -66,6 +63,7 @@ Every rule that governs `policyengine-api` is listed here with how it is enforce
 
 - `api.v2.connectors-no-cross-calls`: 7 remaining in `rules/baselines/import-linter-api-v2-connectors-independent.txt`
 - `api.v2.pure-modules-no-database`: 1 remaining in `rules/baselines/import-linter-api-v2-pure-modules-no-database.txt`
+- `api.v2.routes-no-sessions-or-sql`: 4 remaining in `rules/baselines/v2-layers-routes.txt`
 - `api.v2.types-transformations-no-validators`: 4 remaining in `rules/baselines/import-linter-api-v2-types-transformations-no-validators.txt`
 
 ### Exceptions and review dates
@@ -84,15 +82,18 @@ Every rule that governs `policyengine-api` is listed here with how it is enforce
 | --- | --- | --- | --- | --- | --- | --- | --- |
 | `api.routes.shared-query-parameters` | Routes that accept query parameters define them through the shared typed query models, never with a second manual parser. | mechanical | doc-only | no | nothing | — | — |
 | `api.v2.connectors-mutations-no-reads` | v2 mutation connectors (creates, updates, deletes) never issue select or get reads. | mechanical | enforced | yes | pytest `tests/unit/v2/test_data_crud_boundaries.py` | `pr.yml:test` | **unproven** |
-| `api.v2.connectors-never-commit` | v2 connectors receive a session from the service and never open, commit, roll back or close it. | mechanical | doc-only | no | nothing | — | — |
+| `api.v2.connectors-never-commit` | v2 connectors receive a session from the service and never open (with a session factory or a database_session container), commit, roll back or close it, or import the v1 ORM, the v2 database or a database_session module. | mechanical | enforced | yes | quality guard `v2-layers` | `pr.yml:quality-guards` | yes |
 | `api.v2.connectors-no-cross-calls` | v2 connector modules never import one another, so they cannot call one another to sequence a workflow; the service does the sequencing. | mechanical | ratcheted | yes | import-linter `pyproject.toml` | `pr.yml:quality-guards` | yes |
 | `api.v2.connectors-reads-no-mutations` | v2 read connectors never insert, add or delete rows. | mechanical | enforced | yes | pytest `tests/unit/v2/test_data_crud_boundaries.py` | `pr.yml:test` | **unproven** |
 | `api.v2.crud-checks-cover-every-resource` | The v2 CRUD boundary checks find modules by filename, so every resource package is covered, and every connector module is named as a create, read, update or delete module. | mechanical | enforced | yes | pytest `tests/unit/v2/test_data_crud_boundaries.py` | `pr.yml:test` | **unproven** |
+| `api.v2.files-assigned-to-layers` | Every .py file under services/v2 and fastapi_routes/v2 has a layer, from its file name or from rules/v2-layer-manifest.toml, so the layer rules check it. | mechanical | enforced | yes | quality guard `v2-layers` | `pr.yml:quality-guards` | yes |
 | `api.v2.import-layers` | v2 route modules may import v2 services and v2 data modules, and v2 services may import v2 data modules, never the reverse, directly or through other modules. | mechanical | enforced | yes | import-linter `pyproject.toml` | `pr.yml:quality-guards` | yes |
 | `api.v2.pure-modules-no-database` | v2 validators, transformations and types never reach SQLAlchemy, SQLModel, a connector, a database_session module or the v2 database module, directly or through other modules, except through the v2 table models; imports under TYPE_CHECKING count. | mechanical | ratcheted | yes | import-linter `pyproject.toml` | `pr.yml:quality-guards` | yes |
+| `api.v2.pure-modules-no-session-calls` | v2 validators, transformations, types, process-state modules and package __init__ files import nothing from SQLAlchemy or SQLModel but the Session type, import no connector, database_session, v1 ORM or v2 database module, and call no session method, SQLAlchemy or SQLModel function or session factory; they work on values and rows a connector loaded. | mechanical | enforced | yes | quality guard `v2-layers` | `pr.yml:quality-guards` | yes |
 | `api.v2.routes-import-no-database-access` | v2 route modules never import a connector, a database_session module or the v2 database module; they call a service, which does. | mechanical | enforced | yes | import-linter `pyproject.toml` | `pr.yml:quality-guards` | yes |
-| `api.v2.routes-no-sessions-or-sql` | v2 route modules never open database sessions or construct SQL; they call one service method and map typed failures to HTTP. | mechanical | doc-only | no | nothing | — | — |
-| `api.v2.services-no-sql` | v2 services sequence work but never construct SQL expressions, call Session.exec, or define HTTP response models. | mechanical | doc-only | no | nothing | — | — |
+| `api.v2.routes-no-sessions-or-sql` | v2 route modules never open database sessions or construct SQL: they import no SQLAlchemy, SQLModel, connector, database_session, v1 ORM or v2 database module, and call no SQL builder, session method or session factory; they call one service method and map typed failures to HTTP. | mechanical | ratcheted | yes | quality guard `v2-layers` | `pr.yml:quality-guards` | yes |
+| `api.v2.services-no-sql` | v2 services sequence work but never construct SQL expressions (they import nothing from SQLAlchemy or SQLModel but the Session type and sqlalchemy.exc), call Session.exec or another data-access session method, or define HTTP response models (they import no FastAPI or Starlette). | mechanical | enforced | yes | quality guard `v2-layers` | `pr.yml:quality-guards` | yes |
+| `api.v2.session-modules-no-data-access` | v2 database_session modules sit directly in their resource package and only open, commit, roll back and close sessions: they call no data-access session method (add, get, merge, exec, execute and so on) or SQL builder, and import no connector. | mechanical | enforced | yes | quality guard `v2-layers` | `pr.yml:quality-guards` | yes |
 | `api.v2.session-modules-no-queries` | v2 database_session modules own session lifetime only and never build select, insert, update, delete or exec calls. | mechanical | enforced | yes | pytest `tests/unit/v2/test_data_crud_boundaries.py` | `pr.yml:test` | **unproven** |
 | `api.v2.types-transformations-no-validators` | v2 types and transformations modules never import a validators module, directly or through other modules; validators import types, not the reverse. | mechanical | ratcheted | yes | import-linter `pyproject.toml` | `pr.yml:quality-guards` | yes |
 | `api.v2.user-services-no-sql` | v2 user services (identity, accounts, legacy references and user associations) never build or run SQL themselves or import SQLAlchemy; their connectors do. | mechanical | enforced | yes | pytest `tests/unit/v2/test_data_crud_boundaries.py` | `pr.yml:test` | yes |
@@ -105,34 +106,42 @@ flowchart LR
   r_api_v2_connectors_mutations_no_reads("api.v2.connectors-mutations-no-reads (enforced)")
   e_pytest_tests_unit_v2_test_data_crud_boundaries_py["pytest tests/unit/v2/test_data_crud_boundaries.py"]
   j_CI_pr_yml_test["CI pr.yml:test"]
-  r_api_v2_connectors_never_commit("api.v2.connectors-never-commit (doc-only)")
+  r_api_v2_connectors_never_commit("api.v2.connectors-never-commit (enforced)")
+  e_quality_guard_v2_layers["quality guard v2-layers"]
+  j_CI_pr_yml_quality_guards["CI pr.yml:quality-guards"]
   r_api_v2_connectors_no_cross_calls("api.v2.connectors-no-cross-calls (ratcheted)")
   e_import_linter_pyproject_toml["import-linter pyproject.toml"]
-  j_CI_pr_yml_quality_guards["CI pr.yml:quality-guards"]
   r_api_v2_connectors_reads_no_mutations("api.v2.connectors-reads-no-mutations (enforced)")
   r_api_v2_crud_checks_cover_every_resource("api.v2.crud-checks-cover-every-resource (enforced)")
+  r_api_v2_files_assigned_to_layers("api.v2.files-assigned-to-layers (enforced)")
   r_api_v2_import_layers("api.v2.import-layers (enforced)")
   r_api_v2_pure_modules_no_database("api.v2.pure-modules-no-database (ratcheted)")
+  r_api_v2_pure_modules_no_session_calls("api.v2.pure-modules-no-session-calls (enforced)")
   r_api_v2_routes_import_no_database_access("api.v2.routes-import-no-database-access (enforced)")
-  r_api_v2_routes_no_sessions_or_sql("api.v2.routes-no-sessions-or-sql (doc-only)")
-  r_api_v2_services_no_sql("api.v2.services-no-sql (doc-only)")
+  r_api_v2_routes_no_sessions_or_sql("api.v2.routes-no-sessions-or-sql (ratcheted)")
+  r_api_v2_services_no_sql("api.v2.services-no-sql (enforced)")
+  r_api_v2_session_modules_no_data_access("api.v2.session-modules-no-data-access (enforced)")
   r_api_v2_session_modules_no_queries("api.v2.session-modules-no-queries (enforced)")
   r_api_v2_types_transformations_no_validators("api.v2.types-transformations-no-validators (ratcheted)")
   r_api_v2_user_services_no_sql("api.v2.user-services-no-sql (enforced)")
   r_api_v2_validators_transformations_no_db("api.v2.validators-transformations-no-db (enforced)")
   r_api_routes_shared_query_parameters -.-> gap_unenforced
   r_api_v2_connectors_mutations_no_reads --> e_pytest_tests_unit_v2_test_data_crud_boundaries_py
   e_pytest_tests_unit_v2_test_data_crud_boundaries_py --> j_CI_pr_yml_test
-  r_api_v2_connectors_never_commit -.-> gap_unenforced
+  r_api_v2_connectors_never_commit --> e_quality_guard_v2_layers
+  e_quality_guard_v2_layers --> j_CI_pr_yml_quality_guards
   r_api_v2_connectors_no_cross_calls --> e_import_linter_pyproject_toml
   e_import_linter_pyproject_toml --> j_CI_pr_yml_quality_guards
   r_api_v2_connectors_reads_no_mutations --> e_pytest_tests_unit_v2_test_data_crud_boundaries_py
   r_api_v2_crud_checks_cover_every_resource --> e_pytest_tests_unit_v2_test_data_crud_boundaries_py
+  r_api_v2_files_assigned_to_layers --> e_quality_guard_v2_layers
   r_api_v2_import_layers --> e_import_linter_pyproject_toml
   r_api_v2_pure_modules_no_database --> e_import_linter_pyproject_toml
+  r_api_v2_pure_modules_no_session_calls --> e_quality_guard_v2_layers
   r_api_v2_routes_import_no_database_access --> e_import_linter_pyproject_toml
-  r_api_v2_routes_no_sessions_or_sql -.-> gap_unenforced
-  r_api_v2_services_no_sql -.-> gap_unenforced
+  r_api_v2_routes_no_sessions_or_sql --> e_quality_guard_v2_layers
+  r_api_v2_services_no_sql --> e_quality_guard_v2_layers
+  r_api_v2_session_modules_no_data_access --> e_quality_guard_v2_layers
   r_api_v2_session_modules_no_queries --> e_pytest_tests_unit_v2_test_data_crud_boundaries_py
   r_api_v2_types_transformations_no_validators --> e_import_linter_pyproject_toml
   r_api_v2_user_services_no_sql --> e_pytest_tests_unit_v2_test_data_crud_boundaries_py