fix(firewall): close the firewall's own addresses to every segment, key-only sshd

README.md

@@ -135,8 +135,8 @@ documents for different readers.
   at all. Every pinned digest is also scanned weekly for fixable HIGH and
   CRITICAL CVEs, and each image with findings has an open issue until a scan
   finds it clean.
-- **Documented decisions and runbooks.** 116 ADRs covering what was chosen
-  and what was rejected — including the costs accepted knowingly; 54
+- **Documented decisions and runbooks.** 117 ADRs covering what was chosen
+  and what was rejected — including the costs accepted knowingly; 55
   runbooks for the operations that are easy to get wrong at 1am, one of which
   is the handover page a successor reads first. Every critical alert links to
   one.
@@ -235,9 +235,9 @@ inbound passes may still reach it. Data flow and the maintained Mermaid topology
 │   ├── architecture.md  network.md  hardware.md
 │   ├── observability.md  security.md  roadmap.md  changelog.md
 │   ├── diagrams/             # the network diagram (SVG) and its predecessors
-│   ├── adr/                  # 116 ADRs — architecture decision records
+│   ├── adr/                  # 117 ADRs — architecture decision records
 │   ├── exercises/            # purple-team exercise records — see its README
-│   └── runbooks/             # 54 runbooks; successor-handover.md is the front door
+│   └── runbooks/             # 55 runbooks; successor-handover.md is the front door
 └── Makefile                  # make help
 ```
 

docs/adr/0117-close-the-firewalls-own-addresses-and-take-passwords-off-its-ssh.md

@@ -0,0 +1,116 @@
+# ADR-0117: Close the firewall's own addresses to every segment, and take passwords off its SSH
+
+**Status:** Accepted · 2026-11
+
+## Context
+
+A read-only audit of `morpheus` from `prometheus` on 2026-11-13 found two
+things the documents said were closed.
+
+**Every segment reached the admin UI and `sshd` on the firewall's WAN
+address.** Each interface carries *Block SSH to pfSense*, *Block HTTPS to
+pfSense* and *Block HTTP to pfSense* (Winterfell carries only the last two).
+Each block names that interface's own address and nothing else. The firewall
+holds an address on every segment and one on the WAN, and the interface's
+catch-all `pass ... to any` carried a packet to any of those not already
+blocked. From `prometheus`, `https://` on the WAN address answered 200 and
+port 22 there gave an OpenSSH banner. [`network.md`](../network.md) said
+"Winterfell itself is blocked from it". That held for `10.0.99.1` and for no
+other address. Hicks had a second way in: *Allow Hicks access to
+ImaginationLAN* passes TCP to the whole of `10.0.30.0/24`
+([ADR-0031](0031-narrow-hicks-to-a-named-list-on-winterfell-and-leave-the-lab-open.md)),
+and the lab's gateway, `10.0.30.1`, is in it.
+
+`make check-firewall` could not see this, and that is a gap in its question,
+not a defect in its code. It asks which *segments* a catch-all reaches, and the
+WAN address is in no segment.
+
+**`sshd` accepted passwords, keyboard-interactive and root.** `auth.log`, read
+on 2026-11-14, has `admin` logging in by password from a Hicks desktop on
+2026-11-10, from the Hicks MacBook on 2026-11-13, and six times from `oracle`
+on 2026-11-10. All of
+Hicks reaches port 22 on the firewall by *Allow SSH to Winterfell*, and every
+segment reached it on the WAN address. That includes the two corporate laptops
+on Hicks, which [`network.md`](../network.md) says are to be treated as
+untrusted. Two keys are authorised. One is `prometheus`'s, and it is how every
+collector, the nightly export and `make check-firewall` log in, as `root`.
+
+## Decision
+
+**Every interface blocks ssh, http and https to `This Firewall (self)`, not to
+its own address.** Each block's source is `any` and its family is IPv4+IPv6,
+and the three rows sit above every other rule on the interface except the named
+exceptions below. pfSense expands `(self)` to every address the firewall holds,
+the WAN's included. The three rows keep their names. Hicks' three move up, above
+*Allow Hicks access to ImaginationLAN*.
+
+**The exceptions are named passes above those blocks, and there are three:**
+
+| Segment | Pass | Why |
+| --- | --- | --- |
+| Hicks | *Allow SSH to Winterfell*, already there, 22 to `10.0.99.0/24` | administration from a workstation, key-only below |
+| Hicks | *Allow HTTPS to pfSense on its management address*, already there, 443 to `10.0.99.1` | the admin UI |
+| Winterfell | *Allow SSH to pfSense from prometheus*, new, `10.0.99.20` → `10.0.99.1:22` | the collectors, the export and the claims check |
+
+The switch LAN keeps pfSense's generated anti-lockout rule, which sits above
+every user rule and passes 22, 80 and 443 to the LAN's own address. It is the
+way back in if a rule change locks Hicks out, and the only thing on that
+segment is `neo`. The rest of the switch LAN is closed by the same three blocks.
+
+So the admin UI answers at `10.0.99.1` to Hicks and nowhere else, and `sshd`
+answers at `10.0.99.1` to Hicks and to `prometheus`. `oracle` and `trinity` lose
+port 22 on the firewall. Nothing scheduled on either uses it. The 2026-11-10
+logins from `oracle` were a person at a shell there, and that person now uses
+`prometheus` or Hicks.
+
+**`sshd` is key-only.** *System → Advanced → Admin Access → SSHd Key Only* is
+set to *Public Key Only*, which turns off password and keyboard-interactive
+authentication. Root login stays allowed, because `prometheus`'s jobs log in as
+`root` (`FW_USER`'s default since `backup-firewall.sh`), and with passwords off
+that is a key login too. Every machine that administers the firewall over SSH
+has its key on `admin` under *System → User Manager* before the switch. The
+password path that remains is the console, at the rack.
+
+**The claims file asks the second question, and the timer answers it nightly.**
+`docs/firewall-claims.yaml` gains a `firewall` entry per interface: which of the
+firewall's own addresses a host on that segment reaches on ssh, http and https.
+`check_firewall_claims.py` derives it by evaluating the ruleset the way pf
+does, one firewall address at a time, first `quick` match wins. It names each
+address by the segment it sits on, or `WAN`, or `link-local`, and never prints
+the number. A host-scoped pass like `prometheus`'s is not counted, because the
+question is what the segment reaches. The fixtures run in CI under
+`scripts/self-tests.sh`. The live run stays on `prometheus`, for
+[ADR-0026](0026-check-the-documents-where-the-truth-is.md)'s reason.
+
+## Alternatives considered
+
+- **Bind the admin UI and `sshd` to `10.0.99.1` only.** pfSense offers no
+  listen address for either in its UI, and a change made under it is lost to the
+  next upgrade or restore. Rules are the lever pfSense supports.
+- **One floating rule set instead of rows per interface.** Floating rules are
+  evaluated before every interface's own, so Hicks' two passes would have to
+  move there too. [`network.md`](../network.md) lists what each segment reaches
+  interface by interface, and so does `pfctl`. Three rows per interface keeps
+  those two agreeing.
+- **Disable the anti-lockout rule.** That removes the one recovery path that
+  needs no console. Its cost is that a device on an untagged switch port reaches
+  the UI's login page, and it has been accepted since the switch LAN was closed
+  ([ADR-0025](0025-close-the-switch-lan-to-winterfell.md)).
+- **Narrow Hicks' SSH to named workstations.** Hicks is a `/24` with no
+  per-host source by decision
+  ([ADR-0031](0031-narrow-hicks-to-a-named-list-on-winterfell-and-leave-the-lab-open.md)).
+  Key-only `sshd` is what stops a corporate laptop logging in. A per-host list
+  can follow when Hicks gets one.
+
+## Consequences
+
+- A rule that names one firewall address where it meant all of them now fails
+  the nightly check, by name, rather than waiting for the next audit.
+- An `ssh admin@10.0.99.1` with no key on the MacBook, the desktops or `oracle`
+  fails after the switch. [`close-the-firewall-to-its-segments.md`](../runbooks/close-the-firewall-to-its-segments.md)
+  §1 proves every key that is meant to work before the setting changes.
+- A mistake in the Hicks rows can lock the UI out from Hicks. Recovery is the
+  console's *Restore recent configuration*, or a cable into an untagged port on
+  `neo` and the anti-lockout rule.
+- The GUI's default self-signed certificate, the undeclared Hicks pass into
+  `smaug` and the other audit findings are not decided here.

docs/changelog.md

@@ -17,6 +17,26 @@ roadmap as it read that day, and the *Done* entries keep the shape they had
 there. `check_docs.py` does not check this file, for the reason its module
 docstring gives: it is a record, not a claim about now.
 
+## 2026-11-14
+
+- **The firewall's own addresses are closed to every segment, and its `sshd`
+  takes keys only
+  ([ADR-0117](adr/0117-close-the-firewalls-own-addresses-and-take-passwords-off-its-ssh.md)).**
+  A read-only audit of `morpheus` on 2026-11-13 found every segment reaching
+  the admin UI and `sshd` on the WAN address: each interface's *Block … to
+  pfSense* rows named only that interface's own address. From `prometheus`,
+  https there answered 200 and port 22 gave a banner, though `network.md` said
+  Winterfell was blocked from the UI. Hicks also reached both on the lab's
+  gateway, through *Allow Hicks access to ImaginationLAN*. `sshd` accepted
+  passwords; `admin` had logged in by password from Hicks and from `oracle`
+  that week. The blocks now name *This Firewall (self)*, Winterfell gains
+  *Allow SSH to pfSense from prometheus*, and *SSHd Key Only* is *Public Key
+  Only*, by
+  [`close-the-firewall-to-its-segments.md`](runbooks/close-the-firewall-to-its-segments.md).
+  `make check-firewall` was blind to it, because the WAN address is in no
+  segment. It now also asks which of the firewall's addresses each segment
+  reaches on ssh, http and https, with fixtures in CI.
+
 ## 2026-11-13
 
 - **An audit of `golem`, read-only, found its last verify failed and a

docs/firewall-claims.yaml

@@ -12,6 +12,15 @@
 # other segment is blocked above the catch-all, or there is no catch-all for that
 # family at all. Those are different rulesets and the same posture.
 #
+# WHAT `firewall` MEANS (ADR-0117). For each interface, which of the firewall's
+# OWN addresses a host on that segment reaches on ssh, http and https. Each
+# address is named by the segment it sits on, `WAN`, or `link-local`; never by
+# number. The 2026-11-13 audit found every segment reaching the admin UI and
+# sshd on the WAN address under rules named *Block HTTPS to pfSense*, and
+# `wholesale` could not see it, because the WAN address is in no segment.
+# A host-scoped pass (Winterfell's `prometheus` to 10.0.99.1:22) is not the
+# segment's and is not listed here; docs/network.md names it.
+#
 # It is deliberately NOT a list of what each segment can reach. The narrow
 # host-and-port exceptions — SNMP to the iLO, SSH to the firewall, the passes
 # from Hicks into management — are enumerated in docs/network.md, they change far
@@ -48,13 +57,26 @@ interfaces:
     wholesale:
       inet: []
       inet6: [OPT1, OPT2, OPT3, OPT4, OPT5, OPT6]
+    # The anti-lockout rule pfSense generates on LAN sits above every user rule
+    # and passes 22, 80 and 443 to this interface's own address, which is also
+    # its link-local. Kept as the way back in (ADR-0117).
+    firewall:
+      ssh: [Switch LAN, link-local]
+      http: [Switch LAN, link-local]
+      https: [Switch LAN, link-local]
 
   # Management. The one segment whose catch-all reaches nothing in either family.
   igc0.99:
     segment: Winterfell
     wholesale:
       inet: []
       inet6: []
+    # `prometheus` alone reaches 10.0.99.1:22, by a host-scoped pass above the
+    # blocks; the segment reaches nothing.
+    firewall:
+      ssh: []
+      http: []
+      https: []
 
   # Trusted workstations, and the one IPv4 exception in the estate: nothing
   # blocks Hicks from ImaginationLAN, so the catch-all grants that segment
@@ -71,6 +93,14 @@ interfaces:
     wholesale:
       inet: [OPT4]
       inet6: []
+    # The admin UI on 10.0.99.1:443, and ssh by *Allow SSH to Winterfell*, both
+    # above the blocks to (self). Not the lab's gateway, though *Allow Hicks
+    # access to ImaginationLAN* passes TCP to 10.0.30.0/24: the blocks sit
+    # above it.
+    firewall:
+      ssh: [Winterfell]
+      http: []
+      https: [Winterfell]
 
   # Media, lab, IoT and guest. Identical posture, and the inet6 row is identical
   # for a reason worth stating: each of these carries paired inet/inet6 blocks
@@ -84,21 +114,37 @@ interfaces:
     wholesale:
       inet: []
       inet6: [LAN]
+    firewall:
+      ssh: []
+      http: []
+      https: []
 
   igc0.30:
     segment: ImaginationLAN
     wholesale:
       inet: []
       inet6: [LAN]
+    firewall:
+      ssh: []
+      http: []
+      https: []
 
   igc0.20:
     segment: Skids
     wholesale:
       inet: []
       inet6: [LAN]
+    firewall:
+      ssh: []
+      http: []
+      https: []
 
   igc0.10:
     segment: Degens
     wholesale:
       inet: []
       inet6: [LAN]
+    firewall:
+      ssh: []
+      http: []
+      https: []

docs/network.md

@@ -216,9 +216,21 @@ listed under [Hicks](#hicks--vlan-50--trusted), and nothing else.
   from `10.0.99.40` on the WAN graphs is that and not something to chase. It
   leaves by Winterfell's existing egress rule, and its fetcher refuses every
   private address, this segment's included ([ADR-0057](adr/0057-add-miniflux-to-the-sensitive-tier-with-its-fetcher-kept-off-winterfell.md)).
-- pfSense's admin UI is reachable on this interface from Hicks only, by a
-  named pass to `10.0.99.1:443`. Winterfell itself is blocked from it: the 99
-  interface drops HTTP and HTTPS to `10.0.99.1` above its egress rule.
+- **The firewall answers on its own addresses to two places, since
+  2026-11-14:** its admin UI to Hicks, at `10.0.99.1:443`, and `sshd` to Hicks
+  and `prometheus`, at `10.0.99.1:22`. Every interface blocks ssh, http and
+  https to *This Firewall (self)*, every address the firewall holds, the WAN's
+  included. The blocks sit above everything but three named passes: Hicks'
+  *Allow HTTPS to pfSense on its management address* and *Allow SSH to
+  Winterfell*, and Winterfell's *Allow SSH to pfSense from prometheus*,
+  `10.0.99.20 → 10.0.99.1:22`, for the collectors, the nightly export and
+  `make check-firewall`. Nothing else on Winterfell reaches either. Until then
+  each interface's blocks named only its own address, so every segment
+  reached both on the WAN address; the 2026-11-13 audit proved it from
+  `prometheus`. `sshd` takes keys only.
+  [ADR-0117](adr/0117-close-the-firewalls-own-addresses-and-take-passwords-off-its-ssh.md)
+  records the decision, and `docs/firewall-claims.yaml`'s `firewall` entries
+  are what `make check-firewall` holds the ruleset to.
 - **One pass into Skids, in force since 2026-09-28:**
   `10.0.99.40 → 10.0.20.20` on `80,443/tcp` — `trinity`'s Home Assistant to
   `bifrost`, the Hue bridge, and nothing else on 20 — **above** *Block access
@@ -336,6 +348,15 @@ though not the only one — ImaginationLAN has two host-scoped passes to
   | `10.0.99.30` — `oracle` | `443/tcp` the wiki, served by its Caddy since [#847](https://github.com/Gerrrt/HomeLab/issues/847)'s cutover on 2026-10-07, and `80/tcp`, which only redirects to it ([`stacks/wiki/README.md`](../stacks/wiki/README.md#tls)). 443 had been dropped on 2026-09-30, when nothing listened on it ([#251](https://github.com/Gerrrt/HomeLab/issues/251)) |
   | `10.0.99.40` — `trinity` | `443/tcp` the sensitive tier, since 2026-09-28 |
 
+  **The firewall itself answers Hicks at `10.0.99.1` and nowhere else.** The
+  three *Block … to pfSense* rows name *This Firewall (self)* and sit directly
+  under the `22` and admin-UI passes, above *Allow Hicks access to
+  ImaginationLAN*. That rule passes TCP to all of `10.0.30.0/24`, the lab's
+  gateway included, and until 2026-11-14 it was a second way to the UI and
+  `sshd` ([ADR-0117](adr/0117-close-the-firewalls-own-addresses-and-take-passwords-off-its-ssh.md)).
+  `sshd` takes keys only, so the `22` pass gets a corporate laptop a prompt it
+  cannot answer.
+
   **The source is the segment, not named hosts.** Every one of those passes is
   `vlan50 → …`, so any device on Hicks may use any of them. This note used to
   say the opposite — "only specific hosts, and only on management ports" — and