[murwall]

fix(zap): read the meter DER under Zap's data model v2 local API

Fixes (partially — meter DER only) #170.

Problem

A Zap gateway on firmware serving the data model v2 local API
(srcful-data-models 2.3.0, "version":"v2" in /api/devices/<sn>/data/json)
gives FTW's zap.lua no meter reading at all. emit_meter reads raw.W and
the meter_fields table's v1 names (L1_W, L1_V, L1_A, total_import_Wh,
total_export_Wh) directly; none of those keys exist in a v2 payload, so
number(raw.W) is nil, emit_meter returns false, and the driver logs
Zap: site-meter payload unavailable every poll until METER_FAIL_REDISCOVER
is hit and discovery invalidates — forever, since the rediscovered payload
has the same shape. A site using Zap as is_site_meter: true loses its grid
import/export reading entirely, silently, until someone notices stale
consumption data in the UI.

Reproduced on

• Device: Sourceful Zap (platform: srcful-zap-p1), gateway firmware
  7.0.3 (firmwareBuild: v7.0.3-dirty). (Firmware version, model and the
  v2 field names are the evidence; the gateway's serial, IP and MAC are
  redacted here per CONTRIBUTING.md.)
• FTW host: v2.3.2 (bundled zap.lua id sourceful-zap, local pin at version
  3.1.0); confirmed the same read-path bug exists unchanged at this repo's
  current main (zap.lua id zap, version 3.1.1).
• The gateway's own device log (GET /zap.logg) confirms the HTTP exchange
  itself succeeds on every poll (GET /api/devices/<sn>/data/json →
  returning device data, 200), and a direct curl of that endpoint
  returns a well-formed meter object — the break is entirely in
  emit_meter's field names, not in discovery, connectivity, or payload
  availability.
• Confirmed the device list (GET /api/devices) ders[].type for the P1
  meter is unaffected by the v2 rollout — still "meter", not renamed like
  pv → solar — so discover_devices() needed no change for this fix.

Values compared

Field read by zap.lua (v1)	v2 payload key actually present	Example value seen
W	W_ac	2328
Hz	Hz (unchanged)	null (grid frequency not populated by this meter)
L1_V / L1_A / L1_W	l1_V_ac / l1_A_ac / l1_W_ac	233.8 / 2.3 / 447
L2_V / L2_A / L2_W	l2_V_ac / l2_A_ac / l2_W_ac	233.4 / 2.6 / 607
L3_V / L3_A / L3_W	l3_V_ac / l3_A_ac / l3_W_ac	231.9 / 5.5 / 1273
total_import_Wh	total_import_Wh_ac	58203578
total_export_Wh	total_export_Wh_ac	10630697

Live-site values, MAC/serial redacted. This exactly matches the mapping
#170 already documents from the data model spec — this PR is the
implementation of that issue's "What zap.lua needs" item 1 and 2, scoped to
the meter DER.

Fix

emit_meter now reads every field through a small raw_field(raw, name)
helper: try the exact v1 name, then name .. "_ac", then (for the L1/L2/L3
fields) the lowercased-prefix _ac variant. No version branch is needed —
each lookup just tries progressively more v2-shaped keys, so the same code
path reads either firmware generation. Hz needs no fallback since it is
unchanged between schemas.

Out of scope (tracked by #170, not done here)

• pv → solar rename in both discover_devices() and the data envelope.
• emit_pv / emit_battery have the identical .W/rated_power_W/etc.
  v1-only field reads against v2's _ac/_dc-split names. Not fixed here —
  I don't have hardware with read_pv/read_battery enabled on v2 firmware
  to verify against, and the DC/AC split for those DERs is a genuinely
  different mapping than the meter's (a battery's own measurement is DC;
  see zap: read the Zap's data model v2 local API (pv → solar, W → W_ac, ...) #170's table), not just an added suffix.
• The null-vs-0 rule (44 drivers emit numbers they never read (null, never 0) #169) — not touched; number() already treats a
  JSON null as absent (tonumber(nil) is nil), so this fix doesn't
  regress it, but I haven't separately verified 44 drivers emit numbers they never read (null, never 0) #169 end-to-end.

Testing

• Hardware: the raw_field fallback chain was derived from, and
  verified against, a real Zap gateway on firmware 7.0.3 (v2 payload) —
  confirmed emit_meter now returns true and the meter DER emits with the
  patched driver swapped in locally (FTW custom-drivers/zap.lua bind mount
  override, ahead of this PR existing). Not verified against v1 firmware
  hardware — no Zap still on v1 to test the regression path against; the
  v1 branch is exact-name lookup, unchanged from before this PR, and is also
  covered by the v1 test scenario below.
• This repo's real Lua 5.5 toolchain, built locally from the vendored
  source (bash tools/build_luac.sh --with-interpreter):
  • ./luac55 -p drivers/lua/zap.lua — parses clean.
  • bash tools/check_sandbox.sh — all 92 drivers pass, zap included.
  • ./lua55 drivers/tests/lua_harness/test_zap_meter_schema.lua v1 — exit 0.
  • ./lua55 drivers/tests/lua_harness/test_zap_meter_schema.lua v2 — exit 0.
    Both scenarios assert emit_meter produces the identical normalized
    reading (w, l1_w/l2_w/l3_w, phase V/A, freq_hz,
    total_import_wh/total_export_wh and their import_wh/export_wh
    aliases) from a v1-named and a v2-named (_ac-suffixed, lowercase phase
    prefix) mock payload. Also checked DRIVER.id/version/host_api_*/
    read_only match what the manifest bump below produced.
• python tools/bump_driver.py --id zap --level patch,
  python tools/sync_manifests.py, python tools/generate_index.py,
  python tools/generate_devices.py, python tools/generate_support_status.py,
  and python tools/validate_manifest.py (plain python3, not uv run — no
  uv/package install available where this was drafted, but these tools
  only import the standard library and this repo's own manifest_parser)
  all ran clean: 92 manifests checked, 0 errors, 92 manifests match their source on a second pass (idempotent), index.yaml/devices.yaml/
  SUPPORT_STATUS.md/support-status.json regenerated and included.
• Not run: uv run/make bootstrap itself (no uv where this was
  drafted — the above tools were run directly with python3 instead, which
  worked since none of them need third-party packages), and the
  signed channel accepts this tree check, which needs CI's signing
  infrastructure and can't be run outside it.

Changes in this PR

• drivers/lua/zap.lua: raw_field() helper + emit_meter fix, header
  comment note, DRIVER.version 3.1.1 → 3.1.2.
• manifests/zap.yaml: version: "3.1.2", sha256/size_bytes regenerated
  for real via sync_manifests.py. Did not change tested_devices[].notes
  or firmware_versions — open to adding a v1/v2 compatibility note there
  if a maintainer wants it, wasn't sure what convention to follow.
• index.yaml, devices.yaml, SUPPORT_STATUS.md, support-status.json:
  regenerated to match (mechanical, from the above tools).
• CHANGELOG.md: [Unreleased] entry added.
• drivers/tests/lua_harness/test_zap_meter_schema.lua +
  drivers/tests/test_zap_meter_schema.py: new test pair (see Testing).

Bumped as LEVEL=patch — same registers and fields, no new telemetry — but
LEVEL=minor is defensible too given it's compatibility with a whole new
API generation rather than a narrow register fix; happy to re-bump if a
maintainer prefers minor.

Why only the meter DER

This FTW site runs Zap strictly as is_site_meter: true with
read_pv/read_battery both false (PV and battery come from the
inverter/battery's own native drivers), so the meter path is what broke and
what I could verify end-to-end against real v2 hardware. Filing this as a
focused fix rather than attempting the full #170 scope blind.

[murwall]

@frahlg Noticed yesterday that my ZAP received an OTA that changed naming and some other things.
I'm guessing it's related to #170
This patch here is a draft fix that I have running in my environment that at least seems to work well thus far.
I'm dropping it here in case it can help anyone with similar issue. Feel free to merge or drop it.

[davmoz]

Verified before merge, through FTW's own Lua host (master be11e360) against real Zaps on firmware 7.0.3, GET only:

• Zap-HAN with a real meter, unpatched 3.1.1: no meter, site-meter payload unavailable. Reproduces the report.
• Zap-HAN with a real meter, this PR: meter reads, with phase V/A/W and energy totals.
• Zap-P1 with a simulated Solis (a Modbus simulator on the LAN), this PR: the Solis meter DER reads, export negative. The firmware and its payload are real; the inverter is not.

read_pv and read_battery still read nothing on 7.x, as the PR says. That follows in a separate PR, which also moves the version check in this test.

Thanks for the fix and the evidence.