The SCiO spectrometer: a technical reference

The Consumer Physics SCiO is a pocket near-infrared spectrometer, sold 2015-2019 and now discontinued. It has no offline mode: the device emits opaque blobs and a vendor server turns them into spectra. This repository documents the device end-to-end - transport, framing, every command, the data formats, the calibration rules and the server API - and provides a working Python implementation, so that a SCiO you own stays usable.

Two paths exist:

	status
Capture -> vendor server -> spectrum	works. Verified against 2020/2021 stored spectra to ~1e-15. Needs an active Consumer Physics account.
Capture -> spectrum, offline	unsolved. Scan bodies are opaque, high-entropy and fixed-length; both the class of the transform (encryption or proprietary coding) and where it runs are undetermined. Current state and next steps: `dev/HANDOVER.md`.

Capture itself needs no network. A scan is written as a self-contained record and can be converted to a spectrum later, from anywhere. That separation is the point: the server may be switched off at any time, and a record captured today must still be convertible the day someone breaks the offline path.

1. Quick start

conda env create -f scio_env.yml      # or: conda activate tp
jupyter lab 01_scio_scan_to_spectrum.ipynb

Notebook 01_scio_scan_to_spectrum.ipynb is the whole workflow: name the scan, connect, capture, upload, plot. Everything it does is a call into src/scio/:

import sys; sys.path.insert(0, "src")
from scio import credentials, session, usb

dev  = usb.ScioUSB("COM5").open()
path = session.capture(dev, "bark", comment="north-facing trunk")   # offline
dev.close()

out = session.process(path, credentials.get_token())                # online, later

Raw records land in 01_rawdata/scans/, spectra in 02_processed_data/. session.process_pending() uploads a backlog whenever you next have a connection.

Three notebooks, numbered by role - 01 is the one to open:

notebook	what it is for
`01_scio_scan_to_spectrum.ipynb`	the full workflow: annotate, connect, capture, upload, plot
`02_scio_device_health.ipynb`	identifiers, battery, temperature, firmware file headers
`03_scio_probe.ipynb`	read-only probing of undocumented opcodes

All three import only scio; offline-decoding research is in dev/notebooks/, and superseded notebooks are in archive/notebooks/.

The device must be fully awake. A steady blue LED means it answers commands. A slow light/dark pulse means it is idle or charging, and its USB endpoint goes silent or disappears. Fix: unplug, long-press off, long-press on until steady blue, replug. It re-idles by itself after a period without commands.

Remote power control: no immediate-off or remote-on command has been confirmed. The app does support an automatic-off timer. On an open ScioUSB instance named dev, dev.read_power_saver() queries it; dev.set_power_saver(minutes, allow_write=True) sends the app-compatible setting for 1–30 minutes. Its signed-byte adjustment means the encoded duration can differ from the requested minutes: inspect encoded_seconds in the return value. These optional USB write helpers are hardware-unverified and can alter device behavior. Nothing writes by default. The setter never resets automatically; the app resets after successful writing, so dev.reset_device(allow_write=True) is a separate, disruptive operation, not a power-off/on substitute. Do not retry a timed-out reset blindly. The same helpers exist on ScioBLE (section 2); over BLE they are equally unverified. See power evidence and limitations. The read itself was verified on the connected firmware-147 unit: it returned 360 seconds (6 minutes), between matching identity controls. Actual shutdown timing, writes, reset behavior and remote wake remain unverified.

Scan location: every new record stores where it was taken in a mobile_GPS block shaped like the phone app's (latitude, longitude, locality, country, admin_area, address_line), plus location_meta (source, accuracy, time, why a lookup failed). The coordinates come from the capturing computer's own location service: on Windows the built-in location API, which needs Settings → Privacy & security → Location → Let desktop apps access your location; on Linux GeoClue; on macOS CoreLocationCLI. There is no IP-based lookup and no reverse geocoding, so nothing leaves the machine and the text fields stay empty. When no fix is available the fields are empty and the capture goes ahead. It is off by default. Turn it on with capture(..., geolocate=True) or SCIO_GEOLOCATION=1, or pass a phone's fix with capture(..., location_override={"latitude": …, "longitude": …}).

The location is kept locally, in the raw record and in the processed spectrum file. It is never sent to the server unless you consent with session.process(path, token, send_location=True) or session.process_pending(send_location=True).

GPS data is never published in this repository. tools/check_public_safety.py, run by the pre-commit hook, refuses any file with a real latitude/longitude. Empty fields and the Zurich redaction placeholder (47.3769, 8.5417) pass. A located scan therefore stays on your machine: remove its coordinates before committing it.

2. Transport

USB CDC

The SCiO enumerates as a Texas Instruments CDC serial port, VID:PID 0451:16AA. Baud rate is irrelevant (it is a USB CDC device, not a real UART); DTR/RTS handling does not matter. Responses arrive as one contiguous frame.

Bluetooth LE

The same 0xBA command protocol runs over BLE, and scio.ble.ScioBLE implements it with bleak (pip install bleak). It has the same methods as ScioUSB and drops into session.capture unchanged; the notebooks switch with TRANSPORT = "ble":

from scio import ble
print(ble.find_scio_ble())               # advertisers, likely SCiOs first
with ble.ScioBLE() as dev:               # or ScioBLE("B4:99:4C:59:66:01") / ScioBLE("SCiOmyScio")
    info = dev.read_device_info()
    scan = dev.sample_spectrum(info["firmware_version"])

No pairing. Do not pair the SCiO in the OS Bluetooth settings; the app never bonds and neither does this library.
It must be awake. The SCiO advertises only while on, and switches itself off after its automatic-off timer (6 min on this unit). Press the button first. This unit advertises as SCiOmyScio; anything whose name contains "scio", or that advertises service 3490, is treated as a SCiO.
Synchronous API. bleak is asyncio-only; ScioBLE runs its own event loop in a background thread, so it works the same in a script and in Jupyter.
One connection at a time. While a ScioBLE is connected the SCiO stops advertising. If open() can't find it because an earlier ScioBLE in the same process still holds it (e.g. a re-run notebook cell), it closes that old session, prints a note saying so, and connects; the old object then raises ScioDisconnected. With a named device (ScioBLE(address_or_name)) it skips the first scan, which cannot succeed while the device is held, so the takeover costs no extra scan cycle. A session holding a different SCiO is left alone, and ble.close_all() closes every session. A connection held by another process (a second kernel or script) can't be released from here: close it or restart that kernel. If the link drops, commands fail fast with ScioDisconnected and dev.open() reconnects.
Replies are matched to requests (BLE and USB). Each reply echoes its command id (confirmed in the 2019 BLE captures, all three scan messages included, and in every USB probe log). A late reply to a request that timed out is dropped rather than returned as the next answer (dev.stale_replies; USB gives up with ScioProtocolError after 8 stale frames in a row), and BLE packets belonging to no reply are ignored (dev.stray_packets).
Windows (10/11, verified on hardware): the built-in Bluetooth stack is used through WinRT; nothing to configure.
Linux (BlueZ >= 5.55 over D-Bus, not yet verified on hardware): bluetoothd must be running and the user allowed on the system D-Bus (the default for a desktop user; no sudo needed). Check with bluetoothctl power on and bluetoothctl scan on that the SCiO shows up, then run the four calls above: find_scio_ble() lists it, read_device_info() returns ble_id 01665900004c99b4 / BLE fw 125, sample_spectrum returns 1800/1800/1656-byte blobs. If BlueZ has a stale cache after a firmware change, bluetoothctl remove <MAC> clears it.
Timing (Windows, this unit): identity read 0.3 s, a three-blob scan ~3.2 s (about 280 notifications).

Vendor GATT service 00003490-0000-1000-8000-00805f9b34fb:

Role	UUID	Handle (this unit)
Control (write commands)	`00003492-…`	`0x0029`
Reporter (scan-data notifications)	`00003491-…`	replies on `0x0025`
Button pressed (notification)	`00003493-…`	`0x002c`, reads `0x01` on press

Standard characteristics: device name 0x2a00 (handle 0x0003), system id 00002a23-… (0x0012), manufacturer name 0x2a29 (0x001e), under services 0x1800/0x180a. Each of 3491/3492/3493 carries CCCD (0x2902) and 0x2901 descriptors.

A command is written to the control characteristic; the response arrives as 20-byte notifications on the reporter, each prefixed with the sequence byte (unlike USB, where the frame is contiguous). A real scan sequence, written to handle 0x0029:

01ba050000                    # read battery state
01ba0e0000                    # ready for white reference
01ba0b0900000000000000000000  # set indication LED (9-byte payload)
01ba040000                    # read temperature
01ba020000                    # sample spectrum -> replies on 0x0025

With BlueZ:

sudo gatttool -i hci0 -b <MAC> --char-write-req -a 0x0029 -n 01ba020000 --listen

Requests are split the same way as replies (RequestCommandBuilder): the first packet is 01 BA cmd len_lo len_hi plus up to 15 payload bytes, each later one a sequence byte (2, 3, ...) plus up to 19. Every request starts at sequence 1; the USB session counter does not apply. Writes are with response, except READY_FOR_WR (0x0E), CLEAR_READY_FOR_WR (0x11) and FILE_DOWNLOAD (0x81). A reply message is complete when len bytes have arrived; a scan sends two or three messages back to back, each starting again at sequence 1. See protocol.ble_packets and protocol.BleReassembler.

archive/notebooks/05_scio_ble_devel.ipynb holds an earlier, failed bleak attempt. Why it failed: it wrote the command to the control characteristic and then called read_gatt_char on that same characteristic to get the answer. Replies never arrive there - they arrive as notifications on the reporter characteristic, which must be subscribed to first. (It also wrote the ASCII string "01ba040000" rather than the five bytes bytes.fromhex("01ba040000"), so the device received garbage either way.)

Framing (both directions, both transports)

[seq, 0xBA, cmd, len_lo, len_hi, payload...]

field	size	meaning
`seq`	1 B	sequence counter; `0x01` for a first/only packet
`0xBA`	1 B	protocol marker (`-70` as a signed Java byte)
`cmd`	1 B	command id, see below
`len`	2 B	payload length, little-endian uint16
`payload`	`len` B	command-specific

There is no CRC or checksum. A response reuses the same layout, so a reader should hunt for the 0xBA marker and resynchronise rather than assume alignment

scio.usb.ScioUSB._read_response does exactly that.

Some commands answer with several frames back to back: SAMPLE_SPECTRUM returns two or three, one per blob.

3. Command reference

Legend: R read-only (safe), W writes or changes device state (never sent automatically; power-saver/reset helpers require explicit opt-in), X declared in the app but unimplemented on this firmware (147) - probed on hardware, logs in 01_rawdata/probe_logs/.

The app symbol column names the method or constant in the decompiled Android app that issues each command. Those names are the way back into the decompilation if you ever need to re-derive a layout; keep them.

Read commands

cmd	name	request payload	response	app symbol	notes
`0x00`	READ_DEVICE_STATUS	-	8 B: two `u32` LE, observed `{0, 1}`	`CommandIDs.READ_DEVICE_STATUS`	R
`0x01`	READ_DEVICE_ID	-	≥26 B, see below	`performReadDeviceId`	R
`0x02`	SAMPLE_SPECTRUM	-	2 or 3 frames of blob data	`performSpectrum`	R (capture; persists nothing)
`0x04`	READ_TEMPERATURE	-	12 B: three `u32` LE	`performReadTemperature`	R
`0x05`	READ_BATTERY_STATE	-	8 B, see below	`performReadBattery`	R
`0x84`	READ_BLE_ID	-	≥130 B, see below	`performReadDeviceBleId`	R
`0x85`	READ_BLE_STATUS	-	ble status	`CommandIDs.READ_BLE_STATUS`	R
`0x87`	READ_FILE_HEADER	`<I` file_id	exactly 16 B: four `u32` LE	`performReadFileHeader`, via `SCiOBLeService.performReadFileHeader(int)`	R
`0x94`	READ_FILE_LIST	-	n × 8 B `(u32 type, u32 version)`	`performReadFileList`, via `SCiOBLeService.performReadFileList()`	R
`0x9B`	READ_BLE	-	ble config	`CommandIDs.READ_BLE`	R

0x01 READ_DEVICE_ID

offset	size	field
`[0:8]`	8 B	DSP id (hex, lowercase)
`[8:16]`	8 B	unclassified
`[16:24]`	8 B	Aptina id, stored with 16-bit words byte-swapped
`[24:26]`	2 B	firmware version, `u16` LE

device_id - the value the server wants - is the Aptina field with each 4-hex-char group abcd rewritten as cdab, then uppercased: 328045ab1161f198 -> 8032AB45611198F1.

0x04 READ_TEMPERATURE - three u32 little-endian words:

cmos_t = (w0 - 375.22) / 1.4092        # Aptina CMOS sensor, degC
chip_t = w1 / 100.0                    # DSP chip, degC
obj_t  = w2 / 100.0                    # object/surface; 0.0 on this fw-147 unit

The Android app truncates to integer twice: cmos_t_app = trunc(trunc(raw - 375.22) / 1.4092), so raw 404 gives 19, not 20.42. This matters: the calibration temperature rule compares the truncated value. parse_temperature returns cmos_t, cmos_t_app and raw_u32 so nothing is lost.

obj_t reads 0.0 on this fw-147 unit, but is non-zero and tracks the target on at least one fw-138 unit (a hand ~32.6 °C vs ~20 °C for room-temperature targets), so it is unit/firmware dependent, not universally 0. ScioDevice.read_object_temperature() returns it; data points in dev/DEVICE_FUNCTION_REFERENCE.md.

0x05 READ_BATTERY_STATE

offset	type	field
`0`	`u16`	charge %
`2`	`u8`	health %
`3`	`u8`	health status
`4`	`u16`	charging status
`6`	`u16`	voltage, mV (divide by 1000)

An early notebook read the charging status as 0 = not charging, 4 = full, 6 = battery error, anything else = charging. Treat that as an unverified hypothesis: the code carrying it masked the byte with & 3 before comparing against 4 and 6, so its own branches were unreachable and the mapping was never exercised. parse_battery returns the raw value and interprets nothing.

0x84 READ_BLE_ID

offset	size	field
`[0:8]`	8 B	BLE id (hex)
`[8:10]`	2 B	BLE firmware version, `u16`
`[10:40]`	30 B	serial, first 30 characters - written by `0x89`
`[40:50]`	10 B	serial, last 10 characters (writer unknown)
`[50:66]`	16 B	user device name - written by `0x91`
`[66:130]`	64 B	`i2s_tag_config`, e.g. `20150812-e:PRODUCTION` - written by `0x93`

Text fields are ASCII, NUL-padded. parse_ble_id returns serial_prefix, device_serial_fragment and their concatenation device_serial. No app reads bytes [10:50]; the full serial of this unit is CPPCA0031C6PF0516009W6404386A1DF1816004A.

The i2s ("image to spectrum") tag identifies the binning-table generation and is mandatory in every server request.

Writing the BLE-ID fields

The three text fields are rewritten with one command each. No app sends 0x89 or 0x93 (they were found on this unit, see below); all three take the same payload as the app's rename: raw ASCII, no padding, no NUL, with the frame length giving the size. The device zero-fills the rest of the field, and the value persists across power cycles - no reset is needed (the app resets after a rename anyway). An empty payload clears the field.

cmd	field	max	frame for this unit's value
`0x89`	serial `[10:40]`	30 B	`01ba891e00` + `CPPCA0031C6PF0516009W6404386A1`
`0x91`	name `[50:66]`	16 B	`01ba910600` + `myScio`
`0x93`	i2s tag `[66:130]`	64 B	`01ba931500` + `20150812-e:PRODUCTION`

Each replies with an empty frame of its own opcode.

The device always reports these values itself - they are only blank if something wiped them. Rewriting them is a repair, not part of any workflow, and a wrong value is dangerous: the i2s tag is matched exactly by the vendor server, so a wrong one is stored permanently and every scan from the device is rejected until it is rewritten. The library therefore guards each write twice:

dev.write_i2s_tag("20150812-e:PRODUCTION", allow_write=True)
dev.write_serial_prefix("CPPCA0031C6PF0516009W6404386A1", allow_write=True)

allow_write=True is required, as for every write;
the current value is read first and a warning is shown with the current and the new value; the write goes ahead only if the user types yes (or a confirm= callable returns True). Otherwise nothing is sent.

If the value is already correct nothing is written or asked. The result has previous, readback and verified (from a fresh 0x84 read); nothing resets. Empty and over-long values are refused before anything is sent; longer payloads were never tried on 0x89/0x93. Only write a value you know is the device's original, e.g. from its own earlier records.

0x88 and 0x95 also accept a payload (empty, 30 B or 40 B) and acknowledge it the same way, but change nothing in this record, nor in the device id, file list, file headers or auto-off timer, before or after a power cycle. They are writes of something not visible here and stay forbidden.

Incident and repair (2026-09-07 / 2026-10-04)

From 2026-09-07 every READ_BLE_ID on this unit returned [10:40] and [66:130] zeroed. Reads at 15:52-15:58 that day had the tag; the 18:50 series did not. In between, the reserved-opcode sweep (notebook 03) sent 0x88, 0x89, 0x93 and 0x95 with empty payloads - i.e. it cleared the serial prefix and the tag. The missing tag made every scan unprocessable without a manual fill-in; a reset did not bring it back.

On 2026-10-04 the fields were mapped and restored over BLE with dev/scripts/restore_ble_id_record.py, one write per run, each between full read-only snapshots (BLE-ID record, device id, timer, file list, ten file headers). Raw evidence: 01_rawdata/probe_logs/ble_id_restore_20261004_*.

time	write	effect
14:29:54	`0x93` + tag (21 B)	tag restored at `[66:130]`, nothing else changed
14:30:50	`0x95` + full serial (40 B)	none
14:32:21	(power cycle, read only)	tag still present - persistent
14:33:21	`0x95` + serial prefix (30 B)	none
14:33:40	`0x88` + serial prefix (30 B)	none
14:33:59	`0x89` + serial prefix (30 B)	serial restored at `[10:40]`, nothing else changed

Do not send 0x88, 0x89, 0x93 or 0x95 with an empty or blind payload. The probe refuses all four.

After the repair the device reports both values again, and a scan captured over BLE at 14:44 carried them straight from the device and was processed by the server (331-point spectrum).

The library does not substitute a tag from older records: the device is the source of truth. Should the tag be blank (this or another unit), read_device_info flags i2s_tag_missing, the notebooks say so, and session.process refuses the record. Repair the device with write_i2s_tag first, using the value from its own earlier records.

0x87 READ_FILE_HEADER - request struct.pack("<I", file_id). The response is always 16 bytes: (type, size, version, checksum) as four u32 LE. Appending an offset/length to the request is ignored - there is no readback path (see §5).

0x94 READ_FILE_LIST - a flat array of 8-byte (u32 file_type, u32 version) entries covering the 87-95 band.

Write / state-changing commands

Never sent by this project. ScioUSB refuses them unless allow_write=True.

cmd	name	app symbol	notes
`0x03`	SET_SAMPLE_SETTINGS	-	unused by the app
`0x07`	PARAMETER_SET	`CommandIDs.PARAMETER_SET`	W
`0x0B`	SET_INDICATION_LED	-	W, 9-byte payload
`0x0E`	READY_FOR_WR	`performReadyForWR`	W, LED/UX hint so the device button can trigger a white reference; firmware ≥ 144
`0x11`	CLEAR_READY_FOR_WR	`performClearReadyForWR`	W, counterpart of `0x0E`
`0x81`	FILE_DOWNLOAD	`performFileDownload`	W, host->device only - it uploads firmware, it does not read it
`0x83`	RESET_DEVICE	`CommandIDs.RESET_DEVICE`	W, disruptive
`0x88`	(none; `Cmd.UNKNOWN_WRITE_88`)	-	W, accepts a payload, no visible effect
`0x89`	WRITE_SERIAL_PREFIX (found on this unit)	-	W, BLE-ID `[10:40]`, ASCII ≤ 30 B, persistent
`0x91`	WRITE_USER_DEVICE_NAME	`CommandIDs.WRITE_USER_DEVICE_NAME`	W, BLE-ID `[50:66]`, ASCII ≤ 16 B
`0x93`	WRITE_I2S_TAG (found on this unit)	-	W, BLE-ID `[66:130]`, ASCII ≤ 64 B, persistent
`0x95`	(none; `Cmd.UNKNOWN_WRITE_95`)	-	W, accepts a payload, no visible effect
`0x9A`	WRITE_BLE	`CommandIDs.WRITE_BLE`	W

Declared but unimplemented (firmware 147)

All probed on real hardware:

cmd	name (app symbol)	observed
`0x06`	READ_EVENT_LOG (`CommandIDs.READ_EVENT_LOG`)	X no response
`0x08`	PARAMETER_GET (`CommandIDs.PARAMETER_GET`)	X no response
`0x09`	BIST (built-in self test)	X no response
`0x8A`-`0x8F`	reserved band	no response

0x88, 0x89, 0x93 and 0x95 from the same band turned out to be writes (table above); probing them with empty payloads cleared two BLE-ID fields.

4. Scan data anatomy

SAMPLE_SPECTRUM (0x02) returns one frame per blob. How many depends on the firmware (DeviceInfo.java):

firmware < 136 -> 2 frames (dark, sample);
firmware ≥ 153 with gradient sampling disabled -> 2;
otherwise -> 3 (dark, sample, gradient).

Wire order is dark, sample, gradient.

Sizes depend on the i2s generation:

i2s tag	sample	dark	gradient
`…-e:…` (this unit)	1800 B	1800 B	1656 B
`…-o:…` (older)	1800 B	1800 B	1416 B

Each blob is:

[0:4]   u32 LE  status / type   0x00 for sample and dark, 0x6E (110) for gradient
[4:8]   u32 LE  varies per blob - see below; NOT proven to be a nonce
[8:]            body: an exact multiple of 16 bytes, ~7.9 bits of entropy per byte

The first word is what the vendor's own source calls status: the app reads it as getU32(0) of the sample blob (ScioInternalDevice.java:1085, 2017 build), little-endian. 0 means a healthy scan. Why the gradient carries 0x6E there is unexplained.

The second word is, measured across the whole corpus, indistinguishable from a fresh 32-bit random draw per blob: 300 unique bodies map one-to-one onto 300 unique values with no reuse anywhere; it is not a counter (18 of 29 ascending in a burst of scans 2.13 s apart), not correlated with time (|r| <= 0.11), and not derived from the body (0/291 across eight checksum candidates). Three independent draws per scan, one per blob.

Handy check when reading base64 by eye: sample / dark / white / white-dark all begin AAAAA (leading u32 = 0), while the two gradient blobs begin bgAAA (0x6E). A blob whose base64 starts with anything else is not a SCiO scan blob.

What is established about the body. It is high-entropy (7.89 bits/byte, on the random control), 16-byte aligned, fixed-length regardless of content, and statistically featureless: the pooled byte histogram is flat (chi-square 256.9 on df 255) and not one of the 1792 byte positions deviates more than 4 sigma from uniform. Thirty captures of a physically unchanged target, whose spectra agree to 2.1 % worst case, produce bodies at 0.49986 bit distance from each other with no shared 16-byte block anywhere in the corpus.

What is not established: the class of the transform. Full avalanche looks like encryption, but it does not discriminate - the 2.1 % agreement is measured after binning ~2.7 pixels per band, per-pixel sensor noise differs everywhere, and an entropy coder avalanches on any input difference too. Flat byte histograms are equally what good compression produces.

What the server reveals. Reflectance is a sample-domain quantity divided by a white-domain quantity: swapping sample and white components gives a response table that is multiplicatively separable to ~1e-16, with a stable non-unity self-response factor C(λ). Dark handling is not a simple subtraction, per-band affine map or shared gain. Sample, dark and white blobs are integrity-protected (any tested change to the second header word or body is rejected with Bad_sample_signature); the status bit and gradient edits are accepted and leave the spectrum unchanged. None of this recovers the domain vectors themselves. Evidence: dev/RECOVERY_STATUS.md.

The class remains undetermined. The vendor's vocabulary hints at compression: the i2s tag is called compression_version in the API, the parameter carrying it is literally named i2sTag in the un-obfuscated 2017 build, and the app ships an UnsupportedCompressionConversion error - you cannot convert between encryptions. A supervised test on 92 paired records finds no body bit, byte or 16-bit word that correlates with the returned spectrum, and no held-out linear predictor, although the same test detects a single fixed-position count field in synthetic data of this size (dev/HANDOVER.md). That disfavours a simple fixed-position coder, but does not prove encryption. Encryption cannot be confirmed by software alone: the device could encrypt and the server decrypt, and the Android client is a verified byte-for-byte pass-through that would look identical either way. See dev/README.md.

5. Firmware and calibration files

READ_FILE_LIST / READ_FILE_HEADER expose the device's stored files.

A READ_FILE_HEADER response is (type, size, version, checksum). Read from this fw-147 unit (01_rawdata/device_files/):

id	name	size	version	checksum
87	`ble_runtime`	119233 B	125	12925168
89	`ble`	absent (`0xFFFFFFFF` in every word)
90	`dsp_boot`	7284 B	17	688456
91	`dsp_dec`	14600 B	12	1548938
92	`dsp_op`	32628 B	147	4151168
100	`deadPixelsIndices`	1714 B	3	97267
101	`centers`	96 B	3	6080
102	`bins`	140 B	3	13587
103	`nPixelsPerBin`	1166 B	3	36371

dsp_op's version equals the device firmware version, and the three table files share version 3. Use the size and checksum to validate any flash dump.

The 2017 Android enum calls the BLE image type 89; this fw-147 device reports type 87 as a 119233-byte BLE image. Both observations are kept in protocol.FIRMWARE_FILES rather than reconciled away.

Only headers can be read back. Bodies cannot. FILE_DOWNLOAD (0x81) writes to the device; READ_FILE_HEADER returns 16 bytes and ignores any offset. No body-read operation was found in the reviewed command paths, and the reserved band was probed without finding one. This is why offline decoding is blocked: the transform - and any key - lives in firmware we do not have. It may run on the BF512 (dsp_op and friends) or on the CC2540 BLE SoC (file 87, which has hardware AES); which one is unproven. Routes to the code are a boot-flash dump, debug access to either processor, or a phone that cached the files (see documentation/HARDWARE_ACQUISITION.md).

Hardware (full reference with photos: documentation/HARDWARE.md). A SparkFun teardown shows an ADSP-BF512KBCZ-3 Blackfin DSP (no on-chip program flash), AS4C8M16SA 128 Mbit SDRAM, a CC2540F256 BLE SoC, a probable ADP5062 charger, two unidentified small QFNs and no visible standalone flash chip. The sensor has 12 receptors in a 3 × 4 grid, each with its own filter, aperture and lens over a photodiode array. All the device's reported files together (176,861 B) would fit in the CC2540's 256 KB flash. That makes it plausible, but unverified, that the CC2540 hosts the DSP images and boots the BF512. See documentation/HARDWARE_ACQUISITION.md.

Both SCiO apps cached these files in Android SharedPreferences (/data/data/com.consumerphysics.consumer/shared_prefs/) as base64 with a 4-byte little-endian checksum prefix, listed in a firmware.file.names string-set. The consumer app deletes them after a completed upgrade, so a phone that never finished one is the best candidate. dev/scio_offline/firmware.py extracts them. (For this repository's unit the original phone and backup no longer exist.)

6. White reference and calibration

A white reference (WR) is the same SAMPLE_SPECTRUM command taken with the SCiO in its cover, on the white surface inside. It is stored, reused across scans, and sent to the server with every scan.

The client decides, not the server

This was verified both by reading the decompiled apps and by testing the live API. isCalibrationNeeded() walks these rules in order, each skipped when its threshold is <= 0:

result	condition
`NEVER`	no WR stored, or missing i2s tag / device id
`TIME_THRESHOLD`	WR older than `time_diff`
`EXCEED_SCANS_LIMIT`	`scans_since_calibration >= scan_diff`
`TEMP_THRESHOLD`	\|current CMOS temp − WR temp\| > `temp_diff`
`NO_NEED`	otherwise

Mirrored exactly by store.calibration_status / store.calibration_report. The temperature compared is the app's double-truncated value (see 0x04 above), and the WR temperature is the mean of its before/after readings.

Thresholds come from the server but are only advice:

GET /v1/device/calibration_thresholds?device_id=<DEVICE_ID>
-> {"thresholds": {"time_diff": …, "scan_diff": …, "temp_diff": …,
                   "min_scans_for_new_batch": …}}

time_diff is in minutes. For this device the live values are time_diff = 1e9, scan_diff = 1e9, temp_diff = 10000 - every rule effectively disabled, so a white reference is needed only when none exists. min_scans_for_new_batch is parsed by the app but never read by isCalibrationNeeded; it is stored, labelled unused.

The server never rejects a scan for a stale white reference. 62 stored scans plus 2 deliberately mismatched cross-pairs were replayed, including a WR 2289 days old and a pairing whose WR post-dates its scan. All 64 returned spectra. The experiment is in 02_processed_data/replay_experiment/; the tool is tools/replay_all_scans.py.

There is a separate, purely advisory quality check:

POST /v1/device/{device_id}/user_calibration
-> {"calibration": {"is_valid": true, "calibration_id": "<uuid>"}}

The app stores the WR before calling it, and a false verdict never un-stores it.

Storage

White references are written as YYYYMMDD_HHMM_calibration.json, with the same timestamp inside. They are never overwritten: a same-minute save becomes ..._calibration_2.json, and "latest" is computed by sorting, so the whole calibration history is preserved. Legacy wr_*.json files are still read.

Every canonical scan record embeds its own WR blobs, temperatures and thresholds, so a single scan file is self-contained.

7. Server API

Base: https://api.consumerphysics.com. Requests carry X-SCiO-Client-Version: Android 1.3.8.554.

Login (OAuth implicit flow)

POST /oauth/login       (email + password)
  -> 302 /oauth/authorize
  -> 302 <redirect_uri>#access_token=<token>&expires_in=14&…

Tokens are short-lived - expires_in=14. Fetch a fresh one per request; session.process_pending does. Credentials are asked for once and stored in .scio_credentials.enc (Fernet, PBKDF2-HMAC-SHA256 with 200 000 iterations, key derived from getpass.getuser() | platform.node()), which is gitignored and does not travel between machines.

An inactive account returns HTTP 403 {"user_status": "inactive"}. That is an account state, not a bug - Consumer Physics can reactivate it.

Scan -> spectrum

POST /v2/consumer/spectro-scan

All twelve fields are required unless noted:

field	value
`device_id`	uppercase 16 hex, from `0x01`
`sampled_at`	ISO 8601 with offset and milliseconds
`sampled_white_at`	ditto, from the white reference
`scio_edition`	`"scio_edition"`
`i2s_tag_config`	e.g. `20150812-e:PRODUCTION` - rejected if empty
`mobile_mac_address`	required; this project sends `02:00:00:00:00:00`
`sample`	base64 blob
`sample_dark`	base64 blob
`sample_white`	base64 blob
`sample_white_dark`	base64 blob
`sample_gradient`	base64, optional (3-frame firmware)
`sample_white_gradient`	base64, optional
`widget_scan_attributes`	`[]`

One further field appears in captured request bodies but is not required: mobile_GPS (latitude, longitude, locality, country, admin_area, address_line), which the phone app attached to every scan. The server accepts scans without it, so this project does not send it.

Base64 must be standard alphabet, =-padded, newline every 76 characters - Android's Base64.DEFAULT. URL-safe base64 is rejected.

Response:

{"_type": "POST spectroscan2",
 "spectrum": [ …331 floats… ],
 "wavelengths": {"start": 740, "steps": 1, "num_WL": 331}}

giving reflectance at 740-1070 nm in 1 nm steps.

Error taxonomy seen in the apps and on the wire: invalid_scan, low_signal, high_ambient, material_unknown, novelty, InvalidUsage (a malformed or missing field), plus plain HTTP 401/403 for auth and transient 502s.

Firmware upgrade

GET /v1/device/{ble_id}/firmware-upgrade?…
-> {"new_version": null}     # device already up to date

The only firmware-serving endpoint, and it serves nothing for a current device. Blobs, when present, are base64 with a 4-byte little-endian checksum prefix.

8. Practical gotchas

device_id must be UPPERCASE. Lowercase returns HTTP 404 - proven, see 01_rawdata/probe_logs/calibration_v1_thresholds_lowercase.json.
An empty i2s_tag_config is rejected outright (InvalidUsage). Thirty scans in this repository were captured with an empty tag; they were unusable until the tag was filled in from the device's white reference. The tag can be cleared on the device by an empty 0x93 write and restored with write_i2s_tag (section 3) - after a warning and confirmation, since a wrong tag is rejected too. Capture does not substitute one.
Base64 must be standard, not URL-safe. One legacy group in this repo stored URL-safe unpadded base64; the canonical records re-encode from the authoritative hex.
The device must be steady blue. Pulsing = idle/charging = silent USB. Power-cycle it.
Access tokens expire in seconds. Fetch one per request.
The white-reference decision is yours, not the server's. If you care about photometric accuracy, take a fresh WR; the server will happily accept a six-year-old one and return a meaningless number.
A smooth curve is not a decoded spectrum. Random bytes read as float32 look smooth. Validation requires agreement with a held-out server spectrum.

9. Repository layout

path	contents
`src/scio/`	the working library: `protocol`, `usb`, `probe`, `store`, `cloud`, `credentials`, `session`, `logscan`, `corpus`, `reference`, `paths`
`01`-`03` notebooks	the three live notebooks (see §1)
`dev/`	offline-decoding research (`scio_offline`, scripts, `notebooks/`, its own tests) - start at `dev/HANDOVER.md`; full log in `dev/RECOVERY_STATUS.md`
`dev/notebooks/superseded/`	earlier decoding attempts, each labelled superseded, kept as the exploration record
`tools/`	`replay_all_scans.py`, `analyze_scan.py`, `check_public_safety.py`
`tests/`	offline tests for the working pipeline (`pytest tests/`)
`01_rawdata/scans/`	canonical `scio-scan/2` records - self-contained, ready to process
`01_rawdata/` (rest)	the original captures, untouched: `log_files/`, `log_extracted/`, `scan_json/`, `scan_json_calibration/`, `device_files/`, `probe_logs/`
`02_processed_data/`	records plus their spectra, and the replay experiment
`documentation/`	hardware reference and teardown photos, RE log, hardware acquisition guide, handoff, datasheets, patents
`archive/`	superseded notebooks (`notebooks/`), scripts and raw notes (`more_info/`), frozen - see `archive/README.md`

01_rawdata/scans/ holds 97 records: 26 from 2020/2021 app logs (each carrying the spectrum the server returned at the time - a genuine regression target), 17 more recovered from logs that were never mined, 18 from a 2023 USB series, and 36 from 2026. The canonical records are additional copies; every original file is byte-identical to what it was before.

Scans converted from the 2023 series have no white reference of their own and borrow a later one. The server returns a spectrum, but it is not a calibrated measurement, and every such record says so in provenance.notes.

Tests

pytest tests/          # working pipeline - passes with dev/ deleted
pytest dev/tests/      # offline-decoding research

The two suites are deliberately independent; that is what keeps the strands decoupled.

License and credits

This repository is licensed under the GNU General Public License v3.0 - see LICENSE for the full text. That covers everything written here: the scio and scio_offline libraries, the notebooks, the tools, the tests and this documentation.

Not covered by the GPL, and not ours to license:

Consumer Physics documents and trademarks. SCiO, Consumer Physics and related names, logos and marks belong to their owners. The patents reproduced under documentation/ (US9377396.pdf, US10330531.pdf) are their filings, included for reference only.
Other third-party documents, likewise reference-only: the Analog Devices ADSP-BF51x datasheets (ADSP-BF512.pdf, Rev D, and adsp-bf512-514-516-518.pdf, Rev E), the TI CC2540F256.pdf datasheet, and the notes in archive/more_info/ that came from other people.
SparkFun teardown photos in documentation/teardown/, shown in documentation/HARDWARE.md: © SparkFun Electronics, from "SCiO Pocket Molecular Scanner Teardown" by JOEL_E_B, licensed CC BY-SA 4.0, unmodified.
Captured device data under 01_rawdata/ is measurement output from the author's own unit, not third-party material - it is shared under the same GPL as the rest.

Nothing here redistributes Consumer Physics firmware, application code or decompiled sources. This is independent interoperability work on hardware the author owns, using the vendor's own documented API and the author's own account.

Facts here were established from: the decompiled Android apps (consumer and researcher/Lab), live probing of a fw-147 device over USB, captured app logs, the live server API, the Sparkfun SCiO teardown, and the ADSP-BF512 datasheet. Corrections are welcome - especially anything that turns a stated hypothesis into a fact, or refutes one.

Navigation Menu

Repository navigation

About

Repository files navigation

The SCiO spectrometer: a technical reference

Contents

1. Quick start

2. Transport

USB CDC

Bluetooth LE

Framing (both directions, both transports)

3. Command reference

Read commands

Writing the BLE-ID fields

Incident and repair (2026-09-07 / 2026-10-04)

Write / state-changing commands

Declared but unimplemented (firmware 147)

4. Scan data anatomy

5. Firmware and calibration files

6. White reference and calibration

The client decides, not the server

Storage

7. Server API

Login (OAuth implicit flow)

Scan -> spectrum

Firmware upgrade

8. Practical gotchas

9. Repository layout

Tests

License and credits

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
01_rawdata		01_rawdata
02_processed_data		02_processed_data
archive		archive
dev		dev
documentation		documentation
src/scio		src/scio
tests		tests
tools		tools
.gitignore		.gitignore
01_scio_scan_to_spectrum.ipynb		01_scio_scan_to_spectrum.ipynb
02_scio_device_health.ipynb		02_scio_device_health.ipynb
03_scio_probe.ipynb		03_scio_probe.ipynb
LICENSE		LICENSE
README.md		README.md
conftest.py		conftest.py
scio_env.yml		scio_env.yml

Latest commit

History

Folders and files

Repository files navigation

The SCiO spectrometer: a technical reference

Contents

1. Quick start

2. Transport

USB CDC

Bluetooth LE

Framing (both directions, both transports)

3. Command reference

Read commands

Writing the BLE-ID fields

Incident and repair (2026-09-07 / 2026-10-04)

Write / state-changing commands

Declared but unimplemented (firmware 147)

4. Scan data anatomy

5. Firmware and calibration files

6. White reference and calibration

The client decides, not the server

Storage

7. Server API

Login (OAuth implicit flow)

Scan -> spectrum

Firmware upgrade

8. Practical gotchas

9. Repository layout

Tests

License and credits

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages