Parse FL Studio 2024/2025 playlist items (80-byte records) - #205
Parse FL Studio 2024/2025 playlist items (80-byte records)#205CryptoJones wants to merge 2 commits into
Conversation
FL 2024/2025 (21.2+ / 25.x) grew each playlist item to an 80-byte record: a 20-byte tail (_u4) after FL 21's 28-byte tail (_u3). PlaylistEvent only knew the 32- and 60-byte layouts, so these events failed the size check and their clips were dropped (demberto#177, demberto#199, demberto#200). Add the _u4 block gated on a `fl2025` param, extend SIZES to include 80, and detect it by the fixed size that divides the payload — preferring the 60-byte reading on a 60/80 common multiple so files that parsed before are unaffected. The record layout was verified byte-for-byte against FL Studio 2025 saves, and a synthetic-event test asserts the 80-byte records parse and round-trip. Known limitation (documented in-code): clips edited in FL grow past 80 bytes, so a heavily-edited project's Playlist event can be variable-length and still not match a single fixed size.
|
Additional findings: FL26 needs two more fixes on top of this PR I hit the same wall parsing FL26.1.1 projects (both re-saved FL26 files and legacy FL20.5 files on my rig). This PR's 80-byte layout is correct for FL 2024/2025, but FL26 grows each playlist item to 88 bytes — a further 8-byte tail (_u5) after the 20-byte _u4. Verified byte-for-byte: 32 base + 28 _u3 (FL21) + 20 _u4 (FL2024/25) + 8 _u5 (FL26) = 88.
elif id < TEXT:
if id == EventEnum(0xAC):
size = c.VarInt.parse_stream(stream)
value = stream.read(size)
else:
value = stream.read(4)
With all three changes (this PR's 80-byte + the 88-byte extension + 0xAC varint + phantom-pattern skip) I parseFL26 projects fully: 10 channels / 616 notes, and a 15-channel project at 1302 notes, with legacy FL20.5 filesunaffected (regression-checked). Happy to open a PR with the combined diff + tests if you're interested. Sorry for AI sloping, as I'm really poor at progamming, but it works. |
FL 2025+ can write a small NotesEvent into the project header, before any PatternID.New. Patterns.__iter__ buckets events by the last-seen New, so that event lands in the cur_pat_id = 0 bucket and was yielded as a pattern - one whose every property raises, because Pattern.iid reads PatternID.New and that bucket has none. It also made __iter__ disagree with __len__. Buckets keyed by a real pattern always contain the New that keyed them, so a bucket without one is not a pattern and is skipped. Reported by @Meowrium on demberto#205 and reproduced on a real FL 25.2.5 save: iteration yielded 1000 patterns against a reported length of 999, the extra bucket holding a single PatternID.Notes, and .iid raised KeyError. It now yields 999 with working properties. Checked against 34 real .flp files (FL 24.x and 25.2.5): that one file changes behaviour and the other 33 parse byte-identically. The new test fails without the fix. Also reformats tests/test_arrangement.py, which the previous commit left failing the pinned black 23.3.0 hook. @Meowrium reported two further FL 26 changes on the same thread - 88-byte playlist records and a varint-length 0xAC. Both are held back from this PR pending a real FL 26 save; see the thread for the measurements. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016HrZZMUyXqTmMWKZo3zAfU
|
Thanks — this was a genuinely useful report, and two of the three findings hold up. I have FL Studio 25.2.5 and a corpus of 34 real 3. The phantom pattern — confirmed, fixed, and pushedReproduced exactly on a real FL 25.2.5 save. The Implemented slightly more generally than "skip the id-0 bucket": any bucket with no 2. The 0xAC varint change — the premise doesn't hold for FL 2025This is the one to flag. The patch as written applies the varint read unconditionally, on the reasoning that "old files never contain 0xAC, so gating it as a varint read is regression-free." FL 2025 files do contain Read as a varint, that leading That's fixable with a version gate — But gating alone isn't enough to ship it, because of two things I hit afterwards:
Doing this properly needs a dedicated event class that skips the fixed-size check and serialises with a varint length — a change to 1. The 88-byte record — same shape of problem80 and 88 share multiples and real projects land on them: one of my saves has a 17600-byte playlist, which is both What would unblock bothYou mentioned you have FL 26 projects. Could you attach a few small saves? Specifically:
A zip of those would let both changes land with real fixtures instead of synthetic ones, matching how the rest of Nothing here is meant as a knock on the report — the phantom pattern is a real bug I wouldn't have found, and the 0xAC lead was right about something changing, just not about it being safe to apply unconditionally. AI disclosure
That review is what caught the serialisation and payload-size problems described above; the first draft of this change had both and would have shipped them. Proudly Made in Nebraska. Go Big Red! 🌽 https://xkcd.com/2347/ |
…tom-pattern skip Upstream review (PR demberto#205) verified our FL26 patches against a 34-file FL 25.2.5 corpus and found two regressions in the original unconditional fixes: - Event 0xAC is varint-length only in FL26. FL 2024/2025 also contain 0xAC, always as a fixed 4-byte DWORD (27 occurrences / 34 files); an unconditional varint read derailed 8/12 real FL25.2.5 projects (InvalidEventChunkSize). ProjectID.FLVersion is the first event in the chunk (34/34), so gate the varint read on FLVersion >= 26. - PlaylistEvent 80B (FL 2024/25) and 88B (FL26) records share payload multiples (17600 = 220*80 = 200*88), so length divisibility cannot pick the layout; version-gating is required. FLVersion now flows into PlaylistEvent(id, data, version). 88B = 32+28(_u3)+20(_u4)+8(_u5), so fl26 implies fl2025/new. - Phantom-pattern skip generalized per upstream: any bucket without a PatternID.New is skipped (not just id-0), which also survives a pattern legitimately numbered 0. Verified: full test suite passes; 侠/翩然 (FL26.1.1) and 1.3.flp (FL20.5) still parse with identical channel/pattern/event counts; real FL26 0xAC payload measured as varint length 1 (0x01).
|
Empty FL 2025/2026 project files + some clip edits as requested. |
FL Studio 26 changed two things in the project header that broke parsing entirely (NoModelsFound / StringError on every FL 26 project): 1. Event 0xAC (DWORD+44, ProjectID range) is usually still a plain 4-byte value (matches every occurrence in FL 2024/2025 projects and most occurrences in FL 26 projects), but at least once per FL 26 project it is followed by a 1-byte length and that many bytes of null-terminated UTF-16LE text (seen holding a build/generator tag such as "FL Studio 26.1.5.5618.5618"). The two forms are told apart by the top two bits of the 4th value byte. Verified byte-for-byte against multiple real FL 26 project files. 2. When a project has more than one custom display/track group, FL 26 bundles all of their names into one new container event, and reuses the same numeric ID (194) already claimed by ProjectID.Title. This collision cannot be disambiguated from the ID alone with pyflp's current single-pass, stateless parsing, so rather than crash the whole parse, any text-typed event that fails to decode as text now degrades to a raw UnknownDataEvent with a warning (matching the existing VSTPluginEvent unknown-marker warning style), instead of raising and aborting parse() entirely. See docs/format-notes.md in this fork for the full byte-level traces both of these were reverse engineered from. Verified against a small corpus of FL 11/12/24.2/26 projects: FL 11/12/24.2 files are unaffected (byte-identical parse results), FL 26 projects no longer fail immediately in the project header (they now progress to the already-known-and-reported 80/88-byte Playlist record issue, see demberto#205).
FL Studio 26 changed two things in the project header that broke parsing entirely (NoModelsFound / StringError on every FL 26 project): 1. Event 0xAC (DWORD+44, ProjectID range) is usually still a plain 4-byte value (matches every occurrence in FL 2024/2025 projects and most occurrences in FL 26 projects), but at least once per FL 26 project it is followed by a 1-byte length and that many bytes of null-terminated UTF-16LE text (seen holding a build/generator tag such as "FL Studio 26.1.5.5618.5618"). The two forms are told apart by the top two bits of the 4th value byte. Verified byte-for-byte against multiple real FL 26 project files. 2. When a project has more than one custom display/track group, FL 26 bundles all of their names into one new container event, and reuses the same numeric ID (194) already claimed by ProjectID.Title. This collision cannot be disambiguated from the ID alone with pyflp's current single-pass, stateless parsing, so rather than crash the whole parse, any text-typed event that fails to decode as text now degrades to a raw UnknownDataEvent with a warning (matching the existing VSTPluginEvent unknown-marker warning style), instead of raising and aborting parse() entirely. Verified against a small corpus of FL 11/12/24.2/26 projects: FL 11/12/24.2 files are unaffected (byte-identical parse results), FL 26 projects no longer fail immediately in the project header (they now progress to the already-known-and-reported 80/88-byte Playlist record issue, see demberto#205).
What
FL Studio 2024/2025 (21.2+ / 25.x) grew each playlist item to an 80-byte record — a 20-byte tail (
_u4) after FL 21's 28-byte tail (_u3).PlaylistEventonly knew the 32- and 60-byte layouts, so FL 2024/2025 playlist events fail the size check and their clips are dropped. Fixes #200 and the same underlying cause behind #199 / #177.The record
Full 80-byte layout, verified byte-for-byte against FL Studio 2025 (25.x) saves:
item_indexselects the item type:< 20480references a channel by iid (an audio clip if that channel'sChannelID.Type == 4, an automation clip if== 5);>= 20480is a pattern clip (pattern_base + pattern#).The change
_u4gated on a newfl2025struct param; extendSIZESto include80.Known limitation (documented in-code)
A freshly placed clip is exactly 80 bytes, but clips edited in FL (fades, slices, resizes) carry extra per-clip data and grow past it — so a heavily-edited project's Playlist event can be variable-length and still not match a single fixed size. Fully general handling would need the project's FL version to disambiguate; this PR fixes the common (fresh / programmatically-written) case without regressing anything.
How this came up
I hit this building FL-Studio-MCP-Server — an MCP server that lets Claude Code drive FL Studio and generate/edit
.flpprojects, with PyFLP powering its offline route. Writing FL 2025 audio-clip arrangements is what surfaced the 80-byte record, so the parse fix belongs upstream. Thanks for PyFLP — it's a joy to build on. 🙏🤖 Generated with Claude Code
Proudly Made in Nebraska. Go Big Red! 🌽 https://xkcd.com/2347/