Skip to content

Charge measured context changes by character difference - #233

Draft
norci wants to merge 4 commits into
lambda-symbolics:masterfrom
norci:fix/context-measurement
Draft

norci wants to merge 4 commits into
lambda-symbolics:masterfrom
norci:fix/context-measurement

Conversation

@norci

@norci norci commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

A measured request keeps its occupancy until the measured policy or the measured covered history prefix changes, and every later difference is charged as a signed character difference against that measurement.

Design

graph LR
  S["conversation-context-usage-sample (format 3)"]
  B["conversation-context-usage-baseline"]
  E["context-estimate-known-tokens"]
  A["provider-context--matched-input"]
  H["provider-context--header-charge"]
  S --> E
  S --> A
  B --> E
  E --> H
  A --> H
Loading

The sample holds nine keys: :format, :policy, :chargeables, :history-count, :history-digest, :wire-fields, :wire-characters, :ephemera-characters, :tokens. Both readers price their difference against it through provider-context--header-charge, which walks the union of measured and current (key digest characters) triples so a component that disappeared contributes zero characters. Prompt and tool components are priced from configuration-derived characters on the estimate path and from transported wire-field characters on the admission path.

Conversation storage validates the sample shape and keeps the recorded :tokens and :history-count of every :context-usage record as a baseline, whatever shape its evidence has. The estimate path uses that baseline when no sample is usable, so a conversation loaded from durable records resumes from the last measured total and charges only the history appended since.

One rejection condition: a changed policy (class, model, model-provider, endpoint) or a rewritten covered history prefix (digest or count), because the tokenizer identity and the covered prefix have to be provable before a measurement can be extended.

Why

Correctness

  • One accountant: provider-context--matched-input on admission and context-estimate-known-tokens on the estimate path both price through provider-context--header-charge.
  • One rejection reason: a changed policy or a rewritten covered history prefix. Every other difference is charged, never guessed.
  • The sample is bound to the tokenizer that produced it, so a model change cannot inherit old totals.

Cost

edit before here
one word of the system prompt the whole prompt component, 11,922 tokens for a 47,686-character prompt the edited characters
a tool description the whole tool header, 59,386 tokens for a 237,543-character header the difference, with a credit when tools are removed
request-local input present measurement disabled input charged by character difference
every request re-encoded in full to compare wire fields field character sets compared

Durability

  • The sample persists as format-three evidence with its policy and covered prefix.
  • A recorded total resumes the estimate even when its evidence has an older shape: a 940,185-character history that estimates 235,046 tokens cold resumes at 203,116 from its last measurement.

Evidence

  • ./script/check --suite context-admission --jobs 8: 27 cases, 268 checks, 0 failed.
  • ./script/check --jobs 12: 1,788 cases, 32,438 checks, 2 failed. Both fail without these changes: test-world-server-restart-checkpoint reproduces on f04e6142, and test-lisp-worker-protocol needs an SBCL source root this host does not provision.

Cost

Before any measurement exists the estimate is characters divided by four. Once one exists it resumes from the recorded total, and history appended since then is priced by serialized characters rather than by the tokenizer's own count.

@norci
norci marked this pull request as draft November 15, 2026 11:03
@norci
norci marked this pull request as ready for review November 15, 2026 13:09
@norci

norci commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

this bug fix has been verified on my machine.

I created an extension for billion-context (https://github.com/norci/config-autolith/blob/main/extensions/billion-context.lisp )
it works perfectly.

@norci
norci marked this pull request as draft November 15, 2026 14:11
Evidence written before component charging is no longer usable for pricing
prompt and tools, so a reloaded conversation fell back to dividing the stored
characters of every retained item by four. That re-priced history the provider
had already counted and pushed the estimate over the compaction limit with no
new work in the conversation.

Keep the measured total and history length beside the sample and start the
estimate from them, charging only the history appended after the recorded
count.
@norci
norci force-pushed the fix/context-measurement branch from 487da6b to f357b5b Compare November 15, 2026 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant