Skip to content

plugin: carry a rounded token figure into the next unit in te status - #304

Open
Lanre Shittu (Shizoqua) wants to merge 1 commit into
microsoft:mainfrom
Shizoqua:fix-fmt-tokens-rollover
Open

Lanre Shittu (Shizoqua) wants to merge 1 commit into
microsoft:mainfrom
Shizoqua:fix-fmt-tokens-rollover

Conversation

@Shizoqua

Copy link
Copy Markdown
Contributor

fmt_tokens chose the unit before rounding, so a figure that rounds up to 1000 of its unit was shown in it: 999,950 effective tokens as "1000.0k", 999,999,999 as "1000.0M", and 999.6 (effective tokens are weighted, so they need not be whole) as "1000".

Fix: the unit is chosen as before, then moved up while the figure rounds to 1000 or more of it, giving "1.0M", "1.0B" and "1.0k". Over 28,000 other figures, random and every whole number up to 3,000, the output is unchanged, and None, NaN and negatives read as before.

Tests, in a new tests/test_fmt_tokens.py:

  • 15 figures shown in their unit, which pass on main and with the fix;
  • 6 figures that round up to a thousand, all failing on main.
python -m pytest tests -n 4 --dist worksteal -rfEs --tb=line
1422 passed, 41 skipped

The skips need Windows, a real agent CLI (Codex, Hermes, opencode) or aider, or a non-root user.

Fixes #300

`fmt_tokens` chose the unit before rounding, so a figure that rounds up to 1000 of its unit was
shown in it: 999,950 effective tokens as "1000.0k", 999,999,999 as "1000.0M", and 999.6 (effective
tokens are weighted, so they need not be whole) as "1000".

The unit is now chosen, then moved up while the figure rounds to 1000 or more of it: those are
"1.0M", "1.0B" and "1.0k". Over 28,000 other figures the output is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

plugin: te status shows 999,950 tokens as 1000.0k, not 1.0M

1 participant