Skip to content

Overnight plan and second pass: objects and layers, one delivery path, Slack buttons, Discord webhook, warm jobs - #39

Merged
robdevops merged 82 commits into
mainfrom
claude/kind-newton-ad0z2q
Oct 15, 2026
Merged

Overnight plan and second pass: objects and layers, one delivery path, Slack buttons, Discord webhook, warm jobs#39
robdevops merged 82 commits into
mainfrom
claude/kind-newton-ad0z2q

Conversation

@robdevops

@robdevops robdevops commented Oct 15, 2026

Copy link
Copy Markdown
Owner

Everything on the work branch since #38: the overnight plan (Q2–Q5, M1–M6, M8, D8), its second pass (P2-1 to P2-8), and today's follow-ups. M7 (the Discord bot) and M9 (dropping gevent) stay deferred.

Structure

  • Config as two objects, config.env (finbot.env) and config.prefs (preferences.conf), read where used, with no star import; each preference's default lives on its Prefs field.
  • Cache and state files in their own modules. State files change under a lock (state.edit), and the watchlist and Sharesight portfolios are objects.
  • lib/util.py split into markets, formats, links and render. Reports move into a lib/reports/ package, each with run(dest=None, **kwargs) and job(argv); the top-level scripts are shims, so cron lines keep working.
  • One delivery path: dialects for markup, a transport per way of posting (Telegram bot, Slack bot and webhook, Discord webhook), and Report(lines, chart, buttons) with deliver(). Messages split by size on every network (chunk_maxlines retired).
  • Inbound: a reader per network, kept apart from how updates arrive (webhooks today, room for pull modes later).
  • One command table makes .help and the Telegram keyboard; one argument reader serves every command.
  • Charts are specs (render.Chart), drawn once and only where they go, cached by the spec's hash. Builders are per kind, and a chart's first view is built the way a period button builds it.
  • Scheduler: a deadline per command, a lock, no gevent. Bounded request-body read; signal work runs in a thread.
  • Tabs everywhere, a docstring on every function, and every line within 120 columns, each checked by a test. Stdout prints removed (systemd discarded them). The largest test files are split by subject.

Chat networks

  • Slack
    • Signing secrets.
    • Scheduled reports post as the bot, charts included, with slackChannelID.
    • ⏳ on a command's message while finbot works on it.
    • Block Kit period buttons and .jobs buttons.
    • A period pressed within 15 minutes of its upload shows that upload again. An upload is deleted once no message shows it and it is past those 15 minutes.
  • Discord: the native webhook, with charts, masked links and no link previews.
  • Telegram: every message and chart goes through one Bot API call, which says why Telegram refused. Period buttons appear only where presses arrive.
  • All networks: the typing notice belongs to the transport and stops before any answer goes out.
  • Docs: a setup guide per network under doc/, and a chat-features table in the README.

Commands and reports

  • Commands are matched by their whole name. The local-exchange retry (exchange_suffix, AX by default) applies to every ticker. .hello and .thanks are gone.
  • Period buttons: no 3M, so five at most (Discord's limit for a row). Every chart's Max is 10 years.
  • No data is a plain answer, not an error. Change figures are signed everywhere. The EOFY reminder lives in reminder.json. Only the scheduled earnings report is pinned.
  • Links go to Yahoo wherever Google has no page; with hyperlinks off, the text alone.
  • Every report gathers its data once, then reports per network: rankings (one ratio table), profile, price (one mode instead of five flags), trades, cal, milestone, rating, shorts, podcasts.
  • Price moves over a period always come from Yahoo's history (the figure its chart draws), for lists and single stocks alike.
  • .jobs lists the system jobs under System, without a number or a button.

Data sources and caching

  • Yahoo: raw answers cached everywhere; an outage is an error, not an unknown symbol; history parsed without pandas, dated where the stock trades.
  • Sharesight: one request function with every check.
  • Markets: one table by market code. TSE is Toronto again, and every market has a Yahoo suffix.
  • Flush: SIGUSR1 flushes the cache (systemctl kill -s USR1 finbot, or finbot-flush.service), then runs both warm jobs straight away.
  • Warm jobs, on the scheduler:
    • sharesight_warm.py every 30 minutes; the holdings are cached 40 minutes.
    • yahoo_warm.py every 8 hours; the price histories are cached 12 hours.
    • The bot's warmer thread is gone.
  • Responsiveness: charts are drawn off the gevent hub, a retried update runs once, and Yahoo details are read from the cache first.

Fixes

  • Slack refusals are read from Slack's answer.
  • Slack links and escapes.
  • Secret paths kept out of logs.
  • Silent replies.
  • The smaller fixes listed in each commit.

Upgrading (test environment and production)

  • preferences.conf
    • Delete alliterate, chunk_maxlines and performance_use_sharesight. finbot still starts with them, and logs a note to delete them.
    • performance_cache_seconds now defaults to 2400. A file that sets 1800 should raise it, or drop the line.
    • New: exchange_suffix, default AX.
  • finbot.env: new and optional, slackSigningSecret and slackChannelID.
  • schedule.cron: a data directory with its own copy needs these lines: 
    14,44 * * * * sharesight_warm.py
07 */8 * * * yahoo_warm.py
  • Slack app (steps in doc/slack.md):
    • Turn Interactivity on, with the same URL as the events.
    • Bot scopes chat:write, files:write, files:read and reactions:write.
    • Then reinstall the app.
  • Optional: sudo cp finbot-flush.service /etc/systemd/system/ && sudo systemctl daemon-reload.

Testing

  • 407 offline tests pass on Python 3.13.5 (.venv/bin/python -m unittest discover -s tests -b).
  • At each refactor step, util/record_outputs.py compared what every command, button and scheduled report posts against the previous commit, so only intended changes went in.
  • Yahoo and Sharesight are blocked from the cloud environment, so the live check is the test environment, which pulls this branch.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX

Generated by Claude Code

claude added 30 commits October 14, 2026 22:56
@claude
Slack replies without a bot token say so; user text is escaped; boole…
8160a37 
…ans take yes/no; graph=False means no charts

- Slack: link previews are turned off in the request body, where Slack reads unfurl_links/unfurl_media (they went in the headers
  and never worked). A reply without slackBotToken logs "ERROR slack: a reply to channel … needs slackBotToken", slack_api raises
  the same instead of a TypeError, and the bot warns once at startup when Slack commands are configured without the token.
- Text a user wrote is shown as text: a .plan entry with a < made Telegram refuse every .plan reply after it, and a first name
  is escaped where it arrives. Slack's already escaped text is not escaped twice.
- A true/false setting takes true/false, yes/no, on/off or 1/0; anything else is a config error (scheduler=yes and graph=Ture
  used to mean False). A blank value means the default, as the README says of finbot.env.
- graph=False turns off every chart, not only the price lines.
- performance names each portfolio from the portfolio list: one with no holdings crashed the report.
- Yahoo: the chart request sends the cookie of its crumb; cookie and crumb are fetched together under the lock; price_history no
  longer writes its PNG cache with the cache off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Dead plumbing removed, quiet tests, a chat-networks table and a code …
d71c1b1 
…map for CLAUDE.md

- Replies never quoted the asking message (the worker passed message_id=None even before the command table): the whole chain
  goes, from bot.py through process_request, the report scripts and write/sendPhoto, with Slack's thread_ts/reply_broadcast.
  So do the chat_id=telegramChatID defaults only an interactive run reads, the "make google cloud happy" returns,
  hbar_chart(threshold=) that only a test used, sharesight.warm_once (the tests call warm()[0]), a fontweight DejaVu Sans
  does not have, a dead emoji, an unreachable except, a stranded unittest.main() and a misnamed test.
- A passing test run prints 0.4 KB instead of 23.7 KB: -b in CLAUDE.md and CI, bot imported before requests so gevent patches
  ssl first, a real job child's output sent to /dev/null, and the two chart warnings fixed at their cause (all-zero bars
  divided by zero; a 2-day span made the date locator complain).
- README: a table of what each chat network can do, near the top; the command list as it is (.value and .rating never
  existed; .beta .buy .sell .compare .pe .fpe .peg .plan were missing); 12-hour cache default; no key counts to keep in step.
- CLAUDE.md: a code map, a test map, conventions and token tips.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
SIGUSR1 flushes the cache: systemctl kill -s USR1 finbot, or systemct…
40c51b7 
…l start finbot-flush

- bot.py deletes its cache files on SIGUSR1 and keeps its state (watchlist, seen trades, ratings, schedule). Yahoo's cookie and
  crumb go together under their lock, and the Sharesight caches refill at once (keep_warm can now be woken). The signal is
  ignored until startup is done, like SIGHUP, and job processes ignore it, since systemctl kill signals every process of the unit.
- finbot-flush.service, a oneshot unit, gives it a systemctl verb; finbot.service needs no change. README: how to flush under
  systemd, Docker or by hand, and cleanup.py 0 with the bot stopped.
- cleanup leaves a temp file written in the last minute (an atomic write in flight) and still deletes a crash's old leftovers.
- A cache file deleted between the age check and the read is a miss, not an error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Charts drawn off the gevent hub; a retried update runs once; report s…
551caa7 
…cripts load less; cache-first Yahoo details

- The bot draws charts in one native thread (util.render_off_hub, from bot.py only). Drawn in the hub, the shorts pictogram froze
  every chat and incoming webhook for 1.3 s; now the hub's worst pause is about 0.1 s. One thread, because several at once
  deadlock matplotlib under gevent's patched locks (tested). matplotlib is imported at start instead of by the first chart.
- A Telegram update or a Slack event that arrives again (Slack resends after 3 s without an answer) runs once, by update_id
  and event_id.
- reports imports pandas and the scheduler where it uses them: a price or performance job process no longer loads pandas and
  every script through charts → reports → jobs (import price: 0.53 s → 0.14 s).
- fetch_detail reads its cache before taking Yahoo's auth lock; data files (the simplywall.st lists, 73 KB, read on every
  profile) are parsed again only when they change.
- A scheduled price report fetches the betas of the small caps that moved together, not one at a time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Discord: the native webhook, with charts, masked links and no link pr…
8e09893 
…eviews; reports split by size as well as lines

- Discord reports go to the webhook's own endpoint instead of its Slack-compatible one (a /slack suffix in finbot.env is dropped):
  flags=4 turns link previews off (the README's "garbage link previews" limitation is gone), mentions ping nobody, and
  ?wait=true makes Discord answer 200 instead of an empty 204.
- Scheduled reports to Discord carry their charts, uploaded with the report as the message (up to Discord's 2,000 characters).
- Links on Discord are masked links, [text](<url>), with the brackets and parentheses that would end one escaped.
- A long report is split into messages of at most chunk_maxlines lines and the service's character limit, whichever comes
  first: twenty linked lines made a 3,000-character Discord message, which Discord refuses.
- README: the Discord setup and the chat-networks table say what the webhook now does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Tabs everywhere: the tests, shortman.py and sws_gen.py re-indented; .…
e8b635d 
…editorconfig

A whitespace-only change (git diff -w shows nothing but .editorconfig, .gitignore and CLAUDE.md): the 4-space files now indent
with tabs like the rest of the code. Lines inside multi-line strings that are data were left exactly as they were. One style
means an edit's quoted indentation always matches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Config as two objects: config.env (finbot.env) and config.prefs (pref…
34b11b7 
…erences.conf), read where used; no star import

- lib/config.py builds frozen Env and Prefs dataclasses, one place that lists every setting. A reload replaces config.prefs in one
  assignment, so a request never sees half a reload; it used to rewrite each changed name in every module that had copied it with
  `from lib.config import *`. The job processes' snapshot of the preferences lives in the same object.
- Every module reads config.env.<name> / config.prefs.<name> when it runs: no star import, no UNSET sentinel (defaults are None,
  resolved at call time), no names copied into 19 modules. preferences.conf's keys come from the Prefs fields.
- The greeting words moved to lib/words.py, the country suffixes to util.YAHOO_COUNTRY, and cal reads finbot_adr.json when it
  uses it (it was read once at import, so an edit needed a restart).
- Tests change settings through self.env(...) / self.prefs(...); the tests that policed the star import and the sentinel became
  one guard that nothing star-imports config or freezes a preference into a default.
- Checked by recording every request the bot sends for 70 chat commands on Telegram and Slack, 8 buttons and 13 scheduled reports
  to all three networks: byte for byte the same before and after, charts included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Cache and state files in their own modules: lib/cache.py and lib/stat…
af891b4 
…e.py

util.py's cache_path/read_cache/json_write/json_load family splits by what the file is for. lib/cache.py holds the copies of
Yahoo and Sharesight answers and drawn charts, any of which may vanish at any time; lib/state.py holds what must last in
DATA_DIR (the watchlist, plans, seen trades and so on), written whole and flushed. state.edit() is the locked
read-modify-write for the next step. Output is unchanged: every command, button and scheduled report posts what it did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
State files change under a lock; the watchlist is an object (lib/watc…
1348e87 
…hlist.py)

state.edit() reads a state file, lets the block change it and writes it back only if it changed, while no other edit of
that file runs in the process. Every read-modify-write uses it: the watchlist, plans, chart button refs, keyboard
versions, and the seen trades, ratings, records and podcast episodes, which now add to the file as it is when the run
ends instead of writing back what was read when it began. charts' keyboard lock goes.

The watchlist: tickers(), find() (Yahoo, with the .AX retry), add() and remove(). A ticker is checked with Yahoo before
the file is opened, so two chats adding at once both count; before, both read the list, waited on Yahoo, and the later
write dropped the other's add. prepare_watchlist shrinks to phrasing the outcome: it fetches only the ticker it is
changing, not the whole list it then threw away. yahoo.resolve() is the one .AX retry, shared with the company profile.

Visible changes: the add/remove confirmation links to the stock's exchange (MSFT:NASDAQ, not MSFT:MSFT), and removing a
ticker Yahoo no longer knows says it was removed (it was, but the reply said it could not be found).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Sharesight portfolios are objects: sharesight.portfolios() gives Port…
7d9dcf2 
…folio(name, id)

A Portfolio has holdings(), tickers(), performance(days) and trades(days); sharesight.portfolios(name=None) picks every
portfolio or the one named (in any case, else PortfolioNotFound), and sharesight.tickers() is everything held. They
replace pick_portfolios, get_holdings, get_holdings_new, get_holdings_wrapper and get_performance_wrapper, so callers
pass one object instead of a (name, id) pair. The cached, one-at-a-time requests stay in the module functions.

A performance answer without a report is now an error naming the portfolio (it was a KeyError: 'report' further on),
and is not cached. Output is otherwise unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Stdout prints removed: systemd discarded every one
bd85fb1 
finbot.service sends stdout to null, and job processes inherit it, so the 30 prints there (progress dots, "Preparing
outbound", "Fetching ...", skip notices, a debug payload dump) were never read. They go, and a test keeps the bot and its
reports printing only to stderr; the util/ tools, run by hand, still print to stdout. The one error among them, a data
or cache directory that cannot be created, moves to stderr.

On the way: get_trades printed len(data['trades']) before checking the answer, so a Sharesight error surfaced as
KeyError: 'trades'; get_portfolios' include/exclude filter is one expression.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
util.py split by subject: markets, formats, links and render
5c13613 
lib/util.py (807 lines) held four unrelated things; each moves unchanged into its own module:
- lib/markets.py: market tables, flags, currencies, exchange names, ticker translation;
- lib/formats.py: company names trimmed, units, dates and spans in words, signed percents, ordinals;
- lib/links.py: chat-markup links and Yahoo/Google Finance links;
- lib/render.py: every matplotlib chart and the render pool.
get_holdings_and_watchlist becomes watchlist.with_holdings(). util imported sharesight, which imported util back; that
cycle is gone. Callers and test patch targets follow; the profile's local `links` list is now `sites`. Output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Reports move into a lib/reports/ package; the button helpers into lib…
3da57c0 
…/buttons.py

The ten scheduled scripts move from the top level into lib/reports/ (renamed, so their history follows), and
lib/reports.py (622 lines) splits by subject beside them: about (.help, .jobs), history (.history, .compare), holdings
(.holdings, .who, .pie, .watchlist), profile (.SYMBOL) and rankings (.marketcap, .pe, .fpe, .peg, .beta, .buy, .sell).

The period-button helpers (PERIODS, keyboard, the callback refs) leave lib/charts.py for lib/buttons.py, which the
reports use; charts.py keeps the rebuild behind a button press and imports the reports at the top. That ends the second
import cycle: charts imported reports, which imported jobs, which imported every script, two of which imported charts.

lib/jobrun.py takes a module path (lib.reports.price) and names the run by its last part. The next commit puts a
script back at the top level for each report, for schedule lines and runs by hand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
A top-level script runs each report, as before
03d8147 
price.py, trades.py and the other eight are back at the top level as three-line shims that run lib/reports/<name>.py
with their arguments (reports.cli), so schedule lines, cron entries and runs by hand work unchanged. A test runs two of
them and checks there is one per scheduled report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Markup by dialect: service.bold(text) in place of webhook.bold(text, …
1940085 
…service)

lib/dialects.py holds each network's markup: bold, italic, strike, links, user text made safe, and its message sizes. A
dialect is a str equal to the network's name (TELEGRAM == 'telegram'), so it goes wherever the name did; the bot turns a
name into its dialect where a message comes in or a report goes out (worker, webhook.destinations). The 73 markup calls
that passed the service name along become service.bold(text), service.link(url, text) and the like; webhook's MARKUP
table and links.link go. Tests pass TELEGRAM, SLACK and DISCORD. Output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
A docstring for every function in the bot, lib/ and util/
a3cd2aa 
The 128 functions and methods that had none get one, nested functions included; a comment that said the same on the def
line goes. A test now fails on any function in the bot, lib/ or util/ without a docstring, and CLAUDE.md says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
A docstring for every test and test helper too
db95ce0 
The 421 test functions without one get one: each test method says what it checks, and each helper and stub what it
stands in for; a comment that said the same at the top of a test goes. The guard test now covers tests/ as well, so
every function in the project has a docstring. Four one-line stubs became two-line ones to take theirs; test_config
no longer imports the cal.py shim it never used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
util/record_outputs.py: the old/new output comparison, now in the repo
fda8ed2 
The harness that checked each refactor so far records every HTTP request the bot posts for 159 steps (each chat command
on Telegram and Slack, the period and job buttons, every scheduled report) on the offline fixtures, and compares two
recordings step by step. --against COMMIT (HEAD by default) records that commit, checked out in a temporary worktree,
and this tree in the same run, so dates in the output always match; --diff compares two saved recordings; --record
writes one. The exit status is 1 when they differ.

tests/common.py now names what its fixture replaces with recorders (REAL_SENDING), so the tool knows what to put back
on either side of a change to how the tests capture output. The pre-M1 branches of the scratch version are gone. A test
covers the comparison, and CLAUDE.md points to the tool instead of the by-hand method.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Transports and one delivery path
8b6fa8f 
lib/transports.py has one class per way of posting: TelegramBot,
SlackBot, SlackWebhook and DiscordWebhook. Each one knows its dialect,
what it can carry (charts, buttons, a typing notice) and the network's
size limits. Transport.text() splits long reports and Transport.chart()
fits the caption; each network posts one message its own way.

lib/webhook.py becomes lib/delivery.py:
- Report(lines, chart, buttons) and deliver(dest, report), which draws
  a chart spec only where it can go;
- reply_to() for the chat that asked and default_chats() for scheduled
  reports, with destinations() choosing between them;
- report_error() and the CLI entry point.

Other changes:
- worker: Request.reply() replaces say/send/deliver. Every command
  shows the typing notice after half a second, in place of 18
  `with request.typing()` blocks.
- Buttons are neutral, buttons.Button(label, data, active), and
  TelegramBot.markup() turns them into an inline keyboard. Button
  presses edit through the transport (edit_chart, edit_text).
- cal pins its Telegram messages through the transport.
- util/record_outputs.py imports lib.webhook only for a commit from
  before REAL_SENDING.
- The CLAUDE.md code map is updated.

The output comparison (util/record_outputs.py) against the previous
commit is identical in all 159 steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Inbound updates per network: route, verify, parse, dispatch
4dd8f8a 
lib/inbound.py has one class per network that posts to the bot,
Telegram and Slack. Each class knows:
- the path the network posts to;
- how it proves itself: Telegram by a header, checked before the body
  is read; Slack by a token in the body;
- how to read an update, as a Message to answer, a Press of a button,
  or a Response given straight back (Slack's challenge, or nothing to
  do).

bot.py handle() shrinks from 113 lines to a route, verify, parse and
dispatch sequence, and turns away repeats once an update parses to
something to run. The Telegram bot's name now lives on
inbound.TELEGRAM, set at startup.

Behaviour changes:
- Secrets are compared in constant time, and an empty configured
  secret matches nothing. Before, a request carrying an empty header
  or token was accepted when finbot.env left the token empty.
- Log wording: "warning: Slack auth incorrect", the unhandled Slack
  type is named, and repeats are logged the same way on both networks.
- Slack's unhandled-type answer has the same body as Telegram's.

A scratch comparison put 49 requests through the previous bot.py and
this one: statuses, bodies and the calls handed to the worker match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Slack signing secrets
5f9edf0 
The new startup key slackSigningSecret (a secret) turns on signature
checks for Slack requests. With it set, every request must carry:
- X-Slack-Signature, the HMAC-SHA256 under the signing secret of
  v0:<timestamp>:<body>;
- a timestamp within five minutes of now, so a recorded request cannot
  be replayed.
The verification token is then neither needed nor enough. Without the
signing secret, slackOutgoingToken is checked as before; Slack has
deprecated it because the token travels in every request.

New startup warnings:
- an inbound Slack webhook with neither secret;
- telegramOutgoingWebhook without telegramOutgoingToken.
Either way every request would be refused.

The README documents the key in the finbot.env table and in the Slack
setup steps. A test checks the code against Slack's documented example
signature.

Also fixed: test_config.py imported lib.config before tests/common.py
had set the environment. Run before test_edges, it failed 10 Telegram
tests (on the previous commit as well). It now imports the fixture
first, and every test module passes when run alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
One-line docstrings where they fit
a4328ce 
77 docstrings that spilled a few words onto a second line are reworded
to fit on one, within 120 columns with tabs counting 4. Their meaning
is kept.
- 74 of them overflowed by 30 columns or less once joined.
- 3 had first lines already past 120 columns and only two to four words
  on the second line.

The other 111 two-line docstrings need their second line and stay as
they are.

CLAUDE.md's convention now asks for one line where it fits, and states
the line limit as 120 columns with tabs counting 4.

Also included: a table of cases in test_config's startup-warning test,
which wraps two lines of the previous commit that passed 120 columns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
No data is an answer, not an error (D11)
aebc37f 
lib/answer.py adds Answer(*lines): a reply in place of the report
asked for. The worker and the period buttons send its lines as an
ordinary message. Every other exception is still reported as a
one-line ⚠️ error.

- sharesight.PortfolioNotFound is now an Answer.
- yahoo.NoData is both an Answer and a RuntimeError, so code that
  leaves a ticker out of a list on RuntimeError still does. It is
  raised for:
  - a history without prices;
  - no high/low history;
  - Yahoo's 404 whose body says "Not Found", i.e. no such symbol.
  Any other 404 (an endpoint that moved, say), a refusal or an
  unreachable Yahoo stays a RuntimeError, reported as an error.
- .history and .compare of a ticker without data, compare without
  enough history in the period, and a period button with nothing to
  draw now answer plainly. Before, each was "⚠️ ...: RuntimeError: ...".

The output comparison is identical in all 159 steps. Its fixtures
always have data, so the no-data replies are covered by new tests
instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Telegram: period buttons only where their presses arrive
3339148 
A Telegram bot set up for reports only (no telegramOutgoingWebhook)
still put period buttons under its charts, scheduled ones included.
Telegram posts a button press to the bot's webhook, and with none set
up the press went nowhere: the button spun and did nothing. Charts on
such a bot now come without buttons, and with the webhook they keep
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
A setup guide per chat network, under doc/
d08177f 
Most installs use one of Telegram, Slack and Discord. Each now has a
guide covering its whole setup, webhook part included: doc/telegram.md,
doc/slack.md and doc/discord.md. Where there is a choice, a table puts
the two options side by side before the steps, so it is clear to the
admin that it is a choice:
- Telegram: reports only, or the interactive bot.
- Slack: incoming webhook, or the interactive bot (which builds on the
  webhook).
- Discord: a webhook only, so the guide says there is no choice.

The guides include the nginx front end for each network, now with only
TLS 1.2 and 1.3, plus two steps the README lacked:
- Telegram: finding the chat id, and the allowed characters of the
  webhook password.
- Slack: inviting the bot to the channel.

The README links the guides under its chat networks table and in its
requirements. Its per-network sections and the interactive bot setup
are gone, moved into the guides. Daemonize (systemd) becomes a section
of its own.

CI treats doc/*.md like README.md: a pull request changing only docs
skips the tests. The tests read only the API samples in doc/'s
subdirectories, which the allowlist does not match. CLAUDE.md says so,
and points to the guides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Change figures signed everywhere (D13)
bb371af 
A change figure now gets its sign everywhere, through
formats.signed_percent. Before, the same history figure read +3% in
.history but 3% in a profile.
- The profile's history lines and its pre/post-market line now read
  +3% where they read 3%.
- The forecast growth lines and .compare's caption signed theirs by
  hand; they use the helper now, and thousands get separators.
Levels (dividend yield, short interest, debt ratio, pie shares) stay
unsigned.

The output comparison differs only in the profile's 1Y line
("1Y: 3%" now "1Y: +3%"), on Telegram and Slack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
The EOFY reminder moves into reminder.json (D12)
234ce06 
The Australian end of financial year checklist (23 June) was written
in reminder.py. It is now an AU-gated entry in defaults/reminder.json,
like the other reminders.

reminder.json entries gain what the checklist needs:
- a message may be a list of lines as well as one line;
- [text](url) is a link, in each network's own markup;
- a line wrapped in _underscores_ is in italics;
- the rest is text, now escaped for the network (a & in a reminder
  used to reach Telegram's HTML raw).

reminder.py splits into report(service, today, entries), a pure
function, and lambda_handler(), which reads the clock and the file and
delivers. Only the birthday line stays computed.

Checked old against new: for six dates (the EOFY day, the birthday,
gated, dated and ordinary days), in AU and US, on all three networks,
the output is identical. The README documents the reminder.json
format.

An install that keeps its own reminder.json in the data directory
takes that file instead of the default, so it no longer gets the EOFY
checklist until it adds the entry; util/sync_defaults.py shows the
difference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
One command table makes .help and the keyboard
5d3e8ca 
COMMANDS becomes a table of Command(pattern, handler, usage, section,
buttons). The .help text and the Telegram command keyboard are made
from it, where four lists used to be kept by hand: the patterns,
prepare_help's lines, the keyboard's buttons and the README.
- .help lists each section's commands in table order.
- The keyboard is the commands' buttons, alphabetical. It is the same
  fifteen buttons as before, so its version hash is unchanged and no
  DM keyboard is sent again.
- The typo aliases (earrings, prince, hospital...) stay, as part of
  the patterns (D10).

The table is now in the order .help shows it, with .SYMBOL still last.
None of the reordered patterns can match another command's words, and
the output comparison is identical in every command step but .help and
/start. There .SYMBOL moves from the top of "Company info" to its end,
as the catch-all must stay last in the table.

The command keyboard's upkeep (keyboard_version, mark, push, ensure,
refresh) moves from charts.py to worker.py, beside the table it is made
from; charts.py keeps the period-button rebuild. The keyboard test no
longer counts buttons: each must run its own command (R16).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Every report runs as run(dest=None, **kwargs) (D7)
6254fac 
The scheduled reports' entry point, lambda_handler (a leftover from AWS
Lambda), becomes run(dest=None, **kwargs), with keyword-only arguments.
dest is the chat that asked; without one, the report goes to the
default chats. That replaces the (chat_id, service, interactive)
triple, and the argument orders that differed between reports: worker
called shorts(chat, threshold, ticker, service) but price(chat,
threshold, service, user, ticker).

- delivery.targets(dest) replaces destinations(interactive, service,
  chat_id).
- The worker passes request.dest; jobs, jobrun, the command line shims
  and the cache flush call run(**kwargs).
- milestone, reminder and podcasts take a dest too, so every report
  can be sent to one chat.
- util/record_outputs.py calls run, or lambda_handler for a commit from
  before this one.

The output comparison is identical in all 159 steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
trades and cal: gather once, then report per network (R5, R6)
c9635dd 
Both reports did their gathering inside the per-network code, so it
ran again for every destination. Each now has:
- gather(), which fetches once;
- report(service, ...), which only formats;
- run(), which delivers to each chat.

trades (R5): a Trade dataclass replaces the 9-element lists that
were sorted by position. The Yahoo lookup of each trade's market moves
into gather(), and the closure that collected the new trade ids goes:
run() records the ids of the trades it reported.

cal (R6): primary_listing() takes the lookup that reassigned
market_data inside a loop over it: an ADR's listing in finbot_adr.json,
the NZX for an ASX ticker reporting in NZD, the exchange suffix of a
foreign company's US ticker. gather() returns the dated events once.

Checks:
- The output comparison is identical in all 159 steps.
- An old/new comparison of cal on crafted listings is identical on
  Telegram and Slack. It covered an ADR, an NZD listing, an ASML-style
  suffix, a ticker's own details, and a title mismatch, an ETF and a
  far date that are left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
claude added 27 commits October 15, 2026 09:01
@claude
Reports: blank lines between sections the one way; dead code out
4be22c4 
- formats.sections joins blocks with a blank line between each two, for .help, the profile, .who, .plan, trades,
  the calendar and milestone. A trades report over days no longer starts with a blank line, nor does a milestone
  report of record lows only, and .plan and .who no longer end with one.
- .history builds the line of the period pressed itself; it no longer returns a table its caller threw away.
- Unreachable branches removed: an empty performance report (gather raises first), its rebuild's guard, the
  compare rebuild's; performance.report's default that repeated run's.
- A .jobs button pressed while the scheduler is off is told so directly, not by catching AttributeError.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Price report: every move worked out once, then formatted per network
ab743ba 
price.report ran once per chat and refetched in that loop: Yahoo's histories for the period and the small caps'
details. Now gather() works out each stock's change over the period (Sharesight's for a holding, Yahoo's for the
rest) and the one stock's line chart, moves() picks the moves once (the mode's filters, the threshold, the
scheduled trigger), and report() only formats them for one network.

What the chats see changes in four places:
- The at-close and period headings say "≥" as the others do (≥ 5% at close, Price movement ≥ 5%).
- The top list's chart says "% change over the past 3 months", not "in 3 months".
- With fewer than twice `top` stocks, the Bottom list no longer repeats the Top list's stocks, and with none left
  there is no Bottom list and no gap in the chart.
- One stock with no change over a period is said to have none over that period, not "no interday price".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
tests: a golden image for the pie, before the shared chart heading ch…
e3cc79a 
…anges

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Charts: renderers take their spec; titles fit; the line chart's figur…
3b7ca94 
…es match the caption

The renderers take the Chart and are looked up by kind (render.RENDERERS), so draw() no longer unpacks each kind
its own way, and the tests draw specs instead of calling renderers with positional arguments. One time-axes helper
for the line and compare charts, one table of sign colours, one way to turn (date, close) rows into a series, one
shrink-to-fit, and the row limits in one place (render.trimmed).

What the charts show changes, as decided (D22):
- A long title or subtitle is shrunk to fit, where it ran off the right edge (.fpe negative, long company names).
- The line chart's title is the company and the span; beneath it, the price and the change the caption gives
  (164.12 USD ▲ 6.96%), where the title had Yahoo's figures and the subtitle a change of its own that disagreed.
  A null close no longer breaks the line or draws "nan".
- Dates are unpadded (1 Oct, 5 Jul 2026), and the date ticks are muted like the value ticks, both 10 pt.
- The compare chart's axes match the line chart's; crypto labels lose -USD everywhere (BTC, not BTC-USD).
- The shorts chart marks the stocks it leaves out above 24 with a gap row, as the bar charts do.
- .performance sorts one list for its caption and chart.

The golden images of the line and compare charts are redrawn; the pie has one now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Yahoo: an outage is an error, not an unknown symbol; history without …
3f9e55f 
…pandas, dated where it trades

- One way to read Yahoo's answers, for quotes, details and charts: the first host's 200 is the answer, a 404 Not
  Found is no such symbol, anything else an error. An outage used to read as an unknown symbol for a detail ("I
  could not find TSLA", after trying TSLA.AX too). The chart now falls back to query1 as the others do, and a crumb
  Yahoo refuses (401) is renewed once.
- The cookie and its crumb live in one cache file, renewed together under a plain lock; no cookie or no crumb is an
  error, where a fallback pair from 2024 only turned it into a refusal later. The cache flush needs no lock.
- price_history works on the bars without pandas: each bar dated in its exchange's time zone (an ASX bar carried
  the day before's date all summer, an NZX bar always, so N-day changes slipped a session), the price and today's
  change from the quote where the caller has one (the chart is cached up to 12 hours). It takes 3 ms for every
  period on a 10-year chart (15 ms before), 2 ms for one (5 ms); price and performance jobs no longer import pandas.
- Record highs and lows come from the whole history, and records seeded from 10 years are seeded again.
- Fields that never filled or nothing read are gone (dividend_date, 52-week change, the details' 52-week range,
  price to book, quarterly earnings growth, profit margin, analysts per estimate), and so is the profile's net
  debt/equity line, which Yahoo no longer sends the figures for (D30), with the balance-sheet module.
- Log lines are whole lines (fragments without a newline landed in front of other threads' lines), and the ones
  every ETF set off are gone.
- An unreachable host costs 5 s to connect, not http_timeout (18 s): net sets one default for every request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Sharesight: one request function with every check; gains for the repo…
5d04880 
…rts; one warming pass

- Every request goes through one function: one at a time, retried while Sharesight says too many are in flight (only
  the performance report retried before), and checked: an error answer says Sharesight's error (an OAuth refusal,
  which has no error_code, was a KeyError on it), an answer that is not 200 or lacks what was asked for says so
  (it was a KeyError later).
- One cache-or-fetch, cache.cached, serves Sharesight and Yahoo. A trade's portfolio name is added after the cache,
  not written into it (and out of the sample, which came from that cache).
- Portfolio.gain(days) and Portfolio.gains(days) give the portfolio's and each holding's capital gain, so the price
  and performance reports no longer dig through Sharesight's report.
- The cache warmer refreshes each file at 90% of its lifetime and says when the next is due in the same pass;
  keep_warm lost the stop event only the tests used, and the fetchers are private.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Markets: one table by market code, TSE back to Toronto, a Yahoo suffi…
3fac2f0 
…x for every market

- TSE is the Toronto Stock Exchange in Sharesight, and TYO Tokyo (as in Sharesight's code list, and as the original
  ticker translation had it). Since d2b297c a TSE holding was looked up on Tokyo (SHOP.T), dropped from the reports,
  and shown with Japan's flag and yen; it is SHOP.TO again, with Canada's flag and dollar.
- One table holds each market's flag, currency, Yahoo suffix and Google code. 13 of 40 codes had a suffix, so a
  holding on Xetra (Sharesight's XETR), Frankfurt, Milan, Madrid, Swiss, Nordic, Indian, Chinese or Thai markets
  asked Yahoo for SAP.XETR and dropped out; OTC holdings (Sharesight's OTC) did too. Codes from Sharesight's list
  that were missing are in (XETR, CVE, CNSX, NEO, SHG, BSE, OSL, CSE, HEL, ASE, IDX, KLS, PHS); the old ones stay.
- European markets get their country's flag (Amsterdam was 🇳🇱 as a Yahoo ticker but 🇪🇺 as Sharesight's AMS);
  EURONEXT, five exchanges, keeps 🇪🇺.
- One venue table maps Yahoo's names and ticker suffixes to those codes, so the flag, Google code and country of an
  exchange come from the same row. The price report's heading now says AU for the ASX (it said ASX; the docstring
  always promised AU), US for Cboe US, and GB for London (it said UK); a Google link for an NZX stock says NZE and
  one for Xetra ETR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Config: each preference's default on its Prefs field, read in one loo…
b17d728 
…p by its type

A preference was written twice in lib/config.py (its field, then its read with the default) and twice more in the
tests (a table of every default, and a count of the keys). Now the field holds the default and its type says how it is
read (a tuple is a list of words; exchange_suffix and hyperlinkProvider are tidied as before), so a new preference goes
in three places: the field, defaults/preferences.conf and the README's table. The tests check the template against the
fields' defaults, which are the same as before (checked against the old table before it went).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Small ones: compact cache files, no chdir at import, tests clean up a…
fe74815 
…fter themselves

- Cache files are written compact (a 10-year chart: 832 KB to 297 KB, and quicker to parse); state files keep their
  indentation, for people to read.
- lib/config no longer changes directory when imported, so util/check_config.py and split_env.py take a relative
  path from the shell's directory, as a command-line tool should. A relative DATA_DIR (and the default, var/) is
  still in the checkout, wherever finbot starts.
- A full test run left about 100 directories in /tmp (preferences files, schedules, job records); now none.
- The scheduler's flag that skipped a whole line while its last run went on is gone: the set of commands running
  already skips any command still going, whoever started it. The in-process runner only the tests used moved to them.
- formats: ordinal and strip_url in a line each; td_to_human went with its last caller. util/sws_gen.py runs only as
  a script and leaves the umask as it found it. .holdings and .pie lost their branch for no portfolio, which the
  command now answers with its usage and the portfolios before they are called.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Telegram: messages and charts go through the one Bot API call, which …
9d29939 
…says why Telegram refused

sendMessage and sendPhoto were posted by hand, so a refusal logged "400 error outbound to telegram" (the message) or
the raw answer (the chart). They go through telegram.call now, as the edits and acknowledgements already did: a
refusal logs Telegram's reason ("Bad Request: can't parse entities"), a 200 that is not JSON counts as a failure, and a
timeout on a chart is still MAYBE_SENT. sendPhoto is no longer sent disable_web_page_preview, which it does not take.
The chat id goes in the body; the answers are Telegram's messages, which the pinning reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
worker: a message and a button press arrive as inbound's Message and …
be80746 
…Press

process_request took seven positional arguments and process_callback five, which bot.py unpacked from the Message and
Press that inbound had already made. They take those objects now, so a way of receiving updates other than a webhook
(a pull mode) hands over the same thing, and the .jobs button handler takes the Press too. The test recorder hands
over either form, so it still compares with commits from before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
inbound: reading an update is apart from how it arrives
54e9faa 
A network's class read its updates and also held its webhook: the path, the secret header or signature, the form a
Slack press comes in. The readers (Telegram, Slack) now only read, and a webhook class per network (TelegramWebhook,
SlackWebhook) takes the request in and hands the update to its reader. bot.take(network, update) does what follows
for any way of arriving (read it, drop a repeat, hand it to the worker), so a pull mode (getUpdates, Socket Mode, a
gateway) only has to fetch updates and call it. A test hands updates over without a webhook.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
delivery: deliver() takes chart specs only; the scheduled destination…
f3433bc 
…s in a plain list

- Every report gives its chart as a render.Chart, which deliver() draws where it can go; the path for a chart drawn
  already served only two tests, which give a spec now.
- default_chats() lists the destinations plainly, Telegram, then Slack (the channel as the bot, else the incoming
  webhook), then Discord, where it worked them out from the order of the webhooks plus a Slack afterthought.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
buttons: the data a button carries is made and read in one file
ac0550b 
A period button's data was made in buttons.keyboard but taken apart in worker.process_callback, and a .jobs button's
was made in about.py and taken apart in worker again. buttons.parse reads both (a ChartPress or a JobPress, or None),
beside keyboard and a new job_button, so the format lives in one file; malformed .jobs data is still refused by the
schedule's check, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Charts with period buttons: the first view is built as a button press…
1726b4a 
… builds it

.history's first view came from its button builder, but .compare's and .price SYMBOL PERIOD's were made by other
code, so the chart a command posted and the one its own button redraws could differ. All three go through the
builder now (worker.chart_reply), and the price report lost the code that drew one stock's line, which only that view
used. One difference: with performance_use_sharesight on, .price SYMBOL PERIOD used to give Sharesight's figure and no
chart; it gives Yahoo's figure and line now, as its buttons always did (the lists still take Sharesight's figures).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Slack: .jobs buttons in Block Kit; a pressed chart's earlier file is …
a31190f 
…deleted

- .jobs on Slack lists the commands with a button each, two to a row of Block Kit, where presses reach finbot (the
  bot token and slackOutgoingWebhook), and says to tap one; a press of a stale button puts the current list in its
  place (SlackBot.edit_text, chat.update). Whether a chat gets the buttons is now its transport's say, not a test for
  Telegram.
- A period button pressed under a Slack chart uploads the new chart, shows it, and then deletes the file it replaced
  (D28): nothing reuses an uploaded chart, since a period shown again is uploaded again, so deleting costs nothing
  and the bot's files no longer pile up. Each chart message's file is noted in finbot_slack_charts.json (the newest
  500); a message from before this, or noted no longer, keeps its earlier file. files.delete needs the files:write
  scope the uploads already have.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Retire chunk_maxlines: a long report splits by size alone
501412c 
chunk_maxlines capped the lines per message only of a report already too long for one message, which the README did
not say (it read as a split of any longer report). The size-aware split does the job, so the preference goes (D23): a
long report now fills each message up to the network's limit instead of 20 lines. preferences.conf files that still
set it load as before, and the log at start says to delete the line, as for alliterate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Every line within 120 columns, and a test that keeps it so (D24)
84eae66 
418 lines of code, tests, shipped defaults, units and tools were over 120 columns (tabs counting 4), which every edit
quotes whole. They are rewrapped: comments and docstrings re-flowed (a one-line docstring that no longer fit was
reworded to fit, as the convention asks), code split after a comma inside its brackets or given a name for a part, the
odd long string joined from two. Nothing runs differently: the tests pass and the recorder finds every command,
button and scheduled report the same. A test in test_finbot holds every line to 120 columns from now on; README.md,
CLAUDE.md and doc/ are left out, since CI runs no tests for a change to those alone, and so are data files.

Along the way the tests' repeated "environment without finbot's settings" became common.bare_environment().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
tests: the three largest files split by subject; CLAUDE.md's maps bro…
cb0f412 
…ught up to date (D25, N7)

test_providers.py (65 KB), test_jobs.py (44 KB) and test_edges.py (42 KB) were the biggest files a session opened. They
are split by subject, as moves only (the same 400 tests pass):
- test_yahoo, test_sharesight, test_sending (Telegram, Slack and Discord posting) and test_telegram_api, from
  test_providers; the stand-ins they share (sample, Resp, serve, ProviderCase) moved into tests/common.py;
- test_schedule (the schedule file, job arguments) and test_scheduler (firing, daylight saving, runs from chat, child
  processes), from test_jobs;
- test_inbound (webhook requests, Slack events and signatures, text escaped on arrival), from test_edges, which keeps
  the failures, bad arguments, secrets in logs and signals.

CLAUDE.md: the new test map, the code map for buttons, charts, markets and the test helpers, the worker's entry points,
the README's size (36 KB), and its own lines wrapped at 120 from the Facts down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Price moves over a period: the list and one stock's view take the sam…
e5dcdbb 
…e figures

A list over a period (.price 7d, price.py 7) took each holding's figure from
Sharesight even with performance_use_sharesight off, while one stock's view
(.price BHP 7d) drew Yahoo's history and the figure that goes with it. Now
both take the same source:
- off (the default): Yahoo's price history for every stock, which is the
  change its chart draws, and covers the watchlist too;
- on: each holding's capital gain from Sharesight, which counts only the time
  it was held, for the list and for one stock, whose view then comes without
  a chart: Sharesight has no price history to draw.

The README's row for the preference says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Slack: a period pressed within 15 minutes of its upload shows that up…
c1eeee3 
…load again

A press on a chart's period button uploaded the chart afresh every time and
deleted the file it replaced, so switching back to a period meant building,
drawing and uploading it again, with Slack's wait for a new file to become
showable. Now the upload of each period is noted with the report that came
with it, and a press for that period within 15 minutes, on this message or
another, shows that upload again in one chat.update: nothing is built, drawn
or uploaded. If Slack will not show it (a file deleted outside finbot, say),
the press builds and uploads as before.

An upload is deleted once no chart message shows it and it is a minute past
those 15 minutes (the minute lets a press that is reusing it finish showing
it); a file a message shows is never deleted. The chart messages noted the
old way are forgotten, their files left alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
yahoo_warm.py: the price histories of every stock followed, fetched a…
875a4bd 
…fresh every 8 hours

A period list or a stock's chart reads each ticker's 10-year daily history
from the cache, where it is kept for cache_seconds (12 hours by default);
the first request after it runs out waited on Yahoo, one request per ticker.
A new scheduled job, yahoo_warm.py, fetches every followed stock's history
afresh (the watchlist and the holdings; the watchlist alone while Sharesight
cannot be asked), and the default schedule runs it every 8 hours, so none
runs out. With the cache preference off it fetches nothing, since nothing
would be kept. It is housekeeping, so .jobs does not offer it.

Sharesight's warmer keeps its own timing in the bot (it refreshes each cache
at 90% of its lifetime, and at once after a flush); its thread is now named
sharesight_warm, the name the README gives it beside yahoo_warm.py.

- yahoo.prefetch_history takes the cache age to fetch past (0: all afresh)
  and returns the tickers that failed.
- lib/jobrun.py takes a module name with an underscore.
- A data directory with its own schedule.cron needs the line:
  07 */8 * * * yahoo_warm.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
.jobs: the system jobs listed under System, without a number or a button
5d4f964 
.jobs left the scheduled commands that are not run from chat (cleanup,
podcasts, and now yahoo_warm) out of its list entirely. It now lists them
after the numbered commands under a System heading, unnumbered and without
buttons, marked when running like the rest: they run on their schedule only.
`.jobs run cleanup.py` says it is a system job instead of that it is not in
the schedule, and a schedule of system jobs alone lists them rather than
saying there are no jobs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Price moves over a period always from Yahoo: performance_use_sharesig…
c27ed72 
…ht retired

With Yahoo's price histories kept warm (yahoo_warm.py), Yahoo answers a
period list faster than Sharesight, whose report is per portfolio and per
period, kept 30 minutes and fetched one at a time; and its figure is the
better one: the stock's move over the whole period (Sharesight's capital
gain counts only the time a holding was held), for the watchlist too, and
the change its chart draws. So the Sharesight path goes:
- .price 7d, price.py 7 and .price BHP 7d take every figure from Yahoo's
  history, and one stock over a period is always its chart;
- performance_use_sharesight is retired: a preferences.conf that still sets
  it starts as before, with a note to delete the line;
- Portfolio.gains, which only that path read, goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
sharesight_warm.py: Sharesight's caches refreshed on a fixed schedule…
8bc4fe4 
…, like Yahoo's

The bot kept Sharesight's portfolio list and holdings reports warm from a
thread of its own, which refreshed each cache at 90% of its lifetime and
again at once after a flush. It is now a scheduled job like yahoo_warm.py,
on a fixed interval inside a cache lifetime made a little longer:
- sharesight_warm.py fetches the portfolio list and each portfolio's
  holdings report afresh; the default schedule runs it every 30 minutes
  (at :14 and :44, clear of trades.py);
- performance_cache_seconds defaults to 40 minutes (was 30), so the holdings
  never run out between runs;
- without Sharesight configured, or with the cache off, it fetches nothing;
  a failure is logged, not reported in the chat: the caches it would have
  replaced are served while fresh, and the next run tries again;
- .jobs lists it under System.

The thread goes, with sharesight.keep_warm and wake: after a flush, the
warm jobs fill the cache again on their next runs (a request meanwhile
fetches what it needs). sharesight.warm() now simply fetches both afresh.

A data directory with its own schedule.cron needs the line
`14,44 * * * * sharesight_warm.py`; a preferences.conf that sets
performance_cache_seconds=1800 should raise it, or drop the line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
A flush runs both warm jobs straight away
d4fbd25 
After a flush, the cache stayed empty until each warm job's next run (up to
30 minutes for Sharesight, 8 hours for Yahoo), so requests meanwhile waited
on the sources. Now the flush, once the files are deleted, runs
sharesight_warm.py and yahoo_warm.py at once, one after the other in a
thread of their own (a reload does not wait for them): through the
scheduler, which does not start one already running on its schedule, or
with the scheduler off each in a child process. The log says when they
start and when they are done.

yahoo_warm.py asks Sharesight for the holdings only where Sharesight is
configured, as sharesight_warm.py does: without it, the watchlist's
histories are fetched and no request goes to Sharesight.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@claude
Merge remote-tracking branch 'origin/main' into claude/kind-newton-ad…
9502187 
…0z2q

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@robdevops
robdevops enabled auto-merge October 15, 2026 12:56
@robdevops
robdevops merged commit 65557d3 into main Oct 15, 2026
3 checks passed
@robdevops robdevops mentioned this pull request Oct 16, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@robdevops @claude