Skip to content

Fix 14 code-review findings (request auth, Slack loop, scheduler, state files, Yahoo, .env migration); docs-only PRs skip tests - #35

Merged
robdevops merged 15 commits into
mainfrom
claude/kind-newton-ad0z2q
Oct 14, 2026
Merged

robdevops merged 15 commits into
mainfrom
claude/kind-newton-ad0z2q

Conversation

@robdevops

@robdevops robdevops commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

This PR was opened for the docs-only CI change (last section). The code-review fixes pushed to the same branch afterwards went into it too. The title and description were rewritten after the merge to cover all of it. The merge commit on main (276108a) still carries the original title.

Upgrade notes

  • Old .env in the checkout: finbot refuses to start while <checkout>/.env exists, because nothing reads it any more.
    • Run util/split_env.py. It now reads the old dotenv format itself; util/migrate_env.py is gone.
    • Put finbot.env where your launcher loads it and preferences.conf in the data directory, then delete .env.
  • Schedule: a time that can't happen (minute 61) or never comes (30 February) stops the bot starting. On a reload, the edited file is refused and the running schedule kept. @reboot lines are skipped and reported, like other unsupported lines.
  • Telegram webhook: registered once, as the bot starts. It used to be re-registered on every import, which every job process did.
    • If registration fails, the bot refuses to start, with the token redacted in the log.
    • Without telegramOutgoingWebhook, the webhook is left alone.
  • Reload: use systemctl reload finbot, which signals the bot only. Job processes now ignore SIGHUP anyway.
  • HTTP replies: unknown paths get 404, failed auth 403 and malformed bodies 400. Previously these came back as 200 or 500.
  • Slack: after this PR, Slack replies were text only, because files.upload is retired. Chart uploads came back in Slack chart replies via Slack's current upload API; a failed command stops its typing indicator #36, which needs the files:write and files:read scopes.

Fixes (numbered as in the review)

# Change Commit
2 Slack events not written by a person are ignored: bot posts, other subtypes, the bot's own replies, events with no user. This ends the error-post loop and the .holdings reply loop. A Slack request also no longer overwrites the Telegram bot name. 3e61099
7 Requests are routed by path, and Telegram's secret is checked before the body is read. A failure before authentication is only logged, never posted to the chats. A failure after it is reported once and answered 200, so Telegram and Slack don't retry it into the chat. The expected tokens are no longer printed when an auth check fails. 3e61099
8 The Telegram webhook is unregistered and registered (plus getMe) once, from bot.py's startup. Importing lib.telegram makes no network calls, so job processes and tests no longer touch the webhook. 3e61099
14 SIGHUP is ignored from bot.py's first line until startup is done, so a reload sent during startup can't kill the bot. 3e61099
10 Error lines are HTML-escaped for Telegram and Slack, which had rejected <redacted>, '<' not supported… and similar. 4d511bc
9 Charts go only where they can be sent: no chart is drawn for Slack, and a photo Telegram refuses goes out as text, except after a read timeout. Slack uploads followed in #36. fc02069
4 A failed Yahoo chart fetch skips that ticker, including in scheduled price.py N runs. A single .price TICKER 7d gets a one-line error instead of a raw URL and sys.exit in a worker. A benchmark without history says so in its line of the performance report. A profile goes out without its history lines. d71f686
15 1D is now the change since the previous session's close; it was showing the 5-day change. d71f686
6 State and cache files are written atomically (temp file and rename; state files fsynced), and the process umask is left alone. A corrupt finbot_jobs.json is reported once and no longer stops the bot. d90bf9c
5 Impossible schedule times refuse startup (printed to stderr, nothing posted, so a restart loop can't flood the chats) or reload (reported, and the running schedule carries on). The schedule is now read before anything else at startup. e615154
11 Fire times are right across DST changes on every croniter version (checked 1.3.15, 2.0.7, 6.0.0, 6.2.4). Daily jobs had fired twice or an hour late on the distro versions. 7a27d61
13 Job processes ignore SIGHUP and report their own failures, import errors included, with exit status 3. Any other ending (killed by a signal, an OOM kill) is reported by the scheduler. finbot.service and preferences.conf now give systemctl reload finbot. 86b1321
12 The Sharesight cache warmer follows the cache preference across reloads; it no longer fetches every 5 seconds once cache is off. It now starts whenever Sharesight is configured. cfc7da3
3 The .env refusal and the merged util/split_env.py (dotenv format; splits the checkout's .env when run with no argument). *.new is git-ignored, since that output holds credentials. The README's upgrade note and by-hand example no longer point at ./.env. 0614054

Also included: README notes on the Telegram webhook (aa2364e), a test-fixture fix for typing-indicator threads a failed command left running (d4d332d), and the merge of the README edits from #33 and #34 (ecb094e).

CI (the change this PR was opened for)

PRs that change only README.md, CLAUDE.md or img/*.png skip install, compile and tests. The required checks still report success, and any git error fails the gate closed (8e06623, 010c0a5).

Tests

226 → 255 tests, passing on Python 3.x and 3.13.5.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX

…required checks

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@robdevops
robdevops enabled auto-merge October 13, 2026 09:39
claude added 14 commits October 14, 2026 01:47
Capture the changed-file list first so a git error fails the step, require a
non-empty all-docs list (exact names, --no-renames) to skip, gate on
docs_only != 'true', pass base_ref via env, re-run on base change, name and log
the step, pin ubuntu-24.04, compare the full HEAD sha in the version test, and
document the skip in CLAUDE.md.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
…egister the Telegram webhook only at start, ignore SIGHUP while starting

- Requests are routed on the path first and Telegram's secret header is checked before the body is read. A body that is not a
  JSON object gets 400, a failed check 403, an unknown path 404. A failure before authentication is only logged, so a stranger
  can no longer post error lines into the chats; a failure after it is reported once and answered 200, so Telegram and Slack do
  not retry it into the chat again. The expected tokens are no longer printed on a failed check.
- Slack: events not written by a person (bot_message and the other subtypes, anything from a bot, the bot's own replies, events
  without a user) are ignored, so a webhook report or a reply can no longer set off a loop. A Slack request no longer overwrites
  the Telegram bot name.
- lib/telegram.py no longer unregisters/registers the webhook and calls getMe on import, which every job process did. bot.py
  calls telegram.register() once as it starts (interactive mode only) and refuses to start, with the token redacted, if it fails.
- SIGHUP is ignored from the first line of bot.py until startup is done, so a reload sent while the bot starts cannot kill it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
Telegram parses messages as HTML and refused error lines containing a literal '<', such as the '<redacted>' placeholder or
"'<' not supported", so those reports never arrived. The line is now HTML-escaped (& < >) for Telegram and Slack, which asks for
the same three escapes; Discord gets it unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
… as text

Slack retired files.upload, so the Slack replies that came as charts (.beta, .buy, .sell, .peg, .pie, price, performance and
shorts lists, .history, .compare) arrived as nothing. webhook.takes_charts() now decides where a chart can go (Telegram only):
no chart is drawn for Slack, and webhook.sendPhoto sends the report as text for any other service, or when Telegram refuses
the photo (not after a read timeout, when it may have arrived after all). The README no longer asks for the files:write scope.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
…e since the previous close

- price_history hands fetch_chart_json's (error, None) back to its callers, whose skip-this-ticker checks could never fire
  before: it went on into chart_json_to_df and raised TypeError, so one failed ticker took down .price 7d, the price-list
  buttons, the scheduled price.py N runs and performance (via its SPY/QQQ benchmarks). Now the price list leaves the ticker
  out (scheduled runs too), a single asked-for ticker gets a one-line error instead of a raw URL plus sys.exit in a worker,
  a benchmark without history says so in its line, and a profile goes out without its history lines.
- The 1D figure fell through to the 5D change. It is now measured against the previous session's close (the last close of an
  earlier day in the exchange's time zone).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
…a corrupt jobs state file no longer stops the bot

json_write and write_binary_cache truncated the file and wrote it in place under os.umask(0), then set the umask to a hard-coded
022, and left it at 0 when the write raised. Both now go through util.write_file: a temp file beside the target, renamed over it
(state files are also fsynced), mode 0640, umask untouched. A full disk or a kill mid-write leaves the old file as it was.

Scheduler.load treated an unreadable finbot_jobs.json as fatal (JSONDecodeError at start, and a SIGHUP then killed the bot). It
now reports it once and carries on without a record of past runs; the next run writes the record afresh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
parse_schedule accepted any expression croniter.is_valid() accepts, including dates that never come (30 February, 31 April):
the line loaded, then next_fire raised on every tick, so no job ran again and an error was posted every 30 seconds. Such a line,
and a time that is not valid at all (minute 61), is now an ImpossibleTime: finbot refuses to start with it (ERROR schedule: on
stderr, nothing posted, so a restart loop does not flood the chats), and a reload with it is refused and reported while the
running schedule carries on. Other bad lines are still reported and skipped; @reboot is one of those. bot.py now reads the
schedule before it registers the Telegram webhook or starts anything else.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
next_fire handed croniter a zoneinfo-aware time. croniter up to 6.0, which the distro packages are (Ubuntu 24.04 1.3.15, Debian
trixie 2.0.7), gets zoneinfo's DST changes wrong: with CRON_TZ=Australia/Melbourne a 09:00 daily job fired at 08:00 and 09:00
on the day the clocks go forward and at 10:00 on the day they go back; on 6.2 a frequent job could even step back in time
during the repeated hour. croniter now works on the naive local time and the zone is put back after, compared by the instant.
A repeated time fires once (the first time), a skipped one an hour later by the clock. Checked on 1.3.15, 2.0.7, 6.0.0, 6.2.4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
A command that raises before typing.stop() leaves its indicator thread waiting; a later test that yields to gevent (a subprocess
wait, a sleep) let it wake after the fixture's patches were gone, and it printed KeyError: 'telegram' tracebacks after the run.
The fixture now stops every indicator a test started when the test ends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
… why is reported

The reload command the shipped files gave, systemctl kill -s HUP finbot, signals every process of the unit, so a job process
running at that moment died of SIGHUP; the bot took any non-zero exit as already reported, so the run was lost silently.

- lib/jobrun.py ignores SIGHUP from its first line, and reports any failure itself, a script that cannot be imported included,
  exiting with the status REPORTED (3).
- The scheduler reports a job that ended any other way: "killed by SIGHUP" (or SIGKILL, an OOM kill, ...) or "exited with status
  N without saying why".
- finbot.service and preferences.conf now give systemctl reload finbot, which signals the bot alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
keep_warm was started only if cache was on at startup and never looked at it again. After a reload set cache=False nothing
rewrote the cache files, so everything was always due and the loop fetched the token, the portfolio list and every holdings
report every 5 seconds, competing with chat requests for the single Sharesight slot. It now checks the preference on every pass
and idles (a look every minute) while it is off; bot.py starts it whenever Sharesight is configured, so a reload that turns
cache on starts the warming too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
…it_env.py does the whole migration

finbot reads only the environment now (python-dotenv is gone), so an install upgraded from the single .env, whose old unit had
no EnvironmentFile=, started with no credentials or webhooks and ran silently deaf. While <checkout>/.env exists the bot now
refuses to start, saying to run util/split_env.py and where the two files it writes go.

util/split_env.py now reads the old dotenv format itself (export, spaces around =, quotes, trailing comments), so
util/migrate_env.py is gone. With no argument it splits the checkout's .env, and it says what to do next. *.new (its output,
which holds the credentials) is git-ignored. The README's upgrade note and by-hand run example no longer point at ./.env.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
…efusal stops the start

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
@robdevops
robdevops merged commit 276108a into main Oct 14, 2026
3 checks passed
@robdevops robdevops changed the title CI: docs-only pull requests skip the test steps Fix 14 code-review findings (request auth, Slack loop, scheduler, state files, Yahoo, .env migration); docs-only PRs skip tests Oct 14, 2026
robdevops pushed a commit that referenced this pull request Oct 14, 2026
… push

#35 was opened for a CI tweak, took in the 14 review fixes pushed to the work branch afterwards, and merged under its old title.
The new section: before each push, check for an open PR from the work branch and update its title and description to cover
every commit it holds; never make such a PR mergeable while they are out of date.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants