From 0bb9454c932ab2fe21a63b20563a970124f442cc Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 02:40:14 +0000 Subject: [PATCH 01/41] Retire telegramChatID, slackChannelID and discordChannelID: no home chats Each chat has had its own jobs since jobs per chat, which left the home chats two things, the errors that belong to no chat and the first jobs. Both go: - An error that belongs to no chat (a system job's, a webhook request's, a pulled update's) is logged and posted nowhere; one that belongs to a chat still goes there. No more "Failed to report error: nowhere to post". - A report runs for a chat, the one that asked or whose job it is: without one it raises "nowhere to post" (delivery.targets; default_chats is gone). - The three keys are retired: accepted in finbot.env with a note, at start and from check_config, to delete the line; gone from the template, Env and the startup warnings. - An upgrade from before jobs per chat still needs them once: lib/migrate.py reads them from the environment, for that alone, to give those chats the old schedule.cron, watchlist, plans and records. - util/send_stdin.py takes the chat's key: echo hi | util/send_stdin.py telegram:-1001234567890 - The tests run scheduled reports for a chat (common.scheduled()), and record_outputs gives a tree without home chats each network's chat. util/record_outputs.py: 268 steps, identical. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 15 +++++---- bot.py | 10 +++--- defaults/finbot.env | 18 ++++------ lib/config.py | 35 ++++++++------------ lib/delivery.py | 42 ++++++++--------------- lib/jobs.py | 6 ++-- lib/migrate.py | 30 +++++++++++------ lib/pull.py | 4 +-- lib/reports/__init__.py | 4 +-- lib/reports/cal.py | 2 +- lib/reports/milestone.py | 5 ++- lib/reports/performance.py | 4 +-- lib/reports/podcasts.py | 4 +-- lib/reports/price.py | 2 +- lib/reports/rating.py | 4 +-- lib/reports/reminder.py | 2 +- lib/reports/shorts.py | 3 +- lib/reports/trades.py | 4 +-- lib/transports.py | 2 +- tests/common.py | 8 +++-- tests/test_config.py | 34 +++++++++---------- tests/test_discord.py | 17 ++++------ tests/test_finbot.py | 2 +- tests/test_reports.py | 68 ++++++++++++++++++-------------------- tests/test_schedule.py | 20 +++++++---- tests/test_scheduler.py | 2 +- tests/test_sending.py | 43 +++++++++++++----------- util/record_outputs.py | 17 +++++++--- util/send_stdin.py | 16 ++++++--- util/sync_defaults.py | 2 +- 30 files changed, 214 insertions(+), 211 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 8871df9a..9fe40e45 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -67,15 +67,17 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm percent, a period and a keyword from either end, the words between; a handler raises `Usage` for arguments it cannot take, and gets a ticker with `ticker(args)`. `Request` (`request.reply(lines, chart, buttons)`); each command gets the typing notice while it works. `process_request(message)` and `process_callback(press)` take inbound's objects. -- `lib/reports/`: what finbot says. Scheduled (each has `run(dest=None, **kwargs)`, which posts to the chat that asked, - the job's chat or else the home chats; `job(argv)`, its arguments' parser; and `ARGUMENTS`, what `.jobs add` lists): +- `lib/reports/`: what finbot says. Scheduled (each has `run(dest=None, **kwargs)`, which posts to the chat that asked + or the job's chat, and with no dest raises: there are no home chats; `job(argv)`, its arguments' parser; and + `ARGUMENTS`, what `.jobs add` lists): `price performance trades shorts cal rating milestone reminder podcasts cleanup sharesight_warm yahoo_warm`. trades, rating, milestone and podcasts keep a record per chat of what they told it (`state.load_for`, `state.edit_for`), added to only by a scheduled run (`delivery.interactive(dest)` is false). Chat-only (`prepare_*`): `about` (.help .jobs), `history` (.history .compare), `holdings` (.holdings .who .pie .watchlist), `profile` (.SYMBOL), `rankings` (.marketcap .pe .fpe .peg .beta .buy .sell). -- `lib/delivery.py`: `Report(lines, chart, buttons)`, the destinations (the chat that asked, a chat by its key with - `chat(key)`, or each network's home chat as its bot), `deliver(dest, report)`, the one-line error report. +- `lib/delivery.py`: `Report(lines, chart, buttons)`, the destinations (the chat that asked, or a chat by its key with + `chat(key)`, as its network's bot), `deliver(dest, report)`, the one-line error report (to its chat; one that belongs + to no chat only logged). `lib/transports.py`: `Transport`, the base of each network's bot, every one taking charts and buttons: splitting, captions, charts, buttons, edits, and the typing notice (`start_typing`: after half a second; Slack's ⏳ on the message), which `text()` and `chart()` stop before posting. `DiscordBot`: a slash command's first message replaces @@ -106,8 +108,9 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm `toggle`, `run`, which `.jobs` calls through `jobs.current`) and the fixed `SYSTEM` jobs, its shared state under one lock; each command runs in the bot's process, in a thread (`run_in_process`), reported once if still running after `JOB_TIMEOUT` and left to finish (every request goes through `net`, with its timeouts: a test checks). - `lib/migrate.py`: once, the old schedule.cron, the watchlist, the plans and the reports' shared records become the - home chats'. + `lib/migrate.py`: once, on an upgrade, the old schedule.cron, the watchlist, the plans and the reports' shared + records become the old home chats' (the retired telegramChatID, slackChannelID and discordChannelID, which it alone + reads, from the environment). - `tests/common.py`: `FinbotCase` (offline fixtures). `self.command('.price')` returns what was sent: `('text', lines)` or `('photo', size, caption)`. `wsgi_environ(path, body, **headers)` is a request for `bot.main`. `ProviderCase` runs the real parsers on the answers saved under `doc/` (`sample`), as `Resp` and `serve` return diff --git a/bot.py b/bot.py index 0ef8ee33..fe379f43 100755 --- a/bot.py +++ b/bot.py @@ -76,10 +76,10 @@ def doubled(message): return False def main(environ, start_response): - """WSGI entry point. Never lets an exception escape. A failure handling an authenticated request is reported as one - line to the default channel and answered 200, so the service does not retry it into the chat again. Before - authentication a failure is only logged: a stranger must not be able to post into the chats. Either names the - network, not the path: a webhook's path is a secret (telegramOutgoingWebhook, slackOutgoingWebhook).""" + """WSGI entry point. Never lets an exception escape. A failure handling an authenticated request is logged as an + error report (report_error) and answered 200, so the service does not retry it into the chat again; before + authentication, as a failed request, with a 500. Either names the network, not the path: a webhook's path is a + secret (telegramOutgoingWebhook, slackOutgoingWebhook).""" try: return handle(environ, start_response) except Exception as e: @@ -304,7 +304,7 @@ def http_server(): log.warn('WARN config:', line) if os.getuid() == 0: log.warn("Running as superuser. This is not recommended.") - migrate.run() # once: what was kept for all chats becomes the home chats', schedule.cron among it + migrate.run() # once, on an upgrade: what was kept for all chats becomes the old home chats', schedule.cron among it scheduler = jobs.Scheduler() scheduler.load() render.preload() diff --git a/defaults/finbot.env b/defaults/finbot.env index 2f9e3b7a..9f4010c7 100644 --- a/defaults/finbot.env +++ b/defaults/finbot.env @@ -1,4 +1,4 @@ -# finbot.env: THE STARTUP FILE. Where finbot runs, who it talks to, and where it reports. +# finbot.env: THE STARTUP FILE. Where finbot runs, and who it talks to. # The bot never reads this file. Your launcher loads it into the bot's environment, and a change needs a RESTART: # systemd: EnvironmentFile=/opt/finbot.env (in finbot.service) # docker: --env-file /opt/finbot.env @@ -22,32 +22,26 @@ port=5000 # How many minutes late a scheduled report missed while the bot was down may still fire scheduler_catchup=5 -# Telegram's home chat, a group or channel id: finbot's first start gives it the default jobs (.jobs), and errors are -# reported there -telegramChatID= - # Sharesight API access sharesight_code= sharesight_client_id= sharesight_client_secret= -# Chat networks. A Telegram bot token is enough for reports and commands: finbot fetches its updates (pull mode) +# Chat networks. A Telegram bot token is enough for reports and commands: by default, finbot fetches its updates +# (pull mode) telegramBotToken= slackBotToken= -# Slack's home channel id, as Telegram's home chat above (needs slackBotToken) -slackChannelID= -# Slack's app-level token (xapp-) for commands without a server: finbot opens a socket to Slack (Socket Mode) +# Slack's app-level token (xapp-), for Socket Mode, the default: finbot opens a socket to Slack for its updates slackAppToken= -# Discord's bot token: reports, and commands over the gateway (pull mode); and its home channel id, as Telegram's +# Discord's bot token: reports, and commands over the gateway API (pull mode, the default) discordBotToken= -discordChannelID= # Chat networks: inbound over HTTPS (http mode). Telegram: set both to have it post its updates to your server # instead of finbot fetching them telegramOutgoingWebhook= telegramOutgoingToken= # Discord: the Interactions Endpoint URL and the app's public key, for slash commands and buttons over HTTPS instead of -# the gateway +# the gateway API discordOutgoingWebhook= discordPublicKey= slackOutgoingWebhook= diff --git a/lib/config.py b/lib/config.py index 73186649..a687c037 100644 --- a/lib/config.py +++ b/lib/config.py @@ -7,7 +7,7 @@ from pathlib import Path # TO CONFIGURE, DO NOT EDIT THIS FILE. There are two config files: -# finbot.env the startup file: DATA_DIR, ip, port, credentials, the home chats. Never read by the +# finbot.env the startup file: DATA_DIR, ip, port, credentials and the mode of each network. Never read by the # bot: the launcher loads it into the environment (systemd EnvironmentFile=, docker --env-file, # init.d `set -a; . finbot.env`). A change needs a restart. # preferences.conf how the bot behaves: /preferences.conf, read by the bot itself at start and on every @@ -15,24 +15,28 @@ REPO_DIR = Path(__file__).resolve().parent.parent -STARTUP_KEYS = ('DATA_DIR', 'ip', 'port', 'telegramChatID', 'scheduler_catchup', 'sharesight_code', - 'sharesight_client_id', 'sharesight_client_secret', 'slackBotToken', 'slackAppToken', 'slackChannelID', - 'slackOutgoingToken', 'slackOutgoingWebhook', 'slackSigningSecret', 'telegramBotToken', 'telegramOutgoingToken', - 'telegramOutgoingWebhook', 'discordBotToken', 'discordChannelID', 'discordOutgoingWebhook', 'discordPublicKey') +STARTUP_KEYS = ('DATA_DIR', 'ip', 'port', 'scheduler_catchup', 'sharesight_code', 'sharesight_client_id', + 'sharesight_client_secret', 'slackBotToken', 'slackAppToken', 'slackOutgoingToken', 'slackOutgoingWebhook', + 'slackSigningSecret', 'telegramBotToken', 'telegramOutgoingToken', 'telegramOutgoingWebhook', 'discordBotToken', + 'discordOutgoingWebhook', 'discordPublicKey') SECRET_KEYS = ('sharesight_code', 'sharesight_client_id', 'sharesight_client_secret', 'slackBotToken', 'slackAppToken', 'slackOutgoingToken', 'slackOutgoingWebhook', 'slackSigningSecret', 'telegramBotToken', 'telegramOutgoingToken', 'telegramOutgoingWebhook', 'discordBotToken', 'discordOutgoingWebhook') # never printed PREFERENCES_FILE = 'preferences.conf' TEMPLATE = REPO_DIR / 'defaults' / PREFERENCES_FILE # every key commented out at its default; finbot never reads it +HOME_CHAT = ("each chat has its own jobs (/jobs), which post there as the bot, and an error that belongs to no chat" + ' goes to the log; an upgrade from before jobs per chat reads it once more, to give that chat the old schedule,' + ' watchlist and plans (lib/migrate.py); delete the line after that start') RETIRED = { # keys of either file that do nothing any more: accepted, so that they never stop a start, with a note at # start saying what to do instead 'alliterate': 'delete the line', 'chunk_maxlines': 'delete the line', 'performance_use_sharesight': 'delete the line (price performance comes from Yahoo)', - 'slack_webhook': 'the scheduled reports go to slackChannelID as the bot (slackBotToken); set those, and delete the' - ' line', - 'discord_webhook': 'the scheduled reports go to discordChannelID as the bot (discordBotToken); set those, and' - ' delete the line', + 'slack_webhook': "each chat's jobs (/jobs) post there as the bot (slackBotToken); delete the line", + 'discord_webhook': "each chat's jobs (/jobs) post there as the bot (discordBotToken); delete the line", + 'telegramChatID': HOME_CHAT, + 'slackChannelID': HOME_CHAT, + 'discordChannelID': HOME_CHAT, 'scheduler': 'the scheduler is always on, and a schedule without jobs runs nothing; delete the line', 'debug': 'log_level sets what the log shows (INFO by default; DEBUG adds cache hits); delete the line', } @@ -227,18 +231,15 @@ class Env: ip: str port: int scheduler_catchup: int # minutes: a run missed while the bot was down still fires if it was due this recently - telegramChatID: str # Telegram's home chat (its first jobs, the error reports); '' when unset, like the rest below - telegramBotToken: str + telegramBotToken: str # '' when unset, like the rest below telegramOutgoingWebhook: str telegramOutgoingToken: str slackBotToken: str slackAppToken: str # xapp-: Socket Mode, Slack's pull mode (without slackOutgoingWebhook) - slackChannelID: str # with slackBotToken, Slack's home chat (its first jobs, the error reports) slackOutgoingWebhook: str slackOutgoingToken: str slackSigningSecret: str discordBotToken: str - discordChannelID: str # with discordBotToken, Discord's home chat (its first jobs, the error reports) discordOutgoingWebhook: str # the interactions endpoint URL (http mode); unset, finbot keeps to the gateway discordPublicKey: str # the app's key, which checks the signature of each interaction posted to the endpoint sharesight_auth: dict @@ -262,18 +263,15 @@ def startup_values(getenv): ip=r.str('ip', '127.0.0.1'), port=r.int('port', 5000), scheduler_catchup=r.int('scheduler_catchup', 5), - telegramChatID=r.str('telegramChatID', ''), telegramBotToken=r.str('telegramBotToken', ''), telegramOutgoingWebhook=r.str('telegramOutgoingWebhook', ''), telegramOutgoingToken=r.str('telegramOutgoingToken', ''), slackBotToken=r.str('slackBotToken', ''), slackAppToken=r.str('slackAppToken', ''), - slackChannelID=r.str('slackChannelID', ''), slackOutgoingWebhook=r.str('slackOutgoingWebhook', ''), slackOutgoingToken=r.str('slackOutgoingToken', ''), slackSigningSecret=r.str('slackSigningSecret', ''), discordBotToken=r.str('discordBotToken', ''), - discordChannelID=r.str('discordChannelID', ''), discordOutgoingWebhook=r.str('discordOutgoingWebhook', ''), discordPublicKey=r.str('discordPublicKey', ''), sharesight_auth={ @@ -473,8 +471,6 @@ def startup_warnings(settings=None): if s.slackAppToken and not s.slackAppToken.startswith('xapp-'): warnings.append("slackAppToken does not start with xapp-: it must be an app-level token (the app's Basic" ' Information, App-Level Tokens), not the bot token') - if s.slackChannelID and not s.slackBotToken: - warnings.append('slackChannelID is set but slackBotToken is not: scheduled reports cannot go there as the bot') if s.slackOutgoingWebhook and not (s.slackSigningSecret or s.slackOutgoingToken): warnings.append('slackOutgoingWebhook is set but neither slackSigningSecret nor slackOutgoingToken is:' ' every Slack request will be refused') @@ -489,9 +485,6 @@ def startup_warnings(settings=None): ' updates (pull mode), and the token is not used') if s.discordOutgoingWebhook and not s.discordBotToken: warnings.append('discordOutgoingWebhook is set but discordBotToken is not: Discord commands will get no reply') - if s.discordChannelID and not s.discordBotToken: - warnings.append('discordChannelID is set but discordBotToken is not: scheduled reports cannot go there as the' - ' bot') if s.discordOutgoingWebhook and not re.fullmatch(r'[0-9a-fA-F]{64}', s.discordPublicKey): warnings.append("discordOutgoingWebhook is set but discordPublicKey is not the app's public key (64 hex" ' digits): every Discord interaction will be refused') diff --git a/lib/delivery.py b/lib/delivery.py index 793a5351..58f42afb 100644 --- a/lib/delivery.py +++ b/lib/delivery.py @@ -1,4 +1,4 @@ -"""Where a report goes and how: Report, the destinations (the chat that asked, or the default chats), deliver(), and the +"""Where a report goes and how: Report, the destinations (the chat that asked, or whose job it is), deliver(), and the one-line error report. A destination is a transport: a network's bot (TelegramBot, SlackBot, DiscordBot).""" import dataclasses import html @@ -47,31 +47,16 @@ def chat(key, scheduled=False): def interactive(dest): - """Whether a report answers a command (dest, the chat that asked), not a scheduled run (dest a job's chat, or None - for the home chats): a scheduled run tells each chat only what it has not told it before.""" + """Whether a report answers a command (dest, the chat that asked), not a scheduled run (dest a job's chat): a + scheduled run tells each chat only what it has not told it before.""" return dest is not None and not dest.scheduled -def default_chats(): - """Where a scheduled report goes: each network's home chat (telegramChatID, slackChannelID, discordChannelID), as - its bot. Raises RuntimeError when there is nowhere to post.""" - env = config.env - chats = [] - for chat, token, kind in ((env.telegramChatID, env.telegramBotToken, telegram_common.TelegramBot), - (env.slackChannelID, env.slackBotToken, slack_common.SlackBot), - (env.discordChannelID, env.discordBotToken, discord_common.DiscordBot)): - if chat and token: - chats.append(kind(chat)) - chats[-1].scheduled = True - if not chats: - raise RuntimeError('nowhere to post: finbot.env sets no telegramChatID, slackChannelID or discordChannelID,' - ' each with its bot token') - return chats - - def targets(dest): - """Where a report goes: the chat that asked for it (dest), else the default chats.""" - return [dest] if dest else default_chats() + """Where a report goes: the chat that asked for it, or whose job it is (dest). Raises RuntimeError without one.""" + if dest is None: + raise RuntimeError('nowhere to post: a report goes to the chat that asked for it, or whose job it is') + return [dest] def deliver(dest, report): @@ -105,20 +90,21 @@ def error_line(e, context=None, limit=200): def report_error(e, dest=None, context=None): - """Send a one-line error summary to dest, the chat that asked (its transport), else to every default chat. The line - is escaped for Telegram (sent as HTML, which refuses a stray < such as the one in ) and Slack (which asks - for the same escapes). Never raises: error reporting must not become a second failure.""" + """Log a one-line error summary, with its traceback, and send the line to dest, the chat that asked or whose job it + is (its transport); an error that belongs to no chat stays in the log. The line is escaped for Telegram (sent as + HTML, which refuses a stray < such as the one in ) and Slack (which asks for the same escapes). Never + raises: error reporting must not become a second failure.""" try: line = error_line(e, context) log.error(line) if isinstance(e, BaseException): trace = ''.join(traceback.format_exception(e)) log.error(net.redact(trace), end='') # a request's error carries its URL - for chat in targets(dest): + if dest is not None: try: - chat.text([line if chat.dialect == 'discord' else html.escape(line, quote=False)]) + dest.text([line if dest.dialect == 'discord' else html.escape(line, quote=False)]) except Exception as send_error: - log.error("Failed to report error to", chat.name, ":", send_error) + log.error("Failed to report error to", dest.name, ":", send_error) except Exception as reporting_error: log.error("Failed to report error:", reporting_error) diff --git a/lib/jobs.py b/lib/jobs.py index fe0b8990..548a7d20 100644 --- a/lib/jobs.py +++ b/lib/jobs.py @@ -80,8 +80,8 @@ def checksum(text): def run_in_process(label, module, kwargs, dest=None): - """Run a command here, in the calling thread: its report's run(**kwargs), posting to dest (a chat's job) or else to - the home chats.""" + """Run a command here, in the calling thread: its report's run(**kwargs), posting to dest (a chat's job; None for a + system job, which posts nothing).""" if dest is None: module.run(**kwargs) else: @@ -277,7 +277,7 @@ def claim(self, chat, label): def execute(self, commands, chat=''): """Run commands one after another (a job's), in this thread, posting to the chat whose job it is (its key; '' - for a system job, whose failure goes to the home chats); a failure is reported. One already running for that + for a system job, whose failure goes to the log); a failure is reported. One already running for that chat is skipped. One still running after JOB_TIMEOUT is reported as late, once, and runs on: a thread cannot be stopped, but every request it makes has a timeout, so it ends; until then it stays marked running, and no second copy starts. Returns 'done', 'failed' or 'already running'.""" diff --git a/lib/migrate.py b/lib/migrate.py index 8d397184..4dbcbb15 100644 --- a/lib/migrate.py +++ b/lib/migrate.py @@ -1,8 +1,9 @@ -"""Once, as finbot starts: what the bot kept for all chats at once becomes each home chat's (telegramChatID, -slackChannelID, discordChannelID). The old schedule.cron (the data dir's copy, else the shipped one) is copied to every -home chat as its jobs, each in the zone it ran in, and is read no more; the watchlist and the plans are copied to each -home chat; and so are the reports' records of what they told the chats (trades, ratings, record highs, podcast -episodes), so that nothing old is told again. +"""Once, as an upgraded finbot starts: what the bot kept for all chats at once becomes each old home chat's, the chats +that the retired telegramChatID, slackChannelID and discordChannelID name (read from the environment for this alone). +The old schedule.cron (the data dir's copy, else the shipped one) is copied to every home chat as its jobs, each in the +zone it ran in, and is read no more; the watchlist and the plans are copied to each home chat; and so are the reports' +records of what they told the chats (trades, ratings, record highs, podcast episodes), so that nothing old is told +again. This module, and defaults/schedule.cron, go once every install has started with it: they are its only users.""" import copy @@ -16,7 +17,6 @@ from lib import log from lib import config -from lib import delivery from lib import jobs from lib import state from lib import watchlist @@ -28,14 +28,22 @@ # what a shell reads as more than a word, outside quotes, in a pasted crontab line OPERATOR = re.compile(r'\d*>>?&?\d*|&>>?|;|&&|\|\|?|&|[<`$(){}]') REDIRECT = re.compile(r'\d*>>?&?\d*|&>>?') # > file, >> file, 2>&1, &> file +# the home chats of older versions: (network, the retired finbot.env key of its chat, the key of its bot token) +HOMES = (('telegram', 'telegramChatID', 'telegramBotToken'), ('slack', 'slackChannelID', 'slackBotToken'), + ('discord', 'discordChannelID', 'discordBotToken')) + + +def old_homes(): + """The keys of the chats that the retired home chat keys still name in the environment, each with its bot token.""" + return [f'{network}:{os.environ[key]}' for network, key, token in HOMES + if os.environ.get(key) and getattr(config.env, token)] def run(): - """Make the home chats' own what was kept for all chats, where that has not been done yet. With no home chat set, - it all waits until there is one.""" - try: - homes = [chat.key for chat in delivery.default_chats()] - except RuntimeError: + """Make the old home chats' own what was kept for all chats, where that has not been done yet. Without any (a new + install, or the retired keys deleted), there is nowhere to put it, so it waits.""" + homes = old_homes() + if not homes: return schedule(homes) feeds = sorted(os.path.basename(path) for path in glob.glob(state.path(podcasts.SEEN_FILES))) diff --git a/lib/pull.py b/lib/pull.py index cae30aab..1f3f5c19 100644 --- a/lib/pull.py +++ b/lib/pull.py @@ -53,8 +53,8 @@ def failed(self, what, e): def hand_on(self, take, update): """take(network, update), and its answer delivered where the network must have it: an interaction's, and one - that work waits on. A failure is reported as bot.main reports one of an authenticated request: the update came - from the network, so the default chats are told.""" + that work waits on. A failure is reported as bot.main reports one of an authenticated request: it belongs to no + chat, so the log has it.""" try: answer = take(self.network, update) if getattr(answer, 'interaction', False) or getattr(answer, 'then', None): diff --git a/lib/reports/__init__.py b/lib/reports/__init__.py index 5ce4f3fe..7276706d 100644 --- a/lib/reports/__init__.py +++ b/lib/reports/__init__.py @@ -1,6 +1,6 @@ """What finbot says. A module per scheduled report: cal, cleanup (of the cache), milestone, performance, podcasts, price, rating, reminder, shorts, trades, and sharesight_warm and yahoo_warm (which fill the cache), each with -job(argv), which parses its schedule line, and run(dest=None, **kwargs), which runs it, to the chat that asked (dest) -or else the default chats; the scheduler runs them in the bot's process, and `.jobs run` runs one now. And the +job(argv), which parses its schedule line, and run(dest=None, **kwargs), which runs it, to the chat that asked or +whose job it is (dest); the scheduler runs them in the bot's process, and `.jobs run` runs one now. And the chat-only reports: about (.help, .jobs), history (.history, .compare), holdings (.holdings, .who, .pie, .watchlist), profile (.SYMBOL) and rankings (.marketcap, .pe and the rest).""" diff --git a/lib/reports/cal.py b/lib/reports/cal.py index 4dad9c4d..cb23db27 100644 --- a/lib/reports/cal.py +++ b/lib/reports/cal.py @@ -18,7 +18,7 @@ def primary_listings(): def run(dest=None, *, days=None, specific_stock=None, event='earnings'): """Post the earnings or ex-dividend dates ahead (`event`) for the stocks a chat follows or one (Yahoo's symbol): to - the chat that asked or whose job it is (dest), else to the home chats. The scheduled earnings report is pinned where + the chat that asked or whose job it is (dest). The scheduled earnings report is pinned where the network can; a reply is not, or the chat would collect pinned copies.""" interactive = delivery.interactive(dest) if days is None: diff --git a/lib/reports/milestone.py b/lib/reports/milestone.py index 7ad8dc9d..1981049b 100644 --- a/lib/reports/milestone.py +++ b/lib/reports/milestone.py @@ -13,9 +13,8 @@ STATE_FILE = 'finbot_milestone.json' # each chat's {ticker: {'all_time_high': ..., 'all_time_low': ...}} def run(dest=None): - """Post the stocks a chat follows that set an all-time high or low, against its record, to each of the run's chats - (dest, a chat's job or the chat that asked; else the home chats); a scheduled run records each chat's new - records.""" + """Post the stocks a chat follows that set an all-time high or low, against its record, to the chat that asked or + whose job it is (dest); a scheduled run records the chat's new records.""" interactive = delivery.interactive(dest) for chat in delivery.targets(dest): tickers = watchlist.with_holdings(chat.key) diff --git a/lib/reports/performance.py b/lib/reports/performance.py index 2f05cecd..b2c8ef69 100644 --- a/lib/reports/performance.py +++ b/lib/reports/performance.py @@ -49,8 +49,8 @@ def report(service, past_days, portfolio_select=None, performance=None): def run(dest=None, *, past_days=None, user='', portfolio_select=None): """Post each portfolio's performance over past_days (the past_days preference when None), or the one named, beside - the benchmarks: to the chat that asked or whose job it is (dest), else to the home chats, with its chart and period - buttons where they can go.""" + the benchmarks: to the chat that asked or whose job it is (dest), with its chart and period buttons where they can + go.""" if past_days is None: past_days = config.prefs.past_days performance = gather(past_days, portfolio_select) diff --git a/lib/reports/podcasts.py b/lib/reports/podcasts.py index a7f7cc79..57a5914a 100644 --- a/lib/reports/podcasts.py +++ b/lib/reports/podcasts.py @@ -88,8 +88,8 @@ def fetch_new_episodes(feed_url, seen): return episodes def run(dest=None, *, feed_url): - """Post to each of the run's chats (dest, a chat's job; else the home chats) the feed's newest episodes not posted - to it before, then record every new one as seen by it.""" + """Post to the job's chat (dest) the feed's newest episodes not posted to it before, then record every new one as + seen by it.""" seen_file = get_seen_filename(feed_url) chats = delivery.targets(dest) seen = {chat.key: set(state.load_for(chat.key, seen_file, [])) for chat in chats} diff --git a/lib/reports/price.py b/lib/reports/price.py index d78d38cd..d2282b14 100644 --- a/lib/reports/price.py +++ b/lib/reports/price.py @@ -172,7 +172,7 @@ def nothing(mode, threshold, user, ticker, in_session, days=None): def run(dest=None, *, mode='interday', threshold=None, days=None, top=None, user='', specific_stock=None): """Post the price report (report() has the modes) on the stocks each chat follows: to the chat that asked or whose - job it is (dest), else to the home chats, with its chart and period buttons where they can go.""" + job it is (dest), with its chart and period buttons where they can go.""" interactive = delivery.interactive(dest) if threshold is None: threshold = config.prefs.price_percent diff --git a/lib/reports/rating.py b/lib/reports/rating.py index dc257e46..c5afd616 100644 --- a/lib/reports/rating.py +++ b/lib/reports/rating.py @@ -10,8 +10,8 @@ def run(dest=None): """Post the analysts' consensus ratings of the stocks a chat follows that changed since its last scheduled run: to - the chat that asked (dest), against its record; or, a scheduled run, to each of its chats (dest, a chat's job; else - the home chats), recording for each the ratings it saw.""" + the chat that asked (dest), against its record; or, a scheduled run, to the job's chat (dest), recording the + ratings it saw.""" interactive = delivery.interactive(dest) for chat in delivery.targets(dest): market_data, seen = gather(watchlist.with_holdings(chat.key)) diff --git a/lib/reports/reminder.py b/lib/reports/reminder.py index 1c86763f..36fcb381 100644 --- a/lib/reports/reminder.py +++ b/lib/reports/reminder.py @@ -13,7 +13,7 @@ def run(dest=None): - """Post today's reminders, if there are any, to the chat whose job it is (dest), else to the home chats.""" + """Post today's reminders, if there are any, to the chat whose job it is (dest).""" today = datetime.datetime.now(zoneinfo.ZoneInfo(config.prefs.timezone)).date() entries = config.data_json("reminder.json", []) for chat in delivery.targets(dest): diff --git a/lib/reports/shorts.py b/lib/reports/shorts.py index f829ad73..255c6bfc 100644 --- a/lib/reports/shorts.py +++ b/lib/reports/shorts.py @@ -9,8 +9,7 @@ def run(dest=None, *, threshold=None, specific_stock=None): """Post the stocks a chat follows whose short interest is over `threshold` percent (the shorts_percent preference - when None), or one stock's (Yahoo's symbol): to the chat that asked or whose job it is (dest), else to the home - chats.""" + when None), or one stock's (Yahoo's symbol): to the chat that asked or whose job it is (dest).""" interactive = delivery.interactive(dest) if threshold is None: threshold = config.prefs.shorts_percent diff --git a/lib/reports/trades.py b/lib/reports/trades.py index a68e4118..dda496df 100644 --- a/lib/reports/trades.py +++ b/lib/reports/trades.py @@ -31,8 +31,8 @@ class Trade: def run(dest=None, *, days=None, user='', portfolio_select=None): """Post the buys and sells of the past `days`, in every portfolio or the one named: to the chat that asked (dest); - or, a scheduled run, to each of its chats (dest, a chat's job; else the home chats) those it has not told that chat - of before. Raises sharesight.PortfolioNotFound for a name that is none of them.""" + or, a scheduled run, to the job's chat (dest) those it has not told that chat of before. Raises + sharesight.PortfolioNotFound for a name that is none of them.""" interactive = delivery.interactive(dest) if days is None: days = config.prefs.past_days diff --git a/lib/transports.py b/lib/transports.py index 69feec4c..df5c99ee 100644 --- a/lib/transports.py +++ b/lib/transports.py @@ -19,7 +19,7 @@ class Transport: takes_buttons = True # buttons under a message shows_typing = True # a typing notice while a command works TYPING = (0.5, 7, 10) # the notice: first after 0.5 s (a quick answer gets none), then every 7 s, 10 times at most - scheduled = False # posting a scheduled run's report (a chat's job, or the home chats), not a reply to a command + scheduled = False # posting a scheduled run's report (a chat's job), not a reply to a command _notices = None # the thread sending the notices _quiet = None # set to stop them diff --git a/tests/common.py b/tests/common.py index 48690d76..37bc12a3 100644 --- a/tests/common.py +++ b/tests/common.py @@ -8,8 +8,7 @@ ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) sys.path.insert(0, ROOT) os.chdir(ROOT) -os.environ.update(telegramOutgoingWebhook='https://example.com/telegram', telegramOutgoingToken='tok', - telegramChatID='1') +os.environ.update(telegramOutgoingWebhook='https://example.com/telegram', telegramOutgoingToken='tok') import bot from lib import answer, buttons, cache, charts, config, delivery, dialects, formats, inbound, links, markets, net @@ -31,6 +30,11 @@ def no_jobs(data): Path(data, 'finbot_schedule.json').write_text('{}') +def scheduled(key='telegram:1'): + """A chat as a scheduled run posts to it, the job's (Telegram's chat 1 unless another key is given).""" + return delivery.chat(key, scheduled=True) + + def bare_environment(): """os.environ without finbot's settings (the startup keys and the preferences): for a process that must start from the defaults.""" diff --git a/tests/test_config.py b/tests/test_config.py index 79ec7701..1ceaa27c 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -87,7 +87,7 @@ def test_unknown_key_gets_a_hint_and_a_line_number(self): def test_a_startup_key_in_preferences_says_where_it_belongs(self): """A startup key in preferences.conf is sent to finbot.env, without printing its value.""" - errors = self.errors('slackOutgoingWebhook=https://x\nip=0.0.0.0\ntelegramChatID=5\nscheduler_catchup=3\n') + errors = self.errors('slackOutgoingWebhook=https://x\nip=0.0.0.0\nport=5\nscheduler_catchup=3\n') self.assertEqual(len(errors), 4) for line in errors: self.assertIn('belongs in finbot.env (a change needs a restart)', line) @@ -160,12 +160,15 @@ def test_a_retired_key_is_accepted_and_said_to_do_nothing(self): startup = Path(common.temp_dir(self), 'finbot.env') startup.write_text('slack_webhook=https://hooks.slack.com/services/x\nport=80\n') self.assertEqual(check_config.check(startup, path), []) - self.assertEqual(check_config.notes(path, startup)[0], 'slack_webhook in finbot.env does nothing any more:' - ' the scheduled reports go to slackChannelID as the bot (slackBotToken); set those, and delete the line') - with mock.patch.dict(os.environ, {'discord_webhook': 'https://discord.com/api/webhooks/1/x'}): + self.assertEqual(check_config.notes(path, startup)[0], "slack_webhook in finbot.env does nothing any more:" + " each chat's jobs (/jobs) post there as the bot (slackBotToken); delete the line") + with mock.patch.dict(os.environ, {'discord_webhook': 'https://discord.com/api/webhooks/1/x', + 'telegramChatID': '-1001'}): report = config.startup_report() - self.assertIn('discord_webhook in finbot.env does nothing any more: the scheduled reports go to' - ' discordChannelID as the bot (discordBotToken); set those, and delete the line', report) + self.assertIn("discord_webhook in finbot.env does nothing any more: each chat's jobs (/jobs) post there as" + ' the bot (discordBotToken); delete the line', report) + self.assertIn('telegramChatID in finbot.env does nothing any more: each chat has its own jobs (/jobs)', + '\n'.join(report)) # there are no home chats def test_unreadable_file_is_an_error_naming_it(self): """A preferences file that cannot be read is an error that names it.""" @@ -411,7 +414,7 @@ def test_a_preference_in_the_environment_is_refused_by_the_bot(self): self.assertEqual(len(caught.exception.errors), 2) self.assertIn('future_days is a preference; move it from finbot.env to preferences.conf', '\n'.join(caught.exception.errors)) - with mock.patch.dict(os.environ, {'ip': '1.2.3.4', 'telegramChatID': '5'}): + with mock.patch.dict(os.environ, {'ip': '1.2.3.4', 'port': '80'}): config.check_startup_environment() # startup keys are what the environment is for def test_the_bot_refuses_to_start_with_a_preference_in_its_environment(self): @@ -442,8 +445,8 @@ def test_the_telegram_api_is_the_bot_tokens(self): self.assertEqual(values.telegram_api, api) def test_settings_that_leave_part_of_the_bot_unable_to_work_are_warned_about(self): - """A startup warning for Slack commands or slackChannelID without slackBotToken (no reply, no reports there), - and for an inbound webhook without its secret (every request would be refused).""" + """A startup warning for Slack commands without slackBotToken (no reply), and for an inbound webhook without its + secret (every request would be refused).""" hooks = dict(slackOutgoingWebhook='https://example.com/slack', slackOutgoingToken='stok', slackBotToken='xoxb-1', telegramOutgoingWebhook='https://example.com/telegram', telegramOutgoingToken='tok') deaf = ('slackBotToken is set but neither slackAppToken nor slackOutgoingWebhook is: finbot hears nothing from' @@ -453,7 +456,6 @@ def test_settings_that_leave_part_of_the_bot_unable_to_work_are_warned_about(sel ' every Slack request will be refused') telegram_refused = ('telegramOutgoingWebhook is set but telegramOutgoingToken is not: every Telegram update' ' will be refused') - no_bot = 'slackChannelID is set but slackBotToken is not: scheduled reports cannot go there as the bot' unused = ('telegramOutgoingToken is set but telegramOutgoingWebhook is not: finbot fetches its Telegram updates' ' (pull mode), and the token is not used') app_no_reply = 'slackAppToken is set but slackBotToken is not: Slack commands will get no reply' @@ -464,7 +466,6 @@ def test_settings_that_leave_part_of_the_bot_unable_to_work_are_warned_about(sel bad_secret = ("telegramOutgoingToken is not 1 to 256 letters, digits, _ or -: Telegram takes no other as the" " webhook's secret, so setting the webhook will fail") cases = (({}, []), ({'slackBotToken': ''}, [no_reply]), ({'slackOutgoingToken': ''}, [slack_refused]), - ({'slackChannelID': 'C9'}, []), ({'slackChannelID': 'C9', 'slackBotToken': ''}, [no_reply, no_bot]), ({'slackOutgoingToken': '', 'slackSigningSecret': 's'}, []), ({'telegramOutgoingToken': ''}, [telegram_refused]), ({'telegramOutgoingWebhook': ''}, [unused]), ({'telegramOutgoingWebhook': '', 'telegramOutgoingToken': ''}, []), @@ -481,18 +482,17 @@ def test_settings_that_leave_part_of_the_bot_unable_to_work_are_warned_about(sel self.assertEqual(config.startup_warnings(), warnings) def test_discord_settings_that_leave_part_of_it_unable_to_work_are_warned_about(self): - """A startup warning for Discord's endpoint or channel without the bot token, the endpoint without the app's - public key (every interaction refused), and the key without the endpoint (the gateway is used).""" + """A startup warning for Discord's endpoint without the bot token, the endpoint without the app's public key + (every interaction refused), and the key without the endpoint (the gateway is used).""" key = 'ab' * 32 http = dict(discordBotToken='x', discordOutgoingWebhook='https://example.com/discord', discordPublicKey=key) no_reply = 'discordOutgoingWebhook is set but discordBotToken is not: Discord commands will get no reply' - no_bot = 'discordChannelID is set but discordBotToken is not: scheduled reports cannot go there as the bot' refused = ("discordOutgoingWebhook is set but discordPublicKey is not the app's public key (64 hex digits):" ' every Discord interaction will be refused') unused = ('discordPublicKey is set but discordOutgoingWebhook is not: finbot keeps to the Discord gateway (pull' ' mode), and the key is not used') - cases = (({}, []), ({'discordBotToken': ''}, [no_reply]), ({'discordChannelID': '77'}, []), - ({'discordChannelID': '77', 'discordBotToken': '', 'discordOutgoingWebhook': ''}, [no_bot, unused]), + cases = (({}, []), ({'discordBotToken': ''}, [no_reply]), + ({'discordBotToken': '', 'discordOutgoingWebhook': ''}, [unused]), ({'discordPublicKey': 'not-hex'}, [refused]), ({'discordPublicKey': ''}, [refused]), ({'discordOutgoingWebhook': ''}, [unused]), ({'discordOutgoingWebhook': '', 'discordPublicKey': ''}, [])) for changes, warnings in cases: @@ -572,7 +572,7 @@ def test_split_env(self): self.assertEqual(prefs, ['future_days=3', 'volatility_multiplier=2']) self.assertEqual(len(notes), 5) self.assertTrue(any('slackToken dropped' in n for n in notes) and any('unknown key bogus' in n for n in notes)) - self.assertIn('line 10: slack_webhook dropped: the scheduled reports go to slackChannelID', notes[-1]) + self.assertIn("line 10: slack_webhook dropped: each chat's jobs (/jobs) post there as the bot", notes[-1]) def test_split_env_reads_the_old_dotenv_format_too(self): """split_env reads the old dotenv format: quotes, export, comments after values and # inside URLs.""" diff --git a/tests/test_discord.py b/tests/test_discord.py index 50b94446..265b3516 100644 --- a/tests/test_discord.py +++ b/tests/test_discord.py @@ -157,7 +157,7 @@ class Sending(FinbotCase): def setUp(self): """The bot token set, and Discord's API answered from self.answers (else 200 and a message).""" super().setUp() - self.env(discordBotToken='x', discordChannelID='77') + self.env(discordBotToken='x') self.calls, self.answers = [], [] def request(method, url, **kwargs): """Note the call, and give the next answer.""" @@ -263,16 +263,11 @@ def test_rate_limits_are_waited_out_and_refusals_raise(self): self.assertIn('403 Missing Access (50001)', err.getvalue()) self.assertNotIn('s3cret-token', err.getvalue()) - def test_scheduled_reports_go_to_the_channel_with_their_buttons(self): - """With discordChannelID and the bot token, scheduled reports go to that channel as the bot, buttons and all; - without the token, nowhere on Discord.""" - self.env(telegramChatID='') - chats = delivery.default_chats() - self.assertEqual([(type(chat).__name__, chat.where) for chat in chats], [('DiscordBot', '77')]) - self.assertTrue(chats[0].takes_buttons and chats[0].takes_charts) - self.env(discordBotToken='') - with self.assertRaisesRegex(RuntimeError, 'nowhere to post'): - delivery.default_chats() + def test_a_jobs_reports_go_to_its_channel_with_their_buttons(self): + """A Discord channel's job posts there as the bot, charts and buttons and all.""" + chat = delivery.chat('discord:77', scheduled=True) + self.assertEqual((type(chat).__name__, chat.where, chat.key), ('DiscordBot', '77', 'discord:77')) + self.assertTrue(chat.takes_buttons and chat.takes_charts and chat.scheduled) class Commands(FinbotCase): diff --git a/tests/test_finbot.py b/tests/test_finbot.py index 516ffe72..4a4c6d6c 100644 --- a/tests/test_finbot.py +++ b/tests/test_finbot.py @@ -44,7 +44,7 @@ def test_zoneinfo_offset_is_not_lmt(self): def test_reminder_runs(self): """The reminder report runs with no reminders on file.""" with mock.patch.object(config, 'data_json', lambda name, default=None: []): - reminder.run() + reminder.run(scheduled()) def test_bot_handler(self): """A sticker is answered 200, with nothing logged but the request's line (INFO); git_version matches what git diff --git a/tests/test_reports.py b/tests/test_reports.py index 72906149..e021ce4b 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -176,7 +176,7 @@ def test_the_small_caps_betas_are_fetched_together(self): self.prefs(demote_volatile=True) self.patch(yahoo, 'fetch', lambda ts: {t: market_entry(t) | {'market_cap': 5e8} for t in ts}) - price.run() + price.run(scheduled()) self.assertEqual(len(calls), 1) self.assertTrue(calls[0]) @@ -338,21 +338,21 @@ def test_compare_usage(self): class ScheduledReports(FinbotCase): - """The cron entry points (non-interactive: they post to the default channel).""" + """The scheduled entry points (not interactive: they post to the job's chat).""" def test_price(self): """The scheduled price report posts.""" - price.run() + price.run(scheduled()) self.assertTrue(self.sent) def test_performance(self): """The scheduled performance report posts.""" - performance.run(past_days=7) + performance.run(scheduled(), past_days=7) self.assertTrue(self.sent) def test_shorts(self): """The scheduled shorts report posts.""" - shorts.run() + shorts.run(scheduled()) self.assertTrue(self.sent) def test_earnings_and_dividends(self): @@ -360,7 +360,7 @@ def test_earnings_and_dividends(self): for kwargs in ({'event': 'earnings'}, {'event': 'ex-dividend'}): with self.subTest(kwargs=kwargs): self.sent.clear() - cal.run(**kwargs) + cal.run(scheduled(), **kwargs) self.assertTrue(self.sent) def test_only_the_scheduled_earnings_report_is_pinned(self): @@ -370,13 +370,12 @@ def test_only_the_scheduled_earnings_report_is_pinned(self): self.patch(telegram_common.TelegramBot, 'pin', lambda chat, answers: pins.append(chat.where)) self.command('.earnings') self.assertEqual(pins, []) - self.env(telegramChatID='1') - cal.run(event='ex-dividend') - cal.run(event='earnings') + cal.run(scheduled(), event='ex-dividend') + cal.run(scheduled(), event='earnings') self.assertEqual(pins, ['1']) def test_a_chats_job_runs_its_report_as_scheduled_not_as_a_command(self): - """A report a chat's job runs is a scheduled run, as the home chats' is: it says nothing when nothing passes, + """A report a chat's job runs is a scheduled run: it says nothing when nothing passes, where a command's reply says so, and the earnings report is pinned. price, shorts and cal took any chat they were given for a command.""" pins = [] @@ -399,7 +398,7 @@ def test_earnings_skip_an_adr_whose_primary_listing_yahoo_does_not_know(self): undated = lambda t: {k: v for k, v in market_entry(t).items() if not (t == 'T1' and k == 'earnings_date')} self.patch(yahoo, 'fetch', lambda ts: {t: undated(t) for t in ts}) self.patch(yahoo, 'fetch_detail', lambda t, *a, **k: {} if t == 'T1.XX' else {t: undated(t)}) - cal.run(event='earnings') + cal.run(scheduled(), event='earnings') self.assertIn('Co T2 ', str(self.sent)) self.assertNotIn('Co T1 ', str(self.sent)) @@ -418,7 +417,7 @@ def details(t, *a, **k): return {t: {**(market_entry(t) if own_date else undated(t)), 'profile_country': 'United Kingdom'}} self.patch(yahoo, 'fetch_detail', details) self.sent.clear() - cal.run(event='earnings') + cal.run(scheduled(), event='earnings') self.assertEqual('Co T1 ' in str(self.sent), listed) self.assertEqual([t for t in asked if '.' in t], []) # no T1.L @@ -440,10 +439,10 @@ def test_trades_asks_yahoo_only_for_new_trades(self): fetched = [] self.patch(yahoo, 'fetch', lambda tickers: fetched.append(set(tickers)) or {t: market_entry(t) for t in tickers}) - trades.run() + trades.run(scheduled()) self.assertEqual(fetched, [{'T1'}]) fetched.clear() - trades.run() # the trade is known now: nothing to report and nothing to fetch + trades.run(scheduled()) # the trade is known now: nothing to report and nothing to fetch self.assertEqual(fetched, []) def test_a_scheduled_run_keeps_the_trades_another_run_recorded_meanwhile(self): @@ -451,7 +450,7 @@ def test_a_scheduled_run_keeps_the_trades_another_run_recorded_meanwhile(self): meanwhile = lambda tickers: state.save(trades.STATE_FILE, {'chats': {'telegram:1': [999]}}) or {t: market_entry(t) for t in tickers} self.patch(yahoo, 'fetch', meanwhile) # called after the seen trades were read, before the new ones are recorded - trades.run() + trades.run(scheduled()) known = state.load_for('telegram:1', trades.STATE_FILE, []) self.assertEqual((known[0], len(known)), (999, 2)) @@ -477,7 +476,7 @@ def test_a_trade_or_a_rating_change_reaches_each_chat_that_schedules_the_report( def test_milestone(self): """The milestone report posts.""" - milestone.run() + milestone.run(scheduled()) self.assertTrue(self.sent) def test_milestone_records_a_ticker_only_with_its_new_values(self): @@ -485,11 +484,11 @@ def test_milestone_records_a_ticker_only_with_its_new_values(self): self.patch(yahoo, 'historic_high', lambda t: (150.0, 50.0)) # beyond the 52-week range (120/80): nothing new self.patch(yahoo, 'fetch', lambda ts: {t: {k: v for k, v in market_entry(t).items() if not k.startswith('fiftyTwoWeek')} for t in ts}) - milestone.run() + milestone.run(scheduled()) self.assertFalse(state.load_for('telegram:1', milestone.STATE_FILE, {})) self.patch(yahoo, 'fetch', lambda ts: {t: market_entry(t) for t in ts}) self.sent.clear() - milestone.run() + milestone.run(scheduled()) self.assertNotIn('Record high', str(self.sent)) self.assertEqual(state.load_for('telegram:1', milestone.STATE_FILE, {})['T1'], {'all_time_high': 150.0, 'all_time_low': 50.0}) @@ -499,7 +498,7 @@ def test_a_record_kept_from_ten_years_is_seeded_again_from_the_whole_history(sel above its 10-year high but below its all-time one posted a false record.""" state.save(milestone.STATE_FILE, {'chats': {'telegram:1': {'T1': {'record_high': 110.0, 'record_low': 90.0}}}}) self.patch(yahoo, 'historic_high', lambda t: (150.0, 50.0)) - milestone.run() # the 52-week range (120/80) was past the old record, but not the whole history's + milestone.run(scheduled()) # the 52-week range (120/80) was past the old record, but not the whole history's self.assertNotIn('Co T1 ', str(self.sent)) self.assertEqual(state.load_for('telegram:1', milestone.STATE_FILE, {})['T1'], {'all_time_high': 150.0, 'all_time_low': 50.0}) @@ -508,12 +507,12 @@ def test_milestone_records_are_all_time(self): """A new 52-week high below the all-time high is no record. The record used to become the 52-week high after the first run, so a stock far below its peak posted a record high each time it set a new 52-week one.""" self.patch(yahoo, 'historic_high', lambda t: (150.0, 50.0)) - milestone.run() # first sighting: the 52-week range (120/80) is inside the history (150/50) + milestone.run(scheduled()) # first sighting: the 52-week range (120/80) is inside the history (150/50) for high, expected in ((130.0, False), (160.0, True), (155.0, False)): with self.subTest(high=high): self.patch(yahoo, 'fetch', lambda ts: {t: market_entry(t) | {'fiftyTwoWeekHigh': high} for t in ts}) self.sent.clear() - milestone.run() + milestone.run(scheduled()) self.assertEqual('Record high' in str(self.sent), expected) def test_a_52_week_record_kept_before_is_replaced_by_the_history(self): @@ -521,7 +520,7 @@ def test_a_52_week_record_kept_before_is_replaced_by_the_history(self): state.save(milestone.STATE_FILE, {'chats': {'telegram:1': {f'T{i}': {'high': 110.0, 'low': 90.0} for i in range(25)}}}) self.patch(yahoo, 'historic_high', lambda t: (150.0, 50.0)) - milestone.run() # 120 beats the old 52-week record of 110, but not the history's 150 + milestone.run(scheduled()) # 120 beats the old 52-week record of 110, but not the history's 150 self.assertNotIn('Record high', str(self.sent)) def test_rating(self): @@ -532,13 +531,13 @@ def test_rating(self): def test_rating_keeps_the_last_rating_of_a_ticker_missing_from_a_run(self): """A ticker missing from one run keeps its last rating, so its change is reported when it comes back; it used to be dropped from the record.""" - rating.run() # records every ticker's rating + rating.run(scheduled()) # records every ticker's rating self.patch(yahoo, 'fetch_detail', lambda t, *a, **k: {} if t == 'T1' else {t: market_entry(t)}) - rating.run() # no T1 this time: its rating used to be dropped from the record + rating.run(scheduled()) # no T1 this time: its rating used to be dropped from the record self.patch(yahoo, 'fetch_detail', lambda t, *a, **k: {t: market_entry(t) | ({'recommend': 'sell', 'recommend_index': 4.5} if t == 'T1' else {})}) self.sent.clear() - rating.run() + rating.run(scheduled()) self.assertIn('Co T1 ', str(self.sent)) # so its change from buy to sell went unreported def test_an_interactive_rating_records_nothing(self): @@ -546,10 +545,10 @@ def test_an_interactive_rating_records_nothing(self): rating.run(telegram_common.TelegramBot('5')) self.assertIsNone(state.load(rating.STATE_FILE)) - def test_scheduled_reports_reach_each_home_chat_with_its_own_token(self): - """Scheduled reports reach Telegram's and Slack's home chats, each with its own bot's credentials: Slack's + def test_scheduled_reports_reach_each_jobs_chat_with_its_own_token(self): + """Scheduled reports reach a Telegram and a Slack job's chat, each with its own bot's credentials: Slack's request carries the Slack token and channel, and Telegram's chat id goes only to Telegram.""" - self.env(slackBotToken='xoxb-test', slackChannelID='C9') + self.env(slackBotToken='xoxb-test') self.patch(transports.Transport, 'text', REAL_TEXT) self.patch(transports.Transport, 'chart', REAL_CHART) self.patch(render, 'draw', lambda chart: io.BytesIO(b'png')) @@ -566,14 +565,14 @@ def post(url, **kwargs): with self.subTest(report=report): posts.clear() with contextlib.redirect_stderr(io.StringIO()): - report() + report(scheduled()) + report(scheduled('slack:C9')) self.assertIn(('slack.com', 'Bearer xoxb-test', 'C9'), posts) self.assertTrue(any(host == 'api.telegram.org' and chat == '1' for host, _, chat in posts)) self.assertFalse(any(host == 'slack.com' and chat == '1' for host, _, chat in posts)) def test_nowhere_to_post_is_an_error(self): - """A scheduled report with nowhere to post is an error that says so.""" - self.env(telegramChatID='') + """A report run for no chat (there are no home chats) is an error that says so.""" with self.assertRaisesRegex(RuntimeError, 'nowhere to post'): price.run() @@ -629,13 +628,12 @@ def test_a_long_list_is_stored_and_only_the_newest_are_kept(self): def test_telegram_charts_have_buttons_in_both_modes(self): """A Telegram chart, a scheduled one included, has its period buttons whether Telegram posts their presses to telegramOutgoingWebhook (http mode) or finbot fetches them (pull mode).""" - self.env(telegramChatID='1') label, kwargs = price.job(['7']) # as scheduled: price.py 7 for hook in ('https://example.com/telegram', ''): with self.subTest(telegramOutgoingWebhook=hook): self.env(telegramOutgoingWebhook=hook) self.markups.clear() - price.run(**kwargs) + price.run(scheduled(), **kwargs) self.assertTrue(self.markups and self.markups[0]) def press(self, data): @@ -727,13 +725,13 @@ def item(n, date): 'Tue, 02 Jan 2024 00:00:00 GMT') + '').encode() self.patch(podcasts.net, 'get', lambda url, **k: mock.Mock(content=feed, raise_for_status=lambda: None)) - podcasts.run(feed_url='https://example.com/feed') + podcasts.run(scheduled(), feed_url='https://example.com/feed') self.assertEqual([line.split(' (')[0] for line in self.sent[0][1]], ['📣 Show: ep 3 00:10', '📣 Show: ep 1 00:10', '📣 Show: ep 2 00:10']) self.sent.clear() - podcasts.run(feed_url='https://example.com/feed') + podcasts.run(scheduled(), feed_url='https://example.com/feed') self.assertEqual(self.sent, []) # all seen now diff --git a/tests/test_schedule.py b/tests/test_schedule.py index 4e67246c..ba60839f 100644 --- a/tests/test_schedule.py +++ b/tests/test_schedule.py @@ -1,5 +1,5 @@ """Jobs as .jobs add takes them (lib/jobs.py): their lines, and each report's job(argv); and the one-time migration -(lib/migrate.py) of the old schedule.cron, and of the reports' records, to the home chats.""" +(lib/migrate.py) of the old schedule.cron, and of the reports' records, to an upgrade's home chats.""" import contextlib, hashlib, io, os, unittest from pathlib import Path from unittest import mock @@ -132,12 +132,16 @@ def test_the_checkout_holds_only_the_bot_at_its_top(self): class Migration(FinbotCase): - """As finbot first starts with jobs per chat, the home chats get what was kept for all chats, once.""" + """As an upgraded finbot first starts with jobs per chat, the home chats that the retired keys still name in its + environment get what was kept for all chats, once.""" def setUp(self): - """Two home chats, Telegram's and Slack's; the old schedule ran in UTC.""" + """Two old home chats, Telegram's and Slack's, in the environment; the old schedule ran in UTC.""" super().setUp() - self.env(slackBotToken='xoxb-1', slackChannelID='C1') + self.env(slackBotToken='xoxb-1') + homes = mock.patch.dict(os.environ, {'telegramChatID': '1', 'slackChannelID': 'C1'}) + homes.start() + self.addCleanup(homes.stop) self.patch(migrate, 'local_zone', lambda: jobs.UTC) def migrate(self): @@ -202,10 +206,12 @@ def test_an_empty_schedule_migrates_to_no_jobs_and_is_done(self): self.assertEqual(state.load(jobs.SCHEDULE_FILE), {}) def test_with_no_home_chat_it_waits(self): - """With no home chat set there is nowhere to put the jobs and records, so the migration waits for one.""" - self.env(telegramChatID='', slackChannelID='') + """With no old home chat in the environment (a new install) there is nowhere to put the jobs and records, so + the migration waits for one; so it does for a chat without its bot token.""" state.save(trades.STATE_FILE, ['t1']) - self.assertEqual(self.migrate(), '') + with mock.patch.dict(os.environ, {'telegramChatID': '', 'slackChannelID': 'C1'}): + self.env(slackBotToken='') + self.assertEqual(self.migrate(), '') self.assertFalse(os.path.exists(state.path(jobs.SCHEDULE_FILE))) self.assertEqual(state.load(trades.STATE_FILE), ['t1']) diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index a9c1cc81..6cd94023 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -158,7 +158,7 @@ def setUp(self): self.patch(jobs.SCRIPTS[name], 'run', self.recorder(name)) def recorder(self, name): - """A report's run() that notes the run, and the chat it would post to (its key; None for the home chats).""" + """A report's run() that notes the run, and the chat it would post to (its key; None for a system job).""" def run(dest=None, **kwargs): """Note the run and where it posts.""" self.ran.append((name, kwargs)) diff --git a/tests/test_sending.py b/tests/test_sending.py index 658b0d0b..fdf21a56 100644 --- a/tests/test_sending.py +++ b/tests/test_sending.py @@ -85,12 +85,12 @@ def test_error_reports_are_escaped_for_telegram_and_slack(self): self.patch(transports.Transport, 'text', REAL_TEXT) for kind in (telegram_common.TelegramBot, slack_common.SlackBot, discord_common.DiscordBot): self.patch(kind, 'post_text', lambda dest, text, buttons=None: sent.append((dest.name, text))) - self.env(telegramChatID='5', slackChannelID='C9', slackBotToken='xoxb-1', discordChannelID='77', - discordBotToken='x') + self.env(slackBotToken='xoxb-1', discordBotToken='x') error = RuntimeError("404 for ZZ at https://x/?a=1&crumb=abc; '<' not supported between ") with contextlib.redirect_stderr(io.StringIO()): - REAL_REPORT_ERROR(error, context='.compare ') + for key in ('telegram:5', 'slack:C9', 'discord:77'): + REAL_REPORT_ERROR(error, delivery.chat(key), context='.compare ') REAL_REPORT_ERROR(error, telegram_common.TelegramBot('55'), context='.compare') texts = dict(sent[:3]) self.assertIn('crumb=<redacted>', texts['telegram']) # Telegram's HTML parser rejects a bare @@ -142,15 +142,17 @@ def test_every_bot_takes_charts_and_buttons(self): with self.subTest(kind=kind.__name__): self.assertTrue(kind('1').takes_charts and kind('1').takes_buttons) - def test_scheduled_reports_go_to_each_networks_home_chat_as_its_bot(self): - """Scheduled reports go to each network's home chat (telegramChatID, slackChannelID, discordChannelID), as its - bot: a chat without its bot token is left out.""" - self.env(telegramChatID='1', slackChannelID='C9', slackBotToken='xoxb-test', discordChannelID='77', - discordBotToken='x') - self.assertEqual(list(map(repr, delivery.default_chats())), ['', '', - '']) - self.env(slackBotToken='', telegramBotToken='') - self.assertEqual(list(map(repr, delivery.default_chats())), ['']) + def test_an_error_that_belongs_to_no_chat_goes_only_to_the_log(self): + """There are no home chats: an error that belongs to no chat (a system job's, a webhook request's) is logged, + with nothing posted and no word of a failure to post it.""" + text = mock.Mock() + self.patch(transports.Transport, 'text', text) + err = io.StringIO() + with contextlib.redirect_stderr(err): + REAL_REPORT_ERROR(ValueError('boom'), context='sharesight_warm') + self.assertIn('⚠️ sharesight_warm: ValueError: boom', err.getvalue()) + self.assertNotIn('Failed', err.getvalue()) + text.assert_not_called() def test_slack_replies_upload_their_charts(self): """A Slack reply whose chart has no buttons uploads it in three steps (bot token), the report as its message, @@ -199,18 +201,19 @@ def test_a_slack_upload_that_fails_goes_out_as_text(self): delivery.deliver(slack_common.SlackBot('C1'), delivery.Report(['*Heading*', 'line 2'], CHART)) self.assertEqual(self.sent, expect) - def test_scheduled_reports_carry_their_charts_to_every_home_chat(self): - """A scheduled report's chart goes to each network's home chat, as its bot.""" - self.env(slackBotToken='xoxb-test', slackChannelID='C9', discordBotToken='x', discordChannelID='77') + def test_scheduled_reports_carry_their_charts_on_every_network(self): + """A scheduled report's chart goes to the job's chat on every network, as its bot.""" + self.env(slackBotToken='xoxb-test', discordBotToken='x') sent = [] self.patch(transports.Transport, 'chart', lambda dest, image, caption, buttons=None: sent.append((dest.name, 'chart')) or {'ok': True}) self.patch(transports.Transport, 'text', lambda dest, lines, buttons=None: sent.append((dest.name, 'text')) or []) - for report in (price.run, lambda: performance.run(past_days=7), shorts.run): + for report in (price.run, lambda dest: performance.run(dest, past_days=7), shorts.run): with self.subTest(report=report): sent.clear() - report() + for key in ('telegram:1', 'slack:C9', 'discord:77'): + report(scheduled(key)) self.assertEqual(sorted(set(sent)), [('discord', 'chart'), ('slack', 'chart'), ('telegram', 'chart')]) def test_the_graph_preference_off_sends_no_chart_anywhere(self): @@ -258,12 +261,12 @@ def fake(url, **kwargs): class DiscordSending(FinbotCase): - """Discord's home channel, as the bot: a scheduled report's chart attached, within Discord's 2,000 characters.""" + """A Discord job's channel, as the bot: a scheduled report's chart attached, within Discord's 2,000 characters.""" def setUp(self): """Discord as the only destination, sent through the real path, its calls recorded.""" super().setUp() - self.env(telegramChatID='', discordBotToken='x', discordChannelID='77') + self.env(discordBotToken='x') self.patch(transports.Transport, 'text', REAL_TEXT) self.patch(transports.Transport, 'chart', REAL_CHART) self.calls = [] @@ -274,7 +277,7 @@ def test_a_scheduled_report_carries_its_chart(self): """A scheduled Discord report has its chart attached and the report as its message, within 2,000 characters, without previews or pings.""" with contextlib.redirect_stderr(io.StringIO()): - price.run() + price.run(scheduled('discord:77')) (url, kwargs), = self.calls self.assertEqual(url, 'https://discord.com/api/v10/channels/77/messages') payload = json.loads(kwargs['data']['payload_json']) diff --git a/util/record_outputs.py b/util/record_outputs.py index 9eacf321..c8b549b0 100755 --- a/util/record_outputs.py +++ b/util/record_outputs.py @@ -147,13 +147,22 @@ def step(label, run): for data in BUTTONS: label = f'button {data}' if service == 'telegram' else f'{service} button {data}' step(label, pressing(service, 'cb', chat, message, data)) - # the scheduled reports go to each network's home chat as its bot (a commit from before telegram_api had - # webhooks, the incoming ones among them, and one from before the Discord bot no discordChannelID) + # the scheduled reports go to a chat of each network as its bot: each job's chat, or before they retired, + # the home chats (a commit from before telegram_api had webhooks, the incoming ones among them, and one + # from before the Discord bot no discordChannelID) fields = {field.name for field in common.dataclasses.fields(common.config.env)} api = {'telegram_api': TELEGRAM_API} if 'telegram_api' in fields else { 'webhooks': {'telegram': TELEGRAM_API}} + homes = {'telegramChatID': '1', 'slackChannelID': 'C9'} if 'telegramChatID' in fields else {} discord = {'discordChannelID': '78'} if 'discordChannelID' in fields else {} - self.env(telegramChatID='1', slackChannelID='C9', **api, **discord) + self.env(**homes, **api, **discord) + def scheduled(run, kwargs): + """A scheduled run to a chat of each network: one run to the home chats, or else one run per chat.""" + if homes: + return lambda: run(**kwargs) + from lib import delivery # a tree without home chats has it: an older one may not + keys = ('telegram:1', 'slack:C9', 'discord:78') + return lambda: [run(delivery.chat(key, scheduled=True), **kwargs) for key in keys] for line in SCHEDULED: if hasattr(jobs, 'parse_reports'): label, module, kwargs = jobs.parse_reports(line)[0] @@ -161,7 +170,7 @@ def step(label, run): name, _, arguments = line.partition(' ') label, module, kwargs = jobs.parse_command(f'{name}.py {arguments}'.strip())[0] run = getattr(module, 'run', None) or module.lambda_handler # lambda_handler: a commit before run() - step(f'scheduled {line}', lambda: run(**kwargs)) + step(f'scheduled {line}', scheduled(run, kwargs)) Path(output).write_text(json.dumps(steps, indent=1, sort_keys=True, default=str)) result = unittest.TextTestRunner(verbosity=0, stream=io.StringIO()).run(Record()) diff --git a/util/send_stdin.py b/util/send_stdin.py index b167a5e3..0f26fdaa 100755 --- a/util/send_stdin.py +++ b/util/send_stdin.py @@ -1,5 +1,6 @@ #!/usr/bin/env python3 -"""Post the lines piped in to the home chats, as the scheduled reports go: `echo hello | util/send_stdin.py`.""" +"""Post the lines piped in to a chat, named by its key (its network and id), as a scheduled report goes: +`echo hello | util/send_stdin.py telegram:-1001234567890`.""" import os import sys from pathlib import Path @@ -7,10 +8,15 @@ sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) from lib import delivery +USAGE = 'send_stdin: pipe the lines to post into it, naming the chat: echo hello | util/send_stdin.py telegram:-100123' + if __name__ == '__main__': - if os.isatty(sys.stdin.fileno()): - sys.exit('send_stdin: pipe the lines to post into it, e.g. echo hello | util/send_stdin.py') + if len(sys.argv) != 2 or not sys.argv[1].partition(':')[2] or os.isatty(sys.stdin.fileno()): + sys.exit(USAGE) + try: + dest = delivery.chat(sys.argv[1], scheduled=True) + except ValueError as e: + sys.exit(f'send_stdin: {e}') lines = [line.strip() for line in sys.stdin if line.strip()] - for dest in delivery.default_chats(): - delivery.deliver(dest, delivery.Report(lines)) + delivery.deliver(dest, delivery.Report(lines)) diff --git a/util/sync_defaults.py b/util/sync_defaults.py index 65ef4261..a68a2ec9 100755 --- a/util/sync_defaults.py +++ b/util/sync_defaults.py @@ -33,7 +33,7 @@ def sync(apply=False, force=False, dest=None, create=False): continue if src.name in ('finbot.env', 'schedule.cron'): # the startup file is not data: it holds credentials, lives wherever the launcher loads it from, and the bot - # never reads it; the old schedule is read only to give the home chats their jobs once (lib/migrate.py) + # never reads it; the old schedule is read only to give an upgrade's home chats their jobs (lib/migrate.py) continue name = src.name target = dest_dir / name From 598405eaaed22c1f4940ebe9c8aeec794d906365 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 02:44:13 +0000 Subject: [PATCH 02/41] Calendar: a US listing's home market again, only without a date of its own ARM and CLBT are held on US markets, so their own details come first, as before. Only when those have no earnings or ex-dividend date does the calendar try another listing of the company, where the date may be: the one finbot_adr.json names, the NZX's for an ASX ticker reporting in NZD, or, again, the home market's for a foreign company's US listing (markets.YAHOO_COUNTRY: ASML.AS for ASML). The last two are guesses: one that Yahoo does not know (ARM.L, CLBT.TA: there are none) is logged at DEBUG, not WARN (yahoo.fetch_detail(guess=True)), and the stock left out. Only the earnings and ex-dividend calendar (/earnings, /dividend and the scheduled cal job) uses these fallbacks. .exdividend and .ex-dividend are .dividend; they were taken as a ticker. util/record_outputs.py: 268 steps, identical. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/markets.py | 43 +++++++++++++++++++++++++++++++++++++++++++ lib/reports/cal.py | 28 +++++++++++++++++----------- lib/worker.py | 7 ++++--- lib/yahoo.py | 11 +++++++---- tests/test_reports.py | 28 +++++++++++++++++++--------- tests/test_yahoo.py | 15 ++++++++++++++- 6 files changed, 104 insertions(+), 28 deletions(-) diff --git a/lib/markets.py b/lib/markets.py index 6f191d2b..cf38fa1d 100644 --- a/lib/markets.py +++ b/lib/markets.py @@ -78,6 +78,49 @@ class Market(typing.NamedTuple): 'Shanghai': 'SHG', 'Shenzhen': 'SHE', 'Thailand': 'BKK', 'Jakarta': 'IDX', 'Kuala Lumpur': 'KLS', } +YAHOO_COUNTRY = { # country -> the Yahoo suffix of its home market: the calendar's guess at a US listing's home one + 'Argentina': 'BA', + 'Austria': 'VI', + 'Australia': 'AX', + 'Belgium': 'BR', + 'Brazil': 'SA', + 'Canada': 'TO', + 'Chile': 'SN', + 'Colombia': 'CL', + 'Czech Republic': 'PR', + 'Denmark': 'CO', + 'Estonia': 'TL', + 'Finland': 'HE', + 'France': 'PA', + 'Germany': 'DE', + 'Hungary': 'BD', + 'Iceland': 'IC', + 'India': 'NS', + 'Indonesia': 'JK', + 'Ireland': 'IR', + 'Israel': 'TA', + 'Italy': 'MI', + 'Latvia': 'RG', + 'Lithuania': 'VS', + 'Mexico': 'MX', + 'Netherlands': 'AS', + 'New Zealand': 'NZ', + 'Norway': 'OL', + 'Portugal': 'LS', + 'Qatar': 'QA', + 'Saudi Arabia': 'SR', + 'South Africa': 'JO', + 'Spain': 'MC', + 'Sweden': 'ST', + 'Switzerland': 'SW', + 'Thailand': 'BK', + 'Turkey': 'IS', + 'United Kingdom': 'L', + 'Venezuela': 'CR' + # markets with numeric tickers purposefully omitted +} + + def code(name): """MARKETS' code for an exchange by any name it goes by: Sharesight's code or Google's, the suffix of its Yahoo tickers, or Yahoo's name for it (each Nasdaq tier has one: NasdaqGS); the name itself when unknown.""" diff --git a/lib/reports/cal.py b/lib/reports/cal.py index cb23db27..4d38c6d7 100644 --- a/lib/reports/cal.py +++ b/lib/reports/cal.py @@ -53,20 +53,26 @@ def gather(tickers, days, specific_stock, event): return sorted(found, key=lambda item: item[0]) def primary_listing(ticker, data, field): - """(a listing's ticker, its Yahoo details) with a `field` that a ticker's quote lacks, or (None, None). The ticker's - own details: what is held is that listing, so its own date counts, and a foreign company's US listing (ARM, CLBT) - is never swapped for its home market's (ARM.L, CLBT.TA). Only when it has none, the listing that finbot_adr.json - names, or the NZX's for an ASX ticker reporting in NZD (GTK.AX, SKO.AX).""" + """(a listing's ticker, its Yahoo details) with a `field` that a ticker's quote lacks, or (None, None). First the + ticker's own details: what is held is that listing (a US ADR, say), so its own date counts. Only when it has none, + another listing of the same company, which may have the earnings or ex-dividend date: the one finbot_adr.json names, + the NZX's for an ASX ticker reporting in NZD (GTK.AX, SKO.AX), or the home market's for a foreign company's US + listing (ASML.AS for ASML). The last two are guesses: one that Yahoo does not know (ARM.L, CLBT.TA) is passed over, + logged at DEBUG only.""" own = yahoo.fetch_detail(ticker).get(ticker, {}) - listings = [ticker] + if own.get('profile_title') == data['profile_title'] and field in own: + return ticker, own if mapped := primary_listings().get(ticker): - listings.append(mapped) + other, guess = mapped, False # named in finbot_adr.json: one that Yahoo does not know is a mistake there elif ticker.endswith('.AX') and data.get('financialCurrency') == 'NZD': - listings.append(ticker.replace('.AX', '.NZ')) - for listing in listings: - details = own if listing == ticker else yahoo.fetch_detail(listing).get(listing, {}) - if details.get('profile_title') == data['profile_title'] and field in details: # the same company, with a date - return listing, details + other, guess = ticker.replace('.AX', '.NZ'), True + elif own.get('profile_country') in markets.YAHOO_COUNTRY and '.' not in ticker: # the country: only in the details + other, guess = ticker + '.' + markets.YAHOO_COUNTRY[own['profile_country']], True + else: + return None, None + details = yahoo.fetch_detail(other, guess=guess).get(other, {}) + if details.get('profile_title') == data['profile_title'] and field in details: # the same company, with a date + return other, details return None, None def report(service, events, days, specific_stock, interactive, event): diff --git a/lib/worker.py b/lib/worker.py index 442688d8..a28507dd 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -238,7 +238,7 @@ def on_help(request, args): def on_calendar(request, args): """.earnings or .dividend [period|SYMBOL]: the coming earnings or ex-dividend dates.""" - event = 'ex-dividend' if args.word.startswith('div') else 'earnings' + event = 'ex-dividend' if 'div' in args.word else 'earnings' cal.run(request.dest, days=args.days, specific_stock=ticker(args), event=event) def on_performance(request, args): @@ -447,8 +447,9 @@ class Command: Command(('watchlist', 'wishlist'), on_watchlist, ('.watchlist [add|del SYMBOL]',), TRACKED, ('.watchlist',), keywords=CHANGE), Command(('who',), on_who, ('.who SYMBOL',), TRACKED), - Command(('earnings', 'earning', 'earrings', 'earring', 'dividends', 'dividend'), on_calendar, - ('.dividend [period|SYMBOL]', '.earnings [period|SYMBOL]'), COMPANY, ('.dividend', '.earnings'), period=True), + Command(('earnings', 'earning', 'earrings', 'earring', 'dividends', 'dividend', 'exdividend', 'ex-dividend'), + on_calendar, ('.dividend [period|SYMBOL]', '.earnings [period|SYMBOL]'), COMPANY, ('.dividend', '.earnings'), + period=True), Command(('marketcap', 'maletas', 'marketer'), on_marketcap, ('.marketcap [SYMBOL|top|bottom]',), COMPANY, ('.marketcap',), keywords=TOP), Command(('beta',), on_beta, ('.beta',), PRICE, ('.beta',)), diff --git a/lib/yahoo.py b/lib/yahoo.py index 0668eed1..ced4b6b5 100644 --- a/lib/yahoo.py +++ b/lib/yahoo.py @@ -219,19 +219,22 @@ def _quote_answer(tickers): ) RECOMMENDATION = ('recommend', 'recommend_index', 'recommend_analysts') # all three, or none -def fetch_detail(ticker, seconds=None): +def fetch_detail(ticker, seconds=None, guess=False): """One ticker's quoteSummary: {ticker: {...}}, cached `seconds` (or cache_seconds); {} for a symbol Yahoo does not - know, or without a name or price. Raises RuntimeError when Yahoo cannot be reached or refuses.""" + know, or without a name or price, which is logged as a warning, or for a listing guessed at (guess: the calendar's + other listing of a company), whose absence is no fault, at DEBUG. Raises RuntimeError when Yahoo cannot be reached + or refuses.""" + missing = log.debug if guess else log.warn now = datetime.datetime.now() data = cache.cached("finbot_yahoo_detail_" + ticker + '.json', seconds, lambda: _detail_answer(ticker)) result = dig(data, 'quoteSummary', 'result', 0) if result is None: # Yahoo answers 404 Not Found, or result=null, for a symbol it does not know - log.warn(ticker, "is unknown to Yahoo") + missing(ticker, "is unknown to Yahoo") return {} title = next((name for name in (dig(result, 'price', 'longName'), dig(result, 'price', 'shortName')) if isinstance(name, str) and name), None) if not title: # catches some delisted stocks like "DUB" - log.warn(ticker, "has no name at Yahoo") + missing(ticker, "has no name at Yahoo") return {} detail = {'profile_title': formats.transform_title(title)} for key, path, *convert in DETAIL_FIELDS: diff --git a/tests/test_reports.py b/tests/test_reports.py index e021ce4b..fb30b8ca 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -16,6 +16,13 @@ def test_text_commands(self): with self.subTest(cmd=cmd): self.assertTrue(self.command(cmd), f'{cmd} sent nothing') + def test_exdividend_is_the_ex_dividend_report(self): + """.exdividend and .ex-dividend are .dividend: the ex-dividend dates, not a stock named EXDIVIDEND.""" + self.prefs(graph=False) + for text in ('.dividend', '.exdividend', '.ex-dividend'): + with self.subTest(text=text): + self.assertIn('Upcoming ex-dividends', str(self.command(text))) + def test_a_period_list_and_one_stocks_view_give_the_same_figure(self): """Over a period, the list and one stock's view take a holding's figure from its Yahoo history, which its chart draws, not Sharesight's gain over the time it was held (+33% here), which the list used to take.""" @@ -402,24 +409,27 @@ def test_earnings_skip_an_adr_whose_primary_listing_yahoo_does_not_know(self): self.assertIn('Co T2 ', str(self.sent)) self.assertNotIn('Co T1 ', str(self.sent)) - def test_a_us_listing_of_a_foreign_company_keeps_its_own_dates(self): + def test_a_us_listing_takes_its_home_markets_date_only_without_one_of_its_own(self): """A foreign company's US listing (ARM, CLBT: held on a US market) whose quote lacks the date takes it from its - own details, and is never swapped for its home market's listing (ARM.L, CLBT.TA), even when it has no date at - all: then it has none.""" + own details; only when they have none, from its home market's listing (T1.L, a British company's), a guess: + one that Yahoo does not know (ARM.L, CLBT.TA: there are none) leaves the stock out.""" undated = lambda t: {k: v for k, v in market_entry(t).items() if not (t == 'T1' and k == 'earnings_date')} self.patch(yahoo, 'fetch', lambda ts: {t: undated(t) for t in ts}) - for own_date, listed in ((True, True), (False, False)): - with self.subTest(own_date=own_date): + for own_date, home, listed, guessed in ((True, True, True, []), (False, True, True, [('T1.L', True)]), + (False, False, False, [('T1.L', True)])): + with self.subTest(own_date=own_date, home=home): asked = [] - def details(t, *a, **k): - """T1's details, of a British company, with its date or without; asked for, noted.""" - asked.append(t) + def details(t, *a, guess=False, **k): + """T1's details, of a British company, with its date or without; T1.L's, if Yahoo knows it.""" + asked.append((t, guess)) + if t == 'T1.L': + return {t: market_entry('T1')} if home else {} return {t: {**(market_entry(t) if own_date else undated(t)), 'profile_country': 'United Kingdom'}} self.patch(yahoo, 'fetch_detail', details) self.sent.clear() cal.run(scheduled(), event='earnings') self.assertEqual('Co T1 ' in str(self.sent), listed) - self.assertEqual([t for t in asked if '.' in t], []) # no T1.L + self.assertEqual([(t, guess) for t, guess in asked if '.' in t], guessed) def test_trades(self): """.trades run from a chat posts.""" diff --git a/tests/test_yahoo.py b/tests/test_yahoo.py index f179b293..0096573b 100644 --- a/tests/test_yahoo.py +++ b/tests/test_yahoo.py @@ -1,6 +1,6 @@ """Yahoo's quotes, details and charts: the real parsers on the answers saved under doc/, and failures. HTTP is mocked at lib.net, so the production parsing code runs unchanged.""" -import datetime, os, unittest +import contextlib, datetime, io, os, unittest from tests.common import * @@ -93,6 +93,19 @@ def combined(): merged.update(sample('yahoo', f'sample_yahoo_v10_{module}.json')['quoteSummary']['result'][0]) return {'quoteSummary': {'result': [merged], 'error': None}} + def test_a_listing_guessed_at_that_yahoo_does_not_know_is_logged_at_debug(self): + """A ticker Yahoo does not know is a warning; a listing guessed at (the calendar's home market listing of a US + one) only DEBUG, as its absence is no fault.""" + cases = (('A.L', False, 'INFO', True), ('B.L', True, 'INFO', False), ('C.L', True, 'DEBUG', True)) + self.http(yahoo, *[Resp({'quoteSummary': {'result': None, 'error': None}}) for _ in cases]) + for ticker, guess, level, shown in cases: + with self.subTest(guess=guess, log_level=level): + self.prefs(log_level=level) + err = io.StringIO() + with contextlib.redirect_stderr(err): + self.assertEqual(yahoo.fetch_detail(ticker, guess=guess), {}) + self.assertEqual(f'{ticker} is unknown to Yahoo' in err.getvalue(), shown) + def test_parses_real_quote_summary(self): """A real quoteSummary parses, the company's description included.""" self.http(yahoo, Resp(self.combined())) From ae2c535866a7ef7ae0b9cea72c4a94203bfcfd4f Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 02:48:14 +0000 Subject: [PATCH 03/41] Docs: pull mode by default, Discord's gateway API, a Python quick start, slash commands first - Pull mode is the default, and HTTP mode the alternative for those who prefer to run an HTTPS gateway; the docs no longer say "no server": finbot runs all the same, it only makes outgoing connections. Each guide's table says whether finbot listens on the internet. - Discord's gateway is its gateway API, and the mode gateway API mode. - README: the new features list; beside Docker's quick start, one with Python (git clone and a venv, on Linux, macOS or WSL), both installing every optional library; Issues in place of Suggestions. - Slash commands come first (/price, /jobs add), the dot form (.price) the plain message, quicker to type on a phone; the command list and each report's examples are slash commands, without the @botname copies. - No home chats: the guides lose the chat id steps, each chat schedules its own reports with /jobs add, and configuration.md lists the retired keys. - install.md: a Python venv anywhere else, and what Docker, a venv and a Debian host each install. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- README.md | 65 ++++++++++------ doc/commands.md | 181 +++++++++++++++---------------------------- doc/configuration.md | 19 +++-- doc/discord.md | 52 ++++++------- doc/docker.md | 6 +- doc/install.md | 39 ++++++---- doc/scheduling.md | 37 +++++---- doc/slack.md | 25 +++--- doc/telegram.md | 49 +++++------- 9 files changed, 217 insertions(+), 256 deletions(-) diff --git a/README.md b/README.md index 9d9e2039..64f02bbf 100644 --- a/README.md +++ b/README.md @@ -5,47 +5,66 @@ portfolios and each chat's watchlist with Yahoo Finance data. It posts trades, p dates, analyst rating changes and more on a schedule, and answers commands with reports and charts. ## Features -* Telegram, Slack and Discord, with no server to host: finbot fetches its updates itself (or, in HTTP mode, a network - posts them to your HTTPS server) -* Sharesight trades as they happen, portfolio performance and holdings -* Price moves past a threshold, earnings and ex-dividend dates, analyst rating changes, short interest, record highs -* Stock lookup, price history charts and comparisons, rankings by P/E, PEG, market cap or analyst rating -* Commands as messages (`.price`) or slash commands (`/price`), completed as they are typed where the network allows -* Each chat's own scheduled jobs and watchlist, managed in the chat (`.jobs`, `.watchlist`) -* Docker or a Debian host, hardened either way +* Telegram, Slack and Discord (either the push or the pull method, depending on your environment) +* Sharesight trades as they happen, portfolio performance comparisons +* Charts with buttons to change the time period +* Alerts of daily price movements past a threshold, earnings and ex-dividend dates, analyst rating changes, short + interest, record highs +* Stock lookup and comparisons, rankings by P/E, PEG, market cap or analyst rating +* Slash commands (`/price`), completed as they are typed where the network allows, or messages (`.price`), quicker to + type on a phone +* Each chat's own scheduled jobs and watchlist, managed in the chat (`/jobs`, `/watchlist`) +* Docker or a Debian host ![Screenshot of showing trade notifications on Slack](img/screenshot.png?raw=true "Screenshot showing trade notifications on Slack") ## Quick start -With Docker: +Run finbot with Docker or with Python. Both take the same keys, and both install every optional library, so each +network works in either mode. + +### With Docker +On a Linux host with Docker: 1. `git clone https://github.com/robdevops/finbot /opt/finbot` 2. `sudo /opt/finbot/util/docker_setup.sh` 3. Set the keys of each network you use in `/opt/finbot.env` (below) 4. `cd /opt/finbot && sudo docker compose up -d --build` 5. `journalctl CONTAINER_NAME=finbot -f` -To install on a Debian host with systemd instead, see [doc/install.md](doc/install.md). +### With Python +On Linux, macOS or WSL, with Python 3.12 or later: +1. `git clone https://github.com/robdevops/finbot ~/finbot && cd ~/finbot` +2. `python3 -m venv .venv && .venv/bin/pip install -r requirements.txt -r requirements-optional.txt` (on Debian and + Ubuntu, `sudo apt install python3-venv` first) +3. `cp defaults/finbot.env ~/finbot.env && chmod 600 ~/finbot.env`, then set the keys of each network you use in it + (below) +4. `.venv/bin/python util/check_config.py ~/finbot.env` +5. `(set -a && . ~/finbot.env && exec .venv/bin/python bot.py)`, which logs to the terminal until Ctrl-C + +To run it as a service, under systemd on a Debian host, see [doc/install.md](doc/install.md). -Each network needs a bot of its own, and finbot fetches its updates with no server (its pull mode): -* **Telegram:** create the bot with [@BotFather](https://telegram.me/BotFather), set `telegramBotToken` and - `telegramChatID`, restart finbot ([guide](doc/telegram.md)). +### The chat networks +Each network needs a bot of its own. By default, finbot fetches its updates from each network itself (pull mode). +Alternatively, if you prefer to run an HTTPS gateway (a reverse proxy on a domain with a valid certificate), HTTP mode +subscribes the bot to the updates each network pushes to it (each guide's _HTTP mode_). +* **Telegram:** create the bot with [@BotFather](https://telegram.me/BotFather), set `telegramBotToken`, restart + finbot ([guide](doc/telegram.md)). * **Slack:** create the app from the manifest that `util/slack_manifest.py` prints, add an app-level token, set - `slackBotToken`, `slackAppToken` and `slackChannelID`, restart finbot ([guide](doc/slack.md)). -* **Discord:** create the application and its bot, turn on its Message Content intent, set `discordBotToken` and - `discordChannelID`, restart finbot ([guide](doc/discord.md)). + `slackBotToken` and `slackAppToken`, restart finbot ([guide](doc/slack.md)). +* **Discord:** create the application and its bot, turn on its Message Content intent, set `discordBotToken`, restart + finbot ([guide](doc/discord.md)). -Then send `.help` in the chat. +Then send `/help` in a chat with the bot. Commands work as plain messages too (`.help`), which are quicker to type on a +phone ([Commands](doc/commands.md#supported-commands)). Each chat schedules its own reports with `/jobs add` +([Scheduling](doc/scheduling.md)). ## Documentation * [Commands and reports](doc/commands.md), with screenshots * [Scheduling](doc/scheduling.md): each chat's jobs, and the reports they can run * [Configuration](doc/configuration.md): `finbot.env` and `preferences.conf`, the data directory, Sharesight * [Caching](doc/caching.md) -* Installing: [Docker](doc/docker.md) · [a Debian host with systemd](doc/install.md) +* Installing: [Docker](doc/docker.md) · [Python, or a Debian host with systemd](doc/install.md) * Networks: [Telegram](doc/telegram.md) · [Slack](doc/slack.md) · [Discord](doc/discord.md) -## Suggestions -* Is my code doing something the hard way? -* Is something important missing from the docs? - -Log an [issue](https://github.com/robdevops/finbot/issues)! +## Issues +* [Bug reports](https://github.com/robdevops/finbot/issues) welcomed +* PRs are welcome, but not guaranteed to be merged diff --git a/doc/commands.md b/doc/commands.md index 85ad9889..0a40bf7c 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -7,7 +7,7 @@ What finbot can do on each network. Telegram takes everything; the others take w | Feature | Telegram | Slack | Discord | |---|---|---|---| -| Chat commands | ✓: messages (`.price`), and slash commands (`/price`) from the command menu | ✓: messages, and slash commands (`/price`) from the app's manifest | ✓: slash commands (`/price`), and messages over the gateway | +| Chat commands | ✓: slash commands (`/price`) from the command menu, and messages (`.price`) | ✓: slash commands (`/price`) from the app's manifest, and messages | ✓: slash commands (`/price`), and messages over the gateway API | | "typing…" while a command works | ✓ | ⏳ on the command's message (Slack has no typing notice for bots); none for a slash command, which has no message | ✓ for a message; "thinking…" for a slash command | | Charts in replies | ✓ | ✓ | ✓ | | Charts in scheduled reports | ✓ | ✓ | ✓ | @@ -29,42 +29,44 @@ Where: * _portfolio_ is a Sharesight portfolio name * _symbol_ is a Yahoo Finance symbol; a bare one Yahoo does not know is tried again on your exchange (`exchange_suffix`: `BHP` may be `BHP.AX`) ``` -.symbol -.stock symbol -.beta -.buy -.sell -.compare symbol symbol [symbol...] [period] -.dividend [period|symbol] -.earnings [period|symbol] -.forward pe [symbol|top|bottom|negative] -.history symbol -.holdings portfolio -.jobs [run|toggle] [number] -.jobs add [CRON_TZ=zone] time report -.jobs del number -.marketcap [symbol|top|bottom] -.pe [symbol|top|bottom] -.peg [symbol|top|bottom|negative] -.performance [period] [portfolio] -.pie portfolio [sector] -.plan [your plan] -.premarket [percent|symbol] -.price [percent|symbol|top|bottom] [period] -.session [percent|symbol] -.shorts [percent|symbol] -.trades [period] [portfolio] -.watchlist [add|del symbol] -.who symbol -``` - -`!` or `/` work in place of the dot, and so does the bot's name: `@botname price 7d`, `@botname AAPL`. Each command is also a slash command on every network, which completes its name as it is typed: Telegram's command menu, set as finbot starts; Slack's, from the app's manifest (`util/slack_manifest.py`, see the [Slack guide](slack.md#slash-commands)); Discord's, registered as finbot starts. `/stock symbol` (or `/profile symbol`) is `.symbol`, `/forwardpe` is `.forward pe`, and `/job` is `/jobs`. A command is its whole name, in any case, and so are its keywords (`.price TOP`); any other first word is a symbol. `.help` lists the commands in chat. +/stock symbol +/beta +/buy +/sell +/compare symbol symbol [symbol...] [period] +/dividend [period|symbol] +/earnings [period|symbol] +/forwardpe [symbol|top|bottom|negative] +/help +/history symbol +/holdings portfolio +/jobs [run|toggle] [number] +/jobs add [CRON_TZ=zone] time report +/jobs del number +/marketcap [symbol|top|bottom] +/pe [symbol|top|bottom] +/peg [symbol|top|bottom|negative] +/performance [period] [portfolio] +/pie portfolio [sector] +/plan [your plan] +/premarket [percent|symbol] +/price [percent|symbol|top|bottom] [period] +/session [percent|symbol] +/shorts [percent|symbol] +/trades [period] [portfolio] +/watchlist [add|del symbol] +/who symbol +``` + +Each command is a slash command on every network, which completes its name as it is typed: Telegram's command menu, set as finbot starts; Slack's, from the app's manifest (`util/slack_manifest.py`, see the [Slack guide](slack.md#slash-commands)); Discord's, registered as finbot starts. `/profile` is a second name of `/stock`, and `/job` of `/jobs`. `/help` lists the commands in chat. + +Every command also works as a plain message, which is quicker to type on a phone: a dot in place of the slash (`.price 7d`), or `!`, or the bot's name (`@botname price 7d`). As a message, `.forward pe` is `/forwardpe`, and a symbol alone is `/stock` (`.AAPL`, `@botname AAPL`). A message reaches the bot wherever it reads the chat: on Telegram, a direct chat, or a group where it is an administrator; on Slack, a direct message, a mention, or every message with `--every-message`; on Discord, in gateway API mode. A command is its whole name, in any case, and so are its keywords (`.price TOP`); any other first word is a symbol. Trade notifications are peformed by polling the Sharesight trades API on a schedule (see [Scheduling](scheduling.md)), and notifying each chat that schedules them of any new trades. Thus, it works best if your trades are auto-imported into Sharesight through its broker integrations, and if your environment has persistent storage so that the bot can keep track of known trade IDs between runs. The other various reports can also run on a schedule (e.g. daily or weekly), or on demand with a command. They query the Yahoo Finance API for stock data based on current holdings across your Sharesight portfolios, your friends' Sharesight portfolios, plus a custom watch list. A scheduled report posts to the chat whose job it is, and a command's reply to the chat that sent it. -The bot answers commands in every chat it is in. It fetches its updates from each network itself, with no server to host (pull mode); in HTTP mode instead, a network posts them to a web service you host on a domain with a trusted certificate. It provides: +The bot answers commands in every chat it is in. By default, it fetches its updates from each network itself (pull mode); alternatively, in HTTP mode, each network pushes them to your HTTPS gateway, which passes them to the bot. It provides: * Stock lookup (financials, company profile, price charts) * A watch list per chat, which anyone in it can change, picked up by the reports there * Listing of portfolios and portfolio performance @@ -106,11 +108,9 @@ The stock lookup returns various stats relevant to a stock's valuation, growth a This report is a command only. Example usage: ``` -.AAPL -``` -``` -@botname AAPL +/stock AAPL ``` +As a message: `.AAPL`, or `@botname AAPL`. ### Trades ![trade update in Slack](../img/trades.png?raw=true "Trade update in Slack") @@ -126,25 +126,19 @@ With persistent storage, the default of `past_days=28` suits. This is useful if past_days=28 ``` -Job (see [Scheduling](scheduling.md)), such as `.jobs add */20 * * * Mon-Fri trades`: +Job (see [Scheduling](scheduling.md)), such as `/jobs add */20 * * * Mon-Fri trades`: ``` trades ``` Command: ``` -.trades [period] -``` -``` -@botname trades [period] +/trades [period] ``` You can also specify a portfolio name to get trades for just that portfolio: ``` -.trades [period] [portfolio] -``` -``` -@botname trades [period] [portfolio] +/trades [period] [portfolio] ``` ### Price alerts @@ -161,7 +155,7 @@ price_percent=5 demote_volatile=True ``` -Job (see [Scheduling](scheduling.md)), such as `.jobs add 29 21 * * Mon-Fri price interday`: +Job (see [Scheduling](scheduling.md)), such as `/jobs add 29 21 * * Mon-Fri price interday`: ``` price premarket|midsession|interday|close|DAYS ``` @@ -174,36 +168,27 @@ When scheduled, the mode is its argument: Command (pre-market): ``` -.premarket [percent] -``` -``` -@botname premarket [percent] +/premarket [percent] ``` Command (mid-session): ``` -.session [percent] -``` -``` -@botname session [percent] +/session [percent] ``` Command (interday). ``` -.price [percent] -``` -``` -@botname price [percent] +/price [percent] ``` -For interday, can also specify the time period. A bare number is the percent; a period carries its unit: `7d`, `2w`, `3m`, `1y` or `ytd`. For example, `.price 3 7d` lists the moves of 3% or more over the past week. +For interday, can also specify the time period. A bare number is the percent; a period carries its unit: `7d`, `2w`, `3m`, `1y` or `ytd`. For example, `/price 3 7d` lists the moves of 3% or more over the past week. ``` -.price [percent] [period] +/price [percent] [period] ``` Or a specific stock. In this case, the output includes a graph. ``` -.price [symbol] [period] +/price [symbol] [period] ``` ### Portfolio performance @@ -216,25 +201,19 @@ Config example: past_days=28 ``` -Job (see [Scheduling](scheduling.md)), such as `.jobs add 28 21 * * Sat performance 7`: +Job (see [Scheduling](scheduling.md)), such as `/jobs add 28 21 * * Sat performance 7`: ``` performance [DAYS] ``` Command: ``` -.performance [period] -``` -``` -@botname performance [period] +/performance [period] ``` As a command, it also takes a portfolio: ``` -.performance [period] [[portfolio] -``` -``` -@botname performance [period] [portfolio] +/performance [period] [portfolio] ``` ### Events calendar @@ -249,25 +228,19 @@ The `cal` report sends upcoming earnings and ex-dividend date alerts. The data i future_days=30 ``` -Job (see [Scheduling](scheduling.md)), such as `.jobs add 28 21 * * Sat cal earnings`: +Job (see [Scheduling](scheduling.md)), such as `/jobs add 28 21 * * Sat cal earnings`: ``` cal earnings|ex-dividend ``` Command (earnings): ``` -.earnings [period] -``` -``` -@botname earnings [period] +/earnings [period] ``` Command (ex-dividend): ``` -.dividend [period] -``` -``` -@botname dividend [period] +/dividend [period] ``` ### Highly shorted stock warnings @@ -282,10 +255,7 @@ shorts_percent=15 Command: ``` -.shorts [percent] -``` -``` -@botname shorts [percent] +/shorts [percent] ``` ### pe / forward pe / peg @@ -295,14 +265,9 @@ List securities by P/E ratio, forward P/E ratio, or PEG ratio. A forward P/E is Command: ``` -.pe [SYMBOL] -.fpe [SYMBOL|bottom|negative] -.peg [SYMBOL|bottom|negative] -``` -``` -@botname pe [SYMBOL|top|bottom] -@botname fpe [SYMBOL|top|bottom|negative] -@botname peg [SYMBOL|top|bottom|negative] +/pe [SYMBOL|top|bottom] +/forwardpe [SYMBOL|top|bottom|negative] +/peg [SYMBOL|top|bottom|negative] ``` ### History @@ -313,10 +278,7 @@ It creates a 5Y chart, and then lists performance for several periods (5Y 3Y 1Y Command: ``` -.history -``` -``` -@botname history +/history ``` ### Buy @@ -326,10 +288,7 @@ Show top recommended buys by Yahoo Finance analysts rating, among stocks rated b Command: ``` -.buy -``` -``` -@botname buy +/buy ``` ### Sell @@ -339,10 +298,7 @@ Show top recommended sells by Yahoo Finance analysts rating, among stocks rated Command: ``` -.sell -``` -``` -@botname sell +/sell ``` ### Marketcap @@ -352,16 +308,13 @@ List the top (or bottom) stocks by market cap, in US dollars: market caps in oth Command: ``` -.marketcap [top|bottom|SYMBOL] -``` -``` -@botname marketcap [top|bottom|SYMBOL] +/marketcap [top|bottom|SYMBOL] ``` ### Consensus rating changes ![Rating change report in Slack](../img/finbot_rating.png?raw=true "Rating change report in Slack") -Notifies when Yahoo Finance analyst rating consensus changes category (e.g. from hold to buy). Designed to run on a schedule, such as `.jobs add 28 21 * * Sat rating` (see [Scheduling](scheduling.md)). Requires persistent storage. +Notifies when Yahoo Finance analyst rating consensus changes category (e.g. from hold to buy). Designed to run on a schedule, such as `/jobs add 28 21 * * Sat rating` (see [Scheduling](scheduling.md)). Requires persistent storage. ``` rating @@ -372,18 +325,12 @@ rating Tracks additional securities which are not in your Sharesight holdings. Each chat has its own watchlist, which anyone in the chat can change: the reports in that chat, asked for or scheduled, include its watchlist's stocks as if they were holdings, and no other chat's. The Sharesight holdings are the same in every chat. `yahoo_warm` keeps every chat's watchlist warm in the cache. -The watchlists are kept in `finbot_watchlist.json` in the data directory, by chat (`{"chats": {"telegram:-1001234": ["2454.TW", "ASO.AX", "STEM"]}}`), so there is no need to edit the file. The first start with watchlists per chat gives each home chat the watchlist kept before, and the same goes for the plans that `.plan` records: each chat's `.plan` lists only the plans recorded in it. +The watchlists are kept in `finbot_watchlist.json` in the data directory, by chat (`{"chats": {"telegram:-1001234": ["2454.TW", "ASO.AX", "STEM"]}}`), so there is no need to edit the file. An upgrade from before watchlists per chat gives each old home chat the watchlist kept before, and the same goes for the plans that `/plan` records: each chat's `/plan` lists only the plans recorded in it. Command: ``` -.watchlist -``` -``` -.watchlist [add|del] [symbol] -``` -``` -@botname watchlist +/watchlist ``` ``` -@botname watchlist [add|del] [symbol] +/watchlist [add|del] [symbol] ``` diff --git a/doc/configuration.md b/doc/configuration.md index e209682e..92203c42 100644 --- a/doc/configuration.md +++ b/doc/configuration.md @@ -12,7 +12,7 @@ sudo cp ~/finbot/defaults/finbot.env /opt/finbot.env && sudo chmod 600 /opt/finb * **systemd:** `EnvironmentFile=/opt/finbot.env` in `finbot.service`, plus `ExecStartPre=+.../util/check_config.py /opt/finbot.env` for pre-flight lint. * **Docker:** `compose.yaml` loads it, and checks it before the bot starts (see [docker.md](docker.md)). Write the values without quotes. * **init.d / shell:** `util/check_config.py /opt/finbot.env && set -a && . /opt/finbot.env && set +a && ./bot.py` (`set -a` is what exports the values to the bot). -* **Scheduled jobs:** they are run by `bot.py` (see [Scheduling](scheduling.md)), so they run on the same settings. For a one-off run, send `.jobs run` in the chat. +* **Scheduled jobs:** they are run by `bot.py` (see [Scheduling](scheduling.md)), so they run on the same settings. For a one-off run, send `/jobs run` in the chat. The startup keys, read by your launcher. A change needs a restart. Blank means unset. Each network's guide says which of the chat network keys it needs. @@ -20,27 +20,26 @@ of the chat network keys it needs. | Key | Default | Description | |---|---|---| | `DATA_DIR` | `var/` in the checkout (`$STATE_DIRECTORY` under systemd `StateDirectory=`) | Where state and cache live | -| `ip` | `127.0.0.1` | Address the bot listens on | -| `port` | `5000` | Port the bot listens on | +| `ip` | `127.0.0.1` | Address the bot listens on, in HTTP mode | +| `port` | `5000` | Port the bot listens on, in HTTP mode | | `scheduler_catchup` | `5` | Minutes: a run missed while the bot was down still fires if it was due this recently | -| `telegramChatID` | | Telegram's home chat, a group or channel id: finbot's first start gives it the default jobs, and error reports go there (see [Scheduling](scheduling.md)) | | `sharesight_code` | | Sharesight API access | | `sharesight_client_id` | | Sharesight API access | | `sharesight_client_secret` | | Sharesight API access | | `telegramBotToken` | | Telegram bot token: reports, and commands (finbot fetches its updates unless `telegramOutgoingWebhook` is set) | | `slackBotToken` | | Slack bot token, for replies and chart uploads in Slack | | `slackAppToken` | | Slack app-level token (`xapp-`): commands over a socket finbot opens to Slack (Socket Mode), when `slackOutgoingWebhook` is empty | -| `slackChannelID` | | Slack's home channel id, as `telegramChatID` (with `slackBotToken`) | | `telegramOutgoingWebhook` | | The URL Telegram posts its updates to (HTTP mode); empty, finbot fetches them (pull mode) | | `telegramOutgoingToken` | | Secret token Telegram must present (HTTP mode) | | `slackOutgoingWebhook` | | The URL Slack posts its events and button presses to (HTTP mode); empty, Socket Mode with `slackAppToken` | | `slackOutgoingToken` | | Token Slack must present (HTTP mode): the older way, which Slack has deprecated; not needed with `slackSigningSecret` | | `slackSigningSecret` | | Secret Slack signs each request with (HTTP mode) | -| `discordBotToken` | | Discord bot token: replies, and commands over the gateway unless `discordOutgoingWebhook` is set | -| `discordChannelID` | | Discord's home channel id, as `telegramChatID` (with `discordBotToken`) | -| `discordOutgoingWebhook` | | The Interactions Endpoint URL Discord posts slash commands and button presses to (HTTP mode); empty, the gateway (pull mode) | +| `discordBotToken` | | Discord bot token: replies, and commands over the gateway API unless `discordOutgoingWebhook` is set | +| `discordOutgoingWebhook` | | The Interactions Endpoint URL Discord posts slash commands and button presses to (HTTP mode); empty, the gateway API (pull mode) | | `discordPublicKey` | | The app's public key, which checks the signature of each interaction (HTTP mode) | +Retired keys do nothing, and only get a note at start (and from check_config) saying to delete the line: `telegramChatID`, `slackChannelID` and `discordChannelID` (there are no home chats: each chat has its own jobs, and an error that belongs to no chat goes to the log; an upgrade from before jobs per chat reads them once, as [Scheduling](scheduling.md) says), and `slack_webhook` and `discord_webhook`. + ## `preferences.conf` items The preferences, read by the bot itself from `/preferences.conf`; without that file, finbot runs on the built-in defaults below. `defaults/preferences.conf` is only its template: every key in it is commented out at its default, and finbot never reads it. To change a preference, copy the template into the data directory (this never overwrites a file there): @@ -57,7 +56,7 @@ A change takes effect on a reload. A true/false preference takes `true`/`false`, | `future_days` | `30` | How many days ahead the events calendar looks | | `price_percent` | `5` | Price alert threshold, percent | | `shorts_percent` | `15` | Short interest alert threshold, percent | -| `min_analysts` | `7` | `.buy`, `.sell` and `.forward pe` list only stocks covered by at least this many analysts; with fewer, a consensus says little (thinly covered small caps are nearly all a perfect buy) | +| `min_analysts` | `7` | `/buy`, `/sell` and `/forwardpe` list only stocks covered by at least this many analysts; with fewer, a consensus says little (thinly covered small caps are nearly all a perfect buy) | | `demote_volatile` | `True` | Volatile (small cap, high beta) stocks need a bigger move to alert | | `demote_leveraged` | `True` | Leveraged (2x, 3x) funds need a bigger move to alert | | `volatility_multiplier` | `2` | How much bigger a move volatile stocks need | @@ -70,7 +69,7 @@ A change takes effect on a reload. A true/false preference takes `true`/`false`, | `hyperlinkProvider` | `google` | Where ticker links go: `google` or `yahoo` | | `exchange_suffix` | `AX` | Regional: Yahoo's suffix for your exchange (`AX` for the ASX, `NZ`, `L`, `TO` and so on). A bare symbol Yahoo does not know is tried again with it, in every command that takes a symbol (`BHP` becomes `BHP.AX`); empty, a symbol is taken as written | | `country_code` | `AU` | Regional: your country, for the reminder report | -| `timezone` | `Australia/Melbourne` | Regional: your time zone, for the reminder report, and for each job's next run in `.jobs` (besides UTC) | +| `timezone` | `Australia/Melbourne` | Regional: your time zone, for the reminder report, and for each job's next run in `/jobs` (besides UTC) | | `cache` | `True` | Cache Yahoo and Sharesight responses | | `cache_seconds` | `43200` | Cache lifetime in seconds (12 hours) | | `performance_cache_seconds` | `2400` | Cache lifetime of Sharesight performance reports in seconds: longer than the 30 minutes between `sharesight_warm` runs, so the holdings never run out | diff --git a/doc/discord.md b/doc/discord.md index 80907301..2da5295b 100644 --- a/doc/discord.md +++ b/doc/discord.md @@ -2,30 +2,30 @@ finbot talks to Discord as the bot user of a Discord application that you create in the Developer Portal. The bot posts each channel's scheduled reports there and answers commands: slash commands (`/price`) in both modes, and -messages (`.price`, `@finbot price`) over the gateway. +messages (`.price`, `@finbot price`) over the gateway API. -## Gateway mode or HTTP mode? +## Gateway API mode or HTTP mode? The bot does the same either way. What differs is how it hears from Discord: -| | Gateway mode (the default) | HTTP mode | +| | Gateway API mode (the default) | HTTP mode | |---|---|---| -| How commands and button presses reach finbot | Over Discord's gateway, a connection finbot keeps open | Discord posts them to your HTTPS server | -| Public HTTPS server | Not needed | Needed: a domain name with a valid certificate | +| How commands and button presses reach finbot | Over Discord's gateway API, a connection finbot keeps open to Discord | Discord posts them to your HTTPS server | +| Listens on the internet | No: finbot opens the connection to Discord | Yes, behind your HTTPS server: a domain name with a valid certificate | | Commands | Slash commands (`/price`), and messages (`.price`, `@finbot price`) | Slash commands (`/price`): Discord posts no messages | | Python modules | `discord.py` (Debian: `python3-discord`) | `PyNaCl` and `waitress` (Debian: `python3-nacl`, `python3-waitress`) | -| `finbot.env` keys | `discordBotToken`, `discordChannelID` | Those two, plus `discordOutgoingWebhook` and `discordPublicKey` | +| `finbot.env` keys | `discordBotToken` | `discordBotToken`, `discordOutgoingWebhook` and `discordPublicKey` | -In both: the scheduled reports with their charts; period buttons under charts and the buttons of `.jobs run` and -`.jobs toggle`, which redraw their message in place; linked tickers, without link previews and without pinging anyone; +In both: the scheduled reports with their charts; period buttons under charts and the buttons of `/jobs run` and +`/jobs toggle`, which redraw their message in place; linked tickers, without link previews and without pinging anyone; and direct messages with the bot. A command in a message shows "finbot is typing…" while it works; a slash command shows Discord's "finbot is thinking…" until its answer takes its place. -`discordOutgoingWebhook` chooses the mode: set, HTTP mode; empty, gateway mode. Discord sends the slash commands and -button presses to the app's _Interactions Endpoint URL_ while it is set, and over the gateway while it is empty, so -change both, and restart finbot, to switch. +`discordOutgoingWebhook` chooses the mode: set, HTTP mode; empty, gateway API mode. Discord sends the slash commands +and button presses to the app's _Interactions Endpoint URL_ while it is set, and over the gateway API while it is +empty, so change both, and restart finbot, to switch. -## Gateway mode +## Gateway API mode 1. In the [Developer Portal](https://discord.com/developers/applications), choose _New Application_ and name it (finbot, say). @@ -37,27 +37,26 @@ change both, and restart finbot, to switch. 3. Under _OAuth2 > URL Generator_, tick the scopes `bot` and `applications.commands`, then the bot permissions _View Channels_, _Send Messages_, _Attach Files_ and _Read Message History_. Open the URL it makes, and add the bot to your server. -4. In Discord, turn on _User Settings > Advanced > Developer Mode_; then right-click the channel to be Discord's home - chat, _Copy Channel ID_, and put it in `finbot.env` as `discordChannelID`. finbot's first start gives that channel - the default jobs ([Scheduling](scheduling.md)), and it gets the error reports. Every channel manages its - own jobs with `.jobs`. -5. Install discord.py: `sudo apt install python3-discord` (or `pip install discord.py`). -6. Restart finbot (`sudo systemctl restart finbot`). As it starts, it registers the slash commands, which can take a +4. Install discord.py: `sudo apt install python3-discord` (or `pip install discord.py`; Docker and the Python quick + start have it). +5. Restart finbot (`sudo systemctl restart finbot`). As it starts, it registers the slash commands, which can take a few minutes to show in Discord the first time, and the log says `discord: gateway (no discordOutgoingWebhook)`. -7. Send `/help` (or `.help`) in the channel, or in a direct message to the bot. +6. Send `/help` (or `.help`) in the channel, or in a direct message to the bot. Each channel schedules its own reports + with `/jobs add` ([Scheduling](scheduling.md)). ``` discordBotToken=your-bot-token -discordChannelID=1000000000000000000 ``` ## HTTP mode Discord posts the slash commands and button presses to finbot over HTTPS, each signed with the app's key. finbot listens on `http://127.0.0.1:5000` (`ip` and `port` in `finbot.env`), so it needs a web server in front of it, on a -domain name with a valid certificate. The bot shows as offline in Discord, as it keeps no gateway connection. +domain name with a valid certificate. The bot shows as offline in Discord, as it keeps no connection to the gateway +API. -1. Set up the bot as for [gateway mode](#gateway-mode), steps 1 to 4. The _Message Content Intent_ is not needed. +1. Set up the bot as for [gateway API mode](#gateway-api-mode), steps 1 to 3. The _Message Content Intent_ is not + needed. 2. Install PyNaCl, which checks the signatures, and waitress, the web server: `sudo apt install python3-nacl python3-waitress` (or `pip install PyNaCl waitress`). 3. Put finbot behind an HTTPS server. An nginx example: @@ -86,17 +85,16 @@ domain name with a valid certificate. The bot shows as offline in Discord, as it ``` discordBotToken=your-bot-token -discordChannelID=1000000000000000000 discordOutgoingWebhook=https://www.example.com:8443/discord discordPublicKey=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef ``` ## Slash commands -finbot registers a slash command for each line of `.help`, as it starts, and again only when the list has changed: -`/price`, `/dividend` and `/earnings`, `/buy` and `/sell`, `/forwardpe` for `.forward pe`, `/stock` (or `/profile`) for -`.SYMBOL`, `/jobs` (or `/job`), and the rest. Each takes its arguments, if any, as one text option, as the message -would: `/price` with `5 7d` is `.price 5 7d`. They work in the server's channels and in a direct message with the bot. +finbot registers a slash command for each line of `/help`, as it starts, and again only when the list has changed: +`/price`, `/dividend` and `/earnings`, `/buy` and `/sell`, `/forwardpe`, `/stock` (or `/profile`), `/jobs` (or `/job`), +and the rest. Each takes its arguments, if any, as one text option, as a message would: `/price` with `5 7d` is the +message `.price 5 7d`. They work in the server's channels and in a direct message with the bot. While the arguments are typed, Discord shows finbot's suggestions, in both modes, for the word being typed: * the command's keywords (`top`, `bottom`, `negative`, `sector`, `add`, `del`) and periods (`7d`, `3m`, `ytd`), until diff --git a/doc/docker.md b/doc/docker.md index 61521071..e5d0bc15 100644 --- a/doc/docker.md +++ b/doc/docker.md @@ -8,8 +8,8 @@ The configuration lives on the host, beside the checkout, so a rebuild or an upd 1. `git clone https://github.com/robdevops/finbot /opt/finbot` 2. `sudo /opt/finbot/util/docker_setup.sh` -3. Set `telegramBotToken` and `telegramChatID` in `/opt/finbot.env` (or another network's keys: see the - [Telegram](telegram.md), [Slack](slack.md) and [Discord](discord.md) guides). +3. Set `telegramBotToken` in `/opt/finbot.env` (or another network's keys: see the [Telegram](telegram.md), + [Slack](slack.md) and [Discord](discord.md) guides). 4. `cd /opt/finbot && sudo docker compose up -d --build` 5. `journalctl CONTAINER_NAME=finbot -f` @@ -18,7 +18,7 @@ The configuration lives on the host, beside the checkout, so a rebuild or an upd | Path | What | In the container | |---|---|---| | `/opt/finbot` | the checkout, with `Dockerfile` and `compose.yaml` | the image's `/opt/finbot`, copied at build | -| `/opt/finbot.env` | the startup file: tokens, chat ids, the mode of each network | the environment, and `/run/finbot.env`, read-only, for the check | +| `/opt/finbot.env` | the startup file: the tokens, and the mode of each network | the environment, and `/run/finbot.env`, read-only, for the check | | `/opt/finbot-data` | `DATA_DIR`: `preferences.conf`, each chat's jobs and watchlist, the records, the cache | `/data` | `compose.yaml` finds the other two beside the checkout (`../finbot.env`, `../finbot-data`): clone it elsewhere and diff --git a/doc/install.md b/doc/install.md index 12624d13..5ec08548 100644 --- a/doc/install.md +++ b/doc/install.md @@ -1,21 +1,24 @@ # Installing on a host -finbot runs on any Linux host with Python 3.12 or later; production runs Debian's own packages under systemd, as -below. To run it in a container instead, see [docker.md](docker.md). +finbot runs wherever Python 3.12 or later does (Linux, macOS, WSL); production runs Debian's own packages under +systemd, as below. To run it in a container instead, see [docker.md](docker.md). ## Requirements * Most features need a Sharesight paid plan, preferably with automatic trade imports, and an API key ([configuration.md](configuration.md#sharesight)). * A Telegram bot, a Slack app or a Discord bot: see the network's guide, [Telegram](telegram.md), [Slack](slack.md) or [Discord](discord.md). -* No server: finbot fetches its updates from each network itself (pull mode: Telegram's long polling, Slack's Socket - Mode, Discord's gateway). In HTTP mode instead, a network posts them to a public web server with a domain name and a - matching certificate, which proxies them to the bot (each guide's _HTTP mode_). +* By default, finbot fetches its updates from each network itself (pull mode: Telegram's long polling, Slack's Socket + Mode, Discord's gateway API). It only makes outgoing connections, so nothing has to listen on the internet. + Alternatively, if you prefer to run an HTTPS gateway (a web server with a domain name and a matching certificate), + HTTP mode subscribes the bot to push updates: each network posts its updates to the gateway, which passes them to + the bot (each guide's _HTTP mode_). * Python 3.13.5 (the version in `.python-version`; 3.12 is the minimum), and these modules, pinned to versions with 3.13 support in `requirements.txt`: croniter, python-dateutil, requests, matplotlib and numpy. * For a network in HTTP mode (its webhook URL set), also waitress; for Slack in Socket Mode, websocket-client; for - Discord's gateway, discord.py, and for its HTTP mode, PyNaCl. These are in `requirements-optional.txt`, each imported - only by the mode that needs it. + Discord's gateway API, discord.py, and for its HTTP mode, PyNaCl. These are in `requirements-optional.txt`, each + imported only by the mode that needs it. Docker and a Python venv install them all, Docker in its image and a venv + from that file, so every mode works without choosing; on a Debian host, install the packages of the modes you use. ## Debian 1. The packages, and those of the modes you use: @@ -23,7 +26,7 @@ below. To run it in a container instead, see [docker.md](docker.md). sudo apt install git python3 python3-requests python3-croniter python3-dateutil python3-matplotlib python3-numpy sudo apt install python3-waitress # any network in HTTP mode sudo apt install python3-websocket # Slack's Socket Mode - sudo apt install python3-discord # Discord's gateway + sudo apt install python3-discord # Discord's gateway API sudo apt install python3-nacl # Discord's HTTP mode ``` 2. The code: @@ -37,22 +40,32 @@ below. To run it in a container instead, see [docker.md](docker.md). ``` 4. Check it: `~/finbot/util/check_config.py /opt/finbot.env` 5. Run it under systemd, below. The first start makes the data directory (`var/` in the checkout, or - `/var/lib/finbot` with `DynamicUser=`) and gives each home chat its first jobs ([scheduling.md](scheduling.md)). + `/var/lib/finbot` with `DynamicUser=`); each chat then schedules its own reports with `/jobs add` + ([scheduling.md](scheduling.md)). [configuration.md](configuration.md) has every key and preference. -## Other distributions: pip or uv +## Anywhere else: a Python venv +On Linux, macOS or WSL, with git and Python 3.12 or later (on Debian and Ubuntu, also `python3-venv`): ``` -sudo $(which apt dnf yum) install git python3-pip git clone https://github.com/robdevops/finbot.git ~/finbot cd ~/finbot -pip3 install -r requirements.txt -r requirements-optional.txt +python3 -m venv .venv +.venv/bin/pip install -r requirements.txt -r requirements-optional.txt ``` Or, with [uv](https://docs.astral.sh/uv/), which also fetches the exact Python from `.python-version`: ``` uv venv && uv pip install -r requirements.txt -r requirements-optional.txt ``` -`requirements-optional.txt` can be left out, or cut down to the modes you use. +`requirements-optional.txt` installs the library of every mode; it can be cut down to the modes you use. Then make the +startup file as in step 3 above (anywhere: `~/finbot.env`, say), check it, and start the bot with its values in its +environment: +``` +.venv/bin/python util/check_config.py ~/finbot.env +(set -a && . ~/finbot.env && exec .venv/bin/python bot.py) +``` +Under systemd (below), `ExecStart=` runs the venv's Python: `/home//finbot/.venv/bin/python +/home//finbot/bot.py`. ## Daemonize (systemd) `finbot.service` can take care of keeping `bot.py` running in the background and starting it on boot. Copy diff --git a/doc/scheduling.md b/doc/scheduling.md index 94ee1421..9dd7ba92 100644 --- a/doc/scheduling.md +++ b/doc/scheduling.md @@ -1,14 +1,15 @@ # Scheduling Each chat has its own jobs: reports that run on a schedule and post to that chat. Anyone in the chat manages them there, -with `.jobs` (also `.job`, `.cron`, `.crons` or `.schedule`): +with `/jobs` (or `/job`; as a message, also `.cron`, `.crons` or `.schedule`). A new chat has none until it adds +them: ``` -.jobs this chat's jobs, numbered, each with its next run; then the system jobs -.jobs add the reports a job can run, and what each takes -.jobs add 29 21 * * Mon-Fri price interday -.jobs run the list, with a button per job that runs it now; .jobs run 2 runs job 2 -.jobs toggle the list, with a button per job that switches it on or off; .jobs toggle 2 -.jobs del 2 deletes job 2 +/jobs this chat's jobs, numbered, each with its next run; then the system jobs +/jobs add the reports a job can run, and what each takes +/jobs add 29 21 * * Mon-Fri price interday +/jobs run the list, with a button per job that runs it now; /jobs run 2 runs job 2 +/jobs toggle the list, with a button per job that switches it on or off; /jobs toggle 2 +/jobs del 2 deletes job 2 ``` A job is a time in crontab's syntax, then one or more reports with their arguments: @@ -17,29 +18,31 @@ A job is a time in crontab's syntax, then one or more reports with their argumen saying why. * **Times are in UTC**, which keeps the ASX's jobs and NYSE's each within an hour of their local times all year. To follow one market's clocks exactly, a job names its zone first, as crontab writes it: - `.jobs add CRON_TZ=America/New_York 15 13 * * Mon-Fri price midsession`. `.jobs` shows each next run in UTC and in + `/jobs add CRON_TZ=America/New_York 15 13 * * Mon-Fri price midsession`. `/jobs` shows each next run in UTC and in the `timezone` preference. * **The reports** are `trades`, `price midsession|interday|premarket|close|DAYS`, `cal earnings|ex-dividend`, `performance [DAYS]`, `rating`, `milestone`, `shorts`, `reminder` and `podcasts FEED_URL`. Several, separated by `;`, - run one after another: `.jobs add 28 21 * * Sat cal earnings; cal ex-dividend; price 7; performance 7; rating`. + run one after another: `/jobs add 28 21 * * Sat cal earnings; cal ex-dividend; price 7; performance 7; rating`. Quote an argument that holds spaces or a `;` (a feed URL, say). * A chat has 30 jobs at most. A job posts to its own chat, whichever way that chat's updates reach finbot: the jobs are kept by network and chat id (`finbot_schedule.json` in the data directory), so switching a network between pull and HTTP mode keeps them. Each chat also keeps its own record of what a report has told it (trades, rating changes, record highs, podcast episodes), so the -same report scheduled in two chats tells each of them once. A run from `.jobs run` is a scheduled run: it posts to the +same report scheduled in two chats tells each of them once. A run from `/jobs run` is a scheduled run: it posts to the chat and updates the same record. A change counts at once, with no reload. -**The system jobs**, fixed in `lib/jobs.py` and listed by `.jobs` under _System_, keep the cache full and tidy: +**The system jobs**, fixed in `lib/jobs.py` and listed by `/jobs` under _System_, keep the cache full and tidy: `sharesight_warm` every 30 minutes, `yahoo_warm` every 8 hours (see [Caching](caching.md)), and `cleanup 30` weekly, -which deletes cache files older than 30 days. A chat cannot add, switch or run them. +which deletes cache files older than 30 days. A chat cannot add, switch or run them, and a failure of one goes to the +log. -**From `schedule.cron`**, where the schedule used to be: the first time finbot starts without `finbot_schedule.json`, -it gives each home chat (`telegramChatID`, `slackChannelID`, `discordChannelID`) the jobs of the data directory's -`schedule.cron`, else of the shipped `defaults/schedule.cron`, each in the zone it ran in (the server's), without the -system jobs. It copies the reports' records to each home chat too, so that nothing old is told again. It logs what it -did, and any line it left out, and reads the file no more: delete the data directory's copy afterwards. +**From `schedule.cron`**, where the schedule used to be: on an upgrade from before jobs per chat, the first start +gives each chat that the retired `telegramChatID`, `slackChannelID` or `discordChannelID` still names in `finbot.env` +the jobs of the data directory's `schedule.cron`, else of the shipped `defaults/schedule.cron`, each in the zone it +ran in (the server's), without the system jobs. It copies the reports' records to those chats too, so that nothing old +is told again. It logs what it did, and any line it left out, and reads the file no more: delete the data directory's +copy afterwards, and the retired keys. Behaviour worth knowing: * The reports run in `bot.py` itself, each in a thread of its own, sharing the bot's user, sandbox, environment, data diff --git a/doc/slack.md b/doc/slack.md index c3d3dd01..ea72cd3d 100644 --- a/doc/slack.md +++ b/doc/slack.md @@ -1,9 +1,9 @@ # Slack finbot reaches Slack through a Slack app of your own, whose bot posts each channel's scheduled reports there, charts -and all, and answers commands: `/price`, `.price`, `@finbot price` and the rest -([the list](commands.md#supported-commands)), with charts in its replies, ⏳ on a command's message while it works -(Slack has no typing notice for bots), and period buttons under a chart that redraw it in place. +and all, and answers commands: `/price` and the rest ([the list](commands.md#supported-commands)), or as messages, +`@finbot price` (and `.price` in a direct message), with charts in its replies, ⏳ on a command's message while it +works (Slack has no typing notice for bots), and period buttons under a chart that redraw it in place. ## The bot @@ -14,7 +14,7 @@ The bot does the same either way. What differs is how it hears from Slack: | | Socket Mode (the default) | HTTP mode | |---|---|---| | How messages, slash commands and button presses reach finbot | Over a socket that finbot opens to Slack | Slack posts them to your HTTPS server | -| Public HTTPS server | Not needed | Needed: a domain name with a valid certificate | +| Listens on the internet | No: finbot opens the socket to Slack | Yes, behind your HTTPS server: a domain name with a valid certificate | | Python module | `websocket-client` (Debian: `python3-websocket`) | `waitress` (Debian: `python3-waitress`) | | `finbot.env` keys | `slackBotToken`, `slackAppToken` | `slackBotToken`, `slackOutgoingWebhook`, `slackSigningSecret` | @@ -40,7 +40,6 @@ switch, change both (paste the manifest for the other mode, which sets the app's ``` slackAppToken=xapp-your-app-level-token slackBotToken=xoxb-your-bot-token -slackChannelID=C0123456789 ``` ### HTTP mode @@ -89,7 +88,6 @@ with a valid certificate. slackOutgoingWebhook=https://www.example.com:8443/slack slackSigningSecret=0123456789abcdef0123456789abcdef slackBotToken=xoxb-your-bot-token -slackChannelID=C0123456789 ``` ### Installing the app @@ -99,20 +97,17 @@ slackChannelID=C0123456789 startup and for each reply. 2. Restart finbot, and invite the bot to the channel: `/invite @finbot`. In Socket Mode the log says `slack: Socket Mode (no slackOutgoingWebhook)`. -3. Send `/help` or `@finbot help` in the channel (or `.help`, with `--every-message`), or a direct message to the bot - in its _Messages_ tab. -4. Set `slackChannelID` to the channel's id (in Slack, click the channel's name: the _Channel ID_, which starts with - C, is at the bottom of _About_), and restart finbot. It is Slack's home chat: finbot's first start gives it the - default jobs ([Scheduling](scheduling.md)), and it gets the error reports. Every channel manages its own - jobs with `.jobs`. +3. Send `/help` (or `@finbot help`) in the channel, or in a direct message to the bot in its _Messages_ tab, where + `.help` works too (and in the channels, with `--every-message`). Each channel schedules its own reports with + `/jobs add` ([Scheduling](scheduling.md)). An app set up by hand before takes the manifest too: paste it into _Features > App Manifest_ and _Save Changes_; Slack shows what changes, and asks to reinstall the app when its scopes do. ### Slash commands -Each command is a slash command too, with its usage as Slack's hint: `/price`, `/dividend` and `/earnings`, `/buy` -and `/sell`, `/forwardpe` for `.forward pe`, `/stock` (or `/profile`) for `.SYMBOL`, `/jobs` (or `/job`), and the rest. +Each command is a slash command, with its usage as Slack's hint: `/price`, `/dividend` and `/earnings`, `/buy` and +`/sell`, `/forwardpe`, `/stock` (or `/profile`), `/jobs` (or `/job`), and the rest. Slack makes slash commands only from the app's settings, so they come in the manifest: when finbot's commands change, print it again and paste it into _App Manifest_. @@ -128,5 +123,5 @@ drawn and uploaded anew (Slack cannot replace a file). An upload that no message are past, so the bot's files do not pile up. If Slack will not show a chart with its buttons, the chart is uploaded into the channel without them; if an upload fails (a missing scope, say), the report goes out as text. The log says why. -`.jobs run` and `.jobs toggle` list the channel's jobs with a button for each, as on Telegram; a button from a list +`/jobs run` and `/jobs toggle` list the channel's jobs with a button for each, as on Telegram; a button from a list that has since changed puts the current list in its place. diff --git a/doc/telegram.md b/doc/telegram.md index e74c381a..0ba22b54 100644 --- a/doc/telegram.md +++ b/doc/telegram.md @@ -10,18 +10,18 @@ Telegram: | | Pull mode (the default) | HTTP mode | |---|---|---| | How messages and button presses reach finbot | finbot asks Telegram for them (long polling) | Telegram posts them to your HTTPS server | -| Public HTTPS server | Not needed | Needed: a domain name with a valid certificate, on port 443, 80, 88 or 8443 | -| `finbot.env` keys | `telegramBotToken`, `telegramChatID` | Those two, plus `telegramOutgoingWebhook` and `telegramOutgoingToken` | +| Listens on the internet | No: finbot only makes outgoing requests | Yes, behind your HTTPS server: a domain name with a valid certificate, on port 443, 80, 88 or 8443 | +| `finbot.env` keys | `telegramBotToken` | `telegramBotToken`, `telegramOutgoingWebhook` and `telegramOutgoingToken` | -Everything else is the same in both: the scheduled reports with their charts, the chat commands (`/price` or `.price`, -`.history`, `.performance` and the rest, [the list](commands.md#supported-commands)), period buttons under charts and -the buttons of `.jobs run` and `.jobs toggle`, "typing…" while a command works, and a keyboard of the common commands in -a direct chat with the bot. +Everything else is the same in both: the scheduled reports with their charts, the chat commands (`/price`, `/history`, +`/performance` and the rest, [the list](commands.md#supported-commands), or as messages, `.price`), period buttons +under charts and the buttons of `/jobs run` and `/jobs toggle`, "typing…" while a command works, and a keyboard of the +common commands in a direct chat with the bot. As finbot starts, it sets the bot's command menu, which Telegram shows when `/` is typed: each command with its usage -(`/forwardpe` for `.forward pe`, `/stock` for `.SYMBOL`). `/profile` and `/job`, the second names of `/stock` and -`/jobs`, are left out of the menu, and work when typed, as every name of a command does. In a group, `/price@finbot_bot` -is the same as `/price`. +(`/price`, `/forwardpe`, `/stock` and the rest). `/profile` and `/job`, the second names of `/stock` and `/jobs`, are +left out of the menu, and work when typed, as every name of a command does. In a group, `/price@finbot_bot` is the same +as `/price`. `telegramOutgoingWebhook` chooses the mode: set, HTTP mode; empty, pull mode. A restart switches. As finbot starts in pull mode it deletes the webhook, and in HTTP mode it sets it; the updates sent meanwhile are kept by Telegram and @@ -31,31 +31,19 @@ arrive after the switch. 1. Create the bot: message [@BotFather](https://telegram.me/BotFather), send `/newbot` and answer its questions. It replies with the bot's token: put it in `finbot.env` as `telegramBotToken`. -2. Add the bot to the group or channel the reports should go to. In a channel, make it an administrator, so that it can - post. -3. Let the bot read the chat: make it an administrator of the group. A bot that is not an administrator sees only the - commands that start with `/`, and finbot's start with `.`. Alternatively, send BotFather `/setprivacy` and choose - `Disable`. +2. Add the bot to the groups or channels it should work in, or message it directly. In a channel, make it an + administrator, so that it can post. +3. Optionally, for commands as plain messages (`.price`) in a group: let the bot read the chat, by making it an + administrator of the group, or by sending BotFather `/setprivacy` and choosing `Disable`. Without that, the bot sees + only the commands that start with `/`, which is all that the slash commands need. 4. Stop strangers adding your bot to their groups: send BotFather `/setjoingroups`, pick the bot and choose `Disable`. Do this after step 2, as it stops you adding the bot to groups too. -5. Put the chat's id in `finbot.env` as `telegramChatID`. - * A group's id is negative. A channel's or a supergroup's is negative and 13 characters long: prepend `-100` if - yours is shorter. - * To find it, stop finbot first (`sudo systemctl stop finbot`): Telegram hands each update to one reader only. Send - `/start@` in the chat, then open `https://api.telegram.org/bot/getUpdates` - in a browser and look for `"chat":{"id":`. If the bot is in HTTP mode, getUpdates refuses while the webhook is - set: open `https://api.telegram.org/bot/deleteWebhook` first (finbot sets it again as it - starts). - * A group's id changes when Telegram turns it into a supergroup, which some changes to its settings do. Update - `telegramChatID` then: finbot does not follow the change. -6. Start finbot (`sudo systemctl restart finbot`). The log says `telegram: long polling for updates`. The first start - gives this chat, Telegram's home chat, the default jobs ([Scheduling](scheduling.md)), so the scheduled - reports arrive there; it gets the error reports too. Every chat manages its own jobs with `.jobs`. -7. Send `.help` in the chat. In a direct chat with the bot, the reply also brings a keyboard of the common commands. +5. Start finbot (`sudo systemctl restart finbot`). The log says `telegram: long polling for updates`. +6. Send `/help` in the chat. In a direct chat with the bot, the reply also brings a keyboard of the common commands. + Each chat schedules its own reports with `/jobs add` ([Scheduling](scheduling.md)). ``` telegramBotToken=0000000000:AAAAAAAAAAAAAAAAAAAAAAAAAA -telegramChatID=-1001000000000 ``` ## HTTP mode @@ -92,11 +80,10 @@ with a valid certificate. Telegram posts only to ports 443, 80, 88 and 8443. every update, and finbot turns away a request without it. 4. Restart finbot. As it starts, it sets the webhook again, deleting it first, which also wakes up a stale connection. If Telegram refuses, finbot does not start, and the log says why. -5. Send `.help` in the chat. In a direct chat with the bot, the reply also brings a keyboard of the common commands. +5. Send `/help` in the chat. In a direct chat with the bot, the reply also brings a keyboard of the common commands. ``` telegramBotToken=0000000000:AAAAAAAAAAAAAAAAAAAAAAAAAA -telegramChatID=-1001000000000 telegramOutgoingWebhook=https://www.example.com:8443/telegram telegramOutgoingToken=a-long-random-password ``` From 422d527a7149d15ded8fbdb685f7a26dc396e105 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 06:11:03 +0000 Subject: [PATCH 04/41] Schedules of finbot's own (lib/when.py), for jobs without cron A schedule is which days (daily, weekdays, some weekdays, a day of some months, or a date once), which times on them (a time of day, or a repeat within a window of hours), or a span across days such as Mon 9am to Sat 3pm, which cron cannot hold as one line; and the zone it is in. - parse() reads the English a chat types ("weekly at 9am", "hourly at :01 weekdays from 10am to 9pm", "every 31 mar", "on 28 oct 2026"), with a zone last as a name, a place or what people call it (EST, AEST). - describe() says it back by its zone's clock, with the local time beside it where the clocks differ. - next_after() gives the next run: a skipped time runs at the end of the gap, a repeated one runs once. Nothing uses it yet; the scheduler moves onto it next. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/when.py | 587 +++++++++++++++++++++++++++++++++++++++++++++ tests/test_util.py | 186 ++++++++++++++ 2 files changed, 773 insertions(+) create mode 100644 lib/when.py diff --git a/lib/when.py b/lib/when.py new file mode 100644 index 00000000..cad1cd03 --- /dev/null +++ b/lib/when.py @@ -0,0 +1,587 @@ +"""Schedules of finbot's own, in place of cron: which days a job runs, which times on those days, and the time zone +they are in. A schedule is kept as JSON in its job (to_json, from_json); parse() reads what a chat types ("weekly at +9am", "every 20 mins weekdays from 10am to 9pm New York"), describe() says it back in words, and next_after() gives +the next run. Unlike cron, one schedule holds a span across days ("every 20 mins, Mon 9am – Sat 3pm").""" +import dataclasses +import datetime +import functools +import json +import re +import zoneinfo + +UTC = zoneinfo.ZoneInfo('UTC') +MINUTE = datetime.timedelta(minutes=1) +DAY_MINUTES = 1440 +WEEK = 7 * DAY_MINUTES +DAYS = ('mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun') +DAY_NAMES = ('Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday') +MONTHS = ('jan', 'feb', 'mar', 'apr', 'may', 'jun', 'jul', 'aug', 'sep', 'oct', 'nov', 'dec') +QUARTERS = (1, 4, 7, 10) +HALVES = (1, 7) +DEFAULT_AT = 9 * 60 # a schedule's time of day where none is typed: 9am +ZONE_WORDS = { # what people call a zone -> the zone, with its daylight saving: EST is New York's clock all year + 'utc': 'UTC', 'gmt': 'Europe/London', 'bst': 'Europe/London', 'uk': 'Europe/London', + 'et': 'America/New_York', 'est': 'America/New_York', 'edt': 'America/New_York', + 'ct': 'America/Chicago', 'cst': 'America/Chicago', 'cdt': 'America/Chicago', + 'pt': 'America/Los_Angeles', 'pst': 'America/Los_Angeles', 'pdt': 'America/Los_Angeles', + 'cet': 'Europe/Paris', 'cest': 'Europe/Paris', 'hkt': 'Asia/Hong_Kong', 'jst': 'Asia/Tokyo', + 'aet': 'Australia/Sydney', 'aest': 'Australia/Sydney', 'aedt': 'Australia/Sydney'} +AREAS = ('Africa', 'America', 'Antarctica', 'Asia', 'Atlantic', 'Australia', 'Europe', 'Indian', 'Pacific') +EXAMPLES = ('daily at 9am', 'weekdays at 4:30pm', 'every saturday at 9am', 'first day of each month at 9am', + 'quarterly', 'every 20 mins weekdays from 10am to 9pm', 'every 31 mar', 'on 28 oct 2026', + 'weekdays at 4:30pm New York') +TIME = r'(\d{1,2})(?::(\d{2}))?\s*(am|pm)?' +DAY = r'(mon|tue|wed|thu|fri|sat|sun)(?:day|sday|nesday|rsday|urday|r|s)?s?' +MONTH = r'(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)(?:uary|ruary|ch|il|e|y|ust|tember|t|ober|ember)?' +OF_MONTH = r'(?: of (?:each|the|every) month)?' + + +class ScheduleError(ValueError): + """A schedule, or a job, that cannot be: its text is said to the chat that asked for it.""" + + +@dataclasses.dataclass(frozen=True) +class Schedule: + """When a job runs: on which days (days: 'daily', 'weekdays', a tuple of DAYS, {'day': n, 'months': (...)}, monthly + where months is empty, or {'date': 'YYYY-MM-DD'}, once); at which times on them (at, a minute of the day; or every + so many minutes, counted from midnight, from start to end where given, both included); or in place of both a span + from a minute of the week to another (span, with every); and the zone those are in.""" + days: object = 'daily' + at: int = None + every: int = None + start: int = None + end: int = None + span: tuple = None + zone: str = 'UTC' + + @property + def tz(self): + """The schedule's zone, as zoneinfo has it.""" + return zoneinfo.ZoneInfo(self.zone) + + @property + def key(self): + """The schedule as one string, the same for equal schedules: part of its job's key.""" + return json.dumps(self.to_json(), sort_keys=True, ensure_ascii=False) + + def to_json(self): + """The schedule as its job keeps it: plain JSON, its times as 'HH:MM'.""" + days, data = self.days, {} + if self.span: + data['span'] = [f'{DAYS[m // DAY_MINUTES]} {clock24(m % DAY_MINUTES)}' for m in self.span] + elif isinstance(days, tuple): + data['days'] = list(days) + elif isinstance(days, dict) and 'day' in days: + data['days'] = {'day': days['day'], **({'months': list(days['months'])} if days['months'] else {})} + else: + data['days'] = dict(days) if isinstance(days, dict) else days + for name, value in (('at', self.at), ('from', self.start), ('to', self.end)): + if value is not None: + data[name] = clock24(value) + if self.every is not None: + data['every'] = self.every + data['zone'] = self.zone + return data + + @classmethod + def from_json(cls, data): + """The schedule a job keeps (to_json). Raises ScheduleError for anything that is not one.""" + try: + days = data.get('days', 'daily') + if isinstance(days, list): + days = tuple(sorted(set(days), key=DAYS.index)) + elif isinstance(days, dict): + days = {'date': days['date']} if 'date' in days else {'day': days['day'], + 'months': tuple(sorted(set(days.get('months', ()))))} + span = tuple(week_minute(part) for part in data['span']) if 'span' in data else None + times = {name: None if data.get(name) is None else minutes(data[name]) for name in ('at', 'from', 'to')} + schedule = cls(days, times['at'], data.get('every'), times['from'], times['to'], span, + data.get('zone', 'UTC')) + except (AttributeError, KeyError, TypeError, ValueError) as e: + raise ScheduleError(f'not a schedule: {data!r} ({e})') from None + schedule.check() + return schedule + + def check(self): + """Raise ScheduleError unless the schedule can be: a zone zoneinfo knows, days and times that are some.""" + try: + problem = self.problem() + except (AttributeError, KeyError, TypeError, ValueError) as e: + problem = f'not a schedule ({e})' + if problem: + raise ScheduleError(problem) + + def problem(self): + """What makes the schedule one that cannot be, or None.""" + try: + zoneinfo.ZoneInfo(self.zone) + except (TypeError, ValueError, KeyError): + return f'no time zone {self.zone!r}' + if (self.at is None) == (self.every is None): + return 'a schedule runs at a time of day, or every so many minutes' + if any(value is not None and not (whole(value) and 0 <= value < DAY_MINUTES) + for value in (self.at, self.start, self.end)): + return 'no such time of day' + if self.every is not None and not (whole(self.every) and 1 <= self.every <= DAY_MINUTES): + return 'every 1 minute to every 24 hours' + if self.span is not None: + if self.every is None or self.days != 'daily' or self.start is not None or self.end is not None: + return 'a span of days runs every so many minutes, and says its days itself' + if len(self.span) != 2 or self.span[0] == self.span[1] or not all(whole(m) and 0 <= m < WEEK + for m in self.span): + return 'no such span of days' + return None + days = self.days + if days in ('daily', 'weekdays'): + return None + if isinstance(days, tuple): + return None if days and len(set(days)) == len(days) and set(days) <= set(DAYS) else f'no days {days!r}' + if isinstance(days, dict) and set(days) == {'date'}: + datetime.date.fromisoformat(days['date']) + return None + if (isinstance(days, dict) and set(days) == {'day', 'months'} and whole(days['day']) and 1 <= days['day'] <= 31 + and isinstance(days['months'], tuple) and all(whole(m) and 1 <= m <= 12 for m in days['months'])): + return None + return f'no days {days!r}' + + def day_ok(self, day): + """Whether the schedule runs on this date at all (a span on every date, at the minutes times() gives).""" + days = self.days + if self.span or days == 'daily': + return True + if days == 'weekdays': + return day.weekday() < 5 + if isinstance(days, tuple): + return DAYS[day.weekday()] in days + if 'date' in days: + return day.isoformat() == days['date'] + return day.day == days['day'] and (not days['months'] or day.month in days['months']) + + def times(self, day): + """The minutes of this date at which the schedule runs, earliest first.""" + if not self.day_ok(day): + return [] + if self.at is not None: + return [self.at] + if self.span: + start, end = self.span + base = day.weekday() * DAY_MINUTES + return [m for m in range(0, DAY_MINUTES, self.every) if within(base + m, start, end)] + start = self.start or 0 + end = DAY_MINUTES - 1 if self.end is None else self.end + return [m for m in range(start % self.every, DAY_MINUTES, self.every) if within(m, start, end)] + + def dates(self, first): + """The dates from first on that the schedule may run on, in order, as far ahead as its next run can be: a week + for days of the week, nine years for a day of some months (29 February), the date itself for a one-off.""" + days = self.days + if isinstance(days, dict) and not self.span: + if 'date' in days: + date = datetime.date.fromisoformat(days['date']) + return [date] if date >= first else [] + return month_days(first, days['day'], days['months'] or range(1, 13)) + return (first + datetime.timedelta(days=n) for n in range(8)) + + def next_after(self, moment): + """The schedule's first run after moment (an aware datetime), in UTC (a time its zone repeats compares equal to + no other zone's); None when it has none (a one-off gone by). A time the clocks skip runs as they reach the first + minute after the gap; a time they repeat runs once, the first time.""" + zone, after = self.tz, moment.astimezone(UTC) + local = moment.astimezone(zone) + earliest = local.hour * 60 + local.minute - 120 # today's minutes before that have gone, daylight saving or not + for day in self.dates(local.date()): + for minute in self.times(day): + if day == local.date() and minute < earliest: + continue + run = arrival(day, minute, zone) + if run > after: + return run + return None + + @property + def frequency(self): + """How often its days come round, in a word: daily, weekdays, weekly, monthly, quarterly, bi-annual, annual or + once; None for other days (several weekdays, other months) and for a span.""" + days = self.days + if self.span or isinstance(days, str): + return None if self.span else days + if isinstance(days, tuple): + return 'weekly' if len(days) == 1 else None + if 'date' in days: + return 'once' + return {(): 'monthly', QUARTERS: 'quarterly', HALVES: 'bi-annual'}.get(days['months'], + 'annual' if len(days['months']) == 1 else None) + + def describe(self, local, now=None): + """The schedule in words, by its own zone's clock: plain where that is local (the zone's name), in 24 hours with + "UTC" after it where it is UTC, else with the zone's place after it and, where the local clock differs at the + next run after now, its times in parentheses: "weekdays at 12:45pm New York (3:45am local, Tue–Sat)".""" + if self.zone == local: + return self.words(False) + if self.zone == 'UTC': + return self.words(True) + ' UTC' + return f"{self.words(False)} {place(self.zone)}{self.aside(local, now)}" + + def words(self, utc): + """The schedule in words, its times as its zone's clock has them (in 24 hours where utc).""" + if self.at is not None: + return f"{days_words(self.days)} at {clock(self.at, utc)}" + if self.span: + return f"{repeat_words(self.every, 0, utc)}, {span_words(self.span, utc)}" + head, window = self.repeat(utc) + tail = ' '.join(part for part in ('' if self.days == 'daily' else repeat_days(self.days), window) if part) + return f"{head}, {tail}" if tail else head + + def repeat(self, utc, gap=0): + """A repeat's words, (how often, its window of hours or ''): ('hourly at :01', '10am–9pm'), the window moved by + gap minutes.""" + start, end = self.start or 0, self.end + if end is None and start < self.every: # where in each step it runs, not a window + return repeat_words(self.every, start, utc), '' + offset = start % 60 if self.every % 60 == 0 else 0 + low = bound(start - offset + gap, utc) + window = f"from {low}" if end is None else f"{low}–{bound(end - offset + gap, utc)}" + return repeat_words(self.every, offset, utc), window + + def aside(self, local, now): + """' (3:45am local, Tue–Sat)': the times as the local clock reads them at the next run, where it differs from + the zone's; '' where they agree, and for a repeat with no window.""" + run = self.next_after(now or datetime.datetime.now(UTC)) + if run is None: + return '' + here, there = run.astimezone(self.tz), run.astimezone(zoneinfo.ZoneInfo(local)) + gap = int((there.utcoffset() - here.utcoffset()).total_seconds()) // 60 + utc, days = local == 'UTC', '' + if not gap: + return '' + if self.at is not None: + text = clock(there.hour * 60 + there.minute, utc) + days = shifted(self.days, (there.date() - here.date()).days) + elif self.span: + text = span_words(tuple((m + gap) % WEEK for m in self.span), utc) + else: + text = self.repeat(utc, gap)[1] + if not text: + return '' + return f" ({text} {'UTC' if utc else 'local'}{', ' + days if days else ''})" + + +def whole(value): + """Whether value is a whole number (True and False are not).""" + return isinstance(value, int) and not isinstance(value, bool) + + +def within(value, low, high): + """Whether value is from low to high, both included, where high may come round again before low (a night).""" + return low <= value <= high if low <= high else value >= low or value <= high + + +def month_days(first, day, months): + """The dates on that day of those months, from first on, for nine years.""" + for index in range(first.year * 12 + first.month - 1, (first.year + 9) * 12): + year, month = divmod(index, 12) + if month + 1 in months: + try: + date = datetime.date(year, month + 1, day) + except ValueError: # no such day that month (the 31st, 29 February) + continue + if date >= first: + yield date + + +def arrival(day, minute, zone): + """When a date's minute comes in a zone, in UTC: one the clocks skip comes as they reach the first minute after the + gap; one they repeat, the first time.""" + wall = datetime.datetime.combine(day, datetime.time(minute // 60, minute % 60)) + moment = wall.replace(tzinfo=zone).astimezone(UTC) + if moment.astimezone(zone).replace(tzinfo=None) == wall: + return moment + moment = wall.replace(tzinfo=zone, fold=1).astimezone(UTC) # before the gap, by its length + offset = moment.astimezone(zone).utcoffset() + for _ in range(DAY_MINUTES): + moment += MINUTE + if moment.astimezone(zone).utcoffset() != offset: + break + return moment + + +def minutes(text): + """'08:30' -> 510. Raises ValueError for no time of day.""" + hours, _, mins = text.partition(':') + value = int(hours) * 60 + int(mins) + if not (len(mins) == 2 and 0 <= int(mins) < 60 and 0 <= value < DAY_MINUTES): + raise ValueError(f'no time {text!r}') + return value + + +def week_minute(text): + """'mon 09:00' -> the minute of the week, from Monday midnight.""" + day, time = text.split() + return DAYS.index(day) * DAY_MINUTES + minutes(time) + + +def clock24(minute): + """510 -> '08:30'.""" + return f'{minute // 60:02d}:{minute % 60:02d}' + + +def clock(minute, utc): + """A time of day as shown: '8:30am', '12:45pm'; '21:30' in 24 hours, for UTC.""" + if utc: + return clock24(minute) + hour, mins = divmod(minute, 60) + return f"{hour % 12 or 12}:{mins:02d}{'am' if hour < 12 else 'pm'}" + + +def bound(minute, utc): + """The end of a window or span as shown: '10am', '9:30pm' ('10:00' for UTC); any minute, as the clock reads it.""" + minute %= DAY_MINUTES + if utc: + return clock24(minute) + hour, mins = divmod(minute, 60) + return f"{hour % 12 or 12}{f':{mins:02d}' if mins else ''}{'am' if hour < 12 else 'pm'}" + + +def repeat_words(every, offset, utc): + """How often a repeat runs, and where its steps fall (offset, minutes after midnight): 'every 20 mins', 'every 30 + mins from :14', 'hourly at :01', 'every 8 hours at :07', 'every 2 hours from 1am'.""" + hours = every % 60 == 0 + text = ('every minute' if every == 1 else f'every {every} mins' if not hours else 'hourly' if every == 60 + else f'every {every // 60} hours') + if not offset: + return text + if offset < 60: + return f"{text} {'at' if hours else 'from'} :{offset:02d}" + return f"{text} from {bound(offset, utc)}" + + +def span_words(span, utc): + """'Mon 9am – Sat 3pm': a span from a minute of the week to another.""" + return ' – '.join(f"{DAYS[m // DAY_MINUTES].title()} {bound(m % DAY_MINUTES, utc)}" for m in span) + + +def ordinal(n): + """2 -> '2nd'.""" + return f"{n}{'th' if 10 <= n % 100 <= 20 else {1: 'st', 2: 'nd', 3: 'rd'}.get(n % 10, 'th')}" + + +def plural(day): + """'sun' -> 'Sundays'.""" + return DAY_NAMES[DAYS.index(day)] + 's' + + +def date_words(date): + """A date as shown: '28 Oct 2026'.""" + return f"{date.day} {MONTHS[date.month - 1].title()} {date.year}" + + +def day_list(days): + """Some days of the week as shown: ('tue', 'wed', 'thu', 'fri', 'sat') -> 'Tue–Sat', a run of three or more even + across Sunday ('Sun–Thu'); ('mon', 'thu') -> 'Mon, Thu'; ('sat', 'sun') -> 'weekends'.""" + found = sorted({DAYS.index(day) for day in days}) + if len(found) == 7: + return 'daily' + if found == [5, 6]: + return 'weekends' + for first in range(len(found) if len(found) >= 3 else 0): + run = found[first:] + found[:first] + if all((b - a) % 7 == 1 for a, b in zip(run, run[1:])): + return f'{DAYS[run[0]].title()}–{DAYS[run[-1]].title()}' + return ', '.join(DAYS[i].title() for i in found) + + +def days_words(days): + """Which days, in words: 'daily', 'weekdays', 'weekly, Sundays', 'Tue–Sat', 'monthly, the 2nd', 'quarterly, 2 + Jan/Apr/Jul/Oct', 'bi-annual, 2 Jan and 2 Jul', 'annual, 2 Jan', 'once, 28 Oct 2026'.""" + if isinstance(days, str): + return days + if isinstance(days, tuple): + return f'weekly, {plural(days[0])}' if len(days) == 1 else day_list(days) + if 'date' in days: + return 'once, ' + date_words(datetime.date.fromisoformat(days['date'])) + day, months = days['day'], days['months'] + names = [MONTHS[m - 1].title() for m in months] + if not months: + return f'monthly, the {ordinal(day)}' + if months == HALVES: + return f'bi-annual, {day} {names[0]} and {day} {names[1]}' + kind = 'quarterly' if months == QUARTERS else 'annual' if len(months) == 1 else 'yearly' + return f"{kind}, {day} {'/'.join(names)}" + + +def repeat_days(days): + """The days a repeat runs on, in words: 'weekdays', 'Saturdays', 'Tue–Sat', 'on the 2nd', 'on 31 Mar'.""" + if isinstance(days, str): + return days + if isinstance(days, tuple): + return plural(days[0]) if len(days) == 1 else day_list(days) + if 'date' in days: + return 'on ' + date_words(datetime.date.fromisoformat(days['date'])) + names = '/'.join(MONTHS[m - 1].title() for m in days['months']) + return f"on {days['day']} {names}" if names else f"on the {ordinal(days['day'])}" + + +def shifted(days, shift): + """The days as a clock `shift` days ahead has them: 'weekdays' a day on is 'Tue–Sat'; '' where that says no more + (no shift, daily).""" + if not shift or days == 'daily': + return '' + if isinstance(days, dict): + return 'the day after' if shift > 0 else 'the day before' + moved = tuple(DAYS[(DAYS.index(day) + shift) % 7] for day in (DAYS[:5] if days == 'weekdays' else days)) + return plural(moved[0]) if len(moved) == 1 else day_list(moved) + + +def place(zone): + """A zone's place, as shown: 'America/New_York' -> 'New York'.""" + return zone.rsplit('/', 1)[-1].replace('_', ' ') + + +@functools.cache +def zone_names(): + """(every zone's name in lower case -> the name, every place of the world's areas in lower case -> its zone).""" + names = sorted(zoneinfo.available_timezones()) + places = {} + for name in names: + if name.split('/')[0] in AREAS: + places.setdefault(place(name).lower(), name) + return {name.lower(): name for name in names}, places + + +def zone_named(words): + """The zone that words name (a zone's name, a place, or what people call it: EST, AEST; any case), or None.""" + text = ' '.join(words).lower() + if text in ZONE_WORDS: + return ZONE_WORDS[text] + exact, places = zone_names() + return exact.get(text.replace(' ', '_')) if '/' in text else places.get(text) + + +def time_of(hours, mins, half): + """The minute of the day of a typed time: ('9', None, 'am') -> 540, ('21', '30', None) -> 1290. Raises + ScheduleError for no time of day.""" + hour, minute = int(hours), int(mins or 0) + if half: + if not 1 <= hour <= 12: + raise ScheduleError(f'no time {hours}{half}') + hour = hour % 12 + (12 if half == 'pm' else 0) + if not (0 <= hour < 24 and 0 <= minute < 60): + raise ScheduleError(f"no time {hours}:{mins or '00'}") + return hour * 60 + minute + + +def parse(text, zone, default_at=DEFAULT_AT): + """The Schedule that a chat typed ("weekly at 9am", "every 20 mins weekdays from 10am to 9pm New York"), in the + zone it names last, else in zone (the timezone preference's); at default_at where it says no time. Raises + ScheduleError, with examples, for what it cannot read.""" + words = text.lower().replace('–', '-').replace('—', '-').replace(',', ' ').split() + named = None + for size in (3, 2, 1): + if len(words) > size and (found := zone_named(words[-size:])): + named, words = found, words[:-size] + break + s = f" {' '.join(words)} " + + def take(pattern): + """The match of pattern in what is left of the text, which it then leaves out; None where there is none.""" + nonlocal s + found = re.search(pattern, s) + if found: + s = s[:found.start()] + ' ' + s[found.end():] + return found + + try: + days = read_date(take) + span = window = offset = at = None + if m := take(rf' (?:from|between) {DAY} {TIME} (?:to|and|-) {DAY} {TIME} '): + span = (DAYS.index(m[1]) * DAY_MINUTES + time_of(*m.group(2, 3, 4)), + DAYS.index(m[5]) * DAY_MINUTES + time_of(*m.group(6, 7, 8))) + elif m := take(rf' (?:from|between) {TIME} (?:to|and|-) {TIME} '): + window = (time_of(*m.group(1, 2, 3)), time_of(*m.group(4, 5, 6))) + every = read_every(take) + if m := take(r' at :(\d{2}) '): + offset = int(m[1]) + elif m := take(rf' at {TIME} '): + at = time_of(*m.group(1, 2, 3)) + if days is None: + days = read_days(take) + if s.split(): + raise ScheduleError(f"can't read {' '.join(s.split())!r}") + if days is None and at is None and every is None and span is None: + raise ScheduleError('no days or times') + if every is None: + if span or window or offset is not None: + raise ScheduleError('a window of hours, a span of days or a minute past the hour goes with a repeat' + ' ("every 20 mins")') + at = default_at if at is None else at + elif at is not None: + raise ScheduleError('a schedule runs at a time, or every so many minutes: not both') + if span and days is not None: + raise ScheduleError('a span of days says its days itself') + start, end = window or (None, None) + if offset is not None: + if offset >= 60: + raise ScheduleError(f'no minute :{offset}') + start, end = (start or 0) + offset, None if end is None else end + offset + schedule = Schedule(days or 'daily', at, every, start, end, span, named or zone) + schedule.check() + except ValueError as e: # ScheduleError, or a date that is none (31 February) + raise ScheduleError(f"{e}. For example: {'; '.join(EXAMPLES)}") from None + return schedule + + +def read_date(take): + """The days of a date that the text says (take() finds and leaves out each phrase): a one-off ('on 28 oct 2026') or + a date each year ('every 31 mar'); None for neither.""" + if m := take(rf' on (\d{{1,2}}) {MONTH} (\d{{4}}) '): + return {'date': datetime.date(int(m[3]), MONTHS.index(m[2]) + 1, int(m[1])).isoformat()} + if m := take(r' on (\d{4})-(\d{2})-(\d{2}) '): + return {'date': datetime.date(int(m[1]), int(m[2]), int(m[3])).isoformat()} + if m := take(rf' (?:every|on) (\d{{1,2}}) {MONTH} '): + day, month = int(m[1]), MONTHS.index(m[2]) + 1 + datetime.date(2024, month, day) # a day that month has, in a leap year + take(r' (?:annual|annually|yearly|every year) ') + return {'day': day, 'months': (month,)} + return None + + +def read_every(take): + """The minutes between the runs of a repeat that the text says ('every 20 mins', 'hourly'), or None.""" + if m := take(r' every (\d+) ?(?:m|min|mins|minute|minutes) '): + return int(m[1]) + if m := take(r' every (\d+) ?(?:h|hr|hrs|hour|hours) '): + return int(m[1]) * 60 + if take(r' (?:hourly|every hour) '): + return 60 + return None + + +def read_days(take): + """The days that the text says (take() finds and leaves out each phrase), or None where it says none.""" + if take(r' (?:daily|every day) '): + return 'daily' + if take(r' (?:on )?(?:weekdays|every weekday) '): + return 'weekdays' + if take(r' (?:on )?(?:weekends|every weekend) '): + return ('sat', 'sun') + if m := take(rf' (?:on )?{DAY}(?: ?- ?| to ){DAY} '): + first, last = DAYS.index(m[1]), DAYS.index(m[2]) + return tuple(sorted((DAYS[(first + i) % 7] for i in range((last - first) % 7 + 1)), key=DAYS.index)) + if m := take(rf' weekly(?: on {DAY})? '): + return (m[1] or 'sat',) + if m := take(rf' (?:every |on )?{DAY} '): + return (m[1],) + day = None + if m := take(rf' (?:on )?the (\d{{1,2}})(?:st|nd|rd|th)?{OF_MONTH} ') or take( + rf' (?:on )?(\d{{1,2}})(?:st|nd|rd|th){OF_MONTH} '): + day = int(m[1]) + elif take(r' (?:on )?(?:the )?first (?:day )?of (?:each|the|every) month '): + day = 1 + if day is not None and not 1 <= day <= 31: + raise ScheduleError(f'no day {day} of the month') + for pattern, months in ((r' monthly ', ()), (r' quarterly ', QUARTERS), (r' (?:bi-annual|biannual|bi-annually|' + r'biannually|half-yearly|semi-annual|semi-annually|twice a year) ', HALVES), + (r' (?:annual|annually|yearly|every year) ', (1,))): + if take(pattern): + return {'day': day or 1, 'months': months} + return None if day is None else {'day': day, 'months': ()} diff --git a/tests/test_util.py b/tests/test_util.py index d62d91de..bcef89b4 100644 --- a/tests/test_util.py +++ b/tests/test_util.py @@ -2,6 +2,7 @@ import datetime, json, threading, unittest from pathlib import Path from tests.common import * +from lib import when class Parsing(unittest.TestCase): @@ -317,5 +318,190 @@ def test_holdings_and_watchlist_without_a_watchlist_file(self): self.assertEqual(watchlist.all_followed(), ['AAPL', 'GOOG', 'MSFT', 'NVDA']) +class Schedules(unittest.TestCase): + """lib/when.py: schedules read from English, said back in words, and their next runs, daylight saving included.""" + MEL, SYD, NY = 'Australia/Melbourne', 'Australia/Sydney', 'America/New_York' + OCTOBER = datetime.datetime(2026, 10, 15, tzinfo=when.UTC) # Melbourne and Sydney on summer time, New York too + TRADES = {'span': ['mon 09:00', 'sat 15:00'], 'every': 20, 'zone': SYD} + + def at(self, zone, *args): + """A moment by a zone's clock: at(MEL, 2026, 10, 4, 3, 0).""" + return datetime.datetime(*args, tzinfo=zoneinfo.ZoneInfo(zone)) + + def test_english_becomes_a_schedule(self): + """Each form a chat may type becomes the schedule it says, in the zone it names, else the timezone + preference's.""" + for text, stored in (('daily', {'days': 'daily', 'at': '09:00'}), ('at 9am', {'days': 'daily', 'at': '09:00'}), + ('weekdays at 4:30pm', {'days': 'weekdays', 'at': '16:30'}), + ('weekly', {'days': ['sat'], 'at': '09:00'}), + ('every saturday at 9am', {'days': ['sat'], 'at': '09:00'}), + ('Saturdays at 21:30', {'days': ['sat'], 'at': '21:30'}), + ('weekly on sunday at 8:30am', {'days': ['sun'], 'at': '08:30'}), + ('tue-sat at 8:30am', {'days': ['tue', 'wed', 'thu', 'fri', 'sat'], 'at': '08:30'}), + ('mon to fri at 9', {'days': ['mon', 'tue', 'wed', 'thu', 'fri'], 'at': '09:00'}), + ('weekends at 10am', {'days': ['sat', 'sun'], 'at': '10:00'}), + ('monthly', {'days': {'day': 1}, 'at': '09:00'}), + ('first day of each month at 9am', {'days': {'day': 1}, 'at': '09:00'}), + ('1st of the month', {'days': {'day': 1}, 'at': '09:00'}), + ('monthly on the 2nd at 8:30am', {'days': {'day': 2}, 'at': '08:30'}), + ('quarterly', {'days': {'day': 1, 'months': [1, 4, 7, 10]}, 'at': '09:00'}), + ('quarterly on the 2nd', {'days': {'day': 2, 'months': [1, 4, 7, 10]}, 'at': '09:00'}), + ('bi-annual', {'days': {'day': 1, 'months': [1, 7]}, 'at': '09:00'}), + ('half-yearly', {'days': {'day': 1, 'months': [1, 7]}, 'at': '09:00'}), + ('annual', {'days': {'day': 1, 'months': [1]}, 'at': '09:00'}), + ('every 31 mar at 8:30am', {'days': {'day': 31, 'months': [3]}, 'at': '08:30'}), + ('on 28 oct 2026', {'days': {'date': '2026-10-28'}, 'at': '09:00'}), + ('every 20 mins', {'days': 'daily', 'every': 20}), + ('every 45 minutes weekdays', {'days': 'weekdays', 'every': 45}), + ('hourly', {'days': 'daily', 'every': 60}), + ('every 2 hours between 9am and 5pm', {'days': 'daily', 'every': 120, 'from': '09:00', 'to': '17:00'}), + ('hourly at :01 weekdays from 10am to 9pm', + {'days': 'weekdays', 'every': 60, 'from': '10:01', 'to': '21:01'}), + ('every 20 mins from mon 9am to sat 3pm', {'span': ['mon 09:00', 'sat 15:00'], 'every': 20}), + ('weekdays at 4:30pm New York', {'days': 'weekdays', 'at': '16:30', 'zone': self.NY}), + ('daily at 9am EST', {'days': 'daily', 'at': '09:00', 'zone': self.NY}), + ('daily at 9am america/new_york', {'days': 'daily', 'at': '09:00', 'zone': self.NY}), + ('daily at 21:30 UTC', {'days': 'daily', 'at': '21:30', 'zone': 'UTC'}), + ('saturdays at 9am hong kong', {'days': ['sat'], 'at': '09:00', 'zone': 'Asia/Hong_Kong'})): + with self.subTest(text=text): + self.assertEqual(when.parse(text, self.MEL).to_json(), {'zone': self.MEL, **stored}) + + def test_what_is_no_schedule_is_refused_with_examples(self): + """Words it cannot read, times and dates that are none, and a repeat with a time of day are refused, with + examples.""" + for text in ('fortnightly', 'at 25:00', 'at 13pm', 'every 31 feb', 'every 20 mins at 9am', + 'weekly from 9am to 5pm', 'daily at 9am Atlantis', '', 'every 0 mins', 'the 32nd of each month', + 'monthly; rm -rf /'): + with self.subTest(text=text): + with self.assertRaises(when.ScheduleError) as caught: + when.parse(text, self.MEL) + self.assertIn('For example: daily at 9am', str(caught.exception)) + + def test_a_schedule_says_itself_by_its_own_zones_clock(self): + """In a Melbourne chat in mid-October: Melbourne's zone in plain local time, UTC in 24 hours, another zone by + its place, with the local time beside it only where the clocks differ, its days moved across midnight.""" + syd, ny = self.SYD, self.NY + for stored, words in ((self.TRADES, 'every 20 mins, Mon 9am – Sat 3pm Sydney'), + ({'days': 'weekdays', 'at': '13:00', 'zone': syd}, 'weekdays at 1:00pm Sydney'), + ({'days': 'weekdays', 'at': '12:45', 'zone': ny}, + 'weekdays at 12:45pm New York (3:45am local, Tue–Sat)'), + ({'days': 'weekdays', 'at': '07:15', 'zone': ny}, 'weekdays at 7:15am New York (10:15pm local)'), + ({'days': 'weekdays', 'at': '16:30', 'zone': 'Asia/Hong_Kong'}, + 'weekdays at 4:30pm Hong Kong (7:30pm local)'), + ({'days': 'weekdays', 'at': '17:00', 'zone': 'Europe/London'}, + 'weekdays at 5:00pm London (3:00am local, Tue–Sat)'), + ({'days': ['tue', 'wed', 'thu', 'fri', 'sat'], 'at': '08:30', 'zone': syd}, 'Tue–Sat at 8:30am Sydney'), + ({'days': 'weekdays', 'every': 60, 'from': '10:01', 'to': '21:01', 'zone': syd}, + 'hourly at :01, weekdays 10am–9pm Sydney'), + ({'days': ['sun'], 'at': '08:30', 'zone': syd}, 'weekly, Sundays at 8:30am Sydney'), + ({'days': {'day': 2}, 'at': '08:30', 'zone': syd}, 'monthly, the 2nd at 8:30am Sydney'), + ({'days': {'day': 2, 'months': [1, 4, 7, 10]}, 'at': '08:30', 'zone': syd}, + 'quarterly, 2 Jan/Apr/Jul/Oct at 8:30am Sydney'), + ({'days': {'day': 2, 'months': [1, 7]}, 'at': '08:30', 'zone': syd}, + 'bi-annual, 2 Jan and 2 Jul at 8:30am Sydney'), + ({'days': {'day': 31, 'months': [3]}, 'at': '08:30', 'zone': syd}, 'annual, 31 Mar at 8:30am Sydney'), + ({'days': {'date': '2026-10-28'}, 'at': '08:30', 'zone': syd}, 'once, 28 Oct 2026 at 8:30am Sydney'), + ({'days': ['sat'], 'at': '09:00', 'zone': self.MEL}, 'weekly, Saturdays at 9:00am'), + ({'days': 'daily', 'at': '21:30', 'zone': 'UTC'}, 'daily at 21:30 UTC'), + ({'every': 30, 'from': '00:14', 'zone': 'UTC'}, 'every 30 mins from :14 UTC'), + ({'every': 480, 'from': '00:07', 'zone': 'UTC'}, 'every 8 hours at :07 UTC'), + ({'days': 'weekdays', 'every': 120, 'from': '09:00', 'to': '17:00', 'zone': self.MEL}, + 'every 2 hours, weekdays 9am–5pm')): + with self.subTest(words=words): + self.assertEqual(when.Schedule.from_json(stored).describe(self.MEL, self.OCTOBER), words) + + def test_the_local_clock_beside_another_zones(self): + """Seen from New York, or by a bot on UTC, a Sydney job shows the local times too: a span moved, the days + before, UTC in 24 hours.""" + syd = self.SYD + for stored, local, words in ((self.TRADES, self.NY, + 'every 20 mins, Mon 9am – Sat 3pm Sydney (Sun 6pm – Sat 12am local)'), + ({'days': 'weekdays', 'at': '13:00', 'zone': syd}, self.NY, + 'weekdays at 1:00pm Sydney (10:00pm local, Sun–Thu)'), + ({'days': {'day': 2, 'months': [1]}, 'at': '08:30', 'zone': syd}, self.NY, + 'annual, 2 Jan at 8:30am Sydney (4:30pm local, the day before)'), + ({'days': 'weekdays', 'every': 60, 'from': '10:01', 'to': '21:01', 'zone': syd}, self.NY, + 'hourly at :01, weekdays 10am–9pm Sydney (7pm–6am local)'), + ({'days': 'weekdays', 'at': '13:00', 'zone': syd}, 'UTC', 'weekdays at 1:00pm Sydney (02:00 UTC)')): + with self.subTest(words=words): + self.assertEqual(when.Schedule.from_json(stored).describe(local, self.OCTOBER), words) + + def test_the_next_run_of_each_kind(self): + """next_after gives the first run after a moment, never at it: a week on, the next month with a 31st, 29 + February years on, a one-off's date and then none, a repeat's next step, within its window or its span.""" + mel, syd, trades = self.MEL, self.SYD, self.TRADES + window = {'days': 'weekdays', 'every': 60, 'from': '10:01', 'to': '21:01', 'zone': syd} + once = {'days': {'date': '2026-10-28'}, 'at': '08:30', 'zone': syd} + for stored, after, run in (({'at': '09:00', 'zone': mel}, (2026, 10, 15, 8), (2026, 10, 15, 9)), + ({'at': '09:00', 'zone': mel}, (2026, 10, 15, 9), (2026, 10, 16, 9)), + ({'days': ['sat'], 'at': '09:00', 'zone': mel}, (2026, 10, 17, 10), (2026, 10, 24, 9)), + ({'days': 'weekdays', 'at': '09:00', 'zone': mel}, (2026, 10, 16, 10), (2026, 10, 19, 9)), + ({'days': {'day': 31}, 'at': '09:00', 'zone': mel}, (2026, 4, 1), (2026, 5, 31, 9)), + ({'days': {'day': 2, 'months': [1, 4, 7, 10]}, 'at': '08:30', 'zone': syd}, (2026, 10, 16), + (2027, 1, 2, 8, 30)), + ({'days': {'day': 29, 'months': [2]}, 'at': '09:00', 'zone': mel}, (2026, 3, 1), (2028, 2, 29, 9)), + (once, (2026, 10, 20), (2026, 10, 28, 8, 30)), (once, (2026, 10, 28, 9), None), + ({'every': 20, 'zone': mel}, (2026, 10, 15, 10, 5), (2026, 10, 15, 10, 20)), + ({'every': 45, 'zone': mel}, (2026, 10, 15, 23, 20), (2026, 10, 16, 0, 0)), + (window, (2026, 10, 16, 21, 0), (2026, 10, 16, 21, 1)), + (window, (2026, 10, 16, 21, 30), (2026, 10, 19, 10, 1)), + (trades, (2026, 10, 17, 14, 50), (2026, 10, 17, 15)), (trades, (2026, 10, 17, 15), (2026, 10, 19, 9)), + (trades, (2026, 10, 14, 23, 50), (2026, 10, 15, 0)), + (trades, (2026, 10, 19, 8, 59), (2026, 10, 19, 9))): + schedule = when.Schedule.from_json(stored) + with self.subTest(stored=stored, after=after): + expected = run and datetime.datetime(*run, tzinfo=schedule.tz) + self.assertEqual(schedule.next_after(datetime.datetime(*after, tzinfo=schedule.tz)), expected) + + def test_a_job_keeps_its_zones_clock_through_daylight_saving(self): + """4:30pm New York is 4:30pm there on both sides of a change; a time the clocks skip runs at the first minute + after the gap, and a time they repeat runs once, the first time, as does each step of a repeat.""" + close = when.Schedule.from_json({'days': 'weekdays', 'at': '16:30', 'zone': self.NY}) + friday = close.next_after(self.at(self.NY, 2026, 3, 6, 12)) # New York's clocks go forward on 8 March + monday = close.next_after(friday) + self.assertEqual([(run.weekday(), run.hour, run.minute) for run in (friday.astimezone(close.tz), + monday.astimezone(close.tz))], [(4, 16, 30), (0, 16, 30)]) + self.assertEqual([run.astimezone(when.UTC).hour for run in (friday, monday)], [21, 20]) + self.assertEqual(when.Schedule.from_json({'at': '02:30', 'zone': self.NY}).next_after( + self.at(self.NY, 2026, 3, 7, 12)), datetime.datetime(2026, 3, 8, 7, 0, tzinfo=when.UTC)) # 3am EDT + night = when.Schedule.from_json({'at': '02:30', 'zone': self.MEL}) + self.assertEqual(night.next_after(self.at(self.MEL, 2026, 10, 3, 12)), self.at(self.MEL, 2026, 10, 4, 3, 0)) + first = night.next_after(self.at(self.MEL, 2026, 4, 4, 12)) # Melbourne repeats 2:00-2:59 on 5 April + self.assertEqual(first, datetime.datetime(2026, 4, 4, 15, 30, tzinfo=when.UTC)) + self.assertEqual(night.next_after(first), self.at(self.MEL, 2026, 4, 6, 2, 30)) + steps, seen = when.Schedule.from_json({'every': 20, 'zone': self.MEL}), [] + moment = self.at(self.MEL, 2026, 4, 5, 1, 50) + for _ in range(5): + moment = steps.next_after(moment) + seen.append(f'{moment.astimezone(when.UTC):%H:%M}') + self.assertEqual(seen, ['15:00', '15:20', '15:40', '17:00', '17:20']) # 2:00-2:40 summer time, then 3am + + def test_a_kept_schedule_reads_back_and_junk_is_refused(self): + """to_json and from_json give the same schedule back; a kept schedule that is not one (an edited file) is a + ScheduleError, never another exception.""" + for text in ('weekdays at 4:30pm New York', 'every 20 mins from mon 9am to sat 3pm', 'quarterly on the 2nd', + 'on 28 oct 2026', 'hourly at :01 weekdays from 10am to 9pm', 'tue-sat at 8:30am'): + with self.subTest(text=text): + schedule = when.parse(text, self.MEL) + self.assertEqual(when.Schedule.from_json(json.loads(json.dumps(schedule.to_json()))), schedule) + for junk in ([], 'daily', {'days': 'sometimes', 'at': '09:00'}, {'days': ['mon'], 'at': '25:00'}, + {'days': ['funday'], 'at': '09:00'}, {'at': '09:00', 'zone': '../../etc/passwd'}, + {'at': '09:00', 'zone': 5}, + {'span': ['mon 09:00'], 'every': 20}, {'every': True}, {'at': '09:00', 'every': 20}, + {'days': {'day': 'x'}, 'at': '09:00'}, {'days': {'date': '2026-02-30'}, 'at': '09:00'}, + {'days': [['mon']], 'at': '09:00'}, {'at': '9'}): + with self.subTest(junk=junk), self.assertRaises(when.ScheduleError): + when.Schedule.from_json(junk) + + def test_how_often_in_a_word(self): + """How often a schedule's days come round, in a word; None for several weekdays, and for a span.""" + for text, word in (('daily', 'daily'), ('weekdays', 'weekdays'), ('every sunday', 'weekly'), + ('monthly on the 2nd', 'monthly'), ('quarterly', 'quarterly'), ('bi-annual', 'bi-annual'), + ('annual', 'annual'), ('every 31 mar', 'annual'), ('on 28 oct 2026', 'once'), ('tue-sat', None), + ('every 20 mins from mon 9am to sat 3pm', None)): + with self.subTest(text=text): + self.assertEqual(when.parse(text, self.MEL).frequency, word) + + if __name__ == '__main__': unittest.main() From ca1857271455ae873e3ff850bb5819df920f4eee Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 06:52:27 +0000 Subject: [PATCH 05/41] Jobs without cron: default jobs in every chat, a .jobs menu, message jobs Each chat's jobs now run on schedules of finbot's own (lib/when.py) and whitelisted actions; no cron line is shown, typed or stored, and croniter is gone. - Default jobs (lib/catalog.py): 35 items, every one switched on in each chat finbot knows. That is every chat in the old finbot_schedule.json, every old home chat, and any chat at its first command. Its old cron jobs are not carried over. - Trades runs as one span, Monday 9am to Saturday 3pm Sydney. - Market reports run in their market's zone. - Performance and price cover the days between their runs. - Message jobs replace the reminder report and reminder.json: their texts are default jobs now, AU ones only where country_code is AU. - A job is an action from a fixed table of report modules, with arguments its own parser takes. A stored job naming anything else is logged and left out, never run. - .jobs lists the jobs a page at a time, with buttons: Enable/disable, Delete (jobs added here only), Add (report, how often, day, hour; Back and Cancel), Run now and Next. A stale or forged press acts on nothing. - .jobs add takes English ("price close weekdays at 4:30pm New York", "message every 31 mar "text""); .jobs next and .jobs system are new. - Each list marks a job that has not run yet with *; a one-off past its time is deleted as the list is shown. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- Dockerfile | 2 +- bot.py | 2 +- defaults/preferences.conf | 3 +- defaults/reminder.json | 64 ---- defaults/schedule.cron | 34 -- lib/buttons.py | 42 ++- lib/catalog.py | 159 +++++++++ lib/formats.py | 26 +- lib/jobs.py | 646 ++++++++++++++++++++++-------------- lib/migrate.py | 160 +-------- lib/reports/__init__.py | 10 +- lib/reports/about.py | 283 +++++++++++++--- lib/reports/message.py | 46 +++ lib/reports/reminder.py | 65 ---- lib/when.py | 6 +- lib/worker.py | 155 ++++++--- requirements.txt | 1 - tests/common.py | 8 +- tests/test_discord.py | 40 +-- tests/test_finbot.py | 9 +- tests/test_reports.py | 32 -- tests/test_schedule.py | 323 ++++++++++-------- tests/test_scheduler.py | 680 ++++++++++++++++++++++++-------------- tests/test_sending.py | 5 +- tests/test_util.py | 37 ++- util/record_outputs.py | 24 +- util/sync_defaults.py | 4 +- 27 files changed, 1728 insertions(+), 1138 deletions(-) delete mode 100644 defaults/reminder.json delete mode 100644 defaults/schedule.cron create mode 100644 lib/catalog.py create mode 100644 lib/reports/message.py delete mode 100644 lib/reports/reminder.py diff --git a/Dockerfile b/Dockerfile index 44c7cd2f..14a71cd7 100644 --- a/Dockerfile +++ b/Dockerfile @@ -4,7 +4,7 @@ FROM debian:trixie-slim RUN apt-get update \ - && apt-get install -y --no-install-recommends python3 python3-requests python3-croniter python3-dateutil \ + && apt-get install -y --no-install-recommends python3 python3-requests python3-dateutil \ python3-matplotlib python3-numpy python3-waitress python3-websocket python3-discord python3-nacl \ ca-certificates tzdata \ && rm -rf /var/lib/apt/lists/* diff --git a/bot.py b/bot.py index fe379f43..4c7cf3e9 100755 --- a/bot.py +++ b/bot.py @@ -304,7 +304,7 @@ def http_server(): log.warn('WARN config:', line) if os.getuid() == 0: log.warn("Running as superuser. This is not recommended.") - migrate.run() # once, on an upgrade: what was kept for all chats becomes the old home chats', schedule.cron among it + migrate.run() # once, on an upgrade: what was kept for all chats becomes the old home chats' scheduler = jobs.Scheduler() scheduler.load() render.preload() diff --git a/defaults/preferences.conf b/defaults/preferences.conf index 4440d9e1..73663eab 100644 --- a/defaults/preferences.conf +++ b/defaults/preferences.conf @@ -40,7 +40,8 @@ # Regional: where you are. # exchange_suffix: a ticker Yahoo does not know is tried again on your exchange, by Yahoo's suffix for it (AX: BHP is # tried as BHP.AX; NZ, L, TO and so on for others). Empty: a ticker is taken as written. -# country_code and timezone: the reminder report's country and time zone; .jobs also shows each next run in timezone. +# country_code: your country, for the default jobs' messages that are for one (AU: the tax year's reminders). +# timezone: the clock of the times .jobs takes and shows, and a job's zone where it names none. #exchange_suffix=AX #country_code=AU #timezone=Australia/Melbourne diff --git a/defaults/reminder.json b/defaults/reminder.json deleted file mode 100644 index 1b9d05e0..00000000 --- a/defaults/reminder.json +++ /dev/null @@ -1,64 +0,0 @@ -[ - { - "month_day": "03-31", - "message": "🥳 Happy EOFY 🇬🇧 🇭🇰 🇮🇳 🇰🇷 🇳🇿 🇯🇵 🇿🇦" - }, - { - "month_day": "06-12", - "message": "SPCX IPO anniversary" - }, - { - "month_day": "06-23", - "gate_country": "AU", - "message": [ - "💸 Realise capital gains/losses by EOFY June 30{flag}", - "", - "💰 Finalise any deductables by EOF June 30: {flag}", - "• donations*", - "• Super contributions†", - "• work expenses‡", - "• investment subscriptions§", - "", - "_* Organisation must be [DGR-registered](https://abr.business.gov.au/Tools/DgrListing)_", - "_† Allow transfer time for Super, as contributions can only be deducted for the year they're received. Stake users: see [note about XX PCC in transaction description](https://hellostake.com/au/support/stake-super/employer-and-personal-contributions/33442327233305#h_01JQAPN9A6ZM41NZKP6XZWN1KP)_", - "_‡ See [eligible work expenses](https://www.ato.gov.au/individuals-and-families/income-deductions-offsets-and-records/deductions-you-can-claim)_", - "_§ Must relate directly to specific trades and gains/income (e.g. not general finance news)_" - ] - }, - { - "month_day": "06-30", - "message": "🥳 Happy EOFY 🇦🇺 🇧🇩 🇪🇬 🇪🇹 🇰🇪 🇳🇵 🇵🇰" - }, - { - "month_day": "07-07", - "gate_country": "AU", - "message": "✍️ Submit 'Notice of Intent to Claim' for any post-tax Super contributions. The fund must acknowledge them before you lodge your tax return{flag}" - }, - { - "month_day": "09-26", - "year": "2026", - "gate_country": "AU", - "message": "ubank draw 5/6 https://www.ubank.com.au/campaigns/doubleyourpay → Winners (+ fund ≤ 5000 this month to enter next draw)" - }, - { - "month_day": "10-25", - "gate_country": "AU", - "message": "😓 Self-service individual tax returns are due Oct 31{flag}" - }, - { - "month_day": "10-28", - "year": "2026", - "gate_country": "AU", - "message": "ubank draw 6/6 https://www.ubank.com.au/campaigns/doubleyourpay → Winners" - }, - { - "month_day": "10-31", - "gate_country": "AU", - "message": "😰 Self-service individual tax returns are due today{flag}" - }, - { - "month_day": "11-12", - "year": "2026", - "message": "SPCX 5th mensiversary. End of (statistically) most risky price finding period after an IPO." - } -] diff --git a/defaults/schedule.cron b/defaults/schedule.cron deleted file mode 100644 index 8e340293..00000000 --- a/defaults/schedule.cron +++ /dev/null @@ -1,34 +0,0 @@ -# NO LONGER READ, BUT ONCE: each chat has its own jobs now, changed with .jobs (add, del, toggle, run). As finbot -# starts the first time without them, it gives every home chat (telegramChatID, slackChannelID, discordChannelID) the -# jobs below (or those of the data directory's copy of this file), each in the zone it ran in, and reads this file no -# more. It goes once every install has done that (lib/migrate.py). The system jobs (sharesight_warm, yahoo_warm, -# cleanup) are not copied: they run on their own schedule, in lib/jobs.py. - -# Every 20 minutes on weekdays -*/20 * * * Mon-Fri trades.py - -# Mid-session -00 01 * * Mon-Fri price.py midsession # AU -15 17 * * Mon-Fri price.py midsession # US - -# Daily -30 21 * * * reminder.py - -# Daily on weekdays -29 21 * * Mon-Fri price.py interday -10 11 * * Mon-Fri price.py premarket - -# Weekly -28 21 * * Sat cal.py earnings; cal.py ex-dividend; price.py 7; performance.py 7; rating.py - -# Housekeeping: fetch afresh the Sharesight portfolio list and holdings every 30 minutes (cached 40 minutes) and the -# price histories of the stocks followed every 8 hours (cached 12 hours), so requests never wait on Sharesight or -# Yahoo; delete cache files older than 30 days -14,44 * * * * sharesight_warm.py -07 */8 * * * yahoo_warm.py -51 18 * * Sat cleanup.py 30 - -# Monthly -27 21 1 * * shorts.py; price.py 28; performance.py 28 - - diff --git a/lib/buttons.py b/lib/buttons.py index c11fe28c..653c4c83 100644 --- a/lib/buttons.py +++ b/lib/buttons.py @@ -1,6 +1,7 @@ """Buttons under a message, what their data says when one is pressed (parse), and what a press is told at once (toast): a chart's period buttons (1D | 7D | 1M | 1Y | Max), whose data carries what its builder needs -(charts.BUILDERS), and a job's buttons in .jobs run and .jobs toggle. Each transport draws Button rows its own way.""" +(charts.BUILDERS); the .jobs menu's (menu_button); and the job buttons of older .jobs lists, which still work. Each +transport draws Button rows its own way.""" import hashlib import typing @@ -28,7 +29,16 @@ class JobPress(typing.NamedTuple): action: str = 'run' -JOB_ACTIONS = {'j': 'run', 'e': 'toggle'} # a job button's kind, its data's first letter, and what it does +class MenuPress(typing.NamedTuple): + """A press in the .jobs menu: the screen it asks for, what it says besides (a page, a job's number, the choices of + Add so far), and the checksum of the list it was drawn for (jobs.list_checksum), stale once the list changes.""" + screen: str + args: tuple + checksum: str + + +JOB_ACTIONS = {'j': 'run', 'e': 'toggle'} # an older job button's kind, its data's first letter, and what it does +MENU_SCREENS = ('top', 'on', 'del', 'run', 'add', 'next') # the .jobs menu's screens (lib/reports/about.py) PERIODS = {'d': ('1D', 1), 'w': ('7D', 7), 'm': ('1M', 30), 'y': ('1Y', 365), 'x': ('Max', None)} @@ -75,17 +85,23 @@ def keyboard(kind, args, active=None): ref = _ref(args) return [[Button(PERIODS[key][0], f"c|{kind}|{key}|{ref}", key == active) for key in BUTTONS[kind]]] -def job_button(number, label, checksum, action='run'): - """A job's button, which runs it now (or with action 'toggle', switches it on or off): 'N. label', cut to 30 - characters, its data the job's number and checksum.""" - kind = next(kind for kind, does in JOB_ACTIONS.items() if does == action) - return Button(f"{number}. {label}"[:30], f"{kind}|{number}|{checksum}") +def menu_button(label, screen, args, checksum): + """A button of the .jobs menu: its label, cut to 40 characters, and its data the screen it asks for, what it says + besides (words, numbers) and the list's checksum: 'm|on|0|5|a1b2'.""" + return Button(label[:40], '|'.join(['m', screen, *map(str, args), checksum])) def parse(data): - """What a pressed button's data asks for: a JobPress for a job's button (whatever follows 'j|' or 'e|', which pick - refuses unless it is a job's number and checksum), a ChartPress for a period button of a known kind and period, or - None for data of neither (from an older finbot, or forged).""" + """What a pressed button's data asks for: a MenuPress for a .jobs menu's button of a known screen, its parts plain + letters and digits; a JobPress for an older job button (whatever follows 'j|' or 'e|', which pick refuses unless + it is a job's number and checksum); a ChartPress for a period button of a known kind and period; or None for data + of none of them (from an older finbot, or forged).""" kind, _, rest = data.partition('|') + if kind == 'm': + parts = rest.split('|') + if (len(parts) >= 2 and parts[0] in MENU_SCREENS and len(data.encode()) <= MAX_CALLBACK_BYTES and + all(part.isascii() and part.isalnum() for part in parts[1:])): + return MenuPress(parts[0], tuple(parts[1:-1]), parts[-1]) + return None if kind in JOB_ACTIONS: number, _, checksum = rest.partition('|') return JobPress(number, checksum, JOB_ACTIONS[kind]) @@ -96,9 +112,11 @@ def parse(data): def toast(data): """What a press is told at once, from the button's data alone (the network acknowledges a press before finbot - knows more): 'Loading 1Y…' for a period button, 'Running…' or 'Switching…' for a job's, None for data of - neither.""" + knows more): 'Loading 1Y…' for a period button, 'Updating…' for the .jobs menu's, 'Running…' or 'Switching…' for an + older job button, None for data of none of them.""" pressed = parse(data) + if isinstance(pressed, MenuPress): + return 'Updating…' if isinstance(pressed, JobPress): return 'Running…' if pressed.action == 'run' else 'Switching…' return f"Loading {PERIODS[pressed.period][0]}…" if pressed else None diff --git a/lib/catalog.py b/lib/catalog.py new file mode 100644 index 00000000..e85cd5be --- /dev/null +++ b/lib/catalog.py @@ -0,0 +1,159 @@ +"""The default jobs: every chat has them, each switched on until the chat switches it off (.jobs). An item's name says +what it reports on, never when: its schedule (lib/when.py) says that. For performance and price the period follows how +often the item runs (period_args): quarterly, 90 days. The market reports run in their market's zone, so that they keep +to its session through its daylight saving: pre-market at 7:15am New York, mid-session about halfway through, the close +30 minutes after it. The messages are the reminders that reminder.json held, each at 8:30am Sydney on its date.""" +import dataclasses +import datetime +import zoneinfo + +from lib import formats +from lib import when + +SYDNEY, NEW_YORK, HONG_KONG, LONDON = 'Australia/Sydney', 'America/New_York', 'Asia/Hong_Kong', 'Europe/London' +MORNING = '08:30' # by Sydney's clock: the weekly and longer reports, and the messages +TUE_SAT = ['tue', 'wed', 'thu', 'fri', 'sat'] # the mornings after a US session +FLAGS = {'AU': '🇦🇺'} # what {flag} becomes in a message for one country +PERIODIC = ('performance', 'price') # the reports whose period follows how often they run +PERIOD_DAYS = {'daily': 1, 'weekdays': 1, 'weekly': 7, 'monthly': 30, 'quarterly': 90, 'bi-annual': 180, 'annual': 365} +MONEY_CAFE = ('https://www.omnycontent.com/d/playlist/b034de0e-930d-434b-9822-a7140060b2c0/' + '95d715e5-10a7-4e54-b114-a9b4001be172/c47481a8-e548-4c2b-8592-ad48003dc9c2/podcast.rss') +STOCKTAKE = ('https://www.omnycontent.com/d/playlist/b034de0e-930d-434b-9822-a7140060b2c0/' + '087f773e-34f7-45fb-a4f2-a78b000c6495/b9e412ea-f1ee-40db-a0f8-aaf5005d790d/podcast.rss') + + +@dataclasses.dataclass(frozen=True) +class Item: + """A default job: its key (in a chat's record of the ones it switched off), the name .jobs shows, the action it runs + (a name in jobs.ACTIONS) with its arguments, its schedule, and the country it is for (None: every chat's).""" + key: str + name: str + action: str + args: tuple + schedule: when.Schedule + country: str = None + + +def period_args(action, schedule): + """The arguments of performance or price that make its period follow how often the schedule runs: over the days + between runs (weekly, 7); for price daily, on weekdays or on other days, the day's moves (interday), and for + performance on other days its usual period; [] for any other action.""" + days = PERIOD_DAYS.get(schedule.frequency) + if action == 'price': + return ['interday'] if days in (None, 1) else [str(days)] + if action == 'performance': + return [] if days is None else [str(days)] + return [] + + +def at(days, time=MORNING, zone=SYDNEY): + """A schedule at a time of day on some days: at('weekdays', '16:30', NEW_YORK).""" + return when.Schedule.from_json({'days': days, 'at': time, 'zone': zone}) + + +def periodic(frequency, days): + """The performance and price items that run at a frequency, each over the days between its runs.""" + schedule = at(days) + return tuple(Item(f'{action}-{frequency}', action, action, tuple(period_args(action, schedule)), schedule) + for action in PERIODIC) + + +def message(key, date, lines, country=None, name=None): + """A message item at 8:30am Sydney on its date ('MM-DD' each year, 'YYYY-MM-DD' once), where it is for one country + with that country's flag for {flag}.""" + day = {'date': date} if len(date) == 10 else {'day': int(date[3:]), 'months': [int(date[:2])]} + return Item(key, name or message_name(lines, country), 'message', + tuple(line.replace('{flag}', FLAGS.get(country, '')) for line in lines), at(day), country) + + +def message_name(lines, country=None, size=56): + """What .jobs calls a message job: 'message: ', its first line's words cut to fit, and the country it is for.""" + first = next((formats.plain(line).replace('{flag}', '').strip() for line in lines if line.strip()), '') + if len(first) > size: + first = first[:size].rsplit(' ', 1)[0] + ' …' + return f"message: {first}" + (f' ({country})' if country else '') + + +EOFY_CHECKLIST = ( + "💸 Realise capital gains/losses by EOFY June 30{flag}", + "", + "💰 Finalise any deductables by EOF June 30: {flag}", + "• donations*", + "• Super contributions†", + "• work expenses‡", + "• investment subscriptions§", + "", + "_* Organisation must be [DGR-registered](https://abr.business.gov.au/Tools/DgrListing)_", + "_† Allow transfer time for Super, as contributions can only be deducted for the year they're received. Stake" + " users: see [note about XX PCC in transaction description](https://hellostake.com/au/support/stake-super/" + "employer-and-personal-contributions/33442327233305#h_01JQAPN9A6ZM41NZKP6XZWN1KP)_", + "_‡ See [eligible work expenses](https://www.ato.gov.au/individuals-and-families/" + "income-deductions-offsets-and-records/deductions-you-can-claim)_", + "_§ Must relate directly to specific trades and gains/income (e.g. not general finance news)_") + +ITEMS = ( + Item('trades', 'trades', 'trades', (), + when.Schedule.from_json({'span': ['mon 09:00', 'sat 15:00'], 'every': 20, 'zone': SYDNEY})), + Item('price-midsession-au', 'price midsession AU', 'price', ('midsession',), at('weekdays', '13:00')), + Item('price-midsession-us', 'price midsession US', 'price', ('midsession',), at('weekdays', '12:45', NEW_YORK)), + Item('price-premarket-us', 'price premarket US', 'price', ('premarket',), at('weekdays', '07:15', NEW_YORK)), + Item('price-close-au', 'price close AU', 'price', ('close',), at('weekdays', '16:30')), + Item('price-close-hk', 'price close HK', 'price', ('close',), at('weekdays', '16:30', HONG_KONG)), + Item('price-close-uk', 'price close UK', 'price', ('close',), at('weekdays', '17:00', LONDON)), + Item('price-close-us', 'price close US', 'price', ('close',), at('weekdays', '16:30', NEW_YORK)), + Item('rating', 'rating', 'rating', (), at(TUE_SAT)), + Item('milestone', 'milestone', 'milestone', (), at(TUE_SAT)), + *(Item(f"podcasts-{name.lower().replace(' ', '-')}", f'podcasts {name}', 'podcasts', (feed,), + when.Schedule.from_json({'days': 'weekdays', 'every': 60, 'from': '10:01', 'to': '21:01', 'zone': SYDNEY})) + for name, feed in (('Money Cafe', MONEY_CAFE), ('Stocktake', STOCKTAKE))), + Item('cal-earnings', 'cal earnings', 'cal', ('earnings',), at(['sun'])), + Item('cal-ex-dividend', 'cal ex-dividend', 'cal', ('ex-dividend',), at(['sun'])), + Item('shorts', 'shorts', 'shorts', (), at(['sun'])), + *periodic('weekly', ['sun']), + *periodic('monthly', {'day': 2}), + *periodic('quarterly', {'day': 2, 'months': [1, 4, 7, 10]}), + *periodic('bi-annual', {'day': 2, 'months': [1, 7]}), + *periodic('annual', {'day': 2, 'months': [1]}), + message('message-eofy-world', '03-31', ['🥳 Happy EOFY 🇬🇧 🇭🇰 🇮🇳 🇰🇷 🇳🇿 🇯🇵 🇿🇦']), + message('message-spcx-ipo', '06-12', ['SPCX IPO anniversary']), + message('message-eofy-checklist', '06-23', EOFY_CHECKLIST, 'AU'), + message('message-eofy-au', '06-30', ['🥳 Happy EOFY 🇦🇺 🇧🇩 🇪🇬 🇪🇹 🇰🇪 🇳🇵 🇵🇰']), + message('message-intent-to-claim', '07-07', ["✍️ Submit 'Notice of Intent to Claim' for any post-tax Super" + " contributions. The fund must acknowledge them before you lodge your tax return{flag}"], 'AU'), + message('message-birthday', '08-18', ["It's my {years since 2022-08-18} birthday! 🥳"], + name="message: finbot's birthday"), + message('message-tax-due-soon', '10-25', ['😓 Self-service individual tax returns are due Oct 31{flag}'], 'AU'), + message('message-ubank-draw-6', '2026-10-28', ['ubank draw 6/6 https://www.ubank.com.au/campaigns/doubleyourpay' + ' → Winners'], 'AU'), + message('message-tax-due-today', '10-31', ['😰 Self-service individual tax returns are due today{flag}'], 'AU'), + message('message-spcx-mensiversary', '2026-11-12', ['SPCX 5th mensiversary. End of (statistically) most risky price' + ' finding period after an IPO.']), +) + + +def items(country): + """The default jobs where the country_code preference is country: every item but those for another country.""" + return [item for item in ITEMS if item.country in (None, country)] + + +def same_clock(a, b): + """Whether two zones (by name) keep the same clock, as Melbourne's and Sydney's do: the same offset on the 1st and + 15th of every month for two years.""" + if a == b: + return True + first, second = zoneinfo.ZoneInfo(a), zoneinfo.ZoneInfo(b) + year = datetime.date.today().year + moments = [datetime.datetime(year + month // 12, month % 12 + 1, day, tzinfo=when.UTC) + for month in range(24) for day in (1, 15)] + return all(moment.astimezone(first).utcoffset() == moment.astimezone(second).utcoffset() for moment in moments) + + +def matching(action, args, schedule, country): + """The default job that a job is, or None: the same action and arguments, on the same days at the same times, in a + zone that keeps the same clock as the item's (a chat in Melbourne typing a Sydney item's schedule).""" + for item in items(country): + if ((item.action, item.args) == (action, tuple(args)) and + dataclasses.replace(item.schedule, zone=schedule.zone) == schedule and + same_clock(item.schedule.zone, schedule.zone)): + return item + return None diff --git a/lib/formats.py b/lib/formats.py index 5394e597..09e3f304 100644 --- a/lib/formats.py +++ b/lib/formats.py @@ -1,4 +1,5 @@ -"""Text for people: company names trimmed, numbers in units, dates and spans in words, signed percents, ordinals.""" +"""Text for people: company names trimmed, numbers in units, dates and spans in words, signed percents, ordinals, and a +line of a message job's text in a network's markup.""" import datetime import re import zoneinfo @@ -193,6 +194,29 @@ def ordinal(num): return 'th' if num % 100 in (11, 12, 13) else {1: 'st', 2: 'nd', 3: 'rd'}.get(num % 10, 'th') +LINK = re.compile(r'\[([^\]]+)\]\(([^)\s]+)\)') # [text](url), in a message job's text + + +def markup(line, service): + """A line of a message job's text in the network's markup: [text](url) is a link, a line wrapped in _underscores_ + is in italics, and the rest is text, made safe to show.""" + italic = len(line) > 1 and line.startswith('_') and line.endswith('_') + if italic: + line = line[1:-1] + text, start = '', 0 + for link in LINK.finditer(line): + text += service.escape(line[start:link.start()]) + service.link(link.group(2), service.escape(link.group(1))) + start = link.end() + text += service.escape(line[start:]) + return service.italic(text) if italic else text + + +def plain(line): + """A line of a message job's text as words alone: a link's text without its url, no underscores of italics.""" + line = LINK.sub(lambda link: link.group(1), line) + return line[1:-1] if len(line) > 1 and line.startswith('_') and line.endswith('_') else line + + def zone(): """The timezone preference as a zone: UTC where it names none.""" try: diff --git a/lib/jobs.py b/lib/jobs.py index 548a7d20..24107216 100644 --- a/lib/jobs.py +++ b/lib/jobs.py @@ -1,77 +1,89 @@ -"""The scheduled reports: each chat's jobs, which its people manage with .jobs (lib/worker.py), kept in SCHEDULE_FILE -and each posting to its own chat; and the system jobs (SYSTEM), fixed here, which fill the cache and clear it out. - -A job is a time in crontab's syntax (five fields, or an @macro such as @daily), in UTC unless it names its own zone -first (CRON_TZ=America/New_York), then one or more reports with their arguments, separated by ";": -`*/20 * * * Mon-Fri trades`, `28 21 * * Sat cal earnings; price 7`. The bot runs each command itself, in a thread of -its own, sharing its user, sandbox, environment, caches and connections.""" +"""The scheduled jobs. Each chat's, which its people manage with .jobs (lib/worker.py), each posting to that chat: the +default jobs (lib/catalog.py), which every chat has, switched on until it switches them off, and those it added; both +kept in JOBS_FILE by chat key, whose chats are those finbot knows (see). And the system jobs (SYSTEM), fixed here, which +fill the cache and clear it out. + +A job runs one action of ACTIONS, a fixed table of finbot's own report modules (message among them, which posts a +text), with the arguments that action's parser (its job()) takes: never a command, a script or code. A stored job names +its action and holds its arguments as words, and both are checked again as it loads. Its schedule (lib/when.py) says +which days and times, in its zone. The bot runs each job itself, in a thread of its own, sharing its user, +environment, caches and connections.""" +import dataclasses import datetime +import heapq import hmac +import json import math -import re -import shlex +import os import threading import time import zlib -import zoneinfo -from dataclasses import dataclass - -from croniter import croniter, CroniterBadDateError -from lib import log +from lib import catalog from lib import config -from lib import state from lib import delivery -from lib.reports import cal, cleanup, milestone, performance, podcasts, price, rating, reminder, shorts, trades +from lib import log +from lib import state +from lib import when +from lib.reports import cal, cleanup, message, milestone, performance, podcasts, price, rating, shorts, trades from lib.reports import sharesight_warm, yahoo_warm -SCRIPTS = {'cal': cal, 'cleanup': cleanup, 'milestone': milestone, 'performance': performance, 'podcasts': podcasts, - 'price': price, 'rating': rating, 'reminder': reminder, 'shorts': shorts, 'trades': trades, - 'sharesight_warm': sharesight_warm, 'yahoo_warm': yahoo_warm} +# what a job may run, by name: a report module of finbot's own, whose run() is called with what its job() made of the +# job's arguments; nothing else is ever called, imported or evaluated +ACTIONS = {'cal': cal, 'message': message, 'milestone': milestone, 'performance': performance, 'podcasts': podcasts, + 'price': price, 'rating': rating, 'shorts': shorts, 'trades': trades, + 'cleanup': cleanup, 'sharesight_warm': sharesight_warm, 'yahoo_warm': yahoo_warm} +SYSTEM_ACTIONS = ('cleanup', 'sharesight_warm', 'yahoo_warm') # the system's alone: a chat cannot schedule these # the system jobs, in UTC: the Sharesight portfolios and holdings fetched afresh every 30 minutes (cached 40), the price # histories of the stocks followed every 8 hours (cached 12), so that no request waits on either; and cache files # older than 30 days deleted weekly -SYSTEM = (('14,44 * * * *', 'sharesight_warm'), ('7 */8 * * *', 'yahoo_warm'), ('51 18 * * Sat', 'cleanup 30')) -SYSTEM_SCRIPTS = ('sharesight_warm', 'yahoo_warm', 'cleanup') # a chat cannot schedule these -WARM = 'sharesight_warm; yahoo_warm' # the jobs that fill the cache, run at once after a flush too -SCHEDULE_FILE = 'finbot_schedule.json' # {chat key: [{'when', 'reports', 'zone', 'on'}]}: each chat's jobs, in order -STATE_FILE = 'finbot_jobs.json' # last fire time of every job, so a restart neither repeats a run nor loses one -MAX_JOBS = 30 # a chat's jobs, at most -JOB_TIMEOUT = 5 * 60 # seconds a command may run before it is reported as late, once (it runs on, see execute) -UTC = zoneinfo.ZoneInfo('UTC') +SYSTEM = (('sharesight_warm', (), {'every': 30, 'from': '00:14'}), ('yahoo_warm', (), {'every': 480, 'from': '00:07'}), + ('cleanup', ('30',), {'days': ['sat'], 'at': '18:51'})) +WARM = ('sharesight_warm', 'yahoo_warm') # the jobs that fill the cache, run at once after a flush too +JOBS_FILE = 'finbot_chat_jobs.json' # {chat key: {'off': [catalog keys], 'jobs': [{action, args, schedule, on}]}} +OLD_FILE = 'finbot_schedule.json' # each chat's cron jobs, from before the catalog: read for its chats alone +STATE_FILE = 'finbot_jobs.json' # each job's last run, by key: a restart neither repeats a run nor loses one +MESSAGE_AT = 8 * 60 + 30 # a message's time of day where none is typed: 8:30am +MAX_JOBS = 30 # the jobs a chat may add, at most +JOB_TIMEOUT = 5 * 60 # seconds a job may run before it is reported as late, once (it runs on, see execute) +MINUTE = datetime.timedelta(minutes=1) +QUOTES = '"“”' # what opens and closes a message's text: phones type curly quotes +UTC = when.UTC +ScheduleError = when.ScheduleError current = None # the running Scheduler, set by bot.py; the chat commands use it -class ScheduleError(ValueError): - """A job that cannot be: said to the chat that asked for it.""" - - -class ImpossibleTime(ScheduleError): - """A time that cannot be (minute 61) or never comes (30 February).""" - - -@dataclass +@dataclasses.dataclass class Job: - """A job: its time, in its zone; the commands it runs in turn, [(label, report module, run's keyword arguments)]; - the chat it posts to (its key, as delivery.chat takes it; '' for a system job); whether it is on; and its reports - as written.""" - when: str - commands: list - zone: zoneinfo.ZoneInfo - chat: str - on: bool - text: str + """A job: the action it runs (a name in ACTIONS), the arguments its parser took and what it made of them (a label, + run's keyword arguments); its schedule; the chat it posts to (its key, as delivery.chat takes it; '' for a system + job); whether it is on; its catalog item (its key; None for one its chat added); and the name .jobs shows.""" + action: str + args: tuple + label: str + kwargs: dict + schedule: when.Schedule + chat: str = '' + on: bool = True + item: str = None + name: str = '' @property def key(self): - """What the job is, for its record of runs and the checksum of its buttons: its chat, time, zone and reports.""" - return f'{self.chat}|{self.when}|{self.zone.key}|{self.text}' + """What the job is, for its record of runs and the checksum of its buttons: its chat and catalog item, or its + chat, action, arguments and schedule.""" + if self.item: + return f'{self.chat}|{self.item}' + return f'{self.chat}|{self.action}|{json.dumps(list(self.args), ensure_ascii=False)}|{self.schedule.key}' @property - def line(self): - """The job as .jobs add takes it: its zone if not UTC, its time, its reports.""" - zone = '' if self.zone.key == 'UTC' else f'CRON_TZ={self.zone.key} ' - return f'{zone}{self.when} {self.text}' + def module(self): + """The report module that runs it.""" + return ACTIONS[self.action] + + def stored(self): + """The job as JOBS_FILE keeps one a chat added.""" + return {'action': self.action, 'args': list(self.args), 'schedule': self.schedule.to_json(), 'on': self.on} def checksum(text): @@ -79,8 +91,14 @@ def checksum(text): return format(zlib.crc32(text.encode()) & 0xffff, '04x') +def list_checksum(jobs): + """A chat's list's short stand-in in the .jobs menu's buttons: it tells any change of the list, so that a button + drawn for another list acts on nothing.""" + return checksum('\n'.join(job.key for job in jobs)) + + def run_in_process(label, module, kwargs, dest=None): - """Run a command here, in the calling thread: its report's run(**kwargs), posting to dest (a chat's job; None for a + """Run a job here, in the calling thread: its report's run(**kwargs), posting to dest (a chat's job; None for a system job, which posts nothing).""" if dest is None: module.run(**kwargs) @@ -93,91 +111,136 @@ def start_thread(target, *args): threading.Thread(target=target, args=args, daemon=True).start() +def system_jobs(): + """The system jobs (SYSTEM), as Jobs.""" + return [make_job(action, args, when.Schedule.from_json({**times, 'zone': 'UTC'})) for action, args, times in SYSTEM] + + def warm_now(scheduler=None): - """Run the warm jobs (WARM) now, one after the other: through the scheduler, so that one already running on its - schedule is not started twice; without one (a test, a tool), here, a failure reported in the chat as the scheduler - reports one.""" - commands = parse_reports(WARM, system=True) - log.info('cache: filling it again:', ', '.join(label for label, _, _ in commands)) - if scheduler: - scheduler.execute(commands) - else: - for label, module, kwargs in commands: - try: - run_in_process(label, module, kwargs) - except Exception as e: - delivery.report_error(e, context=label) + """Run the warm jobs (WARM) now, one after the other: through the scheduler, so that one running on its schedule + is not started twice; without one (a test, a tool), here, a failure reported as the scheduler reports it.""" + warm = [job for job in system_jobs() if job.action in WARM] + log.info('cache: filling it again:', ', '.join(job.label for job in warm)) + for job in warm: + if scheduler: + scheduler.execute(job) + continue + try: + run_in_process(job.label, job.module, job.kwargs) + except Exception as e: + delivery.report_error(e, context=job.label) log.info('cache: warm jobs done') -def parse_reports(text, system=False): - """'cal earnings; price 7' -> [(label, report module, run's keyword arguments)]: report names (lib/reports) and - their arguments, quoted where they hold spaces, separated by ";". The system jobs' reports only with system. Raises - ScheduleError, saying what is wrong.""" - lexer = shlex.shlex(text, posix=True, punctuation_chars=';') - lexer.whitespace_split, lexer.commenters = True, '' +def chat_actions(): + """The actions a chat may schedule, by name.""" + return sorted(name for name in ACTIONS if name not in SYSTEM_ACTIONS) + + +def make_job(action, args, schedule, chat='', on=True, item=None, name=None): + """A Job of an action in ACTIONS with the arguments its parser takes (words), on a when.Schedule: the system's + actions only for a system job (no chat). Raises ScheduleError, saying what is wrong.""" + if not isinstance(action, str) or action not in ACTIONS or (chat and action in SYSTEM_ACTIONS): + raise ScheduleError(f"no report {action!r}: send .jobs add for the reports") + if not isinstance(args, (list, tuple)) or not all(isinstance(arg, str) for arg in args): + raise ScheduleError(f'{action} takes words, not {args!r}') try: - tokens = list(lexer) - except ValueError as e: # an unclosed quote - raise ScheduleError(str(e)) - commands, words = [], [] - for token in tokens + [';']: - if token.strip(';'): - words.append(token) - continue - if not words: - continue - name, arguments, words = words[0], words[1:], [] - if name not in SCRIPTS or (name in SYSTEM_SCRIPTS and not system): - raise ScheduleError(f"no report {name!r}: send .jobs add for the reports and what each takes") - try: - label, kwargs = SCRIPTS[name].job(arguments) - except ValueError as e: - raise ScheduleError(f'{name} takes {e}') - commands.append((label, SCRIPTS[name], kwargs)) - if not commands: - raise ScheduleError('no report: send .jobs add for the reports') - return commands - - -def check_time(when): - """Raise ImpossibleTime for a time that cannot be (minute 61) or never comes (30 February).""" - if when == '@reboot': - raise ImpossibleTime('@reboot is no time: the jobs start with the bot') - if not croniter.is_valid(when): - raise ImpossibleTime(f"'{when}' is no time: five fields (minute hour day month weekday) or an @macro") + label, kwargs = ACTIONS[action].job(list(args)) + except ValueError as e: + raise ScheduleError(f'{action} takes {e}') + if not isinstance(schedule, when.Schedule): + raise ScheduleError(f'no schedule: {schedule!r}') + return Job(action, tuple(args), label, kwargs, schedule, chat, on, item, + name or display_name(action, args, schedule, label)) + + +def display_name(action, args, schedule, label): + """What .jobs calls a job a chat added: performance or price alone where the period follows how often it runs + (catalog.period_args), 'message: ' and its first line, else its parser's label ('cal earnings', 'price 30').""" + if action == 'message': + return catalog.message_name(args) + if action in catalog.PERIODIC and list(args) == catalog.period_args(action, schedule): + return action + return label + + +def stored_job(chat, entry): + """A job a chat added, as JOBS_FILE keeps it (Job.stored). Raises ScheduleError for one that cannot be: an action + that is none, arguments its parser refuses, a schedule that is none (an edited file, or an older one).""" + if not isinstance(entry, dict): + raise ScheduleError(f'no job: {entry!r}') + on = entry.get('on', True) + if not isinstance(on, bool): + raise ScheduleError(f'no switch: {on!r}') + schedule = when.Schedule.from_json(entry.get('schedule')) + return make_job(entry.get('action'), entry.get('args', []), schedule, chat, on) + + +def entry_key(chat, entry): + """The key of a job a chat added, as JOBS_FILE keeps it; None for one that cannot be.""" try: - croniter(when, datetime.datetime(2000, 1, 1)).get_next(datetime.datetime) - except CroniterBadDateError: - raise ImpossibleTime(f"'{when}' never comes: no date matches it") + return stored_job(chat, entry).key + except ScheduleError: + return None -def parse_job(line, chat='', on=True, system=False): - """A Job from its line, as .jobs add takes it: [CRON_TZ=Zone] five time fields (or an @macro), then its reports. - Raises ScheduleError (ImpossibleTime for the time), saying what is wrong.""" - line = line.strip() - zone = UTC - if m := re.match(r'CRON_TZ=(\S*)\s*', line): - try: - zone = zoneinfo.ZoneInfo(m.group(1)) - except (ValueError, zoneinfo.ZoneInfoNotFoundError): - raise ScheduleError(f'no time zone {m.group(1)!r} (such as America/New_York)') - line = line[m.end():] - if line.startswith('@'): - when, _, reports = line.partition(' ') +def switch_item(record, key, on): + """Switch a default job on or off in a chat's record: whether it is on now.""" + off = [each for each in record.get('off', []) if each != key] + ([] if on else [key]) + if off: + record['off'] = off else: - fields = line.split(None, 5) - if len(fields) < 6: - raise ScheduleError('a job is five time fields (minute hour day month weekday), then its reports') - when, reports = ' '.join(fields[:5]), fields[5] - check_time(when) - return Job(when, parse_reports(reports, system), zone, chat, on, reports.strip()) + record.pop('off', None) + return on + + +def know(keys): + """Make chats known (by key), each new one with the default jobs switched on and none added: how many were new.""" + with state.edit(JOBS_FILE, {}) as chats: + new = [key for key in dict.fromkeys(keys) if key not in chats] + for key in new: + chats[key] = {} + return len(new) + + +def parse_add(text, zone): + """What .jobs add was given (its text after add) as (action, arguments, when.Schedule): a report, the arguments it + takes and a schedule in English, in zone unless it names one (the arguments end wherever both read), the period of + performance and price following how often it runs where none is given; or message, a schedule, then the text in + double quotes (curly ones too), from the first quote to the last, so that quotes inside it stay, over several lines + if need be, at 8:30am where it says no time. Raises ScheduleError.""" + name, rest = (text.strip().split(None, 1) + [''])[:2] + name = name.lower() + if name == 'message': + start = min((rest.find(quote) for quote in QUOTES if quote in rest), default=-1) + end = max(rest.rfind(quote) for quote in QUOTES) + if start < 0 or end <= start: + raise ScheduleError('a message is a schedule, then its text in double quotes: .jobs add message every 31' + ' mar "🥳 Happy EOFY"') + return 'message', rest[start + 1:end].replace('\r', '').split('\n'), when.parse(rest[:start], zone, MESSAGE_AT) + if name not in ACTIONS or name in SYSTEM_ACTIONS: + raise ScheduleError(f"no report {name!r}: send .jobs add for the reports") + words, wrong, unread = rest.split(), None, None + for cut in range(len(words) + 1): + try: + schedule = when.parse(' '.join(words[cut:]), zone) + except ScheduleError as e: + unread = unread or e + continue + args = words[:cut] or catalog.period_args(name, schedule) + try: + ACTIONS[name].job(args) + except ValueError as e: + wrong = ScheduleError(f'{name} takes {e}') + continue + return name, args, schedule + raise wrong or unread class Scheduler: - """Fires the jobs: each chat's, from SCHEDULE_FILE, and the system jobs. clock, sleep, spawn and runner can be - replaced, which the tests do. Its lock covers what the loop, a change from chat and the runs share: the jobs, their - bases, the record of runs and what is running.""" + """Runs the jobs: each chat's, from JOBS_FILE and the catalog, and the system jobs. clock, sleep, spawn and runner + can be replaced, which the tests do. Its lock covers what the loop, a change from chat and the runs share: the + jobs, their bases and next runs, the record of runs and what is running.""" def __init__(self, catchup=None, clock=None, sleep=None, spawn=None, runner=None): """catchup is how late, in minutes, a missed run may still start (the scheduler_catchup setting when None); @@ -186,41 +249,82 @@ def __init__(self, catchup=None, clock=None, sleep=None, spawn=None, runner=None self.clock = clock or (lambda: datetime.datetime.now(datetime.timezone.utc)) self.sleep = sleep or time.sleep self.spawn = spawn or start_thread - self.runner = runner or run_in_process # (label, module, kwargs, dest): runs the command + self.runner = runner or run_in_process # (label, module, kwargs, dest): runs the job self.jobs = [] + self.chats = set() # the chats JOBS_FILE knows self.base = {} # key -> the time after which the next run is due + self.next_at = {} # key -> the next run after its base, once worked out (None: none, or off) self.last = {} - self.busy = set() # (chat, label) of the commands running now, scheduled or manual + self.busy = set() # the keys of the jobs running now, scheduled or by hand + self.adopted = False # whether the old OLD_FILE's chats were looked at, at the first load self.lock = threading.Lock() def load(self): - """Read every chat's jobs from SCHEDULE_FILE, besides the system jobs: at start, and after every change from a - chat. A job the file holds that cannot be any more (a report since retired) is reported and left out. A job - kept from before keeps its timing; a new one counts from now, as one does that was off.""" - jobs = [parse_job(f'{when} {reports}', system=True) for when, reports in SYSTEM] - for chat, entries in state.load(SCHEDULE_FILE, {}).items(): - for entry in entries: + """Read every job: the system jobs; for each chat JOBS_FILE knows, the catalog's (switched off where its record + says so) and those it added. A stored job that cannot be (an action that is none, arguments its parser refuses: + an edited or older file) is logged and left out, never run. A job kept from before keeps its timing; a new one + counts from now, as one does that was off.""" + jobs = system_jobs() + chats = self.stored_chats() + items = catalog.items(config.prefs.country_code) + for chat, record in chats.items(): + off = {key for key in record.get('off', []) if isinstance(key, str)} if isinstance(record.get('off'), + list) else set() + jobs += [make_job(item.action, item.args, item.schedule, chat, item.key not in off, item.key, item.name) + for item in items] + added = record.get('jobs', []) + keys = set() + for entry in added if isinstance(added, list) else []: try: - zone = '' if entry.get('zone', 'UTC') == 'UTC' else f"CRON_TZ={entry['zone']} " - jobs.append(parse_job(f"{zone}{entry['when']} {entry['reports']}", chat, entry.get('on', True))) - except (ScheduleError, KeyError, TypeError) as e: - delivery.report_error(ScheduleError(f'a job of {chat} left out: {e}: {entry}'), context='schedule') + job = stored_job(chat, entry) + except ScheduleError as e: + log.warn(f'schedule: a job of {chat} left out: {e}') + continue + if job.key not in keys: + keys.add(job.key) + jobs.append(job) with self.lock: # no run is recorded between reading the record and swapping it in try: last, problem = state.load(STATE_FILE), None except (OSError, ValueError) as e: # unreadable or not JSON: start without one; the next run rewrites it last, problem = None, f'cannot read {STATE_FILE}, starting without a record of past runs: {e}' - self.last = last if isinstance(last, dict) else {} + keys = {job.key for job in jobs} + self.last = {key: value for key, value in last.items() if key in keys} if isinstance(last, dict) else {} + if isinstance(last, dict) and self.last != last: # a job gone: one added again later has not run + self.save_runs() now = self.clock() old = {job.key: self.base[job.key] for job in self.jobs if job.key in self.base} self.base = {job.key: old.get(job.key) or self.start_base(job, now) for job in jobs} - self.jobs = jobs + self.next_at = {} + self.jobs, self.chats = jobs, set(chats) if problem: delivery.report_error(ScheduleError(problem), context='schedule') - chats = len({job.chat for job in jobs if job.chat}) - log.info(f"schedule: {len(jobs) - len(SYSTEM)} jobs of {chats} chat{'' if chats == 1 else 's'}, and" + log.info(f"schedule: {len(jobs) - len(SYSTEM)} jobs of {len(chats)} chat{'' if len(chats) == 1 else 's'}, and" f" {len(SYSTEM)} system jobs") + def stored_chats(self): + """The chats JOBS_FILE knows, each with its record. At the first load, those the old OLD_FILE knew join them + first, with the default jobs; the file is read for their keys alone, their cron jobs not carried over.""" + if not self.adopted: + self.adopted = True + if os.path.exists(state.path(OLD_FILE)): + try: + old = state.load(OLD_FILE, {}) + know(key for key in old if isinstance(key, str) and ':' in key) + except (OSError, ValueError, TypeError) as e: + log.warn(f'schedule: {OLD_FILE} left as it is, unreadable: {e}') + log.info(f"schedule: {OLD_FILE} is read no more: each chat's jobs are in {JOBS_FILE}, its chats with" + " the default jobs; it can be deleted") + try: + chats = state.load(JOBS_FILE, {}) + except (OSError, ValueError) as e: + log.error(f'schedule: cannot read {JOBS_FILE}, so no chat has jobs: {e}') + return {} + if not isinstance(chats, dict): + log.error(f'schedule: {JOBS_FILE} is no list of chats, so no chat has jobs') + return {} + return {chat: record for chat, record in chats.items() if isinstance(record, dict)} + def start_base(self, job, now): """Where to count from at start: a run missed in the catch-up window fires once; with no record, none is.""" last = self.last.get(job.key) @@ -228,104 +332,111 @@ def start_base(self, job, now): return now try: return max(datetime.datetime.fromisoformat(last), now - self.catchup) - except ValueError: + except (TypeError, ValueError): return now - def next_fire(self, job, after): - """The first fire time after `after`, in the job's zone. croniter is given the naive local time and the zone is - put back after: croniter up to 6.0 (the distro packages) mishandles zoneinfo across DST changes, firing daily - jobs twice or an hour late. A time the clocks repeat fires once, the first time; one they skip (02:30 as they go - forward) fires an hour later by the clock.""" - times = croniter(job.when, after.astimezone(job.zone).replace(tzinfo=None)) - while True: - # fold 0: the first of a repeated time - t = times.get_next(datetime.datetime).replace(tzinfo=job.zone).timestamp() - # by the instant: comparing two times of one zone ignores which of a repeated hour they are in - if t > after.timestamp(): - return datetime.datetime.fromtimestamp(t, job.zone) - def due(self, now): - """The jobs to run now, as (job, fire time). Several missed fire times of one job run it once. A job that is off - never fires, and counts from now: switched on, it does not catch up.""" + """The jobs to run now, as (job, run time). Several missed runs of one job run it once. A job that is off never + runs, and counts from now: switched on, it does not catch up. Each job's next run is worked out once, and again + only after it runs, so that a tick costs little.""" fired = [] with self.lock: for job in self.jobs: + key = job.key if not job.on: - self.base[job.key] = now + self.base[key], self.next_at[key] = now, None + continue + upcoming = self.next_at[key] if key in self.next_at else job.schedule.next_after(self.base[key]) + if upcoming is None or upcoming > now: + self.next_at[key] = upcoming continue - base, latest = self.base[job.key], None - while (t := self.next_fire(job, base)) <= now: - latest, base = t, t - if latest: - self.base[job.key] = latest - self.last[job.key] = latest.isoformat() - fired.append((job, latest)) + latest = upcoming + while (upcoming := job.schedule.next_after(latest)) is not None and upcoming <= now: + latest = upcoming + self.base[key], self.next_at[key] = latest, upcoming + self.last[key] = latest.isoformat() + fired.append((job, latest)) if fired: - try: - state.save(STATE_FILE, self.last) - except OSError as e: - log.error('schedule: cannot save', STATE_FILE, e) + self.save_runs() return fired - def claim(self, chat, label): - """Mark a chat's command running, unless it already is (scheduled or manual): whether it was claimed.""" + def save_runs(self): + """Write the record of runs (the lock held).""" + try: + state.save(STATE_FILE, self.last) + except OSError as e: + log.error('schedule: cannot save', STATE_FILE, e) + + def claim(self, key): + """Mark a job running, unless it already is (scheduled or by hand): whether it was claimed.""" with self.lock: - if (chat, label) in self.busy: + if key in self.busy: return False - self.busy.add((chat, label)) + self.busy.add(key) return True - def execute(self, commands, chat=''): - """Run commands one after another (a job's), in this thread, posting to the chat whose job it is (its key; '' - for a system job, whose failure goes to the log); a failure is reported. One already running for that - chat is skipped. One still running after JOB_TIMEOUT is reported as late, once, and runs on: a thread cannot be - stopped, but every request it makes has a timeout, so it ends; until then it stays marked running, and no - second copy starts. Returns 'done', 'failed' or 'already running'.""" - dest = delivery.chat(chat, scheduled=True) if chat else None - outcome = 'done' - for label, module, kwargs in commands: - if not self.claim(chat, label): - log.warn(f'schedule: {label} is already running for {chat or "the system"}, skipping it') - outcome = 'already running' - continue - log.info(f'schedule: running {label}' + (f' for {chat}' if chat else '')) - late = threading.Timer(JOB_TIMEOUT, delivery.report_error, args=(TimeoutError(f'still running after' - f' {JOB_TIMEOUT / 60:g} minutes, and left to finish'), dest), kwargs={'context': label}) - late.daemon = True - late.start() - try: - self.runner(label, module, kwargs, dest) - except (Exception, SystemExit) as e: - delivery.report_error(e, dest, context=label) - outcome = 'failed' - finally: - late.cancel() - with self.lock: - self.busy.discard((chat, label)) - return outcome + def execute(self, job): + """Run a job now, in this thread, posting to its chat (a system job posts nothing, and its failure goes to the + log); a failure is reported. A job already running is skipped. One still running after JOB_TIMEOUT is reported + as late, once, and runs on: a thread cannot be stopped, but every request it makes has a timeout, so it ends; + until then it stays marked running, and no second copy starts. Returns 'done', 'failed' or 'already running'.""" + dest = delivery.chat(job.chat, scheduled=True) if job.chat else None + if not self.claim(job.key): + log.warn(f'schedule: {job.label} is already running for {job.chat or "the system"}, skipping it') + return 'already running' + log.info(f'schedule: running {job.label}' + (f' for {job.chat}' if job.chat else '')) + late = threading.Timer(JOB_TIMEOUT, delivery.report_error, args=(TimeoutError(f'still running after' + f' {JOB_TIMEOUT / 60:g} minutes, and left to finish'), dest), kwargs={'context': job.label}) + late.daemon = True + late.start() + try: + self.runner(job.label, job.module, job.kwargs, dest) + return 'done' + except (Exception, SystemExit) as e: + delivery.report_error(e, dest, context=job.label) + return 'failed' + finally: + late.cancel() + with self.lock: + self.busy.discard(job.key) def running(self, chat): - """The labels of the commands running now for a chat (by its key), and for the system.""" + """The keys of the jobs running now for a chat (by its key), and for the system.""" with self.lock: - return {label for where, label in self.busy if where in (chat, '')} + return {key for key in self.busy if key.partition('|')[0] in (chat, '')} def chat_jobs(self, chat): - """A chat's jobs, in their order: .jobs numbers them from 1.""" + """A chat's jobs, as .jobs numbers them from 1: the default ones, then those it added, but a one-off gone by.""" + now = self.clock() with self.lock: - return [job for job in self.jobs if job.chat == chat] + jobs = [job for job in self.jobs if job.chat == chat] + return [job for job in jobs if not self.expired(job, now)] def system_jobs(self): """The system jobs.""" with self.lock: return [job for job in self.jobs if not job.chat] + def expired(self, job, now): + """Whether a one-off job's time has gone by, longer ago than a missed run may still start: run or not.""" + return job.schedule.frequency == 'once' and job.schedule.next_after(now - self.catchup - MINUTE) is None + + def has_run(self, job): + """Whether a job has ever run, on its schedule or by hand.""" + with self.lock: + return job.key in self.last + def job(self, chat, number): """A chat's job by its number in .jobs (a string, from a chat). Raises ScheduleError.""" jobs = self.chat_jobs(chat) if not (number.isascii() and number.isdigit() and 1 <= int(number) <= len(jobs)): - raise ScheduleError(f'no job {number}: send .jobs for the list' if jobs else 'no jobs here yet: .jobs add') + raise ScheduleError(f'no job {number}: send .jobs for the list') return jobs[int(number) - 1] + def number(self, chat, key): + """A chat's job's number in .jobs, by its key.""" + return next(number for number, job in enumerate(self.chat_jobs(chat), 1) if job.key == key) + def pick(self, chat, number, crc): """The job a button stands for. Button data is untrusted input: the number must be plain digits within this chat's list, and the checksum must match that very job, else ScheduleError.""" @@ -335,68 +446,123 @@ def pick(self, chat, number, crc): return job def run(self, job): - """Run a job now, as its schedule would: its outcome (see execute).""" - return self.execute(job.commands, job.chat) + """Run a job now, as its schedule would, and note the run: its outcome (see execute).""" + with self.lock: + self.last[job.key] = self.clock().isoformat() + self.save_runs() + return self.execute(job) + + def see(self, chat): + """A chat seen, at a command of its: one finbot does not know yet starts with the default jobs, switched on.""" + with self.lock: + known = chat in self.chats + if not known: + know([chat]) + self.load() def change(self, chat, edit): - """Change a chat's jobs, as stored: edit(entries) does it to the list (and may raise ScheduleError); then they - are read again, so that the change counts at once. Returns what edit returned.""" - with state.edit(SCHEDULE_FILE, {}) as chats: - entries = chats.setdefault(chat, []) - result = edit(entries) - if not entries: - del chats[chat] + """Change a chat's jobs as stored, and return what edit(record) returned: it changes the chat's record ({'off': + [catalog keys], 'jobs': [...]}), or raises ScheduleError; then every job is read again, to count at once.""" + with state.edit(JOBS_FILE, {}) as chats: + record = chats.setdefault(chat, {}) + if not isinstance(record.get('jobs', []), list): + record['jobs'] = [] + result = edit(record) self.load() return result - def add(self, chat, line): - """Add a job to a chat's list (a line as parse_job takes it), on: its number. Raises ScheduleError.""" - job = parse_job(line, chat) - def append(entries): - """Append the job, within MAX_JOBS.""" + def add(self, chat, action, args, schedule): + """Add a job to a chat, switched on: an action of ACTIONS, its arguments, a when.Schedule. One the catalog has + (catalog.matching) is that default job, switched on. Returns its number in .jobs. Raises ScheduleError for a job + that cannot be, a one-off gone by, one the chat has already, or one more than MAX_JOBS.""" + job = make_job(action, args, schedule, chat) + if schedule.frequency == 'once' and schedule.next_after(self.clock()) is None: + raise ScheduleError(f"{when.days_words(schedule.days)} has gone by: a one-off is for a time ahead") + if item := catalog.matching(job.action, job.args, schedule, config.prefs.country_code): + self.change(chat, lambda record: switch_item(record, item.key, True)) + return self.number(chat, f'{chat}|{item.key}') + def append(record): + """Append the job, unless the chat has it already, within MAX_JOBS.""" + entries = record.setdefault('jobs', []) + if any(entry_key(chat, entry) == job.key for entry in entries): + raise ScheduleError('that job is here already: send .jobs for the list') if len(entries) >= MAX_JOBS: - raise ScheduleError(f'{MAX_JOBS} jobs is the most a chat may have: delete one first') - entries.append({'when': job.when, 'reports': job.text, 'zone': job.zone.key, 'on': True}) - return len(entries) - return self.change(chat, append) + raise ScheduleError(f'{MAX_JOBS} jobs is the most a chat may add: delete one first') + entries.append(job.stored()) + self.change(chat, append) + return self.number(chat, job.key) def delete(self, job): - """Delete a chat's job (from job() or pick()).""" - self.change(job.chat, lambda entries: entries.pop(self.index(entries, job))) + """Delete a job its chat added (from job() or pick()); a default one raises ScheduleError: switch it off.""" + if job.item: + raise ScheduleError(f'{job.name} is a default job: switch it off instead (.jobs toggle)') + self.change(job.chat, lambda record: record['jobs'].pop(self.index(record, job))) def toggle(self, job): """Switch a chat's job on or off: whether it is on now.""" - def flip(entries): - """Flip the job's switch.""" - entry = entries[self.index(entries, job)] + def flip(record): + """Flip the job's switch, as stored.""" + if job.item: + return switch_item(record, job.item, job.item in record.get('off', [])) + entry = record['jobs'][self.index(record, job)] entry['on'] = not entry.get('on', True) return entry['on'] return self.change(job.chat, flip) @staticmethod - def index(entries, job): - """Where a job is among its chat's stored entries. Raises ScheduleError when it is gone meanwhile.""" - for i, entry in enumerate(entries): - zone = entry.get('zone', 'UTC') - if (entry.get('when'), entry.get('reports'), zone) == (job.when, job.text, job.zone.key): + def index(record, job): + """Where a job its chat added is among those stored. Raises ScheduleError when it is gone meanwhile.""" + for i, entry in enumerate(record['jobs']): + if entry_key(job.chat, entry) == job.key: return i raise ScheduleError('that job is gone: send .jobs for the list') + def prune(self, chat): + """Delete a chat's one-off jobs whose time has gone by (run, or missed while off or down): as its list is + shown, never on a tick.""" + now = self.clock() + with self.lock: + gone = {job.key for job in self.jobs if job.chat == chat and not job.item and self.expired(job, now)} + if gone: + self.change(chat, lambda record: record.update(jobs=[entry for entry in record['jobs'] + if entry_key(chat, entry) not in gone])) + def next_run(self, job): - """When a job fires next, or None when it is off.""" + """When a job runs next, or None when it is off (or a one-off gone by).""" if not job.on: return None with self.lock: base = self.base.get(job.key) or self.clock() - return self.next_fire(job, max(base, self.clock())) + return job.schedule.next_after(max(base, self.clock())) + + def upcoming(self, chat, count=20): + """A chat's next runs, in time order, at most count: (first, last, job, number), first and last the same but for + a repeat, whose runs on a day of its zone are one entry, from its first run that day to its last. Each job's + runs come from a heap, so that it takes about count + jobs schedule lookups.""" + now, heap, found = self.clock(), [], [] + for number, job in enumerate(self.chat_jobs(chat), 1): + if job.on and (run := job.schedule.next_after(now)) is not None: + heap.append((run, number, job)) + heapq.heapify(heap) + while heap and len(found) < count: + run, number, job = heapq.heappop(heap) + last, schedule = run, job.schedule + if schedule.every: + day = run.astimezone(schedule.tz).date() + if minutes := schedule.times(day): + last = max(run, when.arrival(day, minutes[-1], schedule.tz)) + found.append((run, last, job, number)) + if (following := schedule.next_after(last)) is not None: + heapq.heappush(heap, (following, number, job)) + return found def tick(self): - """Start what is due; returns the seconds until the next fire time.""" + """Start what is due; returns the seconds until the next run, from 1 to 30.""" now = self.clock() - for job, fire_time in self.due(now): - self.spawn(self.execute, job.commands, job.chat) + for job, _ in self.due(now): + self.spawn(self.execute, job) with self.lock: - waits = [(self.next_fire(job, self.base[job.key]) - now).total_seconds() for job in self.jobs if job.on] + waits = [(upcoming - now).total_seconds() for upcoming in self.next_at.values() if upcoming] return min(30, max(1, math.ceil(min(waits)))) if waits else 30 def run_forever(self): diff --git a/lib/migrate.py b/lib/migrate.py index 4dbcbb15..e53cfae5 100644 --- a/lib/migrate.py +++ b/lib/migrate.py @@ -1,19 +1,13 @@ """Once, as an upgraded finbot starts: what the bot kept for all chats at once becomes each old home chat's, the chats that the retired telegramChatID, slackChannelID and discordChannelID name (read from the environment for this alone). -The old schedule.cron (the data dir's copy, else the shipped one) is copied to every home chat as its jobs, each in the -zone it ran in, and is read no more; the watchlist and the plans are copied to each home chat; and so are the reports' -records of what they told the chats (trades, ratings, record highs, podcast episodes), so that nothing old is told -again. +Each home chat becomes a chat finbot knows, with the default jobs (lib/catalog.py); the watchlist and the plans are +copied to each; and so are the reports' records of what they told the chats (trades, ratings, record highs, podcast +episodes), so that nothing old is told again. -This module, and defaults/schedule.cron, go once every install has started with it: they are its only users.""" +This module goes once every install has started with it: it is its only user.""" import copy -import datetime import glob import os -import re -import shlex -import zoneinfo -from pathlib import Path from lib import log from lib import config @@ -25,9 +19,6 @@ # kept for all chats, each now one record per chat; and each podcast feed's seen file RECORDS = (watchlist.FILE, worker.PLANS, trades.STATE_FILE, rating.STATE_FILE, milestone.STATE_FILE) -# what a shell reads as more than a word, outside quotes, in a pasted crontab line -OPERATOR = re.compile(r'\d*>>?&?\d*|&>>?|;|&&|\|\|?|&|[<`$(){}]') -REDIRECT = re.compile(r'\d*>>?&?\d*|&>>?') # > file, >> file, 2>&1, &> file # the home chats of older versions: (network, the retired finbot.env key of its chat, the key of its bot token) HOMES = (('telegram', 'telegramChatID', 'telegramBotToken'), ('slack', 'slackChannelID', 'slackBotToken'), ('discord', 'discordChannelID', 'discordBotToken')) @@ -45,41 +36,18 @@ def run(): homes = old_homes() if not homes: return - schedule(homes) + try: + added = jobs.know(homes) + except (OSError, ValueError) as e: + log.warn(f'schedule: {jobs.JOBS_FILE} left as it is, unreadable: {e}') + added = 0 + if added: + log.info(f"schedule: {added} old home chat{'' if added == 1 else 's'} start with the default jobs") feeds = sorted(os.path.basename(path) for path in glob.glob(state.path(podcasts.SEEN_FILES))) for name in RECORDS + tuple(feeds): record(name, homes) -def schedule(homes): - """Give every home chat (keys) the jobs of the old schedule.cron, the system jobs aside (they run in code now), - unless the chats' jobs are kept already (jobs.SCHEDULE_FILE). A line that cannot be a job any more is logged and - left out.""" - if os.path.exists(state.path(jobs.SCHEDULE_FILE)): - return - path = config.data_file('schedule.cron') - try: - text = Path(path).read_text(encoding='utf-8') - except OSError: - text = '' - entries = [] - for number, line, zone, words_list in legacy_lines(text, path): - reports = '; '.join(shlex.join([name, *arguments]) for name, arguments in words_list - if name not in jobs.SYSTEM_SCRIPTS) - if not reports: - continue - try: - job = jobs.parse_job(f"{'' if zone == 'UTC' else f'CRON_TZ={zone} '}{line} {reports}") - except jobs.ScheduleError as e: - log.warn(f'schedule: {path} line {number} left out: {e}') - continue - entries.append({'when': job.when, 'reports': job.text, 'zone': job.zone.key, 'on': True}) - # written even when empty: the file being there says the migration is done - state.save(jobs.SCHEDULE_FILE, {key: copy.deepcopy(entries) for key in homes if entries}) - log.info(f"schedule: the {len(entries)} jobs of {path} are now each home chat's ({', '.join(homes)});" - f" finbot reads that file no more: each chat's jobs are in {jobs.SCHEDULE_FILE}, changed with .jobs") - - def record(name, homes): """A file's record kept for all chats at once (anything but {'chats': ...}) becomes each home chat's.""" try: @@ -91,109 +59,3 @@ def record(name, homes): return state.save(name, {'chats': {key: copy.deepcopy(data) for key in homes}}) log.info(f"state: {name} is kept per chat now, its record copied to {', '.join(homes)}") - - -def local_zone(): - """The server's own time zone, which the old schedule ran in, as cron does: UTC by that name when it is UTC by - another (Etc/UTC), or when there is none.""" - try: - if os.environ.get('TZ'): - zone = zoneinfo.ZoneInfo(os.environ['TZ'].lstrip(':')) - elif '/zoneinfo/' in (link := os.path.realpath('/etc/localtime')): - zone = zoneinfo.ZoneInfo(link.split('/zoneinfo/', 1)[1]) - else: - zone = zoneinfo.ZoneInfo(Path('/etc/timezone').read_text().strip()) - except (OSError, ValueError, zoneinfo.ZoneInfoNotFoundError): - return jobs.UTC - year = [datetime.datetime(2026, month, 1, tzinfo=zone).utcoffset() for month in (1, 7)] - return jobs.UTC if year == [datetime.timedelta(0)] * 2 else zone - - -def legacy_lines(text, path): - """The jobs of an old schedule.cron (text, from path): (line number, time, zone name, [(report name, arguments)]) - for each line that parses, in the zone set above it (CRON_TZ=) or the server's own; pasted crontab syntax (paths, - .py, redirects, braces, variables) read as cron read it. A line that does not parse is logged.""" - zone = local_zone().key - for number, raw in enumerate(text.splitlines(), 1): - line = raw.strip() - if not line or line.startswith('#'): - continue - if m := re.fullmatch(r'CRON_TZ=(\S+)', line): - zone = m.group(1) - continue - if re.fullmatch(r'\w+=.*', line): # MAILTO=, PATH= and the like - continue - if line.startswith('@'): - when, _, rest = line.partition(' ') - else: - fields = line.split(None, 5) - if len(fields) < 6: - continue - when, rest = ' '.join(fields[:5]), fields[5] - try: - commands = [(os.path.basename(words[0]).removesuffix('.py'), words[1:]) - for words in split_commands(tokenize(rest))] - except ValueError as e: - log.warn(f'schedule: {path} line {number} left out: {e}') - continue - yield number, when, zone, commands - - -def tokenize(text): - """A command line as ('word', text) and ('op', text) tokens. Quoted text is always a word, whatever it contains.""" - tokens, word, started, i = [], '', False, 0 - def flush(): - """End the word being read, if there is one, and add it to the tokens.""" - nonlocal word, started - if started: - tokens.append(('word', word)) - word, started = '', False - while i < len(text): - c = text[i] - if c.isspace(): - flush() - i += 1 - elif c in '\'"': - end = text.find(c, i + 1) - if end < 0: - raise ValueError('no closing quotation') - word, started, i = word + text[i + 1:end], True, end + 1 - elif c == '\\' and i + 1 < len(text): - word, started, i = word + text[i + 1], True, i + 2 - elif c == '#' and not started: # a trailing comment - break - elif m := OPERATOR.match(text, i): - if word.isdigit() and m.group().startswith('>'): # the 2 of 2>&1 belongs to the redirect - word, started = '', False - flush() - tokens.append(('op', m.group())) - i = m.end() - else: - word, started, i = word + c, True, i + 1 - flush() - return tokens - - -def split_commands(tokens): - """Word lists, one per command: split on unquoted ; and &&, redirects and braces dropped, variables before a command - dropped, other shell syntax refused (ValueError).""" - commands, current, target = [], [], False - for kind, value in tokens: - if target: # the file of a redirect - target = False - if kind == 'word': - continue - if kind == 'word': - if current or not re.fullmatch(r'\w+=\S*', value): # FOO=bar before a command - current.append(value) - elif value in (';', '&&'): - commands.append(current) - current = [] - elif value in ('{', '}'): - continue - elif REDIRECT.fullmatch(value): - target = not re.search(r'&\d+$', value) # 2>&1 has no file after it, > /dev/null does - else: - raise ValueError('only finbot scripts can be scheduled, not shell commands') - commands.append(current) - return [words for words in commands if words] diff --git a/lib/reports/__init__.py b/lib/reports/__init__.py index 7276706d..387268d7 100644 --- a/lib/reports/__init__.py +++ b/lib/reports/__init__.py @@ -1,6 +1,6 @@ -"""What finbot says. A module per scheduled report: cal, cleanup (of the cache), milestone, performance, podcasts, -price, rating, reminder, shorts, trades, and sharesight_warm and yahoo_warm (which fill the cache), each with -job(argv), which parses its schedule line, and run(dest=None, **kwargs), which runs it, to the chat that asked or -whose job it is (dest); the scheduler runs them in the bot's process, and `.jobs run` runs one now. And the -chat-only reports: about (.help, .jobs), history (.history, .compare), holdings (.holdings, .who, .pie, .watchlist), +"""What finbot says. A module per scheduled report: cal, cleanup (of the cache), message (a text, posted as it is), +milestone, performance, podcasts, price, rating, shorts, trades, and sharesight_warm and yahoo_warm (which fill the +cache), each with job(argv), which parses a job's arguments, and run(dest=None, **kwargs), which runs it, to the chat +that asked or whose job it is (dest); the scheduler runs them in the bot's process, and `.jobs run` runs one now. And +the chat-only reports: about (.help, .jobs), history (.history, .compare), holdings (.holdings, .who, .pie, .watchlist), profile (.SYMBOL) and rankings (.marketcap, .pe and the rest).""" diff --git a/lib/reports/about.py b/lib/reports/about.py index ea6c8dbb..ecf885b7 100644 --- a/lib/reports/about.py +++ b/lib/reports/about.py @@ -1,60 +1,257 @@ -"""What finbot can do: the .help text, and the .jobs listing of a chat's jobs and the reports a job can run.""" +"""What finbot can do: the .help text, and the .jobs menu: a chat's jobs a page at a time, with buttons that switch, +delete or run one; the steps of Add; the next runs; the system jobs; and the reports a job can run.""" +import datetime + from lib import buttons +from lib import catalog from lib import config from lib import formats from lib import jobs +from lib import when + +HEADINGS = {'top': 'Jobs here', 'on': 'Tap a job to switch it on or off', 'run': 'Tap a job to run it now', + 'del': 'Tap a job added here to delete it (a default job is switched off instead)'} +TYPED = ['.jobs run|toggle|del NUMBER: run one now, switch it on or off, delete one added here', + '.jobs add REPORT SCHEDULE: add one (.jobs add lists the reports)', '.jobs next: the next runs', + '.jobs system: the system jobs'] # what a chat without buttons types instead +TYPED_ACTIONS = {'on': 'toggle', 'del': 'del', 'run': 'run'} +PAGE_ROOM = 500 # characters of a screen besides its jobs' lines: its headings, a note, the key to * +JOB_ROW = 4 # job buttons in a row +MOST_JOBS = {'discord': 16} # job buttons on a page: Discord takes five rows of five, one kept for the others; else 40 +# Add's reports: (label, action, arguments; None where the period follows how often it runs, or a message's text) +REPORTS = (('trades', 'trades', ()), ('price midsession', 'price', ('midsession',)), + ('price premarket', 'price', ('premarket',)), ('price close', 'price', ('close',)), ('price', 'price', None), + ('performance', 'performance', None), ('cal earnings', 'cal', ('earnings',)), + ('cal ex-dividend', 'cal', ('ex-dividend',)), ('shorts', 'shorts', ()), ('rating', 'rating', ()), + ('milestone', 'milestone', ()), ('message', 'message', None)) +FREQUENCIES = (('o', 'one-off'), ('d', 'daily'), ('k', 'weekdays'), ('w', 'weekly'), ('m', 'monthly'), + ('q', 'quarterly'), ('b', 'bi-annual'), ('a', 'annual'), ('r', 'repeating')) +DAYS = {'d': 'daily', 'k': 'weekdays', 'm': {'day': 1, 'months': ()}, 'q': {'day': 1, 'months': when.QUARTERS}, + 'b': {'day': 1, 'months': when.HALVES}, 'a': {'day': 1, 'months': (1,)}} # the days of Add's frequencies +INTERVALS = (5, 10, 15, 20, 30, 60, 120, 180, 240, 360, 720) # minutes between a repeat's runs, as Add offers them -def prepare_jobs_payload(service, chat, action=None, tappable=False): - """`.jobs`: a chat's jobs (by its key), numbered, each with its line, whether it is on and its next run, in UTC and - in the timezone preference; then the system jobs, under their own heading and unnumbered: they run on their - schedule only; then the subcommands. With action ('run' or 'toggle'), the list of what a tap does, and where the - chat takes buttons (tappable) a button per job that runs it now or switches it on or off, two to a row. Returns - (payload, buttons).""" +def jobs_screen(service, chat, screen='top', page=0, note=None, tappable=True): + """A screen of the .jobs menu that lists a chat's jobs (by its key), numbered: the default ones, then those added + here, each ✅ (on) or ❌ (off), * where it has never run, its name and its schedule in words, by the timezone + preference's clock (lib/when.py). top has what can be done (Enable/disable, Delete, Add, Run now, Next); on, del and + run a button per job, which switches it, deletes it (one added here: a default one has none) or runs it now. A page + at a time, as many jobs as a message of the network takes (◀ and More ▶ for the others), after a note of what the + last press did. A one-off gone by is deleted first. Returns (lines, buttons); where the chat takes no buttons + (tappable false), the whole list and what to type instead.""" scheduler = jobs.current if scheduler is None: return ['No schedule is running'], None + scheduler.prune(chat) own = scheduler.chat_jobs(chat) - running = scheduler.running(chat) - payload = [service.bold('Jobs here:')] + [job_entry(service, scheduler, number, job, running) - for number, job in enumerate(own, 1)] if own else ['No jobs here yet: add one with .jobs add'] - system = scheduler.system_jobs() - payload += ['', service.bold('System:')] + [f"{job.when} {job.text}" + (' (running)' if any( - label in running for label, _, _ in job.commands) else '') for job in system] - if action and own: - payload.append('\n' + ('Tap one to run it now' if action == 'run' else 'Tap one to switch it on or off') - if tappable else f"\nSend .jobs {action} NUMBER") - elif not action: - payload += ['', '.jobs run [NUMBER]: run one now', '.jobs toggle [NUMBER]: switch one on or off', - '.jobs add [CRON_TZ=ZONE] TIME REPORT: add one (.jobs add lists the reports)', - '.jobs del NUMBER: delete one'] - markup = None - if action and tappable and own: - row = [buttons.job_button(number, job.text, jobs.checksum(job.key), action) - for number, job in enumerate(own, 1)] - markup = [row[i:i + 2] for i in range(0, len(row), 2)] - return payload, markup - -def job_entry(service, scheduler, number, job, running=()): - """'1. */20 * * * Mon-Fri trades: on, next Thu 14:20 UTC (Fri 01:20 AEDT)': a job in .jobs.""" - state = 'running' if any(label in running for label, _, _ in job.commands) else 'on' if job.on else 'off' - when = scheduler.next_run(job) - if when is None: - return f"{number}. {job.line}: {state}" - utc, local = when.astimezone(jobs.UTC), when.astimezone(formats.zone()) - shown = f"{utc:%a %H:%M} UTC" + ('' if local.utcoffset() == utc.utcoffset() else f" ({local:%a %H:%M %Z})") - return f"{number}. {job.line}: {state}, next {shown}" + running, local, now = scheduler.running(chat), formats.zone().key, scheduler.clock() + entries = [(number, job, job_entry(service, scheduler, number, job, running, local, now) + + (' (default)' if screen == 'del' and job.item else '')) for number, job in enumerate(own, 1)] + parts = paged(entries, service) if tappable else [entries] + page = min(max(page, 0), len(parts) - 1) + shown = parts[page] + span = f' ({shown[0][0]}–{shown[-1][0]} of {len(own)})' if len(parts) > 1 else '' + lines = ([note, ''] if note else []) + [service.bold(HEADINGS[screen] + span)] + if screen == 'del' and all(job.item for job in own): + lines.append('None added here yet: Add makes one') + kind = None + for number, job, line in shown: + if kind != (job.item is not None): # a heading where the default jobs start, and those added here + kind = job.item is not None + lines += ['', service.bold('Default jobs:' if kind else 'Added here:')] + lines.append(line) + if not all(scheduler.has_run(job) for _, job, _ in shown): + lines += ['', '* not run yet'] + if not tappable: + return lines + [''] + (TYPED if screen == 'top' else [f'Send .jobs {TYPED_ACTIONS[screen]} NUMBER']), None + return lines, screen_buttons(screen, page, len(parts), shown, jobs.list_checksum(own)) + +def paged(entries, service): + """The jobs' entries (number, job, line) a page at a time: as many as fit a message of the network with room to + spare, and as many buttons as it takes in a message besides the others.""" + most, room = MOST_JOBS.get(service, 40), service.text_limit - PAGE_ROOM + parts, size = [[]], 0 + for entry in entries: + if parts[-1] and (len(parts[-1]) == most or size + len(entry[2]) + 1 > room): + parts.append([]) + size = 0 + parts[-1].append(entry) + size += len(entry[2]) + 1 + return parts + +def screen_buttons(screen, page, pages, shown, crc): + """A list screen's buttons: top's, what can be done; on, del and run's, one for each job shown that the screen + acts on, ' ' (with its ✅ or ❌ to switch it), JOB_ROW to a row, then Back; ◀ and More ▶ to the other + pages.""" + if screen == 'top': + rows = [[buttons.menu_button('Enable/disable', 'on', [0], crc), buttons.menu_button('Delete', 'del', [0], crc)], + [buttons.menu_button('Add', 'add', [], crc), buttons.menu_button('Run now', 'run', [0], crc), + buttons.menu_button('Next', 'next', [], crc)]] + nav = [] + else: + tapped = [buttons.menu_button(f"{number} {('✅ ' if job.on else '❌ ') if screen == 'on' else ''}{job.name}", + screen, [page, number], crc) for number, job, _ in shown if screen != 'del' or not job.item] + rows = [tapped[i:i + JOB_ROW] for i in range(0, len(tapped), JOB_ROW)] + nav = [buttons.menu_button('Back', 'top', [0], crc)] + if page > 0: + nav.append(buttons.menu_button('◀', screen, [page - 1], crc)) + if page < pages - 1: + nav.append(buttons.menu_button('More ▶', screen, [page + 1], crc)) + return rows + ([nav] if nav else []) + +def job_entry(service, scheduler, number, job, running=(), local=None, now=None): + """'5. ✅ price close AU · weekdays at 4:30pm Sydney': a job in .jobs, with its number, whether it is on (and * + where it has never run), its name, its schedule in words, and whether it is running now.""" + mark = ('✅' if job.on else '❌') + ('' if scheduler.has_run(job) else '*') + schedule = job.schedule.describe(local or formats.zone().key, now or scheduler.clock()) + return f"{number}. {mark} {service.escape(job.name)} · {schedule}" + (' (running)' if job.key in running else '') def job_line(number, job, outcome): - """'3. trades: done': a job's number in the .jobs list, its reports, and how its run went.""" - return f"{number}. {job.text}: {outcome}" + """'3. trades: done': a job's number in the .jobs list, its name, and how its run went.""" + return f"{number}. {job.name}: {outcome}" + +def next_screen(service, chat, count=20): + """`.jobs next`: a chat's next runs, in time order, at most count, each by the timezone preference's clock (and its + own zone's beside it, where that differs), with its number and name in .jobs; a repeat's runs on a day are one + entry, from its first to its last. Returns (lines, buttons): Back, to the list.""" + scheduler = jobs.current + if scheduler is None: + return ['No schedule is running'], None + local = formats.zone() + lines = [service.bold('Next runs:')] + for first, last, job, number in scheduler.upcoming(chat, count): + said = moment_words(first, local, job.schedule.tz) + if job.schedule.every: + end = last.astimezone(local) + said += (f" – {clock_words(end, local)}" if end.date() == first.astimezone(local).date() else + f" – {moment_words(last, local)}") + f", {when.repeat_words(job.schedule.every, 0, local.key == 'UTC')}" + lines.append(f"{said} · {number}. {service.escape(job.name)}") + if len(lines) == 1: + lines.append('None: every job here is off') + return lines, [[buttons.menu_button('Back', 'top', [0], jobs.list_checksum(scheduler.chat_jobs(chat)))]] + +def clock_words(moment, zone): + """A moment's time of day by a zone's clock: '8:30am', in 24 hours for UTC ('21:30').""" + here = moment.astimezone(zone) + return when.clock(here.hour * 60 + here.minute, zone.key == 'UTC') + +def moment_words(moment, local, zone=None): + """'Sun 18 Oct 8:30am': a moment by the local clock, with a zone's beside it where that differs ('Tue 20 Oct 3:45am + (12:45pm New York)').""" + here = moment.astimezone(local) + said = f"{here:%a} {here.day} {here:%b} {clock_words(moment, local)}" + if zone is not None and moment.astimezone(zone).utcoffset() != here.utcoffset(): + said += f" ({clock_words(moment, zone)} {when.place(zone.key)})" + return said + +def add_screen(service, chat, steps): + """A step of Add, from the choices made so far (steps, from the button pressed: the report's number, how often, + then the date, the day, or a repeat's interval and its days, then the hour): (lines, buttons) of the next choice, + or (None, (action, arguments, schedule)) once every choice is made, at a time by the timezone preference's clock. + Back takes a step back, Cancel goes back to the list. Raises ValueError for choices that are none (forged data).""" + crc = jobs.list_checksum(jobs.current.chat_jobs(chat)) + local = formats.zone() + steps = list(steps) + # Back from the first step goes to the list too, but its data differs from Cancel's: Slack and Discord take a + # button's data as its id, one of its own in a message + back = buttons.menu_button('Back', *(('add', steps[:-1]) if steps else ('top', [])), crc) + cancel = buttons.menu_button('Cancel', 'top', [0], crc) + def choices(heading, offered, row, lines=()): + """A choice's screen: its heading and lines, its buttons (label, data) `row` to a row, then Back and Cancel.""" + tapped = [buttons.menu_button(label, 'add', [*steps, data], crc) for label, data in offered] + return [service.bold(heading), *lines], [tapped[i:i + row] for i in range(0, len(tapped), row)] + [[back, + cancel]] + if not steps: + return choices('Add a job: which report?', [(f'{number} {label}', number) for number, (label, _, _) in + enumerate(REPORTS, 1)], 3, [f'{number}. {label}' for number, (label, _, _) in enumerate(REPORTS, 1)]) + label, action, args = REPORTS[choice(steps[0], 1, len(REPORTS)) - 1] + if action == 'message': + return [service.bold('Add a message'), "A message's text is what you type:", + '.jobs add message SCHEDULE "TEXT"', '', service.bold('For example:'), + '.jobs add message every 31 mar at 8:30am "🥳 Happy EOFY"', + '.jobs add message on 28 oct 2026 "ubank draw 6/6"'], [[back, cancel]] + if len(steps) == 1: + return choices(f'Add {label}: how often?', [(name, code) for code, name in FREQUENCIES], 3) + frequency = steps[1] + if frequency not in dict(FREQUENCIES): + raise ValueError(f'no frequency {frequency!r}') + needed = 4 if frequency in 'owr' else 3 # the choices it takes, the report's and how often included + if len(steps) == 2 and frequency == 'o': + today = jobs.current.clock().astimezone(local).date() + dates = [today + datetime.timedelta(days=n) for n in range(14)] + return choices(f'Add {label} once: on which day?', [(f'{date:%a} {date.day} {date:%b}', f'{date:%y%m%d}') + for date in dates], 4, ['A later one is typed: .jobs add REPORT on 28 oct 2026 at 9am']) + if len(steps) == 2 and frequency == 'w': + return choices(f'Add {label} weekly: on which day?', [(day.title(), number) for number, day in + enumerate(when.DAYS)], 4) + if len(steps) == 2 and frequency == 'r': + return choices(f'Add {label}, repeating: how often?', [(when.repeat_words(minutes, 0, False).removeprefix( + 'every '), minutes) for minutes in INTERVALS], 4, ['A window of hours is typed: .jobs add REPORT every 20' + ' mins weekdays from 10am to 4pm']) + if len(steps) == 3 and frequency == 'r': + choice(steps[2], 1, 1440, INTERVALS) + return choices(f'Add {label}, repeating: on which days?', [('every day', 'd'), ('weekdays', 'k')], 2) + if len(steps) == needed - 1: + hours = [buttons.menu_button(f'{hour:02d}', 'add', [*steps, hour], crc) for hour in range(24)] + [back] + return [service.bold(f'Add {label}: at what hour?'), f"By {when.place(local.key)}'s clock; a time such as" + ' 9:30am is typed: .jobs add REPORT weekly at 9:30am'], [hours[i:i + 5] for i in range(0, 25, 5)] + if len(steps) != needed: + raise ValueError('too many choices') + if frequency == 'r': + if steps[3] not in ('d', 'k'): + raise ValueError(f'no days {steps[3]!r}') + schedule = when.Schedule('daily' if steps[3] == 'd' else 'weekdays', every=choice(steps[2], 1, 1440, + INTERVALS), zone=local.key) + else: + at = choice(steps[-1], 0, 23) * 60 + if frequency == 'o': + date = steps[2] + days = {'date': datetime.date(2000 + int(date[:2]), int(date[2:4]), int(date[4:6])).isoformat()} + elif frequency == 'w': + days = (when.DAYS[choice(steps[2], 0, 6)],) + else: + days = DAYS[frequency] + schedule = when.Schedule(days, at=at, zone=local.key) + schedule.check() + return None, (action, list(args) if args is not None else catalog.period_args(action, schedule), schedule) + +def choice(text, low, high, among=None): + """A choice's number from a button's data, from low to high (and among those offered, where given). Raises + ValueError for one that is none.""" + if not (text.isascii() and text.isdigit() and low <= int(text) <= high and (among is None or int(text) in among)): + raise ValueError(f'no choice {text!r}') + return int(text) + +def added_screen(service, chat, number): + """What the last step of Add shows: the job added, with its number in the list, and buttons to it and to Add.""" + own = jobs.current.chat_jobs(chat) + crc = jobs.list_checksum(own) + return [f"Added: {job_entry(service, jobs.current, number, own[number - 1])}"], [[buttons.menu_button('Jobs', + 'top', [0], crc), buttons.menu_button('Add another', 'add', [], crc)]] + +def add_failed(service, chat, steps, why): + """What the last step of Add shows when the job cannot be added: why, with Back to the last choice and Cancel.""" + crc = jobs.list_checksum(jobs.current.chat_jobs(chat)) + return [service.escape(why)], [[buttons.menu_button('Back', 'add', list(steps)[:-1], crc), + buttons.menu_button('Cancel', 'top', [0], crc)]] + +def prepare_system(service): + """`.jobs system`: the system jobs, which fill the cache and clear it out, each with its schedule in words.""" + scheduler = jobs.current + if scheduler is None: + return ['No schedule is running'] + running, local, now = scheduler.running(''), formats.zone().key, scheduler.clock() + return [service.bold('System jobs:')] + [f"{job.name} · {job.schedule.describe(local, now)}" + + (' (running)' if job.key in running else '') for job in scheduler.system_jobs()] def prepare_reports(service): """`.jobs add` alone: the reports a job can run, each with what it takes, and examples.""" - names = [name for name in jobs.SCRIPTS if name not in jobs.SYSTEM_SCRIPTS] - return [service.bold('Reports a job can run:')] + [f"{name} {jobs.SCRIPTS[name].ARGUMENTS}".strip() - for name in sorted(names)] + ['', service.bold('For example:'), '.jobs add 29 21 * * Mon-Fri price interday', - '.jobs add 28 21 * * Sat cal earnings; price 7 (several in turn)', - '.jobs add CRON_TZ=America/New_York 15 13 * * Mon-Fri price midsession (UTC unless it says)'] + return [service.bold('Reports a job can run:')] + [f"{name} {jobs.ACTIONS[name].ARGUMENTS}".strip() + for name in jobs.chat_actions()] + ['', service.bold('For example:'), '.jobs add price weekly', + '.jobs add price close weekdays at 4:30pm New York', '.jobs add performance first day of each month at 9am', + '.jobs add trades every 20 mins weekdays from 10am to 4pm', + '.jobs add message every 31 mar at 8:30am "🥳 Happy EOFY"', '', + f"Times are by {when.place(formats.zone().key)}'s clock unless one names a zone (UTC, New York, EST);" + " performance and price cover the days between runs"] def prepare_help(service, botName, sections): """The .help text: each section's command lines under its heading (worker.help_sections() has them), then the bot's diff --git a/lib/reports/message.py b/lib/reports/message.py new file mode 100644 index 00000000..a949c798 --- /dev/null +++ b/lib/reports/message.py @@ -0,0 +1,46 @@ +"""A text that a job posts as it is, a reminder say (scheduled: message "TEXT"), in the network's markup: [text](url) is +a link and a line wrapped in _underscores_ is in italics (formats.markup); {years since YYYY-MM-DD} becomes the years +since that date as an ordinal, for an anniversary ("It's my 4th birthday").""" +import datetime +import re + +from lib import delivery +from lib import formats + +ARGUMENTS = '"TEXT"' # what a scheduled run takes: its text, in double quotes, over several lines if need be +MAX_LINES = 40 +MAX_CHARACTERS = 3000 +YEARS = re.compile(r'\{years since (\d{4})-(\d{2})-(\d{2})\}') + + +def run(dest=None, *, lines): + """Post the lines to the chat whose job it is (dest), each placeholder filled, by an explicit replace: the text is + data, never a format string.""" + filled = [YEARS.sub(lambda found: years_since(found, today()), line) for line in lines] + for chat in delivery.targets(dest): + delivery.deliver(chat, delivery.Report([formats.markup(line, chat.dialect) for line in filled])) + + +def today(): + """Today's date, by the timezone preference's clock.""" + return datetime.datetime.now(formats.zone()).date() + + +def years_since(found, on): + """The years from a placeholder's date to the date on, as an ordinal ('4th'); for no date, the placeholder.""" + try: + since = datetime.date(*map(int, found.groups())) + except ValueError: + return found.group(0) + years = on.year - since.year - ((on.month, on.day) < (since.month, since.day)) + return f'{years}{formats.ordinal(years)}' + + +def job(argv): + """A scheduled run's arguments: (label, run's keyword arguments), its text's lines; none or too many: ValueError.""" + lines = list(argv) + if not lines or not all(isinstance(line, str) for line in lines) or not any(line.strip() for line in lines): + raise ValueError(ARGUMENTS) + if len(lines) > MAX_LINES or sum(map(len, lines)) > MAX_CHARACTERS: + raise ValueError(f'a text of {MAX_LINES} lines and {MAX_CHARACTERS} characters at most') + return 'message', {'lines': lines} diff --git a/lib/reports/reminder.py b/lib/reports/reminder.py deleted file mode 100644 index 36fcb381..00000000 --- a/lib/reports/reminder.py +++ /dev/null @@ -1,65 +0,0 @@ -"""Reminders for the day: the entries of reminder.json for today, and finbot's birthday (scheduled: reminder).""" -import datetime -import re -import zoneinfo -from dateutil.relativedelta import relativedelta -from lib import config -from lib import formats -from lib import delivery - -BIRTHDAY = datetime.date(2022, 8, 18) -FLAGS = {'AU': '🇦🇺'} # what {flag} becomes in an entry gated to a country -LINK = re.compile(r'\[([^\]]+)\]\(([^)\s]+)\)') # [text](url) - - -def run(dest=None): - """Post today's reminders, if there are any, to the chat whose job it is (dest).""" - today = datetime.datetime.now(zoneinfo.ZoneInfo(config.prefs.timezone)).date() - entries = config.data_json("reminder.json", []) - for chat in delivery.targets(dest): - delivery.deliver(chat, delivery.Report(report(chat.dialect, today, entries))) - - -def report(service, today, entries): - """Today's reminders in one network's markup, under a heading, or [] when there are none: the entries of - reminder.json for this day (and year, and country, where an entry names one), and finbot's birthday.""" - payload = [] - for entry in entries: - if entry.get("month_day") != today.strftime('%m-%d'): - continue - if entry.get("year") and str(entry["year"]) != today.strftime('%Y'): - continue - country = entry.get("gate_country") - if country and config.prefs.country_code != country: - continue - message = entry["message"] # a line, or a list of them - for line in [message] if isinstance(message, str) else message: - payload.append(markup(line.replace("{flag}", FLAGS.get(country, '')), service)) - if today.strftime('%m-%d') == BIRTHDAY.strftime('%m-%d'): - years = relativedelta(today, BIRTHDAY).years - payload.append(f"It's my {years}{formats.ordinal(years)} birthday! 🥳") - return [service.bold("Finance event reminders:")] + payload if payload else [] - - -def markup(line, service): - """A line of reminder.json in the network's markup: [text](url) is a link, a line wrapped in _underscores_ is in - italics, and the rest is text.""" - italic = len(line) > 1 and line.startswith('_') and line.endswith('_') - if italic: - line = line[1:-1] - text, start = '', 0 - for link in LINK.finditer(line): - text += service.escape(line[start:link.start()]) + service.link(link.group(2), service.escape(link.group(1))) - start = link.end() - text += service.escape(line[start:]) - return service.italic(text) if italic else text - - -ARGUMENTS = '' # what a scheduled run takes: none - -def job(argv): - """A scheduled run's arguments: (label, run's keyword arguments); reminder takes none, and an argument raises - ValueError.""" - if argv: - raise ValueError('no arguments') - return 'reminder', {} diff --git a/lib/when.py b/lib/when.py index cad1cd03..2df198a3 100644 --- a/lib/when.py +++ b/lib/when.py @@ -233,8 +233,7 @@ def words(self, utc): return f"{head}, {tail}" if tail else head def repeat(self, utc, gap=0): - """A repeat's words, (how often, its window of hours or ''): ('hourly at :01', '10am–9pm'), the window moved by - gap minutes.""" + """A repeat's words, (how often, its window of hours or ''), the window moved by gap minutes.""" start, end = self.start or 0, self.end if end is None and start < self.every: # where in each step it runs, not a window return repeat_words(self.every, start, utc), '' @@ -422,8 +421,7 @@ def repeat_days(days): def shifted(days, shift): - """The days as a clock `shift` days ahead has them: 'weekdays' a day on is 'Tue–Sat'; '' where that says no more - (no shift, daily).""" + """The days as a clock `shift` days ahead has them ('weekdays' a day on is 'Tue–Sat'); '' for daily or no shift.""" if not shift or days == 'daily': return '' if isinstance(days, dict): diff --git a/lib/worker.py b/lib/worker.py index a28507dd..1b165bc7 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -2,6 +2,7 @@ table also makes the .help text and the Telegram command keyboard, which is kept current in every DM.""" import dataclasses import hashlib +import hmac import json import math import re @@ -43,15 +44,15 @@ def reply(self, lines, chart=None, buttons=None): return delivery.deliver(self.dest, delivery.Report(lines, chart, buttons)) def process_job_callback(press, pressed): - """A job's button (an inbound.Press; pressed, what its data says): run the job now and say how it went, or switch - it on or off and redraw the list in place. A button from an older list does neither: the list is redrawn as it is - now.""" + """An older job button (an inbound.Press; pressed, what its data says): run the job now and say how it went, or + switch it on or off and redraw the list in place, as the menu's screen that does the same. A button from an older + list does neither: the screen is drawn as the list is now.""" service = dialects.of(press.service) chat = delivery.reply_to(service, press.chat_id, interaction=press.interaction, pressed=True) def redraw(): - """Put the chat's list, as it is now, in place of the pressed one, with the same kind of buttons.""" - payload, markup = about.prepare_jobs_payload(service, chat.key, pressed.action, chat.takes_buttons) - chat.edit_text(press.message_id, '\n'.join(payload), markup or []) + """Put the menu's screen that does the same, as the list is now, in place of the pressed one.""" + lines, markup = about.jobs_screen(service, chat.key, 'on' if pressed.action == 'toggle' else 'run') + chat.edit_text(press.message_id, '\n'.join(lines), markup or []) try: if jobs.current is None: raise jobs.ScheduleError('no schedule is running') @@ -67,10 +68,70 @@ def redraw(): except Exception as e: delivery.report_error(e, chat, context='jobs') +def process_menu_callback(press, pressed): + """A press in the .jobs menu (an inbound.Press; pressed, the buttons.MenuPress its data says): a screen to show; or + first a job to switch, delete or run, as long as the list is the one the button was drawn for (its checksum), else + the screen is drawn as the list is now, saying so; or a step of Add, the last adding the job. The pressed message + then shows the screen; a run's outcome is a message of its own.""" + service = dialects.of(press.service) + chat = delivery.reply_to(service, press.chat_id, interaction=press.interaction, pressed=True) + try: + lines, markup = menu_screen(service, chat, pressed) + chat.edit_text(press.message_id, '\n'.join(lines), markup or []) + except Exception as e: + delivery.report_error(e, chat, context='jobs') + +def menu_screen(service, chat, pressed): + """The screen that a press in the .jobs menu shows, once what it asks for is done (see process_menu_callback).""" + scheduler, key, screen, args = jobs.current, chat.key, pressed.screen, pressed.args + if scheduler is None: + return ['No schedule is running'], None + if screen == 'next': + return about.next_screen(service, key) + if screen == 'add': + return add_step(service, chat, args) + page = int(args[0]) if args and args[0].isdigit() and len(args[0]) < 4 else 0 + note = None + if len(args) == 2 and screen in ('on', 'del', 'run'): + own, number = scheduler.chat_jobs(key), args[1] + if not (hmac.compare_digest(jobs.list_checksum(own), pressed.checksum) and number.isdigit() and + 1 <= int(number) <= len(own)): + note = 'The jobs have changed: here they are now' + else: + job = own[int(number) - 1] + try: + if screen == 'on': + scheduler.toggle(job) + elif screen == 'del': + scheduler.delete(job) + note = f'Deleted {number}. {service.escape(job.name)}' + else: + delivery.deliver(chat, delivery.Report([about.job_line(number, job, scheduler.run(job))])) + except jobs.ScheduleError as e: + note = service.escape(str(e)) + return about.jobs_screen(service, key, screen, page, note) + +def add_step(service, chat, steps): + """A step of Add pressed: the next choice's screen; once every choice is made, the job added, or why it is not.""" + try: + lines, made = about.add_screen(service, chat.key, steps) + except ValueError: # data that is no step of Add: forged, or from an older finbot + return about.jobs_screen(service, chat.key, 'top', 0, 'That is no step of Add') + if lines is not None: + return lines, made + try: + number = jobs.current.add(chat.key, *made) + except jobs.ScheduleError as e: + return about.add_failed(service, chat.key, steps, str(e)) + return about.added_screen(service, chat.key, number) + def process_callback(press): - """A button pressed (an inbound.Press, which bot.take acknowledged as it arrived): a job's (run it, or switch it), - or a chart's, whose message is rebuilt for the chosen period and edited in place. Data of neither is ignored.""" + """A button pressed (an inbound.Press, which bot.take acknowledged as it arrived): the .jobs menu's, an older job + button (run it, or switch it), or a chart's, whose message is rebuilt for the chosen period and edited in place. + Data of none of them is ignored.""" pressed = buttons.parse(press.data) + if isinstance(pressed, buttons.MenuPress): + return process_menu_callback(press, pressed) if isinstance(pressed, buttons.JobPress): return process_job_callback(press, pressed) if not pressed: @@ -99,6 +160,11 @@ def process_request(message): request = Request(dialects.of(message.service), message.chat_id, message.user, message.user_name, message.bot_name, message.message_id, message.interaction) command, args = read(message.text, message.bot_name) + if command and jobs.current: # a chat finbot does not know yet starts with the default jobs + try: + jobs.current.see(request.dest.key) + except Exception as e: + log.warn('schedule: cannot take on', request.dest.key, delivery.error_line(e)) if command: try: request.dest.start_typing() @@ -364,27 +430,36 @@ def on_who(request, args): request.reply(holdings.prepare_who(request.service, re.split(r'[.:]', args.words[0].upper())[0])) def on_jobs(request, args): - """.jobs: this chat's jobs; .jobs run or toggle [NUMBER]: one run now or switched on or off (without a number, the - list with a button per job that does it); .jobs add [CRON_TZ=ZONE] TIME REPORT: one more (alone, the reports it can - run); .jobs del NUMBER: one fewer.""" + """.jobs: this chat's jobs, the default ones and those added here, with the menu's buttons; .jobs next: the next + runs; .jobs system: the system's; .jobs run, toggle or del NUMBER: one run now, switched on or off, or deleted (one + added here), and without a number the list with a button per job that does it; .jobs add REPORT SCHEDULE, or .jobs + add message SCHEDULE "TEXT": one more (alone, the reports a job can run).""" scheduler, chat, service = jobs.current, request.dest.key, request.service + tappable = request.dest.takes_buttons if scheduler is None: raise answer.Answer('No schedule is running') try: if args.keyword is None and not args.words: - request.reply(about.prepare_jobs_payload(service, chat)[0]) + lines, markup = about.jobs_screen(service, chat, tappable=tappable) + request.reply(lines, buttons=markup) + elif args.keyword == 'next' and not args.words: + lines, markup = about.next_screen(service, chat) + request.reply(lines, buttons=markup if tappable else None) + elif args.keyword == 'system' and not args.words: + request.reply(about.prepare_system(service)) elif args.keyword == 'add': - first, line = (args.text.split(None, 1) + [''])[:2] - if first.lower() != 'add': # the job's line follows add + first, text = (args.text.split(None, 1) + [''])[:2] + if first.lower() != 'add': # the job follows add raise Usage - if not line: + if not text.strip(): return request.reply(about.prepare_reports(service)) - number = scheduler.add(chat, line) + number = scheduler.add(chat, *jobs.parse_add(text, formats.zone().key)) request.reply([about.job_entry(service, scheduler, number, scheduler.chat_jobs(chat)[number - 1]), 'Added: .jobs lists them all']) - elif args.keyword in ('run', 'toggle') and not args.words: - payload, markup = about.prepare_jobs_payload(service, chat, args.keyword, request.dest.takes_buttons) - request.reply(payload, buttons=markup) + elif args.keyword in ('run', 'toggle', 'del') and not args.words: + screen = {'run': 'run', 'toggle': 'on', 'del': 'del'}[args.keyword] + lines, markup = about.jobs_screen(service, chat, screen, tappable=tappable) + request.reply(lines, buttons=markup) elif args.keyword in ('run', 'toggle', 'del') and len(args.words) == 1: number = args.words[0] job = scheduler.job(chat, number) @@ -395,7 +470,7 @@ def on_jobs(request, args): request.reply([about.job_entry(service, scheduler, number, scheduler.job(chat, number))]) else: scheduler.delete(job) - request.reply([f"Deleted {number}. {job.line}"]) + request.reply([f"Deleted {number}. {job.name} · {job.schedule.describe(formats.zone().key)}"]) else: raise Usage except jobs.ScheduleError as e: # a job that cannot be, or a number that is none: said, as an answer @@ -433,8 +508,8 @@ class Command: CHANGE = {'add': 'add'} | dict.fromkeys(('del', 'delete', 'rem', 'remove', 'rm'), 'delete') # of the watchlist PLANS = 'finbot_plan.json' # each chat's plans, {user: plan}, as state.load_for keeps them BY = dict.fromkeys(('sector', 'sectors'), 'sector') | dict.fromkeys(('industry', 'industries'), 'industry') # .pie's -JOB_CHANGES = {'run': 'run', 'toggle': 'toggle', 'add': 'add'} | dict.fromkeys(('del', 'delete', 'rem', 'remove', 'rm'), - 'del') # what .jobs does to a chat's jobs +JOB_CHANGES = {'run': 'run', 'toggle': 'toggle', 'add': 'add', 'next': 'next', 'system': 'system'} | dict.fromkeys(( + 'del', 'delete', 'rem', 'remove', 'rm'), 'del') # what .jobs does to a chat's jobs, and shows BUY_SELL = ('buy', 'sell', *(a + sep + b for a, b in (('buy', 'sell'), ('sell', 'buy')) for sep in ('', '&', '/', '+'))) PROFILE = Command(('stock', 'profile'), on_profile, ('.SYMBOL', '.stock SYMBOL'), COMPANY) # any first word that is no # command's name but may be a ticker calls it too @@ -470,8 +545,9 @@ class Command: Command(('shorts', 'short'), on_shorts, ('.shorts [percent|SYMBOL]',), RISK, ('.shorts',), percent=True), Command(('trades', 'trade'), on_trades, ('.trades [period] [portfolio]',), TRADES, ('.trades',), period=True), Command(('plan',), on_plan, ('.plan [your plan]',), TRADES), - Command(('jobs', 'job', 'cron', 'crons', 'schedule'), on_jobs, ('.jobs [run|toggle] [NUMBER]', - '.jobs add [CRON_TZ=ZONE] TIME REPORT', '.jobs del NUMBER'), JOBS, keywords=JOB_CHANGES), + Command(('jobs', 'job', 'cron', 'crons', 'schedule'), on_jobs, ('.jobs [next|system]', + '.jobs run|toggle|del [NUMBER]', '.jobs add REPORT SCHEDULE', '.jobs add message SCHEDULE "TEXT"'), JOBS, + keywords=JOB_CHANGES), PROFILE, ] @@ -528,7 +604,9 @@ def slash_commands(aliases=True): for name, (command, lines) in found.items()] PERIOD_WORDS = ('1d', '7d', '1m', '3m', '6m', 'ytd', '1y', '5y', '10y') # suggested where a command takes a period -JOB_WORDS = ('run', 'toggle', 'add', 'del') # what /jobs suggests first +JOB_WORDS = ('add', 'run', 'toggle', 'del', 'next', 'system') # what /jobs suggests first +SCHEDULE_WORDS = ('daily', 'weekdays', 'weekly', 'monthly', 'quarterly', 'bi-annual', 'annual', 'hourly', 'every', 'at', + 'from', 'between', 'on') # what a schedule in English is made of, suggested after a report def completes(command): """Whether suggest has words to offer for a command's arguments: its keywords, periods, tickers, portfolios, or @@ -581,30 +659,25 @@ def argument_words(command, chat, given): return [(word, word) for word in dict.fromkeys(words)] def job_words(chat, words): - """(word, shown) pairs for /jobs, for suggest, after the words given: first its subcommands; after run, toggle or - del, this chat's job numbers, each shown with its line; after add and a time, the reports, each shown with what it - takes; after a report's name, the words it takes.""" + """(word, shown) pairs for /jobs, for suggest, after the words given: first its subcommands; after run or toggle, + this chat's job numbers, each shown with its name (after del, those of the jobs added here); after add, the reports, + each shown with what it takes; after a report, the words it takes, then those of a schedule.""" if not words: return [(word, word) for word in JOB_WORDS] action = JOB_CHANGES.get(words[0].lower()) if action in ('run', 'toggle', 'del') and len(words) == 1: own = jobs.current.chat_jobs(chat) if jobs.current else [] - return [(str(number), f'{number}: {job.line}') for number, job in enumerate(own, 1)] + return [(str(number), f'{number}: {job.name}') for number, job in enumerate(own, 1) + if action != 'del' or not job.item] if action != 'add': return [] - line = words[1:][1:] if words[1:2] and words[1].startswith('CRON_TZ=') else words[1:] - time = 1 if line and line[0].startswith('@') else 5 - if len(line) < time: - return [] - after = line[time:] - if not after or after[-1].endswith(';'): # a report's name is next - return [(name, f'{name} {module.ARGUMENTS}'.strip()) for name, module in sorted(jobs.SCRIPTS.items()) - if name not in jobs.SYSTEM_SCRIPTS] - start = max(i for i in range(len(after)) if i == 0 or after[i - 1].endswith(';')) # the report being given - module = jobs.SCRIPTS.get(after[start].rstrip(';')) - if module is None or len(after) - start > 1 or after[start].endswith(';'): + if len(words) == 1: + return [(name, f'{name} {jobs.ACTIONS[name].ARGUMENTS}'.strip()) for name in jobs.chat_actions()] + module = jobs.ACTIONS.get(words[1].lower()) + if module is None or words[1].lower() not in jobs.chat_actions(): return [] - return [(word, word) for word in re.findall(r'[a-z][a-z-]*', module.ARGUMENTS)] + takes = re.findall(r'[a-z][a-z-]*', module.ARGUMENTS) if len(words) == 2 else [] + return [(word, word) for word in dict.fromkeys(takes + list(SCHEDULE_WORDS))] def command_keyboard(per_row=3): """Persistent reply keyboard of the commands that have buttons (Telegram DMs), alphabetical, row by row; a press diff --git a/requirements.txt b/requirements.txt index fe66405a..6932aa9e 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,4 @@ # Targets Python 3.13.5 (see .python-version). Lower bounds are the first releases with 3.13 support. -croniter>=1.3.15 matplotlib>=3.9.2,<4 numpy>=2.1.0,<3 python-dateutil>=2.9.0 diff --git a/tests/common.py b/tests/common.py index 37bc12a3..e4860e2a 100644 --- a/tests/common.py +++ b/tests/common.py @@ -15,7 +15,7 @@ from lib import render, sharesight, shortman, state, transports, watchlist, worker, yahoo from lib import discord_common, slack_common, telegram_common # each network's reader and bot from lib.reports import about, cal, history, holdings, milestone, performance, price, profile, rankings, rating -from lib.reports import reminder, shorts, trades +from lib.reports import shorts, trades REAL = {(m.__name__, n): getattr(m, n) for m, n in ((yahoo, 'fetch'), (yahoo, 'fetch_detail'), (yahoo, 'price_series'), (yahoo, 'price_history'), (yahoo, 'historic_high'), @@ -26,8 +26,8 @@ def no_jobs(data): - """No chat's jobs in a data dir, so that a bot started there runs no scheduled report (and migrates none).""" - Path(data, 'finbot_schedule.json').write_text('{}') + """No chat known in a data dir, so that a bot started there runs no scheduled report.""" + Path(data, 'finbot_chat_jobs.json').write_text('{}') def scheduled(key='telegram:1'): @@ -273,5 +273,3 @@ def http(self, module, *responses): UTC = zoneinfo.ZoneInfo('UTC') - -DEFAULT_SCHEDULE = Path(ROOT, 'defaults', 'schedule.cron') diff --git a/tests/test_discord.py b/tests/test_discord.py index 265b3516..a39b28c7 100644 --- a/tests/test_discord.py +++ b/tests/test_discord.py @@ -3,7 +3,7 @@ and the gateway.""" import contextlib, io, json, sys, types, unittest from tests.common import * -from lib import discord_websocket, jobs +from lib import catalog, discord_websocket, jobs try: from nacl.signing import SigningKey @@ -384,32 +384,32 @@ def test_keywords_periods_portfolios_and_tickers(self): self.assertEqual(self.values('/plan buy '), []) def test_the_jobs_subcommands_numbers_and_reports(self): - """/jobs suggests its subcommands; after run, toggle or del this chat's job numbers, each shown with its line; - after add and a time (five fields, an @macro, a zone first), the reports a chat can schedule, each shown with - what it takes, and then what the report takes.""" + """/jobs suggests its subcommands; after run or toggle this chat's job numbers, each shown with its name (after + del, those of the jobs it added); after add, the reports a chat can schedule, each shown with what it takes; + then what the report takes, and the words of a schedule.""" self.patch(jobs, 'SYSTEM', ()) - state.save(jobs.SCHEDULE_FILE, {'discord:77': [ - {'when': '*/20 * * * Mon-Fri', 'reports': 'trades', 'zone': 'UTC', 'on': True}, - {'when': '0 22 * * *', 'reports': 'shorts', 'zone': 'UTC', 'on': False}]}) + self.patch(catalog, 'ITEMS', (catalog.Item('rating', 'rating', 'rating', (), catalog.at('daily')),)) + state.save(jobs.JOBS_FILE, {'discord:77': {'jobs': [ + {'action': 'trades', 'args': [], 'schedule': {'days': 'weekdays', 'every': 20, 'zone': 'UTC'}, 'on': True}, + {'action': 'shorts', 'args': [], 'schedule': {'at': '22:00', 'zone': 'UTC'}, 'on': False}]}}) scheduler = jobs.Scheduler() with contextlib.redirect_stderr(io.StringIO()): scheduler.load() self.patch(jobs, 'current', scheduler) - self.assertEqual(self.values('/jobs '), ['run', 'toggle', 'add', 'del']) + self.assertEqual(self.values('/jobs '), ['add', 'run', 'toggle', 'del', 'next', 'system']) self.assertEqual(self.values('/job T'), ['toggle']) - self.assertEqual(worker.suggest('discord:77', '/jobs run '), [('run 1: */20 * * * Mon-Fri trades', 'run 1'), - ('run 2: 0 22 * * * shorts', 'run 2')]) - self.assertEqual(self.values('/jobs del 2'), ['del 2']) + self.assertEqual(worker.suggest('discord:77', '/jobs run '), [('run 1: rating', 'run 1'), + ('run 2: trades', 'run 2'), ('run 3: shorts', 'run 3')]) + self.assertEqual(self.values('/jobs del '), ['del 2', 'del 3']) self.assertEqual(self.values('/jobs run ', chat='discord:88'), []) - self.assertEqual(self.values('/jobs add 0 22 * * '), []) - reports = worker.suggest('discord:77', '/jobs add 0 22 * * * ') - self.assertIn(('add 0 22 * * * price midsession|interday|premarket|close|DAYS', 'add 0 22 * * * price'), - reports) - self.assertFalse(any(name in value for _, value in reports for name in jobs.SYSTEM_SCRIPTS)) - self.assertEqual(self.values('/jobs add CRON_TZ=America/New_York 15 13 * * Mon-Fri price m'), - ['add CRON_TZ=America/New_York 15 13 * * Mon-Fri price midsession']) - self.assertEqual(self.values('/jobs add @daily cal earnings; s'), ['add @daily cal earnings; shorts']) - self.assertEqual(self.values('/jobs add 0 22 * * * trades '), []) + reports = worker.suggest('discord:77', '/jobs add ') + self.assertIn(('add price midsession|interday|premarket|close|DAYS', 'add price'), reports) + self.assertIn(('add message "TEXT"', 'add message'), reports) + self.assertFalse(any(name in value for _, value in reports for name in jobs.SYSTEM_ACTIONS)) + self.assertEqual(self.values('/jobs add price m'), ['add price midsession', 'add price monthly']) + self.assertEqual(self.values('/jobs add price weekly a'), ['add price weekly annual', 'add price weekly at']) + self.assertEqual(self.values('/jobs add nothere '), []) + self.assertEqual(self.values('/jobs add yahoo_warm '), []) def test_at_most_25_each_of_100_characters_and_a_failure_suggests_nothing(self): """Discord takes 25 suggestions at most, each up to 100 characters; a failure suggests nothing, and is diff --git a/tests/test_finbot.py b/tests/test_finbot.py index 4a4c6d6c..167b67c3 100644 --- a/tests/test_finbot.py +++ b/tests/test_finbot.py @@ -3,6 +3,7 @@ from pathlib import Path from unittest import mock from tests.common import * +from lib.reports import message class Plumbing(FinbotCase): @@ -41,10 +42,10 @@ def test_zoneinfo_offset_is_not_lmt(self): tz = zoneinfo.ZoneInfo('Australia/Melbourne') self.assertEqual(datetime.datetime(2022, 8, 18, tzinfo=tz).utcoffset(), datetime.timedelta(hours=10)) - def test_reminder_runs(self): - """The reminder report runs with no reminders on file.""" - with mock.patch.object(config, 'data_json', lambda name, default=None: []): - reminder.run(scheduled()) + def test_a_message_job_runs(self): + """A message job posts its text to its chat.""" + message.run(scheduled(), lines=['hello']) + self.assertEqual(self.sent, [('text', ['hello'])]) def test_bot_handler(self): """A sticker is answered 200, with nothing logged but the request's line (INFO); git_version matches what git diff --git a/tests/test_reports.py b/tests/test_reports.py index fb30b8ca..ef78eab4 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -678,38 +678,6 @@ def test_a_button_a_chart_does_not_offer_is_ignored(self): self.assertEqual((edits, toasts), ([], [None])) -class Reminders(FinbotCase): - def test_a_reminder_takes_links_and_italics_in_each_networks_markup(self): - """[text](url) is a link and a line in _underscores_ is in italics, an underscore in a url included; the rest - is text, escaped where the network needs it.""" - line = '_See [R&D notes](https://x.example/a_b) & more_' - self.assertEqual(reminder.markup(line, TELEGRAM), - 'See R&D notes & more') - self.assertEqual(reminder.markup(line, SLACK), '_See & more_') - self.assertEqual(reminder.markup(line, DISCORD), '_See [R&D notes]() & more_') - self.assertEqual(reminder.markup('a_b_c and _', TELEGRAM), 'a_b_c and _') # italic only when the whole line is - - def test_the_end_of_financial_year_reminder_comes_from_reminder_json(self): - """On 23 June an Australian gets the EOFY checklist from reminder.json, links and footnotes included; others do - not, and a year or a country an entry names must match.""" - entries = config.data_json('reminder.json', []) - eofy = reminder.report(TELEGRAM, datetime.date(2026, 6, 23), entries) - self.assertEqual(eofy[0], 'Finance event reminders:') - self.assertIn('💸 Realise capital gains/losses by EOFY June 30🇦🇺', eofy) - dgr = 'DGR-registered' - self.assertIn(f'* Organisation must be {dgr}', eofy) - self.prefs(country_code='US') - self.assertEqual(reminder.report(TELEGRAM, datetime.date(2026, 6, 23), entries), []) - dated = [{'month_day': '06-23', 'year': '2027', 'message': 'next year'}, - {'month_day': '06-23', 'message': ['a', 'b']}] - self.assertEqual(reminder.report(SLACK, datetime.date(2026, 6, 23), dated), - ['*Finance event reminders:*', 'a', 'b']) - - def test_the_birthday(self): - """On 18 August finbot counts its years.""" - self.assertEqual(reminder.report(SLACK, datetime.date(2026, 8, 18), [])[1:], ["It's my 4th birthday! 🥳"]) - - class Podcasts(FinbotCase): def test_format_helpers(self): """Durations in any feed format, how long ago, and zoneless dates taken as UTC; nonsense gives nothing.""" diff --git a/tests/test_schedule.py b/tests/test_schedule.py index ba60839f..650cbed3 100644 --- a/tests/test_schedule.py +++ b/tests/test_schedule.py @@ -1,77 +1,64 @@ -"""Jobs as .jobs add takes them (lib/jobs.py): their lines, and each report's job(argv); and the one-time migration -(lib/migrate.py) of the old schedule.cron, and of the reports' records, to an upgrade's home chats.""" -import contextlib, hashlib, io, os, unittest +"""What a job may be (lib/jobs.py): an action of the fixed table, never a command, with the arguments its report's job() +takes, as .jobs add reads them; the default jobs (lib/catalog.py) and the messages they post; and the one-time +migration (lib/migrate.py) of the reports' records to an upgrade's home chats.""" +import contextlib, datetime, hashlib, io, os, re, unittest from pathlib import Path from unittest import mock from tests.common import * -from lib import jobs, migrate, state, watchlist, worker -from lib.reports import cal, cleanup, milestone, performance, podcasts, price, rating, trades - -LOCAL_ZONE = migrate.local_zone # the real one: the migration tests put UTC in its place - - -class JobLines(unittest.TestCase): - def test_a_job_is_a_time_then_its_reports(self): - """A job is five time fields, or an @macro, then one or more reports with their arguments, separated by ";"; - it is on, in UTC, and its reports are kept as written.""" - job = jobs.parse_job('28 21 * * Sat cal earnings; cal ex-dividend;price 7 ; performance 7; rating') - self.assertEqual((job.when, job.zone.key, job.on, job.chat), ('28 21 * * Sat', 'UTC', True, '')) - self.assertEqual([label for label, _, _ in job.commands], - ['cal earnings', 'cal ex-dividend', 'price 7', 'performance 7', 'rating']) - self.assertEqual(job.text, 'cal earnings; cal ex-dividend;price 7 ; performance 7; rating') - self.assertEqual(job.line, '28 21 * * Sat ' + job.text) - self.assertEqual(jobs.parse_job('@daily reminder', 'telegram:1').when, '@daily') - - def test_a_job_may_name_its_zone_first(self): - """A job may name its zone, as crontab writes it, to follow that market's clocks; its line keeps it.""" - job = jobs.parse_job('CRON_TZ=America/New_York 15 13 * * Mon-Fri price midsession') - self.assertEqual((job.zone.key, job.when, job.text), - ('America/New_York', '15 13 * * Mon-Fri', 'price midsession')) - self.assertEqual(job.line, 'CRON_TZ=America/New_York 15 13 * * Mon-Fri price midsession') - for zone in ('Nowhere/Land', '', '../etc/passwd'): - with self.subTest(zone=zone), self.assertRaisesRegex(jobs.ScheduleError, 'no time zone'): - jobs.parse_job(f'CRON_TZ={zone} 0 1 * * * trades') - - def test_quoted_arguments_may_hold_anything(self): - """An argument in quotes may hold anything, a ";" included; commands after a ";" outside quotes still split.""" - url = 'https://example.com/feed?a=1&b=2;c>d $x |y' - for quoted in (f'"{url}"', f"'{url}'"): - job = jobs.parse_job(f'5 4 * * * podcasts {quoted}; shorts') - self.assertEqual([(label.split()[0], kwargs) for label, _, kwargs in job.commands], - [('podcasts', {'feed_url': url}), ('shorts', {})]) - with self.assertRaisesRegex(jobs.ScheduleError, 'No closing quotation'): - jobs.parse_job('5 4 * * * podcasts "https://example.com') - - def test_shell_syntax_is_only_words_that_no_report_takes(self): - """Nothing runs a shell: pipes, redirects, $() and comments are mere words, which no report takes.""" - for line in ('0 1 * * * trades | tee x', '0 1 * * * trades > /dev/null', '0 1 * * * trades $(date)', - '0 1 * * * trades && shorts', '0 1 * * * trades # a note'): - with self.subTest(line=line), self.assertRaisesRegex(jobs.ScheduleError, 'trades takes no arguments'): - jobs.parse_job(line) - - def test_a_time_that_cannot_be_or_never_comes(self): - """A time that cannot be (minute 61, month 13, @bogus) or never comes (30 February) is ImpossibleTime, and so is - @reboot; 29 February does come.""" - for when, why in (('0 0 30 2 *', 'never comes'), ('0 9 31 4,6,9,11 *', 'never comes'), - ('61 1 * * *', 'is no time'), ('0 0 * 13 *', 'is no time'), ('@bogus', 'is no time'), - ('@reboot', 'the jobs start with the bot')): - with self.subTest(when=when), self.assertRaisesRegex(jobs.ImpossibleTime, why): - jobs.parse_job(f'{when} trades') - self.assertEqual(jobs.parse_job('0 0 29 2 * trades').when, '0 0 29 2 *') - - def test_what_is_no_report(self): - """A job runs reports by name: not a script's file, not a system job's (except as one), not a report that is - not there; with arguments it takes, and with at least one report.""" - for line, why in (('0 1 * * * nothere', "no report 'nothere'"), - ('0 1 * * * trades.py', "no report 'trades.py'"), - ('0 1 * * * ./trades', "no report './trades'"), ('0 1 * * * yahoo_warm', "no report 'yahoo_warm'"), - ('0 1 * * * cleanup 30', "no report 'cleanup'"), ('0 1 * * * trades; nothere', "no report 'nothere'"), - ('0 1 * * * price soon', 'price takes midsession|interday|premarket|close|DAYS'), - ('0 1 * * * ;', 'no report: send .jobs add'), ('0 1 * * *', 'a job is five time fields'), - ('trades', 'a job is five time fields')): - with self.subTest(line=line), self.assertRaisesRegex(jobs.ScheduleError, why): - jobs.parse_job(line) - self.assertEqual(jobs.parse_job('51 18 * * Sat cleanup 30', system=True).commands[0][2], {'days': 30}) +from lib import catalog, jobs, migrate, state, watchlist, when, worker +from lib.reports import cal, cleanup, message, milestone, performance, podcasts, price, rating, trades + +MELBOURNE = 'Australia/Melbourne' +OCTOBER = datetime.datetime(2026, 10, 15, tzinfo=when.UTC) + + +class Actions(unittest.TestCase): + def test_a_job_runs_only_a_report_of_finbots_own(self): + """Each action a job may run is a report module of finbot's own, with its run and its parser; a chat may + schedule all but the system's.""" + for name, module in jobs.ACTIONS.items(): + with self.subTest(name=name): + self.assertEqual(module.__name__, f'lib.reports.{name}') + self.assertTrue(callable(module.run) and callable(module.job)) + self.assertEqual(jobs.chat_actions(), ['cal', 'message', 'milestone', 'performance', 'podcasts', 'price', + 'rating', 'shorts', 'trades']) + + def test_anything_else_is_refused(self): + """A name that is no action (a shell command, a module's path, a builtin), a system job for a chat, arguments + that are not words or that its parser refuses: refused as a job is made and as .jobs add reads it.""" + daily = when.parse('daily', 'UTC') + for action, args in (('os.system', ['ls']), ('price; rm -rf /', []), ('lib.reports.price', ['close']), + ('__import__', ['os']), ('eval', ['1']), ('yahoo_warm', []), ('price', ['bogus']), ('trades', [7]), + (['trades'], []), ('price', 'close')): + with self.subTest(action=action, args=args), self.assertRaises(jobs.ScheduleError): + jobs.make_job(action, args, daily, 'telegram:1') + for text in ('os.system daily', 'price; rm -rf / daily', '../../etc/passwd daily', 'trades daily; rm -rf /', + 'message daily', 'trades.py daily'): + with self.subTest(text=text), self.assertRaises(jobs.ScheduleError): + jobs.parse_add(text, 'UTC') + self.assertEqual(jobs.make_job('yahoo_warm', [], daily).label, 'yahoo_warm') # the system's own + + def test_what_jobs_add_reads(self): + """.jobs add takes a report, what it takes and a schedule, wherever both read; performance and price cover the + days between runs where they name no period; a message takes its schedule, then its text in double quotes.""" + url = 'https://example.com/Feed.rss' + for text, action, args, stored in (('price weekly', 'price', ['7'], {'days': ['sat'], 'at': '09:00'}), + ('price close weekdays at 4:30pm New York', 'price', ['close'], + {'days': 'weekdays', 'at': '16:30', 'zone': 'America/New_York'}), + ('performance quarterly', 'performance', ['90'], {'days': {'day': 1, 'months': [1, 4, 7, 10]}, + 'at': '09:00'}), + ('performance tue-sat at 8am', 'performance', [], {'days': ['tue', 'wed', 'thu', 'fri', 'sat'], + 'at': '08:00'}), + ('price every 20 mins weekdays', 'price', ['interday'], {'days': 'weekdays', 'every': 20}), + ('cal ex-dividend every sunday', 'cal', ['ex-dividend'], {'days': ['sun'], 'at': '09:00'}), + (f'podcasts {url} hourly', 'podcasts', [url], {'days': 'daily', 'every': 60}), + ('message every 31 mar "🥳 Happy EOFY"', 'message', ['🥳 Happy EOFY'], + {'days': {'day': 31, 'months': [3]}, 'at': '08:30'}), + ('message on 28 oct 2026 at 9am "a "quoted" word\nand a line"', 'message', + ['a "quoted" word', 'and a line'], {'days': {'date': '2026-10-28'}, 'at': '09:00'})): + with self.subTest(text=text): + got = jobs.parse_add(text, MELBOURNE) + self.assertEqual((got[0], got[1], got[2].to_json()), (action, args, {'zone': MELBOURNE, **stored})) class JobArguments(unittest.TestCase): @@ -107,6 +94,13 @@ def test_podcasts(self): with self.assertRaises(ValueError): podcasts.job(bad) + def test_message(self): + """message takes its text's lines, which may hold anything but must say something, 40 lines and 3000 at most.""" + self.assertEqual(message.job(['a {0}', '', 'b']), ('message', {'lines': ['a {0}', '', 'b']})) + for bad in ([], [''], [' ', ''], ['x'] * 41, ['x' * 3001], [5]): + with self.subTest(bad=bad), self.assertRaises(ValueError): + message.job(bad) + def test_cleanup_arguments(self): """cleanup takes an optional number of days.""" self.assertEqual(cleanup.job([]), ('cleanup', {})) @@ -117,10 +111,10 @@ def test_cleanup_arguments(self): def test_plain_reports(self): """The other reports take no arguments, and say so: their ARGUMENTS, which .jobs add lists, are empty.""" - for name, module in jobs.SCRIPTS.items(): + for name, module in jobs.ACTIONS.items(): with self.subTest(name=name): self.assertIsInstance(module.ARGUMENTS, str) - if name not in ('cal', 'cleanup', 'performance', 'podcasts', 'price'): + if name not in ('cal', 'cleanup', 'message', 'performance', 'podcasts', 'price'): self.assertEqual((module.job([]), module.ARGUMENTS), ((name, {}), '')) with self.assertRaisesRegex(ValueError, 'no arguments'): module.job(['7']) @@ -131,18 +125,119 @@ def test_the_checkout_holds_only_the_bot_at_its_top(self): self.assertEqual(sorted(path.name for path in Path(ROOT).glob('*.py')), ['bot.py']) +class DefaultJobs(unittest.TestCase): + def test_every_default_job_can_be(self): + """Each default job is an action a chat may schedule, with arguments its parser takes, on a schedule that runs; + its key is its own; its name says what it reports on, never when; performance and price over a period cover the + days between their runs.""" + self.assertEqual(len(catalog.ITEMS), 35) + self.assertEqual(len({item.key for item in catalog.ITEMS}), 35) + start = datetime.datetime(2026, 1, 1, tzinfo=when.UTC) + for item in catalog.ITEMS: + with self.subTest(item=item.key): + jobs.make_job(item.action, item.args, item.schedule, 'telegram:1', item=item.key, name=item.name) + self.assertIn(item.action, jobs.chat_actions()) + self.assertIsNotNone(item.schedule.next_after(start)) + self.assertIsNone(re.search(r'\d(am|pm)|daily|weekly|monthly|quarterly|annual', item.name)) + if item.action in catalog.PERIODIC and all(arg.isdigit() for arg in item.args): + self.assertEqual(list(item.args), catalog.period_args(item.action, item.schedule)) + + def test_the_default_jobs_as_a_melbourne_chat_sees_them(self): + """The default jobs by name, each with its schedule in words, as a chat in Melbourne sees them in mid-October: + each in its market's zone, the local time beside it where the clocks differ.""" + described = [f'{item.name} · {item.schedule.describe(MELBOURNE, OCTOBER)}' for item in catalog.items('AU')] + self.assertEqual(described, ['trades · every 20 mins, Mon 9am – Sat 3pm Sydney', + 'price midsession AU · weekdays at 1:00pm Sydney', + 'price midsession US · weekdays at 12:45pm New York (3:45am local, Tue–Sat)', + 'price premarket US · weekdays at 7:15am New York (10:15pm local)', + 'price close AU · weekdays at 4:30pm Sydney', + 'price close HK · weekdays at 4:30pm Hong Kong (7:30pm local)', + 'price close UK · weekdays at 5:00pm London (3:00am local, Tue–Sat)', + 'price close US · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', + 'rating · Tue–Sat at 8:30am Sydney', 'milestone · Tue–Sat at 8:30am Sydney', + 'podcasts Money Cafe · hourly at :01, weekdays 10am–9pm Sydney', + 'podcasts Stocktake · hourly at :01, weekdays 10am–9pm Sydney', + 'cal earnings · weekly, Sundays at 8:30am Sydney', 'cal ex-dividend · weekly, Sundays at 8:30am Sydney', + 'shorts · weekly, Sundays at 8:30am Sydney', 'performance · weekly, Sundays at 8:30am Sydney', + 'price · weekly, Sundays at 8:30am Sydney', 'performance · monthly, the 2nd at 8:30am Sydney', + 'price · monthly, the 2nd at 8:30am Sydney', 'performance · quarterly, 2 Jan/Apr/Jul/Oct at 8:30am Sydney', + 'price · quarterly, 2 Jan/Apr/Jul/Oct at 8:30am Sydney', + 'performance · bi-annual, 2 Jan and 2 Jul at 8:30am Sydney', + 'price · bi-annual, 2 Jan and 2 Jul at 8:30am Sydney', 'performance · annual, 2 Jan at 8:30am Sydney', + 'price · annual, 2 Jan at 8:30am Sydney', + 'message: 🥳 Happy EOFY 🇬🇧 🇭🇰 🇮🇳 🇰🇷 🇳🇿 🇯🇵 🇿🇦 · annual, 31 Mar at 8:30am Sydney', + 'message: SPCX IPO anniversary · annual, 12 Jun at 8:30am Sydney', + 'message: 💸 Realise capital gains/losses by EOFY June 30 (AU) · annual, 23 Jun at 8:30am Sydney', + 'message: 🥳 Happy EOFY 🇦🇺 🇧🇩 🇪🇬 🇪🇹 🇰🇪 🇳🇵 🇵🇰 · annual, 30 Jun at 8:30am Sydney', + "message: ✍️ Submit 'Notice of Intent to Claim' for any post-tax … (AU) · annual, 7 Jul at 8:30am Sydney", + "message: finbot's birthday · annual, 18 Aug at 8:30am Sydney", + 'message: 😓 Self-service individual tax returns are due Oct 31 (AU) · annual, 25 Oct at 8:30am Sydney', + 'message: ubank draw 6/6 … (AU) · once, 28 Oct 2026 at 8:30am Sydney', + 'message: 😰 Self-service individual tax returns are due today (AU) · annual, 31 Oct at 8:30am Sydney', + 'message: SPCX 5th mensiversary. End of (statistically) most … · once, 12 Nov 2026 at 8:30am Sydney']) + + def test_a_message_for_one_country_only_there_with_its_flag(self): + """A message for one country is a default job only where it is the country_code preference, with its flag.""" + au, nz = ({item.key for item in catalog.items(country)} for country in ('AU', 'NZ')) + self.assertEqual(sorted(au - nz), ['message-eofy-checklist', 'message-intent-to-claim', 'message-tax-due-soon', + 'message-tax-due-today', 'message-ubank-draw-6']) + self.assertLess(nz, au) + checklist = next(item for item in catalog.ITEMS if item.key == 'message-eofy-checklist') + self.assertEqual(checklist.args[0], '💸 Realise capital gains/losses by EOFY June 30🇦🇺') + self.assertFalse(any('{flag}' in line for item in catalog.ITEMS for line in item.args)) + + def test_a_job_that_is_a_default_one(self): + """A job is a default one where its action, arguments, days and times are, in a zone that keeps the same clock + (Melbourne's and Sydney's), and not in another (Brisbane's has no summer time).""" + def matching(text): + """The default job that .jobs add's text is in Melbourne, or None.""" + item = catalog.matching(*jobs.parse_add(text, MELBOURNE), 'AU') + return item and item.key + self.assertEqual(matching('trades every 20 mins from mon 9am to sat 3pm'), 'trades') + self.assertIsNone(matching('trades every 20 mins from mon 9am to sat 3pm Brisbane')) + self.assertIsNone(matching('trades every 20 mins weekdays')) + self.assertEqual(matching('price close weekdays at 4:30pm New York'), 'price-close-us') + self.assertEqual(matching('price every sunday at 8:30am'), 'price-weekly') + + +class Messages(FinbotCase): + def test_the_end_of_financial_year_checklist(self): + """On 23 June an Australian chat gets the EOFY checklist, its links and footnotes in its network's markup.""" + checklist = next(item for item in catalog.ITEMS if item.key == 'message-eofy-checklist') + message.run(scheduled(), lines=list(checklist.args)) + lines = self.sent[0][1] + self.assertEqual((len(lines), lines[0]), (12, '💸 Realise capital gains/losses by EOFY June 30🇦🇺')) + self.assertEqual(lines[8], '* Organisation must be ' + 'DGR-registered') + + def test_the_birthday_counts_the_years(self): + """finbot's birthday says how old it is that day, by the timezone preference's clock.""" + birthday = next(item for item in catalog.ITEMS if item.key == 'message-birthday') + for today, said in ((datetime.date(2026, 8, 18), "It's my 4th birthday! 🥳"), + (datetime.date(2026, 8, 17), "It's my 3rd birthday! 🥳")): + with self.subTest(today=today): + self.sent.clear() + self.patch(message, 'today', lambda: today) + message.run(scheduled('slack:C9'), lines=list(birthday.args)) + self.assertEqual(self.sent, [('text', [said])]) + + def test_a_message_is_data(self): + """A message's text is posted as it is, made safe for its network, never formatted: braces stay braces.""" + message.run(scheduled(), lines=['{0} {years since x} & {__class__}']) + self.assertEqual(self.sent, [('text', ['<b>{0} {years since x}</b> & {__class__}'])]) + + class Migration(FinbotCase): """As an upgraded finbot first starts with jobs per chat, the home chats that the retired keys still name in its environment get what was kept for all chats, once.""" def setUp(self): - """Two old home chats, Telegram's and Slack's, in the environment; the old schedule ran in UTC.""" + """Two old home chats, Telegram's and Slack's, in the environment.""" super().setUp() self.env(slackBotToken='xoxb-1') homes = mock.patch.dict(os.environ, {'telegramChatID': '1', 'slackChannelID': 'C1'}) homes.start() self.addCleanup(homes.stop) - self.patch(migrate, 'local_zone', lambda: jobs.UTC) def migrate(self): """Run the migration, as finbot does as it starts: what it logs.""" @@ -150,60 +245,12 @@ def migrate(self): migrate.run() return log.getvalue() - def test_the_old_schedule_becomes_each_home_chats_jobs_once(self): - """Each home chat gets the old schedule's jobs (the data dir's copy), pasted crontab syntax read as cron read - it, each in the zone it ran in, without the system jobs; a line that cannot be a job is logged and left out. - Then the old file is read no more.""" - url = 'https://example.com/feed?a=1&b=2' - Path(config.env.data_dir, 'schedule.cron').write_text('MAILTO=me\n' - '*/20 * * * Mon-Fri ~/finbot/trades.py > ~/finbot.trades 2>&1\n' - '28 21 * * Sat { ./cal.py earnings; ./price.py 7 ;} &> /dev/null # weekly\n' - '@daily /opt/finbot/reminder.py\n' - 'CRON_TZ=Australia/Melbourne\n' - '14,44 * * * * sharesight_warm.py\n' - f'01 * * * * podcasts.py "{url}"; cleanup.py 30\n' - '0 1 * * * ls | wc -l\n' - '0 9 31 6 * price.py close\n' - '0 2 * * * bin/run trades.py\n') - log = self.migrate() - jobs_each = [{'when': '*/20 * * * Mon-Fri', 'reports': 'trades', 'zone': 'UTC', 'on': True}, - {'when': '28 21 * * Sat', 'reports': 'cal earnings; price 7', 'zone': 'UTC', 'on': True}, - {'when': '@daily', 'reports': 'reminder', 'zone': 'UTC', 'on': True}, - {'when': '01 * * * *', 'reports': f"podcasts '{url}'", 'zone': 'Australia/Melbourne', 'on': True}] - self.assertEqual(state.load(jobs.SCHEDULE_FILE), {'telegram:1': jobs_each, 'slack:C1': jobs_each}) - path = config.env.data_dir + '/schedule.cron' - self.assertEqual([line for line in log.splitlines() if 'left out' in line], [ - f'schedule: {path} line 8 left out: only finbot scripts can be scheduled, not shell commands', - f"schedule: {path} line 9 left out: '0 9 31 6 *' never comes: no date matches it", - f"schedule: {path} line 10 left out: no report 'run': send .jobs add for the reports and what each takes"]) - self.assertIn(f"schedule: the 4 jobs of {path} are now each home chat's (telegram:1, slack:C1)", log) - s = jobs.Scheduler() - with contextlib.redirect_stderr(io.StringIO()): - s.load() - self.assertEqual([job.line for job in s.chat_jobs('slack:C1')][-1], f"CRON_TZ=Australia/Melbourne 01 * * * *" - f" podcasts '{url}'") - Path(config.env.data_dir, 'schedule.cron').write_text('0 3 * * * shorts.py\n') # read no more - self.assertEqual(self.migrate(), '') - self.assertEqual(state.load(jobs.SCHEDULE_FILE), {'telegram:1': jobs_each, 'slack:C1': jobs_each}) - - def test_the_shipped_schedule_migrates_whole(self): - """Without a copy in the data dir, the home chats get the shipped schedule's jobs, every line of it but the - system jobs', the weekly line's five reports among them.""" - self.assertNotIn('left out', self.migrate()) - entries = state.load(jobs.SCHEDULE_FILE)['telegram:1'] - self.assertEqual(len(entries), 8) - self.assertIn({'when': '28 21 * * Sat', 'reports': 'cal earnings; cal ex-dividend; price 7; performance 7;' - ' rating', 'zone': 'UTC', 'on': True}, entries) - self.assertFalse(any(name in entry['reports'] for entry in entries for name in jobs.SYSTEM_SCRIPTS)) - - def test_an_empty_schedule_migrates_to_no_jobs_and_is_done(self): - """A schedule with no jobs gives the home chats none, and the migration is still done: it is not tried again.""" - Path(config.env.data_dir, 'schedule.cron').write_text('# nothing\n14,44 * * * * sharesight_warm.py\n') - self.migrate() - self.assertEqual(state.load(jobs.SCHEDULE_FILE), {}) - Path(config.env.data_dir, 'schedule.cron').write_text('0 3 * * * shorts.py\n') - self.assertEqual(self.migrate(), '') - self.assertEqual(state.load(jobs.SCHEDULE_FILE), {}) + def test_each_home_chat_starts_with_the_default_jobs(self): + """Each home chat becomes a chat finbot knows, with the default jobs switched on, unless it is known already.""" + state.save(jobs.JOBS_FILE, {'slack:C1': {'off': ['trades']}}) + self.assertIn('schedule: 1 old home chat start with the default jobs', self.migrate()) + self.assertEqual(state.load(jobs.JOBS_FILE), {'slack:C1': {'off': ['trades']}, 'telegram:1': {}}) + self.assertNotIn('schedule:', self.migrate()) def test_with_no_home_chat_it_waits(self): """With no old home chat in the environment (a new install) there is nowhere to put the jobs and records, so @@ -212,7 +259,7 @@ def test_with_no_home_chat_it_waits(self): with mock.patch.dict(os.environ, {'telegramChatID': '', 'slackChannelID': 'C1'}): self.env(slackBotToken='') self.assertEqual(self.migrate(), '') - self.assertFalse(os.path.exists(state.path(jobs.SCHEDULE_FILE))) + self.assertFalse(os.path.exists(state.path(jobs.JOBS_FILE))) self.assertEqual(state.load(trades.STATE_FILE), ['t1']) def test_the_watchlist_plans_and_records_become_each_home_chats_once(self): @@ -237,16 +284,6 @@ def test_the_watchlist_plans_and_records_become_each_home_chats_once(self): self.assertEqual(Path(state.path(rating.STATE_FILE)).read_text(), '{"NVDA": ["upgr') self.assertIn(f'state: {rating.STATE_FILE} left as it is, unreadable', log) - def test_pasted_redirects_braces_and_paths_are_dropped(self): - """Output redirects, braces, paths, a .py and trailing comments are dropped from an old line's command.""" - for command in ('trades.py > /dev/null', 'trades.py >/dev/null 2>&1', 'trades.py &> /dev/null', - 'trades.py >> ~/x 2>&1', '{ trades.py ;} > ~/x', '/opt/finbot/trades.py # note (x)', 'FOO=1 trades'): - with self.subTest(command=command): - lines = migrate.legacy_lines(f'0 1 * * * {command}\n', 'schedule.cron') - self.assertEqual([commands for _, _, _, commands in lines], [[('trades', [])]]) - - def test_the_local_zone_follows_tz(self): - """The old schedule ran in the server's zone, which TZ sets: UTC by that name when it is UTC by another.""" - for tz, zone in (('Australia/Melbourne', 'Australia/Melbourne'), ('Etc/UTC', 'UTC'), (':UTC', 'UTC')): - with self.subTest(tz=tz), mock.patch.dict(os.environ, {'TZ': tz}): - self.assertEqual(LOCAL_ZONE().key, zone) + +if __name__ == '__main__': + unittest.main() diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index 6cd94023..c2b94bb7 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -1,15 +1,17 @@ -"""The scheduler (lib/jobs.py): each chat's jobs and the system jobs firing on a fake clock, across the clocks' changes; -.jobs in a chat, and its buttons; each command run in the bot's process, past its deadline too.""" +"""The scheduler (lib/jobs.py): each chat's jobs (the default ones and those it added) and the system jobs running on a +fake clock, across the clocks' changes; .jobs in a chat, its menu of buttons and the steps of Add; each job run in the +bot's process, past its deadline too.""" import contextlib, datetime, io, os, subprocess, sys, tempfile, time, unittest import zoneinfo from pathlib import Path from unittest import mock from tests.common import * -from lib import jobs -from lib.reports import cleanup, podcasts, price, sharesight_warm, yahoo_warm +from lib import catalog, jobs, when +from lib.reports import cleanup, message, podcasts, price, sharesight_warm, yahoo_warm from tests import test_sending PREFETCH_HISTORY = yahoo.prefetch_history # FinbotCase replaces it; the warm job's test puts it back +SYSTEM = jobs.SYSTEM # SchedulerBase takes the system jobs out; their own tests put them back def at(day, hour, minute, second=0): @@ -137,25 +139,32 @@ def down(): CHAT, OTHER = 'telegram:55', 'slack:C1' # two chats, by their keys +ITEMS = catalog.ITEMS # the default jobs: SchedulerBase takes them out, DefaultJobs puts them back +FEW = (catalog.Item('rating', 'rating', 'rating', (), catalog.at(catalog.TUE_SAT)), # two default jobs, for the menu + catalog.Item('price-close-us', 'price close US', 'price', ('close',), catalog.at('weekdays', '16:30', + catalog.NEW_YORK))) -def entry(line, on=True): - """A job as SCHEDULE_FILE stores it, from its line as .jobs add takes it.""" - job = jobs.parse_job(line) - return {'when': job.when, 'reports': job.text, 'zone': job.zone.key, 'on': on} +def entry(text, on=True, zone='UTC'): + """A job as JOBS_FILE stores one a chat added, from what .jobs add takes ('trades every 20 mins weekdays'), in UTC + unless it names a zone (or another is given).""" + action, args, schedule = jobs.parse_add(text, zone) + return jobs.make_job(action, args, schedule, 'telegram:1', on).stored() class SchedulerBase(FinbotCase): def setUp(self): - """Chat telegram:55 runs trades every 20 minutes on weekdays; a fake clock; no system jobs (they have tests of - their own); the reports noted with the chat each posts to, not run.""" + """Chat telegram:55 with one job it added, trades every 20 minutes on weekdays in UTC, and no default jobs (they + have tests of their own); a fake clock; no system jobs (nor those); the reports noted with the chat each posts + to, not run.""" super().setUp() self.patch(jobs, 'SYSTEM', ()) - self.store(CHAT, '*/20 * * * Mon-Fri trades') + self.patch(catalog, 'ITEMS', ()) + self.store(CHAT, 'trades every 20 mins weekdays') self.clock = Clock(at(13, 21, 3)) self.ran, self.dests = [], [] - for name in ('trades', 'shorts', 'price', 'rating', 'reminder'): - self.patch(jobs.SCRIPTS[name], 'run', self.recorder(name)) + for name in ('cal', 'message', 'performance', 'price', 'rating', 'shorts', 'trades'): + self.patch(jobs.ACTIONS[name], 'run', self.recorder(name)) def recorder(self, name): """A report's run() that notes the run, and the chat it would post to (its key; None for a system job).""" @@ -165,29 +174,31 @@ def run(dest=None, **kwargs): self.dests.append(dest.key if dest else None) return run - def store(self, chat, *lines, on=True): - """Set a chat's jobs, as .jobs add would have stored them.""" - with state.edit(jobs.SCHEDULE_FILE, {}) as chats: - chats[chat] = [entry(line, on) for line in lines] + def store(self, chat, *texts, on=True, off=()): + """Set a chat's jobs as .jobs would have stored them: those it added (each as .jobs add takes it, in UTC), and + the default ones it switched off (their keys).""" + with state.edit(jobs.JOBS_FILE, {}) as chats: + chats[chat] = {'jobs': [entry(text, on) for text in texts], **({'off': list(off)} if off else {})} def scheduler(self, catchup=5): - """A loaded scheduler on the fake clock that runs commands in this process, at once.""" + """A loaded scheduler on the fake clock that runs jobs in this process, at once.""" s = jobs.Scheduler(catchup=catchup, clock=self.clock, sleep=lambda seconds: None, spawn=lambda f, *a: f(*a)) with contextlib.redirect_stderr(io.StringIO()): s.load() return s -class WarmNow(SchedulerBase): +class WarmNow(FinbotCase): def test_the_warm_jobs_run_through_the_scheduler(self): """warm_now runs both warm jobs through the scheduler, which does not start one that is running already.""" warmed = [] - self.patch(jobs.SCRIPTS['sharesight_warm'], 'run', lambda: warmed.append('sharesight')) - self.patch(jobs.SCRIPTS['yahoo_warm'], 'run', lambda: warmed.append('yahoo')) - s = self.scheduler() + self.patch(sharesight_warm, 'run', lambda: warmed.append('sharesight')) + self.patch(yahoo_warm, 'run', lambda: warmed.append('yahoo')) + s = jobs.Scheduler(clock=Clock(at(13, 21, 3))) with contextlib.redirect_stderr(io.StringIO()) as log: + s.load() jobs.warm_now(s) - s.busy.add(('', 'yahoo_warm')) # on its schedule + s.busy.add(next(job.key for job in s.system_jobs() if job.action == 'yahoo_warm')) # on its schedule jobs.warm_now(s) self.assertEqual(warmed, ['sharesight', 'yahoo', 'sharesight']) self.assertIn('cache: filling it again: sharesight_warm, yahoo_warm', log.getvalue()) @@ -209,21 +220,28 @@ def run(label, module, kwargs, dest=None): class SystemJobs(FinbotCase): def test_the_system_jobs_are_fixed_and_no_chats(self): - """The system jobs run on a schedule of their own, the same for every install: each warm job within its + """The system jobs run on a schedule of their own, in UTC, the same for every install: each warm job within its cache's lifetime (Sharesight's every 30 minutes, cached 40; Yahoo's every 8 hours, cached 12), and cleanup - weekly. A chat can neither add one nor see it among its own jobs.""" + weekly. A chat can neither add one nor see it among its own jobs; .jobs system lists them.""" s = jobs.Scheduler() with contextlib.redirect_stderr(io.StringIO()): s.load() - system = {job.text: job.when for job in s.system_jobs()} - self.assertEqual(system, {'sharesight_warm': '14,44 * * * *', 'yahoo_warm': '7 */8 * * *', - 'cleanup 30': '51 18 * * Sat'}) + self.assertEqual({job.label: job.schedule.describe('Australia/Melbourne') for job in s.system_jobs()}, + {'sharesight_warm': 'every 30 mins from :14 UTC', 'yahoo_warm': 'every 8 hours at :07 UTC', + 'cleanup 30': 'weekly, Saturdays at 18:51 UTC'}) self.assertLess(30 * 60, config.prefs.performance_cache_seconds) self.assertLess(8 * 3600, config.prefs.cache_seconds) self.assertEqual(s.chat_jobs(''), s.system_jobs()) - for name in jobs.SYSTEM_SCRIPTS: - with self.subTest(name=name), self.assertRaisesRegex(jobs.ScheduleError, f"no report '{name}'"): - s.add(CHAT, f'0 1 * * * {name}') + for name in jobs.SYSTEM_ACTIONS: + with self.subTest(name=name): + with self.assertRaisesRegex(jobs.ScheduleError, f"no report '{name}'"): + jobs.parse_add(f'{name} daily', 'UTC') + with self.assertRaisesRegex(jobs.ScheduleError, f"no report '{name}'"): + s.add(CHAT, name, [], when.parse('daily', 'UTC')) + self.patch(jobs, 'current', s) + self.assertEqual(about.prepare_system(TELEGRAM), ['System jobs:', + 'sharesight_warm · every 30 mins from :14 UTC', 'yahoo_warm · every 8 hours at :07 UTC', + 'cleanup 30 · weekly, Saturdays at 18:51 UTC']) class SchedulerTests(SchedulerBase): @@ -246,7 +264,7 @@ def test_tick_runs_the_job_for_its_chat_and_returns_the_wait(self): """tick() runs what is due, posting to the job's chat as a scheduled run, and returns how long to sleep: up to 30 seconds, less when the next run is nearer.""" dests = [] - self.patch(jobs.SCRIPTS['trades'], 'run', lambda dest=None, **k: dests.append(dest)) + self.patch(jobs.ACTIONS['trades'], 'run', lambda dest=None, **k: dests.append(dest)) s = self.scheduler() self.clock.now = at(13, 21, 20) self.assertEqual(s.tick(), 30) # the next run is 20 minutes away: wake at most every 30 seconds @@ -256,7 +274,7 @@ def test_tick_runs_the_job_for_its_chat_and_returns_the_wait(self): def test_each_chats_jobs_post_to_that_chat(self): """Two chats' jobs at the same time each run, each posting to its own chat.""" - self.store(OTHER, '*/20 * * * Mon-Fri trades', '0 22 * * * shorts') + self.store(OTHER, 'trades every 20 mins weekdays', 'shorts daily at 22:00') s = self.scheduler() self.clock.now = at(13, 21, 20) s.tick() @@ -270,7 +288,7 @@ def test_each_chats_jobs_post_to_that_chat(self): def test_a_job_that_is_off_never_fires_nor_catches_up_once_on(self): """A job that is off does not fire; switched on, it fires at its next time, not for the runs it missed.""" - self.store(CHAT, '*/20 * * * Mon-Fri trades', on=False) + self.store(CHAT, 'trades every 20 mins weekdays', on=False) s = self.scheduler() self.clock.now = at(13, 21, 20) self.assertEqual(s.due(self.clock.now), []) @@ -284,8 +302,7 @@ def test_a_job_that_is_off_never_fires_nor_catches_up_once_on(self): def test_a_run_missed_within_the_catch_up_window_fires_once(self): """After a restart, a run missed within the catch-up window fires once; an older one is skipped.""" first = self.scheduler() - first.last = {first.chat_jobs(CHAT)[0].key: at(13, 21, 0).isoformat()} - state.save(jobs.STATE_FILE, first.last) + state.save(jobs.STATE_FILE, {first.chat_jobs(CHAT)[0].key: at(13, 21, 0).isoformat()}) self.clock.now = at(13, 21, 43) s = self.scheduler() # 21:20 is older than 5 minutes: skipped @@ -293,7 +310,7 @@ def test_a_run_missed_within_the_catch_up_window_fires_once(self): def test_a_corrupt_state_file_is_reported_and_the_schedule_starts_afresh(self): """A state file cut off mid-write is reported once, and the schedule runs as usual and writes it afresh.""" - Path(config.env.data_dir, jobs.STATE_FILE).write_text('{"telegram:55|*/20 * * * Mon-Fri|UTC|trades": "2026-1') + Path(config.env.data_dir, jobs.STATE_FILE).write_text('{"telegram:55|trades|[]|{}": "2026-1') s = self.scheduler() self.assertEqual((s.last, len(self.errors)), ({}, 1)) self.assertIn('cannot read finbot_jobs.json', self.errors[0]) @@ -316,73 +333,79 @@ def test_a_restart_just_after_a_run_does_not_repeat_it(self): again = self.scheduler() self.assertEqual(again.due(self.clock.now), []) - def test_a_failing_job_is_reported_to_its_chat_and_the_next_command_still_runs(self): - """A command that fails is reported to the chat whose job it is, and the next command of the job still runs.""" - self.store(CHAT, '*/20 * * * * trades; shorts') + def test_a_failing_job_is_reported_to_its_chat_and_the_others_still_run(self): + """A job that fails is reported to the chat whose job it is; another due at the same time still runs.""" + self.store(CHAT, 'trades every 20 mins', 'shorts every 20 mins') def boom(dest=None, **k): """Fail.""" raise RuntimeError('boom') - self.patch(jobs.SCRIPTS['trades'], 'run', boom) + self.patch(jobs.ACTIONS['trades'], 'run', boom) reported = [] self.patch(delivery, 'report_error', lambda e, dest=None, context=None: reported.append((context, dest.key))) s = self.scheduler() - self.assertEqual(s.execute(s.chat_jobs(CHAT)[0].commands, CHAT), 'failed') + self.clock.now = at(13, 21, 20) + s.tick() self.assertEqual(reported, [('trades', CHAT)]) self.assertEqual(self.ran, [('shorts', {})]) + self.assertEqual(s.execute(s.chat_jobs(CHAT)[0]), 'failed') def test_system_exit_inside_a_job_is_reported_not_raised(self): """A report calling sys.exit is reported like any failure and does not stop the scheduler.""" def quit_(dest=None, **k): """Exit, as a report may.""" raise SystemExit(1) - self.patch(jobs.SCRIPTS['trades'], 'run', quit_) + self.patch(jobs.ACTIONS['trades'], 'run', quit_) s = self.scheduler() s.run(s.chat_jobs(CHAT)[0]) self.assertEqual(len(self.errors), 1) - def test_a_command_past_its_deadline_is_reported_once_and_left_to_finish(self): - """A command still running at its deadline is reported as late, once, and runs on to its end (a thread cannot - be stopped); meanwhile it stays marked running, so a second run of it is turned away; then the rest of its job - runs.""" + def test_a_job_past_its_deadline_is_reported_once_and_left_to_finish(self): + """A job still running at its deadline is reported as late, once, and runs on to its end (a thread cannot be + stopped); meanwhile it stays marked running, so a second run of it is turned away.""" self.patch(jobs, 'JOB_TIMEOUT', 0.1) - self.store(CHAT, '*/20 * * * * trades; shorts') s = self.scheduler() second = [] def slow(dest=None, **k): - """Take three deadlines, and try a second run of this command meanwhile.""" + """Take three deadlines, and try a second run of this job meanwhile.""" time.sleep(0.3) - second.append(s.execute(s.chat_jobs(CHAT)[0].commands[:1], CHAT)) + second.append(s.execute(s.chat_jobs(CHAT)[0])) self.ran.append(('trades', k)) - self.patch(jobs.SCRIPTS['trades'], 'run', slow) + self.patch(jobs.ACTIONS['trades'], 'run', slow) with contextlib.redirect_stderr(io.StringIO()): self.assertEqual(s.run(s.chat_jobs(CHAT)[0]), 'done') self.assertEqual(self.errors, ['⚠️ trades: TimeoutError: still running after 0.00166667 minutes, and left to' ' finish']) self.assertEqual(second, ['already running']) - self.assertEqual(self.ran, [('trades', {}), ('shorts', {})]) + self.assertEqual(self.ran, [('trades', {})]) self.assertEqual(s.busy, set()) - def test_each_command_of_a_job_has_a_deadline_of_its_own(self): - """The deadline is per command (it once covered the whole line): two that each take most of it are not late.""" - self.patch(jobs, 'JOB_TIMEOUT', 0.4) - self.store(CHAT, '*/20 * * * * trades; shorts') - self.patch(jobs.SCRIPTS['trades'], 'run', lambda dest=None, **k: time.sleep(0.25)) - self.patch(jobs.SCRIPTS['shorts'], 'run', lambda dest=None, **k: time.sleep(0.25)) - s = self.scheduler() - with contextlib.redirect_stderr(io.StringIO()): - self.assertEqual(s.run(s.chat_jobs(CHAT)[0]), 'done') - time.sleep(0.3) # past where a deadline of the whole job would have struck + def test_a_stored_job_that_cannot_be_is_logged_left_out_and_never_run(self): + """A job is only ever an action of the fixed table with arguments its parser takes: a stored one that is + anything else (a command, a module's path, a system job, arguments its action refuses, a schedule that is + none, junk) is logged and left out, never run, and the chat's other jobs load.""" + once = {'at': '01:00', 'zone': 'UTC'} + bad = [{'action': 'os.system', 'args': ['rm -rf /'], 'schedule': once}, + {'action': 'price; rm -rf /', 'args': [], 'schedule': once}, + {'action': 'lib.reports.price', 'args': ['close'], 'schedule': once}, + {'action': '__import__', 'args': ['os'], 'schedule': once}, + {'action': 'eval', 'args': ['1'], 'schedule': once}, {'action': 'yahoo_warm', 'args': [], 'schedule': once}, + {'action': 'price', 'args': ['bogus'], 'schedule': once}, + {'action': 'trades', 'args': [], 'schedule': {'at': '25:00'}}, + {'action': 'trades', 'args': [7], 'schedule': once}, {'action': ['trades'], 'args': [], 'schedule': once}, + 'trades', {'action': 'trades', 'schedule': once, 'on': 'yes'}, + {'action': 'trades', 'args': [], 'schedule': {'at': '01:00', 'zone': '../../etc/passwd'}}] + with state.edit(jobs.JOBS_FILE, {}) as chats: + chats[CHAT]['jobs'] += bad + with contextlib.redirect_stderr(io.StringIO()) as log: + s = jobs.Scheduler(clock=self.clock, spawn=lambda f, *a: f(*a)) + s.load() + self.assertEqual([job.name for job in s.chat_jobs(CHAT)], ['trades']) + self.assertEqual(log.getvalue().count('schedule: a job of telegram:55 left out'), len(bad)) self.assertEqual(self.errors, []) - - def test_a_stored_job_that_cannot_be_is_reported_and_the_rest_run(self): - """A stored job that cannot be any more (a report since retired, say) is reported and left out; the chat's other - jobs load.""" - with state.edit(jobs.SCHEDULE_FILE, {}) as chats: - chats[CHAT].append({'when': '0 1 * * *', 'reports': 'nothere', 'zone': 'UTC', 'on': True}) - s = self.scheduler() - self.assertEqual([job.text for job in s.chat_jobs(CHAT)], ['trades']) - self.assertEqual(len(self.errors), 1) - self.assertIn("a job of telegram:55 left out: no report 'nothere'", self.errors[0]) + for day in range(14, 21): + self.clock.now = at(day, 1, 30) + s.tick() + self.assertEqual({name for name, _ in self.ran}, {'trades'}) def test_run_forever_survives_an_error(self): """run_forever reports a failed tick and carries on.""" @@ -401,73 +424,49 @@ def sleep(seconds): self.assertTrue(any('tick failed' in e for e in self.errors)) -class DaylightSaving(FinbotCase): - """Melbourne's clocks go back an hour at 03:00 on 5 April 2026 and forward at 02:00 on 4 October 2026. The fire - times must be right on any croniter: the distro packages (up to 6.0) got zoneinfo's changes wrong.""" - MEL = zoneinfo.ZoneInfo('Australia/Melbourne') - - def fires(self, expr, start, end): - """When a Melbourne job fires between start and end, as Melbourne times ('04 09:00 AEDT').""" - job = jobs.parse_job(f'CRON_TZ=Australia/Melbourne {expr} trades') - s = jobs.Scheduler() - times, t = [], start - while (t := s.next_fire(job, t)) <= end: - # by way of UTC: the real local time - times.append(t.astimezone(UTC).astimezone(self.MEL).strftime('%d %H:%M %Z')) - return times - - def mel(self, *args, **kw): - """A Melbourne time.""" - return datetime.datetime(*args, tzinfo=self.MEL, **kw) - - def test_a_daily_job_fires_once_at_its_time(self): - """A daily job fires at its local time each day across both changes.""" - self.assertEqual(self.fires('0 9 * * *', self.mel(2026, 10, 2), self.mel(2026, 10, 6)), - ['02 09:00 AEST', '03 09:00 AEST', '04 09:00 AEDT', '05 09:00 AEDT']) - self.assertEqual(self.fires('0 9 * * *', self.mel(2026, 4, 3), self.mel(2026, 4, 7)), - ['03 09:00 AEDT', '04 09:00 AEDT', '05 09:00 AEST', '06 09:00 AEST']) - - def test_a_skipped_or_repeated_time_fires_once(self): - """A time the clocks skip fires an hour later, and a repeated one fires once.""" - self.assertEqual(self.fires('30 2 * * *', self.mel(2026, 10, 3), self.mel(2026, 10, 6)), - ['03 02:30 AEST', '04 03:30 AEDT', '05 02:30 AEDT']) - self.assertEqual(self.fires('30 2 * * *', self.mel(2026, 4, 4), self.mel(2026, 4, 7)), - ['04 02:30 AEDT', '05 02:30 AEDT', '06 02:30 AEST']) - self.assertEqual(self.fires('*/20 * * * *', self.mel(2026, 10, 4, 1, 30), self.mel(2026, 10, 4, 3, 50)), - ['04 01:40 AEST', '04 03:00 AEDT', '04 03:20 AEDT', '04 03:40 AEDT']) - self.assertEqual(self.fires('*/20 * * * *', self.mel(2026, 4, 5, 1, 30), self.mel(2026, 4, 5, 3, 30)), - ['05 01:40 AEDT', '05 02:00 AEDT', '05 02:20 AEDT', '05 02:40 AEDT', '05 03:00 AEST', - '05 03:20 AEST']) - - def test_starting_in_the_repeated_hour_runs_nothing_twice(self): - """Starting during the repeated hour runs nothing twice.""" - second_time_round = self.mel(2026, 4, 5, 2, 10, fold=1).astimezone(UTC) - self.assertEqual(self.fires('*/20 * * * *', second_time_round, self.mel(2026, 4, 5, 3, 0, 30)), - ['05 03:00 AEST']) - self.assertEqual(self.fires('30 2 * * *', second_time_round, self.mel(2026, 4, 6, 12)), ['06 02:30 AEST']) - - def test_the_scheduler_runs_a_daily_job_once_across_the_change(self): - """The scheduler runs a chat's daily Melbourne job once a day across the change to daylight time.""" - self.patch(jobs, 'SYSTEM', ()) - with state.edit(jobs.SCHEDULE_FILE, {}) as chats: - chats[CHAT] = [entry('CRON_TZ=Australia/Melbourne 0 9 * * * trades')] - clock = Clock(self.mel(2026, 10, 3, 12).astimezone(UTC)) - s = jobs.Scheduler(clock=clock, runner=lambda *a: None) - with contextlib.redirect_stderr(io.StringIO()): - s.load() - fired = [] - while clock.now < self.mel(2026, 10, 5, 12): - fired += [t.astimezone(UTC).astimezone(self.MEL).strftime('%d %H:%M') for _, t in s.due(clock.now)] - clock.now += datetime.timedelta(minutes=10) - self.assertEqual(fired, ['04 09:00', '05 09:00']) - - -class JobsInChat(SchedulerBase): - """What a chat does with its jobs: .jobs lists them; run, toggle, add and del; their buttons.""" +class DaylightSaving(SchedulerBase): + """A job runs by its zone's clock through its changes: Melbourne's clocks go forward at 02:00 on 4 October 2026, + New York's at 02:00 on 8 March 2026 and back at 02:00 on 1 November 2026.""" + + def runs(self, text, zone, start, end): + """When the scheduler runs CHAT's one job (as .jobs add takes it, in zone) between start and end, its clock + moving ten minutes at a time: each run as UTC's clock and the zone's read it ('Fri 21:30 16:30').""" + with state.edit(jobs.JOBS_FILE, {}) as chats: + chats[CHAT] = {'jobs': [entry(text, zone=zone)]} + self.clock.now = start + s, fired = self.scheduler(), [] + while self.clock.now < end: + fired += [f"{t.astimezone(UTC):%a %H:%M} {t.astimezone(zoneinfo.ZoneInfo(zone)):%H:%M}" + for _, t in s.due(self.clock.now)] + self.clock.now += datetime.timedelta(minutes=10) + return fired + + def test_a_daily_job_runs_once_a_day_at_its_time_across_the_change(self): + """A Melbourne job at 9am runs once a day at 9am Melbourne, before and after its clocks go forward.""" + mel = zoneinfo.ZoneInfo('Australia/Melbourne') + self.assertEqual(self.runs('trades daily at 9am', 'Australia/Melbourne', datetime.datetime(2026, 10, 2, 12, + tzinfo=mel), datetime.datetime(2026, 10, 5, 12, tzinfo=mel)), ['Fri 23:00 09:00', 'Sat 22:00 09:00', + 'Sun 22:00 09:00']) + + def test_a_new_york_job_keeps_new_yorks_clock_in_march_and_november(self): + """A job stored in New York's zone runs at 4:30pm there on both sides of each change: 21:30 UTC, then 20:30.""" + ny = zoneinfo.ZoneInfo('America/New_York') + self.assertEqual(self.runs('price close weekdays at 4:30pm New York', 'America/New_York', + datetime.datetime(2026, 3, 5, 12, tzinfo=ny), datetime.datetime(2026, 3, 11, tzinfo=ny)), + ['Thu 21:30 16:30', 'Fri 21:30 16:30', 'Mon 20:30 16:30', 'Tue 20:30 16:30']) + self.assertEqual(self.runs('price close weekdays at 4:30pm New York', 'America/New_York', + datetime.datetime(2026, 10, 29, 12, tzinfo=ny), datetime.datetime(2026, 11, 4, tzinfo=ny)), + ['Thu 20:30 16:30', 'Fri 20:30 16:30', 'Mon 21:30 16:30', 'Tue 21:30 16:30']) + + +class ChatBase(SchedulerBase): + """A chat's .jobs: the scheduler running, two default jobs (FEW), Telegram's toasts and edits noted, and the + replies with their buttons.""" def setUp(self): - """Chat telegram:55 with its one job, the scheduler running, Telegram's toasts and edits noted.""" + """Chat telegram:55 with the trades job it added, the scheduler running, and what is posted noted.""" super().setUp() + self.patch(catalog, 'ITEMS', FEW) self.s = self.scheduler() self.patch(jobs, 'current', self.s) self.toasts, self.edits = [], [] @@ -486,135 +485,165 @@ def say(self, text, chat='55'): self.assertEqual(self.errors, [], text) return self.replies[-1][1] if self.replies else None + def press(self, data): + """Press a button under the bot's message 9 in chat 55: (the edited text's lines, the buttons' labels by row, + and their data by label).""" + self.edits.clear() + with contextlib.redirect_stderr(io.StringIO()): + self.press_button(data) + self.assertEqual(self.errors, [], data) + if not self.edits: + return None, None, None + _, _, text, markup = self.edits[-1] + return text.split('\n'), [[button.label for button in row] for row in markup], {button.label: button.data + for row in markup for button in row} + + def crc(self, chat=CHAT): + """The checksum of a chat's list, as its menu's buttons carry it.""" + return jobs.list_checksum(self.s.chat_jobs(chat)) + + +class JobsInChat(ChatBase): + """What a chat types: .jobs lists its jobs; run, toggle, add and del; next and system.""" + def test_add_list_and_del(self): - """A job is added, listed with its number, line, state and next run (in UTC and Melbourne), and deleted.""" - added = self.say('.jobs add 29 21 * * Mon-Fri price interday') - self.assertEqual(added, ['2. 29 21 * * Mon-Fri price interday: on, next Tue 21:29 UTC (Wed 08:29 AEDT)', + """A job is added, listed under its chat's default jobs with its number, its switch, * till it runs, its name + and its schedule in words, and deleted; a default job is not.""" + self.assertEqual(self.say('.jobs add shorts daily at 10pm'), ['4. ✅* shorts · daily at 10:00pm', 'Added: .jobs lists them all']) - listed = self.say('.jobs') - self.assertEqual(listed[:3], ['Jobs here:', '1. */20 * * * Mon-Fri trades: on, next Tue 21:20 UTC (Wed' - ' 08:20 AEDT)', '2. 29 21 * * Mon-Fri price interday: on, next Tue 21:29 UTC (Wed 08:29 AEDT)']) - self.assertEqual(listed[-4:], ['.jobs run [NUMBER]: run one now', '.jobs toggle [NUMBER]: switch one on or off', - '.jobs add [CRON_TZ=ZONE] TIME REPORT: add one (.jobs add lists the reports)', - '.jobs del NUMBER: delete one']) - self.assertIsNone(self.replies[-1][2]) # .jobs alone has no buttons - self.assertEqual(self.say('.jobs del 1'), ['Deleted 1. */20 * * * Mon-Fri trades']) - self.assertEqual(state.load(jobs.SCHEDULE_FILE), {CHAT: [entry('29 21 * * Mon-Fri price interday')]}) - self.assertEqual([job.text for job in self.s.chat_jobs(CHAT)], ['price interday']) # at once, no reload - self.assertEqual(self.say('.job del 1'), ['Deleted 1. 29 21 * * Mon-Fri price interday']) - self.assertEqual(state.load(jobs.SCHEDULE_FILE), {}) - self.assertEqual(self.say('.jobs')[0], 'No jobs here yet: add one with .jobs add') - - def test_a_job_names_its_zone_runs_several_reports_and_quotes_its_arguments(self): - """A job is in UTC unless it names its zone; it may run several reports in turn, and an argument in quotes may - hold anything.""" - url = 'https://example.com/feed?a=1&b=2;c' - self.say(f'.jobs add CRON_TZ=America/New_York 15 13 * * Mon-Fri price midsession') - self.say(f'.jobs add 5 4 * * * podcasts "{url}"; shorts') - self.say('.jobs add @daily reminder') - ny, feed, daily = self.s.chat_jobs(CHAT)[1:] - self.assertEqual((ny.zone.key, ny.line), ('America/New_York', 'CRON_TZ=America/New_York 15 13 * * Mon-Fri' - ' price midsession')) - self.assertEqual([(label.split()[0], kwargs) for label, _, kwargs in feed.commands], - [('podcasts', {'feed_url': url}), ('shorts', {})]) - self.assertEqual((daily.when, daily.zone.key), ('@daily', 'UTC')) - self.assertIn('2. CRON_TZ=America/New_York 15 13 * * Mon-Fri price midsession: on, next Wed 17:15 UTC', - self.say('.jobs')[2]) + self.assertEqual(self.say('.jobs'), ['Jobs here', '', 'Default jobs:', + '1. ✅* rating · Tue–Sat at 8:30am Sydney', + '2. ✅* price close US · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', '', 'Added here:', + '3. ✅* trades · every 20 mins, weekdays UTC', '4. ✅* shorts · daily at 10:00pm', '', '* not run yet']) + self.assertEqual([[button.label for button in row] for row in self.replies[-1][2]], + [['Enable/disable', 'Delete'], ['Add', 'Run now', 'Next']]) + self.assertEqual(self.say('.jobs del 3'), ['Deleted 3. trades · every 20 mins, weekdays UTC']) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('shorts daily at 10pm', + zone='Australia/Melbourne')]}) + self.assertEqual([job.name for job in self.s.chat_jobs(CHAT)], ['rating', 'price close US', 'shorts']) + self.assertEqual(self.say('.jobs del 1'), ['rating is a default job: switch it off instead (.jobs toggle)']) + self.assertEqual(self.say('.job del 3'), ['Deleted 3. shorts · daily at 10:00pm']) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': []}) + + def test_add_reads_english_and_the_period_follows_how_often(self): + """A job is a report, what it takes, and a schedule in English, by the timezone preference's clock unless it + names a zone; performance and price cover the days between runs where they name no period.""" + for text, line, args in (('price weekly', 'price · weekly, Saturdays at 9:00am', ['7']), + ('performance first day of each month at 9am', 'performance · monthly, the 1st at 9:00am', ['30']), + ('price daily at 5pm', 'price · daily at 5:00pm', ['interday']), + ('price 30 weekly', 'price 30 · weekly, Saturdays at 9:00am', ['30']), + ('price midsession weekdays at 12:45pm New York', + 'price midsession · weekdays at 12:45pm New York (3:45am local, Tue–Sat)', ['midsession']), + ('cal earnings every sunday at 8:30am', 'cal earnings · weekly, Sundays at 8:30am', ['earnings']), + ('trades every 20 mins weekdays from 10am to 4pm', 'trades · every 20 mins, weekdays 10am–4pm', [])): + with self.subTest(text=text): + added = self.say(f'.jobs add {text}') + self.assertEqual(added, [f'4. ✅* {line}', 'Added: .jobs lists them all']) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT]['jobs'][-1]['args'], args) + self.say('.jobs del 4') + + def test_a_message_takes_its_text_in_quotes(self): + """A message is a schedule, then its text in double quotes (curly ones too), from the first to the last, so + that quotes inside it stay; over several lines if need be; at 8:30am where it says no time. A job runs the + message action with its lines.""" + self.assertEqual(self.say('.jobs add message every 31 mar "🥳 Happy "EOFY" everyone"'), + ['4. ✅* message: 🥳 Happy "EOFY" everyone · annual, 31 Mar at 8:30am', 'Added: .jobs lists them all']) + self.say('.jobs add message on 28 oct 2026 at 9am “line one\nline [two](https://x.example)”') + stored = state.load(jobs.JOBS_FILE)[CHAT]['jobs'] + self.assertEqual([job['args'] for job in stored[1:]], [['🥳 Happy "EOFY" everyone'], + ['line one', 'line [two](https://x.example)']]) + self.assertEqual(self.say('.jobs run 5'), ['5. message: line one: done']) + self.assertEqual(self.ran, [('message', {'lines': ['line one', 'line [two](https://x.example)']})]) + self.assertIn('a message is a schedule, then its text in double quotes', + self.say('.jobs add message daily "no end')[0]) def test_add_alone_lists_the_reports_a_job_can_run(self): - """.jobs add alone lists the reports with what each takes, the system jobs' aside, and examples.""" + """.jobs add alone lists the reports with what each takes, message among them, no system job, and examples.""" lines = self.say('.jobs add') self.assertEqual(lines[0], 'Reports a job can run:') - for line in ('cal earnings|ex-dividend', 'performance [DAYS]', 'podcasts FEED_URL', - 'price midsession|interday|premarket|close|DAYS', 'trades'): + for line in ('cal earnings|ex-dividend', 'message "TEXT"', 'performance [DAYS]', 'podcasts FEED_URL', + 'price midsession|interday|premarket|close|DAYS', 'trades', '.jobs add price weekly'): self.assertIn(line, lines) - self.assertFalse(any(name in line for line in lines for name in jobs.SYSTEM_SCRIPTS)) + self.assertFalse(any(name in line for line in lines for name in jobs.SYSTEM_ACTIONS)) def test_a_job_that_cannot_be_is_refused_saying_why(self): - """A time that cannot be or never comes, an unknown report, a script's file name, a system job, a bad argument, - an unknown zone or too few fields: refused, saying why, and nothing stored.""" - for line, why in (('61 1 * * * trades', "'61 1 * * *' is no time"), - ('0 0 30 2 * trades', "'0 0 30 2 *' never comes"), ('@reboot trades', '@reboot is no time'), - ('0 1 * * * nothere', "no report 'nothere'"), ('0 1 * * * trades.py', "no report 'trades.py'"), - ('0 1 * * * yahoo_warm', "no report 'yahoo_warm'"), - ('0 1 * * * price bogus', 'price takes midsession|interday|premarket|close|DAYS'), - ('0 1 * * * trades 7', 'trades takes no arguments'), - ('CRON_TZ=Nowhere/Land 0 1 * * * trades', "no time zone 'Nowhere/Land'"), - ('0 1 * *', 'a job is five time fields')): - with self.subTest(line=line): - self.assertIn(why, self.say(f'.jobs add {line}')[0]) - self.assertEqual(len(self.s.chat_jobs(CHAT)), 1) - - def test_a_chat_has_thirty_jobs_at_most(self): - """A chat may have 30 jobs; the 31st is refused.""" + """A report that is none (or a script's file name, a system job, a command), arguments it does not take, words + that are no schedule, a zone that is none, a one-off gone by, a job here already: refused, saying why.""" + for text, why in (('nothere daily', "no report 'nothere'"), ('trades.py daily', "no report 'trades.py'"), + ('yahoo_warm daily', "no report 'yahoo_warm'"), ('os.system daily', "no report 'os.system'"), + ('price bogus weekly', 'price takes midsession|interday|premarket|close|DAYS'), + ('trades 7 daily', 'trades takes no arguments'), ('trades fortnightly', "can't read 'fortnightly'"), + ('shorts daily at 9am Atlantis', "can't read 'atlantis'"), ('trades', 'no days or times'), + ('message on 1 jan 2020 "late"', 'once, 1 Jan 2020 has gone by'), + ('trades every 20 mins weekdays UTC', 'that job is here already')): + with self.subTest(text=text): + self.assertIn(why, self.say(f'.jobs add {text}')[0]) + self.assertEqual(len(state.load(jobs.JOBS_FILE)[CHAT]['jobs']), 1) + + def test_a_chat_adds_thirty_jobs_at_most(self): + """A chat may add 30 jobs; the 31st is refused.""" for minute in range(1, 30): - self.s.add(CHAT, f'{minute} 1 * * * trades') - self.assertIn('30 jobs is the most a chat may have', self.say('.jobs add 31 1 * * * trades')[0]) - self.assertEqual(len(self.s.chat_jobs(CHAT)), 30) + self.s.add(CHAT, 'shorts', [], when.parse(f'daily at 1:{minute:02d}', 'UTC')) + self.assertIn('30 jobs is the most a chat may add', self.say('.jobs add shorts daily at 2:00')[0]) + self.assertEqual(len(self.s.chat_jobs(CHAT)), 32) + + def test_a_job_the_defaults_have_switches_that_one_on(self): + """A job typed as a default one is (in a zone that keeps its clock: Melbourne for Sydney) switches that one on, + and nothing more is stored.""" + self.say('.jobs toggle 1') + self.assertEqual(self.say('.jobs add rating tue-sat at 8:30am'), ['1. ✅* rating · Tue–Sat at 8:30am Sydney', + 'Added: .jobs lists them all']) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('trades every 20 mins weekdays')]}) def test_run_a_job_now(self): - """.jobs run N runs job N now, posting to this chat, and says how it went; a number that is none says so.""" - self.assertEqual(self.say('.jobs run 1'), ['1. trades: done']) + """.jobs run N runs job N now, posting to this chat, and says how it went; a number that is none says so; + another chat's numbers are its own.""" + self.assertEqual(self.say('.jobs run 3'), ['3. trades: done']) self.assertEqual((self.ran, self.dests), ([('trades', {})], [CHAT])) - self.assertEqual(self.say('.jobs run 2'), ['no job 2: send .jobs for the list']) - self.assertEqual(self.say('.jobs run 1', chat='56'), ['no jobs here yet: .jobs add']) # another chat's list - self.assertEqual(self.ran, [('trades', {})]) + self.assertEqual(self.say('.jobs run 9'), ['no job 9: send .jobs for the list']) + self.assertEqual(self.say('.jobs run 1', chat='56'), ['1. rating: done']) # a chat seen first has the defaults + self.assertEqual(self.dests, [CHAT, 'telegram:56']) def test_toggle_a_job_off_and_on(self): """.jobs toggle N switches job N off (it no longer fires) and on again.""" - self.assertEqual(self.say('.jobs toggle 1'), ['1. */20 * * * Mon-Fri trades: off']) + self.assertEqual(self.say('.jobs toggle 3'), ['3. ❌* trades · every 20 mins, weekdays UTC']) self.clock.now = at(13, 21, 20) self.assertEqual(self.s.due(self.clock.now), []) - self.assertEqual(self.say('.jobs toggle 1')[0][:30], '1. */20 * * * Mon-Fri trades: ') + self.assertEqual(self.say('.jobs toggle 3'), ['3. ✅* trades · every 20 mins, weekdays UTC']) self.clock.now = at(13, 21, 40) - self.assertEqual(len(self.s.due(self.clock.now)), 1) - - def test_the_run_and_toggle_lists_have_a_button_per_job(self): - """.jobs run and .jobs toggle list the jobs with a button each that runs it now or switches it.""" - self.s.add(CHAT, '0 22 * * * shorts') - for action, kind in (('run', 'j'), ('toggle', 'e')): - with self.subTest(action=action): - self.say(f'.jobs {action}') - lines, markup = self.replies[-1][1:] - tap = 'Tap one to run it now' if action == 'run' else 'Tap one to switch it on or off' - self.assertIn(tap, lines[-1]) - job = self.s.chat_jobs(CHAT)[1] - self.assertEqual(markup[0][1], buttons.Button('2. shorts', f'{kind}|2|{jobs.checksum(job.key)}')) - - def test_a_run_button_runs_its_job_and_a_toggle_button_redraws_the_list(self): - """A run button runs its job and says how it went; a toggle button switches its job and redraws the list in - place, with the buttons still there.""" - job = self.s.chat_jobs(CHAT)[0] - self.press_button(f'j|1|{jobs.checksum(job.key)}') - self.assertEqual((self.toasts, self.ran), (['Running…'], [('trades', {})])) - self.assertEqual(self.replies[-1][1], ['1. trades: done']) - self.press_button(f'e|1|{jobs.checksum(job.key)}') - self.assertEqual(self.toasts[-1], 'Switching…') - self.assertFalse(self.s.chat_jobs(CHAT)[0].on) - chat, message, text, markup = self.edits[-1] - self.assertEqual((chat, message), ('55', '9')) - self.assertIn('1. */20 * * * Mon-Fri trades: off', text) - self.assertEqual(markup[0][0].data, f'e|1|{jobs.checksum(job.key)}') # the key does not hold the switch - - def test_a_stale_or_forged_button_redraws_the_list_and_does_nothing(self): - """A button whose number or checksum is not this chat's job (an older list, another chat's, forged data) runs - and switches nothing: the list is redrawn in its place, with that kind of buttons.""" - self.store(OTHER, '*/20 * * * Mon-Fri trades') - self.s.load() - good = jobs.checksum(self.s.chat_jobs(CHAT)[0].key) - others = jobs.checksum(self.s.chat_jobs(OTHER)[0].key) - for data in ('j|2|' + good, 'j|0|' + good, 'j|1|0000', 'j|1|' + others, 'e|1|0000', 'j|1', 'j|٨|' + good, - 'j| 1|' + good, 'e|99999999999999999999|' + good): - with self.subTest(data=data): - self.edits.clear() - self.press_button(data) - self.assertEqual(self.ran, []) - self.assertTrue(self.s.chat_jobs(CHAT)[0].on) - self.assertEqual(len(self.edits), 1) - self.assertIn('1. */20 * * * Mon-Fri trades: on', self.edits[0][2]) - self.assertEqual(self.edits[0][3][0][0].data[0], data[0]) # the same kind of buttons - self.press_button('j|1|' + good) # the genuine one still works - self.assertEqual(self.ran, [('trades', {})]) + self.assertEqual([job.name for job, _ in self.s.due(self.clock.now)], ['rating', 'trades']) + + def test_a_job_is_marked_till_it_first_runs(self): + """* marks a job that has never run, on its schedule or by hand, and the list says what * means.""" + self.say('.jobs run 3') + lines = self.say('.jobs') + self.assertIn('3. ✅ trades · every 20 mins, weekdays UTC', lines) + self.assertIn('1. ✅* rating · Tue–Sat at 8:30am Sydney', lines) + self.clock.now = at(13, 21, 31) # Wednesday 8:31am in Sydney: rating has run, New York's close not yet + self.s.tick() + lines = self.say('.jobs') + self.assertIn('1. ✅ rating · Tue–Sat at 8:30am Sydney', lines) + self.assertIn('2. ✅* price close US · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', lines) + self.assertEqual(lines[-1], '* not run yet') + + def test_a_one_off_gone_by_is_deleted_as_the_list_is_shown(self): + """A one-off job whose time has gone by (run or off) is deleted as the list is next shown, never on a tick.""" + self.say('.jobs add message on 15 oct 2026 at 9am "soon"') + self.assertEqual(len(state.load(jobs.JOBS_FILE)[CHAT]['jobs']), 2) + self.clock.now = at(16, 0, 0) + self.s.tick() + self.assertEqual(len(state.load(jobs.JOBS_FILE)[CHAT]['jobs']), 2) + self.assertNotIn('soon', '\n'.join(self.say('.jobs'))) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('trades every 20 mins weekdays')]}) + + def test_next_and_system(self): + """.jobs next lists the next runs in order, a repeat's runs on a day as one; .jobs system the system's.""" + lines = self.say('.jobs next') + self.assertEqual(lines[:4], ['Next runs:', + 'Wed 14 Oct 8:20am (21:20 UTC) – 10:40am, every 20 mins · 3. trades', + 'Wed 14 Oct 8:30am · 1. rating', + 'Wed 14 Oct 11:00am (00:00 UTC) – Thu 15 Oct 10:40am, every 20 mins · 3. trades']) + self.assertEqual(self.say('.jobs system'), ['System jobs:']) def test_no_schedule_running(self): """Without a schedule running, .jobs says so.""" @@ -634,25 +663,172 @@ def test_a_job_is_its_chats_whichever_way_its_updates_arrive(self): self.env(**test_sending.EveryNetworkAndMode.MODES[(network, first)]) reader = READERS[network] with contextlib.redirect_stderr(io.StringIO()): - answer = bot.take(reader, updates.update(network, first, chat, '.jobs add 0 23 * * * shorts')) + added = updates.update(network, first, chat, '.jobs add shorts daily at 23:00') + answer = bot.take(reader, added) if answer.then: answer.then() self.join_workers() key = f'{network}:{chat}' - self.assertEqual([job.text for job in self.s.chat_jobs(key)], ['shorts']) + self.assertEqual([job.name for job in self.s.chat_jobs(key)], ['rating', 'price close US', + 'shorts']) self.env(**test_sending.EveryNetworkAndMode.MODES[(network, then)]) self.replies.clear() answer = bot.take(reader, updates.update(network, then, chat, '.jobs')) if answer.then: answer.then() self.join_workers() - self.assertIn('1. 0 23 * * * shorts: on', self.replies[-1][1][1]) + self.assertTrue(any(line.endswith('3. ✅* shorts · daily at 11:00pm') for line in + self.replies[-1][1]), self.replies[-1][1]) self.assertEqual(self.replies[-1][0], key) self.dests.clear() - self.s.run(self.s.chat_jobs(key)[0]) + self.s.run(self.s.chat_jobs(key)[2]) self.assertEqual(self.dests, [key]) # posted to the same chat - self.s.delete(self.s.chat_jobs(key)[0]) - + self.s.delete(self.s.chat_jobs(key)[2]) + + +class JobsMenu(ChatBase): + """The .jobs menu's buttons: each screen drawn in place of the pressed one, with a page of the list and a button + per job; Add's steps; a stale or forged press acting on nothing.""" + + def test_enable_disable_flips_the_switch_in_the_button_and_the_message(self): + """Enable/disable lists the jobs with a button each, ' '; a press switches the job and + redraws, the switch flipped in both; Back goes to the list.""" + crc = self.crc() + lines, rows, data = self.press(f'm|on|0|{crc}') + self.assertEqual(self.toasts, ['Updating…']) + self.assertEqual(lines[0], 'Tap a job to switch it on or off') + self.assertEqual(rows, [['1 ✅ rating', '2 ✅ price close US', '3 ✅ trades'], ['Back']]) + lines, rows, data = self.press(data['1 ✅ rating']) + self.assertIn('1. ❌* rating · Tue–Sat at 8:30am Sydney', lines) + self.assertEqual(rows[0][0], '1 ❌ rating') + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT]['off'], ['rating']) + self.press(data['1 ❌ rating']) + self.assertNotIn('off', state.load(jobs.JOBS_FILE)[CHAT]) + lines, rows, _ = self.press(data['Back']) + self.assertEqual((lines[0], rows[0]), ('Jobs here', ['Enable/disable', 'Delete'])) + + def test_delete_offers_only_the_jobs_added_here(self): + """Delete lists every job, the default ones marked so and without a button, and a button for each one added + here; a press deletes it, saying so; a forged press on a default job deletes nothing.""" + lines, rows, data = self.press(f'm|del|0|{self.crc()}') + self.assertIn('1. ✅* rating · Tue–Sat at 8:30am Sydney (default)', lines) + self.assertEqual(rows, [['3 trades'], ['Back']]) + self.assertEqual(self.press(f'm|del|0|1|{self.crc()}')[0][0], 'rating is a default job: switch it off instead' + ' (.jobs toggle)') + lines, rows, data = self.press(data['3 trades']) + self.assertEqual(lines[:2], ['Deleted 3. trades', '']) + self.assertIn('None added here yet: Add makes one', lines) + self.assertEqual(rows, [['Back']]) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': []}) + + def test_run_now_runs_the_job_and_says_how_it_went(self): + """Run now lists the jobs with a button each; a press runs the job and says how it went.""" + _, _, data = self.press(f'm|run|0|{self.crc()}') + self.replies.clear() + self.press(data['3 trades']) + self.assertEqual(self.ran, [('trades', {})]) + self.assertEqual(self.replies[-1][1], ['3. trades: done']) + + def test_a_stale_or_forged_press_acts_on_nothing(self): + """A press drawn for another list (its checksum), for a number that is none, or of no screen at all switches, + deletes and runs nothing: the screen is drawn as the list is now, or the press is ignored.""" + good = self.crc() + for data in (f'm|on|0|1|0000', f'm|del|0|3|0000', f'm|run|0|3|0000', f'm|on|0|9|{good}', f'm|on|0|0|{good}', + f'm|run|0|x|{good}'): + with self.subTest(data=data): + lines, _, _ = self.press(data) + self.assertEqual(lines[0], 'The jobs have changed: here they are now') + for data in (f'm|zz|0|{good}', f'm|on|0|1|{good}|' + 'x' * 60, 'm|on|٨|1|' + good, 'm|on|0|1|a b'): + with self.subTest(data=data): + self.assertEqual(self.press(data), (None, None, None)) + self.assertEqual((self.ran, len(self.s.chat_jobs(CHAT))), ([], 3)) + self.assertTrue(all(job.on for job in self.s.chat_jobs(CHAT))) + + def test_the_older_job_buttons_still_work(self): + """A job button of an older list ('j|' runs a job, 'e|' switches one) still works, by the job's own checksum; + a switch redraws the menu's screen that switches jobs.""" + job = self.s.chat_jobs(CHAT)[2] + self.press(f'j|3|{jobs.checksum(job.key)}') + self.assertEqual((self.toasts, self.ran), (['Running…'], [('trades', {})])) + lines, rows, _ = self.press(f'e|3|{jobs.checksum(job.key)}') + self.assertFalse(self.s.chat_jobs(CHAT)[2].on) + self.assertEqual(lines[0], 'Tap a job to switch it on or off') + self.assertEqual(rows[0][2], '3 ❌ trades') + lines, _, _ = self.press('e|3|0000') # stale: the screen as the list is now, nothing switched + self.assertFalse(self.s.chat_jobs(CHAT)[2].on) + + def test_add_by_buttons_back_and_cancel(self): + """Add takes the report, how often, the day and the hour from buttons, by the timezone preference's clock, + Back a step back and Cancel back to the list; its last step adds the job and says so, the same job as typed.""" + lines, rows, data = self.press(f'm|add|{self.crc()}') + self.assertEqual(lines[0], 'Add a job: which report?') + self.assertEqual(rows[1:], [['4 price close', '5 price', '6 performance'], ['7 cal earnings', + '8 cal ex-dividend', '9 shorts'], ['10 rating', '11 milestone', '12 message'], ['Back', 'Cancel']]) + lines, rows, data = self.press(data['5 price']) + self.assertEqual((lines[0], rows[1]), ('Add price: how often?', ['weekly', 'monthly', 'quarterly'])) + lines, rows, data = self.press(data['weekly']) + self.assertEqual(rows[:2], [['Mon', 'Tue', 'Wed', 'Thu'], ['Fri', 'Sat', 'Sun']]) + self.assertEqual(data['Cancel'], f'm|top|0|{self.crc()}') + lines, rows, data = self.press(data['Sat']) + self.assertEqual((len(rows), [len(row) for row in rows], rows[-1]), (5, [5] * 5, ['20', '21', '22', '23', + 'Back'])) + self.assertEqual(data['Back'], f'm|add|5|w|{self.crc()}') + lines, rows, data = self.press(data['09']) + self.assertEqual(lines, ['Added: 4. ✅* price · weekly, Saturdays at 9:00am']) + self.assertEqual(rows, [['Jobs', 'Add another']]) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT]['jobs'][-1], entry('price weekly', + zone='Australia/Melbourne')) + self.assertIn('that job is here already', self.say('.jobs add price every saturday at 9am')[0]) + + def test_a_repeat_a_one_off_and_a_message_by_buttons(self): + """Add's repeat takes its interval and days, a one-off its day among the next 14 and its hour; a message's text + cannot come from a button, so its screen says how to type one.""" + crc = self.crc() + self.assertEqual(self.press(f'm|add|1|r|20|k|{crc}')[0], ['Added: 4. ✅* trades · every 20 mins, weekdays']) + lines, rows, data = self.press(f'm|add|9|o|{self.crc()}') + self.assertEqual((rows[0][0], len(sum(rows, [])) - 2), ('Wed 14 Oct', 14)) + self.assertEqual(self.press(data['Tue 20 Oct'].replace('|' + self.crc(), f'|8|{self.crc()}'))[0], + ['Added: 5. ✅* shorts · once, 20 Oct 2026 at 8:00am']) + lines, rows, _ = self.press(f'm|add|12|{self.crc()}') + self.assertIn('.jobs add message SCHEDULE "TEXT"', lines) + self.assertEqual(rows, [['Back', 'Cancel']]) + for data in (f'm|add|13|{crc}', f'm|add|5|x|{crc}', f'm|add|5|w|7|9|{crc}', f'm|add|5|d|24|{crc}', + f'm|add|1|r|7|k|{crc}', f'm|add|5|o|261340|9|{crc}', f'm|add|5|d|9|9|{crc}'): + with self.subTest(data=data): + self.assertEqual(self.press(data)[0][0], 'That is no step of Add') + + def test_every_screen_fits_each_network(self): + """With every default job and 30 jobs added, each page of each screen fits a message of each network (Discord + 2000 characters, Telegram 4096), with at most 25 buttons in five rows of five, each button's data within + Telegram's 64 bytes and its own in the message (Slack and Discord take it as the button's id).""" + def fits(service, lines, markup): + """Check one screen against a network's limits.""" + self.assertLessEqual(len('\n'.join(lines)), service.text_limit) + if service == DISCORD: + self.assertLessEqual(len(markup), 5) + self.assertTrue(all(len(row) <= 5 for row in markup)) + data = [button.data for row in markup for button in row] + self.assertEqual(len(data), len(set(data)), data) + self.assertTrue(all(len(each.encode()) <= buttons.MAX_CALLBACK_BYTES for each in data)) + self.patch(catalog, 'ITEMS', ITEMS) + for minute in range(29): + self.s.add(CHAT, 'message', [f'reminder number {minute} ' * 4], when.parse(f'daily at 1:{minute:02d}', + 'UTC')) + for service in (TELEGRAM, SLACK, DISCORD): + for screen in ('top', 'on', 'del', 'run'): + for page in range(6): + with self.subTest(service=service, screen=screen, page=page): + fits(service, *about.jobs_screen(service, CHAT, screen, page)) + for steps in ([], ['5'], ['5', 'w'], ['5', 'w', '5'], ['5', 'o'], ['5', 'o', '261020'], ['5', 'r'], + ['5', 'r', '20'], ['12']): + with self.subTest(service=service, steps=steps): + fits(service, *about.add_screen(service, CHAT, steps)) + fits(service, *about.next_screen(service, CHAT)) + fits(service, *about.added_screen(service, CHAT, 1)) + fits(service, *about.add_failed(service, CHAT, ['5', 'd', '9'], 'that job is here already')) + pages = len({about.jobs_screen(DISCORD, CHAT, 'on', page)[0][0] for page in range(6)}) + self.assertEqual(pages, 5) # 65 jobs, 16 to a page on Discord + self.assertEqual(len({about.jobs_screen(TELEGRAM, CHAT, 'top', page)[0][0] for page in range(6)}), 2) class ReportImports(FinbotCase): """What importing the reports brings in.""" diff --git a/tests/test_sending.py b/tests/test_sending.py index fdf21a56..2de4a762 100644 --- a/tests/test_sending.py +++ b/tests/test_sending.py @@ -578,8 +578,9 @@ def setUp(self): """A job in each chat for .jobs run to list, and what each command posts noted with its destination.""" super().setUp() self.patch(discord_common.READER, 'bot_id', '99') - every = [{'when': '*/20 * * * Mon-Fri', 'reports': 'trades', 'zone': 'UTC', 'on': True}] - state.save(jobs.SCHEDULE_FILE, {f'{network}:{chat}': every for network, chats in self.CHATS.items() + every = {'jobs': [{'action': 'trades', 'args': [], 'schedule': {'days': 'weekdays', 'every': 20, 'zone': 'UTC'}, + 'on': True}]} + state.save(jobs.JOBS_FILE, {f'{network}:{chat}': every for network, chats in self.CHATS.items() for chat in chats}) scheduler = jobs.Scheduler() with contextlib.redirect_stderr(io.StringIO()): diff --git a/tests/test_util.py b/tests/test_util.py index bcef89b4..2f3c330e 100644 --- a/tests/test_util.py +++ b/tests/test_util.py @@ -147,26 +147,45 @@ def test_date_range_english(self): def test_button_data_reads_back(self): """A button's data says what was pressed: a period button of a known kind and period (as keyboard() makes - them), any .jobs button (pick checks it), and nothing for anything else.""" + them), a .jobs menu's button of a known screen made of plain letters and digits, an older job button (pick + checks it), and nothing for anything else.""" self.assertEqual([buttons.parse(button.data) for button in buttons.keyboard('l', ['5', '0'])[0]], [buttons.ChartPress('l', key, '5,0') for key in 'dwmyx']) - self.assertEqual(buttons.parse(buttons.job_button(8, 'trades', 'abcd').data), buttons.JobPress('8', 'abcd')) + self.assertEqual(buttons.parse(buttons.menu_button('1 ✅ trades', 'on', [0, 1], 'abcd').data), + buttons.MenuPress('on', ('0', '1'), 'abcd')) + self.assertEqual(buttons.parse('m|add|abcd'), buttons.MenuPress('add', (), 'abcd')) + self.assertEqual(buttons.parse('j|8|abcd'), buttons.JobPress('8', 'abcd')) self.assertEqual(buttons.parse('j|8'), buttons.JobPress('8', '')) - for data in ('c|h|d|T1', 'c|z|w|T1', 'c|h|w', 'x|h|w|T1', '', 'c'): + self.assertEqual(buttons.menu_button('x' * 50, 'top', [0], 'abcd').label, 'x' * 40) + for data in ('c|h|d|T1', 'c|z|w|T1', 'c|h|w', 'x|h|w|T1', '', 'c', 'm|zz|0|abcd', 'm|on', 'm|on|0|a b', + 'm|on|٨|abcd', 'm|on|0|1|' + 'a' * 60): with self.subTest(data=data): self.assertIsNone(buttons.parse(data)) def test_a_press_is_told_what_is_coming_from_its_data_alone(self): - """The toast a press gets as it arrives: the period a period button loads, 'Running…' for any .jobs button - (the worker checks it later), and none for data of neither.""" + """The toast a press gets as it arrives: the period a period button loads, 'Updating…' for the .jobs menu's, + 'Running…' or 'Switching…' for an older job button (the worker checks them later), and none for other data.""" self.assertEqual([buttons.toast(button.data) for button in buttons.keyboard('l', ['5', '0'])[0]], ['Loading 1D…', 'Loading 7D…', 'Loading 1M…', 'Loading 1Y…', 'Loading Max…']) - self.assertEqual(buttons.toast(buttons.job_button(8, 'trades', 'abcd').data), 'Running…') + self.assertEqual(buttons.toast('j|8|abcd'), 'Running…') self.assertEqual(buttons.toast('j|99|0000'), 'Running…') + self.assertEqual(buttons.toast('e|99|0000'), 'Switching…') + self.assertEqual(buttons.toast('m|on|0|1|abcd'), 'Updating…') for data in ('c|f|d|', 'c|zz|m|T1', '', 'x'): with self.subTest(data=data): self.assertIsNone(buttons.toast(data)) + def test_a_message_takes_links_and_italics_in_each_networks_markup(self): + """In a message job's text, [text](url) is a link and a line in _underscores_ is in italics, an underscore in a + url included; the rest is text, escaped where the network needs it; plain() keeps the words alone.""" + line = '_See [R&D notes](https://x.example/a_b) & more_' + self.assertEqual(formats.markup(line, TELEGRAM), + 'See R&D notes & more') + self.assertEqual(formats.markup(line, SLACK), '_See & more_') + self.assertEqual(formats.markup(line, DISCORD), '_See [R&D notes]() & more_') + self.assertEqual(formats.markup('a_b_c and _', TELEGRAM), 'a_b_c and _') # italic only when the whole line is + self.assertEqual(formats.plain(line), 'See R&D notes & more') + def test_strip_url(self): """A link's text is its address without the scheme or www.""" for url, text in (('https://www.apple.com', 'apple.com'), ('http://example.org/a?b', 'example.org/a?b'), @@ -329,8 +348,7 @@ def at(self, zone, *args): return datetime.datetime(*args, tzinfo=zoneinfo.ZoneInfo(zone)) def test_english_becomes_a_schedule(self): - """Each form a chat may type becomes the schedule it says, in the zone it names, else the timezone - preference's.""" + """Each form a chat may type becomes the schedule it says, in the zone it names or the timezone preference's.""" for text, stored in (('daily', {'days': 'daily', 'at': '09:00'}), ('at 9am', {'days': 'daily', 'at': '09:00'}), ('weekdays at 4:30pm', {'days': 'weekdays', 'at': '16:30'}), ('weekly', {'days': ['sat'], 'at': '09:00'}), @@ -367,8 +385,7 @@ def test_english_becomes_a_schedule(self): self.assertEqual(when.parse(text, self.MEL).to_json(), {'zone': self.MEL, **stored}) def test_what_is_no_schedule_is_refused_with_examples(self): - """Words it cannot read, times and dates that are none, and a repeat with a time of day are refused, with - examples.""" + """Unreadable words, times and dates that are none, and a repeat at a time of day are refused, with examples.""" for text in ('fortnightly', 'at 25:00', 'at 13pm', 'every 31 feb', 'every 20 mins at 9am', 'weekly from 9am to 5pm', 'daily at 9am Atlantis', '', 'every 0 mins', 'the 32nd of each month', 'monthly; rm -rf /'): diff --git a/util/record_outputs.py b/util/record_outputs.py index c8b549b0..d1b2d6dd 100755 --- a/util/record_outputs.py +++ b/util/record_outputs.py @@ -24,11 +24,13 @@ '.performance 7d', '.performance 1y', '.performance Nope', '.pie Rob', '.pie Rob sector', '.pie', '.compare T1 T2', '.compare T1 T2 3m', '.compare T1', '.history T1', '.history', '.watchlist', '.watchlist add MSFT', '.watchlist add MSFT', '.watchlist del MSFT', '.watchlist frob X', '.T1', '.jobs', '.jobs run', '.jobs toggle', - '.jobs add', '.jobs add 0 22 * * * shorts', '.jobs del 4', '.jobs run 1', 'hi @bot', 'noise'] + '.jobs add', '.jobs add shorts daily at 22:00', '.jobs del 4', '.jobs run 1', '.jobs next', '.jobs system', + 'hi @bot', 'noise'] BUTTONS = ['c|h|m|T1', 'c|h|x|T1', 'c|p|w|T1', 'c|c|y|T1,T2', 'c|f|y|', 'c|l|d|3,0', 'c|l|m|0,10', 'c|zz|m|T1', - 'j|1|0000', 'e|1|0000'] -JOBS = ['*/20 * * * Mon-Fri trades', '29 21 * * Mon-Fri price interday', # each chat's, for .jobs + 'j|1|0000', 'e|1|0000', 'm|on|0|0000', 'm|add|0000', 'm|add|5|w|0000', 'm|next|0000'] +JOBS = ['*/20 * * * Mon-Fri trades', '29 21 * * Mon-Fri price interday', # each chat's, for .jobs (as cron had them) '28 21 * * Sat cal earnings; cal ex-dividend; price 7; performance 7; rating'] +ADDED = ['trades every 20 mins weekdays', 'price daily at 21:29', 'cal earnings every saturday at 21:28'] # since cron SCHEDULED = ['trades', 'price midsession', 'price interday', 'price premarket', 'price close', 'price 7', 'cal earnings', 'cal ex-dividend', 'performance 7', 'rating', 'shorts', 'milestone', 'reminder'] TELEGRAM_API = 'https://api.telegram.org/botX/' @@ -117,10 +119,14 @@ def post(url, method='POST', **kwargs): self.env(slackBotToken='xoxb-test', slackOutgoingWebhook='https://example.com/slack') # Slack's buttons too if 'discordBotToken' in {field.name for field in common.dataclasses.fields(common.config.env)}: self.env(discordBotToken='discord-test') - if hasattr(jobs, 'SCHEDULE_FILE'): # each chat's jobs, since jobs per chat + chats = ('telegram:55', 'slack:C1', 'discord:77') + if hasattr(jobs, 'JOBS_FILE'): # the default jobs and those each chat added, since schedules of finbot's own + added = [jobs.make_job(*jobs.parse_add(text, 'UTC'), 'telegram:55').stored() for text in ADDED] + common.state.save(jobs.JOBS_FILE, {key: {'jobs': added} for key in chats}) + schedule = jobs.Scheduler(clock=lambda: common.datetime.datetime(2026, 10, 15, tzinfo=common.UTC)) + elif hasattr(jobs, 'SCHEDULE_FILE'): # each chat's jobs, since jobs per chat entries = [{'when': job.when, 'reports': job.text, 'zone': 'UTC', 'on': True} for job in map(jobs.parse_job, JOBS)] - chats = ('telegram:55', 'slack:C1', 'discord:77') common.state.save(jobs.SCHEDULE_FILE, {key: entries for key in chats}) schedule = jobs.Scheduler() else: # one schedule for all chats, the shipped one @@ -164,7 +170,13 @@ def scheduled(run, kwargs): keys = ('telegram:1', 'slack:C9', 'discord:78') return lambda: [run(delivery.chat(key, scheduled=True), **kwargs) for key in keys] for line in SCHEDULED: - if hasattr(jobs, 'parse_reports'): + if hasattr(jobs, 'ACTIONS'): # a job is an action and its arguments, since schedules of finbot's own + name, *arguments = line.split() + if name not in jobs.ACTIONS: # the reminder report, which message jobs replaced + continue + module = jobs.ACTIONS[name] + label, kwargs = module.job(arguments) + elif hasattr(jobs, 'parse_reports'): label, module, kwargs = jobs.parse_reports(line)[0] else: # a commit from before jobs per chat, whose reports were scripts name, _, arguments = line.partition(' ') diff --git a/util/sync_defaults.py b/util/sync_defaults.py index a68a2ec9..40084547 100755 --- a/util/sync_defaults.py +++ b/util/sync_defaults.py @@ -31,9 +31,9 @@ def sync(apply=False, force=False, dest=None, create=False): for src in sorted((config.REPO_DIR / 'defaults').iterdir()): if not src.is_file(): continue - if src.name in ('finbot.env', 'schedule.cron'): + if src.name == 'finbot.env': # the startup file is not data: it holds credentials, lives wherever the launcher loads it from, and the bot - # never reads it; the old schedule is read only to give an upgrade's home chats their jobs (lib/migrate.py) + # never reads it continue name = src.name target = dest_dir / name From c64197a8840fb2bfa4735ee40480f8dd6e96172e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 06:56:32 +0000 Subject: [PATCH 06/41] Docs: jobs in English, the default jobs, the .jobs menu; no cron or reminder.json - doc/scheduling.md is rewritten: - the menu and what to type; - schedules in English, with zones and daylight saving; - message jobs; - the 35 default jobs as a Melbourne chat sees them; - the whitelist of actions; - what an upgrade leaves behind (finbot_schedule.json's cron jobs, reminder.json). - The job examples in commands.md now use English. - configuration.md: the country_code and timezone rows, and reminder.json. - install.md drops croniter; docker.md drops reminder.json. - The network guides describe the menu's buttons and say every chat has the default jobs. - CLAUDE.md maps lib/when.py, lib/catalog.py, the new lib/jobs.py, the message report and the menu. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 42 +++++---- README.md | 4 +- doc/commands.md | 17 ++-- doc/configuration.md | 8 +- doc/discord.md | 13 +-- doc/docker.md | 2 +- doc/install.md | 6 +- doc/scheduling.md | 203 ++++++++++++++++++++++++++++++++----------- doc/slack.md | 8 +- doc/telegram.md | 4 +- 10 files changed, 209 insertions(+), 98 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 9fe40e45..9c958c96 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -23,8 +23,7 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm `compose.yaml` does; all except for pull requests that change only README.md, CLAUDE.md, `doc/*.md` or `img/*.png` (install, compile, tests and the build are skipped; the allowlist in `.github/workflows/ci.yml` must never include a path a test reads). This cloud environment cannot reach `deb.debian.org`, so only CI builds the image. -- Production runs the Debian system packages (including `python3-croniter`, for the scheduler in `lib/jobs.py`); - `requirements.txt` is for pip/uv users and CI. The libraries in `requirements-optional.txt` are each imported only +- Production runs the Debian system packages; `requirements.txt` is for pip/uv users and CI. The libraries in `requirements-optional.txt` are each imported only inside the code of the mode that needs it, never at module load (a test starts finbot with them all blocked). - A new preference goes in three places: a `Prefs` field with its default (`lib/config.py`; its type says how it is read), `defaults/preferences.conf` (commented out at its default, `#key=value`: the template sets nothing, and finbot @@ -70,10 +69,10 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm - `lib/reports/`: what finbot says. Scheduled (each has `run(dest=None, **kwargs)`, which posts to the chat that asked or the job's chat, and with no dest raises: there are no home chats; `job(argv)`, its arguments' parser; and `ARGUMENTS`, what `.jobs add` lists): - `price performance trades shorts cal rating milestone reminder podcasts cleanup sharesight_warm yahoo_warm`. trades, + `price performance trades shorts cal rating milestone message podcasts cleanup sharesight_warm yahoo_warm`. trades, rating, milestone and podcasts keep a record per chat of what they told it (`state.load_for`, `state.edit_for`), added to only by a scheduled run (`delivery.interactive(dest)` is false). - Chat-only (`prepare_*`): `about` (.help .jobs), `history` (.history .compare), `holdings` (.holdings .who .pie + Chat-only (`prepare_*`): `about` (.help, and the .jobs menu's screens: `jobs_screen`, `add_screen`, `next_screen`), `history` (.history .compare), `holdings` (.holdings .who .pie .watchlist), `profile` (.SYMBOL), `rankings` (.marketcap .pe .fpe .peg .beta .buy .sell). - `lib/delivery.py`: `Report(lines, chart, buttons)`, the destinations (the chat that asked, or a chat by its key with `chat(key)`, as its network's bot), `deliver(dest, report)`, the one-line error report (to its chat; one that belongs @@ -83,8 +82,10 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm message), which `text()` and `chart()` stop before posting. `DiscordBot`: a slash command's first message replaces its "thinking…" through its `interaction` (application id/token), and a pressed message is edited through the press's. -- `lib/buttons.py`: `Button`, a chart's period buttons (`keyboard`), a `.jobs` command's (`job_button`), what a - pressed button's data asks for (`parse`), and the toast it gets at once (`toast`); each transport draws them. +- `lib/buttons.py`: `Button`, a chart's period buttons (`keyboard`), the `.jobs` menu's (`menu_button`, a + `MenuPress`; older lists' `j|`/`e|` buttons still parse), what a pressed button's data asks for (`parse`), and the + toast it gets at once (`toast`); each transport draws them, and Slack and Discord take a button's data as its id, so + it is unique in a message. `lib/charts.py`: a chart with period buttons, built by its kind's builder (`build`) for the command's first view (`worker.chart_reply`) and for each press. - `lib/dialects.py`: each network's markup and message sizes. `lib/markets.py`: one table by market code (flag, @@ -104,13 +105,20 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm `telegram:-1001234`), the same in both modes. `lib/watchlist.py`: each chat's watchlist (`tickers(chat)`), and `with_holdings(chat)`, the stocks a report in that chat covers; a report takes the chat from its destination. - `lib/config.py`: `config.env` (finbot.env, fixed at start) and `config.prefs` (preferences.conf, a new object on - each reload). `lib/jobs.py`: the scheduler: each chat's jobs (`SCHEDULE_FILE`, by chat key; `add`, `delete`, - `toggle`, `run`, which `.jobs` calls through `jobs.current`) and the fixed `SYSTEM` jobs, its shared state under one - lock; each command runs in the bot's process, in a thread (`run_in_process`), reported once if still running after - `JOB_TIMEOUT` and left to finish (every request goes through `net`, with its timeouts: a test checks). - `lib/migrate.py`: once, on an upgrade, the old schedule.cron, the watchlist, the plans and the reports' shared - records become the old home chats' (the retired telegramChatID, slackChannelID and discordChannelID, which it alone - reads, from the environment). + each reload). `lib/jobs.py`: the scheduler: each chat's jobs, the default ones and those it added (`JOBS_FILE`, by + chat key: the defaults it switched off, the jobs it added; `see` makes a chat known at its first command; `add`, + `delete`, `toggle`, `run`, `upcoming`, which `.jobs` calls through `jobs.current`; `parse_add` reads `.jobs add`), + and the fixed `SYSTEM` jobs, its shared state under one lock. A job is an action of `ACTIONS` (the report modules, + `message` among them) with the arguments its `job()` takes, never a command; one stored that is anything else is + logged and left out. Each job runs in the bot's process, in a thread (`run_in_process`), reported once if still + running after `JOB_TIMEOUT` and left to finish (every request goes through `net`, with its timeouts: a test checks); + each job's next run is worked out once and again only after it runs, so a tick costs little. `lib/when.py`: a job's + `Schedule` (days, a time or a repeat, or a span across days, in a zone): `parse()` of the English a chat types, + `describe()`, `next_after()` through daylight saving. `lib/catalog.py`: the default jobs (`ITEMS`), each with its + name and schedule; the period of performance and price follows how often it runs (`period_args`). + `lib/migrate.py`: once, on an upgrade, the old home chats (the retired telegramChatID, slackChannelID and + discordChannelID, which it alone reads, from the environment) become chats finbot knows, and the watchlist, the + plans and the reports' shared records become theirs. - `tests/common.py`: `FinbotCase` (offline fixtures). `self.command('.price')` returns what was sent: `('text', lines)` or `('photo', size, caption)`. `wsgi_environ(path, body, **headers)` is a request for `bot.main`. `ProviderCase` runs the real parsers on the answers saved under `doc/` (`sample`), as `Resp` and `serve` return @@ -119,10 +127,10 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm - Which test file: commands and scheduled reports `test_reports`; failures, bad input, secrets in logs, signals `test_edges`; webhook requests, Slack events, slash commands and signatures, and the Slack manifest `test_inbound`; the pull modes `test_pull`; the Discord bot `test_discord`; Yahoo `test_yahoo`; Sharesight `test_sharesight`; posting - to each network `test_sending`; Telegram's webhook and command menu `test_telegram_api`; job lines, job arguments and - the migration `test_schedule`; the scheduler and `.jobs` in a chat `test_scheduler`; config `test_config`; charts - `test_charts` (refresh goldens with `UPDATE_GOLDEN=1`); markets, formats, links, buttons, cache and state files - `test_util`; data dir `test_datadir`; log levels, request lines and whole lines `test_edges`; chunking, git_version, and the docstring, stdout, line-width and no-pandas + to each network `test_sending`; Telegram's webhook and command menu `test_telegram_api`; job actions and arguments, + the default jobs, the messages and the migration `test_schedule`; the scheduler, and `.jobs` and its menu in a chat + `test_scheduler`; config `test_config`; charts `test_charts` (refresh goldens with `UPDATE_GOLDEN=1`); markets, + formats, links, buttons, schedules (`lib/when.py`), cache and state files `test_util`; data dir `test_datadir`; log levels, request lines and whole lines `test_edges`; chunking, git_version, and the docstring, stdout, line-width and no-pandas checks `test_finbot`. ## Conventions diff --git a/README.md b/README.md index 64f02bbf..ed351689 100644 --- a/README.md +++ b/README.md @@ -54,8 +54,8 @@ subscribes the bot to the updates each network pushes to it (each guide's _HTTP finbot ([guide](doc/discord.md)). Then send `/help` in a chat with the bot. Commands work as plain messages too (`.help`), which are quicker to type on a -phone ([Commands](doc/commands.md#supported-commands)). Each chat schedules its own reports with `/jobs add` -([Scheduling](doc/scheduling.md)). +phone ([Commands](doc/commands.md#supported-commands)). From its first command, each chat has the default scheduled +jobs, which `/jobs` switches on and off, adds to and runs ([Scheduling](doc/scheduling.md)). ## Documentation * [Commands and reports](doc/commands.md), with screenshots diff --git a/doc/commands.md b/doc/commands.md index 0a40bf7c..bd588c0d 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -40,9 +40,10 @@ Where: /help /history symbol /holdings portfolio -/jobs [run|toggle] [number] -/jobs add [CRON_TZ=zone] time report -/jobs del number +/jobs [next|system] +/jobs run|toggle|del [number] +/jobs add report schedule +/jobs add message schedule "text" /marketcap [symbol|top|bottom] /pe [symbol|top|bottom] /peg [symbol|top|bottom|negative] @@ -126,7 +127,7 @@ With persistent storage, the default of `past_days=28` suits. This is useful if past_days=28 ``` -Job (see [Scheduling](scheduling.md)), such as `/jobs add */20 * * * Mon-Fri trades`: +Job (see [Scheduling](scheduling.md)), such as `/jobs add trades every 20 mins weekdays`: ``` trades ``` @@ -155,7 +156,7 @@ price_percent=5 demote_volatile=True ``` -Job (see [Scheduling](scheduling.md)), such as `/jobs add 29 21 * * Mon-Fri price interday`: +Job (see [Scheduling](scheduling.md)), such as `/jobs add price close weekdays at 4:30pm New York`: ``` price premarket|midsession|interday|close|DAYS ``` @@ -201,7 +202,7 @@ Config example: past_days=28 ``` -Job (see [Scheduling](scheduling.md)), such as `/jobs add 28 21 * * Sat performance 7`: +Job (see [Scheduling](scheduling.md)), such as `/jobs add performance weekly`: ``` performance [DAYS] ``` @@ -228,7 +229,7 @@ The `cal` report sends upcoming earnings and ex-dividend date alerts. The data i future_days=30 ``` -Job (see [Scheduling](scheduling.md)), such as `/jobs add 28 21 * * Sat cal earnings`: +Job (see [Scheduling](scheduling.md)), such as `/jobs add cal earnings every sunday at 8:30am`: ``` cal earnings|ex-dividend ``` @@ -314,7 +315,7 @@ Command: ### Consensus rating changes ![Rating change report in Slack](../img/finbot_rating.png?raw=true "Rating change report in Slack") -Notifies when Yahoo Finance analyst rating consensus changes category (e.g. from hold to buy). Designed to run on a schedule, such as `/jobs add 28 21 * * Sat rating` (see [Scheduling](scheduling.md)). Requires persistent storage. +Notifies when Yahoo Finance analyst rating consensus changes category (e.g. from hold to buy). Designed to run on a schedule, such as `/jobs add rating tue-sat at 8:30am` (see [Scheduling](scheduling.md)). Requires persistent storage. ``` rating diff --git a/doc/configuration.md b/doc/configuration.md index 92203c42..59cadc89 100644 --- a/doc/configuration.md +++ b/doc/configuration.md @@ -68,8 +68,8 @@ A change takes effect on a reload. A true/false preference takes `true`/`false`, | `hyperlinkFooter` | `True` | Add a website link footer to stock lookups | | `hyperlinkProvider` | `google` | Where ticker links go: `google` or `yahoo` | | `exchange_suffix` | `AX` | Regional: Yahoo's suffix for your exchange (`AX` for the ASX, `NZ`, `L`, `TO` and so on). A bare symbol Yahoo does not know is tried again with it, in every command that takes a symbol (`BHP` becomes `BHP.AX`); empty, a symbol is taken as written | -| `country_code` | `AU` | Regional: your country, for the reminder report | -| `timezone` | `Australia/Melbourne` | Regional: your time zone, for the reminder report, and for each job's next run in `/jobs` (besides UTC) | +| `country_code` | `AU` | Regional: your country, for the default jobs' messages that are for one (`AU`: the tax year's reminders) | +| `timezone` | `Australia/Melbourne` | Regional: your time zone, the clock of the times `/jobs` takes and shows, and a job's zone where it names none | | `cache` | `True` | Cache Yahoo and Sharesight responses | | `cache_seconds` | `43200` | Cache lifetime in seconds (12 hours) | | `performance_cache_seconds` | `2400` | Cache lifetime of Sharesight performance reports in seconds: longer than the 30 minutes between `sharesight_warm` runs, so the holdings never run out | @@ -85,7 +85,7 @@ edit `preferences.conf`, then reload if running. The service log shows what chan ## Data directory All state and cache lives under `var/` in the checkout by default. Set `DATA_DIR` to relocate it (cache moves to `$DATA_DIR/cache`); in Docker it is `/data`, the host's `/opt/finbot-data` (see [docker.md](docker.md)). -Reference data (`finbot_adr.json`, `finbot_sws_*.json`, `reminder.json`) ships in `defaults/`. A file of the same name in the data directory takes precedence. Defaults are never copied or overwritten automatically; to see or pull in updates run: +Reference data (`finbot_adr.json`, `finbot_sws_*.json`) ships in `defaults/`. A file of the same name in the data directory takes precedence. Defaults are never copied or overwritten automatically; to see or pull in updates run: ``` util/sync_defaults.py # dry run util/sync_defaults.py --apply # copy missing files (including a starter preferences.conf) @@ -93,7 +93,7 @@ util/sync_defaults.py --force # also replace changed files, keeping .bak copies util/sync_defaults.py --dest /srv/finbot --apply # target another directory instead of the data directory util/sync_defaults.py --dest /srv/finbot --create --apply # and make it (mode 750) if it does not exist yet ``` -`reminder.json` holds the reminders that the `reminder` report posts on their day. Each entry has a `month_day` (`06-30`) and a `message`, which is one line or a list of lines, and may name a `year` (`2026`) and a `gate_country` (posted only when `country_code` in `preferences.conf` matches; `{flag}` in its message becomes that country's flag). A message is text, with two kinds of markup that each network gets in its own form: `[text](https://example.com)` is a link, and a line wrapped in underscores, `_like this_`, is in italics. The Australian end of financial year checklist (23 June) is such an entry: if you keep your own `reminder.json` in the data directory, `util/sync_defaults.py` shows what it lacks. +The reminders that `reminder.json` held are message jobs now, among the default jobs (see [Scheduling](scheduling.md#the-default-jobs)), and finbot reads that file no more: delete the data directory's copy. A chat adds its own with `/jobs add message SCHEDULE "TEXT"`. With systemd `DynamicUser=yes` and `StateDirectory=finbot`, `DATA_DIR` defaults to systemd's `$STATE_DIRECTORY` (`/var/lib/finbot`), so no setting is needed; move existing data once with `cp -a /opt/finbot/var/. /var/lib/finbot/`. An explicit `DATA_DIR` always wins. diff --git a/doc/discord.md b/doc/discord.md index 2da5295b..9ac0d19a 100644 --- a/doc/discord.md +++ b/doc/discord.md @@ -16,8 +16,8 @@ The bot does the same either way. What differs is how it hears from Discord: | Python modules | `discord.py` (Debian: `python3-discord`) | `PyNaCl` and `waitress` (Debian: `python3-nacl`, `python3-waitress`) | | `finbot.env` keys | `discordBotToken` | `discordBotToken`, `discordOutgoingWebhook` and `discordPublicKey` | -In both: the scheduled reports with their charts; period buttons under charts and the buttons of `/jobs run` and -`/jobs toggle`, which redraw their message in place; linked tickers, without link previews and without pinging anyone; +In both: the scheduled reports with their charts; period buttons under charts and the buttons of the `/jobs` menu, +which redraw their message in place; linked tickers, without link previews and without pinging anyone; and direct messages with the bot. A command in a message shows "finbot is typing…" while it works; a slash command shows Discord's "finbot is thinking…" until its answer takes its place. @@ -41,8 +41,8 @@ empty, so change both, and restart finbot, to switch. start have it). 5. Restart finbot (`sudo systemctl restart finbot`). As it starts, it registers the slash commands, which can take a few minutes to show in Discord the first time, and the log says `discord: gateway (no discordOutgoingWebhook)`. -6. Send `/help` (or `.help`) in the channel, or in a direct message to the bot. Each channel schedules its own reports - with `/jobs add` ([Scheduling](scheduling.md)). +6. Send `/help` (or `.help`) in the channel, or in a direct message to the bot. Each channel has the default scheduled + jobs, which `/jobs` changes ([Scheduling](scheduling.md)). ``` discordBotToken=your-bot-token @@ -102,8 +102,9 @@ While the arguments are typed, Discord shows finbot's suggestions, in both modes * the portfolios, for the commands that take one; * tickers, for the commands that take one: the channel's watchlist and the holdings. They come from the cache, never from Sharesight or Yahoo, so the suggestions come at once; the holdings are there once `sharesight_warm` has run; -* for `/jobs`: `run`, `toggle`, `add` and `del`; after `run`, `toggle` or `del` the channel's job numbers, each with - its line; after `add` and a time, the reports, each with what it takes, and then what the report takes. +* for `/jobs`: `add`, `run`, `toggle`, `del`, `next` and `system`; after `run` or `toggle` the channel's job numbers, + each with its name (after `del`, those of the jobs it added); after `add` the reports, each with what it takes, then + what the report takes and the words of a schedule. A message holds up to 2,000 characters, so a long report arrives as several messages. A chart goes as an attachment of the message that carries its report, with its period buttons under it. diff --git a/doc/docker.md b/doc/docker.md index e5d0bc15..c092d5b4 100644 --- a/doc/docker.md +++ b/doc/docker.md @@ -29,7 +29,7 @@ must outlast every container. Run it once, as root; it is safe to run again. It: 1. runs `util/sync_defaults.py --create --apply --dest /opt/finbot-data`, which makes the data dir (mode 750) and copies - the shipped defaults into it: `preferences.conf`, `reminder.json` and the data files. It never replaces a file you + the shipped defaults into it: `preferences.conf` and the data files. It never replaces a file you have changed. The script needs only the standard library, so any `python3` on the host runs it. 2. gives the data dir to the container's user: `chown -R 10001:10001 /opt/finbot-data`. 3. copies `defaults/finbot.env` to `/opt/finbot.env` if there is none, owned by `root:10001`, mode 640: root and the diff --git a/doc/install.md b/doc/install.md index 5ec08548..20ca35ec 100644 --- a/doc/install.md +++ b/doc/install.md @@ -14,7 +14,7 @@ systemd, as below. To run it in a container instead, see [docker.md](docker.md). HTTP mode subscribes the bot to push updates: each network posts its updates to the gateway, which passes them to the bot (each guide's _HTTP mode_). * Python 3.13.5 (the version in `.python-version`; 3.12 is the minimum), and these modules, pinned to versions with - 3.13 support in `requirements.txt`: croniter, python-dateutil, requests, matplotlib and numpy. + 3.13 support in `requirements.txt`: python-dateutil, requests, matplotlib and numpy. * For a network in HTTP mode (its webhook URL set), also waitress; for Slack in Socket Mode, websocket-client; for Discord's gateway API, discord.py, and for its HTTP mode, PyNaCl. These are in `requirements-optional.txt`, each imported only by the mode that needs it. Docker and a Python venv install them all, Docker in its image and a venv @@ -23,7 +23,7 @@ systemd, as below. To run it in a container instead, see [docker.md](docker.md). ## Debian 1. The packages, and those of the modes you use: ``` - sudo apt install git python3 python3-requests python3-croniter python3-dateutil python3-matplotlib python3-numpy + sudo apt install git python3 python3-requests python3-dateutil python3-matplotlib python3-numpy sudo apt install python3-waitress # any network in HTTP mode sudo apt install python3-websocket # Slack's Socket Mode sudo apt install python3-discord # Discord's gateway API @@ -40,7 +40,7 @@ systemd, as below. To run it in a container instead, see [docker.md](docker.md). ``` 4. Check it: `~/finbot/util/check_config.py /opt/finbot.env` 5. Run it under systemd, below. The first start makes the data directory (`var/` in the checkout, or - `/var/lib/finbot` with `DynamicUser=`); each chat then schedules its own reports with `/jobs add` + `/var/lib/finbot` with `DynamicUser=`); each chat then has the default scheduled jobs, which `/jobs` changes ([scheduling.md](scheduling.md)). [configuration.md](configuration.md) has every key and preference. diff --git a/doc/scheduling.md b/doc/scheduling.md index 9dd7ba92..1a779313 100644 --- a/doc/scheduling.md +++ b/doc/scheduling.md @@ -1,57 +1,158 @@ # Scheduling -Each chat has its own jobs: reports that run on a schedule and post to that chat. Anyone in the chat manages them there, -with `/jobs` (or `/job`; as a message, also `.cron`, `.crons` or `.schedule`). A new chat has none until it adds -them: +Each chat has its own jobs: reports and messages that run on a schedule and post to that chat. Anyone in the chat +manages them there, with `/jobs` (or `/job`; as a message, also `.cron`, `.crons` or `.schedule`). + +Every chat starts with the [default jobs](#the-default-jobs), each switched on: a chat finbot knows has them all until +it switches some off. finbot knows a chat from its first command, and knows the chats of an earlier version's schedule +and its home chats. + +## The menu +`/jobs` lists the chat's jobs, numbered, a page at a time: the default jobs, then those added here. Each line shows ✅ +(on) or ❌ (off), a `*` if the job has never run, its name, and its schedule in words: +``` +1. ✅ trades · every 20 mins, Mon 9am – Sat 3pm Sydney +3. ✅* price midsession US · weekdays at 12:45pm New York (3:45am local, Tue–Sat) +``` +Under it are buttons, each screen drawn in place of the last: +* **Enable/disable**: a button per job, its number, switch and name (`3 ✅ price midsession US`); a press switches the + job on or off. +* **Delete**: a button per job added here. A default job has no button: switch it off instead. +* **Add**: the report, how often (one-off, daily, weekdays, weekly, monthly, quarterly, bi-annual, annual or repeating), + then the date, the day, or a repeat's interval and days, then the hour. **Back** takes a step back, **Cancel** goes + back to the list. A message's text cannot come from a button, so its screen says how to type one. +* **Run now**: a button per job; a press runs it now, as its schedule would, and says how it went. +* **Next**: the next runs, in time order. + +A button drawn for a list that has since changed acts on nothing: the screen is drawn again as the list is now. + +Or type: ``` -/jobs this chat's jobs, numbered, each with its next run; then the system jobs -/jobs add the reports a job can run, and what each takes -/jobs add 29 21 * * Mon-Fri price interday -/jobs run the list, with a button per job that runs it now; /jobs run 2 runs job 2 -/jobs toggle the list, with a button per job that switches it on or off; /jobs toggle 2 -/jobs del 2 deletes job 2 +/jobs the chat's jobs, with the buttons above +/jobs next the next 20 runs; a repeat's runs on one day are one entry +/jobs system the system jobs +/jobs add the reports a job can run, what each takes, and examples +/jobs add price weekly +/jobs add price close weekdays at 4:30pm New York +/jobs add message every 31 mar at 8:30am "🥳 Happy EOFY" +/jobs run 2 runs job 2 now; /jobs run alone: the list with a button per job +/jobs toggle 2 switches job 2 on or off +/jobs del 36 deletes job 36, one added here ``` -A job is a time in crontab's syntax, then one or more reports with their arguments: -* **The time** is five fields (minute, hour, day of month, month, day of week; `*/20`, `Mon-Fri`, ranges and lists - work) or an `@macro` such as `@daily`. A time that cannot be or never comes (minute 61, 30 February) is refused, - saying why. -* **Times are in UTC**, which keeps the ASX's jobs and NYSE's each within an hour of their local times all year. To - follow one market's clocks exactly, a job names its zone first, as crontab writes it: - `/jobs add CRON_TZ=America/New_York 15 13 * * Mon-Fri price midsession`. `/jobs` shows each next run in UTC and in - the `timezone` preference. -* **The reports** are `trades`, `price midsession|interday|premarket|close|DAYS`, `cal earnings|ex-dividend`, - `performance [DAYS]`, `rating`, `milestone`, `shorts`, `reminder` and `podcasts FEED_URL`. Several, separated by `;`, - run one after another: `/jobs add 28 21 * * Sat cal earnings; cal ex-dividend; price 7; performance 7; rating`. - Quote an argument that holds spaces or a `;` (a feed URL, say). -* A chat has 30 jobs at most. - -A job posts to its own chat, whichever way that chat's updates reach finbot: the jobs are kept by network and chat id -(`finbot_schedule.json` in the data directory), so switching a network between pull and HTTP mode keeps them. Each chat -also keeps its own record of what a report has told it (trades, rating changes, record highs, podcast episodes), so the -same report scheduled in two chats tells each of them once. A run from `/jobs run` is a scheduled run: it posts to the -chat and updates the same record. A change counts at once, with no reload. - -**The system jobs**, fixed in `lib/jobs.py` and listed by `/jobs` under _System_, keep the cache full and tidy: -`sharesight_warm` every 30 minutes, `yahoo_warm` every 8 hours (see [Caching](caching.md)), and `cleanup 30` weekly, -which deletes cache files older than 30 days. A chat cannot add, switch or run them, and a failure of one goes to the -log. - -**From `schedule.cron`**, where the schedule used to be: on an upgrade from before jobs per chat, the first start -gives each chat that the retired `telegramChatID`, `slackChannelID` or `discordChannelID` still names in `finbot.env` -the jobs of the data directory's `schedule.cron`, else of the shipped `defaults/schedule.cron`, each in the zone it -ran in (the server's), without the system jobs. It copies the reports' records to those chats too, so that nothing old -is told again. It logs what it did, and any line it left out, and reads the file no more: delete the data directory's -copy afterwards, and the retired keys. - -Behaviour worth knowing: -* The reports run in `bot.py` itself, each in a thread of its own, sharing the bot's user, sandbox, environment, data - directory, caches and connections; charts are drawn one at a time, and a slow report holds up no chat command. There +## A job in English +A job is a report and what it takes, then when it runs: +* **Days:** `daily`, `weekdays`, `weekends`, `weekly` (Saturdays), `every saturday` or `saturdays`, `tue-sat`, `monthly` + (the 1st), `first day of each month`, `on the 15th`, `quarterly` (1 Jan, Apr, Jul and Oct), `bi-annual` (1 Jan and 1 + Jul), `annual` (1 Jan), `every 31 mar` (each year) or `on 28 oct 2026` (once). +* **Time:** `at 9`, `at 9am`, `at 9:30am` or `at 21:30`. Without one, 9am, or 8:30am for a message. +* **Repeating:** `every 20 mins`, `hourly`, `hourly at :01`, `every 2 hours` or any other number, counted from + midnight. It can take days, and a window of hours (`from 10am to 9pm`, `between 9am and 5pm`) or a span across days + (`from mon 9am to sat 3pm`). +* **Zone, last:** + * Times are by the `timezone` preference's clock. A job keeps that zone, so it follows its daylight saving: 8am stays + 8am. + * Name `UTC` for a time that never moves. + * Name another zone for its clock: an IANA name (`America/New_York`), a place (`New York`, `Hong Kong`), or a + common abbreviation (`ET`, `EST`, `PT`, `GMT`, `CET`, `HKT`, `JST`, `AEST`). An abbreviation means the zone with its + daylight saving: `EST` follows New York's clock in summer too. + * `/jobs` shows a job by its own zone's clock, with the local time beside it where the clocks differ: `weekdays at + 4:30pm New York (7:30am local, Tue–Sat)`. + +The reports are `trades`, `price midsession|interday|premarket|close|DAYS`, `cal earnings|ex-dividend`, `performance +[DAYS]`, `rating`, `milestone`, `shorts`, `podcasts FEED_URL` and `message "TEXT"`. + +The period of `performance` and `price` follows how often the job runs: weekly covers 7 days, monthly 30, quarterly +90, bi-annual 180 and annual 365, and a daily `price` is the day's moves. A number given (`price 30 weekly`) is used +instead. + +A **message** posts its text as it is: +* The text comes last, in double quotes (or curly quotes, as phones type them). It runs from the first quote to the + last, so quotes inside it stay, and it may run over several lines. +* `[text](url)` is a link, and a line wrapped in `_underscores_` is in italics. +* A one-off message (`on 28 oct 2026`) is deleted after its time has passed, when the list is next shown. + +A job that is the same as a default job (the same report, days and times, in a zone with the same clock) switches that +default job on instead of being added. A chat can add 30 jobs. + +## The default jobs +These are as a chat in Melbourne sees them in mid-October. The local times beside New York's, Hong Kong's and London's +move as either side's daylight saving changes. + +| # | job | when | +|---|---|---| +| 1 | trades | every 20 mins, Mon 9am – Sat 3pm Sydney | +| 2 | price midsession AU | weekdays at 1:00pm Sydney | +| 3 | price midsession US | weekdays at 12:45pm New York (3:45am local, Tue–Sat) | +| 4 | price premarket US | weekdays at 7:15am New York (10:15pm local) | +| 5 | price close AU | weekdays at 4:30pm Sydney | +| 6 | price close HK | weekdays at 4:30pm Hong Kong (7:30pm local) | +| 7 | price close UK | weekdays at 5:00pm London (3:00am local, Tue–Sat) | +| 8 | price close US | weekdays at 4:30pm New York (7:30am local, Tue–Sat) | +| 9 | rating | Tue–Sat at 8:30am Sydney | +| 10 | milestone | Tue–Sat at 8:30am Sydney | +| 11 | podcasts Money Cafe | hourly at :01, weekdays 10am–9pm Sydney | +| 12 | podcasts Stocktake | hourly at :01, weekdays 10am–9pm Sydney | +| 13 | cal earnings | weekly, Sundays at 8:30am Sydney | +| 14 | cal ex-dividend | weekly, Sundays at 8:30am Sydney | +| 15 | shorts | weekly, Sundays at 8:30am Sydney | +| 16 | performance | weekly, Sundays at 8:30am Sydney | +| 17 | price | weekly, Sundays at 8:30am Sydney | +| 18 | performance | monthly, the 2nd at 8:30am Sydney | +| 19 | price | monthly, the 2nd at 8:30am Sydney | +| 20 | performance | quarterly, 2 Jan/Apr/Jul/Oct at 8:30am Sydney | +| 21 | price | quarterly, 2 Jan/Apr/Jul/Oct at 8:30am Sydney | +| 22 | performance | bi-annual, 2 Jan and 2 Jul at 8:30am Sydney | +| 23 | price | bi-annual, 2 Jan and 2 Jul at 8:30am Sydney | +| 24 | performance | annual, 2 Jan at 8:30am Sydney | +| 25 | price | annual, 2 Jan at 8:30am Sydney | +| 26 | message: 🥳 Happy EOFY 🇬🇧 🇭🇰 🇮🇳 🇰🇷 🇳🇿 🇯🇵 🇿🇦 | annual, 31 Mar at 8:30am Sydney | +| 27 | message: SPCX IPO anniversary | annual, 12 Jun at 8:30am Sydney | +| 28 | message: 💸 Realise capital gains/losses by EOFY June 30 (AU) | annual, 23 Jun at 8:30am Sydney | +| 29 | message: 🥳 Happy EOFY 🇦🇺 🇧🇩 🇪🇬 🇪🇹 🇰🇪 🇳🇵 🇵🇰 | annual, 30 Jun at 8:30am Sydney | +| 30 | message: ✍️ Submit 'Notice of Intent to Claim' … (AU) | annual, 7 Jul at 8:30am Sydney | +| 31 | message: finbot's birthday | annual, 18 Aug at 8:30am Sydney | +| 32 | message: 😓 Self-service individual tax returns are due Oct 31 (AU) | annual, 25 Oct at 8:30am Sydney | +| 33 | message: ubank draw 6/6 … (AU) | once, 28 Oct 2026 at 8:30am Sydney | +| 34 | message: 😰 Self-service individual tax returns are due today (AU) | annual, 31 Oct at 8:30am Sydney | +| 35 | message: SPCX 5th mensiversary … | once, 12 Nov 2026 at 8:30am Sydney | + +* **Market reports run in their market's zone**, so they keep to its session through its daylight saving: + * pre-market US at 7:15am New York; + * mid-session about halfway through: 1:00pm Sydney for the ASX, 12:45pm New York for the NYSE; + * the close 30 minutes after it: 4:30pm Sydney for the ASX, 4:30pm Hong Kong, 5:00pm London and 4:30pm New York. +* **The messages marked (AU)** are default jobs only where the `country_code` preference is `AU`. +* **A one-off leaves the list** once its date has passed. + +## How jobs run +* A job is kept by its chat's network and id, in `finbot_chat_jobs.json` in the data directory, so switching a network + between pull and HTTP mode keeps it. Only a chat's switches and the jobs it added are stored: the default jobs come + from finbot's code, so a change to them reaches every chat. +* Each chat keeps its own record of what a report has told it (trades, rating changes, record highs, podcast + episodes), so the same report scheduled in two chats tells each of them once. A run from **Run now** or `/jobs run` + is a scheduled run: it posts to the chat and updates the same record. A change counts at once, with no reload. +* **A job can only run finbot's own reports.** It names an action from a fixed table in `lib/jobs.py` (a report, or + posting a message), with arguments that report's own parser has checked: never a command, a script or code. A stored + job naming anything else is logged and left out, never run. +* **The system jobs**, fixed in `lib/jobs.py` and listed by `/jobs system`, keep the cache full and tidy: + * `sharesight_warm` every 30 minutes and `yahoo_warm` every 8 hours (see [Caching](caching.md)); + * `cleanup 30` weekly, which deletes cache files older than 30 days. + + A chat cannot add, switch or run them, and a failure of one goes to the log. +* **They run in `bot.py` itself**, each job in a thread of its own, sharing the bot's user, sandbox, environment, data + directory, caches and connections. Charts are drawn one at a time, and a slow report holds up no chat command. There is no cron to set up: remove any old crontab entries that ran finbot's scripts, or their reports come twice. -* A run that was due while the bot was down (a restart, a deploy) still fires once if it was due within the last - `scheduler_catchup` minutes (default 5). A restart right after a run does not repeat it. A job that is switched off - does not fire, and does not catch up once it is switched on again. -* A job still running when it is next due is skipped. A command still running after 5 minutes is reported once, and - left to finish (a thread cannot be stopped, but every request it makes has a timeout); until it does, it is not - started again. Each command of a job has 5 minutes of its own. -* A failure is reported to the job's chat. The bot logs each job as it runs (`journalctl -u finbot`). +* **Catch-up:** a run that was due while the bot was down (a restart, a deploy) still fires once, if it was due within + the last `scheduler_catchup` minutes (default 5). A restart right after a run does not repeat it. A job that is + switched off does not fire, and does not catch up when it is switched on again. +* **Daylight saving:** a time the clocks skip runs when they reach the first minute after the gap, and a time they + repeat runs once. +* **Slow and failing jobs:** a job still running when it is next due is skipped. A job still running after 5 minutes is + reported once and left to finish (a thread cannot be stopped, but every request it makes has a timeout); until then + it is not started again. A failure is reported to the job's chat, and the bot logs each job as it runs + (`journalctl -u finbot`). + +**From an earlier version:** +* The cron jobs in `finbot_schedule.json` are not carried over. Its chats start with the default jobs, and the file + can be deleted. +* `reminder.json` is read no more: its reminders are default jobs now. Delete the data directory's copy. diff --git a/doc/slack.md b/doc/slack.md index ea72cd3d..989905e0 100644 --- a/doc/slack.md +++ b/doc/slack.md @@ -98,8 +98,8 @@ slackBotToken=xoxb-your-bot-token 2. Restart finbot, and invite the bot to the channel: `/invite @finbot`. In Socket Mode the log says `slack: Socket Mode (no slackOutgoingWebhook)`. 3. Send `/help` (or `@finbot help`) in the channel, or in a direct message to the bot in its _Messages_ tab, where - `.help` works too (and in the channels, with `--every-message`). Each channel schedules its own reports with - `/jobs add` ([Scheduling](scheduling.md)). + `.help` works too (and in the channels, with `--every-message`). Each channel has the default scheduled jobs, + which `/jobs` changes ([Scheduling](scheduling.md)). An app set up by hand before takes the manifest too: paste it into _Features > App Manifest_ and _Save Changes_; Slack shows what changes, and asks to reinstall the app when its scopes do. @@ -123,5 +123,5 @@ drawn and uploaded anew (Slack cannot replace a file). An upload that no message are past, so the bot's files do not pile up. If Slack will not show a chart with its buttons, the chart is uploaded into the channel without them; if an upload fails (a missing scope, say), the report goes out as text. The log says why. -`/jobs run` and `/jobs toggle` list the channel's jobs with a button for each, as on Telegram; a button from a list -that has since changed puts the current list in its place. +`/jobs` lists the channel's jobs with the menu's buttons, as on Telegram; a button from a list that has since changed +puts the current list in its place. diff --git a/doc/telegram.md b/doc/telegram.md index 0ba22b54..d9d3d8e8 100644 --- a/doc/telegram.md +++ b/doc/telegram.md @@ -15,7 +15,7 @@ Telegram: Everything else is the same in both: the scheduled reports with their charts, the chat commands (`/price`, `/history`, `/performance` and the rest, [the list](commands.md#supported-commands), or as messages, `.price`), period buttons -under charts and the buttons of `/jobs run` and `/jobs toggle`, "typing…" while a command works, and a keyboard of the +under charts and the buttons of the `/jobs` menu, "typing…" while a command works, and a keyboard of the common commands in a direct chat with the bot. As finbot starts, it sets the bot's command menu, which Telegram shows when `/` is typed: each command with its usage @@ -40,7 +40,7 @@ arrive after the switch. Do this after step 2, as it stops you adding the bot to groups too. 5. Start finbot (`sudo systemctl restart finbot`). The log says `telegram: long polling for updates`. 6. Send `/help` in the chat. In a direct chat with the bot, the reply also brings a keyboard of the common commands. - Each chat schedules its own reports with `/jobs add` ([Scheduling](scheduling.md)). + Each chat has the default scheduled jobs, which `/jobs` changes ([Scheduling](scheduling.md)). ``` telegramBotToken=0000000000:AAAAAAAAAAAAAAAAAAAAAAAAAA From bf07e01b92c90cb9ecbfe8401eaa9958b665d42b Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 07:13:03 +0000 Subject: [PATCH 07/41] =?UTF-8?q?.jobs:=20=E2=80=A0=20marks=20a=20default?= =?UTF-8?q?=20job;=20All=20switches=20every=20job=20on=20or=20off?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - In every list, a default job's name carries †, in place of the Delete screen's "(default)". Each page ends with a key to the marks it shows: "* not run yet, † default (no delete, only disable)", or whichever half applies. - Enable/disable has an All button: - "All off" when every job of the chat is on, switching them all off; - otherwise "All on", switching them all on. It says what it did, and a press drawn for a list that has since changed switches nothing. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 2 +- doc/scheduling.md | 12 +++++++---- lib/jobs.py | 14 ++++++++++++ lib/reports/about.py | 44 +++++++++++++++++++++---------------- lib/worker.py | 8 +++++-- tests/test_scheduler.py | 48 ++++++++++++++++++++++++++++++----------- 6 files changed, 91 insertions(+), 37 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 9c958c96..97e79134 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -107,7 +107,7 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm - `lib/config.py`: `config.env` (finbot.env, fixed at start) and `config.prefs` (preferences.conf, a new object on each reload). `lib/jobs.py`: the scheduler: each chat's jobs, the default ones and those it added (`JOBS_FILE`, by chat key: the defaults it switched off, the jobs it added; `see` makes a chat known at its first command; `add`, - `delete`, `toggle`, `run`, `upcoming`, which `.jobs` calls through `jobs.current`; `parse_add` reads `.jobs add`), + `delete`, `toggle`, `switch_all`, `run`, `upcoming`, which `.jobs` calls through `jobs.current`; `parse_add` reads `.jobs add`), and the fixed `SYSTEM` jobs, its shared state under one lock. A job is an action of `ACTIONS` (the report modules, `message` among them) with the arguments its `job()` takes, never a command; one stored that is anything else is logged and left out. Each job runs in the bot's process, in a thread (`run_in_process`), reported once if still diff --git a/doc/scheduling.md b/doc/scheduling.md index 1a779313..09c6a4d5 100644 --- a/doc/scheduling.md +++ b/doc/scheduling.md @@ -9,14 +9,18 @@ and its home chats. ## The menu `/jobs` lists the chat's jobs, numbered, a page at a time: the default jobs, then those added here. Each line shows ✅ -(on) or ❌ (off), a `*` if the job has never run, its name, and its schedule in words: +(on) or ❌ (off), a `*` if the job has never run, its name with a `†` if it is a default job, and its schedule in +words. The page ends with a key to the marks it shows: ``` -1. ✅ trades · every 20 mins, Mon 9am – Sat 3pm Sydney -3. ✅* price midsession US · weekdays at 12:45pm New York (3:45am local, Tue–Sat) +1. ✅ trades† · every 20 mins, Mon 9am – Sat 3pm Sydney +3. ✅* price midsession US† · weekdays at 12:45pm New York (3:45am local, Tue–Sat) + +* not run yet, † default (no delete, only disable) ``` Under it are buttons, each screen drawn in place of the last: * **Enable/disable**: a button per job, its number, switch and name (`3 ✅ price midsession US`); a press switches the - job on or off. + job on or off. **All on** switches every job of the chat on at once, or **All off**, shown when every job is on, + switches them all off. * **Delete**: a button per job added here. A default job has no button: switch it off instead. * **Add**: the report, how often (one-off, daily, weekdays, weekly, monthly, quarterly, bi-annual, annual or repeating), then the date, the day, or a repeat's interval and days, then the hour. **Back** takes a step back, **Cancel** goes diff --git a/lib/jobs.py b/lib/jobs.py index 24107216..5f15a7d6 100644 --- a/lib/jobs.py +++ b/lib/jobs.py @@ -492,6 +492,20 @@ def append(record): self.change(chat, append) return self.number(chat, job.key) + def switch_all(self, chat, on): + """Switch every job of a chat on, or off: the default ones and those it added.""" + keys = [item.key for item in catalog.items(config.prefs.country_code)] + def switch(record): + """Set every switch, as stored.""" + if on: + record.pop('off', None) + else: + record['off'] = keys + for entry in record['jobs']: + if isinstance(entry, dict): + entry['on'] = on + self.change(chat, switch) + def delete(self, job): """Delete a job its chat added (from job() or pick()); a default one raises ScheduleError: switch it off.""" if job.item: diff --git a/lib/reports/about.py b/lib/reports/about.py index ecf885b7..6b93ba5f 100644 --- a/lib/reports/about.py +++ b/lib/reports/about.py @@ -15,6 +15,7 @@ '.jobs add REPORT SCHEDULE: add one (.jobs add lists the reports)', '.jobs next: the next runs', '.jobs system: the system jobs'] # what a chat without buttons types instead TYPED_ACTIONS = {'on': 'toggle', 'del': 'del', 'run': 'run'} +NOT_RUN, DEFAULT = '* not run yet', '† default (no delete, only disable)' # the key to a list's marks PAGE_ROOM = 500 # characters of a screen besides its jobs' lines: its headings, a note, the key to * JOB_ROW = 4 # job buttons in a row MOST_JOBS = {'discord': 16} # job buttons on a page: Discord takes five rows of five, one kept for the others; else 40 @@ -31,21 +32,22 @@ INTERVALS = (5, 10, 15, 20, 30, 60, 120, 180, 240, 360, 720) # minutes between a repeat's runs, as Add offers them def jobs_screen(service, chat, screen='top', page=0, note=None, tappable=True): - """A screen of the .jobs menu that lists a chat's jobs (by its key), numbered: the default ones, then those added - here, each ✅ (on) or ❌ (off), * where it has never run, its name and its schedule in words, by the timezone - preference's clock (lib/when.py). top has what can be done (Enable/disable, Delete, Add, Run now, Next); on, del and - run a button per job, which switches it, deletes it (one added here: a default one has none) or runs it now. A page - at a time, as many jobs as a message of the network takes (◀ and More ▶ for the others), after a note of what the - last press did. A one-off gone by is deleted first. Returns (lines, buttons); where the chat takes no buttons - (tappable false), the whole list and what to type instead.""" + """A screen of the .jobs menu that lists a chat's jobs (by its key), numbered: the default ones (†), then those + added here, each ✅ (on) or ❌ (off), * where it has never run, its name and its schedule in words, by the timezone + preference's clock (lib/when.py), then the key to the marks shown. top has what can be done (Enable/disable, + Delete, Add, Run now, Next); on, del and run a button per job, which switches it (and All, every job), deletes it + (one added here: a default one has none) or runs it now. A page at a time, as many jobs as a message of the + network takes (◀ and More ▶ for the others), after a note of what the last press did. A one-off gone by is deleted + first. Returns (lines, buttons); where the chat takes no buttons (tappable false), the whole list and what to type + instead.""" scheduler = jobs.current if scheduler is None: return ['No schedule is running'], None scheduler.prune(chat) own = scheduler.chat_jobs(chat) running, local, now = scheduler.running(chat), formats.zone().key, scheduler.clock() - entries = [(number, job, job_entry(service, scheduler, number, job, running, local, now) + - (' (default)' if screen == 'del' and job.item else '')) for number, job in enumerate(own, 1)] + entries = [(number, job, job_entry(service, scheduler, number, job, running, local, now)) + for number, job in enumerate(own, 1)] parts = paged(entries, service) if tappable else [entries] page = min(max(page, 0), len(parts) - 1) shown = parts[page] @@ -59,11 +61,13 @@ def jobs_screen(service, chat, screen='top', page=0, note=None, tappable=True): kind = job.item is not None lines += ['', service.bold('Default jobs:' if kind else 'Added here:')] lines.append(line) - if not all(scheduler.has_run(job) for _, job, _ in shown): - lines += ['', '* not run yet'] + key = [text for text, marked in ((NOT_RUN, not all(scheduler.has_run(job) for _, job, _ in shown)), + (DEFAULT, any(job.item for _, job, _ in shown))) if marked] + if key: + lines += ['', ', '.join(key)] if not tappable: return lines + [''] + (TYPED if screen == 'top' else [f'Send .jobs {TYPED_ACTIONS[screen]} NUMBER']), None - return lines, screen_buttons(screen, page, len(parts), shown, jobs.list_checksum(own)) + return lines, screen_buttons(screen, page, len(parts), shown, jobs.list_checksum(own), all(job.on for job in own)) def paged(entries, service): """The jobs' entries (number, job, line) a page at a time: as many as fit a message of the network with room to @@ -78,10 +82,10 @@ def paged(entries, service): size += len(entry[2]) + 1 return parts -def screen_buttons(screen, page, pages, shown, crc): +def screen_buttons(screen, page, pages, shown, crc, every_on=False): """A list screen's buttons: top's, what can be done; on, del and run's, one for each job shown that the screen - acts on, ' ' (with its ✅ or ❌ to switch it), JOB_ROW to a row, then Back; ◀ and More ▶ to the other - pages.""" + acts on, ' ' (with its ✅ or ❌ to switch it), JOB_ROW to a row, then on's All, which switches every + job of the chat on (All off where every_on: off), and Back; ◀ and More ▶ to the other pages.""" if screen == 'top': rows = [[buttons.menu_button('Enable/disable', 'on', [0], crc), buttons.menu_button('Delete', 'del', [0], crc)], [buttons.menu_button('Add', 'add', [], crc), buttons.menu_button('Run now', 'run', [0], crc), @@ -92,6 +96,9 @@ def screen_buttons(screen, page, pages, shown, crc): screen, [page, number], crc) for number, job, _ in shown if screen != 'del' or not job.item] rows = [tapped[i:i + JOB_ROW] for i in range(0, len(tapped), JOB_ROW)] nav = [buttons.menu_button('Back', 'top', [0], crc)] + if screen == 'on': + nav.insert(0, buttons.menu_button('All off' if every_on else 'All on', 'on', [page, 'alloff' if every_on + else 'allon'], crc)) if page > 0: nav.append(buttons.menu_button('◀', screen, [page - 1], crc)) if page < pages - 1: @@ -99,11 +106,12 @@ def screen_buttons(screen, page, pages, shown, crc): return rows + ([nav] if nav else []) def job_entry(service, scheduler, number, job, running=(), local=None, now=None): - """'5. ✅ price close AU · weekdays at 4:30pm Sydney': a job in .jobs, with its number, whether it is on (and * - where it has never run), its name, its schedule in words, and whether it is running now.""" + """'5. ✅ price close AU† · weekdays at 4:30pm Sydney': a job in .jobs, with its number, whether it is on (and * + where it has never run), its name (and † for a default job), its schedule in words, and whether it is running.""" mark = ('✅' if job.on else '❌') + ('' if scheduler.has_run(job) else '*') + name = service.escape(job.name) + ('†' if job.item else '') schedule = job.schedule.describe(local or formats.zone().key, now or scheduler.clock()) - return f"{number}. {mark} {service.escape(job.name)} · {schedule}" + (' (running)' if job.key in running else '') + return f"{number}. {mark} {name} · {schedule}" + (' (running)' if job.key in running else '') def job_line(number, job, outcome): """'3. trades: done': a job's number in the .jobs list, its name, and how its run went.""" diff --git a/lib/worker.py b/lib/worker.py index 1b165bc7..daa8e3e6 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -94,9 +94,13 @@ def menu_screen(service, chat, pressed): note = None if len(args) == 2 and screen in ('on', 'del', 'run'): own, number = scheduler.chat_jobs(key), args[1] - if not (hmac.compare_digest(jobs.list_checksum(own), pressed.checksum) and number.isdigit() and - 1 <= int(number) <= len(own)): + every = screen == 'on' and number in ('allon', 'alloff') # All: every job switched on, or off + if not (hmac.compare_digest(jobs.list_checksum(own), pressed.checksum) and (every or number.isdigit() and + 1 <= int(number) <= len(own))): note = 'The jobs have changed: here they are now' + elif every: + scheduler.switch_all(key, number == 'allon') + note = f"Every job is {number.removeprefix('all')} now" else: job = own[int(number) - 1] try: diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index c2b94bb7..9544b958 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -512,9 +512,10 @@ def test_add_list_and_del(self): self.assertEqual(self.say('.jobs add shorts daily at 10pm'), ['4. ✅* shorts · daily at 10:00pm', 'Added: .jobs lists them all']) self.assertEqual(self.say('.jobs'), ['Jobs here', '', 'Default jobs:', - '1. ✅* rating · Tue–Sat at 8:30am Sydney', - '2. ✅* price close US · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', '', 'Added here:', - '3. ✅* trades · every 20 mins, weekdays UTC', '4. ✅* shorts · daily at 10:00pm', '', '* not run yet']) + '1. ✅* rating† · Tue–Sat at 8:30am Sydney', + '2. ✅* price close US† · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', '', 'Added here:', + '3. ✅* trades · every 20 mins, weekdays UTC', '4. ✅* shorts · daily at 10:00pm', '', + '* not run yet, † default (no delete, only disable)']) self.assertEqual([[button.label for button in row] for row in self.replies[-1][2]], [['Enable/disable', 'Delete'], ['Add', 'Run now', 'Next']]) self.assertEqual(self.say('.jobs del 3'), ['Deleted 3. trades · every 20 mins, weekdays UTC']) @@ -591,7 +592,7 @@ def test_a_job_the_defaults_have_switches_that_one_on(self): """A job typed as a default one is (in a zone that keeps its clock: Melbourne for Sydney) switches that one on, and nothing more is stored.""" self.say('.jobs toggle 1') - self.assertEqual(self.say('.jobs add rating tue-sat at 8:30am'), ['1. ✅* rating · Tue–Sat at 8:30am Sydney', + self.assertEqual(self.say('.jobs add rating tue-sat at 8:30am'), ['1. ✅* rating† · Tue–Sat at 8:30am Sydney', 'Added: .jobs lists them all']) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('trades every 20 mins weekdays')]}) @@ -614,17 +615,20 @@ def test_toggle_a_job_off_and_on(self): self.assertEqual([job.name for job, _ in self.s.due(self.clock.now)], ['rating', 'trades']) def test_a_job_is_marked_till_it_first_runs(self): - """* marks a job that has never run, on its schedule or by hand, and the list says what * means.""" + """* marks a job that has never run, on its schedule or by hand, and † a default job; the list ends with what + the marks it shows mean.""" self.say('.jobs run 3') lines = self.say('.jobs') self.assertIn('3. ✅ trades · every 20 mins, weekdays UTC', lines) - self.assertIn('1. ✅* rating · Tue–Sat at 8:30am Sydney', lines) + self.assertIn('1. ✅* rating† · Tue–Sat at 8:30am Sydney', lines) self.clock.now = at(13, 21, 31) # Wednesday 8:31am in Sydney: rating has run, New York's close not yet self.s.tick() lines = self.say('.jobs') - self.assertIn('1. ✅ rating · Tue–Sat at 8:30am Sydney', lines) - self.assertIn('2. ✅* price close US · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', lines) - self.assertEqual(lines[-1], '* not run yet') + self.assertIn('1. ✅ rating† · Tue–Sat at 8:30am Sydney', lines) + self.assertIn('2. ✅* price close US† · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', lines) + self.assertEqual(lines[-1], '* not run yet, † default (no delete, only disable)') + self.say('.jobs run 2') + self.assertEqual(self.say('.jobs')[-1], '† default (no delete, only disable)') def test_a_one_off_gone_by_is_deleted_as_the_list_is_shown(self): """A one-off job whose time has gone by (run or off) is deleted as the list is next shown, never on a tick.""" @@ -697,9 +701,9 @@ def test_enable_disable_flips_the_switch_in_the_button_and_the_message(self): lines, rows, data = self.press(f'm|on|0|{crc}') self.assertEqual(self.toasts, ['Updating…']) self.assertEqual(lines[0], 'Tap a job to switch it on or off') - self.assertEqual(rows, [['1 ✅ rating', '2 ✅ price close US', '3 ✅ trades'], ['Back']]) + self.assertEqual(rows, [['1 ✅ rating', '2 ✅ price close US', '3 ✅ trades'], ['All off', 'Back']]) lines, rows, data = self.press(data['1 ✅ rating']) - self.assertIn('1. ❌* rating · Tue–Sat at 8:30am Sydney', lines) + self.assertIn('1. ❌* rating† · Tue–Sat at 8:30am Sydney', lines) self.assertEqual(rows[0][0], '1 ❌ rating') self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT]['off'], ['rating']) self.press(data['1 ❌ rating']) @@ -707,11 +711,31 @@ def test_enable_disable_flips_the_switch_in_the_button_and_the_message(self): lines, rows, _ = self.press(data['Back']) self.assertEqual((lines[0], rows[0]), ('Jobs here', ['Enable/disable', 'Delete'])) + def test_all_switches_every_job_on_or_off(self): + """Enable/disable's All switches every job of the chat off where every one is on, else on, and says so; its + label says which. Pressed for another list, it switches nothing.""" + lines, rows, data = self.press(f'm|on|0|{self.crc()}') + self.assertEqual(rows[-1], ['All off', 'Back']) + lines, rows, data = self.press(data['All off']) + self.assertEqual(lines[0], 'Every job is off now') + self.assertEqual(rows, [['1 ❌ rating', '2 ❌ price close US', '3 ❌ trades'], ['All on', 'Back']]) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'off': ['rating', 'price-close-us'], + 'jobs': [entry('trades every 20 mins weekdays', on=False)]}) + lines, rows, data = self.press(data['3 ❌ trades']) + self.assertEqual(rows[-1], ['All on', 'Back']) # some on, some off: All switches them all on + lines, rows, data = self.press(data['All on']) + self.assertEqual(lines[0], 'Every job is on now') + self.assertTrue(all(job.on for job in self.s.chat_jobs(CHAT))) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('trades every 20 mins weekdays')]}) + self.assertEqual(self.press('m|on|0|alloff|0000')[0][0], 'The jobs have changed: here they are now') + self.assertTrue(all(job.on for job in self.s.chat_jobs(CHAT))) + def test_delete_offers_only_the_jobs_added_here(self): """Delete lists every job, the default ones marked so and without a button, and a button for each one added here; a press deletes it, saying so; a forged press on a default job deletes nothing.""" lines, rows, data = self.press(f'm|del|0|{self.crc()}') - self.assertIn('1. ✅* rating · Tue–Sat at 8:30am Sydney (default)', lines) + self.assertIn('1. ✅* rating† · Tue–Sat at 8:30am Sydney', lines) + self.assertEqual(lines[-1], '* not run yet, † default (no delete, only disable)') self.assertEqual(rows, [['3 trades'], ['Back']]) self.assertEqual(self.press(f'm|del|0|1|{self.crc()}')[0][0], 'rating is a default job: switch it off instead' ' (.jobs toggle)') From 189c684513719751e55c3fc090d30c9fc7026686 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 07:44:34 +0000 Subject: [PATCH 08/41] =?UTF-8?q?.jobs=20Add=20down=20to=20the=20minute,?= =?UTF-8?q?=20=E2=80=A0=20defaults;=20DMs'=20jobs=20start=20off;=20quieter?= =?UTF-8?q?=20log?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add: - Every frequency's screens go down to the minute: - one-off: the date, the hour, the minute; - daily and weekdays: the hour, the minute; - weekly: the day of the week, the hour, the minute; - monthly, quarterly and bi-annual: the day of the month, the hour, the minute (two pages on Discord); - annual: the month, the day, the hour, the minute; - repeating: the interval, then the days. - Each screen marks its default with † and says "† indicates default". Nothing is skipped. Direct messages: - A direct message with the bot starts with its default jobs off. That is a person's chat on Telegram, a D channel on Slack, and on Discord a message from no server, which its updates now say (inbound.Message.dm). - Once, at the next start, every job of each direct message chat finbot knows is switched off. Discord's are found by asking Discord, and the old schedule file's DMs are made known with their jobs off. It is recorded in finbot_upgrades.json, so it runs at one start only, and is tried again at the next start if Discord cannot be reached. Log: - These go to DEBUG: a button press, its acknowledgement, the message edited in place (logged='debug' on every network), and a success answered to a network. - The tickers asked of Yahoo for trades are logged sorted. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 11 ++- bot.py | 9 +- doc/configuration.md | 2 +- doc/scheduling.md | 23 +++-- lib/discord_common.py | 14 ++- lib/discord_websocket.py | 2 +- lib/inbound.py | 1 + lib/jobs.py | 82 ++++++++++++------ lib/migrate.py | 74 ++++++++++++++-- lib/net.py | 11 ++- lib/reports/about.py | 180 +++++++++++++++++++++++++++------------ lib/reports/trades.py | 2 +- lib/slack_common.py | 13 +-- lib/telegram_common.py | 7 +- lib/worker.py | 4 +- tests/test_discord.py | 26 +++--- tests/test_finbot.py | 6 +- tests/test_inbound.py | 4 +- tests/test_reports.py | 12 +++ tests/test_schedule.py | 97 ++++++++++++++++++++- tests/test_scheduler.py | 67 ++++++++++----- 21 files changed, 489 insertions(+), 158 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 97e79134..f12e88b7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -106,7 +106,9 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm `with_holdings(chat)`, the stocks a report in that chat covers; a report takes the chat from its destination. - `lib/config.py`: `config.env` (finbot.env, fixed at start) and `config.prefs` (preferences.conf, a new object on each reload). `lib/jobs.py`: the scheduler: each chat's jobs, the default ones and those it added (`JOBS_FILE`, by - chat key: the defaults it switched off, the jobs it added; `see` makes a chat known at its first command; `add`, + chat key: the defaults it switched off, the jobs it added; a direct message's defaults start off, `'defaults': + 'off'` with those switched on; `see` makes a chat known at its first command, a DM as `inbound.Message.dm` or + `is_dm` says; `add`, `delete`, `toggle`, `switch_all`, `run`, `upcoming`, which `.jobs` calls through `jobs.current`; `parse_add` reads `.jobs add`), and the fixed `SYSTEM` jobs, its shared state under one lock. A job is an action of `ACTIONS` (the report modules, `message` among them) with the arguments its `job()` takes, never a command; one stored that is anything else is @@ -118,7 +120,8 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm name and schedule; the period of performance and price follows how often it runs (`period_args`). `lib/migrate.py`: once, on an upgrade, the old home chats (the retired telegramChatID, slackChannelID and discordChannelID, which it alone reads, from the environment) become chats finbot knows, and the watchlist, the - plans and the reports' shared records become theirs. + plans and the reports' shared records become theirs; one-off upgrades run at one start only (`once`, recorded in + `UPGRADES`): `dm_jobs_off` switched off the jobs of the DMs known before DMs started with them off. - `tests/common.py`: `FinbotCase` (offline fixtures). `self.command('.price')` returns what was sent: `('text', lines)` or `('photo', size, caption)`. `wsgi_environ(path, body, **headers)` is a request for `bot.main`. `ProviderCase` runs the real parsers on the answers saved under `doc/` (`sample`), as `Resp` and `serve` return @@ -147,7 +150,9 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm tabs counting 4 (an edit quotes whole lines); a test checks all but README.md, CLAUDE.md and `doc/`. - Log with `lib/log.py`, at a level, with print()'s arguments: `log.error` (failed and lost: a message not sent), `log.warn` (failed and got over: a retry, a fallback, a refused request), `log.info` (what finbot does: a message - sent, a job run; `net` logs each request it sends), `log.debug` (cache misses, hits and writes). The + sent, a job run; `net` logs each request it sends), `log.debug` (cache misses, hits and writes; the routine of a + press: the press, its acknowledgement and the edit in place, sent with `logged='debug'`, and a success answered to + a network). The `log_level` preference sets what reaches the log; `lib/config.py` prints its own errors (it loads before preferences exist). Everything goes to stderr: systemd discards stdout (`StandardOutput=null`), so the bot and its reports never print there (a test checks); only the util/ tools, run by hand, do. diff --git a/bot.py b/bot.py index 4c7cf3e9..792b2cda 100755 --- a/bot.py +++ b/bot.py @@ -102,10 +102,11 @@ def handle(environ, start_response): comes from that network (before its body is read, when the proof is a header), read the update (CONTENT_LENGTH bytes, up to MAX_BODY) and take it (take): the network gets its answer at once.""" def respond(response): - """Answer the request: a status line and a body (logged at INFO, naming the network, not the path). Work that - must wait for the answer starts once the server has sent it.""" + """Answer the request: a status line and a body (logged, naming the network, not the path: at DEBUG when it is + a success, the routine; else at INFO). Work that must wait for the answer starts once the server has sent it.""" network = environ.get(ROUTED) or 'no network' - log.info(f"{environ['REQUEST_METHOD']} from {network}: we answered {response.status}") + (log.debug if response.status.startswith('2') else log.info)( + f"{environ['REQUEST_METHOD']} from {network}: we answered {response.status}") start_response(response.status, [('Content-type', 'application/json')]) return Sent([response.body], response.then) if response.then else [response.body] def print_headers(say=log.info): @@ -211,7 +212,7 @@ def take(network, update): def dispatch(event): """Hand a message or a button press to lib/worker, in a thread of its own, which it returns.""" if isinstance(event, inbound.Press): - log.info(f"[{event.service.capitalize()} button]:", event.user, "chat", event.chat_id, event.data) + log.debug(f"[{event.service.capitalize()} button]:", event.user, "chat", event.chat_id, event.data) target = worker.process_callback else: log.info(f"[{event.service.capitalize()}]:", event.user, event.text) diff --git a/doc/configuration.md b/doc/configuration.md index 59cadc89..ae790789 100644 --- a/doc/configuration.md +++ b/doc/configuration.md @@ -74,7 +74,7 @@ A change takes effect on a reload. A true/false preference takes `true`/`false`, | `cache_seconds` | `43200` | Cache lifetime in seconds (12 hours) | | `performance_cache_seconds` | `2400` | Cache lifetime of Sharesight performance reports in seconds: longer than the 30 minutes between `sharesight_warm` runs, so the holdings never run out | | `http_timeout` | `18` | Seconds to wait on outbound HTTP requests | -| `log_level` | `INFO` | What the log shows: that level and those above it, syslog style. `ERROR`: what failed and was lost (a message not sent); `WARN`: what failed and was got over (a retry, a fallback, a request refused); `INFO`: what finbot does (each message sent, each job run, each request sent and answered); `DEBUG`: each cache miss, hit and write too. Replaces `debug` | +| `log_level` | `INFO` | What the log shows: that level and those above it, syslog style. `ERROR`: what failed and was lost (a message not sent); `WARN`: what failed and was got over (a retry, a fallback, a request refused); `INFO`: what finbot does (each message sent, each job run, each request sent, each request refused); `DEBUG`: the routine of a button press too (the press, its acknowledgement, the message edited in place), each request answered with success, and each cache miss, hit and write. Replaces `debug` | | `print_headers` | `False` | Log inbound request headers | A key in the wrong file, or one that is not listed, stops finbot starting; on a reload it is reported and nothing is applied. A key finbot no longer reads (`alliterate`, `chunk_maxlines`) is ignored, and the log at start says to delete it. diff --git a/doc/scheduling.md b/doc/scheduling.md index 09c6a4d5..71ca1019 100644 --- a/doc/scheduling.md +++ b/doc/scheduling.md @@ -3,9 +3,10 @@ Each chat has its own jobs: reports and messages that run on a schedule and post to that chat. Anyone in the chat manages them there, with `/jobs` (or `/job`; as a message, also `.cron`, `.crons` or `.schedule`). -Every chat starts with the [default jobs](#the-default-jobs), each switched on: a chat finbot knows has them all until -it switches some off. finbot knows a chat from its first command, and knows the chats of an earlier version's schedule -and its home chats. +Every chat starts with the [default jobs](#the-default-jobs): switched on in a group or a channel, and switched off in a +direct message with the bot, until the chat switches them. finbot knows a chat from its first command, and knows the +chats of an earlier version's schedule and its home chats. A direct message is a person's chat on Telegram, a `D…` +channel on Slack, and on Discord a message from no server. ## The menu `/jobs` lists the chat's jobs, numbered, a page at a time: the default jobs, then those added here. Each line shows ✅ @@ -22,9 +23,17 @@ Under it are buttons, each screen drawn in place of the last: job on or off. **All on** switches every job of the chat on at once, or **All off**, shown when every job is on, switches them all off. * **Delete**: a button per job added here. A default job has no button: switch it off instead. -* **Add**: the report, how often (one-off, daily, weekdays, weekly, monthly, quarterly, bi-annual, annual or repeating), - then the date, the day, or a repeat's interval and days, then the hour. **Back** takes a step back, **Cancel** goes - back to the list. A message's text cannot come from a button, so its screen says how to type one. +* **Add**: the report and how often, then, a screen each, the rest down to the minute: + * one-off: the date (the next 14 days), the hour, the minute; + * daily and weekdays: the hour, the minute; + * weekly: the day of the week, the hour, the minute; + * monthly, quarterly and bi-annual: the day of the month, the hour, the minute; + * annual: the month, the day, the hour, the minute; + * repeating: the interval, then every day or weekdays. + + Each screen marks its default with `†` (`Sat†`, `09†`, `09:00†`) and says `† indicates default`. **Back** takes a + step back, **Cancel** goes back to the list. A message's text cannot come from a button, so its screen says how to + type one. * **Run now**: a button per job; a press runs it now, as its schedule would, and says how it went. * **Next**: the next runs, in time order. @@ -159,4 +168,6 @@ move as either side's daylight saving changes. **From an earlier version:** * The cron jobs in `finbot_schedule.json` are not carried over. Its chats start with the default jobs, and the file can be deleted. +* At the first start of this version, every job of each direct message chat finbot knew was switched off, once + (Discord's direct messages as Discord says). It is recorded in `finbot_upgrades.json`, so it never happens again. * `reminder.json` is read no more: its reminders are default jobs now. Delete the data directory's copy. diff --git a/lib/discord_common.py b/lib/discord_common.py index c249a6f6..bfe3aa23 100644 --- a/lib/discord_common.py +++ b/lib/discord_common.py @@ -139,7 +139,7 @@ def interaction(self, update): given = ' '.join(str(option.get('value', '')) for option in data.get('options') or []).strip() text = f"/{data['name']} {given}".strip() return inbound.Message(self.service, channel, f"<@{person.get('id')}>", text, self.bot_name, - self.name(person), interaction=through) + self.name(person), interaction=through, dm=self.dm(update)) if kind == self.AUTOCOMPLETE and data.get('name'): options = data.get('options') or [] typed = next((str(option.get('value', '')) for option in options if option.get('focused')), '') @@ -158,7 +158,13 @@ def message(self, update): return inbound.OK text = re.sub(r'<@!?(\d+)>', r'@\1', update['content']) return inbound.Message(self.service, str(update.get('channel_id')), f"<@{author['id']}>", text, self.bot_name, - self.name(author), str(update.get('id', ''))) + self.name(author), str(update.get('id', '')), dm=self.dm(update)) + + @staticmethod + def dm(update): + """Whether an interaction or a message is in a direct message with the bot: it comes from no server (no + guild_id, no member).""" + return not (update.get('guild_id') or update.get('member')) @staticmethod def name(person): @@ -261,14 +267,14 @@ def edit_chart(self, message, image, caption, buttons=None): image.seek(0) payload = {'content': transports.fit_caption(caption, self.dialect), 'attachments': [{'id': 0, 'filename': 'chart.png'}], 'components': components(buttons or [])} - call('PATCH', self.message_path(message), **message_body(payload, image)) + call('PATCH', self.message_path(message), **message_body(payload, image), logged='debug') def edit_text(self, message, text, buttons=None): """Edit a message's text, and its buttons unless buttons is None ([] removes them); a failure is only logged.""" payload = {'content': text} if buttons is not None: payload['components'] = components(buttons) - try_call('PATCH', self.message_path(message), json=payload) + try_call('PATCH', self.message_path(message), json=payload, logged='debug') DISCORD_ROW = 5 # buttons in a row of components, and rows in a message: Discord's limits diff --git a/lib/discord_websocket.py b/lib/discord_websocket.py index fb6a6776..f38ccdb0 100644 --- a/lib/discord_websocket.py +++ b/lib/discord_websocket.py @@ -74,6 +74,6 @@ def receive(self, raw, take): def answer(self, update, answer): """Post an interaction's answer to its callback, then start the work waiting on it, if any.""" path = f"/interactions/{update['id']}/{update['token']}/callback" - discord_common.call('POST', path, json=json.loads(answer.body)) + discord_common.call('POST', path, json=json.loads(answer.body), logged='debug') if answer.then: answer.then() diff --git a/lib/inbound.py b/lib/inbound.py index 20147a00..9a9c5cc4 100644 --- a/lib/inbound.py +++ b/lib/inbound.py @@ -21,6 +21,7 @@ class Message: user_name: str # what to call them message_id: str = '' # the message itself: Slack shows ⏳ on it while finbot works interaction: str = '' # Discord's slash command: its application id and token, to answer through + dm: bool = None # whether the chat is a direct message with the bot; None where its key says (jobs.is_dm) @dataclasses.dataclass diff --git a/lib/jobs.py b/lib/jobs.py index 5f15a7d6..01334fc6 100644 --- a/lib/jobs.py +++ b/lib/jobs.py @@ -40,7 +40,9 @@ SYSTEM = (('sharesight_warm', (), {'every': 30, 'from': '00:14'}), ('yahoo_warm', (), {'every': 480, 'from': '00:07'}), ('cleanup', ('30',), {'days': ['sat'], 'at': '18:51'})) WARM = ('sharesight_warm', 'yahoo_warm') # the jobs that fill the cache, run at once after a flush too -JOBS_FILE = 'finbot_chat_jobs.json' # {chat key: {'off': [catalog keys], 'jobs': [{action, args, schedule, on}]}} +# {chat key: {'jobs': [{action, args, schedule, on}], 'off': [catalog keys]}}; a chat whose default jobs start off (a +# direct message) holds 'defaults': 'off' and 'on' instead, the catalog keys switched on +JOBS_FILE = 'finbot_chat_jobs.json' OLD_FILE = 'finbot_schedule.json' # each chat's cron jobs, from before the catalog: read for its chats alone STATE_FILE = 'finbot_jobs.json' # each job's last run, by key: a restart neither repeats a run nor loses one MESSAGE_AT = 8 * 60 + 30 # a message's time of day where none is typed: 8:30am @@ -184,22 +186,59 @@ def entry_key(chat, entry): return None +def item_on(record, key): + """Whether a default job (by its catalog key) is on in a chat's record: unless it is switched off, or, where the + default jobs start off (a direct message), where it is switched on.""" + if record.get('defaults') == 'off': + listed = record.get('on', []) + return isinstance(listed, list) and key in listed + listed = record.get('off', []) + return not (isinstance(listed, list) and key in listed) + + def switch_item(record, key, on): """Switch a default job on or off in a chat's record: whether it is on now.""" - off = [each for each in record.get('off', []) if each != key] + ([] if on else [key]) - if off: - record['off'] = off + starts_on = record.get('defaults') != 'off' + name = 'off' if starts_on else 'on' # the list of those switched from how they start + listed = [each for each in record.get(name, []) if each != key] + ([key] if on != starts_on else []) + if listed: + record[name] = listed else: - record.pop('off', None) + record.pop(name, None) return on -def know(keys): - """Make chats known (by key), each new one with the default jobs switched on and none added: how many were new.""" +def switch_record(record, on): + """Switch every job of a chat's record on, or off: those it added, and the default ones, which then start so.""" + record.pop('off', None) + record.pop('on', None) + if on: + record.pop('defaults', None) + else: + record['defaults'] = 'off' + for entry in record.get('jobs', []): + if isinstance(entry, dict): + entry['on'] = on + + +def is_dm(key): + """Whether a chat (by key) is a direct message with the bot, as its key tells: a person's positive id on Telegram, + an id starting D on Slack; None on Discord, whose key does not say.""" + network, _, chat = key.partition(':') + if network == 'telegram': + return not chat.startswith('-') + if network == 'slack': + return chat.startswith('D') + return None + + +def know(keys, dm=None): + """Make chats known (by key), each new one with none added and the default jobs switched on, or off in a direct + message: where dm is True (else as its key tells, is_dm). How many were new.""" with state.edit(JOBS_FILE, {}) as chats: new = [key for key in dict.fromkeys(keys) if key not in chats] for key in new: - chats[key] = {} + chats[key] = {'defaults': 'off'} if (is_dm(key) if dm is None else dm) else {} return len(new) @@ -268,10 +307,8 @@ def load(self): chats = self.stored_chats() items = catalog.items(config.prefs.country_code) for chat, record in chats.items(): - off = {key for key in record.get('off', []) if isinstance(key, str)} if isinstance(record.get('off'), - list) else set() - jobs += [make_job(item.action, item.args, item.schedule, chat, item.key not in off, item.key, item.name) - for item in items] + jobs += [make_job(item.action, item.args, item.schedule, chat, item_on(record, item.key), item.key, + item.name) for item in items] added = record.get('jobs', []) keys = set() for entry in added if isinstance(added, list) else []: @@ -452,12 +489,13 @@ def run(self, job): self.save_runs() return self.execute(job) - def see(self, chat): - """A chat seen, at a command of its: one finbot does not know yet starts with the default jobs, switched on.""" + def see(self, chat, dm=None): + """A chat seen, at a command of its: one finbot does not know yet starts with the default jobs, switched on, + or off in a direct message (dm, from the network, else as the chat's key tells: is_dm).""" with self.lock: known = chat in self.chats if not known: - know([chat]) + know([chat], dm) self.load() def change(self, chat, edit): @@ -494,17 +532,7 @@ def append(record): def switch_all(self, chat, on): """Switch every job of a chat on, or off: the default ones and those it added.""" - keys = [item.key for item in catalog.items(config.prefs.country_code)] - def switch(record): - """Set every switch, as stored.""" - if on: - record.pop('off', None) - else: - record['off'] = keys - for entry in record['jobs']: - if isinstance(entry, dict): - entry['on'] = on - self.change(chat, switch) + self.change(chat, lambda record: switch_record(record, on)) def delete(self, job): """Delete a job its chat added (from job() or pick()); a default one raises ScheduleError: switch it off.""" @@ -517,7 +545,7 @@ def toggle(self, job): def flip(record): """Flip the job's switch, as stored.""" if job.item: - return switch_item(record, job.item, job.item in record.get('off', [])) + return switch_item(record, job.item, not item_on(record, job.item)) entry = record['jobs'][self.index(record, job)] entry['on'] = not entry.get('on', True) return entry['on'] diff --git a/lib/migrate.py b/lib/migrate.py index e53cfae5..e8741cad 100644 --- a/lib/migrate.py +++ b/lib/migrate.py @@ -1,16 +1,20 @@ -"""Once, as an upgraded finbot starts: what the bot kept for all chats at once becomes each old home chat's, the chats -that the retired telegramChatID, slackChannelID and discordChannelID name (read from the environment for this alone). -Each home chat becomes a chat finbot knows, with the default jobs (lib/catalog.py); the watchlist and the plans are -copied to each; and so are the reports' records of what they told the chats (trades, ratings, record highs, podcast -episodes), so that nothing old is told again. +"""Once, as an upgraded finbot starts. The jobs of the direct message chats finbot knows are switched off, as a direct +message now starts with its default jobs off (dm_jobs_off, recorded in UPGRADES so that it runs at one start only). +And what the bot kept for all chats at once becomes each old home chat's, the chats that the retired telegramChatID, +slackChannelID and discordChannelID name (read from the environment for this alone): each becomes a chat finbot knows, +with the default jobs (lib/catalog.py); the watchlist and the plans are copied to each; and so are the reports' records +of what they told the chats (trades, ratings, record highs, podcast episodes), so that nothing old is told again. This module goes once every install has started with it: it is its only user.""" import copy +import datetime import glob import os from lib import log from lib import config +from lib import delivery +from lib import discord_common from lib import jobs from lib import state from lib import watchlist @@ -19,6 +23,7 @@ # kept for all chats, each now one record per chat; and each podcast feed's seen file RECORDS = (watchlist.FILE, worker.PLANS, trades.STATE_FILE, rating.STATE_FILE, milestone.STATE_FILE) +UPGRADES = 'finbot_upgrades.json' # {name: when}: the one-off upgrades done, each at one start only # the home chats of older versions: (network, the retired finbot.env key of its chat, the key of its bot token) HOMES = (('telegram', 'telegramChatID', 'telegramBotToken'), ('slack', 'slackChannelID', 'slackBotToken'), ('discord', 'discordChannelID', 'discordBotToken')) @@ -31,8 +36,10 @@ def old_homes(): def run(): - """Make the old home chats' own what was kept for all chats, where that has not been done yet. Without any (a new - install, or the retired keys deleted), there is nowhere to put it, so it waits.""" + """Switch off the jobs of the direct message chats finbot knows, once (dm_jobs_off). Make the old home chats' own + what was kept for all chats, where that has not been done yet; without any (a new install, or the retired keys + deleted), there is nowhere to put it, so it waits.""" + once('dm jobs off', dm_jobs_off) homes = old_homes() if not homes: return @@ -48,6 +55,59 @@ def run(): record(name, homes) +def once(name, upgrade): + """Run an upgrade unless it has run at an earlier start (UPGRADES). One that fails is logged, and tried again at + the next start.""" + try: + done = state.load(UPGRADES, {}) + except (OSError, ValueError) as e: + log.warn(f'upgrade: {name} waits, {UPGRADES} is unreadable: {e}') + return + done = done if isinstance(done, dict) else {} + if name in done: + return + try: + upgrade() + except Exception as e: + log.warn(f'upgrade: {name} failed, and is tried again at the next start:', delivery.error_line(e)) + return + state.save(UPGRADES, {**done, name: datetime.datetime.now(datetime.timezone.utc).isoformat()}) + + +def dm_jobs_off(): + """Switch off every job of each direct message chat finbot knows, now that one starts with its default jobs off: + Telegram's and Slack's as their keys tell (jobs.is_dm), Discord's as Discord says. The chats of the old schedule + file (jobs.OLD_FILE) that are direct messages become known first, with their jobs off.""" + chats = state.load(jobs.JOBS_FILE, {}) + old = state.load(jobs.OLD_FILE, {}) if os.path.exists(state.path(jobs.OLD_FILE)) else {} + chats, old = (data if isinstance(data, dict) else {} for data in (chats, old)) + keys = [key for key in dict.fromkeys([*chats, *old]) if isinstance(key, str)] + dms = [key for key in keys if jobs.is_dm(key)] + if config.env.discordBotToken: + dms += [key for key in keys if key.startswith('discord:') and discord_dm(key.partition(':')[2])] + jobs.know([key for key in dms if key not in chats], dm=True) + with state.edit(jobs.JOBS_FILE, {}) as stored: + for key in dms: + if isinstance(stored.get(key), dict): + jobs.switch_record(stored[key], False) + if dms: + log.info(f"schedule: every job switched off in {len(dms)} direct message chat{'' if len(dms) == 1 else 's'}" + f" ({', '.join(dms)}), as a direct message now starts with its jobs off") + + +def discord_dm(channel): + """Whether a Discord channel (by id) is a direct message with the bot, as Discord says; one it says nothing of (a + channel the bot has left, an id that is none) is not.""" + if not (channel.isascii() and channel.isdigit()): + return False + try: + answer = discord_common.call('GET', f'/channels/{channel}') + except RuntimeError as e: # refused: the channel is gone, or the bot may not see it + log.warn(f'upgrade: what Discord channel {channel} is, Discord will not say:', delivery.error_line(e)) + return False + return isinstance(answer, dict) and answer.get('type') in (1, 3) # a direct message, or a group one + + def record(name, homes): """A file's record kept for all chats at once (anything but {'chats': ...}) becomes each home chat's.""" try: diff --git a/lib/net.py b/lib/net.py index eef3313c..9edf19ae 100644 --- a/lib/net.py +++ b/lib/net.py @@ -31,11 +31,14 @@ def request(method, url, **kwargs): return session.request(method, url, **timeouts(logged(method, url, kwargs))) def logged(method, url, kwargs): - """The request's arguments without `logged`; the request noted at INFO first, unless logged=False (Telegram's long - polling, asked again and again): its method and URL, without secrets or the scheme, cut to SHOWN bytes.""" - if kwargs.pop('logged', True): + """The request's arguments without `logged`; the request noted first, its method and URL without secrets or the + scheme, cut to SHOWN bytes: at INFO; at DEBUG with logged='debug' (the routine of a press: its acknowledgement, the + message edited in place); not at all with logged=False (Telegram's long polling, asked again and again).""" + shown = kwargs.pop('logged', True) + if shown: line = f"{method} {redact(url).split('://', 1)[-1]}".encode() - log.info(line.decode() if len(line) <= SHOWN else line[:SHOWN - 3].decode(errors='ignore') + '…') + (log.debug if shown == 'debug' else log.info)(line.decode() if len(line) <= SHOWN else + line[:SHOWN - 3].decode(errors='ignore') + '…') return kwargs def timeouts(kwargs): diff --git a/lib/reports/about.py b/lib/reports/about.py index 6b93ba5f..caad270a 100644 --- a/lib/reports/about.py +++ b/lib/reports/about.py @@ -27,9 +27,15 @@ ('milestone', 'milestone', ()), ('message', 'message', None)) FREQUENCIES = (('o', 'one-off'), ('d', 'daily'), ('k', 'weekdays'), ('w', 'weekly'), ('m', 'monthly'), ('q', 'quarterly'), ('b', 'bi-annual'), ('a', 'annual'), ('r', 'repeating')) -DAYS = {'d': 'daily', 'k': 'weekdays', 'm': {'day': 1, 'months': ()}, 'q': {'day': 1, 'months': when.QUARTERS}, - 'b': {'day': 1, 'months': when.HALVES}, 'a': {'day': 1, 'months': (1,)}} # the days of Add's frequencies +# Add's choices after how often, in turn; and the default of each, marked † on its button +LEVELS = {'o': ('date', 'hour', 'minute'), 'd': ('hour', 'minute'), 'k': ('hour', 'minute'), + 'w': ('weekday', 'hour', 'minute'), 'm': ('day', 'hour', 'minute'), 'q': ('day', 'hour', 'minute'), + 'b': ('day', 'hour', 'minute'), 'a': ('month', 'day', 'hour', 'minute'), 'r': ('interval', 'repeat')} +USUAL = {'frequency': 'd', 'weekday': 5, 'day': 1, 'month': 1, 'hour': 9, 'minute': 0, 'interval': 60, 'repeat': 'd'} +MORE = 'p1' # Add's data for the second page of the days of a month INTERVALS = (5, 10, 15, 20, 30, 60, 120, 180, 240, 360, 720) # minutes between a repeat's runs, as Add offers them +MINUTES = tuple(range(0, 60, 5)) # the minutes Add offers +MOST_DAYS = {'discord': 20} # days of the month on a page: Discord takes 25 buttons, elsewhere all 31 def jobs_screen(service, chat, screen='top', page=0, note=None, tappable=True): """A screen of the .jobs menu that lists a chat's jobs (by its key), numbered: the default ones (†), then those @@ -152,22 +158,27 @@ def moment_words(moment, local, zone=None): return said def add_screen(service, chat, steps): - """A step of Add, from the choices made so far (steps, from the button pressed: the report's number, how often, - then the date, the day, or a repeat's interval and its days, then the hour): (lines, buttons) of the next choice, - or (None, (action, arguments, schedule)) once every choice is made, at a time by the timezone preference's clock. - Back takes a step back, Cancel goes back to the list. Raises ValueError for choices that are none (forged data).""" - crc = jobs.list_checksum(jobs.current.chat_jobs(chat)) - local = formats.zone() - steps = list(steps) - # Back from the first step goes to the list too, but its data differs from Cancel's: Slack and Discord take a - # button's data as its id, one of its own in a message + """A step of Add, from the choices made so far (steps, from the button pressed: the report's number, how often, then + the LEVELS of that frequency in turn: a date, a day of the week or of the month, a month, the hour and the minute, + or a repeat's interval and its days): (lines, buttons) of the next choice, its default (USUAL) marked †, or (None, + (action, arguments, schedule)) once every choice is made, at a time by the timezone preference's clock. Back takes a + step back, Cancel goes back to the list. Raises ValueError for choices that are none (forged data).""" + scheduler = jobs.current + crc, local = jobs.list_checksum(scheduler.chat_jobs(chat)), formats.zone() + tomorrow = scheduler.clock().astimezone(local).date() + datetime.timedelta(days=1) + steps, page = list(steps), 0 + if steps and steps[-1] == MORE: + steps, page = steps[:-1], 1 back = buttons.menu_button('Back', *(('add', steps[:-1]) if steps else ('top', [])), crc) cancel = buttons.menu_button('Cancel', 'top', [0], crc) - def choices(heading, offered, row, lines=()): - """A choice's screen: its heading and lines, its buttons (label, data) `row` to a row, then Back and Cancel.""" - tapped = [buttons.menu_button(label, 'add', [*steps, data], crc) for label, data in offered] - return [service.bold(heading), *lines], [tapped[i:i + row] for i in range(0, len(tapped), row)] + [[back, - cancel]] + def choices(heading, offered, row, lines=(), usual=None, nav=None): + """A choice's screen: its heading and lines, and '† indicates default' where one is (usual: its data); its + buttons (label, data), `row` to a row, the default's label marked †; then Back and Cancel (or nav).""" + tapped = [buttons.menu_button(label + ('†' if data == usual else ''), 'add', [*steps, data], crc) + for label, data in offered] + key = [] if usual is None else ['', '† indicates default'] + return [service.bold(heading), *lines, *key], [tapped[i:i + row] for i in range(0, len(tapped), row)] + [ + nav or [back, cancel]] if not steps: return choices('Add a job: which report?', [(f'{number} {label}', number) for number, (label, _, _) in enumerate(REPORTS, 1)], 3, [f'{number}. {label}' for number, (label, _, _) in enumerate(REPORTS, 1)]) @@ -178,49 +189,108 @@ def choices(heading, offered, row, lines=()): '.jobs add message every 31 mar at 8:30am "🥳 Happy EOFY"', '.jobs add message on 28 oct 2026 "ubank draw 6/6"'], [[back, cancel]] if len(steps) == 1: - return choices(f'Add {label}: how often?', [(name, code) for code, name in FREQUENCIES], 3) - frequency = steps[1] - if frequency not in dict(FREQUENCIES): + return choices(f'Add {label}: how often?', [(name, code) for code, name in FREQUENCIES], 3, + usual=USUAL['frequency']) + frequency, levels = steps[1], LEVELS.get(steps[1]) + if levels is None: raise ValueError(f'no frequency {frequency!r}') - needed = 4 if frequency in 'owr' else 3 # the choices it takes, the report's and how often included - if len(steps) == 2 and frequency == 'o': - today = jobs.current.clock().astimezone(local).date() - dates = [today + datetime.timedelta(days=n) for n in range(14)] - return choices(f'Add {label} once: on which day?', [(f'{date:%a} {date.day} {date:%b}', f'{date:%y%m%d}') - for date in dates], 4, ['A later one is typed: .jobs add REPORT on 28 oct 2026 at 9am']) - if len(steps) == 2 and frequency == 'w': - return choices(f'Add {label} weekly: on which day?', [(day.title(), number) for number, day in - enumerate(when.DAYS)], 4) - if len(steps) == 2 and frequency == 'r': + made = steps[2:] + if len(made) > len(levels) or (page and levels[len(made):len(made) + 1] != ('day',)): + raise ValueError('no such choices') + values = {} + for level, data in zip(levels, made): + values[level] = chosen(level, data, values) + if len(made) == len(levels): + if frequency == 'r': + schedule = when.Schedule('daily' if values['repeat'] == 'd' else 'weekdays', every=values['interval'], + zone=local.key) + else: + if frequency == 'o': + days = {'date': values['date'].isoformat()} + elif frequency == 'w': + days = (when.DAYS[values['weekday']],) + elif 'day' in values: + days = {'day': values['day'], 'months': {'m': (), 'q': when.QUARTERS, 'b': when.HALVES}.get(frequency, + (values.get('month'),))} + else: + days = 'daily' if frequency == 'd' else 'weekdays' + schedule = when.Schedule(days, at=values['hour'] * 60 + values['minute'], zone=local.key) + schedule.check() + return None, (action, list(args) if args is not None else catalog.period_args(action, schedule), schedule) + level, said = levels[len(made)], so_far(frequency, values) + if level == 'date': + dates = [tomorrow + datetime.timedelta(days=n) for n in range(-1, 13)] + return choices(f'Add {label}, once: on which day?', [(f'{date:%a} {date.day} {date:%b}', f'{date:%y%m%d}') + for date in dates], 4, ['A later one is typed: .jobs add REPORT on 28 oct 2026 at 9am'], + f'{tomorrow:%y%m%d}') + if level == 'weekday': + return choices(f'Add {label}, weekly: on which day?', [(day.title(), number) for number, day in + enumerate(when.DAYS)], 4, usual=USUAL['weekday']) + if level == 'month': + return choices(f'Add {label}, annual: in which month?', [(month.title(), number) for number, month in + enumerate(when.MONTHS, 1)], 4, usual=USUAL['month']) + if level == 'day': + last = {2: 29, 4: 30, 6: 30, 9: 30, 11: 30}.get(values.get('month'), 31) + most = MOST_DAYS.get(service, 31) + if page * most >= last: + raise ValueError('no such page') + nav = [back, cancel] + if page: + nav.insert(0, buttons.menu_button('◀', 'add', steps, crc)) + elif last > most: + nav.insert(0, buttons.menu_button('More ▶', 'add', [*steps, MORE], crc)) + return choices(f'Add {label}, {said}: on which day of the month?', [(str(day), day) for day in + range(1, last + 1)][page * most:(page + 1) * most], 5, ['A month without that day has no run'], + USUAL['day'], nav) + if level == 'hour': + lines, rows = choices(f'Add {label}, {said}: at what hour?', [(f'{hour:02d}', hour) for hour in range(24)], 5, + [f"By {when.place(local.key)}'s clock"], USUAL['hour']) + hours = [button for row in rows[:-1] for button in row] + [back] # 24 and Back: Discord takes 25 buttons + return lines, [hours[i:i + 5] for i in range(0, 25, 5)] + if level == 'minute': + hour = values['hour'] + return choices(f'Add {label}, {said}: at what time?', [(f'{hour:02d}:{minute:02d}', minute) for minute in + MINUTES], 4, usual=USUAL['minute']) + if level == 'interval': return choices(f'Add {label}, repeating: how often?', [(when.repeat_words(minutes, 0, False).removeprefix( 'every '), minutes) for minutes in INTERVALS], 4, ['A window of hours is typed: .jobs add REPORT every 20' - ' mins weekdays from 10am to 4pm']) - if len(steps) == 3 and frequency == 'r': - choice(steps[2], 1, 1440, INTERVALS) - return choices(f'Add {label}, repeating: on which days?', [('every day', 'd'), ('weekdays', 'k')], 2) - if len(steps) == needed - 1: - hours = [buttons.menu_button(f'{hour:02d}', 'add', [*steps, hour], crc) for hour in range(24)] + [back] - return [service.bold(f'Add {label}: at what hour?'), f"By {when.place(local.key)}'s clock; a time such as" - ' 9:30am is typed: .jobs add REPORT weekly at 9:30am'], [hours[i:i + 5] for i in range(0, 25, 5)] - if len(steps) != needed: - raise ValueError('too many choices') + ' mins weekdays from 10am to 4pm'], USUAL['interval']) + return choices(f'Add {label}, {said}: on which days?', [('every day', 'd'), ('weekdays', 'k')], 2, + usual=USUAL['repeat']) + +def chosen(level, data, values): + """A choice's value at a level of Add, from a button's data (values: the choices before it). Raises ValueError for + one that is none.""" + if level == 'date': + if not (len(data) == 6 and data.isascii() and data.isdigit()): + raise ValueError(f'no date {data!r}') + return datetime.date(2000 + int(data[:2]), int(data[2:4]), int(data[4:])) + if level == 'day': + day = choice(data, 1, 31) + if 'month' in values: + datetime.date(2024, values['month'], day) # a day that month has, in a leap year + return day + if level in ('weekday', 'month', 'hour', 'minute', 'interval'): + return choice(data, *{'weekday': (0, 6), 'month': (1, 12), 'hour': (0, 23), 'minute': (0, 55, MINUTES), + 'interval': (1, 1440, INTERVALS)}[level]) + if data not in ('d', 'k'): + raise ValueError(f'no days {data!r}') + return data + +def so_far(frequency, values): + """How often a job being added runs, as far as it is chosen: 'weekly, Fridays', 'annual, in Mar', 'monthly'.""" if frequency == 'r': - if steps[3] not in ('d', 'k'): - raise ValueError(f'no days {steps[3]!r}') - schedule = when.Schedule('daily' if steps[3] == 'd' else 'weekdays', every=choice(steps[2], 1, 1440, - INTERVALS), zone=local.key) - else: - at = choice(steps[-1], 0, 23) * 60 - if frequency == 'o': - date = steps[2] - days = {'date': datetime.date(2000 + int(date[:2]), int(date[2:4]), int(date[4:6])).isoformat()} - elif frequency == 'w': - days = (when.DAYS[choice(steps[2], 0, 6)],) - else: - days = DAYS[frequency] - schedule = when.Schedule(days, at=at, zone=local.key) - schedule.check() - return None, (action, list(args) if args is not None else catalog.period_args(action, schedule), schedule) + return when.repeat_words(values['interval'], 0, False) if 'interval' in values else 'repeating' + if 'date' in values: + return f"once, {when.date_words(values['date'])}" + if 'weekday' in values: + return f"weekly, {when.plural(when.DAYS[values['weekday']])}" + if 'day' in values: + months = {'m': (), 'q': when.QUARTERS, 'b': when.HALVES}.get(frequency, (values.get('month'),)) + return when.days_words({'day': values['day'], 'months': months}) + if 'month' in values: + return f"annual, in {when.MONTHS[values['month'] - 1].title()}" + return dict(FREQUENCIES)[frequency].replace('one-off', 'once') def choice(text, low, high, among=None): """A choice's number from a button's data, from low to high (and among those offered, where given). Raises diff --git a/lib/reports/trades.py b/lib/reports/trades.py index dda496df..20e8ab9d 100644 --- a/lib/reports/trades.py +++ b/lib/reports/trades.py @@ -60,7 +60,7 @@ def gather(days, portfolio_select, known): if raw and config.prefs.trades_use_yahoo: tickers = {markets.transform_to_yahoo(trade['symbol'], trade['market']) for trade in raw} try: - log.info("Fetching tickers from Yahoo: ", tickers) + log.info("Fetching tickers from Yahoo:", ', '.join(sorted(tickers))) quotes = yahoo.fetch(tickers) except Exception as e: log.warn("Warning: could not fetch Yahoo:", e) diff --git a/lib/slack_common.py b/lib/slack_common.py index 817c94a5..d0b8d19f 100644 --- a/lib/slack_common.py +++ b/lib/slack_common.py @@ -22,14 +22,15 @@ SLACK_REUSE_SECONDS = 15 * 60 # a period pressed within this long of its chart's upload shows that upload again -def call(method, as_app=False, **data): +def call(method, as_app=False, logged=True, **data): """POST a Slack Web API method (form-encoded) as the bot, or with the app-level token (as_app, for Socket Mode's - apps.connections.open), and return the answer; an error raises RuntimeError.""" + apps.connections.open), and return the answer; an error raises RuntimeError. logged: as net logs the request.""" key = 'slackAppToken' if as_app else 'slackBotToken' token = getattr(config.env, key) if not token: raise RuntimeError(f"Slack {method}: needs {key} in finbot.env") - r = net.post('https://slack.com/api/' + method, headers={'Authorization': 'Bearer ' + token}, data=data) + r = net.post('https://slack.com/api/' + method, headers={'Authorization': 'Bearer ' + token}, data=data, + logged=logged) try: output = r.json() except ValueError: @@ -223,7 +224,7 @@ def reshow(self, message, buttons): _, file_id, caption = max(recent) try: call('chat.update', channel=self.where, ts=message, text=caption[:SECTION_LIMIT], - blocks=json.dumps(chart_blocks(caption, buttons, file_id))) + blocks=json.dumps(chart_blocks(caption, buttons, file_id)), logged='debug') except Exception as e: log.warn("Slack chat.update with an earlier upload failed, uploading anew:", net.redact(e)) return False @@ -239,7 +240,7 @@ def edit_chart(self, message, image, caption, buttons=None): file_id = self.unshared(image.read() if hasattr(image, 'read') else bytes(image)) blocks = chart_blocks(caption, buttons or [], file_id) until_shown(lambda: call('chat.update', channel=self.where, ts=message, text=caption[:SECTION_LIMIT], - blocks=json.dumps(blocks))) + blocks=json.dumps(blocks), logged='debug')) self.delete(self.shows(message, file_id, caption, buttons)) def shows(self, message, file_id, caption=None, buttons=None): @@ -284,7 +285,7 @@ def edit_text(self, message, text, buttons=None): if buttons is not None: data['blocks'] = json.dumps(text_blocks(text, buttons)) try: - call('chat.update', **data) + call('chat.update', logged='debug', **data) except Exception as e: log.warn("Slack chat.update failed:", net.redact(e)) diff --git a/lib/telegram_common.py b/lib/telegram_common.py index 8aba0bd8..f09e0b50 100644 --- a/lib/telegram_common.py +++ b/lib/telegram_common.py @@ -62,7 +62,8 @@ def pinChatMessage(chat_id, message_id): def answerCallbackQuery(callback_query_id, text=None): """Acknowledge a button press (the client's spinner stops), `text` as a toast. A failure is only logged.""" - try_call('answerCallbackQuery', json={'callback_query_id': callback_query_id} | ({'text': text} if text else {})) + try_call('answerCallbackQuery', json={'callback_query_id': callback_query_id} | ({'text': text} if text else {}), + logged='debug') class Telegram(inbound.Network): @@ -179,7 +180,7 @@ def edit_chart(self, message, image, caption, buttons=None): if hasattr(image, 'seek'): image.seek(0) try: - call('editMessageMedia', data=data, files={'photo': ('image.png', image)}) + call('editMessageMedia', data=data, files={'photo': ('image.png', image)}, logged='debug') except RuntimeError as e: if 'not modified' not in str(e): raise @@ -192,7 +193,7 @@ def edit_text(self, message, text, buttons=None): 'link_preview_options': self.NO_PREVIEWS} if buttons is not None: payload['reply_markup'] = self.markup(buttons) - try_call('editMessageText', json=payload) + try_call('editMessageText', json=payload, logged='debug') def typing(self): """sendChatAction: "typing…" for a few seconds.""" diff --git a/lib/worker.py b/lib/worker.py index daa8e3e6..3674752e 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -164,9 +164,9 @@ def process_request(message): request = Request(dialects.of(message.service), message.chat_id, message.user, message.user_name, message.bot_name, message.message_id, message.interaction) command, args = read(message.text, message.bot_name) - if command and jobs.current: # a chat finbot does not know yet starts with the default jobs + if command and jobs.current: # a chat finbot does not know yet starts with the default jobs (off in a DM) try: - jobs.current.see(request.dest.key) + jobs.current.see(request.dest.key, message.dm) except Exception as e: log.warn('schedule: cannot take on', request.dest.key, delivery.error_line(e)) if command: diff --git a/tests/test_discord.py b/tests/test_discord.py index a39b28c7..18ffe840 100644 --- a/tests/test_discord.py +++ b/tests/test_discord.py @@ -21,7 +21,8 @@ def interaction(kind, **fields): TYPING = interaction(4, id='902', data={'name': 'price', 'options': [{'name': 'arguments', 'type': 3, 'value': '5 7', 'focused': True}]}) # /price, its arguments being typed PRESS = interaction(3, id='901', data={'custom_id': 'c|h|y|T1', 'component_type': 2}, message={'id': '333'}) -MESSAGE = {'id': '700', 'channel_id': '77', 'author': {'id': '5', 'username': 'rob'}, 'content': '<@99> price 7d'} +MESSAGE = {'id': '700', 'channel_id': '77', 'guild_id': '1', 'author': {'id': '5', 'username': 'rob'}, + 'content': '<@99> price 7d'} # in a server's channel class Now: @@ -44,12 +45,12 @@ def setUp(self): self.patch(discord_common.READER, 'bot_id', '99') def test_a_slash_command_is_the_message_it_stands_for(self): - """/price with its arguments is the message '/price 5 7d', answered through its interaction; in a DM the person - is the interaction's user.""" - expected = inbound.Message('discord', '77', '<@5>', '/price 5 7d', '@99', 'Rob', '', '42/tok') + """/price with its arguments is the message '/price 5 7d', answered through its interaction; in a DM (no + member, no server) the person is the interaction's user, and the message says it is a DM.""" + expected = inbound.Message('discord', '77', '<@5>', '/price 5 7d', '@99', 'Rob', '', '42/tok', dm=False) self.assertEqual(discord_common.READER.parse(COMMAND), expected) dm = {k: v for k, v in COMMAND.items() if k != 'member'} | {'user': {'id': '5', 'username': 'rob'}} - self.assertEqual(discord_common.READER.parse(dm), dataclasses.replace(expected, user_name='rob')) + self.assertEqual(discord_common.READER.parse(dm), dataclasses.replace(expected, user_name='rob', dm=True)) self.assertEqual(discord_common.READER.parse({**COMMAND, 'data': {'name': 'beta'}}).text, '/beta') def test_a_press_is_a_press_of_its_message(self): @@ -59,9 +60,11 @@ def test_a_press_is_a_press_of_its_message(self): def test_a_message_by_a_person_reads_with_its_mentions_as_slacks(self): """A gateway message's mentions read as @id, so that '@bot price 7d' calls .price; one by a bot or a webhook - (this bot's replies included) is not answered.""" + (this bot's replies included) is not answered. One from no server is a DM.""" message = discord_common.READER.parse(MESSAGE) - self.assertEqual(message, inbound.Message('discord', '77', '<@5>', '@99 price 7d', '@99', 'rob', '700')) + self.assertEqual(message, inbound.Message('discord', '77', '<@5>', '@99 price 7d', '@99', 'rob', '700', + dm=False)) + self.assertTrue(discord_common.READER.parse({k: v for k, v in MESSAGE.items() if k != 'guild_id'}).dm) command, args = worker.read(message.text, message.bot_name) self.assertEqual((command.names[0], args.days), ('price', 7)) for other in ({**MESSAGE, 'author': {'id': '99', 'bot': True}}, {**MESSAGE, 'webhook_id': '1'}): @@ -493,7 +496,8 @@ def take(network, update): self.assertTrue(client.debug and client.intents.message_content and client.closed) self.assertFalse(self.library.VoiceClient.warn_nacl) self.assertEqual(taken, ['<@99> price 7d', 2]) - self.assertEqual(self.calls, [('POST', '/interactions/900/tok/callback', {'json': {'type': 5}})]) + self.assertEqual(self.calls, [('POST', '/interactions/900/tok/callback', {'json': {'type': 5}, + 'logged': 'debug'})]) self.assertEqual(order, [('work', 1)]) self.assertIn('discord: the gateway failed, trying again in 5 s: Improper token', err.getvalue()) @@ -503,7 +507,8 @@ def test_arguments_being_typed_over_the_gateway_get_their_suggestions_through_th with contextlib.redirect_stderr(io.StringIO()): discord_websocket.DiscordGateway().poll(bot.take) self.assertEqual(self.calls, [('POST', '/interactions/902/tok/callback', {'json': {'type': 8, 'data': { - 'choices': [{'name': '5 7d', 'value': '5 7d'}]}}})]) + 'choices': [{'name': '5 7d', 'value': '5 7d'}]}}, + 'logged': 'debug'})]) @unittest.skipUnless(SigningKey, 'PyNaCl is not installed') def test_a_press_over_the_gateway_is_the_press_the_endpoint_gets(self): @@ -526,7 +531,8 @@ def test_a_press_over_the_gateway_is_the_press_the_endpoint_gets(self): self.join_workers() self.assertEqual(len(presses), 2) self.assertEqual(presses[0], presses[1]) - self.assertEqual(self.calls[0][1:], ('/interactions/901/tok/callback', {'json': {'type': 6}})) + self.assertEqual(self.calls[0][1:], ('/interactions/901/tok/callback', {'json': {'type': 6}, + 'logged': 'debug'})) def test_discord_is_listened_to_with_its_token_and_no_endpoint(self): """The gateway is used with the bot token and no discordOutgoingWebhook; with the URL (http mode), or without diff --git a/tests/test_finbot.py b/tests/test_finbot.py index 167b67c3..c161f55b 100644 --- a/tests/test_finbot.py +++ b/tests/test_finbot.py @@ -48,8 +48,8 @@ def test_a_message_job_runs(self): self.assertEqual(self.sent, [('text', ['hello'])]) def test_bot_handler(self): - """A sticker is answered 200, with nothing logged but the request's line (INFO); git_version matches what git - says.""" + """A sticker is answered 200, with nothing logged at INFO, and at DEBUG only the request's line: a success is + the routine; git_version matches what git says.""" def post(message): """Post a Telegram message update to the WSGI handler: (its answer, what it logged).""" body = json.dumps({'update_id': 1, 'message': dict({'message_id': 5, 'chat': {'id': 55, 'type': 'private'}, @@ -59,6 +59,8 @@ def post(message): with contextlib.redirect_stderr(err): out = bot.main(env, lambda *a, **k: None) return out, err.getvalue() + self.assertEqual(post({'sticker': {'file_id': 'x'}}), ([b''], '')) + self.prefs(log_level='DEBUG') self.assertEqual(post({'sticker': {'file_id': 'x'}}), ([b''], 'POST from telegram: we answered 200 OK\n')) self.assertGreaterEqual(len(bot.git_version()), 7) head = subprocess.run(['git', 'rev-parse', 'HEAD'], cwd=bot.__file__.rsplit('/', 1)[0], capture_output=True, diff --git a/tests/test_inbound.py b/tests/test_inbound.py index 68f7e576..0336dd1d 100644 --- a/tests/test_inbound.py +++ b/tests/test_inbound.py @@ -154,7 +154,9 @@ def test_an_update_is_taken_however_it_arrived(self): ('slack', 'C1', '.trades')]) def test_button_presses_are_logged(self): - """A button press is logged with who pressed it (handle, first name or unknown), the chat and the button.""" + """A button press is logged at DEBUG (the routine of a menu) with who pressed it (handle, first name or + unknown), the chat and the button.""" + self.prefs(log_level='DEBUG') calls = [] self.patch(worker, 'process_callback', lambda *a: calls.append(a)) secret = {'HTTP_X_TELEGRAM_BOT_API_SECRET_TOKEN': 'tok'} diff --git a/tests/test_reports.py b/tests/test_reports.py index ef78eab4..c1455ebc 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -444,6 +444,18 @@ def test_a_trade_is_in_its_market_currency_not_the_brokerage_one(self): 'quantity': '10', 'price': '5.0', 'value': '50', 'brokerage_currency_code': 'AUD', 'holding_id': 7}]) self.assertIn(' USD 50 ', self.command('.trades')[0][1][-1]) + def test_the_tickers_asked_of_yahoo_are_logged_in_order(self): + """The trades' tickers that Yahoo is asked about are logged sorted, as names.""" + bought = (('WTC', 'ASX'), ('AMD', 'NASDAQ'), ('APE', 'ASX'), ('AMAT', 'NASDAQ')) + self.patch(sharesight, '_trades', lambda pid, days=None: [{'id': n, 'transaction_type': 'BUY', 'symbol': symbol, + 'market': market, 'transaction_date': time.strftime('%Y-%m-%d'), 'quantity': '1', 'price': '1', + 'value': '1', 'brokerage_currency_code': 'AUD', 'holding_id': n} for n, (symbol, market) in + enumerate(bought, 1)]) + self.patch(yahoo, 'fetch', lambda tickers: {t: market_entry(t) for t in tickers}) + with contextlib.redirect_stderr(io.StringIO()) as log: + trades.run(scheduled()) + self.assertIn('Fetching tickers from Yahoo: AMAT, AMD, APE.AX, WTC.AX\n', log.getvalue()) + def test_trades_asks_yahoo_only_for_new_trades(self): """A scheduled trades run asks Yahoo only about trades it has not reported, and nothing once all are known.""" fetched = [] diff --git a/tests/test_schedule.py b/tests/test_schedule.py index 650cbed3..2f0a67c0 100644 --- a/tests/test_schedule.py +++ b/tests/test_schedule.py @@ -5,7 +5,7 @@ from pathlib import Path from unittest import mock from tests.common import * -from lib import catalog, jobs, migrate, state, watchlist, when, worker +from lib import catalog, discord_common, inbound, jobs, migrate, state, watchlist, when, worker from lib.reports import cal, cleanup, message, milestone, performance, podcasts, price, rating, trades MELBOURNE = 'Australia/Melbourne' @@ -249,7 +249,8 @@ def test_each_home_chat_starts_with_the_default_jobs(self): """Each home chat becomes a chat finbot knows, with the default jobs switched on, unless it is known already.""" state.save(jobs.JOBS_FILE, {'slack:C1': {'off': ['trades']}}) self.assertIn('schedule: 1 old home chat start with the default jobs', self.migrate()) - self.assertEqual(state.load(jobs.JOBS_FILE), {'slack:C1': {'off': ['trades']}, 'telegram:1': {}}) + self.assertEqual(state.load(jobs.JOBS_FILE), {'slack:C1': {'off': ['trades']}, + 'telegram:1': {'defaults': 'off'}}) # a person's id: a direct message, whose default jobs start off self.assertNotIn('schedule:', self.migrate()) def test_with_no_home_chat_it_waits(self): @@ -285,5 +286,97 @@ def test_the_watchlist_plans_and_records_become_each_home_chats_once(self): self.assertIn(f'state: {rating.STATE_FILE} left as it is, unreadable', log) +class DirectMessages(FinbotCase): + """A direct message with the bot starts with its default jobs off; those of the DMs finbot knew already were + switched off once, as it started.""" + + def test_a_direct_message_starts_with_its_default_jobs_off(self): + """A chat new to finbot starts with its default jobs on, a direct message with them off: Telegram's (a person's + id) and Slack's (D…) as their keys tell, Discord's as its updates say. One switched on there is on; All on makes + them all start on.""" + jobs.know(['telegram:5', 'telegram:-100', 'slack:D1', 'slack:C1', 'discord:7']) + jobs.know(['discord:8'], dm=True) + self.assertEqual(state.load(jobs.JOBS_FILE), {'telegram:5': {'defaults': 'off'}, 'telegram:-100': {}, + 'slack:D1': {'defaults': 'off'}, 'slack:C1': {}, 'discord:7': {}, 'discord:8': {'defaults': 'off'}}) + s = jobs.Scheduler(clock=lambda: OCTOBER) + with contextlib.redirect_stderr(io.StringIO()): + s.load() + self.assertFalse(any(job.on for job in s.chat_jobs('telegram:5'))) + self.assertTrue(all(job.on for job in s.chat_jobs('telegram:-100'))) + s.toggle(s.chat_jobs('telegram:5')[0]) + self.assertEqual(state.load(jobs.JOBS_FILE)['telegram:5'], {'defaults': 'off', 'on': ['trades']}) + self.assertEqual([job.on for job in s.chat_jobs('telegram:5')][:2], [True, False]) + s.switch_all('telegram:5', True) + self.assertEqual(state.load(jobs.JOBS_FILE)['telegram:5'], {}) + + def test_a_chat_seen_says_whether_it_is_a_direct_message(self): + """At a chat's first command it starts with its default jobs off where it is a direct message: Discord's update + says so, Telegram's chat id does.""" + self.env(discordBotToken='x') + s = jobs.Scheduler(clock=lambda: OCTOBER) + with contextlib.redirect_stderr(io.StringIO()): + s.load() + self.patch(jobs, 'current', s) + for chat, dm in (('88', True), ('77', False)): + worker.process_request(inbound.Message('discord', chat, '<@5>', '.help', '@99', 'rob', dm=dm)) + worker.process_request(inbound.Message('telegram', '5', '@u', '.help', BOT, 'U')) + self.assertEqual(state.load(jobs.JOBS_FILE), {'discord:88': {'defaults': 'off'}, 'discord:77': {}, + 'telegram:5': {'defaults': 'off'}}) + + def test_the_direct_messages_jobs_are_switched_off_once(self): + """As finbot first starts with this version, every job of each direct message chat it knows is switched off, + the default ones and those added: Telegram's and Slack's as their keys tell, Discord's as Discord says; and the + old schedule file's DMs become known, with their jobs off. Other chats are left as they are, and a job switched + on again afterwards stays on.""" + trades = {'action': 'trades', 'args': [], 'schedule': {'days': 'weekdays', 'every': 20, 'zone': 'UTC'}, + 'on': True} + state.save(jobs.JOBS_FILE, {'telegram:5': {'jobs': [trades]}, 'telegram:-100': {'jobs': [trades]}, + 'slack:D1': {'off': ['trades']}, 'discord:7': {}, 'discord:8': {}}) + state.save(jobs.OLD_FILE, {'telegram:6': [], 'discord:9': [], 'telegram:-6': []}) + self.env(discordBotToken='x') + asked = [] + def call(method, path, **kwargs): + """Discord's answer: channels 8 and 9 are direct messages.""" + asked.append(path) + return {'type': 1 if path in ('/channels/8', '/channels/9') else 0} + self.patch(discord_common, 'call', call) + with contextlib.redirect_stderr(io.StringIO()) as log: + migrate.run() + stored = state.load(jobs.JOBS_FILE) + self.assertEqual(stored['telegram:5'], {'defaults': 'off', 'jobs': [{**trades, 'on': False}]}) + for key in ('slack:D1', 'discord:8', 'telegram:6', 'discord:9'): + self.assertEqual(stored[key], {'defaults': 'off'}, key) + self.assertEqual((stored['telegram:-100'], stored['discord:7']), ({'jobs': [trades]}, {})) + self.assertNotIn('telegram:-6', stored) # an old chat that is no DM is known at the scheduler's first load + self.assertEqual(sorted(asked), ['/channels/7', '/channels/8', '/channels/9']) + self.assertIn('every job switched off in 5 direct message chats', log.getvalue()) + self.assertIn('dm jobs off', state.load(migrate.UPGRADES)) + state.save(jobs.JOBS_FILE, {**stored, 'telegram:5': {}}) # switched on again + migrate.run() + self.assertEqual((state.load(jobs.JOBS_FILE)['telegram:5'], len(asked)), ({}, 3)) + + def test_it_waits_for_discord_and_a_channel_discord_will_not_name_is_no_dm(self): + """If Discord cannot be reached, nothing is switched and the upgrade waits for the next start; a channel Discord + refuses to say anything of (the bot has left it) is no direct message.""" + state.save(jobs.JOBS_FILE, {'discord:8': {}, 'telegram:5': {}}) + self.env(discordBotToken='x') + def down(method, path, **kwargs): + """Discord unreachable.""" + raise ConnectionError('Discord unreachable') + self.patch(discord_common, 'call', down) + with contextlib.redirect_stderr(io.StringIO()) as log: + migrate.run() + self.assertIn('upgrade: dm jobs off failed, and is tried again at the next start', log.getvalue()) + self.assertEqual((state.load(jobs.JOBS_FILE), state.load(migrate.UPGRADES)), ({'discord:8': {}, + 'telegram:5': {}}, None)) + def refused(method, path, **kwargs): + """Discord refusing: the channel is gone.""" + raise RuntimeError(f'Discord {method} {path}: 404 Unknown Channel (10003)') + self.patch(discord_common, 'call', refused) + with contextlib.redirect_stderr(io.StringIO()): + migrate.run() + self.assertEqual(state.load(jobs.JOBS_FILE), {'discord:8': {}, 'telegram:5': {'defaults': 'off'}}) + + if __name__ == '__main__': unittest.main() diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index 9544b958..a020e6c0 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -719,7 +719,7 @@ def test_all_switches_every_job_on_or_off(self): lines, rows, data = self.press(data['All off']) self.assertEqual(lines[0], 'Every job is off now') self.assertEqual(rows, [['1 ❌ rating', '2 ❌ price close US', '3 ❌ trades'], ['All on', 'Back']]) - self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'off': ['rating', 'price-close-us'], + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'defaults': 'off', # the default jobs start off now 'jobs': [entry('trades every 20 mins weekdays', on=False)]}) lines, rows, data = self.press(data['3 ❌ trades']) self.assertEqual(rows[-1], ['All on', 'Back']) # some on, some off: All switches them all on @@ -782,42 +782,68 @@ def test_the_older_job_buttons_still_work(self): self.assertFalse(self.s.chat_jobs(CHAT)[2].on) def test_add_by_buttons_back_and_cancel(self): - """Add takes the report, how often, the day and the hour from buttons, by the timezone preference's clock, - Back a step back and Cancel back to the list; its last step adds the job and says so, the same job as typed.""" + """Add takes the report, how often, then the day, the hour and the minute from buttons, by the timezone + preference's clock, each choice's default marked † and the key under it; Back goes a step back and Cancel back + to the list; its last step adds the job and says so, the same job as typed.""" lines, rows, data = self.press(f'm|add|{self.crc()}') self.assertEqual(lines[0], 'Add a job: which report?') self.assertEqual(rows[1:], [['4 price close', '5 price', '6 performance'], ['7 cal earnings', '8 cal ex-dividend', '9 shorts'], ['10 rating', '11 milestone', '12 message'], ['Back', 'Cancel']]) lines, rows, data = self.press(data['5 price']) - self.assertEqual((lines[0], rows[1]), ('Add price: how often?', ['weekly', 'monthly', 'quarterly'])) + self.assertEqual((lines, rows[:2]), (['Add price: how often?', '', '† indicates default'], + [['one-off', 'daily†', 'weekdays'], ['weekly', 'monthly', 'quarterly']])) lines, rows, data = self.press(data['weekly']) - self.assertEqual(rows[:2], [['Mon', 'Tue', 'Wed', 'Thu'], ['Fri', 'Sat', 'Sun']]) + self.assertEqual(rows[:2], [['Mon', 'Tue', 'Wed', 'Thu'], ['Fri', 'Sat†', 'Sun']]) self.assertEqual(data['Cancel'], f'm|top|0|{self.crc()}') - lines, rows, data = self.press(data['Sat']) - self.assertEqual((len(rows), [len(row) for row in rows], rows[-1]), (5, [5] * 5, ['20', '21', '22', '23', - 'Back'])) + lines, rows, data = self.press(data['Fri']) + self.assertEqual(lines[0], 'Add price, weekly, Fridays: at what hour?') + self.assertEqual((len(rows), [len(row) for row in rows], rows[1][4], rows[-1]), (5, [5] * 5, '09†', + ['20', '21', '22', '23', 'Back'])) self.assertEqual(data['Back'], f'm|add|5|w|{self.crc()}') - lines, rows, data = self.press(data['09']) - self.assertEqual(lines, ['Added: 4. ✅* price · weekly, Saturdays at 9:00am']) + lines, rows, data = self.press(data['14']) + self.assertEqual((rows[0], rows[-1]), (['14:00†', '14:05', '14:10', '14:15'], ['Back', 'Cancel'])) + lines, rows, data = self.press(data['14:30']) + self.assertEqual(lines, ['Added: 4. ✅* price · weekly, Fridays at 2:30pm']) self.assertEqual(rows, [['Jobs', 'Add another']]) - self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT]['jobs'][-1], entry('price weekly', + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT]['jobs'][-1], entry('price every friday at 2:30pm', zone='Australia/Melbourne')) - self.assertIn('that job is here already', self.say('.jobs add price every saturday at 9am')[0]) + self.assertIn('that job is here already', self.say('.jobs add price weekly on friday at 14:30')[0]) + + def test_monthly_and_annual_take_their_day_hour_and_minute(self): + """Monthly asks for the day of the month, then the hour and the minute; annual first for its month, offering + only the days it has. On Discord the days of a month come on two pages.""" + lines, rows, data = self.press(f'm|add|6|m|{self.crc()}') + self.assertEqual((lines[0], rows[0], rows[-2], rows[-1]), ('Add performance, monthly: on which day of the' + ' month?', ['1†', '2', '3', '4', '5'], ['31'], ['Back', 'Cancel'])) + lines, rows, data = self.press(data['15']) + self.assertEqual(lines[0], 'Add performance, monthly, the 15th: at what hour?') + lines, rows, data = self.press(data['09†']) + lines, rows, data = self.press(data['09:00†']) + self.assertEqual(lines, ['Added: 4. ✅* performance · monthly, the 15th at 9:00am']) + lines, rows, data = self.press(f'm|add|5|a|{self.crc()}') + self.assertEqual(rows[0], ['Jan†', 'Feb', 'Mar', 'Apr']) + lines, rows, data = self.press(data['Feb']) + self.assertEqual((lines[0], rows[-2]), ('Add price, annual, in Feb: on which day of the month?', + ['26', '27', '28', '29'])) + self.assertEqual(about.add_screen(DISCORD, CHAT, ['5', 'm'])[1][-1][0].label, 'More ▶') + self.assertEqual(about.add_screen(DISCORD, CHAT, ['5', 'm', 'p1'])[1][0][0].label, '21') def test_a_repeat_a_one_off_and_a_message_by_buttons(self): - """Add's repeat takes its interval and days, a one-off its day among the next 14 and its hour; a message's text - cannot come from a button, so its screen says how to type one.""" + """Add's repeat takes its interval and days, a one-off its day among the next 14 (tomorrow the default), its + hour and its minute; a message's text cannot come from a button, so its screen says how to type one.""" crc = self.crc() self.assertEqual(self.press(f'm|add|1|r|20|k|{crc}')[0], ['Added: 4. ✅* trades · every 20 mins, weekdays']) lines, rows, data = self.press(f'm|add|9|o|{self.crc()}') - self.assertEqual((rows[0][0], len(sum(rows, [])) - 2), ('Wed 14 Oct', 14)) - self.assertEqual(self.press(data['Tue 20 Oct'].replace('|' + self.crc(), f'|8|{self.crc()}'))[0], - ['Added: 5. ✅* shorts · once, 20 Oct 2026 at 8:00am']) + self.assertEqual((rows[0][:2], len(sum(rows, [])) - 2), (['Wed 14 Oct', 'Thu 15 Oct†'], 14)) + lines, rows, data = self.press(data['Tue 20 Oct']) + lines, rows, data = self.press(data['08']) + self.assertEqual(self.press(data['08:00†'])[0], ['Added: 5. ✅* shorts · once, 20 Oct 2026 at 8:00am']) lines, rows, _ = self.press(f'm|add|12|{self.crc()}') self.assertIn('.jobs add message SCHEDULE "TEXT"', lines) self.assertEqual(rows, [['Back', 'Cancel']]) for data in (f'm|add|13|{crc}', f'm|add|5|x|{crc}', f'm|add|5|w|7|9|{crc}', f'm|add|5|d|24|{crc}', - f'm|add|1|r|7|k|{crc}', f'm|add|5|o|261340|9|{crc}', f'm|add|5|d|9|9|{crc}'): + f'm|add|1|r|7|k|{crc}', f'm|add|5|o|261340|9|{crc}', f'm|add|5|d|9|9|{crc}', f'm|add|5|d|9|0|0|{crc}', + f'm|add|5|a|2|30|{crc}', f'm|add|5|w|p1|{crc}', f'm|add|5|m|p1|{crc}', f'm|add|5|m|def|{crc}'): with self.subTest(data=data): self.assertEqual(self.press(data)[0][0], 'That is no step of Add') @@ -843,8 +869,11 @@ def fits(service, lines, markup): for page in range(6): with self.subTest(service=service, screen=screen, page=page): fits(service, *about.jobs_screen(service, CHAT, screen, page)) - for steps in ([], ['5'], ['5', 'w'], ['5', 'w', '5'], ['5', 'o'], ['5', 'o', '261020'], ['5', 'r'], + for steps in ([], ['5'], ['5', 'w'], ['5', 'w', '5'], ['5', 'w', '5', '14'], ['5', 'm'], ['5', 'm', 'p1'], + ['5', 'a'], ['5', 'a', '2'], ['5', 'a', '2', '29'], ['5', 'o'], ['5', 'o', '261020'], ['5', 'r'], ['5', 'r', '20'], ['12']): + if steps[-1:] == ['p1'] and service != DISCORD: # Telegram and Slack take every day on one page + continue with self.subTest(service=service, steps=steps): fits(service, *about.add_screen(service, CHAT, steps)) fits(service, *about.next_screen(service, CHAT)) From 20ca1d83635ea28ece61ea257df9ecc69fb71db1 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 07:53:22 +0000 Subject: [PATCH 09/41] .jobs: Cancel, not Back, on the screens the list leads to; Add's reports by name Enable/disable, Delete, Run now, Next and Add's first screen say Cancel, back to the list. Add's report buttons drop their numbers ("price", not "5 price"), and its message no longer lists them; Add's later screens keep Back, a step back, beside Cancel. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- doc/scheduling.md | 10 +++++----- lib/reports/about.py | 20 ++++++++++---------- tests/test_scheduler.py | 39 ++++++++++++++++++++++++--------------- 3 files changed, 39 insertions(+), 30 deletions(-) diff --git a/doc/scheduling.md b/doc/scheduling.md index 71ca1019..8343a09b 100644 --- a/doc/scheduling.md +++ b/doc/scheduling.md @@ -18,12 +18,12 @@ words. The page ends with a key to the marks it shows: * not run yet, † default (no delete, only disable) ``` -Under it are buttons, each screen drawn in place of the last: +Under it are buttons, each screen drawn in place of the last, with **Cancel** back to the list: * **Enable/disable**: a button per job, its number, switch and name (`3 ✅ price midsession US`); a press switches the job on or off. **All on** switches every job of the chat on at once, or **All off**, shown when every job is on, switches them all off. * **Delete**: a button per job added here. A default job has no button: switch it off instead. -* **Add**: the report and how often, then, a screen each, the rest down to the minute: +* **Add**: the report, by name, and how often, then, a screen each, the rest down to the minute: * one-off: the date (the next 14 days), the hour, the minute; * daily and weekdays: the hour, the minute; * weekly: the day of the week, the hour, the minute; @@ -31,9 +31,9 @@ Under it are buttons, each screen drawn in place of the last: * annual: the month, the day, the hour, the minute; * repeating: the interval, then every day or weekdays. - Each screen marks its default with `†` (`Sat†`, `09†`, `09:00†`) and says `† indicates default`. **Back** takes a - step back, **Cancel** goes back to the list. A message's text cannot come from a button, so its screen says how to - type one. + Each screen marks its default with `†` (`Sat†`, `09†`, `09:00†`) and says `† indicates default`. After the report's, + each screen also has **Back**, a step back (the hour's has only Back: Discord takes 25 buttons). A message's text + cannot come from a button, so its screen says how to type one. * **Run now**: a button per job; a press runs it now, as its schedule would, and says how it went. * **Next**: the next runs, in time order. diff --git a/lib/reports/about.py b/lib/reports/about.py index caad270a..a80c8040 100644 --- a/lib/reports/about.py +++ b/lib/reports/about.py @@ -91,7 +91,7 @@ def paged(entries, service): def screen_buttons(screen, page, pages, shown, crc, every_on=False): """A list screen's buttons: top's, what can be done; on, del and run's, one for each job shown that the screen acts on, ' ' (with its ✅ or ❌ to switch it), JOB_ROW to a row, then on's All, which switches every - job of the chat on (All off where every_on: off), and Back; ◀ and More ▶ to the other pages.""" + job of the chat on (All off where every_on: off), and Cancel, back to the list; ◀ and More ▶ to the other pages.""" if screen == 'top': rows = [[buttons.menu_button('Enable/disable', 'on', [0], crc), buttons.menu_button('Delete', 'del', [0], crc)], [buttons.menu_button('Add', 'add', [], crc), buttons.menu_button('Run now', 'run', [0], crc), @@ -101,7 +101,7 @@ def screen_buttons(screen, page, pages, shown, crc, every_on=False): tapped = [buttons.menu_button(f"{number} {('✅ ' if job.on else '❌ ') if screen == 'on' else ''}{job.name}", screen, [page, number], crc) for number, job, _ in shown if screen != 'del' or not job.item] rows = [tapped[i:i + JOB_ROW] for i in range(0, len(tapped), JOB_ROW)] - nav = [buttons.menu_button('Back', 'top', [0], crc)] + nav = [buttons.menu_button('Cancel', 'top', [0], crc)] if screen == 'on': nav.insert(0, buttons.menu_button('All off' if every_on else 'All on', 'on', [page, 'alloff' if every_on else 'allon'], crc)) @@ -126,7 +126,7 @@ def job_line(number, job, outcome): def next_screen(service, chat, count=20): """`.jobs next`: a chat's next runs, in time order, at most count, each by the timezone preference's clock (and its own zone's beside it, where that differs), with its number and name in .jobs; a repeat's runs on a day are one - entry, from its first to its last. Returns (lines, buttons): Back, to the list.""" + entry, from its first to its last. Returns (lines, buttons): Cancel, back to the list.""" scheduler = jobs.current if scheduler is None: return ['No schedule is running'], None @@ -141,7 +141,7 @@ def next_screen(service, chat, count=20): lines.append(f"{said} · {number}. {service.escape(job.name)}") if len(lines) == 1: lines.append('None: every job here is off') - return lines, [[buttons.menu_button('Back', 'top', [0], jobs.list_checksum(scheduler.chat_jobs(chat)))]] + return lines, [[buttons.menu_button('Cancel', 'top', [0], jobs.list_checksum(scheduler.chat_jobs(chat)))]] def clock_words(moment, zone): """A moment's time of day by a zone's clock: '8:30am', in 24 hours for UTC ('21:30').""" @@ -161,15 +161,15 @@ def add_screen(service, chat, steps): """A step of Add, from the choices made so far (steps, from the button pressed: the report's number, how often, then the LEVELS of that frequency in turn: a date, a day of the week or of the month, a month, the hour and the minute, or a repeat's interval and its days): (lines, buttons) of the next choice, its default (USUAL) marked †, or (None, - (action, arguments, schedule)) once every choice is made, at a time by the timezone preference's clock. Back takes a - step back, Cancel goes back to the list. Raises ValueError for choices that are none (forged data).""" + (action, arguments, schedule)) once every choice is made, at a time by the timezone preference's clock. Cancel goes + back to the list, and Back, after the report, a step back. Raises ValueError for choices that are none (forged).""" scheduler = jobs.current crc, local = jobs.list_checksum(scheduler.chat_jobs(chat)), formats.zone() tomorrow = scheduler.clock().astimezone(local).date() + datetime.timedelta(days=1) steps, page = list(steps), 0 if steps and steps[-1] == MORE: steps, page = steps[:-1], 1 - back = buttons.menu_button('Back', *(('add', steps[:-1]) if steps else ('top', [])), crc) + back = buttons.menu_button('Back', 'add', steps[:-1], crc) cancel = buttons.menu_button('Cancel', 'top', [0], crc) def choices(heading, offered, row, lines=(), usual=None, nav=None): """A choice's screen: its heading and lines, and '† indicates default' where one is (usual: its data); its @@ -179,9 +179,9 @@ def choices(heading, offered, row, lines=(), usual=None, nav=None): key = [] if usual is None else ['', '† indicates default'] return [service.bold(heading), *lines, *key], [tapped[i:i + row] for i in range(0, len(tapped), row)] + [ nav or [back, cancel]] - if not steps: - return choices('Add a job: which report?', [(f'{number} {label}', number) for number, (label, _, _) in - enumerate(REPORTS, 1)], 3, [f'{number}. {label}' for number, (label, _, _) in enumerate(REPORTS, 1)]) + if not steps: # a button per report, by its name alone, and Cancel + return choices('Add a job: which report?', [(label, number) for number, (label, _, _) in enumerate(REPORTS, 1)], + 3, nav=[cancel]) label, action, args = REPORTS[choice(steps[0], 1, len(REPORTS)) - 1] if action == 'message': return [service.bold('Add a message'), "A message's text is what you type:", diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index a020e6c0..648075c5 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -696,33 +696,33 @@ class JobsMenu(ChatBase): def test_enable_disable_flips_the_switch_in_the_button_and_the_message(self): """Enable/disable lists the jobs with a button each, ' '; a press switches the job and - redraws, the switch flipped in both; Back goes to the list.""" + redraws, the switch flipped in both; Cancel goes back to the list.""" crc = self.crc() lines, rows, data = self.press(f'm|on|0|{crc}') self.assertEqual(self.toasts, ['Updating…']) self.assertEqual(lines[0], 'Tap a job to switch it on or off') - self.assertEqual(rows, [['1 ✅ rating', '2 ✅ price close US', '3 ✅ trades'], ['All off', 'Back']]) + self.assertEqual(rows, [['1 ✅ rating', '2 ✅ price close US', '3 ✅ trades'], ['All off', 'Cancel']]) lines, rows, data = self.press(data['1 ✅ rating']) self.assertIn('1. ❌* rating† · Tue–Sat at 8:30am Sydney', lines) self.assertEqual(rows[0][0], '1 ❌ rating') self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT]['off'], ['rating']) self.press(data['1 ❌ rating']) self.assertNotIn('off', state.load(jobs.JOBS_FILE)[CHAT]) - lines, rows, _ = self.press(data['Back']) + lines, rows, _ = self.press(data['Cancel']) self.assertEqual((lines[0], rows[0]), ('Jobs here', ['Enable/disable', 'Delete'])) def test_all_switches_every_job_on_or_off(self): """Enable/disable's All switches every job of the chat off where every one is on, else on, and says so; its label says which. Pressed for another list, it switches nothing.""" lines, rows, data = self.press(f'm|on|0|{self.crc()}') - self.assertEqual(rows[-1], ['All off', 'Back']) + self.assertEqual(rows[-1], ['All off', 'Cancel']) lines, rows, data = self.press(data['All off']) self.assertEqual(lines[0], 'Every job is off now') - self.assertEqual(rows, [['1 ❌ rating', '2 ❌ price close US', '3 ❌ trades'], ['All on', 'Back']]) + self.assertEqual(rows, [['1 ❌ rating', '2 ❌ price close US', '3 ❌ trades'], ['All on', 'Cancel']]) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'defaults': 'off', # the default jobs start off now 'jobs': [entry('trades every 20 mins weekdays', on=False)]}) lines, rows, data = self.press(data['3 ❌ trades']) - self.assertEqual(rows[-1], ['All on', 'Back']) # some on, some off: All switches them all on + self.assertEqual(rows[-1], ['All on', 'Cancel']) # some on, some off: All switches them all on lines, rows, data = self.press(data['All on']) self.assertEqual(lines[0], 'Every job is on now') self.assertTrue(all(job.on for job in self.s.chat_jobs(CHAT))) @@ -736,13 +736,13 @@ def test_delete_offers_only_the_jobs_added_here(self): lines, rows, data = self.press(f'm|del|0|{self.crc()}') self.assertIn('1. ✅* rating† · Tue–Sat at 8:30am Sydney', lines) self.assertEqual(lines[-1], '* not run yet, † default (no delete, only disable)') - self.assertEqual(rows, [['3 trades'], ['Back']]) + self.assertEqual(rows, [['3 trades'], ['Cancel']]) self.assertEqual(self.press(f'm|del|0|1|{self.crc()}')[0][0], 'rating is a default job: switch it off instead' ' (.jobs toggle)') lines, rows, data = self.press(data['3 trades']) self.assertEqual(lines[:2], ['Deleted 3. trades', '']) self.assertIn('None added here yet: Add makes one', lines) - self.assertEqual(rows, [['Back']]) + self.assertEqual(rows, [['Cancel']]) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': []}) def test_run_now_runs_the_job_and_says_how_it_went(self): @@ -753,6 +753,13 @@ def test_run_now_runs_the_job_and_says_how_it_went(self): self.assertEqual(self.ran, [('trades', {})]) self.assertEqual(self.replies[-1][1], ['3. trades: done']) + def test_next_lists_the_next_runs_with_cancel(self): + """Next lists the next runs, with Cancel back to the list.""" + lines, rows, data = self.press(f'm|next|{self.crc()}') + self.assertEqual((lines[:2], rows), (['Next runs:', 'Wed 14 Oct 8:20am (21:20 UTC) – 10:40am, every 20' + ' mins · 3. trades'], [['Cancel']])) + self.assertEqual(self.press(data['Cancel'])[0][0], 'Jobs here') + def test_a_stale_or_forged_press_acts_on_nothing(self): """A press drawn for another list (its checksum), for a number that is none, or of no screen at all switches, deletes and runs nothing: the screen is drawn as the list is now, or the press is ignored.""" @@ -782,14 +789,16 @@ def test_the_older_job_buttons_still_work(self): self.assertFalse(self.s.chat_jobs(CHAT)[2].on) def test_add_by_buttons_back_and_cancel(self): - """Add takes the report, how often, then the day, the hour and the minute from buttons, by the timezone - preference's clock, each choice's default marked † and the key under it; Back goes a step back and Cancel back - to the list; its last step adds the job and says so, the same job as typed.""" + """Add takes the report (by its name alone), how often, then the day, the hour and the minute from buttons, by + the timezone preference's clock, each choice's default marked † and the key under it; Cancel goes back to the + list, and after the report Back a step back; its last step adds the job and says so, the same job as typed.""" lines, rows, data = self.press(f'm|add|{self.crc()}') - self.assertEqual(lines[0], 'Add a job: which report?') - self.assertEqual(rows[1:], [['4 price close', '5 price', '6 performance'], ['7 cal earnings', - '8 cal ex-dividend', '9 shorts'], ['10 rating', '11 milestone', '12 message'], ['Back', 'Cancel']]) - lines, rows, data = self.press(data['5 price']) + self.assertEqual(lines, ['Add a job: which report?']) + self.assertEqual(rows, [['trades', 'price midsession', 'price premarket'], ['price close', 'price', + 'performance'], ['cal earnings', 'cal ex-dividend', 'shorts'], ['rating', 'milestone', 'message'], + ['Cancel']]) + self.assertEqual((data['price'], data['Cancel']), (f'm|add|5|{self.crc()}', f'm|top|0|{self.crc()}')) + lines, rows, data = self.press(data['price']) self.assertEqual((lines, rows[:2]), (['Add price: how often?', '', '† indicates default'], [['one-off', 'daily†', 'weekdays'], ['weekly', 'monthly', 'quarterly']])) lines, rows, data = self.press(data['weekly']) From 657fc5a1ee0147e01b65498d7077b1c2c4e57acf Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 08:01:24 +0000 Subject: [PATCH 10/41] .jobs: Next is Upcoming, as a button and typed The menu's Next button reads Upcoming, its screen "Upcoming runs:", and the typed form is .jobs upcoming (help, autocomplete, the slash command's description and the docs); .jobs next still works. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- doc/commands.md | 2 +- doc/scheduling.md | 4 ++-- lib/reports/about.py | 12 ++++++------ lib/worker.py | 19 ++++++++++--------- tests/test_discord.py | 2 +- tests/test_scheduler.py | 20 +++++++++++--------- util/record_outputs.py | 2 +- 7 files changed, 32 insertions(+), 29 deletions(-) diff --git a/doc/commands.md b/doc/commands.md index bd588c0d..ebb2cd16 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -40,7 +40,7 @@ Where: /help /history symbol /holdings portfolio -/jobs [next|system] +/jobs [upcoming|system] /jobs run|toggle|del [number] /jobs add report schedule /jobs add message schedule "text" diff --git a/doc/scheduling.md b/doc/scheduling.md index 8343a09b..c05eb7cf 100644 --- a/doc/scheduling.md +++ b/doc/scheduling.md @@ -35,14 +35,14 @@ Under it are buttons, each screen drawn in place of the last, with **Cancel** ba each screen also has **Back**, a step back (the hour's has only Back: Discord takes 25 buttons). A message's text cannot come from a button, so its screen says how to type one. * **Run now**: a button per job; a press runs it now, as its schedule would, and says how it went. -* **Next**: the next runs, in time order. +* **Upcoming**: the next runs, in time order. A button drawn for a list that has since changed acts on nothing: the screen is drawn again as the list is now. Or type: ``` /jobs the chat's jobs, with the buttons above -/jobs next the next 20 runs; a repeat's runs on one day are one entry +/jobs upcoming the next 20 runs; a repeat's runs on one day are one entry /jobs system the system jobs /jobs add the reports a job can run, what each takes, and examples /jobs add price weekly diff --git a/lib/reports/about.py b/lib/reports/about.py index a80c8040..94500376 100644 --- a/lib/reports/about.py +++ b/lib/reports/about.py @@ -12,7 +12,7 @@ HEADINGS = {'top': 'Jobs here', 'on': 'Tap a job to switch it on or off', 'run': 'Tap a job to run it now', 'del': 'Tap a job added here to delete it (a default job is switched off instead)'} TYPED = ['.jobs run|toggle|del NUMBER: run one now, switch it on or off, delete one added here', - '.jobs add REPORT SCHEDULE: add one (.jobs add lists the reports)', '.jobs next: the next runs', + '.jobs add REPORT SCHEDULE: add one (.jobs add lists the reports)', '.jobs upcoming: the next runs', '.jobs system: the system jobs'] # what a chat without buttons types instead TYPED_ACTIONS = {'on': 'toggle', 'del': 'del', 'run': 'run'} NOT_RUN, DEFAULT = '* not run yet', '† default (no delete, only disable)' # the key to a list's marks @@ -95,7 +95,7 @@ def screen_buttons(screen, page, pages, shown, crc, every_on=False): if screen == 'top': rows = [[buttons.menu_button('Enable/disable', 'on', [0], crc), buttons.menu_button('Delete', 'del', [0], crc)], [buttons.menu_button('Add', 'add', [], crc), buttons.menu_button('Run now', 'run', [0], crc), - buttons.menu_button('Next', 'next', [], crc)]] + buttons.menu_button('Upcoming', 'next', [], crc)]] nav = [] else: tapped = [buttons.menu_button(f"{number} {('✅ ' if job.on else '❌ ') if screen == 'on' else ''}{job.name}", @@ -124,14 +124,14 @@ def job_line(number, job, outcome): return f"{number}. {job.name}: {outcome}" def next_screen(service, chat, count=20): - """`.jobs next`: a chat's next runs, in time order, at most count, each by the timezone preference's clock (and its - own zone's beside it, where that differs), with its number and name in .jobs; a repeat's runs on a day are one - entry, from its first to its last. Returns (lines, buttons): Cancel, back to the list.""" + """`.jobs upcoming`: a chat's next runs, in time order, at most count, each by the timezone preference's clock + (and its own zone's beside it, where that differs), with its number and name in .jobs; a repeat's runs on a day + are one entry, from its first to its last. Returns (lines, buttons): Cancel, back to the list.""" scheduler = jobs.current if scheduler is None: return ['No schedule is running'], None local = formats.zone() - lines = [service.bold('Next runs:')] + lines = [service.bold('Upcoming runs:')] for first, last, job, number in scheduler.upcoming(chat, count): said = moment_words(first, local, job.schedule.tz) if job.schedule.every: diff --git a/lib/worker.py b/lib/worker.py index 3674752e..2c637f68 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -434,10 +434,10 @@ def on_who(request, args): request.reply(holdings.prepare_who(request.service, re.split(r'[.:]', args.words[0].upper())[0])) def on_jobs(request, args): - """.jobs: this chat's jobs, the default ones and those added here, with the menu's buttons; .jobs next: the next - runs; .jobs system: the system's; .jobs run, toggle or del NUMBER: one run now, switched on or off, or deleted (one - added here), and without a number the list with a button per job that does it; .jobs add REPORT SCHEDULE, or .jobs - add message SCHEDULE "TEXT": one more (alone, the reports a job can run).""" + """.jobs: this chat's jobs, the default ones and those added here, with the menu's buttons; .jobs upcoming: the + next runs; .jobs system: the system's; .jobs run, toggle or del NUMBER: one run now, switched on or off, or deleted + (one added here), and without a number the list with a button per job that does it; .jobs add REPORT SCHEDULE, or + .jobs add message SCHEDULE "TEXT": one more (alone, the reports a job can run).""" scheduler, chat, service = jobs.current, request.dest.key, request.service tappable = request.dest.takes_buttons if scheduler is None: @@ -446,7 +446,7 @@ def on_jobs(request, args): if args.keyword is None and not args.words: lines, markup = about.jobs_screen(service, chat, tappable=tappable) request.reply(lines, buttons=markup) - elif args.keyword == 'next' and not args.words: + elif args.keyword == 'upcoming' and not args.words: lines, markup = about.next_screen(service, chat) request.reply(lines, buttons=markup if tappable else None) elif args.keyword == 'system' and not args.words: @@ -512,8 +512,9 @@ class Command: CHANGE = {'add': 'add'} | dict.fromkeys(('del', 'delete', 'rem', 'remove', 'rm'), 'delete') # of the watchlist PLANS = 'finbot_plan.json' # each chat's plans, {user: plan}, as state.load_for keeps them BY = dict.fromkeys(('sector', 'sectors'), 'sector') | dict.fromkeys(('industry', 'industries'), 'industry') # .pie's -JOB_CHANGES = {'run': 'run', 'toggle': 'toggle', 'add': 'add', 'next': 'next', 'system': 'system'} | dict.fromkeys(( - 'del', 'delete', 'rem', 'remove', 'rm'), 'del') # what .jobs does to a chat's jobs, and shows +# what .jobs does to a chat's jobs, and shows; next is what upcoming was called before +JOB_CHANGES = {'run': 'run', 'toggle': 'toggle', 'add': 'add', 'system': 'system'} | dict.fromkeys(('upcoming', + 'next'), 'upcoming') | dict.fromkeys(('del', 'delete', 'rem', 'remove', 'rm'), 'del') BUY_SELL = ('buy', 'sell', *(a + sep + b for a, b in (('buy', 'sell'), ('sell', 'buy')) for sep in ('', '&', '/', '+'))) PROFILE = Command(('stock', 'profile'), on_profile, ('.SYMBOL', '.stock SYMBOL'), COMPANY) # any first word that is no # command's name but may be a ticker calls it too @@ -549,7 +550,7 @@ class Command: Command(('shorts', 'short'), on_shorts, ('.shorts [percent|SYMBOL]',), RISK, ('.shorts',), percent=True), Command(('trades', 'trade'), on_trades, ('.trades [period] [portfolio]',), TRADES, ('.trades',), period=True), Command(('plan',), on_plan, ('.plan [your plan]',), TRADES), - Command(('jobs', 'job', 'cron', 'crons', 'schedule'), on_jobs, ('.jobs [next|system]', + Command(('jobs', 'job', 'cron', 'crons', 'schedule'), on_jobs, ('.jobs [upcoming|system]', '.jobs run|toggle|del [NUMBER]', '.jobs add REPORT SCHEDULE', '.jobs add message SCHEDULE "TEXT"'), JOBS, keywords=JOB_CHANGES), PROFILE, @@ -608,7 +609,7 @@ def slash_commands(aliases=True): for name, (command, lines) in found.items()] PERIOD_WORDS = ('1d', '7d', '1m', '3m', '6m', 'ytd', '1y', '5y', '10y') # suggested where a command takes a period -JOB_WORDS = ('add', 'run', 'toggle', 'del', 'next', 'system') # what /jobs suggests first +JOB_WORDS = ('add', 'run', 'toggle', 'del', 'upcoming', 'system') # what /jobs suggests first SCHEDULE_WORDS = ('daily', 'weekdays', 'weekly', 'monthly', 'quarterly', 'bi-annual', 'annual', 'hourly', 'every', 'at', 'from', 'between', 'on') # what a schedule in English is made of, suggested after a report diff --git a/tests/test_discord.py b/tests/test_discord.py index 18ffe840..c6b2a65a 100644 --- a/tests/test_discord.py +++ b/tests/test_discord.py @@ -399,7 +399,7 @@ def test_the_jobs_subcommands_numbers_and_reports(self): with contextlib.redirect_stderr(io.StringIO()): scheduler.load() self.patch(jobs, 'current', scheduler) - self.assertEqual(self.values('/jobs '), ['add', 'run', 'toggle', 'del', 'next', 'system']) + self.assertEqual(self.values('/jobs '), ['add', 'run', 'toggle', 'del', 'upcoming', 'system']) self.assertEqual(self.values('/job T'), ['toggle']) self.assertEqual(worker.suggest('discord:77', '/jobs run '), [('run 1: rating', 'run 1'), ('run 2: trades', 'run 2'), ('run 3: shorts', 'run 3')]) diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index 648075c5..aea45c5b 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -517,7 +517,7 @@ def test_add_list_and_del(self): '3. ✅* trades · every 20 mins, weekdays UTC', '4. ✅* shorts · daily at 10:00pm', '', '* not run yet, † default (no delete, only disable)']) self.assertEqual([[button.label for button in row] for row in self.replies[-1][2]], - [['Enable/disable', 'Delete'], ['Add', 'Run now', 'Next']]) + [['Enable/disable', 'Delete'], ['Add', 'Run now', 'Upcoming']]) self.assertEqual(self.say('.jobs del 3'), ['Deleted 3. trades · every 20 mins, weekdays UTC']) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('shorts daily at 10pm', zone='Australia/Melbourne')]}) @@ -640,10 +640,12 @@ def test_a_one_off_gone_by_is_deleted_as_the_list_is_shown(self): self.assertNotIn('soon', '\n'.join(self.say('.jobs'))) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('trades every 20 mins weekdays')]}) - def test_next_and_system(self): - """.jobs next lists the next runs in order, a repeat's runs on a day as one; .jobs system the system's.""" - lines = self.say('.jobs next') - self.assertEqual(lines[:4], ['Next runs:', + def test_upcoming_and_system(self): + """.jobs upcoming (or next, as it was called) lists the next runs in order, a repeat's runs on a day as one; + .jobs system the system's.""" + lines = self.say('.jobs upcoming') + self.assertEqual(self.say('.jobs next'), lines) + self.assertEqual(lines[:4], ['Upcoming runs:', 'Wed 14 Oct 8:20am (21:20 UTC) – 10:40am, every 20 mins · 3. trades', 'Wed 14 Oct 8:30am · 1. rating', 'Wed 14 Oct 11:00am (00:00 UTC) – Thu 15 Oct 10:40am, every 20 mins · 3. trades']) @@ -753,11 +755,11 @@ def test_run_now_runs_the_job_and_says_how_it_went(self): self.assertEqual(self.ran, [('trades', {})]) self.assertEqual(self.replies[-1][1], ['3. trades: done']) - def test_next_lists_the_next_runs_with_cancel(self): - """Next lists the next runs, with Cancel back to the list.""" + def test_upcoming_lists_the_next_runs_with_cancel(self): + """Upcoming lists the next runs, with Cancel back to the list.""" lines, rows, data = self.press(f'm|next|{self.crc()}') - self.assertEqual((lines[:2], rows), (['Next runs:', 'Wed 14 Oct 8:20am (21:20 UTC) – 10:40am, every 20' - ' mins · 3. trades'], [['Cancel']])) + self.assertEqual((lines[:2], rows), (['Upcoming runs:', 'Wed 14 Oct 8:20am (21:20 UTC) – 10:40am,' + ' every 20 mins · 3. trades'], [['Cancel']])) self.assertEqual(self.press(data['Cancel'])[0][0], 'Jobs here') def test_a_stale_or_forged_press_acts_on_nothing(self): diff --git a/util/record_outputs.py b/util/record_outputs.py index d1b2d6dd..b3902625 100755 --- a/util/record_outputs.py +++ b/util/record_outputs.py @@ -24,7 +24,7 @@ '.performance 7d', '.performance 1y', '.performance Nope', '.pie Rob', '.pie Rob sector', '.pie', '.compare T1 T2', '.compare T1 T2 3m', '.compare T1', '.history T1', '.history', '.watchlist', '.watchlist add MSFT', '.watchlist add MSFT', '.watchlist del MSFT', '.watchlist frob X', '.T1', '.jobs', '.jobs run', '.jobs toggle', - '.jobs add', '.jobs add shorts daily at 22:00', '.jobs del 4', '.jobs run 1', '.jobs next', '.jobs system', + '.jobs add', '.jobs add shorts daily at 22:00', '.jobs del 4', '.jobs run 1', '.jobs upcoming', '.jobs system', 'hi @bot', 'noise'] BUTTONS = ['c|h|m|T1', 'c|h|x|T1', 'c|p|w|T1', 'c|c|y|T1,T2', 'c|f|y|', 'c|l|d|3,0', 'c|l|m|0,10', 'c|zz|m|T1', 'j|1|0000', 'e|1|0000', 'm|on|0|0000', 'm|add|0000', 'm|add|5|w|0000', 'm|next|0000'] From 17d1ed0a1408261252aefe3edb7b1f127456688f Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 08:05:08 +0000 Subject: [PATCH 11/41] log_width: where a long log line is cut, a preference (71 by default) A request's log line (its method and URL) was cut at a fixed 73 bytes. The log_width preference now sets it, in characters, for any line that can run long: log.cut() keeps a line within it, its last character an ellipsis, and 0 cuts none. The default, 71, gives the same lines as before, and none longer (a line of 72 or 73 characters was shown whole). In Prefs, defaults/preferences.conf and doc/configuration.md. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 3 ++- defaults/preferences.conf | 4 +++- doc/configuration.md | 1 + lib/config.py | 1 + lib/log.py | 9 ++++++++- lib/net.py | 10 ++++------ tests/test_edges.py | 16 +++++++++++++--- 7 files changed, 32 insertions(+), 12 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index f12e88b7..9b332bda 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -98,7 +98,8 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm `yahoo.symbol(typed)` is Yahoo's symbol for a ticker as written, tried on the local exchange (`exchange_suffix`) when Yahoo does not know it bare: every ticker a chat gives goes through it. `sharesight.portfolios(name=None)` gives `Portfolio` objects: `holdings()`, `tickers()`, `gain(days)` (the portfolio's), `trades(days)`. -- `lib/log.py`: the log, by level (`error warn info debug`), as the `log_level` preference lets in; bot.py's `WholeLines` +- `lib/log.py`: the log, by level (`error warn info debug`), as the `log_level` preference lets in, and `cut()`, which + keeps a line that can run long (a request's) within the `log_width` preference; bot.py's `WholeLines` writes each line whole, whichever threads log at once. `lib/cache.py`: cache files (any may vanish at any time). `lib/state.py`: state files in `DATA_DIR`, which must last; change one with `with state.edit(name, default) as data:`, never load-then-save; a file of one record per chat with `state.load_for(chat, name, default)` and `state.edit_for`. A chat's key is its network and id (`Transport.key`, diff --git a/defaults/preferences.conf b/defaults/preferences.conf index 73663eab..8f56920d 100644 --- a/defaults/preferences.conf +++ b/defaults/preferences.conf @@ -53,6 +53,8 @@ #http_timeout=18 # Diagnostics. log_level: what the log shows, that level and those above it: ERROR, WARN, INFO (what finbot does, -# and each request it sends) or DEBUG (each cache miss, hit and write too). +# and each request it sends) or DEBUG (each cache miss, hit and write too). log_width: the most characters of a log +# line that can run long (each request's method and URL), cut there, ending in an ellipsis (…); 0 cuts none. #log_level=INFO +#log_width=71 #print_headers=False diff --git a/doc/configuration.md b/doc/configuration.md index ae790789..5c7f1d14 100644 --- a/doc/configuration.md +++ b/doc/configuration.md @@ -75,6 +75,7 @@ A change takes effect on a reload. A true/false preference takes `true`/`false`, | `performance_cache_seconds` | `2400` | Cache lifetime of Sharesight performance reports in seconds: longer than the 30 minutes between `sharesight_warm` runs, so the holdings never run out | | `http_timeout` | `18` | Seconds to wait on outbound HTTP requests | | `log_level` | `INFO` | What the log shows: that level and those above it, syslog style. `ERROR`: what failed and was lost (a message not sent); `WARN`: what failed and was got over (a retry, a fallback, a request refused); `INFO`: what finbot does (each message sent, each job run, each request sent, each request refused); `DEBUG`: the routine of a button press too (the press, its acknowledgement, the message edited in place), each request answered with success, and each cache miss, hit and write. Replaces `debug` | +| `log_width` | `71` | The most characters of a log line that can run long: a longer one is cut there, ending in `…`. For now that is each request's line, its method and URL. `0`: no line is cut | | `print_headers` | `False` | Log inbound request headers | A key in the wrong file, or one that is not listed, stops finbot starting; on a reload it is reported and nothing is applied. A key finbot no longer reads (`alliterate`, `chunk_maxlines`) is ignored, and the log at start says to delete it. diff --git a/lib/config.py b/lib/config.py index a687c037..dabf3a3a 100644 --- a/lib/config.py +++ b/lib/config.py @@ -306,6 +306,7 @@ class Prefs: hyperlinkProvider: str = 'google' include_portfolios: tuple = () # portfolio ids log_level: str = 'INFO' # what the log shows: one of LOG_LEVELS, and those more severe (lib/log.py) + log_width: int = 71 # the most characters of a log line that can run long (a request's); 0: no cut (log.cut) min_analysts: int = 7 # .buy, .sell and .forward pe list only stocks this many analysts cover past_days: int = 28 performance_cache_seconds: int = 2400 # Sharesight performance reports (40 minutes: sharesight_warm runs every 30) diff --git a/lib/log.py b/lib/log.py index 46eb9c97..6294f924 100644 --- a/lib/log.py +++ b/lib/log.py @@ -1,6 +1,7 @@ """The log, syslog style: each line has a level, ERROR, WARN, INFO or DEBUG, and goes to stderr (which systemd and Docker take to the journal) when the log_level preference lets it in: that level and those above it, so ERROR always -does. Each takes print()'s arguments.""" +does. Each takes print()'s arguments. A line that can run long (a request's URL) goes through cut(), which keeps it +within the log_width preference.""" import sys from lib import config @@ -19,6 +20,12 @@ def write(level, parts, options): print(*parts, file=sys.stderr, **options) +def cut(text): + """A long log line (text) cut to log_width characters, its last an ellipsis (…); whole where log_width is 0.""" + width = config.prefs.log_width + return text if width <= 0 or len(text) <= width else text[:width - 1] + '…' + + def error(*parts, **options): """Something failed and was lost: a message not sent, a request not answered, a start that cannot go on.""" write('ERROR', parts, options) diff --git a/lib/net.py b/lib/net.py index 9edf19ae..2a610bb0 100644 --- a/lib/net.py +++ b/lib/net.py @@ -15,7 +15,6 @@ # no domain may set or receive a cookie session.cookies.set_policy(http.cookiejar.DefaultCookiePolicy(allowed_domains=[])) CONNECT_TIMEOUT = 5 # seconds to wait for a host to answer the connection: one that is down costs this, not http_timeout -SHOWN = 73 # bytes of a request's log line, at most: the method and URL, cut with an ellipsis (3 bytes in UTF-8) def get(url, **kwargs): """requests.get through the shared session, which reuses connections; timed out as timeouts() says; logged.""" @@ -32,13 +31,12 @@ def request(method, url, **kwargs): def logged(method, url, kwargs): """The request's arguments without `logged`; the request noted first, its method and URL without secrets or the - scheme, cut to SHOWN bytes: at INFO; at DEBUG with logged='debug' (the routine of a press: its acknowledgement, the - message edited in place); not at all with logged=False (Telegram's long polling, asked again and again).""" + scheme, cut to the log_width preference (log.cut): at INFO; at DEBUG with logged='debug' (the routine of a press: + its acknowledgement, the message edited in place); not at all with logged=False (Telegram's long polling, asked + again and again).""" shown = kwargs.pop('logged', True) if shown: - line = f"{method} {redact(url).split('://', 1)[-1]}".encode() - (log.debug if shown == 'debug' else log.info)(line.decode() if len(line) <= SHOWN else - line[:SHOWN - 3].decode(errors='ignore') + '…') + (log.debug if shown == 'debug' else log.info)(log.cut(f"{method} {redact(url).split('://', 1)[-1]}")) return kwargs def timeouts(kwargs): diff --git a/tests/test_edges.py b/tests/test_edges.py index 7f394c79..1224d9aa 100644 --- a/tests/test_edges.py +++ b/tests/test_edges.py @@ -414,8 +414,8 @@ class RequestLog(FinbotCase): """Each request that goes out (a cache miss, or a call that has no cache) is a line at INFO.""" def test_each_request_is_a_short_line_without_secrets(self): - """A request logs its method and URL at INFO, without the scheme or a secret, cut to 73 bytes (the ellipsis - takes 3); Telegram's long polling (logged=False) does not, and nothing does below INFO.""" + """A request logs its method and URL at INFO, without the scheme or a secret, cut to log_width characters (71, + the last an ellipsis); Telegram's long polling (logged=False) does not, and nothing does below INFO.""" with contextlib.redirect_stderr(io.StringIO()) as err: net.get('https://query1.finance.yahoo.com/v7/finance/quote?crumb=s3cret') net.post('https://api.telegram.org/bot123:ABC-def/sendMessage', json={}) @@ -427,12 +427,22 @@ def test_each_request_is_a_short_line_without_secrets(self): self.assertEqual(lines, ['GET query1.finance.yahoo.com/v7/finance/quote?crumb=', 'POST api.telegram.org/bot/sendMessage', 'PATCH discord.com/api/v10/channels/7/messages/9', 'GET query2.finance.yahoo.com/v10/finance/quoteSummary/ARM.L?modules=ca…']) - self.assertEqual(len(lines[3].encode()), 73) + self.assertEqual(len(lines[3]), 71) self.prefs(log_level='WARN') with contextlib.redirect_stderr(io.StringIO()) as err: net.get('https://example.com/') self.assertEqual(err.getvalue(), '') + def test_log_width_sets_where_a_long_line_is_cut(self): + """The log_width preference is where a long line is cut, its last character an ellipsis; at 0 none is.""" + whole = 'GET query2.finance.yahoo.com/v10/finance/quoteSummary/ARM.L?modules=calendarEvents' # 82 characters + for width, line in ((20, 'GET query2.finance.…'), (81, whole[:80] + '…'), (82, whole), (0, whole), (1, '…')): + with self.subTest(width=width): + self.prefs(log_width=width) + with contextlib.redirect_stderr(io.StringIO()) as err: + net.get('https://' + whole[4:]) + self.assertEqual(err.getvalue().splitlines(), [line]) + class SharedSession(unittest.TestCase): def test_the_session_keeps_no_cookies(self): From a9fb492ad4d0cbe77b2cf03004e2e771e14ad333 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 10:00:43 +0000 Subject: [PATCH 12/41] when: quarterly, bi-annual and annual from any start month when.cycle(start, count) gives a cycle's months from its start month (quarterly from Feb: Feb/May/Aug/Nov), and month_kind() reads any such cycle as quarterly or bi-annual, so Schedule.frequency, days_words ("quarterly, 15 Feb/May/Aug/Nov", "bi-annual, 1 Mar and 1 Sep") and the period of performance and price follow it. Typed: "quarterly from feb on the 15th", "bi-annual starting in march", "annual from aug"; a bare quarterly or bi-annual still starts in January. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/when.py | 47 ++++++++++++++++++++++++++++++++-------------- tests/test_util.py | 24 +++++++++++++++++++++++ 2 files changed, 57 insertions(+), 14 deletions(-) diff --git a/lib/when.py b/lib/when.py index 2df198a3..1f9038dc 100644 --- a/lib/when.py +++ b/lib/when.py @@ -16,7 +16,7 @@ DAYS = ('mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun') DAY_NAMES = ('Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday') MONTHS = ('jan', 'feb', 'mar', 'apr', 'may', 'jun', 'jul', 'aug', 'sep', 'oct', 'nov', 'dec') -QUARTERS = (1, 4, 7, 10) +QUARTERS = (1, 4, 7, 10) # quarterly and bi-annual where no start month is said: from January HALVES = (1, 7) DEFAULT_AT = 9 * 60 # a schedule's time of day where none is typed: 9am ZONE_WORDS = { # what people call a zone -> the zone, with its daylight saving: EST is New York's clock all year @@ -28,8 +28,8 @@ 'aet': 'Australia/Sydney', 'aest': 'Australia/Sydney', 'aedt': 'Australia/Sydney'} AREAS = ('Africa', 'America', 'Antarctica', 'Asia', 'Atlantic', 'Australia', 'Europe', 'Indian', 'Pacific') EXAMPLES = ('daily at 9am', 'weekdays at 4:30pm', 'every saturday at 9am', 'first day of each month at 9am', - 'quarterly', 'every 20 mins weekdays from 10am to 9pm', 'every 31 mar', 'on 28 oct 2026', - 'weekdays at 4:30pm New York') + 'quarterly', 'quarterly from feb on the 15th', 'every 20 mins weekdays from 10am to 9pm', 'every 31 mar', + 'on 28 oct 2026', 'weekdays at 4:30pm New York') TIME = r'(\d{1,2})(?::(\d{2}))?\s*(am|pm)?' DAY = r'(mon|tue|wed|thu|fri|sat|sun)(?:day|sday|nesday|rsday|urday|r|s)?s?' MONTH = r'(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)(?:uary|ruary|ch|il|e|y|ust|tember|t|ober|ember)?' @@ -209,8 +209,7 @@ def frequency(self): return 'weekly' if len(days) == 1 else None if 'date' in days: return 'once' - return {(): 'monthly', QUARTERS: 'quarterly', HALVES: 'bi-annual'}.get(days['months'], - 'annual' if len(days['months']) == 1 else None) + return month_kind(days['months']) def describe(self, local, now=None): """The schedule in words, by its own zone's clock: plain where that is local (the zone's name), in 24 hours with @@ -400,12 +399,30 @@ def days_words(days): return 'once, ' + date_words(datetime.date.fromisoformat(days['date'])) day, months = days['day'], days['months'] names = [MONTHS[m - 1].title() for m in months] - if not months: + kind = month_kind(months) + if kind == 'monthly': return f'monthly, the {ordinal(day)}' - if months == HALVES: + if kind == 'bi-annual': return f'bi-annual, {day} {names[0]} and {day} {names[1]}' - kind = 'quarterly' if months == QUARTERS else 'annual' if len(months) == 1 else 'yearly' - return f"{kind}, {day} {'/'.join(names)}" + return f"{kind or 'yearly'}, {day} {'/'.join(names)}" + + +def cycle(start, count): + """The months of a cycle that comes count times a year, from its start month (1–12), in order: quarterly from Feb + is (2, 5, 8, 11), bi-annual from Mar (3, 9).""" + return tuple(sorted((start - 1 + i * 12 // count) % 12 + 1 for i in range(count))) + + +def month_kind(months): + """How often some months of the year come round, in a word: monthly for none (every month), quarterly for four 3 + apart, bi-annual for two 6 apart, from any month, annual for one; None for others.""" + if not months: + return 'monthly' + if len(months) == 1: + return 'annual' + if len(months) in (2, 4) and tuple(months) == cycle(months[0], len(months)): + return 'bi-annual' if len(months) == 2 else 'quarterly' + return None def repeat_days(days): @@ -577,9 +594,11 @@ def read_days(take): day = 1 if day is not None and not 1 <= day <= 31: raise ScheduleError(f'no day {day} of the month') - for pattern, months in ((r' monthly ', ()), (r' quarterly ', QUARTERS), (r' (?:bi-annual|biannual|bi-annually|' - r'biannually|half-yearly|semi-annual|semi-annually|twice a year) ', HALVES), - (r' (?:annual|annually|yearly|every year) ', (1,))): - if take(pattern): - return {'day': day or 1, 'months': months} + start = rf'(?: (?:from|starting(?: in)?) {MONTH})?' # the month a cycle starts in: quarterly from feb + for pattern, count in ((r' monthly ', 0), (rf' quarterly{start} ', 4), (r' (?:bi-annual|biannual|bi-annually|' + rf'biannually|half-yearly|semi-annual|semi-annually|twice a year){start} ', 2), + (rf' (?:annual|annually|yearly|every year){start} ', 1)): + if m := take(pattern): + first = MONTHS.index(m[1]) + 1 if count and m[1] else 1 + return {'day': day or 1, 'months': cycle(first, count) if count else ()} return None if day is None else {'day': day, 'months': ()} diff --git a/tests/test_util.py b/tests/test_util.py index 2f3c330e..96b9232f 100644 --- a/tests/test_util.py +++ b/tests/test_util.py @@ -364,8 +364,11 @@ def test_english_becomes_a_schedule(self): ('monthly on the 2nd at 8:30am', {'days': {'day': 2}, 'at': '08:30'}), ('quarterly', {'days': {'day': 1, 'months': [1, 4, 7, 10]}, 'at': '09:00'}), ('quarterly on the 2nd', {'days': {'day': 2, 'months': [1, 4, 7, 10]}, 'at': '09:00'}), + ('quarterly from feb on the 15th', {'days': {'day': 15, 'months': [2, 5, 8, 11]}, 'at': '09:00'}), ('bi-annual', {'days': {'day': 1, 'months': [1, 7]}, 'at': '09:00'}), ('half-yearly', {'days': {'day': 1, 'months': [1, 7]}, 'at': '09:00'}), + ('bi-annual starting in march', {'days': {'day': 1, 'months': [3, 9]}, 'at': '09:00'}), + ('annual from aug', {'days': {'day': 1, 'months': [8]}, 'at': '09:00'}), ('annual', {'days': {'day': 1, 'months': [1]}, 'at': '09:00'}), ('every 31 mar at 8:30am', {'days': {'day': 31, 'months': [3]}, 'at': '08:30'}), ('on 28 oct 2026', {'days': {'date': '2026-10-28'}, 'at': '09:00'}), @@ -416,6 +419,12 @@ def test_a_schedule_says_itself_by_its_own_zones_clock(self): 'quarterly, 2 Jan/Apr/Jul/Oct at 8:30am Sydney'), ({'days': {'day': 2, 'months': [1, 7]}, 'at': '08:30', 'zone': syd}, 'bi-annual, 2 Jan and 2 Jul at 8:30am Sydney'), + ({'days': {'day': 15, 'months': [2, 5, 8, 11]}, 'at': '09:00', 'zone': self.MEL}, + 'quarterly, 15 Feb/May/Aug/Nov at 9:00am'), + ({'days': {'day': 1, 'months': [3, 9]}, 'at': '09:00', 'zone': self.MEL}, + 'bi-annual, 1 Mar and 1 Sep at 9:00am'), + ({'days': {'day': 1, 'months': [1, 2]}, 'at': '09:00', 'zone': self.MEL}, + 'yearly, 1 Jan/Feb at 9:00am'), ({'days': {'day': 31, 'months': [3]}, 'at': '08:30', 'zone': syd}, 'annual, 31 Mar at 8:30am Sydney'), ({'days': {'date': '2026-10-28'}, 'at': '08:30', 'zone': syd}, 'once, 28 Oct 2026 at 8:30am Sydney'), ({'days': ['sat'], 'at': '09:00', 'zone': self.MEL}, 'weekly, Saturdays at 9:00am'), @@ -514,10 +523,25 @@ def test_how_often_in_a_word(self): """How often a schedule's days come round, in a word; None for several weekdays, and for a span.""" for text, word in (('daily', 'daily'), ('weekdays', 'weekdays'), ('every sunday', 'weekly'), ('monthly on the 2nd', 'monthly'), ('quarterly', 'quarterly'), ('bi-annual', 'bi-annual'), + ('quarterly from dec', 'quarterly'), ('bi-annual from mar', 'bi-annual'), ('annual', 'annual'), ('every 31 mar', 'annual'), ('on 28 oct 2026', 'once'), ('tue-sat', None), ('every 20 mins from mon 9am to sat 3pm', None)): with self.subTest(text=text): self.assertEqual(when.parse(text, self.MEL).frequency, word) + self.assertIsNone(when.Schedule({'day': 1, 'months': (1, 2)}, at=540).frequency) # no cycle: yearly + + def test_a_cycle_runs_from_its_start_month(self): + """A quarterly or bi-annual cycle may start in any month: its months are the start and every 3 or 6 months on, + and its next run is the next of them.""" + self.assertEqual([when.cycle(2, 4), when.cycle(12, 4), when.cycle(7, 2), when.cycle(3, 1)], + [(2, 5, 8, 11), (3, 6, 9, 12), (1, 7), (3,)]) + self.assertEqual((when.cycle(1, 4), when.cycle(1, 2)), (when.QUARTERS, when.HALVES)) + october = datetime.datetime(2026, 10, 14, tzinfo=zoneinfo.ZoneInfo(self.MEL)) + for text, run in (('quarterly from feb on the 15th', '2026-11-15 09:00'), + ('bi-annual from mar', '2027-03-01 09:00'), ('quarterly', '2027-01-01 09:00')): + with self.subTest(text=text): + found = when.parse(text, self.MEL).next_after(october).astimezone(zoneinfo.ZoneInfo(self.MEL)) + self.assertEqual(found.strftime('%Y-%m-%d %H:%M'), run) if __name__ == '__main__': From ab34c55d29361aa7e278d3b9de396ce89e1a87cc Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 10:07:27 +0000 Subject: [PATCH 13/41] .jobs Add filled out: hourly, every..., one-off by year, start months, repeats to the minute MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - How often: one-off, hourly, daily, weekly, monthly, quarterly, bi-annual, annual, every... ("weekdays" moves under every... as "weekday", hourly out of it). - one-off: year (this one and the nine after), month, day (from today, tomorrow the default), hour, minute. - quarterly, bi-annual and annual: "start month?", then the day of the month (an annual's only the days its month has), hour and minute. - hourly and every 5 mins ... 12 hours: which days, start hour, start minute, then the end time among the runs after the start, or the end of the day. - every... weekday: hour and minute. - Each level's choices come from one place (offered), which also checks a press: a choice no screen offered (forged, or an older finbot's) is no step of Add. - "† indicates default" is in italics. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/reports/about.py | 247 ++++++++++++++++++++++++---------------- tests/test_scheduler.py | 121 +++++++++++++++----- 2 files changed, 242 insertions(+), 126 deletions(-) diff --git a/lib/reports/about.py b/lib/reports/about.py index 94500376..a1a1aaf0 100644 --- a/lib/reports/about.py +++ b/lib/reports/about.py @@ -1,5 +1,6 @@ """What finbot can do: the .help text, and the .jobs menu: a chat's jobs a page at a time, with buttons that switch, delete or run one; the steps of Add; the next runs; the system jobs; and the reports a job can run.""" +import calendar import datetime from lib import buttons @@ -25,17 +26,26 @@ ('performance', 'performance', None), ('cal earnings', 'cal', ('earnings',)), ('cal ex-dividend', 'cal', ('ex-dividend',)), ('shorts', 'shorts', ()), ('rating', 'rating', ()), ('milestone', 'milestone', ()), ('message', 'message', None)) -FREQUENCIES = (('o', 'one-off'), ('d', 'daily'), ('k', 'weekdays'), ('w', 'weekly'), ('m', 'monthly'), - ('q', 'quarterly'), ('b', 'bi-annual'), ('a', 'annual'), ('r', 'repeating')) -# Add's choices after how often, in turn; and the default of each, marked † on its button -LEVELS = {'o': ('date', 'hour', 'minute'), 'd': ('hour', 'minute'), 'k': ('hour', 'minute'), - 'w': ('weekday', 'hour', 'minute'), 'm': ('day', 'hour', 'minute'), 'q': ('day', 'hour', 'minute'), - 'b': ('day', 'hour', 'minute'), 'a': ('month', 'day', 'hour', 'minute'), 'r': ('interval', 'repeat')} -USUAL = {'frequency': 'd', 'weekday': 5, 'day': 1, 'month': 1, 'hour': 9, 'minute': 0, 'interval': 60, 'repeat': 'd'} +FREQUENCIES = (('o', 'one-off'), ('h', 'hourly'), ('d', 'daily'), ('w', 'weekly'), ('m', 'monthly'), + ('q', 'quarterly'), ('b', 'bi-annual'), ('a', 'annual'), ('r', 'every...')) +WEEKDAY = 'k' # every...'s weekday, among its intervals: a time each weekday +REPEAT = ('repeat', 'start hour', 'start minute', 'end') # a repeat's choices after its interval +# Add's choices after how often, in turn; every... takes a weekday's hour and minute instead, once it is a weekday +LEVELS = {'o': ('year', 'month', 'day', 'hour', 'minute'), 'h': REPEAT, 'd': ('hour', 'minute'), + 'w': ('weekday', 'hour', 'minute'), 'm': ('day', 'hour', 'minute'), 'q': ('month', 'day', 'hour', 'minute'), + 'b': ('month', 'day', 'hour', 'minute'), 'a': ('month', 'day', 'hour', 'minute'), 'r': ('interval', *REPEAT)} +CYCLES = {'q': 4, 'b': 2, 'a': 1} # months a year of quarterly, bi-annual and annual, from the start month chosen +CYCLE_NOTES = {'q': ['Then every 3 months'], 'b': ['And 6 months later']} # under the start month's question +END = 'e' # a repeat's end where it runs to the end of the day +# the default of each choice, marked † on its button; a one-off's are today's year and month, and tomorrow +USUAL = {'frequency': 'd', 'weekday': '5', 'day': '1', 'month': '1', 'hour': '9', 'minute': '0', 'interval': WEEKDAY, + 'repeat': 'd', 'start hour': '0', 'start minute': '0', 'end': END} MORE = 'p1' # Add's data for the second page of the days of a month -INTERVALS = (5, 10, 15, 20, 30, 60, 120, 180, 240, 360, 720) # minutes between a repeat's runs, as Add offers them +INTERVALS = (5, 10, 15, 20, 30, 120, 180, 240, 360, 720) # minutes between a repeat's runs, as every... offers them MINUTES = tuple(range(0, 60, 5)) # the minutes Add offers MOST_DAYS = {'discord': 20} # days of the month on a page: Discord takes 25 buttons, elsewhere all 31 +YEARS = 10 # the years a one-off may be in, this one first +CROWDED = 20 # a screen of more choices takes them in rows of five, Back last: Discord takes 25 buttons def jobs_screen(service, chat, screen='top', page=0, note=None, tappable=True): """A screen of the .jobs menu that lists a chat's jobs (by its key), numbered: the default ones (†), then those @@ -158,27 +168,31 @@ def moment_words(moment, local, zone=None): return said def add_screen(service, chat, steps): - """A step of Add, from the choices made so far (steps, from the button pressed: the report's number, how often, then - the LEVELS of that frequency in turn: a date, a day of the week or of the month, a month, the hour and the minute, - or a repeat's interval and its days): (lines, buttons) of the next choice, its default (USUAL) marked †, or (None, - (action, arguments, schedule)) once every choice is made, at a time by the timezone preference's clock. Cancel goes - back to the list, and Back, after the report, a step back. Raises ValueError for choices that are none (forged).""" + """A step of Add, from the choices made so far (steps, from the button pressed: the report's number, how often, + then the LEVELS of that frequency in turn): (lines, buttons) of the next choice, its default marked †, or (None, + (action, arguments, schedule)) once every choice is made, by the timezone preference's clock. Cancel goes back to + the list, and Back, after the report, a step back. Raises ValueError for choices that are none (forged data, or an + older finbot's).""" scheduler = jobs.current crc, local = jobs.list_checksum(scheduler.chat_jobs(chat)), formats.zone() - tomorrow = scheduler.clock().astimezone(local).date() + datetime.timedelta(days=1) + today = scheduler.clock().astimezone(local).date() steps, page = list(steps), 0 if steps and steps[-1] == MORE: steps, page = steps[:-1], 1 back = buttons.menu_button('Back', 'add', steps[:-1], crc) cancel = buttons.menu_button('Cancel', 'top', [0], crc) - def choices(heading, offered, row, lines=(), usual=None, nav=None): - """A choice's screen: its heading and lines, and '† indicates default' where one is (usual: its data); its - buttons (label, data), `row` to a row, the default's label marked †; then Back and Cancel (or nav).""" + def choices(heading, offer, row, lines=(), usual=None, nav=None): + """A choice's screen: its heading and lines, and '† indicates default' in italics where one is (usual: its + data); its buttons (label, data), `row` to a row, the default's label marked †; then Back and Cancel (or nav). + More than CROWDED choices, with no nav of their own, go in rows of five with Back last.""" tapped = [buttons.menu_button(label + ('†' if data == usual else ''), 'add', [*steps, data], crc) - for label, data in offered] - key = [] if usual is None else ['', '† indicates default'] - return [service.bold(heading), *lines, *key], [tapped[i:i + row] for i in range(0, len(tapped), row)] + [ - nav or [back, cancel]] + for label, data in offer] + key = [] if usual is None else ['', service.italic('† indicates default')] + text = [service.bold(heading), *lines, *key] + if nav is None and len(tapped) > CROWDED: + tapped.append(back) + return text, [tapped[i:i + 5] for i in range(0, len(tapped), 5)] + return text, [tapped[i:i + row] for i in range(0, len(tapped), row)] + [nav or [back, cancel]] if not steps: # a button per report, by its name alone, and Cancel return choices('Add a job: which report?', [(label, number) for number, (label, _, _) in enumerate(REPORTS, 1)], 3, nav=[cancel]) @@ -191,106 +205,141 @@ def choices(heading, offered, row, lines=(), usual=None, nav=None): if len(steps) == 1: return choices(f'Add {label}: how often?', [(name, code) for code, name in FREQUENCIES], 3, usual=USUAL['frequency']) - frequency, levels = steps[1], LEVELS.get(steps[1]) - if levels is None: - raise ValueError(f'no frequency {frequency!r}') - made = steps[2:] + frequency, made = steps[1], steps[2:] + levels = levels_of(frequency, made) if len(made) > len(levels) or (page and levels[len(made):len(made) + 1] != ('day',)): raise ValueError('no such choices') - values = {} + values = {'interval': 60} if frequency == 'h' else {} for level, data in zip(levels, made): - values[level] = chosen(level, data, values) + values[level] = chosen(level, data, frequency, values, today) if len(made) == len(levels): - if frequency == 'r': - schedule = when.Schedule('daily' if values['repeat'] == 'd' else 'weekdays', every=values['interval'], - zone=local.key) - else: - if frequency == 'o': - days = {'date': values['date'].isoformat()} - elif frequency == 'w': - days = (when.DAYS[values['weekday']],) - elif 'day' in values: - days = {'day': values['day'], 'months': {'m': (), 'q': when.QUARTERS, 'b': when.HALVES}.get(frequency, - (values.get('month'),))} - else: - days = 'daily' if frequency == 'd' else 'weekdays' - schedule = when.Schedule(days, at=values['hour'] * 60 + values['minute'], zone=local.key) - schedule.check() + schedule = built(frequency, values, local.key) return None, (action, list(args) if args is not None else catalog.period_args(action, schedule), schedule) level, said = levels[len(made)], so_far(frequency, values) - if level == 'date': - dates = [tomorrow + datetime.timedelta(days=n) for n in range(-1, 13)] - return choices(f'Add {label}, once: on which day?', [(f'{date:%a} {date.day} {date:%b}', f'{date:%y%m%d}') - for date in dates], 4, ['A later one is typed: .jobs add REPORT on 28 oct 2026 at 9am'], - f'{tomorrow:%y%m%d}') - if level == 'weekday': - return choices(f'Add {label}, weekly: on which day?', [(day.title(), number) for number, day in - enumerate(when.DAYS)], 4, usual=USUAL['weekday']) - if level == 'month': - return choices(f'Add {label}, annual: in which month?', [(month.title(), number) for number, month in - enumerate(when.MONTHS, 1)], 4, usual=USUAL['month']) + found, usual = offered(level, frequency, values, today) + offer = [(text, data) for text, data, _ in found] if level == 'day': - last = {2: 29, 4: 30, 6: 30, 9: 30, 11: 30}.get(values.get('month'), 31) most = MOST_DAYS.get(service, 31) - if page * most >= last: + if page * most >= len(offer): raise ValueError('no such page') nav = [back, cancel] if page: nav.insert(0, buttons.menu_button('◀', 'add', steps, crc)) - elif last > most: + elif len(offer) > most: nav.insert(0, buttons.menu_button('More ▶', 'add', [*steps, MORE], crc)) - return choices(f'Add {label}, {said}: on which day of the month?', [(str(day), day) for day in - range(1, last + 1)][page * most:(page + 1) * most], 5, ['A month without that day has no run'], - USUAL['day'], nav) - if level == 'hour': - lines, rows = choices(f'Add {label}, {said}: at what hour?', [(f'{hour:02d}', hour) for hour in range(24)], 5, - [f"By {when.place(local.key)}'s clock"], USUAL['hour']) - hours = [button for row in rows[:-1] for button in row] + [back] # 24 and Back: Discord takes 25 buttons - return lines, [hours[i:i + 5] for i in range(0, 25, 5)] - if level == 'minute': - hour = values['hour'] - return choices(f'Add {label}, {said}: at what time?', [(f'{hour:02d}:{minute:02d}', minute) for minute in - MINUTES], 4, usual=USUAL['minute']) + question = 'on which day?' if frequency == 'o' else 'on which day of the month?' + return choices(f'Add {label}, {said}: {question}', offer[page * most:(page + 1) * most], 5, + ['A month without that day has no run'] if frequency in 'mqb' else [], usual, nav) if level == 'interval': - return choices(f'Add {label}, repeating: how often?', [(when.repeat_words(minutes, 0, False).removeprefix( - 'every '), minutes) for minutes in INTERVALS], 4, ['A window of hours is typed: .jobs add REPORT every 20' - ' mins weekdays from 10am to 4pm'], USUAL['interval']) - return choices(f'Add {label}, {said}: on which days?', [('every day', 'd'), ('weekdays', 'k')], 2, - usual=USUAL['repeat']) + return choices(f'Add {label}: every...', offer, 4, ['A span across days is typed: .jobs add REPORT every 20' + ' mins from mon 9am to sat 3pm'], usual) + clock = [f"By {when.place(local.key)}'s clock"] + month = ('in which month?', 4, []) if frequency == 'o' else ('start month?', 4, CYCLE_NOTES.get(frequency, [])) + question, row, lines = {'year': ('in which year?', 5, []), 'month': month, + 'weekday': ('on which day?', 4, []), 'hour': ('at what hour?', 5, clock), 'minute': ('at what time?', 4, []), + 'repeat': ('on which days?', 2, []), 'start hour': ('start hour?', 5, clock), + 'start minute': ('start minute?', 4, []), 'end': ('end time?', 4, [])}[level] + return choices(f'Add {label}, {said}: {question}', offer, row, lines, usual) -def chosen(level, data, values): - """A choice's value at a level of Add, from a button's data (values: the choices before it). Raises ValueError for - one that is none.""" - if level == 'date': - if not (len(data) == 6 and data.isascii() and data.isdigit()): - raise ValueError(f'no date {data!r}') - return datetime.date(2000 + int(data[:2]), int(data[2:4]), int(data[4:])) +def levels_of(frequency, made): + """The choices Add asks for after how often (LEVELS), as far as those made say: every... asks a weekday's hour and + minute once its interval is a weekday. Raises ValueError for a frequency that is none.""" + if frequency not in LEVELS: + raise ValueError(f'no frequency {frequency!r}') + return ('interval', 'hour', 'minute') if frequency == 'r' and made[:1] == [WEEKDAY] else LEVELS[frequency] + +def offered(level, frequency, values, today): + """What a level of Add offers, from the choices before it (values) and today's date (a one-off's): its choices, + [(label, data, value)] (what each button shows, the data it carries, and what it means), and the default's data.""" + if level == 'year': + return [(str(year), str(year), year) for year in range(today.year, today.year + YEARS)], str(today.year) + if level == 'month': + first = today.month if frequency == 'o' and values['year'] == today.year else 1 + months = [(when.MONTHS[month - 1].title(), str(month), month) for month in range(first, 13)] + return months, str(first) if frequency == 'o' else USUAL[level] if level == 'day': - day = choice(data, 1, 31) - if 'month' in values: - datetime.date(2024, values['month'], day) # a day that month has, in a leap year - return day - if level in ('weekday', 'month', 'hour', 'minute', 'interval'): - return choice(data, *{'weekday': (0, 6), 'month': (1, 12), 'hour': (0, 23), 'minute': (0, 55, MINUTES), - 'interval': (1, 1440, INTERVALS)}[level]) - if data not in ('d', 'k'): - raise ValueError(f'no days {data!r}') - return data + if frequency == 'o': # the days the month has, from today on + year, month = values['year'], values['month'] + first = today.day if (year, month) == (today.year, today.month) else 1 + days = range(first, calendar.monthrange(year, month)[1] + 1) + tomorrow = today + datetime.timedelta(days=1) + usual = tomorrow.day if (tomorrow.year, tomorrow.month) == (year, month) else first + else: # annual: the days its month has, in a leap year; else every day a month may have + last = {2: 29, 4: 30, 6: 30, 9: 30, 11: 30}.get(values['month'], 31) if frequency == 'a' else 31 + days, usual = range(1, last + 1), USUAL[level] + return [(str(day), str(day), day) for day in days], str(usual) + if level in ('hour', 'start hour'): + return [(f'{hour:02d}', str(hour), hour) for hour in range(24)], USUAL[level] + if level in ('minute', 'start minute'): + hour = f"{values['hour']:02d}" if level == 'minute' else '' + return [(f'{hour}:{minute:02d}', str(minute), minute) for minute in MINUTES], USUAL[level] + if level == 'weekday': + return [(day.title(), str(number), number) for number, day in enumerate(when.DAYS)], USUAL[level] + if level == 'interval': + return [(when.repeat_words(minutes, 0, False).removeprefix('every '), str(minutes), minutes) + for minutes in INTERVALS] + [('weekday', WEEKDAY, WEEKDAY)], USUAL[level] + if level == 'repeat': + return [('every day', 'd', 'daily'), ('weekdays', 'k', 'weekdays')], USUAL[level] + # the end: each run after the start, a step apart (an hour, where it repeats more often), or the end of the day + start, step = values['start hour'] * 60 + values['start minute'], max(values['interval'], 60) + return [(when.clock24(end), str(end), end) for end in range(start + step, when.DAY_MINUTES, step)] + [ + ('end of day', END, None)], USUAL[level] + +def chosen(level, data, frequency, values, today): + """What a choice at a level of Add means, from its button's data, among what that level offers (offered). Raises + ValueError for one it does not offer.""" + for _, offer, value in offered(level, frequency, values, today)[0]: + if data == offer: + return value + raise ValueError(f'no {level} {data!r}') def so_far(frequency, values): - """How often a job being added runs, as far as it is chosen: 'weekly, Fridays', 'annual, in Mar', 'monthly'.""" + """How often a job being added runs, as far as it is chosen: 'once, Oct 2026', 'weekly, Fridays', 'quarterly, + Feb/May/Aug/Nov', 'hourly at :05, weekdays from 10am'.""" + if values.get('interval') not in (None, WEEKDAY): + start = values.get('start hour', 0) * 60 + values.get('start minute', 0) + return when.Schedule(values.get('repeat', 'daily'), every=values['interval'], start=start or None).words(False) if frequency == 'r': - return when.repeat_words(values['interval'], 0, False) if 'interval' in values else 'repeating' - if 'date' in values: - return f"once, {when.date_words(values['date'])}" + return 'weekdays' if values.get('interval') == WEEKDAY else 'every...' + if frequency == 'o': + if 'day' in values: + return 'once, ' + when.date_words(datetime.date(values['year'], values['month'], values['day'])) + if 'month' in values: + return f"once, {when.MONTHS[values['month'] - 1].title()} {values['year']}" + return f"once, {values['year']}" if 'year' in values else 'once' if 'weekday' in values: return f"weekly, {when.plural(when.DAYS[values['weekday']])}" + if frequency in CYCLES and 'month' in values: + months = when.cycle(values['month'], CYCLES[frequency]) + if 'day' in values: + return when.days_words({'day': values['day'], 'months': months}) + names = [when.MONTHS[month - 1].title() for month in months] + return dict(FREQUENCIES)[frequency] + ', ' + {'b': ' and '.join(names), 'a': f'in {names[0]}'}.get(frequency, + '/'.join(names)) if 'day' in values: - months = {'m': (), 'q': when.QUARTERS, 'b': when.HALVES}.get(frequency, (values.get('month'),)) - return when.days_words({'day': values['day'], 'months': months}) - if 'month' in values: - return f"annual, in {when.MONTHS[values['month'] - 1].title()}" - return dict(FREQUENCIES)[frequency].replace('one-off', 'once') + return when.days_words({'day': values['day'], 'months': ()}) + return dict(FREQUENCIES)[frequency] + +def built(frequency, values, zone): + """The when.Schedule that Add's choices make (values), in zone. Raises ScheduleError for one that cannot be.""" + if values.get('interval') not in (None, WEEKDAY): # hourly, or every so many minutes or hours + start = values['start hour'] * 60 + values['start minute'] + schedule = when.Schedule(values['repeat'], every=values['interval'], start=start or None, end=values['end'], + zone=zone) + else: + if frequency == 'o': + days = {'date': datetime.date(values['year'], values['month'], values['day']).isoformat()} + elif frequency == 'w': + days = (when.DAYS[values['weekday']],) + elif frequency in CYCLES: + days = {'day': values['day'], 'months': when.cycle(values['month'], CYCLES[frequency])} + elif frequency == 'm': + days = {'day': values['day'], 'months': ()} + else: + days = 'weekdays' if frequency == 'r' else 'daily' + schedule = when.Schedule(days, at=values['hour'] * 60 + values['minute'], zone=zone) + schedule.check() + return schedule def choice(text, low, high, among=None): """A choice's number from a button's data, from low to high (and among those offered, where given). Raises diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index aea45c5b..19f20602 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -792,8 +792,9 @@ def test_the_older_job_buttons_still_work(self): def test_add_by_buttons_back_and_cancel(self): """Add takes the report (by its name alone), how often, then the day, the hour and the minute from buttons, by - the timezone preference's clock, each choice's default marked † and the key under it; Cancel goes back to the - list, and after the report Back a step back; its last step adds the job and says so, the same job as typed.""" + the timezone preference's clock, each choice's default marked † and the key under it in italics; Cancel goes + back to the list, and after the report Back a step back; its last step adds the job and says so, the same job + as typed.""" lines, rows, data = self.press(f'm|add|{self.crc()}') self.assertEqual(lines, ['Add a job: which report?']) self.assertEqual(rows, [['trades', 'price midsession', 'price premarket'], ['price close', 'price', @@ -801,13 +802,14 @@ def test_add_by_buttons_back_and_cancel(self): ['Cancel']]) self.assertEqual((data['price'], data['Cancel']), (f'm|add|5|{self.crc()}', f'm|top|0|{self.crc()}')) lines, rows, data = self.press(data['price']) - self.assertEqual((lines, rows[:2]), (['Add price: how often?', '', '† indicates default'], - [['one-off', 'daily†', 'weekdays'], ['weekly', 'monthly', 'quarterly']])) + self.assertEqual((lines, rows), (['Add price: how often?', '', '† indicates default'], + [['one-off', 'hourly', 'daily†'], ['weekly', 'monthly', 'quarterly'], ['bi-annual', 'annual', 'every...'], + ['Back', 'Cancel']])) lines, rows, data = self.press(data['weekly']) self.assertEqual(rows[:2], [['Mon', 'Tue', 'Wed', 'Thu'], ['Fri', 'Sat†', 'Sun']]) self.assertEqual(data['Cancel'], f'm|top|0|{self.crc()}') lines, rows, data = self.press(data['Fri']) - self.assertEqual(lines[0], 'Add price, weekly, Fridays: at what hour?') + self.assertEqual(lines[:2], ['Add price, weekly, Fridays: at what hour?', "By Melbourne's clock"]) self.assertEqual((len(rows), [len(row) for row in rows], rows[1][4], rows[-1]), (5, [5] * 5, '09†', ['20', '21', '22', '23', 'Back'])) self.assertEqual(data['Back'], f'm|add|5|w|{self.crc()}') @@ -820,43 +822,105 @@ def test_add_by_buttons_back_and_cancel(self): zone='Australia/Melbourne')) self.assertIn('that job is here already', self.say('.jobs add price weekly on friday at 14:30')[0]) - def test_monthly_and_annual_take_their_day_hour_and_minute(self): - """Monthly asks for the day of the month, then the hour and the minute; annual first for its month, offering - only the days it has. On Discord the days of a month come on two pages.""" + def test_the_month_steps_start_a_cycle(self): + """Monthly asks for the day of the month; quarterly, bi-annual and annual first for the start month, the cycle + following from it, then the day, hour and minute; annual offers only the days its month has. On Discord the + days of a month come on two pages.""" lines, rows, data = self.press(f'm|add|6|m|{self.crc()}') - self.assertEqual((lines[0], rows[0], rows[-2], rows[-1]), ('Add performance, monthly: on which day of the' - ' month?', ['1†', '2', '3', '4', '5'], ['31'], ['Back', 'Cancel'])) + self.assertEqual((lines[:2], rows[0], rows[-2], rows[-1]), (['Add performance, monthly: on which day of the' + ' month?', 'A month without that day has no run'], ['1†', '2', '3', '4', '5'], ['31'], + ['Back', 'Cancel'])) lines, rows, data = self.press(data['15']) self.assertEqual(lines[0], 'Add performance, monthly, the 15th: at what hour?') lines, rows, data = self.press(data['09†']) lines, rows, data = self.press(data['09:00†']) self.assertEqual(lines, ['Added: 4. ✅* performance · monthly, the 15th at 9:00am']) - lines, rows, data = self.press(f'm|add|5|a|{self.crc()}') - self.assertEqual(rows[0], ['Jan†', 'Feb', 'Mar', 'Apr']) - lines, rows, data = self.press(data['Feb']) + for frequency, start, said, added in (('quarterly', 'Feb', 'quarterly, Feb/May/Aug/Nov', + 'quarterly, 15 Feb/May/Aug/Nov at 9:00am'), ('bi-annual', 'Mar', 'bi-annual, Mar and Sep', + 'bi-annual, 15 Mar and 15 Sep at 9:00am'), ('annual', 'Apr', 'annual, in Apr', + 'annual, 15 Apr at 9:00am')): + with self.subTest(frequency=frequency): + lines, rows, data = self.press(f'm|add|{self.crc()}') + lines, rows, data = self.press(data['price']) + lines, rows, data = self.press(data[frequency]) + self.assertEqual((lines[0], rows[0]), (f'Add price, {frequency}: start month?', ['Jan†', 'Feb', + 'Mar', 'Apr'])) + lines, rows, data = self.press(data[start]) + self.assertEqual(lines[0], f'Add price, {said}: on which day of the month?') + for pressed in ('15', '09†', '09:00†'): + lines, rows, data = self.press(data[pressed]) + self.assertTrue(lines[0].endswith(f'✅* price · {added}'), lines) + lines, rows, data = self.press(f'm|add|5|a|2|{self.crc()}') self.assertEqual((lines[0], rows[-2]), ('Add price, annual, in Feb: on which day of the month?', ['26', '27', '28', '29'])) self.assertEqual(about.add_screen(DISCORD, CHAT, ['5', 'm'])[1][-1][0].label, 'More ▶') self.assertEqual(about.add_screen(DISCORD, CHAT, ['5', 'm', 'p1'])[1][0][0].label, '21') - def test_a_repeat_a_one_off_and_a_message_by_buttons(self): - """Add's repeat takes its interval and days, a one-off its day among the next 14 (tomorrow the default), its - hour and its minute; a message's text cannot come from a button, so its screen says how to type one.""" - crc = self.crc() - self.assertEqual(self.press(f'm|add|1|r|20|k|{crc}')[0], ['Added: 4. ✅* trades · every 20 mins, weekdays']) + def test_repeats_by_buttons(self): + """hourly and every... take which days, the start hour and minute and the end time (or the end of the day); + every... offers a weekday too, which takes an hour and a minute; every default gives the all-day repeat.""" + lines, rows, data = self.press(f'm|add|5|r|{self.crc()}') + self.assertEqual((lines[0], rows), ('Add price: every...', [['5 mins', '10 mins', '15 mins', '20 mins'], + ['30 mins', '2 hours', '3 hours', '4 hours'], ['6 hours', '12 hours', 'weekday†'], ['Back', 'Cancel']])) + lines, rows, data = self.press(data['weekday†']) + self.assertEqual(lines[0], 'Add price, weekdays: at what hour?') + lines, rows, data = self.press(data['09†']) + self.assertEqual(self.press(data['09:00†'])[0], ['Added: 4. ✅* price · weekdays at 9:00am']) + lines, rows, data = self.press(f'm|add|9|h|{self.crc()}') + self.assertEqual((lines[0], rows[0]), ('Add shorts, hourly: on which days?', ['every day†', 'weekdays'])) + lines, rows, data = self.press(data['weekdays']) + self.assertEqual(lines[:2], ['Add shorts, hourly, weekdays: start hour?', "By Melbourne's clock"]) + lines, rows, data = self.press(data['10']) + self.assertEqual((lines[0], rows[0]), ('Add shorts, hourly, weekdays from 10am: start minute?', + [':00†', ':05', ':10', ':15'])) + lines, rows, data = self.press(data[':05']) + self.assertEqual((lines[0], rows[0], rows[-2:]), ('Add shorts, hourly at :05, weekdays from 10am: end' + ' time?', ['11:05', '12:05', '13:05', '14:05'], [['23:05', 'end of day†'], ['Back', 'Cancel']])) + self.assertEqual(self.press(data['16:05'])[0], ['Added: 5. ✅* shorts · hourly at :05, weekdays 10am–4pm']) + self.assertEqual(self.press(f'm|add|10|h|d|0|0|e|{self.crc()}')[0], ['Added: 6. ✅* rating · hourly']) + self.assertEqual(self.press(f'm|add|11|r|20|k|10|5|965|{self.crc()}')[0], ['Added: 7. ✅* milestone · every 20' + ' mins, weekdays 10:05am–4:05pm']) + self.assertEqual(self.press(f'm|add|7|r|20|k|0|0|e|{self.crc()}')[0], ['Added: 8. ✅* cal earnings · every 20' + ' mins, weekdays']) + screen = about.add_screen(DISCORD, CHAT, ['5', 'h', 'd', '0', '0']) + rows = [[button.label for button in row] for row in screen[1]] + self.assertEqual((len(rows), rows[-1]), (5, ['21:00', '22:00', '23:00', 'end of day†', 'Back'])) # 25 buttons + + def test_a_one_off_by_year_month_day_hour_and_minute(self): + """A one-off takes its year (this one and the nine after it), month, day, hour and minute, never one gone by, + tomorrow its default day; a message's text cannot come from a button, so its screen says how to type one.""" lines, rows, data = self.press(f'm|add|9|o|{self.crc()}') - self.assertEqual((rows[0][:2], len(sum(rows, [])) - 2), (['Wed 14 Oct', 'Thu 15 Oct†'], 14)) - lines, rows, data = self.press(data['Tue 20 Oct']) + self.assertEqual((lines[0], rows[:2]), ('Add shorts, once: in which year?', [['2026†', '2027', '2028', + '2029', '2030'], ['2031', '2032', '2033', '2034', '2035']])) + lines, rows, data = self.press(data['2026†']) + self.assertEqual((lines[0], rows[0]), ('Add shorts, once, 2026: in which month?', + ['Oct†', 'Nov', 'Dec'])) + lines, rows, data = self.press(data['Oct†']) + self.assertEqual((lines[0], rows[0], rows[-2]), ('Add shorts, once, Oct 2026: on which day?', ['14', + '15†', '16', '17', '18'], ['29', '30', '31'])) + lines, rows, data = self.press(f'm|add|9|o|2027|{self.crc()}') + self.assertEqual(rows[0], ['Jan†', 'Feb', 'Mar', 'Apr']) + lines, rows, data = self.press(data['Feb']) + self.assertEqual((rows[0][0], rows[-2]), ('1†', ['26', '27', '28'])) + lines, rows, data = self.press(data['20']) + self.assertEqual(lines[0], 'Add shorts, once, 20 Feb 2027: at what hour?') lines, rows, data = self.press(data['08']) - self.assertEqual(self.press(data['08:00†'])[0], ['Added: 5. ✅* shorts · once, 20 Oct 2026 at 8:00am']) + self.assertEqual(self.press(data['08:00†'])[0], ['Added: 4. ✅* shorts · once, 20 Feb 2027 at 8:00am']) lines, rows, _ = self.press(f'm|add|12|{self.crc()}') self.assertIn('.jobs add message SCHEDULE "TEXT"', lines) self.assertEqual(rows, [['Back', 'Cancel']]) - for data in (f'm|add|13|{crc}', f'm|add|5|x|{crc}', f'm|add|5|w|7|9|{crc}', f'm|add|5|d|24|{crc}', - f'm|add|1|r|7|k|{crc}', f'm|add|5|o|261340|9|{crc}', f'm|add|5|d|9|9|{crc}', f'm|add|5|d|9|0|0|{crc}', - f'm|add|5|a|2|30|{crc}', f'm|add|5|w|p1|{crc}', f'm|add|5|m|p1|{crc}', f'm|add|5|m|def|{crc}'): + + def test_a_step_that_is_none_adds_nothing(self): + """Data that is no step of Add (forged, or an older finbot's: the weekdays frequency, the next 14 days, every... + hourly) adds nothing and says so: a report, frequency, year, month or day not offered, a day gone by, 30 Feb, an + end not among the runs, a page where there is none.""" + crc = self.crc() + for data in ('13', '5|x', '5|k', '5|o|261020', '5|r|60', '5|w|7|9', '5|d|24', '5|d|9|9', '5|d|9|0|0', + '1|r|7', '5|o|2025', '5|o|2036', '5|o|2026|9', '5|o|2026|10|13', '5|o|2027|2|30', '5|a|2|30', '5|q|13', + '5|w|p1', '5|m|def', '5|h|d|10|5|600', '5|h|d|10|5|1000', '5|h|d|10|5|e|0'): with self.subTest(data=data): - self.assertEqual(self.press(data)[0][0], 'That is no step of Add') + self.assertEqual(self.press(f'm|add|{data}|{crc}')[0][0], 'That is no step of Add') + self.assertEqual(len(self.s.chat_jobs(CHAT)), 3) def test_every_screen_fits_each_network(self): """With every default job and 30 jobs added, each page of each screen fits a message of each network (Discord @@ -881,8 +945,11 @@ def fits(service, lines, markup): with self.subTest(service=service, screen=screen, page=page): fits(service, *about.jobs_screen(service, CHAT, screen, page)) for steps in ([], ['5'], ['5', 'w'], ['5', 'w', '5'], ['5', 'w', '5', '14'], ['5', 'm'], ['5', 'm', 'p1'], - ['5', 'a'], ['5', 'a', '2'], ['5', 'a', '2', '29'], ['5', 'o'], ['5', 'o', '261020'], ['5', 'r'], - ['5', 'r', '20'], ['12']): + ['5', 'q'], ['5', 'q', '2'], ['5', 'b', '3'], ['5', 'a'], ['5', 'a', '2'], ['5', 'a', '2', '29'], + ['5', 'o'], ['5', 'o', '2026'], ['5', 'o', '2026', '10'], ['5', 'o', '2027', '1'], + ['5', 'o', '2027', '1', 'p1'], ['5', 'o', '2027', '1', '31'], ['5', 'h'], ['5', 'h', 'd'], + ['5', 'h', 'd', '0'], ['5', 'h', 'd', '0', '0'], ['5', 'r'], ['5', 'r', 'k'], ['5', 'r', 'k', '9'], + ['5', 'r', '20'], ['5', 'r', '20', 'k', '10'], ['5', 'r', '20', 'k', '10', '5'], ['12']): if steps[-1:] == ['p1'] and service != DISCORD: # Telegram and Slack take every day on one page continue with self.subTest(service=service, steps=steps): From b4d046b5fc5e78f9ac3d82ebb445e7f37a56e12c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 10:17:50 +0000 Subject: [PATCH 14/41] .jobs: Add, Remove, Enable, Disable, Run now, Upcoming...; default jobs like any other MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - The top level is Add, Remove, Enable, Disable, Run now and Upcoming..., two to a row, and the list ends with how Add is typed, in italics. - Enable has a button for each job that is off and Enable all; Disable for each one that is on and Disable all; each says what it did ("Enabled 3. trades"), and with nothing to switch the screen says so. A press switches one way only, so a stale one cannot flip a job back. - Remove has a button for every job, the default ones too, and Remove all, which asks "Remove all N jobs here?" first. A removed default job is kept in the chat's record ('deleted') and comes back when the same job is added again. - No "Default jobs:"/"Added here:" headings, no † on job names; the key is "* not run yet" alone. - Typed: .jobs enable|disable|remove|run NUMBER (toggle and del still work); alone, each draws its screen. /jobs suggests the new words, the numbers of jobs off after enable and on after disable. - A slash command's description takes the usage lines that fit Discord's 100 characters. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/buttons.py | 2 +- lib/discord_common.py | 12 ++- lib/jobs.py | 68 +++++++++++++--- lib/reports/about.py | 100 +++++++++++++---------- lib/worker.py | 86 ++++++++++++-------- tests/test_discord.py | 14 ++-- tests/test_scheduler.py | 174 ++++++++++++++++++++++++---------------- util/record_outputs.py | 7 +- 8 files changed, 292 insertions(+), 171 deletions(-) diff --git a/lib/buttons.py b/lib/buttons.py index 653c4c83..8e87ba79 100644 --- a/lib/buttons.py +++ b/lib/buttons.py @@ -38,7 +38,7 @@ class MenuPress(typing.NamedTuple): JOB_ACTIONS = {'j': 'run', 'e': 'toggle'} # an older job button's kind, its data's first letter, and what it does -MENU_SCREENS = ('top', 'on', 'del', 'run', 'add', 'next') # the .jobs menu's screens (lib/reports/about.py) +MENU_SCREENS = ('top', 'on', 'off', 'del', 'run', 'add', 'next') # the .jobs menu's screens (lib/reports/about.py) PERIODS = {'d': ('1D', 1), 'w': ('7D', 7), 'm': ('1M', 30), 'y': ('1Y', 365), 'x': ('Max', None)} diff --git a/lib/discord_common.py b/lib/discord_common.py index bfe3aa23..16a19836 100644 --- a/lib/discord_common.py +++ b/lib/discord_common.py @@ -75,10 +75,18 @@ def command(name, description, arguments, completes=False): """A slash command as Discord takes it: a chat command usable in servers and in a DM with the bot, and the arguments it takes, if any, as one optional text option that they describe; with completes, Discord asks finbot what to suggest as they are typed (an autocomplete interaction).""" - option = {'type': 3, 'name': OPTION, 'description': arguments[:100], 'required': False, 'autocomplete': completes} - return {'name': name, 'type': 1, 'description': description[:100], 'contexts': [GUILD, BOT_DM], + option = {'type': 3, 'name': OPTION, 'description': fit(arguments), 'required': False, 'autocomplete': completes} + return {'name': name, 'type': 1, 'description': fit(description), 'contexts': [GUILD, BOT_DM], 'integration_types': [0], 'options': [option] if arguments else []} +def fit(text, most=100): + """A description within Discord's 100 characters: whole where it fits, else cut after the last of its usage lines + (joined by ' | ') that does.""" + if len(text) <= most: + return text + cut = text.rfind(' | ', 0, most + 1) + return text[:cut] if cut > 0 else text[:most] + def summary(commands): """What of a list of slash commands Discord may differ on from ours, in a form to compare: an answer from Discord diff --git a/lib/jobs.py b/lib/jobs.py index 01334fc6..748b212b 100644 --- a/lib/jobs.py +++ b/lib/jobs.py @@ -40,8 +40,9 @@ SYSTEM = (('sharesight_warm', (), {'every': 30, 'from': '00:14'}), ('yahoo_warm', (), {'every': 480, 'from': '00:07'}), ('cleanup', ('30',), {'days': ['sat'], 'at': '18:51'})) WARM = ('sharesight_warm', 'yahoo_warm') # the jobs that fill the cache, run at once after a flush too -# {chat key: {'jobs': [{action, args, schedule, on}], 'off': [catalog keys]}}; a chat whose default jobs start off (a -# direct message) holds 'defaults': 'off' and 'on' instead, the catalog keys switched on +# {chat key: {'jobs': [{action, args, schedule, on}], 'off': [catalog keys], 'deleted': [catalog keys]}}: the jobs it +# added, the default jobs it switched off and those it removed; a chat whose default jobs start off (a direct message) +# holds 'defaults': 'off' and 'on' instead of 'off', the catalog keys switched on JOBS_FILE = 'finbot_chat_jobs.json' OLD_FILE = 'finbot_schedule.json' # each chat's cron jobs, from before the catalog: read for its chats alone STATE_FILE = 'finbot_jobs.json' # each job's last run, by key: a restart neither repeats a run nor loses one @@ -200,12 +201,22 @@ def switch_item(record, key, on): """Switch a default job on or off in a chat's record: whether it is on now.""" starts_on = record.get('defaults') != 'off' name = 'off' if starts_on else 'on' # the list of those switched from how they start - listed = [each for each in record.get(name, []) if each != key] + ([key] if on != starts_on else []) + keep(record, name, [each for each in record.get(name, []) if each != key] + ([key] if on != starts_on else [])) + return on + + +def removed(record): + """The default jobs (by catalog key) that a chat's record says it removed.""" + listed = record.get('deleted', []) + return [key for key in listed if isinstance(key, str)] if isinstance(listed, list) else [] + + +def keep(record, name, listed): + """Set a list of a chat's record, or leave it out where it is empty.""" if listed: record[name] = listed else: record.pop(name, None) - return on def switch_record(record, on): @@ -307,8 +318,9 @@ def load(self): chats = self.stored_chats() items = catalog.items(config.prefs.country_code) for chat, record in chats.items(): + gone = set(removed(record)) jobs += [make_job(item.action, item.args, item.schedule, chat, item_on(record, item.key), item.key, - item.name) for item in items] + item.name) for item in items if item.key not in gone] added = record.get('jobs', []) keys = set() for entry in added if isinstance(added, list) else []: @@ -511,13 +523,18 @@ def change(self, chat, edit): def add(self, chat, action, args, schedule): """Add a job to a chat, switched on: an action of ACTIONS, its arguments, a when.Schedule. One the catalog has - (catalog.matching) is that default job, switched on. Returns its number in .jobs. Raises ScheduleError for a job - that cannot be, a one-off gone by, one the chat has already, or one more than MAX_JOBS.""" + (catalog.matching) is that default job, switched on, and back in the list where the chat removed it. Returns its + number in .jobs. Raises ScheduleError for a job that cannot be, a one-off gone by, one the chat has already, or + one more than MAX_JOBS.""" job = make_job(action, args, schedule, chat) if schedule.frequency == 'once' and schedule.next_after(self.clock()) is None: raise ScheduleError(f"{when.days_words(schedule.days)} has gone by: a one-off is for a time ahead") if item := catalog.matching(job.action, job.args, schedule, config.prefs.country_code): - self.change(chat, lambda record: switch_item(record, item.key, True)) + def restore(record): + """Bring the default job back where the chat removed it, and switch it on.""" + keep(record, 'deleted', [key for key in removed(record) if key != item.key]) + switch_item(record, item.key, True) + self.change(chat, restore) return self.number(chat, f'{chat}|{item.key}') def append(record): """Append the job, unless the chat has it already, within MAX_JOBS.""" @@ -535,10 +552,37 @@ def switch_all(self, chat, on): self.change(chat, lambda record: switch_record(record, on)) def delete(self, job): - """Delete a job its chat added (from job() or pick()); a default one raises ScheduleError: switch it off.""" - if job.item: - raise ScheduleError(f'{job.name} is a default job: switch it off instead (.jobs toggle)') - self.change(job.chat, lambda record: record['jobs'].pop(self.index(record, job))) + """Remove a chat's job (from job() or pick()): one it added, or a default one, which stays out of its list + until the same job is added again.""" + def remove(record): + """Remove the job, as stored.""" + if not job.item: + return record['jobs'].pop(self.index(record, job)) + for name in ('off', 'on'): + keep(record, name, [key for key in record.get(name, []) if key != job.item]) + record['deleted'] = list(dict.fromkeys([*removed(record), job.item])) + self.change(job.chat, remove) + + def delete_all(self, chat): + """Remove every job of a chat: those it added, and the default ones, which stay out of its list until the same + job is added again.""" + def clear(record): + """Remove every job, as stored.""" + record['jobs'] = [] + record.pop('off', None) + record.pop('on', None) + record['deleted'] = list(dict.fromkeys([*removed(record), *(item.key for item in catalog.ITEMS)])) + self.change(chat, clear) + + def switch(self, job, on): + """Switch a chat's job on, or off, whichever way it is now: whether it is on.""" + def turn(record): + """Set the job's switch, as stored.""" + if job.item: + return switch_item(record, job.item, on) + record['jobs'][self.index(record, job)]['on'] = on + return on + return self.change(job.chat, turn) def toggle(self, job): """Switch a chat's job on or off: whether it is on now.""" diff --git a/lib/reports/about.py b/lib/reports/about.py index a1a1aaf0..5dec75e7 100644 --- a/lib/reports/about.py +++ b/lib/reports/about.py @@ -1,5 +1,5 @@ -"""What finbot can do: the .help text, and the .jobs menu: a chat's jobs a page at a time, with buttons that switch, -delete or run one; the steps of Add; the next runs; the system jobs; and the reports a job can run.""" +"""What finbot can do: the .help text, and the .jobs menu: a chat's jobs a page at a time, with buttons that enable, +disable, remove or run one; the steps of Add; the upcoming runs; the system jobs; and the reports a job can run.""" import calendar import datetime @@ -10,13 +10,16 @@ from lib import jobs from lib import when -HEADINGS = {'top': 'Jobs here', 'on': 'Tap a job to switch it on or off', 'run': 'Tap a job to run it now', - 'del': 'Tap a job added here to delete it (a default job is switched off instead)'} -TYPED = ['.jobs run|toggle|del NUMBER: run one now, switch it on or off, delete one added here', +HEADINGS = {'top': 'Jobs here', 'on': 'Tap a job to enable it', 'off': 'Tap a job to disable it', + 'del': 'Tap a job to remove it', 'run': 'Tap a job to run it now'} +NOTHING = {'on': 'Every job is on', 'off': 'Every job is off'} # what Enable and Disable say with nothing to switch +EVERY = {'on': 'Enable all', 'off': 'Disable all', 'del': 'Remove all'} # each list screen's button for every job +TYPED = ['.jobs enable|disable|remove|run NUMBER: switch one on or off, remove one, run one now', '.jobs add REPORT SCHEDULE: add one (.jobs add lists the reports)', '.jobs upcoming: the next runs', '.jobs system: the system jobs'] # what a chat without buttons types instead -TYPED_ACTIONS = {'on': 'toggle', 'del': 'del', 'run': 'run'} -NOT_RUN, DEFAULT = '* not run yet', '† default (no delete, only disable)' # the key to a list's marks +TYPED_ACTIONS = {'on': 'enable', 'off': 'disable', 'del': 'remove', 'run': 'run'} +ADD_TYPED = ['/jobs add REPORT SCHEDULE', '/jobs add message SCHEDULE "TEXT"'] # how Add is typed: under the list +NOT_RUN = '* not run yet' # the key to a list's mark PAGE_ROOM = 500 # characters of a screen besides its jobs' lines: its headings, a note, the key to * JOB_ROW = 4 # job buttons in a row MOST_JOBS = {'discord': 16} # job buttons on a page: Discord takes five rows of five, one kept for the others; else 40 @@ -48,14 +51,14 @@ CROWDED = 20 # a screen of more choices takes them in rows of five, Back last: Discord takes 25 buttons def jobs_screen(service, chat, screen='top', page=0, note=None, tappable=True): - """A screen of the .jobs menu that lists a chat's jobs (by its key), numbered: the default ones (†), then those - added here, each ✅ (on) or ❌ (off), * where it has never run, its name and its schedule in words, by the timezone - preference's clock (lib/when.py), then the key to the marks shown. top has what can be done (Enable/disable, - Delete, Add, Run now, Next); on, del and run a button per job, which switches it (and All, every job), deletes it - (one added here: a default one has none) or runs it now. A page at a time, as many jobs as a message of the - network takes (◀ and More ▶ for the others), after a note of what the last press did. A one-off gone by is deleted - first. Returns (lines, buttons); where the chat takes no buttons (tappable false), the whole list and what to type - instead.""" + """A screen of the .jobs menu that lists a chat's jobs (by its key), numbered, each ✅ (on) or ❌ (off), * where it + has never run, its name and its schedule in words, by the timezone preference's clock (lib/when.py), then the key + to the marks shown. top has what can be done (Add, Remove, Enable, Disable, Run now, Upcoming...) and how Add is + typed, in italics; on, off, del and run a button for each job the screen acts on (acts), which switches it on or + off, removes it or runs it now, and Enable all, Disable all or Remove all. A page at a time, as many jobs as a + message of the network takes (◀ and More ▶ for the others), after a note of what the last press did. A one-off + gone by is deleted first. Returns (lines, buttons); where the chat takes no buttons (tappable false), the whole + list and what to type instead.""" scheduler = jobs.current if scheduler is None: return ['No schedule is running'], None @@ -69,21 +72,23 @@ def jobs_screen(service, chat, screen='top', page=0, note=None, tappable=True): shown = parts[page] span = f' ({shown[0][0]}–{shown[-1][0]} of {len(own)})' if len(parts) > 1 else '' lines = ([note, ''] if note else []) + [service.bold(HEADINGS[screen] + span)] - if screen == 'del' and all(job.item for job in own): - lines.append('None added here yet: Add makes one') - kind = None - for number, job, line in shown: - if kind != (job.item is not None): # a heading where the default jobs start, and those added here - kind = job.item is not None - lines += ['', service.bold('Default jobs:' if kind else 'Added here:')] - lines.append(line) - key = [text for text, marked in ((NOT_RUN, not all(scheduler.has_run(job) for _, job, _ in shown)), - (DEFAULT, any(job.item for _, job, _ in shown))) if marked] - if key: - lines += ['', ', '.join(key)] + if not own: + lines.append('No jobs here: Add makes one') + elif screen in NOTHING and not any(acts(screen, job) for job in own): + lines.append(NOTHING[screen]) + if shown: + lines += ['', *(line for _, _, line in shown)] + if not all(scheduler.has_run(job) for _, job, _ in shown): + lines += ['', NOT_RUN] if not tappable: return lines + [''] + (TYPED if screen == 'top' else [f'Send .jobs {TYPED_ACTIONS[screen]} NUMBER']), None - return lines, screen_buttons(screen, page, len(parts), shown, jobs.list_checksum(own), all(job.on for job in own)) + if screen == 'top': + lines += ['', *(service.italic(service.escape(line)) for line in ADD_TYPED)] + return lines, screen_buttons(screen, page, len(parts), shown, jobs.list_checksum(own), own) + +def acts(screen, job): + """Whether a list screen acts on a job: Enable on one that is off, Disable on one that is on, the others on any.""" + return {'on': not job.on, 'off': job.on}.get(screen, True) def paged(entries, service): """The jobs' entries (number, job, line) a page at a time: as many as fit a message of the network with room to @@ -98,23 +103,22 @@ def paged(entries, service): size += len(entry[2]) + 1 return parts -def screen_buttons(screen, page, pages, shown, crc, every_on=False): - """A list screen's buttons: top's, what can be done; on, del and run's, one for each job shown that the screen - acts on, ' ' (with its ✅ or ❌ to switch it), JOB_ROW to a row, then on's All, which switches every - job of the chat on (All off where every_on: off), and Cancel, back to the list; ◀ and More ▶ to the other pages.""" +def screen_buttons(screen, page, pages, shown, crc, own): + """A list screen's buttons: top's, what can be done, two to a row; on, off, del and run's, one for each job shown + that the screen acts on (acts), ' ', JOB_ROW to a row, then Enable all, Disable all or Remove all + where it acts on any of the chat's jobs (own), and Cancel, back to the list; ◀ and More ▶ to the other pages.""" if screen == 'top': - rows = [[buttons.menu_button('Enable/disable', 'on', [0], crc), buttons.menu_button('Delete', 'del', [0], crc)], - [buttons.menu_button('Add', 'add', [], crc), buttons.menu_button('Run now', 'run', [0], crc), - buttons.menu_button('Upcoming', 'next', [], crc)]] + rows = [[buttons.menu_button(label, kind, args, crc) for label, kind, args in pair] for pair in ( + (('Add', 'add', []), ('Remove', 'del', [0])), (('Enable', 'on', [0]), ('Disable', 'off', [0])), + (('Run now', 'run', [0]), ('Upcoming...', 'next', [])))] nav = [] else: - tapped = [buttons.menu_button(f"{number} {('✅ ' if job.on else '❌ ') if screen == 'on' else ''}{job.name}", - screen, [page, number], crc) for number, job, _ in shown if screen != 'del' or not job.item] + tapped = [buttons.menu_button(f'{number} {job.name}', screen, [page, number], crc) for number, job, _ in shown + if acts(screen, job)] rows = [tapped[i:i + JOB_ROW] for i in range(0, len(tapped), JOB_ROW)] nav = [buttons.menu_button('Cancel', 'top', [0], crc)] - if screen == 'on': - nav.insert(0, buttons.menu_button('All off' if every_on else 'All on', 'on', [page, 'alloff' if every_on - else 'allon'], crc)) + if screen in EVERY and any(acts(screen, job) for job in own): + nav.insert(0, buttons.menu_button(EVERY[screen], screen, [page, 'all'], crc)) if page > 0: nav.append(buttons.menu_button('◀', screen, [page - 1], crc)) if page < pages - 1: @@ -122,12 +126,20 @@ def screen_buttons(screen, page, pages, shown, crc, every_on=False): return rows + ([nav] if nav else []) def job_entry(service, scheduler, number, job, running=(), local=None, now=None): - """'5. ✅ price close AU† · weekdays at 4:30pm Sydney': a job in .jobs, with its number, whether it is on (and * - where it has never run), its name (and † for a default job), its schedule in words, and whether it is running.""" + """'5. ✅ price close AU · weekdays at 4:30pm Sydney': a job in .jobs, with its number, whether it is on (and * + where it has never run), its name, its schedule in words, and whether it is running.""" mark = ('✅' if job.on else '❌') + ('' if scheduler.has_run(job) else '*') - name = service.escape(job.name) + ('†' if job.item else '') schedule = job.schedule.describe(local or formats.zone().key, now or scheduler.clock()) - return f"{number}. {mark} {name} · {schedule}" + (' (running)' if job.key in running else '') + return f"{number}. {mark} {service.escape(job.name)} · {schedule}" + (' (running)' if job.key in running else '') + +def remove_all_screen(service, chat, page=0): + """Remove all's question, for a chat (by its key): whether to remove every one of its jobs, the default ones too, + with Yes, remove all and Cancel.""" + own = jobs.current.chat_jobs(chat) + crc = jobs.list_checksum(own) + return [service.bold(f"Remove all {len(own)} job{'' if len(own) == 1 else 's'} here?"), 'A default job removed' + ' comes back only when the same job is added again'], [[buttons.menu_button('Yes, remove all', 'del', [page, + 'allyes'], crc), buttons.menu_button('Cancel', 'top', [0], crc)]] def job_line(number, job, outcome): """'3. trades: done': a job's number in the .jobs list, its name, and how its run went.""" diff --git a/lib/worker.py b/lib/worker.py index 2c637f68..64a1ea1e 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -51,7 +51,7 @@ def process_job_callback(press, pressed): chat = delivery.reply_to(service, press.chat_id, interaction=press.interaction, pressed=True) def redraw(): """Put the menu's screen that does the same, as the list is now, in place of the pressed one.""" - lines, markup = about.jobs_screen(service, chat.key, 'on' if pressed.action == 'toggle' else 'run') + lines, markup = about.jobs_screen(service, chat.key, 'top' if pressed.action == 'toggle' else 'run') chat.edit_text(press.message_id, '\n'.join(lines), markup or []) try: if jobs.current is None: @@ -92,23 +92,29 @@ def menu_screen(service, chat, pressed): return add_step(service, chat, args) page = int(args[0]) if args and args[0].isdigit() and len(args[0]) < 4 else 0 note = None - if len(args) == 2 and screen in ('on', 'del', 'run'): + if len(args) == 2 and screen in ('on', 'off', 'del', 'run'): own, number = scheduler.chat_jobs(key), args[1] - every = screen == 'on' and number in ('allon', 'alloff') # All: every job switched on, or off + every = number == 'all' and screen != 'run' or number == 'allyes' and screen == 'del' # every job at once if not (hmac.compare_digest(jobs.list_checksum(own), pressed.checksum) and (every or number.isdigit() and 1 <= int(number) <= len(own))): note = 'The jobs have changed: here they are now' + elif number == 'all' and screen == 'del': # Remove all asks first + return about.remove_all_screen(service, key, page) + elif every and screen == 'del': + scheduler.delete_all(key) + return about.jobs_screen(service, key, 'top', 0, 'Every job here is removed') elif every: - scheduler.switch_all(key, number == 'allon') - note = f"Every job is {number.removeprefix('all')} now" + scheduler.switch_all(key, screen == 'on') + note = f'Every job is {screen} now' else: job = own[int(number) - 1] try: - if screen == 'on': - scheduler.toggle(job) + if screen in ('on', 'off'): + scheduler.switch(job, screen == 'on') + note = f"{'Enabled' if screen == 'on' else 'Disabled'} {number}. {service.escape(job.name)}" elif screen == 'del': scheduler.delete(job) - note = f'Deleted {number}. {service.escape(job.name)}' + note = f'Removed {number}. {service.escape(job.name)}' else: delivery.deliver(chat, delivery.Report([about.job_line(number, job, scheduler.run(job))])) except jobs.ScheduleError as e: @@ -434,10 +440,10 @@ def on_who(request, args): request.reply(holdings.prepare_who(request.service, re.split(r'[.:]', args.words[0].upper())[0])) def on_jobs(request, args): - """.jobs: this chat's jobs, the default ones and those added here, with the menu's buttons; .jobs upcoming: the - next runs; .jobs system: the system's; .jobs run, toggle or del NUMBER: one run now, switched on or off, or deleted - (one added here), and without a number the list with a button per job that does it; .jobs add REPORT SCHEDULE, or - .jobs add message SCHEDULE "TEXT": one more (alone, the reports a job can run).""" + """.jobs: this chat's jobs, with the menu's buttons; .jobs upcoming: the next runs; .jobs system: the system's; + .jobs enable, disable, remove, run or toggle NUMBER: one switched on or off, removed, run now, or switched the + other way, and without a number the list with a button per job that does it; .jobs add REPORT SCHEDULE, or .jobs + add message SCHEDULE "TEXT": one more (alone, the reports a job can run).""" scheduler, chat, service = jobs.current, request.dest.key, request.service tappable = request.dest.takes_buttons if scheduler is None: @@ -460,21 +466,23 @@ def on_jobs(request, args): number = scheduler.add(chat, *jobs.parse_add(text, formats.zone().key)) request.reply([about.job_entry(service, scheduler, number, scheduler.chat_jobs(chat)[number - 1]), 'Added: .jobs lists them all']) - elif args.keyword in ('run', 'toggle', 'del') and not args.words: - screen = {'run': 'run', 'toggle': 'on', 'del': 'del'}[args.keyword] - lines, markup = about.jobs_screen(service, chat, screen, tappable=tappable) + elif args.keyword in JOB_SCREENS and not args.words: + lines, markup = about.jobs_screen(service, chat, JOB_SCREENS[args.keyword], tappable=tappable) request.reply(lines, buttons=markup) - elif args.keyword in ('run', 'toggle', 'del') and len(args.words) == 1: + elif args.keyword in JOB_SCREENS and len(args.words) == 1: number = args.words[0] job = scheduler.job(chat, number) if args.keyword == 'run': request.reply([about.job_line(number, job, scheduler.run(job))]) - elif args.keyword == 'toggle': - scheduler.toggle(job) - request.reply([about.job_entry(service, scheduler, number, scheduler.job(chat, number))]) - else: + elif args.keyword == 'del': scheduler.delete(job) - request.reply([f"Deleted {number}. {job.name} · {job.schedule.describe(formats.zone().key)}"]) + request.reply([f"Removed {number}. {job.name} · {job.schedule.describe(formats.zone().key)}"]) + else: + if args.keyword == 'toggle': + scheduler.toggle(job) + else: + scheduler.switch(job, args.keyword == 'enable') + request.reply([about.job_entry(service, scheduler, number, scheduler.job(chat, number))]) else: raise Usage except jobs.ScheduleError as e: # a job that cannot be, or a number that is none: said, as an answer @@ -513,8 +521,10 @@ class Command: PLANS = 'finbot_plan.json' # each chat's plans, {user: plan}, as state.load_for keeps them BY = dict.fromkeys(('sector', 'sectors'), 'sector') | dict.fromkeys(('industry', 'industries'), 'industry') # .pie's # what .jobs does to a chat's jobs, and shows; next is what upcoming was called before -JOB_CHANGES = {'run': 'run', 'toggle': 'toggle', 'add': 'add', 'system': 'system'} | dict.fromkeys(('upcoming', - 'next'), 'upcoming') | dict.fromkeys(('del', 'delete', 'rem', 'remove', 'rm'), 'del') +JOB_CHANGES = {'run': 'run', 'toggle': 'toggle', 'enable': 'enable', 'disable': 'disable', 'add': 'add', + 'system': 'system'} | dict.fromkeys(('upcoming', 'next'), 'upcoming') | dict.fromkeys(('del', 'delete', 'rem', + 'remove', 'rm'), 'del') +JOB_SCREENS = {'enable': 'on', 'disable': 'off', 'del': 'del', 'run': 'run', 'toggle': 'top'} # each word's screen BUY_SELL = ('buy', 'sell', *(a + sep + b for a, b in (('buy', 'sell'), ('sell', 'buy')) for sep in ('', '&', '/', '+'))) PROFILE = Command(('stock', 'profile'), on_profile, ('.SYMBOL', '.stock SYMBOL'), COMPANY) # any first word that is no # command's name but may be a ticker calls it too @@ -551,8 +561,8 @@ class Command: Command(('trades', 'trade'), on_trades, ('.trades [period] [portfolio]',), TRADES, ('.trades',), period=True), Command(('plan',), on_plan, ('.plan [your plan]',), TRADES), Command(('jobs', 'job', 'cron', 'crons', 'schedule'), on_jobs, ('.jobs [upcoming|system]', - '.jobs run|toggle|del [NUMBER]', '.jobs add REPORT SCHEDULE', '.jobs add message SCHEDULE "TEXT"'), JOBS, - keywords=JOB_CHANGES), + '.jobs enable|disable|remove|run [NUMBER]', '.jobs add REPORT SCHEDULE', '.jobs add message SCHEDULE "TEXT"'), + JOBS, keywords=JOB_CHANGES), PROFILE, ] @@ -605,11 +615,22 @@ def slash_commands(aliases=True): if each and arguments not in found.setdefault(each, (command, []))[1]: found[each][1].append(arguments) return [Slash('help', 'every command, and what it takes', '') if name == 'help' else - Slash(name, f"{name} {' | '.join(lines)}".strip(), ' | '.join(lines), completes(command)) + Slash(name, described(name, lines), ' | '.join(lines), completes(command)) for name, (command, lines) in found.items()] +def described(name, lines, most=100): + """A slash command's description: its name, then as many of its usage lines (the arguments of each, joined by + ' | ') as fit in most characters, Discord's limit.""" + text = name + for line in filter(None, lines): + longer = f"{text}{' ' if text == name else ' | '}{line}" + if len(longer) > most: + break + text = longer + return text + PERIOD_WORDS = ('1d', '7d', '1m', '3m', '6m', 'ytd', '1y', '5y', '10y') # suggested where a command takes a period -JOB_WORDS = ('add', 'run', 'toggle', 'del', 'upcoming', 'system') # what /jobs suggests first +JOB_WORDS = ('add', 'remove', 'enable', 'disable', 'run', 'upcoming', 'system') # what /jobs suggests first SCHEDULE_WORDS = ('daily', 'weekdays', 'weekly', 'monthly', 'quarterly', 'bi-annual', 'annual', 'hourly', 'every', 'at', 'from', 'between', 'on') # what a schedule in English is made of, suggested after a report @@ -664,16 +685,17 @@ def argument_words(command, chat, given): return [(word, word) for word in dict.fromkeys(words)] def job_words(chat, words): - """(word, shown) pairs for /jobs, for suggest, after the words given: first its subcommands; after run or toggle, - this chat's job numbers, each shown with its name (after del, those of the jobs added here); after add, the reports, - each shown with what it takes; after a report, the words it takes, then those of a schedule.""" + """(word, shown) pairs for /jobs, for suggest, after the words given: first its subcommands; after remove, run or + toggle, this chat's job numbers, each shown with its name (after enable, those of the jobs off; after disable, + those on); after add, the reports, each shown with what it takes; after a report, the words it takes, then those + of a schedule.""" if not words: return [(word, word) for word in JOB_WORDS] action = JOB_CHANGES.get(words[0].lower()) - if action in ('run', 'toggle', 'del') and len(words) == 1: + if action in JOB_SCREENS and len(words) == 1: own = jobs.current.chat_jobs(chat) if jobs.current else [] return [(str(number), f'{number}: {job.name}') for number, job in enumerate(own, 1) - if action != 'del' or not job.item] + if {'enable': not job.on, 'disable': job.on}.get(action, True)] if action != 'add': return [] if len(words) == 1: diff --git a/tests/test_discord.py b/tests/test_discord.py index c6b2a65a..00f982b6 100644 --- a/tests/test_discord.py +++ b/tests/test_discord.py @@ -387,9 +387,9 @@ def test_keywords_periods_portfolios_and_tickers(self): self.assertEqual(self.values('/plan buy '), []) def test_the_jobs_subcommands_numbers_and_reports(self): - """/jobs suggests its subcommands; after run or toggle this chat's job numbers, each shown with its name (after - del, those of the jobs it added); after add, the reports a chat can schedule, each shown with what it takes; - then what the report takes, and the words of a schedule.""" + """/jobs suggests its subcommands; after run or remove this chat's job numbers, each shown with its name (after + enable, those of the jobs that are off; after disable, those on); after add, the reports a chat can schedule, + each shown with what it takes; then what the report takes, and the words of a schedule.""" self.patch(jobs, 'SYSTEM', ()) self.patch(catalog, 'ITEMS', (catalog.Item('rating', 'rating', 'rating', (), catalog.at('daily')),)) state.save(jobs.JOBS_FILE, {'discord:77': {'jobs': [ @@ -399,11 +399,13 @@ def test_the_jobs_subcommands_numbers_and_reports(self): with contextlib.redirect_stderr(io.StringIO()): scheduler.load() self.patch(jobs, 'current', scheduler) - self.assertEqual(self.values('/jobs '), ['add', 'run', 'toggle', 'del', 'upcoming', 'system']) - self.assertEqual(self.values('/job T'), ['toggle']) + self.assertEqual(self.values('/jobs '), ['add', 'remove', 'enable', 'disable', 'run', 'upcoming', 'system']) + self.assertEqual(self.values('/job E'), ['enable']) self.assertEqual(worker.suggest('discord:77', '/jobs run '), [('run 1: rating', 'run 1'), ('run 2: trades', 'run 2'), ('run 3: shorts', 'run 3')]) - self.assertEqual(self.values('/jobs del '), ['del 2', 'del 3']) + self.assertEqual(self.values('/jobs remove '), ['remove 1', 'remove 2', 'remove 3']) + self.assertEqual(self.values('/jobs enable '), ['enable 3']) # shorts is off + self.assertEqual(self.values('/jobs disable '), ['disable 1', 'disable 2']) self.assertEqual(self.values('/jobs run ', chat='discord:88'), []) reports = worker.suggest('discord:77', '/jobs add ') self.assertIn(('add price midsession|interday|premarket|close|DAYS', 'add price'), reports) diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index 19f20602..848047d3 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -506,25 +506,26 @@ def crc(self, chat=CHAT): class JobsInChat(ChatBase): """What a chat types: .jobs lists its jobs; run, toggle, add and del; next and system.""" - def test_add_list_and_del(self): - """A job is added, listed under its chat's default jobs with its number, its switch, * till it runs, its name - and its schedule in words, and deleted; a default job is not.""" + def test_add_list_and_remove(self): + """A job is added; the list shows each job with its number, its switch, * till it runs, its name and its + schedule in words, then how Add is typed, in italics, over the buttons; a job is removed, a default one too, + which stays out of the list.""" self.assertEqual(self.say('.jobs add shorts daily at 10pm'), ['4. ✅* shorts · daily at 10:00pm', 'Added: .jobs lists them all']) - self.assertEqual(self.say('.jobs'), ['Jobs here', '', 'Default jobs:', - '1. ✅* rating† · Tue–Sat at 8:30am Sydney', - '2. ✅* price close US† · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', '', 'Added here:', - '3. ✅* trades · every 20 mins, weekdays UTC', '4. ✅* shorts · daily at 10:00pm', '', - '* not run yet, † default (no delete, only disable)']) + self.assertEqual(self.say('.jobs'), ['Jobs here', '', '1. ✅* rating · Tue–Sat at 8:30am Sydney', + '2. ✅* price close US · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', + '3. ✅* trades · every 20 mins, weekdays UTC', '4. ✅* shorts · daily at 10:00pm', '', '* not run yet', '', + '/jobs add REPORT SCHEDULE', '/jobs add message SCHEDULE "TEXT"']) self.assertEqual([[button.label for button in row] for row in self.replies[-1][2]], - [['Enable/disable', 'Delete'], ['Add', 'Run now', 'Upcoming']]) - self.assertEqual(self.say('.jobs del 3'), ['Deleted 3. trades · every 20 mins, weekdays UTC']) - self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('shorts daily at 10pm', - zone='Australia/Melbourne')]}) - self.assertEqual([job.name for job in self.s.chat_jobs(CHAT)], ['rating', 'price close US', 'shorts']) - self.assertEqual(self.say('.jobs del 1'), ['rating is a default job: switch it off instead (.jobs toggle)']) - self.assertEqual(self.say('.job del 3'), ['Deleted 3. shorts · daily at 10:00pm']) - self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': []}) + [['Add', 'Remove'], ['Enable', 'Disable'], ['Run now', 'Upcoming...']]) + self.assertEqual(self.say('.jobs del 3'), ['Removed 3. trades · every 20 mins, weekdays UTC']) + shorts = entry('shorts daily at 10pm', zone='Australia/Melbourne') + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [shorts]}) + self.assertEqual(self.say('.jobs remove 1'), ['Removed 1. rating · Tue–Sat at 8:30am Sydney']) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [shorts], 'deleted': ['rating']}) + self.assertEqual([job.name for job in self.s.chat_jobs(CHAT)], ['price close US', 'shorts']) + self.assertEqual(self.say('.job del 2'), ['Removed 2. shorts · daily at 10:00pm']) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [], 'deleted': ['rating']}) def test_add_reads_english_and_the_period_follows_how_often(self): """A job is a report, what it takes, and a schedule in English, by the timezone preference's clock unless it @@ -590,11 +591,15 @@ def test_a_chat_adds_thirty_jobs_at_most(self): def test_a_job_the_defaults_have_switches_that_one_on(self): """A job typed as a default one is (in a zone that keeps its clock: Melbourne for Sydney) switches that one on, - and nothing more is stored.""" + and brings it back where it was removed; nothing more is stored.""" self.say('.jobs toggle 1') - self.assertEqual(self.say('.jobs add rating tue-sat at 8:30am'), ['1. ✅* rating† · Tue–Sat at 8:30am Sydney', + self.assertEqual(self.say('.jobs add rating tue-sat at 8:30am'), ['1. ✅* rating · Tue–Sat at 8:30am Sydney', 'Added: .jobs lists them all']) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('trades every 20 mins weekdays')]}) + self.say('.jobs remove 1') + self.assertEqual([job.name for job in self.s.chat_jobs(CHAT)], ['price close US', 'trades']) + self.assertEqual(self.say('.jobs add rating tue-sat at 8:30am')[0], '1. ✅* rating · Tue–Sat at 8:30am Sydney') + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('trades every 20 mins weekdays')]}) def test_run_a_job_now(self): """.jobs run N runs job N now, posting to this chat, and says how it went; a number that is none says so; @@ -605,6 +610,18 @@ def test_run_a_job_now(self): self.assertEqual(self.say('.jobs run 1', chat='56'), ['1. rating: done']) # a chat seen first has the defaults self.assertEqual(self.dests, [CHAT, 'telegram:56']) + def test_enable_and_disable_a_job(self): + """.jobs disable N switches job N off and .jobs enable N on, whichever way it was; .jobs enable alone lists the + jobs with a button for each one that is off.""" + for typed, line in (('.jobs disable 3', '3. ❌* trades'), ('.jobs disable 3', '3. ❌* trades'), + ('.jobs enable 3', '3. ✅* trades')): + with self.subTest(typed=typed): + self.assertEqual(self.say(typed), [f'{line} · every 20 mins, weekdays UTC']) + self.say('.jobs disable 1') + self.assertEqual(self.say('.jobs enable')[0], 'Tap a job to enable it') + self.assertEqual([[button.label for button in row] for row in self.replies[-1][2]], + [['1 rating'], ['Enable all', 'Cancel']]) + def test_toggle_a_job_off_and_on(self): """.jobs toggle N switches job N off (it no longer fires) and on again.""" self.assertEqual(self.say('.jobs toggle 3'), ['3. ❌* trades · every 20 mins, weekdays UTC']) @@ -615,20 +632,20 @@ def test_toggle_a_job_off_and_on(self): self.assertEqual([job.name for job, _ in self.s.due(self.clock.now)], ['rating', 'trades']) def test_a_job_is_marked_till_it_first_runs(self): - """* marks a job that has never run, on its schedule or by hand, and † a default job; the list ends with what - the marks it shows mean.""" + """* marks a job that has never run, on its schedule or by hand; the list says what it means while it shows + it.""" self.say('.jobs run 3') lines = self.say('.jobs') self.assertIn('3. ✅ trades · every 20 mins, weekdays UTC', lines) - self.assertIn('1. ✅* rating† · Tue–Sat at 8:30am Sydney', lines) + self.assertIn('1. ✅* rating · Tue–Sat at 8:30am Sydney', lines) self.clock.now = at(13, 21, 31) # Wednesday 8:31am in Sydney: rating has run, New York's close not yet self.s.tick() lines = self.say('.jobs') - self.assertIn('1. ✅ rating† · Tue–Sat at 8:30am Sydney', lines) - self.assertIn('2. ✅* price close US† · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', lines) - self.assertEqual(lines[-1], '* not run yet, † default (no delete, only disable)') + self.assertIn('1. ✅ rating · Tue–Sat at 8:30am Sydney', lines) + self.assertIn('2. ✅* price close US · weekdays at 4:30pm New York (7:30am local, Tue–Sat)', lines) + self.assertIn('* not run yet', lines) self.say('.jobs run 2') - self.assertEqual(self.say('.jobs')[-1], '† default (no delete, only disable)') + self.assertNotIn('* not run yet', self.say('.jobs')) def test_a_one_off_gone_by_is_deleted_as_the_list_is_shown(self): """A one-off job whose time has gone by (run or off) is deleted as the list is next shown, never on a tick.""" @@ -696,56 +713,70 @@ class JobsMenu(ChatBase): """The .jobs menu's buttons: each screen drawn in place of the pressed one, with a page of the list and a button per job; Add's steps; a stale or forged press acting on nothing.""" - def test_enable_disable_flips_the_switch_in_the_button_and_the_message(self): - """Enable/disable lists the jobs with a button each, ' '; a press switches the job and - redraws, the switch flipped in both; Cancel goes back to the list.""" - crc = self.crc() - lines, rows, data = self.press(f'm|on|0|{crc}') + def test_enable_and_disable_switch_one_way(self): + """Disable lists the jobs with a button for each one that is on, ' ', and Enable for each one that + is off; a press switches the job that way, says so and redraws; Cancel goes back to the list.""" + lines, rows, data = self.press(f'm|off|0|{self.crc()}') self.assertEqual(self.toasts, ['Updating…']) - self.assertEqual(lines[0], 'Tap a job to switch it on or off') - self.assertEqual(rows, [['1 ✅ rating', '2 ✅ price close US', '3 ✅ trades'], ['All off', 'Cancel']]) - lines, rows, data = self.press(data['1 ✅ rating']) - self.assertIn('1. ❌* rating† · Tue–Sat at 8:30am Sydney', lines) - self.assertEqual(rows[0][0], '1 ❌ rating') + self.assertEqual((lines[0], rows), ('Tap a job to disable it', [['1 rating', '2 price close US', + '3 trades'], ['Disable all', 'Cancel']])) + lines, rows, data = self.press(data['1 rating']) + self.assertEqual(lines[:2], ['Disabled 1. rating', '']) + self.assertIn('1. ❌* rating · Tue–Sat at 8:30am Sydney', lines) + self.assertEqual(rows[0], ['2 price close US', '3 trades']) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT]['off'], ['rating']) - self.press(data['1 ❌ rating']) - self.assertNotIn('off', state.load(jobs.JOBS_FILE)[CHAT]) - lines, rows, _ = self.press(data['Cancel']) - self.assertEqual((lines[0], rows[0]), ('Jobs here', ['Enable/disable', 'Delete'])) - - def test_all_switches_every_job_on_or_off(self): - """Enable/disable's All switches every job of the chat off where every one is on, else on, and says so; its - label says which. Pressed for another list, it switches nothing.""" + self.assertEqual(self.press(data['2 price close US'])[0][0], 'Disabled 2. price close US') lines, rows, data = self.press(f'm|on|0|{self.crc()}') - self.assertEqual(rows[-1], ['All off', 'Cancel']) - lines, rows, data = self.press(data['All off']) - self.assertEqual(lines[0], 'Every job is off now') - self.assertEqual(rows, [['1 ❌ rating', '2 ❌ price close US', '3 ❌ trades'], ['All on', 'Cancel']]) + self.assertEqual((lines[0], rows), ('Tap a job to enable it', [['1 rating', '2 price close US'], + ['Enable all', 'Cancel']])) + lines, rows, data = self.press(data['1 rating']) + self.assertEqual((lines[0], rows[0]), ('Enabled 1. rating', ['2 price close US'])) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT]['off'], ['price-close-us']) + lines, rows, _ = self.press(data['Cancel']) + self.assertEqual((lines[0], rows[0]), ('Jobs here', ['Add', 'Remove'])) + + def test_enable_all_and_disable_all(self): + """Disable all switches every job of the chat off and Enable all every one on, each saying so, and each shown + only where some job has that to change: with nothing to switch, the screen says so. Pressed for another list, + it switches nothing.""" + lines, rows, data = self.press(f'm|off|0|{self.crc()}') + lines, rows, data = self.press(data['Disable all']) + self.assertEqual((lines[:4], rows), (['Every job is off now', '', 'Tap a job to disable it', + 'Every job is off'], [['Cancel']])) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'defaults': 'off', # the default jobs start off now 'jobs': [entry('trades every 20 mins weekdays', on=False)]}) - lines, rows, data = self.press(data['3 ❌ trades']) - self.assertEqual(rows[-1], ['All on', 'Cancel']) # some on, some off: All switches them all on - lines, rows, data = self.press(data['All on']) - self.assertEqual(lines[0], 'Every job is on now') + lines, rows, data = self.press(f'm|on|0|{self.crc()}') + self.assertEqual(rows, [['1 rating', '2 price close US', '3 trades'], ['Enable all', 'Cancel']]) + lines, rows, data = self.press(data['Enable all']) + self.assertEqual((lines[0], lines[3], rows), ('Every job is on now', 'Every job is on', [['Cancel']])) self.assertTrue(all(job.on for job in self.s.chat_jobs(CHAT))) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [entry('trades every 20 mins weekdays')]}) - self.assertEqual(self.press('m|on|0|alloff|0000')[0][0], 'The jobs have changed: here they are now') + self.assertEqual(self.press('m|off|0|all|0000')[0][0], 'The jobs have changed: here they are now') self.assertTrue(all(job.on for job in self.s.chat_jobs(CHAT))) - def test_delete_offers_only_the_jobs_added_here(self): - """Delete lists every job, the default ones marked so and without a button, and a button for each one added - here; a press deletes it, saying so; a forged press on a default job deletes nothing.""" + def test_remove_offers_every_job_and_remove_all_asks_first(self): + """Remove lists every job with a button each, the default ones too: a press removes it and says so, a default + one staying out of the list (stored as removed). Remove all asks first; Yes empties the list, and a stale + press removes nothing.""" lines, rows, data = self.press(f'm|del|0|{self.crc()}') - self.assertIn('1. ✅* rating† · Tue–Sat at 8:30am Sydney', lines) - self.assertEqual(lines[-1], '* not run yet, † default (no delete, only disable)') - self.assertEqual(rows, [['3 trades'], ['Cancel']]) - self.assertEqual(self.press(f'm|del|0|1|{self.crc()}')[0][0], 'rating is a default job: switch it off instead' - ' (.jobs toggle)') - lines, rows, data = self.press(data['3 trades']) - self.assertEqual(lines[:2], ['Deleted 3. trades', '']) - self.assertIn('None added here yet: Add makes one', lines) - self.assertEqual(rows, [['Cancel']]) - self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': []}) + self.assertEqual((lines[0], lines[-1]), ('Tap a job to remove it', '* not run yet')) + self.assertEqual(rows, [['1 rating', '2 price close US', '3 trades'], ['Remove all', 'Cancel']]) + lines, rows, data = self.press(data['1 rating']) + self.assertEqual((lines[:2], rows), (['Removed 1. rating', ''], [['1 price close US', '2 trades'], + ['Remove all', 'Cancel']])) + self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'deleted': ['rating'], + 'jobs': [entry('trades every 20 mins weekdays')]}) + lines, rows, data = self.press(data['Remove all']) + self.assertEqual((lines, rows), (['Remove all 2 jobs here?', 'A default job removed comes back only when' + ' the same job is added again'], [['Yes, remove all', 'Cancel']])) + self.assertEqual(self.press('m|del|0|allyes|0000')[0][0], 'The jobs have changed: here they are now') + self.assertEqual(len(self.s.chat_jobs(CHAT)), 2) + lines, rows, data = self.press(data['Yes, remove all']) + self.assertEqual(lines[:4], ['Every job here is removed', '', 'Jobs here', + 'No jobs here: Add makes one']) + self.assertEqual((self.s.chat_jobs(CHAT), state.load(jobs.JOBS_FILE)[CHAT]), ([], {'deleted': ['rating', + 'price-close-us'], 'jobs': []})) + self.assertEqual(self.press(f'm|del|0|{self.crc()}')[1], [['Cancel']]) def test_run_now_runs_the_job_and_says_how_it_went(self): """Run now lists the jobs with a button each; a press runs the job and says how it went.""" @@ -766,8 +797,8 @@ def test_a_stale_or_forged_press_acts_on_nothing(self): """A press drawn for another list (its checksum), for a number that is none, or of no screen at all switches, deletes and runs nothing: the screen is drawn as the list is now, or the press is ignored.""" good = self.crc() - for data in (f'm|on|0|1|0000', f'm|del|0|3|0000', f'm|run|0|3|0000', f'm|on|0|9|{good}', f'm|on|0|0|{good}', - f'm|run|0|x|{good}'): + for data in (f'm|on|0|1|0000', f'm|off|0|1|0000', f'm|del|0|3|0000', f'm|run|0|3|0000', f'm|on|0|9|{good}', + f'm|on|0|0|{good}', f'm|run|0|x|{good}', f'm|run|0|all|{good}', f'm|on|0|allyes|{good}'): with self.subTest(data=data): lines, _, _ = self.press(data) self.assertEqual(lines[0], 'The jobs have changed: here they are now') @@ -779,14 +810,14 @@ def test_a_stale_or_forged_press_acts_on_nothing(self): def test_the_older_job_buttons_still_work(self): """A job button of an older list ('j|' runs a job, 'e|' switches one) still works, by the job's own checksum; - a switch redraws the menu's screen that switches jobs.""" + a switch redraws the list.""" job = self.s.chat_jobs(CHAT)[2] self.press(f'j|3|{jobs.checksum(job.key)}') self.assertEqual((self.toasts, self.ran), (['Running…'], [('trades', {})])) lines, rows, _ = self.press(f'e|3|{jobs.checksum(job.key)}') self.assertFalse(self.s.chat_jobs(CHAT)[2].on) - self.assertEqual(lines[0], 'Tap a job to switch it on or off') - self.assertEqual(rows[0][2], '3 ❌ trades') + self.assertEqual((lines[0], rows[0]), ('Jobs here', ['Add', 'Remove'])) + self.assertIn('3. ❌ trades · every 20 mins, weekdays UTC', lines) lines, _, _ = self.press('e|3|0000') # stale: the screen as the list is now, nothing switched self.assertFalse(self.s.chat_jobs(CHAT)[2].on) @@ -940,7 +971,7 @@ def fits(service, lines, markup): self.s.add(CHAT, 'message', [f'reminder number {minute} ' * 4], when.parse(f'daily at 1:{minute:02d}', 'UTC')) for service in (TELEGRAM, SLACK, DISCORD): - for screen in ('top', 'on', 'del', 'run'): + for screen in ('top', 'on', 'off', 'del', 'run'): for page in range(6): with self.subTest(service=service, screen=screen, page=page): fits(service, *about.jobs_screen(service, CHAT, screen, page)) @@ -955,6 +986,7 @@ def fits(service, lines, markup): with self.subTest(service=service, steps=steps): fits(service, *about.add_screen(service, CHAT, steps)) fits(service, *about.next_screen(service, CHAT)) + fits(service, *about.remove_all_screen(service, CHAT)) fits(service, *about.added_screen(service, CHAT, 1)) fits(service, *about.add_failed(service, CHAT, ['5', 'd', '9'], 'that job is here already')) pages = len({about.jobs_screen(DISCORD, CHAT, 'on', page)[0][0] for page in range(6)}) diff --git a/util/record_outputs.py b/util/record_outputs.py index b3902625..2c5cfd50 100755 --- a/util/record_outputs.py +++ b/util/record_outputs.py @@ -24,10 +24,11 @@ '.performance 7d', '.performance 1y', '.performance Nope', '.pie Rob', '.pie Rob sector', '.pie', '.compare T1 T2', '.compare T1 T2 3m', '.compare T1', '.history T1', '.history', '.watchlist', '.watchlist add MSFT', '.watchlist add MSFT', '.watchlist del MSFT', '.watchlist frob X', '.T1', '.jobs', '.jobs run', '.jobs toggle', - '.jobs add', '.jobs add shorts daily at 22:00', '.jobs del 4', '.jobs run 1', '.jobs upcoming', '.jobs system', - 'hi @bot', 'noise'] + '.jobs enable', '.jobs disable 1', '.jobs add', '.jobs add shorts daily at 22:00', '.jobs del 4', '.jobs run 1', + '.jobs upcoming', '.jobs system', 'hi @bot', 'noise'] BUTTONS = ['c|h|m|T1', 'c|h|x|T1', 'c|p|w|T1', 'c|c|y|T1,T2', 'c|f|y|', 'c|l|d|3,0', 'c|l|m|0,10', 'c|zz|m|T1', - 'j|1|0000', 'e|1|0000', 'm|on|0|0000', 'm|add|0000', 'm|add|5|w|0000', 'm|next|0000'] + 'j|1|0000', 'e|1|0000', 'm|on|0|0000', 'm|off|0|0000', 'm|del|0|0000', 'm|add|0000', 'm|add|5|w|0000', + 'm|next|0000'] JOBS = ['*/20 * * * Mon-Fri trades', '29 21 * * Mon-Fri price interday', # each chat's, for .jobs (as cron had them) '28 21 * * Sat cal earnings; cal ex-dividend; price 7; performance 7; rating'] ADDED = ['trades every 20 mins weekdays', 'price daily at 21:29', 'cal earnings every saturday at 21:28'] # since cron From 0b471607b3935b6a3cd4e3a4b69c7270a4aa31d4 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 10:18:43 +0000 Subject: [PATCH 15/41] Telegram keyboard: .schedule in place of .beta The command keyboard of a direct message has .schedule (the jobs) where .beta was, sorted with the rest; its version changes, so each DM gets the new one. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/worker.py | 4 ++-- tests/test_reports.py | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/lib/worker.py b/lib/worker.py index 64a1ea1e..d90662d9 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -542,7 +542,7 @@ class Command: period=True), Command(('marketcap', 'maletas', 'marketer'), on_marketcap, ('.marketcap [SYMBOL|top|bottom]',), COMPANY, ('.marketcap',), keywords=TOP), - Command(('beta',), on_beta, ('.beta',), PRICE, ('.beta',)), + Command(('beta',), on_beta, ('.beta',), PRICE), Command(('compare',), on_compare, ('.compare SYMBOL SYMBOL [... up to 6] [period]',), PRICE, period=True, keywords={'max': 'max', 'all': 'max'}), # max: as the default, 10 years Command(('history', 'hospital', 'visual', 'hosiery'), on_history, ('.history SYMBOL',), PRICE), @@ -562,7 +562,7 @@ class Command: Command(('plan',), on_plan, ('.plan [your plan]',), TRADES), Command(('jobs', 'job', 'cron', 'crons', 'schedule'), on_jobs, ('.jobs [upcoming|system]', '.jobs enable|disable|remove|run [NUMBER]', '.jobs add REPORT SCHEDULE', '.jobs add message SCHEDULE "TEXT"'), - JOBS, keywords=JOB_CHANGES), + JOBS, ('.schedule',), keywords=JOB_CHANGES), PROFILE, ] diff --git a/tests/test_reports.py b/tests/test_reports.py index c1455ebc..50ccb930 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -58,12 +58,12 @@ def test_one_button_asks_for_both_ratings_reports(self): self.assertEqual(len(self.command('.buy & buy')), 1) # each report once def test_the_keyboard_is_alphabetical_and_every_button_is_its_command(self): - """The command keyboard is alphabetical, and each button runs the command it belongs to in the table, not taken - for a ticker by .SYMBOL.""" + """The command keyboard is alphabetical, .schedule (the jobs) in place of .beta, and each button runs the + command it belongs to in the table, not taken for a ticker by .SYMBOL.""" buttons = [button['text'] for row in worker.command_keyboard()['keyboard'] for button in row] self.assertEqual(buttons, sorted(buttons)) - self.assertIn('.buy & sell', buttons) - self.assertIn('.forward pe', buttons) + self.assertEqual(buttons, ['.buy & sell', '.dividend', '.earnings', '.forward pe', '.marketcap', '.pe', '.peg', + '.performance', '.premarket', '.price', '.schedule', '.session', '.shorts', '.trades', '.watchlist']) for command in worker.COMMANDS: for text in command.buttons: with self.subTest(text=text): From f1aefa66defc6507a0917cb57e5b93be74e854aa Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 10:26:21 +0000 Subject: [PATCH 16/41] finbot without Sharesight: the watchlist alone; README says so - sharesight.configured(): whether its keys are in finbot.env (yahoo_warm and sharesight_warm ask it too). - Without them, sharesight.tickers() is empty, so a chat's reports cover its watchlist alone and Sharesight is never asked; sharesight.portfolios() raises NotSetUp, an answer, so .holdings, .trades, .performance, .who and .pie say "Sharesight is not set up: its keys go in finbot.env (doc/configuration.md)". - The trades and performance jobs are default jobs only with Sharesight; a chat cannot add a job that reports on it, and Add does not offer one. - README: "If you don't have Sharesight, you can still add your holdings manually via the watchlist"; doc/configuration.md says what changes without it. Before, every report that covers the watchlist failed without Sharesight's keys, and the trades job posted an error every 20 minutes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- README.md | 2 ++ doc/configuration.md | 1 + lib/catalog.py | 9 ++++++--- lib/jobs.py | 5 ++++- lib/reports/about.py | 7 ++++--- lib/reports/sharesight_warm.py | 2 +- lib/reports/yahoo_warm.py | 4 ++-- lib/sharesight.py | 22 +++++++++++++++++++--- tests/common.py | 3 ++- tests/test_reports.py | 22 ++++++++++++++++++++++ tests/test_schedule.py | 7 +++++++ tests/test_scheduler.py | 10 ++++++++++ 12 files changed, 80 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index ed351689..93c7338c 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,8 @@ finbot is a chat bot for Telegram, Slack and Discord that follows your [Sharesig portfolios and each chat's watchlist with Yahoo Finance data. It posts trades, price moves, earnings and ex-dividend dates, analyst rating changes and more on a schedule, and answers commands with reports and charts. +If you don't have Sharesight, you can still add your holdings manually via the watchlist (`/watchlist add SYMBOL`). + ## Features * Telegram, Slack and Discord (either the push or the pull method, depending on your environment) * Sharesight trades as they happen, portfolio performance comparisons diff --git a/doc/configuration.md b/doc/configuration.md index 5c7f1d14..ead230a3 100644 --- a/doc/configuration.md +++ b/doc/configuration.md @@ -107,6 +107,7 @@ sharesight_code=01d692d4de7 samplefc64bc2e2f01d692d4de72986ea808f6e99813f sharesight_client_id=01d692d4de7 samplefc64bc2e2f01d692d4de72986ea808f6e99813f sharesight_client_secret=01d692d4de7 samplefc64bc2e2f01d692d4de72986ea808f6e99813f ``` +* Sharesight is optional. Without its keys, a chat's reports cover its watchlist alone, so add your holdings to it (`/watchlist add SYMBOL`); `/holdings`, `/trades`, `/performance`, `/who` and `/pie` say Sharesight is not set up, and the trades and performance jobs are not among the default jobs. ## Portfolios Portfolios are auto-discovered, including other people's portfolios which are shared to you. To exclude specific portfolios, add their IDs to `exclude_portfolios` in the `preferences.conf`: diff --git a/lib/catalog.py b/lib/catalog.py index e85cd5be..4a138d23 100644 --- a/lib/catalog.py +++ b/lib/catalog.py @@ -14,6 +14,7 @@ MORNING = '08:30' # by Sydney's clock: the weekly and longer reports, and the messages TUE_SAT = ['tue', 'wed', 'thu', 'fri', 'sat'] # the mornings after a US session FLAGS = {'AU': '🇦🇺'} # what {flag} becomes in a message for one country +SHARESIGHT = ('trades', 'performance', 'holdings', 'pie', 'who') # the actions that report on Sharesight PERIODIC = ('performance', 'price') # the reports whose period follows how often they run PERIOD_DAYS = {'daily': 1, 'weekdays': 1, 'weekly': 7, 'monthly': 30, 'quarterly': 90, 'bi-annual': 180, 'annual': 365} MONEY_CAFE = ('https://www.omnycontent.com/d/playlist/b034de0e-930d-434b-9822-a7140060b2c0/' @@ -131,9 +132,11 @@ def message_name(lines, country=None, size=56): ) -def items(country): - """The default jobs where the country_code preference is country: every item but those for another country.""" - return [item for item in ITEMS if item.country in (None, country)] +def items(country, sharesight=True): + """The default jobs where the country_code preference is country: every item but those for another country, and + where Sharesight is not set up (sharesight false), those that report on it.""" + return [item for item in ITEMS if item.country in (None, country) and (sharesight or item.action not in + SHARESIGHT)] def same_clock(a, b): diff --git a/lib/jobs.py b/lib/jobs.py index 748b212b..a2f84936 100644 --- a/lib/jobs.py +++ b/lib/jobs.py @@ -23,6 +23,7 @@ from lib import config from lib import delivery from lib import log +from lib import sharesight from lib import state from lib import when from lib.reports import cal, cleanup, message, milestone, performance, podcasts, price, rating, shorts, trades @@ -316,7 +317,7 @@ def load(self): counts from now, as one does that was off.""" jobs = system_jobs() chats = self.stored_chats() - items = catalog.items(config.prefs.country_code) + items = catalog.items(config.prefs.country_code, sharesight.configured()) for chat, record in chats.items(): gone = set(removed(record)) jobs += [make_job(item.action, item.args, item.schedule, chat, item_on(record, item.key), item.key, @@ -527,6 +528,8 @@ def add(self, chat, action, args, schedule): number in .jobs. Raises ScheduleError for a job that cannot be, a one-off gone by, one the chat has already, or one more than MAX_JOBS.""" job = make_job(action, args, schedule, chat) + if job.action in catalog.SHARESIGHT and not sharesight.configured(): + raise ScheduleError(f'{job.action} reports on Sharesight, which is not set up') if schedule.frequency == 'once' and schedule.next_after(self.clock()) is None: raise ScheduleError(f"{when.days_words(schedule.days)} has gone by: a one-off is for a time ahead") if item := catalog.matching(job.action, job.args, schedule, config.prefs.country_code): diff --git a/lib/reports/about.py b/lib/reports/about.py index 5dec75e7..a96bbb5f 100644 --- a/lib/reports/about.py +++ b/lib/reports/about.py @@ -8,6 +8,7 @@ from lib import config from lib import formats from lib import jobs +from lib import sharesight from lib import when HEADINGS = {'top': 'Jobs here', 'on': 'Tap a job to enable it', 'off': 'Tap a job to disable it', @@ -205,9 +206,9 @@ def choices(heading, offer, row, lines=(), usual=None, nav=None): tapped.append(back) return text, [tapped[i:i + 5] for i in range(0, len(tapped), 5)] return text, [tapped[i:i + row] for i in range(0, len(tapped), row)] + [nav or [back, cancel]] - if not steps: # a button per report, by its name alone, and Cancel - return choices('Add a job: which report?', [(label, number) for number, (label, _, _) in enumerate(REPORTS, 1)], - 3, nav=[cancel]) + if not steps: # a button per report, by its name alone (none that reports on Sharesight without it), and Cancel + return choices('Add a job: which report?', [(label, number) for number, (label, action, _) in + enumerate(REPORTS, 1) if sharesight.configured() or action not in catalog.SHARESIGHT], 3, nav=[cancel]) label, action, args = REPORTS[choice(steps[0], 1, len(REPORTS)) - 1] if action == 'message': return [service.bold('Add a message'), "A message's text is what you type:", diff --git a/lib/reports/sharesight_warm.py b/lib/reports/sharesight_warm.py index 6a8b4628..eae344aa 100644 --- a/lib/reports/sharesight_warm.py +++ b/lib/reports/sharesight_warm.py @@ -9,7 +9,7 @@ def run(): every 30 minutes, and a flush at once. Nothing without Sharesight configured, or with the cache preference off (nothing would be kept). A failure is only logged: the caches it would have replaced are served while fresh, and the next run tries again. Returns how many portfolios it refreshed.""" - if not (config.prefs.cache and config.env.sharesight_auth.get('client_id')): + if not (config.prefs.cache and sharesight.configured()): return 0 try: return len(sharesight.warm()) diff --git a/lib/reports/yahoo_warm.py b/lib/reports/yahoo_warm.py index 94793686..c6d9ccb1 100644 --- a/lib/reports/yahoo_warm.py +++ b/lib/reports/yahoo_warm.py @@ -1,7 +1,7 @@ """Keeps the price history of every stock followed fresh in the cache (scheduled: yahoo_warm), so that a period list or a chart never waits on Yahoo. Sharesight's caches have their own, sharesight_warm.""" from lib import log -from lib import config, watchlist, yahoo +from lib import config, sharesight, watchlist, yahoo def run(): """Fetch afresh the price history of every stock followed (every chat's watchlist, and the holdings where @@ -11,7 +11,7 @@ def run(): if not config.prefs.cache: return 0 tickers = watchlist.all_watched() - if config.env.sharesight_auth.get('client_id'): + if sharesight.configured(): try: tickers = watchlist.all_followed() except Exception as e: # Sharesight unreachable, say: the watchlists' at least diff --git a/lib/sharesight.py b/lib/sharesight.py index 4a4b829e..761135d2 100644 --- a/lib/sharesight.py +++ b/lib/sharesight.py @@ -106,6 +106,18 @@ def _trades(portfolio_id, days=None): return cache.cached(f"finbot_sharesight_trades_{portfolio_id}_{days}.json", 299, fetch)['trades'] +class NotSetUp(answer.Answer, RuntimeError): + """Sharesight is not set up (configured): what needs it says so, as an answer.""" + def __init__(self): + """The answer: what is missing, and where it goes.""" + super().__init__('Sharesight is not set up: its keys go in finbot.env (doc/configuration.md)') + + +def configured(): + """Whether Sharesight is set up, its keys in finbot.env: without, a chat's reports cover its watchlist alone.""" + return bool(config.env.sharesight_auth.get('client_id')) + + class PortfolioNotFound(answer.Answer, LookupError): """No portfolio is called what was asked for: the answer names it, and the portfolios there are.""" def __init__(self, name, names): @@ -148,8 +160,10 @@ def _chosen(listed): def portfolios(select=None, seconds=None): """The portfolios to report on (after include_portfolios and exclude_portfolios), or the one called `select`, in any - case; the list is cached for `seconds` (cache_seconds when None). Raises RuntimeError when there are none, and - PortfolioNotFound for a name that is none of them.""" + case; the list is cached for `seconds` (cache_seconds when None). Raises NotSetUp without Sharesight's keys, + RuntimeError when there are none, and PortfolioNotFound for a name that is none of them.""" + if not configured(): + raise NotSetUp() found = _chosen(_portfolios(seconds)) if not found: raise RuntimeError("No Sharesight portfolios found") @@ -162,7 +176,9 @@ def portfolios(select=None, seconds=None): def tickers(): - """Every portfolio's holdings as Yahoo tickers, sorted.""" + """Every portfolio's holdings as Yahoo tickers, sorted; none where Sharesight is not set up.""" + if not configured(): + return [] return sorted({ticker for portfolio in portfolios() for ticker in portfolio.tickers()}) diff --git a/tests/common.py b/tests/common.py index e4860e2a..9f0e8444 100644 --- a/tests/common.py +++ b/tests/common.py @@ -132,7 +132,8 @@ def setUp(self): tmp = tempfile.TemporaryDirectory() self.addCleanup(tmp.cleanup) self.env(data_dir=tmp.name, cache_dir=tmp.name, telegramBotToken='X', - telegram_api='https://api.telegram.org/botX/') + telegram_api='https://api.telegram.org/botX/', sharesight_auth={**config.env.sharesight_auth, + 'client_id': 'test'}) # Sharesight set up: its answers are the fixtures below self.patch(bot, '_updates_seen', collections.OrderedDict()) # each test starts with no update taken self.patch(bot, '_said', {}) # nor any message self.patch(bot, 'doubled', lambda message: False) # a test may send the same message twice on purpose diff --git a/tests/test_reports.py b/tests/test_reports.py index 50ccb930..b608e318 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -775,5 +775,27 @@ def test_with_the_graph_preference_off_a_chart_goes_as_text(self): self.assertEqual(self.sent[-1], ('text', ['x'])) +class WithoutSharesight(FinbotCase): + """Without Sharesight's keys, a chat's reports cover its watchlist alone, and what needs Sharesight says it is not + set up.""" + + def setUp(self): + """No Sharesight keys in finbot.env.""" + super().setUp() + self.env(sharesight_auth={**config.env.sharesight_auth, 'client_id': None}) + self.patch(sharesight, '_portfolios', lambda seconds=None: self.fail('Sharesight was asked')) + + def test_the_watchlist_alone_and_never_a_request(self): + """The stocks a chat's reports cover are its watchlist alone, and Sharesight is never asked.""" + self.assertEqual((watchlist._followed(['T9', 'T1']), sharesight.tickers()), (['T1', 'T9'], [])) + + def test_what_needs_sharesight_says_it_is_not_set_up(self): + """.holdings, .trades, .performance, .who and .pie say Sharesight is not set up, and where its keys go.""" + for typed in ('.holdings Rob', '.holdings', '.trades', '.performance', '.who T1', '.pie Rob'): + with self.subTest(typed=typed): + self.assertEqual(self.command(typed), [('text', ['Sharesight is not set up: its keys go in finbot.env' + ' (doc/configuration.md)'])]) + + if __name__ == '__main__': unittest.main() diff --git a/tests/test_schedule.py b/tests/test_schedule.py index 2f0a67c0..986fe97d 100644 --- a/tests/test_schedule.py +++ b/tests/test_schedule.py @@ -142,6 +142,13 @@ def test_every_default_job_can_be(self): if item.action in catalog.PERIODIC and all(arg.isdigit() for arg in item.args): self.assertEqual(list(item.args), catalog.period_args(item.action, item.schedule)) + def test_without_sharesight_no_default_job_reports_on_it(self): + """Without Sharesight, the default jobs leave out those that report on it: trades and performance.""" + names = {item.name for item in catalog.items('AU', sharesight=False)} + self.assertFalse(names & {'trades', 'performance'}) + self.assertTrue({'trades', 'performance', 'price'} <= {item.name for item in catalog.items('AU')}) + self.assertIn('price', names) + def test_the_default_jobs_as_a_melbourne_chat_sees_them(self): """The default jobs by name, each with its schedule in words, as a chat in Melbourne sees them in mid-October: each in its market's zone, the local time beside it where the clocks differ.""" diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index 848047d3..148aa456 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -527,6 +527,16 @@ def test_add_list_and_remove(self): self.assertEqual(self.say('.job del 2'), ['Removed 2. shorts · daily at 10:00pm']) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [], 'deleted': ['rating']}) + def test_without_sharesight_no_job_reports_on_it(self): + """Without Sharesight's keys, a chat cannot add a job that reports on it, and Add does not offer one.""" + self.env(sharesight_auth={'client_id': None}) + self.assertEqual(self.say('.jobs add trades every 20 mins weekdays from 10am to 4pm'), ['trades reports on' + ' Sharesight, which is not set up']) + labels = [button.label for row in about.add_screen(TELEGRAM, CHAT, [])[1] for button in row] + self.assertNotIn('trades', labels) + self.assertNotIn('performance', labels) + self.assertIn('price', labels) + def test_add_reads_english_and_the_period_follows_how_often(self): """A job is a report, what it takes, and a schedule in English, by the timezone preference's clock unless it names a zone; performance and price cover the days between runs where they name no period.""" From aa37efb1a58aad07c70bdd529d0fcfafef4e87de Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 10:41:56 +0000 Subject: [PATCH 17/41] Jobs run the commands of .help: pe, forward pe, peg, marketcap, beta, buy & sell, history, compare... A job may run any command of .help (worker.COMMANDS, the table .help is made from) but those that change something (watchlist, plan, jobs; Command.job false) and those a scheduled report already runs (price, performance, trades, shorts, earnings/dividend, session/premarket). That leaves holdings, pie, who, marketcap, stock, beta, compare, history, buy/sell, pe, forward pe and peg; a new command is a job too unless it says otherwise. - worker.CommandJob stands in for a report module: its usage is what it takes, job() reads its words (none where it takes none, at least one SYMBOL or PORTFOLIO where it needs one), and run() answers in the job's chat as the command would, through a Request aimed at that chat (Request takes a dest). Nothing is evaluated: the job names a command of the table, its arguments stay words, and a stored job naming anything else is still left out. - jobs.actions() is the reports and the commands; .jobs add takes a command by its name, two-word ones too (forward pe), the words before the schedule (".jobs add pe top weekly", ".jobs add compare AAPL MSFT 1y weekly at 9am"); another of its names (fpe, a typo) stands for it, but buy, sell or both say which. - Add lists the commands after the reports (buy & sell as one); one that needs a stock or a portfolio says how to type it. On Discord the list comes on two pages. - .jobs add alone lists what each takes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/jobs.py | 41 +++++++++++++++------ lib/reports/about.py | 52 ++++++++++++++++++++++---- lib/worker.py | 81 ++++++++++++++++++++++++++++++++++++----- tests/test_reports.py | 17 +++++++++ tests/test_schedule.py | 43 +++++++++++++++++++--- tests/test_scheduler.py | 49 +++++++++++++++++++++---- 6 files changed, 242 insertions(+), 41 deletions(-) diff --git a/lib/jobs.py b/lib/jobs.py index a2f84936..9161bd53 100644 --- a/lib/jobs.py +++ b/lib/jobs.py @@ -82,8 +82,8 @@ def key(self): @property def module(self): - """The report module that runs it.""" - return ACTIONS[self.action] + """What runs it: a report module, or a command of .help (worker.CommandJob).""" + return actions()[self.action] def stored(self): """The job as JOBS_FILE keeps one a chat added.""" @@ -136,20 +136,37 @@ def warm_now(scheduler=None): log.info('cache: warm jobs done') +def actions(): + """Every action a job may name: the report modules (ACTIONS), and the commands of .help that a job may run + (command_actions), each standing in for a report module.""" + return {**command_actions(), **ACTIONS} + + +def command_actions(): + """The commands of .help that a job may run, by each name a job may call one by (worker.job_commands).""" + from lib import worker # here, not at the top: worker imports this module + return worker.job_commands() + + def chat_actions(): - """The actions a chat may schedule, by name.""" - return sorted(name for name in ACTIONS if name not in SYSTEM_ACTIONS) + """The actions a chat may schedule, by name: the reports, then the commands of .help a job may run, as .help names + them.""" + return sorted(name for name in ACTIONS if name not in SYSTEM_ACTIONS) + [name for name, command in + command_actions().items() if command.listed] def make_job(action, args, schedule, chat='', on=True, item=None, name=None): - """A Job of an action in ACTIONS with the arguments its parser takes (words), on a when.Schedule: the system's - actions only for a system job (no chat). Raises ScheduleError, saying what is wrong.""" - if not isinstance(action, str) or action not in ACTIONS or (chat and action in SYSTEM_ACTIONS): + """A Job of an action (actions()) with the arguments its parser takes (words), on a when.Schedule: the system's + actions only for a system job (no chat); a command by the name a job calls it (its name in .help). Raises + ScheduleError, saying what is wrong.""" + known = actions() + if not isinstance(action, str) or action not in known or (chat and action in SYSTEM_ACTIONS): raise ScheduleError(f"no report {action!r}: send .jobs add for the reports") if not isinstance(args, (list, tuple)) or not all(isinstance(arg, str) for arg in args): raise ScheduleError(f'{action} takes words, not {args!r}') + action = getattr(known[action], 'name', action) # a command's other name (a typo too) stands for its first try: - label, kwargs = ACTIONS[action].job(list(args)) + label, kwargs = known[action].job(list(args)) except ValueError as e: raise ScheduleError(f'{action} takes {e}') if not isinstance(schedule, when.Schedule): @@ -269,9 +286,11 @@ def parse_add(text, zone): raise ScheduleError('a message is a schedule, then its text in double quotes: .jobs add message every 31' ' mar "🥳 Happy EOFY"') return 'message', rest[start + 1:end].replace('\r', '').split('\n'), when.parse(rest[:start], zone, MESSAGE_AT) - if name not in ACTIONS or name in SYSTEM_ACTIONS: + known, words = actions(), text.split() + name = next((' '.join(words[:size]).lower() for size in (2, 1) if ' '.join(words[:size]).lower() in known), name) + if name not in known or name in SYSTEM_ACTIONS: raise ScheduleError(f"no report {name!r}: send .jobs add for the reports") - words, wrong, unread = rest.split(), None, None + words, wrong, unread = words[len(name.split()):], None, None for cut in range(len(words) + 1): try: schedule = when.parse(' '.join(words[cut:]), zone) @@ -280,7 +299,7 @@ def parse_add(text, zone): continue args = words[:cut] or catalog.period_args(name, schedule) try: - ACTIONS[name].job(args) + known[name].job(args) except ValueError as e: wrong = ScheduleError(f'{name} takes {e}') continue diff --git a/lib/reports/about.py b/lib/reports/about.py index a96bbb5f..b6e96c43 100644 --- a/lib/reports/about.py +++ b/lib/reports/about.py @@ -2,6 +2,7 @@ disable, remove or run one; the steps of Add; the upcoming runs; the system jobs; and the reports a job can run.""" import calendar import datetime +import re from lib import buttons from lib import catalog @@ -48,6 +49,7 @@ INTERVALS = (5, 10, 15, 20, 30, 120, 180, 240, 360, 720) # minutes between a repeat's runs, as every... offers them MINUTES = tuple(range(0, 60, 5)) # the minutes Add offers MOST_DAYS = {'discord': 20} # days of the month on a page: Discord takes 25 buttons, elsewhere all 31 +MOST_REPORTS = {'discord': 16} # Add's reports on a page, four to a row: Discord takes five rows, elsewhere all YEARS = 10 # the years a one-off may be in, this one first CROWDED = 20 # a screen of more choices takes them in rows of five, Back last: Discord takes 25 buttons @@ -206,15 +208,31 @@ def choices(heading, offer, row, lines=(), usual=None, nav=None): tapped.append(back) return text, [tapped[i:i + 5] for i in range(0, len(tapped), 5)] return text, [tapped[i:i + row] for i in range(0, len(tapped), row)] + [nav or [back, cancel]] + reports = add_reports() if not steps: # a button per report, by its name alone (none that reports on Sharesight without it), and Cancel - return choices('Add a job: which report?', [(label, number) for number, (label, action, _) in - enumerate(REPORTS, 1) if sharesight.configured() or action not in catalog.SHARESIGHT], 3, nav=[cancel]) - label, action, args = REPORTS[choice(steps[0], 1, len(REPORTS)) - 1] + offer = [(label, number) for number, (label, action, _) in enumerate(reports, 1) + if sharesight.configured() or action not in catalog.SHARESIGHT] + most = MOST_REPORTS.get(service, len(offer)) + if page * most >= len(offer): + raise ValueError('no such page') + nav = [cancel] + if page: + nav.insert(0, buttons.menu_button('◀', 'add', [], crc)) + elif len(offer) > most: + nav.insert(0, buttons.menu_button('More ▶', 'add', [MORE], crc)) + return choices('Add a job: which report?', offer[page * most:(page + 1) * most], 4 if service in + MOST_REPORTS else 3, nav=nav) + label, action, args = reports[choice(steps[0], 1, len(reports)) - 1] if action == 'message': return [service.bold('Add a message'), "A message's text is what you type:", '.jobs add message SCHEDULE "TEXT"', '', service.bold('For example:'), '.jobs add message every 31 mar at 8:30am "🥳 Happy EOFY"', '.jobs add message on 28 oct 2026 "ubank draw 6/6"'], [[back, cancel]] + command = jobs.command_actions().get(action) + if command and command.needs: # what it is about, a stock or a portfolio, comes from typing, not a button + return [service.bold(f'Add {label}'), f'What {label} is about is typed:', + f'.jobs add {action} {command.ARGUMENTS} SCHEDULE', '', service.bold('For example:'), + f'.jobs add {action} {example(command.ARGUMENTS)} weekly'], [[back, cancel]] if len(steps) == 1: return choices(f'Add {label}: how often?', [(name, code) for code, name in FREQUENCIES], 3, usual=USUAL['frequency']) @@ -254,6 +272,24 @@ def choices(heading, offer, row, lines=(), usual=None, nav=None): 'start minute': ('start minute?', 4, []), 'end': ('end time?', 4, [])}[level] return choices(f'Add {label}, {said}: {question}', offer, row, lines, usual) +def add_reports(): + """What Add offers a job to run: the reports (REPORTS), then the commands of .help a job may run, in its order (buy + and sell as one), each (label, action, arguments; None where the period follows how often it runs). Each one's + place in this list is its button's data, whichever are offered.""" + commands = [] + for name, command in jobs.command_actions().items(): + if command.listed and name != 'sell': + name = 'buy&sell' if name == 'buy' else name + commands.append((name.replace('&', ' & '), name, ())) + return REPORTS + tuple(commands) + +def example(arguments): + """What a command's arguments may be, for an example: its words outside brackets, each SYMBOL a ticker in turn and + PORTFOLIO a portfolio's name (the first that Sharesight's cache has, else Growth).""" + tickers, names = iter(('AAPL', 'MSFT', 'GOOG')), sharesight.cached()[0] + ['Growth'] + return ' '.join(next(tickers) if word == 'SYMBOL' else names[0] if word == 'PORTFOLIO' else word + for word in re.sub(r'\[[^\]]*\]', '', arguments).split()) + def levels_of(frequency, made): """The choices Add asks for after how often (LEVELS), as far as those made say: every... asks a weekday's hour and minute once its interval is a weekday. Raises ValueError for a frequency that is none.""" @@ -384,12 +420,14 @@ def prepare_system(service): (' (running)' if job.key in running else '') for job in scheduler.system_jobs()] def prepare_reports(service): - """`.jobs add` alone: the reports a job can run, each with what it takes, and examples.""" - return [service.bold('Reports a job can run:')] + [f"{name} {jobs.ACTIONS[name].ARGUMENTS}".strip() + """`.jobs add` alone: what a job can run (the reports, then the commands of .help), each with what it takes, and + examples.""" + known = jobs.actions() + return [service.bold('What a job can run:')] + [f"{name} {known[name].ARGUMENTS}".strip() for name in jobs.chat_actions()] + ['', service.bold('For example:'), '.jobs add price weekly', '.jobs add price close weekdays at 4:30pm New York', '.jobs add performance first day of each month at 9am', - '.jobs add trades every 20 mins weekdays from 10am to 4pm', - '.jobs add message every 31 mar at 8:30am "🥳 Happy EOFY"', '', + '.jobs add trades every 20 mins weekdays from 10am to 4pm', '.jobs add pe top weekly', + '.jobs add compare AAPL MSFT 1y monthly', '.jobs add message every 31 mar at 8:30am "🥳 Happy EOFY"', '', f"Times are by {when.place(formats.zone().key)}'s clock unless one names a zone (UTC, New York, EST);" " performance and price cover the days between runs"] diff --git a/lib/worker.py b/lib/worker.py index d90662d9..f74b996d 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -1,6 +1,7 @@ """Chat commands: a message is answered by the Command in COMMANDS whose name it starts with (read), or .SYMBOL. The table also makes the .help text and the Telegram command keyboard, which is kept current in every DM.""" import dataclasses +import functools import hashlib import hmac import json @@ -25,7 +26,8 @@ @dataclasses.dataclass class Request: - """A message being answered: where it came from (service is its dialect) and who sent it.""" + """A message being answered: where it came from (service is its dialect) and who sent it; or a job running a + command (CommandJob), its chat's transport given as dest.""" service: str chat_id: str user: str # the sender's handle: whose watchlist add or plan it is @@ -33,10 +35,12 @@ class Request: bot_name: str message_id: str = None # the message being answered interaction: str = '' # what the network answers through, where it has one (Discord's slash command) + dest: object = None # the chat's transport, where it is given def __post_init__(self): - """The chat's transport, which also keeps the typing notice going.""" - self.dest = delivery.reply_to(self.service, self.chat_id, self.message_id, self.interaction) + """The chat's transport, which also keeps the typing notice going, unless one is given.""" + if self.dest is None: + self.dest = delivery.reply_to(self.service, self.chat_id, self.message_id, self.interaction) def reply(self, lines, chart=None, buttons=None): """Answer in the chat (see delivery.deliver): the lines, perhaps a chart (drawn, or a spec drawn only where it @@ -511,6 +515,7 @@ class Command: percent: bool = False period: bool = False keywords: dict = dataclasses.field(default_factory=dict) + job: bool = True # whether a job may run it (job_commands): not one that changes something SECTIONS = TRACKED, COMPANY, PRICE, RISK, TRADES, JOBS = ('Tracked securities', 'Company info', 'Price', 'Risk & Value', 'Trades', 'Scheduled jobs') # the headings of .help, in order @@ -531,11 +536,11 @@ class Command: COMMANDS = [ # .help lists a section's commands in this order. The odd spellings (earrings, prince, hospital...) are # kept on purpose. - Command(('help', 'usage', 'start'), on_help), # /start is what Telegram sends when a chat opens + Command(('help', 'usage', 'start'), on_help, job=False), # /start is what Telegram sends when a chat opens Command(('holdings', 'holding'), on_holdings, ('.holdings PORTFOLIO',), TRACKED), Command(('pie',), on_pie, ('.pie PORTFOLIO [sector]',), TRACKED, keywords=BY), Command(('watchlist', 'wishlist'), on_watchlist, ('.watchlist [add|del SYMBOL]',), TRACKED, ('.watchlist',), - keywords=CHANGE), + keywords=CHANGE, job=False), Command(('who',), on_who, ('.who SYMBOL',), TRACKED), Command(('earnings', 'earning', 'earrings', 'earring', 'dividends', 'dividend', 'exdividend', 'ex-dividend'), on_calendar, ('.dividend [period|SYMBOL]', '.earnings [period|SYMBOL]'), COMPANY, ('.dividend', '.earnings'), @@ -559,10 +564,10 @@ class Command: Command(('peg',), on_ratio('peg'), ('.peg [SYMBOL|top|bottom|negative]',), RISK, ('.peg',), keywords=NEGATIVE), Command(('shorts', 'short'), on_shorts, ('.shorts [percent|SYMBOL]',), RISK, ('.shorts',), percent=True), Command(('trades', 'trade'), on_trades, ('.trades [period] [portfolio]',), TRADES, ('.trades',), period=True), - Command(('plan',), on_plan, ('.plan [your plan]',), TRADES), + Command(('plan',), on_plan, ('.plan [your plan]',), TRADES, job=False), Command(('jobs', 'job', 'cron', 'crons', 'schedule'), on_jobs, ('.jobs [upcoming|system]', '.jobs enable|disable|remove|run [NUMBER]', '.jobs add REPORT SCHEDULE', '.jobs add message SCHEDULE "TEXT"'), - JOBS, ('.schedule',), keywords=JOB_CHANGES), + JOBS, ('.schedule',), keywords=JOB_CHANGES, job=False), PROFILE, ] @@ -629,6 +634,64 @@ def described(name, lines, most=100): text = longer return text +REPORTED = (on_price, on_performance, on_trades, on_shorts, on_calendar, on_session) # what a scheduled report runs + +@dataclasses.dataclass(frozen=True) +class CommandJob: + """A command of .help as a job's action (job_commands), standing in for a report module: ARGUMENTS is what it takes, + from its usage; job() reads a scheduled run's words; run() answers in the job's chat as the command would.""" + command: Command + name: str # what the job calls it, its name in .help, but buy, sell or both: on_rating reads which from the name + ARGUMENTS: str = '' + listed: bool = True # whether .jobs add lists it: not a command's other names, nor its typos + + @property + def needs(self): + """How many words it needs, each SYMBOL or PORTFOLIO of its usage outside brackets: none comes from a button.""" + return len(re.findall(r'\b[A-Z]{2,}\b', re.sub(r'\[[^\]]*\]', '', self.ARGUMENTS))) + + def job(self, argv): + """A scheduled run's arguments (words): (label, run's keyword arguments). Raises ValueError, with what it takes, + for words where its usage takes none, or fewer than it needs.""" + args = read_args(self.command, self.name, ' '.join(argv)) + if (argv and not self.ARGUMENTS) or len(args.words) < self.needs: + raise ValueError(self.ARGUMENTS or 'no arguments') + return ' '.join([self.name.replace('&', ' & '), *argv]), {'words': list(argv)} + + def run(self, dest, *, words): + """Answer in the job's chat (dest) as the command would to these words, typed there: its usage where it cannot + take them, an answer as an answer; any other failure is the job's to report.""" + request = Request(dest.dialect, dest.where, '', '', '', dest=dest) + try: + self.command.handler(request, read_args(self.command, self.name, ' '.join(words))) + except Usage as e: + request.reply([f"Usage: {line}" for line in self.command.usage] + list(e.lines)) + except answer.Answer as e: + request.reply(e.lines) + +@functools.cache +def job_commands(): + """The commands of .help a job may run, by each name a job may call one by: a CommandJob each. Every one but those + that change something (job false) and those a scheduled report runs already (REPORTED): first each by its name in + .help, with what it takes; then its other names (typos too), each standing for the first, but buy and sell's.""" + found = {} + for command in COMMANDS: + if not (command.job and command.usage) or command.handler in REPORTED: + continue + listed = [] + for line in command.usage: + words = line.lstrip('.').split() + name = next((' '.join(words[:size]).lower() for size in (2, 1) if ' '.join(words[:size]).lower() in + command.names), None) + if name: + found[name] = CommandJob(command, name, ' '.join(words[len(name.split()):])) + listed.append(name) + first = found[listed[0]] + for name in command.names: + found.setdefault(name, dataclasses.replace(first, name=name if command.handler is on_rating else first.name, + listed=False)) + return found + PERIOD_WORDS = ('1d', '7d', '1m', '3m', '6m', 'ytd', '1y', '5y', '10y') # suggested where a command takes a period JOB_WORDS = ('add', 'remove', 'enable', 'disable', 'run', 'upcoming', 'system') # what /jobs suggests first SCHEDULE_WORDS = ('daily', 'weekdays', 'weekly', 'monthly', 'quarterly', 'bi-annual', 'annual', 'hourly', 'every', 'at', @@ -699,8 +762,8 @@ def job_words(chat, words): if action != 'add': return [] if len(words) == 1: - return [(name, f'{name} {jobs.ACTIONS[name].ARGUMENTS}'.strip()) for name in jobs.chat_actions()] - module = jobs.ACTIONS.get(words[1].lower()) + return [(name, f'{name} {jobs.actions()[name].ARGUMENTS}'.strip()) for name in jobs.chat_actions()] + module = jobs.actions().get(words[1].lower()) if module is None or words[1].lower() not in jobs.chat_actions(): return [] takes = re.findall(r'[a-z][a-z-]*', module.ARGUMENTS) if len(words) == 2 else [] diff --git a/tests/test_reports.py b/tests/test_reports.py index b608e318..4e9a9256 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -775,6 +775,23 @@ def test_with_the_graph_preference_off_a_chart_goes_as_text(self): self.assertEqual(self.sent[-1], ('text', ['x'])) +class CommandJobs(FinbotCase): + """A job that runs a command of .help posts in its chat what the command posts when typed there.""" + + def test_a_command_job_posts_what_the_typed_command_posts(self): + """pe, beta, buy & sell and compare, run as a job's, post what each posts when typed in the chat; one that + cannot take its words posts its usage.""" + chat = delivery.chat('telegram:55', scheduled=True) + for name, words, typed in (('pe', [], '.pe'), ('beta', [], '.beta'), ('buy&sell', [], '.buy & sell'), + ('compare', ['T1', 'T2'], '.compare T1 T2'), ('history', [], '.history')): + with self.subTest(name=name): + posted = self.command(typed) + self.sent.clear() + worker.job_commands()[name].run(chat, words=words) + self.assertEqual(self.sent, posted) + self.assertEqual(posted, [('text', ['Usage: .history SYMBOL'])]) + + class WithoutSharesight(FinbotCase): """Without Sharesight's keys, a chat's reports cover its watchlist alone, and what needs Sharesight says it is not set up.""" diff --git a/tests/test_schedule.py b/tests/test_schedule.py index 986fe97d..ae275d31 100644 --- a/tests/test_schedule.py +++ b/tests/test_schedule.py @@ -13,15 +13,44 @@ class Actions(unittest.TestCase): - def test_a_job_runs_only_a_report_of_finbots_own(self): - """Each action a job may run is a report module of finbot's own, with its run and its parser; a chat may - schedule all but the system's.""" + def test_a_job_runs_only_a_report_or_a_command_of_finbots_own(self): + """Each action a job may run is a report module of finbot's own, with its run and its parser, or a command of + .help that changes nothing, by one of its names; a chat may schedule all but the system's, the commands listed + by their names in .help.""" for name, module in jobs.ACTIONS.items(): with self.subTest(name=name): self.assertEqual(module.__name__, f'lib.reports.{name}') self.assertTrue(callable(module.run) and callable(module.job)) + commands = jobs.command_actions() + for name, command in commands.items(): + with self.subTest(name=name): + self.assertIn(command.command, worker.COMMANDS) + self.assertTrue(command.command.job and name in command.command.names) + self.assertNotIn(command.command.handler, worker.REPORTED) + self.assertFalse({'help', 'start', 'watchlist', 'wishlist', 'plan', 'jobs', 'cron', 'schedule', 'price', + 'session', 'earnings', 'trades'} & set(commands)) self.assertEqual(jobs.chat_actions(), ['cal', 'message', 'milestone', 'performance', 'podcasts', 'price', - 'rating', 'shorts', 'trades']) + 'rating', 'shorts', 'trades', 'holdings', 'pie', 'who', 'marketcap', 'beta', 'compare', 'history', 'buy', + 'sell', 'pe', 'forward pe', 'peg', 'stock']) + + def test_a_command_of_help_as_a_job(self): + """A job may run a command of .help by its name, the words it takes before the schedule; another of its names + (a typo too) stands for it, but buy and sell's, which say which; one that takes no words refuses them, and one + that needs a stock or a portfolio refuses none.""" + weekly = when.parse('weekly', 'UTC') + for text, action, args in (('pe top weekly', 'pe', ['top']), ('forward pe weekly', 'forward pe', []), + ('compare AAPL MSFT 1y weekly at 9am', 'compare', ['AAPL', 'MSFT', '1y']), + ('buy&sell weekly', 'buy&sell', []), ('holdings Rob weekly', 'holdings', ['Rob'])): + with self.subTest(text=text): + self.assertEqual(jobs.parse_add(text, 'UTC')[:2], (action, args)) + for text in ('beta x weekly', 'history weekly', 'compare AAPL weekly', 'holdings weekly'): + with self.subTest(text=text), self.assertRaises(jobs.ScheduleError): + jobs.parse_add(text, 'UTC') + for action, args, made, name in (('hospital', ['AAPL'], 'history', 'history AAPL'), ('fpe', [], 'forward pe', + 'forward pe'), ('buy&sell', [], 'buy&sell', 'buy & sell'), ('sell', [], 'sell', 'sell')): + with self.subTest(action=action): + job = jobs.make_job(action, args, weekly, 'telegram:1') + self.assertEqual((job.action, job.name, job.kwargs), (made, name, {'words': args})) def test_anything_else_is_refused(self): """A name that is no action (a shell command, a module's path, a builtin), a system job for a chat, arguments @@ -29,11 +58,13 @@ def test_anything_else_is_refused(self): daily = when.parse('daily', 'UTC') for action, args in (('os.system', ['ls']), ('price; rm -rf /', []), ('lib.reports.price', ['close']), ('__import__', ['os']), ('eval', ['1']), ('yahoo_warm', []), ('price', ['bogus']), ('trades', [7]), - (['trades'], []), ('price', 'close')): + (['trades'], []), ('price', 'close'), ('jobs', []), ('plan', ['x']), ('watchlist', ['add', 'X']), + ('help', []), ('on_jobs', [])): with self.subTest(action=action, args=args), self.assertRaises(jobs.ScheduleError): jobs.make_job(action, args, daily, 'telegram:1') for text in ('os.system daily', 'price; rm -rf / daily', '../../etc/passwd daily', 'trades daily; rm -rf /', - 'message daily', 'trades.py daily'): + 'message daily', 'trades.py daily', 'jobs add price daily daily', 'plan buy low daily', + 'watchlist add X daily'): with self.subTest(text=text), self.assertRaises(jobs.ScheduleError): jobs.parse_add(text, 'UTC') self.assertEqual(jobs.make_job('yahoo_warm', [], daily).label, 'yahoo_warm') # the system's own diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index 148aa456..4152766c 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -527,6 +527,16 @@ def test_add_list_and_remove(self): self.assertEqual(self.say('.job del 2'), ['Removed 2. shorts · daily at 10:00pm']) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [], 'deleted': ['rating']}) + def test_a_command_of_help_as_a_job(self): + """.jobs add takes a command of .help, with its words; the job is listed by its name and words, and runs as the + command does, posting what it posts; a command that changes something is no job.""" + self.assertEqual(self.say('.jobs add pe top weekly'), ['4. ✅* pe top · weekly, Saturdays at 9:00am', + 'Added: .jobs lists them all']) + self.sent.clear() + self.assertEqual(self.say('.jobs run 4'), ['4. pe top: done']) + self.assertTrue(self.sent or self.replies[:-1]) # what .pe top posts + self.assertIn("no report 'plan'", self.say('.jobs add plan buy low weekly')[0]) + def test_without_sharesight_no_job_reports_on_it(self): """Without Sharesight's keys, a chat cannot add a job that reports on it, and Add does not offer one.""" self.env(sharesight_auth={'client_id': None}) @@ -570,11 +580,13 @@ def test_a_message_takes_its_text_in_quotes(self): self.say('.jobs add message daily "no end')[0]) def test_add_alone_lists_the_reports_a_job_can_run(self): - """.jobs add alone lists the reports with what each takes, message among them, no system job, and examples.""" + """.jobs add alone lists the reports and the commands with what each takes, message among them, no system job, + and examples.""" lines = self.say('.jobs add') - self.assertEqual(lines[0], 'Reports a job can run:') + self.assertEqual(lines[0], 'What a job can run:') for line in ('cal earnings|ex-dividend', 'message "TEXT"', 'performance [DAYS]', 'podcasts FEED_URL', - 'price midsession|interday|premarket|close|DAYS', 'trades', '.jobs add price weekly'): + 'price midsession|interday|premarket|close|DAYS', 'trades', '.jobs add price weekly', 'beta', + 'pe [SYMBOL|top|bottom]', 'compare SYMBOL SYMBOL [... up to 6] [period]', '.jobs add pe top weekly'): self.assertIn(line, lines) self.assertFalse(any(name in line for line in lines for name in jobs.SYSTEM_ACTIONS)) @@ -840,7 +852,8 @@ def test_add_by_buttons_back_and_cancel(self): self.assertEqual(lines, ['Add a job: which report?']) self.assertEqual(rows, [['trades', 'price midsession', 'price premarket'], ['price close', 'price', 'performance'], ['cal earnings', 'cal ex-dividend', 'shorts'], ['rating', 'milestone', 'message'], - ['Cancel']]) + ['holdings', 'pie', 'who'], ['marketcap', 'beta', 'compare'], ['history', 'buy & sell', 'pe'], + ['forward pe', 'peg', 'stock'], ['Cancel']]) self.assertEqual((data['price'], data['Cancel']), (f'm|add|5|{self.crc()}', f'm|top|0|{self.crc()}')) lines, rows, data = self.press(data['price']) self.assertEqual((lines, rows), (['Add price: how often?', '', '† indicates default'], @@ -951,12 +964,31 @@ def test_a_one_off_by_year_month_day_hour_and_minute(self): self.assertIn('.jobs add message SCHEDULE "TEXT"', lines) self.assertEqual(rows, [['Back', 'Cancel']]) + def test_the_commands_of_help_by_buttons(self): + """Add offers the commands of .help after the reports: one that takes no stock adds as a report does; one that + needs a stock or a portfolio says how to type it. On Discord the list comes on two pages.""" + lines, rows, data = self.press(f'm|add|{self.crc()}') + for pressed in ('pe', 'weekly', 'Fri', '09†', '09:00†'): + lines, rows, data = self.press(data[pressed]) + self.assertEqual(lines, ['Added: 4. ✅* pe · weekly, Fridays at 9:00am']) + self.assertEqual(self.s.chat_jobs(CHAT)[-1].action, 'pe') + lines, rows, data = self.press(f'm|add|{self.crc()}') + lines, rows, _ = self.press(data['history']) + self.assertEqual((lines, rows), (['Add history', 'What history is about is typed:', + '.jobs add history SYMBOL SCHEDULE', '', 'For example:', '.jobs add history AAPL weekly'], + [['Back', 'Cancel']])) + first, second = (about.add_screen(DISCORD, CHAT, steps)[1] for steps in ([], ['p1'])) + self.assertEqual(([len(row) for row in first], [button.label for button in first[-1]]), ([4, 4, 4, 4, 2], + ['More ▶', 'Cancel'])) + self.assertEqual([button.label for row in second for button in row], ['beta', 'compare', 'history', + 'buy & sell', 'pe', 'forward pe', 'peg', 'stock', '◀', 'Cancel']) + def test_a_step_that_is_none_adds_nothing(self): """Data that is no step of Add (forged, or an older finbot's: the weekdays frequency, the next 14 days, every... hourly) adds nothing and says so: a report, frequency, year, month or day not offered, a day gone by, 30 Feb, an end not among the runs, a page where there is none.""" crc = self.crc() - for data in ('13', '5|x', '5|k', '5|o|261020', '5|r|60', '5|w|7|9', '5|d|24', '5|d|9|9', '5|d|9|0|0', + for data in ('25', '5|x', '5|k', '5|o|261020', '5|r|60', '5|w|7|9', '5|d|24', '5|d|9|9', '5|d|9|0|0', '1|r|7', '5|o|2025', '5|o|2036', '5|o|2026|9', '5|o|2026|10|13', '5|o|2027|2|30', '5|a|2|30', '5|q|13', '5|w|p1', '5|m|def', '5|h|d|10|5|600', '5|h|d|10|5|1000', '5|h|d|10|5|e|0'): with self.subTest(data=data): @@ -985,9 +1017,10 @@ def fits(service, lines, markup): for page in range(6): with self.subTest(service=service, screen=screen, page=page): fits(service, *about.jobs_screen(service, CHAT, screen, page)) - for steps in ([], ['5'], ['5', 'w'], ['5', 'w', '5'], ['5', 'w', '5', '14'], ['5', 'm'], ['5', 'm', 'p1'], - ['5', 'q'], ['5', 'q', '2'], ['5', 'b', '3'], ['5', 'a'], ['5', 'a', '2'], ['5', 'a', '2', '29'], - ['5', 'o'], ['5', 'o', '2026'], ['5', 'o', '2026', '10'], ['5', 'o', '2027', '1'], + for steps in ([], ['p1'], ['19'], ['5'], ['5', 'w'], ['5', 'w', '5'], ['5', 'w', '5', '14'], ['5', 'm'], + ['5', 'm', 'p1'], ['5', 'q'], ['5', 'q', '2'], ['5', 'b', '3'], ['5', 'a'], ['5', 'a', '2'], + ['5', 'a', '2', '29'], ['5', 'o'], ['5', 'o', '2026'], ['5', 'o', '2026', '10'], + ['5', 'o', '2027', '1'], ['5', 'o', '2027', '1', 'p1'], ['5', 'o', '2027', '1', '31'], ['5', 'h'], ['5', 'h', 'd'], ['5', 'h', 'd', '0'], ['5', 'h', 'd', '0', '0'], ['5', 'r'], ['5', 'r', 'k'], ['5', 'r', 'k', '9'], ['5', 'r', '20'], ['5', 'r', '20', 'k', '10'], ['5', 'r', '20', 'k', '10', '5'], ['12']): From 2e2ed55b739079e07f0a33502d97645ff86afa6e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 10:46:12 +0000 Subject: [PATCH 18/41] schedule log: group chats and DMs counted apart "schedule: 85 jobs of 1 group chat and 1 DM, and 3 system jobs". A chat's record marks a direct message ('dm': True) from when finbot first knows it; a one-off upgrade (dm flag) marks the DMs it knew already, Telegram's and Slack's by key, Discord's as Discord says, switching nothing. A DM's record kept 'defaults': 'off' only until All on, and All off set it in a group too, so it could not tell. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/jobs.py | 15 +++++++----- lib/migrate.py | 21 ++++++++++++++--- tests/test_schedule.py | 52 +++++++++++++++++++++++++++++++----------- 3 files changed, 66 insertions(+), 22 deletions(-) diff --git a/lib/jobs.py b/lib/jobs.py index 9161bd53..ca233c40 100644 --- a/lib/jobs.py +++ b/lib/jobs.py @@ -42,8 +42,8 @@ ('cleanup', ('30',), {'days': ['sat'], 'at': '18:51'})) WARM = ('sharesight_warm', 'yahoo_warm') # the jobs that fill the cache, run at once after a flush too # {chat key: {'jobs': [{action, args, schedule, on}], 'off': [catalog keys], 'deleted': [catalog keys]}}: the jobs it -# added, the default jobs it switched off and those it removed; a chat whose default jobs start off (a direct message) -# holds 'defaults': 'off' and 'on' instead of 'off', the catalog keys switched on +# added, the default jobs it switched off and those it removed; a chat whose default jobs start off holds 'defaults': +# 'off' and 'on' instead of 'off', the catalog keys switched on; a direct message holds 'dm': True JOBS_FILE = 'finbot_chat_jobs.json' OLD_FILE = 'finbot_schedule.json' # each chat's cron jobs, from before the catalog: read for its chats alone STATE_FILE = 'finbot_jobs.json' # each job's last run, by key: a restart neither repeats a run nor loses one @@ -263,11 +263,11 @@ def is_dm(key): def know(keys, dm=None): """Make chats known (by key), each new one with none added and the default jobs switched on, or off in a direct - message: where dm is True (else as its key tells, is_dm). How many were new.""" + message (marked 'dm'): where dm is True (else as its key tells, is_dm). How many were new.""" with state.edit(JOBS_FILE, {}) as chats: new = [key for key in dict.fromkeys(keys) if key not in chats] for key in new: - chats[key] = {'defaults': 'off'} if (is_dm(key) if dm is None else dm) else {} + chats[key] = {'defaults': 'off', 'dm': True} if (is_dm(key) if dm is None else dm) else {} return len(new) @@ -368,8 +368,11 @@ def load(self): self.jobs, self.chats = jobs, set(chats) if problem: delivery.report_error(ScheduleError(problem), context='schedule') - log.info(f"schedule: {len(jobs) - len(SYSTEM)} jobs of {len(chats)} chat{'' if len(chats) == 1 else 's'}, and" - f" {len(SYSTEM)} system jobs") + dms = sum(1 for key, record in chats.items() if isinstance(record, dict) and record.get('dm') is True or + is_dm(key)) + log.info(f"schedule: {len(jobs) - len(SYSTEM)} jobs of {len(chats) - dms} group" + f" chat{'' if len(chats) - dms == 1 else 's'} and {dms} DM{'' if dms == 1 else 's'}, and {len(SYSTEM)}" + " system jobs") def stored_chats(self): """The chats JOBS_FILE knows, each with its record. At the first load, those the old OLD_FILE knew join them diff --git a/lib/migrate.py b/lib/migrate.py index e8741cad..3ef1f4b6 100644 --- a/lib/migrate.py +++ b/lib/migrate.py @@ -36,10 +36,11 @@ def old_homes(): def run(): - """Switch off the jobs of the direct message chats finbot knows, once (dm_jobs_off). Make the old home chats' own - what was kept for all chats, where that has not been done yet; without any (a new install, or the retired keys - deleted), there is nowhere to put it, so it waits.""" + """Switch off the jobs of the direct message chats finbot knows, and mark them as such, once each (dm_jobs_off, + mark_dms). Make the old home chats' own what was kept for all chats, where that has not been done yet; without any + (a new install, or the retired keys deleted), there is nowhere to put it, so it waits.""" once('dm jobs off', dm_jobs_off) + once('dm flag', mark_dms) homes = old_homes() if not homes: return @@ -95,6 +96,20 @@ def dm_jobs_off(): f" ({', '.join(dms)}), as a direct message now starts with its jobs off") +def mark_dms(): + """Mark each direct message chat finbot knows as one ('dm' in its record), for the schedule's count of group chats + and DMs: Telegram's and Slack's as their keys tell (jobs.is_dm), Discord's as Discord says. Nothing is switched.""" + chats = state.load(jobs.JOBS_FILE, {}) + keys = [key for key in (chats if isinstance(chats, dict) else {}) if isinstance(key, str)] + dms = [key for key in keys if jobs.is_dm(key)] + if config.env.discordBotToken: + dms += [key for key in keys if key.startswith('discord:') and discord_dm(key.partition(':')[2])] + with state.edit(jobs.JOBS_FILE, {}) as stored: + for key in dms: + if isinstance(stored.get(key), dict): + stored[key]['dm'] = True + + def discord_dm(channel): """Whether a Discord channel (by id) is a direct message with the bot, as Discord says; one it says nothing of (a channel the bot has left, an id that is none) is not.""" diff --git a/tests/test_schedule.py b/tests/test_schedule.py index ae275d31..8117c27f 100644 --- a/tests/test_schedule.py +++ b/tests/test_schedule.py @@ -288,7 +288,7 @@ def test_each_home_chat_starts_with_the_default_jobs(self): state.save(jobs.JOBS_FILE, {'slack:C1': {'off': ['trades']}}) self.assertIn('schedule: 1 old home chat start with the default jobs', self.migrate()) self.assertEqual(state.load(jobs.JOBS_FILE), {'slack:C1': {'off': ['trades']}, - 'telegram:1': {'defaults': 'off'}}) # a person's id: a direct message, whose default jobs start off + 'telegram:1': {'defaults': 'off', 'dm': True}}) # a person's id: a DM, whose default jobs start off self.assertNotIn('schedule:', self.migrate()) def test_with_no_home_chat_it_waits(self): @@ -334,18 +334,19 @@ def test_a_direct_message_starts_with_its_default_jobs_off(self): them all start on.""" jobs.know(['telegram:5', 'telegram:-100', 'slack:D1', 'slack:C1', 'discord:7']) jobs.know(['discord:8'], dm=True) - self.assertEqual(state.load(jobs.JOBS_FILE), {'telegram:5': {'defaults': 'off'}, 'telegram:-100': {}, - 'slack:D1': {'defaults': 'off'}, 'slack:C1': {}, 'discord:7': {}, 'discord:8': {'defaults': 'off'}}) + dm = {'defaults': 'off', 'dm': True} + self.assertEqual(state.load(jobs.JOBS_FILE), {'telegram:5': dm, 'telegram:-100': {}, 'slack:D1': dm, + 'slack:C1': {}, 'discord:7': {}, 'discord:8': dm}) s = jobs.Scheduler(clock=lambda: OCTOBER) with contextlib.redirect_stderr(io.StringIO()): s.load() self.assertFalse(any(job.on for job in s.chat_jobs('telegram:5'))) self.assertTrue(all(job.on for job in s.chat_jobs('telegram:-100'))) s.toggle(s.chat_jobs('telegram:5')[0]) - self.assertEqual(state.load(jobs.JOBS_FILE)['telegram:5'], {'defaults': 'off', 'on': ['trades']}) + self.assertEqual(state.load(jobs.JOBS_FILE)['telegram:5'], {**dm, 'on': ['trades']}) self.assertEqual([job.on for job in s.chat_jobs('telegram:5')][:2], [True, False]) s.switch_all('telegram:5', True) - self.assertEqual(state.load(jobs.JOBS_FILE)['telegram:5'], {}) + self.assertEqual(state.load(jobs.JOBS_FILE)['telegram:5'], {'dm': True}) # still a DM def test_a_chat_seen_says_whether_it_is_a_direct_message(self): """At a chat's first command it starts with its default jobs off where it is a direct message: Discord's update @@ -358,8 +359,8 @@ def test_a_chat_seen_says_whether_it_is_a_direct_message(self): for chat, dm in (('88', True), ('77', False)): worker.process_request(inbound.Message('discord', chat, '<@5>', '.help', '@99', 'rob', dm=dm)) worker.process_request(inbound.Message('telegram', '5', '@u', '.help', BOT, 'U')) - self.assertEqual(state.load(jobs.JOBS_FILE), {'discord:88': {'defaults': 'off'}, 'discord:77': {}, - 'telegram:5': {'defaults': 'off'}}) + self.assertEqual(state.load(jobs.JOBS_FILE), {'discord:88': {'defaults': 'off', 'dm': True}, 'discord:77': {}, + 'telegram:5': {'defaults': 'off', 'dm': True}}) def test_the_direct_messages_jobs_are_switched_off_once(self): """As finbot first starts with this version, every job of each direct message chat it knows is switched off, @@ -381,17 +382,18 @@ def call(method, path, **kwargs): with contextlib.redirect_stderr(io.StringIO()) as log: migrate.run() stored = state.load(jobs.JOBS_FILE) - self.assertEqual(stored['telegram:5'], {'defaults': 'off', 'jobs': [{**trades, 'on': False}]}) + self.assertEqual(stored['telegram:5'], {'defaults': 'off', 'dm': True, 'jobs': [{**trades, 'on': False}]}) for key in ('slack:D1', 'discord:8', 'telegram:6', 'discord:9'): - self.assertEqual(stored[key], {'defaults': 'off'}, key) + self.assertEqual(stored[key], {'defaults': 'off', 'dm': True}, key) self.assertEqual((stored['telegram:-100'], stored['discord:7']), ({'jobs': [trades]}, {})) self.assertNotIn('telegram:-6', stored) # an old chat that is no DM is known at the scheduler's first load - self.assertEqual(sorted(asked), ['/channels/7', '/channels/8', '/channels/9']) + self.assertEqual(sorted(asked), ['/channels/7', '/channels/7', '/channels/8', '/channels/8', '/channels/9', + '/channels/9']) # once by each upgrade, at the one start self.assertIn('every job switched off in 5 direct message chats', log.getvalue()) - self.assertIn('dm jobs off', state.load(migrate.UPGRADES)) + self.assertTrue({'dm jobs off', 'dm flag'} <= set(state.load(migrate.UPGRADES))) state.save(jobs.JOBS_FILE, {**stored, 'telegram:5': {}}) # switched on again migrate.run() - self.assertEqual((state.load(jobs.JOBS_FILE)['telegram:5'], len(asked)), ({}, 3)) + self.assertEqual((state.load(jobs.JOBS_FILE)['telegram:5'], len(asked)), ({}, 6)) def test_it_waits_for_discord_and_a_channel_discord_will_not_name_is_no_dm(self): """If Discord cannot be reached, nothing is switched and the upgrade waits for the next start; a channel Discord @@ -413,7 +415,31 @@ def refused(method, path, **kwargs): self.patch(discord_common, 'call', refused) with contextlib.redirect_stderr(io.StringIO()): migrate.run() - self.assertEqual(state.load(jobs.JOBS_FILE), {'discord:8': {}, 'telegram:5': {'defaults': 'off'}}) + self.assertEqual(state.load(jobs.JOBS_FILE), {'discord:8': {}, 'telegram:5': {'defaults': 'off', 'dm': True}}) + + def test_the_direct_messages_known_are_marked_once(self): + """Where the jobs of the DMs were switched off at an earlier start, the DMs finbot knows are marked as such + once, switching nothing: Telegram's and Slack's as their keys tell, Discord's as Discord says.""" + state.save(migrate.UPGRADES, {'dm jobs off': '2026-10-16T00:00:00+00:00'}) + state.save(jobs.JOBS_FILE, {'telegram:5': {'on': ['rating']}, 'telegram:-100': {}, 'slack:D1': {}, + 'discord:8': {}, 'discord:7': {}}) + self.env(discordBotToken='x') + self.patch(discord_common, 'call', lambda method, path, **kwargs: {'type': 1 if path == '/channels/8' else 0}) + with contextlib.redirect_stderr(io.StringIO()): + migrate.run() + self.assertEqual(state.load(jobs.JOBS_FILE), {'telegram:5': {'on': ['rating'], 'dm': True}, + 'telegram:-100': {}, 'slack:D1': {'dm': True}, 'discord:8': {'dm': True}, 'discord:7': {}}) + self.assertIn('dm flag', state.load(migrate.UPGRADES)) + + def test_the_schedule_counts_group_chats_and_dms(self): + """The schedule's line in the log at each load counts the chats' jobs, the group chats and the DMs apart (a + DM by its mark, or its key), and the system's jobs.""" + self.patch(catalog, 'ITEMS', ()) + self.patch(jobs, 'SYSTEM', ()) + state.save(jobs.JOBS_FILE, {'telegram:-100': {}, 'discord:8': {'dm': True}, 'slack:C1': {}}) + with contextlib.redirect_stderr(io.StringIO()) as log: + jobs.Scheduler(clock=lambda: OCTOBER).load() + self.assertIn('schedule: 0 jobs of 2 group chats and 1 DM, and 0 system jobs', log.getvalue()) if __name__ == '__main__': From ecfc7bb55229d0f4bb19e33968ce34d50e347caf Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 10:48:25 +0000 Subject: [PATCH 19/41] Docs: the .jobs menu filled out, removable defaults, command jobs, no Sharesight needed - doc/scheduling.md: the list with how Add is typed under it; the six buttons (Add, Remove, Enable, Disable, Run now, Upcoming...) with Remove all, Enable all and Disable all; Add step by step, a table of every frequency and its levels down to the minute; typed enable/disable/remove; quarterly and bi-annual from a start month; the commands of /help a job can run; a removed default job comes back when added again; trades and performance only with Sharesight; the DM mark. - doc/commands.md: the /jobs lines. - CLAUDE.md: the code map (job_commands and CommandJob, actions(), 'deleted' and 'dm', delete_all and switch, when.cycle, mark_dms, sharesight.configured and NotSetUp) and a new command's job=False. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 30 +++++++++------ doc/commands.md | 2 +- doc/scheduling.md | 97 ++++++++++++++++++++++++++++++----------------- 3 files changed, 81 insertions(+), 48 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 9b332bda..e4301fe8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -58,7 +58,8 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm (`Message`, `Press`, `Suggest`, `Response`; the `Network` and `Webhook` bases), `lib/pull.py` (the `Listener` base, which waits longer after each failure in a row: `backoff`), `lib/transports.py` (the `Transport` base, below). - `lib/worker.py`: `COMMANDS`, a table of `Command(names, handler, usage, section, buttons, percent, period, - keywords)`, which also makes `.help`, the Telegram command keyboard, kept current in DMs, and `slash_commands()`, which + keywords, job)`, which also makes `.help`, the commands a job may run (`job_commands()`, each a `CommandJob`: every + one but those that change something, `job=False`, and those a report runs, `REPORTED`), the Telegram command keyboard, kept current in DMs, and `slash_commands()`, which every network gets (Telegram's menu, Slack's manifest from `util/slack_manifest.py`, Discord's registered commands); `suggest(chat, text)` completes a slash command's arguments as they are typed (Discord's autocomplete), from the cache alone. `read()` finds the @@ -97,7 +98,9 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm deletes cache files by when they were written (`--before`/`--after TIME`, or `all`), never the state. `yahoo.symbol(typed)` is Yahoo's symbol for a ticker as written, tried on the local exchange (`exchange_suffix`) when Yahoo does not know it bare: every ticker a chat gives goes through it. `sharesight.portfolios(name=None)` gives - `Portfolio` objects: `holdings()`, `tickers()`, `gain(days)` (the portfolio's), `trades(days)`. + `Portfolio` objects: `holdings()`, `tickers()`, `gain(days)` (the portfolio's), `trades(days)`. Sharesight is + optional (`sharesight.configured()`): without its keys `tickers()` is empty, so reports cover the watchlist alone, and + `portfolios()` raises `NotSetUp`, an answer. - `lib/log.py`: the log, by level (`error warn info debug`), as the `log_level` preference lets in, and `cut()`, which keeps a line that can run long (a request's) within the `log_width` preference; bot.py's `WholeLines` writes each line whole, whichever threads log at once. `lib/cache.py`: cache files (any may vanish at any time). `lib/state.py`: state files in `DATA_DIR`, which must last; @@ -107,22 +110,24 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm `with_holdings(chat)`, the stocks a report in that chat covers; a report takes the chat from its destination. - `lib/config.py`: `config.env` (finbot.env, fixed at start) and `config.prefs` (preferences.conf, a new object on each reload). `lib/jobs.py`: the scheduler: each chat's jobs, the default ones and those it added (`JOBS_FILE`, by - chat key: the defaults it switched off, the jobs it added; a direct message's defaults start off, `'defaults': - 'off'` with those switched on; `see` makes a chat known at its first command, a DM as `inbound.Message.dm` or - `is_dm` says; `add`, - `delete`, `toggle`, `switch_all`, `run`, `upcoming`, which `.jobs` calls through `jobs.current`; `parse_add` reads `.jobs add`), - and the fixed `SYSTEM` jobs, its shared state under one lock. A job is an action of `ACTIONS` (the report modules, - `message` among them) with the arguments its `job()` takes, never a command; one stored that is anything else is - logged and left out. Each job runs in the bot's process, in a thread (`run_in_process`), reported once if still + chat key: the defaults it switched off and those it removed, `'deleted'`, the jobs it added; a direct message's + defaults start off, `'defaults': 'off'` with those switched on, and it is marked `'dm'`; `see` makes a chat known at + its first command, a DM as `inbound.Message.dm` or `is_dm` says; `add`, `delete`, `delete_all`, `switch`, + `toggle`, `switch_all`, `run`, `upcoming`, which `.jobs` calls through `jobs.current`; `parse_add` reads `.jobs add`), + and the fixed `SYSTEM` jobs, its shared state under one lock. A job is an action of `actions()`: a report module of + `ACTIONS` (`message` among them) or a command of `.help` (`command_actions()`), with the arguments its `job()` takes + as words, never a command line; one stored that is anything else is logged and left out. Each job runs in the bot's process, in a thread (`run_in_process`), reported once if still running after `JOB_TIMEOUT` and left to finish (every request goes through `net`, with its timeouts: a test checks); each job's next run is worked out once and again only after it runs, so a tick costs little. `lib/when.py`: a job's `Schedule` (days, a time or a repeat, or a span across days, in a zone): `parse()` of the English a chat types, - `describe()`, `next_after()` through daylight saving. `lib/catalog.py`: the default jobs (`ITEMS`), each with its + `describe()`, `next_after()` through daylight saving; `cycle()` the months of a quarterly or bi-annual one from its + start month. `lib/catalog.py`: the default jobs (`ITEMS`), each with its name and schedule; the period of performance and price follows how often it runs (`period_args`). `lib/migrate.py`: once, on an upgrade, the old home chats (the retired telegramChatID, slackChannelID and discordChannelID, which it alone reads, from the environment) become chats finbot knows, and the watchlist, the plans and the reports' shared records become theirs; one-off upgrades run at one start only (`once`, recorded in - `UPGRADES`): `dm_jobs_off` switched off the jobs of the DMs known before DMs started with them off. + `UPGRADES`): `dm_jobs_off` switched off the jobs of the DMs known before DMs started with them off, and `mark_dms` + marked them `'dm'`, for the log's count of group chats and DMs. - `tests/common.py`: `FinbotCase` (offline fixtures). `self.command('.price')` returns what was sent: `('text', lines)` or `('photo', size, caption)`. `wsgi_environ(path, body, **headers)` is a request for `bot.main`. `ProviderCase` runs the real parsers on the answers saved under `doc/` (`sample`), as `Resp` and `serve` return @@ -162,7 +167,8 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm - A chart renderer in `lib/render.py` takes its `Chart` and is listed in `RENDERERS` by kind; it gets `@_drawn`, which draws one chart at a time (matplotlib's caches are shared by every thread). - A new command: a `Command` in `COMMANDS` (its names, and what its arguments may hold), whose usage, section and - buttons make its `.help` lines and keyboard buttons, and the command list of `doc/commands.md`. + buttons make its `.help` lines and keyboard buttons, and the command list of `doc/commands.md`. A job may run every + command of `.help`, so one that changes anything takes `job=False`. ## Saving tokens - Run the tests quietly: `.venv/bin/python -m unittest discover -s tests -b` prints only failures (about 0.4 KB when diff --git a/doc/commands.md b/doc/commands.md index ebb2cd16..43bc146b 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -41,7 +41,7 @@ Where: /history symbol /holdings portfolio /jobs [upcoming|system] -/jobs run|toggle|del [number] +/jobs enable|disable|remove|run [number] /jobs add report schedule /jobs add message schedule "text" /marketcap [symbol|top|bottom] diff --git a/doc/scheduling.md b/doc/scheduling.md index c05eb7cf..1156d644 100644 --- a/doc/scheduling.md +++ b/doc/scheduling.md @@ -9,55 +9,71 @@ chats of an earlier version's schedule and its home chats. A direct message is a channel on Slack, and on Discord a message from no server. ## The menu -`/jobs` lists the chat's jobs, numbered, a page at a time: the default jobs, then those added here. Each line shows ✅ -(on) or ❌ (off), a `*` if the job has never run, its name with a `†` if it is a default job, and its schedule in -words. The page ends with a key to the marks it shows: +`/jobs` lists the chat's jobs, numbered, a page at a time. Each line shows ✅ (on) or ❌ (off), a `*` if the job has +never run, its name and its schedule in words. Under the list, in italics, is how Add is typed: ``` -1. ✅ trades† · every 20 mins, Mon 9am – Sat 3pm Sydney -3. ✅* price midsession US† · weekdays at 12:45pm New York (3:45am local, Tue–Sat) +1. ✅ trades · every 20 mins, Mon 9am – Sat 3pm Sydney +3. ✅* price midsession US · weekdays at 12:45pm New York (3:45am local, Tue–Sat) -* not run yet, † default (no delete, only disable) +* not run yet + +/jobs add REPORT SCHEDULE +/jobs add message SCHEDULE "TEXT" ``` -Under it are buttons, each screen drawn in place of the last, with **Cancel** back to the list: -* **Enable/disable**: a button per job, its number, switch and name (`3 ✅ price midsession US`); a press switches the - job on or off. **All on** switches every job of the chat on at once, or **All off**, shown when every job is on, - switches them all off. -* **Delete**: a button per job added here. A default job has no button: switch it off instead. -* **Add**: the report, by name, and how often, then, a screen each, the rest down to the minute: - * one-off: the date (the next 14 days), the hour, the minute; - * daily and weekdays: the hour, the minute; - * weekly: the day of the week, the hour, the minute; - * monthly, quarterly and bi-annual: the day of the month, the hour, the minute; - * annual: the month, the day, the hour, the minute; - * repeating: the interval, then every day or weekdays. - - Each screen marks its default with `†` (`Sat†`, `09†`, `09:00†`) and says `† indicates default`. After the report's, - each screen also has **Back**, a step back (the hour's has only Back: Discord takes 25 buttons). A message's text - cannot come from a button, so its screen says how to type one. +Under it are six buttons, two to a row. Each screen is drawn in place of the last, with **Cancel** back to the list: +* **Add**: what the job runs, how often, then the rest down to the minute ([below](#add-step-by-step)). +* **Remove**: a button per job, the default ones too. **Remove all** asks first ("Remove all 37 jobs here?"), then + removes every job. A default job that is removed comes back when the same job is added again. +* **Enable**: a button per job that is off; **Enable all** switches every job on. +* **Disable**: a button per job that is on; **Disable all** switches every job off. * **Run now**: a button per job; a press runs it now, as its schedule would, and says how it went. -* **Upcoming**: the next runs, in time order. +* **Upcoming...**: the next runs, in time order. A button drawn for a list that has since changed acts on nothing: the screen is drawn again as the list is now. +### Add, step by step +Each screen marks its default with `†` (`Sat†`, `09†`) and says _† indicates default_; after the first, **Back** takes a +step back. Times are by the `timezone` preference's clock. + +First what the job runs: a report, or a command of `/help` ([below](#a-job-in-english)). A message's text, and a +command's stock or portfolio, cannot come from a button, so their screen says how to type them. Then how often: + +| how often | then | added as, for example | +|---|---|---| +| one-off | the year (this one and the nine after), the month, the day (from today; tomorrow by default), the hour, the minute | once, 20 Mar 2027 at 8:00am | +| hourly | every day or weekdays, the start hour, the start minute, the end time (a run after the start, or the end of the day) | hourly at :05, weekdays 10am–4pm | +| daily | the hour, the minute | daily at 9:00am | +| weekly | the day of the week, the hour, the minute | weekly, Saturdays at 9:00am | +| monthly | the day of the month, the hour, the minute | monthly, the 1st at 9:00am | +| quarterly | the start month (then every 3 months), the day of the month, the hour, the minute | quarterly, 15 Feb/May/Aug/Nov at 9:00am | +| bi-annual | the start month (and 6 months later), the day of the month, the hour, the minute | bi-annual, 1 Mar and 1 Sep at 9:00am | +| annual | the start month, the day of the month (those it has), the hour, the minute | annual, 31 Mar at 9:00am | +| every... | 5, 10, 15, 20 or 30 mins, or 2, 3, 4, 6 or 12 hours, then as hourly; or weekday, then the hour and the minute | every 20 mins, weekdays 10:05am–4:05pm; weekdays at 9:00am | + +A span across days (`every 20 mins from mon 9am to sat 3pm`), or a window of hours across midnight, is typed. + Or type: ``` /jobs the chat's jobs, with the buttons above /jobs upcoming the next 20 runs; a repeat's runs on one day are one entry /jobs system the system jobs -/jobs add the reports a job can run, what each takes, and examples +/jobs add what a job can run, what each takes, and examples /jobs add price weekly /jobs add price close weekdays at 4:30pm New York +/jobs add pe top weekly +/jobs add compare AAPL MSFT 1y monthly /jobs add message every 31 mar at 8:30am "🥳 Happy EOFY" /jobs run 2 runs job 2 now; /jobs run alone: the list with a button per job -/jobs toggle 2 switches job 2 on or off -/jobs del 36 deletes job 36, one added here +/jobs enable 2 switches job 2 on; /jobs disable 2 off; /jobs toggle 2 the other way +/jobs remove 36 removes job 36 (or /jobs del 36) ``` ## A job in English A job is a report and what it takes, then when it runs: * **Days:** `daily`, `weekdays`, `weekends`, `weekly` (Saturdays), `every saturday` or `saturdays`, `tue-sat`, `monthly` - (the 1st), `first day of each month`, `on the 15th`, `quarterly` (1 Jan, Apr, Jul and Oct), `bi-annual` (1 Jan and 1 - Jul), `annual` (1 Jan), `every 31 mar` (each year) or `on 28 oct 2026` (once). + (the 1st), `first day of each month`, `on the 15th`, `quarterly` (1 Jan, Apr, Jul and Oct; `quarterly from feb`: Feb, + May, Aug and Nov), `bi-annual` (1 Jan and 1 Jul; `bi-annual starting mar`: Mar and Sep), `annual` (1 Jan; + `annual from aug`), `every 31 mar` (each year) or `on 28 oct 2026` (once). * **Time:** `at 9`, `at 9am`, `at 9:30am` or `at 21:30`. Without one, 9am, or 8:30am for a message. * **Repeating:** `every 20 mins`, `hourly`, `hourly at :01`, `every 2 hours` or any other number, counted from midnight. It can take days, and a window of hours (`from 10am to 9pm`, `between 9am and 5pm`) or a span across days @@ -75,6 +91,12 @@ A job is a report and what it takes, then when it runs: The reports are `trades`, `price midsession|interday|premarket|close|DAYS`, `cal earnings|ex-dividend`, `performance [DAYS]`, `rating`, `milestone`, `shorts`, `podcasts FEED_URL` and `message "TEXT"`. +A job can also run a command of `/help`, with what it takes, and posts what the command posts when typed in the chat: +`pe [SYMBOL|top|bottom]`, `forward pe`, `peg`, `marketcap`, `beta`, `buy`, `sell` (`buy&sell` both), `history SYMBOL`, +`compare SYMBOL SYMBOL…`, `stock SYMBOL`, and with Sharesight `holdings PORTFOLIO`, `pie PORTFOLIO` and `who SYMBOL`. +The commands that change something (`watchlist`, `plan`, `jobs`) are no jobs, and price, performance, trades, shorts, +earnings, dividends, session and premarket are run by their reports above. + The period of `performance` and `price` follows how often the job runs: weekly covers 7 days, monthly 30, quarterly 90, bi-annual 180 and annual 365, and a daily `price` is the day's moves. A number given (`price 30 weekly`) is used instead. @@ -86,7 +108,7 @@ A **message** posts its text as it is: * A one-off message (`on 28 oct 2026`) is deleted after its time has passed, when the list is next shown. A job that is the same as a default job (the same report, days and times, in a zone with the same clock) switches that -default job on instead of being added. A chat can add 30 jobs. +default job on instead of being added, and brings it back where the chat removed it. A chat can add 30 jobs. ## The default jobs These are as a chat in Melbourne sees them in mid-October. The local times beside New York's, Hong Kong's and London's @@ -135,18 +157,22 @@ move as either side's daylight saving changes. * mid-session about halfway through: 1:00pm Sydney for the ASX, 12:45pm New York for the NYSE; * the close 30 minutes after it: 4:30pm Sydney for the ASX, 4:30pm Hong Kong, 5:00pm London and 4:30pm New York. * **The messages marked (AU)** are default jobs only where the `country_code` preference is `AU`. +* **trades and performance** are default jobs only where Sharesight is set up (see + [Configuration](configuration.md#sharesight)). * **A one-off leaves the list** once its date has passed. ## How jobs run * A job is kept by its chat's network and id, in `finbot_chat_jobs.json` in the data directory, so switching a network - between pull and HTTP mode keeps it. Only a chat's switches and the jobs it added are stored: the default jobs come - from finbot's code, so a change to them reaches every chat. + between pull and HTTP mode keeps it. Only a chat's switches, the default jobs it removed and the jobs it added are + stored: the default jobs come from finbot's code, so a change to them reaches every chat, and a new one arrives in + every chat. * Each chat keeps its own record of what a report has told it (trades, rating changes, record highs, podcast episodes), so the same report scheduled in two chats tells each of them once. A run from **Run now** or `/jobs run` is a scheduled run: it posts to the chat and updates the same record. A change counts at once, with no reload. -* **A job can only run finbot's own reports.** It names an action from a fixed table in `lib/jobs.py` (a report, or - posting a message), with arguments that report's own parser has checked: never a command, a script or code. A stored - job naming anything else is logged and left out, never run. +* **A job can only run finbot's own reports and commands.** It names a report from a fixed table in `lib/jobs.py` (or + posting a message), or a command of the table `/help` is made from (`lib/worker.py`) that changes nothing, with + arguments its parser has checked, kept as words: never a command line, a script or code. A stored job naming anything + else is logged and left out, never run. * **The system jobs**, fixed in `lib/jobs.py` and listed by `/jobs system`, keep the cache full and tidy: * `sharesight_warm` every 30 minutes and `yahoo_warm` every 8 hours (see [Caching](caching.md)); * `cleanup 30` weekly, which deletes cache files older than 30 days. @@ -169,5 +195,6 @@ move as either side's daylight saving changes. * The cron jobs in `finbot_schedule.json` are not carried over. Its chats start with the default jobs, and the file can be deleted. * At the first start of this version, every job of each direct message chat finbot knew was switched off, once - (Discord's direct messages as Discord says). It is recorded in `finbot_upgrades.json`, so it never happens again. + (Discord's direct messages as Discord says), and those chats are marked as direct messages, for the log's count of + group chats and DMs at each start. Each is recorded in `finbot_upgrades.json`, so it never happens again. * `reminder.json` is read no more: its reminders are default jobs now. Delete the data directory's copy. From d07cdf73b7774a3193810ddf5b741b84e67b1ca2 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 11:20:11 +0000 Subject: [PATCH 20/41] Charts attach again to long reports: Telegram's caption counted as Telegram counts it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Scheduled price reports (close, midsession) went out as text without their chart. Telegram counts a caption's 1,024 characters in UTF-16 code units, after its HTML is read: each 🔼 or 🔻 counts 2. finbot counted code points, so a list of about thirty moves, cut to 1,012 of them, came to about 1,042 by Telegram's count. Telegram refused the chart and the report went as text. The shorter names (30 characters) fit more lines, and so more emoji, into a caption, which is why it showed up now. - dialects.Telegram.visible() counts as Telegram does: the text the HTML shows (no tags, & one character), in UTF-16 code units. - deliver(): a chart the network refuses goes once more with its heading (the first line) alone as its caption and its buttons, and the report follows as text. If that is refused too, the report goes as text alone, as before. A chart that timed out (it may have arrived) is not sent again. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 5 +-- doc/slack.md | 3 +- lib/delivery.py | 14 +++++++-- lib/dialects.py | 5 +-- lib/transports.py | 4 +-- tests/test_sending.py | 71 +++++++++++++++++++++++++++++++++++++++++-- 6 files changed, 89 insertions(+), 13 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index e4301fe8..8ac62966 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -76,8 +76,9 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm Chat-only (`prepare_*`): `about` (.help, and the .jobs menu's screens: `jobs_screen`, `add_screen`, `next_screen`), `history` (.history .compare), `holdings` (.holdings .who .pie .watchlist), `profile` (.SYMBOL), `rankings` (.marketcap .pe .fpe .peg .beta .buy .sell). - `lib/delivery.py`: `Report(lines, chart, buttons)`, the destinations (the chat that asked, or a chat by its key with - `chat(key)`, as its network's bot), `deliver(dest, report)`, the one-line error report (to its chat; one that belongs - to no chat only logged). + `chat(key)`, as its network's bot), `deliver(dest, report)` (a chart refused goes again with its heading alone as the + caption, the lines as text after it), the one-line error report (to its chat; one that belongs to no chat only + logged). A caption is cut to fit as the network counts it (`Dialect.visible`: Telegram's in UTF-16 code units). `lib/transports.py`: `Transport`, the base of each network's bot, every one taking charts and buttons: splitting, captions, charts, buttons, edits, and the typing notice (`start_typing`: after half a second; Slack's ⏳ on the message), which `text()` and `chart()` stop before posting. `DiscordBot`: a slash command's first message replaces diff --git a/doc/slack.md b/doc/slack.md index 989905e0..70823393 100644 --- a/doc/slack.md +++ b/doc/slack.md @@ -121,7 +121,8 @@ its id) and the buttons. A press shows the chart for that period in the message that period from the last 15 minutes, for this message or another, with the report that came with it, or else the chart drawn and uploaded anew (Slack cannot replace a file). An upload that no message shows is deleted once those 15 minutes are past, so the bot's files do not pile up. If Slack will not show a chart with its buttons, the chart is uploaded into -the channel without them; if an upload fails (a missing scope, say), the report goes out as text. The log says why. +the channel without them. If an upload fails, it is tried once more with the report's heading alone as its message, the +report following as text; if that fails too (a missing scope, say), the report goes out as text alone. The log says why. `/jobs` lists the channel's jobs with the menu's buttons, as on Telegram; a button from a list that has since changed puts the current list in its place. diff --git a/lib/delivery.py b/lib/delivery.py index 58f42afb..09448960 100644 --- a/lib/delivery.py +++ b/lib/delivery.py @@ -62,17 +62,25 @@ def targets(dest): def deliver(dest, report): """Post a report: its chart, drawn here only where it can go (a destination that takes charts, with the graph preference on), with the lines as its caption; else the lines, and the buttons under the last message unless they - were the chart's. A chart the network refuses goes as text without its buttons, unless it may have arrived after all - (a timeout). Returns the network's answers, one per message.""" + were the chart's. A chart the network refuses goes again with its heading (the first line) alone as its caption, the + lines following as text; refused again, the lines go as text alone, without its buttons. A chart that may have + arrived after all (a timeout) is not sent again. Returns the network's answers, one per message.""" chart = report.chart if chart is None or not (dest.takes_charts and config.prefs.graph and chart.rows): return dest.text(report.lines, report.buttons if chart is None else None) caption = '\n'.join(map(str, report.lines)) or chart.title - answer = dest.chart(render.draw(chart), caption, report.buttons) + image = render.draw(chart) + answer = dest.chart(image, caption, report.buttons) if answer is transports.MAYBE_SENT: return [] if answer: return [answer] + heading = caption.split('\n', 1)[0] + if heading != caption: # the caption may be what was refused: too long, or markup the network cannot read + log.warn("chart not sent, sending it again with its heading alone, and the report as text") + answer = dest.chart(image, heading, report.buttons) + if answer: + return ([] if answer is transports.MAYBE_SENT else [answer]) + dest.text(caption.split('\n')) log.warn("chart not sent, sending the report as text") return dest.text(caption.split('\n')) diff --git a/lib/dialects.py b/lib/dialects.py index c6d2f131..94766612 100644 --- a/lib/dialects.py +++ b/lib/dialects.py @@ -57,8 +57,9 @@ def _link(self, url, text): return '' + text + '' def visible(self, text): - """The length without the HTML tags, which Telegram does not count.""" - return len(re.sub(r'<[^>]+>', '', text)) + """The length as Telegram counts it: the text the HTML shows (no tags, & one character), in UTF-16 code + units, so an emoji such as 🔼 counts 2.""" + return len(html.unescape(re.sub(r'<[^>]+>', '', text)).encode('utf-16-le')) // 2 class Slack(Dialect): diff --git a/lib/transports.py b/lib/transports.py index df5c99ee..ee70bb59 100644 --- a/lib/transports.py +++ b/lib/transports.py @@ -139,8 +139,8 @@ def chunked(lines, limit): def fit_caption(caption, dialect): - """The caption cut to the dialect's caption limit (none for Slack), as the network counts it (Telegram leaves the - tags out): the whole lines that fit, and how many were left out.""" + """The caption cut to the dialect's caption limit (none for Slack), as the network counts it (Telegram: the text + its HTML shows, in UTF-16 code units): the whole lines that fit, and how many were left out.""" limit = dialect.caption_limit if limit is None or dialect.visible(caption) <= limit: return caption diff --git a/tests/test_sending.py b/tests/test_sending.py index 2de4a762..7f93e4f2 100644 --- a/tests/test_sending.py +++ b/tests/test_sending.py @@ -1,6 +1,6 @@ """Posting to the networks against their answers and failures: Telegram, Slack (text, charts, Block Kit buttons) and Discord.""" -import contextlib, io, json, threading, unittest +import contextlib, html.parser, io, json, threading, unittest from unittest import mock import requests from tests.common import * @@ -181,7 +181,8 @@ def test_slack_replies_upload_their_charts(self): self.assertNotIn(share['data']['initial_comment'].split('\n'), [payload for _, payload in self.sent]) def test_a_slack_upload_that_fails_goes_out_as_text(self): - """A Slack upload failing at any step goes as text, but not on a last-step timeout, when it may have arrived.""" + """A Slack upload failing at any step, then again with its heading alone, goes as text, but not on a last-step + timeout, when it may have arrived.""" self.env(slackBotToken='xoxb-test') self.patch(render, 'draw', lambda chart: io.BytesIO(b'png')) @@ -195,11 +196,12 @@ def test_a_slack_upload_that_fails_goes_out_as_text(self): ([good[0], good[1], slow], []), # it may have been shared after all (good, [])): with self.subTest(outcomes=outcomes): - self.slack_answers(*outcomes) + calls = self.slack_answers(*outcomes, *outcomes) # a refused chart is tried again, failing the same way self.sent.clear() with contextlib.redirect_stderr(io.StringIO()), contextlib.redirect_stdout(io.StringIO()): delivery.deliver(slack_common.SlackBot('C1'), delivery.Report(['*Heading*', 'line 2'], CHART)) self.assertEqual(self.sent, expect) + self.assertEqual(len(calls), len(outcomes) * (2 if expect else 1)) def test_scheduled_reports_carry_their_charts_on_every_network(self): """A scheduled report's chart goes to the job's chat on every network, as its bot.""" @@ -259,6 +261,69 @@ def fake(url, **kwargs): delivery.deliver(telegram_common.TelegramBot('5'), report) self.assertEqual(self.sent, expect) + def telegram(self, refused): + """net.post as Telegram, refusing each caption refused(caption) is true for: the captions sent, in turn.""" + captions = [] + def post(url, data, **kwargs): + """Telegram's answer to sendPhoto: the message, or Bad Request.""" + captions.append(data['caption']) + if refused(data['caption']): + return Resp({'ok': False, 'error_code': 400, 'description': 'Bad Request: caption refused'}, status=400) + return Resp({'ok': True, 'result': {'message_id': len(captions)}}) + self.patch(net, 'post', post) + return captions + + def test_a_caption_fits_as_telegram_counts_it(self): + """A caption is cut to Telegram's 1,024 characters as it counts them: the text its HTML shows (no tags, an + entity one character), in UTF-16 code units, so each 🔼 counts 2. Counted by code points, a list of thirty + moves passed for 1,012 and was refused, and the chart went as text.""" + self.patch(render, 'draw', lambda chart: io.BytesIO(b'png')) + class Shown(html.parser.HTMLParser): + """The text an HTML caption shows, as Telegram parses it.""" + def handle_data(self, data): + """Keep the text between the tags, its entities read.""" + self.text += data + def length(caption): + """A caption's length as Telegram counts it: its text in UTF-16 code units.""" + shown = Shown() + shown.text = '' + shown.feed(caption) + return sum(2 if ord(char) > 0xFFFF else 1 for char in shown.text) + self.assertEqual(length('🔼 S&P'), TELEGRAM.visible('🔼 S&P')) + self.assertEqual(TELEGRAM.visible('🔼 S&P T1'), 9) + lines = ['Price alerts 3% (ASX, NASDAQ)'] + [ + f'{"🔼🔻"[i % 2]} Company Number {i:02d} Holdings Lt (T{i}' + f') {"+-"[i % 2]}{i % 9 + 3}%' for i in range(40)] + captions = self.telegram(lambda caption: length(caption) > 1024) + with contextlib.redirect_stderr(io.StringIO()): + delivery.deliver(telegram_common.TelegramBot('5'), delivery.Report(lines, CHART)) + self.assertEqual(len(captions), 1) # accepted at once: nothing sent again, nothing as text + self.assertEqual(self.sent, []) + self.assertLessEqual(length(captions[0]), 1024) + self.assertGreater(length(captions[0]), 990) # as many whole lines as fit + self.assertTrue(captions[0].endswith(' more in the chart'), captions[0][-40:]) + + def test_a_chart_refused_with_its_caption_goes_again_with_its_heading(self): + """A chart refused with its report as the caption goes again with the heading alone and its buttons, the + report following as text; refused again, or where the heading was the whole caption, the report goes as text.""" + self.patch(render, 'draw', lambda chart: io.BytesIO(b'png')) + markup = [[buttons.Button('7D', 'l|3|0|w')]] + for refused, captions, sent in ((lambda caption: '\n' in caption, ['Heading\nline 2', 'Heading'], + [('text', ['Heading', 'line 2'])]), (lambda caption: True, ['Heading\nline 2', + 'Heading'], [('text', ['Heading', 'line 2'])])): + with self.subTest(captions=captions): + self.sent.clear() + asked = self.telegram(refused) + with contextlib.redirect_stderr(io.StringIO()): + delivery.deliver(telegram_common.TelegramBot('5'), + delivery.Report(['Heading', 'line 2'], CHART, markup)) + self.assertEqual((asked, self.sent), (captions, sent)) + asked = self.telegram(lambda caption: True) + self.sent.clear() + with contextlib.redirect_stderr(io.StringIO()): + delivery.deliver(telegram_common.TelegramBot('5'), delivery.Report(['Heading'], CHART)) + self.assertEqual((asked, self.sent), (['Heading'], [('text', ['Heading'])])) + class DiscordSending(FinbotCase): """A Discord job's channel, as the bot: a scheduled report's chart attached, within Discord's 2,000 characters.""" From 5e5581af65b5dcd6fa4c505d656a5b4f45b6891d Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 11:23:00 +0000 Subject: [PATCH 21/41] log_width cuts chat messages too, each logged on one line A chat message's log line ("[Telegram]: @rob /price ...") went out as it was typed, over as many lines as the message had, at any length. - log.cut() puts a line on one line (each run of line breaks and spaces one space), then cuts it to log_width as before. At log_width=0 the line is whole, still on one line. - The message line and the line for a message sent twice within a second (now "sent it again within 1 s, not run again: TEXT", the text last so a cut takes only the text) go through it, as each request's line does. - The log_width docs (preferences.conf, configuration.md, the Prefs comment) name chat messages too. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 4 ++-- bot.py | 5 +++-- defaults/preferences.conf | 3 ++- doc/configuration.md | 2 +- lib/config.py | 2 +- lib/log.py | 9 +++++---- tests/test_edges.py | 20 ++++++++++++++++++++ 7 files changed, 34 insertions(+), 11 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 8ac62966..75cc46c0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -103,8 +103,8 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm optional (`sharesight.configured()`): without its keys `tickers()` is empty, so reports cover the watchlist alone, and `portfolios()` raises `NotSetUp`, an answer. - `lib/log.py`: the log, by level (`error warn info debug`), as the `log_level` preference lets in, and `cut()`, which - keeps a line that can run long (a request's) within the `log_width` preference; bot.py's `WholeLines` - writes each line whole, whichever threads log at once. `lib/cache.py`: cache files (any may vanish at any time). `lib/state.py`: state files in `DATA_DIR`, which must last; + keeps a line that can run long (a request's, a chat message's) on one line, within the `log_width` preference; + bot.py's `WholeLines` writes each line whole, whichever threads log at once. `lib/cache.py`: cache files (any may vanish at any time). `lib/state.py`: state files in `DATA_DIR`, which must last; change one with `with state.edit(name, default) as data:`, never load-then-save; a file of one record per chat with `state.load_for(chat, name, default)` and `state.edit_for`. A chat's key is its network and id (`Transport.key`, `telegram:-1001234`), the same in both modes. `lib/watchlist.py`: each chat's watchlist (`tickers(chat)`), and diff --git a/bot.py b/bot.py index 792b2cda..4ee73883 100755 --- a/bot.py +++ b/bot.py @@ -201,7 +201,8 @@ def take(network, update): log.info(f"[{network.service}]: update {network.update_id(update)} seen already, not run again") return inbound.OK if isinstance(event, inbound.Message) and not event.interaction and doubled(event): # each interaction is answered - log.info(f"[{network.service}]: {event.user} sent {event.text!r} again within {DOUBLED} s, not run again") + log.info(log.cut(f"[{network.service}]: {event.user} sent it again within {DOUBLED} s, not run again: " + + event.text)) return inbound.OK answer = network.acknowledge(event) if answer is not None: @@ -215,7 +216,7 @@ def dispatch(event): log.debug(f"[{event.service.capitalize()} button]:", event.user, "chat", event.chat_id, event.data) target = worker.process_callback else: - log.info(f"[{event.service.capitalize()}]:", event.user, event.text) + log.info(log.cut(f"[{event.service.capitalize()}]: {event.user} {event.text}")) target = worker.process_request thread = threading.Thread(target=target, args=(event,)) thread.start() diff --git a/defaults/preferences.conf b/defaults/preferences.conf index 8f56920d..84f1663a 100644 --- a/defaults/preferences.conf +++ b/defaults/preferences.conf @@ -54,7 +54,8 @@ # Diagnostics. log_level: what the log shows, that level and those above it: ERROR, WARN, INFO (what finbot does, # and each request it sends) or DEBUG (each cache miss, hit and write too). log_width: the most characters of a log -# line that can run long (each request's method and URL), cut there, ending in an ellipsis (…); 0 cuts none. +# line that can run long (each request's method and URL, each chat message), cut there, ending in an ellipsis (…); +# 0 cuts none. A chat message of several lines is logged on one line either way. #log_level=INFO #log_width=71 #print_headers=False diff --git a/doc/configuration.md b/doc/configuration.md index ead230a3..37e13c0d 100644 --- a/doc/configuration.md +++ b/doc/configuration.md @@ -75,7 +75,7 @@ A change takes effect on a reload. A true/false preference takes `true`/`false`, | `performance_cache_seconds` | `2400` | Cache lifetime of Sharesight performance reports in seconds: longer than the 30 minutes between `sharesight_warm` runs, so the holdings never run out | | `http_timeout` | `18` | Seconds to wait on outbound HTTP requests | | `log_level` | `INFO` | What the log shows: that level and those above it, syslog style. `ERROR`: what failed and was lost (a message not sent); `WARN`: what failed and was got over (a retry, a fallback, a request refused); `INFO`: what finbot does (each message sent, each job run, each request sent, each request refused); `DEBUG`: the routine of a button press too (the press, its acknowledgement, the message edited in place), each request answered with success, and each cache miss, hit and write. Replaces `debug` | -| `log_width` | `71` | The most characters of a log line that can run long: a longer one is cut there, ending in `…`. For now that is each request's line, its method and URL. `0`: no line is cut | +| `log_width` | `71` | The most characters of a log line that can run long: a longer one is cut there, ending in `…`. That is each request's line (its method and URL) and each chat message's (who sent what, in which network). A message of several lines is logged on one line, its line breaks spaces. `0`: no line is cut | | `print_headers` | `False` | Log inbound request headers | A key in the wrong file, or one that is not listed, stops finbot starting; on a reload it is reported and nothing is applied. A key finbot no longer reads (`alliterate`, `chunk_maxlines`) is ignored, and the log at start says to delete it. diff --git a/lib/config.py b/lib/config.py index dabf3a3a..ce16e19f 100644 --- a/lib/config.py +++ b/lib/config.py @@ -306,7 +306,7 @@ class Prefs: hyperlinkProvider: str = 'google' include_portfolios: tuple = () # portfolio ids log_level: str = 'INFO' # what the log shows: one of LOG_LEVELS, and those more severe (lib/log.py) - log_width: int = 71 # the most characters of a log line that can run long (a request's); 0: no cut (log.cut) + log_width: int = 71 # the most characters of a log line that can run long (a request's, a chat message's); 0: no cut min_analysts: int = 7 # .buy, .sell and .forward pe list only stocks this many analysts cover past_days: int = 28 performance_cache_seconds: int = 2400 # Sharesight performance reports (40 minutes: sharesight_warm runs every 30) diff --git a/lib/log.py b/lib/log.py index 6294f924..fb07f256 100644 --- a/lib/log.py +++ b/lib/log.py @@ -1,7 +1,7 @@ """The log, syslog style: each line has a level, ERROR, WARN, INFO or DEBUG, and goes to stderr (which systemd and Docker take to the journal) when the log_level preference lets it in: that level and those above it, so ERROR always -does. Each takes print()'s arguments. A line that can run long (a request's URL) goes through cut(), which keeps it -within the log_width preference.""" +does. Each takes print()'s arguments. A line that can run long (a request's URL, a chat message) goes through cut(), +which puts it on one line within the log_width preference.""" import sys from lib import config @@ -21,8 +21,9 @@ def write(level, parts, options): def cut(text): - """A long log line (text) cut to log_width characters, its last an ellipsis (…); whole where log_width is 0.""" - width = config.prefs.log_width + """A log line that can run long (text) on one line, each run of line breaks and spaces one space, and cut to + log_width characters, its last an ellipsis (…); whole where log_width is 0.""" + text, width = ' '.join(str(text).split()), config.prefs.log_width return text if width <= 0 or len(text) <= width else text[:width - 1] + '…' diff --git a/tests/test_edges.py b/tests/test_edges.py index 1224d9aa..353fdc04 100644 --- a/tests/test_edges.py +++ b/tests/test_edges.py @@ -443,6 +443,26 @@ def test_log_width_sets_where_a_long_line_is_cut(self): net.get('https://' + whole[4:]) self.assertEqual(err.getvalue().splitlines(), [line]) + def test_a_chat_message_is_one_line_cut_to_log_width(self): + """A chat message is logged on one line, its line breaks spaces, cut to log_width characters like a request's; + at 0 it is whole, still on one line. So is one not run again, sent twice within a second.""" + self.patch(worker, 'process_request', lambda message: None) + self.patch(bot, 'doubled', REAL_DOUBLED) + text = '/jobs add message daily at 9am "Morning!\nThe ASX opens at 10am,\n\nso coffee first"' + def logged(update_id, chat=55): + """What taking a Telegram message of that text in that chat logs.""" + update = {'update_id': update_id, 'message': {'message_id': update_id, 'chat': {'id': chat}, + 'from': {'first_name': 'Rob', 'username': 'rob'}, 'text': text}} + with contextlib.redirect_stderr(io.StringIO()) as err: + bot.take(telegram_common.READER, update) + self.join_workers() + return err.getvalue().splitlines() + whole = '[Telegram]: @rob /jobs add message daily at 9am "Morning! The ASX opens at 10am, so coffee first"' + self.assertEqual(logged(1), [whole[:70] + '…']) + self.assertEqual(logged(2), ['[telegram]: @rob sent it again within 1 s, not run again: /jobs add me…']) + self.prefs(log_width=0) + self.assertEqual(logged(3, chat=66), [whole]) # another chat's: taken + class SharedSession(unittest.TestCase): def test_the_session_keeps_no_cookies(self): From ea5828f23af8a3387471df4dfbc34fab9cf3f683 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 11:26:02 +0000 Subject: [PATCH 22/41] .performance lists the portfolios by name, then the benchmarks The list and its chart were sorted by gain, best first, so a portfolio's place changed from one report to the next. Now the portfolios come by name (ignoring case), then the S&P 500 and NASDAQ 100 benchmarks, in the same order in the list and the chart. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- doc/commands.md | 2 +- lib/reports/performance.py | 9 ++++----- tests/test_charts.py | 19 ++++++++++++++----- 3 files changed, 19 insertions(+), 11 deletions(-) diff --git a/doc/commands.md b/doc/commands.md index 43bc146b..f1bd9724 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -195,7 +195,7 @@ Or a specific stock. In this case, the output includes a graph. ### Portfolio performance ![Screenshot of Portfolio performance notification on Slack](../img/performance.png?raw=true "Screenshot performance notification on Slack") -The `performance` report covers Sharesight portfolio performance over a given period. The default is set by `past_days` in the `preferences.conf`, which can be overridden at runtime by specifying an integer as the argument. +The `performance` report covers Sharesight portfolio performance over a given period. The default is set by `past_days` in the `preferences.conf`, which can be overridden at runtime by specifying an integer as the argument. The portfolios are listed by name, then the S&P 500 and NASDAQ 100 benchmarks, in the list and the chart alike. Config example: ``` diff --git a/lib/reports/performance.py b/lib/reports/performance.py index b2c8ef69..c09df2d9 100644 --- a/lib/reports/performance.py +++ b/lib/reports/performance.py @@ -28,19 +28,18 @@ def benchmark(ticker, market, text, past_days, service): return f"{formats.get_emoji(percent)} {link} {formats.signed_percent(percent, 2)}", percent def report(service, past_days, portfolio_select=None, performance=None): - """(payload, chart) of the performance report for one service: each portfolio and the benchmarks, best first. - performance: gather()'s, fetched when not given.""" + """(payload, chart) of the performance report for one service: each portfolio by name, in any case, then the + benchmarks. performance: gather()'s, fetched when not given.""" if performance is None: performance = gather(past_days, portfolio_select) - rows = [] # (percent or None, line, chart label), one list for the caption and the chart, best first - for portfolio, percent in performance: + rows = [] # (percent or None, line, chart label), one list for the caption and the chart, in the same order + for portfolio, percent in sorted(performance, key=lambda each: each[0].name.casefold()): link = service.link(f"https://portfolio.sharesight.com/portfolios/{portfolio.id}", portfolio.name) line = f"{formats.get_emoji(percent)} {link} {formats.signed_percent(percent, 2)}" rows.append((percent, line, portfolio.name)) for ticker, market, text in BENCHMARKS: line, percent = benchmark(ticker, market, text, past_days, service) rows.append((None if percent is None else float(percent), line, text)) - rows.sort(key=lambda row: float('-inf') if row[0] is None else row[0], reverse=True) # an error line sorts last period = formats.days_english(past_days) payload = [service.bold(f"Performance over {period}")] + [line for _, line, _ in rows] chart_rows = [(label, percent) for percent, _, label in rows if percent is not None] diff --git a/tests/test_charts.py b/tests/test_charts.py index df98ac6c..cceb3120 100644 --- a/tests/test_charts.py +++ b/tests/test_charts.py @@ -4,7 +4,7 @@ 3. one golden image per chart type, compared with a tolerance. Refresh after an intended restyle with: UPDATE_GOLDEN=1 python -m unittest tests.test_charts """ -import datetime, os, subprocess, sys, tempfile, threading, unittest +import datetime, os, re, subprocess, sys, tempfile, threading, unittest from unittest import mock import numpy as np from PIL import Image @@ -338,11 +338,20 @@ def test_price_over_a_period_has_a_dated_title(self): chart = self.run_report(lambda: self.command('.price 7d')) self.assertRegex(chart.title, r'\d{1,2} \w{3} - \d{1,2} \w{3} \d{4}') # 28 Sep - 5 Oct 2026 - def test_performance_is_sorted_by_percent(self): - """.performance 7d: the chart is sorted best first.""" + def test_performance_is_sorted_by_name(self): + """.performance 7d: the portfolios by name, in any case, whatever their gains, then the two benchmarks: the + list and the chart in the same order.""" + self.patch(sharesight, '_portfolios', lambda seconds=None: [{'name': name, 'id': i} for i, name in + enumerate(('super', 'Rob', 'kids', 'Bonds'), 1)]) + gains = {1: '9.5', 2: '-3.25', 3: '0.5', 4: '12'} + self.patch(sharesight, '_performance', lambda pid, days=None, seconds=None: + {'capital_gain_percent': gains[pid], 'holdings': []}) chart = self.run_report(lambda: self.command('.performance 7d')) - values = [v for _, v in chart.rows if v is not None] - self.assertEqual(values, sorted(values, reverse=True)) + names = ['Bonds', 'kids', 'Rob', 'super', 'S&P 500', 'NASDAQ 100'] + self.assertEqual([label for label, _ in chart.rows], names) + self.assertEqual([value for _, value in chart.rows][:4], [12.0, 0.5, -3.25, 9.5]) + payload, _ = performance.report(TELEGRAM, 7) + self.assertEqual([re.search(r'>([^<]+)', line).group(1) for line in payload[1:]], names) def test_shorts_is_sorted_and_carries_the_threshold(self): """.shorts: the pictogram gets the stocks over the threshold, most shorted first, and the threshold.""" From f2d228edafcaa92224bc695a8a614a5535ab62ba Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 12:00:59 +0000 Subject: [PATCH 23/41] Add .treemap: a portfolio's holdings as tiles grouped by industry or sector Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 2 +- doc/commands.md | 2 + doc/configuration.md | 2 +- doc/scheduling.md | 2 +- lib/catalog.py | 2 +- lib/render.py | 79 +++++++++++++++++++++++++++++++++++++-- lib/reports/__init__.py | 2 +- lib/reports/holdings.py | 51 +++++++++++++++++++------ lib/worker.py | 8 ++++ tests/golden/treemap.png | Bin 0 -> 5582 bytes tests/test_charts.py | 22 +++++++++++ tests/test_reports.py | 56 ++++++++++++++++++++++++++- tests/test_schedule.py | 4 +- tests/test_scheduler.py | 10 ++--- util/record_outputs.py | 3 +- 15 files changed, 216 insertions(+), 29 deletions(-) create mode 100644 tests/golden/treemap.png diff --git a/CLAUDE.md b/CLAUDE.md index 75cc46c0..e52825c9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -73,7 +73,7 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm `price performance trades shorts cal rating milestone message podcasts cleanup sharesight_warm yahoo_warm`. trades, rating, milestone and podcasts keep a record per chat of what they told it (`state.load_for`, `state.edit_for`), added to only by a scheduled run (`delivery.interactive(dest)` is false). - Chat-only (`prepare_*`): `about` (.help, and the .jobs menu's screens: `jobs_screen`, `add_screen`, `next_screen`), `history` (.history .compare), `holdings` (.holdings .who .pie + Chat-only (`prepare_*`): `about` (.help, and the .jobs menu's screens: `jobs_screen`, `add_screen`, `next_screen`), `history` (.history .compare), `holdings` (.holdings .who .pie .treemap .watchlist), `profile` (.SYMBOL), `rankings` (.marketcap .pe .fpe .peg .beta .buy .sell). - `lib/delivery.py`: `Report(lines, chart, buttons)`, the destinations (the chat that asked, or a chat by its key with `chat(key)`, as its network's bot), `deliver(dest, report)` (a chart refused goes again with its heading alone as the diff --git a/doc/commands.md b/doc/commands.md index f1bd9724..3a78a613 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -55,6 +55,7 @@ Where: /session [percent|symbol] /shorts [percent|symbol] /trades [period] [portfolio] +/treemap portfolio [sector] /watchlist [add|del symbol] /who symbol ``` @@ -71,6 +72,7 @@ The bot answers commands in every chat it is in. By default, it fetches its upda * Stock lookup (financials, company profile, price charts) * A watch list per chat, which anyone in it can change, picked up by the reports there * Listing of portfolios and portfolio performance +* A pie or a treemap of a portfolio by industry or sector (`/pie portfolio [sector]`, `/treemap portfolio [sector]`): the treemap tiles each holding by value, grouped and coloured by industry (the default) or sector * Listing of holdings sorted by various attributes * Running the other stock reports on demand diff --git a/doc/configuration.md b/doc/configuration.md index 37e13c0d..d7ad143f 100644 --- a/doc/configuration.md +++ b/doc/configuration.md @@ -107,7 +107,7 @@ sharesight_code=01d692d4de7 samplefc64bc2e2f01d692d4de72986ea808f6e99813f sharesight_client_id=01d692d4de7 samplefc64bc2e2f01d692d4de72986ea808f6e99813f sharesight_client_secret=01d692d4de7 samplefc64bc2e2f01d692d4de72986ea808f6e99813f ``` -* Sharesight is optional. Without its keys, a chat's reports cover its watchlist alone, so add your holdings to it (`/watchlist add SYMBOL`); `/holdings`, `/trades`, `/performance`, `/who` and `/pie` say Sharesight is not set up, and the trades and performance jobs are not among the default jobs. +* Sharesight is optional. Without its keys, a chat's reports cover its watchlist alone, so add your holdings to it (`/watchlist add SYMBOL`); `/holdings`, `/trades`, `/performance`, `/who`, `/pie` and `/treemap` say Sharesight is not set up, and the trades and performance jobs are not among the default jobs. ## Portfolios Portfolios are auto-discovered, including other people's portfolios which are shared to you. To exclude specific portfolios, add their IDs to `exclude_portfolios` in the `preferences.conf`: diff --git a/doc/scheduling.md b/doc/scheduling.md index 1156d644..36d4e2e7 100644 --- a/doc/scheduling.md +++ b/doc/scheduling.md @@ -93,7 +93,7 @@ The reports are `trades`, `price midsession|interday|premarket|close|DAYS`, `cal A job can also run a command of `/help`, with what it takes, and posts what the command posts when typed in the chat: `pe [SYMBOL|top|bottom]`, `forward pe`, `peg`, `marketcap`, `beta`, `buy`, `sell` (`buy&sell` both), `history SYMBOL`, -`compare SYMBOL SYMBOL…`, `stock SYMBOL`, and with Sharesight `holdings PORTFOLIO`, `pie PORTFOLIO` and `who SYMBOL`. +`compare SYMBOL SYMBOL…`, `stock SYMBOL`, and with Sharesight `holdings PORTFOLIO`, `pie PORTFOLIO`, `treemap PORTFOLIO` and `who SYMBOL`. The commands that change something (`watchlist`, `plan`, `jobs`) are no jobs, and price, performance, trades, shorts, earnings, dividends, session and premarket are run by their reports above. diff --git a/lib/catalog.py b/lib/catalog.py index 4a138d23..4fa5ade1 100644 --- a/lib/catalog.py +++ b/lib/catalog.py @@ -14,7 +14,7 @@ MORNING = '08:30' # by Sydney's clock: the weekly and longer reports, and the messages TUE_SAT = ['tue', 'wed', 'thu', 'fri', 'sat'] # the mornings after a US session FLAGS = {'AU': '🇦🇺'} # what {flag} becomes in a message for one country -SHARESIGHT = ('trades', 'performance', 'holdings', 'pie', 'who') # the actions that report on Sharesight +SHARESIGHT = ('trades', 'performance', 'holdings', 'pie', 'treemap', 'who') # the actions that report on Sharesight PERIODIC = ('performance', 'price') # the reports whose period follows how often they run PERIOD_DAYS = {'daily': 1, 'weekdays': 1, 'weekly': 7, 'monthly': 30, 'quarterly': 90, 'bi-annual': 180, 'annual': 365} MONEY_CAFE = ('https://www.omnycontent.com/d/playlist/b034de0e-930d-434b-9822-a7140060b2c0/' diff --git a/lib/render.py b/lib/render.py index 09a8134c..db2bbdba 100644 --- a/lib/render.py +++ b/lib/render.py @@ -17,8 +17,9 @@ @dataclasses.dataclass(frozen=True) class Chart: """A chart not yet drawn, for draw(). kind 'bars': rows of (label, value), first on top, a (label, None) row a gap; - 'pie': rows of (label, value); 'shorts': rows of (label, percent), and the threshold; 'line': rows of (date, close), - a price line, and the change it shows; 'compare': rows of (label, ((date, close), ...)), two to six price lines.""" + 'pie': rows of (label, value); 'treemap': rows of (group, ((label, value), ...)); 'shorts': rows of (label, + percent), and the threshold; 'line': rows of (date, close), a price line, and the change it shows; 'compare': + rows of (label, ((date, close), ...)), two to six price lines.""" kind: str title: str = '' subtitle: str = '' @@ -510,6 +511,77 @@ def draw_pie(chart): _heading(fig, chart.title, chart.subtitle) return _png(fig, bg) +def _worst(row, side): + """The worst aspect ratio among tiles of areas `row` laid along a side of length `side`.""" + total = sum(row) + return max(max(side * side * area / (total * total), total * total / (side * side * area)) for area in row) + +def _squarify(values, x, y, w, h): + """Tiles (x, y, w, h) filling the box, one per value (largest first), areas in proportion and as square as + the squarified layout makes them.""" + scale = w * h / sum(values) + areas = [value * scale for value in values] + tiles, i = [], 0 + while i < len(areas): + side = min(w, h) + row, j = [areas[i]], i + 1 + while j < len(areas) and _worst(row + [areas[j]], side) <= _worst(row, side): + row.append(areas[j]) + j += 1 + span = sum(row) / side + offset = 0.0 + for area in row: + length = area / span + tiles.append((x, y + offset, span, length) if w >= h else (x + offset, y, length, span)) + offset += length + if w >= h: + x, w = x + span, w - span + else: + y, h = y + span, h - span + i = j + return tiles + +def _mixed(color, other, share): + """`color` (#rrggbb) with `share` of `other` mixed in.""" + mixed = [round(int(color[i:i + 2], 16) * (1 - share) + int(other[i:i + 2], 16) * share) for i in (1, 3, 5)] + return '#%02x%02x%02x' % tuple(mixed) + +@_drawn +def draw_treemap(chart): + """Treemap of (group, ((label, value), ...)) rows, largest group first: a block a group, in a colour of its own + (the dark categorical slots in order, gray for 'Other') under its name, a tile a holding sized by value: a + 1280x800 PNG.""" + from matplotlib.patches import Rectangle + rows = chart.rows + bg, ink, ink2, _ = PALETTE + fig, _ = _figure(bg) + ax = fig.add_axes([0, 0, 1, 1], facecolor=bg) + ax.set_xlim(0, 1280) + ax.set_ylim(800, 0) + ax.axis('off') + total = sum(value for _, held in rows for _, value in held) or 1.0 + box = _squarify([sum(value for _, value in held) for _, held in rows], 24, 92, 1232, 684) + char, n = 12.5, 0 # pixels a character of the 8pt tile text takes + for (label, held), (x, y, w, h) in zip(rows, box): + color = '#6b7280' if label == 'Other' else CATEGORIES[n % len(CATEGORIES)] + n += label != 'Other' + x, y, w, h = x + 1, y + 1, w - 2, h - 2 + head = 22 if h > 60 and w > 3 * char else 0 + if head: + room = int((w - 4) / char) + ax.text(x + 2, y + head / 2, label if len(label) <= room else label[:max(room - 1, 1)] + '…', + color=color, fontsize=8, fontweight='bold', ha='left', va='center', clip_on=True) + tiles = _squarify([value for _, value in held], x, y + head, w, h - head) + for k, ((name, value), (tx, ty, tw, th)) in enumerate(zip(held, tiles)): + ax.add_patch(Rectangle((tx + 1, ty + 1), tw - 2, th - 2, facecolor=_mixed(color, bg, k % 3 * 0.14), + edgecolor='none')) + if tw > len(name) * char + 8 and th > 34: + lines = name + ('\n%.1f%%' % (value / total * 100) if th > 64 and tw > 5 * char else '') + ax.text(tx + tw / 2, ty + th / 2, lines, color='#ffffff', fontsize=8, ha='center', va='center', + linespacing=1.3) + _heading(fig, chart.title, chart.subtitle) + return _png(fig, bg) + def _code_version(): """What else decides how a spec looks: this file and the matplotlib release.""" try: @@ -520,7 +592,8 @@ def _code_version(): CODE_VERSION = _code_version() -RENDERERS = {'bars': draw_bars, 'pie': draw_pie, 'shorts': draw_shorts, 'line': draw_line, 'compare': draw_compare} +RENDERERS = {'bars': draw_bars, 'pie': draw_pie, 'treemap': draw_treemap, 'shorts': draw_shorts, 'line': draw_line, + 'compare': draw_compare} ROWS = 40 # bars: the rows a 1280x800 image has room for; more make it taller (draw_bars) LIMITS = {'shorts': 24} # the rows a chart of a fixed size has room for diff --git a/lib/reports/__init__.py b/lib/reports/__init__.py index 387268d7..2216a61b 100644 --- a/lib/reports/__init__.py +++ b/lib/reports/__init__.py @@ -2,5 +2,5 @@ milestone, performance, podcasts, price, rating, shorts, trades, and sharesight_warm and yahoo_warm (which fill the cache), each with job(argv), which parses a job's arguments, and run(dest=None, **kwargs), which runs it, to the chat that asked or whose job it is (dest); the scheduler runs them in the bot's process, and `.jobs run` runs one now. And -the chat-only reports: about (.help, .jobs), history (.history, .compare), holdings (.holdings, .who, .pie, .watchlist), +the chat-only reports: about (.help, .jobs), history (.history, .compare), holdings (.holdings, .who, .pie, .treemap, .watchlist), profile (.SYMBOL) and rankings (.marketcap, .pe and the rest).""" diff --git a/lib/reports/holdings.py b/lib/reports/holdings.py index 0d2336df..3392d1f0 100644 --- a/lib/reports/holdings.py +++ b/lib/reports/holdings.py @@ -1,4 +1,4 @@ -"""What is held and followed: .holdings, .who, .pie and .watchlist.""" +"""What is held and followed: .holdings, .who, .pie, .treemap and .watchlist.""" from lib import watchlist from lib import sharesight from lib import formats @@ -32,20 +32,29 @@ def prepare_who(service, code): blocks.append(found and [service.bold(f"{portfolio.name}:")] + found) return formats.sections(*blocks) or [f'{code} not found in any portfolio'] -def prepare_pie(portfolio_name, by, service, max_slices=8): - """Portfolio value by industry (or by='sector'): (payload, chart or None). Raises sharesight.PortfolioNotFound.""" - portfolio, = sharesight.portfolios(portfolio_name) - name = portfolio.name +def _groups(portfolio, by): + """The holdings of a portfolio with a value, by industry (or by='sector'): {group: [(code, value), ...]}.""" key = 'sector_classification_name' if by == 'sector' else 'industry_classification_name' - totals = {} + groups = {} for holding in portfolio.holdings().values(): try: value = float(holding.get('value')) except (TypeError, ValueError): continue if value > 0: - label = holding.get(key) or 'Unclassified' - totals[label] = totals.get(label, 0.0) + value + groups.setdefault(holding.get(key) or 'Unclassified', []).append((holding['code'], value)) + return groups + +def _noun(by, count): + """'3 industries', '1 sector'.""" + singular, plural = {'sector': ('sector', 'sectors'), 'industry': ('industry', 'industries')}[by] + return f"{count} {singular if count == 1 else plural}" + +def prepare_pie(portfolio_name, by, service, max_slices=8): + """Portfolio value by industry (or by='sector'): (payload, chart or None). Raises sharesight.PortfolioNotFound.""" + portfolio, = sharesight.portfolios(portfolio_name) + name = portfolio.name + totals = {label: sum(value for _, value in held) for label, held in _groups(portfolio, by).items()} if not totals: return [f"{name} has no holdings with a value"], None rows = sorted(totals.items(), key=lambda kv: kv[1], reverse=True) @@ -57,9 +66,29 @@ def prepare_pie(portfolio_name, by, service, max_slices=8): title = f"{name} by {by}" payload = [service.bold(title)] payload += [f"{label}: {value / total * 100:.1f}%" for label, value in rows] - noun = {'sector': ('sector', 'sectors'), 'industry': ('industry', 'industries')}[by] - subtitle = f"{len(totals)} {noun[0] if len(totals) == 1 else noun[1]}" - return payload, render.Chart('pie', title, subtitle, tuple(rows)) + return payload, render.Chart('pie', title, _noun(by, len(totals)), tuple(rows)) + +def prepare_treemap(portfolio_name, by, service, max_groups=8): + """The holdings by industry (or by='sector') as tiles grouped by it: (payload, chart or None). Beyond `max_groups` + the smallest groups are one 'Other'. Raises sharesight.PortfolioNotFound.""" + portfolio, = sharesight.portfolios(portfolio_name) + name = portfolio.name + groups = _groups(portfolio, by) + if not groups: + return [f"{name} has no holdings with a value"], None + rows = sorted(((label, sorted(held, key=lambda item: item[1], reverse=True)) for label, held in groups.items()), + key=lambda row: sum(value for _, value in row[1]), reverse=True) + if len(rows) > max_groups: + rest = sorted((item for _, held in rows[max_groups - 1:] for item in held), key=lambda item: item[1], + reverse=True) + rows = rows[:max_groups - 1] + [('Other', rest)] + total = sum(value for _, held in rows for _, value in held) + title = f"{name} treemap by {by}" + payload = [service.bold(title)] + payload += [f"{label}: {sum(value for _, value in held) / total * 100:.1f}%" for label, held in rows] + count = sum(len(held) for held in groups.values()) + subtitle = f"{_noun(by, len(groups))}, {count} {'holding' if count == 1 else 'holdings'}" + return payload, render.Chart('treemap', title, subtitle, tuple((label, tuple(held)) for label, held in rows)) def prepare_watchlist(service, chat, user, action=None, ticker=None): """The .watchlist reply in a chat (by its key): the companies it watches, or one line on adding ('add') or removing diff --git a/lib/worker.py b/lib/worker.py index f74b996d..c1b95d40 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -391,6 +391,13 @@ def on_pie(request, args): payload, chart = holdings.prepare_pie(args.rest, args.keyword or 'industry', request.service) request.reply(payload, chart) +def on_treemap(request, args): + """.treemap PORTFOLIO [sector]: its holdings as tiles, grouped by industry or by sector.""" + if not args.words: + raise Usage(portfolios()) + payload, chart = holdings.prepare_treemap(args.rest, args.keyword or 'industry', request.service) + request.reply(payload, chart) + def on_rating(request, args): """.buy, .sell or both (.buy & sell, a keyboard button): the analysts' strongest buy or sell ratings, as asked.""" for action in dict.fromkeys(re.findall('buy|sell', f'{args.word} {args.text}'.lower())): # each once, as asked @@ -539,6 +546,7 @@ class Command: Command(('help', 'usage', 'start'), on_help, job=False), # /start is what Telegram sends when a chat opens Command(('holdings', 'holding'), on_holdings, ('.holdings PORTFOLIO',), TRACKED), Command(('pie',), on_pie, ('.pie PORTFOLIO [sector]',), TRACKED, keywords=BY), + Command(('treemap',), on_treemap, ('.treemap PORTFOLIO [sector]',), TRACKED, keywords=BY), Command(('watchlist', 'wishlist'), on_watchlist, ('.watchlist [add|del SYMBOL]',), TRACKED, ('.watchlist',), keywords=CHANGE, job=False), Command(('who',), on_who, ('.who SYMBOL',), TRACKED), diff --git a/tests/golden/treemap.png b/tests/golden/treemap.png new file mode 100644 index 0000000000000000000000000000000000000000..4660048a0d4d52381c4b141d87188548fed53dd2 GIT binary patch literal 5582 zcmV;<6*20GP)003kN0ssI2j?}E!000%0NklFbpFsqvM=&0OFimYqi#hXe}*3%i{4E z00co`tQDLaV~nwmBb8E!XpCWug`py%n&)Yy8Dqv6t+fzBYaR8zCaB8;jEGw6M;`g7 zhadd>z?Oj*Uwmoj&YhJ?W$)fS-}uJY4;^|#DdqY8efJ)C=)upQI(@peHSM~tF{Y!v zJ)6z8wzjslr3-~(cXwAJ5f@yzt{aBo=FOXwQpT9>?(SqVSuU5$mC8d8e&KVU`^=$3 zZ^Ywq&N*kSrL|>lcJ|9(`7$DwOQnwX_Cm1;01tfX{x5y$&t84?U@Dc0$Kyc|^!4>1 z;@7|Scjo`RZ4H`m>+kzJGsrcjr@2J+uGz z{r#IaJ@Mo(9(dq^Sj_c3KMccv{kMPr<~RQ#lWBYW@h86W<-aIbDo;K2^iO~CqhvC9 z*GE6n)!8{RIwm+1Ql32d_Rj`?v}scxW9-}C{#HEZ{?`wFc;}se)W4~3GCTd9@BHJD zBggK%^Nz96vCn?y)8GHkN5A;RhhKW><-0%jvChtpQmK6M?YH;t+4CRY``*QgNk_`2 ztX?E`xdH~x`N-&)>$>;cefRWqE|baZ*s)_KpZ99D+itxTNF6zP^z`Y$TGbmJ8~f!m zzr5$;cVE0Xv2FXd+xPFA&Sv|1`>NH-kt0Wkhev9j_wd6HZQR&1(7$=-&K>!D-m7`H z-F7S4FquqFO=Z*R^!WJrrcImgJ@AR&{`M6n;~O??=I zr_P-l^t|emzj*5Kk)xjHz4E(PCnhEb2M33SM@C0Sj~_o#ES6q*<#)B3ci_N*=bwN6 z@yDN327KR7CKH3_&;R!iet7@=_YDt^JpJ@D>2%un{lUTWLqkL37sihqJ$n53$*#_h z7hih$;K76K?HgV@_FMml4?pzS|2#H${=C}+ z_;SK4tmtB~m`WyVz8`blN~PjBj+C-mtun?^sTAW(g{oGoNhvw!@p!yasS;5<9`k)) zDV0bhlv0eb$PNbpV~k&|esKT3fq{XipLlXZR~P3z2!dM81E8+1t}lGy^Upo^>r$x{ zF^V)OA_Abb#`pbLEG`7ERH{-6*L8y+;G83}Fbo{WvDWV1y=&jTeNR653(k2H(f0s& z%hv64v-z4=lR_$`g1~QY?+C-NR;vJ@luja%s8q_yWU5puT5F}0ks+OGY3c3VG<1G2 z)zYH1s+3DoI>}T^ty<~m?3&HzOXX4`9*<19MI?+d<#IWhOmfZ_ou7BQZ3};qwU;?; z(U&j`ecw-|QcXDD&B#K;Fbstd-95b{L+5wyz6FtaUahUIy;?2jrn4J+`oh3ZBvYPO z3&Zf1Js${s4*-YGpPQbV?Ck2^vVBLXRFsahX;XipFvq#zoM~+`ZEbCB8{&zC=Xr=I zgw$G{JbpCdo5^IXwGmX&pkJ1ZF;{$9Bq?QN&o|+G4;{>fXz%DOmx`Y6r&6uc(^H9f zLK&?>mCj^5&m*E*tyU@)Jg=5arb@+vB^!z5a;enXn!Y%G!5UMoR^#!6wWd-obIz4g zzVGK}a*VOg&aN;BEZM@`tTCou(TP8>5d>febomjyIYcy)RX~=@xHcNCHd=A%T0kL$ z(VB4%1lAhP1(A)$TqIYelmMu;S{ijCvW)TQLPWIIATlBv*s?8`%hswD#@1S2B7i0c zEyVaG=Rpx5fIt8MTF(Gm!;>96o(Xd2z$&L>2al)2{IKx~M3!-ftOjJrI2i>@AQ!MC zOt@ec!Lk~3=@F$A=lq?zc5TpYcVJV3oi5pm2996|HYYH!FcT2Nd6?Iu4n}5mE!Fp- z#K1?q@wY^>Q>J^7J3)R38RLoeaBft#bQ`Z=gR0xLyY%)me)cqzF(3;72>33&{|emQ zn24g$YwhnNcnbDs=&$aaCV3}301skyn+oK0DGedDh9-ekJMUz zHDzZ4N(-C`sA?d=At~shNBxK;MI7t=qB$Z0*fqqM00csU&-Ya9*f;=)q*eL@56RXZ z8&;7!L>3tbWTXQi0>%LdfFS~rMJAE48!DVYjEibHfBWPUx3yr^%L2pk3%eR^TqMjA&fT3G21{%Omu`)p7Fj7&=cxMTWk3e&jL3ra$tzYgkeQ<#({B$z!yGi0C2=;k?9%$j4;pfl6BdP^{j7d0^9_+Nlk#8)C9OmO@N!! z1h`2}fSc3=_(q{9W_*2V-lF*$8;*zw2mwheU>vLkhPcxFh$g^|CsZ955XoAnt*(`-2T)P@>0^CR=*k~r*mTh}vYa0Q9rS?1Sojdkg zqHkO2+_B=RHxc=@%IYS-4P`)cAgjs+RW0dC2^pU|`dabKVLvy)gapJDXyPWo4dnhZ z$1R;Z7HyglIYpJ>Q%pM5(X#}=gly{IRZ)$XWj3#B!XXg~2}`)4aDXgI2Up6v3GjQp zbFBaojEwX4ir+)O{0*teTu!)HV|0}T1h7N^zz_`)5g>rmfGfYOMJGmD zm0o__)Jn`vAmhLoSla}6dD0d-Qp{M0OB5WukXm9aCec?klGm)!Am#$dR*(J`kpYmY z7Qq-`967sz78S1@zzm_L>DC^(r`MUS+S#g!NemTKLR$+-8H!146#0ipq3>?pi-@vo zb8z9T^(v`5?pOIKL4FM*Q9M4ay6N=&zU5V9zI0GVJ45h#Z zdSgV$g{hT;vD39vZ!pKDFetwKb5qQt<1#0H18oMd)&Soz;8p?~m-y=W;KKt>C7?{4 z%~s8YIn!7=mt*JO2}1(26-HnOH8I8bzQB4i9Ei$*Y?R&`i3M6(@L^qUSWw;v~`1ogA`UXn4x~H`m|*umE=6 zsiaHG2ImV{)H8J|Dt$sXPfk`NpzDxL5`-2w1KdP+9AjZI9;IcKE zF0=$L1Bo{uz!o40^k~t2RxynX1*7RQ1PXf6Fjs?60|5vI0Cijn#{g;uDwb+(_mYdcg!8lQ!ZE+7Ix=m8+0YJjPjeSH|b5bBhoO#lpl z<9ywuMcZi=y{rYM4&H1cnt&vCr;6-1#{rQfkIEDi5Jqo2>0(>Vgdz9=)HMg6t{h_Q zdUXN2da1>Egpnk_*>o@g0JSG=Pa3oZcBujuA%d}(Nl+#MpWF?M)pwNP6#_I@0apND zSrI(~#;8z31dx207Knf|Dtd5kI_d}Fvj4~e+4Tsn1t16%yS<;?y9+A8B}E%`c|$?3!dw8p^|m+A{%Qwxru^EpI-q_a#5(Fr4Yb0!;!iX{c2g@ZDIRyT0n zBt%@1D?=8YBsy{M6_k9aUPa3yK%iF}VDJygq_T6fgsYk%Y z=IIZ(B?7<&lWN%vYO#8+gRXcVYl6WlF|tIKD4L*+5I{tMtiJ3FSxW>&3|RyWT_^E8ID7mOhO?ZY_Q{g@RxAeBOoy||SCt8d(hc2EsWR~3- z01-W{?%lrE71CIXh}sxHcx&{WM;2EPU?LD4t)-0_XRfMal84&HUEzl|Pe=cX_ETBV#=Z>If1A?FCWv}e~7U}fxvl#Dwp?l`j*6=_6QAmVCmKte>4v#D9{ zwt-Bp;OnY2mRjOcYgP4h%;DERhzS6ZNhaRR4HLnJSSs(A8KTRjp}Os?VgUd_lP%Or z88De_{%=RVQ4rb2}6?2j?x^BY`5q%XN7`Sy)y8W?}2fudveZz%mm&=ht?$E_STs~@! zq1Lzc^pDTYhDZeeAStb^>-$FHY9{VYw7w`J^^qd zGMDqq+gm#sBGtugC*a5!V?fZ?vLP^PtS~b;JGH5GL#rD%mm5tBSCA`ZXEGfc-J9M} z3&XS16NOrJTSw38x=BQ|tvkNCJy!9}XZCj-9xiXpI2GT#{PvvERDUB2!CFejVN%ouD>faUGQExK1KC;`5+kk&Dcq04MRLzm{QgZmvaJny&MAqlAOOZa?;__ z;o{N5FDYxZwa2ofmWW@)86%?qJ#!>9YIN2=J~8TWF+P{$$hhRRN;)`_Hs>a5g{o$V z$HyuYbHPxywz(rFIfmM@RfvKz2G-IHib9<&*WQYtI7lMco(U3u4tEGdZ-6${21VdS0 zaD4H^EN2J+7iR;>8N>RcO9XR;r1g{;2jq&wm0*-3%|9lr>^ zdY@`>5pYgpvygB=@D4E)THeG+kbNKAdd a job: which report?']) self.assertEqual(rows, [['trades', 'price midsession', 'price premarket'], ['price close', 'price', 'performance'], ['cal earnings', 'cal ex-dividend', 'shorts'], ['rating', 'milestone', 'message'], - ['holdings', 'pie', 'who'], ['marketcap', 'beta', 'compare'], ['history', 'buy & sell', 'pe'], - ['forward pe', 'peg', 'stock'], ['Cancel']]) + ['holdings', 'pie', 'treemap'], ['who', 'marketcap', 'beta'], ['compare', 'history', 'buy & sell'], + ['pe', 'forward pe', 'peg'], ['stock'], ['Cancel']]) self.assertEqual((data['price'], data['Cancel']), (f'm|add|5|{self.crc()}', f'm|top|0|{self.crc()}')) lines, rows, data = self.press(data['price']) self.assertEqual((lines, rows), (['Add price: how often?', '', '† indicates default'], @@ -980,15 +980,15 @@ def test_the_commands_of_help_by_buttons(self): first, second = (about.add_screen(DISCORD, CHAT, steps)[1] for steps in ([], ['p1'])) self.assertEqual(([len(row) for row in first], [button.label for button in first[-1]]), ([4, 4, 4, 4, 2], ['More ▶', 'Cancel'])) - self.assertEqual([button.label for row in second for button in row], ['beta', 'compare', 'history', - 'buy & sell', 'pe', 'forward pe', 'peg', 'stock', '◀', 'Cancel']) + self.assertEqual([button.label for row in second for button in row], ['marketcap', 'beta', 'compare', + 'history', 'buy & sell', 'pe', 'forward pe', 'peg', 'stock', '◀', 'Cancel']) def test_a_step_that_is_none_adds_nothing(self): """Data that is no step of Add (forged, or an older finbot's: the weekdays frequency, the next 14 days, every... hourly) adds nothing and says so: a report, frequency, year, month or day not offered, a day gone by, 30 Feb, an end not among the runs, a page where there is none.""" crc = self.crc() - for data in ('25', '5|x', '5|k', '5|o|261020', '5|r|60', '5|w|7|9', '5|d|24', '5|d|9|9', '5|d|9|0|0', + for data in ('26', '5|x', '5|k', '5|o|261020', '5|r|60', '5|w|7|9', '5|d|24', '5|d|9|9', '5|d|9|0|0', '1|r|7', '5|o|2025', '5|o|2036', '5|o|2026|9', '5|o|2026|10|13', '5|o|2027|2|30', '5|a|2|30', '5|q|13', '5|w|p1', '5|m|def', '5|h|d|10|5|600', '5|h|d|10|5|1000', '5|h|d|10|5|e|0'): with self.subTest(data=data): diff --git a/util/record_outputs.py b/util/record_outputs.py index 2c5cfd50..458e9cf1 100755 --- a/util/record_outputs.py +++ b/util/record_outputs.py @@ -21,7 +21,8 @@ '.earnings', '.earnings T1', '.dividend', '.dividend T1', '.trades', '.trades 30d', '.trades 7d Nope', '.price', '.price 7d', '.price 3', '.price 3 7d', '.price top', '.price top 7d', '.price T1', '.price T1 7d', '.session', '.session T1', '.premarket', '.postmarket', '.shorts', '.shorts 30', '.shorts T1', '.performance', - '.performance 7d', '.performance 1y', '.performance Nope', '.pie Rob', '.pie Rob sector', '.pie', '.compare T1 T2', + '.performance 7d', '.performance 1y', '.performance Nope', '.pie Rob', '.pie Rob sector', '.pie', '.treemap Rob', + '.treemap Rob sector', '.treemap', '.compare T1 T2', '.compare T1 T2 3m', '.compare T1', '.history T1', '.history', '.watchlist', '.watchlist add MSFT', '.watchlist add MSFT', '.watchlist del MSFT', '.watchlist frob X', '.T1', '.jobs', '.jobs run', '.jobs toggle', '.jobs enable', '.jobs disable 1', '.jobs add', '.jobs add shorts daily at 22:00', '.jobs del 4', '.jobs run 1', From f8dc446339e5ef4f7ee1e17229ae500b228e7ac1 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 12:02:31 +0000 Subject: [PATCH 24/41] Keep the reports docstring within 120 columns Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/reports/__init__.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/reports/__init__.py b/lib/reports/__init__.py index 2216a61b..02679cc3 100644 --- a/lib/reports/__init__.py +++ b/lib/reports/__init__.py @@ -2,5 +2,5 @@ milestone, performance, podcasts, price, rating, shorts, trades, and sharesight_warm and yahoo_warm (which fill the cache), each with job(argv), which parses a job's arguments, and run(dest=None, **kwargs), which runs it, to the chat that asked or whose job it is (dest); the scheduler runs them in the bot's process, and `.jobs run` runs one now. And -the chat-only reports: about (.help, .jobs), history (.history, .compare), holdings (.holdings, .who, .pie, .treemap, .watchlist), -profile (.SYMBOL) and rankings (.marketcap, .pe and the rest).""" +the chat-only reports: about (.help, .jobs), history (.history, .compare), holdings (.holdings, .who, .pie, .treemap, +.watchlist), profile (.SYMBOL) and rankings (.marketcap, .pe and the rest).""" From 745d997e2546b8f392b164446c2da2dbb0a015be Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 12:08:19 +0000 Subject: [PATCH 25/41] Fit treemap group titles to their block by measured width Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/render.py | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/lib/render.py b/lib/render.py index db2bbdba..4481d676 100644 --- a/lib/render.py +++ b/lib/render.py @@ -561,16 +561,20 @@ def draw_treemap(chart): ax.axis('off') total = sum(value for _, held in rows for _, value in held) or 1.0 box = _squarify([sum(value for _, value in held) for _, held in rows], 24, 92, 1232, 684) - char, n = 12.5, 0 # pixels a character of the 8pt tile text takes + renderer = fig.canvas.get_renderer() + char, n = 14, 0 # pixels a character of the 8pt tile text takes, at most for (label, held), (x, y, w, h) in zip(rows, box): color = '#6b7280' if label == 'Other' else CATEGORIES[n % len(CATEGORIES)] n += label != 'Other' x, y, w, h = x + 1, y + 1, w - 2, h - 2 - head = 22 if h > 60 and w > 3 * char else 0 + head = 22 if h > 60 and w > 4 * char else 0 if head: - room = int((w - 4) / char) - ax.text(x + 2, y + head / 2, label if len(label) <= room else label[:max(room - 1, 1)] + '…', - color=color, fontsize=8, fontweight='bold', ha='left', va='center', clip_on=True) + title = ax.text(x + 2, y + head / 2, label, color=color, fontsize=8, fontweight='bold', ha='left', + va='center') + while len(title.get_text()) > 2 and title.get_window_extent(renderer).width > w - 12: + title.set_text(title.get_text().rstrip('… ')[:-1].rstrip() + '…') + if title.get_window_extent(renderer).width > w - 12: + title.remove() # not even a few letters fit tiles = _squarify([value for _, value in held], x, y + head, w, h - head) for k, ((name, value), (tx, ty, tw, th)) in enumerate(zip(held, tiles)): ax.add_patch(Rectangle((tx + 1, ty + 1), tw - 2, th - 2, facecolor=_mixed(color, bg, k % 3 * 0.14), From 81a4c0262a4a81a7ab57f1f524c9b26796957c77 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 12:10:39 +0000 Subject: [PATCH 26/41] Use a real portfolio's industry names and holdings for the treemap golden image Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- tests/golden/treemap.png | Bin 5582 -> 12940 bytes tests/test_charts.py | 15 ++++++++++++--- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/tests/golden/treemap.png b/tests/golden/treemap.png index 4660048a0d4d52381c4b141d87188548fed53dd2..c67b1dd1d7c8b0395c3ff433386502ac46b77d0a 100644 GIT binary patch literal 12940 zcmV;7GIPy|P)003kN0ssI2j?}E!001&mNklfie1gD`X*#~5QR1F*GL z2$7~~{?l4ZLRf3#I7Y-YO_@24V-fw)07#A zFv~J#-rNxRr!aGVUI+oqQcA6Lep&vIdy)cNaeor|NKAxtI^T1GiT4Vn$6$* zt>1X?frFPXUmhJD$ueCi6kOM>)vDv;<3yAs$@uuV5Ta16IU5nMR_|eeQE}b90lElRy5^k39SAw?FampVZoX=R4o| z>@WO69LF!b@WQYC+UL97fDligKJz!<_}l;H_r7@g?D^Hz)t~x_kA+e4zx=_!K6do& z!J)xNjy&}E6OVVh-L=)VhaP_D5C8Cw*4EZ0CntXX=YQ@OKKq$QqwyDC`tpD9E1w%1 z8++;Hmw*5FfA6(dUwiuL4<%`OaR0t%p8eLZ|N4J;d3N^GpZe6LnVJ6n{+C~V<>?Q7 z@W1_+|Gn$F?*$rPC8DL2Nt}H6=?^~f#0R1{dh!F0PfScmDIY$3Xk~Tv)alb}YpcKg zpZ)gY!lDpjZf<^QY1wg|Z+`Qe6BFZ~|H2n$F3k)L5B=dE{!yA{>+9=JJn{IYi zmABq}Yk7J3AOHFnPM$ciXL|af!-r>PE>`*~)mklzqI2iY@87p~YHD)t-s$o2@m9MX z1mV=4$@AyWpFMT_AODm8Xl`!)kw+f+^FRNK-}&M{UtC!H(wDzdsZ>7ytN)0B&!4}L zW!d3FhkV~}x7$GwOifK4df;IHz`#o{{@@dz`1r!YqUU+fJoD_QKmDnH@h|@6^78Wk z{=fa}Kl#%?J9_kJp-_0w5!e{>=wputLHA2v{>zgmPYw?BzxCETU-`;kc7ou{nKQF< zbB`Q3^2|4%ed*;_vMl>Q{?GqwX=$m^XlSiRhll^-OJA9}Jahj1g;!pA)$?2_<=WcX z{{8z;o;)>kY37YL-l*4`t7~h2_UB((U0YjRTzd7j*O!)-FI>2A>C&Z{naih6oo+Q- z-}=^f_DoG4K78m;|LiYbeDNjec%5!{V4(lI-+S))=b!(=7ryZ2FMs92g$thNb-Ufu zXU<%nxxBKn^4;%#FNz{-%`?w@>%xT#g@X6J=bmrWn?VqK|AiN;)#}K|$Sbe>aDIN? z82hPDeew_f;D5E&AigKmFmAJ=8;u44x~@Y+Q4~oj3xz@yMF60bilRse>3MDxMg9H# zNfK*q5HX5l&vkSA#rJ({3=(Q>yWNiGc~VNHRHxGsLTnmDN-1MZZXxG(mNq7iqR)Qz zGu2w{jaOeA85;wDFpQEkVQZgw{P8EBdh%;u`)eTt0EA(fThY1xj-p6wQz{j;)^Qwr zo<~BYN#eS0mSxN!rLfk1;u9Ztl=I#1eXmd`Y&EUlGX!Qh@W2DrwbhBK>GfJQ2)cqq z97o2O(b2J1tEsaLi5%B;9k<PI=4Ue>1jZ&#^e(v)4#ALm` zuCq)^RVtOjC>$IdYBd|V0qi)A)*6wRZEocuVz=8BLgYqr?&Rb)c9JA<9G6O^O?Pgq z@t^;iZmbI1$HzYAfQEZGU7K`s4H|dIzg@|dIxUOf61qNnz9QT0-4>3a=M?=FS z^;)&l?Tn0!Wm)R`ei(*XmOb{wQ{7HS2yy1r@ia}NC>$IfK5+2CBu$nV=O?E2Oit}d z(=?|OA{C0oW~1&G3UL?-A(Z1Z8}$q4&Pb(lmdaIQPTibl`G1sBuIuJ*($@2v%4Mrd zef6KNobi4C+I!zaZW04bOzqJ+GuEswFBJ*}rJVKJTB%r0l9(-*N~I);wbn_J)Yog; zXunWsHX1^Z=lNk2I!g7GE7j_%Agb5vX_BOA;ui{Ol7vB5N)^XZ9LM!qwNNY$4h;w0 z4j``Aszi7_3T@@oD}W4MErin?wow#pqT+!BJp)O*cw?JQT=<;)`~S5A_D`M%o};oCT&DytC0;La?(|m2~iw2 z-8Doe5~4H$R%dZwvjj=9$w&}DPj}+RksC+7Ux=(Rwc6Sy40%fj0~{&ipo41(q!ui| z!6I6Q$n<_ifW(3Yundp(;d%-Oi|7zGGO!FE8bDuQV4$me(-@gmsjT zO~bTy(H(o3O(wib)?RW3_M?#T+F9xMr_BX#?1&DkC|s0|=~h+$bm!c6(%J?T8OwHKv9pg50&tLI-@@~AsQKX zGz==3^f9q;y3GfR*vMF0@Fg@;kO)IA{(rMNH9!b_2izV#KrW^rtFLC68+ zMC4rP!eJQMmMj!92{@m~bFV^SA z+)_P@mXq#c+)k_}#FrcM50!==C=8aQvmAHl;^u(za~k9D_&z{nomC$H71cLklP*eU z8)KnRY&5V%BDMxeGFwDsYf!oX4AufLTZ6&@YXA|2x)-hpf;GZ(E`R6Kct$y%u~sv< z1OSj(Py~^Iy#*X{@SUc02rPrZTs>oGI9Ix~p{-*Bggq@T5G(^CsEsp(aK6h90o}R= zQ3P`#k4?4r^qVNf)ZkRV9w?Yo%dQ}12FoajjYUDZN0+ZE51^qVV5W{v?dj=4v8ywO z1Op4C)EbFo*t#SWt0j^F&Q&r25@IgNXQSrduANgzyx}Gx+pO}$=gWtG)+XH@EH&nh zGg~bZ?pY33Oa=tz4O`_l_Wdph%sO!^E7-TmTuZph?w7HnW^XQZ+IWT0AQKR(tUn?=4-2G925+^=?9}t zIy$J^qyz6_3=i2DZFQE^0i0J4)$JJ@AvHSK7sEHsd}Hl|9}sx zGkDhrE#4>a?H^wM5GnO8r}6s)z8%SIjdqHC`+n-Ps!(QY-_5f6eFERsTazjcJ30!> z6VrAl1o+)^$oB%r^-zRfUN_uX#0YJcFudxZOb9_5ez0{Nxj?dRNmM^HL)aIEpSe{6Mnds&r;2nenDOJ3i= z;%m=E^(ruz#`lm^+UVVcl9?F{=TR}F-p}2yg)_~@;$~KgAiSYKy88r11WAYp+Oi-R z?UVJuSib32yC)kBg*4q}x^h{K@6W1>C?pV}SA=#9Lfo=&VJ4}XGw0YC6r#JfB%}g_ z&dR)yY8N{OnOVAhg|R2HU`@DX9k&W2k9HScNxN%EHxqUUtZm`QUeQ<3^$q}tvacK6 z`0^?Wx=#eQ3*#n$;5|I$hth_?LX{C_6ij5+2#{Sg9(jRgv`<>EcE~0~#qriAoXz98X zX79i_Ab1DHZIY2AgJYMrho$Z{2Qx!6v{JxeNd$=n10yBjNE|JTbF10?=3wP|2Om#n zPqEIt1CN^ax{X8AuHT{Ox>5OPH`L&66*x%<){d8?XPPsex$nFE2g9|aI%uNAnwLpZRAiJ|2B-%%Oj*$VdaDdb!p32I?k$uKAcd!1 zwMpc~Av*4l$e%#~8u`F)G|zt%&<-5_T+*6Dq+t2=q_v3RhGa(MBy%5_@E_V!|HiBR zKl7vsVse#@lHk-FSbIJ5?w(lD`vzsA`M{7^58_Wh;-6hfeMLcPe{eq5_P&|Z3__T8 zz5Ro~MjpU zrJH>QgCt$=fU)kLVNvpcFkN2vADqnA+sRxNot?e#d!T~>1mRjoTZW2HCzsN5tC_Jd z)i0F5%#AY-lNv z85Wyb5z_s4hp&8dzWUdj)5#$yt6NXmVJJ%Sg|R#BqqM!+ivTbn0qacbY!xjk>A6!1oo>3iS@11JIp$W1CYe?15i@$XT^7XoO8F%Ld6NPLShonaVIxD;E> zJ2A1uEjfP-2)KK=k1z*BdyK*WH{+OlgJf;toNfH`_vpxJ9O#|JB`^RRAaSioFbobt zYN2!!i2}FtYI?7Ax0k;cqDgmV_2M&RLuXA8iz%RKOMh-{;`F zCn@i7Wy7YCwapV{Yf(AQ-UI2%Wi_!6Nyve*Y<)RA_Tu{~J9p90fmQ}GyN5zC_e?Xs z#zkf8j82<&-5J{}`-Zx2Jtqgp)X*e49>*cP`#L24{Ua~|Bo^LoTbWwEm+e%vg~Wm& z$TsnFW>ju)?8V#vn;X< zY%*hwlun+W$IO68NX%C2RFEW+*V{E$2yD~9L@k?k?!IY_07z`~-5{{lnw3tG?rl&i zuhL(O`MEW_JzUt!py<(u03r)6fiHSA&I`b2+wj%b`uoN*n)4_nMnH}1SUr@@EMM9{df&HDJ8iebF z0+A3A0V9*SBy65elz$y0N~`BX;joVGf$wB>PG(i8}CYmFF1#kt9 zxEqbobA6AGVcoUQU)C6k`vsh9$xM^XxeXejGg z-X8%mXvCCkpH_>WbJ=HyF+(tU@By{fIPN}j__K2IgMwtHlfBbFP9(ypWo_118rLQv zlD^lsaO&$T?Ny~zng-VBT%gMhu^=~PsMEj48++I$-McTM*#cemr=JEkcg(>?Is@YD zm)V$I`JLC}^WAMViBOOm94$rvBOe9;S?o*JS4`9m<}QF#E^$GEFTf?p14D+vCs2yj zoIl)f(y~U2IusNN0DzWa&j0{R1_rD-tSk*>tgKvjDCzYCG3<0Z%coEOO_BxxY^?Fz zvb8#jTUw{3;*gZC)@hmsNHo^JM+#@Pc0Pz(M1n|yD!>*F&Z(eTr);u&LSQzW#dpE2 zVZ#{hcB3#`onfvWuIf6+ND_Bp^loZawnq9DD)h0=K)J<(kA@4Ej5$lvvCNjia_?dU zung9sRc!Z%t*1LS&os0Mm{<5C0APlqu56#~wf~?50Fnb?uFr|cg!-xO$B*p&WNM<^ z+OpPc^c*8=jb%e55de|$BtStD5R~+mt7l()?cYl2GPB%fwe8S;?=C4}K!m%}Vhm}w z&q3^e{oj91bc*zKe*vq==c|rL}A`(R6RtStlVrz}X zyM@4;?K(GR_uL&m?p!=6yn>0NWPO#C!$z~Uz?PJ=8&hMWC=9JmQMvBO6!|3!7%b0~ z_CH+M_i$AI0pAU4Uh52rSX&?vbT^6eZXvK?Foq>zW|$eIxa0IZM4M&)=$_G!eP-^j z|DTbMe2SE7vlNABoOsoQo!vY`Ab11}jINuUe_JMDHoIT05UE1X) zM%bxDi-D&HrkinV-@v|R)Ld#Vp}-rXYYKeX?2^ zpKRxO(QNCpNCFUn^HShApAhCCFU34CfVnN*v!bttE;r|P0RRcI8=dnIQCln=-q=%_zFePKX)Nz(xLFHAfH5EhWEm@k`Ogji{Ei8X0NV7r=Ly7ATL2J*;tqH% ze~*p8c48Z*{P2Ww1Xg@nY-qk4VxO7o@BF|-L0nrxp-k9`R^|YaZkG6-D@Vf+XqE&F zfWqC-pb{(t2w=9FH9=y)B`65M4L`Ub!65)7*6{G~!-t0t=Nz2b>|E{Kg~o*q%L>f)K)!uZ3Yq0IE14jOGG1*P0MLdd-Eo>NB4=sZs3Hkw z1_+Ua&C-n|_?vG&k6_g8DGRH!D=k<_Ff&OBLO95&<)baGXP8(Z#LX2 z9a{hhQ%-c=5gzVNiacUorP6DX-m~I%-TDzAV<2V{vN$zO%CR~tPwwlkEFe+RZiMwJ zDXAHXU}TmsDX`k2P}VAx3>GB3vHS)j6A}VWmM3E!0|9Fwq)gjK#~&pUrxs3q=JC%g zwiXBdflKQ%v#r@}YqS=Y7IdMAolCZ_UmSfSstkyC-U#~##ClEp1yT-X=^d1=SO(vr z=T0`X;o-8Ht;H+NR0-TW><#rfbG2CA7Q-Pzs`)dI`D-0Jx2`|+h<~Y?O;p73#q{#J zQFkSm?MgpazCAflL?(?#jyzQw-xI7Y6-OtFBje?XeJ8*2NBO>CosR68c;L{1c6C+y z1#k;P!y~VM^ZQ_ot5k*h*Sc#3x8O-{zCE9r3@D#^!VGMzogUab-<<#0zK^|d>4lP8 z`t;BP|Ni7(tv7009b_U1yEY7T=lLEuc>*tTRmoDol zTz>myw_I62cU<~KZZ~pC%gXKhz{j(Tr@Y~b_2WOx;yCHHfdpIp`2LT7_4rpmvG)^? zjXZX~alY)7pSko*Evz9T5p~mUwOeK8KY8;{8d0Miw}CyXIMv|s)yN0}0A#w?0eD;B zHm_S1#8HNlu7cdP21F193>;-h$jqoD06?li3Ls>*Bpomo1c9*#0FDA?Hu<<0((QI& zTd_G$BrSlWu-7>Y?bpwJJ+aPyS_Vb1mcj3~D`LU2wUF5!B*6fdakR`q0*M9$NHpZt?+Dmm;8a7b>CrvB+~j(Y zx4HqtI9%dXLpuT~@bEaaA}Ba~W(@$7kv=gSK&pWNbcXwfd9{rLMJT#_rV0Qc1sbC* z%fn;xY>hL!A@I1HS@99nmb+>9O5)D7J&-Ij=FQse7QcCCc#JJ*W<(%@&F=4a4d}=p z6{Qw8DhCn4+~<1V-asK-ePo(sABQmjpb}7Yx!S>k1CHQYfTLy522umQ;&u#<#FEQr zR_vRz;$wYWE<-bfk_*P7tH709?ciY1o?oNsF&r&(m~y28f<(o$OD*X0AV_hv42Wj_ zB<&jmW8tYXI1)Ss#v(J6J-c@lE>`W)1@I&=5L`W!YX`N*thk3)ZzH7O>rv)b@ugda zD(Pyh_(#&E5sg)E&k4VugBVIKBpM42*8?0bagYKtEY<-4gg^w2QXm3C=)|09e0vs@ z;98d+9OZfdzJkm^nBrI;YYi`+17pFJQ1%d!&%^=|2a6o0*o`4bL^s0I%iIWXsvipu zw_*?oj?CpYr<$`2kijwtx{*7EKy9h)&XSLWuq|^}GFI*<7R`4j$;KEgd$4ye76yRL zX099N+A;6`(o2qF*qXW}5cBC63`hvt;-S%;65LI=9l_o~&>M@)VFN+{;Jj}c1B{mO z@E912L|`m4!{o+(#~##!PfUYla3x=^L9(dXC9)$0NY#40lHReF@r04p~=kk#-Bk6eDFhH8?u)M18%M6@~O zmH~v7a?Rnc+>yXYWV?3V%$~mR{W$IHzUv~hbyQ(~`RHc%TX{buGYS*8lhxCJH|#IR zfKny33y6Z8;*N<|3_##y-8Gc`Om`y(DEEvBH)m79bscQXhaEBGGmCE2n-K+BYaV>( ziPGXwmZnDQVyV2mw2*t!LWn2|Gm|1*H#f9^{YT;JZ@vBRZ`sUYMiO~K);0q3)};Kq zd00bkpF8r<%~z&en{<$*@G934zf9y)t9buwH4~j#W=TXykxmr}MS^e0O#Nen#=)`i ziB6~ODCdEJ!=C5o)96nedmD)JxFPOntTG@-r4;O}#$hL~Ybz7y z^Oec5mBqz^Uogfnun>~1G1e;O5aqp(kb!%10eI(-C+-{|>6JKay9*JlWwTLcaAoqD ziJH2*X4CL)yFSQZIg5dI(c}*Cs#K+0+qU=WFwA|fe{f(dG1-yfsn-@SxKceaws)yh zzd@l9A%66ye!4t5%*N(1f7n1oa}+x7cS%HH7_2NUmdfR8N0Sp#r_~x99#QJ5S&)dB zWm&yeE%#OI7GcKBQmA^pdi0$)rIcIJW(m{Q5;{fJ@!jWOtF1`^1#tH6tdhQ&>yMo@ zUhXueE5jG+D`Ulqw)Xt`@=b5_Jik=x(`gC-c8jyamWe2*E)i*Mlu}AL)^g5&`Jb${ zLI|mp<2Xt=+X#$^QYqy)Tek%>OR1z(*NA9lRUUN)_j4BBi^TV|wRC$m@VRA^=K%i* z+iVG>R~F74Tb=D@2?AJVNy1mojn`!Jndrv2uH$>IF_r-k(REb2(;u#H)rz`$Eg-`iEdm{~Jcs(f<%@$Xzb`b&pC7@N#myB4%BHdb$10*8p&nCX$B zvC&~*o){lpTv|PQeX`k~WjFD)!CODPFS8#6jOym!xJ6vw{jC20x_X`0T=&gaib zGY!mnHc$RDr)!pJX13M>0wGyznX~I5S@*QE;WivpR#F_E>Z`Sqho&p7AU!@)R}ydf zNt$gALrJ1kYs0Ka5Z-n@A`?Pp^udAAe!nzSD3@LTa&zscM;{5Yc(K!PZrBZ+83c(& zvo$$Bnq~UcH;#@D5B2qx>WvmN=MHWj{I}LlPK9gmpwU*iUy(14D zPSPX{Vw(Gg@tMZWO*8<_w%;lE(jmlJ+}T$it~maDx1Ji^&63-rF*7J3 zUz|Gw46iR;Y$s7SjV{$!Bb{vw@x9^xAdI8r^|y{QbEfrLt?no}zp(7Nj+7$jMI~je zR)6N(-zP!4uuzI}Yp)C6t}3En$B8OMn+#xHnFORG}Kg6~)Q%1=CYWOc1tE|sz@ zL&Qe2_4s2)jvqhHSB-eTzlUW|0xh$zgs~i@HlOiut!Wd)**9LDD|+huV(Y?ETS;QR zVekquGiI#N_bUgUTIe-OV87NT>&w>b%}p{`YYVf+v0sy}6#Zf#qHWJ54z0oA1XTJ5E=ef;h+x1*yOcX`y>rJh%%<7W2 zEpr)qbEt30AC6P|--OLo1 zNVT~8c5N=m5??we*B2*?{ihmB4^~DOgGOw!+vqWv(^v_?%pmhr8g`}P-I{%9v>6^A zoS9vC_|U*)f+z^XoT$}Wz1t0Fqpt3j32ZGRp*D;NMNh`5xg|0a zVJp-hexPrz7B8&FuA)b#iVO8*D+?oHlA0$U8eDC}tswp7AK$y$hydZ@QhR2l?aCYR ziX>5wgRjrL<4ctqEs##s7lJIUI6ed4h`J0nXbQobrs)+xnXhYM@HWVukuV6tfq}}U z%kzuNtBq!>)9GqsdcJWF&#;c;3=Z~3aop~78;#b%1AEKm((=-Bp7@uWR{$WTK!hYo zh)4)QThbkwSxP~4TeV4fr|+MAa$>!mj1BmN7$!PO^&97EyWGjlAP5)hNf0;Mk?J5I1=Dif(>|`*okFO5(vRSUO|>+`oyVoLWtZSl0wWct`MS> zGIxDQV7D8*_QpHb+Be@mo|9^=T9Z;rDUfJeHasFqDRaPN>u}6`6Kv&{-6!2)lt%wV3jWNdL0fyWl2PA8)AOc`x3=tV)hy)^Txw5x?gb>W!Xwd3t1_9!r7qwdAdt&Q`)n!qrB|3_@TKX(3P>Ziktm8#!?dAc#6?{N)R; zX=~=wsGUYL?b=MIR*(+eux~yl8e?@aQ>rE-;EsJPG%#7JLS{ffL%&v8R z=_7lCILp=W3&&UV?o#g>?Q|e4}{Vqg+6UfW{ox`+jKv2B5d3uEo-)pkeM+I?4nFE13mnat+j?3 z5w$i%M1%mSwRRk}i7%E)<$;0WEK72W)!N=Ba9k%%6Kf4@Kv!E#JWjLh@kbvT8XQ_G|eU_MyK~owA!6| zy;&|7iTY;e7IRDy5skI`_fGBGJ5^h63=Q^su4}DrwAz3u1@%|@;yAAKm91sp_g2@c z0N}dL-29?3RtTZB&Mh*}b8pj4LWE5BYG4Yk%#Al8?t1=59_`;A@W;)mI4th@+%g71 zbYl7PTw|>hN0P)^rx~QNC$|G(3`|JJY758f3uWa4z;ladO2WC)rI0WUlgqOUzVBVU zbh%h8`o3q3xjeTpG}v#g>A}&=j#A4jYs)KZ#X{l4sdG~kW5(J%GhR9Bqd)dktJN+P zywhhc?wK52SzQamV0A?!^?;X6r7lP`n~mE4@zT^De$v2u|SJD;y*)sdmnJ)FO z#qE!bKad*j3K?hVv+L)=Ep~wKdygJD{Nl^6KlbQD(^C`8c3Vk#;^es?40|eH>(P;+ zvC+ZfC(b^48?C$@TDe>v=L}5lJmO!j>4}Q441-1b{<&Bi!<5~h(UOEXJvchwsMf>I z#p?3oQ~S%VUk}=VS9An!(74|%tjFEC&U%m~4^_qs((SO_l#^N;$5ErB!=C3xaWcQK zeE7kGLqh}YPFJa(?^G)K1>Y<8m4prKk*Z99X&ZSbBX9$5zTztk@;N&{CDy2E7x#l?0^pkhXeta zT#Pd=D#$w^^$=sb@$b$fmzI0a+`ytGQZwlO-HF!_@NbX5ntQj>7&O^YON8jWa4ef zs+2C~8S(cZqXU&oPVuG$Mxjvo42}mS77Nb z5ZIC6Do_#}3Don0dDC%6nO`Q*xe{T>fet0XRp2R*77)}%f}>=Hf@74BQYz&j10qIo zf{4EF^On(=LLdoWkrCbNYnXKq}R4H8-^b z4v+8a-D5IfzGq2~2)phI32d3UH?;BU$PM?RZwf75Df+&18s}vVxAwqZR?2nmVjwca za8;t*4J`NT<`iPDY5xv^*R5x-0003kN0ssI2j?}E!000%0NklFbpFsqvM=&0OFimYqi#hXe}*3%i{4E z00co`tQDLaV~nwmBb8E!XpCWug`py%n&)Yy8Dqv6t+fzBYaR8zCaB8;jEGw6M;`g7 zhadd>z?Oj*Uwmoj&YhJ?W$)fS-}uJY4;^|#DdqY8efJ)C=)upQI(@peHSM~tF{Y!v zJ)6z8wzjslr3-~(cXwAJ5f@yzt{aBo=FOXwQpT9>?(SqVSuU5$mC8d8e&KVU`^=$3 zZ^Ywq&N*kSrL|>lcJ|9(`7$DwOQnwX_Cm1;01tfX{x5y$&t84?U@Dc0$Kyc|^!4>1 z;@7|Scjo`RZ4H`m>+kzJGsrcjr@2J+uGz z{r#IaJ@Mo(9(dq^Sj_c3KMccv{kMPr<~RQ#lWBYW@h86W<-aIbDo;K2^iO~CqhvC9 z*GE6n)!8{RIwm+1Ql32d_Rj`?v}scxW9-}C{#HEZ{?`wFc;}se)W4~3GCTd9@BHJD zBggK%^Nz96vCn?y)8GHkN5A;RhhKW><-0%jvChtpQmK6M?YH;t+4CRY``*QgNk_`2 ztX?E`xdH~x`N-&)>$>;cefRWqE|baZ*s)_KpZ99D+itxTNF6zP^z`Y$TGbmJ8~f!m zzr5$;cVE0Xv2FXd+xPFA&Sv|1`>NH-kt0Wkhev9j_wd6HZQR&1(7$=-&K>!D-m7`H z-F7S4FquqFO=Z*R^!WJrrcImgJ@AR&{`M6n;~O??=I zr_P-l^t|emzj*5Kk)xjHz4E(PCnhEb2M33SM@C0Sj~_o#ES6q*<#)B3ci_N*=bwN6 z@yDN327KR7CKH3_&;R!iet7@=_YDt^JpJ@D>2%un{lUTWLqkL37sihqJ$n53$*#_h z7hih$;K76K?HgV@_FMml4?pzS|2#H${=C}+ z_;SK4tmtB~m`WyVz8`blN~PjBj+C-mtun?^sTAW(g{oGoNhvw!@p!yasS;5<9`k)) zDV0bhlv0eb$PNbpV~k&|esKT3fq{XipLlXZR~P3z2!dM81E8+1t}lGy^Upo^>r$x{ zF^V)OA_Abb#`pbLEG`7ERH{-6*L8y+;G83}Fbo{WvDWV1y=&jTeNR653(k2H(f0s& z%hv64v-z4=lR_$`g1~QY?+C-NR;vJ@luja%s8q_yWU5puT5F}0ks+OGY3c3VG<1G2 z)zYH1s+3DoI>}T^ty<~m?3&HzOXX4`9*<19MI?+d<#IWhOmfZ_ou7BQZ3};qwU;?; z(U&j`ecw-|QcXDD&B#K;Fbstd-95b{L+5wyz6FtaUahUIy;?2jrn4J+`oh3ZBvYPO z3&Zf1Js${s4*-YGpPQbV?Ck2^vVBLXRFsahX;XipFvq#zoM~+`ZEbCB8{&zC=Xr=I zgw$G{JbpCdo5^IXwGmX&pkJ1ZF;{$9Bq?QN&o|+G4;{>fXz%DOmx`Y6r&6uc(^H9f zLK&?>mCj^5&m*E*tyU@)Jg=5arb@+vB^!z5a;enXn!Y%G!5UMoR^#!6wWd-obIz4g zzVGK}a*VOg&aN;BEZM@`tTCou(TP8>5d>febomjyIYcy)RX~=@xHcNCHd=A%T0kL$ z(VB4%1lAhP1(A)$TqIYelmMu;S{ijCvW)TQLPWIIATlBv*s?8`%hswD#@1S2B7i0c zEyVaG=Rpx5fIt8MTF(Gm!;>96o(Xd2z$&L>2al)2{IKx~M3!-ftOjJrI2i>@AQ!MC zOt@ec!Lk~3=@F$A=lq?zc5TpYcVJV3oi5pm2996|HYYH!FcT2Nd6?Iu4n}5mE!Fp- z#K1?q@wY^>Q>J^7J3)R38RLoeaBft#bQ`Z=gR0xLyY%)me)cqzF(3;72>33&{|emQ zn24g$YwhnNcnbDs=&$aaCV3}301skyn+oK0DGedDh9-ekJMUz zHDzZ4N(-C`sA?d=At~shNBxK;MI7t=qB$Z0*fqqM00csU&-Ya9*f;=)q*eL@56RXZ z8&;7!L>3tbWTXQi0>%LdfFS~rMJAE48!DVYjEibHfBWPUx3yr^%L2pk3%eR^TqMjA&fT3G21{%Omu`)p7Fj7&=cxMTWk3e&jL3ra$tzYgkeQ<#({B$z!yGi0C2=;k?9%$j4;pfl6BdP^{j7d0^9_+Nlk#8)C9OmO@N!! z1h`2}fSc3=_(q{9W_*2V-lF*$8;*zw2mwheU>vLkhPcxFh$g^|CsZ955XoAnt*(`-2T)P@>0^CR=*k~r*mTh}vYa0Q9rS?1Sojdkg zqHkO2+_B=RHxc=@%IYS-4P`)cAgjs+RW0dC2^pU|`dabKVLvy)gapJDXyPWo4dnhZ z$1R;Z7HyglIYpJ>Q%pM5(X#}=gly{IRZ)$XWj3#B!XXg~2}`)4aDXgI2Up6v3GjQp zbFBaojEwX4ir+)O{0*teTu!)HV|0}T1h7N^zz_`)5g>rmfGfYOMJGmD zm0o__)Jn`vAmhLoSla}6dD0d-Qp{M0OB5WukXm9aCec?klGm)!Am#$dR*(J`kpYmY z7Qq-`967sz78S1@zzm_L>DC^(r`MUS+S#g!NemTKLR$+-8H!146#0ipq3>?pi-@vo zb8z9T^(v`5?pOIKL4FM*Q9M4ay6N=&zU5V9zI0GVJ45h#Z zdSgV$g{hT;vD39vZ!pKDFetwKb5qQt<1#0H18oMd)&Soz;8p?~m-y=W;KKt>C7?{4 z%~s8YIn!7=mt*JO2}1(26-HnOH8I8bzQB4i9Ei$*Y?R&`i3M6(@L^qUSWw;v~`1ogA`UXn4x~H`m|*umE=6 zsiaHG2ImV{)H8J|Dt$sXPfk`NpzDxL5`-2w1KdP+9AjZI9;IcKE zF0=$L1Bo{uz!o40^k~t2RxynX1*7RQ1PXf6Fjs?60|5vI0Cijn#{g;uDwb+(_mYdcg!8lQ!ZE+7Ix=m8+0YJjPjeSH|b5bBhoO#lpl z<9ywuMcZi=y{rYM4&H1cnt&vCr;6-1#{rQfkIEDi5Jqo2>0(>Vgdz9=)HMg6t{h_Q zdUXN2da1>Egpnk_*>o@g0JSG=Pa3oZcBujuA%d}(Nl+#MpWF?M)pwNP6#_I@0apND zSrI(~#;8z31dx207Knf|Dtd5kI_d}Fvj4~e+4Tsn1t16%yS<;?y9+A8B}E%`c|$?3!dw8p^|m+A{%Qwxru^EpI-q_a#5(Fr4Yb0!;!iX{c2g@ZDIRyT0n zBt%@1D?=8YBsy{M6_k9aUPa3yK%iF}VDJygq_T6fgsYk%Y z=IIZ(B?7<&lWN%vYO#8+gRXcVYl6WlF|tIKD4L*+5I{tMtiJ3FSxW>&3|RyWT_^E8ID7mOhO?ZY_Q{g@RxAeBOoy||SCt8d(hc2EsWR~3- z01-W{?%lrE71CIXh}sxHcx&{WM;2EPU?LD4t)-0_XRfMal84&HUEzl|Pe=cX_ETBV#=Z>If1A?FCWv}e~7U}fxvl#Dwp?l`j*6=_6QAmVCmKte>4v#D9{ zwt-Bp;OnY2mRjOcYgP4h%;DERhzS6ZNhaRR4HLnJSSs(A8KTRjp}Os?VgUd_lP%Or z88De_{%=RVQ4rb2}6?2j?x^BY`5q%XN7`Sy)y8W?}2fudveZz%mm&=ht?$E_STs~@! zq1Lzc^pDTYhDZeeAStb^>-$FHY9{VYw7w`J^^qd zGMDqq+gm#sBGtugC*a5!V?fZ?vLP^PtS~b;JGH5GL#rD%mm5tBSCA`ZXEGfc-J9M} z3&XS16NOrJTSw38x=BQ|tvkNCJy!9}XZCj-9xiXpI2GT#{PvvERDUB2!CFejVN%ouD>faUGQExK1KC;`5+kk&Dcq04MRLzm{QgZmvaJny&MAqlAOOZa?;__ z;o{N5FDYxZwa2ofmWW@)86%?qJ#!>9YIN2=J~8TWF+P{$$hhRRN;)`_Hs>a5g{o$V z$HyuYbHPxywz(rFIfmM@RfvKz2G-IHib9<&*WQYtI7lMco(U3u4tEGdZ-6${21VdS0 zaD4H^EN2J+7iR;>8N>RcO9XR;r1g{;2jq&wm0*-3%|9lr>^ zdY@`>5pYgpvygB=@D4E)THeG+kbNK Date: Fri, 16 Oct 2026 12:11:51 +0000 Subject: [PATCH 27/41] Treemap: 2px edges and the subtitle beside the title, so the tiles take the height it freed Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/render.py | 20 +++++++++++++------- tests/golden/treemap.png | Bin 12940 -> 13089 bytes 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/lib/render.py b/lib/render.py index 4481d676..9d62351b 100644 --- a/lib/render.py +++ b/lib/render.py @@ -123,13 +123,19 @@ def _shrink(texts, room): text.set_fontsize(text.get_fontsize() * factor) return factor -def _heading(fig, title, subtitle='', subtitle_color=None): - """A chart's title at the top left, and its subtitle (if any) beneath, each shrunk to fit the width.""" +def _heading(fig, title, subtitle='', subtitle_color=None, inline=False): + """A chart's title at the top left, and its subtitle (if any) beneath, or with `inline` to the right of it, each + shrunk to fit the width.""" _, ink, ink2, _ = PALETTE room = 0.94 * fig.bbox.width - _shrink([fig.text(0.03, _from_top(0.957, fig), title, color=ink, fontsize=13, fontweight='bold', ha='left', - va='center')], room) - if subtitle: + heading = fig.text(0.03, _from_top(0.957, fig), title, color=ink, fontsize=13, fontweight='bold', ha='left', + va='center') + _shrink([heading], room) + if subtitle and inline: + right = heading.get_window_extent(fig.canvas.get_renderer()).x1 / fig.bbox.width + _shrink([fig.text(right + 0.015, _from_top(0.957, fig), subtitle, color=subtitle_color or ink2, fontsize=8.5, + ha='left', va='center')], (0.97 - right - 0.015) * fig.bbox.width) + elif subtitle: _shrink([fig.text(0.03, _from_top(0.910, fig), subtitle, color=subtitle_color or ink2, fontsize=8.5, ha='left', va='center')], room) @@ -560,7 +566,7 @@ def draw_treemap(chart): ax.set_ylim(800, 0) ax.axis('off') total = sum(value for _, held in rows for _, value in held) or 1.0 - box = _squarify([sum(value for _, value in held) for _, held in rows], 24, 92, 1232, 684) + box = _squarify([sum(value for _, value in held) for _, held in rows], 0, 58, 1280, 742) # 2px edges, with the gaps renderer = fig.canvas.get_renderer() char, n = 14, 0 # pixels a character of the 8pt tile text takes, at most for (label, held), (x, y, w, h) in zip(rows, box): @@ -583,7 +589,7 @@ def draw_treemap(chart): lines = name + ('\n%.1f%%' % (value / total * 100) if th > 64 and tw > 5 * char else '') ax.text(tx + tw / 2, ty + th / 2, lines, color='#ffffff', fontsize=8, ha='center', va='center', linespacing=1.3) - _heading(fig, chart.title, chart.subtitle) + _heading(fig, chart.title, chart.subtitle, inline=True) return _png(fig, bg) def _code_version(): diff --git a/tests/golden/treemap.png b/tests/golden/treemap.png index c67b1dd1d7c8b0395c3ff433386502ac46b77d0a..2646cb043546d8ddf85055473ab89b478b1908a8 100644 GIT binary patch literal 13089 zcmV++Gv3UJP)003kN0ssI2j?}E!001)RNkl$;SX5%A9H;E6 zl#@Rk#aSw*9VfOd%aSP3k{wbK7m*+VVgO7E5M zU~sEz>dkxa-uK;mzO((#h65veVbeiG%#4TtAR>sk`CrK21t6lUKHa#&#_2?~{&mEa-WQRp7PfIm8_Tz-V&*uG5iw0u5sBlNlmZcJt%!J@mp{-T z2+}lDii|NtWUXap$8pj$6#*j5vP?uaHbnj_B9gzalmZd0wJ|3DT7HrIu@(viV@&={ zt#y`Vh^Uk@#sDBQBSOBsAPDkPj4{^Q{LA^~V`dQ+5o-)7mG2Kk1i+2gIq~xE=hs0* zSl^n~noS!NL?AzFV|#6E#Es*g5f~A(Ec?o@|LrgS<^M1U!o|gfVzF2%mD=t0=;&yt z(>;9n@IUzb|NZXWyJlx+fBRqli{r;%`L$pB&p-C`)8%sc*Z-@(iAa@djT3kr5(Fl4N9LL@8A)7IMOmj0|@= zUDtI-MurhFilQJ0|LfoQ%Fxi@=`&|~dV10{tyC&RWQ_UMU;S&7lT(F)Unmq>t(NOL zf8%fb_5J(zpE~tkwOa8!*BHBF$Bt5|^tb-Yzqz=yoFvK6&`_~hY`5F}{ry>%S!+w> za=|aUuGil`7zAC{b(18v))Ip2d1;zjYlS&~t+fV#B#DI?)@4{<-v9u3zMt<8rBsq6 zY^~?{Y*}gL7YcC{F3tx!i_{A4r{H0&|E8T8D zh-c26{l+){^uPT(UpaH`!otGBr$70LFiL*wfBDDno_M>jzwhBAhaNlnShw3-oLCeP>Uf>gyi>fS$g-C=5j;P18c5kR(YQg)^6?iKtjAA2@i} zFBHz5c`pjXM~)tw93S%w1;=sIB~=eBt+7U!m39#E($r*{CLb6pUm>Ek)=8Xv z?CB?u9z7Pt(c{M+8yy|hS|2|6(EP%}=`&{*7Z-l*zxcJ;nOUXO^z@~2lnpSxpT*!J-bInMq2H55QIB-?YMB^!nxBYfB9Gblj-S8 z4?q0y@BQBI|N2+{_Uz2;AAIc(dup{`{Ka1okP8@ zzo)nN@+PQOgrCKw_^!E3ML6D|N zPj7FrTxQFbxzp*mt~)R=SSpo~(Dw_T=OH39`@Y}TKTs-_%H>MX?RGkCrPK$-!H56| z!hmRuEtX1ezw^$wzw=HBZv3zd%!Ogo6Y9`_z(Z5cBeBlJA3x*`N6?~YPI_Hum9Umf9g|jzWLVT!s6-E zr~lD!{jWs$!V5q6)TcfLAd?eQ)mjw*!Z18_>QuYkuFuU)OiX?JW`~^Ltp#DKYIMw(ck`8zw_E_uSH?>!Vi9c2+a21e*NpMb|*>FO0_aMG4aF`kMG*G z>$~6mUT<&D$DV%jop(=lI-PHQ>)S8C{POb3%Km-({`o)uzXfDzX=!m`;rPohmrEr9 z$mk@EjI`?&3dJOj z%0(|VY>WlOIEp;aa~#)LTP~NawTPG`NfgDd>nNpM*X?$@MC5s16vu?9losJ!_vdz& zu{MsP&wu{smX?>_`0;Cl!@~d&hEb9x!hH0x#~y#;@qhR4{=L>JvCs}eUy~+eW>iiT zM__HSRAOtwC@vO!B$cHJIv(0Awc=<^z@PriXGlBW{oeO0rJ@31X3MNd0RjTDh{DT` zz-BvtL+NhV_5h%ives_c?N?d&0KjYolwou=SStvYfe6=(e??*$h(H(sVcpgQBB6ES zAwVWLxOrO;q7})+7D!(~`<;*dv?}(8Q?HU!5Mk|=z(mf#!DMBe zfCbQIiBqVW?xH(*SQo0%%v+3FX;=3?U^?@VMa-G@t85aR1Ua?Vtu`WB^Iq1h;!QVy ztzVHpqi(mWlr9u}V@&>7^ZQPcB#z^1r5s!MSTF4@tCyB-I|BiDumnX-r`q76O&NP! zYGv#bbS!?hAKq%g?gEw+ywkKv96wc)fol0w%Pc0K5EP;n(U>y<##UsL^OGAUvd%r5 z{i16D6?Q!ZVo`ZEZt7w`2)iSXsX|Y@aK`D`Ehbi8 zmE2OiaM~YztaIki!|B(^Ev{{I0L$we5pm=7`ipB05dbch${=E_1tb7q$=?UZbv?fT zY*Ud>43L8|x)X;yoCu^I%U|7v|7P0$%!rx@#3OKsJc5N-9OxqdoSqQOGh zQy3fWb<}|}`U)lk*emL1y_2MK-hK)BG-zQY1w8H+B*8TAxxGo*^ojZn2B)mO4r-=3QFU!5qHT@F%e zB~nt(XZrT;@GE-@J*zEz}Ci(MWH%>Qhnk^Ft4fhpQFsWo+9f zK|mbZbHp1w$Z3em;WR|;0#YunIqvB4Alp<-iloRZbKFv;0ZuxchGH|+9(iRDPM62r zfdi(ym@Hi&?cEw^uq9vtBt!j3k9WGwkJhlq!A2&5fe{}=BfTz$7CzBWudQ;&NXjtm z;qeu&IB=*!?MzAvmQyeS4#I$k8lh{X-^F??E2$hT)9VcmEzr7#LjZ-hj#IEeUx5z} zrweW6JE94mdoVmV@77xmA!sFTqZ?k91_SbHR#(ghzh)|8RSEiAey}I_k2H z@{kg?`700x5PXKM;JsqAQ!WcGAM+M zp{$|b#kp8E9Iy?#=@bI?D|hyu2`i2xCG-(kn!5*2M@M6ARv%U_VzWa7mzjN6@Wo} zQWfQCP6LixlEv_WqiEYFumw_*#a#oPOu`HZuZs^uY;Rjw0Hh3E0|`TD;1&pM1QLeC zU^|9xf<$X-J2qRrf~PP{C1KOb;tnSSw4DV=Mmn^a2Bh`B`<}okfC>-^VdL)xF@QTj z;QacSzkqEMI6oJ!St@RO0Z531NZVP00IZ+&(eV>~d_>@n2>j9Ue(_-Xh`>L&hloic z5c!C}Kas>0a@@fqPbl|py8a&#_+E)-U{;Ph`0x{j{$Vj0-mgB$M+Ck_{zav-DCo}5 zXwSbhp4r_w^hdFMW{q5r*a^_EdTiA-VUbR=OMci^0v;bS)po_4O-rh1K zGk}Oh!WKkOtLu{a4ih*vLV30WfTO@NuoWV_ov7+UlnQ=bA6RC-PRBV_YkQhKYVmH=tbukJSSs<1`nXs;)3PJr00ZZb!^qy3T@ z8^ly_socf@aGN}f`mJW;l;PUngz?K z@Fv$m6k~ONvHIyp#4uPE5y?#ZrBiaH1iixq&Z8f_McUb;nh}G*YQVdD)xH7eQp1cb zW%tIJ(T*P4oy||#&Z_Pk=5C9VNYdzbO|1=RLpCX-t;-dJESUHEk7V6B6D~RBk(E>b zdP~&Z^tiikUF0Zr`YQma`zt{Cxp$3&5kLgKdph!6WCj-KEmCIX)rrJaSa5J7UEu>{ z-^ksuull(!um17>iC)Qn@G(hZDpi9wzi-+rNc!G*H6ln5)KAB&=WSv(;{4>mmZ&R& zT4f83Y_SOnP|>wFbmur&z;jR$#F=Qqm98}`mO%uM2aAefYtPEAvpS~OZ!sDgYz_LIq z;@2jV4_X0>pmxmCZ0mb}0C_nOGaz(d|1J`tcDE=DW&Llv3hPE^AtDnC0#R{$gp z=6=j+%O3~>uY09;Ne3esmY2t3 z5tzRai@-NegjT>ZOe`A!(1iB|$g`bAAiz>00${hFljSIwiv{JnOE5wG-HrTBR1mLD z0RXBkS!+m?)>nZA6`Iu!XhOia*+s3dUF99+V1_JN&j?4byq1yl;XSOtqcu8I#^jc= zDyJNJ`vw32YPu{}gi+(RHXmxWriQ#V036lzCDWbsgYha$>cdRh4LbF@I9kA%w@qMA zK@{IOiYR=AMA@!zyX>BAPWiq2P~}hFVFXf!^jZRMB&}aooU)m~h>+a}D(NEvZ$T`r z|6RdCX#xPKv{-{m=kViZ8sq&T!agGKEqH`X#j<%~8IM;=IL$I$UF zn@%4Q_`@ust?nJ-B;xj}SR+XkU3dr8eOmkTp(L;XumH5%>-q9zjt9@LhbKdG)4G(7F!%VV%KU90$(84&6H#o;>c1 zKHv=RNoFrf6u^C!diS9u@VSnpuEd$#J(w&n0H2Sg&%ILeilE~q6Q{&lu_ie25(gdA z_%UintOcU`EwS=LN#MjvY$X;!zKaC50*Ms_1iW0}o=a)dS_K3|G*K5x0g@PVBONSq ztMRCH9dQd0t&LOkT0ZIhU0}26Q3-$ydN^9Z*xW^{CKu6i0|**b&KQDfO&|+iFO&m7 z2wHo`zfItu`U zeGk?bPZMbX(aHmWFm7t)_4VvFCRu4tJ$C2|KJj3)u=kWpzdbqcU$(pnf&XPR5kV|P)N(5Yjxkw5MpX|lZ03ir~CTIni7J0F1 z{EVP%Z8ZY{3bg9zYBd$iSgEX%`ATY|V82o#pwHJtL0}2`|O1f>()>PLg!AQyX#n_T4QG zMgp6r-CDn} z1k>lEr8%Y7(jl+uXb5O9-4lg}gUDQI2P2^1nmT3x1z<&>~gBUi&MYhFY# zSz%~rE>Ue93bU|vt9^hN&0j*jEv+HsILUIoFtSH^e(T&jUhiPMG)s;nH#D^YESml- zI}|~*ZZd4dkFV+!OIN*V<^8#ejR6z9B0Ec<=GRzQL=uyfoKiDtUa?CAfB`cE18htfw^9gjTYa*GwUec3 z_!CNC0U+We6t?7gt8c$V+5tqHL^kZ|Vr31W*oJlx^8KD8Pc6UkLgj%Y-oS`WV;hCt z$unZ!H%fSI!hk^0E&j@*zcSIBcyjc~cNgDPsJ?mbn}Tu$_H&9V!hqqQ>EY!!^IwjG9Pz#|ts~{NJ4+11s9@*jS>C;grD_tI`DMEPRY@8Um zT{4RRkxioZ`4h_Zn^8+-qmuv%{rfB;=cA{4h(QNIQ4CQOJ&qqyPcff_5;97!l?!WmmDX z-UU)>EMNbu7DtO9T**{!p1ljEfYD7p43|6QWa@j*gsE9Hv zVS2qv=3AH5e6{Glm2E;Mi92&s$}gntW@G%USLq2B>t3ZNTv-4`EHLQ7ZWSe3gRlT< z;;!ovT_>*^*Ud82`NWSrW#$FbUR5ixxIF#p7j)|9r0qvo}14y3$+4WYM71o z7xx1&Cf4}Xo{=X$G57k5k*m>FINMAB;MIwk z5Z}L&m;tP8yQCA~CXpn>b*Na@;Nd)XlL3t}PPNae_W1)NrO`d*UHgL7#r`8t8f%x& zo>0nX0LxN;@te@M!?adR5^0TD6oIB|MYMourAuxpG0AXccr{$@#GR{6)*FECdG3Z% z&~Dmtg_aj>6mpty|Dbjp%rd!Zt8B%{LS|`(W~A3;k#20K7GnU|(d$htMz`rK5kk>L z7RY!lsYM~)^pYi%nu9J~V~X+G5U}fP`mDD8Jra3?eBH%lDn#|?lemTJ$ zg${`o94JF#WHng3oG^F}5JH%OR@hsF#DFFU5=jje+&JS-Xi6?|W})iasrSeNRJ*d= zAy91Fh=4#eREe9hHnvuBSk~e>M34n0+Q06bUXJ{TQmW!rW?Hk3a?UTE8?BALJ9}ay ze`kwMe%>d@ao0qh(%08$dW(F<0&D~kJO@SAd-mLmFcD<-#XY;Q*b6}`ONW zM6CFdSuhrTR~i9Ued#7xaKTjoV83%(9oP>=SG?w0R=cmU;ED*unbcdfYZQ0)f&ibc zLo3AngRt0EskMp0s*k-z8D9nwdTbA$se>l;9dML10z5d3&5##5=1d)(H4|9~C?eQ* zOUwEO}d~Fa<#H9f%Ep zK(zpd!7{KwYC$Vt01>Db;AbA;{0hTW$#~tG5*RE)Y9)%XR^%WBLNo%g5f=0uSq@M{ z!X^md-hKcO768D7w&cKD2&k1m62isgz|87vP$2|j=1=e~XR zJBNo4@2>6o=IL+Vjn45O2!Vz333zuvUo!)W`r_m2z&=UhO$i=y8SJ?{^-Qi*;G5;q zl^GS5`Mo;s=)J55Hz52q^ZFVrfAc%n`4qrGT+Z<35Xy-NX@WWQ2wRn%%8sZ zXXh5rom)INSQGT3c-jgrsK6~6lfMnb2!MeO%xtG478QMnMM5G0V? zp-j-ce@tp1O7EBk%KIoq1{AZ$gcb={SOCz+H<#YXe|PcH0%!mO^=J;=P1e5vXc0%i z1=6}ys3~x^o6+IAg@RYPs{|H6;-nkRzD?S@{n}X(CbyC{reVKc1NH9|f%8kzor6|# zi{C+9(B-o|HWh2Kg6A1ym~Gze2nn@va@z{wnn(kIUUfl9r?Y(go!>Q(Y!AH^QQBQ? zOy6A&Mg$R-G~zoVqJ;%DY^C4d-7@diDnwv?3bec*`h|wF!RC%pGt9_QQke|R*IvxwvCxlyH0J#)>{X@KY_`s zs6y>tnetJV0)SQu0Fo^8wL|aIwH84b$RqX*12~Qw$6={jexUzB0I=3hPFzeW$p@pa zS=fLAa#ivj3i@tsy|21l7l~RgQo2rnLL+G;2Z--N+N_{*08mVNxlHOhT)u`7ZhS>< z<3ZZ(=9 zKtwF8Q1|x_ZCsz3jWKkCl4oY7Ya#3PkMd8i`;j(b!*mzdXmA%SEKUQU_X~9<87o4| z!jccn1VqquUBL3*v?fA;)Y=`@-rco+Pdg9x4v#k%`wFEeM)!Yj{N1AC@%8pD5k+Zw z?C>MU4j)UiY<+I?GjaLBf`0wVU05 z#`nVa2GB~agq_q{S2;nJ43~PW@MLpQk-F(3mD0G*nrp>sM4J`)bwt|r-obTdT}!4# z$g<4Kr#4;w2t*{G01kj6qMgc!+qYLtdY392($z^2zHWzg0Z@oXb{3pY3BQ`2&8G| zI9H%o%$ymsj=@MS%w5MpM9W+%6|*!&B4z=RojZn?mmB%%#bSY3MA&s4*K^Y(HP*&) zq=hpv>D?pn`*=qPkXfEtNd_v;#f1<6=Nd_AUafXBt?>Sd_9H;bc7N~iP^tR*?8VPN z^u%h^CB%BCIp1w+Qa23}Afhqm=%a_bK{(joM~YTf8bKHh4fdZsKb|BBk;+L}E|;p6 zvN5(;ER>3cnc4dA@X-9iVi*QlW)2^GaL4HITz#QjDmGhf5$Wsg{ozaVwQA-0&wZvo zzc4yFoFu8|x(ka-y}dnu{3qY|0BV!o7f4fha=II5tO!Mf8LUXrRr)qMORrm%R~odp znOg+F%*TfJzPEVk$iU7{8h6s@;lW-1@3}WSX{7NcG!_s*jN*9j?j1>zT4ojz;i19) zGiNWZH@yTAaUAE_k3Es3S+^T3EiL~CKlP~-Cr?LljEG9B)kZ6dW7l=sov!D3S(bIW zfiU;fs;iAA09?E{Q7HJGPS9vJ_XBE2>wBx)2;kX+wNumGksfbHpW6v-V)#NmOby>! z&L#q;Y*-{8aBf8ikcAzkZ+-uYAOIo0IXCVp=cVbh^=`v3@2d8}7Ux0V{aYqQk!_4T}T{I$XUzJl*JR#zRZFOE&7 zY2y2S5QMJl`o3!|7mJ0dOS4lm^SN$IE%(7LaCZ{hM zYiDQYJ%%PWoNo_(s>Y8hi&?M}Ph0)c0rd17|19!2q?gAX>FZENlF z%4)0CyryG%el^QGa&GZ{@EHAXuOUtpqt+x>uk&5GgcITn7*3zM#(N-EMq|fX) z*h<66&dTkq)*8>{cdL>g!CKo>t&~c|lkc5-_UU6`6ltZpL3rlen5?6f zmf82cV~-s<{_2~LKlX6BTw>-RjK;>NEL+d>hKBmJqeSH4BL_~scdoy`uiFjg=IX^_ zVPt5~WLY-|k3M?j?RQS?7#*2kT$-=X>+MLtHH0+}vJu@bphSe-*!=i>+tny*kwBbp zrhvED|6fEr<(yufn-5!tnGjiIuu%ENqgAKVnVgS*Nf4SeWn-C3jyKg^5s|UhqNns0#ON*T z0lfJg3uvWQRvL}fma+D=H{Nv|4FL7|MM7G4i}Rv?7#o`cfph0B3P8KvnVGF4pi+tu z$Hu4gMh*)L%UY|hNTb+#hr{>_!+Hh|H5sIGH zijvGuE_BD{+ODJIW~11UocEbRC7Wz*ywNjTONV4s?)%jBB4T2*eU*W~yzhyBdg_Je z_dZh6UT9NaIWNtfZzO?wAALy(uHJS7OoZ#7l6PUzgouc1|1ygegNVNGZ4v=%h(~^8 zSvEO!N$a(zs+8&mp`!c>=y|TSBm$0;BP8NDZX5*w;QOAn762T_5fLI(N{10{sdNPa z4(+ZD_Y^u&<~b_PY?PXTs&{@?ZZZf4QA}#Oi%1RtZ=W;PqV8+mzl6!{*r@e>-dEm2 zjOd}-utF-k{(JMYPYmweUmbjHp|kPeuCeA3nxVwof%9ZNdEeD=z1f?h6*N00Q)wEf zY4ZL~0THEXiik?-LV-3_4hoUVQRsHNWtn+nGLi@)TDDOTSdmt%dF8DMA%#I#Z$&tJ zuKvM^x>DpjYN-_u*8IMTJF(F9wBq+;UqK|!qGb63uD9>nUaSJOjT*W-;td=SlL!bp z1yIh7_LPXYO3!o~|Ig`{SY)!ZtVxAgGSyijL`U6N*yPyXc1=4?(hBj&DE1WLs>yu< z=qT0jg0nO8b90%^Zbo2?qbLkRB)TpZ9frZqoqKwE`^=f?>1t1JmZlp& zk)vk5@3||>ThhTIGSpL?Z$ynwGFbByV<#56!dn}(=lE{l12@X(z;*tS>u6m;a!ki} z`u7!gKWURtKvbb8Svm`H@_nKXgxJmE9JooWAw(k<81Q}h9vlTl_qv$L5m9b(10XZIu0up&7!hrug+soYPd+mI+PS5rR{W*U>^nEp8tp5- zeR28ROxw9>b)kru^g8wn0<*9HB4mcOB7L=>6ClZvR8B+5LIDxwib-!8Q`+!%3D!fA zxP|+?a?6T{5f%ZrummRCxpVw$8q zJ+;4h;YT8}rNnn;xiH)6ExU1Q=T^cCv+X#w12zA`TzgA$_j))ei};={c{(+`bD*@? zj8@vo)e$7zGyxqcM@F`yCU2H3ycMs3D}J%$dL_5;(9q5xi8@L2&dm6=(O(3x!UKMJ zuvjt7^{}bAl( zHf)dX*f+jhZ$+Udx)SF`7AZLH&+K{h^y(ZU{_;c5e)Z({1=d&%5i(Z z`daH;2j>qH5Ko>u$HI=*X_mQ;lSdM_(KY# znL9br@ZI+}**;(bW0)oCDUk}2h6o7Q>2~e`7L5Rw<(b1nC&pF=hy264dsf@A6&aiB zbfQeFdmraV#5hf#-g9uJ+n#PL{-+;*etLD$Rr>70%!Q>{XH$Vm4j@-t|M8(+e>QdE z`Q48cwI?90Bz$A_(&jN}Y-PvjAQ4?0n|$cNKG$^+aAmd8Xtp-esYFC;wP*Lv`GuvP zS``tp%!EOt6fv{bdUASp`qFH%SP+p;Hy|RdRHfp}WvHg7j_shrw77>xrTHj!~HdwB;qQJ0ya?gRH>)TC!s;pIC znm)fYQtGKVz7<}KIuG>>KTsLS*lw8b`JU+S>n#=vPe1use_u~uU(d0}juZ-ooUIYj z7&|oBcVPdXYNfnu=kPO6AA9J)10Q?pXzl_LVw$GyZZN;Fw79eq$4Q!IIYKft*njk~ zhX)7x_wLy_Fwj>j6`y|cnCog|%st>>5pgsvG(tq|gy}b4ot{|^&d;h3oriYHDhg>rhUCVBD|5AMT9{ZwL4v{RioKXGcz+Yzi0Q3QnA1y3IGwcB1Fm) z5b}N!#@HRBBMdB+aD~;LNS)GST?oOuh+K7d##zF{yA~-R& zLWD%P+=?AdORXp;{1qcSH%?s3n_qt+aEI4?<=_kf(L}S&Wf9S&T49i*!HuA!HZnxl z!wCo&W$C$~u9cER5J4felZYIZSAUC$uqdS#mzL}Ei(0F8r<1enbaz%Ml?NM`*>T)v zv%S2$LPWW*OhhN&JL|fxRw_>eVHQN*Au2H+!B3XsvS}KI&qKGJ1s~o!u$wK9z_RuL{9&d39s2p$zTn%FH z*qTN1*uZAl>*_otAZk5jTf-(VE+4$32^>Y9Q&4)N&58hM0d!R65E5yv892w8H*BDM zH@U96k!PutCQ`2JB4M6$q_q|S#}TD vW!c838}D~IZELOM)r*&Z9^yFacG~|RZP09Dnv74r00000NkvXXu0mjf+A)3* literal 12940 zcmV;7GIPy|P)003kN0ssI2j?}E!001&mNklfie1gD`X*#~5QR1F*GL z2$7~~{?l4ZLRf3#I7Y-YO_@24V-fw)07#A zFv~J#-rNxRr!aGVUI+oqQcA6Lep&vIdy)cNaeor|NKAxtI^T1GiT4Vn$6$* zt>1X?frFPXUmhJD$ueCi6kOM>)vDv;<3yAs$@uuV5Ta16IU5nMR_|eeQE}b90lElRy5^k39SAw?FampVZoX=R4o| z>@WO69LF!b@WQYC+UL97fDligKJz!<_}l;H_r7@g?D^Hz)t~x_kA+e4zx=_!K6do& z!J)xNjy&}E6OVVh-L=)VhaP_D5C8Cw*4EZ0CntXX=YQ@OKKq$QqwyDC`tpD9E1w%1 z8++;Hmw*5FfA6(dUwiuL4<%`OaR0t%p8eLZ|N4J;d3N^GpZe6LnVJ6n{+C~V<>?Q7 z@W1_+|Gn$F?*$rPC8DL2Nt}H6=?^~f#0R1{dh!F0PfScmDIY$3Xk~Tv)alb}YpcKg zpZ)gY!lDpjZf<^QY1wg|Z+`Qe6BFZ~|H2n$F3k)L5B=dE{!yA{>+9=JJn{IYi zmABq}Yk7J3AOHFnPM$ciXL|af!-r>PE>`*~)mklzqI2iY@87p~YHD)t-s$o2@m9MX z1mV=4$@AyWpFMT_AODm8Xl`!)kw+f+^FRNK-}&M{UtC!H(wDzdsZ>7ytN)0B&!4}L zW!d3FhkV~}x7$GwOifK4df;IHz`#o{{@@dz`1r!YqUU+fJoD_QKmDnH@h|@6^78Wk z{=fa}Kl#%?J9_kJp-_0w5!e{>=wputLHA2v{>zgmPYw?BzxCETU-`;kc7ou{nKQF< zbB`Q3^2|4%ed*;_vMl>Q{?GqwX=$m^XlSiRhll^-OJA9}Jahj1g;!pA)$?2_<=WcX z{{8z;o;)>kY37YL-l*4`t7~h2_UB((U0YjRTzd7j*O!)-FI>2A>C&Z{naih6oo+Q- z-}=^f_DoG4K78m;|LiYbeDNjec%5!{V4(lI-+S))=b!(=7ryZ2FMs92g$thNb-Ufu zXU<%nxxBKn^4;%#FNz{-%`?w@>%xT#g@X6J=bmrWn?VqK|AiN;)#}K|$Sbe>aDIN? z82hPDeew_f;D5E&AigKmFmAJ=8;u44x~@Y+Q4~oj3xz@yMF60bilRse>3MDxMg9H# zNfK*q5HX5l&vkSA#rJ({3=(Q>yWNiGc~VNHRHxGsLTnmDN-1MZZXxG(mNq7iqR)Qz zGu2w{jaOeA85;wDFpQEkVQZgw{P8EBdh%;u`)eTt0EA(fThY1xj-p6wQz{j;)^Qwr zo<~BYN#eS0mSxN!rLfk1;u9Ztl=I#1eXmd`Y&EUlGX!Qh@W2DrwbhBK>GfJQ2)cqq z97o2O(b2J1tEsaLi5%B;9k<PI=4Ue>1jZ&#^e(v)4#ALm` zuCq)^RVtOjC>$IdYBd|V0qi)A)*6wRZEocuVz=8BLgYqr?&Rb)c9JA<9G6O^O?Pgq z@t^;iZmbI1$HzYAfQEZGU7K`s4H|dIzg@|dIxUOf61qNnz9QT0-4>3a=M?=FS z^;)&l?Tn0!Wm)R`ei(*XmOb{wQ{7HS2yy1r@ia}NC>$IfK5+2CBu$nV=O?E2Oit}d z(=?|OA{C0oW~1&G3UL?-A(Z1Z8}$q4&Pb(lmdaIQPTibl`G1sBuIuJ*($@2v%4Mrd zef6KNobi4C+I!zaZW04bOzqJ+GuEswFBJ*}rJVKJTB%r0l9(-*N~I);wbn_J)Yog; zXunWsHX1^Z=lNk2I!g7GE7j_%Agb5vX_BOA;ui{Ol7vB5N)^XZ9LM!qwNNY$4h;w0 z4j``Aszi7_3T@@oD}W4MErin?wow#pqT+!BJp)O*cw?JQT=<;)`~S5A_D`M%o};oCT&DytC0;La?(|m2~iw2 z-8Doe5~4H$R%dZwvjj=9$w&}DPj}+RksC+7Ux=(Rwc6Sy40%fj0~{&ipo41(q!ui| z!6I6Q$n<_ifW(3Yundp(;d%-Oi|7zGGO!FE8bDuQV4$me(-@gmsjT zO~bTy(H(o3O(wib)?RW3_M?#T+F9xMr_BX#?1&DkC|s0|=~h+$bm!c6(%J?T8OwHKv9pg50&tLI-@@~AsQKX zGz==3^f9q;y3GfR*vMF0@Fg@;kO)IA{(rMNH9!b_2izV#KrW^rtFLC68+ zMC4rP!eJQMmMj!92{@m~bFV^SA z+)_P@mXq#c+)k_}#FrcM50!==C=8aQvmAHl;^u(za~k9D_&z{nomC$H71cLklP*eU z8)KnRY&5V%BDMxeGFwDsYf!oX4AufLTZ6&@YXA|2x)-hpf;GZ(E`R6Kct$y%u~sv< z1OSj(Py~^Iy#*X{@SUc02rPrZTs>oGI9Ix~p{-*Bggq@T5G(^CsEsp(aK6h90o}R= zQ3P`#k4?4r^qVNf)ZkRV9w?Yo%dQ}12FoajjYUDZN0+ZE51^qVV5W{v?dj=4v8ywO z1Op4C)EbFo*t#SWt0j^F&Q&r25@IgNXQSrduANgzyx}Gx+pO}$=gWtG)+XH@EH&nh zGg~bZ?pY33Oa=tz4O`_l_Wdph%sO!^E7-TmTuZph?w7HnW^XQZ+IWT0AQKR(tUn?=4-2G925+^=?9}t zIy$J^qyz6_3=i2DZFQE^0i0J4)$JJ@AvHSK7sEHsd}Hl|9}sx zGkDhrE#4>a?H^wM5GnO8r}6s)z8%SIjdqHC`+n-Ps!(QY-_5f6eFERsTazjcJ30!> z6VrAl1o+)^$oB%r^-zRfUN_uX#0YJcFudxZOb9_5ez0{Nxj?dRNmM^HL)aIEpSe{6Mnds&r;2nenDOJ3i= z;%m=E^(ruz#`lm^+UVVcl9?F{=TR}F-p}2yg)_~@;$~KgAiSYKy88r11WAYp+Oi-R z?UVJuSib32yC)kBg*4q}x^h{K@6W1>C?pV}SA=#9Lfo=&VJ4}XGw0YC6r#JfB%}g_ z&dR)yY8N{OnOVAhg|R2HU`@DX9k&W2k9HScNxN%EHxqUUtZm`QUeQ<3^$q}tvacK6 z`0^?Wx=#eQ3*#n$;5|I$hth_?LX{C_6ij5+2#{Sg9(jRgv`<>EcE~0~#qriAoXz98X zX79i_Ab1DHZIY2AgJYMrho$Z{2Qx!6v{JxeNd$=n10yBjNE|JTbF10?=3wP|2Om#n zPqEIt1CN^ax{X8AuHT{Ox>5OPH`L&66*x%<){d8?XPPsex$nFE2g9|aI%uNAnwLpZRAiJ|2B-%%Oj*$VdaDdb!p32I?k$uKAcd!1 zwMpc~Av*4l$e%#~8u`F)G|zt%&<-5_T+*6Dq+t2=q_v3RhGa(MBy%5_@E_V!|HiBR zKl7vsVse#@lHk-FSbIJ5?w(lD`vzsA`M{7^58_Wh;-6hfeMLcPe{eq5_P&|Z3__T8 zz5Ro~MjpU zrJH>QgCt$=fU)kLVNvpcFkN2vADqnA+sRxNot?e#d!T~>1mRjoTZW2HCzsN5tC_Jd z)i0F5%#AY-lNv z85Wyb5z_s4hp&8dzWUdj)5#$yt6NXmVJJ%Sg|R#BqqM!+ivTbn0qacbY!xjk>A6!1oo>3iS@11JIp$W1CYe?15i@$XT^7XoO8F%Ld6NPLShonaVIxD;E> zJ2A1uEjfP-2)KK=k1z*BdyK*WH{+OlgJf;toNfH`_vpxJ9O#|JB`^RRAaSioFbobt zYN2!!i2}FtYI?7Ax0k;cqDgmV_2M&RLuXA8iz%RKOMh-{;`F zCn@i7Wy7YCwapV{Yf(AQ-UI2%Wi_!6Nyve*Y<)RA_Tu{~J9p90fmQ}GyN5zC_e?Xs z#zkf8j82<&-5J{}`-Zx2Jtqgp)X*e49>*cP`#L24{Ua~|Bo^LoTbWwEm+e%vg~Wm& z$TsnFW>ju)?8V#vn;X< zY%*hwlun+W$IO68NX%C2RFEW+*V{E$2yD~9L@k?k?!IY_07z`~-5{{lnw3tG?rl&i zuhL(O`MEW_JzUt!py<(u03r)6fiHSA&I`b2+wj%b`uoN*n)4_nMnH}1SUr@@EMM9{df&HDJ8iebF z0+A3A0V9*SBy65elz$y0N~`BX;joVGf$wB>PG(i8}CYmFF1#kt9 zxEqbobA6AGVcoUQU)C6k`vsh9$xM^XxeXejGg z-X8%mXvCCkpH_>WbJ=HyF+(tU@By{fIPN}j__K2IgMwtHlfBbFP9(ypWo_118rLQv zlD^lsaO&$T?Ny~zng-VBT%gMhu^=~PsMEj48++I$-McTM*#cemr=JEkcg(>?Is@YD zm)V$I`JLC}^WAMViBOOm94$rvBOe9;S?o*JS4`9m<}QF#E^$GEFTf?p14D+vCs2yj zoIl)f(y~U2IusNN0DzWa&j0{R1_rD-tSk*>tgKvjDCzYCG3<0Z%coEOO_BxxY^?Fz zvb8#jTUw{3;*gZC)@hmsNHo^JM+#@Pc0Pz(M1n|yD!>*F&Z(eTr);u&LSQzW#dpE2 zVZ#{hcB3#`onfvWuIf6+ND_Bp^loZawnq9DD)h0=K)J<(kA@4Ej5$lvvCNjia_?dU zung9sRc!Z%t*1LS&os0Mm{<5C0APlqu56#~wf~?50Fnb?uFr|cg!-xO$B*p&WNM<^ z+OpPc^c*8=jb%e55de|$BtStD5R~+mt7l()?cYl2GPB%fwe8S;?=C4}K!m%}Vhm}w z&q3^e{oj91bc*zKe*vq==c|rL}A`(R6RtStlVrz}X zyM@4;?K(GR_uL&m?p!=6yn>0NWPO#C!$z~Uz?PJ=8&hMWC=9JmQMvBO6!|3!7%b0~ z_CH+M_i$AI0pAU4Uh52rSX&?vbT^6eZXvK?Foq>zW|$eIxa0IZM4M&)=$_G!eP-^j z|DTbMe2SE7vlNABoOsoQo!vY`Ab11}jINuUe_JMDHoIT05UE1X) zM%bxDi-D&HrkinV-@v|R)Ld#Vp}-rXYYKeX?2^ zpKRxO(QNCpNCFUn^HShApAhCCFU34CfVnN*v!bttE;r|P0RRcI8=dnIQCln=-q=%_zFePKX)Nz(xLFHAfH5EhWEm@k`Ogji{Ei8X0NV7r=Ly7ATL2J*;tqH% ze~*p8c48Z*{P2Ww1Xg@nY-qk4VxO7o@BF|-L0nrxp-k9`R^|YaZkG6-D@Vf+XqE&F zfWqC-pb{(t2w=9FH9=y)B`65M4L`Ub!65)7*6{G~!-t0t=Nz2b>|E{Kg~o*q%L>f)K)!uZ3Yq0IE14jOGG1*P0MLdd-Eo>NB4=sZs3Hkw z1_+Ua&C-n|_?vG&k6_g8DGRH!D=k<_Ff&OBLO95&<)baGXP8(Z#LX2 z9a{hhQ%-c=5gzVNiacUorP6DX-m~I%-TDzAV<2V{vN$zO%CR~tPwwlkEFe+RZiMwJ zDXAHXU}TmsDX`k2P}VAx3>GB3vHS)j6A}VWmM3E!0|9Fwq)gjK#~&pUrxs3q=JC%g zwiXBdflKQ%v#r@}YqS=Y7IdMAolCZ_UmSfSstkyC-U#~##ClEp1yT-X=^d1=SO(vr z=T0`X;o-8Ht;H+NR0-TW><#rfbG2CA7Q-Pzs`)dI`D-0Jx2`|+h<~Y?O;p73#q{#J zQFkSm?MgpazCAflL?(?#jyzQw-xI7Y6-OtFBje?XeJ8*2NBO>CosR68c;L{1c6C+y z1#k;P!y~VM^ZQ_ot5k*h*Sc#3x8O-{zCE9r3@D#^!VGMzogUab-<<#0zK^|d>4lP8 z`t;BP|Ni7(tv7009b_U1yEY7T=lLEuc>*tTRmoDol zTz>myw_I62cU<~KZZ~pC%gXKhz{j(Tr@Y~b_2WOx;yCHHfdpIp`2LT7_4rpmvG)^? zjXZX~alY)7pSko*Evz9T5p~mUwOeK8KY8;{8d0Miw}CyXIMv|s)yN0}0A#w?0eD;B zHm_S1#8HNlu7cdP21F193>;-h$jqoD06?li3Ls>*Bpomo1c9*#0FDA?Hu<<0((QI& zTd_G$BrSlWu-7>Y?bpwJJ+aPyS_Vb1mcj3~D`LU2wUF5!B*6fdakR`q0*M9$NHpZt?+Dmm;8a7b>CrvB+~j(Y zx4HqtI9%dXLpuT~@bEaaA}Ba~W(@$7kv=gSK&pWNbcXwfd9{rLMJT#_rV0Qc1sbC* z%fn;xY>hL!A@I1HS@99nmb+>9O5)D7J&-Ij=FQse7QcCCc#JJ*W<(%@&F=4a4d}=p z6{Qw8DhCn4+~<1V-asK-ePo(sABQmjpb}7Yx!S>k1CHQYfTLy522umQ;&u#<#FEQr zR_vRz;$wYWE<-bfk_*P7tH709?ciY1o?oNsF&r&(m~y28f<(o$OD*X0AV_hv42Wj_ zB<&jmW8tYXI1)Ss#v(J6J-c@lE>`W)1@I&=5L`W!YX`N*thk3)ZzH7O>rv)b@ugda zD(Pyh_(#&E5sg)E&k4VugBVIKBpM42*8?0bagYKtEY<-4gg^w2QXm3C=)|09e0vs@ z;98d+9OZfdzJkm^nBrI;YYi`+17pFJQ1%d!&%^=|2a6o0*o`4bL^s0I%iIWXsvipu zw_*?oj?CpYr<$`2kijwtx{*7EKy9h)&XSLWuq|^}GFI*<7R`4j$;KEgd$4ye76yRL zX099N+A;6`(o2qF*qXW}5cBC63`hvt;-S%;65LI=9l_o~&>M@)VFN+{;Jj}c1B{mO z@E912L|`m4!{o+(#~##!PfUYla3x=^L9(dXC9)$0NY#40lHReF@r04p~=kk#-Bk6eDFhH8?u)M18%M6@~O zmH~v7a?Rnc+>yXYWV?3V%$~mR{W$IHzUv~hbyQ(~`RHc%TX{buGYS*8lhxCJH|#IR zfKny33y6Z8;*N<|3_##y-8Gc`Om`y(DEEvBH)m79bscQXhaEBGGmCE2n-K+BYaV>( ziPGXwmZnDQVyV2mw2*t!LWn2|Gm|1*H#f9^{YT;JZ@vBRZ`sUYMiO~K);0q3)};Kq zd00bkpF8r<%~z&en{<$*@G934zf9y)t9buwH4~j#W=TXykxmr}MS^e0O#Nen#=)`i ziB6~ODCdEJ!=C5o)96nedmD)JxFPOntTG@-r4;O}#$hL~Ybz7y z^Oec5mBqz^Uogfnun>~1G1e;O5aqp(kb!%10eI(-C+-{|>6JKay9*JlWwTLcaAoqD ziJH2*X4CL)yFSQZIg5dI(c}*Cs#K+0+qU=WFwA|fe{f(dG1-yfsn-@SxKceaws)yh zzd@l9A%66ye!4t5%*N(1f7n1oa}+x7cS%HH7_2NUmdfR8N0Sp#r_~x99#QJ5S&)dB zWm&yeE%#OI7GcKBQmA^pdi0$)rIcIJW(m{Q5;{fJ@!jWOtF1`^1#tH6tdhQ&>yMo@ zUhXueE5jG+D`Ulqw)Xt`@=b5_Jik=x(`gC-c8jyamWe2*E)i*Mlu}AL)^g5&`Jb${ zLI|mp<2Xt=+X#$^QYqy)Tek%>OR1z(*NA9lRUUN)_j4BBi^TV|wRC$m@VRA^=K%i* z+iVG>R~F74Tb=D@2?AJVNy1mojn`!Jndrv2uH$>IF_r-k(REb2(;u#H)rz`$Eg-`iEdm{~Jcs(f<%@$Xzb`b&pC7@N#myB4%BHdb$10*8p&nCX$B zvC&~*o){lpTv|PQeX`k~WjFD)!CODPFS8#6jOym!xJ6vw{jC20x_X`0T=&gaib zGY!mnHc$RDr)!pJX13M>0wGyznX~I5S@*QE;WivpR#F_E>Z`Sqho&p7AU!@)R}ydf zNt$gALrJ1kYs0Ka5Z-n@A`?Pp^udAAe!nzSD3@LTa&zscM;{5Yc(K!PZrBZ+83c(& zvo$$Bnq~UcH;#@D5B2qx>WvmN=MHWj{I}LlPK9gmpwU*iUy(14D zPSPX{Vw(Gg@tMZWO*8<_w%;lE(jmlJ+}T$it~maDx1Ji^&63-rF*7J3 zUz|Gw46iR;Y$s7SjV{$!Bb{vw@x9^xAdI8r^|y{QbEfrLt?no}zp(7Nj+7$jMI~je zR)6N(-zP!4uuzI}Yp)C6t}3En$B8OMn+#xHnFORG}Kg6~)Q%1=CYWOc1tE|sz@ zL&Qe2_4s2)jvqhHSB-eTzlUW|0xh$zgs~i@HlOiut!Wd)**9LDD|+huV(Y?ETS;QR zVekquGiI#N_bUgUTIe-OV87NT>&w>b%}p{`YYVf+v0sy}6#Zf#qHWJ54z0oA1XTJ5E=ef;h+x1*yOcX`y>rJh%%<7W2 zEpr)qbEt30AC6P|--OLo1 zNVT~8c5N=m5??we*B2*?{ihmB4^~DOgGOw!+vqWv(^v_?%pmhr8g`}P-I{%9v>6^A zoS9vC_|U*)f+z^XoT$}Wz1t0Fqpt3j32ZGRp*D;NMNh`5xg|0a zVJp-hexPrz7B8&FuA)b#iVO8*D+?oHlA0$U8eDC}tswp7AK$y$hydZ@QhR2l?aCYR ziX>5wgRjrL<4ctqEs##s7lJIUI6ed4h`J0nXbQobrs)+xnXhYM@HWVukuV6tfq}}U z%kzuNtBq!>)9GqsdcJWF&#;c;3=Z~3aop~78;#b%1AEKm((=-Bp7@uWR{$WTK!hYo zh)4)QThbkwSxP~4TeV4fr|+MAa$>!mj1BmN7$!PO^&97EyWGjlAP5)hNf0;Mk?J5I1=Dif(>|`*okFO5(vRSUO|>+`oyVoLWtZSl0wWct`MS> zGIxDQV7D8*_QpHb+Be@mo|9^=T9Z;rDUfJeHasFqDRaPN>u}6`6Kv&{-6!2)lt%wV3jWNdL0fyWl2PA8)AOc`x3=tV)hy)^Txw5x?gb>W!Xwd3t1_9!r7qwdAdt&Q`)n!qrB|3_@TKX(3P>Ziktm8#!?dAc#6?{N)R; zX=~=wsGUYL?b=MIR*(+eux~yl8e?@aQ>rE-;EsJPG%#7JLS{ffL%&v8R z=_7lCILp=W3&&UV?o#g>?Q|e4}{Vqg+6UfW{ox`+jKv2B5d3uEo-)pkeM+I?4nFE13mnat+j?3 z5w$i%M1%mSwRRk}i7%E)<$;0WEK72W)!N=Ba9k%%6Kf4@Kv!E#JWjLh@kbvT8XQ_G|eU_MyK~owA!6| zy;&|7iTY;e7IRDy5skI`_fGBGJ5^h63=Q^su4}DrwAz3u1@%|@;yAAKm91sp_g2@c z0N}dL-29?3RtTZB&Mh*}b8pj4LWE5BYG4Yk%#Al8?t1=59_`;A@W;)mI4th@+%g71 zbYl7PTw|>hN0P)^rx~QNC$|G(3`|JJY758f3uWa4z;ladO2WC)rI0WUlgqOUzVBVU zbh%h8`o3q3xjeTpG}v#g>A}&=j#A4jYs)KZ#X{l4sdG~kW5(J%GhR9Bqd)dktJN+P zywhhc?wK52SzQamV0A?!^?;X6r7lP`n~mE4@zT^De$v2u|SJD;y*)sdmnJ)FO z#qE!bKad*j3K?hVv+L)=Ep~wKdygJD{Nl^6KlbQD(^C`8c3Vk#;^es?40|eH>(P;+ zvC+ZfC(b^48?C$@TDe>v=L}5lJmO!j>4}Q441-1b{<&Bi!<5~h(UOEXJvchwsMf>I z#p?3oQ~S%VUk}=VS9An!(74|%tjFEC&U%m~4^_qs((SO_l#^N;$5ErB!=C3xaWcQK zeE7kGLqh}YPFJa(?^G)K1>Y<8m4prKk*Z99X&ZSbBX9$5zTztk@;N&{CDy2E7x#l?0^pkhXeta zT#Pd=D#$w^^$=sb@$b$fmzI0a+`ytGQZwlO-HF!_@NbX5ntQj>7&O^YON8jWa4ef zs+2C~8S(cZqXU&oPVuG$Mxjvo42}mS77Nb z5ZIC6Do_#}3Don0dDC%6nO`Q*xe{T>fet0XRp2R*77)}%f}>=Hf@74BQYz&j10qIo zf{4EF^On(=LLdoWkrCbNYnXKq}R4H8-^b z4v+8a-D5IfzGq2~2)phI32d3UH?;BU$PM?RZwf75Df+&18s}vVxAwqZR?2nmVjwca za8;t*4J`NT<`iPDY5xv^*R5x-0 Date: Fri, 16 Oct 2026 12:14:52 +0000 Subject: [PATCH 28/41] Check the treemap's 2px edge in the image sanity test Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- tests/test_charts.py | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/tests/test_charts.py b/tests/test_charts.py index cae738e4..dcda72d7 100644 --- a/tests/test_charts.py +++ b/tests/test_charts.py @@ -97,9 +97,9 @@ def shorts(rows, threshold=15): class ImageSanity(unittest.TestCase): - def check_image(self, buf, name, bg=BG, edge_clean=True, size=(1280, 800)): + def check_image(self, buf, name, bg=BG, edge_clean=True, size=(1280, 800), edge=3): """Fail unless buf is a PNG of this size (1280x800) that is not blank, starts with the background, and - (edge_clean) draws nothing on its edges, where clipped text or bars would show.""" + (edge_clean) draws nothing in `edge` pixels at its edges, where clipped text or bars would show.""" image = decode(buf) self.assertEqual(image.size, size, name) pixels = np.asarray(image) @@ -109,9 +109,9 @@ def check_image(self, buf, name, bg=BG, edge_clean=True, size=(1280, 800)): self.assertIn(tuple(int(c) for c in pixels[0, 0]), allowed, name) if not edge_clean: # the shorts beach draws its sea right to the edge return - edge = np.concatenate([pixels[:3].reshape(-1, 3), pixels[-3:].reshape(-1, 3), pixels[:, :3].reshape(-1, 3), - pixels[:, -3:].reshape(-1, 3)]) - self.assertTrue((edge == bg).all(), f'{name}: something is drawn on the image edge (clipped text or bars?)') + border = np.concatenate([pixels[:edge].reshape(-1, 3), pixels[-edge:].reshape(-1, 3), + pixels[:, :edge].reshape(-1, 3), pixels[:, -edge:].reshape(-1, 3)]) + self.assertTrue((border == bg).all(), f'{name}: something is drawn on the image edge (clipped text or bars?)') def test_each_chart_type(self): """Every chart type draws a sane image.""" @@ -119,6 +119,8 @@ def test_each_chart_type(self): with self.subTest(chart=name): if name == 'shorts': self.check_image(make(), name, bg=SHORTS_BG, edge_clean=False) + elif name == 'treemap': + self.check_image(make(), name, edge=2) else: self.check_image(make(), name) From 8f9ebf82cf409895e15a1b6846936945d23b2906 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 12:48:01 +0000 Subject: [PATCH 29/41] Buttons to pick a portfolio or tickers for a bare .pie .treemap .holdings .history .compare A picker is one message edited as buttons are pressed (a 1x1 image where the answer is a chart), .compare ticks up to 6 tickers with an alert on a seventh, .treemap toggles Sector/Industry, tickers sit behind a sub menu. The Telegram keyboard now has .compare, .pie and .treemap in place of .dividend, .marketcap and .pe. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 4 + doc/commands.md | 1 + lib/buttons.py | 46 ++++++++++- lib/discord_common.py | 9 ++- lib/render.py | 7 ++ lib/reports/pick.py | 102 ++++++++++++++++++++++++ lib/telegram_common.py | 14 ++-- lib/worker.py | 122 +++++++++++++++++++++++++---- tests/test_charts.py | 6 ++ tests/test_discord.py | 6 +- tests/test_edges.py | 8 +- tests/test_reports.py | 174 ++++++++++++++++++++++++++++++++++++++--- tests/test_util.py | 14 ++++ util/record_outputs.py | 2 +- 14 files changed, 472 insertions(+), 43 deletions(-) create mode 100644 lib/reports/pick.py diff --git a/CLAUDE.md b/CLAUDE.md index e52825c9..d948d2e0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -84,6 +84,10 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm message), which `text()` and `chart()` stop before posting. `DiscordBot`: a slash command's first message replaces its "thinking…" through its `interaction` (application id/token), and a pressed message is edited through the press's. +- `lib/reports/pick.py` and `PickPress` (`lib/buttons.py`, `pick_button`): the buttons of `.pie .treemap .holdings + .history .compare` typed bare (`worker.picker`, `process_pick_callback`): one message edited as it is pressed (a + 1x1 image, `render.blank()`, where its answer is a chart), its list of items checked by checksum, `.compare`'s ticks a + bit each in the data's mask, a seventh refused by an alert toast (`buttons.alert`). - `lib/buttons.py`: `Button`, a chart's period buttons (`keyboard`), the `.jobs` menu's (`menu_button`, a `MenuPress`; older lists' `j|`/`e|` buttons still parse), what a pressed button's data asks for (`parse`), and the toast it gets at once (`toast`); each transport draws them, and Slack and Discord take a button's data as its id, so diff --git a/doc/commands.md b/doc/commands.md index 3a78a613..8ecde9c2 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -73,6 +73,7 @@ The bot answers commands in every chat it is in. By default, it fetches its upda * A watch list per chat, which anyone in it can change, picked up by the reports there * Listing of portfolios and portfolio performance * A pie or a treemap of a portfolio by industry or sector (`/pie portfolio [sector]`, `/treemap portfolio [sector]`): the treemap tiles each holding by value, grouped and coloured by industry (the default) or sector +* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap`, `/holdings` (portfolios; `/treemap` also toggles Sector/Industry), `/history` (a ticker) and `/compare` (up to 6 ticked tickers and a period), the tickers those held (else the chat's watchlist), behind a `Tickers ▶` button. One message is edited as buttons are pressed, and becomes the answer. A job, or a chat with no buttons, gets the usage instead * Listing of holdings sorted by various attributes * Running the other stock reports on demand diff --git a/lib/buttons.py b/lib/buttons.py index 8e87ba79..73f7a95e 100644 --- a/lib/buttons.py +++ b/lib/buttons.py @@ -1,6 +1,7 @@ """Buttons under a message, what their data says when one is pressed (parse), and what a press is told at once (toast): a chart's period buttons (1D | 7D | 1M | 1Y | Max), whose data carries what its builder needs -(charts.BUILDERS); the .jobs menu's (menu_button); and the job buttons of older .jobs lists, which still work. Each +(charts.BUILDERS); the .jobs menu's (menu_button); the pickers of a command typed with nothing to say what it is +about (pick_button); and the job buttons of older .jobs lists, which still work. Each transport draws Button rows its own way.""" import hashlib import typing @@ -37,10 +38,28 @@ class MenuPress(typing.NamedTuple): checksum: str +class PickPress(typing.NamedTuple): + """A press in a picker (lib/reports/pick.py): the command it picks for (a key of PICKERS), how it groups (a + treemap's or pie's: i industry, s sector, 0 neither), the period (a key of PERIODS, or 0), the items ticked (a + bit of the mask each), what the press asks for (act: p the first screen, m the tickers' screen, r then an item's + number to run it, g to run the ticked ones, f a tick refused, as too many) and the checksum of the list of + items it was drawn for, stale once that changes.""" + cmd: str + group: str + period: str + mask: int + act: str + checksum: str + + JOB_ACTIONS = {'j': 'run', 'e': 'toggle'} # an older job button's kind, its data's first letter, and what it does MENU_SCREENS = ('top', 'on', 'off', 'del', 'run', 'add', 'next') # the .jobs menu's screens (lib/reports/about.py) +PICKERS = {'p': 'pie', 't': 'treemap', 'h': 'holdings', 'y': 'history', 'c': 'compare'} # command key -> its name +MAX_PICKED = 6 # tickers .compare takes +TOO_MANY = f"⚠️ Max {MAX_PICKED} tickers: untick one first" + PERIODS = {'d': ('1D', 1), 'w': ('7D', 7), 'm': ('1M', 30), 'y': ('1Y', 365), 'x': ('Max', None)} BUTTONS = {'h': 'wmyx', 'p': 'wmyx', 'c': 'wmyx', 'f': 'wmyx', 'l': 'dwmyx'} # chart kind -> its buttons, five at most # (Discord's limit for a row): h history, p a stock's price chart, c compare, f performance, l the price list @@ -90,12 +109,27 @@ def menu_button(label, screen, args, checksum): besides (words, numbers) and the list's checksum: 'm|on|0|5|a1b2'.""" return Button(label[:40], '|'.join(['m', screen, *map(str, args), checksum])) +def pick_button(label, press, active=False): + """A picker's button: its label (cut to 40 characters) and a PickPress as its data: 'k|c|0|x|3|p|a1b2'.""" + data = '|'.join(['k', press.cmd, press.group, press.period, format(press.mask, 'x'), press.act, press.checksum]) + return Button(label[:40], data, active) + def parse(data): """What a pressed button's data asks for: a MenuPress for a .jobs menu's button of a known screen, its parts plain letters and digits; a JobPress for an older job button (whatever follows 'j|' or 'e|', which pick refuses unless it is a job's number and checksum); a ChartPress for a period button of a known kind and period; or None for data of none of them (from an older finbot, or forged).""" kind, _, rest = data.partition('|') + if kind == 'k': + parts = rest.split('|') + if (len(parts) == 6 and parts[0] in PICKERS and parts[1] in ('0', 'i', 's') and (parts[2] in PERIODS or + parts[2] == '0') and len(parts[3]) <= 8 and parts[4][:1] in ('p', 'm', 'r', 'g', 'f') and + all(part.isascii() and part.isalnum() for part in parts) and len(data.encode()) <= MAX_CALLBACK_BYTES): + try: + return PickPress(parts[0], parts[1], parts[2], int(parts[3], 16), parts[4], parts[5]) + except ValueError: + return None + return None if kind == 'm': parts = rest.split('|') if (len(parts) >= 2 and parts[0] in MENU_SCREENS and len(data.encode()) <= MAX_CALLBACK_BYTES and @@ -110,11 +144,19 @@ def parse(data): return ChartPress(*parts) return None +def alert(data): + """Whether a press's toast is an alert the person must dismiss: a picker refusing a seventh ticker.""" + pressed = parse(data) + return isinstance(pressed, PickPress) and pressed.act == 'f' + def toast(data): """What a press is told at once, from the button's data alone (the network acknowledges a press before finbot knows more): 'Loading 1Y…' for a period button, 'Updating…' for the .jobs menu's, 'Running…' or 'Switching…' for an - older job button, None for data of none of them.""" + older job button, 'Loading…' or 'Updating…' for a picker's (an error for a refused tick), None for data of none + of them.""" pressed = parse(data) + if isinstance(pressed, PickPress): + return TOO_MANY if pressed.act == 'f' else 'Loading…' if pressed.act[0] in 'rg' else 'Updating…' if isinstance(pressed, MenuPress): return 'Updating…' if isinstance(pressed, JobPress): diff --git a/lib/discord_common.py b/lib/discord_common.py index 16a19836..73b11c29 100644 --- a/lib/discord_common.py +++ b/lib/discord_common.py @@ -289,11 +289,12 @@ def edit_text(self, message, text, buttons=None): def components(buttons): - """Rows of buttons.Button as Discord's action rows: DISCORD_ROW buttons to a row and as many rows at most (the - buttons past that are left out), the one shown now blurple and the rest grey.""" - flat = [button for row in buttons for button in row][:DISCORD_ROW * DISCORD_ROW] + """Rows of buttons.Button as Discord's action rows: a row of at most DISCORD_ROW buttons (a longer one in rows of + that), and as many rows at most (the buttons past that are left out), the one shown now blurple and the rest + grey. A row of buttons that fits stays a row of its own, so a picker's period buttons keep theirs.""" + rows = [row[i:i + DISCORD_ROW] for row in buttons for i in range(0, len(row), DISCORD_ROW)][:DISCORD_ROW] return [{'type': 1, 'components': [{'type': 2, 'style': 1 if button.active else 2, 'label': button.label[:80], - 'custom_id': button.data} for button in flat[i:i + DISCORD_ROW]]} for i in range(0, len(flat), DISCORD_ROW)] + 'custom_id': button.data} for button in row]} for row in rows] def message_body(payload, image=None): diff --git a/lib/render.py b/lib/render.py index 9d62351b..0436f9f2 100644 --- a/lib/render.py +++ b/lib/render.py @@ -592,6 +592,13 @@ def draw_treemap(chart): _heading(fig, chart.title, chart.subtitle, inline=True) return _png(fig, bg) +def blank(): + """A 1x1 PNG in the chart background: the image of a picker's message, which an answer's chart then replaces (a text + message cannot become a photo).""" + fig, _ = _figure(PALETTE[0]) + fig.set_size_inches(1 / 200, 1 / 200) + return _png(fig, PALETTE[0]) + def _code_version(): """What else decides how a spec looks: this file and the matplotlib release.""" try: diff --git a/lib/reports/pick.py b/lib/reports/pick.py new file mode 100644 index 00000000..2aab9873 --- /dev/null +++ b/lib/reports/pick.py @@ -0,0 +1,102 @@ +"""The pickers: what a chat gets when it types .pie, .treemap, .holdings, .history or .compare with nothing to say what +it is about: buttons for the portfolios or tickers, one message edited as each is pressed (buttons.PickPress), the +last press turning it into the answer.""" +import hashlib + +from lib import answer +from lib import buttons +from lib import config +from lib import sharesight +from lib import watchlist + +PER_ROW = {'portfolio': 3, 'ticker': 4} # buttons to a row of items +MOST_ITEMS = 32 # items offered; Discord has room for fewer (DISCORD_ITEMS) +DISCORD_ITEMS = 16 # four rows of four: a fifth row of its 5 holds the way back and the compare button +CHARTS = 'ptyc' # the pickers whose answer is a chart + +def as_photo(chat, cmd): + """Whether a picker's message is a picture (a 1x1 image whose caption is the picker): when its answer is a chart, + and this chat takes charts (the graph preference on); else it is text.""" + return cmd in CHARTS and chat.takes_charts and config.prefs.graph + +def noun(cmd): + """What a picker picks: 'portfolio' or 'ticker'.""" + return 'ticker' if cmd in 'yc' else 'portfolio' + +def items(service, cmd, chat): + """What a picker offers, sorted: portfolio names (a portfolio command), or tickers (history, compare): the ones + held, or the chat's watchlist where it holds none. Raises answer.Answer where there are none.""" + if noun(cmd) == 'portfolio': + found = sorted(portfolio.name for portfolio in sharesight.portfolios()) + else: + found = sharesight.tickers() or watchlist.tickers(chat) + found = sorted(found) + if not found: + raise answer.Answer("No tickers to pick from: .watchlist add SYMBOL adds one") + return found[:DISCORD_ITEMS if service == 'discord' else MOST_ITEMS] + +def checksum(found): + """A short stand-in for a list of items, which tells any change of it.""" + return hashlib.sha1('\n'.join(found).encode()).hexdigest()[:4] + +def press(cmd, group, period, mask, act, found): + """The PickPress a button of this picker sends, for the list `found` of items.""" + return buttons.PickPress(cmd, group, period, mask, act, checksum(found)) + +def start(service, cmd, chat, group='0', period='0'): + """A picker's first screen: (lines, rows).""" + return screen(service, cmd, items(service, cmd, chat), group, period, 0, 'p') + +def screen(service, cmd, found, group, period, mask, view, note=None): + """A picker's screen, as (lines, rows): view 'p', its first (the portfolios, or for tickers the period and a + button for the tickers' screen), or 'm', the tickers'. `mask` has a bit for each ticked ticker.""" + ticked = [name for i, name in enumerate(found) if mask >> i & 1] + name = buttons.PICKERS[cmd] + rows = [] + if period != '0' and view == 'p': + rows.append([buttons.pick_button(buttons.PERIODS[key][0], press(cmd, group, key, mask, 'p', found), + key == period) for key in buttons.BUTTONS['c']]) + if cmd == 't' and view == 'p': + other = 's' if group != 's' else 'i' + rows.append([buttons.pick_button(f"Group by: {'Sector' if group == 's' else 'Industry'}", + press(cmd, other, period, mask, 'p', found))]) + if noun(cmd) == 'portfolio': + heading = f"{service.bold(name.capitalize())}: pick a portfolio" + rows += chunks([buttons.pick_button(name, press(cmd, group, period, mask, f'r{i}', found)) + for i, name in enumerate(found)], PER_ROW['portfolio']) + elif view == 'p': + heading = f"{service.bold(name.capitalize())}: pick " + ('up to 6 tickers' if cmd == 'c' else 'a ticker') + label = f"Tickers ({len(ticked)}) ▶" if ticked else "Tickers ▶" + rows.append([buttons.pick_button(label, press(cmd, group, period, mask, 'm', found))]) + else: + heading = f"{service.bold(name.capitalize())}: pick " + ('up to 6 tickers' if cmd == 'c' else 'a ticker') + full = len(ticked) >= buttons.MAX_PICKED + tickers = [] + for i, ticker in enumerate(found): + if cmd == 'y': + tickers.append(buttons.pick_button(ticker, press(cmd, group, period, mask, f'r{i}', found))) + elif mask >> i & 1: + tickers.append(buttons.pick_button(f"✅ {ticker}", press(cmd, group, period, mask ^ 1 << i, 'm', found))) + else: + tickers.append(buttons.pick_button(ticker, press(cmd, group, period, mask, 'f', found) if full else + press(cmd, group, period, mask | 1 << i, 'm', found))) + rows += chunks(tickers, PER_ROW['ticker']) + if cmd == 'c' and len(ticked) >= 2: + last = [buttons.pick_button('Compare', press(cmd, group, period, mask, 'g', found))] + else: + last = [] + if cmd in 'yc' and view == 'm': + last.insert(0, buttons.pick_button('◀ Back', press(cmd, group, period, mask, 'p', found))) + elif last: + rows.append(last) + last = [] + if last: + rows.append(last) + lines = ([note] if note else []) + [heading] + if ticked: + lines.append(f"Ticked: {', '.join(ticked)}") + return lines, rows + +def chunks(row, size): + """A list of buttons as rows of at most `size`.""" + return [row[i:i + size] for i in range(0, len(row), size)] diff --git a/lib/telegram_common.py b/lib/telegram_common.py index f09e0b50..520e9f62 100644 --- a/lib/telegram_common.py +++ b/lib/telegram_common.py @@ -60,10 +60,11 @@ def pinChatMessage(chat_id, message_id): try_call('pinChatMessage', json={'chat_id': chat_id, 'message_id': message_id, 'disable_notification': True}) -def answerCallbackQuery(callback_query_id, text=None): - """Acknowledge a button press (the client's spinner stops), `text` as a toast. A failure is only logged.""" - try_call('answerCallbackQuery', json={'callback_query_id': callback_query_id} | ({'text': text} if text else {}), - logged='debug') +def answerCallbackQuery(callback_query_id, text=None, alert=False): + """Acknowledge a button press (the client's spinner stops), `text` as a toast, or with `alert` as an alert the + person dismisses. A failure is only logged.""" + try_call('answerCallbackQuery', json={'callback_query_id': callback_query_id} | ({'text': text} if text else {}) | + ({'show_alert': True} if alert else {}), logged='debug') class Telegram(inbound.Network): @@ -116,7 +117,10 @@ def acknowledge(self, event): """A press: answerCallbackQuery, so that the client's spinner stops and a toast made from the button's data says what is coming (buttons.toast: 'Loading 1Y…'). A failure is only logged. Nothing goes back with the update.""" if isinstance(event, inbound.Press): - answerCallbackQuery(event.press_id, buttons.toast(event.data)) + if buttons.alert(event.data): + answerCallbackQuery(event.press_id, buttons.toast(event.data), alert=True) + else: + answerCallbackQuery(event.press_id, buttons.toast(event.data)) return None diff --git a/lib/worker.py b/lib/worker.py index c1b95d40..6b68cb56 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -17,7 +17,8 @@ from lib import render from lib import buttons from lib import charts -from lib.reports import about, cal, holdings, performance, price, profile, rankings, shorts, trades +from lib import transports +from lib.reports import about, cal, holdings, performance, pick, price, profile, rankings, shorts, trades from lib import sharesight from lib import telegram_common from lib import jobs @@ -139,11 +140,79 @@ def add_step(service, chat, steps): return about.add_failed(service, chat.key, steps, str(e)) return about.added_screen(service, chat.key, number) +def show_pick(chat, message, cmd, lines, rows): + """Put a picker's screen (lines and button rows) in place of the pressed message.""" + text = '\n'.join(lines) + if pick.as_photo(chat, cmd): + chat.edit_chart(message, render.blank(), text, rows) + else: + chat.edit_text(message, text, rows) + +def process_pick_callback(press, pressed): + """A press in a picker (an inbound.Press; pressed, the buttons.PickPress its data says): the screen it asks for, in + place of the pressed message, or a button of an item, or Compare, turning the message into the answer. Buttons + drawn for another list of items start the picker again, saying so.""" + service = dialects.of(press.service) + chat = delivery.reply_to(service, press.chat_id, interaction=press.interaction, pressed=True) + cmd = pressed.cmd + try: + found = pick.items(service, cmd, chat.key) + stale = pick.checksum(found) != pressed.checksum + mask, act = (0, 'p') if stale else (pressed.mask, pressed.act) + note = 'The list has changed: here it is now' if stale else buttons.TOO_MANY if act == 'f' else None + if act[0] in 'rg' and pick_answer(service, chat, press.message_id, pressed, found, mask, act): + return + lines, rows = pick.screen(service, cmd, found, pressed.group, pressed.period, mask, 'm' if act in 'mf' else 'p', + note) + show_pick(chat, press.message_id, cmd, lines, rows) + except answer.Answer as e: + delivery.deliver(chat, delivery.Report(e.lines)) + except Exception as e: + delivery.report_error(e, chat, context='pick') + +def pick_answer(service, chat, message, pressed, found, mask, act): + """The pressed picker message turned into the answer of its command for the item (act r and its number) or the + ticked items (g) it asks for; False when there is nothing to answer yet (a number out of the list, fewer than two + to compare).""" + cmd = pressed.cmd + if act == 'g': + names = [name for i, name in enumerate(found) if mask >> i & 1] + elif act[1:].isdigit() and int(act[1:]) < len(found): + names = [found[int(act[1:])]] + else: + return False + if cmd == 'c' and len(names) < 2: + return False + by = 'sector' if pressed.group == 's' else 'industry' + if cmd == 'h': + lines = holdings.prepare_holdings_payload(names[0], service) or [f"{names[0]} holds nothing"] + parts = transports.chunked(lines, chat.dialect.text_limit) + chat.edit_text(message, '\n'.join(parts[0]), []) + for part in parts[1:]: + chat.text(part) + return True + markup = [] + if cmd in 'pt': + prepare = holdings.prepare_pie if cmd == 'p' else holdings.prepare_treemap + lines, chart = prepare(names[0], by, service) + caption = '\n'.join(lines) + else: + period = pressed.period if cmd == 'c' else None + caption, chart = charts.build('c' if cmd == 'c' else 'h', names, service, chat.key, period=period) + markup = buttons.keyboard('c' if cmd == 'c' else 'h', names, period if cmd == 'c' else 'x') + if pick.as_photo(chat, cmd): + chat.edit_chart(message, render.draw(chart) if chart and chart.rows else render.blank(), caption, markup) + else: + chat.edit_text(message, caption, []) + return True + def process_callback(press): """A button pressed (an inbound.Press, which bot.take acknowledged as it arrived): the .jobs menu's, an older job button (run it, or switch it), or a chart's, whose message is rebuilt for the chosen period and edited in place. Data of none of them is ignored.""" pressed = buttons.parse(press.data) + if isinstance(pressed, buttons.PickPress): + return process_pick_callback(press, pressed) if isinstance(pressed, buttons.MenuPress): return process_menu_callback(press, pressed) if isinstance(pressed, buttons.JobPress): @@ -354,9 +423,9 @@ def on_trades(request, args): portfolio_select=args.rest or None) def on_holdings(request, args): - """.holdings PORTFOLIO: what it holds.""" + """.holdings PORTFOLIO: what it holds; with no portfolio, buttons for them.""" if not args.words: - raise Usage(portfolios()) + return picker(request, 'h', args) request.reply(holdings.prepare_holdings_payload(args.rest, request.service)) def on_marketcap(request, args): @@ -385,16 +454,17 @@ def on_beta(request, args): request.reply(payload, chart) def on_pie(request, args): - """.pie PORTFOLIO [sector]: its value by industry, or by sector.""" + """.pie PORTFOLIO [sector]: its value by industry, or by sector; with no portfolio, buttons for them.""" if not args.words: - raise Usage(portfolios()) + return picker(request, 'p', args) payload, chart = holdings.prepare_pie(args.rest, args.keyword or 'industry', request.service) request.reply(payload, chart) def on_treemap(request, args): - """.treemap PORTFOLIO [sector]: its holdings as tiles, grouped by industry or by sector.""" + """.treemap PORTFOLIO [sector]: its holdings as tiles, grouped by industry or by sector; with no portfolio, buttons + for them.""" if not args.words: - raise Usage(portfolios()) + return picker(request, 't', args) payload, chart = holdings.prepare_treemap(args.rest, args.keyword or 'industry', request.service) request.reply(payload, chart) @@ -405,7 +475,10 @@ def on_rating(request, args): request.reply(payload, chart) def on_compare(request, args): - """.compare SYMBOL SYMBOL... [period]: their price changes on one chart, with period buttons where they work.""" + """.compare SYMBOL SYMBOL... [period]: their price changes on one chart, with period buttons where they work; with + no symbols, buttons to pick them.""" + if not args.words: + return picker(request, 'c', args) if not 2 <= len(args.words) <= 6: raise Usage tickers = list(dict.fromkeys(yahoo.symbols(args.words))) # each once, in the order asked @@ -414,11 +487,28 @@ def on_compare(request, args): chart_reply(request, 'c', tickers, args.days) def on_history(request, args): - """.history SYMBOL: the stock's change over every period, with its price chart, and period buttons where they go.""" + """.history SYMBOL: the stock's change over every period, with its price chart, and period buttons where they go; + with no symbol, buttons to pick one.""" + if not args.words: + return picker(request, 'y', args) if len(args.words) != 1: raise Usage chart_reply(request, 'h', [ticker(args)]) +def picker(request, cmd, args): + """A command typed with nothing to say what it is about (cmd, a key of buttons.PICKERS): where there are buttons, + their picker as one message, a 1x1 image (pick.as_photo) that the answer then replaces; else the command's usage.""" + dest = request.dest + if not (dest.takes_buttons and delivery.interactive(dest)): + raise Usage(portfolios()) if pick.noun(cmd) == 'portfolio' else Usage + group = {'sector': 's', 'industry': 'i'}.get(args.keyword, '0') + period = buttons.period_for_days(args.days, 'x') if cmd == 'c' else '0' + lines, rows = pick.start(request.service, cmd, dest.key, group, period) + if pick.as_photo(dest, cmd): + dest.chart(render.blank(), '\n'.join(lines), rows) + else: + dest.text(lines, rows) + def chart_reply(request, kind, args, days=None): """The first view of a chart with period buttons, built as a button press builds it (charts.build): its caption, chart and buttons where they go, the period shown marked (Max when days is None).""" @@ -545,19 +635,19 @@ class Command: # kept on purpose. Command(('help', 'usage', 'start'), on_help, job=False), # /start is what Telegram sends when a chat opens Command(('holdings', 'holding'), on_holdings, ('.holdings PORTFOLIO',), TRACKED), - Command(('pie',), on_pie, ('.pie PORTFOLIO [sector]',), TRACKED, keywords=BY), - Command(('treemap',), on_treemap, ('.treemap PORTFOLIO [sector]',), TRACKED, keywords=BY), + Command(('pie',), on_pie, ('.pie PORTFOLIO [sector]',), TRACKED, ('.pie',), keywords=BY), + Command(('treemap',), on_treemap, ('.treemap PORTFOLIO [sector]',), TRACKED, ('.treemap',), keywords=BY), Command(('watchlist', 'wishlist'), on_watchlist, ('.watchlist [add|del SYMBOL]',), TRACKED, ('.watchlist',), keywords=CHANGE, job=False), Command(('who',), on_who, ('.who SYMBOL',), TRACKED), Command(('earnings', 'earning', 'earrings', 'earring', 'dividends', 'dividend', 'exdividend', 'ex-dividend'), - on_calendar, ('.dividend [period|SYMBOL]', '.earnings [period|SYMBOL]'), COMPANY, ('.dividend', '.earnings'), + on_calendar, ('.dividend [period|SYMBOL]', '.earnings [period|SYMBOL]'), COMPANY, ('.earnings',), period=True), Command(('marketcap', 'maletas', 'marketer'), on_marketcap, ('.marketcap [SYMBOL|top|bottom]',), COMPANY, - ('.marketcap',), keywords=TOP), + keywords=TOP), Command(('beta',), on_beta, ('.beta',), PRICE), - Command(('compare',), on_compare, ('.compare SYMBOL SYMBOL [... up to 6] [period]',), PRICE, period=True, - keywords={'max': 'max', 'all': 'max'}), # max: as the default, 10 years + Command(('compare',), on_compare, ('.compare SYMBOL SYMBOL [... up to 6] [period]',), PRICE, ('.compare',), + period=True, keywords={'max': 'max', 'all': 'max'}), # max: as the default, 10 years Command(('history', 'hospital', 'visual', 'hosiery'), on_history, ('.history SYMBOL',), PRICE), Command(('performance', 'performanc'), on_performance, ('.performance [period] [portfolio]',), PRICE, ('.performance',), period=True), @@ -566,7 +656,7 @@ class Command: Command(('session', 'premarket', 'postmarket', 'permarket'), on_session, ('.session [percent|SYMBOL]', '.premarket [percent|SYMBOL]'), PRICE, ('.session', '.premarket'), percent=True), Command(BUY_SELL, on_rating, ('.buy', '.sell'), RISK, ('.buy & sell',)), # .buy & sell (or .buy sell): both - Command(('pe',), on_ratio('pe'), ('.pe [SYMBOL|top|bottom]',), RISK, ('.pe',), keywords=TOP), + Command(('pe',), on_ratio('pe'), ('.pe [SYMBOL|top|bottom]',), RISK, keywords=TOP), Command(('forward pe', 'forwardpe', 'fpe'), on_ratio('forward pe'), ('.forward pe [SYMBOL|top|bottom|negative]',), RISK, ('.forward pe',), keywords=NEGATIVE), Command(('peg',), on_ratio('peg'), ('.peg [SYMBOL|top|bottom|negative]',), RISK, ('.peg',), keywords=NEGATIVE), diff --git a/tests/test_charts.py b/tests/test_charts.py index dcda72d7..3d04613f 100644 --- a/tests/test_charts.py +++ b/tests/test_charts.py @@ -458,3 +458,9 @@ def test_pie(self): if __name__ == '__main__': unittest.main() + + +class BlankImage(unittest.TestCase): + def test_the_picker_image_is_one_pixel(self): + """The image of a picker's message is a 1x1 PNG, which the answer's chart replaces.""" + self.assertEqual(decode(render.blank()).size, (1, 1)) diff --git a/tests/test_discord.py b/tests/test_discord.py index 00f982b6..ef623901 100644 --- a/tests/test_discord.py +++ b/tests/test_discord.py @@ -172,7 +172,7 @@ def request(method, url, **kwargs): def test_a_message_has_no_previews_or_pings_and_its_buttons_in_rows_of_five(self): """A message suppresses link previews and pings nobody; its buttons go in rows of five, the one shown now - blurple, and past 25 they are left out.""" + blurple, a row that fits stays a row, and past five rows they are left out.""" row = [buttons.Button(f'B{i}', f'c|h|w|{i}', active=i == 0) for i in range(12)] with contextlib.redirect_stderr(io.StringIO()): discord_common.DiscordBot('77').text(['hello'], [row[:7], row[7:]]) @@ -181,9 +181,11 @@ def test_a_message_has_no_previews_or_pings_and_its_buttons_in_rows_of_five(self payload = kwargs['json'] self.assertEqual((payload['content'], payload['flags'], payload['allowed_mentions']), ('hello', 4, {'parse': []})) - self.assertEqual([len(r['components']) for r in payload['components']], [5, 5, 2]) + self.assertEqual([len(r['components']) for r in payload['components']], [5, 2, 5]) self.assertEqual([b['style'] for b in payload['components'][0]['components']], [1, 2, 2, 2, 2]) self.assertEqual(len(discord_common.components([[buttons.Button('x', 'y')] * 30])), 5) + self.assertEqual([len(r['components']) for r in discord_common.components([[buttons.Button('x', 'y')] * 5, + [buttons.Button('x', 'y')] * 2])], [5, 2]) def test_a_chart_is_attached_to_its_report(self): """A chart goes as an attachment of the message that carries the report and the buttons.""" diff --git a/tests/test_edges.py b/tests/test_edges.py index 353fdc04..887b6a59 100644 --- a/tests/test_edges.py +++ b/tests/test_edges.py @@ -133,7 +133,7 @@ def test_removing_a_ticker_yahoo_no_longer_knows(self): class BadArguments(FinbotCase): def test_compare_argument_errors_give_usage(self): """.compare with too few or too many tickers, or one twice, gets its usage.""" - for cmd in ('.compare', '.compare T1', '.compare T1 T2 T3 T4 T5 T6 T7', '.compare T1 T1'): + for cmd in ('.compare T1', '.compare T1 T2 T3 T4 T5 T6 T7', '.compare T1 T1'): with self.subTest(cmd=cmd): self.assertIn('Usage', str(self.command(cmd))) @@ -152,9 +152,9 @@ def no_history(t, *a, **k): self.assertEqual((self.sent, self.errors), ([('text', ['no price history for T1'])], [])) def test_history_without_a_ticker_or_with_extra_argument_gives_usage(self): - """.history with no ticker or an extra argument gets its usage, not the profile of a ticker called HISTORY.""" - # not the profile of a ticker called HISTORY - self.assertEqual(self.command('.history'), [('text', ['Usage: .history SYMBOL'])]) + """.history with an extra argument gets its usage, not the profile of a ticker called HISTORY; with no + ticker, the buttons for one.""" + self.assertEqual(self.command('.history')[0][0], 'photo') self.assertEqual(self.command('.history T1 1y'), [('text', ['Usage: .history SYMBOL'])]) diff --git a/tests/test_reports.py b/tests/test_reports.py index c3ba5fee..5d03a927 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -62,8 +62,8 @@ def test_the_keyboard_is_alphabetical_and_every_button_is_its_command(self): command it belongs to in the table, not taken for a ticker by .SYMBOL.""" buttons = [button['text'] for row in worker.command_keyboard()['keyboard'] for button in row] self.assertEqual(buttons, sorted(buttons)) - self.assertEqual(buttons, ['.buy & sell', '.dividend', '.earnings', '.forward pe', '.marketcap', '.pe', '.peg', - '.performance', '.premarket', '.price', '.schedule', '.session', '.shorts', '.trades', '.watchlist']) + self.assertEqual(buttons, ['.buy & sell', '.compare', '.earnings', '.forward pe', '.peg', '.performance', + '.pie', '.premarket', '.price', '.schedule', '.session', '.shorts', '.trades', '.treemap', '.watchlist']) for command in worker.COMMANDS: for text in command.buttons: with self.subTest(text=text): @@ -114,10 +114,18 @@ def test_a_portfolio_with_no_holdings_is_reported_by_name(self): self.assertTrue(any('>Rob' in line for line in payload), payload) def test_a_portfolio_command_without_a_portfolio_lists_them(self): - """.holdings and .pie without a portfolio say how to ask, with the portfolios there are.""" + """.holdings, .pie and .treemap without a portfolio give buttons for the portfolios there are, and name them + where there are no buttons (a job's).""" + chat = delivery.chat('telegram:55', scheduled=True) for text in ('.holdings', '.pie', '.treemap'): with self.subTest(text=text): - self.assertIn('Portfolios: Rob', str(self.command(text))) + self.sent.clear() + self.command(text) + self.assertEqual(self.sent[0][0], 'text' if text == '.holdings' else 'photo') + self.sent.clear() + worker.job_commands()[text[1:]].run(chat, words=[]) + usage = f"Usage: {text} PORTFOLIO" + ('' if text == '.holdings' else ' [sector]') + self.assertEqual(self.sent, [('text', [usage, 'Portfolios: Rob'])]) def test_prepare_marketcap(self): """The market cap list is built.""" @@ -764,7 +772,7 @@ def test_folds_into_other_and_renders(self): def test_unknown_or_missing_portfolio_lists_known(self): """.pie with no portfolio, or an unknown one, lists the portfolios there are.""" - self.assertEqual(self.command('.pie'), [('text', ['Usage: .pie PORTFOLIO [sector]', 'Portfolios: Rob'])]) + self.assertEqual(self.command('.pie')[0][0], 'photo') # the buttons of the portfolios self.assertEqual(self.command('.pie Nope'), [('text', ['Nope portfolio not found. I only know about:', 'Rob'])]) def test_with_the_graph_preference_off_a_chart_goes_as_text(self): @@ -821,8 +829,7 @@ def test_nothing_with_a_value(self): def test_unknown_or_missing_portfolio_lists_known(self): """.treemap with no portfolio, or an unknown one, lists the portfolios there are.""" - self.assertEqual(self.command('.treemap'), - [('text', ['Usage: .treemap PORTFOLIO [sector]', 'Portfolios: Rob'])]) + self.assertEqual(self.command('.treemap')[0][0], 'photo') # the buttons of the portfolios self.assertEqual(self.command('.treemap Nope'), [('text', ['Nope portfolio not found. I only know about:', 'Rob'])]) @@ -835,13 +842,15 @@ def test_a_command_job_posts_what_the_typed_command_posts(self): cannot take its words posts its usage.""" chat = delivery.chat('telegram:55', scheduled=True) for name, words, typed in (('pe', [], '.pe'), ('beta', [], '.beta'), ('buy&sell', [], '.buy & sell'), - ('compare', ['T1', 'T2'], '.compare T1 T2'), ('history', [], '.history')): + ('compare', ['T1', 'T2'], '.compare T1 T2')): with self.subTest(name=name): posted = self.command(typed) self.sent.clear() worker.job_commands()[name].run(chat, words=words) self.assertEqual(self.sent, posted) - self.assertEqual(posted, [('text', ['Usage: .history SYMBOL'])]) + self.sent.clear() + worker.job_commands()['history'].run(chat, words=[]) + self.assertEqual(self.sent, [('text', ['Usage: .history SYMBOL'])]) # a job has no buttons to pick with class WithoutSharesight(FinbotCase): @@ -868,3 +877,150 @@ def test_what_needs_sharesight_says_it_is_not_set_up(self): if __name__ == '__main__': unittest.main() + + +class Pickers(FinbotCase): + """A command typed with nothing to say what it is about answers with buttons: one message, edited as they are + pressed.""" + + def setUp(self): + """Telegram's edits and toasts of a press are noted: self.edits (kind, lines, buttons by row), self.toasts.""" + super().setUp() + self.edits, self.toasts, self.shown = [], [], [] + bot_class = telegram_common.TelegramBot + def edited(kind, lines, buttons): + """Note an edit, which is the screen shown now.""" + self.edits.append((kind, lines, self.rows(buttons))) + self.shown = self.edits[-1][2] + def posted(dest, lines, buttons=None): + """Transport.text in these tests: noted as the others are, and its buttons are the screen shown now.""" + self.shown = self.rows(buttons) + return self.record_text(dest, lines) + real_chart = transports.Transport.chart + self.patch(bot_class, 'edit_chart', lambda dest, msg, image, caption, buttons=None: + edited('photo', caption.split('\n'), buttons)) + self.patch(bot_class, 'edit_text', lambda dest, msg, text, buttons=None: + edited('text', text.split('\n'), buttons)) + self.patch(transports.Transport, 'text', posted) + self.patch(transports.Transport, 'chart', lambda dest, image, caption, markup=None: + (setattr(self, 'shown', self.rows(markup)), self.record_chart(dest, image, caption, markup))[1]) + self.patch(telegram_common, 'answerCallbackQuery', lambda cid, text=None, alert=False: + self.toasts.append((text, alert))) + + @staticmethod + def rows(buttons): + """Rows of (label, data) of the buttons a message was left with.""" + return [[(b.label if not b.active else '● ' + b.label, b.data) for b in row] for row in buttons or []] + + def sent_rows(self): + """The buttons of the picker as shown now (posted or edited), by row: (label, data).""" + return self.shown + + def tap(self, label, rows=None): + """Press the button with this label (as marked, '✅ T1' or '● Max') in the last screen shown.""" + rows = rows or self.shown + data = next(data for row in rows for text, data in row if text == label) + self.press_button(data) + self.assertEqual(self.errors, []) + return self.edits[-1] + + def test_a_portfolio_command_offers_the_portfolios(self): + """.pie, .treemap and .holdings: a button for each portfolio; .treemap and its Group by button toggle.""" + self.command('.pie') + self.assertEqual(self.sent[-1], ('photo', self.sent[-1][1], 'Pie: pick a portfolio')) + self.assertEqual([[text for text, _ in row] for row in self.sent_rows()], [['Rob']]) + self.command('.treemap sector') + self.assertEqual([[text for text, _ in row] for row in self.sent_rows()], [['Group by: Sector'], ['Rob']]) + kind, lines, rows = self.tap('Group by: Sector') + self.assertEqual((kind, [[text for text, _ in row] for row in rows]), + ('photo', [['Group by: Industry'], ['Rob']])) + self.command('.holdings') + self.assertEqual((self.sent[-1][0], [[t for t, _ in row] for row in self.shown]), ('text', [['Rob']])) + + def test_a_portfolio_button_turns_its_message_into_the_answer(self): + """.pie, .treemap (by the group chosen) and .holdings: the pressed message becomes the chart, or the list.""" + self.command('.treemap') + kind, lines, rows = self.tap('Group by: Industry') + kind, lines, rows = self.tap('Rob') + self.assertEqual((kind, lines[0], rows), ('photo', 'Rob treemap by sector', [])) + self.command('.pie') + self.assertEqual(self.tap('Rob')[:2], ('photo', ['Rob by industry', 'Unclassified: 100.0%'])) + self.patch(holdings, 'prepare_holdings_payload', lambda name, service: + [f'Holdings for {name} (25)', 'x']) + self.command('.holdings') + kind, lines, rows = self.tap('Rob') + self.assertEqual((kind, lines, rows), ('text', ['Holdings for Rob (25)', 'x'], [])) + self.assertEqual(self.toasts[-1], ('Loading…', False)) + + def test_history_picks_one_ticker_in_a_sub_menu(self): + """.history has a button for its tickers' screen, whose buttons are the tickers held, and Back.""" + self.command('.history') + self.assertEqual([[t for t, _ in row] for row in self.sent_rows()], [['Tickers ▶']]) + kind, lines, rows = self.tap('Tickers ▶') + self.assertEqual((kind, [t for t, _ in rows[0]], [t for t, _ in rows[-1]]), + ('photo', ['T0', 'T1', 'T10', 'T11'], ['◀ Back'])) + self.assertEqual(self.tap('◀ Back')[2][0][0][0], 'Tickers ▶') + self.tap('Tickers ▶') + kind, lines, rows = self.tap('T3') + self.assertEqual(kind, 'photo') + self.assertTrue(rows and rows[0][0][1].startswith('c|h|'), rows) # the chart's period buttons + + def test_compare_ticks_up_to_six_and_refuses_a_seventh(self): + """.compare: ticks toggle, Compare appears with two, a seventh is refused with an alert and a note.""" + self.command('.compare') + self.assertEqual([[t for t, _ in row] for row in self.sent_rows()], + [['7D', '1M', '1Y', '● Max'], ['Tickers ▶']]) + self.tap('Tickers ▶') + for ticker in ('T0', 'T1', 'T2', 'T3', 'T4', 'T5'): + kind, lines, rows = self.tap(ticker) + labels = [t for row in rows for t, _ in row] + self.assertIn('✅ T5', labels) + self.assertEqual(lines[-1], 'Ticked: T0, T1, T2, T3, T4, T5') + self.assertIn('Compare', labels) + self.assertEqual(self.toasts[-1], ('Updating…', False)) + kind, lines, rows = self.tap('T6') + self.assertEqual((self.toasts[-1], lines[0]), ((buttons.TOO_MANY, True), buttons.TOO_MANY)) + self.assertNotIn('✅ T6', [t for row in rows for t, _ in row]) + kind, lines, rows = self.tap('✅ T5') # unticking makes room + self.assertNotIn(buttons.TOO_MANY, lines) + self.assertEqual(lines[-1], 'Ticked: T0, T1, T2, T3, T4') + + def test_compare_turns_its_message_into_the_chart_of_the_ticked(self): + """Compare with the period chosen edits the picker into the chart, with the chart's own period buttons.""" + self.command('.compare 1y') + self.assertIn('● 1Y', [t for row in self.sent_rows() for t, _ in row]) + self.tap('Tickers ▶') + self.tap('T1') + self.assertNotIn('Compare', [t for row in self.edits[-1][2] for t, _ in row]) # one is too few + self.tap('T2') + kind, lines, rows = self.tap('Compare') + self.assertEqual((kind, rows[0][2]), ('photo', ('● 1Y', 'c|c|y|T1,T2'))) + self.assertEqual(self.toasts[-1], ('Loading…', False)) + + def test_buttons_of_another_list_start_again(self): + """A button drawn for other tickers than there are now starts the picker again, saying so.""" + self.command('.compare') + stale = self.shown[1][0][1] # Tickers ▶ + self.patch(sharesight, 'tickers', lambda: ['A1', 'B1']) + self.press_button(stale) + kind, lines, rows = self.edits[-1] + self.assertEqual(lines[0], 'The list has changed: here it is now') + + def test_without_holdings_the_watchlist_is_offered(self): + """Tickers to pick are the holdings', else the chat's watchlist; none at all says how to add one.""" + self.patch(sharesight, 'tickers', lambda: []) + self.patch(watchlist, 'tickers', lambda chat: ['MSFT', 'AAPL']) + self.command('.history') + self.tap('Tickers ▶') + self.assertEqual([t for row in self.edits[-1][2] for t, _ in row], ['AAPL', 'MSFT', '◀ Back']) + self.patch(watchlist, 'tickers', lambda chat: []) + self.assertEqual(self.command('.compare'), + [('text', ['No tickers to pick from: .watchlist add SYMBOL adds one'])]) + + def test_with_the_graph_off_the_picker_is_text_and_edits_as_text(self): + """Where charts are not drawn, the picker is a text message, and its answer the chart's lines as text.""" + self.prefs(graph=False) + self.command('.pie') + self.assertEqual(self.sent[-1][0], 'text') + kind, lines, rows = self.tap('Rob') + self.assertEqual((kind, lines[0], rows), ('text', 'Rob by industry', [])) diff --git a/tests/test_util.py b/tests/test_util.py index 96b9232f..8c6a06a4 100644 --- a/tests/test_util.py +++ b/tests/test_util.py @@ -162,6 +162,20 @@ def test_button_data_reads_back(self): with self.subTest(data=data): self.assertIsNone(buttons.parse(data)) + def test_a_pickers_data_reads_back_in_few_bytes(self): + """A picker's button says what it asks for, with up to 32 ticked tickers in 64 bytes; data of no picker, with + an act or a command it has not, is nothing, and only a refused tick is an alert.""" + press = buttons.PickPress('c', 's', 'x', 2 ** 32 - 1, 'r31', 'a1b2') + button = buttons.pick_button('x' * 99, press) + self.assertEqual((buttons.parse(button.data), len(button.label), len(button.data)), (press, 40, 25)) + for data in ('k|z|0|x|0|p|a1b2', 'k|c|0|x|0|q|a1b2', 'k|c|0|x|zz|p|a1b2', 'k|c|0|q|0|p|a1b2', 'k|c|0|x|0|p', + 'k|c|0|x|0|p|a-b2'): + with self.subTest(data=data): + self.assertIsNone(buttons.parse(data)) + self.assertEqual([buttons.toast(f'k|c|0|x|3|{act}|a1b2') for act in ('p', 'm', 'r2', 'g', 'f')], + ['Updating…', 'Updating…', 'Loading…', 'Loading…', buttons.TOO_MANY]) + self.assertEqual([buttons.alert(f'k|c|0|x|3|{act}|a1b2') for act in ('m', 'f')], [False, True]) + def test_a_press_is_told_what_is_coming_from_its_data_alone(self): """The toast a press gets as it arrives: the period a period button loads, 'Updating…' for the .jobs menu's, 'Running…' or 'Switching…' for an older job button (the worker checks them later), and none for other data.""" diff --git a/util/record_outputs.py b/util/record_outputs.py index 458e9cf1..3f343b61 100755 --- a/util/record_outputs.py +++ b/util/record_outputs.py @@ -22,7 +22,7 @@ '.price 7d', '.price 3', '.price 3 7d', '.price top', '.price top 7d', '.price T1', '.price T1 7d', '.session', '.session T1', '.premarket', '.postmarket', '.shorts', '.shorts 30', '.shorts T1', '.performance', '.performance 7d', '.performance 1y', '.performance Nope', '.pie Rob', '.pie Rob sector', '.pie', '.treemap Rob', - '.treemap Rob sector', '.treemap', '.compare T1 T2', + '.treemap Rob sector', '.treemap', '.history', '.compare', '.holdings', '.compare T1 T2', '.compare T1 T2 3m', '.compare T1', '.history T1', '.history', '.watchlist', '.watchlist add MSFT', '.watchlist add MSFT', '.watchlist del MSFT', '.watchlist frob X', '.T1', '.jobs', '.jobs run', '.jobs toggle', '.jobs enable', '.jobs disable 1', '.jobs add', '.jobs add shorts daily at 22:00', '.jobs del 4', '.jobs run 1', From 7e7eb5264707f422a4bbd1ead8b797de18e10048 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 14:42:14 +0000 Subject: [PATCH 30/41] Pickers: 32 items a page with < and >, portfolios four to a row with Group by after, buttons kept under the answer A picker's buttons stay under its answer (the item shown marked), so another item, the Sector/Industry toggle or a period redoes it in place; a Slack chart message keeps its button rows; the log prefix reads 'Telegram @user' not '[Telegram]: @user'. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 5 +- bot.py | 16 ++-- doc/commands.md | 2 +- lib/buttons.py | 30 +++++--- lib/reports/pick.py | 108 ++++++++++++++------------ lib/slack_common.py | 11 ++- lib/telegram_common.py | 2 +- lib/worker.py | 72 +++++++++--------- tests/test_edges.py | 4 +- tests/test_inbound.py | 2 +- tests/test_reports.py | 169 +++++++++++++++++++++++++++++++++-------- tests/test_sending.py | 7 ++ tests/test_util.py | 15 ++-- 13 files changed, 286 insertions(+), 157 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index d948d2e0..6ef4bde8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -86,8 +86,9 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm press's. - `lib/reports/pick.py` and `PickPress` (`lib/buttons.py`, `pick_button`): the buttons of `.pie .treemap .holdings .history .compare` typed bare (`worker.picker`, `process_pick_callback`): one message edited as it is pressed (a - 1x1 image, `render.blank()`, where its answer is a chart), its list of items checked by checksum, `.compare`'s ticks a - bit each in the data's mask, a seventh refused by an alert toast (`buttons.alert`). + 1x1 image, `render.blank()`, where its answer is a chart) that keeps its buttons under the answer (`shown`), 32 items + a page (`pick.layout`; fewer on Discord) turned by `<` `>`, its list of items checked by checksum, `.compare`'s + ticks a bit each in the data's mask, a seventh refused by an alert toast (`buttons.alert`). - `lib/buttons.py`: `Button`, a chart's period buttons (`keyboard`), the `.jobs` menu's (`menu_button`, a `MenuPress`; older lists' `j|`/`e|` buttons still parse), what a pressed button's data asks for (`parse`), and the toast it gets at once (`toast`); each transport draws them, and Slack and Discord take a button's data as its id, so diff --git a/bot.py b/bot.py index 4ee73883..90fecdd9 100755 --- a/bot.py +++ b/bot.py @@ -127,18 +127,18 @@ def print_headers(say=log.info): return respond(inbound.FORBIDDEN) length = environ.get('CONTENT_LENGTH') or '0' if not length.isdigit() or int(length) > MAX_BODY: - log.warn(f"[{source.service}]: a body of {length!r} bytes refused unread") + log.warn(f"{source.service.capitalize()} a body of {length!r} bytes refused unread") return respond(inbound.TOO_LARGE if length.isdigit() else inbound.BAD_REQUEST) body = environ['wsgi.input'].read(int(length)) # the body and no more: a server need not end the stream there try: update = source.decode(body) except ValueError: - log.warn(f"[{source.service}]: request body is not a JSON object") + log.warn(f"{source.service.capitalize()} request body is not a JSON object") return respond(inbound.BAD_REQUEST) if why := source.refuse(environ, body, update): shown = {k: '' if k == 'token' else v for k, v in update.items()} # Slack's verification token log.warn("warning:", why) - log.warn(f"[{source.service}]: refused update", json.dumps(shown, indent=4, default=str)) + log.warn(f"{source.service.capitalize()} refused update", json.dumps(shown, indent=4, default=str)) return respond(source.REFUSED) environ[AUTHENTICATED] = True return respond(take(source.network, update)) @@ -198,11 +198,11 @@ def take(network, update): if isinstance(event, inbound.Suggest): # answered here and at once, while the command is typed return network.suggestions(worker.suggest(f'{event.service}:{event.chat_id}', event.text)) if seen_before(network.service, network.update_id(update)): - log.info(f"[{network.service}]: update {network.update_id(update)} seen already, not run again") + log.info(f"{network.service.capitalize()} update {network.update_id(update)} seen already, not run again") return inbound.OK if isinstance(event, inbound.Message) and not event.interaction and doubled(event): # each interaction is answered - log.info(log.cut(f"[{network.service}]: {event.user} sent it again within {DOUBLED} s, not run again: " - + event.text)) + log.info(log.cut(f"{network.service.capitalize()} {event.user} sent it again within {DOUBLED} s, not run " + "again: " + event.text)) return inbound.OK answer = network.acknowledge(event) if answer is not None: @@ -213,10 +213,10 @@ def take(network, update): def dispatch(event): """Hand a message or a button press to lib/worker, in a thread of its own, which it returns.""" if isinstance(event, inbound.Press): - log.debug(f"[{event.service.capitalize()} button]:", event.user, "chat", event.chat_id, event.data) + log.debug(f"{event.service.capitalize()} button", event.user, "chat", event.chat_id, event.data) target = worker.process_callback else: - log.info(log.cut(f"[{event.service.capitalize()}]: {event.user} {event.text}")) + log.info(log.cut(f"{event.service.capitalize()} {event.user} {event.text}")) target = worker.process_request thread = threading.Thread(target=target, args=(event,)) thread.start() diff --git a/doc/commands.md b/doc/commands.md index 8ecde9c2..6dea27c6 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -73,7 +73,7 @@ The bot answers commands in every chat it is in. By default, it fetches its upda * A watch list per chat, which anyone in it can change, picked up by the reports there * Listing of portfolios and portfolio performance * A pie or a treemap of a portfolio by industry or sector (`/pie portfolio [sector]`, `/treemap portfolio [sector]`): the treemap tiles each holding by value, grouped and coloured by industry (the default) or sector -* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap`, `/holdings` (portfolios; `/treemap` also toggles Sector/Industry), `/history` (a ticker) and `/compare` (up to 6 ticked tickers and a period), the tickers those held (else the chat's watchlist), behind a `Tickers ▶` button. One message is edited as buttons are pressed, and becomes the answer. A job, or a chat with no buttons, gets the usage instead +* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap`, `/holdings` (portfolios; `/treemap` also toggles Sector/Industry), `/history` (a ticker) and `/compare` (up to 6 ticked tickers and a period), the tickers those held (else the chat's watchlist), behind a `Tickers ▶` button. Portfolios come four to a row, `/treemap`'s Sector/Industry toggle after them; thirty-two items make a page, with `<` and `>` to turn it (fewer on Discord, which holds five rows of five buttons). One message is edited as buttons are pressed and becomes the answer, the buttons staying under it with the item shown marked: another item, the toggle or a period redoes the answer in place. A job, or a chat with no buttons, gets the usage instead * Listing of holdings sorted by various attributes * Running the other stock reports on demand diff --git a/lib/buttons.py b/lib/buttons.py index 73f7a95e..a7823f32 100644 --- a/lib/buttons.py +++ b/lib/buttons.py @@ -40,13 +40,16 @@ class MenuPress(typing.NamedTuple): class PickPress(typing.NamedTuple): """A press in a picker (lib/reports/pick.py): the command it picks for (a key of PICKERS), how it groups (a - treemap's or pie's: i industry, s sector, 0 neither), the period (a key of PERIODS, or 0), the items ticked (a - bit of the mask each), what the press asks for (act: p the first screen, m the tickers' screen, r then an item's - number to run it, g to run the ticked ones, f a tick refused, as too many) and the checksum of the list of - items it was drawn for, stale once that changes.""" + treemap's or pie's: i industry, s sector, 0 neither), the period (a key of PERIODS, or 0), what the message shows + (shown: 0 nothing yet, an item's number plus one, or 1 for the ticked tickers), the items ticked (a bit of the mask + each), what the press asks for (act: a letter and a number: p the first screen and m the tickers' screen, each at + a page; f a tick refused, as too many, the tickers' screen at a page; r or s an item's number, to run it and show + the tickers' screen or the first; g the ticked ones, to run and show the tickers' screen at a page; h to run them + and show the first screen) and the checksum of the list of items it was drawn for, stale once that changes.""" cmd: str group: str period: str + shown: int mask: int act: str checksum: str @@ -110,8 +113,9 @@ def menu_button(label, screen, args, checksum): return Button(label[:40], '|'.join(['m', screen, *map(str, args), checksum])) def pick_button(label, press, active=False): - """A picker's button: its label (cut to 40 characters) and a PickPress as its data: 'k|c|0|x|3|p|a1b2'.""" - data = '|'.join(['k', press.cmd, press.group, press.period, format(press.mask, 'x'), press.act, press.checksum]) + """A picker's button: its label (cut to 40 characters) and a PickPress as its data: 'k|c|0|x|1|3|p0|a1b2'.""" + data = '|'.join(['k', press.cmd, press.group, press.period, format(press.shown, 'x'), format(press.mask, 'x'), + press.act, press.checksum]) return Button(label[:40], data, active) def parse(data): @@ -122,11 +126,13 @@ def parse(data): kind, _, rest = data.partition('|') if kind == 'k': parts = rest.split('|') - if (len(parts) == 6 and parts[0] in PICKERS and parts[1] in ('0', 'i', 's') and (parts[2] in PERIODS or - parts[2] == '0') and len(parts[3]) <= 8 and parts[4][:1] in ('p', 'm', 'r', 'g', 'f') and - all(part.isascii() and part.isalnum() for part in parts) and len(data.encode()) <= MAX_CALLBACK_BYTES): + if (len(parts) == 7 and parts[0] in PICKERS and parts[1] in ('0', 'i', 's') and (parts[2] in PERIODS or + parts[2] == '0') and len(parts[3]) <= 3 and len(parts[4]) <= 25 and parts[5][:1] in 'pmfrsgh' and + parts[5][1:].isdigit() and all(part.isascii() and part.isalnum() for part in parts) and + len(data.encode()) <= MAX_CALLBACK_BYTES): try: - return PickPress(parts[0], parts[1], parts[2], int(parts[3], 16), parts[4], parts[5]) + return PickPress(parts[0], parts[1], parts[2], int(parts[3], 16), int(parts[4], 16), parts[5], + parts[6]) except ValueError: return None return None @@ -147,7 +153,7 @@ def parse(data): def alert(data): """Whether a press's toast is an alert the person must dismiss: a picker refusing a seventh ticker.""" pressed = parse(data) - return isinstance(pressed, PickPress) and pressed.act == 'f' + return isinstance(pressed, PickPress) and pressed.act[0] == 'f' def toast(data): """What a press is told at once, from the button's data alone (the network acknowledges a press before finbot @@ -156,7 +162,7 @@ def toast(data): of them.""" pressed = parse(data) if isinstance(pressed, PickPress): - return TOO_MANY if pressed.act == 'f' else 'Loading…' if pressed.act[0] in 'rg' else 'Updating…' + return TOO_MANY if pressed.act[0] == 'f' else 'Loading…' if pressed.act[0] in 'rsgh' else 'Updating…' if isinstance(pressed, MenuPress): return 'Updating…' if isinstance(pressed, JobPress): diff --git a/lib/reports/pick.py b/lib/reports/pick.py index 2aab9873..7e97d0be 100644 --- a/lib/reports/pick.py +++ b/lib/reports/pick.py @@ -1,6 +1,6 @@ """The pickers: what a chat gets when it types .pie, .treemap, .holdings, .history or .compare with nothing to say what -it is about: buttons for the portfolios or tickers, one message edited as each is pressed (buttons.PickPress), the -last press turning it into the answer.""" +it is about: buttons for the portfolios or tickers, one message edited as each is pressed (buttons.PickPress). An +answer takes the message over and leaves the buttons under it, the item shown marked, so another can be picked.""" import hashlib from lib import answer @@ -9,9 +9,7 @@ from lib import sharesight from lib import watchlist -PER_ROW = {'portfolio': 3, 'ticker': 4} # buttons to a row of items -MOST_ITEMS = 32 # items offered; Discord has room for fewer (DISCORD_ITEMS) -DISCORD_ITEMS = 16 # four rows of four: a fifth row of its 5 holds the way back and the compare button +MOST_ITEMS = 100 # items offered, paged: what the mask of ticks in a button's data has room for CHARTS = 'ptyc' # the pickers whose answer is a chart def as_photo(chat, cmd): @@ -23,80 +21,90 @@ def noun(cmd): """What a picker picks: 'portfolio' or 'ticker'.""" return 'ticker' if cmd in 'yc' else 'portfolio' +def layout(service, cmd): + """(items to a page, items to a row) that a picker's screen has room for: 32 in rows of 4; Discord holds five rows + of five buttons, so a page leaves a row for the group and page buttons (12 portfolios, 20 tickers).""" + if service == 'discord': + return (20, 5) if noun(cmd) == 'ticker' else (12, 4) + return 32, 4 + def items(service, cmd, chat): """What a picker offers, sorted: portfolio names (a portfolio command), or tickers (history, compare): the ones held, or the chat's watchlist where it holds none. Raises answer.Answer where there are none.""" if noun(cmd) == 'portfolio': found = sorted(portfolio.name for portfolio in sharesight.portfolios()) else: - found = sharesight.tickers() or watchlist.tickers(chat) - found = sorted(found) + found = sorted(sharesight.tickers() or watchlist.tickers(chat)) if not found: raise answer.Answer("No tickers to pick from: .watchlist add SYMBOL adds one") - return found[:DISCORD_ITEMS if service == 'discord' else MOST_ITEMS] + return found[:MOST_ITEMS] def checksum(found): """A short stand-in for a list of items, which tells any change of it.""" return hashlib.sha1('\n'.join(found).encode()).hexdigest()[:4] -def press(cmd, group, period, mask, act, found): - """The PickPress a button of this picker sends, for the list `found` of items.""" - return buttons.PickPress(cmd, group, period, mask, act, checksum(found)) - def start(service, cmd, chat, group='0', period='0'): """A picker's first screen: (lines, rows).""" - return screen(service, cmd, items(service, cmd, chat), group, period, 0, 'p') + return screen(service, cmd, items(service, cmd, chat), group, period, 0, 0, 'p') -def screen(service, cmd, found, group, period, mask, view, note=None): - """A picker's screen, as (lines, rows): view 'p', its first (the portfolios, or for tickers the period and a - button for the tickers' screen), or 'm', the tickers'. `mask` has a bit for each ticked ticker.""" +def screen(service, cmd, found, group, period, shown, mask, view, page=0, note=None): + """A picker's screen, as (lines, rows): `view` 'p', the first (the portfolios; for tickers the period and the way + to the tickers), or 'm', the tickers'; `page` of the items; `shown` what the message answers now (an item's number + plus one, or 1 for the ticked tickers, 0 for nothing yet, which a button then carries to redo with its change); + `mask` has a bit for each ticked ticker.""" + per_page, per_row = layout(service, cmd) + pages = -(-len(found) // per_page) + page = min(max(page, 0), pages - 1) ticked = [name for i, name in enumerate(found) if mask >> i & 1] name = buttons.PICKERS[cmd] + def button(label, act, active=False, **changes): + """A button of this screen, whatever it changes (group, period, shown or mask) in its data.""" + state = {'group': group, 'period': period, 'shown': shown, 'mask': mask} | changes + return buttons.pick_button(label, buttons.PickPress(cmd, state['group'], state['period'], state['shown'], + state['mask'], act, checksum(found)), active) + redo = 'p0' if not shown else 'h0' if cmd == 'c' else f"s{shown - 1}" if cmd == 'y' else f"r{shown - 1}" rows = [] - if period != '0' and view == 'p': - rows.append([buttons.pick_button(buttons.PERIODS[key][0], press(cmd, group, key, mask, 'p', found), - key == period) for key in buttons.BUTTONS['c']]) - if cmd == 't' and view == 'p': - other = 's' if group != 's' else 'i' - rows.append([buttons.pick_button(f"Group by: {'Sector' if group == 's' else 'Industry'}", - press(cmd, other, period, mask, 'p', found))]) + if cmd in 'yc' and view == 'p': # the period, and the way to the tickers + rows.append([button(buttons.PERIODS[key][0], redo, key == period, period=key) for key in buttons.BUTTONS['c']]) + more = f"Tickers ({len(ticked)}) ▶" if ticked else "Tickers ▶" + rows.append([button(more, 'm0')] + ([button('Compare', 'h0')] if cmd == 'c' and len(ticked) >= 2 else [])) + shown_page = items_on(found, page, per_page) if noun(cmd) == 'portfolio': - heading = f"{service.bold(name.capitalize())}: pick a portfolio" - rows += chunks([buttons.pick_button(name, press(cmd, group, period, mask, f'r{i}', found)) - for i, name in enumerate(found)], PER_ROW['portfolio']) - elif view == 'p': - heading = f"{service.bold(name.capitalize())}: pick " + ('up to 6 tickers' if cmd == 'c' else 'a ticker') - label = f"Tickers ({len(ticked)}) ▶" if ticked else "Tickers ▶" - rows.append([buttons.pick_button(label, press(cmd, group, period, mask, 'm', found))]) - else: - heading = f"{service.bold(name.capitalize())}: pick " + ('up to 6 tickers' if cmd == 'c' else 'a ticker') + rows += chunks([button(name, f"r{i}", shown == i + 1, shown=i + 1) for i, name in shown_page], per_row) + if cmd == 't': + rows.append([button(f"Group by: {'Sector' if group == 's' else 'Industry'}", redo, + group='i' if group == 's' else 's')]) + elif view == 'm': full = len(ticked) >= buttons.MAX_PICKED tickers = [] - for i, ticker in enumerate(found): + for i, ticker in shown_page: if cmd == 'y': - tickers.append(buttons.pick_button(ticker, press(cmd, group, period, mask, f'r{i}', found))) + tickers.append(button(ticker, f"r{i}", shown == i + 1, shown=i + 1)) elif mask >> i & 1: - tickers.append(buttons.pick_button(f"✅ {ticker}", press(cmd, group, period, mask ^ 1 << i, 'm', found))) + tickers.append(button(f"✅ {ticker}", f"m{page}", mask=mask ^ 1 << i)) else: - tickers.append(buttons.pick_button(ticker, press(cmd, group, period, mask, 'f', found) if full else - press(cmd, group, period, mask | 1 << i, 'm', found))) - rows += chunks(tickers, PER_ROW['ticker']) - if cmd == 'c' and len(ticked) >= 2: - last = [buttons.pick_button('Compare', press(cmd, group, period, mask, 'g', found))] - else: - last = [] - if cmd in 'yc' and view == 'm': - last.insert(0, buttons.pick_button('◀ Back', press(cmd, group, period, mask, 'p', found))) - elif last: - rows.append(last) - last = [] - if last: - rows.append(last) - lines = ([note] if note else []) + [heading] + tickers.append(button(ticker, f"f{page}") if full else button(ticker, f"m{page}", mask=mask | 1 << i)) + rows += chunks(tickers, per_row) + nav = [button('◀ Back', 'p0')] if cmd in 'yc' and view == 'm' else [] + if pages > 1: + act = 'm' if view == 'm' else 'p' + nav += ([button('<', f"{act}{page - 1}")] if page else []) + ( + [button('>', f"{act}{page + 1}")] if page < pages - 1 else []) + if cmd == 'c' and view == 'm' and len(ticked) >= 2: + nav.append(button('Compare', f"g{page}")) + if nav: + rows.append(nav) + heading = f"{service.bold(name.capitalize())}: pick " + ( + 'a portfolio' if noun(cmd) == 'portfolio' else 'up to 6 tickers' if cmd == 'c' else 'a ticker') + lines = ([note] if note else []) + [heading + (f" (page {page + 1} of {pages})" if pages > 1 else '')] if ticked: lines.append(f"Ticked: {', '.join(ticked)}") return lines, rows +def items_on(found, page, per_page): + """(number, item) of the items on a page.""" + return list(enumerate(found))[page * per_page:(page + 1) * per_page] + def chunks(row, size): """A list of buttons as rows of at most `size`.""" return [row[i:i + size] for i in range(0, len(row), size)] diff --git a/lib/slack_common.py b/lib/slack_common.py index d0b8d19f..5d0aac4b 100644 --- a/lib/slack_common.py +++ b/lib/slack_common.py @@ -61,7 +61,7 @@ def parse(self, update): if kind == 'block_actions': return self.press(update) if kind != 'event_callback': - log.info(f"[slack]: unhandled 'type' {kind!r}") + log.info(f"Slack unhandled 'type' {kind!r}") return inbound.UNHANDLED event = update['event'] bot_user = update['authorizations'][0]['user_id'] # QWERTY @@ -69,7 +69,7 @@ def parse(self, update): event['user'] == bot_user: # not by a person: answering a bot (this one's replies included) could loop return inbound.OK if event['type'] not in ('message', 'app_mention') or 'text' not in event: - log.info("[slack]: unhandled event callback type", event['type']) + log.info("Slack unhandled event callback type", event['type']) return inbound.UNHANDLED user = '<@' + event['user'] + '>' # a mention: Slack shows the name return inbound.Message(self.service, event['channel'], user, self.plain(event['text']), '@' + bot_user, user, @@ -307,11 +307,10 @@ def actions_block(row): def chart_blocks(caption, buttons, file_id): - """Block Kit for a chart message: the report, the chart (an uploaded file, by its id) and the buttons in one - row.""" - row = [button for line in buttons for button in line] + """Block Kit for a chart message: the report, the chart (an uploaded file, by its id) and the buttons, a row of + them to a block.""" return section_blocks(caption) + [{'type': 'image', 'slack_file': {'id': file_id}, 'alt_text': 'chart'}] + \ - ([actions_block(row)] if row else []) + [actions_block(row) for row in buttons if row] def marked(buttons): diff --git a/lib/telegram_common.py b/lib/telegram_common.py index 520e9f62..5f0ae932 100644 --- a/lib/telegram_common.py +++ b/lib/telegram_common.py @@ -108,7 +108,7 @@ def parse(self, update): elif self.IGNORED & message.keys(): return inbound.OK else: - log.info("[telegram]: unhandled: 'message' without 'text/photo'") + log.info("Telegram unhandled: 'message' without 'text/photo'") return inbound.UNHANDLED return inbound.Message(self.service, chat_id, user, text, self.bot_name, user_name, str(message.get('message_id', ''))) diff --git a/lib/worker.py b/lib/worker.py index 6b68cb56..ff2dcfa6 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -149,62 +149,62 @@ def show_pick(chat, message, cmd, lines, rows): chat.edit_text(message, text, rows) def process_pick_callback(press, pressed): - """A press in a picker (an inbound.Press; pressed, the buttons.PickPress its data says): the screen it asks for, in - place of the pressed message, or a button of an item, or Compare, turning the message into the answer. Buttons - drawn for another list of items start the picker again, saying so.""" + """A press in a picker (an inbound.Press; pressed, the buttons.PickPress its data says): the screen it asks for in + place of the pressed message, or the answer for an item, or Compare, with the picker's buttons left under it (the + item shown marked). Buttons drawn for another list of items start the picker again, saying so.""" service = dialects.of(press.service) chat = delivery.reply_to(service, press.chat_id, interaction=press.interaction, pressed=True) - cmd = pressed.cmd + cmd, group, period, shown, mask = pressed.cmd, pressed.group, pressed.period, pressed.shown, pressed.mask try: found = pick.items(service, cmd, chat.key) stale = pick.checksum(found) != pressed.checksum - mask, act = (0, 'p') if stale else (pressed.mask, pressed.act) - note = 'The list has changed: here it is now' if stale else buttons.TOO_MANY if act == 'f' else None - if act[0] in 'rg' and pick_answer(service, chat, press.message_id, pressed, found, mask, act): - return - lines, rows = pick.screen(service, cmd, found, pressed.group, pressed.period, mask, 'm' if act in 'mf' else 'p', - note) + kind, number = (pressed.act[0], int(pressed.act[1:])) if not stale else ('p', 0) + if stale: + shown, mask = 0, 0 + per_page = pick.layout(service, cmd)[0] + view = 'm' if kind in 'mfg' or kind == 'r' and noun_is_ticker(cmd) else 'p' + page = number if kind in 'pmfg' else number // per_page if kind == 'r' else 0 + note = 'The list has changed: here it is now' if stale else buttons.TOO_MANY if kind == 'f' else None + names = None + if kind in 'rs' and number < len(found): + names, shown = [found[number]], number + 1 + elif kind in 'gh' and bin(mask).count('1') >= 2: + names, shown = [name for i, name in enumerate(found) if mask >> i & 1], 1 + lines, rows = pick.screen(service, cmd, found, group, period, shown, mask, view, page, note) + if names: + return pick_answer(service, chat, press.message_id, cmd, group, period, names, rows) show_pick(chat, press.message_id, cmd, lines, rows) except answer.Answer as e: delivery.deliver(chat, delivery.Report(e.lines)) except Exception as e: delivery.report_error(e, chat, context='pick') -def pick_answer(service, chat, message, pressed, found, mask, act): - """The pressed picker message turned into the answer of its command for the item (act r and its number) or the - ticked items (g) it asks for; False when there is nothing to answer yet (a number out of the list, fewer than two - to compare).""" - cmd = pressed.cmd - if act == 'g': - names = [name for i, name in enumerate(found) if mask >> i & 1] - elif act[1:].isdigit() and int(act[1:]) < len(found): - names = [found[int(act[1:])]] - else: - return False - if cmd == 'c' and len(names) < 2: - return False - by = 'sector' if pressed.group == 's' else 'industry' +def noun_is_ticker(cmd): + """Whether a picker picks tickers (history, compare), not portfolios.""" + return pick.noun(cmd) == 'ticker' + +def pick_answer(service, chat, message, cmd, group, period, names, rows): + """The pressed picker message turned into the answer of its command for the item (history, a portfolio's) or items + (compare), the picker's button rows left under it: a chart, or the holdings as text.""" + by = 'sector' if group == 's' else 'industry' if cmd == 'h': lines = holdings.prepare_holdings_payload(names[0], service) or [f"{names[0]} holds nothing"] parts = transports.chunked(lines, chat.dialect.text_limit) - chat.edit_text(message, '\n'.join(parts[0]), []) - for part in parts[1:]: - chat.text(part) - return True - markup = [] + chat.edit_text(message, '\n'.join(parts[0]), rows if len(parts) == 1 else []) + for i, part in enumerate(parts[1:], 1): + chat.text(part, rows if i == len(parts) - 1 else None) + return if cmd in 'pt': prepare = holdings.prepare_pie if cmd == 'p' else holdings.prepare_treemap lines, chart = prepare(names[0], by, service) caption = '\n'.join(lines) else: - period = pressed.period if cmd == 'c' else None - caption, chart = charts.build('c' if cmd == 'c' else 'h', names, service, chat.key, period=period) - markup = buttons.keyboard('c' if cmd == 'c' else 'h', names, period if cmd == 'c' else 'x') + days = period if period != '0' else 'x' + caption, chart = charts.build('c' if cmd == 'c' else 'h', names, service, chat.key, period=days) if pick.as_photo(chat, cmd): - chat.edit_chart(message, render.draw(chart) if chart and chart.rows else render.blank(), caption, markup) + chat.edit_chart(message, render.draw(chart) if chart and chart.rows else render.blank(), caption, rows) else: - chat.edit_text(message, caption, []) - return True + chat.edit_text(message, caption, rows) def process_callback(press): """A button pressed (an inbound.Press, which bot.take acknowledged as it arrived): the .jobs menu's, an older job @@ -502,7 +502,7 @@ def picker(request, cmd, args): if not (dest.takes_buttons and delivery.interactive(dest)): raise Usage(portfolios()) if pick.noun(cmd) == 'portfolio' else Usage group = {'sector': 's', 'industry': 'i'}.get(args.keyword, '0') - period = buttons.period_for_days(args.days, 'x') if cmd == 'c' else '0' + period = buttons.period_for_days(args.days, 'x') if cmd in 'cy' else '0' lines, rows = pick.start(request.service, cmd, dest.key, group, period) if pick.as_photo(dest, cmd): dest.chart(render.blank(), '\n'.join(lines), rows) diff --git a/tests/test_edges.py b/tests/test_edges.py index 887b6a59..e54d68a2 100644 --- a/tests/test_edges.py +++ b/tests/test_edges.py @@ -457,9 +457,9 @@ def logged(update_id, chat=55): bot.take(telegram_common.READER, update) self.join_workers() return err.getvalue().splitlines() - whole = '[Telegram]: @rob /jobs add message daily at 9am "Morning! The ASX opens at 10am, so coffee first"' + whole = 'Telegram @rob /jobs add message daily at 9am "Morning! The ASX opens at 10am, so coffee first"' self.assertEqual(logged(1), [whole[:70] + '…']) - self.assertEqual(logged(2), ['[telegram]: @rob sent it again within 1 s, not run again: /jobs add me…']) + self.assertEqual(logged(2), ['Telegram @rob sent it again within 1 s, not run again: /jobs add messa…']) self.prefs(log_width=0) self.assertEqual(logged(3, chat=66), [whole]) # another chat's: taken diff --git a/tests/test_inbound.py b/tests/test_inbound.py index 0336dd1d..3fe6bdf9 100644 --- a/tests/test_inbound.py +++ b/tests/test_inbound.py @@ -167,7 +167,7 @@ def test_button_presses_are_logged(self): err = io.StringIO() with contextlib.redirect_stderr(err): bot.main(wsgi_environ('/telegram', body, **secret), lambda *a, **k: None) - self.assertIn(f'[Telegram button]: {who} chat 55 j|7|abcd', err.getvalue()) + self.assertIn(f'Telegram button {who} chat 55 j|7|abcd', err.getvalue()) def test_a_press_is_acknowledged_once_before_the_worker_gets_it(self): """bot.take acknowledges a press, the one place that does: Telegram's with the toast its data makes, before the diff --git a/tests/test_reports.py b/tests/test_reports.py index 5d03a927..83efe444 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -4,6 +4,7 @@ import datetime import re from tests.common import * +from lib.reports import pick class Commands(FinbotCase): @@ -924,78 +925,102 @@ def tap(self, label, rows=None): self.assertEqual(self.errors, []) return self.edits[-1] + def labels(self, rows=None): + """The labels of a screen's buttons, by row (the screen shown now unless given).""" + return [[text for text, _ in row] for row in (self.shown if rows is None else rows)] + def test_a_portfolio_command_offers_the_portfolios(self): - """.pie, .treemap and .holdings: a button for each portfolio; .treemap and its Group by button toggle.""" + """.pie, .treemap and .holdings: a button for each portfolio; .treemap's Group by button comes after them.""" self.command('.pie') self.assertEqual(self.sent[-1], ('photo', self.sent[-1][1], 'Pie: pick a portfolio')) - self.assertEqual([[text for text, _ in row] for row in self.sent_rows()], [['Rob']]) + self.assertEqual(self.labels(), [['Rob']]) self.command('.treemap sector') - self.assertEqual([[text for text, _ in row] for row in self.sent_rows()], [['Group by: Sector'], ['Rob']]) - kind, lines, rows = self.tap('Group by: Sector') - self.assertEqual((kind, [[text for text, _ in row] for row in rows]), - ('photo', [['Group by: Industry'], ['Rob']])) + self.assertEqual(self.labels(), [['Rob'], ['Group by: Sector']]) self.command('.holdings') - self.assertEqual((self.sent[-1][0], [[t for t, _ in row] for row in self.shown]), ('text', [['Rob']])) + self.assertEqual((self.sent[-1][0], self.labels()), ('text', [['Rob']])) - def test_a_portfolio_button_turns_its_message_into_the_answer(self): - """.pie, .treemap (by the group chosen) and .holdings: the pressed message becomes the chart, or the list.""" + def test_portfolios_come_four_to_a_row(self): + """Portfolios are in rows of four before the Group by row.""" + names = [f'P{i}' for i in range(6)] + self.patch(sharesight, '_portfolios', lambda seconds=None: [{'name': n, 'id': i} for i, n in enumerate(names)]) + self.command('.treemap') + self.assertEqual(self.labels(), [names[:4], names[4:], ['Group by: Industry']]) + + def test_the_answer_keeps_the_buttons_and_the_group_toggle_redraws_it(self): + """A portfolio button turns its message into the chart, the buttons still under it with that portfolio + marked; the Group by button and another portfolio redo it in place.""" self.command('.treemap') - kind, lines, rows = self.tap('Group by: Industry') kind, lines, rows = self.tap('Rob') - self.assertEqual((kind, lines[0], rows), ('photo', 'Rob treemap by sector', [])) + self.assertEqual((kind, lines[0], self.labels(rows)), ('photo', 'Rob treemap by industry', + [['● Rob'], ['Group by: Industry']])) + kind, lines, rows = self.tap('Group by: Industry') + self.assertEqual((kind, lines[0], self.labels(rows)), ('photo', 'Rob treemap by sector', + [['● Rob'], ['Group by: Sector']])) + self.assertEqual(self.toasts[-1], ('Loading…', False)) self.command('.pie') - self.assertEqual(self.tap('Rob')[:2], ('photo', ['Rob by industry', 'Unclassified: 100.0%'])) + kind, lines, rows = self.tap('Rob') + self.assertEqual((kind, lines[:2], self.labels(rows)), ('photo', ['Rob by industry', + 'Unclassified: 100.0%'], [['● Rob']])) + + def test_a_holdings_answer_keeps_its_buttons(self): + """.holdings: the list as text, the portfolio buttons still under it.""" self.patch(holdings, 'prepare_holdings_payload', lambda name, service: [f'Holdings for {name} (25)', 'x']) self.command('.holdings') kind, lines, rows = self.tap('Rob') - self.assertEqual((kind, lines, rows), ('text', ['Holdings for Rob (25)', 'x'], [])) - self.assertEqual(self.toasts[-1], ('Loading…', False)) + self.assertEqual((kind, lines, self.labels(rows)), ('text', ['Holdings for Rob (25)', 'x'], [['● Rob']])) def test_history_picks_one_ticker_in_a_sub_menu(self): - """.history has a button for its tickers' screen, whose buttons are the tickers held, and Back.""" + """.history has a period row and a button for its tickers' screen, whose buttons are the tickers held, and + Back; a ticker turns the message into its chart, the screen kept with that ticker marked.""" self.command('.history') - self.assertEqual([[t for t, _ in row] for row in self.sent_rows()], [['Tickers ▶']]) + self.assertEqual(self.labels(), [['7D', '1M', '1Y', '● Max'], ['Tickers ▶']]) kind, lines, rows = self.tap('Tickers ▶') - self.assertEqual((kind, [t for t, _ in rows[0]], [t for t, _ in rows[-1]]), + self.assertEqual((kind, self.labels(rows)[0], self.labels(rows)[-1]), ('photo', ['T0', 'T1', 'T10', 'T11'], ['◀ Back'])) - self.assertEqual(self.tap('◀ Back')[2][0][0][0], 'Tickers ▶') + self.assertEqual(self.labels(self.tap('◀ Back')[2])[1], ['Tickers ▶']) self.tap('Tickers ▶') kind, lines, rows = self.tap('T3') self.assertEqual(kind, 'photo') - self.assertTrue(rows and rows[0][0][1].startswith('c|h|'), rows) # the chart's period buttons + self.assertIn('● T3', [t for row in self.labels(rows) for t in row]) # the tickers still there, T3 marked + kind, lines, rows = self.tap('◀ Back') + kind, lines, rows = self.tap('1Y') # a period redoes the chart for T3 + self.assertEqual((self.toasts[-1], self.labels(rows)[0]), (('Loading…', False), ['7D', '1M', '● 1Y', 'Max'])) def test_compare_ticks_up_to_six_and_refuses_a_seventh(self): """.compare: ticks toggle, Compare appears with two, a seventh is refused with an alert and a note.""" self.command('.compare') - self.assertEqual([[t for t, _ in row] for row in self.sent_rows()], - [['7D', '1M', '1Y', '● Max'], ['Tickers ▶']]) + self.assertEqual(self.labels(), [['7D', '1M', '1Y', '● Max'], ['Tickers ▶']]) self.tap('Tickers ▶') for ticker in ('T0', 'T1', 'T2', 'T3', 'T4', 'T5'): kind, lines, rows = self.tap(ticker) - labels = [t for row in rows for t, _ in row] + labels = [t for row in self.labels(rows) for t in row] self.assertIn('✅ T5', labels) self.assertEqual(lines[-1], 'Ticked: T0, T1, T2, T3, T4, T5') self.assertIn('Compare', labels) self.assertEqual(self.toasts[-1], ('Updating…', False)) kind, lines, rows = self.tap('T6') self.assertEqual((self.toasts[-1], lines[0]), ((buttons.TOO_MANY, True), buttons.TOO_MANY)) - self.assertNotIn('✅ T6', [t for row in rows for t, _ in row]) + self.assertNotIn('✅ T6', [t for row in self.labels(rows) for t in row]) kind, lines, rows = self.tap('✅ T5') # unticking makes room self.assertNotIn(buttons.TOO_MANY, lines) self.assertEqual(lines[-1], 'Ticked: T0, T1, T2, T3, T4') - def test_compare_turns_its_message_into_the_chart_of_the_ticked(self): - """Compare with the period chosen edits the picker into the chart, with the chart's own period buttons.""" + def test_compare_keeps_its_buttons_under_the_chart(self): + """Compare edits the picker into the chart of the ticked, at the period chosen; the picker stays, and a period + button redoes the comparison.""" self.command('.compare 1y') - self.assertIn('● 1Y', [t for row in self.sent_rows() for t, _ in row]) + self.assertIn('● 1Y', [t for row in self.labels() for t in row]) self.tap('Tickers ▶') self.tap('T1') - self.assertNotIn('Compare', [t for row in self.edits[-1][2] for t, _ in row]) # one is too few + self.assertNotIn('Compare', [t for row in self.labels() for t in row]) # one is too few self.tap('T2') kind, lines, rows = self.tap('Compare') - self.assertEqual((kind, rows[0][2]), ('photo', ('● 1Y', 'c|c|y|T1,T2'))) - self.assertEqual(self.toasts[-1], ('Loading…', False)) + self.assertEqual((kind, self.toasts[-1]), ('photo', ('Loading…', False))) + self.assertIn('✅ T2', [t for row in self.labels(rows) for t in row]) + kind, lines, rows = self.tap('◀ Back') + kind, lines, rows = self.tap('7D') + self.assertEqual((kind, self.labels(rows)[0]), ('photo', ['● 7D', '1M', '1Y', 'Max'])) def test_buttons_of_another_list_start_again(self): """A button drawn for other tickers than there are now starts the picker again, saying so.""" @@ -1012,7 +1037,7 @@ def test_without_holdings_the_watchlist_is_offered(self): self.patch(watchlist, 'tickers', lambda chat: ['MSFT', 'AAPL']) self.command('.history') self.tap('Tickers ▶') - self.assertEqual([t for row in self.edits[-1][2] for t, _ in row], ['AAPL', 'MSFT', '◀ Back']) + self.assertEqual([t for row in self.labels(self.edits[-1][2]) for t in row], ['AAPL', 'MSFT', '◀ Back']) self.patch(watchlist, 'tickers', lambda chat: []) self.assertEqual(self.command('.compare'), [('text', ['No tickers to pick from: .watchlist add SYMBOL adds one'])]) @@ -1023,4 +1048,86 @@ def test_with_the_graph_off_the_picker_is_text_and_edits_as_text(self): self.command('.pie') self.assertEqual(self.sent[-1][0], 'text') kind, lines, rows = self.tap('Rob') - self.assertEqual((kind, lines[0], rows), ('text', 'Rob by industry', [])) + self.assertEqual((kind, lines[0], self.labels(rows)), ('text', 'Rob by industry', [['● Rob']])) + + def many(self, count=70): + """Held tickers S00, S01..., enough to need pages.""" + self.patch(sharesight, 'tickers', lambda: [f'S{i:02d}' for i in range(count)]) + + def test_tickers_are_paged_32_to_a_page(self): + """Thirty-two tickers to a page, in rows of four, with < and > where there is another page: the first page has + only >, a middle one both, the last only <; Back stays; ticks last across pages.""" + self.many() + self.command('.compare') + kind, lines, rows = self.tap('Tickers ▶') + self.assertEqual((len(rows) - 1, [t for row in self.labels(rows)[:-1] for t in row][-1], self.labels(rows)[-1], + lines[0]), (8, 'S31', ['◀ Back', '>'], 'Compare: pick up to 6 tickers (page 1 of 3)')) + kind, lines, rows = self.tap('S00') + kind, lines, rows = self.tap('>') + self.assertEqual((self.labels(rows)[0], self.labels(rows)[-1]), (['S32', 'S33', 'S34', 'S35'], + ['◀ Back', '<', '>'])) + kind, lines, rows = self.tap('S40') + kind, lines, rows = self.tap('>') + self.assertEqual((self.labels(rows)[-1], [t for row in self.labels(rows)[:-1] for t in row][-1]), + (['◀ Back', '<', 'Compare'], 'S69')) + self.assertEqual(lines[-1], 'Ticked: S00, S40') + kind, lines, rows = self.tap('<') + kind, lines, rows = self.tap('<') + self.assertIn('✅ S00', [t for row in self.labels(rows) for t in row]) + + def test_a_seventh_tick_is_refused_on_a_later_page(self): + """The limit of six counts every page's ticks, and the refusal stays on the page.""" + self.many() + self.command('.compare') + self.tap('Tickers ▶') + for ticker in ('S00', 'S01', 'S02', 'S03', 'S04'): + self.tap(ticker) + self.tap('>') + self.tap('S33') + kind, lines, rows = self.tap('S34') + self.assertEqual((self.toasts[-1], lines[0], self.labels(rows)[0][:2]), ((buttons.TOO_MANY, True), + buttons.TOO_MANY, ['S32', '✅ S33'])) + + def test_an_item_on_a_later_page_is_the_one_picked(self): + """A ticker's button on page two picks that ticker, and the answer's screen stays on its page.""" + self.many() + self.command('.history') + self.tap('Tickers ▶') + self.tap('>') + kind, lines, rows = self.tap('S40') + self.assertEqual(kind, 'photo') + self.assertIn('● S40', [t for row in self.labels(rows) for t in row]) + self.assertEqual(self.labels(rows)[-1], ['◀ Back', '<', '>']) + + def test_portfolios_page_too(self): + """Thirty-two portfolios to a page, in rows of four, then Group by, then the page buttons.""" + names = [f'P{i:02d}' for i in range(40)] + self.patch(sharesight, '_portfolios', lambda seconds=None: [{'name': n, 'id': i} for i, n in enumerate(names)]) + self.command('.treemap') + rows = self.labels() + self.assertEqual((len(rows), rows[7], rows[8], rows[9]), (10, ['P28', 'P29', 'P30', 'P31'], + ['Group by: Industry'], ['>'])) + kind, lines, rows = self.tap('>') + self.assertEqual((self.labels(rows)[0], self.labels(rows)[-1]), (['P32', 'P33', 'P34', 'P35'], ['<'])) + + def test_discord_pages_hold_five_rows(self): + """Discord's five rows of five buttons hold 12 portfolios with Group by and the page buttons, or 20 tickers + with Back, the page buttons and Compare.""" + for cmd, items in (('t', 12), ('p', 12), ('c', 20), ('y', 20)): + with self.subTest(cmd=cmd): + names = [f'N{i:02d}' for i in range(60)] + rows = pick.screen(DISCORD, cmd, names, '0', '0', 0, 0b11 if cmd == 'c' else 0, + 'm' if cmd in 'yc' else 'p', 1)[1] + self.assertLessEqual(len(rows), 5) + self.assertTrue(all(len(row) <= 5 for row in rows)) + counted = sum(button.label.endswith(tuple('0123456789')) for row in rows for button in row) + self.assertEqual(counted, items) + + def test_pick_data_carries_the_page_and_what_is_shown(self): + """A button's data says its page and what the message answers, and stays within a button's 64 bytes with + 100 items.""" + press = buttons.PickPress('c', 's', 'x', 0x3e, (1 << 100) - 1, 'm2', 'a1b2') + button = buttons.pick_button('x', press) + self.assertEqual((buttons.parse(button.data), len(button.data) <= 64), (press, True)) + for data in ('k|c|0|x|0|0|z1|a1b2', 'k|c|0|x|0|0|p|a1b2', 'k|c|0|x|0|0|r-1|a1b2'): + self.assertIsNone(buttons.parse(data)) diff --git a/tests/test_sending.py b/tests/test_sending.py index 7f93e4f2..e9c462d7 100644 --- a/tests/test_sending.py +++ b/tests/test_sending.py @@ -606,6 +606,13 @@ def test_a_long_report_takes_several_sections(self): self.assertTrue(all(len(block['text']['text']) <= 3000 for block in blocks[:3])) self.assertEqual([block['type'] for block in slack_common.chart_blocks('', [], 'F1')], ['image']) + def test_a_chart_messages_buttons_keep_their_rows(self): + """The buttons of a chart message are a block a row, as a picker lays them out: never one row of them all, past + the 25 elements a block takes.""" + rows = [[buttons.Button(f'B{i}{j}', f'k{i}{j}') for j in range(4)] for i in range(8)] + blocks = slack_common.chart_blocks('x', rows, 'F1') + self.assertEqual([len(block.get('elements', [])) for block in blocks], [0, 0] + [4] * 8) + def test_a_chart_message_that_times_out_is_not_posted_again(self): """A chart with its buttons whose chat.postMessage times out may have arrived: no plain copy follows.""" record = net.post diff --git a/tests/test_util.py b/tests/test_util.py index 8c6a06a4..5ef26b25 100644 --- a/tests/test_util.py +++ b/tests/test_util.py @@ -165,16 +165,17 @@ def test_button_data_reads_back(self): def test_a_pickers_data_reads_back_in_few_bytes(self): """A picker's button says what it asks for, with up to 32 ticked tickers in 64 bytes; data of no picker, with an act or a command it has not, is nothing, and only a refused tick is an alert.""" - press = buttons.PickPress('c', 's', 'x', 2 ** 32 - 1, 'r31', 'a1b2') + press = buttons.PickPress('c', 's', 'x', 1, 2 ** 32 - 1, 'r31', 'a1b2') button = buttons.pick_button('x' * 99, press) - self.assertEqual((buttons.parse(button.data), len(button.label), len(button.data)), (press, 40, 25)) - for data in ('k|z|0|x|0|p|a1b2', 'k|c|0|x|0|q|a1b2', 'k|c|0|x|zz|p|a1b2', 'k|c|0|q|0|p|a1b2', 'k|c|0|x|0|p', - 'k|c|0|x|0|p|a-b2'): + self.assertEqual((buttons.parse(button.data), len(button.label), len(button.data)), (press, 40, 27)) + for data in ('k|z|0|x|0|0|p0|a1b2', 'k|c|0|x|0|0|q0|a1b2', 'k|c|0|x|0|zz|p0|a1b2', 'k|c|0|q|0|0|p0|a1b2', + 'k|c|0|x|0|0|p0', 'k|c|0|x|0|0|p0|a-b2', 'k|c|0|x|0|0|p|a1b2'): with self.subTest(data=data): self.assertIsNone(buttons.parse(data)) - self.assertEqual([buttons.toast(f'k|c|0|x|3|{act}|a1b2') for act in ('p', 'm', 'r2', 'g', 'f')], - ['Updating…', 'Updating…', 'Loading…', 'Loading…', buttons.TOO_MANY]) - self.assertEqual([buttons.alert(f'k|c|0|x|3|{act}|a1b2') for act in ('m', 'f')], [False, True]) + acts = ('p0', 'm1', 'r2', 's2', 'g0', 'h0', 'f0') + self.assertEqual([buttons.toast(f'k|c|0|x|0|3|{act}|a1b2') for act in acts], + ['Updating…', 'Updating…', 'Loading…', 'Loading…', 'Loading…', 'Loading…', buttons.TOO_MANY]) + self.assertEqual([buttons.alert(f'k|c|0|x|0|3|{act}|a1b2') for act in ('m0', 'f0')], [False, True]) def test_a_press_is_told_what_is_coming_from_its_data_alone(self): """The toast a press gets as it arrives: the period a period button loads, 'Updating…' for the .jobs menu's, From 2db69fb41104ec1cc54dcfc3018677876a920c3e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 15:01:29 +0000 Subject: [PATCH 31/41] .jobs menu: Remove is Delete, and a top-level button shows only where it has a job to act on Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- doc/commands.md | 2 +- doc/scheduling.md | 8 +++--- lib/reports/about.py | 45 +++++++++++++++++++--------------- lib/worker.py | 18 +++++++------- tests/test_discord.py | 2 +- tests/test_scheduler.py | 54 ++++++++++++++++++++++++++--------------- 6 files changed, 74 insertions(+), 55 deletions(-) diff --git a/doc/commands.md b/doc/commands.md index 6dea27c6..e118d2c8 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -41,7 +41,7 @@ Where: /history symbol /holdings portfolio /jobs [upcoming|system] -/jobs enable|disable|remove|run [number] +/jobs enable|disable|delete|run [number] /jobs add report schedule /jobs add message schedule "text" /marketcap [symbol|top|bottom] diff --git a/doc/scheduling.md b/doc/scheduling.md index 36d4e2e7..fb36763e 100644 --- a/doc/scheduling.md +++ b/doc/scheduling.md @@ -20,10 +20,10 @@ never run, its name and its schedule in words. Under the list, in italics, is ho /jobs add REPORT SCHEDULE /jobs add message SCHEDULE "TEXT" ``` -Under it are six buttons, two to a row. Each screen is drawn in place of the last, with **Cancel** back to the list: +Under it are up to six buttons, two to a row, each there only where it has something to act on (Add always; Delete and Run now with any job, Enable with a job that is off, Disable and Upcoming... with one that is on). Each screen is drawn in place of the last, with **Cancel** back to the list: * **Add**: what the job runs, how often, then the rest down to the minute ([below](#add-step-by-step)). -* **Remove**: a button per job, the default ones too. **Remove all** asks first ("Remove all 37 jobs here?"), then - removes every job. A default job that is removed comes back when the same job is added again. +* **Delete**: a button per job, the default ones too. **Delete all** asks first ("Delete all 37 jobs here?"), then + deletes every job. A default job that is deleted comes back when the same job is added again. * **Enable**: a button per job that is off; **Enable all** switches every job on. * **Disable**: a button per job that is on; **Disable all** switches every job off. * **Run now**: a button per job; a press runs it now, as its schedule would, and says how it went. @@ -65,7 +65,7 @@ Or type: /jobs add message every 31 mar at 8:30am "🥳 Happy EOFY" /jobs run 2 runs job 2 now; /jobs run alone: the list with a button per job /jobs enable 2 switches job 2 on; /jobs disable 2 off; /jobs toggle 2 the other way -/jobs remove 36 removes job 36 (or /jobs del 36) +/jobs delete 36 deletes job 36 (or /jobs remove 36, /jobs del 36) ``` ## A job in English diff --git a/lib/reports/about.py b/lib/reports/about.py index b6e96c43..cf9f0772 100644 --- a/lib/reports/about.py +++ b/lib/reports/about.py @@ -1,5 +1,5 @@ """What finbot can do: the .help text, and the .jobs menu: a chat's jobs a page at a time, with buttons that enable, -disable, remove or run one; the steps of Add; the upcoming runs; the system jobs; and the reports a job can run.""" +disable, delete or run one; the steps of Add; the upcoming runs; the system jobs; and the reports a job can run.""" import calendar import datetime import re @@ -13,13 +13,13 @@ from lib import when HEADINGS = {'top': 'Jobs here', 'on': 'Tap a job to enable it', 'off': 'Tap a job to disable it', - 'del': 'Tap a job to remove it', 'run': 'Tap a job to run it now'} + 'del': 'Tap a job to delete it', 'run': 'Tap a job to run it now'} NOTHING = {'on': 'Every job is on', 'off': 'Every job is off'} # what Enable and Disable say with nothing to switch -EVERY = {'on': 'Enable all', 'off': 'Disable all', 'del': 'Remove all'} # each list screen's button for every job -TYPED = ['.jobs enable|disable|remove|run NUMBER: switch one on or off, remove one, run one now', +EVERY = {'on': 'Enable all', 'off': 'Disable all', 'del': 'Delete all'} # each list screen's button for every job +TYPED = ['.jobs enable|disable|delete|run NUMBER: switch one on or off, delete one, run one now', '.jobs add REPORT SCHEDULE: add one (.jobs add lists the reports)', '.jobs upcoming: the next runs', '.jobs system: the system jobs'] # what a chat without buttons types instead -TYPED_ACTIONS = {'on': 'enable', 'off': 'disable', 'del': 'remove', 'run': 'run'} +TYPED_ACTIONS = {'on': 'enable', 'off': 'disable', 'del': 'delete', 'run': 'run'} ADD_TYPED = ['/jobs add REPORT SCHEDULE', '/jobs add message SCHEDULE "TEXT"'] # how Add is typed: under the list NOT_RUN = '* not run yet' # the key to a list's mark PAGE_ROOM = 500 # characters of a screen besides its jobs' lines: its headings, a note, the key to * @@ -56,11 +56,12 @@ def jobs_screen(service, chat, screen='top', page=0, note=None, tappable=True): """A screen of the .jobs menu that lists a chat's jobs (by its key), numbered, each ✅ (on) or ❌ (off), * where it has never run, its name and its schedule in words, by the timezone preference's clock (lib/when.py), then the key - to the marks shown. top has what can be done (Add, Remove, Enable, Disable, Run now, Upcoming...) and how Add is - typed, in italics; on, off, del and run a button for each job the screen acts on (acts), which switches it on or - off, removes it or runs it now, and Enable all, Disable all or Remove all. A page at a time, as many jobs as a - message of the network takes (◀ and More ▶ for the others), after a note of what the last press did. A one-off - gone by is deleted first. Returns (lines, buttons); where the chat takes no buttons (tappable false), the whole + to the marks shown. top has what can be done (Add, and where there is a job to act on, Delete, Enable, Disable, Run + now, Upcoming...) and how Add is typed, in italics; on, off, del and run a button for each job the screen acts on + (acts), which switches it on or off, deletes it or runs it now, and Enable all, Disable all or Delete all. A page at + a time, as many jobs as a message of the network takes (◀ and More ▶ for the others), after a note of what the last + press did. A one-off gone by is deleted first. Returns (lines, buttons); where the chat takes no buttons (tappable + false), the whole list and what to type instead.""" scheduler = jobs.current if scheduler is None: @@ -107,13 +108,17 @@ def paged(entries, service): return parts def screen_buttons(screen, page, pages, shown, crc, own): - """A list screen's buttons: top's, what can be done, two to a row; on, off, del and run's, one for each job shown - that the screen acts on (acts), ' ', JOB_ROW to a row, then Enable all, Disable all or Remove all + """A list screen's buttons: top's, what can be done (Add; Delete, Run now with any job, Enable with one off, + Disable with one on, Upcoming... with one on), two to a row; on, off, del and run's, one for each job shown + that the screen acts on (acts), ' ', JOB_ROW to a row, then Enable all, Disable all or Delete all where it acts on any of the chat's jobs (own), and Cancel, back to the list; ◀ and More ▶ to the other pages.""" if screen == 'top': - rows = [[buttons.menu_button(label, kind, args, crc) for label, kind, args in pair] for pair in ( - (('Add', 'add', []), ('Remove', 'del', [0])), (('Enable', 'on', [0]), ('Disable', 'off', [0])), - (('Run now', 'run', [0]), ('Upcoming...', 'next', [])))] + possible = [('Add', 'add', [])] + [(label, kind, args) for label, kind, args, there in ( + ('Delete', 'del', [0], bool(own)), ('Enable', 'on', [0], any(acts('on', job) for job in own)), + ('Disable', 'off', [0], any(acts('off', job) for job in own)), ('Run now', 'run', [0], bool(own)), + ('Upcoming...', 'next', [], any(job.on for job in own))) if there] + pairs = [possible[i:i + 2] for i in range(0, len(possible), 2)] + rows = [[buttons.menu_button(label, kind, args, crc) for label, kind, args in pair] for pair in pairs] nav = [] else: tapped = [buttons.menu_button(f'{number} {job.name}', screen, [page, number], crc) for number, job, _ in shown @@ -135,13 +140,13 @@ def job_entry(service, scheduler, number, job, running=(), local=None, now=None) schedule = job.schedule.describe(local or formats.zone().key, now or scheduler.clock()) return f"{number}. {mark} {service.escape(job.name)} · {schedule}" + (' (running)' if job.key in running else '') -def remove_all_screen(service, chat, page=0): - """Remove all's question, for a chat (by its key): whether to remove every one of its jobs, the default ones too, - with Yes, remove all and Cancel.""" +def delete_all_screen(service, chat, page=0): + """Delete all's question, for a chat (by its key): whether to delete every one of its jobs, the default ones too, + with Yes, delete all and Cancel.""" own = jobs.current.chat_jobs(chat) crc = jobs.list_checksum(own) - return [service.bold(f"Remove all {len(own)} job{'' if len(own) == 1 else 's'} here?"), 'A default job removed' - ' comes back only when the same job is added again'], [[buttons.menu_button('Yes, remove all', 'del', [page, + return [service.bold(f"Delete all {len(own)} job{'' if len(own) == 1 else 's'} here?"), 'A default job deleted' + ' comes back only when the same job is added again'], [[buttons.menu_button('Yes, delete all', 'del', [page, 'allyes'], crc), buttons.menu_button('Cancel', 'top', [0], crc)]] def job_line(number, job, outcome): diff --git a/lib/worker.py b/lib/worker.py index ff2dcfa6..73a534d6 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -103,11 +103,11 @@ def menu_screen(service, chat, pressed): if not (hmac.compare_digest(jobs.list_checksum(own), pressed.checksum) and (every or number.isdigit() and 1 <= int(number) <= len(own))): note = 'The jobs have changed: here they are now' - elif number == 'all' and screen == 'del': # Remove all asks first - return about.remove_all_screen(service, key, page) + elif number == 'all' and screen == 'del': # Delete all asks first + return about.delete_all_screen(service, key, page) elif every and screen == 'del': scheduler.delete_all(key) - return about.jobs_screen(service, key, 'top', 0, 'Every job here is removed') + return about.jobs_screen(service, key, 'top', 0, 'Every job here is deleted') elif every: scheduler.switch_all(key, screen == 'on') note = f'Every job is {screen} now' @@ -119,7 +119,7 @@ def menu_screen(service, chat, pressed): note = f"{'Enabled' if screen == 'on' else 'Disabled'} {number}. {service.escape(job.name)}" elif screen == 'del': scheduler.delete(job) - note = f'Removed {number}. {service.escape(job.name)}' + note = f'Deleted {number}. {service.escape(job.name)}' else: delivery.deliver(chat, delivery.Report([about.job_line(number, job, scheduler.run(job))])) except jobs.ScheduleError as e: @@ -542,7 +542,7 @@ def on_who(request, args): def on_jobs(request, args): """.jobs: this chat's jobs, with the menu's buttons; .jobs upcoming: the next runs; .jobs system: the system's; - .jobs enable, disable, remove, run or toggle NUMBER: one switched on or off, removed, run now, or switched the + .jobs enable, disable, delete, run or toggle NUMBER: one switched on or off, deleted, run now, or switched the other way, and without a number the list with a button per job that does it; .jobs add REPORT SCHEDULE, or .jobs add message SCHEDULE "TEXT": one more (alone, the reports a job can run).""" scheduler, chat, service = jobs.current, request.dest.key, request.service @@ -577,7 +577,7 @@ def on_jobs(request, args): request.reply([about.job_line(number, job, scheduler.run(job))]) elif args.keyword == 'del': scheduler.delete(job) - request.reply([f"Removed {number}. {job.name} · {job.schedule.describe(formats.zone().key)}"]) + request.reply([f"Deleted {number}. {job.name} · {job.schedule.describe(formats.zone().key)}"]) else: if args.keyword == 'toggle': scheduler.toggle(job) @@ -664,7 +664,7 @@ class Command: Command(('trades', 'trade'), on_trades, ('.trades [period] [portfolio]',), TRADES, ('.trades',), period=True), Command(('plan',), on_plan, ('.plan [your plan]',), TRADES, job=False), Command(('jobs', 'job', 'cron', 'crons', 'schedule'), on_jobs, ('.jobs [upcoming|system]', - '.jobs enable|disable|remove|run [NUMBER]', '.jobs add REPORT SCHEDULE', '.jobs add message SCHEDULE "TEXT"'), + '.jobs enable|disable|delete|run [NUMBER]', '.jobs add REPORT SCHEDULE', '.jobs add message SCHEDULE "TEXT"'), JOBS, ('.schedule',), keywords=JOB_CHANGES, job=False), PROFILE, ] @@ -791,7 +791,7 @@ def job_commands(): return found PERIOD_WORDS = ('1d', '7d', '1m', '3m', '6m', 'ytd', '1y', '5y', '10y') # suggested where a command takes a period -JOB_WORDS = ('add', 'remove', 'enable', 'disable', 'run', 'upcoming', 'system') # what /jobs suggests first +JOB_WORDS = ('add', 'delete', 'enable', 'disable', 'run', 'upcoming', 'system') # what /jobs suggests first SCHEDULE_WORDS = ('daily', 'weekdays', 'weekly', 'monthly', 'quarterly', 'bi-annual', 'annual', 'hourly', 'every', 'at', 'from', 'between', 'on') # what a schedule in English is made of, suggested after a report @@ -846,7 +846,7 @@ def argument_words(command, chat, given): return [(word, word) for word in dict.fromkeys(words)] def job_words(chat, words): - """(word, shown) pairs for /jobs, for suggest, after the words given: first its subcommands; after remove, run or + """(word, shown) pairs for /jobs, for suggest, after the words given: first its subcommands; after delete, run or toggle, this chat's job numbers, each shown with its name (after enable, those of the jobs off; after disable, those on); after add, the reports, each shown with what it takes; after a report, the words it takes, then those of a schedule.""" diff --git a/tests/test_discord.py b/tests/test_discord.py index ef623901..b208a13d 100644 --- a/tests/test_discord.py +++ b/tests/test_discord.py @@ -401,7 +401,7 @@ def test_the_jobs_subcommands_numbers_and_reports(self): with contextlib.redirect_stderr(io.StringIO()): scheduler.load() self.patch(jobs, 'current', scheduler) - self.assertEqual(self.values('/jobs '), ['add', 'remove', 'enable', 'disable', 'run', 'upcoming', 'system']) + self.assertEqual(self.values('/jobs '), ['add', 'delete', 'enable', 'disable', 'run', 'upcoming', 'system']) self.assertEqual(self.values('/job E'), ['enable']) self.assertEqual(worker.suggest('discord:77', '/jobs run '), [('run 1: rating', 'run 1'), ('run 2: trades', 'run 2'), ('run 3: shorts', 'run 3')]) diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index 4fea431f..579f4b42 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -517,14 +517,14 @@ def test_add_list_and_remove(self): '3. ✅* trades · every 20 mins, weekdays UTC', '4. ✅* shorts · daily at 10:00pm', '', '* not run yet', '', '/jobs add REPORT SCHEDULE', '/jobs add message SCHEDULE "TEXT"']) self.assertEqual([[button.label for button in row] for row in self.replies[-1][2]], - [['Add', 'Remove'], ['Enable', 'Disable'], ['Run now', 'Upcoming...']]) - self.assertEqual(self.say('.jobs del 3'), ['Removed 3. trades · every 20 mins, weekdays UTC']) + [['Add', 'Delete'], ['Disable', 'Run now'], ['Upcoming...']]) + self.assertEqual(self.say('.jobs del 3'), ['Deleted 3. trades · every 20 mins, weekdays UTC']) shorts = entry('shorts daily at 10pm', zone='Australia/Melbourne') self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [shorts]}) - self.assertEqual(self.say('.jobs remove 1'), ['Removed 1. rating · Tue–Sat at 8:30am Sydney']) + self.assertEqual(self.say('.jobs delete 1'), ['Deleted 1. rating · Tue–Sat at 8:30am Sydney']) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [shorts], 'deleted': ['rating']}) self.assertEqual([job.name for job in self.s.chat_jobs(CHAT)], ['price close US', 'shorts']) - self.assertEqual(self.say('.job del 2'), ['Removed 2. shorts · daily at 10:00pm']) + self.assertEqual(self.say('.job remove 2'), ['Deleted 2. shorts · daily at 10:00pm']) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'jobs': [], 'deleted': ['rating']}) def test_a_command_of_help_as_a_job(self): @@ -755,7 +755,21 @@ def test_enable_and_disable_switch_one_way(self): self.assertEqual((lines[0], rows[0]), ('Enabled 1. rating', ['2 price close US'])) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT]['off'], ['price-close-us']) lines, rows, _ = self.press(data['Cancel']) - self.assertEqual((lines[0], rows[0]), ('Jobs here', ['Add', 'Remove'])) + self.assertEqual((lines[0], rows[0]), ('Jobs here', ['Add', 'Delete'])) + + def test_the_top_level_offers_only_what_can_be_done(self): + """Delete, Run now, Enable, Disable and Upcoming... show only where a job is there to act on: with every job on + there is no Enable; with every job off no Disable and no Upcoming...; with none, only Add.""" + def top(): + """The top screen's button labels, by row.""" + return [[button.label for button in row] for row in about.jobs_screen(TELEGRAM, CHAT)[1]] + self.assertEqual(top(), [['Add', 'Delete'], ['Disable', 'Run now'], ['Upcoming...']]) + self.s.switch(self.s.chat_jobs(CHAT)[0], False) + self.assertEqual(top(), [['Add', 'Delete'], ['Enable', 'Disable'], ['Run now', 'Upcoming...']]) + self.s.switch_all(CHAT, False) + self.assertEqual(top(), [['Add', 'Delete'], ['Enable', 'Run now']]) + self.s.delete_all(CHAT) + self.assertEqual(top(), [['Add']]) def test_enable_all_and_disable_all(self): """Disable all switches every job of the chat off and Enable all every one on, each saying so, and each shown @@ -776,25 +790,25 @@ def test_enable_all_and_disable_all(self): self.assertEqual(self.press('m|off|0|all|0000')[0][0], 'The jobs have changed: here they are now') self.assertTrue(all(job.on for job in self.s.chat_jobs(CHAT))) - def test_remove_offers_every_job_and_remove_all_asks_first(self): - """Remove lists every job with a button each, the default ones too: a press removes it and says so, a default - one staying out of the list (stored as removed). Remove all asks first; Yes empties the list, and a stale - press removes nothing.""" + def test_delete_offers_every_job_and_delete_all_asks_first(self): + """Delete lists every job with a button each, the default ones too: a press deletes it and says so, a default + one staying out of the list (stored as deleted). Delete all asks first; Yes empties the list, and a stale + press deletes nothing.""" lines, rows, data = self.press(f'm|del|0|{self.crc()}') - self.assertEqual((lines[0], lines[-1]), ('Tap a job to remove it', '* not run yet')) - self.assertEqual(rows, [['1 rating', '2 price close US', '3 trades'], ['Remove all', 'Cancel']]) + self.assertEqual((lines[0], lines[-1]), ('Tap a job to delete it', '* not run yet')) + self.assertEqual(rows, [['1 rating', '2 price close US', '3 trades'], ['Delete all', 'Cancel']]) lines, rows, data = self.press(data['1 rating']) - self.assertEqual((lines[:2], rows), (['Removed 1. rating', ''], [['1 price close US', '2 trades'], - ['Remove all', 'Cancel']])) + self.assertEqual((lines[:2], rows), (['Deleted 1. rating', ''], [['1 price close US', '2 trades'], + ['Delete all', 'Cancel']])) self.assertEqual(state.load(jobs.JOBS_FILE)[CHAT], {'deleted': ['rating'], 'jobs': [entry('trades every 20 mins weekdays')]}) - lines, rows, data = self.press(data['Remove all']) - self.assertEqual((lines, rows), (['Remove all 2 jobs here?', 'A default job removed comes back only when' - ' the same job is added again'], [['Yes, remove all', 'Cancel']])) + lines, rows, data = self.press(data['Delete all']) + self.assertEqual((lines, rows), (['Delete all 2 jobs here?', 'A default job deleted comes back only when' + ' the same job is added again'], [['Yes, delete all', 'Cancel']])) self.assertEqual(self.press('m|del|0|allyes|0000')[0][0], 'The jobs have changed: here they are now') self.assertEqual(len(self.s.chat_jobs(CHAT)), 2) - lines, rows, data = self.press(data['Yes, remove all']) - self.assertEqual(lines[:4], ['Every job here is removed', '', 'Jobs here', + lines, rows, data = self.press(data['Yes, delete all']) + self.assertEqual(lines[:4], ['Every job here is deleted', '', 'Jobs here', 'No jobs here: Add makes one']) self.assertEqual((self.s.chat_jobs(CHAT), state.load(jobs.JOBS_FILE)[CHAT]), ([], {'deleted': ['rating', 'price-close-us'], 'jobs': []})) @@ -838,7 +852,7 @@ def test_the_older_job_buttons_still_work(self): self.assertEqual((self.toasts, self.ran), (['Running…'], [('trades', {})])) lines, rows, _ = self.press(f'e|3|{jobs.checksum(job.key)}') self.assertFalse(self.s.chat_jobs(CHAT)[2].on) - self.assertEqual((lines[0], rows[0]), ('Jobs here', ['Add', 'Remove'])) + self.assertEqual((lines[0], rows[0]), ('Jobs here', ['Add', 'Delete'])) self.assertIn('3. ❌ trades · every 20 mins, weekdays UTC', lines) lines, _, _ = self.press('e|3|0000') # stale: the screen as the list is now, nothing switched self.assertFalse(self.s.chat_jobs(CHAT)[2].on) @@ -1029,7 +1043,7 @@ def fits(service, lines, markup): with self.subTest(service=service, steps=steps): fits(service, *about.add_screen(service, CHAT, steps)) fits(service, *about.next_screen(service, CHAT)) - fits(service, *about.remove_all_screen(service, CHAT)) + fits(service, *about.delete_all_screen(service, CHAT)) fits(service, *about.added_screen(service, CHAT, 1)) fits(service, *about.add_failed(service, CHAT, ['5', 'd', '9'], 'that job is here already')) pages = len({about.jobs_screen(DISCORD, CHAT, 'on', page)[0][0] for page in range(6)}) From 796ee5bcbafbcad84ddbe8f6fde84e0ee32fbc47 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 15:08:16 +0000 Subject: [PATCH 32/41] Bare .pie .treemap .history .compare answer at once with a default and keep the picker under it The first portfolio (pie, treemap), the first ticker (history) or the first two (compare), alphabetically, with the picker's buttons under the chart; a press redoes the answer in place from what is shown; the page buttons are only under the tickers menu; a pie or treemap draws its other charts in a background thread to fill the cache. The 1x1 placeholder image is gone. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 10 +- doc/commands.md | 2 +- lib/buttons.py | 2 +- lib/reports/pick.py | 68 ++++++++++--- lib/worker.py | 74 ++++++++------ tests/common.py | 1 + tests/test_edges.py | 2 +- tests/test_reports.py | 217 +++++++++++++++++++++++++----------------- tests/test_util.py | 4 +- 9 files changed, 240 insertions(+), 140 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6ef4bde8..63143885 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -85,10 +85,12 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm its "thinking…" through its `interaction` (application id/token), and a pressed message is edited through the press's. - `lib/reports/pick.py` and `PickPress` (`lib/buttons.py`, `pick_button`): the buttons of `.pie .treemap .holdings - .history .compare` typed bare (`worker.picker`, `process_pick_callback`): one message edited as it is pressed (a - 1x1 image, `render.blank()`, where its answer is a chart) that keeps its buttons under the answer (`shown`), 32 items - a page (`pick.layout`; fewer on Discord) turned by `<` `>`, its list of items checked by checksum, `.compare`'s - ticks a bit each in the data's mask, a seventh refused by an alert toast (`buttons.alert`). + .history .compare` typed bare (`worker.picker`, `process_pick_callback`): the first answer at once (the first + portfolio or ticker, the first two to compare; `.holdings` only its buttons), the buttons kept under it and every + press re-editing the answer from `shown` (an item's number plus one; compare's tickers, `pick.pack`), 32 items a page + (`pick.layout`; fewer on Discord) turned by `<` `>` under the tickers menu, its list of items checked by checksum, + `.compare`'s ticks a bit each in the data's mask, a seventh refused by an alert toast (`buttons.alert`); a pie or + treemap primes its other charts in a thread (`pick.prime`, started by `worker.prime_in_background`). - `lib/buttons.py`: `Button`, a chart's period buttons (`keyboard`), the `.jobs` menu's (`menu_button`, a `MenuPress`; older lists' `j|`/`e|` buttons still parse), what a pressed button's data asks for (`parse`), and the toast it gets at once (`toast`); each transport draws them, and Slack and Discord take a button's data as its id, so diff --git a/doc/commands.md b/doc/commands.md index e118d2c8..61d63c97 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -73,7 +73,7 @@ The bot answers commands in every chat it is in. By default, it fetches its upda * A watch list per chat, which anyone in it can change, picked up by the reports there * Listing of portfolios and portfolio performance * A pie or a treemap of a portfolio by industry or sector (`/pie portfolio [sector]`, `/treemap portfolio [sector]`): the treemap tiles each holding by value, grouped and coloured by industry (the default) or sector -* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap`, `/holdings` (portfolios; `/treemap` also toggles Sector/Industry), `/history` (a ticker) and `/compare` (up to 6 ticked tickers and a period), the tickers those held (else the chat's watchlist), behind a `Tickers ▶` button. Portfolios come four to a row, `/treemap`'s Sector/Industry toggle after them; thirty-two items make a page, with `<` and `>` to turn it (fewer on Discord, which holds five rows of five buttons). One message is edited as buttons are pressed and becomes the answer, the buttons staying under it with the item shown marked: another item, the toggle or a period redoes the answer in place. A job, or a chat with no buttons, gets the usage instead +* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap` (the first portfolio alphabetically, by industry or by `sector`), `/history` (the first ticker) and `/compare` (the first two tickers, at the period typed) answer at once, with the buttons under the chart: portfolios four to a row and `/treemap`'s Sector/Industry toggle after them; for tickers a period row and a `Tickers ▶` menu (those held, else the chat's watchlist) with `<` `>` pages, up to 6 ticked to compare. `/holdings` shows its portfolio buttons alone. Another portfolio or ticker, the toggle or a period redoes the chart in place, and a `/pie` or `/treemap` draws its other charts in the background so the next press is quick. Thirty-two items make a page (fewer on Discord, which holds five rows of five buttons). A job, or a chat with no buttons, gets the usage instead * Listing of holdings sorted by various attributes * Running the other stock reports on demand diff --git a/lib/buttons.py b/lib/buttons.py index a7823f32..2a327cf1 100644 --- a/lib/buttons.py +++ b/lib/buttons.py @@ -127,7 +127,7 @@ def parse(data): if kind == 'k': parts = rest.split('|') if (len(parts) == 7 and parts[0] in PICKERS and parts[1] in ('0', 'i', 's') and (parts[2] in PERIODS or - parts[2] == '0') and len(parts[3]) <= 3 and len(parts[4]) <= 25 and parts[5][:1] in 'pmfrsgh' and + parts[2] == '0') and len(parts[3]) <= 12 and len(parts[4]) <= 25 and parts[5][:1] in 'pmfrgh' and parts[5][1:].isdigit() and all(part.isascii() and part.isalnum() for part in parts) and len(data.encode()) <= MAX_CALLBACK_BYTES): try: diff --git a/lib/reports/pick.py b/lib/reports/pick.py index 7e97d0be..91638798 100644 --- a/lib/reports/pick.py +++ b/lib/reports/pick.py @@ -6,8 +6,11 @@ from lib import answer from lib import buttons from lib import config +from lib import log +from lib import render from lib import sharesight from lib import watchlist +from lib.reports import holdings MOST_ITEMS = 100 # items offered, paged: what the mask of ticks in a button's data has room for CHARTS = 'ptyc' # the pickers whose answer is a chart @@ -43,41 +46,78 @@ def checksum(found): """A short stand-in for a list of items, which tells any change of it.""" return hashlib.sha1('\n'.join(found).encode()).hexdigest()[:4] -def start(service, cmd, chat, group='0', period='0'): - """A picker's first screen: (lines, rows).""" - return screen(service, cmd, items(service, cmd, chat), group, period, 0, 0, 'p') +def pack(numbers): + """The numbers of the tickers on show (compare's), as one number for a button's data: two hex digits to each, its + number plus one; 0 for none.""" + return int(''.join(f'{number + 1:02x}' for number in numbers), 16) if numbers else 0 + +def unpack(shown): + """The numbers of the tickers that pack made `shown` of.""" + text = format(shown, 'x') + text = text.rjust(len(text) + len(text) % 2, '0') + return [int(text[i:i + 2], 16) - 1 for i in range(0, len(text), 2)] + +def default(cmd, found): + """What a picker answers at first, as (shown, mask): the first portfolio (pie, treemap) or ticker (history), or the + first two tickers (compare), alphabetically; (0, 0) for nothing yet (holdings). Raises answer.Answer where compare + has fewer than two tickers.""" + if cmd == 'c': + if len(found) < 2: + raise answer.Answer("Compare needs two tickers: .watchlist add SYMBOL adds one") + return pack([0, 1]), 0b11 + return (0, 0) if cmd == 'h' else (1, 0) + +def shown_names(cmd, found, shown): + """The items a picker's message shows now (shown, as a PickPress has it): none when it shows nothing yet.""" + if cmd == 'c': + return [found[number] for number in unpack(shown) if 0 <= number < len(found)] if shown else [] + return [found[shown - 1]] if 0 < shown <= len(found) else [] + +def prime(service, cmd, group, found, index): + """Draw, to fill the cache, the charts a button of a pie or treemap picker would ask for: first the other grouping + of the portfolio shown (found[index]), then every other portfolio in both. A failure is logged, never raised.""" + prepare = holdings.prepare_pie if cmd == 'p' else holdings.prepare_treemap + other = 'industry' if group == 's' else 'sector' + wanted = [(found[index], other)] + [(name, by) for name in found if name != found[index] + for by in ('industry', 'sector')] + for name, by in wanted: + try: + _, chart = prepare(name, by, service) + if chart and chart.rows: + render.draw(chart) + except Exception as e: + log.warn("priming", cmd, name, by, "failed:", e) def screen(service, cmd, found, group, period, shown, mask, view, page=0, note=None): """A picker's screen, as (lines, rows): `view` 'p', the first (the portfolios; for tickers the period and the way to the tickers), or 'm', the tickers'; `page` of the items; `shown` what the message answers now (an item's number - plus one, or 1 for the ticked tickers, 0 for nothing yet, which a button then carries to redo with its change); - `mask` has a bit for each ticked ticker.""" + plus one, or compare's tickers, see pack; 0 for nothing yet); `mask` has a bit for each ticked ticker.""" per_page, per_row = layout(service, cmd) pages = -(-len(found) // per_page) page = min(max(page, 0), pages - 1) ticked = [name for i, name in enumerate(found) if mask >> i & 1] + to_compare = cmd == 'c' and len(ticked) >= 2 and pack([i for i in range(len(found)) if mask >> i & 1]) != shown name = buttons.PICKERS[cmd] def button(label, act, active=False, **changes): """A button of this screen, whatever it changes (group, period, shown or mask) in its data.""" state = {'group': group, 'period': period, 'shown': shown, 'mask': mask} | changes return buttons.pick_button(label, buttons.PickPress(cmd, state['group'], state['period'], state['shown'], state['mask'], act, checksum(found)), active) - redo = 'p0' if not shown else 'h0' if cmd == 'c' else f"s{shown - 1}" if cmd == 'y' else f"r{shown - 1}" rows = [] if cmd in 'yc' and view == 'p': # the period, and the way to the tickers - rows.append([button(buttons.PERIODS[key][0], redo, key == period, period=key) for key in buttons.BUTTONS['c']]) + rows.append([button(buttons.PERIODS[key][0], 'p0', key == period, period=key) for key in buttons.BUTTONS['c']]) more = f"Tickers ({len(ticked)}) ▶" if ticked else "Tickers ▶" - rows.append([button(more, 'm0')] + ([button('Compare', 'h0')] if cmd == 'c' and len(ticked) >= 2 else [])) - shown_page = items_on(found, page, per_page) + rows.append([button(more, 'm0')] + ([button('Compare', 'h0')] if to_compare else [])) + on_page = items_on(found, page, per_page) if noun(cmd) == 'portfolio': - rows += chunks([button(name, f"r{i}", shown == i + 1, shown=i + 1) for i, name in shown_page], per_row) + rows += chunks([button(name, f"r{i}", shown == i + 1, shown=i + 1) for i, name in on_page], per_row) if cmd == 't': - rows.append([button(f"Group by: {'Sector' if group == 's' else 'Industry'}", redo, + rows.append([button(f"Group by: {'Sector' if group == 's' else 'Industry'}", f"p{page}", group='i' if group == 's' else 's')]) elif view == 'm': full = len(ticked) >= buttons.MAX_PICKED tickers = [] - for i, ticker in shown_page: + for i, ticker in on_page: if cmd == 'y': tickers.append(button(ticker, f"r{i}", shown == i + 1, shown=i + 1)) elif mask >> i & 1: @@ -86,11 +126,11 @@ def button(label, act, active=False, **changes): tickers.append(button(ticker, f"f{page}") if full else button(ticker, f"m{page}", mask=mask | 1 << i)) rows += chunks(tickers, per_row) nav = [button('◀ Back', 'p0')] if cmd in 'yc' and view == 'm' else [] - if pages > 1: + if pages > 1 and (view == 'm' or noun(cmd) == 'portfolio'): # the page buttons: under the tickers, not above them act = 'm' if view == 'm' else 'p' nav += ([button('<', f"{act}{page - 1}")] if page else []) + ( [button('>', f"{act}{page + 1}")] if page < pages - 1 else []) - if cmd == 'c' and view == 'm' and len(ticked) >= 2: + if view == 'm' and to_compare: nav.append(button('Compare', f"g{page}")) if nav: rows.append(nav) diff --git a/lib/worker.py b/lib/worker.py index 73a534d6..e4c56cec 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -7,6 +7,7 @@ import json import math import re +import threading import typing from lib import log from lib import answer @@ -141,17 +142,14 @@ def add_step(service, chat, steps): return about.added_screen(service, chat.key, number) def show_pick(chat, message, cmd, lines, rows): - """Put a picker's screen (lines and button rows) in place of the pressed message.""" - text = '\n'.join(lines) - if pick.as_photo(chat, cmd): - chat.edit_chart(message, render.blank(), text, rows) - else: - chat.edit_text(message, text, rows) + """Put a picker's screen (lines and button rows) in place of the pressed message: a picker with no answer yet, which + is text.""" + chat.edit_text(message, '\n'.join(lines), rows) def process_pick_callback(press, pressed): - """A press in a picker (an inbound.Press; pressed, the buttons.PickPress its data says): the screen it asks for in - place of the pressed message, or the answer for an item, or Compare, with the picker's buttons left under it (the - item shown marked). Buttons drawn for another list of items start the picker again, saying so.""" + """A press in a picker (an inbound.Press; pressed, the buttons.PickPress its data says): the message shows the + answer for what it shows now (`shown`), or for the item or Compare pressed, with the screen the press asks for + under it. Buttons drawn for another list of items start the picker again at its first answer, saying so.""" service = dialects.of(press.service) chat = delivery.reply_to(service, press.chat_id, interaction=press.interaction, pressed=True) cmd, group, period, shown, mask = pressed.cmd, pressed.group, pressed.period, pressed.shown, pressed.mask @@ -160,19 +158,19 @@ def process_pick_callback(press, pressed): stale = pick.checksum(found) != pressed.checksum kind, number = (pressed.act[0], int(pressed.act[1:])) if not stale else ('p', 0) if stale: - shown, mask = 0, 0 + shown, mask = pick.default(cmd, found) per_page = pick.layout(service, cmd)[0] view = 'm' if kind in 'mfg' or kind == 'r' and noun_is_ticker(cmd) else 'p' page = number if kind in 'pmfg' else number // per_page if kind == 'r' else 0 note = 'The list has changed: here it is now' if stale else buttons.TOO_MANY if kind == 'f' else None - names = None - if kind in 'rs' and number < len(found): - names, shown = [found[number]], number + 1 + if kind == 'r' and number < len(found): + shown = number + 1 elif kind in 'gh' and bin(mask).count('1') >= 2: - names, shown = [name for i, name in enumerate(found) if mask >> i & 1], 1 + shown = pick.pack([i for i in range(len(found)) if mask >> i & 1]) + names = pick.shown_names(cmd, found, shown) lines, rows = pick.screen(service, cmd, found, group, period, shown, mask, view, page, note) if names: - return pick_answer(service, chat, press.message_id, cmd, group, period, names, rows) + return pick_answer(service, chat, press.message_id, cmd, group, period, names, rows, note) show_pick(chat, press.message_id, cmd, lines, rows) except answer.Answer as e: delivery.deliver(chat, delivery.Report(e.lines)) @@ -183,24 +181,27 @@ def noun_is_ticker(cmd): """Whether a picker picks tickers (history, compare), not portfolios.""" return pick.noun(cmd) == 'ticker' -def pick_answer(service, chat, message, cmd, group, period, names, rows): +def pick_content(service, chat, cmd, group, period, names): + """A picker's answer for the portfolio (pie, treemap) or tickers (history, compare) `names`: (caption, chart or + None).""" + if cmd in 'pt': + prepare = holdings.prepare_pie if cmd == 'p' else holdings.prepare_treemap + lines, chart = prepare(names[0], 'sector' if group == 's' else 'industry', service) + return '\n'.join(lines), chart + return charts.build('c' if cmd == 'c' else 'h', names, service, chat.key, period=period if period != '0' else 'x') + +def pick_answer(service, chat, message, cmd, group, period, names, rows, note=None): """The pressed picker message turned into the answer of its command for the item (history, a portfolio's) or items - (compare), the picker's button rows left under it: a chart, or the holdings as text.""" - by = 'sector' if group == 's' else 'industry' + (compare), the picker's button rows left under it, after a note if there is one: a chart, or text (the holdings).""" if cmd == 'h': lines = holdings.prepare_holdings_payload(names[0], service) or [f"{names[0]} holds nothing"] - parts = transports.chunked(lines, chat.dialect.text_limit) + parts = transports.chunked(([note] if note else []) + lines, chat.dialect.text_limit) chat.edit_text(message, '\n'.join(parts[0]), rows if len(parts) == 1 else []) for i, part in enumerate(parts[1:], 1): chat.text(part, rows if i == len(parts) - 1 else None) return - if cmd in 'pt': - prepare = holdings.prepare_pie if cmd == 'p' else holdings.prepare_treemap - lines, chart = prepare(names[0], by, service) - caption = '\n'.join(lines) - else: - days = period if period != '0' else 'x' - caption, chart = charts.build('c' if cmd == 'c' else 'h', names, service, chat.key, period=days) + caption, chart = pick_content(service, chat, cmd, group, period, names) + caption = f"{note}\n{caption}" if note else caption if pick.as_photo(chat, cmd): chat.edit_chart(message, render.draw(chart) if chart and chart.rows else render.blank(), caption, rows) else: @@ -497,17 +498,30 @@ def on_history(request, args): def picker(request, cmd, args): """A command typed with nothing to say what it is about (cmd, a key of buttons.PICKERS): where there are buttons, - their picker as one message, a 1x1 image (pick.as_photo) that the answer then replaces; else the command's usage.""" + its answer for the first portfolio or ticker (the first two to compare), alphabetically, with the picker's buttons + under it, one message that a press then edits (holdings, which has no first answer, is the picker alone); a pie or + treemap's other charts are then drawn in the background to fill the cache. Else the command's usage.""" dest = request.dest if not (dest.takes_buttons and delivery.interactive(dest)): raise Usage(portfolios()) if pick.noun(cmd) == 'portfolio' else Usage group = {'sector': 's', 'industry': 'i'}.get(args.keyword, '0') period = buttons.period_for_days(args.days, 'x') if cmd in 'cy' else '0' - lines, rows = pick.start(request.service, cmd, dest.key, group, period) + found = pick.items(request.service, cmd, dest.key) + shown, mask = pick.default(cmd, found) + lines, rows = pick.screen(request.service, cmd, found, group, period, shown, mask, 'p') + if not shown: + return dest.text(lines, rows) + caption, chart = pick_content(request.service, dest, cmd, group, period, pick.shown_names(cmd, found, shown)) if pick.as_photo(dest, cmd): - dest.chart(render.blank(), '\n'.join(lines), rows) + request.reply(caption.split('\n'), chart, rows) else: - dest.text(lines, rows) + dest.text(caption.split('\n'), rows) # no chart goes: the answer as text, its buttons kept + if cmd in 'pt' and chart and pick.as_photo(dest, cmd): + prime_in_background(request.service, cmd, group, found, 0) + +def prime_in_background(service, cmd, group, found, index): + """Start drawing a picker's other charts, to fill the cache, in a thread of its own (see pick.prime).""" + threading.Thread(target=pick.prime, args=(service, cmd, group, found, index), daemon=True).start() def chart_reply(request, kind, args, days=None): """The first view of a chart with period buttons, built as a button press builds it (charts.build): its caption, diff --git a/tests/common.py b/tests/common.py index 9f0e8444..a9669e34 100644 --- a/tests/common.py +++ b/tests/common.py @@ -148,6 +148,7 @@ def setUp(self): self.record_chart(dest, image, caption, markup)) self.patch(transports.Transport, 'text', lambda dest, lines, markup=None: self.record_text(dest, lines)) self.patch(watchlist, 'with_holdings', lambda chat: TICKERS) + self.patch(worker, 'prime_in_background', lambda *args: None) # no thread draws charts behind a test self.patch(yahoo, 'fetch', lambda ts: {t: market_entry(t) for t in ts}) self.patch(yahoo, 'fetch_detail', lambda t, *a, **k: {t: market_entry(t)}) moved = {t: float(i * 4 - 40) for i, t in enumerate(TICKERS)} # each stock's move over any period diff --git a/tests/test_edges.py b/tests/test_edges.py index e54d68a2..486e68bb 100644 --- a/tests/test_edges.py +++ b/tests/test_edges.py @@ -154,7 +154,7 @@ def no_history(t, *a, **k): def test_history_without_a_ticker_or_with_extra_argument_gives_usage(self): """.history with an extra argument gets its usage, not the profile of a ticker called HISTORY; with no ticker, the buttons for one.""" - self.assertEqual(self.command('.history')[0][0], 'photo') + self.assertEqual(self.command('.history')[0][0], 'text') # the chart of the first ticker, as text here self.assertEqual(self.command('.history T1 1y'), [('text', ['Usage: .history SYMBOL'])]) diff --git a/tests/test_reports.py b/tests/test_reports.py index 83efe444..0afc2a92 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -929,38 +929,44 @@ def labels(self, rows=None): """The labels of a screen's buttons, by row (the screen shown now unless given).""" return [[text for text, _ in row] for row in (self.shown if rows is None else rows)] - def test_a_portfolio_command_offers_the_portfolios(self): - """.pie, .treemap and .holdings: a button for each portfolio; .treemap's Group by button comes after them.""" + def portfolios(self, *names): + """Sharesight has these portfolios.""" + self.patch(sharesight, '_portfolios', lambda seconds=None: [{'name': n, 'id': i} for i, n in enumerate(names)]) + + def test_a_portfolio_command_answers_with_the_first_portfolio(self): + """.pie and .treemap answer with the first portfolio alphabetically at once, its button marked, the Group by + button after the portfolios; .holdings has no first answer, only its buttons.""" + self.portfolios('Zed', 'Amy') self.command('.pie') - self.assertEqual(self.sent[-1], ('photo', self.sent[-1][1], 'Pie: pick a portfolio')) - self.assertEqual(self.labels(), [['Rob']]) + self.assertEqual((self.sent[-1][0], self.sent[-1][2]), ('photo', 'Amy by industry')) + self.assertEqual(self.labels(), [['● Amy', 'Zed']]) self.command('.treemap sector') - self.assertEqual(self.labels(), [['Rob'], ['Group by: Sector']]) + self.assertEqual((self.sent[-1][2], self.labels()), ('Amy treemap by sector', + [['● Amy', 'Zed'], ['Group by: Sector']])) self.command('.holdings') - self.assertEqual((self.sent[-1][0], self.labels()), ('text', [['Rob']])) + self.assertEqual((self.sent[-1], self.labels()), (('text', ['Holdings: pick a portfolio']), + [['Amy', 'Zed']])) def test_portfolios_come_four_to_a_row(self): """Portfolios are in rows of four before the Group by row.""" names = [f'P{i}' for i in range(6)] - self.patch(sharesight, '_portfolios', lambda seconds=None: [{'name': n, 'id': i} for i, n in enumerate(names)]) + self.portfolios(*names) self.command('.treemap') - self.assertEqual(self.labels(), [names[:4], names[4:], ['Group by: Industry']]) + self.assertEqual(self.labels(), [['● P0', 'P1', 'P2', 'P3'], names[4:], ['Group by: Industry']]) - def test_the_answer_keeps_the_buttons_and_the_group_toggle_redraws_it(self): - """A portfolio button turns its message into the chart, the buttons still under it with that portfolio - marked; the Group by button and another portfolio redo it in place.""" + def test_the_group_toggle_and_another_portfolio_redraw_the_answer(self): + """The Group by button and another portfolio's button redo the chart in place, the buttons staying.""" + self.portfolios('Amy', 'Zed') self.command('.treemap') - kind, lines, rows = self.tap('Rob') - self.assertEqual((kind, lines[0], self.labels(rows)), ('photo', 'Rob treemap by industry', - [['● Rob'], ['Group by: Industry']])) kind, lines, rows = self.tap('Group by: Industry') - self.assertEqual((kind, lines[0], self.labels(rows)), ('photo', 'Rob treemap by sector', - [['● Rob'], ['Group by: Sector']])) - self.assertEqual(self.toasts[-1], ('Loading…', False)) - self.command('.pie') - kind, lines, rows = self.tap('Rob') - self.assertEqual((kind, lines[:2], self.labels(rows)), ('photo', ['Rob by industry', - 'Unclassified: 100.0%'], [['● Rob']])) + self.assertEqual((kind, lines[0], self.labels(rows)), ('photo', 'Amy treemap by sector', + [['● Amy', 'Zed'], ['Group by: Sector']])) + self.assertEqual(self.toasts[-1], ('Updating…', False)) + kind, lines, rows = self.tap('Zed') + self.assertEqual((kind, lines[0], self.labels(rows)), ('photo', 'Zed treemap by sector', + [['Amy', '● Zed'], ['Group by: Sector']])) + kind, lines, rows = self.tap('Group by: Sector') + self.assertEqual((lines[0], self.labels(rows)[1]), ('Zed treemap by industry', ['Group by: Industry'])) def test_a_holdings_answer_keeps_its_buttons(self): """.holdings: the list as text, the portfolio buttons still under it.""" @@ -970,66 +976,63 @@ def test_a_holdings_answer_keeps_its_buttons(self): kind, lines, rows = self.tap('Rob') self.assertEqual((kind, lines, self.labels(rows)), ('text', ['Holdings for Rob (25)', 'x'], [['● Rob']])) - def test_history_picks_one_ticker_in_a_sub_menu(self): - """.history has a period row and a button for its tickers' screen, whose buttons are the tickers held, and - Back; a ticker turns the message into its chart, the screen kept with that ticker marked.""" + def test_history_answers_with_the_first_ticker_and_picks_in_a_sub_menu(self): + """.history answers with the first ticker at once, a period row and a button for the tickers' screen under it; + that screen has the tickers held, the one shown marked, and Back; a ticker redoes the chart, a period too.""" self.command('.history') self.assertEqual(self.labels(), [['7D', '1M', '1Y', '● Max'], ['Tickers ▶']]) kind, lines, rows = self.tap('Tickers ▶') - self.assertEqual((kind, self.labels(rows)[0], self.labels(rows)[-1]), - ('photo', ['T0', 'T1', 'T10', 'T11'], ['◀ Back'])) - self.assertEqual(self.labels(self.tap('◀ Back')[2])[1], ['Tickers ▶']) - self.tap('Tickers ▶') + self.assertEqual((self.labels(rows)[0], self.labels(rows)[-1]), (['● T0', 'T1', 'T10', 'T11'], ['◀ Back'])) kind, lines, rows = self.tap('T3') - self.assertEqual(kind, 'photo') - self.assertIn('● T3', [t for row in self.labels(rows) for t in row]) # the tickers still there, T3 marked - kind, lines, rows = self.tap('◀ Back') - kind, lines, rows = self.tap('1Y') # a period redoes the chart for T3 - self.assertEqual((self.toasts[-1], self.labels(rows)[0]), (('Loading…', False), ['7D', '1M', '● 1Y', 'Max'])) + self.assertIn('● T3', [t for row in self.labels(rows) for t in row]) + self.tap('◀ Back') + kind, lines, rows = self.tap('1Y') + self.assertEqual((self.toasts[-1], self.labels(rows)[0]), (('Updating…', False), ['7D', '1M', '● 1Y', 'Max'])) + self.assertIn('● T3', [t for row in self.labels(self.tap('Tickers ▶')[2]) for t in row]) + + def test_compare_answers_with_the_first_two_tickers(self): + """.compare answers with the first two tickers at once, at the period typed, both ticked on the tickers screen; + ticking changes the buttons, not the chart, until Compare is pressed.""" + self.command('.compare 1y') + first = self.sent[-1][2] + self.assertEqual(self.labels(), [['7D', '1M', '● 1Y', 'Max'], ['Tickers (2) ▶']]) + kind, lines, rows = self.tap('Tickers (2) ▶') + self.assertEqual(self.labels(rows)[0][:3], ['✅ T0', '✅ T1', 'T10']) + kind, lines, rows = self.tap('T10') + self.assertEqual((lines[0], self.labels(rows)[0][:3]), (first, ['✅ T0', '✅ T1', '✅ T10'])) + self.assertFalse(any('Co T10' in line for line in lines)) # the chart is still of the first two + self.assertEqual(self.labels(rows)[-1], ['◀ Back', 'Compare']) + kind, lines, rows = self.tap('✅ T0') + kind, lines, rows = self.tap('Compare') + self.assertTrue(any('Co T10' in line for line in lines) and not any('Co T0' in line for line in lines)) + self.assertEqual((self.toasts[-1], self.labels(rows)[-1]), (('Loading…', False), ['◀ Back'])) def test_compare_ticks_up_to_six_and_refuses_a_seventh(self): - """.compare: ticks toggle, Compare appears with two, a seventh is refused with an alert and a note.""" + """.compare: ticks toggle, a seventh is refused with an alert and a note.""" self.command('.compare') - self.assertEqual(self.labels(), [['7D', '1M', '1Y', '● Max'], ['Tickers ▶']]) - self.tap('Tickers ▶') - for ticker in ('T0', 'T1', 'T2', 'T3', 'T4', 'T5'): + self.tap('Tickers (2) ▶') + for ticker in ('T10', 'T11', 'T12', 'T13'): kind, lines, rows = self.tap(ticker) - labels = [t for row in self.labels(rows) for t in row] - self.assertIn('✅ T5', labels) - self.assertEqual(lines[-1], 'Ticked: T0, T1, T2, T3, T4, T5') - self.assertIn('Compare', labels) + self.assertEqual([t for row in self.labels(rows) for t in row if t.startswith('✅')], + ['✅ T0', '✅ T1', '✅ T10', '✅ T11', '✅ T12', '✅ T13']) self.assertEqual(self.toasts[-1], ('Updating…', False)) - kind, lines, rows = self.tap('T6') + kind, lines, rows = self.tap('T14') self.assertEqual((self.toasts[-1], lines[0]), ((buttons.TOO_MANY, True), buttons.TOO_MANY)) - self.assertNotIn('✅ T6', [t for row in self.labels(rows) for t in row]) - kind, lines, rows = self.tap('✅ T5') # unticking makes room + self.assertNotIn('✅ T14', [t for row in self.labels(rows) for t in row]) + kind, lines, rows = self.tap('✅ T13') # unticking makes room self.assertNotIn(buttons.TOO_MANY, lines) - self.assertEqual(lines[-1], 'Ticked: T0, T1, T2, T3, T4') - - def test_compare_keeps_its_buttons_under_the_chart(self): - """Compare edits the picker into the chart of the ticked, at the period chosen; the picker stays, and a period - button redoes the comparison.""" - self.command('.compare 1y') - self.assertIn('● 1Y', [t for row in self.labels() for t in row]) - self.tap('Tickers ▶') - self.tap('T1') - self.assertNotIn('Compare', [t for row in self.labels() for t in row]) # one is too few - self.tap('T2') - kind, lines, rows = self.tap('Compare') - self.assertEqual((kind, self.toasts[-1]), ('photo', ('Loading…', False))) - self.assertIn('✅ T2', [t for row in self.labels(rows) for t in row]) - kind, lines, rows = self.tap('◀ Back') - kind, lines, rows = self.tap('7D') - self.assertEqual((kind, self.labels(rows)[0]), ('photo', ['● 7D', '1M', '1Y', 'Max'])) + self.assertEqual(len([t for row in self.labels(rows) for t in row if t.startswith('✅')]), 5) def test_buttons_of_another_list_start_again(self): - """A button drawn for other tickers than there are now starts the picker again, saying so.""" + """A button drawn for other tickers than there are now starts the picker again, at its first answer, saying + so.""" self.command('.compare') stale = self.shown[1][0][1] # Tickers ▶ - self.patch(sharesight, 'tickers', lambda: ['A1', 'B1']) + self.patch(sharesight, 'tickers', lambda: ['A1', 'B1', 'C1']) self.press_button(stale) kind, lines, rows = self.edits[-1] self.assertEqual(lines[0], 'The list has changed: here it is now') + self.assertEqual(self.labels(rows)[1], ['Tickers (2) ▶']) def test_without_holdings_the_watchlist_is_offered(self): """Tickers to pick are the holdings', else the chat's watchlist; none at all says how to add one.""" @@ -1037,32 +1040,37 @@ def test_without_holdings_the_watchlist_is_offered(self): self.patch(watchlist, 'tickers', lambda chat: ['MSFT', 'AAPL']) self.command('.history') self.tap('Tickers ▶') - self.assertEqual([t for row in self.labels(self.edits[-1][2]) for t in row], ['AAPL', 'MSFT', '◀ Back']) + self.assertEqual([t for row in self.labels(self.edits[-1][2]) for t in row], ['● AAPL', 'MSFT', '◀ Back']) self.patch(watchlist, 'tickers', lambda chat: []) self.assertEqual(self.command('.compare'), [('text', ['No tickers to pick from: .watchlist add SYMBOL adds one'])]) + self.patch(watchlist, 'tickers', lambda chat: ['MSFT']) + self.assertEqual(self.command('.compare'), + [('text', ['Compare needs two tickers: .watchlist add SYMBOL adds one'])]) def test_with_the_graph_off_the_picker_is_text_and_edits_as_text(self): - """Where charts are not drawn, the picker is a text message, and its answer the chart's lines as text.""" + """Where charts are not drawn, the answer and its buttons are a text message, edited as text.""" self.prefs(graph=False) + self.portfolios('Amy', 'Zed') self.command('.pie') self.assertEqual(self.sent[-1][0], 'text') - kind, lines, rows = self.tap('Rob') - self.assertEqual((kind, lines[0], self.labels(rows)), ('text', 'Rob by industry', [['● Rob']])) + self.assertEqual(self.labels(), [['● Amy', 'Zed']]) + kind, lines, rows = self.tap('Zed') + self.assertEqual((kind, lines[0], self.labels(rows)), ('text', 'Zed by industry', [['Amy', '● Zed']])) def many(self, count=70): """Held tickers S00, S01..., enough to need pages.""" self.patch(sharesight, 'tickers', lambda: [f'S{i:02d}' for i in range(count)]) - def test_tickers_are_paged_32_to_a_page(self): - """Thirty-two tickers to a page, in rows of four, with < and > where there is another page: the first page has - only >, a middle one both, the last only <; Back stays; ticks last across pages.""" + def test_tickers_are_paged_32_to_a_page_under_the_tickers_menu(self): + """The top level has no < or >; the tickers menu has 32 tickers to a page, in rows of four, with < and > where + there is another page, Back always, and ticks lasting across pages.""" self.many() self.command('.compare') - kind, lines, rows = self.tap('Tickers ▶') - self.assertEqual((len(rows) - 1, [t for row in self.labels(rows)[:-1] for t in row][-1], self.labels(rows)[-1], - lines[0]), (8, 'S31', ['◀ Back', '>'], 'Compare: pick up to 6 tickers (page 1 of 3)')) - kind, lines, rows = self.tap('S00') + self.assertEqual(self.labels(), [['7D', '1M', '1Y', '● Max'], ['Tickers (2) ▶']]) + kind, lines, rows = self.tap('Tickers (2) ▶') + self.assertEqual((len(rows) - 1, [t for row in self.labels(rows)[:-1] for t in row][-1], self.labels(rows)[-1]), + (8, 'S31', ['◀ Back', '>'])) kind, lines, rows = self.tap('>') self.assertEqual((self.labels(rows)[0], self.labels(rows)[-1]), (['S32', 'S33', 'S34', 'S35'], ['◀ Back', '<', '>'])) @@ -1070,39 +1078,38 @@ def test_tickers_are_paged_32_to_a_page(self): kind, lines, rows = self.tap('>') self.assertEqual((self.labels(rows)[-1], [t for row in self.labels(rows)[:-1] for t in row][-1]), (['◀ Back', '<', 'Compare'], 'S69')) - self.assertEqual(lines[-1], 'Ticked: S00, S40') kind, lines, rows = self.tap('<') kind, lines, rows = self.tap('<') self.assertIn('✅ S00', [t for row in self.labels(rows) for t in row]) + kind, lines, rows = self.tap('◀ Back') + self.assertEqual(self.labels(rows)[1], ['Tickers (3) ▶', 'Compare']) def test_a_seventh_tick_is_refused_on_a_later_page(self): """The limit of six counts every page's ticks, and the refusal stays on the page.""" self.many() self.command('.compare') - self.tap('Tickers ▶') - for ticker in ('S00', 'S01', 'S02', 'S03', 'S04'): + self.tap('Tickers (2) ▶') + for ticker in ('S02', 'S03', 'S04', 'S05'): self.tap(ticker) self.tap('>') - self.tap('S33') - kind, lines, rows = self.tap('S34') + kind, lines, rows = self.tap('S33') self.assertEqual((self.toasts[-1], lines[0], self.labels(rows)[0][:2]), ((buttons.TOO_MANY, True), - buttons.TOO_MANY, ['S32', '✅ S33'])) + buttons.TOO_MANY, ['S32', 'S33'])) def test_an_item_on_a_later_page_is_the_one_picked(self): - """A ticker's button on page two picks that ticker, and the answer's screen stays on its page.""" + """A ticker's button on page two picks that ticker, and the screen stays on its page.""" self.many() self.command('.history') self.tap('Tickers ▶') self.tap('>') kind, lines, rows = self.tap('S40') - self.assertEqual(kind, 'photo') self.assertIn('● S40', [t for row in self.labels(rows) for t in row]) self.assertEqual(self.labels(rows)[-1], ['◀ Back', '<', '>']) def test_portfolios_page_too(self): """Thirty-two portfolios to a page, in rows of four, then Group by, then the page buttons.""" names = [f'P{i:02d}' for i in range(40)] - self.patch(sharesight, '_portfolios', lambda seconds=None: [{'name': n, 'id': i} for i, n in enumerate(names)]) + self.portfolios(*names) self.command('.treemap') rows = self.labels() self.assertEqual((len(rows), rows[7], rows[8], rows[9]), (10, ['P28', 'P29', 'P30', 'P31'], @@ -1124,10 +1131,46 @@ def test_discord_pages_hold_five_rows(self): self.assertEqual(counted, items) def test_pick_data_carries_the_page_and_what_is_shown(self): - """A button's data says its page and what the message answers, and stays within a button's 64 bytes with - 100 items.""" - press = buttons.PickPress('c', 's', 'x', 0x3e, (1 << 100) - 1, 'm2', 'a1b2') + """A button's data says its page and what the message shows (compare's six tickers packed), within a button's + 64 bytes with 100 items.""" + press = buttons.PickPress('c', 's', 'x', pick.pack([0, 1, 2, 3, 98, 99]), (1 << 100) - 1, 'm2', 'a1b2') button = buttons.pick_button('x', press) self.assertEqual((buttons.parse(button.data), len(button.data) <= 64), (press, True)) - for data in ('k|c|0|x|0|0|z1|a1b2', 'k|c|0|x|0|0|p|a1b2', 'k|c|0|x|0|0|r-1|a1b2'): + self.assertEqual(pick.unpack(press.shown), [0, 1, 2, 3, 98, 99]) + for data in ('k|c|0|x|0|0|z1|a1b2', 'k|c|0|x|0|0|p|a1b2', 'k|c|0|x|0|0|r-1|a1b2', 'k|c|0|x|0|0|s1|a1b2'): self.assertIsNone(buttons.parse(data)) + + def test_a_pie_or_treemap_primes_the_other_charts_in_the_background(self): + """Once the first chart is out, a pie or treemap starts drawing the others in a thread (the first portfolio's + other grouping first); history, compare and holdings start none, nor does a chat without charts.""" + started = [] + self.patch(worker, 'prime_in_background', lambda *args: started.append(args)) + self.portfolios('Amy', 'Zed') + for text in ('.pie', '.treemap sector', '.history', '.compare', '.holdings'): + self.command(text) + self.assertEqual([args[1:] for args in started], [('p', '0', ['Amy', 'Zed'], 0), ('t', 's', ['Amy', 'Zed'], 0)]) + self.prefs(graph=False) + self.command('.pie') + self.assertEqual(len(started), 2) + + def test_priming_draws_the_other_grouping_then_the_other_portfolios(self): + """pick.prime draws the shown portfolio's other grouping, then each other portfolio by industry and by sector; + an empty chart is skipped, and a failure is logged and the rest still drawn.""" + drawn = [] + self.patch(render, 'draw', lambda chart: drawn.append(chart.title)) + self.portfolios('Amy', 'Bob', 'Zed') + pick.prime(TELEGRAM, 't', '0', ['Amy', 'Bob', 'Zed'], 0) + self.assertEqual(drawn, ['Amy treemap by sector', 'Bob treemap by industry', 'Bob treemap by sector', + 'Zed treemap by industry', 'Zed treemap by sector']) + drawn.clear() + real = holdings.prepare_pie + def prepare(name, by, service): + """Fail for Bob, answer nothing for Zed, as usual for the rest.""" + if name == 'Bob': + raise RuntimeError('boom') + return real(name, by, service) if name != 'Zed' else (['x'], None) + self.patch(holdings, 'prepare_pie', prepare) + with contextlib.redirect_stderr(io.StringIO()) as err: + pick.prime(TELEGRAM, 'p', 's', ['Amy', 'Bob', 'Zed'], 0) + self.assertEqual(drawn, ['Amy by industry']) + self.assertIn('priming p Bob industry failed: boom', err.getvalue()) diff --git a/tests/test_util.py b/tests/test_util.py index 5ef26b25..78ac5591 100644 --- a/tests/test_util.py +++ b/tests/test_util.py @@ -172,9 +172,9 @@ def test_a_pickers_data_reads_back_in_few_bytes(self): 'k|c|0|x|0|0|p0', 'k|c|0|x|0|0|p0|a-b2', 'k|c|0|x|0|0|p|a1b2'): with self.subTest(data=data): self.assertIsNone(buttons.parse(data)) - acts = ('p0', 'm1', 'r2', 's2', 'g0', 'h0', 'f0') + acts = ('p0', 'm1', 'r2', 'g0', 'h0', 'f0') self.assertEqual([buttons.toast(f'k|c|0|x|0|3|{act}|a1b2') for act in acts], - ['Updating…', 'Updating…', 'Loading…', 'Loading…', 'Loading…', 'Loading…', buttons.TOO_MANY]) + ['Updating…', 'Updating…', 'Loading…', 'Loading…', 'Loading…', buttons.TOO_MANY]) self.assertEqual([buttons.alert(f'k|c|0|x|0|3|{act}|a1b2') for act in ('m0', 'f0')], [False, True]) def test_a_press_is_told_what_is_coming_from_its_data_alone(self): From 27d61c4847a3b883ae2c1e7cf63860e2b2979f03 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 15:11:03 +0000 Subject: [PATCH 33/41] A bare .compare starts on 7D Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- doc/commands.md | 2 +- lib/worker.py | 2 +- tests/test_reports.py | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/commands.md b/doc/commands.md index 61d63c97..a05f8ed3 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -73,7 +73,7 @@ The bot answers commands in every chat it is in. By default, it fetches its upda * A watch list per chat, which anyone in it can change, picked up by the reports there * Listing of portfolios and portfolio performance * A pie or a treemap of a portfolio by industry or sector (`/pie portfolio [sector]`, `/treemap portfolio [sector]`): the treemap tiles each holding by value, grouped and coloured by industry (the default) or sector -* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap` (the first portfolio alphabetically, by industry or by `sector`), `/history` (the first ticker) and `/compare` (the first two tickers, at the period typed) answer at once, with the buttons under the chart: portfolios four to a row and `/treemap`'s Sector/Industry toggle after them; for tickers a period row and a `Tickers ▶` menu (those held, else the chat's watchlist) with `<` `>` pages, up to 6 ticked to compare. `/holdings` shows its portfolio buttons alone. Another portfolio or ticker, the toggle or a period redoes the chart in place, and a `/pie` or `/treemap` draws its other charts in the background so the next press is quick. Thirty-two items make a page (fewer on Discord, which holds five rows of five buttons). A job, or a chat with no buttons, gets the usage instead +* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap` (the first portfolio alphabetically, by industry or by `sector`), `/history` (the first ticker) and `/compare` (the first two tickers, over 7D unless a period is typed) answer at once, with the buttons under the chart: portfolios four to a row and `/treemap`'s Sector/Industry toggle after them; for tickers a period row and a `Tickers ▶` menu (those held, else the chat's watchlist) with `<` `>` pages, up to 6 ticked to compare. `/holdings` shows its portfolio buttons alone. Another portfolio or ticker, the toggle or a period redoes the chart in place, and a `/pie` or `/treemap` draws its other charts in the background so the next press is quick. Thirty-two items make a page (fewer on Discord, which holds five rows of five buttons). A job, or a chat with no buttons, gets the usage instead * Listing of holdings sorted by various attributes * Running the other stock reports on demand diff --git a/lib/worker.py b/lib/worker.py index e4c56cec..765b422b 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -505,7 +505,7 @@ def picker(request, cmd, args): if not (dest.takes_buttons and delivery.interactive(dest)): raise Usage(portfolios()) if pick.noun(cmd) == 'portfolio' else Usage group = {'sector': 's', 'industry': 'i'}.get(args.keyword, '0') - period = buttons.period_for_days(args.days, 'x') if cmd in 'cy' else '0' + period = buttons.period_for_days(args.days, 'w' if cmd == 'c' else 'x') if cmd in 'cy' else '0' # compare: 7D found = pick.items(request.service, cmd, dest.key) shown, mask = pick.default(cmd, found) lines, rows = pick.screen(request.service, cmd, found, group, period, shown, mask, 'p') diff --git a/tests/test_reports.py b/tests/test_reports.py index 0afc2a92..59ab07ec 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -1067,7 +1067,7 @@ def test_tickers_are_paged_32_to_a_page_under_the_tickers_menu(self): there is another page, Back always, and ticks lasting across pages.""" self.many() self.command('.compare') - self.assertEqual(self.labels(), [['7D', '1M', '1Y', '● Max'], ['Tickers (2) ▶']]) + self.assertEqual(self.labels(), [['● 7D', '1M', '1Y', 'Max'], ['Tickers (2) ▶']]) # compare starts on 7D kind, lines, rows = self.tap('Tickers (2) ▶') self.assertEqual((len(rows) - 1, [t for row in self.labels(rows)[:-1] for t in row][-1], self.labels(rows)[-1]), (8, 'S31', ['◀ Back', '>'])) From 1f284eca76983eb4fc8392ee5b33609058c7f84c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 15:13:06 +0000 Subject: [PATCH 34/41] Compare goes back to the top level, with the period row Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/buttons.py | 10 +++++----- lib/worker.py | 2 +- tests/test_reports.py | 3 ++- 3 files changed, 8 insertions(+), 7 deletions(-) diff --git a/lib/buttons.py b/lib/buttons.py index 2a327cf1..c4c7506f 100644 --- a/lib/buttons.py +++ b/lib/buttons.py @@ -41,11 +41,11 @@ class MenuPress(typing.NamedTuple): class PickPress(typing.NamedTuple): """A press in a picker (lib/reports/pick.py): the command it picks for (a key of PICKERS), how it groups (a treemap's or pie's: i industry, s sector, 0 neither), the period (a key of PERIODS, or 0), what the message shows - (shown: 0 nothing yet, an item's number plus one, or 1 for the ticked tickers), the items ticked (a bit of the mask - each), what the press asks for (act: a letter and a number: p the first screen and m the tickers' screen, each at - a page; f a tick refused, as too many, the tickers' screen at a page; r or s an item's number, to run it and show - the tickers' screen or the first; g the ticked ones, to run and show the tickers' screen at a page; h to run them - and show the first screen) and the checksum of the list of items it was drawn for, stale once that changes.""" + (shown: 0 nothing yet, an item's number plus one, or the ticked tickers packed), the items ticked (a bit of the mask + each), what the press asks for (act: a letter and a number: p the first screen and m the tickers' screen, each at a + page; f a tick refused, as too many, the tickers' screen at a page; r an item's number, to run it and show the + tickers' screen (the first for portfolios); g or h the ticked ones, to run them and show the first screen) and the + checksum of the list of items it was drawn for, stale once that changes.""" cmd: str group: str period: str diff --git a/lib/worker.py b/lib/worker.py index 765b422b..a7f0bd8d 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -160,7 +160,7 @@ def process_pick_callback(press, pressed): if stale: shown, mask = pick.default(cmd, found) per_page = pick.layout(service, cmd)[0] - view = 'm' if kind in 'mfg' or kind == 'r' and noun_is_ticker(cmd) else 'p' + view = 'm' if kind in 'mf' or kind == 'r' and noun_is_ticker(cmd) else 'p' # Compare goes back to the top page = number if kind in 'pmfg' else number // per_page if kind == 'r' else 0 note = 'The list has changed: here it is now' if stale else buttons.TOO_MANY if kind == 'f' else None if kind == 'r' and number < len(found): diff --git a/tests/test_reports.py b/tests/test_reports.py index 59ab07ec..769bcd9b 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -1005,7 +1005,8 @@ def test_compare_answers_with_the_first_two_tickers(self): kind, lines, rows = self.tap('✅ T0') kind, lines, rows = self.tap('Compare') self.assertTrue(any('Co T10' in line for line in lines) and not any('Co T0' in line for line in lines)) - self.assertEqual((self.toasts[-1], self.labels(rows)[-1]), (('Loading…', False), ['◀ Back'])) + self.assertEqual((self.toasts[-1], self.labels(rows)), (('Loading…', False), [['7D', '1M', '● 1Y', 'Max'], + ['Tickers (2) ▶']])) # back at the top, with the period row def test_compare_ticks_up_to_six_and_refuses_a_seventh(self): """.compare: ticks toggle, a seventh is refused with an alert and a note.""" From dfc859366f664e00f6da3990e4de9f26dca48fca Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 15:22:42 +0000 Subject: [PATCH 35/41] Prime the other periods of every chart with a period row; typed .compare gets the tickers menu Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- CLAUDE.md | 4 +++- doc/commands.md | 2 +- lib/charts.py | 21 +++++++++++++++++++ lib/worker.py | 49 +++++++++++++++++++++++++++++++++++++++++-- tests/common.py | 1 + tests/test_reports.py | 47 +++++++++++++++++++++++++++++++++++++++++ 6 files changed, 120 insertions(+), 4 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 63143885..cf77e0a9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -90,7 +90,9 @@ An open PR takes in every commit pushed to its branch, so a PR opened for one sm press re-editing the answer from `shown` (an item's number plus one; compare's tickers, `pick.pack`), 32 items a page (`pick.layout`; fewer on Discord) turned by `<` `>` under the tickers menu, its list of items checked by checksum, `.compare`'s ticks a bit each in the data's mask, a seventh refused by an alert toast (`buttons.alert`); a pie or - treemap primes its other charts in a thread (`pick.prime`, started by `worker.prime_in_background`). + treemap primes its other charts in a thread (`pick.prime`, started by `worker.prime_in_background`); every chart + with period buttons primes its other periods (`charts.prime`, `worker.prime_periods` after `.compare`, `.price`, + `.performance`, `chart_reply` and the pickers; never for a scheduled run). - `lib/buttons.py`: `Button`, a chart's period buttons (`keyboard`), the `.jobs` menu's (`menu_button`, a `MenuPress`; older lists' `j|`/`e|` buttons still parse), what a pressed button's data asks for (`parse`), and the toast it gets at once (`toast`); each transport draws them, and Slack and Discord take a button's data as its id, so diff --git a/doc/commands.md b/doc/commands.md index a05f8ed3..f9b34f9d 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -73,7 +73,7 @@ The bot answers commands in every chat it is in. By default, it fetches its upda * A watch list per chat, which anyone in it can change, picked up by the reports there * Listing of portfolios and portfolio performance * A pie or a treemap of a portfolio by industry or sector (`/pie portfolio [sector]`, `/treemap portfolio [sector]`): the treemap tiles each holding by value, grouped and coloured by industry (the default) or sector -* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap` (the first portfolio alphabetically, by industry or by `sector`), `/history` (the first ticker) and `/compare` (the first two tickers, over 7D unless a period is typed) answer at once, with the buttons under the chart: portfolios four to a row and `/treemap`'s Sector/Industry toggle after them; for tickers a period row and a `Tickers ▶` menu (those held, else the chat's watchlist) with `<` `>` pages, up to 6 ticked to compare. `/holdings` shows its portfolio buttons alone. Another portfolio or ticker, the toggle or a period redoes the chart in place, and a `/pie` or `/treemap` draws its other charts in the background so the next press is quick. Thirty-two items make a page (fewer on Discord, which holds five rows of five buttons). A job, or a chat with no buttons, gets the usage instead +* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap` (the first portfolio alphabetically, by industry or by `sector`), `/history` (the first ticker) and `/compare` (the first two tickers, over 7D unless a period is typed) answer at once, with the buttons under the chart: portfolios four to a row and `/treemap`'s Sector/Industry toggle after them; for tickers a period row and a `Tickers ▶` menu (those held, else the chat's watchlist) with `<` `>` pages, up to 6 ticked to compare. `/holdings` shows its portfolio buttons alone. Another portfolio or ticker, the toggle or a period redoes the chart in place, and a `/pie` or `/treemap` draws its other charts in the background so the next press is quick. `/compare T1 T2 [period]` typed with tickers gets the same period row and `Tickers ▶` menu when the tickers are among those held (else watched) and the period is a button one. Every chart with a period row, from any command, draws its other periods in the background once it is sent, so a period press is quick. Thirty-two items make a page (fewer on Discord, which holds five rows of five buttons). A job, or a chat with no buttons, gets the usage instead * Listing of holdings sorted by various attributes * Running the other stock reports on demand diff --git a/lib/charts.py b/lib/charts.py index 185bb35f..2a128258 100644 --- a/lib/charts.py +++ b/lib/charts.py @@ -1,10 +1,14 @@ """A chart with period buttons, built by its kind's builder: the command's first view, and again for the period of a button pressed, so that both look alike. Each builder takes the network's dialect, the chat's key (a price list covers the stocks that chat follows), the period, its days, then what the buttons carry.""" +import threading + from lib import answer from lib import buttons from lib import formats from lib import links +from lib import log +from lib import render from lib import watchlist from lib import yahoo from lib.reports import history, performance, price @@ -57,3 +61,20 @@ def build(kind, args, service, chat, period=None, days=None): if kind not in BUILDERS: raise ValueError(f"unknown chart kind {kind}") return BUILDERS[kind](service, chat, period, buttons.PERIODS[period][1] if period else days, *args) + +def prime(kind, args, service, chat, shown): + """Draw, to fill the cache, the chart of each period button of a kind (but `shown`, the period already sent), as a + press would ask for it. A period with nothing to draw is skipped, and a failure is logged, never raised.""" + for period in buttons.BUTTONS[kind]: + if period == shown: + continue + try: + _, chart = build(kind, args, service, chat, period=period) + if chart and chart.rows: + render.draw(chart) + except Exception as e: + log.warn("priming", kind, args, period, "failed:", e) + +def prime_in_background(kind, args, service, chat, shown): + """Start drawing a chart's other periods, to fill the cache, in a thread of its own (see prime).""" + threading.Thread(target=prime, args=(kind, args, service, chat, shown), daemon=True).start() diff --git a/lib/worker.py b/lib/worker.py index a7f0bd8d..e030791d 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -11,6 +11,7 @@ import typing from lib import log from lib import answer +from lib import config from lib import dialects from lib import state from lib import formats @@ -170,7 +171,10 @@ def process_pick_callback(press, pressed): names = pick.shown_names(cmd, found, shown) lines, rows = pick.screen(service, cmd, found, group, period, shown, mask, view, page, note) if names: - return pick_answer(service, chat, press.message_id, cmd, group, period, names, rows, note) + pick_answer(service, chat, press.message_id, cmd, group, period, names, rows, note) + if cmd in 'yc' and (stale or kind in 'rgh') and pick.as_photo(chat, cmd): # a new ticker or set + charts.prime_in_background('c' if cmd == 'c' else 'h', names, service, chat.key, period) + return show_pick(chat, press.message_id, cmd, lines, rows) except answer.Answer as e: delivery.deliver(chat, delivery.Report(e.lines)) @@ -397,6 +401,7 @@ def on_performance(request, args): if days < 1: # 0d, or ytd on 1 January raise answer.Answer("The period must cover a day or more, such as 7d, 3m or ytd") performance.run(request.dest, past_days=days, user=request.user, portfolio_select=args.rest or None) + prime_periods(request, 'f', [args.rest or ''], buttons.period_for_days(days)) def on_session(request, args): """.session or .premarket (.postmarket) [percent|SYMBOL]: the moves mid-session, or pre-market (and post-market).""" @@ -413,6 +418,9 @@ def on_price(request, args): top = 10 if args.keyword else None # top or bottom: one chart shows both ends price.run(request.dest, mode='period' if args.days else 'interday', threshold=0 if top else args.percent, days=args.days, top=top, user=request.user, specific_stock=symbol) + threshold = 0 if top else config.prefs.price_percent if args.percent is None else args.percent + prime_periods(request, 'l', [f"{threshold:g}", str(top or 0)], buttons.period_for_days(args.days, 'd' + if not args.days else None)) # the price list's period buttons (price.run's) def on_shorts(request, args): """.shorts [percent|SYMBOL]: the most shorted stocks followed, or one stock's short interest.""" @@ -485,7 +493,26 @@ def on_compare(request, args): tickers = list(dict.fromkeys(yahoo.symbols(args.words))) # each once, in the order asked if len(tickers) < 2: raise Usage - chart_reply(request, 'c', tickers, args.days) + if not compare_picker(request, tickers, args.days): + chart_reply(request, 'c', tickers, args.days) + +def compare_picker(request, tickers, days): + """Compare of the tickers typed, with the picker's buttons under the chart (the period row and the tickers' menu), + where the chat has buttons and the tickers are among those it offers (held, else watched) and the period is one + of the buttons: True when it did; else False, and the chart has its period buttons alone.""" + dest = request.dest + period = buttons.period_for_days(days, 'x' if days is None else None) + if not (dest.takes_buttons and delivery.interactive(dest)) or period is None: + return False + try: + found = pick.items(request.service, 'c', dest.key) + except answer.Answer: + return False + if not set(tickers) <= set(found): + return False + numbers = [i for i, name in enumerate(found) if name in tickers] + send_picker(request, 'c', found, '0', period, pick.pack(numbers), sum(1 << i for i in numbers)) + return True def on_history(request, args): """.history SYMBOL: the stock's change over every period, with its price chart, and period buttons where they go; @@ -508,6 +535,12 @@ def picker(request, cmd, args): period = buttons.period_for_days(args.days, 'w' if cmd == 'c' else 'x') if cmd in 'cy' else '0' # compare: 7D found = pick.items(request.service, cmd, dest.key) shown, mask = pick.default(cmd, found) + send_picker(request, cmd, found, group, period, shown, mask) + +def send_picker(request, cmd, found, group, period, shown, mask): + """The first message of a picker of the items `found`: the answer for what `shown` says (or the picker alone, for + 0), the buttons under it, and the answer's other charts drawn in the background to fill the cache.""" + dest = request.dest lines, rows = pick.screen(request.service, cmd, found, group, period, shown, mask, 'p') if not shown: return dest.text(lines, rows) @@ -518,6 +551,8 @@ def picker(request, cmd, args): dest.text(caption.split('\n'), rows) # no chart goes: the answer as text, its buttons kept if cmd in 'pt' and chart and pick.as_photo(dest, cmd): prime_in_background(request.service, cmd, group, found, 0) + elif cmd in 'yc' and chart: + prime_periods(request, 'c' if cmd == 'c' else 'h', pick.shown_names(cmd, found, shown), period) def prime_in_background(service, cmd, group, found, index): """Start drawing a picker's other charts, to fill the cache, in a thread of its own (see pick.prime).""" @@ -531,6 +566,16 @@ def chart_reply(request, kind, args, days=None): if chart and request.dest.takes_buttons: markup = buttons.keyboard(kind, args, buttons.period_for_days(days) if days else 'x') request.reply(caption.split('\n'), chart, markup) + if chart: + prime_periods(request, kind, args, buttons.period_for_days(days) if days else 'x') + +def prime_periods(request, kind, args, shown): + """After a chart with period buttons is sent (kind, args and the period `shown`: charts.build), the other periods + are drawn in the background to fill the cache, where the chat can use them: a command asked for, with charts and + buttons.""" + dest = request.dest + if dest.takes_buttons and dest.takes_charts and config.prefs.graph and delivery.interactive(dest): + charts.prime_in_background(kind, args, request.service, dest.key, shown) def on_plan(request, args): """.plan [text]: record the sender's plan in this chat, and list everyone's here.""" diff --git a/tests/common.py b/tests/common.py index a9669e34..e6bc47b4 100644 --- a/tests/common.py +++ b/tests/common.py @@ -149,6 +149,7 @@ def setUp(self): self.patch(transports.Transport, 'text', lambda dest, lines, markup=None: self.record_text(dest, lines)) self.patch(watchlist, 'with_holdings', lambda chat: TICKERS) self.patch(worker, 'prime_in_background', lambda *args: None) # no thread draws charts behind a test + self.patch(charts, 'prime_in_background', lambda *args: None) self.patch(yahoo, 'fetch', lambda ts: {t: market_entry(t) for t in ts}) self.patch(yahoo, 'fetch_detail', lambda t, *a, **k: {t: market_entry(t)}) moved = {t: float(i * 4 - 40) for i, t in enumerate(TICKERS)} # each stock's move over any period diff --git a/tests/test_reports.py b/tests/test_reports.py index 769bcd9b..c80f2473 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -1175,3 +1175,50 @@ def prepare(name, by, service): pick.prime(TELEGRAM, 'p', 's', ['Amy', 'Bob', 'Zed'], 0) self.assertEqual(drawn, ['Amy by industry']) self.assertIn('priming p Bob industry failed: boom', err.getvalue()) + + def test_compare_typed_with_tickers_has_the_tickers_button_too(self): + """.compare T1 T2 [period] answers with the period row and the tickers' menu under its chart, those typed ticked + and shown; tickers the chat does not offer, or a period no button has, leave the chart its period buttons.""" + self.command('.compare T3 T1 1y') + self.assertEqual(self.labels(), [['7D', '1M', '● 1Y', 'Max'], ['Tickers (2) ▶']]) + kind, lines, rows = self.tap('Tickers (2) ▶') + self.assertEqual([t for row in self.labels(rows) for t in row if t.startswith('✅')], ['✅ T1', '✅ T3']) + self.command('.compare T1 NOPE') + self.assertEqual(self.labels(), [['7D', '1M', '1Y', '● Max']]) + self.command('.compare T1 T2 3m') + self.assertEqual(self.labels(), [['7D', '1M', '1Y', 'Max']]) + + def test_every_chart_with_a_period_row_primes_its_other_periods(self): + """Once a chart with period buttons is out, its other periods are drawn in the background: .compare, + .performance, the price list and a picker's answer, but not where no chart goes.""" + started = [] + self.patch(charts, 'prime_in_background', lambda *args: started.append(args[:1] + args[1:2] + args[4:])) + self.command('.compare T1 T2') + self.command('.performance') + self.command('.price') + self.command('.price top') + self.command('.compare') + self.assertEqual(started, [('c', ['T1', 'T2'], 'x'), ('f', [''], 'm'), ('l', ['5', '0'], 'd'), + ('l', ['0', '10'], 'd'), ('c', ['T0', 'T1'], 'w')]) + started.clear() + self.prefs(graph=False) + self.command('.compare T1 T2') + self.assertEqual(started, []) + + def test_priming_draws_each_other_period_and_survives_a_failure(self): + """charts.prime draws the chart of each period button but the one shown, skipping one with nothing to draw, + and logs a failure.""" + drawn = [] + self.patch(render, 'draw', lambda chart: drawn.append(chart.kind)) + built = [] + def build(kind, args, service, chat, period=None, days=None): + """Fail for 1M, answer nothing for 1Y, a chart for the rest.""" + built.append(period) + if period == 'm': + raise RuntimeError('boom') + return 'caption', None if period == 'y' else render.Chart('bars', rows=(('a', 1.0),)) + self.patch(charts, 'build', build) + with contextlib.redirect_stderr(io.StringIO()) as err: + charts.prime('h', ['T1'], TELEGRAM, 'telegram:55', 'x') + self.assertEqual((built, drawn), (['w', 'm', 'y'], ['bars'])) + self.assertIn("priming h ['T1'] m failed: boom", err.getvalue()) From 9c4aee90626568207ae713a81fa1f67ea1df2d37 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 15:28:29 +0000 Subject: [PATCH 36/41] cache_seconds defaults to 6 hours; yahoo_warm runs every 4 hours to stay inside it Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- defaults/preferences.conf | 2 +- doc/caching.md | 6 +++--- doc/configuration.md | 2 +- doc/scheduling.md | 2 +- lib/config.py | 2 +- lib/jobs.py | 4 ++-- lib/reports/yahoo_warm.py | 4 ++-- tests/test_scheduler.py | 8 ++++---- 8 files changed, 15 insertions(+), 15 deletions(-) diff --git a/defaults/preferences.conf b/defaults/preferences.conf index 84f1663a..5dd2540c 100644 --- a/defaults/preferences.conf +++ b/defaults/preferences.conf @@ -48,7 +48,7 @@ # Caching and network, in seconds #cache=True -#cache_seconds=43200 +#cache_seconds=21600 #performance_cache_seconds=2400 #http_timeout=18 diff --git a/doc/caching.md b/doc/caching.md index 40d53431..83fa4e28 100644 --- a/doc/caching.md +++ b/doc/caching.md @@ -1,9 +1,9 @@ # Caching -Many objects are cached for 12 hours by default (`cache_seconds`). Cache is controlled by the settings below. Trades IDs are stored on disk, but trade queries generally don't result in a cache hit for functional reasons. +Many objects are cached for 6 hours by default (`cache_seconds`). Cache is controlled by the settings below. Trades IDs are stored on disk, but trade queries generally don't result in a cache hit for functional reasons. ``` cache=True -cache_seconds=43200 +cache_seconds=21600 performance_cache_seconds=2400 ``` @@ -23,4 +23,4 @@ A time is a number and `s` (seconds), `h` (hours), `d` (days), `w` (weeks), `m` Signalling the bot flushes every file too, and then runs both warm jobs (`sharesight_warm`, `yahoo_warm`) at once to fill the cache again: `sudo systemctl kill -s USR1 finbot`, or `sudo docker compose kill -s USR1 finbot`. The log says `cache: flushed N files`, then `cache: warm jobs done`. -Two system jobs keep the caches full, each on a fixed interval shorter than the cache lifetime, so that chat requests do not wait on Sharesight or Yahoo (see [Scheduling](scheduling.md)): `sharesight_warm` fetches the Sharesight portfolio list and each portfolio's holdings report afresh every 30 minutes (`performance_cache_seconds`: 40 minutes), and `yahoo_warm` the price history of every stock followed every 8 hours (`cache_seconds`: 12 hours). Neither fetches anything with `cache=False`, and `sharesight_warm` nothing without Sharesight configured. A refresh that fails is logged, the previous cache keeps being served while fresh, and the next run tries again. A flush runs both at once (see above). +Two system jobs keep the caches full, each on a fixed interval shorter than the cache lifetime, so that chat requests do not wait on Sharesight or Yahoo (see [Scheduling](scheduling.md)): `sharesight_warm` fetches the Sharesight portfolio list and each portfolio's holdings report afresh every 30 minutes (`performance_cache_seconds`: 40 minutes), and `yahoo_warm` the price history of every stock followed every 4 hours (`cache_seconds`: 6 hours). Neither fetches anything with `cache=False`, and `sharesight_warm` nothing without Sharesight configured. A refresh that fails is logged, the previous cache keeps being served while fresh, and the next run tries again. A flush runs both at once (see above). diff --git a/doc/configuration.md b/doc/configuration.md index d7ad143f..74c82095 100644 --- a/doc/configuration.md +++ b/doc/configuration.md @@ -71,7 +71,7 @@ A change takes effect on a reload. A true/false preference takes `true`/`false`, | `country_code` | `AU` | Regional: your country, for the default jobs' messages that are for one (`AU`: the tax year's reminders) | | `timezone` | `Australia/Melbourne` | Regional: your time zone, the clock of the times `/jobs` takes and shows, and a job's zone where it names none | | `cache` | `True` | Cache Yahoo and Sharesight responses | -| `cache_seconds` | `43200` | Cache lifetime in seconds (12 hours) | +| `cache_seconds` | `21600` | Cache lifetime in seconds (6 hours) | | `performance_cache_seconds` | `2400` | Cache lifetime of Sharesight performance reports in seconds: longer than the 30 minutes between `sharesight_warm` runs, so the holdings never run out | | `http_timeout` | `18` | Seconds to wait on outbound HTTP requests | | `log_level` | `INFO` | What the log shows: that level and those above it, syslog style. `ERROR`: what failed and was lost (a message not sent); `WARN`: what failed and was got over (a retry, a fallback, a request refused); `INFO`: what finbot does (each message sent, each job run, each request sent, each request refused); `DEBUG`: the routine of a button press too (the press, its acknowledgement, the message edited in place), each request answered with success, and each cache miss, hit and write. Replaces `debug` | diff --git a/doc/scheduling.md b/doc/scheduling.md index fb36763e..b7093b31 100644 --- a/doc/scheduling.md +++ b/doc/scheduling.md @@ -174,7 +174,7 @@ move as either side's daylight saving changes. arguments its parser has checked, kept as words: never a command line, a script or code. A stored job naming anything else is logged and left out, never run. * **The system jobs**, fixed in `lib/jobs.py` and listed by `/jobs system`, keep the cache full and tidy: - * `sharesight_warm` every 30 minutes and `yahoo_warm` every 8 hours (see [Caching](caching.md)); + * `sharesight_warm` every 30 minutes and `yahoo_warm` every 4 hours (see [Caching](caching.md)); * `cleanup 30` weekly, which deletes cache files older than 30 days. A chat cannot add, switch or run them, and a failure of one goes to the log. diff --git a/lib/config.py b/lib/config.py index ce16e19f..1b76ad7a 100644 --- a/lib/config.py +++ b/lib/config.py @@ -292,7 +292,7 @@ class Prefs: and read as its type says (preference_values). Read them as config.prefs. when they are used, never into a default or a module constant, so that a reload is seen.""" cache: bool = True - cache_seconds: int = 43200 + cache_seconds: int = 21600 country_code: str = 'AU' demote_leveraged: bool = True demote_volatile: bool = True diff --git a/lib/jobs.py b/lib/jobs.py index ca233c40..93a68f4d 100644 --- a/lib/jobs.py +++ b/lib/jobs.py @@ -36,9 +36,9 @@ 'cleanup': cleanup, 'sharesight_warm': sharesight_warm, 'yahoo_warm': yahoo_warm} SYSTEM_ACTIONS = ('cleanup', 'sharesight_warm', 'yahoo_warm') # the system's alone: a chat cannot schedule these # the system jobs, in UTC: the Sharesight portfolios and holdings fetched afresh every 30 minutes (cached 40), the price -# histories of the stocks followed every 8 hours (cached 12), so that no request waits on either; and cache files +# histories of the stocks followed every 4 hours (cached 6), so that no request waits on either; and cache files # older than 30 days deleted weekly -SYSTEM = (('sharesight_warm', (), {'every': 30, 'from': '00:14'}), ('yahoo_warm', (), {'every': 480, 'from': '00:07'}), +SYSTEM = (('sharesight_warm', (), {'every': 30, 'from': '00:14'}), ('yahoo_warm', (), {'every': 240, 'from': '00:07'}), ('cleanup', ('30',), {'days': ['sat'], 'at': '18:51'})) WARM = ('sharesight_warm', 'yahoo_warm') # the jobs that fill the cache, run at once after a flush too # {chat key: {'jobs': [{action, args, schedule, on}], 'off': [catalog keys], 'deleted': [catalog keys]}}: the jobs it diff --git a/lib/reports/yahoo_warm.py b/lib/reports/yahoo_warm.py index c6d9ccb1..7683e1bf 100644 --- a/lib/reports/yahoo_warm.py +++ b/lib/reports/yahoo_warm.py @@ -5,8 +5,8 @@ def run(): """Fetch afresh the price history of every stock followed (every chat's watchlist, and the holdings where - Sharesight is configured), so each is cached for cache_seconds (12 hours by default) from now: the system job runs - this every 8 hours, before any runs out, and a flush at once. With the cache preference off nothing would be kept, + Sharesight is configured), so each is cached for cache_seconds (6 hours by default) from now: the system job runs + this every 4 hours, before any runs out, and a flush at once. With the cache preference off nothing would be kept, so nothing is fetched. Returns how many it fetched.""" if not config.prefs.cache: return 0 diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index 579f4b42..179dc9f4 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -221,16 +221,16 @@ def run(label, module, kwargs, dest=None): class SystemJobs(FinbotCase): def test_the_system_jobs_are_fixed_and_no_chats(self): """The system jobs run on a schedule of their own, in UTC, the same for every install: each warm job within its - cache's lifetime (Sharesight's every 30 minutes, cached 40; Yahoo's every 8 hours, cached 12), and cleanup + cache's lifetime (Sharesight's every 30 minutes, cached 40; Yahoo's every 4 hours, cached 6), and cleanup weekly. A chat can neither add one nor see it among its own jobs; .jobs system lists them.""" s = jobs.Scheduler() with contextlib.redirect_stderr(io.StringIO()): s.load() self.assertEqual({job.label: job.schedule.describe('Australia/Melbourne') for job in s.system_jobs()}, - {'sharesight_warm': 'every 30 mins from :14 UTC', 'yahoo_warm': 'every 8 hours at :07 UTC', + {'sharesight_warm': 'every 30 mins from :14 UTC', 'yahoo_warm': 'every 4 hours at :07 UTC', 'cleanup 30': 'weekly, Saturdays at 18:51 UTC'}) self.assertLess(30 * 60, config.prefs.performance_cache_seconds) - self.assertLess(8 * 3600, config.prefs.cache_seconds) + self.assertLess(4 * 3600, config.prefs.cache_seconds) self.assertEqual(s.chat_jobs(''), s.system_jobs()) for name in jobs.SYSTEM_ACTIONS: with self.subTest(name=name): @@ -240,7 +240,7 @@ def test_the_system_jobs_are_fixed_and_no_chats(self): s.add(CHAT, name, [], when.parse('daily', 'UTC')) self.patch(jobs, 'current', s) self.assertEqual(about.prepare_system(TELEGRAM), ['System jobs:', - 'sharesight_warm · every 30 mins from :14 UTC', 'yahoo_warm · every 8 hours at :07 UTC', + 'sharesight_warm · every 30 mins from :14 UTC', 'yahoo_warm · every 4 hours at :07 UTC', 'cleanup 30 · weekly, Saturdays at 18:51 UTC']) From 555c5acd56651b5f2585206aa2119d3b86e83865 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 15:32:00 +0000 Subject: [PATCH 37/41] Compare's tickers button reads Select... (n) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- doc/commands.md | 2 +- lib/reports/pick.py | 2 +- tests/test_reports.py | 22 +++++++++++----------- 3 files changed, 13 insertions(+), 13 deletions(-) diff --git a/doc/commands.md b/doc/commands.md index f9b34f9d..855090ff 100644 --- a/doc/commands.md +++ b/doc/commands.md @@ -73,7 +73,7 @@ The bot answers commands in every chat it is in. By default, it fetches its upda * A watch list per chat, which anyone in it can change, picked up by the reports there * Listing of portfolios and portfolio performance * A pie or a treemap of a portfolio by industry or sector (`/pie portfolio [sector]`, `/treemap portfolio [sector]`): the treemap tiles each holding by value, grouped and coloured by industry (the default) or sector -* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap` (the first portfolio alphabetically, by industry or by `sector`), `/history` (the first ticker) and `/compare` (the first two tickers, over 7D unless a period is typed) answer at once, with the buttons under the chart: portfolios four to a row and `/treemap`'s Sector/Industry toggle after them; for tickers a period row and a `Tickers ▶` menu (those held, else the chat's watchlist) with `<` `>` pages, up to 6 ticked to compare. `/holdings` shows its portfolio buttons alone. Another portfolio or ticker, the toggle or a period redoes the chart in place, and a `/pie` or `/treemap` draws its other charts in the background so the next press is quick. `/compare T1 T2 [period]` typed with tickers gets the same period row and `Tickers ▶` menu when the tickers are among those held (else watched) and the period is a button one. Every chart with a period row, from any command, draws its other periods in the background once it is sent, so a period press is quick. Thirty-two items make a page (fewer on Discord, which holds five rows of five buttons). A job, or a chat with no buttons, gets the usage instead +* Buttons to pick from where a command needs a portfolio or tickers and is typed without them: `/pie`, `/treemap` (the first portfolio alphabetically, by industry or by `sector`), `/history` (the first ticker) and `/compare` (the first two tickers, over 7D unless a period is typed) answer at once, with the buttons under the chart: portfolios four to a row and `/treemap`'s Sector/Industry toggle after them; for tickers a period row and a menu button (`Tickers ▶`; `Select... (2)` for `/compare`, with the number ticked) (those held, else the chat's watchlist) with `<` `>` pages, up to 6 ticked to compare. `/holdings` shows its portfolio buttons alone. Another portfolio or ticker, the toggle or a period redoes the chart in place, and a `/pie` or `/treemap` draws its other charts in the background so the next press is quick. `/compare T1 T2 [period]` typed with tickers gets the same period row and `Select...` menu when the tickers are among those held (else watched) and the period is a button one. Every chart with a period row, from any command, draws its other periods in the background once it is sent, so a period press is quick. Thirty-two items make a page (fewer on Discord, which holds five rows of five buttons). A job, or a chat with no buttons, gets the usage instead * Listing of holdings sorted by various attributes * Running the other stock reports on demand diff --git a/lib/reports/pick.py b/lib/reports/pick.py index 91638798..c304ae86 100644 --- a/lib/reports/pick.py +++ b/lib/reports/pick.py @@ -106,7 +106,7 @@ def button(label, act, active=False, **changes): rows = [] if cmd in 'yc' and view == 'p': # the period, and the way to the tickers rows.append([button(buttons.PERIODS[key][0], 'p0', key == period, period=key) for key in buttons.BUTTONS['c']]) - more = f"Tickers ({len(ticked)}) ▶" if ticked else "Tickers ▶" + more = ("Select..." + (f" ({len(ticked)})" if ticked else "")) if cmd == 'c' else "Tickers ▶" rows.append([button(more, 'm0')] + ([button('Compare', 'h0')] if to_compare else [])) on_page = items_on(found, page, per_page) if noun(cmd) == 'portfolio': diff --git a/tests/test_reports.py b/tests/test_reports.py index c80f2473..512d66eb 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -995,8 +995,8 @@ def test_compare_answers_with_the_first_two_tickers(self): ticking changes the buttons, not the chart, until Compare is pressed.""" self.command('.compare 1y') first = self.sent[-1][2] - self.assertEqual(self.labels(), [['7D', '1M', '● 1Y', 'Max'], ['Tickers (2) ▶']]) - kind, lines, rows = self.tap('Tickers (2) ▶') + self.assertEqual(self.labels(), [['7D', '1M', '● 1Y', 'Max'], ['Select... (2)']]) + kind, lines, rows = self.tap('Select... (2)') self.assertEqual(self.labels(rows)[0][:3], ['✅ T0', '✅ T1', 'T10']) kind, lines, rows = self.tap('T10') self.assertEqual((lines[0], self.labels(rows)[0][:3]), (first, ['✅ T0', '✅ T1', '✅ T10'])) @@ -1006,12 +1006,12 @@ def test_compare_answers_with_the_first_two_tickers(self): kind, lines, rows = self.tap('Compare') self.assertTrue(any('Co T10' in line for line in lines) and not any('Co T0' in line for line in lines)) self.assertEqual((self.toasts[-1], self.labels(rows)), (('Loading…', False), [['7D', '1M', '● 1Y', 'Max'], - ['Tickers (2) ▶']])) # back at the top, with the period row + ['Select... (2)']])) # back at the top, with the period row def test_compare_ticks_up_to_six_and_refuses_a_seventh(self): """.compare: ticks toggle, a seventh is refused with an alert and a note.""" self.command('.compare') - self.tap('Tickers (2) ▶') + self.tap('Select... (2)') for ticker in ('T10', 'T11', 'T12', 'T13'): kind, lines, rows = self.tap(ticker) self.assertEqual([t for row in self.labels(rows) for t in row if t.startswith('✅')], @@ -1033,7 +1033,7 @@ def test_buttons_of_another_list_start_again(self): self.press_button(stale) kind, lines, rows = self.edits[-1] self.assertEqual(lines[0], 'The list has changed: here it is now') - self.assertEqual(self.labels(rows)[1], ['Tickers (2) ▶']) + self.assertEqual(self.labels(rows)[1], ['Select... (2)']) def test_without_holdings_the_watchlist_is_offered(self): """Tickers to pick are the holdings', else the chat's watchlist; none at all says how to add one.""" @@ -1068,8 +1068,8 @@ def test_tickers_are_paged_32_to_a_page_under_the_tickers_menu(self): there is another page, Back always, and ticks lasting across pages.""" self.many() self.command('.compare') - self.assertEqual(self.labels(), [['● 7D', '1M', '1Y', 'Max'], ['Tickers (2) ▶']]) # compare starts on 7D - kind, lines, rows = self.tap('Tickers (2) ▶') + self.assertEqual(self.labels(), [['● 7D', '1M', '1Y', 'Max'], ['Select... (2)']]) # compare starts on 7D + kind, lines, rows = self.tap('Select... (2)') self.assertEqual((len(rows) - 1, [t for row in self.labels(rows)[:-1] for t in row][-1], self.labels(rows)[-1]), (8, 'S31', ['◀ Back', '>'])) kind, lines, rows = self.tap('>') @@ -1083,13 +1083,13 @@ def test_tickers_are_paged_32_to_a_page_under_the_tickers_menu(self): kind, lines, rows = self.tap('<') self.assertIn('✅ S00', [t for row in self.labels(rows) for t in row]) kind, lines, rows = self.tap('◀ Back') - self.assertEqual(self.labels(rows)[1], ['Tickers (3) ▶', 'Compare']) + self.assertEqual(self.labels(rows)[1], ['Select... (3)', 'Compare']) def test_a_seventh_tick_is_refused_on_a_later_page(self): """The limit of six counts every page's ticks, and the refusal stays on the page.""" self.many() self.command('.compare') - self.tap('Tickers (2) ▶') + self.tap('Select... (2)') for ticker in ('S02', 'S03', 'S04', 'S05'): self.tap(ticker) self.tap('>') @@ -1180,8 +1180,8 @@ def test_compare_typed_with_tickers_has_the_tickers_button_too(self): """.compare T1 T2 [period] answers with the period row and the tickers' menu under its chart, those typed ticked and shown; tickers the chat does not offer, or a period no button has, leave the chart its period buttons.""" self.command('.compare T3 T1 1y') - self.assertEqual(self.labels(), [['7D', '1M', '● 1Y', 'Max'], ['Tickers (2) ▶']]) - kind, lines, rows = self.tap('Tickers (2) ▶') + self.assertEqual(self.labels(), [['7D', '1M', '● 1Y', 'Max'], ['Select... (2)']]) + kind, lines, rows = self.tap('Select... (2)') self.assertEqual([t for row in self.labels(rows) for t in row if t.startswith('✅')], ['✅ T1', '✅ T3']) self.command('.compare T1 NOPE') self.assertEqual(self.labels(), [['7D', '1M', '1Y', '● Max']]) From 69a4acfd1dd92f589cf030945c382b3e31870f7e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 15:34:39 +0000 Subject: [PATCH 38/41] Pie legend text is bigger, shrunk only where a name would run into its share Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/render.py | 12 +++++++----- tests/golden/pie.png | Bin 7312 -> 7872 bytes 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/lib/render.py b/lib/render.py index 0436f9f2..12656f38 100644 --- a/lib/render.py +++ b/lib/render.py @@ -505,15 +505,17 @@ def draw_pie(chart): ax.set_xlim(-1.03, 1.03) # matplotlib's default 1.25 leaves a wide margin around the pie ax.set_ylim(-1.03, 1.03) ax.set_aspect('equal') - step = min(0.075, 0.74 / max(len(rows), 1)) + step = min(0.1, 0.74 / max(len(rows), 1)) y = 0.5 + step * (len(rows) - 1) / 2 - 0.03 # legend block centred on the pie + names = [] for (label, value), color in zip(rows, colors): - fig.add_artist(Rectangle((0.57, y - 0.014), 0.018, 0.028, transform=fig.transFigure, facecolor=color, + fig.add_artist(Rectangle((0.57, y - 0.019), 0.024, 0.038, transform=fig.transFigure, facecolor=color, edgecolor='none')) - fig.text(0.60, y, label if len(label) <= 28 else label[:27] + '…', color=ink, fontsize=8.5, ha='left', - va='center') - fig.text(0.97, y, f"{value / total * 100:.1f}%", color=ink2, fontsize=8.5, ha='right', va='center') + names.append(fig.text(0.61, y, label if len(label) <= 28 else label[:27] + '…', color=ink, fontsize=11.5, + ha='left', va='center')) + fig.text(0.97, y, f"{value / total * 100:.1f}%", color=ink2, fontsize=11.5, ha='right', va='center') y -= step + _shrink(names, 0.26 * fig.bbox.width) # no name runs into its share _heading(fig, chart.title, chart.subtitle) return _png(fig, bg) diff --git a/tests/golden/pie.png b/tests/golden/pie.png index 822c096c9cc44eb3a6a5ed9e6e2edae67865b080..72fc81cf8b6f6473b4cfbeec30b4f35fff578ed4 100644 GIT binary patch literal 7872 zcmV;x9zWrUP)003kN0ssI2j?}E!0016`NkleTEb>=yDc}t_w-2g~{Ai1|XR2ng zGhDN#2wE;>dZJ9yJ&|mClWRt(0O6D^<_2?I?dWaU2UF0ElyLj8RG< zqA_$HTIRYA0EA(vjS-w{t+h6mCF3|oM6ESrEQ(?x@_b*#iZQ0OR!RvWlv040%%|y5 zD}WJEDfQ(q|F=gsJu-j6{1;w$v9q(YQmL$5dHrWU`{`}lwkxGdC&Lrf;kG$b_ySZni{Pk*YZDLYgsa?Zc_$6r{tZe19J zzjNoE>(}2F$ML@1yKY;5`vdpiyLRoGdGqJrbmQ7}x2zLF?%loPLm&RI>$;zN;9WTQY*cF{rU$UxNqh4D_dJz zAKkop-@bjtQb}+=4awvVHr4oYLy((PNHnPg~-e+u$gU zzw@2%=CawJKl@zk%o*FZZU6btpUq~oy?wnu{Na<@U=RkWhSbqx$N%B`Ke+ead%C(# zJoVJSX0zEK2#y{-cKrD9GpEn&-m~Yex881@Ipc*FU)s8LYfDS>maVU!=sKab-nsM5 z-Fx;dTDWNEt~a~7y88NhAANN5kDmO|(PPK#X@F}maI_bVh{ZyoA>H8nfohdlUPth1VCeS5Co3n3c)Lts+7XE z?Jx{E=ZGwdLd&v@F-wqG9LHf8w6x5K zqR8_q04Sx^(9lpWm(q=mLxY2rN+q36GtQ0H7b7$}QUV}j-iftF;-V;uf*?qz)6+=Q zXo!&`V~k~4H>|$#_^~6N=XEY!hRA&1Yieq#R?Gc;y|dcqL}8F>Nc+AQMbXmbD^(mB zV~!s^GB}tY0q^3WwZ7nBWJoD(+rIex^r(pqW{g>u<$0dxRnqCkzP{d6Div!LM^QGH z^L?L)JkKi?3%>6)G^B?H2a&Njj!LECyVkMOcDH^un21yyb0J2UEMtriBB3b~Dd(IM zkuipd5>SUJt?BVTSJ8=Sv}`*eBojnA7bE0iBI1d_F~$H9&KuO}=lGie@TC+cjiXzq zAKz~tz;!bXqr@JQ$%4qvPa6UtpmC?y^mxAkoW#e;>Xb1$G2_Wv6*49m>DWhPfC$Ku zabyCRIKQH0rWMBf6X0P=O)V{%OxAT9EX(4Y z1E6KQy{8Xu>202~>CaSk2oOofCi1j57#Gim14o1IeL>IusBlUL9&(FG7a5zD=_vy^ zVX2}h0;0xDwtaR-$K3fXGiIhzX)ZV!GR7FAlQm`{B7i80NQKHPYQKnF03r%o<~ppF zh3O@C5gF|l!u})PiPtN~UJ82m>%e1H0~6LX%yk7gfi?_7Aw);VyhTfv&YsheN~MWN zYo%1IR0J@>SC9N>hzN+tI5G~%002mn(xgHnLImV8)p29P{99UX`?TNvR%!p!OF#s)Cuz2O_8)vr8B4gq>9);_qOd>YyuSHcN z5~-FMcl=Ry^<#zIPY!PXp04yWCkp`6F4&|3oFv^T-PGK&_QrMd<}V^LQ5e>i-a>{5 zq+{jfnU$LP!7pT1JTma=H;Q|If?P7`OpEg9!QR715h6thB9prN?gthw>Wso5NwKR3 zd5MTj0BPmrWoFjwO@H3;*k6jqS=uikn_gOe?*MZKehiI{^~Bxkqif#UwL|GC$&5r~ zf>fyeLgTUr7X0BqHm=y9ydgk@X@>mW1I!T00d}^E|8PGnnqjY=f3#hlc>93ux@4xp z1R_JmwO`;)w&VByqV=9HnaBg9haK{1fUg?eYRJ|v7N5ThGY1hKVo&CZ?!mHZHw za;gGTWSn$l!t%^JzcA;~{{_fMM}T}9;HyNp60&<&@K4+ZL=b7rz%V&)DuKItLDxJyw&yK7 zHARa09h8Og@~MTl&HepvGpm7Aa4qXagYh^4W;JZ)bZFiQCMK&YZl{)-8N@X3E3?R`C~VO7oDe>#thO%xK{Ac((OC9 z^7nyOL?%>qsA0kS*&Dx3S|uya(*R#acD)EYTlvS=L8Jia(hrKE@gjHSyi<$ik=?te zTu}o6a-qC@X2piqd%mQDA~H4&@Wl@aAU4ow^FO#1BnNHAi!T|PN@zd4dRO6Cv8P{H z)|6*VA`_}Q*nI1!a%(=Kydh*_8sLi%R+@kGW^Btq5D$A*;|ycCC8LLzZ{PJgUP_Zs z0mg_XsXfY^E=2suitT&e00Zv_S{RY*u+p;r39EU5QNhG7DJj_!oHsT$#&KM&R--6N z%56N)S4zcktd#P7KRNI`KZ>F-3@07qqz)lLR0uJGtLvtc+5@E&yU3Beuzd6#0UxtBQq+P^}ttJ@A{MM4UcA*KU6vLd{8aTnbYx!Py9i#PzZwX`s-J4&j0g& z`qHxHOY`|W0A{mUA^80H^QDwh%EgNp+m_YV)|SubC$VG_5yK$pm^c5rT_B-zi zf}pRb`=0wAYDhO2L(5iN&lp35N~I#iJHF&)yh;K<;P);E!61590VHRp5;*s-+VicK z+fUEUwzsLvV_ZM>BPeB8ZYu2h$FT1pO7{}k0Yox8$DVneQJx7(fGm-WzW)A9W8?kz z-xtU6s#U8F9XbSv&CSglHazUQ?yg;Lwlp`-oj3Q$k)y8bE?T&NbMe=I^EbV{eMyIt z_kQt4>xRb0&ZWy$9A~mQ0CZC+ZFEof*-R$e(9pmTmvk=e>+SKX)nw83@+z7k1Tij~ zh0EK)kFHvLM?EagnA*7VjlJ7I>-U3WLPm2d-TZ-1ke=vlnO z;d!2va?YH&#llb~n;py#oIZ83ucv!vYnx?RbiP9pTu^*AN<@B97ysf{f{h>eA-VoG$xgLSkiSKoSIwN|kdSMNuY` zNdQhPbSBf-+uO$`m0B^zC>7hbLqtu@Eyco+l+tlrZFI3va2z*^LN}H2J#S3e$b|qS zLZom`PW=wgipGk>0I+R4IU|IS zQjGL|lTu14lH;Vzx~9>Qlrs5g&IK1@q=NTiopBMs+CbXDRdXOxlfy$HQV`kmTKjKq zd29PN+i}g*QXhy$g;w*z)Vy0v96ZfDM5yaq&!u z^(3zoAX1?0IW75H+xBjM!*ZOdRMd#1LYeJw+LoL6%B&U877B&VB};NmP4ni>&E;}LuxQaD&Uv|9u3fh;A`(I_zkXH6yaj>p zuejnD&J_q4aqs%bMQ##j`E2lnsNN}WA@s;{SKFh2kYw&MyRq_jBai#wOL z&+bU1{PHS#b`EM6$skgstXWM%ceL-`_PRBdy$c!X*lJq9oh%qNj=Dlbt#lZL)vC8- z$r2(;HKYz4IIwTu+d$CYKVaK-_qp!Hix=x!vUSE7DXj+|-c+rWvrSD>io(#~`~{2R zI5Nic_4b@OeX_5&=VB&HT;T3vTV|5N-^(g$Qx4qQmmNIx%A7+-+OJzodP;KTkxVSo zGex>J%6Fj@!ySVF&INS1LZOC=22EsHR(E&zGtd0IT&@fb4!-{S8^vNV`Hrrx6Hydp zv)L_Mwib#-+qP@xSReoZz53GghKyEv@9tem-^roD!Bj(PXmHTB?cSd5lP8Y*p2s*J zksL8bd@as$E+`E|V8(nmOk#QlB1DRBUcLQqcRbkMZa1cF{Z!24*?kGiE8nEFZnNo zh@uD)y^F|T_6~}M$hGv$3^)?g4o}-WZSj=F9my@h1!D{u01_b)n6ZD9{7sk)j%Os*hc2cs?~9v#AmiFYt-6)(n^dcC+l#{Qc5A^NJ2q!CCP!@u-UT6WrQLtp9;EEZwN=W}u1zTmQx>|CIg;CgS)5DGIAaW1BGMz!8UjN^=l@>} z_h?O!VoBe&0>|=QTc<3Yn$RRjLbmgh0LTykpdX8kH*Ejz zo}1?_Vj?{y7;`uG+s$N&(I(gxIJ?DeBjgAi0Ra#Q2*{8&6zeEbp|6Y=Q^lilk&0?~ z5)2tu09Hg^;-buD#j<=yhK}vKmP*-XcoIl-EanJq3vO{{a|S3oS}L2D?*99&;<^t_ zjWcbfCK&OL5DSTN9_iLOdUKgNP|z1xlabDnm)pEXj1B<>qS6#-!i&@vA|hvuL!$#( z1ct~F*rg^xsKP**YD|?Xl?zmip`dX{L&2~D@Zm`yWHr$Qwp1-QQRuHl7)dz%3 zOeC}!agQCx@qNEqt)@~b+qO%kQgX6VsYofU8mEFMjbY(@S6E|=W!YM*D2k-Cl!~R4 z7kKjGVnfL;=PwfxfEu-^a>+@+lS#mUA#xZq35+3aOsv94`+iK-m?|FmWp=nV`nIVW zGu2riiF^O?b;D$B5bN)}78R=<^XG2fxba_~e)^FO8(w_rrE1m7<+6Q!eQm96#Zt+2 z-NT0u-*Lz7XU?4M?(P;s)Cgm()vdSRdG74#{cr7kV8f<^`}g&nJG-QFX=~f8p6+wm zT&}OTr&1~B2L?vZGckt9nt+bM^N#|?p9JRelRz*odB%kt0wTJcFDZookdF*e-Qh*j z@t9jCz~%;=orOq2H3o)6gaFOWO`A4tjH0NerDg5fwSx0=Xq<_t_j0% z(V~Ul|NajqD+%k8e9NhmT?nwab7^zS499VlR!0sWv}{``C4^jjshq4a00$ZrO@flV z@+2To=MN(Wgxm>-l!`6HIy+h>_O9SX6Cq>!_Psrj&!^LAOUlB~(A)d>uUWlX2)_To zfpWRLYSl_0s-5hPU_5(H2WR~Fu_Iv^pFMl-bD#apYg@L|&cqOrw5%hC4lu@hd%Amj zx`hyyW$)RwLt2*Wrh0q2l{$UuWY=YxEQU}~R8^GXSkFF0hQQ&x(NqQ^5A7UiRX0zI zh_pBGj>|y=lItJu9D_JxDpmlHPNzN38@&l5j4FEg%m4dVrBdkvqJPpyeqaCqY}+OR zt+nS>lis~th~#1}%VZG%M<^>8j6kruw%H&e!B`HwIH(}m=hQs>bCWWb(cLC?BO2Ybp#E{K^%BZ#Kq@HgK#;Q;_$F*vw&)xDB-Yt z-v@vo2$Qf}HpqEV55Za1=b0<+{9t-s^0*0y%7bR35-KL3j#@KdSO=1m)=C666D-qzOE-`}6j zWPbUpUpF>3W-^)n{{B^~RvtQZxDI)DNiFO06)W54bR63M_Q{i7_doPVVQ8@X+_`xR z7IdFGTOEhV0sxLsiebl)xv_~?>&OpkLmA1WKWL1p2f{=s?6`O)=sCco17j1`)9J=s zQ;UjK6#1Mp&+~4(?bhz@p66e9;ZMKv<$e413C`PRw{Lj(p(RU}D5ZLOdOY9Xy=Si! zb!q@Y0Gx}3OFEApIn>+T-I&Qtw7M_}mR=h$k*+K0ZwS(o+tH9Vx(=Tg(d>Wv5y`(I(J^j;~)RnOE16d`931P@WP9gN~K&bpFDN4x36#e_8q!j+n003 zy3d_K#NM9nx$_qc4dutLE)*9$=#+rHMblNK`3)G>MVt^o6Qu-qr3juN~K&F$|ow?7$c?Z?dyB$si!lU#$(5hcXZ5&W950?sZ*z4dG)nyCNoqhAYy$s zKTdkyYpr)|+fphP9oGe-?Qd);779TS^mLy)IS!LG{2E~}g6I3q6B+Jn;8-6CCP(tL zE)7J1!@GA$-sCs#O3GLPSwGZcf%HrDSt>rbn0JI-xu4-*J|qY8+M(Sxuu- zBEw{=eDD|QO8R*W*Cs@gG%2MNLX1R5)+iRKp1hEdGCAO!UtM*f0dP6IU8L9YCM}>o zF)KfUAhg~)> z6|gl=Po2}*+6!^QR^}w2pQjoH!bHgJxZIoH`Yn`KVSRsVCD;%+9KLZ(_xND3nlHC# zqMSq?4Ax3~Npm}$-||<{;0Y$2YiGwpHjd787@Vlm|30c3L~RSw5J3|(N+!c0T?4j~ zjOI?Za{QH{9Y5es_L|x`i;XK)kh9=NXY^C&Op`U7Fk3sWBQ2Q|4QU-Cu*nr(Vf!Jg&KIRj4>uGspKhDRB<%9t(J%kw^F^o{94d^kXa4a z+UAB=0vNz%Fc86)-ijjwvS!BE5ZRnJ>avP_L^fHG5*Z~LTJo>|ZDIF6iBuCA1=rZP zvNF^J838*6>A$`mJ8T&BT7*fG^9)tvs0wUyQtwDBk!mR&c&6|9uX86e^{vU@p8x}Z zB51PUCuhyq4yvrYib~+CQKZBfYr;VJrAd)sGDkunSjaD3crN_Cvx0u z!FRg!*AJHKczj;)D?SM)P zU@*zC69F~}W&HP=dY9PUO4pXtPk!1&RTR#(yleE;1tf-`$%1bk*I(MJID*3`Gj=in zO~Q~j^zJvkB?Dwc!;IovKRffI&zRUp;Q(FJ`$b>%qfB)5kI(2&?u`3FYLcKP!z2&q zR!YDiPw#4dm5fGiF}tz<<^OZ`r(XmjBqr-R~|fByG<&wrg+DPa6s*39n-;KZ-aNEnFdbGzcN9#GmqRt`7Z zd6yE5pb0V-w}#ou3Cb{$Q7CNg=E?`3JMq2WFCTbXq?(3X!%UCMZptw+Ikp)P!ncp> zR|d?VTql0Fm1%;i8g3JC^sVf~P?(5FBez5<7v_%-y!wse-k%_s+|6B6 zRC5ghhVx!@vjxYi^!Z)!y{+ssi};${Fw8VgH;7ByU~nZ*t7170Epd5AnhX&!VRI`N zmwN}d{oUa9@99cEbFzsgo(6T&03K1-E(ZX3uGhRWVD4*UkI&<4a!evrHN*yBVn|cM zR+`FC{Bchz72K@A#d9K&o z(#$r`=IdKns|^GYXo$?PvyA|FDZCAV5Ijp>98|C|2Z{ibwnYF;FzFzdCi1*9yGr|? zE+6`3Ts+I9D^g8l%rv^S1$e|N13*rKA$VoLywq>zrEq-*oy9f>Ee3`lHlPS} z(gJa;4F+Uc3{+4;CQ%p^HZnnEv|sRg_IoE^uN-|T=-qE3k68^O)kHK6Z5;s~4Rcxm z0`!IS!_)f7)4I)p&NN$<#bu3behO#Wm==%{z!6xYh!K>5CX2PpNQY%IDk_{&rS5Rx zXwbbc=-D3?PMXL^ZcR(=)rBXQ*cfXPza4-f5I}4oG@uM{ga!dQ3C$KZTbPm1z)><_ z2$2CzaJZz7^0^XCT003kN0ssI2j?}E!0010UNkleTEmEJk`-nVr(x&bsc0^lN&kVuLWNm-+XvSmruW?Pgr8QBwiD$Z2Adn&F>D)EnO zPNl|4l`B7zj32 zC7?kMaDM=eSFand?|063mV55G*wM2ZrVT_SBF3081`z>(h>|Nrl>CW^$!&-T$yI>l ze#Y3;J=5Pfb$@atxhMILQ`cd6EWqHYOzZT&l~Rmhz2Uo#6UQ+TS(ZgaQc58N0CCQZ zF-j>!G=?rg%RJ8ofGCQzF@kfgwbsVAEh!};YONV#aV&|*4+13>V@zwUloCQHr2sLJ zr{%Fw03)JO>Wg3ew@*I)*y=T_UwrWw0|Nu~dVTZen?L>OPww8mM=4dU)t-2K#|J<7 z{@uIxc%H{O=Zv*vTS~>^M?U%yM66Y-9r=8@QUQR6AA0aZAA0IHzuB3|WIWG{qNuNL z1tNa(Bnt?z*5*^6{6t$@u2imY&j0kEeg2L+ z?uf$Zz4zX`ef!-~%EN>E@7{jTBM(2gZQIsWt5?74wrzLZd4~|z!NI-1{|A5IdERF} z`{@sU;L(QffApjOaQj_%24S>q+ZM+76Hn||+28-U&whG*eC!jS__$Vj`}XaRJo3=y zn>TlLbv^mS6Ne8Uu2idn^JPfpcd(fT7&zyr&zyA}=iT?*Gd_N?wY7Eq`t=iqLg4#1 zZ`wpAPN&nOqhq;T?)>@l{rxNNd+)u!{N=Bh<+ZoB_x7$B85#Nci@$i_{`>lRdrzG{ z?RlQ#IFCN^&@G!cpE-Nh5BzQ0whay*IQ;72TrT^g=U@2Z7ytd|KKmJ^)zHuh*Kw9D z@q*jnSjumFh-#9+m>ZD8V$x+CX?ZuhhY$gkuj#Hr)S5G$6k2h$JJ^zxhJuphzNkj=r9aj*As%* z>kZ2ij^jj8#5qS~aU9vUZH&3;rW?0z-TM9S|9~%(^FjlRh;bbE_Vvp+ZZztaWlJfe zD9q-9_~!#Fou&qio^qy#|5yb)_n#YJ%(hhdn> zWR{UwpdqGS8DnhQzIDrOCr=#reShGl4TvlV{I<4yqfxs!KGw6MFOI@gIuitb9LG0p z+^l45j5#@UyjUzufj4_-tuK2qH7v_=9B20R<*^_(m@#JCw(tACU(aM(#>dA}sg%@8 z#&NE-H3$MC@_oNrDF=a{PG?HRA~GgrT&-5#Hped8-R9X~B2rRvA*M1}#uy<)lBP(s zoO4b@#uy?>K%Go!EsyJ*q7&0-J5G|2Oc3QsQOgsQG%djmzzzMKgYo#P2OQq7?J-r@i?T8tO$>!2hr`j=;__J?`p9i&dC1tZPA?k3 zNtP;(V<2kD=2rCf_pe-?@90dWGF)&nWQ;LJCu__^L;!IdlZupI(Ln{d07Mjy)!J|8 zZ^*2_kI3kt99=x_pWanJ@lrVUnht$trjyS$*m!eiR}W(>$z~@vTynDB>PJqK1OOrvHxhSj z3hC>)kTFuqX}Hdd5V1*rEp8Bz zNaZ`;{U^CCPn8FsDen1}u3uzs4gi*2uz3YIDY{d-tv$c(wmVj>UQ1-+C|cO`7BWO2 zEtOwjcB=C|pU-Z3Y~t5nsT}+sa*J8+vMNs>>>a!nAy$MSvZ?#;f8>U>19224C3ek7 zULqnBKw9|)E8Eli_c5MY9vzgCEiWyw4=_Whg*eb9{>8(vw!_)7dT50@{pu0N^T;fQ z2}FjB>!8ftT>n4#^R5TJU}7JPo^;5U0lsE*8xh;SPJHe@%(@W75K4Q;)_tQ#>V*=w zti=jUk#W+oiE5qqe!lO?{{qNJ$AEkp;A=#;9jFb@>&wqIRzP&p+ z=ZhJTA_HWqF_GK+_{tA|m02lL;S$G~T{F7x-^`zS7X%Uv0mJ0FsfXT{HK$hLi35k6 z)FKh}8z>9q7g9Iez48yg%Iq|$a4B;&yEb&+w}C&o4g6>_Mw;exkssTSZQVEVdTqSG zZEJB0agYh^mr^U=)&JCAps>NnC5!P}``VmAJz{s>z@NGu{PD{*Uegi-R^-i!T|PdgOdy%l`6-%IHO5 z+lyW?iA<QO`H@L_4?{~b^q7!ve zLBQQy_rw1V0GHbOF}upVfYjKT<&WJ8p}dC7A7`c>I*;Bu*f>)f9udyjGH+Hkz*MJ|rx#^vgs0Vt7Z_XUux6koBb6pU1WVLkVw||;sh)c#2U*UTa z00Mt-BMA03i(xo3_0WCzmIKkb;<;h#I@tS*Oz5zh+x)n@;#L#;OT^n^bE|JCaYGMo zTmeCR&FcGxln`f3?b!Uv!QG(s^{~T*jOKQx{r5gbI=FV)v{E&fZZi8=?uMyS8U^Ywp!Od#?wgM#gniZ@K9~r)z_WgSjj*nz*j~ zk`Bs1h6tpTmSx3pETs&?FbqSjbTw@@-mOuF9N?Mzt{Pdo2U^lU(qH0Ww;o+D%xk zpLmhmsToldBI!7P=O@~3|D(*h`=YTUI;=dlV~6iI5FwY#edt3^S;8unO0Mg=j^ntl z=egNzODdI)qNr)QD`h-zw)Rr7xN_AR$MxFt`Ibz}tFIoE@f!wc7Qos-#>HFu zAXf8}heD($aK zSR>K^Fg!dQ_zfwg@B8(7Ba=>F7#T6fgkgvPM~@vd#;jVk%J==N^f;J5{vpO#7z77@ zyN@wesg#vcmTeab7sp11%f&)VHh0-O#H{E@;f7A^&O)r_Em-8t)C2eaO^1hm*?01E z$Lf_biWbx7B0^*>0}s@WJbxK|GvSQ=>_0Ng12VEYA}sg#>#xUg%ov+;rjmveh&VWS zz_Kg=ICJK#ZQD)zM#h-ob7wErjU!{MQYtdegCHmt3!L-mPa!h*^4#=j)m(wLF##4CqSf7Ieq!UUEK%wykfiVVtUkw zq#`TV?{;rA^75<}6E?RzZh2GJr&?A_jpWbi7Z-zuXr^G{oKM1L_|HmXM+UoEKLT?^ZIGH^_`N1=ms z`3%380Qx+%;XVBRkbcGNrdi&Fv%<463 z-Bil5tnQu_EiJiJI-RrzzOs{L2zp!RBQ&})H^$2DX)C>Z#o+E;ci#U%;QNb}!Xqv1 zwl&<%k%|&C%n*D9|7wGHcPFb!_}HF!sD}5w_ug8)(bLnzId>hGah6JXXU?2$$z*o! zdPNA{WEx_O#c`a?wQ?ca+Vizq)oIDzy7gV7Bg3gwdTey$?3q(nw!xfXcNWY-J~Bg7 z3%z?b9L?0uA3JV)E-lJ-F+?&_WI9Eri}V}u0s&+Mn*oz+qNx? zLeF)(y1G-T)Vg)+Y}+=b*;d9F#?k1=aIsj(cXYZlGlpV@ z`r4eedZ08Aftg0#$uFEgevB1=V#}Vd?|pQ|3a2HrIH?b#nVq(>J<-MEcnMvB@Zv=? z98##ECV>I`PIub=u zsaWKkE2XB}KJf;MnkZ%K&MbE=v;;!{ATR(#UzzTuKew; zo$q<{vBmO28Mj1BHyMqHL_h>KgRh;^Ns2bj!QnVjC`_T4bg2h`N~H<_i_$Dyi` zOfb$7G0DadMG2_M=D)dYFaaTOrMG6$cYuVVlPA>#1dyaME(>;r z5U_Yn7!gria;B!bBqZNrHlv6)QdkBz*yhjek*ZJCDpe(v6_hoW4J!jHBd;=16IR`} z0>=(rJMbKxvUTb$2|*I0T@nHyLjZt*6gzI+^UVXduUgAQW>Mygd##rf;1mzde4gn? zi%d=QVqtt)(_R2V#JQ+-dbe2uPhfA}=_cd|9036k2nfiKHY9Z%t0+*$msIzuR-uZT z6oQgrWndNLRW54Ws@rzpT9NAno~=@jnG^yGU7NXrJA&JsIh+An{rUP6Hw}J!r?~O= z7sr{lQh#&MjGbHa_bZA&SMfH5@2OxN!w1Dre$V~jCQWD;_oZ8gMn zb`pq6Q>Y1NH8nv*aUqAm5IF*yO9&zrg~~J})f-eRQ$<2qV@X5VurlyTArMw0 zVFJfe`P?bh=x@{$8Emf8Lde7$tcD0uvxhcfZyUtQ{PZ;{`;Wf=(Q2jQx!$3}hdJk* zb3_QkNJ^Pbr$MeAIr8eEO`C5$d-|kOdhPlFDW%c6P`Fqwm8LIhiP?sd zUCC7)f0Z)X00Yl_4LkNr^ZH!b=tb;&OLsH$RpvGSBioa?a##{$gABNcG zKD?zL{Mh{ZI2BBPUtezH#xrNnZom63$8n-K7MydzjW&$)AO7&!*?!GJqbsE}#`N~} zuUmg(p>VM{F(HIiDVJ8Qz9E}yeeFuig$SVDLRJ+o$vRwJ2+Wm*Krn9ctOulDWUhQn(Q+e z0-N&|Q&+K%j5V`o9T91N;xd;5Eslugh>FK$7(JbWRJ<4UEnb=wvx)%f`M*!cMD0;r~`FBhUxE(gBvdEWTgSR6&wY6ThQ zJG#cl#{%D<3f^XDdByX*QmK^7<*L={;NSt*b)zWM+AL~C3TmzP?b#)zbX@oN>qmpYFBT@Q zyj%!C05*f+fKE4Pa}Iq4P2*|<&}st@3Wl1LszgMi;^4v{GBMpFWsDs@xZfB9fR<$? z$As1EbsGbRi}#+ct|z@-z9W~bF=$RJ0kE7)5yHfy2=BWOce zi&8M+Ctz6znL(xZfers5M(Uw#3gNX6Z(1(87AD|+&;w1+?`2T|?FfMl!1VS-}pI8C_LzxZt5M5fe^aJmsI+&Ft%-M@*9eCIW}Uk4AM>0&f;g6M)TGmKri@AhHF< z*DxzpJNgqf7eD{~2iO339A2%^Zwe+O7Ont(R$#|?D+Vg6EwlzB6DlZH4*!6OMXK0c zT!5!M+o#WJX`1J1F~l~EbWmB)_9$dDcQW;3Ka0kXGTU1UJ+s){KCOV43+6Wk(_%GC ziIKGlrGqjoqy{77I<6P@ejB;HL=L9Dn!0@Ll#-@N=?(&D7n*_+09&A$C!@HVtsMGc zc;O(kQ%mFk9(KjRRKU&xJ$pgt7G8;yY-Otj7!+uM0%0QLj;xIpcK#(=OTzlTi3S@2 zm%~?1=+OY0kl4~htrqgIxKQd#nmd`o%l{)To@T;bIvp$7RXEdSaJoVNbx5Vf!p6ji zpb1(mW<@2v0NF}1n!CCB$zPTBeuulcC3SKZyQ)?}s}0{hub;hO+U&`K*@eq>WGv>! zC9Nd_n_uNcZt1Wx_LKhrKuhgBCH7{M12%&{J0efiDa{wk7-NzpwRlQZl#Fp+{W>DU z?bO&$zZ8xgWp;Y09d7t$00TG-CSv%)AsHL67R(qMB8T%9T~l!|wUugKJZF*47j}KU zJovXF)ka3a5*u$$hMFKNU~iHB&8yO7lc+C5m}GIDrG|_fz~*P6gR~Nhqr=8y>zyP2K+HCmVu=&zam9r+di_W)~OPsMb3{_B_7Y!z(M7phU`sE8h{FlfC zAX}P;NWWEgMiaE#@QqXY%g0o!y&!swjJ2_NUOKXqQ6kmmpWAo-@BV~T4BT3>=L%ng zznVDJe=(%LbVOwYFqr4qi2&P#GQni&-8;0q^;0jOd*+iSZlG|N@~P3+7LXW%HXHu( zr2fJ|#SvUSpRtnxXcLC~(%V1umJE;)>5j^wADsX0r;H3xxIjz#yy$D5%7m-`?!5l^ zJ~4`dfckJMx@JwN18$SsrtA zo4po2b{G)CS5N9+O_+avqj+Bz(*zAQ*(TuHKakRbxw5RawBB^1FcFbPZi`fFR5&^D z>#tM}eh<0Dz1Ag_YFNoN}Ya4x(Jfe<`Lz6|Tw(h7=_7RZPI zOfbtuZkgEk&+o6k_FV1QPi1A8S)NF>kul5YE-b)PRv7?VEieSXnlLY2G^0m}cN&rV-2@xYG18p{2 zVZ}PCkx_B^ysC~w6GP$1;c)b|xO~RM0djj;Yp*Rl>F8O#fS(SK9HS!*Ld zwNw!(GuiFa09@K&pZHnmP)r2K1+d9|d_-nBuDxC;dTtMSW}8e4!4{LiUnU7eV9>$I q_n2i^e>VWWI^MGbre$E}Yy5u#Ay2Qk&t8rI0000 Date: Fri, 16 Oct 2026 15:39:41 +0000 Subject: [PATCH 39/41] History: a ticker goes back to the top level; pie legend 2pt bigger, the pie a little smaller to make room Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- lib/buttons.py | 2 +- lib/render.py | 10 +++++----- lib/worker.py | 2 +- tests/golden/pie.png | Bin 7872 -> 8305 bytes tests/test_reports.py | 14 ++++++++------ 5 files changed, 15 insertions(+), 13 deletions(-) diff --git a/lib/buttons.py b/lib/buttons.py index c4c7506f..3a3d6037 100644 --- a/lib/buttons.py +++ b/lib/buttons.py @@ -44,7 +44,7 @@ class PickPress(typing.NamedTuple): (shown: 0 nothing yet, an item's number plus one, or the ticked tickers packed), the items ticked (a bit of the mask each), what the press asks for (act: a letter and a number: p the first screen and m the tickers' screen, each at a page; f a tick refused, as too many, the tickers' screen at a page; r an item's number, to run it and show the - tickers' screen (the first for portfolios); g or h the ticked ones, to run them and show the first screen) and the + first screen; g or h the ticked ones, to run them and show the first screen) and the checksum of the list of items it was drawn for, stale once that changes.""" cmd: str group: str diff --git a/lib/render.py b/lib/render.py index 12656f38..0430c76f 100644 --- a/lib/render.py +++ b/lib/render.py @@ -499,7 +499,7 @@ def draw_pie(chart): colors.append(slots[n % len(slots)]) n += 1 fig, _ = _figure(bg) - ax = fig.add_axes([0.02, 0.02, 0.525, 0.84], facecolor=bg) # square: 0.84 * 4in = 0.525 * 6.4in + ax = fig.add_axes([0.02, 0.09, 0.475, 0.76], facecolor=bg) # square: 0.76 * 4in = 0.475 * 6.4in ax.pie([v for _, v in rows], colors=colors, startangle=90, counterclock=False, wedgeprops={'edgecolor': bg, 'linewidth': 2}) # 2px surface gap between slices ax.set_xlim(-1.03, 1.03) # matplotlib's default 1.25 leaves a wide margin around the pie @@ -509,13 +509,13 @@ def draw_pie(chart): y = 0.5 + step * (len(rows) - 1) / 2 - 0.03 # legend block centred on the pie names = [] for (label, value), color in zip(rows, colors): - fig.add_artist(Rectangle((0.57, y - 0.019), 0.024, 0.038, transform=fig.transFigure, facecolor=color, + fig.add_artist(Rectangle((0.52, y - 0.021), 0.026, 0.042, transform=fig.transFigure, facecolor=color, edgecolor='none')) - names.append(fig.text(0.61, y, label if len(label) <= 28 else label[:27] + '…', color=ink, fontsize=11.5, + names.append(fig.text(0.56, y, label if len(label) <= 28 else label[:27] + '…', color=ink, fontsize=13.5, ha='left', va='center')) - fig.text(0.97, y, f"{value / total * 100:.1f}%", color=ink2, fontsize=11.5, ha='right', va='center') + fig.text(0.985, y, f"{value / total * 100:.1f}%", color=ink2, fontsize=13.5, ha='right', va='center') y -= step - _shrink(names, 0.26 * fig.bbox.width) # no name runs into its share + _shrink(names, 0.32 * fig.bbox.width) # no name runs into its share _heading(fig, chart.title, chart.subtitle) return _png(fig, bg) diff --git a/lib/worker.py b/lib/worker.py index e030791d..b7ca03fb 100755 --- a/lib/worker.py +++ b/lib/worker.py @@ -161,7 +161,7 @@ def process_pick_callback(press, pressed): if stale: shown, mask = pick.default(cmd, found) per_page = pick.layout(service, cmd)[0] - view = 'm' if kind in 'mf' or kind == 'r' and noun_is_ticker(cmd) else 'p' # Compare goes back to the top + view = 'm' if kind in 'mf' else 'p' # picking a ticker, or Compare, goes back to the top page = number if kind in 'pmfg' else number // per_page if kind == 'r' else 0 note = 'The list has changed: here it is now' if stale else buttons.TOO_MANY if kind == 'f' else None if kind == 'r' and number < len(found): diff --git a/tests/golden/pie.png b/tests/golden/pie.png index 72fc81cf8b6f6473b4cfbeec30b4f35fff578ed4..7d03bb9217cfcf5387214153a36dbaf29e005d04 100644 GIT binary patch literal 8305 zcmV-%AdcUOP)003kN0ssI2j?}E!001C3NkleV~b>=zUeeZo6-hzDx2@>3QSriwow)Ty-6DP5p#7S(&aWa$2B$b+RRc1U>v&1#2 zn#xpalA0}hyd^cZV^1VykwjS%CGHeSi6Q}#AOK>2c-!5(`^=9EfFQVtl<+_$^G@9& z-@AC^YuvBT>2tn5=QK99_P_!`L?U90X{`|v0Ej5MLqySAM2tR$h!EWch(6C4oBzzh ze>ng7=uY&R=o{znLrtn6Jdw&Sd@rSB3@f%1v#cNph{!MuA_~J$2mwHxbFHa1_m$RybETA0+BA(Y3=vT&#TW~Mkcb@Dm7!#eDW#;8LI^1(AV%}4Ce;EM z5v7!0``W+$^L$6tT_Kq{4r$K$^5cXYHN;+MYg z3z<+{5)bXRc_w0S_(TBHh-`3sJbJvdTJMY;kgz@(A!=L)GAB)H1zx0(aKlS9} zw&Oha+)wS;FMs9B6BFZK{NfjsRD1U9dF-)AHf`M0+}!-h zefv(GI#n!{1m`tKmR8u+3ovlbFAiR^EbHFAdnYC)v)SzG)vKqbr(MU{xM2h7Ael^# zjg4nAnc?B#&d#m}AGrV3SAWEeczu0+dwbjH=;%u?zw+^i9_(muzi{zlJRY|!>+#1P z*}Q4f;H68B>ule?{rK?{r%s>BWYT~0{0m?E+Q0qful$l!vcG>IW?8kBxaw+Ko&JFVs|Iit1Q&YIh*&HXlBuNYdX{BXDizZ-4a2Z) zn=zJ3r8wuF=X##6wQg-~edd{`UwGlgQmGVuCR#xe5dgJTp6A74aUpo6VjG6AEX((O z&N(6r0^c-Et@YZqYqo9M_Py_YpVy3YwE;%NAPCw!I>R8a?TTTTVHoa_Qc0y!jB~BjwFoVYlmJNUE3wvmToeR>=Xt4As)j_BM9ja^ zTAQZ1W$TXf1Lqva>07%Fk-4su%Qe_`d2(XBwXGxYy+ktQx=s)TYu9g*VW71>-+wNj zpPmQrT2V?ZdND7CVOW-R?fsfmWesMGnWpJDj^k8Psk(`Y@kAmKDj5brCYyC#mxvt4 zDHRK@>m-w@nS34@3&WsPDt=%aTU&RRZw)3Q8HQYlc_vG1Erf_@ib%^j=R~BnCZY(` zSxT!WA8`{MEsdsSMTBI8DCc6HT#QsaQaHvKAi~^&T6>QFH~_w$!K9nm*4pFyM@Dqi zG37p+e)U{RMBMNY&tXgXshFWXJnQ6xA>Jo`WGM-3URxB1zxRv~nov7=3NZNVr zm!v%<%j02rLOFTmxrm4&hQdO`8qyyjfDsvyjy96x@kDb=TXS=3LsN4mosB2rLYRy* zATX`?*rj(T$LpIv{_Dyv5;7pALr=MdusH0GpK(Txd6(Y~W`;-w%uFz05fPBC0d~g# zM#hwsGW1Q$>gww0UeVjy){#o3x!|NpX{D8xG6V@kT4@S`fTS<&qH<<{aYO`UB2jPD zwIq9X<>+UXJ?)O3s+@bNGO$0K83ZPnnFN3u%*zchGNzRZTsKu$*Sm2`@2WMq`Ub|h zl%bTNlmWo(Cd8bd#Ta9V2#ClyG7fB(1tS%b3TF9gBc57yUtQlrVR@|5|4QNbU%JD` z0Fhg%n$ccXfPK$RBvYHW?C4v&K2?`iQifp&5H4`CZo|Z1akoinzd}UJOy#!!WM<>D zmHz$H2miNo=@1h!X2eL>80@M7%n&qbqIK&xZ`pcRI+F>*&~sf}vg|5^=zk(5odPme zxBjtI--E@I-=BK@_k;XJ9?#B2RyDbyEZ*?H#U~F4;44M?{s$i3{;^#QAqax2?eI;% zl?%HxJ>tac`W~Xt17z>#l`{a;p$nuF-5*P@e_V%7WU(1HY8WpmlZ6PHzz+B)*0G=8 zGdeOX!%%89&sSG5s%9JzrIY8ebo|4)lKQeW;JTc8pWBGP=5s`7}+V6$N`Sa{d|!L3LHS*`RU)w?f7gB;|~FFq`SGe_>~8-Jqr~dIE(IU zCx~zAzO>3Xcl@}Oh%cL=o*NhkEp_O&J@ub*JD!s@Tm3-*Mubp7(iFe=0JfyT31-oq z!(LZ>>&dN03j@WmNnx7Hoq~f5h;(4LJ^3FpoA%W(zI}j+06=_x4|eAkqZ^6V+%$qG z*B?5100Ck7^E(iMw3dF`(_c^a>{4E_W~*-*V20oZ{L?#dQ~MRme9dRHXp7r>aj0l0k<~0A*4hZD?8(vLZi16tRhfcf&I%1WV zA7Y{0nYuL(=XUE2(CwQSUZeKSLi2*_ryx%tZ&TSUptZhU=YWWGKK=f*i#caQTt@xlSKn=HM!qnmB- z1}6Y^Yri39y5hwi-F(6y$`6egccz#ONl6yVG(GTZH{Wg|8BplXUL>QNwr!V6r6~P7 zg4S`IAPA$^3niKd(fwBU;sdE=-3@iWYfUuQRh$jG!C z=~j`b7pW%DilncI?%J{AuDf=OkB{f``7jIt@Sc0_7D9}Vk0W9jhEZ~vQYtE>vu!(S ze-H$r5WL#DPDGtuJ>^oVr*~CHXZOrZUP_tCW_wrlNhxFTL~~0^rBZHeX|XGnXfWai zzE!{;=)D(A-?^u&-7wbMI<{@}^r6FB@7?R#_Oi|HMGvE4*B9-9AJMfLAjov+ zv^?@#sa5wYx4;wiWB>Q7qi_7t#_hYFdG_gVee2s&%4eQ=nuuodGoSn1XYGn};LSIW z9zB*xr(Ms}T4l4@wzjs{UVHr$AAe|aYRWXtbUOX&t3OgoRS9E6=6UYQ-qm~Wd+=|c z|G|ottF+QV;GaBxboadvq*8Sk&i8NGc9&&Yx%!5Q@v$%nV(~Z;T~C8GfGPM--Hmbc z_SAQ7evmJXHLatEPq$I3t|6!7;$OR}8azACRAAMw^u|v4QwI+o+`MVCFvNTBoy}x2@4a`nwYBAi=U+s| z;_>*{*x1z6bTXM39UT)w3=a=~=R1Fu&1TBw%1vr$KcomDO2y*Qi98#X_M}EKE*J7=}SwUsYWQh!9BZ$ckTkcu7`4h#BGk zeD$%myBd4CLf@Znyk(fNiQ%&cCv$C2{JSWtvMQ^vRw9v`dj0pu|NiSDk-N&NCKBmZ zX2gLsAd{6z?UxCd<2neCOeUjR*;p*5lm;Lvr4UAx!EYEw5CqW+05A;0^E{e;Ygb9_~bg!;&_z0fe z^46aoe!QgxIa}u38bqe0pIY_6)a!q+IAlT$3&Zm|ja&e71gJ};B4Ze#rL{H;0RRha zZv^~&lY;rT%MkZ#t)qHR(==yQ#V`Os2s82}g@}H7#SdVMbQeL0QT5O`RD+fy()gV$J&DPw^y=})2zG*Hue~Bc0vvHM3)Pt7u zoHGG*#Bps3`%<{J4l@QyBF5Og_wK##zP*NFc%E_uRA7bKULR zw>gg8+1a&w_b$e8a&mH+dwW^2SQrFr*Kd?k&g7@>-nDmfVq$V~{9`+J$6~RT*0#EI zrlGMZlg&D|UC8Gn|BEej(I93(HV&bhb+~9rA9Nu3zO9GOp9snh<9zwHYe_L9Ze+Va zy-%}Kr8P3-LMWw@iNxkj8yj|^2R<^0s-<&h!Am~B^*sXEVdFP!f$77KNC>H3Do=GL~TxhuN@fFexCL`#~- z1dkcq6kIUIkO3eO5`n(@BD#%9fIy)#8Z(9ayN(??V3^BisEGg>H?ke{er1WAGbPpN z<;%Ujy?giW2|}64WG;_fzIbsko=6l+rFbIY*iL_c|6`9n8U>)sXQH(>E$hdA;&YWs z*|sYiH*cMqnq-W1_pD53via%BOg1+)G4A{R4Vf%Nkc#ZL1|eZW+{C!SVg|PaGdVXn zA(qpZAOOzAiDQQ~ z>2x}CY4Cz$+YwiF^zd7~y{r4ry=Pk1)a3Z}nJk9jD%hBUpJ}Dh5KEdBfEAFJm?#UQ zVj8YxdNDJITPk6yghdGx6IV|H!4^$|sFC5^BmmL`nvk==cGG`+;$z*tN8GlSNB)g= z3sB7*Ins*9v&EC&z5KmjXR!>Bik?@Cg$WKCfFVr!RF=R19LFJ|L?RJ}p_DS>6QbVb zQ780huP|!ix$Gey1%Z!%hG}Z8qh%r*nD6_BVQ|iG$YdcvprI?petNxC35fxaCZ#D< zK_GosYA2+ML**hBB@`qS6wYW?09Hg^;-V~!ieb7j!;e{R+>{AR&rSjZVtv&vy2gI6BElZ>XK$@8eU~E273js#lD<)eYa0CEI%`sV)WyNB#dEOxM zr#R<^Va)g7T=pOsW3fcUZA6S!M3hF&6c*lJ$YhCy)|3GheO(M^e_KRE&KQTf7-SF_ zB1ec66oG)QOafo}p44_o6`RThDuz%{SWqydSpoR`B(Nh(U>UL@b3xjjb|u=_s`6SE zHF+@>eUHhm1Pb%M#C&wI2*Mk^UiM%~FB-$#6ucpmC9a{KX0L_6jid$t_?l^jobhN9 zFkpxru9^f|lhQhrexO`8Bs-*vLvER!Z4Um2wzXDwiXkEdx&rwXCTspQl7&y|BXVrW z{BF&{Aw>k%!b4ORj`k{mLxSt-6VBVZ8`Gnb0Q8L~fneO=={RH&5HZgA>Vp60(}5)~uVH7)vBmQp%xA7ZGuI=+edW1I5BjU0vPvm@EK}FcVTCAdbs1R>4>noY1Qv zR+qMnD$~Nk2;P+OCQY$eY`FPgp31JGgd;bLnkZ@0T(y&gd_Exo}PaBmHlVVoayT7vMf_6 z#ksI;`@n&>W*tc>8Df9`z+;a+x?<&u@Bj5*M`mz&)K?frsaTX!27!O{@LNNJ7yZDm ztExvi60DJ}gHa*z9EE@rc^*mH=p}J*&M97V> zG7%O>7&m9#W+E`S`l};g3D5)#spPSQ<-6Wcd8nbGp-?C+JZZ69E-#Ey5b^Zs(+3Y8 znwXd}m$7jLB3?Y-ue7!-E0wA%mrIB^H8DX%!QjplO;)DIvr{)wyO{kWQR< zn=!Wh2R)gV6@|R>XDbN900482T)P>Qr3p+1@06$zf>C8Z3js7?P7rwcsxAgZgp9i* z$D@>;tKcx$f~bjZ!~|WJx=JJxcZ986_@xVnN;e)qTSy033@$nJURlRD=qe9h17uC$ z?zE;VQkBS<4qfNc!D~}?a5i4M>e~^K^D8ArH|py)*ZBHQQd$cKd8^fh(Vo&9)7oHF z)?fssDQz&r%WJKxg)k9{m_L2qo2_2CikDDIN*PKir3~jeKi~HYg+i3s1OQRB+3fiw zbBS$trqpUNrRDp+HyZ)ow7QTj;`A&I`zLidglY#%njmFxE0|GQRjIEBk!HfO2lgv> zan}waN~O}d`o?T7m#c3yO%oA(-|y}1{mGyBOlM~Y04Sw8+S_BXSh-v#qKa+%e&G9l zl$Cb}byq3bw`SeC4V$%6TI)&J@godcT04%p!YVgii9dv6(N8k zN(degi&6%>N)>sGE5A}Y^CA;hl?+8SwfF44ucNcu_q|vw{@SZAoj!HEyQ_QOzGq&1 z;l;-vf86(d%Q7b?CVN)&yt@BKXV0E}>d7bi2L?RP-?(A@EBp7`wzK@IjEF(tcXsu( zcXWE5+uOJ1?PEuF?Y;jd)rE_9lNdlq@ZAx0Un8UHYB0qNo>HZ$z>CE!(rSrAVN%RW zR|a13Mo(SkKVN?;py&B3dsmK*j=lN%%NsUqSg~ToYp=bY%Vqn|_4lso-Me>>VerPL zrkG_NJAS-aEL(S^4+#Kc+_r7Ub()%5q*O}l{iqPS z5MuEhR0JsjFHh<-Wr}mC9-B@8CTD4~L*GVR`ZYgt-3rIP4fOsylB6jfkH2^3y_S~d zC!YRvHkUng_;6!m1LvZDV8C_VQn~DT{-J}1rt|rW7cbHs>N+0ghyMMtEy$CH9SeDuBTJ!MyHoGrA3-qsiD98rOJE%z~kv_ zn{^OTGMN&FL8K9x>o`H+GseO&^gJ&fk8{o)#}UF1LTIfx=dSAtA)-QuJ6^^T1Oa1A z2qC4so$A6l<0%k|A^gFB{JrglQ#D4NkmZC_vq^Wfbl@r;&y>&nZRPAsELL~jRxF5E zsg$C#xd4bU9ubp4RAX-Xmfwvj(6$O-mODlU6*Io_y{1z46mGcb;j1z&~AKH=z;}94^@O+h@W= zb(l*9K^`8u?dUH0naw*wL zf;xj4{)}1@wvrSx5@F%e*z^AekOS0CHGV&UqhO{^z}L^n{gXOpEHTEKC}pr%Fr!q6 zAM&7E(!dPT^73E*W>~t+j9Be}v|9q0q9kX8-#8JTDp6+XFvchpHh4l7Wf**@R!c;H zj0tP>hhMcXzR6>m+L>9m3UH*v7K5UMUpX3{uTVxT8OEB(;=E3kW#A%y(9}E9fXK|` z=->QS@znQ3qQ18D-X6eF)Rf>b>d_aDgl8+1T_Qau0E6>1*!m8 zt20NxSwr^&0vutS6fow}=MM#M=5?b9>O&WZ6A@=Dhn{pxA3%diOEeRp)$srM=QGE? zwYU>jO>PU|Sv?j|3gH(Ihu^rQawc&6up9_7mSe$fpRzziT8Tt`SQxtWjh`#O^EV<< zUqko90vus%aR7j?o|3V&OI3A%BXbP^IRh#Z{>wS_onifS8+*2$ zudPFY;AsdnL|L)3o6n0hIASYJZWd>hR)|r4G7%9VATyX5LngxVSm~V~7LNawJA4!n zMLbKSYv?Wq!1JD$K&UsN5Ym4?uf8>;?yhGKH?y7ftTT=l2SpHS2sLQ9YSV%M0!=Z_ zte~J|U?2l#AUBX3L|WO?l}m>z=U%D|>?~xEGNJbR1wTM6BNPA6Z#(} zbb|>S(rjB6H)q(21U6Zi;9#(5g~s`HIG5x(Diu&zNP#mQ7DxQ?GtS5{@ABKh%n+%7 znF$`xMvZ!EfZcW=dTq+i61e6F2oP%UG)N5wgS3FA7&cqjY(dt*Is;5Zh7b@0QawK% zD>Q6n&QwqyS597sE+V3cp|B8t*;*uCaWej vfNN3%xF$7#Yf=NaCN+R-0M{PDKW+JcJU0&ZJEyUd00000NkvXXu0mjf?@SQT literal 7872 zcmV;x9zWrUP)003kN0ssI2j?}E!0016`NkleTEb>=yDc}t_w-2g~{Ai1|XR2ng zGhDN#2wE;>dZJ9yJ&|mClWRt(0O6D^<_2?I?dWaU2UF0ElyLj8RG< zqA_$HTIRYA0EA(vjS-w{t+h6mCF3|oM6ESrEQ(?x@_b*#iZQ0OR!RvWlv040%%|y5 zD}WJEDfQ(q|F=gsJu-j6{1;w$v9q(YQmL$5dHrWU`{`}lwkxGdC&Lrf;kG$b_ySZni{Pk*YZDLYgsa?Zc_$6r{tZe19J zzjNoE>(}2F$ML@1yKY;5`vdpiyLRoGdGqJrbmQ7}x2zLF?%loPLm&RI>$;zN;9WTQY*cF{rU$UxNqh4D_dJz zAKkop-@bjtQb}+=4awvVHr4oYLy((PNHnPg~-e+u$gU zzw@2%=CawJKl@zk%o*FZZU6btpUq~oy?wnu{Na<@U=RkWhSbqx$N%B`Ke+ead%C(# zJoVJSX0zEK2#y{-cKrD9GpEn&-m~Yex881@Ipc*FU)s8LYfDS>maVU!=sKab-nsM5 z-Fx;dTDWNEt~a~7y88NhAANN5kDmO|(PPK#X@F}maI_bVh{ZyoA>H8nfohdlUPth1VCeS5Co3n3c)Lts+7XE z?Jx{E=ZGwdLd&v@F-wqG9LHf8w6x5K zqR8_q04Sx^(9lpWm(q=mLxY2rN+q36GtQ0H7b7$}QUV}j-iftF;-V;uf*?qz)6+=Q zXo!&`V~k~4H>|$#_^~6N=XEY!hRA&1Yieq#R?Gc;y|dcqL}8F>Nc+AQMbXmbD^(mB zV~!s^GB}tY0q^3WwZ7nBWJoD(+rIex^r(pqW{g>u<$0dxRnqCkzP{d6Div!LM^QGH z^L?L)JkKi?3%>6)G^B?H2a&Njj!LECyVkMOcDH^un21yyb0J2UEMtriBB3b~Dd(IM zkuipd5>SUJt?BVTSJ8=Sv}`*eBojnA7bE0iBI1d_F~$H9&KuO}=lGie@TC+cjiXzq zAKz~tz;!bXqr@JQ$%4qvPa6UtpmC?y^mxAkoW#e;>Xb1$G2_Wv6*49m>DWhPfC$Ku zabyCRIKQH0rWMBf6X0P=O)V{%OxAT9EX(4Y z1E6KQy{8Xu>202~>CaSk2oOofCi1j57#Gim14o1IeL>IusBlUL9&(FG7a5zD=_vy^ zVX2}h0;0xDwtaR-$K3fXGiIhzX)ZV!GR7FAlQm`{B7i80NQKHPYQKnF03r%o<~ppF zh3O@C5gF|l!u})PiPtN~UJ82m>%e1H0~6LX%yk7gfi?_7Aw);VyhTfv&YsheN~MWN zYo%1IR0J@>SC9N>hzN+tI5G~%002mn(xgHnLImV8)p29P{99UX`?TNvR%!p!OF#s)Cuz2O_8)vr8B4gq>9);_qOd>YyuSHcN z5~-FMcl=Ry^<#zIPY!PXp04yWCkp`6F4&|3oFv^T-PGK&_QrMd<}V^LQ5e>i-a>{5 zq+{jfnU$LP!7pT1JTma=H;Q|If?P7`OpEg9!QR715h6thB9prN?gthw>Wso5NwKR3 zd5MTj0BPmrWoFjwO@H3;*k6jqS=uikn_gOe?*MZKehiI{^~Bxkqif#UwL|GC$&5r~ zf>fyeLgTUr7X0BqHm=y9ydgk@X@>mW1I!T00d}^E|8PGnnqjY=f3#hlc>93ux@4xp z1R_JmwO`;)w&VByqV=9HnaBg9haK{1fUg?eYRJ|v7N5ThGY1hKVo&CZ?!mHZHw za;gGTWSn$l!t%^JzcA;~{{_fMM}T}9;HyNp60&<&@K4+ZL=b7rz%V&)DuKItLDxJyw&yK7 zHARa09h8Og@~MTl&HepvGpm7Aa4qXagYh^4W;JZ)bZFiQCMK&YZl{)-8N@X3E3?R`C~VO7oDe>#thO%xK{Ac((OC9 z^7nyOL?%>qsA0kS*&Dx3S|uya(*R#acD)EYTlvS=L8Jia(hrKE@gjHSyi<$ik=?te zTu}o6a-qC@X2piqd%mQDA~H4&@Wl@aAU4ow^FO#1BnNHAi!T|PN@zd4dRO6Cv8P{H z)|6*VA`_}Q*nI1!a%(=Kydh*_8sLi%R+@kGW^Btq5D$A*;|ycCC8LLzZ{PJgUP_Zs z0mg_XsXfY^E=2suitT&e00Zv_S{RY*u+p;r39EU5QNhG7DJj_!oHsT$#&KM&R--6N z%56N)S4zcktd#P7KRNI`KZ>F-3@07qqz)lLR0uJGtLvtc+5@E&yU3Beuzd6#0UxtBQq+P^}ttJ@A{MM4UcA*KU6vLd{8aTnbYx!Py9i#PzZwX`s-J4&j0g& z`qHxHOY`|W0A{mUA^80H^QDwh%EgNp+m_YV)|SubC$VG_5yK$pm^c5rT_B-zi zf}pRb`=0wAYDhO2L(5iN&lp35N~I#iJHF&)yh;K<;P);E!61590VHRp5;*s-+VicK z+fUEUwzsLvV_ZM>BPeB8ZYu2h$FT1pO7{}k0Yox8$DVneQJx7(fGm-WzW)A9W8?kz z-xtU6s#U8F9XbSv&CSglHazUQ?yg;Lwlp`-oj3Q$k)y8bE?T&NbMe=I^EbV{eMyIt z_kQt4>xRb0&ZWy$9A~mQ0CZC+ZFEof*-R$e(9pmTmvk=e>+SKX)nw83@+z7k1Tij~ zh0EK)kFHvLM?EagnA*7VjlJ7I>-U3WLPm2d-TZ-1ke=vlnO z;d!2va?YH&#llb~n;py#oIZ83ucv!vYnx?RbiP9pTu^*AN<@B97ysf{f{h>eA-VoG$xgLSkiSKoSIwN|kdSMNuY` zNdQhPbSBf-+uO$`m0B^zC>7hbLqtu@Eyco+l+tlrZFI3va2z*^LN}H2J#S3e$b|qS zLZom`PW=wgipGk>0I+R4IU|IS zQjGL|lTu14lH;Vzx~9>Qlrs5g&IK1@q=NTiopBMs+CbXDRdXOxlfy$HQV`kmTKjKq zd29PN+i}g*QXhy$g;w*z)Vy0v96ZfDM5yaq&!u z^(3zoAX1?0IW75H+xBjM!*ZOdRMd#1LYeJw+LoL6%B&U877B&VB};NmP4ni>&E;}LuxQaD&Uv|9u3fh;A`(I_zkXH6yaj>p zuejnD&J_q4aqs%bMQ##j`E2lnsNN}WA@s;{SKFh2kYw&MyRq_jBai#wOL z&+bU1{PHS#b`EM6$skgstXWM%ceL-`_PRBdy$c!X*lJq9oh%qNj=Dlbt#lZL)vC8- z$r2(;HKYz4IIwTu+d$CYKVaK-_qp!Hix=x!vUSE7DXj+|-c+rWvrSD>io(#~`~{2R zI5Nic_4b@OeX_5&=VB&HT;T3vTV|5N-^(g$Qx4qQmmNIx%A7+-+OJzodP;KTkxVSo zGex>J%6Fj@!ySVF&INS1LZOC=22EsHR(E&zGtd0IT&@fb4!-{S8^vNV`Hrrx6Hydp zv)L_Mwib#-+qP@xSReoZz53GghKyEv@9tem-^roD!Bj(PXmHTB?cSd5lP8Y*p2s*J zksL8bd@as$E+`E|V8(nmOk#QlB1DRBUcLQqcRbkMZa1cF{Z!24*?kGiE8nEFZnNo zh@uD)y^F|T_6~}M$hGv$3^)?g4o}-WZSj=F9my@h1!D{u01_b)n6ZD9{7sk)j%Os*hc2cs?~9v#AmiFYt-6)(n^dcC+l#{Qc5A^NJ2q!CCP!@u-UT6WrQLtp9;EEZwN=W}u1zTmQx>|CIg;CgS)5DGIAaW1BGMz!8UjN^=l@>} z_h?O!VoBe&0>|=QTc<3Yn$RRjLbmgh0LTykpdX8kH*Ejz zo}1?_Vj?{y7;`uG+s$N&(I(gxIJ?DeBjgAi0Ra#Q2*{8&6zeEbp|6Y=Q^lilk&0?~ z5)2tu09Hg^;-buD#j<=yhK}vKmP*-XcoIl-EanJq3vO{{a|S3oS}L2D?*99&;<^t_ zjWcbfCK&OL5DSTN9_iLOdUKgNP|z1xlabDnm)pEXj1B<>qS6#-!i&@vA|hvuL!$#( z1ct~F*rg^xsKP**YD|?Xl?zmip`dX{L&2~D@Zm`yWHr$Qwp1-QQRuHl7)dz%3 zOeC}!agQCx@qNEqt)@~b+qO%kQgX6VsYofU8mEFMjbY(@S6E|=W!YM*D2k-Cl!~R4 z7kKjGVnfL;=PwfxfEu-^a>+@+lS#mUA#xZq35+3aOsv94`+iK-m?|FmWp=nV`nIVW zGu2riiF^O?b;D$B5bN)}78R=<^XG2fxba_~e)^FO8(w_rrE1m7<+6Q!eQm96#Zt+2 z-NT0u-*Lz7XU?4M?(P;s)Cgm()vdSRdG74#{cr7kV8f<^`}g&nJG-QFX=~f8p6+wm zT&}OTr&1~B2L?vZGckt9nt+bM^N#|?p9JRelRz*odB%kt0wTJcFDZookdF*e-Qh*j z@t9jCz~%;=orOq2H3o)6gaFOWO`A4tjH0NerDg5fwSx0=Xq<_t_j0% z(V~Ul|NajqD+%k8e9NhmT?nwab7^zS499VlR!0sWv}{``C4^jjshq4a00$ZrO@flV z@+2To=MN(Wgxm>-l!`6HIy+h>_O9SX6Cq>!_Psrj&!^LAOUlB~(A)d>uUWlX2)_To zfpWRLYSl_0s-5hPU_5(H2WR~Fu_Iv^pFMl-bD#apYg@L|&cqOrw5%hC4lu@hd%Amj zx`hyyW$)RwLt2*Wrh0q2l{$UuWY=YxEQU}~R8^GXSkFF0hQQ&x(NqQ^5A7UiRX0zI zh_pBGj>|y=lItJu9D_JxDpmlHPNzN38@&l5j4FEg%m4dVrBdkvqJPpyeqaCqY}+OR zt+nS>lis~th~#1}%VZG%M<^>8j6kruw%H&e!B`HwIH(}m=hQs>bCWWb(cLC?BO2Ybp#E{K^%BZ#Kq@HgK#;Q;_$F*vw&)xDB-Yt z-v@vo2$Qf}HpqEV55Za1=b0<+{9t-s^0*0y%7bR35-KL3j#@KdSO=1m)=C666D-qzOE-`}6j zWPbUpUpF>3W-^)n{{B^~RvtQZxDI)DNiFO06)W54bR63M_Q{i7_doPVVQ8@X+_`xR z7IdFGTOEhV0sxLsiebl)xv_~?>&OpkLmA1WKWL1p2f{=s?6`O)=sCco17j1`)9J=s zQ;UjK6#1Mp&+~4(?bhz@p66e9;ZMKv<$e413C`PRw{Lj(p(RU}D5ZLOdOY9Xy=Si! zb!q@Y0Gx}3OFEApIn>+T-I&Qtw7M_}mR=h$k*+K0ZwS(o+tH9Vx(=Tg(d>Wv5y`(I(J^j;~)RnOE16d`931P@WP9gN~K&bpFDN4x36#e_8q!j+n003 zy3d_K#NM9nx$_qc4dutLE)*9$=#+rHMblNK`3)G>MVt^o6Qu-qr3juN~K&F$|ow?7$c?Z?dyB$si!lU#$(5hcXZ5&W950?sZ*z4dG)nyCNoqhAYy$s zKTdkyYpr)|+fphP9oGe-?Qd);779TS^mLy)IS!LG{2E~}g6I3q6B+Jn;8-6CCP(tL zE)7J1!@GA$-sCs#O3GLPSwGZcf%HrDSt>rbn0JI-xu4-*J|qY8+M(Sxuu- zBEw{=eDD|QO8R*W*Cs@gG%2MNLX1R5)+iRKp1hEdGCAO!UtM*f0dP6IU8L9YCM}>o zF)KfUAhg~)> z6|gl=Po2}*+6!^QR^}w2pQjoH!bHgJxZIoH`Yn`KVSRsVCD;%+9KLZ(_xND3nlHC# zqMSq?4Ax3~Npm}$-||<{;0Y$2YiGwpHjd787@Vlm|30c3L~RSw5J3|(N+!c0T?4j~ zjOI?Za{QH{9Y5es_L|x`i;XK)kh9=NXY^C&Op`U7Fk3sWBQ2Q|4QU-Cu*nr(Vf!Jg&KIRj4>uGspKhDRB<%9t(J%kw^F^o{94d^kXa4a z+UAB=0vNz%Fc86)-ijjwvS!BE5ZRnJ>avP_L^fHG5*Z~LTJo>|ZDIF6iBuCA1=rZP zvNF^J838*6>A$`mJ8T&BT7*fG^9)tvs0wUyQtwDBk!mR&c&6|9uX86e^{vU@p8x}Z zB51PUCuhyq4yvrYib~+CQKZBfYr;VJrAd)sGDkunSjaD3crN_Cvx0u z!FRg!*AJHKczj;)D?SM)P zU@*zC69F~}W&HP=dY9PUO4pXtPk!1&RTR#(yleE;1tf-`$%1bk*I(MJID*3`Gj=in zO~Q~j^zJvkB?Dwc!;IovKRffI&zRUp;Q(FJ`$b>%qfB)5kI(2&?u`3FYLcKP!z2&q zR!YDiPw#4dm5fGiF}tz<<^OZ`r(XmjBqr-R~|fByG<&wrg+DPa6s*39n-;KZ-aNEnFdbGzcN9#GmqRt`7Z zd6yE5pb0V-w}#ou3Cb{$Q7CNg=E?`3JMq2WFCTbXq?(3X!%UCMZptw+Ikp)P!ncp> zR|d?VTql0Fm1%;i8g3JC^sVf~P?(5FBez5<7v_%-y!wse-k%_s+|6B6 zRC5ghhVx!@vjxYi^!Z)!y{+ssi};${Fw8VgH;7ByU~nZ*t7170Epd5AnhX&!VRI`N zmwN}d{oUa9@99cEbFzsgo(6T&03K1-E(ZX3uGhRWVD4*UkI&<4a!evrHN*yBVn|cM zR+`FC{Bchz72K@A#d9K&o z(#$r`=IdKns|^GYXo$?PvyA|FDZCAV5Ijp>98|C|2Z{ibwnYF;FzFzdCi1*9yGr|? zE+6`3Ts+I9D^g8l%rv^S1$e|N13*rKA$VoLywq>zrEq-*oy9f>Ee3`lHlPS} z(gJa;4F+Uc3{+4;CQ%p^HZnnEv|sRg_IoE^uN-|T=-qE3k68^O)kHK6Z5;s~4Rcxm z0`!IS!_)f7)4I)p&NN$<#bu3behO#Wm==%{z!6xYh!K>5CX2PpNQY%IDk_{&rS5Rx zXwbbc=-D3?PMXL^ZcR(=)rBXQ*cfXPza4-f5I}4oG@uM{ga!dQ3C$KZTbPm1z)><_ z2$2CzaJZz7^0^XCT') kind, lines, rows = self.tap('S40') + self.assertEqual(self.labels(rows), [['7D', '1M', '1Y', '● Max'], ['Tickers ▶']]) + self.tap('Tickers ▶') + kind, lines, rows = self.tap('>') self.assertIn('● S40', [t for row in self.labels(rows) for t in row]) - self.assertEqual(self.labels(rows)[-1], ['◀ Back', '<', '>']) def test_portfolios_page_too(self): """Thirty-two portfolios to a page, in rows of four, then Group by, then the page buttons.""" From 220b82c6bad47a1eeb4828e49975f2fc9a6da123 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 15:59:08 +0000 Subject: [PATCH 40/41] CI: checkout v5 and setup-python v6, which run on Node 24 (Node 20 is deprecated) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- .github/workflows/ci.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7b8d4834..313a45aa 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -17,7 +17,7 @@ jobs: matrix: python-version: ['3.x', '3.13.5'] # latest release, and the production version steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 with: fetch-depth: 0 # the base branch is needed to see what a pull request changes - id: changes @@ -41,7 +41,7 @@ jobs: fi echo "docs_only=$docs_only" >> "$GITHUB_OUTPUT" - if: steps.changes.outputs.docs_only != 'true' - uses: actions/setup-python@v5 + uses: actions/setup-python@v6 with: python-version: ${{ matrix.python-version }} cache: pip @@ -56,7 +56,7 @@ jobs: # production meets it. Then every module imports in it, and the container starts as compose.yaml starts it. runs-on: ubuntu-24.04 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 with: fetch-depth: 0 # the base branch is needed to see what a pull request changes - id: changes From b1f35c9d90707b19fa87b8f0144e23c106d92e4d Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 16:00:51 +0000 Subject: [PATCH 41/41] finbot.service: the wrapped comments are one short line each Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01JVEkZofdPjQ4jtXFKipeUX --- finbot.service | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/finbot.service b/finbot.service index 993302ba..c3d84e22 100644 --- a/finbot.service +++ b/finbot.service @@ -4,12 +4,11 @@ Description=Finbot interactive chat bot [Service] Type=simple User=CHANGEME -# Or, instead of User=: DynamicUser=yes plus StateDirectory=finbot (state goes to /var/lib/finbot automatically) +# Or instead of User=: DynamicUser=yes and StateDirectory=finbot (state in /var/lib/finbot) ExecStart=/home/CHANGEME/finbot/bot.py # systemctl reload finbot re-reads preferences.conf without a restart ExecReload=/bin/kill -HUP $MAINPID -# Refuse to start on a mistake in finbot.env or preferences.conf (the bot never reads finbot.env itself, so this is -# where its typos are caught). "+" runs it as root. +# Refuse to start on a typo in finbot.env or preferences.conf ("+" runs the check as root) ExecStartPre=+/home/CHANGEME/finbot/util/check_config.py /opt/finbot.env Restart=on-failure RestartSec=30s @@ -17,8 +16,7 @@ StandardOutput=null StandardError=journal SyslogIdentifier=finbot Environment=PYTHONUNBUFFERED=1 -# finbot.env is the startup file: systemd loads it (as root), the bot only inherits the values. chmod 600. A change -# needs a restart. +# finbot.env: loaded by systemd as root (chmod 600); a change needs a restart EnvironmentFile=/opt/finbot.env [Install]