diff options
Diffstat (limited to 'dev-docs')
| -rw-r--r-- | dev-docs/alfred.md | 270 | ||||
| -rw-r--r-- | dev-docs/contessa.md | 92 | ||||
| -rw-r--r-- | dev-docs/daffy.md | 157 | ||||
| -rw-r--r-- | dev-docs/daft.md | 132 | ||||
| -rw-r--r-- | dev-docs/herald.md | 140 | ||||
| -rw-r--r-- | dev-docs/jeeves.md | 242 | ||||
| -rw-r--r-- | dev-docs/osterman.md | 274 | ||||
| -rw-r--r-- | dev-docs/salvador.md | 143 | ||||
| -rw-r--r-- | dev-docs/shireen.md | 173 |
9 files changed, 1623 insertions, 0 deletions
diff --git a/dev-docs/alfred.md b/dev-docs/alfred.md new file mode 100644 index 0000000..74876c5 --- /dev/null +++ b/dev-docs/alfred.md @@ -0,0 +1,270 @@ +# alfred - developer documentation + +## Overview + +Alfred is the owner's personal server management bot. It is not a general-purpose +bot. Every command is owner-only. It consolidates server administration tasks +(process management, system stats, soju bouncer control, 0x0 file hosting, and +coffee logging) into a single IRC interface. + +Alfred runs as its own Sopel instance using a dedicated config. The other bots +it manages are also Sopel instances, each with their own configs. + +--- + +## Module structure + +Alfred's directory contains multiple .py files, each loaded as an independent +Sopel plugin module. Sopel loads them individually (not as a package), so +relative imports do not work. Modules that need shared data use a sys.path +insertion to import `helpstrings` as a plain module — see the Help system +section below. + +``` +alfred/ + helpstrings.py - help string constants shared across all modules + help.py - !help command handler + coffee.py - coffee intake logger + server.py - system stats (!srv / !server) + soju.py - soju bouncer management (!irc) + files.py - 0x0 file hosting management (!files) + bots.py - bot process controller (!bot) + __init__.py - empty, marks directory as Sopel package +``` + +The `enable` list in alfred.cfg controls which modules load. All six above +are listed. + +--- + +## Help system + +### Design + +Help strings live in `helpstrings.py` as plain string constants and dicts. +No Sopel imports, no decorators. Each plugin imports it at the top with: + +```python +import os as _os, sys as _sys +_alfred_dir = _os.path.dirname(_os.path.abspath(__file__)) +if _alfred_dir not in _sys.path: + _sys.path.insert(0, _alfred_dir) +import helpstrings as h +``` + +Relative imports (`from . import helpstrings`) do not work because Sopel loads +each file as a standalone module, not as part of a package. The sys.path +insertion ensures `helpstrings` is findable regardless of how Sopel resolves +the module. + +This central location means help text only needs updating in one place. Each +plugin uses its topic string for the bare-call output (e.g. `!srv` with no +args) and `help.py` uses the same strings for `!help srv` responses. + +### Three-level hierarchy + +``` +!help → TOPICS (list of topic names) +!help <topic> → topic line (same as bare !<topic>) +!help <topic> <cmd> → single command description + usage +``` + +All responses are sent via `bot.notice()` to the requester — not said publicly +in channel. This matches the osterman and jeeves help behaviour. + +### Dispatch + +`help.py` delegates all resolution to `h.lookup(args)` in `helpstrings.py`. +`helpstrings.py` exposes `TOPIC_MAP`, `_ALL`, and `lookup()` alongside the +existing string constants — the same pattern used by osterman and jeeves. +Unknown args fall back to the TOPICS line with a "no help for X" prefix. + +### Bare-call consistency + +Every command plugin calls `bot.say(h.<TOPIC>_TOPIC)` when invoked with no +arguments. This means `!srv` and `!help srv` produce identical output. The +USAGE lists that previously existed in each plugin have been removed. + +### bot topic special case + +`!help bot <name>` where `<name>` is not `list` or `all` falls back to +`h.BOT_NAME`, the generic bot-name description. This is handled inside +`h.lookup()` — not in `help.py` — since managed bot names are dynamic and +not enumerable at help-text definition time. + +### coffee +n / -n lookup + +The `+n` and `-n` keys in `h.COFFEE` are looked up with `args[1].lower()`. +Sopel preserves the `+` and `-` characters in the argument string, so +`!help coffee +n` correctly maps to the `+n` entry. + +--- + +## coffee.py + +### Design + +The coffee tracker is owner-only and deliberately simple. It stores data as +a JSON file with a single `entries` dict mapping ISO date strings to cup counts. +No database, no schema migration, easy to inspect and edit manually. + +### Pending confirmation flow + +Some operations trigger a warning before applying: setting a count above +`WARN_HIGH` (5 cups) or bringing a count down to 0. The bot stores the pending +action in a module-level `_pending` dict keyed by nick, then waits for `!coffee +yes` or `!coffee no`. This is in-memory only and does not persist across +restarts. That is acceptable since pending actions are ephemeral by nature. + +The `_pending` dict is module-level (not in `bot.memory`) because Sopel plugins +can safely use module-level state when the state is simple and does not need to +be shared across plugins. + +### Week bar graph + +`_bar()` maps 7 days of cup counts to Unicode block characters (U+2581 through +U+2588) scaled relative to the week's maximum. Zero is always a space. + +### Timezone handling + +All date calculations use `ZoneInfo` from the standard library. The timezone +is read from `[coffee] timezone` in the config. This matters for the owner's +local midnight boundary: without timezone awareness, a cup logged at 11pm +could land on the wrong date if the server is in UTC. + +--- + +## server.py + +### Design + +A thin wrapper around standard Unix commands and /proc files. Each subcommand +is a function that takes `(bot, args)` and runs a subprocess or reads a file. +The dispatch table `_SUBCOMMANDS` maps subcommand names to functions, making +it easy to add new ones without touching the command handler. + +Commands that pass raw system output directly to IRC without formatting: +`!srv disk` (df -h), `!srv mem` (free -h), `!srv conns` (ss -s), `!srv who` (who). +These need proper parsing and formatted output. + +`!server` and `!srv` are both registered as aliases via `@plugin.commands()`. + +### top / topmem + +These parse `top -b -n 1` output. The column indices are hardcoded to match +the Alpine Linux `top` format from busybox: +``` +PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND +``` +If the system uses a different `top` (procps-ng for example), the column +indices may be wrong. This is documented as a known platform dependency. + +### logs + +The `LOG_PATHS` dict maps service names to candidate log file paths. The +function tries each path in order and reads from the first that exists. This +handles services that log to different locations depending on configuration. +Services not in `LOG_PATHS` fall back to `/var/log/<service>.log` and +`/var/log/<service>/error.log`. + +--- + +## soju.py + +### Design + +A thin wrapper around the `sojuctl` CLI. Every operation shells out to +`sojuctl` with the appropriate arguments. This avoids having to speak the +soju management protocol directly and stays compatible with future soju +versions as long as the CLI interface is stable. + +All commands except `!irc net presets` pass raw sojuctl output to IRC without +formatting. These need proper parsing and formatted output. + +### Multiline output + +`_send()` splits `sojuctl` output on newlines and sends each non-empty line +as a separate `bot.say()`. This handles `sojuctl` commands that return tables +or multiple status lines. + +### Network presets + +Common IRC networks are hardcoded in `NETWORK_PRESETS`. When adding a network +with just a name, the preset address is used. When adding with a full address, +the preset is bypassed. This avoids requiring the user to remember server +addresses for common networks. + +--- + +## files.py + +### Design + +Direct SQLite access to the 0x0 database rather than going through the HTTP +API for most operations. The HTTP API is used only for shorten and mirror +because those create new records. Stats, listing, and removal go to the +database directly for speed and to avoid HTTP overhead. + +### URL encoding + +0x0 uses a custom base-N encoding for file IDs. The alphabet is stored in +`URL_ALPHABET`. `_enbase()` and `_debase()` implement the encoding used by +0x0 to convert between integer IDs and URL-safe strings. These must match +0x0's implementation exactly or file lookups will fail. + +### File removal + +`_cmd_remove()` parses the filename to extract the base name and extension, +converts it back to a database ID via `_debase()`, then deletes the file from +disk and marks the record as removed in the database. It handles double +extensions like `.tar.gz` by taking the last two suffixes. + +The pattern `p.name[:-len(sufs) or None]` is intentional: when `sufs` is +empty (no extension), `-0 or None` evaluates to `None`, so `p.name[:None]` +returns the full name. When `sufs` is non-empty, it removes the extension +characters. + +### Raw flask output + +`!files prune` and `!files vscan` pass raw flask CLI stdout to IRC without +formatting. These need proper parsing and formatted output. + +### Stats labeling + +The stats query separates files by expiration state: +- `expiration IS NOT NULL` = live files with a set expiry +- `expiration IS NULL` = permanent files (no expiry set) + +--- + +## bots.py + +### Design + +Bots are managed as plain OS processes, not services. Alfred starts them with +`subprocess.Popen`, stops them with `pkill -f`, and checks if they are running +with `pgrep -f`. The match pattern is `sopel.*<config_path>`, which is specific +enough to avoid false matches. + +### Bot discovery + +`_discover()` globs `~/.config/sopel/*.cfg` and returns a dict of name to +config path. Alfred itself is excluded via the `EXCLUDE` set. This means adding +a new bot just requires dropping a config file in that directory. No hardcoded +list. + +### Restart race + +`_restart()` stops the bot then polls `_running()` up to 10 times with 0.2s +sleeps (2 seconds total) before starting it again. This avoids starting a new +process before the old one has fully exited. The poll is necessary because +`pkill` returns immediately after sending the signal, not after the process exits. + +--- + +## Config + +Alfred's config file is `alfred.cfg`. The `extra` key points to the script +directory. The `enable` list must include all six module names or they will +not load. Each module that uses config reads its own section via a +`StaticSection` subclass registered in `setup()`. diff --git a/dev-docs/contessa.md b/dev-docs/contessa.md new file mode 100644 index 0000000..07b4140 --- /dev/null +++ b/dev-docs/contessa.md @@ -0,0 +1,92 @@ +# contessa - developer documentation + +## Overview + +Contessa is a recipe suggestion bot. It loads a JSON recipe file at startup +and serves random suggestions or full recipes on demand. There is no external +API, no database, and no persistent state beyond what is in memory. + +--- + +## Module structure + +``` +contessa/ + commands.py - all commands and recipe logic + recipes.json - bundled recipe data + __init__.py - empty +``` + +--- + +## Data loading + +Recipes are loaded once in `setup()` and stored in `bot.memory["ct_recipes"]`. +The structure is a dict with meal-type keys mapping to lists of dish objects: + +```json +{ + "breakfast": [{"name": "...", "description": "...", "ingredients": [...], "steps": [...]}], + "lunch": [...], + "dinner": [...], + "snack": [...] +} +``` + +If the file is missing or malformed, `setup()` catches the exception and +initializes all categories to empty lists. The bot will respond with "no +recipes loaded" rather than crashing. + +There is no hot-reload. A restart is required to pick up changes to the +recipe file. + +--- + +## Recipe lookup + +`_find_dish()` does two passes over all categories: +1. Exact name match (case-insensitive) +2. Substring match (case-insensitive) + +This order matters. If someone has a dish named "Rice" and another named +"Rice Pudding", searching for "rice" returns "Rice" not "Rice Pudding". The +first exact match wins. + +The function is a generator consumer over `_all_dishes()`, which yields +`(category, dish)` tuples from all categories. The double traversal (once +for exact, once for partial) is acceptable given that recipe sets are small. + +--- + +## Output format + +`_fmt_recipe()` returns three strings meant to be sent as three separate IRC +messages: +1. Name and description +2. Ingredients joined by comma +3. All steps joined into one line, numbered + +The steps line can get long for recipes with many steps. IRC servers typically +truncate or drop messages over ~512 bytes. This is a known limitation. The +decision to keep steps on one line was made to minimize flood (three messages +per recipe is already pushing it in a shared channel). + +--- + +## Channel requirement + +All meal and recipe commands require channel context (`@plugin.require_chanmsg`). +The admin commands (join, part, chans, help) do not, since the owner needs to +manage the bot from PM. + +--- + +## Config + +| Key | Default | Description | +|-----|---------|-------------| +| `[contessa] recipes_path` | `/var/contessa/recipes.json` | Path to recipe JSON | + +The bundled `recipes.json` contains 24 recipes per category (96 total) covering +Egyptian, Middle Eastern, and Western dishes. It is the default data file and +can be replaced or extended without code changes. diff --git a/dev-docs/daffy.md b/dev-docs/daffy.md new file mode 100644 index 0000000..5ac7815 --- /dev/null +++ b/dev-docs/daffy.md @@ -0,0 +1,157 @@ +# daffy - developer documentation + +## Overview + +Daffy is a duck hunting game. Ducks spawn at random intervals in each channel. +Players fire with `!bang`, score points, and compete on a per-channel leaderboard. +Scores are persistent in SQLite. Game state is in memory only and resets on restart. + +--- + +## Module structure + +``` +daffy/ + daffy.py - all game logic, commands, and admin + __init__.py - empty +``` + +--- + +## Per-channel state + +The original design used global bot.memory keys for game state. This meant all +channels shared one duck, one timer, and one game toggle. A duck spawning would +broadcast to every channel the bot was in simultaneously. That was wrong. + +The fix was a per-channel state dict stored in `bot.memory["df_channels"]`, +keyed by the lowercased channel name. `_chan_state(bot, channel)` handles lazy +initialization and key normalization: + +```python +key = str(channel).lower() +bot.memory["df_channels"][key] = { + "active": False, # duck is currently up + "spawn_time": 0.0, # when the current duck spawned + "next_spawn": ..., # when the next duck will spawn + "empty_rounds": 0, # consecutive rounds with no shooter + "enabled": True, # hunt is running for this channel + "idle_rounds": 3, # auto-stop threshold +} +``` + +The key **must** be lowercased. Sopel's `bot.channels` yields `Identifier` objects +whose `__hash__` returns `hash(irc_lower(name))`, while command handlers produce +plain `str` from `trigger.sender` whose hash is case-sensitive. Without +normalization, `_tick` and command handlers key into different dict entries: +the tick sets `active=True` under `Identifier("#Chan")`, but `cmd_bang` looks up +`"#Chan"` (different hash), finds no entry, creates a fresh one with +`active=False`, and fires MOCK_MSG instead of registering the shot. The same +split caused `!hunt start` to never affect the state the tick was reading. + +State is created on first access (first tick or first command). Channels the +bot is in before the first tick may not have state yet. This is fine because +the tick runs every 5 seconds and creates state on its first pass. + +--- + +## Game tick + +`_tick()` runs every 5 seconds via `@plugin.interval(5)`. It holds +`df_game_lock` for its entire run and iterates all joined channels. For each: + +1. If a duck is active and the flee timeout has passed, the duck flees. +2. If no duck is active and the spawn timer has fired, a new duck spawns. + +The flee check increments `empty_rounds`. When `empty_rounds` reaches +`idle_rounds`, the hunt auto-stops for that channel. Each channel manages this +independently. + +--- + +## Threading + +A single `threading.Lock()` (`df_game_lock`) protects all channel states. +Per-channel locks were considered but rejected: the tick iterates all channels +under one lock anyway, so per-channel locks would not improve concurrency and +would add complexity. + +`cmd_bang` acquires the lock to check and clear `active`. The ammo check is +outside the lock since ammo is per-nick and Sopel's event handlers run in a +single thread. Two concurrent bangs from the same nick cannot happen under +normal Sopel operation. + +--- + +## Ammo + +Ammo is tracked in `bot.memory["df_players"]`, keyed by nick, and is not +per-channel. A nick carries their loaded/unloaded state across all channels. +This is intentional: if you fire in one channel you are empty in all channels +until you reload. Reload has a 3-second cooldown tracked by `df_reload_cd`. + +--- + +## Scoring + +Scores are stored in SQLite with `(nick, channel)` as the primary key. The +table uses `ON CONFLICT DO UPDATE` (upsert) so there is no separate select +before incrementing. The database path comes from config, defaulting to +`~/daffy_scores.db`. The directory is created automatically if it does not exist. + +--- + +## Hunt control + +`!hunt` requires channel context (`@plugin.require_chanmsg`) after the +per-channel refactor. It no longer makes sense in PM since each hunt is +tied to a specific channel. + +`!hunt idle 0` disables auto-stop for that channel. The idle counter still +increments but is never compared to a threshold. + +`!hunt idle N` checks the current `empty_rounds` immediately on set. If the +counter already meets the new threshold and no duck is active, the hunt stops +right away with an announcement — same behaviour as if the flee had just +triggered it. This prevents a silent bypass where setting a lower threshold +while `empty_rounds` was already at or above it would have no effect until +the next flee, and a single kill in between would reset the counter and escape +the threshold entirely. + +--- + +## !score from PM (owner only) + +The owner can use `!score` from PM to see scores across all channels, or +`!score clear all` / `!score clear #channel` to manage them. Regular users +get "use !score in a channel" from PM. + +--- + +## Known issues and tradeoffs + +**No persistence for game state.** On restart, all channels start fresh with +`enabled=True` and a new random spawn timer. There is no way to stop the +hunt, restart the bot, and have it stay stopped. This is acceptable since +the hunt auto-starts silently without bothering anyone until a duck actually +appears. + +**Idle auto-stop is per-restart.** `empty_rounds` resets to 0 on restart +because it is in-memory. A channel that was about to auto-stop will get +a fresh counter. + +--- + +## Config + +| Key | Default | Description | +|-----|---------|-------------| +| `[daffy] db_path` | `~/daffy_scores.db` | SQLite scores database | +| `[daffy] idle_rounds` | 3 | Empty rounds before auto-stop (0 = disabled) | +| `[daffy] spawn_min` | 60 | Lower bound of spawn interval (seconds) | +| `[daffy] spawn_max` | 300 | Upper bound of spawn interval (seconds) | +| `[daffy] flee_time` | 45 | Seconds before an unshot duck flees | + +All three timing values are loaded into `bot.memory` as floats (`df_spawn_min`, +`df_spawn_max`, `df_flee_time`) at startup and read from there at runtime, so +they fall back to the module-level constants if the config section is missing. diff --git a/dev-docs/daft.md b/dev-docs/daft.md new file mode 100644 index 0000000..880b95c --- /dev/null +++ b/dev-docs/daft.md @@ -0,0 +1,132 @@ +# daft - developer documentation + +## Overview + +Daft is a YouTube playlist manager for IRC. Anyone can queue tracks. Ops control +playback. The bot announces tracks and advances the queue on a fixed timer. +There is no actual audio playback. The timer simulates track duration for +queue progression purposes. + +--- + +## Module structure + +``` +daft/ + dj.py - all commands and queue logic + __init__.py - empty +``` + +--- + +## Queue model + +The queue is a plain list stored in `bot.memory["dj_queue"]`. There is no +persistence. The queue is lost on restart. This is intentional: a music queue +is a live social activity, not something that needs to survive a restart. + +`dj_pos` is the index of the currently playing track. `-1` means nothing is +playing. When a track ends, `_on_track_end()` advances `dj_pos` by 1 and +calls `_play_at()` for the next position. + +--- + +## Track duration + +`TRACK_DURATION = 120` seconds. This is a fixed constant, not the actual +duration of the YouTube video. The bot has no way to query video duration +without an API key. The timer is just for queue advancement in the IRC +session. If you want a different advancement interval, change the constant. + +--- + +## URL validation + +`_extract_video_id()` validates and normalizes YouTube URLs: + +1. Ensures the scheme is http or https. +2. Checks the domain against a whitelist of known YouTube hosts. +3. Rejects shorts, embeds, and live stream paths. +4. Extracts the 11-character video ID. + +All queued URLs are normalized to `https://youtu.be/<id>`. This means the +same video added with different URL formats (watch?v=, youtu.be/, m.youtube.com) +is detected as a duplicate. + +The regex `^[A-Za-z0-9_-]{11}$` validates the video ID format. This is the +format YouTube has used since the beginning. If YouTube ever changes their +ID format, this regex needs to be updated. + +--- + +## Threading + +`_on_track_end()` is called from a `threading.Timer` callback, which runs +in a separate thread from Sopel's main event loop. It reads and writes shared +state in `bot.memory`. This requires locking. + +A `threading.RLock()` (`dj_lock`) is used instead of a regular `Lock()` because +`cmd_skip` calls `_on_track_end()` directly from the main thread while holding +the lock. An RLock allows the same thread to acquire it multiple times without +deadlocking. + +The lock is acquired in: +- `_on_track_end()` (timer thread) +- `cmd_play`, `cmd_stop`, `cmd_skip`, `cmd_remove`, `cmd_clear` (main thread) +- `cmd_dj`, `cmd_np`, `cmd_queue` (main thread, for read consistency) + +`_play_at()` and `_cancel_timer()` do not acquire the lock themselves because +they are always called by code that already holds it. + +--- + +## Removing a playing track + +`cmd_remove()` removes the current track and advances to the next one. +The logic: + +1. `queue.pop(pos)` removes the current track. The element that was at + `pos+1` is now at `pos`. +2. `dj_pos` is set to `pos - 1`. +3. `_on_track_end()` is called, which computes `next_pos = (pos-1) + 1 = pos`, + which points to the track that was previously next. + +This correctly advances to the next track without skipping it. + +If the removed track was the last one, `next_pos` will be out of bounds and +playback stops normally. + +--- + +## Loop mode + +In loop mode, `_on_track_end()` wraps around to index 0 when it reaches the +end of the queue. The `!queue` display in loop mode shows up to 4 upcoming +tracks wrapping around the current position, excluding the current track +itself. + +--- + +## dj_channel + +`bot.memory["dj_channel"]` stores the channel where playback was started. +This is used when `_on_track_end()` fires from the timer thread and needs to +know where to send "now playing" messages. It is also used as a fallback in +`cmd_remove()` when `trigger.sender` might differ from the original playback +channel. + +--- + +## Known issues and tradeoffs + +**No actual playback.** The bot announces tracks but does not play audio. +It was designed as a queue coordinator for a separate external player. The +TRACK_DURATION timer is a rough approximation. + +**No duration info.** Without the YouTube Data API, there is no way to know +the actual length of a video. Using a fixed 120s timer means long videos get +cut short and short videos have dead air. + +**Queue position display.** `!queue` shows position 1-indexed from the next +track. The currently playing track is not shown in the queue list, only in +`!np`. diff --git a/dev-docs/herald.md b/dev-docs/herald.md new file mode 100644 index 0000000..c23cf44 --- /dev/null +++ b/dev-docs/herald.md @@ -0,0 +1,140 @@ +# herald - developer documentation + +## Overview + +Herald (also deployed as bikabot) does two things: it follows the owner around +IRC channels automatically, and it responds to configured keywords in channel +messages. These are implemented in two independent plugin files. + +--- + +## Module structure + +``` +herald/ + commands.py - follow logic, admin commands, owner presence tracking + respond.py - keyword-based auto-responder + __init__.py - empty +``` + +--- + +## commands.py: follow logic + +### How follow works + +When `!follow on` is active, the bot sends a WHOIS for the owner every 30 +seconds via `@plugin.interval(30)`. The server responds with one or more IRC +numeric replies: + +- **319 RPL_WHOISCHANNELS**: lists the channels the owner is in. +- **318 RPL_ENDOFWHOIS**: signals the end of the WHOIS response. + +If 319 arrives, `_whois_channels()` parses the channel list and joins any +channel the owner is in that the bot is not already in. Joined channels are +tracked in `bot.memory["follow_joined"]`. + +If 318 arrives with no preceding 319 (the flag `whois_got_channels` is still +False), the owner is offline. The bot leaves all follow-joined channels. + +### Preconfigured channels + +Channels listed under `channels` in the bot config are never auto-left, even +if the owner leaves or goes offline. `_configured_channels()` reads +`bot.settings.core.channels` and returns a set of lowercase channel names. + +This matters for the PART and QUIT event handlers: if the owner leaves a +channel, the bot only follows if the channel is not preconfigured. + +### PART and QUIT handling + +`on_owner_part()` fires on any PART. If the parting nick is the owner and the +channel is not preconfigured, the bot parts too. + +`on_owner_quit()` fires on QUIT and leaves all non-preconfigured channels +immediately without waiting for the next WHOIS cycle. + +### Why WHOIS instead of tracking JOIN/PART directly + +Tracking JOIN/PART directly would miss channels the owner was already in before +the bot connected. WHOIS gives a complete current picture regardless of when +the bot joined the network. + +### RPL_WHOISCHANNELS parsing + +The 319 reply contains channel names prefixed with membership status characters +(`@`, `+`, `~`, `&`, `%`). The parser strips these prefixes before comparing +channel names: + +```python +chan = part.lstrip("@+~&%") +``` + +--- + +## commands.py: !pause / !resume + +`!pause` sets `bot.memory["herald_paused"] = True`; `!resume` clears it. Both +are owner-only and work in any context (channel or PM). The `respond()` handler +in `respond.py` checks this flag before processing any message. + +--- + +## commands.py: !unfollow + +`!unfollow` leaves all channels in `follow_joined` immediately and clears the +set. It does not disable `follow_enabled`. After `!unfollow`, the bot stops +following for that session but `!follow on` can restart it. + +--- + +## respond.py: keyword responder + +### How it works + +`respond()` fires on every channel message. It loads `keywords` and `responses` +from the `[respond]` config section (both are `ListAttribute`, supporting +newline-separated values in the config file). If any keyword appears anywhere +in the message (case-insensitive using `casefold()`), a random response is sent. + +### Why casefold over lower + +`casefold()` is more aggressive than `lower()` for Unicode. For a bot that +might see messages in Arabic or other scripts, casefold handles more edge +cases correctly. + +### Error handling + +The match is wrapped in a try/except for `UnicodeError` and `AttributeError`. +This guards against malformed trigger data from the IRC layer. + +--- + +## State + +``` +bot.memory["follow_enabled"] - bool, follow mode on/off +bot.memory["follow_joined"] - set of channel names joined via follow +bot.memory["whois_got_channels"] - bool, reset each WHOIS cycle +bot.memory["herald_paused"] - bool, keyword responses paused on/off +``` + +None of this persists across restarts. Both `follow_enabled` and `herald_paused` +are off by default on startup. + +`herald_paused` is set by `!pause` / `!resume` in `commands.py` and read by +`respond()` in `respond.py`. The two plugins share `bot.memory`, so no import +or coupling between files is needed. + +--- + +## Known issues and tradeoffs + +**WHOIS interval is fixed at 30 seconds.** If the owner moves channels quickly, +the bot may lag behind by up to 30 seconds. Shortening the interval increases +WHOIS traffic on the server. + +**Follow only joins, never leaves proactively.** With `!follow on`, the bot +joins channels the owner is in but only leaves them on QUIT/PART events or +when the owner goes offline. It does not automatically leave a channel if the +owner leaves without the bot noticing (network split, for example). diff --git a/dev-docs/jeeves.md b/dev-docs/jeeves.md new file mode 100644 index 0000000..27ba163 --- /dev/null +++ b/dev-docs/jeeves.md @@ -0,0 +1,242 @@ +# jeeves - developer documentation + +## Overview + +Jeeves is a combined-features IRC bot built on Sopel. It merges the functionality +of herald (admin/follow/keywords), daft (DJ queue), daffy (duck hunting), contessa +(recipes), shireen (news/weather), and salvador (image art), and adds a new quote +bank with auto-posting. All tunable parameters are runtime-configurable via `!set` +and persisted to a single JSON file. + +--- + +## Module structure + +``` +jeeves/ + jv_core.py - shared helpers, CONFIG_KEYS, db I/O, auth utils (not in enable list) + helpstrings.py - help string constants and lookup() (not in enable list) + admin.py - !join, !part, !chans, !pause, !resume + dj.py - YouTube DJ queue + duck.py - duck hunting game + recipes.py - recipe bot + news.py - weather (wttr.in) + RSS news + art.py - mIRC colour art renderer (Pillow) + respond.py - keyword responder + quotes.py - quote bank + auto-post + config.py - !set, !config + help.py - !help + __init__.py - empty +``` + +--- + +## Shared state (bot.memory) + +`jv_core.ensure_setup()` initializes all persistent state on first call: + +``` +bot.memory["jv_db"] - main data dict loaded from JSON +bot.memory["jv_db_path"] - path to db.json +bot.memory["jv_duck_db_path"] - path to scores.db +bot.memory["jv_recipes_path"] - path to recipes.json +bot.memory["jv_lock"] - threading.Lock() for db writes +``` + +Additional per-module memory keys (non-persistent, set in setup()): + +``` +bot.memory["dj_queue"] - list of normalized YouTube URLs +bot.memory["dj_pos"] - current position (-1 = not playing) +bot.memory["dj_playing"] - bool +bot.memory["dj_timer"] - threading.Timer or None +bot.memory["dj_loop"] - bool +bot.memory["dj_channel"] - str or None +bot.memory["dj_lock"] - threading.RLock() (separate from jv_lock) +bot.memory["dk_channels"] - {channel_lower: state_dict} duck state +bot.memory["dk_reload_cd"] - {nick: timestamp} reload cooldown +bot.memory["dk_players"] - {nick: {"ammo": int}} +bot.memory["dk_lock"] - threading.Lock() +bot.memory["jv_paused"] - bool (keyword responder) +bot.memory["jv_recipes"] - dict loaded from recipes.json +``` + +--- + +## Database structure + +### JSON database (jv_db) + +```json +{ + "config": {"dj_track_duration": 90}, + "channels": { + "#example": { + "config": {"duck_spawn_min": 30}, + "news": {"enabled": true, "last_broadcast": 0}, + "quotes": {"enabled": false, "last_broadcast": 0} + } + }, + "quotes": ["quote text", ...], + "news_feeds": [...9 feeds...], + "news_seen": ["guid1", ...] +} +``` + +Config resolution is a two-level merge: global config override → channel config +override, applied on top of hardcoded defaults from `CONFIG_KEYS`. + +`cfg(bot, channel)` returns the merged dict. +`cfg_val(bot, key, channel)` returns a single typed value. + +### SQLite database (scores.db) + +Single `scores` table: `nick`, `channel`, `score`. Primary key `(nick, channel)`. + +--- + +## CONFIG_KEYS + +```python +CONFIG_KEYS = { + "dj_track_duration": (int, 120, True, "seconds per track before auto-advance"), + "dj_queue_cap": (int, 50, True, "max tracks in queue"), + "art_height": (int, 6, True, "default lines for !draw (4-12)"), + "duck_spawn_min": (int, 60, False, "min seconds between duck spawns"), + "duck_spawn_max": (int, 300, False, "max seconds between duck spawns"), + "duck_flee_time": (int, 45, False, "seconds before duck flees"), + "duck_idle_rounds": (int, 3, False, "empty rounds before auto-stop (0=off)"), + "news_interval": (int, 90, False, "minutes between news broadcasts"), + "news_count": (int, 1, False, "headlines per broadcast cycle"), + "quotes_interval": (int, 60, False, "minutes between quote auto-posts"), +} +``` + +The tuple is `(type, default, global_only, description)`. + +`global_only=True` means the key can only be changed in PM by the owner. Setting +it in a channel is rejected with a notice. This is used for keys that have no +per-channel meaning (DJ track duration, art height). + +--- + +## Help system + +Same three-level pattern as alfred and osterman: + +``` +!help → TOPICS +!help <topic> → topic line (list of commands) +!help <cmd> → command description +!help <topic> <cmd> → same as above +``` + +`helpstrings.lookup(args)` in `helpstrings.py` handles all dispatch. `help.py` +imports it via sys.path insertion. Unknown args fall back to the TOPICS line. + +--- + +## Module import pattern + +All plugin files use sys.path insertion to import jv_core: + +```python +import os as _os, sys as _sys +_d = _os.path.dirname(_os.path.abspath(__file__)) +if _d not in _sys.path: + _sys.path.insert(0, _d) +import jv_core as jv +``` + +`jv_core.py` and `helpstrings.py` are not in the `enable` list and are never +loaded directly by Sopel. They are shared modules, importable via the sys.path +trick above. + +--- + +## Message routing + +| Output | Method | Destination | +|--------|--------|-------------| +| Game events (duck spawn/kill/flee) | `bot.say` | channel | +| Score leaderboard | `bot.say` | channel | +| Owner-only score queries | `bot.notice` | nick | +| DJ now-playing, queue | `bot.say` | channel | +| DJ feedback (queued, removed) | `bot.say` | channel | +| DJ error feedback | `bot.notice` | nick | +| News/weather | `bot.say` | channel/sender | +| News broadcast (tick) | `bot.say` | channel | +| Quotes auto-post | `bot.say` | channel | +| Config feedback | `bot.notice` | nick | +| Admin feedback | `bot.notice` | nick | +| Help | `bot.notice` | nick | + +--- + +## DJ notes + +- Track duration: read fresh on each play via `jv.cfg_val(bot, "dj_track_duration")` — + changing `dj_track_duration` affects tracks queued after the change, not any + currently running timer. +- `!loop` (no args): shows state to anyone, no privilege check. +- `!loop on|off`: requires op (checked manually via `jv.is_authorized`). +- Queue display: no leading indentation — `{i}. {url}` format. + +--- + +## Duck notes + +- Spawn times are read per-tick from `jv.cfg_val(bot, "duck_spawn_min", channel)`. + A config change takes effect on the next spawn scheduling. +- Each channel has independent state in `dk_channels[channel.lower()]`. +- Hunt is off by default; ops must `!hunt start` per channel. +- `!hunt` status output includes current `duck_spawn_min`, `duck_spawn_max`, and + `duck_flee_time` values. Authorized users also receive a notice reminding them + these are adjustable via `!set`. + +--- + +## News notes + +- `!news interval` and `!news count` commands from shireen are replaced by + `!set news_interval` / `!set news_count`. The `!news` command only handles + on/off toggle and status display. +- Per-channel news state (`enabled`, `last_broadcast`) lives in + `jv.chan_db(bot, channel)["news"]`. +- Broadcast tick runs every 60s; the interval check is done in-tick against + `cfg_val(bot, "news_interval", channel) * 60`. + +--- + +## Quotes notes + +- Quotes are stored globally in `jv_db["quotes"]` — shared across all channels. +- Per-channel auto-post state is in `jv.chan_db(bot, channel)["quotes"]`. +- `!quote del N` uses 1-based indexing for user-facing display. + +--- + +## Pillow dependency (art.py) + +`art.py` checks `_PILLOW` at import time. If Pillow is not installed, `!draw` +responds with a notice and returns immediately rather than crashing. + +`CHAR_RATIO = 2.0` in `art.py` corrects for IRC monospace character cells +being approximately twice as tall as they are wide. `_fit()` multiplies the +image's pixel aspect ratio by this factor before computing output dimensions, +preventing the 2× vertical stretch that would otherwise occur. + +--- + +## Known issues and tradeoffs + +**DJ timer not persisted.** If the bot restarts mid-track the timer is lost. +The queue survives (it's in memory only while the bot runs anyway — not stored +to disk). Reissue `!play` after restart. + +**Duck hunt off by default.** Unlike daffy, jeeves does not auto-start the hunt +on any channel. Ops must `!hunt start`. This avoids unwanted duck spam in channels +that don't want the game. + +**Global quotes bank.** All channels share the same quote pool. There is no +per-channel quote list. diff --git a/dev-docs/osterman.md b/dev-docs/osterman.md new file mode 100644 index 0000000..7676def --- /dev/null +++ b/dev-docs/osterman.md @@ -0,0 +1,274 @@ +# osterman - developer documentation + +## Overview + +Osterman is a full channel moderation bot. It handles automatic protection +(flood, caps, repeat, badword, clone, join flood, nick flood), manual +moderation commands, auto-modes on join, persistent ban tracking, user +event logging, and idle detection. + +It is split across five plugin files that share state through `bot.memory`. + +--- + +## Module structure + +``` +osterman/ + protect.py - setup, passive protection, on_join, on_message, MODE/NICK/PART/QUIT handlers + commands.py - all user-facing moderation and config commands + tracking.py - SQLite event log, greeting, !seen, !info, !log + idle.py - idle detection tick, !afk, !idle + help.py - three-level help system (!help) + helpstrings.py - help string constants shared across modules + __init__.py - empty +``` + +--- + +## Help system + +Help strings live in `helpstrings.py` as plain string constants and dicts. +No Sopel imports, no decorators. `help.py` imports it via sys.path insertion +(same pattern as alfred): + +```python +import os as _os, sys as _sys +_osterman_dir = _os.path.dirname(_os.path.abspath(__file__)) +if _osterman_dir not in _sys.path: + _sys.path.insert(0, _osterman_dir) +import helpstrings as h +``` + +Three-level hierarchy: +``` +!help → TOPICS (list of topic names and commands) +!help <topic> → topic line listing commands +!help <cmd> → single command description + usage +!help <topic> <cmd> → same as above +``` + +`helpstrings.lookup(args)` handles all dispatch. Unknown args fall back to +the TOPICS line with an "no help for X" prefix. + +--- + +## Shared state + +`protect.py`'s `setup()` initializes all shared state in `bot.memory`: + +``` +bot.memory["os_db"] - full database dict (loaded from JSON) +bot.memory["os_db_path"] - path to the JSON db file +bot.memory["os_lock"] - threading.Lock() for db writes +bot.memory["os_flood"] - {(channel, nick): [timestamps]} flood tracking +bot.memory["os_repeat"] - {(channel, nick): [(hash, timestamp)]} repeat tracking +bot.memory["os_join_flood"] - {(channel, host): [timestamps]} join flood (per channel) +bot.memory["os_nick_flood"] - {"user@host": [timestamps]} nick flood (network-wide) +bot.memory["os_last"] - {(channel, nick): timestamp} last message time +``` + +`os_join_flood` is keyed by `(channel, host)` so join flood counts are isolated +per channel. A user joining multiple channels quickly does not bleed the flood +counter across channels. `os_nick_flood` is network-wide by design — NICK changes +apply across all channels. + +The other modules access these keys with safe fallbacks in case of load order +differences. Sopel does not guarantee plugin load order, so every module that +touches `os_db` uses `bot.memory.get("os_db", {})` rather than a direct dict +access. + +`tracking.py` derives its SQLite database path from `os_db_path` by replacing +the filename with `track.db`. This keeps both files in the same directory +without requiring a separate config key. + +--- + +## Database structure + +### JSON database (os_db) + +```json +{ + "config": {"flood_threshold": 5, ...}, + "acl": ["nick1"], + "whitelist": ["*!*@services.dal.net"], + "blacklist": [], + "exceptions": [], + "channels": { + "#example": { + "config": {"flood_threshold": 3}, + "bans": [{"mask": "...", "added_by": "...", "timestamp": 0}], + "filters": ["regex1"], + "badwords": ["word1"], + "exceptions": [], + "autoop": [], + "autovoice": [], + "autohalfop": [] + } + } +} +``` + +Config resolution is a three-level merge: hardcoded defaults → global config +override → channel config override. `_cfg(bot, channel)` returns the merged +result. `!set` in a channel writes to channel config; `!set` in PM writes to +global config (owner only). + +### SQLite database (track.db) + +Single `events` table: `id`, `nick`, `type`, `channel`, `detail`, `timestamp`. +Indexed on `nick` (NOCASE) and `timestamp DESC`. + +Event types: `join`, `part`, `quit`, `kick`, `ban`, `nick`, `mute`, `unmute`, +`lock+m`, `lock+i`, `unlock-m`, `unlock-i`. + +--- + +## Message routing + +| Output | Method | Destination | +|--------|--------|-------------| +| `!seen`, `!idle <nick>` results | `bot.say` | channel (`trigger.sender`) | +| `!info`, `!log` results | `bot.say` | PM (`trigger.nick`) | +| `!ban list`, `!config`, `!filter list`, `!badword list` | `bot.notice` | nick | +| All other command feedback | `bot.notice` | nick | +| Greetings, idle warns, bad word warn | `bot.notice` | nick | + +`_notice(bot, nick, text)` in `commands.py` and `help.py` calls +`bot.notice(text, nick)` — a true IRC NOTICE, not a PRIVMSG. This is the +standard approach for automated bot feedback that should not trigger other +bots or appear as a regular chat message. + +--- + +## protect.py: passive protection + +### on_join + +Runs in order: +1. Blacklist check (global instant-ban, bypasses whitelist). +2. Whitelist/exception check. If exempt, still applies auto-modes. +3. Persistent ban reapply. +4. Clone detection. +5. Join flood detection (keyed per channel+host). +6. Auto-modes (`_apply_auto_modes()`). + +Each step returns early if it takes action. + +### on_message + +Runs in order under `os_lock`: +1. Regex content filters. +2. Badword list. +3. Caps filter. +4. Repeat filter. +5. Flood control. + +Each filter returns early on action. Flood is last because it is the least +specific. + +### Privilege degradation + +- Op: full enforcement (kick, ban, +b mode). +- Halfop: kick only (no +b, no re-op). +- None: tracking only (`os_last` updates, no enforcement). + +### Takeover mitigation + +`on_mode()` watches for `-o` events on ACL nicks. If the bot has op, it +immediately re-ops them. + +--- + +## commands.py + +### ban vs bankick + +`!ban` and `!bankick` are functionally identical. They both ban and kick. +`!ban` also handles the `!ban list` subcommand. `!bankick` exists as an +explicit alias without the list subcommand ambiguity. + +### tempban + +Auto-unban is implemented with `threading.Timer`. The timer is not persisted. +If the bot restarts before the timer fires, the ban is never lifted. The expiry +is stored in the JSON db (`"expires"` key), so the data is there; it is just +not acted on at startup. + +### !filter del / !badword del + +Deleted by index, not by value. Use `!filter list` / `!badword list` to see +indices, then `del N`. This avoids ambiguity with regex special characters. + +### !unban db cleanup + +`cmd_unban` removes matching entries from the internal ban database using +`fnmatch.fnmatch(b["mask"], mask)` — tests whether the stored ban mask matches +the target being unbanned. The argument order matters: `fnmatch(pattern, string)` +where the stored mask is the pattern and the unban target is the string being +tested against it. + +--- + +## tracking.py + +### Greeting + +On JOIN, checks for prior events before logging the join itself. If the nick +has any prior events: "Welcome back, nick!". If not: "Welcome to #channel, nick!". +Sent as IRC NOTICE to the joining user. Controlled by the `greet` config key. + +### !seen scoping + +In channel: checks only the current channel — live presence check against +`bot.channels[channel]`, then DB query with `AND channel = ?`. Reports in channel. + +In PM: owner/ACL only. Cross-channel lookup across all joined channels and +the full event log. Reports as IRC NOTICE to nick. All other callers get +"use !seen in a channel". + +### !info scoping + +In channel: all DB queries scoped to `channel = ?`. Alias lookups (nick-change +events) are global because NICK events have no channel. Reports in PM. + +In PM: owner/ACL only. All queries are global. Reports in PM. All other +callers get "use !info in a channel". + +--- + +## idle.py + +### Idle tick + +`@plugin.interval(60)` checks all users in all channels against +`idle_warn_min` and `idle_kick_min` (minutes). Uses `os_last` timestamps. +Users with no `os_last` entry are treated as active. + +--- + +## Known issues and tradeoffs + +**tempban does not survive restarts.** The expiry timestamp is stored in the +JSON db but not checked on startup. Tempbans that expire while the bot is down +are never lifted. + +**Clone detection is host-only.** Checks `trigger.host`, not `user@host`. +Users behind a BNC share the same host. Either raise `clone_limit` or whitelist +the BNC host. + +**ACL management requires a channel.** `!acl` has `@plugin.require_chanmsg`. +Cannot manage the ACL from PM. Workaround: call from any channel the bot is in. + +**`!ban list` only shows bot-tracked bans.** Bans set manually by channel ops +(not through osterman) do not appear. The list reflects the internal JSON db, +not the IRC +b list. + +--- + +## Config + +| Key | Description | +|-----|-------------| +| `[osterman] db_path` | Path to JSON database (auto-created if missing) | diff --git a/dev-docs/salvador.md b/dev-docs/salvador.md new file mode 100644 index 0000000..fa6e75f --- /dev/null +++ b/dev-docs/salvador.md @@ -0,0 +1,143 @@ +# salvador - developer documentation + +## Overview + +Salvador converts an image URL to mIRC colour art using half-block characters +and Floyd-Steinberg dithering. It uses Pillow for image loading and resizing +and implements its own color matching and dithering against the 99-color mIRC +palette. + +--- + +## Module structure + +``` +salvador/ + salvador.py - image fetching, rendering, !draw command + commands.py - admin commands (!join, !part, !chans, !help) + __init__.py - empty +``` + +--- + +## Half-block rendering technique + +Each IRC character represents two vertical pixels. The character used is +U+2584 (LOWER HALF BLOCK). The foreground color represents the lower pixel +and the background color represents the upper pixel. IRC color codes are +written as `\x03fg,bg`. + +To render an image at N character lines, the image is resized to N*2 pixels +tall. Pairs of rows are then combined: row 0 and row 1 become character line 0, +row 2 and row 3 become character line 1, and so on. + +This doubles the effective vertical resolution compared to full-block +approaches where each character is one pixel. + +--- + +## Color matching + +`_nearest(r, g, b)` finds the closest mIRC palette color using a +perceptually-weighted RGB distance formula: + +```python +rmean = (r + 128) / 2 +dist = (2 + rmean/256) * dr*dr + 4*dg*dg + (2 + (255-rmean)/256) * db*db +``` + +This is a standard approximation of human color perception. Green is weighted +highest (coefficient 4). Red and blue are weighted by the red channel mean. +Pure Euclidean distance produces noticeably worse results especially in +saturated reds and blues. + +The full 99-color mIRC palette is hardcoded in `MIRC`. The first 16 are the +original mIRC colors. Colors 16-98 are the extended palette added in later +mIRC versions. Colors 88-98 are grayscale. + +--- + +## Floyd-Steinberg dithering + +`_dither()` is a manual implementation of Floyd-Steinberg dithering. +PIL's built-in dithering only targets the web palette or a custom palette +using PIL's quantize methods, which do not map cleanly to the mIRC palette +index scheme. Implementing it directly gives full control over the palette +and error diffusion. + +The algorithm quantizes each pixel to the nearest palette color, computes +the quantization error (difference between original and quantized RGB), and +distributes that error to neighboring pixels: + +``` +(x+1, y ): 7/16 of error +(x-1, y+1): 3/16 of error +(x , y+1): 5/16 of error +(x+1, y+1): 1/16 of error +``` + +The buffer is floats to accumulate fractional errors. Final values are clamped +to 0-255 before color matching. + +--- + +## Fetch and size limits + +`_fetch()` sends a request with a custom User-Agent and reads up to `MAX_BYTES` +(5MB). It checks the Content-Type header before reading the body. If the +response is not an image type, it raises a ValueError immediately rather than +reading the full body. + +The size limit prevents memory exhaustion from large images. 5MB is generous +for web images and strict enough to catch accidental links to large files. + +--- + +## Aspect ratio fitting + +`_fit()` computes the output dimensions to fit within `MAX_WIDTH x height` +while preserving the aspect ratio. IRC monospace character cells are +approximately twice as tall as they are wide, so a naïve pixel aspect ratio +would produce output that is 2× stretched vertically. + +`CHAR_RATIO = 2.0` corrects for this: the image's pixel aspect ratio is +multiplied by `CHAR_RATIO` before the fit comparison, so the renderer allocates +twice as many character columns as the raw pixel ratio would suggest, cancelling +the cell height distortion. + +--- + +## Flood protection + +A `FLOOD_DELAY` of 0.4 seconds is inserted between output lines via +`time.sleep()`. Without this, sending 6-12 lines in rapid succession triggers +flood protection on most IRC servers and the lines get dropped. + +`MAX_HEIGHT = 12` lines. Sending more than 12 lines would be disruptive in +any channel. + +--- + +## URL handling + +The command strips trailing punctuation from the URL (`.`, `,`, `)`, `>`). +This handles the common case where someone pastes a URL at the end of a +sentence and the punctuation gets included in the copied text. + +Only `http://` and `https://` URLs are accepted. This blocks `file://` and +other schemes that could access local resources. + +--- + +## Known issues and tradeoffs + +**Color accuracy degrades on complex gradients.** The 99-color mIRC palette +is coarse. Dithering helps significantly but cannot fully compensate for the +palette limitation. + +**No image caching.** Every `!draw` fetches the image fresh. Repeated calls +with the same URL re-download every time. + +**Blocking fetch.** `_fetch()` runs in the command handler (main Sopel thread) +and blocks while downloading. Large images or slow servers will block the bot +for the duration of the download (up to 10 seconds timeout). diff --git a/dev-docs/shireen.md b/dev-docs/shireen.md new file mode 100644 index 0000000..85563a5 --- /dev/null +++ b/dev-docs/shireen.md @@ -0,0 +1,173 @@ +# shireen - developer documentation + +## Overview + +Shireen is a news and weather bot. It fetches weather from wttr.in (no API +key required) and headlines from RSS/Atom feeds. It posts fresh headlines +automatically at a configurable per-channel interval and deduplicates across +restarts using a persisted GUID list. + +--- + +## Module structure + +``` +shireen/ + commands.py - all logic, commands, broadcast tick + __init__.py - empty +``` + +--- + +## Data persistence + +All state is stored in a single JSON file at `data_path`. The structure is: + +```json +{ + "channels": { + "#example": { + "news_enabled": true, + "broadcast_interval": 90, + "broadcast_count": 1, + "last_broadcast": 0 + } + }, + "feeds": [...], + "seen": ["guid1", "guid2"] +} +``` + +The file is read once at startup into `bot.memory["sh_data"]` and written +back on every change. This means the in-memory dict is the live state and +the file is the persistence layer. Changes made to the file while the bot +is running will be overwritten on the next save. + +### _DEFAULT_CHANNEL mutation + +`setup()` mutates the module-level `_DEFAULT_CHANNEL` dict to apply the +config-specified defaults. New channels added to `data["channels"]` after +startup will use these mutated defaults. This is intentional: the config +file controls what a "fresh" channel looks like. + +--- + +## Feed parsing + +`_fetch_feed()` parses both RSS 2.0 and Atom feeds: + +- RSS: looks for `<item>` elements with `<title>`, `<link>`, and `<guid>`. +- Atom: falls back if no items found, looks for `<entry>` elements with the + Atom namespace. + +The fallback order means Atom feeds are only tried if RSS parsing finds +nothing. Feeds that mix both formats will be parsed as RSS. + +GUIDs are used as unique identifiers. If a feed does not provide a GUID, +the link URL is used instead. This is less stable but still prevents +immediate re-posting. + +--- + +## URL cleaning + +`_clean_url()` strips query strings and fragments from feed item links before +storing and posting them. RSS feeds from major outlets routinely include +tracking parameters in article links (`utm_source`, `ref`, etc.). Stripping +them produces cleaner, shorter URLs. + +--- + +## Deduplication + +`data["seen"]` is a list of GUIDs that have already been posted. Before +posting an item, its GUID is checked against this list. After posting, +the GUID is added. The list is pruned to 1000 entries (oldest removed) on +every save. This prevents the file from growing unbounded. + +The seen list persists across restarts so the same article is never reposted +even if the bot is restarted between broadcast cycles. + +--- + +## Broadcast tick and locking + +`@plugin.interval(60)` runs every minute and checks each channel. + +The original implementation held `sh_lock` for the entire tick including all +HTTP fetches. This blocked `!headlines` and `!news` commands for the full +duration of the RSS fetches (multiple feeds, multiple channels). On a slow +network this could mean several seconds of blocked commands. + +The fix splits the tick into three phases: + +1. Under lock: read config, determine which channels need a broadcast, + snapshot `feeds` and `seen`. +2. Outside lock: fetch headlines using the snapshot. HTTP happens here. +3. Under lock: write seen GUIDs, update `last_broadcast`, save to disk. + +The `seen` set is updated between channels in step 3 to prevent the same +article from being posted to multiple channels in the same tick cycle. + +`!headlines` still holds the lock during its fetch since it is a single +user-triggered request. The wait is acceptable for interactive commands. + +--- + +## Weather + +Weather data comes from `wttr.in` using the JSON format `?format=j1`. No +API key is required. The response includes current conditions, hourly data +for 3 days, and nearest area information. + +`_format_location()` builds a clean location string using a country code +lookup. "United States of America" becomes "US", etc. If the country is not +in the lookup dict, the full country name is used as-is. + +`!forecast` uses `day["hourly"][4]` for the weather description. Index 4 +corresponds to roughly midday (wttr.in hourly data is in 3-hour intervals +starting at midnight, so index 4 = 12:00). This gives a more representative +daily description than midnight or early morning conditions. + +--- + +## Feed defaults + +Nine feeds are hardcoded in `_DEFAULT_DATA`. They are written to the data +file on first run. After that, the feeds in the data file are used. To +add, remove, or disable a feed, edit the `feeds` array in the data file +while the bot is stopped. + +Each feed has an `enabled` field. Setting it to `false` excludes it from +all headline fetches without removing it from the file. + +--- + +## Per-channel configuration + +Each channel's config is stored separately in `data["channels"]`. `_chan()` +creates a new channel entry with `_DEFAULT_CHANNEL` defaults if it does not +exist. Channel settings are modified via `!news interval` and `!news count`. +These write directly to the data dict and save immediately. + +--- + +## Known issues and tradeoffs + +**No feed management via IRC.** Feeds can only be added or modified by +editing the JSON file directly. There are no IRC commands for feed management. +This was a deliberate simplicity choice. + +**RSS fetches can fail silently.** If a feed is unreachable or returns bad +XML, the error is logged via `log.warning()` but not reported in IRC. The +tick continues with the remaining feeds. This prevents a single broken feed +from affecting all others. + +**Seen list is global, not per-channel.** An article posted in one channel +will never be posted in another channel either. If you want different channels +to receive the same articles independently, this would require reworking the +seen list to be per-channel. + +**last_broadcast is stored in UTC epoch seconds.** The interval check is +`now - last_broadcast < interval_sec`. This is wall-clock time, not adjusted +for timezones. It works correctly as long as the bot's system clock is stable. |