path: root/dev-docs/osterman.md

# osterman - developer documentation

## Overview

Osterman is a full channel moderation bot. It handles automatic protection
(flood, caps, repeat, badword, clone, join flood, nick flood), manual
moderation commands, auto-modes on join, persistent ban tracking, user
event logging, and idle detection.

It is split across five plugin files that share state through `bot.memory`.

---

## Module structure

```
osterman/
  protect.py    - setup, passive protection, on_join, on_message, MODE/NICK/PART/QUIT handlers
  commands.py   - all user-facing moderation and config commands
  tracking.py   - SQLite event log, greeting, !seen, !info, !log
  idle.py       - idle detection tick, !afk, !idle
  help.py       - three-level help system (!help)
  helpstrings.py - help string constants shared across modules
  __init__.py   - empty
```

---

## Help system

Help strings live in `helpstrings.py` as plain string constants and dicts.
No Sopel imports, no decorators. `help.py` imports it via sys.path insertion
(same pattern as alfred):

```python
import os as _os, sys as _sys
_osterman_dir = _os.path.dirname(_os.path.abspath(__file__))
if _osterman_dir not in _sys.path:
    _sys.path.insert(0, _osterman_dir)
import helpstrings as h
```

Three-level hierarchy:
```
!help                       → TOPICS (list of topic names and commands)
!help <topic>               → topic line listing commands
!help <cmd>                 → single command description + usage
!help <topic> <cmd>         → same as above
```

`helpstrings.lookup(args)` handles all dispatch. Unknown args fall back to
the TOPICS line with an "no help for X" prefix.

---

## Shared state

`protect.py`'s `setup()` initializes all shared state in `bot.memory`:

```
bot.memory["os_db"]         - full database dict (loaded from JSON)
bot.memory["os_db_path"]    - path to the JSON db file
bot.memory["os_lock"]       - threading.Lock() for db writes
bot.memory["os_flood"]      - {(channel, nick): [timestamps]} flood tracking
bot.memory["os_repeat"]     - {(channel, nick): [(hash, timestamp)]} repeat tracking
bot.memory["os_join_flood"] - {(channel, host): [timestamps]} join flood (per channel)
bot.memory["os_nick_flood"] - {"user@host": [timestamps]} nick flood (network-wide)
bot.memory["os_last"]       - {(channel, nick): timestamp} last message time
```

`os_join_flood` is keyed by `(channel, host)` so join flood counts are isolated
per channel. A user joining multiple channels quickly does not bleed the flood
counter across channels. `os_nick_flood` is network-wide by design — NICK changes
apply across all channels.

The other modules access these keys with safe fallbacks in case of load order
differences. Sopel does not guarantee plugin load order, so every module that
touches `os_db` uses `bot.memory.get("os_db", {})` rather than a direct dict
access.

`tracking.py` derives its SQLite database path from `os_db_path` by replacing
the filename with `track.db`. This keeps both files in the same directory
without requiring a separate config key.

---

## Database structure

### JSON database (os_db)

```json
{
  "config": {"flood_threshold": 5, ...},
  "acl": ["nick1"],
  "whitelist": ["*!*@services.dal.net"],
  "blacklist": [],
  "exceptions": [],
  "channels": {
    "#example": {
      "config": {"flood_threshold": 3},
      "bans": [{"mask": "...", "added_by": "...", "timestamp": 0}],
      "filters": ["regex1"],
      "badwords": ["word1"],
      "exceptions": [],
      "autoop": [],
      "autovoice": [],
      "autohalfop": []
    }
  }
}
```

Config resolution is a three-level merge: hardcoded defaults → global config
override → channel config override. `_cfg(bot, channel)` returns the merged
result. `!set` in a channel writes to channel config; `!set` in PM writes to
global config (owner only).

### SQLite database (track.db)

Single `events` table: `id`, `nick`, `type`, `channel`, `detail`, `timestamp`.
Indexed on `nick` (NOCASE) and `timestamp DESC`.

Event types: `join`, `part`, `quit`, `kick`, `ban`, `nick`, `mute`, `unmute`,
`lock+m`, `lock+i`, `unlock-m`, `unlock-i`.

---

## Message routing

| Output | Method | Destination |
|--------|--------|-------------|
| `!seen`, `!idle <nick>` results | `bot.say` | channel (`trigger.sender`) |
| `!info`, `!log` results | `bot.say` | PM (`trigger.nick`) |
| `!ban list`, `!config`, `!filter list`, `!badword list` | `bot.notice` | nick |
| All other command feedback | `bot.notice` | nick |
| Greetings, idle warns, bad word warn | `bot.notice` | nick |

`_notice(bot, nick, text)` in `commands.py` and `help.py` calls
`bot.notice(text, nick)` — a true IRC NOTICE, not a PRIVMSG. This is the
standard approach for automated bot feedback that should not trigger other
bots or appear as a regular chat message.

---

## protect.py: passive protection

### on_join

Runs in order:
1. Blacklist check (global instant-ban, bypasses whitelist).
2. Whitelist/exception check. If exempt, still applies auto-modes.
3. Persistent ban reapply.
4. Clone detection.
5. Join flood detection (keyed per channel+host).
6. Auto-modes (`_apply_auto_modes()`).

Each step returns early if it takes action.

### on_message

Runs in order under `os_lock`:
1. Regex content filters.
2. Badword list.
3. Caps filter.
4. Repeat filter.
5. Flood control.

Each filter returns early on action. Flood is last because it is the least
specific.

### Privilege degradation

- Op: full enforcement (kick, ban, +b mode).
- Halfop: kick only (no +b, no re-op).
- None: tracking only (`os_last` updates, no enforcement).

### Takeover mitigation

`on_mode()` watches for `-o` events on ACL nicks. If the bot has op, it
immediately re-ops them.

---

## commands.py

### ban vs bankick

`!ban` and `!bankick` are functionally identical. They both ban and kick.
`!ban` also handles the `!ban list` subcommand. `!bankick` exists as an
explicit alias without the list subcommand ambiguity.

### tempban

Auto-unban is implemented with `threading.Timer`. The timer is not persisted.
If the bot restarts before the timer fires, the ban is never lifted. The expiry
is stored in the JSON db (`"expires"` key), so the data is there; it is just
not acted on at startup.

### !filter del / !badword del

Deleted by index, not by value. Use `!filter list` / `!badword list` to see
indices, then `del N`. This avoids ambiguity with regex special characters.

### !unban db cleanup

`cmd_unban` removes matching entries from the internal ban database using
`fnmatch.fnmatch(b["mask"], mask)` — tests whether the stored ban mask matches
the target being unbanned. The argument order matters: `fnmatch(pattern, string)`
where the stored mask is the pattern and the unban target is the string being
tested against it.

---

## tracking.py

### Greeting

On JOIN, checks for prior events before logging the join itself. If the nick
has any prior events: "Welcome back, nick!". If not: "Welcome to #channel, nick!".
Sent as IRC NOTICE to the joining user. Controlled by the `greet` config key.

### !seen scoping

In channel: checks only the current channel — live presence check against
`bot.channels[channel]`, then DB query with `AND channel = ?`. Reports in channel.

In PM: owner/ACL only. Cross-channel lookup across all joined channels and
the full event log. Reports as IRC NOTICE to nick. All other callers get
"use !seen in a channel".

### !info scoping

In channel: all DB queries scoped to `channel = ?`. Alias lookups (nick-change
events) are global because NICK events have no channel. Reports in PM.

In PM: owner/ACL only. All queries are global. Reports in PM. All other
callers get "use !info in a channel".

---

## idle.py

### Idle tick

`@plugin.interval(60)` checks all users in all channels against
`idle_warn_min` and `idle_kick_min` (minutes). Uses `os_last` timestamps.
Users with no `os_last` entry are treated as active.

---

## Known issues and tradeoffs

**tempban does not survive restarts.** The expiry timestamp is stored in the
JSON db but not checked on startup. Tempbans that expire while the bot is down
are never lifted.

**Clone detection is host-only.** Checks `trigger.host`, not `user@host`.
Users behind a BNC share the same host. Either raise `clone_limit` or whitelist
the BNC host.

**ACL management requires a channel.** `!acl` has `@plugin.require_chanmsg`.
Cannot manage the ACL from PM. Workaround: call from any channel the bot is in.

**`!ban list` only shows bot-tracked bans.** Bans set manually by channel ops
(not through osterman) do not appear. The list reflects the internal JSON db,
not the IRC +b list.

---

## Config

| Key | Description |
|-----|-------------|
| `[osterman] db_path` | Path to JSON database (auto-created if missing) |