ttrpg-npc 1.0.0__tar.gz

ttrpg_npc-1.0.0/.claude/settings.local.json

@@ -0,0 +1,16 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(uv run *)",
+      "Bash(git add *)",
+      "Bash(git restore *)",
+      "Bash(git commit -m ' *)",
+      "Bash(git push *)",
+      "Bash(curl -s --max-time 2 http://localhost:11434/api/version)",
+      "Bash(ollama list *)",
+      "Read(//home/kenneth/.local/share/npc/voices/**)",
+      "Bash(git -C /home/kenneth/documents/github/npc log --oneline -5)",
+      "Bash(grep -n \"^#\\\\{1,3\\\\} \\\\|!\\\\[\\\\|badge\\\\|shields.io\\\\|uv sync\\\\|pip install\\\\|git clone\" README.md)"
+    ]
+  }
+}

ttrpg_npc-1.0.0/.gitignore

@@ -0,0 +1,10 @@
+.venv/
+__pycache__/
+*.pyc
+dist/
+.pytest_cache/
+.ruff_cache/
+
+# Personal campaign content (characters, logbooks, session transcripts).
+# Remove this line if you want to version your campaigns.
+campaigns/

ttrpg_npc-1.0.0/ARCHITECTURE.md

@@ -0,0 +1,102 @@
+# Architecture
+
+`npc` is a fully-offline voice agent that role-plays a TTRPG NPC: hold a
+key, speak, and a local LLM answers aloud in character. This document is the
+map for people who want to read or change the code.
+
+## The pipeline
+
+```mermaid
+flowchart LR
+    PTT[push-to-talk\nevdev hotkey] --> REC[Recorder\nsounddevice]
+    REC --> STT[faster-whisper\nsv/en auto-detect]
+    STT --> PROMPT[prompt assembly\nsheet + lore + secrets\n+ logbook + GM notes]
+    PROMPT --> LLM[local LLM\nOllama / OpenAI-compatible]
+    LLM -->|token stream| SENT[sentence regrouper]
+    SENT --> CLEAN[dialogue sanitizer\nstrips narration & markers]
+    CLEAN --> TTS[Piper TTS]
+    TTS --> OUT[speaker]
+    LLM -. events .-> BUS
+    STT -. events .-> BUS
+    BUS[(event bus)] --> CLI[terminal UI]
+    BUS --> WS[overlay websocket\nOBS / table display]
+```
+
+Everything runs on one machine. Nothing ever calls the internet.
+
+## Threads and state
+
+Four threads, coordinated by `NPCApp` (`src/npc/app.py`), a state machine
+`IDLE → RECORDING → PROCESSING → SPEAKING`:
+
+1. **Main** — the `prompt_toolkit` REPL (`gm>`), where typed lines are
+   out-of-character GM instructions and `/commands`.
+2. **Hotkey listener** — evdev key-down/key-up (terminals don't deliver
+   key-up, which is why this exists). Pressing the key while the NPC speaks
+   is barge-in: playback stops, recording starts.
+3. **Worker** — a single queue serializes every turn (STT → LLM → TTS) and
+   every state-touching command, so nothing races.
+4. **Audio callback** — sounddevice's thread, which only copies samples.
+
+Voice input is *always* in-character player dialogue; the keyboard is
+*always* the GM. That split is the core interaction design.
+
+## Events, not prints
+
+The app never prints. It emits typed events (`src/npc/events.py`) —
+`PlayerSpoke`, `NpcReplied`, `SecretRevealRequested`, … — to one callback;
+the CLI renders them, tests assert on them, and the overlay broadcasts them
+as JSON. Two class-level flags form the exposure contract:
+
+- `dm_only` — never leaves the machine (everything about secrets).
+- `table_safe` — the only events broadcast when the overlay listens on the
+  LAN; GM notes, status, and errors are neither and stay home.
+
+## The campaign directory
+
+A campaign is a folder of plain markdown — the whole "database":
+
+```
+character.md      who the NPC is (first # heading = display name)
+characters/*.md   or several NPCs; /npc switches at the table
+adventure.md      GM background
+secrets.md        GM-gated clues — see below
+lore/             reference docs injected as established fact
+logbook.md        per-NPC session memory, written by the LLM
+sessions/         raw transcripts, appended crash-safe every turn
+config.toml       models, voices, hotkey — all optional
+```
+
+Every piece of per-NPC state (sheet, history, GM notes, logbook, secrets,
+lore, voice) lives on one `CharacterSlot` (`src/npc/roster.py`); switching
+NPCs swaps the slot. Memory is strictly per-NPC, in-session and across
+sessions.
+
+## Five guarantees
+
+1. **Fully offline.** Whisper, the LLM, and Piper are all local; the only
+   network listener is the overlay, loopback-bound unless you opt in.
+2. **Per-NPC isolation.** What players tell one NPC never reaches another —
+   not in the prompt, not in the logbook, not in the summaries.
+3. **Secrets can't leak.** Gated clues never enter the prompt until the GM
+   approves at the console; the model cannot reveal text it has never seen.
+   (`scripts/probe_secrets.py` pins the behavior against a real model.)
+4. **English out, always.** Players can speak Swedish; replies are forced to
+   English (the TTS voice's language) by prompt rules plus a detector with
+   a corrective re-ask — the prompt alone doesn't survive baiting.
+5. **Crash-safe writes.** Logbooks and secrets write via temp-file +
+   `os.replace`; transcripts append per turn. A crash costs nothing.
+
+## Seams
+
+Every hardware/model dependency sits behind a small Protocol —
+`Transcriber`, `Speaker`, `Recorder`, the LLM clients — and the entire test
+suite (280+ tests) runs against fakes: no GPU, no microphone, no server.
+`tests/test_app_pipeline.py` is the best starting point for how the pieces
+meet; `npc doctor` is the runtime mirror of the same seams.
+
+## Reading order
+
+`events.py` → `app.py` → `session/prompt.py` → `roster.py`, then whichever
+seam you care about (`stt.py`, `tts.py`, `llm.py`, `overlay.py`,
+`session/secrets.py`, `session/lore.py`).

ttrpg_npc-1.0.0/CHANGELOG.md

@@ -0,0 +1,59 @@
+# Changelog
+
+All notable changes to this project are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.0] — 2026-07-19
+
+First public release, published to PyPI as
+[`ttrpg-npc`](https://pypi.org/project/ttrpg-npc/) (the installed command is
+`npc`). Everything below works fully offline.
+
+### Voice pipeline
+
+- Push-to-talk (hold) and tap-to-talk (voice-activity detection) recording via
+  evdev — works on Wayland, barge-in interrupts a speaking NPC.
+- Speech-to-text with faster-whisper, Swedish/English auto-detect, with a
+  three-layer guard against whisper hallucinations on near-silence (energy
+  gate, no-speech segment filter, phantom-phrase blocklist).
+- Local LLM replies via Ollama (native) or any OpenAI-compatible server
+  (Jan, LM Studio, llama.cpp, vLLM), with timeouts and retry.
+- Text-to-speech with Piper (British English `en_GB-alba-medium` by default),
+  streamed sentence-by-sentence for ~3.4× faster time-to-first-sound.
+- Replies locked to in-character, voice-only English: narration/stage-direction
+  sanitizer and a foreign-language re-ask that survives direct baiting.
+
+### At the table
+
+- Multiple NPCs per campaign (`characters/`, `/npc <name>` switching), each
+  with its own Piper voice, memory, logbook, and session summaries.
+- GM-gated secrets: locked clue bodies never enter the prompt until the GM
+  approves at the console (`/yes`, `/no`, `/later`, `/reveal`, `/secrets`) —
+  the model mechanically cannot leak text it never saw.
+- Lore documents (`lore/`, .md/.txt/.pdf) injected as established fact, with
+  `[llm] num_ctx` support and doctor measuring the real prompt size.
+- OBS/table overlay over WebSocket with an opt-in LAN mode for a tablet at
+  the table; DM-only events never cross the network.
+- Crash-safe session transcripts and per-session logbook summaries.
+
+### Tooling & onboarding
+
+- `npc init` guided campaign wizard, campaign-aware quick-start on `npc run`,
+  grouped `/help`, did-you-mean command suggestions.
+- `npc doctor [--fix]` checks (and safely sets up) models, voices, audio, and
+  permissions; prints the exact `num_ctx` to configure.
+- `npc run --timings` per-stage latency instrumentation; last/avg in `/status`.
+- Debug helpers: `npc say`, `npc transcribe`.
+- Example campaign gallery (`examples/`), recorded demo, `ARCHITECTURE.md`.
+
+### Pre-release history
+
+Development milestones before this release were tracked internally as
+v1.x (timeouts, hallucination guard, streaming), v2.0 (voice-only hardening,
+tap-to-talk, overlay, doctor --fix, timings), v2.1 (multiple NPCs, gated
+secrets, UTF-8 + onboarding, lore), and v2.2 (LAN overlay, docs) — see
+`ROADMAP.md` for the map.
+
+[1.0.0]: https://github.com/seetee/npc/releases/tag/v1.0.0

ttrpg_npc-1.0.0/CLAUDE.md

@@ -0,0 +1,58 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project
+
+`npc` — a fully-offline terminal voice agent that role-plays a TTRPG NPC at the game table. Hold a push-to-talk key → mic records → faster-whisper transcribes (Swedish/English auto-detect) → a local LLM via Ollama replies in character → Piper speaks it aloud (British English, `en_GB-alba-medium`). Voice input is ALWAYS in-character player dialogue; typed terminal lines are out-of-character GM instructions to the LLM. Design doc: `/home/kenneth/.claude/plans/the-project-is-as-squishy-pond.md`.
+
+## Commands
+
+```bash
+uv sync --group dev --extra cuda   # install (cuda extra = GPU whisper; this machine has an RTX 3060)
+uv run pytest                      # unit tests — hardware-free, fakes injected
+uv run pytest -m integration      # real whisper/piper/ollama smoke tests (auto-skip if missing)
+uv run pytest tests/test_app_pipeline.py::test_voice_turn_end_to_end   # single test
+uv run ruff check .
+uv run npc init|doctor|run <campaign-dir>     # scaffold / setup-check / play
+uv run npc doctor --fix <dir>      # also offer safe fixes (model pull, voice download); sudo fixes stay print-only
+uv run npc run <dir> --timings --overlay      # per-turn stage timings / OBS-overlay server
+uv run npc say "text" <dir>  /  npc transcribe file.wav <dir>   # debug TTS / STT
+uv run python scripts/probe_narration.py      # re-check the prompt against real Ollama after prompt/model changes
+uv run python scripts/probe_secrets.py        # re-check the secrets gate (markers, leaks) after locked_block/deny_note changes
+```
+
+## Architecture
+
+The pipeline is orchestrated by `src/npc/app.py` (`NPCApp`) — a state machine (`IDLE → RECORDING → PROCESSING → SPEAKING`) with four threads: the prompt_toolkit REPL (main), an evdev hotkey listener (`hotkey.py`), a single worker that serializes STT → LLM → TTS turns via one `queue.Queue`, and sounddevice's callback thread. Push-to-talk during SPEAKING is barge-in (stops playback, starts recording); during PROCESSING it reports busy.
+
+Multiple NPCs (`src/npc/roster.py`): the roster = legacy `character.md` (keeps `logbook.md`) ∪ `characters/*.md` (each gets `logbooks/<stem>.md`); a `CharacterSlot` holds ALL per-NPC state (sheet, history, OOC notes, own logbook, summary turn buffer, `[tts.voices]` voice) and `NPCApp.active` points at one — `history`/`npc_name`/etc. are delegating properties. Memory is strictly separate per NPC, in-session AND across sessions: summaries are per-NPC from per-NPC turn buffers (dirty-gated so checkpoints skip unchanged NPCs); the session number is the max across all logbooks. `/npc` switches serialize through the worker queue (like bare OOC lines — so a note typed right after `/npc` lands on the right character); the switch assigns `self.speaker` BEFORE `self.active` for barge-in coherence. Speakers are cached per voice via the `make_speaker` factory; ALL PiperSpeakers must share ONE `AudioPlayer` (the persistent-PCM-stream crash guarantee).
+
+The app never prints: it emits typed events (`events.py` — `PlayerSpoke`, `NpcReplied`, `StateChanged`, `LogbookWritten`, …) to a single `on_event` callback; the CLI renders them with `format_event`, tests assert on event objects, and a future OBS overlay/web remote subscribes to the same stream (`StateChanged` is deliberately not rendered in the terminal). Emit events outside `self._lock`; `_emit` swallows subscriber exceptions so a broken subscriber can't kill a session.
+
+Key seams (all Protocol-typed, faked in `tests/test_app_pipeline.py`):
+- `audio/recorder.py` — `Recorder` protocol; `PushToTalkRecorder` (hold) and `VadRecorder` (`[hotkey] mode = "tap"`: tap to start, trailing silence or `stt.vad_max_seconds` fires `on_auto_stop(clip)` from a finalizer thread — the sounddevice callback never closes the stream, and an internal lock guarantees at-most-once delivery). `app.on_auto_stop` reuses the RECORDING→PROCESSING CAS so exactly one of {second tap, VAD} enqueues; tap-toggle semantics live in `cli._ptt_callbacks`, not the app. The pure `SilenceTracker` is the tunable part (threshold = `stt.silence_threshold_db`).
+- `overlay.py` — `[overlay] enabled`/`npc run --overlay`: one daemon thread + private asyncio loop serving `static/overlay.html` over HTTP and broadcasting events as JSON over `/ws`. Binds `[overlay] listen` (default 127.0.0.1; LAN is an explicit opt-in). The exposure contract is TWO ClassVar tiers gated in `cli.py:on_event`: `dm_only` events never reach the websocket anywhere, and beyond loopback ONLY `table_safe` events cross (the assignment block at the end of events.py is the reviewable table-facing surface — GM notes/status/errors stay home). `cli.py:overlay_announcement` prints the tablet URL + warning (`lan_address` resolves 0.0.0.0 via the UDP-connect trick). `publish()` hops threads via `call_soon_threadsafe`; `websockets.broadcast` drops slow clients so OBS can never stall a turn; event class/field renames break external consumers.
+- `stt.py` / `tts.py` / `llm.py` — `Transcriber`, `Speaker`, and the LLM clients. `llm.py:make_llm_client` picks the backend from `[llm].backend`: native `OllamaClient` (default) or `OpenAICompatClient` for Jan/LM Studio/llama.cpp/vLLM (tested via httpx MockTransport in `test_llm.py`). `stt.py` preloads pip-installed CUDA libs and falls back to CPU if CUDA breaks; resamples non-16kHz input. Whisper hallucinates YouTube outros/subtitle credits on near-silence, so three guards protect the LLM: an energy gate in `app.py:_handle_utterance` (`AudioClip.dbfs()` vs `stt.silence_threshold_db`, skips whisper entirely), a per-segment `no_speech_prob` filter (`stt.py:join_segments`), and the `PHANTOM_PHRASES` blocklist (`looks_like_hallucination`, en+sv, fires only when the WHOLE transcript is phantom).
+
+Streaming replies (`[llm] stream = true`, the default): `chat_stream()` on both clients → `session/sentences.py:iter_sentences` regroups tokens into ≥25-char sentences → `strip_decoration` per sentence (empties skipped, so a leading `*bows*` is never spoken) → `tts.py:say_stream`, where synthesis (worker thread) runs up to 3 sentences ahead of playback (internal thread); `stop()` cancels both, so barge-in needs no special casing. History/transcript/`NpcReplied` get the whole cleaned reply AFTER the stream ends — after a barge-in, only the sentences the table heard. `NpcReplyChunk` events stream for overlays (CLI renders them as None). Connection failures retry only before the first token; a server that rejects streaming (`StreamingNotSupported`) falls back to non-streaming `chat()`.
+
+Prompt assembly (`session/prompt.py`): system prompt is rebuilt every turn from `character.md` + `adventure.md` + logbook tail + accumulated OOC notes. OOC lines appear twice on purpose — inline in history (timing) and in the standing-instructions block (survives history trimming, `session/history.py`). The logbook (`session/logbook.py`) upserts one `## Session N — date` section per session (checkpoints/`/save`/`/end` re-summarize and replace, never duplicate); raw turns are appended crash-safe to `sessions/*-transcript.md`. Logbook writes are hardened against three failure modes — keep them that way: atomic temp-file+`os.replace` (crash can't truncate), `re.sub` with a lambda (backslashes in LLM output must stay literal), and demotion of `## Session N` lines inside summary bodies (would corrupt section parsing).
+
+GM-gated secrets (`session/secrets.py`): `secrets.md` (legacy) / `secrets/<stem>.md` (NOT `characters/` — the roster glob would load it as a character) holds `## id` + `hint:` + optional `mode: hesitate|deflect` + body. THE LEAK GUARANTEE IS MECHANICAL: only hints enter the prompt (`locked_block`); the body joins `revealed_block` after the GM approves — the model can't leak text it never saw. The model flags a locked topic by ending its stall line with `[CHECK:id]`; detection runs on RAW LLM output (`find_markers`) because `strip_decoration`'s `_BRACKETED` rule deletes the marker from everything spoken/recorded (which is also why the marker needs no TTS special-casing — but `strip_markers` must run before `extract_dialogue`, whose de-asterisk fallback would otherwise speak it, and `MarkerScrubber` filters `NpcReplyChunk`, whose raw feed reaches the table screen, holding back split markers across chunks). Flow: `_handle_markers` sets `slot.pending_secret` (per-slot, survives `/npc`) + emits `SecretRevealRequested`; `/yes [rider]`→ worker `_resolve_secret`: write-back `revealed:` (atomic), then a delivery turn (`_deliver_secret` → shared `_reply_turn`) with a ONE-SHOT GM instruction (never a standing note); `/no [rider]`→ standing deny note via `_add_ooc(emit=False)`, denial is session-scoped, id in `slot.denied_secrets`. Secret events are `dm_only=True` (ClassVar) and `cli.py:on_event` skips `overlay.publish` for them — ids/hints must NEVER reach the table websocket; `SecretPondering` (hesitate mode only) is the content-free overlay signal. Re-validate wording changes to `locked_block` with `scripts/probe_secrets.py`.
+
+Lore (`session/lore.py`): per-NPC reference docs — `lore/*` (legacy NPC; subdirs ignored) / `lore/<stem>/*` (`.md`/`.txt`/`.pdf`, pypdf lazy-imported, per-file failures collected not raised so a broken PDF can't block a session). Whole-document injection (tier 1, no RAG): `lore_block` goes into the prompt DIRECTLY after the character sheet — sheet+lore change only on `/reload`, keeping Ollama's KV prefix cache warm; mid-session-mutable blocks (secrets, logbook, OOC) come after. The budget is the context window: `[llm] num_ctx` (Ollama `options`, per-request → applies live on `/reload`; OpenAI-compat ignores it), `estimate_tokens` = chars//4, and TWO guards against Ollama's silent truncation of oversized prompts (which plays like the NPC forgetting its instructions): doctor's `npc_lore_checks` builds the real prompt and prints the `num_ctx` to set (`suggest_num_ctx`: 8192/16384/32768 with 25% headroom), and `app._warn_if_over_budget` warns once per session. Both gate on `config.llm.backend == "ollama"` (string compare — testable with fakes).
+
+A campaign directory (see `src/npc/templates/`) is the unit of play; `config.py` loads its optional `config.toml` over dataclass defaults. The NPC's display name is the first `# heading` of `character.md`. `/reload` re-reads the markdown files AND `config.toml`: the LLM model applies live (it's per-request), while `[stt]`/`[tts]`/`[hotkey]` changes are flagged as needing a restart (`app.py:_reload_config`).
+
+## Gotchas
+
+- Terminals never deliver key-up events — that's why hotkeys use evdev (works on Wayland, needs the user in the `input` group). `evdev.list_devices()` silently hides devices you lack permission for; `find_ptt_devices` compensates.
+- A held spacebar still types spaces into the REPL; `cli.py:_ptt_callbacks` snapshots/restores the prompt buffer around each press, and treats a press as typing (no recording) when the line already has text — so spaces typed inside `/say …` or an OOC note never trigger PTT. Both mitigations apply only to typing keys (`_key_types_text`); F12/right-ctrl/a grabbed USB button still allow PTT mid-line. A dedicated USB button should set `hotkey.grab = true`.
+- In `config.toml` templates, top-level keys must stay above `[sections]` (TOML semantics).
+- `sounddevice` needs the system package `libportaudio2` on Linux.
+- Replies must always be English (Alba voice) even for Swedish input — enforced in `ROLE_FRAMING`, tested in `test_prompt.py`. The prompt alone does NOT survive direct baiting ("svara på svenska!" flips the model 4/4), so `looks_foreign` (session/prompt.py; foreign function words + non-ASCII letters, ≥2 combined hits) is the real lock: non-stream replies get one corrective re-ask, a streaming reply that OPENS foreign is abandoned before any audio (`_NotEnglish` → non-stream fallback → re-ask), and a mid-stream foreign sentence is skipped from TTS only. Verified live: the re-ask converts real flips to English.
+- Replies must be voice-only: `ROLE_FRAMING` forbids narration/stage directions/assistant behavior (incl. a WRONG/RIGHT few-shot pair — 7B models need it), and `session/prompt.py:extract_dialogue` strips what small models slip through anyway (`*actions*`, `(parentheticals)`, speaker labels, quote wrapping) before the reply reaches history/transcript/TTS. When narration surrounds double-quoted speech, `_extract_quoted_dialogue` keeps only the quoted spans — the tell is terminal punctuation inside the quotes, so quoted titles (`"The Broken Crown"`) survive; the accepted false positive (quoting someone else loses framing words) is pinned in a test. Re-validate prompt changes with `scripts/probe_narration.py`.
+- `TurnCompleted` timing events are emitted every turn but rendered only via `npc run --timings` (like `StateChanged`, they're overlay material); `/status` shows last/avg turn seconds.
+
+License is AGPL-3.0-or-later — new dependencies must be compatible (Piper is GPL-3.0, fine).

ttrpg_npc-1.0.0/CONTRIBUTING.md

@@ -0,0 +1,59 @@
+# Contributing
+
+Thanks for your interest! This is a small project, but contributions are
+welcome — bug reports, campaign examples, and code alike.
+
+## Development setup
+
+Requirements: Linux, Python 3.11+, [uv](https://docs.astral.sh/uv/), and the
+system package `libportaudio2` (for sounddevice). A local
+[Ollama](https://ollama.com) install is only needed for integration tests and
+actual play.
+
+```bash
+git clone https://github.com/seetee/npc.git
+cd npc
+uv sync --group dev            # add `--extra cuda` for GPU whisper
+uv run npc --version
+```
+
+## Tests
+
+```bash
+uv run pytest                  # unit tests — hardware-free, fakes injected
+uv run pytest -m integration   # real whisper/piper/ollama smoke tests
+                               # (auto-skip when the services are missing)
+uv run ruff check .            # lint
+```
+
+Unit tests must stay hardware-free: the STT/LLM/TTS/recorder seams are all
+Protocol-typed and faked in `tests/test_app_pipeline.py` — follow that pattern
+for new features.
+
+## Prompt changes need probing
+
+The system prompt is only as good as its measured behavior against a real
+model. If you touch prompt wording, re-run the relevant probe against live
+Ollama and include the numbers in your PR:
+
+- `uv run python scripts/probe_narration.py` — after changes to
+  `ROLE_FRAMING` / narration handling in `src/npc/session/prompt.py`.
+- `uv run python scripts/probe_secrets.py` — after changes to
+  `locked_block` / `deny_note` in `src/npc/session/secrets.py`.
+
+## Architecture in one paragraph
+
+The app never prints: `NPCApp` (`src/npc/app.py`) emits typed events
+(`src/npc/events.py`) to a single callback; the CLI renders them, tests assert
+on them, and the overlay broadcasts them. One worker thread serializes
+STT → LLM → TTS turns. Read `ARCHITECTURE.md` for the full picture and
+`CLAUDE.md` for the invariants that must not break (per-NPC memory isolation,
+the secrets leak guarantee, the single shared `AudioPlayer`, DM-only event
+tiers on the overlay).
+
+## Pull requests
+
+- Keep `uv run pytest` and `uv run ruff check .` green.
+- New dependencies must be AGPL-3.0-or-later compatible.
+- By contributing you agree your work is licensed under the project license,
+  AGPL-3.0-or-later (see `LICENSE`).