@cad0p/pi-napkin 0.2.4-20260518.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +22 -0
- package/README.md +451 -0
- package/extensions/distill/README.md +86 -0
- package/extensions/distill/auto-setup.ts +392 -0
- package/extensions/distill/distill-prompt.md +58 -0
- package/extensions/distill/distill-prompt.ts +212 -0
- package/extensions/distill/distill-workspace.ts +1248 -0
- package/extensions/distill/index.ts +1828 -0
- package/extensions/distill/scripts/distill-wrapper.sh +1472 -0
- package/extensions/distill/scripts-paths.ts +21 -0
- package/extensions/distill/session-touched-files.ts +216 -0
- package/extensions/distill/should-distill-on-shutdown.ts +68 -0
- package/extensions/napkin-context/index.ts +259 -0
- package/package.json +71 -0
- package/skills/napkin/SKILL.md +44 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Michael Liv (upstream Michaelliv/pi-napkin)
|
|
4
|
+
Copyright (c) 2026 Pier Carlo Cadoppi (cad0p/pi-napkin fork)
|
|
5
|
+
|
|
6
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
7
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
8
|
+
in the Software without restriction, including without limitation the rights
|
|
9
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
10
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
11
|
+
furnished to do so, subject to the following conditions:
|
|
12
|
+
|
|
13
|
+
The above copyright notice and this permission notice shall be included in all
|
|
14
|
+
copies or substantial portions of the Software.
|
|
15
|
+
|
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
17
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
18
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
19
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
20
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
21
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
22
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,451 @@
|
|
|
1
|
+
# pi-napkin
|
|
2
|
+
|
|
3
|
+
π§» [Napkin](https://github.com/cad0p/napkin) integration for [pi](https://github.com/badlogic/pi-mono).
|
|
4
|
+
|
|
5
|
+
Gives a pi agent first-class access to an Obsidian-compatible knowledge vault, with automatic knowledge distillation that safely captures conversation context into notes as you work.
|
|
6
|
+
|
|
7
|
+
## Install
|
|
8
|
+
|
|
9
|
+
pi-napkin depends on the [`@cad0p/napkin`](https://github.com/cad0p/napkin) CLI. Install it first:
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
npm install -g @cad0p/napkin
|
|
13
|
+
# or: bun add -g @cad0p/napkin
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Then install the pi-napkin extension:
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
pi install npm:@cad0p/pi-napkin
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
### Requirements
|
|
23
|
+
|
|
24
|
+
- **bash 4+** β the auto-distill wrapper uses bash arrays, `local -n`, and other bash-4 features. macOS ships bash 3.2 by default; install a newer one via `brew install bash` (the wrapper resolves bash via its `#!/usr/bin/env bash` shebang, so it picks up homebrew's bash if `brew` is on `PATH`).
|
|
25
|
+
- **`timeout(1)` from coreutils** β used to bound the agent's wall-clock budget (`distill.maxDurationMinutes`). Linux distros ship it as `timeout`; macOS ships it as `gtimeout` after `brew install coreutils`. The wrapper detects either binary and falls back fast with a helpful error if neither is present.
|
|
26
|
+
- **git 2.20+** β needed for `git worktree`, `merge-base --is-ancestor`, and `symbolic-ref --short HEAD`.
|
|
27
|
+
- **A `pi` configured with at least one model provider** β auto-distill spawns `pi -p` against the model in `distill.model.{provider,id}`. Manual `/distill` reuses the parent session's provider.
|
|
28
|
+
|
|
29
|
+
<details>
|
|
30
|
+
<summary>Pre-release / dev installs</summary>
|
|
31
|
+
|
|
32
|
+
- Pre-release (calver snapshots from `main`, published to npm `@next` on every push):
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
pi install npm:@cad0p/pi-napkin@next
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
pi pins npm installs with an explicit tag or version β `pi update` won't auto-bump this. Re-run the install to pick up newer `@next` builds.
|
|
39
|
+
|
|
40
|
+
- Install from source for local development:
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
pi install git:github.com/cad0p/pi-napkin
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
</details>
|
|
47
|
+
|
|
48
|
+
## What you get
|
|
49
|
+
|
|
50
|
+
Two pi extensions, one skill, two slash commands, one agent tool.
|
|
51
|
+
|
|
52
|
+
### Extensions
|
|
53
|
+
|
|
54
|
+
| Extension | What it does |
|
|
55
|
+
|---|---|
|
|
56
|
+
| `napkin-context` | On session start, injects the vault overview into the agent's context. Registers `kb_search` + `kb_read` tools. |
|
|
57
|
+
| `napkin-distill` | Automatic knowledge distillation. Runs on a timer and at shutdown, forks the conversation, and asks a cheap model to extract structured notes into the vault. |
|
|
58
|
+
|
|
59
|
+
### Skill
|
|
60
|
+
|
|
61
|
+
The [`napkin` skill](skills/napkin/SKILL.md) gives the agent a full CLI reference β all commands, flags, and patterns.
|
|
62
|
+
|
|
63
|
+
### Slash commands
|
|
64
|
+
|
|
65
|
+
- `/distill` β Trigger a manual distill now. Works in any vault; does not require git.
|
|
66
|
+
- `/distill-auto-this-session` β Turn the automatic timer off / on for the current session. Persists across pi restarts.
|
|
67
|
+
- `/distill-status` β Show active background distill processes for the current vault.
|
|
68
|
+
|
|
69
|
+
### Agent tool
|
|
70
|
+
|
|
71
|
+
- `napkin_distill_status` β JSON version of `/distill-status`, for the agent to query programmatically before making concurrent vault edits.
|
|
72
|
+
|
|
73
|
+
## Auto-distill requires subdir vault layout
|
|
74
|
+
|
|
75
|
+
Auto-distill (interval + shutdown) uses git worktrees for concurrency safety.
|
|
76
|
+
That requires the vault to use napkin's **subdir layout** β the config in a
|
|
77
|
+
`.napkin/` subdir distinct from the content root:
|
|
78
|
+
|
|
79
|
+
```
|
|
80
|
+
my-vault/
|
|
81
|
+
.napkin/
|
|
82
|
+
config.json β napkin config here, with `"vault": { "root": ".." }`
|
|
83
|
+
NAPKIN.md
|
|
84
|
+
changelog/
|
|
85
|
+
daily/
|
|
86
|
+
β¦
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
`napkin init` creates this layout by default, so freshly created vaults work out
|
|
90
|
+
of the box. If your vault uses the **legacy embedded layout** (config at
|
|
91
|
+
`<vault>/config.json` with no `.napkin/` subdir, where `configPath ===
|
|
92
|
+
contentPath`), you'll see a migration notification at session start and
|
|
93
|
+
auto-distill will be disabled for the session. Legacy layout continues to
|
|
94
|
+
work for manual `/distill` (which doesn't need concurrency safety) and for
|
|
95
|
+
napkin CLI commands generally β only auto-distill requires the subdir
|
|
96
|
+
layout.
|
|
97
|
+
|
|
98
|
+
### Why
|
|
99
|
+
|
|
100
|
+
Worktree-based concurrency relies on the worktree having a `.napkin/`
|
|
101
|
+
subdir post-checkout, so the wrapper's `git add -A` + `git commit` +
|
|
102
|
+
`git merge --squash` operations on the worktree see the isolated
|
|
103
|
+
vault layout. On a subdir-layout vault, the branch tracks
|
|
104
|
+
`.napkin/config.json`, so every checked-out worktree has the
|
|
105
|
+
`.napkin/` subdir. On a legacy-embedded vault, the branch has no
|
|
106
|
+
`.napkin/` subdir at all (`.napkin/` IS the vault), so the worktree
|
|
107
|
+
has nothing to operate on β the wrapper would silently produce empty
|
|
108
|
+
commits, and the per-distill napkin shim's `--vault $WORKTREE` would
|
|
109
|
+
point at a directory napkin doesn't recognize as a vault. The
|
|
110
|
+
concurrency guarantee silently degrades to nothing.
|
|
111
|
+
|
|
112
|
+
### Where worktrees live
|
|
113
|
+
|
|
114
|
+
Active distill worktrees are placed under
|
|
115
|
+
`$XDG_CACHE_HOME/napkin-distill/<vault-hash>/<id>/` (typically
|
|
116
|
+
`~/.cache/napkin-distill/β¦`), outside your vault. `<vault-hash>` is
|
|
117
|
+
`sha256(contentPath).slice(0, 16)` so worktrees from multiple vaults never
|
|
118
|
+
collide. External placement avoids:
|
|
119
|
+
|
|
120
|
+
- **Cloud-sync pollution** β OneDrive/Dropbox don't respect `.gitignore`;
|
|
121
|
+
in-vault worktrees would upload tens of MB per distill spawn.
|
|
122
|
+
- **Filesystem-walker pollution** β Obsidian plugins and `find` descend
|
|
123
|
+
into worktrees and see N full vault copies for N concurrent distills.
|
|
124
|
+
- **Autocommit-cron noise** β gitlinks can surface in `git status` under
|
|
125
|
+
some command sequences; external placement eliminates the surface.
|
|
126
|
+
|
|
127
|
+
Inspect active distills with `/distill-status`. If you ever need to nuke
|
|
128
|
+
all distill state for a vault (stuck lock, etc.):
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
rm -rf ~/.cache/napkin-distill/<hash>/
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
Safe β anything valuable is either already committed to main or was
|
|
135
|
+
never going to commit.
|
|
136
|
+
|
|
137
|
+
### Migration from legacy layout
|
|
138
|
+
|
|
139
|
+
```bash
|
|
140
|
+
# From your vault directory (e.g. ~/.napkin or wherever configPath == contentPath)
|
|
141
|
+
mkdir .napkin
|
|
142
|
+
mv config.json .napkin/config.json
|
|
143
|
+
# Edit .napkin/config.json and add at the top level:
|
|
144
|
+
# "vault": { "root": ".." }
|
|
145
|
+
# After editing, reload pi (or /quit and restart).
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Verify with `napkin vault --json` β the `path` field should point at
|
|
149
|
+
`<vault>/.napkin/` (that's the new configPath) and napkin should still
|
|
150
|
+
find all your notes.
|
|
151
|
+
|
|
152
|
+
## Vault resolution
|
|
153
|
+
|
|
154
|
+
Both extensions use napkin's built-in vault resolution. The resolution order is:
|
|
155
|
+
|
|
156
|
+
1. **Local project vault** β walk up from cwd looking for `.napkin/` (or `.obsidian/.napkin/`)
|
|
157
|
+
2. **Global fallback** β read `$XDG_CONFIG_HOME/napkin/config.json` (defaults to `~/.config/napkin/config.json`)
|
|
158
|
+
3. **Bare vault** β create a new vault at cwd as a last resort
|
|
159
|
+
|
|
160
|
+
```json
|
|
161
|
+
// ~/.config/napkin/config.json
|
|
162
|
+
{
|
|
163
|
+
"vault": "~/path/to/vault"
|
|
164
|
+
}
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
Local project vaults take priority when present.
|
|
168
|
+
|
|
169
|
+
### Migrating from `~/.pi/agent/napkin.json`
|
|
170
|
+
|
|
171
|
+
If you previously configured your vault in `~/.pi/agent/napkin.json`, move it to the new location:
|
|
172
|
+
|
|
173
|
+
```bash
|
|
174
|
+
mkdir -p ~/.config/napkin
|
|
175
|
+
cp ~/.pi/agent/napkin.json ~/.config/napkin/config.json
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
## Auto-distill
|
|
179
|
+
|
|
180
|
+
Auto-distill is the core feature. It runs in the background without prompting the user, periodically forking your pi session and asking a cheap model to extract knowledge into the vault. It's off by default.
|
|
181
|
+
|
|
182
|
+
### Enable it
|
|
183
|
+
|
|
184
|
+
```bash
|
|
185
|
+
napkin --vault ~/path/to/vault config set --key distill.enabled --value true
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
Or edit `<vault>/.napkin/config.json` directly:
|
|
189
|
+
|
|
190
|
+
```jsonc
|
|
191
|
+
{
|
|
192
|
+
"distill": {
|
|
193
|
+
"enabled": true,
|
|
194
|
+
"intervalMinutes": 60,
|
|
195
|
+
"onShutdown": true,
|
|
196
|
+
"model": { "provider": "kiro", "id": "claude-sonnet-4-6" }
|
|
197
|
+
}
|
|
198
|
+
}
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
### Config keys
|
|
202
|
+
|
|
203
|
+
| Setting | Default | Description |
|
|
204
|
+
|---|---|---|
|
|
205
|
+
| `distill.enabled` | `false` | Master switch. When false, nothing auto-distill related happens. |
|
|
206
|
+
| `distill.intervalMinutes` | `60` | Timer interval. |
|
|
207
|
+
| `distill.maxDurationMinutes` | `10` | Maximum wall-clock duration a detached distill subprocess is allowed before the parent's poll loop declares it stuck and force-cleans its worktree. Values `<= 0` or non-finite silently fall back to the 10-minute default so a bad config can't disable the timeout entirely. Lower this for short-session vaults where a 10-minute hang is unacceptable; raise it for vaults with large merge windows or slow providers. |
|
|
208
|
+
| `distill.onShutdown` | `true` | Also run a final distill at pi shutdown, to capture anything the interval missed. |
|
|
209
|
+
| `distill.model.provider` | `"anthropic"` | Model provider for the distill subprocess. |
|
|
210
|
+
| `distill.model.id` | `"claude-sonnet-4-6"` | Model ID. Prefer a cheap, fast model β distill is automated, not interactive. |
|
|
211
|
+
|
|
212
|
+
### How it works
|
|
213
|
+
|
|
214
|
+
When enabled, on session start the extension:
|
|
215
|
+
|
|
216
|
+
1. Runs `ensureVaultReadyForAutoDistill` on the vault (see below) β git-inits if needed and scaffolds `.gitignore`.
|
|
217
|
+
2. Sweeps stale distill worktrees left by crashed pi instances (`cleanupStaleWorktrees`).
|
|
218
|
+
3. Arms a timer (`intervalMinutes`) that spawns a detached `pi -p` subprocess on tick.
|
|
219
|
+
4. On shutdown (unless `distill.onShutdown` is false or the session file is already captured), spawns one final distill.
|
|
220
|
+
|
|
221
|
+
Each distill subprocess gets its own isolated copy of the vault via `git worktree add`. See [Concurrency](#concurrency) below.
|
|
222
|
+
|
|
223
|
+
The distill subprocess runs a prompt that asks the model to:
|
|
224
|
+
- Learn the vault structure via `napkin overview` and `_about.md` files
|
|
225
|
+
- Read the templates via `napkin template list`
|
|
226
|
+
- Search for existing notes on a topic before creating duplicates
|
|
227
|
+
- Create or append to notes as appropriate
|
|
228
|
+
- Add `[[wikilinks]]` to related notes
|
|
229
|
+
- Tag superseded notes with `supersedes: ["path/to/old.md"]` frontmatter for a future janitor to archive
|
|
230
|
+
|
|
231
|
+
### Auto-init on first use
|
|
232
|
+
|
|
233
|
+
When you enable `distill.enabled: true` on a vault that isn't a git repo, pi-napkin auto-initializes it on the next session start:
|
|
234
|
+
|
|
235
|
+
1. Runs `git init`.
|
|
236
|
+
2. Scaffolds `.gitignore` (excludes `.napkin/distill/` β the per-worktree session fork). Distill worktrees themselves live outside the vault (see [Where worktrees live](#where-worktrees-live)), so `.gitignore` doesn't need to exclude them. No `.gitattributes` is written β the agent-driven merge architecture has no driver to register.
|
|
237
|
+
3. Commits everything as `napkin: initial vault commit (auto-distill setup)`.
|
|
238
|
+
4. Notifies you once, with instructions to undo (`rm -rf <vault>/.git`) or opt out (`distill.enabled: false`).
|
|
239
|
+
|
|
240
|
+
Auto-distill requires git **and** the subdir vault layout (see [Auto-distill requires subdir vault layout](#auto-distill-requires-subdir-vault-layout)). Manual `/distill` requires neither β it just spawns a detached `pi -p` with the session forked to a temp dir, no worktree.
|
|
241
|
+
|
|
242
|
+
### Running an auto-distill loop
|
|
243
|
+
|
|
244
|
+
Once configured, nothing further is required: open any pi session in a directory under the vault and the timer arms automatically. You'll see a `distill: Xm..s` countdown in the status bar and a one-time notice when a distill begins, completes, or fails.
|
|
245
|
+
|
|
246
|
+
To pause for the current session only (e.g., you're drafting sensitive content you don't want captured):
|
|
247
|
+
|
|
248
|
+
```
|
|
249
|
+
/distill-auto-this-session off
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
Toggles back on with `/distill-auto-this-session on`. State persists across pi restarts of that same session.
|
|
253
|
+
|
|
254
|
+
## Concurrency
|
|
255
|
+
|
|
256
|
+
Running multiple pi sessions against the same vault (for example, autonomous agent fleets spawning many sessions in parallel) or having Obsidian open while a distill is running would race on file writes. pi-napkin uses git worktrees to make this safe.
|
|
257
|
+
|
|
258
|
+
### Worktree per distill
|
|
259
|
+
|
|
260
|
+
Each auto-distill invocation (interval fire or shutdown) creates its own temporary branch and worktree under `$XDG_CACHE_HOME/napkin-distill/<vault-hash>/<branch-suffix>/` (typically `~/.cache/napkin-distill/β¦`). pi runs at the **parent session's cwd** (not the worktree) so the system prompt's `Current working directory:` line stays byte-identical to the parent's, preserving prompt-cache hits. Vault writes from the agent's bash tool are routed back to the worktree by a per-distill napkin shim at `<worktree>/.napkin/distill/bin/napkin` that injects `--vault <worktree>` into every napkin invocation. See [Where worktrees live](#where-worktrees-live) for why the worktree dir is external to the vault.
|
|
261
|
+
|
|
262
|
+
### How distill resolves conflicts
|
|
263
|
+
|
|
264
|
+
The distill agent owns the full lifecycle: produce content β merge default branch into the distill branch β squash to default β push to origin (if configured) β clean up. The wrapper does NOT invoke a per-file merge driver; the model that wrote the content also resolves any conflicts that surface when the worktree is reconciled with main.
|
|
265
|
+
|
|
266
|
+
Flow:
|
|
267
|
+
|
|
268
|
+
1. **Wrapper** sets up the worktree, installs the per-distill napkin shim, and spawns one `pi -p $PROMPT` invocation under `timeout ${maxDurationMinutes}m`. The prompt instructs the agent to walk steps 1β10 (distill β merge β squash β push β cleanup).
|
|
269
|
+
2. **Agent** distills content into the worktree, then runs `git -C <worktree> merge --no-edit <default>`. If conflicts surface, it edits each file in place using the conversation history as context.
|
|
270
|
+
3. **Agent** squash-merges into the vault's default branch (`git -C <vault> merge --squash <distill-branch>` then `git commit`).
|
|
271
|
+
4. **Agent** pushes to `origin/<default>` if origin is configured. On non-fast-forward failures it recovers via `git pull --no-rebase origin <default>` then re-pushes. It NEVER uses `--force` or `--force-with-lease`.
|
|
272
|
+
5. **Wrapper** post-validates: no conflict markers in tracked `*.md`, vault HEAD on default branch, push (if attempted) didn't rewrite shared history. Writes an outcome sidecar and force-cleans the worktree + distill branch.
|
|
273
|
+
|
|
274
|
+
The full prompt template lives at `extensions/distill/distill-prompt.md`. The wrapper bounds the agent's wall-clock with `timeout(1)`; everything else (retry policy, network handling, conflict-resolution shape) is the agent's call.
|
|
275
|
+
|
|
276
|
+
### Outcome classes
|
|
277
|
+
|
|
278
|
+
Each distill produces a sidecar at `<vault>/.napkin/distill/errors/<timestamp>-<pid>-<branch>.outcome` whose first line is a machine-readable class:
|
|
279
|
+
|
|
280
|
+
| Class | When | UI severity |
|
|
281
|
+
|---|---|---|
|
|
282
|
+
| `merged-content` | Agent produced content, integrated, squashed, and pushed to origin (or origin not configured). | info β |
|
|
283
|
+
| `merged-local` | Agent integrated + squashed, but origin/`<default>` is configured AND local `<default>` is ahead of `origin/<default>` (push didn't land). | warning β β "distilled but not pushed" |
|
|
284
|
+
| `no-content` | Agent produced nothing (genuine no-op β selective filter), or committed to the distill branch but skipped squash (default branch never moved). | warning β |
|
|
285
|
+
| `failed:<reason>` | Wrapper validation rejected the agent's output, or the agent didn't complete. | error β |
|
|
286
|
+
|
|
287
|
+
Known `failed:<reason>` codes:
|
|
288
|
+
|
|
289
|
+
| Reason | Meaning | Recovery |
|
|
290
|
+
|---|---|---|
|
|
291
|
+
| `markers-after-agent-exit` | Conflict markers (`<<<<<<<` / `=======` / `>>>>>>>`) found in tracked `*.md` files that the agent did not have at distill start. The agent left an unfinished merge. | The squash commit may already be on `<default>`. Inspect with `git show HEAD`, then `git revert HEAD --no-edit` to undo cleanly. The dangling distill branch is recoverable from `git reflog` for ~2 weeks. |
|
|
292
|
+
| `pre-existing-markers` | Markers were present in the same files BEFORE the agent ran. Validation refuses to misattribute them. | Resolve the pre-existing markers in the vault yourself, then re-run distill. |
|
|
293
|
+
| `internal-validator-error` | The wrapper's post-distill marker validator could NOT run (e.g. `mktemp` failed because of a full disk or locked-down `TMPDIR`), so the vault was never scanned after the agent exited. | Inspect the vault manually for unresolved `<<<<<<< / ======= / >>>>>>>` markers before relying on the squash commit. If clean, the squash is keepable; otherwise `git -C <vault> revert HEAD --no-edit`. Distill content is recoverable from `git reflog` for ~90 days. |
|
|
294
|
+
| `head-not-on-default` | Vault HEAD is not on the default branch after the agent exited (detached HEAD or different branch). | Manually `git checkout <default>` after confirming nothing is in flight. The distill branch lives in `git reflog` if you want to recover its content. |
|
|
295
|
+
| `divergent-history` | After the agent's push, `origin/<default>` and local `<default>` diverged in a way that doesn't look like a fast-forward (unexpected; benign third-party push or β defensively β a force-push). | Inspect `origin/<default>` vs local. If a teammate landed a commit during distill, `git pull --no-rebase` to integrate. |
|
|
296
|
+
| `agent-exit-nonzero` | The agent's `pi -p` invocation exited non-zero with no other diagnostic. | See the error log next to the sidecar for the agent's stderr. |
|
|
297
|
+
| `agent-timeout` | `timeout(1)` killed the agent after `distill.maxDurationMinutes` minutes. | Bump `distill.maxDurationMinutes` if your conversations routinely produce long distills, or check the error log for what the agent was doing when it timed out. |
|
|
298
|
+
|
|
299
|
+
A missing sidecar AND missing error log means the wrapper itself was killed before writing either (SIGKILL, OOM). The JS-side poller surfaces this as "distill terminated abnormally".
|
|
300
|
+
|
|
301
|
+
### Salvage when validation fails
|
|
302
|
+
|
|
303
|
+
The salvage path is deliberately narrow: the wrapper janitors the **worktree and distill branch**, never the main vault's commit history. If validation fails:
|
|
304
|
+
|
|
305
|
+
- Force-remove the worktree (`git worktree remove --force` + path-safety guard, falling back to `rm -rf` only if the worktree path is a descendant of the resolved cache root).
|
|
306
|
+
- Force-delete the distill branch (`git branch -D`).
|
|
307
|
+
- Write a `failed:<reason>` sidecar with a recovery hint that points at the corrupt squash commit if one landed on `<default>`.
|
|
308
|
+
- Exit 1.
|
|
309
|
+
|
|
310
|
+
The wrapper deliberately does NOT `git reset --hard` or otherwise rewrite the vault's commit history. If the agent's squash landed corrupt content, the user's `git revert HEAD --no-edit` (forward-only) is preferred over a destructive reset that could clobber concurrent edits the user made between distill spawn and validation. The recovery hint in the failed sidecar names the exact command.
|
|
311
|
+
|
|
312
|
+
### Linear history
|
|
313
|
+
|
|
314
|
+
Successful distill lifecycles produce exactly one squash commit on `main`, with a one-line summary the agent generates from the distill content. This keeps history clean and makes it easy to revert a bad distill with `git revert <sha>`.
|
|
315
|
+
|
|
316
|
+
## Agent visibility
|
|
317
|
+
|
|
318
|
+
When a background distill completes a squash-merge that touches files you've also written this session, pi-napkin posts a one-line notice into the conversation as a custom session message so the agent knows its recent writes may have been merged or overwritten:
|
|
319
|
+
|
|
320
|
+
```
|
|
321
|
+
β οΈ Background napkin distill is editing files you've also touched: notes/foo.md.
|
|
322
|
+
Recent writes to these files may be overwritten or merged automatically at distill
|
|
323
|
+
completion; consider re-reading before further edits.
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
Key properties:
|
|
327
|
+
- **Trigger**: per distill completion (when the wrapper's squash-merge has landed and the worktree is gone), not per agent turn. Files actually changing in the parent's view is the right moment to surface the overlap.
|
|
328
|
+
- **Channel**: posted via `appendCustomMessageEntry` with `customType: "napkin-distill-overlap"`. Lives in session history (so distill subprocesses inherit it cleanly via session-fork) and is displayed in the TUI so you see it too.
|
|
329
|
+
- **Cache parity**: custom messages land at the END of the message array, so Anthropic-style prompt caching's prefix stays byte-identical. The notice itself becomes a one-time cache write rather than the recurring cache-bust the previous per-turn `appendSystemPrompt` mechanism produced.
|
|
330
|
+
- **Frequency**: bounded by the per-distill-completion trigger β typically ~5β12 messages/day for an active session, only when actual file overlap exists.
|
|
331
|
+
- **Cursor**: each completion only considers entries added since the previous completion. On a fresh session this starts at 0; on a resumed session it starts at the pre-resume entry count, so resume doesn't surface stale notices for files written in earlier pi processes.
|
|
332
|
+
|
|
333
|
+
The session-touched-files detector reimplements pi's internal `extractFileOpsFromMessage` (not exported from pi) and tracks `write`, `edit`, and bash-redirection-style writes. A companion version-check test (`session-touched-files.version-check.test.ts`) pins pi's upstream utility so we get an explicit test failure if pi renames / removes it.
|
|
334
|
+
|
|
335
|
+
## Vault setup
|
|
336
|
+
|
|
337
|
+
### New vault
|
|
338
|
+
|
|
339
|
+
```bash
|
|
340
|
+
mkdir my-vault && cd my-vault
|
|
341
|
+
napkin init
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
Then edit `.napkin/config.json` to set `distill.enabled: true`. On first pi session in the vault, auto-distill will auto-init git for you.
|
|
345
|
+
|
|
346
|
+
### Existing Obsidian vault
|
|
347
|
+
|
|
348
|
+
Just `cd` into it and run `pi`. napkin will detect `.obsidian/` and treat it as a vault. The `.napkin/` config dir lives alongside `.obsidian/`.
|
|
349
|
+
|
|
350
|
+
If you want auto-distill on an existing Obsidian vault:
|
|
351
|
+
|
|
352
|
+
1. Set `distill.enabled: true` in `.napkin/config.json` (create the file if missing).
|
|
353
|
+
2. Start a pi session in the vault.
|
|
354
|
+
3. On session start, pi-napkin prompts to auto-init git if the vault isn't already one.
|
|
355
|
+
|
|
356
|
+
If your vault IS already a git repo (common for "vault as project"), pi-napkin just adds the `.gitignore` scaffold and moves on.
|
|
357
|
+
|
|
358
|
+
## Troubleshooting
|
|
359
|
+
|
|
360
|
+
### "No vault in cwd"
|
|
361
|
+
|
|
362
|
+
Some command or tool couldn't resolve a vault from the current directory. Either:
|
|
363
|
+
- Run from a directory under a vault (any ancestor with `.napkin/` or `.obsidian/`)
|
|
364
|
+
- Set the global fallback in `~/.config/napkin/config.json`
|
|
365
|
+
- Pass `--vault <path>` to the napkin CLI
|
|
366
|
+
|
|
367
|
+
### Auto-distill stopped working
|
|
368
|
+
|
|
369
|
+
Check `/distill-status` to see if there are stale / dead worktrees. The next session start will sweep them automatically (`cleanupStaleWorktrees`). If that doesn't help, nuke the per-vault cache dir:
|
|
370
|
+
|
|
371
|
+
```bash
|
|
372
|
+
rm -rf ~/.cache/napkin-distill/<vault-hash>/
|
|
373
|
+
# Or, if you're not sure which hash matches your vault, the nuclear option:
|
|
374
|
+
rm -rf ~/.cache/napkin-distill/
|
|
375
|
+
```
|
|
376
|
+
|
|
377
|
+
Safe β anything valuable is either already committed to main or was never going to commit.
|
|
378
|
+
|
|
379
|
+
### Distill keeps failing (`failed:<reason>`)
|
|
380
|
+
|
|
381
|
+
Outcome sidecar: `<vault>/.napkin/distill/errors/<ISO-timestamp>-<pid>-<branch-hash>.outcome`. First line is the class string (`failed:<reason>`); remaining lines are a human-readable recovery hint.
|
|
382
|
+
|
|
383
|
+
Companion error log (if produced): `<ISO-timestamp>-<pid>-<branch-hash>.log` β wrapper diagnostics + the agent's stderr.
|
|
384
|
+
|
|
385
|
+
See the [Outcome classes](#outcome-classes) table for what each `<reason>` means and the recommended recovery action. Common patterns:
|
|
386
|
+
|
|
387
|
+
- `agent-timeout` recurring β bump `distill.maxDurationMinutes` (default 10) or investigate what's taking the agent so long. Log shows the last tool calls before SIGTERM.
|
|
388
|
+
- `markers-after-agent-exit` β the agent's squash commit may already be on `<default>` with literal `<<<<<<<` markers in tracked files. The recovery hint in the sidecar names the exact `git revert HEAD --no-edit` command.
|
|
389
|
+
- `agent-exit-nonzero` β check the model provider (rate limits, auth refresh, network). The agent's stderr is in the `.log` file.
|
|
390
|
+
- `divergent-history` β a teammate likely landed a commit on `origin/<default>` while the agent's distill ran. `git pull --no-rebase` to integrate.
|
|
391
|
+
|
|
392
|
+
### "vault not a git repo" / "legacy embedded layout"
|
|
393
|
+
|
|
394
|
+
Auto-distill requires **git** and the **subdir vault layout**. If you see:
|
|
395
|
+
|
|
396
|
+
- `vault not a git repo` β either set `distill.enabled: true` and let pi-napkin auto-init git for you on next session, run `git init` in the vault root manually, or disable auto-distill with `distill.enabled: false`.
|
|
397
|
+
- `legacy embedded layout` β follow the [migration steps](#migration-from-legacy-layout). Auto-distill stays off for this session; manual `/distill` works regardless.
|
|
398
|
+
|
|
399
|
+
Manual `/distill` works without git and works on any vault layout.
|
|
400
|
+
|
|
401
|
+
### Testing hooks
|
|
402
|
+
|
|
403
|
+
The distill wrapper (`extensions/distill/scripts/distill-wrapper.sh`) reads several environment variables that exist solely to make integration tests deterministic. Production code never sets them; documenting here so future maintainers know they exist and what they do, and so a future-you grepping for one of these names can land on this section.
|
|
404
|
+
|
|
405
|
+
| Variable | Purpose |
|
|
406
|
+
|---|---|
|
|
407
|
+
| `NAPKIN_DISTILL_PI_BIN` | Override the `pi` binary path. Integration tests under `extensions/distill/test-fixtures/agent-stubs/` point this at a bash stub that simulates a specific agent behavior class (clean-distill, conflict-resolve-clean, agent-timeout, etc.) so the wrapper completes its lifecycle without contacting a real LLM. |
|
|
408
|
+
| `NAPKIN_DISTILL_SKIP_PI=1` | Skip BOTH the napkin shim install AND the `pi` invocation. Used by tests that pre-stage file changes manually and only want to exercise the wrapper's lifecycle (validation, salvage, sidecar emission). |
|
|
409
|
+
| `NAPKIN_DISTILL_NO_RECURSE=1` | Exported by the wrapper into the agent's environment so a nested `pi` won't auto-distill recursively. Tests sometimes set it directly to suppress recursion when invoking the wrapper from inside another distill. |
|
|
410
|
+
| `NAPKIN_DISTILL_HALT_AFTER_META=1` | Halt right after rewriting `meta.json`'s pid to the wrapper's pid. Lets tests inspect the updated meta without the cleanup trap wiping the worktree. Clears the `EXIT` trap so cleanup is skipped β caller is responsible for tearing down the worktree afterwards. |
|
|
411
|
+
| `NAPKIN_DISTILL_HALT_AFTER_SHIM=1` | Halt right after the per-distill napkin shim is installed at `<worktree>/.napkin/distill/bin/napkin`. Lets tests inspect the shim contents and PATH injection without the cleanup trap firing. |
|
|
412
|
+
| `NAPKIN_DISTILL_FORCE_CLEANUP=1` | Force the salvage / cleanup path to run unconditionally (even on success). Used to test salvage idempotency. |
|
|
413
|
+
| `NAPKIN_DISTILL_TIMEOUT_KILL_GRACE_SECS=<n>` | Override the `timeout(1) -k` grace window (default 30s) β the delay between SIGTERM and SIGKILL when the agent exceeds `distill.maxDurationMinutes`. Used to keep timeout tests fast. |
|
|
414
|
+
|
|
415
|
+
Production never sets any of these. If you find one in your environment by accident, `unset` it and re-run β the wrapper falls back to its normal behaviour automatically.
|
|
416
|
+
|
|
417
|
+
## Migration from PR #11
|
|
418
|
+
|
|
419
|
+
pi-napkin v0.1.x shipped an LLM-backed git merge driver (`.napkin-distill-merge`). PR #12 (v0.2.x) deleted the driver entirely; the agent now resolves its own conflicts as part of the distill task. New vaults set up by v0.2+ never see the driver. Existing v0.1.x vaults retain inert fragments that are safe to leave in place but cleaner to remove:
|
|
420
|
+
|
|
421
|
+
```bash
|
|
422
|
+
# In each existing vault that ran v0.1.x distill:
|
|
423
|
+
git config --local --remove-section merge.napkin-distill-merge 2>/dev/null || true
|
|
424
|
+
|
|
425
|
+
# Edit <vault>/.gitattributes and remove this line if present:
|
|
426
|
+
# *.md merge=napkin-distill-merge
|
|
427
|
+
```
|
|
428
|
+
|
|
429
|
+
Why manual: PR #12 deliberately avoids automatic migration. The orphaned `.gitattributes` rule falls back to git's built-in merge driver once the script is gone (the rule becomes inert, not harmful), so the cost of automating cleanup outweighs the benefit for a project with a small user base. New vaults aren't affected.
|
|
430
|
+
|
|
431
|
+
## Maintenance
|
|
432
|
+
|
|
433
|
+
### Verifying the distill flow end-to-end against a real LLM
|
|
434
|
+
|
|
435
|
+
CI uses bash-stub fixtures (`extensions/distill/test-fixtures/agent-stubs/`) to cover the agent-behavior space without burning tokens. For ad-hoc re-validation that the full runtime β wrapper subprocess + agent + JS-side polling β still walks cleanly through a real model, run:
|
|
436
|
+
|
|
437
|
+
```bash
|
|
438
|
+
bun run verify:e2e
|
|
439
|
+
```
|
|
440
|
+
|
|
441
|
+
The script (`scripts/verify-e2e.ts`) creates a tmpdir vault with a bare-repo origin, registers the distill extension against a captured mock `ExtensionAPI`, triggers the `/distill` command handler so the production `runDistillWith` path runs (real wrapper subprocess, real 2-second `setInterval` poller), and asserts on the dispatched UI notification's severity + message together with filesystem post-conditions: no conflict markers, HEAD on default branch, agent's squash commit landed, distill branch removed, worktree removed, outcome sidecar with class `merged-content`, and origin advanced. Exits 0 on PASS, 1 on FAIL. Manual-only β not in CI; cost is roughly $0.50 per run.
|
|
442
|
+
|
|
443
|
+
This replaces the earlier prompt-only gate. The strict superset matters because the wrapperβJS-poller seam β where worktree-teardown and outcome-write race β is invisible to a prompt-only harness.
|
|
444
|
+
|
|
445
|
+
## Future: builder-deleter
|
|
446
|
+
|
|
447
|
+
Next major feature: a "builder-deleter" janitor that acts on the `supersedes:` frontmatter convention that auto-distill already writes. When a note lists `supersedes: ["old/note.md"]`, the janitor archives the superseded file. Threshold-triggered to avoid running on every distill, git gc as the safety net. Design pending.
|
|
448
|
+
|
|
449
|
+
## License
|
|
450
|
+
|
|
451
|
+
MIT
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
# napkin-distill
|
|
2
|
+
|
|
3
|
+
Pi extension that automatically distills knowledge from conversations into your napkin vault.
|
|
4
|
+
|
|
5
|
+
## How it works
|
|
6
|
+
|
|
7
|
+
1. Runs on a configurable interval (default: 60 minutes)
|
|
8
|
+
2. Checks if the conversation changed since last distill
|
|
9
|
+
3. Sends the new conversation to a model
|
|
10
|
+
4. The model extracts structured notes using your vault's templates
|
|
11
|
+
5. Notes are written directly into the vault
|
|
12
|
+
|
|
13
|
+
## Setup
|
|
14
|
+
|
|
15
|
+
Enable distill in your vault config:
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
napkin config set --key distill.enabled --value true
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Or edit `.napkin/config.json` directly:
|
|
22
|
+
|
|
23
|
+
```json
|
|
24
|
+
{
|
|
25
|
+
"distill": {
|
|
26
|
+
"enabled": true,
|
|
27
|
+
"intervalMinutes": 60,
|
|
28
|
+
"model": {
|
|
29
|
+
"provider": "anthropic",
|
|
30
|
+
"id": "claude-sonnet-4-6"
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Configuration
|
|
37
|
+
|
|
38
|
+
All distill settings live in `.napkin/config.json` under the `distill` key:
|
|
39
|
+
|
|
40
|
+
| Field | Default | Description |
|
|
41
|
+
|-------|---------|-------------|
|
|
42
|
+
| `distill.enabled` | `false` | Enable automatic distillation |
|
|
43
|
+
| `distill.intervalMinutes` | `60` | How often to check for new content |
|
|
44
|
+
| `distill.model.provider` | `"anthropic"` | LLM provider |
|
|
45
|
+
| `distill.model.id` | `"claude-sonnet-4-6"` | Model for distillation |
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+
## Manual trigger
|
|
49
|
+
|
|
50
|
+
Use `/distill` in pi to manually trigger distillation of the full conversation.
|
|
51
|
+
|
|
52
|
+
## Turn off for this session
|
|
53
|
+
|
|
54
|
+
`/distill-auto-this-session` turns auto-distill off or on for the current session only. Manual `/distill` still works regardless.
|
|
55
|
+
|
|
56
|
+
| Invocation | Effect |
|
|
57
|
+
|------------|--------|
|
|
58
|
+
| `/distill-auto-this-session` | Toggle (on β off) |
|
|
59
|
+
| `/distill-auto-this-session on` | Turn auto-distill on |
|
|
60
|
+
| `/distill-auto-this-session off` | Turn auto-distill off |
|
|
61
|
+
| `/distill-auto-this-session status` | Report current state and time to next run |
|
|
62
|
+
|
|
63
|
+
### Status bar indicator
|
|
64
|
+
|
|
65
|
+
| Display | Meaning |
|
|
66
|
+
|---------|---------|
|
|
67
|
+
| `distill: 59m30s` | Auto-distill is on β countdown to next run |
|
|
68
|
+
| `distill: off (session)` | Auto-distill is off for this session |
|
|
69
|
+
| `distill: off` | Distill is disabled at the vault level (`distill.enabled=false`) |
|
|
70
|
+
| `β distill 12s` | A distillation is currently running |
|
|
71
|
+
|
|
72
|
+
### Persistence
|
|
73
|
+
|
|
74
|
+
The off/on state is written into the session file as a `CustomEntry` (it does not enter the LLM context). Resuming the same session with `pi --session <path>` β or picking it from the session list β restores the state. Other sessions (and new sessions) are unaffected.
|
|
75
|
+
|
|
76
|
+
On resume, a brief notification reminds you when auto-distill is still off, so you don't lose track of it.
|
|
77
|
+
|
|
78
|
+
Turning auto-distill back on resets the countdown so the next run respects the full interval instead of firing immediately.
|
|
79
|
+
|
|
80
|
+
Persistent vault-wide opt-out still lives in `.napkin/config.json` (`distill.enabled`).
|
|
81
|
+
|
|
82
|
+
### Notes
|
|
83
|
+
|
|
84
|
+
- `/distill-auto-this-session off` does **not** interrupt a distillation already in flight. It only suppresses future scheduled runs.
|
|
85
|
+
- If `distill.enabled` is `false` in the vault config, the command warns that the session flag has no effect β enable distill in the vault config first.
|
|
86
|
+
- Forking a session with auto-distill off carries the state forward (the CustomEntry is copied with the rest of the session). Run `/distill-auto-this-session on` in the fork to turn it back on.
|