@basicmemory/openclaw-basic-memory 0.23.1 → 0.24.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -7
- package/bm-client.ts +107 -7
- package/dist/bm-client.d.ts +33 -1
- package/dist/bm-client.js +77 -6
- package/dist/index.js +9 -1
- package/dist/tools/edit-note.js +8 -1
- package/dist/tools/list-memory-projects.js +23 -5
- package/dist/tools/list-workspaces.js +3 -1
- package/dist/tools/memory-provider.js +2 -1
- package/dist/tools/read-note.js +18 -3
- package/dist/tools/search-notes.js +3 -1
- package/dist/tools/write-note.js +65 -5
- package/index.ts +9 -1
- package/openclaw.plugin.json +2 -0
- package/package.json +1 -1
- package/scripts/fetch-skills.ts +87 -36
- package/scripts/setup-bm.sh +4 -1
- package/skills/manifest.json +31 -16
- package/skills/memory-agents/SKILL.md +170 -0
- package/skills/memory-capture/SKILL.md +7 -5
- package/skills/memory-comark/SKILL.md +204 -0
- package/skills/memory-comark/references/components.md +740 -0
- package/skills/memory-continue/SKILL.md +11 -1
- package/skills/memory-curate/SKILL.md +32 -39
- package/skills/memory-defrag/SKILL.md +106 -57
- package/skills/memory-ingest/SKILL.md +14 -3
- package/skills/memory-lifecycle/SKILL.md +21 -21
- package/skills/memory-literary-analysis/SKILL.md +258 -28
- package/skills/memory-metadata-search/SKILL.md +29 -4
- package/skills/memory-notes/SKILL.md +69 -17
- package/skills/memory-okf/SKILL.md +135 -0
- package/skills/memory-onboarding/SKILL.md +2 -2
- package/skills/memory-onboarding/references/assistant-setup.md +8 -0
- package/skills/memory-onboarding/references/conventions.md +3 -3
- package/skills/memory-onboarding/references/domain-playbooks.md +1 -1
- package/skills/memory-onboarding/references/schema-guide.md +3 -3
- package/skills/memory-quest/SKILL.md +129 -0
- package/skills/memory-quest/agents/openai.yaml +9 -0
- package/skills/memory-reflect/SKILL.md +125 -44
- package/skills/memory-research/SKILL.md +9 -15
- package/skills/memory-schema/SKILL.md +29 -21
- package/skills/memory-tasks/SKILL.md +36 -10
- package/tools/edit-note.ts +15 -1
- package/tools/list-memory-projects.ts +24 -5
- package/tools/list-workspaces.ts +3 -1
- package/tools/memory-provider.ts +2 -1
- package/tools/read-note.ts +22 -3
- package/tools/search-notes.ts +3 -1
- package/tools/write-note.ts +94 -4
- package/skills/memory-ci-capture/SKILL.md +0 -56
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: memory-literary-analysis
|
|
3
|
-
description: "Analyze a complete literary work into a structured Basic Memory knowledge graph
|
|
3
|
+
description: "Analyze a complete literary work into a structured Basic Memory knowledge graph: schema design, entity seeding, chapter-by-chapter processing, cross-referencing, and validation."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Memory Literary Analysis
|
|
7
7
|
|
|
8
|
-
Transform a complete literary work into a structured knowledge graph. Characters, themes, chapters, locations, symbols, and literary devices become interconnected notes — searchable, validatable, and
|
|
8
|
+
Transform a complete literary work into a structured knowledge graph. Characters, themes, chapters, locations, symbols, and literary devices become interconnected notes — searchable, validatable, and traversable.
|
|
9
9
|
|
|
10
10
|
## When to Use
|
|
11
11
|
|
|
@@ -23,15 +23,56 @@ Phase 1: Seed → stub notes for known major entities
|
|
|
23
23
|
Phase 2: Process → chapter-by-chapter notes in batches
|
|
24
24
|
Phase 3: Cross-ref → enrich arcs, add parallels, write analysis
|
|
25
25
|
Phase 4: Validate → schema checks, drift detection, consistency
|
|
26
|
-
Phase 5:
|
|
26
|
+
Phase 5: Explore → traverse the graph, write synthesis notes
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
+
## Tools
|
|
30
|
+
|
|
31
|
+
Writing always goes through `write_note` and `edit_note`. For *reading* — which is most of
|
|
32
|
+
the work in a long analysis — prefer the POSIX read verbs where they are available
|
|
33
|
+
(the MCP `cat`, `find`, `grep`, and `ls` tools are on by default; the `bm` CLI verbs are always
|
|
34
|
+
available):
|
|
35
|
+
|
|
36
|
+
| Need | Use | Instead of |
|
|
37
|
+
|------|-----|-----------|
|
|
38
|
+
| A section of a long note | `cat <note> --section Observations` | reading the whole note |
|
|
39
|
+
| A line range of the source text | `cat <source>.txt --lines 4200-4890` | pulling the whole book into context |
|
|
40
|
+
| Notes matching frontmatter | `find --meta status=active` | reading notes to check fields |
|
|
41
|
+
| Fields across many notes | `find --meta ... --fields pov,setting` | one read per note |
|
|
42
|
+
| Where something lives | `ls`, `tree`, `find --name '*.md'` | listing everything |
|
|
43
|
+
|
|
44
|
+
The two rules that matter across a 100+ chapter run:
|
|
45
|
+
|
|
46
|
+
- **Never read a note to check a field.** That is what `--meta` predicates and `--fields`
|
|
47
|
+
projection are for — one call answers what a read-per-note loop would cost.
|
|
48
|
+
- **Never pull a whole file into context to reach one part of it.** Sections and line ranges
|
|
49
|
+
slice the *output*: the full note is still fetched, then cut down before it is returned. What
|
|
50
|
+
they save is context, not I/O — a long chapter or a full source text costs you the tokens of
|
|
51
|
+
the relevant part, not of the whole file.
|
|
52
|
+
|
|
53
|
+
Two sharp edges to know before you write a query. The first fails *quietly* — a short
|
|
54
|
+
answer, exit 0, no warning — so learn it here rather than from a graph you thought you had
|
|
55
|
+
audited:
|
|
56
|
+
|
|
57
|
+
- **`find` pages, and the default page is 10.** Any query whose answer is "all N chapters"
|
|
58
|
+
needs `--page-size 200` (the maximum) — see [Coverage Checks](#coverage-checks).
|
|
59
|
+
- **`--name` cannot combine with `--meta`.** The metadata search has no filename glob. Scope a
|
|
60
|
+
`--meta` query with the positional path instead: `find /characters --meta
|
|
61
|
+
'note_type=character'`. That path is matched on a directory boundary against the *file path*
|
|
62
|
+
a note is indexed under — where the note actually lives, not its permalink, which stops
|
|
63
|
+
mirroring the file path once a note pins `permalink:` in frontmatter or is moved. So
|
|
64
|
+
`/characters` reaches everything filed under `characters/` (including `characters/major/`),
|
|
65
|
+
and never `characters-cut/`.
|
|
66
|
+
|
|
67
|
+
If the POSIX verbs are unavailable, every step below still works with `search_notes`,
|
|
68
|
+
`read_note`, and `list_directory` — it just costs more.
|
|
69
|
+
|
|
29
70
|
## Phase 0: Setup
|
|
30
71
|
|
|
31
72
|
### Create the Project
|
|
32
73
|
|
|
33
74
|
```python
|
|
34
|
-
create_memory_project(
|
|
75
|
+
create_memory_project(project_name="<work-name>", project_path="~/basic-memory/<work-name>")
|
|
35
76
|
```
|
|
36
77
|
|
|
37
78
|
Use a kebab-case slug of the work's title (e.g., `great-gatsby`, `hamlet`, `beloved`).
|
|
@@ -212,6 +253,7 @@ Schema for literary technique and device notes.
|
|
|
212
253
|
|
|
213
254
|
```
|
|
214
255
|
<project>/
|
|
256
|
+
<work>.txt # the source text, verbatim (see Phase 2)
|
|
215
257
|
schema/ # 6 schema definitions
|
|
216
258
|
chapters/ # one note per chapter/section + prologue/epilogue
|
|
217
259
|
characters/
|
|
@@ -271,6 +313,46 @@ Stubs don't need to be complete — they give `[[wiki-link]]` targets and will b
|
|
|
271
313
|
|
|
272
314
|
Obtain the full text and identify chapter/section boundaries. For public domain works, Project Gutenberg is a good source. For copyrighted works, work from a physical or licensed digital copy.
|
|
273
315
|
|
|
316
|
+
**Put the source text inside the project directory, as `.txt`, and index it once.** `bm cat`
|
|
317
|
+
resolves a *note identifier*, not a filesystem path — it can only reach a file the project
|
|
318
|
+
index has observed. A one-time index pass gives the raw text an entity row, after which the
|
|
319
|
+
line-range slice works against it:
|
|
320
|
+
|
|
321
|
+
```bash
|
|
322
|
+
cp ~/Downloads/moby-dick.txt ~/basic-memory/moby-dick/moby-dick.txt
|
|
323
|
+
bm reindex --search -p moby-dick # one pass; the .txt becomes readable
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
Two constraints that make this the right shape, both worth respecting:
|
|
327
|
+
|
|
328
|
+
- **Keep it `.txt`, do not convert it to `.md`.** Basic Memory injects frontmatter into
|
|
329
|
+
markdown notes, which shifts every line number by the height of that block — an offset map
|
|
330
|
+
built from the original file would then be silently wrong. A `.txt` is stored verbatim, so
|
|
331
|
+
its line numbers stay 1:1 with the file on disk.
|
|
332
|
+
- **Keep it inside the project.** A source text elsewhere on disk is not an entity, and
|
|
333
|
+
`bm cat` answers `Error: Entity not found`. If you must leave it outside, drop the BM verbs
|
|
334
|
+
for the source and use plain shell (`sed -n '4200,4890p' <path>`) — the notes still get the
|
|
335
|
+
BM verbs, only the raw source falls back to the shell.
|
|
336
|
+
|
|
337
|
+
**Then build a chapter offset map once, before processing.** Scan the text for chapter
|
|
338
|
+
headings and record the line range of each chapter, then read chapters by range rather than
|
|
339
|
+
re-reading the whole book into context:
|
|
340
|
+
|
|
341
|
+
```bash
|
|
342
|
+
grep -n '^CHAPTER ' ~/basic-memory/moby-dick/moby-dick.txt # heading -> line number
|
|
343
|
+
bm cat moby-dick/moby-dick.txt --lines 4200-4890 --plain # one chapter, not the whole text
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
`grep -n` here is the shell's grep on a filesystem path (this is the map-building step, and
|
|
347
|
+
it needs the file). `bm cat` then takes the *note identifier* and returns exactly that slice plus a
|
|
348
|
+
`lines 4200-4890 of N` footer. Spell the identifier project-qualified
|
|
349
|
+
(`<work>/<work>.txt`) so it routes to the right project. `bm head moby-dick/moby-dick.txt -n 40` is the cheap
|
|
350
|
+
way to eyeball the heading format before writing the grep pattern.
|
|
351
|
+
|
|
352
|
+
Store the map in the project (a note or a small JSON file) so later batches — and a resumed
|
|
353
|
+
run after context compaction — do not have to rediscover it. On a long work this is the
|
|
354
|
+
single largest context saving in the pipeline.
|
|
355
|
+
|
|
274
356
|
### Batching Strategy
|
|
275
357
|
|
|
276
358
|
Process ~10 chapters per batch to balance depth with progress. Group by narrative arc or thematic focus:
|
|
@@ -289,7 +371,9 @@ Adjust batch size based on chapter length and density. Short, action-heavy chapt
|
|
|
289
371
|
|
|
290
372
|
For each chapter:
|
|
291
373
|
|
|
292
|
-
**1. Read the chapter carefully.**
|
|
374
|
+
**1. Read the chapter carefully.** Read the chapter's line range from the offset map
|
|
375
|
+
(`bm cat <source>.txt --lines <start>-<end>`), not the whole file. Read the actual text —
|
|
376
|
+
never work from memory or a summary; textual evidence is the entire point.
|
|
293
377
|
|
|
294
378
|
**2. Create the chapter note:**
|
|
295
379
|
|
|
@@ -332,13 +416,17 @@ write_note(
|
|
|
332
416
|
```python
|
|
333
417
|
edit_note(
|
|
334
418
|
identifier="characters/major/<character-slug>",
|
|
335
|
-
operation="
|
|
336
|
-
|
|
419
|
+
operation="insert_after_section",
|
|
420
|
+
section="Observations",
|
|
337
421
|
content="""- [arc] Ch.<N>: <What happens to this character>
|
|
338
422
|
- [quote] "<Attributed quote>" (Ch.<N>)"""
|
|
339
423
|
)
|
|
340
424
|
```
|
|
341
425
|
|
|
426
|
+
`append` and `prepend` do not target a section — they write at the end of the note and the
|
|
427
|
+
start of the body. Use `insert_after_section` / `insert_before_section` to land lines under a
|
|
428
|
+
heading; they fail loudly if the heading is missing.
|
|
429
|
+
|
|
342
430
|
**4. Track progress** using the memory-tasks skill to create a processing task that survives context compaction.
|
|
343
431
|
|
|
344
432
|
### What to Capture Per Chapter
|
|
@@ -364,7 +452,7 @@ As each chapter is processed, append observations to relevant entities:
|
|
|
364
452
|
|
|
365
453
|
### Adding Prose and Interpretation
|
|
366
454
|
|
|
367
|
-
After the structured observations are in place, consider adding interpretive prose to major entity notes.
|
|
455
|
+
After the structured observations are in place, consider adding interpretive prose to major entity notes. Insert 2-4 paragraphs of critical essay before the Observations section with `edit_note(operation="insert_before_section", section="Observations", content=...)`. This prose should:
|
|
368
456
|
|
|
369
457
|
- Argue for a reading of the character, theme, or symbol — not just describe it
|
|
370
458
|
- Connect the entity to the work's larger concerns and to literary tradition
|
|
@@ -377,6 +465,25 @@ The prose adds the interpretive texture that structured observations alone canno
|
|
|
377
465
|
|
|
378
466
|
After all chapters are processed:
|
|
379
467
|
|
|
468
|
+
### Find What Needs Enriching
|
|
469
|
+
|
|
470
|
+
Do not re-read every note to decide what is thin. Query for it:
|
|
471
|
+
|
|
472
|
+
```bash
|
|
473
|
+
bm find --meta 'note_type=chapter' --fields chapter_number,pov,setting --page-size 200
|
|
474
|
+
bm find --meta 'note_type=character' --fields role,status --page-size 200 # who is still a stub
|
|
475
|
+
bm find --meta 'chapter_number>100' --fields pov --page-size 200 # late-book POV drift
|
|
476
|
+
```
|
|
477
|
+
|
|
478
|
+
A field a note never set comes back as a blank cell (`null` under `--json`), so rows with
|
|
479
|
+
blanks are the work queue. This turns "audit the graph" from a read of every note into one
|
|
480
|
+
call per question.
|
|
481
|
+
|
|
482
|
+
`write_note` snake-cases `note_type` (`Chapter` is stored as `chapter`), and `--meta`
|
|
483
|
+
type predicates are normalized the same way, so either spelling matches. `--page-size 200`
|
|
484
|
+
is not decoration: without it these return the first 10 rows and the work queue looks ten
|
|
485
|
+
items long.
|
|
486
|
+
|
|
380
487
|
### Character Arcs
|
|
381
488
|
For each major character, write a full `[arc]` summary observation covering their trajectory across the work.
|
|
382
489
|
|
|
@@ -426,18 +533,18 @@ During chapter processing, new minor characters, locations, and symbols will eme
|
|
|
426
533
|
|
|
427
534
|
```python
|
|
428
535
|
# Validate each entity type
|
|
429
|
-
schema_validate(
|
|
430
|
-
schema_validate(
|
|
431
|
-
schema_validate(
|
|
432
|
-
schema_validate(
|
|
433
|
-
schema_validate(
|
|
434
|
-
schema_validate(
|
|
536
|
+
schema_validate(note_type="Character")
|
|
537
|
+
schema_validate(note_type="Theme")
|
|
538
|
+
schema_validate(note_type="Chapter")
|
|
539
|
+
schema_validate(note_type="Location")
|
|
540
|
+
schema_validate(note_type="Symbol")
|
|
541
|
+
schema_validate(note_type="LiteraryDevice")
|
|
435
542
|
```
|
|
436
543
|
|
|
437
544
|
### Drift Detection
|
|
438
545
|
|
|
439
546
|
```python
|
|
440
|
-
schema_diff(
|
|
547
|
+
schema_diff(note_type="Character")
|
|
441
548
|
# ... for each type
|
|
442
549
|
```
|
|
443
550
|
|
|
@@ -446,26 +553,147 @@ Fix issues found — common fixes:
|
|
|
446
553
|
- Enum values outside allowed set → correct metadata
|
|
447
554
|
- Fields in notes but not schema → add as optional to schema if legitimate
|
|
448
555
|
|
|
556
|
+
### Coverage Checks
|
|
557
|
+
|
|
558
|
+
Schema validation proves notes match their shape. These prove the graph is *complete*:
|
|
559
|
+
|
|
560
|
+
```bash
|
|
561
|
+
bm find --meta 'note_type=chapter' --fields chapter_number --page-size 200 # every chapter present?
|
|
562
|
+
bm find --meta 'note_type=chapter' --fields pov,setting --page-size 200 # missing context?
|
|
563
|
+
bm find /characters --meta 'note_type=character' --fields role --page-size 200 # inventory vs. seed list
|
|
564
|
+
```
|
|
565
|
+
|
|
566
|
+
**A coverage check that pages is not a coverage check.** `bm find` defaults to
|
|
567
|
+
`--page-size 10`, so the un-sized form of the first query "proves" a 138-chapter work has 10
|
|
568
|
+
chapters. 200 is the maximum page size; past that, iterate with `--page 2`, `--page 3`, … .
|
|
569
|
+
Scope a `--meta` query with the positional path (`/characters`), never `--name` — the two
|
|
570
|
+
options are mutually exclusive, because the metadata search has no filename glob. The
|
|
571
|
+
positional path scopes by the *file path* a note is indexed under, matched on a directory
|
|
572
|
+
boundary: `/characters` admits `characters/major/ahab.md` but never `characters-cut/`. It is
|
|
573
|
+
not a permalink match, so a note that pins its own `permalink:` is still found where its file
|
|
574
|
+
lives.
|
|
575
|
+
|
|
576
|
+
Read the count off the footer, not off the rows you can see. In a terminal every `find`
|
|
577
|
+
result reports `page 1 • total 138`, and appends `• more available (--page)` when the page
|
|
578
|
+
truncated the answer — that suffix appearing is the check *failing*, whatever the visible
|
|
579
|
+
rows say. **The footer is a TTY feature.** Piped output without `--plain` is JSON, which
|
|
580
|
+
carries `total` and `has_more`; with `--plain` the rows go to stdout and a has-more notice
|
|
581
|
+
goes to stderr. For scripts, read `has_more` from `--json`.
|
|
582
|
+
|
|
583
|
+
For the sequence gap — the failure that a count alone cannot catch — take the numbers from
|
|
584
|
+
`--json`, which carries `total`, `total_is_exact`, and `has_more`:
|
|
585
|
+
|
|
586
|
+
```bash
|
|
587
|
+
expected=138; page=1; rows='[]'
|
|
588
|
+
while :; do
|
|
589
|
+
resp=$(bm find --meta 'note_type=chapter' --fields chapter_number \
|
|
590
|
+
--page-size 200 --page $page --project <work> --json)
|
|
591
|
+
rows=$(jq -n --argjson acc "$rows" --argjson r "$resp" \
|
|
592
|
+
'$acc + [$r.results[] | {title, n: .fields.chapter_number}]')
|
|
593
|
+
[ "$(jq -r '.has_more' <<<"$resp")" = "true" ] || break
|
|
594
|
+
page=$((page + 1))
|
|
595
|
+
done
|
|
596
|
+
jq -n --argjson rows "$rows" --argjson expected "$expected" '
|
|
597
|
+
([$rows[] | select(.n != null and (.n | tostring | test("^[0-9]+$"))) | .n | tonumber]) as $n
|
|
598
|
+
| { total: ($rows | length),
|
|
599
|
+
unnumbered: [$rows[] | select(.n == null or (.n | tostring | test("^[0-9]+$") | not))
|
|
600
|
+
| .title],
|
|
601
|
+
missing: ([range(1; $expected + 1)] - $n),
|
|
602
|
+
duplicates: ($n | group_by(.) | map(select(length > 1) | .[0])),
|
|
603
|
+
out_of_range: ($n | map(select(. < 1 or . > $expected)) | unique) }'
|
|
604
|
+
```
|
|
605
|
+
|
|
606
|
+
The loop is not ceremony. `--page-size` caps at 200, so a single call cannot inventory a work
|
|
607
|
+
with more than 200 chapters — and rerunning it with `--page 2` *replaces* the numbers rather
|
|
608
|
+
than accumulating them, which reports chapters 1-200 as missing on a corpus that is complete.
|
|
609
|
+
Walk until `has_more` is false and check the union.
|
|
610
|
+
|
|
611
|
+
`--fields` returns YAML-native types, so `chapter_number: 63` comes back as `63`. The
|
|
612
|
+
`tostring | test(...) | tonumber` guard still catches hand-typed values such as `"63"` or
|
|
613
|
+
`"12a"`.
|
|
614
|
+
|
|
615
|
+
Pass the work's **actual** chapter count as `$expected` — deriving the range from the highest
|
|
616
|
+
number found lets an incomplete graph pass. With 138 rows numbered 1..137 plus one duplicate,
|
|
617
|
+
a max-derived check reports `missing: []` while a chapter is genuinely absent: the duplicate
|
|
618
|
+
keeps the count right and the missing tail moves the goalpost. The check passes on
|
|
619
|
+
`unnumbered: []`, `missing: []`, `duplicates: []`, **and** `out_of_range: []` together, over
|
|
620
|
+
the combined pages.
|
|
621
|
+
|
|
622
|
+
`unnumbered` is not decoration either. A `chapter` note that never got a `chapter_number` comes
|
|
623
|
+
back as `null`, and feeding that straight to `tonumber` aborts the whole pipeline with `null
|
|
624
|
+
cannot be parsed as a number` — so the check *crashes on exactly the malformed inventory it
|
|
625
|
+
exists to find*. Partitioning first turns that into a named row.
|
|
626
|
+
|
|
627
|
+
`out_of_range` is not hypothetical: a prologue or epilogue typed as `chapter` lands at 0 or at
|
|
628
|
+
`$expected + 1`, and without that key the report reads clean — every expected number present,
|
|
629
|
+
none repeated — while the inventory holds a note the numbering does not account for. Type
|
|
630
|
+
front and back matter as its own note type, or widen `$expected` deliberately.
|
|
631
|
+
|
|
632
|
+
A gap in the middle of a batch is the most common processing failure and the easiest to miss
|
|
633
|
+
by eye; a duplicated chapter number is the second, and it hides the first.
|
|
634
|
+
|
|
449
635
|
### Relation Consistency
|
|
450
636
|
Spot-check bidirectional relations: if Chapter X `features [[Character]]`, does Character have observations referencing Chapter X? Fix gaps.
|
|
451
637
|
|
|
452
|
-
|
|
638
|
+
Orphans are the other half of this check — a note with no inbound or outbound relations is
|
|
639
|
+
either genuinely isolated or was never linked back into the graph:
|
|
640
|
+
|
|
641
|
+
```bash
|
|
642
|
+
bm orphans # entities with no relations in the graph
|
|
643
|
+
```
|
|
644
|
+
|
|
645
|
+
`orphans` finds notes with no relations. It does not find a `[[target]]` that resolves to
|
|
646
|
+
nothing — a misspelled or renamed entity name. Check a chapter's links with `bm tool build-context memory://chapters/<slug> --depth 1
|
|
647
|
+
--project <work> --json` and look for relations whose `to_entity_id` is `null`; fix the
|
|
648
|
+
spelling or add the alias, then re-run the chapter.
|
|
649
|
+
|
|
650
|
+
Graph quality is relation *density*, not note count. A pass that adds notes while leaving
|
|
651
|
+
orphans behind has made the graph worse.
|
|
652
|
+
|
|
653
|
+
## Phase 5: Explore the Graph
|
|
453
654
|
|
|
454
|
-
|
|
655
|
+
With the graph complete, traverse it to find what the chapter-by-chapter pass could not see:
|
|
455
656
|
|
|
456
|
-
```
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
{"id": "ishmael", "type": "file", "file": "characters/ishmael.md", "x": 500, "y": 0, "width": 400, "height": 300}
|
|
461
|
-
],
|
|
462
|
-
"edges": [
|
|
463
|
-
{"id": "e1", "fromNode": "ishmael", "toNode": "ahab", "label": "narrates"}
|
|
464
|
-
]
|
|
465
|
-
}
|
|
657
|
+
```bash
|
|
658
|
+
bm grep "features [[" --page-size 200 --project <work> --json # every chapter's cast, one call
|
|
659
|
+
bm find --meta 'note_type=theme' --fields prevalence --page-size 200 # thematic weight
|
|
660
|
+
bm grep "doubloon" --page-size 100 --project <work> # every mention of a symbol
|
|
466
661
|
```
|
|
467
662
|
|
|
468
|
-
|
|
663
|
+
"Which characters share the most chapters" is the first line plus a local parse of each
|
|
664
|
+
row's `content` for `features [[...]]` — one call instead of a read per chapter. Do not
|
|
665
|
+
reach for `bm tool build-context 'memory://characters/major/*'` here: a
|
|
666
|
+
wildcard context is capped at 100 related rows across all primaries and returns one primary
|
|
667
|
+
row per indexed observation, so it neither enumerates the cast nor walks the web.
|
|
668
|
+
`build-context` on a *single* note is the right tool for a different question, below.
|
|
669
|
+
`build-context` takes its URL as a positional argument — there is no `--url` option.
|
|
670
|
+
|
|
671
|
+
`grep` matches keywords and returns a page of 10, which quietly truncates "where does this
|
|
672
|
+
appear?" — a symbol in 40 chapters comes back as 10. For symbol tracing, raise
|
|
673
|
+
`--page-size`; the meaning shifts you are hunting are usually in the later occurrences, which
|
|
674
|
+
the default page would have dropped. `--semantic` ranks by meaning instead, which answers
|
|
675
|
+
"what is this about?" rather than "where does this word appear?".
|
|
676
|
+
|
|
677
|
+
Two more facts about `grep` rows. Matching is case-insensitive and note-level: a hit is a
|
|
678
|
+
note, not a line, and there is no `-n` or context. And each row's `content` is capped at
|
|
679
|
+
4000 characters; `content_truncated: true` marks a cut row, so a long note's tail (such as a
|
|
680
|
+
final chapter's `follows`/`precedes` relations) may be missing from a grep-driven scan.
|
|
681
|
+
When a parse depends on the end of a note, `cat` that note.
|
|
682
|
+
|
|
683
|
+
`--page-size` raises the ceiling, it does not remove it. A symbol in a long work can exceed
|
|
684
|
+
even 100, so check whether the last page was full and walk `--page 2`, `--page 3` until it is
|
|
685
|
+
not. A truncated symbol search fails the same silent way as an unpaginated `find`: a plausible
|
|
686
|
+
answer, exit 0, and no sign that the tail is missing.
|
|
687
|
+
|
|
688
|
+
And `grep` searches **your notes, not the source**. The `<work>.txt` is indexed as an entity,
|
|
689
|
+
but its body is not in the searchable text, so an occurrence you never carried into a note is
|
|
690
|
+
unreachable: a word present only in the source returns `total: 0`. So this answers "where have I written about the doubloon", not
|
|
691
|
+
"where does the doubloon appear in the book". For the latter, search the file itself and use
|
|
692
|
+
the [chapter offset map](#source-text-preparation) to turn a hit into a chapter.
|
|
693
|
+
|
|
694
|
+
Traversal is where second-order questions get answered — which characters share the most
|
|
695
|
+
chapters, which themes converge in the final act, where a symbol's meaning shifts. Capture
|
|
696
|
+
what you find as `analysis/` notes; those syntheses are the payoff of having built the graph.
|
|
469
697
|
|
|
470
698
|
## Adapting to Other Genres
|
|
471
699
|
|
|
@@ -504,6 +732,8 @@ This pipeline works for any literary text. Adjust schemas for genre:
|
|
|
504
732
|
- **Seed before processing.** Create entity stubs first so wiki-links resolve immediately during chapter processing.
|
|
505
733
|
- **Batch for sanity.** Processing ~10 chapters at a time balances depth with momentum. Track progress with a Task note.
|
|
506
734
|
- **Read the source text.** Don't rely on memory or summaries. Read (or re-read) the actual text for each batch before creating notes. Textual evidence is everything.
|
|
735
|
+
- **Read narrowly.** Keep the source text in the project as `.txt`, index it once, build the chapter offset map once, then read chapters by line range and notes by section. On a long work, whole files landing in context are the largest avoidable cost in the pipeline.
|
|
736
|
+
- **Query, don't scan.** When you need to know which notes have a field, ask with `--meta` predicates and `--fields` projection. Reading notes to check frontmatter is the mistake this pipeline makes at scale. One way these queries lie quietly: `find` returns 10 rows unless you pass `--page-size`.
|
|
507
737
|
- **Observations are your index.** The knowledge graph's value comes from categorized observations. Be generous with categories and specific with content.
|
|
508
738
|
- **Relations are your web.** Every chapter should link to characters, themes, locations, and devices. Every entity should link back to chapters where it appears.
|
|
509
739
|
- **Enrich iteratively.** Entity notes grow richer with each chapter. Don't try to write the perfect character note upfront — append as you go.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: memory-metadata-search
|
|
3
|
-
description: "
|
|
3
|
+
description: "Search Basic Memory notes by custom frontmatter fields with equality, range, array, and nested filters. Use when finding notes by status, priority, confidence, or any YAML field, not free text."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Memory Metadata Search
|
|
@@ -56,6 +56,17 @@ Numeric values use numeric comparison; strings use lexicographic comparison.
|
|
|
56
56
|
{"score": {"$between": [0.3, 0.8]}}
|
|
57
57
|
```
|
|
58
58
|
|
|
59
|
+
### Null (field missing or explicitly null)
|
|
60
|
+
|
|
61
|
+
```json
|
|
62
|
+
{"owner": null}
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Matches notes with no `owner` key and notes whose `owner` is explicitly null.
|
|
66
|
+
Null works only as a plain equality value — inside `$in`, `$between`, an
|
|
67
|
+
array-contains list, or a comparison it is rejected, because those compare
|
|
68
|
+
against the value and a comparison with null is never true.
|
|
69
|
+
|
|
59
70
|
### Nested Access (dot notation)
|
|
60
71
|
|
|
61
72
|
```json
|
|
@@ -67,20 +78,34 @@ Numeric values use numeric comparison; strings use lexicographic comparison.
|
|
|
67
78
|
| Operator | Syntax | Example |
|
|
68
79
|
|----------|--------|---------|
|
|
69
80
|
| Equality | `{"field": "value"}` | `{"status": "active"}` |
|
|
81
|
+
| Is null | `{"field": null}` | `{"owner": null}` |
|
|
70
82
|
| Array contains | `{"field": ["a", "b"]}` | `{"tags": ["security", "oauth"]}` |
|
|
71
83
|
| `$in` | `{"field": {"$in": [...]}}` | `{"priority": {"$in": ["high", "critical"]}}` |
|
|
72
84
|
| `$gt` / `$gte` | `{"field": {"$gt": N}}` | `{"confidence": {"$gt": 0.7}}` |
|
|
73
85
|
| `$lt` / `$lte` | `{"field": {"$lt": N}}` | `{"score": {"$lt": 0.5}}` |
|
|
74
86
|
| `$between` | `{"field": {"$between": [lo, hi]}}` | `{"score": {"$between": [0.3, 0.8]}}` |
|
|
87
|
+
| `$contains` | `{"field": {"$contains": value}}` | `{"tags": {"$contains": "security"}}` |
|
|
88
|
+
| `$exists` | `{"field": {"$exists": true}}` | `{"owner": {"$exists": false}}` |
|
|
75
89
|
| Nested | `{"a.b": "value"}` | `{"schema.version": "2"}` |
|
|
76
90
|
|
|
77
91
|
**Rules:**
|
|
78
92
|
- Keys must match `[A-Za-z0-9_-]+` (dots separate nesting levels)
|
|
79
|
-
-
|
|
93
|
+
- Several operators on one field AND together: `{"started": {"$gte": "2026-01-01", "$lt": "2026-02-01"}}`
|
|
80
94
|
- `$in` and array-contains require non-empty lists
|
|
81
95
|
- `$between` requires exactly `[min, max]`
|
|
82
|
-
|
|
83
|
-
|
|
96
|
+
- `null` is an is-null match and only valid as a plain equality value
|
|
97
|
+
- Comparison and `$between` bounds must be finite numbers — a magnitude no float
|
|
98
|
+
can hold (a 400-digit integer, which JSON keeps as an ordinary `int`) is
|
|
99
|
+
refused rather than compared against an infinite bound
|
|
100
|
+
- Metadata filters match Markdown notes only — indexed PDFs, images and other
|
|
101
|
+
regular files carry no frontmatter and are never hits, not even for `null`
|
|
102
|
+
|
|
103
|
+
> The leading `$` is optional (`gte` works), but write it for clarity.
|
|
104
|
+
|
|
105
|
+
Frontmatter fields say when a note was filed or reviewed. To ask what a note
|
|
106
|
+
*claims was true* on a date, use observation qualifiers (see **memory-notes**)
|
|
107
|
+
and `search_notes(valid_at="2026-07-01")` or
|
|
108
|
+
`search_notes(valid_overlaps="[2026-07-01,2026-08-01)")`.
|
|
84
109
|
|
|
85
110
|
## Using `search_notes` with Metadata
|
|
86
111
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: memory-notes
|
|
3
|
-
description: "How to write well-structured Basic Memory notes: frontmatter, observations
|
|
3
|
+
description: "How to write well-structured Basic Memory notes: frontmatter, categorized observations, wiki-link relations, and knowledge-graph best practices. Use when creating or improving notes."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Memory Notes
|
|
@@ -49,7 +49,7 @@ permalink: custom-path # optional — auto-generated from title if omitted
|
|
|
49
49
|
- The `title` must match the `# Heading` in the body
|
|
50
50
|
- Tags are searchable and help with discovery
|
|
51
51
|
- Custom `type` values (Task, Meeting, Person, etc.) work with the schema system. See the **memory-schema** skill for defining schemas, validating notes against them, and detecting drift.
|
|
52
|
-
- The `permalink` is auto-generated from the `title` and `directory`. For example, title "API Design Decisions" in directory "specs" produces permalink `specs/api-design-decisions` and memory URL `memory://specs/api-design-decisions`. If no directory is specified, the permalink is just the kebab-cased title.
|
|
52
|
+
- The `permalink` is auto-generated from the `title` and `directory`. For example, title "API Design Decisions" in directory "specs" produces permalink `specs/api-design-decisions` and memory URL `memory://specs/api-design-decisions`. If no directory is specified, the permalink is just the kebab-cased title. By default permalinks stay stable across file moves (unless the project enables `update_permalinks_on_move`). You rarely need to set one manually.
|
|
53
53
|
|
|
54
54
|
> **Note:** When using `write_note`, you don't write frontmatter yourself. The `title`, `tags`, `note_type`, and `metadata` are separate parameters — Basic Memory generates the frontmatter automatically. Your `content` parameter is just the markdown body starting with `# Heading`.
|
|
55
55
|
|
|
@@ -99,7 +99,42 @@ A few examples to illustrate the range:
|
|
|
99
99
|
- **One fact per observation.** Don't pack multiple ideas into one line.
|
|
100
100
|
- **Be specific.** `[decision] Use JWT` is less useful than `[decision] Use JWT with 15-minute expiry for API auth`.
|
|
101
101
|
- **Use tags for cross-cutting concerns.** `[risk] Rate limiting needed #api #security` makes this findable under both topics.
|
|
102
|
-
- **Categories are queryable.** `search_notes("[decision]
|
|
102
|
+
- **Categories are queryable.** `search_notes(entity_types=["observation"], categories=["decision"])` returns every decision observation across your knowledge base.
|
|
103
|
+
|
|
104
|
+
### When a Claim Holds
|
|
105
|
+
|
|
106
|
+
An observation can say when it is true. Put a qualifier after the category:
|
|
107
|
+
|
|
108
|
+
```
|
|
109
|
+
- [decision] @effective:2026-06-10 The cache layer uses Redis
|
|
110
|
+
- [event] @occurred:2026-10-14 Launch sync moved the date to Thursday
|
|
111
|
+
- [deadline] @due:2026-10-20 Draft ready for review
|
|
112
|
+
- [stage] @valid:2026-10 Series B, about 200 employees
|
|
113
|
+
- [decision] @effective[2026-06-10,2026-10-14) Launch on Friday
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
- **Kinds**: `effective` (a decision or rule takes effect), `occurred` (an
|
|
117
|
+
event happened), `due` (a commitment), `mentioned` (when it was said), and
|
|
118
|
+
`valid`, the default (`@2026-06-10` means `@valid:2026-06-10`).
|
|
119
|
+
- **Points** cover the span their precision names: `@2026` is that year,
|
|
120
|
+
`@2026-06` that month, and `@2026-06-10` from that date onward.
|
|
121
|
+
- **Ranges** use `[start,end)`, and either end can be open: `@effective[2026-06-10,)`.
|
|
122
|
+
- **Dates** are written out. An unquoted date is one token (`@occurred:2026-06-10`).
|
|
123
|
+
Quote anything longer: `@occurred:"June 10, 2026"`. A relative date like
|
|
124
|
+
`yesterday` is never read as a qualifier; it stays in the text.
|
|
125
|
+
- **When a fact changes**, close the old one's range and start the new one,
|
|
126
|
+
instead of deleting the old one. Both stay searchable by date.
|
|
127
|
+
|
|
128
|
+
Search by what notes say was true, rather than by when they were edited:
|
|
129
|
+
|
|
130
|
+
```python
|
|
131
|
+
search_notes(valid_at="2026-07-01") # true on that date
|
|
132
|
+
search_notes(time_kind="due", valid_overlaps="[2026-10-14,2026-10-21)") # due this week
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
A valid-time filter returns only dated observations; drop it to search
|
|
136
|
+
everything. Day points run "from that date onward", so a due-date window also
|
|
137
|
+
returns items that are already overdue.
|
|
103
138
|
|
|
104
139
|
## Relations
|
|
105
140
|
|
|
@@ -218,10 +253,10 @@ For people, try full name and last name. For organizations, try the full name an
|
|
|
218
253
|
When a note already exists, make targeted edits instead of rewriting the whole file:
|
|
219
254
|
|
|
220
255
|
```python
|
|
221
|
-
#
|
|
256
|
+
# Add a new observation under the existing Observations heading
|
|
222
257
|
edit_note(
|
|
223
258
|
identifier="API Design Decisions",
|
|
224
|
-
operation="
|
|
259
|
+
operation="insert_after_section",
|
|
225
260
|
section="Observations",
|
|
226
261
|
content="- [decision] Switched to OpenAPI 3.1 for spec generation #api"
|
|
227
262
|
)
|
|
@@ -237,7 +272,7 @@ edit_note(
|
|
|
237
272
|
# Add a new relation
|
|
238
273
|
edit_note(
|
|
239
274
|
identifier="API Design Decisions",
|
|
240
|
-
operation="
|
|
275
|
+
operation="insert_after_section",
|
|
241
276
|
section="Relations",
|
|
242
277
|
content="- depends_on [[Rate Limiter]]"
|
|
243
278
|
)
|
|
@@ -276,16 +311,20 @@ Basic Memory auto-generates frontmatter (including the permalink and memory URL)
|
|
|
276
311
|
|
|
277
312
|
### Editing an Existing Note
|
|
278
313
|
|
|
279
|
-
Use `edit_note` to update a note in place —
|
|
314
|
+
Use `edit_note` to update a note in place — six operations: `append`, `prepend`,
|
|
315
|
+
`find_replace`, `replace_section`, `insert_before_section`, `insert_after_section`.
|
|
280
316
|
|
|
281
317
|
```python
|
|
282
|
-
#
|
|
318
|
+
# insert_after_section / insert_before_section — add lines at an existing heading
|
|
283
319
|
edit_note(
|
|
284
320
|
identifier="API Design Decisions",
|
|
285
|
-
operation="
|
|
321
|
+
operation="insert_after_section",
|
|
286
322
|
section="Observations",
|
|
287
323
|
content="- [decision] Use OpenAPI 3.1 for spec generation #api"
|
|
288
324
|
)
|
|
325
|
+
|
|
326
|
+
# append / prepend — add to the end of the note or the start of the body
|
|
327
|
+
# (use for time-ordered logs; neither one targets a section)
|
|
289
328
|
edit_note(
|
|
290
329
|
identifier="API Design Decisions",
|
|
291
330
|
operation="prepend",
|
|
@@ -309,8 +348,23 @@ edit_note(
|
|
|
309
348
|
)
|
|
310
349
|
```
|
|
311
350
|
|
|
312
|
-
When an edit is destructive (`replace_section`, `find_replace`),
|
|
313
|
-
|
|
351
|
+
When an edit is destructive (`replace_section`, `find_replace`), read the note first
|
|
352
|
+
and pass its checksum, so the edit applies only to the revision you read:
|
|
353
|
+
|
|
354
|
+
```python
|
|
355
|
+
note = read_note(identifier="API Design Decisions", output_format="json")
|
|
356
|
+
edit_note(
|
|
357
|
+
identifier="API Design Decisions",
|
|
358
|
+
operation="replace_section",
|
|
359
|
+
section="Summary",
|
|
360
|
+
content="Concise, current summary of the decision and its rationale.",
|
|
361
|
+
expected_checksum=note["checksum"]
|
|
362
|
+
)
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
A checksum conflict means someone changed the note after you read it. Read it
|
|
366
|
+
again and redo the edit against the new text. `write_note(overwrite=True)` takes
|
|
367
|
+
`expected_checksum` the same way.
|
|
314
368
|
|
|
315
369
|
### Moving a Note
|
|
316
370
|
|
|
@@ -323,7 +377,7 @@ move_note(
|
|
|
323
377
|
)
|
|
324
378
|
```
|
|
325
379
|
|
|
326
|
-
|
|
380
|
+
By default the permalink stays the same after a move, so links keep resolving. Projects with `update_permalinks_on_move` enabled rewrite it from the new path.
|
|
327
381
|
|
|
328
382
|
## Best Practices
|
|
329
383
|
|
|
@@ -333,10 +387,8 @@ The permalink stays the same after a move, so all `[[wiki-links]]` and `memory:/
|
|
|
333
387
|
|
|
334
388
|
3. **Build incrementally.** Add to existing notes rather than creating duplicates. Use `edit_note` to append new observations or relations as you learn more.
|
|
335
389
|
|
|
336
|
-
4. **
|
|
337
|
-
|
|
338
|
-
5. **Use consistent titles.** Note titles are identifiers in the knowledge graph. `API Design Decisions` and `Api Design decisions` are different entities. Pick a convention and stick with it.
|
|
390
|
+
4. **Use consistent titles.** Note titles are identifiers in the knowledge graph. `API Design Decisions` and `Api Design decisions` are different entities. Pick a convention and stick with it.
|
|
339
391
|
|
|
340
|
-
|
|
392
|
+
5. **Link related concepts.** The value of a knowledge graph compounds with connections. A note with zero relations is an island — useful, but not as powerful as a connected one.
|
|
341
393
|
|
|
342
|
-
|
|
394
|
+
6. **Let the graph grow naturally.** Don't try to design a perfect taxonomy upfront. Write notes as you work, add relations as connections emerge, and periodically use the **memory-reflect** or **memory-defrag** skills to consolidate.
|