@basicmemory/openclaw-basic-memory 0.23.2 → 0.24.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. package/README.md +24 -7
  2. package/bm-client.ts +107 -7
  3. package/dist/bm-client.d.ts +33 -1
  4. package/dist/bm-client.js +77 -6
  5. package/dist/index.js +9 -1
  6. package/dist/tools/edit-note.js +8 -1
  7. package/dist/tools/list-memory-projects.js +23 -5
  8. package/dist/tools/list-workspaces.js +3 -1
  9. package/dist/tools/memory-provider.js +2 -1
  10. package/dist/tools/read-note.js +18 -3
  11. package/dist/tools/search-notes.js +3 -1
  12. package/dist/tools/write-note.js +65 -5
  13. package/index.ts +9 -1
  14. package/openclaw.plugin.json +2 -0
  15. package/package.json +1 -1
  16. package/scripts/fetch-skills.ts +87 -36
  17. package/scripts/setup-bm.sh +4 -1
  18. package/skills/manifest.json +31 -16
  19. package/skills/memory-agents/SKILL.md +170 -0
  20. package/skills/memory-capture/SKILL.md +7 -5
  21. package/skills/memory-comark/SKILL.md +204 -0
  22. package/skills/memory-comark/references/components.md +740 -0
  23. package/skills/memory-continue/SKILL.md +11 -1
  24. package/skills/memory-curate/SKILL.md +32 -39
  25. package/skills/memory-defrag/SKILL.md +106 -57
  26. package/skills/memory-ingest/SKILL.md +14 -3
  27. package/skills/memory-lifecycle/SKILL.md +21 -21
  28. package/skills/memory-literary-analysis/SKILL.md +258 -28
  29. package/skills/memory-metadata-search/SKILL.md +29 -4
  30. package/skills/memory-notes/SKILL.md +69 -17
  31. package/skills/memory-okf/SKILL.md +135 -0
  32. package/skills/memory-onboarding/SKILL.md +2 -2
  33. package/skills/memory-onboarding/references/assistant-setup.md +8 -0
  34. package/skills/memory-onboarding/references/conventions.md +3 -3
  35. package/skills/memory-onboarding/references/domain-playbooks.md +1 -1
  36. package/skills/memory-onboarding/references/schema-guide.md +3 -3
  37. package/skills/memory-quest/SKILL.md +129 -0
  38. package/skills/memory-quest/agents/openai.yaml +9 -0
  39. package/skills/memory-reflect/SKILL.md +125 -44
  40. package/skills/memory-research/SKILL.md +9 -15
  41. package/skills/memory-schema/SKILL.md +29 -21
  42. package/skills/memory-tasks/SKILL.md +36 -10
  43. package/tools/edit-note.ts +15 -1
  44. package/tools/list-memory-projects.ts +24 -5
  45. package/tools/list-workspaces.ts +3 -1
  46. package/tools/memory-provider.ts +2 -1
  47. package/tools/read-note.ts +22 -3
  48. package/tools/search-notes.ts +3 -1
  49. package/tools/write-note.ts +94 -4
  50. package/skills/memory-ci-capture/SKILL.md +0 -56
@@ -1,11 +1,11 @@
1
1
  ---
2
2
  name: memory-literary-analysis
3
- description: "Analyze a complete literary work into a structured Basic Memory knowledge graph. Covers schema design, entity seeding, chapter-by-chapter processing, cross-referencing, validation, and visualization."
3
+ description: "Analyze a complete literary work into a structured Basic Memory knowledge graph: schema design, entity seeding, chapter-by-chapter processing, cross-referencing, and validation."
4
4
  ---
5
5
 
6
6
  # Memory Literary Analysis
7
7
 
8
- Transform a complete literary work into a structured knowledge graph. Characters, themes, chapters, locations, symbols, and literary devices become interconnected notes — searchable, validatable, and visualizable.
8
+ Transform a complete literary work into a structured knowledge graph. Characters, themes, chapters, locations, symbols, and literary devices become interconnected notes — searchable, validatable, and traversable.
9
9
 
10
10
  ## When to Use
11
11
 
@@ -23,15 +23,56 @@ Phase 1: Seed → stub notes for known major entities
23
23
  Phase 2: Process → chapter-by-chapter notes in batches
24
24
  Phase 3: Cross-ref → enrich arcs, add parallels, write analysis
25
25
  Phase 4: Validate → schema checks, drift detection, consistency
26
- Phase 5: Visualize → Obsidian canvas files for character webs, timelines
26
+ Phase 5: Explore → traverse the graph, write synthesis notes
27
27
  ```
28
28
 
29
+ ## Tools
30
+
31
+ Writing always goes through `write_note` and `edit_note`. For *reading* — which is most of
32
+ the work in a long analysis — prefer the POSIX read verbs where they are available
33
+ (the MCP `cat`, `find`, `grep`, and `ls` tools are on by default; the `bm` CLI verbs are always
34
+ available):
35
+
36
+ | Need | Use | Instead of |
37
+ |------|-----|-----------|
38
+ | A section of a long note | `cat <note> --section Observations` | reading the whole note |
39
+ | A line range of the source text | `cat <source>.txt --lines 4200-4890` | pulling the whole book into context |
40
+ | Notes matching frontmatter | `find --meta status=active` | reading notes to check fields |
41
+ | Fields across many notes | `find --meta ... --fields pov,setting` | one read per note |
42
+ | Where something lives | `ls`, `tree`, `find --name '*.md'` | listing everything |
43
+
44
+ The two rules that matter across a 100+ chapter run:
45
+
46
+ - **Never read a note to check a field.** That is what `--meta` predicates and `--fields`
47
+ projection are for — one call answers what a read-per-note loop would cost.
48
+ - **Never pull a whole file into context to reach one part of it.** Sections and line ranges
49
+ slice the *output*: the full note is still fetched, then cut down before it is returned. What
50
+ they save is context, not I/O — a long chapter or a full source text costs you the tokens of
51
+ the relevant part, not of the whole file.
52
+
53
+ Two sharp edges to know before you write a query. The first fails *quietly* — a short
54
+ answer, exit 0, no warning — so learn it here rather than from a graph you thought you had
55
+ audited:
56
+
57
+ - **`find` pages, and the default page is 10.** Any query whose answer is "all N chapters"
58
+ needs `--page-size 200` (the maximum) — see [Coverage Checks](#coverage-checks).
59
+ - **`--name` cannot combine with `--meta`.** The metadata search has no filename glob. Scope a
60
+ `--meta` query with the positional path instead: `find /characters --meta
61
+ 'note_type=character'`. That path is matched on a directory boundary against the *file path*
62
+ a note is indexed under — where the note actually lives, not its permalink, which stops
63
+ mirroring the file path once a note pins `permalink:` in frontmatter or is moved. So
64
+ `/characters` reaches everything filed under `characters/` (including `characters/major/`),
65
+ and never `characters-cut/`.
66
+
67
+ If the POSIX verbs are unavailable, every step below still works with `search_notes`,
68
+ `read_note`, and `list_directory` — it just costs more.
69
+
29
70
  ## Phase 0: Setup
30
71
 
31
72
  ### Create the Project
32
73
 
33
74
  ```python
34
- create_memory_project(name="<work-name>", path="~/basic-memory/<work-name>")
75
+ create_memory_project(project_name="<work-name>", project_path="~/basic-memory/<work-name>")
35
76
  ```
36
77
 
37
78
  Use a kebab-case slug of the work's title (e.g., `great-gatsby`, `hamlet`, `beloved`).
@@ -212,6 +253,7 @@ Schema for literary technique and device notes.
212
253
 
213
254
  ```
214
255
  <project>/
256
+ <work>.txt # the source text, verbatim (see Phase 2)
215
257
  schema/ # 6 schema definitions
216
258
  chapters/ # one note per chapter/section + prologue/epilogue
217
259
  characters/
@@ -271,6 +313,46 @@ Stubs don't need to be complete — they give `[[wiki-link]]` targets and will b
271
313
 
272
314
  Obtain the full text and identify chapter/section boundaries. For public domain works, Project Gutenberg is a good source. For copyrighted works, work from a physical or licensed digital copy.
273
315
 
316
+ **Put the source text inside the project directory, as `.txt`, and index it once.** `bm cat`
317
+ resolves a *note identifier*, not a filesystem path — it can only reach a file the project
318
+ index has observed. A one-time index pass gives the raw text an entity row, after which the
319
+ line-range slice works against it:
320
+
321
+ ```bash
322
+ cp ~/Downloads/moby-dick.txt ~/basic-memory/moby-dick/moby-dick.txt
323
+ bm reindex --search -p moby-dick # one pass; the .txt becomes readable
324
+ ```
325
+
326
+ Two constraints that make this the right shape, both worth respecting:
327
+
328
+ - **Keep it `.txt`, do not convert it to `.md`.** Basic Memory injects frontmatter into
329
+ markdown notes, which shifts every line number by the height of that block — an offset map
330
+ built from the original file would then be silently wrong. A `.txt` is stored verbatim, so
331
+ its line numbers stay 1:1 with the file on disk.
332
+ - **Keep it inside the project.** A source text elsewhere on disk is not an entity, and
333
+ `bm cat` answers `Error: Entity not found`. If you must leave it outside, drop the BM verbs
334
+ for the source and use plain shell (`sed -n '4200,4890p' <path>`) — the notes still get the
335
+ BM verbs, only the raw source falls back to the shell.
336
+
337
+ **Then build a chapter offset map once, before processing.** Scan the text for chapter
338
+ headings and record the line range of each chapter, then read chapters by range rather than
339
+ re-reading the whole book into context:
340
+
341
+ ```bash
342
+ grep -n '^CHAPTER ' ~/basic-memory/moby-dick/moby-dick.txt # heading -> line number
343
+ bm cat moby-dick/moby-dick.txt --lines 4200-4890 --plain # one chapter, not the whole text
344
+ ```
345
+
346
+ `grep -n` here is the shell's grep on a filesystem path (this is the map-building step, and
347
+ it needs the file). `bm cat` then takes the *note identifier* and returns exactly that slice plus a
348
+ `lines 4200-4890 of N` footer. Spell the identifier project-qualified
349
+ (`<work>/<work>.txt`) so it routes to the right project. `bm head moby-dick/moby-dick.txt -n 40` is the cheap
350
+ way to eyeball the heading format before writing the grep pattern.
351
+
352
+ Store the map in the project (a note or a small JSON file) so later batches — and a resumed
353
+ run after context compaction — do not have to rediscover it. On a long work this is the
354
+ single largest context saving in the pipeline.
355
+
274
356
  ### Batching Strategy
275
357
 
276
358
  Process ~10 chapters per batch to balance depth with progress. Group by narrative arc or thematic focus:
@@ -289,7 +371,9 @@ Adjust batch size based on chapter length and density. Short, action-heavy chapt
289
371
 
290
372
  For each chapter:
291
373
 
292
- **1. Read the chapter carefully.** If working from a source text file, read the relevant section.
374
+ **1. Read the chapter carefully.** Read the chapter's line range from the offset map
375
+ (`bm cat <source>.txt --lines <start>-<end>`), not the whole file. Read the actual text —
376
+ never work from memory or a summary; textual evidence is the entire point.
293
377
 
294
378
  **2. Create the chapter note:**
295
379
 
@@ -332,13 +416,17 @@ write_note(
332
416
  ```python
333
417
  edit_note(
334
418
  identifier="characters/major/<character-slug>",
335
- operation="append",
336
- heading="Observations",
419
+ operation="insert_after_section",
420
+ section="Observations",
337
421
  content="""- [arc] Ch.<N>: <What happens to this character>
338
422
  - [quote] "<Attributed quote>" (Ch.<N>)"""
339
423
  )
340
424
  ```
341
425
 
426
+ `append` and `prepend` do not target a section — they write at the end of the note and the
427
+ start of the body. Use `insert_after_section` / `insert_before_section` to land lines under a
428
+ heading; they fail loudly if the heading is missing.
429
+
342
430
  **4. Track progress** using the memory-tasks skill to create a processing task that survives context compaction.
343
431
 
344
432
  ### What to Capture Per Chapter
@@ -364,7 +452,7 @@ As each chapter is processed, append observations to relevant entities:
364
452
 
365
453
  ### Adding Prose and Interpretation
366
454
 
367
- After the structured observations are in place, consider adding interpretive prose to major entity notes. Prepend 2-4 paragraphs of critical essay before the Observations section using `edit_note(operation="prepend")`. This prose should:
455
+ After the structured observations are in place, consider adding interpretive prose to major entity notes. Insert 2-4 paragraphs of critical essay before the Observations section with `edit_note(operation="insert_before_section", section="Observations", content=...)`. This prose should:
368
456
 
369
457
  - Argue for a reading of the character, theme, or symbol — not just describe it
370
458
  - Connect the entity to the work's larger concerns and to literary tradition
@@ -377,6 +465,25 @@ The prose adds the interpretive texture that structured observations alone canno
377
465
 
378
466
  After all chapters are processed:
379
467
 
468
+ ### Find What Needs Enriching
469
+
470
+ Do not re-read every note to decide what is thin. Query for it:
471
+
472
+ ```bash
473
+ bm find --meta 'note_type=chapter' --fields chapter_number,pov,setting --page-size 200
474
+ bm find --meta 'note_type=character' --fields role,status --page-size 200 # who is still a stub
475
+ bm find --meta 'chapter_number>100' --fields pov --page-size 200 # late-book POV drift
476
+ ```
477
+
478
+ A field a note never set comes back as a blank cell (`null` under `--json`), so rows with
479
+ blanks are the work queue. This turns "audit the graph" from a read of every note into one
480
+ call per question.
481
+
482
+ `write_note` snake-cases `note_type` (`Chapter` is stored as `chapter`), and `--meta`
483
+ type predicates are normalized the same way, so either spelling matches. `--page-size 200`
484
+ is not decoration: without it these return the first 10 rows and the work queue looks ten
485
+ items long.
486
+
380
487
  ### Character Arcs
381
488
  For each major character, write a full `[arc]` summary observation covering their trajectory across the work.
382
489
 
@@ -426,18 +533,18 @@ During chapter processing, new minor characters, locations, and symbols will eme
426
533
 
427
534
  ```python
428
535
  # Validate each entity type
429
- schema_validate(noteType="Character")
430
- schema_validate(noteType="Theme")
431
- schema_validate(noteType="Chapter")
432
- schema_validate(noteType="Location")
433
- schema_validate(noteType="Symbol")
434
- schema_validate(noteType="LiteraryDevice")
536
+ schema_validate(note_type="Character")
537
+ schema_validate(note_type="Theme")
538
+ schema_validate(note_type="Chapter")
539
+ schema_validate(note_type="Location")
540
+ schema_validate(note_type="Symbol")
541
+ schema_validate(note_type="LiteraryDevice")
435
542
  ```
436
543
 
437
544
  ### Drift Detection
438
545
 
439
546
  ```python
440
- schema_diff(noteType="Character")
547
+ schema_diff(note_type="Character")
441
548
  # ... for each type
442
549
  ```
443
550
 
@@ -446,26 +553,147 @@ Fix issues found — common fixes:
446
553
  - Enum values outside allowed set → correct metadata
447
554
  - Fields in notes but not schema → add as optional to schema if legitimate
448
555
 
556
+ ### Coverage Checks
557
+
558
+ Schema validation proves notes match their shape. These prove the graph is *complete*:
559
+
560
+ ```bash
561
+ bm find --meta 'note_type=chapter' --fields chapter_number --page-size 200 # every chapter present?
562
+ bm find --meta 'note_type=chapter' --fields pov,setting --page-size 200 # missing context?
563
+ bm find /characters --meta 'note_type=character' --fields role --page-size 200 # inventory vs. seed list
564
+ ```
565
+
566
+ **A coverage check that pages is not a coverage check.** `bm find` defaults to
567
+ `--page-size 10`, so the un-sized form of the first query "proves" a 138-chapter work has 10
568
+ chapters. 200 is the maximum page size; past that, iterate with `--page 2`, `--page 3`, … .
569
+ Scope a `--meta` query with the positional path (`/characters`), never `--name` — the two
570
+ options are mutually exclusive, because the metadata search has no filename glob. The
571
+ positional path scopes by the *file path* a note is indexed under, matched on a directory
572
+ boundary: `/characters` admits `characters/major/ahab.md` but never `characters-cut/`. It is
573
+ not a permalink match, so a note that pins its own `permalink:` is still found where its file
574
+ lives.
575
+
576
+ Read the count off the footer, not off the rows you can see. In a terminal every `find`
577
+ result reports `page 1 • total 138`, and appends `• more available (--page)` when the page
578
+ truncated the answer — that suffix appearing is the check *failing*, whatever the visible
579
+ rows say. **The footer is a TTY feature.** Piped output without `--plain` is JSON, which
580
+ carries `total` and `has_more`; with `--plain` the rows go to stdout and a has-more notice
581
+ goes to stderr. For scripts, read `has_more` from `--json`.
582
+
583
+ For the sequence gap — the failure that a count alone cannot catch — take the numbers from
584
+ `--json`, which carries `total`, `total_is_exact`, and `has_more`:
585
+
586
+ ```bash
587
+ expected=138; page=1; rows='[]'
588
+ while :; do
589
+ resp=$(bm find --meta 'note_type=chapter' --fields chapter_number \
590
+ --page-size 200 --page $page --project <work> --json)
591
+ rows=$(jq -n --argjson acc "$rows" --argjson r "$resp" \
592
+ '$acc + [$r.results[] | {title, n: .fields.chapter_number}]')
593
+ [ "$(jq -r '.has_more' <<<"$resp")" = "true" ] || break
594
+ page=$((page + 1))
595
+ done
596
+ jq -n --argjson rows "$rows" --argjson expected "$expected" '
597
+ ([$rows[] | select(.n != null and (.n | tostring | test("^[0-9]+$"))) | .n | tonumber]) as $n
598
+ | { total: ($rows | length),
599
+ unnumbered: [$rows[] | select(.n == null or (.n | tostring | test("^[0-9]+$") | not))
600
+ | .title],
601
+ missing: ([range(1; $expected + 1)] - $n),
602
+ duplicates: ($n | group_by(.) | map(select(length > 1) | .[0])),
603
+ out_of_range: ($n | map(select(. < 1 or . > $expected)) | unique) }'
604
+ ```
605
+
606
+ The loop is not ceremony. `--page-size` caps at 200, so a single call cannot inventory a work
607
+ with more than 200 chapters — and rerunning it with `--page 2` *replaces* the numbers rather
608
+ than accumulating them, which reports chapters 1-200 as missing on a corpus that is complete.
609
+ Walk until `has_more` is false and check the union.
610
+
611
+ `--fields` returns YAML-native types, so `chapter_number: 63` comes back as `63`. The
612
+ `tostring | test(...) | tonumber` guard still catches hand-typed values such as `"63"` or
613
+ `"12a"`.
614
+
615
+ Pass the work's **actual** chapter count as `$expected` — deriving the range from the highest
616
+ number found lets an incomplete graph pass. With 138 rows numbered 1..137 plus one duplicate,
617
+ a max-derived check reports `missing: []` while a chapter is genuinely absent: the duplicate
618
+ keeps the count right and the missing tail moves the goalpost. The check passes on
619
+ `unnumbered: []`, `missing: []`, `duplicates: []`, **and** `out_of_range: []` together, over
620
+ the combined pages.
621
+
622
+ `unnumbered` is not decoration either. A `chapter` note that never got a `chapter_number` comes
623
+ back as `null`, and feeding that straight to `tonumber` aborts the whole pipeline with `null
624
+ cannot be parsed as a number` — so the check *crashes on exactly the malformed inventory it
625
+ exists to find*. Partitioning first turns that into a named row.
626
+
627
+ `out_of_range` is not hypothetical: a prologue or epilogue typed as `chapter` lands at 0 or at
628
+ `$expected + 1`, and without that key the report reads clean — every expected number present,
629
+ none repeated — while the inventory holds a note the numbering does not account for. Type
630
+ front and back matter as its own note type, or widen `$expected` deliberately.
631
+
632
+ A gap in the middle of a batch is the most common processing failure and the easiest to miss
633
+ by eye; a duplicated chapter number is the second, and it hides the first.
634
+
449
635
  ### Relation Consistency
450
636
  Spot-check bidirectional relations: if Chapter X `features [[Character]]`, does Character have observations referencing Chapter X? Fix gaps.
451
637
 
452
- ## Phase 5: Visualization
638
+ Orphans are the other half of this check — a note with no inbound or outbound relations is
639
+ either genuinely isolated or was never linked back into the graph:
640
+
641
+ ```bash
642
+ bm orphans # entities with no relations in the graph
643
+ ```
644
+
645
+ `orphans` finds notes with no relations. It does not find a `[[target]]` that resolves to
646
+ nothing — a misspelled or renamed entity name. Check a chapter's links with `bm tool build-context memory://chapters/<slug> --depth 1
647
+ --project <work> --json` and look for relations whose `to_entity_id` is `null`; fix the
648
+ spelling or add the alias, then re-run the chapter.
649
+
650
+ Graph quality is relation *density*, not note count. A pass that adds notes while leaving
651
+ orphans behind has made the graph worse.
652
+
653
+ ## Phase 5: Explore the Graph
453
654
 
454
- Write [JSON Canvas](https://jsoncanvas.org/) files (`.canvas`) into the project directory for visual exploration in Obsidian. Query the graph first (`search_notes`, `build_context`), then lay out the results as canvas nodes and edges:
655
+ With the graph complete, traverse it to find what the chapter-by-chapter pass could not see:
455
656
 
456
- ```json
457
- {
458
- "nodes": [
459
- {"id": "ahab", "type": "file", "file": "characters/captain-ahab.md", "x": 0, "y": 0, "width": 400, "height": 300},
460
- {"id": "ishmael", "type": "file", "file": "characters/ishmael.md", "x": 500, "y": 0, "width": 400, "height": 300}
461
- ],
462
- "edges": [
463
- {"id": "e1", "fromNode": "ishmael", "toNode": "ahab", "label": "narrates"}
464
- ]
465
- }
657
+ ```bash
658
+ bm grep "features [[" --page-size 200 --project <work> --json # every chapter's cast, one call
659
+ bm find --meta 'note_type=theme' --fields prevalence --page-size 200 # thematic weight
660
+ bm grep "doubloon" --page-size 100 --project <work> # every mention of a symbol
466
661
  ```
467
662
 
468
- Useful canvases: character relationship web (protagonist/antagonist/supporting), theme connections, chapter timeline with key events.
663
+ "Which characters share the most chapters" is the first line plus a local parse of each
664
+ row's `content` for `features [[...]]` — one call instead of a read per chapter. Do not
665
+ reach for `bm tool build-context 'memory://characters/major/*'` here: a
666
+ wildcard context is capped at 100 related rows across all primaries and returns one primary
667
+ row per indexed observation, so it neither enumerates the cast nor walks the web.
668
+ `build-context` on a *single* note is the right tool for a different question, below.
669
+ `build-context` takes its URL as a positional argument — there is no `--url` option.
670
+
671
+ `grep` matches keywords and returns a page of 10, which quietly truncates "where does this
672
+ appear?" — a symbol in 40 chapters comes back as 10. For symbol tracing, raise
673
+ `--page-size`; the meaning shifts you are hunting are usually in the later occurrences, which
674
+ the default page would have dropped. `--semantic` ranks by meaning instead, which answers
675
+ "what is this about?" rather than "where does this word appear?".
676
+
677
+ Two more facts about `grep` rows. Matching is case-insensitive and note-level: a hit is a
678
+ note, not a line, and there is no `-n` or context. And each row's `content` is capped at
679
+ 4000 characters; `content_truncated: true` marks a cut row, so a long note's tail (such as a
680
+ final chapter's `follows`/`precedes` relations) may be missing from a grep-driven scan.
681
+ When a parse depends on the end of a note, `cat` that note.
682
+
683
+ `--page-size` raises the ceiling, it does not remove it. A symbol in a long work can exceed
684
+ even 100, so check whether the last page was full and walk `--page 2`, `--page 3` until it is
685
+ not. A truncated symbol search fails the same silent way as an unpaginated `find`: a plausible
686
+ answer, exit 0, and no sign that the tail is missing.
687
+
688
+ And `grep` searches **your notes, not the source**. The `<work>.txt` is indexed as an entity,
689
+ but its body is not in the searchable text, so an occurrence you never carried into a note is
690
+ unreachable: a word present only in the source returns `total: 0`. So this answers "where have I written about the doubloon", not
691
+ "where does the doubloon appear in the book". For the latter, search the file itself and use
692
+ the [chapter offset map](#source-text-preparation) to turn a hit into a chapter.
693
+
694
+ Traversal is where second-order questions get answered — which characters share the most
695
+ chapters, which themes converge in the final act, where a symbol's meaning shifts. Capture
696
+ what you find as `analysis/` notes; those syntheses are the payoff of having built the graph.
469
697
 
470
698
  ## Adapting to Other Genres
471
699
 
@@ -504,6 +732,8 @@ This pipeline works for any literary text. Adjust schemas for genre:
504
732
  - **Seed before processing.** Create entity stubs first so wiki-links resolve immediately during chapter processing.
505
733
  - **Batch for sanity.** Processing ~10 chapters at a time balances depth with momentum. Track progress with a Task note.
506
734
  - **Read the source text.** Don't rely on memory or summaries. Read (or re-read) the actual text for each batch before creating notes. Textual evidence is everything.
735
+ - **Read narrowly.** Keep the source text in the project as `.txt`, index it once, build the chapter offset map once, then read chapters by line range and notes by section. On a long work, whole files landing in context are the largest avoidable cost in the pipeline.
736
+ - **Query, don't scan.** When you need to know which notes have a field, ask with `--meta` predicates and `--fields` projection. Reading notes to check frontmatter is the mistake this pipeline makes at scale. One way these queries lie quietly: `find` returns 10 rows unless you pass `--page-size`.
507
737
  - **Observations are your index.** The knowledge graph's value comes from categorized observations. Be generous with categories and specific with content.
508
738
  - **Relations are your web.** Every chapter should link to characters, themes, locations, and devices. Every entity should link back to chapters where it appears.
509
739
  - **Enrich iteratively.** Entity notes grow richer with each chapter. Don't try to write the perfect character note upfront — append as you go.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: memory-metadata-search
3
- description: "Structured metadata search for Basic Memory: query notes by custom frontmatter fields using equality, range, array, and nested filters. Use when finding notes by status, priority, confidence, or any custom YAML field rather than free-text content."
3
+ description: "Search Basic Memory notes by custom frontmatter fields with equality, range, array, and nested filters. Use when finding notes by status, priority, confidence, or any YAML field, not free text."
4
4
  ---
5
5
 
6
6
  # Memory Metadata Search
@@ -56,6 +56,17 @@ Numeric values use numeric comparison; strings use lexicographic comparison.
56
56
  {"score": {"$between": [0.3, 0.8]}}
57
57
  ```
58
58
 
59
+ ### Null (field missing or explicitly null)
60
+
61
+ ```json
62
+ {"owner": null}
63
+ ```
64
+
65
+ Matches notes with no `owner` key and notes whose `owner` is explicitly null.
66
+ Null works only as a plain equality value — inside `$in`, `$between`, an
67
+ array-contains list, or a comparison it is rejected, because those compare
68
+ against the value and a comparison with null is never true.
69
+
59
70
  ### Nested Access (dot notation)
60
71
 
61
72
  ```json
@@ -67,20 +78,34 @@ Numeric values use numeric comparison; strings use lexicographic comparison.
67
78
  | Operator | Syntax | Example |
68
79
  |----------|--------|---------|
69
80
  | Equality | `{"field": "value"}` | `{"status": "active"}` |
81
+ | Is null | `{"field": null}` | `{"owner": null}` |
70
82
  | Array contains | `{"field": ["a", "b"]}` | `{"tags": ["security", "oauth"]}` |
71
83
  | `$in` | `{"field": {"$in": [...]}}` | `{"priority": {"$in": ["high", "critical"]}}` |
72
84
  | `$gt` / `$gte` | `{"field": {"$gt": N}}` | `{"confidence": {"$gt": 0.7}}` |
73
85
  | `$lt` / `$lte` | `{"field": {"$lt": N}}` | `{"score": {"$lt": 0.5}}` |
74
86
  | `$between` | `{"field": {"$between": [lo, hi]}}` | `{"score": {"$between": [0.3, 0.8]}}` |
87
+ | `$contains` | `{"field": {"$contains": value}}` | `{"tags": {"$contains": "security"}}` |
88
+ | `$exists` | `{"field": {"$exists": true}}` | `{"owner": {"$exists": false}}` |
75
89
  | Nested | `{"a.b": "value"}` | `{"schema.version": "2"}` |
76
90
 
77
91
  **Rules:**
78
92
  - Keys must match `[A-Za-z0-9_-]+` (dots separate nesting levels)
79
- - Operator dicts must contain exactly one operator
93
+ - Several operators on one field AND together: `{"started": {"$gte": "2026-01-01", "$lt": "2026-02-01"}}`
80
94
  - `$in` and array-contains require non-empty lists
81
95
  - `$between` requires exactly `[min, max]`
82
-
83
- > **Warning:** Operators MUST include the `$` prefix — write `$gte`, not `gte`. Without the prefix the filter is treated as an exact-match key and will silently return no results. Correct: `{"confidence": {"$gte": 0.7}}`. Wrong: `{"confidence": {"gte": 0.7}}`.
96
+ - `null` is an is-null match and only valid as a plain equality value
97
+ - Comparison and `$between` bounds must be finite numbers — a magnitude no float
98
+ can hold (a 400-digit integer, which JSON keeps as an ordinary `int`) is
99
+ refused rather than compared against an infinite bound
100
+ - Metadata filters match Markdown notes only — indexed PDFs, images and other
101
+ regular files carry no frontmatter and are never hits, not even for `null`
102
+
103
+ > The leading `$` is optional (`gte` works), but write it for clarity.
104
+
105
+ Frontmatter fields say when a note was filed or reviewed. To ask what a note
106
+ *claims was true* on a date, use observation qualifiers (see **memory-notes**)
107
+ and `search_notes(valid_at="2026-07-01")` or
108
+ `search_notes(valid_overlaps="[2026-07-01,2026-08-01)")`.
84
109
 
85
110
  ## Using `search_notes` with Metadata
86
111
 
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: memory-notes
3
- description: "How to write well-structured Basic Memory notes: frontmatter, observations with semantic categories, relations with wiki-links, and best practices for building a rich knowledge graph. Use when creating or improving notes."
3
+ description: "How to write well-structured Basic Memory notes: frontmatter, categorized observations, wiki-link relations, and knowledge-graph best practices. Use when creating or improving notes."
4
4
  ---
5
5
 
6
6
  # Memory Notes
@@ -49,7 +49,7 @@ permalink: custom-path # optional — auto-generated from title if omitted
49
49
  - The `title` must match the `# Heading` in the body
50
50
  - Tags are searchable and help with discovery
51
51
  - Custom `type` values (Task, Meeting, Person, etc.) work with the schema system. See the **memory-schema** skill for defining schemas, validating notes against them, and detecting drift.
52
- - The `permalink` is auto-generated from the `title` and `directory`. For example, title "API Design Decisions" in directory "specs" produces permalink `specs/api-design-decisions` and memory URL `memory://specs/api-design-decisions`. If no directory is specified, the permalink is just the kebab-cased title. Permalinks stay stable across file moves. You rarely need to set one manually.
52
+ - The `permalink` is auto-generated from the `title` and `directory`. For example, title "API Design Decisions" in directory "specs" produces permalink `specs/api-design-decisions` and memory URL `memory://specs/api-design-decisions`. If no directory is specified, the permalink is just the kebab-cased title. By default permalinks stay stable across file moves (unless the project enables `update_permalinks_on_move`). You rarely need to set one manually.
53
53
 
54
54
  > **Note:** When using `write_note`, you don't write frontmatter yourself. The `title`, `tags`, `note_type`, and `metadata` are separate parameters — Basic Memory generates the frontmatter automatically. Your `content` parameter is just the markdown body starting with `# Heading`.
55
55
 
@@ -99,7 +99,42 @@ A few examples to illustrate the range:
99
99
  - **One fact per observation.** Don't pack multiple ideas into one line.
100
100
  - **Be specific.** `[decision] Use JWT` is less useful than `[decision] Use JWT with 15-minute expiry for API auth`.
101
101
  - **Use tags for cross-cutting concerns.** `[risk] Rate limiting needed #api #security` makes this findable under both topics.
102
- - **Categories are queryable.** `search_notes("[decision]")` finds all decisions across your knowledge base.
102
+ - **Categories are queryable.** `search_notes(entity_types=["observation"], categories=["decision"])` returns every decision observation across your knowledge base.
103
+
104
+ ### When a Claim Holds
105
+
106
+ An observation can say when it is true. Put a qualifier after the category:
107
+
108
+ ```
109
+ - [decision] @effective:2026-06-10 The cache layer uses Redis
110
+ - [event] @occurred:2026-10-14 Launch sync moved the date to Thursday
111
+ - [deadline] @due:2026-10-20 Draft ready for review
112
+ - [stage] @valid:2026-10 Series B, about 200 employees
113
+ - [decision] @effective[2026-06-10,2026-10-14) Launch on Friday
114
+ ```
115
+
116
+ - **Kinds**: `effective` (a decision or rule takes effect), `occurred` (an
117
+ event happened), `due` (a commitment), `mentioned` (when it was said), and
118
+ `valid`, the default (`@2026-06-10` means `@valid:2026-06-10`).
119
+ - **Points** cover the span their precision names: `@2026` is that year,
120
+ `@2026-06` that month, and `@2026-06-10` from that date onward.
121
+ - **Ranges** use `[start,end)`, and either end can be open: `@effective[2026-06-10,)`.
122
+ - **Dates** are written out. An unquoted date is one token (`@occurred:2026-06-10`).
123
+ Quote anything longer: `@occurred:"June 10, 2026"`. A relative date like
124
+ `yesterday` is never read as a qualifier; it stays in the text.
125
+ - **When a fact changes**, close the old one's range and start the new one,
126
+ instead of deleting the old one. Both stay searchable by date.
127
+
128
+ Search by what notes say was true, rather than by when they were edited:
129
+
130
+ ```python
131
+ search_notes(valid_at="2026-07-01") # true on that date
132
+ search_notes(time_kind="due", valid_overlaps="[2026-10-14,2026-10-21)") # due this week
133
+ ```
134
+
135
+ A valid-time filter returns only dated observations; drop it to search
136
+ everything. Day points run "from that date onward", so a due-date window also
137
+ returns items that are already overdue.
103
138
 
104
139
  ## Relations
105
140
 
@@ -218,10 +253,10 @@ For people, try full name and last name. For organizations, try the full name an
218
253
  When a note already exists, make targeted edits instead of rewriting the whole file:
219
254
 
220
255
  ```python
221
- # Append a new observation to an existing note
256
+ # Add a new observation under the existing Observations heading
222
257
  edit_note(
223
258
  identifier="API Design Decisions",
224
- operation="append",
259
+ operation="insert_after_section",
225
260
  section="Observations",
226
261
  content="- [decision] Switched to OpenAPI 3.1 for spec generation #api"
227
262
  )
@@ -237,7 +272,7 @@ edit_note(
237
272
  # Add a new relation
238
273
  edit_note(
239
274
  identifier="API Design Decisions",
240
- operation="append",
275
+ operation="insert_after_section",
241
276
  section="Relations",
242
277
  content="- depends_on [[Rate Limiter]]"
243
278
  )
@@ -276,16 +311,20 @@ Basic Memory auto-generates frontmatter (including the permalink and memory URL)
276
311
 
277
312
  ### Editing an Existing Note
278
313
 
279
- Use `edit_note` to update a note in place — four operations:
314
+ Use `edit_note` to update a note in place — six operations: `append`, `prepend`,
315
+ `find_replace`, `replace_section`, `insert_before_section`, `insert_after_section`.
280
316
 
281
317
  ```python
282
- # append / prepend — add to the end or start (use for time-ordered logs)
318
+ # insert_after_section / insert_before_section — add lines at an existing heading
283
319
  edit_note(
284
320
  identifier="API Design Decisions",
285
- operation="append",
321
+ operation="insert_after_section",
286
322
  section="Observations",
287
323
  content="- [decision] Use OpenAPI 3.1 for spec generation #api"
288
324
  )
325
+
326
+ # append / prepend — add to the end of the note or the start of the body
327
+ # (use for time-ordered logs; neither one targets a section)
289
328
  edit_note(
290
329
  identifier="API Design Decisions",
291
330
  operation="prepend",
@@ -309,8 +348,23 @@ edit_note(
309
348
  )
310
349
  ```
311
350
 
312
- When an edit is destructive (`replace_section`, `find_replace`), it's good practice to
313
- read the note first and confirm the change before applying it.
351
+ When an edit is destructive (`replace_section`, `find_replace`), read the note first
352
+ and pass its checksum, so the edit applies only to the revision you read:
353
+
354
+ ```python
355
+ note = read_note(identifier="API Design Decisions", output_format="json")
356
+ edit_note(
357
+ identifier="API Design Decisions",
358
+ operation="replace_section",
359
+ section="Summary",
360
+ content="Concise, current summary of the decision and its rationale.",
361
+ expected_checksum=note["checksum"]
362
+ )
363
+ ```
364
+
365
+ A checksum conflict means someone changed the note after you read it. Read it
366
+ again and redo the edit against the new text. `write_note(overwrite=True)` takes
367
+ `expected_checksum` the same way.
314
368
 
315
369
  ### Moving a Note
316
370
 
@@ -323,7 +377,7 @@ move_note(
323
377
  )
324
378
  ```
325
379
 
326
- The permalink stays the same after a move, so all `[[wiki-links]]` and `memory://` URLs continue to resolve.
380
+ By default the permalink stays the same after a move, so links keep resolving. Projects with `update_permalinks_on_move` enabled rewrite it from the new path.
327
381
 
328
382
  ## Best Practices
329
383
 
@@ -333,10 +387,8 @@ The permalink stays the same after a move, so all `[[wiki-links]]` and `memory:/
333
387
 
334
388
  3. **Build incrementally.** Add to existing notes rather than creating duplicates. Use `edit_note` to append new observations or relations as you learn more.
335
389
 
336
- 4. **Review AI-generated content.** When an AI writes notes for you, review them for accuracy. The AI captures structure well but may miss nuance.
337
-
338
- 5. **Use consistent titles.** Note titles are identifiers in the knowledge graph. `API Design Decisions` and `Api Design decisions` are different entities. Pick a convention and stick with it.
390
+ 4. **Use consistent titles.** Note titles are identifiers in the knowledge graph. `API Design Decisions` and `Api Design decisions` are different entities. Pick a convention and stick with it.
339
391
 
340
- 6. **Link related concepts.** The value of a knowledge graph compounds with connections. A note with zero relations is an island — useful, but not as powerful as a connected one.
392
+ 5. **Link related concepts.** The value of a knowledge graph compounds with connections. A note with zero relations is an island — useful, but not as powerful as a connected one.
341
393
 
342
- 7. **Let the graph grow naturally.** Don't try to design a perfect taxonomy upfront. Write notes as you work, add relations as connections emerge, and periodically use `/reflect` or `/defrag` to consolidate.
394
+ 6. **Let the graph grow naturally.** Don't try to design a perfect taxonomy upfront. Write notes as you work, add relations as connections emerge, and periodically use the **memory-reflect** or **memory-defrag** skills to consolidate.