wiki-graph 0.3.1 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/README.md +17 -0
  2. package/dist/cli.js +2221 -37548
  3. package/dist/cli.js.map +1 -1
  4. package/dist/data/editor/text_compressor.jinja +17 -13
  5. package/dist/data/help/commands/archive/next.jinja +7 -2
  6. package/dist/data/help/commands/gc.jinja +2 -1
  7. package/dist/data/help/commands/legacy/migrate.jinja +6 -1
  8. package/dist/data/help/commands/legacy.jinja +3 -2
  9. package/dist/data/help/commands/library-predicate.jinja +296 -0
  10. package/dist/data/help/commands/library.jinja +237 -0
  11. package/dist/data/help/commands/maintenance/chapter/add.jinja +3 -3
  12. package/dist/data/help/commands/maintenance/chapter/list.jinja +4 -4
  13. package/dist/data/help/commands/maintenance/chapter/move.jinja +4 -4
  14. package/dist/data/help/commands/maintenance/chapter/remove.jinja +1 -1
  15. package/dist/data/help/commands/maintenance/chapter/reset.jinja +1 -1
  16. package/dist/data/help/commands/maintenance/chapter/set-source.jinja +2 -2
  17. package/dist/data/help/commands/maintenance/chapter/set-summary.jinja +1 -1
  18. package/dist/data/help/commands/maintenance/chapter/set-title.jinja +3 -3
  19. package/dist/data/help/commands/maintenance/chapter/tree.jinja +6 -6
  20. package/dist/data/help/commands/maintenance/chapter.jinja +10 -10
  21. package/dist/data/help/commands/maintenance/meta.jinja +7 -7
  22. package/dist/data/help/commands/maintenance/upgrade.jinja +35 -0
  23. package/dist/data/help/commands/maintenance.jinja +23 -0
  24. package/dist/data/help/commands/predicate.jinja +204 -28
  25. package/dist/data/help/commands/root.jinja +10 -3
  26. package/dist/data/help/commands/uri.jinja +165 -6
  27. package/dist/data/help/topics/config.jinja +10 -3
  28. package/dist/data/help/topics/library.jinja +90 -0
  29. package/dist/data/help/topics/readiness.jinja +32 -2
  30. package/dist/data/help/topics/recipe.jinja +42 -17
  31. package/dist/data/help/topics/runtime.jinja +20 -3
  32. package/dist/data/help/topics/uri.jinja +53 -25
  33. package/dist/gc-worker.d.ts +1 -0
  34. package/dist/gc-worker.js +548 -0
  35. package/dist/gc-worker.js.map +1 -0
  36. package/dist/index.cjs +12678 -33444
  37. package/dist/index.cjs.map +1 -1
  38. package/dist/index.d.cts +36 -2649
  39. package/dist/index.d.ts +36 -2649
  40. package/dist/index.js +12901 -33331
  41. package/dist/index.js.map +1 -1
  42. package/dist/queue-worker.js +1367 -24052
  43. package/dist/queue-worker.js.map +1 -1
  44. package/package.json +15 -3
@@ -25,7 +25,7 @@ Generation job behavior:
25
25
  - Reading Graph, Reading Summary, and Knowledge Graph generation run as device-local jobs.
26
26
  - Generation jobs consume LLM tokens, may incur provider charges, and can run for minutes to hours on large archives.
27
27
  - `wikg://local/job add` requires `--accept-cost` as an explicit cost/time acknowledgement.
28
- - Job records live under `~/.{{ commandName }}/jobs`.
28
+ - Job records live under `~/.wikigraph/jobs`.
29
29
  - `wikg://local/job add`, `wikg://local/job/<job-id> resume`, `wikg://local/job/<job-id> boost`, and `wikg://local/job/<job-id>/target set` try to start a transient worker.
30
30
  - The worker exits after idle time; it is not intended to be a permanent daemon.
31
31
  - `wikg://local/job/<job-id> watch --jsonl` is the stable agent-facing progress stream.
@@ -41,15 +41,30 @@ Progress and diagnostics:
41
41
  Workspace behavior:
42
42
  - Archive creation and EPUB import use temporary workspaces.
43
43
  - `transform --digest-dir <path>` uses the provided digest workspace for source digest runs.
44
- - Generation jobs use per-job directories under `~/.{{ commandName }}/jobs/work`, `~/.{{ commandName }}/jobs/cache`, and `~/.{{ commandName }}/jobs/logs`.
44
+ - Generation jobs use per-job directories under `~/.wikigraph/jobs/work`, `~/.wikigraph/jobs/cache`, and `~/.wikigraph/jobs/logs`.
45
45
  - Succeeded jobs delete their workspace by default.
46
46
  - Failed and canceled jobs keep their workspace for debugging until cleanup.
47
47
  - Job events, logs, and LLM cache are kept until the job record is cleaned.
48
48
 
49
+ Local state map:
50
+ - `~/.wikigraph/core.sqlite`: home registry and shared local state.
51
+ - `~/.wikigraph/cache/search-sessions.sqlite`: search query cache sessions used by paged search results.
52
+ - `~/.wikigraph/cache/continuation-cursors.sqlite`: opaque `wg next` continuation cursor records.
53
+ - `~/.wikigraph/cache/cache.sqlite`: Wikipedia page and other reusable local cache records.
54
+ - `~/.wikigraph/jobs`: durable generation job records, events, workspaces, caches, and logs.
55
+ - `~/.wikigraph/jobs/work`: per-job workspaces; failed and canceled jobs keep these for debugging.
56
+ - `~/.wikigraph/jobs/cache`: per-job LLM/cache artifacts kept with job records until cleanup.
57
+ - `~/.wikigraph/jobs/logs`: per-job diagnostic logs.
58
+ - `~/.wikigraph/staging`: archive coordinator state, materialized archive entries, local archive index staging, and library aggregate indexes.
59
+ - Search query caches and continuation cursors are local derived state; use `{{ commandName }} next --help` for cursor continuation and `{{ commandName }} gc --help` for cleanup.
60
+ - Do not manually delete active job, staging, or cursor state while commands are running; prefer `{{ commandName }} gc` and command-specific cleanup paths.
61
+
49
62
  In-place archive edits:
50
63
  - Mutating `.wikg` commands go through the archive coordinator.
51
64
  - Active generation jobs block destructive chapter maintenance edits for the same chapter.
52
- - Archive writes are coordinated through `~/.{{ commandName }}/staging`.
65
+ - Archive writes are coordinated through `~/.wikigraph/staging`.
66
+ - Library aggregate indexes are local derived state under staging and can be removed by cleanup when orphaned.
67
+ - Library cursors store the library index scope and re-check index freshness before continuing.
53
68
 
54
69
  Where to go next:
55
70
  - Need the normal workflow before debugging:
@@ -60,3 +75,5 @@ Where to go next:
60
75
  {{ commandName }} help format
61
76
  - Need URI command routing:
62
77
  {{ commandName }} help uri
78
+ - Need library registry, membership, path binding, or index behavior:
79
+ {{ commandName }} help library
@@ -8,7 +8,8 @@ Purpose:
8
8
  Use this page to choose a target, understand scope/object behavior, and select retrieval shape.
9
9
 
10
10
  Archive locator:
11
- - Archive/object command URIs must contain a `.wikg` locator.
11
+ - Normal archive/object command URIs contain a `.wikg` locator.
12
+ - Library URIs and local runtime URIs are separate URI families and do not contain `.wikg` locators.
12
13
  - Do not pass a bare filesystem path as a command target.
13
14
  /Users/me/book.wikg -> wikg:///Users/me/book.wikg
14
15
  ./book.wikg -> wikg://book.wikg
@@ -19,17 +20,31 @@ Archive locator:
19
20
  - Home archive locators may start with `wikg://~/`; `~` is expanded only in that leading form.
20
21
  - The first path segment ending in `.wikg` is the archive/object boundary.
21
22
 
23
+ Library locators:
24
+ - `wikg://lib` addresses the default library.
25
+ - `wikg://lib/<lib-id>` addresses a non-default library.
26
+ - Naked library scopes enumerate library-wide objects; `--query` on a naked library scope searches library-wide objects through the library index.
27
+ - Use explicit object lenses such as `/entity`, `/triple`, `/chapter`, or `/chunk` for archive object search.
28
+ - `wikg://lib/arc/<archive-id>/` addresses one archive membership in the default library.
29
+ - `wikg://lib/<lib-id>/arc/<archive-id>/` addresses one archive membership in a non-default library.
30
+ - Add ordinary archive object suffixes after the trailing slash to query a managed archive:
31
+ wikg://lib/arc/<archive-id>/chapter
32
+ wikg://lib/arc/<archive-id>/entity
33
+ wikg://lib/arc/<archive-id>/triple
34
+ - Library-wide scopes such as `wikg://lib/entity` query the aggregate library index.
35
+ - Read `{{ commandName }} help library` before managing registry, membership, path binding, or library index lifecycle.
36
+
22
37
  Short and located URIs:
23
38
  Commands may return short object URIs without local file paths:
24
- wikg://chapter/3/title
25
- wikg://chapter/3/source#4..8
39
+ wikg://chapter/part/title
40
+ wikg://chapter/part/source#4..8
26
41
  wikg://chunk/12
27
42
  wikg://entity/Q8018
28
43
  wikg://triple/Q8018/discusses/Q123
29
44
  Returned short URIs are archive-relative handles, not complete command targets.
30
45
  Add the archive locator before passing them back to commands:
31
46
  wikg://book.wikg/entity/Q8018
32
- wikg:///books/book.wikg/chapter/3/source#4..8
47
+ wikg:///books/book.wikg/chapter/part/source#4..8
33
48
 
34
49
  Command routing:
35
50
  - `{{ commandName }} <scope-uri>` enumerates objects.
@@ -50,21 +65,27 @@ Common command shapes:
50
65
  {{ commandName }} <archive-uri> export --output-format <format>
51
66
  {{ commandName }} <archive-uri> inspect
52
67
  {{ commandName }} transform [options]
68
+ {{ commandName }} wikg://lib --help
69
+ {{ commandName }} wikg://lib/entity --query <query>
70
+ {{ commandName }} wikg://lib/arc/<archive-id>/entity --query <query>
53
71
  {{ commandName }} wikg://local/job
54
72
  {{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task <reading-graph|reading-summary|knowledge-graph> --accept-cost
55
73
  {{ commandName }} wikg://local/job/<job-id>
56
- {{ commandName }} <object-uri> related
57
- {{ commandName }} <object-uri> evidence
58
- {{ commandName }} <object-uri> pack --budget 5000
59
74
 
60
75
  Scope URIs:
61
76
  Scope URIs enumerate objects by default. With `--query <query>`, they apply keyword retrieval within that scope.
77
+ A chapter URI used as a scope covers that chapter's full subtree by default.
78
+ Add `--depth <N>` to limit chapter subtree expansion:
79
+ - no `--depth`: full subtree
80
+ - `--depth 0`: only the current chapter, or root-level chapters for `<archive-uri>/chapter`
81
+ - `--depth 1`: current/root-level chapters plus direct children
82
+ `--depth` must include a non-negative integer value; naked `--depth`, negative values, and non-integers are invalid.
62
83
  wikg://book.wikg
63
84
  wikg://book.wikg/chapter
64
- wikg://book.wikg/chapter/3
65
- wikg://book.wikg/chapter/3/chunk
66
- wikg://book.wikg/chapter/3/entity
67
- wikg://book.wikg/chapter/3/triple
85
+ wikg://book.wikg/chapter/part
86
+ wikg://book.wikg/chapter/part/chunk
87
+ wikg://book.wikg/chapter/part/entity
88
+ wikg://book.wikg/chapter/part/triple
68
89
  wikg://book.wikg/chunk
69
90
  wikg://book.wikg/entity
70
91
  wikg://book.wikg/triple
@@ -74,15 +95,15 @@ Scope URIs:
74
95
 
75
96
  Object URIs:
76
97
  Object URIs read one object by default.
77
- wikg://book.wikg/chapter/3/title
78
- wikg://book.wikg/chapter/3/state
79
- wikg://book.wikg/chapter/3/source
80
- wikg://book.wikg/chapter/3/source#4..8
81
- wikg://book.wikg/chapter/3/summary
82
- wikg://book.wikg/chapter/3/summary#4..8
83
- wikg://book.wikg/chapter/3/chunk/12
84
- wikg://book.wikg/chapter/3/entity/Q8018
85
- wikg://book.wikg/chapter/3/triple/Q8018/discusses/Q123
98
+ wikg://book.wikg/chapter/part/title
99
+ wikg://book.wikg/chapter/part/state
100
+ wikg://book.wikg/chapter/part/source
101
+ wikg://book.wikg/chapter/part/source#4..8
102
+ wikg://book.wikg/chapter/part/summary
103
+ wikg://book.wikg/chapter/part/summary#4..8
104
+ wikg://book.wikg/chapter/part/chunk/12
105
+ wikg://book.wikg/chapter/part/entity/Q8018
106
+ wikg://book.wikg/chapter/part/triple/Q8018/discusses/Q123
86
107
  wikg://book.wikg/chunk/12
87
108
  wikg://book.wikg/entity/Q8018
88
109
  wikg://book.wikg/entity/Q8018/wikipage
@@ -99,9 +120,12 @@ Triple pattern scopes:
99
120
  A complete triple URI with concrete subject, predicate, and object is an object URI.
100
121
 
101
122
  Object boundaries:
102
- - `chapter/<id>` is a scope, not an object.
103
- - `chapter/<id>/title` is the queryable chapter title object.
123
+ - `chapter/<path>` is a scope, not an object.
124
+ - `chapter/<path>/title` is the queryable chapter title object.
104
125
  - Whole `source` and `summary` objects are readable text streams, not collection scopes.
126
+ - Chapter-qualified scope URIs such as `<chapter-uri>/chunk`, `<chapter-uri>/entity`, and `<chapter-uri>/triple` expand to the chapter subtree and support `--depth`.
127
+ - Chapter-qualified resource and object URIs such as `/source`, `/summary`, `/title`, `/state`, `/chunk/<id>`, `/entity/<qid>`, and complete `/triple/<subject>/<predicate>/<object>` do not expand to subtrees and do not support `--depth`.
128
+ - `move`, `remove`, and `reset` are chapter node maintenance predicates; they target the chapter node itself, not the subtree. Use `remove --recursive` for explicit recursive deletion.
105
129
  - Ranged fragments such as `/source#4..8` and `/summary#4..8` are structured range objects.
106
130
  - Source and summary range numbers are 1-based and inclusive. `/source#1..2` means the first two source sentences.
107
131
  - Source text is the grounding material. `evidence` is not available for source, summary, or chapter scope URIs.
@@ -126,13 +150,13 @@ Lens selection:
126
150
  - `/chunk` targets Reading Graph chunks.
127
151
  - `/entity` targets grounded entities.
128
152
  - `/triple` targets Knowledge Graph relations.
129
- - `chapter/<id>` is a scope; `--query` over chapter titles returns `chapter/<id>/title` objects.
153
+ - `chapter/<path>` is a scope; `--query` over chapter titles returns `chapter/<path>/title` objects.
130
154
 
131
155
  Literal source passages:
132
156
  Use `--query` on the narrowest useful scope when the task is to find source wording or passages mentioning a literal term:
133
157
  {{ commandName }} <archive-uri> --query "term"
134
- {{ commandName }} <archive-uri>/chapter/<id> --query "term"
135
- {{ commandName }} <archive-uri>/chapter/<id>/source#20..30
158
+ {{ commandName }} <archive-uri>/chapter/<path> --query "term"
159
+ {{ commandName }} <archive-uri>/chapter/<path>/source#20..30
136
160
 
137
161
  Entity-grounded evidence:
138
162
  Use entity evidence when you already have an entity URI or QID and need grounded mentions, aliases, translations, variants, or source-backed support:
@@ -143,6 +167,7 @@ Entity-grounded evidence:
143
167
  Pagination:
144
168
  - Use default paging or `--limit <n>` while exploring.
145
169
  - Use `{{ commandName }} next <cursor>` to continue a paged result.
170
+ - Use `{{ commandName }} next <uri> <cursor>` when you want the cursor's stored index scope checked against the archive or library URI you expect.
146
171
  - Use `--all` only when you intentionally need every item from the selected scope.
147
172
  - Prefer the narrowest scope before using `--all`.
148
173
  - Read `{{ commandName }} help format` before choosing text, `--json`, `--jsonl`, or `--all --jsonl` for machine consumption.
@@ -158,6 +183,7 @@ Recovery hints:
158
183
  {{ commandName }} help readiness
159
184
  {{ commandName }} help runtime
160
185
  - URI rejected: convert bare filesystem paths to Wiki Graph URIs, and prepend the archive locator to short object URIs.
186
+ - Library URI rejected: non-default libraries use wikg://lib/<lib-id>; archive members use <lib-scope>/arc/<archive-id>.
161
187
  - Help did not appear: append `--help` to the URI or URI-bound predicate you are trying to inspect.
162
188
 
163
189
  Job URIs:
@@ -176,3 +202,5 @@ Related help:
176
202
  {{ commandName }} help format
177
203
  - Local jobs, cache, logs, and runtime debugging:
178
204
  {{ commandName }} help runtime
205
+ - Library registry, membership, path binding, and aggregate index:
206
+ {{ commandName }} help library
@@ -0,0 +1 @@
1
+ #!/usr/bin/env node