wiki-graph 0.3.1 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -0
- package/dist/cli.js +2221 -37548
- package/dist/cli.js.map +1 -1
- package/dist/data/editor/text_compressor.jinja +17 -13
- package/dist/data/help/commands/archive/next.jinja +7 -2
- package/dist/data/help/commands/gc.jinja +2 -1
- package/dist/data/help/commands/legacy/migrate.jinja +6 -1
- package/dist/data/help/commands/legacy.jinja +3 -2
- package/dist/data/help/commands/library-predicate.jinja +296 -0
- package/dist/data/help/commands/library.jinja +237 -0
- package/dist/data/help/commands/maintenance/chapter/add.jinja +3 -3
- package/dist/data/help/commands/maintenance/chapter/list.jinja +4 -4
- package/dist/data/help/commands/maintenance/chapter/move.jinja +4 -4
- package/dist/data/help/commands/maintenance/chapter/remove.jinja +1 -1
- package/dist/data/help/commands/maintenance/chapter/reset.jinja +1 -1
- package/dist/data/help/commands/maintenance/chapter/set-source.jinja +2 -2
- package/dist/data/help/commands/maintenance/chapter/set-summary.jinja +1 -1
- package/dist/data/help/commands/maintenance/chapter/set-title.jinja +3 -3
- package/dist/data/help/commands/maintenance/chapter/tree.jinja +6 -6
- package/dist/data/help/commands/maintenance/chapter.jinja +10 -10
- package/dist/data/help/commands/maintenance/meta.jinja +7 -7
- package/dist/data/help/commands/maintenance/upgrade.jinja +35 -0
- package/dist/data/help/commands/maintenance.jinja +23 -0
- package/dist/data/help/commands/predicate.jinja +204 -28
- package/dist/data/help/commands/root.jinja +10 -3
- package/dist/data/help/commands/uri.jinja +165 -6
- package/dist/data/help/topics/config.jinja +10 -3
- package/dist/data/help/topics/library.jinja +90 -0
- package/dist/data/help/topics/readiness.jinja +32 -2
- package/dist/data/help/topics/recipe.jinja +42 -17
- package/dist/data/help/topics/runtime.jinja +20 -3
- package/dist/data/help/topics/uri.jinja +53 -25
- package/dist/gc-worker.d.ts +1 -0
- package/dist/gc-worker.js +548 -0
- package/dist/gc-worker.js.map +1 -0
- package/dist/index.cjs +12678 -33444
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +36 -2649
- package/dist/index.d.ts +36 -2649
- package/dist/index.js +12901 -33331
- package/dist/index.js.map +1 -1
- package/dist/queue-worker.js +1367 -24052
- package/dist/queue-worker.js.map +1 -1
- package/package.json +15 -3
|
@@ -25,7 +25,7 @@ Generation job behavior:
|
|
|
25
25
|
- Reading Graph, Reading Summary, and Knowledge Graph generation run as device-local jobs.
|
|
26
26
|
- Generation jobs consume LLM tokens, may incur provider charges, and can run for minutes to hours on large archives.
|
|
27
27
|
- `wikg://local/job add` requires `--accept-cost` as an explicit cost/time acknowledgement.
|
|
28
|
-
- Job records live under `~/.
|
|
28
|
+
- Job records live under `~/.wikigraph/jobs`.
|
|
29
29
|
- `wikg://local/job add`, `wikg://local/job/<job-id> resume`, `wikg://local/job/<job-id> boost`, and `wikg://local/job/<job-id>/target set` try to start a transient worker.
|
|
30
30
|
- The worker exits after idle time; it is not intended to be a permanent daemon.
|
|
31
31
|
- `wikg://local/job/<job-id> watch --jsonl` is the stable agent-facing progress stream.
|
|
@@ -41,15 +41,30 @@ Progress and diagnostics:
|
|
|
41
41
|
Workspace behavior:
|
|
42
42
|
- Archive creation and EPUB import use temporary workspaces.
|
|
43
43
|
- `transform --digest-dir <path>` uses the provided digest workspace for source digest runs.
|
|
44
|
-
- Generation jobs use per-job directories under `~/.
|
|
44
|
+
- Generation jobs use per-job directories under `~/.wikigraph/jobs/work`, `~/.wikigraph/jobs/cache`, and `~/.wikigraph/jobs/logs`.
|
|
45
45
|
- Succeeded jobs delete their workspace by default.
|
|
46
46
|
- Failed and canceled jobs keep their workspace for debugging until cleanup.
|
|
47
47
|
- Job events, logs, and LLM cache are kept until the job record is cleaned.
|
|
48
48
|
|
|
49
|
+
Local state map:
|
|
50
|
+
- `~/.wikigraph/core.sqlite`: home registry and shared local state.
|
|
51
|
+
- `~/.wikigraph/cache/search-sessions.sqlite`: search query cache sessions used by paged search results.
|
|
52
|
+
- `~/.wikigraph/cache/continuation-cursors.sqlite`: opaque `wg next` continuation cursor records.
|
|
53
|
+
- `~/.wikigraph/cache/cache.sqlite`: Wikipedia page and other reusable local cache records.
|
|
54
|
+
- `~/.wikigraph/jobs`: durable generation job records, events, workspaces, caches, and logs.
|
|
55
|
+
- `~/.wikigraph/jobs/work`: per-job workspaces; failed and canceled jobs keep these for debugging.
|
|
56
|
+
- `~/.wikigraph/jobs/cache`: per-job LLM/cache artifacts kept with job records until cleanup.
|
|
57
|
+
- `~/.wikigraph/jobs/logs`: per-job diagnostic logs.
|
|
58
|
+
- `~/.wikigraph/staging`: archive coordinator state, materialized archive entries, local archive index staging, and library aggregate indexes.
|
|
59
|
+
- Search query caches and continuation cursors are local derived state; use `{{ commandName }} next --help` for cursor continuation and `{{ commandName }} gc --help` for cleanup.
|
|
60
|
+
- Do not manually delete active job, staging, or cursor state while commands are running; prefer `{{ commandName }} gc` and command-specific cleanup paths.
|
|
61
|
+
|
|
49
62
|
In-place archive edits:
|
|
50
63
|
- Mutating `.wikg` commands go through the archive coordinator.
|
|
51
64
|
- Active generation jobs block destructive chapter maintenance edits for the same chapter.
|
|
52
|
-
- Archive writes are coordinated through `~/.
|
|
65
|
+
- Archive writes are coordinated through `~/.wikigraph/staging`.
|
|
66
|
+
- Library aggregate indexes are local derived state under staging and can be removed by cleanup when orphaned.
|
|
67
|
+
- Library cursors store the library index scope and re-check index freshness before continuing.
|
|
53
68
|
|
|
54
69
|
Where to go next:
|
|
55
70
|
- Need the normal workflow before debugging:
|
|
@@ -60,3 +75,5 @@ Where to go next:
|
|
|
60
75
|
{{ commandName }} help format
|
|
61
76
|
- Need URI command routing:
|
|
62
77
|
{{ commandName }} help uri
|
|
78
|
+
- Need library registry, membership, path binding, or index behavior:
|
|
79
|
+
{{ commandName }} help library
|
|
@@ -8,7 +8,8 @@ Purpose:
|
|
|
8
8
|
Use this page to choose a target, understand scope/object behavior, and select retrieval shape.
|
|
9
9
|
|
|
10
10
|
Archive locator:
|
|
11
|
-
-
|
|
11
|
+
- Normal archive/object command URIs contain a `.wikg` locator.
|
|
12
|
+
- Library URIs and local runtime URIs are separate URI families and do not contain `.wikg` locators.
|
|
12
13
|
- Do not pass a bare filesystem path as a command target.
|
|
13
14
|
/Users/me/book.wikg -> wikg:///Users/me/book.wikg
|
|
14
15
|
./book.wikg -> wikg://book.wikg
|
|
@@ -19,17 +20,31 @@ Archive locator:
|
|
|
19
20
|
- Home archive locators may start with `wikg://~/`; `~` is expanded only in that leading form.
|
|
20
21
|
- The first path segment ending in `.wikg` is the archive/object boundary.
|
|
21
22
|
|
|
23
|
+
Library locators:
|
|
24
|
+
- `wikg://lib` addresses the default library.
|
|
25
|
+
- `wikg://lib/<lib-id>` addresses a non-default library.
|
|
26
|
+
- Naked library scopes enumerate library-wide objects; `--query` on a naked library scope searches library-wide objects through the library index.
|
|
27
|
+
- Use explicit object lenses such as `/entity`, `/triple`, `/chapter`, or `/chunk` for archive object search.
|
|
28
|
+
- `wikg://lib/arc/<archive-id>/` addresses one archive membership in the default library.
|
|
29
|
+
- `wikg://lib/<lib-id>/arc/<archive-id>/` addresses one archive membership in a non-default library.
|
|
30
|
+
- Add ordinary archive object suffixes after the trailing slash to query a managed archive:
|
|
31
|
+
wikg://lib/arc/<archive-id>/chapter
|
|
32
|
+
wikg://lib/arc/<archive-id>/entity
|
|
33
|
+
wikg://lib/arc/<archive-id>/triple
|
|
34
|
+
- Library-wide scopes such as `wikg://lib/entity` query the aggregate library index.
|
|
35
|
+
- Read `{{ commandName }} help library` before managing registry, membership, path binding, or library index lifecycle.
|
|
36
|
+
|
|
22
37
|
Short and located URIs:
|
|
23
38
|
Commands may return short object URIs without local file paths:
|
|
24
|
-
wikg://chapter/
|
|
25
|
-
wikg://chapter/
|
|
39
|
+
wikg://chapter/part/title
|
|
40
|
+
wikg://chapter/part/source#4..8
|
|
26
41
|
wikg://chunk/12
|
|
27
42
|
wikg://entity/Q8018
|
|
28
43
|
wikg://triple/Q8018/discusses/Q123
|
|
29
44
|
Returned short URIs are archive-relative handles, not complete command targets.
|
|
30
45
|
Add the archive locator before passing them back to commands:
|
|
31
46
|
wikg://book.wikg/entity/Q8018
|
|
32
|
-
wikg:///books/book.wikg/chapter/
|
|
47
|
+
wikg:///books/book.wikg/chapter/part/source#4..8
|
|
33
48
|
|
|
34
49
|
Command routing:
|
|
35
50
|
- `{{ commandName }} <scope-uri>` enumerates objects.
|
|
@@ -50,21 +65,27 @@ Common command shapes:
|
|
|
50
65
|
{{ commandName }} <archive-uri> export --output-format <format>
|
|
51
66
|
{{ commandName }} <archive-uri> inspect
|
|
52
67
|
{{ commandName }} transform [options]
|
|
68
|
+
{{ commandName }} wikg://lib --help
|
|
69
|
+
{{ commandName }} wikg://lib/entity --query <query>
|
|
70
|
+
{{ commandName }} wikg://lib/arc/<archive-id>/entity --query <query>
|
|
53
71
|
{{ commandName }} wikg://local/job
|
|
54
72
|
{{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task <reading-graph|reading-summary|knowledge-graph> --accept-cost
|
|
55
73
|
{{ commandName }} wikg://local/job/<job-id>
|
|
56
|
-
{{ commandName }} <object-uri> related
|
|
57
|
-
{{ commandName }} <object-uri> evidence
|
|
58
|
-
{{ commandName }} <object-uri> pack --budget 5000
|
|
59
74
|
|
|
60
75
|
Scope URIs:
|
|
61
76
|
Scope URIs enumerate objects by default. With `--query <query>`, they apply keyword retrieval within that scope.
|
|
77
|
+
A chapter URI used as a scope covers that chapter's full subtree by default.
|
|
78
|
+
Add `--depth <N>` to limit chapter subtree expansion:
|
|
79
|
+
- no `--depth`: full subtree
|
|
80
|
+
- `--depth 0`: only the current chapter, or root-level chapters for `<archive-uri>/chapter`
|
|
81
|
+
- `--depth 1`: current/root-level chapters plus direct children
|
|
82
|
+
`--depth` must include a non-negative integer value; naked `--depth`, negative values, and non-integers are invalid.
|
|
62
83
|
wikg://book.wikg
|
|
63
84
|
wikg://book.wikg/chapter
|
|
64
|
-
wikg://book.wikg/chapter/
|
|
65
|
-
wikg://book.wikg/chapter/
|
|
66
|
-
wikg://book.wikg/chapter/
|
|
67
|
-
wikg://book.wikg/chapter/
|
|
85
|
+
wikg://book.wikg/chapter/part
|
|
86
|
+
wikg://book.wikg/chapter/part/chunk
|
|
87
|
+
wikg://book.wikg/chapter/part/entity
|
|
88
|
+
wikg://book.wikg/chapter/part/triple
|
|
68
89
|
wikg://book.wikg/chunk
|
|
69
90
|
wikg://book.wikg/entity
|
|
70
91
|
wikg://book.wikg/triple
|
|
@@ -74,15 +95,15 @@ Scope URIs:
|
|
|
74
95
|
|
|
75
96
|
Object URIs:
|
|
76
97
|
Object URIs read one object by default.
|
|
77
|
-
wikg://book.wikg/chapter/
|
|
78
|
-
wikg://book.wikg/chapter/
|
|
79
|
-
wikg://book.wikg/chapter/
|
|
80
|
-
wikg://book.wikg/chapter/
|
|
81
|
-
wikg://book.wikg/chapter/
|
|
82
|
-
wikg://book.wikg/chapter/
|
|
83
|
-
wikg://book.wikg/chapter/
|
|
84
|
-
wikg://book.wikg/chapter/
|
|
85
|
-
wikg://book.wikg/chapter/
|
|
98
|
+
wikg://book.wikg/chapter/part/title
|
|
99
|
+
wikg://book.wikg/chapter/part/state
|
|
100
|
+
wikg://book.wikg/chapter/part/source
|
|
101
|
+
wikg://book.wikg/chapter/part/source#4..8
|
|
102
|
+
wikg://book.wikg/chapter/part/summary
|
|
103
|
+
wikg://book.wikg/chapter/part/summary#4..8
|
|
104
|
+
wikg://book.wikg/chapter/part/chunk/12
|
|
105
|
+
wikg://book.wikg/chapter/part/entity/Q8018
|
|
106
|
+
wikg://book.wikg/chapter/part/triple/Q8018/discusses/Q123
|
|
86
107
|
wikg://book.wikg/chunk/12
|
|
87
108
|
wikg://book.wikg/entity/Q8018
|
|
88
109
|
wikg://book.wikg/entity/Q8018/wikipage
|
|
@@ -99,9 +120,12 @@ Triple pattern scopes:
|
|
|
99
120
|
A complete triple URI with concrete subject, predicate, and object is an object URI.
|
|
100
121
|
|
|
101
122
|
Object boundaries:
|
|
102
|
-
- `chapter/<
|
|
103
|
-
- `chapter/<
|
|
123
|
+
- `chapter/<path>` is a scope, not an object.
|
|
124
|
+
- `chapter/<path>/title` is the queryable chapter title object.
|
|
104
125
|
- Whole `source` and `summary` objects are readable text streams, not collection scopes.
|
|
126
|
+
- Chapter-qualified scope URIs such as `<chapter-uri>/chunk`, `<chapter-uri>/entity`, and `<chapter-uri>/triple` expand to the chapter subtree and support `--depth`.
|
|
127
|
+
- Chapter-qualified resource and object URIs such as `/source`, `/summary`, `/title`, `/state`, `/chunk/<id>`, `/entity/<qid>`, and complete `/triple/<subject>/<predicate>/<object>` do not expand to subtrees and do not support `--depth`.
|
|
128
|
+
- `move`, `remove`, and `reset` are chapter node maintenance predicates; they target the chapter node itself, not the subtree. Use `remove --recursive` for explicit recursive deletion.
|
|
105
129
|
- Ranged fragments such as `/source#4..8` and `/summary#4..8` are structured range objects.
|
|
106
130
|
- Source and summary range numbers are 1-based and inclusive. `/source#1..2` means the first two source sentences.
|
|
107
131
|
- Source text is the grounding material. `evidence` is not available for source, summary, or chapter scope URIs.
|
|
@@ -126,13 +150,13 @@ Lens selection:
|
|
|
126
150
|
- `/chunk` targets Reading Graph chunks.
|
|
127
151
|
- `/entity` targets grounded entities.
|
|
128
152
|
- `/triple` targets Knowledge Graph relations.
|
|
129
|
-
- `chapter/<
|
|
153
|
+
- `chapter/<path>` is a scope; `--query` over chapter titles returns `chapter/<path>/title` objects.
|
|
130
154
|
|
|
131
155
|
Literal source passages:
|
|
132
156
|
Use `--query` on the narrowest useful scope when the task is to find source wording or passages mentioning a literal term:
|
|
133
157
|
{{ commandName }} <archive-uri> --query "term"
|
|
134
|
-
{{ commandName }} <archive-uri>/chapter/<
|
|
135
|
-
{{ commandName }} <archive-uri>/chapter/<
|
|
158
|
+
{{ commandName }} <archive-uri>/chapter/<path> --query "term"
|
|
159
|
+
{{ commandName }} <archive-uri>/chapter/<path>/source#20..30
|
|
136
160
|
|
|
137
161
|
Entity-grounded evidence:
|
|
138
162
|
Use entity evidence when you already have an entity URI or QID and need grounded mentions, aliases, translations, variants, or source-backed support:
|
|
@@ -143,6 +167,7 @@ Entity-grounded evidence:
|
|
|
143
167
|
Pagination:
|
|
144
168
|
- Use default paging or `--limit <n>` while exploring.
|
|
145
169
|
- Use `{{ commandName }} next <cursor>` to continue a paged result.
|
|
170
|
+
- Use `{{ commandName }} next <uri> <cursor>` when you want the cursor's stored index scope checked against the archive or library URI you expect.
|
|
146
171
|
- Use `--all` only when you intentionally need every item from the selected scope.
|
|
147
172
|
- Prefer the narrowest scope before using `--all`.
|
|
148
173
|
- Read `{{ commandName }} help format` before choosing text, `--json`, `--jsonl`, or `--all --jsonl` for machine consumption.
|
|
@@ -158,6 +183,7 @@ Recovery hints:
|
|
|
158
183
|
{{ commandName }} help readiness
|
|
159
184
|
{{ commandName }} help runtime
|
|
160
185
|
- URI rejected: convert bare filesystem paths to Wiki Graph URIs, and prepend the archive locator to short object URIs.
|
|
186
|
+
- Library URI rejected: non-default libraries use wikg://lib/<lib-id>; archive members use <lib-scope>/arc/<archive-id>.
|
|
161
187
|
- Help did not appear: append `--help` to the URI or URI-bound predicate you are trying to inspect.
|
|
162
188
|
|
|
163
189
|
Job URIs:
|
|
@@ -176,3 +202,5 @@ Related help:
|
|
|
176
202
|
{{ commandName }} help format
|
|
177
203
|
- Local jobs, cache, logs, and runtime debugging:
|
|
178
204
|
{{ commandName }} help runtime
|
|
205
|
+
- Library registry, membership, path binding, and aggregate index:
|
|
206
|
+
{{ commandName }} help library
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
#!/usr/bin/env node
|