wiki-graph 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -271
- package/dist/cli.d.ts +1 -0
- package/dist/cli.js +2983 -4
- package/dist/cli.js.map +1 -0
- package/{data → dist/data}/editor/text_compressor.jinja +17 -13
- package/{data → dist/data}/help/commands/archive/next.jinja +7 -2
- package/{data → dist/data}/help/commands/gc.jinja +2 -1
- package/{data → dist/data}/help/commands/legacy/migrate.jinja +6 -1
- package/{data → dist/data}/help/commands/legacy.jinja +3 -2
- package/dist/data/help/commands/library-predicate.jinja +296 -0
- package/dist/data/help/commands/library.jinja +235 -0
- package/{data → dist/data}/help/commands/maintenance/chapter/add.jinja +3 -3
- package/{data → dist/data}/help/commands/maintenance/chapter/list.jinja +4 -4
- package/{data → dist/data}/help/commands/maintenance/chapter/move.jinja +4 -4
- package/{data → dist/data}/help/commands/maintenance/chapter/remove.jinja +1 -1
- package/{data → dist/data}/help/commands/maintenance/chapter/reset.jinja +1 -1
- package/{data → dist/data}/help/commands/maintenance/chapter/set-source.jinja +2 -2
- package/{data → dist/data}/help/commands/maintenance/chapter/set-summary.jinja +1 -1
- package/{data → dist/data}/help/commands/maintenance/chapter/set-title.jinja +3 -3
- package/{data → dist/data}/help/commands/maintenance/chapter/tree.jinja +6 -6
- package/{data → dist/data}/help/commands/maintenance/chapter.jinja +10 -10
- package/{data → dist/data}/help/commands/maintenance/meta.jinja +7 -7
- package/dist/data/help/commands/maintenance/upgrade.jinja +35 -0
- package/dist/data/help/commands/maintenance.jinja +23 -0
- package/{data → dist/data}/help/commands/predicate.jinja +204 -28
- package/{data → dist/data}/help/commands/root.jinja +10 -3
- package/{data → dist/data}/help/commands/uri.jinja +165 -6
- package/{data → dist/data}/help/topics/config.jinja +10 -3
- package/dist/data/help/topics/library.jinja +87 -0
- package/{data → dist/data}/help/topics/readiness.jinja +32 -2
- package/{data → dist/data}/help/topics/recipe.jinja +42 -17
- package/{data → dist/data}/help/topics/runtime.jinja +20 -3
- package/{data → dist/data}/help/topics/uri.jinja +51 -25
- package/dist/gc-worker.d.ts +1 -0
- package/dist/gc-worker.js +548 -0
- package/dist/gc-worker.js.map +1 -0
- package/dist/index.cjs +12463 -19690
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +36 -842
- package/dist/index.d.ts +36 -842
- package/dist/index.js +13871 -9
- package/dist/index.js.map +1 -0
- package/dist/queue-worker.d.ts +1 -0
- package/dist/queue-worker.js +2110 -0
- package/dist/queue-worker.js.map +1 -0
- package/package.json +18 -45
- package/dist/cli.cjs +0 -41734
- package/dist/cli.cjs.map +0 -1
- package/docs/images/flowchart.excalidraw +0 -3293
- package/docs/images/flowchart.svg +0 -4
- package/docs/images/terminal-cn.png +0 -0
- package/docs/images/terminal-en.png +0 -0
- package/docs/wikispine-runtime.md +0 -143
- /package/{data → dist/data}/editor/clue_reviewer.jinja +0 -0
- /package/{data → dist/data}/editor/clue_reviewer_generator.jinja +0 -0
- /package/{data → dist/data}/editor/revision_feedback.jinja +0 -0
- /package/{data → dist/data}/guaranteed/response_intent_classifier.jinja +0 -0
- /package/{data → dist/data}/help/commands/maintenance/cover.jinja +0 -0
- /package/{data → dist/data}/help/commands/transform.jinja +0 -0
- /package/{data → dist/data}/help/topics/format.jinja +0 -0
- /package/{data → dist/data}/output/epub/cover.xhtml.jinja +0 -0
- /package/{data → dist/data}/output/epub/nav.xhtml.jinja +0 -0
- /package/{data → dist/data}/output/epub/package.opf.xml.jinja +0 -0
- /package/{data → dist/data}/output/epub/section.xhtml.jinja +0 -0
- /package/{data → dist/data}/topologization/book_coherence_extraction.jinja +0 -0
- /package/{data → dist/data}/topologization/chunk_extraction.jinja +0 -0
- /package/{data → dist/data}/topologization/evidence_choice.jinja +0 -0
- /package/{data → dist/data}/topologization/translate_chunks.jinja +0 -0
- /package/{data → dist/data}/topologization/user_focused_extraction.jinja +0 -0
|
@@ -25,7 +25,7 @@ Generation job behavior:
|
|
|
25
25
|
- Reading Graph, Reading Summary, and Knowledge Graph generation run as device-local jobs.
|
|
26
26
|
- Generation jobs consume LLM tokens, may incur provider charges, and can run for minutes to hours on large archives.
|
|
27
27
|
- `wikg://local/job add` requires `--accept-cost` as an explicit cost/time acknowledgement.
|
|
28
|
-
- Job records live under `~/.
|
|
28
|
+
- Job records live under `~/.wikigraph/jobs`.
|
|
29
29
|
- `wikg://local/job add`, `wikg://local/job/<job-id> resume`, `wikg://local/job/<job-id> boost`, and `wikg://local/job/<job-id>/target set` try to start a transient worker.
|
|
30
30
|
- The worker exits after idle time; it is not intended to be a permanent daemon.
|
|
31
31
|
- `wikg://local/job/<job-id> watch --jsonl` is the stable agent-facing progress stream.
|
|
@@ -41,15 +41,30 @@ Progress and diagnostics:
|
|
|
41
41
|
Workspace behavior:
|
|
42
42
|
- Archive creation and EPUB import use temporary workspaces.
|
|
43
43
|
- `transform --digest-dir <path>` uses the provided digest workspace for source digest runs.
|
|
44
|
-
- Generation jobs use per-job directories under `~/.
|
|
44
|
+
- Generation jobs use per-job directories under `~/.wikigraph/jobs/work`, `~/.wikigraph/jobs/cache`, and `~/.wikigraph/jobs/logs`.
|
|
45
45
|
- Succeeded jobs delete their workspace by default.
|
|
46
46
|
- Failed and canceled jobs keep their workspace for debugging until cleanup.
|
|
47
47
|
- Job events, logs, and LLM cache are kept until the job record is cleaned.
|
|
48
48
|
|
|
49
|
+
Local state map:
|
|
50
|
+
- `~/.wikigraph/core.sqlite`: home registry and shared local state.
|
|
51
|
+
- `~/.wikigraph/cache/search-sessions.sqlite`: search query cache sessions used by paged search results.
|
|
52
|
+
- `~/.wikigraph/cache/continuation-cursors.sqlite`: opaque `wg next` continuation cursor records.
|
|
53
|
+
- `~/.wikigraph/cache/cache.sqlite`: Wikipedia page and other reusable local cache records.
|
|
54
|
+
- `~/.wikigraph/jobs`: durable generation job records, events, workspaces, caches, and logs.
|
|
55
|
+
- `~/.wikigraph/jobs/work`: per-job workspaces; failed and canceled jobs keep these for debugging.
|
|
56
|
+
- `~/.wikigraph/jobs/cache`: per-job LLM/cache artifacts kept with job records until cleanup.
|
|
57
|
+
- `~/.wikigraph/jobs/logs`: per-job diagnostic logs.
|
|
58
|
+
- `~/.wikigraph/staging`: archive coordinator state, materialized archive entries, local archive index staging, and library aggregate indexes.
|
|
59
|
+
- Search query caches and continuation cursors are local derived state; use `{{ commandName }} next --help` for cursor continuation and `{{ commandName }} gc --help` for cleanup.
|
|
60
|
+
- Do not manually delete active job, staging, or cursor state while commands are running; prefer `{{ commandName }} gc` and command-specific cleanup paths.
|
|
61
|
+
|
|
49
62
|
In-place archive edits:
|
|
50
63
|
- Mutating `.wikg` commands go through the archive coordinator.
|
|
51
64
|
- Active generation jobs block destructive chapter maintenance edits for the same chapter.
|
|
52
|
-
- Archive writes are coordinated through `~/.
|
|
65
|
+
- Archive writes are coordinated through `~/.wikigraph/staging`.
|
|
66
|
+
- Library aggregate indexes are local derived state under staging and can be removed by cleanup when orphaned.
|
|
67
|
+
- Library cursors store the library index scope and re-check index freshness before continuing.
|
|
53
68
|
|
|
54
69
|
Where to go next:
|
|
55
70
|
- Need the normal workflow before debugging:
|
|
@@ -60,3 +75,5 @@ Where to go next:
|
|
|
60
75
|
{{ commandName }} help format
|
|
61
76
|
- Need URI command routing:
|
|
62
77
|
{{ commandName }} help uri
|
|
78
|
+
- Need library registry, membership, path binding, or index behavior:
|
|
79
|
+
{{ commandName }} help library
|
|
@@ -8,7 +8,8 @@ Purpose:
|
|
|
8
8
|
Use this page to choose a target, understand scope/object behavior, and select retrieval shape.
|
|
9
9
|
|
|
10
10
|
Archive locator:
|
|
11
|
-
-
|
|
11
|
+
- Normal archive/object command URIs contain a `.wikg` locator.
|
|
12
|
+
- Library URIs and local runtime URIs are separate URI families and do not contain `.wikg` locators.
|
|
12
13
|
- Do not pass a bare filesystem path as a command target.
|
|
13
14
|
/Users/me/book.wikg -> wikg:///Users/me/book.wikg
|
|
14
15
|
./book.wikg -> wikg://book.wikg
|
|
@@ -19,17 +20,29 @@ Archive locator:
|
|
|
19
20
|
- Home archive locators may start with `wikg://~/`; `~` is expanded only in that leading form.
|
|
20
21
|
- The first path segment ending in `.wikg` is the archive/object boundary.
|
|
21
22
|
|
|
23
|
+
Library locators:
|
|
24
|
+
- `wikg://lib` addresses the default library.
|
|
25
|
+
- `wikg://lib/<lib-id>` addresses a non-default library.
|
|
26
|
+
- `wikg://lib/arc/<archive-id>/` addresses one archive membership in the default library.
|
|
27
|
+
- `wikg://lib/<lib-id>/arc/<archive-id>/` addresses one archive membership in a non-default library.
|
|
28
|
+
- Add ordinary archive object suffixes after the trailing slash to query a managed archive:
|
|
29
|
+
wikg://lib/arc/<archive-id>/chapter
|
|
30
|
+
wikg://lib/arc/<archive-id>/entity
|
|
31
|
+
wikg://lib/arc/<archive-id>/triple
|
|
32
|
+
- Library-wide scopes such as `wikg://lib/entity` query the aggregate library index.
|
|
33
|
+
- Read `{{ commandName }} help library` before managing registry, membership, path binding, or library index lifecycle.
|
|
34
|
+
|
|
22
35
|
Short and located URIs:
|
|
23
36
|
Commands may return short object URIs without local file paths:
|
|
24
|
-
wikg://chapter/
|
|
25
|
-
wikg://chapter/
|
|
37
|
+
wikg://chapter/part/title
|
|
38
|
+
wikg://chapter/part/source#4..8
|
|
26
39
|
wikg://chunk/12
|
|
27
40
|
wikg://entity/Q8018
|
|
28
41
|
wikg://triple/Q8018/discusses/Q123
|
|
29
42
|
Returned short URIs are archive-relative handles, not complete command targets.
|
|
30
43
|
Add the archive locator before passing them back to commands:
|
|
31
44
|
wikg://book.wikg/entity/Q8018
|
|
32
|
-
wikg:///books/book.wikg/chapter/
|
|
45
|
+
wikg:///books/book.wikg/chapter/part/source#4..8
|
|
33
46
|
|
|
34
47
|
Command routing:
|
|
35
48
|
- `{{ commandName }} <scope-uri>` enumerates objects.
|
|
@@ -50,21 +63,27 @@ Common command shapes:
|
|
|
50
63
|
{{ commandName }} <archive-uri> export --output-format <format>
|
|
51
64
|
{{ commandName }} <archive-uri> inspect
|
|
52
65
|
{{ commandName }} transform [options]
|
|
66
|
+
{{ commandName }} wikg://lib --help
|
|
67
|
+
{{ commandName }} wikg://lib/entity --query <query>
|
|
68
|
+
{{ commandName }} wikg://lib/arc/<archive-id>/entity --query <query>
|
|
53
69
|
{{ commandName }} wikg://local/job
|
|
54
70
|
{{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task <reading-graph|reading-summary|knowledge-graph> --accept-cost
|
|
55
71
|
{{ commandName }} wikg://local/job/<job-id>
|
|
56
|
-
{{ commandName }} <object-uri> related
|
|
57
|
-
{{ commandName }} <object-uri> evidence
|
|
58
|
-
{{ commandName }} <object-uri> pack --budget 5000
|
|
59
72
|
|
|
60
73
|
Scope URIs:
|
|
61
74
|
Scope URIs enumerate objects by default. With `--query <query>`, they apply keyword retrieval within that scope.
|
|
75
|
+
A chapter URI used as a scope covers that chapter's full subtree by default.
|
|
76
|
+
Add `--depth <N>` to limit chapter subtree expansion:
|
|
77
|
+
- no `--depth`: full subtree
|
|
78
|
+
- `--depth 0`: only the current chapter, or root-level chapters for `<archive-uri>/chapter`
|
|
79
|
+
- `--depth 1`: current/root-level chapters plus direct children
|
|
80
|
+
`--depth` must include a non-negative integer value; naked `--depth`, negative values, and non-integers are invalid.
|
|
62
81
|
wikg://book.wikg
|
|
63
82
|
wikg://book.wikg/chapter
|
|
64
|
-
wikg://book.wikg/chapter/
|
|
65
|
-
wikg://book.wikg/chapter/
|
|
66
|
-
wikg://book.wikg/chapter/
|
|
67
|
-
wikg://book.wikg/chapter/
|
|
83
|
+
wikg://book.wikg/chapter/part
|
|
84
|
+
wikg://book.wikg/chapter/part/chunk
|
|
85
|
+
wikg://book.wikg/chapter/part/entity
|
|
86
|
+
wikg://book.wikg/chapter/part/triple
|
|
68
87
|
wikg://book.wikg/chunk
|
|
69
88
|
wikg://book.wikg/entity
|
|
70
89
|
wikg://book.wikg/triple
|
|
@@ -74,15 +93,15 @@ Scope URIs:
|
|
|
74
93
|
|
|
75
94
|
Object URIs:
|
|
76
95
|
Object URIs read one object by default.
|
|
77
|
-
wikg://book.wikg/chapter/
|
|
78
|
-
wikg://book.wikg/chapter/
|
|
79
|
-
wikg://book.wikg/chapter/
|
|
80
|
-
wikg://book.wikg/chapter/
|
|
81
|
-
wikg://book.wikg/chapter/
|
|
82
|
-
wikg://book.wikg/chapter/
|
|
83
|
-
wikg://book.wikg/chapter/
|
|
84
|
-
wikg://book.wikg/chapter/
|
|
85
|
-
wikg://book.wikg/chapter/
|
|
96
|
+
wikg://book.wikg/chapter/part/title
|
|
97
|
+
wikg://book.wikg/chapter/part/state
|
|
98
|
+
wikg://book.wikg/chapter/part/source
|
|
99
|
+
wikg://book.wikg/chapter/part/source#4..8
|
|
100
|
+
wikg://book.wikg/chapter/part/summary
|
|
101
|
+
wikg://book.wikg/chapter/part/summary#4..8
|
|
102
|
+
wikg://book.wikg/chapter/part/chunk/12
|
|
103
|
+
wikg://book.wikg/chapter/part/entity/Q8018
|
|
104
|
+
wikg://book.wikg/chapter/part/triple/Q8018/discusses/Q123
|
|
86
105
|
wikg://book.wikg/chunk/12
|
|
87
106
|
wikg://book.wikg/entity/Q8018
|
|
88
107
|
wikg://book.wikg/entity/Q8018/wikipage
|
|
@@ -99,9 +118,12 @@ Triple pattern scopes:
|
|
|
99
118
|
A complete triple URI with concrete subject, predicate, and object is an object URI.
|
|
100
119
|
|
|
101
120
|
Object boundaries:
|
|
102
|
-
- `chapter/<
|
|
103
|
-
- `chapter/<
|
|
121
|
+
- `chapter/<path>` is a scope, not an object.
|
|
122
|
+
- `chapter/<path>/title` is the queryable chapter title object.
|
|
104
123
|
- Whole `source` and `summary` objects are readable text streams, not collection scopes.
|
|
124
|
+
- Chapter-qualified scope URIs such as `<chapter-uri>/chunk`, `<chapter-uri>/entity`, and `<chapter-uri>/triple` expand to the chapter subtree and support `--depth`.
|
|
125
|
+
- Chapter-qualified resource and object URIs such as `/source`, `/summary`, `/title`, `/state`, `/chunk/<id>`, `/entity/<qid>`, and complete `/triple/<subject>/<predicate>/<object>` do not expand to subtrees and do not support `--depth`.
|
|
126
|
+
- `move`, `remove`, and `reset` are chapter node maintenance predicates; they target the chapter node itself, not the subtree. Use `remove --recursive` for explicit recursive deletion.
|
|
105
127
|
- Ranged fragments such as `/source#4..8` and `/summary#4..8` are structured range objects.
|
|
106
128
|
- Source and summary range numbers are 1-based and inclusive. `/source#1..2` means the first two source sentences.
|
|
107
129
|
- Source text is the grounding material. `evidence` is not available for source, summary, or chapter scope URIs.
|
|
@@ -126,13 +148,13 @@ Lens selection:
|
|
|
126
148
|
- `/chunk` targets Reading Graph chunks.
|
|
127
149
|
- `/entity` targets grounded entities.
|
|
128
150
|
- `/triple` targets Knowledge Graph relations.
|
|
129
|
-
- `chapter/<
|
|
151
|
+
- `chapter/<path>` is a scope; `--query` over chapter titles returns `chapter/<path>/title` objects.
|
|
130
152
|
|
|
131
153
|
Literal source passages:
|
|
132
154
|
Use `--query` on the narrowest useful scope when the task is to find source wording or passages mentioning a literal term:
|
|
133
155
|
{{ commandName }} <archive-uri> --query "term"
|
|
134
|
-
{{ commandName }} <archive-uri>/chapter/<
|
|
135
|
-
{{ commandName }} <archive-uri>/chapter/<
|
|
156
|
+
{{ commandName }} <archive-uri>/chapter/<path> --query "term"
|
|
157
|
+
{{ commandName }} <archive-uri>/chapter/<path>/source#20..30
|
|
136
158
|
|
|
137
159
|
Entity-grounded evidence:
|
|
138
160
|
Use entity evidence when you already have an entity URI or QID and need grounded mentions, aliases, translations, variants, or source-backed support:
|
|
@@ -143,6 +165,7 @@ Entity-grounded evidence:
|
|
|
143
165
|
Pagination:
|
|
144
166
|
- Use default paging or `--limit <n>` while exploring.
|
|
145
167
|
- Use `{{ commandName }} next <cursor>` to continue a paged result.
|
|
168
|
+
- Use `{{ commandName }} next <uri> <cursor>` when you want the cursor's stored index scope checked against the archive or library URI you expect.
|
|
146
169
|
- Use `--all` only when you intentionally need every item from the selected scope.
|
|
147
170
|
- Prefer the narrowest scope before using `--all`.
|
|
148
171
|
- Read `{{ commandName }} help format` before choosing text, `--json`, `--jsonl`, or `--all --jsonl` for machine consumption.
|
|
@@ -158,6 +181,7 @@ Recovery hints:
|
|
|
158
181
|
{{ commandName }} help readiness
|
|
159
182
|
{{ commandName }} help runtime
|
|
160
183
|
- URI rejected: convert bare filesystem paths to Wiki Graph URIs, and prepend the archive locator to short object URIs.
|
|
184
|
+
- Library URI rejected: non-default libraries use wikg://lib/<lib-id>; archive members use <lib-scope>/arc/<archive-id>.
|
|
161
185
|
- Help did not appear: append `--help` to the URI or URI-bound predicate you are trying to inspect.
|
|
162
186
|
|
|
163
187
|
Job URIs:
|
|
@@ -176,3 +200,5 @@ Related help:
|
|
|
176
200
|
{{ commandName }} help format
|
|
177
201
|
- Local jobs, cache, logs, and runtime debugging:
|
|
178
202
|
{{ commandName }} help runtime
|
|
203
|
+
- Library registry, membership, path binding, and aggregate index:
|
|
204
|
+
{{ commandName }} help library
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
#!/usr/bin/env node
|