wiki-graph 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli.js +917 -578
- package/dist/cli.js.map +1 -1
- package/dist/data/help/commands/library-predicate.jinja +19 -19
- package/dist/data/help/commands/library.jinja +20 -15
- package/dist/data/help/commands/predicate.jinja +108 -98
- package/dist/data/help/commands/root.jinja +1 -1
- package/dist/data/help/commands/uri.jinja +46 -19
- package/dist/data/help/topics/config.jinja +27 -0
- package/dist/data/help/topics/library.jinja +19 -12
- package/dist/data/help/topics/readiness.jinja +71 -34
- package/dist/data/help/topics/recipe.jinja +11 -8
- package/dist/data/help/topics/uri.jinja +12 -3
- package/dist/gc-worker.js +109 -84
- package/dist/gc-worker.js.map +1 -1
- package/dist/index.cjs +2681 -1283
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +3886 -2468
- package/dist/index.js.map +1 -1
- package/dist/queue-worker.js +571 -355
- package/dist/queue-worker.js.map +1 -1
- package/package.json +3 -3
- package/dist/cli.d.ts +0 -1
- package/dist/gc-worker.d.ts +0 -1
- package/dist/index.d.cts +0 -36
- package/dist/queue-worker.d.ts +0 -1
|
@@ -12,7 +12,9 @@ URI type:
|
|
|
12
12
|
|
|
13
13
|
Default behavior:
|
|
14
14
|
- `{{ commandName }} {{ uri }}` enumerates archive-level objects.
|
|
15
|
-
- `{{ commandName }} {{ uri }} --query <query>` applies
|
|
15
|
+
- `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval across archive scopes.
|
|
16
|
+
- Broad query results are bucketed by object family: chapter/title hits, graph objects, chunks, then text streams.
|
|
17
|
+
- Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
|
|
16
18
|
|
|
17
19
|
Use when:
|
|
18
20
|
- You need the archive entry point before narrowing to chapters, chunks, entities, triples, source, or summaries.
|
|
@@ -28,7 +30,8 @@ URI type:
|
|
|
28
30
|
|
|
29
31
|
Default behavior:
|
|
30
32
|
- `{{ commandName }} {{ uri }}` enumerates chapters.
|
|
31
|
-
- `{{ commandName }} {{ uri }} --query <query>` applies
|
|
33
|
+
- `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval to chapter objects.
|
|
34
|
+
- Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
|
|
32
35
|
- Chapter collection scope covers the full chapter tree by default; use `--depth 0` for root-level chapters only.
|
|
33
36
|
- Chapter path URIs are the public CLI handles; internal chapter ids are not command targets.
|
|
34
37
|
|
|
@@ -40,7 +43,9 @@ URI type:
|
|
|
40
43
|
|
|
41
44
|
Default behavior:
|
|
42
45
|
- `{{ commandName }} {{ uri }}` enumerates objects inside this chapter subtree.
|
|
43
|
-
- `{{ commandName }} {{ uri }} --query <query>` applies
|
|
46
|
+
- `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval inside this chapter subtree.
|
|
47
|
+
- Broad query results are bucketed by object family: chapter/title hits, graph objects, chunks, then text streams.
|
|
48
|
+
- Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
|
|
44
49
|
- Use `--depth 0` to restrict the scope to this chapter only; `--depth 1` includes direct children.
|
|
45
50
|
- The path in this URI is the public CLI handle for the chapter; internal chapter ids are not command targets.
|
|
46
51
|
|
|
@@ -52,7 +57,8 @@ URI type:
|
|
|
52
57
|
|
|
53
58
|
Default behavior:
|
|
54
59
|
- `{{ commandName }} {{ uri }}` enumerates Reading Graph chunks.
|
|
55
|
-
- `{{ commandName }} {{ uri }} --query <query>` applies
|
|
60
|
+
- `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval to chunks in this scope.
|
|
61
|
+
- Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
|
|
56
62
|
{% if uriContext.isChapterQualified %}
|
|
57
63
|
- This chapter-qualified scope covers the selected chapter subtree by default.
|
|
58
64
|
- Use `--depth 0` for only the selected chapter, or `--depth 1` for the selected chapter plus direct children.
|
|
@@ -69,7 +75,8 @@ URI type:
|
|
|
69
75
|
|
|
70
76
|
Default behavior:
|
|
71
77
|
- `{{ commandName }} {{ uri }}` enumerates Knowledge Graph entities.
|
|
72
|
-
- `{{ commandName }} {{ uri }} --query <query>` applies
|
|
78
|
+
- `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval to entities in this scope.
|
|
79
|
+
- Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
|
|
73
80
|
{% if uriContext.isChapterQualified %}
|
|
74
81
|
- This chapter-qualified scope covers the selected chapter subtree by default.
|
|
75
82
|
- Use `--depth 0` for only the selected chapter, or `--depth 1` for the selected chapter plus direct children.
|
|
@@ -86,7 +93,8 @@ URI type:
|
|
|
86
93
|
|
|
87
94
|
Default behavior:
|
|
88
95
|
- `{{ commandName }} {{ uri }}` enumerates Knowledge Graph triples matching this scope or pattern.
|
|
89
|
-
- `{{ commandName }} {{ uri }} --query <query>` applies
|
|
96
|
+
- `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval to triples in this scope.
|
|
97
|
+
- Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
|
|
90
98
|
{% if uriContext.isChapterQualified %}
|
|
91
99
|
- This chapter-qualified scope covers the selected chapter subtree by default.
|
|
92
100
|
- Use `--depth 0` for only the selected chapter, or `--depth 1` for the selected chapter plus direct children.
|
|
@@ -129,6 +137,7 @@ Default behavior:
|
|
|
129
137
|
- Summary range fragments such as `#4..8` read sentence numbers 4 through 8, inclusive.
|
|
130
138
|
- Range sentence numbers are 1-based. `#1` is the first sentence; `#1..2` is the first two sentences.
|
|
131
139
|
- Whole summary reads are plain text streams and do not support `--json`; summary range fragments are structured range objects and support `--json`.
|
|
140
|
+
- Summary readiness means a summary object exists and is current for the chapter source. It does not guarantee the prose answers every question about the chapter.
|
|
132
141
|
|
|
133
142
|
Use when:
|
|
134
143
|
- You need compact readable prose before tracing source evidence.
|
|
@@ -170,6 +179,17 @@ Default behavior:
|
|
|
170
179
|
|
|
171
180
|
Use when:
|
|
172
181
|
- You need to know whether source, Reading Graph, Reading Summary, or Knowledge Graph data exists for a chapter.
|
|
182
|
+
{% elif target.name == "chapter-index-artifact" %}
|
|
183
|
+
URI type:
|
|
184
|
+
Chapter index artifact object
|
|
185
|
+
|
|
186
|
+
Default behavior:
|
|
187
|
+
- `{{ commandName }} {{ uri }}` reads this chapter index artifact status and summary.
|
|
188
|
+
|
|
189
|
+
Use when:
|
|
190
|
+
- You need to build or delete one chapter-level index artifact stored in the `.wikg` archive.
|
|
191
|
+
- Use `/index/fts` for lexical retrieval, `/index/embedding/source` for Dense source retrieval, or `/index/embedding/summary` for Dense summary retrieval.
|
|
192
|
+
- Build enqueues a local job; read the returned job status or watch the job before assuming the artifact is ready.
|
|
173
193
|
{% elif target.name == "chunk-object" %}
|
|
174
194
|
URI type:
|
|
175
195
|
Chunk object
|
|
@@ -217,20 +237,21 @@ Use when:
|
|
|
217
237
|
- You need to redirect or pipe the cover resource.
|
|
218
238
|
{% elif target.name == "index-object" %}
|
|
219
239
|
URI type:
|
|
220
|
-
Index object
|
|
240
|
+
Index cache object
|
|
221
241
|
|
|
222
242
|
Default behavior:
|
|
223
|
-
- `{{ commandName }} {{ uri }}` reads
|
|
224
|
-
-
|
|
243
|
+
- `{{ commandName }} {{ uri }}` reads index cache status and capabilities.
|
|
244
|
+
- Chapter-level `/index/...` URIs manage index artifacts. Archive and library `/index` URIs manage derived cache.
|
|
225
245
|
|
|
226
246
|
Use when:
|
|
227
|
-
- You need to inspect
|
|
247
|
+
- You need to inspect, sync, or clean query cache materialization.
|
|
228
248
|
{% elif target.name == "job-collection-scope" %}
|
|
229
249
|
URI type:
|
|
230
250
|
Local job collection scope
|
|
231
251
|
|
|
232
252
|
Default behavior:
|
|
233
253
|
- `{{ commandName }} {{ uri }}` shows local generation jobs.
|
|
254
|
+
- Use `--json` for a machine-readable job list. Job list does not support `--jsonl`; `watch --jsonl` is the streaming job interface.
|
|
234
255
|
|
|
235
256
|
Use when:
|
|
236
257
|
- You need to inspect, add, or clean local generation jobs.
|
|
@@ -268,6 +289,7 @@ Use when:
|
|
|
268
289
|
|
|
269
290
|
Sections:
|
|
270
291
|
- `{{ commandName }} wikg://local/config/llm --help`
|
|
292
|
+
- `{{ commandName }} wikg://local/config/embeddings --help`
|
|
271
293
|
- `{{ commandName }} wikg://local/config/concurrent --help`
|
|
272
294
|
- `{{ commandName }} wikg://local/config/wikispine --help`
|
|
273
295
|
{% elif target.name == "local-config-section" %}
|
|
@@ -279,6 +301,9 @@ Default behavior:
|
|
|
279
301
|
|
|
280
302
|
Use when:
|
|
281
303
|
- You need to read, replace, edit, clear, or test local configuration.
|
|
304
|
+
{% if "/embeddings" in uri %}
|
|
305
|
+
- This section is used by Dense embedding artifact builds and Dense query embeddings.
|
|
306
|
+
{% endif %}
|
|
282
307
|
{% if "/wikispine" in uri %}
|
|
283
308
|
|
|
284
309
|
WikiSpine runtime guide:
|
|
@@ -357,10 +382,16 @@ Predicate commands:
|
|
|
357
382
|
{% endif %}
|
|
358
383
|
{% elif predicate == "boost" %}
|
|
359
384
|
- boost: move a waiting job forward. Help: {{ commandName }} {{ uri }} boost --help
|
|
385
|
+
{% elif predicate == "build" %}
|
|
386
|
+
- build: build or replace this chapter index artifact. Help: {{ commandName }} {{ uri }} build --help
|
|
360
387
|
{% elif predicate == "cancel" %}
|
|
361
388
|
- cancel: cancel a local job. Help: {{ commandName }} {{ uri }} cancel --help
|
|
362
389
|
{% elif predicate == "clean" %}
|
|
390
|
+
{% if target.name == "index-object" %}
|
|
391
|
+
- clean: remove derived index cache materialization. Help: {{ commandName }} {{ uri }} clean --help
|
|
392
|
+
{% else %}
|
|
363
393
|
- clean: remove completed local jobs. Help: {{ commandName }} {{ uri }} clean --help
|
|
394
|
+
{% endif %}
|
|
364
395
|
{% elif predicate == "clear" %}
|
|
365
396
|
{% if target.name == "metadata-object" %}
|
|
366
397
|
- clear: clear this metadata map. Help: {{ commandName }} {{ uri }} clear --help
|
|
@@ -370,23 +401,19 @@ Predicate commands:
|
|
|
370
401
|
{% elif predicate == "create" %}
|
|
371
402
|
- create: create the archive; use `--replace` only when discarding an existing archive. Help: {{ commandName }} {{ uri }} create --help
|
|
372
403
|
{% elif predicate == "delete" %}
|
|
373
|
-
{% if target.name == "
|
|
404
|
+
{% if target.name == "chapter-index-artifact" %}
|
|
405
|
+
- delete: delete this chapter index artifact. Help: {{ commandName }} {{ uri }} delete --help
|
|
406
|
+
{% elif target.name == "metadata-object" %}
|
|
374
407
|
- delete: delete one metadata key. Help: {{ commandName }} {{ uri }} delete --help
|
|
375
408
|
{% else %}
|
|
376
409
|
- delete: delete a config key. Help: {{ commandName }} {{ uri }} delete --help
|
|
377
410
|
{% endif %}
|
|
378
|
-
{% elif predicate == "disable" %}
|
|
379
|
-
- disable: disable searchable index support and remove index materialization. Help: {{ commandName }} {{ uri }} disable --help
|
|
380
|
-
{% elif predicate == "embed" %}
|
|
381
|
-
- embed: use archive-embedded searchable index storage. Help: {{ commandName }} {{ uri }} embed --help
|
|
382
|
-
{% elif predicate == "enable" %}
|
|
383
|
-
- enable: enable searchable index support and bring it to current. Help: {{ commandName }} {{ uri }} enable --help
|
|
384
411
|
{% elif predicate == "evidence" %}
|
|
385
412
|
- evidence: trace this object to source-backed evidence; supports `--reverse` without `--query`. Help: {{ commandName }} {{ uri }} evidence --help
|
|
386
413
|
{% elif predicate == "export" %}
|
|
387
414
|
- export: export an archive projection. Help: {{ commandName }} {{ uri }} export --help
|
|
388
|
-
{% elif predicate == "
|
|
389
|
-
-
|
|
415
|
+
{% elif predicate == "sync" %}
|
|
416
|
+
- sync: rebuild derived index cache from chapter index artifacts. Help: {{ commandName }} {{ uri }} sync --help
|
|
390
417
|
{% elif predicate == "inspect" %}
|
|
391
418
|
- inspect: inspect archive readiness. Help: {{ commandName }} {{ uri }} inspect --help
|
|
392
419
|
{% elif predicate == "move" %}
|
|
@@ -11,6 +11,12 @@ Sections:
|
|
|
11
11
|
`wikg://local/config/llm`
|
|
12
12
|
LLM provider, model, optional base URL, client name, and API key.
|
|
13
13
|
|
|
14
|
+
`wikg://local/config/embeddings`
|
|
15
|
+
Embedding provider for Dense search index vectors and Dense query vectors:
|
|
16
|
+
- `provider`: `openai` or `openai-compatible`.
|
|
17
|
+
- `model`: embedding model name.
|
|
18
|
+
- `dimensions`: optional positive integer when the provider/model supports dimension control.
|
|
19
|
+
|
|
14
20
|
`wikg://local/config/concurrent`
|
|
15
21
|
Local concurrency settings:
|
|
16
22
|
- `job`: simultaneous generation jobs. Default: 3.
|
|
@@ -25,17 +31,21 @@ Sections:
|
|
|
25
31
|
Operations:
|
|
26
32
|
Read a section:
|
|
27
33
|
{{ commandName }} wikg://local/config/llm
|
|
34
|
+
{{ commandName }} wikg://local/config/embeddings
|
|
28
35
|
{{ commandName }} wikg://local/config/concurrent
|
|
29
36
|
{{ commandName }} wikg://local/config/wikispine
|
|
30
37
|
|
|
31
38
|
Replace a section with a JSON object:
|
|
32
39
|
{{ commandName }} wikg://local/config/llm set '{"provider":"openai","model":"gpt-4.1"}'
|
|
40
|
+
{{ commandName }} wikg://local/config/embeddings set '{"provider":"openai","model":"text-embedding-3-small"}'
|
|
33
41
|
{{ commandName }} wikg://local/config/concurrent set '{"job":3,"request":6}'
|
|
34
42
|
{{ commandName }} wikg://local/config/wikispine set '{"provider":"fetch"}'
|
|
35
43
|
|
|
36
44
|
Update one key:
|
|
37
45
|
{{ commandName }} wikg://local/config/llm put provider openai
|
|
38
46
|
{{ commandName }} wikg://local/config/llm put model gpt-4.1
|
|
47
|
+
{{ commandName }} wikg://local/config/embeddings put provider openai
|
|
48
|
+
{{ commandName }} wikg://local/config/embeddings put model text-embedding-3-small
|
|
39
49
|
{{ commandName }} wikg://local/config/concurrent put job 3
|
|
40
50
|
{{ commandName }} wikg://local/config/concurrent put request 6
|
|
41
51
|
{{ commandName }} wikg://local/config/wikispine put provider cli
|
|
@@ -43,6 +53,7 @@ Operations:
|
|
|
43
53
|
|
|
44
54
|
Store the API key:
|
|
45
55
|
{{ commandName }} wikg://local/config/llm put apiKey --secret
|
|
56
|
+
{{ commandName }} wikg://local/config/embeddings put apiKey --secret
|
|
46
57
|
|
|
47
58
|
Secret handling:
|
|
48
59
|
- `apiKey` is read through an interactive hidden prompt.
|
|
@@ -56,6 +67,18 @@ Minimum LLM-backed operation configuration:
|
|
|
56
67
|
- `llm.model`
|
|
57
68
|
- Provider credentials when required by the selected provider.
|
|
58
69
|
|
|
70
|
+
Minimum Dense embedding configuration:
|
|
71
|
+
- `embeddings.provider`
|
|
72
|
+
- `embeddings.model`
|
|
73
|
+
- `embeddings.baseURL` when using `openai-compatible`
|
|
74
|
+
- Provider credentials when required by the selected provider.
|
|
75
|
+
|
|
76
|
+
Dense search notes:
|
|
77
|
+
- FTS artifact builds do not use this section.
|
|
78
|
+
- Source and summary embedding artifact builds require this section.
|
|
79
|
+
- Index cache sync does not call the embeddings provider; it uses vectors already stored in artifacts.
|
|
80
|
+
- Query uses this section when the current index cache includes Dense vectors. If Dense query embedding fails, Hybrid query can still fall back to FTS.
|
|
81
|
+
|
|
59
82
|
One-run overrides:
|
|
60
83
|
`--llm <json>` can provide an LLM object for one invocation.
|
|
61
84
|
It accepts direct LLM objects and common OpenAI-compatible aliases such as `baseUrl`.
|
|
@@ -68,6 +91,10 @@ Examples:
|
|
|
68
91
|
{{ commandName }} wikg://local/config/llm put model gpt-4.1
|
|
69
92
|
{{ commandName }} wikg://local/config/llm put apiKey --secret
|
|
70
93
|
{{ commandName }} wikg://local/config/llm test
|
|
94
|
+
{{ commandName }} wikg://local/config/embeddings put provider openai
|
|
95
|
+
{{ commandName }} wikg://local/config/embeddings put model text-embedding-3-small
|
|
96
|
+
{{ commandName }} wikg://local/config/embeddings put apiKey --secret
|
|
97
|
+
{{ commandName }} wikg://local/config/embeddings test
|
|
71
98
|
{{ commandName }} wikg://local/config/wikispine put provider fetch
|
|
72
99
|
{{ commandName }} wikg://local/config/wikispine test
|
|
73
100
|
|
|
@@ -12,7 +12,8 @@ Core model:
|
|
|
12
12
|
- The default library is addressed as `wikg://lib`.
|
|
13
13
|
- A non-default library is addressed as `wikg://lib/<lib-id>`.
|
|
14
14
|
- The `arc` reserved segment keeps library ids distinct from archive member ids.
|
|
15
|
-
- `{{ commandName }} wikg://lib` reads the default library scope
|
|
15
|
+
- `{{ commandName }} wikg://lib` reads the default library root scope.
|
|
16
|
+
- `{{ commandName }} wikg://lib --query <query>` searches library-wide objects through the library index.
|
|
16
17
|
|
|
17
18
|
Registry discovery:
|
|
18
19
|
- `{{ commandName }} wikg://lib/registry add --path <folder>` prints the new non-default library URI.
|
|
@@ -30,15 +31,16 @@ Library folders:
|
|
|
30
31
|
Archive memberships:
|
|
31
32
|
- `{{ commandName }} wikg://lib/arc scan` recursively discovers `.wikg` files under the default library folder and reconciles registry state.
|
|
32
33
|
- `{{ commandName }} wikg://lib/arc add --input <path> [--to <relative-wikg-path>]` copies one `.wikg` file into the default library folder and registers it.
|
|
33
|
-
- `{{ commandName }} wikg://lib/arc
|
|
34
|
+
- `{{ commandName }} wikg://lib` and `{{ commandName }} wikg://lib/arc` separate the library root scope from archive membership management; use `{{ commandName }} wikg://lib/arc/tree` when you want a file tree visualization.
|
|
34
35
|
- For a non-default library, replace `wikg://lib` with `wikg://lib/<lib-id>` in those `arc` commands.
|
|
35
36
|
- `{{ commandName }} <library-archive-uri>/path set <relative-wikg-path>` moves or renames one managed archive inside the same folder.
|
|
36
37
|
- `{{ commandName }} <library-archive-uri> remove --confirm` deletes one managed archive file and unregisters it.
|
|
37
38
|
- Missing registered files remain visible as missing or stale until `arc scan`, archive `path set`, archive `remove`, or library `path set` resolves them.
|
|
38
39
|
|
|
39
40
|
Library archive shortcuts:
|
|
40
|
-
- `wikg://lib/arc/<archive-id>/` addresses one archive
|
|
41
|
-
- `wikg://lib/<lib-id>/arc/<archive-id>/` addresses one archive
|
|
41
|
+
- `wikg://lib/arc/<archive-id>/` addresses one archive member page in the default library.
|
|
42
|
+
- `wikg://lib/<lib-id>/arc/<archive-id>/` addresses one archive member page in a non-default library.
|
|
43
|
+
- The archive member page exposes the managed archive's business entry URIs (`/chapter`, `/entity`, `/triple`, and `inspect`) and keeps registry fields as diagnostic metadata.
|
|
42
44
|
- Add normal archive object suffixes after the trailing slash to query that archive's contents:
|
|
43
45
|
{{ commandName }} wikg://lib/arc/<archive-id>/chapter
|
|
44
46
|
{{ commandName }} wikg://lib/arc/<archive-id>/entity --query "term"
|
|
@@ -48,9 +50,11 @@ Library archive shortcuts:
|
|
|
48
50
|
- `{{ commandName }} <library-archive-uri>/entity --help` explains the archive object scope reached through that library shortcut.
|
|
49
51
|
|
|
50
52
|
Library-wide query:
|
|
53
|
+
- `{{ commandName }} wikg://lib --query <archive-term>` searches library-wide objects through the library index.
|
|
51
54
|
- `{{ commandName }} wikg://lib/entity --query <query>` searches across present archive members through the library index.
|
|
52
55
|
- `{{ commandName }} wikg://lib/triple --query <query>` searches library-wide Knowledge Graph triples.
|
|
53
56
|
- `{{ commandName }} wikg://lib/chunk --query <query>` searches library-wide Reading Graph chunks.
|
|
57
|
+
- Add `--skip-unindexed` only when you intentionally want to search indexed chapters and ignore uncovered chapters.
|
|
54
58
|
- Library-wide object predicates such as `evidence`, `related`, and `pack` use the same aggregate library index.
|
|
55
59
|
- Library-wide object URIs such as `wikg://lib/entity/<qid>` are aggregate views over matching objects from present archive members.
|
|
56
60
|
- Evidence and related output should be interpreted with the returned archive/member provenance; cite the specific returned evidence or member, not just the library-wide URI.
|
|
@@ -58,18 +62,21 @@ Library-wide query:
|
|
|
58
62
|
|
|
59
63
|
Library index lifecycle:
|
|
60
64
|
- `{{ commandName }} <library-uri>/index` reads library index status.
|
|
61
|
-
- `{{ commandName }} <library-uri>/index
|
|
62
|
-
- `{{ commandName }} <library-uri>/index
|
|
65
|
+
- `{{ commandName }} <library-uri>/index sync` rebuilds the aggregate index cache from registered present archives' artifacts.
|
|
66
|
+
- `{{ commandName }} <library-uri>/index clean` removes the local library index cache.
|
|
63
67
|
- Library indexes are local derived state under Wiki Graph staging, not files inside the library folder.
|
|
64
|
-
-
|
|
65
|
-
-
|
|
66
|
-
-
|
|
68
|
+
- Sync does not build missing chapter artifacts and does not call an embeddings provider.
|
|
69
|
+
- Dense query still requires `wikg://local/config/embeddings` to embed the query text.
|
|
70
|
+
- When both FTS and Dense exist, library query combines them automatically; there is no separate query mode flag.
|
|
71
|
+
- Library membership changes can make the aggregate index stale.
|
|
72
|
+
- Managed archive writes try to sync an existing library index cache immediately; if sync cannot complete, library-wide query reports the cache as stale or not ready.
|
|
73
|
+
- Sync the library index again when library-wide query reports that the index is missing, stale, or not ready.
|
|
67
74
|
|
|
68
75
|
Concurrency:
|
|
69
76
|
- Library registry and membership writes hold the library write lock.
|
|
70
|
-
- `arc scan`, `arc add`, library `path set`, library archive `path set`, library archive `remove`, and library index `
|
|
71
|
-
- Library-wide `--query`,
|
|
72
|
-
- If a write
|
|
77
|
+
- `arc scan`, `arc add`, library `path set`, library archive `path set`, library archive `remove`, and library index `sync` or `clean` cannot run concurrently for the same library.
|
|
78
|
+
- Library-wide object `--query`, object scope enumeration, evidence, related, and pack commands read the current aggregate index.
|
|
79
|
+
- If a write cannot resync the aggregate index, rebuild it with `{{ commandName }} <library-uri>/index sync` before relying on library-wide query results.
|
|
73
80
|
|
|
74
81
|
Upgrade and cleanup:
|
|
75
82
|
- Upgrade a whole library with `{{ commandName }} maintenance upgrade wikg://lib` or `{{ commandName }} maintenance upgrade wikg://lib/<lib-id>`.
|
|
@@ -10,47 +10,67 @@ Purpose:
|
|
|
10
10
|
Start with inspect:
|
|
11
11
|
{{ commandName }} <archive-uri> inspect
|
|
12
12
|
|
|
13
|
-
Archive
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
13
|
+
Archive index readiness:
|
|
14
|
+
Ordinary archive source query needs chapter index artifacts. For each content chapter, at least one of these must be current:
|
|
15
|
+
- FTS artifact: `<chapter-uri>/index/fts`
|
|
16
|
+
- Source embedding artifact: `<chapter-uri>/index/embedding/source`
|
|
17
|
+
|
|
18
|
+
Summary semantic retrieval is separate:
|
|
19
|
+
- Summary embedding artifact: `<chapter-uri>/index/embedding/summary`
|
|
20
|
+
- Summary embedding alone does not satisfy ordinary archive source query readiness.
|
|
21
|
+
|
|
22
|
+
Artifact behavior:
|
|
23
|
+
Chapter artifacts are stored inside the `.wikg` archive.
|
|
24
|
+
Build FTS artifacts with the chapter index URI or queue target `index-fts`.
|
|
25
|
+
Build source embedding artifacts with the chapter index URI or queue target `index-embedding-source`.
|
|
26
|
+
Build summary embedding artifacts with the chapter index URI or queue target `index-embedding-summary`.
|
|
27
|
+
Building embedding artifacts requires `wikg://local/config/embeddings` and may call the configured provider.
|
|
28
|
+
|
|
29
|
+
Cache behavior:
|
|
30
|
+
`{{ commandName }} <archive-uri>/index sync` builds or repairs local `index.db` cache from artifacts.
|
|
31
|
+
Sync does not build missing artifacts and does not call an embeddings provider.
|
|
32
|
+
Archive-level query can create a missing cache from current artifacts on first use, but explicit sync avoids the first-query delay.
|
|
33
|
+
If query reports a missing or outdated cache, run `{{ commandName }} <archive-uri>/index sync`; library query always requires a current library cache.
|
|
34
|
+
`{{ commandName }} <archive-uri>/index clean` deletes only the local cache.
|
|
35
|
+
|
|
36
|
+
Query behavior:
|
|
37
|
+
FTS artifact only: FTS query.
|
|
38
|
+
Source embedding artifact only: semantic/Dense-only query.
|
|
39
|
+
Both artifacts: Hybrid query with automatic fusion.
|
|
40
|
+
Broad query is grouped by result type first: chapter/title, object, chunk, then source/summary text.
|
|
41
|
+
FTS, Dense, and Hybrid ranking affect ordering inside those result groups and evidence-backed object ranking; they do not flatten the type groups into one global rerank.
|
|
42
|
+
Missing both FTS and source embedding for any content chapter in scope: ordinary query fails.
|
|
43
|
+
Add `--skip-unindexed` to a scope query, `related --query`, or `evidence --query` only when you want to search indexed chapters and ignore uncovered chapters.
|
|
44
|
+
Results produced with `--skip-unindexed` are intentionally partial; inspect coverage before treating absent chapters as negative evidence.
|
|
45
|
+
Summary embedding artifacts are for summary semantic retrieval and are not the minimum requirement for ordinary source query.
|
|
28
46
|
|
|
29
47
|
Commands:
|
|
30
48
|
{{ commandName }} <archive-uri>/index
|
|
31
|
-
{{ commandName }} <
|
|
32
|
-
{{ commandName }} <
|
|
33
|
-
{{ commandName }} <
|
|
34
|
-
{{ commandName }} <archive-uri>/index
|
|
49
|
+
{{ commandName }} <chapter-uri>/index/fts --help
|
|
50
|
+
{{ commandName }} <chapter-uri>/index/embedding/source --help
|
|
51
|
+
{{ commandName }} <chapter-uri>/index/embedding/summary --help
|
|
52
|
+
{{ commandName }} <archive-uri>/index sync --help
|
|
53
|
+
{{ commandName }} <archive-uri>/index clean --help
|
|
35
54
|
|
|
36
55
|
Library index readiness:
|
|
37
|
-
Library-wide scopes use an aggregate library index
|
|
38
|
-
|
|
56
|
+
Library-wide scopes use an aggregate library index cache built from member archives' artifacts.
|
|
57
|
+
Unlike a free archive, library query does not lazy-sync this cache. It must already be current.
|
|
39
58
|
|
|
40
59
|
Default behavior:
|
|
41
|
-
`{{ commandName }} wikg://lib/index
|
|
42
|
-
`{{ commandName }} wikg://lib/<lib-id>/index
|
|
60
|
+
`{{ commandName }} wikg://lib/index sync` rebuilds the default library index cache from registered present archives.
|
|
61
|
+
`{{ commandName }} wikg://lib/<lib-id>/index sync` rebuilds a non-default library index cache.
|
|
43
62
|
The index is local derived state under Wiki Graph staging, not a file inside the library folder.
|
|
63
|
+
Sync does not build missing chapter artifacts and does not call an embeddings provider.
|
|
44
64
|
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
65
|
+
Stale behavior:
|
|
66
|
+
Library membership changes can make the aggregate index stale.
|
|
67
|
+
Managed archive writes try to sync an existing library index cache immediately.
|
|
68
|
+
Sync the library index when library-wide query reports that it is missing, stale, or not ready.
|
|
49
69
|
|
|
50
70
|
Commands:
|
|
51
71
|
{{ commandName }} wikg://lib/index
|
|
52
|
-
{{ commandName }} wikg://lib/index
|
|
53
|
-
{{ commandName }} wikg://lib/index
|
|
72
|
+
{{ commandName }} wikg://lib/index sync --help
|
|
73
|
+
{{ commandName }} wikg://lib/index clean --help
|
|
54
74
|
{{ commandName }} help library
|
|
55
75
|
|
|
56
76
|
Generation readiness:
|
|
@@ -82,6 +102,22 @@ LLM readiness:
|
|
|
82
102
|
{{ commandName }} wikg://local/config/llm put apiKey --secret
|
|
83
103
|
{{ commandName }} wikg://local/config/llm test
|
|
84
104
|
|
|
105
|
+
Embeddings readiness:
|
|
106
|
+
`wikg://local/config/embeddings` is required for source or summary embedding artifact builds and for querying an index cache that includes Dense vectors.
|
|
107
|
+
Hybrid query can still fall back to FTS if a Dense query embedding request fails.
|
|
108
|
+
|
|
109
|
+
Minimum configuration:
|
|
110
|
+
provider
|
|
111
|
+
model
|
|
112
|
+
baseURL when using openai-compatible
|
|
113
|
+
provider credentials when required
|
|
114
|
+
|
|
115
|
+
Common path:
|
|
116
|
+
{{ commandName }} wikg://local/config/embeddings put provider openai
|
|
117
|
+
{{ commandName }} wikg://local/config/embeddings put model text-embedding-3-small
|
|
118
|
+
{{ commandName }} wikg://local/config/embeddings put apiKey --secret
|
|
119
|
+
{{ commandName }} wikg://local/config/embeddings test
|
|
120
|
+
|
|
85
121
|
WikiSpine readiness:
|
|
86
122
|
Knowledge Graph generation also requires WikiSpine.
|
|
87
123
|
WikiSpine provides entity candidate lookup for Knowledge Graph construction.
|
|
@@ -108,13 +144,14 @@ WikiSpine readiness:
|
|
|
108
144
|
https://raw.githubusercontent.com/oomol-lab/wiki-graph/refs/heads/main/docs/wikispine-runtime.md
|
|
109
145
|
|
|
110
146
|
How to choose:
|
|
111
|
-
- Need
|
|
112
|
-
- Need
|
|
113
|
-
- Need
|
|
114
|
-
- Need
|
|
147
|
+
- Need lexical retrieval: build chapter FTS artifacts, then sync cache if you do not want lazy sync.
|
|
148
|
+
- Need semantic retrieval: configure embeddings, build source embedding artifacts, then sync cache.
|
|
149
|
+
- Need both lexical and semantic retrieval: build both FTS and source embedding artifacts, then sync the archive index cache if it already exists or query reports it outdated; query can create a missing archive cache and chooses Hybrid automatically once the cache is current.
|
|
150
|
+
- Need searchable retrieval across a managed folder: sync the library index cache.
|
|
151
|
+
- Need to intentionally drop local cache: clean index cache; artifacts remain in the archive.
|
|
115
152
|
- Need generated Reading Graph or summaries: configure LLM and use local jobs.
|
|
116
153
|
- Need Knowledge Graph: configure LLM and WikiSpine, then use local jobs.
|
|
117
|
-
- Not sure
|
|
154
|
+
- Not sure what is missing: run inspect and check index artifact coverage before syncing cache.
|
|
118
155
|
|
|
119
156
|
Where to go next:
|
|
120
157
|
- Normal workflow:
|
|
@@ -50,15 +50,18 @@ After inspect:
|
|
|
50
50
|
{{ commandName }} wikg:///Users/me/book.wikg --query "attention memory"
|
|
51
51
|
{{ commandName }} wikg:///Users/me/book.wikg/chapter/part --query "attention"
|
|
52
52
|
|
|
53
|
-
- If inspect reports missing
|
|
53
|
+
- If inspect reports missing index artifacts or cache, inspect the index targets before syncing cache.
|
|
54
54
|
{{ commandName }} <archive-uri>/index --help
|
|
55
|
-
{{ commandName }} <archive-uri>/index
|
|
56
|
-
|
|
55
|
+
{{ commandName }} <archive-uri>/chapter/<chapter-id>/index/fts --help
|
|
56
|
+
{{ commandName }} <archive-uri>/chapter/<chapter-id>/index/embedding/source --help
|
|
57
|
+
{{ commandName }} <archive-uri>/chapter/<chapter-id>/index/embedding/summary --help
|
|
58
|
+
{{ commandName }} <archive-uri>/index sync --help
|
|
59
|
+
Read readiness before deciding whether artifacts or cache are missing:
|
|
57
60
|
{{ commandName }} help readiness
|
|
58
61
|
|
|
59
|
-
- If library-wide query reports a missing or
|
|
62
|
+
- If library-wide query reports a missing, stale, or not-ready index cache, inspect the library index target and sync it when appropriate.
|
|
60
63
|
{{ commandName }} wikg://lib/index --help
|
|
61
|
-
{{ commandName }} wikg://lib/index
|
|
64
|
+
{{ commandName }} wikg://lib/index sync --help
|
|
62
65
|
{{ commandName }} help library
|
|
63
66
|
|
|
64
67
|
- If Reading Graph data is missing, chunk `--query`, related expansion, and pack may be unavailable or incomplete.
|
|
@@ -83,8 +86,8 @@ Finding material:
|
|
|
83
86
|
{{ commandName }} wikg://book.wikg/chapter/part
|
|
84
87
|
|
|
85
88
|
- Need to search across all archives in a managed library:
|
|
86
|
-
Use library-wide scopes after
|
|
87
|
-
{{ commandName }} wikg://lib/index
|
|
89
|
+
Use library-wide scopes after syncing the library index cache.
|
|
90
|
+
{{ commandName }} wikg://lib/index sync
|
|
88
91
|
{{ commandName }} wikg://lib/entity --query "attention" --evidence 2
|
|
89
92
|
{{ commandName }} wikg://lib/triple --query "attention memory" --evidence 2
|
|
90
93
|
{{ commandName }} wikg://lib/chunk --query "attention memory" --evidence 2
|
|
@@ -168,7 +171,7 @@ When to read deeper:
|
|
|
168
171
|
{{ commandName }} help format
|
|
169
172
|
- LLM, concurrency, and WikiSpine provider setup:
|
|
170
173
|
{{ commandName }} help config
|
|
171
|
-
-
|
|
174
|
+
- Search index, embeddings, LLM, WikiSpine, and generated-data prerequisites:
|
|
172
175
|
{{ commandName }} help readiness
|
|
173
176
|
- Library registry, membership, path binding, and aggregate index:
|
|
174
177
|
{{ commandName }} help library
|
|
@@ -23,6 +23,8 @@ Archive locator:
|
|
|
23
23
|
Library locators:
|
|
24
24
|
- `wikg://lib` addresses the default library.
|
|
25
25
|
- `wikg://lib/<lib-id>` addresses a non-default library.
|
|
26
|
+
- Naked library scopes enumerate library-wide objects; `--query` on a naked library scope searches library-wide objects through the library index.
|
|
27
|
+
- Use explicit object lenses such as `/entity`, `/triple`, `/chapter`, or `/chunk` for archive object search.
|
|
26
28
|
- `wikg://lib/arc/<archive-id>/` addresses one archive membership in the default library.
|
|
27
29
|
- `wikg://lib/<lib-id>/arc/<archive-id>/` addresses one archive membership in a non-default library.
|
|
28
30
|
- Add ordinary archive object suffixes after the trailing slash to query a managed archive:
|
|
@@ -46,7 +48,7 @@ Short and located URIs:
|
|
|
46
48
|
|
|
47
49
|
Command routing:
|
|
48
50
|
- `{{ commandName }} <scope-uri>` enumerates objects.
|
|
49
|
-
- `{{ commandName }} <scope-uri> --query <query>` applies
|
|
51
|
+
- `{{ commandName }} <scope-uri> --query <query>` applies search index retrieval within that scope.
|
|
50
52
|
- `{{ commandName }} <object-uri>` reads one object.
|
|
51
53
|
- `{{ commandName }} <uri> --help` explains that URI target and its supported predicates.
|
|
52
54
|
- `{{ commandName }} <uri> <predicate> --help` explains a URI-bound predicate.
|
|
@@ -67,12 +69,16 @@ Common command shapes:
|
|
|
67
69
|
{{ commandName }} wikg://lib/entity --query <query>
|
|
68
70
|
{{ commandName }} wikg://lib/arc/<archive-id>/entity --query <query>
|
|
69
71
|
{{ commandName }} wikg://local/job
|
|
70
|
-
{{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task <reading-graph|reading-summary|knowledge-graph> --accept-cost
|
|
72
|
+
{{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task <index-fts|index-embedding-source|index-embedding-summary|reading-graph|reading-summary|knowledge-graph> [--accept-cost]
|
|
71
73
|
{{ commandName }} wikg://local/job/<job-id>
|
|
72
74
|
|
|
73
75
|
Scope URIs:
|
|
74
|
-
Scope URIs enumerate objects by default. With `--query <query>`, they apply
|
|
76
|
+
Scope URIs enumerate objects by default. With `--query <query>`, they apply search index retrieval within that scope.
|
|
75
77
|
A chapter URI used as a scope covers that chapter's full subtree by default.
|
|
78
|
+
Query is strict by default: every chapter in scope must have a current FTS artifact or source embedding artifact.
|
|
79
|
+
Add `--skip-unindexed` only when you intentionally want to search indexed chapters and ignore uncovered chapters.
|
|
80
|
+
Broad query is grouped by result type before ranking within each group. Use `/chunk`, `/entity`, or `/triple` lenses when you want one object family instead of the mixed broad result. Query an archive or chapter scope to find source or summary text, then read the returned `/source` or `/summary` object.
|
|
81
|
+
Search uses FTS-only, Dense-only, or Hybrid automatically from the current artifacts and index cache; use `{{ commandName }} help readiness` to inspect what must exist.
|
|
76
82
|
Add `--depth <N>` to limit chapter subtree expansion:
|
|
77
83
|
- no `--depth`: full subtree
|
|
78
84
|
- `--depth 0`: only the current chapter, or root-level chapters for `<archive-uri>/chapter`
|
|
@@ -154,6 +160,7 @@ Literal source passages:
|
|
|
154
160
|
Use `--query` on the narrowest useful scope when the task is to find source wording or passages mentioning a literal term:
|
|
155
161
|
{{ commandName }} <archive-uri> --query "term"
|
|
156
162
|
{{ commandName }} <archive-uri>/chapter/<path> --query "term"
|
|
163
|
+
{{ commandName }} <archive-uri> --query "term" --skip-unindexed
|
|
157
164
|
{{ commandName }} <archive-uri>/chapter/<path>/source#20..30
|
|
158
165
|
|
|
159
166
|
Entity-grounded evidence:
|
|
@@ -176,6 +183,8 @@ Recovery hints:
|
|
|
176
183
|
{{ commandName }} <archive-uri>/chapter/tree
|
|
177
184
|
{{ commandName }} help readiness
|
|
178
185
|
- Missing generated objects: start the relevant generation job for the chapter.
|
|
186
|
+
{{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task index-fts
|
|
187
|
+
{{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task index-embedding-source --accept-cost
|
|
179
188
|
{{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task reading-graph --accept-cost
|
|
180
189
|
{{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task reading-summary --accept-cost
|
|
181
190
|
{{ commandName }} help readiness
|