wiki-graph 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -12,7 +12,9 @@ URI type:
12
12
 
13
13
  Default behavior:
14
14
  - `{{ commandName }} {{ uri }}` enumerates archive-level objects.
15
- - `{{ commandName }} {{ uri }} --query <query>` applies keyword retrieval across archive scopes.
15
+ - `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval across archive scopes.
16
+ - Broad query results are bucketed by object family: chapter/title hits, graph objects, chunks, then text streams.
17
+ - Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
16
18
 
17
19
  Use when:
18
20
  - You need the archive entry point before narrowing to chapters, chunks, entities, triples, source, or summaries.
@@ -28,7 +30,8 @@ URI type:
28
30
 
29
31
  Default behavior:
30
32
  - `{{ commandName }} {{ uri }}` enumerates chapters.
31
- - `{{ commandName }} {{ uri }} --query <query>` applies keyword retrieval to chapter objects.
33
+ - `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval to chapter objects.
34
+ - Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
32
35
  - Chapter collection scope covers the full chapter tree by default; use `--depth 0` for root-level chapters only.
33
36
  - Chapter path URIs are the public CLI handles; internal chapter ids are not command targets.
34
37
 
@@ -40,7 +43,9 @@ URI type:
40
43
 
41
44
  Default behavior:
42
45
  - `{{ commandName }} {{ uri }}` enumerates objects inside this chapter subtree.
43
- - `{{ commandName }} {{ uri }} --query <query>` applies keyword retrieval inside this chapter subtree.
46
+ - `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval inside this chapter subtree.
47
+ - Broad query results are bucketed by object family: chapter/title hits, graph objects, chunks, then text streams.
48
+ - Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
44
49
  - Use `--depth 0` to restrict the scope to this chapter only; `--depth 1` includes direct children.
45
50
  - The path in this URI is the public CLI handle for the chapter; internal chapter ids are not command targets.
46
51
 
@@ -52,7 +57,8 @@ URI type:
52
57
 
53
58
  Default behavior:
54
59
  - `{{ commandName }} {{ uri }}` enumerates Reading Graph chunks.
55
- - `{{ commandName }} {{ uri }} --query <query>` applies keyword retrieval to chunks in this scope.
60
+ - `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval to chunks in this scope.
61
+ - Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
56
62
  {% if uriContext.isChapterQualified %}
57
63
  - This chapter-qualified scope covers the selected chapter subtree by default.
58
64
  - Use `--depth 0` for only the selected chapter, or `--depth 1` for the selected chapter plus direct children.
@@ -69,7 +75,8 @@ URI type:
69
75
 
70
76
  Default behavior:
71
77
  - `{{ commandName }} {{ uri }}` enumerates Knowledge Graph entities.
72
- - `{{ commandName }} {{ uri }} --query <query>` applies keyword retrieval to entities in this scope.
78
+ - `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval to entities in this scope.
79
+ - Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
73
80
  {% if uriContext.isChapterQualified %}
74
81
  - This chapter-qualified scope covers the selected chapter subtree by default.
75
82
  - Use `--depth 0` for only the selected chapter, or `--depth 1` for the selected chapter plus direct children.
@@ -86,7 +93,8 @@ URI type:
86
93
 
87
94
  Default behavior:
88
95
  - `{{ commandName }} {{ uri }}` enumerates Knowledge Graph triples matching this scope or pattern.
89
- - `{{ commandName }} {{ uri }} --query <query>` applies keyword retrieval to triples in this scope.
96
+ - `{{ commandName }} {{ uri }} --query <query>` applies search index retrieval to triples in this scope.
97
+ - Query is strict by default; add `--skip-unindexed` only to intentionally search indexed chapters and ignore uncovered chapters.
90
98
  {% if uriContext.isChapterQualified %}
91
99
  - This chapter-qualified scope covers the selected chapter subtree by default.
92
100
  - Use `--depth 0` for only the selected chapter, or `--depth 1` for the selected chapter plus direct children.
@@ -129,6 +137,7 @@ Default behavior:
129
137
  - Summary range fragments such as `#4..8` read sentence numbers 4 through 8, inclusive.
130
138
  - Range sentence numbers are 1-based. `#1` is the first sentence; `#1..2` is the first two sentences.
131
139
  - Whole summary reads are plain text streams and do not support `--json`; summary range fragments are structured range objects and support `--json`.
140
+ - Summary readiness means a summary object exists and is current for the chapter source. It does not guarantee the prose answers every question about the chapter.
132
141
 
133
142
  Use when:
134
143
  - You need compact readable prose before tracing source evidence.
@@ -170,6 +179,17 @@ Default behavior:
170
179
 
171
180
  Use when:
172
181
  - You need to know whether source, Reading Graph, Reading Summary, or Knowledge Graph data exists for a chapter.
182
+ {% elif target.name == "chapter-index-artifact" %}
183
+ URI type:
184
+ Chapter index artifact object
185
+
186
+ Default behavior:
187
+ - `{{ commandName }} {{ uri }}` reads this chapter index artifact status and summary.
188
+
189
+ Use when:
190
+ - You need to build or delete one chapter-level index artifact stored in the `.wikg` archive.
191
+ - Use `/index/fts` for lexical retrieval, `/index/embedding/source` for Dense source retrieval, or `/index/embedding/summary` for Dense summary retrieval.
192
+ - Build enqueues a local job; read the returned job status or watch the job before assuming the artifact is ready.
173
193
  {% elif target.name == "chunk-object" %}
174
194
  URI type:
175
195
  Chunk object
@@ -217,20 +237,21 @@ Use when:
217
237
  - You need to redirect or pipe the cover resource.
218
238
  {% elif target.name == "index-object" %}
219
239
  URI type:
220
- Index object
240
+ Index cache object
221
241
 
222
242
  Default behavior:
223
- - `{{ commandName }} {{ uri }}` reads archive FTS index status, readiness, storage policy, and materialization state.
224
- - Use this page when `inspect`, `--query`, `related`, or `evidence` reports missing, stale, or unavailable index support.
243
+ - `{{ commandName }} {{ uri }}` reads index cache status and capabilities.
244
+ - Chapter-level `/index/...` URIs manage index artifacts. Archive and library `/index` URIs manage derived cache.
225
245
 
226
246
  Use when:
227
- - You need to inspect currentness, enable, disable, embed, or externalize searchable FTS index support.
247
+ - You need to inspect, sync, or clean query cache materialization.
228
248
  {% elif target.name == "job-collection-scope" %}
229
249
  URI type:
230
250
  Local job collection scope
231
251
 
232
252
  Default behavior:
233
253
  - `{{ commandName }} {{ uri }}` shows local generation jobs.
254
+ - Use `--json` for a machine-readable job list. Job list does not support `--jsonl`; `watch --jsonl` is the streaming job interface.
234
255
 
235
256
  Use when:
236
257
  - You need to inspect, add, or clean local generation jobs.
@@ -268,6 +289,7 @@ Use when:
268
289
 
269
290
  Sections:
270
291
  - `{{ commandName }} wikg://local/config/llm --help`
292
+ - `{{ commandName }} wikg://local/config/embeddings --help`
271
293
  - `{{ commandName }} wikg://local/config/concurrent --help`
272
294
  - `{{ commandName }} wikg://local/config/wikispine --help`
273
295
  {% elif target.name == "local-config-section" %}
@@ -279,6 +301,9 @@ Default behavior:
279
301
 
280
302
  Use when:
281
303
  - You need to read, replace, edit, clear, or test local configuration.
304
+ {% if "/embeddings" in uri %}
305
+ - This section is used by Dense embedding artifact builds and Dense query embeddings.
306
+ {% endif %}
282
307
  {% if "/wikispine" in uri %}
283
308
 
284
309
  WikiSpine runtime guide:
@@ -357,10 +382,16 @@ Predicate commands:
357
382
  {% endif %}
358
383
  {% elif predicate == "boost" %}
359
384
  - boost: move a waiting job forward. Help: {{ commandName }} {{ uri }} boost --help
385
+ {% elif predicate == "build" %}
386
+ - build: build or replace this chapter index artifact. Help: {{ commandName }} {{ uri }} build --help
360
387
  {% elif predicate == "cancel" %}
361
388
  - cancel: cancel a local job. Help: {{ commandName }} {{ uri }} cancel --help
362
389
  {% elif predicate == "clean" %}
390
+ {% if target.name == "index-object" %}
391
+ - clean: remove derived index cache materialization. Help: {{ commandName }} {{ uri }} clean --help
392
+ {% else %}
363
393
  - clean: remove completed local jobs. Help: {{ commandName }} {{ uri }} clean --help
394
+ {% endif %}
364
395
  {% elif predicate == "clear" %}
365
396
  {% if target.name == "metadata-object" %}
366
397
  - clear: clear this metadata map. Help: {{ commandName }} {{ uri }} clear --help
@@ -370,23 +401,19 @@ Predicate commands:
370
401
  {% elif predicate == "create" %}
371
402
  - create: create the archive; use `--replace` only when discarding an existing archive. Help: {{ commandName }} {{ uri }} create --help
372
403
  {% elif predicate == "delete" %}
373
- {% if target.name == "metadata-object" %}
404
+ {% if target.name == "chapter-index-artifact" %}
405
+ - delete: delete this chapter index artifact. Help: {{ commandName }} {{ uri }} delete --help
406
+ {% elif target.name == "metadata-object" %}
374
407
  - delete: delete one metadata key. Help: {{ commandName }} {{ uri }} delete --help
375
408
  {% else %}
376
409
  - delete: delete a config key. Help: {{ commandName }} {{ uri }} delete --help
377
410
  {% endif %}
378
- {% elif predicate == "disable" %}
379
- - disable: disable searchable index support and remove index materialization. Help: {{ commandName }} {{ uri }} disable --help
380
- {% elif predicate == "embed" %}
381
- - embed: use archive-embedded searchable index storage. Help: {{ commandName }} {{ uri }} embed --help
382
- {% elif predicate == "enable" %}
383
- - enable: enable searchable index support and bring it to current. Help: {{ commandName }} {{ uri }} enable --help
384
411
  {% elif predicate == "evidence" %}
385
412
  - evidence: trace this object to source-backed evidence; supports `--reverse` without `--query`. Help: {{ commandName }} {{ uri }} evidence --help
386
413
  {% elif predicate == "export" %}
387
414
  - export: export an archive projection. Help: {{ commandName }} {{ uri }} export --help
388
- {% elif predicate == "external" %}
389
- - external: use local-cache searchable index storage. Help: {{ commandName }} {{ uri }} external --help
415
+ {% elif predicate == "sync" %}
416
+ - sync: rebuild derived index cache from chapter index artifacts. Help: {{ commandName }} {{ uri }} sync --help
390
417
  {% elif predicate == "inspect" %}
391
418
  - inspect: inspect archive readiness. Help: {{ commandName }} {{ uri }} inspect --help
392
419
  {% elif predicate == "move" %}
@@ -11,6 +11,12 @@ Sections:
11
11
  `wikg://local/config/llm`
12
12
  LLM provider, model, optional base URL, client name, and API key.
13
13
 
14
+ `wikg://local/config/embeddings`
15
+ Embedding provider for Dense search index vectors and Dense query vectors:
16
+ - `provider`: `openai` or `openai-compatible`.
17
+ - `model`: embedding model name.
18
+ - `dimensions`: optional positive integer when the provider/model supports dimension control.
19
+
14
20
  `wikg://local/config/concurrent`
15
21
  Local concurrency settings:
16
22
  - `job`: simultaneous generation jobs. Default: 3.
@@ -25,17 +31,21 @@ Sections:
25
31
  Operations:
26
32
  Read a section:
27
33
  {{ commandName }} wikg://local/config/llm
34
+ {{ commandName }} wikg://local/config/embeddings
28
35
  {{ commandName }} wikg://local/config/concurrent
29
36
  {{ commandName }} wikg://local/config/wikispine
30
37
 
31
38
  Replace a section with a JSON object:
32
39
  {{ commandName }} wikg://local/config/llm set '{"provider":"openai","model":"gpt-4.1"}'
40
+ {{ commandName }} wikg://local/config/embeddings set '{"provider":"openai","model":"text-embedding-3-small"}'
33
41
  {{ commandName }} wikg://local/config/concurrent set '{"job":3,"request":6}'
34
42
  {{ commandName }} wikg://local/config/wikispine set '{"provider":"fetch"}'
35
43
 
36
44
  Update one key:
37
45
  {{ commandName }} wikg://local/config/llm put provider openai
38
46
  {{ commandName }} wikg://local/config/llm put model gpt-4.1
47
+ {{ commandName }} wikg://local/config/embeddings put provider openai
48
+ {{ commandName }} wikg://local/config/embeddings put model text-embedding-3-small
39
49
  {{ commandName }} wikg://local/config/concurrent put job 3
40
50
  {{ commandName }} wikg://local/config/concurrent put request 6
41
51
  {{ commandName }} wikg://local/config/wikispine put provider cli
@@ -43,6 +53,7 @@ Operations:
43
53
 
44
54
  Store the API key:
45
55
  {{ commandName }} wikg://local/config/llm put apiKey --secret
56
+ {{ commandName }} wikg://local/config/embeddings put apiKey --secret
46
57
 
47
58
  Secret handling:
48
59
  - `apiKey` is read through an interactive hidden prompt.
@@ -56,6 +67,18 @@ Minimum LLM-backed operation configuration:
56
67
  - `llm.model`
57
68
  - Provider credentials when required by the selected provider.
58
69
 
70
+ Minimum Dense embedding configuration:
71
+ - `embeddings.provider`
72
+ - `embeddings.model`
73
+ - `embeddings.baseURL` when using `openai-compatible`
74
+ - Provider credentials when required by the selected provider.
75
+
76
+ Dense search notes:
77
+ - FTS artifact builds do not use this section.
78
+ - Source and summary embedding artifact builds require this section.
79
+ - Index cache sync does not call the embeddings provider; it uses vectors already stored in artifacts.
80
+ - Query uses this section when the current index cache includes Dense vectors. If Dense query embedding fails, Hybrid query can still fall back to FTS.
81
+
59
82
  One-run overrides:
60
83
  `--llm <json>` can provide an LLM object for one invocation.
61
84
  It accepts direct LLM objects and common OpenAI-compatible aliases such as `baseUrl`.
@@ -68,6 +91,10 @@ Examples:
68
91
  {{ commandName }} wikg://local/config/llm put model gpt-4.1
69
92
  {{ commandName }} wikg://local/config/llm put apiKey --secret
70
93
  {{ commandName }} wikg://local/config/llm test
94
+ {{ commandName }} wikg://local/config/embeddings put provider openai
95
+ {{ commandName }} wikg://local/config/embeddings put model text-embedding-3-small
96
+ {{ commandName }} wikg://local/config/embeddings put apiKey --secret
97
+ {{ commandName }} wikg://local/config/embeddings test
71
98
  {{ commandName }} wikg://local/config/wikispine put provider fetch
72
99
  {{ commandName }} wikg://local/config/wikispine test
73
100
 
@@ -12,7 +12,8 @@ Core model:
12
12
  - The default library is addressed as `wikg://lib`.
13
13
  - A non-default library is addressed as `wikg://lib/<lib-id>`.
14
14
  - The `arc` reserved segment keeps library ids distinct from archive member ids.
15
- - `{{ commandName }} wikg://lib` reads the default library scope; it does not list all library registries or archive memberships.
15
+ - `{{ commandName }} wikg://lib` reads the default library root scope.
16
+ - `{{ commandName }} wikg://lib --query <query>` searches library-wide objects through the library index.
16
17
 
17
18
  Registry discovery:
18
19
  - `{{ commandName }} wikg://lib/registry add --path <folder>` prints the new non-default library URI.
@@ -30,15 +31,16 @@ Library folders:
30
31
  Archive memberships:
31
32
  - `{{ commandName }} wikg://lib/arc scan` recursively discovers `.wikg` files under the default library folder and reconciles registry state.
32
33
  - `{{ commandName }} wikg://lib/arc add --input <path> [--to <relative-wikg-path>]` copies one `.wikg` file into the default library folder and registers it.
33
- - `{{ commandName }} wikg://lib/arc/tree` shows the archive member file tree; use this instead of naked `wikg://lib` or `wikg://lib/arc` when you want to see archive members.
34
+ - `{{ commandName }} wikg://lib` and `{{ commandName }} wikg://lib/arc` separate the library root scope from archive membership management; use `{{ commandName }} wikg://lib/arc/tree` when you want a file tree visualization.
34
35
  - For a non-default library, replace `wikg://lib` with `wikg://lib/<lib-id>` in those `arc` commands.
35
36
  - `{{ commandName }} <library-archive-uri>/path set <relative-wikg-path>` moves or renames one managed archive inside the same folder.
36
37
  - `{{ commandName }} <library-archive-uri> remove --confirm` deletes one managed archive file and unregisters it.
37
38
  - Missing registered files remain visible as missing or stale until `arc scan`, archive `path set`, archive `remove`, or library `path set` resolves them.
38
39
 
39
40
  Library archive shortcuts:
40
- - `wikg://lib/arc/<archive-id>/` addresses one archive membership in the default library.
41
- - `wikg://lib/<lib-id>/arc/<archive-id>/` addresses one archive membership in a non-default library.
41
+ - `wikg://lib/arc/<archive-id>/` addresses one archive member page in the default library.
42
+ - `wikg://lib/<lib-id>/arc/<archive-id>/` addresses one archive member page in a non-default library.
43
+ - The archive member page exposes the managed archive's business entry URIs (`/chapter`, `/entity`, `/triple`, and `inspect`) and keeps registry fields as diagnostic metadata.
42
44
  - Add normal archive object suffixes after the trailing slash to query that archive's contents:
43
45
  {{ commandName }} wikg://lib/arc/<archive-id>/chapter
44
46
  {{ commandName }} wikg://lib/arc/<archive-id>/entity --query "term"
@@ -48,9 +50,11 @@ Library archive shortcuts:
48
50
  - `{{ commandName }} <library-archive-uri>/entity --help` explains the archive object scope reached through that library shortcut.
49
51
 
50
52
  Library-wide query:
53
+ - `{{ commandName }} wikg://lib --query <archive-term>` searches library-wide objects through the library index.
51
54
  - `{{ commandName }} wikg://lib/entity --query <query>` searches across present archive members through the library index.
52
55
  - `{{ commandName }} wikg://lib/triple --query <query>` searches library-wide Knowledge Graph triples.
53
56
  - `{{ commandName }} wikg://lib/chunk --query <query>` searches library-wide Reading Graph chunks.
57
+ - Add `--skip-unindexed` only when you intentionally want to search indexed chapters and ignore uncovered chapters.
54
58
  - Library-wide object predicates such as `evidence`, `related`, and `pack` use the same aggregate library index.
55
59
  - Library-wide object URIs such as `wikg://lib/entity/<qid>` are aggregate views over matching objects from present archive members.
56
60
  - Evidence and related output should be interpreted with the returned archive/member provenance; cite the specific returned evidence or member, not just the library-wide URI.
@@ -58,18 +62,21 @@ Library-wide query:
58
62
 
59
63
  Library index lifecycle:
60
64
  - `{{ commandName }} <library-uri>/index` reads library index status.
61
- - `{{ commandName }} <library-uri>/index enable` rebuilds the aggregate index from registered present archives.
62
- - `{{ commandName }} <library-uri>/index disable` removes the local library index materialization.
65
+ - `{{ commandName }} <library-uri>/index sync` rebuilds the aggregate index cache from registered present archives' artifacts.
66
+ - `{{ commandName }} <library-uri>/index clean` removes the local library index cache.
63
67
  - Library indexes are local derived state under Wiki Graph staging, not files inside the library folder.
64
- - `scan`, `add`, archive `path set`, archive `remove`, and archive writes can make the aggregate index dirty.
65
- - Enable the library index again when library-wide query reports that the index is missing, dirty, or stale.
66
- - Library indexes do not support archive-style `embed` or `external` modes.
68
+ - Sync does not build missing chapter artifacts and does not call an embeddings provider.
69
+ - Dense query still requires `wikg://local/config/embeddings` to embed the query text.
70
+ - When both FTS and Dense exist, library query combines them automatically; there is no separate query mode flag.
71
+ - Library membership changes can make the aggregate index stale.
72
+ - Managed archive writes try to sync an existing library index cache immediately; if sync cannot complete, library-wide query reports the cache as stale or not ready.
73
+ - Sync the library index again when library-wide query reports that the index is missing, stale, or not ready.
67
74
 
68
75
  Concurrency:
69
76
  - Library registry and membership writes hold the library write lock.
70
- - `arc scan`, `arc add`, library `path set`, library archive `path set`, library archive `remove`, and library index `enable` or `disable` cannot run concurrently for the same library.
71
- - Library-wide `--query`, naked scope enumeration, evidence, related, and pack commands read the current aggregate index.
72
- - If a write marks the aggregate index dirty, rebuild it with `{{ commandName }} <library-uri>/index enable` before relying on library-wide query results.
77
+ - `arc scan`, `arc add`, library `path set`, library archive `path set`, library archive `remove`, and library index `sync` or `clean` cannot run concurrently for the same library.
78
+ - Library-wide object `--query`, object scope enumeration, evidence, related, and pack commands read the current aggregate index.
79
+ - If a write cannot resync the aggregate index, rebuild it with `{{ commandName }} <library-uri>/index sync` before relying on library-wide query results.
73
80
 
74
81
  Upgrade and cleanup:
75
82
  - Upgrade a whole library with `{{ commandName }} maintenance upgrade wikg://lib` or `{{ commandName }} maintenance upgrade wikg://lib/<lib-id>`.
@@ -10,47 +10,67 @@ Purpose:
10
10
  Start with inspect:
11
11
  {{ commandName }} <archive-uri> inspect
12
12
 
13
- Archive FTS readiness:
14
- Archive `--query` uses that archive's FTS index. Without a current index, keyword retrieval is unavailable for archive scopes, evidence, and related results.
15
-
16
- Default behavior:
17
- `{{ commandName }} <archive-uri>/index enable` enables the searchable index as local cache outside the `.wikg` archive.
18
- This keeps the archive smaller, but the index does not travel with copied archive files.
19
- If another user receives the `.wikg`, they usually need to enable their own local index.
20
- After an index is enabled, CLI archive writes keep it synchronized automatically.
21
- Do not enable the index again after every generation job.
22
-
23
- Portable behavior:
24
- `{{ commandName }} <archive-uri>/index embed` switches to an archive-embedded searchable index and ensures it is current.
25
- This makes the archive more self-contained, but increases file size.
26
- After embedding, CLI archive writes keep the embedded index synchronized automatically.
27
- If you are unsure, enable the local index first and embed only when distribution convenience matters.
13
+ Archive index readiness:
14
+ Ordinary archive source query needs chapter index artifacts. For each content chapter, at least one of these must be current:
15
+ - FTS artifact: `<chapter-uri>/index/fts`
16
+ - Source embedding artifact: `<chapter-uri>/index/embedding/source`
17
+
18
+ Summary semantic retrieval is separate:
19
+ - Summary embedding artifact: `<chapter-uri>/index/embedding/summary`
20
+ - Summary embedding alone does not satisfy ordinary archive source query readiness.
21
+
22
+ Artifact behavior:
23
+ Chapter artifacts are stored inside the `.wikg` archive.
24
+ Build FTS artifacts with the chapter index URI or queue target `index-fts`.
25
+ Build source embedding artifacts with the chapter index URI or queue target `index-embedding-source`.
26
+ Build summary embedding artifacts with the chapter index URI or queue target `index-embedding-summary`.
27
+ Building embedding artifacts requires `wikg://local/config/embeddings` and may call the configured provider.
28
+
29
+ Cache behavior:
30
+ `{{ commandName }} <archive-uri>/index sync` builds or repairs local `index.db` cache from artifacts.
31
+ Sync does not build missing artifacts and does not call an embeddings provider.
32
+ Archive-level query can create a missing cache from current artifacts on first use, but explicit sync avoids the first-query delay.
33
+ If query reports a missing or outdated cache, run `{{ commandName }} <archive-uri>/index sync`; library query always requires a current library cache.
34
+ `{{ commandName }} <archive-uri>/index clean` deletes only the local cache.
35
+
36
+ Query behavior:
37
+ FTS artifact only: FTS query.
38
+ Source embedding artifact only: semantic/Dense-only query.
39
+ Both artifacts: Hybrid query with automatic fusion.
40
+ Broad query is grouped by result type first: chapter/title, object, chunk, then source/summary text.
41
+ FTS, Dense, and Hybrid ranking affect ordering inside those result groups and evidence-backed object ranking; they do not flatten the type groups into one global rerank.
42
+ Missing both FTS and source embedding for any content chapter in scope: ordinary query fails.
43
+ Add `--skip-unindexed` to a scope query, `related --query`, or `evidence --query` only when you want to search indexed chapters and ignore uncovered chapters.
44
+ Results produced with `--skip-unindexed` are intentionally partial; inspect coverage before treating absent chapters as negative evidence.
45
+ Summary embedding artifacts are for summary semantic retrieval and are not the minimum requirement for ordinary source query.
28
46
 
29
47
  Commands:
30
48
  {{ commandName }} <archive-uri>/index
31
- {{ commandName }} <archive-uri>/index enable --help
32
- {{ commandName }} <archive-uri>/index disable --help
33
- {{ commandName }} <archive-uri>/index embed --help
34
- {{ commandName }} <archive-uri>/index external --help
49
+ {{ commandName }} <chapter-uri>/index/fts --help
50
+ {{ commandName }} <chapter-uri>/index/embedding/source --help
51
+ {{ commandName }} <chapter-uri>/index/embedding/summary --help
52
+ {{ commandName }} <archive-uri>/index sync --help
53
+ {{ commandName }} <archive-uri>/index clean --help
35
54
 
36
55
  Library index readiness:
37
- Library-wide scopes use an aggregate library index instead of one standalone archive index.
38
- Without a current library index, library-wide `--query`, evidence, related, pack, and scope enumeration may be unavailable.
56
+ Library-wide scopes use an aggregate library index cache built from member archives' artifacts.
57
+ Unlike a free archive, library query does not lazy-sync this cache. It must already be current.
39
58
 
40
59
  Default behavior:
41
- `{{ commandName }} wikg://lib/index enable` rebuilds the default library index from registered present archives.
42
- `{{ commandName }} wikg://lib/<lib-id>/index enable` rebuilds a non-default library index.
60
+ `{{ commandName }} wikg://lib/index sync` rebuilds the default library index cache from registered present archives.
61
+ `{{ commandName }} wikg://lib/<lib-id>/index sync` rebuilds a non-default library index cache.
43
62
  The index is local derived state under Wiki Graph staging, not a file inside the library folder.
63
+ Sync does not build missing chapter artifacts and does not call an embeddings provider.
44
64
 
45
- Dirty behavior:
46
- `scan`, `add`, archive `path set`, archive `remove`, archive writes, and membership changes can mark the aggregate index dirty.
47
- Re-enable the library index when library-wide query reports that it is missing, dirty, or stale.
48
- Library indexes do not support archive-style `embed` or `external` modes.
65
+ Stale behavior:
66
+ Library membership changes can make the aggregate index stale.
67
+ Managed archive writes try to sync an existing library index cache immediately.
68
+ Sync the library index when library-wide query reports that it is missing, stale, or not ready.
49
69
 
50
70
  Commands:
51
71
  {{ commandName }} wikg://lib/index
52
- {{ commandName }} wikg://lib/index enable --help
53
- {{ commandName }} wikg://lib/index disable --help
72
+ {{ commandName }} wikg://lib/index sync --help
73
+ {{ commandName }} wikg://lib/index clean --help
54
74
  {{ commandName }} help library
55
75
 
56
76
  Generation readiness:
@@ -82,6 +102,22 @@ LLM readiness:
82
102
  {{ commandName }} wikg://local/config/llm put apiKey --secret
83
103
  {{ commandName }} wikg://local/config/llm test
84
104
 
105
+ Embeddings readiness:
106
+ `wikg://local/config/embeddings` is required for source or summary embedding artifact builds and for querying an index cache that includes Dense vectors.
107
+ Hybrid query can still fall back to FTS if a Dense query embedding request fails.
108
+
109
+ Minimum configuration:
110
+ provider
111
+ model
112
+ baseURL when using openai-compatible
113
+ provider credentials when required
114
+
115
+ Common path:
116
+ {{ commandName }} wikg://local/config/embeddings put provider openai
117
+ {{ commandName }} wikg://local/config/embeddings put model text-embedding-3-small
118
+ {{ commandName }} wikg://local/config/embeddings put apiKey --secret
119
+ {{ commandName }} wikg://local/config/embeddings test
120
+
85
121
  WikiSpine readiness:
86
122
  Knowledge Graph generation also requires WikiSpine.
87
123
  WikiSpine provides entity candidate lookup for Knowledge Graph construction.
@@ -108,13 +144,14 @@ WikiSpine readiness:
108
144
  https://raw.githubusercontent.com/oomol-lab/wiki-graph/refs/heads/main/docs/wikispine-runtime.md
109
145
 
110
146
  How to choose:
111
- - Need keyword retrieval on this machine: enable local index.
112
- - Need keyword retrieval across a managed folder: enable the library index.
113
- - Need to send a more self-contained `.wikg`: consider embedding the index.
114
- - Need to intentionally drop query support or index materialization: disable index.
147
+ - Need lexical retrieval: build chapter FTS artifacts, then sync cache if you do not want lazy sync.
148
+ - Need semantic retrieval: configure embeddings, build source embedding artifacts, then sync cache.
149
+ - Need both lexical and semantic retrieval: build both FTS and source embedding artifacts, then sync the archive index cache if it already exists or query reports it outdated; query can create a missing archive cache and chooses Hybrid automatically once the cache is current.
150
+ - Need searchable retrieval across a managed folder: sync the library index cache.
151
+ - Need to intentionally drop local cache: clean index cache; artifacts remain in the archive.
115
152
  - Need generated Reading Graph or summaries: configure LLM and use local jobs.
116
153
  - Need Knowledge Graph: configure LLM and WikiSpine, then use local jobs.
117
- - Not sure whether to embed index or generate missing data: keep the archive external/local first, inspect again, and decide later.
154
+ - Not sure what is missing: run inspect and check index artifact coverage before syncing cache.
118
155
 
119
156
  Where to go next:
120
157
  - Normal workflow:
@@ -50,15 +50,18 @@ After inspect:
50
50
  {{ commandName }} wikg:///Users/me/book.wikg --query "attention memory"
51
51
  {{ commandName }} wikg:///Users/me/book.wikg/chapter/part --query "attention"
52
52
 
53
- - If inspect reports missing or unavailable index support, inspect the index target before enabling anything.
53
+ - If inspect reports missing index artifacts or cache, inspect the index targets before syncing cache.
54
54
  {{ commandName }} <archive-uri>/index --help
55
- {{ commandName }} <archive-uri>/index enable --help
56
- Read readiness before deciding whether to keep the index local or embed it:
55
+ {{ commandName }} <archive-uri>/chapter/<chapter-id>/index/fts --help
56
+ {{ commandName }} <archive-uri>/chapter/<chapter-id>/index/embedding/source --help
57
+ {{ commandName }} <archive-uri>/chapter/<chapter-id>/index/embedding/summary --help
58
+ {{ commandName }} <archive-uri>/index sync --help
59
+ Read readiness before deciding whether artifacts or cache are missing:
57
60
  {{ commandName }} help readiness
58
61
 
59
- - If library-wide query reports a missing or dirty index, inspect the library index target and rebuild it when appropriate.
62
+ - If library-wide query reports a missing, stale, or not-ready index cache, inspect the library index target and sync it when appropriate.
60
63
  {{ commandName }} wikg://lib/index --help
61
- {{ commandName }} wikg://lib/index enable --help
64
+ {{ commandName }} wikg://lib/index sync --help
62
65
  {{ commandName }} help library
63
66
 
64
67
  - If Reading Graph data is missing, chunk `--query`, related expansion, and pack may be unavailable or incomplete.
@@ -83,8 +86,8 @@ Finding material:
83
86
  {{ commandName }} wikg://book.wikg/chapter/part
84
87
 
85
88
  - Need to search across all archives in a managed library:
86
- Use library-wide scopes after enabling the library index.
87
- {{ commandName }} wikg://lib/index enable
89
+ Use library-wide scopes after syncing the library index cache.
90
+ {{ commandName }} wikg://lib/index sync
88
91
  {{ commandName }} wikg://lib/entity --query "attention" --evidence 2
89
92
  {{ commandName }} wikg://lib/triple --query "attention memory" --evidence 2
90
93
  {{ commandName }} wikg://lib/chunk --query "attention memory" --evidence 2
@@ -168,7 +171,7 @@ When to read deeper:
168
171
  {{ commandName }} help format
169
172
  - LLM, concurrency, and WikiSpine provider setup:
170
173
  {{ commandName }} help config
171
- - FTS index, LLM, WikiSpine, and generated-data prerequisites:
174
+ - Search index, embeddings, LLM, WikiSpine, and generated-data prerequisites:
172
175
  {{ commandName }} help readiness
173
176
  - Library registry, membership, path binding, and aggregate index:
174
177
  {{ commandName }} help library
@@ -23,6 +23,8 @@ Archive locator:
23
23
  Library locators:
24
24
  - `wikg://lib` addresses the default library.
25
25
  - `wikg://lib/<lib-id>` addresses a non-default library.
26
+ - Naked library scopes enumerate library-wide objects; `--query` on a naked library scope searches library-wide objects through the library index.
27
+ - Use explicit object lenses such as `/entity`, `/triple`, `/chapter`, or `/chunk` for archive object search.
26
28
  - `wikg://lib/arc/<archive-id>/` addresses one archive membership in the default library.
27
29
  - `wikg://lib/<lib-id>/arc/<archive-id>/` addresses one archive membership in a non-default library.
28
30
  - Add ordinary archive object suffixes after the trailing slash to query a managed archive:
@@ -46,7 +48,7 @@ Short and located URIs:
46
48
 
47
49
  Command routing:
48
50
  - `{{ commandName }} <scope-uri>` enumerates objects.
49
- - `{{ commandName }} <scope-uri> --query <query>` applies keyword retrieval within that scope.
51
+ - `{{ commandName }} <scope-uri> --query <query>` applies search index retrieval within that scope.
50
52
  - `{{ commandName }} <object-uri>` reads one object.
51
53
  - `{{ commandName }} <uri> --help` explains that URI target and its supported predicates.
52
54
  - `{{ commandName }} <uri> <predicate> --help` explains a URI-bound predicate.
@@ -67,12 +69,16 @@ Common command shapes:
67
69
  {{ commandName }} wikg://lib/entity --query <query>
68
70
  {{ commandName }} wikg://lib/arc/<archive-id>/entity --query <query>
69
71
  {{ commandName }} wikg://local/job
70
- {{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task <reading-graph|reading-summary|knowledge-graph> --accept-cost
72
+ {{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task <index-fts|index-embedding-source|index-embedding-summary|reading-graph|reading-summary|knowledge-graph> [--accept-cost]
71
73
  {{ commandName }} wikg://local/job/<job-id>
72
74
 
73
75
  Scope URIs:
74
- Scope URIs enumerate objects by default. With `--query <query>`, they apply keyword retrieval within that scope.
76
+ Scope URIs enumerate objects by default. With `--query <query>`, they apply search index retrieval within that scope.
75
77
  A chapter URI used as a scope covers that chapter's full subtree by default.
78
+ Query is strict by default: every chapter in scope must have a current FTS artifact or source embedding artifact.
79
+ Add `--skip-unindexed` only when you intentionally want to search indexed chapters and ignore uncovered chapters.
80
+ Broad query is grouped by result type before ranking within each group. Use `/chunk`, `/entity`, or `/triple` lenses when you want one object family instead of the mixed broad result. Query an archive or chapter scope to find source or summary text, then read the returned `/source` or `/summary` object.
81
+ Search uses FTS-only, Dense-only, or Hybrid automatically from the current artifacts and index cache; use `{{ commandName }} help readiness` to inspect what must exist.
76
82
  Add `--depth <N>` to limit chapter subtree expansion:
77
83
  - no `--depth`: full subtree
78
84
  - `--depth 0`: only the current chapter, or root-level chapters for `<archive-uri>/chapter`
@@ -154,6 +160,7 @@ Literal source passages:
154
160
  Use `--query` on the narrowest useful scope when the task is to find source wording or passages mentioning a literal term:
155
161
  {{ commandName }} <archive-uri> --query "term"
156
162
  {{ commandName }} <archive-uri>/chapter/<path> --query "term"
163
+ {{ commandName }} <archive-uri> --query "term" --skip-unindexed
157
164
  {{ commandName }} <archive-uri>/chapter/<path>/source#20..30
158
165
 
159
166
  Entity-grounded evidence:
@@ -176,6 +183,8 @@ Recovery hints:
176
183
  {{ commandName }} <archive-uri>/chapter/tree
177
184
  {{ commandName }} help readiness
178
185
  - Missing generated objects: start the relevant generation job for the chapter.
186
+ {{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task index-fts
187
+ {{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task index-embedding-source --accept-cost
179
188
  {{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task reading-graph --accept-cost
180
189
  {{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task reading-summary --accept-cost
181
190
  {{ commandName }} help readiness