@bodhi-ventures/aiocs 0.5.3 → 0.6.0

package/docs/json-contract.md

@@ -47,11 +47,16 @@ All of these support the root-level `--json` flag:
 - `doctor`
 - `source upsert`
 - `source list`
+- `source describe`
+- `source context show`
+- `source context upsert`
 - `fetch`
 - `canary`
 - `refresh due`
 - `snapshot list`
 - `diff`
+- `page list`
+- `page show`
 - `project link`
 - `project unlink`
 - `backup export`
@@ -61,6 +66,9 @@ All of these support the root-level `--json` flag:
 - `embeddings clear`
 - `embeddings run`
 - `search`
+- `retrieve`
+- `learning save`
+- `learning list`
 - `verify coverage`
 - `show`
 
@@ -177,6 +185,63 @@ Summary status values:
 }
 ```
 
+### `source.describe`
+
+```json
+{
+  "source": {
+    "id": "hyperliquid",
+    "kind": "web",
+    "label": "Hyperliquid",
+    "specPath": "/absolute/path/to/spec.yaml"
+  },
+  "context": {
+    "sourceId": "hyperliquid",
+    "context": {
+      "purpose": "Primary exchange docs",
+      "summary": "Use for exchange API and auth questions.",
+      "topicHints": ["authentication", "orders"],
+      "commonLocations": [],
+      "gotchas": [],
+      "authNotes": []
+    },
+    "createdAt": "2026-04-08T12:00:00.000Z",
+    "updatedAt": "2026-04-08T12:00:00.000Z"
+  },
+  "latestSnapshot": {
+    "snapshotId": "snp_...",
+    "detectedVersion": null,
+    "createdAt": "2026-04-08T12:05:00.000Z",
+    "pageCount": 139
+  },
+  "recentLearnings": []
+}
+```
+
+### `source.context.show` and `source.context.upsert`
+
+```json
+{
+  "sourceId": "hyperliquid",
+  "context": {
+    "purpose": "Primary exchange docs",
+    "summary": "Use for exchange API and auth questions.",
+    "topicHints": ["authentication", "orders"],
+    "commonLocations": [
+      {
+        "label": "Auth reference",
+        "url": "https://example.dev/docs/auth",
+        "note": "Start here for API setup questions."
+      }
+    ],
+    "gotchas": ["Read the full page before answering from isolated chunks."],
+    "authNotes": ["Wallet signing docs are separate from trader docs."]
+  },
+  "createdAt": "2026-04-08T12:00:00.000Z",
+  "updatedAt": "2026-04-08T12:00:00.000Z"
+}
+```
+
 ### `fetch` and `refresh.due`
 
 ```json
@@ -236,6 +301,46 @@ Summary status values:
 }
 ```
 
+### `page.list`
+
+```json
+{
+  "sourceId": "hyperliquid",
+  "snapshotId": "snp_...",
+  "total": 2,
+  "limit": 50,
+  "offset": 0,
+  "hasMore": false,
+  "pages": [
+    {
+      "url": "https://example.dev/docs/start",
+      "title": "Docs Start",
+      "pageKind": "document",
+      "filePath": null,
+      "language": null,
+      "markdownLength": 245
+    }
+  ]
+}
+```
+
+### `page.show`
+
+```json
+{
+  "sourceId": "hyperliquid",
+  "snapshotId": "snp_...",
+  "page": {
+    "url": "https://example.dev/docs/start",
+    "title": "Docs Start",
+    "markdown": "# Docs Start\n\nFull page content...",
+    "pageKind": "document",
+    "filePath": null,
+    "language": null
+  }
+}
+```
+
 ### `project.link` and `project.unlink`
 
 ```json
@@ -285,6 +390,97 @@ Summary status values:
 }
 ```
 
+### `retrieve`
+
+```json
+{
+  "query": "where is maker flow documented",
+  "modeRequested": "lexical",
+  "modeUsed": "lexical",
+  "sourceScope": ["hyperliquid"],
+  "sourceHints": [
+    {
+      "sourceId": "hyperliquid",
+      "score": 2,
+      "context": {
+        "purpose": "Primary exchange docs",
+        "summary": "Use for exchange API and auth questions.",
+        "topicHints": ["maker flow"],
+        "commonLocations": [],
+        "gotchas": [],
+        "authNotes": []
+      },
+      "matchedCommonLocations": []
+    }
+  ],
+  "matchedLearnings": [],
+  "avoidedLearnings": [],
+  "search": {
+    "total": 1,
+    "limit": 20,
+    "offset": 0,
+    "hasMore": false,
+    "results": []
+  },
+  "pages": [
+    {
+      "sourceId": "hyperliquid",
+      "snapshotId": "snp_...",
+      "url": "https://example.dev/docs/start",
+      "title": "Docs Start",
+      "markdown": "# Docs Start\n\nFull page content...",
+      "pageKind": "document",
+      "filePath": null,
+      "language": null
+    }
+  ]
+}
+```
+
+### `learning.save`
+
+```json
+{
+  "learning": {
+    "learningId": "rte_...",
+    "sourceId": "hyperliquid",
+    "snapshotId": "snp_...",
+    "learningType": "discovery",
+    "intent": "where is maker flow documented",
+    "pageUrl": "https://example.dev/docs/start",
+    "filePath": null,
+    "title": "Docs Start",
+    "note": "Start here for maker-flow docs questions.",
+    "searchTerms": ["maker flow"],
+    "createdAt": "2026-04-08T12:00:00.000Z",
+    "updatedAt": "2026-04-08T12:00:00.000Z"
+  }
+}
+```
+
+### `learning.list`
+
+```json
+{
+  "learnings": [
+    {
+      "learningId": "rte_...",
+      "sourceId": "hyperliquid",
+      "snapshotId": "snp_...",
+      "learningType": "discovery",
+      "intent": "where is maker flow documented",
+      "pageUrl": "https://example.dev/docs/start",
+      "filePath": null,
+      "title": "Docs Start",
+      "note": "Start here for maker-flow docs questions.",
+      "searchTerms": ["maker flow"],
+      "createdAt": "2026-04-08T12:00:00.000Z",
+      "updatedAt": "2026-04-08T12:00:00.000Z"
+    }
+  ]
+}
+```
+
 ### `search`
 
 ```json

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bodhi-ventures/aiocs",
-  "version": "0.5.3",
+  "version": "0.6.0",
   "license": "MIT",
   "type": "module",
   "description": "Local-only documentation store, fetcher, and search CLI for AI agents.",

package/skills/aiocs/SKILL.md

@@ -22,6 +22,11 @@ Use this skill when you need authoritative local documentation lookup through th
 - Check `source_list` or scoped `search` before assuming a source is missing.
 - Use `aiocs` first for the bundled `hyperliquid` source and for any repo or machine that already relies on `~/.aiocs`.
 - This skill is the default read/search path. If the task requires source creation, force fetch, targeted refresh, or canary remediation, also load `aiocs-curation`.
+- Default to the awareness loop before answering:
+  - `source describe` to understand the source
+  - `page list` to inspect likely full pages
+  - `search` to shortlist candidate chunks
+  - `retrieve` or `page show` to read the full stored page before synthesizing an answer
 - Only fall back to live browsing when:
   - the source is not present in `aiocs`
   - the user explicitly wants the live site
@@ -38,6 +43,7 @@ Use this skill when you need authoritative local documentation lookup through th
 
 ## Search defaults for agents
 
+- Prefer `retrieve` when the user wants an answer grounded in the right document, not just raw chunk matches.
 - Default to `search` with `mode=auto`.
 - Use `mode=lexical` for exact identifiers, section titles, endpoint names, and error strings.
 - Use `--path` / `pathPatterns` and `--language` / `languages` when searching repo/code sources.
@@ -61,6 +67,15 @@ aiocs --json init --no-fetch
 
 ## Core commands
 
+Inspect source awareness and stored pages:
+
+```bash
+aiocs --json source describe hyperliquid
+aiocs --json source context show hyperliquid
+aiocs --json page list hyperliquid --query "auth"
+aiocs --json page show hyperliquid --url "https://hyperliquid.gitbook.io/hyperliquid-docs/for-developers/api"
+```
+
 Search the shared catalog:
 
 ```bash
@@ -77,6 +92,13 @@ Inspect a specific chunk:
 aiocs --json show 42
 ```
 
+Assemble answer context from awareness, search, and full-page reads:
+
+```bash
+aiocs --json retrieve "where is maker flow documented" --source hyperliquid --mode lexical
+aiocs --json retrieve "api introduction" --source bulk-trade --mode lexical --page-limit 2
+```
+
 Inspect source availability and health:
 
 ```bash
@@ -119,9 +141,13 @@ The `aiocs-mcp` server exposes the same core operations without shell parsing:
 - `doctor`
 - `init`
 - `source_list`
+- `source_describe`
+- `source_context_show`
 - `canary`
 - `snapshot_list`
 - `diff_snapshots`
+- `page_list`
+- `page_show`
 - `project_link`
 - `project_unlink`
 - `embeddings_status`
@@ -131,20 +157,24 @@ The `aiocs-mcp` server exposes the same core operations without shell parsing:
 - `backup_export`
 - `backup_import`
 - `search`
+- `retrieve_context`
 - `show`
+- `learning_list`
 - `verify_coverage`
 - `batch`
 
-Mutation-capable MCP tools such as `source_upsert`, `refresh_due`, and `fetch` belong to `aiocs-curation`.
+Mutation-capable MCP tools such as `source_upsert`, `source_context_upsert`, `learning_save`, `refresh_due`, and `fetch` belong to `aiocs-curation`.
 
 ## Recommended Codex workflow
 
 1. If runtime health is in doubt, run `doctor`.
-2. Run `source_list` to see whether the source already exists.
-3. Use `search` in `auto` mode first, then `show` for the selected chunk.
-4. Use `canary`, `diff_snapshots`, or `verify_coverage` when the question is about drift, changes, or completeness.
-5. If the source is missing or stale and the next step is to mutate `aiocs`, load `aiocs-curation`.
-6. Use `batch` when combining list/search/show or diff/coverage checks in one pass.
+2. Run `source_list` or `source_describe` to see whether the source already exists and what it covers.
+3. Use `page_list` when the question is likely page-oriented, or `search` when chunk recall is the fastest shortlist path.
+4. Use `retrieve` when the answer should be grounded in full-page reads rather than isolated chunks.
+5. Use `page_show` when you already know the exact page to read.
+6. Use `canary`, `diff_snapshots`, or `verify_coverage` when the question is about drift, changes, or completeness.
+7. If the source is missing or stale and the next step is to mutate `aiocs`, load `aiocs-curation`.
+8. Use `batch` when combining describe/page-list/search/show or diff/coverage checks in one pass.
 
 ## Operational notes
 
@@ -152,6 +182,7 @@ Mutation-capable MCP tools such as `source_upsert`, `refresh_due`, and `fetch` b
 - Default state root: `~/.aiocs/data`, `~/.aiocs/config`, and `~/.aiocs/sources`.
 - Use `aiocs daemon` or the Docker daemon service when the catalog should stay fresh automatically.
 - `aiocs search --mode auto` is the right default for agents; it uses hybrid retrieval only when embeddings are current and healthy for the requested scope.
+- `aiocs retrieve` is the right default when the agent should answer from the best full document, not just the top chunk.
 - The Docker Compose stack includes a dedicated `aiocs-qdrant` container and expects Ollama to be reachable separately.
 - Canaries are the first place to look when a docs site changed and fetches started degrading.
 - Newly added or changed sources become due immediately, so `refresh due <source-id>` is the safe first refresh path after upsert.

package/skills/aiocs-curation/SKILL.md

@@ -13,6 +13,8 @@ Use this skill when you need to add, refresh, repair, or otherwise mutate `aiocs
 - An existing source is stale and should be refreshed instead of bypassed.
 - A source spec needs to be created, updated, or upserted under `~/.aiocs/sources`.
 - A reusable external git repository should be added as a `kind: git` source under `~/.aiocs/sources`.
+- A source should gain curated metadata such as purpose, topic hints, common locations, gotchas, or auth notes.
+- A durable routing hint should be saved because a discovery or failed path will help future retrieval.
 - A canary is failing and the source needs remediation or targeted refetch.
 - The user explicitly wants `aiocs` maintenance, source onboarding, or catalog repair.
 
@@ -59,6 +61,12 @@ mkdir -p ~/.aiocs/sources
 aiocs --json source upsert ~/.aiocs/sources/my-source.yaml
 ```
 
+Add or update curated source context:
+
+```bash
+aiocs --json source context upsert my-source ~/.aiocs/source-context/my-source.yaml
+```
+
 Refresh only what is needed:
 
 ```bash
@@ -69,6 +77,13 @@ aiocs --json fetch my-source
 aiocs --json canary my-source
 ```
 
+Persist durable routing learnings when they will help future retrieval:
+
+```bash
+aiocs --json learning save --source my-source --kind discovery --intent "where is auth documented" --page-url "https://..."
+aiocs --json learning save --source my-source --kind negative --intent "where is auth documented" --page-url "https://..." --note "Overview page is not enough."
+```
+
 Heavy maintenance remains explicit:
 
 ```bash
@@ -84,9 +99,11 @@ The `aiocs-mcp` server exposes the same curation operations without shell parsin
 - `doctor`
 - `source_list`
 - `source_upsert`
+- `source_context_upsert`
 - `canary`
 - `fetch`
 - `refresh_due`
+- `learning_save`
 - `embeddings_status`
 - `embeddings_backfill`
 - `embeddings_clear`
@@ -98,13 +115,17 @@ The `aiocs-mcp` server exposes the same curation operations without shell parsin
 1. Run `doctor` or `source_list` if runtime health, presence, or freshness is unclear.
 2. If the source already exists and is due, prefer `refresh due <source-id>`.
 3. If the source is missing but worth curating, create a spec under `~/.aiocs/sources`, then `source_upsert` it.
-4. After upsert, use `refresh due <source-id>` as the safe first fetch path.
-5. Use `canary` when the site changed or extraction drift is suspected.
-6. Escalate to `fetch <source-id>` or `fetch all` only for explicit maintenance or when due-based refresh is not enough.
+4. If source-level context will help future retrieval, upsert a curated source-context file.
+5. After upsert, use `refresh due <source-id>` as the safe first fetch path.
+6. Use `canary` when the site changed or extraction drift is suspected.
+7. Save routing learnings only when the discovery is durable enough to help future runs.
+8. Escalate to `fetch <source-id>` or `fetch all` only for explicit maintenance or when due-based refresh is not enough.
 
 ## Operational notes
 
 - New or changed sources become due immediately after `source_upsert`.
+- `source context upsert` is the right place for durable source-level guidance; do not overload source specs with retrieval notes.
+- `learning save` is for durable routing memory, not one-off scratch notes.
 - `~/.aiocs/sources` and bundled repo sources behave the same once bootstrapped into the catalog.
 - Targeted refresh is the default. Broad refresh is a maintenance task, not a normal answering step.
 - Use `aiocs` for read/search flows and this skill only for catalog mutation.