@bodhi-ventures/aiocs 0.1.2 → 0.3.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bodhi-ventures/aiocs",
-  "version": "0.1.2",
+  "version": "0.3.0",
   "license": "MIT",
   "type": "module",
   "description": "Local-only documentation store, fetcher, and search CLI for AI agents.",
@@ -48,27 +48,30 @@
     "test:watch": "vitest"
   },
   "dependencies": {
-    "@modelcontextprotocol/sdk": "^1.28.0",
-    "@mozilla/readability": "^0.6.0",
+    "@lmstudio/sdk": "1.5.0",
+    "@modelcontextprotocol/sdk": "1.28.0",
+    "@mozilla/readability": "0.6.0",
     "@qdrant/js-client-rest": "1.17.0",
-    "better-sqlite3": "^12.4.1",
-    "commander": "^14.0.1",
-    "jsdom": "^27.0.1",
-    "playwright": "^1.57.0",
-    "turndown": "^7.2.1",
-    "turndown-plugin-gfm": "^1.0.2",
-    "yaml": "^2.8.1",
-    "zod": "^4.1.12"
+    "better-sqlite3": "12.4.1",
+    "commander": "14.0.1",
+    "csv-parse": "6.2.1",
+    "jsdom": "27.0.1",
+    "pdf-parse": "2.4.5",
+    "playwright": "1.57.0",
+    "turndown": "7.2.1",
+    "turndown-plugin-gfm": "1.0.2",
+    "yaml": "2.8.1",
+    "zod": "4.1.12"
   },
   "devDependencies": {
-    "@types/better-sqlite3": "^7.6.13",
-    "@types/jsdom": "^21.1.7",
-    "@types/node": "^24.7.2",
-    "@types/turndown": "^5.0.5",
-    "execa": "^9.6.0",
-    "tsup": "^8.5.0",
-    "tsx": "^4.20.6",
-    "typescript": "^5.9.3",
-    "vitest": "^3.2.4"
+    "@types/better-sqlite3": "7.6.13",
+    "@types/jsdom": "21.1.7",
+    "@types/node": "24.7.2",
+    "@types/turndown": "5.0.5",
+    "execa": "9.6.0",
+    "tsup": "8.5.0",
+    "tsx": "4.20.6",
+    "typescript": "5.9.3",
+    "vitest": "3.2.4"
   }
 }

package/skills/aiocs/SKILL.md

@@ -1,30 +1,33 @@
+---
+name: aiocs
+description: Use when authoritative local documentation lookup should come from the shared aiocs catalog under ~/.aiocs instead of live browsing.
+---
+
 # aiocs
 
-Use this skill when you need authoritative local documentation search, inspection, safe refresh, or bootstrap through the shared `aiocs` catalog under `~/.aiocs`.
+Use this skill when you need authoritative local documentation lookup through the shared `aiocs` catalog under `~/.aiocs`.
 
 ## When to use it
 
 - The user is asking about exchange or product docs that may already exist in the local `aiocs` catalog.
 - You need authoritative local docs for an exchange, SDK, or product without browsing the live site every time.
-- You want machine-readable search/show results for an AI agent.
-- You need to detect source drift or compare snapshot changes over time.
-- You want hybrid docs retrieval with lexical plus semantic/vector recall.
-- You need to bootstrap or validate `aiocs` on a new machine.
-- You want to keep the local docs catalog warm through the `aiocs` daemon or MCP server.
-- You need to back up or restore the shared catalog.
+- You need reusable reference search over a curated external git repository that already lives in `aiocs`.
+- You need to read from a compiled `aiocs` workspace wiki or inspect derived workspace artifacts.
+- You need to inspect workspace status, queued compile health, or raw ingested evidence before deciding whether curation is needed.
+- You want machine-readable search/show/diff/coverage results for an AI agent.
+- You need hybrid docs retrieval with lexical plus semantic/vector recall.
+- You need to validate runtime health before relying on the local docs catalog.
 
 ## Trigger guidance for Codex
 
 - Prefer `aiocs` before live web browsing when the requested docs may already be in the local catalog.
 - Check `source_list` or scoped `search` before assuming a source is missing.
 - Use `aiocs` first for the bundled `hyperliquid` source and for any repo or machine that already relies on `~/.aiocs`.
-- If a source is missing, only add it when it is worth curating for future reuse.
-- Prefer `refresh due <source-id>` over force `fetch <source-id>` whenever freshness is the real goal.
-- Do not use `fetch all` as a normal answering path; reserve it for explicit user requests or maintenance flows.
+- This skill is the default read/search path. If the task requires source creation, force fetch, targeted refresh, or canary remediation, also load `aiocs-curation`.
 - Only fall back to live browsing when:
   - the source is not present in `aiocs`
   - the user explicitly wants the live site
-  - the local catalog is stale or broken and the answer cannot wait for refresh/canary remediation
+  - the local catalog is stale or broken and the answer cannot wait for curation/remediation
 - If you need multiple docs operations in MCP, use `batch` instead of many small round trips.
 
 ## Preferred interfaces
@@ -39,6 +42,7 @@ Use this skill when you need authoritative local documentation search, inspectio
 
 - Default to `search` with `mode=auto`.
 - Use `mode=lexical` for exact identifiers, section titles, endpoint names, and error strings.
+- Use `--path` / `pathPatterns` and `--language` / `languages` when searching repo/code sources.
 - Use `mode=hybrid` for conceptual questions when embeddings are healthy.
 - Use `mode=semantic` only when you explicitly want vector-only recall.
 - When citing results, include `sourceId`, `snapshotId`, and `pageUrl` when they materially help traceability.
@@ -57,12 +61,6 @@ Bootstrap managed sources from the repo bundle and `~/.aiocs/sources`:
 docs --json init --no-fetch
 ```
 
-User-managed source specs live under:
-
-```bash
-~/.aiocs/sources
-```
-
 ## Core commands
 
 Search the shared catalog:
@@ -72,6 +70,7 @@ docs --json search "maker flow" --source hyperliquid
 docs --json search "maker flow" --all
 docs --json search "maker flow" --source hyperliquid --limit 5 --offset 0
 docs --json search "maker flow" --source hyperliquid --mode hybrid
+docs --json search "WebSocketTransport" --source nktkas-hyperliquid --path "src/**" --language typescript --mode lexical
 ```
 
 Inspect a specific chunk:
@@ -80,22 +79,24 @@ Inspect a specific chunk:
 docs --json show 42
 ```
 
-Refresh the catalog:
+Search or inspect a compiled workspace:
 
 ```bash
-docs --json source list
-docs --json refresh due hyperliquid
-docs --json canary hyperliquid
-docs --json embeddings status
-docs --json embeddings backfill all
-docs --json embeddings run
+docs --json workspace status market-structure
+docs --json workspace search market-structure "transport design" --scope mixed
+docs --json workspace ingest list market-structure
+docs --json workspace ingest search market-structure "fee tier"
+docs --json workspace artifact list market-structure
+docs --json workspace artifact show market-structure derived/index.md
+docs --json workspace lint market-structure
 ```
 
-Force fetch is still available for explicit maintenance:
+Inspect source availability and health:
 
 ```bash
-docs --json fetch hyperliquid
-docs --json fetch all
+docs --json source list
+docs --json canary hyperliquid
+docs --json embeddings status
 ```
 
 Inspect what changed between snapshots:
@@ -131,11 +132,8 @@ The `aiocs-mcp` server exposes the same core operations without shell parsing:
 - `version`
 - `doctor`
 - `init`
-- `source_upsert`
 - `source_list`
 - `canary`
-- `fetch`
-- `refresh_due`
 - `snapshot_list`
 - `diff_snapshots`
 - `project_link`
@@ -148,17 +146,28 @@ The `aiocs-mcp` server exposes the same core operations without shell parsing:
 - `backup_import`
 - `search`
 - `show`
+- `workspace_list`
+- `workspace_status`
+- `workspace_search`
+- `workspace_ingest_list`
+- `workspace_ingest_show`
+- `workspace_ingest_search`
+- `workspace_artifact_list`
+- `workspace_artifact_show`
+- `workspace_lint`
 - `verify_coverage`
 - `batch`
 
+Mutation-capable MCP tools such as `source_upsert`, `refresh_due`, and `fetch` belong to `aiocs-curation`.
+
 ## Recommended Codex workflow
 
-1. If runtime health or freshness is in doubt, run `doctor`.
-2. Run `source_list` to see whether the source already exists and whether it is due.
-3. If the source exists and is due, prefer `refresh due <source-id>` over force fetch.
-4. If the source is missing but likely to be reused, add a spec under `~/.aiocs/sources`, upsert it, then refresh only that source.
-5. Use `search` in `auto` mode first, then `show` for the selected chunk.
-6. Use `canary`, `diff_snapshots`, or `verify_coverage` when the question is about drift, changes, or completeness.
+1. If runtime health is in doubt, run `doctor`.
+2. Run `source_list` to see whether the source already exists.
+3. Use `search` in `auto` mode first, then `show` for the selected chunk.
+4. If the user is asking against a compiled research wiki, prefer `workspace_search` and `workspace_artifact_show`.
+5. Use `canary`, `diff_snapshots`, or `verify_coverage` when the question is about drift, changes, or completeness.
+6. If the source is missing or stale and the next step is to mutate `aiocs`, load `aiocs-curation`.
 7. Use `batch` when combining list/search/show or diff/coverage checks in one pass.
 
 ## Operational notes
@@ -168,6 +177,7 @@ The `aiocs-mcp` server exposes the same core operations without shell parsing:
 - Use `docs daemon` or the Docker daemon service when the catalog should stay fresh automatically.
 - `docs search --mode auto` is the right default for agents; it uses hybrid retrieval only when embeddings are current and healthy for the requested scope.
 - The Docker Compose stack includes a dedicated `aiocs-qdrant` container and expects Ollama to be reachable separately.
+- Compiled research workspaces use LM Studio as the v1 compiler backend and expect `google/gemma-4-26b-a4b` to be loaded unless the environment overrides it.
 - Canaries are the first place to look when a docs site changed and fetches started degrading.
 - Newly added or changed sources become due immediately, so `refresh due <source-id>` is the safe first refresh path after upsert.
 - CLI failures expose machine-readable `error.code` fields in `--json` mode.

package/skills/aiocs-curation/SKILL.md

@@ -0,0 +1,145 @@
+---
+name: aiocs-curation
+description: Use when aiocs sources or snapshots need mutation, such as source onboarding, targeted refresh, canary remediation, or catalog repair.
+---
+
+# aiocs-curation
+
+Use this skill when you need to add, refresh, repair, or otherwise mutate `aiocs` sources under `~/.aiocs`.
+
+## When to use it
+
+- The requested docs source is missing from the local `aiocs` catalog and is worth curating for reuse.
+- An existing source is stale and should be refreshed instead of bypassed.
+- A source spec needs to be created, updated, or upserted under `~/.aiocs/sources`.
+- A reusable external git repository should be added as a `kind: git` source under `~/.aiocs/sources`.
+- A canary is failing and the source needs remediation or targeted refetch.
+- A research workspace needs to be created, rebound, compiled, or asked to generate a report/slide artifact.
+- A workspace needs raw evidence ingest, auto-compile setup, queued compile processing, or Obsidian sync.
+- The user explicitly wants `aiocs` maintenance, source onboarding, or catalog repair.
+
+## Trigger guidance for Codex
+
+- Load this skill when the next step requires a mutating `aiocs` operation.
+- Prefer targeted maintenance:
+  - `refresh due <source-id>` for existing sources
+  - `source_upsert` plus targeted refresh for newly curated sources
+- Avoid `fetch all` unless the user explicitly asks for broad maintenance.
+- If the source is missing, only curate it when it is likely to be reused across sessions or projects.
+- Keep the read/search path in `aiocs`; use this skill only for the curation step.
+
+## Preferred interfaces
+
+1. Prefer `aiocs-mcp` when an MCP client can use it directly.
+2. Otherwise use the CLI with the root `--json` flag.
+3. Assume `docs` and `aiocs-mcp` come from the globally installed `@bodhi-ventures/aiocs` package unless the user explicitly asks for a checkout-local development build.
+4. Use `npx -y -p @bodhi-ventures/aiocs ...` only as a fallback when the global install is unavailable.
+
+## User-managed sources
+
+Machine-local source specs live under:
+
+```bash
+~/.aiocs/sources
+```
+
+Create or update source specs there instead of editing the bundled repo sources.
+
+## Core commands
+
+Validate the machine before curation:
+
+```bash
+docs --json doctor
+docs --json source list
+```
+
+Add or update a machine-local source:
+
+```bash
+mkdir -p ~/.aiocs/sources
+docs --json source upsert ~/.aiocs/sources/my-source.yaml
+```
+
+Refresh only what is needed:
+
+```bash
+docs --json refresh due my-source
+docs --json refresh due hyperliquid
+docs --json refresh due nktkas-hyperliquid
+docs --json fetch my-source
+docs --json canary my-source
+```
+
+Compile or regenerate a workspace wiki:
+
+```bash
+docs --json workspace create market-structure --label "Market Structure"
+docs --json workspace bind market-structure hyperliquid nktkas-hyperliquid
+docs --json workspace compile market-structure
+docs --json workspace configure market-structure --auto-compile true
+docs --json workspace queue-run
+docs --json workspace ingest add market-structure markdown-dir /absolute/path/to/notes --label "Research notes"
+docs --json workspace ingest add market-structure csv /absolute/path/to/fills.csv --label "Fills CSV"
+docs --json workspace ingest add market-structure json /absolute/path/to/manifest.json --label "Research manifest"
+docs --json workspace ingest add market-structure jsonl /absolute/path/to/events.jsonl --label "Events JSONL"
+docs --json workspace output market-structure report --name weekly-brief
+docs --json workspace output market-structure slides --name weekly-brief
+docs --json workspace answer market-structure note "What changed?" --name quick-note
+docs --json workspace sync obsidian market-structure /absolute/path/to/vault
+```
+
+Heavy maintenance remains explicit:
+
+```bash
+docs --json fetch all
+docs --json embeddings backfill all
+docs --json embeddings run
+```
+
+## MCP tools
+
+The `aiocs-mcp` server exposes the same curation operations without shell parsing:
+
+- `doctor`
+- `source_list`
+- `source_upsert`
+- `canary`
+- `fetch`
+- `refresh_due`
+- `embeddings_status`
+- `embeddings_backfill`
+- `embeddings_clear`
+- `embeddings_run`
+- `workspace_create`
+- `workspace_bind`
+- `workspace_unbind`
+- `workspace_update`
+- `workspace_compile`
+- `workspace_queue_run`
+- `workspace_ingest_add`
+- `workspace_ingest_remove`
+- `workspace_output`
+- `workspace_answer`
+- `workspace_sync_obsidian`
+- `batch`
+
+## Recommended Codex workflow
+
+1. Run `doctor` or `source_list` if runtime health, presence, or freshness is unclear.
+2. If the source already exists and is due, prefer `refresh due <source-id>`.
+3. If the source is missing but worth curating, create a spec under `~/.aiocs/sources`, then `source_upsert` it.
+4. After upsert, use `refresh due <source-id>` as the safe first fetch path.
+5. Use `canary` when the site changed or extraction drift is suspected.
+6. For research workspaces, bind curated sources and/or ingest raw evidence, then run `workspace compile`.
+7. Use `workspace output` and `workspace answer` only after a successful compile so outputs have source-backed provenance.
+8. Prefer `autoCompileEnabled` plus targeted queue processing over broad global rebuilds.
+9. Escalate to `fetch <source-id>` or `fetch all` only for explicit maintenance or when due-based refresh is not enough.
+
+## Operational notes
+
+- New or changed sources become due immediately after `source_upsert`.
+- `~/.aiocs/sources` and bundled repo sources behave the same once bootstrapped into the catalog.
+- Targeted refresh is the default. Broad refresh is a maintenance task, not a normal answering step.
+- Use `aiocs` for read/search flows and this skill only for catalog mutation.
+- Workspace compilation uses LM Studio and expects the configured model to be loaded before `workspace compile`, `workspace output`, or `workspace answer`.

package/sources/nktkas-hyperliquid.yaml

@@ -0,0 +1,30 @@
+kind: git
+id: nktkas-hyperliquid
+label: nktkas/hyperliquid Repo
+repo:
+  url: https://github.com/nktkas/hyperliquid.git
+  ref: main
+  include:
+    - README.md
+    - CONTRIBUTING.md
+    - SECURITY.md
+    - LICENSE
+    - docs/**
+    - src/**
+    - tests/**
+  exclude:
+    - .github/**
+    - .dev/**
+  maxFiles: 1000
+  textFileMaxBytes: 262144
+schedule:
+  everyHours: 24
+canary:
+  everyHours: 6
+  checks:
+    - path: README.md
+      expectedText: Hyperliquid
+      minContentLength: 120
+    - path: src/mod.ts
+      expectedText: export
+      minContentLength: 20