@bodhi-ventures/aiocs 0.2.0 → 0.3.0

package/docs/codex-integration.md

@@ -46,6 +46,7 @@ command = "aiocs-mcp"
 9. Prefer `refresh due <source-id>` over force `fetch <source-id>` when the source already exists.
 10. Use MCP `batch` when multiple list/search/show or search/diff/coverage steps are needed.
 11. Cite `sourceId`, `snapshotId`, and `pageUrl` when they materially improve traceability.
+12. If the user is operating on a compiled research workspace, prefer `workspace_search` and `workspace_artifact_show` over raw source search.
 
 ## Automatic use in Codex
 
@@ -113,6 +114,27 @@ docs --json diff hyperliquid
 docs --json verify coverage hyperliquid /absolute/path/to/reference.md
 ```
 
+Research workspaces:
+
+```bash
+docs --json workspace create market-structure --label "Market Structure" --auto-compile
+docs --json workspace bind market-structure hyperliquid nktkas-hyperliquid
+docs --json workspace compile market-structure
+docs --json workspace queue-run
+docs --json workspace search market-structure "transport design" --scope mixed
+docs --json workspace artifact show market-structure derived/index.md
+docs --json workspace lint market-structure
+docs --json workspace output market-structure report --name weekly-brief
+docs --json workspace answer market-structure note "What changed in websocket transport?" --name websocket-note
+docs --json workspace ingest add market-structure markdown-dir /absolute/path/to/notes --label "Research notes"
+docs --json workspace ingest add market-structure csv /absolute/path/to/fills.csv --label "Fills CSV"
+docs --json workspace ingest add market-structure json /absolute/path/to/manifest.json --label "Research manifest"
+docs --json workspace ingest add market-structure jsonl /absolute/path/to/events.jsonl --label "Events JSONL"
+docs --json workspace sync obsidian market-structure /absolute/path/to/vault
+```
+
+If bound source snapshots have changed since the last compile, rerun `workspace compile` before `workspace output`. The output path now fails closed on stale derived artifacts instead of synthesizing reports from outdated summaries.
+
 Catalog maintenance:
 
 ```bash
@@ -134,8 +156,26 @@ If a Codex agent has access to the `aiocs-mcp` server, prefer these MCP tools ov
 - `diff_snapshots`
 - `verify_coverage`
 - `embeddings_status`
+- `workspace_list`
+- `workspace_status`
+- `workspace_search`
+- `workspace_ingest_list`
+- `workspace_ingest_search`
+- `workspace_artifact_list`
+- `workspace_artifact_show`
+- `workspace_lint`
 - `batch`
 
-Use mutating tools such as `source_upsert`, `refresh_due`, and `fetch` only through the `aiocs-curation` workflow.
+Use mutating tools such as `source_upsert`, `refresh_due`, `fetch`, `workspace_create`, `workspace_bind`, `workspace_compile`, and `workspace_output` only through the `aiocs-curation` workflow.
+
+## LM Studio requirement for workspaces
+
+Workspace compilation and output generation use LM Studio in v1. Codex should assume:
+
+- provider: LM Studio
+- model: `google/gemma-4-26b-a4b`
+- default API endpoint: `ws://127.0.0.1:1234`
+
+If `doctor` reports the `lmstudio` check as `warn` or `fail`, Codex should not claim workspace compilation is available until LM Studio is running and the configured model is loaded.
 
 The CLI remains the fallback and should always be invoked with `--json` for agent use. For normal answering flows, avoid `fetch all`; use targeted due refresh or explicit user-approved force fetches.

package/docs/json-contract.md

@@ -54,6 +54,26 @@ All of these support the root-level `--json` flag:
 - `diff`
 - `project link`
 - `project unlink`
+- `workspace create`
+- `workspace configure`
+- `workspace list`
+- `workspace bind`
+- `workspace unbind`
+- `workspace compile`
+- `workspace queue-run`
+- `workspace status`
+- `workspace search`
+- `workspace ingest add`
+- `workspace ingest list`
+- `workspace ingest show`
+- `workspace ingest search`
+- `workspace ingest remove`
+- `workspace artifact list`
+- `workspace artifact show`
+- `workspace lint`
+- `workspace output`
+- `workspace answer`
+- `workspace sync obsidian`
 - `backup export`
 - `backup import`
 - `embeddings status`
@@ -131,6 +151,7 @@ Check ids are currently:
 - `source-spec-dirs`
 - `freshness`
 - `daemon-heartbeat`
+- `lmstudio`
 - `embedding-provider`
 - `vector-store`
 - `embeddings`
@@ -245,6 +266,405 @@ Summary status values:
 }
 ```
 
+### Workspace commands
+
+Workspace commands manage derived wiki artifacts backed by canonical source snapshots and LM Studio compilation.
+
+#### `workspace.create`
+
+```json
+{
+  "workspace": {
+    "id": "market-structure",
+    "label": "Market Structure",
+    "compilerProfile": {
+      "provider": "lmstudio",
+      "model": "google/gemma-4-26b-a4b",
+      "temperature": 0.1,
+      "topP": 0.9,
+      "maxInputChars": 12000,
+      "maxOutputTokens": 4096,
+      "concurrency": 1
+    },
+    "defaultOutputFormats": ["report", "slides"]
+  }
+}
+```
+
+#### `workspace.list`
+
+```json
+{
+  "workspaces": [
+    {
+      "id": "market-structure",
+      "label": "Market Structure",
+      "bindingCount": 2,
+      "artifactCount": 5,
+      "lastCompileStatus": "success"
+    }
+  ]
+}
+```
+
+#### `workspace.bind` and `workspace.unbind`
+
+```json
+{
+  "workspaceId": "market-structure",
+  "sourceIds": ["hyperliquid", "nktkas-hyperliquid"]
+}
+```
+
+#### `workspace.configure`
+
+```json
+{
+  "workspace": {
+    "id": "market-structure",
+    "label": "Market Structure",
+    "autoCompileEnabled": true
+  }
+}
+```
+
+#### `workspace.compile`
+
+```json
+{
+  "workspaceId": "market-structure",
+  "skipped": false,
+  "sourceFingerprint": "sha256...",
+  "changedSourceIds": ["hyperliquid"],
+  "changedRawInputIds": [],
+  "updatedArtifactPaths": [
+    "derived/sources/hyperliquid/summary.md",
+    "derived/concepts/hyperliquid.md",
+    "derived/index.md"
+  ],
+  "artifactCount": 3,
+  "compileRunId": "wrkcmp_..."
+}
+```
+
+#### `workspace.queue-run`
+
+```json
+{
+  "processedJobs": 1,
+  "succeededJobs": [
+    {
+      "workspaceId": "market-structure",
+      "sourceFingerprint": "sha256...",
+      "changedSourceIds": ["hyperliquid"],
+      "changedRawInputIds": []
+    }
+  ],
+  "failedJobs": []
+}
+```
+
+#### `workspace.status`
+
+```json
+{
+  "workspace": {
+    "id": "market-structure",
+    "label": "Market Structure",
+    "autoCompileEnabled": true
+  },
+  "bindings": [
+    {
+      "workspaceId": "market-structure",
+      "sourceId": "hyperliquid",
+      "createdAt": "2026-04-03T10:00:00.000Z"
+    }
+  ],
+  "artifacts": [
+    {
+      "workspaceId": "market-structure",
+      "path": "derived/index.md",
+      "kind": "index",
+      "stale": false,
+      "chunkCount": 3
+    }
+  ],
+  "compileJob": {
+    "workspaceId": "market-structure",
+    "status": "pending",
+    "requestedSourceIds": ["hyperliquid"],
+    "requestedRawInputIds": [],
+    "requestedFingerprint": null
+  },
+  "rawInputs": [],
+  "syncTargets": [],
+  "questionRuns": [],
+  "links": [],
+  "graph": {
+    "linkCount": 4,
+    "brokenLinkCount": 0,
+    "orphanArtifactCount": 0,
+    "backlinkCount": 4,
+    "relationCounts": {
+      "explicit_link": 0,
+      "derived_from": 1,
+      "mentions": 1,
+      "related_to": 2,
+      "expands": 1,
+      "index_entry": 2,
+      "summary_of": 1,
+      "concept_of": 1,
+      "output_depends_on": 0
+    },
+    "mostLinkedArtifacts": [
+      {
+        "artifactPath": "derived/index.md",
+        "incomingCount": 0,
+        "outgoingCount": 2
+      }
+    ]
+  },
+  "lintSummary": {
+    "status": "pass",
+    "findingCount": 0,
+    "staleArtifactCount": 0,
+    "missingProvenanceCount": 0,
+    "missingArtifactCount": 0,
+    "brokenLinkCount": 0,
+    "orphanArtifactCount": 0,
+    "suggestedConceptCount": 0,
+    "duplicateConceptCandidateCount": 0,
+    "missingArticleCandidateCount": 0,
+    "followUpQuestionCount": 0
+  },
+  "health": {
+    "status": "healthy",
+    "staleArtifactCount": 0,
+    "pendingCompileJobs": 0,
+    "failedCompileJobs": 0,
+    "brokenLinkCount": 0,
+    "orphanArtifactCount": 0,
+    "rawInputCount": 0,
+    "lintFindingCount": 0,
+    "duplicateConceptCandidateCount": 0,
+    "missingArticleCandidateCount": 0,
+    "followUpQuestionCount": 0
+  },
+  "compileRuns": [
+    {
+      "id": "wrkcmp_...",
+      "status": "success"
+    }
+  ]
+}
+```
+
+#### `workspace.ingest.*`
+
+```json
+{
+  "workspaceId": "market-structure",
+  "rawInput": {
+    "id": "csv-fills-abc123def0",
+    "workspaceId": "market-structure",
+    "kind": "csv",
+    "label": "Fills CSV",
+    "sourcePath": "/absolute/path/to/fills.csv",
+    "storagePath": "raw/csv-fills-abc123def0/fills.csv",
+    "extractedTextPath": "raw/csv-fills-abc123def0/fills.csv.txt",
+    "contentHash": "sha256...",
+    "chunkCount": 12
+  }
+}
+```
+
+Raw-input search returns:
+
+```json
+{
+  "workspaceId": "market-structure",
+  "query": "fee tier",
+  "total": 1,
+  "limit": 10,
+  "offset": 0,
+  "hasMore": false,
+  "results": [
+    {
+      "rawInputId": "csv-fills-abc123def0",
+      "kind": "csv",
+      "label": "Fills CSV",
+      "sectionTitle": "Fills CSV rows 1-2",
+      "markdown": "...",
+      "filePath": "fills.csv",
+      "score": 0.42
+    }
+  ]
+}
+```
+
+#### `workspace.search`
+
+```json
+{
+  "workspaceId": "market-structure",
+  "query": "transport design",
+  "scope": "mixed",
+  "limit": 10,
+  "offset": 0,
+  "hasMore": false,
+  "modeRequested": "auto",
+  "modeUsed": "hybrid",
+  "total": 2,
+  "results": [
+    {
+      "kind": "source",
+      "scope": "source",
+      "chunkId": 42,
+      "sourceId": "nktkas-hyperliquid",
+      "snapshotId": "snp_...",
+      "pageUrl": "file://src/transports/websocket.ts",
+      "pageTitle": "src/transports/websocket.ts",
+      "sectionTitle": "WebSocketTransport",
+      "markdown": "...",
+      "pageKind": "file",
+      "filePath": "src/transports/websocket.ts",
+      "language": "typescript",
+      "score": 0.91,
+      "signals": ["lexical", "vector"]
+    },
+    {
+      "kind": "derived",
+      "scope": "derived",
+      "artifactPath": "derived/concepts/nktkas-hyperliquid.md",
+      "artifactKind": "concept",
+      "sectionTitle": "Transport design",
+      "markdown": "...",
+      "stale": false,
+      "score": 0.74
+    }
+  ]
+}
+```
+
+#### `workspace.artifact.list`
+
+```json
+{
+  "workspaceId": "market-structure",
+  "artifacts": [
+    {
+      "workspaceId": "market-structure",
+      "path": "derived/index.md",
+      "kind": "index",
+      "stale": false,
+      "chunkCount": 3
+    }
+  ]
+}
+```
+
+#### `workspace.artifact.show`
+
+```json
+{
+  "workspaceId": "market-structure",
+  "artifact": {
+    "workspaceId": "market-structure",
+    "path": "derived/index.md",
+    "kind": "index",
+    "stale": false
+  },
+  "content": "# Workspace Index\n...",
+  "provenance": [
+    {
+      "workspaceId": "market-structure",
+      "path": "derived/index.md",
+      "sourceId": "hyperliquid",
+      "snapshotId": "snp_...",
+      "chunkIds": [42, 43]
+    }
+  ],
+  "rawInputProvenance": []
+}
+```
+
+#### `workspace.lint`
+
+```json
+{
+  "workspaceId": "market-structure",
+  "summary": {
+    "status": "warn",
+    "findingCount": 1,
+    "staleArtifactCount": 1,
+    "missingProvenanceCount": 0,
+    "missingArtifactCount": 0,
+    "brokenLinkCount": 0,
+    "orphanArtifactCount": 0,
+    "suggestedConceptCount": 0,
+    "duplicateConceptCandidateCount": 1,
+    "missingArticleCandidateCount": 1,
+    "followUpQuestionCount": 2
+  },
+  "findings": [
+    {
+      "kind": "stale-artifact",
+      "severity": "warn",
+      "summary": "Artifact provenance points at an older snapshot.",
+      "artifactPath": "derived/sources/hyperliquid/summary.md"
+    },
+    {
+      "kind": "follow-up-question-suggestion",
+      "severity": "warn",
+      "summary": "What important workflows, caveats, or open questions remain unresolved for derived/concepts/hyperliquid.md?",
+      "artifactPath": "derived/concepts/hyperliquid.md"
+    }
+  ],
+  "suggestionsArtifactPath": "outputs/suggestions/lint.md"
+}
+```
+
+#### `workspace.output`
+
+```json
+{
+  "workspaceId": "market-structure",
+  "format": "report",
+  "path": "outputs/reports/weekly-brief.md",
+  "artifactCount": 8
+}
+```
+
+#### `workspace.answer`
+
+```json
+{
+  "workspaceId": "market-structure",
+  "format": "note",
+  "path": "derived/notes/websocket-note.md",
+  "artifactCount": 9,
+  "questionRun": {
+    "id": "wrkq_...",
+    "workspaceId": "market-structure",
+    "question": "What changed in websocket transport?",
+    "format": "note",
+    "artifactPath": "derived/notes/websocket-note.md",
+    "status": "success"
+  }
+}
+```
+
+#### `workspace.sync.obsidian`
+
+```json
+{
+  "workspaceId": "market-structure",
+  "vaultPath": "/absolute/path/to/vault",
+  "targetPath": "/absolute/path/to/vault/aiocs/market-structure",
+  "exportSubdir": "aiocs/market-structure"
+}
+```
+
 ### `diff`
 
 ```json
@@ -494,6 +914,11 @@ Current stable CLI error codes include:
 
 - `INVALID_ARGUMENT`
 - `SOURCE_NOT_FOUND`
+- `WORKSPACE_NOT_FOUND`
+- `WORKSPACE_ARTIFACT_NOT_FOUND`
+- `WORKSPACE_ARTIFACTS_STALE`
+- `WORKSPACE_COMPILER_CONFIG_INVALID`
+- `WORKSPACE_COMPILER_UNAVAILABLE`
 - `SNAPSHOT_NOT_FOUND`
 - `NO_PAGES_FETCHED`
 - `NO_PROJECT_SCOPE`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bodhi-ventures/aiocs",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "license": "MIT",
   "type": "module",
   "description": "Local-only documentation store, fetcher, and search CLI for AI agents.",
@@ -48,12 +48,15 @@
     "test:watch": "vitest"
   },
   "dependencies": {
+    "@lmstudio/sdk": "1.5.0",
     "@modelcontextprotocol/sdk": "1.28.0",
     "@mozilla/readability": "0.6.0",
     "@qdrant/js-client-rest": "1.17.0",
     "better-sqlite3": "12.4.1",
     "commander": "14.0.1",
+    "csv-parse": "6.2.1",
     "jsdom": "27.0.1",
+    "pdf-parse": "2.4.5",
     "playwright": "1.57.0",
     "turndown": "7.2.1",
     "turndown-plugin-gfm": "1.0.2",

package/skills/aiocs/SKILL.md

@@ -12,6 +12,8 @@ Use this skill when you need authoritative local documentation lookup through th
 - The user is asking about exchange or product docs that may already exist in the local `aiocs` catalog.
 - You need authoritative local docs for an exchange, SDK, or product without browsing the live site every time.
 - You need reusable reference search over a curated external git repository that already lives in `aiocs`.
+- You need to read from a compiled `aiocs` workspace wiki or inspect derived workspace artifacts.
+- You need to inspect workspace status, queued compile health, or raw ingested evidence before deciding whether curation is needed.
 - You want machine-readable search/show/diff/coverage results for an AI agent.
 - You need hybrid docs retrieval with lexical plus semantic/vector recall.
 - You need to validate runtime health before relying on the local docs catalog.
@@ -77,6 +79,18 @@ Inspect a specific chunk:
 docs --json show 42
 ```
 
+Search or inspect a compiled workspace:
+
+```bash
+docs --json workspace status market-structure
+docs --json workspace search market-structure "transport design" --scope mixed
+docs --json workspace ingest list market-structure
+docs --json workspace ingest search market-structure "fee tier"
+docs --json workspace artifact list market-structure
+docs --json workspace artifact show market-structure derived/index.md
+docs --json workspace lint market-structure
+```
+
 Inspect source availability and health:
 
 ```bash
@@ -132,6 +146,15 @@ The `aiocs-mcp` server exposes the same core operations without shell parsing:
 - `backup_import`
 - `search`
 - `show`
+- `workspace_list`
+- `workspace_status`
+- `workspace_search`
+- `workspace_ingest_list`
+- `workspace_ingest_show`
+- `workspace_ingest_search`
+- `workspace_artifact_list`
+- `workspace_artifact_show`
+- `workspace_lint`
 - `verify_coverage`
 - `batch`
 
@@ -142,9 +165,10 @@ Mutation-capable MCP tools such as `source_upsert`, `refresh_due`, and `fetch` b
 1. If runtime health is in doubt, run `doctor`.
 2. Run `source_list` to see whether the source already exists.
 3. Use `search` in `auto` mode first, then `show` for the selected chunk.
-4. Use `canary`, `diff_snapshots`, or `verify_coverage` when the question is about drift, changes, or completeness.
-5. If the source is missing or stale and the next step is to mutate `aiocs`, load `aiocs-curation`.
-6. Use `batch` when combining list/search/show or diff/coverage checks in one pass.
+4. If the user is asking against a compiled research wiki, prefer `workspace_search` and `workspace_artifact_show`.
+5. Use `canary`, `diff_snapshots`, or `verify_coverage` when the question is about drift, changes, or completeness.
+6. If the source is missing or stale and the next step is to mutate `aiocs`, load `aiocs-curation`.
+7. Use `batch` when combining list/search/show or diff/coverage checks in one pass.
 
 ## Operational notes
 
@@ -153,6 +177,7 @@ Mutation-capable MCP tools such as `source_upsert`, `refresh_due`, and `fetch` b
 - Use `docs daemon` or the Docker daemon service when the catalog should stay fresh automatically.
 - `docs search --mode auto` is the right default for agents; it uses hybrid retrieval only when embeddings are current and healthy for the requested scope.
 - The Docker Compose stack includes a dedicated `aiocs-qdrant` container and expects Ollama to be reachable separately.
+- Compiled research workspaces use LM Studio as the v1 compiler backend and expect `google/gemma-4-26b-a4b` to be loaded unless the environment overrides it.
 - Canaries are the first place to look when a docs site changed and fetches started degrading.
 - Newly added or changed sources become due immediately, so `refresh due <source-id>` is the safe first refresh path after upsert.
 - CLI failures expose machine-readable `error.code` fields in `--json` mode.

package/skills/aiocs-curation/SKILL.md

@@ -14,6 +14,8 @@ Use this skill when you need to add, refresh, repair, or otherwise mutate `aiocs
 - A source spec needs to be created, updated, or upserted under `~/.aiocs/sources`.
 - A reusable external git repository should be added as a `kind: git` source under `~/.aiocs/sources`.
 - A canary is failing and the source needs remediation or targeted refetch.
+- A research workspace needs to be created, rebound, compiled, or asked to generate a report/slide artifact.
+- A workspace needs raw evidence ingest, auto-compile setup, queued compile processing, or Obsidian sync.
 - The user explicitly wants `aiocs` maintenance, source onboarding, or catalog repair.
 
 ## Trigger guidance for Codex
@@ -69,6 +71,24 @@ docs --json fetch my-source
 docs --json canary my-source
 ```
 
+Compile or regenerate a workspace wiki:
+
+```bash
+docs --json workspace create market-structure --label "Market Structure"
+docs --json workspace bind market-structure hyperliquid nktkas-hyperliquid
+docs --json workspace compile market-structure
+docs --json workspace configure market-structure --auto-compile true
+docs --json workspace queue-run
+docs --json workspace ingest add market-structure markdown-dir /absolute/path/to/notes --label "Research notes"
+docs --json workspace ingest add market-structure csv /absolute/path/to/fills.csv --label "Fills CSV"
+docs --json workspace ingest add market-structure json /absolute/path/to/manifest.json --label "Research manifest"
+docs --json workspace ingest add market-structure jsonl /absolute/path/to/events.jsonl --label "Events JSONL"
+docs --json workspace output market-structure report --name weekly-brief
+docs --json workspace output market-structure slides --name weekly-brief
+docs --json workspace answer market-structure note "What changed?" --name quick-note
+docs --json workspace sync obsidian market-structure /absolute/path/to/vault
+```
+
 Heavy maintenance remains explicit:
 
 ```bash
@@ -91,6 +111,17 @@ The `aiocs-mcp` server exposes the same curation operations without shell parsin
 - `embeddings_backfill`
 - `embeddings_clear`
 - `embeddings_run`
+- `workspace_create`
+- `workspace_bind`
+- `workspace_unbind`
+- `workspace_update`
+- `workspace_compile`
+- `workspace_queue_run`
+- `workspace_ingest_add`
+- `workspace_ingest_remove`
+- `workspace_output`
+- `workspace_answer`
+- `workspace_sync_obsidian`
 - `batch`
 
 ## Recommended Codex workflow
@@ -100,7 +131,10 @@ The `aiocs-mcp` server exposes the same curation operations without shell parsin
 3. If the source is missing but worth curating, create a spec under `~/.aiocs/sources`, then `source_upsert` it.
 4. After upsert, use `refresh due <source-id>` as the safe first fetch path.
 5. Use `canary` when the site changed or extraction drift is suspected.
-6. Escalate to `fetch <source-id>` or `fetch all` only for explicit maintenance or when due-based refresh is not enough.
+6. For research workspaces, bind curated sources and/or ingest raw evidence, then run `workspace compile`.
+7. Use `workspace output` and `workspace answer` only after a successful compile so outputs have source-backed provenance.
+8. Prefer `autoCompileEnabled` plus targeted queue processing over broad global rebuilds.
+9. Escalate to `fetch <source-id>` or `fetch all` only for explicit maintenance or when due-based refresh is not enough.
 
 ## Operational notes
 
@@ -108,3 +142,4 @@ The `aiocs-mcp` server exposes the same curation operations without shell parsin
 - `~/.aiocs/sources` and bundled repo sources behave the same once bootstrapped into the catalog.
 - Targeted refresh is the default. Broad refresh is a maintenance task, not a normal answering step.
 - Use `aiocs` for read/search flows and this skill only for catalog mutation.
+- Workspace compilation uses LM Studio and expects the configured model to be loaded before `workspace compile`, `workspace output`, or `workspace answer`.