orc-ai 0.1.34 → 0.1.36

package/README.md

@@ -28,14 +28,14 @@ ORC fixes this. Shared memory across every session. A task board where agents su
 | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
 | **Agent orchestration**   | Task loop spawns workers across backends (Claude Code, Codex, Gemini, Copilot, A2A), manages concurrency and review cycles         |
 | **Human-in-the-loop**     | Agents submit work for your approval via CLI, Telegram, or Slack before anything lands                                             |
-| **Shared memory**         | Decisions, rules, and discoveries stored once, searchable by any session via ranked full-text search                               |
+| **Shared memory**         | Short project facts (ports, conventions, gotchas, decisions), ranked by priority and loaded into every session's context           |
 | **Task board**            | `todo → queued → doing → review → done` with dependency tracking, priority, and automatic unblocking                               |
 | **Multi-backend routing** | Route to Claude (in-process, nothing to install), ACPX (14+ agents), agentapi, or remote A2A endpoints — with `orc doctor` reporting which are usable |
 | **Job runner**            | Cron, file-watch, webhook, or manual triggers with full run history                                                                |
-| **MCP server**            | 34 tools connect any [Model Context Protocol](https://modelcontextprotocol.io) (MCP) compatible agent — stdio or Streamable HTTP   |
+| **MCP server**            | 54 tools connect any [Model Context Protocol](https://modelcontextprotocol.io) (MCP) compatible agent — stdio or Streamable HTTP   |
 | **Session continuity**    | Snapshots survive context compaction so agents resume where they left off                                                          |
 | **Gateway**               | Approve work, search memory, and chat with live agents from Telegram or Slack                                                      |
-| **Knowledge search**      | Index document collections (markdown, notes, wikis) and search them via BM25 or hybrid (vector + reranking)                        |
+| **Knowledge bases**       | Upload PDFs, Office files and Markdown or copy in local folders; hybrid full-text + vector search with cited passages            |
 | **Skill library**         | Discoverable workflow templates (coder, reviewer, planner, bugfix) that encode your standards                                      |
 | **Task flows**            | Per-task graphs: conditional edges, bounded loops, parallel fan-out with joins, and human gates — with hard termination rails      |
 
@@ -112,7 +112,7 @@ orc project use my-app
 
 # 3. Add tasks and memories - everything auto-scopes to my-app
 orc task add "Fix the auth bug" --priority high
-orc mem add "Use RWMutex for token refresh" --type decision
+orc mem add token-refresh-lock -d "Check before touching token refresh" -c "Use RWMutex for token refresh" --priority 60
 orc job add nightly --command "bun run test" --trigger cron --cron "0 22 * * *"
 
 # 4. See everything
@@ -138,6 +138,8 @@ The web dashboard ships inside the `orc` binary and is served by the API process
 | `GET /mcp`                         | MCP server (streamable HTTP)                                                                                                   |
 | `GET /tasks/<id>`, `/knowledge`, … | `index.html` again - a navigation request matching no static file falls back to the SPA shell, so dashboard links are shareable |
 
+`/mcp?mode=meta` serves three meta-tools (`tool_search`, `tool_describe`, `tool_invoke`) instead of the full tool list, which keeps client context small; the model finds a tool, reads its schema, then calls it through `tool_invoke` with the same validation as a direct call. Stdio parity: `orc mcp --mode meta`. Any other `mode` value returns HTTP 400.
+
 Override the served dist directory with `ORC_WEB_DIST=/path/to/web/dist` if you want to host a custom build (e.g. a fork). If no dist is found, the server runs in pure-API mode.
 
 ### Desktop app
@@ -181,7 +183,9 @@ Or with [docker-compose.yml](docker-compose.yml):
 docker compose up -d
 ```
 
-The container defaults to the `claude` backend (direct Anthropic API via SDK — no host CLI needed). To delegate to an agent running on the host instead, set `ORC_AGENT_LOOP_DEFAULT_BACKEND=agentapi` and run [coder/agentapi](https://github.com/coder/agentapi) on the host:
+The source Docker image includes Node, ACPX, Claude Code and Codex CLI, with versions pinned in the Dockerfile. Its default backend is `claude` through the Anthropic SDK. Tool installation alone does not supply authentication: provide `ANTHROPIC_API_KEY` for Claude, `OPENAI_API_KEY` or a container-local Codex login for `codex-cli`, and the credentials required by whichever agent ACPX launches. Mount your repository and use its **container path** as the project folder. Agent configuration/session files can persist in the Compose `orc-home` volume. Existing ORC policies still apply; native Codex rule interception is not certified by CLI/tool availability. These source changes do not update an already published image.
+
+To delegate to an agent running on the host instead, set `ORC_AGENT_LOOP_DEFAULT_BACKEND=agentapi` and run [coder/agentapi](https://github.com/coder/agentapi) on the host:
 
 ```bash
 agentapi server --allowed-hosts '*' -- \
@@ -202,7 +206,7 @@ Add to `~/.claude/settings.json`:
   "hooks": {
     "PostToolUse": [
       {
-        "matcher": "Write|Edit|MultiEdit|StrReplace|EditNotebook|Bash|Shell|Agent|EnterPlanMode|ExitPlanMode|mcp__orc__task_|mcp__orc__memory_store|mcp__orc__memory_delete|mcp__orc__job_run|mcp__orc__job_create|mcp__orc__job_update",
+        "matcher": "Write|Edit|MultiEdit|StrReplace|EditNotebook|Bash|Shell|Agent|EnterPlanMode|ExitPlanMode|mcp__orc__task_|mcp__orc__memory_create|mcp__orc__memory_delete|mcp__orc__job_run|mcp__orc__job_create|mcp__orc__job_update",
         "hooks": [
           {
             "type": "command",
@@ -372,7 +376,7 @@ orc project add my-app -d "Main application"
 orc project use my-app       # set active
 
 orc task list                 # scoped to my-app
-orc mem search "auth"         # scoped to my-app
+orc mem list -q auth          # scoped to my-app
 orc task list -p infra        # override to different project
 orc task list --no-project    # see everything
 ```
@@ -383,45 +387,55 @@ MCP tools follow the same logic - pass `project: "name"` to scope, or omit to us
 
 ### Memory
 
-Store decisions, conventions, and discoveries that persist across all sessions:
+A memory is a short project fact worth knowing next session: ports, conventions, gotchas,
+decisions, pointers. Each has a lowercase-hyphenated `name` (unique per project), a
+`description` saying when it matters, `content` (up to 800 characters by default) and an
+optional `priority` from 0 to 100. `context` loads the top 30 memories by priority; agents
+then read the ones they need. Procedures and long guidance belong in skills.
 
 ```bash
-orc mem add "All IDs are ULIDs" --type rule
-orc mem add "Use PostgreSQL for concurrent writes" --type decision
-orc mem search "authentication"
+orc mem add ulid-ids -d "Check before adding a table or ID field" -c "All IDs are ULIDs" --priority 90
+orc mem list -q ids                    # search names and descriptions
+orc mem show ulid-ids
+orc mem edit ulid-ids --priority 80 -r "Narrower than we thought"
+orc mem history ulid-ids               # every change, with before/after
+orc mem revert ulid-ids <change-id> --expected-change-id <latest-change-id> -r "Undo mistaken edit"
 ```
 
-| Type        | Weight | Use for                                         |
-| ----------- | ------ | ----------------------------------------------- |
-| `rule`      | High   | Conventions: "all IDs are ULIDs"                |
-| `decision`  | High   | Choices: "use PostgreSQL for concurrent writes" |
-| `discovery` | Medium | Findings: "token refresh has a race condition"  |
-| `event`     | Low    | Things that happened: "deployed v1.0"           |
-| `fact`      | Low    | General knowledge (default)                     |
+### Knowledge bases
 
-### Knowledge
-
-Index existing document collections (markdown, notes, code docs) and make them searchable by any agent. Unlike memory (short agent-authored notes), knowledge searches pre-existing files on disk.
+Knowledge bases hold human reference documents for agents to search and cite. A knowledge base belongs to a project or is global. Add documents by uploading files (PDF, Word, PowerPoint, Excel, HTML, CSV, Markdown, text), writing Markdown in the dashboard, or registering a folder whose files are copied in. ORC never writes the folder: a copy follows its file on each scan until it is edited in ORC, and a file deleted from the folder keeps its copy. Documents have folders, labels and `[[links]]` with backlinks.
 
 ```bash
-# Add a document collection
-orc knowledge add my-docs --path ~/projects/docs --pattern "**/*.md"
+orc kb create Runbooks --project my-app
+orc kb add Runbooks ./docs/*.pdf          # upload and parse
+orc kb sources add Runbooks ./handbook    # copy a folder in, rescanned periodically
+orc kb editable-by Runbooks ui,cli,api    # agents (mcp) may read but not edit
+orc kb search "rollback procedure"        # cited passages
+orc kb evidence "how do we rotate keys"   # budgeted evidence pack
+orc kb usage Runbooks                     # reads by source (mcp, api, cli, ui)
+```
 
-# Search across indexed documents
-orc knowledge search "authentication flow"
+Comments on a knowledge base or one of its documents ask for a change: a correction, a source to digest, a lesson to record. Each has an author and source, a status (open, resolved, dismissed) and one closing reply.
 
-# List collections
-orc knowledge list
+Any knowledge base can have a digest: an agent (Claude, Codex, ...) that follows the knowledge base's own instructions on a schedule, handling open comments first and then reading memories, session summaries and audit records to add, correct and clean up documents. The schedule is an ordinary ORC job (cron or manual) with run history. In the dashboard choose **Add digest** on a knowledge base; it starts from a generic template. Scheduled runs are skipped when nothing new happened. Each active project gets a starter **Learnings** knowledge base with a daily 03:00 digest.
 
-# Re-index after files change
-orc knowledge update
+```bash
+orc kb comment add Learnings "Digest https://example.com/postmortem"
+orc kb digest template > digest.md        # edit, then:
+orc kb digest set Runbooks -b codex --cron "0 3 * * *" -i digest.md
+orc kb digest run Runbooks
 ```
 
-Knowledge uses [QMD](https://github.com/nicholasgriffintn/qmd) as the search engine. By default it runs BM25 full-text search (no LLM needed). Set `search_mode: "hybrid"` in config for vector search with reranking - embeddings are generated automatically when documents are indexed.
+Search combines title/path matches, SQLite full-text search and, when embeddings are configured (Ollama or an OpenAI-compatible API), vectors stored with sqlite-vec. Set the embedding URL, model and API key or token in Settings or with `orc kb settings`; the key is stored encrypted and never returned by the API. Results cite passages as `kb:<doc>@<rev>#<n>`. Every read is counted per document and source.
+
+The built-in parser handles all formats without extra installs. For better PDF layout, tables and OCR, install the optional [Docling](https://github.com/docling-project/docling) parser from Settings or with `orc kb docling install` (about 2 GB); in `auto` mode it is then used for PDFs and images. Each knowledge base can override the parser.
 
-Collections can be scoped to projects. When scoped, searches and listing filter to only that project's collections.
+Each knowledge base lists the call sources that may change its documents (`editable_by`: `ui`, `cli`, `api`, `mcp`; all four by default). Agents write with `kb_write` and `kb_document` (partial edits by exact text or line range) where `mcp` is allowed; every change is a revision recording its source and session. Knowledge base settings and knowledge settings cannot be changed over MCP, so an agent cannot widen its own access. Rendered Markdown shows remote images as links instead of loading them.
 
-**MCP tools:** `knowledge_search`, `knowledge_get`, `knowledge_collections`, `knowledge_collection_add`, `knowledge_collection_remove`, `knowledge_update`
+To back up knowledge bases, stop ORC and copy `~/.orc/orc.db` (with its `-wal` and `-shm` files) and `~/.orc/knowledge/originals/`, plus `~/.orc/secret.key`, without which a saved embedding API key cannot be decrypted and must be entered again. The search index, parse results and Markdown copies are rebuilt on the next start.
+
+**MCP tools:** `kb_list`, `kb_search`, `kb_evidence`, `kb_read`, `kb_documents`, `kb_write`, `kb_document`, `kb_comment`, `kb_manage`, `kb_digest`, `kb_maintenance`
 
 ### Task status flow
 
@@ -449,21 +463,18 @@ The graph itself is deterministic; only the nodes are agents. That means loops,
 
 ```bash
 orc flow list                                    # what graphs are available
-orc flow show orc-plan-build-verify              # nodes, edges, limits
-orc task add "Fix the flaky test" --flow orc-fix-verify
+orc flow show orc-default              # nodes, edges, limits
+orc task add "Fix the flaky test" --flow orc-default --skill orc-bugfix
 orc flow status <taskId>                         # active nodes + the ledger of outcomes
 orc flow resume <taskId> --outcome approved      # resolve a human gate
 orc flow halt <taskId>                           # stop it and kill live nodes
 ```
 
-| Built-in flow           | Shape                                                                        |
-| ----------------------- | ---------------------------------------------------------------------------- |
-| `orc-default`           | build → review → done, looping back while `max_review_rounds` allows         |
-| `orc-review-only`       | A single review pass for a task you moved straight to `review`               |
-| `orc-plan-build-verify` | planner → coder → reviewer, looping until acceptance criteria pass           |
-| `orc-fix-verify`        | bugfix → independent confirmation the fix holds and a regression test exists  |
-| `orc-supervisor`        | Executor keeping its context, plus a supervisor re-verifying from a clean one |
-| `orc-parallel-review`   | Fan out to correctness / security / tests reviewers, join on all three       |
+| Built-in workflow | Use it for |
+| --- | --- |
+| `orc-default` | Development: understand, plan as needed, implement, verify and independently review; bounded rework. `required_review: false` explicitly skips agent review. |
+| `orc-human-review` | The same development process plus human sign-off. Agent review and human approval are always required, even if `required_review` is false. |
+| `orc-review-only` | Review existing work; approve, request changes, or pause when evidence is insufficient. |
 
 **Nodes** are `agent` (runs a session with a skill), `gate` (deterministic branching, fan-out, join points), `human` (waits for you), or `terminal` (ends the run and sets the task's final status). **Edges** carry declarative conditions — `outcome`, `visits`, `var`, `elapsed_secs`, composed with `all`/`any`/`not` — which are data, never evaluated code.
 
@@ -561,13 +572,29 @@ npx skills add niradler/orc --all -g
 | --------------- | -------------------------------------------------------------------------- |
 | `orc-session`   | Session start, context compaction, resuming work                           |
 | `orc-tasks`     | Task creation, status updates, HITL review, task decomposition             |
-| `orc-knowledge` | Storing decisions, searching memory, "remember this", "what did we decide" |
+| `orc-memory`    | Short project facts, "remember this", "what did we decide"                 |
+| `orc-knowledge` | Searching indexed docs, managing knowledge collections                     |
 | `orc-gateway`   | Telegram/Slack setup, remote approval, live agent sessions                 |
 
 ### Built-in skill templates
 
 Skill templates live in `skills/*/SKILL.md` (built-in) and `~/.orc/skills/` (user-defined). Agents discover them via `skill_list` and load with `skill_read`.
 
+On the **Skills** page, each built-in has an **Agent discovery** toggle, plus
+**Hide all built-ins** and **Show all built-ins** controls. Visibility applies
+across the installation and defaults to visible. Hidden built-ins stay on the
+Skills page for management and remain readable by name, including supporting
+files; existing task flows continue to use them. User/imported skills and authored
+global/project instruction overrides remain discoverable.
+
+The policy is saved under `skills.hidden_builtins` in `~/.orc/config.json` and
+read afresh for discovery, including by existing MCP processes. `GET /api/skills`
+omits hidden built-ins; `include_hidden=true` returns the management catalog with
+`discovery_hidden` metadata. `GET /api/skills/settings/visibility` reads the policy and
+`PUT /api/skills/settings/visibility` replaces its `hidden_builtins` array using the usual
+API authentication. This installation-wide preference ignores project config.
+New built-ins default to visible until hidden explicitly.
+
 | Skill              | Type     | Purpose                                                                     |
 | ------------------ | -------- | --------------------------------------------------------------------------- |
 | `orc-worker-base`  | Base     | Default worker behavior - ORC awareness, status updates, deliverable format |
@@ -648,12 +675,12 @@ Tools are available to any connected agent. Start every session with `context`.
 | Category      | Tools                                                                                                                                       |
 | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
 | **Project**   | `project_list`                                                                                                                              |
-| **Memory**    | `context`, `memory_search`, `memory_get`, `memory_store`, `memory_update`                                                                   |
+| **Memory**    | `context`, `memory_list`, `memory_read`, `memory_create`, `memory_update`, `memory_delete`, `memory_history`, `memory_revert`                |
 | **Task**      | `task_list`, `task_get`, `task_create`, `task_update`, `task_batch_create`                                                                  |
 | **Skill**     | `skill_list`, `skill_read`, `skill_create`                                                                                                  |
 | **Agent**     | `agent_list`, `agent_read`, `agent_create`, `agent_package_list`, `agent_package_read`, `agent_package_import`                               |
 | **Flow**      | `flow_report`, `flow_status`, `flow_list`, `flow_read`, `flow_create`, `flow_attach`                                                        |
-| **Knowledge** | `knowledge_search`, `knowledge_get`, `knowledge_collections`, `knowledge_collection_add`, `knowledge_collection_remove`, `knowledge_update` |
+| **Knowledge** | `kb_list`, `kb_search`, `kb_evidence`, `kb_read`, `kb_documents`, `kb_write`, `kb_maintenance`                                               |
 | **Search**    | `search`                                                                                                                                    |
 | **Job**       | `job_list`, `job_run`, `job_status`                                                                                                         |
 | **Session**   | `session_event`, `session_snapshot`, `session_restore`, `session_log`                                                                       |
@@ -677,8 +704,10 @@ Runs on port 7700 with auto-generated OpenAPI spec.
 | `GET/POST/PATCH/DELETE` | `/tasks`                       | CRUD tasks                     |
 | `GET/POST`              | `/tasks/{id}/notes`            | Task notes                     |
 | `GET/POST/DELETE`       | `/tasks/{id}/links`            | Task dependencies              |
-| `GET/POST/DELETE`       | `/memories`                    | CRUD memories                  |
-| `GET`                   | `/memories/search`             | BM25 search                    |
+| `GET/POST`              | `/memories`                    | List (`q`, `sort`) / create    |
+| `GET/PATCH/DELETE`      | `/memories/{name}`             | Read / update / delete by name |
+| `GET`                   | `/memories/{name}/history`     | Change history                 |
+| `POST`                  | `/memories/{name}/revert`      | Revert a change                |
 | `GET`                   | `/knowledge/search`            | Search documents (BM25/hybrid) |
 | `GET`                   | `/knowledge/documents/{id}`    | Get document by docid          |
 | `GET/POST/DELETE`       | `/knowledge/collections`       | CRUD collections               |
@@ -714,7 +743,7 @@ orc doctor                       Check which agent backends are usable here
 
 orc project list|add|show|use|update|archive
 orc task list|add|show|update|done|review|approve|reject|delete
-orc mem list|add|search
+orc mem list|show|add|edit|delete|history|revert
 orc job list|add|run|runs
 orc session list|show|log
 orc skill list|show
@@ -757,8 +786,13 @@ ORC merges config in priority order (later wins):
 | `ORC_AGENT_LOOP_DEFAULT_BACKEND` | `claude`              | Default agent backend                             |
 | `ORC_AGENT_LOOP_IDLE_TIMEOUT`    | `20`                  | Session idle timeout (minutes)                    |
 | `ORC_AGENT_LOOP_AUTO_APPROVE`    | `true`                | Auto-approve worker tool permissions              |
-| `ORC_KNOWLEDGE_DB_PATH`          | `~/.orc/knowledge.db` | Knowledge search database path                    |
-| `ORC_KNOWLEDGE_SEARCH_MODE`      | `lexical`             | `lexical` (BM25) or `hybrid` (vector + reranking) |
+| `ORC_KNOWLEDGE_ROOT`             | beside `orc.db`       | Knowledge base storage (files, index)             |
+| `ORC_KNOWLEDGE_PARSER`           | `auto`                | `auto`, `basic` or `docling`                      |
+| `ORC_KNOWLEDGE_EMBEDDING_PROVIDER` | `none`              | `none`, `ollama` or `openai`                      |
+| `ORC_KNOWLEDGE_EMBEDDING_URL`    | provider default      | Embedding endpoint                                |
+| `ORC_KNOWLEDGE_EMBEDDING_MODEL`  | provider default      | Embedding model                                   |
+| `ORC_KNOWLEDGE_EMBEDDING_API_KEY` | -                    | API key for OpenAI-compatible embeddings          |
+| `ORC_SQLITE_VEC_PATH`            | bundled               | Path to a sqlite-vec `vec0` library               |
 
 </details>
 
@@ -838,7 +872,7 @@ See [AGENTS.md](./AGENTS.md) for the full development guide and coding conventio
 
 ## Learn more
 
-- [Evolving Wiki](./docs/evolving-wiki.md) - cited retrieval, automatic session consolidation, evaluated skill promotion, and human revert
+- [Knowledge learning loop](./docs/knowledge-loop.md) - memories, knowledge base digests and comments, skill edits, and human revert
 - [Agent Rules](./docs/agent-rules.md) - opt-in tool protection, revision history, deterministic job actions, and evidence-based skill evaluation
 - [Usage Guide](./docs/usage-guide.md) - best practices for memory, tasks, multi-agent workflows, and configuration
 - [Vision](./docs/vision.md) - why ORC exists and the problem it solves
@@ -848,4 +882,4 @@ See [AGENTS.md](./AGENTS.md) for the full development guide and coding conventio
 
 ### Knowledge learning
 
-Session review and a daily digest compile verified notes into wiki lessons and edited skills. Reuse them through ordinary wiki/KB search and skill reads. The dashboard shows learning history, audit records and manual skill revert. See [Knowledge learning loop](docs/knowledge-loop.md) for scopes, scheduling and validation limits.
+Session review records verified lessons, and each project's Learnings knowledge base has a daily digest that turns memories, session summaries and comments into documents. Reuse them through ordinary KB search and skill reads. The dashboard shows learning history, audit records and manual skill revert. See [Knowledge learning loop](docs/knowledge-loop.md) for scopes, scheduling and validation limits.