agentic-cli 0.5.0__tar.gz → 0.5.2__tar.gz

{agentic_cli-0.5.0 → agentic_cli-0.5.2}/CHANGELOG.md

@@ -5,6 +5,44 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.2] - 2026-05-01
+
+### Security
+- **`kb_ingest` split into `kb_ingest_text` / `kb_ingest_file` / `kb_ingest_url`**: the previous unified tool declared only `kb.write` (auto-allowed by the builtin `kb.*` rule) while internally reading arbitrary local paths and fetching arbitrary URLs. The split lets the permission engine gate `filesystem.read` and `http.read` at the right entry point, and the URL path now flows through the hardened `ContentFetcher` (URLValidator + manual redirect revalidation + DNS-rebinding mitigation).
+- **Concept-page slug traversal blocked** (`ConceptStore`): explicit slugs are validated against a strict `[a-z0-9-]+` allowlist, and `_concept_path` asserts the resolved path stays under `base_dir`. A slug like `../../etc/passwd` is now rejected before any filesystem write.
+- **Webfetch redirect handling rewritten**: `ContentFetcher.fetch` no longer follows redirects via httpx. Each `Location` is revalidated through `URLValidator` *before* the next GET, cross-host redirects are reported without contacting the new origin, same-host redirects re-check `robots.txt`, and the redirect chain is capped at 5.
+- **`os_sandbox_enabled=True` now fails closed**: when shell or Python execution is requested with sandboxing on, both executors refuse to run if the resolved sandbox is `NoOpSandbox` or wrap fails — instead of silently dropping back to plain subprocess/ulimit.
+
+### Changed
+- **`kb_ingest` tool removed** in favor of the three split entry points; `KB_WRITER_TOOLS` and `make_kb_tools` updated. Internal helper renamed: `_ingest_document_with_kb` → `_ingest_text_with_kb` / `_ingest_file_with_kb` / `_ingest_url_with_kb`.
+- **research_demo agent guide** updated to direct ingestion at `ingest_arxiv_paper` for arXiv papers and at the typed `kb_ingest_*` tools for everything else.
+
+## [0.5.1] - 2026-04-18
+
+### Added
+- **Capability-based permission engine** (PR #72): Framework-independent engine that replaces the ConfirmationPlugin-based HITL. Tools declare capabilities (e.g. `filesystem.write(path=...)`, `http.read`, `shell.exec`) via `@register_tool(capabilities=...)`. Rules are evaluated from four sources — builtin defaults, user `~/.{app_name}/settings.json`, project `./.{app_name}/settings.json`, and in-memory session. When no rule matches, the user is prompted with `Allow once / for session / always (save to project) / Deny`; "always" grants persist into project settings.
+- **Matchers**: `PathMatcher` (with `**` glob), `URLMatcher`, `ShellMatcher`, and `StringGlobMatcher`.
+- **ADK `PermissionPlugin`**: capability gating for ADK agents.
+- **LangGraph `wrap_tool_for_permission`**: per-tool permission wrapping for LangGraph `ToolNode`.
+- **`permissions` and `permissions_enabled` settings** for runtime control.
+- **`EXEMPT` sentinel**: opts a tool out of the permission engine (used for backend state tools and ADK `transfer_to_agent`).
+- **Real BM25 backends**: `bm25s` (preferred) and `rank_bm25` fallback.
+
+### Changed
+- **`@register_tool`** now requires the `capabilities=` kwarg.
+- **Default grants broadened**: `filesystem.*` auto-extends to the parent directory; `memory.*` and `kb.*` are allowed by default.
+- **Ask prompt** now shows the directory-scope target.
+- **Workflow init**: `_ensure_managers_initialized` runs in a worker thread to avoid blocking the event loop.
+- **HITL confirmation** extracted into a backend-neutral module during the permissions migration.
+- **Memory tools**: deduped registry-bound and factory-bound entry points.
+
+### Fixed
+- Matcher canonicalize/matches preserves the `*` wildcard.
+- `KnowledgeBaseManager` concurrency contract tightened.
+
+### Removed
+- **`PermissionLevel`** (SAFE / CAUTION / DANGEROUS), `ConfirmationPlugin`, `_wrap_for_confirmation`, `hitl_tools`, and the `hitl_enabled` setting — superseded by the permission engine.
+
 ## [0.5.0] - 2026-04-17
 
 ### Added

{agentic_cli-0.5.0 → agentic_cli-0.5.2}/CLAUDE.md

@@ -47,7 +47,7 @@ agentic-cli/
 │   │       ├── persistence/  # Checkpointers, stores
 │   │       └── tools/        # LangChain-compatible wrappers
 │   ├── tools/
-│   │   ├── registry.py       # ToolRegistry, @register_tool, ToolCategory, PermissionLevel
+│   │   ├── registry.py       # ToolRegistry, @register_tool, ToolCategory
 │   │   ├── executor.py       # SafePythonExecutor
 │   │   ├── knowledge_tools.py # kb_search, kb_ingest, kb_list, kb_read
 │   │   ├── arxiv_tools.py    # search_arxiv, fetch_arxiv_paper, analyze_arxiv_paper
@@ -62,7 +62,7 @@ agentic-cli/
 │   │   ├── memory_tools.py   # save_memory, search_memory + MemoryStore
 │   │   ├── planning_tools.py # save_plan, get_plan + PlanStore
 │   │   ├── task_tools.py     # save_tasks, get_tasks + TaskStore
-│   │   ├── hitl_tools.py     # request_approval + ApprovalManager, HITLConfig
+│   │   ├── reflection_tools.py # save_reflection + ToolReflectionStore
 │   │   ├── shell/            # 8-layer shell security
 │   │   └── webfetch/         # Fetcher, converter, validator, robots
 │   ├── knowledge_base/
@@ -114,6 +114,9 @@ Workflow:
 2. For features: create `feature/<name>` (or `fix/<name>` or `refactor/<name>`) from `develop`, work there, merge back to `develop`
 3. When ready to release: merge `develop` → `main` and tag the release
 
+### What NOT to commit
+- `docs/` is gitignored on purpose (see `.gitignore`). It is a scratchpad for review notes, plans, and internal analysis. **Never `git add docs/…` or suggest committing anything under `docs/`.** If a document belongs in the repo, it lives elsewhere (README, CHANGELOG, top-level `*.md`).
+
 ## Development Principles
 
 ### Code Style
@@ -130,7 +133,8 @@ Workflow:
 
 ### Key Design Patterns
 - **Tool error handling**: All tools return `{"success": bool, ...}` dicts. Never raise `ToolError`.
-- **Tool registration**: Use `@register_tool(category=..., permission_level=..., description=...)` decorator. Tools are auto-discovered via the global `ToolRegistry`.
+- **Tool registration**: Use `@register_tool(category=..., capabilities=..., description=...)` decorator. `capabilities=` is required — pass `EXEMPT` for tools that need no permission check or a list of `Capability(name, target_arg=...)` tuples the engine matches against rules. Tools are auto-discovered via the global `ToolRegistry`.
+- **Permissions**: `workflow/permissions/` holds a framework-independent engine that evaluates declared capabilities against rules from four sources (builtin, user `~/.{app_name}/settings.json`, project `./.{app_name}/settings.json`, in-memory session). ADK + LangGraph gate tool calls via `workflow/adk/permission_plugin.py::PermissionPlugin` and `workflow/langgraph/permission_wrap.py::wrap_tool_for_permission`. See `docs/superpowers/specs/2026-04-18-permissions-system-design.md`.
 - **Service registry**: Tools access services and shared state via `get_service(key)` from `workflow.service_registry`. A single ContextVar holds a `dict[str, Any]` set by the workflow manager during processing. Complex services (KBManager, SandboxManager, MemoryStore) are lazily created; simple state (plan string, task list) lives directly in the registry dict.
 - **Manager detection**: Tools decorated with `@requires("kb_manager")` etc. are scanned by `BaseWorkflowManager._detect_required_managers()` which lazily creates only the needed services.
 - **Atomic writes**: Use `atomic_write_json`/`atomic_write_text` from `persistence/_utils.py` for file persistence.

{agentic_cli-0.5.0 → agentic_cli-0.5.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentic-cli
-Version: 0.5.0
+Version: 0.5.2
 Summary: A framework for building domain-specific agentic CLI applications
 Project-URL: Homepage, https://github.com/shoom1/agentic-cli
 Project-URL: Repository, https://github.com/shoom1/agentic-cli
@@ -32,7 +32,9 @@ Requires-Dist: pytest-asyncio>=0.24.0; extra == 'dev'
 Requires-Dist: pytest-cov>=6.0.0; extra == 'dev'
 Requires-Dist: pytest>=8.0.0; extra == 'dev'
 Provides-Extra: kb
+Requires-Dist: bm25s>=0.2.0; extra == 'kb'
 Requires-Dist: faiss-cpu>=1.7.0; extra == 'kb'
+Requires-Dist: rank-bm25>=0.2.2; extra == 'kb'
 Requires-Dist: sentence-transformers<3.0.0,>=2.0.0; extra == 'kb'
 Requires-Dist: torch>=2.2.0; extra == 'kb'
 Provides-Extra: langgraph
@@ -84,7 +86,7 @@ Agentic CLI provides the core infrastructure for building interactive CLI applic
 ├─────────────────────────────┬───────────────────────────────────────┤
 │   GoogleADKWorkflowManager  │     LangGraphWorkflowManager          │
 │   (default)                 │     (optional: langgraph extra)       │
-│   + ConfirmationPlugin      │     + confirmation tool wrapper       │
+│   + PermissionPlugin        │     + wrap_tool_for_permission        │
 │   + LLMLoggingPlugin        │     + native ToolNode                 │
 │   + TaskProgressPlugin      │                                       │
 └─────────────────────────────┴───────────────────────────────────────┘
@@ -191,7 +193,7 @@ manager = GoogleADKWorkflowManager(
 ```
 
 ADK integrations:
-- **ConfirmationPlugin** — intercepts `DANGEROUS` tools and prompts the user for approval
+- **PermissionPlugin** — evaluates each tool's declared capabilities against the permission engine and prompts the user when no rule matches
 - **LLMLoggingPlugin** — structured logging of LLM requests/responses
 - **TaskProgressPlugin** — streams the task checklist into its own thinking box
 
@@ -217,7 +219,7 @@ Features:
 - **Explicit provider support**: Uses `langchain-google-genai` for Gemini (not VertexAI)
 - **Thinking mode**: Native support for Claude and Gemini thinking/reasoning
 - **Retry policies**: Automatic retry with exponential backoff
-- **Confirmation wrapper**: Wraps `DANGEROUS` tools to request HITL approval
+- **Permission wrapper**: Wraps each tool to gate execution through the permission engine
 - **Event streaming**: Real-time workflow events via `WorkflowEvent`
 
 Requires: `pip install agentic-cli[langgraph]`
@@ -232,7 +234,7 @@ Requires: `pip install agentic-cli[langgraph]`
 | State persistence | In-memory | Memory, PostgreSQL, or SQLite |
 | Thinking support | Native (Gemini) | Native (Claude & Gemini) |
 | Retry handling | Built-in | Built-in with backoff |
-| HITL confirmation | ConfirmationPlugin | Tool wrapper |
+| Permission gate | PermissionPlugin | wrap_tool_for_permission |
 | Context trimming | Native | Native |
 
 ### Auto-selection via Settings
@@ -366,11 +368,12 @@ configs = [coordinator, researcher, analyst]
 Tools are regular Python functions with type hints and docstrings. **All tools return `{"success": bool, ...}` dicts** — never raise exceptions.
 
 ```python
-from agentic_cli.tools import register_tool, ToolCategory, PermissionLevel
+from agentic_cli.tools import register_tool, ToolCategory
+from agentic_cli.workflow.permissions import Capability, EXEMPT
 
 @register_tool(
-    category=ToolCategory.DATA,
-    permission_level=PermissionLevel.SAFE,
+    category=ToolCategory.NETWORK,
+    capabilities=[Capability("http.read")],
     description="Search the database for matching records.",
 )
 def search_database(query: str, limit: int = 10) -> dict:
@@ -387,15 +390,14 @@ def search_database(query: str, limit: int = 10) -> dict:
     return {"success": True, "results": results, "count": len(results)}
 ```
 
-Registering via `@register_tool` is optional — you can pass raw callables into `AgentConfig.tools`. Registering gives the tool metadata for the registry, permission-aware HITL wrapping, and tool-summary formatting.
+Registering via `@register_tool` is optional — you can pass raw callables into `AgentConfig.tools`. Registering gives the tool metadata for the registry, capability-aware permission gating, and tool-summary formatting.
 
-### Permission Levels
+### Capabilities
 
-| Level | Behavior |
-|-------|----------|
-| `SAFE` | Runs silently |
-| `CAUTION` | Runs; result is surfaced prominently |
-| `DANGEROUS` | Intercepted by HITL — user must approve before execution |
+Tool access is gated by the **permission engine** (see the HITL section below). Each registered tool declares what it touches via `capabilities=`:
+
+- **`Capability("namespace.action", target_arg="...")`** — e.g. `Capability("filesystem.write", target_arg="path")`, `Capability("http.read", target_arg="url")`, `Capability("shell.exec", target_arg="command")`. The engine resolves `target_arg` against the actual tool-call arguments and matches against rules.
+- **`EXEMPT`** — opts the tool out of the engine entirely. Reserved for backend-internal tools (e.g. ADK `transfer_to_agent`, backend state tools).
 
 ### Built-in Tools
 
@@ -458,7 +460,7 @@ results = search_arxiv("transformer attention", max_results=10, categories=["cs.
 paper = fetch_arxiv_paper("1706.03762")  # "Attention Is All You Need"
 ```
 
-The `ArxivSearchSource` (in `knowledge_base/sources.py`) is cached via the service registry and reused across tools, including `kb_ingest` for arXiv URLs.
+The `ArxivSearchSource` (in `knowledge_base/sources.py`) is cached via the service registry and reused across tools. To pull a paper into the KB, use `ingest_arxiv_paper(arxiv_id)` — it composes the arxiv source with `kb_manager`.
 
 #### File Operations
 
@@ -476,7 +478,7 @@ list_dir("src/", include_hidden=False)
 diff_compare(source_a="old.txt", source_b="new.txt")
 ```
 
-**WRITE tools (caution)**
+**WRITE tools**
 
 ```python
 from agentic_cli.tools import write_file, edit_file
@@ -511,7 +513,8 @@ Renamed tools and a new concept-pages subsystem:
 
 ```python
 from agentic_cli.tools import (
-    kb_search, kb_ingest, kb_read, kb_list,
+    kb_search, kb_ingest_text, kb_ingest_file, kb_ingest_url,
+    kb_read, kb_list,
     kb_write_concept, kb_search_concepts,
     KB_READER_TOOLS, KB_WRITER_TOOLS,
 )
@@ -520,12 +523,16 @@ from agentic_cli.tools import (
 | Tool | Purpose |
 |------|---------|
 | `kb_search` | Hybrid BM25 + vector search with RRF fusion, filters by source/date |
-| `kb_ingest` | Ingest content or URL (arXiv URLs auto-fetch metadata + PDF) |
+| `kb_ingest_text` | Ingest in-memory text content (no FS or network access) |
+| `kb_ingest_file` | Ingest a local file; declares `filesystem.read(path)` for the permission engine |
+| `kb_ingest_url` | Ingest content from an http(s) URL; routed through the hardened `ContentFetcher` and declares `http.read(url)` |
 | `kb_read` | Return the per-document markdown sidecar (lazy-generated on first read) |
 | `kb_list` | List documents, optionally filtered |
 | `kb_write_concept` | Agent-authored concept/summary page (slugged, merged on overwrite) |
 | `kb_search_concepts` | Search concept pages (title-weighted, case-insensitive) |
 
+The single `kb_ingest` tool was split in 0.5.2 so each entry point declares the right capability — text-only stays under `kb.write`, while file and URL ingestion correctly trip `filesystem.read` / `http.read` checks. For arXiv papers, prefer `ingest_arxiv_paper(arxiv_id)`.
+
 Bundle convenience:
 
 ```python
@@ -580,15 +587,9 @@ Each tool keeps at most N reflections (FIFO eviction). Reflections can be inject
 
 #### HITL (Human-in-the-Loop)
 
-Two mechanisms:
-
-1. **Automatic confirmation for `DANGEROUS` tools** — handled by ADK's `ConfirmationPlugin` and LangGraph's tool wrapper. No code changes needed; just mark the tool `DANGEROUS`.
-2. **Explicit `request_approval` tool** — for domain-level checkpoints:
+Tool calls are gated by the **permission engine** (`workflow/permissions/`). Each tool declares a list of capabilities (e.g. `filesystem.write(path=...)`); the engine evaluates them against rules from four sources (builtin defaults, user `~/.{app_name}/settings.json`, project `./.{app_name}/settings.json`, in-memory session). When no rule matches, the user is prompted with `Allow once / Allow for session / Allow always (save to project) / Deny`. Always-grants persist into the project settings file so the next run picks them up automatically.
 
-   ```python
-   from agentic_cli.tools import hitl_tools
-   # request_approval(message, options) -> {"success": True, "choice": "..."}
-   ```
+See `docs/superpowers/specs/2026-04-18-permissions-system-design.md` for the full design.
 
 ## CLI Commands
 
@@ -764,22 +765,32 @@ agentic-cli/
 │   │   ├── adk/
 │   │   │   ├── manager.py                # GoogleADKWorkflowManager
 │   │   │   ├── event_processor.py
-│   │   │   ├── plugins.py                # ConfirmationPlugin, LLMLoggingPlugin
+│   │   │   ├── plugins.py                # LLMLoggingPlugin
+│   │   │   ├── permission_plugin.py      # PermissionPlugin (capability gating)
 │   │   │   └── task_progress_plugin.py
-│   │   └── langgraph/
-│   │       ├── manager.py                # LangGraphWorkflowManager
-│   │       ├── graph_builder.py
-│   │       ├── state.py
-│   │       └── persistence/              # Checkpointers and stores
+│   │   ├── langgraph/
+│   │   │   ├── manager.py                # LangGraphWorkflowManager
+│   │   │   ├── graph_builder.py
+│   │   │   ├── permission_wrap.py        # wrap_tool_for_permission
+│   │   │   ├── state.py
+│   │   │   └── persistence/              # Checkpointers and stores
+│   │   └── permissions/                  # Framework-independent permission engine
+│   │       ├── capabilities.py           # Capability, ResolvedCapability, EXEMPT
+│   │       ├── rules.py                  # Rule, Effect, RuleSource, CheckResult, AskScope
+│   │       ├── matchers.py               # Path/URL/Shell/StringGlob matchers
+│   │       ├── store.py                  # PermissionContext, BUILTIN_RULES, JSON load/save
+│   │       ├── prompt.py                 # build_request + parse_response
+│   │       └── engine.py                 # PermissionEngine
 │   ├── tools/
-│   │   ├── registry.py           # ToolRegistry, ToolCategory, PermissionLevel
+│   │   ├── registry.py           # ToolRegistry, ToolCategory, register_tool
 │   │   ├── factories.py          # Backend-aware tool factories
 │   │   ├── executor.py           # SafePythonExecutor
 │   │   ├── arxiv_tools.py        # search_arxiv, fetch_arxiv_paper, ingest_arxiv_paper
 │   │   ├── arxiv_source.py       # ArxivSearchSource (service-registered)
 │   │   ├── execution_tools.py    # execute_python
 │   │   ├── interaction_tools.py  # ask_clarification
-│   │   ├── knowledge_tools.py    # kb_search / kb_ingest / kb_read / kb_list /
+│   │   ├── knowledge_tools.py    # kb_search / kb_ingest_text / kb_ingest_file /
+│   │   │                         #   kb_ingest_url / kb_read / kb_list /
 │   │   │                         #   kb_write_concept / kb_search_concepts
 │   │   ├── file_read.py          # read_file, diff_compare
 │   │   ├── file_write.py         # write_file, edit_file (atomic)
@@ -790,7 +801,6 @@ agentic-cli/
 │   │   ├── pdf_utils.py          # PDF text extraction helpers
 │   │   ├── memory_tools.py       # save/search/update/delete + MemoryStore
 │   │   ├── reflection_tools.py   # save_reflection + ToolReflectionStore
-│   │   ├── hitl_tools.py         # request_approval
 │   │   ├── _core/                # Shared planning/task logic
 │   │   │   ├── planning.py
 │   │   │   └── tasks.py

{agentic_cli-0.5.0 → agentic_cli-0.5.2}/README.md

@@ -38,7 +38,7 @@ Agentic CLI provides the core infrastructure for building interactive CLI applic
 ├─────────────────────────────┬───────────────────────────────────────┤
 │   GoogleADKWorkflowManager  │     LangGraphWorkflowManager          │
 │   (default)                 │     (optional: langgraph extra)       │
-│   + ConfirmationPlugin      │     + confirmation tool wrapper       │
+│   + PermissionPlugin        │     + wrap_tool_for_permission        │
 │   + LLMLoggingPlugin        │     + native ToolNode                 │
 │   + TaskProgressPlugin      │                                       │
 └─────────────────────────────┴───────────────────────────────────────┘
@@ -145,7 +145,7 @@ manager = GoogleADKWorkflowManager(
 ```
 
 ADK integrations:
-- **ConfirmationPlugin** — intercepts `DANGEROUS` tools and prompts the user for approval
+- **PermissionPlugin** — evaluates each tool's declared capabilities against the permission engine and prompts the user when no rule matches
 - **LLMLoggingPlugin** — structured logging of LLM requests/responses
 - **TaskProgressPlugin** — streams the task checklist into its own thinking box
 
@@ -171,7 +171,7 @@ Features:
 - **Explicit provider support**: Uses `langchain-google-genai` for Gemini (not VertexAI)
 - **Thinking mode**: Native support for Claude and Gemini thinking/reasoning
 - **Retry policies**: Automatic retry with exponential backoff
-- **Confirmation wrapper**: Wraps `DANGEROUS` tools to request HITL approval
+- **Permission wrapper**: Wraps each tool to gate execution through the permission engine
 - **Event streaming**: Real-time workflow events via `WorkflowEvent`
 
 Requires: `pip install agentic-cli[langgraph]`
@@ -186,7 +186,7 @@ Requires: `pip install agentic-cli[langgraph]`
 | State persistence | In-memory | Memory, PostgreSQL, or SQLite |
 | Thinking support | Native (Gemini) | Native (Claude & Gemini) |
 | Retry handling | Built-in | Built-in with backoff |
-| HITL confirmation | ConfirmationPlugin | Tool wrapper |
+| Permission gate | PermissionPlugin | wrap_tool_for_permission |
 | Context trimming | Native | Native |
 
 ### Auto-selection via Settings
@@ -320,11 +320,12 @@ configs = [coordinator, researcher, analyst]
 Tools are regular Python functions with type hints and docstrings. **All tools return `{"success": bool, ...}` dicts** — never raise exceptions.
 
 ```python
-from agentic_cli.tools import register_tool, ToolCategory, PermissionLevel
+from agentic_cli.tools import register_tool, ToolCategory
+from agentic_cli.workflow.permissions import Capability, EXEMPT
 
 @register_tool(
-    category=ToolCategory.DATA,
-    permission_level=PermissionLevel.SAFE,
+    category=ToolCategory.NETWORK,
+    capabilities=[Capability("http.read")],
     description="Search the database for matching records.",
 )
 def search_database(query: str, limit: int = 10) -> dict:
@@ -341,15 +342,14 @@ def search_database(query: str, limit: int = 10) -> dict:
     return {"success": True, "results": results, "count": len(results)}
 ```
 
-Registering via `@register_tool` is optional — you can pass raw callables into `AgentConfig.tools`. Registering gives the tool metadata for the registry, permission-aware HITL wrapping, and tool-summary formatting.
+Registering via `@register_tool` is optional — you can pass raw callables into `AgentConfig.tools`. Registering gives the tool metadata for the registry, capability-aware permission gating, and tool-summary formatting.
 
-### Permission Levels
+### Capabilities
 
-| Level | Behavior |
-|-------|----------|
-| `SAFE` | Runs silently |
-| `CAUTION` | Runs; result is surfaced prominently |
-| `DANGEROUS` | Intercepted by HITL — user must approve before execution |
+Tool access is gated by the **permission engine** (see the HITL section below). Each registered tool declares what it touches via `capabilities=`:
+
+- **`Capability("namespace.action", target_arg="...")`** — e.g. `Capability("filesystem.write", target_arg="path")`, `Capability("http.read", target_arg="url")`, `Capability("shell.exec", target_arg="command")`. The engine resolves `target_arg` against the actual tool-call arguments and matches against rules.
+- **`EXEMPT`** — opts the tool out of the engine entirely. Reserved for backend-internal tools (e.g. ADK `transfer_to_agent`, backend state tools).
 
 ### Built-in Tools
 
@@ -412,7 +412,7 @@ results = search_arxiv("transformer attention", max_results=10, categories=["cs.
 paper = fetch_arxiv_paper("1706.03762")  # "Attention Is All You Need"
 ```
 
-The `ArxivSearchSource` (in `knowledge_base/sources.py`) is cached via the service registry and reused across tools, including `kb_ingest` for arXiv URLs.
+The `ArxivSearchSource` (in `knowledge_base/sources.py`) is cached via the service registry and reused across tools. To pull a paper into the KB, use `ingest_arxiv_paper(arxiv_id)` — it composes the arxiv source with `kb_manager`.
 
 #### File Operations
 
@@ -430,7 +430,7 @@ list_dir("src/", include_hidden=False)
 diff_compare(source_a="old.txt", source_b="new.txt")
 ```
 
-**WRITE tools (caution)**
+**WRITE tools**
 
 ```python
 from agentic_cli.tools import write_file, edit_file
@@ -465,7 +465,8 @@ Renamed tools and a new concept-pages subsystem:
 
 ```python
 from agentic_cli.tools import (
-    kb_search, kb_ingest, kb_read, kb_list,
+    kb_search, kb_ingest_text, kb_ingest_file, kb_ingest_url,
+    kb_read, kb_list,
     kb_write_concept, kb_search_concepts,
     KB_READER_TOOLS, KB_WRITER_TOOLS,
 )
@@ -474,12 +475,16 @@ from agentic_cli.tools import (
 | Tool | Purpose |
 |------|---------|
 | `kb_search` | Hybrid BM25 + vector search with RRF fusion, filters by source/date |
-| `kb_ingest` | Ingest content or URL (arXiv URLs auto-fetch metadata + PDF) |
+| `kb_ingest_text` | Ingest in-memory text content (no FS or network access) |
+| `kb_ingest_file` | Ingest a local file; declares `filesystem.read(path)` for the permission engine |
+| `kb_ingest_url` | Ingest content from an http(s) URL; routed through the hardened `ContentFetcher` and declares `http.read(url)` |
 | `kb_read` | Return the per-document markdown sidecar (lazy-generated on first read) |
 | `kb_list` | List documents, optionally filtered |
 | `kb_write_concept` | Agent-authored concept/summary page (slugged, merged on overwrite) |
 | `kb_search_concepts` | Search concept pages (title-weighted, case-insensitive) |
 
+The single `kb_ingest` tool was split in 0.5.2 so each entry point declares the right capability — text-only stays under `kb.write`, while file and URL ingestion correctly trip `filesystem.read` / `http.read` checks. For arXiv papers, prefer `ingest_arxiv_paper(arxiv_id)`.
+
 Bundle convenience:
 
 ```python
@@ -534,15 +539,9 @@ Each tool keeps at most N reflections (FIFO eviction). Reflections can be inject
 
 #### HITL (Human-in-the-Loop)
 
-Two mechanisms:
-
-1. **Automatic confirmation for `DANGEROUS` tools** — handled by ADK's `ConfirmationPlugin` and LangGraph's tool wrapper. No code changes needed; just mark the tool `DANGEROUS`.
-2. **Explicit `request_approval` tool** — for domain-level checkpoints:
+Tool calls are gated by the **permission engine** (`workflow/permissions/`). Each tool declares a list of capabilities (e.g. `filesystem.write(path=...)`); the engine evaluates them against rules from four sources (builtin defaults, user `~/.{app_name}/settings.json`, project `./.{app_name}/settings.json`, in-memory session). When no rule matches, the user is prompted with `Allow once / Allow for session / Allow always (save to project) / Deny`. Always-grants persist into the project settings file so the next run picks them up automatically.
 
-   ```python
-   from agentic_cli.tools import hitl_tools
-   # request_approval(message, options) -> {"success": True, "choice": "..."}
-   ```
+See `docs/superpowers/specs/2026-04-18-permissions-system-design.md` for the full design.
 
 ## CLI Commands
 
@@ -718,22 +717,32 @@ agentic-cli/
 │   │   ├── adk/
 │   │   │   ├── manager.py                # GoogleADKWorkflowManager
 │   │   │   ├── event_processor.py
-│   │   │   ├── plugins.py                # ConfirmationPlugin, LLMLoggingPlugin
+│   │   │   ├── plugins.py                # LLMLoggingPlugin
+│   │   │   ├── permission_plugin.py      # PermissionPlugin (capability gating)
 │   │   │   └── task_progress_plugin.py
-│   │   └── langgraph/
-│   │       ├── manager.py                # LangGraphWorkflowManager
-│   │       ├── graph_builder.py
-│   │       ├── state.py
-│   │       └── persistence/              # Checkpointers and stores
+│   │   ├── langgraph/
+│   │   │   ├── manager.py                # LangGraphWorkflowManager
+│   │   │   ├── graph_builder.py
+│   │   │   ├── permission_wrap.py        # wrap_tool_for_permission
+│   │   │   ├── state.py
+│   │   │   └── persistence/              # Checkpointers and stores
+│   │   └── permissions/                  # Framework-independent permission engine
+│   │       ├── capabilities.py           # Capability, ResolvedCapability, EXEMPT
+│   │       ├── rules.py                  # Rule, Effect, RuleSource, CheckResult, AskScope
+│   │       ├── matchers.py               # Path/URL/Shell/StringGlob matchers
+│   │       ├── store.py                  # PermissionContext, BUILTIN_RULES, JSON load/save
+│   │       ├── prompt.py                 # build_request + parse_response
+│   │       └── engine.py                 # PermissionEngine
 │   ├── tools/
-│   │   ├── registry.py           # ToolRegistry, ToolCategory, PermissionLevel
+│   │   ├── registry.py           # ToolRegistry, ToolCategory, register_tool
 │   │   ├── factories.py          # Backend-aware tool factories
 │   │   ├── executor.py           # SafePythonExecutor
 │   │   ├── arxiv_tools.py        # search_arxiv, fetch_arxiv_paper, ingest_arxiv_paper
 │   │   ├── arxiv_source.py       # ArxivSearchSource (service-registered)
 │   │   ├── execution_tools.py    # execute_python
 │   │   ├── interaction_tools.py  # ask_clarification
-│   │   ├── knowledge_tools.py    # kb_search / kb_ingest / kb_read / kb_list /
+│   │   ├── knowledge_tools.py    # kb_search / kb_ingest_text / kb_ingest_file /
+│   │   │                         #   kb_ingest_url / kb_read / kb_list /
 │   │   │                         #   kb_write_concept / kb_search_concepts
 │   │   ├── file_read.py          # read_file, diff_compare
 │   │   ├── file_write.py         # write_file, edit_file (atomic)
@@ -744,7 +753,6 @@ agentic-cli/
 │   │   ├── pdf_utils.py          # PDF text extraction helpers
 │   │   ├── memory_tools.py       # save/search/update/delete + MemoryStore
 │   │   ├── reflection_tools.py   # save_reflection + ToolReflectionStore
-│   │   ├── hitl_tools.py         # request_approval
 │   │   ├── _core/                # Shared planning/task logic
 │   │   │   ├── planning.py
 │   │   │   └── tasks.py

{agentic_cli-0.5.0 → agentic_cli-0.5.2}/examples/research_demo/agents.py

@@ -50,7 +50,7 @@ When asked to research papers on a topic:
 1. Check if a relevant concept page already exists with `kb_search_concepts(topic)`. If one is relevant, read it first — it already synthesizes prior work and lists the source `doc_ids` you can follow.
 2. Search arXiv for relevant papers. If a search returns `success: false`, report the error instead of retrying with simpler queries.
 3. Fetch metadata for papers of interest.
-4. Ingest papers into the knowledge base by passing an arXiv PDF URL to `kb_ingest` — this auto-fetches metadata, downloads the PDF, extracts text, embeds chunks, and generates the sidecar in one call. The response includes the LLM-generated `summary` and the `document_id`.
+4. Ingest papers into the knowledge base. For arXiv papers, call `ingest_arxiv_paper(arxiv_id)` — it handles metadata, PDF download, and rate limiting. For other web sources use `kb_ingest_url(url)`; for a local PDF use `kb_ingest_file(path)`; for raw text you already hold use `kb_ingest_text(content)`. Each returns a `document_id` and the LLM-generated `summary`.
 5. Read each paper's sidecar with `kb_read(doc_id)` to plan the per-paper analysis.
 6. Pull full text with `kb_read(doc_id, full=True)` for any sections where the sidecar is insufficient (specific numbers, quotes, methodology details).
 7. Save detailed per-paper analyses via `write_file`.

{agentic_cli-0.5.0 → agentic_cli-0.5.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "agentic-cli"
-version = "0.5.0"
+version = "0.5.2"
 description = "A framework for building domain-specific agentic CLI applications"
 readme = "README.md"
 license = "MIT"
@@ -46,6 +46,8 @@ kb = [
     "torch>=2.2.0",
     "sentence-transformers>=2.0.0,<3.0.0",
     "faiss-cpu>=1.7.0",
+    "bm25s>=0.2.0",
+    "rank-bm25>=0.2.2",
 ]
 langgraph = [
     "langgraph>=0.2.0",

{agentic_cli-0.5.0 → agentic_cli-0.5.2}/src/agentic_cli/__init__.py

@@ -92,4 +92,4 @@ __all__ = [
     "CLISettingsMixin",
 ]
 
-__version__ = "0.5.0"
+__version__ = "0.5.2"

agentic_cli-0.5.2/src/agentic_cli/knowledge_base/_bm25_backends.py

@@ -0,0 +1,137 @@
+"""Real BM25 index backends (bm25s, rank_bm25).
+
+Both implement the same interface as MockBM25Index so create_bm25_index()
+can return any of them interchangeably. Tokenization is lowercase whitespace
+split to match MockBM25Index; the scoring model is the library's own BM25.
+
+Neither underlying library supports true incremental add/remove, so we keep
+tokenized docs + chunk_ids in memory and rebuild the model lazily on search.
+"""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+from agentic_cli.file_utils import atomic_write_json
+
+
+def _tokenize(text: str) -> list[str]:
+    return text.lower().split()
+
+
+class _BM25BackendBase:
+    _INDEX_FILE: str = ""
+
+    def __init__(self):
+        self._chunk_ids: list[str] = []
+        self._tokenized: list[list[str]] = []
+        self._model = None
+
+    @property
+    def size(self) -> int:
+        return len(self._chunk_ids)
+
+    def add_documents(self, chunk_ids: list[str], texts: list[str]) -> None:
+        for cid, text in zip(chunk_ids, texts):
+            self._chunk_ids.append(cid)
+            self._tokenized.append(_tokenize(text))
+        self._model = None
+
+    def remove_documents(self, chunk_ids: list[str]) -> None:
+        remove_set = set(chunk_ids)
+        keep = [
+            (cid, toks)
+            for cid, toks in zip(self._chunk_ids, self._tokenized)
+            if cid not in remove_set
+        ]
+        if keep:
+            self._chunk_ids, self._tokenized = map(list, zip(*keep))
+        else:
+            self._chunk_ids, self._tokenized = [], []
+        self._model = None
+
+    def rebuild(self, chunk_ids: list[str], texts: list[str]) -> None:
+        self._chunk_ids = []
+        self._tokenized = []
+        self._model = None
+        self.add_documents(chunk_ids, texts)
+
+    def save(self, path: Path) -> None:
+        path.mkdir(parents=True, exist_ok=True)
+        atomic_write_json(
+            path / self._INDEX_FILE,
+            {"chunk_ids": self._chunk_ids, "tokenized": self._tokenized},
+        )
+
+    def load(self, path: Path) -> None:
+        index_path = path / self._INDEX_FILE
+        if not index_path.exists():
+            return
+        data = json.loads(index_path.read_text())
+        self._chunk_ids = data["chunk_ids"]
+        self._tokenized = data["tokenized"]
+        self._model = None
+
+
+class RankBM25Index(_BM25BackendBase):
+    """BM25 backed by rank_bm25.BM25Plus (pure Python).
+
+    BM25Plus rather than BM25Okapi because Okapi's IDF can go zero or
+    negative when a term appears in most of the corpus; BM25Plus adds a
+    delta offset that guarantees positive contributions on real matches.
+    """
+
+    _INDEX_FILE = "bm25_rank.json"
+
+    def search(self, query: str, top_k: int = 10) -> list[tuple[str, float]]:
+        if not self._chunk_ids:
+            return []
+        query_tokens = _tokenize(query)
+        if not query_tokens:
+            return []
+        query_set = set(query_tokens)
+        if self._model is None:
+            from rank_bm25 import BM25Plus
+
+            self._model = BM25Plus(self._tokenized)
+        scores = self._model.get_scores(query_tokens)
+        scored: list[tuple[str, float]] = []
+        for cid, doc_tokens, score in zip(
+            self._chunk_ids, self._tokenized, scores
+        ):
+            if query_set.intersection(doc_tokens):
+                scored.append((cid, float(score)))
+        scored.sort(key=lambda x: x[1], reverse=True)
+        return scored[:top_k]
+
+
+class BM25sIndex(_BM25BackendBase):
+    """BM25 backed by bm25s (NumPy/C-accelerated)."""
+
+    _INDEX_FILE = "bm25s_sidecar.json"
+
+    def _build_model(self) -> None:
+        import bm25s
+
+        model = bm25s.BM25()
+        model.index(self._tokenized, show_progress=False)
+        self._model = model
+
+    def search(self, query: str, top_k: int = 10) -> list[tuple[str, float]]:
+        if not self._chunk_ids:
+            return []
+        query_tokens = _tokenize(query)
+        if not query_tokens:
+            return []
+        if self._model is None:
+            self._build_model()
+        k = min(top_k, len(self._chunk_ids))
+        docs, scores = self._model.retrieve(
+            [query_tokens], k=k, show_progress=False
+        )
+        results: list[tuple[str, float]] = []
+        for idx, score in zip(docs[0], scores[0]):
+            if score > 0:
+                results.append((self._chunk_ids[int(idx)], float(score)))
+        return results