agentic-cli 0.5.1__tar.gz → 0.5.2__tar.gz

{agentic_cli-0.5.1 → agentic_cli-0.5.2}/CHANGELOG.md

@@ -5,6 +5,18 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.2] - 2026-05-01
+
+### Security
+- **`kb_ingest` split into `kb_ingest_text` / `kb_ingest_file` / `kb_ingest_url`**: the previous unified tool declared only `kb.write` (auto-allowed by the builtin `kb.*` rule) while internally reading arbitrary local paths and fetching arbitrary URLs. The split lets the permission engine gate `filesystem.read` and `http.read` at the right entry point, and the URL path now flows through the hardened `ContentFetcher` (URLValidator + manual redirect revalidation + DNS-rebinding mitigation).
+- **Concept-page slug traversal blocked** (`ConceptStore`): explicit slugs are validated against a strict `[a-z0-9-]+` allowlist, and `_concept_path` asserts the resolved path stays under `base_dir`. A slug like `../../etc/passwd` is now rejected before any filesystem write.
+- **Webfetch redirect handling rewritten**: `ContentFetcher.fetch` no longer follows redirects via httpx. Each `Location` is revalidated through `URLValidator` *before* the next GET, cross-host redirects are reported without contacting the new origin, same-host redirects re-check `robots.txt`, and the redirect chain is capped at 5.
+- **`os_sandbox_enabled=True` now fails closed**: when shell or Python execution is requested with sandboxing on, both executors refuse to run if the resolved sandbox is `NoOpSandbox` or wrap fails — instead of silently dropping back to plain subprocess/ulimit.
+
+### Changed
+- **`kb_ingest` tool removed** in favor of the three split entry points; `KB_WRITER_TOOLS` and `make_kb_tools` updated. Internal helper renamed: `_ingest_document_with_kb` → `_ingest_text_with_kb` / `_ingest_file_with_kb` / `_ingest_url_with_kb`.
+- **research_demo agent guide** updated to direct ingestion at `ingest_arxiv_paper` for arXiv papers and at the typed `kb_ingest_*` tools for everything else.
+
 ## [0.5.1] - 2026-04-18
 
 ### Added

{agentic_cli-0.5.1 → agentic_cli-0.5.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentic-cli
-Version: 0.5.1
+Version: 0.5.2
 Summary: A framework for building domain-specific agentic CLI applications
 Project-URL: Homepage, https://github.com/shoom1/agentic-cli
 Project-URL: Repository, https://github.com/shoom1/agentic-cli
@@ -460,7 +460,7 @@ results = search_arxiv("transformer attention", max_results=10, categories=["cs.
 paper = fetch_arxiv_paper("1706.03762")  # "Attention Is All You Need"
 ```
 
-The `ArxivSearchSource` (in `knowledge_base/sources.py`) is cached via the service registry and reused across tools, including `kb_ingest` for arXiv URLs.
+The `ArxivSearchSource` (in `knowledge_base/sources.py`) is cached via the service registry and reused across tools. To pull a paper into the KB, use `ingest_arxiv_paper(arxiv_id)` — it composes the arxiv source with `kb_manager`.
 
 #### File Operations
 
@@ -513,7 +513,8 @@ Renamed tools and a new concept-pages subsystem:
 
 ```python
 from agentic_cli.tools import (
-    kb_search, kb_ingest, kb_read, kb_list,
+    kb_search, kb_ingest_text, kb_ingest_file, kb_ingest_url,
+    kb_read, kb_list,
     kb_write_concept, kb_search_concepts,
     KB_READER_TOOLS, KB_WRITER_TOOLS,
 )
@@ -522,12 +523,16 @@ from agentic_cli.tools import (
 | Tool | Purpose |
 |------|---------|
 | `kb_search` | Hybrid BM25 + vector search with RRF fusion, filters by source/date |
-| `kb_ingest` | Ingest content or URL (arXiv URLs auto-fetch metadata + PDF) |
+| `kb_ingest_text` | Ingest in-memory text content (no FS or network access) |
+| `kb_ingest_file` | Ingest a local file; declares `filesystem.read(path)` for the permission engine |
+| `kb_ingest_url` | Ingest content from an http(s) URL; routed through the hardened `ContentFetcher` and declares `http.read(url)` |
 | `kb_read` | Return the per-document markdown sidecar (lazy-generated on first read) |
 | `kb_list` | List documents, optionally filtered |
 | `kb_write_concept` | Agent-authored concept/summary page (slugged, merged on overwrite) |
 | `kb_search_concepts` | Search concept pages (title-weighted, case-insensitive) |
 
+The single `kb_ingest` tool was split in 0.5.2 so each entry point declares the right capability — text-only stays under `kb.write`, while file and URL ingestion correctly trip `filesystem.read` / `http.read` checks. For arXiv papers, prefer `ingest_arxiv_paper(arxiv_id)`.
+
 Bundle convenience:
 
 ```python
@@ -784,7 +789,8 @@ agentic-cli/
 │   │   ├── arxiv_source.py       # ArxivSearchSource (service-registered)
 │   │   ├── execution_tools.py    # execute_python
 │   │   ├── interaction_tools.py  # ask_clarification
-│   │   ├── knowledge_tools.py    # kb_search / kb_ingest / kb_read / kb_list /
+│   │   ├── knowledge_tools.py    # kb_search / kb_ingest_text / kb_ingest_file /
+│   │   │                         #   kb_ingest_url / kb_read / kb_list /
 │   │   │                         #   kb_write_concept / kb_search_concepts
 │   │   ├── file_read.py          # read_file, diff_compare
 │   │   ├── file_write.py         # write_file, edit_file (atomic)

{agentic_cli-0.5.1 → agentic_cli-0.5.2}/README.md

@@ -412,7 +412,7 @@ results = search_arxiv("transformer attention", max_results=10, categories=["cs.
 paper = fetch_arxiv_paper("1706.03762")  # "Attention Is All You Need"
 ```
 
-The `ArxivSearchSource` (in `knowledge_base/sources.py`) is cached via the service registry and reused across tools, including `kb_ingest` for arXiv URLs.
+The `ArxivSearchSource` (in `knowledge_base/sources.py`) is cached via the service registry and reused across tools. To pull a paper into the KB, use `ingest_arxiv_paper(arxiv_id)` — it composes the arxiv source with `kb_manager`.
 
 #### File Operations
 
@@ -465,7 +465,8 @@ Renamed tools and a new concept-pages subsystem:
 
 ```python
 from agentic_cli.tools import (
-    kb_search, kb_ingest, kb_read, kb_list,
+    kb_search, kb_ingest_text, kb_ingest_file, kb_ingest_url,
+    kb_read, kb_list,
     kb_write_concept, kb_search_concepts,
     KB_READER_TOOLS, KB_WRITER_TOOLS,
 )
@@ -474,12 +475,16 @@ from agentic_cli.tools import (
 | Tool | Purpose |
 |------|---------|
 | `kb_search` | Hybrid BM25 + vector search with RRF fusion, filters by source/date |
-| `kb_ingest` | Ingest content or URL (arXiv URLs auto-fetch metadata + PDF) |
+| `kb_ingest_text` | Ingest in-memory text content (no FS or network access) |
+| `kb_ingest_file` | Ingest a local file; declares `filesystem.read(path)` for the permission engine |
+| `kb_ingest_url` | Ingest content from an http(s) URL; routed through the hardened `ContentFetcher` and declares `http.read(url)` |
 | `kb_read` | Return the per-document markdown sidecar (lazy-generated on first read) |
 | `kb_list` | List documents, optionally filtered |
 | `kb_write_concept` | Agent-authored concept/summary page (slugged, merged on overwrite) |
 | `kb_search_concepts` | Search concept pages (title-weighted, case-insensitive) |
 
+The single `kb_ingest` tool was split in 0.5.2 so each entry point declares the right capability — text-only stays under `kb.write`, while file and URL ingestion correctly trip `filesystem.read` / `http.read` checks. For arXiv papers, prefer `ingest_arxiv_paper(arxiv_id)`.
+
 Bundle convenience:
 
 ```python
@@ -736,7 +741,8 @@ agentic-cli/
 │   │   ├── arxiv_source.py       # ArxivSearchSource (service-registered)
 │   │   ├── execution_tools.py    # execute_python
 │   │   ├── interaction_tools.py  # ask_clarification
-│   │   ├── knowledge_tools.py    # kb_search / kb_ingest / kb_read / kb_list /
+│   │   ├── knowledge_tools.py    # kb_search / kb_ingest_text / kb_ingest_file /
+│   │   │                         #   kb_ingest_url / kb_read / kb_list /
 │   │   │                         #   kb_write_concept / kb_search_concepts
 │   │   ├── file_read.py          # read_file, diff_compare
 │   │   ├── file_write.py         # write_file, edit_file (atomic)

{agentic_cli-0.5.1 → agentic_cli-0.5.2}/examples/research_demo/agents.py

@@ -50,7 +50,7 @@ When asked to research papers on a topic:
 1. Check if a relevant concept page already exists with `kb_search_concepts(topic)`. If one is relevant, read it first — it already synthesizes prior work and lists the source `doc_ids` you can follow.
 2. Search arXiv for relevant papers. If a search returns `success: false`, report the error instead of retrying with simpler queries.
 3. Fetch metadata for papers of interest.
-4. Ingest papers into the knowledge base by passing an arXiv PDF URL to `kb_ingest` — this auto-fetches metadata, downloads the PDF, extracts text, embeds chunks, and generates the sidecar in one call. The response includes the LLM-generated `summary` and the `document_id`.
+4. Ingest papers into the knowledge base. For arXiv papers, call `ingest_arxiv_paper(arxiv_id)` — it handles metadata, PDF download, and rate limiting. For other web sources use `kb_ingest_url(url)`; for a local PDF use `kb_ingest_file(path)`; for raw text you already hold use `kb_ingest_text(content)`. Each returns a `document_id` and the LLM-generated `summary`.
 5. Read each paper's sidecar with `kb_read(doc_id)` to plan the per-paper analysis.
 6. Pull full text with `kb_read(doc_id, full=True)` for any sections where the sidecar is insufficient (specific numbers, quotes, methodology details).
 7. Save detailed per-paper analyses via `write_file`.

{agentic_cli-0.5.1 → agentic_cli-0.5.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "agentic-cli"
-version = "0.5.1"
+version = "0.5.2"
 description = "A framework for building domain-specific agentic CLI applications"
 readme = "README.md"
 license = "MIT"

{agentic_cli-0.5.1 → agentic_cli-0.5.2}/src/agentic_cli/__init__.py

@@ -92,4 +92,4 @@ __all__ = [
     "CLISettingsMixin",
 ]
 
-__version__ = "0.5.1"
+__version__ = "0.5.2"

{agentic_cli-0.5.1 → agentic_cli-0.5.2}/src/agentic_cli/knowledge_base/concepts.py

@@ -21,9 +21,28 @@ _SLUG_WHITESPACE = re.compile(r"[\s_]+")
 _SLUG_DASH_RUN = re.compile(r"-+")
 _FRONTMATTER_RE = re.compile(r"^---\n(.*?)\n---\n?(.*)", re.DOTALL)
 
+# Strict allowlist for explicit slugs. Auto-generated slugs from
+# slug_from_title() already conform; explicit slugs from callers must be
+# checked against this pattern to prevent path traversal (``../escape``)
+# and unsafe filenames.
+_SAFE_SLUG_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$")
+
 MAX_SLUG_LENGTH = 80
 
 
+def _is_safe_explicit_slug(slug: str) -> bool:
+    """Return True iff ``slug`` is safe to use as a concept filename.
+
+    Rejects empty strings, anything containing path separators, ``..``,
+    leading/trailing dashes, uppercase letters, or characters outside the
+    ``[a-z0-9-]`` set. This protects ``ConceptStore`` from writes outside
+    its base directory regardless of where ``slug`` originated.
+    """
+    if not slug or len(slug) > MAX_SLUG_LENGTH:
+        return False
+    return bool(_SAFE_SLUG_RE.match(slug))
+
+
 def slug_from_title(title: str) -> str:
     """Convert a title into a kebab-case ASCII slug.
 
@@ -119,7 +138,17 @@ class ConceptStore:
         self.base_dir = Path(base_dir)
 
     def _concept_path(self, slug: str) -> Path:
-        return self.base_dir / f"{slug}.md"
+        # Defense in depth: even if a caller bypasses the explicit-slug
+        # validation, refuse to compose a path that escapes base_dir.
+        candidate = (self.base_dir / f"{slug}.md").resolve()
+        base_resolved = self.base_dir.resolve()
+        try:
+            candidate.relative_to(base_resolved)
+        except ValueError as exc:
+            raise ValueError(
+                f"Refusing to compose concept path outside base_dir: {slug!r}"
+            ) from exc
+        return candidate
 
     def write(
         self,
@@ -150,6 +179,19 @@ class ConceptStore:
             Dict with keys ``success``, ``slug``, ``path``, ``action``
             (``"created"`` or ``"updated"``), and ``invalid_sources``.
         """
+        if slug and not _is_safe_explicit_slug(slug):
+            return {
+                "success": False,
+                "slug": "",
+                "path": "",
+                "action": "failed",
+                "error": (
+                    f"invalid slug {slug!r}: must match [a-z0-9-]+, no path "
+                    f"separators, max {MAX_SLUG_LENGTH} chars"
+                ),
+                "invalid_sources": [],
+            }
+
         valid_sources, invalid = _partition_sources(sources, valid_ids_check)
 
         if not valid_sources:

{agentic_cli-0.5.1 → agentic_cli-0.5.2}/src/agentic_cli/tools/__init__.py

@@ -28,7 +28,9 @@ from agentic_cli.tools.file_write import write_file, edit_file
 
 from agentic_cli.tools.knowledge_tools import (
     kb_search,
-    kb_ingest,
+    kb_ingest_text,
+    kb_ingest_file,
+    kb_ingest_url,
     kb_read,
     kb_list,
     kb_write_concept,
@@ -36,7 +38,13 @@ from agentic_cli.tools.knowledge_tools import (
 )
 
 KB_READER_TOOLS = [kb_search, kb_read, kb_list, kb_search_concepts]
-KB_WRITER_TOOLS = [*KB_READER_TOOLS, kb_ingest, kb_write_concept]
+KB_WRITER_TOOLS = [
+    *KB_READER_TOOLS,
+    kb_ingest_text,
+    kb_ingest_file,
+    kb_ingest_url,
+    kb_write_concept,
+]
 from agentic_cli.tools.arxiv_tools import (
     search_arxiv,
     fetch_arxiv_paper,
@@ -87,7 +95,9 @@ __all__ = [
     "google_search_tool",
     # Standard tool functions (ready to use with agents)
     "kb_search",
-    "kb_ingest",
+    "kb_ingest_text",
+    "kb_ingest_file",
+    "kb_ingest_url",
     "kb_read",
     "kb_list",
     "kb_write_concept",

{agentic_cli-0.5.1 → agentic_cli-0.5.2}/src/agentic_cli/tools/executor.py

@@ -247,6 +247,28 @@ class SafePythonExecutor:
                 from agentic_cli.tools.shell.os_sandbox import get_os_sandbox
 
                 sandbox = get_os_sandbox()
+                # Fail closed when sandboxing was requested but no real
+                # isolation is available — silently dropping back to a plain
+                # subprocess would defeat the user's opt-in.
+                if sandbox.sandbox_type == "none":
+                    logger.error(
+                        "python_executor.os_sandbox_required_but_unavailable",
+                        sandbox_type=sandbox.sandbox_type,
+                    )
+                    elapsed = (time.time() - start_time) * 1000
+                    return {
+                        "success": False,
+                        "output": "",
+                        "result": None,
+                        "error": (
+                            "OS sandbox is required (os_sandbox_enabled=True) "
+                            "but no supported sandbox tool is available on "
+                            "this system (install sandbox-exec on macOS or "
+                            "bwrap on Linux). Refusing to execute without "
+                            "isolation."
+                        ),
+                        "execution_time_ms": round(elapsed, 2),
+                    }
                 wrap_result = sandbox.wrap_python_command(
                     [sys.executable, "-c", script],
                     Path.cwd(),
@@ -266,17 +288,22 @@ class SafePythonExecutor:
                         timeout=timeout,
                     )
                 else:
-                    logger.warning(
+                    logger.error(
                         "python_executor.os_sandbox_wrap_failed",
                         error=wrap_result.error,
                     )
-                    proc = subprocess.run(
-                        [sys.executable, "-c", script],
-                        input=code,
-                        capture_output=True,
-                        text=True,
-                        timeout=timeout,
-                    )
+                    elapsed = (time.time() - start_time) * 1000
+                    return {
+                        "success": False,
+                        "output": "",
+                        "result": None,
+                        "error": (
+                            f"OS sandbox wrap failed "
+                            f"({wrap_result.sandbox_type}): {wrap_result.error}. "
+                            f"Refusing to execute without isolation."
+                        ),
+                        "execution_time_ms": round(elapsed, 2),
+                    }
             else:
                 proc = subprocess.run(
                     [sys.executable, "-c", script],

{agentic_cli-0.5.1 → agentic_cli-0.5.2}/src/agentic_cli/tools/factories.py

@@ -107,18 +107,22 @@ def make_kb_tools(kb_manager, user_kb_manager=None) -> list[Callable]:
         user_kb_manager: Optional user-scoped KnowledgeBaseManager.
 
     Returns:
-        [kb_search, kb_ingest, kb_read, kb_list, kb_write_concept, kb_search_concepts]
-        (six tools — the concept pages pair was added in Phase 2.)
+        [kb_search, kb_ingest_text, kb_ingest_file, kb_ingest_url,
+         kb_read, kb_list, kb_write_concept, kb_search_concepts]
     """
     from agentic_cli.tools.knowledge_tools import (
         READ_DOCUMENT_MAX_CHARS,
-        _ingest_document_with_kb,
+        _ingest_file_with_kb,
+        _ingest_text_with_kb,
+        _ingest_url_with_kb,
         _list_documents_in_kbs,
         _read_document_from_kbs,
         _search_concepts_with_kb,
         _search_kbs,
         _write_concept_with_kb,
-        kb_ingest as _orig_ingest,
+        kb_ingest_file as _orig_ingest_file,
+        kb_ingest_text as _orig_ingest_text,
+        kb_ingest_url as _orig_ingest_url,
         kb_list as _orig_list,
         kb_read as _orig_read,
         kb_search as _orig_search,
@@ -133,9 +137,8 @@ def make_kb_tools(kb_manager, user_kb_manager=None) -> list[Callable]:
     ) -> dict[str, Any]:
         return _search_kbs(kb_manager, user_kb_manager, query, filters, top_k)
 
-    async def kb_ingest(
-        content: str = "",
-        url_or_path: str = "",
+    async def kb_ingest_text(
+        content: str,
         title: str = "",
         source_type: str = "user",
         source_url: str | None = None,
@@ -143,10 +146,49 @@ def make_kb_tools(kb_manager, user_kb_manager=None) -> list[Callable]:
         abstract: str = "",
         tags: list[str] | None = None,
     ) -> dict[str, Any]:
-        return await _ingest_document_with_kb(
+        return await _ingest_text_with_kb(
             kb_manager,
             content=content,
-            url_or_path=url_or_path,
+            title=title,
+            source_type=source_type,
+            source_url=source_url,
+            authors=authors,
+            abstract=abstract,
+            tags=tags,
+        )
+
+    async def kb_ingest_file(
+        path: str,
+        title: str = "",
+        source_type: str = "local",
+        source_url: str | None = None,
+        authors: list[str] | None = None,
+        abstract: str = "",
+        tags: list[str] | None = None,
+    ) -> dict[str, Any]:
+        return await _ingest_file_with_kb(
+            kb_manager,
+            path=path,
+            title=title,
+            source_type=source_type,
+            source_url=source_url,
+            authors=authors,
+            abstract=abstract,
+            tags=tags,
+        )
+
+    async def kb_ingest_url(
+        url: str,
+        title: str = "",
+        source_type: str = "web",
+        source_url: str | None = None,
+        authors: list[str] | None = None,
+        abstract: str = "",
+        tags: list[str] | None = None,
+    ) -> dict[str, Any]:
+        return await _ingest_url_with_kb(
+            kb_manager,
+            url=url,
             title=title,
             source_type=source_type,
             source_url=source_url,
@@ -194,8 +236,12 @@ def make_kb_tools(kb_manager, user_kb_manager=None) -> list[Callable]:
 
     kb_search.__name__ = "kb_search"
     kb_search.__doc__ = _orig_search.__doc__
-    kb_ingest.__name__ = "kb_ingest"
-    kb_ingest.__doc__ = _orig_ingest.__doc__
+    kb_ingest_text.__name__ = "kb_ingest_text"
+    kb_ingest_text.__doc__ = _orig_ingest_text.__doc__
+    kb_ingest_file.__name__ = "kb_ingest_file"
+    kb_ingest_file.__doc__ = _orig_ingest_file.__doc__
+    kb_ingest_url.__name__ = "kb_ingest_url"
+    kb_ingest_url.__doc__ = _orig_ingest_url.__doc__
     kb_read.__name__ = "kb_read"
     kb_read.__doc__ = _orig_read.__doc__
     kb_list.__name__ = "kb_list"
@@ -207,7 +253,9 @@ def make_kb_tools(kb_manager, user_kb_manager=None) -> list[Callable]:
 
     return [
         kb_search,
-        kb_ingest,
+        kb_ingest_text,
+        kb_ingest_file,
+        kb_ingest_url,
         kb_read,
         kb_list,
         kb_write_concept,