cat-stack 2.5.0__tar.gz → 2.5.2__tar.gz

{cat_stack-2.5.0 → cat_stack-2.5.2}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: cat-stack
-Version: 2.5.0
+Version: 2.5.2
 Summary: Domain-agnostic text, image, PDF, and DOCX classification engine powered by LLMs
 Project-URL: Documentation, https://github.com/chrissoria/cat-stack#readme
 Project-URL: Issues, https://github.com/chrissoria/cat-stack/issues
@@ -24,9 +24,9 @@ Requires-Dist: pandas
 Requires-Dist: requests
 Requires-Dist: tqdm
 Provides-Extra: agent
-Requires-Dist: cat-claws[claude]>=0.3.0; extra == 'agent'
+Requires-Dist: cat-claws[claude]>=0.3.2; extra == 'agent'
 Provides-Extra: codex-agent
-Requires-Dist: cat-claws[codex]>=0.3.0; extra == 'codex-agent'
+Requires-Dist: cat-claws[codex]>=0.3.2; extra == 'codex-agent'
 Provides-Extra: docx
 Requires-Dist: python-docx>=1.0.0; extra == 'docx'
 Provides-Extra: embeddings
@@ -138,7 +138,7 @@ cat.classify(
 ```
 
 ### `extract()`
-Discover categories from a corpus using LLM-driven exploration.
+Discover categories from a corpus using LLM-driven exploration. Since v2.5.0, text consolidation runs the full explore → `collapse_themes()` pipeline (`engine="collapse"`, the default): the entire raw label inventory reaches the semantic merge — Jaro-Winkler dedup, embedding pre-merge, quality-controlled LLM passes, then a count-guided reduction to at most `max_categories`. Pass `engine="legacy"` to reproduce pre-2.5 runs (single merge call over a truncated inventory). `collapse_kwargs` forwards options to `collapse_themes()`; `max_workers` parallelizes both extraction and consolidation.
 
 ```python
 cat.extract(
@@ -197,7 +197,7 @@ cat.collapse_themes(
 | Parameter | Default | Description |
 | --- | --- | --- |
 | `input_data` | — | List of category labels, or a frequency `Series`/`dict` (`label -> count`). |
-| `api_key` | `None` | API key for the LLM provider (required). |
+| `api_key` | `None` | API key for the LLM provider. Not required for subscription/CLI backends (`claude-code`, `claude-agent`, `codex-agent`) or `ollama`. |
 | `description` | `""` | The survey question or context, used in the merge prompt. |
 | `passes` | `1` | Number of merge iterations, or `"auto"` to iterate until the embedding-quality benchmark peaks. |
 | `max_passes` | `10` | Cap on iterations when `passes="auto"`. |
@@ -207,6 +207,8 @@ cat.collapse_themes(
 | `embedding_merge_threshold` | `0.92` | Cosine similarity at/above which labels are merged in the pre-LLM embedding step. `None`/`>=1.0` disables it. |
 | `shuffle` | `True` | Randomize order each pass so batch composition varies (improves convergence stability). |
 | `final_consolidation` | `0.82` | Cosine threshold for one greedy global embedding re-merge after all passes, collapsing cross-batch duplicates. Conservative by design (errs toward keeping categories). `False`/`None` skips it. |
+| `top_n` | `None` | If set, a final global LLM call consolidates the surviving list into at most N categories, guided by each label's generation count (frequent themes favored; overlapping labels merged rather than dropped). Guaranteed `<= top_n` via truncation plus a deterministic top-N-by-count fallback. |
+| `prune` | `False` | `True` = drop conceptual duplicates keeping one representative verbatim; never renames or merges merely-related labels. |
 | `user_model` | `"gpt-4o"` | Model for the merge phase. Use a capable model — small models can degenerate. |
 | `model_source` | `"auto"` | Provider for `user_model` (`"auto"`, `"openai"`, `"huggingface"`, …). |
 | `unique_model` | `None` | If set, run an initial extract-unique thinning phase on this (typically cheaper) model before the merge phase. `None` skips the phase (backward compatible). |
@@ -237,6 +239,10 @@ OpenAI, Anthropic, Google (Gemini), Mistral, Perplexity, xAI (Grok), HuggingFace
 
 All providers use the same `(model_name, provider, api_key)` tuple format. Provider is auto-detected from model name if omitted.
 
+**Subscription backends (no API key).** Three `model_source` values authenticate through a chat subscription instead of a metered key — leave `api_key` unset: `"claude-agent"` (Claude subscription via the Agent SDK; `pip install "cat-stack[agent]"`), `"claude-code"` (the Claude Code CLI, if installed — no extra needed), and `"codex-agent"` (ChatGPT subscription; `pip install "cat-stack[codex-agent]"`). Classification, extraction, exploration, and summarization all route through them.
+
+**Reproducibility note:** the same nominal model can behave differently across access routes. In a seed-matched benchmark (Claude Sonnet 5, temperature 0), extraction through the Agent SDK produced more varied label phrasings than the direct API (89% vs. 74% unique raw labels), while final consolidated taxonomies were equivalent. If raw label counts matter to your analysis, record the access route alongside the model version.
+
 ## Features
 
 - **Automatic prompt optimization** (`prompt_tune`) — correct a small sample in a browser UI, and the system generates per-category instructions that improve accuracy

{cat_stack-2.5.0 → cat_stack-2.5.2}/README.md

@@ -97,7 +97,7 @@ cat.classify(
 ```
 
 ### `extract()`
-Discover categories from a corpus using LLM-driven exploration.
+Discover categories from a corpus using LLM-driven exploration. Since v2.5.0, text consolidation runs the full explore → `collapse_themes()` pipeline (`engine="collapse"`, the default): the entire raw label inventory reaches the semantic merge — Jaro-Winkler dedup, embedding pre-merge, quality-controlled LLM passes, then a count-guided reduction to at most `max_categories`. Pass `engine="legacy"` to reproduce pre-2.5 runs (single merge call over a truncated inventory). `collapse_kwargs` forwards options to `collapse_themes()`; `max_workers` parallelizes both extraction and consolidation.
 
 ```python
 cat.extract(
@@ -156,7 +156,7 @@ cat.collapse_themes(
 | Parameter | Default | Description |
 | --- | --- | --- |
 | `input_data` | — | List of category labels, or a frequency `Series`/`dict` (`label -> count`). |
-| `api_key` | `None` | API key for the LLM provider (required). |
+| `api_key` | `None` | API key for the LLM provider. Not required for subscription/CLI backends (`claude-code`, `claude-agent`, `codex-agent`) or `ollama`. |
 | `description` | `""` | The survey question or context, used in the merge prompt. |
 | `passes` | `1` | Number of merge iterations, or `"auto"` to iterate until the embedding-quality benchmark peaks. |
 | `max_passes` | `10` | Cap on iterations when `passes="auto"`. |
@@ -166,6 +166,8 @@ cat.collapse_themes(
 | `embedding_merge_threshold` | `0.92` | Cosine similarity at/above which labels are merged in the pre-LLM embedding step. `None`/`>=1.0` disables it. |
 | `shuffle` | `True` | Randomize order each pass so batch composition varies (improves convergence stability). |
 | `final_consolidation` | `0.82` | Cosine threshold for one greedy global embedding re-merge after all passes, collapsing cross-batch duplicates. Conservative by design (errs toward keeping categories). `False`/`None` skips it. |
+| `top_n` | `None` | If set, a final global LLM call consolidates the surviving list into at most N categories, guided by each label's generation count (frequent themes favored; overlapping labels merged rather than dropped). Guaranteed `<= top_n` via truncation plus a deterministic top-N-by-count fallback. |
+| `prune` | `False` | `True` = drop conceptual duplicates keeping one representative verbatim; never renames or merges merely-related labels. |
 | `user_model` | `"gpt-4o"` | Model for the merge phase. Use a capable model — small models can degenerate. |
 | `model_source` | `"auto"` | Provider for `user_model` (`"auto"`, `"openai"`, `"huggingface"`, …). |
 | `unique_model` | `None` | If set, run an initial extract-unique thinning phase on this (typically cheaper) model before the merge phase. `None` skips the phase (backward compatible). |
@@ -196,6 +198,10 @@ OpenAI, Anthropic, Google (Gemini), Mistral, Perplexity, xAI (Grok), HuggingFace
 
 All providers use the same `(model_name, provider, api_key)` tuple format. Provider is auto-detected from model name if omitted.
 
+**Subscription backends (no API key).** Three `model_source` values authenticate through a chat subscription instead of a metered key — leave `api_key` unset: `"claude-agent"` (Claude subscription via the Agent SDK; `pip install "cat-stack[agent]"`), `"claude-code"` (the Claude Code CLI, if installed — no extra needed), and `"codex-agent"` (ChatGPT subscription; `pip install "cat-stack[codex-agent]"`). Classification, extraction, exploration, and summarization all route through them.
+
+**Reproducibility note:** the same nominal model can behave differently across access routes. In a seed-matched benchmark (Claude Sonnet 5, temperature 0), extraction through the Agent SDK produced more varied label phrasings than the direct API (89% vs. 74% unique raw labels), while final consolidated taxonomies were equivalent. If raw label counts matter to your analysis, record the access route alongside the model version.
+
 ## Features
 
 - **Automatic prompt optimization** (`prompt_tune`) — correct a small sample in a browser UI, and the system generates per-category instructions that improve accuracy

{cat_stack-2.5.0 → cat_stack-2.5.2}/pyproject.toml

@@ -42,8 +42,8 @@ embeddings = ["sentence-transformers>=2.2.0"]
 # `agent` keeps its historical meaning (the Claude backend) so every shipped
 # install hint stays true; `codex-agent` matches the provider string so the
 # error-message hint is copy-pasteable.
-agent = ["cat-claws[claude]>=0.3.0"]
-codex-agent = ["cat-claws[codex]>=0.3.0"]
+agent = ["cat-claws[claude]>=0.3.2"]
+codex-agent = ["cat-claws[codex]>=0.3.2"]
 
 [project.urls]
 Documentation = "https://github.com/chrissoria/cat-stack#readme"

{cat_stack-2.5.0 → cat_stack-2.5.2}/src/catstack/__about__.py

@@ -1,7 +1,7 @@
 # SPDX-FileCopyrightText: 2025-present Christopher Soria <chrissoria@berkeley.edu>
 #
 # SPDX-License-Identifier: GPL-3.0-or-later
-__version__ = "2.5.0"
+__version__ = "2.5.2"
 __author__ = "Chris Soria"
 __email__ = "chrissoria@berkeley.edu"
 __title__ = "cat-stack"

{cat_stack-2.5.0 → cat_stack-2.5.2}/src/catstack/_providers.py

@@ -735,8 +735,10 @@ PROVIDER_CONFIG = {
 
 
 # Providers that route through complete() with no HTTP endpoint of their own
-# (subscription logins / CLI). Features that build a direct HTTP request
-# (image, PDF) can't use them — guard with a clear error, not a deep crash.
+# (subscription logins / CLI). Features that build a direct HTTP request can't
+# use them — guard with a clear error, not a deep crash. (Images and PDF pages
+# DO work on the cat-claws agent backends: they go through the adapters'
+# images= argument, see _split_agent_content / _call_agent_image.)
 _SUBSCRIPTION_PROVIDERS = ("claude-code", "claude-agent", "codex-agent")
 
 # Agent-SDK backends routed through cat-claws: provider -> (adapter name,
@@ -748,6 +750,46 @@ _AGENT_BACKENDS = {
 }
 
 
+def _split_agent_content(content):
+    """Split one message's content into (text, images) for the cat-claws
+    adapters.
+
+    Plain-string content is text only. Multimodal content is a list of blocks
+    in whichever provider shape the prompt builder produced (Anthropic
+    `image` + base64 `source`, OpenAI `image_url` data URL, Google
+    `inline_data`); text blocks are joined and image blocks become the
+    adapters' ``{"media_type", "data"}`` dicts. Unknown block types are
+    dropped rather than stringified into the prompt.
+    """
+    if not isinstance(content, list):
+        return content, []
+    texts, images = [], []
+    for block in content:
+        if not isinstance(block, dict):
+            continue
+        kind = block.get("type")
+        if kind == "text":
+            texts.append(block.get("text", ""))
+        elif kind == "image":
+            src = block.get("source") or {}
+            if src.get("type") == "base64" and src.get("data"):
+                images.append({"media_type": src.get("media_type") or "image/png",
+                               "data": src["data"]})
+        elif kind == "image_url":
+            url = (block.get("image_url") or {}).get("url", "")
+            if url.startswith("data:") and ";base64," in url:
+                header, data = url.split(",", 1)
+                images.append({"media_type": header[5:].split(";", 1)[0] or "image/png",
+                               "data": data})
+        elif kind == "inline_data" and block.get("data"):
+            images.append({"media_type": block.get("mime_type") or "image/png",
+                           "data": block["data"]})
+    for im in images:  # the API wants image/jpeg, not image/jpg
+        if im["media_type"] == "image/jpg":
+            im["media_type"] = "image/jpeg"
+    return "\n\n".join(t for t in texts if t), images
+
+
 def _require_http_provider(model_source, feature):
     """Raise a clear error when an HTTP-only feature is used with a
     subscription/CLI provider (claude-code / claude-agent / codex-agent)."""
@@ -1272,17 +1314,26 @@ class UnifiedLLMClient:
             )
         import asyncio
 
+        # Multimodal content (image summaries, rendered PDF pages) arrives as
+        # a list of blocks: text goes to the prompt, images to the adapter's
+        # `images=` argument (base.AgentAdapter.one_shot contract).
         system_parts = []
         user_parts = []
+        images = []
         for msg in messages:
+            text, msg_images = _split_agent_content(msg["content"])
             if msg["role"] == "system":
-                system_parts.append(msg["content"])
+                system_parts.append(text)
             elif msg["role"] in ("user", "assistant"):
-                user_parts.append(msg["content"])
+                user_parts.append(text)
+                images.extend(msg_images)
         system_prompt = "\n\n".join(system_parts) if system_parts else None
         user_prompt = "\n\n".join(user_parts)
 
         adapter = get_adapter(adapter_name)
+        # Only pass images when there are some, so text-only calls are
+        # byte-identical to before.
+        image_kwargs = {"images": images} if images else {}
         try:
             return asyncio.run(
                 adapter.one_shot(
@@ -1290,6 +1341,7 @@ class UnifiedLLMClient:
                     system_prompt=system_prompt,
                     model=self.model,
                     thinking_budget=thinking_budget or 0,
+                    **image_kwargs,
                 )
             )
         except Exception as e:

{cat_stack-2.5.0 → cat_stack-2.5.2}/src/catstack/collapse_themes.py

@@ -25,7 +25,7 @@ import numpy as np
 import pandas as pd
 from jellyfish import jaro_winkler_similarity
 
-from ._providers import UnifiedLLMClient, detect_provider
+from ._providers import UnifiedLLMClient, detect_provider, _SUBSCRIPTION_PROVIDERS
 from ._utils import _clean_label
 
 __all__ = [
@@ -301,6 +301,19 @@ def _count_guidance(current, input_data):
     return {lbl: max(int(c), orig.get(_norm_key(lbl), 0)) for lbl, c in cur.items()}
 
 
+def _as_label_list(x):
+    """Enforce the documented list[str] return contract at exit points.
+
+    Dict / Series / DataFrame inputs can reach a return untouched (e.g.
+    passes=0 with a top_n no-op), which would leak the input container —
+    and, via the filename write, save counts instead of labels."""
+    if isinstance(x, list):
+        return [str(v) for v in x]
+    if isinstance(x, dict):
+        return [str(k) for k in x]
+    return [str(k) for k in _to_counts(x)]
+
+
 def _to_counts(input_data):
     """Coerce the accepted input forms into a {category: count} dict."""
     if isinstance(input_data, pd.DataFrame):
@@ -425,7 +438,9 @@ def collapse_themes(
         input_data: Themes to collapse. list[str] (duplicates allowed), pandas
             Series, dict {category: count}, or DataFrame with "category"
             [and optional "count"] columns.
-        api_key (str): API key for the model provider.
+        api_key (str): API key for the model provider. Not required for
+            subscription/CLI backends (`claude-code`, `claude-agent`,
+            `codex-agent`) or `ollama`.
         description (str): Data/question context, injected into the prompt — e.g.
             the survey question the categories came from. Helps the model judge
             which distinctions matter.
@@ -505,9 +520,6 @@ def collapse_themes(
         ...     aggressive=True, passes="auto", max_workers=8,
         ... )
     """
-    if not api_key:
-        raise ValueError("collapse_themes() needs an api_key for the LLM call.")
-
     mode = "merge" if aggressive else "unique"
 
     # The main (merge) phase runs on merge_model if given, else user_model. A separate
@@ -518,6 +530,20 @@ def collapse_themes(
     merge_name = merge_model or user_model
     merge_src = merge_model_source if merge_model else model_source
     merge_provider = detect_provider(merge_name, merge_src)
+
+    # A key is only needed for providers that bill one; the subscription/CLI
+    # backends and ollama run keyless.
+    _keyless = set(_SUBSCRIPTION_PROVIDERS) | {"ollama"}
+    _providers_used = {merge_provider}
+    if unique_model:
+        _providers_used.add(detect_provider(unique_model, unique_model_source))
+    if not api_key and not _providers_used <= _keyless:
+        raise ValueError(
+            "collapse_themes() needs an api_key for the LLM call. "
+            "(Not required for the claude-code/claude-agent/codex-agent "
+            "backends or ollama.)"
+        )
+
     client = UnifiedLLMClient(provider=merge_provider, api_key=api_key, model=merge_name)
 
     def _run(cl, items, md, p):
@@ -589,6 +615,7 @@ def collapse_themes(
         if top_n and len(items) > int(top_n):
             items = _select_top_n(client, _count_guidance(items, input_data),
                                   int(top_n), description, creativity, dedupe_threshold)
+        items = _as_label_list(items)
         if filename:
             pd.DataFrame({"category": items}).to_csv(filename, index=False)
             print(f"Collapsed categories saved to {filename}")
@@ -674,6 +701,7 @@ def collapse_themes(
         current = _select_top_n(client, _count_guidance(current, input_data),
                                 int(top_n), description, creativity, dedupe_threshold)
 
+    current = _as_label_list(current)
     if filename:
         pd.DataFrame({"category": current}).to_csv(filename, index=False)
         print(f"Collapsed categories saved to {filename}")

{cat_stack-2.5.0 → cat_stack-2.5.2}/src/catstack/explore.py

@@ -16,7 +16,7 @@ from .text_functions import explore_common_categories
 
 def explore(
     input_data,
-    api_key,
+    api_key=None,
     description="",
     max_categories=12,
     categories_per_chunk=10,
@@ -46,7 +46,9 @@ def explore(
 
     Args:
         input_data: List of text responses or pandas Series.
-        api_key (str): API key for the model provider.
+        api_key (str, optional): API key for the model provider. Not required
+            for subscription/CLI backends (claude-code, claude-agent,
+            codex-agent) or ollama.
         description (str): Description of the data context. Content-neutral —
             for survey responses this is the question that was asked; for
             documents or posts this describes what the content is about.

{cat_stack-2.5.0 → cat_stack-2.5.2}/src/catstack/extract.py

@@ -41,7 +41,7 @@ from .collapse_themes import collapse_themes
 
 def extract(
     input_data,
-    api_key,
+    api_key=None,
     input_type="auto",
     description="",
     survey_question=None,
@@ -79,7 +79,9 @@ def extract(
             - For text: list of text responses or pandas Series
             - For image: directory path, single file, or list of image paths
             - For pdf: directory path, single file, or list of PDF paths
-        api_key (str): API key for the model provider.
+        api_key (str, optional): API key for the model provider. Not required
+            for subscription/CLI backends (claude-code, claude-agent,
+            codex-agent) or ollama.
         input_type (str): Type of input data. Options:
             - "auto" (default): Auto-detect from file extensions
             - "text": Text responses

{cat_stack-2.5.0 → cat_stack-2.5.2}/src/catstack/image_functions.py

@@ -154,12 +154,6 @@ def image_multi_class(
             "(the text-only CLI shim). Use model_source='claude-agent' (the cat-claws "
             "subscription backend) or an API-key provider."
         )
-    if model_source == "codex-agent":
-        raise ValueError(
-            "Image classification is not yet supported with "
-            "model_source='codex-agent'. Use model_source='claude-agent' (the "
-            "multimodal subscription backend) or an API-key provider."
-        )
 
     image_files = _load_image_files(image_input)
 
@@ -677,15 +671,18 @@ Provide the final categorization in the same JSON format:"""
 
         return """{"1":"e"}""", "Max retries exceeded"
 
-    def _call_claude_agent_image(base_text, encoded, media_type):
-        """Image classification via the cat-claws multimodal adapter (Claude
-        subscription, no API key). Returns (reply, error) like _call_anthropic."""
+    def _call_agent_image(base_text, encoded, media_type):
+        """Image classification via a cat-claws multimodal adapter
+        (claude-agent or codex-agent: subscription login, no API key).
+        Returns (reply, error) like _call_anthropic."""
+        from ._providers import _AGENT_BACKENDS
+        adapter_name, install_hint = _AGENT_BACKENDS[model_source]
         try:
             from catclaws._adapters import get_adapter
         except ImportError:
-            return None, ("cat-claws is not installed. Run: pip install cat-stack[agent]")
+            return None, f"cat-claws is not installed. Run: {install_hint}"
         import asyncio
-        adapter = get_adapter("claude")
+        adapter = get_adapter(adapter_name)
         _system = ("You are an image classification engine. Follow the user's "
                    "instructions exactly and reply with only what they ask for.")
         try:
@@ -738,9 +735,11 @@ Provide the final categorization in the same JSON format:"""
             image_content = {"type": "image_url", "image_url": {"url": encoded_image, "detail": "high"}}
             return _call_mistral(prompt, step2_prompt, step3_prompt, step4_prompt, image_content)
 
-        elif model_source == "claude-agent":
+        elif model_source in ("claude-agent", "codex-agent"):
             media_type = f"image/{ext}" if ext else "image/jpeg"
-            return _call_claude_agent_image(base_prompt_text, encoded, media_type)
+            if media_type == "image/jpg":
+                media_type = "image/jpeg"
+            return _call_agent_image(base_prompt_text, encoded, media_type)
 
         else:
             raise ValueError("Unknown source! Choose from OpenAI, Anthropic, Perplexity, Google, xAI, Huggingface, or Mistral")

{cat_stack-2.5.0 → cat_stack-2.5.2}/src/catstack/pdf_functions.py

@@ -399,12 +399,6 @@ def pdf_multi_class(
             "(the text-only CLI shim). Use model_source='claude-agent' (the cat-claws "
             "subscription backend) or an API-key provider."
         )
-    if model_source == "codex-agent":
-        raise ValueError(
-            "PDF classification is not yet supported with "
-            "model_source='codex-agent'. Use model_source='claude-agent' (the "
-            "multimodal subscription backend) or an API-key provider."
-        )
 
     # Providers with native PDF support (only used in image/both modes)
     native_pdf_providers = {"anthropic", "google"}
@@ -1192,23 +1186,27 @@ Provide the final categorization in the same JSON format:"""
 
         return """{"1":"e"}""", "Max retries exceeded"
 
-    def _call_claude_agent_pdf(base_text, encoded_image):
-        """PDF-page classification via the cat-claws multimodal adapter. The page
-        is rendered to an image (PDF-as-images); Claude subscription, no API key.
-        Returns (reply, error)."""
+    def _call_agent_pdf(base_text, encoded_image):
+        """PDF-page classification via a cat-claws adapter (claude-agent or
+        codex-agent: subscription login, no API key). The page is rendered to
+        an image (PDF-as-images); encoded_image=None sends text only (mode
+        "text"). Returns (reply, error)."""
+        from ._providers import _AGENT_BACKENDS
+        adapter_name, install_hint = _AGENT_BACKENDS[model_source]
         try:
             from catclaws._adapters import get_adapter
         except ImportError:
-            return None, ("cat-claws is not installed. Run: pip install cat-stack[agent]")
+            return None, f"cat-claws is not installed. Run: {install_hint}"
         import asyncio
-        adapter = get_adapter("claude")
+        adapter = get_adapter(adapter_name)
         _system = ("You are a document-page classification engine. Follow the "
                    "user's instructions exactly and reply with only what they ask for.")
         try:
             reply, error = asyncio.run(adapter.one_shot(
                 base_text, system_prompt=_system, model=user_model,
                 thinking_budget=thinking_budget or 0,
-                images=[{"media_type": "image/png", "data": encoded_image}],
+                **({"images": [{"media_type": "image/png", "data": encoded_image}]}
+                   if encoded_image else {}),
             ))
             return (None, error) if error else (reply, None)
         except Exception as e:
@@ -1244,6 +1242,8 @@ Provide the final categorization in the same JSON format:"""
                 return _call_openai_text_only(base_prompt_text, step2_prompt, step3_prompt, step4_prompt)
             elif model_source == "mistral":
                 return _call_mistral_text_only(base_prompt_text, step2_prompt, step3_prompt, step4_prompt)
+            elif model_source in ("claude-agent", "codex-agent"):
+                return _call_agent_pdf(base_prompt_text, None)
             else:
                 raise ValueError(f"Unknown source! Choose from OpenAI, Anthropic, Perplexity, Google, xAI, Huggingface, or Mistral")
 
@@ -1293,12 +1293,12 @@ Provide the final categorization in the same JSON format:"""
             prompt_data = _build_prompt_google_pdf(encoded_pdf, base_prompt_text)
             return _call_google(prompt_data, step2_prompt, step3_prompt, step4_prompt, base_prompt_text)
 
-        elif model_source == "claude-agent":
+        elif model_source in ("claude-agent", "codex-agent"):
             image_bytes, is_valid = _extract_page_as_image_bytes(pdf_path, page_index)
             if not is_valid:
                 return None, "Failed to render PDF page to image"
             encoded_image = _encode_bytes_to_base64(image_bytes)
-            return _call_claude_agent_pdf(base_prompt_text, encoded_image)
+            return _call_agent_pdf(base_prompt_text, encoded_image)
 
         # Handle providers requiring image conversion
         else:

{cat_stack-2.5.0 → cat_stack-2.5.2}/src/catstack/text_functions_ensemble.py

@@ -3863,6 +3863,17 @@ multi_class_ensemble = classify_ensemble
 # Summarization helpers
 # =============================================================================
 
+def _format_item_errors(errors: dict, multi_model: bool) -> str:
+    """One row's failure reasons for the `error_message` column: the bare
+    message for a single model, "model: message" pairs for an ensemble.
+    Empty string when the row had no errors."""
+    if not errors:
+        return ""
+    if not multi_model:
+        return "; ".join(str(e) for e in errors.values())
+    return "; ".join(f"{m}: {e}" for m, e in errors.items())
+
+
 def _save_partial_summarize_results(all_results, model_configs, model_names, is_pdf_mode, filename, save_directory):
     """Save partial summarization results to CSV for safety/incremental saves."""
     rows = []
@@ -3895,6 +3906,7 @@ def _save_partial_summarize_results(all_results, model_configs, model_names, is_
             ) else "partial"
         else:
             row["processing_status"] = "success"
+        row["error_message"] = _format_item_errors(entry["errors"], len(model_configs) > 1)
 
         rows.append(row)
 
@@ -4511,6 +4523,9 @@ def summarize_ensemble(
 
             if error and error != "skipped":
                 still_failed.append((idx, model_name))
+                # Keep the latest reason so the output reports what the
+                # final attempt actually hit.
+                all_results[idx]["errors"][model_name] = error
             else:
                 # Update the stored result
                 all_results[idx]["model_results"][model_name] = json_result
@@ -4631,10 +4646,20 @@ def summarize_ensemble(
         else:
             row["processing_status"] = "success"
 
+        # Why a row failed: previously collected per item but never written
+        # out, so failures were silent (status "error", no reason).
+        row["error_message"] = _format_item_errors(entry["errors"], len(model_configs) > 1)
+
         rows.append(row)
 
     df = pd.DataFrame(rows)
 
+    n_failed = int((df["error_message"] != "").sum()) if "error_message" in df else 0
+    if n_failed:
+        first = df.loc[df["error_message"] != "", "error_message"].iloc[0]
+        print(f"\n[CatLLM] WARNING: {n_failed} of {len(df)} item(s) had errors "
+              f"(see the error_message column). First error: {first}")
+
     # Save to file if requested
     if filename:
         save_path = os.path.join(save_directory, filename) if save_directory else filename