cat-stack 2.5.0__tar.gz → 2.5.1__tar.gz

{cat_stack-2.5.0 → cat_stack-2.5.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cat-stack
-Version: 2.5.0
+Version: 2.5.1
 Summary: Domain-agnostic text, image, PDF, and DOCX classification engine powered by LLMs
 Project-URL: Documentation, https://github.com/chrissoria/cat-stack#readme
 Project-URL: Issues, https://github.com/chrissoria/cat-stack/issues
@@ -24,9 +24,9 @@ Requires-Dist: pandas
 Requires-Dist: requests
 Requires-Dist: tqdm
 Provides-Extra: agent
-Requires-Dist: cat-claws[claude]>=0.3.0; extra == 'agent'
+Requires-Dist: cat-claws[claude]>=0.3.1; extra == 'agent'
 Provides-Extra: codex-agent
-Requires-Dist: cat-claws[codex]>=0.3.0; extra == 'codex-agent'
+Requires-Dist: cat-claws[codex]>=0.3.1; extra == 'codex-agent'
 Provides-Extra: docx
 Requires-Dist: python-docx>=1.0.0; extra == 'docx'
 Provides-Extra: embeddings
@@ -138,7 +138,7 @@ cat.classify(
 ```
 
 ### `extract()`
-Discover categories from a corpus using LLM-driven exploration.
+Discover categories from a corpus using LLM-driven exploration. Since v2.5.0, text consolidation runs the full explore → `collapse_themes()` pipeline (`engine="collapse"`, the default): the entire raw label inventory reaches the semantic merge — Jaro-Winkler dedup, embedding pre-merge, quality-controlled LLM passes, then a count-guided reduction to at most `max_categories`. Pass `engine="legacy"` to reproduce pre-2.5 runs (single merge call over a truncated inventory). `collapse_kwargs` forwards options to `collapse_themes()`; `max_workers` parallelizes both extraction and consolidation.
 
 ```python
 cat.extract(
@@ -197,7 +197,7 @@ cat.collapse_themes(
 | Parameter | Default | Description |
 | --- | --- | --- |
 | `input_data` | — | List of category labels, or a frequency `Series`/`dict` (`label -> count`). |
-| `api_key` | `None` | API key for the LLM provider (required). |
+| `api_key` | `None` | API key for the LLM provider. Not required for subscription/CLI backends (`claude-code`, `claude-agent`, `codex-agent`) or `ollama`. |
 | `description` | `""` | The survey question or context, used in the merge prompt. |
 | `passes` | `1` | Number of merge iterations, or `"auto"` to iterate until the embedding-quality benchmark peaks. |
 | `max_passes` | `10` | Cap on iterations when `passes="auto"`. |
@@ -207,6 +207,8 @@ cat.collapse_themes(
 | `embedding_merge_threshold` | `0.92` | Cosine similarity at/above which labels are merged in the pre-LLM embedding step. `None`/`>=1.0` disables it. |
 | `shuffle` | `True` | Randomize order each pass so batch composition varies (improves convergence stability). |
 | `final_consolidation` | `0.82` | Cosine threshold for one greedy global embedding re-merge after all passes, collapsing cross-batch duplicates. Conservative by design (errs toward keeping categories). `False`/`None` skips it. |
+| `top_n` | `None` | If set, a final global LLM call consolidates the surviving list into at most N categories, guided by each label's generation count (frequent themes favored; overlapping labels merged rather than dropped). Guaranteed `<= top_n` via truncation plus a deterministic top-N-by-count fallback. |
+| `prune` | `False` | `True` = drop conceptual duplicates keeping one representative verbatim; never renames or merges merely-related labels. |
 | `user_model` | `"gpt-4o"` | Model for the merge phase. Use a capable model — small models can degenerate. |
 | `model_source` | `"auto"` | Provider for `user_model` (`"auto"`, `"openai"`, `"huggingface"`, …). |
 | `unique_model` | `None` | If set, run an initial extract-unique thinning phase on this (typically cheaper) model before the merge phase. `None` skips the phase (backward compatible). |
@@ -237,6 +239,8 @@ OpenAI, Anthropic, Google (Gemini), Mistral, Perplexity, xAI (Grok), HuggingFace
 
 All providers use the same `(model_name, provider, api_key)` tuple format. Provider is auto-detected from model name if omitted.
 
+**Subscription backends (no API key).** Three `model_source` values authenticate through a chat subscription instead of a metered key — leave `api_key` unset: `"claude-agent"` (Claude subscription via the Agent SDK; `pip install "cat-stack[agent]"`), `"claude-code"` (the Claude Code CLI, if installed — no extra needed), and `"codex-agent"` (ChatGPT subscription; `pip install "cat-stack[codex-agent]"`). Classification, extraction, exploration, and summarization all route through them.
+
 ## Features
 
 - **Automatic prompt optimization** (`prompt_tune`) — correct a small sample in a browser UI, and the system generates per-category instructions that improve accuracy

{cat_stack-2.5.0 → cat_stack-2.5.1}/README.md

@@ -97,7 +97,7 @@ cat.classify(
 ```
 
 ### `extract()`
-Discover categories from a corpus using LLM-driven exploration.
+Discover categories from a corpus using LLM-driven exploration. Since v2.5.0, text consolidation runs the full explore → `collapse_themes()` pipeline (`engine="collapse"`, the default): the entire raw label inventory reaches the semantic merge — Jaro-Winkler dedup, embedding pre-merge, quality-controlled LLM passes, then a count-guided reduction to at most `max_categories`. Pass `engine="legacy"` to reproduce pre-2.5 runs (single merge call over a truncated inventory). `collapse_kwargs` forwards options to `collapse_themes()`; `max_workers` parallelizes both extraction and consolidation.
 
 ```python
 cat.extract(
@@ -156,7 +156,7 @@ cat.collapse_themes(
 | Parameter | Default | Description |
 | --- | --- | --- |
 | `input_data` | — | List of category labels, or a frequency `Series`/`dict` (`label -> count`). |
-| `api_key` | `None` | API key for the LLM provider (required). |
+| `api_key` | `None` | API key for the LLM provider. Not required for subscription/CLI backends (`claude-code`, `claude-agent`, `codex-agent`) or `ollama`. |
 | `description` | `""` | The survey question or context, used in the merge prompt. |
 | `passes` | `1` | Number of merge iterations, or `"auto"` to iterate until the embedding-quality benchmark peaks. |
 | `max_passes` | `10` | Cap on iterations when `passes="auto"`. |
@@ -166,6 +166,8 @@ cat.collapse_themes(
 | `embedding_merge_threshold` | `0.92` | Cosine similarity at/above which labels are merged in the pre-LLM embedding step. `None`/`>=1.0` disables it. |
 | `shuffle` | `True` | Randomize order each pass so batch composition varies (improves convergence stability). |
 | `final_consolidation` | `0.82` | Cosine threshold for one greedy global embedding re-merge after all passes, collapsing cross-batch duplicates. Conservative by design (errs toward keeping categories). `False`/`None` skips it. |
+| `top_n` | `None` | If set, a final global LLM call consolidates the surviving list into at most N categories, guided by each label's generation count (frequent themes favored; overlapping labels merged rather than dropped). Guaranteed `<= top_n` via truncation plus a deterministic top-N-by-count fallback. |
+| `prune` | `False` | `True` = drop conceptual duplicates keeping one representative verbatim; never renames or merges merely-related labels. |
 | `user_model` | `"gpt-4o"` | Model for the merge phase. Use a capable model — small models can degenerate. |
 | `model_source` | `"auto"` | Provider for `user_model` (`"auto"`, `"openai"`, `"huggingface"`, …). |
 | `unique_model` | `None` | If set, run an initial extract-unique thinning phase on this (typically cheaper) model before the merge phase. `None` skips the phase (backward compatible). |
@@ -196,6 +198,8 @@ OpenAI, Anthropic, Google (Gemini), Mistral, Perplexity, xAI (Grok), HuggingFace
 
 All providers use the same `(model_name, provider, api_key)` tuple format. Provider is auto-detected from model name if omitted.
 
+**Subscription backends (no API key).** Three `model_source` values authenticate through a chat subscription instead of a metered key — leave `api_key` unset: `"claude-agent"` (Claude subscription via the Agent SDK; `pip install "cat-stack[agent]"`), `"claude-code"` (the Claude Code CLI, if installed — no extra needed), and `"codex-agent"` (ChatGPT subscription; `pip install "cat-stack[codex-agent]"`). Classification, extraction, exploration, and summarization all route through them.
+
 ## Features
 
 - **Automatic prompt optimization** (`prompt_tune`) — correct a small sample in a browser UI, and the system generates per-category instructions that improve accuracy

{cat_stack-2.5.0 → cat_stack-2.5.1}/pyproject.toml

@@ -42,8 +42,8 @@ embeddings = ["sentence-transformers>=2.2.0"]
 # `agent` keeps its historical meaning (the Claude backend) so every shipped
 # install hint stays true; `codex-agent` matches the provider string so the
 # error-message hint is copy-pasteable.
-agent = ["cat-claws[claude]>=0.3.0"]
-codex-agent = ["cat-claws[codex]>=0.3.0"]
+agent = ["cat-claws[claude]>=0.3.1"]
+codex-agent = ["cat-claws[codex]>=0.3.1"]
 
 [project.urls]
 Documentation = "https://github.com/chrissoria/cat-stack#readme"

{cat_stack-2.5.0 → cat_stack-2.5.1}/src/catstack/__about__.py

@@ -1,7 +1,7 @@
 # SPDX-FileCopyrightText: 2025-present Christopher Soria <chrissoria@berkeley.edu>
 #
 # SPDX-License-Identifier: GPL-3.0-or-later
-__version__ = "2.5.0"
+__version__ = "2.5.1"
 __author__ = "Chris Soria"
 __email__ = "chrissoria@berkeley.edu"
 __title__ = "cat-stack"

{cat_stack-2.5.0 → cat_stack-2.5.1}/src/catstack/collapse_themes.py

@@ -25,7 +25,7 @@ import numpy as np
 import pandas as pd
 from jellyfish import jaro_winkler_similarity
 
-from ._providers import UnifiedLLMClient, detect_provider
+from ._providers import UnifiedLLMClient, detect_provider, _SUBSCRIPTION_PROVIDERS
 from ._utils import _clean_label
 
 __all__ = [
@@ -301,6 +301,19 @@ def _count_guidance(current, input_data):
     return {lbl: max(int(c), orig.get(_norm_key(lbl), 0)) for lbl, c in cur.items()}
 
 
+def _as_label_list(x):
+    """Enforce the documented list[str] return contract at exit points.
+
+    Dict / Series / DataFrame inputs can reach a return untouched (e.g.
+    passes=0 with a top_n no-op), which would leak the input container —
+    and, via the filename write, save counts instead of labels."""
+    if isinstance(x, list):
+        return [str(v) for v in x]
+    if isinstance(x, dict):
+        return [str(k) for k in x]
+    return [str(k) for k in _to_counts(x)]
+
+
 def _to_counts(input_data):
     """Coerce the accepted input forms into a {category: count} dict."""
     if isinstance(input_data, pd.DataFrame):
@@ -425,7 +438,9 @@ def collapse_themes(
         input_data: Themes to collapse. list[str] (duplicates allowed), pandas
             Series, dict {category: count}, or DataFrame with "category"
             [and optional "count"] columns.
-        api_key (str): API key for the model provider.
+        api_key (str): API key for the model provider. Not required for
+            subscription/CLI backends (`claude-code`, `claude-agent`,
+            `codex-agent`) or `ollama`.
         description (str): Data/question context, injected into the prompt — e.g.
             the survey question the categories came from. Helps the model judge
             which distinctions matter.
@@ -505,9 +520,6 @@ def collapse_themes(
         ...     aggressive=True, passes="auto", max_workers=8,
         ... )
     """
-    if not api_key:
-        raise ValueError("collapse_themes() needs an api_key for the LLM call.")
-
     mode = "merge" if aggressive else "unique"
 
     # The main (merge) phase runs on merge_model if given, else user_model. A separate
@@ -518,6 +530,20 @@ def collapse_themes(
     merge_name = merge_model or user_model
     merge_src = merge_model_source if merge_model else model_source
     merge_provider = detect_provider(merge_name, merge_src)
+
+    # A key is only needed for providers that bill one; the subscription/CLI
+    # backends and ollama run keyless.
+    _keyless = set(_SUBSCRIPTION_PROVIDERS) | {"ollama"}
+    _providers_used = {merge_provider}
+    if unique_model:
+        _providers_used.add(detect_provider(unique_model, unique_model_source))
+    if not api_key and not _providers_used <= _keyless:
+        raise ValueError(
+            "collapse_themes() needs an api_key for the LLM call. "
+            "(Not required for the claude-code/claude-agent/codex-agent "
+            "backends or ollama.)"
+        )
+
     client = UnifiedLLMClient(provider=merge_provider, api_key=api_key, model=merge_name)
 
     def _run(cl, items, md, p):
@@ -589,6 +615,7 @@ def collapse_themes(
         if top_n and len(items) > int(top_n):
             items = _select_top_n(client, _count_guidance(items, input_data),
                                   int(top_n), description, creativity, dedupe_threshold)
+        items = _as_label_list(items)
         if filename:
             pd.DataFrame({"category": items}).to_csv(filename, index=False)
             print(f"Collapsed categories saved to {filename}")
@@ -674,6 +701,7 @@ def collapse_themes(
         current = _select_top_n(client, _count_guidance(current, input_data),
                                 int(top_n), description, creativity, dedupe_threshold)
 
+    current = _as_label_list(current)
     if filename:
         pd.DataFrame({"category": current}).to_csv(filename, index=False)
         print(f"Collapsed categories saved to {filename}")

{cat_stack-2.5.0 → cat_stack-2.5.1}/src/catstack/explore.py

@@ -16,7 +16,7 @@ from .text_functions import explore_common_categories
 
 def explore(
     input_data,
-    api_key,
+    api_key=None,
     description="",
     max_categories=12,
     categories_per_chunk=10,
@@ -46,7 +46,9 @@ def explore(
 
     Args:
         input_data: List of text responses or pandas Series.
-        api_key (str): API key for the model provider.
+        api_key (str, optional): API key for the model provider. Not required
+            for subscription/CLI backends (claude-code, claude-agent,
+            codex-agent) or ollama.
         description (str): Description of the data context. Content-neutral —
             for survey responses this is the question that was asked; for
             documents or posts this describes what the content is about.

{cat_stack-2.5.0 → cat_stack-2.5.1}/src/catstack/extract.py

@@ -41,7 +41,7 @@ from .collapse_themes import collapse_themes
 
 def extract(
     input_data,
-    api_key,
+    api_key=None,
     input_type="auto",
     description="",
     survey_question=None,
@@ -79,7 +79,9 @@ def extract(
             - For text: list of text responses or pandas Series
             - For image: directory path, single file, or list of image paths
             - For pdf: directory path, single file, or list of PDF paths
-        api_key (str): API key for the model provider.
+        api_key (str, optional): API key for the model provider. Not required
+            for subscription/CLI backends (claude-code, claude-agent,
+            codex-agent) or ollama.
         input_type (str): Type of input data. Options:
             - "auto" (default): Auto-detect from file extensions
             - "text": Text responses