gutenberg-sdk 0.1.0__tar.gz

gutenberg_sdk-0.1.0/.gitignore

@@ -0,0 +1,53 @@
+# Claude Code
+.claude/
+.claude-handoff.md
+CLAUDE_TODO.md
+
+# Python
+__pycache__/
+*.pyc
+*.pyo
+*.egg-info/
+.eggs/
+dist/
+build/
+*.egg
+
+# Virtual environments
+.venv/
+venv/
+
+# Environment
+.env
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# Node
+node_modules/
+.next/
+
+# OS
+.DS_Store
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+
+# Dev database seed (contains prod data, do NOT commit)
+scripts/dev-seed.sql.gz
+
+# Re-allow committed test fixtures (root .gitignore blocks *.parquet globally)
+!tests/fixtures/**/*.parquet
+!gutenberg-sdk/examples/**/*.parquet
+
+# Local dev env-state backups (worktree-dev.sh writes these — never commit)
+.env.*-backup
+.env.prePlanB
+# Ad-hoc backups (cp .env .env.bak-<date>, .env.<thing>-snapshot-<date>, etc.)
+.env.bak*
+.env.*-snapshot*

gutenberg_sdk-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Gutenberg PBC
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

gutenberg_sdk-0.1.0/PKG-INFO

@@ -0,0 +1,116 @@
+Metadata-Version: 2.4
+Name: gutenberg-sdk
+Version: 0.1.0
+Summary: Python SDK for the Gutenberg SAE Activation API
+Project-URL: Homepage, https://gutenberg.ai
+Project-URL: Console, https://console.gutenberg.ai
+Project-URL: Documentation, https://console.gutenberg.ai/d/docs
+Project-URL: Repository, https://github.com/gutenbergpbc/code
+Author: Gutenberg PBC
+License: MIT
+License-File: LICENSE
+Keywords: activations,interpretability,llm,mechanistic-interpretability,observability,sae
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: httpx>=0.28
+Requires-Dist: pydantic>=2.0
+Requires-Dist: tqdm>=4.66
+Provides-Extra: pandas
+Requires-Dist: pandas>=2.0; extra == 'pandas'
+Requires-Dist: pyarrow>=14; extra == 'pandas'
+Description-Content-Type: text/markdown
+
+# gutenberg-sdk
+
+Python SDK for the **Gutenberg** SAE activation API — interpretability-based
+observability for language models. Upload text, read it through a sparse
+autoencoder (SAE) feature dictionary, and find the features that separate any
+two classes of documents.
+
+- **Docs:** https://console.gutenberg.ai/d/docs
+- **Console:** https://console.gutenberg.ai
+- **Get an API key:** https://console.gutenberg.ai/d/keys
+
+## Install
+
+```bash
+uv add gutenberg-sdk          # or: pip install gutenberg-sdk
+```
+
+The distribution is `gutenberg-sdk`; the import is `gutenberg`:
+
+```python
+from gutenberg import gutenberg
+```
+
+Optional `pandas` extra (for `load_activations_df` and parquet helpers):
+
+```bash
+uv add "gutenberg-sdk[pandas]"
+```
+
+## Quickstart
+
+```python
+from gutenberg import gutenberg
+
+client = gutenberg(api_key="gtn_...")   # or set GUTENBERG_API_KEY
+
+# 1. upload a parquet dataset (text + a binary target column)
+dataset = client.datasets.upload("examples/simple_binary_features_extraction_100.parquet")
+
+# 2. launch hosted SAE feature extraction
+job = client.jobs.create(
+    dataset_id=dataset.dataset_id,
+    model_id="google/gemma-3-27b-it",
+    sae_id="layer_31_width_262k_l0_medium",
+)
+job = client.jobs.wait(job.job_id)
+
+# 3. score every feature against the target with AUROC
+exp = client.experiments.create(
+    job_id=job.job_id,
+    target_column="is_ai",
+    target_column_type="binary",
+    positive_value="1",
+    scoring_method="auroc",
+)
+exp = client.experiments.wait(exp.experiment_id)
+
+# 4. read back ranked features and token-level examples
+for feature in client.experiments.features(exp.experiment_id)[:10]:
+    print(feature.rank, feature.feature_id, feature.score)
+```
+
+The full runnable script lives in
+[`examples/simple_binary_features_extraction.py`](examples/simple_binary_features_extraction.py),
+with a companion 100-row parquet. On production the whole flow runs in a couple
+of minutes. See [`docs/getting-started.md`](docs/getting-started.md) for the
+walkthrough, SAE selection guidance, and how token examples are served.
+
+> The bundled example is a **curated showcase**, not a benchmark — its 50 AI
+> passages were picked to exhibit a handful of recognizable AI-writing features,
+> so those features separate the two classes near-perfectly there. Run it on
+> your own data to see a realistic ranking.
+
+## API surface
+
+A single `gutenberg(...)` client with namespaced resources: `datasets`,
+`jobs`, `experiments`, `aggregations`, `autointerp`, `meta_autointerp`,
+`subsets`, plus the sync helpers `activations()`, `interpret()`, `models()`,
+and `saes()`.
+
+## License
+
+MIT

gutenberg_sdk-0.1.0/README.md

@@ -0,0 +1,83 @@
+# gutenberg-sdk
+
+Python SDK for the **Gutenberg** SAE activation API — interpretability-based
+observability for language models. Upload text, read it through a sparse
+autoencoder (SAE) feature dictionary, and find the features that separate any
+two classes of documents.
+
+- **Docs:** https://console.gutenberg.ai/d/docs
+- **Console:** https://console.gutenberg.ai
+- **Get an API key:** https://console.gutenberg.ai/d/keys
+
+## Install
+
+```bash
+uv add gutenberg-sdk          # or: pip install gutenberg-sdk
+```
+
+The distribution is `gutenberg-sdk`; the import is `gutenberg`:
+
+```python
+from gutenberg import gutenberg
+```
+
+Optional `pandas` extra (for `load_activations_df` and parquet helpers):
+
+```bash
+uv add "gutenberg-sdk[pandas]"
+```
+
+## Quickstart
+
+```python
+from gutenberg import gutenberg
+
+client = gutenberg(api_key="gtn_...")   # or set GUTENBERG_API_KEY
+
+# 1. upload a parquet dataset (text + a binary target column)
+dataset = client.datasets.upload("examples/simple_binary_features_extraction_100.parquet")
+
+# 2. launch hosted SAE feature extraction
+job = client.jobs.create(
+    dataset_id=dataset.dataset_id,
+    model_id="google/gemma-3-27b-it",
+    sae_id="layer_31_width_262k_l0_medium",
+)
+job = client.jobs.wait(job.job_id)
+
+# 3. score every feature against the target with AUROC
+exp = client.experiments.create(
+    job_id=job.job_id,
+    target_column="is_ai",
+    target_column_type="binary",
+    positive_value="1",
+    scoring_method="auroc",
+)
+exp = client.experiments.wait(exp.experiment_id)
+
+# 4. read back ranked features and token-level examples
+for feature in client.experiments.features(exp.experiment_id)[:10]:
+    print(feature.rank, feature.feature_id, feature.score)
+```
+
+The full runnable script lives in
+[`examples/simple_binary_features_extraction.py`](examples/simple_binary_features_extraction.py),
+with a companion 100-row parquet. On production the whole flow runs in a couple
+of minutes. See [`docs/getting-started.md`](docs/getting-started.md) for the
+walkthrough, SAE selection guidance, and how token examples are served.
+
+> The bundled example is a **curated showcase**, not a benchmark — its 50 AI
+> passages were picked to exhibit a handful of recognizable AI-writing features,
+> so those features separate the two classes near-perfectly there. Run it on
+> your own data to see a realistic ranking.
+
+## API surface
+
+A single `gutenberg(...)` client with namespaced resources: `datasets`,
+`jobs`, `experiments`, `aggregations`, `autointerp`, `meta_autointerp`,
+`subsets`, plus the sync helpers `activations()`, `interpret()`, `models()`,
+and `saes()`.
+
+## License
+
+MIT

gutenberg_sdk-0.1.0/docs/getting-started.md

@@ -0,0 +1,106 @@
+# Getting Started
+
+First, obtain an API key at https://console.gutenberg.ai/d/keys.
+
+Once the package is published, install it with:
+
+```bash
+uv add gutenberg-sdk        # or: pip install gutenberg-sdk
+```
+
+> Until the first PyPI release lands, install from source (from the repo root):
+> `uv pip install ./gutenberg-sdk` (or `pip install -e gutenberg-sdk`). The
+> distribution is `gutenberg-sdk`; the import is `from gutenberg import gutenberg`.
+
+## Running your first experiment
+
+Open [simple_binary_features_extraction.py](../examples/simple_binary_features_extraction.py) complete binary feature-extraction experiment with the Gutenberg SDK.
+
+This example mirrors the public Pangram EditLens "human vs AI" experiment, using
+the companion 100-row [parquet](../examples/simple_binary_features_extraction_100.parquet): 50
+human-written rows and 50 AI rows (ai_generated + ai_edited). The target is is_ai.
+
+> The same curated 100-row dataset is also published as a HuggingFace dataset,
+> `gutenbergpbc/pangram-editlens-100` (private for now, pending the upstream
+> Pangram EditLens public-use grant).
+
+The script uses the public SDK surface to:
+  1. upload a parquet dataset
+  2. launch hosted SAE feature extraction
+  3. create a binary AUROC experiment over the extracted features
+  4. read back ranked features and token examples
+
+This quickstart dataset is intentionally tiny (100 rows), so the whole flow runs
+in a couple of minutes and feature browsing is live and fast the moment the
+experiment completes.
+
+> Note: this is a **curated showcase**, not a benchmark — the 50 AI passages were
+> picked to exhibit a handful of recognizable AI-writing features, so those
+> features separate the two classes near-perfectly here. On a representative
+> human-vs-AI sample the same features score lower and other features lead; run it
+> on your own data to see a realistic ranking.
+
+### Choosing an SAE
+
+A `(model, SAE)` pair determines which feature dictionary your text is read
+through. This quickstart pins the **layer-31** SAE
+(`layer_31_width_262k_l0_medium`) — the one the Pangram EditLens detector was
+validated on (AUROC ~0.853), so the human-vs-AI result reproduces here. We
+recommend layer-31 when replicating Pangram-style detection.
+
+For your own datasets the **platform default is layer-41**
+(`layer_41_width_262k_l0_big`) — omit `sae_id` and the server resolves it. To
+choose explicitly, pass `sae_id=` to `client.jobs.create(...)` (the example
+exposes this as `--sae-id`), and list what's available for a model with:
+
+```python
+client.saes("google/gemma-3-27b-it")   # -> [{sae_id, is_default, ...}, ...]
+```
+
+After running the script, you should see something like the following in the console:
+
+```
+Console URL: https://console.gutenberg.ai/d/datasets/7d8a0ee4-d73b-4e2f-8ab8-bec3e1c5d7da/exp/24d3590c-e0b6-486e-b0cf-8050fd51f383
+```
+
+(The URL above is the public example experiment — feel free to open it to follow
+along. Its top features are recognizable AI tells: "delve", "it's not X, it's Y",
+overformatting, formulaic conclusions, and dense formal description.)
+
+## Viewing features
+
+List of all features, ordered by absolute value of `score` as defined in the script above (`auroc` here). Click question mark to view keyboard shortcuts, or view experiment metadata or search for a specific feature or explanation.
+
+### How token examples are served
+
+Examples are **cache-first with an automatic on-the-fly fallback**, so a feature's
+activating windows always come back without you running any manual aggregation
+step. The response's `pick_semantics` field tells you which path served it:
+
+- `v2` — the calibrated pre-built cache (the `max` / `p95` / `stratified` / `random`
+  classes). This is the default for the features you'll browse first.
+- `fallback_topn` — a top-by-activation rebuild, served live for features not yet
+  in the cache.
+
+For a small dataset like this quickstart, every feature is served fast and
+calibrated. For large datasets, the complete calibrated cache is built in the
+background after extraction (so the experiment viewer and autointerp come up
+immediately); until it finishes, less-common features are served via the fast
+`fallback_topn` path.
+
+The example-selection strategy is **`max`** by default — the feature's strongest
+activating examples (exact top-K by activation value). The full vocabulary is
+`[max, p95, stratified, random]`: `p95` is a random sample of firings at/above the
+95th-percentile activation, `stratified` anchors across the activation quartiles,
+and `random` samples uniformly. (`top_k` / `strongest` are accepted as deprecated
+aliases for `p95`.)
+
+![feature_view_1.png](./feature_view_1.png)
+
+Detailed feature view, here you can see the score on various subsets of activating examples, auto-generated explanations ("autointerp") for the feature, or you can add your own custom label. If you have any meta-autointerp runs, they will also be linked here.
+
+![feature_view_2.png](./feature_view_2.png)
+
+## Meta-autointerp runs
+
+TODO: Add UI screenshots

gutenberg_sdk-0.1.0/examples/simple_binary_features_extraction.py

@@ -0,0 +1,212 @@
+"""Run a complete binary feature-extraction experiment with the Gutenberg SDK.
+
+This example mirrors the public Pangram EditLens "human vs AI" experiment, but
+uses the companion 100-row parquet in this directory: 50 human-written rows and
+50 AI rows (ai_generated + ai_edited). The target column is is_ai.
+
+The script uses only the public SDK surface:
+  1. upload a parquet dataset
+  2. launch hosted SAE feature extraction (layer-31 SAE — the one the Pangram
+     detector was validated on; the platform default is layer-41)
+  3. create a binary AUROC experiment over the extracted features
+  4. read back ranked features and token examples
+
+Running it on production starts hosted compute and may also trigger any
+server-side completion hooks configured for the environment.
+
+Usage:
+    export GUTENBERG_API_KEY=gtn_...
+    python examples/simple_binary_features_extraction.py
+
+For local repo use, the script also loads GUTENBERG_API_KEY from the nearest
+.env file if it is not already set in the environment.
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+import sys
+from pathlib import Path
+
+from gutenberg import gutenberg
+
+
+DEFAULT_DATASET_PATH = Path(__file__).with_name(
+    "simple_binary_features_extraction_100.parquet"
+)
+DEFAULT_MODEL_ID = "google/gemma-3-27b-it"
+# This quickstart runs the layer-31 SAE — the one the Pangram EditLens detector
+# was validated on (AUROC ~0.853), so the human-vs-AI result is reproducible here.
+# The platform default for new datasets is the layer-41 SAE
+# (layer_41_width_262k_l0_big); pass --sae-id to override either way, or call
+# client.saes("google/gemma-3-27b-it") to list what's available.
+DEFAULT_SAE_ID = "layer_31_width_262k_l0_medium"
+DEFAULT_BASE_URL = "https://api.gutenberg.ai"
+
+
+def log(message: str = "") -> None:
+    print(message, flush=True)
+
+
+def _strip_inline_comment(value: str) -> str:
+    quote: str | None = None
+    chars: list[str] = []
+    for i, ch in enumerate(value):
+        if ch in ("'", '"'):
+            if quote == ch:
+                quote = None
+            elif quote is None:
+                quote = ch
+            chars.append(ch)
+        elif ch == "#" and quote is None and (i == 0 or value[i - 1].isspace()):
+            break
+        else:
+            chars.append(ch)
+    return "".join(chars).strip().strip('"').strip("'")
+
+
+def _load_nearest_dotenv(start: Path) -> None:
+    """Load missing env vars from the nearest .env without extra dependencies."""
+    for directory in [start, *start.parents]:
+        dotenv = directory / ".env"
+        if not dotenv.exists():
+            continue
+        for raw in dotenv.read_text().splitlines():
+            line = raw.strip()
+            if not line or line.startswith("#") or "=" not in line:
+                continue
+            key, value = line.split("=", 1)
+            os.environ.setdefault(key.strip(), _strip_inline_comment(value))
+        return
+
+
+def _die_if_failed(kind: str, obj) -> None:
+    if obj.status == "failed":
+        error = getattr(obj, "error", None) or "unknown error"
+        raise RuntimeError(f"{kind} {obj.status}: {error}")
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description=(
+            "Upload the 100-row EditLens binary sample, run hosted SAE extraction, "
+            "then score features against is_ai with AUROC."
+        )
+    )
+    parser.add_argument(
+        "--dataset-path",
+        type=Path,
+        default=DEFAULT_DATASET_PATH,
+        help="Parquet file to upload.",
+    )
+    parser.add_argument(
+        "--base-url",
+        default=os.environ.get("GUTENBERG_API_URL", DEFAULT_BASE_URL),
+        help="Gutenberg API base URL.",
+    )
+    parser.add_argument("--model-id", default=DEFAULT_MODEL_ID)
+    parser.add_argument("--sae-id", default=DEFAULT_SAE_ID)
+    parser.add_argument("--top-k", type=int, default=100)
+    parser.add_argument("--num-workers", type=int, default=1)
+    parser.add_argument("--timeout-seconds", type=int, default=21_600)
+    parser.add_argument("--poll-interval", type=float, default=15.0)
+    parser.add_argument("--feature-limit", type=int, default=10)
+    return parser.parse_args()
+
+
+def main() -> int:
+    _load_nearest_dotenv(Path(__file__).resolve().parent)
+    args = parse_args()
+
+    api_key = os.environ.get("GUTENBERG_API_KEY")
+    if not api_key:
+        raise RuntimeError(
+            "Set GUTENBERG_API_KEY or put it in a parent .env file."
+        )
+    if not args.dataset_path.exists():
+        raise FileNotFoundError(args.dataset_path)
+
+    with gutenberg(api_key=api_key, base_url=args.base_url, timeout=300) as client:
+        log(f"Uploading dataset: {args.dataset_path}")
+        dataset = client.datasets.upload(args.dataset_path)
+        dataset = client.datasets.update(
+            dataset.dataset_id,
+            name="SDK example: EditLens human vs AI (100 rows)",
+            description=(
+                "100-row Pangram EditLens demo: 50 human-written and 50 AI rows "
+                "(ai_generated + ai_edited). Target is is_ai."
+            ),
+        )
+        log(f"Dataset: {dataset.dataset_id}")
+        log("Columns: " + ", ".join(client.datasets.columns(dataset.dataset_id)))
+
+        log(f"Starting extraction: {args.model_id} / {args.sae_id}")
+        job = client.jobs.create(
+            dataset_id=dataset.dataset_id,
+            model_id=args.model_id,
+            sae_id=args.sae_id,
+            top_k=args.top_k,
+            num_workers=args.num_workers,
+            timeout_seconds=args.timeout_seconds,
+        )
+        log(f"Job: {job.job_id} ({job.status})")
+        job = client.jobs.wait(job.job_id, poll_interval=args.poll_interval)
+        _die_if_failed("job", job)
+        log(f"Job completed: {job.job_id}")
+
+        log("Creating binary AUROC experiment over target column is_ai")
+        exp = client.experiments.create(
+            job_id=job.job_id,
+            name="SDK example: human vs AI",
+            target_column="is_ai",
+            target_column_type="binary",
+            positive_value="1",
+            scoring_method="auroc",
+        )
+        log(f"Experiment: {exp.experiment_id} ({exp.status})")
+        exp = client.experiments.wait(
+            exp.experiment_id,
+            poll_interval=args.poll_interval,
+        )
+        _die_if_failed("experiment", exp)
+        log(f"Experiment completed: {exp.experiment_id}")
+
+        features = client.experiments.features(exp.experiment_id)
+        if not features:
+            raise RuntimeError("Experiment completed but returned no features.")
+
+        log("\nTop features")
+        for feature in features[: args.feature_limit]:
+            log(
+                f"#{feature.rank:>3} feature={feature.feature_id:<8} "
+                f"score={feature.score:.4f} p={feature.p_value:.3g} "
+                f"n={feature.n_samples}"
+            )
+
+        top = features[0]
+        examples = client.experiments.examples(exp.experiment_id, top.feature_id)
+        log(f"\nExamples for top feature {top.feature_id}: {len(examples)}")
+        for example in examples[:3]:
+            max_value = max(example.values) if example.values else 0.0
+            log(
+                f"- sample_id={example.sample_id} tokens={len(example.tokens)} "
+                f"max_activation={max_value:.4f}"
+            )
+
+        if args.base_url.rstrip("/") == DEFAULT_BASE_URL:
+            log(
+                "\nConsole URL: "
+                f"https://console.gutenberg.ai/d/datasets/"
+                f"{dataset.dataset_id}/exp/{exp.experiment_id}"
+            )
+
+    return 0
+
+
+if __name__ == "__main__":
+    try:
+        raise SystemExit(main())
+    except KeyboardInterrupt:
+        print("\nInterrupted.", file=sys.stderr)
+        raise SystemExit(130)

gutenberg_sdk-0.1.0/gutenberg/__init__.py

@@ -0,0 +1,10 @@
+from importlib.metadata import PackageNotFoundError, version
+
+from gutenberg.client import gutenberg
+
+try:
+    __version__ = version("gutenberg-sdk")
+except PackageNotFoundError:  # editable / source checkout without metadata
+    __version__ = "0.0.0+unknown"
+
+__all__ = ["gutenberg", "__version__"]