loom-batch 0.1.0__tar.gz

loom_batch-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jan Nehring
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

loom_batch-0.1.0/PKG-INFO

@@ -0,0 +1,377 @@
+Metadata-Version: 2.4
+Name: loom-batch
+Version: 0.1.0
+Summary: Weave batch LLM jobs across OpenAI, Anthropic, and Google.
+Author: Jan Nehring
+License: MIT
+Project-URL: Homepage, https://github.com/jannehring/loom
+Project-URL: Repository, https://github.com/jannehring/loom
+Project-URL: Issues, https://github.com/jannehring/loom/issues
+Keywords: llm,batch,openai,anthropic,gemini,cli
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Utilities
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: typer>=0.12
+Requires-Dist: pydantic>=2
+Requires-Dist: pandas>=2
+Requires-Dist: python-dotenv>=1
+Requires-Dist: openai>=1.40
+Requires-Dist: anthropic>=0.34
+Requires-Dist: google-genai>=0.3
+Requires-Dist: rich>=13
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: pytest-cov>=4; extra == "dev"
+Requires-Dist: build>=1; extra == "dev"
+Dynamic: license-file
+
+# Loom: LLM Batch Processing Made Easy
+
+> Weave LLM jobs across OpenAI, Anthropic, Google, and OpenRouter — in batch or live.
+
+## 1. Introduction
+
+Loom is a small Python CLI for running a dataset of prompts (JSON or CSV) through an LLM and merging the responses back into the original file. It speaks two modes:
+
+- **Batch** (`loom run`, default): submits the dataset to the provider's batch API, persists the batch id locally, and later you call `loom fetch` to download and merge results. Cheap (50% off on OpenAI / Anthropic) but asynchronous — can take up to 24 hours.
+- **Sequential** (`loom run --sync`): calls the chat-completion endpoint per prompt with a concurrent worker pool, writes the output file immediately, and uses an on-disk response cache.
+
+It also ships a `loom tokens` command that uses each provider's token-counting API where available.
+
+### Supported providers
+
+| Provider | Batch (`loom run`) | Sequential (`loom run --sync`) | Token counter (`loom tokens`) |
+| --- | --- | --- | --- |
+| OpenAI | ✓ | ✓ | ✗ — no remote API |
+| Anthropic | ✓ | ✓ | ✓ |
+| Google (Gemini) | ✓ | ✓ | ✓ |
+| OpenRouter | ✗ | ✓ | ✗ — no remote API |
+
+### Table of contents
+
+- [1. Introduction](#1-introduction)
+- [2. Getting started](#2-getting-started)
+  - [Installation](#installation)
+  - [Preparing the data](#preparing-the-data)
+  - [Submit a batch request](#submit-a-batch-request)
+- [3. Usage](#3-usage)
+  - [Command-line reference](#command-line-reference)
+  - [Batch vs sequential](#batch-vs-sequential)
+  - [Storing API keys](#storing-api-keys)
+  - [Caching](#caching)
+  - [Token counter](#token-counter)
+  - [Where Loom stores state](#where-loom-stores-state)
+  - [Troubleshooting](#troubleshooting)
+- [4. Developer instructions](#4-developer-instructions)
+- [5. License](#5-license)
+
+## 2. Getting started
+
+### Installation
+
+```bash
+pip install loom-batch
+```
+
+The PyPI package is `loom-batch` (the name `loom` was taken); the CLI command is `loom`.
+
+From source, for hacking or running tests:
+
+```bash
+git clone https://github.com/jannehring/loom
+cd loom
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+> **Tip:** if you create the venv with `uv venv`, `pip` is not installed inside it. Use `uv pip install -e ".[dev]"` instead, or recreate the venv with stdlib `python -m venv` (see [Troubleshooting](#troubleshooting)).
+
+### Preparing the data
+
+Loom accepts two input formats — plain or **gzip-compressed** (`.json.gz`, `.csv.gz`). Compressed inputs are decompressed transparently; outputs are always written uncompressed (`.json` / `.csv`).
+
+**JSON** — a list of `{id, prompt}` objects. The `id` is reused as the row key in the merged output.
+
+```json
+[
+  {"id": "task-001", "prompt": "Summarize the plot of Hamlet in one sentence."},
+  {"id": "task-002", "prompt": "Translate 'Good morning' to French."}
+]
+```
+
+**CSV** — any schema; you tell Loom which column holds the prompt with `--col`. All original columns are preserved; a new `llm_response` column is appended.
+
+```csv
+id,text,priority
+1,"Explain quantum physics in one paragraph",low
+2,"Write a haiku about rust",high
+```
+
+### Submit a batch request
+
+Minimal end-to-end run, passing the API key inline (see [Storing API keys](#storing-api-keys) for cleaner options):
+
+```bash
+loom run --file prompts.json \
+         --provider openai \
+         --model gpt-4o-mini \
+         --api-key sk-...
+# -> Batch submitted. id=batch_abc123 provider=openai
+
+# ...minutes or hours later...
+loom fetch              # --all is the default; fetches every pending batch
+# -> Fabric complete. id=batch_abc123 -> prompts_results_openai_gpt-4o-mini.json
+```
+
+The output is written next to the input as `<name>_results_<provider>_<model>.<ext>`. Forward slashes and other unsafe characters in the model id are replaced with underscores (e.g. `openai/gpt-4o-mini` → `openai_gpt-4o-mini`). For gzipped inputs the `.gz` is dropped — `data.csv.gz` → `data_results_<provider>_<model>.csv`. Override the path entirely with `--output`.
+
+## 3. Usage
+
+### Command-line reference
+
+#### `loom run`
+
+Submit a dataset as a batch job (default) or run it synchronously with `--sync`.
+
+| Flag | Default | Description |
+| --- | --- | --- |
+| `--file`, `-f` | _required_ | Input `.json`, `.csv`, `.json.gz`, or `.csv.gz`. |
+| `--provider`, `-p` | _required_ | `openai`, `anthropic`, `google`, or `openrouter`. |
+| `--model`, `-m` | _required_ | Provider-specific model id (e.g. `gpt-4o-mini`, `claude-3-5-sonnet-latest`, `gemini-2.0-flash`, `openai/gpt-4o-mini`). |
+| `--col`, `-c` | — | Prompt column name (required for CSV). |
+| `--api-key` | env / `.env` | Override the resolved API key for this run. |
+| `--output`, `-o` | `<input>_results_<provider>_<model>.<ext>` | Custom output file path. |
+| `--sync` / `--batch` | `--batch` | `--sync` calls the provider per prompt and writes the output immediately. `--batch` uses the provider's batch API. |
+| `--workers`, `-w` | `8` | Concurrent workers in `--sync` mode. |
+| `--no-cache` | off | Disable the on-disk response cache (`--sync` only). |
+| `--force` | off | Overwrite an existing output file without prompting (`--sync` only). |
+| `--with-meta` | off | Add `llm_provider` and `llm_model` columns (CSV) or fields (JSON) to the output, alongside `llm_response`. |
+
+OpenRouter has no batch API; using `--provider openrouter` without `--sync` exits with a helpful error.
+
+#### `loom fetch`
+
+Poll the provider, download results, merge into the output file.
+
+| Flag | Default | Description |
+| --- | --- | --- |
+| `--id`, `-i` | — | Fetch a single batch by id. If set, implies `--no-all`. |
+| `--all` / `--no-all`, `-a` | `--all` | Process every pending batch under `~/.loom/batches/`. This is the default — `loom fetch` with no args walks all batches. |
+| `--api-key` | env / `.env` | Override the resolved API key. |
+| `--keep`, `-k` | off | Keep the metadata file in `~/.loom/batches/` after a successful fetch (default: delete it). |
+| `--force` | off | Overwrite existing output files without prompting. |
+
+For pending batches, `loom fetch` prints the current status and a one-sentence explanation. The full set of possible statuses:
+
+| Status | Meaning |
+| --- | --- |
+| `validating` | Provider has accepted the batch and is queueing/preparing it; no work has started yet. |
+| `in_progress` | Provider is actively running the prompts; check back later. |
+| `completed` | All prompts finished and results were downloaded — the merged output file has been written. |
+| `failed` | Provider reported the batch as failed; results are not available. |
+| `expired` | Batch exceeded the provider's time limit (typically 24h) before completing. |
+| `cancelled` | Batch was cancelled — either by you on the provider's dashboard, or by the provider itself. |
+| `unknown` | The last fetch attempt raised an error (invalid id, auth failure, network glitch, or an API response Loom doesn't recognise). Re-run `loom fetch` to retry; if it persists, inspect the metadata file under `~/.loom/batches/`. |
+
+`validating` and `in_progress` are the only non-terminal states — `loom fetch` will pick the batch up again on the next run. The other states are terminal: `completed` means the output file is on disk, and `failed` / `expired` / `cancelled` mean no merge happened.
+
+#### `loom list`
+
+List every batch known to Loom, with last-seen status, model, and source file. No flags.
+
+#### `loom tokens`
+
+Count input tokens for every prompt using the provider's token-counting API. See [Token counter](#token-counter).
+
+| Flag | Default | Description |
+| --- | --- | --- |
+| `--file`, `-f` | _required_ | Input `.json`, `.csv`, `.json.gz`, or `.csv.gz`. |
+| `--provider`, `-p` | _required_ | `openai`, `anthropic`, `google`, or `openrouter`. |
+| `--model`, `-m` | _required_ | Provider-specific model id. |
+| `--col`, `-c` | — | Prompt column name (required for CSV). |
+| `--api-key` | env / `.env` | Override the resolved API key. |
+| `--workers`, `-w` | `8` | Concurrent workers. |
+
+#### `loom cache clear`
+
+Delete every cached response under `~/.loom/cache/`. See [Caching](#caching).
+
+| Flag | Default | Description |
+| --- | --- | --- |
+| `--yes`, `-y` | off | Skip the confirmation prompt. |
+
+### Batch vs sequential
+
+| | `loom run` (batch, default) | `loom run --sync` (sequential) |
+| --- | --- | --- |
+| Latency | Up to 24h | Real-time |
+| Pricing (OpenAI, Anthropic) | 50% off | Standard |
+| Steps | `run` → wait → `fetch` | Single command |
+| Cache | n/a | On-disk, on by default |
+| OpenRouter | ✗ | ✓ (only mode) |
+| State on disk | `~/.loom/batches/` | None (cache only) |
+
+Pick **batch** when you have a large dataset and don't care about wall-clock time. Pick **sync** when you want results now, or when the provider has no batch API (OpenRouter).
+
+### Storing API keys
+
+Loom resolves keys in this order: **`--api-key` flag → environment variable → `.env` file** in the current working directory (loaded via `python-dotenv`, does not overwrite existing env vars).
+
+Recognised environment variables:
+
+```ini
+OPENAI_API_KEY=sk-...
+ANTHROPIC_API_KEY=sk-ant-...
+GOOGLE_API_KEY=...
+OPENROUTER_API_KEY=sk-or-...
+```
+
+A `.env` in the working directory is the friction-free option for daily use; `--api-key` is handy for one-offs or shared workstations.
+
+### Caching
+
+In `--sync` mode, Loom caches every response under `~/.loom/cache/`. The cache key is `sha256("<provider>|<model>|<prompt>")`, so changing any of those misses the cache. There is no TTL or eviction — the cache grows monotonically until you clear it.
+
+```bash
+loom run --sync -p openai -m gpt-4o-mini -f data.csv -c text   # first run: API calls
+loom run --sync -p openai -m gpt-4o-mini -f data.csv -c text   # second run: 100% cache hits
+loom run --sync -p openai -m gpt-4o-mini -f data.csv -c text --no-cache  # bypass
+loom cache clear                                                # wipe ~/.loom/cache/
+```
+
+`loom run --sync` reports cache hits live in its progress bar.
+
+### Token counter
+
+```bash
+loom tokens --file prompts.json --provider anthropic --model claude-3-5-sonnet-latest
+# Counting tokens ████████░░░░  340/1000  est_total≈36,210  errors=0  0:01:12  eta 0:02:35
+# -> Total input tokens: 12,345 across 100 prompts (provider=anthropic, model=claude-3-5-sonnet-latest, errors=0)
+```
+
+`loom tokens` calls each provider's official count-tokens endpoint, one prompt at a time, with a concurrent worker pool. The live progress bar shows:
+
+- `done/total` prompts processed,
+- `est_total` — running estimate of the final input-token count, computed as the mean tokens-per-prompt-so-far multiplied by `total` (refines as more prompts complete),
+- `errors`,
+- elapsed time and `eta` (estimated time remaining, based on the current rate).
+
+| Provider | Endpoint | Available |
+| --- | --- | --- |
+| Anthropic | `client.messages.count_tokens(...)` → `input_tokens` | ✓ |
+| Google | `client.models.count_tokens(...)` → `total_tokens` | ✓ |
+| OpenAI | — | ✗ (no remote API; use `tiktoken` locally) |
+| OpenRouter | — | ✗ |
+
+For unsupported providers, `loom tokens` prints _"Token counting not available: ..."_ and exits with code 2.
+
+### Where Loom stores state
+
+```
+~/.loom/
+├── batches/        # one <provider>_<batch_id>.json per pending or kept batch
+└── cache/          # one <sha256>.json per cached --sync response
+```
+
+- `~/.loom/batches/<provider>_<safe_id>.json` is created by `loom run` (batch mode) and contains `batch_id`, `provider`, `model`, `original_file_path`, `file_type`, the prompt column, an `id_map` mapping internal `custom_id` → original row id, `created_at`, and the last-seen `status`. `loom fetch` updates `status`, downloads results, and (unless `--keep` is passed) deletes the file on success.
+- `~/.loom/cache/<sha256>.json` is the response cache used by `--sync`. Each file holds `{provider, model, response, created_at}`.
+
+Both directories are safe to delete by hand: cache will rebuild itself; deleting `batches/` orphans any in-flight batch jobs (they still complete on the provider's side, you just lose Loom's view of them).
+
+### Troubleshooting
+
+**`ModuleNotFoundError: No module named 'pandas'` right after `pip install -e ".[dev]"`**
+Your `.venv` was probably created with `uv venv`, which doesn't install `pip` inside. Your `pip install` ran against the system / conda `pip` and dropped the packages elsewhere. Fix with `uv pip install -e ".[dev]"`, or recreate the venv with `python -m venv .venv && source .venv/bin/activate && pip install -e ".[dev]"`.
+
+**`which loom` shows `/opt/miniconda3/bin/loom` even after `source .venv/bin/activate`**
+Conda's path is being prepended after the venv. Either reorder your shell init, or just call the venv binary directly: `./.venv/bin/loom <cmd>`.
+
+**Google batch `Invalid batch job name: jqpem7...`**
+The stored `batch_id` is missing the required `batches/` prefix (an older Loom version stripped it). Edit `~/.loom/batches/google_<id>.json`, change `"batch_id": "<id>"` to `"batch_id": "batches/<id>"`, and rename the file to `google_batches_<id>.json` so the on-disk filename and the in-file id stay consistent.
+
+**`status=unknown` in `loom fetch`**
+The previous fetch attempt raised an exception (bad id, network blip, expired key, or an API response Loom doesn't recognise). Re-running `loom fetch` retries; if it persists, run with the provider's SDK directly to surface the underlying error.
+
+**`Error: OpenRouter has no batch API`**
+OpenRouter doesn't offer batch processing. Re-run with `--sync`.
+
+## 4. Developer instructions
+
+### Repository layout
+
+```
+loom/
+  main.py                       # CLI entry point (Typer commands)
+  core/
+    orchestrator.py             # run_batch, fetch_batch, generate_sync, count_tokens
+    models.py                   # Pydantic models, ProviderName, BatchStatus
+  providers/
+    base.py                     # Batch provider ABC (submit/check_status/download)
+    sync_base.py                # Sync provider ABC (generate/count_tokens)
+    openai.py, anthropic.py,
+    google.py                   # Batch implementations
+    openai_sync.py, anthropic_sync.py,
+    google_sync.py, openrouter_sync.py   # Sync implementations
+  utils/
+    converters.py               # Load / merge JSON & CSV
+    storage.py                  # ~/.loom/batches/ persistence
+    cache.py                    # ~/.loom/cache/ response cache
+    keys.py                     # API-key resolution
+tests/                          # pytest suite
+.github/workflows/              # CI: test.yml, publish.yml
+pyproject.toml                  # Dependencies and package metadata
+```
+
+### Running unit tests
+
+```bash
+pip install -e ".[dev]"
+pytest                 # quiet
+pytest -v              # verbose
+pytest tests/test_converters.py     # one file
+pytest tests/test_storage.py::test_save_and_load_roundtrip   # one test
+```
+
+### GitHub Actions
+
+- [`.github/workflows/test.yml`](.github/workflows/test.yml) — runs on every push and PR, with a matrix over Python 3.10 / 3.11 / 3.12. Installs the project with `pip install -e ".[dev]"` and runs `pytest -v`.
+- [`.github/workflows/publish.yml`](.github/workflows/publish.yml) — fires when you push a `v*` tag or publish a GitHub Release. It runs tests, builds an sdist + wheel, and uploads to PyPI via **OIDC Trusted Publishing** — no PyPI token is stored in repo secrets.
+
+### Releasing
+
+**One-time PyPI setup:**
+
+1. Create the project on PyPI: https://pypi.org/manage/account/publishing/
+2. Add a **trusted publisher** with:
+   - Owner: `jannehring`
+   - Repository: `loom`
+   - Workflow: `publish.yml`
+   - Environment: `pypi`
+3. In GitHub: **Settings → Environments → New environment → `pypi`**. Enable manual approval here if you want a human in the loop for every release.
+
+**Cutting a release:**
+
+```bash
+# 1. Bump version in pyproject.toml
+# 2. Commit and tag
+git commit -am "Release v0.1.1"
+git tag v0.1.1
+git push origin main --tags
+```
+
+The `publish` workflow runs on the tag, executes the test suite, builds, and uploads. You can also trigger it manually from the Actions tab (`workflow_dispatch`).
+
+## 5. License
+
+MIT — see [LICENSE](LICENSE).