batchor 0.0.3__tar.gz → 0.0.5__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {batchor-0.0.3 → batchor-0.0.5}/.agents/plugins/marketplace.json +12 -0
- {batchor-0.0.3 → batchor-0.0.5}/.agents/skills/batchor-dev/SKILL.md +2 -1
- {batchor-0.0.3 → batchor-0.0.5}/PKG-INFO +146 -14
- {batchor-0.0.3 → batchor-0.0.5}/README.md +138 -11
- batchor-0.0.5/docs/design_docs/ANTHROPIC_BATCHING.md +33 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/design_docs/ARCHITECTURE.md +29 -9
- batchor-0.0.5/docs/design_docs/GEMINI_BATCHING.md +96 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/design_docs/OPENAI_BATCHING.md +6 -2
- {batchor-0.0.3 → batchor-0.0.5}/docs/design_docs/ROADMAP.md +2 -5
- {batchor-0.0.3 → batchor-0.0.5}/docs/design_docs/STORAGE_AND_RUNS.md +17 -3
- {batchor-0.0.3 → batchor-0.0.5}/docs/doc-map.md +4 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/getting-started/cli.md +64 -0
- batchor-0.0.5/docs/getting-started/installation.md +104 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/getting-started/python-api.md +67 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/index.md +2 -2
- {batchor-0.0.3 → batchor-0.0.5}/docs/policies/contributing.md +1 -1
- {batchor-0.0.3 → batchor-0.0.5}/docs/reference/api.md +20 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/smoke-test.md +45 -1
- {batchor-0.0.3 → batchor-0.0.5}/mkdocs.yml +3 -1
- batchor-0.0.5/plugins/batchor/.codex-plugin/plugin.json +44 -0
- batchor-0.0.5/plugins/batchor/.mcp.json +10 -0
- batchor-0.0.5/plugins/batchor/scripts/batchor_user_mcp.py +218 -0
- batchor-0.0.5/plugins/batchor/skills/use-batchor/SKILL.md +57 -0
- batchor-0.0.5/plugins/batchor/skills/use-batchor/agents/openai.yaml +4 -0
- batchor-0.0.5/plugins/batchor/skills/use-batchor/references/cli-workflows.md +47 -0
- batchor-0.0.5/plugins/batchor/skills/use-batchor/references/python-pipelines.md +66 -0
- {batchor-0.0.3 → batchor-0.0.5}/plugins/batchor-agent-tools/scripts/batchor_repo_mcp.py +21 -3
- {batchor-0.0.3 → batchor-0.0.5}/pyproject.toml +13 -4
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/__init__.py +14 -2
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/cli.py +114 -3
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/core/enums.py +17 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/core/exceptions.py +11 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/core/models.py +215 -1
- batchor-0.0.5/src/batchor/core/responses.py +60 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/core/types.py +10 -7
- batchor-0.0.5/src/batchor/providers/anthropic.py +177 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/providers/base.py +33 -3
- batchor-0.0.5/src/batchor/providers/gemini.py +554 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/providers/registry.py +34 -7
- batchor-0.0.5/src/batchor/runtime/execution.py +105 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/ingestion.py +13 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/polling.py +5 -2
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/run_handle.py +5 -13
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/runner.py +14 -10
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/submission.py +13 -9
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/validation.py +12 -67
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/memory.py +2 -1
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/sqlite_queries.py +14 -1
- batchor-0.0.5/tests/integration/test_batchor_gemini_runner.py +219 -0
- batchor-0.0.5/tests/integration/test_batchor_live_anthropic.py +60 -0
- batchor-0.0.5/tests/integration/test_batchor_live_gemini.py +157 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/integration/test_batchor_runner.py +130 -0
- batchor-0.0.5/tests/unit/test_agent_tooling.py +206 -0
- batchor-0.0.5/tests/unit/test_batchor_anthropic_provider.py +117 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_architecture.py +57 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_cli.py +159 -1
- batchor-0.0.5/tests/unit/test_batchor_gemini_provider.py +355 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_runtime_polling.py +3 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_runtime_submission.py +94 -2
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_storage_contracts.py +24 -0
- {batchor-0.0.3 → batchor-0.0.5}/uv.lock +399 -1
- batchor-0.0.3/docs/getting-started/installation.md +0 -56
- batchor-0.0.3/tests/unit/test_agent_tooling.py +0 -49
- {batchor-0.0.3 → batchor-0.0.5}/.agents/skills/batchor-dev/agents/openai.yaml +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/.github/workflows/ci.yml +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/.github/workflows/docs.yml +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/.github/workflows/publish.yml +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/.gitignore +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/.pre-commit-config.yaml +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/AGENTS.md +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/CONTRIBUTING.md +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/LICENSE +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/SUPPORT.md +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/VERSIONING.md +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/assets/stylesheets/extra.css +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/design_docs/BOUNDARY_AND_PHILOSOPHY.md +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/design_docs/STORAGE_MIGRATIONS.md +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/getting-started/use-cases.md +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/policies/support.md +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/docs/policies/versioning.md +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/plugins/batchor-agent-tools/.codex-plugin/plugin.json +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/plugins/batchor-agent-tools/.mcp.json +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/artifacts/__init__.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/artifacts/base.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/artifacts/local.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/core/__init__.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/providers/__init__.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/providers/openai.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/py.typed +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/__init__.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/artifacts.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/context.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/results.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/retry.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/runtime/tokens.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/sources/__init__.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/sources/base.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/sources/composite.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/sources/files.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/__init__.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/postgres.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/postgres_store.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/registry.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/sqlite.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/sqlite_codec.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/sqlite_lifecycle.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/sqlite_protocol.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/sqlite_results.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/sqlite_schema.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/sqlite_store.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/state.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/src/batchor/storage/state_models.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/integration/test_batchor_live_openai.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_artifacts.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_openai_provider.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_results.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_retry.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_runtime_ingestion.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_sources.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_sqlite_storage_flow.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_storage.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_tokens.py +0 -0
- {batchor-0.0.3 → batchor-0.0.5}/tests/unit/test_batchor_validation.py +0 -0
|
@@ -15,6 +15,18 @@
|
|
|
15
15
|
"authentication": "ON_INSTALL"
|
|
16
16
|
},
|
|
17
17
|
"category": "Coding"
|
|
18
|
+
},
|
|
19
|
+
{
|
|
20
|
+
"name": "batchor",
|
|
21
|
+
"source": {
|
|
22
|
+
"source": "local",
|
|
23
|
+
"path": "./plugins/batchor"
|
|
24
|
+
},
|
|
25
|
+
"policy": {
|
|
26
|
+
"installation": "AVAILABLE",
|
|
27
|
+
"authentication": "ON_INSTALL"
|
|
28
|
+
},
|
|
29
|
+
"category": "Coding"
|
|
18
30
|
}
|
|
19
31
|
]
|
|
20
32
|
}
|
|
@@ -14,13 +14,14 @@ Use this skill for normal contributor work in this repository.
|
|
|
14
14
|
3. Load only the design docs that match the task:
|
|
15
15
|
- `docs/design_docs/ARCHITECTURE.md` for package layout and runtime boundaries
|
|
16
16
|
- `docs/design_docs/OPENAI_BATCHING.md` for provider-specific batching behavior
|
|
17
|
+
- `docs/design_docs/ANTHROPIC_BATCHING.md` for Anthropic Message Batches behavior
|
|
17
18
|
- `docs/design_docs/STORAGE_AND_RUNS.md` for durable state, run lifecycle, and retention
|
|
18
19
|
- `docs/smoke-test.md` for the full validation matrix
|
|
19
20
|
|
|
20
21
|
## Repo map
|
|
21
22
|
|
|
22
23
|
- `src/batchor/runtime/`: orchestration, run handles, validation, retry, token budgeting
|
|
23
|
-
- `src/batchor/providers/`: provider interfaces and OpenAI
|
|
24
|
+
- `src/batchor/providers/`: provider interfaces and OpenAI, Anthropic, and Gemini implementations
|
|
24
25
|
- `src/batchor/storage/`: SQLite, Postgres, and in-memory durability backends
|
|
25
26
|
- `src/batchor/sources/`: file-backed checkpointable item sources
|
|
26
27
|
- `src/batchor/artifacts/`: durable request/output artifact handling
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: batchor
|
|
3
|
-
Version: 0.0.
|
|
4
|
-
Summary: Structured-first
|
|
3
|
+
Version: 0.0.5
|
|
4
|
+
Summary: Structured-first provider Batch runner with typed Pydantic results.
|
|
5
5
|
Project-URL: Documentation, https://github.com/AnsonDev42/batchor#readme
|
|
6
6
|
Project-URL: Repository, https://github.com/AnsonDev42/batchor
|
|
7
7
|
Project-URL: Issues, https://github.com/AnsonDev42/batchor/issues
|
|
@@ -79,7 +79,7 @@ License: # PolyForm Noncommercial License 1.0.0
|
|
|
79
79
|
|
|
80
80
|
**Use** means anything you do with the software requiring one of your licenses.
|
|
81
81
|
License-File: LICENSE
|
|
82
|
-
Keywords: batch,cli,openai,pydantic,sqlite
|
|
82
|
+
Keywords: anthropic,batch,cli,gemini,openai,pydantic,sqlite
|
|
83
83
|
Classifier: Development Status :: 3 - Alpha
|
|
84
84
|
Classifier: Environment :: Console
|
|
85
85
|
Classifier: Intended Audience :: Developers
|
|
@@ -99,13 +99,18 @@ Requires-Dist: python-dotenv<2,>=1.0.1
|
|
|
99
99
|
Requires-Dist: sqlalchemy<3,>=2.0.43
|
|
100
100
|
Requires-Dist: tiktoken>=0.12.0
|
|
101
101
|
Requires-Dist: typer<1,>=0.16.0
|
|
102
|
+
Provides-Extra: anthropic
|
|
103
|
+
Requires-Dist: anthropic<1,>=0.71.0; extra == 'anthropic'
|
|
104
|
+
Provides-Extra: gemini
|
|
105
|
+
Requires-Dist: google-cloud-storage<4,>=3.4.0; extra == 'gemini'
|
|
106
|
+
Requires-Dist: google-genai<2,>=1.55.0; extra == 'gemini'
|
|
102
107
|
Description-Content-Type: text/markdown
|
|
103
108
|
|
|
104
109
|
# batchor
|
|
105
110
|
|
|
106
111
|
`batchor` grew out of a recurring problem in academic research: running large datasets through LLMs in batch — reliably, reproducibly, and without reinventing the same glue code across every project. The patterns that kept emerging (durable state, typed results, safe resume after failure) were extracted into this library so they do not have to be rebuilt each time.
|
|
107
112
|
|
|
108
|
-
`batchor` is a durable
|
|
113
|
+
`batchor` is a durable provider Batch runner for Python teams that want:
|
|
109
114
|
|
|
110
115
|
- typed Pydantic results
|
|
111
116
|
- resumable durable runs
|
|
@@ -115,7 +120,7 @@ Description-Content-Type: text/markdown
|
|
|
115
120
|
- library-first run controls
|
|
116
121
|
- a small operator CLI for CSV and JSONL jobs
|
|
117
122
|
|
|
118
|
-
It is intentionally narrow today: OpenAI-
|
|
123
|
+
It is intentionally narrow today: OpenAI is the CLI default, Anthropic and Gemini are opt-in through provider extras, SQLite is the CLI durability backend, and the Python API exposes the broadest configuration surface.
|
|
119
124
|
|
|
120
125
|
## What problem it solves
|
|
121
126
|
|
|
@@ -140,6 +145,8 @@ Most OpenAI Batch examples stop at "upload a JSONL file and poll until it finish
|
|
|
140
145
|
Built-in implementations:
|
|
141
146
|
|
|
142
147
|
- `OpenAIProviderConfig` + `OpenAIBatchProvider`
|
|
148
|
+
- `AnthropicProviderConfig` + `AnthropicBatchProvider` for Claude Message Batches
|
|
149
|
+
- `GeminiProviderConfig` + `GeminiBatchProvider` for text-only Gemini Batch jobs
|
|
143
150
|
- `SQLiteStorage`
|
|
144
151
|
- `PostgresStorage` as an opt-in durable control-plane backend
|
|
145
152
|
- `MemoryStateStore`
|
|
@@ -155,6 +162,8 @@ Important constraints:
|
|
|
155
162
|
- the CLI supports file-backed inputs only
|
|
156
163
|
- users still own selecting and ordering input files or partitions
|
|
157
164
|
- the built-in CLI uses SQLite durability only
|
|
165
|
+
- the CLI supports OpenAI, Anthropic, Gemini Developer API, and Vertex AI text jobs
|
|
166
|
+
- Gemini support is text-only for now and does not build multimodal requests
|
|
158
167
|
- structured-output rehydration requires an importable module-level Pydantic model
|
|
159
168
|
- raw output artifacts are retained by default and must be exported before raw pruning
|
|
160
169
|
- pause/resume/cancel and incremental terminal-result APIs are library-first today
|
|
@@ -179,6 +188,14 @@ Durability is split on purpose:
|
|
|
179
188
|
|
|
180
189
|
That split is what allows retries and fresh-process resume without keeping every request inline in the control-plane store.
|
|
181
190
|
On resume, existing active provider batches are polled before `batchor` materializes or submits new local work, so completed batches and retry backoff are reconciled first. Deterministic sources also use stored checkpoint completion metadata to avoid opening a fully materialized Parquet or composite source just to discover there are no rows left.
|
|
191
|
+
Incomplete checkpointed ingestion keeps the run non-terminal. In the process that
|
|
192
|
+
started the job, `Run.resume()` can continue from the attached source. After a
|
|
193
|
+
fresh-process rehydration, call `start(job, run_id=...)`; `refresh()` otherwise
|
|
194
|
+
raises `RunIngestionSourceRequiredError` instead of silently completing only the
|
|
195
|
+
already-materialized rows.
|
|
196
|
+
Cancelling an incompletely ingested checkpointed source abandons its
|
|
197
|
+
unmaterialized tail and finalizes the checkpoint after active batches drain, so
|
|
198
|
+
the cancelled run can still reach a terminal lifecycle state.
|
|
182
199
|
|
|
183
200
|
## Architecture
|
|
184
201
|
|
|
@@ -189,10 +206,13 @@ graph LR
|
|
|
189
206
|
subgraph runtime["runtime/"]
|
|
190
207
|
BatchRunner
|
|
191
208
|
Run["Run handle"]
|
|
209
|
+
Executor["RunExecutor"]
|
|
192
210
|
end
|
|
193
211
|
|
|
194
212
|
subgraph providers["providers/"]
|
|
195
213
|
OpenAI["OpenAIBatchProvider"]
|
|
214
|
+
Anthropic["AnthropicBatchProvider"]
|
|
215
|
+
Gemini["GeminiBatchProvider"]
|
|
196
216
|
end
|
|
197
217
|
|
|
198
218
|
subgraph sources["sources/"]
|
|
@@ -211,7 +231,10 @@ graph LR
|
|
|
211
231
|
|
|
212
232
|
User -->|"start() / run_and_wait()"| BatchRunner
|
|
213
233
|
BatchRunner --> Run
|
|
234
|
+
BatchRunner --> Executor
|
|
214
235
|
BatchRunner --> OpenAI
|
|
236
|
+
BatchRunner --> Anthropic
|
|
237
|
+
BatchRunner --> Gemini
|
|
215
238
|
BatchRunner --> SQLite
|
|
216
239
|
BatchRunner --> LocalFS
|
|
217
240
|
Files -->|"BatchItem stream"| BatchRunner
|
|
@@ -247,16 +270,37 @@ Operational semantics for resume, run control, and artifact retention live in
|
|
|
247
270
|
pip install batchor
|
|
248
271
|
```
|
|
249
272
|
|
|
250
|
-
|
|
273
|
+
For Gemini Batch support, install the optional extra:
|
|
251
274
|
|
|
252
|
-
|
|
275
|
+
```bash
|
|
276
|
+
pip install "batchor[gemini]"
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
For Anthropic Message Batches support:
|
|
280
|
+
|
|
281
|
+
```bash
|
|
282
|
+
pip install "batchor[anthropic]"
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
## Agent setup
|
|
286
|
+
|
|
287
|
+
Batchor keeps contributor tooling separate from the tools intended for researchers and downstream projects:
|
|
288
|
+
|
|
289
|
+
- `plugins/batchor/` is the user-facing Codex plugin. Its `$use-batchor` skill and MCP helpers turn datasets and prompts into safe CLI or Python workflows without assuming a Batchor source checkout.
|
|
290
|
+
- `.agents/skills/batchor-dev/` and `plugins/batchor-agent-tools/` are contributor-only. They teach agents how to change this repository, find design docs, and run its validation suite.
|
|
291
|
+
- `.agents/plugins/marketplace.json` registers both plugins for local development.
|
|
292
|
+
- `.vscode/mcp.json` configures the contributor MCP for this workspace.
|
|
253
293
|
|
|
254
|
-
|
|
255
|
-
- repo-local plugin marketplace: `.agents/plugins/marketplace.json`
|
|
256
|
-
- repo-local MCP plugin: `plugins/batchor-agent-tools/`
|
|
257
|
-
- VS Code workspace MCP config: `.vscode/mcp.json`
|
|
294
|
+
The PyPI package remains the runtime dependency: `pip install batchor` installs the library and CLI. Agent skills and MCP configuration are distributed as the separate `batchor` plugin so Python environments do not receive Codex-specific files implicitly.
|
|
258
295
|
|
|
259
|
-
|
|
296
|
+
To try the user plugin from a local checkout:
|
|
297
|
+
|
|
298
|
+
```bash
|
|
299
|
+
codex plugin marketplace add /path/to/batchor
|
|
300
|
+
codex plugin add batchor@batchor-local
|
|
301
|
+
```
|
|
302
|
+
|
|
303
|
+
Start a new Codex task after installation, then ask: `Use $use-batchor to turn my CSV and research prompt into a resumable batch job.`
|
|
260
304
|
|
|
261
305
|
Supported Python versions:
|
|
262
306
|
|
|
@@ -268,8 +312,9 @@ Supported Python versions:
|
|
|
268
312
|
|
|
269
313
|
For Python API usage, auth resolution is:
|
|
270
314
|
|
|
271
|
-
1. explicit `OpenAIProviderConfig(api_key=...)`
|
|
272
|
-
2. ambient `OPENAI_API_KEY`
|
|
315
|
+
1. explicit provider config credentials such as `OpenAIProviderConfig(api_key=...)`, `AnthropicProviderConfig(api_key=...)`, or `GeminiProviderConfig(api_key=...)`
|
|
316
|
+
2. ambient provider environment variables: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `GEMINI_API_KEY`
|
|
317
|
+
3. Vertex AI Application Default Credentials when `GeminiProviderConfig(vertexai=True, ...)` or `GOOGLE_GENAI_USE_VERTEXAI=true` is used
|
|
273
318
|
|
|
274
319
|
The Python library does not auto-load `.env`.
|
|
275
320
|
|
|
@@ -298,6 +343,66 @@ run = runner.run_and_wait(
|
|
|
298
343
|
print(run.results()[0].output_text)
|
|
299
344
|
```
|
|
300
345
|
|
|
346
|
+
### Anthropic text job
|
|
347
|
+
|
|
348
|
+
```python
|
|
349
|
+
from batchor import AnthropicProviderConfig, BatchItem, BatchJob, BatchRunner, PromptParts
|
|
350
|
+
|
|
351
|
+
|
|
352
|
+
runner = BatchRunner(storage="memory")
|
|
353
|
+
run = runner.run_and_wait(
|
|
354
|
+
BatchJob(
|
|
355
|
+
items=[BatchItem(item_id="row1", payload="Summarize this text")],
|
|
356
|
+
build_prompt=lambda item: PromptParts(prompt=item.payload),
|
|
357
|
+
provider_config=AnthropicProviderConfig(
|
|
358
|
+
model="claude-sonnet-4-5",
|
|
359
|
+
max_tokens=1024,
|
|
360
|
+
),
|
|
361
|
+
)
|
|
362
|
+
)
|
|
363
|
+
|
|
364
|
+
print(run.results()[0].output_text)
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
Anthropic support maps Batchor attempt identifiers deterministically to Claude-safe `custom_id` values and supports system prompts, `message_params`, and structured outputs through `output_config.format`.
|
|
368
|
+
|
|
369
|
+
### Gemini text job
|
|
370
|
+
|
|
371
|
+
```python
|
|
372
|
+
from batchor import BatchItem, BatchJob, BatchRunner, GeminiProviderConfig, PromptParts
|
|
373
|
+
|
|
374
|
+
|
|
375
|
+
runner = BatchRunner(storage="memory")
|
|
376
|
+
run = runner.run_and_wait(
|
|
377
|
+
BatchJob(
|
|
378
|
+
items=[BatchItem(item_id="row1", payload="Summarize this text")],
|
|
379
|
+
build_prompt=lambda item: PromptParts(prompt=item.payload),
|
|
380
|
+
provider_config=GeminiProviderConfig(
|
|
381
|
+
model="gemini-2.5-flash",
|
|
382
|
+
api_key="YOUR_GEMINI_API_KEY",
|
|
383
|
+
),
|
|
384
|
+
)
|
|
385
|
+
)
|
|
386
|
+
|
|
387
|
+
print(run.results()[0].output_text)
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
Gemini support currently builds text-only `GenerateContent` batch requests. It uses Gemini JSONL `key` values internally while keeping `batchor`'s durable item and attempt tracking unchanged.
|
|
391
|
+
|
|
392
|
+
For Vertex AI, provide a Cloud Storage staging prefix and use Application Default Credentials:
|
|
393
|
+
|
|
394
|
+
```python
|
|
395
|
+
provider_config = GeminiProviderConfig(
|
|
396
|
+
model="gemini-2.5-flash",
|
|
397
|
+
vertexai=True,
|
|
398
|
+
project="my-project",
|
|
399
|
+
location="europe-west8",
|
|
400
|
+
gcs_uri="gs://my-bucket/batchor",
|
|
401
|
+
)
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
Vertex AI stages JSONL input and output in that prefix. The Gemini Developer API uses inline requests for batches below 20 MB and the Files API for larger batches; `input_mode=` can override that automatic choice. Vertex request/output correlation uses a generated request label because Vertex output does not include the Developer API JSONL `key`.
|
|
405
|
+
|
|
301
406
|
### Structured output
|
|
302
407
|
|
|
303
408
|
```python
|
|
@@ -351,6 +456,8 @@ Structured-output models are validated up front against the OpenAI strict-schema
|
|
|
351
456
|
|
|
352
457
|
If you need a field to be optional in Python, model it as nullable in the schema shape OpenAI accepts rather than relying on omitted required fields.
|
|
353
458
|
|
|
459
|
+
The same `structured_output=` API is available with `GeminiProviderConfig`; batchor sends the schema through Gemini `generation_config.response_json_schema` and validates the returned JSON text with the same Pydantic model.
|
|
460
|
+
|
|
354
461
|
### Rehydrate a durable run
|
|
355
462
|
|
|
356
463
|
```python
|
|
@@ -498,6 +605,31 @@ The CLI is intentionally narrower than the Python API:
|
|
|
498
605
|
- CSV and JSONL only
|
|
499
606
|
- SQLite-backed durable runs only
|
|
500
607
|
|
|
608
|
+
OpenAI remains the default provider. For Gemini, install `batchor[gemini]` and select the backend explicitly or let `auto` follow `GOOGLE_GENAI_USE_VERTEXAI`:
|
|
609
|
+
|
|
610
|
+
```bash
|
|
611
|
+
batchor start \
|
|
612
|
+
--input input/items.jsonl \
|
|
613
|
+
--id-field id \
|
|
614
|
+
--prompt-field text \
|
|
615
|
+
--provider gemini \
|
|
616
|
+
--model gemini-2.5-flash \
|
|
617
|
+
--gemini-backend developer
|
|
618
|
+
```
|
|
619
|
+
|
|
620
|
+
Vertex AI additionally needs a writable staging prefix:
|
|
621
|
+
|
|
622
|
+
```bash
|
|
623
|
+
batchor start \
|
|
624
|
+
--input input/items.jsonl \
|
|
625
|
+
--id-field id \
|
|
626
|
+
--prompt-field text \
|
|
627
|
+
--provider gemini \
|
|
628
|
+
--model gemini-2.5-flash \
|
|
629
|
+
--gemini-backend vertex \
|
|
630
|
+
--gcs-uri gs://my-bucket/batchor
|
|
631
|
+
```
|
|
632
|
+
|
|
501
633
|
Start a run from JSONL:
|
|
502
634
|
|
|
503
635
|
```bash
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
`batchor` grew out of a recurring problem in academic research: running large datasets through LLMs in batch — reliably, reproducibly, and without reinventing the same glue code across every project. The patterns that kept emerging (durable state, typed results, safe resume after failure) were extracted into this library so they do not have to be rebuilt each time.
|
|
4
4
|
|
|
5
|
-
`batchor` is a durable
|
|
5
|
+
`batchor` is a durable provider Batch runner for Python teams that want:
|
|
6
6
|
|
|
7
7
|
- typed Pydantic results
|
|
8
8
|
- resumable durable runs
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
- library-first run controls
|
|
13
13
|
- a small operator CLI for CSV and JSONL jobs
|
|
14
14
|
|
|
15
|
-
It is intentionally narrow today: OpenAI-
|
|
15
|
+
It is intentionally narrow today: OpenAI is the CLI default, Anthropic and Gemini are opt-in through provider extras, SQLite is the CLI durability backend, and the Python API exposes the broadest configuration surface.
|
|
16
16
|
|
|
17
17
|
## What problem it solves
|
|
18
18
|
|
|
@@ -37,6 +37,8 @@ Most OpenAI Batch examples stop at "upload a JSONL file and poll until it finish
|
|
|
37
37
|
Built-in implementations:
|
|
38
38
|
|
|
39
39
|
- `OpenAIProviderConfig` + `OpenAIBatchProvider`
|
|
40
|
+
- `AnthropicProviderConfig` + `AnthropicBatchProvider` for Claude Message Batches
|
|
41
|
+
- `GeminiProviderConfig` + `GeminiBatchProvider` for text-only Gemini Batch jobs
|
|
40
42
|
- `SQLiteStorage`
|
|
41
43
|
- `PostgresStorage` as an opt-in durable control-plane backend
|
|
42
44
|
- `MemoryStateStore`
|
|
@@ -52,6 +54,8 @@ Important constraints:
|
|
|
52
54
|
- the CLI supports file-backed inputs only
|
|
53
55
|
- users still own selecting and ordering input files or partitions
|
|
54
56
|
- the built-in CLI uses SQLite durability only
|
|
57
|
+
- the CLI supports OpenAI, Anthropic, Gemini Developer API, and Vertex AI text jobs
|
|
58
|
+
- Gemini support is text-only for now and does not build multimodal requests
|
|
55
59
|
- structured-output rehydration requires an importable module-level Pydantic model
|
|
56
60
|
- raw output artifacts are retained by default and must be exported before raw pruning
|
|
57
61
|
- pause/resume/cancel and incremental terminal-result APIs are library-first today
|
|
@@ -76,6 +80,14 @@ Durability is split on purpose:
|
|
|
76
80
|
|
|
77
81
|
That split is what allows retries and fresh-process resume without keeping every request inline in the control-plane store.
|
|
78
82
|
On resume, existing active provider batches are polled before `batchor` materializes or submits new local work, so completed batches and retry backoff are reconciled first. Deterministic sources also use stored checkpoint completion metadata to avoid opening a fully materialized Parquet or composite source just to discover there are no rows left.
|
|
83
|
+
Incomplete checkpointed ingestion keeps the run non-terminal. In the process that
|
|
84
|
+
started the job, `Run.resume()` can continue from the attached source. After a
|
|
85
|
+
fresh-process rehydration, call `start(job, run_id=...)`; `refresh()` otherwise
|
|
86
|
+
raises `RunIngestionSourceRequiredError` instead of silently completing only the
|
|
87
|
+
already-materialized rows.
|
|
88
|
+
Cancelling an incompletely ingested checkpointed source abandons its
|
|
89
|
+
unmaterialized tail and finalizes the checkpoint after active batches drain, so
|
|
90
|
+
the cancelled run can still reach a terminal lifecycle state.
|
|
79
91
|
|
|
80
92
|
## Architecture
|
|
81
93
|
|
|
@@ -86,10 +98,13 @@ graph LR
|
|
|
86
98
|
subgraph runtime["runtime/"]
|
|
87
99
|
BatchRunner
|
|
88
100
|
Run["Run handle"]
|
|
101
|
+
Executor["RunExecutor"]
|
|
89
102
|
end
|
|
90
103
|
|
|
91
104
|
subgraph providers["providers/"]
|
|
92
105
|
OpenAI["OpenAIBatchProvider"]
|
|
106
|
+
Anthropic["AnthropicBatchProvider"]
|
|
107
|
+
Gemini["GeminiBatchProvider"]
|
|
93
108
|
end
|
|
94
109
|
|
|
95
110
|
subgraph sources["sources/"]
|
|
@@ -108,7 +123,10 @@ graph LR
|
|
|
108
123
|
|
|
109
124
|
User -->|"start() / run_and_wait()"| BatchRunner
|
|
110
125
|
BatchRunner --> Run
|
|
126
|
+
BatchRunner --> Executor
|
|
111
127
|
BatchRunner --> OpenAI
|
|
128
|
+
BatchRunner --> Anthropic
|
|
129
|
+
BatchRunner --> Gemini
|
|
112
130
|
BatchRunner --> SQLite
|
|
113
131
|
BatchRunner --> LocalFS
|
|
114
132
|
Files -->|"BatchItem stream"| BatchRunner
|
|
@@ -144,16 +162,37 @@ Operational semantics for resume, run control, and artifact retention live in
|
|
|
144
162
|
pip install batchor
|
|
145
163
|
```
|
|
146
164
|
|
|
147
|
-
|
|
165
|
+
For Gemini Batch support, install the optional extra:
|
|
148
166
|
|
|
149
|
-
|
|
167
|
+
```bash
|
|
168
|
+
pip install "batchor[gemini]"
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
For Anthropic Message Batches support:
|
|
172
|
+
|
|
173
|
+
```bash
|
|
174
|
+
pip install "batchor[anthropic]"
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
## Agent setup
|
|
178
|
+
|
|
179
|
+
Batchor keeps contributor tooling separate from the tools intended for researchers and downstream projects:
|
|
180
|
+
|
|
181
|
+
- `plugins/batchor/` is the user-facing Codex plugin. Its `$use-batchor` skill and MCP helpers turn datasets and prompts into safe CLI or Python workflows without assuming a Batchor source checkout.
|
|
182
|
+
- `.agents/skills/batchor-dev/` and `plugins/batchor-agent-tools/` are contributor-only. They teach agents how to change this repository, find design docs, and run its validation suite.
|
|
183
|
+
- `.agents/plugins/marketplace.json` registers both plugins for local development.
|
|
184
|
+
- `.vscode/mcp.json` configures the contributor MCP for this workspace.
|
|
150
185
|
|
|
151
|
-
|
|
152
|
-
- repo-local plugin marketplace: `.agents/plugins/marketplace.json`
|
|
153
|
-
- repo-local MCP plugin: `plugins/batchor-agent-tools/`
|
|
154
|
-
- VS Code workspace MCP config: `.vscode/mcp.json`
|
|
186
|
+
The PyPI package remains the runtime dependency: `pip install batchor` installs the library and CLI. Agent skills and MCP configuration are distributed as the separate `batchor` plugin so Python environments do not receive Codex-specific files implicitly.
|
|
155
187
|
|
|
156
|
-
|
|
188
|
+
To try the user plugin from a local checkout:
|
|
189
|
+
|
|
190
|
+
```bash
|
|
191
|
+
codex plugin marketplace add /path/to/batchor
|
|
192
|
+
codex plugin add batchor@batchor-local
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Start a new Codex task after installation, then ask: `Use $use-batchor to turn my CSV and research prompt into a resumable batch job.`
|
|
157
196
|
|
|
158
197
|
Supported Python versions:
|
|
159
198
|
|
|
@@ -165,8 +204,9 @@ Supported Python versions:
|
|
|
165
204
|
|
|
166
205
|
For Python API usage, auth resolution is:
|
|
167
206
|
|
|
168
|
-
1. explicit `OpenAIProviderConfig(api_key=...)`
|
|
169
|
-
2. ambient `OPENAI_API_KEY`
|
|
207
|
+
1. explicit provider config credentials such as `OpenAIProviderConfig(api_key=...)`, `AnthropicProviderConfig(api_key=...)`, or `GeminiProviderConfig(api_key=...)`
|
|
208
|
+
2. ambient provider environment variables: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `GEMINI_API_KEY`
|
|
209
|
+
3. Vertex AI Application Default Credentials when `GeminiProviderConfig(vertexai=True, ...)` or `GOOGLE_GENAI_USE_VERTEXAI=true` is used
|
|
170
210
|
|
|
171
211
|
The Python library does not auto-load `.env`.
|
|
172
212
|
|
|
@@ -195,6 +235,66 @@ run = runner.run_and_wait(
|
|
|
195
235
|
print(run.results()[0].output_text)
|
|
196
236
|
```
|
|
197
237
|
|
|
238
|
+
### Anthropic text job
|
|
239
|
+
|
|
240
|
+
```python
|
|
241
|
+
from batchor import AnthropicProviderConfig, BatchItem, BatchJob, BatchRunner, PromptParts
|
|
242
|
+
|
|
243
|
+
|
|
244
|
+
runner = BatchRunner(storage="memory")
|
|
245
|
+
run = runner.run_and_wait(
|
|
246
|
+
BatchJob(
|
|
247
|
+
items=[BatchItem(item_id="row1", payload="Summarize this text")],
|
|
248
|
+
build_prompt=lambda item: PromptParts(prompt=item.payload),
|
|
249
|
+
provider_config=AnthropicProviderConfig(
|
|
250
|
+
model="claude-sonnet-4-5",
|
|
251
|
+
max_tokens=1024,
|
|
252
|
+
),
|
|
253
|
+
)
|
|
254
|
+
)
|
|
255
|
+
|
|
256
|
+
print(run.results()[0].output_text)
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
Anthropic support maps Batchor attempt identifiers deterministically to Claude-safe `custom_id` values and supports system prompts, `message_params`, and structured outputs through `output_config.format`.
|
|
260
|
+
|
|
261
|
+
### Gemini text job
|
|
262
|
+
|
|
263
|
+
```python
|
|
264
|
+
from batchor import BatchItem, BatchJob, BatchRunner, GeminiProviderConfig, PromptParts
|
|
265
|
+
|
|
266
|
+
|
|
267
|
+
runner = BatchRunner(storage="memory")
|
|
268
|
+
run = runner.run_and_wait(
|
|
269
|
+
BatchJob(
|
|
270
|
+
items=[BatchItem(item_id="row1", payload="Summarize this text")],
|
|
271
|
+
build_prompt=lambda item: PromptParts(prompt=item.payload),
|
|
272
|
+
provider_config=GeminiProviderConfig(
|
|
273
|
+
model="gemini-2.5-flash",
|
|
274
|
+
api_key="YOUR_GEMINI_API_KEY",
|
|
275
|
+
),
|
|
276
|
+
)
|
|
277
|
+
)
|
|
278
|
+
|
|
279
|
+
print(run.results()[0].output_text)
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
Gemini support currently builds text-only `GenerateContent` batch requests. It uses Gemini JSONL `key` values internally while keeping `batchor`'s durable item and attempt tracking unchanged.
|
|
283
|
+
|
|
284
|
+
For Vertex AI, provide a Cloud Storage staging prefix and use Application Default Credentials:
|
|
285
|
+
|
|
286
|
+
```python
|
|
287
|
+
provider_config = GeminiProviderConfig(
|
|
288
|
+
model="gemini-2.5-flash",
|
|
289
|
+
vertexai=True,
|
|
290
|
+
project="my-project",
|
|
291
|
+
location="europe-west8",
|
|
292
|
+
gcs_uri="gs://my-bucket/batchor",
|
|
293
|
+
)
|
|
294
|
+
```
|
|
295
|
+
|
|
296
|
+
Vertex AI stages JSONL input and output in that prefix. The Gemini Developer API uses inline requests for batches below 20 MB and the Files API for larger batches; `input_mode=` can override that automatic choice. Vertex request/output correlation uses a generated request label because Vertex output does not include the Developer API JSONL `key`.
|
|
297
|
+
|
|
198
298
|
### Structured output
|
|
199
299
|
|
|
200
300
|
```python
|
|
@@ -248,6 +348,8 @@ Structured-output models are validated up front against the OpenAI strict-schema
|
|
|
248
348
|
|
|
249
349
|
If you need a field to be optional in Python, model it as nullable in the schema shape OpenAI accepts rather than relying on omitted required fields.
|
|
250
350
|
|
|
351
|
+
The same `structured_output=` API is available with `GeminiProviderConfig`; batchor sends the schema through Gemini `generation_config.response_json_schema` and validates the returned JSON text with the same Pydantic model.
|
|
352
|
+
|
|
251
353
|
### Rehydrate a durable run
|
|
252
354
|
|
|
253
355
|
```python
|
|
@@ -395,6 +497,31 @@ The CLI is intentionally narrower than the Python API:
|
|
|
395
497
|
- CSV and JSONL only
|
|
396
498
|
- SQLite-backed durable runs only
|
|
397
499
|
|
|
500
|
+
OpenAI remains the default provider. For Gemini, install `batchor[gemini]` and select the backend explicitly or let `auto` follow `GOOGLE_GENAI_USE_VERTEXAI`:
|
|
501
|
+
|
|
502
|
+
```bash
|
|
503
|
+
batchor start \
|
|
504
|
+
--input input/items.jsonl \
|
|
505
|
+
--id-field id \
|
|
506
|
+
--prompt-field text \
|
|
507
|
+
--provider gemini \
|
|
508
|
+
--model gemini-2.5-flash \
|
|
509
|
+
--gemini-backend developer
|
|
510
|
+
```
|
|
511
|
+
|
|
512
|
+
Vertex AI additionally needs a writable staging prefix:
|
|
513
|
+
|
|
514
|
+
```bash
|
|
515
|
+
batchor start \
|
|
516
|
+
--input input/items.jsonl \
|
|
517
|
+
--id-field id \
|
|
518
|
+
--prompt-field text \
|
|
519
|
+
--provider gemini \
|
|
520
|
+
--model gemini-2.5-flash \
|
|
521
|
+
--gemini-backend vertex \
|
|
522
|
+
--gcs-uri gs://my-bucket/batchor
|
|
523
|
+
```
|
|
524
|
+
|
|
398
525
|
Start a run from JSONL:
|
|
399
526
|
|
|
400
527
|
```bash
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Anthropic Batching
|
|
2
|
+
|
|
3
|
+
The built-in Anthropic provider targets the Claude Message Batches API.
|
|
4
|
+
|
|
5
|
+
## Request mapping
|
|
6
|
+
|
|
7
|
+
Each Batchor item attempt becomes one Anthropic batch request:
|
|
8
|
+
|
|
9
|
+
- Batchor's durable correlation identifier is deterministically hashed into Anthropic's 64-character-safe `custom_id` alphabet
|
|
10
|
+
- `PromptParts.prompt` becomes one user message
|
|
11
|
+
- `PromptParts.system_prompt` becomes the top-level `system` parameter
|
|
12
|
+
- `model` and `max_tokens` come from `AnthropicProviderConfig`
|
|
13
|
+
- `message_params` supplies optional Messages API parameters
|
|
14
|
+
- structured output schemas use `output_config.format`
|
|
15
|
+
|
|
16
|
+
Batchor rejects provider-owned fields in `message_params` so replayed requests cannot silently change model, token limit, messages, system prompt, or streaming behavior.
|
|
17
|
+
|
|
18
|
+
## Submission and lifecycle
|
|
19
|
+
|
|
20
|
+
Anthropic accepts request objects directly rather than an uploaded JSONL file. The provider therefore stages Batchor's durable request artifact in memory only for the upload/create boundary, then calls `messages.batches.create(requests=...)`.
|
|
21
|
+
|
|
22
|
+
Anthropic `in_progress` maps to Batchor's active status and `ended` maps to `completed`. Once ended, the provider streams `messages.batches.results(...)` into Batchor's raw output artifact. Result order is not assumed; correlation always uses `custom_id`.
|
|
23
|
+
|
|
24
|
+
The four Anthropic result types map as follows:
|
|
25
|
+
|
|
26
|
+
- `succeeded`: Batchor success
|
|
27
|
+
- `errored`, `canceled`, and `expired`: Batchor item error, subject to the configured retry policy
|
|
28
|
+
|
|
29
|
+
## Limits and retention
|
|
30
|
+
|
|
31
|
+
Anthropic currently limits a Message Batch to 100,000 requests or 256 MB. Batchor's default chunk limits are lower than both limits. Claude batch results remain available from Anthropic for 29 days; Batchor downloads and retains them according to its artifact policy, so durable runs do not depend on that remote retention window after ingestion.
|
|
32
|
+
|
|
33
|
+
Message Batches are not eligible for Anthropic Zero Data Retention. Users should account for Anthropic's batch retention policy when selecting workloads.
|