tokenrail 0.4.0__tar.gz

tokenrail-0.4.0/.github/workflows/ci.yml

@@ -0,0 +1,37 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Run tests
+        run: uv run --python ${{ matrix.python-version }} pytest -q
+
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+      - name: Ruff check
+        run: uv run ruff check src tests
+
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+      - name: Build sdist and wheel
+        run: uv build

tokenrail-0.4.0/.github/workflows/release.yml

@@ -0,0 +1,43 @@
+name: Release
+
+on:
+  push:
+    tags: ["v*"]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+      - name: Check tag matches package version
+        run: |
+          PKG_VERSION=$(grep -m1 '^version' pyproject.toml | cut -d'"' -f2)
+          TAG_VERSION="${GITHUB_REF_NAME#v}"
+          if [ "$PKG_VERSION" != "$TAG_VERSION" ]; then
+            echo "Tag $GITHUB_REF_NAME does not match pyproject.toml version $PKG_VERSION" >&2
+            exit 1
+          fi
+      - name: Run tests
+        run: uv run pytest -q
+      - name: Build sdist and wheel
+        run: uv build
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/p/tokenrail
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - uses: pypa/gh-action-pypi-publish@release/v1

tokenrail-0.4.0/.gitignore

@@ -0,0 +1,14 @@
+# Python-generated files
+__pycache__/
+*.py[oc]
+build/
+dist/
+wheels/
+*.egg-info
+
+# Virtual environments
+.venv
+
+# Other
+.env
+sample_out*

tokenrail-0.4.0/.python-version

@@ -0,0 +1 @@
+3.12

tokenrail-0.4.0/CHANGELOG.md

@@ -0,0 +1,138 @@
+# Changelog
+
+All notable changes to this project are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
+for its 0.x development releases, with a one-time version reset in 0.4.0.
+Earlier 1.x and 2.x entries are retained as published history; their stable
+designation was premature and does not describe the project's current status.
+
+## [Unreleased]
+
+## [0.4.0] - 2026-10-08
+
+### Added
+
+- GPT-6 Astra, GPT-6.1 Sol, GPT-6 Sol, and GPT-6 Luna catalog entries with
+  standard-tier base prices and cache-write rates.
+- GPT-6 reasoning-effort and sampling-parameter validation for both
+  `responses.create(...)` and `responses.parse(...)`.
+- OpenAI prompt-cache settings are forwarded unchanged for both
+  `responses.create(...)` and `responses.parse(...)`.
+- Conflicting request fields in `extra_body` are rejected before sending.
+- `cache_write_tokens` usage metrics on responses and batch/model statistics.
+
+### Changed
+
+- Reset the package version from 2.0.0 to 0.4.0 and the development status from
+  Production/Stable to Beta. This continues development from the existing code;
+  it does not restore an older implementation. Public API compatibility is not
+  yet guaranteed between 0.x minor releases.
+- GPT-5.6 Sol and the `gpt-5.6` alias use OpenAI's current promotional prices:
+  $4.00 input, $0.40 cached input, $5.00 cache writes, and $20.00 output per
+  million tokens, verified on October 8, 2026.
+- GPT-5.6 cache writes are included in cost estimates at 1.25 times the normal
+  input rate.
+- GPT-5.6 Terra and Luna prices now match OpenAI's August 2026 price cuts,
+  including cached-input and 1.25-times cache-write rates.
+- Prompt caching uses only OpenAI's standard request settings; tokenrail does
+  not plan prefixes, generate or shard keys, prewarm caches, or impose per-key
+  submit limits. `BatchExecutor.prompt_cache` and `PromptCacheConfig` are removed.
+- Progress output reports cache reads and writes independently of request settings.
+- Progress output preserves unknown payer strings instead of displaying `?`,
+  and falls back to `billing.payer` when the cost breakdown has no payer.
+
+## [2.0.0] - 2026-08-03
+
+### Added
+
+- GPT-5.6 Sol, Terra, and Luna capability and base-price catalog entries,
+  including the `gpt-5.6` alias for Sol.
+- `ModelCatalogFallbackWarning` identifies unregistered models and the catalog
+  entries used as capability or pricing fallbacks.
+- Payer state, payer switch count, and raw payer request counts on
+  `StatsSnapshot`.
+
+### Changed
+
+- `RollingMetricsMonitor` now prints a compact request line, explicit payer
+  transitions, periodic summaries, and a final batch summary by default.
+  Pass `verbose=True` to retain the previous progress format.
+- Monitor output supports automatic or forced ANSI emphasis without an added
+  dependency.
+- GPT-5.6 cost estimates use current base standard-tier prices. Long-context,
+  cache-write, Batch/Fast/Flex, and regional price adjustments remain outside
+  the estimator.
+
+## [1.1.0] - 2026-06-14
+
+### Added
+
+- Structured output parsing for batch jobs. `BatchExecutor` now calls
+  `responses.parse(...)` automatically for items that include `text_format`.
+- `client.responses.parse(...)` and `OpenAIProvider.parse(...)`.
+- `NormalizedResponse.output_parsed` and `NormalizedResponse.refusal`.
+- Default JSONL result records now include `output_parsed` and `refusal`.
+- README example for Pydantic structured output batches.
+
+### Changed
+
+- `response_format` remains the low-level JSON Schema path for
+  `responses.create(...)`; `text_format` is the high-level Pydantic parsing path.
+  They are rejected when used together.
+
+## [1.0.0] - 2026-06-11
+
+First stable release. The public API surface is now covered by semantic
+versioning: `RailClient`, `BatchExecutor`, `batch_items_from_queries`,
+`RollingMetricsMonitor`, `ResultsJsonlSink`, `PerRequestJsonSink`,
+`OpenAIProvider`, and the types exported from `tokenrail`.
+
+### Added
+
+- `py.typed` marker — the package now ships inline type information (PEP 561).
+- `tokenrail.__version__`.
+- Docstrings across the public API.
+- Complete packaging metadata (license, classifiers, project URLs).
+- CI test workflow across Python 3.10–3.14.
+- First release published to [PyPI](https://pypi.org/project/tokenrail/);
+  releases are published automatically from `v*` tags via PyPI Trusted
+  Publishing.
+
+### Changed
+
+- Minimum supported Python lowered from 3.11 to 3.10.
+- `catalog.get_model_pricing` no longer carries a dead `service_tier` branch;
+  non-default tiers explicitly fall back to default-tier pricing.
+
+## [0.2.1] - 2026-06-10
+
+### Added
+
+- Client-side RPM and TPM submit throttling in `BatchExecutor`
+  (`max_rpm` / `max_tpm`).
+
+### Removed
+
+- vLLM provider support; the library is OpenAI-only.
+
+## [0.1.3] - 2026-05-20
+
+### Changed
+
+- Removed the custom OpenAI retry loop in favor of the SDK's built-in
+  `max_retries`.
+
+## [0.1.2] - 2026-05-15
+
+### Added
+
+- ETA progress reporting in `RollingMetricsMonitor`.
+
+## [0.1.0] - 2026-05-13
+
+### Added
+
+- Initial release: `RailClient`, `BatchExecutor`, rolling metrics monitor,
+  JSONL and per-request sinks, model capability/pricing catalog.

tokenrail-0.4.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Takumi Shibata
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

tokenrail-0.4.0/PKG-INFO

@@ -0,0 +1,338 @@
+Metadata-Version: 2.5
+Name: tokenrail
+Version: 0.4.0
+Summary: Thin client and batch execution helpers for OpenAI Responses API workloads.
+Project-URL: Homepage, https://github.com/takumi0shibata/tokenrail
+Project-URL: Repository, https://github.com/takumi0shibata/tokenrail
+Project-URL: Issues, https://github.com/takumi0shibata/tokenrail/issues
+Project-URL: Changelog, https://github.com/takumi0shibata/tokenrail/blob/main/CHANGELOG.md
+Author: Takumi Shibata
+License-Expression: MIT
+License-File: LICENSE
+Keywords: batch,cost-tracking,llm,openai,rate-limit,responses-api,token
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: openai>=1.108.0
+Description-Content-Type: text/markdown
+
+# tokenrail
+
+[![CI](https://github.com/takumi0shibata/tokenrail/actions/workflows/ci.yml/badge.svg)](https://github.com/takumi0shibata/tokenrail/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/tokenrail)](https://pypi.org/project/tokenrail/)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://pypi.org/project/tokenrail/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+
+`tokenrail` is a small Python library for running OpenAI Responses API jobs with a `client.responses`-style surface.
+
+It focuses on:
+
+- thread-based OpenAI batch execution
+- structured output parsing for Pydantic models
+- client-side RPM / TPM submit throttling
+- OpenAI prompt-cache settings passed through unchanged, with read/write usage tracking
+- per-model token / cost monitoring with ETA progress reporting
+- resumable JSONL and per-request result writing
+
+Fully typed (PEP 561), supports Python 3.10+.
+
+The library is in beta. The public API is still evolving, and breaking changes
+may occur between 0.x minor versions.
+
+## Installation
+
+```bash
+uv add tokenrail
+# or
+pip install tokenrail
+```
+
+To track an unreleased revision instead, depend on the Git repository directly:
+
+```toml
+[tool.uv.sources]
+tokenrail = { git = "https://github.com/takumi0shibata/tokenrail", branch = "main" }
+```
+
+Set your OpenAI API key in the consuming project before use:
+
+```bash
+export OPENAI_API_KEY=...
+```
+
+## Quick start
+
+```python
+from tokenrail import BatchExecutor, ResultsJsonlSink, PerRequestJsonSink, RailClient, RollingMetricsMonitor
+from tokenrail.executor import batch_items_from_queries
+
+client = RailClient.openai(max_retries=6)
+
+queries = {
+    "1": [{"role": "user", "content": "Summarize this paper in 3 bullets."}],
+    "2": [{"role": "user", "content": "Extract the key assumptions."}],
+}
+
+items = batch_items_from_queries(
+    queries,
+    model="gpt-5.4-mini-2026-03-17",
+    reasoning_effort="medium",
+    verbosity="low",
+)
+
+# Consolidate only the necessary elements from all processing results into a single file.
+result_sink = ResultsJsonlSink(
+    "out/results.jsonl",
+    projector=lambda response: {
+        "id": response.id,
+        "text": response.output_text,
+        "model": response.model,
+        "usage": response.usage.to_dict(),
+    },
+)
+# Save the raw output of each query.
+per_request_sink = PerRequestJsonSink("out/")
+
+executor = BatchExecutor(
+    client=client,
+    max_workers=16,
+    max_rpm=500,
+    max_tpm=200_000,
+    sinks=[result_sink, per_request_sink],
+    monitor=RollingMetricsMonitor(),
+)
+
+stats = executor.run(items)
+print(stats.to_dict())
+```
+
+## Prompt caching
+
+tokenrail forwards OpenAI's prompt-cache settings and input content as supplied.
+It does not detect shared prefixes, insert breakpoints, generate or split keys,
+prewarm caches, or impose cache-specific submit limits. These settings work with
+both `responses.create(...)` and `responses.parse(...)`, including batch items.
+
+| Setting | Purpose |
+| --- | --- |
+| `prompt_cache_options` | Cache mode, TTL, prewarming, and diagnostics where supported |
+| `prompt_cache_breakpoint` inside `input` content | The end of a reusable prefix |
+| `prompt_cache_key` | A caller-supplied key, preserved exactly |
+| `prompt_cache_retention` | Retention for older models that support it |
+
+Omitting these settings keeps the OpenAI API defaults, including implicit
+caching. On GPT-5.6 and later, writing an eligible prefix costs 1.25 times the
+ordinary input rate, so a prefix that is never reused can cost more than an
+uncached input. Setting support and cache eligibility depend on the model.
+
+Use the API defaults:
+
+```python
+from tokenrail import RailClient
+
+client = RailClient.openai()
+response = client.responses.create(model="gpt-6.1-sol", input="Summarize this paper.")
+```
+
+Cache only a reusable prefix on GPT-5.6 and later. Put the shared content in its
+own block and mark its end; content after the breakpoint stays outside the cache
+write. The prefix must render to at least 1,024 visible input tokens. Top-level
+`instructions` cannot contain a breakpoint, so supply reusable instructions in
+a developer content block:
+
+```python
+from tokenrail import BatchExecutor, batch_items_from_queries
+
+shared_block = {
+    "type": "input_text",
+    "text": "Shared rubric, examples, and reference material ...",  # At least 1,024 tokens.
+    "prompt_cache_breakpoint": {"mode": "explicit"},
+}
+items = batch_items_from_queries(
+    {
+        item_id: [
+            {"role": "developer", "content": [shared_block]},
+            {"role": "user", "content": question},
+        ]
+        for item_id, question in {
+            "paper-1": "Summarize paper one.",
+            "paper-2": "Summarize paper two.",
+        }.items()
+    },
+    model="gpt-6.1-sol",
+    prompt_cache_options={"mode": "explicit"},
+)
+stats = BatchExecutor(client=client, max_workers=16, max_rpm=120).run(items)
+print(stats.cached_tokens)       # Tokens read from cache.
+print(stats.cache_write_tokens)  # Tokens newly written to cache.
+```
+
+Avoid prompt-cache reads and writes on GPT-5.6 and later by using explicit mode
+with no breakpoints:
+
+```python
+response = client.responses.create(
+    model="gpt-6.1-sol",
+    input="A one-off prompt ...",
+    prompt_cache_options={"mode": "explicit"},
+)
+```
+
+Choose keys, breakpoints, and reuse according to your workload. When supplying a
+key, keep it stable across related requests. tokenrail records cache read/write
+tokens and estimates their costs; it does not guarantee a hit or a saving.
+Conflicting values supplied in normal request arguments and `extra_body` raise
+`ValueError` before sending. `store=False` controls response storage, not prompt
+caching. See the [OpenAI prompt caching guide](https://developers.openai.com/api/docs/guides/prompt-caching)
+for current model support, pricing, and cache lifetime.
+
+## Structured output batches
+
+Pass a Pydantic model as `text_format` when building batch items. `BatchExecutor`
+will call `responses.parse(...)` for those items and store the validated object
+on `response.output_parsed`.
+
+```python
+from pydantic import BaseModel
+
+from tokenrail import BatchExecutor, RailClient, ResultsJsonlSink
+from tokenrail.executor import batch_items_from_queries
+
+
+class PaperSummary(BaseModel):
+    title: str
+    key_assumptions: list[str]
+
+
+client = RailClient.openai(max_retries=6)
+
+items = batch_items_from_queries(
+    {
+        "paper-1": [{"role": "user", "content": "Extract the title and assumptions from this paper: ..."}],
+        "paper-2": [{"role": "user", "content": "Extract the title and assumptions from this paper: ..."}],
+    },
+    model="gpt-5.4-mini-2026-03-17",
+    reasoning_effort="medium",
+    text_format=PaperSummary,
+)
+
+sink = ResultsJsonlSink(
+    "out/structured-results.jsonl",
+    projector=lambda response: {
+        "id": response.id,
+        "summary": response.output_parsed.model_dump(mode="json") if response.output_parsed else None,
+        "refusal": response.refusal,
+        "usage": response.usage.to_dict(),
+    },
+)
+
+stats = BatchExecutor(client=client, sinks=[sink], max_workers=16).run(items)
+print(stats.to_dict())
+```
+
+Use `response_format={...}` with `client.responses.create(...)` when you want to
+provide a raw JSON Schema yourself. Use `text_format=YourModel` for Pydantic
+parsing; `response_format` and `text_format` cannot be used together. Do not set
+`verbosity` on structured output batches that use `text_format`, because the
+OpenAI SDK's `responses.parse(...)` path does not accept that combination.
+
+## Configuration notes
+
+- `max_retries` configures the OpenAI Python SDK client's built-in retry behavior. `tokenrail` does not add its own retry loop on top.
+- `max_rpm` and `max_tpm` are optional client-side submit limits. When a limit is set, `BatchExecutor` waits before submitting more work instead of raising its effective concurrency above the configured rate.
+- Request failures are captured as error records (written to sinks and counted in stats) rather than raised, so one failing item does not abort the batch.
+- `base_url` is passed through to the OpenAI Python SDK for callers that need an SDK-level custom endpoint.
+
+## Resume behavior
+
+`BatchExecutor` reads completed ids from the first configured sink before it starts. Re-running the same job with the same output path skips records that are already present, then writes only the remaining requests.
+
+If you use a custom `projector` with `ResultsJsonlSink`, make sure it keeps an `"id"` field — resume relies on it.
+
+## Progress output
+
+`RollingMetricsMonitor` keeps request-specific details short and prints batch
+metrics separately:
+
+```text
+tokenrail · 100 requests
+   PAYER: openai — costs are covered
+  0001  ok   req-0001      model=gpt-5.6-terra  1.3k tok (40% cached / 0% cache-write)  $0.002000  oai  1.4s
+── 50/100 · 50% · 00:00:14 · ETA 00:00:14 · 58 rpm · 74k tpm · $0.100 (oai 100% / dev $0.000) · cache r40%/w2%
+!! PAYER SWITCH: openai → developer at 53/100 (00:00:15) — now billed to you
+  0053  ok   req-0053      1.2k tok (38% cached / 0% cache-write)  $0.001900  DEV  1.1s
+Done 100/100 · 99 ok / 1 errors · 00:00:29
+Total $0.198 — openai $0.104 (53%) / developer $0.094 (47%)
+Prompt cache: 48k read / 2.4k written
+Payer switches: 1
+```
+
+Periodic summaries are emitted every 50 requests or 30 seconds by default.
+Configure them with `summary_every` and `summary_interval`, adjust payer
+transition hysteresis with `payer_switch_threshold`, and control ANSI styling
+with `color`. Pass `verbose=True` to use the legacy `[n/total] id=...` format.
+`printer=None` still disables all monitor output.
+
+Request lines show `oai` for `openai`, `DEV` for `developer`, and the original
+string for any other payer value. A missing or empty payer is shown as `?`.
+The monitor reads the payer from `cost`, falling back to `billing.payer` when
+unavailable. Unknown payer values count toward `unknown_payer_requests` but
+do not drive payer transitions.
+
+## Cost tracking
+
+Standard-tier base prices in USD per million tokens, checked against the
+[official OpenAI pricing page](https://developers.openai.com/api/docs/pricing)
+on October 8, 2026:
+
+| Model | Input | Cached input | Cache writes | Output |
+| --- | ---: | ---: | ---: | ---: |
+| `gpt-6-astra` | $10.00 | $1.00 | $12.50 | $50.00 |
+| `gpt-6.1-sol` | $2.00 | $0.10 | $2.50 | $10.00 |
+| `gpt-6-sol` | $2.00 | $0.20 | $2.50 | $10.00 |
+| `gpt-6-luna` | $0.10 | $0.01 | $0.125 | $0.50 |
+| `gpt-5.6-sol` / `gpt-5.6` | $4.00 | $0.40 | $5.00 | $20.00 |
+| `gpt-5.6-terra` | $2.00 | $0.20 | $2.50 | $12.00 |
+| `gpt-5.6-luna` | $0.20 | $0.02 | $0.25 | $1.20 |
+
+GPT-5.6 Sol uses the current promotional rate, available at least through
+November 21, 2026 according to OpenAI.
+
+- GPT-5.6 and GPT-6 cache writes are estimated at 1.25 times each model's ordinary input rate and are tracked separately from cache reads.
+- Costs use the checked-in base, standard-tier pricing table (`tokenrail.catalog`). The estimate does not apply GPT-5.6/GPT-6 long-context rates above 272K input tokens, Batch/Fast/Flex/Ultrafast pricing, or regional uplifts. The official pricing page is authoritative.
+- Models without a pricing entry get `cost=None`. If an unregistered model partially matches an older catalog entry, tokenrail emits `ModelCatalogFallbackWarning` once per model and names the capability and pricing entries used as fallbacks.
+- OpenAI cost allocation is inferred from `billing.payer` in the response body. When `payer == "openai"`, the nominal request cost is counted as OpenAI-covered rather than developer-billed.
+- `reasoning_effort` is gated to supported `gpt-5`, `gpt-6`, and `o`-series models in the checked-in capability registry.
+- GPT-6 Astra and GPT-6.1 Sol support `low`, `medium`, `high`, `xhigh`, and `max` reasoning efforts; `temperature` and `top_p` are rejected before sending a request. GPT-6 Sol and Luna additionally support `none`, which must be explicitly selected to use `temperature` or `top_p`. These checks apply to both `responses.create(...)` and `responses.parse(...)`. See the [GPT-6 guide](https://developers.openai.com/api/docs/guides/latest-model).
+
+Fallback warnings can be filtered with Python's standard warning controls:
+
+```python
+import warnings
+
+from tokenrail.catalog import ModelCatalogFallbackWarning
+
+warnings.filterwarnings("ignore", category=ModelCatalogFallbackWarning)
+```
+
+## Development
+
+```bash
+uv sync
+uv run pytest
+uv run ruff check src tests
+```
+
+## License
+
+[MIT](LICENSE)