typellm 0.3.0.dev1__tar.gz → 0.3.1__tar.gz

{typellm-0.3.0.dev1/typellm.egg-info → typellm-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: typellm
-Version: 0.3.0.dev1
+Version: 0.3.1
 Summary: Type-Safe Decoding for Autoregressive LLMs
 License-Expression: Apache-2.0
 Project-URL: Homepage, https://typellm.ai/
@@ -23,12 +23,14 @@ Dynamic: license-file
   <a href="https://typellm.ai/">Homepage</a>&nbsp; • &nbsp;
   <a href="https://typellm.ai/blog">Blog</a>&nbsp; • &nbsp;
   <a href="https://typellm.ai/docs">Docs</a>&nbsp; • &nbsp;
-  <a href="https://typellm.ai/early-access">Early Access</a>&nbsp; • &nbsp;
+  <a href="https://typellm.ai/examples">Examples</a>&nbsp; • &nbsp;
+  <a href="https://typellm.ai/dashboard">API</a>&nbsp; • &nbsp;
   <a href="https://typellm.ai/contact">Contact</a>&nbsp;
 </h4>
 </div>
 
 ## Updates
+- **[2026/09/29]** The [TypeLLM API](https://typellm.ai) is live: typed outputs without serving a model yourself, and $5 of credit for new accounts. [Get early access](https://typellm.ai/early-access) · [Quick start](#typellm-api-cloud) · [Examples](https://typellm.ai/examples).
 - **[2026/09/24]** Added [image input](#image-input) for vision-language models, tested with Qwen3.8-27B.
 - **[2026/09/23]** Added [JevBench results](https://github.com/TypeLLM/TypeLLM/blob/main/evals/jevbench/README.md): TypeLLM scored 195/231 without thinking and 228/231 with thinking.
 - **[2026/09/23]** Added [permutation averaging](#per-question-permutation-averaging) to improve the predictive distribution. See the [blog post](https://typellm.ai/blog/fair-die).
@@ -66,37 +68,33 @@ Evaluated on 231 public [JevBench](https://github.com/fstandhartinger/jevbench)
 
 ## Quick start
 
-### Hosted API (early access)
+```bash
+pip install -U typellm
+```
 
-No GPU? [Early-access](https://typellm.ai/early-access) users get an API key
-by email, and the hosted API runs the model:
+Call the hosted TypeLLM API, or run TypeLLM against a model you serve yourself.
+Both clients take the same requests.
 
-```python
-from typellm import TypeLLMClient
+### TypeLLM API (cloud)
+
+[Sign in](https://typellm.ai/login), enter your access code on the
+[API keys](https://typellm.ai/dashboard/keys) page and create a key. No code yet?
+[Get early access](https://typellm.ai/early-access).
 
-client = TypeLLMClient(api_key="YOUR_API_KEY")  # calls https://api.typellm.ai
+```bash
+export TYPELLM_API_KEY="tl-sk-..."
 ```
 
-`generate()` then works as in the example below, with these differences:
+```python
+import os
+from typellm import TypeLLMClient
 
-- The service compiles and runs the schema. Use `model`, `mode`, `temperature`
-  (for sampling), `seed` and `timeout`; local compilation settings do not apply.
-  A call runs for at most 300 seconds, 60 by default. Set `timeout` on the
-  client for a new default, or on `generate()` to override it for one call.
-- Pass `questions`; `schema=`, `cancel` and `print_final_prompt` need your
-  own server. Images can be files, bytes, PIL images or data: URIs, not URLs.
-- Each key runs a limited number of calls at once, told with the key. Calls
-  beyond it fail right away with HTTP 429.
-- Errors raise `SGLangError` with the HTTP status in `.status`, and a call
-  that runs out of time raises `GenerationTimeout`. The service caps the
-  context size and the images, answering 400 past them, and stops string
-  answers at 128 tokens (`maxLength` is refused).
-- Successful calls set `last_usage` from the service's input and thinking token
-  counts. Failed hosted calls may not return usage.
+client = TypeLLMClient(api_key=os.environ["TYPELLM_API_KEY"])
+```
 
-To run on your own GPU instead, start here:
+See the [API reference](https://typellm.ai/docs/api) to call it over HTTP instead.
 
-### 1. Serve a model with SGLang
+### Self-hosted with SGLang
 
 Use [SGLang](https://github.com/sgl-project/sglang) to configure and serve a
 compatible autoregressive model on your local GPU server. This example uses
@@ -106,12 +104,6 @@ to start it with prefix caching enabled.
 
 See [Supported models](#supported-models) for tested checkpoints and thinking behavior.
 
-### 2. Run TypeLLM
-
-```bash
-pip install -U typellm
-```
-
 Point `TypeLLMClient` at the SGLang server's HTTP endpoint:
 
 ```python
@@ -123,10 +115,12 @@ client = TypeLLMClient(
 )
 ```
 
-Example request:
+### Make a request
+
+With either client:
 
 ```python
-result = client.generate(
+response = client.generate(
     context="""
     Receipt from Hilton London
     Total: £324.50
@@ -158,10 +152,12 @@ result = client.generate(
     },
 )
 
-print(result)
+print(response.result)
 ```
 
-Example output:
+`generate()` returns what the HTTP API does: the typed answers in `.result`, the
+reasoning of each field that thought in `.thinking`, and the call's tokens in
+`.usage`. Example `.result`:
 
 ```python
 {
@@ -190,7 +186,7 @@ Enum choices support `string`, `integer`, and `number` types, with at most 24 va
 A string without `enum` generates free text:
 
 ```python
-result = client.generate(
+response = client.generate(
     context="The train ticket is for a client meeting.",
     questions={
         "summary": {"type": "string", "instructions": "Summarize in one sentence."},
@@ -199,12 +195,13 @@ result = client.generate(
 ```
 
 Free text stops after 128 tokens. Pass `text_max_tokens=` to the client for
-longer answers, or set `maxLength` on a field to cap its characters.
+longer answers, and ask for the length you want in the field's instructions.
+`maxLength` is not supported.
 
 Ask for a numeric answer without enumerating every possible value:
 
 ```python
-result = client.generate(
+response = client.generate(
     context="Calculate the requested value accurately.",
     questions={
         "answer": {
@@ -214,7 +211,7 @@ result = client.generate(
     },
 )
 
-print(result)
+print(response.result)
 # {"answer": 70.0}
 ```
 
@@ -240,7 +237,7 @@ Add `"null"` to the type to allow a missing value. The field returns `None`
 when the input has no value for it:
 
 ```python
-result = client.generate(
+response = client.generate(
     context="Read the attached receipt.",
     images=["receipt.jpg"],
     questions={
@@ -266,7 +263,7 @@ Thinking is off by default. Turn it on for the fields that need it; the others
 answer at once, and the fields that think reason side by side:
 
 ```python
-result = client.generate(context=context, questions={
+response = client.generate(context=context, questions={
     "total": {"type": "number"},
     "category": {"type": "string", "enum": ["meal", "travel", "equipment"]},
     "policy_ok": {"type": "boolean", "instructions": "Does it meet the travel policy?",
@@ -274,13 +271,12 @@ result = client.generate(context=context, questions={
 })
 ```
 
-`thinking_budget` caps a field's reasoning; there is no budget by default.
-`TypeLLMClient(..., thinking_budget=2048)` sets one for every field that thinks
-without its own. When reasoning reaches the budget, TypeLLM closes it and moves
-on to the typed answer.
+`thinking_budget` caps a field's reasoning. Without one, a field may reason until
+the model's context is full. When reasoning reaches the budget, TypeLLM closes it
+and moves on to the typed answer.
 
-After a call, `client.last_thinking` maps each field that thought to its
-reasoning, and `client.last_usage.thinking_tokens` counts the reasoning tokens.
+`response.thinking` maps each field that thought to its reasoning, and
+`response.usage.thinking_tokens` counts the reasoning tokens.
 
 Models with always-on thinking reason on every field; `thinking_budget` applies
 to them too. See [Supported models](#supported-models).
@@ -291,7 +287,7 @@ Pass images with `images=` alongside the text context. The served model must be
 a vision-language model, such as `Qwen/Qwen3.8-27B`.
 
 ```python
-result = client.generate(
+response = client.generate(
     context="The customer says this receipt was charged twice.",
     images=["receipt.png"],
     questions={
@@ -311,7 +307,7 @@ Fields run together by default, and each sees only the original context.
 When a field needs earlier results, list them in `depends_on`:
 
 ```python
-result = client.generate(
+response = client.generate(
     context="The payments service is returning errors after a deployment.",
     questions={
         "system": {
@@ -349,7 +345,7 @@ step reuses its parent's cached prompt. Unknown names and cycles raise
 Set `return_probabilities` on individual enum or boolean fields:
 
 ```python
-result = client.generate(
+response = client.generate(
     context=context,
     questions={
         "expense_type": {
@@ -377,19 +373,20 @@ result = client.generate(
 Only opted-in fields return `value` and `probabilities`; other fields return plain values.
 The option is not supported on open Numeric or Text fields.
 
-Argmax is the default. To enable sampling:
+`temperature` is 0 by default: each field gets its most likely answer. Above 0,
+TypeLLM samples at that temperature:
 
 ```python
 client = TypeLLMClient(
     "http://127.0.0.1:30000",
-    mode="sample",
     temperature=0.8,
     seed=42,
 )
 ```
 
 Sampling applies to finite candidates for Choice fields and to token generation
-for Numeric and Text fields. `temperature` controls sampling in each case.
+for Numeric and Text fields. `generate(temperature=...)` sets it for one call.
+The older `mode="argmax"` / `mode="sample"` still works for now but is deprecated.
 
 A seed fixes TypeLLM's own random choices. SGLang's log probabilities can shift
 with its prefix cache and batching, so the same seed may still give different
@@ -400,7 +397,7 @@ For a one-off request, use the convenience function:
 ```python
 from typellm import run_schema
 
-result = run_schema(
+response = run_schema(
     context=context,
     questions=questions,
     base_url="http://127.0.0.1:30000",
@@ -413,7 +410,7 @@ result = run_schema(
 Add `permutations` to an `enum` question to reduce option-order bias. TypeLLM averages the probabilities and keeps the same return format.
 
 ```python
-result = client.generate(
+response = client.generate(
     context="A single roll of a fair die.",
     questions={"roll": {
         "type": "string",
@@ -440,25 +437,25 @@ support this option.
 ## Serving
 
 One `TypeLLMClient` can be shared by many threads. Each call keeps its own
-prompts and usage, and connections to SGLang are reused.
+prompts and usage, and connections to SGLang or the hosted API are reused.
 
 ```python
 import threading
 
 cancel = threading.Event()
-result = client.generate(
+response = client.generate(
     context=context,
     questions=questions,
     seed=7,          # this call's random choices only
     timeout=30,      # seconds for the whole call; raises GenerationTimeout
     cancel=cancel,   # set it from another thread; raises GenerationCancelled
 )
-print(client.last_usage)
-# Usage(requests=4, prompt_tokens=1830, cached_tokens=1504, completion_tokens=7, thinking_tokens=0, input_tokens=410)
+print(response.usage)
+# Usage(input_tokens=410, thinking_tokens=0, requests=4, prompt_tokens=1830, cached_tokens=1504, completion_tokens=7)
 ```
 
-`last_usage` reports the tokens of the last call made in the current thread,
-including a call that failed partway. `input_tokens` is what you sent, each part
+`usage` reports the tokens of the call. When a call fails partway, the
+exception's `.usage` holds what it spent. `input_tokens` is what you sent, each part
 counted once: the context, the questions as JSON, and the images. The other
 counts are SGLang's for the call's requests, where every prompt carries the
 shared context. A timeout or cancel stops
@@ -497,8 +494,9 @@ Other sizes in the Qwen3.5 and Qwen3.8 families are expected to be compatible.
 
 [Image input](#image-input) has been tested with `Qwen/Qwen3.8-27B`.
 
-Use the checkpoint ID as `model=`. If the server's tokenizer path is unavailable
-locally, set `tokenizer=` to its matching Hugging Face ID or local directory.
+Use the checkpoint ID as `model=`. TypeLLM loads the tokenizer from the server's
+paths, then its served model name; if none of those loads locally, set
+`tokenizer=` to the matching Hugging Face ID or local directory.
 The tokenizer must load from standard artifacts without custom model code.
 
 ## Comparison with Jev-style models

{typellm-0.3.0.dev1 → typellm-0.3.1}/README.md

@@ -8,12 +8,14 @@
   <a href="https://typellm.ai/">Homepage</a>&nbsp; • &nbsp;
   <a href="https://typellm.ai/blog">Blog</a>&nbsp; • &nbsp;
   <a href="https://typellm.ai/docs">Docs</a>&nbsp; • &nbsp;
-  <a href="https://typellm.ai/early-access">Early Access</a>&nbsp; • &nbsp;
+  <a href="https://typellm.ai/examples">Examples</a>&nbsp; • &nbsp;
+  <a href="https://typellm.ai/dashboard">API</a>&nbsp; • &nbsp;
   <a href="https://typellm.ai/contact">Contact</a>&nbsp;
 </h4>
 </div>
 
 ## Updates
+- **[2026/09/29]** The [TypeLLM API](https://typellm.ai) is live: typed outputs without serving a model yourself, and $5 of credit for new accounts. [Get early access](https://typellm.ai/early-access) · [Quick start](#typellm-api-cloud) · [Examples](https://typellm.ai/examples).
 - **[2026/09/24]** Added [image input](#image-input) for vision-language models, tested with Qwen3.8-27B.
 - **[2026/09/23]** Added [JevBench results](https://github.com/TypeLLM/TypeLLM/blob/main/evals/jevbench/README.md): TypeLLM scored 195/231 without thinking and 228/231 with thinking.
 - **[2026/09/23]** Added [permutation averaging](#per-question-permutation-averaging) to improve the predictive distribution. See the [blog post](https://typellm.ai/blog/fair-die).
@@ -51,37 +53,33 @@ Evaluated on 231 public [JevBench](https://github.com/fstandhartinger/jevbench)
 
 ## Quick start
 
-### Hosted API (early access)
+```bash
+pip install -U typellm
+```
 
-No GPU? [Early-access](https://typellm.ai/early-access) users get an API key
-by email, and the hosted API runs the model:
+Call the hosted TypeLLM API, or run TypeLLM against a model you serve yourself.
+Both clients take the same requests.
 
-```python
-from typellm import TypeLLMClient
+### TypeLLM API (cloud)
+
+[Sign in](https://typellm.ai/login), enter your access code on the
+[API keys](https://typellm.ai/dashboard/keys) page and create a key. No code yet?
+[Get early access](https://typellm.ai/early-access).
 
-client = TypeLLMClient(api_key="YOUR_API_KEY")  # calls https://api.typellm.ai
+```bash
+export TYPELLM_API_KEY="tl-sk-..."
 ```
 
-`generate()` then works as in the example below, with these differences:
+```python
+import os
+from typellm import TypeLLMClient
 
-- The service compiles and runs the schema. Use `model`, `mode`, `temperature`
-  (for sampling), `seed` and `timeout`; local compilation settings do not apply.
-  A call runs for at most 300 seconds, 60 by default. Set `timeout` on the
-  client for a new default, or on `generate()` to override it for one call.
-- Pass `questions`; `schema=`, `cancel` and `print_final_prompt` need your
-  own server. Images can be files, bytes, PIL images or data: URIs, not URLs.
-- Each key runs a limited number of calls at once, told with the key. Calls
-  beyond it fail right away with HTTP 429.
-- Errors raise `SGLangError` with the HTTP status in `.status`, and a call
-  that runs out of time raises `GenerationTimeout`. The service caps the
-  context size and the images, answering 400 past them, and stops string
-  answers at 128 tokens (`maxLength` is refused).
-- Successful calls set `last_usage` from the service's input and thinking token
-  counts. Failed hosted calls may not return usage.
+client = TypeLLMClient(api_key=os.environ["TYPELLM_API_KEY"])
+```
 
-To run on your own GPU instead, start here:
+See the [API reference](https://typellm.ai/docs/api) to call it over HTTP instead.
 
-### 1. Serve a model with SGLang
+### Self-hosted with SGLang
 
 Use [SGLang](https://github.com/sgl-project/sglang) to configure and serve a
 compatible autoregressive model on your local GPU server. This example uses
@@ -91,12 +89,6 @@ to start it with prefix caching enabled.
 
 See [Supported models](#supported-models) for tested checkpoints and thinking behavior.
 
-### 2. Run TypeLLM
-
-```bash
-pip install -U typellm
-```
-
 Point `TypeLLMClient` at the SGLang server's HTTP endpoint:
 
 ```python
@@ -108,10 +100,12 @@ client = TypeLLMClient(
 )
 ```
 
-Example request:
+### Make a request
+
+With either client:
 
 ```python
-result = client.generate(
+response = client.generate(
     context="""
     Receipt from Hilton London
     Total: £324.50
@@ -143,10 +137,12 @@ result = client.generate(
     },
 )
 
-print(result)
+print(response.result)
 ```
 
-Example output:
+`generate()` returns what the HTTP API does: the typed answers in `.result`, the
+reasoning of each field that thought in `.thinking`, and the call's tokens in
+`.usage`. Example `.result`:
 
 ```python
 {
@@ -175,7 +171,7 @@ Enum choices support `string`, `integer`, and `number` types, with at most 24 va
 A string without `enum` generates free text:
 
 ```python
-result = client.generate(
+response = client.generate(
     context="The train ticket is for a client meeting.",
     questions={
         "summary": {"type": "string", "instructions": "Summarize in one sentence."},
@@ -184,12 +180,13 @@ result = client.generate(
 ```
 
 Free text stops after 128 tokens. Pass `text_max_tokens=` to the client for
-longer answers, or set `maxLength` on a field to cap its characters.
+longer answers, and ask for the length you want in the field's instructions.
+`maxLength` is not supported.
 
 Ask for a numeric answer without enumerating every possible value:
 
 ```python
-result = client.generate(
+response = client.generate(
     context="Calculate the requested value accurately.",
     questions={
         "answer": {
@@ -199,7 +196,7 @@ result = client.generate(
     },
 )
 
-print(result)
+print(response.result)
 # {"answer": 70.0}
 ```
 
@@ -225,7 +222,7 @@ Add `"null"` to the type to allow a missing value. The field returns `None`
 when the input has no value for it:
 
 ```python
-result = client.generate(
+response = client.generate(
     context="Read the attached receipt.",
     images=["receipt.jpg"],
     questions={
@@ -251,7 +248,7 @@ Thinking is off by default. Turn it on for the fields that need it; the others
 answer at once, and the fields that think reason side by side:
 
 ```python
-result = client.generate(context=context, questions={
+response = client.generate(context=context, questions={
     "total": {"type": "number"},
     "category": {"type": "string", "enum": ["meal", "travel", "equipment"]},
     "policy_ok": {"type": "boolean", "instructions": "Does it meet the travel policy?",
@@ -259,13 +256,12 @@ result = client.generate(context=context, questions={
 })
 ```
 
-`thinking_budget` caps a field's reasoning; there is no budget by default.
-`TypeLLMClient(..., thinking_budget=2048)` sets one for every field that thinks
-without its own. When reasoning reaches the budget, TypeLLM closes it and moves
-on to the typed answer.
+`thinking_budget` caps a field's reasoning. Without one, a field may reason until
+the model's context is full. When reasoning reaches the budget, TypeLLM closes it
+and moves on to the typed answer.
 
-After a call, `client.last_thinking` maps each field that thought to its
-reasoning, and `client.last_usage.thinking_tokens` counts the reasoning tokens.
+`response.thinking` maps each field that thought to its reasoning, and
+`response.usage.thinking_tokens` counts the reasoning tokens.
 
 Models with always-on thinking reason on every field; `thinking_budget` applies
 to them too. See [Supported models](#supported-models).
@@ -276,7 +272,7 @@ Pass images with `images=` alongside the text context. The served model must be
 a vision-language model, such as `Qwen/Qwen3.8-27B`.
 
 ```python
-result = client.generate(
+response = client.generate(
     context="The customer says this receipt was charged twice.",
     images=["receipt.png"],
     questions={
@@ -296,7 +292,7 @@ Fields run together by default, and each sees only the original context.
 When a field needs earlier results, list them in `depends_on`:
 
 ```python
-result = client.generate(
+response = client.generate(
     context="The payments service is returning errors after a deployment.",
     questions={
         "system": {
@@ -334,7 +330,7 @@ step reuses its parent's cached prompt. Unknown names and cycles raise
 Set `return_probabilities` on individual enum or boolean fields:
 
 ```python
-result = client.generate(
+response = client.generate(
     context=context,
     questions={
         "expense_type": {
@@ -362,19 +358,20 @@ result = client.generate(
 Only opted-in fields return `value` and `probabilities`; other fields return plain values.
 The option is not supported on open Numeric or Text fields.
 
-Argmax is the default. To enable sampling:
+`temperature` is 0 by default: each field gets its most likely answer. Above 0,
+TypeLLM samples at that temperature:
 
 ```python
 client = TypeLLMClient(
     "http://127.0.0.1:30000",
-    mode="sample",
     temperature=0.8,
     seed=42,
 )
 ```
 
 Sampling applies to finite candidates for Choice fields and to token generation
-for Numeric and Text fields. `temperature` controls sampling in each case.
+for Numeric and Text fields. `generate(temperature=...)` sets it for one call.
+The older `mode="argmax"` / `mode="sample"` still works for now but is deprecated.
 
 A seed fixes TypeLLM's own random choices. SGLang's log probabilities can shift
 with its prefix cache and batching, so the same seed may still give different
@@ -385,7 +382,7 @@ For a one-off request, use the convenience function:
 ```python
 from typellm import run_schema
 
-result = run_schema(
+response = run_schema(
     context=context,
     questions=questions,
     base_url="http://127.0.0.1:30000",
@@ -398,7 +395,7 @@ result = run_schema(
 Add `permutations` to an `enum` question to reduce option-order bias. TypeLLM averages the probabilities and keeps the same return format.
 
 ```python
-result = client.generate(
+response = client.generate(
     context="A single roll of a fair die.",
     questions={"roll": {
         "type": "string",
@@ -425,25 +422,25 @@ support this option.
 ## Serving
 
 One `TypeLLMClient` can be shared by many threads. Each call keeps its own
-prompts and usage, and connections to SGLang are reused.
+prompts and usage, and connections to SGLang or the hosted API are reused.
 
 ```python
 import threading
 
 cancel = threading.Event()
-result = client.generate(
+response = client.generate(
     context=context,
     questions=questions,
     seed=7,          # this call's random choices only
     timeout=30,      # seconds for the whole call; raises GenerationTimeout
     cancel=cancel,   # set it from another thread; raises GenerationCancelled
 )
-print(client.last_usage)
-# Usage(requests=4, prompt_tokens=1830, cached_tokens=1504, completion_tokens=7, thinking_tokens=0, input_tokens=410)
+print(response.usage)
+# Usage(input_tokens=410, thinking_tokens=0, requests=4, prompt_tokens=1830, cached_tokens=1504, completion_tokens=7)
 ```
 
-`last_usage` reports the tokens of the last call made in the current thread,
-including a call that failed partway. `input_tokens` is what you sent, each part
+`usage` reports the tokens of the call. When a call fails partway, the
+exception's `.usage` holds what it spent. `input_tokens` is what you sent, each part
 counted once: the context, the questions as JSON, and the images. The other
 counts are SGLang's for the call's requests, where every prompt carries the
 shared context. A timeout or cancel stops
@@ -482,8 +479,9 @@ Other sizes in the Qwen3.5 and Qwen3.8 families are expected to be compatible.
 
 [Image input](#image-input) has been tested with `Qwen/Qwen3.8-27B`.
 
-Use the checkpoint ID as `model=`. If the server's tokenizer path is unavailable
-locally, set `tokenizer=` to its matching Hugging Face ID or local directory.
+Use the checkpoint ID as `model=`. TypeLLM loads the tokenizer from the server's
+paths, then its served model name; if none of those loads locally, set
+`tokenizer=` to the matching Hugging Face ID or local directory.
 The tokenizer must load from standard artifacts without custom model code.
 
 ## Comparison with Jev-style models

{typellm-0.3.0.dev1 → typellm-0.3.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "typellm"
-version = "0.3.0.dev1"
+version = "0.3.1"
 description = "Type-Safe Decoding for Autoregressive LLMs"
 readme = "README.md"
 license = "Apache-2.0"