courier-encode 0.1.0__py3-none-any.whl

courier_encode-0.1.0.dist-info/METADATA

@@ -0,0 +1,1158 @@
+Metadata-Version: 2.4
+Name: courier-encode
+Version: 0.1.0
+Summary: Python SDK for OpenAI-compatible inference endpoints (Courier and friends) with auto tool-call loops, structured outputs, and Whisper.
+Project-URL: Homepage, https://getcourier.ai
+Project-URL: Documentation, https://getcourier.ai/docs
+Project-URL: Repository, https://github.com/recursion-ai/encode
+Author: Recursion AI
+License:                                  Apache License
+                                   Version 2.0, January 2004
+                                http://www.apache.org/licenses/
+        
+           TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+        
+           1. Definitions.
+        
+              "License" shall mean the terms and conditions for use, reproduction,
+              and distribution as defined by Sections 1 through 9 of this document.
+        
+              "Licensor" shall mean the copyright owner or entity authorized by
+              the copyright owner that is granting the License.
+        
+              "Legal Entity" shall mean the union of the acting entity and all
+              other entities that control, are controlled by, or are under common
+              control with that entity. For the purposes of this definition,
+              "control" means (i) the power, direct or indirect, to cause the
+              direction or management of such entity, whether by contract or
+              otherwise, or (ii) ownership of fifty percent (50%) or more of the
+              outstanding shares, or (iii) beneficial ownership of such entity.
+        
+              "You" (or "Your") shall mean an individual or Legal Entity
+              exercising permissions granted by this License.
+        
+              "Source" form shall mean the preferred form for making modifications,
+              including but not limited to software source code, documentation
+              source, and configuration files.
+        
+              "Object" form shall mean any form resulting from mechanical
+              transformation or translation of a Source form, including but
+              not limited to compiled object code, generated documentation,
+              and conversions to other media types.
+        
+              "Work" shall mean the work of authorship, whether in Source or
+              Object form, made available under the License, as indicated by a
+              copyright notice that is included in or attached to the work
+              (an example is provided in the Appendix below).
+        
+              "Derivative Works" shall mean any work, whether in Source or Object
+              form, that is based on (or derived from) the Work and for which the
+              editorial revisions, annotations, elaborations, or other modifications
+              represent, as a whole, an original work of authorship. For the purposes
+              of this License, Derivative Works shall not include works that remain
+              separable from, or merely link (or bind by name) to the interfaces of,
+              the Work and Derivative Works thereof.
+        
+              "Contribution" shall mean any work of authorship, including
+              the original version of the Work and any modifications or additions
+              to that Work or Derivative Works thereof, that is intentionally
+              submitted to Licensor for inclusion in the Work by the copyright owner
+              or by an individual or Legal Entity authorized to submit on behalf of
+              the copyright owner. For the purposes of this definition, "submitted"
+              means any form of electronic, verbal, or written communication sent
+              to the Licensor or its representatives, including but not limited to
+              communication on electronic mailing lists, source code control systems,
+              and issue tracking systems that are managed by, or on behalf of, the
+              Licensor for the purpose of discussing and improving the Work, but
+              excluding communication that is conspicuously marked or otherwise
+              designated in writing by the copyright owner as "Not a Contribution."
+        
+              "Contributor" shall mean Licensor and any individual or Legal Entity
+              on behalf of whom a Contribution has been received by Licensor and
+              subsequently incorporated within the Work.
+        
+           2. Grant of Copyright License. Subject to the terms and conditions of
+              this License, each Contributor hereby grants to You a perpetual,
+              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+              copyright license to reproduce, prepare Derivative Works of,
+              publicly display, publicly perform, sublicense, and distribute the
+              Work and such Derivative Works in Source or Object form.
+        
+           3. Grant of Patent License. Subject to the terms and conditions of
+              this License, each Contributor hereby grants to You a perpetual,
+              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+              (except as stated in this section) patent license to make, have made,
+              use, offer to sell, sell, import, and otherwise transfer the Work,
+              where such license applies only to those patent claims licensable
+              by such Contributor that are necessarily infringed by their
+              Contribution(s) alone or by combination of their Contribution(s)
+              with the Work to which such Contribution(s) was submitted. If You
+              institute patent litigation against any entity (including a
+              cross-claim or counterclaim in a lawsuit) alleging that the Work
+              or a Contribution incorporated within the Work constitutes direct
+              or contributory patent infringement, then any patent licenses
+              granted to You under this License for that Work shall terminate
+              as of the date such litigation is filed.
+        
+           4. Redistribution. You may reproduce and distribute copies of the
+              Work or Derivative Works thereof in any medium, with or without
+              modifications, and in Source or Object form, provided that You
+              meet the following conditions:
+        
+              (a) You must give any other recipients of the Work or
+                  Derivative Works a copy of this License; and
+        
+              (b) You must cause any modified files to carry prominent notices
+                  stating that You changed the files; and
+        
+              (c) You must retain, in the Source form of any Derivative Works
+                  that You distribute, all copyright, patent, trademark, and
+                  attribution notices from the Source form of the Work,
+                  excluding those notices that do not pertain to any part of
+                  the Derivative Works; and
+        
+              (d) If the Work includes a "NOTICE" text file as part of its
+                  distribution, then any Derivative Works that You distribute must
+                  include a readable copy of the attribution notices contained
+                  within such NOTICE file, excluding those notices that do not
+                  pertain to any part of the Derivative Works, in at least one
+                  of the following places: within a NOTICE text file distributed
+                  as part of the Derivative Works; within the Source form or
+                  documentation, if provided along with the Derivative Works; or,
+                  within a display generated by the Derivative Works, if and
+                  wherever such third-party notices normally appear. The contents
+                  of the NOTICE file are for informational purposes only and
+                  do not modify the License. You may add Your own attribution
+                  notices within Derivative Works that You distribute, alongside
+                  or as an addendum to the NOTICE text from the Work, provided
+                  that such additional attribution notices cannot be construed
+                  as modifying the License.
+        
+              You may add Your own copyright statement to Your modifications and
+              may provide additional or different license terms and conditions
+              for use, reproduction, or distribution of Your modifications, or
+              for any such Derivative Works as a whole, provided Your use,
+              reproduction, and distribution of the Work otherwise complies with
+              the conditions stated in this License.
+        
+           5. Submission of Contributions. Unless You explicitly state otherwise,
+              any Contribution intentionally submitted for inclusion in the Work
+              by You to the Licensor shall be under the terms and conditions of
+              this License, without any additional terms or conditions.
+              Notwithstanding the above, nothing herein shall supersede or modify
+              the terms of any separate license agreement you may have executed
+              with Licensor regarding such Contributions.
+        
+           6. Trademarks. This License does not grant permission to use the trade
+              names, trademarks, service marks, or product names of the Licensor,
+              except as required for reasonable and customary use in describing the
+              origin of the Work and reproducing the content of the NOTICE file.
+        
+           7. Disclaimer of Warranty. Unless required by applicable law or
+              agreed to in writing, Licensor provides the Work (and each
+              Contributor provides its Contributions) on an "AS IS" BASIS,
+              WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+              implied, including, without limitation, any warranties or conditions
+              of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+              PARTICULAR PURPOSE. You are solely responsible for determining the
+              appropriateness of using or redistributing the Work and assume any
+              risks associated with Your exercise of permissions under this License.
+        
+           8. Limitation of Liability. In no event and under no legal theory,
+              whether in tort (including negligence), contract, or otherwise,
+              unless required by applicable law (such as deliberate and grossly
+              negligent acts) or agreed to in writing, shall any Contributor be
+              liable to You for damages, including any direct, indirect, special,
+              incidental, or consequential damages of any character arising as a
+              result of this License or out of the use or inability to use the
+              Work (including but not limited to damages for loss of goodwill,
+              work stoppage, computer failure or malfunction, or any and all
+              other commercial damages or losses), even if such Contributor
+              has been advised of the possibility of such damages.
+        
+           9. Accepting Warranty or Additional Liability. While redistributing
+              the Work or Derivative Works thereof, You may choose to offer,
+              and charge a fee for, acceptance of support, warranty, indemnity,
+              or other liability obligations and/or rights consistent with this
+              License. However, in accepting such obligations, You may act only
+              on Your own behalf and on Your sole responsibility, not on behalf
+              of any other Contributor, and only if You agree to indemnify,
+              defend, and hold each Contributor harmless for any liability
+              incurred by, or claims asserted against, such Contributor by reason
+              of your accepting any such warranty or additional liability.
+        
+           END OF TERMS AND CONDITIONS
+        
+           APPENDIX: How to apply the Apache License to your work.
+        
+              To apply the Apache License to your work, attach the following
+              boilerplate notice, with the fields enclosed by brackets "[]"
+              replaced with your own identifying information. (Don't include
+              the brackets!)  The text should be enclosed in the appropriate
+              comment syntax for the file format. We also recommend that a
+              file or class name and description of purpose be included on the
+              same "printed page" as the copyright notice for easier
+              identification within third-party archives.
+        
+           Copyright [yyyy] [name of copyright owner]
+        
+           Licensed under the Apache License, Version 2.0 (the "License");
+           you may not use this file except in compliance with the License.
+           You may obtain a copy of the License at
+        
+               http://www.apache.org/licenses/LICENSE-2.0
+        
+           Unless required by applicable law or agreed to in writing, software
+           distributed under the License is distributed on an "AS IS" BASIS,
+           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+           See the License for the specific language governing permissions and
+           limitations under the License.
+License-File: LICENSE
+Keywords: agents,courier,llm,openai,sdk,tool-calling,whisper
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: httpx>=0.27
+Requires-Dist: pexpect>=4.9
+Requires-Dist: pydantic>=2.7
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: typing-extensions>=4.10
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: mypy>=1.10; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: respx>=0.21; extra == 'dev'
+Requires-Dist: ruff>=0.5; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# Courier — ENCODE
+
+Courier — ENCODE is an SDK specifically designed for developing AI apps and agents with open source
+LLMs.
+
+## Features
+
+- Python SDK
+- Request wrapper that sends messages to v1/chat/completions and v1/responses, configurable for each, auto formatting,
+  and returns a pydantic response with props the end user can use in their code.
+    - The request wrapper includes agentic processing with an optional 'tools' prop. Python functions can be passed and
+      automatically converted into tools for an agent. The SDK should parse out tool responses and call the functions,
+      and automatically send a follow up message to the agent. The SDK should be considered an option that runs until
+      it's completed it's tool calling, and then return a full list of messages in openai format.
+    - The request wrapper should optionally accept Pydantic models as a response_format for structured JSON responses
+      and automatically apply the format. The response object should be auto formatted to that pydantic model and usable
+      as such.
+    - The request wrapper should accept a pydantic model of messages. Images and audio should be included as well.
+    - a model name should be able to be selected.
+- A whisper request wrapper that allows for transcriptions and translations
+
+## Notes
+
+**encode should be a `uv` project and structured to be deployed on PyPi as an open source pip package.**
+
+Everywhere that pydantic is accepted JSON should be accepted as well.
+
+The tool call loop runs until the model doesn't make a tool call (or a hard capped limit that can be specified). So, if
+a tool call is present, the code should execute and the response appended to the messages, and then another API requests
+sent to the model. Once a response comes in with no acions the code continues past the relay loop.
+
+Web search should be a boolean to enable. If enabled it should automatically be appended using the shorthand schema.
+
+### function names
+
+- `encode.relay()` — the chat completions and responses wrapper
+    - `.intercept() ` — listener that can be attached to a `relay()` call and execute code whenever a tool call loop is
+      engaged, where the model is processing requests until the loop ends. Intercept runs every time a tool call
+      finishes even if the model is continuing the loop.
+- `encode.whisper()` — whisper function. Accepts audio and can translate or transcribe.
+
+### Courier Docs
+
+[Courier](https://getcourier.ai/docs)
+
+#### API docs
+
+## API Docs
+
+### Courier Inference API
+
+Courier provides a custom inference API optimized for n8n and other workflows.
+
+#### POST /inference/
+
+```json
+{
+  "model_name": "Solar Open 100B",
+  "model_id": "Model_UUID",
+  "model_type": "text-text",
+  "messages": [
+    {
+      "role": "system",
+      "content": "You are a helpful assistant"
+    },
+    {
+      "role": "user",
+      "content": "hello"
+    }
+  ],
+  "temperature": 0.7
+}
+```
+
+#### Authentication
+
+For the Courier **/inference/** endpoint, use token authentication:
+
+```
+Authorization: API_KEY
+```
+
+### OpenAI Compatible Endpoints
+
+Courier supports OpenAI-compatible APIs for completions and responses workflows.
+
+#### POST /v1/chat/completions
+
+```json
+{
+  "model": "Solar Open 100B",
+  "messages": [
+    {
+      "role": "system",
+      "content": "You are a helpful assistant"
+    },
+    {
+      "role": "user",
+      "content": "hello"
+    }
+  ],
+  "temperature": 0.7
+}
+```
+
+#### GET /v1/models
+
+```json
+{
+  "object": "list",
+  "data": [
+    {
+      "id": "model_name",
+      "object": "model",
+      "created": 1686935092,
+      "owned_by": "recursion-ai"
+    }
+  ]
+}
+```
+
+#### POST /v1/responses
+
+```json
+{
+  "model": "my-shared-model",
+  "input": "Summarize this in one sentence.",
+  "instructions": "optional system/developer instruction",
+  "tools": [],
+  "tool_choice": "auto",
+  "text": {
+    "format": {
+      "type": "text"
+    }
+  },
+  "stream": false,
+  "max_output_tokens": 256
+}
+```
+
+#### Authentication (OpenAI Endpoints)
+
+Use Bearer authentication:
+
+```
+Authorization: Bearer API_KEY
+```
+
+#### Tool Calling
+
+## Tool Calling API
+
+Industry-leading tool calling for self-hosted AI stacks, with production-ready reliability for text and fused modality (
+vision) models. One of the most robust OpenAI-compatible tool-calling implementations available on an API platform you
+can own.
+
+### Global OpenAI Compatibility
+
+- Auth header: `Authorization: Bearer <api_key>` is required.
+- Model matching is case-insensitive against workbench `name` or `nickname`, and only models available to the API key
+  are usable.
+
+#### Error Envelope
+
+```json
+{
+  "error": {
+    "message": "....",
+    "type": "invalid_request_error",
+    "code": "...."
+  }
+}
+```
+
+### POST /v1/chat/completions
+
+#### Supported Request Fields
+
+`model`, `messages`, `tools`, `tool_choice`, `stream`, `response_format`, `stop`, `max_tokens`, `temperature`, `top_p`,
+`presence_penalty`, `frequency_penalty`, `user`. `n` is accepted but currently returns one choice (`index: 0`).
+
+#### Request Body Example
+
+```json
+{
+  "model": "Solar Open 100B",
+  "messages": [
+    {
+      "role": "system",
+      "content": "You are a helpful assistant."
+    },
+    {
+      "role": "user",
+      "content": "What is the weather in Denver?"
+    }
+  ],
+  "tools": [
+    {
+      "type": "function",
+      "function": {
+        "name": "get_weather",
+        "description": "Get current weather by city",
+        "parameters": {
+          "type": "object",
+          "properties": {
+            "city": {
+              "type": "string"
+            }
+          },
+          "required": [
+            "city"
+          ]
+        }
+      }
+    }
+  ],
+  "tool_choice": "auto",
+  "stream": false
+}
+```
+
+#### Tool Support Rules
+
+- Only tools with `type: "function"` are used.
+- Forced `tool_choice` names must exist in `tools`, or requests fail with `400 invalid_tool_choice`.
+- Text and fused modality (image-text-text) models support the tool-calling pipeline. Audio models do not support tools
+  or streaming.
+- Tool arguments are normalized to JSON strings; strict invalid arguments fail with `400 invalid_tool_call`.
+
+#### Response Body Example
+
+```json
+{
+  "choices": [
+    {
+      "index": 0,
+      "message": {
+        "role": "assistant",
+        "tool_calls": [
+          {
+            "id": "call_abc123",
+            "type": "function",
+            "function": {
+              "name": "get_weather",
+              "arguments": "{\"city\":\"Denver\"}"
+            }
+          }
+        ]
+      },
+      "finish_reason": "tool_calls"
+    }
+  ]
+}
+```
+
+#### Streaming Behavior (SSE)
+
+1. Emits `chat.completion.chunk` events.
+2. Streams content via `choices[0].delta.content`.
+3. Tool calls stream incrementally through `delta.tool_calls` argument chunks.
+4. Final chunk sets `finish_reason` to `tool_calls` or `stop`, then emits `[DONE]`.
+
+### POST /v1/responses
+
+#### Supported Request Fields
+
+`model`, `input`, `messages`, `input_content`, `instructions`, `text`, `tools`, `tool_choice`, `stream`, `stop`,
+`max_tokens`, `max_output_tokens`, `temperature`, `top_p`, `presence_penalty`, `frequency_penalty`, `user`. `n`,
+`logit_bias`, and `input_type` are accepted but not used in generation logic.
+
+#### Request Body Example
+
+```json
+{
+  "model": "Solar Open 100B",
+  "input": [
+    {
+      "type": "message",
+      "role": "user",
+      "content": "Find today's top headline"
+    }
+  ],
+  "tools": [
+    {
+      "type": "function",
+      "function": {
+        "name": "search_news",
+        "description": "Search current headlines",
+        "parameters": {
+          "type": "object",
+          "properties": {
+            "query": {
+              "type": "string"
+            }
+          },
+          "required": [
+            "query"
+          ]
+        }
+      }
+    }
+  ],
+  "tool_choice": "auto",
+  "stream": false
+}
+```
+
+#### Input Normalization
+
+- `input` can be a string or a list of typed items.
+- Supported item types: `message`, `input_text`, `function_call`, `function_call_output`.
+- `reasoning` input items are rejected with `400 invalid_input`.
+
+#### Response Body Example
+
+```json
+{
+  "object": "response",
+  "output": [
+    {
+      "id": "msg_123",
+      "type": "message",
+      "status": "completed",
+      "role": "assistant",
+      "content": [
+        {
+          "type": "output_text",
+          "text": "Searching now...",
+          "annotations": []
+        }
+      ]
+    },
+    {
+      "id": "fc_123",
+      "type": "function_call",
+      "status": "completed",
+      "call_id": "call_abc123",
+      "name": "search_news",
+      "arguments": "{\"query\":\"top headline today\"}"
+    }
+  ]
+}
+```
+
+#### Streaming Behavior (Responses SSE Events)
+
+`response.created`, `response.in_progress`, `response.output_item.added`, `response.content_part.added`,
+`response.output_text.delta`, `response.output_text.done`, `response.content_part.done`,
+`response.function_call_arguments.delta`, `response.function_call_arguments.done`, `response.output_item.done`,
+`response.completed`, `error`
+
+Streams end with `data: [DONE]`.
+
+#### Multimodal Restrictions on /v1/responses
+
+- For `audio-*` models, `stream` and `tools` are not supported.
+- When provided for audio models, `text.format` must be plain text.
+
+#### Tool Calling Parity Notes
+
+- OpenAI-style behavior with function tools only.
+- Tool arguments must normalize to valid JSON strings.
+- Audio models do not support tools and/or streaming. Text and fused modality (image-text-text) models have full
+  tool-calling parity.
+
+#### Whisper API
+
+## Whisper API
+
+OpenAI-compatible Whisper transcription and translation endpoints built for production automation workflows.
+
+### Implemented Endpoints
+
+- `POST /v1/audio/transcriptions`
+- `POST /v1/audio/translations`
+- `/v1/audio/speech` is not currently implemented.
+
+### Common Request Behavior
+
+- Multipart upload with required `file` and `model`.
+- Allowed extensions: `.mp3`, `.mp4`, `.mpeg`, `.mpga`, `.m4a`, `.wav`, `.webm`. Max size: 25 MB.
+- Error mapping: unsupported format → `invalid_audio`, too large → `invalid_request_error`, invalid format value →
+  `invalid_response_format`.
+- `model=whisper-1` maps to `UCE_WHISPER_MODEL` (default `mlx-community/whisper-large-v3-turbo`); other model names pass
+  through unchanged.
+
+### POST /v1/audio/transcriptions
+
+#### Request Example (multipart)
+
+```bash
+curl -X POST "$BASE_URL/v1/audio/transcriptions" \
+  -H "Authorization: Bearer $API_KEY" \
+  -F "file=@audio.wav" \
+  -F "model=whisper-1" \
+  -F "response_format=verbose_json" \
+  -F "timestamp_granularities[]=word"
+```
+
+#### Response Example (json)
+
+```json
+{
+  "text": "Hello from Courier Whisper."
+}
+```
+
+#### Response Example (verbose_json)
+
+```json
+{
+  "text": "Hello from Courier Whisper.",
+  "language": "en",
+  "segments": [
+    {
+      "id": 0,
+      "start": 0.0,
+      "end": 1.8,
+      "text": "Hello from Courier Whisper."
+    }
+  ]
+}
+```
+
+> `timestamp_granularities` values `segment` and `word` are only allowed when `response_format=verbose_json`.
+
+### POST /v1/audio/translations
+
+#### Request Example (multipart)
+
+```bash
+curl -X POST "$BASE_URL/v1/audio/translations" \
+  -H "Authorization: Bearer $API_KEY" \
+  -F "file=@audio-es.mp3" \
+  -F "model=whisper-1" \
+  -F "response_format=json"
+```
+
+#### Response Example
+
+```json
+{
+  "text": "This audio was translated into English."
+}
+```
+
+### Behavior Notes
+
+- Uses translate operation internally.
+- `word_timestamps` is disabled for translations.
+- For `verbose_json`, language defaults to `en` if upstream language is missing.
+
+#### JSON Response Formatting
+
+## JSON Response Formatting
+
+### Structured JSON Outputs with Outlines
+
+Courier supports guaranteed structured JSON outputs using the [Outlines](https://github.com/outlines-dev/outlines)
+library. This feature enables models to generate responses that strictly adhere to a provided JSON schema through
+FSM-based logit masking.
+
+### Overview
+
+Courier's structured JSON output feature uses Outlines to ensure models generate responses that strictly follow your
+JSON schema. This is achieved through Finite State Machine (FSM) based logit masking that constrains token generation to
+only produce valid JSON matching your schema.
+
+### Technical Architecture
+
+- **FSM-Based Logit Masking:** Outlines builds a Finite State Machine from your JSON schema that constrains token
+  generation to only produce valid JSON matching the schema.
+- **Generator Caching:** The first time a schema is used, there's a 0.1-1s cold start while the FSM is compiled.
+  Subsequent uses are instant (cached in memory per worker).
+- **Thought Field Pattern:** To prevent "probability tunneling", schemas are automatically enhanced with a "thought"or "
+  reasoning" field if one isn't present, allowing natural language processing before data constraints.
+- **Zero-Copy Integration:** The Outlines wrapper shares the same MLX model weights in memory, providing minimal
+  overhead when structured output is requested.
+
+### Usage
+
+Both **/v1/chat/completions** and **/inference/** endpoints use the OpenAI-compatible **response_format** parameter:
+
+#### POST /v1/chat/completions
+
+```json
+{
+  "model": "Solar Open 100B",
+  "messages": [
+    {
+      "role": "user",
+      "content": "What is 123 * 456?"
+    }
+  ],
+  "response_format": {
+    "type": "json_schema",
+    "json_schema": {
+      "schema": {
+        "type": "object",
+        "properties": {
+          "thought": {
+            "type": "string"
+          },
+          "answer": {
+            "type": "number"
+          }
+        },
+        "required": [
+          "thought",
+          "answer"
+        ]
+      }
+    }
+  }
+}
+```
+
+#### POST /inference/
+
+```json
+{
+  "model_id": "uuid-here",
+  "model_name": "your-model",
+  "model_type": "text-text",
+  "messages": [
+    {
+      "role": "user",
+      "content": "Classify: 'Great product!'"
+    }
+  ],
+  "temperature": 0.7,
+  "response_format": {
+    "type": "json_schema",
+    "json_schema": {
+      "schema": {
+        "type": "object",
+        "properties": {
+          "reasoning": {
+            "type": "string"
+          },
+          "sentiment": {
+            "type": "string",
+            "enum": [
+              "positive",
+              "negative",
+              "neutral"
+            ]
+          }
+        },
+        "required": [
+          "reasoning",
+          "sentiment"
+        ]
+      }
+    }
+  }
+}
+```
+
+#### Authentication
+
+For the Courier **/inference/** endpoint, use token authentication:
+
+```
+Authorization: API_KEY
+```
+
+For the OpenAI **/v1/chat/completions** endpoint, use bearer authentication:
+
+```
+Authorization: Bearer API_KEY
+```
+
+#### Example Structured Response
+
+```json
+{
+  "content": "{\n    \"reasoning\": \"The user sent a positive sentiment message: 'Great product!'. I need to classify this as positive sentiment.\",\n    \"sentiment\": \"positive\"\n  }"
+}
+```
+
+As you can see, the LLM responded in the specified JSON structure with both reasoning and sentiment fields. You can
+parse this response using `JSON.parse()` without any extra formatting or validation needed.
+
+### Schema Examples
+
+#### Simple Classification
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "thought": {
+      "type": "string"
+    },
+    "classification": {
+      "type": "string",
+      "enum": [
+        "urgent",
+        "normal",
+        "low_priority"
+      ]
+    }
+  },
+  "required": [
+    "thought",
+    "classification"
+  ]
+}
+```
+
+#### Nested Objects
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "analysis": {
+      "type": "string"
+    },
+    "person": {
+      "type": "object",
+      "properties": {
+        "name": {
+          "type": "string"
+        },
+        "age": {
+          "type": "integer"
+        },
+        "email": {
+          "type": "string"
+        }
+      },
+      "required": [
+        "name",
+        "age"
+      ]
+    }
+  },
+  "required": [
+    "analysis",
+    "person"
+  ]
+}
+```
+
+#### Arrays and Lists
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "reasoning": {
+      "type": "string"
+    },
+    "items": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "name": {
+            "type": "string"
+          },
+          "quantity": {
+            "type": "integer"
+          }
+        },
+        "required": [
+          "name",
+          "quantity"
+        ]
+      }
+    }
+  },
+  "required": [
+    "reasoning",
+    "items"
+  ]
+}
+```
+
+#### Enums and Constraints
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "thought": {
+      "type": "string"
+    },
+    "rating": {
+      "type": "integer",
+      "minimum": 1,
+      "maximum": 5
+    },
+    "category": {
+      "type": "string",
+      "enum": [
+        "electronics",
+        "clothing",
+        "food",
+        "other"
+      ]
+    }
+  },
+  "required": [
+    "thought",
+    "rating",
+    "category"
+  ]
+}
+```
+
+### Limitations
+
+- **No Streaming:** Structured output currently doesn't support streaming. The response is returned as a complete JSON
+  object.
+- **Schema Complexity:** Extremely complex schemas with deep nesting may take longer to compile or potentially fail.
+- **Model Capabilities:** The underlying model must be capable of understanding and following instructions. Smaller
+  models may struggle with complex schemas.
+- **Text & Vision Models:** Supported for `text-text` and `image-text-text` (fused modality) model types. Audio and
+  image generation models use standard unconstrained generation.
+
+### Backward Compatibility
+
+When no `response_format` is provided, the system works exactly as it did before. There is zero impact on existing
+inference flows. This feature is purely opt-in.
+
+```json
+// This still works exactly as before
+{
+  "model": "your-model",
+  "messages": [
+    {
+      "role": "user",
+      "content": "Hello!"
+    }
+  ]
+  // No response_format = standard unconstrained generation
+}
+```
+
+#### Web search
+
+## Web Search
+
+Built-in web search that lets models automatically ground responses with real-time information from the web. Powered by
+the Brave Search API.
+
+When enabled, models can decide to search the web mid-inference. The server handles the search transparently and returns
+a grounded response - no extra client-side logic required.
+
+### Setup
+
+#### 1. Get a Brave Search API Key
+
+Sign up at [brave.com/search/api](https://brave.com/search/api/) to get your API key. New accounts receive $5/month in
+free credits (~1,000 searches).
+
+#### 2. Configure the Key
+
+- **Option A: Courier TUI Installer** - Run `courier` and enter your key in the "Brave Search API Key" field (in the
+  System Configuration section, below the ngrok fields).
+- **Option B: Manual** - Add to your `~/.courier/.env`:
+
+```bash
+BRAVE_SEARCH_API_KEY=your_key_here
+```
+
+Then restart Courier.
+
+### Usage
+
+Include `web_search` in your request's `tools` array. Two formats are supported:
+
+#### Shorthand Format
+
+```json
+{
+  "model": "Qwen3 30B",
+  "messages": [
+    {
+      "role": "user",
+      "content": "What happened in the news today?"
+    }
+  ],
+  "tools": [
+    {
+      "type": "web_search"
+    }
+  ]
+}
+```
+
+#### Standard Function Format
+
+```json
+{
+  "model": "Qwen3 30B",
+  "messages": [
+    {
+      "role": "user",
+      "content": "What is the current price of Bitcoin?"
+    }
+  ],
+  "tools": [
+    {
+      "type": "function",
+      "function": {
+        "name": "web_search",
+        "description": "Search the web for current information",
+        "parameters": {
+          "type": "object",
+          "properties": {
+            "query": {
+              "type": "string"
+            }
+          },
+          "required": [
+            "query"
+          ]
+        }
+      }
+    }
+  ]
+}
+```
+
+#### cURL Example
+
+```bash
+curl -X POST http://localhost:9100/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer YOUR_API_KEY" \
+  -d '{
+    "model": "Qwen3 30B",
+    "messages": [{"role": "user", "content": "What are the latest developments in AI?"}],
+    "tools": [{"type": "web_search"}]
+  }'
+```
+
+#### Streaming
+
+```bash
+curl -X POST http://localhost:9100/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer YOUR_API_KEY" \
+  -d '{
+    "model": "Qwen3 30B",
+    "messages": [{"role": "user", "content": "What are the latest developments in AI?"}],
+    "tools": [{"type": "web_search"}],
+    "stream": true
+  }'
+```
+
+Streaming works with web search enabled. Add `"stream": true` to your request. The server resolves all searches before
+streaming the final grounded response.
+
+### Mixing Web Search with Other Tools
+
+Web search works alongside your own function tools. The server executes `web_search` calls automatically while returning
+your custom tool calls to the client as normal.
+
+#### Example
+
+```json
+{
+  "model": "Qwen3 30B",
+  "messages": [
+    {
+      "role": "user",
+      "content": "What's the weather in Denver and what's trending on Hacker News?"
+    }
+  ],
+  "tools": [
+    {
+      "type": "web_search"
+    },
+    {
+      "type": "function",
+      "function": {
+        "name": "get_weather",
+        "description": "Get current weather by city",
+        "parameters": {
+          "type": "object",
+          "properties": {
+            "city": {
+              "type": "string"
+            }
+          },
+          "required": [
+            "city"
+          ]
+        }
+      }
+    }
+  ]
+}
+```
+
+In this case, the model may call both tools. `web_search` is resolved server-side and `get_weather` is returned to the
+client for execution.
+
+### How It Works
+
+1. The model decides whether a search is needed based on the user's question
+2. If the model calls `web_search`, the server intercepts the call and queries the Brave Search API
+3. The top 5 results (title, URL, description) are injected back as context
+4. The model generates a final grounded response using the search results
+5. The client receives only the final answer — the search loop is invisible
+
+The server caps search iterations at 3 per request to prevent runaway loops.
+
+### Behavior Without a Key
+
+If `BRAVE_SEARCH_API_KEY` is not configured and a request includes `web_search`, the tool is silently dropped. The
+request proceeds normally as if no tools were provided. A warning is logged server-side.
+
+### Pricing
+
+Brave Search API charges $5 per 1,000 queries. New accounts get $5/month in free credits. There is no markup from
+Courier — you pay Brave directly for what you use.