dot-inference 2.2.0__tar.gz → 3.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (66) hide show
  1. {dot_inference-2.2.0 → dot_inference-3.1.0}/.gitlab-ci.yml +22 -14
  2. {dot_inference-2.2.0 → dot_inference-3.1.0}/.pre-commit-config.yaml +3 -3
  3. {dot_inference-2.2.0 → dot_inference-3.1.0}/AUTHORS.md +1 -0
  4. {dot_inference-2.2.0 → dot_inference-3.1.0}/CHANGELOG.md +85 -1
  5. dot_inference-3.1.0/CONTRIBUTING.md +37 -0
  6. dot_inference-3.1.0/LICENSE.md +21 -0
  7. {dot_inference-2.2.0 → dot_inference-3.1.0}/PKG-INFO +96 -18
  8. {dot_inference-2.2.0 → dot_inference-3.1.0}/README.md +94 -16
  9. {dot_inference-2.2.0 → dot_inference-3.1.0}/docs/ARCHITECTURE.md +2 -2
  10. {dot_inference-2.2.0 → dot_inference-3.1.0}/docs/DEVELOPMENT.md +25 -26
  11. {dot_inference-2.2.0 → dot_inference-3.1.0}/docs/VERSIONING.md +3 -3
  12. dot_inference-3.1.0/docs/provider-quirks/openrouter-errors.md +40 -0
  13. dot_inference-3.1.0/docs/provider-quirks/strict-schemas.md +175 -0
  14. dot_inference-3.1.0/docs/provider-quirks/tool-choice.md +130 -0
  15. {dot_inference-2.2.0 → dot_inference-3.1.0}/examples/demo.py +0 -3
  16. {dot_inference-2.2.0 → dot_inference-3.1.0}/examples/demo_streaming.py +0 -3
  17. {dot_inference-2.2.0 → dot_inference-3.1.0}/pyproject.toml +2 -2
  18. dot_inference-3.1.0/scripts/provider-quirks/tool-choice/mistral_compare.sh +56 -0
  19. dot_inference-3.1.0/scripts/provider-quirks/tool-choice/mistral_runaway.sh +31 -0
  20. dot_inference-3.1.0/scripts/provider-quirks/tool-choice/openrouter_compare.sh +60 -0
  21. dot_inference-3.1.0/scripts/provider-quirks/tool-choice/openrouter_runaway.sh +39 -0
  22. {dot_inference-2.2.0 → dot_inference-3.1.0}/scripts/test_all_models.py +0 -3
  23. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/__init__.py +18 -4
  24. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/_client.py +122 -14
  25. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/_cost_tracker.py +0 -3
  26. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/_llm_registry.py +31 -3
  27. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/_local_check.py +23 -9
  28. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/_mistral_compat.py +38 -27
  29. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/_openai_compat.py +85 -31
  30. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/_rate_limit.py +0 -3
  31. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/_retry.py +0 -3
  32. dot_inference-3.1.0/src/dot_inference/_strict_schema.py +78 -0
  33. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/_sync.py +0 -3
  34. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/_types.py +125 -9
  35. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/mock.py +5 -3
  36. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/models.yaml +66 -0
  37. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/settings.py +2 -4
  38. dot_inference-3.1.0/tests/__init__.py +0 -0
  39. dot_inference-3.1.0/tests/test_constrained_live.py +273 -0
  40. {dot_inference-2.2.0 → dot_inference-3.1.0}/tests/test_cost_tracker.py +0 -3
  41. {dot_inference-2.2.0 → dot_inference-3.1.0}/tests/test_factories.py +53 -3
  42. dot_inference-3.1.0/tests/test_images.py +220 -0
  43. {dot_inference-2.2.0 → dot_inference-3.1.0}/tests/test_llm_registry.py +45 -3
  44. {dot_inference-2.2.0 → dot_inference-3.1.0}/tests/test_local_check.py +10 -4
  45. {dot_inference-2.2.0 → dot_inference-3.1.0}/tests/test_mistral_compat.py +68 -5
  46. {dot_inference-2.2.0 → dot_inference-3.1.0}/tests/test_mistral_live.py +2 -5
  47. {dot_inference-2.2.0 → dot_inference-3.1.0}/tests/test_openai_compat.py +486 -7
  48. {dot_inference-2.2.0 → dot_inference-3.1.0}/tests/test_rate_limit.py +0 -3
  49. {dot_inference-2.2.0 → dot_inference-3.1.0}/tests/test_retry.py +0 -3
  50. {dot_inference-2.2.0 → dot_inference-3.1.0}/tests/test_settings.py +5 -3
  51. dot_inference-3.1.0/tests/test_strict_schema.py +69 -0
  52. dot_inference-3.1.0/tests/test_types.py +116 -0
  53. {dot_inference-2.2.0 → dot_inference-3.1.0}/uv.lock +1 -1
  54. dot_inference-2.2.0/CONTRIBUTING.md +0 -87
  55. dot_inference-2.2.0/DCO +0 -34
  56. dot_inference-2.2.0/LICENSE.md +0 -660
  57. dot_inference-2.2.0/docs/provider-quirks/openrouter-errors.md +0 -19
  58. dot_inference-2.2.0/docs/provider-quirks/tool-choice.md +0 -31
  59. dot_inference-2.2.0/tests/__init__.py +0 -2
  60. dot_inference-2.2.0/tests/test_types.py +0 -59
  61. {dot_inference-2.2.0 → dot_inference-3.1.0}/.env.template +0 -0
  62. {dot_inference-2.2.0 → dot_inference-3.1.0}/.gitignore +0 -0
  63. {dot_inference-2.2.0 → dot_inference-3.1.0}/.python-version +0 -0
  64. {dot_inference-2.2.0 → dot_inference-3.1.0}/docs/PUBLISHING.md +0 -0
  65. {dot_inference-2.2.0 → dot_inference-3.1.0}/examples/doc_stream.md +0 -0
  66. {dot_inference-2.2.0 → dot_inference-3.1.0}/src/dot_inference/README.md +0 -0
@@ -83,36 +83,43 @@ test:
83
83
  path: coverage.xml
84
84
 
85
85
  # The offline suite stubs SDK responses, so it cannot notice the real API or
86
- # SDK changing shape. This job calls the real Mistral API on Mistral Small 4
87
- # with tiny prompts (~$0.0005 a run), only when something it guards changed;
88
- # otherwise it stays available as a manual job.
86
+ # SDK changing shape, nor a provider ignoring a strict schema. This job runs
87
+ # every test marked live: the Mistral smoke tests (tests/test_mistral_live.py)
88
+ # and the constrained-decoding tests on Mistral, GPT and Claude through
89
+ # OpenRouter (tests/test_constrained_live.py), with cheap models and short
90
+ # prompts. It runs only when something it guards changed; otherwise it stays
91
+ # available as a manual job.
89
92
  #
90
- # Needs MISTRAL_API_KEY under Settings > CI/CD > Variables: masked, NOT
91
- # protected (MR pipelines from unprotected branches would not see it). Use a
92
- # dedicated key from a workspace with a low spending cap — any developer can
93
- # read an unprotected variable from their branch.
94
- test_live_mistral:
93
+ # Needs MISTRAL_API_KEY and OPENROUTER_API_KEY under Settings > CI/CD >
94
+ # Variables: masked, NOT protected (MR pipelines from unprotected branches
95
+ # would not see them). Use dedicated keys with a low spending cap — any
96
+ # developer can read an unprotected variable from their branch.
97
+ test_live:
95
98
  stage: test
96
99
  before_script:
97
100
  - uv sync --all-extras --all-groups
98
101
  script:
99
102
  - |
100
- if [ -z "$MISTRAL_API_KEY" ]; then
101
- echo "MISTRAL_API_KEY is not set. Add it under Settings > CI/CD > Variables"
102
- echo "(masked, not protected) — see the comment on this job."
103
- exit 1
104
- fi
103
+ for key in MISTRAL_API_KEY OPENROUTER_API_KEY; do
104
+ if [ -z "$(printenv "$key")" ]; then
105
+ echo "$key is not set. Add it under Settings > CI/CD > Variables"
106
+ echo "(masked, not protected) — see the comment on this job."
107
+ exit 1
108
+ fi
109
+ done
105
110
  - uv run pytest -m live -v
106
111
  retry:
107
112
  max: 1
108
113
  when: script_failure
109
114
  rules:
110
- # Fork MRs never get the project's key: a maintainer runs the pipeline
115
+ # Fork MRs never get the project's keys: a maintainer runs the pipeline
111
116
  # for the MR in the parent project instead.
112
117
  - if: $CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_SOURCE_PROJECT_PATH != $CI_MERGE_REQUEST_PROJECT_PATH
113
118
  when: never
114
119
  - changes:
115
120
  - src/dot_inference/_mistral_compat.py
121
+ - src/dot_inference/_openai_compat.py
122
+ - src/dot_inference/_strict_schema.py
116
123
  - src/dot_inference/_client.py
117
124
  - src/dot_inference/_types.py
118
125
  - src/dot_inference/_cost_tracker.py
@@ -120,6 +127,7 @@ test_live_mistral:
120
127
  - src/dot_inference/_llm_registry.py
121
128
  - src/dot_inference/models.yaml
122
129
  - tests/test_mistral_live.py
130
+ - tests/test_constrained_live.py
123
131
  # SDK upgrades are the main source of silent breakage.
124
132
  - pyproject.toml
125
133
  - uv.lock
@@ -1,6 +1,6 @@
1
- # Verbatim legal texts: the AGPL and the DCO both forbid modification,
2
- # so no hook may normalise their whitespace or line endings.
3
- exclude: '^(LICENSE\.md|DCO)$'
1
+ # Verbatim legal text (MIT licence): no hook may normalise its whitespace
2
+ # or line endings.
3
+ exclude: '^LICENSE\.md$'
4
4
 
5
5
  repos:
6
6
  - repo: https://github.com/pre-commit/pre-commit-hooks
@@ -7,6 +7,7 @@ here does not confer any ownership of, or rights in, the project.
7
7
  ## Kannon For Deep Tech
8
8
 
9
9
  - Adrien Marjollet
10
+ - Charles Capon
10
11
  - Clément Olivier
11
12
  - Jean Fechter
12
13
  - Louis Le Tarnec
@@ -7,6 +7,88 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [3.1.0] - 2026-10-29
11
+
12
+ ### Added
13
+
14
+ - Image input: `image_part()` builds an image content part from an `http(s)://`
15
+ or `data:` URL, a local file path or raw bytes (inlined as base64, media type
16
+ guessed from the extension or the bytes), with an optional `detail`;
17
+ `text_part()` builds the text part next to it. `has_images()` tells whether
18
+ messages carry any. Both the OpenAI-compatible and the Mistral clients send
19
+ the parts as-is.
20
+ - Registry: `supports_vision` (default `false`, can be overridden per
21
+ provider). Sending an image to a model registered without it raises
22
+ `ValueError` before any request; unregistered models are not checked. Set to
23
+ `true` for the GPT-5/GPT-6, Gemini, Gemma 4, Mistral, Qwen 3.5 and Claude
24
+ entries.
25
+ - Request summaries in warning logs count each message's images, never their
26
+ content.
27
+ - `max_tokens` per call on `call`, `stream`, `stream_text` and `call_sync`, and
28
+ as an override on `get_llm_client`.
29
+ - `list_served_models(base_url)`: the model ids an OpenAI-compatible server
30
+ reports via `GET /models`, for any model, registered or not.
31
+ - `reasoning_tokens` on `LLMResponse` and `StreamUsage`, read from
32
+ `usage.completion_tokens_details.reasoning_tokens` (or `usage.reasoning_tokens`).
33
+ - `ProviderModelConfig` is now exported, so `register_model` can declare
34
+ provider routes.
35
+ - `ToolDef.strict`: when true, the tool is sent with `"strict": true` so the
36
+ server constrains its arguments to the schema. Defaults to `false`, and the
37
+ field is omitted then.
38
+ - `tool_choice` also accepts the name of one of the provided tools, to force
39
+ that tool. An unknown name raises `ValueError` before any request.
40
+ - `extra_body` per call on `call`, `stream`, `stream_text` and `call_sync`:
41
+ provider-specific request fields (e.g. vLLM's `ignore_eos`), merged over the
42
+ client's own. The Mistral client rejects it.
43
+ - `max_attempts` on `OpenAICompatClient` and `MistralClient`, and as an override
44
+ on `get_llm_client`: total attempts per request (1 = no retry). Defaults are
45
+ unchanged (4 and 2).
46
+ - `ToolDef.parameters` also takes a Pydantic model. Its schema is sent; with
47
+ `strict=True` it is rewritten to OpenAI's strict rules for every provider
48
+ (OpenAI and Claude reject a strict tool whose objects are not closed).
49
+ `ToolDef.parse_arguments()` validates a call's `arguments` into the model
50
+ (`None` for a dict schema).
51
+ - `to_strict_schema()` is exported, to rewrite a hand-written schema to
52
+ OpenAI's strict rules.
53
+
54
+ ### Fixed
55
+
56
+ - OpenAI-compatible clients no longer stack the OpenAI SDK's own retries (2 by
57
+ default) on top of theirs: a failing request is now sent at most 4 times by
58
+ default, not up to 12. A client passed in through `client=` keeps its own
59
+ settings.
60
+ - Structured output on OpenAI models (the `openai` provider, or `openai/...`
61
+ on OpenRouter, gpt-oss aside): a Pydantic `response_format` with a nested
62
+ model or a field with a default no longer fails with a 400. Its schema is
63
+ rewritten to OpenAI's strict rules (every object closed, every property
64
+ required, optional ones nullable). Dict schemas and other providers are
65
+ unchanged.
66
+ - A dict `response_format` is no longer modified in place: the top-level
67
+ `additionalProperties: false` was added to the caller's own dict.
68
+
69
+ ## [3.0.0] - 2026-10-15
70
+
71
+ ### Added
72
+
73
+ - Registry: `supports_temperature` (default `true`, can be overridden per
74
+ provider), independent of `supports_reasoning`. OpenAI's `gpt-5*` and
75
+ `gpt-6*` models are marked `false`: their API accepts only the default
76
+ temperature (1).
77
+
78
+ ### Changed
79
+
80
+ - **Breaking** — temperature no longer depends on reasoning: a temperature is
81
+ sent whenever one is set, reasoning model or not (gpt-oss, Claude, Mistral,
82
+ Qwen… honour it), and dropped only for a model whose registry entry says
83
+ `supports_temperature: false`. A per-call `temperature=` is treated the
84
+ same way.
85
+ - **Breaking** — `GenerationSettings.temperature` defaults to `None` instead of
86
+ `0`: unset, no temperature is sent and the provider applies its own default.
87
+ Set `DOTI_GENERATION__TEMPERATURE=0` to keep the previous behaviour for
88
+ non-reasoning models.
89
+ - Licence: from `AGPL-3.0-or-later` to `MIT`. Releases already published
90
+ stay under the AGPL.
91
+
10
92
  ## [2.2.0] - 2026-10-08
11
93
 
12
94
  ### Added
@@ -107,7 +189,9 @@ this release.
107
189
  - `LLMCallable` and `EmbeddingCallable` return a `Coroutine` rather than the
108
190
  looser `Awaitable`, matching what `asyncio.run_coroutine_threadsafe` accepts.
109
191
 
110
- [Unreleased]: https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/-/compare/v2.2.0...main
192
+ [Unreleased]: https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/-/compare/v3.1.0...main
193
+ [3.1.0]: https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/-/tags/v3.1.0
194
+ [3.0.0]: https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/-/tags/v3.0.0
111
195
  [2.2.0]: https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/-/tags/v2.2.0
112
196
  [2.1.0]: https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/-/tags/v2.1.0
113
197
  [2.0.3]: https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/-/tags/v2.0.3
@@ -0,0 +1,37 @@
1
+ # Contributing
2
+
3
+ Thanks for your interest in this project. Contributions are welcome.
4
+
5
+ This project is maintained by **Kannon For Deep Tech** and released under the
6
+ MIT License (`MIT`).
7
+
8
+ ## 1. Licensing of your contribution
9
+
10
+ By contributing to this project, you agree that your contribution is licensed
11
+ under the MIT License, the same terms as the rest of the project. You remain
12
+ the author and copyright holder of your contribution.
13
+
14
+ You confirm that you are entitled to do so: that your contribution is your
15
+ original work or that you hold the necessary rights over it, and that it does
16
+ not infringe any third-party rights. If you are contributing in the course of
17
+ your employment or under a service agreement, you confirm that your employer or
18
+ client has authorised you to contribute under these terms.
19
+
20
+ ## 2. Contractors and employees of Kannon For Deep Tech
21
+
22
+ If you are working for Kannon For Deep Tech as an employee or under a service
23
+ agreement, the assignment of rights set out in your contract governs, and takes
24
+ precedence over this file.
25
+
26
+ ## 3. Submitting a change
27
+
28
+ 1. Open an issue first for anything substantial, so we can agree on the approach.
29
+ 2. Fork, branch, and keep commits focused.
30
+ 3. Add or update tests where relevant.
31
+ 4. If you would like to be credited individually, add yourself to
32
+ [AUTHORS.md](AUTHORS.md) in the same pull request.
33
+ 5. Open a pull request.
34
+
35
+ ## 4. Questions
36
+
37
+ For anything about licensing, contact <louis.letarnec@deepika.ai>.
@@ -0,0 +1,21 @@
1
+ # MIT License
2
+
3
+ Copyright (c) 2026 Kannon For Deep Tech
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -1,12 +1,12 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: dot-inference
3
- Version: 2.2.0
3
+ Version: 3.1.0
4
4
  Summary: Unified async Python library for LLM inference and embeddings across multiple providers
5
5
  Project-URL: Homepage, https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference
6
6
  Project-URL: Repository, https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference
7
7
  Project-URL: Issues, https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/-/issues
8
8
  Author-email: Kannon For Deep Tech <louis.letarnec@deepika.ai>
9
- License-Expression: AGPL-3.0-or-later
9
+ License-Expression: MIT
10
10
  License-File: LICENSE.md
11
11
  Keywords: embeddings,inference,llm,providers
12
12
  Classifier: Development Status :: 3 - Alpha
@@ -29,7 +29,7 @@ Description-Content-Type: text/markdown
29
29
 
30
30
  [![PyPI](https://img.shields.io/pypi/v/dot-inference)](https://pypi.org/project/dot-inference/)
31
31
  ![Python Version](https://img.shields.io/badge/python-3.12%2B-blue)
32
- [![Licence: AGPL v3](https://img.shields.io/badge/licence-AGPL--3.0--or--later-blue)](LICENSE.md)
32
+ [![Licence: MIT](https://img.shields.io/badge/licence-MIT-blue)](LICENSE.md)
33
33
  [![Pipeline](https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/badges/main/pipeline.svg)](https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/-/pipelines)
34
34
 
35
35
  **One async interface for LLM inference and embeddings, across every provider.**
@@ -61,7 +61,9 @@ instead of silently doing nothing.
61
61
  - Multi-provider: OpenAI, Mistral, OpenRouter, and any OpenAI-compatible server (vLLM, SGLang, Ollama)
62
62
  - Structured outputs backed by Pydantic models
63
63
  - Rich event streaming: text, reasoning, incremental tool-call arguments, tool calls, usage and cost
64
- - Native tool calling with `tool_choice` control (`auto` / `required` / `none`)
64
+ - Image input: URLs, local files or bytes, with per-model vision support in the registry
65
+ - Native tool calling with `tool_choice` control (`auto` / `required` / `none` / a tool name) and strict tool schemas
66
+ - Provider-specific request fields per call (`extra_body`)
65
67
  - Embeddings with automatic batching
66
68
  - Retries on transient errors, honouring `Retry-After`, with exponential backoff
67
69
  - Cost tracking in tokens and USD
@@ -90,10 +92,14 @@ nesting delimiter — or from an env file, modelled on `.env.template`:
90
92
 
91
93
  ```bash
92
94
  DOTI_GENERATION__PROVIDER=openai
93
- DOTI_GENERATION__OPENAI__MODEL_NAME=gpt-5-mini
95
+ DOTI_GENERATION__OPENAI__MODEL_NAME=openai/gpt-5-mini
94
96
  DOTI_GENERATION__OPENAI__API_KEY=sk-...
95
97
  ```
96
98
 
99
+ Model names are registry ids (`openai/gpt-5-mini`, whatever the provider); each
100
+ provider is sent its own API name. A name missing from the registry is sent as
101
+ is, with a warning, and without the registry's capabilities and prices.
102
+
97
103
  API keys are held as `SecretStr`, so a settings object caught in a traceback or
98
104
  an error report never carries the key in clear text.
99
105
 
@@ -121,7 +127,7 @@ print(response.content)
121
127
  from dot_inference import InferenceSettings, Message, Provider, Role, get_llm_client
122
128
 
123
129
  settings = InferenceSettings()
124
- client = get_llm_client(settings, provider=Provider.OPENAI, model="gpt-5-mini")
130
+ client = get_llm_client(settings, provider=Provider.OPENAI, model="openai/gpt-5-mini")
125
131
  response = asyncio.run(client.call([Message(role=Role.USER, content="Hello!")]))
126
132
  ```
127
133
 
@@ -143,7 +149,7 @@ settings = InferenceSettings()
143
149
  client = get_llm_client(
144
150
  settings,
145
151
  provider=Provider.OPENAI,
146
- model="gpt-5-mini",
152
+ model="openai/gpt-5-mini",
147
153
  reasoning_effort=ReasoningEffort.HIGH,
148
154
  )
149
155
  response = asyncio.run(client.call([Message(role=Role.USER, content="Solve this step by step...")]))
@@ -162,7 +168,7 @@ class Answer(BaseModel):
162
168
 
163
169
 
164
170
  settings = InferenceSettings()
165
- client = get_llm_client(settings, provider=Provider.OPENAI, model="gpt-5-mini")
171
+ client = get_llm_client(settings, provider=Provider.OPENAI, model="openai/gpt-5-mini")
166
172
  response = asyncio.run(
167
173
  client.call(
168
174
  [Message(role=Role.USER, content="What is 2+2?")],
@@ -172,6 +178,32 @@ response = asyncio.run(
172
178
  print(response.structured_output) # Answer(answer='4', confidence=1.0)
173
179
  ```
174
180
 
181
+ OpenAI's strict mode wants every object closed (`"additionalProperties": false`) and every property listed in `required`. When the request reaches OpenAI (the `openai` provider, or an `openai/...` model on OpenRouter), a Pydantic model's schema is rewritten to those rules: an optional field becomes required but nullable, so the model answers `null` and Pydantic still gives `None`. A dict schema is sent as written and must follow the rules itself. Details and examples: [docs/provider-quirks/strict-schemas.md](docs/provider-quirks/strict-schemas.md).
182
+
183
+ ### Images
184
+
185
+ Give a message a list of content parts: `text_part()` for text, `image_part()`
186
+ for an image — an `http(s)://` or `data:` URL, a local file path, or raw bytes
187
+ (inlined as base64).
188
+
189
+ ```python
190
+ from dot_inference import Message, Role, image_part, text_part
191
+
192
+ message = Message(
193
+ role=Role.USER,
194
+ content=[
195
+ text_part("What is on this receipt?"),
196
+ image_part("receipt.jpg"),
197
+ image_part("https://example.com/logo.png", detail="low"),
198
+ ],
199
+ )
200
+ response = client.call_sync([message])
201
+ ```
202
+
203
+ The registry marks the models that read images (`supports_vision`). Sending an
204
+ image to a model registered without it raises `ValueError` before any request;
205
+ an unregistered model gets the request as-is.
206
+
175
207
  ### Streaming
176
208
 
177
209
  `stream()` yields typed `StreamEvent` objects:
@@ -279,6 +311,37 @@ client = get_llm_client(settings)
279
311
  client.on_retry = lambda attempt, max_attempts, err: print(f"Retry {attempt}/{max_attempts}: {err}")
280
312
  ```
281
313
 
314
+ `max_attempts` sets the total number of attempts per request (default 4 for OpenAI-compatible clients, 2 for Mistral). Set it to 1 to surface every error immediately, e.g. when measuring latency:
315
+
316
+ ```python
317
+ client = get_llm_client(settings, max_attempts=1)
318
+ ```
319
+
320
+ ### Strict tools, forced tool, provider-specific fields
321
+
322
+ ```python
323
+ class WeatherArgs(BaseModel):
324
+ city: str
325
+ unit: Literal["C", "F"] | None = None
326
+
327
+
328
+ tools = [
329
+ ToolDef(name="get_weather", description="Get weather", parameters=WeatherArgs, strict=True)
330
+ ]
331
+
332
+ response = await client.call(
333
+ messages,
334
+ tools=tools,
335
+ tool_choice="get_weather", # force this tool; or "auto" / "required" / "none"
336
+ extra_body={"ignore_eos": True}, # sent as-is (vLLM, SGLang, llama.cpp…)
337
+ )
338
+ args = tools[0].parse_arguments(response.tool_calls[0].arguments) # WeatherArgs(city='Paris', unit=None)
339
+ ```
340
+
341
+ `parameters` is a JSON schema dict, sent as written, or a Pydantic model. A model's schema is sent, rewritten to OpenAI's strict rules when `strict=True` (for every provider: OpenAI and Claude both reject a strict tool with an open object). A call's `arguments` stays the JSON string the provider sent; `ToolDef.parse_arguments()` validates it into the model, raising `ValidationError` when it does not match.
342
+
343
+ `strict=True` asks the server to constrain the arguments to the schema exactly; it is only sent when set, and OpenAI then imposes its own schema rules ([strict-schemas.md](docs/provider-quirks/strict-schemas.md)). Some models cannot be forced to call a tool (Claude Opus 5.5 and Sonnet 5.5): `"required"` or a tool name raises `ValueError` there ([tool-choice.md](docs/provider-quirks/tool-choice.md)). `extra_body` fields win over those the client adds itself (OpenRouter routing, `cache_control`); the Mistral client rejects `extra_body`.
344
+
282
345
  ## On-premise inference (vLLM, SGLang, Ollama)
283
346
 
284
347
  Point the `local` provider at any OpenAI-compatible endpoint:
@@ -297,6 +360,25 @@ response = client.call_sync([Message(role=Role.USER, content="How tall is the Ei
297
360
  print(response.content)
298
361
  ```
299
362
 
363
+ `list_served_models(base_url)` returns the model ids the server actually loaded
364
+ (`GET /models`), for any model. To send `reasoning_effort` to a model the
365
+ registry does not know, declare it (`temperature` is still sent alongside
366
+ it, as for any model with `supports_temperature`, the default):
367
+
368
+ ```python
369
+ from dot_inference import ModelConfig, ProviderModelConfig, list_served_models, register_model
370
+
371
+ base_url = "http://localhost:8000/v1"
372
+ served = (await list_served_models(base_url))[0]
373
+ register_model(ModelConfig(
374
+ model_id=served,
375
+ supports_reasoning=True,
376
+ supports_vision=True, # if the served model reads images
377
+ default_provider="local",
378
+ providers={"local": ProviderModelConfig(api_model_name=served)},
379
+ ))
380
+ ```
381
+
300
382
  ## Testing without API calls
301
383
 
302
384
  ```python
@@ -313,7 +395,7 @@ package is covered, anything underscore-prefixed is internal and may change in
313
395
  any release. Public names are never removed without a deprecation period.
314
396
 
315
397
  ```toml
316
- dependencies = ["dot-inference>=2.0,<3"]
398
+ dependencies = ["dot-inference>=3.0,<4"]
317
399
  ```
318
400
 
319
401
  See [docs/VERSIONING.md](docs/VERSIONING.md) for the full policy.
@@ -332,21 +414,17 @@ See [docs/VERSIONING.md](docs/VERSIONING.md) for the full policy.
332
414
  | [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) | Environment setup, tests, code style |
333
415
  | [docs/VERSIONING.md](docs/VERSIONING.md) | Versioning, deprecation policy, how to depend on this package |
334
416
  | [docs/PUBLISHING.md](docs/PUBLISHING.md) | Cutting a release |
335
- | [docs/provider-quirks/](docs/provider-quirks/) | Provider behaviours that differ from the OpenAI format: `tool_choice` support, OpenRouter error bodies |
417
+ | [docs/provider-quirks/](docs/provider-quirks/) | Provider behaviours that differ from the OpenAI format: `tool_choice` support, strict schemas, OpenRouter error bodies |
336
418
  | [CHANGELOG.md](CHANGELOG.md) | Release history |
337
419
 
338
420
  ## Contributing
339
421
 
340
- Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for the DCO
341
- sign-off requirement, the licensing terms that apply to contributions, and how to
342
- submit a change.
422
+ Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for the
423
+ licensing terms that apply to contributions and how to submit a change.
343
424
 
344
425
  ## Licence
345
426
 
346
427
  Copyright (C) 2026 Kannon For Deep Tech (deepika)
347
428
 
348
- This software is distributed under the GNU Affero General Public License,
349
- version 3 or later — see [LICENSE.md](LICENSE.md).
350
-
351
- A commercial licence is available for use in proprietary environments.
352
- Contact: louis.letarnec@deepika.ai
429
+ This software is distributed under the MIT License — see
430
+ [LICENSE.md](LICENSE.md).
@@ -2,7 +2,7 @@
2
2
 
3
3
  [![PyPI](https://img.shields.io/pypi/v/dot-inference)](https://pypi.org/project/dot-inference/)
4
4
  ![Python Version](https://img.shields.io/badge/python-3.12%2B-blue)
5
- [![Licence: AGPL v3](https://img.shields.io/badge/licence-AGPL--3.0--or--later-blue)](LICENSE.md)
5
+ [![Licence: MIT](https://img.shields.io/badge/licence-MIT-blue)](LICENSE.md)
6
6
  [![Pipeline](https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/badges/main/pipeline.svg)](https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-inference/-/pipelines)
7
7
 
8
8
  **One async interface for LLM inference and embeddings, across every provider.**
@@ -34,7 +34,9 @@ instead of silently doing nothing.
34
34
  - Multi-provider: OpenAI, Mistral, OpenRouter, and any OpenAI-compatible server (vLLM, SGLang, Ollama)
35
35
  - Structured outputs backed by Pydantic models
36
36
  - Rich event streaming: text, reasoning, incremental tool-call arguments, tool calls, usage and cost
37
- - Native tool calling with `tool_choice` control (`auto` / `required` / `none`)
37
+ - Image input: URLs, local files or bytes, with per-model vision support in the registry
38
+ - Native tool calling with `tool_choice` control (`auto` / `required` / `none` / a tool name) and strict tool schemas
39
+ - Provider-specific request fields per call (`extra_body`)
38
40
  - Embeddings with automatic batching
39
41
  - Retries on transient errors, honouring `Retry-After`, with exponential backoff
40
42
  - Cost tracking in tokens and USD
@@ -63,10 +65,14 @@ nesting delimiter — or from an env file, modelled on `.env.template`:
63
65
 
64
66
  ```bash
65
67
  DOTI_GENERATION__PROVIDER=openai
66
- DOTI_GENERATION__OPENAI__MODEL_NAME=gpt-5-mini
68
+ DOTI_GENERATION__OPENAI__MODEL_NAME=openai/gpt-5-mini
67
69
  DOTI_GENERATION__OPENAI__API_KEY=sk-...
68
70
  ```
69
71
 
72
+ Model names are registry ids (`openai/gpt-5-mini`, whatever the provider); each
73
+ provider is sent its own API name. A name missing from the registry is sent as
74
+ is, with a warning, and without the registry's capabilities and prices.
75
+
70
76
  API keys are held as `SecretStr`, so a settings object caught in a traceback or
71
77
  an error report never carries the key in clear text.
72
78
 
@@ -94,7 +100,7 @@ print(response.content)
94
100
  from dot_inference import InferenceSettings, Message, Provider, Role, get_llm_client
95
101
 
96
102
  settings = InferenceSettings()
97
- client = get_llm_client(settings, provider=Provider.OPENAI, model="gpt-5-mini")
103
+ client = get_llm_client(settings, provider=Provider.OPENAI, model="openai/gpt-5-mini")
98
104
  response = asyncio.run(client.call([Message(role=Role.USER, content="Hello!")]))
99
105
  ```
100
106
 
@@ -116,7 +122,7 @@ settings = InferenceSettings()
116
122
  client = get_llm_client(
117
123
  settings,
118
124
  provider=Provider.OPENAI,
119
- model="gpt-5-mini",
125
+ model="openai/gpt-5-mini",
120
126
  reasoning_effort=ReasoningEffort.HIGH,
121
127
  )
122
128
  response = asyncio.run(client.call([Message(role=Role.USER, content="Solve this step by step...")]))
@@ -135,7 +141,7 @@ class Answer(BaseModel):
135
141
 
136
142
 
137
143
  settings = InferenceSettings()
138
- client = get_llm_client(settings, provider=Provider.OPENAI, model="gpt-5-mini")
144
+ client = get_llm_client(settings, provider=Provider.OPENAI, model="openai/gpt-5-mini")
139
145
  response = asyncio.run(
140
146
  client.call(
141
147
  [Message(role=Role.USER, content="What is 2+2?")],
@@ -145,6 +151,32 @@ response = asyncio.run(
145
151
  print(response.structured_output) # Answer(answer='4', confidence=1.0)
146
152
  ```
147
153
 
154
+ OpenAI's strict mode wants every object closed (`"additionalProperties": false`) and every property listed in `required`. When the request reaches OpenAI (the `openai` provider, or an `openai/...` model on OpenRouter), a Pydantic model's schema is rewritten to those rules: an optional field becomes required but nullable, so the model answers `null` and Pydantic still gives `None`. A dict schema is sent as written and must follow the rules itself. Details and examples: [docs/provider-quirks/strict-schemas.md](docs/provider-quirks/strict-schemas.md).
155
+
156
+ ### Images
157
+
158
+ Give a message a list of content parts: `text_part()` for text, `image_part()`
159
+ for an image — an `http(s)://` or `data:` URL, a local file path, or raw bytes
160
+ (inlined as base64).
161
+
162
+ ```python
163
+ from dot_inference import Message, Role, image_part, text_part
164
+
165
+ message = Message(
166
+ role=Role.USER,
167
+ content=[
168
+ text_part("What is on this receipt?"),
169
+ image_part("receipt.jpg"),
170
+ image_part("https://example.com/logo.png", detail="low"),
171
+ ],
172
+ )
173
+ response = client.call_sync([message])
174
+ ```
175
+
176
+ The registry marks the models that read images (`supports_vision`). Sending an
177
+ image to a model registered without it raises `ValueError` before any request;
178
+ an unregistered model gets the request as-is.
179
+
148
180
  ### Streaming
149
181
 
150
182
  `stream()` yields typed `StreamEvent` objects:
@@ -252,6 +284,37 @@ client = get_llm_client(settings)
252
284
  client.on_retry = lambda attempt, max_attempts, err: print(f"Retry {attempt}/{max_attempts}: {err}")
253
285
  ```
254
286
 
287
+ `max_attempts` sets the total number of attempts per request (default 4 for OpenAI-compatible clients, 2 for Mistral). Set it to 1 to surface every error immediately, e.g. when measuring latency:
288
+
289
+ ```python
290
+ client = get_llm_client(settings, max_attempts=1)
291
+ ```
292
+
293
+ ### Strict tools, forced tool, provider-specific fields
294
+
295
+ ```python
296
+ class WeatherArgs(BaseModel):
297
+ city: str
298
+ unit: Literal["C", "F"] | None = None
299
+
300
+
301
+ tools = [
302
+ ToolDef(name="get_weather", description="Get weather", parameters=WeatherArgs, strict=True)
303
+ ]
304
+
305
+ response = await client.call(
306
+ messages,
307
+ tools=tools,
308
+ tool_choice="get_weather", # force this tool; or "auto" / "required" / "none"
309
+ extra_body={"ignore_eos": True}, # sent as-is (vLLM, SGLang, llama.cpp…)
310
+ )
311
+ args = tools[0].parse_arguments(response.tool_calls[0].arguments) # WeatherArgs(city='Paris', unit=None)
312
+ ```
313
+
314
+ `parameters` is a JSON schema dict, sent as written, or a Pydantic model. A model's schema is sent, rewritten to OpenAI's strict rules when `strict=True` (for every provider: OpenAI and Claude both reject a strict tool with an open object). A call's `arguments` stays the JSON string the provider sent; `ToolDef.parse_arguments()` validates it into the model, raising `ValidationError` when it does not match.
315
+
316
+ `strict=True` asks the server to constrain the arguments to the schema exactly; it is only sent when set, and OpenAI then imposes its own schema rules ([strict-schemas.md](docs/provider-quirks/strict-schemas.md)). Some models cannot be forced to call a tool (Claude Opus 5.5 and Sonnet 5.5): `"required"` or a tool name raises `ValueError` there ([tool-choice.md](docs/provider-quirks/tool-choice.md)). `extra_body` fields win over those the client adds itself (OpenRouter routing, `cache_control`); the Mistral client rejects `extra_body`.
317
+
255
318
  ## On-premise inference (vLLM, SGLang, Ollama)
256
319
 
257
320
  Point the `local` provider at any OpenAI-compatible endpoint:
@@ -270,6 +333,25 @@ response = client.call_sync([Message(role=Role.USER, content="How tall is the Ei
270
333
  print(response.content)
271
334
  ```
272
335
 
336
+ `list_served_models(base_url)` returns the model ids the server actually loaded
337
+ (`GET /models`), for any model. To send `reasoning_effort` to a model the
338
+ registry does not know, declare it (`temperature` is still sent alongside
339
+ it, as for any model with `supports_temperature`, the default):
340
+
341
+ ```python
342
+ from dot_inference import ModelConfig, ProviderModelConfig, list_served_models, register_model
343
+
344
+ base_url = "http://localhost:8000/v1"
345
+ served = (await list_served_models(base_url))[0]
346
+ register_model(ModelConfig(
347
+ model_id=served,
348
+ supports_reasoning=True,
349
+ supports_vision=True, # if the served model reads images
350
+ default_provider="local",
351
+ providers={"local": ProviderModelConfig(api_model_name=served)},
352
+ ))
353
+ ```
354
+
273
355
  ## Testing without API calls
274
356
 
275
357
  ```python
@@ -286,7 +368,7 @@ package is covered, anything underscore-prefixed is internal and may change in
286
368
  any release. Public names are never removed without a deprecation period.
287
369
 
288
370
  ```toml
289
- dependencies = ["dot-inference>=2.0,<3"]
371
+ dependencies = ["dot-inference>=3.0,<4"]
290
372
  ```
291
373
 
292
374
  See [docs/VERSIONING.md](docs/VERSIONING.md) for the full policy.
@@ -305,21 +387,17 @@ See [docs/VERSIONING.md](docs/VERSIONING.md) for the full policy.
305
387
  | [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) | Environment setup, tests, code style |
306
388
  | [docs/VERSIONING.md](docs/VERSIONING.md) | Versioning, deprecation policy, how to depend on this package |
307
389
  | [docs/PUBLISHING.md](docs/PUBLISHING.md) | Cutting a release |
308
- | [docs/provider-quirks/](docs/provider-quirks/) | Provider behaviours that differ from the OpenAI format: `tool_choice` support, OpenRouter error bodies |
390
+ | [docs/provider-quirks/](docs/provider-quirks/) | Provider behaviours that differ from the OpenAI format: `tool_choice` support, strict schemas, OpenRouter error bodies |
309
391
  | [CHANGELOG.md](CHANGELOG.md) | Release history |
310
392
 
311
393
  ## Contributing
312
394
 
313
- Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for the DCO
314
- sign-off requirement, the licensing terms that apply to contributions, and how to
315
- submit a change.
395
+ Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for the
396
+ licensing terms that apply to contributions and how to submit a change.
316
397
 
317
398
  ## Licence
318
399
 
319
400
  Copyright (C) 2026 Kannon For Deep Tech (deepika)
320
401
 
321
- This software is distributed under the GNU Affero General Public License,
322
- version 3 or later — see [LICENSE.md](LICENSE.md).
323
-
324
- A commercial licence is available for use in proprietary environments.
325
- Contact: louis.letarnec@deepika.ai
402
+ This software is distributed under the MIT License — see
403
+ [LICENSE.md](LICENSE.md).