tunarag-python 0.3.0__py3-none-any.whl → 0.3.1__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
tunarag/__init__.py CHANGED
@@ -165,4 +165,4 @@ __all__ = [
165
165
  "__version__",
166
166
  ]
167
167
 
168
- __version__ = "0.3.0"
168
+ __version__ = "0.3.1"
@@ -1,12 +1,14 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: tunarag-python
3
- Version: 0.3.0
3
+ Version: 0.3.1
4
4
  Summary: Tune LangChain, LangGraph, and Google ADK RAG pipelines
5
+ Project-URL: Documentation, https://shivamshinde123.github.io/tunarag-python/
5
6
  Project-URL: Repository, https://github.com/shivamshinde123/tunarag-python
6
7
  Project-URL: Issues, https://github.com/shivamshinde123/tunarag-python/issues
7
8
  Project-URL: Changelog, https://github.com/shivamshinde123/tunarag-python/blob/main/CHANGELOG.md
8
9
  Author: Shivam Shinde
9
10
  License-Expression: MIT
11
+ License-File: LICENSE
10
12
  Keywords: evaluation,llm,optimization,rag,retrieval
11
13
  Classifier: Development Status :: 3 - Alpha
12
14
  Classifier: Intended Audience :: Developers
@@ -102,21 +104,21 @@ indexes, provider clients, and credentials; TunaRAG owns the optimization workfl
102
104
 
103
105
  **Lightweight distribution:** pip installs runtime code and metadata, not tests, datasets,
104
106
  or documentation images. Framework dependencies are optional extras. See
105
- [package contents and release checks](docs/package-distribution.md).
107
+ [package contents and release checks](https://shivamshinde123.github.io/tunarag-python/package-distribution/).
106
108
 
107
- **Live integration checks:** [OpenRouter, RAGAS, and MLflow test plan](docs/live-integration-test-plan.md).
109
+ **Live integration checks:** [OpenRouter, RAGAS, and MLflow test plan](https://shivamshinde123.github.io/tunarag-python/live-integration-test-plan/).
108
110
 
109
111
  **End-to-end verification:** [Full-PDF campaigns, held-out quality checks, crash recovery,
110
- and clean-wheel testing](docs/comprehensive-testing.md). Paid tests are explicit opt-ins;
112
+ and clean-wheel testing](https://shivamshinde123.github.io/tunarag-python/comprehensive-testing/). Paid tests are explicit opt-ins;
111
113
  ordinary CI never makes paid provider calls.
112
114
  These opt-in tests include spending safeguards; normal tests never make paid calls.
113
115
 
114
116
  **Google ADK validation:** [Real OpenRouter RAG tuning, RAGAS scoring, SQLite resume,
115
- and MLflow checks](docs/live-google-adk-testing.md).
117
+ and MLflow checks](https://shivamshinde123.github.io/tunarag-python/live-google-adk-testing/).
116
118
 
117
- **Quality and reliability:** [Ordered validation plan](docs/quality-validation-plan.md)
118
- and [bounded repeated tuning, ADK tools, billing and recovery checks](docs/reliability-validation.md).
119
- See [mixed-topic validation](docs/multicorpus-validation.md) for cross-model QA,
119
+ **Quality and reliability:** [Ordered validation plan](https://shivamshinde123.github.io/tunarag-python/quality-validation-plan/)
120
+ and [bounded repeated tuning, ADK tools, billing and recovery checks](https://shivamshinde123.github.io/tunarag-python/reliability-validation/).
121
+ See [mixed-topic validation](https://shivamshinde123.github.io/tunarag-python/multicorpus-validation/) for cross-model QA,
120
122
  simulated faults/load, and the unresolved adversarial-prompt failure.
121
123
 
122
124
  ## Comprehensive Usage Guide
@@ -154,27 +156,12 @@ TunaRAG compares configurations of an **existing** RAG pipeline. You provide the
154
156
  evaluation dataset, a search space, evaluators, and an objective. TunaRAG schedules trials, isolates
155
157
  failures, persists state, and returns an inspectable result.
156
158
 
157
- ```mermaid
158
- flowchart LR
159
- D[Versioned dataset] --> O[Optimizer]
160
- S[Search strategy] -->|Candidate| O
161
- O -->|Candidate + example| A[RAG adapter]
162
- A -->|Output + usage| E[Evaluators]
163
- E -->|Metrics| J[Objective]
164
- J -->|Benefit score| O
165
- O -->|Observe| S
166
- O --> DB[(SQLite store)]
167
- O <--> C[(Evaluation cache)]
168
- DB --> CB[Post-commit callbacks]
169
- O --> R[OptimizationResult]
170
- ```
159
+ ![Optimization loop: a strategy proposes candidates, the framework adapter runs each example, evaluators produce metrics, and the objective scores trials; SQLite persists results before callbacks](https://shivamshinde123.github.io/tunarag-python/assets/diagrams/optimization-loop.png)
171
160
 
172
161
  A **study** contains ordered **trials**. Each trial evaluates one complete candidate configuration
173
162
  against every dataset example. Same-named sample metrics are aggregated into trial metrics, then the
174
163
  objective converts those metrics into one larger-is-better score.
175
164
 
176
- ![A hand-drawn Xiaohei operating TunaRAG's iterative optimization loop](docs/assets/tunarag-python-readme-illustrations/01-optimization-loop.png)
177
-
178
165
  TunaRAG does not create or host a RAG application, provision indexes, manage provider credentials,
179
166
  or expose a hosted API. Your application keeps ownership of those concerns.
180
167
 
@@ -298,7 +285,10 @@ chain = RunnableLambda(lambda inputs: {"answer": inputs["question"]})
298
285
  adapter = LangChainAdapter(chain)
299
286
  ```
300
287
 
301
- ![A hand-drawn Xiaohei connecting a framework RAG pipeline through TunaRAG](docs/assets/tunarag-python-readme-illustrations/02-rag-adapter-boundary.png)
288
+ ![TunaRAG sends settings and a query through an official adapter to a separate framework RAG; answers and usage return in the opposite direction](https://shivamshinde123.github.io/tunarag-python/assets/tunarag-python-readme-illustrations/07-framework-adapter-control.png)
289
+
290
+ The adapter connects two responsibilities; it does not move your RAG into TunaRAG.
291
+ Your framework must consume the candidate settings for tuning to change its behavior.
302
292
 
303
293
  Candidate parameters go into framework configuration or initial ADK session state. A parameter
304
294
  only changes behavior if the framework consumes it. For model, prompt, retriever, or index choices
@@ -472,8 +462,6 @@ and [integration reference](https://github.com/shivamshinde123/tunarag-python/bl
472
462
 
473
463
  ## Evaluation datasets
474
464
 
475
- ![A hand-drawn Xiaohei normalizing, validating, hashing, and splitting evaluation records](docs/assets/tunarag-python-readme-illustrations/03-dataset-preparation.png)
476
-
477
465
  ### Canonical model
478
466
 
479
467
  ```python
@@ -542,6 +530,13 @@ optimization_data = split_dataset.select(DatasetSplit.OPTIMIZE)
542
530
  Splits are hash-based and stable across input order. Group splitting keeps related examples
543
531
  together. `select()` rejects an empty split.
544
532
 
533
+ ![Tune and choose a configuration using optimization data while keeping the test set untouched for a final check](https://shivamshinde123.github.io/tunarag-python/assets/tunarag-python-readme-illustrations/08-held-out-evaluation.png)
534
+
535
+ Pass only the selected optimization subset to the tuning study. Use validation data for
536
+ development decisions, and evaluate the frozen winner on the untouched test subset.
537
+ TunaRAG does not automatically enforce this separation or run the final held-out evaluation;
538
+ calling `optimize()` with the test set would use it for parameter selection and leak test data.
539
+
545
540
  For custom examples, use a user-managed version:
546
541
 
547
542
  ```python
@@ -608,7 +603,7 @@ using them as gold data. Generation usage is separate from optimization usage.
608
603
 
609
604
  ## Search spaces and strategies
610
605
 
611
- ![A hand-drawn Xiaohei validating candidates from RandomSearch and LLMSearch](docs/assets/tunarag-python-readme-illustrations/04-search-strategies.png)
606
+ ![A hand-drawn Xiaohei validating candidates from RandomSearch and LLMSearch](https://shivamshinde123.github.io/tunarag-python/assets/tunarag-python-readme-illustrations/04-search-strategies.png)
612
607
 
613
608
  ### Typed space
614
609
 
@@ -748,6 +743,11 @@ configuration remain owned by your RAGAS components.
748
743
 
749
744
  ## Objectives
750
745
 
746
+ ![A hand-drawn Xiaohei adjusting quality, cost, and latency tradeoffs to choose a configuration](https://shivamshinde123.github.io/tunarag-python/assets/tunarag-python-readme-illustrations/06-results-and-reports.png)
747
+
748
+ Your objective weights define the tradeoff; the winning configuration is best among the
749
+ evaluated trials under that objective, not a guarantee of the globally best RAG.
750
+
751
751
  ```python
752
752
  from tunarag import Objective, ObjectiveDirection, ObjectiveTerm
753
753
 
@@ -985,7 +985,7 @@ labels are metadata, not encryption.
985
985
 
986
986
  ## SQLite persistence and resume
987
987
 
988
- ![A hand-drawn Xiaohei recovering and resuming a committed SQLite study after interruption](docs/assets/tunarag-python-readme-illustrations/05-persistence-and-resume.png)
988
+ ![A hand-drawn Xiaohei recovering and resuming a committed SQLite study after interruption](https://shivamshinde123.github.io/tunarag-python/assets/tunarag-python-readme-illustrations/05-persistence-and-resume.png)
989
989
 
990
990
  ```python
991
991
  from pathlib import Path
@@ -1044,18 +1044,7 @@ Events cover study, trial, and attempt lifecycles plus callback failures. Handle
1044
1044
  types defensively. Callback failures never roll back committed work; TunaRAG records them as durable
1045
1045
  events. Make callbacks idempotent.
1046
1046
 
1047
- ```mermaid
1048
- sequenceDiagram
1049
- participant O as Optimizer
1050
- participant S as SQLite
1051
- participant C as Callback
1052
- O->>S: Commit terminal trial and event
1053
- S-->>O: Committed
1054
- O->>C: on_event(event)
1055
- alt callback fails
1056
- O->>S: Append callback.failed
1057
- end
1058
- ```
1047
+ ![SQLite commits the terminal trial and event before the callback runs; a failing callback appends a callback.failed event without rolling back the trial](https://shivamshinde123.github.io/tunarag-python/assets/diagrams/post-commit-callbacks.png)
1059
1048
 
1060
1049
  MLflow is an optional post-commit projection; SQLite remains authoritative:
1061
1050
 
@@ -1084,8 +1073,6 @@ reconciliation is not included in V0.2.
1084
1073
 
1085
1074
  ## Results and reports
1086
1075
 
1087
- ![A hand-drawn Xiaohei balancing quality, cost, and latency before exporting the winning result](docs/assets/tunarag-python-readme-illustrations/06-results-and-reports.png)
1088
-
1089
1076
  `OptimizationResult` exposes:
1090
1077
 
1091
1078
  ```python
@@ -1245,6 +1232,24 @@ LangChain, LangGraph, and Google ADK 2.x are official integrations. Additional f
1245
1232
  | Trial | One candidate evaluated over the whole dataset |
1246
1233
  | Usage | Tokens, cost, latency, confidence, and pricing provenance |
1247
1234
 
1248
- For deeper contracts and architecture, see [`docs/api-specification.md`](docs/api-specification.md),
1249
- [`docs/system-architecture-document.md`](docs/system-architecture-document.md), and
1250
- [`docs/technical-design-document.md`](docs/technical-design-document.md).
1235
+ For deeper contracts and architecture, see the [API specification](https://shivamshinde123.github.io/tunarag-python/api-specification/),
1236
+ [system architecture](https://shivamshinde123.github.io/tunarag-python/system-architecture-document/), and
1237
+ [technical design](https://shivamshinde123.github.io/tunarag-python/technical-design-document/).
1238
+
1239
+ ## License and responsibility
1240
+
1241
+ TunaRAG is distributed under the [MIT License](https://opensource.org/license/mit).
1242
+ The full text and copyright notice are in the repository and distribution `LICENSE` file.
1243
+ It permits use, modification,
1244
+ and redistribution subject to retaining the copyright and permission notice. The software
1245
+ is provided **as is, without warranty**, and the license includes a limitation of authors'
1246
+ and copyright holders' liability. The complete license text, not this summary, governs.
1247
+
1248
+ You are responsible for testing fitness for your use case, validating generated answers,
1249
+ protecting data and credentials, setting provider-side spending limits, and monitoring costs.
1250
+ TunaRAG does not guarantee accuracy, security, optimization improvements, or provider bills.
1251
+ See the [Security Policy](https://github.com/shivamshinde123/tunarag-python/blob/main/SECURITY.md)
1252
+ for boundaries and known responsibilities.
1253
+
1254
+ No license can guarantee protection from every claim or override applicable mandatory law.
1255
+ Seek qualified legal advice for liability-sensitive use or distribution.
@@ -1,4 +1,4 @@
1
- tunarag/__init__.py,sha256=RlM5mZYGfa6ZsvmYY3xJCd9MRCRr3vMUqgC3DmaxmuI,3695
1
+ tunarag/__init__.py,sha256=1ugGAQZJbEWCvjzENV0mizMhgQKpAIIvfGt_gpc3ojk,3695
2
2
  tunarag/cache.py,sha256=prra0uAjW1JYVez6nA9xXFnRwANODHo1rj6o1EiGLI8,11556
3
3
  tunarag/config.py,sha256=EzoRWhViL4OQRMFadhiNwEwvne64p2XYMgigaV4tDqg,9325
4
4
  tunarag/contracts.py,sha256=H3rpzKJjN9F34t6ukJj4q7iBrOjFa9PzOwLk-s8A_Pk,2073
@@ -20,6 +20,7 @@ tunarag/integrations/__init__.py,sha256=7eE5ScMAhleHTFwqglB5ZXR4oyjblovIIosIYYcO
20
20
  tunarag/integrations/google_adk.py,sha256=P_fL0g60Ok7dirwQlGxBq6BBMxNrP8OSEroSk9ciB8Q,5942
21
21
  tunarag/integrations/mlflow.py,sha256=Ig96h8CItgEnm-OX41Sp_MQvW9cbQureneZR2xCdikM,5807
22
22
  tunarag/integrations/runnables.py,sha256=B9RLnrpn8nZnYnOqkbDQmMmul2SfxOG8oOE-fiPv948,10268
23
- tunarag_python-0.3.0.dist-info/METADATA,sha256=RwyQBcK6PfD1wMlpO-OQNYs2jxVQiQSl7cu3EdQvz8E,44254
24
- tunarag_python-0.3.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
25
- tunarag_python-0.3.0.dist-info/RECORD,,
23
+ tunarag_python-0.3.1.dist-info/METADATA,sha256=oC8AqMNj6aJWy1hKa5kss90R4q6EYW7V93TlBYnxI3U,46536
24
+ tunarag_python-0.3.1.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
25
+ tunarag_python-0.3.1.dist-info/licenses/LICENSE,sha256=ve5VOCp_tHEW3KWjdX4JuIMa0TKdOyAXyaSIS9okbUA,1070
26
+ tunarag_python-0.3.1.dist-info/RECORD,,
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Shivam Shinde
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.