AutoRAG 0.2.5__tar.gz → 0.2.7__tar.gz

{autorag-0.2.5 → autorag-0.2.7}/.github/workflows/sphinx.yml

@@ -22,6 +22,9 @@ jobs:
     - name: Build HTML
       run: |
         sphinx-build -b html docs/source docs/build/html
+    - name: Copy CNAME
+      run: |
+        cp docs/source/CNAME docs/build/html/CNAME
     - name: Upload artifacts
       uses: actions/upload-artifact@v3
       with:

{autorag-0.2.5 → autorag-0.2.7}/AutoRAG.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: AutoRAG
-Version: 0.2.5
+Version: 0.2.7
 Summary: Automatically Evaluate RAG pipelines with your own data. Find optimal structure for new RAG product.
 Author-email: Marker-Inc <vkehfdl1@gmail.com>
 License:                                  Apache License
@@ -210,16 +210,17 @@ Keywords: RAG,AutoRAG,autorag,rag-evaluation,evaluation,rag-auto,AutoML,AutoML-R
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Information Technology
 Classifier: Intended Audience :: Science/Research
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Classifier: Topic :: Software Development :: Libraries
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: numpy
+Requires-Dist: numpy<2.0.0
 Requires-Dist: pandas
 Requires-Dist: tqdm
 Requires-Dist: tiktoken>=0.7.0
@@ -248,16 +249,17 @@ Requires-Dist: bert_score
 Requires-Dist: sentence-transformers
 Requires-Dist: FlagEmbedding
 Requires-Dist: ragas
-Requires-Dist: ray
 Requires-Dist: kiwipiepy
 Requires-Dist: llmlingua
 Requires-Dist: llama-index>=0.10.1
 Requires-Dist: llama-index-core>=0.10.1
+Requires-Dist: llama-index-readers-file
 Requires-Dist: llama-index-embeddings-openai
 Requires-Dist: llama-index-embeddings-huggingface
 Requires-Dist: llama-index-llms-openai
 Requires-Dist: llama-index-llms-huggingface
 Requires-Dist: llama-index-llms-openai-like
+Requires-Dist: llama-index-llms-ollama
 Requires-Dist: llama-index-retrievers-bm25
 Requires-Dist: streamlit
 Requires-Dist: langchain-core>=0.1.6
@@ -271,19 +273,20 @@ Requires-Dist: ipywidgets_bokeh
 
 RAG AutoML tool for automatically finds an optimal RAG pipeline for your data.
 
-Explore our 📖 [Document](https://marker-inc-korea.github.io/AutoRAG/)!!
+Explore our 📖 [Document](https://docs.auto-rag.com)!!
 
 Plus, join our 📞 [Discord](https://discord.gg/P4DYXfmSAs) Community.
 
 ---
 
-📌 Colab Tutorial
+### 💪 Colab Tutorial
 
 - [Step 1: Basic of AutoRAG | Optimizing your RAG pipeline](https://colab.research.google.com/drive/19OEQXO_pHN6gnn2WdfPd4hjnS-4GurVd?usp=sharing)
 - [Step 2: Create evaluation dataset](https://colab.research.google.com/drive/1HXjVHCLTaX7mkmZp3IKlEPt0B3jVeHvP#scrollTo=cgFUCuaUZvTr)
 
 ---
-🚨 YouTube Tutorial
+
+### 🚨 YouTube Tutorial
 
 https://github.com/Marker-Inc-Korea/AutoRAG/assets/96727832/c0d23896-40c0-479f-a17b-aa2ec3183a26
 
@@ -291,24 +294,23 @@ _Muted by default, enable sound for voice-over_
 
 You can see on [YouTube](https://youtu.be/2ojK8xjyXAU?feature=shared)
 
-# 📑 Index
-
-- [Introduction](#introduction)
-- [Quick Install](#-quick-install)
-- [Index](#-index)
-- [Strengths](#-strengths)
-- [QuickStart](#-quickstart)
-  - [1. Prepare your evaluation data](#1-prepare-your-evaluation-data)
-  - [2. Evaluate your data to various RAG modules](#2-evaluate-your-data-to-various-rag-modules)
-  - [3. Use a found optimal RAG pipeline](#3-use-a-found-optimal-rag-pipeline)
-  - [4. Run Dashboard to see the result](#4-run-dashboard)
-  - [5. Share your RAG pipeline](#5-share-your-rag-pipeline)
-  - [+ Config yaml file](#-create-your-own-config-yaml-file)
-- [Supporting RAG modules](#supporting-nodes--modules)
-- [Roadmap](#roadmap)
-- [Contribution](#contribution)
-
-# Introduction
+---
+
+### ☎️ FaQ
+
+🛣️ [Support plans & Roadmap](https://edai.notion.site/Support-plans-Roadmap-02ca7c97376340c393885855e2d99630?pvs=4)
+
+💻 [Hardware Specs](https://edai.notion.site/Hardware-specs-28cefcf2a26246ffadc91e2f3dc3d61c?pvs=4)
+
+⭐ [Running AutoRAG](https://edai.notion.site/About-running-AutoRAG-44a8058307af42068fc218a073ee480b?pvs=4)
+
+🍯 [Tips/Tricks](https://edai.notion.site/Tips-Tricks-10708a0e36ff461cb8a5d4fb3279ff15?pvs=4)
+
+☎️ [TroubleShooting](https://medium.com/@autorag/autorag-troubleshooting-5cf872b100e3)
+
+---
+
+# 😁 Introduction
 
 There are many RAG pipelines and modules out there,
 but you don’t know what pipeline is great for “your own data” and "your own use-case."
@@ -322,34 +324,35 @@ and find the best RAG pipeline for your own use-case.
 AutoRAG supports a simple way to evaluate many RAG module combinations.
 Try now and find the best RAG pipeline for your own use-case.
 
-# ⚡ Quick Install
+# 📌AutoRAG Structure
 
-```bash
-pip install AutoRAG
-```
+![Node](https://github.com/Marker-Inc-Korea/AutoRAG/assets/96727832/79dda7ba-e9d8-4552-9e7b-6a5f9edc4c1a)
 
-# 💪 Strengths
+## ❗Supporting Nodes & modules
 
-### **1. Find your RAG baseline**
+![module_1](https://github.com/Marker-Inc-Korea/AutoRAG/assets/96727832/828d8752-13f2-4918-8ca4-2605a86596cb)
+![module_2](https://github.com/Marker-Inc-Korea/AutoRAG/assets/96727832/4db3d83f-3178-4b81-bad1-b751dcd31a13)
+You can check our all supporting Nodes & modules
+at [here](https://edai.notion.site/Supporting-Nodes-modules-0ebc7810649f4e41aead472a92976be4?pvs=4)
 
-Benchmark RAG pipelines with few lines of code. You can quickly get a high-performance RAG
-pipeline just for your data. Don’t waste time dealing with complex RAG modules and academic paper. Focus on your data.
+## ❗Supporting Evaluation Metrics
 
-### **2. Analyze where is wrong**
+![Metrics](https://github.com/Marker-Inc-Korea/AutoRAG/assets/96727832/9f090e3c-d762-41cb-9f1d-cd8d7da137dd)
 
-Sometimes it is hard to keep tracking where is the major problem within your RAG pipeline.
-AutoRAG gives you the data of it, so you can analyze and focus where is the major problem and where you to focus on.
+You can check our all supporting Evaluation Metrics
+at [here](https://edai.notion.site/Supporting-metrics-867d71caefd7401c9264dd91ba406043?pvs=4)
 
-### **3. Quick Starter Pack for your new RAG product**
+- [Retrieval Metrics](https://edai.notion.site/Retrieval-Metrics-dde3d9fa1d9547cdb8b31b94060d21e7?pvs=4)
+- [Retrieval Token Metrics](https://edai.notion.site/Retrieval-Token-Metrics-c3e2d83358e04510a34b80429ebb543f?pvs=4)
+- [Generation Metrics](https://edai.notion.site/Retrieval-Token-Metrics-c3e2d83358e04510a34b80429ebb543f?pvs=4)
 
-Get the most effective RAG workflow among many pipelines, and start from
-there.
-Don’t start at toy-project level, start from advanced level.
+# ⚡ Quick Install
 
-### **4. Share your experiment to others**
+We recommend using Python version 3.9 or higher for AutoRAG.
 
-It's really easy to share your experiment to others. Share your config yaml file and
-summary csv files. Plus, check out others result and adapt to your use-case.
+```bash
+pip install AutoRAG
+```
 
 # ⚡ QuickStart
 
@@ -357,16 +360,16 @@ summary csv files. Plus, check out others result and adapt to your use-case.
 
 For evaluation, you need to prepare just three files.
 
-- QA dataset file (qa.parquet)
-- Corpus dataset file (corpus.parquet)
-- Config yaml file (config.yaml)
+1. **QA** dataset file (qa.parquet)
+2. **Corpus** dataset file (corpus.parquet)
+3. **Config yaml file** (config.yaml)
 
 There is a template for your evaluation data for using AutoRAG.
 
 - Check out how to make evaluation data
-  at [here](https://marker-inc-korea.github.io/AutoRAG/data_creation/tutorial.html).
+  at [here](https://docs.auto-rag.com/data_creation/tutorial.html).
 - Check out the evaluation data rule
-  at [here](https://marker-inc-korea.github.io/AutoRAG/data_creation/data_format.html).
+  at [here](https://docs.auto-rag.com/data_creation/data_format.html).
 - Plus, you can get example datasets for testing AutoRAG at [here](./sample_dataset).
 
 ### 2. Evaluate your data to various RAG modules
@@ -397,7 +400,7 @@ At the trial folder named to numbers (like 0),
 you can check `summary.csv` file that summarizes the evaluation results and the best RAG pipeline for your data.
 
 For more details, you can check out how the folder structure looks like
-at [here](https://marker-inc-korea.github.io/AutoRAG/optimization/folder_structure.html).
+at [here](https://docs.auto-rag.com/optimization/folder_structure.html).
 
 ### 3. Use a found optimal RAG pipeline
 
@@ -417,7 +420,7 @@ runner.run('your question')
 
 Or, you can run this pipeline as api server.
 You can use python code or CLI command.
-Check out API endpoint at [here](https://marker-inc-korea.github.io/AutoRAG/deploy/api_endpoint.html).
+Check out API endpoint at [here](https://docs.auto-rag.com/deploy/api_endpoint.html).
 
 ```python
 from autorag.deploy import Runner
@@ -440,6 +443,10 @@ You can run dashboard to easily see the result.
 autorag dashboard --trial_dir /your/path/to/trial_dir
 ```
 
+- sample dashboard:
+
+![dashboard](https://github.com/Marker-Inc-Korea/AutoRAG/assets/96727832/3798827d-31d7-4c4e-a9b1-54340b964e53)
+
 ### 5. Share your RAG pipeline
 
 You can use your RAG pipeline from extracted pipeline yaml file.
@@ -454,90 +461,61 @@ from autorag.deploy import extract_best_config
 pipeline_dict = extract_best_config(trial_path='your/path/to/trial_folder', output_path='your/path/to/pipeline.yaml')
 ```
 
-### ➕ Create your own Config yaml file
-
-You can build your own evaluation process with config yaml file.
-You can check detailed explanation how to configure each module and node
-at [here](https://marker-inc-korea.github.io/AutoRAG/nodes/index.html#).
-
-There is a simple yaml file example.
-
-It evaluates two retrieval modules which are BM25 and Vector Retriever, and three reranking modules.
-Lastly, it generates prompt and makes generation with two other LLM models and three temperatures.
-
-```yaml
-node_lines:
-  - node_line_name: retrieve_node_line
-    nodes:
-      - node_type: retrieval
-        strategy:
-          metric: retrieval_f1
-        top_k: 50
-        modules:
-          - module_type: bm25
-          - module_type: vector
-            embedding_model: [ openai, openai_embed_3_large ]
-          - module_type: hybrid_rrf
-            target_modules: ('bm25', 'vectordb')
-            rrf_k: [ 3, 5, 10 ]
-      - node_type: reranker
-        strategy:
-          metric: retrieval_precision
-          speed_threshold: 5
-        top_k: 3
-        modules:
-          - module_type: upr
-          - module_type: tart
-            prompt: Arrange the following sentences in the correct order.
-          - module_type: monoT5
-  - node_line_name: generate_node_line
-    nodes:
-      - node_type: prompt_maker
-        modules:
-          - module_type: fstring
-            prompt: "This is a news dataset, crawled from finance news site. You need to make detailed question about finance news. Do not make questions that not relevant to economy or finance domain.\n{retrieved_contents}\n\nQ: {query}\nA:"
-      - node_type: generator
-        strategy:
-          metric:
-            - metric_name: meteor
-            - metric_name: rouge
-            - metric_name: sem_score
-              embedding_model: openai
-            - metric_name: g_eval
-              model: gpt-3.5-turbo
-        modules:
-          - module_type: llama_index_llm
-            llm: openai
-            model: [ gpt-3.5-turbo-16k, gpt-3.5-turbo-1106 ]
-            temperature: [ 0.5, 1.0, 1.5 ]
+### 6. Deploy your optimal RAG pipeline (for testing)
+
+### 6-1. Run as a CLI
+
+You can use a found optimal RAG pipeline right away with extracted yaml file.
+
+```python
+from autorag.deploy import Runner
+
+runner = Runner.from_yaml('your/path/to/pipeline.yaml')
+runner.run('your question')
+```
+
+### 6-2. Run as an API server
+
+You can run this pipeline as an API server.
+
+Check out API endpoint at [here](deploy/api_endpoint.md).
+
+```python
+from autorag.deploy import Runner
+
+runner = Runner.from_yaml('your/path/to/pipeline.yaml')
+runner.run_api_server()
+```
+
+```bash
+autorag run_api --config_path your/path/to/pipeline.yaml --host 0.0.0.0 --port 8000
+```
+
+### 6-3. Run as a Web Interface
+
+you can run this pipeline as a web interface.
 
+Check out web interface at [here](deploy/web.md).
+
+```bash
+autorag run_web --trial_path your/path/to/trial_path
 ```
 
-# ❗Supporting Nodes & modules
+- sample web interface:
 
-You can check our all supporting Nodes & modules
-at [here](https://edai.notion.site/Supporting-Nodes-modules-0ebc7810649f4e41aead472a92976be4?pvs=4)
+<img width="1491" alt="web_interface" src="https://github.com/Marker-Inc-Korea/AutoRAG/assets/96727832/f6b00353-f6bb-4d8f-8740-1c264c0acbb8">
 
-# ❗Supporting Evaluation Metrics
+# Star History
 
-You can check our all supporting Evaluation Metrics
-at [here](https://edai.notion.site/Supporting-metrics-867d71caefd7401c9264dd91ba406043?pvs=4)
+[![Star History Chart](https://api.star-history.com/svg?repos=Marker-Inc-Korea/AutoRAG&type=Date)](https://star-history.com/#Marker-Inc-Korea/AutoRAG&Date)
 
-- [Retrieval Metrics](https://edai.notion.site/Retrieval-Metrics-dde3d9fa1d9547cdb8b31b94060d21e7?pvs=4)
-- [Retrieval Token Metrics](https://edai.notion.site/Retrieval-Token-Metrics-c3e2d83358e04510a34b80429ebb543f?pvs=4)
-- [Generation Metrics](https://edai.notion.site/Retrieval-Token-Metrics-c3e2d83358e04510a34b80429ebb543f?pvs=4)
+# Contributors ⭐
 
-# 🛣Roadmap
+Thanks go to these wonderful people:
 
-- [ ] Policy Module for modular RAG pipeline
-- [x] Visualize evaluation result
-- [ ] Visualize config yaml file
-- [x] More RAG modules support
-- [x] Token usage strategy
-- [ ] Multi-modal support
-- [x] More evaluation metrics
-- [ ] Answer Filtering Module
-- [x] Restart optimization from previous trial
+<a href="https://github.com/Marker-Inc-Korea/AutoRAG/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=Marker-Inc-Korea/AutoRAG" />
+</a>
 
 # Contribution
 
@@ -545,4 +523,4 @@ We are developing AutoRAG as open-source.
 
 So this project welcomes contributions and suggestions. Feel free to contribute to this project.
 
-Plus, check out our detailed documentation at [here](https://marker-inc-korea.github.io/AutoRAG/index.html).
+Plus, check out our detailed documentation at [here](https://docs.auto-rag.com/index.html).

{autorag-0.2.5 → autorag-0.2.7}/AutoRAG.egg-info/SOURCES.txt

@@ -38,20 +38,20 @@ autorag/data/qacreation/ragas.py
 autorag/data/qacreation/simple.py
 autorag/data/utils/__init__.py
 autorag/data/utils/util.py
-autorag/evaluate/__init__.py
-autorag/evaluate/generation.py
-autorag/evaluate/retrieval.py
-autorag/evaluate/retrieval_contents.py
-autorag/evaluate/util.py
-autorag/evaluate/metric/__init__.py
-autorag/evaluate/metric/generation.py
-autorag/evaluate/metric/retrieval.py
-autorag/evaluate/metric/retrieval_contents.py
-autorag/evaluate/metric/util.py
-autorag/evaluate/metric/g_eval_prompts/coh_detailed.txt
-autorag/evaluate/metric/g_eval_prompts/con_detailed.txt
-autorag/evaluate/metric/g_eval_prompts/flu_detailed.txt
-autorag/evaluate/metric/g_eval_prompts/rel_detailed.txt
+autorag/evaluation/__init__.py
+autorag/evaluation/generation.py
+autorag/evaluation/retrieval.py
+autorag/evaluation/retrieval_contents.py
+autorag/evaluation/util.py
+autorag/evaluation/metric/__init__.py
+autorag/evaluation/metric/generation.py
+autorag/evaluation/metric/retrieval.py
+autorag/evaluation/metric/retrieval_contents.py
+autorag/evaluation/metric/util.py
+autorag/evaluation/metric/g_eval_prompts/coh_detailed.txt
+autorag/evaluation/metric/g_eval_prompts/con_detailed.txt
+autorag/evaluation/metric/g_eval_prompts/flu_detailed.txt
+autorag/evaluation/metric/g_eval_prompts/rel_detailed.txt
 autorag/nodes/__init__.py
 autorag/nodes/generator/__init__.py
 autorag/nodes/generator/base.py
@@ -129,6 +129,7 @@ autorag/utils/util.py
 docs/Makefile
 docs/make.bat
 docs/requirements.txt
+docs/source/CNAME
 docs/source/conf.py
 docs/source/index.rst
 docs/source/install.md
@@ -169,8 +170,8 @@ docs/source/api_spec/autorag.data.corpus.rst
 docs/source/api_spec/autorag.data.qacreation.rst
 docs/source/api_spec/autorag.data.rst
 docs/source/api_spec/autorag.data.utils.rst
-docs/source/api_spec/autorag.evaluate.metric.rst
-docs/source/api_spec/autorag.evaluate.rst
+docs/source/api_spec/autorag.evaluation.metric.rst
+docs/source/api_spec/autorag.evaluation.rst
 docs/source/api_spec/autorag.nodes.generator.rst
 docs/source/api_spec/autorag.nodes.passageaugmenter.rst
 docs/source/api_spec/autorag.nodes.passagecompressor.rst
@@ -252,6 +253,7 @@ sample_config/simple_local.yaml
 sample_config/simple_openai.yaml
 sample_dataset/README.md
 sample_dataset/eli5/load_eli5_dataset.py
+sample_dataset/hotpotqa/load_hotpotqa_dataset.py
 sample_dataset/msmarco/load_msmarco_dataset.py
 sample_dataset/triviaqa/load_triviaqa_dataset.py
 tests/conftest.py

{autorag-0.2.5 → autorag-0.2.7}/AutoRAG.egg-info/requires.txt

@@ -1,4 +1,4 @@
-numpy
+numpy<2.0.0
 pandas
 tqdm
 tiktoken>=0.7.0
@@ -27,16 +27,17 @@ bert_score
 sentence-transformers
 FlagEmbedding
 ragas
-ray
 kiwipiepy
 llmlingua
 llama-index>=0.10.1
 llama-index-core>=0.10.1
+llama-index-readers-file
 llama-index-embeddings-openai
 llama-index-embeddings-huggingface
 llama-index-llms-openai
 llama-index-llms-huggingface
 llama-index-llms-openai-like
+llama-index-llms-ollama
 llama-index-retrievers-bm25
 streamlit
 langchain-core>=0.1.6