AutoRAG 0.0.0__py3-none-any.whl

autorag-0.0.0.dist-info/METADATA

@@ -0,0 +1,780 @@
+Metadata-Version: 2.4
+Name: AutoRAG
+Version: 0.0.0
+Summary: Automatically Evaluate RAG pipelines with your own data. Find optimal structure for new RAG product.
+Author-email: Marker-Inc <vkehfdl1@gmail.com>
+License:                                  Apache License
+                                   Version 2.0, January 2004
+                                http://www.apache.org/licenses/
+        
+           TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+        
+           1. Definitions.
+        
+              "License" shall mean the terms and conditions for use, reproduction,
+              and distribution as defined by Sections 1 through 9 of this document.
+        
+              "Licensor" shall mean the copyright owner or entity authorized by
+              the copyright owner that is granting the License.
+        
+              "Legal Entity" shall mean the union of the acting entity and all
+              other entities that control, are controlled by, or are under common
+              control with that entity. For the purposes of this definition,
+              "control" means (i) the power, direct or indirect, to cause the
+              direction or management of such entity, whether by contract or
+              otherwise, or (ii) ownership of fifty percent (50%) or more of the
+              outstanding shares, or (iii) beneficial ownership of such entity.
+        
+              "You" (or "Your") shall mean an individual or Legal Entity
+              exercising permissions granted by this License.
+        
+              "Source" form shall mean the preferred form for making modifications,
+              including but not limited to software source code, documentation
+              source, and configuration files.
+        
+              "Object" form shall mean any form resulting from mechanical
+              transformation or translation of a Source form, including but
+              not limited to compiled object code, generated documentation,
+              and conversions to other media types.
+        
+              "Work" shall mean the work of authorship, whether in Source or
+              Object form, made available under the License, as indicated by a
+              copyright notice that is included in or attached to the work
+              (an example is provided in the Appendix below).
+        
+              "Derivative Works" shall mean any work, whether in Source or Object
+              form, that is based on (or derived from) the Work and for which the
+              editorial revisions, annotations, elaborations, or other modifications
+              represent, as a whole, an original work of authorship. For the purposes
+              of this License, Derivative Works shall not include works that remain
+              separable from, or merely link (or bind by name) to the interfaces of,
+              the Work and Derivative Works thereof.
+        
+              "Contribution" shall mean any work of authorship, including
+              the original version of the Work and any modifications or additions
+              to that Work or Derivative Works thereof, that is intentionally
+              submitted to Licensor for inclusion in the Work by the copyright owner
+              or by an individual or Legal Entity authorized to submit on behalf of
+              the copyright owner. For the purposes of this definition, "submitted"
+              means any form of electronic, verbal, or written communication sent
+              to the Licensor or its representatives, including but not limited to
+              communication on electronic mailing lists, source code control systems,
+              and issue tracking systems that are managed by, or on behalf of, the
+              Licensor for the purpose of discussing and improving the Work, but
+              excluding communication that is conspicuously marked or otherwise
+              designated in writing by the copyright owner as "Not a Contribution."
+        
+              "Contributor" shall mean Licensor and any individual or Legal Entity
+              on behalf of whom a Contribution has been received by Licensor and
+              subsequently incorporated within the Work.
+        
+           2. Grant of Copyright License. Subject to the terms and conditions of
+              this License, each Contributor hereby grants to You a perpetual,
+              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+              copyright license to reproduce, prepare Derivative Works of,
+              publicly display, publicly perform, sublicense, and distribute the
+              Work and such Derivative Works in Source or Object form.
+        
+           3. Grant of Patent License. Subject to the terms and conditions of
+              this License, each Contributor hereby grants to You a perpetual,
+              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+              (except as stated in this section) patent license to make, have made,
+              use, offer to sell, sell, import, and otherwise transfer the Work,
+              where such license applies only to those patent claims licensable
+              by such Contributor that are necessarily infringed by their
+              Contribution(s) alone or by combination of their Contribution(s)
+              with the Work to which such Contribution(s) was submitted. If You
+              institute patent litigation against any entity (including a
+              cross-claim or counterclaim in a lawsuit) alleging that the Work
+              or a Contribution incorporated within the Work constitutes direct
+              or contributory patent infringement, then any patent licenses
+              granted to You under this License for that Work shall terminate
+              as of the date such litigation is filed.
+        
+           4. Redistribution. You may reproduce and distribute copies of the
+              Work or Derivative Works thereof in any medium, with or without
+              modifications, and in Source or Object form, provided that You
+              meet the following conditions:
+        
+              (a) You must give any other recipients of the Work or
+                  Derivative Works a copy of this License; and
+        
+              (b) You must cause any modified files to carry prominent notices
+                  stating that You changed the files; and
+        
+              (c) You must retain, in the Source form of any Derivative Works
+                  that You distribute, all copyright, patent, trademark, and
+                  attribution notices from the Source form of the Work,
+                  excluding those notices that do not pertain to any part of
+                  the Derivative Works; and
+        
+              (d) If the Work includes a "NOTICE" text file as part of its
+                  distribution, then any Derivative Works that You distribute must
+                  include a readable copy of the attribution notices contained
+                  within such NOTICE file, excluding those notices that do not
+                  pertain to any part of the Derivative Works, in at least one
+                  of the following places: within a NOTICE text file distributed
+                  as part of the Derivative Works; within the Source form or
+                  documentation, if provided along with the Derivative Works; or,
+                  within a display generated by the Derivative Works, if and
+                  wherever such third-party notices normally appear. The contents
+                  of the NOTICE file are for informational purposes only and
+                  do not modify the License. You may add Your own attribution
+                  notices within Derivative Works that You distribute, alongside
+                  or as an addendum to the NOTICE text from the Work, provided
+                  that such additional attribution notices cannot be construed
+                  as modifying the License.
+        
+              You may add Your own copyright statement to Your modifications and
+              may provide additional or different license terms and conditions
+              for use, reproduction, or distribution of Your modifications, or
+              for any such Derivative Works as a whole, provided Your use,
+              reproduction, and distribution of the Work otherwise complies with
+              the conditions stated in this License.
+        
+           5. Submission of Contributions. Unless You explicitly state otherwise,
+              any Contribution intentionally submitted for inclusion in the Work
+              by You to the Licensor shall be under the terms and conditions of
+              this License, without any additional terms or conditions.
+              Notwithstanding the above, nothing herein shall supersede or modify
+              the terms of any separate license agreement you may have executed
+              with Licensor regarding such Contributions.
+        
+           6. Trademarks. This License does not grant permission to use the trade
+              names, trademarks, service marks, or product names of the Licensor,
+              except as required for reasonable and customary use in describing the
+              origin of the Work and reproducing the content of the NOTICE file.
+        
+           7. Disclaimer of Warranty. Unless required by applicable law or
+              agreed to in writing, Licensor provides the Work (and each
+              Contributor provides its Contributions) on an "AS IS" BASIS,
+              WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+              implied, including, without limitation, any warranties or conditions
+              of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+              PARTICULAR PURPOSE. You are solely responsible for determining the
+              appropriateness of using or redistributing the Work and assume any
+              risks associated with Your exercise of permissions under this License.
+        
+           8. Limitation of Liability. In no event and under no legal theory,
+              whether in tort (including negligence), contract, or otherwise,
+              unless required by applicable law (such as deliberate and grossly
+              negligent acts) or agreed to in writing, shall any Contributor be
+              liable to You for damages, including any direct, indirect, special,
+              incidental, or consequential damages of any character arising as a
+              result of this License or out of the use or inability to use the
+              Work (including but not limited to damages for loss of goodwill,
+              work stoppage, computer failure or malfunction, or any and all
+              other commercial damages or losses), even if such Contributor
+              has been advised of the possibility of such damages.
+        
+           9. Accepting Warranty or Additional Liability. While redistributing
+              the Work or Derivative Works thereof, You may choose to offer,
+              and charge a fee for, acceptance of support, warranty, indemnity,
+              or other liability obligations and/or rights consistent with this
+              License. However, in accepting such obligations, You may act only
+              on Your own behalf and on Your sole responsibility, not on behalf
+              of any other Contributor, and only if You agree to indemnify,
+              defend, and hold each Contributor harmless for any liability
+              incurred by, or claims asserted against, such Contributor by reason
+              of your accepting any such warranty or additional liability.
+        
+           END OF TERMS AND CONDITIONS
+        
+           APPENDIX: How to apply the Apache License to your work.
+        
+              To apply the Apache License to your work, attach the following
+              boilerplate notice, with the fields enclosed by brackets "[]"
+              replaced with your own identifying information. (Don't include
+              the brackets!)  The text should be enclosed in the appropriate
+              comment syntax for the file format. We also recommend that a
+              file or class name and description of purpose be included on the
+              same "printed page" as the copyright notice for easier
+              identification within third-party archives.
+        
+           Copyright [yyyy] [name of copyright owner]
+        
+           Licensed under the Apache License, Version 2.0 (the "License");
+           you may not use this file except in compliance with the License.
+           You may obtain a copy of the License at
+        
+               http://www.apache.org/licenses/LICENSE-2.0
+        
+           Unless required by applicable law or agreed to in writing, software
+           distributed under the License is distributed on an "AS IS" BASIS,
+           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+           See the License for the specific language governing permissions and
+           limitations under the License.
+        
+Project-URL: Homepage, https://github.com/Marker-Inc-Korea/AutoRAG
+Keywords: RAG,AutoRAG,autorag,rag-evaluation,evaluation,rag-auto,AutoML,AutoML-RAG
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Information Technology
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic>=2.9.2
+Requires-Dist: numpy==1.26.4
+Requires-Dist: pandas>=2.2.3
+Requires-Dist: tqdm>=4.67.1
+Requires-Dist: tiktoken>=0.9.0
+Requires-Dist: openai>=1.90.0
+Requires-Dist: rank_bm25>=0.2.2
+Requires-Dist: pyyaml>=6.0.2
+Requires-Dist: pyarrow>=20.0.0
+Requires-Dist: fastparquet>=2024.11.0
+Requires-Dist: sacrebleu>=2.5.1
+Requires-Dist: evaluate>=0.4.4
+Requires-Dist: rouge_score>=0.1.2
+Requires-Dist: rich>=14.0.0
+Requires-Dist: click>=8.2.1
+Requires-Dist: cohere>=5.18.0
+Requires-Dist: tokenlog>=0.0.3
+Requires-Dist: aiohttp>=3.12.13
+Requires-Dist: voyageai>=0.3.2
+Requires-Dist: mixedbread-ai>=2.2.6
+Requires-Dist: scikit-learn>=1.7.0
+Requires-Dist: emoji>=2.14.1
+Requires-Dist: fastapi>=0.115.13
+Requires-Dist: banks>=2.1.2
+Requires-Dist: grpcio<1.68.0,>=1.66.2
+Requires-Dist: grpcio-health-checking<1.68.0,>=1.66.2
+Requires-Dist: grpcio-status<1.68.0,>=1.66.2
+Requires-Dist: grpcio-tools<1.68.0,>=1.66.2
+Requires-Dist: datasets>=3.5.1
+Requires-Dist: pyarrow>=20.0.0
+Requires-Dist: pymilvus>=2.6.0b0
+Requires-Dist: chromadb>=1.0.0
+Requires-Dist: weaviate-client>=4.15.2
+Requires-Dist: pinecone[grpc]
+Requires-Dist: couchbase>=4.4.0
+Requires-Dist: qdrant-client>=1.12.1
+Requires-Dist: quart>=0.20.0
+Requires-Dist: pyngrok>=7.2.11
+Requires-Dist: llama-index>=0.12.42
+Requires-Dist: llama-index-core>=0.12.42
+Requires-Dist: llama-index-readers-file>=0.4.9
+Requires-Dist: llama-index-embeddings-openai>=0.3.1
+Requires-Dist: llama-index-embeddings-ollama>=0.6.0
+Requires-Dist: llama-index-embeddings-openai-like>=0.1.1
+Requires-Dist: llama-index-llms-openai>=0.4.7
+Requires-Dist: llama-index-llms-openai-like>=0.4.0
+Requires-Dist: llama-index-retrievers-bm25>=0.5.2
+Requires-Dist: llama-index-llms-bedrock>=0.3.8
+Requires-Dist: streamlit>=1.46.0
+Requires-Dist: gradio>=5.34.2
+Requires-Dist: langchain-core==0.2.43
+Requires-Dist: langchain-unstructured==0.1.2
+Requires-Dist: langchain-upstage==0.1.5
+Requires-Dist: langchain-community==0.2.19
+Requires-Dist: langsmith==0.1.147
+Requires-Dist: panel>=1.7.1
+Requires-Dist: seaborn>=0.13.2
+Requires-Dist: ipykernel>=6.29.5
+Requires-Dist: ipywidgets>=8.1.7
+Requires-Dist: ipywidgets_bokeh>=1.6.0
+Provides-Extra: ko
+Requires-Dist: kiwipiepy>=0.18.0; extra == "ko"
+Requires-Dist: konlpy>=0.6.0; extra == "ko"
+Provides-Extra: parse
+Requires-Dist: PyMuPDF>=1.26.1; extra == "parse"
+Requires-Dist: pdfminer.six>=20250506; extra == "parse"
+Requires-Dist: pdfplumber>=0.11.7; extra == "parse"
+Requires-Dist: jq>=1.9.1; extra == "parse"
+Requires-Dist: unstructured[pdf]>=0.17.2; extra == "parse"
+Requires-Dist: PyPDF2==2.12.1; extra == "parse"
+Requires-Dist: pdf2image>=1.17.0; extra == "parse"
+Provides-Extra: ja
+Requires-Dist: sudachipy>=0.6.8; extra == "ja"
+Requires-Dist: sudachidict_core; extra == "ja"
+Provides-Extra: gpu
+Requires-Dist: torch>=2.7.1; extra == "gpu"
+Requires-Dist: sentencepiece>=0.2.0; extra == "gpu"
+Requires-Dist: bert_score>=0.3.13; extra == "gpu"
+Requires-Dist: peft>=0.15.2; extra == "gpu"
+Requires-Dist: llmlingua>=0.2.2; extra == "gpu"
+Requires-Dist: FlagEmbedding>=1.2.11; extra == "gpu"
+Requires-Dist: sentence-transformers>=4.1.0; extra == "gpu"
+Requires-Dist: transformers>=4.51.3; extra == "gpu"
+Requires-Dist: llama-index-llms-ollama>=0.6.0; extra == "gpu"
+Requires-Dist: llama-index-embeddings-huggingface>=0.5.4; extra == "gpu"
+Requires-Dist: llama-index-llms-huggingface>=0.5.0; extra == "gpu"
+Requires-Dist: onnxruntime>=1.22.0; extra == "gpu"
+Requires-Dist: vllm>=0.11.0; extra == "gpu"
+Provides-Extra: all
+Requires-Dist: AutoRAG[gpu]; extra == "all"
+Requires-Dist: AutoRAG[ko]; extra == "all"
+Requires-Dist: AutoRAG[parse]; extra == "all"
+Requires-Dist: AutoRAG[ja]; extra == "all"
+Dynamic: license-file
+
+# AutoRAG
+
+RAG AutoML tool for automatically finding an optimal RAG pipeline for your data.
+
+![Thumbnail](https://github.com/user-attachments/assets/6bab243d-a4b3-431a-8ac0-fe17336ab4de)
+
+![PyPI - Downloads](https://img.shields.io/pypi/dm/AutoRAG)
+[![LinkedIn](https://img.shields.io/badge/LinkedIn-Connect-blue?style=flat-square&logo=linkedin)](https://www.linkedin.com/company/104375108/admin/dashboard/)
+![X (formerly Twitter) Follow](https://img.shields.io/twitter/follow/AutoRAG_HQ)
+[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-Follow-orange?style=flat-square&logo=huggingface)](https://huggingface.co/AutoRAG)
+
+<a href="https://trendshift.io/repositories/7832" target="_blank"><img src="https://trendshift.io/api/badge/repositories/7832" alt="Marker-Inc-Korea%2FAutoRAG | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
+
+There are many RAG pipelines and modules out there,
+but you don’t know what pipeline is great for “your own data” and "your own use-case."
+Making and evaluating all RAG modules is very time-consuming and hard to do.
+But without it, you will never know which RAG pipeline is the best for your own use-case.
+
+AutoRAG is a tool for finding the optimal RAG pipeline for “your data.”
+You can evaluate various RAG modules automatically with your own evaluation data
+and find the best RAG pipeline for your own use-case.
+
+AutoRAG supports a simple way to evaluate many RAG module combinations.
+Try now and find the best RAG pipeline for your own use-case.
+
+Explore our 📖 [Document](https://marker-inc-korea.github.io/AutoRAG/)!!
+
+---
+
+## YouTube Tutorial
+
+https://github.com/Marker-Inc-Korea/AutoRAG/assets/96727832/c0d23896-40c0-479f-a17b-aa2ec3183a26
+
+_Muted by default, enable sound for voice-over_
+
+You can see on [YouTube](https://youtu.be/2ojK8xjyXAU?feature=shared)
+
+## Use AutoRAG in HuggingFace Space 🚀
+
+- [💬 Naive RAG Chatbot](https://huggingface.co/spaces/AutoRAG/Naive-RAG-chatbot)
+- [✏️ AutoRAG Data Creation](https://huggingface.co/spaces/AutoRAG/AutoRAG-data-creation)
+- [🚀 AutoRAG RAG Pipeline Optimization](https://huggingface.co/spaces/AutoRAG/RAG-Pipeline-Optimization)
+
+## Colab Tutorial
+
+- [Step 1: Basic of AutoRAG | Optimizing your RAG pipeline](https://colab.research.google.com/drive/19OEQXO_pHN6gnn2WdfPd4hjnS-4GurVd?usp=sharing)
+- [Step 2: Data Creation | Create your own Data for RAG Optimization](https://colab.research.google.com/drive/1BOdzMndYgMY_iqhwKcCCS7ezHbZ4Oz5X?usp=sharing)
+- [Step 3: Use Custom LLM & Embedding Model | Use Custom Model](https://colab.research.google.com/drive/12VpWcSTSOsLSyW0BKb-kPoEzK22ACxvS?usp=sharing)
+
+# Index
+
+- [Quick Install](#quick-install)
+- [Data Creation](#data-creation)
+    - [Parsing](#1-parsing)
+    - [Chunking](#2-chunking)
+    - [QA Creation](#3-qa-creation)
+- [RAG Optimization](#rag-optimization)
+    - [How AutoRAG optimizes RAG pipeline?](#how-autorag-optimizes-rag-pipeline)
+    - [Metrics](#metrics)
+    - [Quick Start](#quick-start-1)
+        - [Set YAML File](#1-set-yaml-file)
+        - [Run AutoRAG](#2-run-autorag)
+        - [Run Dashboard](#3-run-dashboard)
+        - [Deploy your optimal RAG pipeline](#4-deploy-your-optimal-rag-pipeline)
+- [FaQ](#-faq)
+
+# Quick Install
+
+We recommend using Python version 3.10 or higher for AutoRAG.
+
+```bash
+pip install AutoRAG
+```
+
+If you want to use the local models, you need to install gpu version.
+
+```bash
+pip install "AutoRAG[gpu]"
+```
+
+Or for parsing, you can use the parsing version.
+
+```bash
+pip install "AutoRAG[gpu,parse]"
+```
+
+# Data Creation
+
+<a href="https://huggingface.co/spaces/AutoRAG/AutoRAG-data-creation">
+<img src="https://github.com/user-attachments/assets/8c6e4b02-3938-4560-b817-c95764965b50" alt="Hugging Face Sticker" style="width:200px;height:auto;">
+</a>
+
+![Image](https://github.com/user-attachments/assets/146d005d-dcb9-4460-a8b3-25126e5e3dc2)
+
+![image](https://github.com/user-attachments/assets/6079f696-207c-4221-8d28-5561a203dfe2)
+
+RAG Optimization requires two types of data: QA dataset and Corpus dataset.
+
+1. **QA** dataset file (qa.parquet)
+2. **Corpus** dataset file (corpus.parquet)
+
+**QA** dataset is important for accurate and reliable evaluation and optimization.
+
+**Corpus** dataset is critical to the performance of RAGs.
+This is because RAG uses the corpus to retrieve documents and generate answers using it.
+
+### 📌 Supporting Data Creation Modules
+
+![Image](https://github.com/user-attachments/assets/c6f15fab-6c69-4627-9685-6c218b66f5d6)
+
+- [Supporting Parsing Modules List](https://edai.notion.site/Supporting-Parsing-Modules-e0b7579c7c0e4fb2963e408eeccddd75?pvs=4)
+- [Supporting Chunking Modules List](https://edai.notion.site/Supporting-Chunk-Modules-8db803dba2ec4cd0a8789659106e86a3?pvs=4)
+
+## Quick Start
+
+### 1. Parsing
+
+#### Set YAML File
+
+```yaml
+modules:
+  - module_type: langchain_parse
+    parse_method: pdfminer
+```
+
+You can also use multiple Parse modules at once.
+However, in this case, you'll need to return a new process for each parsed result.
+
+#### Start Parsing
+
+You can parse your raw documents with just a few lines of code.
+
+```python
+from autorag.parser import Parser
+
+parser = Parser(data_path_glob="your/data/path/*")
+parser.start_parsing("your/path/to/parse_config.yaml")
+```
+
+### 2. Chunking
+
+#### Set YAML File
+
+```yaml
+modules:
+  - module_type: llama_index_chunk
+    chunk_method: Token
+    chunk_size: 1024
+    chunk_overlap: 24
+    add_file_name: en
+```
+
+You can also use multiple Chunk modules at once.
+In this case, you need to use one corpus to create QA and then map the rest of the corpus to QA Data.
+If the chunk method is different, the retrieval_gt will be different, so we need to remap it to the QA dataset.
+
+#### Start Chunking
+
+You can chunk your parsed results with just a few lines of code.
+
+```python
+from autorag.chunker import Chunker
+
+chunker = Chunker.from_parquet(parsed_data_path="your/parsed/data/path")
+chunker.start_chunking("your/path/to/chunk_config.yaml")
+```
+
+### 3. QA Creation
+
+You can create QA dataset with just a few lines of code.
+
+```python
+import pandas as pd
+from llama_index.llms.openai import OpenAI
+
+from autorag.data.qa.filter.dontknow import dontknow_filter_rule_based
+from autorag.data.qa.generation_gt.llama_index_gen_gt import (
+	make_basic_gen_gt,
+	make_concise_gen_gt,
+)
+from autorag.data.qa.schema import Raw, Corpus
+from autorag.data.qa.query.llama_gen_query import factoid_query_gen
+from autorag.data.qa.sample import random_single_hop
+
+llm = OpenAI()
+raw_df = pd.read_parquet("your/path/to/parsed.parquet")
+raw_instance = Raw(raw_df)
+
+corpus_df = pd.read_parquet("your/path/to/corpus.parquet")
+corpus_instance = Corpus(corpus_df, raw_instance)
+
+initial_qa = (
+	corpus_instance.sample(random_single_hop, n=3)
+	.map(
+		lambda df: df.reset_index(drop=True),
+	)
+	.make_retrieval_gt_contents()
+	.batch_apply(
+		factoid_query_gen,  # query generation
+		llm=llm,
+	)
+	.batch_apply(
+		make_basic_gen_gt,  # answer generation (basic)
+		llm=llm,
+	)
+	.batch_apply(
+		make_concise_gen_gt,  # answer generation (concise)
+		llm=llm,
+	)
+	.filter(
+		dontknow_filter_rule_based,  # filter don't know
+		lang="en",
+	)
+)
+
+initial_qa.to_parquet('./qa.parquet', './corpus.parquet')
+```
+
+# RAG Optimization
+
+<a href="https://huggingface.co/spaces/AutoRAG/RAG-Pipeline-Optimization">
+<img src="https://github.com/user-attachments/assets/8c6e4b02-3938-4560-b817-c95764965b50" alt="Hugging Face Sticker" style="width:200px;height:auto;">
+</a>
+
+![Image](https://github.com/user-attachments/assets/b814928d-54a4-4b96-af34-adba0ac6803b)
+
+![rag](https://github.com/user-attachments/assets/214d842e-fc67-4113-9c24-c94158b00c23)
+
+## How AutoRAG optimizes RAG pipeline?
+
+Here is the AutoRAG RAG Structure that only show Nodes.
+
+![Image](https://github.com/user-attachments/assets/cbc60938-e211-4fbf-be74-31bd9a997581)
+
+Here is the image showing all the nodes and modules.
+
+![Image](https://github.com/user-attachments/assets/9489e803-f47a-49d4-97ec-0dd9b270394f)
+
+![rag_opt_gif](https://github.com/user-attachments/assets/55bd09cd-8420-4f6d-bc7d-0a66af288317)
+
+### 📌 Supporting RAG Optimization Nodes & modules
+
+- [Supporting RAG Modules list](https://edai.notion.site/Supporting-Nodes-modules-0ebc7810649f4e41aead472a92976be4?pvs=4)
+
+## Metrics
+
+The metrics used by each node in AutoRAG are shown below.
+
+![Image](https://github.com/user-attachments/assets/5b342f68-d25c-4cba-aa85-1e257801afea)
+
+![Image](https://github.com/user-attachments/assets/393d3ad6-1bde-4e75-b314-5c150eadaeee)
+
+- [Supporting metrics list](https://edai.notion.site/Supporting-metrics-867d71caefd7401c9264dd91ba406043?pvs=4)
+
+Here is the detailed information about the metrics that AutoRAG supports.
+
+- [Retrieval Metrics](https://edai.notion.site/Retrieval-Metrics-dde3d9fa1d9547cdb8b31b94060d21e7?pvs=4)
+- [Retrieval Token Metrics](https://edai.notion.site/Retrieval-Token-Metrics-c3e2d83358e04510a34b80429ebb543f?pvs=4)
+- [Generation Metrics](https://github.com/user-attachments/assets/7d4a3069-9186-4854-885d-ca0f7bcc17e8)
+
+## Quick Start
+
+### 1. Set YAML File
+
+First, you need to set the config YAML file for your RAG optimization.
+
+We highly recommend using pre-made config YAML files for starter.
+
+- [Get Sample YAML](sample_config/rag)
+    - [Sample YAML Guide](https://marker-inc-korea.github.io/AutoRAG/optimization/sample_config.html)
+- [Make Custom YAML Guide](https://marker-inc-korea.github.io/AutoRAG/optimization/custom_config.html)
+
+Here is an example of the config YAML file to use three retrieval nodes, `prompt_maker`, and `generator` nodes.
+
+```yaml
+node_lines:
+  - node_line_name: retrieve_node_line
+    nodes:
+      - node_type: lexical_retrieval
+        strategy:
+          metrics: [ retrieval_f1, retrieval_recall, retrieval_ndcg, retrieval_mrr ]
+        top_k: 3
+        modules:
+          - module_type: bm25
+      - node_type: semantic_retrieval
+        strategy:
+          metrics: [ retrieval_f1, retrieval_recall, retrieval_ndcg, retrieval_mrr ]
+        top_k: 3
+        modules:
+          - module_type: vectordb
+            vectordb: default
+      - node_type: hybrid_retrieval
+        strategy:
+          metrics: [ retrieval_f1, retrieval_recall, retrieval_ndcg, retrieval_mrr ]
+        top_k: 3
+        modules:
+          - module_type: hybrid_rrf
+            weight_range: (4,80)
+  - node_line_name: post_retrieve_node_line
+    nodes:
+      - node_type: prompt_maker  # Set Prompt Maker Node
+        strategy:
+          metrics: # Set Generation Metrics
+            - metric_name: meteor
+            - metric_name: rouge
+            - metric_name: sem_score
+              embedding_model: openai
+        modules:
+          - module_type: fstring
+            prompt: "Read the passages and answer the given question. \n Question: {query} \n Passage: {retrieved_contents} \n Answer : "
+      - node_type: generator  # Set Generator Node
+        strategy:
+          metrics: # Set Generation Metrics
+            - metric_name: meteor
+            - metric_name: rouge
+            - metric_name: sem_score
+              embedding_model: openai
+        modules:
+          - module_type: openai_llm
+            llm: gpt-4o-mini
+            batch: 16
+```
+
+### 2. Run AutoRAG
+
+You can evaluate your RAG pipeline with just a few lines of code.
+
+```python
+from autorag.evaluator import Evaluator
+
+evaluator = Evaluator(qa_data_path='your/path/to/qa.parquet', corpus_data_path='your/path/to/corpus.parquet')
+evaluator.start_trial('your/path/to/config.yaml')
+```
+
+or you can use the command line interface
+
+```bash
+autorag evaluate --config your/path/to/default_config.yaml --qa_data_path your/path/to/qa.parquet --corpus_data_path your/path/to/corpus.parquet
+```
+
+Once it is done, you can see several files and folders created in your current directory.
+At the trial folder named to numbers (like 0),
+you can check `summary.csv` file that summarizes the evaluation results and the best RAG pipeline for your data.
+
+For more details, you can check out how the folder structure looks like
+at [here](https://marker-inc-korea.github.io/AutoRAG/optimization/folder_structure.html).
+
+### 3. Run Dashboard
+
+You can run a dashboard to easily see the result.
+
+```bash
+autorag dashboard --trial_dir /your/path/to/trial_dir
+```
+
+#### sample dashboard
+
+![dashboard](https://github.com/Marker-Inc-Korea/AutoRAG/assets/96727832/3798827d-31d7-4c4e-a9b1-54340b964e53)
+
+### 4. Deploy your optimal RAG pipeline
+
+### 4-1. Run as a Code
+
+You can use an optimal RAG pipeline right away from the trial folder.
+The trial folder is the directory used in the running dashboard. (like 0, 1, 2, ...)
+
+```python
+from autorag.deploy import Runner
+
+runner = Runner.from_trial_folder('/your/path/to/trial_dir')
+runner.run('your question')
+```
+
+### 4-2. Run as an API server
+
+You can run this pipeline as an API server.
+
+Check out the API endpoint at [here](./docs/source/deploy/api_endpoint.md).
+
+```python
+import nest_asyncio
+from autorag.deploy import ApiRunner
+
+nest_asyncio.apply()
+
+runner = ApiRunner.from_trial_folder('/your/path/to/trial_dir')
+runner.run_api_server()
+```
+
+```bash
+autorag run_api --trial_dir your/path/to/trial_dir --host 0.0.0.0 --port 8000
+```
+
+The cli command uses extracted config YAML file. If you want to know it more, check
+out [here](https://marker-inc-korea.github.io/AutoRAG/tutorial.html#extract-pipeline-and-evaluate-test-dataset).
+
+### 4-3. Run as a Web Interface
+
+you can run this pipeline as a web interface.
+
+Check out the web interface at [here](deploy/web.md).
+
+```bash
+autorag run_web --trial_path your/path/to/trial_path
+```
+
+#### sample web interface
+
+<img width="1491" alt="web_interface" src="https://github.com/Marker-Inc-Korea/AutoRAG/assets/96727832/f6b00353-f6bb-4d8f-8740-1c264c0acbb8">
+
+## ☎️ FaQ
+
+💻 [Hardware Specs](https://edai.notion.site/Hardware-specs-28cefcf2a26246ffadc91e2f3dc3d61c?pvs=4)
+
+⭐ [Running AutoRAG](https://edai.notion.site/About-running-AutoRAG-44a8058307af42068fc218a073ee480b?pvs=4)
+
+🍯 [Tips/Tricks](https://edai.notion.site/Tips-Tricks-10708a0e36ff461cb8a5d4fb3279ff15?pvs=4)
+
+☎️ [TroubleShooting](https://medium.com/@autorag/autorag-troubleshooting-5cf872b100e3)
+
+## Thanks for shoutout
+
+### Company
+
+<a href="https://www.linkedin.com/posts/llamaindex_rag-pipelines-have-a-lot-of-hyperparameters-activity-7182053546593247232-HFMN/">
+<img src="https://github.com/user-attachments/assets/b8fdaaf6-543a-4019-8dbe-44191a5269b9" alt="llama index" style="width:200px;height:auto;">
+</a>
+
+### Individual
+
+- [Shubham Saboo](https://www.linkedin.com/posts/shubhamsaboo_just-found-the-solution-to-the-biggest-rag-activity-7255404464054939648-ISQ8/)
+- [Kalyan KS](https://www.linkedin.com/posts/kalyanksnlp_rag-autorag-llms-activity-7258677155574788097-NgS0/)
+
+---
+
+# ✨ Contributors ✨
+
+Thanks go to these wonderful people:
+
+<a href="https://github.com/Marker-Inc-Korea/AutoRAG/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=Marker-Inc-Korea/AutoRAG" />
+</a>
+
+# Contribution
+
+We are developing AutoRAG as open-source.
+
+So this project welcomes contributions and suggestions. Feel free to contribute to this project.
+
+Plus, check out our detailed documentation at [here](https://marker-inc-korea.github.io/AutoRAG/index.html).
+
+## Citation
+
+```bibtex
+@misc{kim2024autoragautomatedframeworkoptimization,
+      title={AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline},
+      author={Dongkyu Kim and Byoungwook Kim and Donggeon Han and Matouš Eibich},
+      year={2024},
+      eprint={2410.20878},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2410.20878},
+}
+```