flashrag-dev 0.1.4.dev20250403__tar.gz → 0.1.4.dev20250410__tar.gz

{flashrag_dev-0.1.4.dev20250403 → flashrag_dev-0.1.4.dev20250410}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: flashrag_dev
-Version: 0.1.4.dev20250403
+Version: 0.1.4.dev20250410
 Summary: A library for efficient Retrieval-Augmented Generation research
 Home-page: https://github.com/RUC-NLPIR/FlashRAG
 Author: Jiajie Jin, Yutao Zhu, Chenghao Zhang, Xinyu Yang, Zhicheng Dou
@@ -94,6 +94,7 @@ FlashRAG is still under development and there are many issues and room for impro
 - [ ] Inlcude more RAG approaches
 - [ ] Add more evaluation metrics (e.g., Unieval, name-entity F1) and benchmarks (e.g., RGB benchmark)
 - [ ] Enhance code adaptability and readability
+- [ ] Add support for api-based retriever (vllm server)
 
 ## :page_with_curl: Changelog
 [25/03/21] We have added support for **Reasoning Pipeline**, which is a new paradigm that combines reasoning ability and retrieval, representing work that includes [Search-o1](https://github.com/sunnynexus/Search-o1), [R1-Searcher](https://github.com/SsmallSong/R1-Searcher), [ReSearch](https://github.com/Agent-RL/ReSearch). We evaluate the performance of the pipeline on various RAG benchmarks, it can achieve F1 scores close to 60 on multi hop inference datasets such as Hotpotqe. See it in [**result table**](#robot-supporting-methods).
@@ -108,6 +109,8 @@ FlashRAG is still under development and there are many issues and room for impro
 
 [25/01/07] We have currently support the aggregation of multiple retrievers, see it in [**multi retriever usage**](https://github.com/RUC-NLPIR/FlashRAG/blob/main/docs/original_docs/multi_retriever_usage.md).
 
+[25/01/07] We have integrated a very flexible and lightweight corpus chunking library [**Chunkie**](https://github.com/chonkie-ai/chonkie?tab=readme-ov-file#usage), which supports various custom chunking methods (tokens, sentences, semantic, etc.). Use it in [<u>chunking doc corpus</u>](docs/original_docs/chunk-doc-corpus.md).
+
 [24/10/21] We have released a version based on the Paddle framework that supports Chinese hardware platforms. Please refer to [FlashRAG Paddle](https://github.com/RUC-NLPIR/FlashRAG-Paddle) for details.
 
 [24/10/13] A new in-domain dataset and corpus - [DomainRAG](https://arxiv.org/pdf/2406.05654) have been added to the dataset. The dataset is based on the internal enrollment data of Renmin University of China, covering seven types of tasks, which can be used for conducting domain-specific RAG testing.

{flashrag_dev-0.1.4.dev20250403 → flashrag_dev-0.1.4.dev20250410}/README.md

@@ -77,6 +77,7 @@ FlashRAG is still under development and there are many issues and room for impro
 - [ ] Inlcude more RAG approaches
 - [ ] Add more evaluation metrics (e.g., Unieval, name-entity F1) and benchmarks (e.g., RGB benchmark)
 - [ ] Enhance code adaptability and readability
+- [ ] Add support for api-based retriever (vllm server)
 
 ## :page_with_curl: Changelog
 [25/03/21] We have added support for **Reasoning Pipeline**, which is a new paradigm that combines reasoning ability and retrieval, representing work that includes [Search-o1](https://github.com/sunnynexus/Search-o1), [R1-Searcher](https://github.com/SsmallSong/R1-Searcher), [ReSearch](https://github.com/Agent-RL/ReSearch). We evaluate the performance of the pipeline on various RAG benchmarks, it can achieve F1 scores close to 60 on multi hop inference datasets such as Hotpotqe. See it in [**result table**](#robot-supporting-methods).
@@ -91,6 +92,8 @@ FlashRAG is still under development and there are many issues and room for impro
 
 [25/01/07] We have currently support the aggregation of multiple retrievers, see it in [**multi retriever usage**](https://github.com/RUC-NLPIR/FlashRAG/blob/main/docs/original_docs/multi_retriever_usage.md).
 
+[25/01/07] We have integrated a very flexible and lightweight corpus chunking library [**Chunkie**](https://github.com/chonkie-ai/chonkie?tab=readme-ov-file#usage), which supports various custom chunking methods (tokens, sentences, semantic, etc.). Use it in [<u>chunking doc corpus</u>](docs/original_docs/chunk-doc-corpus.md).
+
 [24/10/21] We have released a version based on the Paddle framework that supports Chinese hardware platforms. Please refer to [FlashRAG Paddle](https://github.com/RUC-NLPIR/FlashRAG-Paddle) for details.
 
 [24/10/13] A new in-domain dataset and corpus - [DomainRAG](https://arxiv.org/pdf/2406.05654) have been added to the dataset. The dataset is based on the internal enrollment data of Renmin University of China, covering seven types of tasks, which can be used for conducting domain-specific RAG testing.

{flashrag_dev-0.1.4.dev20250403 → flashrag_dev-0.1.4.dev20250410}/flashrag/retriever/reranker.py

@@ -44,7 +44,7 @@ class BaseReranker:
             doc_list = [doc_list]
 
         assert len(query_list) == len(doc_list)
-        if topk < min([len(docs) for docs in doc_list]):
+        if topk > min([len(docs) for docs in doc_list]):
             warnings.warn("The number of doc returned by the retriever is less than the topk.")
 
         # get doc contents

flashrag_dev-0.1.4.dev20250410/flashrag/version.py

@@ -0,0 +1 @@
+__version__ = "0.1.4dev20250410"

{flashrag_dev-0.1.4.dev20250403 → flashrag_dev-0.1.4.dev20250410}/flashrag_dev.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: flashrag-dev
-Version: 0.1.4.dev20250403
+Version: 0.1.4.dev20250410
 Summary: A library for efficient Retrieval-Augmented Generation research
 Home-page: https://github.com/RUC-NLPIR/FlashRAG
 Author: Jiajie Jin, Yutao Zhu, Chenghao Zhang, Xinyu Yang, Zhicheng Dou
@@ -94,6 +94,7 @@ FlashRAG is still under development and there are many issues and room for impro
 - [ ] Inlcude more RAG approaches
 - [ ] Add more evaluation metrics (e.g., Unieval, name-entity F1) and benchmarks (e.g., RGB benchmark)
 - [ ] Enhance code adaptability and readability
+- [ ] Add support for api-based retriever (vllm server)
 
 ## :page_with_curl: Changelog
 [25/03/21] We have added support for **Reasoning Pipeline**, which is a new paradigm that combines reasoning ability and retrieval, representing work that includes [Search-o1](https://github.com/sunnynexus/Search-o1), [R1-Searcher](https://github.com/SsmallSong/R1-Searcher), [ReSearch](https://github.com/Agent-RL/ReSearch). We evaluate the performance of the pipeline on various RAG benchmarks, it can achieve F1 scores close to 60 on multi hop inference datasets such as Hotpotqe. See it in [**result table**](#robot-supporting-methods).
@@ -108,6 +109,8 @@ FlashRAG is still under development and there are many issues and room for impro
 
 [25/01/07] We have currently support the aggregation of multiple retrievers, see it in [**multi retriever usage**](https://github.com/RUC-NLPIR/FlashRAG/blob/main/docs/original_docs/multi_retriever_usage.md).
 
+[25/01/07] We have integrated a very flexible and lightweight corpus chunking library [**Chunkie**](https://github.com/chonkie-ai/chonkie?tab=readme-ov-file#usage), which supports various custom chunking methods (tokens, sentences, semantic, etc.). Use it in [<u>chunking doc corpus</u>](docs/original_docs/chunk-doc-corpus.md).
+
 [24/10/21] We have released a version based on the Paddle framework that supports Chinese hardware platforms. Please refer to [FlashRAG Paddle](https://github.com/RUC-NLPIR/FlashRAG-Paddle) for details.
 
 [24/10/13] A new in-domain dataset and corpus - [DomainRAG](https://arxiv.org/pdf/2406.05654) have been added to the dataset. The dataset is based on the internal enrollment data of Renmin University of China, covering seven types of tasks, which can be used for conducting domain-specific RAG testing.

{flashrag_dev-0.1.4.dev20250403 → flashrag_dev-0.1.4.dev20250410}/flashrag_dev.egg-info/requires.txt

@@ -16,7 +16,7 @@ transformers>=4.40.0
 bm25s[core]==0.2.0
 fschat
 streamlit
-chonkie>=0.4.0
+chonkie<1.1.0,>=1.0.2
 gradio>=5.0.0
 rouge-chinese
 jieba
@@ -40,7 +40,7 @@ transformers>=4.40.0
 bm25s[core]==0.2.0
 fschat
 streamlit
-chonkie>=0.4.0
+chonkie<1.1.0,>=1.0.2
 gradio>=5.0.0
 rouge-chinese
 jieba
@@ -64,7 +64,7 @@ transformers>=4.40.0
 bm25s[core]==0.2.0
 fschat
 streamlit
-chonkie>=0.4.0
+chonkie<1.1.0,>=1.0.2
 gradio>=5.0.0
 rouge-chinese
 jieba

flashrag_dev-0.1.4.dev20250403/flashrag/version.py

@@ -1 +0,0 @@
-__version__ = "0.1.4dev20250403"