flashrag-dev 0.1.3.dev20241111__tar.gz → 0.1.3.dev20241115__tar.gz

{flashrag-dev-0.1.3.dev20241111 → flashrag-dev-0.1.3.dev20241115}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: flashrag-dev
-Version: 0.1.3.dev20241111
+Version: 0.1.3.dev20241115
 Summary: A library for efficient Retrieval-Augmented Generation research
 Home-page: https://github.com/RUC-NLPIR/FlashRAG
 Author: Jiajie Jin, Yutao Zhu, Chenghao Zhang, Xinyu Yang, Zhicheng Dou
@@ -536,6 +536,11 @@ In the academic research, Wikipedia and MS MARCO are the most commonly used retr
 
 For MS MARCO, it is already processed upon release and can be directly downloaded from its [<u>hosting link</u>](https://huggingface.co/datasets/Tevatron/msmarco-passage-corpus) on Hugging Face.
 
+### Index
+
+To facilitate easier replication of the experiments, we now provide a preprocessed index available in the ModelScope dataset page: [FlashRAG_Dataset/retrieval_corpus/wiki18_100w_e5_index.zip](https://www.modelscope.cn/datasets/hhjinjiajie/FlashRAG_Dataset/file/view/master?id=47985&status=2&fileName=retrieval_corpus%252Fwiki18_100w_e5_index.zip). 
+
+The index was created using the e5-base-v2 retriever on our uploaded wiki18_100w dataset, which is consistent with the index used in our experiments.
 
 ## :raised_hands: Additional FAQs
 

flashrag-dev-0.1.3.dev20241111/flashrag_dev.egg-info/PKG-INFO → flashrag-dev-0.1.3.dev20241115/README.md

@@ -1,19 +1,3 @@
-Metadata-Version: 2.1
-Name: flashrag-dev
-Version: 0.1.3.dev20241111
-Summary: A library for efficient Retrieval-Augmented Generation research
-Home-page: https://github.com/RUC-NLPIR/FlashRAG
-Author: Jiajie Jin, Yutao Zhu, Chenghao Zhang, Xinyu Yang, Zhicheng Dou
-Author-email: jinjiajie@ruc.edu.cn
-License: MIT License
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-Provides-Extra: core
-Provides-Extra: retriever
-Provides-Extra: generator
-Provides-Extra: full
-License-File: LICENSE
-
 # <div align="center">⚡FlashRAG: A Python Toolkit for Efficient RAG Research<div>
 
 
@@ -536,6 +520,11 @@ In the academic research, Wikipedia and MS MARCO are the most commonly used retr
 
 For MS MARCO, it is already processed upon release and can be directly downloaded from its [<u>hosting link</u>](https://huggingface.co/datasets/Tevatron/msmarco-passage-corpus) on Hugging Face.
 
+### Index
+
+To facilitate easier replication of the experiments, we now provide a preprocessed index available in the ModelScope dataset page: [FlashRAG_Dataset/retrieval_corpus/wiki18_100w_e5_index.zip](https://www.modelscope.cn/datasets/hhjinjiajie/FlashRAG_Dataset/file/view/master?id=47985&status=2&fileName=retrieval_corpus%252Fwiki18_100w_e5_index.zip). 
+
+The index was created using the e5-base-v2 retriever on our uploaded wiki18_100w dataset, which is consistent with the index used in our experiments.
 
 ## :raised_hands: Additional FAQs
 

{flashrag-dev-0.1.3.dev20241111 → flashrag-dev-0.1.3.dev20241115}/flashrag/config/basic_config.yaml

@@ -2,7 +2,7 @@
 # Paths to various models
 model2path:
   e5: "intfloat/e5-base-v2"
-  bge: "intfloat/e5-base-v2"
+  bge: "BAAI/bge-base-en-v1.5"
   contriever: "facebook/contriever"
   llama2-7B-chat: "meta-llama/Llama-2-7b-chat-hf"
   llama2-7B: "meta-llama/Llama-2-7b-hf"

{flashrag-dev-0.1.3.dev20241111 → flashrag-dev-0.1.3.dev20241115}/flashrag/config/config.py

@@ -131,7 +131,8 @@ class Config:
                 print("Index is empty!!")
                 assert False
 
-        self.final_config["retrieval_model_path"] = model2path.get(retrieval_method, retrieval_method)
+        if self.final_config.get("retrieval_model_path") is None:
+            self.final_config["retrieval_model_path"] = model2path.get(retrieval_method, retrieval_method)
         # TODO: not support when `retrieval_model` is path
 
         def set_pooling_method(method, model2pooling):

{flashrag-dev-0.1.3.dev20241111 → flashrag-dev-0.1.3.dev20241115}/flashrag/dataset/dataset.py

@@ -100,8 +100,11 @@ class Dataset:
             self.data = self._load_data(self.dataset_name, self.dataset_path)
         else:
             print("Load data from provided data")
-            assert isinstance(data[0], dict)
-            self.data = [Item(item_dict) for item_dict in data]
+            if isinstance(data[0], dict):
+                self.data = [Item(item_dict) for item_dict in data]
+            else:
+                assert isinstance(data[0], Item)
+                self.data = data
 
     def _load_data(self, dataset_name: str, dataset_path: str) -> List[Item]:
         """Load data from the provided dataset_path or directly download the file(TODO)."""

flashrag-dev-0.1.3.dev20241115/flashrag/version.py

@@ -0,0 +1 @@
+__version__ = "0.1.3dev20241115"

flashrag-dev-0.1.3.dev20241111/README.md → flashrag-dev-0.1.3.dev20241115/flashrag_dev.egg-info/PKG-INFO

@@ -1,3 +1,19 @@
+Metadata-Version: 2.1
+Name: flashrag-dev
+Version: 0.1.3.dev20241115
+Summary: A library for efficient Retrieval-Augmented Generation research
+Home-page: https://github.com/RUC-NLPIR/FlashRAG
+Author: Jiajie Jin, Yutao Zhu, Chenghao Zhang, Xinyu Yang, Zhicheng Dou
+Author-email: jinjiajie@ruc.edu.cn
+License: MIT License
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Provides-Extra: core
+Provides-Extra: retriever
+Provides-Extra: generator
+Provides-Extra: full
+License-File: LICENSE
+
 # <div align="center">⚡FlashRAG: A Python Toolkit for Efficient RAG Research<div>
 
 
@@ -520,6 +536,11 @@ In the academic research, Wikipedia and MS MARCO are the most commonly used retr
 
 For MS MARCO, it is already processed upon release and can be directly downloaded from its [<u>hosting link</u>](https://huggingface.co/datasets/Tevatron/msmarco-passage-corpus) on Hugging Face.
 
+### Index
+
+To facilitate easier replication of the experiments, we now provide a preprocessed index available in the ModelScope dataset page: [FlashRAG_Dataset/retrieval_corpus/wiki18_100w_e5_index.zip](https://www.modelscope.cn/datasets/hhjinjiajie/FlashRAG_Dataset/file/view/master?id=47985&status=2&fileName=retrieval_corpus%252Fwiki18_100w_e5_index.zip). 
+
+The index was created using the e5-base-v2 retriever on our uploaded wiki18_100w dataset, which is consistent with the index used in our experiments.
 
 ## :raised_hands: Additional FAQs
 

flashrag-dev-0.1.3.dev20241111/flashrag/version.py

@@ -1 +0,0 @@
-__version__ = "0.1.3dev20241111"