flashrag-dev 0.1.4.dev20250319__tar.gz → 0.1.4.dev20250321__tar.gz

{flashrag_dev-0.1.4.dev20250319 → flashrag_dev-0.1.4.dev20250321}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: flashrag_dev
-Version: 0.1.4.dev20250319
+Version: 0.1.4.dev20250321
 Summary: A library for efficient Retrieval-Augmented Generation research
 Home-page: https://github.com/RUC-NLPIR/FlashRAG
 Author: Jiajie Jin, Yutao Zhu, Chenghao Zhang, Xinyu Yang, Zhicheng Dou
@@ -41,7 +41,7 @@ License-File: LICENSE
 </h4>
 
 
-FlashRAG is a Python toolkit for the reproduction and development of Retrieval Augmented Generation (RAG) research. Our toolkit includes 36 pre-processed benchmark RAG datasets and 16 state-of-the-art RAG algorithms.
+FlashRAG is a Python toolkit for the reproduction and development of Retrieval Augmented Generation (RAG) research. Our toolkit includes 36 pre-processed benchmark RAG datasets and 17 state-of-the-art RAG algorithms.
 
 <p align="center">
 <img src="asset/framework.jpg">
@@ -75,7 +75,7 @@ https://github.com/user-attachments/assets/8ca00873-5df2-48a7-b853-89e7b18bc6e9
 
 - **Comprehensive Benchmark Datasets**: A collection of 36 pre-processed RAG benchmark datasets to test and validate RAG models' performances.
 
-- **Pre-implemented Advanced RAG Algorithms**: Features 16 advancing RAG algorithms with reported results, based on our framework. Easily reproducing results under different settings.
+- **Pre-implemented Advanced RAG Algorithms**: Features 17 advancing RAG algorithms with reported results, based on our framework. Easily reproducing results under different settings.
 
 - **Efficient Preprocessing Stage**: Simplifies the RAG workflow preparation by providing various scripts like corpus processing for retrieval, retrieval index building, and pre-retrieval of documents.
 
@@ -96,6 +96,8 @@ FlashRAG is still under development and there are many issues and room for impro
 - [ ] Enhance code adaptability and readability
 
 ## :page_with_curl: Changelog
+[25/03/21] We have added support for **Reasoning Pipeline**, which is a new paradigm that combines reasoning ability and retrieval, representing work that includes [Search-o1](https://github.com/sunnynexus/Search-o1), [R1-Searcher](https://github.com/SsmallSong/R1-Searcher), [ReSearch](https://github.com/Agent-RL/ReSearch). We evaluate the performance of the pipeline on various RAG benchmarks, it can achieve F1 scores close to 60 on multi hop inference datasets such as Hotpotqe. See it in [**result table**](#robot-supporting-methods).
+
 [25/02/24] 🔥🔥🔥 We have added support for **multimodal RAG**, including [**MLLMs like Llava, Qwen, InternVL**](https://ruc-nlpir.github.io/FlashRAG/#/zh-cn/component/generator?id=%e5%a4%9a%e6%a8%a1%e6%80%81%e7%94%9f%e6%88%90%e5%99%a8), and various [**multimodal retrievers with Clip architecture**](https://ruc-nlpir.github.io/FlashRAG/#/zh-cn/component/retriever?id=%e5%a4%9a%e6%a8%a1%e6%80%81%e6%a3%80%e7%b4%a2%e5%99%a8). More information can be found in our new version of arxiv article and our documentation. Try it!
 
 [25/01/21] Our technical paper [FlashRAG: A Python Toolkit for Efficient RAG Research](https://arxiv.org/abs/2405.13576) is honored to have been accepted to the Resource Track of the 2025 **ACM Web Conference (WWW 2025)**. Please Check it out!
@@ -482,7 +484,7 @@ In each category, we have implemented corresponding common pipelines. Some pipel
           <td>Ranking and merging generated results based on each document</td>
         </tr>
         <tr>
-            <td rowspan="5">Loop</td>
+            <td rowspan="6">Loop</td>
             <td>Iterative Pipeline</td>
             <td>Alternating retrieval and generation</td>
         </tr>
@@ -502,6 +504,10 @@ In each category, we have implemented corresponding common pipelines. Some pipel
             <td>IRCoT Pipeline</td>
             <td>Integrate retrieval process with CoT</td>
         </tr>
+        <tr>
+            <td>Reasoning Pipeline</td>
+            <td>Reasoning with retrieval</td>
+        </tr>
     </tbody>
 </table>
 
@@ -582,6 +588,8 @@ It’s important to note that, to ensure consistency, we have utilized a uniform
 | [Iter-Retgen](https://arxiv.org/abs/2305.15294), [ITRG](https://arxiv.org/abs/2310.05149) | Loop        | 36.8    | 60.1          | 38.3          | 21.6       | 37.9       | 18.2      |                                                 |
 | [IRCoT](https://aclanthology.org/2023.acl-long.557.pdf)                                   | Loop        | 33.3    | 56.9          | 41.5          | 32.4       | 45.6       | 20.7      |                                                 |
 | [RQRAG](https://arxiv.org/abs/2404.00610)                                   | Loop        | 32.6    | 52.5          | 33.5          | 35.8       | 46.4       | 26.2      |  Use trained rqrag-llama2-7B                                               |
+| [R1-Searcher](https://arxiv.org/pdf/2503.05592) | Reasoning | 37.3 | 58.7 | 59.5 | 55.5 | 43.9 | 27.5 | Using trained Qwen2.5-7B model |
+ 
 
 
 ## :notebook: Supporting Datasets & Document Corpus

flashrag_dev-0.1.4.dev20250319/flashrag_dev.egg-info/PKG-INFO → flashrag_dev-0.1.4.dev20250321/README.md

@@ -1,20 +1,3 @@
-Metadata-Version: 2.1
-Name: flashrag-dev
-Version: 0.1.4.dev20250319
-Summary: A library for efficient Retrieval-Augmented Generation research
-Home-page: https://github.com/RUC-NLPIR/FlashRAG
-Author: Jiajie Jin, Yutao Zhu, Chenghao Zhang, Xinyu Yang, Zhicheng Dou
-Author-email: jinjiajie@ruc.edu.cn
-License: MIT License
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-Provides-Extra: core
-Provides-Extra: retriever
-Provides-Extra: generator
-Provides-Extra: multimodal
-Provides-Extra: full
-License-File: LICENSE
-
 # <div align="center">⚡FlashRAG: A Python Toolkit for Efficient RAG Research<div>
 \[ English | [中文](README_zh.md) \]
 <div align="center">
@@ -41,7 +24,7 @@ License-File: LICENSE
 </h4>
 
 
-FlashRAG is a Python toolkit for the reproduction and development of Retrieval Augmented Generation (RAG) research. Our toolkit includes 36 pre-processed benchmark RAG datasets and 16 state-of-the-art RAG algorithms.
+FlashRAG is a Python toolkit for the reproduction and development of Retrieval Augmented Generation (RAG) research. Our toolkit includes 36 pre-processed benchmark RAG datasets and 17 state-of-the-art RAG algorithms.
 
 <p align="center">
 <img src="asset/framework.jpg">
@@ -75,7 +58,7 @@ https://github.com/user-attachments/assets/8ca00873-5df2-48a7-b853-89e7b18bc6e9
 
 - **Comprehensive Benchmark Datasets**: A collection of 36 pre-processed RAG benchmark datasets to test and validate RAG models' performances.
 
-- **Pre-implemented Advanced RAG Algorithms**: Features 16 advancing RAG algorithms with reported results, based on our framework. Easily reproducing results under different settings.
+- **Pre-implemented Advanced RAG Algorithms**: Features 17 advancing RAG algorithms with reported results, based on our framework. Easily reproducing results under different settings.
 
 - **Efficient Preprocessing Stage**: Simplifies the RAG workflow preparation by providing various scripts like corpus processing for retrieval, retrieval index building, and pre-retrieval of documents.
 
@@ -96,6 +79,8 @@ FlashRAG is still under development and there are many issues and room for impro
 - [ ] Enhance code adaptability and readability
 
 ## :page_with_curl: Changelog
+[25/03/21] We have added support for **Reasoning Pipeline**, which is a new paradigm that combines reasoning ability and retrieval, representing work that includes [Search-o1](https://github.com/sunnynexus/Search-o1), [R1-Searcher](https://github.com/SsmallSong/R1-Searcher), [ReSearch](https://github.com/Agent-RL/ReSearch). We evaluate the performance of the pipeline on various RAG benchmarks, it can achieve F1 scores close to 60 on multi hop inference datasets such as Hotpotqe. See it in [**result table**](#robot-supporting-methods).
+
 [25/02/24] 🔥🔥🔥 We have added support for **multimodal RAG**, including [**MLLMs like Llava, Qwen, InternVL**](https://ruc-nlpir.github.io/FlashRAG/#/zh-cn/component/generator?id=%e5%a4%9a%e6%a8%a1%e6%80%81%e7%94%9f%e6%88%90%e5%99%a8), and various [**multimodal retrievers with Clip architecture**](https://ruc-nlpir.github.io/FlashRAG/#/zh-cn/component/retriever?id=%e5%a4%9a%e6%a8%a1%e6%80%81%e6%a3%80%e7%b4%a2%e5%99%a8). More information can be found in our new version of arxiv article and our documentation. Try it!
 
 [25/01/21] Our technical paper [FlashRAG: A Python Toolkit for Efficient RAG Research](https://arxiv.org/abs/2405.13576) is honored to have been accepted to the Resource Track of the 2025 **ACM Web Conference (WWW 2025)**. Please Check it out!
@@ -482,7 +467,7 @@ In each category, we have implemented corresponding common pipelines. Some pipel
           <td>Ranking and merging generated results based on each document</td>
         </tr>
         <tr>
-            <td rowspan="5">Loop</td>
+            <td rowspan="6">Loop</td>
             <td>Iterative Pipeline</td>
             <td>Alternating retrieval and generation</td>
         </tr>
@@ -502,6 +487,10 @@ In each category, we have implemented corresponding common pipelines. Some pipel
             <td>IRCoT Pipeline</td>
             <td>Integrate retrieval process with CoT</td>
         </tr>
+        <tr>
+            <td>Reasoning Pipeline</td>
+            <td>Reasoning with retrieval</td>
+        </tr>
     </tbody>
 </table>
 
@@ -582,6 +571,8 @@ It’s important to note that, to ensure consistency, we have utilized a uniform
 | [Iter-Retgen](https://arxiv.org/abs/2305.15294), [ITRG](https://arxiv.org/abs/2310.05149) | Loop        | 36.8    | 60.1          | 38.3          | 21.6       | 37.9       | 18.2      |                                                 |
 | [IRCoT](https://aclanthology.org/2023.acl-long.557.pdf)                                   | Loop        | 33.3    | 56.9          | 41.5          | 32.4       | 45.6       | 20.7      |                                                 |
 | [RQRAG](https://arxiv.org/abs/2404.00610)                                   | Loop        | 32.6    | 52.5          | 33.5          | 35.8       | 46.4       | 26.2      |  Use trained rqrag-llama2-7B                                               |
+| [R1-Searcher](https://arxiv.org/pdf/2503.05592) | Reasoning | 37.3 | 58.7 | 59.5 | 55.5 | 43.9 | 27.5 | Using trained Qwen2.5-7B model |
+ 
 
 
 ## :notebook: Supporting Datasets & Document Corpus

{flashrag_dev-0.1.4.dev20250319 → flashrag_dev-0.1.4.dev20250321}/flashrag/config/basic_config.yaml

@@ -66,6 +66,8 @@ retrieval_cache_path: ~ # path to the retrieval cache
 retrieval_pooling_method: ~ # set automatically if not provided
 bm25_backend: bm25s # pyserini, bm25s
 use_sentence_transformer: False
+silent_retrieval: True # whether to silent the retrieval process
+
 
 use_reranker: False # whether to use reranker
 rerank_model_name: ~ # same as retrieval_method

{flashrag_dev-0.1.4.dev20250319 → flashrag_dev-0.1.4.dev20250321}/flashrag/dataset/dataset.py

@@ -48,6 +48,13 @@ class Item:
                 except AttributeError:
                     raise AttributeError(f"Attribute `{attr_name}` not found")
 
+    def __setattr__(self, attr_name: str, value: Any) -> None:
+        predefined_attrs = ["id", "question", "golden_answers", "metadata", "output", "choices", 'data']
+        if attr_name in predefined_attrs:
+            super().__setattr__(attr_name, value)
+        else:
+            self.update_output(attr_name, value)
+
     def to_dict(self) -> Dict[str, Any]:
         """Convert all information within the data sample into a dict. Information generated
         during the inference will be saved into output field.

{flashrag_dev-0.1.4.dev20250319 → flashrag_dev-0.1.4.dev20250321}/flashrag/pipeline/__init__.py

@@ -1,4 +1,5 @@
 from flashrag.pipeline.mm_pipeline import *
 from flashrag.pipeline.pipeline import *
 from flashrag.pipeline.branching_pipeline import REPLUGPipeline, SuRePipeline
-from flashrag.pipeline.active_pipeline import IterativePipeline, SelfRAGPipeline, FLAREPipeline, SelfAskPipeline, IRCOTPipeline, RQRAGPipeline
+from flashrag.pipeline.active_pipeline import IterativePipeline, SelfRAGPipeline, FLAREPipeline, SelfAskPipeline, IRCOTPipeline, RQRAGPipeline
+from flashrag.pipeline.reasoning_pipeline import ReasoningPipeline

flashrag_dev-0.1.4.dev20250321/flashrag/pipeline/reasoning_pipeline.py

@@ -0,0 +1,154 @@
+import re
+from tqdm import tqdm
+from typing import List, Tuple, Dict
+import math
+import numpy as np
+import copy
+from transformers import AutoTokenizer, PreTrainedTokenizer, PreTrainedTokenizerFast
+from flashrag.utils import get_retriever, get_generator, selfask_pred_parse, ircot_pred_parse
+from flashrag.pipeline import BasicPipeline
+from flashrag.dataset.utils import get_batch_dataset, merge_batch_dataset
+from flashrag.prompt import PromptTemplate
+from flashrag.utils.utils import extract_between
+
+class ReasoningPipeline(BasicPipeline):
+    system_prompt=""
+    user_prompt=(
+        "The User asks a question, and the Assistant solves it.\n"
+        "The Assistant first thinks about the reasoning process in the mind and then provides the User with the final answer.\n"
+        "The output format of reasoning process and final answer are enclosed within <think> </think> and <answer> </answer> tags, "
+        "respectively, i.e., \"<think> reasoning process here </think>\\n\\n<answer> final answer here </answer>\".\n"
+        "During the thinking process, the Assistant can perform searching for uncertain knowledge if necessary with the format of "
+        "\"<|begin_of_query|> search query (only keywords) here <|end_of_query|>\". **A query must involve only a single triple**.\n"
+        "Then, the system will provide the Assistant with helpful information with the format of "
+        "\"<|begin_of_documents|> ...search results... <|end_of_documents|>\".\n\n"
+        "User:{question}\n"
+        "Assistant: <think>"
+    )
+
+
+    def __init__(self, config, 
+        prompt_template=None, 
+        max_retrieval_num=5, 
+        begin_of_query_token="<|begin_of_query|>",
+        end_of_query_token="<|end_of_query|>",
+        begin_of_documents_token="<|begin_of_documents|>",
+        end_of_documents_token="<|end_of_documents|>",
+        begin_of_answer_token="<answer>",
+        end_of_answer_token='</answer>',
+        retriever=None, 
+        generator=None
+    ):
+        if prompt_template is None:
+            prompt_template = PromptTemplate(
+                config=config,
+                system_prompt=self.system_prompt,
+                user_prompt=self.user_prompt
+            )
+        super().__init__(config, prompt_template)
+
+        if generator is None:
+            self.generator = get_generator(config)
+        else:
+            self.generator = generator
+        if retriever is None:
+            self.retriever = get_retriever(config)
+        else:
+            self.retriever = retriever
+
+        self.max_retrieval_num = max_retrieval_num
+
+        self.begin_of_query_token = begin_of_query_token
+        self.end_of_query_token = end_of_query_token
+        self.begin_of_documents_token = begin_of_documents_token
+        self.end_of_documents_token = end_of_documents_token
+        self.begin_of_answer_token = begin_of_answer_token
+        self.end_of_answer_token = end_of_answer_token
+
+        self.stop_tokens = ["<|im_end|>", "<|endoftext|>", self.end_of_answer_token, self.end_of_query_token]
+    
+    def _retrieved_docs_to_string(self, retrieved_docs: List[Dict]):
+        format_doc_string = ""
+        for idx, doc in enumerate(retrieved_docs):
+            contents = doc['contents']
+            title = contents.split('\n')[0]
+            text = '\n'.join(contents.split('\n')[1:])
+            doc_string = f"Title: {title} Text: {text}"
+            doc_string = re.sub(r'^\d+\s+', '', doc_string)
+            format_doc_string += f'({idx+1}){doc_string}\n'
+        format_doc_string = f'\n\n{self.begin_of_documents_token}\n{format_doc_string}\n{self.end_of_documents_token}\n\n'
+        return format_doc_string
+
+    def run(self, dataset, do_eval=True, pred_process_fun=None):
+        prompts = [self.prompt_template.get_string(question=question) for question in dataset.question]
+        dataset.update_output('prompt', prompts)
+        dataset.update_output('finish_flag', [False] * len(prompts))
+        dataset.update_output('retrieval_results', [{}] * len(prompts))
+        dataset.update_output('retrieved_times', [0] * len(prompts))
+
+        # Logic of reasoning
+        for current_step_idx in range(self.max_retrieval_num + 1):
+            exist_items = [item for item in dataset if item.finish_flag == False]
+            exist_prompts = [item.prompt for item in exist_items]
+            
+            print(f"Current step: {current_step_idx}, exist_items: {len(exist_items)}")
+
+            if len(exist_items) == 0:
+                print("All prompts are finished")
+                break
+            if current_step_idx == self.max_retrieval_num:
+                print("Max retrieval number reached")
+                for item in exist_items:
+                    item.pred = 'No valid answer found'
+                    item.finish_flag = True
+                    item.finish_reason = 'Reach max retrieval number'
+                break
+
+            step_outputs = self.generator.generate(exist_prompts, stop=self.stop_tokens)
+            step_query_list = [] # store generated queries for retrieval
+
+            # parse each sample's step output
+            for item, step_output in zip(exist_items, step_outputs):
+                item.prompt = item.prompt + step_output.strip()
+                if self.end_of_answer_token in step_output and step_output.endswith(self.end_of_answer_token):
+                    item.pred = str(extract_between(step_output, self.begin_of_answer_token, self.end_of_answer_token))
+                    item.finish_flag = True
+                    item.finish_reason = "Finished"
+                
+                elif self.begin_of_query_token in step_output and step_output.endswith(self.end_of_query_token):
+                    query = extract_between(step_output, self.begin_of_query_token, self.end_of_query_token)
+                    if query is not None:
+                        step_query_list.append({'item': item, 'query': query})
+                    else:
+                        item.pred = 'No valid answer found'
+                        item.finish_flag = True
+                        item.finish_reason = 'Query instruction error'
+
+                else:
+                    item.pred = step_output.strip()
+                    item.finish_flag = True
+                    item.finish_reason = 'Normal finish without answer pattern'
+                
+            # do retrieval and add retrieved docs to prompt
+            if len(step_query_list) > 0:
+                retrieved_docs = self.retriever.batch_search([it['query'] for it in step_query_list])
+                for it, item_retrieved_docs in zip(step_query_list, retrieved_docs):
+                    item = it['item']
+                    query = it['query']
+                    item.retrieval_results[item.retrieved_times] = {'query': query,'docs': copy.copy(item_retrieved_docs)}
+                    #item.retrieved_docs += [item_retrieved_docs]
+                    format_doc_string = self._retrieved_docs_to_string(item_retrieved_docs)
+                    item.prompt += format_doc_string
+                    item.retrieved_times += 1
+
+        dataset = self.evaluate(dataset, do_eval=do_eval, pred_process_fun=pred_process_fun)
+        return dataset
+
+            
+                
+                        
+
+                
+            
+        
+            

{flashrag_dev-0.1.4.dev20250319 → flashrag_dev-0.1.4.dev20250321}/flashrag/retriever/encoder.py

@@ -24,13 +24,14 @@ class Encoder:
             Encodes a list of queries into embeddings.
     """
 
-    def __init__(self, model_name, model_path, pooling_method, max_length, use_fp16, instruction):
+    def __init__(self, model_name, model_path, pooling_method, max_length, use_fp16, instruction, silent=False):
         self.model_name = model_name
         self.model_path = model_path
         self.pooling_method = pooling_method
         self.max_length = max_length
         self.use_fp16 = use_fp16
         self.instruction = instruction
+        self.silent = silent
         self.gpu_num = torch.cuda.device_count()
         self.model, self.tokenizer = load_model(model_path=model_path, use_fp16=use_fp16)
 
@@ -67,7 +68,7 @@ class Encoder:
     @torch.inference_mode()
     def encode(self, query_list: List[str], batch_size=64, is_query=True) -> np.ndarray:
         query_emb = []
-        for i in tqdm(range(0, len(query_list), batch_size), desc="Encoding process: "):
+        for i in tqdm(range(0, len(query_list), batch_size), desc="Encoding process: ", disable=self.silent):
             query_emb.append(self.single_batch_encode(query_list[i : i + batch_size], is_query))
         query_emb = np.concatenate(query_emb, axis=0)
         return query_emb
@@ -98,7 +99,7 @@ class STEncoder:
             Encodes a list of queries into embeddings using multiple GPUs.
     """
 
-    def __init__(self, model_name, model_path, max_length, use_fp16, instruction):
+    def __init__(self, model_name, model_path, max_length, use_fp16, instruction, silent=False):
         import torch
         from sentence_transformers import SentenceTransformer
 
@@ -107,6 +108,7 @@ class STEncoder:
         self.max_length = max_length
         self.use_fp16 = use_fp16
         self.instruction = instruction
+        self.silent = silent
         self.model = SentenceTransformer(
             model_path, trust_remote_code=True, model_kwargs={"torch_dtype": torch.float16 if use_fp16 else torch.float}
         )
@@ -115,7 +117,7 @@ class STEncoder:
     def encode(self, query_list: Union[List[str], str], batch_size=64, is_query=True) -> np.ndarray:
         query_list = parse_query(self.model_name, query_list, self.instruction, is_query)
         query_emb = self.model.encode(
-            query_list, batch_size=batch_size, convert_to_numpy=True, normalize_embeddings=True, show_progress_bar=True
+            query_list, batch_size=batch_size, convert_to_numpy=True, normalize_embeddings=True, show_progress_bar=not self.silent
         )
         query_emb = query_emb.astype(np.float32, order="C")
 
@@ -130,7 +132,7 @@ class STEncoder:
             pool,
             normalize_embeddings=True,
             batch_size=batch_size,
-            show_progress_bar=True,
+            show_progress_bar=not self.silent,
         )
         self.model.stop_multi_process_pool(pool)
         query_emb = query_emb.astype(np.float32, order="C")
@@ -190,7 +192,7 @@ class ClipEncoder:
         if not isinstance(query_list, list):
             query_list = [query_list]
         query_emb = []
-        for i in tqdm(range(0, len(query_list), batch_size), desc="Encoding process: "):
+        for i in tqdm(range(0, len(query_list), batch_size), desc="Encoding process: ", disable=self.silent):
             query_emb.append(self.single_batch_encode(query_list[i : i + batch_size], modal))
         query_emb = np.concatenate(query_emb, axis=0)
         return query_emb

{flashrag_dev-0.1.4.dev20250319 → flashrag_dev-0.1.4.dev20250321}/flashrag/retriever/retriever.py

@@ -1,6 +1,7 @@
 import json
 import os
 import time
+
 os.environ["TOKENIZERS_PARALLELISM"] = "false"
 import warnings
 from typing import List, Dict, Union
@@ -116,7 +117,7 @@ class BaseRetriever:
     def __init__(self, config):
         self._config = config
         self.update_config()
-    
+
     @property
     def config(self):
         return self._config
@@ -125,11 +126,11 @@ class BaseRetriever:
     def config(self, config_data):
         self._config = config_data
         self.update_config()
-    
+
     def update_config(self):
         self.update_base_setting()
         self.update_additional_setting()
-    
+
     def update_base_setting(self):
         self.retrieval_method = self._config["retrieval_method"]
         self.topk = self._config["retrieval_topk"]
@@ -154,8 +155,10 @@ class BaseRetriever:
             assert self.cache_path is not None
             with open(self.cache_path, "r") as f:
                 self.cache = json.load(f)
+        self.silent = self._config["silent_retrieval"] if "silent_retrieval" in self._config else False
     def update_additional_setting(self):
         pass
+
     def _save_cache(self):
         self.cache = convert_numpy(self.cache)
 
@@ -163,6 +166,7 @@ class BaseRetriever:
             if isinstance(obj, np.float32):
                 return float(obj)
             raise TypeError(f"Type {type(obj)} not serializable")
+
         with open(self.cache_save_path, "w") as f:
             json.dump(self.cache, f, indent=4, default=custom_serializer)
 
@@ -220,9 +224,10 @@ class BM25Retriever(BaseTextRetriever):
     def __init__(self, config, corpus=None):
         super().__init__(config)
         self.load_model_corpus(corpus)
+
     def update_additional_setting(self):
         self.backend = self._config["bm25_backend"]
-    
+
     def load_model_corpus(self, corpus):
         if self.backend == "pyserini":
             # Warning: the method based on pyserini will be deprecated
@@ -237,24 +242,24 @@ class BM25Retriever(BaseTextRetriever):
                     self.corpus = corpus
             self.max_process_num = 8
 
-            is_zh = judge_zh(self.corpus[0]['contents'])
+            is_zh = judge_zh(self.corpus[0]["contents"])
             if is_zh:
-                self.searcher.set_language('zh')
+                self.searcher.set_language("zh")
         elif self.backend == "bm25s":
             import Stemmer
             import bm25s
 
             self.corpus = load_corpus(self.corpus_path)
-            is_zh = judge_zh(self.corpus[0]['contents'])
+            is_zh = judge_zh(self.corpus[0]["contents"])
 
             self.searcher = bm25s.BM25.load(self.index_path, mmap=True, load_corpus=False)
             if is_zh:
-                self.tokenizer = bm25s.tokenization.Tokenizer(stopwords='zh')
+                self.tokenizer = bm25s.tokenization.Tokenizer(stopwords="zh")
                 self.tokenizer.load_stopwords(self.index_path)
                 self.tokenizer.load_vocab(self.index_path)
             else:
                 stemmer = Stemmer.Stemmer("english")
-                self.tokenizer = bm25s.tokenization.Tokenizer(stopwords='en', stemmer=stemmer)
+                self.tokenizer = bm25s.tokenization.Tokenizer(stopwords="en", stemmer=stemmer)
                 self.tokenizer.load_stopwords(self.index_path)
                 self.tokenizer.load_vocab(self.index_path)
 
@@ -298,7 +303,7 @@ class BM25Retriever(BaseTextRetriever):
             else:
                 results = load_docs(self.corpus, [hit.docid for hit in hits])
         elif self.backend == "bm25s":
-            query_tokens = self.tokenizer.tokenize([query], return_as='tuple', update_vocab=False)
+            query_tokens = self.tokenizer.tokenize([query], return_as="tuple", update_vocab=False)
             results, scores = self.searcher.retrieve(query_tokens, k=num)
             results = list(results[0])
             scores = list(scores[0])
@@ -320,12 +325,12 @@ class BM25Retriever(BaseTextRetriever):
                 results.append(item_result)
                 scores.append(item_score)
         elif self.backend == "bm25s":
-            query_tokens = self.tokenizer.tokenize(query, return_as='tuple', update_vocab=False)
+            query_tokens = self.tokenizer.tokenize(query, return_as="tuple", update_vocab=False)
             results, scores = self.searcher.retrieve(query_tokens, k=num)
         else:
             assert False, "Invalid bm25 backend!"
-        results = results.tolist() if isinstance(results,np.ndarray) else results
-        scores = scores.tolist() if isinstance(scores,np.ndarray) else scores
+        results = results.tolist() if isinstance(results, np.ndarray) else results
+        scores = scores.tolist() if isinstance(scores, np.ndarray) else scores
         if return_score:
             return results, scores
         else:
@@ -337,17 +342,17 @@ class DenseRetriever(BaseTextRetriever):
 
     def __init__(self, config: dict, corpus=None):
         super().__init__(config)
-        
+
         self.load_corpus(corpus)
         self.load_index()
         self.load_model()
-    
+
     def load_corpus(self, corpus):
         if corpus is None:
             self.corpus = load_corpus(self.corpus_path)
         else:
             self.corpus = corpus
-    
+
     def load_index(self):
         if self.index_path is None or not os.path.exists(self.index_path):
             raise Warning(f"Index file {self.index_path} does not exist!")
@@ -358,17 +363,16 @@ class DenseRetriever(BaseTextRetriever):
             co.shard = True
             self.index = faiss.index_cpu_to_all_gpus(self.index, co=co)
 
-    
     def update_additional_setting(self):
         self.query_max_length = self._config["retrieval_query_max_length"]
-        self.pooling_method = self._config['retrieval_pooling_method']
-        self.use_fp16 = self._config['retrieval_use_fp16']
+        self.pooling_method = self._config["retrieval_pooling_method"]
+        self.use_fp16 = self._config["retrieval_use_fp16"]
         self.batch_size = self._config["retrieval_batch_size"]
         self.instruction = self._config["instruction"]
 
-        self.retreival_model_path = self._config['retrieval_model_path']
+        self.retreival_model_path = self._config["retrieval_model_path"]
         self.use_st = self._config["use_sentence_transformer"]
-        self.use_faiss_gpu = self._config['faiss_gpu']
+        self.use_faiss_gpu = self._config["faiss_gpu"]
 
     def load_model(self):
         if self.use_st:
@@ -378,8 +382,11 @@ class DenseRetriever(BaseTextRetriever):
                 max_length = self.query_max_length,
                 use_fp16 = self.use_fp16,
                 instruction = self.instruction,
+                silent = self.silent
             )
         else:
+            # check pooling method
+            self._check_pooling_method(self.retrieval_model_path, self.pooling_method)
             self.encoder = Encoder(
                 model_name = self.retrieval_method,
                 model_path = self.retreival_model_path,
@@ -387,6 +394,7 @@ class DenseRetriever(BaseTextRetriever):
                 max_length = self.query_max_length,
                 use_fp16 = self.use_fp16,
                 instruction = self.instruction,
+                silent = self.silent
             )
 
     def _search(self, query: str, num: int = None, return_score=False):
@@ -413,11 +421,8 @@ class DenseRetriever(BaseTextRetriever):
 
         results = []
         scores = []
-
         emb = self.encoder.encode(query, batch_size=batch_size, is_query=True)
-        print("Begin faiss searching...")
         scores, idxs = self.index.search(emb, k=num)
-        print("End faiss searching")
         scores = scores.tolist()
         idxs = idxs.tolist()
 
@@ -459,6 +464,7 @@ class MultiModalRetriever(BaseRetriever):
         self.encoder = ClipEncoder(
             model_name=self.retrieval_method,
             model_path=config["retrieval_model_path"],
+            silent=self.silent
         )
 
     def _judge_input_modal(self, query):
@@ -480,10 +486,12 @@ class MultiModalRetriever(BaseRetriever):
         )
         if query_modal == "image" and isinstance(query, str):
             from PIL import Image
+
             if os.path.exists(query):
                 query = Image.open(query)
             else:
                 import requests
+
                 query = Image.open(requests.get(query, stream=True).raw)
 
         query_emb = self.encoder.encode(query, modal=query_modal)
@@ -511,6 +519,7 @@ class MultiModalRetriever(BaseRetriever):
         if query_modal == "image" and isinstance(query[0], str):
             from PIL import Image
             import requests
+
             if os.path.exists(query[0]):
                 query = [Image.open(q) for q in query]
             else:
@@ -519,7 +528,7 @@ class MultiModalRetriever(BaseRetriever):
         results = []
         scores = []
 
-        for start_idx in tqdm(range(0, len(query), batch_size), desc="Retrieval process: "):
+        for start_idx in tqdm(range(0, len(query), batch_size), desc="Retrieval process: ", disable=self.silent):
             query_batch = query[start_idx : start_idx + batch_size]
             batch_emb = self.encoder.encode(query_batch, modal=query_modal)
             batch_scores, batch_idxs = self.index_dict[target_modal].search(batch_emb, k=num)
@@ -547,10 +556,10 @@ class MultiRetrieverRouter:
         self.retriever_list = self.load_all_retriever(config)
         self.config = config
 
-        if self.merge_method == 'rerank':
-            config['multi_retriever_setting']['rerank_topk'] = self.final_topk
-            config['multi_retriever_setting']['device'] = config['device']
-            self.reranker = get_reranker(config['multi_retriever_setting'])
+        if self.merge_method == "rerank":
+            config["multi_retriever_setting"]["rerank_topk"] = self.final_topk
+            config["multi_retriever_setting"]["device"] = config["device"]
+            self.reranker = get_reranker(config["multi_retriever_setting"])
 
     def load_all_retriever(self, config):
         retriever_config_list = config["multi_retriever_setting"]["retriever_list"]
@@ -586,7 +595,7 @@ class MultiRetrieverRouter:
                 try:
                     model_config = AutoConfig.from_pretrained(retrieval_model_path)
                     arch = model_config.architectures[0]
-                    print("arch: ",arch)
+                    print("arch: ", arch)
                     if "clip" in arch.lower():
                         retriever = MultiModalRetriever(retriever_config, corpus)
                     else:
@@ -609,11 +618,11 @@ class MultiRetrieverRouter:
                 for _item in item:
                     _item["source"] = retrieval_method
                     _item["corpus_path"] = corpus_path
-                    _item['is_multimodal'] = is_multimodal
+                    _item["is_multimodal"] = is_multimodal
             else:
                 item["source"] = retrieval_method
                 item["corpus_path"] = corpus_path
-                item['is_multimodal'] = is_multimodal
+                item["is_multimodal"] = is_multimodal
         return result
 
     def _search_or_batch_search(self, query: Union[str, list], target_modal, num, return_score, method, retriever_list):
@@ -626,7 +635,7 @@ class MultiRetrieverRouter:
         def process_retriever(retriever):
             is_multimodal = isinstance(retriever, MultiModalRetriever)
             params = {"query": query, "return_score": return_score}
-            
+
             if is_multimodal:
                 params["target_modal"] = target_modal
 
@@ -645,7 +654,9 @@ class MultiRetrieverRouter:
             return result, score
 
         with ThreadPoolExecutor(max_workers=4) as executor:
-            future_to_retriever = {executor.submit(process_retriever, retriever): retriever for retriever in retriever_list}
+            future_to_retriever = {
+                executor.submit(process_retriever, retriever): retriever for retriever in retriever_list
+            }
             for future in as_completed(future_to_retriever):
                 try:
                     result, score = future.result()
@@ -661,7 +672,6 @@ class MultiRetrieverRouter:
         else:
             return result_list
 
-
     def reorder(self, result_list, score_list, retriever_list):
         """
         batch_search:
@@ -670,7 +680,7 @@ class MultiRetrieverRouter:
 
         navie search:
         original result like: [bm25-d1, bm25-d2, e5-d1, e5-d2]
-        
+
         """
 
         retriever_num = len(retriever_list)
@@ -705,8 +715,8 @@ class MultiRetrieverRouter:
                 for query_idx, query_doc_list in enumerate(result_list):
                     exist_id = set()
                     for doc_idx, doc in enumerate(query_doc_list):
-                        if doc['id'] not in exist_id:
-                            exist_id.add(doc['id'])
+                        if doc["id"] not in exist_id:
+                            exist_id.add(doc["id"])
                         else:
                             query_doc_list.remove(doc)
                             if score_list != []:
@@ -726,14 +736,14 @@ class MultiRetrieverRouter:
             else:
                 result_list, score_list = self.rrf_merge(result_list, num, k=60)
             return result_list, score_list
-        elif self.merge_method == 'rerank':
+        elif self.merge_method == "rerank":
             if isinstance(result_list[0], dict):
                 query, result_list, score_list = [query], [result_list], [score_list]
             # parse the result of multimodal corpus
             for item_result in result_list:
                 for item in item_result:
-                    if item['is_multimodal']:
-                        item['contents'] = item['text']
+                    if item["is_multimodal"]:
+                        item["contents"] = item["text"]
             # rerank all docs
             print(result_list)
             result_list, score_list = self.reranker.rerank(query, result_list, topk=num)
@@ -793,50 +803,75 @@ class MultiRetrieverRouter:
         # query: str or PIL.Image
         # judge query type: text or image
         if judge_image(query):
-            retriever_list = [retriever for retriever in self.retriever_list if isinstance(retriever, MultiModalRetriever)]
+            retriever_list = [
+                retriever for retriever in self.retriever_list if isinstance(retriever, MultiModalRetriever)
+            ]
         else:
             retriever_list = self.retriever_list
-        if target_modal == 'image':
+        if target_modal == "image":
             # remove text retriever
             retriever_list = [retriever for retriever in retriever_list if isinstance(retriever, MultiModalRetriever)]
 
-        return self._search_or_batch_search(query, target_modal, num, return_score, method="search", retriever_list=retriever_list)
+        return self._search_or_batch_search(
+            query, target_modal, num, return_score, method="search", retriever_list=retriever_list
+        )
 
     def batch_search(self, query, target_modal="text", num: Union[list, int, None] = None, return_score=False):
         # judge query type: text or image
         if not isinstance(query, list):
             query = [query]
-        if target_modal == 'image':
-            self._retriever_list = [retriever for retriever in self.retriever_list if isinstance(retriever, MultiModalRetriever)]
+        if target_modal == "image":
+            self._retriever_list = [
+                retriever for retriever in self.retriever_list if isinstance(retriever, MultiModalRetriever)
+            ]
         else:
             self._retriever_list = self.retriever_list
         query_type_list = [judge_image(q) for q in query]
         if all(query_type_list):
             # all query is image
-            if self.merge_method == 'rerank':
-                warnings.warn('merge_method is rerank, but all query is image, use default method `concat` instead')
-                self.merge_method = 'concat'
-            retriever_list = [retriever for retriever in self._retriever_list if isinstance(retriever, MultiModalRetriever)]
-
-            return self._search_or_batch_search(query, target_modal, num, return_score, method="batch_search", retriever_list=retriever_list)
+            if self.merge_method == "rerank":
+                warnings.warn("merge_method is rerank, but all query is image, use default method `concat` instead")
+                self.merge_method = "concat"
+            retriever_list = [
+                retriever for retriever in self._retriever_list if isinstance(retriever, MultiModalRetriever)
+            ]
+
+            return self._search_or_batch_search(
+                query, target_modal, num, return_score, method="batch_search", retriever_list=retriever_list
+            )
         elif all([not t for t in query_type_list]):
             # all query is text
             # if exist text retriever, don't use mm retriever for text-text search
             if any([isinstance(retriever, BaseTextRetriever) for retriever in self._retriever_list]):
-                self._retriever_list = [retriever for retriever in self._retriever_list if not isinstance(retriever, MultiModalRetriever)]
-            return self._search_or_batch_search(query, target_modal, num, return_score, method="batch_search", retriever_list=self._retriever_list)
+                self._retriever_list = [
+                    retriever for retriever in self._retriever_list if not isinstance(retriever, MultiModalRetriever)
+                ]
+            return self._search_or_batch_search(
+                query, target_modal, num, return_score, method="batch_search", retriever_list=self._retriever_list
+            )
         else:
             # query list is the mix of image and text
-            if self.merge_method == 'rerank':
-                warnings.warn('merge_method is rerank, but some query is image, use default method `concat` instead')
-                self.merge_method = 'concat'
+            if self.merge_method == "rerank":
+                warnings.warn("merge_method is rerank, but some query is image, use default method `concat` instead")
+                self.merge_method = "concat"
             image_query_idx = [i for i, t in enumerate(query_type_list) if t]
             image_query_list = [query[i] for i in image_query_idx]
             text_query_list = [q for q in query if q not in image_query_list]
 
-            text_output = self._search_or_batch_search(text_query_list, target_modal, num, return_score, method="batch_search", retriever_list=self._retriever_list)
-            retriever_list = [retriever for retriever in self._retriever_list if isinstance(retriever, MultiModalRetriever)]
-            image_output = self._search_or_batch_search(text_query_list, target_modal, num, return_score, method="batch_search", retriever_list=retriever_list)
+            text_output = self._search_or_batch_search(
+                text_query_list,
+                target_modal,
+                num,
+                return_score,
+                method="batch_search",
+                retriever_list=self._retriever_list,
+            )
+            retriever_list = [
+                retriever for retriever in self._retriever_list if isinstance(retriever, MultiModalRetriever)
+            ]
+            image_output = self._search_or_batch_search(
+                text_query_list, target_modal, num, return_score, method="batch_search", retriever_list=retriever_list
+            )
 
             # merge text output and image output
             if return_score:
@@ -854,7 +889,7 @@ class MultiRetrieverRouter:
                     else:
                         final_result.append(image_result[image_idx])
                         final_score.append(image_score[image_idx])
-                        image_idx += 1 
+                        image_idx += 1
                 return final_result, final_score
             else:
                 final_result = []
@@ -866,6 +901,5 @@ class MultiRetrieverRouter:
                         text_idx += 1
                     else:
                         final_result.append(image_result[image_idx])
-                        image_idx += 1 
+                        image_idx += 1
                 return final_result
-

{flashrag_dev-0.1.4.dev20250319 → flashrag_dev-0.1.4.dev20250321}/flashrag/retriever/utils.py

@@ -8,6 +8,8 @@ import re
 import langid
 from transformers import AutoTokenizer, AutoModel, AutoConfig
 
+_has_printed_instruction = False  # trigger instruction print once
+
 def convert_numpy(obj: Union[Dict, list, np.ndarray, np.generic]) -> Any:
     """Recursively convert numpy objects in nested dictionaries or lists to native Python types."""
     if isinstance(obj, dict):
@@ -101,6 +103,7 @@ def parse_query(model_name, query_list, instruction=None, is_query=True):
     """
     processing query for different encoders
     """
+    global _has_printed_instruction
 
     if isinstance(query_list, str):
         query_list = [query_list]
@@ -109,10 +112,14 @@ def parse_query(model_name, query_list, instruction=None, is_query=True):
         instruction = instruction.strip() + " "
     else:
         instruction = set_default_instruction(model_name, is_query=is_query, is_zh=judge_zh(query_list[0]))
-    print(f"Use `{instruction}` as retreival instruction")
-    if instruction == "":
-        warnings.warn('Instruction is not set')
-
+    
+    if not _has_printed_instruction:
+        if instruction == "":
+            warnings.warn('Instruction is not set')
+        else:
+            print(f"Use `{instruction}` as retreival instruction")
+        _has_printed_instruction = True
+        
     query_list = [instruction + query for query in query_list]
 
     return query_list

{flashrag_dev-0.1.4.dev20250319 → flashrag_dev-0.1.4.dev20250321}/flashrag/utils/utils.py

@@ -1,4 +1,5 @@
 import os
+import re
 import json
 import importlib
 from transformers import AutoConfig
@@ -173,3 +174,10 @@ def hash_object(o) -> str:
         dill.dump(o, buffer)
         m.update(buffer.getbuffer())
         return base58.b58encode(m.digest()).decode()
+
+def extract_between(text: str, start_tag: str, end_tag: str):
+    pattern = re.escape(start_tag) + r"(.*?)" + re.escape(end_tag)
+    matches = re.findall(pattern, text, flags=re.DOTALL)
+    if matches:
+        return matches[-1].strip()
+    return None

flashrag_dev-0.1.4.dev20250321/flashrag/version.py

@@ -0,0 +1 @@
+__version__ = "0.1.4dev20250321"

flashrag_dev-0.1.4.dev20250319/README.md → flashrag_dev-0.1.4.dev20250321/flashrag_dev.egg-info/PKG-INFO

@@ -1,3 +1,20 @@
+Metadata-Version: 2.1
+Name: flashrag-dev
+Version: 0.1.4.dev20250321
+Summary: A library for efficient Retrieval-Augmented Generation research
+Home-page: https://github.com/RUC-NLPIR/FlashRAG
+Author: Jiajie Jin, Yutao Zhu, Chenghao Zhang, Xinyu Yang, Zhicheng Dou
+Author-email: jinjiajie@ruc.edu.cn
+License: MIT License
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Provides-Extra: core
+Provides-Extra: retriever
+Provides-Extra: generator
+Provides-Extra: multimodal
+Provides-Extra: full
+License-File: LICENSE
+
 # <div align="center">⚡FlashRAG: A Python Toolkit for Efficient RAG Research<div>
 \[ English | [中文](README_zh.md) \]
 <div align="center">
@@ -24,7 +41,7 @@
 </h4>
 
 
-FlashRAG is a Python toolkit for the reproduction and development of Retrieval Augmented Generation (RAG) research. Our toolkit includes 36 pre-processed benchmark RAG datasets and 16 state-of-the-art RAG algorithms.
+FlashRAG is a Python toolkit for the reproduction and development of Retrieval Augmented Generation (RAG) research. Our toolkit includes 36 pre-processed benchmark RAG datasets and 17 state-of-the-art RAG algorithms.
 
 <p align="center">
 <img src="asset/framework.jpg">
@@ -58,7 +75,7 @@ https://github.com/user-attachments/assets/8ca00873-5df2-48a7-b853-89e7b18bc6e9
 
 - **Comprehensive Benchmark Datasets**: A collection of 36 pre-processed RAG benchmark datasets to test and validate RAG models' performances.
 
-- **Pre-implemented Advanced RAG Algorithms**: Features 16 advancing RAG algorithms with reported results, based on our framework. Easily reproducing results under different settings.
+- **Pre-implemented Advanced RAG Algorithms**: Features 17 advancing RAG algorithms with reported results, based on our framework. Easily reproducing results under different settings.
 
 - **Efficient Preprocessing Stage**: Simplifies the RAG workflow preparation by providing various scripts like corpus processing for retrieval, retrieval index building, and pre-retrieval of documents.
 
@@ -79,6 +96,8 @@ FlashRAG is still under development and there are many issues and room for impro
 - [ ] Enhance code adaptability and readability
 
 ## :page_with_curl: Changelog
+[25/03/21] We have added support for **Reasoning Pipeline**, which is a new paradigm that combines reasoning ability and retrieval, representing work that includes [Search-o1](https://github.com/sunnynexus/Search-o1), [R1-Searcher](https://github.com/SsmallSong/R1-Searcher), [ReSearch](https://github.com/Agent-RL/ReSearch). We evaluate the performance of the pipeline on various RAG benchmarks, it can achieve F1 scores close to 60 on multi hop inference datasets such as Hotpotqe. See it in [**result table**](#robot-supporting-methods).
+
 [25/02/24] 🔥🔥🔥 We have added support for **multimodal RAG**, including [**MLLMs like Llava, Qwen, InternVL**](https://ruc-nlpir.github.io/FlashRAG/#/zh-cn/component/generator?id=%e5%a4%9a%e6%a8%a1%e6%80%81%e7%94%9f%e6%88%90%e5%99%a8), and various [**multimodal retrievers with Clip architecture**](https://ruc-nlpir.github.io/FlashRAG/#/zh-cn/component/retriever?id=%e5%a4%9a%e6%a8%a1%e6%80%81%e6%a3%80%e7%b4%a2%e5%99%a8). More information can be found in our new version of arxiv article and our documentation. Try it!
 
 [25/01/21] Our technical paper [FlashRAG: A Python Toolkit for Efficient RAG Research](https://arxiv.org/abs/2405.13576) is honored to have been accepted to the Resource Track of the 2025 **ACM Web Conference (WWW 2025)**. Please Check it out!
@@ -465,7 +484,7 @@ In each category, we have implemented corresponding common pipelines. Some pipel
           <td>Ranking and merging generated results based on each document</td>
         </tr>
         <tr>
-            <td rowspan="5">Loop</td>
+            <td rowspan="6">Loop</td>
             <td>Iterative Pipeline</td>
             <td>Alternating retrieval and generation</td>
         </tr>
@@ -485,6 +504,10 @@ In each category, we have implemented corresponding common pipelines. Some pipel
             <td>IRCoT Pipeline</td>
             <td>Integrate retrieval process with CoT</td>
         </tr>
+        <tr>
+            <td>Reasoning Pipeline</td>
+            <td>Reasoning with retrieval</td>
+        </tr>
     </tbody>
 </table>
 
@@ -565,6 +588,8 @@ It’s important to note that, to ensure consistency, we have utilized a uniform
 | [Iter-Retgen](https://arxiv.org/abs/2305.15294), [ITRG](https://arxiv.org/abs/2310.05149) | Loop        | 36.8    | 60.1          | 38.3          | 21.6       | 37.9       | 18.2      |                                                 |
 | [IRCoT](https://aclanthology.org/2023.acl-long.557.pdf)                                   | Loop        | 33.3    | 56.9          | 41.5          | 32.4       | 45.6       | 20.7      |                                                 |
 | [RQRAG](https://arxiv.org/abs/2404.00610)                                   | Loop        | 32.6    | 52.5          | 33.5          | 35.8       | 46.4       | 26.2      |  Use trained rqrag-llama2-7B                                               |
+| [R1-Searcher](https://arxiv.org/pdf/2503.05592) | Reasoning | 37.3 | 58.7 | 59.5 | 55.5 | 43.9 | 27.5 | Using trained Qwen2.5-7B model |
+ 
 
 
 ## :notebook: Supporting Datasets & Document Corpus

{flashrag_dev-0.1.4.dev20250319 → flashrag_dev-0.1.4.dev20250321}/flashrag_dev.egg-info/SOURCES.txt

@@ -29,6 +29,7 @@ flashrag/pipeline/active_pipeline.py
 flashrag/pipeline/branching_pipeline.py
 flashrag/pipeline/mm_pipeline.py
 flashrag/pipeline/pipeline.py
+flashrag/pipeline/reasoning_pipeline.py
 flashrag/pipeline/replug_utils.py
 flashrag/prompt/__init__.py
 flashrag/prompt/base_prompt.py

flashrag_dev-0.1.4.dev20250319/flashrag/version.py

@@ -1 +0,0 @@
-__version__ = "0.1.4dev20250319"