sdg-hub 0.1.2__tar.gz → 0.1.4__tar.gz

{sdg_hub-0.1.2/src/sdg_hub.egg-info → sdg_hub-0.1.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sdg_hub
-Version: 0.1.2
+Version: 0.1.4
 Summary: Synthetic Data Generation
 Author-email: Red Hat AI Innovation <abhandwa@redhat.com>
 License: Apache-2.0
@@ -36,7 +36,7 @@ Requires-Dist: flask>=3.0.2; extra == "web-interface"
 Requires-Dist: pyyaml>=6.0.1; extra == "web-interface"
 Requires-Dist: flask-wtf>=1.2.2; extra == "web-interface"
 Provides-Extra: vllm
-Requires-Dist: vllm<0.8.4,>=0.8.0; extra == "vllm"
+Requires-Dist: vllm>=0.9.1; extra == "vllm"
 Requires-Dist: torch>=2.0.0; extra == "vllm"
 Requires-Dist: transformers>=4.37.0; extra == "vllm"
 Requires-Dist: accelerate>=0.21.0; extra == "vllm"

sdg_hub-0.1.4/examples/knowledge_tuning/README.md

@@ -0,0 +1,115 @@
+# Synthetic Data Generation for Knowledge Tuning
+
+## What is Knowledge Tuning?
+
+**Knowledge tuning** is the process of adapting a large language model (LLM) to new factual content by training it on specific documents. The goal is to enable the model to **recall and reason over document-grounded information** when performing downstream tasks such as:
+
+* Question Answering
+* Summarization
+* Entity Extraction
+* Other document-specific reasoning tasks
+
+This adaptation can be used:
+
+* As a **standalone fine-tuned model**, or
+* As part of a **Retrieval-Augmented Generation (RAG)** pipeline to enhance factual accuracy and contextuality.
+
+---
+
+### Setup Instructions
+
+#### Install sdg-hub
+
+```bash 
+pip install sdg-hub==0.1.0a4
+```
+
+#### Install with optional dependencies
+
+If you want to use the vLLM server, you can install it with the following command:
+
+```bash 
+pip install sdg-hub[vllm] 
+```
+
+In order to use docling, you need to install it with the following command:
+
+```bash
+pip install sdg-hub[examples]
+```
+
+### Serving the Teacher Model
+
+#### vLLM Server
+
+Launch the vLLM server with the following command:
+
+```bash
+vllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 4
+```
+
+## Repository Structure
+
+This repository demonstrates how to generate synthetic data for knowledge tuning using different approaches:
+
+### Examples
+
+1. [`instructlab/`](instructlab/):
+   Implements knowledge tuning using the **InstructLab** pipeline, which supports a two-phase approach:
+
+   * Phase 1: Knowledge tuning via synthetic QAs
+   * Phase 2: Instruction tuning to generalize reasoning skills
+
+2. [`knowledge_tuning_with_reasoning_model/`](knowledge_tuning_with_reasoning_model/):
+   Uses **Nemotron Super** as the teacher model to generate reasoning-focused synthetic data grounded in document content. We also show how to edit the knowledge pipeline to introduce new types of summaries
+
+Each example includes:
+
+* Source document processing
+* QA generation with a teacher model
+* Filtering and validation logic
+* Dataset formatting for fine-tuning
+
+3. [`translation_example`](translation_example/):
+    Implements a translation block to translate article into a target language for generating knowledge QA. The example scripts show how to translate Kannada Wikipedia article into English and generate synthetic QA to train a model.
+---
+
+## Data Post-Processing
+
+Once synthetic QA data is generated, you’ll need to prepare it for training:
+
+### Key Practices
+
+* Append source document content to the generated QA to improve memorization and coverage.
+* During training, backpropagate on both the **prompt** (document + question) and the **response** (answer).
+* For `instructlab.training`, you can use the `unmask` field to enable pretraining-style loss computation over the full prompt-response.
+
+### Creating QA dataset
+
+* You can use below function to transform the generated dataset into Prompt + Response pair for training in messages format.
+* You can control various parameters like appending document to question, adding document outline to document etc.
+```python
+from knowledge_utils import generate_knowledge_qa_dataset
+
+knowl_train = generate_knowledge_qa_dataset(
+    generated_dataset=generated_data,
+    keep_context_separate=False,
+    keep_document_outline=True,
+    keep_columns=['document', 'document_outline', 'raw_document']
+)
+```
+* `keep_context_separate=False`: Includes the document in the prompt
+* `keep_document_outline=True`: Adds structure to the prompt using outline
+* `keep_columns`: Retains metadata for record-keeping (not used in training)
+
+
+### Workflow: InstructLab (Knowledge + RAFT)
+You can find steps for data post-processing [here](instructlab/README.md#data-post-processing)
+
+### Workflow: Fine-tuning Instruct Model
+
+* You can simply take the generated data and continue instruction tuning an existing instruct model (e.g. Qwen 2.5 8B instruct, LLama 3.3 8B/70B etc.)
+* Simply follow [Creating QA dataset](#creating-qa-dataset) section for creating the training data.
+* Note: The model might suffer catastrophic forgetting and might need a replay buffer of instruction data. Or you might need to explore alternate methods like Parameter efficient fine-tuning.
+
+

sdg_hub-0.1.4/examples/knowledge_tuning/instructlab/README.md

@@ -0,0 +1,110 @@
+# InstructLab: Synthetic Data Generation for Knowledge Tuning
+![InstructLab Banner](../../../assets/imgs/instructlab-banner.png)
+
+The provided notebooks show a sdg_hub pipeline for generating high-quality synthetic data from your documents. By following the methodology of the **LAB (Large-scale Alignment for Chatbots)** framework, as detailed in our [research paper](https://arxiv.org/pdf/2403.01081), you can effectively tune language models to master the knowledge contained within your specific-domain documentation.
+
+## How It Works: A Three-Phase Pipeline
+
+Our data generation process is designed to operate in three distinct phases, ensuring the creation of a robust and reliable dataset for model training.
+
+### 1\. Document Summarization
+
+To kickstart the process, we generate three unique summaries of your source documents. This multi-faceted approach helps the model to thoroughly memorize and recall the key information. The summaries include:
+
+  * **Detailed Summaries:** Comprehensive overviews of the content.
+  * **Extractive Summaries:** Key sentences and passages pulled directly from the text.
+  * **Atomic Facts:** A list of the most critical, standalone pieces of information.
+
+### 2\. Synthetic Q\&A Generation
+
+Next, our pipeline leverages user-provided "seed examples"—sample questions and answers—to generate a wealth of synthetic Q\&A pairs. These new pairs are contextually grounded in the summarized documents, effectively scaling up your initial examples into a diverse training dataset.
+
+### 3\. Quality Control
+
+To ensure the integrity of our generated data, we employ a quality-checking phase. Using a "teacher" model, we perform a faithfulness evaluation by:
+
+1.  Providing the model with a generated answer and the original source document.
+2.  Tasking the model to extract every claim made in the answer.
+3.  Verifying that each claim is factually supported by the provided document.
+
+This process filters out inaccuracies and ensures that only high-quality, faithful Q\&A pairs make it into the final dataset.
+
+## Getting Started
+
+To begin using the pipeline, simply install the `sdg_hub` library. From there, you can instantiate and run the synthetic data generation process with the following code:
+
+```python
+from sdg_hub.flow import Flow
+from sdg_hub.sdg import SDG
+
+# Load the knowledge generation pipeline from the YAML file
+knowledge_agentic_pipeline = "../../../src/instructlab/sdg/flows/generation/knowledge/synth_knowledge1.5.yaml"
+flow = Flow(openai_client).get_flow_from_file(knowledge_agentic_pipeline)
+
+# Initialize the Synthetic Data Generator
+generator = SDG(
+    flows=[flow],
+    num_workers=1,
+    batch_size=1,
+    save_freq=1000,
+)
+```
+
+## InstructLab Training Methodology
+
+Our training process is structured to build upon a pre-trained model, systematically enhancing its skills and knowledge.
+
+1.  **Foundation Training:** We begin by training a pre-trained model on foundational skills such as logic, coding, and math. The instruction data in this phase features short, direct responses.
+2.  **Foundation Knowledge:** Next, we expand the model's general knowledge base, by training it on general textbooks and benchmarking it against MMLU. The result of these first two stages is what we term the **starter model**.
+
+This **starter model** then serves as the base for our specialized, two-phase knowledge tuning:
+
+  * **Phase 1: Knowledge Tuning:** We do pre-training style training on the document generated data by our pipeline. This allows the model to internalize the new knowledge and be able to recall and answer questions based on it.
+  * **Phase 2: Skills Tuning:** Building on the previous phase, we do instruction tuning on general skills (combination of instruction tuning dataset and skills generated with sdg_hub). To prevent the model from forgetting the newly acquired knowledge, we mix in data from the previous stage. We also incorporate [RAFT-style](https://openreview.net/forum?id=rzQGHXNReU) data to enhance the model's robustness for RAG on the target documents.
+
+## Data Post-Processing
+
+After generating your data, use the provided utility functions to prepare it for the two-phase training process. All helper functions are located in `examples/knowledge_tuning/knowledge_utils.py`.
+
+### 1\. Knowledge Dataset (for Phase 1)
+
+This dataset is used for the initial knowledge-tuning phase. You can also merge datasets from multiple documents to train on a set of documents simultaneously.
+
+This function also creates a summarization dataset that formats the generated summaries as task: document + instruction -> document summary.
+
+```python
+from knowledge_utils import create_knowledge_pretraining_ds
+
+# Create the dataset for knowledge pre-training
+knowledge_data = create_knowledge_pretraining_ds(generated_dataset=generated_data)
+```
+
+### 2\. Skills Dataset (for Phase 2)
+
+This dataset combines the knowledge-specific data with RAFT-style examples for the second phase of tuning. It can also be mixed with general instruction-tuning data to grant the model broad instruction-following abilities while retaining the specialized knowledge.
+
+```python
+from knowledge_utils import create_knowledge_regular_ds
+from datasets import concatenate_datasets
+
+# Create the dataset with RAFT and summary data
+raft_and_summary_data = create_knowledge_regular_ds(generated_dataset=generated_data)
+
+# Create the core knowledge dataset. 
+# add_auxiliary_dataset parameter controls wheter to add the summarization dataset that was mentioned above in the returned dataset
+knowledge_data = create_knowledge_pretraining_ds(generated_dataset=generated_data, add_auxiliary_dataset=False)
+
+# Combine the datasets for the skills tuning phase
+knowledge_skills_data = concatenate_datasets([raft_and_summary_data, knowledge_data])
+```
+
+---
+
+## Generation Statistics
+
+Default generation parameters (based on `llama-3.3-70B`) are defined in:
+[`synth_knowledge1.5.yaml`](../../src/sdg_hub/flows/generation/knowledge/synth_knowledge1.5.yaml)
+
+* The pipeline converts each input document into **3 summaries** 
+* Outputs vary based on teacher model and generation parameters (e.g. `temperature`, `top_p`, `top_k`) and can be entered in the `gen_kwargs` section of the flow.
+* Generation currently uses temperature=0.0 and is deterministic.

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/examples/knowledge_tuning/instructlab/knowledge_generation_and_mixing.ipynb

@@ -34,7 +34,6 @@
     "\n",
     "# First Party\n",
     "from sdg_hub.flow import Flow\n",
-    "from sdg_hub.pipeline import Pipeline\n",
     "from sdg_hub.sdg import SDG\n",
     "import sys\n",
     "import os\n",
@@ -86,9 +85,9 @@
    "outputs": [],
    "source": [
     "knowledge_agentic_pipeline = \"../../../src/instructlab/sdg/flows/generation/knowledge/synth_knowledge1.5.yaml\"\n",
-    "flow_cfg = Flow(client).get_flow_from_file(knowledge_agentic_pipeline)\n",
+    "flow = Flow(client).get_flow_from_file(knowledge_agentic_pipeline)\n",
     "sdg = SDG(\n",
-    "    [Pipeline(flow_cfg)],\n",
+    "    flows=[flow],\n",
     "    num_workers=1,\n",
     "    batch_size=1,\n",
     "    save_freq=1000,\n",

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/examples/knowledge_tuning/knowledge_utils.py

@@ -268,7 +268,22 @@ def build_raft_dataset(ds: Dataset, p, num_doc_in_context=4):
 
 
 def create_knowledge_regular_ds(generated_dataset: Dataset):
-    # Phase 1.0
+    """  
+    Create a knowledge dataset for the Skills Phase of knowledge tuning.  
+    
+    This function generates QA datasets with RAFT-style context separation  
+    and optionally includes auxiliary datasets for enhanced training.  
+    
+    Parameters  
+    ----------  
+    generated_dataset : Dataset  
+        The input dataset containing generated knowledge content  
+        
+    Returns  
+    -------  
+    Dataset  
+        Processed dataset ready for skills phase training 
+    """
     knowledge_ds = generate_knowledge_qa_dataset(
         generated_dataset, keep_context_separate=True
     )
@@ -276,26 +291,36 @@ def create_knowledge_regular_ds(generated_dataset: Dataset):
 
     auxiliary_dataset = create_auxiliary_dataset(generated_dataset)
     if auxiliary_dataset is not None:
-        transformed_data = safe_concatenate_datasets([knowledge_ds, auxiliary_dataset])
-    else:
-        transformed_data = knowledge_ds
-    return transformed_data
+        knowledge_ds = safe_concatenate_datasets([knowledge_ds, auxiliary_dataset])
+    return knowledge_ds
 
 
-def create_knowledge_pretraining_ds(generated_dataset: Dataset):
-    # Phase 0.7
+def create_knowledge_pretraining_ds(generated_dataset: Dataset, add_auxiliary_dataset: bool = True):
+    # Phase 0.7 (Knowledge Phase)
+    """  
+    Create a knowledge dataset for the Knowledge Phase of knowledge tuning.  
+    
+    This function generates QA datasets for pretraining-style knowledge tuning  
+    with optional auxiliary dataset inclusion.  
+    
+    Parameters  
+    ----------  
+    generated_dataset (Dataset): The dataset containing generated knowledge data.  
+    add_auxiliary_dataset (bool): Whether to include an auxiliary dataset.  
+    
+    Returns  
+    -------  
+    Dataset: The generated knowledge dataset.  
+    """
     knowledge_ds = generate_knowledge_qa_dataset(
-        generated_dataset, keep_context_separate=False
-    )
+        generated_dataset, keep_context_separate=False)
     knowledge_ds = knowledge_ds.map(_conv_pretrain)
 
     auxiliary_dataset = create_auxiliary_dataset(generated_dataset)
-    if auxiliary_dataset is not None:
+    if auxiliary_dataset is not None and add_auxiliary_dataset:
         auxiliary_dataset = auxiliary_dataset.map(_conv_pretrain)
-        transformed_data = safe_concatenate_datasets([knowledge_ds, auxiliary_dataset])
-    else:
-        transformed_data = knowledge_ds
-    return transformed_data
+        knowledge_ds = safe_concatenate_datasets([knowledge_ds, auxiliary_dataset])
+    return knowledge_ds
 
 
 def fuse_texts(text_list, short_length_threshold=100):

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/pyproject.toml

@@ -55,7 +55,7 @@ web_interface = [
     "flask-wtf>=1.2.2",
 ]
 vllm = [
-    "vllm>=0.8.0,<0.8.4",
+    "vllm>=0.9.1",
     "torch>=2.0.0",
     "transformers>=4.37.0",
     "accelerate>=0.21.0",

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/src/sdg_hub/_version.py

@@ -17,5 +17,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '0.1.2'
-__version_tuple__ = version_tuple = (0, 1, 2)
+__version__ = version = '0.1.4'
+__version_tuple__ = version_tuple = (0, 1, 4)

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/src/sdg_hub/configs/annotations/simple_annotations.yaml

@@ -1,4 +1,4 @@
-system: null
+system: "You are a helpful assistant that annotates text."
 introduction: "Task Description: Data Annotation"
 principles: null
 examples: null

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/src/sdg_hub/configs/knowledge/evaluate_relevancy.yaml

@@ -9,7 +9,7 @@ principles: |
   
   For each question, assign a score of 1 point if the response meets the criteria, and 0 points if it does not. After evaluating each question, provide detailed feedback explaining your reasoning behind the scores awarded.
 
-  Conclude your evaluation with a final result, strictly using the following format: 'Total Score: X'. The total score should represent the sum of points assigned for each question, with a maximum possible score of 2 points.
+  Conclude your evaluation with a total score as a final result. The total score should represent the sum of points assigned for each question, with a maximum possible score of 2 points.
   Only evaluate the response based on the above criteria, do not create new questions.
 
 examples: |
@@ -49,7 +49,6 @@ examples: |
   0
   [End of Score]
 
-
   Example 3:
   [Start of Question]
   What are the benefits of electric vehicles?

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/src/sdg_hub/flow_runner.py

@@ -42,6 +42,7 @@ def run_flow(
     debug: bool = False,
     dataset_start_index: int = 0,
     dataset_end_index: Optional[int] = None,
+    api_key: Optional[str] = None,
 ) -> None:
     """Process the dataset using the specified configuration.
 
@@ -69,6 +70,8 @@ def run_flow(
         Start index for dataset slicing, by default 0.
     dataset_end_index : Optional[int], optional
         End index for dataset slicing, by default None.
+    api_key : Optional[str], optional
+        API key for the remote endpoint. If not provided, will use OPENAI_API_KEY environment variable, by default None.
 
     Returns
     -------
@@ -137,9 +140,9 @@ def run_flow(
             ) from e
 
         # Validate API configuration
-        openai_api_key = os.environ.get("OPENAI_API_KEY")
+        openai_api_key = api_key or os.environ.get("OPENAI_API_KEY")
         if not openai_api_key or openai_api_key == "EMPTY":
-            logger.warning("OPENAI_API_KEY not set or is 'EMPTY'. API calls may fail.")
+            logger.warning("API key not provided and OPENAI_API_KEY not set or is 'EMPTY'. API calls may fail.")
 
         openai_api_base = endpoint
         if not openai_api_base:
@@ -349,6 +352,12 @@ def run_flow(
 @click.option(
     "--dataset_end_index", type=int, default=None, help="End index of the dataset."
 )
+@click.option(
+    "--api_key",
+    type=str,
+    default=None,
+    help="API key for the remote endpoint. If not provided, will use OPENAI_API_KEY environment variable.",
+)
 def main(
     ds_path: str,
     bs: int,
@@ -361,6 +370,7 @@ def main(
     debug: bool,
     dataset_start_index: int,
     dataset_end_index: Optional[int],
+    api_key: Optional[str],
 ) -> None:
     """CLI entry point for running data generation flows.
 
@@ -388,6 +398,8 @@ def main(
         Start index for dataset slicing.
     dataset_end_index : Optional[int]
         End index for dataset slicing.
+    api_key : Optional[str]
+        API key for the remote endpoint. If not provided, will use OPENAI_API_KEY environment variable.
 
     Returns
     -------
@@ -406,6 +418,7 @@ def main(
             debug=debug,
             dataset_start_index=dataset_start_index,
             dataset_end_index=dataset_end_index,
+            api_key=api_key,
         )
     except (
         DatasetLoadError,

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/src/sdg_hub/flows/generation/knowledge/mmlu_bench.yaml

@@ -2,7 +2,7 @@
   block_config:
     block_name: gen_mmlu_knowledge
     config_path: configs/knowledge/mcq_generation.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - mmlubench_question
       - mmlubench_answer

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/src/sdg_hub/flows/generation/knowledge/simple_knowledge.yaml

@@ -2,7 +2,7 @@
   block_config:
     block_name: gen_knowledge
     config_path: configs/knowledge/simple_generate_qa.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - output
   gen_kwargs:

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/src/sdg_hub/flows/generation/knowledge/synth_knowledge.yaml

@@ -2,7 +2,7 @@
   block_config:
     block_name: gen_knowledge
     config_path: configs/knowledge/generate_questions_responses.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - question
       - response
@@ -20,7 +20,7 @@
   block_config:
     block_name: eval_faithfulness_qa_pair
     config_path: configs/knowledge/evaluate_faithfulness.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - explanation
       - judgment
@@ -43,7 +43,7 @@
   block_config:
     block_name: eval_relevancy_qa_pair
     config_path: configs/knowledge/evaluate_relevancy.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - feedback
       - score
@@ -67,7 +67,7 @@
   block_config:
     block_name: eval_verify_question
     config_path: configs/knowledge/evaluate_question.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - explanation
       - rating

{sdg_hub-0.1.2 → sdg_hub-0.1.4}/src/sdg_hub/flows/generation/knowledge/synth_knowledge1.5.yaml

@@ -8,35 +8,31 @@
   block_config:
     block_name: gen_detailed_summary
     config_path: configs/knowledge/detailed_summary.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - summary_detailed
   gen_kwargs:
-    max_tokens: 4096
-    temperature: 0.7
-    n: 50
+    max_tokens: 2048
 
 - block_type: LLMBlock
   block_config:
     block_name: gen_atomic_facts
     config_path: configs/knowledge/atomic_facts.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - summary_atomic_facts
   gen_kwargs:
-    max_tokens: 4096
-    temperature: 0.7
+    max_tokens: 2048
 
 - block_type: LLMBlock
   block_config:
     block_name: gen_extractive_summary
     config_path: configs/knowledge/extractive_summary.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - summary_extractive
   gen_kwargs:
-    max_tokens: 4096
-    temperature: 0.7
+    max_tokens: 2048
 
 - block_type: FlattenColumnsBlock
   block_config:
@@ -59,33 +55,25 @@
 - block_type: LLMBlock
   block_config:
     block_name: knowledge generation
-    config_path: configs/knowledge/generate_questions.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    config_path: configs/knowledge/generate_questions_responses.yaml
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - question
+      - response
     parser_kwargs:
       parser_name: custom
-      parsing_pattern: "\\[(?:Question|QUESTION)\\]\\s*(.*?)\\s*(?=\\[(?:Question|QUESTION)\\]|$)"
-  gen_kwargs:
-    temperature: 0.7
-    max_tokens: 100
-
-- block_type: LLMBlock
-  block_config:
-    block_name: knowledge generation
-    config_path: configs/knowledge/generate_responses.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
-    output_cols:
-      - response
+      parsing_pattern: "\\[(?:Question|QUESTION)\\]\\s*(.*?)\\s*\\[(?:Answer|ANSWER)\\]\\s*(.*?)\\s*(?=\\[(?:Question|QUESTION)\\]|$)"
+      parser_cleanup_tags:
+        - "[END]"
   gen_kwargs:
-    temperature: 0.7
+    temperature: 0.0
     max_tokens: 2048
 
 - block_type: LLMBlock
   block_config:
     block_name: eval_faithfulness_qa_pair
     config_path: configs/knowledge/evaluate_faithfulness.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - explanation
       - judgment
@@ -106,7 +94,7 @@
   block_config:
     block_name: eval_relevancy_qa_pair
     config_path: configs/knowledge/evaluate_relevancy.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - feedback
       - score
@@ -128,7 +116,7 @@
   block_config:
     block_name: eval_verify_question
     config_path: configs/knowledge/evaluate_question.yaml
-    model_id: mistralai/Mixtral-8x7B-Instruct-v0.1
+    model_id: meta-llama/Llama-3.3-70B-Instruct
     output_cols:
       - explanation
       - rating