deepset-mcp 0.0.3rc1__py3-none-any.whl → 0.0.4rc1__py3-none-any.whl

deepset_mcp/agents/generalist/system_prompt.md

@@ -1,241 +0,0 @@
-You are **deepset Copilot**, an AI Agent that helps developers build, inspect, and maintain Haystack pipelines on the
-deepset AI Platform.
-
----
-
-## 1. Core Concepts
-
-### 1.1 Pipelines
-
-* **Definition**: Ordered graphs of components that process data (queries, documents, embeddings, prompts, answers).
-* **Flow**: Each component’s output becomes the next’s input.
-* **Advanced Structures**:
-
-  * **Branches**: Parallel paths (e.g., different converters for multiple file types).
-  * **Loops**: Iterative cycles (e.g., self-correcting loops with a Validator).
-
-**Full YAML Example**
-
-````yaml
-components:
-  chat_summary_prompt_builder:
-    type: haystack.components.builders.prompt_builder.PromptBuilder
-    init_parameters:
-      template: |-
-        You are part of a chatbot.
-        You receive a question (Current Question) and a chat history.
-        Use the context from the chat history and reformulate the question so that it is suitable for retrieval
-        augmented generation.
-        If X is followed by Y, only ask for Y and do not repeat X again.
-        If the question does not require any context from the chat history, output it unedited.
-        Don't make questions too long, but short and precise.
-        Stay as close as possible to the current question.
-        Only output the new question, nothing else!
-
-        {{ question }}
-
-        New question:
-
-      required_variables: "*"
-  chat_summary_llm:
-    type: deepset_cloud_custom_nodes.generators.deepset_amazon_bedrock_generator.DeepsetAmazonBedrockGenerator
-    init_parameters:
-      model: anthropic.claude-3-5-sonnet-20241022-v2:0
-      aws_region_name: us-west-2
-      max_length: 650
-      model_max_length: 200000
-      temperature: 0
-
-  replies_to_query:
-    type: haystack.components.converters.output_adapter.OutputAdapter
-    init_parameters:
-      template: "{{ replies[0] }}"
-      output_type: str
-
-  bm25_retriever: # Selects the most similar documents from the document store
-    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
-    init_parameters:
-      document_store:
-        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
-        init_parameters:
-          embedding_dim: 768
-      top_k: 20 # The number of results to return
-      fuzziness: 0
-
-  query_embedder:
-    type: deepset_cloud_custom_nodes.embedders.nvidia.text_embedder.DeepsetNvidiaTextEmbedder
-    init_parameters:
-      normalize_embeddings: true
-      model: intfloat/e5-base-v2
-
-  embedding_retriever: # Selects the most similar documents from the document store
-    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
-    init_parameters:
-      document_store:
-        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
-        init_parameters:
-          embedding_dim: 768
-      top_k: 20 # The number of results to return
-
-  document_joiner:
-    type: haystack.components.joiners.document_joiner.DocumentJoiner
-    init_parameters:
-      join_mode: concatenate
-
-  ranker:
-    type: deepset_cloud_custom_nodes.rankers.nvidia.ranker.DeepsetNvidiaRanker
-    init_parameters:
-      model: intfloat/simlm-msmarco-reranker
-      top_k: 8
-
-  meta_field_grouping_ranker:
-    type: haystack.components.rankers.meta_field_grouping_ranker.MetaFieldGroupingRanker
-    init_parameters:
-      group_by: file_id
-      subgroup_by: null
-      sort_docs_by: split_id
-
-  qa_prompt_builder:
-    type: haystack.components.builders.prompt_builder.PromptBuilder
-    init_parameters:
-      template: |-
-        You are a technical expert.
-        You answer questions truthfully based on provided documents.
-        If the answer exists in several documents, summarize them.
-        Ignore documents that don't contain the answer to the question.
-        Only answer based on the documents provided. Don't make things up.
-        If no information related to the question can be found in the document, say so.
-        Always use references in the form [NUMBER OF DOCUMENT] when using information from a document,
-        e.g. [3] for Document [3] .
-        Never name the documents, only enter a number in square brackets as a reference.
-        The reference must only refer to the number that comes in square brackets after the document.
-        Otherwise, do not use brackets in your answer and reference ONLY the number of the document without mentioning
-        the word document.
-
-        These are the documents:
-        {%- if documents|length > 0 %}
-        {%- for document in documents %}
-        Document [{{ loop.index }}] :
-        Name of Source File: {{ document.meta.file_name }}
-        {{ document.content }}
-        {% endfor -%}
-        {%- else %}
-        No relevant documents found.
-        Respond with "Sorry, no matching documents were found, please adjust the filters or try a different question."
-        {% endif %}
-
-        Question: {{ question }}
-        Answer:
-
-      required_variables: "*"
-  qa_llm:
-    type: deepset_cloud_custom_nodes.generators.deepset_amazon_bedrock_generator.DeepsetAmazonBedrockGenerator
-    init_parameters:
-      model: anthropic.claude-3-5-sonnet-20241022-v2:0
-      aws_region_name: us-west-2
-      max_length: 650
-      model_max_length: 200000
-      temperature: 0
-
-  answer_builder:
-    type: deepset_cloud_custom_nodes.augmenters.deepset_answer_builder.DeepsetAnswerBuilder
-    init_parameters:
-      reference_pattern: acm
-
-connections:  # Defines how the components are connected
-- sender: chat_summary_prompt_builder.prompt
-  receiver: chat_summary_llm.prompt
-- sender: chat_summary_llm.replies
-  receiver: replies_to_query.replies
-- sender: replies_to_query.output
-  receiver: bm25_retriever.query
-- sender: replies_to_query.output
-  receiver: query_embedder.text
-- sender: replies_to_query.output
-  receiver: ranker.query
-- sender: replies_to_query.output
-  receiver: qa_prompt_builder.question
-- sender: replies_to_query.output
-  receiver: answer_builder.query
-- sender: bm25_retriever.documents
-  receiver: document_joiner.documents
-- sender: query_embedder.embedding
-  receiver: embedding_retriever.query_embedding
-- sender: embedding_retriever.documents
-  receiver: document_joiner.documents
-- sender: document_joiner.documents
-  receiver: ranker.documents
-- sender: ranker.documents
-  receiver: meta_field_grouping_ranker.documents
-- sender: meta_field_grouping_ranker.documents
-  receiver: qa_prompt_builder.documents
-- sender: meta_field_grouping_ranker.documents
-  receiver: answer_builder.documents
-- sender: qa_prompt_builder.prompt
-  receiver: qa_llm.prompt
-- sender: qa_prompt_builder.prompt
-  receiver: answer_builder.prompt
-- sender: qa_llm.replies
-  receiver: answer_builder.replies
-
-inputs:  # Define the inputs for your pipeline
-  query:  # These components will receive the query as input
-  - "chat_summary_prompt_builder.question"
-
-  filters:  # These components will receive a potential query filter as input
-  - "bm25_retriever.filters"
-  - "embedding_retriever.filters"
-
-outputs:  # Defines the output of your pipeline
-  documents: "meta_field_grouping_ranker.documents"  # The output of the pipeline is the retrieved documents
-  answers: "answer_builder.answers" # The output of the pipeline is the generated answers
-
-### 1.2 Components
-- **Identification**: Each has a unique `type` (fully qualified class path).
-- **Configuration**: `init_parameters` control models, thresholds, credentials, etc.
-- **I/O Signatures**: Named inputs and outputs, with specific data types (e.g., `List[Document]`, `List[Answer]`).
-
-**Component Example**:
-```yaml
-my_converter:
-  type: haystack.components.converters.xlsx.XLSXToDocument
-  init_parameters:
-    metadata_filters: ["*.sheet1"]
-````
-
-**Connection Example**:
-
-```yaml
-- sender: my_converter.documents
-  receiver: text_converter.sources
-```
-
-### 1.3 YAML Structure
-
-1. **components**: Declare each block’s name, `type`, and `init_parameters`.
-2. **connections**: Link `sender:<component>.<output>` → `receiver:<component>.<input>`.
-3. **inputs**: Map external inputs (`query`, `filters`) to component inputs.
-4. **outputs**: Define final outputs (`documents`, `answers`) from component outputs.
-5. **max\_loops\_allowed**: (Optional) Cap on loop iterations.
-
----
-
-## 2. Agent Workflow
-
-1. **Inspect & Discover**
-
-   * Always call listing/fetch tools (`list_pipelines`, `get_component_definition`, etc.) to gather current state.
-   * Check the pipeline templates, oftentimes you can start off of an existing template when the user wants to create a
-        new pipeline.
-   * Ask targeted questions if requirements are unclear.
-2. **Architect Phase**
-
-   * Reason about the changes you will need to make.
-   * Do NOT ask the user for confirmation, go ahead with execution once you know what you need to do.
-
-3. **Execute Phase**
-   * Execute the changes to help the user fix their pipeline or index.
-
-4. **Integrity**
-
-   * Never invent components; rely exclusively on tool-derived definitions.

deepset_mcp/benchmark/README.md

@@ -1,425 +0,0 @@
-# Deepset MCP Benchmark System
-
-A comprehensive benchmarking and testing framework for the Deepset Cloud Platform that enables automated testing of AI agents against predefined test cases.
-
-## Getting Started
-
-### Prerequisites
-
-- Python 3.11+
-- Access to Deepset Cloud Platform
-- Required environment variables:
-  - `DEEPSET_API_KEY`: Your Deepset Cloud API key
-  - `DEEPSET_WORKSPACE`: Your workspace name
-
-### Installation
-
-Install the benchmark dependencies:
-
-```bash
-pip install -e .[benchmark]
-```
-
-For agent testing, also install:
-
-```bash
-pip install -e .[agents]
-```
-
-### Quick Start
-
-1. **Check your environment**:
-   ```bash
-   deepset agent check-env agent_configs/generalist_agent.yml
-   ```
-
-2. **List available test cases**:
-   ```bash
-   deepset test list
-   ```
-
-3. **Run a single test**:
-   ```bash
-   deepset agent run agent_configs/generalist_agent.yml chat_rag_answers_wrong_format
-   ```
-
-## Core Concepts
-
-### Test Cases
-
-Test cases define specific scenarios for testing agents. Each test case includes:
-
-- **Pipeline configuration**: YAML files defining Haystack pipelines
-- **Index configuration**: YAML files for document indexing
-- **Test prompt**: The input message sent to the agent
-- **Validation criteria**: Expected behavior and outputs
-
-Test cases are stored as YAML files in `tasks/` directory.
-
-### Agent Configurations
-
-Agent configurations define how to instantiate and run AI agents. They specify:
-
-- **Agent factory function**: Python function that creates the agent
-- **Environment variables**: Required API keys and settings
-- **Display name**: Human-readable identifier
-
-### Pipelines and Indexes
-
-- **Pipelines**: Define the processing workflow for queries and documents
-- **Indexes**: Configure document storage and retrieval systems
-- Both are managed as YAML configurations on the Deepset platform
-
-## Tutorials
-
-### Running Your First Benchmark
-
-1. **Prepare your environment**:
-   ```bash
-   export DEEPSET_API_KEY="your_api_key"
-   export DEEPSET_WORKSPACE="your_workspace"
-   export ANTHROPIC_API_KEY="your_anthropic_key"
-   ```
-
-2. **Validate your agent configuration**:
-   ```bash
-   deepset agent validate-config agent_configs/generalist_agent.yml
-   ```
-
-3. **Run a single test case**:
-   ```bash
-   deepset agent run agent_configs/generalist_agent.yml chat_rag_answers_wrong_format
-   ```
-
-   This will:
-   - Create necessary pipelines and indexes
-   - Run the agent against the test case
-   - Validate the results
-   - Clean up resources
-   - Save detailed results to disk
-
-4. **View the results**:
-   Results are saved in `agent_runs/` directory with:
-   - Full message transcripts (`messages.json`)
-   - Performance metrics (`test_results.csv`)
-   - Pipeline configurations (`post_run_pipeline.yml`)
-
-### Running Multiple Test Cases
-
-Run all available test cases:
-
-```bash
-deepset agent run-all agent_configs/generalist_agent.yml
-```
-
-With parallel execution:
-
-```bash
-deepset agent run-all agent_configs/generalist_agent.yml --concurrency 3
-```
-
-### Creating Test Cases
-
-1. **Create a test case YAML file** in `tasks/`:
-
-   ```yaml
-   name: "my_test_case"
-   objective: "Test pipeline validation"
-   prompt: "Please check my pipeline configuration"
-   query_yaml: "pipelines/my_pipeline.yml"
-   query_name: "test-pipeline"
-   index_yaml: "pipelines/my_index.yml"
-   index_name: "test-index"
-   tags:
-     - "validation"
-     - "debugging"
-   ```
-
-2. **Create the referenced pipeline files** in `tasks/pipelines/`
-
-3. **Test your new case**:
-   ```bash
-   deepset agent run agent_configs/generalist_agent.yml my_test_case
-   ```
-
-## How-To Guides
-
-### Managing Test Resources
-
-#### Setup Test Cases Manually
-
-Create all test case resources on the platform:
-
-```bash
-deepset test setup-all --workspace your-workspace --concurrency 5
-```
-
-Setup a specific test case:
-
-```bash
-deepset test setup my_test_case --workspace your-workspace
-```
-
-#### Cleanup Test Resources
-
-Remove all test case resources:
-
-```bash
-deepset test teardown-all --workspace your-workspace
-```
-
-Remove a specific test case:
-
-```bash
-deepset test teardown my_test_case --workspace your-workspace
-```
-
-### Managing Pipelines and Indexes
-
-#### Create Individual Resources
-
-Create a pipeline from YAML file:
-
-```bash
-deepset pipeline create --path pipeline.yml --name my-pipeline --workspace your-workspace
-```
-
-Create an index from YAML content:
-
-```bash
-deepset index create --content "$(cat index.yml)" --name my-index --workspace your-workspace
-```
-
-#### Delete Resources
-
-Delete a pipeline:
-
-```bash
-deepset pipeline delete --name my-pipeline --workspace your-workspace
-```
-
-Delete an index:
-
-```bash
-deepset index delete --name my-index --workspace your-workspace
-```
-
-### Environment Configuration
-
-#### Using Environment Files
-
-Create a `.env` file:
-
-```bash
-DEEPSET_API_KEY=your_api_key
-DEEPSET_WORKSPACE=your_workspace
-ANTHROPIC_API_KEY=your_anthropic_key
-```
-
-Use it with any command:
-
-```bash
-deepset agent run --env-file .env agent_configs/generalist_agent.yml test_case
-```
-
-#### Override Settings
-
-Override workspace and API key:
-
-```bash
-deepset agent run agent_configs/generalist_agent.yml test_case \
-  --workspace different-workspace \
-  --api-key different-key
-```
-
-### Custom Output Directories
-
-Specify where to save results:
-
-```bash
-deepset agent run agent_configs/generalist_agent.yml test_case \
-  --output-dir ./my_results
-```
-
-Specify test case directory:
-
-```bash
-deepset agent run agent_configs/generalist_agent.yml test_case \
-  --test-base-dir ./my_test_cases
-```
-
-### Debugging and Monitoring
-
-#### Check Environment Variables
-
-Verify all required environment variables are set:
-
-```bash
-deepset agent check-env agent_configs/generalist_agent.yml
-```
-
-#### Validate Configurations
-
-Check agent configuration syntax:
-
-```bash
-deepset agent validate-config agent_configs/generalist_agent.yml
-```
-
-#### View Test Case Lists
-
-List available test cases:
-
-```bash
-deepset test list --test-dir ./my_test_cases
-```
-
-## Command Reference
-
-### Agent Commands
-
-- `deepset agent run` - Run agent against single test case
-- `deepset agent run-all` - Run agent against all test cases
-- `deepset agent check-env` - Verify environment configuration
-- `deepset agent validate-config` - Validate agent configuration
-
-### Test Management Commands
-
-- `deepset test list` - List available test cases
-- `deepset test setup` - Setup single test case resources
-- `deepset test setup-all` - Setup all test case resources
-- `deepset test teardown` - Remove single test case resources
-- `deepset test teardown-all` - Remove all test case resources
-
-### Pipeline Management Commands
-
-- `deepset pipeline create` - Create new pipeline
-- `deepset pipeline delete` - Delete existing pipeline
-
-### Index Management Commands
-
-- `deepset index create` - Create new index
-- `deepset index delete` - Delete existing index
-
-## Configuration Files
-
-### Agent Configuration Format
-
-```yaml
-agent_factory_function: "module.path.to.get_agent"
-display_name: "My Agent"
-required_env_vars:
-  - DEEPSET_API_KEY
-  - DEEPSET_WORKSPACE
-  - ANTHROPIC_API_KEY
-```
-
-### Test Case Configuration Format
-
-```yaml
-name: "test_case_name"
-objective: "Description of what this test validates"
-prompt: "The message sent to the agent"
-query_yaml: "relative/path/to/pipeline.yml"  # Optional
-query_name: "pipeline-name"                   # Required if query_yaml present
-index_yaml: "relative/path/to/index.yml"     # Optional
-index_name: "index-name"                      # Required if index_yaml present
-expected_query: "path/to/expected.yml"        # Optional validation reference
-tags:
-  - "category"
-  - "type"
-judge_prompt: "Optional prompt for LLM validation"  # Optional
-```
-
-## Result Analysis
-
-### Understanding Output Files
-
-Each test run produces:
-
-1. **`messages.json`**: Complete conversation transcript with the agent
-2. **`test_results.csv`**: Performance metrics and validation results
-3. **`post_run_pipeline.yml`**: Final pipeline configuration after agent modifications
-
-### Performance Metrics
-
-The system tracks:
-
-- **Token usage**: Prompt and completion tokens consumed
-- **Tool calls**: Number of API calls made by the agent
-- **Validation status**: Pre and post-run pipeline validation results
-- **Model information**: Which AI model was used
-
-### Aggregate Analysis
-
-When running multiple test cases, the system provides:
-
-- Success/failure counts
-- Total resource consumption
-- Per-test case breakdowns
-- Cleanup status reports
-
-## Troubleshooting
-
-### Common Issues
-
-**Environment variable errors**:
-- Ensure all required variables are set
-- Use `deepset agent check-env` to verify configuration
-
-**Test case not found**:
-- Check test case directory path
-- Verify YAML file exists and is properly named
-- Use `deepset test list` to see available cases
-
-**Validation failures**:
-- Review pipeline YAML syntax
-- Check component type names and parameters
-- Use Deepset Cloud UI to validate manually
-
-**Resource conflicts**:
-- Ensure unique names for pipelines and indexes
-- Clean up existing resources before running tests
-- Use different workspace for testing
-
-**Permission errors**:
-- Verify API key has sufficient permissions
-- Check workspace access rights
-- Confirm network connectivity to Deepset Cloud
-
-### Getting Help
-
-1. **Check logs**: Review detailed error messages in command output
-2. **Validate configs**: Use validation commands before running tests
-3. **Test incrementally**: Start with single test cases before batch runs
-4. **Clean environment**: Remove conflicting resources and retry
-
-## Best Practices
-
-### Test Organization
-
-- Use descriptive test case names with underscores
-- Group related tests with consistent tag names
-- Keep pipeline files organized in subdirectories
-- Document test objectives clearly
-
-### Resource Management
-
-- Always clean up test resources after experiments
-- Use unique names to avoid conflicts
-- Prefer automated setup/teardown over manual management
-- Monitor resource usage in your workspace
-
-### Performance Optimization
-
-- Use appropriate concurrency levels (start with 1-3)
-- Set reasonable token limits for cost control
-- Cache common pipeline configurations
-- Run expensive tests separately from quick validation tests
-
-### Environment Management
-
-- Use environment files for consistent configuration
-- Never commit API keys to version control
-- Use different workspaces for development and testing
-- Validate environment before important test runs

deepset_mcp/benchmark/__init__.py

@@ -1 +0,0 @@
-

deepset_mcp/benchmark/agent_configs/debugging_agent.yml

@@ -1,10 +0,0 @@
-agent_factory_function: "deepset_mcp.agents.debugging.debugging_agent.get_agent"
-display_name: "debugging-agent"
-interactive: true
-required_env_vars:
-  - DEEPSET_API_KEY
-  - DEEPSET_WORKSPACE
-  - ANTHROPIC_API_KEY
-  - DEEPSET_DOCS_WORKSPACE
-  - DEEPSET_DOCS_API_KEY
-  - DEEPSET_DOCS_PIPELINE_NAME

deepset_mcp/benchmark/agent_configs/generalist_agent.yml

@@ -1,6 +0,0 @@
-agent_factory_function: "deepset_mcp.agents.generalist.generalist_agent.get_agent"
-display_name: "generalist-agent"
-required_env_vars:
-  - DEEPSET_API_KEY
-  - DEEPSET_WORKSPACE
-  - ANTHROPIC_API_KEY