qtype 0.1.11__py3-none-any.whl → 0.1.13__py3-none-any.whl

qtype/docs/index.md

@@ -0,0 +1,81 @@
+# Overview
+
+**QType is a domain-specific language (DSL) for rapid prototyping of AI applications.**  
+It is designed to help developers define modular, composable AI systems using a structured YAML-based specification. QType supports models, prompts, tools, retrievers, and flow orchestration, and is extensible for code generation or live interpretation.
+## 🚀 Quick Start
+
+Install QType:
+
+```bash
+pip install qtype[interpreter]
+```
+
+Create a file `hello_world.qtype.yaml` that answers a question:
+```yaml
+id: hello_world
+
+models:
+  - type: Model
+    id: gpt-4
+    provider: openai
+    model_id: gpt-4-turbo
+
+flows:
+  - type: Flow
+    id: chat_example
+    description: A simple chat flow with OpenAI
+    interface:
+      type: Conversational
+    variables:
+      - id: user_message
+        type: ChatMessage
+      - id: response
+        type: ChatMessage
+    inputs:
+      - user_message
+    outputs:
+      - response
+    steps:
+      - type: LLMInference
+        id: llm_inference_step
+        model: gpt-4
+        system_message: "You are a helpful assistant."
+        inputs:
+          - user_message
+        outputs:
+          - response
+```
+
+Put your OpenAI API key into your `.env` file:
+```
+echo "OPENAI_API_KEY=sk-..." >> .env
+```
+
+Validate its semantic correctness:
+
+```bash
+qtype validate hello_world.qtype.yaml 
+```
+
+You should see:
+
+```
+INFO: ✅ Schema validation successful.
+INFO: ✅ Model validation successful.
+INFO: ✅ Language validation successful
+INFO: ✅ Semantic validation successful
+```
+
+Launch the interpreter:
+
+```bash
+qtype serve hello_world.qtype.yaml
+```
+
+
+And go to [http://localhost:8000/ui](http://localhost:8000/ui) to see the user interface for your application:
+
+![Example UI](example_ui.png)
+
+
+Check out the [Tutorials](Tutorials/01-first-qtype-application.md) guide to learn more.

qtype/docs/legacy_how_tos/.pages

@@ -0,0 +1,6 @@
+title: How-To Guides
+nav:
+  - Configuration
+  - Data Types
+  - Tools
+  - Debugging

qtype/docs/legacy_how_tos/Configuration/modular-yaml.md

@@ -0,0 +1,366 @@
+# Include References and use Modular YAML
+
+QType supports including external files in your YAML specifications, allowing you to break large configurations into smaller, manageable modules and reuse common components across multiple applications.
+
+## Include Directives
+
+### `!include` - YAML File Inclusion
+
+The `!include` directive loads and parses external YAML files, merging their content into your main specification:
+
+```yaml
+# main.qtype.yaml
+id: main_app
+
+# Include reusable components from other files
+auths: !include common/auth.qtype.yaml
+models: !include common/models.qtype.yaml
+tools: !include common/tools.qtype.yaml
+## Conditional References
+```
+
+```yaml
+# common/models.qtype.yaml
+- id: shared_gpt4
+  provider: openai
+  auth: shared_openai_auth
+  model_id: gpt-4
+  
+- id: shared_claude
+  provider: anthropic
+  auth: shared_anthropic_auth
+  model_id: claude-3-5-sonnet-20241022
+```
+
+```yaml
+# common/auth.qtype.yaml
+- id: shared_openai_auth
+  type: api_key
+  api_key: ${OPENAI_KEY}
+
+- id: shared_anthropic_auth
+  type: api_key
+  api_key: ${ANTHROPIC_KEY}
+```
+
+### `!include_raw` - Raw Text Inclusion
+
+The `!include_raw` directive loads external text files as raw strings, useful for prompts, templates, or documentation:
+
+```yaml
+# Application with external prompt files
+id: prompt_app
+
+flows:
+  - id: chat_flow
+    steps:
+      - id: system_setup
+        model: gpt-4
+        system_message: !include_raw prompts/expert_assistant.txt
+        
+      - id: user_interaction
+        model: gpt-4
+        template: !include_raw templates/response_format.txt
+```
+
+```txt
+# prompts/expert_assistant.txt
+You are an expert AI assistant with deep knowledge across multiple domains.
+Always provide accurate, helpful, and well-structured responses.
+
+When answering questions:
+
+1. Be concise but thorough
+2. Provide examples when helpful
+3. Cite sources when making factual claims
+4. Ask for clarification if the question is ambiguous
+```
+
+```txt
+# templates/response_format.txt
+Based on the user's question: "{user_question}s"
+
+Please provide a response in the following format:
+- **Summary**: Brief answer to the question
+- **Details**: More comprehensive explanation
+- **Examples**: Relevant examples if applicable
+- **Next Steps**: Suggested follow-up actions
+```
+
+## File Path Resolution
+
+QType supports multiple path types and protocols:
+
+### Local File Paths
+
+```yaml
+# Relative paths (relative to the current YAML file)
+models: !include ../shared/models.yaml
+prompts: !include_raw ./prompts/system.txt
+
+# Absolute paths
+config: !include /etc/qtype/global-config.yaml
+```
+
+### Remote URLs
+
+```yaml
+# HTTP/HTTPS URLs
+shared_config: !include https://config.example.com/qtype/base.yaml
+prompt_library: !include_raw https://prompts.example.com/expert-system.txt
+
+# GitHub URLs (via fsspec)
+tools: !include github://company/qtype-configs/main/tools.yaml
+```
+
+### Cloud Storage
+
+```yaml
+# S3 URLs
+production_config: !include s3://company-configs/qtype/prod.yaml
+training_data: !include_raw s3://data-bucket/prompts/training.txt
+```
+
+## Environment-Specific Configurations
+
+Use file inclusion to manage different environments:
+
+```yaml
+# base.qtype.yaml - Common configuration
+id: my_app
+
+# Include environment-specific overrides
+models: !include environments/${ENV:-development}/models.yaml
+auths: !include environments/${ENV:-development}/auth.yaml
+
+flows:
+  - id: main_flow
+    steps:
+      - id: llm_step
+        model: primary_model
+        system_message: !include_raw prompts/base_system.txt
+```
+
+```yaml
+# environments/development/models.yaml
+- id: primary_model
+  provider: openai
+  model_id: gpt-3.5-turbo  # Cheaper for development
+  auth: dev_openai_auth
+
+- id: secondary_model
+  provider: anthropic
+  model_id: claude-3-haiku-20240307  # Fast for testing
+  auth: dev_anthropic_auth
+```
+
+```yaml
+# environments/production/models.yaml
+- id: primary_model
+  provider: openai
+  model_id: gpt-4o  # Best quality for production
+  auth: prod_openai_auth
+  inference_params:
+    temperature: 0.1
+    max_tokens: 4000
+
+- id: secondary_model
+  provider: anthropic
+  model_id: claude-3-5-sonnet-20241022  # High quality fallback
+  auth: prod_anthropic_auth
+```
+
+## Nested Inclusions
+
+Files can include other files, creating a hierarchy of modular components:
+
+```yaml
+# main.qtype.yaml
+id: complex_app
+flows: !include flows/index.yaml
+```
+
+```yaml
+# flows/index.yaml
+- id: data_processing
+  description: !include_raw descriptions/data_processing.md
+  steps: !include flows/data_processing_steps.yaml
+
+- id: user_interaction  
+  description: !include_raw descriptions/user_interaction.md
+  steps: !include flows/user_interaction_steps.yaml
+```
+
+```yaml
+# flows/data_processing_steps.yaml
+- id: extract_step
+  tools: !include ../tools/extraction.yaml
+  
+- id: transform_step
+  model: !include ../models/transformation_model.yaml
+  template: !include_raw ../templates/transform_prompt.txt
+
+- id: load_step
+  tools: !include ../tools/database.yaml
+```
+
+## Component Libraries
+
+Create reusable component libraries:
+
+```yaml
+# libraries/ai-models.yaml
+- id: gpt4_creative
+  provider: openai
+  model_id: gpt-4
+  auth: openai_auth
+  inference_params:
+    temperature: 0.8
+    max_tokens: 2000
+
+- id: gpt4_analytical  
+  provider: openai
+  model_id: gpt-4
+  auth: openai_auth
+  inference_params:
+    temperature: 0.1
+    max_tokens: 4000
+
+- id: claude_creative
+  provider: anthropic
+  model_id: claude-3-5-sonnet-20241022
+  auth: anthropic_auth
+  inference_params:
+    temperature: 0.7
+    max_tokens: 8000
+```
+
+```yaml
+# libraries/common-tools.yaml
+- id: web_search
+  name: search_web
+  description: Search the web for current information
+  endpoint: https://api.search.com/v1/search
+  method: GET
+  auth: search_api_auth
+  
+- id: email_sender
+  name: send_email
+  description: Send emails via SMTP
+  function_name: send_email
+  module_path: qtype.commons.email
+```
+
+```yaml
+# applications/research-assistant.yaml
+id: research_assistant
+
+# Import component libraries
+models: !include ../libraries/ai-models.yaml
+tools: !include ../libraries/common-tools.yaml
+
+flows:
+  - id: research_flow
+    steps:
+      - id: search_step
+        tools:
+          - web_search  # From imported library
+          
+      - id: analysis_step
+        model: gpt4_analytical  # From imported library
+        system_message: !include_raw prompts/research_analysis.txt
+```
+
+## Best Practices
+
+### 1. Organize by Logical Groupings
+
+```
+project/
+├── qtype/
+│   ├── main.qtype.yaml
+│   ├── auth/
+│   │   ├── development.yaml
+│   │   └── production.yaml
+│   ├── models/
+│   │   ├── openai.yaml
+│   │   └── anthropic.yaml
+│   ├── tools/
+│   │   ├── api-tools.yaml
+│   │   └── python-tools.yaml
+│   └── prompts/
+│       ├── system-messages/
+│       └── templates/
+```
+
+### 2. Use Consistent Naming Conventions
+
+```yaml
+# ✅ Clear, consistent naming
+models: !include models/openai-models.yaml
+tools: !include tools/api-tools.yaml
+prompts: !include_raw prompts/system-messages/expert.txt
+
+# ❌ Inconsistent naming  
+models: !include models.yaml
+tools: !include tool_definitions.yml
+prompts: !include_raw prompt.txt
+```
+
+### 3. Document File Dependencies
+
+```yaml
+# main.qtype.yaml
+# Dependencies:
+# - auth/production.yaml (auth providers)
+# - models/openai.yaml (LLM models)
+# - tools/api-tools.yaml (external API tools)
+# - prompts/system.txt (system message)
+
+id: production_app
+auths: !include auth/production.yaml
+models: !include models/openai.yaml
+tools: !include tools/api-tools.yaml
+
+flows:
+  - id: main_flow
+    steps:
+      - model: production_gpt4
+        system_message: !include_raw prompts/system.txt
+```
+
+## Security Considerations
+
+### 1. Path Traversal Protection
+QType resolves paths relative to the including file, preventing unauthorized access:
+
+```yaml
+# Safe - resolves relative to current file
+config: !include ../shared/config.yaml
+
+# Potentially unsafe - absolute paths should be used carefully
+config: !include /etc/passwd  # Will fail with appropriate error
+```
+
+### 2. Environment Variable Security
+Use environment variables for sensitive data in included files:
+
+```yaml
+# auth.yaml - Don't commit secrets
+- id: production_auth
+  type: api_key
+  api_key: ${PROD_API_KEY}  # Load from environment
+```
+
+### 3. URL Validation
+Remote URLs are validated and must use secure protocols where appropriate:
+
+```yaml
+# ✅ Secure HTTPS
+config: !include https://secure-config.example.com/config.yaml
+
+# ⚠️ HTTP should be avoided for sensitive configs
+config: !include http://config.example.com/config.yaml
+```
+
+File inclusion makes QType specifications more modular, maintainable, and suitable for complex, multi-environment deployments while keeping sensitive information secure and configurations organized.

qtype/docs/legacy_how_tos/Configuration/reference-by-id.md

@@ -0,0 +1,251 @@
+# Referencing Entities by ID
+
+One of QType's core principles is the ability to define reusable components once and reference them by their unique ID throughout your application. This promotes modularity, reduces duplication, and makes your specifications easier to maintain and validate.
+
+## How It Works
+
+QType uses a **"define once, reference by ID"** pattern for most components. Components are defined at the application level with a unique ID, then referenced by that ID (as a string) wherever they're needed.
+
+The QType validation and resolution system automatically resolves ID references to their actual object definitions, ensuring all components are properly connected and that all references are valid.
+
+## Basic Reference Pattern
+
+### Models
+
+Models are defined at the application level and referenced by ID:
+
+```yaml
+# ✅ Clean reference-based approach
+id: my_app
+
+models:
+  - type: Model  # Type discriminator required
+    id: gpt-4
+    provider: openai
+    model_id: gpt-4-turbo
+
+flows:
+  - type: Flow
+    id: my_flow
+    steps:
+      - type: LLMInference
+        id: step1
+        model: gpt-4  # Reference model by ID (string)
+      - type: LLMInference
+        id: step2
+        model: gpt-4  # Reuse the same model
+```
+
+### Authorization Providers
+
+Authorization providers follow the same pattern:
+
+```yaml
+authorization_providers:
+  - type: APIKeyAuthProvider
+    id: openai_auth
+    api_key: ${OPENAI_API_KEY}
+
+models:
+  - type: Model
+    id: gpt-4
+    provider: openai
+    auth: openai_auth  # Reference auth by ID
+```
+
+### Memory
+
+Memory configurations are centralized and referenced by ID:
+
+```yaml
+memories:
+  - id: chat_memory
+    token_limit: 50000
+
+flows:
+  - type: Flow
+    id: chat_flow
+    steps:
+      - type: LLMInference
+        model: gpt-4
+        memory: chat_memory  # Reference memory by ID
+```
+
+### Variables
+
+Variables are declared in the flow's `variables` section and referenced by ID in step inputs/outputs:
+
+```yaml
+flows:
+  - type: Flow
+    id: process_question
+    variables:
+      - id: user_question
+        type: text
+      - id: llm_response
+        type: text
+      - id: formatted_output
+        type: text
+    inputs:
+      - user_question
+    outputs:
+      - formatted_output
+    steps:
+      - type: LLMInference
+        id: llm_step
+        model: gpt-4
+        inputs:
+          - user_question  # Reference by ID
+        outputs:
+          - llm_response   # Reference by ID
+
+      - type: PromptTemplate
+        id: format_step
+        template: "Response: {llm_response}"
+        inputs:
+          - llm_response   # Reuse the same variable
+        outputs:
+          - formatted_output
+```
+
+### Tools
+
+Tools are defined at the application level and referenced by ID:
+
+```yaml
+tools:
+  - type: PythonFunctionTool
+    id: calculator
+    name: calculate
+    function_name: calculate
+    module_path: my_tools
+
+flows:
+  - type: Flow
+    id: my_flow
+    steps:
+      - type: InvokeTool
+        tool: calculator  # Reference by ID
+      - type: Agent
+        model: gpt-4
+        tools:
+          - calculator  # Can also be used by agents
+```
+
+## Advanced Reference Patterns
+
+### Custom Types and Complex Data Structures
+
+```yaml
+id: type_reference_example
+
+# Define reusable custom types
+types:
+  - id: Person
+    properties:
+      name: text
+      age: int
+      email: text
+  - id: PersonList
+    properties:
+      items: list[Person]
+
+# Define variables using custom types
+variables:
+  - id: current_user
+    type: Person  # Reference custom type by ID
+  - id: all_users
+    type: PersonList  # Reference array type by ID
+
+flows:
+  - id: user_management
+    steps:
+      - id: get_user
+        inputs:
+          - id: user_id
+            type: text
+        outputs:
+          - current_user  # Uses Person type
+
+      - id: list_users
+        outputs:
+          - all_users  # Uses PersonList type
+```
+
+## Best Practices
+
+### 1. Define Components at the Appropriate Level
+- **Application-level**: Components used across multiple flows
+- **Flow-level**: Components specific to one flow
+- **Step-level**: Simple, one-off configurations
+
+### 2. Use Descriptive IDs
+```yaml
+# ✅ Clear and descriptive
+models:
+  - id: openai_gpt4_chat
+  - id: anthropic_claude_reasoning
+
+# ❌ Unclear
+models:
+  - id: model1
+  - id: m2
+```
+
+### 3. Group Related Components
+```yaml
+# ✅ Well-organized
+auths:
+  - id: openai_auth
+  - id: anthropic_auth
+
+models:
+  - id: gpt4_model
+    auth: openai_auth
+  - id: claude_model  
+    auth: anthropic_auth
+```
+
+### 4. Leverage References for Configuration Management
+```yaml
+# Different environments can reference different components
+# development.qtype.yaml
+models:
+  - id: main_model
+    provider: openai
+    model_id: gpt-3.5-turbo  # Cheaper for dev
+
+# production.qtype.yaml  
+models:
+  - id: main_model
+    provider: openai
+    model_id: gpt-4o         # Better for prod
+```
+
+## Validation and Error Handling
+
+QType's validation system ensures:
+
+1. **Unique IDs**: No duplicate component IDs within the same scope
+2. **Valid References**: All ID references resolve to actual components
+3. **Type Safety**: Referenced components match expected types
+
+Common validation errors:
+
+```yaml
+# ❌ This will fail - duplicate ID
+models:
+  - id: gpt-4
+    provider: openai
+  - id: gpt-4  # Error: Duplicate ID
+    provider: anthropic
+
+# ❌ This will fail - missing reference  
+flows:
+  - id: my_flow
+    steps:
+      - model: nonexistent_model  # Error: Reference not found
+```
+
+The reference-by-ID system makes QType specifications more maintainable, reusable, and easier to understand by eliminating duplication and creating clear component relationships.
+