qtype 0.1.12__py3-none-any.whl → 0.1.14__py3-none-any.whl

qtype/docs/skills/architect/SKILL.md

@@ -0,0 +1,188 @@
+---
+name: qtype-architect
+description: Expert assistant for designing and building AI applications using QType's declarative YAML specification language. Helps users architect flows from concept to implementation, validates QType code, generates visualizations, and recommends best practices. Use when users want to build chatbots, RAG systems, agents, data pipelines, or any AI application with QType. Use when the user mentions building AI apps, RAG pipelines, or YAML-based agent orchestration.
+compatibility: Requires the 'qtype-mcp' MCP server to be active for schema lookup and validation.
+metadata:
+  author: qtype
+  version: "1.0"
+allowed-tools: qtype-mcp:*
+---
+
+# QType Architect
+
+Expert assistant for designing and building AI applications using the QType DSL.
+
+## When to use this skill
+
+Activate when users want to:
+- Design QType applications (chatbots, RAG, agents, pipelines)
+- Convert requirements into QType YAML specifications
+- Validate or debug QType code
+- Visualize application architecture
+- Learn QType best practices and patterns
+
+## What is QType?
+
+**QType is a domain-specific language (DSL) for rapid prototyping of AI applications.**
+
+Qtype is a declarative, text-based language that lets you specify *what* your AI application should do, not *how* to do it. You write YAML specifications that describe flows, steps, models, and data transformations, and QType handles the execution.
+
+**Elevator pitch:** QType turns AI application prototypes from days of Python coding into hours of YAML configuration, without sacrificing maintainability or requiring you to learn yet another GUI tool.
+
+## Core Mental Model
+
+Understanding QType requires understanding core concepts and how they relate:
+
+**Think of QType like this:**
+
+**Variables** are your data containers (typed boxes)  
+**Steps** are your transformations (functions on boxes)  
+**Flows** are your pipelines (sequences of transformations)  
+**The DSL** is your specification language (what you write)  
+**The Semantic layer** is your validator (what checks it)  
+**The Interpreter** is your executor (what runs it)  
+
+## Core workflow
+
+### 1. Understand requirements
+
+Clarify the user's goal through targeted questions:
+- What is the overall goal?
+- What are the inputs and outputs?
+- What processing steps are needed?
+- Which models, APIs, or tools are required?
+- Authentication requirements?
+- Conversational vs batch processing?
+
+Prefer inferring reasonable defaults over excessive questions.
+
+### 2. Use MCP tools to research
+
+**CRITICAL: Always use the qtype-mcp server to look up information rather than guessing.**
+
+- Use `list_components` to discover available component types
+- Use `get_component_schema` to get detailed field requirements for specific components
+- Use `search_library` to find relevant documentation and examples
+- Use `list_examples` and `get_example` to see real QType code patterns
+- Use `list_documentation` and `get_documentation` to get specific guides, tutorials, or references
+
+**Don't make assumptions about component fields, syntax, or capabilities - look them up!**
+
+### 3. Design architecture
+
+Map requirements to QType components. For guidance:
+- See [references/patterns.md](references/patterns.md) for common architecture patterns
+- See [references/cheatsheet.md](references/cheatsheet.md) for quick component reference
+- See `assets/*.qtype.yaml` for complete working examples
+
+Use MCP tools to verify component schemas and field requirements as you design.
+
+### 4. Generate and validate
+
+**ALWAYS validate:**
+
+1. Generate complete QType YAML
+2. Call `validate_qtype_yaml` with your YAML
+3. If errors: fix specific issues and validate again
+4. Present validated code to user
+5. If success and you have mermaid preview capability, call `visualize_qtype_architecture` and use the mermaid preview to visalize the result to the user.
+
+**Never skip validation.** Invalid code wastes user time.
+
+### 5. Visualization (Optional)
+
+The `visualize` tool in qtype-mcp converts the qtype yaml to an archiecture diagram.
+If you or the user desires it, use the tool and then open it with either the claude-mermaid mcp (if you are claude code) or the mermaid preview (if you are in vscode).
+If neither of those tools are available, just let the user know and save the mermaid to disk.
+
+### 6. Provide context
+
+After presenting validated code:
+- Explain key architectural decisions
+- Show how to run: `qtype run`, or `qtype validate`, `qtype serve`
+- Point to relevant documentation
+- Suggest next steps or improvements
+
+### 7. Optional Next Steps
+
+If the user desires, or you think they would like it, you can run `qtype serve --reload <yaml_file>` to start the interpreter server. The serve command will print a url, open that in the users web browser so they can see the app.
+
+The `--reload` will refresh each time the yaml is changed.
+
+## Using the qtype-mcp server
+
+**CRITICAL: Use MCP tools constantly - don't guess or rely on memory!**
+
+The qtype-mcp server provides tools for:
+- **Discovery**: Browse components, docs, and examples
+- **Schema lookup**: Get exact field requirements with `get_component_schema`
+- **Search**: Full-text search across all documentation and examples
+- **Validation**: `validate_qtype_yaml` - **Use before every code presentation**
+- **Visualization**: Generate Mermaid diagrams of flows
+- **Code generation**: Convert APIs/Python to QType tools
+
+**Key workflow patterns:**
+1. Unsure about a component? → Use `get_component_schema`
+2. Need an example? → Use `search_library` or `list_examples`
+3. Generated YAML? → **Always call `validate_qtype_yaml`**
+4. Complex flow? → Use `visualize_qtype_architecture`
+
+## Reference materials
+
+See bundled reference files for quick lookup:
+- [references/patterns.md](references/patterns.md) - Common architecture patterns
+- [references/cheatsheet.md](references/cheatsheet.md) - Component quick reference
+- `assets/examples/*.qtype.yaml` for complete working examples.
+
+## Key guidelines
+
+### Self-correction on validation failure
+If validation fails:
+- **Do not guess the fix**
+- Read the error message carefully
+- Call `get_component_schema` for the failing component
+- Compare your YAML against the official schema
+- Make precise corrections based on schema requirements
+
+### Use MCP for everything
+- Component details? → `get_component_schema`
+- Need examples? → `list_examples` + `get_example`
+- Unclear syntax? → `search_library` for relevant docs
+- Best practices? → `get_documentation` for concept guides
+
+### Common pitfalls
+❌ Skipping validation
+❌ Guessing component fields instead of using `get_component_schema`
+❌ Undefined variables
+❌ Missing authentication
+❌ Hardcoding secrets instead of `${ENV_VAR}`
+
+### Respect Virtual Environments
+
+The user is likely running in a virtual environment.  The `qtype` executable may not be in your default `path`.
+
+You may try:
+```
+uv run qtype
+```
+If the user is using `uv`
+
+or `source .venv/bin/activate` before calling `qtype`.
+
+
+## Response format
+
+0. **High-level plan** - Briefly state which components you've chosen and why they fit the user's requirements based on [references/patterns.md](references/patterns.md) and the components listed in the mcp server.
+1. **Brief explanation** (what it does, how data flows)
+2. **Complete validated YAML** (with comments)
+3. **Visualization** (for complex flows)
+4. **Next steps** (`qtype validate`, `qtype run`, `qtype serve`)
+
+## When NOT to use QType
+
+QType doesn't fit:
+- Complex conditional branching (use Python)
+- Pure data pipelines without AI (use Dagster/Airflow)
+- Simple API calls (use SDKs directly)
+
+Be honest if QType isn't the right tool.

qtype/docs/skills/architect/references/cheatsheet.md

@@ -0,0 +1,198 @@
+# Cheatsheet
+
+## Abbreviated Shape Reference
+
+### Application (Root Object)
+```yaml
+id: <string>                    # REQUIRED
+flows: [<Flow>]                 # REQUIRED (at least one)
+models: [<Model>]               # OPTIONAL
+tools: [<Tool>]                 # OPTIONAL
+types: [<CustomType>]           # OPTIONAL
+```
+
+### Flow
+```yaml
+type: Flow
+id: <string>                    # REQUIRED
+steps: [<Step>]                 # REQUIRED (at least one)
+variables: [<Variable>]         # OPTIONAL
+inputs: [<string>]              # OPTIONAL (Variable IDs)
+outputs: [<string>]             # OPTIONAL (Variable IDs)
+interface:                      # OPTIONAL
+  type: <chat|form|agent>
+```
+
+### Variable
+```yaml
+type: Variable
+id: <string>                    # REQUIRED
+value_type: <type>              # REQUIRED (see Type Reference)
+ui:                             # OPTIONAL
+  type: <text-input|text-area|...>
+  label: <string>
+```
+
+**Type Reference:**
+- Primitives: `string`, `int`, `float`, `bool`
+- Collections: `list`, `dict`
+- Built-ins: `ChatMessage`, `Document`, `Embedding`
+- Custom: Any CustomType `id`
+- List syntax: `value_type: list` + `item_type: <type>`
+
+### Model
+```yaml
+type: Model
+id: <string>                    # REQUIRED
+provider: <bedrock|openai|anthropic>  # REQUIRED
+model_id: <string>              # REQUIRED
+auth: <string>                  # OPTIONAL (AuthProvider ID)
+```
+
+### CustomType
+```yaml
+id: <string>                    # REQUIRED
+fields:                         # REQUIRED (at least one)
+  - name: <string>              # REQUIRED
+    field_type: <type>          # REQUIRED
+  - name: <string>
+    field_type: list[<type>]
+```
+
+---
+
+## Abbreviated Step Reference
+
+### LLMInference
+```yaml
+type: LLMInference
+id: <string>                    # REQUIRED
+model: <string>                 # REQUIRED (Model ID)
+inputs: [<string>]              # OPTIONAL (Variable IDs)
+outputs: [<string>]             # OPTIONAL (Variable IDs)
+system_message: <string>        # OPTIONAL
+memory: <string>                # OPTIONAL (Variable ID for chat history)
+```
+
+### Agent
+```yaml
+type: Agent
+id: <string>                    # REQUIRED
+model: <string>                 # REQUIRED (Model ID)
+tools: [<string>]               # REQUIRED (Tool IDs)
+inputs: [<string>]              # OPTIONAL
+outputs: [<string>]             # OPTIONAL
+system_message: <string>        # OPTIONAL
+```
+
+### PromptTemplate
+```yaml
+type: PromptTemplate
+id: <string>                    # REQUIRED
+template: <string>              # REQUIRED (use {{var_id}} syntax)
+inputs: [<string>]              # REQUIRED (Variable IDs to interpolate)
+outputs: [<string>]             # REQUIRED (single Variable ID)
+```
+
+### InvokeFlow
+```yaml
+type: InvokeFlow
+id: <string>                    # REQUIRED
+flow: <string>                  # REQUIRED (Flow ID)
+input_bindings:                 # REQUIRED (even if empty: {})
+  <flow_input_id>: <local_var_id>
+output_bindings:                # REQUIRED (even if empty: {})
+  <local_var_id>: <flow_output_id>
+```
+
+### InvokeTool
+```yaml
+type: InvokeTool
+id: <string>                    # REQUIRED
+tool: <string>                  # REQUIRED (Tool ID)
+input_bindings:                 # REQUIRED
+  <tool_param>: <local_var_id>
+output_bindings:                # REQUIRED
+  <tool_output>: <local_var_id>
+```
+
+### Decoder
+```yaml
+type: Decoder
+id: <string>                    # REQUIRED
+format: <json|xml>              # REQUIRED
+inputs: [<string>]              # REQUIRED (single Variable ID with text)
+outputs: [<string>]             # REQUIRED (single Variable ID)
+```
+
+### Construct
+```yaml
+type: Construct
+id: <string>                    # REQUIRED
+output_type: <string>           # REQUIRED (CustomType ID)
+field_bindings:                 # REQUIRED
+  <field_name>: <var_id>
+outputs: [<string>]             # REQUIRED (single Variable ID)
+```
+
+### VectorSearch
+```yaml
+type: VectorSearch
+id: <string>                    # REQUIRED
+index: <string>                 # REQUIRED (DocumentIndex ID)
+inputs: [<string>]              # REQUIRED (query Variable ID)
+outputs: [<string>]             # REQUIRED (results Variable ID)
+top_k: <int>                    # OPTIONAL (default: 5)
+```
+
+### Explode
+```yaml
+type: Explode
+id: <string>                    # REQUIRED
+inputs: [<string>]              # REQUIRED (list Variable ID)
+outputs: [<string>]             # REQUIRED (single item Variable ID)
+```
+
+### FileSource
+```yaml
+type: FileSource
+id: <string>                    # REQUIRED
+path: <string>                  # REQUIRED
+outputs: [<string>]             # REQUIRED (Variable ID)
+```
+
+## Critical Rules
+
+### ID Rules
+- All `id` fields must be unique within their scope
+- IDs must be valid identifiers (alphanumeric + underscore/hyphen)
+- Reference other objects by their exact `id` string
+
+### Variable Rules
+- Variables in `inputs`/`outputs` must be defined in `variables` list
+- Variables must have `value_type` specified
+- List variables REQUIRE `item_type` field
+- Variable types must match their usage in steps
+
+### Step Rules
+- Steps execute in order within `steps` array
+- Step `inputs` must reference existing variables (from previous steps or flow inputs)
+- Step `outputs` create or update variables for subsequent steps
+- All referenced Model/Tool/Flow/CustomType IDs must exist
+
+### Binding Rules
+- `input_bindings`: Map local variables TO external inputs
+  - Format: `{external_input_name: local_variable_id}`
+- `output_bindings`: Map external outputs TO local variables
+  - Format: `{local_variable_id: external_output_name}`
+- Both must be present even if empty: `{}`
+
+### Template Rules
+- Use `{{variable_id}}` syntax only
+- All variables in template must be in `inputs` list
+- Template must be a single string (use `\n` for newlines)
+
+### Type Rules
+- Decoder outputs must match expected structure
+- Construct `field_bindings` must map to all required CustomType fields
+- List operations (Explode/Collect) require matching list types

qtype/docs/skills/architect/references/patterns.md

@@ -0,0 +1,29 @@
+# QType Examples
+
+## Pattern: Simple Chatbot
+
+Basic conversational interface with memory.
+
+```yaml
+--8<-- "../examples/conversational_ai/simple_chatbot.qtype.yaml"
+```
+
+---
+
+## Pattern: Tool-Based Research
+
+Web search with synthesis using external tools.
+
+```yaml
+--8<-- "../examples/research_assistant/research_assistant.qtype.yaml"
+```
+
+---
+
+## Pattern: Data Pipeline with LLM Processing
+
+Parallel processing of database records with LLM analysis.
+
+```yaml
+--8<-- "../examples/data_processing/dataflow_pipelines.qtype.yaml"
+```

qtype/docs/stylesheets/extra.css

@@ -0,0 +1,27 @@
+/* Custom styles for QType documentation */
+
+/* Increase max width of main content area */
+.md-grid {
+  max-width: 75rem;  /* Default is 61rem (976px), this increases to 1280px */
+}
+
+/* Adjust content width for better readability */
+.md-content {
+  max-width: none;
+}
+
+/* Ensure sidebars remain proportional */
+.md-sidebar--primary {
+  left: 0;
+}
+
+.md-sidebar--secondary {
+  right: 0;
+}
+
+/* Make section headers darker gray */
+.md-nav__title,
+.md-nav__item--section > .md-nav__link {
+  color: #424242 !important;
+  font-weight: 600;
+}

qtype/dsl/linker.py

@@ -291,6 +291,10 @@ def _resolve_list_references(
                 field_value[i] = _resolve_reference(
                     item.ref, type(item), lookup_map
                 )
+            case str():
+                # Resolve string IDs to their objects (e.g., variable IDs to Variables)
+                if item in lookup_map:
+                    field_value[i] = lookup_map[item]
             case BaseModel():
                 _resolve_all_references(item, lookup_map)
 
@@ -309,6 +313,10 @@ def _resolve_dict_references(
         match v:
             case base_types.Reference():
                 field_value[k] = _resolve_reference(v.ref, type(v), lookup_map)
+            case str():
+                # Resolve string IDs to their objects (e.g., variable IDs to Variables)
+                if v in lookup_map:
+                    field_value[k] = lookup_map[v]
             case BaseModel():
                 _resolve_all_references(v, lookup_map)