thothctl 0.22.2__tar.gz → 0.23.0__tar.gz

{thothctl-0.22.2 → thothctl-0.23.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: thothctl
-Version: 0.22.2
+Version: 0.23.0
 Summary: A CLI for Developer Control Plane. Accelerate your cloud IaC deployments.
 Project-URL: SourceCode, https://github.com/thothforge/thothctl
 Project-URL: HomePage, https://github.com/thothforge/thothctl

thothctl-0.23.0/docs/framework/commands/document/mermaid_examples.md

@@ -0,0 +1,350 @@
+# Mermaid Dependency Graphs - Examples
+
+## Overview
+
+ThothCTL generates professional Mermaid diagrams for Terragrunt projects, showing module dependencies with explicit input/output relationships.
+
+## Basic Usage
+
+```bash
+# Generate Mermaid diagram
+thothctl document iac --framework terragrunt --graph-type mermaid
+
+# Output: graph.mmd in current directory
+```
+
+## Example 1: Simple Two-Module Stack
+
+### Project Structure
+```
+stacks/
+├── vpc/
+│   └── terragrunt.hcl
+└── security-groups/
+    └── terragrunt.hcl
+```
+
+### vpc/terragrunt.hcl
+```hcl
+terraform {
+  source = "git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git?ref=v5.0.0"
+}
+
+inputs = {
+  name = "my-vpc"
+  cidr = "10.0.0.0/16"
+}
+```
+
+### security-groups/terragrunt.hcl
+```hcl
+terraform {
+  source = "git::https://github.com/terraform-aws-modules/terraform-aws-security-group.git?ref=v5.0.0"
+}
+
+dependency "vpc" {
+  config_path = "../vpc"
+  
+  mock_outputs = {
+    vpc_id = "vpc-mock-12345"
+    subnet_ids = ["subnet-1", "subnet-2"]
+  }
+}
+
+inputs = {
+  vpc_id = dependency.vpc.outputs.vpc_id
+  name = "my-sg"
+}
+```
+
+### Generated Diagram
+
+```bash
+cd stacks
+thothctl document iac -f terragrunt --graph-type mermaid
+```
+
+**Output** (`graph.mmd`):
+```mermaid
+%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#e3f2fd','primaryTextColor':'#1565c0','primaryBorderColor':'#1976d2','lineColor':'#42a5f5','secondaryColor':'#fff3e0','tertiaryColor':'#f3e5f5','fontSize':'14px'}}}%%
+graph LR
+    vpc["<b>vpc</b>"]:::rootNode
+    security_groups["<b>security-groups</b><br/><small>📥 vpc: vpc_id, subnet_ids</small>"]:::normalNode
+    security_groups -->|vpc_id, subnet_ids| vpc
+
+    classDef rootNode fill:#4caf50,stroke:#2e7d32,stroke-width:3px,color:#fff
+    classDef normalNode fill:#2196f3,stroke:#1565c0,stroke-width:2px,color:#fff
+    classDef complexNode fill:#ff9800,stroke:#e65100,stroke-width:2px,color:#fff
+```
+
+**Visual Result**:
+- 🟢 **vpc** (green) - Root module with no dependencies
+- 🔵 **security-groups** (blue) - Normal module with 1 dependency
+- Edge shows: `vpc_id, subnet_ids` - the actual inputs being passed
+
+## Example 2: Multi-Tier Application
+
+### Project Structure
+```
+stacks/
+├── vpc/
+├── rds/
+├── security-groups/
+└── app/
+```
+
+### rds/terragrunt.hcl
+```hcl
+dependency "vpc" {
+  config_path = "../vpc"
+  
+  mock_outputs = {
+    vpc_id = "vpc-123"
+    subnet_ids = ["subnet-1", "subnet-2"]
+    database_subnet_group = "db-subnet-group"
+  }
+}
+
+inputs = {
+  vpc_id = dependency.vpc.outputs.vpc_id
+  subnet_ids = dependency.vpc.outputs.subnet_ids
+}
+```
+
+### app/terragrunt.hcl
+```hcl
+dependency "vpc" {
+  config_path = "../vpc"
+  
+  mock_outputs = {
+    vpc_id = "vpc-123"
+  }
+}
+
+dependency "rds" {
+  config_path = "../rds"
+  
+  mock_outputs = {
+    endpoint = "db.example.com"
+    port = 5432
+  }
+}
+
+dependency "sg" {
+  config_path = "../security-groups"
+  
+  mock_outputs = {
+    security_group_id = "sg-123"
+  }
+}
+
+inputs = {
+  vpc_id = dependency.vpc.outputs.vpc_id
+  db_endpoint = dependency.rds.outputs.endpoint
+  security_group_id = dependency.sg.outputs.security_group_id
+}
+```
+
+### Generated Diagram
+
+```mermaid
+%%{init: {'theme':'base', 'themeVariables': {...}}}%%
+graph LR
+    vpc["<b>vpc</b>"]:::rootNode
+    rds["<b>rds</b><br/><small>📥 vpc: vpc_id, subnet_ids...</small>"]:::normalNode
+    security_groups["<b>security-groups</b><br/><small>📥 vpc: vpc_id</small>"]:::normalNode
+    app["<b>app</b><br/><small>📥 vpc: vpc_id<br/>📥 rds: endpoint, port</small><br/><small>+1 more...</small>"]:::complexNode
+    
+    rds -->|vpc_id, subnet_ids...| vpc
+    security_groups -->|vpc_id| vpc
+    app -->|vpc_id| vpc
+    app -->|endpoint, port| rds
+    app -->|security_group_id| security_groups
+
+    classDef rootNode fill:#4caf50,stroke:#2e7d32,stroke-width:3px,color:#fff
+    classDef normalNode fill:#2196f3,stroke:#1565c0,stroke-width:2px,color:#fff
+    classDef complexNode fill:#ff9800,stroke:#e65100,stroke-width:2px,color:#fff
+```
+
+**Visual Result**:
+- 🟢 **vpc** (green) - Root module
+- 🔵 **rds, security-groups** (blue) - Normal modules (1-2 dependencies)
+- 🟠 **app** (orange) - Complex module (3+ dependencies)
+
+## Example 3: Running from Subdirectory
+
+```bash
+cd stacks/app
+thothctl document iac -f terragrunt --graph-type mermaid
+```
+
+**Output** shows relative paths:
+```mermaid
+graph LR
+    app["<b>app</b><br/><small>📥 vpc: vpc_id<br/>📥 rds: endpoint, port</small><br/><small>+1 more...</small>"]:::complexNode
+    ..vpc["<b>..vpc</b>"]:::rootNode
+    ..rds["<b>..rds</b><br/><small>📥 vpc: vpc_id, subnet_ids...</small>"]:::normalNode
+    ..security_groups["<b>..security-groups</b><br/><small>📥 vpc: vpc_id</small>"]:::normalNode
+    
+    app -->|vpc_id| ..vpc
+    app -->|endpoint, port| ..rds
+    app -->|security_group_id| ..security_groups
+```
+
+## Node Color Coding
+
+| Color | Meaning | Dependencies |
+|-------|---------|--------------|
+| 🟢 Green | Root node | 0 dependencies |
+| 🔵 Blue | Normal node | 1-2 dependencies |
+| 🟠 Orange | Complex node | 3+ dependencies |
+
+## Edge Labels
+
+Edge labels show the **actual input keys** from `mock_outputs`:
+
+```hcl
+dependency "vpc" {
+  mock_outputs = {
+    vpc_id = "vpc-123"
+    subnet_ids = ["subnet-1"]
+  }
+}
+```
+
+Results in edge label: `vpc_id, subnet_ids`
+
+### Truncation
+
+If more than 2 keys, the label is truncated:
+- Shows first 2 keys
+- Adds "..." to indicate more
+
+Example: `vpc_id, subnet_ids...`
+
+## Node Labels
+
+Node labels show:
+1. **Bold module name**
+2. **Dependency details** (if any):
+   - 📥 emoji indicator
+   - Dependency name
+   - Input keys (up to 3)
+3. **Overflow indicator** (if more than 2 dependencies)
+
+Example:
+```
+<b>app</b>
+📥 vpc: vpc_id
+📥 rds: endpoint, port
++1 more...
+```
+
+## Rendering
+
+### GitHub/GitLab
+Mermaid diagrams render automatically in:
+- README.md files
+- Pull request descriptions
+- Issue comments
+- Wiki pages
+
+### VS Code
+Install extension: "Markdown Preview Mermaid Support"
+
+### Mermaid Live Editor
+https://mermaid.live/ - Paste and preview
+
+### Export to Image
+```bash
+# Install mermaid-cli
+npm install -g @mermaid-js/mermaid-cli
+
+# Generate PNG
+mmdc -i graph.mmd -o graph.png
+
+# Generate SVG
+mmdc -i graph.mmd -o graph.svg
+```
+
+## Comparison: Mermaid vs SVG
+
+| Feature | Mermaid | SVG |
+|---------|---------|-----|
+| **Format** | Text | Binary |
+| **Version Control** | Easy to diff | Hard to diff |
+| **Editability** | Edit text file | Regenerate only |
+| **Rendering** | GitHub/GitLab | Browser/viewer |
+| **Input Details** | Shows on edges | Not shown |
+| **Complexity Colors** | Yes | No |
+| **File Size** | Small (~1KB) | Larger (~10KB+) |
+
+## Best Practices
+
+1. **Use Mermaid for documentation**: Better for README files and wikis
+2. **Use SVG for presentations**: Better for slides and reports
+3. **Commit both formats**: Provide options for different use cases
+4. **Update on changes**: Regenerate when dependencies change
+5. **Add to CI/CD**: Automate diagram generation
+
+## CI/CD Integration
+
+### GitHub Actions
+```yaml
+name: Update Dependency Graphs
+
+on:
+  push:
+    paths:
+      - '**/terragrunt.hcl'
+
+jobs:
+  update-graphs:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      
+      - name: Install ThothCTL
+        run: pip install thothctl
+        
+      - name: Generate Mermaid Diagrams
+        run: |
+          cd stacks
+          thothctl document iac -f terragrunt --graph-type mermaid
+        
+      - name: Commit Diagrams
+        uses: stefanzweifel/git-auto-commit-action@v4
+        with:
+          commit_message: "docs: Update dependency graphs"
+          file_pattern: "**/graph.mmd"
+```
+
+## Troubleshooting
+
+### No Edge Labels
+**Issue**: Edges show no input keys
+
+**Causes**:
+1. No `mock_outputs` in dependency block
+2. Dependency name doesn't match module name
+3. Empty `mock_outputs = {}`
+
+**Solution**: Add mock_outputs to dependency blocks
+
+### Wrong Module Names
+**Issue**: Shows "." or "../module" instead of clean names
+
+**Solution**: This is expected when running from subdirectories. Run from project root for clean names.
+
+### Colors Not Showing
+**Issue**: All nodes same color
+
+**Cause**: Rendering platform doesn't support Mermaid themes
+
+**Solution**: Use Mermaid Live Editor or GitHub/GitLab
+
+## Additional Resources
+
+- [Mermaid Documentation](https://mermaid.js.org/)
+- [Terragrunt Dependencies](https://terragrunt.gruntwork.io/docs/features/execute-terraform-commands-on-multiple-modules-at-once/#dependencies-between-modules)
+- [ThothCTL Documentation](https://github.com/thothforge/thothctl)

thothctl-0.23.0/docs/framework/commands/document/mermaid_quickref.md

@@ -0,0 +1,72 @@
+# Mermaid Graphs - Quick Reference
+
+## Command
+
+```bash
+thothctl document iac --framework terragrunt --graph-type mermaid
+```
+
+## Output
+
+`graph.mmd` - Text-based Mermaid diagram
+
+## Features
+
+| Feature | Description |
+|---------|-------------|
+| **Edge Labels** | Shows input keys from `mock_outputs` |
+| **Color Coding** | 🟢 Root, 🔵 Normal, 🟠 Complex |
+| **Dependency Details** | Lists inputs in node labels |
+| **Professional Styling** | ThothCTL brand colors |
+| **Version Control** | Text-based, easy to diff |
+| **Renders Everywhere** | GitHub, GitLab, VS Code |
+
+## Example
+
+### Input (terragrunt.hcl)
+```hcl
+dependency "vpc" {
+  config_path = "../vpc"
+  mock_outputs = {
+    vpc_id = "vpc-123"
+    subnet_ids = ["subnet-1"]
+  }
+}
+```
+
+### Output (graph.mmd)
+```mermaid
+graph LR
+    app["<b>app</b><br/><small>📥 vpc: vpc_id, subnet_ids</small>"]:::normalNode
+    vpc["<b>vpc</b>"]:::rootNode
+    app -->|vpc_id, subnet_ids| vpc
+```
+
+## Color Guide
+
+- 🟢 **Green** = 0 dependencies (root)
+- 🔵 **Blue** = 1-2 dependencies (normal)
+- 🟠 **Orange** = 3+ dependencies (complex)
+
+## Quick Tips
+
+✅ Run from project root for clean names  
+✅ Commit `graph.mmd` to version control  
+✅ Renders automatically in GitHub/GitLab  
+✅ Use for architecture documentation  
+✅ Shows explicit data flow between modules  
+
+## Troubleshooting
+
+**No edge labels?**  
+→ Add `mock_outputs` to dependency blocks
+
+**Shows "." instead of module name?**  
+→ Expected from subdirectories, or run from root
+
+**Want SVG instead?**  
+→ Use `--graph-type dot`
+
+## More Info
+
+See: `docs/framework/commands/document/mermaid_examples.md`

thothctl-0.23.0/docs/framework/commands/generate/generate_iac.md

@@ -0,0 +1,227 @@
+# Generate IaC — Intent-to-Infrastructure
+
+Generate governed Infrastructure as Code from natural language descriptions. ThothCTL reads your organizational conventions (`.thothcf.toml`, steering docs, existing patterns) and produces compliant code validated by Checkov and OPA.
+
+## Quick Start
+
+```bash
+# Generate a VPC (dry-run by default — shows code without writing)
+thothctl generate iac -i "VPC with 3 private subnets and NAT gateway for production"
+
+# Write to disk
+thothctl generate iac -i "EKS cluster with managed node groups" --apply -o ./stacks/platform/eks
+
+# Use AWS Bedrock instead of local Ollama
+thothctl generate iac -i "S3 bucket with versioning and encryption" -p bedrock
+```
+
+## How It Works
+
+```
+Your intent (natural language)
+       ↓
+1. Load Context — reads .thothcf.toml, steering docs, existing patterns, OPA policies
+       ↓
+2. Generate — AI produces IaC code following your org conventions
+       ↓
+3. Validate — Checkov + OPA scan the generated code
+       ↓
+4. Self-Correct — if violations found, AI fixes them (up to 3 attempts)
+       ↓
+5. Output — display (dry-run) or write to disk (--apply)
+```
+
+The AI produces compliant code because your organizational rules are injected directly into its context — not because of a separate policy engine. Existing Checkov and OPA validate the output as a safety net.
+
+## Options
+
+| Flag | Short | Default | Description |
+|------|-------|---------|-------------|
+| `--intent` | `-i` | (required) | What infrastructure to create |
+| `--project-type` | `-pt` | auto | `terraform`, `terraform-terragrunt`, `terragrunt`, `cloudformation`, `cdkv2` |
+| `--provider` | `-p` | ollama | AI provider: `ollama`, `bedrock`, `openai`, `azure` |
+| `--model` | `-m` | (provider default) | Model override (e.g., `llama3`, `claude-sonnet-4-20250514`) |
+| `--output-dir` | `-o` | current dir | Where to write generated files |
+| `--dry-run / --no-dry-run` | | dry-run | Preview without writing |
+| `--apply` | | off | Write files to disk |
+| `--self-correct / --no-self-correct` | | on | Fix validation violations automatically |
+| `--max-iterations` | | 3 | Maximum self-correction attempts |
+| `--skip-validation` | | off | Skip Checkov/OPA (trust AI output) |
+| `--include-diagram / --no-diagram` | | on | Generate architecture diagram |
+
+## Examples
+
+### Basic VPC (Terragrunt)
+
+```bash
+thothctl generate iac \
+  -i "VPC with 3 private subnets, 3 public subnets, NAT gateway per AZ, flow logs enabled" \
+  -pt terraform-terragrunt \
+  --apply -o ./stacks/foundation/network/vpc
+```
+
+Generates:
+```
+stacks/foundation/network/vpc/
+├── terragrunt.hcl    # include root, dependency blocks, inputs
+├── main.tf           # terraform-aws-modules/vpc/aws with flow logs
+├── variables.tf      # cidr, environment, tags
+└── outputs.tf        # vpc_id, subnet_ids, nat_gateway_ids
+```
+
+### EKS Cluster (Terraform)
+
+```bash
+thothctl generate iac \
+  -i "Production EKS cluster with managed node groups, Karpenter autoscaler, and IRSA" \
+  -pt terraform \
+  -p bedrock -m claude-sonnet-4-20250514 \
+  --apply -o ./modules/eks
+```
+
+### S3 with Security (validates automatically)
+
+```bash
+thothctl generate iac \
+  -i "S3 bucket for application logs with encryption, versioning, lifecycle rules, and no public access"
+```
+
+The self-correction loop ensures:
+- ✅ Server-side encryption enabled (CKV_AWS_145)
+- ✅ Public access blocked (CKV_AWS_53, CKV_AWS_54, CKV_AWS_55, CKV_AWS_56)
+- ✅ Versioning enabled (CKV_AWS_21)
+- ✅ Logging configured (CKV_AWS_18)
+
+### CloudFormation
+
+```bash
+thothctl generate iac \
+  -i "Application Load Balancer with HTTPS listener and WAF" \
+  -pt cloudformation \
+  --apply -o ./stacks/application/web-tier.yaml
+```
+
+### Skip Validation (fast iteration)
+
+```bash
+thothctl generate iac \
+  -i "CloudWatch dashboard with CPU, memory, and request metrics" \
+  --skip-validation --apply
+```
+
+## Context Sources
+
+ThothCTL compiles organizational context from these sources (in priority order):
+
+| Source | What's Used | Token Budget |
+|--------|-------------|-------------|
+| `.thothcf.toml` | Project type, naming patterns, environment, tags | ~500 |
+| `.kiro/steering/iac-rules.md` or `.claude/rules/*.md` | IaC composition rules, module preferences, prohibited practices | ~2000 |
+| `.kiro/steering/product.md` or `CLAUDE.md` | Project purpose, architecture layers | ~500 |
+| Existing files in `stacks/` | Real examples from your project (few-shot) | ~2000 |
+| `policies/*.rego` or `THOTH_ORG_POLICY` | OPA rule names and descriptions | ~500 |
+
+**Total context: ~5,500 tokens** — leaves room for generation within most model limits.
+
+### No Context? No Problem
+
+If none of these files exist, the command still works — it generates standard Terraform following AWS best practices. Context just makes the output match *your* conventions.
+
+## AI Providers
+
+| Provider | Flag | Requires | Best For |
+|----------|------|----------|----------|
+| Ollama | `-p ollama` | Local Ollama running | Offline, private, fast iteration |
+| AWS Bedrock | `-p bedrock` | AWS credentials | Production quality, Claude models |
+| OpenAI | `-p openai` | `OPENAI_API_KEY` | GPT-4 Turbo |
+| Azure OpenAI | `-p azure` | Azure endpoint configured | Enterprise Azure environments |
+
+### Recommended Models
+
+| Provider | Model | Quality | Speed |
+|----------|-------|---------|-------|
+| Ollama | `llama3` (default) | Good | Fast |
+| Ollama | `codellama:34b` | Better | Slower |
+| Bedrock | `claude-sonnet-4-20250514` | Excellent | Fast |
+| OpenAI | `gpt-4-turbo` | Excellent | Medium |
+
+## Self-Correction
+
+When Checkov finds violations, the AI automatically fixes them:
+
+```
+🤖 Generating infrastructure code...
+  ✅ Generated 4 files
+
+🔒 Validating with Checkov...
+  ⚠️ 2 findings: CKV_AWS_130 (VPC flow logs), CKV_AWS_178 (NAT HA)
+
+🔄 Self-correcting (iteration 1/3)...
+  ✅ Fixed: added flow logs + multi-AZ NAT
+
+🔒 Re-validating...
+  ✅ Validation passed
+```
+
+Disable with `--no-self-correct` if you want raw output without fixes.
+
+## MCP Integration
+
+The command is also available as an MCP tool for AI assistants (Kiro, Claude Code):
+
+```json
+{
+  "name": "thothctl_generate_iac",
+  "description": "Generate governed IaC from natural language intent",
+  "parameters": {
+    "intent": "VPC with 3 private subnets",
+    "project_type": "terraform-terragrunt",
+    "self_correct": true,
+    "apply": false
+  }
+}
+```
+
+Start the MCP server: `thothctl mcp server`
+
+## Tips
+
+1. **Be specific** — "VPC with 3 private subnets, NAT per AZ, flow logs" produces better results than "create a VPC"
+2. **Use your scaffold** — run from a project that has `.thothcf.toml` and existing stacks for best context
+3. **Start with dry-run** — review the generated code before `--apply`
+4. **Iterate** — if the output isn't perfect, refine your intent and re-run
+5. **Use Bedrock for production** — Claude Sonnet produces the highest quality IaC
+
+## Troubleshooting
+
+### "AI returned no files"
+
+The AI provider couldn't parse the intent or returned invalid JSON. Try:
+- Simplify the intent
+- Use a more capable model (`-p bedrock -m claude-sonnet-4-20250514`)
+- Check provider connectivity (`thothctl check environment`)
+
+### "Validation: N violations remain"
+
+Self-correction reached max iterations without passing all checks. Options:
+- Increase iterations: `--max-iterations 5`
+- Skip validation: `--skip-validation` (fix manually)
+- Fix the generated code yourself and run `thothctl scan iac -t checkov`
+
+### Slow generation with Ollama
+
+Large models + long context = slow. Solutions:
+- Use a smaller model: `-m llama3` (7B is fastest)
+- Reduce context: remove unnecessary steering files
+- Switch to API provider: `-p bedrock` (faster inference)
+
+### Provider not initialized
+
+```
+Failed to initialize AI provider: ...
+```
+
+Check provider configuration:
+- Ollama: ensure `ollama serve` is running
+- Bedrock: ensure AWS credentials are configured
+- OpenAI: ensure `OPENAI_API_KEY` is set