thothctl 0.29.0__tar.gz → 0.30.1__tar.gz

{thothctl-0.29.0 → thothctl-0.30.1}/CHANGELOG.md

@@ -5,6 +5,43 @@ All notable changes to ThothCTL are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.30.0] - 2026-09-22
+
+### Fixed
+
+- **Intent-to-IaC plan validation** — per-stack plan validation is now skipped for
+  terragrunt projects when no `stack_path` is given (previously it wrote stray `.tf`
+  files into the project root and ran `terragrunt plan` against `root.hcl`).
+  Single-mode validation now routes `full-project` mode to `validate_full_project`.
+- **HCL-aware block stripping** — `terraform {}` / `provider {}` removal now uses a
+  string/comment-aware brace scanner that correctly handles arbitrarily nested
+  blocks (e.g. `backend`, `assume_role`, `default_tags`) and braces inside strings
+  or comments, replacing the previous single-level regex.
+- Added a missing module logger in `generate iac` CLI (undefined `logger` in the
+  history-save error handler).
+
+### Added
+
+- **`--plan-filter` wiring** — the flag (plus `THOTH_PLAN_FILTER` env and
+  `[generation.plan].stack_filter`) is now honored end-to-end; `validate_full_project`
+  routes to `validate_filtered` when set.
+- **Real dependency-graph diagrams** — composition-mode architecture diagrams now
+  render actual `depends_on` edges and layer subgraphs (new `_build_stack_diagram`)
+  instead of inferred/hardcoded edges.
+- **Raw-response debugging** — the raw AI response is persisted to generation history
+  on parse failure (new `raw_response` column + idempotent migration) and surfaced in
+  the CLI.
+- Single-mode `--mode project` now resolves `#{...}#` placeholders (previously only
+  the composition path did).
+
+### Changed
+
+- Unified self-correction into a shared loop (with stagnation detection) used by both
+  single and composition paths; deduplicated stack file processing.
+- `generation_tokens` is now estimated consistently across single and composition modes.
+- Removed dead scaffold helpers; added a public `ScaffoldLoader.scaffold_dir` property;
+  warn when scaffold examples are unavailable (missing `gh` CLI).
+
 ## [0.29.0] - 2026-09-15
 
 ### Added

{thothctl-0.29.0 → thothctl-0.30.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: thothctl
-Version: 0.29.0
+Version: 0.30.1
 Summary: A CLI for Developer Control Plane. Accelerate your cloud IaC deployments.
 Project-URL: SourceCode, https://github.com/thothforge/thothctl
 Project-URL: HomePage, https://github.com/thothforge/thothctl

{thothctl-0.29.0 → thothctl-0.30.1}/docs/framework/commands/mcp/mcp.md

@@ -82,7 +82,7 @@ thothctl mcp register
 This command will internally use the correct MCP registration syntax:
 
 ```bash
-q mcp add --name thothctl --command "thothctl mcp server"
+kiro-cli mcp add --name thothctl --command "thothctl" --args "mcp" --args "server" --args "--stdio"
 ```
 
 Register the MCP server running on a custom port:
@@ -209,7 +209,7 @@ This tool generates governed Infrastructure as Code from natural language. It su
 
 **Example via AI Assistant**:
 ```
-q chat "Generate a VPC with 3 private subnets and NAT gateway for production in us-east-1 using terraform-terragrunt composition full mode"
+kiro-cli chat "Generate a VPC with 3 private subnets and NAT gateway for production in us-east-1 using terraform-terragrunt composition full mode"
 ```
 
 ## Integration with AI Assistants
@@ -217,7 +217,7 @@ q chat "Generate a VPC with 3 private subnets and NAT gateway for production in
 After starting the MCP server and registering it with your AI assistant, you can interact with ThothCTL using natural language:
 
 ```bash
-q chat "List all ThothCTL projects"
+kiro-cli chat "List all ThothCTL projects"
 ```
 
 Example interactions:
@@ -264,57 +264,61 @@ This typically means one of the following:
 3. **Wrong command in registration**: Verify your registration command.
    ```bash
    # List your registered MCP servers
-   q mcp list
+   kiro-cli mcp list
    
    # Remove incorrect registration if needed
-   q mcp remove thothctl
+   kiro-cli mcp remove --name thothctl
    
    # Register correctly
-   q mcp add --name thothctl --command "thothctl mcp server"
+   kiro-cli mcp add --name thothctl --command "thothctl" --args "mcp" --args "server" --args "--stdio"
    ```
 
 4. **MCP configuration file is missing or corrupted**: Your AI assistant stores MCP server configurations in JSON files.
    
-   **Configuration file locations**:
-   - Global Configuration: `~/.aws/amazonq/mcp.json` - Applies to all workspaces
-   - Workspace Configuration: `.amazonq/mcp.json` - Specific to the current workspace
+   **Configuration file locations (Kiro CLI)**:
+   - Global Configuration: `~/.kiro/settings/mcp.json` - Applies to all workspaces
+   - Workspace Configuration: `.kiro/settings/mcp.json` - Specific to the current workspace (takes precedence over global)
+
+   For **Claude Desktop**, MCP servers are configured in:
+   - macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
+   - Windows: `%APPDATA%\Claude\claude_desktop_config.json`
    
    ```bash
-   # Check if the configuration files exist
-   ls -la ~/.aws/amazonq/mcp.json  # Global config (Linux/macOS)
-   ls -la .amazonq/mcp.json        # Workspace config (Linux/macOS)
+   # Check if the configuration files exist (Kiro CLI)
+   ls -la ~/.kiro/settings/mcp.json  # Global config (Linux/macOS)
+   ls -la .kiro/settings/mcp.json    # Workspace config (Linux/macOS)
    
    # View the current configuration
-   cat ~/.aws/amazonq/mcp.json     # Global config (Linux/macOS)
-   cat .amazonq/mcp.json           # Workspace config (Linux/macOS)
+   cat ~/.kiro/settings/mcp.json     # Global config (Linux/macOS)
+   cat .kiro/settings/mcp.json       # Workspace config (Linux/macOS)
    
-   # If missing or corrupted, recreate it
-   q mcp remove thothctl
-   q mcp add --name thothctl --command "thothctl mcp server"
+   # If missing or corrupted, recreate it (stdio is the default)
+   thothctl mcp register
+   # ...or register manually with the Kiro CLI (note the required --stdio flag)
+   kiro-cli mcp remove --name thothctl
+   kiro-cli mcp add --name thothctl --command "thothctl" --args "mcp" --args "server" --args "--stdio"
    ```
 
-   The MCP configuration file (`mcp.json`) should have the following structure:
+   The MCP configuration file (`mcp.json`) should have the following structure.
+   The `--stdio` flag is required — Kiro CLI and Claude talk to the server over
+   stdin/stdout, so stdio mode must be enabled explicitly. Without it,
+   `thothctl mcp server` starts in HTTP mode and the assistant cannot connect:
    ```json
    {
      "mcpServers": {
        "thothctl": {
          "command": "thothctl",
-         "args": ["mcp", "server"],
-         "env": {},
-         "timeout": 60000
+         "args": ["mcp", "server", "--stdio"]
        }
      }
    }
    ```
 
-   You can manually create or edit this file if needed, but it's recommended to use the `q mcp add` command to ensure proper formatting.
+   You can manually create or edit this file if needed, but it's recommended to use `thothctl mcp register` (or `kiro-cli mcp add`) to ensure proper formatting.
 
-5. **Detailed debugging**: Run with increased logging as suggested in the error message.
+5. **Detailed debugging**: Run with increased logging.
    ```bash
-   Q_LOG_LEVEL=trace q chat "List ThothCTL projects"
-   
-   # Then check the logs
-   cat $TMPDIR/qchat/latest.log
+   kiro-cli chat --verbose "List ThothCTL projects"
    ```
 
 #### Port Already in Use
@@ -334,8 +338,8 @@ thothctl mcp server -p 8081
 And update your registration accordingly:
 
 ```bash
-q mcp remove thothctl
-q mcp add --name thothctl --command "thothctl" --args "mcp" --args "server" --args "-p" --args "8081"
+kiro-cli mcp remove --name thothctl
+kiro-cli mcp add --name thothctl --command "thothctl" --args "mcp" --args "server" --args "-p" --args "8081"
 ```
 
 ## Architecture

{thothctl-0.29.0 → thothctl-0.30.1}/docs/framework/software_architecture.md

@@ -578,7 +578,7 @@ Model Context Protocol server for AI integration.
   "mcpServers": {
     "thothctl": {
       "command": "thothctl",
-      "args": ["mcp", "server"]
+      "args": ["mcp", "server", "--stdio"]
     }
   }
 }

{thothctl-0.29.0 → thothctl-0.30.1}/docs/mcp.md

@@ -36,16 +36,25 @@ This will start the MCP server on port 8080 (default).
 
 ### Registering with Kiro CLI
 
-To use ThothCTL with Kiro CLI or Claude, you need to register the MCP server:
+To use ThothCTL with Kiro CLI or Claude, you need to register the MCP server.
+The easiest way is the built-in command, which writes a stdio-mode config to
+`~/.kiro/settings/mcp.json` (or `.kiro/settings/mcp.json` with `--scope workspace`):
 
 ```bash
-kiro mcp add thothctl --command "thothctl mcp server"
+thothctl mcp register
+```
+
+Alternatively, register it directly with the Kiro CLI. Note the `--stdio` flag —
+it is required so the server speaks MCP over stdin/stdout instead of HTTP:
+
+```bash
+kiro-cli mcp add --name thothctl --command "thothctl" --args "mcp" --args "server" --args "--stdio"
 ```
 
 Once registered, you can interact with ThothCTL through Kiro CLI using natural language:
 
 ```bash
-kiro chat "List all ThothCTL projects using the MCP integration"
+kiro-cli chat "List all ThothCTL projects using the MCP integration"
 ```
 
 ### Checking Server Status
@@ -88,29 +97,41 @@ This typically means one of the following:
    ```
 
 3. **MCP configuration file is missing or corrupted**: Your AI assistant stores MCP server configurations in JSON files.
-   
-   **Configuration file locations**:
-   - Global Configuration: `~/.aws/amazonq/mcp.json` - Applies to all workspaces
-   - Workspace Configuration: `.amazonq/mcp.json` - Specific to the current workspace
-   
-   The MCP configuration file should have this structure:
+
+   **Configuration file locations (Kiro CLI)**:
+   - Global Configuration: `~/.kiro/settings/mcp.json` - Applies to all workspaces
+   - Workspace Configuration: `.kiro/settings/mcp.json` - Specific to the current workspace (takes precedence over global)
+
+   For **Claude Desktop**, MCP servers are configured in:
+   - macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
+   - Windows: `%APPDATA%\Claude\claude_desktop_config.json`
+
+   The MCP configuration file should have this structure. Note the `--stdio`
+   flag — Kiro CLI and Claude communicate with the server over stdin/stdout,
+   so stdio mode must be enabled explicitly (without it, `thothctl mcp server`
+   starts in HTTP mode and the assistant cannot connect):
    ```json
    {
      "mcpServers": {
        "thothctl": {
          "command": "thothctl",
-         "args": ["mcp", "server"],
-         "env": {},
-         "timeout": 60000
+         "args": ["mcp", "server", "--stdio"]
        }
      }
    }
    ```
 
-   To recreate the configuration:
+   To recreate the configuration, the simplest way is the built-in registration
+   command (stdio is the default and writes to `~/.kiro/settings/mcp.json`):
+   ```bash
+   thothctl mcp register              # global scope, stdio mode
+   thothctl mcp register --scope workspace   # writes .kiro/settings/mcp.json
+   ```
+
+   Or register it manually via the Kiro CLI:
    ```bash
-   q mcp remove thothctl
-   q mcp add --name thothctl --command "thothctl" --args "mcp" --args "server"
+   kiro-cli mcp remove --name thothctl
+   kiro-cli mcp add --name thothctl --command "thothctl" --args "mcp" --args "server" --args "--stdio"
    ```
 
 For more detailed troubleshooting, see the [MCP Command Documentation](./framework/commands/mcp/mcp.md#troubleshooting).

{thothctl-0.29.0 → thothctl-0.30.1}/docs/whats_new.md

@@ -1,5 +1,40 @@
 # What's New
 
+## v0.30.0 — Intent-to-IaC Hardening (September 2026)
+
+### Highlights
+
+!!! success "Accurate dependency-graph diagrams for multi-stack generation"
+    Composition-mode diagrams now render the real stack dependency graph
+    (from each stack's `depends_on`) grouped by layer — no more inferred edges.
+
+!!! success "Targeted plan validation with --plan-filter"
+    ```bash
+    thothctl generate iac -i "add an RDS stack" \
+      --composition incremental \
+      --plan-validation full-project \
+      --plan-filter "stacks/platform/data/*"
+    ```
+
+- **Safer plan validation** — per-stack plan validation no longer runs against the
+  terragrunt project root (`root.hcl`); single-stack + `full-project` mode is routed
+  correctly
+- **`--plan-filter` wired end-to-end** — also settable via `THOTH_PLAN_FILTER` or
+  `[generation.plan].stack_filter`
+- **HCL-aware block stripping** — `terraform {}` / `provider {}` blocks are removed
+  with a string/comment-aware scanner that handles nested blocks (`backend`,
+  `assume_role`, `default_tags`) reliably
+- **Single-mode placeholder resolution** — `--mode project` now resolves `#{...}#`
+  placeholders for single-stack generation too
+- **Raw-response debugging** — when the AI returns unparseable output, the raw
+  response is saved to generation history for troubleshooting
+- **Unified self-correction** — single and composition paths now share the same
+  validate → fix → re-validate loop with stagnation detection
+- Consistent `generation_tokens` metrics across modes; clearer warning when scaffold
+  examples are unavailable (missing `gh` CLI)
+
+---
+
 ## v0.28.0 — Security Platform Integrations (August 2026)
 
 ### Highlights

{thothctl-0.29.0 → thothctl-0.30.1}/src/thothctl/commands/generate/commands/iac.py

@@ -1,5 +1,6 @@
 """CLI command: thothctl generate iac — Generate IaC from natural language intent."""
 
+import logging
 from typing import Optional
 
 import click
@@ -7,6 +8,8 @@ import click
 from ....core.cli_ui import CliUI
 from ....core.commands import ClickCommand
 
+logger = logging.getLogger(__name__)
+
 
 class GenerateIaCCommand(ClickCommand):
     """Generate governed Infrastructure as Code from natural language intent."""
@@ -65,6 +68,7 @@ class GenerateIaCCommand(ClickCommand):
             plan_iam_role=plan_iam_role,
             plan_profile=plan_profile,
             project_type=project_type,
+            plan_filter=plan_filter,
         )
 
         # Show intent
@@ -149,11 +153,21 @@ class GenerateIaCCommand(ClickCommand):
         # Display results
         if not result.success:
             self.ui.print_error(f"Generation failed: {result.error}")
+            if result.raw_response:
+                self.ui.print_info(
+                    "The raw AI response was saved to generation history for "
+                    "debugging (view via the dashboard Generation tab or "
+                    "generation_history.db)."
+                )
             # Still record failed runs to history
             try:
                 import time as _time
 
-                _elapsed = _time.time() - self._start_time if hasattr(self, "_start_time") else 0
+                _elapsed = (
+                    _time.time() - self._start_time
+                    if hasattr(self, "_start_time")
+                    else 0
+                )
                 from ....services.generate.intent.generation_history import (
                     save_generation_run,
                 )
@@ -308,6 +322,7 @@ class GenerateIaCCommand(ClickCommand):
         plan_iam_role: Optional[str],
         plan_profile: Optional[str],
         project_type: str,
+        plan_filter: Optional[str] = None,
     ) -> Optional[dict]:
         """Build plan validation config from CLI flags + .thothcf.toml.
 
@@ -346,6 +361,12 @@ class GenerateIaCCommand(ClickCommand):
         elif os.environ.get("THOTH_PLAN_AWS_PROFILE"):
             config["aws_profile"] = os.environ["THOTH_PLAN_AWS_PROFILE"]
 
+        # Stack filter for targeted plan validation
+        if plan_filter:
+            config["stack_filter"] = plan_filter
+        elif os.environ.get("THOTH_PLAN_FILTER"):
+            config["stack_filter"] = os.environ["THOTH_PLAN_FILTER"]
+
         # Always pass project type for routing
         config["project_type"] = project_type
 
@@ -492,6 +513,8 @@ cli = GenerateIaCCommand.as_click_command(
     click.option(
         "--plan-filter",
         default=None,
-        help="Terragrunt filter pattern for targeted plan validation",
+        help="Terragrunt filter pattern for targeted plan validation "
+        "(scopes 'full-project' plan validation to matching stacks). "
+        "Also settable via THOTH_PLAN_FILTER or [generation.plan].stack_filter.",
     ),
 )

{thothctl-0.29.0 → thothctl-0.30.1}/src/thothctl/services/generate/intent/generation_history.py

@@ -6,8 +6,6 @@ and generated file listings. Powers the dashboard Generation tab.
 Database: ~/.thothcf/generation_history.db
 """
 
-import json
-import os
 import sqlite3
 import uuid
 from datetime import datetime
@@ -37,7 +35,8 @@ CREATE TABLE IF NOT EXISTS generation_runs (
     violations_initial INTEGER DEFAULT 0,
     violations_final INTEGER DEFAULT 0,
     output_dir TEXT DEFAULT '',
-    plan_validation TEXT DEFAULT 'disabled'
+    plan_validation TEXT DEFAULT 'disabled',
+    raw_response TEXT DEFAULT ''
 );
 
 CREATE TABLE IF NOT EXISTS generation_files (
@@ -82,9 +81,26 @@ def _get_conn() -> sqlite3.Connection:
     conn = sqlite3.connect(str(DB_PATH))
     conn.row_factory = sqlite3.Row
     conn.executescript(_SCHEMA)
+    _migrate(conn)
     return conn
 
 
+def _migrate(conn: sqlite3.Connection) -> None:
+    """Apply lightweight, idempotent migrations for pre-existing databases.
+
+    CREATE TABLE IF NOT EXISTS does not add columns to an already-existing
+    table, so newly introduced columns are added here via ALTER TABLE.
+    """
+    existing = {
+        row[1] for row in conn.execute("PRAGMA table_info(generation_runs)").fetchall()
+    }
+    if "raw_response" not in existing:
+        conn.execute(
+            "ALTER TABLE generation_runs ADD COLUMN raw_response TEXT DEFAULT ''"
+        )
+        conn.commit()
+
+
 def save_generation_run(
     intent: str,
     result: Any,
@@ -126,8 +142,9 @@ def save_generation_run(
             (id, timestamp, intent, project_type, composition, output_mode, 
              space, provider, model, success, error, files_count, iterations,
              context_tokens, generation_tokens, duration_seconds,
-             violations_initial, violations_final, output_dir, plan_validation)
-            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
+             violations_initial, violations_final, output_dir, plan_validation,
+             raw_response)
+            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
             (
                 run_id,
                 datetime.now().isoformat(),
@@ -149,6 +166,8 @@ def save_generation_run(
                 result.validation.total_violations if result.validation else 0,
                 output_dir or "",
                 plan_validation,
+                # Only persisted on failed parses; truncate defensively.
+                (getattr(result, "raw_response", None) or "")[:10000],
             ),
         )
 
@@ -188,7 +207,7 @@ def save_generation_run(
         conn.commit()
         return run_id
 
-    except Exception as e:
+    except Exception:
         conn.rollback()
         raise
     finally:
@@ -335,33 +354,43 @@ def _compute_metrics(
         }
 
     # Add success condition to existing WHERE
-    success_where = (
-        f"{where} AND success = 1" if where else "WHERE success = 1"
-    )
+    success_where = f"{where} AND success = 1" if where else "WHERE success = 1"
 
     success = conn.execute(
         f"SELECT COUNT(*) FROM generation_runs {success_where}", params
     ).fetchone()[0]
 
-    avg_duration = conn.execute(
-        f"SELECT AVG(duration_seconds) FROM generation_runs {success_where}",
-        params,
-    ).fetchone()[0] or 0
+    avg_duration = (
+        conn.execute(
+            f"SELECT AVG(duration_seconds) FROM generation_runs {success_where}",
+            params,
+        ).fetchone()[0]
+        or 0
+    )
 
-    avg_iterations = conn.execute(
-        f"SELECT AVG(iterations) FROM generation_runs {success_where}",
-        params,
-    ).fetchone()[0] or 0
+    avg_iterations = (
+        conn.execute(
+            f"SELECT AVG(iterations) FROM generation_runs {success_where}",
+            params,
+        ).fetchone()[0]
+        or 0
+    )
 
-    total_tokens = conn.execute(
-        f"SELECT SUM(context_tokens + generation_tokens) FROM generation_runs {where}",
-        params,
-    ).fetchone()[0] or 0
+    total_tokens = (
+        conn.execute(
+            f"SELECT SUM(context_tokens + generation_tokens) FROM generation_runs {where}",
+            params,
+        ).fetchone()[0]
+        or 0
+    )
 
-    avg_files = conn.execute(
-        f"SELECT AVG(files_count) FROM generation_runs {success_where}",
-        params,
-    ).fetchone()[0] or 0
+    avg_files = (
+        conn.execute(
+            f"SELECT AVG(files_count) FROM generation_runs {success_where}",
+            params,
+        ).fetchone()[0]
+        or 0
+    )
 
     # Composition breakdown
     compositions = conn.execute(