@opengeni/runtime 0.2.0

package/src/bundled_hashicorp_terraform_skills/terraform-stacks/SKILL.md

@@ -0,0 +1,480 @@
+---
+name: terraform-stacks
+description: Comprehensive guide for working with HashiCorp Terraform Stacks. Use when creating, modifying, or validating Terraform Stack configurations (.tfcomponent.hcl, .tfdeploy.hcl files), working with stack components and deployments from local modules, public registry, or private registry sources, managing multi-region or multi-environment infrastructure, or troubleshooting Terraform Stacks syntax and structure.
+metadata:
+  copyright: Copyright IBM Corp. 2026
+  version: "0.0.1"
+---
+
+# Terraform Stacks
+
+Terraform Stacks simplify infrastructure provisioning and management at scale by providing a configuration layer above traditional Terraform modules. Stacks enable declarative orchestration of multiple components across environments, regions, and cloud accounts.
+
+## Core Concepts
+
+**Stack**: A complete unit of infrastructure composed of components and deployments that can be managed together.
+
+**Component**: An abstraction around a Terraform module that defines infrastructure pieces. Each component specifies a source module, inputs, and providers.
+
+**Deployment**: An instance of all components in a stack with specific input values. Use deployments for different environments (dev/staging/prod), regions, or cloud accounts.
+
+**Stack Language**: A separate HCL-based language (not regular Terraform HCL) with distinct blocks and file extensions.
+
+## File Structure
+
+Terraform Stacks use specific file extensions:
+
+- **Component configuration**: `.tfcomponent.hcl`
+- **Deployment configuration**: `.tfdeploy.hcl`
+- **Provider lock file**: `.terraform.lock.hcl` (generated by CLI)
+
+All configuration files must be at the root level of the Stack repository. HCP Terraform processes all files in dependency order.
+
+### Recommended File Organization
+
+```
+my-stack/
+├── .terraform-version               # The required Terraform version for this Stack
+├── variables.tfcomponent.hcl        # Variable declarations
+├── providers.tfcomponent.hcl        # Provider configurations
+├── components.tfcomponent.hcl       # Component definitions
+├── outputs.tfcomponent.hcl          # Stack outputs
+├── deployments.tfdeploy.hcl         # Deployment definitions
+├── .terraform.lock.hcl              # Provider lock file (generated)
+└── modules/                         # Local modules (optional - only if using local modules)
+    ├── s3/
+    └── compute/
+```
+
+**Note**: The `modules/` directory is only required when using local module sources. Components can reference modules from:
+- Local file paths: `./modules/vpc`
+- Public registry: `terraform-aws-modules/vpc/aws`
+- Private registry: `app.terraform.io/<org-name>/vpc/aws`
+- Git: `git::https://github.com/org/repo.git//path?ref=v1.0.0`
+
+HCP Terraform processes all `.tfcomponent.hcl` and `.tfdeploy.hcl` files in dependency order.
+
+## Required Terraform version (.terraform-version)
+
+Use Terraform v1.13.x or later to access the Stacks CLI plugin and to run
+terraform stacks CLI commands. Begin by adding a .terraform-version file to
+your Stack's root directory to specify the Terraform version required for your
+Stack. For example, the following file specifies Terraform v1.14.5:
+
+```
+1.14.5
+```
+
+## Component Configuration (.tfcomponent.hcl)
+
+### Variable Block
+
+Declare input variables for the Stack configuration. Variables must define a `type` field and do not support the `validation` argument.
+
+```hcl
+variable "aws_region" {
+  type        = string
+  description = "AWS region for deployments"
+  default     = "us-west-1"
+}
+
+variable "identity_token" {
+  type        = string
+  description = "OIDC identity token"
+  ephemeral   = true  # Does not persist to state file
+}
+
+variable "instance_count" {
+  type     = number
+  nullable = false
+}
+```
+
+**Important**: Use `ephemeral = true` for credentials and tokens (identity tokens, API keys, passwords) to prevent them from persisting in state files. Use `stable` for longer-lived values like license keys that need to persist across runs.
+
+### Required Providers Block
+
+```hcl
+required_providers {
+  aws = {
+    source  = "hashicorp/aws"
+    version = "~> 6.0"
+  }
+  random = {
+    source  = "hashicorp/random"
+    version = "~> 3.5.0"
+  }
+}
+```
+
+### Provider Block
+
+Provider blocks differ from traditional Terraform:
+
+1. Support `for_each` meta-argument
+2. Define aliases in the block header (not as an argument)
+3. Accept configuration through a `config` block
+
+**Single Provider Configuration:**
+
+```hcl
+provider "aws" "this" {
+  config {
+    region = var.aws_region
+    assume_role_with_web_identity {
+      role_arn           = var.role_arn
+      web_identity_token = var.identity_token
+    }
+  }
+}
+```
+
+**Multiple Provider Configurations with for_each:**
+
+```hcl
+provider "aws" "configurations" {
+  for_each = var.regions
+
+  config {
+    region = each.value
+    assume_role_with_web_identity {
+      role_arn           = var.role_arn
+      web_identity_token = var.identity_token
+    }
+  }
+}
+```
+
+**Authentication Best Practice**: Use **workload identity** (OIDC) as the preferred authentication method for Stacks. This approach:
+- Avoids long-lived static credentials
+- Provides temporary, scoped credentials per deployment run
+- Integrates with cloud provider IAM (AWS IAM Roles, Azure Managed Identities, GCP Service Accounts)
+- Eliminates need for platform-managed environment variables
+
+Configure workload identity using `identity_token` blocks and `assume_role_with_web_identity` in provider configuration. For detailed setup instructions for AWS, Azure, and GCP, see: https://developer.hashicorp.com/terraform/cloud-docs/dynamic-provider-credentials
+
+### Component Block
+
+Each Stack requires at least one component block. Add a component for each module to include in the Stack. Components reference modules from local paths, registries, or Git.
+
+```hcl
+component "vpc" {
+  source  = "app.terraform.io/my-org/vpc/aws"  # Local, registry, or Git URL
+  version = "2.1.0"          # For registry modules
+
+  inputs = {
+    cidr_block  = var.vpc_cidr
+    name_prefix = var.name_prefix
+  }
+
+  providers = {
+    aws = provider.aws.this
+  }
+}
+```
+
+See `references/component-blocks.md` for examples of dependencies, for_each, public registry modules, Git sources, and more.
+
+**Key Points:**
+- Reference outputs: `component.<name>.<output>` or `component.<name>[key].<output>` for for_each
+- Dependencies inferred automatically from component references
+- Aggregate with for expressions: `[for x in component.s3 : x.bucket_name]`
+- For components with `for_each`, reference specific instances: `component.<name>[each.value].<output>`
+- Provider references are normal values: `provider.<type>.<alias>` or `provider.<type>.<alias>[each.value]`
+
+### Output Block
+
+Outputs require a `type` argument and do not support `preconditions`:
+
+```hcl
+output "vpc_id" {
+  type        = string
+  description = "VPC ID"
+  value       = component.vpc.vpc_id
+}
+
+output "endpoint_urls" {
+  type      = map(string)
+  value     = {
+    for region, comp in component.api : region => comp.endpoint_url
+  }
+  sensitive = false
+}
+```
+
+### Locals Block
+
+Locals blocks work the same in both `.tfcomponent.hcl` and `.tfdeploy.hcl` files:
+
+```hcl
+locals {
+  common_tags = {
+    Environment = var.environment
+    ManagedBy   = "Terraform Stacks"
+    Project     = var.project_name
+  }
+
+  region_config = {
+    for region in var.regions : region => {
+      name_suffix = "${var.environment}-${region}"
+    }
+  }
+}
+```
+
+### Removed Block
+
+Use to safely remove components from a Stack. HCP Terraform requires the component's providers to remove it.
+
+```hcl
+removed {
+  from   = component.old_component
+  source = "./modules/old-module"
+  
+  providers = {
+    aws = provider.aws.this
+  }
+}
+```
+
+## Deployment Configuration (.tfdeploy.hcl)
+
+### Identity Token Block
+
+Generate JWT tokens for OIDC authentication with cloud providers:
+
+```hcl
+identity_token "aws" {
+  audience = ["aws.workload.identity"]
+}
+
+identity_token "azure" {
+  audience = ["api://AzureADTokenExchange"]
+}
+```
+
+Reference tokens in deployments using `identity_token.<name>.jwt`
+
+### Store Block
+
+Access HCP Terraform variable sets within Stack deployments:
+
+```hcl
+store "varset" "aws_credentials" {
+  id       = "varset-ABC123"  # Alternatively use: name = "varset_name"
+  source   = "tfc-cloud-shared"
+  category = "terraform"      # Alternatively use: category = "env" for environment variables
+}
+
+deployment "production" {
+  inputs = {
+    aws_access_key = store.varset.aws_credentials.AWS_ACCESS_KEY_ID
+  }
+}
+```
+
+Use to centralize credentials and share variables across Stacks. See `references/deployment-blocks.md` for details.
+
+### Deployment Block
+
+Define deployment instances (minimum 1, maximum 20 per Stack):
+
+```hcl
+deployment "production" {
+  inputs = {
+    aws_region     = "us-west-1"
+    instance_count = 3
+    role_arn       = local.role_arn
+    identity_token = identity_token.aws.jwt
+  }
+}
+
+# Create multiple deployments for different environments
+deployment "development" {
+  inputs = {
+    aws_region     = "us-east-1"
+    instance_count = 1
+    name_suffix    = "dev"
+    role_arn       = local.role_arn
+    identity_token = identity_token.aws.jwt
+  }
+}
+```
+
+**To destroy a deployment**: Set `destroy = true`, upload configuration, approve destroy run, then remove the deployment block. See `references/deployment-blocks.md` for details.
+
+### Deployment Group Block
+
+Group deployments together for shared settings (HCP Terraform Premium tier feature). Free/standard tiers use default groups named `{deployment-name}_default`.
+
+```hcl
+deployment_group "canary" {
+  auto_approve_checks = [deployment_auto_approve.safe_changes]
+}
+
+deployment "dev" {
+  inputs = { /* ... */ }
+  deployment_group = deployment_group.canary
+}
+```
+
+Multiple deployments can reference the same group. See `references/deployment-blocks.md` for details.
+
+### Deployment Auto-Approve Block
+
+Define rules to automatically approve deployment plans (HCP Terraform Premium tier feature):
+
+```hcl
+deployment_auto_approve "safe_changes" {
+  deployment_group = deployment_group.canary
+
+  check {
+    condition = context.plan.changes.remove == 0
+    reason    = "Cannot auto-approve plans with resource deletions"
+  }
+}
+```
+
+**Available context variables**: `context.plan.applyable`, `context.plan.changes.add/change/remove/total`, `context.success`
+
+**Note:** `orchestrate` blocks are deprecated. Use `deployment_group` and `deployment_auto_approve` instead.
+
+See `references/deployment-blocks.md` for all context variables and patterns.
+
+### Publish Output and Upstream Input Blocks
+
+Link Stacks together by publishing outputs from one Stack and consuming them in another:
+
+```hcl
+# In network Stack - publish outputs
+publish_output "vpc_id_network" {
+  type  = string
+  value = deployment.network.vpc_id
+}
+
+# In application Stack - consume outputs
+upstream_input "network_stack" {
+  type   = "stack"
+  source = "app.terraform.io/my-org/my-project/networking-stack"
+}
+
+deployment "app" {
+  inputs = {
+    vpc_id = upstream_input.network_stack.vpc_id_network
+  }
+}
+```
+
+See `references/linked-stacks.md` for complete documentation and examples.
+
+## Terraform Stacks CLI
+
+**Note**: Terraform Stacks is Generally Available (GA) as of Terraform CLI v1.13+. Stacks now count toward Resources Under Management (RUM) for HCP Terraform billing.
+
+### Initialize and Validate
+
+```bash
+terraform stacks init              # Download providers, modules, generate lock file
+terraform stacks providers-lock    # Regenerate lock file (add platforms if needed)
+terraform stacks validate          # Check syntax without uploading
+```
+
+### Deployment Workflow
+
+**Important**: No `plan` or `apply` commands. Upload configuration triggers deployment runs automatically.
+
+```bash
+# 1. Upload configuration (triggers deployment runs)
+terraform stacks configuration upload
+
+# 2. Monitor deployments
+terraform stacks deployment-run list                          # List runs (non-interactive)
+terraform stacks deployment-group watch -deployment-group=... # Stream status updates
+
+# 3. Approve deployments (if auto-approve not configured)
+terraform stacks deployment-run approve-all-plans -deployment-run-id=...
+terraform stacks deployment-group approve-all-plans -deployment-group=...
+terraform stacks deployment-run cancel -deployment-run-id=...  # Cancel if needed
+```
+
+### Configuration Management
+
+```bash
+terraform stacks configuration list                    # List configuration versions
+terraform stacks configuration fetch -configuration-id=...  # Download configuration
+terraform stacks configuration watch                   # Monitor upload status
+```
+
+### Other Commands
+
+```bash
+terraform stacks create              # Create new Stack (interactive)
+terraform stacks fmt                 # Format Stack files
+terraform stacks list                # Show all Stacks
+terraform stacks version             # Display version
+terraform stacks deployment-group rerun -deployment-group=...  # Rerun deployment
+```
+
+## Monitoring Deployments with HCP Terraform API
+
+For programmatic monitoring in automation, CI/CD, or non-interactive environments (like AI agents), use the HCP Terraform API instead of CLI watch commands. The API provides endpoints for:
+
+- Configuration status and validation
+- Deployment group summaries
+- Deployment run status
+- Deployment step details (plan/apply)
+- Error diagnostics with file locations and code snippets
+- Stack outputs via artifacts endpoint
+
+**Key points:**
+- CLI watch commands stream indefinitely and don't work in automation
+- Use artifacts endpoint to retrieve Stack outputs: `GET /api/v2/stack-deployment-steps/{step-id}/artifacts?name=apply-description`
+- Diagnostics endpoint requires `stack_deployment_step_id` query parameter
+- Artifacts endpoint returns HTTP 307 redirect (use `curl -L`)
+
+For complete API workflow, authentication, polling best practices, and example scripts, see `references/api-monitoring.md`.
+
+## Common Patterns
+
+**Component Dependencies**: Dependencies are automatically inferred when one component references another's output (e.g., `subnet_ids = component.vpc.private_subnet_ids`).
+
+**Multi-Region Deployment**: Use `for_each` on providers and components to deploy across multiple regions. Each region gets its own provider configuration and component instances.
+
+**Deferred Changes**: Stacks support deferred changes to handle dependencies where values are only known after apply. This enables complex multi-component deployments where some resources depend on runtime values from other components (cluster endpoints, generated passwords, etc.).
+
+For complete examples including multi-region deployments, component dependencies, deferred changes patterns, and linked Stacks, see `references/examples.md`.
+
+## Best Practices
+
+1. **Component Granularity**: Create components for logical infrastructure units that share a lifecycle
+2. **Module Compatibility**:
+   - Modules used with Stacks cannot include provider blocks (configure providers in Stack configuration)
+   - **Test public registry modules** before using in production Stacks - some modules may have compatibility issues
+   - Consider using raw resources for critical infrastructure if module compatibility is uncertain
+   - Example: Some terraform-aws-modules versions have been found to have compatibility issues with Stacks (e.g., ALB and ECS modules)
+3. **State Isolation**: Each deployment has its own isolated state
+4. **Input Variables**: Use variables for values that differ across deployments; use locals for shared values
+5. **Provider Lock Files**: Always generate and commit `.terraform.lock.hcl` to version control
+6. **Naming Conventions**: Use descriptive names for components and deployments
+7. **Deployment Groups**: You can organize deployments into deployment groups. Deployment groups enable auto-approval rules, logical organization, and provide a foundation for scaling. Deployment groups are an HCP Terraform Premium tier feature
+8. **Testing**: Test Stack configurations in dev/staging deployments before production
+
+## Troubleshooting
+
+**Circular Dependencies**: Refactor to break circular references or use intermediate components.
+
+**Deployment Destruction**: Cannot destroy from UI. Set `destroy = true` in deployment block, upload configuration, and HCP Terraform creates a destroy run.
+
+**Empty Diagnostics**: Add required `stack_deployment_step_id` query parameter to diagnostics API requests.
+
+**Module Compatibility**: Test public registry modules before production use. Some modules may have compatibility issues with Stacks.
+
+## References
+
+For detailed documentation, see:
+- `references/component-blocks.md` - Complete component block reference with all arguments and syntax
+- `references/deployment-blocks.md` - Complete deployment block reference with all configuration options
+- `references/linked-stacks.md` - Publish outputs and upstream inputs for linking Stacks together
+- `references/examples.md` - Complete working examples for multi-region and component dependencies
+- `references/api-monitoring.md` - Full API workflow for programmatic monitoring and automation
+- `references/troubleshooting.md` - Detailed troubleshooting guide for common issues and solutions