@stackgenhq/backstage-plugin-stackgen-backend 0.3.6-beta.1

package/README.md

@@ -0,0 +1,485 @@
+# Backstage Plugin StackGen Backend
+
+Published to **[npmjs.org](https://www.npmjs.com/package/@stackgenhq/backstage-plugin-stackgen-backend)** as `@stackgenhq/backstage-plugin-stackgen-backend`. Install works without a GitHub PAT. Release process: [docs/PUBLISH.md](docs/PUBLISH.md).
+
+## Installation
+
+This plugin is installed via the `@stackgenhq/backstage-plugin-stackgen-backend` package. To install it to your backend package, run the following command:
+
+```bash
+# From your root directory
+yarn --cwd packages/backend add @stackgenhq/backstage-plugin-stackgen-backend
+```
+
+No `.yarnrc.yml` GitHub Packages scope is required for released versions.
+Then add the plugin to your backend in `packages/backend/src/index.ts`:
+
+```ts
+const backend = createBackend();
+// ...
+backend.add(import('@stackgenhq/backstage-plugin-stackgen-backend'));
+```
+
+
+add configuration required for plugin in backstage app-config.yaml
+```yaml
+stackGen:
+  # The base URL for the StackGen API. Default: `https://cloud.stackgen.com`
+  baseUrl: "${BACKSTAGE_ADAPTER_URL}"
+  apiToken: "${STACKGEN_PAT}"
+  # Optional: role assigned when createAppStack auto-adds PAT user to project/team
+  # Defaults to admin role 00000000-0000-0000-0000-000000000002
+  projectMemberRoleId: "${STACKGEN_PROJECT_MEMBER_ROLE_ID}"
+  allowedTeams: ""
+```
+
+allowedTeams is optional so by default, all the teams the STACKGEN_PAT has access will be available.
+Access can be restricted to a few teams by adding their ids in the configuration. exa:
+
+```yaml
+stackGen:
+  baseUrl: "${BACKSTAGE_ADAPTER_URL}" 
+  apiToken: "${STACKGEN_PAT}"
+  allowedTeams: "20f0e211-15ce-4d2c-1e12-0555bffee7bd,20f0e211-15ce-4d2c-2e22-0555bffee7bd"
+```
+
+## API specification
+
+The canonical Integrations Gateway API spec is maintained in the [integrations](https://github.com/appcd-dev/integrations) repo:
+
+**https://github.com/appcd-dev/integrations/blob/main/api-docs/api-spec.yml**
+
+Use that spec as the source of truth for request/response schemas and endpoints. This plugin is not generated from the spec; implementation is in code and may be updated independently.
+
+## Compatibility
+
+| Package | Tested / required notes |
+|---------|-------------------------|
+| `@backstage/plugin-scaffolder-node` | Declared dependency **`^0.12.5`** (tested). Schema declarations use the **function form** (`schema.input: () => zodObject` or `(z) => z.object(…)`) required by `parseSchemas`. Do not pass raw `ZodObject` instances — modern scaffolder-node ignores them and leaves `schema.input` / `schema.output` undefined. Older scaffolder-node **0.6.x** accepted raw Zod objects via `safeParseAsync` detection; that path is obsolete. |
+| Node.js | `>=20` |
+
+**ENG-3394:** Releases **≤0.3.2** (built against scaffolder-node 0.6.x with raw ZodObject schemas) show blank input fields on Backstage “Installed actions” when paired with scaffolder-node **≥0.8** (including **0.12.5**) and skip scaffolder upfront validation. Fixed in **0.3.3** (package track `0.3.3-beta.3+`): function-form schemas + scaffolder-node `^0.12.5`.
+
+## Available Actions
+
+### Setup
+
+To use these actions, add the following code in your backend in `packages/backend/src/index.ts`:
+
+```ts
+import { createAppStackAction, createProjectAction, downloadIaCAction, exportAppStackToGitAction } from '@stackgenhq/backstage-plugin-stackgen-backend';
+import { scaffolderActionsExtensionPoint } from '@backstage/plugin-scaffolder-node';
+import { coreServices, createBackendModule } from '@backstage/backend-plugin-api';
+
+...
+
+export const scaffolderCustomExtension = createBackendModule({
+  pluginId: 'scaffolder',
+  moduleId: 'custom-extensions',
+  register(env) {
+    env.registerInit({
+      deps: {
+        scaffolder: scaffolderActionsExtensionPoint,
+        config: coreServices.rootConfig,
+        logger: coreServices.logger,
+      },
+      async init({ scaffolder, config, logger }) {
+        scaffolder.addActions(
+          createProjectAction(config, logger),
+          createAppStackAction(config, logger),
+          downloadIaCAction(config, logger),
+          exportAppStackToGitAction(config, logger),
+          // ...other actions
+        );
+      },
+    });
+  },
+});
+
+...
+
+backend.add(scaffolderCustomExtension);
+```
+
+### stackGen:createProject
+
+Creates a Project in StackGen with optional git and environment configuration.
+
+A project in StackGen maps to a team. After creation, the `teamId` output can be passed to `stackGen:createAppStack` to create an appstack under that project/team.
+
+**Inputs:**
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `project.name` | string | yes | Project name |
+| `project.description` | string | yes | Project description |
+| `project.gitConfig` | object | no | Default git configuration (scmType, repoUrl, targetBranch, path, vaultReference) |
+| `project.environmentConfig` | object | no | Default environment configuration (templates, variables, overrides) |
+
+**Outputs:**
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `projectId` | string | UUID of the created project |
+| `projectName` | string | Name of the project |
+| `teamId` | string | UUID of the team (same as project; use this for appstack creation) |
+| `alreadyExisted` | boolean | `true` if a project with this name already existed (409), `false` if newly created (201) |
+
+**Status code handling:**
+
+- **201** -- Project created. Outputs `projectId`, `projectName`, `teamId`, `alreadyExisted: false`.
+- **409** -- Project already exists. Treated as success. Outputs `projectId`, `projectName`, `teamId`, `alreadyExisted: true`.
+- **403** -- Throws error: invalid token permissions.
+- **400** -- Throws error: invalid project name.
+
+Important behavior for project initialization:
+
+- Auxiliary initialization settings (for example environment templates, variables, overrides, default configs like `gitConfig`/`environmentConfig`) are only applied when the plugin actually creates a new project.
+- If the requested project already exists (`409` path), the action returns that project/team identifiers and proceeds, but it does **not** patch/update the existing project with those extra settings.
+- Updating existing projects with those parameters is planned for a future update.
+
+Example:
+```yaml
+steps:
+  - id: create-project
+    name: Create StackGen Project
+    action: stackGen:createProject
+    input:
+      project:
+        name: ${{ parameters.projectName }}
+        description: "Project created via Backstage"
+```
+
+Example with git and environment config:
+```yaml
+steps:
+  - id: create-project
+    name: Create StackGen Project
+    action: stackGen:createProject
+    input:
+      project:
+        name: ${{ parameters.projectName }}
+        description: "Project created via Backstage"
+        gitConfig:
+          name: "default-git-config"
+          type: "git"
+          parameters:
+            scmType: "GITHUB"
+            repoUrl: "https://github.com/org/repo.git"
+            targetBranch: "main"
+          vaultReference: "550e8400-e29b-41d4-a716-446655440000"
+          default: true
+        environmentConfig:
+          name: "default-env-config"
+          environmentTemplates:
+            - name: "production"
+              color: "#FF0000"
+              stateBackendTemplate:
+                type: "S3"
+                configJson: '{"bucket": "my-terraform-state"}'
+          variables:
+            - name: "region"
+              type: "string"
+              defaultValueJson: '"us-east-1"'
+              description: "AWS region for deployment"
+          variableOverrides:
+            - variableName: "region"
+              environmentTemplateName: "production"
+              valueJson: '"us-west-2"'
+```
+
+Environment variable value types (`defaultValueJson` and `valueJson`):
+
+- These fields are **JSON literals encoded as strings** in YAML.
+- The plugin passes them through to StackGen, so you can use non-string JSON values.
+- Set `variables[].type` to match your intended semantic type in StackGen.
+
+Examples:
+
+```yaml
+variables:
+  - name: "region"
+    type: "string"
+    defaultValueJson: '"us-east-1"'
+  - name: "replicas"
+    type: "number"
+    defaultValueJson: '3'
+  - name: "enableAutoscaling"
+    type: "boolean"
+    defaultValueJson: 'true'
+  - name: "subnetIds"
+    type: "list"
+    defaultValueJson: '["subnet-1","subnet-2"]'
+  - name: "extraTags"
+    type: "object"
+    defaultValueJson: '{"team":"platform","env":"dev"}'
+
+variableOverrides:
+  - variableName: "replicas"
+    environmentTemplateName: "prod"
+    valueJson: '5'
+```
+
+### stackGen:createAppStack
+
+Creates an AppStack in StackGen for a given team with specified resources and configurations.
+
+Behavior by input type:
+
+- **Before appstack creation**: action resolves the PAT user via `GET /appcd/api/v1/auth/me` and attempts to assign that user to the target project/team (`appstack.teamId`) via `POST /appcd/api/v1/orgs/{projectId}/users` using role `projectMemberRoleId` (default admin). If this preflight assignment fails, appstack creation still proceeds and the membership error is logged as a warning—this usually happens if the PAT user was already a member of the project. If the PAT was newly added, it **stays** a project member after the action completes (no automatic removal).
+- **When no `resourcePackId` is present**: uses the legacy create path `POST /stackgen/v1/appstacks` with payload shape `name`, `coreConfig` (provider, targetCompute), `resources`, and `teamId`.
+- **When `templateAppstackId` is set** (and `resources` is empty): loads the source AppStack via `GET /appcd/api/v1/appstacks/{uuid}`, creates a blank appstack through integrations, then clones topology with `POST /iac-gen/v1/topologies?orgId=…` (`appstackRefId`). Deployment type and IaC type come from the source `coreConfig` (defaults: `k8s`, `Terraform` if missing).
+- **When any `resourcePackId` is present**: uses a two-step integrations-compatible flow:
+  1. Create a blank appstack via `POST /integrations/api/v1/projects/{teamId}/appstacks` (empty topology)
+  2. Add each resource pack into that topology via `POST /iac-gen/v1/topologies/{topologyId}/resources?orgId={orgScope}`, preserving `resourcePackId` semantics.
+
+**Inputs:**
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `appstack.name` or `appstack.appstackName` | string | yes | AppStack name |
+| `appstack.teamId` | string | yes | Team UUID (use `teamId` output from `stackGen:createProject`) |
+| `appstack.cloudProvider` | string | conditional | Required unless `templateAppstackId` is set (then read from appcd `coreConfig.provider`) |
+| `appstack.templateAppstackId` | string (UUID) | no | If set: load source AppStack from appcd, then clone topology via iac-gen (`appstackRefId`). Cannot be used with a non-empty `resources` array |
+| `appstack.resources` | array | yes | List of resources (can be empty `[]`) |
+
+Integrations `POST …/appstacks` only accepts `name`, `cloudProvider`, and `topology` (no extra metadata fields).
+
+Each resource in the array:
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `resourceType` | string | yes | Valid StackGen resource type |
+| `configuration` | object | no | Resource configuration (mutually exclusive with `resourcePackId`) |
+| `tfVars` | object | no | Terraform variables |
+| `resourcePackId` | string | no | Resource pack ID (mutually exclusive with `configuration`) |
+| `children` | array | no | Child resources for group resource types |
+
+**Outputs:**
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `appstackId` | string | UUID of the created appstack |
+| `appstackName` | string | Name of the created appstack |
+| `appStackURL` | string | Direct URL to the created appstack in StackGen |
+
+Example (chained with project creation):
+```yaml
+steps:
+  - id: create-project
+    name: Create StackGen Project
+    action: stackGen:createProject
+    input:
+      project:
+        name: ${{ parameters.projectName }}
+        description: "Project created via Backstage"
+
+  - id: stackGen
+    name: Create AppStack
+    action: stackGen:createAppStack
+    input:
+      appstack:
+        name: "MyAppStack"
+        # teamId comes from the output of the create-project step
+        teamId: ${{ steps['create-project'].output.teamId }}
+        cloudProvider: "aws"
+        resources:
+          - resourceType: "aws_s3"
+            # Only one of 'configuration' or 'resourcePackId' should be provided
+            configuration:
+              bucket_name: "mybucket"
+          - resourceType: "resourcePack"
+            resourcePackId: "20f0e212-15ce-4d2c-1e12-0555bffee7bd"
+            # No configuration here because resourcePackId is used (legacy path preserves resourcePackId)
+          - resourceType: "helm_workload"
+            children:
+              - resourceType: "helm_workload"
+                configuration:
+                  name: test-workload
+                  image: test-image
+              - resourceType: helm_service
+                configuration:
+                  name: test-service
+```
+
+Example output:
+```yaml
+output:
+  links:
+    - title: 'View Created AppStack in StackGen'
+      url: '${{ steps.stackGen.output.appStackURL }}'
+```
+
+## Example templates walkthrough
+
+The templates in `examples/template/` demonstrate end-to-end StackGen scaffolding flows:
+
+| Template | Description |
+|----------|-------------|
+| `stackgen-export-iac` | Manually fetch IaC from an AppStack and push it to GitHub via Backstage — useful when you prefer Backstage to handle the git push instead of StackGen's native git integration. |
+| `stackgen-appstack-with-resources` | Creates a project with environments, then an AppStack with a resource pack, standalone S3 bucket, and custom Terraform module (via `templateId`). |
+| `stackgen-full-setup` | The most comprehensive template: project + Git config + AppStack (resource pack + S3 + custom module) + production/staging environments with per-environment TF variables + S3 state backends. |
+| `stackgen-project-basic` | Creates an empty project — a blank starting point. |
+| `stackgen-project-advanced` | Creates a project with Git integration, environments, variables, and per-environment overrides. |
+
+Project initialization in these templates can include:
+
+- Environment templates (for example `dev`, `stage`, `prod`).
+- Default variables and per-environment variable overrides.
+- Git configuration via `project.gitConfig`.
+- Per-environment storage backend configuration via `stateBackendTemplate`.
+
+## Disabling the Frontend Plugin
+
+The StackGen plugin includes an optional frontend component that adds a "StackGen" item to the Backstage sidebar. If you only need the backend scaffolder actions (templates) and don't want the sidebar entry, you can toggle it off by commenting out two lines:
+
+**1. Remove the sidebar item** in `packages/app/src/components/Root/Root.tsx`:
+```tsx
+// Comment out this line:
+{/* <SidebarItem icon={StackGenIcon} to="stackgen" text="StackGen"/> */}
+```
+
+**2. Remove the route** in `packages/app/src/App.tsx`:
+```tsx
+// Comment out this line:
+{/* <Route path="/stackgen" element={<StackGenPage />} /> */}
+```
+
+The imports can remain in place — they'll be unused but won't cause errors. To re-enable the frontend later, simply uncomment both lines and restart Backstage.
+
+
+### stackGen:downloadIaC
+
+This action downloads and extracts the Infrastructure as Code (IaC) files for a specified AppStack from StackGen.
+
+Example input:
+```yaml
+steps:
+  - id: download-iac
+    name: Download IaC
+    action: stackGen:downloadIaC
+    input:
+      appstackName: "MyAppStack"
+      # Optional: teamId can be omitted if not needed
+      teamId: "team-123"
+      # Optional: path where IaC files should be extracted, defaults to "./"
+      extractedIaCPath: "./infrastructure"
+```
+### stackGen:exportToGit
+
+This action is a compatibility wrapper over the new stack-exporter flow.
+It resolves topology for `teamId + appstackName`, selects an existing git exporter
+config for the team (or uses an explicit id), and triggers exporter `POST /export`.
+
+**Inputs (high level)**
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `appstackName`, `teamId` | yes | AppStack and team / org id |
+| `configId` | no | When set (non-empty), used as stack-exporter git `config_id`; skips listing, `chooseConfig`, and bootstrap. **Precedence:** `configId` overrides `gitConfig`. In Software Templates, map from the frontend field **`existingExporterConfigId`** (`ExporterGitConfigSelect`). |
+| `topologyId`, `appstackId` | no | Optional topology / appstack hints for resolution |
+| `gitConfig` | no | Bootstrap shape (same as `createProject.project.gitConfig`). Ignored when `configId` is set. |
+| `baseBranch` | no | Branch to base the change on (PR base / commit destination). Maps to the exporter `target_branch` override. Blank uses the git config default. Prefill via `ExporterBaseBranch`. |
+| `pushBranch` | no | Branch StackGen pushes the generated IaC to (source/head branch). Maps to the exporter `branch_format` override. Blank creates a new one. Prefill via `ExporterPushBranch.branch`. |
+| `useSameBranch` | no | When `true`, sets exporter `use_same_branch=true` (reuse previous export head). Set by `ExporterPushBranch` when the history suggestion is left unchanged. |
+| `createPr` | no | Boolean, default `true`. When `false`, changes are committed directly to the branch without opening a PR (exporter `commit_only`). |
+| `overrides` | no | Raw exporter overrides (e.g. `pr_title`, `commit_msg`). An explicit entry wins over the derived key for the same override. |
+
+These give the Backstage plugin parity with the StackGen UI git export (base branch, push branch, create PR). `baseBranch`, `pushBranch`, `useSameBranch`, and `createPr` are translated to exporter override keys in one place (`buildExportOverrides`), so templates do not need to know the internal keys.
+
+**Outputs**
+
+| Field | When present | Description |
+|-------|--------------|-------------|
+| `pullRequestUrl`, `externalLink` | PR created (`createPr` true) | URL of the opened pull request |
+| `committedBranch` | commit-only (`createPr` false) | Branch StackGen committed the changes to |
+| `notes` | always (when returned) | Human-readable export notes (e.g. "committed to branch X") |
+
+> A commit-only export (`createPr: false`) does not open a pull request, so `pullRequestUrl` is absent; use `committedBranch` / `notes` in that case.
+
+Notes:
+- **Explicit `configId`:** Use when the user picks a config in Backstage (`ExporterGitConfigSelect` writes the config uuid to `existingExporterConfigId`; pass it as `configId` in the action input). **Precedence:** `configId` overrides `gitConfig`.
+- **Listing configs:** If `configId` and `gitConfig` are omitted, `GET .../exporter/.../config?orgId={teamId}&type=git&appstackId={appstackId}` — if a single/default git config exists for that AppStack, it is used for `POST .../export`. Multiple configs require `configId`.
+- **Bootstrap (no git config for the AppStack yet):** Project-level `gitConfig` from `stackGen:createProject` is stored as stack-exporter **templates** (`GET .../exporter/.../config/template?orgId={teamId}`). When the list step finds nothing usable, this action loads that template and `POST .../exporter/.../config?orgId={teamId}` **with `appstack_id`** so the config is scoped to the AppStack.
+- **Optional `gitConfig` on this action:** Same shape as `createProject.project.gitConfig`. When provided (and `configId` is not set), creates a new appstack-scoped config from that input and uses it. Not used when `configId` is set.
+- **`appstackId`:** Optional. When omitted, resolved from `teamId` + `appstackName` via appcd `GET /appstacks` (`appstackId` for exporter binding, `uuid` for topology).
+
+Example input:
+```yaml
+  steps:
+    - id: exportToGit
+      name: Export AppStack to Git via StackGen
+      action: stackGen:exportToGit
+      input:
+        appstackName: ${{ parameters.appstackName }}
+        teamId: ${{ parameters.teamId }}
+        # Optional: explicit exporter git config (from ExporterGitConfigSelect → existingExporterConfigId)
+        configId: ${{ parameters.existingExporterConfigId }}
+        # gitConfig omitted: uses project template from createProject when no appstack config exists
+        # First-class git export controls (parity with the StackGen UI):
+        baseBranch: ${{ parameters.baseBranch }}   # PR base / commit destination (target_branch)
+        pushBranch: ${{ parameters.pushBranch.branch }}   # branch StackGen pushes to (branch_format)
+        useSameBranch: ${{ parameters.pushBranch.useSameBranch }}  # reuse history head when unchanged
+        createPr: ${{ parameters.createPr }}       # false => commit_only (no PR)
+        overrides:
+          - key: pr_title
+            value: Export IaC from Backstage
+          - key: commit_msg
+            value: chore: export iac
+```
+
+
+Example output:
+```
+output:
+  links:
+    - title: 'Pull Request'
+      url: '${{ steps.exportToGit.output.pullRequestUrl }}'
+```
+
+## Development
+
+Use the [stackgen-backstage](https://github.com/appcd-dev/stackgen-backstage) app to run Backstage with your local backend plugin build.
+
+1. **Lay out repositories** so the backend plugin and the Backstage app are siblings (adjust paths below if yours differ):
+
+   ```text
+   <workspace>/
+     stackgen-backstage/
+     backstage-plugin-stackgen-backend/
+   ```
+
+2. **Point the backend at your local plugin** in `stackgen-backstage/packages/backend/package.json`:
+
+   Set the dependency to a Yarn link relative to `packages/backend` (three levels up to the parent of `stackgen-backstage`, then into the backend plugin repo):
+
+   ```json
+   "@stackgenhq/backstage-plugin-stackgen-backend": "link:../../../backstage-plugin-stackgen-backend"
+   ```
+
+   If your clone lives elsewhere, change the `link:` path accordingly (it must resolve from `packages/backend` to the root of `backstage-plugin-stackgen-backend`).
+
+3. **Build the backend plugin**:
+
+   ```bash
+   yarn install
+   yarn tsc && yarn build
+   ```
+
+4. **Install and run stackgen-backstage** from the Backstage repo root:
+
+   ```bash
+   yarn install
+   ```
+
+   Configure `app-config.yaml` (or local overrides) with `stackGen.baseUrl`, `stackGen.apiToken`, and any optional keys you need (see [Installation](#installation)). The stackgen-backstage README also documents env vars such as `STACKGEN_API_TOKEN` if you wire config that way.
+
+5. **Start the app** (frontend + backend in parallel):
+
+   ```bash
+   yarn dev
+   ```
+
+   Alternatively, run workspaces separately: `yarn start-backend` and `yarn start` (app only), as described in the stackgen-backstage `package.json` scripts.
+
+6. **After backend plugin code changes**, rebuild the plugin (`yarn tsc && yarn build` in `backstage-plugin-stackgen-backend`), then install the dependencies again.

package/config.d.ts

@@ -0,0 +1,39 @@
+export interface Config {
+  /**
+   * @visibility backend
+   */
+  stackGen: {
+    /**
+     * @visibility secret
+     */
+    baseUrl: string;
+    /**
+     * @visibility secret
+     */
+    apiToken: string;
+    /**
+     * @visibility backend
+     */
+    allowedTeams: string;
+    /**
+     * Default StackGen org id when actions omit orgId (optional).
+     * @visibility backend
+     */
+    orgId?: string;
+    /**
+     * Base path for iac-gen HTTP API (default /iac-gen).
+     * @visibility backend
+     */
+    iacGenBasePath?: string;
+    /**
+     * Base path for stack-exporter HTTP API (default /exporter/api/v1).
+     * @visibility backend
+     */
+    exporterBasePath?: string;
+    /**
+     * Base path for stackgen-vault HTTP API (default /api/vault/v1).
+     * @visibility backend
+     */
+    vaultBasePath?: string;
+  };
+}