@aifabrix/builder 2.36.2 → 2.37.0

package/README.md

@@ -1,9 +1,13 @@
-# AI Fabrix - Builder SDK
+# AI Fabrix - Builder
 
 [![npm version](https://img.shields.io/npm/v/@aifabrix/builder.svg)](https://www.npmjs.com/package/@aifabrix/builder)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-Local development infrastructure + Azure deployment tool.
+Install the AI Fabrix platform and test it locally. Then add external integrations or build your own applications.
+
+← **Full documentation:** [docs/README.md](docs/README.md) (table of contents for all guides)
+
+---
 
 ## Install
 
@@ -11,126 +15,86 @@ Local development infrastructure + Azure deployment tool.
 npm install -g @aifabrix/builder
 ```
 
-## Quick Start
+**Alias:** You can use `aifx` instead of `aifabrix` in any command.
 
-```bash
-aifabrix up              # Start Postgres + Redis
-aifabrix create myapp    # Create your app
-aifabrix build myapp     # Build Docker image
-aifabrix run myapp       # Run locally
-# Stop the app (optionally remove its data volume)
-aifabrix down myapp
-# aifabrix down myapp --volumes
-```
+---
 
-→ [Full Guide](docs/quick-start.md) | [CLI Commands](docs/cli-reference.md)
+## Goal 1: Start and test the AI Fabrix platform
 
-## What You Get
+Get the platform running locally so you can try it.
 
-- **Local Postgres + Redis infrastructure** - Runs in Docker
-- **Auto-generated Dockerfiles** - TypeScript and Python templates
-- **Environment variable management** - Secret resolution with kv:// references
-- **Azure deployment pipeline** - Push to ACR and deploy via controller
+1. **Start local infrastructure** (Postgres, Redis, optional Traefik):
 
-## Optional Platform Apps
+   ```bash
+   aifabrix up-infra
+   ```
 
-Want authentication or deployment controller?
+2. **Start the platform** (Keycloak, Miso Controller, Dataplane) from community images:
 
-**Quick install from images (no build):**
-```bash
-aifabrix up              # Start Postgres + Redis first
-aifabrix up-miso         # Install Keycloak + Miso Controller from images (auto-generated secrets for testing)
-```
+   ```bash
+   aifabrix up-platform
+   ```
 
-**Or create and build from templates:**
-```bash
-# Keycloak for authentication
-aifabrix create keycloak --port 8082 --database --template keycloak
-aifabrix build keycloak
-aifabrix run keycloak
-
-# Miso Controller for Azure deployments
-aifabrix create miso-controller --port 3000 --database --redis --template miso-controller
-aifabrix build miso-controller
-aifabrix run miso-controller
-```
+   Or run platform apps separately: `aifabrix up-miso` then `aifabrix up-dataplane`. Infra must be up first.
 
-**Dataplane in dev (after login):**
-```bash
-aifabrix login --environment dev
-aifabrix up-dataplane    # Register or rotate, run, and deploy dataplane in dev
-```
+3. **Configure secrets** – You need either **OpenAI** or **Azure OpenAI**:
+
+   - **OpenAI:** set your API key:
+     ```bash
+     aifabrix secrets set secrets-openaiApiKeyVault <your-openai-secret-key>
+     ```
+   - **Azure OpenAI:** set endpoint and API key:
+     ```bash
+     aifabrix secrets set azure-openaiapi-urlKeyVault <your-azure-openai-endpoint-url>
+     aifabrix secrets set secrets-azureOpenaiApiKeyVault <your-azure-openai-secret-key>
+     ```
+
+Secrets are stored in `~/.aifabrix/secrets.local.yaml` or the file from `aifabrix-secrets` in your config (e.g. `builder/secrets.local.yaml`).
+
+→ [Infrastructure guide](docs/infrastructure.md)
 
-→ [Infrastructure Guide](docs/infrastructure.md)
+---
+
+## Goal 2: External system integration
+
+Create and deploy an external system (e.g. HubSpot): wizard or manual setup, then validate and deploy.
+
+**Example: HubSpot**
+
+- Create: `aifabrix create hubspot-test --type external` (or `aifabrix wizard` for guided setup).
+- Configure auth and datasources under `integration/hubspot-test/`.
+- Validate: `aifabrix validate hubspot-test`
+- Deploy: `aifabrix deploy hubspot-test`
+
+→ [External systems guide](docs/external-systems.md) · [Wizard](docs/wizard.md)
+
+---
+
+## Goal 3: Build your own application
+
+Create, configure, and run your own AI Fabrix application locally or deploy it (create app → configure → build → run / deploy).
+
+→ [Your own applications](docs/your-own-applications.md)
+
+---
 
 ## Documentation
 
-- [Quick Start](docs/quick-start.md) - Get running in 5 minutes
-- [Infrastructure](docs/infrastructure.md) - What runs and why
-- [Configuration](docs/configuration.md) - Config file reference
-- [Building](docs/building.md) - Build process explained
-- [Running](docs/running.md) - Run apps locally
-- [Deploying](docs/deploying.md) - Deploy to Azure
-- [CLI Reference](docs/cli-reference.md) - All commands
-
-## How It Works
-
-1. **Infrastructure** - Minimal baseline (Postgres + Redis)
-2. **Create** - Generate config files for your app
-3. **Build** - Auto-detect runtime and build Docker image
-4. **Run** - Start locally, connected to infrastructure
-5. **Deploy** - Push to ACR and deploy via controller
-
-```mermaid
-%%{init: {
-  "theme": "base",
-  "themeVariables": {
-    "fontFamily": "Poppins, Arial Rounded MT Bold, Arial, sans-serif",
-    "fontSize": "16px",
-    "background": "#FFFFFF",
-    "primaryColor": "#F8FAFC",
-    "primaryTextColor": "#0B0E15",
-    "primaryBorderColor": "#E2E8F0",
-    "lineColor": "#E2E8F0",
-    "textColor": "#0B0E15",
-    "borderRadius": 16
-  },
-  "flowchart": {
-    "curve": "linear",
-    "nodeSpacing": 34,
-    "rankSpacing": 34,
-    "padding": 10
-  }
-}}%%
-
-flowchart TD
-
-%% =======================
-%% Styles
-%% =======================
-classDef base fill:#FFFFFF,color:#0B0E15,stroke:#E2E8F0,stroke-width:1.5px;
-classDef primary fill:#0062FF,color:#ffffff,stroke-width:0px;
-
-%% =======================
-%% Flow
-%% =======================
-Install[Install CLI]:::primary --> Up[Start Infrastructure]:::base
-Up --> Create[Create App]:::base
-Create --> Build[Build Image]:::base
-Build --> Run[Run Locally]:::base
-Run --> Deploy[Deploy to Azure]:::primary
-```
+All guides and references are listed in **[docs/README.md](docs/README.md)** (table of contents).
 
-## Development
+- [CLI Reference](docs/cli-reference.md) – All commands
+- [Infrastructure](docs/infrastructure.md) – What runs and why
+- [Configuration](docs/configuration.md) – Config files
 
-- **Tests**: `npm test` (runs via wrapper; handles known Jest/Node exit issues).
-- **Coverage**: `npm run test:coverage` — runs tests with coverage through the same wrapper. May take 3–5 minutes for the full suite. If the process exits with a signal after "Ran all test suites", the wrapper treats it as success and coverage is written to `coverage/`. Use `test:coverage:nyc` only if you need nyc-specific reporters.
+---
 
 ## Requirements
 
-- **Docker Desktop** - For running containers
-- **Node.js 18+** - For running the CLI
-- **Azure CLI** - For deploying to Azure (optional)
+- **Docker Desktop** – For running containers
+- **Node.js 18+** – For running the CLI
+- **Azure CLI** – For deploying to Azure (optional)
+
+---
 
 ## License
 

package/lib/app/display.js

@@ -50,7 +50,7 @@ function displayWebappSuccess(appName, config, envConversionMessage) {
 
   logger.log(chalk.green('\nNext steps:'));
   logger.log(chalk.white('1. Copy env.template to .env and fill in your values'));
-  logger.log(chalk.white('2. Run: aifabrix up'));
+  logger.log(chalk.white('2. Run: aifabrix up-infra'));
   logger.log(chalk.white('3. Run: aifabrix build ' + appName));
   logger.log(chalk.white('4. Run: aifabrix run ' + appName));
 }

package/lib/app/run-helpers.js

@@ -177,7 +177,7 @@ async function checkPrerequisites(appName, appConfig, debug = false, skipInfraCh
     .map(([service, _]) => service);
 
   if (unhealthyServices.length > 0) {
-    throw new Error(`Infrastructure services not healthy: ${unhealthyServices.join(', ')}\nRun 'aifabrix up' first`);
+    throw new Error(`Infrastructure services not healthy: ${unhealthyServices.join(', ')}\nRun 'aifabrix up-infra' first`);
   }
   logger.log(chalk.green('✓ Infrastructure is running'));
 }

package/lib/cli/index.js

@@ -0,0 +1,45 @@
+/**
+ * AI Fabrix Builder CLI Command Definitions
+ *
+ * This module wires all CLI command setup and re-exports setupCommands,
+ * validateCommand, and handleCommandError for backward compatibility.
+ *
+ * @fileoverview CLI entry for AI Fabrix Builder; command setup orchestration
+ * @author AI Fabrix Team
+ * @version 2.0.0
+ */
+
+const { validateCommand, handleCommandError } = require('../utils/cli-utils');
+const { setupAuthCommands } = require('./setup-auth');
+const { setupInfraCommands } = require('./setup-infra');
+const { setupAppCommands } = require('./setup-app');
+const { setupEnvironmentCommands } = require('./setup-environment');
+const { setupUtilityCommands } = require('./setup-utility');
+const { setupDevCommands } = require('./setup-dev');
+const { setupSecretsCommands } = require('./setup-secrets');
+const { setupExternalSystemCommands } = require('./setup-external-system');
+const { setupAppCommands: setupAppManagementCommands } = require('../commands/app');
+const { setupDatasourceCommands } = require('../commands/datasource');
+
+/**
+ * Sets up all CLI commands on the Commander program instance
+ * @param {Command} program - Commander program instance
+ */
+function setupCommands(program) {
+  setupInfraCommands(program);
+  setupAuthCommands(program);
+  setupAppCommands(program);
+  setupEnvironmentCommands(program);
+  setupAppManagementCommands(program);
+  setupDatasourceCommands(program);
+  setupUtilityCommands(program);
+  setupExternalSystemCommands(program);
+  setupDevCommands(program);
+  setupSecretsCommands(program);
+}
+
+module.exports = {
+  setupCommands,
+  validateCommand,
+  handleCommandError
+};

package/lib/cli/setup-app.js

@@ -0,0 +1,229 @@
+/**
+ * CLI application lifecycle command setup (create, wizard, build, run, push, deploy, dockerfile).
+ *
+ * @fileoverview Application command definitions for AI Fabrix Builder CLI
+ * @author AI Fabrix Team
+ * @version 2.0.0
+ */
+
+const chalk = require('chalk');
+const path = require('path');
+const app = require('../app');
+const logger = require('../utils/logger');
+const { handleCommandError } = require('../utils/cli-utils');
+
+/**
+ * Normalize options for external system creation
+ * @param {Object} options - Raw CLI options
+ * @returns {Object} Normalized options
+ */
+function normalizeExternalOptions(options) {
+  const normalized = { ...options };
+  if (options.displayName) normalized.systemDisplayName = options.displayName;
+  if (options.description) normalized.systemDescription = options.description;
+  if (options.systemType) normalized.systemType = options.systemType;
+  if (options.authType) normalized.authType = options.authType;
+  if (options.datasources !== undefined) {
+    const parsedCount = parseInt(options.datasources, 10);
+    if (Number.isNaN(parsedCount) || parsedCount < 1 || parsedCount > 10) {
+      throw new Error('Datasources count must be a number between 1 and 10');
+    }
+    normalized.datasourceCount = parsedCount;
+  }
+  if (options.controller) {
+    normalized.controller = true;
+    normalized.controllerUrl = options.controller;
+  }
+  return normalized;
+}
+
+/**
+ * Validate required options for non-interactive external creation
+ * @param {Object} normalizedOptions - Normalized options
+ * @throws {Error} If required options are missing
+ */
+function validateNonInteractiveExternalOptions(normalizedOptions) {
+  const missing = [];
+  if (!normalizedOptions.systemDisplayName) missing.push('--display-name');
+  if (!normalizedOptions.systemDescription) missing.push('--description');
+  if (!normalizedOptions.systemType) missing.push('--system-type');
+  if (!normalizedOptions.authType) missing.push('--auth-type');
+  if (!normalizedOptions.datasourceCount) missing.push('--datasources');
+  if (missing.length > 0) {
+    throw new Error(`Missing required options for non-interactive external create: ${missing.join(', ')}`);
+  }
+  if (!Object.prototype.hasOwnProperty.call(normalizedOptions, 'github')) {
+    normalizedOptions.github = false;
+  }
+  if (!Object.prototype.hasOwnProperty.call(normalizedOptions, 'controller')) {
+    normalizedOptions.controller = false;
+  }
+}
+
+/**
+ * Handle create command execution
+ * @async
+ * @param {string} appName - Application name
+ * @param {Object} options - CLI options
+ */
+async function handleCreateCommand(appName, options) {
+  const validTypes = ['webapp', 'api', 'service', 'functionapp', 'external'];
+  if (options.type && !validTypes.includes(options.type)) {
+    throw new Error(`Invalid type: ${options.type}. Must be one of: ${validTypes.join(', ')}`);
+  }
+
+  const wizardOptions = { app: appName, ...options };
+  const normalizedOptions = normalizeExternalOptions(options);
+
+  const isExternalType = options.type === 'external';
+  const isNonInteractive = process.stdin && process.stdin.isTTY === false;
+
+  if (isExternalType && !options.wizard && isNonInteractive) {
+    validateNonInteractiveExternalOptions(normalizedOptions);
+  }
+
+  const shouldUseWizard = options.wizard && (options.type === 'external' || (!options.type && validTypes.includes('external')));
+  if (shouldUseWizard) {
+    const { handleWizard } = require('../commands/wizard');
+    await handleWizard(wizardOptions);
+  } else {
+    await app.createApp(appName, normalizedOptions);
+  }
+}
+
+/**
+ * Sets up application lifecycle commands
+ * @param {Command} program - Commander program instance
+ */
+function setupAppCommands(program) {
+  program.command('create <app>')
+    .description('Create new application with configuration files')
+    .option('-p, --port <port>', 'Application port', '3000')
+    .option('-d, --database', 'Requires database')
+    .option('-r, --redis', 'Requires Redis')
+    .option('-s, --storage', 'Requires file storage')
+    .option('-a, --authentication', 'Requires authentication/RBAC')
+    .option('-l, --language <lang>', 'Runtime language (typescript/python)')
+    .option('-t, --template <name>', 'Template to use (e.g., miso-controller, keycloak)')
+    .option('--type <type>', 'Application type (webapp, api, service, functionapp, external)', 'webapp')
+    .option('--app', 'Generate minimal application files (package.json, index.ts or requirements.txt, main.py)')
+    .option('-g, --github', 'Generate GitHub Actions workflows')
+    .option('--github-steps <steps>', 'Extra GitHub workflow steps (comma-separated, e.g., npm,test)')
+    .option('--main-branch <branch>', 'Main branch name for workflows', 'main')
+    .option('--wizard', 'Use interactive wizard for external system creation')
+    .option('--display-name <name>', 'External system display name')
+    .option('--description <desc>', 'External system description')
+    .option('--system-type <type>', 'External system type (openapi, mcp, custom)')
+    .option('--auth-type <type>', 'External system auth type (oauth2, apikey, basic)')
+    .option('--datasources <count>', 'Number of datasources to create')
+    .action(async(appName, options) => {
+      try {
+        await handleCreateCommand(appName, options);
+      } catch (error) {
+        handleCommandError(error, 'create');
+        process.exit(1);
+      }
+    });
+
+  program.command('wizard [appName]')
+    .description('Create or extend external systems (OpenAPI, MCP, or known platforms like HubSpot) via guided steps or a config file')
+    .option('-a, --app <app>', 'Application name (synonym for positional appName)')
+    .option('--config <file>', 'Run headless using a wizard.yaml file (appName, mode, source, credential, preferences)')
+    .option('--silent', 'Run with saved integration/<app>/wizard.yaml only; no prompts (requires app name and existing wizard.yaml)')
+    .addHelpText('after', `
+Examples:
+  $ aifabrix wizard                    Run interactively (mode first, then prompts)
+  $ aifabrix wizard my-integration      Load wizard.yaml if present → show summary → "Run with saved config?" or start from step 1
+  $ aifabrix wizard my-integration --silent  Run headless with integration/my-integration/wizard.yaml (no prompts)
+  $ aifabrix wizard -a my-integration   Same as above (app name set)
+  $ aifabrix wizard --config wizard.yaml  Run headless from a wizard config file
+
+Config path: When appName is provided, integration/<appName>/wizard.yaml is used for load/save and error.log.
+To change settings after a run, edit that file and run "aifabrix wizard <app>" again.
+Headless config must include: appName, mode (create-system|add-datasource), source (type + filePath/url/platform).
+See integration/hubspot/wizard-hubspot-e2e.yaml for an example.`)
+    .action(async(positionalAppName, options) => {
+      try {
+        const appName = positionalAppName || options.app;
+        const configPath = appName ? path.join(process.cwd(), 'integration', appName, 'wizard.yaml') : null;
+        const { handleWizard } = require('../commands/wizard');
+        await handleWizard({ ...options, app: appName, config: options.config, configPath });
+      } catch (error) {
+        handleCommandError(error, 'wizard');
+        process.exit(1);
+      }
+    });
+
+  program.command('build <app>')
+    .description('Build container image (auto-detects runtime)')
+    .option('-l, --language <lang>', 'Override language detection')
+    .option('-f, --force-template', 'Force rebuild from template')
+    .option('-t, --tag <tag>', 'Image tag (default: latest). Set image.tag in variables.yaml to match for deploy.')
+    .action(async(appName, options) => {
+      try {
+        const imageTag = await app.buildApp(appName, options);
+        logger.log(`✅ Built image: ${imageTag}`);
+      } catch (error) {
+        handleCommandError(error, 'build');
+        process.exit(1);
+      }
+    });
+
+  program.command('run <app>')
+    .description('Run application locally')
+    .option('-p, --port <port>', 'Override local port')
+    .option('-d, --debug', 'Enable debug output with detailed container information')
+    .action(async(appName, options) => {
+      try {
+        await app.runApp(appName, options);
+      } catch (error) {
+        handleCommandError(error, 'run');
+        process.exit(1);
+      }
+    });
+
+  program.command('push <app>')
+    .description('Push image to Azure Container Registry')
+    .option('-r, --registry <registry>', 'ACR registry URL (overrides variables.yaml)')
+    .option('-t, --tag <tag>', 'Image tag(s) - comma-separated for multiple (default: latest)')
+    .action(async(appName, options) => {
+      try {
+        await app.pushApp(appName, options);
+      } catch (error) {
+        handleCommandError(error, 'push');
+        process.exit(1);
+      }
+    });
+
+  program.command('deploy <app>')
+    .description('Deploy to Azure via Miso Controller')
+    .option('--client-id <id>', 'Client ID (overrides config)')
+    .option('--client-secret <secret>', 'Client Secret (overrides config)')
+    .option('--poll', 'Poll for deployment status', true)
+    .option('--no-poll', 'Do not poll for status')
+    .action(async(appName, options) => {
+      try {
+        await app.deployApp(appName, options);
+      } catch (error) {
+        handleCommandError(error, 'deploy');
+        process.exit(1);
+      }
+    });
+
+  program.command('dockerfile <app>')
+    .description('Generate Dockerfile for an application')
+    .option('-l, --language <lang>', 'Override language detection')
+    .option('-f, --force', 'Overwrite existing Dockerfile')
+    .action(async(appName, options) => {
+      try {
+        const dockerfilePath = await app.generateDockerfileForApp(appName, options);
+        logger.log(chalk.green('\n✅ Dockerfile generated successfully!'));
+        logger.log(chalk.gray(`Location: ${dockerfilePath}`));
+      } catch (error) {
+        handleCommandError(error, 'dockerfile');
+        process.exit(1);
+      }
+    });
+}
+
+module.exports = { setupAppCommands };

package/lib/cli/setup-auth.js

@@ -0,0 +1,88 @@
+/**
+ * CLI authentication command setup (login, logout, auth status/config).
+ *
+ * @fileoverview Authentication command definitions for AI Fabrix Builder CLI
+ * @author AI Fabrix Team
+ * @version 2.0.0
+ */
+
+const chalk = require('chalk');
+const logger = require('../utils/logger');
+const { handleCommandError } = require('../utils/cli-utils');
+const { handleLogin } = require('../commands/login');
+const { handleLogout } = require('../commands/logout');
+const { handleAuthStatus } = require('../commands/auth-status');
+const { handleAuthConfig } = require('../commands/auth-config');
+
+/**
+ * Sets up authentication commands
+ * @param {Command} program - Commander program instance
+ */
+function setupAuthCommands(program) {
+  program.command('login')
+    .description('Authenticate with Miso Controller')
+    .option('-c, --controller <url>', 'Controller URL (default: from config or developer ID, e.g. http://localhost:3000)')
+    .option('-m, --method <method>', 'Authentication method (device|credentials)', 'device')
+    .option('-a, --app <app>', 'Application name (required for credentials method, reads from secrets.local.yaml)')
+    .option('--client-id <id>', 'Client ID (for credentials method, overrides secrets.local.yaml)')
+    .option('--client-secret <secret>', 'Client Secret (for credentials method, overrides secrets.local.yaml)')
+    .option('-e, --environment <env>', 'Environment key (updates root-level environment in config.yaml, e.g., miso, dev, tst, pro)')
+    .option('--online', 'Request online-only token (excludes offline_access scope, device flow only)')
+    .option('--scope <scopes>', 'Custom OAuth2 scope string (device flow only, default: "openid profile email offline_access")')
+    .action(async(options) => {
+      try {
+        await handleLogin(options);
+      } catch (error) {
+        logger.error(chalk.red('\n❌ Login failed:'), error.message);
+        process.exit(1);
+      }
+    });
+
+  program.command('logout')
+    .description('Clear authentication tokens')
+    .option('-c, --controller <url>', 'Clear device tokens for specific controller')
+    .option('-e, --environment <env>', 'Clear client tokens for specific environment')
+    .option('-a, --app <app>', 'Clear client tokens for specific app (requires --environment)')
+    .action(async(options) => {
+      try {
+        await handleLogout(options);
+      } catch (error) {
+        handleCommandError(error, 'logout');
+        process.exit(1);
+      }
+    });
+
+  const authStatusHandler = async(options) => {
+    try {
+      await handleAuthStatus(options);
+    } catch (error) {
+      handleCommandError(error, 'auth status');
+      process.exit(1);
+    }
+  };
+
+  const auth = program
+    .command('auth')
+    .description('Authentication commands');
+
+  auth
+    .command('status')
+    .description('Display authentication status for current controller and environment')
+    .action(authStatusHandler);
+
+  auth
+    .command('config')
+    .description('Configure authentication settings (controller, environment)')
+    .option('--set-controller <url>', 'Set default controller URL')
+    .option('--set-environment <env>', 'Set default environment')
+    .action(async(options) => {
+      try {
+        await handleAuthConfig(options);
+      } catch (error) {
+        handleCommandError(error, 'auth config');
+        process.exit(1);
+      }
+    });
+}
+
+module.exports = { setupAuthCommands };