elestio 1.2.0 → 1.3.0

package/README.md

@@ -345,26 +345,47 @@ written as `random_password` are generated, `[EMAIL]` becomes your account
 email, and `[CI_CD_DOMAIN]` is resolved by the platform once the pipeline has a
 domain.
 
-**Two routes:**
-
-| | compose (default) | git (`--owner <git-user>`) |
-|---|---|---|
-| What it does | Inlines the template's `docker-compose.yml` | Generates the template repo into your Git account, then builds from it |
-| Needs a Git account | No | Yes, connected in the dashboard |
-| Lifecycle scripts | Skipped | Run |
-| Repo files the compose mounts | Unavailable | Available |
-| Works for | Templates that need no files from the repo | Every template |
-
-The compose route has no checkout, so a template whose `docker-compose.yml`
-bind-mounts a file from its repo cannot work: Docker creates the missing source
-as an empty directory and the container fails to start. The CLI detects this
-from the compose file and refuses upfront, naming the files, rather than
-letting the build fail a minute later with a `runc` error. `--force` overrides.
+**Three routes. The default needs nothing from you:**
+
+| | repo (default) | inline (`--inline-compose`) | git (`--owner <git-user>`) |
+|---|---|---|---|
+| What it does | Points the pipeline at the `elestio-examples` repo, which Elestio clones on the VM | Inlines the template's `docker-compose.yml` only | Generates the template repo into your Git account, then builds from it |
+| Needs a Git account | No | No | Yes, connected in the dashboard |
+| Install scripts | **Run** | Skipped | Run |
+| Files the compose mounts or reads | **Present** | Missing | Present |
+| Works for | The whole catalogue | Templates that need no file from the repo | Every template |
+
+The default route sends `imageData.dockerExample` (the template repo URL),
+`branch` and `repoName`. The backend stores them as the pipeline's
+`gitConfig.repoURL`, and the deployment agent clones that repo with Elestio's
+own Git token - the same thing the dashboard does. Nothing is copied into your
+account, and no Git account is needed.
+
+`--inline-compose` sends only the compose file, which is what you want for a
+compose of your own. That route has no checkout and runs no install script, so a template
+whose `docker-compose.yml` bind-mounts a file cannot work: Docker creates the
+missing source as an empty directory and the container fails to start. The file
+may ship in the template repo (n8n's `./n8n-task-runners.json`) or be generated
+by an install script (Kafka writes its `./jaas/*.conf` in `preInstall`) - either
+way it is absent here. The CLI detects this from the compose file and refuses
+upfront, naming the files, rather than letting the build fail a minute later
+with a `runc` error. `--force` overrides, and will most likely still fail.
+
+Quoted mounts (`- "./.env:/app/.env"`) and the long `type: bind` syntax count
+too; missing them is how Chromadb once deployed into a broken container.
 
 This affects more templates than you would expect - n8n
-(`./n8n-task-runners.json`), Rybbit (four files under `./configs/`) and
-WordPress (`./php.ini`) are all in this category. Vaultwarden, Redis and
-Metabase deploy cleanly on the compose route.
+(`./n8n-task-runners.json`), Chromadb (`./.env`, `./overrides/system.js`),
+Rybbit (four files under `./configs/`) and WordPress (`./php.ini`) are all in
+this category. Vaultwarden, Redis and Metabase deploy cleanly on the compose
+route.
+
+**Every template also declares install scripts** (`preInstallCommand`,
+`postInstallCommand`) that the compose route cannot run, and what they do
+cannot be read from outside Elestio. Most software does not need them -
+vaultwarden, redis and metabase were verified to deploy and run - so the CLI
+warns and continues rather than refusing. If the software comes up without its
+admin account or initial data, deploy it as a managed service instead.
 
 > **The git route is currently unavailable.** It needs
 > `POST /api/cicd/createRepoByTemplate`, which the Elestio API returns 404 for:
@@ -383,7 +404,8 @@ Metabase deploy cleanly on the compose route.
 | `elestio cicd pipeline-delete <vmID> <pipelineID> --force` | Delete pipeline |
 | `elestio cicd pipeline-resync <vmID> <pipelineID>` | Re-sync pipeline |
 | `elestio cicd pipeline-logs <vmID> <pipelineID>` | View pipeline logs |
-| `elestio cicd pipeline-history <vmID> <pipelineID>` | Build history |
+| `elestio cicd pipeline-history <vmID> <pipelineID>` | Build history, with each build's log file |
+| `elestio cicd pipeline-log <vmID> --pipeline <id> [--file <logID>]` | Open a build log (latest, or one from the history) |
 | `elestio cicd create --auto --target <vmID> --name X --repo owner/repo` | Pipeline from your own repo |
 | `elestio cicd create <config.json>` | Create from config file |
 | `elestio cicd template [mode]` | Generate config template |
@@ -407,7 +429,7 @@ Metabase deploy cleanly on the compose route.
 | `--build-cmd`, `--run-cmd`, `--install-cmd`, `--build-dir` | Override the values from `elestio.yml` (git route only) |
 | `--variables <KEY=VALUE...>` | Override the environment variables, newline-separated (git route only) |
 | `--force` | Deploy on the compose route even when the compose mounts a repo file |
-| `--dry-run` | Print the plan, create nothing |
+| `--dry-run` | Print the plan, create nothing - including on the git route, which never creates the repository |
 
 #### Pipelines from your own repository
 
@@ -488,7 +510,8 @@ you to supply all of that yourself.
 
 **The software starts, then exits, or the build fails.**
 Check the build log: `elestio cicd pipeline-history <vmID> <pipelineID>`, then
-`elestio cicd pipeline-log <vmID> --pipeline <id> --file <log>`. On the compose
+`elestio cicd pipeline-log <vmID> --pipeline <id> --file <logID>` (or without
+`--file` for the latest build). It answers a temporary URL tailing the log. On the compose
 route the usual causes are a repo file the compose mounts (the CLI refuses
 these upfront unless you passed `--force`) or a template that genuinely needs
 its lifecycle scripts. Both need the git route, which is currently

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "elestio",
-  "version": "1.2.0",
+  "version": "1.3.0",
   "description": "Elestio CLI - Deploy and manage services on the Elestio DevOps platform",
   "type": "module",
   "main": "src/cli.js",

package/src/api.js

@@ -50,6 +50,28 @@ async function parseJson(response, endpoint) {
   }
 }
 
+/**
+ * The API throttles bursts and then answers "Access temporarily restricted"
+ * for a while, on authentication as well as on any endpoint. Saying so plainly
+ * matters: it reads like a credentials or permissions problem otherwise, and a
+ * script that treats it as "no data" silently produces wrong results.
+ */
+export function rateLimitMessage(message) {
+  if (!/temporarily restricted|too many requests|rate limit/i.test(String(message ?? ''))) return null;
+  return 'The Elestio API has temporarily restricted this account for sending too many requests. ' +
+    'Wait a few minutes before retrying, and space out bulk loops (one request at a time). ' +
+    `API said: "${String(message).trim()}"`;
+}
+
+function raiseIfRateLimited(data) {
+  const message = rateLimitMessage(data?.message);
+  if (message) {
+    const error = new Error(message);
+    error.rateLimited = true;
+    throw error;
+  }
+}
+
 async function authenticate(email, token) {
   const response = await httpRequest(`${BASE_URL}/api/auth/checkAPIToken`, {
     method: 'POST',
@@ -59,6 +81,8 @@ async function authenticate(email, token) {
 
   const data = await parseJson(response, '/api/auth/checkAPIToken');
 
+  raiseIfRateLimited(data);
+
   if (data.status !== 'OK' || !data.jwt) {
     throw new Error(data.message || 'Authentication failed');
   }
@@ -118,6 +142,10 @@ export async function apiRequest(endpoint, method = 'POST', body = {}, retried =
 
   const data = await parseJson(response, endpoint);
 
+  // Before the auth check: a throttled account answers like a bad token, and
+  // clearing the JWT then re-authenticating only adds requests.
+  raiseIfRateLimited(data);
+
   const isAuthError = !retried && (
     (data.status === 'error' && data.message?.toLowerCase().includes('auth')) ||
     (data.code === 'InvalidToken') ||

package/src/commands/cicd-template.js

@@ -3,7 +3,8 @@ import { loadConfig } from '../config.js';
 import { log, colors, outputJson, formatTable } from '../utils.js';
 import { getTemplates } from './templates.js';
 import { createSubstitutions, normalizeElestioConfig, repoFileMounts } from '../templates/elestio-config.js';
-import { buildGitPipelinePayload, buildComposePipelinePayload } from '../payloads/pipeline.js';
+import { buildGitPipelinePayload, buildComposePipelinePayload, buildTemplateRepoPipelinePayload } from '../payloads/pipeline.js';
+import { parseGitAuthId } from './cicd.js';
 import { TEMPLATE_REPO_OWNER, TEMPLATE_REPO_HOST } from '../constants.js';
 
 /**
@@ -15,11 +16,18 @@ import { TEMPLATE_REPO_OWNER, TEMPLATE_REPO_HOST } from '../constants.js';
  * produces a pipeline that starts nothing, which is why this is a dedicated
  * command rather than a flag on `cicd create`.
  *
- * Two routes:
- *   git     - generate the template repo into the user's own Git account, then
- *             point the pipeline at it. Keeps the lifecycle scripts.
- *   compose - inline the template's docker-compose.yml. No Git account needed,
- *             but lifecycle scripts are lost because there is no checkout.
+ * Three routes:
+ *   repo (default) - point the pipeline at the elestio-examples repo itself.
+ *             Elestio clones it on the VM with its own Git token, so the repo's
+ *             files exist and its install scripts run. No Git account needed,
+ *             nothing copied into the user's account. This is what the
+ *             dashboard does.
+ *   inline  - inline the docker-compose.yml only (--inline-compose). No
+ *             checkout: install scripts are skipped and any file the compose
+ *             mounts or reads (env_file) is missing. Half the catalogue cannot
+ *             work this way.
+ *   git     - generate the template repo into the user's own Git account
+ *             (--owner). Needs an endpoint the API does not expose (404).
  */
 
 function templateRepoUrl(repoName) {
@@ -99,12 +107,9 @@ async function resolveCicdTarget(vmID, projectId) {
   };
 }
 
-async function findGitAuth(gitType, projectId) {
+async function findGitAuth(gitType) {
   const response = await apiRequest('/api/cicd/getCICDUserAuthAccount', 'POST', {});
-  const accounts = response.data?.accounts || response.data || response.accounts || [];
-  if (!Array.isArray(accounts)) return null;
-  const match = accounts.find(a => (a.externalProviderName || a.gitType) === gitType);
-  return match ? String(match.id || match.authID) : null;
+  return parseGitAuthId(response, gitType);
 }
 
 /**
@@ -210,13 +215,16 @@ export async function deployTemplate(nameOrId, options = {}) {
   }
   log('success', `Config loaded: runtime "${elestioConfig.config.runTime}", ${elestioConfig.variables ? elestioConfig.variables.split('\n').length : 0} env vars, ${elestioConfig.ports.length} port(s)`);
 
-  const payload = useGit
-    ? await buildGitRoute({ options, projectId, target, template, repoName, pipelineName, branch, elestioConfig })
-    : await buildComposeRoute({ projectId, target, repoName, pipelineName, branch, sourceRepoUrl, elestioConfig, force: options.force });
+  const route = useGit ? 'git' : (options.inlineCompose ? 'inline' : 'repo');
+  const payload = route === 'git'
+    ? await buildGitRoute({ options, projectId, target, template, repoName, pipelineName, branch, elestioConfig, dryRun: !!options.dryRun })
+    : route === 'inline'
+      ? await buildComposeRoute({ projectId, target, repoName, pipelineName, branch, sourceRepoUrl, elestioConfig, force: options.force })
+      : await buildTemplateRepoRoute({ projectId, target, repoName, pipelineName, branch, sourceRepoUrl, elestioConfig });
 
   if (options.dryRun) {
-    if (options.json) { outputJson({ dryRun: true, route: useGit ? 'git' : 'compose', payload }); return payload; }
-    printPlan({ template, repoName, pipelineName, target, elestioConfig, useGit, branch });
+    if (options.json) { outputJson({ dryRun: true, route, payload }); return payload; }
+    printPlan({ template, repoName, pipelineName, target, elestioConfig, route, branch });
     log('info', 'To deploy, run the same command without --dry-run');
     return payload;
   }
@@ -231,7 +239,7 @@ export async function deployTemplate(nameOrId, options = {}) {
   log('success', `Pipeline "${pipelineName}" created on ${target.displayName}`);
 
   if (options.json) {
-    outputJson({ status: 'OK', pipelineName, target: target.vmID, route: useGit ? 'git' : 'compose', webUI: elestioConfig.webUI, response });
+    outputJson({ status: 'OK', pipelineName, target: target.vmID, route, webUI: elestioConfig.webUI, response });
     return response;
   }
 
@@ -239,10 +247,10 @@ export async function deployTemplate(nameOrId, options = {}) {
   return response;
 }
 
-async function buildGitRoute({ options, projectId, target, template, repoName, pipelineName, branch, elestioConfig }) {
+async function buildGitRoute({ options, projectId, target, template, repoName, pipelineName, branch, elestioConfig, dryRun = false }) {
   const gitType = (options.gitType || 'GITHUB').toUpperCase();
 
-  let authID = options.authId ? String(options.authId) : await findGitAuth(gitType, projectId);
+  let authID = options.authId ? String(options.authId) : await findGitAuth(gitType);
   if (!authID) {
     throw new Error(
       `No ${gitType} account connected. Connect one in the dashboard (CI/CD > Git accounts), pass --auth-id, ` +
@@ -255,14 +263,21 @@ async function buildGitRoute({ options, projectId, target, template, repoName, p
     throw new Error('The Git account or organisation to create the repo in is required (--owner <user-or-org>)');
   }
 
-  log('info', `Creating ${owner}/${repoName} from ${TEMPLATE_REPO_OWNER}/${repoName}...`);
-  const repo = await generateRepoFromTemplate({
-    gitType, authID, owner, repoName,
-    templateName: repoName,
-    isPrivate: !!options.private,
-    isNonOrg: !!options.nonOrg
-  });
-  log('success', `Repo created: ${owner}/${repoName}`);
+  // A dry run must create nothing, and this step creates a repository in the
+  // user's Git account.
+  let repo = null;
+  if (dryRun) {
+    log('info', `Would create ${owner}/${repoName} from ${TEMPLATE_REPO_OWNER}/${repoName}`);
+  } else {
+    log('info', `Creating ${owner}/${repoName} from ${TEMPLATE_REPO_OWNER}/${repoName}...`);
+    repo = await generateRepoFromTemplate({
+      gitType, authID, owner, repoName,
+      templateName: repoName,
+      isPrivate: !!options.private,
+      isNonOrg: !!options.nonOrg
+    });
+    log('success', `Repo created: ${owner}/${repoName}`);
+  }
 
   return buildGitPipelinePayload({
     target,
@@ -281,6 +296,22 @@ async function buildGitRoute({ options, projectId, target, template, repoName, p
   });
 }
 
+/**
+ * Default route: the pipeline points at the template repo, which Elestio clones
+ * on the VM. Nothing to refuse -- the repo's files are there and its install
+ * scripts run.
+ */
+async function buildTemplateRepoRoute({ projectId, target, repoName, pipelineName, branch, sourceRepoUrl, elestioConfig }) {
+  log('info', `Fetching docker-compose from ${TEMPLATE_REPO_OWNER}/${repoName}...`);
+  const compose = await fetchCompose(sourceRepoUrl, branch, projectId);
+  log('success', 'Compose file loaded');
+
+  const hooks = installHooks(elestioConfig);
+  if (hooks.length > 0) log('info', `Install scripts run from the cloned repo: ${hooks.join(', ')}`);
+
+  return buildTemplateRepoPipelinePayload({ target, projectId, pipelineName, compose, repoName, branch, elestioConfig });
+}
+
 async function buildComposeRoute({ projectId, target, repoName, pipelineName, branch, sourceRepoUrl, elestioConfig, force }) {
   log('info', `Fetching docker-compose from ${TEMPLATE_REPO_OWNER}/${repoName}...`);
   const compose = await fetchCompose(sourceRepoUrl, branch, projectId);
@@ -291,16 +322,17 @@ async function buildComposeRoute({ projectId, target, repoName, pipelineName, br
   const fileMounts = repoFileMounts(compose);
   if (fileMounts.length > 0 && !force) {
     throw new Error(
-      `"${repoName}" bind-mounts ${fileMounts.length} file(s) that live in the template repo ` +
-      `(${fileMounts.join(', ')}). The compose route has no checkout, so Docker creates each one as an ` +
-      'empty directory and the container fails to start. Use the Git route (--owner <git-user>) for this ' +
-      'software, or --force to deploy anyway.'
+      `"${repoName}" bind-mounts ${fileMounts.length} file(s) that will not exist on the compose route ` +
+      `(${fileMounts.join(', ')}): either they ship in the template repo, or its install scripts create them ` +
+      '(Kafka generates its jaas/ files in preInstall). This route has no checkout and runs no install ' +
+      'script, so Docker creates each path as an empty directory and the container fails to start. The Git ' +
+      'route would handle both, but the Elestio API does not expose it today: deploy this software from the ' +
+      'dashboard, or as a managed service. --force deploys anyway, and will most likely fail.'
     );
   }
 
-  if (hasLifecycleHooks(elestioConfig)) {
-    log('warn', `"${repoName}" defines lifecycle scripts that need a repo checkout; they are skipped on the compose route.`);
-  }
+  const hooks = installHooks(elestioConfig);
+  if (hooks.length > 0) log('warn', lifecycleWarning(repoName, hooks));
 
   return buildComposePipelinePayload({
     target, projectId, pipelineName, compose, elestioConfig
@@ -311,6 +343,25 @@ function hasLifecycleHooks(elestioConfig) {
   return Object.values(elestioConfig.lifeCycleCommand).some(v => v && v !== '');
 }
 
+/**
+ * Install hooks live in the template repo, which the compose route never
+ * checks out, so they are skipped. Every catalog template declares them, and
+ * plenty deploy perfectly without them (vaultwarden, redis and metabase were
+ * verified live), so this reports rather than refuses: what the scripts
+ * actually do cannot be read, the repos are private to Elestio.
+ */
+export function installHooks(elestioConfig) {
+  return Object.entries(elestioConfig.lifeCycleCommand || {})
+    .filter(([name, value]) => value && value !== '' && /^(pre|post)Install/.test(name))
+    .map(([name, value]) => `${name}=${value}`);
+}
+
+export function lifecycleWarning(repoName, hooks) {
+  return `"${repoName}" declares install scripts the compose route cannot run (${hooks.join(', ')}). ` +
+    'Whatever they set up is skipped: an admin account, initial data or a sign-up lockdown may be missing. ' +
+    'Check the software once deployed; if it needs them, deploy it as a managed service instead.';
+}
+
 function collectOverrides(options) {
   const overrides = {};
   if (options.buildCmd !== undefined) overrides.buildCmd = options.buildCmd;
@@ -321,13 +372,18 @@ function collectOverrides(options) {
   return overrides;
 }
 
-function printPlan({ template, repoName, pipelineName, target, elestioConfig, useGit, branch }) {
+function printPlan({ template, repoName, pipelineName, target, elestioConfig, route, branch }) {
   console.log(`\n${colors.bold}Pipeline Preview (--dry-run)${colors.reset}\n`);
   console.log(`  Software:   ${colors.cyan}${template.title}${colors.reset} (ID: ${template.id})`);
   console.log(`  Template:   ${TEMPLATE_REPO_OWNER}/${repoName} @ ${branch}`);
   console.log(`  Pipeline:   ${pipelineName}`);
   console.log(`  Target:     ${target.displayName} (${target.vmID})`);
-  console.log(`  Route:      ${useGit ? 'git (repo generated in your account, lifecycle scripts kept)' : 'compose (inlined, lifecycle scripts skipped)'}`);
+  const routes = {
+    repo: 'repo (Elestio clones the template repo on the VM, install scripts run)',
+    inline: 'inline compose (no checkout: install scripts skipped, mounted files missing)',
+    git: 'git (repo generated in your account, install scripts run)'
+  };
+  console.log(`  Route:      ${routes[route]}`);
   console.log(`  Runtime:    ${elestioConfig.config.runTime || 'N/A'}`);
   console.log(`  Ports:      ${elestioConfig.ports.map(p => `${p.listeningPort}->${p.targetPort}`).join(', ') || 'default'}`);
   console.log(`  Env vars:   ${elestioConfig.variables ? elestioConfig.variables.split('\n').map(v => v.split('=')[0]).join(', ') : 'none'}`);

package/src/commands/cicd.js

@@ -169,16 +169,70 @@ export async function getPipelineHistory(vmID, pipelineID, projectId, json = fal
   return history;
 }
 
-export async function viewPipelineLog(vmID, pipelineID, filepath, projectId) {
+/**
+ * getCICDUserAuthAccount answers an object keyed by provider:
+ *   {data: {GITHUB: [{name, value, type}], GITLAB: [], GITLAB_SELF_HOSTED: []}}
+ * The auth ID is the account's `value`. Reading it as an array found nothing,
+ * so the git route reported no connected account even when there was one.
+ */
+export function parseGitAuthId(response, gitType) {
+  const data = response?.data ?? response ?? {};
+
+  const byProvider = data[gitType];
+  if (Array.isArray(byProvider) && byProvider.length > 0) {
+    const account = byProvider[0];
+    const id = account.value ?? account.id ?? account.authID;
+    return id === undefined || id === null ? null : String(id);
+  }
+
+  // Older shape: a flat list of accounts carrying their provider.
+  const flat = Array.isArray(data) ? data : (Array.isArray(data.accounts) ? data.accounts : null);
+  if (flat) {
+    const match = flat.find(a => (a.externalProviderName || a.gitType || a.type) === gitType);
+    if (match) {
+      const id = match.value ?? match.id ?? match.authID;
+      return id === undefined || id === null ? null : String(id);
+    }
+  }
+
+  return null;
+}
+
+/**
+ * viewPipelineLog requires isLatest, and spells the file `filePath`. Sending
+ * `filepath` without isLatest was rejected as a missing parameter, so no build
+ * log could be read. The file name is the logID from pipeline-history; it is
+ * resolved inside the pipeline's own log directory, so a full path is trimmed.
+ */
+export function buildViewLogPayload(vmID, pipelineID, projectId, file = null) {
+  const payload = {
+    vmID: String(vmID),
+    projectID: String(projectId),
+    pipelineID: parseInt(pipelineID),
+    isLatest: !file
+  };
+  if (file) payload.filePath = String(file).slice(String(file).lastIndexOf('/') + 1);
+  return payload;
+}
+
+/**
+ * Opens a temporary web view tailing one build log, like the dashboard does.
+ * The endpoint answers a URL, not the log's content.
+ */
+export async function viewPipelineLog(vmID, pipelineID, filepath, projectId, json = false) {
   const config = loadConfig();
   const pid = projectId || config.defaultProject;
   if (!pid) throw new Error('Project ID required');
+  if (!pipelineID) throw new Error('Pipeline ID required (--pipeline <id>)');
 
-  const response = await apiRequest('/api/cicd/viewPipelineLog', 'POST', {
-    vmID: String(vmID), projectID: String(pid), pipelineID: parseInt(pipelineID), filepath
-  });
-  if (response.status !== 'OK') throw new Error(response.message || 'Failed');
-  console.log(response.data?.content || response.content || '');
+  const response = await apiRequest('/api/cicd/viewPipelineLog', 'POST', buildViewLogPayload(vmID, pipelineID, pid, filepath));
+  if (!response.url) throw new Error(response.message || response.details || 'Failed to open the build log');
+
+  if (json) { outputJson({ url: response.url }); return response; }
+
+  console.log(`\n${colors.bold}Build log${filepath ? ` (${filepath})` : ' (latest)'}${colors.reset}\n`);
+  console.log(`  URL: ${colors.cyan}${response.url}${colors.reset}`);
+  console.log(`\n  ${colors.dim}Temporary page tailing the log on the target.${colors.reset}\n`);
   return response;
 }
 
@@ -238,6 +292,14 @@ async function findGitAuthID(gitType, projectId) {
   const config = loadConfig();
   const pid = projectId || config.defaultProject;
 
+  // The connected accounts are the authoritative source; the scan below only
+  // covers accounts that are not listed but are in use by a pipeline.
+  try {
+    const accounts = await apiRequest('/api/cicd/getCICDUserAuthAccount', 'POST', {});
+    const authID = parseGitAuthId(accounts, gitType);
+    if (authID) return authID;
+  } catch { /* fall through to the scan */ }
+
   try {
     const cicdResp = await apiRequest('/api/cicd/getCICDServices', 'POST', { projectID: String(pid) });
     const cicdServices = Array.isArray(cicdResp) ? cicdResp : (cicdResp.data?.services || []);

package/src/payloads/pipeline.js

@@ -1,4 +1,4 @@
-import { RUNTIME_PRESETS } from '../constants.js';
+import { RUNTIME_PRESETS, TEMPLATE_REPO_OWNER, TEMPLATE_REPO_HOST } from '../constants.js';
 import { EMPTY_LIFECYCLE } from '../templates/elestio-config.js';
 
 /**
@@ -147,6 +147,55 @@ export function buildGitPipelinePayload({
   });
 }
 
+/**
+ * Pipeline running catalog software from its own template repo.
+ *
+ * This is what the dashboard sends: `imageData.dockerExample` carries the
+ * elestio-examples URL, and the backend then stores it as the pipeline's
+ * gitConfig.repoURL (createCiCdExistServer). The deployment agent clones that
+ * repo on the VM with Elestio's own Git token, so the repo's files exist and
+ * its lifecycle scripts run -- without copying anything into the user's Git
+ * account, and without needing a connected Git account at all.
+ *
+ * The inline route below cannot do either, which is why half the catalogue
+ * (Kafka, Chromadb, Weaviate, ClickHouse, WordPress...) is undeployable
+ * through it.
+ */
+export function buildTemplateRepoPipelinePayload({
+  target, projectId, pipelineName, compose, repoName, branch = 'main', elestioConfig, overrides = {}
+}) {
+  if (!compose) throw new Error('A docker-compose file is required');
+  if (!repoName) throw new Error('A template repo name is required');
+  const fromFile = elestioConfig && elestioConfig.hasConfig ? elestioConfig : null;
+
+  return buildPipelinePayload({
+    target, projectId, pipelineName,
+    cicdMode: 'DockerCompose',
+    gitData: {},
+    imageData: {
+      isPrivate: false,
+      compose,
+      // The three fields that turn this into a cloned-repo pipeline.
+      dockerExample: `${TEMPLATE_REPO_HOST}/${TEMPLATE_REPO_OWNER}/${repoName}`,
+      branch,
+      repoName
+    },
+    configData: {
+      runTime: 'NodeJs', framework: 'NoFramework', version: '20',
+      buildDir: '/', rootDir: '/', buildCmd: '', runCmd: '', installCmd: ''
+    },
+    ports: overrides.ports || fromFile?.ports,
+    exposedPorts: overrides.exposedPorts || fromFile?.exposedPorts,
+    variables: overrides.variables ?? fromFile?.variables ?? '',
+    // Kept, unlike the inline route: the repo is checked out, so they can run.
+    lifeCycleCommand: fromFile?.lifeCycleCommand,
+    copyCommandConfig: fromFile?.copyCommandConfig || [],
+    authID: null,
+    isPublicGitRepo: false,
+    isNeedToCreateRepo: false
+  });
+}
+
 /**
  * Pipeline running an inline docker-compose file, with no Git repo attached.
  *

package/src/registry.js

@@ -632,6 +632,7 @@ export const registry = {
             owner: args.owner, branch: args.branch, repoName: args['repo-name'],
             gitType: args['git-type'], authId: args['auth-id'],
             git: !args['no-git'] && (args.git === true || args.git === 'true' || !!args.owner),
+            inlineCompose: !!args['inline-compose'],
             private: !!args.private, nonOrg: !!args['non-org'],
             buildCmd: args['build-cmd'], runCmd: args['run-cmd'],
             installCmd: args['install-cmd'], buildDir: args['build-dir'],
@@ -654,7 +655,7 @@ export const registry = {
       'pipeline-resync': { summary: 'Re-sync a pipeline', usage: 'pipeline-resync <vmID> <pipelineID>', async run({ args }) { await (await load.cicd()).resyncPipeline(requireArg(args._[2], 'cicd pipeline-resync <vmID> <pipelineID>'), positional(args, 3, 'pipeline'), args.project); } },
       'pipeline-logs': { summary: 'Tail the running pipeline logs', usage: 'pipeline-logs <vmID> <pipelineID>', async run({ args }) { await (await load.cicd()).getPipelineLogs(requireArg(args._[2], 'cicd pipeline-logs <vmID> <pipelineID>'), positional(args, 3, 'pipeline'), args.project); } },
       'pipeline-history': { summary: 'Show the build history', usage: 'pipeline-history <vmID> <pipelineID>', async run({ args, json }) { await (await load.cicd()).getPipelineHistory(requireArg(args._[2], 'cicd pipeline-history <vmID> <pipelineID>'), positional(args, 3, 'pipeline'), args.project, json); } },
-      'pipeline-log': { summary: 'Print one build log file', usage: 'pipeline-log <vmID> --pipeline <id> --file <path>', async run({ args }) { await (await load.cicd()).viewPipelineLog(requireArg(args._[2], 'cicd pipeline-log <vmID> --pipeline <id> --file <path>'), args.pipeline, args.file, args.project); } },
+      'pipeline-log': { summary: 'Open a build log (latest, or --file <logID> from pipeline-history)', usage: 'pipeline-log <vmID> --pipeline <id> [--file <logID>]', async run({ args, json }) { await (await load.cicd()).viewPipelineLog(requireArg(args._[2], 'cicd pipeline-log <vmID> --pipeline <id> [--file <logID>]'), args.pipeline, args.file, args.project, json); } },
       domains: { summary: 'List pipeline domains', usage: 'domains <vmID> <pipelineID>', async run({ args, json }) { await (await load.cicd()).listPipelineDomains(requireArg(args._[2], 'cicd domains <vmID> <pipelineID>'), positional(args, 3, 'pipeline'), args.project, json); } },
       'domain-add': { summary: 'Add a pipeline domain', usage: 'domain-add <vmID> --pipeline <id> --domain <d>', async run({ args }) { await (await load.cicd()).addPipelineDomain(requireArg(args._[2], 'cicd domain-add <vmID> --pipeline <id> --domain <d>'), args.pipeline, args.domain || args._[3], args.project); } },
       'domain-remove': { summary: 'Remove a pipeline domain', usage: 'domain-remove <vmID> --pipeline <id> --domain <d>', async run({ args }) { await (await load.cicd()).removePipelineDomain(requireArg(args._[2], 'cicd domain-remove <vmID> --pipeline <id> --domain <d>'), args.pipeline, args.domain || args._[3], args.project); } },

package/src/templates/elestio-config.js

@@ -179,10 +179,34 @@ export function repoFileMounts(compose) {
   if (typeof compose !== 'string') return [];
 
   const mounts = new Set();
-  for (const match of compose.matchAll(/^\s*-\s+(\.{1,2}\/[^\s:]+):[^\s]+/gm)) {
-    const source = match[1];
+  const add = (source) => {
     const lastSegment = source.slice(source.lastIndexOf('/') + 1);
     if (/\.[A-Za-z0-9]{1,10}$/.test(lastSegment)) mounts.add(source);
+  };
+
+  // Short syntax, quoted or not: - ./file:/in/container[:ro]
+  for (const match of compose.matchAll(/^\s*-\s+["']?(\.{1,2}\/[^\s:"']+)["']?\s*:/gm)) {
+    add(match[1]);
+  }
+
+  // Long syntax: - type: bind / source: ./file / target: /in/container
+  for (const match of compose.matchAll(/^\s*source:\s*["']?(\.{1,2}\/[^\s"']+)["']?\s*$/gm)) {
+    add(match[1]);
+  }
+
+  // env_file, inline or as a list. Docker refuses to start when the file is
+  // missing, and 8 of the 22 catalogue templates sampled read one.
+  for (const match of compose.matchAll(/^\s*env_file:\s*(?:\[([^\]]*)\]|["']?([^\s"'\[]+)["']?)\s*$/gm)) {
+    for (const entry of (match[1] ?? match[2] ?? '').split(',')) {
+      const file = entry.trim().replace(/^["']|["']$/g, '');
+      if (file.startsWith('.')) add(file);
+    }
   }
+  for (const block of compose.matchAll(/^\s*env_file:\s*$((?:\n\s*-\s+.*)+)/gm)) {
+    for (const item of block[1].matchAll(/^\s*-\s+["']?([^\s"']+)["']?\s*$/gm)) {
+      if (item[1].startsWith('.')) add(item[1]);
+    }
+  }
+
   return [...mounts];
 }