@camunda8/cli 4.1.0-alpha.1 → 4.1.0-alpha.2

package/dist/default-plugins/element-template/AGENTS.md

@@ -1,7 +1,7 @@
 # AGENTS.md — `element-template` plugin
 
 Read [docs/design.md](./docs/design.md) before changing this plugin. It
-covers the **why**: vendor bundle, marketplace endpoint choice, cache
+covers the **why**: vendor bundle, release-bundle source choice, cache
 strategy (`upstreamRef`-keyed dedup), lazy bootstrap, and semver-based
 version resolution.
 
@@ -12,7 +12,8 @@ version resolution.
 | `c8ctl-plugin.ts` | Plugin API (metadata + commands export), subcommand dispatch table |
 | `commands/<name>.ts` | One file per subcommand: `apply`, `edit`, `get`, `get-properties`, `info`, `search`, `sync` |
 | `template-ref.ts` | `parseTemplateRef`, `readBpmnInput`, `getExecutionPlatformVersion`, `resolveOotbTemplate`, `loadTemplate` |
-| `marketplace.ts` | Cache I/O, `/ootb-connectors` fetch, sync, search, version resolution |
+| `cache.ts` | Cache I/O, sync, search, version resolution |
+| `releases.ts` | `camunda/connectors` GitHub releases: listing, newest-release-per-minor selection, bundle download, tar.gz reader |
 | `helpers.ts` | `--set` parsing, file/URL fetch, glob → regex, multi-binding lookup, condition warnings, `atomicOverwriteFile` |
 | `binding.ts` | Binding-target resolution shared by `apply`/`edit` — which moddle child + property a `zeebe:input`/etc. binding writes to. Hand-rolled stand-in for bpmn-js-element-templates' internal `setPropertyValue`; see the note at the top of the file about replacing it once bpmn-io/bpmn-js-element-templates#236 ships. |
 | `vendor.ts` | `resolveVendorBundle()` + the minimal bpmn-js `Modeler`/`modeling` type surface shared by `apply`/`edit` |
@@ -29,13 +30,13 @@ version resolution.
   without reason — `metadata.upstreamRef` is the dedup key for
   incremental sync.
 - **No subcommand auto-bootstraps the cache.** OOTB-id resolution
-  is guarded by `requireCachePresent()` in `marketplace.ts`, which
+  is guarded by `requireCachePresent()` in `cache.ts`, which
   throws the shared `CACHE_NOT_FOUND_MESSAGE` when the cache is
   absent. The reason auto-bootstrap is forbidden: bootstrap progress
   goes through `logger.info`, which writes to stdout in text mode —
   it would corrupt `apply | bpmn lint` and `get <id> > template.json`
-  pipelines, and racing cold-cache invocations would both fetch the
-  same ~14 MB index. Don't re-add a bootstrap call to any subcommand
+  pipelines, and racing cold-cache invocations would both download the
+  same bundles. Don't re-add a bootstrap call to any subcommand
   without changing the logger story first.
 - **Path/URL apply paths must not trigger the cache check.**
   Detection happens in `parseTemplateRef()` in `template-ref.ts`
@@ -45,7 +46,7 @@ version resolution.
   overwrites a user-owned file (cache or BPMN) must follow the same
   pattern — a kill mid-write must not leave a truncated file.
 - **`syncTemplates` is serialised by an advisory lockfile.** The
-  helper `withSyncLock` in `marketplace.ts` holds
+  helper `withSyncLock` in `cache.ts` holds
   `<cacheDir>/.sync.lock` while the body runs, with stale-lock
   recovery (dead PID or > 60 min old) and signal handlers
   (SIGINT/SIGTERM/SIGHUP) that release before re-raising. Don't
@@ -79,9 +80,16 @@ version resolution.
   WeakMap-based `sourceByDetail` side table in `loadTemplate` preserves
   this identity so `get-properties` doesn't collapse them and `--set`
   writes to all matching duplicates.
-- **Marketplace endpoint URL is overridable via
-  `C8CTL_OOTB_ELEMENT_TEMPLATES_URL`** — useful for tests against a
-  local fixture server.
+- **Templates come from the `camunda/connectors` GitHub releases, not
+  `raw.githubusercontent.com`** (blocked in many enterprise networks —
+  c8ctl#530). `sync` keeps the newest release of each of the 4 newest
+  minor lines (`MAX_MINOR_LINES`), skipping drafts, release candidates
+  and releases whose bundle asset isn't published yet. A bundle URL
+  changes with every patch, so `--prune` is skipped whenever a download
+  failed — otherwise one transient error would drop a whole minor line.
+  The release listing endpoint is overridable via
+  `C8CTL_CONNECTORS_RELEASES_URL` — useful for tests against a local
+  fixture server.
 
 ## Testing
 
@@ -89,7 +97,8 @@ version resolution.
 npm run build:vendor && node --experimental-strip-types --test tests/unit/element-template.test.ts
 ```
 
-Smoke-test against the live marketplace (writes to a throwaway dir):
+Smoke-test against the live connectors releases (writes to a throwaway
+dir):
 
 ```bash
 C8CTL_DATA_DIR=/tmp/c8ctl-smoke node --experimental-strip-types src/index.ts \

package/dist/default-plugins/element-template/README.md

@@ -98,7 +98,7 @@ c8ctl element-template get io.camunda.connectors.HttpJson.v2 --no-icon  # drop t
 
 # Refresh the local OOTB template cache
 c8ctl element-template sync
-c8ctl element-template sync --prune    # also drop entries no longer in the index
+c8ctl element-template sync --prune    # also drop entries no longer in a selected release
 ```
 
 ## Inspecting a template
@@ -302,10 +302,14 @@ warning at the end (the property won't be applied).
 
 ## How OOTB templates are resolved
 
-1. **Index**: fetched from the Camunda marketplace
-   (`https://marketplace.cloud.camunda.io/api/v1/ootb-connectors`) — the
-   same source Desktop Modeler uses. Override via
-   `C8CTL_OOTB_ELEMENT_TEMPLATES_URL` for testing.
+1. **Source**: the `camunda/connectors` GitHub releases. `sync` lists
+   the releases, keeps the newest release of each of the 4 newest minor
+   lines (8.8.x, 8.9.x, ... — alphas count, release candidates don't),
+   and downloads
+   each one's `connectors-bundle-templates-<tag>.tar.gz` asset from
+   `github.com`. `raw.githubusercontent.com` is deliberately not used:
+   it is blocked in many enterprise networks. Override the release
+   listing endpoint via `C8CTL_CONNECTORS_RELEASES_URL` for testing.
 2. **Populate the cache once** via `c8ctl element-template sync`.
    Subcommands that resolve OOTB ids (`search`, `info`, `get-properties`,
    `apply`, `get`) exit non-zero with a hint to run `sync` when the
@@ -314,14 +318,14 @@ warning at the end (the property won't be applied).
    `apply | bpmn lint` or `get <id> > template.json`.
 3. **Local file or URL** template args (paths containing `/` or `\`,
    starting with `.`, ending in `.json`, or starting with `http(s)://`)
-   skip the index entirely.
+   skip the cache entirely.
 4. **Version selection** uses `semver.satisfies` against the BPMN's
    `modeler:executionPlatformVersion` and each template's
    `engines.camunda` constraint. Without `@<version>`, the highest
    compatible version wins.
 5. **Stale cache** (>7 days) prints a hint to run `sync`. No automatic
-   refresh — `sync` only fetches refs not already cached (commit-pinned
-   URLs make incremental sync free).
+   refresh — `sync` only downloads bundles not already cached (release
+   assets are tag-pinned, so incremental sync is free).
 6. **Concurrent syncs** are serialised by an advisory lock at
    `<cache-dir>/.sync.lock`. A second `sync` started while one is
    already running exits non-zero rather than racing.
@@ -339,5 +343,5 @@ Set `C8CTL_DATA_DIR` to override.
 ## Design
 
 See [`docs/design.md`](./docs/design.md) for the full reasoning behind
-the marketplace endpoint choice, the vendor bundle, the cache strategy
+the release-bundle source choice, the vendor bundle, the cache strategy
 (mirrored from Desktop Modeler), and version resolution.

package/dist/default-plugins/element-template/c8ctl-plugin.js

@@ -8610,8 +8610,8 @@ function parseTemplateJson(content) {
 import { existsSync as existsSync3, readFileSync as readFileSync3 } from "node:fs";
 import { resolve as resolvePath2 } from "node:path";
 
-// default-plugins/element-template/marketplace.ts
-var import_semver2 = __toESM(require_semver2(), 1);
+// default-plugins/element-template/cache.ts
+var import_semver3 = __toESM(require_semver2(), 1);
 import {
   closeSync,
   existsSync as existsSync2,
@@ -8624,14 +8624,140 @@ import {
   writeSync
 } from "node:fs";
 import { join } from "node:path";
-var DEFAULT_OOTB_URL = "https://marketplace.cloud.camunda.io/api/v1/ootb-connectors";
-var FETCH_CONCURRENCY = 12;
+
+// default-plugins/element-template/releases.ts
+var import_semver2 = __toESM(require_semver2(), 1);
+import { gunzipSync } from "node:zlib";
+var DEFAULT_RELEASES_URL = "https://api.github.com/repos/camunda/connectors/releases?per_page=100";
+var TEMPLATES_ASSET_PREFIX = "connectors-bundle-templates-";
+var TEMPLATES_ASSET_SUFFIX = ".tar.gz";
+var RELEASES_FETCH_TIMEOUT_MS = 3e4;
+var ASSET_FETCH_TIMEOUT_MS = 12e4;
+var MAX_BUNDLE_BYTES = 128 * 1024 * 1024;
+var MAX_MINOR_LINES = 4;
+function getReleasesUrl() {
+  return process.env.C8CTL_CONNECTORS_RELEASES_URL || DEFAULT_RELEASES_URL;
+}
+function isReleaseCandidate(version) {
+  const prerelease = import_semver2.default.prerelease(version);
+  if (!prerelease) return false;
+  return prerelease.flatMap((part) => typeof part === "string" ? part.split("-") : []).some((part) => /^rc\d*$/i.test(part));
+}
+function findTemplatesAssetUrl(release) {
+  if (!Array.isArray(release.assets)) return null;
+  for (const asset of release.assets) {
+    if (!isRecord(asset)) continue;
+    const { name: name2, browser_download_url: url } = asset;
+    if (typeof name2 !== "string" || typeof url !== "string") continue;
+    if (name2.startsWith(TEMPLATES_ASSET_PREFIX) && name2.endsWith(TEMPLATES_ASSET_SUFFIX)) {
+      return url;
+    }
+  }
+  return null;
+}
+function parseReleases(raw) {
+  if (!Array.isArray(raw)) {
+    throw new Error("Connector releases response is not a JSON array");
+  }
+  const releases = [];
+  for (const entry of raw) {
+    if (!isRecord(entry)) continue;
+    if (entry.draft === true) continue;
+    const tag = entry.tag_name;
+    if (typeof tag !== "string") continue;
+    const version = import_semver2.default.valid(tag);
+    if (!version || isReleaseCandidate(version)) continue;
+    const assetUrl = findTemplatesAssetUrl(entry);
+    if (!assetUrl) continue;
+    releases.push({ tag, version, assetUrl });
+  }
+  return releases;
+}
+function selectLatestPerMinor(releases) {
+  const latest = /* @__PURE__ */ new Map();
+  for (const release of releases) {
+    const key = `${import_semver2.default.major(release.version)}.${import_semver2.default.minor(release.version)}`;
+    const current = latest.get(key);
+    if (!current || import_semver2.default.gt(release.version, current.version)) {
+      latest.set(key, release);
+    }
+  }
+  return [...latest.values()].sort((a, b) => import_semver2.default.rcompare(a.version, b.version)).slice(0, MAX_MINOR_LINES);
+}
+async function fetchConnectorReleases() {
+  const url = getReleasesUrl();
+  const response = await fetch(url, {
+    headers: {
+      "User-Agent": USER_AGENT,
+      Accept: "application/vnd.github+json"
+    },
+    signal: AbortSignal.timeout(RELEASES_FETCH_TIMEOUT_MS)
+  });
+  if (!response.ok) {
+    const hint = response.status === 403 || response.status === 429 ? "\nThe GitHub API rate limit may be exhausted \u2014 retry later, or point C8CTL_CONNECTORS_RELEASES_URL at a mirror of the release listing." : "";
+    throw new Error(
+      `HTTP ${response.status} ${response.statusText} for ${url}${hint}`
+    );
+  }
+  return selectLatestPerMinor(parseReleases(await response.json()));
+}
+async function fetchReleaseAsset(url) {
+  const response = await fetch(url, {
+    headers: { "User-Agent": USER_AGENT },
+    signal: AbortSignal.timeout(ASSET_FETCH_TIMEOUT_MS)
+  });
+  if (!response.ok) {
+    throw new Error(
+      `HTTP ${response.status} ${response.statusText} for ${url}`
+    );
+  }
+  return new Uint8Array(await response.arrayBuffer());
+}
+var TAR_BLOCK_SIZE = 512;
+function readTarString(tar, offset, length) {
+  const raw = tar.toString("utf-8", offset, offset + length);
+  const nul = raw.indexOf("\0");
+  return nul === -1 ? raw : raw.slice(0, nul);
+}
+function extractJsonEntries(gzipped) {
+  const tar = gunzipSync(gzipped, { maxOutputLength: MAX_BUNDLE_BYTES });
+  const entries = [];
+  let offset = 0;
+  while (offset + TAR_BLOCK_SIZE <= tar.length) {
+    if (tar.subarray(offset, offset + TAR_BLOCK_SIZE).every((b) => b === 0)) {
+      break;
+    }
+    const name2 = readTarString(tar, offset, 100);
+    const prefix2 = readTarString(tar, offset + 345, 155);
+    const sizeField = readTarString(tar, offset + 124, 12).trim();
+    const size = Number.parseInt(sizeField, 8);
+    if (!Number.isFinite(size) || size < 0) {
+      throw new Error(
+        `Malformed tar header (bad size field) at byte ${offset}`
+      );
+    }
+    const typeFlag = String.fromCharCode(tar[offset + 156]);
+    const dataStart = offset + TAR_BLOCK_SIZE;
+    const dataEnd = dataStart + size;
+    if (dataEnd > tar.length) {
+      throw new Error(`Truncated tar entry '${name2}' at byte ${offset}`);
+    }
+    const fullName = prefix2 ? `${prefix2}/${name2}` : name2;
+    if ((typeFlag === "0" || typeFlag === "\0") && fullName.endsWith(".json")) {
+      entries.push({
+        name: fullName,
+        content: tar.toString("utf-8", dataStart, dataEnd)
+      });
+    }
+    offset = dataStart + Math.ceil(size / TAR_BLOCK_SIZE) * TAR_BLOCK_SIZE;
+  }
+  return entries;
+}
+
+// default-plugins/element-template/cache.ts
+var FETCH_CONCURRENCY = 4;
 var STALE_AFTER_MS = 7 * 24 * 60 * 60 * 1e3;
-var FETCH_TIMEOUT_MS = 3e4;
 var SYNC_LOCK_STALE_AFTER_MS = 60 * 60 * 1e3;
-function getMarketplaceUrl() {
-  return process.env.C8CTL_OOTB_ELEMENT_TEMPLATES_URL || DEFAULT_OOTB_URL;
-}
 function getCacheDir() {
   if (!globalThis.c8ctl?.getUserDataDir) {
     throw new Error(
@@ -8796,7 +8922,7 @@ function atomicWriteFileSync(target, contents) {
     throw error3;
   }
 }
-function saveCache(templates) {
+function saveCache(templates, { stampFetchedAt = true } = {}) {
   const dir = getCacheDir();
   mkdirSync(dir, { recursive: true });
   atomicWriteFileSync(
@@ -8804,7 +8930,9 @@ function saveCache(templates) {
     `${JSON.stringify(templates, null, 2)}
 `
   );
-  atomicWriteFileSync(getFetchedAtPath(), String(Date.now()));
+  if (stampFetchedAt) {
+    atomicWriteFileSync(getFetchedAtPath(), String(Date.now()));
+  }
 }
 function isCacheStale() {
   const fetchedAt = loadFetchedAt();
@@ -8825,50 +8953,41 @@ function nudgeIfStale(logger) {
     `Element template cache is ${ageText}. Run 'c8ctl element-template sync' to refresh.`
   );
 }
-async function fetchJson(url) {
-  const response = await fetch(url, {
-    headers: { "User-Agent": USER_AGENT },
-    signal: AbortSignal.timeout(FETCH_TIMEOUT_MS)
-  });
-  if (!response.ok) {
-    throw new Error(
-      `HTTP ${response.status} ${response.statusText} for ${url}`
-    );
-  }
-  return response.json();
-}
-async function fetchIndex() {
-  const raw = await fetchJson(getMarketplaceUrl());
-  if (!isRecord(raw)) {
-    throw new Error("Marketplace index is not a JSON object");
-  }
-  const result = {};
-  for (const [id, value] of Object.entries(raw)) {
-    if (!Array.isArray(value)) continue;
-    const entries = [];
-    for (const entry of value) {
-      if (!isRecord(entry)) continue;
-      entries.push({
-        version: typeof entry.version === "number" ? entry.version : void 0,
-        ref: typeof entry.ref === "string" ? entry.ref : void 0,
-        engine: isRecord(entry.engine) ? {
-          camunda: typeof entry.engine.camunda === "string" ? entry.engine.camunda : void 0
-        } : void 0
-      });
-    }
-    result[id] = entries;
-  }
-  return result;
-}
 function isTemplateLike(value) {
   return isRecord(value) && Array.isArray(value.properties);
 }
-async function fetchTemplate(url) {
-  const raw = await fetchJson(url);
-  if (!isTemplateLike(raw)) {
-    throw new Error(`Fetched template at ${url} did not match expected shape`);
+function buildUpstreamRef(assetUrl, entryName) {
+  return `${assetUrl}#${entryName}`;
+}
+function assetUrlOfRef(ref) {
+  const hash = ref.indexOf("#");
+  return hash === -1 ? ref : ref.slice(0, hash);
+}
+async function fetchReleaseTemplates(release) {
+  const bundle = await fetchReleaseAsset(release.assetUrl);
+  const templates = [];
+  for (const entry of extractJsonEntries(bundle)) {
+    let parsed;
+    try {
+      parsed = JSON.parse(entry.content);
+    } catch {
+      continue;
+    }
+    if (!isTemplateLike(parsed) || typeof parsed.version !== "number") {
+      continue;
+    }
+    parsed.metadata = {
+      ...parsed.metadata,
+      upstreamRef: buildUpstreamRef(release.assetUrl, entry.name)
+    };
+    templates.push(parsed);
+  }
+  if (templates.length === 0) {
+    throw new Error(
+      `Bundle for release ${release.tag} contained no element templates`
+    );
   }
-  return raw;
+  return templates;
 }
 async function pool(items, concurrency, fn) {
   const queue = items.slice();
@@ -8884,20 +9003,12 @@ async function pool(items, concurrency, fn) {
   );
   await Promise.all(workers);
 }
-function flattenIndex(index) {
-  const entries = [];
-  for (const [id, versions] of Object.entries(index)) {
-    for (const entry of versions) {
-      if (entry.version === void 0 || entry.ref === void 0) continue;
-      entries.push({
-        id,
-        version: entry.version,
-        ref: entry.ref,
-        engine: entry.engine
-      });
-    }
-  }
-  return entries;
+function sortTemplates(templates) {
+  return [...templates].sort((a, b) => {
+    const idCmp = (a.id ?? "").localeCompare(b.id ?? "");
+    if (idCmp !== 0) return idCmp;
+    return (a.version ?? 0) - (b.version ?? 0);
+  });
 }
 async function syncTemplates({
   logger,
@@ -8909,11 +9020,16 @@ async function syncTemplatesLocked({
   logger,
   prune
 }) {
-  logger.info(`Fetching index from ${getMarketplaceUrl()} ...`);
-  const index = await fetchIndex();
-  const entries = flattenIndex(index);
+  const releasesUrl = getReleasesUrl();
+  logger.info(`Fetching connector releases from ${releasesUrl} ...`);
+  const releases = await fetchConnectorReleases();
+  if (releases.length === 0) {
+    throw new Error(
+      `No connector release with an element-template bundle found at ${releasesUrl}.`
+    );
+  }
   logger.info(
-    `Index has ${entries.length} template versions across ${Object.keys(index).length} connectors.`
+    `Latest release per minor: ${releases.map((r) => r.tag).join(", ")}.`
   );
   let existing;
   try {
@@ -8923,70 +9039,90 @@ async function syncTemplatesLocked({
     logger.warn(`Corrupt cache \u2014 starting fresh: ${message}`);
     existing = [];
   }
-  const byUpstreamRef = /* @__PURE__ */ new Map();
+  const selectedAssets = new Set(releases.map((r) => r.assetUrl));
+  const cachedByAsset = /* @__PURE__ */ new Map();
+  const staleCached = [];
   for (const tpl of existing) {
     const ref = tpl.metadata?.upstreamRef;
-    if (ref) byUpstreamRef.set(ref, tpl);
+    const asset = ref ? assetUrlOfRef(ref) : void 0;
+    if (asset && selectedAssets.has(asset)) {
+      const bucket = cachedByAsset.get(asset);
+      if (bucket) bucket.push(tpl);
+      else cachedByAsset.set(asset, [tpl]);
+    } else {
+      staleCached.push(tpl);
+    }
   }
-  const freshRefs = new Set(entries.map((e) => e.ref));
-  const toFetch = entries.filter((e) => !byUpstreamRef.has(e.ref));
+  const toFetch = releases.filter((r) => !cachedByAsset.has(r.assetUrl));
+  const reusedCount = [...cachedByAsset.values()].reduce(
+    (sum, bucket) => sum + bucket.length,
+    0
+  );
   logger.info(
-    `${byUpstreamRef.size > 0 ? `Reusing ${entries.length - toFetch.length} cached, ` : ""}fetching ${toFetch.length} new...`
+    `${reusedCount > 0 ? `Reusing ${reusedCount} cached templates from ${cachedByAsset.size} bundle(s), ` : ""}downloading ${toFetch.length} bundle(s)...`
   );
-  let fetched = 0;
   let errors = 0;
   let progress = 0;
-  const fetchedTemplates = [];
-  await pool(toFetch, FETCH_CONCURRENCY, async (entry) => {
+  const fetchedByAsset = /* @__PURE__ */ new Map();
+  await pool(toFetch, FETCH_CONCURRENCY, async (release) => {
     progress += 1;
     const myProgress = progress;
-    const label = `${entry.id}@${entry.version}`;
     try {
-      const template = await fetchTemplate(entry.ref);
-      template.metadata = template.metadata || {};
-      template.metadata.upstreamRef = entry.ref;
-      fetchedTemplates.push(template);
-      fetched += 1;
-      logger.info(`  [${myProgress}/${toFetch.length}] ${label}`);
+      const templates = await fetchReleaseTemplates(release);
+      fetchedByAsset.set(release.assetUrl, templates);
+      logger.info(
+        `  [${myProgress}/${toFetch.length}] ${release.tag} \u2014 ${templates.length} templates`
+      );
     } catch (error3) {
       errors += 1;
       const message = error3 instanceof Error ? error3.message : String(error3);
-      logger.warn(`  [${myProgress}/${toFetch.length}] ${label} \u2014 ${message}`);
+      logger.warn(
+        `  [${myProgress}/${toFetch.length}] ${release.tag} \u2014 ${message}`
+      );
     }
   });
-  fetchedTemplates.sort((a, b) => {
-    const idCmp = (a.id ?? "").localeCompare(b.id ?? "");
-    if (idCmp !== 0) return idCmp;
-    return (a.version ?? 0) - (b.version ?? 0);
-  });
   const next = [];
-  for (const tpl of existing) {
-    const ref = tpl.metadata?.upstreamRef;
-    if (ref && freshRefs.has(ref)) {
-      next.push(tpl);
+  const seen = /* @__PURE__ */ new Set();
+  let fetched = 0;
+  let cached = 0;
+  const add = (tpl) => {
+    const key = `${tpl.id}@${tpl.version}`;
+    if (seen.has(key)) return false;
+    seen.add(key);
+    next.push(tpl);
+    return true;
+  };
+  for (const release of releases) {
+    const fresh = fetchedByAsset.get(release.assetUrl);
+    const templates = fresh ?? cachedByAsset.get(release.assetUrl) ?? [];
+    for (const tpl of sortTemplates(templates)) {
+      if (!add(tpl)) continue;
+      if (fresh) fetched += 1;
+      else cached += 1;
     }
   }
-  next.push(...fetchedTemplates);
-  let pruned = 0;
-  if (prune) {
-    pruned = existing.filter((t) => {
-      const ref = t.metadata?.upstreamRef;
-      return ref === void 0 || !freshRefs.has(ref);
-    }).length;
+  const pruning = prune && errors === 0;
+  if (prune && errors > 0) {
+    logger.warn(
+      `Skipping prune: ${errors} bundle(s) failed to download \u2014 cached templates were kept.`
+    );
   }
-  if (!prune) {
-    for (const tpl of existing) {
-      const ref = tpl.metadata?.upstreamRef;
-      if (ref && !freshRefs.has(ref) && !next.includes(tpl)) {
-        next.push(tpl);
-      }
+  if (!pruning) {
+    for (const tpl of sortTemplates(staleCached)) {
+      add(tpl);
     }
   }
-  saveCache(next);
+  const pruned = pruning ? staleCached.length : 0;
+  if (next.length === 0) {
+    throw new Error(
+      `No element templates could be downloaded (${errors} bundle(s) failed). Check the warnings above and retry 'c8ctl element-template sync'.`
+    );
+  }
+  saveCache(next, { stampFetchedAt: errors === 0 });
   const summary = {
     total: next.length,
     fetched,
-    cached: entries.length - toFetch.length,
+    cached,
     errors,
     pruned
   };
@@ -9015,12 +9151,12 @@ function pickVersion(templates, { version, executionPlatformVersion } = {}) {
   }
   let candidates = templates.filter((t) => t.version !== void 0);
   if (executionPlatformVersion) {
-    const coerced = import_semver2.default.coerce(executionPlatformVersion);
+    const coerced = import_semver3.default.coerce(executionPlatformVersion);
     if (coerced) {
       candidates = candidates.filter((t) => {
         const constraint = t.engines?.camunda;
         if (!constraint) return true;
-        return import_semver2.default.satisfies(coerced, constraint);
+        return import_semver3.default.satisfies(coerced, constraint);
       });
     }
   }
@@ -9040,12 +9176,12 @@ function searchTemplates(query, { executionPlatformVersion } = {}) {
     return name2.includes(q) || description.includes(q) || id.includes(q) || keywords.includes(q);
   }).filter((t) => !t.deprecated);
   if (executionPlatformVersion) {
-    const coerced = import_semver2.default.coerce(executionPlatformVersion);
+    const coerced = import_semver3.default.coerce(executionPlatformVersion);
     if (coerced) {
       matches = matches.filter((t) => {
         const constraint = t.engines?.camunda;
         if (!constraint) return true;
-        return import_semver2.default.satisfies(coerced, constraint);
+        return import_semver3.default.satisfies(coerced, constraint);
       });
     }
   }
@@ -10463,7 +10599,7 @@ var commands = {
       },
       prune: {
         type: "boolean",
-        description: "Drop cached entries no longer in the index [sync]"
+        description: "Drop cached entries no longer in a selected release [sync]"
       },
       "no-icon": {
         type: "boolean",

package/dist/default-plugins/element-template/docs/design.md

@@ -24,29 +24,61 @@ shipped layout but during development we run directly from
 
 ## OOTB template integration
 
-Three sources publish the OOTB connector templates:
+Four sources publish the OOTB connector templates:
 
 | | Source-of-truth | Inlines templates? | Has all versions? | Used by |
 |---|---|---|---|---|
+| `github.com/camunda/connectors` releases (bundle asset) | yes | yes (~1 MB gzip per line) | per minor line | c8ctl |
 | `marketplace.cloud.camunda.io/api/v1/ootb-connectors` | mirrors GH | no, URL refs | yes | Desktop Modeler |
 | `github.com/camunda/connectors/connector-templates.json` | yes | no, URL refs | yes | (the marketplace) |
 | `@camunda/connectors-element-templates` (npm) | derived | yes (~9MB) | yes, may lag | (Web Modeler skill) |
 
-We chose the **marketplace endpoint** for parity with Desktop Modeler:
-
-- Desktop Modeler is released and deployed widely; the endpoint contract
-  is effectively stable.
-- The endpoint already proxies the GH source-of-truth; using it means
-  we benefit from any future server-side filtering/versioning the
-  marketplace adds.
+We use the **release bundle assets** — every connectors release ships a
+`connectors-bundle-templates-<tag>.tar.gz` with one JSON file per
+template:
+
+- The marketplace endpoint (Desktop Modeler's source, and c8ctl's
+  original choice) only publishes *refs* into
+  `raw.githubusercontent.com`, a host routinely blocked in enterprise
+  networks — the reason for the switch, see c8ctl#530. Release
+  downloads are served from `github.com`, which those environments
+  generally do reach.
+- One request per minor line replaces ~650 per-template requests, so a
+  cold sync is a few MB and a few seconds instead of ~14 MB and ~30 s.
 - npm package was rejected: confirmed lag against the GH source and
   some entries are missing `version`/`engines` (pre-versioned legacy
   templates).
-- GH directly was rejected because the marketplace gives us the same
-  content with one less reason to drift from Modeler's behavior.
 
-The endpoint is overridable via `C8CTL_OOTB_ELEMENT_TEMPLATES_URL`
-(useful for testing).
+### Which releases
+
+`sync` lists the releases (`api.github.com/repos/camunda/connectors/
+releases?per_page=100`, overridable via
+`C8CTL_CONNECTORS_RELEASES_URL`) and keeps **the newest release of each
+minor line** — 8.8.x, 8.9.x, 8.10.x, ...:
+
+- Bundles are cumulative within a line, so the newest patch of 8.8
+  already contains everything 8.8.17 shipped. Across lines they differ,
+  which is why one release per line is kept rather than just the newest
+  release overall.
+- Alphas count (`8.10.0-alpha3` is the only source for a line that has
+  no stable release yet); release candidates do not — they are
+  superseded within days. Note that semver ranks `8.10.0-alpha5-rc3`
+  *above* `8.10.0-alpha5`, so RCs are filtered explicitly rather than
+  by ordering.
+- Draft releases and releases without a bundle asset (a release that is
+  still being built) are skipped, so an in-flight release falls back to
+  the previous one on that line.
+- Only the **4 newest minor lines** are kept (`MAX_MINOR_LINES` in
+  `releases.ts`), roughly Camunda's supported-version window. This keeps
+  the selection deterministic: the newest release of each selected line
+  is always within the first page of the listing, whereas an EOL line's
+  newest release drifts down the listing until it falls off the page and
+  would silently vanish from the selection.
+
+Bundles contain a superset of the marketplace index: the `-hybrid`
+variants and a few templates the marketplace does not list are cached
+too, and templates without a numeric `version` (pre-versioned legacy
+entries that version resolution cannot rank) are skipped.
 
 ## Cache strategy
 
@@ -56,12 +88,17 @@ We mirror Desktop Modeler's approach
 - Cache lives in `<userDataDir>/element-templates/`:
   - `templates.json` — flat array of all template objects (matches
     Modeler's `.camunda-connector-templates.json` shape).
-  - `fetched-at` — epoch ms of last index sync.
-- Each cached template gets `metadata.upstreamRef = <ref-url>` injected.
-  Since each `ref` is a commit-pinned `raw.githubusercontent.com` URL,
-  "upstreamRef unchanged" ⇒ "content unchanged" ⇒ no re-fetch needed.
-  Subsequent syncs only fetch refs that aren't already in the cache.
-- Per-template fetch failures are logged + counted, never abort the run.
+  - `fetched-at` — epoch ms of last sync.
+- Each cached template gets
+  `metadata.upstreamRef = <assetUrl>#<file-in-bundle>` injected. Release
+  assets are tag-pinned and therefore immutable, so "asset URL already
+  in the cache" ⇒ "bundle already ingested" ⇒ no re-download.
+- `id@version` is deduplicated across bundles, newest release wins.
+- Per-release fetch failures are logged + counted, and never abort the
+  run — unless *every* bundle failed and nothing was reusable from the
+  cache, in which case `sync` errors out rather than writing an empty
+  cache. A partial sync also leaves `fetched-at` untouched, so the
+  staleness nudge keeps asking for a full refresh.
 
 ### Lifecycle
 
@@ -79,11 +116,16 @@ We mirror Desktop Modeler's approach
   arg in `parseTemplateRef` before any cache call.
 - **Stale cache** (>7 days since `fetched-at`): warn-only, suggesting
   `c8ctl element-template sync`. We don't auto-refresh — surprise
-  network activity inside `apply` is undesirable and the index is
-  small enough that manual sync is cheap.
-- **`sync`** always re-fetches the index but skips already-cached
-  refs. **`sync --prune`** drops cached entries no longer in the
-  fresh index (opt-in: a user may keep a legacy version intentionally).
+  network activity inside `apply` is undesirable and a manual sync is
+  cheap.
+- **`sync`** always re-fetches the release listing but only downloads
+  bundles that aren't cached yet. **`sync --prune`** drops cached
+  entries that no longer belong to a selected release (opt-in: a user
+  may keep a legacy line intentionally). Pruning is skipped when any
+  bundle failed to download: a bundle URL changes with every patch, so
+  the previous patch's cached templates are always "stale", and dropping
+  them while their replacement failed would delete a whole minor line
+  over one transient HTTP error.
 - **Atomicity**: `sync` writes `templates.json` and `fetched-at`
   via a sibling temp file + `renameSync`, so a kill mid-sync leaves
   the previous cache intact. `apply --in-place` uses the same recipe
@@ -100,9 +142,9 @@ We mirror Desktop Modeler's approach
 
 Earlier draft considered fetching only the requested `(id, version)`
 on demand. Rejected because **search needs the template names**, which
-live inside the template files (the index has only `id`/`version`/`ref`).
-Eager bulk fetch makes search instant; the ~14 MB / ~30 s first-run
-cost is acceptable, and incremental syncs are cheap.
+means every template has to be read anyway. Bundles make this moot: the
+whole line arrives in one request, so eager bulk fetch is both simpler
+and faster than any lazy scheme.
 
 ## Version resolution
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@camunda8/cli",
-  "version": "4.1.0-alpha.1",
+  "version": "4.1.0-alpha.2",
   "description": "Camunda 8 CLI - minimal-dependency CLI for Camunda 8 operations",
   "type": "module",
   "engines": {