@jc_stack/ez-agents 0.1.0-beta.21 → 0.1.0-beta.24

package/CHANGELOG.md

@@ -1,5 +1,27 @@
 # Changelog
 
+## 0.1.0-beta.24
+
+- Preserve host-owned private plugin networks across generated Compose services,
+  one-shot commands and compatible updates. Bind routes to reviewed plugin
+  revisions and retain plugin-local connectivity.
+- Support existing deployments on their first upgrade. Network authorization
+  remains outside the agent-writable registry. Existing beta acceptance limits remain.
+
+## 0.1.0-beta.23
+
+- Identify failed publisher API reads and missing release tags without exposing
+  credentials or signed asset URLs. Document verified tag/artifact staging and
+  recovery before a fresh publication attempt.
+- Include beta.22 incomplete-update-receipt recovery; the earlier unpublished
+  candidate remains preserved. Existing beta acceptance limits remain.
+
+## 0.1.0-beta.22
+
+- Ignore incomplete or malformed update-receipt directories until a valid,
+  atomically committed `job.json` exists. They have no activation authority and
+  must not take the host executor offline.
+
 ## 0.1.0-beta.21
 
 - Initialize fresh Telegram bots before reading their identity. Beta.19 could

package/docs/plugins.md

@@ -198,12 +198,33 @@ Secrets persist privately across reinstall and are never included in registry
 responses. Twenty provides a complete v2 backend example.
 
 Compose JSON is generated, not accepted from untrusted arbitrary Compose input.
-No ports, host network, privileged services, host env inheritance, arbitrary bind mounts,
-Docker socket or raw Compose args can be supplied. Project names include a hash
-of the canonical registry path; networks and volumes inherit that namespace.
+No ports, host-network mode, privileged services, host env inheritance, arbitrary bind mounts,
+Docker socket or raw Compose args can be supplied. Package descriptors cannot select
+Docker networks. Project names include a hash of the canonical registry path;
+networks and volumes inherit that namespace.
 Images use release-specific names. All containers drop capabilities and run as
 UID 1000 by default; v2 can declare another non-root UID:GID. Plugins sharing a profile remain in the same owning deployment.
 
+### Host-owned private networks
+
+Some private deployments need a reviewed plugin service to reach another private
+Compose project. The installation host may save `pluginNetworkBindings` on that
+agent's entry in its host-owned `host-executor.json`, keyed by plugin id. Each
+route records the explicitly approved reviewed plugin `revisions` and literal
+`{service, network}` entries. The manager re-hashes the installed snapshot
+before accepting a route, then emits it for both persistent services and
+one-shot command containers. Bound services keep their plugin's default network
+as well, so they retain access to plugin-local dependencies. Routes therefore
+survive command-time Compose regeneration and restarts. Before a routed plugin
+update, the host adds the inspected candidate revision to that route; both the
+current and candidate revisions may remain authorized during the transition.
+
+This is deployment configuration, not a package-descriptor field or an `ez`
+command: agents and plugins cannot request or alter host networks. The host
+installer owns the binding and must ensure the selected service is stopped
+before changing its network topology. The executor's writable tool registry is
+never a source of network attachment authority.
+
 Commands run in one-shot client containers against their own service's volumes;
 stdin, stdout, stderr, literal arguments and exit codes are preserved. SIGINT/
 SIGTERM cancel the Docker call and remove its unique client container. The manager

package/docs/trusted-publishing.md

@@ -65,8 +65,28 @@ jobs from its latest attempt. Tag, PR and other workflow runs cannot shadow it;
 failed, pending or incomplete main CI cannot fall back to an older success.
 
 Create the `vVERSION` tag at that exact source commit and a **draft prerelease**
-with these assets, using native `gh release create --draft --prerelease` and
-`gh release upload` under existing release authority:
+with these assets under existing release authority. **Creating a draft release
+with `--target` does not create a Git tag.** Create and push the tag explicitly,
+then use `--verify-tag` to prevent staging against a missing remote tag:
+
+```sh
+# VERSION and SOURCE_SHA identify the reviewed, current-main candidate.
+git tag "v$VERSION" "$SOURCE_SHA"
+git push origin "refs/tags/v$VERSION"
+git ls-remote origin "refs/tags/v$VERSION"
+# Copy the already tested bytes, without rebuilding, to this exact basename.
+cp /absolute/tested-package.tgz /absolute/release/candidate.tgz
+gh release create "v$VERSION" --repo OWNER/REPO --verify-tag --draft --prerelease \
+  --title "v$VERSION" --notes-file /absolute/release/notes.md \
+  /absolute/release/candidate.tgz /absolute/release/release-receipt.json
+```
+
+If the tag already exists, read and verify its commit (peel annotated tags) rather
+than recreating or force-pushing it. Read back the draft by numeric ID and check
+asset names and downloaded SHA-256 before dispatch. `npm pack`'s default filename
+is not the publisher's asset name; GitHub asset labels do not rename the asset.
+Use `gh release upload` only to add a missing asset, never `--clobber` to replace
+candidate bytes or receipts. The required assets are:
 
 - `candidate.tgz`: the exact Mac-tested bytes from `npm pack --ignore-scripts`.
   Do not rebuild it on Actions.
@@ -114,6 +134,22 @@ identity on the release PR. A failed command after the publish call may mean npm
 accepted it: inspect registry state first. A rerun may verify an existing exact
 version; if the version is absent it refuses a second write. Reconcile first,
 then create a fresh authorized dispatch if appropriate. Never repeat or overwrite that version or silently repair tags.
+When validation fails, use the logged GitHub API path to identify the missing
+input. A tag lookup 404 is not evidence of a draft-release permission problem.
+A draft/asset 404 can mean absent evidence or insufficient access; verify with
+the existing authorized identity before diagnosing credentials. Do not broaden
+token permissions based on a generic 404.
+
+For a failure before the publish job starts, confirm that job was skipped and
+read npm for the exact version. If absent, repair missing staging inputs against
+the same approved commit/bytes, verify tag, asset names and digest, and create a
+fresh dispatch. If main or the package changes, prepare a new reviewed version;
+preserve the old draft instead of moving its tag or replacing its artifact.
+If any npm write may have started, use exact registry/artifact reconciliation
+above first. Publication recovery never means rolling back a running agent.
+Runtime upgrades and `failed`/`rolled-back` versus `recovery-required` recovery
+follow [upgrades](upgrades.md) under the installation's saved policy.
+
 Missing trust or registry access is an external dependency, not a reason to use
 a token workaround.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@jc_stack/ez-agents",
-  "version": "0.1.0-beta.21",
+  "version": "0.1.0-beta.24",
   "private": false,
   "type": "module",
   "description": "A lightweight foundation for persistent business AI assistants using existing AI harnesses, workspaces and plugins.",

package/scripts/trusted-beta.mjs

@@ -87,8 +87,8 @@ export function tarManifest(path) {
   return JSON.parse(execFileSync('tar', ['-xOf', path, 'package/package.json'], { ...options, maxBuffer: 1024 * 1024 }));
 }
 
-async function responseBytes(response, max = MAX_ARTIFACT) {
-  assert(response.ok, `HTTP ${response.status} while reading release evidence`);
+async function responseBytes(response, max = MAX_ARTIFACT, context = 'release evidence') {
+  assert(response.ok, `HTTP ${response.status} while reading ${context}`);
   assert(Number(response.headers.get('content-length') || 0) <= max, 'Response too large');
   let size = 0; const chunks = [];
   for await (const chunk of response.body) { size += chunk.length; assert(size <= max, 'Response too large'); chunks.push(chunk); }
@@ -98,6 +98,8 @@ async function responseBytes(response, max = MAX_ARTIFACT) {
 export function githubClient(token, fetcher = fetch) {
   return async (path, binary = false) => {
     assert(path.startsWith('/repos/'), 'Invalid GitHub API path');
+    // Identify the failed read without logging tokens, response bodies or signed URLs.
+    const context = `GitHub GET ${path.split('?')[0]}`;
     const response = await fetcher(`https://api.github.com${path}`, {
       headers: { Accept: binary ? 'application/octet-stream' : 'application/vnd.github+json', ...(token ? { Authorization: `Bearer ${token}` } : {}), 'X-GitHub-Api-Version': '2022-11-28' },
       redirect: 'manual', signal: AbortSignal.timeout(30_000),
@@ -105,9 +107,12 @@ export function githubClient(token, fetcher = fetch) {
     if (binary && [301, 302, 303, 307, 308].includes(response.status)) {
       const location = new URL(response.headers.get('location'));
       assert(location.protocol === 'https:' && (location.hostname === 'release-assets.githubusercontent.com' || location.hostname === 'objects.githubusercontent.com'), 'Unexpected asset redirect');
-      return responseBytes(await fetcher(location, { signal: AbortSignal.timeout(60_000), redirect: 'error' }));
+      return responseBytes(await fetcher(location, { signal: AbortSignal.timeout(60_000), redirect: 'error' }), MAX_ARTIFACT, `${context} asset download`);
     }
-    const bytes = await responseBytes(response, binary ? MAX_ARTIFACT : 8 * 1024 * 1024);
+    if (response.status === 404 && path.includes('/git/ref/tags/')) {
+      throw new Error(`HTTP 404 while reading ${context}: release tag is missing or inaccessible; a draft release does not create its Git tag. Verify the remote tag at the reviewed source before dispatch.`);
+    }
+    const bytes = await responseBytes(response, binary ? MAX_ARTIFACT : 8 * 1024 * 1024, context);
     return binary ? bytes : JSON.parse(bytes.toString());
   };
 }

package/src/host-executor.ts

@@ -13,7 +13,8 @@ import { taskWorkspace } from './task-workspace.js'
 import { packageVersion } from './version.js'
 import { installedPluginVersions } from './software-status.js'
 
-export type HostBinding = { name: string; workspace: string; controlDir: string; binDir: string; toolsHome?: string; sharedWorkspace?: string }
+export type PluginNetworkRoute = { revisions:string[]; bindings:{service:string;network:string}[] }
+export type HostBinding = { name: string; workspace: string; controlDir: string; binDir: string; toolsHome?: string; sharedWorkspace?: string; pluginNetworkBindings?: Record<string, PluginNetworkRoute> }
 export type HostInstallation = { cli: string; agents: HostBinding[] }
 
 export const serveHostExecutor = async (installation: HostInstallation, signal: AbortSignal, launch = startExecutorJob) => {

package/src/plugins/manager.mjs

@@ -5,9 +5,11 @@ import path from 'node:path';
 import { fileURLToPath } from 'node:url';
 import { createHash, randomUUID, randomBytes } from 'node:crypto';
 import { spawn } from 'node:child_process';
+import { readFileSync, realpathSync } from 'node:fs';
 
 const reserved = new Set(['status','updates','plugins','tools','message','owner','approval','react','setup','help','version']);
 const id = value => { if(typeof value !== 'string' || !/^[a-z][a-z0-9-]{0,39}$/.test(value)) throw Error('Invalid identifier'); return value; };
+const dockerNetwork = value => { if(typeof value !== 'string' || !/^[A-Za-z0-9][A-Za-z0-9_.-]{0,127}$/.test(value)) throw Error('Invalid Docker network'); return value; };
 const hash = data => createHash('sha256').update(data).digest('hex');
 const json = async file => JSON.parse(await fs.readFile(file,'utf8'));
 const emit = value => console.log(JSON.stringify(value));
@@ -142,9 +144,67 @@ export async function checkFolders(config, record) {
       throw Error('Folder source must remain an existing real directory');
   }
 }
-export function compose(config, record, secrets={}) {
-  const services={}, volumes={};
+const childOf = (candidate, parent) => candidate === parent || candidate.startsWith(parent + path.sep);
+// Host-owned private network bindings remain separate from portable package descriptors.
+// The executor may write toolsHome, so the binding source must stay in its host config.
+export async function hostNetworkBindings(config, record, home) {
+  const configuredHost = config.hostConfig ?? (typeof config.deploymentDir === 'string' && path.isAbsolute(config.deploymentDir) ? path.join(config.deploymentDir, 'host-executor.json') : undefined);
+  if (configuredHost === undefined) return [];
+  if (!home || typeof configuredHost !== 'string' || !path.isAbsolute(configuredHost)) throw Error('Invalid host network binding');
+  home = realpathSync(home);
+  const hostConfig = realpathSync(configuredHost);
+  if (hostConfig !== configuredHost || path.basename(hostConfig) !== 'host-executor.json' || childOf(hostConfig, home))
+    throw Error('Host network bindings require an external host config');
+  const host = JSON.parse(readFileSync(hostConfig, 'utf8'));
+  if (!host || typeof host !== 'object' || !Array.isArray(host.agents)) throw Error('Invalid host network bindings');
+  const candidates = host.agents.filter(agent => agent && typeof agent === 'object' && typeof agent.toolsHome === 'string' && realpathSync(agent.toolsHome) === home);
+  if (candidates.length !== 1) throw Error('Host network binding does not belong to this registry');
+  const agent = candidates[0], deployment = path.dirname(hostConfig);
+  if (typeof agent.workspace !== 'string' || typeof agent.controlDir !== 'string' ||
+      realpathSync(agent.workspace) !== config.workspace || realpathSync(path.join(deployment, 'mind')) !== config.workspace ||
+      realpathSync(path.join(deployment, 'control')) !== realpathSync(agent.controlDir) ||
+      childOf(hostConfig, config.workspace) || childOf(hostConfig, realpathSync(agent.controlDir)))
+    throw Error('Invalid host network binding deployment');
+  const configured = agent.pluginNetworkBindings;
+  if (configured !== undefined && (!configured || typeof configured !== 'object' || Array.isArray(configured)))
+    throw Error('Invalid host network bindings');
+  const route = configured?.[record.manifest?.id];
+  if (route === undefined) return [];
+  keys(route, ['revisions', 'bindings']);
+  const trusted = await snapshot(record.source);
+  if (record.revision !== trusted.revision || record.source !== trusted.source ||
+      JSON.stringify(record.manifest) !== JSON.stringify(trusted.manifest) ||
+      JSON.stringify(record.deployment) !== JSON.stringify(trusted.deployment))
+    throw Error('Registry plugin identity is not the reviewed source');
+  if (record.manifest.id !== trusted.manifest.id) throw Error('Registry plugin identity is not the reviewed source');
+  if (!Array.isArray(route.revisions) || route.revisions.some(revision => typeof revision !== 'string' || !/^sha256:[a-f0-9]{64}$/.test(revision)) ||
+      !route.revisions.includes(trusted.revision)) throw Error('Host network binding is not pinned to the reviewed plugin revision');
+  const bindings = route.bindings;
+  if (!Array.isArray(bindings)) throw Error('Invalid host network bindings');
+  const seen = new Set();
+  for (const binding of bindings) {
+    keys(binding, ['service', 'network']);
+    id(binding.service); dockerNetwork(binding.network);
+    if (!record.deployment.services[binding.service]) throw Error('Unknown host network service');
+    const key = `${binding.service}\0${binding.network}`;
+    if (seen.has(key)) throw Error('Duplicate host network binding');
+    seen.add(key);
+  }
+  return bindings;
+}
+export async function compose(config, record, secrets={}, home) {
+  const services={}, volumes={}, networks={}, serviceNetworks=new Map();
   const folders = folderMounts(config, record);
+  for (const binding of await hostNetworkBindings(config, record, home)) {
+    // A service with an explicit network list loses Compose's implicit default.
+    // Keep the plugin's own network so bound services retain sibling access.
+    networks.default = {};
+    const key = `host-${hash(binding.network).slice(0,16)}`;
+    networks[key] = { external: true, name: binding.network };
+    const attached = serviceNetworks.get(binding.service) || ['default'];
+    if (!attached.includes(key)) attached.push(key);
+    serviceNetworks.set(binding.service, attached);
+  }
   for(const [name,s] of Object.entries(record.deployment.services)) {
     const mounts=[];
     for(const [volume,target] of Object.entries(s.volumes||{})) { volumes[volume]={};mounts.push({type:'volume',source:volume,target}); }
@@ -156,13 +216,14 @@ export function compose(config, record, secrets={}) {
       ...(s.dependsOn?{depends_on:Object.fromEntries(s.dependsOn.map(dep=>[dep,{condition:'service_healthy'}]))}:{}),
       ...(s.memoryMiB?{mem_limit:`${s.memoryMiB}m`}:{}),
       ...(s.cpus?{cpus:s.cpus}:{}),
+      ...(serviceNetworks.get(name)?.length?{networks:serviceNetworks.get(name)}:{}),
       ...(s.environment?{environment:Object.fromEntries(Object.entries(s.environment).map(([key,value])=>{
         if(typeof value==='string')return [key,value];
         if(!/^[a-f0-9]{64}$/.test(secrets[value.secret]||''))throw Error('Missing or invalid private deployment secret');
         return [key,(value.prefix||'')+secrets[value.secret]+(value.suffix||'')];
       }))}:{}),...(s.command?{command:s.command}:{})};
   }
-  return attachShared({name:record.project,services,volumes}, record);
+  return attachShared({name:record.project,services,volumes,...(Object.keys(networks).length?{networks}:{})}, record);
 }
 function dockerEnv() {
   return Object.fromEntries(['HOME','PATH','LANG','LC_ALL','TMPDIR','DOCKER_HOST','DOCKER_CONTEXT','DOCKER_CONFIG','BUILDX_CONFIG'].filter(k=>process.env[k]!==undefined).map(k=>[k,process.env[k]]));
@@ -205,14 +266,14 @@ async function registry(home) {
 export async function init(home,workspace,catalogFile,hostConfig,standalone=false) {
   if(standalone && hostConfig) throw Error('Standalone setup cannot bind a relay host config');
   if(typeof home!=='string'||typeof workspace!=='string'||!path.isAbsolute(home)||!path.isAbsolute(workspace)||/[\r\n\0$:,]/.test(home+workspace)) throw Error('Explicit absolute home/workspace required');
-  workspace=await fs.realpath(workspace);await privateDir(home);home=await fs.realpath(home);
+  workspace=await fs.realpath(workspace);await privateDir(home);home=await fs.realpath(home);if(hostConfig)hostConfig=await fs.realpath(hostConfig);
   if(await fs.lstat(path.join(home,'registry.json')).catch(()=>null)) throw Error('Registry already exists; refusing replacement');
   catalogFile=path.resolve(catalogFile||fileURLToPath(new URL('../../default-plugins.json',import.meta.url)));
   const sources=await json(catalogFile);
   const catalog={};
   for(const [name,source] of Object.entries(sources)) {id(name);if(typeof source!=='string'||!source)throw Error('Catalog source must be a nonempty path');const resolved=path.resolve(path.dirname(catalogFile),source);const p=await snapshot(resolved).catch(error=>{throw Error(`Cannot load reviewed plugin ${name} from ${resolved}: ${error.message}. Supply its checkout or an explicit --catalog file.`)});if(p.manifest.id!==name) throw Error('Catalog ID mismatch');catalog[name]={source:p.source,revision:p.revision};}
   await locked(home,async()=>{
-    await atomic(path.join(home,'config.json'),{schemaVersion:1,workspace,catalog});
+    await atomic(path.join(home,'config.json'),{schemaVersion:1,workspace,catalog,...(hostConfig&&path.basename(hostConfig)==='host-executor.json'?{hostConfig}:{})});
     await atomic(path.join(home,'registry.json'),{schemaVersion:1,owner:home,plugins:{},commands:{}});
     const bin=path.join(home,'bin');await privateDir(bin);
     await fs.writeFile(path.join(bin,'ez'),`#!${process.execPath}\nimport(${JSON.stringify(new URL('./manager.mjs',import.meta.url).href)}).then(m=>m.main(['--home',${JSON.stringify(home)},...process.argv.slice(2)])).catch(e=>{console.error(e.message);process.exitCode=1});\n`,{mode:0o700,flag:'wx'});
@@ -262,7 +323,7 @@ export async function install(home,config,name,source,revision) {
     let secrets;try{secrets=await json(secretsFile);}catch(error){if(error.code!=='ENOENT')throw error;secrets={};}
     for(const name of p.deployment.secrets||[])if(secrets[name]===undefined)secrets[name]=randomBytes(32).toString('hex');
     if(p.deployment.secrets?.length)await atomic(secretsFile,secrets);
-    await atomic(record.compose,compose(config,record,secrets));
+    await atomic(record.compose,await compose(config,record,secrets,home));
     // Build/pull is explicit installation mechanics. No services start and no onboarding is executed.
     for(const [service,s] of Object.entries(record.deployment.services)) await checked([...composeArgs(record),s.image?'pull':'build',service]);
     r.plugins[name]=record;for(const alias of Object.keys(p.manifest.commands)) r.commands[alias]=name;
@@ -327,7 +388,7 @@ export async function main(args) {
         settings.folders={...settings.folders,[name]:folders};
         await checkFolders(settings,latest);
         const secrets=await json(path.join(home,'packages',name,'secrets.json')).catch(e=>{if(e.code==='ENOENT')return {};throw e;});
-        const generated=compose(settings,latest,secrets);
+        const generated=await compose(settings,latest,secrets,home);
         await atomic(path.join(home,'config.json'),settings);
         await atomic(latest.compose,generated);
         return emit({ok:true,plugin:name,folders,readOnly:true,started:false});
@@ -345,7 +406,7 @@ export async function main(args) {
         const result = action === 'shared-enable' ? await sharedService(latest, key, 'enable', run) : { state: 'detached' };
         latest.sharedEnabled = [...new Set([...(latest.sharedEnabled || []).filter(k => k !== key), ...(action === 'shared-enable' ? [key] : [])])];
         const secrets = await json(path.join(home, 'packages', name, 'secrets.json')).catch(e => { if (e.code === 'ENOENT') return {}; throw e; });
-        await atomic(latest.compose, compose(currentConfig, latest, secrets));
+        await atomic(latest.compose, await compose(currentConfig, latest, secrets, home));
         // Persist the binding before recreating clients; start can recover an interrupted recreation.
         await atomic(path.join(home, 'registry.json'), current);
         await checked([...composeArgs(latest), 'up', '-d', '--wait']);
@@ -371,7 +432,7 @@ export async function main(args) {
         const currentConfig=await json(path.join(home,'config.json'));
         await checkFolders(currentConfig,current.plugins[name]);
         const secrets=await json(path.join(home,'packages',name,'secrets.json')).catch(e=>{if(e.code==='ENOENT')return {};throw e;});
-        await atomic(record.compose,compose(currentConfig,current.plugins[name],secrets));
+        await atomic(record.compose,await compose(currentConfig,current.plugins[name],secrets,home));
       }
       await checked([...composeArgs(record),...(action==='start'?['up','-d','--wait']:action==='stop'?['stop']:['down'])]);
       if(action==='uninstall') {delete current.plugins[name];for(const [alias,owner] of Object.entries(current.commands))if(owner===name)delete current.commands[alias];await atomic(path.join(home,'registry.json'),current);}
@@ -384,7 +445,7 @@ export async function main(args) {
   if(!binding)throw Error('Unknown registered CLI');
   await checkFolders(config,record);
   const secrets=await json(path.join(home,'packages',record.manifest.id,'secrets.json')).catch(e=>{if(e.code==='ENOENT')return {};throw e;});
-  await atomic(record.compose,compose(config,record,secrets));
+  await atomic(record.compose,await compose(config,record,secrets,home));
   // Docker exec does not reliably forward cancellation to the in-container process.
   // Run each client as a one-shot Compose container; docker compose run forwards signals.
   const name=`${record.project}-call-${randomUUID()}`;

package/src/updates/binding.mjs

@@ -11,7 +11,7 @@ export async function bindUpdates(home,hostConfig,packageRoot=fileURLToPath(new
   if(await fs.realpath(path.join(deploymentDir,'mind'))!==config.workspace||await fs.realpath(path.join(deploymentDir,'control'))!==host.agents[0].controlDir)throw Error('Noncanonical deployment state paths');
   packageRoot=await fs.realpath(packageRoot);
   await fs.mkdir(path.join(home,'updates'),{recursive:true,mode:0o700});
-  await atomic(path.join(home,'config.json'),{...config,packageRoot:packageRoot.replace(/\/$/,''),deploymentDir});
+  await atomic(path.join(home,'config.json'),{...config,hostConfig,packageRoot:packageRoot.replace(/\/$/,''),deploymentDir});
   const pkg=await read(path.join(packageRoot,'package.json')),bin=path.join(home,'bin');
   // Every invocation resolves the active root. Already-running workers retain
   // their old code; the next worker receives the new package after activation.

package/src/updates/control.mjs

@@ -7,6 +7,7 @@ import { digest, extract, newer, compatible, version, releaseContract, registryV
 export const read = async file => JSON.parse(await fs.readFile(file,'utf8'));
 export const missing = error => {if(error.code!=='ENOENT')throw error;return null;};
 export const targetId = value => {if(value!=='main'&&!/^[a-z][a-z0-9-]{0,39}$/.test(value))throw Error('Invalid update target');return value;};
+const jobId=/^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$/;
 export const updateHome = home => path.join(home,'updates');
 export async function state(home) {
   const config=await read(path.join(home,'config.json'));
@@ -77,12 +78,15 @@ export async function prepare(home,target,{file,release}) {
   } catch(error){await fs.rm(dir,{recursive:true,force:true});throw error;}
 }
 export function jobPath(home,id) {
-  if(!/^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$/.test(id))throw Error('Invalid upgrade job ID');
+  if(!jobId.test(id))throw Error('Invalid upgrade job ID');
   return path.join(updateHome(home),id);
 }
 export async function jobs(home) {
   const entries=await fs.readdir(updateHome(home),{withFileTypes:true}).catch(error=>{if(error.code==='ENOENT')return [];throw error;});
-  return Promise.all(entries.filter(e=>e.isDirectory()&&/^[a-f0-9-]{36}$/.test(e.name)).map(e=>read(path.join(jobPath(home,e.name),'job.json'))));
+  // A job becomes visible only when its receipt is atomically committed. A crash
+  // before that point leaves no activation authority and must not stop the host.
+  const found=await Promise.all(entries.filter(e=>e.isDirectory()&&jobId.test(e.name)).map(e=>read(path.join(jobPath(home,e.name),'job.json')).catch(missing)));
+  return found.filter(Boolean);
 }
 async function requireSupervisor(directory) {
   const h=await read(path.join(directory,'supervisor.json'));

package/src/updates/runtime.mjs

@@ -58,7 +58,7 @@ function envValue(text,key,value) {
 }
 async function backupVolume(run,record,name,directory) {
   const services=Object.entries(record.deployment.services),service=services.find(([,s])=>s.volumes?.[name])?.[0];
-  const c=compose({workspace:'/unused'},record),image=c.services[service].image;
+  const c=await compose({workspace:'/unused'},record),image=c.services[service].image;
   // No writable profile mount, network, Docker socket, provider command or secrets.
   const found=(await run('docker',['volume','ls','--format','{{.Name}}','--filter',`name=^${record.project}_${name}$`])).trim();
   if(!found)return; // Installed but never started: no profile exists yet.
@@ -112,7 +112,7 @@ export async function perform(home,job,hooks) {
       const s=await snapshot(root),candidate={...old,source:root,revision:s.revision,manifest:s.manifest,deployment:s.deployment,sharedRevisions:s.sharedRevisions};
       for (const key of old.sharedEnabled || []) if (sharedIdentity(old, key).fingerprint !== sharedIdentity(candidate, key).fingerprint) throw Error('Shared worker changed; disable this client and coordinate an explicit shared worker upgrade before updating');
       await checkFolders(config,candidate);
-      const stage={...candidate,compose:path.join(dir,'compose.json')};await atomic(stage.compose,compose(config,stage,secrets));
+      const stage={...candidate,compose:path.join(dir,'compose.json')};await atomic(stage.compose,await compose(config,stage,secrets,home));
       for(const [service,spec] of Object.entries(stage.deployment.services))await run('docker',[...pluginArgs(stage),spec.image?'pull':'build',service]);
       const running=Boolean((await run('docker',[...pluginArgs(old),'ps','-q'])).trim());
       job.rollback={record:old,registry:r,compose:await read(old.compose),running};await save();
@@ -120,7 +120,7 @@ export async function perform(home,job,hooks) {
       const backup=path.join(dir,'backup');await fs.mkdir(backup,{mode:0o700});
       const volumes=new Set(Object.values(old.deployment.services).flatMap(s=>Object.keys(s.volumes||{})));
       for(const name of volumes)await backupVolume(run,old,name,backup);
-      await atomic(old.compose,compose(config,candidate,secrets));
+      await atomic(old.compose,await compose(config,candidate,secrets,home));
       if(running)await run('docker',[...pluginArgs(candidate),'up','-d','--wait','--wait-timeout','90','--no-build']);
       r.plugins[job.target]=candidate;await atomic(path.join(home,'registry.json'),r);
       job.runtimeVerified=running;

package/test/plugin-manager.test.mjs

@@ -30,8 +30,8 @@ for(const cleanup of ['success','failure','already-removed']) test(`cancel remov
 });
 async function fixture(t) {
  const root=await fs.mkdtemp(path.join(tmpdir(),'ez-tools-'));t.after(()=>fs.rm(root,{recursive:true,force:true}));
- const source=path.join(root,'source'),home=path.join(root,'tools'),workspace=path.join(root,'mind'),fake=path.join(root,'fake');
- for(const d of [source,workspace,fake])await fs.mkdir(d);
+ const source=path.join(root,'source'),home=path.join(root,'tools'),deploymentDir=path.join(root,'deployment'),workspace=path.join(deploymentDir,'mind'),control=path.join(deploymentDir,'control'),hostConfig=path.join(deploymentDir,'host-executor.json'),fake=path.join(root,'fake');
+ for(const d of [source,workspace,control,fake])await fs.mkdir(d,{recursive:true});
  const manifest={schemaVersion:1,id:'sample',version:'0.1.0',description:'Synthetic',commands:{sample:{executable:'client.mjs',args:[]}},skills:['SKILL.md']};
  const deployment={schemaVersion:1,services:{sample:{buildTarget:'runtime',volumes:{data:'/data'},workspace:true,healthcheck:['node','--version']}},commands:{sample:{service:'sample',argv:['node','/app/client.mjs'],suffix:[]}}};
  for(const [name,value]of Object.entries({'package.json':JSON.stringify({files:['client.mjs','SKILL.md']}),'ez-plugin.json':JSON.stringify(manifest),'ez-deployment.json':JSON.stringify(deployment),'Dockerfile':'FROM scratch AS runtime','.dockerignore':'','client.mjs':'','SKILL.md':'Synthetic'}))await fs.writeFile(path.join(source,name),value);
@@ -39,7 +39,7 @@ async function fixture(t) {
  await fs.writeFile(path.join(fake,'docker'),`#!${process.execPath}\nrequire('fs').appendFileSync(${JSON.stringify(log)},JSON.stringify({argv:process.argv.slice(2),secret:process.env.TELEGRAM_BOT_TOKEN})+'\\n');\nif(process.argv.includes('run')){console.log(JSON.stringify(process.argv.slice(process.argv.indexOf('/app/client.mjs')+1)));process.exit(process.argv.includes('fail')?17:0)}\n`,{mode:0o700});
  const env={...process.env,PATH:fake+path.delimiter+process.env.PATH,TELEGRAM_BOT_TOKEN:'must-not-leak'};
  const call=(...args)=>exec(process.execPath,[bin,'--home',home,...args],{env});
- return {root,source,home,workspace,fake,log,manifest,deployment,env,call};
+ return {root,source,home,deploymentDir,workspace,control,hostConfig,fake,log,manifest,deployment,env,call};
 }
 test('catalog paths resolve relative to the catalog and pin each new agent independently',async t=>{
  const f=await fixture(t),catalog=path.join(f.root,'defaults.json');
@@ -95,13 +95,45 @@ test('registry corruption, active writer lock and alias collision fail closed',a
 });
 test('Compose resources are namespaced, private, and restricted to owning workspace',async t=>{
  const f=await fixture(t),p=await snapshot(f.source);
- const c=compose({workspace:f.workspace},{source:f.source,project:'ezp-synthetic',revision:p.revision,deployment:p.deployment});
+ const c=await compose({workspace:f.workspace},{source:f.source,project:'ezp-synthetic',revision:p.revision,deployment:p.deployment});
  assert.equal(c.name,'ezp-synthetic');assert.equal(c.services.sample.ports,undefined);assert.equal(c.services.sample.privileged,undefined);assert.equal(c.services.sample.volumes[1].read_only,true);assert.deepEqual(c.services.sample.cap_drop,['ALL']);
 });
+test('host-owned private networks survive registered-call Compose regeneration',async t=>{
+ const f=await fixture(t),p=await snapshot(f.source);await init(f.home,f.workspace);
+ await f.call('plugins','install','sample','--source',f.source,'--revision',p.revision);
+ const config=JSON.parse(await fs.readFile(path.join(f.home,'config.json'),'utf8'));
+ const binding={service:'sample',network:'stocks-vm_default'};
+ const host={cli:'synthetic',agents:[{name:'sample',workspace:f.workspace,controlDir:f.control,binDir:path.join(f.home,'bin'),toolsHome:f.home,pluginNetworkBindings:{sample:{revisions:[p.revision],bindings:[binding]}}}]};
+ await fs.writeFile(f.hostConfig,JSON.stringify(host));
+ config.hostConfig=await fs.realpath(f.hostConfig);
+ await fs.writeFile(path.join(f.home,'config.json'),JSON.stringify(config));
+ const file=path.join(f.home,'packages/sample/compose.json');await fs.writeFile(file,'{}');
+ await f.call('sample','read');
+ const c=JSON.parse(await fs.readFile(file,'utf8'));
+ assert.deepEqual(c.networks.default,{});
+ const [key,network]=Object.entries(c.networks).find(([name])=>name!=='default');
+ assert.equal(network.name,'stocks-vm_default');assert.equal(network.external,true);
+ assert.deepEqual(c.services.sample.networks,['default',key]);
+ const compromised={...config,hostNetworkBindings:{sample:[{service:'sample',network:'untrusted_default'}]}};
+ assert.deepEqual((await compose(compromised,p,{},f.home)).services.sample.networks,['default',key]);
+ const upgraded={...config,hostConfig:undefined,deploymentDir:path.dirname(config.hostConfig)};
+ assert.deepEqual((await compose(upgraded,p,{},f.home)).services.sample.networks,['default',key]);
+ const unsafeHost=path.join(f.home,'host-executor.json');await fs.writeFile(unsafeHost,JSON.stringify(host));
+ await assert.rejects(compose({...config,hostConfig:unsafeHost},p,{},f.home),/external host config/);
+ const forged=structuredClone(p);forged.manifest.id='forged';
+ host.agents[0].pluginNetworkBindings={forged:{revisions:[p.revision],bindings:[binding]}};await fs.writeFile(f.hostConfig,JSON.stringify(host));
+ await assert.rejects(compose(config,forged,{},f.home),/reviewed source/);
+ for(const bindings of [
+   {sample:[{service:'missing',network:'stocks-vm_default'}]},
+   {sample:[{service:'sample',network:'bad/network'}]},
+   {sample:[{service:'sample',network:'stocks-vm_default'},{service:'sample',network:'stocks-vm_default'}]},
+   'not-a-binding-map'
+ ]) { host.agents[0].pluginNetworkBindings=bindings==='not-a-binding-map'?bindings:{sample:{revisions:[p.revision],bindings}};await fs.writeFile(f.hostConfig,JSON.stringify(host));await assert.rejects(compose(config,{source:p.source,project:'ezp-synthetic',revision:p.revision,manifest:p.manifest,deployment:p.deployment},{},f.home),/host network|Docker network/); }
+});
 test('host onboarding binds local ez without replacing global commands',async t=>{
  const f=await fixture(t),native=path.join(f.root,'native'),config=path.join(f.root,'host.json');await fs.mkdir(native);await fs.writeFile(path.join(native,'native-tool'),'hello');
  await fs.writeFile(config,JSON.stringify({cli:'synthetic',agents:[{name:'demo',workspace:f.workspace,binDir:native}]}));
- await init(f.home,f.workspace,undefined,config);const host=JSON.parse(await fs.readFile(config));assert.equal(host.agents[0].binDir,path.join(await fs.realpath(f.home),'bin'));assert.equal(host.agents[0].toolsHome,await fs.realpath(f.home));assert.equal(await fs.readFile(path.join(host.agents[0].binDir,'native-tool'),'utf8'),'hello');
+ await init(f.home,f.workspace,undefined,config);const host=JSON.parse(await fs.readFile(config));assert.equal(host.agents[0].binDir,path.join(await fs.realpath(f.home),'bin'));assert.equal(host.agents[0].toolsHome,await fs.realpath(f.home));assert.equal((JSON.parse(await fs.readFile(path.join(f.home,'config.json'),'utf8'))).hostConfig,undefined);assert.equal(await fs.readFile(path.join(host.agents[0].binDir,'native-tool'),'utf8'),'hello');
   await assert.rejects(init(f.home,f.workspace),/exists/);
 });
 test('host install exposes runnable public commands without source aliases',async t=>{
@@ -128,10 +160,10 @@ test('v2 supports bounded dependency graphs and generated private secrets',async
  d.services.database={image:'example/database@sha256:'+'a'.repeat(64),user:'999:999',healthcheck:['check'],memoryMiB:128,cpus:2,environment:{PASSWORD:{secret:'db-password'}}};
  d.services.sample.dependsOn=['database'];d.services.sample.environment={URL:{secret:'db-password',prefix:'db://',suffix:'@database'}};
  validate(f.manifest,d,p.files);
- const c=compose({workspace:f.workspace},{source:f.source,project:'ezp-synthetic',revision:p.revision,deployment:d},{'db-password':'b'.repeat(64)});
+ const c=await compose({workspace:f.workspace},{source:f.source,project:'ezp-synthetic',revision:p.revision,deployment:d},{'db-password':'b'.repeat(64)});
  assert.equal(c.services.database.user,'999:999');assert.equal(c.services.sample.depends_on.database.condition,'service_healthy');assert.equal(c.services.sample.environment.URL,'db://'+'b'.repeat(64)+'@database');
- assert.equal(c.services.database.mem_limit,'128m');assert.throws(()=>compose({workspace:f.workspace}, {source:f.source,project:'ezp-synthetic',revision:p.revision,deployment:d}),/Missing/);
- assert.equal(c.services.database.cpus,2);assert.throws(()=>compose({workspace:f.workspace}, {source:f.source,project:'ezp-synthetic',revision:p.revision,deployment:d}),/Missing/);
+ assert.equal(c.services.database.mem_limit,'128m');await assert.rejects(compose({workspace:f.workspace}, {source:f.source,project:'ezp-synthetic',revision:p.revision,deployment:d}),/Missing/);
+ assert.equal(c.services.database.cpus,2);await assert.rejects(compose({workspace:f.workspace}, {source:f.source,project:'ezp-synthetic',revision:p.revision,deployment:d}),/Missing/);
  for(const change of [x=>x.services.database.user='0:0',x=>x.services.database.cpus=0.01,x=>x.services.database.cpus=9,x=>x.services.database.environment.PASSWORD={secret:'undeclared'},x=>x.services.database.environment.PASSWORD='${HOST_SECRET}',x=>x.services.database.dependsOn=['sample'],x=>x.services.sample.dependsOn=['missing'],x=>x.services.sample.ports=['9999:9999']]){const bad=structuredClone(d);change(bad);assert.throws(()=>validate(f.manifest,bad,p.files));}
  const old=structuredClone(d);old.schemaVersion=1;assert.throws(()=>validate(f.manifest,old,p.files));
 });
@@ -252,9 +284,9 @@ test('folder bindings survive compatible descriptors and reject incompatible upd
  const f=await fixture(t);const p=await snapshot(f.source);
  const record={...p,project:'ezp-test-sample'};
  const config={workspace:f.workspace,folders:{sample:[{service:'sample',source:f.workspace,target:'/data/files'}]}};
- assert.equal(compose(config,record).services.sample.volumes.find(v=>v.target==='/data/files').read_only,true);
+ assert.equal((await compose(config,record)).services.sample.volumes.find(v=>v.target==='/data/files').read_only,true);
  const next=structuredClone(record);next.deployment.services.sample.volumes={data:'/new-state'};
- assert.throws(()=>compose(config,next),/child of a declared volume/);
+ await assert.rejects(compose(config,next),/child of a declared volume/);
 });
 
 test('folder rebind rejects every live project container including one-shots',async t=>{

package/test/trusted-beta.test.mjs

@@ -222,3 +222,63 @@ test('workflow runs and attempt jobs are paginated before selecting required evi
     [jobsPath.replace('&page=1', '&page=2')]: { jobs: [job] },
   }));
 });
+
+test('GitHub errors identify the failed read without exposing credentials or signed URLs', async () => {
+  const missing = githubClient('private-token', async () => new Response('private-body', { status: 404 }));
+  await assert.rejects(missing('/repos/jdorado/ez-agents/git/ref/tags/v1.2.3-beta.1'), /GitHub GET .*\/git\/ref\/tags\/v1\.2\.3-beta\.1: release tag is missing or inaccessible/);
+  await assert.rejects(missing('/repos/jdorado/ez-agents/releases/123'), error => {
+    assert.match(error.message, /HTTP 404.*GitHub GET .*\/releases\/123/);
+    assert.doesNotMatch(error.message, /private-token|private-body|release tag/);
+    return true;
+  });
+  let request = 0;
+  const download = githubClient('private-token', async () => request++ === 0
+    ? new Response('', { status: 302, headers: { location: 'https://release-assets.githubusercontent.com/file?signature=private-signature' } })
+    : new Response('private-body', { status: 403 }));
+  await assert.rejects(download('/repos/jdorado/ez-agents/releases/assets/456', true), error => {
+    assert.match(error.message, /HTTP 403.*\/releases\/assets\/456 asset download/);
+    assert.doesNotMatch(error.message, /private-token|private-body|private-signature|signature=/);
+    return true;
+  });
+});
+
+test('staging recovery requires a real source tag and canonical asset before validating exact bytes', async t => {
+  const { validate } = await import('../scripts/trusted-beta.mjs');
+  const { readFile } = await import('node:fs/promises');
+  const dir = await mkdtemp(join(tmpdir(), 'beta-staging-recovery-'));
+  t.after(() => rm(dir, { recursive: true, force: true }));
+  await mkdir(join(dir, 'package'));
+  await writeFile(join(dir, 'package/package.json'), JSON.stringify(manifest));
+  execFileSync('tar', ['-czf', join(dir, 'candidate.tgz'), '-C', dir, 'package/package.json']);
+  const bytes = await readFile(join(dir, 'candidate.tgz'));
+  const candidateEnv = { ...env, RELEASE_SHA256: sha256(bytes) };
+  let tagExists = false, assetName = 'jc_stack-ez-agents-1.2.3-beta.1.tgz';
+  const reads = [];
+  const source = sourceApi();
+  const api = async path => {
+    reads.push(path);
+    if (path.includes('/git/ref/tags/') && !tagExists) {
+      return githubClient('test', async () => new Response('', { status: 404 }))(path);
+    }
+    if (path.endsWith('/releases/123')) return { id: 123, draft: true, prerelease: true, tag_name: `v${expected.version}`, assets: [
+      { id: 456, name: assetName, state: 'uploaded' }, { id: 457, name: 'release-receipt.json', state: 'uploaded' },
+    ] };
+    if (path.endsWith('/releases/assets/456')) return bytes;
+    if (path.endsWith('/releases/assets/457')) return Buffer.from(JSON.stringify({ ...receipt, sha256: candidateEnv.RELEASE_SHA256 }));
+    if (path.endsWith('/pulls/41')) return { merged: true, base: { repo: { full_name: expected.repository }, ref: 'main' }, merge_commit_sha: expected.sourceSha };
+    return source(path);
+  };
+  const output = join(dir, 'validated');
+  await assert.rejects(validate(output, candidateEnv, api), /release tag is missing/);
+  assert.ok(!reads.some(path => path.includes('/releases/')));
+  tagExists = true;
+  await assert.rejects(validate(output, candidateEnv, api), /Missing or ambiguous asset: candidate.tgz/);
+  assert.ok(!reads.some(path => path.includes('/releases/assets/')));
+  assetName = 'candidate.tgz';
+  await assert.rejects(validate(output, { ...candidateEnv, RELEASE_SHA256: '0'.repeat(64) }, api), /SHA256 mismatch/);
+  const result = await validate(output, candidateEnv, api);
+  assert.equal(result.sha256, sha256(bytes));
+  assert.deepEqual(await readFile(join(output, 'candidate.tgz')), bytes);
+  // Retry cannot silently replace an existing validated bundle.
+  await assert.rejects(validate(output, candidateEnv, api), /EEXIST/);
+});

package/test/updates.test.mjs

@@ -7,7 +7,7 @@ import { gzipSync } from 'node:zlib';
 import { spawn, execFile } from 'node:child_process';
 import { promisify } from 'node:util';
 import { extract, digest, version, newer, compatible } from '../src/updates/artifact.mjs';
-import { prepare, submit, command, read, jobPath, eligibility } from '../src/updates/control.mjs';
+import { prepare, submit, command, read, jobPath, eligibility, jobs } from '../src/updates/control.mjs';
 import { perform, environment, packageManager } from '../src/updates/runtime.mjs';
 import { atomic, snapshot, compose } from '../src/plugins/manager.mjs';
 import { bindUpdates } from '../src/updates/binding.mjs';
@@ -44,7 +44,7 @@ async function fixture(t,kind='main') {
  if(kind==='plugin') {
   const s=await snapshot(old),base=path.join(home,'packages',target);await fs.mkdir(base,{recursive:true});
   record={revision:s.revision,source:old,project:`ezp-${digest(home).slice(0,16)}-${target}`,manifest:s.manifest,deployment:s.deployment,compose:path.join(base,'compose.json')};
-  await atomic(record.compose,compose(config,record));
+  await atomic(record.compose,await compose(config,record,{},home));
  }
  await atomic(path.join(home,'registry.json'),{schemaVersion:1,owner:home,plugins:record?{sample:record}:{},commands:record?{sample:'sample'}:{}});
  await fs.cp(old,source,{recursive:true});pkg.version='0.1.1';await atomic(path.join(source,'package.json'),pkg);
@@ -252,6 +252,11 @@ test('missing or broken managers fail with repair guidance before installing or
 
 for(const provider of ['pnpm','corepack']) test(`supervisor with only ${provider} drains work, replaces host PID and recovers after restart`,async t=>{
  const f=await fixture(t),fake=path.join(f.root,'fake');await fs.mkdir(fake);
+ // A process may die after creating a job directory but before atomically
+ // committing its receipt. That directory must not block transport startup.
+ await fs.mkdir(path.join(f.home,'updates','d18e847a-bb59-49c6-96f3-27fdc43ca44f'));
+ await fs.mkdir(path.join(f.home,'updates','a'.repeat(36)));
+ assert.deepEqual(await jobs(f.home),[]);
  const hostCode=`import fs from 'node:fs';import path from 'node:path';const c=JSON.parse(fs.readFileSync(process.argv[2])).agents[0];const d=path.join(c.controlDir,'host-executor');fs.mkdirSync(d,{recursive:true});const beat=()=>{fs.writeFileSync(path.join(d,'heartbeat.json'),JSON.stringify({pid:process.pid,at:Date.now()}));};beat();const timer=setInterval(()=>{try{process.kill(Number(process.env.EZ_HOST_SUPERVISOR_PID),0)}catch{process.exit(0)}beat()},100);process.on('SIGTERM',()=>{clearInterval(timer);process.exit(0)});`;
  for(const dir of [f.old,f.source]) {
   await fs.mkdir(path.join(dir,'node_modules/tsx/dist'),{recursive:true});await fs.writeFile(path.join(dir,'node_modules/tsx/dist/loader.mjs'),'');