@rubytech/create-realagent 1.0.867 → 1.0.868

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@rubytech/create-realagent",
-  "version": "1.0.867",
+  "version": "1.0.868",
   "description": "Install Real Agent — Built for agents. By agents.",
   "bin": {
     "create-realagent": "./dist/index.js"

package/payload/platform/plugins/admin/skills/unzip-attachment/SKILL.md

@@ -17,18 +17,31 @@ The `[ATTACHMENTS:]` block is contractually present on every turn that originall
 These are not guidelines. Every one of them is mechanically enforced by the shell primitives below; if any check fails, abort the flow, emit the named log line, and tell the operator verbatim what tripped it.
 
 - **No byte of the archive may land outside `{accountDir}/extracted/{attachmentId}/`.** The destination is fresh per upload. Extraction uses `unzip -oq` into that directory and only that directory.
-- **Sum of declared uncompressed sizes must be ≤ `MAX_ZIP_UNCOMPRESSED_BYTES` (100 MiB, defined in [platform/ui/app/lib/attachments.ts](../../../../ui/app/lib/attachments.ts)).** Checked via `unzip -Zt` *before* any write.
-- **No entry may be a symlink.** Checked via `unzip -Z1v` output parsing before extraction; the post-extraction loop also runs `find <dest> -type l` as a ground-truth backstop.
-- **Every extracted file's `realpath --relative-to <dest>` must not start with `../`.** This is the canonical zip-slip guard — it catches malicious entry names like `../../etc/passwd` that some older `unzip` builds silently accept.
-- **Password-protected archives are refused with a fixed message.** Never prompt for a key.
+- **Sum of declared uncompressed sizes must be ≤ `MAX_ZIP_UNCOMPRESSED_BYTES` (100 MiB, defined in [platform/ui/app/lib/attachments.ts](../../../../ui/app/lib/attachments.ts)).** Checked by summing column 4 of `unzip -Z` output *before* any write.
+- **No entry may be a symlink.** Checked by parsing column 1 of `unzip -Z` output (permission column; leading `l` = symlink) before extraction; the post-extraction loop also runs `find <dest> -type l` as a ground-truth backstop.
+- **No entry name may start with `../`.** Checked by parsing the name column of `unzip -Z` output before extraction; the post-extraction `realpath --relative-to <dest>` loop is a defense-in-depth backstop for `unzip` versions that silently sanitise `../` during extraction.
+- **Password-protected archives are refused with a fixed message.** Detected by column 5 of `unzip -Z` output: a leading uppercase `T` or `B` (text/binary, encrypted) marks an encrypted entry. Never prompt for a key.
 
 See [references/safety.md](references/safety.md) for the precise shell commands, attack examples, and refusal templates.
 
 ## Flow
 
 1. **Resolve paths.** From the `[ATTACHMENTS:]` line read `attachmentId` and `storagePath`. Set `dest="${ACCOUNT_DIR}/extracted/${attachmentId}"`. Create it with `mkdir -p "$dest"`.
-2. **Pre-scan for password + symlink + byte total.** Run `unzip -Z -1 "$storagePath"` for the entry list; if it exits non-zero, the archive is corrupt or password-protected — refuse. Run `unzip -Z -v "$storagePath"` and scan the permission column for a leading `l` (symlink) — refuse on match. Run `unzip -Zt "$storagePath"` to obtain the summary line and parse the total uncompressed bytes; if > `100 * 1024 * 1024`, refuse.
-3. **Emit the start log line** — `[skill:unzip] start attachmentId=<uuid> dest=<path> declaredBytes=<n>` — exactly once, before the extraction command.
+2. **Single-pass pre-flight.** One `unzip -Z "$storagePath"` invocation produces the zipinfo column listing; pipe through an awk parser that checks all four invariants in one read of the output. Non-zero exit from `unzip -Z` → corrupt archive, refuse with `unreadable reason=corrupt`. Otherwise the awk gates (in order: encrypted, symlink, zip-slip, oversize) fire the matching refusal log line and exit before extraction. Inline this single bash block — do **not** split into two or more `unzip` calls:
+   ```sh
+   ZINFO=$(unzip -Z "$storagePath" 2>&1) || { echo "[skill:unzip] unreadable attachmentId=$attachmentId reason=corrupt"; exit 1; }
+   echo "$ZINFO" | awk -v aid="$attachmentId" -v lim=$((100*1024*1024)) '
+     NR>2 && $1 ~ /^[-lrwxd?]/ {
+       if ($5 ~ /^[TB]/)           { print "[skill:unzip] unreadable attachmentId=" aid " reason=password"; bad=1; exit }
+       if ($1 ~ /^l/)              { print "[skill:unzip] symlink-blocked attachmentId=" aid " entry=" $9; bad=1; exit }
+       for (i=9; i<=NF; i++) if ($i ~ /^\.\.\//) { print "[skill:unzip] zip-slip-blocked attachmentId=" aid " entry=" $i; bad=1; exit }
+       sum += $4
+     }
+     END { if (!bad && sum > lim) { print "[skill:unzip] oversize attachmentId=" aid " uncompressed=" sum " limit=" lim; bad=1 }; exit bad }
+   ' || exit 1
+   DECLARED_BYTES=$(echo "$ZINFO" | awk 'NR>2 && $1 ~ /^[-lrwxd?]/ { s += $4 } END { print s+0 }')
+   ```
+3. **Emit the start log line** — `[skill:unzip] start attachmentId=<uuid> dest=<path> declaredBytes=<n> preflightPasses=1` — exactly once, before the extraction command. The `preflightPasses=1` field is mandatory and always `1`; its absence in `server.log` after a clean upload means the skill was not invoked or step 2 was split into multiple passes (a regression).
 4. **Extract.** `unzip -oq "$storagePath" -d "$dest"`. Non-zero exit → refuse with the underlying `unzip` stderr quoted.
 5. **Post-extraction realpath check.** For every path emitted by `find "$dest" -mindepth 1 \( -type f -o -type l \)`:
    - If `-type l`, refuse with `symlink-blocked` (ground-truth guard — pre-scan caught most, this catches the rest).
@@ -45,7 +58,7 @@ On any refusal step the operator gets the refusal message verbatim, no re-try, n
 
 | When | Line |
 |------|------|
-| Before extraction | `[skill:unzip] start attachmentId=<uuid> dest=<path> declaredBytes=<n>` |
+| Before extraction | `[skill:unzip] start attachmentId=<uuid> dest=<path> declaredBytes=<n> preflightPasses=1` |
 | On success | `[skill:unzip] done attachmentId=<uuid> entries=<n> uncompressed=<bytes>` |
 | Oversize refusal | `[skill:unzip] oversize attachmentId=<uuid> uncompressed=<bytes> limit=104857600` |
 | Zip-slip refusal | `[skill:unzip] zip-slip-blocked attachmentId=<uuid> entry=<path>` |

package/payload/platform/plugins/admin/skills/unzip-attachment/__tests__/preflight.sh

@@ -0,0 +1,148 @@
+#!/usr/bin/env bash
+# Regression suite for the single-pass pre-flight prescribed by SKILL.md step 2.
+# Each test builds a fixture zip, runs the exact awk one-liners from SKILL.md
+# against `unzip -Z` output, and asserts the expected gate fires.
+#
+# Run from anywhere: `bash platform/plugins/admin/skills/unzip-attachment/__tests__/preflight.sh`
+# Exits 0 if all assertions pass, non-zero on the first failure.
+
+set -uo pipefail
+
+WORK=$(mktemp -d)
+trap 'rm -rf "$WORK"' EXIT
+
+PASS=0
+FAIL=0
+fail() { echo "FAIL: $1"; FAIL=$((FAIL+1)); }
+pass() { echo "PASS: $1"; PASS=$((PASS+1)); }
+
+# --- The four awk gates from SKILL.md step 2 -----------------------------------
+# Each takes `unzip -Z $zip` on stdin and exits 0 if the gate FIRES (attack
+# detected), non-zero if the zip is clean for that gate.
+
+detect_symlink()   { awk 'NR>2 && $1 ~ /^l/ { found=1; print $0 } END { exit !found }'; }
+detect_encrypted() { awk 'NR>2 && $5 ~ /^[TB]/ { found=1; print $0 } END { exit !found }'; }
+detect_slip()      { awk 'NR>2 && $1 ~ /^[-lrwxd?]/ { for (i=9; i<=NF; i++) if ($i ~ /^\.\.\//) { found=1; print $i } } END { exit !found }'; }
+sum_uncompressed() { awk 'NR>2 && $1 ~ /^[-lrwxd?]/ { s += $4 } END { print s+0 }'; }
+extract_names()    { awk 'NR>2 && $1 ~ /^[-lrwxd?]/ { for (i=9; i<=NF; i++) printf "%s%s", $i, (i<NF?" ":"\n") }'; }
+
+LIMIT=$((100 * 1024 * 1024))
+
+# --- (a) Password-protected archive refused ----------------------------------
+cd "$WORK"
+echo "secret" > a-secret.txt
+zip -e -P testpass a-pw.zip a-secret.txt >/dev/null 2>&1
+if unzip -Z a-pw.zip 2>/dev/null | detect_encrypted >/dev/null; then
+  pass "(a) password — col-5 uppercase first letter detected"
+else
+  fail "(a) password — col-5 uppercase first letter NOT detected"
+fi
+
+# --- (b) Symlink entry refused -----------------------------------------------
+cd "$WORK"
+ln -s /etc/passwd b-link
+echo "ok" > b-ok.txt
+zip -y b-sym.zip b-link b-ok.txt >/dev/null 2>&1
+if unzip -Z b-sym.zip 2>/dev/null | detect_symlink >/dev/null; then
+  pass "(b) symlink — col-1 leading 'l' detected"
+else
+  fail "(b) symlink — col-1 leading 'l' NOT detected"
+fi
+
+# --- (c) Zip-slip entry refused by pre-scan ---------------------------------
+# The PRE-scan must detect `../`-prefixed entry names in `unzip -Z` column 9+
+# BEFORE extraction runs. macOS unzip silently sanitises these paths during
+# extraction (the file lands inside dest with `../` stripped), so the
+# post-extraction realpath check is a defense-in-depth backstop for unzip
+# versions that don't sanitise — not the primary gate.
+# zip(1) refuses `..` paths, so python forges the central-directory entry.
+cd "$WORK"
+python3 - <<'PY' >/dev/null 2>&1
+import zipfile
+with zipfile.ZipFile("c-slip.zip", "w") as z:
+    z.writestr("../../escapee.txt", "evil payload")
+    z.writestr("good.txt", "ok")
+PY
+if unzip -Z c-slip.zip 2>/dev/null | detect_slip >/dev/null; then
+  pass "(c) zip-slip — pre-scan caught '../' in entry name"
+else
+  fail "(c) zip-slip — pre-scan missed '../' in entry name"
+fi
+
+# --- (d) Oversize archive refused (declared > 100 MiB) -----------------------
+# Building a 100 MiB+ fixture is expensive on disk and CI. Instead, simulate
+# the awk gate against a synthetic `unzip -Z`-shaped fixture whose column-4
+# sum exceeds the limit. This tests the *gate logic*, not unzip itself.
+cd "$WORK"
+cat > d-zfake.txt <<'FAKE'
+Archive:  d-big.zip
+Zip file size: 999 bytes, number of entries: 2
+-rw-r--r--  3.0 unx 60000000 tx stor 26-May-10 21:08 big1.bin
+-rw-r--r--  3.0 unx 60000000 tx stor 26-May-10 21:08 big2.bin
+2 files, 120000000 bytes uncompressed, 999 bytes compressed:  0.0%
+FAKE
+SUM=$(sum_uncompressed < d-zfake.txt)
+if [ "$SUM" -gt "$LIMIT" ]; then
+  pass "(d) oversize — sum=$SUM > limit=$LIMIT detected"
+else
+  fail "(d) oversize — sum=$SUM did not exceed limit=$LIMIT (gate broken)"
+fi
+
+# --- (e) Clean zip = exactly 1 pre-flight + 1 extraction ---------------------
+cd "$WORK"
+mkdir e-stage && cd e-stage
+echo "alpha" > a.txt
+mkdir sub && echo "beta" > sub/b.txt
+zip -r ../e-clean.zip a.txt sub >/dev/null 2>&1
+cd "$WORK"
+
+# Wrap unzip in a tracer that increments a counter file per invocation, scoped
+# by the flag combination so we can count pre-flight vs extraction calls.
+TRACE_DIR=$(mktemp -d)
+mkdir -p "$TRACE_DIR/bin"
+cat > "$TRACE_DIR/bin/unzip" <<TRACE
+#!/usr/bin/env bash
+# Classify the call: -Z* = pre-flight (zipinfo mode), -o* = extraction.
+case "\$1" in
+  -Z*) echo "Z" >> "$TRACE_DIR/calls" ;;
+  -o*) echo "O" >> "$TRACE_DIR/calls" ;;
+  -t*) echo "T" >> "$TRACE_DIR/calls" ;;
+  *)   echo "?" >> "$TRACE_DIR/calls" ;;
+esac
+exec /usr/bin/unzip "\$@"
+TRACE
+chmod +x "$TRACE_DIR/bin/unzip"
+: > "$TRACE_DIR/calls"
+
+# Run the prescribed clean-path: 1 `unzip -Z` + 1 `unzip -oq`.
+mkdir e-dest
+PATH="$TRACE_DIR/bin:/usr/bin:/bin" unzip -Z e-clean.zip >/dev/null 2>&1
+PATH="$TRACE_DIR/bin:/usr/bin:/bin" unzip -oq e-clean.zip -d e-dest >/dev/null 2>&1
+
+Z_COUNT=$(grep -c '^Z$' "$TRACE_DIR/calls" || true)
+O_COUNT=$(grep -c '^O$' "$TRACE_DIR/calls" || true)
+T_COUNT=$(grep -c '^T$' "$TRACE_DIR/calls" || true)
+
+if [ "$Z_COUNT" = "1" ] && [ "$O_COUNT" = "1" ] && [ "$T_COUNT" = "0" ]; then
+  pass "(e) clean — exactly 1 pre-flight + 1 extraction (Z=$Z_COUNT O=$O_COUNT T=$T_COUNT)"
+else
+  fail "(e) clean — wrong invocation count Z=$Z_COUNT O=$O_COUNT T=$T_COUNT"
+fi
+
+# --- (f) Filenames with spaces survive name extraction -----------------------
+cd "$WORK"
+mkdir f-stage && cd f-stage
+echo "hello" > "hello world.txt"
+zip ../f-space.zip "hello world.txt" >/dev/null 2>&1
+cd "$WORK"
+NAMES=$(unzip -Z f-space.zip 2>/dev/null | extract_names)
+if [ "$NAMES" = "hello world.txt" ]; then
+  pass "(f) whitespace name — extracted intact: '$NAMES'"
+else
+  fail "(f) whitespace name — corrupted: got '$NAMES' want 'hello world.txt'"
+fi
+
+# --- Summary -----------------------------------------------------------------
+echo
+echo "RESULT: $PASS passed, $FAIL failed"
+exit "$FAIL"

package/payload/platform/plugins/admin/skills/unzip-attachment/references/safety.md

@@ -1,6 +1,20 @@
 # Zip extraction safety reference
 
-This file is the authoritative source for the exact shell commands, attack examples, and refusal messages referenced by [SKILL.md](../SKILL.md). Changes here propagate to every extraction — the skill file inlines none of this so the reference is the single point of truth.
+This file is the authoritative source for the exact shell commands, attack examples, and refusal messages referenced by [SKILL.md](../SKILL.md). Changes here propagate to every extraction — the skill file inlines the pre-flight bash block; this reference explains *why* each gate is shaped the way it is.
+
+## The single-pass pre-flight
+
+One `unzip -Z "$zip"` call produces the zipinfo column listing. Every attack-class gate reads the same output:
+
+```
+Archive:  example.zip
+Zip file size: 317 bytes, number of entries: 2
+-rw-r--r--  3.0 unx        6 tx stor 26-May-10 21:08 foo.txt
+lrwxr-xr-x  3.0 unx       11 bx stor 26-May-10 21:08 link
+2 files, 17 bytes uncompressed, 17 bytes compressed:  0.0%
+```
+
+Columns: `$1=permissions $2=zip-version $3=os $4=size-bytes $5=text/binary+crypto-flag $6=method $7=date $8=time $9+=name`. Each gate keys off one column; no second `unzip` invocation is needed.
 
 ## Attack classes
 
@@ -8,13 +22,20 @@ This file is the authoritative source for the exact shell commands, attack examp
 
 A malicious archive contains an entry whose name starts with `../` (or uses Windows `..\` on some unzip builds). A naive extractor writing relative paths without validation lands the file *outside* the intended destination.
 
-Example listing (`unzip -Z1`):
+Example listing (`unzip -Z`):
 ```
-good/readme.md
-../../etc/cron.d/malicious
+?rw-------  2.0 unx        4 b- stor 26-May-11 06:31 ../../escapee.txt
+?rw-------  2.0 unx        2 b- stor 26-May-11 06:31 good.txt
 ```
 
-Ground-truth guard: after extraction, for every path under `$dest`, compute `realpath --relative-to "$dest" "<entry>"`. If the relative path begins with `../`, the entry escaped. Refusal line: `[skill:unzip] zip-slip-blocked attachmentId=<uuid> entry=<path>`.
+Pre-scan detection (one awk pass over column 9+):
+```awk
+NR>2 && $1 ~ /^[-lrwxd?]/ { for (i=9; i<=NF; i++) if ($i ~ /^\.\.\//) { print "blocked: " $i; exit } }
+```
+
+Ground-truth backstop: after extraction, for every path under `$dest`, compute `realpath --relative-to "$dest" "<entry>"`. If the relative path begins with `../`, the entry escaped. On macOS InfoZIP 6.00 the extractor silently sanitises `../` prefixes (the file lands inside `$dest` with the prefix stripped), so the realpath check is a no-op in that case — but it still catches the attack on `unzip` builds that don't sanitise. Defense in depth: pre-scan refuses the obvious case, realpath catches the version-specific escape.
+
+Refusal line: `[skill:unzip] zip-slip-blocked attachmentId=<uuid> entry=<path>`.
 
 Refusal message to the operator:
 
@@ -24,9 +45,9 @@ Refusal message to the operator:
 
 A malicious archive contains a symlink entry — for example, a zero-byte file with permission `lrwxrwxrwx` pointing at `/etc/passwd`. If the extractor materialises it, subsequent operations that follow the symlink read or overwrite the target.
 
-Pre-scan detection:
-```sh
-unzip -Z -v "$zip" | awk 'NR>3 && $1 ~ /^l/ { print $NF; found=1 } END { exit !found }'
+Pre-scan detection (same `unzip -Z` output, column 1):
+```awk
+NR>2 && $1 ~ /^l/ { print "blocked: " $9; exit }
 ```
 First field beginning with `l` is the POSIX symlink flag in the permission column.
 
@@ -42,12 +63,11 @@ Refusal line: `[skill:unzip] symlink-blocked attachmentId=<uuid> entry=<path>`.
 
 A 42 KB archive expands to 4.5 GB — the classic `42.zip`. Not filesystem-escape but resource exhaustion: fills the account disk, causes OOM on downstream processing.
 
-Pre-scan gate:
-```sh
-unzip -Zt "$zip"
-# → "N files, X bytes uncompressed, Y bytes compressed: Z%"
+Pre-scan gate (sum column 4 over the same `unzip -Z` output):
+```awk
+NR>2 && $1 ~ /^[-lrwxd?]/ { s += $4 } END { if (s > 100*1024*1024) print "oversize: " s }
 ```
-Parse the `X bytes uncompressed` field. If `X > 100 * 1024 * 1024` (the `MAX_ZIP_UNCOMPRESSED_BYTES` export from `attachments.ts`), refuse *before* calling `unzip -oq`.
+Column 4 is the uncompressed size in bytes per entry. If the running sum exceeds `MAX_ZIP_UNCOMPRESSED_BYTES` (100 MiB, defined in `attachments.ts`), refuse *before* calling `unzip -oq`.
 
 Refusal line: `[skill:unzip] oversize attachmentId=<uuid> uncompressed=<bytes> limit=104857600`.
 
@@ -55,7 +75,12 @@ Note: a hostile archive can lie in its declared sizes. The post-extraction realp
 
 ### 4. Password-protected archive
 
-Refused without prompting. `unzip -Z -1` exits non-zero for encrypted entries (or the `-v` listing shows `E` in the attribute column).
+Refused without prompting. The signal is **not** the exit code — `unzip -Z` exits 0 on password-protected archives because the central directory itself is unencrypted; only entry payloads are. The signal is column 5: a leading uppercase `T` (encrypted text) or `B` (encrypted binary). Lowercase `t`/`b` = unencrypted.
+
+Pre-scan detection:
+```awk
+NR>2 && $5 ~ /^[TB]/ { print "password"; exit }
+```
 
 Refusal line: `[skill:unzip] unreadable attachmentId=<uuid> reason=password`.
 
@@ -63,19 +88,29 @@ Refusal message:
 
 > This archive is password-protected. Please extract it locally and upload the individual files you want me to look at.
 
+A genuinely corrupt zip (no central directory) makes `unzip -Z` exit non-zero (typically 9). That is the only exit-code-driven branch: non-zero → `[skill:unzip] unreadable attachmentId=<uuid> reason=corrupt`.
+
 ## Canonical commands
 
 | Purpose | Command |
 |---------|---------|
-| Entry list (names only, exits non-zero on password/corrupt) | `unzip -Z -1 "$zip"` |
-| Verbose listing with permission + size columns | `unzip -Z -v "$zip"` |
-| Summary totals parsed for byte-sum gate | `unzip -Zt "$zip"` |
+| Single-pass pre-flight (listing + permissions + sizes + crypto flag) | `unzip -Z "$zip"` |
 | Extract | `unzip -oq "$zip" -d "$dest"` |
 | Post-extraction symlink backstop | `find "$dest" -mindepth 1 -type l` |
-| Per-entry zip-slip check | `realpath --relative-to "$dest" "<entry>"` → must not start with `../` |
+| Per-entry zip-slip backstop | `realpath --relative-to "$dest" "<entry>"` → must not start with `../` |
 
 All of these are from `unzip` (InfoZIP) + `coreutils` — installed on every Pi image by the installer. No additional dependency.
 
+## Why one pass
+
+Each `Bash` tool_use is one agent turn. `effortToMaxTurns("low") = 5` ([platform/ui/app/lib/claude-agent/budget.ts:175](../../../../ui/app/lib/claude-agent/budget.ts#L175)). The previous flow used three separate `unzip -Z` invocations (entry list, verbose-for-symlink, totals) plus the extraction — four bash calls inside the skill alone, before counting the surrounding specialist dispatch and `plugin-read` calls. At `effort=low` the floor was six turns; one upload exhausted the budget and tripped `error_max_turns`. The fix is structural: every signal the four gates need is already in `unzip -Z`'s column output, so the three pre-flight calls collapse to one.
+
+The observability anchor is `preflightPasses=1` on the `[skill:unzip] start` log line. Absence after a clean upload means either the skill did not activate or the flow regressed back to multiple passes — both regressions are caught by `grep '[skill:unzip] start' server.log | grep -v preflightPasses=1`.
+
 ## Why this is a skill, not a tool
 
 Doctrine: the admin agent has `Bash`; the security-critical code path is a sequence of shell commands whose determinism is enforced by the shell primitives themselves, not by LLM reasoning. Wrapping this in an MCP tool would add a translation layer without adding enforcement — a tool wrapper is still LLM-mediated at the decision boundary. The shell script is the deterministic primitive; the skill tells the agent which primitive to invoke, in which order, against which argument.
+
+## Regression suite
+
+[`__tests__/preflight.sh`](../__tests__/preflight.sh) is the executable specification of the four pre-flight gates. Run it after any change to this file or `SKILL.md`. It builds fixtures for each attack class, pipes `unzip -Z` output through the same awk gates inlined in `SKILL.md` step 2, and asserts each gate fires (or stays silent) on the right input. The clean-zip case also traces `unzip` invocations to assert exactly 1 pre-flight + 1 extraction — the structural invariant `preflightPasses=1` is built on.