instar 1.3.484 → 1.3.486
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dashboard/index.html +4 -1
- package/dist/core/MessagingToneGate.d.ts +23 -2
- package/dist/core/MessagingToneGate.d.ts.map +1 -1
- package/dist/core/MessagingToneGate.js +15 -1
- package/dist/core/MessagingToneGate.js.map +1 -1
- package/dist/core/PostUpdateMigrator.d.ts.map +1 -1
- package/dist/core/PostUpdateMigrator.js +21 -0
- package/dist/core/PostUpdateMigrator.js.map +1 -1
- package/dist/core/SessionManager.d.ts.map +1 -1
- package/dist/core/SessionManager.js +22 -0
- package/dist/core/SessionManager.js.map +1 -1
- package/dist/core/TelegramRelay.d.ts +1 -0
- package/dist/core/TelegramRelay.d.ts.map +1 -1
- package/dist/core/TelegramRelay.js +9 -1
- package/dist/core/TelegramRelay.js.map +1 -1
- package/dist/core/raw-file-path.d.ts +33 -0
- package/dist/core/raw-file-path.d.ts.map +1 -0
- package/dist/core/raw-file-path.js +105 -0
- package/dist/core/raw-file-path.js.map +1 -0
- package/dist/messaging/OutboundAdvisory.d.ts +152 -0
- package/dist/messaging/OutboundAdvisory.d.ts.map +1 -0
- package/dist/messaging/OutboundAdvisory.js +453 -0
- package/dist/messaging/OutboundAdvisory.js.map +1 -0
- package/dist/messaging/TelegramAdapter.d.ts +6 -0
- package/dist/messaging/TelegramAdapter.d.ts.map +1 -1
- package/dist/messaging/TelegramAdapter.js +4 -1
- package/dist/messaging/TelegramAdapter.js.map +1 -1
- package/dist/messaging/pending-relay-store.d.ts +7 -0
- package/dist/messaging/pending-relay-store.d.ts.map +1 -1
- package/dist/messaging/pending-relay-store.js +9 -3
- package/dist/messaging/pending-relay-store.js.map +1 -1
- package/dist/monitoring/delivery-failure-sentinel.d.ts +6 -1
- package/dist/monitoring/delivery-failure-sentinel.d.ts.map +1 -1
- package/dist/monitoring/delivery-failure-sentinel.js +15 -3
- package/dist/monitoring/delivery-failure-sentinel.js.map +1 -1
- package/dist/scaffold/templates.d.ts.map +1 -1
- package/dist/scaffold/templates.js +7 -1
- package/dist/scaffold/templates.js.map +1 -1
- package/dist/scheduler/JobScheduler.d.ts.map +1 -1
- package/dist/scheduler/JobScheduler.js +8 -0
- package/dist/scheduler/JobScheduler.js.map +1 -1
- package/dist/server/AgentServer.d.ts +1 -0
- package/dist/server/AgentServer.d.ts.map +1 -1
- package/dist/server/AgentServer.js +22 -0
- package/dist/server/AgentServer.js.map +1 -1
- package/dist/server/PeerStreamProxy.d.ts +7 -0
- package/dist/server/PeerStreamProxy.d.ts.map +1 -1
- package/dist/server/PeerStreamProxy.js +11 -2
- package/dist/server/PeerStreamProxy.js.map +1 -1
- package/dist/server/WebSocketManager.d.ts +10 -1
- package/dist/server/WebSocketManager.d.ts.map +1 -1
- package/dist/server/WebSocketManager.js +27 -3
- package/dist/server/WebSocketManager.js.map +1 -1
- package/dist/server/routes.d.ts +12 -0
- package/dist/server/routes.d.ts.map +1 -1
- package/dist/server/routes.js +216 -3
- package/dist/server/routes.js.map +1 -1
- package/package.json +1 -1
- package/src/data/builtin-manifest.json +67 -67
- package/src/scaffold/templates.ts +7 -1
- package/src/templates/scripts/telegram-reply.sh +202 -10
- package/upgrades/1.3.485.md +31 -0
- package/upgrades/1.3.486.md +38 -0
- package/upgrades/side-effects/outbound-advisory-inform-only.md +252 -0
- package/upgrades/side-effects/pool-stream-cross-machine-fix.md +94 -0
|
@@ -18,6 +18,28 @@
|
|
|
18
18
|
# --stdin-base64 Decode stdin/argument text from base64 before sending.
|
|
19
19
|
# Use this for content that may contain shell syntax or
|
|
20
20
|
# heredoc delimiters such as a literal EOF line.
|
|
21
|
+
# --ack-advisory Acknowledge a preflight advisory and send the message
|
|
22
|
+
# unchanged. The preflight still runs (so the override is
|
|
23
|
+
# audited) but never withholds. FLAG ONLY — there is
|
|
24
|
+
# deliberately no env form (a standing env export would be
|
|
25
|
+
# a blanket pre-ack that silently disables the inform
|
|
26
|
+
# layer; spec outbound-jargon-filepath-gap §2.4(4)).
|
|
27
|
+
#
|
|
28
|
+
# Outbound advisory preflight (inform-only — spec outbound-jargon-filepath-gap §2.4):
|
|
29
|
+
# When this send comes from an automated LLM job session (the scheduler
|
|
30
|
+
# stamps INSTAR_MESSAGE_KIND=automated + INSTAR_SENDER_CLASS=llm-session
|
|
31
|
+
# into the session env — the model types nothing), the script first asks
|
|
32
|
+
# the server's deterministic detectors about the text. If they flag
|
|
33
|
+
# something (raw file path, dev jargon, localhost link), the message is
|
|
34
|
+
# NOT sent yet: the advisory prints to stdout (so the agent reads it in
|
|
35
|
+
# its transcript) and the script exits 0. The agent then either fixes the
|
|
36
|
+
# text and re-runs, or re-runs with --ack-advisory to send unchanged.
|
|
37
|
+
# The advisory layer NEVER blocks: the ack path always delivers past it,
|
|
38
|
+
# and every error path (server down, timeout, bad JSON) proceeds straight
|
|
39
|
+
# to the send as if the preflight returned nothing. Script-class senders
|
|
40
|
+
# (INSTAR_SENDER_CLASS=script) skip the preflight — there is no agent to
|
|
41
|
+
# inform. Conversational sessions have none of these env vars and are
|
|
42
|
+
# completely unaffected.
|
|
21
43
|
#
|
|
22
44
|
# Port resolution (in order):
|
|
23
45
|
# 1. INSTAR_PORT environment variable (explicit operator override).
|
|
@@ -33,6 +55,7 @@
|
|
|
33
55
|
|
|
34
56
|
FORMAT=""
|
|
35
57
|
STDIN_BASE64=0
|
|
58
|
+
ACK_ADVISORY=0
|
|
36
59
|
|
|
37
60
|
# Parse leading flags before positional args.
|
|
38
61
|
while [ $# -gt 0 ]; do
|
|
@@ -49,6 +72,10 @@ while [ $# -gt 0 ]; do
|
|
|
49
72
|
STDIN_BASE64=1
|
|
50
73
|
shift
|
|
51
74
|
;;
|
|
75
|
+
--ack-advisory)
|
|
76
|
+
ACK_ADVISORY=1
|
|
77
|
+
shift
|
|
78
|
+
;;
|
|
52
79
|
--)
|
|
53
80
|
shift
|
|
54
81
|
break
|
|
@@ -117,11 +144,14 @@ v = c.get('authToken', '')
|
|
|
117
144
|
print(v if isinstance(v, str) else '')
|
|
118
145
|
print(c.get('projectName', ''))
|
|
119
146
|
print(c.get('port', ''))
|
|
147
|
+
t = (((c.get('messaging') or {}).get('outboundAdvisory') or {}).get('timeoutMs', ''))
|
|
148
|
+
print(t if isinstance(t, (int, float)) else '')
|
|
120
149
|
" 2>/dev/null)
|
|
121
150
|
CONFIG_AUTH=$(printf '%s\n' "$CONFIG_VALUES" | sed -n '1p')
|
|
122
151
|
[ -z "$AUTH_TOKEN" ] && AUTH_TOKEN="$CONFIG_AUTH"
|
|
123
152
|
AGENT_ID=$(printf '%s\n' "$CONFIG_VALUES" | sed -n '2p')
|
|
124
153
|
CONFIG_PORT=$(printf '%s\n' "$CONFIG_VALUES" | sed -n '3p')
|
|
154
|
+
CONFIG_ADVISORY_TIMEOUT_MS=$(printf '%s\n' "$CONFIG_VALUES" | sed -n '4p')
|
|
125
155
|
fi
|
|
126
156
|
|
|
127
157
|
if [ -n "$INSTAR_PORT" ]; then
|
|
@@ -133,21 +163,162 @@ else
|
|
|
133
163
|
echo "WARN: telegram-reply.sh — no INSTAR_PORT env and no port in .instar/config.json; falling back to 4040" >&2
|
|
134
164
|
fi
|
|
135
165
|
|
|
136
|
-
#
|
|
137
|
-
|
|
166
|
+
# ── Outbound advisory preflight (inform-only; spec outbound-jargon-filepath-gap §2.4) ──
|
|
167
|
+
# Validate the scheduler-stamped env values against the literal enums BEFORE
|
|
168
|
+
# use (an unexpected value forwards nothing — the server additionally coerces
|
|
169
|
+
# unknowns server-side).
|
|
170
|
+
MESSAGE_KIND=""
|
|
171
|
+
case "${INSTAR_MESSAGE_KIND:-}" in
|
|
172
|
+
reply|health-alert|unknown|automated) MESSAGE_KIND="$INSTAR_MESSAGE_KIND" ;;
|
|
173
|
+
esac
|
|
174
|
+
SENDER_CLASS=""
|
|
175
|
+
case "${INSTAR_SENDER_CLASS:-}" in
|
|
176
|
+
script|llm-session) SENDER_CLASS="$INSTAR_SENDER_CLASS" ;;
|
|
177
|
+
esac
|
|
178
|
+
# Job slug rides the metadata for server-side audit keying; charset-clamped
|
|
179
|
+
# here so it can never carry quotes/injection into a JSON or SQL context.
|
|
180
|
+
JOB_SLUG=$(printf '%s' "${INSTAR_JOB_SLUG:-}" | tr -c 'A-Za-z0-9._-' '_' | head -c 128)
|
|
181
|
+
|
|
182
|
+
ADVISORY_CODES_CSV=""
|
|
183
|
+
if [ "$MESSAGE_KIND" = "automated" ] && [ "$SENDER_CLASS" = "llm-session" ]; then
|
|
184
|
+
# Timeout: config messaging.outboundAdvisory.timeoutMs (default 2000ms),
|
|
185
|
+
# converted ms→SECONDS for curl --max-time with ceil division, clamped to
|
|
186
|
+
# [1, 10]. A raw `--max-time 2000` would be a ~33-minute fail-HANG and
|
|
187
|
+
# `$((MS/1000))` on values <1000 would yield 0 = no timeout — both invert
|
|
188
|
+
# the fail-open contract.
|
|
189
|
+
ADV_MS="${CONFIG_ADVISORY_TIMEOUT_MS:-2000}"
|
|
190
|
+
case "$ADV_MS" in (*[!0-9]*|'') ADV_MS=2000 ;; esac
|
|
191
|
+
ADV_SECS=$(( (ADV_MS + 999) / 1000 ))
|
|
192
|
+
[ "$ADV_SECS" -lt 1 ] && ADV_SECS=1
|
|
193
|
+
[ "$ADV_SECS" -gt 10 ] && ADV_SECS=10
|
|
194
|
+
|
|
195
|
+
# Preflight body via python3 (already a hard dependency of this script). If
|
|
196
|
+
# python3 is unavailable the preflight is skipped entirely — fail-open.
|
|
197
|
+
PREFLIGHT_BODY=$(PF_TOPIC="$TOPIC_ID" PF_SLUG="$JOB_SLUG" PF_KIND="$MESSAGE_KIND" python3 -c '
|
|
198
|
+
import sys, json, os
|
|
199
|
+
print(json.dumps({
|
|
200
|
+
"text": sys.stdin.read(),
|
|
201
|
+
"messageKind": os.environ.get("PF_KIND", "automated"),
|
|
202
|
+
"topicId": int(os.environ.get("PF_TOPIC") or 0),
|
|
203
|
+
"jobSlug": os.environ.get("PF_SLUG", ""),
|
|
204
|
+
}))
|
|
205
|
+
' <<<"$MSG" 2>/dev/null)
|
|
206
|
+
|
|
207
|
+
if [ -n "$PREFLIGHT_BODY" ]; then
|
|
208
|
+
PREFLIGHT_CURL=(-s -X POST "http://localhost:${PORT}/messaging/preflight"
|
|
209
|
+
-H 'Content-Type: application/json'
|
|
210
|
+
--max-time "$ADV_SECS"
|
|
211
|
+
-d "$PREFLIGHT_BODY")
|
|
212
|
+
if [ -n "$AUTH_TOKEN" ]; then
|
|
213
|
+
PREFLIGHT_CURL+=(-H "Authorization: Bearer ${AUTH_TOKEN}")
|
|
214
|
+
fi
|
|
215
|
+
if [ -n "$AGENT_ID" ]; then
|
|
216
|
+
PREFLIGHT_CURL+=(-H "X-Instar-AgentId: ${AGENT_ID}")
|
|
217
|
+
fi
|
|
218
|
+
PREFLIGHT_RESP=$(curl "${PREFLIGHT_CURL[@]}" 2>/dev/null)
|
|
219
|
+
|
|
220
|
+
# Parse advisories — every failure mode (empty/timeout/malformed/disabled)
|
|
221
|
+
# yields an empty result and the send proceeds (fail-OPEN end-to-end).
|
|
222
|
+
ADVISORY_RENDERED=$(printf '%s' "$PREFLIGHT_RESP" | python3 -c '
|
|
138
223
|
import sys, json
|
|
224
|
+
try:
|
|
225
|
+
resp = json.load(sys.stdin)
|
|
226
|
+
advisories = resp.get("advisories") or []
|
|
227
|
+
except Exception:
|
|
228
|
+
advisories = []
|
|
229
|
+
codes = ",".join(a.get("code", "") for a in advisories if isinstance(a, dict) and a.get("code"))
|
|
230
|
+
print(codes)
|
|
231
|
+
for a in advisories:
|
|
232
|
+
if not isinstance(a, dict):
|
|
233
|
+
continue
|
|
234
|
+
print("- " + str(a.get("code", "")))
|
|
235
|
+
m = a.get("match")
|
|
236
|
+
if m:
|
|
237
|
+
# Inert, delimited, quoted token under a fixed label — never spliced
|
|
238
|
+
# into instruction-shaped prose (injection-pinned rendering).
|
|
239
|
+
print(" detected: " + json.dumps(str(m)[:120]))
|
|
240
|
+
g = a.get("guidance")
|
|
241
|
+
if g:
|
|
242
|
+
print(" guidance: " + str(g))
|
|
243
|
+
' 2>/dev/null)
|
|
244
|
+
ADVISORY_CODES_CSV=$(printf '%s\n' "$ADVISORY_RENDERED" | sed -n '1p')
|
|
245
|
+
ADVISORY_DETAIL=$(printf '%s\n' "$ADVISORY_RENDERED" | sed '1d')
|
|
246
|
+
|
|
247
|
+
if [ -n "$ADVISORY_CODES_CSV" ] && [ "$ACK_ADVISORY" != "1" ]; then
|
|
248
|
+
# Inform the sender BEFORE the user sees anything. The FIRST line is
|
|
249
|
+
# machine-unmissable and literal (the E2E asserts this exact string).
|
|
250
|
+
# Exit 0 is deliberate: non-zero means delivery failure and triggers
|
|
251
|
+
# queue/retry semantics in callers; an advisory is neither — the message
|
|
252
|
+
# was deliberately not yet sent and the next move belongs to the agent.
|
|
253
|
+
echo "NOT SENT — advisory (fix and re-run, or re-run with --ack-advisory to send unchanged)"
|
|
254
|
+
echo ""
|
|
255
|
+
echo "The outbound advisory flagged this automated message BEFORE delivery:"
|
|
256
|
+
printf '%s\n' "$ADVISORY_DETAIL"
|
|
257
|
+
echo ""
|
|
258
|
+
echo "Next move (yours — the advisory layer never blocks):"
|
|
259
|
+
echo " 1. FIX: revise the message and re-run this script (preferred)."
|
|
260
|
+
echo " 2. SEND AS-IS: re-run with --ack-advisory to deliver unchanged (the override is audited)."
|
|
261
|
+
exit 0
|
|
262
|
+
fi
|
|
263
|
+
fi
|
|
264
|
+
fi
|
|
265
|
+
|
|
266
|
+
# ── Serialized kind metadata — computed ONCE, shared by both body builders
|
|
267
|
+
# AND both queue writers (a queued send must carry the metadata whole so the
|
|
268
|
+
# sentinel redrive doesn't mis-kind it / drop an ack — spec §2.5). Every
|
|
269
|
+
# component is enum-validated or charset-clamped above, so this fragment is
|
|
270
|
+
# safe to interpolate into JSON and (parameterized) SQL contexts.
|
|
271
|
+
METADATA_JSON=""
|
|
272
|
+
if [ -n "$MESSAGE_KIND" ] || [ -n "$SENDER_CLASS" ] || [ -n "$JOB_SLUG" ]; then
|
|
273
|
+
META_PARTS=""
|
|
274
|
+
[ -n "$MESSAGE_KIND" ] && META_PARTS="\"messageKind\":\"${MESSAGE_KIND}\""
|
|
275
|
+
if [ -n "$SENDER_CLASS" ]; then
|
|
276
|
+
[ -n "$META_PARTS" ] && META_PARTS="${META_PARTS},"
|
|
277
|
+
META_PARTS="${META_PARTS}\"senderClass\":\"${SENDER_CLASS}\""
|
|
278
|
+
fi
|
|
279
|
+
if [ -n "$JOB_SLUG" ]; then
|
|
280
|
+
[ -n "$META_PARTS" ] && META_PARTS="${META_PARTS},"
|
|
281
|
+
META_PARTS="${META_PARTS}\"jobSlug\":\"${JOB_SLUG}\""
|
|
282
|
+
fi
|
|
283
|
+
if [ "$ACK_ADVISORY" = "1" ] && [ "$MESSAGE_KIND" = "automated" ] && [ "$SENDER_CLASS" = "llm-session" ]; then
|
|
284
|
+
# REQUIRED annotation (§2.4(4)) — how the server audits "acked" as the
|
|
285
|
+
# single writer. Carries the overridden codes, including [] for a
|
|
286
|
+
# preemptive ack on a clean message (itself a signal).
|
|
287
|
+
ACK_CODES_JSON=$(printf '%s' "$ADVISORY_CODES_CSV" | tr -cd 'A-Z_,' | awk -F',' '{out=""; for(i=1;i<=NF;i++){if($i!=""){out=out (out==""?"":",") "\""$i"\""}} print "["out"]"}')
|
|
288
|
+
[ -z "$ACK_CODES_JSON" ] && ACK_CODES_JSON="[]"
|
|
289
|
+
[ -n "$META_PARTS" ] && META_PARTS="${META_PARTS},"
|
|
290
|
+
META_PARTS="${META_PARTS}\"advisoryAck\":true,\"advisoryCodes\":${ACK_CODES_JSON}"
|
|
291
|
+
fi
|
|
292
|
+
METADATA_JSON="{${META_PARTS}}"
|
|
293
|
+
fi
|
|
294
|
+
|
|
295
|
+
# Build JSON body (text + optional format + optional kind metadata). A
|
|
296
|
+
# conversational session (no kind env) produces the identical body it
|
|
297
|
+
# always did.
|
|
298
|
+
JSON_BODY=$(B_META="$METADATA_JSON" python3 -c '
|
|
299
|
+
import sys, json, os
|
|
139
300
|
msg = sys.argv[1]
|
|
140
301
|
fmt = sys.argv[2]
|
|
141
302
|
body = {"text": msg}
|
|
142
303
|
if fmt:
|
|
143
304
|
body["format"] = fmt
|
|
305
|
+
meta_raw = os.environ.get("B_META", "")
|
|
306
|
+
if meta_raw:
|
|
307
|
+
try:
|
|
308
|
+
body["metadata"] = json.loads(meta_raw)
|
|
309
|
+
except Exception:
|
|
310
|
+
pass
|
|
144
311
|
print(json.dumps(body))
|
|
145
312
|
' "$MSG" "$FORMAT" 2>/dev/null)
|
|
146
313
|
|
|
147
314
|
if [ -z "$JSON_BODY" ]; then
|
|
148
315
|
# Fallback if python3 not available: basic escape, no format override.
|
|
316
|
+
# BOTH builders must carry the kind fields, or a python-degraded agent
|
|
317
|
+
# silently drops them and the incident class recurs (spec §2.1).
|
|
149
318
|
ESCAPED=$(printf '%s' "$MSG" | sed 's/\\/\\\\/g; s/"/\\"/g; s/\n/\\n/g')
|
|
150
|
-
|
|
319
|
+
META_FIELD=""
|
|
320
|
+
[ -n "$METADATA_JSON" ] && META_FIELD=",\"metadata\":${METADATA_JSON}"
|
|
321
|
+
JSON_BODY="{\"text\":\"${ESCAPED}\"${META_FIELD}}"
|
|
151
322
|
fi
|
|
152
323
|
|
|
153
324
|
# Assemble curl args. Always include X-Instar-AgentId when we can resolve it
|
|
@@ -293,6 +464,7 @@ except Exception:
|
|
|
293
464
|
Q_PORT="$PORT" \
|
|
294
465
|
Q_ATTEMPTED_AT="$ATTEMPTED_AT" \
|
|
295
466
|
Q_TRUNCATED="$TRUNCATED" \
|
|
467
|
+
Q_METADATA="$METADATA_JSON" \
|
|
296
468
|
Q_DB_PATH="$QUEUE_DB" \
|
|
297
469
|
python3 -c '
|
|
298
470
|
import os, sqlite3, sys, json, datetime
|
|
@@ -324,7 +496,8 @@ try:
|
|
|
324
496
|
state TEXT NOT NULL,
|
|
325
497
|
claimed_by TEXT,
|
|
326
498
|
status_history TEXT NOT NULL DEFAULT "[]",
|
|
327
|
-
truncated INTEGER NOT NULL DEFAULT 0
|
|
499
|
+
truncated INTEGER NOT NULL DEFAULT 0,
|
|
500
|
+
message_metadata TEXT
|
|
328
501
|
)""")
|
|
329
502
|
# Idempotent column add for older schemas.
|
|
330
503
|
try:
|
|
@@ -332,6 +505,13 @@ try:
|
|
|
332
505
|
except sqlite3.OperationalError as e:
|
|
333
506
|
if "duplicate column name" not in str(e):
|
|
334
507
|
raise
|
|
508
|
+
# message_metadata (kind/senderClass/advisoryAck) — the redrive must carry
|
|
509
|
+
# the metadata whole (spec outbound-jargon-filepath-gap §2.5). Idempotent.
|
|
510
|
+
try:
|
|
511
|
+
conn.execute("ALTER TABLE entries ADD COLUMN message_metadata TEXT")
|
|
512
|
+
except sqlite3.OperationalError as e:
|
|
513
|
+
if "duplicate column name" not in str(e):
|
|
514
|
+
raise
|
|
335
515
|
conn.execute("CREATE INDEX IF NOT EXISTS idx_state_next ON entries(state, next_attempt_at)")
|
|
336
516
|
conn.execute("CREATE INDEX IF NOT EXISTS idx_text_hash_topic ON entries(text_hash, topic_id)")
|
|
337
517
|
# 5s dedup window (spec § 2b step 2).
|
|
@@ -355,8 +535,8 @@ try:
|
|
|
355
535
|
delivery_id, topic_id, text_hash, text, format,
|
|
356
536
|
http_code, error_body, attempted_port,
|
|
357
537
|
attempted_at, attempts, next_attempt_at,
|
|
358
|
-
state, claimed_by, status_history, truncated
|
|
359
|
-
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 1, NULL, "queued", NULL, ?, ?)""",
|
|
538
|
+
state, claimed_by, status_history, truncated, message_metadata
|
|
539
|
+
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 1, NULL, "queued", NULL, ?, ?, ?)""",
|
|
360
540
|
(
|
|
361
541
|
os.environ["Q_DELIVERY_ID"],
|
|
362
542
|
int(os.environ["Q_TOPIC_ID"]),
|
|
@@ -369,6 +549,7 @@ try:
|
|
|
369
549
|
os.environ["Q_ATTEMPTED_AT"],
|
|
370
550
|
initial_history,
|
|
371
551
|
int(os.environ.get("Q_TRUNCATED", "0")),
|
|
552
|
+
os.environ.get("Q_METADATA") or None,
|
|
372
553
|
),
|
|
373
554
|
)
|
|
374
555
|
conn.commit()
|
|
@@ -386,6 +567,15 @@ except Exception as exc:
|
|
|
386
567
|
# with IF NOT EXISTS so this is safe even if the Python path
|
|
387
568
|
# partially ran.
|
|
388
569
|
printf '%s' "$QUEUE_TEXT" > "${QUEUE_DB}.tmp.text"
|
|
570
|
+
# message_metadata rides via a temp file + readfile (same pattern as the
|
|
571
|
+
# text BLOB) so no shell value is ever interpolated into SQL — the
|
|
572
|
+
# metadata is enum-built, but the no-raw-interpolation rule holds anyway
|
|
573
|
+
# (spec outbound-jargon-filepath-gap §2.5).
|
|
574
|
+
META_SQL_VALUE="NULL"
|
|
575
|
+
if [ -n "$METADATA_JSON" ]; then
|
|
576
|
+
printf '%s' "$METADATA_JSON" > "${QUEUE_DB}.tmp.meta"
|
|
577
|
+
META_SQL_VALUE="CAST(readfile('${QUEUE_DB}.tmp.meta') AS TEXT)"
|
|
578
|
+
fi
|
|
389
579
|
sqlite3 "$QUEUE_DB" >/dev/null 2>&1 <<SQL
|
|
390
580
|
PRAGMA journal_mode=WAL;
|
|
391
581
|
PRAGMA synchronous=NORMAL;
|
|
@@ -405,22 +595,24 @@ CREATE TABLE IF NOT EXISTS entries (
|
|
|
405
595
|
state TEXT NOT NULL,
|
|
406
596
|
claimed_by TEXT,
|
|
407
597
|
status_history TEXT NOT NULL DEFAULT '[]',
|
|
408
|
-
truncated INTEGER NOT NULL DEFAULT 0
|
|
598
|
+
truncated INTEGER NOT NULL DEFAULT 0,
|
|
599
|
+
message_metadata TEXT
|
|
409
600
|
);
|
|
601
|
+
ALTER TABLE entries ADD COLUMN message_metadata TEXT;
|
|
410
602
|
CREATE INDEX IF NOT EXISTS idx_state_next ON entries(state, next_attempt_at);
|
|
411
603
|
CREATE INDEX IF NOT EXISTS idx_text_hash_topic ON entries(text_hash, topic_id);
|
|
412
604
|
INSERT OR IGNORE INTO entries (
|
|
413
605
|
delivery_id, topic_id, text_hash, text, format,
|
|
414
606
|
http_code, error_body, attempted_port, attempted_at,
|
|
415
|
-
attempts, state, status_history, truncated
|
|
607
|
+
attempts, state, status_history, truncated, message_metadata
|
|
416
608
|
) VALUES (
|
|
417
609
|
'$DELIVERY_ID', $TOPIC_ID, '$TEXT_HASH',
|
|
418
610
|
CAST(readfile('${QUEUE_DB}.tmp.text') AS BLOB), $( [ -n "$FORMAT" ] && printf "'%s'" "$FORMAT" || echo "NULL"),
|
|
419
611
|
$HTTP_CODE, NULL, $PORT, '$ATTEMPTED_AT',
|
|
420
|
-
1, 'queued', '[]', $TRUNCATED
|
|
612
|
+
1, 'queued', '[]', $TRUNCATED, $META_SQL_VALUE
|
|
421
613
|
);
|
|
422
614
|
SQL
|
|
423
|
-
rm -f "${QUEUE_DB}.tmp.text" 2>/dev/null
|
|
615
|
+
rm -f "${QUEUE_DB}.tmp.text" "${QUEUE_DB}.tmp.meta" 2>/dev/null
|
|
424
616
|
chmod 600 "$QUEUE_DB" 2>/dev/null
|
|
425
617
|
fi
|
|
426
618
|
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Upgrade Guide — vNEXT
|
|
2
|
+
|
|
3
|
+
<!-- assembled-by: assemble-next-md -->
|
|
4
|
+
<!-- bump: patch -->
|
|
5
|
+
|
|
6
|
+
## What Changed
|
|
7
|
+
|
|
8
|
+
Fixes the two live bugs from 2026-06-08 testing that left cross-machine dashboard streaming connecting-but-blank with no recovery (POOL-DASHBOARD-STREAM-SPEC, shipped #950–#970):
|
|
9
|
+
|
|
10
|
+
- **Closed-proxy eviction** (`WebSocketManager.peerProxyFor`): a peer's stream proxy that reached `closed` — after the bounded reconnect failed, or after the 60s idle grace once the last viewer left — stayed cached forever, and a closed proxy silently ignores every later subscribe while the server still answered `subscribed`. One hiccup (or one minute of nobody watching) made that machine permanently unstreamable until a server restart. The get-or-create chokepoint now evicts a closed proxy and opens a fresh episode with its own bounded reconnect budget (P19 no-storm guarantee preserved: reconnects remain one-per-episode, new episodes only on explicit user subscribes).
|
|
11
|
+
- **Cross-machine scrollback** (`history` relay): the terminal-history fetch only ever captured locally — structurally empty for a session owned by another machine. It now relays upstream for remote-subscribed sessions exactly like input/key (spec §2.2: capture happens ONLY on the owning machine), via a new read-only `PeerStreamProxy.relayFrame`; the dashboard client sends `machineId` on history requests for remote tiles.
|
|
12
|
+
- **Honest history miss**: the "no output for session" reply was a sessionless error frame, which the peer fan-out structurally drops; it now carries `session` + `code:'session-not-found'` and renders in the terminal (§2.4 failure honesty).
|
|
13
|
+
|
|
14
|
+
The serving side of the protocol is unchanged — an updated machine streaming from a not-yet-updated peer degrades to exactly today's behavior at worst. The remote-input default-off security gate is untouched.
|
|
15
|
+
|
|
16
|
+
## What to Tell Your User
|
|
17
|
+
|
|
18
|
+
- "Watching one machine's terminal from another machine's dashboard now actually shows the terminal — including scrolling back through its history — and if the link between machines hiccups, clicking the session again recovers it instead of staying dead until a restart."
|
|
19
|
+
|
|
20
|
+
## Summary of New Capabilities
|
|
21
|
+
|
|
22
|
+
| Capability | How to Use |
|
|
23
|
+
|-----------|-----------|
|
|
24
|
+
| Remote session scrollback | Automatic — scroll up in any remote session's terminal in the dashboard |
|
|
25
|
+
| Stream recovery after a dropped peer link | Automatic — re-clicking the session tile opens a fresh, bounded connection episode |
|
|
26
|
+
|
|
27
|
+
## Evidence
|
|
28
|
+
|
|
29
|
+
- 7 new unit tests (5 behavior + 2 regression guards) across `tests/unit/WebSocketManager.test.ts` and `tests/unit/PeerStreamProxy.test.ts`; each behavior test was run against the UNFIXED code first and failed for the stated reason (missing relayFrame, ignored re-subscribe after closed, sessionless error), then green with the fix.
|
|
30
|
+
- Full unit suite green in the worktree; `pnpm build` clean.
|
|
31
|
+
- Live verification on the real laptop + Mac Mini (a Mini session terminal rendering on the laptop dashboard, then recovery after an induced link drop) — performed post-deploy, results recorded in topic 13481.
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# Upgrade Guide — vNEXT
|
|
2
|
+
|
|
3
|
+
<!-- assembled-by: assemble-next-md -->
|
|
4
|
+
<!-- bump: minor -->
|
|
5
|
+
|
|
6
|
+
## What Changed
|
|
7
|
+
|
|
8
|
+
Implements `docs/specs/outbound-jargon-filepath-gap.md` (r2, converged + approved) — the durable Structure-over-Willpower fix for the 2026-06-10 incident where a background job sent the operator a reminder containing dev jargon and a raw repo path, bypassing both standing standards because nothing told the infrastructure the message was an automated alert. One PR, no deferrals:
|
|
9
|
+
|
|
10
|
+
- **Structural message kind (§2.1)** — the scheduler stamps `INSTAR_MESSAGE_KIND=automated` + `INSTAR_JOB_SLUG` + `INSTAR_SENDER_CLASS` into job session environments (BOTH `SessionManager.spawnSession` spawn lanes — headless AND rerouted-interactive — plus `JobScheduler.runScriptJob` with `senderClass=script`). The model types nothing; interactive sessions get none of these and stay `'reply'`. The `messageKind` union widens with `'automated'` (single-sourced as `MessageKind` in MessagingToneGate; all five prior sites now use it) + a `renderMessageKind` branch describing the kind to the authority. NO new block rule and NO B12 re-scope — the operator constraint is zero new blocking power.
|
|
11
|
+
- **Jargon signal single-sourced (§2.2)** — `evaluateOutbound` computes `detectJargon` itself for non-`reply` kinds (`health-alert | automated`), replacing the `options.jargon` caller opt-in (the one vestigial caller is cleaned up). Conversational replies deliberately get NO jargon signal (over-block scope). Live-config kill switch `messaging.outboundFloor.jargonAlways` (absence = on).
|
|
12
|
+
- **Raw-file-path SIGNAL (§2.3)** — new `detectRawFilePath` (`src/core/raw-file-path.ts`), a linear-regex sibling of `localhost-link.ts`: indexOf prescreen, bounded segments, never matches inside an `http(s)://` URL, match echo bounded to 120 chars and stopping before `?` (an adjacent secret can never ride along), fail-OPEN at every call site. Feeds `signals.filePath` to the existing authority on all kinds — anchoring the B2_FILE_PATH judgment it already has.
|
|
13
|
+
- **The preflight advisory (§2.4)** — `telegram-reply.sh` (template) gains an inform-only preflight, active ONLY for `automated` + `llm-session` senders: it calls the new `POST /messaging/preflight` (deterministic detectors only, no LLM; 64KB cap; runtime enum coercion at both boundaries) and on findings prints an advisory whose FIRST line is the literal `NOT SENT — advisory (fix and re-run, or re-run with --ack-advisory to send unchanged)` and exits 0 — the message is withheld until the SENDER resolves it (fix, or `--ack-advisory`, both always deliver past the advisory layer). Fail-OPEN end-to-end: route down / timeout / malformed JSON / disabled → the send proceeds untouched. The curl timeout converts config ms→seconds with ceil + a [1,10]s clamp (a raw `--max-time 2000` would be a ~33-minute fail-HANG). Flag-only ack (no env form — a standing pre-ack would silently disable the inform layer).
|
|
14
|
+
- **Server-side single-writer audit + escalation (§2.4(5–6))** — every preflight outcome is one JSONL line in `logs/outbound-advisory.jsonl` (`clean|advised|acked`; `acked` written only by `/telegram/reply` when a send carries the REQUIRED `metadata.advisoryAck` + overridden codes); size-rotated; read via the new `GET /messaging/advisory-log` (bounded tail, never the whole file). An in-memory write-time index drives the repeated-ignore escalation: N unresolved advised per signature (default 3) or per-slug aggregate across topics (default 5) raises ONE deduped HIGH Attention item with the FIXED `sourceContext: outbound-advisory-escalation` (P17 — the per-source topic budget genuinely binds); an interleaved clean heartbeat does NOT reset the count (reset-gaming closed); a near-identical clean re-send (the fix landing) resolves it; habitual `--ack-advisory` overriding raises the preemptive-ack consumer (NORMAL).
|
|
15
|
+
- **The kind survives every hop (§2.5)** — BOTH `telegram-reply.sh` body builders (python3 + sed fallback) forward `metadata.{messageKind,senderClass,jobSlug}`; BOTH script queue writers AND `PendingRelayStore` gain a nullable `message_metadata` column via the existing idempotent-ALTER conventions (a pre-column legacy DB upgrades on open — tested); the `DeliveryFailureSentinel` redrive forwards the metadata whole (an acked-then-queued send still lands its `acked` row — no escalation false-fire on transient delivery failure); the cross-machine relay hop (`sendToTopic → outboundRelay → relayOutbound`) carries it to the holder. Observability breadcrumbs (never gates): kindless send to a job-session topic (exempt for a redrive whose `X-Instar-DeliveryId` matches a REAL queue row), automated send with no recent preflight row (class-spoof visibility), declared `script` class on a non-script job.
|
|
16
|
+
- **Migration parity (§5)** — the current live `telegram-reply.sh` SHA (`3e30b2cd…`) joins `TELEGRAM_REPLY_PRIOR_SHIPPED_SHAS` so deployed stock scripts actually receive the new template; CLAUDE.md template + `migrateClaudeMd` gain the "Outbound advisory for automated messages" awareness section; the hand-curl `/telegram/reply` example is REPLACED by the relay-script form (no printed map to the bypass); the section is registered in `migrateFrameworkShadowCapabilities` markers[] (Codex/Gemini mirrors) and the feature-delivery-completeness guard.
|
|
17
|
+
|
|
18
|
+
## What to Tell Your User
|
|
19
|
+
|
|
20
|
+
- "My background jobs' messages now get a quality check before they reach you: if an automated reminder contains a raw file path, internal dev jargon, or a link that only works on my machine, the job is told to fix it BEFORE you ever see it — and the fix guidance is built in. Nothing can get stuck: the check informs the sender, it never blocks, and if a job keeps ignoring its own feedback you get one consolidated heads-up instead of silence."
|
|
21
|
+
- "Your normal conversations with me are completely unaffected — this only touches messages composed by scheduled background jobs."
|
|
22
|
+
|
|
23
|
+
## Summary of New Capabilities
|
|
24
|
+
|
|
25
|
+
| Capability | How to Use |
|
|
26
|
+
|-----------|-----------|
|
|
27
|
+
| Advisory preflight on automated sends | Automatic — scheduler-stamped; job models type nothing |
|
|
28
|
+
| `--ack-advisory` send-as-is override | Re-run `telegram-reply.sh --ack-advisory …` after an advisory (audited) |
|
|
29
|
+
| Advisory audit trail | `GET /messaging/advisory-log?limit=50` (Bearer-authed) |
|
|
30
|
+
| Repeated-ignore escalation | Automatic — one deduped Attention item per misbehaving job |
|
|
31
|
+
| Rollback levers (live, no restart) | `messaging.outboundAdvisory.enabled:false` · `messaging.outboundFloor.jargonAlways:false` |
|
|
32
|
+
|
|
33
|
+
## Evidence
|
|
34
|
+
|
|
35
|
+
- Founding incident reproduction shape covered end-to-end: the REAL shipped `telegram-reply.sh` run under bash with a recording curl shim proves (1) a raw-path automated send does NOT deliver on the first call, prints the LITERAL `NOT SENT — advisory…` first line, exits 0; (2) corrected text delivers; (3) `--ack-advisory` delivers the original and the send body carries `advisoryAck` + codes; (4) preflight network-failure AND malformed-JSON both DELIVER (fail-open proof); (5) a conversational session makes NO preflight call and produces the byte-identical legacy body (`tests/unit/telegram-reply-advisory-script.test.ts`, 12 tests).
|
|
36
|
+
- 26 detector tests incl. a 4KB-pathological-input bounded-time ReDoS test and the adjacent-secret echo bound (`raw-file-path.test.ts`); 17 audit/escalation tests incl. the reset-gaming case (interleaved clean does NOT reset) and the per-slug topic-varying aggregate (`outbound-advisory.test.ts`); 15 route tests incl. "the server DELIVERS a raw-path automated message exactly as today — never blocks" (`outbound-advisory-routes.test.ts`); queue pre-column-DB ALTER + sentinel metadata-forward tests (`pending-relay-metadata.test.ts`); relay-hop body test (`relay-kind-forward.test.ts`); spawn-lane env assertions (`session-manager-behavioral.test.ts`).
|
|
37
|
+
- Tier-3 alive: the REAL AgentServer on the production init path answers `POST /messaging/preflight` 200-with-advisories and `GET /messaging/advisory-log` 200, enforces Bearer auth (401 without), writes the JSONL where documented, and the live-config kill switch flips the preflight off WITHOUT a restart (`tests/e2e/outbound-advisory-alive.test.ts`, 5 tests).
|
|
38
|
+
- `tsc --noEmit` exit 0; full unit suite green before push.
|