@runeditrun/rer 0.2.88 → 0.2.89

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -62,7 +62,11 @@ export async function serveHosted(root, { declaration: declarationPath = 'deploy
62
62
  const nativeBindings = Object.fromEntries(Object.entries(nativeWorkers).map(([entrypoint, binding]) => [binding, { name: 'app', entrypoint }]));
63
63
  const modules = [entry, ...(await files(build)).map(path => join(build, path)).filter(path => path !== entry && moduleTypes[extname(path)])].map(path => ({ type: moduleTypes[extname(path)], path }));
64
64
  const identity = { sourceSha: createHash('sha1').update(randomBytes(16)).digest('hex'), fingerprint: createHash('sha256').update(randomBytes(16)).digest('hex'), versionId: crypto.randomUUID() };
65
- const secrets = Object.fromEntries((declaration.secrets ?? []).map(item => [item.binding, standIn(item, manifest.fields?.find(field => field.name === item.binding))]));
65
+ // Optional integrations stay unconfigured unless verification opts in. Invented
66
+ // endpoint/certificate values can enable an invalid native auth configuration.
67
+ const secrets = Object.fromEntries((declaration.secrets ?? [])
68
+ .filter(item => item.optional !== true || typeof item.verifyStandIn === 'string' && item.verifyStandIn)
69
+ .map(item => [item.binding, standIn(item, manifest.fields?.find(field => field.name === item.binding))]));
66
70
  for (const binding of await hostedMailCredentials(root)) {
67
71
  if (!Object.hasOwn(secrets, binding)) secrets[binding] = standIn({}, manifest.fields?.find(field => field.name === binding));
68
72
  }
@@ -1,5 +1,17 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.2.89
4
+
5
+ - Publication email hygiene recognizes parsed dependency package contact metadata and registry deprecation notices in pnpm, npm and Yarn lockfiles. Comments, custom keys, repository author fields, source files and secret checks retain their findings; no address allowlist is needed. This scanner source update can be adopted independently of a launcher package release.
6
+
7
+ - Re-execute completed read-only approval replays under current native grants, so aggregate reads cannot disclose rows retained before client revocation. Native empty results and refusals replace cached payloads; unchanged visibility retains the result and approval receipt. Mutation replays still execute their effect once.
8
+
9
+ - Leave optional deploy secrets unconfigured during hosted output and agent-door verification unless they declare a nonempty `verifyStandIn`. This prevents invented SSO endpoints from causing HTTP 500 before signed owner bootstrap. Required and explicitly selected stand-ins remain; public-door coverage retains invalid-signature and replay refusals. See `ADOPTING.md`.
10
+
11
+ - Verify legacy v2 backups at their recorded migration schema before applying the additive bridge or a declared row upgrade. Missing or unknown history, schema drift and altered archive digests remain refused before staging.
12
+
13
+ - Reconstruct excluded-table objects omitted by older v2 inventories only when recorded replay proves that historical inventory and the complete live destination. Unrecorded excluded objects remain refused.
14
+
3
15
  ## 0.2.88
4
16
 
5
17
  - Agent doors can declare the mandatory tools/policy/pause/audit/idempotency core plus events, resources, prompts and subscriptions. Discovery and conformance report declared, refused and unscoped families; omitted supported capabilities are hidden and return typed refusals. Core authorization and security checks remain mandatory, and absent declarations retain the legacy full requirements. Other lifecycle families remain explicitly unscoped with their existing checks intact.
@@ -481,14 +481,24 @@ is available. Current archives carry the complete schema inventory and restore
481
481
  without any acknowledgment. Version 2 backups written by rer 0.2.79 or earlier
482
482
  can lack `schemaObjects`. Ordinary import verifies such a backup without an
483
483
  acknowledgment when its nonempty recorded migration history, resolved against the
484
- destination's own applied files in order, replays to the complete live destination
485
- schema identity, including indexes, views and triggers. The same replay's tables
486
- must match the archive's table definitions. The archive's ledger, when present,
484
+ destination's own applied files in order, reproduces the archive's table
485
+ definitions. Replay reconstructs that recorded schema's complete identity,
486
+ including indexes, views and triggers, before the ordinary additive bridge checks
487
+ later migrations. A seed's explicit `acceptedSchemas` row upgrade remains
488
+ available for its declared archive identity. The complete applied replay must
489
+ also match the live destination. The archive's ledger, when present,
487
490
  must agree with its recorded names, and available hashes must match. Missing
488
491
  history, failed replay, table drift or a different complete destination identity
489
492
  still refuses `schema_mismatch:legacy_schema` before staging. Source SQL is never
490
493
  installed. This verified path records no `schemaUnverified` receipt.
491
494
 
495
+ Some later v2 writers carried `schemaObjects` but omitted indexes and triggers
496
+ attached to excluded tables. Their historical table/object projection must match
497
+ the recorded replay exactly before its complete identity can be used for bridge
498
+ placement. An excluded-table object absent from the destination's migration
499
+ replay still refuses before staging. Original archive bytes and digests remain
500
+ unchanged in both paths.
501
+
492
502
  When that proof is unavailable, the owner can explicitly accept that its schema
493
503
  objects are unverified:
494
504
 
@@ -70,9 +70,10 @@ async function placement(db,manifest,cat,replay,ledger,history,legacyRecovery,un
70
70
  return {applied,earlier,replays,rebuilt};
71
71
  }
72
72
  // Old v2 writers recorded tables and applied migration names, but no behavioral
73
- // object inventory. Verify both projections: the recorded files must reproduce
74
- // the complete live schema, and their tables must reproduce the archive. Never
75
- // guess an unrecorded prefix or install source SQL to fill missing evidence.
73
+ // object inventory (later writers omitted objects attached to excluded tables).
74
+ // Verify the archive's historical projection and the complete live replay, then
75
+ // return the recorded schema's complete identity for ordinary bridge placement.
76
+ // Never guess an unrecorded prefix or install source SQL to fill missing evidence.
76
77
  export async function verifiedLegacySchema(db,manifest,cat,migrations,history) {
77
78
  if(!migrations)return false;
78
79
  const replay=await migrations(),ledger=await destinationLedger(db)??[];
@@ -84,9 +85,9 @@ export async function verifiedLegacySchema(db,manifest,cat,migrations,history) {
84
85
  if(!recorded.length || canonicalJson(applied)!==canonicalJson(replay.names)
85
86
  || !recorded.every((name,i)=>applied.indexOf(name)>(i?applied.indexOf(recorded[i-1]):-1)))return false;
86
87
  if(archived && manifest.migrations && canonicalJson(matchMigrationLedger(archived,files,replay))!==canonicalJson(recorded))return false;
87
- return await replay.schemaAfter(applied)===cat.identity
88
- && await replay.schemaAfter(recorded)===cat.identity
89
- && await replay.schemaAfter(recorded,{tablesOnly:true})===manifest.schema;
88
+ if(await replay.schemaAfter(applied)!==cat.identity
89
+ || await replay.schemaAfter(recorded,manifest.schemaObjects===undefined?{tablesOnly:true}:{legacyExcludedObjects:true})!==manifest.schema)return false;
90
+ return await replay.schemaAfter(recorded);
90
91
  }catch{return false;} // Unknown history or a failed replay is not verification.
91
92
  }
92
93
  // An archive with the destination's schema identity can still predate later
@@ -31,13 +31,14 @@ export async function createImport(context,manifest,actor='direct',legacyRecover
31
31
  const cat=await catalogue(db,config),upgrade=config.acceptedSchemas[original.schema];
32
32
  const legacy=original.version===2 && manifest.schemaObjects===undefined;
33
33
  // An explicit acknowledgment keeps its existing recovery semantics. Without
34
- // it, legacy v2 needs positive replay evidence for the complete destination.
34
+ // it, legacy v2 needs positive replay evidence for its recorded schema and
35
+ // the complete destination. The ordinary bridge then checks later migrations.
35
36
  const unverified=legacy && acknowledgeUnverifiedSchemaObjects===true;
36
37
  const verifyingLegacy=legacy && !unverified && !legacyRecovery;
37
38
  const history=()=>archivedHistory(original,ledgers,{allowOlderShape:unverified || verifyingLegacy});
38
- const verified=verifyingLegacy && await verifiedLegacySchema(db,manifest,cat,context.migrations,history);
39
+ const verified=(verifyingLegacy || (!legacy && manifest.schema!==cat.identity)) && await verifiedLegacySchema(db,manifest,cat,context.migrations,history);
39
40
  insist(!legacy || legacyRecovery || unverified || verified,'schema_mismatch:legacy_schema',409);
40
- if(verified)manifest={...manifest,schema:cat.identity,schemaObjects:cat.schemaObjects};
41
+ if(verified)manifest={...manifest,schema:verified};
41
42
  if(unverified)for(const table of manifest.tables) {
42
43
  const target=cat.tables.find(t=>t.name===table.name);
43
44
  insist(target,'schema_mismatch:table:'+table.name,409);
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@runeditrun/rer",
3
- "version": "0.2.88",
3
+ "version": "0.2.89",
4
4
  "description": "The rer command: install, set up, run and launch Run Edit Run checkouts.",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -61,6 +61,15 @@ bounded public coauthor address rule described in the command guide; a file name
61
61
  masks only the accepted address for email inspection, leaving names, other
62
62
  addresses and every other rule intact.
63
63
 
64
+ `lockfile_metadata.py` uses `lockfile_metadata.mjs` and the maintained vendored
65
+ YAML parser to identify dependency package contact and deprecation scalar spans
66
+ in pnpm, npm (lock versions 1–3) and Yarn Berry locks. A bounded Yarn v1 parser
67
+ handles its colon-free scalar members and grouped selectors. Only email
68
+ inspection masks these spans; JSON/YAML duplicates, aliases, tags, malformed
69
+ input and unrecognized shapes grant no exception. Comments and custom fields
70
+ retain their own locations, even beside an accepted value on a minified line.
71
+ Repository package metadata, including npm's empty package key, stays inspected.
72
+
64
73
  `png_text.py` separates the fields of framed PNG `tEXt` and uncompressed `iTXt`
65
74
  chunks before text hygiene inspection. Container length, type and CRC bytes
66
75
  cannot extend a mailbox, and these explicit text fields do not use the binary
@@ -0,0 +1,141 @@
1
+ // Return only scalar value locations in dependency package contact metadata.
2
+ // Use the maintained YAML parser; no package installation or lockfile execution.
3
+ import { readFileSync } from 'node:fs';
4
+ import { isMap, isScalar, isSeq, parseDocument, visit } from '../../edition-authoring/vendor/yaml/index.mjs';
5
+
6
+ const personFields = new Set(['name', 'email', 'url']);
7
+ const single = new Set(['author', 'maintainer']);
8
+ const multiple = new Set(['maintainers', 'contributors']);
9
+ const packageName = /^(?:@[^\s/@]+\/)?[^\s/@]+$/;
10
+ const selector = /^(?:@[^\s/@]+\/)?[^\s/@]+@[^\s]+$/;
11
+ const pnpmSelector = /^(?:\/?(?:@[^\s/@]+\/)?[^\s/@]+)(?:@|\/)[^\s]+$/;
12
+ const key = node => isScalar(node) && typeof node.value === 'string' ? node.value : null;
13
+ const field = (node, name) => isMap(node) ? node.items.find(pair => key(pair.key) === name)?.value : undefined;
14
+ const string = node => isScalar(node) && typeof node.value === 'string';
15
+
16
+ function metadata(node, spans) {
17
+ if (!isMap(node)) return;
18
+ const scalar = value => { if (string(value)) spans.push(value.range.slice(0, 2)); };
19
+ const person = value => {
20
+ if (string(value)) scalar(value);
21
+ else if (isMap(value) && value.items.every(pair => personFields.has(key(pair.key)) && string(pair.value))) {
22
+ for (const pair of value.items) scalar(pair.value);
23
+ }
24
+ };
25
+ for (const pair of node.items) {
26
+ const name = key(pair.key), value = pair.value;
27
+ if (single.has(name)) person(value);
28
+ else if (multiple.has(name) && isSeq(value)) for (const entry of value.items) person(entry);
29
+ // npm's deprecation advisory is registry-supplied package metadata too.
30
+ else if (name === 'email' || name === 'deprecated') scalar(value);
31
+ }
32
+ }
33
+
34
+ function structured(text, filename) {
35
+ if (filename === 'package-lock.json') JSON.parse(text);
36
+ const document = parseDocument(text, { uniqueKeys: true, strict: true });
37
+ if (document.errors.length || !isMap(document.contents)) return [];
38
+ let ambiguous = false;
39
+ visit(document, { Node(_, node) { if (node.tag || node.anchor || node.constructor.name === 'Alias') ambiguous = true; } });
40
+ if (ambiguous) return [];
41
+ const root = document.contents, spans = [];
42
+ if (filename === 'pnpm-lock.yaml') {
43
+ if (!/^(?:[1-9]\d*)(?:\.\d+)?$/.test(String(field(root, 'lockfileVersion')?.value ?? ''))) return [];
44
+ const packages = field(root, 'packages');
45
+ if (isMap(packages)) for (const pair of packages.items) {
46
+ if (pnpmSelector.test(key(pair.key) ?? '')) metadata(pair.value, spans);
47
+ }
48
+ } else if (filename === 'package-lock.json') {
49
+ const version = field(root, 'lockfileVersion')?.value;
50
+ if (![1, 2, 3].includes(version)) return [];
51
+ const packages = field(root, 'packages');
52
+ if (version >= 2 && isMap(packages)) for (const pair of packages.items) {
53
+ // The empty key is the repository's own package, never a dependency.
54
+ if (/(?:^|\/)node_modules\/(?:@[^/\s]+\/)?[^/\s]+$/.test(key(pair.key) ?? '') &&
55
+ string(field(pair.value, 'version'))) metadata(pair.value, spans);
56
+ }
57
+ const dependencies = node => {
58
+ if (!isMap(node)) return;
59
+ for (const pair of node.items) if (packageName.test(key(pair.key) ?? '') && isMap(pair.value)) {
60
+ if (string(field(pair.value, 'version'))) metadata(pair.value, spans);
61
+ dependencies(field(pair.value, 'dependencies'));
62
+ }
63
+ };
64
+ if (version <= 2) dependencies(field(root, 'dependencies'));
65
+ } else {
66
+ if (!isMap(field(root, '__metadata')) || !field(field(root, '__metadata'), 'version')) return [];
67
+ for (const pair of root.items) {
68
+ if ((key(pair.key) ?? '').split(/,\s*/).every(name => selector.test(name))) metadata(pair.value, spans);
69
+ }
70
+ }
71
+ return spans;
72
+ }
73
+
74
+ function classic(text) {
75
+ // Yarn v1 is not YAML: scalar members omit the colon and selectors can be
76
+ // grouped. Parse its indentation and JSON-quoted strings before classifying.
77
+ if (!/^# yarn lockfile v1\r?$/m.test(text)) return [];
78
+ const spans = [], stack = [], records = [], selectors = new Set();
79
+ let offset = 0, current = null;
80
+ for (const line of text.split(/(?<=\n)/)) {
81
+ const raw = line.replace(/\r?\n$/, '');
82
+ const start = offset;
83
+ offset += line.length;
84
+ if (/^\s*(?:#.*)?$/.test(raw)) continue;
85
+ if (!raw.startsWith(' ')) {
86
+ const header = raw.match(/^((?:"(?:[^"\\]|\\.)*"|[^\s",]+)(?:,\s*(?:"(?:[^"\\]|\\.)*"|[^\s",]+))*):$/);
87
+ if (!header) return [];
88
+ const names = header[1].match(/"(?:[^"\\]|\\.)*"|[^\s,]+/g).map(value => value.startsWith('"') ? JSON.parse(value) : value);
89
+ if (!names.every(name => selector.test(name) && !selectors.has(name))) return [];
90
+ for (const name of names) selectors.add(name);
91
+ current = { items: [], keys: new Set() };
92
+ records.push(current);
93
+ stack.splice(0, stack.length, { indent: 0, node: current });
94
+ continue;
95
+ }
96
+ if (!current) return [];
97
+ const member = raw.match(/^( +)("(?:[^"\\]|\\.)*"|[A-Za-z][A-Za-z0-9_-]*)(?::|[ \t]+((?:"(?:[^"\\]|\\.)*"|[^\s"#]+)))([ \t]*(?:#.*)?)$/);
98
+ if (!member) return [];
99
+ const indent = member[1].length;
100
+ while (stack.length && stack.at(-1).indent >= indent) stack.pop();
101
+ const parent = stack.at(-1);
102
+ if (!parent || indent !== parent.indent + 2) return [];
103
+ const name = member[2].startsWith('"') ? JSON.parse(member[2]) : member[2];
104
+ if (parent.node.keys.has(name)) return [];
105
+ parent.node.keys.add(name);
106
+ let value;
107
+ if (member[3] === undefined) {
108
+ value = { items: [], keys: new Set() };
109
+ stack.push({ indent, node: value });
110
+ } else {
111
+ const at = start + member[1].length + member[2].length + raw.slice(member[1].length + member[2].length).search(/\S/);
112
+ const decoded = member[3].startsWith('"') ? JSON.parse(member[3]) : member[3];
113
+ value = { value: decoded, range: [at, at + member[3].length] };
114
+ }
115
+ parent.node.items.push({ name, value });
116
+ }
117
+ // Only direct package metadata and documented person members qualify.
118
+ const person = value => {
119
+ if (typeof value.value === 'string') spans.push(value.range);
120
+ else if (value.items.every(pair => personFields.has(pair.name) && typeof pair.value.value === 'string')) {
121
+ for (const pair of value.items) spans.push(pair.value.range);
122
+ }
123
+ };
124
+ for (const record of records) {
125
+ for (const pair of record.items) {
126
+ if (single.has(pair.name)) person(pair.value);
127
+ else if (['email', 'deprecated'].includes(pair.name) && typeof pair.value.value === 'string') spans.push(pair.value.range);
128
+ }
129
+ }
130
+ return spans;
131
+ }
132
+
133
+ try {
134
+ const text = readFileSync(0, 'utf8');
135
+ const filename = process.argv[2];
136
+ const spans = filename === 'yarn.lock' && /^# yarn lockfile v1\r?$/m.test(text) ? classic(text) : structured(text, filename);
137
+ process.stdout.write(JSON.stringify(spans.map(([a, b]) => [Buffer.byteLength(text.slice(0, a)), Buffer.byteLength(text.slice(0, b))])));
138
+ } catch {
139
+ // Malformed/unsupported input grants no metadata exception.
140
+ process.stdout.write('[]');
141
+ }
@@ -0,0 +1,30 @@
1
+ """Exact dependency metadata scalar spans; parser failure keeps email findings."""
2
+ from functools import lru_cache
3
+ import json
4
+ from pathlib import Path
5
+ import subprocess
6
+
7
+ PARSER = Path(__file__).with_suffix('.mjs')
8
+ LOCKFILES = {'pnpm-lock.yaml', 'package-lock.json', 'yarn.lock'}
9
+
10
+
11
+ @lru_cache(maxsize=32)
12
+ def without_package_contacts(data, filename):
13
+ if filename not in LOCKFILES:
14
+ return data
15
+ try:
16
+ data.decode('utf-8')
17
+ result = subprocess.run(['node', str(PARSER), filename], input=data, capture_output=True, timeout=10)
18
+ if result.returncode:
19
+ return data
20
+ spans = json.loads(result.stdout)
21
+ if any(not isinstance(a, int) or not isinstance(b, int) or not 0 <= a <= b <= len(data) for a, b in spans):
22
+ return data
23
+ except (OSError, ValueError, UnicodeError, subprocess.TimeoutExpired):
24
+ return data
25
+ output, offset = [], 0
26
+ for start, end in sorted(spans):
27
+ output.extend((data[offset:start], b' '))
28
+ offset = max(offset, end)
29
+ output.append(data[offset:])
30
+ return b''.join(output)
@@ -11,6 +11,7 @@ from trust_bundle import anchor_values
11
11
  from png_text import text_chunks
12
12
  from media import sample_spans
13
13
  from c2pa import certificate_spans
14
+ from lockfile_metadata import without_package_contacts
14
15
  URI_AUTHORITY = re.compile(r'''//(?P<authority>[^/?#\s<>"']+)''')
15
16
  ACTION_REFERENCE = re.compile(r'''uses:\s*["']?[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+@[^\s"']+''')
16
17
  SCHEME_CHARACTERS = frozenset('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789+.-')
@@ -188,7 +189,8 @@ def inspect_text(data, path, config, *, commit_message=False, binary_content=Non
188
189
  if any(pattern.search(text) for pattern in config['internalReferencePatterns']):
189
190
  rules.add('hygiene.internal-reference')
190
191
  if '@' in text:
191
- email_text = decoded_text(without_public_coauthor_addresses(data)) if commit_message else text
192
+ email_data = without_public_coauthor_addresses(data) if commit_message else without_package_contacts(data, path.rsplit('/', 1)[-1])
193
+ email_text = decoded_text(email_data)
192
194
  emails = private_emails(email_text, config)
193
195
  bundle = config.get('trustBundles', {}).get(path)
194
196
  if emails and not (bundle and certificate_metadata(data, bundle, emails, config)):