@synapsor/runner 1.4.122 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +71 -19
- package/README.md +56 -46
- package/dist/cli.d.ts.map +1 -1
- package/dist/local-ui.d.ts.map +1 -1
- package/dist/runner.mjs +14531 -8137
- package/dist/runtime.d.ts +50 -0
- package/dist/runtime.d.ts.map +1 -0
- package/dist/runtime.mjs +15066 -0
- package/docs/README.md +19 -3
- package/docs/capability-authoring.md +17 -1
- package/docs/contract-review.md +33 -0
- package/docs/database-enforced-scope.md +273 -0
- package/docs/dependency-license-inventory.md +6 -2
- package/docs/doctor.md +17 -0
- package/docs/effect-regression.md +142 -0
- package/docs/getting-started-own-database.md +7 -0
- package/docs/local-mode.md +7 -3
- package/docs/mcp-apps.md +188 -0
- package/docs/mcp-audit.md +54 -0
- package/docs/mcp-client-setup.md +16 -4
- package/docs/mcp-clients.md +8 -0
- package/docs/production.md +8 -0
- package/docs/release-notes.md +67 -11
- package/docs/release-policy.md +9 -5
- package/docs/runner-config-reference.md +25 -0
- package/docs/schema-api-candidates.md +68 -0
- package/docs/security-boundary.md +14 -1
- package/docs/shadow-studies.md +198 -0
- package/examples/mcp-postgres-billing-app-handler/app-handler.mjs +9 -1
- package/examples/mcp-postgres-billing-app-handler/schema.sql +11 -0
- package/examples/mcp-postgres-billing-app-handler/synapsor-handler.mjs +3 -0
- package/examples/raw-sql-vs-synapsor/Makefile +2 -2
- package/examples/raw-sql-vs-synapsor/README.md +6 -4
- package/examples/support-billing-agent/Makefile +5 -1
- package/examples/support-billing-agent/README.md +39 -5
- package/examples/support-billing-agent/db/schema.sql +88 -1
- package/examples/support-billing-agent/db/seed.sql +15 -15
- package/examples/support-billing-agent/scripts/run-demo.sh +4 -0
- package/examples/support-billing-agent/scripts/run-evaluation.sh +62 -0
- package/examples/support-billing-agent/shadow-study/cases.jsonl +6 -0
- package/examples/support-billing-agent/shadow-study/outcomes.jsonl +2 -0
- package/examples/support-billing-agent/synapsor.runner.json +17 -5
- package/fixtures/contracts/capability-surface-fitness.contract.json +190 -0
- package/fixtures/effects/changed/eff_support_late_fee_v1.result.json +58 -0
- package/fixtures/effects/dataset.json +8 -0
- package/fixtures/effects/results/eff_support_late_fee_v1.result.json +59 -0
- package/fixtures/effects/support-late-fee.effect.json +71 -0
- package/fixtures/generators/drizzle/malicious.ts +11 -0
- package/fixtures/generators/drizzle/schema.ts +12 -0
- package/fixtures/generators/openapi/external-ref.yaml +15 -0
- package/fixtures/generators/openapi/openapi.yaml +81 -0
- package/fixtures/generators/prisma/schema.prisma +34 -0
- package/package.json +13 -2
- package/schemas/effect-dataset.schema.json +19 -0
- package/schemas/effect-fixture.schema.json +191 -0
- package/schemas/effect-result.schema.json +52 -0
- package/schemas/mcp-audit-candidates.schema.json +41 -0
- package/schemas/mcp-audit-report.schema.json +104 -0
- package/schemas/schema-candidate-review.schema.json +167 -0
- package/schemas/schema-candidates.schema.json +49 -0
- package/schemas/synapsor.runner.schema.json +51 -0
package/docs/README.md
CHANGED
|
@@ -28,9 +28,10 @@ no-database demo, wire your database, then read deeper concepts.
|
|
|
28
28
|
Guardrails](why-synapsor-vs-app-guardrails.md): where SQL authority lives,
|
|
29
29
|
what a hand-built semantic tool already gets right, and when the shared
|
|
30
30
|
contract, approval, receipt, and replay layer is worth adopting.
|
|
31
|
-
- [MCP Audit](mcp-audit.md): static review for risky database MCP tools
|
|
32
|
-
`execute_sql`, broad query tools, model-controlled tenant filters, or
|
|
33
|
-
model-facing approval/commit tools
|
|
31
|
+
- [MCP Audit](mcp-audit.md): concise static review for risky database MCP tools
|
|
32
|
+
such as `execute_sql`, broad query tools, model-controlled tenant filters, or
|
|
33
|
+
model-facing approval/commit tools, plus explicit generation of disabled
|
|
34
|
+
canonical review candidates.
|
|
34
35
|
|
|
35
36
|
## 03 Run The Demo
|
|
36
37
|
|
|
@@ -49,6 +50,9 @@ no-database demo, wire your database, then read deeper concepts.
|
|
|
49
50
|
links to the canonical own-database guide.
|
|
50
51
|
- [Doctor](doctor.md): redacted setup checks, handler probes, direct SQL
|
|
51
52
|
writeback probes, and receipt-table guidance.
|
|
53
|
+
- [Database-Enforced Scope](database-enforced-scope.md): application-level
|
|
54
|
+
scope, hardened PostgreSQL RLS, tenant-bound credentials/deployments, and
|
|
55
|
+
honest MySQL alternatives.
|
|
52
56
|
- [Guarded Single-Row CRUD Writeback](guarded-crud-writeback.md): native
|
|
53
57
|
INSERT/UPDATE/DELETE, receipt modes, crash semantics, privileges, and
|
|
54
58
|
reconciliation.
|
|
@@ -60,6 +64,10 @@ no-database demo, wire your database, then read deeper concepts.
|
|
|
60
64
|
|
|
61
65
|
## 05 Generate Capabilities
|
|
62
66
|
|
|
67
|
+
- [Reviewed Prisma, Drizzle, And OpenAPI
|
|
68
|
+
Candidates](schema-api-candidates.md): turn structural developer artifacts
|
|
69
|
+
into deterministic, disabled canonical review candidates without executing
|
|
70
|
+
adopter code or inferring authority.
|
|
63
71
|
- [Capability Authoring](capability-authoring.md): define read/proposal
|
|
64
72
|
capabilities, model-facing descriptions, result envelopes, trusted context,
|
|
65
73
|
and writeback guards.
|
|
@@ -74,11 +82,17 @@ no-database demo, wire your database, then read deeper concepts.
|
|
|
74
82
|
and deterministic lint/SARIF output.
|
|
75
83
|
- [Contract Testing](contract-testing.md): adopter-owned static and disposable
|
|
76
84
|
PostgreSQL/MySQL allow/deny/redaction assertions.
|
|
85
|
+
- [Agent Effect Regression](effect-regression.md): provider-neutral,
|
|
86
|
+
propose-only fixtures that catch changed capability calls, business diffs,
|
|
87
|
+
policy outcomes, tenant handling, and hidden-field behavior.
|
|
77
88
|
- [Bounded Aggregate Reads](aggregate-reads.md): fixed scalar count/sum/avg,
|
|
78
89
|
tenant scope, suppression, and no-row evidence.
|
|
79
90
|
|
|
80
91
|
## 06 Serve MCP
|
|
81
92
|
|
|
93
|
+
- [Inline Proposal Review With MCP Apps](mcp-apps.md): display-only proposal
|
|
94
|
+
cards where supported, exact protocol versions, tested compatibility, and
|
|
95
|
+
secure standalone review fallback.
|
|
82
96
|
- [MCP Client Setup](mcp-client-setup.md): connect Claude, Cursor, VS Code, or
|
|
83
97
|
another stdio MCP client.
|
|
84
98
|
- [MCP Client Configs](mcp-clients.md): complete Claude, Cursor, OpenAI Agents,
|
|
@@ -96,6 +110,8 @@ no-database demo, wire your database, then read deeper concepts.
|
|
|
96
110
|
|
|
97
111
|
- [Local Mode](local-mode.md): local store, proposals, approval, replay, and
|
|
98
112
|
writeback flow.
|
|
113
|
+
- [Shadow Studies](shadow-studies.md): compare what an agent would propose
|
|
114
|
+
with explicit authorized outcomes before granting write authority.
|
|
99
115
|
- [Writeback Executors](writeback-executors.md): app-owned writeback handlers
|
|
100
116
|
for approved proposals.
|
|
101
117
|
- [App-Owned Executors](app-owned-executors.md): short entry point for rich
|
|
@@ -121,6 +121,22 @@ synapsor-runner mcp serve --config ./synapsor.runner.json --store ./.synapsor/lo
|
|
|
121
121
|
capabilities cannot accidentally lose descriptions, returns hints, or numeric
|
|
122
122
|
patch bounds during review.
|
|
123
123
|
|
|
124
|
+
After compilation, run the canonical contract through the surface review:
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
synapsor-runner contract lint ./synapsor.contract.json
|
|
128
|
+
synapsor-runner contract lint ./synapsor.contract.json --strict
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
The default command keeps capability-surface findings advisory; `--strict` opts
|
|
132
|
+
into warning-as-failure CI policy. Review capabilities using a simple fitness
|
|
133
|
+
test: if the name is not a business operation an audit log would recognize, ask
|
|
134
|
+
whether the model should receive it. Lint also surfaces generic query-like
|
|
135
|
+
string arguments, more than eight capabilities on one target, and structurally
|
|
136
|
+
near-duplicate tools. These checks make breadth drift visible; they do not
|
|
137
|
+
change canonical validity or replace human review. See [Contract
|
|
138
|
+
Review](contract-review.md) for exact codes and behavior.
|
|
139
|
+
|
|
124
140
|
## DSL / JSON Capability Parity
|
|
125
141
|
|
|
126
142
|
The DSL compiles to canonical `@synapsor/spec` JSON. It must not silently weaken
|
|
@@ -134,7 +150,7 @@ reviewed runner JSON capabilities. Current parity:
|
|
|
134
150
|
| arg `minimum` | `ARG amount NUMBER MIN 1` | 0.1.8 | NUMBER args only. |
|
|
135
151
|
| arg `maximum` | `ARG amount NUMBER MAX 2500` | 0.1.8 | NUMBER args only. |
|
|
136
152
|
| arg `max_length` | `ARG reason TEXT MAX LENGTH 500` | 0.1.8 | STRING/TEXT args only. Legacy `MAX 500` is still accepted for existing DSL files, but `MAX LENGTH` is the reviewed spelling. |
|
|
137
|
-
| arg `enum` |
|
|
153
|
+
| arg `enum` | `ARG risk_level STRING ENUM('low', 'medium', 'high') REQUIRED` | 1.4.121 | Supports 1..64 same-type string, number, or boolean values; duplicates and mixed types fail compilation. |
|
|
138
154
|
| proposal `numeric_bounds` | `BOUND column 1..2500`, `BOUND column ..2500`, or `BOUND column 1..` | 0.1.8 | Applies to patched numeric columns. Strict mode warns when a NUMBER arg is patched without arg min/max or a matching `BOUND`. |
|
|
139
155
|
| proposal `transition_guards` | `TRANSITION status ALLOW pending -> approved\|rejected` or `TRANSITION status FROM current_status ALLOW open -> closed` | 0.1.8 | Values are state strings; use `|` for multiple target states. |
|
|
140
156
|
| proposal `conflict_guard` | `CONFLICT GUARD updated_at` | 0.1 | If omitted, DSL emits an explicit weak-guard acknowledgement. Prefer a real row-version column. |
|
package/docs/contract-review.md
CHANGED
|
@@ -33,9 +33,42 @@ cover objective review gaps such as missing descriptions/evidence, unbounded
|
|
|
33
33
|
strings, unresolved wiring, irreversible operations, and policy/writeback
|
|
34
34
|
contradictions. Lint does not claim to discover every sensitive column.
|
|
35
35
|
|
|
36
|
+
### Review Capability-Surface Fitness
|
|
37
|
+
|
|
38
|
+
Runner structurally limits capability **depth**: contracts cannot turn a model
|
|
39
|
+
argument into raw SQL or a free-form predicate, and trusted tenant/principal
|
|
40
|
+
scope is never model input. A separate review concern is capability **breadth**:
|
|
41
|
+
many individually narrow tools can accumulate until the total model-facing
|
|
42
|
+
surface is difficult to understand.
|
|
43
|
+
|
|
44
|
+
Use this fitness test during review:
|
|
45
|
+
|
|
46
|
+
> If a capability cannot be described as a named business operation an audit
|
|
47
|
+
> log would recognize, it probably should not be exposed yet.
|
|
48
|
+
|
|
49
|
+
`contract lint` reports these deterministic advisory warnings:
|
|
50
|
+
|
|
51
|
+
| Code | Review signal |
|
|
52
|
+
| --- | --- |
|
|
53
|
+
| `SURFACE_GENERIC_ARGUMENT` | An un-enumerated string argument is literally named `filter`, `query`, `where`, `predicate`, or `sql`. Runner still does not interpolate it into SQL. |
|
|
54
|
+
| `SURFACE_TARGET_DENSITY` | More than eight capabilities target the same normalized source and object. Eight is a review threshold, not a runtime limit. |
|
|
55
|
+
| `SURFACE_OPERATION_NAMING` | A capability name does not match the conservative named-business-operation heuristic. |
|
|
56
|
+
| `SURFACE_NEAR_DUPLICATE` | Two capabilities have the same target, kind, reviewed fields, targeting, write/approval shape, and identical or directionally loosened arguments. |
|
|
57
|
+
|
|
58
|
+
The default command exits successfully when these advisories are the only
|
|
59
|
+
findings. `--strict` (or `--fail-on warning`) deliberately lets a team turn all
|
|
60
|
+
warnings into a CI policy gate without changing canonical contract validity.
|
|
61
|
+
Text, JSON, and SARIF use the same stable finding set and deterministic order.
|
|
62
|
+
|
|
63
|
+
Passing these checks does not prove that a surface is safe or well designed.
|
|
64
|
+
Review which operations each agent actually needs; use narrower contracts or
|
|
65
|
+
Runner/client deployments instead of exposing every organization capability to
|
|
66
|
+
every model.
|
|
67
|
+
|
|
36
68
|
Expected successful output ends with:
|
|
37
69
|
|
|
38
70
|
```text
|
|
71
|
+
Surface: N model-facing capabilities across M targets; 0 target(s) above the advisory density threshold of 8
|
|
39
72
|
Summary: 0 error / 0 warning / N info
|
|
40
73
|
```
|
|
41
74
|
|
|
@@ -0,0 +1,273 @@
|
|
|
1
|
+
# Database-Enforced Tenant And Principal Scope
|
|
2
|
+
|
|
3
|
+
Runner always binds tenant and principal from trusted server-side context, not
|
|
4
|
+
from model arguments. Choose the database enforcement mode deliberately:
|
|
5
|
+
|
|
6
|
+
| Diagnostic mode | What enforces scope | Protects against | Does not protect against |
|
|
7
|
+
| --- | --- | --- | --- |
|
|
8
|
+
| `application_scope` | Runner's fixed, parameterized predicates with one least-privilege credential | A model trying to widen tenant or principal arguments | A defect in Runner's predicate construction |
|
|
9
|
+
| `postgres_rls` | Runner predicates plus PostgreSQL row-level security | Omitted/wrong Runner predicates and pooled-session context leakage | A fully compromised process that can choose trusted settings while holding a broad credential |
|
|
10
|
+
| `tenant_bound` | Runner predicates plus a credential or process that cannot access other tenants | Query mistakes and a process that never receives organization-wide database authority | Incorrect grants or isolation in the credential/deployment itself |
|
|
11
|
+
|
|
12
|
+
These modes are defense in depth, not substitutes for least-privilege roles,
|
|
13
|
+
restricted views, application authorization, or staging-first validation.
|
|
14
|
+
|
|
15
|
+
## Default: Application-Level Scope
|
|
16
|
+
|
|
17
|
+
Existing source configs remain compatible. An omitted `database_scope`, or:
|
|
18
|
+
|
|
19
|
+
```json
|
|
20
|
+
{
|
|
21
|
+
"database_scope": { "mode": "application" },
|
|
22
|
+
"credential_scope": { "mode": "shared" }
|
|
23
|
+
}
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
means Runner adds reviewed tenant and optional principal predicates to every
|
|
27
|
+
supported read and write using one environment-bound credential. This stops
|
|
28
|
+
the model from selecting another tenant, because scope is not a tool argument.
|
|
29
|
+
It is still application-level isolation: a defect in Runner's SQL construction
|
|
30
|
+
could cross that boundary. Keep database permissions, restricted views, and
|
|
31
|
+
RLS where available.
|
|
32
|
+
|
|
33
|
+
## Trusted-Context Provenance
|
|
34
|
+
|
|
35
|
+
The tenant value is part of the security boundary. Runner accepts it through
|
|
36
|
+
these reviewed paths:
|
|
37
|
+
|
|
38
|
+
| Deployment | Trusted-context source | Diagnostic binding |
|
|
39
|
+
| --- | --- | --- |
|
|
40
|
+
| Local stdio or one explicitly single-tenant process | Operator-controlled environment or local development binding | `process_bound` |
|
|
41
|
+
| Shared Streamable HTTP | Verified signed JWT claims configured with `http_claims` and `session_auth` | `verified_http_session` |
|
|
42
|
+
| Cloud embedding | A verified per-session binding supplied by the embedding control plane | `verified_external_session` |
|
|
43
|
+
|
|
44
|
+
Runner does not treat a tool argument, query parameter, arbitrary MCP `_meta`
|
|
45
|
+
value, `X-Tenant-*` header, or unverified forwarded header as tenant or
|
|
46
|
+
principal authority. A shared HTTP catalog cannot mix claims-bound capabilities
|
|
47
|
+
with environment-bound capabilities.
|
|
48
|
+
|
|
49
|
+
For shared production HTTP, prefer asymmetric JWT verification so Runner holds
|
|
50
|
+
only public verification material. HS256 remains useful for local development
|
|
51
|
+
and controlled deployments but gives Runner access to the signing secret.
|
|
52
|
+
|
|
53
|
+
Run `doctor --json` or `tools preview --json` to inspect the effective
|
|
54
|
+
per-source assurance mode and trusted-context binding. Server startup prints
|
|
55
|
+
the same non-secret summary. An `application_scope` source used with verified
|
|
56
|
+
HTTP sessions is deliberately reported with a warning: authentication makes
|
|
57
|
+
the tenant value trustworthy, but it does not add an independent database
|
|
58
|
+
boundary.
|
|
59
|
+
|
|
60
|
+
## PostgreSQL RLS Mode
|
|
61
|
+
|
|
62
|
+
Add fixed setting names to a PostgreSQL source:
|
|
63
|
+
|
|
64
|
+
```json
|
|
65
|
+
{
|
|
66
|
+
"sources": {
|
|
67
|
+
"billing_postgres": {
|
|
68
|
+
"engine": "postgres",
|
|
69
|
+
"read_url_env": "BILLING_POSTGRES_READ_URL",
|
|
70
|
+
"write_url_env": "BILLING_POSTGRES_WRITE_URL",
|
|
71
|
+
"database_scope": {
|
|
72
|
+
"mode": "postgres_rls",
|
|
73
|
+
"tenant_setting": "synapsor.tenant_id",
|
|
74
|
+
"principal_setting": "synapsor.principal"
|
|
75
|
+
},
|
|
76
|
+
"credential_scope": { "mode": "shared" }
|
|
77
|
+
}
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
For each scoped transaction, Runner binds both values with parameterized
|
|
83
|
+
transaction-local `set_config(..., true)`. Reads, evidence, aggregates,
|
|
84
|
+
bounded-set selection, guarded writes, reconciliation, compensation, and
|
|
85
|
+
verification therefore use the same trusted context. Transaction-local values
|
|
86
|
+
are cleared on commit or rollback and are not reused on the next pooled
|
|
87
|
+
checkout.
|
|
88
|
+
|
|
89
|
+
Runner refuses hardened serving or writeback when the role or target cannot be
|
|
90
|
+
attested. Required properties include:
|
|
91
|
+
|
|
92
|
+
- RLS and `FORCE ROW LEVEL SECURITY` are enabled;
|
|
93
|
+
- the effective role is not a superuser, table owner, or `BYPASSRLS` role;
|
|
94
|
+
- applicable policies are permissive only where the reviewed access requires
|
|
95
|
+
them;
|
|
96
|
+
- every applicable operation policy references both configured settings;
|
|
97
|
+
- `SELECT`/`DELETE` have a suitable `USING` expression;
|
|
98
|
+
- `INSERT` has a suitable `WITH CHECK` expression;
|
|
99
|
+
- `UPDATE` has both `USING` and `WITH CHECK`.
|
|
100
|
+
|
|
101
|
+
### Example PostgreSQL policy
|
|
102
|
+
|
|
103
|
+
Use a non-owner read role and a separate non-owner write role. Adapt names and
|
|
104
|
+
grants to your schema:
|
|
105
|
+
|
|
106
|
+
Runner does not silently create or widen these policies. Review and apply the
|
|
107
|
+
SQL as a database owner, then run doctor using the steady-state non-owner role.
|
|
108
|
+
|
|
109
|
+
```sql
|
|
110
|
+
ALTER TABLE public.invoices ENABLE ROW LEVEL SECURITY;
|
|
111
|
+
ALTER TABLE public.invoices FORCE ROW LEVEL SECURITY;
|
|
112
|
+
|
|
113
|
+
CREATE POLICY invoices_reader_scope
|
|
114
|
+
ON public.invoices
|
|
115
|
+
FOR SELECT
|
|
116
|
+
TO runner_reader
|
|
117
|
+
USING (
|
|
118
|
+
tenant_id = current_setting('synapsor.tenant_id', true)
|
|
119
|
+
AND assigned_to = current_setting('synapsor.principal', true)
|
|
120
|
+
);
|
|
121
|
+
|
|
122
|
+
CREATE POLICY invoices_writer_select_scope
|
|
123
|
+
ON public.invoices
|
|
124
|
+
FOR SELECT
|
|
125
|
+
TO runner_writer
|
|
126
|
+
USING (
|
|
127
|
+
tenant_id = current_setting('synapsor.tenant_id', true)
|
|
128
|
+
AND assigned_to = current_setting('synapsor.principal', true)
|
|
129
|
+
);
|
|
130
|
+
|
|
131
|
+
CREATE POLICY invoices_writer_insert_scope
|
|
132
|
+
ON public.invoices
|
|
133
|
+
FOR INSERT
|
|
134
|
+
TO runner_writer
|
|
135
|
+
WITH CHECK (
|
|
136
|
+
tenant_id = current_setting('synapsor.tenant_id', true)
|
|
137
|
+
AND assigned_to = current_setting('synapsor.principal', true)
|
|
138
|
+
);
|
|
139
|
+
|
|
140
|
+
CREATE POLICY invoices_writer_update_scope
|
|
141
|
+
ON public.invoices
|
|
142
|
+
FOR UPDATE
|
|
143
|
+
TO runner_writer
|
|
144
|
+
USING (
|
|
145
|
+
tenant_id = current_setting('synapsor.tenant_id', true)
|
|
146
|
+
AND assigned_to = current_setting('synapsor.principal', true)
|
|
147
|
+
)
|
|
148
|
+
WITH CHECK (
|
|
149
|
+
tenant_id = current_setting('synapsor.tenant_id', true)
|
|
150
|
+
AND assigned_to = current_setting('synapsor.principal', true)
|
|
151
|
+
);
|
|
152
|
+
|
|
153
|
+
CREATE POLICY invoices_writer_delete_scope
|
|
154
|
+
ON public.invoices
|
|
155
|
+
FOR DELETE
|
|
156
|
+
TO runner_writer
|
|
157
|
+
USING (
|
|
158
|
+
tenant_id = current_setting('synapsor.tenant_id', true)
|
|
159
|
+
AND assigned_to = current_setting('synapsor.principal', true)
|
|
160
|
+
);
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Grant only the operations and columns needed by the reviewed capabilities.
|
|
164
|
+
Policy expressions are not a replacement for grants.
|
|
165
|
+
|
|
166
|
+
Run the normal metadata checks before serving:
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
synapsor-runner doctor --config ./synapsor.runner.json
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
On a disposable or explicitly approved live target, also prove that a known
|
|
173
|
+
row disappears under a different tenant and under a different principal:
|
|
174
|
+
|
|
175
|
+
```bash
|
|
176
|
+
synapsor-runner doctor --config ./synapsor.runner.json --check-rls
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
`--check-rls` reads only through the configured read role. It does not mutate
|
|
180
|
+
the source. Startup and per-connection checks still fail closed when the
|
|
181
|
+
required role or policy properties are absent.
|
|
182
|
+
|
|
183
|
+
PostgreSQL RLS catches a missing application predicate. It does not make a
|
|
184
|
+
fully compromised Runner process safe if that process can choose arbitrary
|
|
185
|
+
trusted setting values while holding a credential that spans tenants. Use a
|
|
186
|
+
tenant-bound credential or isolated deployment for that threat model.
|
|
187
|
+
|
|
188
|
+
## Tenant-Bound Credentials
|
|
189
|
+
|
|
190
|
+
Contracts contain a resolver identifier, never a connection string:
|
|
191
|
+
|
|
192
|
+
```json
|
|
193
|
+
{
|
|
194
|
+
"credential_scope": {
|
|
195
|
+
"mode": "tenant_resolver",
|
|
196
|
+
"resolver": "production_tenant_credentials"
|
|
197
|
+
}
|
|
198
|
+
}
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
An embedding application supplies the matching resolver:
|
|
202
|
+
|
|
203
|
+
```ts
|
|
204
|
+
import {
|
|
205
|
+
startRunnerStreamableHttp,
|
|
206
|
+
type TenantCredentialResolver
|
|
207
|
+
} from "@synapsor/runner/runtime";
|
|
208
|
+
|
|
209
|
+
const credentialResolver: TenantCredentialResolver = {
|
|
210
|
+
id: "production_tenant_credentials",
|
|
211
|
+
async resolve({ source_name, access, tenant_id, principal }) {
|
|
212
|
+
const lease = await credentialBroker.issue({
|
|
213
|
+
source: source_name,
|
|
214
|
+
access,
|
|
215
|
+
tenant: tenant_id,
|
|
216
|
+
principal
|
|
217
|
+
});
|
|
218
|
+
|
|
219
|
+
return {
|
|
220
|
+
connection_url: lease.connectionUrl,
|
|
221
|
+
credential_id: lease.id,
|
|
222
|
+
expires_at: lease.expiresAt
|
|
223
|
+
};
|
|
224
|
+
}
|
|
225
|
+
};
|
|
226
|
+
|
|
227
|
+
await startRunnerStreamableHttp({
|
|
228
|
+
configPath: "./synapsor.runner.json",
|
|
229
|
+
credentialResolver
|
|
230
|
+
});
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
The resolver receives only verified trusted context. Runner partitions pools
|
|
234
|
+
by source, access type, tenant, principal, and credential identity; an expired
|
|
235
|
+
or rotated lease is closed before reuse. Resolution failures are fail-closed,
|
|
236
|
+
and connection URLs are not logged or stored in contracts.
|
|
237
|
+
|
|
238
|
+
The stock CLI does not load arbitrary executable resolver modules. For direct
|
|
239
|
+
CLI operation, run one Runner process per tenant with a tenant-bound credential
|
|
240
|
+
in the normal `read_url_env`/`write_url_env` variables and keep
|
|
241
|
+
`credential_scope.mode` as `shared`. This gives the process no cross-tenant
|
|
242
|
+
credential to misuse.
|
|
243
|
+
|
|
244
|
+
## MySQL
|
|
245
|
+
|
|
246
|
+
MySQL does not provide a native equivalent to PostgreSQL RLS. Runner therefore
|
|
247
|
+
does not label session variables or application predicates as independent
|
|
248
|
+
database enforcement.
|
|
249
|
+
|
|
250
|
+
Use one or more of:
|
|
251
|
+
|
|
252
|
+
- tenant-bound read/write credentials;
|
|
253
|
+
- restricted tenant-aware views or stored procedures owned and reviewed by
|
|
254
|
+
your application;
|
|
255
|
+
- separate schemas or databases;
|
|
256
|
+
- one isolated Runner deployment per tenant.
|
|
257
|
+
|
|
258
|
+
Runner's trusted-context predicates still narrow model behavior, but a shared
|
|
259
|
+
MySQL credential remains application-level scope unless the database grants,
|
|
260
|
+
views, or deployment physically prevent cross-tenant access.
|
|
261
|
+
|
|
262
|
+
## Verification
|
|
263
|
+
|
|
264
|
+
The repository's disposable PostgreSQL proof covers:
|
|
265
|
+
|
|
266
|
+
```bash
|
|
267
|
+
corepack pnpm test:database-scope
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
It verifies an intentionally unscoped query is still denied by RLS, correct
|
|
271
|
+
tenant/principal access succeeds, cross-scope access fails, `WITH CHECK`
|
|
272
|
+
prevents row moves, pooled scope does not leak, guarded writeback and
|
|
273
|
+
compensation remain scoped, and unsafe roles/policies fail doctor.
|
|
@@ -10,11 +10,15 @@ Summary from the current dependency set:
|
|
|
10
10
|
|
|
11
11
|
| License | Package entries |
|
|
12
12
|
| --- | ---: |
|
|
13
|
-
| MIT |
|
|
13
|
+
| MIT | 200 |
|
|
14
14
|
| Apache-2.0 | 4 |
|
|
15
15
|
| BSD-3-Clause | 3 |
|
|
16
16
|
| BSD-2-Clause | 1 |
|
|
17
|
-
| ISC |
|
|
17
|
+
| ISC | 15 |
|
|
18
|
+
| `(BSD-3-Clause OR GPL-2.0)` | 1 |
|
|
19
|
+
| `(MIT AND Zlib)` | 1 |
|
|
20
|
+
| `0BSD` | 1 |
|
|
21
|
+
| `(MIT OR CC0-1.0)` | 1 |
|
|
18
22
|
|
|
19
23
|
Apache-2.0 dependency entries reported by pnpm:
|
|
20
24
|
|
package/docs/doctor.md
CHANGED
|
@@ -20,6 +20,23 @@ The default check validates:
|
|
|
20
20
|
- MCP tool boundary, including absence of raw SQL and commit tools;
|
|
21
21
|
- local store stats.
|
|
22
22
|
|
|
23
|
+
For a source configured with `database_scope.mode = "postgres_rls"`, doctor
|
|
24
|
+
also verifies RLS/FORCE status, effective-role bypass risk, applicable
|
|
25
|
+
operation policies, `USING`/`WITH CHECK`, and both configured trusted-setting
|
|
26
|
+
names. Hardened mode fails rather than silently falling back to
|
|
27
|
+
application-only predicates.
|
|
28
|
+
|
|
29
|
+
On a disposable or explicitly approved live target, add:
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
synapsor-runner doctor --config synapsor.runner.json --check-rls
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
This performs a read-only cross-tenant and cross-principal canary and checks
|
|
36
|
+
that transaction-local values do not survive the pooled transaction. See
|
|
37
|
+
[Database-Enforced Tenant And Principal
|
|
38
|
+
Scope](database-enforced-scope.md).
|
|
39
|
+
|
|
23
40
|
## App-Owned Handler Checks
|
|
24
41
|
|
|
25
42
|
For `http_handler` executors, add `--check-handlers`:
|
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
# Agent Effect Regression
|
|
2
|
+
|
|
3
|
+
Contract tests answer whether a reviewed capability boundary is still valid.
|
|
4
|
+
Effect regression answers a different question:
|
|
5
|
+
|
|
6
|
+
> Did a model, prompt, policy, capability, or application change alter the
|
|
7
|
+
> business effect the agent proposes?
|
|
8
|
+
|
|
9
|
+
Runner effect evaluation is provider-neutral and offline. Runner snapshots
|
|
10
|
+
existing replay evidence into a versioned fixture, then compares a result
|
|
11
|
+
exported by your agent harness. It does not invoke a model, query a source
|
|
12
|
+
database, approve a proposal, or apply a write.
|
|
13
|
+
|
|
14
|
+
## Create A Baseline
|
|
15
|
+
|
|
16
|
+
Start from an existing replay, proposal, or shadow-study case:
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
synapsor-runner effect fixture create \
|
|
20
|
+
--from-replay replay_wrp_... \
|
|
21
|
+
--request "Waive the late fee on invoice INV-3001" \
|
|
22
|
+
--contract ./synapsor.contract.json \
|
|
23
|
+
--capability-call billing.inspect_invoice \
|
|
24
|
+
--capability-call billing.propose_late_fee_waiver \
|
|
25
|
+
--store ./.synapsor/local.db \
|
|
26
|
+
--out ./effects/late-fee.json
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Use `--from-proposal wrp_...` or `--from-shadow-case shc_...` for the other
|
|
30
|
+
sources. The contract is required so Runner records its version and the
|
|
31
|
+
reviewed kept-out fields. Every `--capability-call` must exist in that
|
|
32
|
+
contract.
|
|
33
|
+
|
|
34
|
+
The fixture contains:
|
|
35
|
+
|
|
36
|
+
- the business request;
|
|
37
|
+
- trusted tenant and principal context from the stored proposal;
|
|
38
|
+
- bounded evidence and query-fingerprint snapshots already in the ledger;
|
|
39
|
+
- the permitted capability calls;
|
|
40
|
+
- expected target, business diff, policy result, conflict code, and outcome;
|
|
41
|
+
- the contract version and fields that must remain hidden;
|
|
42
|
+
- a digest that catches unreviewed fixture edits.
|
|
43
|
+
|
|
44
|
+
It contains no database URL, provider credential, or live source connection.
|
|
45
|
+
The secret and hidden-field checks fail closed while creating or loading it.
|
|
46
|
+
|
|
47
|
+
## Import A Provider-Neutral Result
|
|
48
|
+
|
|
49
|
+
Create a template:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
synapsor-runner effect result init \
|
|
53
|
+
--fixture ./effects/late-fee.json \
|
|
54
|
+
--out ./effects/late-fee.result.json
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Have your agent test harness populate this JSON after a propose-only run. The
|
|
58
|
+
format records:
|
|
59
|
+
|
|
60
|
+
- capability names and model-controlled arguments;
|
|
61
|
+
- trusted context supplied by the application outside those arguments;
|
|
62
|
+
- target, proposed diff, policy, conflict, and outcome category;
|
|
63
|
+
- fields observed by the harness;
|
|
64
|
+
- whether the harness made any new source read;
|
|
65
|
+
- whether the source database changed.
|
|
66
|
+
|
|
67
|
+
The adapter may use any model provider or application framework. Runner imports
|
|
68
|
+
the result; it does not require a proprietary LLM API.
|
|
69
|
+
|
|
70
|
+
The evaluation harness must disable apply/writeback. A result reporting a
|
|
71
|
+
source mutation always fails. A new source read also fails by default. Pass
|
|
72
|
+
`--allow-live-read` only when the test environment is explicitly disposable;
|
|
73
|
+
that flag never permits a write.
|
|
74
|
+
|
|
75
|
+
## Compare In CI
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
synapsor-runner effect run \
|
|
79
|
+
--fixture ./effects/late-fee.json \
|
|
80
|
+
--result ./effects/late-fee.result.json
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
`effect compare` is an alias. Both return a nonzero status when any reviewed
|
|
84
|
+
effect changes. Reports cover:
|
|
85
|
+
|
|
86
|
+
- capability calls and capability-surface expansion;
|
|
87
|
+
- trusted-context drift or model arguments selecting tenant/principal;
|
|
88
|
+
- target-object and business-diff changes;
|
|
89
|
+
- policy, conflict, outcome-category, and contract-version changes;
|
|
90
|
+
- kept-out field exposure;
|
|
91
|
+
- source mutation or an unapproved new read.
|
|
92
|
+
|
|
93
|
+
Use `--format json` or `--format junit` for CI:
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
synapsor-runner effect run \
|
|
97
|
+
--dataset ./effects/dataset.json \
|
|
98
|
+
--results-dir ./effects/results \
|
|
99
|
+
--format junit \
|
|
100
|
+
--out ./artifacts/effect-results.xml
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Dataset fixture paths are relative and cannot escape the dataset directory.
|
|
104
|
+
Each result is named `<fixture_id>.result.json`.
|
|
105
|
+
|
|
106
|
+
## Accept An Intentional Change
|
|
107
|
+
|
|
108
|
+
Runner never updates a baseline during `run` or `compare`. After reviewing an
|
|
109
|
+
intentional business change, accept it explicitly:
|
|
110
|
+
|
|
111
|
+
```bash
|
|
112
|
+
synapsor-runner effect accept \
|
|
113
|
+
--fixture ./effects/late-fee.json \
|
|
114
|
+
--result ./effects/late-fee.result.json \
|
|
115
|
+
--actor release-engineer@example.com \
|
|
116
|
+
--reason "Reviewed billing policy v2" \
|
|
117
|
+
--in-place \
|
|
118
|
+
--yes
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Use `--out ./effects/late-fee.v2.json` instead of `--in-place` to preserve the
|
|
122
|
+
old fixture. Acceptance appends actor, reason, timestamp, and before/after
|
|
123
|
+
baseline digests. It cannot waive:
|
|
124
|
+
|
|
125
|
+
- a mismatched fixture identity;
|
|
126
|
+
- source mutation;
|
|
127
|
+
- a new source read;
|
|
128
|
+
- trusted-context drift or model-controlled tenant/principal;
|
|
129
|
+
- hidden-field exposure.
|
|
130
|
+
|
|
131
|
+
Commit the accepted fixture and review it like application code.
|
|
132
|
+
|
|
133
|
+
## Boundaries
|
|
134
|
+
|
|
135
|
+
- This is not a general agent workflow engine.
|
|
136
|
+
- This does not execute a provider command or application code.
|
|
137
|
+
- This does not prove model quality outside the imported observations.
|
|
138
|
+
- This complements [`contract test`](contract-testing.md); a contract can
|
|
139
|
+
conform while an agent's proposed business effect still regresses.
|
|
140
|
+
- Replay remains read-only. Creating an effect fixture does not replay or
|
|
141
|
+
reapply the original write.
|
|
142
|
+
|
|
@@ -66,6 +66,13 @@ export DATABASE_URL="<postgres-or-mysql-read-url>"
|
|
|
66
66
|
|
|
67
67
|
Use a read-only credential for inspection and model-facing read/proposal tools.
|
|
68
68
|
|
|
69
|
+
The generated compatibility setup uses application-level trusted-scope
|
|
70
|
+
predicates. Before production-like multi-tenant use, choose and document one of
|
|
71
|
+
the modes in [Database-Enforced Tenant And Principal
|
|
72
|
+
Scope](database-enforced-scope.md): shared application scope, PostgreSQL RLS
|
|
73
|
+
plus Runner checks, or tenant-bound credentials/deployments. MySQL has no
|
|
74
|
+
native RLS equivalent.
|
|
75
|
+
|
|
69
76
|
### TLS notes for AWS RDS and other managed databases
|
|
70
77
|
|
|
71
78
|
If you see an error like:
|
package/docs/local-mode.md
CHANGED
|
@@ -30,9 +30,9 @@ Current local-mode foundation:
|
|
|
30
30
|
- local approved proposal to `synapsor.writeback-job.v1` job generation;
|
|
31
31
|
- guarded Postgres/MySQL writeback adapters for approved structured jobs.
|
|
32
32
|
|
|
33
|
-
|
|
33
|
+
Runtime requirement:
|
|
34
34
|
|
|
35
|
-
The
|
|
35
|
+
The stable package requires Node >= 22.5.0 because the local evidence/replay
|
|
36
36
|
ledger uses Node's `node:sqlite` runtime. The published package declares that
|
|
37
37
|
engine requirement and the CLI exits early with a clear message on older Node
|
|
38
38
|
versions. The Docker-backed demo remains the recommended path when you do not
|
|
@@ -184,7 +184,11 @@ npx -y -p @synapsor/runner synapsor-runner proposals reject wrp_123 \
|
|
|
184
184
|
|
|
185
185
|
Approval records the approver against the exact proposal hash/version. The proposal patch is immutable after creation.
|
|
186
186
|
|
|
187
|
-
Shadow-mode proposals are inspectable through `proposals show` and `replay
|
|
187
|
+
Shadow-mode proposals are inspectable through `proposals show` and `replay
|
|
188
|
+
show`, but `proposals approve` and `proposals writeback-job` reject them.
|
|
189
|
+
Shadow mode never mutates Postgres/MySQL or invokes an app-owned handler. To
|
|
190
|
+
compare agent behavior with explicit application/operator outcomes across a
|
|
191
|
+
pilot, use [Shadow Studies](shadow-studies.md).
|
|
188
192
|
|
|
189
193
|
## Browser review UI
|
|
190
194
|
|