enigma-memory 0.1.11 → 0.1.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -0
- package/apps/cli/bin/enigma.mjs +362 -10
- package/deploy/SIMULATION.md +152 -0
- package/deploy/docker-compose.local-production-simulation.yml +237 -0
- package/deploy/docker-compose.production.example.yml +19 -0
- package/deploy/kms-mock.mjs +64 -0
- package/deploy/nginx.local-production-simulation.conf +33 -0
- package/deploy/siem-mock.mjs +50 -0
- package/docs/benchmark-attestation-network.md +488 -0
- package/docs/benchmark-reproducibility.md +19 -2
- package/docs/blockchain-only-mechanisms.md +388 -0
- package/docs/client-connectors.md +512 -0
- package/docs/demo-proof-network.md +275 -0
- package/docs/developer-ecosystem.md +47 -4
- package/docs/developer-proof-quickstart.md +325 -0
- package/docs/enigma-memory-ready-conformance.md +376 -0
- package/docs/enterprise-proof-control-plane.md +365 -0
- package/docs/install-anywhere.md +517 -0
- package/docs/market-category-narrative.md +398 -0
- package/docs/memory-drive-health-model.md +649 -0
- package/docs/memory-drive-strategy.md +458 -0
- package/docs/memory-passport-standard.md +445 -0
- package/docs/novelty-invention-candidates.md +161 -0
- package/docs/privacy-ledger-model.md +229 -0
- package/docs/proof-network-build-notes.md +240 -0
- package/docs/proof-network-claim-boundaries.md +318 -0
- package/docs/proof-network-dashboard-spec.md +773 -0
- package/docs/proof-network-glossary.md +27 -0
- package/docs/proof-network-launch-plan.md +421 -0
- package/docs/proof-network-operator-protocol.md +432 -0
- package/docs/proof-network-roadmap.md +431 -0
- package/docs/proof-network-test-plan.md +216 -0
- package/docs/proof-network-threat-model.md +373 -0
- package/docs/proof-network.md +257 -0
- package/docs/sdk-api.md +132 -10
- package/docs/solana-devnet-acceptance.md +226 -0
- package/docs/solana-proof-rail.md +453 -0
- package/examples/ci/github-actions.yml +6 -3
- package/examples/proof-network-anchor.json +37 -0
- package/examples/proof-network-attestation.json +35 -0
- package/examples/proof-network-grant.json +27 -0
- package/examples/proof-network-packet.json +71 -0
- package/package.json +42 -3
- package/packages/mcp-server/src/index.js +1 -1
- package/packages/proof-network/src/index.js +570 -0
- package/scripts/build-hosted-api-key-lifecycle.mjs +1 -1
- package/scripts/build-hosted-customer-lifecycle.mjs +1 -1
- package/scripts/build-installer-assets.mjs +1 -1
- package/scripts/build-proof-network-packet.mjs +213 -0
- package/scripts/run-standard-memory-benchmarks.mjs +1 -1
- package/scripts/simulate-production-env.mjs +210 -0
- package/scripts/verify-registry-install.mjs +1 -0
- package/scripts/wait-for-backend-ready.mjs +101 -0
- package/specs/goal-completion-audit-v1.schema.json +1 -0
- package/specs/proof-network-anchor-batch-v1.schema.json +125 -0
- package/specs/proof-network-benchmark-attestation-v1.schema.json +103 -0
- package/specs/proof-network-capability-grant-v1.schema.json +132 -0
- package/specs/proof-network-packet-v1.schema.json +171 -0
|
@@ -0,0 +1,229 @@
|
|
|
1
|
+
# Privacy ledger model
|
|
2
|
+
|
|
3
|
+
Enigma is the private memory controller for AI. The privacy ledger model separates private memory operations from public proof coordination with two ledgers: a local private ledger and a public proof ledger. Solana is optional proof, permission, and settlement infrastructure for opaque commitments; it is not a memory database.
|
|
4
|
+
|
|
5
|
+
A public proof artifact can show that a commitment, root, grant, revocation, attestation, settlement ref, or packet hash existed in a declared envelope. It must not reveal raw memory, prompts, transcripts, completions, embeddings, policy bodies, access-control bodies, provider responses, private identities, tenant names, API keys, private keys, seed phrases, or credential-bearing URLs. It also must not claim provider deletion, model forgetting, live SaaS operation, live Solana deployment, compliance certification, ROI, or benchmark superiority.
|
|
6
|
+
|
|
7
|
+
## 1. Ledger split
|
|
8
|
+
|
|
9
|
+
| Layer | Location | Primary reader | Purpose | Publication rule |
|
|
10
|
+
| --- | --- | --- | --- | --- |
|
|
11
|
+
| Local private ledger | User/operator-controlled Enigma vault, BYOC store, on-prem store, or offline audit bundle | Memory owner, authorized operator, confidential auditor | Complete event history for Enigma-mediated memory creation, retrieval, context-pack export, policy decision, grant, revocation, tombstone, compaction, and verifier activity | Never public by default. Contains private payload refs and may contain sensitive operational metadata. |
|
|
12
|
+
| Local proof ledger | Local proof artifacts and proof packets | Operator, reviewer, CI/verifier, confidential auditor | Public-safe receipts derived from the private ledger: roots, refs, counts, policy hashes, capability commitments, leakage-scan results, verifier output | Shareable only after private-payload checks pass. |
|
|
13
|
+
| Public proof ledger | Optional Solana anchor, registry, release packet, public repo artifact, or third-party verifier record | External observer, partner verifier, ecosystem auditor | Durable commitment to a batch root, grant/ref, revocation/nullifier, benchmark attestation, settlement ref, or packet hash | Hashes/roots/refs/counts/signatures only. No raw memory, prompts, transcripts, completions, embeddings, ACL bodies, tenant names, provider responses, secrets, wallet seed material, or private local paths. |
|
|
14
|
+
|
|
15
|
+
The private ledger is the source of operational truth. The public proof ledger is a derived commitment layer. If they disagree, the public layer cannot invent context; the operator must regenerate the proof from an approved local snapshot or mark the proof unavailable.
|
|
16
|
+
|
|
17
|
+
## 2. Field placement
|
|
18
|
+
|
|
19
|
+
### 2.1 Local private ledger fields
|
|
20
|
+
|
|
21
|
+
The private ledger may hold fields needed for local control, replay, and confidential audit:
|
|
22
|
+
|
|
23
|
+
| Field family | Examples | Notes |
|
|
24
|
+
| --- | --- | --- |
|
|
25
|
+
| Event identity | Private event id, monotonic sequence, local clock reading, prior event hash | Public packets should expose only commitments, rounded timestamps, or sequence ranges when needed. |
|
|
26
|
+
| Memory material | Memory body, source excerpt, user note, imported record, derived summary, context-pack body | Never copied into public proof artifacts. |
|
|
27
|
+
| Retrieval material | Query text, selected memory ids, omitted memory ids, scores, scorer internals, embedding/vector refs | Public packets may expose selected/omitted roots and counts, not queries, scores, embeddings, or selected text. |
|
|
28
|
+
| Policy material | Policy body, rule text, private subject attributes, purpose details, data labels, legal-hold rationale | Public packets use policy hashes, rule refs, decision refs, and allow/deny status only. |
|
|
29
|
+
| Access control | Private ACL body, internal group membership, operator identity, account mapping | Public grants use issuer refs, subject refs, capability names, scope commitments, expiry, nonces, and signatures. |
|
|
30
|
+
| Provider/tool material | Provider request, response, tool payload, transcript, completion, error body | Public packets use provider/tool refs only if non-identifying and approved. |
|
|
31
|
+
| Commitment openings | Per-record random nonce, epoch salt id, private salt/HMAC key reference, Merkle proof path, canonicalization version | Open only to authorized auditors under a confidential review workflow. |
|
|
32
|
+
| Operations | Tombstone reason, compaction trace, import/export logs, SIEM mapping, support notes | Public packets use reason refs and counts; no free-text incident detail. |
|
|
33
|
+
|
|
34
|
+
Local does not mean unconstrained. Local ledger stores and exports still need least privilege, encryption where available, secret redaction, retention limits, and a release review before anything leaves the boundary.
|
|
35
|
+
|
|
36
|
+
### 2.2 Local proof ledger fields
|
|
37
|
+
|
|
38
|
+
The local proof ledger is the public-safe derivative that can be shared after validation:
|
|
39
|
+
|
|
40
|
+
| Artifact family | Public-safe fields | Excluded fields |
|
|
41
|
+
| --- | --- | --- |
|
|
42
|
+
| Anchor batch | `schema`, `batch_id`, `created_at`, `anchor_root`, `root_count`, root kinds, root hashes, root counts, `public_refs`, `privacy_boundary`, `solana_anchor.payload_hash`, `opaque_payload`, signer refs, signatures, `transaction_submitted:false`, `raw_memory_on_chain:false` | Raw memory, prompts, transcripts, completions, embeddings, private paths, tenant names, provider responses, wallet private keys, seed phrases, RPC credentials. |
|
|
43
|
+
| Capability grant | Grant id/hash, issuer ref, subject ref, capability name, scope commitment/ref, issued-at, expires-at, nonce commitment, audience ref, policy hash, signer ref, signature | Private ACL body, human tenant or account name, private subject attributes, raw policy text, bearer tokens. |
|
|
44
|
+
| Capability revocation | Revocation id/hash, grant ref/hash, scope ref/hash, nullifier commitment, reason ref, effective time, issuer/signer ref | Private incident text, subject identity details, ACL body, customer name. |
|
|
45
|
+
| Benchmark attestation | Report hash, dataset ref/hash, runner ref/hash, package ref/hash, metric digest/count, environment ref, leakage-scan result, claim boundaries | Dataset rows, questions, answers, prompts, completions, provider responses, hidden challenge records. |
|
|
46
|
+
| Proof packet | Packet id, packet root, artifact hashes, artifact refs, artifact counts, leakage scan, claim boundaries, verifier refs | Any nested private artifact body or raw operational log. |
|
|
47
|
+
| Settlement reference | Quote/receipt hash, service ref, amount/currency bounds if approved, settlement ref, signer refs | Raw workload, invoice secret, customer name, provider billing detail, ROI/cost-savings claims. |
|
|
48
|
+
|
|
49
|
+
### 2.3 Public proof ledger fields
|
|
50
|
+
|
|
51
|
+
A public chain or registry should store the smallest useful commitment:
|
|
52
|
+
|
|
53
|
+
- domain separator, such as `enigma.proof_network.anchor_batch.v1`;
|
|
54
|
+
- schema version and canonicalization ref;
|
|
55
|
+
- batch root, artifact hash, packet hash, or settlement ref hash;
|
|
56
|
+
- root kind and count when necessary for verifier routing;
|
|
57
|
+
- opaque public refs approved for publication;
|
|
58
|
+
- signer or program refs that do not identify private tenants;
|
|
59
|
+
- optional memo ref that points to a public-safe packet, not private evidence;
|
|
60
|
+
- explicit boundary flags when represented in the artifact: `transaction_submitted:false` for local plans and `raw_memory_on_chain:false` for chain-bound artifacts.
|
|
61
|
+
|
|
62
|
+
The public proof ledger must not include direct hashes of low-entropy private values such as names, emails, short prompts, small ACL bodies, local usernames, account labels, or private dataset names. Those values are guessable even when hashed.
|
|
63
|
+
|
|
64
|
+
## 3. Commitments and addresses
|
|
65
|
+
|
|
66
|
+
Commitments bind private state without publishing it. Every commitment should be domain-separated, canonicalized, and selected for the entropy of the underlying value.
|
|
67
|
+
|
|
68
|
+
### 3.1 Canonical input
|
|
69
|
+
|
|
70
|
+
Every committed object should first be converted to a deterministic representation:
|
|
71
|
+
|
|
72
|
+
1. Select only the fields intended for that commitment domain.
|
|
73
|
+
2. Normalize field order, encodings, timestamps, and absent/null values.
|
|
74
|
+
3. Prefix a domain string such as `enigma.memory_event.v1`, `enigma.capability_scope.v1`, `enigma.grant_nullifier.v1`, or `enigma.anchor_batch.v1`.
|
|
75
|
+
4. Include schema version and canonicalization version.
|
|
76
|
+
5. Hash or HMAC the canonical bytes.
|
|
77
|
+
|
|
78
|
+
Domain separation prevents a digest from one context from being replayed as a valid digest in another context.
|
|
79
|
+
|
|
80
|
+
### 3.2 High-entropy public refs
|
|
81
|
+
|
|
82
|
+
A plain `sha256:` digest is acceptable for material that is already high entropy or already public-safe, such as a random packet id, generated nonce, package tarball hash, public release artifact, or canonical public-safe proof packet.
|
|
83
|
+
|
|
84
|
+
```text
|
|
85
|
+
sha256(domain || canonical_public_safe_bytes)
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Do not use a plain hash for short private strings, tenant labels, email addresses, human-readable prompts, ACL snippets, account labels, or private dataset names.
|
|
89
|
+
|
|
90
|
+
### 3.3 Salted private commitments
|
|
91
|
+
|
|
92
|
+
Salted commitments are useful when a verifier may later receive the salt through a private audit process:
|
|
93
|
+
|
|
94
|
+
```text
|
|
95
|
+
commitment = sha256(domain || record_nonce_128bit_or_more || canonical_private_tuple)
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Rules:
|
|
99
|
+
|
|
100
|
+
- Generate `record_nonce` with at least 128 bits of randomness.
|
|
101
|
+
- Store the nonce only in the local private ledger or a private audit package.
|
|
102
|
+
- Publish the digest and, if necessary, an opaque salt ref; do not publish the salt in the public ledger.
|
|
103
|
+
- Open salts only for selected records and authorized auditors.
|
|
104
|
+
- Do not reveal neighboring record nonces when opening one record.
|
|
105
|
+
|
|
106
|
+
Salted commitments are weaker than HMAC commitments for low-entropy public values because an exposed salt enables dictionary checks.
|
|
107
|
+
|
|
108
|
+
### 3.4 HMAC commitments
|
|
109
|
+
|
|
110
|
+
For stable public refs where dictionary resistance matters, prefer keyed commitments:
|
|
111
|
+
|
|
112
|
+
```text
|
|
113
|
+
commitment = hmac_sha256(epoch_commitment_key, domain || canonical_tuple || optional_record_nonce)
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Rules:
|
|
117
|
+
|
|
118
|
+
- Keep `epoch_commitment_key` local or inside customer-controlled KMS/BYOK custody.
|
|
119
|
+
- Public artifacts may include a non-sensitive `key_ref` or `epoch_ref`, never the key.
|
|
120
|
+
- Rotate keys by epoch, ledger, deployment, or customer boundary to reduce cross-batch linkability.
|
|
121
|
+
- Derive separate keys for memory addresses, content commitments, capability nullifiers, benchmark private refs, and settlement refs.
|
|
122
|
+
- Never reuse HMAC commitment keys for signatures, encryption, API authentication, provider credentials, or wallet operations.
|
|
123
|
+
|
|
124
|
+
A public HMAC commitment proves possession of a keyed digest, not the truth, completeness, legality, deletion, or business value of the underlying memory.
|
|
125
|
+
|
|
126
|
+
### 3.5 Merkle roots and proof paths
|
|
127
|
+
|
|
128
|
+
The local proof ledger should aggregate commitments into Merkle roots before publication:
|
|
129
|
+
|
|
130
|
+
- leaf = domain-separated commitment for one event, grant, revocation, attestation, settlement ref, or packet;
|
|
131
|
+
- internal node = hash of domain, left child, right child, tree version;
|
|
132
|
+
- root = public-safe commitment to the batch;
|
|
133
|
+
- proof path = private or auditor-scoped unless every leaf in the path is already public-safe.
|
|
134
|
+
|
|
135
|
+
Merkle roots support auditor sampling: the public ledger can hold one root, while a confidential auditor can inspect selected private records, nonces, and proof paths without exposing the full ledger.
|
|
136
|
+
|
|
137
|
+
## 4. Batch cadence
|
|
138
|
+
|
|
139
|
+
Batching is both a cost control and a privacy control. Enigma should not publish one public event per sensitive memory operation unless the operator has explicitly accepted the metadata leakage.
|
|
140
|
+
|
|
141
|
+
| Cadence | Use when | Privacy effect | Caution |
|
|
142
|
+
| --- | --- | --- | --- |
|
|
143
|
+
| Per operation, local only | Append to the private ledger immediately after Enigma-mediated action | Preserves replayability without public timing leakage | Local clock and sequence still need tamper-evident chaining. |
|
|
144
|
+
| Per event, public | A counterparty needs immediate evidence for one bounded operation | Minimal; timing and count are easy to correlate | Highest freshness, highest leakage; require explicit approval. |
|
|
145
|
+
| Short rolling batch | Interactive workflows need near-real-time proof refs | Hides individual event timing inside a small group | Exact cadence can reveal workload rhythm; use jitter and minimum batch size. |
|
|
146
|
+
| Fixed window | Routine memory operations can wait for scheduled anchoring | Makes timing less tied to individual actions | Delays public evidence. |
|
|
147
|
+
| Threshold batch | Publish only after enough events accumulate | Avoids revealing sparse activity as a single event | Public timing may reveal threshold crossings. |
|
|
148
|
+
| Manual release batch | High-risk environments require human review | Strongest review control before publication | Lowest automation and freshness. |
|
|
149
|
+
| Revocation-sensitive batch | Grant revocation/nullifier evidence needs faster propagation | Reduces acceptance window for revoked capabilities | Immediate publication can reveal incident timing; use reason refs, not incident text. |
|
|
150
|
+
| Audit/export batch | Produce packet for a reviewer or customer checkpoint | Keeps publication scoped to an approved review moment | Export bundle must pass leakage scan and claim-boundary review. |
|
|
151
|
+
|
|
152
|
+
Recommended default: append private events immediately, derive local proof roots continuously, and publish public anchor batches only at approved checkpoints with fixed windows or manual release batches, minimum batch sizes, rounded timestamps, and opaque refs.
|
|
153
|
+
|
|
154
|
+
## 5. Leakage risks and mitigations
|
|
155
|
+
|
|
156
|
+
Even hash-only artifacts can leak information through structure and timing.
|
|
157
|
+
|
|
158
|
+
| Risk | How it leaks | Mitigation |
|
|
159
|
+
| --- | --- | --- |
|
|
160
|
+
| Dictionary attack on hashes | Observer hashes likely tenant names, emails, prompts, ACL strings, or dataset names and compares outputs | Use HMAC or high-entropy random nonces; never publish plain hashes of low-entropy private values. |
|
|
161
|
+
| Cross-batch linkability | Same subject, scope, policy, memory ref, signer, or settlement ref appears in many public artifacts | Rotate epoch keys, use scoped refs, avoid stable human-readable ids, and aggregate into batches. |
|
|
162
|
+
| Timing correlation | Anchor time matches a customer incident, model run, support ticket, benchmark window, or revocation | Round timestamps, use batch windows, add operational jitter, and avoid incident-specific public reason text. |
|
|
163
|
+
| Count leakage | Root counts reveal customer size, workload spikes, benchmark set size, deletion volume, or support load | Publish coarse counts where possible, bucket counts for public artifacts, or keep exact counts in confidential packets. |
|
|
164
|
+
| Ref leakage | `public_refs` reveal package, dataset, customer, connector, branch, path, or internal project names | Use approved opaque refs; forbid tenant names, local paths, emails, account ids, and private dataset labels. |
|
|
165
|
+
| Signature/key linkage | Same signer ref links multiple customers or deployments | Use deployment-scoped signer refs, key rotation, and customer-controlled key custody where appropriate. |
|
|
166
|
+
| Solana account linkage | Wallet address, memo, program account, or explorer trail ties proofs to a tenant/operator | Use operator-approved public refs only; do not include seed phrases, private keys, credentialed RPC URLs, tenant labels, or account names. |
|
|
167
|
+
| Auditor overexposure | Audit bundle includes full private ledger when sampling would suffice | Provide sampled openings, Merkle paths, verifier output, and scoped extracts; keep raw payload access need-to-know. |
|
|
168
|
+
| Unknown-field smuggling | A public artifact adds harmless-looking free-text or metadata fields that contain private content | Schemas must be allowlist-only and reject unknown fields. |
|
|
169
|
+
| Public verifier overclaim | A verifier treats a valid root as proof of content truth, deletion, compliance, live settlement, or benchmark superiority | Carry claim boundaries with every packet and repeat that roots prove commitments to snapshots, not external facts. |
|
|
170
|
+
|
|
171
|
+
## 6. Auditor views
|
|
172
|
+
|
|
173
|
+
Auditors should receive the narrowest view that answers the review question.
|
|
174
|
+
|
|
175
|
+
| View | Audience | Contents | Not included |
|
|
176
|
+
| --- | --- | --- | --- |
|
|
177
|
+
| Public verifier view | Anyone validating a packet, release artifact, or optional anchor payload | Schema id, artifact type, hashes, roots, refs, counts, coarse timestamps, signatures, verifier status | Private preimages, salts, memory, prompts, transcripts, completions, embeddings, provider responses, policy bodies, private identities. |
|
|
178
|
+
| Counterparty proof view | Party verifying a bounded grant, revocation, attestation, or settlement ref | Public verifier view plus the specific private disclosure approved for that counterparty, if any | Unrelated local ledger rows, unrelated subjects, broad history, secrets. |
|
|
179
|
+
| Confidential audit view | Approved auditor under private review terms | Selected local ledger rows, selected salts or preimages when required, mapping from private rows to public commitments, release-gate evidence, verifier transcript hashes | Secrets not needed for audit, unrelated records, provider response bodies unless explicitly in scope and permitted. |
|
|
180
|
+
| Operator incident view | Internal operator investigating a failed gate or suspected leak | Local diagnostic context, failed artifact, rejection reasons, and remediation notes | Material beyond the incident scope or any unredacted export outside the local boundary. |
|
|
181
|
+
|
|
182
|
+
An auditor view is not automatically publishable. Anything derived from a private audit view must pass the same public proof release gates before it can become a public artifact.
|
|
183
|
+
|
|
184
|
+
A confidential auditor should be able to recompute selected commitments and confirm they roll up to the public root while seeing no records outside the approved audit scope.
|
|
185
|
+
|
|
186
|
+
## 7. End-to-end flow
|
|
187
|
+
|
|
188
|
+
1. Enigma mediates a memory operation locally.
|
|
189
|
+
2. The private ledger appends an event containing private operational detail and a chained event hash.
|
|
190
|
+
3. The commitment engine derives domain-separated salted or HMAC commitments for the event fields that need public accountability.
|
|
191
|
+
4. The local proof ledger builds roots over commitments and emits public-safe artifacts.
|
|
192
|
+
5. A leakage scanner rejects forbidden field names, secret-looking values, raw payloads, private paths, tenant names, prompts, transcripts, completions, embeddings, provider responses, and keys.
|
|
193
|
+
6. A proof packet records artifact hashes, roots, refs, counts, verifier refs, and claim boundaries.
|
|
194
|
+
7. At an approved cadence, an anchor batch commits the packet/root set and remains local with `transaction_submitted:false` unless a separate operator-controlled submission workflow is approved.
|
|
195
|
+
8. Optional public anchoring publishes only the compact commitment, never the private ledger or its openings.
|
|
196
|
+
9. Auditors verify either the public-safe artifact alone or a confidential sample opening against the same root.
|
|
197
|
+
|
|
198
|
+
## 8. Privacy release gates
|
|
199
|
+
|
|
200
|
+
Before any public proof ledger artifact, documentation example, release packet, or optional Solana-ready payload leaves the local boundary, all gates must pass:
|
|
201
|
+
|
|
202
|
+
1. **Field allowlist gate**: every public field is in the approved schema and every unknown field is rejected.
|
|
203
|
+
2. **Private-payload gate**: no raw memory, prompt, transcript, completion, embedding, provider response, private policy body, ACL body, tenant name, user name, customer name, local path, credential, private key, seed phrase, or credential-bearing URL appears at any depth.
|
|
204
|
+
3. **Commitment gate**: public refs are hashes, HMAC commitments, salted commitments without public salts, roots, nullifiers, or opaque ids; raw preimages are absent.
|
|
205
|
+
4. **Domain-separation gate**: commitments identify their purpose or version so values from one domain cannot be replayed as another domain.
|
|
206
|
+
5. **Metadata gate**: timestamps, counts, refs, sequence ids, signer refs, and batch windows are reviewed for correlation risk.
|
|
207
|
+
6. **Batching gate**: per-event publication is rejected unless freshness is required and the metadata risk is explicitly accepted.
|
|
208
|
+
7. **Auditor-view gate**: private audit material is separated from public verifier material, and only the public verifier material is exported.
|
|
209
|
+
8. **Solana boundary gate**: optional chain payloads contain public-safe roots/refs only and clearly distinguish local planning from submitted transactions.
|
|
210
|
+
9. **Claim boundary gate**: copy does not imply provider deletion, model forgetting, legal compliance, chain finality, live hosted operation, ROI, token value, or universal benchmark superiority without separate approved evidence.
|
|
211
|
+
10. **Human review gate**: a reviewer confirms the artifact is public-safe before publication or anchor submission.
|
|
212
|
+
|
|
213
|
+
Failure is closed: if a gate cannot be evaluated, the artifact remains local and unpublished.
|
|
214
|
+
|
|
215
|
+
## 9. Publication checklist
|
|
216
|
+
|
|
217
|
+
Before any privacy-ledger artifact leaves the operator boundary, confirm:
|
|
218
|
+
|
|
219
|
+
- [ ] The artifact contains no raw memory, prompts, transcripts, completions, embeddings, ACL bodies, tenant names, provider responses, secrets, private keys, seed phrases, API keys, credentialed URLs, or local identifying paths.
|
|
220
|
+
- [ ] Low-entropy private values are protected with HMAC or high-entropy nonces, not plain hashes.
|
|
221
|
+
- [ ] Public refs are opaque and approved for publication.
|
|
222
|
+
- [ ] Counts and timestamps are no more precise than the review use case requires.
|
|
223
|
+
- [ ] Batch cadence does not reveal sensitive operational timing.
|
|
224
|
+
- [ ] Commitment domains, schema versions, and canonicalization versions are recorded.
|
|
225
|
+
- [ ] HMAC keys, salts, and nonces remain local except for scoped confidential audit openings.
|
|
226
|
+
- [ ] Solana or other public rails carry only hashes, roots, refs, counts, and signatures.
|
|
227
|
+
- [ ] Claim boundaries state that the proof does not establish provider deletion, model forgetting, compliance certification, benchmark superiority, ROI, token value, live hosted operation, or live chain settlement.
|
|
228
|
+
|
|
229
|
+
Keep private memory useful locally and public proof artifacts boring publicly. Public ledgers should be easy to verify, hard to correlate, and impossible to use as a source of private memory reconstruction.
|
|
@@ -0,0 +1,240 @@
|
|
|
1
|
+
# Proof Network build notes
|
|
2
|
+
|
|
3
|
+
These notes are for future engineers extending Enigma's privacy-preserving Proof Network into a Solana program or hosted verifier. The 0.1.13 scope is local-only: create and verify public-safe proof artifacts, prepare opaque root batches for future chain anchoring, and never submit transactions or write private memory material into proof payloads.
|
|
4
|
+
|
|
5
|
+
Paths below are relative to `enigma/`.
|
|
6
|
+
|
|
7
|
+
## Current implementation inventory
|
|
8
|
+
|
|
9
|
+
| Surface | Current file | Notes for builders |
|
|
10
|
+
| --- | --- | --- |
|
|
11
|
+
| Pure package API | `packages/proof-network/src/index.js` | Exports constructors, validators, schema constants, `assertNoPrivateProofPayload`, and `sha256Json`. Keep it pure: no filesystem, network, provider SDK, Solana RPC, subprocess, or mutable runtime state. |
|
|
12
|
+
| Package export | `package.json` | Includes `packages/proof-network/src/` in `files` and exports `./proof-network`. |
|
|
13
|
+
| CLI commands | `apps/cli/bin/enigma.mjs` | Implements `enigma chain anchor`, `grant`, `revoke`, `attest`, and `verify` as local planning/verification commands. The help text states these commands do not submit Solana transactions or put raw memory on-chain. |
|
|
14
|
+
| Packet builder | `scripts/build-proof-network-packet.mjs` | Builds a public-safe packet from hashes/refs and hashes benchmark report files without copying their body or path into the packet. |
|
|
15
|
+
| Schemas | `specs/proof-network-anchor-batch-v1.schema.json`, `specs/proof-network-capability-grant-v1.schema.json`, `specs/proof-network-benchmark-attestation-v1.schema.json`, `specs/proof-network-packet-v1.schema.json` | Add `specs/proof-network-capability-revocation-v1.schema.json` before release so every artifact schema has a JSON Schema. |
|
|
16
|
+
| Tests | `test/enigma-proof-network.test.mjs`, `test/enigma-chain-cli.test.mjs` | Keep fixtures public-safe. Add cases when any field, schema, or CLI flag changes. |
|
|
17
|
+
| Root product docs | `../docs/proof-network.md`, `../docs/proof-network-faq.md`, `../docs/proof-network-threat-model.md`, `../docs/proof-network-build-notes.md` | Keep positioning bounded to local proof generation, local verification, Solana-ready payload shape, and public-safe commitments. |
|
|
18
|
+
|
|
19
|
+
Do not touch website, Cloudflare, live infrastructure, publishing, account creation, or external provider integrations while building this layer.
|
|
20
|
+
|
|
21
|
+
## Scope boundary
|
|
22
|
+
|
|
23
|
+
Allowed public evidence:
|
|
24
|
+
|
|
25
|
+
- exact schema ids
|
|
26
|
+
- artifact ids
|
|
27
|
+
- `sha256:` digests
|
|
28
|
+
- Merkle roots
|
|
29
|
+
- opaque refs
|
|
30
|
+
- counts
|
|
31
|
+
- timestamps
|
|
32
|
+
- reviewed public keys or signature refs
|
|
33
|
+
- boolean claim-boundary flags
|
|
34
|
+
- transaction planning metadata with `transaction_submitted:false`
|
|
35
|
+
|
|
36
|
+
Forbidden public evidence: raw memory, prompts, transcripts, completions, embeddings, ACL bodies, tenant names, private keys, API keys, seed phrases, provider responses, customer identifiers, private local paths, and raw benchmark rows. The Proof Network proves commitments to private memory workflows; it does not publish those workflows.
|
|
37
|
+
|
|
38
|
+
## Public artifact contracts
|
|
39
|
+
|
|
40
|
+
Future Solana and hosted-verifier code should switch on exact schema id, not filename. Preserve the package's current field names unless there is a coordinated clean cutover across schemas, CLI, tests, and docs.
|
|
41
|
+
|
|
42
|
+
### `enigma.proof_network.anchor_batch.v1`
|
|
43
|
+
|
|
44
|
+
Purpose: group public commitments into one Solana-ready opaque anchoring payload.
|
|
45
|
+
|
|
46
|
+
Current package fields include:
|
|
47
|
+
|
|
48
|
+
- `schema:"enigma.proof_network.anchor_batch.v1"`
|
|
49
|
+
- `generated_at`
|
|
50
|
+
- `anchor_ref`
|
|
51
|
+
- `chain`
|
|
52
|
+
- `cluster_ref`
|
|
53
|
+
- `commitment_count`
|
|
54
|
+
- `commitment_root`
|
|
55
|
+
- `commitments`: objects containing public `kind`, `root`, and optional `ref`
|
|
56
|
+
- `solana_ready_anchor.payload_hash`
|
|
57
|
+
- `solana_ready_anchor.account_seed`
|
|
58
|
+
- `solana_ready_anchor.instruction_ref`
|
|
59
|
+
- `solana_ready_anchor.opaque_payload_only:true`
|
|
60
|
+
- `anchor_batch_id`
|
|
61
|
+
- `anchor_batch_hash`
|
|
62
|
+
- `transaction_submitted:false`
|
|
63
|
+
- `raw_memory_on_chain:false`
|
|
64
|
+
- `provider_deletion_claim:false`
|
|
65
|
+
- `model_forgetting_claim:false`
|
|
66
|
+
- `hosted_saas_claim:false`
|
|
67
|
+
|
|
68
|
+
The Solana program should only need the compact payload hash and public metadata for schema/version/count auditing. It should not need ACL bodies, memory plaintext, benchmark reports, or provider outputs.
|
|
69
|
+
|
|
70
|
+
### `enigma.proof_network.capability_grant.v1`
|
|
71
|
+
|
|
72
|
+
Purpose: represent a scoped, public-safe grant without revealing the underlying tenant, memory, ACL, or account body.
|
|
73
|
+
|
|
74
|
+
Current package fields include `issued_at`, `expires_at`, `grant_ref`, `issuer_ref`, `subject_ref`, `audience_ref`, `scopes`, `resource_root`, `resource_roots`, `max_uses`, `nonce_hash`, `signature_ref`, `capability_grant_id`, `capability_grant_hash`, and all safety flags listed above.
|
|
75
|
+
|
|
76
|
+
Use refs and hashes for scopes. Never embed policy documents that contain names, tenant identifiers, access-control bodies, or business-sensitive terms.
|
|
77
|
+
|
|
78
|
+
### `enigma.proof_network.capability_revocation.v1`
|
|
79
|
+
|
|
80
|
+
Purpose: revoke a grant with a public nullifier/commitment.
|
|
81
|
+
|
|
82
|
+
Current package fields include `revoked_at`, `revocation_ref`, `grant_id`, `grant_hash`, `issuer_ref`, `reason_ref`, `nullifier_root`, `signature_ref`, `capability_revocation_id`, `capability_revocation_hash`, and all safety flags listed above.
|
|
83
|
+
|
|
84
|
+
The nullifier must not be reversible to a tenant name, account id, private ACL, or raw grant body. Add the missing JSON Schema file for this artifact before release.
|
|
85
|
+
|
|
86
|
+
### `enigma.proof_network.benchmark_attestation.v1`
|
|
87
|
+
|
|
88
|
+
Purpose: attest benchmark evidence without publishing raw datasets, prompts, completions, judge responses, provider outputs, or report bodies.
|
|
89
|
+
|
|
90
|
+
Current package fields include `attested_at`, `benchmark_ref`, `dataset_ref`, `runner_ref`, `package_ref`, `report_hash`, `metric_root`, `metric_roots`, `sample_count`, `run_count`, `signature_ref`, `benchmark_attestation_id`, `benchmark_attestation_hash`, and all safety flags listed above.
|
|
91
|
+
|
|
92
|
+
If a CLI or script accepts a report file, hash the file locally and emit only the digest plus public refs unless the report has already been reviewed as public-safe.
|
|
93
|
+
|
|
94
|
+
### `enigma.proof_network.packet.v1`
|
|
95
|
+
|
|
96
|
+
Purpose: bundle supported proof-network artifacts into one verifier-ready packet.
|
|
97
|
+
|
|
98
|
+
Current package fields include `created_at`, `packet_ref`, `artifact_count`, `artifact_root`, `artifact_hashes`, `artifacts`, `proof_network_packet_id`, `proof_network_packet_hash`, and all safety flags listed above.
|
|
99
|
+
|
|
100
|
+
Packets must recursively validate nested artifacts. A packet hash must not hide unsupported schemas, invalid counts, or private nested fields. Nested packets are rejected by current validation.
|
|
101
|
+
|
|
102
|
+
## Pure package API contract
|
|
103
|
+
|
|
104
|
+
`packages/proof-network/src/index.js` exports:
|
|
105
|
+
|
|
106
|
+
```js
|
|
107
|
+
assertNoPrivateProofPayload(value)
|
|
108
|
+
sha256Json(value)
|
|
109
|
+
createProofNetworkAnchorBatch(input = {})
|
|
110
|
+
validateProofNetworkAnchorBatch(batch)
|
|
111
|
+
createCapabilityGrant(input = {})
|
|
112
|
+
validateCapabilityGrant(grant)
|
|
113
|
+
createCapabilityRevocation(input = {})
|
|
114
|
+
validateCapabilityRevocation(revocation)
|
|
115
|
+
createBenchmarkAttestation(input = {})
|
|
116
|
+
validateBenchmarkAttestation(attestation)
|
|
117
|
+
createProofNetworkPacket(input = {})
|
|
118
|
+
validateProofNetworkPacket(packet)
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Implementation rules:
|
|
122
|
+
|
|
123
|
+
1. Keep every function pure: no filesystem, network, environment reads, provider SDKs, Solana RPC, subprocesses, or mutable module state.
|
|
124
|
+
2. Constructors may accept caller-provided ids and timestamps. If defaults exist, keep them deterministic or label them local artifact metadata only.
|
|
125
|
+
3. `sha256Json` must canonicalize object keys before hashing so equivalent JSON yields the same digest.
|
|
126
|
+
4. Validators should return a validation result or normalized artifact according to the existing package pattern; do not silently coerce private or malformed fields.
|
|
127
|
+
5. Every constructor and validator must call `assertNoPrivateProofPayload` before returning success.
|
|
128
|
+
6. Freeze schema constants and enums to prevent mutation.
|
|
129
|
+
7. Error messages must be public-safe: name the rejected key/path, never echo the rejected private value.
|
|
130
|
+
|
|
131
|
+
## Forbidden fields and values
|
|
132
|
+
|
|
133
|
+
Reject these exact keys anywhere in proof artifacts, nested packet artifacts, docs examples, and test fixtures:
|
|
134
|
+
|
|
135
|
+
- `memory`
|
|
136
|
+
- `memory_text`
|
|
137
|
+
- `raw_memory`
|
|
138
|
+
- `prompt`
|
|
139
|
+
- `prompts`
|
|
140
|
+
- `transcript`
|
|
141
|
+
- `transcripts`
|
|
142
|
+
- `completion`
|
|
143
|
+
- `completions`
|
|
144
|
+
- `embedding`
|
|
145
|
+
- `embeddings`
|
|
146
|
+
- `acl`
|
|
147
|
+
- `acl_body`
|
|
148
|
+
- `access_control_list`
|
|
149
|
+
- `tenant`
|
|
150
|
+
- `tenant_name`
|
|
151
|
+
- `customer_name`
|
|
152
|
+
- `organization_name`
|
|
153
|
+
- `org_name`
|
|
154
|
+
- `private_key`
|
|
155
|
+
- `secret_key`
|
|
156
|
+
- `api_key`
|
|
157
|
+
- `seed_phrase`
|
|
158
|
+
- `mnemonic`
|
|
159
|
+
- `provider_response`
|
|
160
|
+
- `provider_responses`
|
|
161
|
+
- `response_body`
|
|
162
|
+
- `access_token`
|
|
163
|
+
- `refresh_token`
|
|
164
|
+
- `password`
|
|
165
|
+
- `credential`
|
|
166
|
+
- `credentials`
|
|
167
|
+
|
|
168
|
+
Also reject values that look private even under disguised keys: seed phrases, PEM private keys, bearer/basic/API-token prefixes, provider transcript blocks, raw embedding arrays, absolute private local paths, tenant/customer/account/company names, and raw benchmark prompt/completion/dataset rows.
|
|
169
|
+
|
|
170
|
+
Safe replacements: use `memory_root`, `memory_hash`, `receipt_root`, `prompt_hash`, `transcript_hash`, `completion_hash`, `embedding_set_hash`, `scope_ref`, `policy_hash`, `subject_ref`, `issuer_ref`, `opaque_account_ref`, `provider_ref`, `report_hash`, aggregate counts, public key refs, and signature refs.
|
|
171
|
+
|
|
172
|
+
## CLI contract
|
|
173
|
+
|
|
174
|
+
`enigma chain` is a local command group. It must not submit transactions, read Solana keypairs, call Solana RPC, contact hosted Enigma services, or call external providers.
|
|
175
|
+
|
|
176
|
+
- `enigma chain anchor`: accepts one or more `--root <sha256:...>` values plus optional public refs and writes an anchor batch.
|
|
177
|
+
- `enigma chain grant`: accepts public `--subject`, `--capability`, `--scope`, optional resource refs, policy hash, expiry, and writes a grant.
|
|
178
|
+
- `enigma chain revoke`: accepts `--grant-hash`, `--reason`, optional public refs, and writes a revocation with `nullifier_root`.
|
|
179
|
+
- `enigma chain attest`: accepts `--report-hash` or `--report-file`, dataset/runner/package refs, optional public scores, and writes a benchmark attestation.
|
|
180
|
+
- `enigma chain verify --file <json>`: loads one supported artifact, runs the matching validator, and prints a public-safe validation result.
|
|
181
|
+
|
|
182
|
+
Every command output must include or summarize `transaction_submitted:false` and `raw_memory_on_chain:false`.
|
|
183
|
+
|
|
184
|
+
## Packet builder script
|
|
185
|
+
|
|
186
|
+
`scripts/build-proof-network-packet.mjs` exists for a local release/demo packet. Preserve these rules: hash the benchmark report file; never copy its body into the packet; reject absolute output paths; keep `transaction_submitted:false` and `raw_memory_on_chain:false`; do not call a network, deploy contracts, create accounts, sign transactions, or write private payload classes.
|
|
187
|
+
|
|
188
|
+
## Solana program handoff
|
|
189
|
+
|
|
190
|
+
The future Solana program should treat Enigma artifacts as commitments, not data stores.
|
|
191
|
+
|
|
192
|
+
Recommended instruction shape:
|
|
193
|
+
|
|
194
|
+
1. `anchor_batch(payload_hash, schema_hash, commitment_count, created_at_bucket)`
|
|
195
|
+
2. `grant_capability(grant_hash, subject_ref_hash, issuer_ref_hash, scope_hash, expires_at)`
|
|
196
|
+
3. `revoke_capability(nullifier_root, grant_hash, revoked_at, reason_code)`
|
|
197
|
+
4. `attest_benchmark(attestation_hash, report_hash, dataset_ref_hash, runner_ref_hash, package_ref_hash, sample_count)`
|
|
198
|
+
|
|
199
|
+
On-chain accounts should store compact hashes, counters, timestamps, and public status. They should not store JSON blobs unless the blob has passed the same forbidden-payload scanner and strict size caps. Even then, prefer storing only a digest plus schema discriminator.
|
|
200
|
+
|
|
201
|
+
## Hosted verifier handoff
|
|
202
|
+
|
|
203
|
+
A hosted verifier can add convenience, not authority over private content. It may validate schema ids, JSON Schemas, package validators, canonical hashes, packet nesting, artifact counts, and public registry/chain observations. It must return bounded public-safe error codes and never echo private submitted values.
|
|
204
|
+
|
|
205
|
+
Verifier non-goals: no raw memory ingestion, provider transcript ingestion, embedding upload endpoint, tenant-name lookup endpoint, Solana transaction submission, or hosted secret custody in the 0.1.13 proof-network layer.
|
|
206
|
+
|
|
207
|
+
## Targeted test plan
|
|
208
|
+
|
|
209
|
+
Run only targeted tests while building this layer. The orchestrator owns project-wide gates.
|
|
210
|
+
|
|
211
|
+
Core package tests in `test/enigma-proof-network.test.mjs` should cover constructor schemas, safety flags, deterministic `sha256Json`, valid/invalid validators, recursive packet validation, forbidden key/value rejection without value echoing, Solana-ready anchor content, and benchmark attestation privacy boundaries.
|
|
212
|
+
|
|
213
|
+
CLI tests in `test/enigma-chain-cli.test.mjs` should cover `chain anchor`, `chain grant`, `chain revoke`, `chain attest --report-hash`, `chain attest --report-file`, `chain verify --file` success/failure paths, required false safety flags, and the absence of transaction submission, keypair reads, Solana RPC, provider calls, or hosted credentials.
|
|
214
|
+
|
|
215
|
+
Use fixtures that contain only placeholder refs and hashes. Do not create realistic memory text, tenant names, prompts, completions, embeddings, provider responses, private keys, seed phrases, or API tokens in tests.
|
|
216
|
+
|
|
217
|
+
## Release gates
|
|
218
|
+
|
|
219
|
+
Do not mark the release ready unless all of these are true:
|
|
220
|
+
|
|
221
|
+
1. `packages/proof-network/src/index.js` exposes the full API contract listed above.
|
|
222
|
+
2. `package.json` includes the package export and publish file path for `./proof-network`.
|
|
223
|
+
3. JSON Schemas exist for all five schema ids, including `proof-network-capability-revocation-v1.schema.json`.
|
|
224
|
+
4. CLI commands exist for `chain anchor`, `chain grant`, `chain revoke`, `chain attest`, and `chain verify`.
|
|
225
|
+
5. Every generated CLI artifact includes `transaction_submitted:false` and `raw_memory_on_chain:false`.
|
|
226
|
+
6. Validators reject forbidden keys at arbitrary nesting depth.
|
|
227
|
+
7. Validators reject private-looking values without printing them.
|
|
228
|
+
8. Packets recursively validate nested artifacts and counts.
|
|
229
|
+
9. Tests cover pure package APIs, packet builder behavior, and CLI behavior with public-safe fixtures.
|
|
230
|
+
10. Documentation examples contain only hashes, roots, refs, counts, and signatures.
|
|
231
|
+
11. No Solana deploy, RPC submission, account creation, npm publish, Cloudflare change, external provider call, or live infrastructure action is part of the release path.
|
|
232
|
+
12. Benchmark attestations are claim-bounded to reviewed report hashes/refs and aggregate metrics, not raw datasets or model outputs.
|
|
233
|
+
13. Hosted verifier docs state that it verifies public commitments and never ingests raw memory.
|
|
234
|
+
14. Public positioning avoids claims of provider deletion proof, model forgetting proof, hosted SaaS proof, compliance certification, or official benchmark superiority unless separate audited evidence exists.
|
|
235
|
+
|
|
236
|
+
## Copy review checklist
|
|
237
|
+
|
|
238
|
+
Before merging proof-network docs or examples, search changed files for private payload language. Any occurrence of the forbidden keys above should be absent or inside a warning list like this document. Examples should use `subject_ref`, `issuer_ref`, `scope_ref`, `sha256:...`, `root_...`, `packet_...`, and `signature:...`; they should not use customer names, real local paths, account ids, provider transcripts, memory snippets, or keys.
|
|
239
|
+
|
|
240
|
+
The safest engineering rule is simple: if a future Solana explorer, README, CI log, npm tarball, hosted verifier response, or benchmark artifact could show it publicly, the value must already be public-safe before it enters the Proof Network.
|