enigma-memory 0.1.11 → 0.1.13

package/docs/market-category-narrative.md

@@ -0,0 +1,398 @@
+# Market category narrative
+
+## One-line thesis
+
+Enigma is the **private Memory Controller for AI**: a user- or customer-controlled layer that stores durable memory outside the model, governs how that memory is used, and emits public-safe proofs for memory operations without exposing the memory itself.
+
+## The category shift
+
+The first wave of AI memory is being described as a chatbot feature: a model remembers a preference, a product saves a conversation, or an app retrieves a useful fact from a vector store.
+
+That framing is too small.
+
+Memory is not just recall. Memory is a durable asset. It contains identity, context, preferences, decisions, permissions, work history, relationships, project state, and institutional knowledge. Once AI systems become operators across apps, workflows, devices, teams, and agents, memory stops being a convenience feature and becomes infrastructure.
+
+The category shift is:
+
+> From chatbot memory to AI Memory Controller.
+
+A chatbot memory feature asks, “What should this assistant remember?”
+
+An AI Memory Controller asks:
+
+- Who owns the memory?
+- Where does the canonical memory live?
+- Which model, app, agent, or workflow can use it?
+- What context was retrieved, injected, exported, revoked, or withheld?
+- Which operations can be verified later?
+- What can be proven publicly without revealing the private memory behind it?
+- Which proofs, permissions, roots, and refs can move across ecosystems?
+
+That is the product system Enigma should own.
+
+## The SanDisk / SSD analogy
+
+The useful analogy is not “Enigma is a database.” The better analogy is **SanDisk plus the SSD controller layer for AI memory**.
+
+An SSD is not valuable because it is a folder. It is valuable because it provides durable storage that applications can trust without knowing the details of flash translation, wear leveling, bad-block handling, integrity checks, erase behavior, caching, interfaces, firmware, and lifecycle management.
+
+The controller is the important part. It turns fragile underlying media into a usable system.
+
+AI memory has a similar problem. The underlying “media” is scattered across:
+
+- model-provider memory features;
+- chat histories;
+- browser sessions;
+- IDE agents;
+- local vector indexes;
+- enterprise logs;
+- agent scratchpads;
+- RAG stores;
+- CRM and ticketing systems;
+- imported exports;
+- benchmark runs;
+- governance and audit records.
+
+Without a controller, every app invents its own memory layer, custody model, deletion semantics, permission surface, export format, and proof story. That creates lock-in for users, risk for enterprises, and repetitive infrastructure work for developers.
+
+Enigma’s category role is to be the controller layer:
+
+- keep canonical memory local or customer-controlled;
+- expose memory through connectors instead of one walled garden;
+- treat summaries, vector indexes, and retrieval caches as derived artifacts;
+- govern which clients and agents can use which memory;
+- record Enigma-controlled lifecycle operations;
+- produce receipts, proof packets, benchmark attestations, and verifier artifacts;
+- keep raw memory out of public artifacts;
+- optionally anchor opaque roots or refs on Solana when a public timestamp or settlement rail is useful.
+
+The SSD analogy should stay operational, not financial. It does not imply valuation outcomes, market share, token value, or investment returns. It gives buyers and developers a simple mental model: AI systems need durable private memory, and that memory needs a controller.
+
+## Why models need external private memory
+
+A model is not the right place to put canonical memory.
+
+Models are powerful reasoning and generation engines, but they are not controlled memory drives. They are opaque, vendor-specific, expensive to inspect, difficult to port, and often outside the owner’s direct custody. Even when a provider offers memory features, that memory usually lives inside one product boundary and cannot reliably become the user’s or enterprise’s cross-model source of truth.
+
+External private memory matters because AI work is becoming long-lived.
+
+A useful AI system needs to remember more than a prompt window can hold:
+
+- stable user preferences;
+- project constraints;
+- prior decisions;
+- open tasks;
+- security boundaries;
+- workflow state;
+- team norms;
+- imported history;
+- permissions and revocations;
+- benchmark and evaluation records;
+- enterprise retention and legal-hold state.
+
+That memory should not be trapped inside one chatbot subscription or silently duplicated across every model provider a user tries. It should be portable across models, clients, and workflows.
+
+External memory also creates a cleaner architecture:
+
+- **The model reasons.** It can use relevant context when authorized.
+- **The controller governs.** It decides what memory is available, under what policy, through which connector, with which receipt.
+- **The owner retains custody.** The canonical record can remain local, customer-controlled, or explicitly hosted under declared boundaries.
+- **The verifier checks artifacts.** Proof packets and attestations can be inspected without exposing private memory.
+
+This separation gives users portability, gives developers a reusable memory layer, and gives enterprises a control surface before memory crosses a provider boundary.
+
+## What makes memory different from search
+
+Search returns relevant information. Memory carries responsibility.
+
+A vector database can answer “what is similar to this query?” That is useful, but it is not enough for durable AI memory. A memory controller has to answer harder questions:
+
+- Is this memory canonical or derived?
+- Is it active, expired, tombstoned, exported, or under legal hold?
+- Which connector requested it?
+- Was it allowed by policy?
+- Was it injected into a model context?
+- Can the operation be reconstructed later from public-safe evidence?
+- Can an outside reviewer verify the artifact without seeing private content?
+
+This is why Enigma should not be positioned as another semantic search tool. Search is one capability inside the system. The category is controlled, portable, verifiable AI memory.
+
+## Why proof matters
+
+AI memory creates trust problems that ordinary product logs do not solve.
+
+If a team claims that memory was batched, exported, revoked, withheld, benchmarked, or made available to a specific tool under a specific permission, another party may need evidence. That party might be a developer, auditor, enterprise security reviewer, ecosystem partner, or future operator debugging an incident.
+
+But the evidence cannot simply be the private memory itself.
+
+Public artifacts must not contain raw memory, prompts, transcripts, completions, embeddings, tenant names, private keys, API keys, seed phrases, provider responses, ACL bodies, customer records, or private benchmark rows. Publishing those values would turn proof into a privacy failure.
+
+The proof layer exists to make a narrow, useful promise:
+
+> Enigma can produce public-safe evidence for Enigma-controlled memory operations and declared boundary decisions while keeping private memory private.
+
+Proof artifacts should carry values such as:
+
+- schema ids;
+- artifact ids;
+- hashes;
+- Merkle roots;
+- public-safe refs;
+- counts;
+- timestamps;
+- signer or verifier metadata;
+- policy ids;
+- grant and revocation refs;
+- benchmark dataset refs, runner refs, package refs, config refs, and report hashes;
+- explicit boundary notes.
+
+They should not claim more than they can prove. A receipt can support an Enigma-side memory lifecycle claim. It cannot prove that a closed model forgot something, that a provider deleted every internal copy, that an enterprise is compliant, that a benchmark implies universal superiority, or that a deployment has a financial outcome.
+
+That boundary is not a weakness. It is the source of credibility.
+
+## Where blockchain fits
+
+Blockchain is useful only after the memory boundary is clear.
+
+The product is not “put AI memory on-chain.” Raw memory does not belong on-chain. Public chains are replicated, durable, and hard to erase. Even encrypted sensitive content can become a long-term disclosure risk if keys, algorithms, metadata, or access patterns change.
+
+The correct blockchain role is narrower:
+
+> Solana can be an optional proof, permission, timestamp, and settlement rail for opaque roots and refs. The memory stays private.
+
+In Enigma’s product system, Solana should carry public-safe commitments such as:
+
+- memory batch roots;
+- proof-packet roots;
+- benchmark-attestation roots;
+- capability-grant refs;
+- revocation nullifiers;
+- settlement or metering refs;
+- verifier refs.
+
+It should not carry the underlying memory, prompt text, transcript, completion, embedding, tenant name, ACL body, key, seed phrase, provider response, or private dataset row.
+
+This lets Enigma speak to the blockchain ecosystem without becoming chain-first. The local Memory Controller is the trust anchor. Solana is an optional public rail when a team wants outside observers to verify that an opaque commitment existed, a grant was issued, a revocation was recorded, or a report hash was anchored.
+
+The message is simple:
+
+> Public roots for private memory systems.
+
+## Product architecture as narrative
+
+The marketable system has five layers.
+
+### 1. Private Memory Controller
+
+The controller is the core category object. It manages the canonical memory record and its lifecycle.
+
+It should make memory feel like owned infrastructure: durable, portable, inspectable, exportable, revocable where Enigma controls the boundary, and verifiable through receipts.
+
+### 2. Connector layer
+
+AI memory becomes valuable when many clients can use it. Connectors let AI apps, agents, IDEs, browsers, SDKs, CLIs, and enterprise gateways request memory through declared boundaries rather than copying memory into every product.
+
+The connector message is:
+
+> One private memory layer, many AI clients.
+
+### 3. Governance layer
+
+Enterprises need policy before memory crosses boundaries. A memory controller should support allow/deny decisions, scoped grants, revocations, retention posture, residency policy, legal-hold state, SIEM-safe events, and offline review artifacts.
+
+The enterprise message is:
+
+> Govern AI memory before it enters a model context.
+
+### 4. Proof layer
+
+Receipts and proof packets turn memory operations into bounded evidence. They make Enigma more than a storage feature because they create a verifier surface that can travel outside the private data plane.
+
+The proof message is:
+
+> Proofs can be public; memory stays private.
+
+### 5. Optional Solana rail
+
+Solana can add public timestamping, root anchoring, nullifier publication, permission references, and settlement references when operators choose that path.
+
+The blockchain message is:
+
+> Anchor roots, not memory.
+
+## Blockchain-specific product system
+
+The blockchain-specific version of Enigma should still begin with private memory, not with chain mechanics.
+
+The product system is:
+
+| Layer | Product object | Buyer-visible purpose | Public surface |
+| --- | --- | --- | --- |
+| Private Memory Controller | Local or customer-controlled memory record | Keep durable AI memory portable and governed | None by default |
+| Connector boundary | MCP, SDK, CLI, browser, agent, or gateway path | Decide which client can use which memory under which policy | Public-safe connector refs and receipt ids |
+| Proof packet | Hashes, roots, refs, counts, timestamps, signatures, and boundary notes | Let a reviewer verify an Enigma-controlled event without seeing private memory | Schema-bound packet or attestation |
+| Permission object | Capability grant, expiry, scope digest, and revocation path | Give agents and operators narrow memory permissions | Public-safe grant refs or digest commitments |
+| Revocation object | Nullifier, revoked grant ref, and effective time | Tell verifiers which grants or proof references should no longer be accepted | Public-safe nullifier or revocation ref |
+| Optional Solana rail | Anchor, registry entry, permission state, nullifier set, or settlement reference | Provide neutral ordering, discoverability, and shared state when local evidence is not enough | Opaque roots, refs, public keys, slots, signatures, and settlement status |
+
+This gives the blockchain story product weight without leaking the memory story into the chain. Enigma is not asking the market to believe that blockchains should store AI memory. It is saying that AI memory needs a private controller, and some controller events benefit from a public rail.
+
+## Public-safe example patterns
+
+The category should be explained with examples that are safe to publish.
+
+A good memory commitment example says:
+
+> A workspace generated a memory-batch root, a count, a timestamp, and a verifier packet. The private memory records remained in the customer-controlled memory store. A reviewer with approved access can recompute the root; a public observer only sees the opaque commitment.
+
+A good permission example says:
+
+> An agent received a scoped capability to request a context pack for a public-safe project ref until a declared expiry. The grant refers to a scope digest, not the private policy body or raw memory behind that scope.
+
+A good revocation example says:
+
+> A prior capability was revoked through a nullifier. Verifiers that honor the nullifier set should reject the old grant. The revocation does not claim downstream provider deletion or model forgetting.
+
+A good benchmark example says:
+
+> A benchmark report is bound to a dataset ref, runner ref, package ref, config ref, metric summary, report hash, and verifier status. The public artifact does not include raw prompts, answers, provider responses, private rows, or hidden evaluation notes.
+
+A good Solana example says:
+
+> An operator may anchor an opaque proof-packet root or publish a public-safe nullifier. The chain shows ordering and shared state for the commitment. The chain does not see the private memory that produced the root.
+
+These examples make the product concrete while keeping the public record clean.
+
+## Spec thinking for the category
+
+The narrative should push toward a spec surface that other developers can implement and reviewers can inspect. The important spec ideas are simple:
+
+1. **Canonical private source, derived public proof.** The memory record remains private. Public artifacts are derived from it through stable hashes, roots, refs, and counts.
+2. **Explicit schema families.** Anchors, grants, revocations, benchmark attestations, proof packets, and settlement refs should have separate schema ids instead of one vague proof blob.
+3. **Public-safety by construction.** Proof artifacts should reject or avoid private payload classes: raw memory, prompts, transcripts, completions, embeddings, tenant names, ACL bodies, private keys, seed phrases, API keys, provider responses, and private benchmark rows.
+4. **Boundary notes are part of the artifact.** Every proof should say what it supports and what it does not support.
+5. **Offline verification first.** A reviewer should be able to verify local artifacts without trusting a hosted Enigma service or submitting a transaction.
+6. **Chain submission is explicit.** Local planning artifacts should not imply live deployment or transaction submission. A real chain workflow needs explicit operator action, key handling, fee policy, confirmation semantics, and irreversible public-data warnings.
+7. **Nullifiers over vague deletion claims.** Revocation should create a verifier rule for Enigma-controlled capabilities, not a claim that every downstream model or provider erased state.
+8. **Settlement references, not private invoices.** If settlement uses a chain rail, it should reference proof roots and public-safe job refs rather than private commercial terms or customer data.
+
+This spec posture turns category design into engineering discipline. The marketable promise is not “trust us.” It is “inspect the controller boundary, verify the artifact, and keep the memory private.”
+
+## Market wedge
+
+The wedge is not a general-purpose chain product. The wedge is the painful gap between durable AI memory and trustworthy control.
+
+Users feel the problem as trapped memory: each assistant remembers differently, exports are awkward, and switching tools means losing context.
+
+Developers feel the problem as repeated infrastructure: every AI product needs storage, retrieval, import/export, permissions, deletion posture, receipts, and evaluation evidence.
+
+Enterprises feel the problem as risk: memory can become an ungoverned copy of customer facts, employee knowledge, regulated context, and decision history.
+
+The crypto ecosystem feels the problem as credibility: chain-based AI claims often become either privacy failures or unverifiable narratives.
+
+Enigma’s wedge cuts across those pains:
+
+> Put the memory under owner control, expose it through connectors, prove bounded operations, and use Solana only where public roots or shared state add value.
+
+## Category positioning
+
+Enigma should avoid the crowded language of “AI memory app,” “RAG database,” or “crypto AI storage.” Those phrases undersell the control-plane problem.
+
+The stronger category is:
+
+> **Private AI Memory Controller**
+
+Expanded:
+
+> **Private memory infrastructure for AI systems, with public-safe proofs and optional Solana anchoring.**
+
+This positioning separates Enigma from adjacent categories:
+
+| Adjacent category | What it usually does | Enigma’s sharper position |
+| --- | --- | --- |
+| Chatbot memory | Saves preferences inside one product | Portable memory outside any one model or chatbot |
+| Vector database | Retrieves similar records | Controls canonical memory lifecycle, permissions, receipts, and proofs |
+| RAG framework | Injects context into prompts | Governs which memory may enter context and records declared boundary operations |
+| Enterprise logging | Records operational events | Produces public-safe verifier artifacts without exposing private memory |
+| Crypto storage | Publishes or coordinates data availability | Keeps raw memory off-chain and anchors only opaque roots, refs, or nullifiers |
+| Benchmark leaderboard | Markets scores | Binds benchmark claims to scoped attestations and reproducibility metadata |
+
+## Messaging spine
+
+Use this sequence when explaining Enigma:
+
+1. **AI work needs memory.** Prompt windows and provider-specific memories are not enough for durable, cross-client AI work.
+2. **Memory needs an owner.** The canonical memory record should be local or customer-controlled, not trapped in one model account.
+3. **Memory needs a controller.** Search is not enough; memory needs lifecycle, permission, context, export, revocation, and governance semantics.
+4. **Memory needs proof.** Teams need receipts and attestations for what Enigma controlled or mediated, without disclosing the private memory.
+5. **Blockchain is optional infrastructure.** Solana can anchor roots, refs, grants, revocations, and attestations, but raw memory stays off-chain.
+
+## Claim-safe category lines
+
+Primary line:
+
+> Enigma is the private Memory Controller for AI.
+
+Expanded line:
+
+> Enigma gives AI systems a private, portable memory layer governed by the owner and backed by public-safe proof artifacts.
+
+Developer line:
+
+> Add durable AI memory, connector access, receipts, and verifier artifacts without building a memory control plane from scratch.
+
+Enterprise line:
+
+> Govern AI memory before it crosses model, app, or workflow boundaries.
+
+Proof line:
+
+> Verify Enigma-controlled memory operations with hashes, roots, refs, counts, timestamps, and signatures instead of private content.
+
+Solana line:
+
+> Use Solana as an optional public rail for opaque proof roots and permission refs, not as a place to store memory.
+
+Boundary line:
+
+> Enigma proves Enigma-controlled memory state and declared boundary operations; it does not prove closed-provider deletion, model forgetting, compliance certification, benchmark superiority, ROI, token value, or investment outcomes.
+
+## Language to avoid
+
+Avoid claims that make the category less credible:
+
+- “Enigma stores AI memory on-chain.”
+- “Encrypted private memory belongs on-chain.”
+- “Enigma proves a model forgot.”
+- “Enigma proves every provider deleted user data.”
+- “Enigma guarantees compliance.”
+- “Enigma guarantees benchmark superiority.”
+- “Enigma guarantees ROI, savings, token value, valuation, or investment upside.”
+- “Solana is required for Enigma to work.”
+- “The blockchain is the memory layer.”
+
+Use bounded replacements:
+
+- “Enigma keeps raw memory private and can publish public-safe commitments.”
+- “Enigma proves Enigma-controlled lifecycle and boundary operations.”
+- “Benchmark claims should be tied to dataset refs, runner refs, package refs, config refs, report hashes, and verifier status.”
+- “Solana anchoring is optional and root-only.”
+- “The Memory Controller is the product; the chain is an optional proof rail.”
+
+## Final narrative
+
+AI systems are moving from one-off conversations to persistent work. Persistent work needs memory. But memory cannot remain a scattered chatbot feature if it contains identity, preferences, permissions, project state, enterprise knowledge, and long-running workflow context.
+
+The market needs a new layer: a private Memory Controller for AI.
+
+Like the controller inside an SSD, this layer makes fragile underlying storage usable. It manages lifecycle, interfaces, integrity, policy, portability, and verification so every AI app does not have to reinvent memory custody from scratch.
+
+Enigma should own that layer.
+
+Models can change. Apps can change. Providers can change. The user’s or customer’s memory should remain portable, governed, and verifiable under declared boundaries.
+
+Proof makes that promise credible. Enigma can emit public-safe artifacts that show what was committed, granted, revoked, benchmarked, or packaged without publishing the private memory behind those operations.
+
+Blockchain fits when public anchoring is useful. Solana can carry hashes, roots, refs, grants, revocations, nullifiers, and settlement references. It should not carry raw memory. The chain is a public clock and proof rail; the private Memory Controller is the product.
+
+That is the category: private memory infrastructure for AI systems, governed by owners, usable across clients, verified by public-safe proofs, and optionally anchored through Solana without putting memory on-chain.