persona-ai-sdk 0.0.0-stage → 1.0.0

package/.env.example

@@ -0,0 +1,14 @@
+# OpenAI
+OPENAI_API_KEY=
+OPENAI_MODEL=
+OPENAI_EMBEDDING_MODEL=text-embedding-3-small
+RUN_LIVE_PROVIDER_TESTS=0
+
+# Amazon Bedrock uses the standard AWS credential provider chain.
+AWS_REGION=us-east-1
+AWS_PROFILE=
+BEDROCK_MODEL_ID=
+BEDROCK_EMBEDDING_MODEL_ID=amazon.titan-embed-text-v2:0
+
+# Hosted service example. Use different scoped keys for different tenants.
+CHARACTER_MEMORY_API_KEYS=local-development-key:local

package/ARCHITECTURE.md

@@ -0,0 +1,164 @@
+<div align="center">
+
+# Architecture
+
+Internal boundaries and extension rules for `persona-ai-sdk`.
+
+[README](./README.md) · [Documentation](./docs/README.md) · [API reference](./docs/API_REFERENCE.md)
+
+</div>
+
+---
+
+## Design goals
+
+The package is infrastructure, not a chat frontend. It must remain usable from text chat, voice,
+games, bots, and server applications while allowing providers and databases to change independently.
+
+The dependency rule is:
+
+```text
+application
+    |
+    v
+bundled persona card ---> persona manifest
+    |
+    v
+persona runtime ---> character model <--- OpenAI / Bedrock adapters
+    |
+    v
+memory runtime ---> extractor port <--- OpenAI / Bedrock adapters
+    |
+    +------------> embedding port <--- OpenAI / Titan adapters
+    |
+    +------------> store port <------- memory / JSON / future databases
+```
+
+Core code does not import OpenAI, AWS, or filesystem modules. Vendor and persistence integrations
+depend inward on core contracts.
+
+## Modules
+
+### Persona catalog
+
+`personas/*.json` contains the versioned Tavern Labs cards. `src/personas.mjs` loads the catalog and
+exports full-card lookup, ID listing, summaries, and the default pair. The build copies the loader
+to the package root, where it is re-exported alongside the core runtime.
+
+Persona cards are product defaults rather than canonical memory records. Applications map a card's
+prompt, traits, greeting, and memory seeds onto their own persona manifest and seeding policy.
+
+### Core
+
+`src/core` owns:
+
+- The canonical memory schema and lifecycle
+- Extraction and embedding ports
+- Ingestion, provenance validation, consolidation, and lifecycle controls
+- Retrieval constraints and prompt-budget packing
+- Provider-independent errors
+
+It does not own credentials, HTTP clients, or database connections.
+
+### Providers
+
+`src/providers` adapts vendor SDKs to the core ports.
+
+- OpenAI extraction uses Responses API structured output.
+- Bedrock extraction uses Converse structured output.
+- OpenAI embeddings use the embeddings API.
+- Bedrock embeddings use Titan Text Embeddings V2 through InvokeModel.
+
+Provider responses are validated again with Zod at the boundary. Structured output protects syntax;
+runtime validation protects domain constraints and produces stable errors.
+
+### Stores
+
+`src/stores` owns persistence adapters.
+
+- `InMemoryMemoryStore` is deterministic and intended for tests or short-lived processes.
+- `JsonFileMemoryStore` is a local reference adapter with versioned files, serialized writes, and
+  temporary-file replacement.
+- `SQLiteMemoryStore` uses WAL mode, indexes, and transactional batch writes for durable
+  deployments.
+
+The store persists complete canonical records. Vectors can always be regenerated from `content`.
+
+## Ingestion lifecycle
+
+```text
+messages
+  -> normalize and validate
+  -> provider extraction
+  -> validate schema and source-message provenance
+  -> discard low-confidence candidates
+  -> consolidation (merge, supersede, conflict, or keep)
+  -> generate embeddings for new records
+  -> batch persistence
+```
+
+The runtime never accepts a source message ID invented by a model. Candidates with no remaining
+valid provenance are discarded.
+
+Consolidation is a replaceable policy rather than database behavior. The default policy merges
+normalized/semantic duplicates, supersedes mutable state, and surfaces contradictions to the caller.
+
+## Recall lifecycle
+
+```text
+query
+  -> optional query embedding
+  -> scope and identity filters
+  -> semantic + lexical + importance + recency scoring
+  -> expiry filtering
+  -> token-budget packing
+  -> bounded, explicitly untrusted memory context
+```
+
+The prompt renderer labels recalled content as recollection rather than instruction. Applications
+should place the resulting block in contextual data, never above their security or behavior policy.
+
+## Identity and isolation
+
+`characterId` is required on every operation. `userId` defaults to `null` and is included in
+retrieval filters. Applications should use opaque stable IDs, not mutable display names.
+
+The service adapter resolves API keys to tenant IDs, but its store factory must also provide
+physical or logical tenant partitioning. Do not encode tenancy only in vector metadata.
+
+## Failure behavior
+
+- Provider refusals, truncated output, malformed JSON, and schema violations fail ingestion without
+  partially writing new records.
+- Embedding count mismatches fail before persistence.
+- File writes are serialized and atomically replaced.
+- Empty conversations and empty recalls are valid and return empty results.
+
+## Higher layers
+
+`src/persona` owns manifests, prompt compilation, relationship state, delivery hints, and response
+orchestration. `src/service` wraps persona runtimes in tenant-aware HTTP routes. `src/sillytavern`
+is a compatibility boundary, so card fields do not leak into the canonical schema. `src/evaluation`
+measures retrieval hit rate/MRR and aggregates deterministic drift checks.
+
+## Package exports
+
+The root export contains core, the in-memory store, and the bundled persona catalog. Heavy or
+environment-specific adapters use explicit subpaths:
+
+```text
+persona-ai-sdk
+persona-ai-sdk/personas
+persona-ai-sdk/personas/*
+persona-ai-sdk/openai
+persona-ai-sdk/bedrock
+persona-ai-sdk/stores/json-file
+persona-ai-sdk/stores/sqlite
+persona-ai-sdk/persona
+persona-ai-sdk/sillytavern
+persona-ai-sdk/service
+persona-ai-sdk/evaluation
+```
+
+This prevents browser or edge consumers from loading AWS and filesystem code accidentally and gives
+the package room to split adapters into optional packages later without changing the core API.

package/CHANGELOG.md

@@ -0,0 +1,11 @@
+# Changelog
+
+## 1.0.0 - 2026-10-31
+
+- Initial public release.
+- Character memory runtime: extraction, hybrid recall, consolidation, lifecycle controls, atomic
+  JSON-file and SQLite stores.
+- Provider adapters: OpenAI Responses API and Amazon Bedrock Converse, Titan V2 embeddings,
+  character models.
+- Persona response runtime with relationship state and JSON state persistence.
+- Eight default persona cards: atlas, sage, vera, scout, wren, ember, lumen, mica.

package/CONTRIBUTING.md

@@ -0,0 +1,193 @@
+<div align="center">
+
+# Contributing
+
+Thank you for helping characters stay coherent, portable, and safe.
+
+[README](./README.md) · [Documentation](./docs/README.md) · [Architecture](./ARCHITECTURE.md)
+
+</div>
+
+---
+
+## Prerequisites
+
+- Node.js 20 or newer
+- npm compatible with the committed lockfile
+- Git
+- A native build environment if your platform cannot use a prebuilt `better-sqlite3` binary
+
+Provider credentials are not required for normal development or CI.
+
+## Set up
+
+```bash
+npm install
+npm run check
+```
+
+The check must pass before and after a change:
+
+1. Prettier formatting check
+2. Strict TypeScript type check
+3. Deterministic Vitest suite
+4. Distribution build
+
+## Development commands
+
+| Command                | Purpose                                    |
+| ---------------------- | ------------------------------------------ |
+| `npm run build`        | Compile distributable ESM and declarations |
+| `npm run format`       | Format code and Markdown                   |
+| `npm run format:check` | Check formatting without changing files    |
+| `npm run docs:check`   | Verify local Markdown links resolve        |
+| `npm test`             | Run deterministic tests once               |
+| `npm run test:watch`   | Run tests in watch mode                    |
+| `npm run typecheck`    | Check strict TypeScript without emitting   |
+| `npm run check`        | Run every required validation step         |
+| `npm pack --dry-run`   | Inspect the npm package payload            |
+
+## Change workflow
+
+1. Read the relevant guide and source boundary.
+2. Keep the change focused on one user-visible outcome.
+3. Add or update deterministic tests.
+4. Update the API reference and task guide when behavior changes.
+5. Add a changelog entry for notable user-facing changes.
+6. Run `npm run check` and `npm pack --dry-run`.
+7. Review the diff for credentials, generated junk, unrelated formatting, and accidental breaking
+   changes.
+
+Use Conventional Commit-style subjects where practical:
+
+```text
+feat(memory): add postgres store adapter
+fix(sillytavern): preserve multiline example dialogue
+docs(api): document consolidation decisions
+test(service): cover tenant isolation
+```
+
+Do not alter commit timestamps or fabricate development history.
+
+## Architectural rules
+
+### Core remains provider-neutral
+
+Code in `src/core` must not import OpenAI, AWS, HTTP frameworks, or Node filesystem/database
+modules. Define a port in core and implement vendor behavior in an adapter.
+
+### Validate at boundaries
+
+- Validate provider structured output even when the provider claims schema conformance.
+- Validate untrusted cards, chat exports, HTTP bodies, and persisted records.
+- Preserve source-message provenance.
+- Fail before persistence when a batch is inconsistent.
+
+### Preserve canonical records
+
+Store adapters must persist the complete `MemoryRecord`. Embeddings are rebuildable indexes;
+content, provenance, lifecycle, visibility, and identity are not.
+
+### Keep tenant isolation explicit
+
+Do not add a production service store that relies only on vector metadata or process-local
+filtering. Tenant identity must be part of durable keys and queries.
+
+### Respect licensing boundaries
+
+This repository is MIT licensed. Do not copy AGPL SillyTavern source or third-party extension code
+into it. Implement compatibility from public formats and APIs, and preserve required notices for any
+new dependency or adapted code.
+
+## Testing expectations
+
+Every public behavior needs a test covering the outcome and relevant failure path.
+
+| Change                | Minimum coverage                                            |
+| --------------------- | ----------------------------------------------------------- |
+| Core lifecycle        | Created/merged/superseded/conflict result and stored state  |
+| Store                 | Round-trip, filtering, and failure/transaction behavior     |
+| Provider adapter      | Exact SDK request shape and malformed response handling     |
+| Persona               | Compiled instruction or state transition and response trace |
+| Importer              | Representative fixture plus malformed input                 |
+| Service route         | Authentication, tenant behavior, status, and response body  |
+| Documentation example | Typechecked equivalent or nearby tested usage               |
+
+Normal tests must not call paid APIs. Inject provider clients and return deterministic fixtures.
+
+Live smoke tests are explicitly opt-in:
+
+```bash
+RUN_LIVE_PROVIDER_TESTS=1 npm test
+```
+
+Only run them in an approved account with spend controls.
+
+## Documentation expectations
+
+Documentation follows a layered approach: a navigable README, task tutorials, conceptual
+explanation, API reference, contribution guidance, license, and version history.
+
+When adding a public function, document:
+
+- What it does
+- Import path
+- Parameters and defaults
+- Return value
+- Side effects
+- Failure behavior
+- A minimal example when usage is not obvious
+
+Use descriptive filenames, stable headings, relative links, accessible image alt text, and explicit
+placeholders. Do not add badges for CI, npm, coverage, security, or compatibility unless the
+underlying claim is real and maintained.
+
+## Style
+
+- Prefer plain language and precise names.
+- Keep provider terminology at provider boundaries.
+- Use ISO 8601 timestamps in data and documentation.
+- Explain why a constraint exists when it is not obvious.
+- Avoid comments that merely repeat code.
+- Avoid introducing a dependency for behavior that is small and auditable.
+- Preserve strict TypeScript settings; do not suppress errors with broad `any` casts.
+
+## Compatibility and breaking changes
+
+Version `0.x` may still evolve, but breaking changes should be deliberate. A breaking proposal must
+describe:
+
+- Affected imports and types
+- Migration path
+- Persistence/schema implications
+- Provider behavior changes
+- Whether old exports or store files remain readable
+
+Store format changes need a forward migration and a fixture proving the previous supported version
+still loads, unless the release explicitly documents otherwise.
+
+## Security-sensitive changes
+
+Read [docs/SECURITY.md](./docs/SECURITY.md) before modifying authentication, CORS, tenancy,
+visibility, imports, export/erasure, logging, or provider credential handling.
+
+Do not put a vulnerability, credential, or real private conversation into a public issue or test
+fixture. Use the hosting platform's private vulnerability-reporting channel for sensitive reports.
+
+## Review checklist
+
+- [ ] The change has one clear outcome
+- [ ] Public inputs are validated
+- [ ] Tests are deterministic and meaningful
+- [ ] Provider calls do not silently change defaults
+- [ ] Tenant/user/character identity is preserved
+- [ ] Secrets and message content are not logged
+- [ ] Relevant documentation and changelog are updated
+- [ ] `npm run check` passes
+- [ ] `npm pack --dry-run` contains only intended files
+- [ ] No unrelated user changes were overwritten
+
+---
+
+By contributing, you agree that your contribution may be distributed under the project's
+[MIT License](./LICENSE).

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tavern Labs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.