@simonsbs/keylore 1.0.0-rc4 → 1.0.0-rc6

package/.env.example

@@ -1,4 +1,8 @@
-KEYLORE_DATABASE_URL=postgresql://keylore:keylore@127.0.0.1:5432/keylore
+KEYLORE_DATABASE_MODE=local
+KEYLORE_LOCAL_DATABASE_FILE=keylore.db
+# Optional advanced override:
+# KEYLORE_DATABASE_MODE=postgres
+# KEYLORE_DATABASE_URL=postgresql://keylore:keylore@127.0.0.1:5432/keylore
 KEYLORE_DATABASE_POOL_MAX=10
 KEYLORE_AUTH_CLIENTS_FILE=auth-clients.json
 KEYLORE_LOCAL_SECRETS_FILE=local-secrets.enc.json

package/README.md

@@ -22,7 +22,7 @@ This repository is incubating privately today, but it is structured to be publis
 - MCP server for `stdio` and Streamable HTTP
 - metadata-only catalogue search and retrieval tools
 - default-deny policy engine with domain and operation constraints
-- PostgreSQL-backed catalogue, policy, and audit persistence with startup migrations
+- file-backed local persistence by default with startup migrations, plus explicit PostgreSQL support for advanced deployments
 - OAuth-style client credentials token issuance for remote HTTP and MCP access
 - PKCE-bound `authorization_code` plus rotating `refresh_token` support for interactive public or confidential clients
 - protected-resource metadata for REST and MCP surfaces
@@ -64,13 +64,13 @@ This repository is incubating privately today, but it is structured to be publis
 
 ## What is intentionally deferred
 
-The full `KeyLore.md` specification is broader than a sane `v1.0.0-rc4` delivery. The main remaining work before `v1.0.0` is:
+The full `KeyLore.md` specification is broader than a sane `v1.0.0-rc6` delivery. The main remaining work before `v1.0.0` is:
 
 - public release polish and final operator documentation cleanup
 
 Those items are tracked in [docs/roadmap.md](/home/simon/keylore/docs/roadmap.md) and mapped back to the spec in [docs/keylore-spec-map.md](/home/simon/keylore/docs/keylore-spec-map.md).
 
-The active post-`v1.0.0-rc4` refocus is documented in [docs/core-mode-plan.md](/home/simon/keylore/docs/core-mode-plan.md): make the default user journey "add secret, add context, connect MCP, use it" and push broader operator features behind an advanced path.
+The active post-`v1.0.0-rc6` refocus is documented in [docs/core-mode-plan.md](/home/simon/keylore/docs/core-mode-plan.md): make the default user journey "add secret, add context, connect MCP, use it" and push broader operator features behind an advanced path.
 
 The handoff from local core mode to advanced self-hosted mode is documented in [docs/production-handoff.md](/home/simon/keylore/docs/production-handoff.md).
 
@@ -88,14 +88,16 @@ npm install
 npm run quickstart
 ```
 
+That starts KeyLore in the background. Use `keylore-http status`, `keylore-http stop`, and `keylore-http restart` to manage it later. Use `keylore-http run` only when you intentionally want the server attached to the terminal for debugging.
+
 For a clean Linux VM install from npm instead of cloning the repo:
 
 ```bash
 npm install -g @simonsbs/keylore@next
-keylore-http
+keylore-http start
 ```
 
-That starts KeyLore from the packaged migrations and seed data, while writable state defaults to `~/.keylore`.
+That starts KeyLore from the packaged migrations and seed data with no Docker or external PostgreSQL required. Writable state defaults to `~/.keylore`.
 
 To simulate a brand-new user install on the same machine without reusing your normal checkout or shell environment:
 
@@ -105,8 +107,8 @@ npm run ops:fresh-user-env
 
 That launches an isolated disposable user, a fresh clone, and a separate KeyLore UI port for onboarding and MCP testing. By default it clones from the current local repo source so it also works while the repo is private.
 
-This starts local PostgreSQL, waits for readiness, and boots KeyLore at `http://127.0.0.1:8787`.
-If KeyLore is already running locally on that port, the command reuses the existing instance instead of failing.
+This boots KeyLore at `http://127.0.0.1:8787` with a local embedded database and encrypted local secret store.
+If KeyLore is already running locally, the command reuses the existing background instance instead of failing.
 
 3. Open KeyLore in your browser:
 
@@ -116,28 +118,36 @@ KeyLore now redirects `/` to `/admin` and automatically opens a local operator s
 
 If that local session bootstrap fails for any reason, use `Start working locally` or the manual sign-in form shown on the page.
 
-4. In `Save token`, choose the closest template for the token you are adding, such as `GitHub read-only`, `GitHub write-capable`, `npm read-only`, or `Internal service token`, then fill in:
+4. In `Quick start`, follow the short path: add token, test token, then connect your AI tool.
+
+5. In `Your tokens`, click `Add token`, choose the closest template for the token you are adding, such as `GitHub read-only`, `GitHub write-capable`, `npm read-only`, or `Internal service token`, then fill in:
 - `Name shown in KeyLore`
 - `Token key`
 - `Paste token`
 - `Where can it be used?`
+- `Explain this token for people`
 - `Tell the AI when to use this token`
 
-That stores the raw token outside the searchable catalogue and keeps only the LLM-facing metadata in the credential record.
+That stores the raw token outside the searchable catalogue and keeps only the metadata record in the credential catalogue.
 
-5. Review `Writing help` and `What the AI will see` in the form to confirm the agent-facing record is specific, useful, and secret-free. `Token key` is the unique identifier for the token; if KeyLore says a token already exists, change that field and save again. Open `Advanced token settings` only if you need to change storage mode, risk level, service name, tags, or write access.
+6. Review `Writing help` and `What the AI will see` in the form to confirm the record is specific, useful, and secret-free. `LLM context` is the primary retrieval hint for agents. `User context` explains the human purpose of the token. `Token key` is the unique identifier for the token; if KeyLore says a token already exists, change that field and save again. Open `Advanced token settings` only if you need to change storage mode, risk level, service name, tags, or write access.
 
-6. In `Saved tokens`, look under `Your tokens` for the ones you added yourself. `Included examples` are seeded local records and are shown separately so they do not get confused with your own tokens.
+7. In `Saved tokens`, everything is now listed together in one place. Example records are marked as examples, and they can be edited or deleted from the same list.
 
-7. In `Test credential`, choose `Token to check`, set the `URL to call with this token`, and run the check.
+8. In `Test credential`, choose `Token to check`, set the `URL to call with this token`, and run the check.
 
 The check makes a real brokered `http.get` call with that token and URL. Success means the token, the target domain, and KeyLore policy all allowed the request.
 
-8. In `Connect your AI tool`, copy the generated Codex or Gemini CLI local snippet for the easiest setup. Use the built-in `First prompt to try` examples after you restart the client. If you want remote HTTP MCP instead, open `Remote or advanced connection options` and mint an `/mcp` token there.
+9. In `Connect your AI tool`, pick the tool tab you want:
+- `Codex`: copy the snippet or click `Apply to my Codex settings` to merge it into `~/.codex/config.toml`, then restart Codex
+- `Gemini CLI`: copy the snippet or click `Apply to my Gemini settings` to merge it into `~/.gemini/settings.json`, then restart Gemini
+- `Claude CLI`: copy the command or click `Apply to my Claude settings` to register KeyLore, then restart Claude
+
+Use the built-in `First prompt to try` example after restarting the client. If you want remote HTTP MCP instead, open `Remote or advanced connection options` and mint an `/mcp` token there.
 
 Everything beyond that now sits behind `Show advanced controls` in the UI, so a first-run user can ignore tenants, OAuth client administration, approvals, backups, audit, and system internals entirely.
 
-After creation, use `Inspect or edit AI-facing context` inside `Save token` if you need to refine the metadata without re-entering or exposing the stored secret. Saved token cards also support lightweight lifecycle actions such as rename, retag, and archive/restore under `More actions`.
+After creation, use `Edit token` from the saved-token list if you need to refine the metadata or replace the stored value for a locally stored token. Token creation and editing both happen in a popup now, while the main page stays focused on the token list, testing, and MCP connection.
 
 When that local path stops being enough, use [docs/production-handoff.md](/home/simon/keylore/docs/production-handoff.md) to decide when to switch to external secret backends, real OAuth clients, approvals, and tenant-separated self-hosting.
 
@@ -164,12 +174,14 @@ KEYLORE_SANDBOX_COMMAND_ALLOWLIST=/usr/bin/env,node
 
 ## Advanced local usage
 
-Start PostgreSQL only:
+If you want production-style external persistence locally, start PostgreSQL first:
 
 ```bash
 npm run db:up
 ```
 
+Then either set `KEYLORE_DATABASE_MODE=postgres` and `KEYLORE_DATABASE_URL=...` in `.env`, or export them for one run.
+
 Start the HTTP server directly:
 
 ```bash

package/bin/keylore-http.js

@@ -1,3 +1,11 @@
 #!/usr/bin/env node
-process.argv.push("--transport", "http");
-await import("../dist/index.js");
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const binDir = path.dirname(fileURLToPath(import.meta.url));
+const runtimeEntryPath = path.resolve(binDir, "../dist/index.js");
+const serviceManagerPath = path.resolve(binDir, "../dist/http-service.js");
+
+const { runHttpServiceCommand } = await import(serviceManagerPath);
+const exitCode = await runHttpServiceCommand(process.argv.slice(2), runtimeEntryPath);
+process.exit(exitCode);

package/data/catalog.json

@@ -13,6 +13,8 @@
       "expiresAt": null,
       "rotationPolicy": "Rotate every 90 days",
       "lastValidatedAt": "2026-03-13T00:00:00.000Z",
+      "userContext": "Shared example credential for GitHub metadata lookups in local demos. Keep it read-only and prefer a real user-owned token when one exists.",
+      "llmContext": "Fallback GitHub read-only demo credential. Use for repository metadata, issue lookup, release inspection, and rate-limit reads only when no better user-owned GitHub token is available. Never use for write operations.",
       "selectionNotes": "Use for repository metadata, issue lookup, and release inspection. Never use for write operations.",
       "binding": {
         "adapter": "env",
@@ -37,6 +39,8 @@
       "expiresAt": null,
       "rotationPolicy": "Rotate every 90 days",
       "lastValidatedAt": "2026-03-13T00:00:00.000Z",
+      "userContext": "Shared example credential for npm package metadata inspection in local demos. Keep it read-only and do not use it for publish workflows.",
+      "llmContext": "Fallback npm read-only demo credential. Use for package metadata inspection, dependency lookup, and registry read operations only. Never use for publish or package mutation workflows.",
       "selectionNotes": "Use for package metadata inspection only. Publishing is intentionally excluded from the default demo policy.",
       "binding": {
         "adapter": "env",

package/dist/app.js

@@ -38,13 +38,13 @@ import { TenantService } from "./services/tenant-service.js";
 import { TraceExportService } from "./services/trace-export-service.js";
 import { TraceService } from "./services/trace-service.js";
 import { bootstrapFromFiles } from "./storage/bootstrap.js";
-import { createPostgresDatabase } from "./storage/database.js";
+import { createSqlDatabase } from "./storage/database.js";
 import { runMigrations } from "./storage/migrations.js";
 import { SandboxRunner } from "./runtime/sandbox-runner.js";
 export async function createKeyLoreApp() {
     const config = loadConfig();
     const logger = pino({ name: config.appName, level: config.logLevel });
-    const database = createPostgresDatabase(config);
+    const database = await createSqlDatabase(config);
     const telemetry = new TelemetryService();
     const traces = new TraceService(config.traceCaptureEnabled, config.traceRecentSpanLimit);
     const traceExports = new TraceExportService(config.traceExportUrl, config.traceExportAuthHeader, config.traceExportBatchSize, config.traceExportIntervalMs, config.traceExportTimeoutMs, telemetry);

package/dist/config.js

@@ -3,7 +3,6 @@ import os from "node:os";
 import path from "node:path";
 import { fileURLToPath } from "node:url";
 import * as z from "zod/v4";
-const LOCAL_DATABASE_URL = "postgresql://keylore:keylore@127.0.0.1:5432/keylore";
 const PACKAGE_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
 const LOCAL_ADMIN_CLIENT_ID = "keylore-admin-local";
 const LOCAL_ADMIN_CLIENT_SECRET = "keylore-local-admin";
@@ -73,9 +72,6 @@ function hydrateEnvironment(cwd) {
         ...fileEnv,
         ...process.env,
     };
-    if (isMissing(effectiveEnv.KEYLORE_DATABASE_URL)) {
-        effectiveEnv.KEYLORE_DATABASE_URL = LOCAL_DATABASE_URL;
-    }
     const environment = effectiveEnv.KEYLORE_ENVIRONMENT?.trim() || "development";
     const httpHost = effectiveEnv.KEYLORE_HTTP_HOST?.trim() || "127.0.0.1";
     const bootstrapFromFiles = effectiveEnv.KEYLORE_BOOTSTRAP_FROM_FILES !== "false";
@@ -113,14 +109,16 @@ function resolveDefaultDataDir(cwd) {
     return path.join(os.homedir(), ".keylore");
 }
 const envSchema = z.object({
+    KEYLORE_DATABASE_MODE: z.enum(["local", "postgres"]).optional(),
     KEYLORE_DATA_DIR: z.string().optional(),
     KEYLORE_CATALOG_FILE: z.string().optional(),
     KEYLORE_POLICY_FILE: z.string().optional(),
     KEYLORE_AUTH_CLIENTS_FILE: z.string().optional(),
     KEYLORE_MIGRATIONS_DIR: z.string().optional(),
+    KEYLORE_LOCAL_DATABASE_FILE: z.string().optional(),
     KEYLORE_LOCAL_SECRETS_FILE: z.string().optional(),
     KEYLORE_LOCAL_SECRETS_KEY_FILE: z.string().optional(),
-    KEYLORE_DATABASE_URL: z.string().min(1),
+    KEYLORE_DATABASE_URL: optionalString,
     KEYLORE_DATABASE_POOL_MAX: z.coerce.number().int().min(1).max(100).default(10),
     KEYLORE_HTTP_HOST: z.string().default("127.0.0.1"),
     KEYLORE_HTTP_PORT: z.coerce.number().int().min(1).max(65535).default(8787),
@@ -196,22 +194,28 @@ export function loadConfig(cwd = process.cwd()) {
     const env = envSchema.parse(hydrated.env);
     const runtimeRoot = resolveRuntimeRoot(cwd);
     const dataDir = path.resolve(env.KEYLORE_DATA_DIR ?? resolveDefaultDataDir(cwd));
+    const databaseMode = env.KEYLORE_DATABASE_MODE ?? (env.KEYLORE_DATABASE_URL ? "postgres" : "local");
     const publicBaseUrl = env.KEYLORE_PUBLIC_BASE_URL ?? `http://${env.KEYLORE_HTTP_HOST}:${env.KEYLORE_HTTP_PORT}`;
     const oauthIssuerUrl = env.KEYLORE_OAUTH_ISSUER_URL ?? `${publicBaseUrl}/oauth`;
     const localQuickstartEnabled = env.KEYLORE_ENVIRONMENT !== "production" &&
         env.KEYLORE_BOOTSTRAP_FROM_FILES &&
         isLoopbackHost(env.KEYLORE_HTTP_HOST);
+    if (databaseMode === "postgres" && isMissing(env.KEYLORE_DATABASE_URL)) {
+        throw new Error("KEYLORE_DATABASE_URL is required when KEYLORE_DATABASE_MODE=postgres.");
+    }
     return {
         appName: "keylore",
-        version: "1.0.0-rc4",
+        version: "1.0.0-rc6",
         dataDir,
         bootstrapCatalogPath: path.resolve(runtimeRoot, "data", env.KEYLORE_CATALOG_FILE ?? "catalog.json"),
         bootstrapPolicyPath: path.resolve(runtimeRoot, "data", env.KEYLORE_POLICY_FILE ?? "policies.json"),
         bootstrapAuthClientsPath: path.resolve(runtimeRoot, "data", env.KEYLORE_AUTH_CLIENTS_FILE ?? "auth-clients.json"),
         migrationsDir: path.resolve(runtimeRoot, env.KEYLORE_MIGRATIONS_DIR ?? "migrations"),
+        databaseMode,
+        localDatabasePath: path.resolve(dataDir, env.KEYLORE_LOCAL_DATABASE_FILE ?? "keylore.db"),
         localSecretsFilePath: path.resolve(dataDir, env.KEYLORE_LOCAL_SECRETS_FILE ?? "local-secrets.enc.json"),
         localSecretsKeyPath: path.resolve(dataDir, env.KEYLORE_LOCAL_SECRETS_KEY_FILE ?? "local-secrets.key"),
-        databaseUrl: env.KEYLORE_DATABASE_URL,
+        databaseUrl: env.KEYLORE_DATABASE_URL || undefined,
         databasePoolMax: env.KEYLORE_DATABASE_POOL_MAX,
         httpHost: env.KEYLORE_HTTP_HOST,
         httpPort: env.KEYLORE_HTTP_PORT,

package/dist/domain/types.js

@@ -113,6 +113,8 @@ export const credentialRecordSchema = z.object({
     rotationPolicy: z.string().min(1),
     lastValidatedAt: z.string().datetime().nullable(),
     selectionNotes: z.string().min(1),
+    userContext: z.string().min(1).optional(),
+    llmContext: z.string().min(1).optional(),
     binding: credentialBindingSchema,
     tags: z.array(z.string().min(1)).default([]),
     status: credentialStatusSchema.default("active"),
@@ -210,6 +212,52 @@ export const catalogSearchInputSchema = z.object({
 export const createCredentialInputSchema = credentialRecordSchema;
 const secretLikeSelectionNotesPattern = /(gh[pousr]_[A-Za-z0-9_]+|github_pat_[A-Za-z0-9_]+|sk-[A-Za-z0-9_-]+|AKIA[0-9A-Z]{16})/;
 const vagueSelectionNotesPattern = /^(use when needed|general use|general purpose|for api|api token|token for api|default token|main token)$/i;
+function normalizedLlmContext(value) {
+    return (value.llmContext ?? value.selectionNotes ?? "").trim();
+}
+function normalizedUserContext(value) {
+    return (value.userContext ?? normalizedLlmContext(value)).trim();
+}
+function validateLlmContext(llmContext, ctx, path) {
+    if (!llmContext) {
+        ctx.addIssue({
+            code: z.ZodIssueCode.custom,
+            path,
+            message: "LLM context is required. Explain when the agent should use this credential.",
+        });
+        return;
+    }
+    if (llmContext.length < 16) {
+        ctx.addIssue({
+            code: z.ZodIssueCode.custom,
+            path,
+            message: "LLM context must explain when the agent should use this credential in more detail.",
+        });
+    }
+    if (vagueSelectionNotesPattern.test(llmContext)) {
+        ctx.addIssue({
+            code: z.ZodIssueCode.custom,
+            path,
+            message: "LLM context is too vague. Describe the target service, intended use, and what the agent should avoid.",
+        });
+    }
+    if (secretLikeSelectionNotesPattern.test(llmContext)) {
+        ctx.addIssue({
+            code: z.ZodIssueCode.custom,
+            path,
+            message: "LLM context must not contain token-like secret material.",
+        });
+    }
+}
+function validateUserContext(userContext, ctx, path) {
+    if (!userContext) {
+        ctx.addIssue({
+            code: z.ZodIssueCode.custom,
+            path,
+            message: "User context is required. Describe the human purpose of this credential.",
+        });
+    }
+}
 export const coreCredentialCreateInputSchema = z
     .object({
     credentialId: z.string().min(1),
@@ -221,7 +269,9 @@ export const coreCredentialCreateInputSchema = z
     sensitivity: sensitivitySchema.default("high"),
     allowedDomains: z.array(z.string().min(1)).min(1),
     permittedOperations: z.array(operationSchema).min(1).default(["http.get"]),
-    selectionNotes: z.string().min(1),
+    selectionNotes: z.string().min(1).optional(),
+    userContext: z.string().min(1).optional(),
+    llmContext: z.string().min(1).optional(),
     rotationPolicy: z.string().default("Managed locally"),
     tags: z.array(z.string().min(1)).default([]),
     status: credentialStatusSchema.default("active"),
@@ -251,28 +301,8 @@ export const coreCredentialCreateInputSchema = z
             message: "Bearer credentials should include a non-empty header prefix.",
         });
     }
-    const selectionNotes = value.selectionNotes.trim();
-    if (selectionNotes.length < 16) {
-        ctx.addIssue({
-            code: z.ZodIssueCode.custom,
-            path: ["selectionNotes"],
-            message: "Selection notes must explain when the agent should use this credential in more detail.",
-        });
-    }
-    if (vagueSelectionNotesPattern.test(selectionNotes)) {
-        ctx.addIssue({
-            code: z.ZodIssueCode.custom,
-            path: ["selectionNotes"],
-            message: "Selection notes are too vague. Describe the target service, intended use, and what the agent should avoid.",
-        });
-    }
-    if (secretLikeSelectionNotesPattern.test(selectionNotes)) {
-        ctx.addIssue({
-            code: z.ZodIssueCode.custom,
-            path: ["selectionNotes"],
-            message: "Selection notes must not contain token-like secret material.",
-        });
-    }
+    validateLlmContext(normalizedLlmContext(value), ctx, ["llmContext"]);
+    validateUserContext(normalizedUserContext(value), ctx, ["userContext"]);
     if (value.permittedOperations.includes("http.post") && value.scopeTier === "read_only") {
         ctx.addIssue({
             code: z.ZodIssueCode.custom,
@@ -293,6 +323,8 @@ export const coreCredentialContextUpdateInputSchema = z
     allowedDomains: z.array(z.string().min(1)).min(1).optional(),
     permittedOperations: z.array(operationSchema).min(1).optional(),
     selectionNotes: z.string().min(1).optional(),
+    userContext: z.string().min(1).optional(),
+    llmContext: z.string().min(1).optional(),
     tags: z.array(z.string().min(1)).optional(),
     status: credentialStatusSchema.optional(),
 })
@@ -300,29 +332,11 @@ export const coreCredentialContextUpdateInputSchema = z
     message: "At least one context field must be provided.",
 })
     .superRefine((value, ctx) => {
-    if (value.selectionNotes !== undefined) {
-        const selectionNotes = value.selectionNotes.trim();
-        if (selectionNotes.length < 16) {
-            ctx.addIssue({
-                code: z.ZodIssueCode.custom,
-                path: ["selectionNotes"],
-                message: "Selection notes must explain when the agent should use this credential in more detail.",
-            });
-        }
-        if (vagueSelectionNotesPattern.test(selectionNotes)) {
-            ctx.addIssue({
-                code: z.ZodIssueCode.custom,
-                path: ["selectionNotes"],
-                message: "Selection notes are too vague. Describe the target service, intended use, and what the agent should avoid.",
-            });
-        }
-        if (secretLikeSelectionNotesPattern.test(selectionNotes)) {
-            ctx.addIssue({
-                code: z.ZodIssueCode.custom,
-                path: ["selectionNotes"],
-                message: "Selection notes must not contain token-like secret material.",
-            });
-        }
+    if (value.selectionNotes !== undefined ||
+        value.llmContext !== undefined ||
+        value.userContext !== undefined) {
+        validateLlmContext(normalizedLlmContext(value), ctx, ["llmContext"]);
+        validateUserContext(normalizedUserContext(value), ctx, ["userContext"]);
     }
     const scopeTier = value.scopeTier;
     const permittedOperations = value.permittedOperations;