loreli 0.0.0 → 1.0.0

package/packages/session/README.md

@@ -0,0 +1,165 @@
+# loreli/session
+
+Persistent session storage for detached Loreli agents.
+
+This package manages the `~/.loreli/` state directory where agent sessions and their runtime data are persisted. It exists as a standalone package because session persistence is needed by multiple packages (agent, orchestrator, MCP server) — centralizing it avoids duplication and provides a single, generic solution for `~/.loreli` state management.
+
+## Installation
+
+```bash
+pnpm add loreli
+```
+
+## Directory Layout
+
+Storage organizes state into per-session directories under `~/.loreli/sessions/`:
+
+```
+~/.loreli/
+  sessions/
+    <session-id>/
+      config.json          # Session metadata (id, created, repo, etc.)
+      agents/
+        optimus-0.json     # Agent runtime state
+        megatron-0.json
+      logs/                # Session-scoped log files
+      registry.json        # Identity registry snapshot
+```
+
+## API Reference
+
+### `new Storage(opts?)`
+
+Create a new Storage instance.
+
+- `opts.home` `{string}` — Override the base directory. Falls back to `LORELI_HOME` env var, then `~/.loreli`.
+
+The following example shows the precedence order for resolving the home directory. Options take priority over environment variables, which take priority over the default.
+
+```js
+import { Storage } from 'loreli/session';
+
+// Uses ~/.loreli by default
+const storage = new Storage();
+
+// Explicit override
+const custom = new Storage({ home: '/tmp/test-loreli' });
+
+// Environment variable fallback
+// LORELI_HOME=/opt/loreli node app.js
+const envStorage = new Storage(); // uses /opt/loreli
+```
+
+### `storage.init(id, config?)` → `Promise<string>`
+
+Initialize a session directory. Creates the `agents/` and `logs/` subdirectories and writes `config.json` with the session ID, creation timestamp, and any additional config fields. Returns the session ID.
+
+Safe to call multiple times (idempotent) — existing directories are preserved. The `github.token` field is automatically stripped from the persisted `config.json` to prevent plaintext credential storage on disk.
+
+```js
+const id = await storage.init('session-abc', { repo: 'owner/repo', theme: 'transformers' });
+```
+
+### `storage.save(sessionId, agentName, data)` → `Promise<void>`
+
+Save agent state to disk. Writes atomically (temp file + rename) to prevent corruption from crashes during write.
+
+The following example shows saving an agent's runtime state, which is typically called during state transitions or periodic checkpoints.
+
+```js
+await storage.save('session-abc', 'optimus-0', {
+  identity: { name: 'optimus-0', provider: 'openai' },
+  state: 'working',
+  paneId: '%3',
+  claimedIssue: 42
+});
+```
+
+### `storage.load(sessionId, agentName)` → `Promise<object|null>`
+
+Load agent state from disk. Returns `null` if the agent file does not exist.
+
+```js
+const data = await storage.load('session-abc', 'optimus-0');
+if (data) {
+  console.log(`Agent ${data.identity.name} was in state: ${data.state}`);
+}
+```
+
+### `storage.agents(sessionId)` → `Promise<string[]>`
+
+List all agent names in a session (filenames without `.json` extension). Returns empty array for missing sessions.
+
+```js
+const names = await storage.agents('session-abc');
+// ['optimus-0', 'megatron-0']
+```
+
+### `storage.sessions()` → `Promise<string[]>`
+
+List all session IDs (directory names under `sessions/`). Returns empty array when no sessions exist.
+
+```js
+const ids = await storage.sessions();
+// ['session-abc', 'session-def']
+```
+
+### `storage.sessionDir(id)` → `string`
+
+Get the absolute path to a session's directory. Synchronous.
+
+```js
+storage.sessionDir('session-abc');
+// '/Users/you/.loreli/sessions/session-abc'
+```
+
+### `storage.remove(id)` → `Promise<void>`
+
+Remove a single session and all its contents (agents, logs, config). Does not throw when the session does not exist.
+
+```js
+await storage.remove('session-abc');
+```
+
+### `storage.prune(maxAge)` → `Promise<string[]>`
+
+Remove sessions older than `maxAge` milliseconds. Reads each session's `config.json` for the `created` timestamp. Sessions with missing or unparseable config are treated as stale and pruned. Returns an array of pruned session IDs.
+
+Called automatically at start when `cleanup.autoprune` is `true` (the default), using `cleanup.retention` as the max age.
+
+```js
+const pruned = await storage.prune(12 * 60 * 60 * 1000); // 12 hours
+// ['old-session-1', 'old-session-2']
+```
+
+### `storage.sweep()` → `Promise<string[]>`
+
+Remove known stray files from the loreli home root directory. Files like `mcp-ready` can end up at the root level when the MCP server's working directory happens to be `~/.loreli`. Returns an array of removed file names.
+
+```js
+const swept = await storage.sweep();
+// ['mcp-ready']
+```
+
+### `storage.atomic(filePath, content)` → `Promise<void>`
+
+Write a file atomically using temp file + rename. Used internally by `save()` and `init()`, but exposed for custom state files that need crash-safe writes.
+
+## Error Handling
+
+| Scenario | Behavior |
+|----------|----------|
+| Session directory missing | `load()` and `agents()` return `null` / `[]` — no throw |
+| No sessions directory | `sessions()` and `prune()` return `[]` — no throw |
+| Missing session for `remove()` | Silent no-op |
+| Missing/invalid `config.json` during prune | Session treated as stale and pruned |
+| `github.token` in config | Automatically stripped during `init()` |
+| File write interrupted | Atomic writes prevent partial files from appearing at the target path |
+
+## Testing
+
+```bash
+node --test packages/session/test/index.test.js
+```
+
+Tests use temporary directories and do not require tmux.

package/packages/session/src/index.js

@@ -0,0 +1,215 @@
+import { mkdir, writeFile, readFile, readdir, rename, rm, unlink, stat } from 'node:fs/promises';
+import { join } from 'node:path';
+import { homedir } from 'node:os';
+
+/**
+ * Persistent session storage for detached agents.
+ *
+ * Writes session and agent state to ~/.loreli/ (or LORELI_HOME)
+ * so agents survive orchestrator shutdown and can be reconnected.
+ *
+ * Directory layout:
+ * ```
+ * ~/.loreli/sessions/<id>/
+ *   config.json
+ *   agents/
+ *     optimus-0.json
+ *     megatron-0.json
+ *   registry.json
+ * ```
+ */
+export class Storage {
+  /**
+   * @param {object} [opts]
+   * @param {string} [opts.home] - Override the base directory.
+   */
+  constructor(opts = {}) {
+    /** @type {string} Base storage directory. */
+    this.home = opts.home ?? process.env.LORELI_HOME ?? join(homedir(), '.loreli');
+  }
+
+  /**
+   * Initialize a session directory.
+   *
+   * @param {string} id - Session ID.
+   * @param {object} [config] - Session config to persist.
+   * @returns {Promise<string>} The session ID.
+   */
+  async init(id, config = {}) {
+    const dir = this.sessionDir(id);
+    await mkdir(join(dir, 'agents'), { recursive: true });
+    await mkdir(join(dir, 'logs'), { recursive: true });
+
+    const safe = { ...config };
+    if (safe.github) {
+      safe.github = { ...safe.github };
+      delete safe.github.token;
+    }
+
+    const configPath = join(dir, 'config.json');
+    await this.atomic(configPath, JSON.stringify({
+      id,
+      created: new Date().toISOString(),
+      ...safe
+    }, null, 2));
+
+    return id;
+  }
+
+  /**
+   * Save agent state to disk.
+   *
+   * @param {string} sessionId - Session ID.
+   * @param {string} agentName - Agent identity name.
+   * @param {object} data - Agent state data.
+   * @returns {Promise<void>}
+   */
+  async save(sessionId, agentName, data) {
+    const filePath = join(this.sessionDir(sessionId), 'agents', `${agentName}.json`);
+    await this.atomic(filePath, JSON.stringify(data, null, 2));
+  }
+
+  /**
+   * Load agent state from disk.
+   *
+   * @param {string} sessionId - Session ID.
+   * @param {string} agentName - Agent identity name.
+   * @returns {Promise<object>} Parsed agent state data.
+   */
+  async load(sessionId, agentName) {
+    const filePath = join(this.sessionDir(sessionId), 'agents', `${agentName}.json`);
+    try {
+      const content = await readFile(filePath, 'utf8');
+      return JSON.parse(content);
+    } catch {
+      return null;
+    }
+  }
+
+  /**
+   * List all agent files in a session.
+   *
+   * @param {string} sessionId - Session ID.
+   * @returns {Promise<string[]>} Array of agent names.
+   */
+  async agents(sessionId) {
+    const dir = join(this.sessionDir(sessionId), 'agents');
+    try {
+      const files = await readdir(dir);
+      return files.filter(function isJson(f) { return f.endsWith('.json'); })
+        .map(function stripExt(f) { return f.replace('.json', ''); });
+    } catch {
+      return [];
+    }
+  }
+
+  /**
+   * List all session IDs.
+   *
+   * @returns {Promise<string[]>} Array of session ID strings.
+   */
+  async sessions() {
+    const dir = join(this.home, 'sessions');
+    try {
+      return await readdir(dir);
+    } catch {
+      return [];
+    }
+  }
+
+  /**
+   * Get the directory path for a session.
+   *
+   * @param {string} id - Session ID.
+   * @returns {string} Absolute path to the session directory.
+   */
+  sessionDir(id) {
+    return join(this.home, 'sessions', id);
+  }
+
+  /**
+   * Remove a single session and all its contents.
+   *
+   * @param {string} id - Session ID to remove.
+   * @returns {Promise<void>}
+   */
+  async remove(id) {
+    await rm(this.sessionDir(id), { recursive: true, force: true });
+  }
+
+  /**
+   * Prune sessions older than a given age.
+   *
+   * Reads each session's `config.json` for the `created` timestamp.
+   * Sessions older than `maxAge` milliseconds are removed. Sessions
+   * with missing or unparseable config are treated as stale and pruned.
+   *
+   * @param {number} maxAge - Maximum age in milliseconds.
+   * @returns {Promise<string[]>} IDs of pruned sessions.
+   */
+  async prune(maxAge) {
+    const ids = await this.sessions();
+    const now = Date.now();
+    const pruned = [];
+
+    for (const id of ids) {
+      let created = 0;
+      try {
+        const raw = await readFile(join(this.sessionDir(id), 'config.json'), 'utf8');
+        const config = JSON.parse(raw);
+        created = new Date(config.created).getTime();
+      } catch {
+        // Missing or invalid config — treat as infinitely old
+      }
+
+      if (now - created > maxAge) {
+        await this.remove(id);
+        pruned.push(id);
+      }
+    }
+
+    return pruned;
+  }
+
+  /**
+   * Remove known stray files from the loreli home root.
+   *
+   * Files like `mcp-ready` can end up at the home root level when
+   * the MCP server's cwd happens to be `~/.loreli`. This sweeps
+   * only files from a known allowlist — directories and unknown
+   * files are left untouched.
+   *
+   * @returns {Promise<string[]>} Names of removed files.
+   */
+  async sweep() {
+    const STRAYS = ['mcp-ready'];
+    const removed = [];
+
+    for (const name of STRAYS) {
+      const path = join(this.home, name);
+      try {
+        const s = await stat(path);
+        if (!s.isFile()) continue;
+        await unlink(path);
+        removed.push(name);
+      } catch {
+        // File doesn't exist — nothing to sweep
+      }
+    }
+
+    return removed;
+  }
+
+  /**
+   * Write a file atomically (temp file + rename).
+   *
+   * @param {string} filePath - Target file path.
+   * @param {string} content - File content.
+   * @returns {Promise<void>}
+   */
+  async atomic(filePath, content) {
+    const tmp = `${filePath}.tmp`;
+    await writeFile(tmp, content, 'utf8');
+    await rename(tmp, filePath);
+  }
+}

package/packages/test-utils/README.md

@@ -0,0 +1,96 @@
+# loreli/test-utils
+
+Shared test infrastructure for Loreli integration tests. Consolidates helpers from `hub/test/helpers.js` and `mcp/test/helpers.js` into a single canonical location.
+
+## API Reference
+
+### Constants
+
+| Name | Description |
+|------|-------------|
+| `TEST_REPO` | Test repository — set via `LORELI_TEST_REPO` env var (no default; tests skip when unset) |
+| `TEST_SESSION` | Tmux session name for test agents (`loreli-test-<pid>`) |
+
+## Test Modes
+
+Set `LORELI_TEST_MODE=unit` to disable agentic tests that require real CLI backends. This matters for CI environments where LLM CLIs are unavailable or expensive, and you should expect backend-dependent suites to skip with a clear reason when this mode is active.
+
+### Functions
+
+#### `testIdentity()` → Identity
+
+Create a stable test identity for integration tests. All automated test interactions use this identity.
+
+#### `ensureTestLabels(hub, repo)` → Promise\<void\>
+
+Ensure the test identity's labels exist in the test repository. Must be called once before scoped hub operations.
+
+#### `agenticSkipReason()` → string | false
+
+Return a skip reason when agentic tests are disabled.
+
+#### `skipWithoutToken(t)` → boolean
+
+Skip guard for tests requiring `GITHUB_TOKEN`.
+
+#### `skipWithoutTestRepo(t)` → boolean
+
+Skip guard for tests requiring `LORELI_TEST_REPO`.
+
+#### `skipWithoutBackends(t, backends)` → Promise\<boolean\>
+
+Skip guard for tests requiring real CLI backends (claude, codex, etc.) on PATH and a working tmux install. This guard skips immediately when `LORELI_TEST_MODE=unit`.
+
+#### `createHub()` → GitHubHub
+
+Create a real GitHubHub instance from the environment token.
+
+#### `createConfig(hub?, overrides?)` → Promise\<Config\>
+
+Create a real Config instance, optionally loading from the test repo.
+
+#### `createAgent(backends, registry, role, provider?, opts?)` → Promise\<object\>
+
+Discover real backends and create a real agent for the given provider and role. Agents are assigned to the `loreli-test` tmux session.
+
+#### `createTmpDir(prefix?)` → Promise\<string\>
+
+Create a temporary directory for test artifacts.
+
+#### `resetTestSession()` → Promise\<void\>
+
+Kill and recreate the test tmux session with retry logic.
+
+#### `killTestSession()` → Promise\<void\>
+
+Kill the test tmux session entirely.
+
+### Classes
+
+#### `Cleanup`
+
+Tracks GitHub resources (issues, PRs, branches, discussions) created during tests and provides `flush()` for best-effort cleanup.
+
+The example below demonstrates tracking a created issue for cleanup. This matters because integration tests create real GitHub resources, and you should expect `flush()` to close or delete tracked items without throwing even if some were already removed.
+
+```js
+const cleanup = new Cleanup(hub, TEST_REPO);
+cleanup.issues.push(issue.number);
+await cleanup.flush();
+```
+
+### Re-exports
+
+This package re-exports core classes used frequently in integration tests so test files can import from one place:
+
+- `GitHubHub` (from `loreli/hub`)
+- `Config` (from `loreli/config`)
+- `Identity`, `Registry` (from `loreli/identity`)
+- `BackendRegistry` (from `loreli/agent`)
+- `Storage` (from `loreli/session`)
+
+## Scope Boundary
+
+**In scope**: Test skip guards, GitHub resource cleanup, agent creation helpers, tmux session management, test identity.
+
+**Out of scope**: Test framework configuration, assertions, test runner setup.