@timqi/pier 0.1.14 → 0.3.0

package/README.md

@@ -9,9 +9,9 @@ something is worth showing.
 One instance, one account, your own machine. The agent runs shell commands in
 directories you name — meant for a machine you own, not a shared host.
 
-**Status: pre-release.** The version is `0.0.x`; the database schema is
-versioned from `0.0.1` on. Read `docs/deploy.md` before putting it anywhere
-reachable.
+**Status: pre-release.** Releases may still change behaviour and the
+database schema between versions; schema upgrades are one-way. Read
+`docs/deploy.md` before putting it anywhere reachable.
 
 ## Requirements
 
@@ -30,15 +30,35 @@ pier serve
 
 It listens on `127.0.0.1:3141` (`PORT`, `HOST`) and keeps everything under
 `~/.pier` (`PIER_HOME`): one SQLite database, the boards it serves, the
-`master.key` that seals stored credentials, and the Pi runtime with its
-session transcripts (`~/.pier/pi`, unless `PI_CODING_AGENT_DIR` says
-otherwise).
+`master.key` that seals stored credentials, the conversation's memory
+(`~/.pier/home`), and the Pi runtime with its session transcripts
+(`~/.pier/pi`, unless `PI_CODING_AGENT_DIR` says otherwise).
 
 **The first start generates a password and prints it once.** Lost it?
 `sqlite3 ~/.pier/db/pier.db 'DELETE FROM auth'` and restart. Open
-`http://localhost:3141`, sign in; **Console → Settings** configures Models,
-Agent, Channels, Vault, the public URL, password and master key; **New session**
-picks the directory the agent's shell runs in.
+`http://localhost:3141`, sign in; **Console → Settings** has Models, Agent,
+Channels, Vault, Tasks, Boards, Instance (public URL, accent, notifications,
+reload) and Security (master key, password, passkeys, signed-in devices).
+
+## How the conversation works
+
+The web chat is one conversation, Pier. Its current session answers,
+remembers and dispatches; it runs in `~/.pier/home`, which holds
+memory only: `MEMORY.md` (one-line facts) and daily notes in
+`memory/YYYY-MM-DD.md`.
+
+- Real work is delegated with `pier task run`: a worker for a small task, a
+  lead for a feature, which designs with you or builds with its own workers.
+  Each is its own session; its result comes back to Pier as a callback.
+- After an idle hour, or past 60K tokens, the next message starts a fresh
+  session, seeded with the memory, the open items, recent runs and the last
+  exchanges. The old transcripts stay readable.
+- `/status` lists what is open, `/new` starts a fresh session now, `/stop` stops
+  the reply in progress, `/skills` lists the skills. `%` works in place of
+  `/` (`%status`), because Slack's client eats an unregistered `/`.
+- One IM DM, on Lark or Slack, can be the conversation: Console → Settings →
+  Channels, the DM's row, "This DM is the conversation". Pier then
+  answers there, from the phone; other chats keep one session per thread.
 
 ## Configure Pi
 
@@ -58,14 +78,15 @@ Console → Settings:
 - **Models** — endpoints, API-key/OAuth login, pinned models. Credentials are
   sealed in Pier's SQLite database, never written back to `models.json`.
   **Test** sends one real request and shows the body as sent and the reply.
-- **Agent** — edits `SYSTEM.md`, `AGENTS.md`, `settings.json` and the
-  `models.json` structure, globally or per project scope (with that project's
-  `.pi/skills` and `.pi/extensions` listed). Changes apply when a session next
-  opens; saving recycles idle sessions; **Settings → Instance → Reload** does
-  the same for files changed elsewhere. Also here: Pier's own skills
-  (`pier web search|fetch` reaches the web through the provider's hosted
-  tools — Anthropic or OpenAI, no key of your own) and the
-  managed CLI tools (`rtk`, `rg`, `fd`, `wt`, `jq`, or your own as a
+- **Agent** — edits `SYSTEM.md`, `AGENTS.md` and the `models.json`
+  structure, globally or per project scope (with that project's
+  `.pi/skills` and `.pi/extensions` listed); `settings.json` is Pier's and
+  shown read-only. Changes apply when a session next opens; saving recycles
+  idle sessions; **Settings → Instance → Reload** does the same for files
+  changed elsewhere. Also here: packages (Pi's registry, **Add package**),
+  Pier's own skills as the built-in `pier` package (`pier web search|fetch`
+  reaches the web through the provider's hosted tools — Anthropic or OpenAI,
+  no key of your own), **Browse files**, and the managed CLI tools (`rtk`, `rg`, `fd`, `wt`, `jq`, or your own as a
   [ubix](https://github.com/timqi/ubix) block), installed into
   `~/.pier/tools/bin`, first on every session's PATH beside the `pier` shim
   Pier writes there at start.
@@ -79,7 +100,7 @@ store and renamed `auth.json.imported`; literal keys in `models.json` likewise
 (`models.json.imported`). The editor rejects plaintext keys or header values.
 Pi's provider environment variables are inherited from the Pier process; a
 systemd service does not inherit your shell, so put non-secret settings in a
-unit override and API keys in the Providers UI.
+unit override and API keys in Settings → Models.
 
 ## Run it as a service
 
@@ -118,8 +139,9 @@ npm test          # vitest
 
 ## Releases
 
-Pier asks `registry.npmjs.org` at boot and every 30 minutes; the footer version
-becomes `v0.0.1 → 0.0.2` when a release is out and opens a panel: source link,
+Pier asks `registry.npmjs.org` at boot and every 30 minutes; the version in
+the Settings header turns into a badge when a release is out (`Pier <version>
+is out`) and opens a panel: source link,
 **Update now**, **Update automatically** (idle instance only). Both drain first
 and hand the install to the updater unit; off systemd the panel says `pier
 update`. The updater writes `~/.pier/db/backups/pier.db.release-<version>.bak`
@@ -128,10 +150,10 @@ update`. The updater writes `~/.pier/db/backups/pier.db.release-<version>.bak`
 
 `main` is the only development line. `just release [patch|minor|major]` checks,
 tags and pushes; the tag builds and publishes to npm and a GitHub Release. The
-footer version is `package.json`'s.
+version shown is `package.json`'s.
 
 ## License
 
 [AGPL-3.0-only](LICENSE). Run it, change it, deploy it. If you offer a modified
 Pier to other people over a network, they are entitled to your source — the
-version in the footer links to this repository for exactly that reason.
+version in the Settings header links to this repository for exactly that reason.

package/dist/agent/config.js

@@ -4,7 +4,8 @@
 import { randomUUID } from "node:crypto";
 import { promises as fs } from "node:fs";
 import { join } from "node:path";
-import { isProviderApi, isThinkingLevel, validateEndpoint, validateProviderSetup } from "../core/types.js";
+import { isThinkingLevel } from "../core/types.js";
+import { isProviderApi, SNAPSHOT_FILES, validateEndpoint, validateProviderSetup } from "./types.js";
 import { pierPath } from "../paths.js";
 import { mergeSnapshotProviders, normalizeAgentSnapshot, snapshotProviders } from "./config-sync.js";
 const GLOBAL_FILES = ["SYSTEM.md", "AGENTS.md", "settings.json", "models.json"];
@@ -13,7 +14,6 @@ const PROJECT_FILES = ["AGENTS.md"];
 // go through writeDefaults, and are the only fields a snapshot carries. The
 // Console shows the file and never edits it; the rest of it is machine-local.
 const READONLY_FILES = ["settings.json"];
-const SNAPSHOT_FILES = ["SYSTEM.md", "AGENTS.md", "models.json", "settings.json"];
 /** What a first boot writes (docs/deploy.md says why each key): the list the
  *  Console installs into, Pi's attribution headers off — a server is not a
  *  person to survey — and a retry budget of minutes, because an unattended

package/dist/agent/events.js

@@ -1,7 +1,7 @@
 // Pure Pi-event → SessionEventPayload translation. Structurally typed: no Pi
 // imports, so it is unit-testable without Pi and Pi types never leak past the
 // seam. The golden table in events.test.ts is the mapping's spec.
-import { isThinkingLevel, MAX_STEP_OUTPUT } from "../core/types.js";
+import { isChatCommand, isThinkingLevel, MAX_STEP_OUTPUT } from "../core/types.js";
 /** An assistant message that calls a tool is work in progress, not a reply. */
 export const hasToolCalls = (message) => Array.isArray(message?.content) && message.content.some((part) => part.type === "toolCall");
 export function textOf(content) {
@@ -20,14 +20,36 @@ function systemOrigin(message) {
     const value = message.details;
     if (!value || typeof value !== "object")
         return null;
-    const { source: raw, ...origin } = value;
+    const { source: raw, cwd, ...origin } = value;
+    if (origin.kind === "session-seed") {
+        const { reason, previousSessionId } = origin;
+        return (reason === "first" || reason === "idle" || reason === "lost" || reason === "full" || reason === "new") &&
+            (previousSessionId === null || typeof previousSessionId === "string")
+            ? { kind: "session-seed", reason, previousSessionId }
+            : null;
+    }
+    if (origin.kind === "chat-command") {
+        const { command, sessions } = origin;
+        if (!isChatCommand(command))
+            return null;
+        // A malformed map costs the card its links, not the card.
+        const links = sessions && typeof sessions === "object" && !Array.isArray(sessions) && Object.values(sessions).every((v) => typeof v === "string")
+            ? { sessions: sessions }
+            : {};
+        return { kind: "chat-command", command, ...links };
+    }
     if (typeof origin.taskId !== "string" ||
         typeof origin.runId !== "string" ||
         (origin.sourceSessionId !== null && typeof origin.sourceSessionId !== "string"))
         return null;
     // A half-valid `source` drawn by the card is an `undefined` in a chip.
     const source = inputSource(raw);
-    const shape = { ...origin, ...(source ? { source } : {}) };
+    // A callback's, and relative paths resolve against it, so only an absolute one is kept.
+    const shape = {
+        ...origin,
+        ...(source ? { source } : {}),
+        ...(origin.kind === "task-callback" && typeof cwd === "string" && cwd.startsWith("/") ? { cwd } : {}),
+    };
     if (origin.kind === "task-delegation" || origin.kind === "task-callback") {
         return shape;
     }
@@ -168,6 +190,13 @@ export function toChatTurns(messages) {
             }
         }
         const text = textOf(m.content);
+        // Only the attempt that ended the turn: one Pi retried past is no failure.
+        if (m.role === "assistant" && m.stopReason === "error" && messages[i + 1]?.role !== "assistant") {
+            flush("assistant", hasTools ? "" : text, turnMetaAt(messages, i), undefined, m.timestamp);
+            turns[turns.length - 1].error = m.errorMessage || "unknown agent error";
+            candidate = undefined;
+            continue;
+        }
         // Tool-bearing messages are intermediate work, even when they include text.
         if (!text || hasTools)
             continue;

package/dist/agent/listing.js

@@ -80,16 +80,15 @@ function fold(acc, line, clean) {
 }
 /** A picker, not a results page. */
 const SEARCH_LIMIT = 8;
-/** A trigram token is one character, so `snippet()` is asked for its ceiling
- *  of 64 tokens; the default 12 fits one word. */
+/** Characters of context a snippet keeps around its first match. */
 const SNIPPET_CHARS = 64;
 /** Whitespace splits a query: a space is "and this too", not a character to
  *  find. A term is quoted where it has to be, so a phrase is still findable. */
 const termsOf = (query) => query.split(/\s+/).filter(Boolean);
-/** Delimited like `snippet()` (\u0001 … \u0002, `…` for a cut), so a surface
- *  draws one shape for both paths. Every term is marked where it first
- *  appears inside the window, as `snippet()` marks every phrase it kept.
- *  ASCII case folding: LIKE found the row by the same rule. */
+/** Cut here rather than by FTS `snippet()`, which can only cut the indexed
+ *  text, markup and all. Each match delimited \u0001 … \u0002, `…` for a cut;
+ *  every term is marked where it first appears inside the window. A match only
+ *  inside stripped markup leaves the message's opening. */
 function around(text, terms) {
     const lower = text.toLowerCase();
     const found = terms
@@ -98,7 +97,7 @@ function around(text, terms) {
         .sort((a, b) => a.at - b.at);
     const first = found[0];
     if (!first)
-        return text.slice(0, SNIPPET_CHARS);
+        return text.length > SNIPPET_CHARS ? `${text.slice(0, SNIPPET_CHARS)}…` : text;
     const start = Math.max(0, first.at - SNIPPET_CHARS / 2);
     const end = Math.min(text.length, first.at + first.length + SNIPPET_CHARS / 2);
     let out = start ? "…" : "";
@@ -115,14 +114,19 @@ function around(text, terms) {
 export class IndexedListing {
     dir;
     clean;
+    plain;
     #db;
     #statements;
     constructor(dir = join(defaultAgentDir(), "sessions"), db, 
     /** Handed in rather than imported: the header rule is core's, and agent/
      *  does not import core at runtime. */
-    clean = (text) => text) {
+    clean = (text) => text, 
+    /** Chat markup off what was said, before a snippet is cut (core/reply.ts):
+     *  applied at search time, so rows indexed before it are covered too. */
+    plain = (text) => text) {
         this.dir = dir;
         this.clean = clean;
+        this.plain = plain;
         this.#db = db;
     }
     /** Opened on the first scan: building a factory must not open the database. */
@@ -231,7 +235,7 @@ export class IndexedListing {
             return [];
         const rows = terms.every((term) => [...term].length >= 3)
             // Quoted: a term is a string to find, never FTS syntax.
-            ? sql(`SELECT session_id, role, at, snippet(session_fts, 0, char(1), char(2), '…', ${SNIPPET_CHARS}) AS snippet
+            ? sql(`SELECT session_id, role, at, text
          FROM session_fts WHERE text MATCH ? ORDER BY bm25(session_fts), at DESC`).iterate(terms.map((term) => `"${term.replaceAll('"', '""')}"`).join(" AND "))
             : sql(`SELECT session_id, role, at, text FROM session_fts WHERE ${terms.map(() => "text LIKE ? ESCAPE '\\'").join(" AND ")} ORDER BY at DESC`).iterate(...terms.map((term) => `%${term.replaceAll(/[\\%_]/g, "\\$&")}%`));
         const hits = [];
@@ -245,7 +249,7 @@ export class IndexedListing {
                 sessionId: row.session_id,
                 role: row.role,
                 at: row.at,
-                snippet: row.snippet ?? around(row.text ?? "", terms),
+                snippet: around(this.plain(row.text), terms),
             });
             if (hits.length >= limit)
                 break;

package/dist/agent/packages.js

@@ -5,17 +5,16 @@
 import { existsSync, promises as fs } from "node:fs";
 import { basename, dirname, join, relative, resolve } from "node:path";
 import { DefaultPackageManager, loadSkills, SettingsManager, } from "@earendil-works/pi-coding-agent";
-import { PackageError, } from "../core/types.js";
+import { PackageError, } from "./types.js";
 import { logger } from "../log.js";
 import { defaultAgentDir } from "./config.js";
 const log = logger("packages");
 /** Set by how long a released fix may sit unnoticed, like update.ts. */
 const CHECK_EVERY_MS = 24 * 60 * 60_000;
 const KINDS = { extensions: "extension", skills: "skill" };
-/** Written by `rtk init -g --agent pi` (the rtk block's `post_install` hook): a `local` file whose
- *  switch is the rtk tool's, so a settings.json pattern would fight the tool. */
+/** Written by `rtk init -g --agent pi` (the rtk block's `post_install` hook): the rtk tool's, so it
+ *  is left out of `local` — the Console switches it as that tool, and a settings.json pattern would fight it. */
 const RTK_FILE = join("extensions", "rtk.ts");
-const RTK_STATE = "installed by the rtk tool";
 export const kindOf = (source) => source.startsWith("npm:") ? "npm" : /^(git:|(https?|ssh|git):\/\/)/.test(source) ? "git" : "path";
 /** A trailing `@ref` on an npm or git source; the leading `@` of a scope is not one. */
 export const pinnedRef = (source) => {
@@ -120,13 +119,12 @@ export class PiPackageStore {
         row("local", "local", "global", this.agentDir);
         for (const key of Object.keys(KINDS)) {
             for (const r of resolved[key]) {
+                if (r.path === rtkPath)
+                    continue;
                 const own = r.metadata.origin === "top-level";
                 const scope = r.metadata.scope === "project" ? "project" : "global";
-                const resource = resourceRow(KINDS[key], r);
-                if (r.path === rtkPath)
-                    Object.assign(resource, { state: RTK_STATE, locked: true });
                 row(own ? "local" : r.metadata.source, own ? "local" : kindOf(r.metadata.source), scope, r.metadata.baseDir ?? null)
-                    .resources.push(resource);
+                    .resources.push(resourceRow(KINDS[key], r));
             }
         }
         // Configured but unresolved (not installed, or empty): still a row.
@@ -277,8 +275,6 @@ export class PiPackageStore {
     async setEnabled(change) {
         const { source, kind, path, enabled, cwd } = change;
         const { pkg, resource } = await this.#find(change);
-        if (resource.locked)
-            throw new PackageError("refused", `${resource.name} is ${resource.state} — its switch is under Tools`);
         if (pkg.kind === "pier") {
             this.pier.settings.setSkillsOff(withName(this.pier.settings.get().skillsOff, resource.name, !enabled));
         }