herdrchat 1.2.0 → 1.2.1

package/docs/guide.md

@@ -2,9 +2,11 @@
 
 How herdrchat works and every setting, for when the README is not enough.
 
-herdrchat is a chat front-end for hc: every terminal hc runs, followed and answered in a
-chat app. Feishu is the first (one bot); Slack, Discord and Telegram may follow, on the
-same core (hc events, a reader per agent, dialogs, the overview) with an adapter each.
+herdrchat is a chat front-end for hc: every herdr-app tab, followed and answered in a chat
+app, Feishu or Telegram. Both sit on the same core (hc events, a reader per agent,
+dialogs, the overview), each behind one contract (`docs/architecture.md`); Slack and
+Discord may follow the same way. A profile is one chat app (`HERDRCHAT_CHAT`); to use
+both, run a profile each.
 
 ## Feishu
 
@@ -21,6 +23,24 @@ herdr-app, moved into one Feishu bot:
 Everything comes from hc, so it works the same for Claude Code, codex, pi or any program
 hc shows, on this machine or on a machine hc has added. Nothing is installed in the agents.
 
+## Telegram
+
+A Telegram bot cannot make groups, so it uses topics in your private chat with the bot
+(topic mode must be on in @BotFather):
+
+| herdr-app | Telegram |
+|---|---|
+| a tab | a topic named `<space> / <tab>` in the bot's private chat; type there to talk to it |
+| a space | one topic icon color for all of its topics (Telegram offers six) |
+| a terminal that is gone | its topic stays, its name starting with 🔚 (taken off if it comes back) |
+| herdr itself | the chat's main thread: the overview message, edited in place; `/ls`, `/new`, `/kill`; `/` shows the commands |
+
+What differs: an image goes as its own photo after the text (a message cannot hold one);
+a dialog's free answer is typed in the topic (a message has no text box); there are no
+tags, labels, pins or `/auth` (a bot cannot act as you). Telegram takes about one message
+a second in a chat, so the bot queues and waits as Telegram asks. Setting it up:
+`skills/herdrchat/references/telegram.md`.
+
 ## How it works: hc, plus a reader per agent
 
 **hc is the backbone.** The bot follows `hc events --json` for each session (format:
@@ -36,13 +56,14 @@ each agent a reader reads the agent's own records. The status hc reports names t
 | `status.app` | reader | reads |
 |---|---|---|
 | `claude-code` | `lib/core/readers/claude.mjs` | its transcript: the terminal's foreground pid names the session in `~/.claude/sessions/<pid>.json` |
-| anything else | none (default) | the screen, through `hc read` |
+| anything else | none | nothing by itself; what you type gets its screen in reply |
 
 On another machine, a reader reads the same files through `hc exec --on`. To add an
-agent, write `lib/core/readers/<agent>.mjs` (the contract is at the top of
-`lib/core/readers/index.mjs`) and list it there.
+agent: `docs/architecture.md`, "Adding a harness".
+
+## What a room shows
 
-## What a group shows
+A room is a Feishu group or a Telegram topic.
 
 - A program with a reader:
   - while it works: one "⌛️ Working…" card per turn (then "✅ Worked"), edited every 5
@@ -50,9 +71,12 @@ agent, write `lib/core/readers/<agent>.mjs` (the contract is at the top of
     mid-turn) as **You:**, and each tool it calls as one line (`▸ Bash: Run the tests`).
     It shows the program is alive, as herdr-app does, and since an edit is not a new
     message, the phone does not ring every few seconds;
-  - when the turn ends: its last answer, as a new message. Without `/auth` you are
-    notified when a turn starts and when it is done; with it, groups are muted and only
-    an @ under the answer rings (see "Optional: let the bot act as you").
+  - when the turn ends: its last answer, as a new message. In a space of
+    `HERDR_MUTED_SPACES` (all, by default) only "needs you" and "done" ring: in Feishu
+    the groups are muted (after `/auth`) and an @ under the answer rings; in Telegram
+    the posts go silently and a short reply rings. `HERDR_SILENT_SPACES` rings nothing.
+  - A message another terminal sent with `hc send` shows as **From t7 review**, not as
+    yours (hc wraps it in `<hc from="…">`, for any agent).
   - Images too: ones it looked at (a screenshot, an image file it read) and ones you
     pasted in the terminal, shown in the card.
   - What you sent from the group is not repeated.
@@ -63,15 +87,21 @@ agent, write `lib/core/readers/<agent>.mjs` (the contract is at the top of
   a red "⚠️ Needs you" card. When the screen shows a numbered menu (as Claude Code's
   permission and question dialogs do), the card shows only the dialog, as text, with a
   button per option; a multi-select also has "Next ⇥", a "Type something" option a box to
-  type in, and every card "Esc". Without a menu the card shows the screen and how to
-  answer by typing. Once the dialog is gone the card turns to "✅ Done".
-
-## Answering a dialog from Feishu
-
-A button presses that option's number key in the terminal (Next presses tab, Esc esc).
+  type in (Feishu; in Telegram you type the answer in the topic), and every card "Esc".
+  Without a menu the card shows the last lines of the screen and how to answer by typing.
+  Once the dialog is gone the card turns to "✅ Done". A card is posted only for a dialog
+  that still waits: one answered while its screen was read gets none.
+
+## Answering a dialog from the chat
+
+A button presses that option's number key in the terminal (Esc presses esc). A question
+with previews (a box beside its options) takes no number, so there the button moves with
+the arrows from the cursor to that option and presses Enter. In a multi-select, a box is
+ticked on the card only, at once; Next sends them all: each box that differs from the
+screen is pressed, then tab.
 Before it presses anything, the bot reads the screen again and checks that it still
-shows the same dialog (a hash of it, from its top border to its last option) and the
-same option, so an old card can never answer a newer dialog that looks alike. After the
+shows the same dialog (a hash of it, from its top border to its last option, without the
+cursor, the check marks and the preview box) and the same option, so an old card can never answer a newer dialog that looks alike. After the
 key, the card turns into the dialog's next step (a toggled box, the next question, the
 review) or says what was picked. Text you type in the group while it waits goes into its
 "Type something" box when it has one.
@@ -123,14 +153,17 @@ delivered (the terminal has ended) says so plainly.
   recalled), `/new <session> <space>/<tab> [command]` (start one; default `claude`;
   `--cwd <dir>` first to pick its folder), `/kill <session> <address>`, `/auth` (see
   below).
-- In a terminal's group: any text goes to it (marked THINKING once typed in, then OK when
-  the turn that took it is over); an image (alone or in a rich text message) is sent
+- In a terminal's room: any text goes to it (marked once typed in, and again when the
+  turn that took it is over); an image (alone or in a rich text message) is sent
   beside the terminal with `hc put`, on its machine, and its path is typed in with your
   text, so an agent like Claude Code opens it; `/screen`, `/answer <text>` (type into the
-  dialog it waits on, as is), `/esc`. Dialog cards have buttons.
+  dialog it waits on, as is), `/esc`, `/history` (the conversation so far, as a file and
+  a folded post). Dialog cards have buttons.
 
 ## Set up
 
+Telegram: `skills/herdrchat/references/telegram.md`. Feishu:
+
 Steps 1, 2, 5 and 6 are in the Feishu developer console; the herdrchat skill
 (`/herdrchat setup`, see "Skills") does them with an agent, or guides you through them.
 
@@ -152,6 +185,24 @@ Steps 1, 2, 5 and 6 are in the Feishu developer console; the herdrchat skill
 7. Open `https://applink.feishu.cn/client/bot/open?appId=<app id>` and send anything.
    The bot answers with your `open_id`; put it in `FEISHU_OWNER_OPEN_ID` and run
    `herdrchat start` again.
+8. Once it works: `herdrchat install` (see "Running as a service").
+
+## Running as a service
+
+`herdrchat start` runs the bot in an hc terminal, which ends when that hc core stops,
+and nothing starts it after a reboot. `herdrchat install` makes it a service of the
+system instead: a launchd agent on macOS, a systemd user unit on Linux. It starts at
+login, comes back by itself after a crash, and does not depend on any hc core. It runs
+node and the bot by their full paths with only the variables it needs (hc's path, PATH,
+the profile, the folders), so no agent's variables come along. With it installed,
+`start`, `stop` and `status` go through the service; `herdrchat uninstall` removes it.
+Each profile is its own service (`herdrchat install --profile telegram`).
+
+- After a node upgrade that moves node, run `herdrchat install` again; `herdrchat status`
+  says when the service's node is gone.
+- On Linux, `loginctl enable-linger` keeps it running after you log out.
+- What the process prints before its log is open (a crash at start) goes to
+  `logs/service.out.log` in the state folder.
 
 ## Optional: let the bot act as you
 
@@ -181,9 +232,10 @@ refreshed, the bot says so once and falls back.
 ## Skills
 
 `skills/herdrchat/` is one skill for an agent (Claude Code or another that reads
-`SKILL.md`): `/herdrchat setup`, `status`, `start`, `stop`, `logs`, `auth`, `config`, `fix`
-and `feishu <task>` (the developer console, with Ego Lite from https://lite.ego.app/ or as
-a guided walk-through). Details per part are in `skills/herdrchat/references/`. To use it
+`SKILL.md`): `/herdrchat setup`, `status`, `start`, `stop`, `install`, `logs`, `auth`,
+`config`, `fix`, `feishu <task>` (the developer console, with Ego Lite from
+https://lite.ego.app/ or as a guided walk-through) and `telegram <task>` (the bot, from
+@BotFather to the first topic). Details per part are in `skills/herdrchat/references/`. To use it
 with Claude Code, link it where skills are read, for example
 `ln -s "$PWD/skills/herdrchat" ~/.claude/skills/herdrchat`.
 
@@ -191,7 +243,7 @@ with Claude Code, link it where skills are read, for example
 
 Everything the bot logs goes to `logs/<YYYY-MM-DD>.log` in the state folder (one file a
 day, the last 14 kept) as `<time> <LEVEL> <text>`, besides its terminal: `grep ERROR
-~/.local/state/herdrchat/default/logs/*.log` finds
+~/.local/state/herdrchat/<profile>/logs/*.log` finds
 what went wrong, a retry that recovered is a `WARN`, and a crash leaves its stack there
 before it is started again. Tokens and secrets are never logged. The
 skill's `references/ops.md` explains the lines.
@@ -199,12 +251,15 @@ skill's `references/ops.md` explains the lines.
 ## Limits
 
 - Narration blocks are a condensed version of the text Claude showed, not the exact words.
-- Only Claude Code has a reader so far. codex, pi and the rest get their screen, at turn
-  end only, until each has a reader.
+- Only Claude Code has a reader so far. codex, pi and the rest post nothing by themselves
+  until each has a reader; what you type to them gets their screen in reply.
 - Only text, images and rich text are passed on; files, stickers and the rest are not.
 - An image that fails to upload three times is not sent; the group says so and why.
 - Feishu takes tag names of at most 16 characters; a longer session or machine name is cut
   at its last `-`, `_` or `.` that fits (`build-agent-server` becomes `build-agent`).
 - Avatars are drawn with `sips`, so only on macOS; elsewhere groups keep Feishu's default.
+- Telegram: a bot cannot mute a topic for you; a topic you mute in Telegram stays fully
+  quiet, "needs you" included. Closing a topic is documented for supergroups only, so an
+  ended topic is marked by its name.
 - Dialog buttons come from the screen's numbered menu; a dialog drawn some other way gets
   the screen and `/answer`, `/esc`.

package/lib/chats/feishu/client.mjs

@@ -258,7 +258,8 @@ export async function renameChat(client, chatId, name) {
 // Messages in (im.message.receive_v1) and card clicks (card.action.trigger), both over
 // the long connection. A click must be answered within 3 seconds; what `onCardAction`
 // returns ({toast, card}) is that answer.
-export function startEvents(cfg, onMessage, onCardAction, onReady = () => {}) {
+// onConnect runs at every connection, the first and each one after a drop.
+export function startEvents(cfg, onMessage, onCardAction, onReady = () => {}, onConnect = () => {}) {
   const ws = new Lark.WSClient({
     appId: cfg.appId,
     appSecret: cfg.appSecret,
@@ -266,9 +267,13 @@ export function startEvents(cfg, onMessage, onCardAction, onReady = () => {}) {
     onReady: () => {
       console.log('[feishu] connected');
       onReady();
+      onConnect();
     },
     onReconnecting: () => console.warn('[feishu] reconnecting'),
-    onReconnected: () => console.log('[feishu] reconnected'),
+    onReconnected: () => {
+      console.log('[feishu] reconnected');
+      onConnect();
+    },
     onError: (err) => console.error(`[feishu] connection failed: ${err.message}`),
   });
   ws.start({

package/lib/chats/feishu/index.mjs

@@ -401,13 +401,40 @@ export function create({ cfg, store, log, logWarn, logError, stateDir, lane, sco
 
   // Messages (deduplicated, the owner's only) and card clicks, turned into the core's
   // shape: {room (null: the direct chat), where, message, text, images}.
+  //
+  // Feishu wants an event handled within 3 seconds or it pushes it again, so a message is
+  // taken at once and then handled in its chat's own queue (in order). Whether Feishu
+  // sends again what came while the bot was offline is not documented, so the bot looks
+  // itself: per chat it keeps the time of the last message it handled (state.json
+  // feishuRead), and at every connection it lists what came after that and handles what
+  // it missed.
   const seen = new Set();
+  const queues = new Map(); // chat -> its last queued message
+  function queued(chatId, fn) {
+    const next = (queues.get(chatId) || Promise.resolve()).then(fn).catch((err) => logError(`feishu message in ${chatId}:`, err));
+    queues.set(chatId, next);
+    next.then(() => { if (queues.get(chatId) === next) queues.delete(chatId); });
+  }
+  function markRead(chatId, at) {
+    const state = read();
+    if ((state.feishuRead?.[chatId] || 0) >= at) return;
+    state.feishuRead = { ...(state.feishuRead || {}), [chatId]: at };
+    write(state);
+  }
+
   function start({ onMessage, onAction, onReady }) {
-    const handleMessage = async (data) => {
+    const take = (data) => {
       const msg = data.message;
       if (seen.has(msg.message_id)) return;
       seen.add(msg.message_id);
-      if (seen.size > 1000) seen.delete(seen.values().next().value);
+      if (seen.size > 2000) seen.delete(seen.values().next().value);
+      queued(msg.chat_id, async () => {
+        await handleMessage(data);
+        markRead(msg.chat_id, Number(msg.create_time) || Date.now());
+      });
+    };
+    const handleMessage = async (data) => {
+      const msg = data.message;
       const openId = data.sender?.sender_id?.open_id;
       if (!cfg.ownerOpenId) {
         log(`owner not set; message from open_id ${openId}`);
@@ -437,7 +464,30 @@ export function create({ cfg, store, log, logWarn, logError, stateDir, lane, sco
       if (out.card?.kind === 'dialog') res.card = { type: 'raw', data: menuCard({ title: out.card.dialog.title, markdown: out.card.dialog.markdown, menu: out.card.dialog.menu, value: out.card.value }) };
       return res;
     };
-    startEvents(cfg, handleMessage, handleAction, onReady);
+    // What came while the bot was away: the owner's messages after the last one handled,
+    // in every chat it reads (a chat with none handled yet starts from now).
+    const catchUp = async () => {
+      if (!cfg.ownerOpenId) return;
+      const state = read();
+      const chats = new Set([...Object.values(state.chats || {}).filter((c) => !c.ended).map((c) => c.chat), ...Object.keys(state.feishuRead || {})]);
+      for (const chatId of chats) {
+        const since = state.feishuRead?.[chatId];
+        if (!since) {
+          markRead(chatId, Date.now());
+          continue;
+        }
+        try {
+          const res = await client.im.v1.message.list({ params: { container_id_type: 'chat', container_id: chatId, start_time: String(Math.floor(since / 1000)), sort_type: 'ByCreateTimeAsc', page_size: 50 } });
+          const missed = (res?.data?.items || []).filter((m) => Number(m.create_time) > since && m.sender?.sender_type === 'user' && m.sender?.id === cfg.ownerOpenId && !seen.has(m.message_id));
+          if (missed.length) log(`feishu: ${missed.length} message(s) in ${chatId} came while the bot was away`);
+          const p2p = !Object.values(state.chats || {}).some((c) => c.chat === chatId);
+          for (const m of missed) take({ message: { message_id: m.message_id, chat_id: chatId, chat_type: p2p ? 'p2p' : 'group', message_type: m.msg_type, content: m.body?.content || '{}', create_time: m.create_time }, sender: { sender_id: { open_id: m.sender.id } } });
+        } catch (err) {
+          logWarn(`feishu: could not look for missed messages in ${chatId}: ${err.message}`);
+        }
+      }
+    };
+    startEvents(cfg, take, handleAction, onReady, () => catchUp().catch((err) => logError(`feishu catch-up: ${err.message}`)));
     setInterval(() => asYou().catch((err) => logError(`as you: ${err.message}`)), YOU_CHECK_MS);
     asYou().catch((err) => logError(`as you: ${err.message}`));
   }

package/lib/chats/telegram/index.mjs

@@ -344,9 +344,12 @@ export function create({ cfg, store, log, logWarn, logError }) {
   }
 
   // Long polling: getUpdates holds up to POLL_SECONDS for news; a failure waits RETRY_MS
-  // and tries again (409: another copy of the bot polls the same token).
+  // and tries again (409: another copy of the bot polls the same token). Telegram keeps
+  // what the bot has not confirmed for 24 hours, so nothing sent while it was away is
+  // lost; which update it handled last is kept in state.json (telegramOffset), so after a
+  // restart nothing is handled twice.
   function start({ onMessage, onAction, onReady = () => {} }) {
-    let offset = 0;
+    let offset = read().telegramOffset || 0;
     let connected = false;
     (async () => {
       for (;;) {
@@ -365,13 +368,17 @@ export function create({ cfg, store, log, logWarn, logError }) {
           }
           const updates = await tg.call('getUpdates', { offset, timeout: POLL_SECONDS, allowed_updates: ['message', 'callback_query'] }, { timeoutMs: (POLL_SECONDS + 15) * 1000 });
           for (const u of updates) {
-            offset = u.update_id + 1;
             try {
               if (u.message) await handleMessage(u.message, onMessage);
               if (u.callback_query) await handleClick(u.callback_query, onAction);
             } catch (err) {
               logError(`telegram update ${u.update_id}:`, err);
             }
+            // saved once handled: a crash halfway handles it again, never skips it
+            offset = u.update_id + 1;
+            const state = read();
+            state.telegramOffset = offset;
+            write(state);
           }
         } catch (err) {
           logError(`telegram poll: ${err.message}`);

package/lib/core/engine.mjs

@@ -797,6 +797,7 @@ export function createEngine({ cfg, chat, hc, readers, store, log, logWarn, logE
     if (now !== was) since.set(k, Date.now());
     if (now !== 'blocked') {
       postedDialog.delete(k);
+      if (now) sendWaiting(k);
       if (chat.pinned().includes(k)) later(session, address, 'unpin', () => chat.unpin(k));
       if (openCard.has(k)) later(session, address, 'dialog', () => closeCard(k, '✔ Answered.'));
       return;
@@ -888,6 +889,23 @@ export function createEngine({ cfg, chat, hc, readers, store, log, logWarn, logE
     }
   }
 
+  // Messages sent while it waited on a dialog with no box to type in, per terminal, kept
+  // in state.json (waiting) so a restart does not lose them; they go in, in order, once it
+  // no longer waits (onStatus, and at start).
+  function hold(k, m) {
+    const state = read();
+    state.waiting = { ...(state.waiting || {}), [k]: [...(state.waiting?.[k] || []), m] };
+    write(state);
+  }
+  function sendWaiting(k) {
+    const state = read();
+    const held = state.waiting?.[k];
+    if (!held?.length) return;
+    delete state.waiting[k];
+    write(state);
+    for (const m of held) laneOf(k, 'held message', () => typeInto(m.where, m.message, m.session, m.address, m.text));
+  }
+
   async function typeInto(where, message, session, address, text) {
     const record = views.get(session)?.terminals.get(address);
     if (!record || record.state !== 'live') {
@@ -908,16 +926,21 @@ export function createEngine({ cfg, chat, hc, readers, store, log, logWarn, logE
       return;
     }
     const answer = text.match(/^\/answer\s+([\s\S]+)$/);
-    if (!answer && agent && record.status?.state === 'blocked') {
+    // whether it waits on a dialog now: asked of hc, since the status followed from its
+    // events can be a moment behind (a dialog just answered still looks blocked)
+    const waits = !answer && agent && (await hc.getTerminal(session, address).catch(() => record))?.status?.state === 'blocked';
+    if (waits) {
       // It waits on a dialog: words go into its "Type something" box when it has one;
-      // otherwise the dialog card comes again, to pick there.
+      // otherwise they wait for the dialog to be answered, then go in (never dropped).
       const menu = menuOf(await hc.screenTail(session, address, SCREEN_LINES));
       if (menu?.free) {
         await answerFree(session, address, menu, text);
         log(`-> ${session}/${address}: answered ${text.slice(0, 80)}`);
         await markWorking(key(session, address), message);
       } else {
-        await postBlocked(session, address, record, { again: true });
+        hold(key(session, address), { where, message, session, address, text });
+        log(`-> ${session}/${address}: held until its dialog is answered: ${text.slice(0, 80)}`);
+        await chat.reply(where, 'It asks something first (the card above). Your message goes in once that is answered.');
       }
       return;
     }
@@ -1057,6 +1080,7 @@ export function createEngine({ cfg, chat, hc, readers, store, log, logWarn, logE
       const target = terminalOfRoom(room);
       if (!target) {
         log(`message in a room this bot did not make: ${room}`);
+        await chat.reply(where, 'Not delivered: this room is not a terminal\'s. Each terminal has its own room; the direct chat takes /ls, /new and /kill.');
         return;
       }
       if (views.get(target.session)?.terminals.get(target.address)?.state !== 'live') {
@@ -1083,6 +1107,11 @@ export function createEngine({ cfg, chat, hc, readers, store, log, logWarn, logE
     for (const session of cfg.sessions) watch(session);
     ready.then(() => setTimeout(async () => {
       await endGone().catch((err) => logError(`end gone rooms: ${err.message}`));
+      // messages held before a restart, for terminals that no longer wait
+      for (const k of Object.keys(read().waiting || {})) {
+        const address = addressOfKey(k);
+        if (!cfg.sessions.some((s) => views.get(s)?.terminals.get(address)?.status?.state === 'blocked')) sendWaiting(k);
+      }
       // pins left from before a restart, for terminals that no longer wait
       for (const k of chat.pinned()) {
         const address = addressOfKey(k);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "herdrchat",
-  "version": "1.2.0",
+  "version": "1.2.1",
   "description": "Chat front-ends for hc: every terminal hc runs, in the chat app you use",
   "repository": {
     "type": "git",

package/skills/herdrchat/references/telegram.md

@@ -39,6 +39,9 @@ Do these in order; the user does the Telegram steps, you check each one.
    takes about one message a second in a chat, so twenty tabs take twenty seconds. A new
    topic gets the conversation before it (`HERDR_HISTORY`); `/history` in a topic posts it
    again.
+7. **Make it a service.** Once it works: `herdrchat install --profile telegram`. It then
+   starts at login and comes back by itself after a crash, whatever happens to the hc
+   cores; `herdrchat status --profile telegram` says it runs as the service.
 
 ## What rings