openclaw-scheduler 0.2.0

package/CONTRIBUTING.md

@@ -0,0 +1,73 @@
+# Contributing
+
+## Scope
+
+Contributions should improve one of these areas:
+
+- runtime reliability
+- workflow and queue semantics
+- installation and service management
+- package/install ergonomics
+- documentation and tests
+
+## Ground Rules
+
+- preserve durable runtime behavior
+- do not remove backward-compatible CLI or schema behavior casually
+- update docs when installation or runtime behavior changes
+- update tests when changing scheduler semantics, payload validation, or delivery behavior
+
+## Development
+
+```bash
+npm install
+npm test
+npm run lint
+```
+
+### Local Verification Gate
+
+Before pushing or opening a PR, run the full local gate:
+
+```bash
+npm run verify:local
+```
+
+This runs, in order:
+
+1. Lint (`eslint`)
+2. TypeScript declaration smoke tests
+3. Full test suite (in-memory SQLite) -- must end with **0 failed**
+4. Coverage floor checks (statement, branch, function, line)
+
+The same gate runs automatically via `prepublishOnly` before any `npm publish`.
+
+If you add new features or fix bugs, add tests. The test count should only go up. Coverage expectations are enforced by the verify script -- if you drop below the floor, the gate fails.
+
+## Branch Model
+
+All PRs target `main`. There are no long-lived feature branches.
+
+## Release Process
+
+1. `npm run verify:local` -- must pass completely
+2. `npm version <patch|minor|major>`
+3. `npm publish` -- `prepublishOnly` re-runs the verification gate
+4. Push the version commit and tag: `git push && git push --tags`
+
+### Agent-Facing Documentation
+
+The following files ship in the npm package for agent adoption:
+
+- `AGENTS.md` -- discovery flow, working rules, CLI commands
+- `CONTEXT.md` -- repo positioning, design bias
+- `JOB-QUICK-REF.md` -- copy-paste job patterns, field reference
+- `docs/` -- gateway contract, trust architecture, ADRs
+
+Update these when adding new features or changing the CLI API.
+
+## Pull Requests
+
+- explain whether the change affects runtime behavior, package/install behavior, or both
+- call out migration or compatibility risk explicitly
+- include verification steps

package/IMPLEMENTATION_SPEC.md

@@ -0,0 +1,170 @@
+# Implementation Spec — Scheduler v5 Features
+
+## New Schema (consolidated into schema baseline)
+
+### Jobs table — new columns:
+```sql
+ALTER TABLE jobs ADD COLUMN delivery_guarantee TEXT DEFAULT 'at-most-once';  -- 'at-most-once'|'at-least-once'
+ALTER TABLE jobs ADD COLUMN job_class TEXT DEFAULT 'standard';               -- 'standard'|'pre_compaction_flush'
+ALTER TABLE jobs ADD COLUMN approval_required INTEGER DEFAULT 0;             -- HITL gate
+ALTER TABLE jobs ADD COLUMN approval_timeout_s INTEGER DEFAULT 3600;
+ALTER TABLE jobs ADD COLUMN approval_auto TEXT DEFAULT 'reject';             -- 'approve'|'reject'
+ALTER TABLE jobs ADD COLUMN context_retrieval TEXT DEFAULT 'none';           -- 'none'|'recent'|'hybrid'
+ALTER TABLE jobs ADD COLUMN context_retrieval_limit INTEGER DEFAULT 5;
+```
+
+### Runs table — new columns:
+```sql
+ALTER TABLE runs ADD COLUMN context_summary TEXT;   -- JSON: {messages_injected,scope,aliases_resolved,...}
+ALTER TABLE runs ADD COLUMN replay_of TEXT;          -- run id if this is a crash replay
+```
+
+### Messages table — new column:
+```sql
+ALTER TABLE messages ADD COLUMN owner TEXT;          -- originator of typed message
+```
+(kind enum extends to include: 'decision','constraint','fact','preference' alongside existing)
+
+### New table: approvals
+```sql
+CREATE TABLE IF NOT EXISTS approvals (
+  id              TEXT PRIMARY KEY,
+  job_id          TEXT NOT NULL REFERENCES jobs(id) ON DELETE CASCADE,
+  run_id          TEXT REFERENCES runs(id) ON DELETE SET NULL,
+  status          TEXT NOT NULL DEFAULT 'pending',    -- pending|approved|rejected|timed_out
+  requested_at    TEXT NOT NULL DEFAULT (datetime('now')),
+  resolved_at     TEXT,
+  resolved_by     TEXT,                               -- 'operator'|'timeout'|'api'
+  notes           TEXT
+);
+CREATE INDEX IF NOT EXISTS idx_approvals_status ON approvals(status) WHERE status = 'pending';
+CREATE INDEX IF NOT EXISTS idx_approvals_job ON approvals(job_id);
+```
+
+### schema_migrations: baseline includes these fields/tables
+
+---
+
+## Function Signatures (contracts between modules)
+
+### approval.js (NEW FILE) exports:
+```js
+export function createApproval(jobId, runId)           // returns approval record
+export function getApproval(id)                         // by approval id
+export function getPendingApproval(jobId)               // latest pending for a job
+export function listPendingApprovals()                  // all pending
+export function resolveApproval(id, status, resolvedBy, notes)  // approve/reject/timed_out
+export function getTimedOutApprovals()                  // pending approvals past timeout
+export function pruneApprovals(retentionDays)           // clean old resolved approvals
+```
+
+### retrieval.js (NEW FILE) exports:
+```js
+export function getRecentRunSummaries(jobId, limit)     // last N run summaries (non-null)
+export function searchRunSummaries(jobId, query, limit) // hybrid: substring + TF-IDF scoring
+export function buildRetrievalContext(job)               // returns string to inject into prompt (or '')
+```
+
+### jobs.js — add to createJob/updateJob allowed fields:
+delivery_guarantee, job_class, approval_required, approval_timeout_s, approval_auto, context_retrieval, context_retrieval_limit
+
+### runs.js — new export:
+```js
+export function updateContextSummary(runId, summaryObj) // store JSON context_summary
+```
+
+### messages.js — update:
+- sendMessage: accept `owner` field
+- getInbox: sort by typed priority (constraint > decision > fact > task > preference > text/other)
+- New kinds accepted: 'decision','constraint','fact','preference'
+
+### dispatcher.js — new functions:
+```js
+async function replayOrphanedRuns()        // called from main() after initDb, before tick loop
+async function checkApprovals()            // called from tick(), checks timeouts + approved gates
+// Note: context metadata is built inline within buildJobPrompt() and returned as contextMeta
+```
+
+### cli.js — new commands:
+- `jobs approve <job-id>` — resolve pending approval as approved
+- `jobs reject <job-id> [reason]` — resolve as rejected
+- `approvals list` — list pending
+- `approvals pending` — alias
+
+---
+
+## Feature Details
+
+### F1: Delivery Semantics Contract
+- New field `delivery_guarantee` on jobs ('at-most-once' default | 'at-least-once')
+- at-most-once: current behavior. On crash, orphaned runs marked 'crashed'.
+- at-least-once: on startup, orphaned runs are replayed (new run with replay_of set).
+- Document in job creation. Expose in CLI `jobs list` table.
+
+### F2: Flush-Before-Compaction Hook
+- New field `job_class` on jobs ('standard' default | 'pre_compaction_flush')
+- In buildJobPrompt: if job_class === 'pre_compaction_flush', prepend:
+  ```
+  [SYSTEM: Pre-compaction flush required]
+  Write a structured summary of: active decisions, constraints, task owners, open questions.
+  Format as labeled sections. If nothing needs flushing, respond with exactly: NO_FLUSH
+  [END SYSTEM]
+  ```
+- In dispatch result handling: if content.trim() === 'NO_FLUSH', skip delivery and log 'Flush: nothing to flush'
+
+### F3: Context Summary / Memory Observability
+- New field `context_summary` on runs (TEXT, stores JSON)
+- In buildJobPrompt: collect metadata into an object: { messages_injected: N, scope: 'own'|'global', aliases_resolved: [...], job_class, delivery_guarantee, context_retrieval, retrieval_results: N }
+- After creating the run, store the summary via updateContextSummary()
+- Expose in `runs list` and `runs get` CLI output (note: CLI exposure is deferred)
+
+### F4: Typed Message Contract
+- New message kinds: 'decision', 'constraint', 'fact', 'preference'
+- New field `owner` on messages
+- In sendMessage: validate kind against full enum, accept owner
+- In getInbox: sort results by typed priority order:
+  1. constraint (highest)
+  2. decision
+  3. fact
+  4. task
+  5. preference
+  6. text, result, status, system, spawn (lowest)
+- In buildJobPrompt: display kind and owner for typed messages:
+  ```
+  [constraint] (owner: ops-agent) Never deploy during business hours
+  ```
+
+### F5: HITL Approval Gates
+- New fields on jobs: approval_required (int 0/1), approval_timeout_s (int), approval_auto (text)
+- New run status: 'awaiting_approval'
+- New table: approvals
+- Flow:
+  1. In dispatchJob: if job.approval_required AND job is chain-triggered (has parent_id):
+     - Create run with status 'awaiting_approval'
+     - Create approval record
+     - Send notification: "⚠️ Job '{name}' requires approval. Approve: `node cli.js jobs approve {job_id}`"
+     - Return (don't dispatch yet)
+  2. In tick: call checkApprovals():
+     - For each pending approval: check if resolved or timed out
+     - If approved: change run status to 'pending', dispatch the job
+     - If timed_out: apply approval_auto policy
+     - If rejected: mark run as 'cancelled'
+- CLI: approve/reject commands resolve the approval record
+
+### F6: Run Replay on Startup
+- New field `replay_of` on runs
+- In main(), after initDb(), before starting tick loop:
+  - Query: SELECT r.*, j.delivery_guarantee, j.name FROM runs r JOIN jobs j ON r.job_id = j.id WHERE r.status = 'running'
+  - For each orphaned run:
+    - If delivery_guarantee = 'at-least-once': create new run with replay_of = old run id, set old run status = 'crashed', queue for dispatch
+    - If delivery_guarantee = 'at-most-once': set old run status = 'crashed', advance job schedule
+  - Log all actions
+
+### F7: Hybrid Retrieval for Job Context
+- New fields on jobs: context_retrieval ('none'|'recent'|'hybrid'), context_retrieval_limit (int)
+- In buildJobPrompt: if context_retrieval !== 'none', call buildRetrievalContext(job) and append to prompt
+- retrieval.js implements:
+  - getRecentRunSummaries: SELECT summary FROM runs WHERE job_id=? AND summary IS NOT NULL ORDER BY started_at DESC LIMIT ?
+  - searchRunSummaries: combine substring matching + simple TF-IDF scoring
+  - TF-IDF: tokenize query and summaries, compute term frequency * inverse document frequency, rank by score
+  - buildRetrievalContext: format results as "--- Prior Run Context ---\n[date] summary\n..."

package/INSTALL-ADDITIONAL-HOST.md

@@ -0,0 +1,333 @@
+# Installing OpenClaw Scheduler on an Additional Host
+
+This guide is for setting up the scheduler on a **second or additional OpenClaw instance**. Each host runs its own independent SQLite database and its own service (launchd on macOS, systemd on Linux, PM2 on Windows) -- they don't share state. This is not a replication setup; each host schedules and dispatches jobs independently.
+
+> **Starting fresh:** Unlike migrating from OC cron, on an additional host you'll typically create jobs from scratch. Use the job examples in README.md.
+> **Need copy-paste examples?** See [Starter Recipes in the README](README.md#starter-recipes) and [Common Migrations](README.md#common-migrations).
+
+---
+
+## Prerequisites
+
+| Requirement | Notes |
+|-------------|-------|
+| macOS or Linux | Tested on macOS arm64 |
+| Node.js >= 20 | `node --version` (use full path if needed: `/opt/homebrew/bin/node --version`) |
+| OpenClaw gateway running | With auth token |
+| Git or SCP access | To clone/copy the repo |
+
+> **macOS PATH note:** If installed via Homebrew, put the minimal PATH bootstrap in `~/.zshenv`, not only `~/.zprofile`, so non-interactive commands like `ssh host 'node cli.js status'` can find `node` too.
+
+---
+
+## Step 1: Install Scheduler Files
+
+```bash
+cd ~/.openclaw
+git clone https://github.com/amittell/openclaw-scheduler.git scheduler
+cd scheduler
+```
+
+Or copy from an existing host:
+```bash
+scp -r user@source-host:~/.openclaw/scheduler ~/.openclaw/scheduler
+```
+
+Or npm-first install (no git clone):
+```bash
+mkdir -p ~/.openclaw/scheduler
+npm install --prefix ~/.openclaw/scheduler openclaw-scheduler@latest
+npm exec --prefix ~/.openclaw/scheduler openclaw-scheduler -- help
+```
+
+Runtime state for npm installs defaults to `~/.openclaw/scheduler/`, not the package directory under `node_modules/`.
+
+---
+
+## Step 2: Install Dependencies
+
+If you used the npm-first install path in Step 1, dependencies are already installed; skip to Step 3.
+
+```bash
+cd ~/.openclaw/scheduler
+npm install
+```
+
+Installs `better-sqlite3` (native, compiles for your arch) and `croner`.
+
+If `better-sqlite3` fails: `xcode-select --install` (macOS).
+
+If Node changes later on this host, rebuild the native binding before restarting the scheduler:
+
+```bash
+cd ~/.openclaw/scheduler
+npm rebuild better-sqlite3
+```
+
+This is especially common after `brew upgrade node` on macOS or any major Node version switch.
+
+---
+
+## Step 2.5: Fix macOS shell PATH and completions
+
+If this additional host uses `zsh`, configure shell startup so both interactive terminals and non-interactive remote commands can find Homebrew Node.
+
+Recommended `~/.zshenv`:
+
+```zsh
+# ~/.zshenv — sourced by all zsh instances, including non-interactive SSH commands
+if [ -x /opt/homebrew/bin/brew ]; then
+  eval "$(/opt/homebrew/bin/brew shellenv)"
+fi
+
+export PATH="$HOME/.local/bin:$HOME/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
+```
+
+If you load OpenClaw completions in `~/.zshrc`, initialize completions first:
+
+```zsh
+autoload -Uz compinit
+compinit
+
+if [ -f "$HOME/.openclaw/completions/openclaw.zsh" ]; then
+  source "$HOME/.openclaw/completions/openclaw.zsh"
+fi
+```
+
+Avoid version-pinned Node paths like `/opt/homebrew/opt/node@22/bin`. Prefer `/opt/homebrew/bin/node`.
+
+Quick verification:
+
+```bash
+ssh "$HOST" 'command -v node && node -v'
+```
+
+---
+
+## Step 3: Run Tests
+
+```bash
+SCHEDULER_DB=:memory: node test.js
+```
+
+**All tests must pass before proceeding.**
+
+---
+
+## Step 4: Enable Chat Completions on Gateway
+
+```bash
+openclaw config set gateway.http.endpoints.chatCompletions.enabled true
+openclaw gateway restart
+```
+
+Verify:
+```bash
+curl -s -o /dev/null -w "%{http_code}" \
+  -X POST \
+  -H "Authorization: Bearer YOUR_GATEWAY_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"model":"openclaw:main","messages":[{"role":"user","content":"reply OK"}]}' \
+  http://127.0.0.1:18789/v1/chat/completions
+```
+
+Expected: `200`
+
+---
+
+## Step 5: Disable OC Built-in Cron
+
+If this host had OC built-in cron jobs enabled, disable them so they don't conflict with the scheduler.
+
+```bash
+openclaw cron list
+# For each enabled job:
+openclaw cron edit <job-id> --disable
+openclaw config set cron.enabled false
+```
+
+Also set `OPENCLAW_SKIP_CRON=1` in your OpenClaw gateway service environment (launchctl/systemd/pm2), then restart the gateway.
+
+Verify: `openclaw cron list` shows no enabled jobs (or "No cron jobs").
+
+---
+
+## Step 6: Disable OC Heartbeat
+
+If this host had OC heartbeat enabled, disable it:
+
+```bash
+openclaw config set agents.defaults.heartbeat.every "0m"
+# If you have per-agent heartbeat overrides, set/remove those too:
+# agents.list[].heartbeat.every = "0m"
+openclaw gateway restart
+```
+
+---
+
+## Step 7: Choose a macOS launchd mode
+
+> **Linux hosts:** For Linux additional hosts, follow the systemd setup in [INSTALL-LINUX.md](INSTALL-LINUX.md) instead of this step.
+
+On an additional host, the same choice applies:
+
+- **LaunchAgent**: best for a personal Mac with auto-login
+- **LaunchDaemon**: best for a headless host or startup before login
+
+Use the setup wizard to install the mode you want:
+
+```bash
+cd ~/.openclaw/scheduler
+node setup.mjs --service-mode agent
+# or:
+node setup.mjs --service-mode daemon
+```
+
+If you installed from npm:
+
+```bash
+npm exec --prefix ~/.openclaw/scheduler openclaw-scheduler -- setup --service-mode agent
+# or:
+npm exec --prefix ~/.openclaw/scheduler openclaw-scheduler -- setup --service-mode daemon
+```
+
+Verify the mode you chose:
+
+```bash
+# LaunchAgent
+launchctl print gui/$UID/ai.openclaw.scheduler
+
+# LaunchDaemon
+sudo launchctl print system/ai.openclaw.scheduler
+
+# Either mode
+sleep 5 && tail -5 /tmp/openclaw-scheduler.log
+```
+
+---
+
+## Step 8: Smoke Tests
+
+> **Note:** These smoke test commands use direct file imports and are for the git-clone install path. For npm installs, use `openclaw-scheduler` CLI commands instead.
+
+### Isolated dispatch
+```bash
+cd ~/.openclaw/scheduler
+node --input-type=module -e "
+import { initDb, getDb } from './db.js';
+import { createJob } from './jobs.js';
+initDb();
+const job = createJob({
+  name: 'Smoke Test',
+  schedule_cron: '0 0 31 2 *',
+  payload_message: 'Reply with exactly: SCHEDULER_OK',
+  delivery_mode: 'none',
+  delete_after_run: true,
+  origin: 'system',
+  run_timeout_ms: 300000,
+});
+getDb().prepare(\"UPDATE jobs SET next_run_at = datetime('now', '-1 second') WHERE id = ?\").run(job.id);
+console.log('Created smoke test:', job.id);
+"
+sleep 20 && tail -10 /tmp/openclaw-scheduler.log
+```
+
+Look for: `Dispatching: Smoke Test` → `Completed: Smoke Test`
+
+### Telegram delivery
+```bash
+node --input-type=module -e "
+import { initDb, getDb } from './db.js';
+import { createJob } from './jobs.js';
+initDb();
+const job = createJob({
+  name: 'Telegram Test',
+  schedule_cron: '0 0 31 2 *',
+  payload_message: 'Confirm scheduler is working. Send a brief greeting.',
+  delivery_mode: 'announce',
+  delivery_channel: 'telegram',
+  delivery_to: 'YOUR_CHAT_ID',
+  delete_after_run: true,
+  origin: 'system',
+  run_timeout_ms: 300000,
+});
+getDb().prepare(\"UPDATE jobs SET next_run_at = datetime('now', '-1 second') WHERE id = ?\").run(job.id);
+console.log('Created Telegram test:', job.id);
+"
+```
+
+You should receive a Telegram message within 30 seconds.
+
+---
+
+## Step 9: Create Your Jobs
+
+Since this is a fresh host, create jobs from scratch using the CLI:
+
+```bash
+node cli.js jobs add '{
+  "name": "My First Job",
+  "schedule_cron": "0 * * * *",
+  "payload_message": "Run your task here",
+  "delivery_mode": "announce",
+  "delivery_channel": "telegram",
+  "delivery_to": "YOUR_CHAT_ID"
+}'
+node cli.js jobs list
+node cli.js status
+```
+
+See README.md for full job examples including shell jobs, workflow chains, approval gates, and more.
+
+---
+
+## Step 10: Verify First Real Job
+
+Wait for the next scheduled job and confirm:
+```bash
+tail -f /tmp/openclaw-scheduler.log
+# Or after it fires:
+node cli.js runs list <job-id>
+```
+
+---
+
+## Rollback
+
+If anything goes wrong:
+
+```bash
+# 1. Stop scheduler
+launchctl bootout gui/$UID/ai.openclaw.scheduler     # if using LaunchAgent
+sudo launchctl bootout system/ai.openclaw.scheduler  # if using LaunchDaemon
+
+# 2. Re-enable OC cron (if you disabled it)
+openclaw cron edit <job-id> --enable  # for each job
+openclaw config set cron.enabled true
+# remove OPENCLAW_SKIP_CRON=1 from gateway service env
+
+# 3. Re-enable heartbeat (if you disabled it)
+openclaw config set agents.defaults.heartbeat.every "5m"
+openclaw gateway restart
+```
+
+---
+
+## Validation Checklist
+
+- [ ] `SCHEDULER_DB=:memory: node test.js` -- all passing, 0 failed
+- [ ] `node cli.js status` → shows jobs, 0 stale
+- [ ] `launchctl print gui/$UID/ai.openclaw.scheduler` or `sudo launchctl print system/ai.openclaw.scheduler` → running
+- [ ] Log file has startup lines, no errors
+- [ ] OC cron → all disabled (if applicable)
+- [ ] OC heartbeat → `0m` (if applicable)
+- [ ] Chat completions → 200
+- [ ] Smoke test → dispatched + completed in log
+- [ ] Telegram test → message received
+- [ ] First real job → fires on schedule
+
+---
+
+## Upgrading
+
+Already have the scheduler installed and need to update to a newer version? See [UPGRADING.md](UPGRADING.md).