@soffits/dsh-cron-jobs 0.14.0 → 0.15.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +26 -1
- package/client.js +1 -1
- package/index.js +5 -3
- package/package.json +1 -1
- package/src/config.js +6 -0
- package/src/controller.js +36 -8
- package/src/job.js +43 -9
- package/src/session.js +60 -27
- package/src/watchers.js +3 -3
package/README.md
CHANGED
|
@@ -119,6 +119,8 @@ Next to `jobs`, the row also takes:
|
|
|
119
119
|
| `autoArchive` | Default for `agent.autoArchive`. Default `true`. |
|
|
120
120
|
| `reuseWithinHours` | Default for `agent.reuseWithinHours`. Default `0`. |
|
|
121
121
|
| `deliver` | Default delivery target, `{ via, to }`, for jobs that do not set `deliver`. Unset: results are not delivered. |
|
|
122
|
+
| `problems.channel` | Also emit every problem note as an event on this channel, so an agent job with `on: [<channel>]` can handle it. See [Problem notes](#problem-notes). |
|
|
123
|
+
| `problems.deliver` | Where problem notes are sent instead of the job's own target: `{ via, to }`, or `false` for none once a job took the event. Unset: the job's target, as before. |
|
|
122
124
|
| `historyRecords` | Runs kept per job for the Cron tasks page. Default 50, at most 1000. |
|
|
123
125
|
| `runLog` | Optional JSON Lines file of run starts and ends, see below. |
|
|
124
126
|
| `watchers` | Long-running commands whose output lines are events: `[{ id, channel, command, cwd, enabled }]`. See [Watchers](#watchers). |
|
|
@@ -167,6 +169,29 @@ Where results go is decided per job: its own `deliver` (or `false`), otherwise n
|
|
|
167
169
|
|
|
168
170
|
Each run's delivery shows on the Cron tasks page and in the run log: not sent yet (waiting for the answer or for the channel), sent, nothing to send, or not sent with the reason. A delivery still waiting when DSH stops or the plugin reloads, whether for a turn that is still running or for a channel that is not there, is picked up and delivered after the next start.
|
|
169
171
|
|
|
172
|
+
### Problem notes
|
|
173
|
+
|
|
174
|
+
Failure notes (a command or gate that failed, a wake or rerun that failed, a watcher that is down or works again) go to the same place as the job's results unless the row says otherwise. That is the right default when whoever reads results also fixes the jobs. When the results go to someone who should not get raw failures, such as a person reading an agent's reports, the row's `problems` setting sends them elsewhere:
|
|
175
|
+
|
|
176
|
+
```yaml
|
|
177
|
+
- id: cron-jobs
|
|
178
|
+
config:
|
|
179
|
+
deliver: { via: telegramTopics, to: notify }
|
|
180
|
+
problems:
|
|
181
|
+
channel: cron.problem # hand each note to the jobs on this channel
|
|
182
|
+
deliver: false # and don't also send it to the job's target
|
|
183
|
+
jobs:
|
|
184
|
+
- name: Fix broken jobs
|
|
185
|
+
on: [cron.problem]
|
|
186
|
+
agent:
|
|
187
|
+
prompt: A scheduled job or watcher failed; the note is below. Find the cause and fix it. Tell the owner only what needs them.
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
- `channel`: each note becomes an event with the note as its text and details as data: `kind` (`command-failed`, `gate-failed`, `wake-failed`, `rerun-failed`, `watcher-down`, `watcher-up`), `job` and `jobId` (or `watcher` and its `channel`), `trigger`, `startedAt`, `error`, `exitCode`, `sessionId` when known, and `text`. Events batch and are capped like any others. A run that this channel started never emits a note on it, so a failing handler can't feed itself.
|
|
191
|
+
- `deliver`: a target sends every note there instead (a log topic, an ops chat), with or without `channel`. `false` sends nothing, but only when a job listening on `channel` took the event; a note no job took (no channel, nobody listening, or the handler's own failure) still goes to the job's target, so a broken job never goes quiet.
|
|
192
|
+
|
|
193
|
+
Ordinary results are not affected. Both are off by default.
|
|
194
|
+
|
|
170
195
|
### Runs cut off by a restart
|
|
171
196
|
|
|
172
197
|
A woken turn that a DSH restart cuts off before it finishes is run again once. A turn a person stopped, or one that failed, is not. Finding the cut turns is left to [dsh-restart-resume](https://github.com/soffits/dsh-restart-resume) (0.4.0 or later), which reads every cut turn from DSH's log after a restart and hands the ones a job started back to this plugin; without it, a cut-off run is recorded but not run again. The cut-off turn delivers nothing; when it is handed back, the plugin sends a short follow-up into the same session saying the earlier turn was cut off and asking the agent to finish the job without repeating what is done. The session already holds the job's prompt, command output and events, and whatever the cut-off turn did. The rerun's answer is delivered like any other.
|
|
@@ -270,7 +295,7 @@ A watcher is a long-running command the plugin starts with the scheduler, restar
|
|
|
270
295
|
- any other non-empty line is the event's text;
|
|
271
296
|
- `{"wakeAgent": false}` lines are ignored.
|
|
272
297
|
|
|
273
|
-
A watcher that exits is restarted after 1 s, doubling up to 5 minutes, and back to 1 s once a run lasted a minute. If it has not managed to stay up for a minute in 15 minutes, one note goes to the row's `deliver` target, and one more when it works again; exits, stderr tails and these notes are in the run log. The command runs with `bash -c` in its own process group, with `DSH_CRON_WATCHER` (its id), `DSH_CRON_CHANNEL` and `DSH_CRON_STATE_DIR` (`$DSH_HOME/cron-jobs/watchers/<id>`, a directory for its cursor) in its environment.
|
|
298
|
+
A watcher that exits is restarted after 1 s, doubling up to 5 minutes, and back to 1 s once a run lasted a minute. If it has not managed to stay up for a minute in 15 minutes, one note goes to the row's `deliver` target (or as set by [`problems`](#problem-notes)), and one more when it works again; exits, stderr tails and these notes are in the run log. The command runs with `bash -c` in its own process group, with `DSH_CRON_WATCHER` (its id), `DSH_CRON_CHANNEL` and `DSH_CRON_STATE_DIR` (`$DSH_HOME/cron-jobs/watchers/<id>`, a directory for its cursor) in its environment.
|
|
274
299
|
|
|
275
300
|
Watchers come from the row's `watchers` list, or from a conversation with the `cron_job_watch` tool (`create`, `delete`, `list`), which keeps them in DSH storage. A tool-made watcher can be `once`: it runs one time and removes itself when it exits, for a one-off wait that has to survive a DSH restart:
|
|
276
301
|
|
package/client.js
CHANGED
|
@@ -455,7 +455,7 @@ window.__ModuleLoader__.load({
|
|
|
455
455
|
@media (width<=1100px){.dcj-detail{--detail-gutter:20px}}
|
|
456
456
|
@media (width<=760px){.dcj-hasDetails .dcj-listPane{display:none}.dcj-detail{border-left:0;flex-basis:100%}.dcj-detailScroll{padding-top:20px}}
|
|
457
457
|
@media (width<=400px){.dcj-detail{--detail-gutter:16px}.dcj-detailTabsBar{gap:16px}}
|
|
458
|
-
@media (pointer:coarse){.dcj-searchField input{font-size:16px}}
|
|
458
|
+
@media (pointer:coarse){.dcj-searchField input{font-size:16px}.dcj-filterTab,.dcj-btn,.dcj-btnSm,.dcj-iconButton,.dcj-footerLink,.dcj-menuItem{min-height:40px}.dcj-iconButton{min-width:40px}.dcj-filters{gap:4px 8px}}
|
|
459
459
|
`
|
|
460
460
|
|
|
461
461
|
// ------------------------------------------------------------- helpers
|
package/index.js
CHANGED
|
@@ -32,7 +32,7 @@ export { Config }
|
|
|
32
32
|
|
|
33
33
|
/**
|
|
34
34
|
* @param {import('@deepseek-ai/cordis').Context} ctx
|
|
35
|
-
* @param {{ jobs: import('./src/job.js').Job[], watchers?: object[], events?: { coalesceSeconds?: number, maxRunsPerHour?: number }, runLog?: string, historyRecords?: number, defaultAgentCwd?: string, autoArchive?: boolean, reuseWithinHours?: number, deliver?: { via?: string, to?: string } }} config
|
|
35
|
+
* @param {{ jobs: import('./src/job.js').Job[], watchers?: object[], events?: { coalesceSeconds?: number, maxRunsPerHour?: number }, runLog?: string, historyRecords?: number, defaultAgentCwd?: string, autoArchive?: boolean, reuseWithinHours?: number, deliver?: { via?: string, to?: string }, problems?: { channel?: string, deliver?: { via?: string, to?: string } | false } }} config
|
|
36
36
|
*/
|
|
37
37
|
export function apply(ctx, config) {
|
|
38
38
|
const logger = ctx.logger('cron-jobs')
|
|
@@ -71,8 +71,9 @@ export function apply(ctx, config) {
|
|
|
71
71
|
jobs: config.jobs ?? [],
|
|
72
72
|
logger,
|
|
73
73
|
run: runCommand,
|
|
74
|
-
|
|
75
|
-
|
|
74
|
+
// Both wait for session services DSH has not registered yet (a run at startup).
|
|
75
|
+
wake: request => openAgentSession(ctx, request, logger, { signal: controller.signal }),
|
|
76
|
+
reuse: request => deliverToSession(ctx, request, { signal: controller.signal }),
|
|
76
77
|
settle: (sessionId, options) => archiveWhenIdle(ctx, sessionId, options),
|
|
77
78
|
// The channel is found by name at send time, so it may load after
|
|
78
79
|
// this plugin, reload, or be missing for a while (the delivery waits).
|
|
@@ -83,6 +84,7 @@ export function apply(ctx, config) {
|
|
|
83
84
|
autoArchive: config.autoArchive ?? true,
|
|
84
85
|
reuseWithinHours: config.reuseWithinHours ?? 0,
|
|
85
86
|
deliver: config.deliver,
|
|
87
|
+
problems: config.problems,
|
|
86
88
|
},
|
|
87
89
|
signal: controller.signal,
|
|
88
90
|
record,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@soffits/dsh-cron-jobs",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.15.1",
|
|
4
4
|
"description": "Scheduled jobs for DeepSeek Harness: shell commands and agent prompts on cron, RRULE or one-time schedules, a gate that wakes an agent only when there is something to do, result delivery through any plugin that can send messages, tools to manage jobs from a conversation, and a Cron tasks page in the Web GUI.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./index.js",
|
package/src/config.js
CHANGED
|
@@ -57,6 +57,11 @@ const Events = z.object({
|
|
|
57
57
|
maxRunsPerHour: z.natural().default(DEFAULT_MAX_RUNS_PER_HOUR).description('Most event-started runs of one job in any hour; past that, events wait and run together. 0 means no cap.'),
|
|
58
58
|
})
|
|
59
59
|
|
|
60
|
+
const Problems = z.object({
|
|
61
|
+
channel: z.string().description('Also emit every problem note (a failed command or gate, a failed wake or rerun, a watcher down or back) as an event on this channel, such as cron.problem, so a job with on: [cron.problem] can handle it. A run this channel started never emits one. Empty: no event.'),
|
|
62
|
+
deliver: z.union([z.const(false), DeliveryTarget]).description('Where problem notes are sent instead of the job\'s own target (or, for watchers, the plugin\'s deliver). false sends none when a job took the event on channel; a note no job took still goes to the job\'s target. Unset: the job\'s target, as before.'),
|
|
63
|
+
}).description('Where notes about failures go, so they can reach whoever fixes them rather than every job\'s audience.')
|
|
64
|
+
|
|
60
65
|
export const Config = z.object({
|
|
61
66
|
jobs: z.array(Job).default([]).description('Scheduled jobs.'),
|
|
62
67
|
watchers: z.array(Watcher).default([]).description('Long-running commands whose output lines are events (see a job\'s on). Watchers can also be added from a conversation with the watch tool.'),
|
|
@@ -65,6 +70,7 @@ export const Config = z.object({
|
|
|
65
70
|
autoArchive: z.boolean().default(true).description('Archive woken sessions once their turn ends, for jobs that do not set agent.autoArchive. Archived sessions leave the sidebar\'s active list but stay in DSH and in session search.'),
|
|
66
71
|
reuseWithinHours: z.number().min(0).default(0).description('Reuse window for jobs that do not set agent.reuseWithinHours: send the prompt into the job\'s previous woken session when it was opened within this many hours. 0 always opens a new session.'),
|
|
67
72
|
deliver: DeliveryTarget.description('Default delivery target for jobs that do not set deliver. Jobs that post into an existing session do not use it. Unset: results are not delivered.'),
|
|
73
|
+
problems: Problems,
|
|
68
74
|
historyRecords: z.natural().min(1).max(1000).default(DEFAULT_HISTORY_RECORDS)
|
|
69
75
|
.description('Runs kept per job for the Cron tasks page (start, duration, exit code, woken session, output tails). Older runs are dropped.'),
|
|
70
76
|
runLog: z.string().description('JSON Lines file that gets one line when a run starts and one when it ends (outcome, exit code, duration, output tails), so past runs can be checked later. Directories are created as needed. No file when empty.'),
|
package/src/controller.js
CHANGED
|
@@ -70,7 +70,7 @@ export class CronJobs {
|
|
|
70
70
|
* @param {Function} [options.settle] - Archives a session after its turn.
|
|
71
71
|
* @param {Function} [options.send] - Hands a result to its delivery service (see deliver.js sendMessage).
|
|
72
72
|
* @param {Function} [options.answer] - Waits for a woken agent's final answer (see session.js waitForAnswer).
|
|
73
|
-
* @param {{ agentCwd?: string, autoArchive?: boolean, reuseWithinHours?: number, deliver?: object }} [options.defaults] - Plugin-level agent settings
|
|
73
|
+
* @param {{ agentCwd?: string, autoArchive?: boolean, reuseWithinHours?: number, deliver?: object, problems?: { channel?: string, deliver?: object | false } }} [options.defaults] - Plugin-level agent settings, default delivery target, and where problem notes go (see reportProblem).
|
|
74
74
|
* @param {AbortSignal} [options.signal] - Stops running commands when aborted.
|
|
75
75
|
* @param {(entry: object) => Promise<void>} [options.record] - Run log, see runlog.js.
|
|
76
76
|
* @param {import('./history.js').HistoryStore} [options.history] - Saved runs, pause state and stored jobs.
|
|
@@ -96,7 +96,7 @@ export class CronJobs {
|
|
|
96
96
|
this.watchdogMs = watchdogMs
|
|
97
97
|
this.overdueMs = overdueMs
|
|
98
98
|
this.record = record ?? (async () => {})
|
|
99
|
-
this.deps = { run, wake, reuse, settle, send, answer, defaults, logger, signal, record, history, now }
|
|
99
|
+
this.deps = { run, wake, reuse, settle, send, answer, defaults, logger, signal, record, history, now, reportProblem: (problem, triggers) => this.reportProblem(problem, triggers) }
|
|
100
100
|
this.eventSettings = {
|
|
101
101
|
coalesceMs: (events.coalesceSeconds ?? DEFAULT_COALESCE_SECONDS) * 1000,
|
|
102
102
|
maxRunsPerHour: events.maxRunsPerHour ?? DEFAULT_MAX_RUNS_PER_HOUR,
|
|
@@ -237,7 +237,7 @@ export class CronJobs {
|
|
|
237
237
|
emit: event => this.emit(event),
|
|
238
238
|
logger: this.logger,
|
|
239
239
|
record: this.record,
|
|
240
|
-
notify: (text, def) => this.
|
|
240
|
+
notify: (text, def, kind) => this.notifyProblem(text, { kind, watcher: def.id, channel: def.channel }),
|
|
241
241
|
onDone: id => this.watcherDone(id),
|
|
242
242
|
})
|
|
243
243
|
this.supervisor.set(this.watcherDefinitions())
|
|
@@ -253,13 +253,41 @@ export class CronJobs {
|
|
|
253
253
|
return defs
|
|
254
254
|
}
|
|
255
255
|
|
|
256
|
-
/**
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
256
|
+
/**
|
|
257
|
+
* Hand a problem note to the jobs on the `problems.channel` setting, as an
|
|
258
|
+
* event with the note as text and the details as data. Not emitted for a
|
|
259
|
+
* run that this channel started, so a failing handler cannot feed itself.
|
|
260
|
+
* @param {{ kind: string, text: string }} problem
|
|
261
|
+
* @param {string[]} [triggers] - Event channels that started the failed run.
|
|
262
|
+
* @returns {boolean} Whether a job listening on the channel took it.
|
|
263
|
+
*/
|
|
264
|
+
reportProblem(problem, triggers = []) {
|
|
265
|
+
const channel = this.defaults.problems?.channel?.trim()
|
|
266
|
+
if (!channel || this.stopped || triggers.includes(channel)) return false
|
|
267
|
+
try {
|
|
268
|
+
return this.emit({ channel, source: 'cron-jobs', text: problem.text, data: problem }).matched.length > 0
|
|
269
|
+
} catch (error) {
|
|
270
|
+
this.logger.warn('problem note not emitted on %s: %s', channel, error instanceof Error ? error.message : String(error))
|
|
271
|
+
return false
|
|
272
|
+
}
|
|
273
|
+
}
|
|
274
|
+
|
|
275
|
+
/**
|
|
276
|
+
* A watcher's down or back-up note: emitted on the problems channel, and
|
|
277
|
+
* sent to `problems.deliver` or the plugin's default target (see
|
|
278
|
+
* JobRunner.problemTarget for the same rule).
|
|
279
|
+
*/
|
|
280
|
+
async notifyProblem(text, details) {
|
|
281
|
+
const from = `watcher ${details.watcher}`
|
|
282
|
+
const handled = this.reportProblem({ ...details, text })
|
|
283
|
+
const routed = deliveryTarget(this.defaults.problems?.deliver)
|
|
284
|
+
const fallback = resolveDelivery({}, this.defaults.deliver)
|
|
285
|
+
const target = routed === undefined ? fallback : routed === false ? (handled ? undefined : fallback) : routed
|
|
286
|
+
if (target === undefined) {
|
|
287
|
+
if (!handled) this.logger.warn('%s: %s (no delivery target)', from, text)
|
|
261
288
|
return
|
|
262
289
|
}
|
|
290
|
+
if (this.deps.send === undefined) return
|
|
263
291
|
const sent = await this.deps.send(target, { text, files: [] }, { job: from }, {})
|
|
264
292
|
if (sent.status !== 'sent') this.logger.warn('%s: note not delivered: %s', from, sent.error)
|
|
265
293
|
}
|
package/src/job.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { randomUUID } from 'node:crypto'
|
|
2
2
|
import { decideWake } from './wake.js'
|
|
3
|
-
import { commandResultText, cutOff, describeTarget, messageFromText, resolveDelivery } from './deliver.js'
|
|
3
|
+
import { commandResultText, cutOff, deliveryTarget, describeTarget, messageFromText, resolveDelivery } from './deliver.js'
|
|
4
4
|
import { buildAgentPrompt, buildRerunPrompt, formatLocalTime, sessionTitle } from './prompt.js'
|
|
5
5
|
import { HISTORY_TAIL_CHARS } from './history.js'
|
|
6
6
|
import { batchSummary, eventInput } from './events.js'
|
|
@@ -105,14 +105,15 @@ export class JobRunner {
|
|
|
105
105
|
* @param {(sessionId: string, options: { signal?: AbortSignal, isLatest: () => boolean, lock: Function }) => Promise<{ archive: string, note?: string }>} [deps.settle] - Archives a session after its turn (see session.js archiveWhenIdle); without it nothing is archived.
|
|
106
106
|
* @param {(target: { via: string, to: string }, message: { text: string, files: string[] }, from: { job: string, sessionId?: string }, options: { waitMs?: number, onWait?: Function }) => Promise<{ status: 'sent' | 'failed' | 'unavailable' | 'stopped', attempts: number, error?: string }>} [deps.send] - Hands a result to its delivery service (see deliver.js sendMessage); without it nothing is delivered.
|
|
107
107
|
* @param {(sessionId: string, messageId: string, options: { signal?: AbortSignal }) => Promise<{ pending: true } | { text: string, reason?: string } | { error: string }>} [deps.answer] - Waits for the final answer to a prompt (see session.js waitForAnswer).
|
|
108
|
-
* @param {{ agentCwd?: string, autoArchive?: boolean, reuseWithinHours?: number, deliver?: object }} [deps.defaults] - Plugin-level agent settings
|
|
108
|
+
* @param {{ agentCwd?: string, autoArchive?: boolean, reuseWithinHours?: number, deliver?: object, problems?: { deliver?: object | false } }} [deps.defaults] - Plugin-level agent settings, the default delivery target, and where problem notes go.
|
|
109
|
+
* @param {(problem: object, triggers: string[]) => boolean} [deps.reportProblem] - Emits a problem note as an event (see CronJobs.reportProblem); true when a job took it.
|
|
109
110
|
* @param {{ info: Function, warn: Function }} deps.logger
|
|
110
111
|
* @param {AbortSignal} [deps.signal] - Stops a running command when aborted.
|
|
111
112
|
* @param {(entry: object) => Promise<void>} [deps.record] - Appends a run record (see runlog.js).
|
|
112
113
|
* @param {{ begin: Function, finish: Function }} [deps.history] - Saved run history (see history.js).
|
|
113
114
|
* @param {() => Date} [deps.now]
|
|
114
115
|
*/
|
|
115
|
-
constructor(job, { run, wake, reuse, keepSession, settle, send, answer, defaults = {}, logger, signal, record = async () => {}, history = NO_HISTORY, now = () => new Date() }) {
|
|
116
|
+
constructor(job, { run, wake, reuse, keepSession, settle, send, answer, reportProblem, defaults = {}, logger, signal, record = async () => {}, history = NO_HISTORY, now = () => new Date() }) {
|
|
116
117
|
this.job = job
|
|
117
118
|
this.send = send
|
|
118
119
|
this.answer = answer
|
|
@@ -120,6 +121,7 @@ export class JobRunner {
|
|
|
120
121
|
this.keepSession = keepSession
|
|
121
122
|
this.settle = settle
|
|
122
123
|
this.defaults = defaults
|
|
124
|
+
this.reportProblem = reportProblem
|
|
123
125
|
/** Latest run that sent a prompt to each session; only that run archives it. */
|
|
124
126
|
this.latestRun = new Map()
|
|
125
127
|
/** Per-session queue that keeps a delivery and an archive from overlapping. */
|
|
@@ -245,22 +247,28 @@ export class JobRunner {
|
|
|
245
247
|
* nothing.
|
|
246
248
|
*/
|
|
247
249
|
async deliverResult({ messageId, ...outcome }, result, { rerun = false } = {}) {
|
|
248
|
-
const target = this.deliveryTarget()
|
|
249
|
-
if (target === undefined) return outcome
|
|
250
250
|
const { job } = this
|
|
251
|
-
const where = { via: target.via, to: target.to }
|
|
252
251
|
if (outcome.status === 'woke') {
|
|
253
|
-
|
|
254
|
-
|
|
252
|
+
const target = this.deliveryTarget()
|
|
253
|
+
if (target === undefined || outcome.turn?.status !== 'pending') return outcome
|
|
254
|
+
return { ...outcome, delivery: { status: 'pending', via: target.via, to: target.to } }
|
|
255
255
|
}
|
|
256
256
|
let text
|
|
257
|
+
let problem
|
|
257
258
|
if (outcome.status === 'wake-failed') {
|
|
259
|
+
problem = rerun ? 'rerun-failed' : 'wake-failed'
|
|
258
260
|
text = rerun
|
|
259
261
|
? `⚠️ ${job.name}: the run cut off by a DSH restart could not run again: ${outcome.error}`
|
|
260
262
|
: `⚠️ ${job.name}: could not wake the agent: ${outcome.error}`
|
|
261
263
|
}
|
|
262
|
-
else if (result !== undefined && !result.aborted && (outcome.status === 'done' || outcome.status === 'failed'))
|
|
264
|
+
else if (result !== undefined && !result.aborted && (outcome.status === 'done' || outcome.status === 'failed')) {
|
|
265
|
+
text = commandResultText(job, result)
|
|
266
|
+
if (!result.ok) problem = isGated(job) ? 'gate-failed' : 'command-failed'
|
|
267
|
+
}
|
|
263
268
|
else return outcome
|
|
269
|
+
const target = problem === undefined ? this.deliveryTarget() : this.problemTarget(problem, text, outcome, result)
|
|
270
|
+
if (target === undefined) return outcome
|
|
271
|
+
const where = { via: target.via, to: target.to }
|
|
264
272
|
const message = messageFromText(text, job.cwd || undefined)
|
|
265
273
|
if (message === undefined) return { ...outcome, delivery: { status: 'silent', ...where } }
|
|
266
274
|
// No waiting inside the run: a missing service is waited for in the
|
|
@@ -269,6 +277,32 @@ export class JobRunner {
|
|
|
269
277
|
return { ...outcome, delivery: delivery.status === 'pending' ? { ...delivery, message } : delivery }
|
|
270
278
|
}
|
|
271
279
|
|
|
280
|
+
/**
|
|
281
|
+
* Where a problem note (a failed command or gate, a failed wake or rerun)
|
|
282
|
+
* goes, after handing it to the problems channel. Without a `problems`
|
|
283
|
+
* setting, the job's own target. `problems.deliver` set to a target sends
|
|
284
|
+
* it there instead; set to false sends nothing, but only when a job took
|
|
285
|
+
* the event, so a problem nobody handles still reaches the job's target.
|
|
286
|
+
*/
|
|
287
|
+
problemTarget(kind, text, outcome, result) {
|
|
288
|
+
const { job, current } = this
|
|
289
|
+
const handled = this.reportProblem?.({
|
|
290
|
+
kind,
|
|
291
|
+
job: job.name,
|
|
292
|
+
jobId: jobKey(job),
|
|
293
|
+
trigger: current?.rerunOf ? 'rerun' : current?.trigger ?? 'schedule',
|
|
294
|
+
...(current?.startedAt ? { startedAt: current.startedAt } : {}),
|
|
295
|
+
...(outcome.sessionId ? { sessionId: outcome.sessionId } : {}),
|
|
296
|
+
...(outcome.error ? { error: outcome.error } : {}),
|
|
297
|
+
...(result ? { exitCode: result.exitCode ?? null } : {}),
|
|
298
|
+
text,
|
|
299
|
+
}, current?.events?.channels ?? []) === true
|
|
300
|
+
const routed = deliveryTarget(this.defaults.problems?.deliver)
|
|
301
|
+
if (routed === undefined) return this.deliveryTarget()
|
|
302
|
+
if (routed === false) return handled ? undefined : this.deliveryTarget()
|
|
303
|
+
return this.send === undefined ? undefined : routed
|
|
304
|
+
}
|
|
305
|
+
|
|
272
306
|
/**
|
|
273
307
|
* Send one message and log the result; resolves the run's delivery record.
|
|
274
308
|
* Not sent because the plugin is stopping, or (with `waitMs` 0) because
|
package/src/session.js
CHANGED
|
@@ -9,10 +9,11 @@ import { finalAnswer } from './deliver.js'
|
|
|
9
9
|
* follow-up (which starts a turn), and wait until session persistence has
|
|
10
10
|
* stored the message.
|
|
11
11
|
*
|
|
12
|
-
* Uses only public Host services,
|
|
13
|
-
*
|
|
14
|
-
* `
|
|
15
|
-
*
|
|
12
|
+
* Uses only public Host services, so that hosts without them still run
|
|
13
|
+
* plain jobs: `sessionController` and `sessions` (required), `agents`,
|
|
14
|
+
* `workspaceRegistry`, `sessionTitle`, and `llm` when a model is requested.
|
|
15
|
+
* Every one is read through {@link sessionServices}, which waits for a
|
|
16
|
+
* service the host has not registered yet (a run right after a DSH start).
|
|
16
17
|
*
|
|
17
18
|
* A requested model is checked with `llm.resolveCallConfig()` before the
|
|
18
19
|
* session is created, then set on the new session's agent with
|
|
@@ -30,16 +31,19 @@ import { finalAnswer } from './deliver.js'
|
|
|
30
31
|
* @param {string} [request.cwd] - Session directory; the host default when omitted.
|
|
31
32
|
* @param {{ provider: string, model: string, reasoningEffort?: string }} [request.model] - Model of the session; the preset or host default when omitted.
|
|
32
33
|
* @param {{ warn: Function }} logger
|
|
34
|
+
* @param {ServiceWait} [wait]
|
|
33
35
|
* @returns {Promise<string>} The new session id.
|
|
34
36
|
*/
|
|
35
|
-
export async function openAgentSession(ctx, { text, messageId, principal, title, agentPreset, cwd, model }, logger) {
|
|
36
|
-
const controller = ctx
|
|
37
|
-
|
|
37
|
+
export async function openAgentSession(ctx, { text, messageId, principal, title, agentPreset, cwd, model }, logger, wait = {}) {
|
|
38
|
+
const { sessionController: controller, sessions, agents, workspaceRegistry, sessionTitle, llm } = await sessionServices(ctx, {
|
|
39
|
+
required: ['sessionController', 'sessions', ...(model !== undefined ? ['llm'] : [])],
|
|
40
|
+
optional: ['agents', 'workspaceRegistry', 'sessionTitle'],
|
|
41
|
+
}, wait)
|
|
38
42
|
if (controller === undefined || sessions === undefined) {
|
|
39
43
|
throw new Error('this host cannot create sessions (no sessionController service)')
|
|
40
44
|
}
|
|
41
45
|
const open = async () => {
|
|
42
|
-
if (model !== undefined) await checkModel(
|
|
46
|
+
if (model !== undefined) await checkModel(llm, model)
|
|
43
47
|
const { sessionId } = await controller.create({
|
|
44
48
|
...(cwd ? { cwd } : {}),
|
|
45
49
|
...(agentPreset ? { agentPreset } : {}),
|
|
@@ -49,8 +53,8 @@ export async function openAgentSession(ctx, { text, messageId, principal, title,
|
|
|
49
53
|
const { agent } = resolved
|
|
50
54
|
if (model !== undefined) agent.model.select({ ...model })
|
|
51
55
|
|
|
52
|
-
await joinWorkspace(
|
|
53
|
-
nameSession(
|
|
56
|
+
await joinWorkspace(workspaceRegistry, sessionId, cwd, logger)
|
|
57
|
+
nameSession(sessionTitle, agent, title, logger)
|
|
54
58
|
|
|
55
59
|
agent.followup(userMessage(text, messageId, principal))
|
|
56
60
|
if (!await sessions.flush(agent.session)) {
|
|
@@ -60,7 +64,6 @@ export async function openAgentSession(ctx, { text, messageId, principal, title,
|
|
|
60
64
|
}
|
|
61
65
|
// A timer callback has no initiating agent; clearing it anyway keeps the new
|
|
62
66
|
// session a top-level session rather than a child of whatever is running.
|
|
63
|
-
const agents = ctx.get('agents')
|
|
64
67
|
return agents === undefined ? open() : agents.withoutInitiator(open)
|
|
65
68
|
}
|
|
66
69
|
|
|
@@ -87,12 +90,15 @@ export async function openAgentSession(ctx, { text, messageId, principal, title,
|
|
|
87
90
|
* @param {object} [request.principal] - Who sends the message, recorded on its source.
|
|
88
91
|
* @param {boolean} [request.mayUnarchive]
|
|
89
92
|
* @param {boolean} [request.ignoreHold] - Deliver even to a pinned session or one a person uses.
|
|
93
|
+
* @param {ServiceWait} [wait]
|
|
90
94
|
* @returns {Promise<{ sessionId: string, queued: boolean, unarchived: boolean }>}
|
|
91
95
|
* `queued`: the session was busy and the message waits for its next turn.
|
|
92
96
|
*/
|
|
93
|
-
export async function deliverToSession(ctx, { sessionId, text, messageId, principal, mayUnarchive = false, ignoreHold = false }) {
|
|
94
|
-
const controller = ctx
|
|
95
|
-
|
|
97
|
+
export async function deliverToSession(ctx, { sessionId, text, messageId, principal, mayUnarchive = false, ignoreHold = false }, wait = {}) {
|
|
98
|
+
const { sessionController: controller, sessions, agents, workspaceRegistry: registry } = await sessionServices(ctx, {
|
|
99
|
+
required: ['sessionController', 'sessions'],
|
|
100
|
+
optional: ['agents', 'workspaceRegistry'],
|
|
101
|
+
}, wait)
|
|
96
102
|
if (controller === undefined || sessions === undefined) {
|
|
97
103
|
throw new Error('this host cannot resume sessions (no sessionController service)')
|
|
98
104
|
}
|
|
@@ -100,9 +106,8 @@ export async function deliverToSession(ctx, { sessionId, text, messageId, princi
|
|
|
100
106
|
const resolved = await controller.resolveAgent(sessionId)
|
|
101
107
|
if ('error' in resolved) throw resolved.error
|
|
102
108
|
const { agent } = resolved
|
|
103
|
-
const hold = ignoreHold ? undefined : userHold(
|
|
109
|
+
const hold = ignoreHold ? undefined : userHold(registry, sessionId, agent.session)
|
|
104
110
|
if (hold !== undefined) throw new Error(`the session is in use (${hold})`)
|
|
105
|
-
const registry = ctx.get('workspaceRegistry')
|
|
106
111
|
const archived = registry?.archivedSessionIds?.includes(sessionId) === true
|
|
107
112
|
if (archived) {
|
|
108
113
|
if (!mayUnarchive) throw new Error('the session is archived')
|
|
@@ -115,7 +120,6 @@ export async function deliverToSession(ctx, { sessionId, text, messageId, princi
|
|
|
115
120
|
}
|
|
116
121
|
return { sessionId, queued, unarchived: archived }
|
|
117
122
|
}
|
|
118
|
-
const agents = ctx.get('agents')
|
|
119
123
|
return agents === undefined ? deliver() : agents.withoutInitiator(deliver)
|
|
120
124
|
}
|
|
121
125
|
|
|
@@ -128,13 +132,13 @@ export async function deliverToSession(ctx, { sessionId, text, messageId, princi
|
|
|
128
132
|
* results) have their own kinds. Such a session is neither archived nor
|
|
129
133
|
* reused.
|
|
130
134
|
*
|
|
131
|
-
* @param {
|
|
135
|
+
* @param {{ pinnedSessionIds?: string[] } | undefined} registry - The Workspace registry, when the host has one.
|
|
132
136
|
* @param {string} sessionId
|
|
133
137
|
* @param {{ log?: Array<{ type: string, data?: any }> }} [session] - The live session, to read its message log.
|
|
134
138
|
* @returns {string | undefined}
|
|
135
139
|
*/
|
|
136
|
-
export function userHold(
|
|
137
|
-
if (
|
|
140
|
+
export function userHold(registry, sessionId, session) {
|
|
141
|
+
if (registry?.pinnedSessionIds?.includes(sessionId)) return 'pinned'
|
|
138
142
|
if (session?.log?.some(event => event.type === 'user/message' && event.data?.source?.kind === 'user')) {
|
|
139
143
|
return 'taken over by a person'
|
|
140
144
|
}
|
|
@@ -189,7 +193,7 @@ export async function archiveWhenIdle(ctx, sessionId, {
|
|
|
189
193
|
if (turnCutByShutdown(agent.session?.log ?? [])) return { archive: 'kept', note: 'its turn was cut off by a DSH stop' }
|
|
190
194
|
const result = await lock(async () => {
|
|
191
195
|
if (!isLatest()) return { archive: 'superseded' }
|
|
192
|
-
const hold = userHold(
|
|
196
|
+
const hold = userHold(registry, sessionId, agent.session)
|
|
193
197
|
if (hold !== undefined) return { archive: 'kept', note: hold }
|
|
194
198
|
try {
|
|
195
199
|
await registry.archiveSession(sessionId)
|
|
@@ -280,6 +284,38 @@ export async function hostService(ctx, name, { signal, waitMs = SERVICE_WAIT_MS,
|
|
|
280
284
|
}
|
|
281
285
|
}
|
|
282
286
|
|
|
287
|
+
/** How long an optional service is waited for once the required ones are there. */
|
|
288
|
+
export const OPTIONAL_SERVICE_WAIT_MS = 10_000
|
|
289
|
+
|
|
290
|
+
/**
|
|
291
|
+
* @typedef {object} ServiceWait
|
|
292
|
+
* @property {AbortSignal} [signal] - Plugin unload: stop waiting.
|
|
293
|
+
* @property {number} [waitMs] - Wait for a required service; {@link SERVICE_WAIT_MS} by default.
|
|
294
|
+
* @property {number} [optionalWaitMs] - Wait for an optional one after that; {@link OPTIONAL_SERVICE_WAIT_MS} by default.
|
|
295
|
+
* @property {number} [pollMs]
|
|
296
|
+
*/
|
|
297
|
+
|
|
298
|
+
/**
|
|
299
|
+
* Read the Host services a session path needs, waiting for them the way
|
|
300
|
+
* {@link hostService} does: a job that fires while DSH is still starting
|
|
301
|
+
* (a catch-up run, an event from a watcher) would otherwise find no
|
|
302
|
+
* `sessionController` and fail (2026-11-16, a check-in's catch-up run ten
|
|
303
|
+
* seconds after a restart). Required services are waited for fully; optional
|
|
304
|
+
* ones, which come from other plugins of the same start, a short time more,
|
|
305
|
+
* so a host that lacks one still runs jobs without a long delay.
|
|
306
|
+
*
|
|
307
|
+
* @param {import('@deepseek-ai/cordis').Context} ctx
|
|
308
|
+
* @param {{ required: string[], optional?: string[] }} names
|
|
309
|
+
* @param {ServiceWait} [wait]
|
|
310
|
+
* @returns {Promise<Record<string, any>>} Each name mapped to its service, or undefined.
|
|
311
|
+
*/
|
|
312
|
+
export async function sessionServices(ctx, { required, optional = [] }, { signal, waitMs = SERVICE_WAIT_MS, optionalWaitMs = OPTIONAL_SERVICE_WAIT_MS, pollMs } = {}) {
|
|
313
|
+
const read = (names, ms) => Promise.all(names.map(name => hostService(ctx, name, { signal, waitMs: ms, ...(pollMs !== undefined ? { pollMs } : {}) })))
|
|
314
|
+
const found = await read(required, waitMs)
|
|
315
|
+
const extra = await read(optional, found.every(service => service !== undefined) ? optionalWaitMs : 0)
|
|
316
|
+
return Object.fromEntries([...required, ...optional].map((name, i) => [name, i < found.length ? found[i] : extra[i - found.length]]))
|
|
317
|
+
}
|
|
318
|
+
|
|
283
319
|
function sleep(ms, signal) {
|
|
284
320
|
return new Promise((resolve) => {
|
|
285
321
|
const timer = setTimeout(done, ms)
|
|
@@ -317,8 +353,7 @@ export function userMessage(text, id = randomUUID(), principal) {
|
|
|
317
353
|
}
|
|
318
354
|
|
|
319
355
|
/** Reject a model the host cannot route, before any session is created. */
|
|
320
|
-
async function checkModel(
|
|
321
|
-
const llm = ctx.get('llm')
|
|
356
|
+
async function checkModel(llm, model) {
|
|
322
357
|
if (llm === undefined) throw new Error('this host cannot select models (no llm service)')
|
|
323
358
|
try {
|
|
324
359
|
await llm.resolveCallConfig({ ...model })
|
|
@@ -328,8 +363,7 @@ async function checkModel(ctx, model) {
|
|
|
328
363
|
}
|
|
329
364
|
}
|
|
330
365
|
|
|
331
|
-
async function joinWorkspace(
|
|
332
|
-
const registry = ctx.get('workspaceRegistry')
|
|
366
|
+
async function joinWorkspace(registry, sessionId, cwd, logger) {
|
|
333
367
|
if (registry === undefined || !cwd) return
|
|
334
368
|
try {
|
|
335
369
|
await (await registry.resolveByPath(cwd))?.attachSession(sessionId)
|
|
@@ -338,8 +372,7 @@ async function joinWorkspace(ctx, sessionId, cwd, logger) {
|
|
|
338
372
|
}
|
|
339
373
|
}
|
|
340
374
|
|
|
341
|
-
function nameSession(
|
|
342
|
-
const titles = ctx.get('sessionTitle')
|
|
375
|
+
function nameSession(titles, agent, title, logger) {
|
|
343
376
|
if (titles === undefined) return
|
|
344
377
|
try {
|
|
345
378
|
titles.rename(agent.session, title)
|
package/src/watchers.js
CHANGED
|
@@ -81,7 +81,7 @@ export class WatcherSupervisor {
|
|
|
81
81
|
* @param {(event: object) => void} options.emit - Receives each event.
|
|
82
82
|
* @param {{ info: Function, warn: Function }} options.logger
|
|
83
83
|
* @param {(entry: object) => Promise<void>} [options.record] - Run log.
|
|
84
|
-
* @param {(text: string, watcher: object) => Promise<void>} [options.notify] - Sends a down or back-up note.
|
|
84
|
+
* @param {(text: string, watcher: object, kind: 'watcher-down' | 'watcher-up') => Promise<void>} [options.notify] - Sends a down or back-up note.
|
|
85
85
|
* @param {(id: string) => Promise<void>} [options.onDone] - A once watcher finished.
|
|
86
86
|
* @param {string} [options.stateRoot] - Each watcher gets `<stateRoot>/<id>` as DSH_CRON_STATE_DIR.
|
|
87
87
|
* @param {() => number} [options.now]
|
|
@@ -270,7 +270,7 @@ class Watcher {
|
|
|
270
270
|
if (wasDown && this.alerted) {
|
|
271
271
|
this.alerted = false
|
|
272
272
|
sup.record({ time: new Date(sup.now()).toISOString(), job: null, event: 'watcher-up', watcher: def.id })
|
|
273
|
-
sup.notify(`✅ Watcher ${def.id} (${def.channel}) works again.`, def).catch(() => {})
|
|
273
|
+
sup.notify(`✅ Watcher ${def.id} (${def.channel}) works again.`, def, 'watcher-up').catch(() => {})
|
|
274
274
|
}
|
|
275
275
|
}
|
|
276
276
|
|
|
@@ -324,7 +324,7 @@ class Watcher {
|
|
|
324
324
|
// The last run's, not this.stderr: a restarted run may be going already, with its own empty tail.
|
|
325
325
|
const stderr = (exit?.stderrTail ?? '').slice(-300)
|
|
326
326
|
sup.record({ time: new Date(sup.now()).toISOString(), job: null, event: 'watcher-down', watcher: def.id, downMs: sup.now() - this.downSince })
|
|
327
|
-
sup.notify(`⚠️ Watcher ${def.id} (${def.channel}) has been down for ${minutes} min (${this.restarts} restarts, last: ${how})${stderr ? `: ${stderr}` : ''}. Jobs on this channel only run on their schedule until it is back.`, def).catch(() => {})
|
|
327
|
+
sup.notify(`⚠️ Watcher ${def.id} (${def.channel}) has been down for ${minutes} min (${this.restarts} restarts, last: ${how})${stderr ? `: ${stderr}` : ''}. Jobs on this channel only run on their schedule until it is back.`, def, 'watcher-down').catch(() => {})
|
|
328
328
|
}
|
|
329
329
|
|
|
330
330
|
async stop() {
|