@kingcrab/pi-imessage 0.0.43 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (187) hide show
  1. package/README.md +101 -127
  2. package/dist/agent/behavior.js +79 -0
  3. package/dist/agent/chats.js +67 -0
  4. package/dist/agent/commands.js +163 -0
  5. package/dist/agent/compaction.js +51 -0
  6. package/dist/agent/deliver.js +139 -0
  7. package/dist/agent/direct-send.js +68 -0
  8. package/dist/agent/harness.js +79 -0
  9. package/dist/agent/health.js +32 -0
  10. package/dist/agent/models.js +69 -0
  11. package/dist/agent/progress-format.js +87 -0
  12. package/dist/agent/progress.js +135 -0
  13. package/dist/agent/prompt.js +79 -0
  14. package/dist/agent/replies.js +9 -0
  15. package/dist/agent/run.js +159 -0
  16. package/dist/agent/scheduling.js +290 -0
  17. package/dist/agent/time.js +18 -0
  18. package/dist/cli.js +88 -191
  19. package/dist/config/settings.js +60 -0
  20. package/dist/extensions/browser.js +326 -0
  21. package/dist/extensions/final-text.js +23 -0
  22. package/dist/extensions/memory.js +109 -0
  23. package/dist/extensions/message.js +33 -0
  24. package/dist/extensions/read-image.js +72 -0
  25. package/dist/extensions/schedules.js +41 -0
  26. package/dist/extensions/subagent.js +54 -0
  27. package/dist/extensions/web.js +99 -0
  28. package/dist/extensions/workspace.js +98 -0
  29. package/dist/main.js +568 -88
  30. package/dist/transport/attachments.js +37 -0
  31. package/dist/transport/http.js +110 -0
  32. package/dist/{rich-text.js → transport/rich-text.js} +25 -13
  33. package/dist/{send.js → transport/send.js} +13 -13
  34. package/dist/transport/watch.js +124 -0
  35. package/dist/web/app.js +748 -0
  36. package/dist/web/history.js +87 -0
  37. package/dist/web/memory.js +81 -0
  38. package/dist/web/page.js +28 -0
  39. package/dist/web/schedules.js +74 -0
  40. package/dist/web/server.js +348 -0
  41. package/dist/web/style.css +543 -0
  42. package/dist/web/tasks.js +63 -0
  43. package/docs/adr/0001-durable-owns-state.md +11 -0
  44. package/docs/adr/0002-module-layering.md +11 -0
  45. package/docs/adr/0003-replies-at-most-once.md +11 -0
  46. package/docs/adr/0004-native-extensions.md +11 -0
  47. package/docs/adr/0005-history-migration.md +11 -0
  48. package/docs/adr/0006-jsonl-storage.md +11 -0
  49. package/docs/adr/0007-images-by-path.md +11 -0
  50. package/docs/adr/0008-compaction-and-run.md +11 -0
  51. package/docs/adr/0009-one-shot-cutover.md +11 -0
  52. package/docs/adr/0010-test-budget.md +11 -0
  53. package/docs/adr/0011-reconcile-unsent-answers.md +24 -0
  54. package/docs/adr/0012-run-in-owned-conversation.md +22 -0
  55. package/docs/adr/0013-thin-adapters-around-durable.md +18 -0
  56. package/docs/adr/0014-ephemeral-private-storage.md +13 -0
  57. package/docs/adr/0015-ordinary-stop-keeps-native-inbox.md +21 -0
  58. package/docs/adr/0016-process-http-transport.md +21 -0
  59. package/docs/adr/0017-browser-cli-owned-scope.md +22 -0
  60. package/docs/adr/0018-tasks-without-prompt-api.md +19 -0
  61. package/docs/adr/0019-remove-scheduled-services.md +41 -0
  62. package/docs/adr/0020-chat-task-progress.md +41 -0
  63. package/docs/adr/0021-readable-progress-without-link-previews.md +33 -0
  64. package/docs/adr/0022-progress-admission-and-settlement.md +35 -0
  65. package/docs/adr/0023-retain-progress-type-prefixes.md +18 -0
  66. package/docs/adr/0024-status-marker-before-type-prefix.md +13 -0
  67. package/docs/adr/0025-reusable-progress-and-command-preview.md +49 -0
  68. package/docs/adr/0026-inline-command-only-requests.md +16 -0
  69. package/docs/adr/0027-command-preview-width.md +10 -0
  70. package/docs/adr/0028-command-preview-120.md +8 -0
  71. package/docs/adr/0030-shared-web-read-cache.md +33 -0
  72. package/docs/adr/0031-quiet-compaction-progress.md +32 -0
  73. package/docs/adr/0032-native-background-schedules.md +58 -0
  74. package/docs/adr/0033-operator-scheduled-run.md +23 -0
  75. package/docs/adr/0034-machine-local-schedule-time.md +36 -0
  76. package/docs/adr/0035-external-business-task-extensions.md +56 -0
  77. package/docs/adr/0036-workspace-policy-and-neutral-service-id.md +37 -0
  78. package/docs/adr/0037-restart-during-in-flight-work.md +30 -0
  79. package/docs/adr/0038-restart-any-time.md +25 -0
  80. package/docs/adr/0039-rerun-interrupted-tools.md +24 -0
  81. package/docs/adr/0040-chat-progress-notices.md +44 -0
  82. package/docs/adr/0041-restart-and-recovery.md +28 -0
  83. package/docs/adr/0042-background-schedules-and-workspace-extensions.md +44 -0
  84. package/docs/adr/0043-remove-legacy-migration.md +23 -0
  85. package/docs/adr/0044-message-channels.md +32 -0
  86. package/docs/adr/0045-chat-scheduling-tools.md +51 -0
  87. package/docs/adr/0046-selective-command-redaction.md +31 -0
  88. package/docs/adr/0047-one-scheduling-path.md +158 -0
  89. package/docs/adr/README.md +50 -0
  90. package/docs/api.md +92 -0
  91. package/docs/architecture.md +164 -0
  92. package/docs/browser.md +93 -0
  93. package/docs/memory.md +27 -0
  94. package/docs/screenshot.png +0 -0
  95. package/docs/workspace-extensions.md +133 -0
  96. package/examples/workspace-extensions/system-context/index.ts +69 -0
  97. package/examples/workspace-extensions/system-context/package.json +3 -0
  98. package/ops/README.md +267 -0
  99. package/ops/preflight.mjs +28 -0
  100. package/package.json +36 -37
  101. package/dist/agent.d.ts +0 -84
  102. package/dist/agent.d.ts.map +0 -1
  103. package/dist/agent.js +0 -780
  104. package/dist/agent.js.map +0 -1
  105. package/dist/cli.d.ts +0 -15
  106. package/dist/cli.d.ts.map +0 -1
  107. package/dist/cli.js.map +0 -1
  108. package/dist/imessage.d.ts +0 -33
  109. package/dist/imessage.d.ts.map +0 -1
  110. package/dist/imessage.js +0 -76
  111. package/dist/imessage.js.map +0 -1
  112. package/dist/logger.d.ts +0 -33
  113. package/dist/logger.d.ts.map +0 -1
  114. package/dist/logger.js +0 -104
  115. package/dist/logger.js.map +0 -1
  116. package/dist/main.d.ts +0 -5
  117. package/dist/main.d.ts.map +0 -1
  118. package/dist/main.js.map +0 -1
  119. package/dist/memory.d.ts +0 -53
  120. package/dist/memory.d.ts.map +0 -1
  121. package/dist/memory.js +0 -196
  122. package/dist/memory.js.map +0 -1
  123. package/dist/model-health.d.ts +0 -11
  124. package/dist/model-health.d.ts.map +0 -1
  125. package/dist/model-health.js +0 -69
  126. package/dist/model-health.js.map +0 -1
  127. package/dist/pipeline.d.ts +0 -27
  128. package/dist/pipeline.d.ts.map +0 -1
  129. package/dist/pipeline.js +0 -59
  130. package/dist/pipeline.js.map +0 -1
  131. package/dist/queue.d.ts +0 -23
  132. package/dist/queue.d.ts.map +0 -1
  133. package/dist/queue.js +0 -89
  134. package/dist/queue.js.map +0 -1
  135. package/dist/reminders.d.ts +0 -42
  136. package/dist/reminders.d.ts.map +0 -1
  137. package/dist/reminders.js +0 -207
  138. package/dist/reminders.js.map +0 -1
  139. package/dist/rich-text.d.ts +0 -26
  140. package/dist/rich-text.d.ts.map +0 -1
  141. package/dist/rich-text.js.map +0 -1
  142. package/dist/self-echo.d.ts +0 -19
  143. package/dist/self-echo.d.ts.map +0 -1
  144. package/dist/self-echo.js +0 -54
  145. package/dist/self-echo.js.map +0 -1
  146. package/dist/send.d.ts +0 -24
  147. package/dist/send.d.ts.map +0 -1
  148. package/dist/send.js.map +0 -1
  149. package/dist/settings.d.ts +0 -44
  150. package/dist/settings.d.ts.map +0 -1
  151. package/dist/settings.js +0 -80
  152. package/dist/settings.js.map +0 -1
  153. package/dist/store.d.ts +0 -57
  154. package/dist/store.d.ts.map +0 -1
  155. package/dist/store.js +0 -124
  156. package/dist/store.js.map +0 -1
  157. package/dist/tasks.d.ts +0 -95
  158. package/dist/tasks.d.ts.map +0 -1
  159. package/dist/tasks.js +0 -384
  160. package/dist/tasks.js.map +0 -1
  161. package/dist/types.d.ts +0 -93
  162. package/dist/types.d.ts.map +0 -1
  163. package/dist/types.js +0 -38
  164. package/dist/types.js.map +0 -1
  165. package/dist/watch.d.ts +0 -21
  166. package/dist/watch.d.ts.map +0 -1
  167. package/dist/watch.js +0 -219
  168. package/dist/watch.js.map +0 -1
  169. package/dist/web/data.d.ts +0 -11
  170. package/dist/web/data.d.ts.map +0 -1
  171. package/dist/web/data.js +0 -56
  172. package/dist/web/data.js.map +0 -1
  173. package/dist/web/html.d.ts +0 -5
  174. package/dist/web/html.d.ts.map +0 -1
  175. package/dist/web/html.js +0 -16
  176. package/dist/web/html.js.map +0 -1
  177. package/dist/web/index.d.ts +0 -25
  178. package/dist/web/index.d.ts.map +0 -1
  179. package/dist/web/index.js +0 -335
  180. package/dist/web/index.js.map +0 -1
  181. package/dist/web/render.d.ts +0 -11
  182. package/dist/web/render.d.ts.map +0 -1
  183. package/dist/web/render.js +0 -56
  184. package/dist/web/render.js.map +0 -1
  185. package/dist/web/templates/logs.eta +0 -75
  186. package/dist/web/templates/memory.eta +0 -91
  187. package/dist/web/templates/page.eta +0 -98
@@ -0,0 +1,158 @@
1
+ # 0047. One scheduling path for recurring and one-off work
2
+
3
+ Status: accepted (owner requested the minimal implementation on 2026-10-06)
4
+
5
+ ## Problem
6
+
7
+ Recurring workspace jobs currently enter through a native scheduler task, while
8
+ `serve --run-scheduled JOB --request-id ID` calls `runOnce()` and creates an
9
+ occurrence directly. Both use Durable, but admission follows two paths, and a
10
+ manual trial requires a service restart.
11
+
12
+ Calling all work "scheduled" is not sufficient. The common path must own deadline
13
+ persistence, deduplication and non-overlap. A trial must also retain the original
14
+ job's business state, rather than create an empty English learning history or
15
+ bypass the monthly quota.
16
+
17
+ ## Example
18
+
19
+ Workplace English has a daily 07:45 trigger. An operator requests one trial now.
20
+ The trial receives its own stable trigger ID, but targets the existing English
21
+ job and business conversation. Reloading twice does not run it twice, and the
22
+ regular 07:45 deadline does not move. Existing per-date card deduplication still
23
+ applies: a trial is not permission to resend today's card.
24
+
25
+ ## Evaluation
26
+
27
+ - **One common scheduling path:** recommended. Recurrence and an immediate trial
28
+ differ in their deadlines and repetition, not their executor or delivery path.
29
+ - **One physical configuration file:** unnecessary. Keep trusted business code
30
+ and configuration in the workspace extensions required by ADR 0042. Their
31
+ declarations pass through one framework admission function.
32
+ - **Automatic configuration polling:** unnecessary for the first version. The
33
+ existing `reload_extensions` tool and `/reload` already apply configuration
34
+ through the owning service without restart. Polling would add partial-write,
35
+ repeated-load and failure-handling concerns without helping deadline execution.
36
+ - **A local CLI/socket or HTTP run API:** unnecessary here. A trial can be a
37
+ one-off declaration followed by the existing reload. This trades one explicit
38
+ reload operation for avoiding another control protocol.
39
+ - **A separate business executor for every cadence:** rejected. Reminder,
40
+ script and model workflows remain native business tasks; they do not become
41
+ separate schedulers.
42
+
43
+ ## Decision
44
+
45
+ ### One admission and execution path
46
+
47
+ ```text
48
+ Workspace extension configuration
49
+ |
50
+ startup / reload_extensions / /reload
51
+ |
52
+ validate and register triggers
53
+ |
54
+ native background Scheduler task
55
+ |
56
+ sleep(persisted absolute deadline)
57
+ |
58
+ native occurrence wrapper
59
+ |
60
+ registered native business task
61
+ |
62
+ native receipts -> read-only Scheduled Web
63
+ |
64
+ optional outbox -> existing send receipts
65
+ ```
66
+
67
+ The reload commands are configuration-loading entry points, not direct execution
68
+ paths. No file edit alone invokes business work. An immediate trial becomes due
69
+ once its declaration has been accepted; it is not a promise of zero latency.
70
+ The service must be running, and publication of validated code/configuration must
71
+ complete before newly due work can execute under that definition.
72
+
73
+ The framework retains one owning Harness. Do not introduce another daemon,
74
+ operating-system timer, SDK scheduler bridge, database, or polling watcher.
75
+
76
+ ### Job identity and trigger identity
77
+
78
+ - A **job** names registered business code, input, destination and its existing
79
+ business conversation/state. It is not an arbitrary command or prompt supplied
80
+ by a new public execution API.
81
+ - A **trigger** names when that job is eligible to run. Recurring triggers and
82
+ one-off triggers use the same registration function and scheduler definition.
83
+ - The minimal version preserves elapsed-time intervals and one local daily slot,
84
+ and adds a one-off absolute deadline or `now`. Multiple daily slots and monthly
85
+ calendars are deferred, not part of this first change. Do not add a cron parser.
86
+ - Resolve `now` to an absolute timestamp once, in the admission transaction.
87
+ Persist the stable trigger ID, target job and deadline together. Subsequent
88
+ reloads/restarts recover that trigger, including a terminal one; they never
89
+ reevaluate `now`. A new intended trial needs a new trigger ID. Conflicting reuse
90
+ of a one-off ID fails explicitly.
91
+ - A one-off declaration targets an existing registered job. Do not give it a
92
+ fresh business conversation, reset learning/checkpoints, or change the regular
93
+ trigger's deadline. A workspace extension must provide this declarative trial
94
+ binding; trial requests do not require generating a new extension per run.
95
+ - Enforce non-overlap by target job, not merely by trigger. A due one-off can wait
96
+ for that job's active occurrence, then admit its own occurrence. Do not report
97
+ another trigger's work as completion of the new request.
98
+ - Commit occurrence admission and the scheduler checkpoint together. A one-off
99
+ scheduler becomes terminal after its occurrence; a recurring scheduler saves
100
+ the next deadline. Failures remain visible in native occurrence records.
101
+
102
+ ### Recovery, reload and delivery
103
+
104
+ - Preserve ADR 0042's machine-local calendar/DST rules, saved absolute deadlines,
105
+ daily/interval overdue behavior and lack of historical backfill. A one-off
106
+ admitted before shutdown runs once when overdue after restart.
107
+ - Load and validate a complete candidate before accepting declarations. A failed
108
+ reload leaves old definitions and schedules active and must not start a trial.
109
+ Reload does not interrupt existing business calls. Pausing/removing a declaration
110
+ prevents future admission, not undoing already admitted effects.
111
+ - Use native task versions/migrations where persisted structures change. Preserve
112
+ existing job/conversation/task IDs, learning state, checkpoints, execution
113
+ history and delivery receipts. Do not recreate terminal trials on startup.
114
+ - Business tasks keep their own replay and success policy under ADR 0041. A shared
115
+ clock does not make a script's external effects transactional or replay-safe.
116
+ Reflection advances its checkpoint and starts backup only after verified success.
117
+ - Keep the existing outgoing queue and at-most-once transport receipts. An
118
+ interrupted send becomes `unknown` and is not automatically resent. One-off
119
+ execution does not override a business quota, per-date dedupe or a send receipt.
120
+ - Reuse the compact Scheduled page and native occurrence history. Show one-off
121
+ executions under their target job, with links to their Task tree. Viewing the
122
+ page starts no work; add no execution buttons.
123
+
124
+ ### Entry points removed or deferred
125
+
126
+ After the unified one-off path is available, remove the operator startup-run
127
+ flags and the direct-occurrence `runOnce()` bypass. Keep their historical request
128
+ and execution records; do not replay them during migration. Do not add a CLI or
129
+ Unix socket replacement.
130
+
131
+ ADR 0045 remains a separate proposal about chat scheduling tools. This ADR does
132
+ not enable or reject those tools. If accepted, their declarations must use this
133
+ same admission/clock/occurrence path, preserve chat ownership boundaries, and
134
+ must not introduce another scheduler or call a business executor directly.
135
+
136
+ ## Scope and consequences
137
+
138
+ Framework changes concern the schedule declaration contract, common admission,
139
+ calendar calculation, scheduler lifecycle and removal of the operator bypass.
140
+ Personal messages, weather/news logic, memory updates, photo-date checks and
141
+ Reflection remain workspace extensions, not framework-specific branches.
142
+
143
+ Restoring the selected legacy jobs uses this path. Stop only their competing old
144
+ cron producers during the authorized cutover; do not disable unrelated system
145
+ jobs. Fixed-message business tasks do not restore the retired reminder service,
146
+ PT workers or HTTP `/prompt` / `/send`. No restoration or migration is performed
147
+ by accepting a file reload alone
148
+ unless those jobs are explicitly declared.
149
+
150
+ The first version requires an explicit reload after editing configuration. This
151
+ is intentional: it avoids a new watcher and a new management service. Persistence
152
+ cannot wake a stopped process or machine. Exactly-once external delivery is not
153
+ claimed.
154
+
155
+ This supersedes only ADR 0042's operator-run clause (which
156
+ consolidated ADR 0033), and extends its cadence/declaration contract. Its workspace
157
+ ownership, recovery and delivery rules remain in force. Accepted ADRs stay
158
+ unchanged.
@@ -0,0 +1,50 @@
1
+ # Architecture decision records
2
+
3
+ Accepted ADR files are never edited; this index records which ones are current.
4
+ "Active" means the decision is in force. A replaced ADR points to its successor.
5
+
6
+ ## Current
7
+
8
+ | ADR | Decision | Status |
9
+ |---|---|---|
10
+ | [0001](0001-durable-owns-state.md) | Pi Durable owns all chat state | Active |
11
+ | [0002](0002-module-layering.md) | Modules in strict layers | Active |
12
+ | [0003](0003-replies-at-most-once.md) | Replies are sent at most once | Active |
13
+ | [0004](0004-native-extensions.md) | Native Durable extensions, not an SDK adapter | Active; Codex tier clause replaced by 0036 |
14
+ | [0006](0006-jsonl-storage.md) | One JSONL storage, backed up by hand | Active |
15
+ | [0007](0007-images-by-path.md) | Store images as file paths | Active |
16
+ | [0008](0008-compaction-and-run.md) | Six-hour compaction | Active; `/run` clause replaced by 0012 |
17
+ | [0010](0010-test-budget.md) | About 40 tests | Active |
18
+ | [0011](0011-reconcile-unsent-answers.md) | Reconcile unsent answers | Active |
19
+ | [0012](0012-run-in-owned-conversation.md) | `/run` in an owned conversation | Active |
20
+ | [0013](0013-thin-adapters-around-durable.md) | Thin adapters around Durable | Active; scheduler clause replaced by 0042 |
21
+ | [0016](0016-process-http-transport.md) | One process-owned HTTP transport | Active |
22
+ | [0017](0017-browser-cli-owned-scope.md) | Conversation-scoped browser CLI tool | Accepted, to be implemented |
23
+ | [0018](0018-tasks-without-prompt-api.md) | Ordinary `/stop`; no HTTP `/prompt` or `/send` | Active |
24
+ | [0036](0036-workspace-policy-and-neutral-service-id.md) | Workspace policy and neutral service ID | Active |
25
+ | [0040](0040-chat-progress-notices.md) | Chat progress notices | Active; consolidates 0020–0028, 0031 |
26
+ | [0041](0041-restart-and-recovery.md) | Restart and recovery | Active; consolidates 0037–0039 |
27
+ | [0042](0042-background-schedules-and-workspace-extensions.md) | Background schedules and workspace extensions | Active; operator-run clause replaced by 0047 |
28
+ | [0043](0043-remove-legacy-migration.md) | Remove the legacy migration code | Active; archived at tag `archive/legacy-migration` |
29
+ | [0044](0044-message-channels.md) | Organize message sources as channels | Proposed |
30
+ | [0045](0045-chat-scheduling-tools.md) | Native chat scheduling tools | Proposed |
31
+
32
+ | [0046](0046-selective-command-redaction.md) | Selective command-preview redaction | Proposed; owner-requested implementation in #36 |
33
+ | [0047](0047-one-scheduling-path.md) | One scheduling path for recurring and one-off work | Active |
34
+
35
+ ## Replaced or rejected
36
+
37
+ | ADR | Decision | Replaced by |
38
+ |---|---|---|
39
+ | [0005](0005-history-migration.md) | Import old history once | 0043 (done) |
40
+ | [0009](0009-one-shot-cutover.md) | Switch over in one step | 0043 (done) |
41
+ | [0014](0014-ephemeral-private-storage.md) | Ephemeral private storage | 0018 |
42
+ | [0015](0015-ordinary-stop-keeps-native-inbox.md) | `/stop` keeps the native inbox | 0018 |
43
+ | [0030](0030-shared-web-read-cache.md) | Shared web read cache | Rejected; removed |
44
+ | [0019](0019-remove-scheduled-services.md) | Remove scheduled services | 0042 |
45
+ | [0020](0020-chat-task-progress.md)–[0028](0028-command-preview-120.md) | Progress notice format | 0040 |
46
+ | [0031](0031-quiet-compaction-progress.md) | Quiet six-hour compaction progress | 0040 |
47
+ | [0032](0032-native-background-schedules.md)–[0035](0035-external-business-task-extensions.md) | Native schedules, operator run, local time, workspace extensions | 0042 |
48
+ | [0037](0037-restart-during-in-flight-work.md)–[0039](0039-rerun-interrupted-tools.md) | Restart and tool rerun | 0041 |
49
+
50
+ Number 0029 was never used.
package/docs/api.md ADDED
@@ -0,0 +1,92 @@
1
+ # HTTP API
2
+
3
+ [Back to README](../README.md)
4
+
5
+ Default address: `http://localhost:7750` (configured by `WEB_HOST` and `WEB_PORT`).
6
+ The server has no authentication; do not expose it to untrusted networks.
7
+
8
+ The pages `/`, `/chat`, `/tasks`, `/scheduled`, `/settings`, `/memory` and `/logs` are read-only.
9
+ The Tasks tab shows real chat and native schedule ownership trees, including
10
+ subagent child conversations and terminal tasks. Internal-only conversations such as model-health
11
+ are hidden, including when selected by URL; their stored data is not deleted.
12
+ `/tasks/data` retains all conversation records for native diagnostics
13
+ and lightweight task summaries, reading every native scan page. Input and payloads
14
+ are fetched only when a task is expanded, through `/tasks/task/data?taskId=ID`.
15
+ `/tasks?conversationId=ID` focuses one conversation and its descendants. The tree
16
+ and expanded tasks update only on manual refresh. Conversation lists and sidebars
17
+ use newest message activity first, with conversation ID as a descending tie-break;
18
+ Tasks uses the same activity clock as Recent Chats, not conversation creation order.
19
+ Conversation windows have a maximum height of `min(560px, 70dvh)` and scroll inside. These are latest task states,
20
+ not checkpoint history: terminal tasks retain input and outcome, but no checkpoint.
21
+ Existing redaction and large-string truncation apply to task details too.
22
+ `/chat/data?conversationId=ID` exposes history/live state; `/events` announces
23
+ committed changes without executing anything. Native history cursors use
24
+ base64url-encoded JSON in `cursor`; long results are paged and large text is
25
+ truncated. HTML, JSON, JavaScript and CSS responses of at least 1 KiB support
26
+ negotiated gzip compression with `Vary: Accept-Encoding`. SSE stays uncompressed
27
+ so notifications are not buffered.
28
+
29
+ The chat overview embeds at most 15 displayed messages per conversation in its
30
+ initial HTML, ordered by message timestamps (including imported dates). Group
31
+ names are retained even when their source message is outside that window.
32
+ `/chat/data?view=overview` batches the same bounded view for refreshes, without
33
+ system prompts, tool results, image payloads or agent configuration.
34
+ `/chat/data?conversationId=ID&view=display` returns display-only history, chat
35
+ metadata, reply settings and a revision marker without separate index/settings
36
+ requests; detail history and its older-page cursors are not reduced to 15
37
+ messages. Existing `/chat/data` and conversation detail responses without these
38
+ view parameters retain their native state shapes.
39
+
40
+ Every request reads current state directly; there is no server-side read cache
41
+ (ADR 0030 was rejected).
42
+
43
+ Pages load once and refresh only through the Refresh button, memory filtering/
44
+ pagination or explicit navigation. There is no browser SSE connection, periodic
45
+ polling or visibility-triggered refresh. Expanded task payloads are still loaded
46
+ on demand. The standalone `/events` API remains compatible for existing clients;
47
+ the web pages do not use it. Cache invalidation still makes manual reads current.
48
+
49
+ Memory pages use server-side filtering and 20-record pagination. The additive
50
+ `/memory/data?view=records` view accepts `q`, `namespace`, `kind`, `status` and
51
+ `page`; status defaults to `active`, and an empty status selects all records.
52
+ Responses include only the selected page plus counts and filter choices.
53
+ Supersession is resolved across namespaces before filtering. Complete JSONL
54
+ records and long memory text are never character-truncated; invalid source lines
55
+ are counted and reported without modifying source files. The original
56
+ `/memory/data` raw namespace-array shape remains available with complete content.
57
+ Known credential fields/token patterns are redacted, not a guarantee
58
+ that arbitrary user-written secrets can be identified.
59
+
60
+ The agent sends files and extra messages with its immediate `send_message`
61
+ tool. HTTP `/prompt` and `/send` remain retired. Legacy cron management,
62
+ reminders and PT Automation routes remain removed and return 404 (ADR 0019).
63
+
64
+ `GET /scheduled` and `GET /scheduled/data` expose configured native background
65
+ jobs (ADR 0032), not the retained legacy cron files. The JSON view contains `jobs`
66
+ with cadence, enabled state, native scheduler task/conversation IDs, phase and
67
+ `nextAt`, and each job's latest ten execution tasks. `recent` contains the latest
68
+ ten executions across all jobs. Results separate native task outcome from the
69
+ message transport receipt (`sent` is not recipient confirmation). These views
70
+ never resume the Harness or invoke a model. POST/DELETE return 404; there are no
71
+ run, pause or resend buttons. Scheduling configuration takes effect at startup.
72
+ Daily slots and learning dates use the service machine's local timezone (or an
73
+ explicit process `TZ`). Each job includes `timezone`; Web deadlines and history
74
+ use it even when the browser is in another timezone. Saved absolute deadlines
75
+ remain unchanged on restart (ADR 0034).
76
+
77
+ | Endpoint | Description | Example |
78
+ |---|---|---|
79
+ | `GET /health/model` | Make a live request to the configured default AI model; returns HTTP 200 when healthy or 503 on failure | `curl localhost:7750/health/model` |
80
+
81
+ The model health endpoint can incur model usage. It is not a passive startup
82
+ probe and never sends its answer to an iMessage chat.
83
+
84
+ Set `progressMessages: true` in `settings.json` to send one short iMessage notice
85
+ for each newly admitted native task/tool in reply-enabled chats, except
86
+ `pi.generation`. Child calls go to their parent chat. Notices use native names
87
+ without IDs, with bounded argument summaries and screened path/search-query
88
+ previews. Raw outputs, commands, prompts and document contents stay local.
89
+ Known credential indicators are redacted; arbitrary user-authored secrets cannot
90
+ be guaranteed detectable. Missing or false keeps final-only delivery. The setting
91
+ is reread for each notice; old tasks and unknown sends are not replayed
92
+ ([ADR 0020](adr/0020-chat-task-progress.md)).
@@ -0,0 +1,164 @@
1
+ # Architecture
2
+
3
+ ## 1. Packages
4
+
5
+ ```text
6
+ ┌────────────────────────────────────────────┐
7
+ │ 1 main.ts · cli.ts │
8
+ ├────────────────────────────────────────────┤
9
+ │ 2 web/ │
10
+ ├────────────────────────────────────────────┤
11
+ │ 3 agent/ │
12
+ ├────────────────────────────────────────────┤
13
+ │ 4 extensions/ │
14
+ ├────────────────────────────────────────────┤
15
+ │ 5 transport/ │
16
+ ├────────────────────────────────────────────┤
17
+ │ 6 config/ │
18
+ └────────────────────────────────────────────┘
19
+ Imports go downward only.
20
+ ```
21
+
22
+ ### Notes
23
+
24
+ These are folders under `src/`, not separate npm packages.
25
+ The root `main.ts` and `cli.ts` files form one host package.
26
+ Lower layer numbers may import higher ones. Different packages in the same layer
27
+ cannot import each other; imports within a package are allowed.
28
+
29
+ | Layer | Package | Responsibility |
30
+ |---|---|---|
31
+ | 1 | `main.ts`, `cli.ts` | Startup, wiring and installation. |
32
+ | 2 | `web/` | HTTP API; read-only pages for chat history and live state. |
33
+ | 3 | `agent/` | Harness, models, chat mapping, commands, delivery, prompt, `/run`, native background scheduling and maintenance. |
34
+ | 4 | `extensions/` | Built-in native tools and loading of trusted workspace business extensions. |
35
+ | 5 | `transport/` | Poll chat.db, archive attachments, send text and files through Messages.app. No agent logic. |
36
+ | 6 | `config/` | Settings, environment variables and chat allowlist. |
37
+
38
+ Enforcement: `test/architecture.test.ts`. Rationale: [ADRs](adr/).
39
+
40
+ Personal tool and task code lives under `WORKING_DIR/extensions/<name>/index.ts` with
41
+ its own `config.json`. Startup and `/reload` scan direct child directories. Native
42
+ CommonJS module factories (`module.exports`) receive the pinned Durable primitives;
43
+ the loader never scans skills or attachments, installs dependencies or adapts SDK
44
+ extensions. Source and configuration reload without interrupting current calls.
45
+ Failed reloads retain old definitions and schedule configuration. Removed task
46
+ code stays in memory for calls that can still admit children; missing required
47
+ code fails explicitly on restart. The assistant's `reload_extensions` tool uses
48
+ the same service owner. See [workspace extensions](workspace-extensions.md),
49
+ ADR 0035 and ADR 0036.
50
+
51
+ The built-in native `browser` extension wraps the pinned pi-browser CLI with
52
+ host-derived conversation scopes. Browser children get private HOME/cache,
53
+ profile/output and cwd; other coding tools are unchanged. The host owns scoped
54
+ stop/reset/shutdown/restart cleanup, while Durable owns invocation state and
55
+ unsafe replay policy. Read-only contexts do not offer the tool. This is not an
56
+ OS sandbox; see [browser operations and retention](browser.md) and ADR 0017.
57
+
58
+
59
+ ### Native schedules
60
+
61
+ `agent/scheduling.ts` registers an extension with native task definitions. A
62
+ job's dedicated conversation owns a background scheduler, whose checkpoint saves
63
+ the next deadline. Each occurrence is a child execution task delegating to an
64
+ extension's native business task; terminal receipts supply Web history, not a
65
+ parallel UI log. The framework owns daily/interval cadence, non-overlap, recovery
66
+ and quiet six-hour native compaction. Workspace tasks own business policy and
67
+ documents, and can commit their state and the generic outgoing item atomically.
68
+ The host delivery poll uses existing direct-send receipts and passive chat records.
69
+ `web/schedules.ts` projects these native records, including latest-ten history.
70
+ Daily slots and learning dates use the service machine's local timezone; the Web
71
+ view uses the same timezone. Saved absolute deadlines survive restart. See
72
+ [ADR 0032](adr/0032-native-background-schedules.md) and
73
+ [ADR 0034](adr/0034-machine-local-schedule-time.md).
74
+
75
+ ## 2. Workflow
76
+
77
+ ### Overall flow
78
+
79
+ ```text
80
+ User (iMessage / SMS)
81
+ |
82
+ v
83
+ transport/watch.ts
84
+ Poll chat.db, filter, save attachments
85
+ |
86
+ v
87
+ agent/chats.ts
88
+ Find conversation; submit with GUID as requestId
89
+ Save row cursor after acceptance; steer if busy
90
+ |
91
+ v
92
+ Pi Durable Harness <--------> durable/ (JSONL)
93
+ | |
94
+ | final answer +--> web/ (history and live state)
95
+ v
96
+ agent/deliver.ts
97
+ Record sending -> transport/send.ts -> record sent
98
+ |
99
+ v
100
+ Messages.app -> User
101
+ ```
102
+
103
+ ### Inside the Harness
104
+
105
+ These concepts and files are native to Pi Durable's JSONL storage.
106
+ Paths are relative to `WORKING_DIR/durable/`.
107
+
108
+ | Member | Role | File |
109
+ |---|---|---|
110
+ | Conversation | Conversation scope linking history and tasks. | `main.jsonl` |
111
+ | Generation | Requests the model, waits for tools and advances one model turn. | Active: `task-<id>.jsonl`; terminal: `main.jsonl` |
112
+ | Entry | Immutable history record: user input, model response or tool result. | `main.jsonl` |
113
+ | `pi.live` | Current run state, partial model response and running tool output. | `doc-<id>.jsonl` |
114
+ | `pi.inbox` | Queued submissions received while the conversation is busy. | `doc-<id>.jsonl` |
115
+
116
+ Example: "Show the current directory.", then a `steer` while the tool runs.
117
+ Based on a verified Durable 1.0.2 run; test timing details are omitted.
118
+ The second Generation is not shown. File mappings are in the table above.
119
+
120
+ ```text
121
+ Conversation / Tasks Model Tool
122
+ | | |
123
+ | Conversation 1 (main.jsonl) - create | |
124
+ | | |
125
+ | Generation task (task-9.jsonl) | |
126
+ | ▶ phase=prepare; state=pending | |
127
+ | pi.live - run.taskId=9, inputs=[8] | |
128
+ | Prepare request; Entry 10 (main.jsonl) - pi.system | |
129
+ | ▶ phase=request; state=running | |
130
+ | pi.live - generation.attempt=1 | |
131
+ | ------------------ request with committed context ------------------> |
132
+ | <------------------ complete response: bash call -------------------| |
133
+ | Entry 11 (main.jsonl) - pi.assistant, byTaskId=9 | |
134
+ | pi.usage - update model usage | |
135
+ | ▶ phase=tools; state=waiting, on=[12] | |
136
+ | pi.live - clear partial; tool 12 pending | |
137
+ | | |
138
+ | Tool Task T12 (task-12.jsonl), owner=9 | |
139
+ | ▶ phase=call; state=pending | |
140
+ | Resolve tool; validate arguments | |
141
+ | ▶ phase=execute; state=running | |
142
+ | Save arguments + replay=safe; pi.live tool running | |
143
+ | -------------------------------- execute command="pwd" --------------------------------->
144
+ | <----------------------------- result: current directory -------------------------------|
145
+ | Entry 14 (main.jsonl) - pi.tool-result, byTaskId=12 | |
146
+ | | |
147
+ | ▶ T12 (main.jsonl): state=terminal, outcome=completed | |
148
+ | pi.live - tool done, entry=14; clear output | |
149
+ | | |
150
+ | ▶ phase=tools (reused); state=running | |
151
+ | Place steer: Entry 15 (main.jsonl) - pi.user, byTaskId=9 | |
152
+ | Submission 13 (main.jsonl) - status=placed | |
153
+ | pi.inbox - remove item 13 | |
154
+ | | |
155
+ | ▶ G9 (main.jsonl): state=terminal, outcome=completed | |
156
+ | pi.live - remove tools; run.taskId=16 | |
157
+ | | |
158
+ | Next generation continues the run; final settlement is not shown. | |
159
+ | | |
160
+ ```
161
+
162
+ Upstream reference: [Watching a Conversation](https://github.com/earendil-works/pi/blob/main/packages/durable/README.md#watching-a-conversation)
163
+ and [Busy Conversations](https://github.com/earendil-works/pi/blob/main/packages/durable/README.md#busy-conversations).
164
+ Upstream `main` is experimental; the examples here describe the pinned version.
@@ -0,0 +1,93 @@
1
+ # Conversation-owned browser
2
+
3
+ The host's built-in native Durable `browser` extension implements ADR 0017. It is
4
+ registered with the other built-in tools, not loaded as a personal workspace
5
+ business extension. It invokes the installed `PI_CODING_AGENT_DIR/bin/pi-browser`
6
+ wrapper and its pinned Playwright CLI by argv; there is no SDK shim, private CLI
7
+ import or second browser driver.
8
+
9
+ ## Operations
10
+
11
+ `browser` takes `action` and only the arguments needed by that action:
12
+
13
+ - `open`, `goto`, `tab-new`: optional `url` (`goto` requires one), HTTP(S) without credentials.
14
+ - `snapshot`, `screenshot`: optional `ref` from the latest snapshot.
15
+ - `click`, `hover`, `check`, `uncheck`: required `ref`.
16
+ - `fill`, `select`: required `ref` and `text`.
17
+ - `press`: required `key`.
18
+ - `tab-select`: required `tab`; `tab-close` accepts an optional index.
19
+ - `go-back`, `go-forward`, `tab-list`, `close`: no further arguments.
20
+
21
+ Open before interacting. A snapshot includes bounded accessibility text with
22
+ current refs. Screenshot/other artifact links resolve from the reported private
23
+ cwd; the result also gives the absolute output directory. Page output is
24
+ untrusted. No attachment, discovery, headed overrides, global cleanup, arbitrary
25
+ profile/config/filename, storage export/import or arbitrary code execution is
26
+ exposed to the model. Positional text follows `--`, so strings resembling flags
27
+ remain literal. Native health checks and read-only subagents do not offer the tool.
28
+
29
+ ## Isolation and retention
30
+
31
+ Scope is derived from the canonical Durable storage directory and native
32
+ conversation ID, never a model-selected session. Ordinary chats, run conversations
33
+ and other tool-enabled native conversations receive distinct scope directories:
34
+
35
+ ```text
36
+ WORKING_DIR/browser/<sha256-of-canonical-storage>/<conversation-id>/
37
+ owner.json
38
+ home/ # private HOME/cache/CLI daemon registry
39
+ cwd/.playwright/ # private workspace identity and managed headless config
40
+ state/profiles/owned/ # dedicated non-personal persistent Chrome profile
41
+ state/output/ # snapshots/screenshots
42
+ state/auth.json # private last completed storage-state checkpoint
43
+ ```
44
+
45
+ Browser child processes alone get the private HOME/cwd and a small environment
46
+ allowlist. Browser target/config/test environment overrides are not inherited.
47
+ Ordinary bash/read keep their original environment. Scope directories are 0700;
48
+ authentication checkpoints are 0600, not returned to the model. Public scoped
49
+ `state-save`/`state-load` preserve cookie/local-storage login state across browser
50
+ restarts alongside the retained profile. Restore before navigating the supplied
51
+ URL, not by repeating the navigation. Before orderly close/reopen, a host-only
52
+ public scoped session query checks the owned profile and refreshes its checkpoint,
53
+ including asynchronous token changes since the last command. Checkpoints publish
54
+ by atomic rename, so an interrupted save cannot truncate the last completed state. On recovery a live
55
+ owned daemon is checkpointed before cleanup; if it already died, only the last
56
+ completed checkpoint is available and may require renewed login. Profiles,
57
+ checkpoints and outputs are not automatically deleted. Real-site login expiration
58
+ and IndexedDB-based workflows are not guaranteed by the synthetic fixture.
59
+
60
+ Operations serialize per conversation, while other conversations remain
61
+ independent. `/stop` and `/new` close the chat's owned browsers, including run/task
62
+ child scopes, after aborting current work and before admitting the next boundary.
63
+ Host shutdown stops native work, closes owned sessions, then releases the native
64
+ owner lock. Startup closes saved owned sessions after acquiring that lock, before
65
+ resuming work.
66
+ Only matching ownership markers and scoped `close` are used; malformed ownership
67
+ or a cleanup failure is an explicit error, never grounds for global killing or
68
+ silent profile deletion.
69
+
70
+ The tool declares `replay: "unsafe"` even though the host otherwise defaults tools
71
+ to safe. A click/press can already have submitted something when its caller is
72
+ interrupted. Cancellation/timeout does not prove the effect was undone; do not
73
+ blindly retry. Browser calls have request-bound deadlines and await CLI child exit
74
+ on cancellation. SIGTERM gets a short grace period, then only that exact owned CLI
75
+ child is terminated; this is never a global Chrome/service kill. Raw subprocess
76
+ argv/errors (which can contain form contents) are not surfaced, and snapshots are
77
+ read with a byte limit.
78
+
79
+ This is a tool boundary, **not an OS sandbox**. Unrestricted coding tools can
80
+ still access host files or invoke another browser process. Stronger containment
81
+ requires a separate decision. Never copy a personal Chrome profile, bypass a
82
+ CAPTCHA/2FA or perform an unapproved irreversible action.
83
+
84
+ ## Verification
85
+
86
+ - `test/browser.test.ts` uses a faux model and fake CLI boundary to check native
87
+ registration, forced arguments/environment, forbidden controls, literal flag-like
88
+ form text, owned stop/reset/restart, retention, cancellation and unsafe replay.
89
+ - `node --experimental-strip-types ops/browser-smoke.mjs` uses the real installed
90
+ CLI and headless Chrome with temporary profiles and loopback pages. It checks
91
+ same-name sessions, daemon discovery, cookies/local storage, tabs/output files,
92
+ literal arguments, selective cleanup and preserved login checkpoints on reopen.
93
+ It never opens a personal profile, calls a paid model or sends an iMessage.
package/docs/memory.md ADDED
@@ -0,0 +1,27 @@
1
+ # Structured Memory
2
+
3
+ [Back to README](../README.md)
4
+
5
+ Durable memories are stored as namespaced JSONL records under
6
+ `WORKING_DIR/skills/file-memory/namespaces/`. Each record includes a stable ID,
7
+ creation timestamp, factual event date when known, source, kind, subjects,
8
+ importance, confidence, and status. This keeps every memory traceable without
9
+ inventing dates for older facts.
10
+
11
+ The agent uses typed tools to load complete namespaces, search for specific
12
+ records, and append validated memories. Corrections append a new record that
13
+ supersedes the old ID, preserving an auditable history instead of silently
14
+ rewriting it. Legacy global and per-chat `MEMORY.md` files remain read-only
15
+ archives.
16
+
17
+ The service calls the store's existing `memory_cli.py` in that directory using
18
+ Python 3 and argument arrays, not a shell or an embedded backend. The CLI is a
19
+ runtime prerequisite; missing or failed calls return tool errors. Tool
20
+ cancellation terminates the subprocess, but an already completed write is not
21
+ undone. Saves are replay-safe because the store assigns deterministic IDs and
22
+ skips identical records; corrections pass the existing `supersedes` option.
23
+
24
+ Core memory (`core.md`) and the namespace list are loaded at startup and refreshed
25
+ by `/reload`. `load_memory` and `search_memory` always read the current store.
26
+ Explicit extension selection can replace the service's default memory tools for
27
+ isolated callers or restricted execution.
Binary file