@pmndre/pine 0.22.1 → 0.33.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (99) hide show
  1. package/README.md +2 -1
  2. package/changes.json +264 -0
  3. package/docs/SKILL.md +31 -7
  4. package/docs/references/agents.md +245 -21
  5. package/docs/references/coding.md +1 -1
  6. package/docs/references/computer.md +1 -1
  7. package/docs/references/configuration.md +14 -5
  8. package/docs/references/mcp.md +9 -0
  9. package/docs/references/operations.md +21 -10
  10. package/docs/references/overview.md +3 -1
  11. package/docs/references/security.md +83 -24
  12. package/docs/references/setup.md +55 -9
  13. package/docs/references/source-types.md +1418 -0
  14. package/docs/references/sources.md +333 -19
  15. package/docs/references/stack.md +1 -0
  16. package/docs/references/tui.md +86 -12
  17. package/examples/home/AGENTS.md +4 -0
  18. package/examples/home/TOOLS.md +5 -0
  19. package/examples/home/agents/main/workspace/AGENTS.md +1 -1
  20. package/examples/home/agents/main/workspace/TOOLS.md +5 -0
  21. package/mcp/browser.d.ts +6 -0
  22. package/mcp/browser.js +19 -0
  23. package/mcp/render.d.ts +1 -0
  24. package/mcp/render.js +237 -0
  25. package/package.json +6 -4
  26. package/src/actions.d.ts +6 -1
  27. package/src/actions.js +17 -4
  28. package/src/admin.d.ts +4 -0
  29. package/src/admin.js +25 -6
  30. package/src/agents.d.ts +167 -2
  31. package/src/agents.js +912 -45
  32. package/src/catalogue.d.ts +60 -0
  33. package/src/catalogue.js +791 -0
  34. package/src/cli.js +10 -7
  35. package/src/config.d.ts +53 -4
  36. package/src/config.js +30 -5
  37. package/src/goals.d.ts +1 -1
  38. package/src/goals.js +14 -1
  39. package/src/manual.js +24 -2
  40. package/src/mime.d.ts +78 -0
  41. package/src/mime.js +342 -0
  42. package/src/net.d.ts +11 -0
  43. package/src/net.js +29 -2
  44. package/src/pine.js +10 -1
  45. package/src/router.js +3 -3
  46. package/src/sandbox-linux.d.ts +3 -1
  47. package/src/sandbox-linux.js +5 -0
  48. package/src/security.d.ts +22 -6
  49. package/src/security.js +44 -13
  50. package/src/slots.d.ts +106 -0
  51. package/src/slots.js +250 -0
  52. package/src/source-types.d.ts +20 -0
  53. package/src/sources/calendar.d.ts +24 -0
  54. package/src/sources/calendar.js +265 -0
  55. package/src/sources/command.js +109 -4
  56. package/src/sources/discord.d.ts +4 -0
  57. package/src/sources/discord.js +298 -0
  58. package/src/sources/feed.d.ts +19 -0
  59. package/src/sources/feed.js +257 -0
  60. package/src/sources/http.js +154 -103
  61. package/src/sources/mail.d.ts +51 -0
  62. package/src/sources/mail.js +1013 -0
  63. package/src/sources/matrix.d.ts +4 -0
  64. package/src/sources/matrix.js +232 -0
  65. package/src/sources/modbus.d.ts +15 -0
  66. package/src/sources/modbus.js +240 -0
  67. package/src/sources/onvif.d.ts +35 -0
  68. package/src/sources/onvif.js +326 -0
  69. package/src/sources/presence.d.ts +80 -0
  70. package/src/sources/presence.js +480 -0
  71. package/src/sources/serial.d.ts +21 -0
  72. package/src/sources/serial.js +220 -0
  73. package/src/sources/slack.d.ts +16 -0
  74. package/src/sources/slack.js +1062 -0
  75. package/src/sources/smtp.d.ts +4 -0
  76. package/src/sources/smtp.js +360 -0
  77. package/src/sources/stream.d.ts +68 -0
  78. package/src/sources/stream.js +534 -0
  79. package/src/sources/syslog.d.ts +32 -0
  80. package/src/sources/syslog.js +243 -0
  81. package/src/sources/system.d.ts +14 -0
  82. package/src/sources/system.js +236 -0
  83. package/src/sources/system.swift +102 -0
  84. package/src/sources/telegram.js +34 -1
  85. package/src/sources.d.ts +41 -1
  86. package/src/sources.js +147 -24
  87. package/src/tools.d.ts +1 -1
  88. package/src/tools.js +304 -23
  89. package/src/tui/app.d.ts +35 -2
  90. package/src/tui/app.js +306 -43
  91. package/src/tui/levels.d.ts +68 -0
  92. package/src/tui/levels.js +273 -0
  93. package/src/tui/picture.d.ts +17 -1
  94. package/src/tui/picture.js +72 -5
  95. package/src/types.d.ts +3 -0
  96. package/src/update.d.ts +9 -1
  97. package/src/update.js +75 -3
  98. package/src/watching.d.ts +156 -0
  99. package/src/watching.js +609 -4
@@ -0,0 +1,1418 @@
1
+ # Source types: what each one watches, what it takes, what it is for
2
+
3
+ A source reports what happens somewhere else, and an agent that is subscribed to it is woken
4
+ with each report. What a source and a subscription are, who is woken, what trust an event has
5
+ and how secrets are kept is in [sources.md](sources.md). This file is the catalogue: every
6
+ type there is, the options it takes, the events it reports with their fields, and the services
7
+ and devices each one reaches, with an entry to copy for each.
8
+
9
+ The seven everyday types (`folder`, `mqtt`, `webhook`, `ftp`, `telegram`, `http`, `command`)
10
+ are described in [sources.md](sources.md), "The built-in types". The others are described
11
+ here.
12
+
13
+ ## Which type for what
14
+
15
+ Two kinds of type exist. One **is told**: it listens on a port or holds a connection open, and
16
+ an event arrives the moment something happens. The other **looks**: it asks again and again
17
+ and reports what is different from the last look. A look that finds nothing new costs
18
+ nothing: no event, nothing stored, no model woken.
19
+
20
+ | Type | Told or looks | Reports | For |
21
+ |---|---|---|---|
22
+ | `folder` | told | a new or changed file | scans, downloads, exports, a project folder |
23
+ | `webhook` | told | a request another service sends to pine | GitHub, Home Assistant, a shortcut on a phone, anything that can call a web address |
24
+ | `stream` | told | each message a service pushes over Server-Sent Events or a WebSocket | ntfy, Gotify, Home Assistant, Mastodon, Bluesky, exchange tickers, webhook relays, signal-cli |
25
+ | `mqtt` | told | messages on the topics of a broker | Zigbee and Z-Wave hubs, sensors, plugs, printers |
26
+ | `telegram` | told | what you write to your bot | talking to an agent from anywhere |
27
+ | `mail` | told | a new mail in a mailbox | invoices, notifications of services that only write mail, a shared inbox |
28
+ | `slack` | told | what happens in a Slack workspace: messages (by channel, mention, keyword or sender, also those of other apps), reactions, edits, people joining, channels, pins, slash commands | being told when somebody needs you, an alert bot posts, a customer reacts; answering in a thread, with a reaction or a file |
29
+ | `matrix` | told | messages in the rooms of a Matrix account | Element rooms, bridges to other networks |
30
+ | `discord` | told | messages in the channels a bot can see | a server's channels, direct messages to the bot |
31
+ | `ftp` | told | a file a device uploads | cameras, doorbells, scanners |
32
+ | `smtp` | told | a mail a device sends | NAS boxes, printers, UPS units, routers, cron jobs |
33
+ | `syslog` | told | the log lines of a device | routers, firewalls, switches, access points, NAS boxes |
34
+ | `onvif` | told | what a network camera detects | motion, a person, a vehicle, tampering, a doorbell button |
35
+ | `serial` | told | lines from a serial port | an Arduino, a smart meter's P1 port, a GPS receiver |
36
+ | `system` | told | what this computer does | sleep, wake, lock, power, network, disks, apps |
37
+ | `command` with `follow` | told | each line a command prints while it runs | logs, `docker events`, `kubectl -w`, any broker or database with a client |
38
+ | `http` | looks | a change of a page or of one value in it | a price, a version, a status, a device's state page, an API |
39
+ | `feed` | looks | new entries of an RSS, Atom or JSON feed | releases, blogs, videos, podcasts, papers, status pages |
40
+ | `calendar` | looks | entries of a calendar that are new, moved or gone | appointments, school and club calendars, waste collection |
41
+ | `command` | looks | a change in what a command prints or how it ends | disk space, a build, a branch, whether a machine answers |
42
+ | `modbus` | looks | a change of a value in a Modbus device | solar inverters, heat pumps, wall boxes, energy meters |
43
+ | `presence` | looks | a device joining or leaving your network | a phone coming home, a TV switched on, a printer online |
44
+
45
+ What should happen at a time is not a source but a schedule ([events.md](events.md),
46
+ "Schedules"). A `calendar` source reports that an entry was added or moved; "ten minutes
47
+ before it starts" is a schedule the agent sets when it is told of the entry.
48
+
49
+ ## Which type for which service
50
+
51
+ | Service or device | Type | How |
52
+ |---|---|---|
53
+ | GitHub: releases, commits, tags of a repository | `feed` | `https://github.com/<owner>/<repo>/releases.atom` (or `commits.atom`, `tags.atom`). No account needed for a public repository. |
54
+ | GitHub: pushes, issues, pull requests as they happen | `webhook` | A webhook of the repository, with the secret. On a machine without a public address: a relay (smee.io) and a `stream` source. |
55
+ | YouTube channel, podcast, blog, arXiv, Reddit, Hacker News | `feed` | The feed's address. |
56
+ | A shop's price, a version number, a status page without a feed | `http` | `match`, `select` or `path` for the one value that counts. |
57
+ | A page that only shows its content after its scripts ran | `http` | `render`: pine's browser opens the page at every look. |
58
+ | An API that is asked with POST (GraphQL) | `http` | `method: "POST"` and `body`. |
59
+ | Home Assistant | `stream`, `mqtt` or `webhook` | Its WebSocket API (`stream` with `hello`), its MQTT broker, or an automation that calls pine's webhook. |
60
+ | Zigbee and Z-Wave devices | `mqtt` | Through zigbee2mqtt or zwave-js-ui. |
61
+ | ntfy, Gotify | `stream` | The topic's `/sse` address; Gotify's `/stream`. |
62
+ | Signal | `stream` | signal-cli's HTTP daemon on this machine, which sends events over SSE. |
63
+ | Mastodon, Bluesky, Nostr | `stream` | The streaming API, Jetstream, a relay. A Mastodon account's posts alone: `feed` (`https://<server>/@<name>.rss`). |
64
+ | Slack | `slack` | An app in Socket Mode (through Slack's own SDK). Also the way to what other tools post into Slack: with `bots`, the messages of GitHub, PagerDuty or an alert bot are reported. |
65
+ | Discord | `discord` | A bot. |
66
+ | Matrix (Element) | `matrix` | An access token of an account. |
67
+ | Telegram | `telegram` | A bot, paired with your account. |
68
+ | Gmail, iCloud, Fastmail, your own mail server | `mail` | IMAP with an app password. |
69
+ | Outlook.com, Hotmail, Microsoft 365 | `mail` | `"signIn": "microsoft"`: you sign in at Microsoft in your browser, pine holds no password. |
70
+ | Google Calendar, iCloud, Outlook, Nextcloud | `calendar` | The calendar's iCal address. |
71
+ | Synology, QNAP, TrueNAS, Proxmox, UniFi, a UPS, a printer | `smtp` | The device's "send notifications by e-mail" setting, pointed at pine. |
72
+ | A router, firewall, switch or NAS that can send its log | `syslog` | The device's "remote log server" setting, pointed at pine. |
73
+ | A camera or doorbell that uploads pictures | `ftp` | The camera's "upload to FTP server" setting. |
74
+ | A camera's motion, person and tamper detection | `onvif` | The camera's ONVIF user. |
75
+ | A solar inverter, heat pump, wall box, energy meter | `modbus` | The device's Modbus TCP port and its register list. |
76
+ | A 3D printer | `mqtt` | Bambu printers are their own broker; OctoPrint and Klipper publish through an MQTT plug-in. |
77
+ | An Arduino, an ESP32 on USB, a smart meter's P1 cable, a GPS | `serial` | The device under `/dev`. |
78
+ | "Is my phone home", a TV switched on | `presence` | The device's fixed address on your network. |
79
+ | This Mac: lock, sleep, battery, network, disks | `system` | Nothing to set up. |
80
+ | A log file, Docker, Kubernetes, journald | `command` with `follow` | `tail -F`, `docker events`, `kubectl get -w`, `journalctl -f`. |
81
+ | PostgreSQL, Redis, NATS, Kafka, RabbitMQ | `command` with `follow` | The system's command line client, printing one line per message. |
82
+ | Anything with a command that prints its state | `command` | Run again and again; a change is the event. |
83
+
84
+ ## How every type is used
85
+
86
+ ### Ask the agent
87
+
88
+ Say what you want watched. The agent picks the type, pine shows you what the source will do,
89
+ and you confirm:
90
+
91
+ > Tell me on Telegram when pmndre/pine has a new release.
92
+
93
+ The agent calls `add_source` with `type: "feed"`, the address, a description and a note for
94
+ itself. The dialog says "read the feed at https://github.com/pmndre/pine/releases.atom
95
+ every 15m and report each new entry". A password, token or key is never typed into the conversation: the
96
+ agent leaves it out, and pine asks you for it in a dialog of its own and keeps it in `.env`
97
+ ([sources.md](sources.md), "Secrets").
98
+
99
+ An agent is told the everyday types with their options. The other types it knows by name and
100
+ purpose, and what one takes it looks up in this manual (a search for "source type" and the
101
+ name finds a part made from the type's own description), or learns from the answer to a call
102
+ that leaves a needed option out.
103
+
104
+ ### In the configuration
105
+
106
+ A source is an entry under `sources` in `pine.json` (every agent can subscribe) or in an
107
+ agent's `agent.json` (only that agent can). The options of the type lie next to `type`:
108
+
109
+ ```json
110
+ "sources": {
111
+ "releases": { "type": "feed", "url": "https://github.com/pmndre/pine/releases.atom", "every": "30m", "description": "new releases of pine" }
112
+ }
113
+ ```
114
+
115
+ The keys every source has, whatever its type (`description`, `agents`, `trust`, `start`,
116
+ `enabled`), are in [sources.md](sources.md), "Configuring sources". Then an agent subscribes:
117
+ in a conversation ("watch the releases"), or with `pine subscribe main releases`.
118
+
119
+ ### What an event looks like
120
+
121
+ Every report is an event on pine's bus:
122
+
123
+ | Field | |
124
+ |---|---|
125
+ | `topic` | `source/<name>/<event>`, or `source/<name>@<agent>/<event>` for an agent's own source. `<event>` is the word in the tables below: `entry`, `message`, `changed`. |
126
+ | `payload` | The fields in the tables below. |
127
+ | `trust` | `"untrusted"` unless the source is marked as yours. The agent reads the event as data, not as instructions. |
128
+ | `ts`, `id` | When it was reported, and its number in the event log. |
129
+
130
+ `pine events --topic 'source/#'` shows them as they come, and `pine events --topic
131
+ 'source/releases/#'` those of one source. The agent receives the event as a message with the
132
+ note of its subscription in front ([sources.md](sources.md), "What the agent is woken with").
133
+
134
+ ### Being woken only for what counts
135
+
136
+ A subscription can carry a filter, `when`, so that the agent's turn is spent only on the
137
+ events that matter. It names fields of the event and what each has to be:
138
+
139
+ ```json
140
+ { "source": "price", "when": { "payload.number": { "lt": 500 } }, "note": "Tell me: the fridge is under 500." }
141
+ ```
142
+
143
+ | In `when` | Holds when the field |
144
+ |---|---|
145
+ | `"payload.kind": "dm"` | equals the value |
146
+ | `{ "in": ["err", "crit"] }` | is one of these |
147
+ | `{ "gt": 90 }`, `{ "gte": 90 }`, `{ "lt": 5 }`, `{ "lte": 5 }` | is a number above, at least, below, at most |
148
+ | `{ "contains": "backup" }` | is a text or a list that contains it |
149
+ | `{ "regex": "\\.pdf$" }` | matches the regular expression |
150
+ | `{ "exists": true }` | is there at all |
151
+ | `{ "ne": "ok" }` | is anything but the value |
152
+
153
+ All fields named have to hold. The agent sets `when` with its `subscribe` tool; in a file it
154
+ is a key of the subscription. Many types also narrow at the source (`match`, `keywords`,
155
+ `topics`, `severity`): what is filtered there is not even an event. `throttle` and `debounce`
156
+ of a subscription keep a source that reports in bursts from waking the agent for each one,
157
+ and a subscription with `"session": "new"` gives every event a conversation of its own
158
+ ([sources.md](sources.md), "A fresh conversation for every event").
159
+
160
+ ### What all types have in common
161
+
162
+ - **Nothing runs until somebody is subscribed.** A configured source opens no port and no
163
+ connection. The first subscription starts it, the last one to end stops it.
164
+ - **The start is the starting point.** What is there when a source starts (the entries of a
165
+ feed, the mail in a mailbox, the state of a device) is not reported. Only what comes after is.
166
+ - **What is reported is bounded.** A text is cut to a few thousand characters, and most types
167
+ that are told things report at most so many events a minute (`perMinute` where it can be
168
+ set); what was left out is counted in the next event's `dropped`.
169
+ - **An address is the one you confirmed.** A public address has to stay on the public
170
+ internet: if its name one day leads into your network, the source fails and says so. A local
171
+ address is said to be local in the dialog. A redirect to another host is not followed, and a
172
+ token goes only to the host it was given for.
173
+ - **A source that also sends, sends only where you listed.** `sendTo` of `mail`, `slack`,
174
+ `matrix` and `discord`, `publish` of `mqtt`. Without the list nothing can be sent.
175
+ - **How it is doing is said where sources are listed** (`pine sources`, `/sources`, the
176
+ agent's `sources` tool): connected since when, last looked when, failing since when and why.
177
+
178
+ ## The types
179
+
180
+ ### feed
181
+
182
+ A feed that is read again and again: each entry that was not there before is one event. For
183
+ everything that publishes itself as RSS or Atom, which is more than it looks like: releases,
184
+ blogs, video channels, podcasts, searches, status pages.
185
+
186
+ > Tell me when a new version of Node comes out.
187
+
188
+ ```json
189
+ "node": { "type": "feed", "url": "https://github.com/nodejs/node/releases.atom", "every": "1h", "description": "releases of Node.js" }
190
+ ```
191
+
192
+ | Option | Default | |
193
+ |---|---|---|
194
+ | `url` | | The address of the feed itself, `https://...`. Not the web page that links to it. |
195
+ | `every` | `"15m"` | How often to read it. At least `"1m"` when an agent sets it up. |
196
+ | `match` | | Only entries whose title or text this regular expression finds: `"security\|CVE"`, `"^v[0-9]+\\.0"`. The others are passed over without a word. |
197
+ | `headers` | | Headers to send. Not for secrets. |
198
+ | `token` | | An access token or key, sent as `Authorization: Bearer <token>`. Kept in `.env`. |
199
+ | `tokenHeader` | | Send the token as this header instead, as it is: `"x-api-key"`. |
200
+ | `local` | `false` | The address is on your own network under a name that does not say so. Not needed for an IP address, `localhost` or a `.local` name. |
201
+
202
+ | Event | | |
203
+ |---|---|---|
204
+ | `entry` | `{ feed, title, link, id, published?, author?, summary, media?, more? }` | One for each new entry. `feed` is the feed's title, `summary` the entry's text as a reader sees it (up to 1500 characters), `published` a time in UTC, `media` the sound or video file of a podcast episode. |
205
+
206
+ The entries that are in the feed when the source is set up are the starting point and are not
207
+ reported. After that every look reports what is new, oldest first, ten entries at most: when
208
+ there were more (a feed that was moved, a week without power), the newest ten are reported and
209
+ the last one carries `more`, the number that was left out. An entry is known by its id (`guid`
210
+ in RSS, `id` in Atom), else by its link, else by its title and date. One that leaves the feed
211
+ and comes back is not new: the last 500 that left are remembered.
212
+
213
+ It reads RSS 2.0, RSS 1.0, Atom and JSON Feed. A feed that can say "nothing new since you last
214
+ asked" is asked that way, so a look at a quiet feed costs it one line.
215
+
216
+ | What it is for | The feed's address |
217
+ |---|---|
218
+ | Releases of a project on GitHub | `https://github.com/<owner>/<repo>/releases.atom` |
219
+ | Commits on a branch on GitHub | `https://github.com/<owner>/<repo>/commits/<branch>.atom` |
220
+ | Tags on GitHub | `https://github.com/<owner>/<repo>/tags.atom` |
221
+ | A YouTube channel | `https://www.youtube.com/feeds/videos.xml?channel_id=<id>` (the id starts with `UC`; a playlist: `?playlist_id=<id>`) |
222
+ | A blog | Usually `/feed`, `/rss.xml`, `/atom.xml` or `/index.xml` on the blog's address. The page names it in a `<link rel="alternate">` tag. |
223
+ | A podcast | The address a podcast directory shows as "RSS feed". The episode's file is in `media`. |
224
+ | arXiv | `https://export.arxiv.org/rss/<category>`, e.g. `cs.AI`. It has entries on working days only. |
225
+ | A subreddit, a Reddit search | `https://www.reddit.com/r/<name>/.rss`, `https://www.reddit.com/search.rss?q=<words>` |
226
+ | Hacker News | `https://hnrss.org/frontpage`, `https://hnrss.org/newest?q=<words>` (stories that mention the words), `https://hnrss.org/newest?points=300` |
227
+ | A status page | Most have `/history.atom` or `/history.rss` (GitHub: `https://www.githubstatus.com/history.atom`). |
228
+ | A Mastodon account | `https://<instance>/@<name>.rss` |
229
+ | Weather warnings, earthquakes, court decisions, job boards, classifieds | Whatever the site offers as RSS or Atom. Many searches have a feed of their own results. |
230
+
231
+ With a pattern the agent is woken only for what your owner cares about, without a model
232
+ reading the rest:
233
+
234
+ ```json
235
+ "cves": { "type": "feed", "url": "https://hnrss.org/newest?points=100", "match": "(?i:postgres|sqlite)", "description": "stories about databases on Hacker News" }
236
+ ```
237
+
238
+ What it says where sources are listed is a count (`now: 25 entries`), never a title: that line
239
+ is read by agents without the mark that outside content gets.
240
+
241
+ Refused when it is set up: an address that is not `http(s)`, a user name or password in the
242
+ address, a header that carries a secret (it belongs into `token`), a `match` that is not a
243
+ regular expression. A failed look and not an event: an address that answers with a web page
244
+ instead of a feed (the page is not searched for its feed; set the source up with the feed's own
245
+ address, or watch the page with an `http` source), and a redirect to another host. An answer is
246
+ read up to 1 MB, which for a long feed is its newest entries. Like every source that looks, it
247
+ follows the rules under "Looking again and again".
248
+
249
+ ### calendar
250
+
251
+ A calendar that is read again and again: reports the entries that were added, changed or
252
+ removed. For any calendar that has an address of its own in iCalendar format (`.ics`), which
253
+ nearly every calendar service can give you: a shared family calendar, a team's, a school's, a
254
+ club's fixtures, the waste collection.
255
+
256
+ > Keep an eye on the family calendar and tell me when somebody adds or moves something this month.
257
+
258
+ ```json
259
+ "family": { "type": "calendar", "url": "https://calendar.google.com/calendar/ical/.../basic.ics", "ahead": "30d", "description": "the family calendar" }
260
+ ```
261
+
262
+ | Option | Default | |
263
+ |---|---|---|
264
+ | `url` | | The calendar's address in iCalendar format, `https://...` or `webcal://...` (which is the same address, read over https). |
265
+ | `every` | `"15m"` | How often to read it. At least `"1m"` when an agent sets it up. |
266
+ | `ahead` | `"60d"` | Only entries that begin within this from now. What is added further away is not reported, also not later when it comes nearer. At least `"1h"`. |
267
+ | `headers` | | Headers to send. Not for secrets. |
268
+ | `token` | | An access token or key, sent as `Authorization: Bearer <token>`. Kept in `.env`. |
269
+ | `tokenHeader` | | Send the token as this header instead, as it is. |
270
+ | `local` | `false` | The address is on your own network under a name that does not say so (a Nextcloud at `cloud.home`). |
271
+
272
+ | Event | | |
273
+ |---|---|---|
274
+ | `added` | `{ calendar, uid, summary, start, end?, allDay?, zone?, location?, description?, repeats? }` | An entry that was not in the calendar at the last look. |
275
+ | `changed` | the same, with `was: { summary, start, end?, location? }` | Its title, its time or its place is different. A new description alone is not reported. |
276
+ | `removed` | `{ calendar, uid, summary, start }` | An entry that was still ahead is gone, or was cancelled. |
277
+
278
+ How a time is written says what kind it is:
279
+
280
+ | `start` | Means |
281
+ |---|---|
282
+ | `2026-12-24T17:00:00Z` | A moment in UTC. |
283
+ | `2026-12-24T18:00:00` with `zone: "Europe/Berlin"` | That time on the clocks of that zone. |
284
+ | `2026-12-24T18:00:00` without `zone` | That time wherever the reader is ("floating"). |
285
+ | `2026-12-24` with `allDay: true` | The whole day. `end` is then the last day, and is left out for a single day. |
286
+
287
+ **It reports changes to the calendar and never fires on a clock.** That an entry begins in ten
288
+ minutes is not an event of this source. What should happen at a time is a schedule: an agent
289
+ that is to remind you before an appointment adds one when the entry is reported (`add_schedule`
290
+ with `at`), and removes it when the entry is reported `removed`. In the same way an entry that
291
+ has simply passed is not `removed`: it leaves without a word.
292
+
293
+ A repeating entry is one entry. It is not unfolded into its dates: `repeats` carries its rule
294
+ as the calendar writes it (`FREQ=WEEKLY;BYDAY=MO`), and it counts as inside `ahead` for as long
295
+ as it runs. One date of it that was moved is an entry of its own with the same `uid`.
296
+
297
+ The entries that are in the calendar when the source is set up are the starting point. One look
298
+ reports twenty changes at most, earliest first; the last one carries `more` when there were
299
+ more (a calendar that was emptied and filled again).
300
+
301
+ | What it is for | Where its address is |
302
+ |---|---|
303
+ | Google Calendar | Settings of the calendar, "Integrate calendar", "Secret address in iCal format". Google refreshes that file only every few hours. |
304
+ | iCloud | Share the calendar as a "Public Calendar" and copy the link (`webcal://...`). |
305
+ | Outlook, Microsoft 365 | Settings, "Shared calendars", "Publish a calendar", the ICS link. |
306
+ | Nextcloud | The calendar's menu, "Copy subscription link" (add `?export` to a share link). |
307
+ | Fastmail, Proton, Zoho and others | The calendar's sharing settings, as "iCal", "ICS" or "subscription" link. |
308
+ | Schools, clubs, leagues, public holidays, waste collection, cinema and concert programmes | A "subscribe" or "add to calendar" link on their site. |
309
+
310
+ A secret address like Google's is a key in itself, and pine does not treat it as one: it is
311
+ stored and shown like any address. Give such a link only where you would give the calendar.
312
+
313
+ Refused when it is set up: an address that is not `https://`, `http://` or `webcal://`, a user
314
+ name or password in the address, a header that carries a secret, `ahead` under an hour. A
315
+ failed look and not an event: an answer that is not a calendar (a web page with a "subscribe"
316
+ button is not one), a redirect to another host, and a calendar of more than 8 MB, which is not
317
+ read in part because half a calendar would make the other half look removed. A zone under a
318
+ name only the program that wrote the calendar knows (`W. Europe Standard Time`) is read as this
319
+ computer's own zone. Like every source that looks, it follows the rules under "Looking again
320
+ and again".
321
+
322
+ ### stream
323
+
324
+ A service that pushes: pine connects to it once, stays connected, and the service says by
325
+ itself when something happens. Two ways of doing that are understood: Server-Sent Events (one
326
+ long answer to a web request, `https://`) and a WebSocket (`wss://`). For what would otherwise
327
+ have to be asked again and again, and for the many services whose only way out is such a
328
+ stream.
329
+
330
+ > Tell me when somebody opens the front door. Home Assistant knows.
331
+
332
+ ```json
333
+ "door": {
334
+ "type": "stream", "url": "ws://homeassistant.local:8123/api/websocket", "token": "${HA_TOKEN}",
335
+ "hello": [
336
+ { "type": "auth", "access_token": "{{token}}" },
337
+ { "id": 1, "type": "subscribe_trigger", "trigger": { "platform": "state", "entity_id": "binary_sensor.front_door" } }
338
+ ],
339
+ "path": "event.variables.trigger.to_state.state", "changed": true,
340
+ "description": "whether the front door is open"
341
+ }
342
+ ```
343
+
344
+ | Option | Default | |
345
+ |---|---|---|
346
+ | `url` | | The address of the stream: `https://...` for Server-Sent Events, `wss://...` for a WebSocket. `http://` and `ws://` only for an address on your own network or this computer. |
347
+ | `kind` | by the address | `"sse"` or `"websocket"`. Only needed for a WebSocket that is given as an `https://` address. |
348
+ | `hello` | | For a WebSocket: what is sent right after connecting, to log in or to say what is wanted. One message or a list; each is a text or a JSON object. `{{token}}` in it is replaced by the token, so the token itself is never in the configuration. |
349
+ | `token` | | An access token or key, sent as `Authorization: Bearer <token>` when connecting. Kept in `.env`. When `hello` says `{{token}}`, it goes there and is not sent as a header as well. |
350
+ | `tokenHeader` | | Send the token as this header instead, as it is: `"x-gotify-key"`, `"x-api-key"`. |
351
+ | `headers` | | Headers to send when connecting. Not for secrets. |
352
+ | `event` | | For Server-Sent Events: only events of this name (`"notification"`). Without it every event counts. |
353
+ | `path` | | For messages in JSON: take only this field of each, `"event.data.new_state.state"` or `"data.0.last"`. A message that does not have it is left out. |
354
+ | `match` | | Only messages this regular expression finds something in (in the field, if `path` is given). If it has a group, what the group finds is the value; without a group it only says which messages count. |
355
+ | `changed` | `false` | Report a value only when it differs from the last one. The first value is the starting point and is not reported. For a stream that repeats a state. |
356
+ | `perMinute` | `30` | At most this many reports a minute, 1 to 600. What comes on top is dropped and counted. |
357
+ | `local` | `false` | The address is on your own network under a name that does not say so (`ha.lan`). Not needed for an IP address, `localhost` or a `.local` name. |
358
+
359
+ | Event | | |
360
+ |---|---|---|
361
+ | `message` | `{ value, event?, id?, previous?, dropped? }` | One message of the service. `value` is the parsed JSON, or the text, or what `path` and `match` picked, cut to 4000 characters. `event` and `id` are what Server-Sent Events call them; `previous` comes with `changed`; `dropped` says how many messages the cap left out before this one. |
362
+ | `down` | `{ url, error }` | After three attempts in a row that did not connect, once. |
363
+ | `up` | `{ url }` | When it is connected again. |
364
+
365
+ It reports what the service sends from the moment it is connected: nothing that was sent
366
+ before, and nothing that was sent while it was not connected, unless the service itself makes
367
+ up for that (Server-Sent Events can: pine says which event was the last it got, as
368
+ `Last-Event-ID`, when it connects again).
369
+
370
+ **Staying connected.** A connection that ends is made again: after a second, then after two,
371
+ four and so on up to five minutes, and from a second again once a connection has lasted. A
372
+ service on this computer is asked again sooner. A hang-up that is mended this way is not an
373
+ event; only three failed attempts in a row are (`down`). Where the sources are listed the line
374
+ says `connected since 14:02:11`, or `cannot connect since 14:02:11 (4 attempts): it answered
375
+ 503`. A WebSocket is pinged every 45 seconds so that a line that died without a word is
376
+ noticed; an event stream that says nothing for five minutes is connected to again.
377
+
378
+ **Not too much.** A stream can say a great deal. `path`, `match`, `event` and `changed` decide
379
+ what counts; `perMinute` is the limit for what is left. Neither a filter nor the cap costs a
380
+ model turn: only a reported message wakes the agent, and the subscription's own `throttle`
381
+ applies on top.
382
+
383
+ An address that is not on your own network has to be `https://` or `wss://`, and has to stay
384
+ on the public internet: a name that leads into your network is refused, with a sentence that
385
+ says so. A redirect is followed only on the same host, and the token is sent to that host
386
+ only. Refused when it is set up: a user name or password in the address, a header that
387
+ carries a secret (`authorization`, `cookie`; it belongs into `token`), `hello` for
388
+ Server-Sent Events, `event` for a WebSocket. One message may be 256 KB long; longer ones are
389
+ left out.
390
+
391
+ #### What a stream reaches
392
+
393
+ Each line is an entry for `"sources"`; an agent sets the same up with `add_source` (type
394
+ `stream`), and is asked for nothing but the token.
395
+
396
+ | Service | Entry | What arrives |
397
+ |---|---|---|
398
+ | ntfy (a topic on ntfy.sh or your own server) | `{ "type": "stream", "url": "https://ntfy.sh/my-topic/sse", "event": "message", "path": "message" }` | The text of every notification on the topic. Without `path`: title, priority, tags as well. A protected topic takes a `token`. |
399
+ | Gotify | `{ "type": "stream", "url": "wss://gotify.example.com/stream", "token": "${GOTIFY_CLIENT_TOKEN}", "tokenHeader": "x-gotify-key" }` | Every message of the server: `{ title, message, priority, appid }`. |
400
+ | Home Assistant | the example above | The new state of one entity. With `{ "id": 1, "type": "subscribe_events", "event_type": "state_changed" }` in `hello` instead: every state change of the house, which wants a `match` and a low `perMinute`. The token is a long-lived access token from your profile page. |
401
+ | signal-cli (Signal messages on this machine) | `{ "type": "stream", "url": "http://127.0.0.1:8080/api/v1/events", "event": "receive", "path": "envelope" }` | Each Signal message the account gets, for `signal-cli daemon --http 127.0.0.1:8080`. |
402
+ | Mastodon | `{ "type": "stream", "url": "https://mastodon.social/api/v1/streaming/user", "event": "notification", "token": "${MASTODON_TOKEN}" }` | Mentions, boosts, follows. Use the streaming address your instance names: a redirect to another host is not followed. |
403
+ | Bluesky (Jetstream) | `{ "type": "stream", "url": "wss://jetstream2.us-east.bsky.network/subscribe?wantedCollections=app.bsky.feed.post&wantedDids=did:plc:...", "path": "commit.record.text" }` | What one account posts. |
404
+ | A webhook relay (smee.io) | `{ "type": "stream", "url": "https://smee.io/your-channel", "path": "body" }` | Webhooks for a machine that has no public address: GitHub, Stripe and the like call the relay, and pine reads them from it. Whoever knows the channel's address can write to it, so treat it like any outside content. |
405
+ | A ticker of an exchange (Kraken) | `{ "type": "stream", "url": "wss://ws.kraken.com/v2", "hello": { "method": "subscribe", "params": { "channel": "ticker", "symbol": ["BTC/EUR"] } }, "path": "data.0.last", "changed": true, "perMinute": 2 }` | The last price, at most twice a minute. With `when` on the subscription: only below or above a number. |
406
+ | A Nostr relay | `{ "type": "stream", "url": "wss://relay.damus.io", "hello": [["REQ", "pine", { "kinds": [1], "authors": ["<public key in hex>"] }]], "path": "2.content" }` | The notes of one author. (`hello` is a list of messages, and a Nostr message is itself a list.) |
407
+ | Wikimedia's public event stream, to try it | `{ "type": "stream", "url": "https://stream.wikimedia.org/v2/stream/recentchange", "path": "title", "perMinute": 2 }` | The title of a page somebody just edited, twice a minute, with `dropped` saying how many edits were left out. |
408
+
409
+ What has no stream of its own but can call a web address is a `webhook` source; what can only
410
+ be asked is an `http` source; what has a command that prints as things happen (`tail -f`,
411
+ `docker events`) is a `command` source.
412
+
413
+ ### mail
414
+
415
+ A mailbox: reports every new mail the moment the mail server says there is one, with its text
416
+ and the files that came with it. It reads over IMAP and only reads: nothing is marked as read,
417
+ moved or deleted, so your mail program shows the same as before.
418
+
419
+ > Watch my mailbox and tell me on Telegram when the bank sends a statement. Put the PDF into
420
+ > ~/Documents/Bank.
421
+
422
+ ```json
423
+ "inbox": {
424
+ "type": "mail", "host": "imap.fastmail.com", "user": "me@example.com", "password": "${MAIL_APP_PASSWORD}",
425
+ "from": ["@bank.example"], "subject": "statement|kontoauszug",
426
+ "description": "statements from the bank"
427
+ }
428
+ ```
429
+
430
+ | Option | Default | |
431
+ |---|---|---|
432
+ | `host` | | The IMAP server, by name alone: `imap.gmail.com`. No `imap://`, no port, no path. Needed, except with `signIn`. |
433
+ | `user` | | The user name of the mailbox, usually the e-mail address. |
434
+ | `password` | | The password. Kept in `.env`. Most providers want an app password made for this, see the table below. An agent that sets a mailbox up leaves it out, and pine asks you for it. Not used with `signIn`. |
435
+ | `signIn` | | `"microsoft"`: no password. You sign in at Microsoft in your browser, once, with a code pine shows you. For Outlook.com, Hotmail, Live and Microsoft 365, which take no password over IMAP. See "A Microsoft mailbox" below. |
436
+ | `clientId` | | With `signIn`: the Application (client) ID of an app you registered at Microsoft. Not a secret. |
437
+ | `tenant` | `"common"` | With `signIn`: whose accounts the app signs in. `"common"`: any. `"consumers"`: private accounts. `"organizations"`: work and school accounts. Or the domain or id of one organisation. |
438
+ | `port` | `993` | `143` with `security` `"starttls"` or `"none"`. |
439
+ | `security` | `"tls"` | `"tls"`: encrypted from the start. `"starttls"`: encrypted after connecting. `"none"`: only for a server on this computer or your own network. |
440
+ | `mailbox` | `"INBOX"` | The folder to watch, as the server names it. |
441
+ | `from` | everyone | Only mail that says it is from one of these: an address (`"bank@example.com"`) or a whole domain (`"@example.com"`). |
442
+ | `subject` | | Only mail whose subject this regular expression finds. Capitals do not matter. |
443
+ | `maxMb` | `10` | Attachments are kept up to this size (50 at most). A larger one is named in the event, without a file. |
444
+ | `keepDays` | `7` | Kept attachments are deleted after this many days. |
445
+ | `smtpHost` | | For sending: the provider's SMTP server. Only together with `sendTo`. |
446
+ | `smtpPort` | `465` | `465` is encrypted from the start; any other port (`587`) uses STARTTLS. |
447
+ | `sendTo` | none | The addresses an agent may write to: `["me@example.com"]`. Without it the source only reads. |
448
+ | `sender` | `user` | The address mail is sent as. |
449
+ | `insecureTls` | `false` | Accept a certificate that cannot be verified, as a server of your own with a self-made one needs. |
450
+ | `local` | `false` | The server is on your own network or this computer under a name that does not say so (`mail.home`). Not needed for an IP address, `localhost` or a `.local` name. |
451
+
452
+ | Event | | |
453
+ |---|---|---|
454
+ | `mail` | `{ from, fromName?, to, subject, date?, text, attachments, images?, cut? }` | One for every new mail that passes `from` and `subject`. `text` is the plain text, or for a mail that is only HTML what a reader sees of it, up to 6000 characters. `attachments` is a list of `{ file, path?, type, size }`: `file` is the name the mail gave it, `path` where pine put it. A file without a `path` was larger than `maxMb`. `cut` says that the mail was larger than what is fetched of one. |
455
+
456
+ Pictures among the attachments (jpg, png, webp, gif under 10 MB, four at most) are shown to
457
+ the agent if its model can look at images. The files lie in the source's own folder under
458
+ names pine makes, and the agents subscribed to the source can read them there (see "Files a
459
+ source received"); to keep one, the agent copies it into its workspace.
460
+
461
+ **The mail that is in the mailbox when the source first connects is the starting point** and is
462
+ not reported. From then on every new one is, also what arrived while pine was not running or
463
+ the connection was down: the source remembers the last mail it reported, not a time. If more
464
+ than 200 mails arrived meanwhile, the oldest of them are left out.
465
+
466
+ **Who a mail says it is from can be forged.** The sender line is what the sender wrote, and
467
+ anyone can write any address there. So a mail is always outside content, whoever it claims to
468
+ come from: it wakes the agent the way a web page it read would, and after it the agent asks
469
+ before it sends anything out. `from` is a filter that keeps noise away, not a proof, and there
470
+ is no way to make a mailbox speak for you (`trust: "owner"` on a mailbox would hand your agent
471
+ to whoever knows your address; pine does not stop you writing it into the file, and
472
+ `pine check` names it under its security notes).
473
+
474
+ | Provider | `host` | `smtpHost` | The password |
475
+ |---|---|---|---|
476
+ | Gmail, Google Workspace | `imap.gmail.com` | `smtp.gmail.com` | An app password (Google account, Security, 2-Step Verification, App passwords). The account's own is refused. |
477
+ | iCloud Mail | `imap.mail.me.com` | `smtp.mail.me.com`, `smtpPort` 587 | An app-specific password from account.apple.com. |
478
+ | Fastmail | `imap.fastmail.com` | `smtp.fastmail.com` | An app password (Settings, Privacy & Security, Integrations) with IMAP, and SMTP if it sends. |
479
+ | GMX, WEB.DE, mailbox.org, Posteo | `imap.gmx.net`, `imap.web.de`, `imap.mailbox.org`, `posteo.de` | `mail.gmx.net`, `smtp.web.de`, `smtp.mailbox.org`, `posteo.de` | The account's password; at GMX and WEB.DE, IMAP has to be switched on in the settings first. |
480
+ | A server of your own (Dovecot, Mailcow, Synology MailPlus) | its name | its name | The mailbox's password. On your own network: `"local": true` if the name does not say so, `insecureTls` for a self-made certificate. |
481
+ | Proton Mail | `127.0.0.1`, `port` 1143, `security` `"starttls"`, `insecureTls` | `127.0.0.1`, `smtpPort` 1025 | Through Proton's Bridge running on this machine, with the password the Bridge shows. |
482
+ | Outlook.com, Hotmail, Live, Microsoft 365 | filled in: `outlook.office365.com` | filled in: `smtp.office365.com`, port 587 | None: Microsoft takes no password over IMAP any more. `"signIn": "microsoft"` and a `clientId`, see "A Microsoft mailbox" below. |
483
+
484
+ #### A Microsoft mailbox: signing in instead of a password
485
+
486
+ Outlook.com, Hotmail, Live and Microsoft 365 mailboxes take no password over IMAP. For these
487
+ you sign in at Microsoft yourself, and Microsoft gives pine a sign-in for that one mailbox:
488
+
489
+ > Watch my Outlook mailbox me@outlook.com and tell me when the school writes.
490
+
491
+ ```json
492
+ "outlook": {
493
+ "type": "mail", "user": "me@outlook.com", "signIn": "microsoft",
494
+ "clientId": "11111111-2222-3333-4444-555555555555", "tenant": "consumers",
495
+ "from": ["@school.example"], "description": "mail from the school"
496
+ }
497
+ ```
498
+
499
+ What happens when the source starts for the first time:
500
+
501
+ 1. pine asks Microsoft for a code and shows you a dialog: "Open
502
+ https://microsoft.com/devicelogin in your browser, enter the code ABCD-EFGH, and sign in as
503
+ me@outlook.com". The same is in the source's status line (`pine sources`), so you find the
504
+ code again if the dialog is gone.
505
+ 2. You open that page on any device, enter the code and sign in the way you always do, with
506
+ your password and second factor. Microsoft shows what the app asks for: reading the mailbox
507
+ over IMAP, and sending if the source has `sendTo`.
508
+ 3. pine notices by itself that you have signed in, closes the dialog and connects. You have
509
+ about fifteen minutes; "no" in the dialog gives up.
510
+
511
+ pine never sees your password. What Microsoft hands over lets pine open that mailbox, is
512
+ renewed by pine whenever it connects, without asking you, and is kept in pine's own database
513
+ in its home folder, where no agent reads. You are asked again only when Microsoft no longer
514
+ takes it: after you signed the app out, changed your password, or the source was not used for
515
+ about ninety days. You take the access away at Microsoft (account.microsoft.com, Privacy,
516
+ "Apps and services you've given access to"; for a work account myapps.microsoft.com) or by
517
+ removing the source, which deletes what pine kept.
518
+
519
+ `clientId` names an app that you register at Microsoft, once, for free. pine does not ship
520
+ one of its own: an app id says who is asking for your mail, and that should be you.
521
+
522
+ | Step | Where |
523
+ |---|---|
524
+ | Register an app | entra.microsoft.com, "App registrations", "New registration". Any name ("pine mail"). Supported account types: the choice that includes personal Microsoft accounts if the mailbox is a private one. No redirect address. |
525
+ | Copy its id | The app's "Overview" page: "Application (client) ID". That is `clientId`. |
526
+ | Allow signing in with a code | "Authentication", "Allow public client flows": Yes. Without it Microsoft refuses to give a code. |
527
+ | Allow reading mail | "API permissions", "Add a permission", "APIs my organization uses", "Office 365 Exchange Online", "Delegated permissions": `IMAP.AccessAsUser.All`, and `SMTP.Send` if the source sends. For a private account the question at sign-in is enough; in an organisation an administrator may have to agree. |
528
+
529
+ `tenant` has to fit how the app was registered: `"consumers"` for an app that is only for
530
+ private accounts, the organisation's domain for one that is only for that organisation,
531
+ `"common"` (the default) for one that takes both.
532
+
533
+ In an organisation two settings of the mailbox can stand in the way, and the status line says
534
+ which: IMAP switched off for the mailbox, and for sending "Authenticated SMTP" switched off
535
+ (Microsoft 365 admin center, the user, "Mail", "Manage email apps"). An organisation that
536
+ forbids apps its users registered themselves cannot be reached this way at all.
537
+
538
+ Refused when it is set up: `signIn` other than `"microsoft"`, `signIn` without a `clientId` or
539
+ together with a `password`, `clientId` or `tenant` without `signIn`. A wrong `clientId` or
540
+ `tenant` is found out when the source starts, and is why it is not running.
541
+
542
+ This follows Microsoft's device code flow and the XOAUTH2 login of IMAP and SMTP, and is
543
+ tested against stand-ins for all three. The steps in the table are from Microsoft's
544
+ documentation; the names of the pages there change now and then.
545
+
546
+ Gmail has no sign-in of this kind for mail (its device sign-in does not cover the mailbox);
547
+ an app password works there, also for a Google Workspace account unless its administrator
548
+ turned app passwords off.
549
+
550
+ **Sending.** With `smtpHost` and `sendTo` an agent can write an e-mail through the source:
551
+ `{"to": "me@example.com", "subject": "...", "text": "..."}` with its `send` tool. It reaches
552
+ the addresses on `sendTo` and nobody else; the type checks that itself, whatever a call names.
553
+ With one address on the list, `to` can be left out. Mail can reach anybody, so unlike a
554
+ Telegram message to you it counts as information leaving: after outside content the agent asks
555
+ before it sends (see "Sending through a source"). The text goes as plain text; no files.
556
+
557
+ Refused when it is set up: a `host` with a scheme, port or path in it, `"security": "none"` for
558
+ a server that is not local, a `from` entry that is neither an address nor `@domain`, `sendTo`
559
+ without `smtpHost` and the other way round, `maxMb` above 50. A server on the public internet
560
+ has to stay there: if its name one day leads into your network the connection fails and says
561
+ so. A password is never sent unencrypted to a server that is not local, when reading and when
562
+ sending.
563
+
564
+ A refused login (a wrong password, a mailbox that does not exist) is why the source is not
565
+ running, shown where sources are listed; fix it and subscribe again or restart. A server that
566
+ cannot be reached right now is tried again, after one second at first and up to five minutes
567
+ between tries, and the status line says since when. It waits with IMAP IDLE, renewed every 25
568
+ minutes; a server without IDLE is asked once a minute. Folder names with other than plain
569
+ letters are written as the server spells them (`"INBOX.Rechnungen"`, `"[Gmail]/All Mail"`).
570
+
571
+ ### slack
572
+
573
+ A Slack workspace: reports what is written in channels and in direct messages, all of it or
574
+ only what mentions somebody, contains a word, or comes from certain people, and if you ask
575
+ for it also reactions, edits, who joins and leaves, changes to channels, pins, the slash
576
+ commands of your app, and what other apps post (alerts, GitHub). An agent can answer through
577
+ it: post to the channels and people you list, put an emoji on a message, send a file. It uses
578
+ Slack's Socket Mode: your Slack app connects out to Slack and is told what happens where it
579
+ may look. Nothing listens on a port, so it works behind any router. The connection and every
580
+ call are made with Slack's own SDK.
581
+
582
+ > Watch the #ops channel in Slack. Tell me on Telegram when somebody mentions me or writes
583
+ > "outage", when an alert comes in there, and always when Rainer writes anything, anywhere.
584
+ > Put :eyes: on what you have passed on.
585
+
586
+ ```json
587
+ "work": {
588
+ "type": "slack", "appToken": "${SLACK_APP_TOKEN}", "token": "${SLACK_TOKEN}",
589
+ "channels": ["ops"], "mentions": ["me"], "keywords": ["outage"], "bots": true,
590
+ "from": ["U0456RAINER"], "sendTo": ["C0123OPS"],
591
+ "description": "what is written in Slack that concerns me"
592
+ }
593
+ ```
594
+
595
+ The four things most people want, each with the one setting it takes. They need a **user
596
+ token** (see "Making the Slack app"), because they are about what is written to you:
597
+
598
+ | You want to know when | Set | The event says |
599
+ |---|---|---|
600
+ | somebody writes, anywhere you are | nothing: this is what it does without filters | `why: ["channel"]`, or `["dm"]` |
601
+ | somebody writes to you personally | nothing (`dms` is on) | `kind: "dm"`, `why: ["dm"]` |
602
+ | somebody tags you | `"mentions": ["me"]` | `why: ["mention"]`, `mention`: your id |
603
+ | somebody writes about a topic | `"keywords": ["invoice", "outage"]` | `why: ["keyword"]`, `keyword`: the word |
604
+
605
+ With `mentions` or `keywords` set, a message in a channel counts only when one of them is
606
+ found; a direct message always counts. So `"mentions": ["me"], "keywords": ["outage"]` is
607
+ "what is written to me, what tags me, and what is about an outage", and nothing else. A
608
+ message that is several of these says so (`why: ["dm", "mention"]`), and a subscription can
609
+ be woken for some and not for others:
610
+
611
+ ```json
612
+ "subscribe": [{ "source": "work", "when": { "payload.why": { "contains": "mention" } }, "note": "Somebody tagged me: tell me on Telegram, with who and where." }]
613
+ ```
614
+
615
+ What you write yourself is never reported to you.
616
+
617
+ | Option | Default | |
618
+ |---|---|---|
619
+ | `appToken` | | The app-level token of your Slack app (`xapp-...`), with the scope `connections:write`. Socket Mode connects with it. Kept in `.env`. |
620
+ | `token` | | A bot token (`xoxb-...`) or a user token (`xoxp-...`) of the same app. With it people and channels have names in the events, a channel can be given by its name, `"me"` is known, the files of a message are fetched, and things can be sent. Without it events carry ids only and the source only listens. Kept in `.env`. An agent that sets a source up leaves it out, and pine asks you for it. |
621
+ | `events` | `["message"]` | Which kinds of thing are reported: `"message"`, `"reaction"`, `"edit"`, `"member"`, `"channel"`, `"pin"`, `"command"`, `"other"`. See "The kinds of event". A kind that is not listed is not reported. |
622
+ | `channels` | every channel the app sees | Only these channels, by name (`"ops"`, `"#ops"`) or by id (`"C0123ABCD"`; a private channel's starts with `G` or `C`). A name is looked up when the source starts and needs `token`. |
623
+ | `mentions` | | Only messages that mention one of these: user ids, `"me"`, `"here"`, `"channel"`, `"everyone"` (somebody wrote @here, @channel, @everyone), or the id of a user group (`S...`). |
624
+ | `keywords` | | Only messages that contain one of these words. Capitals do not matter. |
625
+ | `from` | | User ids whose messages always count, in any channel and whatever they say. An app's id (`B...`) is taken too. `"me"` here has your own messages reported, which they are not otherwise. |
626
+ | `dms` | `true` | Direct messages count, without having to mention anybody or contain a keyword. |
627
+ | `bots` | `false` | `true`: what other apps and integrations post counts too, as a message with `bot: true`. What your own app posts is never reported. |
628
+ | `ignore` | | User ids that are never reported: a colleague's bot account, a channel's chatterbox. An app's id (`B...`) is taken too. What the token's own account writes (with a user token: you) is left out without being listed. |
629
+ | `sendTo` | none | Where an agent may write, by id only: channels (`"C0123ABCD"`) and people (`"U0123ABCD"`, a direct message). Without it the source only listens. Needs `token`. See "Sending" below. |
630
+ | `perMinute` | `60` | At most this many reports a minute, 1 to 600. What comes on top is counted and dropped, and the next report says how many were (`dropped`). |
631
+ | `maxMb` | `10` | Files that come with a message are fetched up to this size (50 at most). A larger one is named in the event, without a `path`. Needs `token`. |
632
+ | `keepDays` | `7` | Fetched files are deleted after this many days. |
633
+ | `apiBase` | `https://slack.com/api` | A Slack-compatible server of your own: reached over https, or on this machine. |
634
+
635
+ **`"me"`** is the account the `token` belongs to. With a user token that is you; with a bot
636
+ token it is the bot. So `"mentions": ["me"]` with a user token is "when somebody mentions me",
637
+ and with a bot token "when somebody mentions the bot".
638
+
639
+ **Names and ids.** `channels` takes names; `mentions`, `from`, `ignore` and `sendTo` take ids
640
+ (and `"me"`). A person's id is in their profile under "Copy member ID"; a channel's id is the
641
+ last part of its link ("Copy link"). `sendTo` takes ids only because it is the list that says
642
+ where an agent may write: a name can be given to another channel tomorrow, an id cannot. A
643
+ channel name that the app cannot find is why the source does not start, and the reason is
644
+ shown where the sources are listed: `there is no channel named "opps" that the app can see`.
645
+
646
+ #### The kinds of event
647
+
648
+ | `events` | Event | Fields | When |
649
+ |---|---|---|---|
650
+ | `"message"` | `message` | `{ text, user, userName?, channel, channelName?, kind, ts, thread?, reply?, bot?, mention?, keyword?, from?, files?, images?, dropped? }` | Somebody wrote something that counts. |
651
+ | `"reaction"` | `reaction` | `{ reaction, added, user, userName?, channel, channelName?, ts, author?, authorName? }` | Somebody put an emoji on a message (`added: true`) or took it away (`false`). `ts` is the message, `author` who wrote it. |
652
+ | `"edit"` | `edited` | `{ text, previous?, user, userName?, channel, channelName?, kind, ts, mention?, keyword?, from? }` | A message was changed. `previous` is what it said before. The preview Slack puts under a link is not an edit. |
653
+ | `"edit"` | `deleted` | `{ channel, channelName?, ts, previous? }` | A message was deleted. `previous` is what it said. |
654
+ | `"member"` | `joined`, `left` | `{ user, userName?, channel?, channelName? }` | Somebody joined or left a channel. Without `channel`: somebody joined the workspace. |
655
+ | `"channel"` | `channel` | `{ what, channel, channelName? }` | A channel was `"created"`, `"renamed"`, `"archived"`, `"unarchived"` or `"deleted"`. After `"renamed"`, `channelName` is the new name. |
656
+ | `"pin"` | `pin` | `{ added, channel, channelName?, ts?, user, userName? }` | A message was pinned (`added: true`) or unpinned. |
657
+ | `"command"` | `command` | `{ command, text, user, userName?, channel, channelName? }` | Somebody typed a slash command of your app: `command` is `"/deploy"`, `text` what followed it. Whoever typed it reads "Passed on." |
658
+ | `"other"` | `event` | `{ type, data }` | Any other event your app is subscribed to. `type` is Slack's name for it, `data` the event as Slack sent it, cut at 4000 characters. |
659
+
660
+ Any event can carry `dropped`: how many reports were left out before it, when more than
661
+ `perMinute` counted in a minute.
662
+
663
+ The fields of a `message`:
664
+
665
+ | Field | |
666
+ |---|---|
667
+ | `text` | What was written, as a reader sees it, cut to 4000 characters. `<@U0123>` in it is written as `@Name` (with a `token`), a link as `its words (https://its.address)`. For a message of an app (`bots`) it is put together from its text, its attachments and its blocks, since an alert often says nothing in the text itself. |
668
+ | `user`, `userName` | Who wrote it: the id, and with a `token` the name. For an app the id is the app's (`B...` or its user id), and `userName` what it posts as. |
669
+ | `channel`, `channelName` | Where: the id, and for a channel the name. |
670
+ | `kind` | `"dm"` for a direct message, `"group"` for a group conversation or a private channel, `"channel"` for a public one. |
671
+ | `ts` | Slack's time stamp of the message. It is what names a message: `send` takes it to react to it. |
672
+ | `thread`, `reply` | For an answer in a thread: `reply: true`, and `thread` is the time stamp of the thread. `thread` is what `send` takes to answer in that thread. |
673
+ | `bot` | `true` when an app or integration posted it. |
674
+ | `why` | Why it is reported, a word for each reason: `"dm"` (written to you directly), `"mention"`, `"keyword"`, `"from"` (its sender is on `from`), or `"channel"` when it is none of these and simply was written where the source listens. A list, since a message can be several: a subscription picks with `"payload.why": { "contains": "mention" }`. |
675
+ | `mention`, `keyword` | Which entry of `mentions` it mentions (for `"me"`: the id), and which word on `keywords` it contains. |
676
+ | `from` | `true` when its sender is on `from`. |
677
+ | `files` | Files that came with it, ten at most: `[{ name, type, size, path? }]`. With a `token`, `path` is where pine put the file. A file without a `path` could not be had: larger than `maxMb`, not readable with this token, or the source has no token. |
678
+ | `images` | The pictures among those files (jpg, png, webp, gif, four at most), which are shown to the agent if its model can look at images. |
679
+
680
+ The files lie in the source's own folder under names pine makes, and the agents subscribed to
681
+ the source can read them there (see "Files a source received" in [sources.md](sources.md));
682
+ to keep one, the agent copies it into its workspace. A file is fetched from Slack's file host
683
+ (`files.slack.com`) and from nowhere else, whatever address a message names, and a redirect
684
+ is not followed.
685
+
686
+ **Which message counts**, in this order:
687
+
688
+ 1. Never: a post of your own app, and a message of somebody on `ignore`.
689
+ 2. Never, unless `bots` is on: a message of another app or integration.
690
+ 3. A message of somebody on `from` counts, wherever it was written.
691
+ 4. A direct message counts when `dms` is on.
692
+ 5. Any other message must be in a channel on `channels`, if there is such a list.
693
+ 6. If `mentions` or `keywords` are given, it must mention one of those or contain one of
694
+ those words. If neither is given, every message in the channel counts.
695
+
696
+ The other kinds go the same way as far as they have somebody who did it and a channel:
697
+ `ignore`, `from`, `dms` and `channels` apply to a reaction, a pin, somebody joining, a slash
698
+ command. `mentions` and `keywords` apply to messages and to edits only. An event without a
699
+ channel (somebody joined the workspace) is not held back by `channels`.
700
+
701
+ What counts beyond that is a subscription's business: `when` picks among the events
702
+ (`{"payload.from": true}`, `{"payload.kind": "dm"}`, `{"payload.bot": true}`,
703
+ `{"payload.channel": "C0123OPS"}`), and the note says what to do with each.
704
+
705
+ #### Making the Slack app
706
+
707
+ On api.slack.com/apps, "Create New App", "From scratch":
708
+
709
+ 1. "Socket Mode": switch it on.
710
+ 2. "Basic Information", "App-Level Tokens": make a token with the scope `connections:write`.
711
+ That is `appToken`.
712
+ 3. "Event Subscriptions": switch on, and under "Subscribe to bot events" (or "on behalf of
713
+ users", see below) add the events for what you want reported, from the table.
714
+ 4. "OAuth & Permissions": add the scopes from the table. Slack adds the ones an event needs
715
+ by itself; those for names, files and sending you add.
716
+ 5. For slash commands: "Slash Commands", "Create New Command".
717
+ 6. "Install to Workspace". The token shown then is `token`. After adding an event or a scope
718
+ later, install the app again.
719
+
720
+ | To get | Event subscription | Scope |
721
+ |---|---|---|
722
+ | the connection | | `connections:write`, on the app-level token |
723
+ | messages in public channels | `message.channels` | `channels:history` |
724
+ | messages in private channels | `message.groups` | `groups:history` |
725
+ | direct messages | `message.im` | `im:history` |
726
+ | group conversations | `message.mpim` | `mpim:history` |
727
+ | only messages that mention the bot, without reading channels | `app_mention` | `app_mentions:read` |
728
+ | what other apps post (`bots`), edits and deletions (`"edit"`) | the `message.*` ones above: they arrive as messages | as above |
729
+ | reactions (`"reaction"`) | `reaction_added`, `reaction_removed` | `reactions:read` |
730
+ | who joins and leaves a channel (`"member"`) | `member_joined_channel`, `member_left_channel` | `channels:read`, and `groups:read` for private channels |
731
+ | who joins the workspace (`"member"`) | `team_join` | `users:read` |
732
+ | changes to channels (`"channel"`) | `channel_created`, `channel_rename`, `channel_archive`, `channel_unarchive`, `channel_deleted`; for private channels `group_rename`, `group_archive`, `group_unarchive`, `group_deleted` | `channels:read`, `groups:read` |
733
+ | pins (`"pin"`) | `pin_added`, `pin_removed` | `pins:read` |
734
+ | slash commands (`"command"`) | none: the command is made under "Slash Commands" | `commands` |
735
+ | anything else (`"other"`) | the event you want | the scope Slack names for it |
736
+ | names of people | | `users:read` |
737
+ | names of channels, and `channels` by name | | `channels:read`, and `groups:read` for private channels |
738
+ | the files of a message | | `files:read` |
739
+ | posting a message | | `chat:write` (and `chat:write.public` to post in a public channel the bot is not in) |
740
+ | a direct message to a person on `sendTo` | | `chat:write`, `im:write` |
741
+ | an emoji on a message | | `reactions:write` |
742
+ | sending a file | | `files:write` |
743
+
744
+ A scope that is missing does not stop the source, with one exception: a name is left out, a
745
+ file has no `path`, and a `send` says which scope Slack wants. The exception is a channel
746
+ given by its name, which cannot be found without `channels:read`.
747
+
748
+ A **bot token** (`xoxb-`) sees the channels the bot was invited to (`/invite @yourapp`) and
749
+ direct messages written to the bot. A **user token** (`xoxp-`, with the events subscribed "on
750
+ behalf of users" and the scopes added as user scopes) sees what you see: every channel you
751
+ are in and your own direct messages. To be told when somebody writes to *you*, it has to be a
752
+ user token. What you write yourself is then left out by itself (put `"me"` on `from` if you
753
+ want it reported all the same). What is sent with a user token is sent in your name.
754
+
755
+ #### Sending
756
+
757
+ An agent sends with the `send` tool: `text` for the message, `file` for a file from its
758
+ workspace, and everything else in `data`.
759
+
760
+ | To | The agent passes |
761
+ |---|---|
762
+ | post in a channel on `sendTo` | `text`, and `data`: `{"channel": "C0123OPS"}` |
763
+ | answer in a thread | the same with `"thread"`: the `thread` of the event, or the `ts` of the message that starts one |
764
+ | write to a person on `sendTo` directly | `text`, and `data`: `{"channel": "U0456RAINER"}` |
765
+ | put an emoji on a message | `data`: `{"channel": "C0123OPS", "ts": "1712345678.000100", "react": "eyes"}` |
766
+ | send a file | `file`, with `text` as the message beside it, and `data`: `{"channel": "C0123OPS"}` |
767
+
768
+ With one entry on `sendTo`, `channel` can be left out. Where things may go is the source's to
769
+ decide, not the agent's: a channel or a person that is not on `sendTo` is refused, whatever
770
+ the call names. The one thing beside the list: when a person is on it, the direct conversation
771
+ with them counts as on it too, so that an agent can answer where their message came from
772
+ (`"channel"`: the `channel` of that event). `@channel`, `@here` and `@everyone` in what an
773
+ agent writes are sent as plain words and call on nobody.
774
+
775
+ A Slack channel reaches other people, so a `send` through it is asked about after outside
776
+ content like any other, and what Slack reports is other people's words, never yours: its
777
+ events are outside content whatever `trust` says about your other sources. What the agent
778
+ sent comes back from Slack as a message like anybody's; it is not reported.
779
+
780
+ What Slack refuses, the agent is told in Slack's own word, with what to do where pine knows
781
+ it: `Slack refused chat.postMessage: not_in_channel. The app is not in that channel: somebody
782
+ there writes /invite and its name`.
783
+
784
+ #### Staying connected
785
+
786
+ If the connection ends it is made again, as described under `stream` ("Staying connected").
787
+ Slack itself closes it now and then and says so beforehand; pine connects again at once.
788
+ Slack is pinged every few seconds, so a line that died without a word is noticed. Messages
789
+ written while it was not connected are not asked for afterwards. When Slack says "slow down"
790
+ to a call, pine waits as long as Slack says and asks again.
791
+
792
+ Where the sources are listed, the line says how things stand: `connected since 14:02:11.
793
+ listening to 2 channels and direct messages, for messages, reactions`, or `cannot connect
794
+ since 14:02:11 (3 attempts): Slack refused apps.connections.open: invalid_auth`. A `token`
795
+ that Slack does not take, and a channel name it does not know, are why a source does not
796
+ start at all.
797
+
798
+ #### Coming from the old Slack monitor
799
+
800
+ The OpenClaw plugin `openclaw-slack-monitor`:
801
+
802
+ | There | Here |
803
+ |---|---|
804
+ | `appToken`, `userToken` | `appToken`, `token` |
805
+ | `watchChannels` | `channels` (ids as before, or names) |
806
+ | `watchUserIds` | `mentions` (who has to be mentioned; `"me"` for yourself). Your own messages are left out without a setting. |
807
+ | `watchKeywords` | `keywords` |
808
+ | `watchDMs` | `dms` |
809
+ | `urgentUserIds` | `from` |
810
+ | `notifyChannel`, `notifyTo` | the subscription's note: "tell me on Telegram who wrote and what" |
811
+ | `urgentPrefix`, `urgentRepeat`, `urgentRouteOnly` | the note again: "when `from` is true, say so first and send it twice", or a conversation of its own subscribed with `"when": {"payload.from": true}` |
812
+ | `urgentCooldownSec` | `throttle` on that subscription |
813
+
814
+ The one difference in what counts: a user on `ignore` is never reported, also not in a
815
+ channel when a keyword is found in what they wrote.
816
+
817
+ #### Limits, and what is refused
818
+
819
+ - One source is one Slack app in one workspace. For a second workspace, a second source.
820
+ - Only what happens while it is connected is reported. Nothing is read from a channel's past.
821
+ - A message's text is cut at 4000 characters, its files at ten, its pictures at four. A name
822
+ is looked up once and kept until pine restarts; a renamed channel is noticed when the app
823
+ is subscribed to `channel_rename`.
824
+ - Files are fetched from `files.slack.com` only, with the `token`. A file larger than `maxMb`
825
+ is not fetched; one that turns out larger than Slack said is given up while reading.
826
+ - A button somebody presses and a form somebody fills in (Slack's "interactivity") are
827
+ acknowledged and not reported.
828
+ - A slash command is answered with "Passed on." and nothing else: what the agent has to say,
829
+ it sends as a message.
830
+ - It never speaks for you by itself: there is no `trust: "owner"` for what is written in
831
+ Slack, and nothing is sent anywhere that is not on `sendTo`.
832
+ - Refused when it is set up: an `appToken` that does not start with `xapp-`, a `token` that
833
+ does not start with `xoxb-` or `xoxp-` (a token that rotates, `xoxe...`, is not taken), a
834
+ name where an id belongs (`mentions`, `from`, `ignore`, `sendTo`), a kind in `events` that
835
+ there is none of, `perMinute` outside 1 to 600, `maxMb` above 50, and an `apiBase` that is
836
+ neither https nor on this machine.
837
+ - Refused when it starts: `"me"` or a channel by name without a `token`, a channel name that
838
+ is not found or an app that may not list channels (the scope `channels:read`), and a `token`
839
+ Slack does not take. The reason is shown where the sources are listed.
840
+
841
+ ### matrix
842
+
843
+ A Matrix account: reports what is written in the rooms it has joined, all of it or only what
844
+ mentions somebody or contains a word. It asks the homeserver with one request that is
845
+ answered the moment something arrives, so nothing listens on a port.
846
+
847
+ > Keep an eye on the family room on Matrix and tell me when somebody asks for me.
848
+
849
+ ```json
850
+ "family": {
851
+ "type": "matrix", "homeserver": "https://matrix.org", "token": "${MATRIX_TOKEN}",
852
+ "rooms": ["!AbCdEfGh:matrix.org"], "mentions": ["@paul:matrix.org"],
853
+ "description": "the family room"
854
+ }
855
+ ```
856
+
857
+ | Option | Default | |
858
+ |---|---|---|
859
+ | `homeserver` | | The address of the server the account is on: `https://matrix.org`. `http://` only for one on your own network or this computer. |
860
+ | `token` | | An access token of the account. Kept in `.env`. |
861
+ | `rooms` | every joined room | Only these rooms, by id (`!AbCdEfGh:matrix.org`). |
862
+ | `mentions` | | User ids (`@paul:matrix.org`): only messages that mention one of them. |
863
+ | `keywords` | | Only messages that contain one of these words. Capitals do not matter. |
864
+ | `ignore` | | User ids that are never reported. What the account itself writes never is. |
865
+ | `sendTo` | none | Room ids an agent may write to. Without it the source only listens. See "Sending through a source". |
866
+ | `local` | `false` | The homeserver is on your own network under a name that does not say so. |
867
+
868
+ Event `message`: `{ text, user, room, roomName?, kind, id, mention?, keyword?, dropped? }`.
869
+ `kind` is `"dm"` for a room the account keeps as a direct chat, else `"room"`; `id` is the
870
+ event's id; `mention` and `keyword` say which entry of those lists was found. Edits of a
871
+ message are not reported a second time.
872
+
873
+ **Where the access token is.** In Element: "All settings", "Help & About", "Advanced",
874
+ "Access Token". It is the key to the whole account, so it is best taken from a session of its
875
+ own (log in once in a private window, copy the token, and close the window without logging
876
+ out: logging out ends the token). An account made only for pine, invited to the rooms it is
877
+ to read, is the cleaner way.
878
+
879
+ **The starting point.** The first look only notes where "now" is: what was written before is
880
+ not reported. From then on pine remembers how far it has read, also across a restart, so what
881
+ was written while pine was not running is reported when it is back (up to 50 messages a
882
+ room).
883
+
884
+ **Encrypted rooms cannot be read.** pine holds no keys. Messages in a room with end-to-end
885
+ encryption are counted, and where the sources are listed the line says how many: `connected
886
+ since 14:02:11. 3 messages in encrypted rooms could not be read`. For such a room, use one
887
+ that is not encrypted, or a bridge.
888
+
889
+ Sending writes to a room on `sendTo` and nowhere else, as the account: `{"room":
890
+ "!AbCdEfGh:matrix.org", "text": "..."}`. What is written in a room is other people's words:
891
+ its events are outside content.
892
+
893
+ Refused when it is set up: a room given by its alias (`#family:matrix.org`) instead of its id
894
+ (the id is in the room's settings under "Advanced"), a user id that is not of the form
895
+ `@name:server`, and a homeserver outside your network that is not `https://`. A token the
896
+ homeserver does not take shows as `cannot connect since ...: the homeserver refused:
897
+ M_UNKNOWN_TOKEN`.
898
+
899
+ ### discord
900
+
901
+ A Discord bot: reports what is written in the servers it was invited to and in direct
902
+ messages to it, all of it or only what mentions the bot or contains a word. It connects out
903
+ to Discord's gateway, so nothing listens on a port.
904
+
905
+ > Watch #announcements on our Discord and tell me when a release is mentioned.
906
+
907
+ ```json
908
+ "community": {
909
+ "type": "discord", "token": "${DISCORD_BOT_TOKEN}",
910
+ "channels": ["1234567890123456789"], "keywords": ["release"],
911
+ "description": "announcements on our Discord server"
912
+ }
913
+ ```
914
+
915
+ | Option | Default | |
916
+ |---|---|---|
917
+ | `token` | | The bot's token from the Developer Portal. Kept in `.env`. |
918
+ | `channels` | every channel the bot can read | Only these channels, by id. |
919
+ | `guilds` | every server the bot is in | Only these servers, by id. |
920
+ | `mentions` | `false` | `true`: in servers, only messages that mention the bot. |
921
+ | `keywords` | | In servers, only messages that contain one of these words. Capitals do not matter. With `mentions` as well, either is enough. |
922
+ | `dms` | `true` | Direct messages to the bot count, without a mention or a keyword. |
923
+ | `bots` | `false` | What other bots and webhooks write counts too. |
924
+ | `sendTo` | none | Channel ids an agent may write to. Without it the source only listens. See "Sending through a source". |
925
+ | `apiBase` | `https://discord.com/api/v10` | A Discord-compatible server of your own: reached over https, or on this machine. |
926
+
927
+ Ids are long numbers. With "Developer Mode" on (User Settings, Advanced), a right click on a
928
+ channel or a server offers "Copy ID". A name is refused when the source is set up.
929
+
930
+ Event `message`:
931
+ `{ text, user, userName, channel, channelName?, guild?, guildName?, kind, id, mention?, keyword?, files?, dropped? }`.
932
+ `kind` is `"dm"` for a direct message to the bot and `"channel"` for one in a server; `mention`
933
+ is `true` when the bot is mentioned; `files` lists what was attached (`[{ name, type, size }]`,
934
+ not the contents). In `text`, `<@123...>` is written as `@name` and `<#123...>` as `#channel`.
935
+ What the bot itself writes is never reported, and neither is somebody joining or a pinned
936
+ message.
937
+
938
+ **Making the bot.** On discord.com/developers/applications:
939
+
940
+ 1. "New Application", then "Bot": "Reset Token" shows the token once. That is `token`.
941
+ 2. On the same page, under "Privileged Gateway Intents", switch on **Message Content
942
+ Intent**. Without it Discord closes the connection, and the source says so where the
943
+ sources are listed: `the bot is not allowed to read messages yet: switch on "Message
944
+ Content Intent" ...`.
945
+ 3. "OAuth2", "URL Generator": tick `bot`, and under "Bot Permissions" tick "View Channels" and
946
+ "Read Message History" (and "Send Messages" if an agent is to write). Open the address it
947
+ makes and pick your server: that is the invitation.
948
+
949
+ The bot sees the channels its role may read. A private channel has to let the bot in.
950
+
951
+ The connection is kept the way Discord asks: a heartbeat at the interval Discord names, and a
952
+ session that is taken up again after the line dropped, so that what was written in the
953
+ meantime still arrives. If the session is gone, a new one is started, and what was written
954
+ while the bot was away is not reported. See `stream`, "Staying connected", for how often it
955
+ tries.
956
+
957
+ Sending writes to a channel on `sendTo` and nowhere else, as the bot: `{"channel":
958
+ "1234567890123456789", "text": "..."}`. A text of more than 2000 characters goes out as
959
+ several messages, and it pings nobody: no `@everyone`, no role, no person. A Discord channel reaches
960
+ other people, and what is written there is other people's words: its events are outside
961
+ content.
962
+
963
+ ### smtp
964
+
965
+ A mail drop: a small mail server that accepts messages and nothing else. For devices and
966
+ programs whose only way of reporting is "send an e-mail": they are told this computer as their
967
+ mail server, and what they send becomes an event instead of a mail.
968
+
969
+ > My NAS can only send e-mail alerts. Take them and tell me on Telegram when a disk is failing.
970
+
971
+ ```json
972
+ "nas": { "type": "smtp", "port": 2525, "user": "nas", "password": "${NAS_MAIL_PASSWORD}", "to": ["alerts@pine.home"], "description": "alerts the NAS sends" }
973
+ ```
974
+
975
+ | Option | Default | |
976
+ |---|---|---|
977
+ | `port` | | The port to listen on, `2525`. (25, 465 and 587 are the usual ones; below 1024 many systems want root.) |
978
+ | `user` | `"device"` | The user name the device logs in with. |
979
+ | `password` | made by pine | The password the device logs in with, 8 characters or more. Left out, pine makes one and keeps it in `.env`. Left out together with `allow` given, no login is asked for. |
980
+ | `allow` | everyone who logs in | Only these client addresses, as prefixes: `["192.168.1.40", "192.168.2."]`. For a device that cannot log in, give its address here and no password. |
981
+ | `host` | `"0.0.0.0"` | The address to listen on. The default lets devices on your network reach it. |
982
+ | `to` | any | Recipient addresses it accepts: `["alerts@pine.home"]`. Mail for any other is refused. The address does not have to exist anywhere; it is only what you enter in the device. |
983
+ | `maxMb` | `10` | The largest message accepted (50 at most). |
984
+ | `keepDays` | `7` | Received attachments are deleted after this many days. |
985
+
986
+ | Event | | |
987
+ |---|---|---|
988
+ | `mail` | `{ from, fromName?, to, subject, text, attachments, images?, client }` | One for every message. `from` is what the message says, `to` the recipients the device named, `client` the address it really came from. `attachments` and `images` as for `mail`. |
989
+
990
+ In the device, enter this computer's address as the mail (SMTP) server, the port, **no
991
+ encryption** (no SSL, no TLS, no STARTTLS), the user and the password, and any sender and
992
+ recipient address (one from `to`, if you set that). The agent that set the source up tells you
993
+ where the password is: in pine's `.env`, as `PINE_SOURCE_<NAME>_<AGENT>_PASSWORD`.
994
+
995
+ | What it is for | How |
996
+ |---|---|
997
+ | Synology DSM, QNAP QTS | Control Panel, Notification, Email: a custom SMTP server, this computer and the port, no secure connection. Disk, volume, UPS and backup alerts. |
998
+ | TrueNAS, Proxmox, Unraid | Their e-mail settings take an SMTP server with user and password ("outgoing mail server", "notification target"). |
999
+ | UniFi, routers | UniFi's Network application takes a custom SMTP server in its e-mail settings; a Fritz!Box under System, Push Service, Sender, with the box for a secure connection left unticked. |
1000
+ | Printers and scanners | "Scan to e-mail": the scan arrives as an attachment, a PDF or a picture, and the agent can file it. Most cannot log in: give `allow` with the printer's address and no password. |
1001
+ | UPS units, network cards of UPS units | APC, Eaton and CyberPower mail on power loss and low battery. |
1002
+ | Cameras and recorders | A snapshot by mail on motion (Reolink, Hikvision, Dahua). The picture is shown to the agent. A camera that can upload by FTP is better served by `ftp`. |
1003
+ | A Linux server | cron, smartd, unattended-upgrades, fail2ban and mdadm all mail to root. With `msmtp` (`/etc/msmtprc`: `host`, `port`, `auth on`, `user`, `password`, `tls off`) or a Postfix `relayhost` that mail arrives here. |
1004
+ | Your own scripts | Anything that can call `sendmail`, `mail` or `curl smtp://`. |
1005
+
1006
+ It speaks plain SMTP, as these devices do: the password and the mail cross your network
1007
+ unencrypted, so it is for your own network. It **only receives**. Nothing is passed on to
1008
+ anybody (it is no relay), nothing can be listed or read back, and every attachment lands under
1009
+ a name of pine's choosing inside the source's own folder, whatever the mail called it. It
1010
+ answers EHLO, HELO, AUTH (PLAIN and LOGIN), MAIL, RCPT, DATA, RSET, NOOP and QUIT, and says
1011
+ "not implemented" to everything else.
1012
+
1013
+ What a mail says is outside content, also from your own NAS: who it is "from" is a line the
1014
+ sender wrote. For a device nobody else can reach, on an address you fixed with `allow`, you can
1015
+ say `own` when you ask the agent to set it up (or `"trust": "owner"` in the file), as for any
1016
+ source of yours.
1017
+
1018
+ One of `password` and `allow` is needed, or the source does not start: without either, anyone
1019
+ on your network could hand text to an agent. Refused: a message above `maxMb`, a recipient not
1020
+ on `to`, a client not on `allow` (the connection is closed without a greeting), three wrong
1021
+ passwords in a row (the connection is closed), more than 20 connections at a time, a connection
1022
+ that says nothing for a minute, and more than 30 messages in a minute, which get "try again
1023
+ later", so that a device in a loop does not fill a conversation; a mail program sends them
1024
+ again by itself. The status line counts what was received and what was refused.
1025
+
1026
+ ### syslog
1027
+
1028
+ A syslog listener: takes the log lines that routers, firewalls, switches, access points and NAS
1029
+ boxes send once they are told a "remote log server", and reports the ones that matter.
1030
+
1031
+ > Have the router send its log here and tell me when the internet line goes down or somebody
1032
+ > fails to log in.
1033
+
1034
+ ```json
1035
+ "router": { "type": "syslog", "port": 5514, "allow": ["192.168.1.1"], "match": "link down|PPP|login failed", "severity": "notice", "description": "the router's log" }
1036
+ ```
1037
+
1038
+ | Option | Default | |
1039
+ |---|---|---|
1040
+ | `port` | | The port to listen on, `5514`. (514 is the usual one, which on many systems only root may open.) |
1041
+ | `host` | `"0.0.0.0"` | The address to listen on. The default lets devices on your network reach it. |
1042
+ | `protocol` | `"udp"` | `"udp"` is what most devices send. `"tcp"`, or `"both"`. |
1043
+ | `allow` | anyone | Only these sender addresses, as prefixes: `["192.168.1.1", "192.168.2."]`. |
1044
+ | `match` | | Only lines in which this regular expression finds something. |
1045
+ | `severity` | `"warning"` | The least severe level still reported: `emerg`, `alert`, `crit`, `err`, `warning`, `notice`, `info`, `debug`. `"warning"` reports warnings and everything worse. |
1046
+ | `perMinute` | `20` | At most this many reports a minute (1 to 600). What comes on top is counted and dropped. |
1047
+
1048
+ | Event | | |
1049
+ |---|---|---|
1050
+ | `line` | `{ host, app?, severity, facility, message, from, time?, dropped? }` | One for every line that is severe enough and matches. `host` and `app` are what the line says about itself (`host` is the sender's address when the line names none), `from` is the address it really came from, `message` the text up to 2000 characters. `dropped` says how many lines were left out by `perMinute` since the last report. |
1051
+
1052
+ A line has to pass both `severity` and `match`. To report every line that matches, whatever its
1053
+ level, set `"severity": "debug"`. Both formats in use are read: the older one (RFC 3164,
1054
+ `<13>Oct 11 22:14:15 router dnsmasq[1873]: ...`), also where a device leaves out the time or
1055
+ its name, and the newer one (RFC 5424), whose `time` is passed on. A line that is neither is
1056
+ reported as it is, with the severity `notice`. Over TCP a line ends with a line break or has
1057
+ its length in front; both are read.
1058
+
1059
+ | What it is for | How |
1060
+ |---|---|
1061
+ | OpenWrt | System, System, Logging: "External system log server" and its port; or `uci set system.@system[0].log_ip=<address>` and `log_port=5514`. |
1062
+ | UniFi | In the Network application's settings: "Remote syslog server" (called "SIEM server" in newer versions), with the address and the port. |
1063
+ | MikroTik RouterOS | `/system logging action set remote remote=<address> remote-port=5514`, then `/system logging add topics=error action=remote`, one such line for each topic you want. |
1064
+ | pfSense, OPNsense | Status, System Logs, Settings, "Remote log servers" (pfSense); System, Settings, Logging, Targets (OPNsense). |
1065
+ | Synology DSM | Log Center, Log Sending: the server, the port, UDP or TCP, format BSD or IETF. |
1066
+ | A Linux server | rsyslog: a line `*.warn @<address>:5514` in `/etc/rsyslog.d/pine.conf` (one `@` is UDP, `@@` is TCP). systemd-journald alone cannot send; rsyslog or syslog-ng does it. |
1067
+ | Switches, access points, printers, IP phones | Almost all have a "syslog server" field in their settings. |
1068
+
1069
+ **Syslog has no login.** Whoever can reach the port can write a line under any host name.
1070
+ Without `allow` the dialog you confirm says so in capitals; with it, lines from other addresses
1071
+ are counted and dropped. Over UDP even the sender's address can be forged by somebody on the
1072
+ same network, so a line is outside content like everything else a source reports. It only
1073
+ receives: nothing is answered, nothing is stored but the events.
1074
+
1075
+ A device that logs many times a second costs little: a line that is not severe enough or does
1076
+ not match is counted and nothing else, and what passes is capped by `perMinute`. The status
1077
+ line says how many were reported, how many were not severe enough or did not match, how many
1078
+ were over the limit and how many came from addresses not allowed; it never repeats a line.
1079
+ Also set a `throttle` on the subscription that fits ("5m"), so that a burst wakes the agent
1080
+ once. Up to 20 TCP connections at a time, lines up to 8 KB.
1081
+
1082
+ ### onvif
1083
+
1084
+ A network camera's own detections: motion, a person, a vehicle, tampering, a line crossed, a
1085
+ doorbell's button. The camera does the looking; this source asks it what it saw, in the
1086
+ language most cameras and recorders share (ONVIF, Profile S and T).
1087
+
1088
+ > When the camera at the gate sees a person, tell me on Telegram.
1089
+
1090
+ ```json
1091
+ "gate": { "type": "onvif", "host": "192.168.1.40", "port": 8000, "user": "pine", "password": "${GATE_CAMERA_PASSWORD}", "topics": ["Motion", "People", "Visitor"], "description": "what the camera at the gate detects" }
1092
+ ```
1093
+
1094
+ | Option | Default | |
1095
+ |---|---|---|
1096
+ | `host` | | The camera's address on your network: `192.168.1.40`. |
1097
+ | `user`, `password` | | An account on the camera. Make one for pine with the lowest rights that may read events. The password is kept in `.env`. |
1098
+ | `port` | `80` | The camera's ONVIF port. See the table of makes below. |
1099
+ | `path` | `"/onvif/device_service"` | Where its ONVIF service is. Nearly every camera uses this one. |
1100
+ | `topics` | everything | Only events whose topic contains one of these words; capitals do not matter. `["Motion", "Person"]`. |
1101
+ | `perMinute` | `20` | At most this many reports a minute. What comes on top is dropped, and the next event says how many in `dropped`. |
1102
+ | `local` | `false` | The address is a name on your own network that does not say so (`gate.lan`). Not needed for an IP address or a `.local` name. |
1103
+
1104
+ Event `event`: `{ topic, source, data, time?, active?, dropped? }`.
1105
+
1106
+ | Field | |
1107
+ |---|---|
1108
+ | `topic` | What kind of thing happened, as the camera calls it: `RuleEngine/CellMotionDetector/Motion`. |
1109
+ | `source` | Which part of the camera it is about, as names and values: `{ "VideoSourceConfigurationToken": "1" }`. A recorder with eight cameras tells them apart here. |
1110
+ | `data` | What it says: `{ "IsMotion": true }`. `true`, `false` and numbers come as such. |
1111
+ | `active` | When `data` holds exactly one yes-or-no value, that value: `true` when motion starts, `false` when it ends. To be woken only at the start, subscribe with the filter `{"payload.active": true}`. |
1112
+ | `time` | The camera's own time for it. |
1113
+
1114
+ | Make | Port | To switch on |
1115
+ |---|---|---|
1116
+ | Reolink | 8000 | Settings › Network › Advanced › Port settings: ONVIF. |
1117
+ | Hikvision | 80 | Network › Advanced › Integration protocol: enable ONVIF and add an ONVIF user there (it is separate from the camera's own users). |
1118
+ | Dahua, Amcrest | 80 | On by default on most; otherwise Network › Access platform › ONVIF. |
1119
+ | Axis | 80 | System › ONVIF: add an ONVIF account. |
1120
+ | TP-Link Tapo | 2020 | In the app: Advanced settings › Camera account. That account is the user here. |
1121
+ | Others (Uniview, Foscam, Annke, most recorders) | 80, 8080 or 8899 | Look for "ONVIF" in the network settings. |
1122
+
1123
+ | What it reports | Topic contains |
1124
+ |---|---|
1125
+ | Motion | `Motion` (`CellMotionDetector/Motion` with `IsMotion`, or `VideoSource/MotionAlarm` with `State`) |
1126
+ | A person, a vehicle, an animal | `People`, `Person`, `Vehicle`, `DogCat`, `FaceDetect`, depending on the make |
1127
+ | Tampering: covered, moved, defocused | `Tamper` |
1128
+ | A line crossed, somebody inside a zone | `LineDetector/Crossed`, `FieldDetector/ObjectsInside` |
1129
+ | A doorbell's button | `Visitor` |
1130
+ | A wired alarm input | `DigitalInput` |
1131
+
1132
+ The names differ from make to make. Set the source up without `topics` first, walk past the
1133
+ camera, and look at what arrives (`pine events --topic 'source/gate/#'`); then narrow it down.
1134
+
1135
+ pine asks the camera for a subscription and then keeps one question open with it, which the
1136
+ camera answers the moment something happens (a "pull point"). The camera needs no way to reach
1137
+ pine, and nothing has to be opened in a firewall. If the camera restarts or forgets the
1138
+ subscription, a new one is made, after a pause that grows while the camera stays away.
1139
+
1140
+ **The first thing a camera says is where things stand**: no motion, not tampered with. That is
1141
+ the starting point and is not reported; a change from it is. A state that is said again
1142
+ without having changed is not reported either. Something that just happens rather than is (a
1143
+ line crossed) is reported every time.
1144
+
1145
+ Every request goes to the host and port you confirmed. A camera names addresses in its
1146
+ answers (where its event service is, where the subscription lives); of those only the path is
1147
+ used, never the host. The password never crosses the network: what is sent is a digest over a
1148
+ random number, the camera's time and the password, which is what ONVIF prescribes. The rest
1149
+ is plain HTTP, as cameras speak it on a local network. What a camera writes into an error is
1150
+ not repeated where agents read the state of a source.
1151
+
1152
+ It only listens: nothing on the camera is changed, moved or switched. It brings the event, not
1153
+ a picture. For the picture, let the camera upload one when it sees something (the `ftp` type),
1154
+ or have the agent fetch a snapshot when it is woken.
1155
+
1156
+ This type was written against the ONVIF specification and a stand-in for a camera, and has
1157
+ not been tried on every make. Two things it does not do: cameras that accept only HTTP
1158
+ digest authentication for ONVIF (a setting on some Axis and Hikvision models: choose "digest
1159
+ and WS-UsernameToken"), and ONVIF over HTTPS.
1160
+
1161
+ ### modbus
1162
+
1163
+ A device that speaks Modbus TCP, read again and again: reports each value when it changes. For
1164
+ what sits in the cellar or on the roof and has a network socket but no way of telling anybody
1165
+ anything: a solar inverter, a battery, a heat pump, a wall box, an energy meter.
1166
+
1167
+ > Tell me when the house battery drops below 20%, and when it is full again.
1168
+
1169
+ ```json
1170
+ "battery": {
1171
+ "type": "modbus", "host": "192.168.1.60", "unit": 100, "every": "1m",
1172
+ "read": [
1173
+ { "name": "soc", "address": 843 },
1174
+ { "name": "power", "address": 842, "type": "i16", "step": 200 },
1175
+ { "name": "grid", "address": 820, "type": "i16", "step": 200 },
1176
+ { "name": "voltage", "address": 840, "scale": 0.1, "step": 0.5 }
1177
+ ],
1178
+ "description": "the house battery: charge in percent, power in watts, power from the grid, voltage"
1179
+ }
1180
+ ```
1181
+
1182
+ | Option | Default | |
1183
+ |---|---|---|
1184
+ | `host` | | The device's address on your network: `192.168.1.60`. |
1185
+ | `read` | | The values to read, 1 to 32 of them. See the next table. |
1186
+ | `port` | `502` | The port. Nearly always 502; SolarEdge inverters use 1502. |
1187
+ | `unit` | `1` | The device's unit id, which some call slave id. A gateway that fronts several devices tells them apart by it. |
1188
+ | `every` | `"30s"` | How often to read. At least `"2s"` when an agent sets it up. |
1189
+ | `local` | `false` | The address is a name on your own network that does not say so (`inverter.lan`). Not needed for an IP address or a `.local` name. |
1190
+
1191
+ Each entry of `read`:
1192
+
1193
+ | Key | Default | |
1194
+ |---|---|---|
1195
+ | `name` | | A short word for the value: `power`, `soc`. It is what the event carries, and what a filter names. |
1196
+ | `address` | | The register's number as the protocol counts it, from 0. A register list that counts from 40001 (holding registers) or 30001 (input registers) means the number minus that; one that says "base 1" means the number minus 1. |
1197
+ | `kind` | `"holding"` | `"holding"`, `"input"`, `"coil"` or `"discrete"`: which of the device's four tables the value is in. The register list says so; `coil` and `discrete` are single bits and come as `true` or `false`. |
1198
+ | `type` | `"u16"` | How the register is read: `"u16"`, `"i16"` (may be negative), `"u32"`, `"i32"`, `"f32"` (a floating-point number). The last three are two registers. |
1199
+ | `scale` | `1` | The raw value is multiplied by it: `0.1` for a voltage given in tenths of a volt. |
1200
+ | `step` | | Report the value only when it has moved at least this far since it was last reported: `200` for a power in watts that never stands still. Without it every change is an event. |
1201
+ | `words` | `"be"` | For a 32-bit value: `"be"` if the first register is the high half (most devices), `"le"` if it is the low half. A value that reads as nonsense with one is right with the other. |
1202
+
1203
+ Event `changed`: `{ name, value, previous }`, one for each value that moved. To be woken only
1204
+ below a threshold, subscribe with a filter: `{"payload.name": "soc", "payload.value": {"lt": 20}}`.
1205
+
1206
+ The example is a Victron GX device (Cerbo, Venus OS). Its system values are at unit 100:
1207
+ register 843 is the battery's state of charge in percent, 842 the battery's power in watts
1208
+ (negative while it discharges), 820 the power drawn from the grid on the first phase, 840 the
1209
+ battery voltage in tenths of a volt. Modbus TCP is switched on under Settings › Services, and
1210
+ Victron publishes the whole list as a spreadsheet ("CCGX Modbus TCP register list").
1211
+
1212
+ | What it is for | How |
1213
+ |---|---|
1214
+ | Solar inverters and batteries (SolarEdge, Fronius, SMA, Huawei, Sungrow, Victron) | Most speak SunSpec, a common register layout, over Modbus TCP once it is switched on in the device's settings or installer menu. The maker's "Modbus" or "SunSpec" document lists the registers. |
1215
+ | Heat pumps (Stiebel Eltron, Nibe, Vaillant, Daikin with a gateway) | Flow and outside temperature, operating state, a fault code. Usually an extra network module, and its manual has the register list. |
1216
+ | Wall boxes (go-e, Keba, Wallbox, openWB) | Whether a car is plugged in, charging power, energy of the session. |
1217
+ | Energy meters (Eastron SDM, Shelly Pro 3EM, a smart meter gateway) | Power per phase, energy totals. Meters with only a serial bus (Modbus RTU) need a small RTU-to-TCP gateway in front. |
1218
+ | Anything with a PLC | A coil or a discrete input is a pump that runs, a door that is open, an alarm contact. |
1219
+
1220
+ The device's register list is the one thing this type cannot know. Ask for the document by
1221
+ the device's exact model ("... Modbus register map"); an agent can read it and pick the
1222
+ registers. Where the list gives a value a scale in a register of its own (SunSpec's `_SF`
1223
+ registers), read the value raw and the scale beside it, or use the device's floating-point
1224
+ registers if it has them.
1225
+
1226
+ **It only reads.** Of the protocol's functions only the four that read are ever sent (1 to 4),
1227
+ so nothing can be switched, set or charged through this source, whatever an agent is told.
1228
+ Each look is one short connection: several devices allow only one Modbus connection at a
1229
+ time, and one that is held would lock a home automation system out.
1230
+
1231
+ A device that says no to a register (`illegal data address`: the number is not one it has),
1232
+ does not answer within five seconds or closes the connection is a failed look, shown with its
1233
+ reason wherever sources are listed, and not an event. The address has to be on your own
1234
+ network, or it has to stay on the public internet for good, as with `http`.
1235
+
1236
+ Refused when it is set up: more than 32 values, two values of one name, an address above
1237
+ 65535, a `type` or `kind` that does not exist, `every` below `"2s"`.
1238
+
1239
+ ### serial
1240
+
1241
+ A serial port: reports the lines a device prints on it. For what is plugged into this computer
1242
+ by USB or a serial cable and writes text: a microcontroller that prints its readings, a smart
1243
+ meter, a GPS receiver, a weather station, a UPS.
1244
+
1245
+ > The Arduino on the USB port prints the greenhouse temperature every second. Tell me when it
1246
+ > goes above 35.
1247
+
1248
+ ```json
1249
+ "greenhouse": { "type": "serial", "port": "/dev/cu.usbmodem1101", "baud": 115200, "match": "temp=([0-9.]+)", "changed": true, "description": "the temperature in the greenhouse" }
1250
+ ```
1251
+
1252
+ | Option | Default | |
1253
+ |---|---|---|
1254
+ | `port` | | The device: `/dev/cu.usbserial-1420` or `/dev/cu.usbmodem1101` on a Mac (`ls /dev/cu.*` lists them), `/dev/ttyUSB0` or `/dev/ttyACM0` on Linux, or its lasting name under `/dev/serial/by-id/`. |
1255
+ | `baud` | `9600` | The speed the device sends at: 9600, 19200, 38400, 57600, 115200 and the other usual ones. With the wrong one nothing readable arrives. |
1256
+ | `match` | | Report only lines this regular expression finds. Its first group becomes `value`, as a number when it is one. |
1257
+ | `changed` | `false` | Report a line (or its `value`) only when it differs from the one before. For a device that prints the same reading every second. |
1258
+ | `perMinute` | `30` | At most this many reports a minute. What comes on top is dropped, and the next event says how many in `dropped`. |
1259
+
1260
+ Event `line`: `{ port, line, value?, previous?, dropped? }`. Without `match`, `value` is there
1261
+ when the line is JSON (`{"temp": 21.5}`) or a number. With `changed`, `previous` is what it was
1262
+ before. To be woken only above a threshold, subscribe with a filter:
1263
+ `{"payload.value": {"gt": 35}}`.
1264
+
1265
+ | What it is for | How |
1266
+ |---|---|
1267
+ | An Arduino, ESP32 or Pico that prints readings | `Serial.println()` in its sketch is all it takes. Print one reading a line, as `name=value` or as JSON. |
1268
+ | A smart meter's P1 port (Dutch and Belgian meters, many others) | A USB P1 cable, 115200 baud. It sends a block of lines every second or ten; `"match": "1-0:1\\.7\\.0\\(([0-9.]+)"` is the power drawn right now in kW, `1-0:1.8.1` and `1-0:1.8.2` are the meter readings. |
1269
+ | A GPS receiver | NMEA sentences at 9600 baud (4800 for old ones); `"match": "^\\$G.GGA,(.*)"` is one position a second. |
1270
+ | A weather station or a sensor with a data cable | Whatever its manual says it prints. |
1271
+ | A UPS with a serial or USB cable | The ones that print status lines. Most want to be asked first, which this type does not do: use the maker's tool or NUT with the `command` type instead. |
1272
+
1273
+ **It only listens.** Nothing is ever written to the port, so a device that has to be asked
1274
+ before it answers stays silent. The port is read as 8 data bits, no parity, one stop bit,
1275
+ which is what nearly everything uses.
1276
+
1277
+ Only a serial port is read, and it is told by its name: `/dev/cu.*` or `/dev/tty.*` on a Mac,
1278
+ `/dev/ttyUSB0`, `/dev/ttyACM0`, `/dev/ttyS0`, `/dev/ttyAMA0`, `/dev/rfcomm0` and the like on
1279
+ Linux. A terminal is a device of the same kind, and reading from one would take away what
1280
+ somebody types there; those are called otherwise and are refused, like every path that is not
1281
+ a device under `/dev`.
1282
+
1283
+ A device that is unplugged is not an error: the source waits, looks for it again with a
1284
+ growing pause (a minute at most), and says `waiting for cu.usbmodem1101: it is not there
1285
+ (unplugged?)` where sources are listed. On Linux the user pine runs as has to be allowed to
1286
+ read the port (the group `dialout` on most systems). Lines are cleaned of control characters
1287
+ and cut at 1000 characters; a device that never ends its line is cut every 4096.
1288
+
1289
+ ### presence
1290
+
1291
+ Who is on your own network: reports when a device joins it and when it has left. For "is my
1292
+ phone home", a TV or a games console that was switched on, a printer that came online.
1293
+
1294
+ > When my phone comes home after having been away for more than an hour, switch the hallway
1295
+ > light on.
1296
+
1297
+ ```json
1298
+ "home": { "type": "presence", "hosts": { "phone": "192.168.1.23", "tv": "192.168.1.30" }, "services": ["_ipp._tcp"], "away": "10m", "description": "who is at home: my phone, the TV, and any printer" }
1299
+ ```
1300
+
1301
+ | Option | Default | |
1302
+ |---|---|---|
1303
+ | `hosts` | | The devices to look for, as name and address: an IP address on your network, or a `.local` name. Up to 32. |
1304
+ | `services` | | Kinds of device that announce themselves on the network (Bonjour, also called mDNS), by their service type. Every device of that kind is reported under its own name. Up to 16. |
1305
+ | `every` | `"30s"` | How often to look. At least `"10s"` when an agent sets it up. |
1306
+ | `away` | `"10m"` | How long a device must have been silent before it counts as gone. |
1307
+
1308
+ At least one of `hosts` and `services`.
1309
+
1310
+ | Event | | |
1311
+ |---|---|---|
1312
+ | `arrived` | `{ name, address?, service?, awayFor? }` | It is there again. `awayFor` is how long it was gone, in seconds; to be woken only after a real absence, subscribe with the filter `{"payload.name": "phone", "payload.awayFor": {"gt": 3600}}`. |
1313
+ | `left` | `{ name, address?, service?, lastSeen }` | It has not answered for as long as `away` says. `lastSeen` is when it last did. |
1314
+
1315
+ | What it is for | How |
1316
+ |---|---|
1317
+ | Is my phone home | `hosts`, with the phone's address. Tell the router to always give the phone the same address (a "DHCP reservation"), and on an iPhone switch off "Private Wi-Fi Address" for the home network, or it changes. |
1318
+ | A TV, a console or a computer that was switched on | `hosts`: it answers while it is on and is silent in standby. With `away` at `"1m"` you know within a minute or two. |
1319
+ | A printer, a speaker, a media player that came online | `services`: `_ipp._tcp` (printers), `_airplay._tcp` (Apple TV, HomePod, AirPlay TVs), `_googlecast._tcp` (Chromecast, Google speakers), `_sonos._tcp`, `_spotify-connect._tcp`, `_hap._tcp` (HomeKit accessories), `_smb._tcp` (file shares, a NAS). |
1320
+ | A device whose address you do not know | `services`, if it announces itself; or its `.local` name under `hosts`. |
1321
+
1322
+ A host is there when it answers one ping, or when it answers a connection at all, even by
1323
+ refusing it. Services are asked for on the whole network at once: on a Mac through the
1324
+ system's own Bonjour browser, elsewhere with a question to the network that every device of
1325
+ that kind answers.
1326
+
1327
+ **Who is there at the start is the starting point** and is not reported. From then on an
1328
+ arrival is reported at the next look, and a departure once the device has been silent for
1329
+ `away`. That pause is there for phones: one that sleeps answers only now and then, and
1330
+ without it a phone on the sofa would leave and arrive all evening. If a phone still flickers,
1331
+ make `away` longer. A device that says goodbye when it is switched off (announced services
1332
+ do) is reported gone at once.
1333
+
1334
+ Limits:
1335
+
1336
+ - It sees your own network and nothing else. An address on the internet is refused; whether
1337
+ one of those answers is what the `http` type's `down` and `up` are for.
1338
+ - A guest network or a router setting that keeps devices from seeing each other ("client
1339
+ isolation") hides everything from it.
1340
+ - A phone is at home when it is on the Wi-Fi, not when its owner is. One with Wi-Fi off, or
1341
+ a flat battery, has left.
1342
+ - When this computer itself has no network, nobody answering says nothing about who is at
1343
+ home: the look fails and says so, and no departures are reported.
1344
+ - On a Mac the system may ask once whether the app pine runs in may find devices on your
1345
+ local network. Without a yes nothing answers.
1346
+
1347
+ It sends one ping and three connection attempts to each host per look, and one question
1348
+ about services to the network. Nothing else is sent to any device, and nothing on a device
1349
+ is read or changed. On Linux a `.local` name needs the system to resolve such names (Avahi
1350
+ with `nss-mdns`); an IP address always works.
1351
+
1352
+ ### system
1353
+
1354
+ This computer: it reports when it goes to sleep and wakes, when its screen is locked and
1355
+ unlocked, when it changes between mains and battery or runs low, when its network comes and
1356
+ goes, when a disk is plugged in or taken out, and when an app you named starts or quits. On a
1357
+ Mac only, so far.
1358
+
1359
+ > When I unlock the Mac in the morning, tell me what happened overnight.
1360
+
1361
+ ```json
1362
+ "mac": { "type": "system", "watch": ["lock", "sleep", "power"], "apps": ["Xcode"], "battery": 15, "description": "what this Mac does: lock, sleep, power, Xcode" }
1363
+ ```
1364
+
1365
+ | Option | Default | |
1366
+ |---|---|---|
1367
+ | `watch` | all five | What to report: `"sleep"`, `"lock"`, `"power"`, `"network"`, `"volumes"`. |
1368
+ | `apps` | none | Apps whose start and end are reported, by name or bundle id: `["Xcode", "us.zoom.xos"]`. Without it no app is reported. |
1369
+ | `battery` | `20` | Report `battery-low` once when the charge falls to this many percent while on battery. |
1370
+
1371
+ | Event | | `watch` |
1372
+ |---|---|---|
1373
+ | `sleeping`, `woke` | `{}` | `sleep`: the computer goes to sleep, and is awake again. |
1374
+ | `display-off`, `display-on` | `{}` | `sleep`: the screens go dark, and light up again. |
1375
+ | `locked`, `unlocked` | `{}` | `lock`: the screen lock. |
1376
+ | `power` | `{ on, percent? }` | `power`: the supply changed. `on` is `"ac"`, `"battery"` or `"ups"`. |
1377
+ | `battery-low` | `{ percent }` | `power`: once per discharge, said again only after it was charged. |
1378
+ | `network` | `{ online, via? }` | `network`: the connection came, went, or changed its way (`"wifi"`, `"ethernet"`, `"cellular"`, `"other"`). |
1379
+ | `mounted`, `unmounted` | `{ name, path? }` | `volumes`: a disk, a memory card, a disk image, a network share. |
1380
+ | `app-started`, `app-quit` | `{ app, bundle? }` | Only for the apps on `apps`. |
1381
+
1382
+ | What it is for | How |
1383
+ |---|---|
1384
+ | A summary when you sit down | `watch: ["lock"]`, and a note: "when it is unlocked after more than an hour, say what came in". |
1385
+ | Something to do when the backup disk is plugged in | `watch: ["volumes"]`, subscribed with `when: {"payload.name": "BACKUP"}`. |
1386
+ | Finish work before the battery is empty | `watch: ["power"]`, `battery: 15`. |
1387
+ | Know that the connection dropped | `watch: ["network"]`. An agent that needs the network waits for `online: true`. |
1388
+ | Start something when a meeting begins | `apps: ["zoom.us"]`: `app-started`, and `app-quit` when it is over. |
1389
+
1390
+ How things are when the source starts (on mains or battery, online or not) is the starting
1391
+ point and is not reported; only what changes afterwards is. The charge moves all day and is no
1392
+ event: `power` is reported when the supply changes, and carries the charge it had then.
1393
+
1394
+ The system tells a small helper program, which pine builds from its own source with `swiftc`
1395
+ the first time the source starts (a few seconds, and Apple's command line developer tools have
1396
+ to be installed: `xcode-select --install`). The helper lies in the source's folder in pine's
1397
+ data folder, where no agent can write, gets none of pine's environment, and only repeats what
1398
+ the system says. It needs no permission from you: none of this is behind a privacy grant.
1399
+
1400
+ What it does not report: the name of the Wi-Fi network (macOS gives that only to an app with
1401
+ access to your location), other apps' notifications, and what you type or look at. A disk's
1402
+ name is text from outside like a file name, so these events are outside content unless you
1403
+ mark the source as your own.
1404
+
1405
+ On Linux and Windows a `system` source cannot be set up. What a command can tell there
1406
+ (`upower`, `nmcli monitor`, `udevadm monitor`) the `command` type reports.
1407
+
1408
+ ## What there is no type for
1409
+
1410
+ | Wanted | Instead |
1411
+ |---|---|
1412
+ | A page that a script fills in after loading | It has no type of its own: `http` with `render` has a browser make the look ([sources.md](sources.md), "A page that a script fills in"). Where the page's script asks an address that answers JSON, watching that with `path` is lighter. |
1413
+ | A calendar's "ten minutes before" | A schedule the agent sets when `calendar` reports the entry ([events.md](events.md), "Schedules"). |
1414
+ | Bluetooth sensors and beacons | A bridge that publishes them to MQTT (Theengs Gateway, OpenMQTTGateway), and the `mqtt` type. |
1415
+ | iMessage, WhatsApp | No type. Signal works through signal-cli and `stream`; Telegram, Slack, Matrix and Discord have types. |
1416
+ | KNX, Z-Wave, Zigbee, HomeKit, Matter | Through the hub that speaks them (Home Assistant, zigbee2mqtt, zwave-js-ui), by `mqtt` or `stream`. |
1417
+ | Something with a command line client and nothing else | `command` with `follow` ([sources.md](sources.md), "Following a command that keeps running"). |
1418
+ | Something of your own | A source type of your own is a small module ([sources.md](sources.md), "A source type of your own"); the main agent can write and register one when you ask it. |