pi-roundtable-line 0.0.0-stage → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +33 -0
- package/LICENSE +21 -0
- package/README.md +329 -2
- package/package.json +46 -5
- package/src/api.ts +265 -0
- package/src/bot.ts +64 -0
- package/src/consumer.ts +503 -0
- package/src/content.ts +210 -0
- package/src/errors.ts +43 -0
- package/src/events.ts +104 -0
- package/src/icon.ts +46 -0
- package/src/inbox.ts +760 -0
- package/src/index.ts +55 -0
- package/src/intake.ts +119 -0
- package/src/join.ts +136 -0
- package/src/messages.ts +267 -0
- package/src/migrations.ts +144 -0
- package/src/options.ts +147 -0
- package/src/outbox-worker.ts +469 -0
- package/src/outbox.ts +273 -0
- package/src/plugin.ts +221 -0
- package/src/port.ts +42 -0
- package/src/quota.ts +195 -0
- package/src/retention.ts +243 -0
- package/src/route.ts +71 -0
- package/src/sender.ts +110 -0
- package/src/sources.ts +186 -0
- package/src/surface.ts +76 -0
- package/src/token.ts +215 -0
- package/src/webhook.ts +110 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|
5
|
+
|
|
6
|
+
## [Unreleased]
|
|
7
|
+
|
|
8
|
+
## [0.10.0] - 2026-10-13
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
|
|
12
|
+
- The consumer. `LINE.subscribe(handler)` takes the one subscriber (a second is refused; the returned function detaches it) and the package's own inbox processor hands it typed events: `message` (text, image, video, audio, file, sticker, location, and `unsupported` for a type it does not know), `edited`, `unsent`, `joined` (`notice` says whether the join notice went out), `left`, `memberJoined`, `memberLeft`, and `outboxResult`. A `message` is answered with `{ refs }` (anything else fails with `bad_refs`), opaque references stored one to many in `line_message_refs` and given back on `edited` and `unsent`; a redelivery of a recorded message is not handed out again and stores nothing twice. The subscriber's third argument has `keep(ref)`, to store the reference of each post as it lands under the same lock and tombstone check as the final store (a ref kept after an unsend is handed back as `unsent` at once and `keep` resolves `{ unsent: true }`); a handler that throws after keeping some refs leaves them stored for the retry and for a later `unsend`.
|
|
13
|
+
- Tombstones. An `unsend` writes a tombstone, nulls every inbox payload of the message and deletes its metadata in one transaction under a per-message lock, then hands the refs to the subscriber and forgets them once it returns. A `message` checks the tombstone before it calls the subscriber and again wherever its refs are stored (`keep` or the final store); an unsend that lands meanwhile has the new refs handed straight back. Only a crash between a post and its `keep` can leave a post the package does not know of. Edits arrive out of order and the latest one wins.
|
|
14
|
+
- `LINE.content(messageId, { kind, signal })`: message content into memory only, capped by `media.maxBytes` (default 10 MiB) and answered as `{ bytes, contentType }`, `{ tooLarge }` or `{ expired, reason }`; a video or audio waits for `/content/transcoding` for at most `media.transcodingMaxMs` in all, requests included (each request is given what is left of the wait). It uses the content host, reissues the token once on a `401`, stops on the signal, and never fetches a message that was unsent.
|
|
15
|
+
- Reply and quote tokens are kept per message in `line_messages` for a later reply mode, never logged or handed to the subscriber, and deleted with the message.
|
|
16
|
+
- The `line-retention` service and the `retention` option: inbox events that are done, skipped, dead or never handled, and message metadata, refs and tombstones, are removed after 7 days by default; it logs counts only. A subscriber that stays detached longer than the window loses the unsends and refs that waited for it. New migration `messages` (the released `inbox` migration is untouched).
|
|
17
|
+
|
|
18
|
+
- The outbox. `LINE.outbox.enqueue(sourceKey, messages, { sender?, origin? })` queues message objects for a chat on the allowlist. `sender` (`{ name, iconUrl }`) is validated before anything is queued: a name of 1 to 20 characters without a forbidden word (`sender.forbiddenWords`, default `LINE` and `LY`, whole words in any case and width, where a change of script between Latin and other letters ends a word; LINE still has the last word), an `https` icon URL of at most 2000 characters. Objects of one chat and sender queued within `outbox.coalesceMs` (2 seconds) share a push of at most five, in order; the pushes of a chat go out in order. Every push has a UUID retry key stored when its row is created, reused on every attempt for 24 hours and never replaced (a push still unsent then is `dead`, counted from its first request even if a stop cut that request short). `2xx` and `409` are `sent`, `429`, `5xx` and no answer back off and retry (up to `outbox.maxAttempts`), any other `4xx` is `dead` with a code (`sender_name_refused`, `sender_icon_refused`, or `http_<status>`). A quota gate reads LINE's quota and consumption (cached 60 seconds), counts the recipients (1, or the member count of a group or room) and this process's reservations, and refuses a push that would pass the lower of the quota and `quota.monthlyCap`. LINE calls its consumption figure approximate, so a sent push (or one whose outcome is unknown) stays reserved until consumption has grown by its recipients or `quota.holdMs` (10 minutes) has passed; keep `monthlyCap` below the quota with headroom. The subscriber hears an `outboxResult` (`sent`, `dead`, `refused`) per `origin`, at least once. The `line-outbox` service, migration `outbox` (tables `line_outbox` and `line_notices`) and `retention.outboxMs` (7 days) are new; message objects are nulled when a push finishes.
|
|
19
|
+
- The join notice. `joinNotice: { text, sha256 }` answers a `join` with one Reply, on the event's reply token, only when the SHA-256 of the text's UTF-8 bytes equals `sha256`, in any chat the bot joins, allowlisted or not; never as a push. A malformed digest throws when the plugin is made; a digest that does not match sends nothing. `joined.notice` is now `"sent"`, `"failed"` (LINE refused the Reply, its token lapsed, or the digest did not match) or `"unconfirmed"` (no notice configured, the Reply got no answer, or the first try was cut short), and the outcome is recorded before the Reply so a redelivered join never replies twice. `digestOf(text)` is exported.
|
|
20
|
+
- `route: true` hands the text of messages of allowed chats to the host's router as messages of the `line` surface, inside the transaction that records the message and never for one unsent meanwhile (at least once: only a failed commit right after the hand-over can repeat it) (off by default; the README warns never to admit `line` through `everyone`).
|
|
21
|
+
- The `line` chat surface (channel = source key), which queues what the host writes as text pushes and passes `describeSurfaceContract`, and `senderIconRoute: { path, png }`, which serves fixed PNG bytes with a one-year cache header so a sender can name a stable `iconUrl`.
|
|
22
|
+
- The retention sweep hands strays back first: refs stored for a message that has a tombstone, more than `retention.strayMs` (default 15 minutes) ago, whose tombstone is that old too, go to the subscriber as `unsent` until they are gone. A handler that outlived its lease could keep a ref after the unsend had run and its take-back was refused, and nothing else would have handed the ref over.
|
|
23
|
+
|
|
24
|
+
### Changed
|
|
25
|
+
|
|
26
|
+
- A ref a handler kept and also returns at the end is no longer handed back a second time when the message was unsent meanwhile.
|
|
27
|
+
- **Breaking (published in 0.9.13):** `LINE.attachProcessor` is removed in favour of `LINE.subscribe`: a raw processor could read events without the tombstone rules. There is no deprecated shim. Code that attached a processor must subscribe instead; the next lockstep release is 0.10.0. `InboxProcessor` and its types stay exported for the inbox options.
|
|
28
|
+
|
|
29
|
+
## [0.9.13] - 2026-10-13
|
|
30
|
+
|
|
31
|
+
### Added
|
|
32
|
+
|
|
33
|
+
- First slice of the package: `line(options)`, the `LINE` service, a signed webhook that stores every event before it answers, an allowlist of chats, and a durable inbox worker. Stateless channel access tokens (`client_credentials` with the channel id and secret) are issued on demand, kept in memory and never logged. The inbox claims with `FOR UPDATE SKIP LOCKED` per chat, recovers expired leases, retries with backoff and sets an event aside as dead after 12 attempts. Delivery is at least once: a claim is renewed while its handler runs and fences every write that ends it, edits and unsends of one message run strictly in order and wait for an original that is backing off, and nothing of a blocked chat is handed out again. A handler is bounded by `maxHandleMs` (10 minutes: its signal is aborted and the event fails with `handler_timeout`), the worker refills free slots on a kick, a poll or a finished handler while others are still running, and what a chat the bot left still has waiting is dropped with its payload, apart from `unsend` events, which are still delivered so a message the customer took back is taken down. The processor's signal is also aborted when the worker stops or loses the event's lease, the processor is looked up after the claim, and `maxHandleMs`, `leaseMs` and `pollMs` above 2147483647 ms are refused.
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 wayne930242
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,330 @@
|
|
|
1
|
-
#
|
|
1
|
+
# pi-roundtable-line
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
A LINE Messaging API transport for [pi-roundtable][roundtable].
|
|
4
|
+
It receives the events of a LINE Official Account's group chats on the host's HTTP listener, stores each one before it answers, hands the stored events of the chats you allow to one consumer, in order and at least once, and sends pushes back to those chats under a sender name, a retry key and a monthly budget.
|
|
5
|
+
Source lives in [`packages/line`][source] in the pi-roundtable repository and releases in lockstep with the core.
|
|
6
|
+
|
|
7
|
+
[roundtable]: https://www.npmjs.com/package/pi-roundtable
|
|
8
|
+
[source]: https://github.com/wayne930242/pi-roundtable/tree/master/packages/line
|
|
9
|
+
|
|
10
|
+
It covers intake (the credentials and token source, the webhook, the allowlist of chats, and the inbox worker), the consumer side (typed events for one subscriber, tombstones for unsent messages, message content, and retention) and the sending side: an outbox with per-chat ordering and coalescing, sender names, persisted retry keys and a quota gate, the join notice, an optional router delivery, a `line` chat surface and a route that serves a sender icon.
|
|
11
|
+
|
|
12
|
+
## Requirements
|
|
13
|
+
|
|
14
|
+
- Bun 1.3 or later, and pi-roundtable `>=0.10.0 <0.11.0` as a peer dependency.
|
|
15
|
+
- PostgreSQL, which the host already needs.
|
|
16
|
+
- An HTTP listener on the host that LINE can reach over HTTPS (the webhook URL is set in the channel's settings).
|
|
17
|
+
- A Messaging API channel: its channel id and channel secret.
|
|
18
|
+
|
|
19
|
+
## Install
|
|
20
|
+
|
|
21
|
+
```sh
|
|
22
|
+
bun add pi-roundtable-line
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
```ts
|
|
26
|
+
import type { RoundtableConfig } from "pi-roundtable";
|
|
27
|
+
import { line } from "pi-roundtable-line";
|
|
28
|
+
|
|
29
|
+
export default {
|
|
30
|
+
// ...
|
|
31
|
+
http: { port: 3000, hostname: "127.0.0.1" },
|
|
32
|
+
plugins: [
|
|
33
|
+
line({
|
|
34
|
+
credentials: {
|
|
35
|
+
channelId: process.env.LINE_CHANNEL_ID ?? "",
|
|
36
|
+
channelSecret: process.env.LINE_CHANNEL_SECRET ?? "",
|
|
37
|
+
},
|
|
38
|
+
}),
|
|
39
|
+
],
|
|
40
|
+
} satisfies RoundtableConfig;
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Point the channel's webhook URL at `https://<your host>/line/webhook` and turn the webhook on.
|
|
44
|
+
The route attaches to the `public` listener; set `listener` for another, and `webhookPath` to move it.
|
|
45
|
+
|
|
46
|
+
## Credentials
|
|
47
|
+
|
|
48
|
+
The channel id and secret are all a stateless setup needs.
|
|
49
|
+
The package asks LINE for a channel access token (`POST /oauth2/v3/token`, `grant_type=client_credentials`), keeps it in memory until a minute before it lapses (about 15 minutes), asks again on demand, and asks again once when LINE answers a call with `401`.
|
|
50
|
+
Concurrent callers share one request.
|
|
51
|
+
|
|
52
|
+
```ts
|
|
53
|
+
credentials: { channelId, channelSecret, token: "stateless" } // the default
|
|
54
|
+
credentials: { channelId, channelSecret, token: { static: longLivedToken } } // a token a console issued
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
A static token cannot be replaced, so a `401` on it is final.
|
|
58
|
+
The channel secret also verifies the webhook signature.
|
|
59
|
+
|
|
60
|
+
Neither the secret nor a token is written to disk, logged, or put in an error message, and neither shows in `JSON.stringify` or `console.log` of the objects that hold them.
|
|
61
|
+
Errors carry a short code (`token_refused`, `api_unreachable`, `http_429`, ...) and LINE's request id, never a response body.
|
|
62
|
+
|
|
63
|
+
## The webhook
|
|
64
|
+
|
|
65
|
+
`POST /line/webhook` answers:
|
|
66
|
+
|
|
67
|
+
| Status | When |
|
|
68
|
+
|---|---|
|
|
69
|
+
| `200` | the events were stored (or there were none: LINE's Verify button) |
|
|
70
|
+
| `400` | the body is signed but is not a LINE events object |
|
|
71
|
+
| `401` | `x-line-signature` is missing or is not the HMAC-SHA256 of the raw body under the channel secret |
|
|
72
|
+
| `403` | the body's `destination` is not this bot (checked once the bot's user id is known) |
|
|
73
|
+
| `405` | not a `POST` |
|
|
74
|
+
| `413` | the body is over 1 MiB |
|
|
75
|
+
| `500` | the database could not store the events |
|
|
76
|
+
|
|
77
|
+
The signature is checked on the raw bytes, in constant time, before the body is parsed.
|
|
78
|
+
All events of a request are stored in one transaction, and `200` follows the commit.
|
|
79
|
+
An event with a `webhookEventId` already stored is not stored again, so LINE's redelivery (a channel setting of its own) is safe.
|
|
80
|
+
|
|
81
|
+
The bot's user id comes from `GET /v2/bot/info` when the host starts.
|
|
82
|
+
If LINE cannot be reached then, the plugin logs a warning with a code and keeps trying (at most every 30 seconds, when a webhook arrives); until it knows the id a webhook is vouched for by its signature alone.
|
|
83
|
+
|
|
84
|
+
## Chats and the allowlist
|
|
85
|
+
|
|
86
|
+
The first event from a chat the bot has not seen records it as `pending`: its key, when it was first and last seen, and counts.
|
|
87
|
+
Nothing a member wrote is stored, and the event's payload is dropped.
|
|
88
|
+
A chat is a key `line:group:<id>`, `line:room:<id>` or `line:user:<id>`.
|
|
89
|
+
|
|
90
|
+
```ts
|
|
91
|
+
import { LINE } from "pi-roundtable-line";
|
|
92
|
+
|
|
93
|
+
const { sources } = context.services.get(LINE);
|
|
94
|
+
await sources.pending(); // chats waiting for a decision
|
|
95
|
+
await sources.allow("line:group:C...");
|
|
96
|
+
await sources.block("line:group:C...");
|
|
97
|
+
await sources.summary("line:group:C..."); // adds the group's name, asked of LINE
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
| Status | Meaning |
|
|
101
|
+
|---|---|
|
|
102
|
+
| `pending` | seen, not decided; events are dropped unread |
|
|
103
|
+
| `allowed` | events are stored and handed to the consumer |
|
|
104
|
+
| `blocked` | events are dropped; `block` also drops what was waiting |
|
|
105
|
+
| `left` | the bot was removed; a `join` makes it `pending` again, because a new group needs a new decision |
|
|
106
|
+
|
|
107
|
+
The `join` and `leave` events carry no customer text and are kept for every chat that is not blocked. An `unsend` is stored only while its chat is allowed, but one that is already stored is still delivered after the bot has left the chat (see [The inbox worker](#the-inbox-worker)).
|
|
108
|
+
Allowing a chat does not recover earlier events: they were dropped unread.
|
|
109
|
+
This package never creates anything for a chat; what a chat becomes is the consumer's business.
|
|
110
|
+
|
|
111
|
+
## The inbox worker
|
|
112
|
+
|
|
113
|
+
Stored events wait in `line_inbox`.
|
|
114
|
+
The worker runs inside the host as the `line-inbox` service and hands each event to the package's own processor, which turns it into a typed event for the one subscriber (see [The consumer](#the-consumer)).
|
|
115
|
+
Until a subscriber is attached, events stay in the inbox with their payload, are not counted as handed out, and are removed by retention if they wait past its window.
|
|
116
|
+
|
|
117
|
+
- Delivery is at least once. A handler that outlives its lease (its process died, or it could not renew) may run again for the same event, so a subscriber must be idempotent.
|
|
118
|
+
- A claim is `FOR UPDATE SKIP LOCKED` under a per-chat lock: while leases are kept no chat has two events in hand at once, across workers and processes, and events of a chat go out in order of LINE's event time among those that are due. An event that is backing off does not hold the ones behind it.
|
|
119
|
+
- A claimed event has a 2-minute lease that the worker renews every 40 seconds while the handler runs, so a slow handler is not taken for a crashed one. Every write that ends a handling is fenced by its claim: a handler that lost the event cannot change what the next claim wrote. The worker puts back events whose lease ran out when it starts and every minute after, other than ones it is handling itself, and that counts as a handing out, so an event that keeps crashing its worker ends up dead.
|
|
120
|
+
- A handler runs for at most `maxHandleMs` (10 minutes). The processor (and so the subscriber) receives an `AbortSignal` as its second argument that is aborted then; the lease is no longer renewed and the event fails with `handler_timeout` like any thrown error, so a handler that never settles cannot hold its chat or the shutdown. What it settles with later is ignored. The signal is also aborted when the worker stops (`signal.reason` has the code `worker_stopped`; a handler that then throws the reason or an abort error puts the event back without counting as a failure, while any other error it throws counts) and when the worker finds the event is no longer its own (`lease_lost`). Only a handler that honours the signal gives up: one that ignores it may still be running while the same chat's next event, or the event's own retry, is handed out, so a subscriber must be safe against its own overlap. To give up, throw (for example `throw signal.reason`): a handler that returns has handled the event, and it is marked done. A subscriber's result is never read as a retry request; a `message` answer that is neither nothing nor `{ refs }` (a `{ retryInMs }`, say) fails with `bad_refs`. The subscriber is looked up again just before it is called, so one detached or replaced while the event waited gets nothing. `maxHandleMs`, `leaseMs` and `pollMs` are at most 2147483647 (about 24 days), the longest delay a timer holds.
|
|
121
|
+
- The worker looks at the inbox on a kick, on every poll and whenever a handler ends, also while handlers are running, so a slow handler holds one of the `concurrency` slots and nothing else; expired leases are put back on the same schedule.
|
|
122
|
+
- A thrown error retries with exponential backoff and jitter (5 seconds doubling to 1 hour); after 12 handings out the event is `dead`. Only the error's lower-case `code` word is stored, never its message, because messages may quote what a customer wrote.
|
|
123
|
+
- An `unsend` or `messageEdited` event runs strictly after the message it acts on and after the earlier events on that message: it waits, however long, while any of them is still pending, being handled or backing off, and goes to the processor once they are done or dead. The wait does not count against `maxRequeues`. It waits until the soonest of them is due again, and at least `dependencyRetryMs` (5 seconds). A message that has not reached the inbox at all is waited for up to 10 minutes.
|
|
124
|
+
- Only events of `allowed` chats, and the `join`, `leave` and `unsend` of a chat that is not blocked, are handed out. When a chat is blocked, what it still has waiting is dropped with its payload, and an event that was in hand at that moment is dropped instead of retried. The same goes for a chat the bot left: when its `leave` is applied, what else it has waiting is dropped with its payload, and an event of a `left` or undecided chat is dropped at the claim, so nothing surfaces after a rejoin. An `unsend` is the exception, because it carries no content and only takes a mirrored message down: it is kept and delivered even after the bot left, while `block` drops it like anything else.
|
|
125
|
+
- A finished event keeps its row and loses its payload. U+0000 and lone surrogates in an event are replaced with U+FFFD when it is stored, so one odd event never costs the others of its request.
|
|
126
|
+
|
|
127
|
+
Options tune the worker: `inbox: { leaseMs, maxHandleMs, maxAttempts, backoffBaseMs, backoffCapMs, concurrency, pollMs, ... }`.
|
|
128
|
+
|
|
129
|
+
## The consumer
|
|
130
|
+
|
|
131
|
+
```ts
|
|
132
|
+
const detach = context.services.get(LINE).subscribe(async (event, signal, line) => {
|
|
133
|
+
switch (event.type) {
|
|
134
|
+
case "message": {
|
|
135
|
+
// mirror event.body; keep the opaque reference of each post as it lands
|
|
136
|
+
await line.keep("discord-message-1");
|
|
137
|
+
await line.keep("discord-message-2");
|
|
138
|
+
return { refs: ["discord-message-1", "discord-message-2"] };
|
|
139
|
+
}
|
|
140
|
+
case "edited": // event.text, event.refs (the original's)
|
|
141
|
+
case "unsent": // event.refs: take these down; may be heard more than once
|
|
142
|
+
case "joined": // event.notice: "sent" | "unconfirmed" | "failed"
|
|
143
|
+
case "left":
|
|
144
|
+
case "memberJoined": // event.userIds
|
|
145
|
+
case "memberLeft":
|
|
146
|
+
case "outboxResult": // how a push you queued ended
|
|
147
|
+
}
|
|
148
|
+
});
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
There is exactly one subscriber: a second `subscribe` throws, and the function it returns detaches the first.
|
|
152
|
+
With none attached the inbox worker claims nothing (events wait, uncounted).
|
|
153
|
+
The processor that serves it is the package's own; there is no way to read raw inbox events through the service, so the rules below cannot be bypassed.
|
|
154
|
+
|
|
155
|
+
| Event | What it carries |
|
|
156
|
+
|---|---|
|
|
157
|
+
| `message` | `messageId`, `userId`, `quotedMessageId`, and a `body`: `text`, `image`, `video`, `audio` (with `durationMs`), `file` (`fileName`, `fileSize`), `sticker` (`packageId`, `stickerId`, `keywords`), `location` (`title`, `address`, `latitude`, `longitude`), or `unsupported` for a type this version does not know, so nothing disappears silently |
|
|
158
|
+
| `edited` | the new `text` and the `refs` of the original, in the order the subscriber gave them; groups only, and only text can be edited |
|
|
159
|
+
| `unsent` | the `refs` to take down |
|
|
160
|
+
| `joined` | `notice`: `"sent"`, `"failed"` or `"unconfirmed"`, truthfully (see [The join notice](#the-join-notice)) |
|
|
161
|
+
| `left` | |
|
|
162
|
+
| `memberJoined`, `memberLeft` | `userIds` |
|
|
163
|
+
| `outboxResult` | `sourceKey`, `originRef`, `status` (`sent`, `dead`, `refused`) and `errorCode` (see [Sending](#sending)) |
|
|
164
|
+
|
|
165
|
+
Every event but `outboxResult` has `eventId` (LINE's `webhookEventId`, the same on every redelivery), `sourceKey` and `timestamp`; a push is not a webhook event, so `outboxResult` has none of the three.
|
|
166
|
+
No event carries a reply token, a quote token or a URL.
|
|
167
|
+
|
|
168
|
+
A `message` is answered with `{ refs }`: opaque strings of at most 512 characters (at most 100 per message, kept and answered together) that the package stores one to many in `line_message_refs`, in order, and gives back on `edited` and `unsent`.
|
|
169
|
+
Answering nothing is allowed; the message is then still recorded as mirrored. Any other answer (an object without `refs`, a `{ retryInMs }`) fails the event with `bad_refs`; to give up, throw.
|
|
170
|
+
A redelivery of a message that is already recorded is not handed to the subscriber again.
|
|
171
|
+
|
|
172
|
+
One message may become several posts (a long text is cut into chunks, media and text are posted apart), and the last one can fail.
|
|
173
|
+
So the third argument of the subscriber, `line`, has `keep(ref)`: call it as each post lands.
|
|
174
|
+
It stores the reference at once, under the same per-message lock and tombstone check as the answer at the end, and resolves `{ unsent }`.
|
|
175
|
+
Refs answered at the end merge with the kept ones (one already stored keeps its place), so keeping and answering the same ref is fine; a ref that was kept is never handed back a second time because it is answered again.
|
|
176
|
+
A handler that throws after keeping some refs leaves them stored: the retry mirrors the message again, and an `unsent` hands over the posts of every attempt, so none is left behind.
|
|
177
|
+
If the sender had already taken the message back, `keep` stores the ref, hands it to the subscriber as `unsent` at once (the same take-back the final store does) and resolves `{ unsent: true }`: stop posting.
|
|
178
|
+
Keep each post as it lands, and throw to give up.
|
|
179
|
+
Only a crash between a post and its `keep` can leave a post the package does not know of; the message is then mirrored again, and the subscriber should deduplicate on `eventId` if that matters.
|
|
180
|
+
|
|
181
|
+
### Unsent messages
|
|
182
|
+
|
|
183
|
+
An `unsend` event writes a tombstone for the message, nulls the stored payload of every event that mentions the message (its own, edits of it), and deletes the message's metadata, tokens included, in one transaction under a per-message lock.
|
|
184
|
+
Only then are the stored refs handed to the subscriber as `unsent`; they are forgotten once it has returned, so a subscriber that failed halfway is handed the same refs again.
|
|
185
|
+
|
|
186
|
+
- A `message` checks the tombstone before it calls the subscriber: an unsent message is never mirrored. Refs left over from a mirror that was cut short are still handed back.
|
|
187
|
+
- The refs a subscriber keeps or returns are stored under the same lock and the tombstone is checked again there. If the sender took the message back while the subscriber was working, the new refs are stored, handed straight back as `unsent` (at the `keep` for a kept ref, at the end for the rest), and the message record is not written.
|
|
188
|
+
- An `unsend` for a message the package never saw (it arrived first, or is older than retention) still writes the tombstone and is heard with `refs: []`.
|
|
189
|
+
- An edit of a message that was unsent, was never mirrored, or is older than the last edit applied is skipped; edits arrive out of order, and the one with the latest time wins.
|
|
190
|
+
|
|
191
|
+
## Message content
|
|
192
|
+
|
|
193
|
+
```ts
|
|
194
|
+
const result = await context.services.get(LINE).content(event.messageId, {
|
|
195
|
+
kind: event.body.kind === "text" ? undefined : event.body.kind,
|
|
196
|
+
signal,
|
|
197
|
+
});
|
|
198
|
+
if ("bytes" in result) upload(result.bytes, result.contentType);
|
|
199
|
+
else if ("tooLarge" in result) placeholder("too large");
|
|
200
|
+
else placeholder(result.reason); // "expired" | "unsent" | "transcoding_failed"
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
The content is read into memory, never to disk, never logged, and never exposed at a URL.
|
|
204
|
+
`media: { maxBytes }` (default 10 MiB) caps it: a `content-length` over the cap is refused before reading, and a body without one is counted as it streams and dropped past the cap, giving `{ tooLarge: true }`.
|
|
205
|
+
LINE keeps content for a limited time; a missing one is `{ expired: true }`.
|
|
206
|
+
Pass the event's `kind` for a video or audio: the call then polls `/v2/bot/message/{id}/content/transcoding` (every `media.transcodingPollMs`, 2 seconds, until `media.transcodingMaxMs`, 60 seconds, have passed since the call began: a real deadline, so each request gets only what is left of it) until it has `succeeded`; `failed` is `{ expired: true, reason: "transcoding_failed" }` and running out of time is a `transcoding_timeout` error, which the inbox retries.
|
|
207
|
+
The call uses the same token source as every other call (a `401` reissues the token once) and the content host `api-data.line.me`.
|
|
208
|
+
Pass the handler's `signal` and an aborted event stops the download.
|
|
209
|
+
A message that was unsent is never fetched.
|
|
210
|
+
|
|
211
|
+
## Reply and quote tokens
|
|
212
|
+
|
|
213
|
+
The reply token and the quote token of a mirrored message are kept in `line_messages` (`reply_token`, `reply_token_at`, `quote_token`) for a later reply mode.
|
|
214
|
+
They are never logged and never passed to the subscriber, and they are deleted with the message when it is unsent or when retention runs.
|
|
215
|
+
|
|
216
|
+
## Retention
|
|
217
|
+
|
|
218
|
+
The `line-retention` service sweeps once at start and every hour (`retention.intervalMs`):
|
|
219
|
+
|
|
220
|
+
| What | Removed after (option) |
|
|
221
|
+
|---|---|
|
|
222
|
+
| inbox events that are `done`, `skipped` or `dead` | 7 days (`retention.inboxMs`) |
|
|
223
|
+
| inbox events still `pending`, which no subscriber handled | 7 days (`retention.inboxMs`), payload and all |
|
|
224
|
+
| message metadata and tokens, refs, tombstones | 7 days (`retention.messagesMs`) |
|
|
225
|
+
| pushes that finished, pushes that never got sent, join notice records | 7 days (`retention.outboxMs`) |
|
|
226
|
+
|
|
227
|
+
Before it trims, every sweep hands back strays: refs still stored for a message that has a tombstone, where both the ref and the tombstone are more than `retention.strayMs` (15 minutes) old, go to the subscriber as `unsent`, and are forgotten once it returns.
|
|
228
|
+
A stray happens when a handler that outlived its lease keeps a ref after its message was unsent and the take-back is refused, so nobody else would hand the ref over.
|
|
229
|
+
A take-back that fails is logged by its code and retried at the next sweep, until the ref is gone; with no subscriber attached the refs wait.
|
|
230
|
+
|
|
231
|
+
A subscriber that stays detached longer than the window loses what was waiting for it: an `unsend` still pending is removed with the rest, and the refs of the message it names are removed too, so that mirror can no longer be taken back. Keep a subscriber attached, or raise `retention.inboxMs` and `retention.messagesMs` beyond the longest outage.
|
|
232
|
+
|
|
233
|
+
It logs counts only, never ids, text or tokens, and counts as `busy` while a sweep runs so the shutdown waits for it.
|
|
234
|
+
A table that cannot be trimmed is logged by its code and the others go on.
|
|
235
|
+
|
|
236
|
+
## Sending
|
|
237
|
+
|
|
238
|
+
```ts
|
|
239
|
+
const { outbox } = context.services.get(LINE);
|
|
240
|
+
await outbox.enqueue(
|
|
241
|
+
"line:group:C...",
|
|
242
|
+
[{ type: "text", text: "On it." }],
|
|
243
|
+
{
|
|
244
|
+
sender: { name: "Ada | Support", iconUrl: "https://example.com/line/staff-icon.png" },
|
|
245
|
+
origin: "draft-42", // your own reference, given back in the outboxResult
|
|
246
|
+
},
|
|
247
|
+
);
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
`enqueue` queues LINE message objects for a chat on the allowlist; it throws before queuing anything when the sender is one LINE would refuse (`bad_sender`), an object cannot be sent (`bad_messages`: not an object with a `type`, an empty or over-long text, a `sender` of its own, over 50,000 bytes, more than 100 in one call), the key is not a source key (`bad_source_key`), or the chat is not allowed (`source_not_allowed`).
|
|
251
|
+
Message objects hold what you send, so they live in `line_outbox.messages` until the push finishes and are then set to null; the row itself goes after the retention window.
|
|
252
|
+
|
|
253
|
+
- **Pushes of at most five objects, per chat and sender.** Objects queued for the same chat and the same sender within `outbox.coalesceMs` (2 seconds, counted from the first object, not from the last) share one push, in the order they were queued; a sixth object starts the next push, and a push that holds five goes at once. Different senders never share a push, and a push is never joined if a later push of the chat exists (that would reorder the chat). The chat's pushes go out strictly in order: a push that is backing off holds the ones behind it (a dead or refused one does not).
|
|
254
|
+
- **A retry key from the start.** Each push gets a UUID the moment its row exists, before any attempt, and every attempt and every retry after a crash sends it as `X-Line-Retry-Key`. LINE keeps a key for 24 hours (`outbox.retryKeyMs`): a push still unsent then is `dead` with `retry_key_expired` and is never sent under a new key.
|
|
255
|
+
- **Outcomes.** `2xx` and `409` (LINE already has this key) are `sent`. `429`, `5xx` and no answer back off (5 seconds doubling to 1 hour, at least `Retry-After`) and retry, up to `outbox.maxAttempts` (12), then `dead` with the last code. Any other `4xx` is `dead` at once with a code: `sender_name_refused` or `sender_icon_refused` when LINE names that property of the request, else `http_<status>`; nothing else of LINE's answer is kept. A push the quota gate refuses is `refused` and is never sent. A push for a chat that stopped being allowed is `dead` with `source_not_allowed`.
|
|
256
|
+
- **Results.** The subscriber hears an `outboxResult` for each push once it has ended, once per distinct `origin` the push carries (once with `originRef: null` when none was given), at least once: a result waits while no subscriber is attached and is told again later, up to 12 times, if the subscriber throws.
|
|
257
|
+
- **Crashes.** A push in hand when the process dies has a 2-minute lease; the worker puts it back and sends it again under the same key, which LINE answers with `409` if it had taken the first request.
|
|
258
|
+
- **Service.** The `line-outbox` service runs the worker and counts as `busy` while a push is in flight, so the shutdown waits for it; a push still in flight when the host stops is put back as it was.
|
|
259
|
+
- **Nothing sensitive in logs.** The worker logs push ids, attempt numbers, status words and codes; never message objects, sender names, retry keys or tokens.
|
|
260
|
+
|
|
261
|
+
### Senders
|
|
262
|
+
|
|
263
|
+
A sender is `{ name?, iconUrl? }`, shown by LINE as "<name> from '<channel>'".
|
|
264
|
+
It is checked when you enqueue: the name is 1 to 20 characters (counted as characters, so an emoji is one), has no leading or trailing space or control character, and contains none of the forbidden words (`sender.forbiddenWords`, default `["LINE", "LY"]`, compared case-insensitively, with full-width letters read as plain ones, and as whole words, where a word is a run of Latin letters and digits or a run of letters of another script, so "Online" and "Sally" pass while "LINE" next to kana or hanzi does not); the icon is an `https` URL of at most 2000 characters without credentials.
|
|
265
|
+
LINE has the last word: a name it refuses anyway comes back as a dead push with `sender_name_refused`.
|
|
266
|
+
|
|
267
|
+
### Quota
|
|
268
|
+
|
|
269
|
+
Before a push is sent the gate reads LINE's monthly limit (`/v2/bot/message/quota`) and consumption (`/v2/bot/message/quota/consumption`), cached for 60 seconds, and counts the push's recipients: 1 for a user chat, the member count (`/members/count`, cached for 60 seconds) for a group or room.
|
|
270
|
+
The push goes only if consumption, plus what this process has already reserved for pushes it sent or is sending, plus its own recipients, stays within the lower of the plan's quota and `quota.monthlyCap`; otherwise it is `refused` with `quota_exhausted`.
|
|
271
|
+
A reservation for a push that went out stays counted until the consumption LINE reports has grown by at least the push's recipients since it was sent, or until `quota.holdMs` (10 minutes by default) has passed. A push whose outcome is unknown (a timeout, no answer, or a `5xx`) is held as if it had been sent; one LINE refused with a `4xx` or `429` is dropped at once. When the quota cannot be read the push backs off and retries; it is not refused and not sent blind.
|
|
272
|
+
This rests on an assumption: LINE calls its consumption figure approximate, and the gate treats it as approximate. It may count late, so keep `monthlyCap` below the plan's quota with some headroom, and keep it lower still when other processes, or the LINE Official Account Manager, send too. Only this process's reservations are known, and they are lost on a restart.
|
|
273
|
+
|
|
274
|
+
```ts
|
|
275
|
+
line({ credentials, quota: { monthlyCap: 400 } })
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
## The join notice
|
|
279
|
+
|
|
280
|
+
```ts
|
|
281
|
+
line({
|
|
282
|
+
credentials,
|
|
283
|
+
joinNotice: { text: NOTICE_TEXT, sha256: NOTICE_SHA256 },
|
|
284
|
+
})
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
When the bot joins a chat the package answers the `join` event with **one Reply**, using the event's reply token, if and only if the SHA-256 of the text's UTF-8 bytes (lower-case hex) equals `joinNotice.sha256`.
|
|
288
|
+
`digestOf(text)` computes it. A digest that is not 64 lower-case hex digits, or a text over 5000 characters, throws when the plugin is made; a well-formed digest that does not match the text does not stop the host, but nothing is ever sent and every join is `failed` (and a warning is logged once at start).
|
|
289
|
+
Without `joinNotice` nothing is ever sent.
|
|
290
|
+
|
|
291
|
+
- It applies to any chat the bot joins that is not blocked, allowlisted or not.
|
|
292
|
+
- It is never a push. When LINE refuses the Reply, or its token has lapsed (reply tokens live a short time), the notice is `failed` and nothing is sent in its place. When the Reply gets no answer (a timeout or an unreachable LINE) nobody knows whether LINE took it, so the notice is `unconfirmed` and is not tried again.
|
|
293
|
+
- The subscriber's `joined` event says what happened: `"sent"`, `"failed"`, or `"unconfirmed"` (no notice configured, or the Reply got no answer, or the first try was cut short, and nobody knows whether LINE got the Reply). The outcome is recorded before the Reply is made, so a join that is handed out again never replies twice.
|
|
294
|
+
- The inbox worker only runs while a subscriber is attached (or `route` is on), so attach one before the bot can be added to chats; a join that waits past its reply token's life is `failed`.
|
|
295
|
+
|
|
296
|
+
## Router delivery
|
|
297
|
+
|
|
298
|
+
`route: true` hands the text of messages of allowed chats to the host's router, as messages of the `line` surface (channel `line:group:<id>`, author the LINE user id, `mentionsBot` when the bot is @mentioned, with the bot's mention cut out of the text), after the subscriber has mirrored them, and never for a message that was unsent meanwhile.
|
|
299
|
+
The hand-over happens in the same transaction that records the message, so a router that throws makes the event retry and a failed store hands over nothing. Delivery is at least once: only a failure of the commit itself, right after the router took the text, can hand the same text over a second time.
|
|
300
|
+
Images, stickers and other kinds are not routed, nor are messages LINE did not name a sender for.
|
|
301
|
+
It is off by default.
|
|
302
|
+
**Never admit `line` through the core's `everyone` access**: anyone who can add the bot to a group, or write in a chat you allow, would reach the assistant. Admit only identities you have linked to a person.
|
|
303
|
+
With `route` on the inbox is served even when no subscriber is attached.
|
|
304
|
+
|
|
305
|
+
## The `line` chat surface
|
|
306
|
+
|
|
307
|
+
The plugin contributes a chat surface under the prefix `line`; its channel is a source key (`line:group:<id>`, `line:room:<id>`, `line:user:<id>`).
|
|
308
|
+
What the host writes there (`sendReply`) is queued as text objects, one per chunk (a chunk over 5000 characters is cut, never inside a surrogate pair), and goes out as a push with no sender; it reaches only a chat on the allowlist.
|
|
309
|
+
The surface shows no typing indicator, stop control, progress, files or approval prompts: LINE has none the bot can show.
|
|
310
|
+
It passes `describeSurfaceContract` (`pi-roundtable/testing`).
|
|
311
|
+
|
|
312
|
+
## The sender icon route
|
|
313
|
+
|
|
314
|
+
```ts
|
|
315
|
+
line({ credentials, senderIconRoute: { path: "/line/staff-icon.png", png: readFileSync("staff-icon.png") } })
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
Serves fixed PNG bytes at `path` to `GET` and `HEAD` with `cache-control: public, max-age=31536000, immutable`, on the same listener as the webhook, so the host can give LINE a stable `iconUrl` (`https://<your host>/line/staff-icon.png`) without another service.
|
|
319
|
+
The bytes are copied when the plugin is made and checked: they must be a PNG of at most 1 MiB, and `path` must differ from `webhookPath`.
|
|
320
|
+
|
|
321
|
+
## Safety
|
|
322
|
+
|
|
323
|
+
Do not admit a `line` surface through the core's `everyone` access: anyone who can add the bot to a group would reach the assistant (see [Router delivery](#router-delivery)).
|
|
324
|
+
A push goes only to a chat on the allowlist, under a sender that was checked before it was queued, inside the month's budget.
|
|
325
|
+
The access token, the channel secret, reply and quote tokens, retry keys and message text appear in no log line and no error.
|
|
326
|
+
|
|
327
|
+
## Tests
|
|
328
|
+
|
|
329
|
+
PostgreSQL-backed tests run when `ROUNDTABLE_TEST_DATABASE_URL` is set and are skipped otherwise.
|
|
330
|
+
No test reaches LINE: the plugin takes a `fetch` option, and the tests inject a fake.
|
package/package.json
CHANGED
|
@@ -1,6 +1,47 @@
|
|
|
1
1
|
{
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
2
|
+
"name": "pi-roundtable-line",
|
|
3
|
+
"version": "0.10.0",
|
|
4
|
+
"description": "A LINE Messaging API transport for pi-roundtable: a signed webhook that persists events before it answers, a durable inbox worker, and an allowlist of the groups and chats it mirrors",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"license": "MIT",
|
|
7
|
+
"engines": {
|
|
8
|
+
"bun": ">=1.3.0"
|
|
9
|
+
},
|
|
10
|
+
"exports": {
|
|
11
|
+
".": "./src/index.ts"
|
|
12
|
+
},
|
|
13
|
+
"files": [
|
|
14
|
+
"src",
|
|
15
|
+
"!src/**/*.test.ts",
|
|
16
|
+
"!src/testing",
|
|
17
|
+
"LICENSE",
|
|
18
|
+
"README.md",
|
|
19
|
+
"CHANGELOG.md"
|
|
20
|
+
],
|
|
21
|
+
"repository": {
|
|
22
|
+
"type": "git",
|
|
23
|
+
"url": "git+https://github.com/wayne930242/pi-roundtable.git",
|
|
24
|
+
"directory": "packages/line"
|
|
25
|
+
},
|
|
26
|
+
"publishConfig": {
|
|
27
|
+
"access": "public",
|
|
28
|
+
"provenance": true
|
|
29
|
+
},
|
|
30
|
+
"scripts": {
|
|
31
|
+
"typecheck": "tsc --noEmit",
|
|
32
|
+
"lint": "biome check .",
|
|
33
|
+
"test": "bun test"
|
|
34
|
+
},
|
|
35
|
+
"peerDependencies": {
|
|
36
|
+
"pi-roundtable": ">=0.10.0 <0.11.0"
|
|
37
|
+
},
|
|
38
|
+
"devDependencies": {
|
|
39
|
+
"@biomejs/biome": "2.5.15",
|
|
40
|
+
"@earendil-works/pi-ai": ">=1.0.0 <2",
|
|
41
|
+
"@earendil-works/pi-coding-agent": ">=1.0.0 <2",
|
|
42
|
+
"@types/bun": "1.4.2",
|
|
43
|
+
"pi-roundtable": "0.10.0",
|
|
44
|
+
"typebox": "1.3.34",
|
|
45
|
+
"typescript": "7.0.2"
|
|
46
|
+
}
|
|
47
|
+
}
|