@framers/agentos-ext-meeting-browser 0.1.11 → 0.1.12

package/README.md

@@ -131,7 +131,7 @@ The unread callback is optional on the internal channel peer and cannot fail val
 - **The sandbox.** Chromium runs without its own sandbox, which needs user namespaces that the container's dropped capabilities and seccomp profile refuse; the container is the boundary. `image/seccomp.json` is Docker's default seccomp profile, unchanged (`seccomp/default.json` of moby/profiles at `6fe7deb1`); the scheduler sends it with each container when `MEETING_SECCOMP_PROFILE` names it.
 - **Logs.** The bot writes JSON lines through pino, each with the run's id: events, phases, reasons, codes and counts, never a link, a name or a chat line.
 
-**The adapters.** `image/src/adapters/google-meet.ts` and `microsoft-teams.ts` each hold a selector table and a `selectorsVersion`, the date its selectors were last read against the live client, and the hosts the client may load. Every selector is one Playwright engine: CSS, a comma list included, one `text=` selector, or one `role=` selector, an element by its role and its accessible name, which survives a suffix on a label and a wrapper the label sits on. Their checks read what is visible, a read waits two seconds at most for its element, and the chat is read only while its panel is open. The Meet chat is read as open-source Meet clients read it in 2026, after the nightly check's first Meet run (24 October 2026) found no textarea to type the notice into: the toolbar's Chat key is a button named "Chat with everyone", and the composer is a `<textarea>` labelled "Send a message" in a meeting whose chat is Meet's own, or Google Chat's editable in a meeting whose chat is a Google Chat conversation (Workspace's Chat in Meet: the chat of a meeting scheduled in Calendar, for the host's own domain), a `contenteditable` div with the textbox role labelled "History is on", which one reading of April 2026 found under a wrapper that carries the "Send a message" label instead. A Google Chat conversation's panel draws that editable, its Send key and its messages inside a frame the panel embeds, on chat.google.com, where a page-level locator never looks (Playwright's `page.locator` is the main frame's): the nightly check's Meet run of 25 October 2026 opened the panel on its key and found no composer by any lookup (`chat_input_missing`). The adapter looks for the composer in every frame of the page, the main frame first and the frames read anew on each poll since the chat's frame attaches after the panel opens, by role and name first, then as the textarea, then as an editable by its shape, in any language, and, while the Chat key says its panel is open, as a textarea, a textbox or an editable inside the side panel (an `aside`, or the complementary role), whatever its label, which is how the nightly check's Meet run of 27 October 2026 found the composer: drawn disabled and aria-disabled under another label, which the outline reads as the meeting's chat being off for the bot, a signed-in guest from another organisation (an inference from the outline, not something a Google page states); a composer the client draws disabled, read-only or aria-disabled is named `chat_input_blocked` at once, by Playwright's read of its state (`isEditable`, whose actionability page, https://playwright.dev/docs/actionability, names the `disabled` attribute, `aria-disabled="true"` on an element above, `readonly` and `aria-readonly`; Playwright 1.63.0, which the image pins, reads `aria-disabled="true"` on the composer itself as well, which the lifecycle test pins), before any text is typed, with the panel's outline and `tried=state:<lookup>`. The key's word is its `aria-pressed` or `aria-expanded` (that run read `expanded:true` once the panel had opened): open when either reads true, closed when either reads false, and a key that marks neither is trusted once the panel opened on it in the page; so a field of another side panel never stands for the composer while the chat panel is closed, a composer found by the panel lookup alone is read again on every later post (the farewell, a message the manager commands) and in the observer's reads, and a key that says its panel is open is never pressed, since the key toggles the panel and a press would close it. Each lookup reads its first visible match, never its first match, since the client keeps screens it is not showing in the page; it closes a tip laid over the call ("Got it", "Dismiss", "Not now", "No thanks") before it presses the Chat key, opens the panel with Meet's shortcut (Ctrl+Alt+c) when the key is missing or will not take the click, types with Playwright's `fill`, sends with Enter and presses the Send key in the composer's frame once when Enter left the text there, and reads its own message back as a new message among those its observer reads in that frame: an element carrying `data-message-id` (in a Google Chat conversation the message's name, `spaces/…/messages/…`; the Pin key beside the message carries the same id and is never read as one) or an element with its sender's name. The observer reads the panel's messages from the frame the composer is in, so a stop phrase typed in a Google Chat conversation is heard. Every step waits ten seconds at most (five after the shortcut) and names itself when it fails, with the panel's outline beside the name. The Teams chat is read as open-source Teams clients read it: the composer is CKEditor's editable, which some builds draw inside `[data-tid="ckeditor"]` and others as that element itself, and the adapter types into either; a message is its card's body (`[data-tid="messageBodyContent"]`), apart from the author's name and the time the card also shows. A message whose element holds no body element, in either client, is read from the whole element and marked so (`bodyRead: false`). `fixture.ts` runs the adapter of the platform the link's `platform` parameter names against the test fixture page, whose markup carries that platform's selectors; the bot takes it only for a job marked `adapter: 'fixture'`, and its network policy admits the fixture's own host alone, over plain `http` too.
+**The adapters.** `image/src/adapters/google-meet.ts` and `microsoft-teams.ts` each hold a selector table and a `selectorsVersion`, the date its selectors were last read against the live client, and the hosts the client may load. Every selector is one Playwright engine: CSS, a comma list included, one `text=` selector, or one `role=` selector, an element by its role and its accessible name, which survives a suffix on a label and a wrapper the label sits on. Their checks read what is visible, a read waits two seconds at most for its element, and the chat is read only while its panel is open. The Meet chat is read as open-source Meet clients read it in 2026, after the nightly check's first Meet run (24 October 2026) found no textarea to type the notice into: the toolbar's Chat key is a button named "Chat with everyone", and the composer is a `<textarea>` labelled "Send a message" in a meeting whose chat is Meet's own, or Google Chat's editable in a meeting whose chat is a Google Chat conversation (Workspace's Chat in Meet: the chat of a meeting scheduled in Calendar, for the host's own domain), a `contenteditable` div with the textbox role labelled "History is on", which one reading of April 2026 found under a wrapper that carries the "Send a message" label instead. A Google Chat conversation's panel draws that editable, its Send key and its messages inside a frame the panel embeds, on chat.google.com, where a page-level locator never looks (Playwright's `page.locator` is the main frame's): the nightly check's Meet run of 25 October 2026 opened the panel on its key and found no composer by any lookup (`chat_input_missing`). The adapter looks for the composer in every frame of the page, the main frame first and the frames read anew on each poll since the chat's frame attaches after the panel opens, by role and name first, then as the textarea, then as an editable by its shape, in any language, and, while the Chat key says its panel is open, as a textarea, a textbox or an editable inside the side panel (an `aside`, or the complementary role), whatever its label, which is how the nightly check's Meet run of 27 October 2026 found the composer: drawn disabled and aria-disabled under another label, which the outline reads as the meeting's chat being off for the bot, a signed-in guest from another organisation (an inference from the outline, not something a Google page states); a composer the client draws disabled, read-only or aria-disabled is named `chat_input_blocked` at once, by Playwright's read of its state (`isEditable`, whose actionability page, https://playwright.dev/docs/actionability, names the `disabled` attribute, `aria-disabled="true"` on an element above, `readonly` and `aria-readonly`; Playwright 1.63.0, which the image pins, reads `aria-disabled="true"` on the composer itself as well, which the lifecycle test pins), before any text is typed, with the panel's outline and `tried=state:<lookup>`. The key's word is its `aria-pressed` or `aria-expanded` (that run read `expanded:true` once the panel had opened): open when either reads true, closed when either reads false, and a key that marks neither is trusted once the panel opened on it in the page; so a field of another side panel never stands for the composer while the chat panel is closed, a composer found by the panel lookup alone is read again on every later post (the farewell, a message the manager commands) and in the observer's reads, and a key that says its panel is open is never pressed, since the key toggles the panel and a press would close it. Each lookup reads its first visible match, never its first match, since the client keeps screens it is not showing in the page; it closes a tip laid over the call ("Got it", "Dismiss", "Not now", "No thanks") before it presses the Chat key, opens the panel with Meet's shortcut (Ctrl+Alt+c) when the key is missing or will not take the click, types with Playwright's `fill`, sends with Enter and presses the Send key in the composer's frame once when Enter left the text there, and reads its own message back as a new message among those its observer reads in that frame: an element carrying `data-message-id` (in a Google Chat conversation the message's name, `spaces/…/messages/…`; the Pin key beside the message carries the same id and is never read as one) or an element with its sender's name. The observer reads the panel's messages from the frame the composer is in, so a stop phrase typed in a Google Chat conversation is heard. Every step waits ten seconds at most (five after the shortcut) and names itself when it fails, with the panel's outline beside the name. The Teams adapter reads the client Microsoft serves an anonymous guest as it drew its pages on 30 October 2026 (`teams.microsoft.com/light-meetings`, after the launcher's "Continue on this browser"), after the nightly check's Teams runs of 29 and 30 October 2026 ended `join_control_missing` with requests to `teams.public.onecdn.static.microsoft` blocked: the launcher and that client load their scripts from Microsoft's static CDN, so the hosts the client may load include the ones Microsoft's list of Microsoft 365 URLs and IP address ranges (https://learn.microsoft.com/en-us/microsoft-365/enterprise/urls-and-ip-address-ranges) names as required, `*.static.microsoft` and `*.usercontent.microsoft` (ID 193) and `*.lync.com` among the Teams hosts (ID 12). The name field is read by its `data-tid`, since its id is generated, and Join now is pressed once the name is in it. The lobby is the screen under `[data-tid="calling-lobby-screen"]` or the lobby's words ("Someone will let you in when the meeting starts." before the meeting has started). The call draws no calling stage: its Leave key, `button#hangup-button` named by its title alone, is drawn in the call and not in the lobby, so it stands for the call and is the key the adapter presses to leave. The waits for the lobby and the call read the first shown match. The People key is the roster key (`button#roster-button`, or a key named People and nothing else), read at its first shown match, since a client may keep a hidden copy of it in the page and draws keys that name people without being it (Invite people). It reads "2 People" with one other person in the call and "People" while the bot is alone, so a key that reads "People" alone counts no one besides the bot, and other words with no number, or a key not read, are no count. The composer is CKEditor's editable, which that client draws as `[data-tid="ckeditor"]` itself and other builds inside it, and the adapter types into either; a message's element (`[data-tid="chat-pane-message"]`, the message's id in its own id) holds the text in `div#content-<id>`, and the author's name and the time sit beside it in the message's item (`[data-tid="chat-pane-item"]`), from which the adapter reads the author. A message whose element holds no body element, in either client, is read from the whole element and marked so (`bodyRead: false`). `fixture.ts` runs the adapter of the platform the link's `platform` parameter names against the test fixture page, whose markup carries that platform's selectors; the bot takes it only for a job marked `adapter: 'fixture'`, and its network policy admits the fixture's own host alone, over plain `http` too.
 
 ## Tests
 
@@ -139,7 +139,7 @@ The unread callback is optional on the internal channel peer and cannot fail val
 
 The diagnostic cases also cover every exact refusal body and malformed or unknown bodies, both output streams, fixed error metadata, a real HTTP/WebSocket channel's invalid JSON and schema failures, the warning's run id, one allowance across reconnect and persisted adoption, a new run and replacement lease, valid reports and command acknowledgements, and a throwing diagnostic logger. Driver tests assert that an unset policy omits LogConfig, an explicit journal selection changes only that option, and unsupported values produce a fixed error. The real executable entry is imported with filesystem, Docker and server leaves substituted to prove environment-to-policy wiring. CI typechecks these fixtures separately, checks the built scheduler policy and public refusal dependency, and retains the existing package and browser lifecycle suites. Host journal persistence and post-removal retention need separate deployment evidence. `test/dispatch-refusals.spec.ts`, `test/channel-diagnostics.spec.ts`, `test/log-driver.spec.ts` and `test/scheduler-entry.spec.ts` hold these cases, and CI runs them with the seven above. The refusal and driver files also start a scheduler on an ephemeral port with the fake Docker: the provider reads the scheduler's own answers when it is paused, when its slots are full, when a live run holds its one signed-in identity, when its identity pool cannot be read, and when the pool is empty or holds only signed-out or unprovisioned folders, a run started under the journal policy gets that `LogConfig` and one started without it gets none, and an unsupported setting's error message is the fixed word alone.
 
-**The lifecycle test.** `image/fixture/` is a meeting page that carries the Google Meet adapter's selectors: a pre-join screen (the name field, the camera and microphone toggles, Ask to join), a waiting screen, the call's tiles, the participant count, the chat panel and the Leave key, and an `<audio>` element that plays a known recording into the page's output, which is the call's audio. Its Meet chat is drawn as open-source Meet clients read it: the messages are the items of a list labelled "In-call messages", each with `data-message-id`, and the composer is the textarea labelled "Send a message" (the default), Google Chat's editable, a textbox div editable by a bare `contenteditable` attribute and labelled "History is on" (`chat=history`), that editable under a wrapper that carries the "Send a message to everyone" label with no role or label of its own (`chat=nested`), or a frame the panel embeds (`chat=frame`, the fixture's `chat-frame.html` at `/chat-frame`) that holds that editable, its Send key and the messages, with none of them in the page, as a Google Chat conversation's panel embeds its frame from chat.google.com; with `chat=none` the panel holds the messages and no composer at all, and with `chat=off` it holds the textarea drawn disabled and aria-disabled under another label, in a side panel that is an `aside`, as Meet drew it on 27 October 2026 for a participant the outline reads as having its chat off. With `chat=aria-disabled` it holds that textarea under the same label marked `aria-disabled="true"` on itself alone, with no `disabled` attribute, so the page still takes text into it, and with `chat=relabelled` that textarea enabled under a label of its own, which no lookup by name reads either. With `platform=microsoft_teams` in its link it carries the Microsoft Teams adapter's selectors instead, and draws the Teams chat as open-source Teams clients read it: a message card (`[data-tid="chat-pane-message"]`) holds its author, its time and its body (`[data-tid="messageBodyContent"]`), and the composer is CKEditor's editable inside `[data-tid="ckeditor"]`, or that element itself with `composer=self`, the two shapes Teams builds show. Its server, `serve.mjs` on port 8787, queues scripted events from `POST /control` (`admit`, `late_joiner`, `chat` with a sender and a text, `noId` for a Meet message drawn without `data-message-id` and `noBody` for a Teams card drawn without its body element, `hide_chat`, `show_chat`, `leave_all`) for the page to poll, and keeps what the page did on `GET /state`: its knocks, the texts the bot posted, its presses of Leave. The recording is spoken from `recording.txt` by espeak-ng when the fixture's image is built: about twenty seconds of 16 kHz mono speech. `image/test/compose.yml` runs LiveKit 1.13.7 in development mode (its documented key pair, `devkey` and `secret`), the fixture, a fake product that records every report with a hash of the token it came under, and the scheduler on this pack's build, which starts real bot containers from `agentos-meeting-bot:ci` on the `meeting-bots` network under its fixed container policy, the seccomp profile included. `image/test/lifecycle.mjs` sends twelve runs through the scheduler's API and listens to each run's room with `@livekit/rtc-node`:
+**The lifecycle test.** `image/fixture/` is a meeting page that carries the Google Meet adapter's selectors: a pre-join screen (the name field, the camera and microphone toggles, Ask to join), a waiting screen, the call's tiles, the participant count, the chat panel and the Leave key, and an `<audio>` element that plays a known recording into the page's output, which is the call's audio. Its Meet chat is drawn as open-source Meet clients read it: the messages are the items of a list labelled "In-call messages", each with `data-message-id`, and the composer is the textarea labelled "Send a message" (the default), Google Chat's editable, a textbox div editable by a bare `contenteditable` attribute and labelled "History is on" (`chat=history`), that editable under a wrapper that carries the "Send a message to everyone" label with no role or label of its own (`chat=nested`), or a frame the panel embeds (`chat=frame`, the fixture's `chat-frame.html` at `/chat-frame`) that holds that editable, its Send key and the messages, with none of them in the page, as a Google Chat conversation's panel embeds its frame from chat.google.com; with `chat=none` the panel holds the messages and no composer at all, and with `chat=off` it holds the textarea drawn disabled and aria-disabled under another label, in a side panel that is an `aside`, as Meet drew it on 27 October 2026 for a participant the outline reads as having its chat off. With `chat=aria-disabled` it holds that textarea under the same label marked `aria-disabled="true"` on itself alone, with no `disabled` attribute, so the page still takes text into it, and with `chat=relabelled` that textarea enabled under a label of its own, which no lookup by name reads either. With `platform=microsoft_teams` in its link it carries the Microsoft Teams adapter's selectors instead, drawn as the client Microsoft served an anonymous guest on 30 October 2026: the name field known by its `data-tid` with a generated id and Join now disabled until a name is typed, the lobby under `[data-tid="calling-lobby-screen"]` with the lobby's words, a call with no calling stage whose Leave key is `button#hangup-button` named by its title, a People key that reads "People" while the bot is alone and "2 People" with one other person, after a hidden copy of it and an Invite people key, and a message as an item (`[data-tid="chat-pane-item"]`) that holds the author's name and the time beside the message's element (`[data-tid="chat-pane-message"]`), whose text is in `div#content-<id>`; the composer is CKEditor's editable inside `[data-tid="ckeditor"]`, or that element itself with `composer=self`, the shape that client draws. Its server, `serve.mjs` on port 8787, queues scripted events from `POST /control` (`admit`, `late_joiner`, `chat` with a sender and a text, `noId` for a Meet message drawn without `data-message-id` and `noBody` for a Teams message drawn without its body element, `hide_chat`, `show_chat`, `leave_all`) for the page to poll, and keeps what the page did on `GET /state`: its knocks, the texts the bot posted, its presses of Leave. The recording is spoken from `recording.txt` by espeak-ng when the fixture's image is built: about twenty seconds of 16 kHz mono speech. `image/test/compose.yml` runs LiveKit 1.13.7 in development mode (its documented key pair, `devkey` and `secret`), the fixture, a fake product that records every report with a hash of the token it came under, and the scheduler on this pack's build, which starts real bot containers from `agentos-meeting-bot:ci` on the `meeting-bots` network under its fixed container policy, the seccomp profile included. `image/test/lifecycle.mjs` sends twelve runs through the scheduler's API and listens to each run's room with `@livekit/rtc-node`:
 
 - The first run joins, waits in the waiting room, is admitted, posts the notice, reads it back and reports a heartbeat within 12 seconds of it. No audio frame reaches the room before `startCapture`.
 - After `startCapture` the recording is heard in the room: above 0.01 RMS in at least three of five seconds, the first frame after `notice_posted`.
@@ -148,7 +148,7 @@ The diagnostic cases also cover every exact refusal body and malformed or unknow
 - Another bot's notice in the chat, which names the stop phrase, is reported as `chat_message`, and no `stop_command` or `left` follows in five seconds. A participant's stop phrase is reported as `chat_message` and `stop_command`. The bot posts "Leaving now; everything I took is being deleted.", presses Leave, and reports `left` (`stopped_by_participant`) and `done`. No audio reaches the room after `left`, and the container is removed.
 - A second run, admitted and confirmed but never sent `startCapture`, publishes no track for 15 seconds, then leaves on the `leave` command with its reason.
 - A third run, sent two `startCapture` commands at once, publishes one track. The same short message sent twice by messages without a `data-message-id` gives two `chat_message` reports, and the stop phrase from a participant whose name is You is obeyed.
-- A Microsoft Teams run with the chat's editable inside `[data-tid="ckeditor"]`, and another with that element itself as the editable, each post the notice and read it back, report the same short message sent twice as two `chat_message` reports, report each message as its body with its author, and leave on the stop phrase, which the second run sends in a card drawn without its body element.
+- A Microsoft Teams run with the chat's editable inside `[data-tid="ckeditor"]`, and another with that element itself as the editable, each join through the name field by its `data-tid`, wait in the lobby, count the one other person on the People key, past a hidden copy of it and an Invite people key, once admitted to the call its Leave key stands for, post the notice and read it back, report the same short message sent twice as two `chat_message` reports, report each message as its body with its author read from the message's item, and leave on the stop phrase by `button#hangup-button`, which the second run sends in a message drawn without its body element.
 - A Google Meet run whose chat panel holds Google Chat's editable, a textbox named "History is on", and another whose panel holds that editable under a wrapper carrying the "Send a message" label, neither with a textarea, each post the notice and read it back, report a participant's message as typed, and leave on the stop phrase with the farewell posted through the same composer. Before the adapter read these shapes, each ended as the nightly check's Meet run of 24 October 2026 did: `notice_failed`, thirty seconds after `in_call`.
 - A Google Meet run whose chat's composer, Send key and messages are inside a frame the panel embeds posts the notice through the frame's editable and reads it back from the frame, reports a participant's message drawn in the frame as typed, and leaves on the stop phrase typed there. Before the adapter looked in the page's frames, it ended as the nightly check's Meet run of 25 October 2026 did: `notice_failed` with `notice: chat_input_missing`, ten seconds after `in_call`.
 - A Google Meet run whose panel holds no composer at all reports `notice: chat_input_missing` with the panel's outline after it (the fixture's host, the counts of controls by role and of the message drawn before the panel opened, the Chat key pressed, the pre-join name field as the one composer-shaped control, undrawn under its hidden section, and the lookups tried), in the record line's alphabet and quoting neither that message, its sender nor a tile's name, and leaves with `notice_failed`.

package/manifest.json

@@ -2,7 +2,7 @@
   "$schema": "https://agentos.sh/schemas/extension-manifest-v1.json",
   "id": "com.framers.meetings.browser",
   "name": "Browser Meeting Bot Provider",
-  "version": "0.1.11",
+  "version": "0.1.12",
   "description": "A meeting-participant provider that joins Google Meet and Microsoft Teams through a browser in a container, posts a notice, publishes the call's audio into a LiveKit room, and leaves on any participant's stop command.",
   "categories": ["meetings", "voice"],
   "platforms": ["google_meet", "microsoft_teams"],

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@framers/agentos-ext-meeting-browser",
-  "version": "0.1.11",
+  "version": "0.1.12",
   "description": "Browser-join meeting bot provider for AgentOS: Google Meet and Microsoft Teams through Chromium in a container, audio into a LiveKit room",
   "type": "module",
   "main": "dist/index.js",