@framers/agentos-ext-meeting-browser 0.1.8 → 0.1.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -5
- package/manifest.json +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -114,7 +114,7 @@ TLS belongs to the reverse proxy in front of the scheduler. Every route but `GET
|
|
|
114
114
|
`image/` builds the container the scheduler starts for each run, from this folder (`docker build -f image/Dockerfile .`), since the bot shares `src/protocol.ts` with the scheduler: Debian 12 with Node 24, Chromium through Playwright 1.63.0, PulseAudio, Xvfb, `@livekit/rtc-node`, and the bot program in `image/src/`. One container is one run.
|
|
115
115
|
|
|
116
116
|
- **Start.** `entrypoint.sh` starts PulseAudio on `image/pulse/default.pa` alone: a null sink `meeting`, the browser's output, whose monitor the bot publishes, and a null sink `silence`, whose monitor is the browser's microphone. The root file system is read-only, so PulseAudio's runtime and state folders and its cookie are made under the `/run/pulse` tmpfs as the bot user, and Chromium's config folder (`XDG_CONFIG_HOME`), where its crash handler keeps its database and without which the browser does not start, is `/tmp/.config` in the `/tmp` tmpfs. Then Xvfb on display `:99`, then `bot.js`, which reads the run's token from stdin and opens the command channel at `MEETING_CHANNEL_URL` with it.
|
|
117
|
-
- **The run.** The job is the channel's first message. The bot joins the run's LiveKit room without publishing, launches Chromium on the identity's profile at `/home/bot/profile` when the run has one (a Google Meet identity from the pool) or on a fresh profile, and drives the platform's adapter: it reports `joining`, `waiting_room` and `in_call`, posts the notice, reads it back from the chat panel and reports `notice_posted`, then reads the panel every second. Each step of the posting is bounded and named, so a night that fails there reports the step rather than a bare timeout: `notice: chat_button_missing` (no Chat key was shown and the keyboard shortcut opened no panel), `chat_button_blocked` (the key would not take the click, a dialog over it, and the shortcut opened no panel), `chat_input_missing` (the panel opened without a composer-shaped control in any frame), `chat_input_blocked` (the composer is drawn disabled, read-only or aria-disabled, the meeting's chat off for the bot, or would not take the text), `chat_enter_blocked` (it would not take the Enter) or `chat_echo_missing` (the message was typed and sent and never read back). The message is read back as a new message among those the chat observer reads, so a line the panel already held, the bot's own notice after a rejoin among them, never stands for the one being sent. A failed notice step's report carries, after its code, an outline of the chat panel (`image/src/adapters/chat-outline.ts`): for the main frame and each frame the page embeds, the frame's host and its counts of drawn controls by role (textbox, combobox, button, menuitem), of editables, of list items and of elements carrying a message id, the accessible names of the composer-shaped controls (and of the buttons in an embedded frame, whose controls are the chat's own; a menu item is counted and never named, since no lookup targets one and a menu may list people), each cut to forty characters of letters, digits and a few marks and at most twelve a frame after the filter below, the Chat key's state, the composer-shaped controls by their fixed facts, and the lookups the step tried, for example `notice: chat_input_missing; panel: frames=2; f1 meet.google.com textbox=0 combobox=1 button=23 menuitem=0 editable=0 listitem=0 messageid=0 key=pressed:true,expanded:none; f2 chat.google.com textbox=1 combobox=0 button=6 menuitem=2 editable=1 listitem=3 messageid=3 cands=div,textbox,ce:1,dis:0,ro:0,ad:0,drawn:1,box:1,vis:visible,ah:0,name:aria,lbl:history,chat:0,land:none names=History is on|Send message; tried=composer:role,textarea,editable`. `key=` is the toolbar's Chat key (the first drawn button named "Chat with everyone") by its `aria-pressed` and `aria-expanded` values (`true`, `false`, `mixed`, `other`, or `none` when absent), the sign of an open panel where the client marks one. `cands=` lists up to four composer-shaped controls (a textbox by role, a textarea, a text-like input, an editable), the drawn ones first and `more:<n>` for the rest, each by fixed facts alone: its tag and role, then `ce` (editable), `dis` (the disabled attribute), `ro` (readonly), `ad` (aria-disabled on it or above it), `drawn` (it has a client rect), `box` (a width and a height, which Playwright's visibility read needs), `vis` (its computed visibility), `ah` (aria-hidden on it or above it, which Playwright's role lookup skips), `name` (how it is named: `labelledby`, `aria`, `label`, `title`, `placeholder` or `none`), `lbl` (whether its name begins as the lookups expect: `send`, `history`, else `other` or `none`), `chat` (an element above it is labelled for the chat)
|
|
117
|
+
- **The run.** The job is the channel's first message. The bot joins the run's LiveKit room without publishing, launches Chromium on the identity's profile at `/home/bot/profile` when the run has one (a Google Meet identity from the pool) or on a fresh profile, and drives the platform's adapter: it reports `joining`, `waiting_room` and `in_call`, posts the notice, reads it back from the chat panel and reports `notice_posted`, then reads the panel every second. Each step of the posting is bounded and named, so a night that fails there reports the step rather than a bare timeout: `notice: chat_button_missing` (no Chat key was shown and the keyboard shortcut opened no panel), `chat_button_blocked` (the key would not take the click, a dialog over it, and the shortcut opened no panel), `chat_input_missing` (the panel opened without a composer-shaped control in any frame), `chat_input_blocked` (the composer is drawn disabled, read-only or aria-disabled, which the adapter reads as the meeting's chat being off for the bot, or would not take the text), `chat_enter_blocked` (it would not take the Enter) or `chat_echo_missing` (the message was typed and sent and never read back). The message is read back as a new message among those the chat observer reads, so a line the panel already held, the bot's own notice after a rejoin among them, never stands for the one being sent. A failed notice step's report carries, after its code, an outline of the chat panel (`image/src/adapters/chat-outline.ts`): for the main frame and each frame the page embeds, the frame's host and its counts of drawn controls by role (textbox, combobox, button, menuitem), of editables, of list items and of elements carrying a message id, the accessible names of the composer-shaped controls (and of the buttons in an embedded frame, whose controls are the chat's own; a menu item is counted and never named, since no lookup targets one and a menu may list people), each cut to forty characters of letters, digits and a few marks and at most twelve a frame after the filter below, the Chat key's state, the side panel's buttons by the allow-listed words of their names, the composer-shaped controls by their fixed facts and the allow-listed words of their names, and the lookups the step tried, for example `notice: chat_input_missing; panel: frames=2; f1 meet.google.com textbox=0 combobox=1 button=23 menuitem=0 editable=0 listitem=0 messageid=0 key=pressed:true,expanded:none pbtn=3,on:3,send:1(1); f2 chat.google.com textbox=1 combobox=0 button=6 menuitem=2 editable=1 listitem=3 messageid=3 cands=div,textbox,ce:1,dis:0,ro:0,ad:0,drawn:1,box:1,vis:visible,ah:0,name:aria,lbl:history,chat:0,land:none,words:history-on names=History is on|Send message; tried=composer:role,textarea,editable`. `key=` is the toolbar's Chat key (the first drawn button named "Chat with everyone") by its `aria-pressed` and `aria-expanded` values (`true`, `false`, `mixed`, `other`, or `none` when absent), the sign of an open panel where the client marks one. `cands=` lists up to four composer-shaped controls (a textbox by role, a textarea, a text-like input, an editable), the drawn ones first and `more:<n>` for the rest, each by fixed facts alone: its tag and role, then `ce` (editable), `dis` (the disabled attribute), `ro` (readonly), `ad` (aria-disabled on it or above it), `drawn` (it has a client rect), `box` (a width and a height, which Playwright's visibility read needs), `vis` (its computed visibility), `ah` (aria-hidden on it or above it, which Playwright's role lookup skips), `name` (how it is named: `labelledby`, `aria`, `label`, `title`, `placeholder` or `none`), `lbl` (whether its name begins as the lookups expect: `send`, `history`, else `other` or `none`), `chat` (an element above it is labelled for the chat), `land` (the nearest landmark above it) and `words` (the words of a fixed allow-list, `LABEL_WORDS` in `chat-outline.ts`, that its name holds, joined by `-` in the name's order, each once, a plural read as its word and an apostrophe dropped, or `none`: words for a chat stopped for the participant such as `off`, `turned`, `disabled`, `host`, `only`, `cant`; for a step a guest takes first such as `join`, `accept`, `invite`, `request`, `continue`; for the composer's own wordings such as `send`, `message`, `history`, `read`, `view`; and for an outside guest such as `guest`, `external`, `signed`; no word names a person). `pbtn=` is written when the frame draws a side panel (a complementary landmark: an `aside`, or the role written on a div): the drawn buttons under it (whatever landmark lies between, a header, a form or a region of the panel's own), `on:` how many of them are enabled (no `disabled` attribute and no `aria-disabled="true"` on them or above them), and, for each allow-listed word a button's name holds, `<word>:<drawn>(<enabled>)` in the list's order; a button's name is read for its words alone, and a button inside a message, or carrying a message's id (the Pin key beside one), counts for nothing, as it names nothing. The facts tell apart a composer the adapter's reads pass over (boxless, hidden, under an aria-hidden element, named another way), a disabled one, a panel that never opened on its key, and, for a composer the client draws disabled under a label the filter below drops, whether the label and the panel's buttons speak of a stopped chat, of a step to take first (a `join` or an `accept` on an enabled button, say), of a view-only chat or of a guest's standing; a name is read for its kind, its start and its words and never written. The outline is fitted to its length (1,900 characters): when the frames' lines would pass it, every frame's counts, key and panel totals are written first, then each frame's first candidate, then the panels' tallies by word in rounds, then each frame's second candidate, and so on while they fit, then each frame's names while they fit, no token cut; when the counts alone pass it, the frames at the end are dropped whole; the lookups tried always stay at its end. It never holds an address or a link, nor a message's text or a participant's name that the frames draw: a control inside a message element names nothing; a name is tested whole, before any cut, and is dropped when it holds the whole of a tile's, a sender's or a message's text (a message's text read from the element marked as its text, from the element carrying its id, which is all a Google Chat conversation marks, from an item of a list, and from each element of the message that holds no other, so its sender's name and each of its lines count on their own), when such a text holds the whole of it, when it shares a run of three words with one, or when it holds a word of a participant's name, however the name spells them, each text compared whole, read both as the page gives it and with each element in it a word apart (a sender's name beside a message's first line, two lines split by a line break, a tile's name beside an icon's word), with a mark between two words read both as nothing and as a space (`alpha/beta`), a modifier letter written as an apostrophe (`Alʼs`) read as the apostrophe is, and a text or a word of fewer than three characters matched only as a word of the name, except in a script whose words run together (Chinese, Japanese, Korean, Thai, Lao, Khmer, Burmese), where it is matched anywhere in the name and each letter also counts as a word of a run; a name longer than two hundred characters is dropped without its comparison; a name that holds anything shaped like an address or a link (an `@` address, a scheme such as `https://`, `mailto:` or `tel:`, a protocol-relative `//`, a bare host such as `docs.example.com` with or without a path after it, or an IPv4 or an IPv6 address), read as written, with its compatibility forms decomposed and every invisible or combining mark dropped (a format character such as a zero-width space or a soft hyphen, a default-ignorable code point such as a combining grapheme joiner, a combining mark such as a dot above: none sets a scheme's colon or a host's dot apart) and ideographic full stops read as dots, or as it would be shown, is dropped whole, never cleaned and kept; the names are withheld altogether (`names=withheld`) when any frame went unread or any frame's reading stopped at its element limit, since the texts they must not quote may then be unread, and when those texts come to more than twenty thousand terms, which bounds the filter's work; and a frame is named by its host alone. A text the frames do not draw as a message, a sender or a tile (a message outside the part of a long conversation the panel shows, a participant with no tile) is unknown to the filter. The frames are read at once, all of them within three seconds, and a frame that does not answer in time is written as `unread` and holds up no other. A `monitor_heartbeat` goes every ten seconds, and only while the panel was read through in the last ten seconds.
|
|
118
118
|
- **The audio.** The call's audio is published into the room only on `startCapture`: a 48 kHz mono microphone track read from the `meeting` monitor with `parec`, with nothing written to disk. Thirty seconds without a good read of the chat suspend the track inside the container, whatever the manager's clock says, and `startCapture` is refused while the read is that stale, before the notice's chat has been read, or while the bot is leaving. Starts and suspends run one at a time, in the order they come: a `startCapture` that arrives while another is in flight publishes nothing more, so two commands close together publish one track and run one `parec`, and a suspend that arrives while a start is in flight waits for it, then stops the track and the capture that start began.
|
|
119
119
|
- **The chat and the stop.** Each message the bot did not post itself is reported as `chat_message`. A message that is the stop phrase is also reported as `stop_command`, and the bot leaves on it at once with "Leaving now; everything I took is being deleted.". The message is compared with the phrase as AgentOS's meeting manager compares it (`image/src/stop-phrase.ts`): case ignored, each run of whitespace read as one space and the whitespace around it dropped, and `.`, `!` or `?` allowed after it. A message that holds the phrase among other words is reported as `chat_message` alone, another bot's notice in the same meeting among them. A report carries a message's first 4,000 characters, and the bot decides the stop on that text, which is the text the manager reads (`image/src/chat-report.ts`). A message the adapter read from its whole element, because the client drew no body element for it, holds the sender's name and the time on lines of their own, and a line of it that is the phrase, other than the line of the sender's name, is a stop too; another bot's notice is one line that holds the phrase among other words, and stays a chat message. A job whose stop phrase is blank once its whitespace is folded is refused (`src/protocol.ts`), so no run starts with a stop phrase no message could match. The bot's own posts, the notice that names the stop phrase among them, are never reported: the bot knows them by the texts it posted, never by a sender's name, so a participant whose name is You is heard like any other. A message is remembered by the client's id for it, or, when its element carries none, by the element and its text, so the same text sent twice is reported twice.
|
|
120
120
|
- **The limits.** The bot leaves when the waiting room's limit passes (`not_admitted`), when the client's participant count shows nobody else for the alone limit (`alone`), and at the run's whole length (`cap`); the manager's clock is the backstop. A count the client does not show is unknown, never zero. The bot reports `participant_joined` and `participant_left` when the count changes, and `usage` once a minute.
|
|
@@ -124,13 +124,13 @@ TLS belongs to the reverse proxy in front of the scheduler. Every route but `GET
|
|
|
124
124
|
- **The sandbox.** Chromium runs without its own sandbox, which needs user namespaces that the container's dropped capabilities and seccomp profile refuse; the container is the boundary. `image/seccomp.json` is Docker's default seccomp profile, unchanged (`seccomp/default.json` of moby/profiles at `6fe7deb1`); the scheduler sends it with each container when `MEETING_SECCOMP_PROFILE` names it.
|
|
125
125
|
- **Logs.** The bot writes JSON lines through pino, each with the run's id: events, phases, reasons, codes and counts, never a link, a name or a chat line.
|
|
126
126
|
|
|
127
|
-
**The adapters.** `image/src/adapters/google-meet.ts` and `microsoft-teams.ts` each hold a selector table and a `selectorsVersion`, the date its selectors were last read against the live client, and the hosts the client may load. Every selector is one Playwright engine: CSS, a comma list included, one `text=` selector, or one `role=` selector, an element by its role and its accessible name, which survives a suffix on a label and a wrapper the label sits on. Their checks read what is visible, a read waits two seconds at most for its element, and the chat is read only while its panel is open. The Meet chat is read as open-source Meet clients read it in 2026, after the nightly check's first Meet run (24 October 2026) found no textarea to type the notice into: the toolbar's Chat key is a button named "Chat with everyone", and the composer is a `<textarea>` labelled "Send a message" in a meeting whose chat is Meet's own, or Google Chat's editable in a meeting whose chat is a Google Chat conversation (Workspace's Chat in Meet: the chat of a meeting scheduled in Calendar, for the host's own domain), a `contenteditable` div with the textbox role labelled "History is on", which one reading of April 2026 found under a wrapper that carries the "Send a message" label instead. A Google Chat conversation's panel draws that editable, its Send key and its messages inside a frame the panel embeds, on chat.google.com, where a page-level locator never looks (Playwright's `page.locator` is the main frame's): the nightly check's Meet run of 25 October 2026 opened the panel on its key and found no composer by any lookup (`chat_input_missing`). The adapter looks for the composer in every frame of the page, the main frame first and the frames read anew on each poll since the chat's frame attaches after the panel opens, by role and name first, then as the textarea, then as an editable by its shape, in any language, and, while the Chat key says its panel is open, as a textarea, a textbox or an editable inside the side panel (an `aside`, or the complementary role), whatever its label, which is how the nightly check's Meet run of 27 October 2026 found the composer: drawn disabled and aria-disabled under another label, the meeting's chat off for the bot, a signed-in guest from another organisation; a composer the client draws disabled, read-only or aria-disabled is named `chat_input_blocked` at once, by Playwright's read of its state, before any text is typed, with the panel's outline and `tried=state:<lookup>`. The key's word is its `aria-pressed` or `aria-expanded` (that run read `expanded:true` once the panel had opened): open when either reads true, closed when either reads false, and a key that marks neither is trusted once the panel opened on it in the page; so a field of another side panel never stands for the composer while the chat panel is closed, a composer found by the panel lookup alone is read again on every later post (the farewell, a message the manager commands) and in the observer's reads, and a key that says its panel is open is never pressed, since the key toggles the panel and a press would close it. Each lookup reads its first visible match, never its first match, since the client keeps screens it is not showing in the page; it closes a tip laid over the call ("Got it", "Dismiss", "Not now", "No thanks") before it presses the Chat key, opens the panel with Meet's shortcut (Ctrl+Alt+c) when the key is missing or will not take the click, types with Playwright's `fill`, sends with Enter and presses the Send key in the composer's frame once when Enter left the text there, and reads its own message back as a new message among those its observer reads in that frame: an element carrying `data-message-id` (in a Google Chat conversation the message's name, `spaces/…/messages/…`; the Pin key beside the message carries the same id and is never read as one) or an element with its sender's name. The observer reads the panel's messages from the frame the composer is in, so a stop phrase typed in a Google Chat conversation is heard. Every step waits ten seconds at most (five after the shortcut) and names itself when it fails, with the panel's outline beside the name. The Teams chat is read as open-source Teams clients read it: the composer is CKEditor's editable, which some builds draw inside `[data-tid="ckeditor"]` and others as that element itself, and the adapter types into either; a message is its card's body (`[data-tid="messageBodyContent"]`), apart from the author's name and the time the card also shows. A message whose element holds no body element, in either client, is read from the whole element and marked so (`bodyRead: false`). `fixture.ts` runs the adapter of the platform the link's `platform` parameter names against the test fixture page, whose markup carries that platform's selectors; the bot takes it only for a job marked `adapter: 'fixture'`, and its network policy admits the fixture's own host alone, over plain `http` too.
|
|
127
|
+
**The adapters.** `image/src/adapters/google-meet.ts` and `microsoft-teams.ts` each hold a selector table and a `selectorsVersion`, the date its selectors were last read against the live client, and the hosts the client may load. Every selector is one Playwright engine: CSS, a comma list included, one `text=` selector, or one `role=` selector, an element by its role and its accessible name, which survives a suffix on a label and a wrapper the label sits on. Their checks read what is visible, a read waits two seconds at most for its element, and the chat is read only while its panel is open. The Meet chat is read as open-source Meet clients read it in 2026, after the nightly check's first Meet run (24 October 2026) found no textarea to type the notice into: the toolbar's Chat key is a button named "Chat with everyone", and the composer is a `<textarea>` labelled "Send a message" in a meeting whose chat is Meet's own, or Google Chat's editable in a meeting whose chat is a Google Chat conversation (Workspace's Chat in Meet: the chat of a meeting scheduled in Calendar, for the host's own domain), a `contenteditable` div with the textbox role labelled "History is on", which one reading of April 2026 found under a wrapper that carries the "Send a message" label instead. A Google Chat conversation's panel draws that editable, its Send key and its messages inside a frame the panel embeds, on chat.google.com, where a page-level locator never looks (Playwright's `page.locator` is the main frame's): the nightly check's Meet run of 25 October 2026 opened the panel on its key and found no composer by any lookup (`chat_input_missing`). The adapter looks for the composer in every frame of the page, the main frame first and the frames read anew on each poll since the chat's frame attaches after the panel opens, by role and name first, then as the textarea, then as an editable by its shape, in any language, and, while the Chat key says its panel is open, as a textarea, a textbox or an editable inside the side panel (an `aside`, or the complementary role), whatever its label, which is how the nightly check's Meet run of 27 October 2026 found the composer: drawn disabled and aria-disabled under another label, which the outline reads as the meeting's chat being off for the bot, a signed-in guest from another organisation (an inference from the outline, not something a Google page states); a composer the client draws disabled, read-only or aria-disabled is named `chat_input_blocked` at once, by Playwright's read of its state (`isEditable`, whose actionability page, https://playwright.dev/docs/actionability, names the `disabled` attribute, `aria-disabled="true"` on an element above, `readonly` and `aria-readonly`; Playwright 1.63.0, which the image pins, reads `aria-disabled="true"` on the composer itself as well, which the lifecycle test pins), before any text is typed, with the panel's outline and `tried=state:<lookup>`. The key's word is its `aria-pressed` or `aria-expanded` (that run read `expanded:true` once the panel had opened): open when either reads true, closed when either reads false, and a key that marks neither is trusted once the panel opened on it in the page; so a field of another side panel never stands for the composer while the chat panel is closed, a composer found by the panel lookup alone is read again on every later post (the farewell, a message the manager commands) and in the observer's reads, and a key that says its panel is open is never pressed, since the key toggles the panel and a press would close it. Each lookup reads its first visible match, never its first match, since the client keeps screens it is not showing in the page; it closes a tip laid over the call ("Got it", "Dismiss", "Not now", "No thanks") before it presses the Chat key, opens the panel with Meet's shortcut (Ctrl+Alt+c) when the key is missing or will not take the click, types with Playwright's `fill`, sends with Enter and presses the Send key in the composer's frame once when Enter left the text there, and reads its own message back as a new message among those its observer reads in that frame: an element carrying `data-message-id` (in a Google Chat conversation the message's name, `spaces/…/messages/…`; the Pin key beside the message carries the same id and is never read as one) or an element with its sender's name. The observer reads the panel's messages from the frame the composer is in, so a stop phrase typed in a Google Chat conversation is heard. Every step waits ten seconds at most (five after the shortcut) and names itself when it fails, with the panel's outline beside the name. The Teams chat is read as open-source Teams clients read it: the composer is CKEditor's editable, which some builds draw inside `[data-tid="ckeditor"]` and others as that element itself, and the adapter types into either; a message is its card's body (`[data-tid="messageBodyContent"]`), apart from the author's name and the time the card also shows. A message whose element holds no body element, in either client, is read from the whole element and marked so (`bodyRead: false`). `fixture.ts` runs the adapter of the platform the link's `platform` parameter names against the test fixture page, whose markup carries that platform's selectors; the bot takes it only for a job marked `adapter: 'fixture'`, and its network policy admits the fixture's own host alone, over plain `http` too.
|
|
128
128
|
|
|
129
129
|
## Tests
|
|
130
130
|
|
|
131
|
-
`test/provider.spec.ts` runs the provider against a stubbed `fetch`: the declaration, the join-link checks, dispatch and the `409`, report verification and parsing, commands, deletion and its refusals, and usage. `test/protocol.spec.ts` parses each channel message, refuses an unknown report event and a stop phrase that is blank once its whitespace is folded, and checks the event ids. `test/scheduler.spec.ts` runs the scheduler on an ephemeral port with a fake Docker (`test/fakes/docker.ts`), a fake product that records every report (`test/fakes/product.ts`) and real `ws` sockets: the credential, the slots and the container policy, the join-link checks, the channel's job, commands and reports, a second channel, exits, the bot's own end, restarts and stray containers, the kills, capacity and pausing, the fixture host, a dispatch sent again, deletion, a leave taken while the bot was away, report order and retries, the identity pool, refused bodies, start failures and the listen addresses. `image/src/__tests__/adapters.test.ts` checks the bot's adapters with no browser: every selector in the Google Meet and Microsoft Teams tables is one Playwright engine, the Meet chat's key and composer are named by role and accessible name with the earlier markup kept as fallbacks, the panel's composer is read by its shape inside the side panel alone and a composer drawn disabled is named `chat_input_blocked` by its state before any text is typed, a second post through that composer presses the Chat key once whichever state the key marks (`aria-pressed`, `aria-expanded` or neither), a key that says its panel is open is never pressed, another panel's field is never taken while the key says the panel closed, each lookup reads its first visible match past a hidden one, the network policy's rule on each platform's hosts, schemes and address literals, the fixture adapter's platform, code and hosts, each adapter's reading of a message with and without its body element, through a fake page, the Meet observer's reading of the panel through Google Chat's editable alone, its reading of the messages from the frame the composer is in, its refusal of a panel that shows no composer, its reading of the panel through the composer under another label while the Chat key says the panel is open and its refusal of another panel's field when the key says it is closed or there is no key, and the Meet adapter's posting of a message through a fake call: the panel opened on its key and the notice typed into each composer shape, in the page or in a frame the panel embeds, and read back as a new message from the composer's frame (one the panel already held stands for nothing), a tip closed first, the shortcut when the key is under a dialog or missing, the Send key in the composer's frame when Enter left the text, with the composer's text read within the step's wait, the step named when the composer, the key, the click or the echo is missing or the composer refuses the text or the Enter, and the panel's outline carried by the failed step's error (each frame by its host, the counts, the names, the lookups tried, in the record line's alphabet). The outline's own tests read a fake tree that draws the tiles, a sender, a message's text and a Pin key inside the message, and check that the outline counts them and quotes none of them, names no button of the main frame and no menu item anywhere, drops a name that quotes a participant in another spelling or by a word of the name or that holds an address or a link, keeps a safe name after a dozen that quoted a participant, withholds every name when the main frame went unread or a reading was cut short, stops at its element limit, names a frame by its host alone, reads the frames at once so that a frame that never answers holds up none after it,
|
|
131
|
+
`test/provider.spec.ts` runs the provider against a stubbed `fetch`: the declaration, the join-link checks, dispatch and the `409`, report verification and parsing, commands, deletion and its refusals, and usage. `test/protocol.spec.ts` parses each channel message, refuses an unknown report event and a stop phrase that is blank once its whitespace is folded, and checks the event ids. `test/scheduler.spec.ts` runs the scheduler on an ephemeral port with a fake Docker (`test/fakes/docker.ts`), a fake product that records every report (`test/fakes/product.ts`) and real `ws` sockets: the credential, the slots and the container policy, the join-link checks, the channel's job, commands and reports, a second channel, exits, the bot's own end, restarts and stray containers, the kills, capacity and pausing, the fixture host, a dispatch sent again, deletion, a leave taken while the bot was away, report order and retries, the identity pool, refused bodies, start failures and the listen addresses. `image/src/__tests__/adapters.test.ts` checks the bot's adapters with no browser: every selector in the Google Meet and Microsoft Teams tables is one Playwright engine, the Meet chat's key and composer are named by role and accessible name with the earlier markup kept as fallbacks, the panel's composer is read by its shape inside the side panel alone and a composer drawn disabled is named `chat_input_blocked` by its state before any text is typed, a second post through that composer presses the Chat key once whichever state the key marks (`aria-pressed`, `aria-expanded` or neither), a key that says its panel is open is never pressed, another panel's field is never taken while the key says the panel closed, each lookup reads its first visible match past a hidden one, the network policy's rule on each platform's hosts, schemes and address literals, the fixture adapter's platform, code and hosts, each adapter's reading of a message with and without its body element, through a fake page, the Meet observer's reading of the panel through Google Chat's editable alone, its reading of the messages from the frame the composer is in, its refusal of a panel that shows no composer, its reading of the panel through the composer under another label while the Chat key says the panel is open and its refusal of another panel's field when the key says it is closed or there is no key, and the Meet adapter's posting of a message through a fake call: the panel opened on its key and the notice typed into each composer shape, in the page or in a frame the panel embeds, and read back as a new message from the composer's frame (one the panel already held stands for nothing), a tip closed first, the shortcut when the key is under a dialog or missing, the Send key in the composer's frame when Enter left the text, with the composer's text read within the step's wait, the step named when the composer, the key, the click or the echo is missing or the composer refuses the text or the Enter, and the panel's outline carried by the failed step's error (each frame by its host, the counts, the names, the lookups tried, in the record line's alphabet). The outline's own tests read a fake tree that draws the tiles, a sender, a message's text and a Pin key inside the message, and check that the outline counts them and quotes none of them, names no button of the main frame and no menu item anywhere, drops a name that quotes a participant in another spelling or by a word of the name or that holds an address or a link, keeps a safe name after a dozen that quoted a participant, withholds every name when the main frame went unread or a reading was cut short, stops at its element limit, names a frame by its host alone, reads the frames at once so that a frame that never answers holds up none after it, writes that frame as unread, writes each composer-shaped control's allow-listed label words in the name's order (a plural as its word, an apostrophe dropped) or `none`, writes the side panel's buttons by those words with how many are drawn and how many of them enabled while naming no button, and writes no panel token for a frame that draws no side panel. `image/src/__tests__/audio.test.ts` runs the audio publisher against fakes of `@livekit/rtc-node` and `parec` whose publish waits until the test answers it: two starts at once publish one track and start one capture, and a suspend that comes while a start waits stops the track and the capture that start began. `image/src/__tests__/stop-phrase.test.ts` checks which chat messages are the stop phrase: the phrase whatever its case, the whitespace around and inside it, and its closing punctuation is; another bot's notice, its short notice to a late arrival and the phrase among other words are not. `image/src/__tests__/chat-report.test.ts` checks the bot's report of a message: its text and sender cut to the protocol's ceilings, the stop decided on the text as reported, and a line that is the phrase taken as a stop in a message read from its whole element only. CI runs all seven with the pack's tests.
|
|
132
132
|
|
|
133
|
-
**The lifecycle test.** `image/fixture/` is a meeting page that carries the Google Meet adapter's selectors: a pre-join screen (the name field, the camera and microphone toggles, Ask to join), a waiting screen, the call's tiles, the participant count, the chat panel and the Leave key, and an `<audio>` element that plays a known recording into the page's output, which is the call's audio. Its Meet chat is drawn as open-source Meet clients read it: the messages are the items of a list labelled "In-call messages", each with `data-message-id`, and the composer is the textarea labelled "Send a message" (the default), Google Chat's editable, a textbox div editable by a bare `contenteditable` attribute and labelled "History is on" (`chat=history`), that editable under a wrapper that carries the "Send a message to everyone" label with no role or label of its own (`chat=nested`), or a frame the panel embeds (`chat=frame`, the fixture's `chat-frame.html` at `/chat-frame`) that holds that editable, its Send key and the messages, with none of them in the page, as a Google Chat conversation's panel embeds its frame from chat.google.com; with `chat=none` the panel holds the messages and no composer at all, and with `chat=off` it holds the textarea drawn disabled and aria-disabled under another label, in a side panel that is an `aside`, as Meet drew it on 27 October 2026 for a participant
|
|
133
|
+
**The lifecycle test.** `image/fixture/` is a meeting page that carries the Google Meet adapter's selectors: a pre-join screen (the name field, the camera and microphone toggles, Ask to join), a waiting screen, the call's tiles, the participant count, the chat panel and the Leave key, and an `<audio>` element that plays a known recording into the page's output, which is the call's audio. Its Meet chat is drawn as open-source Meet clients read it: the messages are the items of a list labelled "In-call messages", each with `data-message-id`, and the composer is the textarea labelled "Send a message" (the default), Google Chat's editable, a textbox div editable by a bare `contenteditable` attribute and labelled "History is on" (`chat=history`), that editable under a wrapper that carries the "Send a message to everyone" label with no role or label of its own (`chat=nested`), or a frame the panel embeds (`chat=frame`, the fixture's `chat-frame.html` at `/chat-frame`) that holds that editable, its Send key and the messages, with none of them in the page, as a Google Chat conversation's panel embeds its frame from chat.google.com; with `chat=none` the panel holds the messages and no composer at all, and with `chat=off` it holds the textarea drawn disabled and aria-disabled under another label, in a side panel that is an `aside`, as Meet drew it on 27 October 2026 for a participant the outline reads as having its chat off. With `chat=aria-disabled` it holds that textarea under the same label marked `aria-disabled="true"` on itself alone, with no `disabled` attribute, so the page still takes text into it, and with `chat=relabelled` that textarea enabled under a label of its own, which no lookup by name reads either. With `platform=microsoft_teams` in its link it carries the Microsoft Teams adapter's selectors instead, and draws the Teams chat as open-source Teams clients read it: a message card (`[data-tid="chat-pane-message"]`) holds its author, its time and its body (`[data-tid="messageBodyContent"]`), and the composer is CKEditor's editable inside `[data-tid="ckeditor"]`, or that element itself with `composer=self`, the two shapes Teams builds show. Its server, `serve.mjs` on port 8787, queues scripted events from `POST /control` (`admit`, `late_joiner`, `chat` with a sender and a text, `noId` for a Meet message drawn without `data-message-id` and `noBody` for a Teams card drawn without its body element, `hide_chat`, `show_chat`, `leave_all`) for the page to poll, and keeps what the page did on `GET /state`: its knocks, the texts the bot posted, its presses of Leave. The recording is spoken from `recording.txt` by espeak-ng when the fixture's image is built: about twenty seconds of 16 kHz mono speech. `image/test/compose.yml` runs LiveKit 1.13.7 in development mode (its documented key pair, `devkey` and `secret`), the fixture, a fake product that records every report with a hash of the token it came under, and the scheduler on this pack's build, which starts real bot containers from `agentos-meeting-bot:ci` on the `meeting-bots` network under its fixed container policy, the seccomp profile included. `image/test/lifecycle.mjs` sends twelve runs through the scheduler's API and listens to each run's room with `@livekit/rtc-node`:
|
|
134
134
|
|
|
135
135
|
- The first run joins, waits in the waiting room, is admitted, posts the notice, reads it back and reports a heartbeat within 12 seconds of it. No audio frame reaches the room before `startCapture`.
|
|
136
136
|
- After `startCapture` the recording is heard in the room: above 0.01 RMS in at least three of five seconds, the first frame after `notice_posted`.
|
|
@@ -145,10 +145,11 @@ TLS belongs to the reverse proxy in front of the scheduler. Every route but `GET
|
|
|
145
145
|
- A Google Meet run whose panel holds no composer at all reports `notice: chat_input_missing` with the panel's outline after it (the fixture's host, the counts of controls by role and of the message drawn before the panel opened, the Chat key pressed, the pre-join name field as the one composer-shaped control, undrawn under its hidden section, and the lookups tried), in the record line's alphabet and quoting neither that message, its sender nor a tile's name, and leaves with `notice_failed`.
|
|
146
146
|
- A Google Meet run whose panel holds the composer drawn disabled and aria-disabled under another label, in a side panel that is an `aside`, reports `notice: chat_input_blocked` at once, with the panel's outline after it (the Chat key pressed, the disabled textarea as the drawn candidate in the complementary landmark, the pre-join name field undrawn after it, and `tried=state:panel`), posts nothing, quotes no tile's name, and leaves with `notice_failed`. Before the adapter read the panel's composer by its shape and its state, it ended as the nightly check's Meet run of 27 October 2026 did: `notice: chat_input_missing`, ten seconds after `in_call`.
|
|
147
147
|
- A Google Meet run whose panel holds the composer enabled under another label, which the adapter finds by its shape and its place in the panel alone, posts the notice and reads it back, reports a participant's message as typed, and leaves on the stop phrase with the farewell posted through the same composer, the panel left open: the Chat key, which toggles the panel and marks its state in `aria-pressed`, is pressed once. Before the adapter read the key's state, the farewell pressed the key again, which closed the panel, and failed as `chat_input_missing`.
|
|
148
|
+
- A Google Meet run whose panel holds the textarea of the disabled run above, under its label, marked `aria-disabled="true"` on itself alone, with no `disabled` attribute, reports `notice: chat_input_blocked` at once, its outline reading the composer as `dis:0,ad:1` with `tried=state:panel`, posts nothing though the page would take the text, and leaves with `notice_failed`. Playwright's actionability page names `aria-disabled` only on an element above the target; this run pins that Playwright 1.63.0 reads it on the composer itself too.
|
|
148
149
|
- `docker inspect` of each live container shows neither its meeting link nor a token, and shows no bind mount, the browser profile in a tmpfs, a read-only root, no capabilities and the seccomp profile.
|
|
149
150
|
- Every report reaches the product under its run's token.
|
|
150
151
|
|
|
151
|
-
Each step prints one line. The first failed check of the first two runs and of the last two checks prints the reports received so far, by sequence number, event and time, and exits 1. The
|
|
152
|
+
Each step prints one line. The first failed check of the first two runs and of the last two checks prints the reports received so far, by sequence number, event and time, and exits 1. The ten runs between them go on past a failed check where their run allows, and the test then fails at its end naming every check of theirs that failed. From this folder, once the pack is built:
|
|
152
153
|
|
|
153
154
|
```bash
|
|
154
155
|
docker build -f image/Dockerfile -t agentos-meeting-bot:ci .
|
package/manifest.json
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
"$schema": "https://agentos.sh/schemas/extension-manifest-v1.json",
|
|
3
3
|
"id": "com.framers.meetings.browser",
|
|
4
4
|
"name": "Browser Meeting Bot Provider",
|
|
5
|
-
"version": "0.1.
|
|
5
|
+
"version": "0.1.9",
|
|
6
6
|
"description": "A meeting-participant provider that joins Google Meet and Microsoft Teams through a browser in a container, posts a notice, publishes the call's audio into a LiveKit room, and leaves on any participant's stop command.",
|
|
7
7
|
"categories": ["meetings", "voice"],
|
|
8
8
|
"platforms": ["google_meet", "microsoft_teams"],
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@framers/agentos-ext-meeting-browser",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.9",
|
|
4
4
|
"description": "Browser-join meeting bot provider for AgentOS: Google Meet and Microsoft Teams through Chromium in a container, audio into a LiveKit room",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|