@dxos/plugin-transcription 0.8.4-main.fd6878d → 0.8.4-staging.60fe92afc8

package/LICENSE

@@ -1,8 +1,105 @@
-MIT License 
-Copyright (c) 2022 DXOS
+# Functional Source License, Version 1.1, ALv2 Future License
 
-Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+## Abbreviation
 
-The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+FSL-1.1-Apache-2.0
 
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+## Notice
+
+Copyright 2026 DXOS
+
+## Terms and Conditions
+
+### Licensor ("We")
+
+The party offering the Software under these Terms and Conditions.
+
+### The Software
+
+The "Software" is each version of the software that we make available under
+these Terms and Conditions, as indicated by our inclusion of these Terms and
+Conditions with the Software.
+
+### License Grant
+
+Subject to your compliance with this License Grant and the Patents,
+Redistribution and Trademark clauses below, we hereby grant you the right to
+use, copy, modify, create derivative works, publicly perform, publicly display
+and redistribute the Software for any Permitted Purpose identified below.
+
+### Permitted Purpose
+
+A Permitted Purpose is any purpose other than a Competing Use. A Competing Use
+means making the Software available to others in a commercial product or
+service that:
+
+1. substitutes for the Software;
+
+2. substitutes for any other product or service we offer using the Software
+   that exists as of the date we make the Software available; or
+
+3. offers the same or substantially similar functionality as the Software.
+
+Permitted Purposes specifically include using the Software:
+
+1. for your internal use and access;
+
+2. for non-commercial education;
+
+3. for non-commercial research; and
+
+4. in connection with professional services that you provide to a licensee
+   using the Software in accordance with these Terms and Conditions.
+
+### Patents
+
+To the extent your use for a Permitted Purpose would necessarily infringe our
+patents, the license grant above includes a license under our patents. If you
+make a claim against any party that the Software infringes or contributes to
+the infringement of any patent, then your patent license to the Software ends
+immediately.
+
+### Redistribution
+
+The Terms and Conditions apply to all copies, modifications and derivatives of
+the Software.
+
+If you redistribute any copies, modifications or derivatives of the Software,
+you must include a copy of or a link to these Terms and Conditions and not
+remove any copyright notices provided in or with the Software.
+
+### Disclaimer
+
+THE SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR
+PURPOSE, MERCHANTABILITY, TITLE OR NON-INFRINGEMENT.
+
+IN NO EVENT WILL WE HAVE ANY LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE
+SOFTWARE, INCLUDING INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES,
+EVEN IF WE HAVE BEEN INFORMED OF THEIR POSSIBILITY IN ADVANCE.
+
+### Trademarks
+
+Except for displaying the License Details and identifying us as the origin of
+the Software, you have no right under these Terms and Conditions to use our
+trademarks, trade names, service marks or product names.
+
+## Grant of Future License
+
+We hereby irrevocably grant you an additional license to use the Software under
+the Apache License, Version 2.0 that is effective on the second anniversary of
+the date we make the Software available. On or after that date, you may use the
+Software under the Apache License, Version 2.0, in which case the following
+will apply:
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use
+this file except in compliance with the License.
+
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed
+under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.

package/PLUGIN.mdl

@@ -0,0 +1,401 @@
+---
+id: org.dxos.plugin.transcription
+name: TranscriptionPlugin
+version: 0.1.0
+---
+
+A real-time voice-to-text transcription plugin for DXOS Composer.
+Captures spoken audio from a microphone, converts it to timestamped text segments via the
+Whisper speech-recognition service, and stores the results in an ECHO `Transcript` object
+backed by a `Feed`.
+The plugin also exposes blueprint-compatible operations that let AI assistants open and
+summarize transcripts inside agent workflows.
+
+## Extensions
+
+The following extension dialects are used in this document.
+Each extension is defined in the Appendix or resolved via its URI.
+
+| Term        | URI                            |
+|-------------|--------------------------------|
+| `type`      | `org.dxos.mdl.type@1.0`       |
+| `feat`      | `org.dxos.mdl.feat@1.0`       |
+| `test`      | `org.dxos.mdl.test@1.0`       |
+| `component` | `org.dxos.mdl.component@1.0`  |
+| `op`        | `org.dxos.mdl.op@1.0`         |
+
+## Types
+
+```mdl
+type TranscriptSegment
+  fields:
+    text: string              # transcribed text for this segment
+    started: string           # ISO-8601 wall-clock timestamp of segment start
+```
+
+```mdl
+type Message
+  fields:
+    id: string
+    created: string           # ISO-8601 creation timestamp
+    blocks: TranscriptSegment[]
+    sender:
+      identityDid?: string    # DID of the speaker, if known
+```
+
+```mdl
+type Transcript
+  fields:
+    id: string
+    feed: Ref<Feed>           # ECHO feed that stores Message objects
+    started?: string          # ISO-8601 timestamp when recording started
+```
+
+```mdl
+type WhisperWord
+  fields:
+    word: string
+    start: number             # seconds from start of audio chunk
+    end: number
+```
+
+```mdl
+type WhisperSegment
+  fields:
+    text: string
+    start: number
+    end: number
+    no_speech_prob: number    # probability of silence; used to filter noise
+    words: WhisperWord[]
+```
+
+## Components
+
+```mdl
+component Transcription
+  desc: Read-only CodeMirror editor that renders transcript messages as Markdown with byline headings.
+  props:
+    transcript?: Transcript
+    model: SerializationModel<Message>
+  slots:
+    (none — self-contained editor viewport)
+  layout: |
+    ┌─────────────────────────────────┐
+    │ [CodeMirror editor — read-only] │
+    │  ## Speaker · HH:MM             │
+    │  Transcribed text…              │
+    │  …                              │
+    └─────────────────────────────────┘
+```
+
+```mdl
+component Oracle
+  desc: |
+    Ambient display surface with an SVG graph canvas and a centered text overlay.
+    Used to show the latest transcribed utterance alongside a live visualization.
+  props:
+    text?: string             # latest utterance to display
+    delay?: number            # per-character reveal delay in ms (default 12)
+  slots:
+    canvas?: ReactNode        # replaces default SVG.Graph when provided
+  layout: |
+    ┌─────────────────────────────────┐
+    │  [SVG canvas — absolutely fills]│
+    │         [Oracle.Text]           │
+    │   "latest utterance…"           │
+    └─────────────────────────────────┘
+```
+
+```mdl
+component TranscriptionArticle
+  desc: Full-page article panel combining the Transcription editor with a start/stop recording toolbar action.
+  props:
+    subject: Transcript
+    role: string
+    attendableId?: string
+  state:
+    recording: boolean        # whether microphone capture is active
+  actions:
+    toggleRecording()
+  layout: |
+    ┌─────────────────────────────────┐
+    │ [Toolbar]  [🎤 Start/Stop]      │
+    ├─────────────────────────────────┤
+    │ [Transcription editor]          │
+    │  …                              │
+    └─────────────────────────────────┘
+```
+
+## Operations
+
+Pure operations — no side-effects.
+
+```mdl
+op openTranscript
+  desc: Loads a Transcript from ECHO, reads all Messages from its Feed, and returns the full text content rendered with speaker bylines.
+  input:
+    transcript: Ref<Transcript>
+  output: string
+  errors:
+    TranscriptNotFound: no Transcript object found for the given ref
+    FeedNotFound: the Transcript's feed ref cannot be resolved
+  requires: [Database, FeedService]
+```
+
+Effectful operations — produce output via AI or mutate persistent state.
+
+```mdl
+op createTranscript
+  desc: Creates a new Feed and a Transcript object linked to it, then writes both into the current ECHO space.
+  input:
+    name?: string
+    space: Space
+  output: Transcript
+  effects: [echo:write]
+```
+
+```mdl
+op summarizeTranscript
+  desc: Sends a transcript string and optional participant notes to an LLM and returns a Markdown summary with key points, verbatim quotes, and actionable tasks extracted from the conversation.
+  input:
+    transcript: string
+    notes?: string
+  output: string
+  effects: [http]
+  requires: [AiService]
+  note: |
+    Uses Claude Sonnet. Prompt instructs the model to bold speaker names,
+    include verbatim quotes, and emit only tasks that are directly actionable
+    and clearly assigned.
+```
+
+```mdl
+op normalizeSentences
+  desc: Post-processes an array of raw transcript messages, merging word-level tokens into grammatically complete sentences.
+  input:
+    messages: Message[]
+  output: Message[]
+  note: pure transformation; no effects or service dependencies
+```
+
+## Features
+
+```mdl
+feat F-1: Recording Lifecycle
+
+  req F-1.1: User can start microphone capture from TranscriptionArticle toolbar.
+  req F-1.2: User can stop microphone capture; final buffered chunks are flushed and transcribed.
+  req F-1.3: TranscriptionManager restarts the Transcriber automatically when the audio track changes.
+```
+
+```mdl
+feat F-2: Audio Chunking and Transcription
+
+  req F-2.1:
+    when: user is speaking and chunk count reaches transcribeAfterChunksAmount
+    then: Transcriber schedules a deferred transcription task
+
+  req F-2.2:
+    when: transcription task runs
+    then: buffered PCM chunks are merged into a WAV file and POSTed to the Whisper endpoint
+
+  req F-2.3:
+    when: Whisper response is received
+    then: segments are filtered by no_speech_prob and aligned to absolute wall-clock timestamps
+
+  req F-2.4:
+    when: user is not speaking
+    then: only prefixBufferChunksAmount most-recent chunks are retained; older chunks discarded
+```
+
+```mdl
+feat F-3: Transcript Persistence
+
+  req F-3.1:
+    when: aligned segments are produced
+    then: a Message object with the segments is appended to the Transcript's ECHO Feed
+
+  req F-3.2:
+    when: a messageEnricher is configured
+    then: Message is passed through the enricher before being written to the Feed
+
+  req F-3.3:
+    when: a peer's ECHO space receives the Feed update
+    then: TranscriptionArticle re-renders with the new messages
+    tags: [collaborative]
+```
+
+```mdl
+feat F-4: Transcript Display
+
+  req F-4.1:
+    when: a Transcript object is opened
+    then: all Messages are rendered in the Transcription editor with speaker bylines and timestamps
+
+  req F-4.2:
+    when: new messages arrive during live recording
+    then: the SerializationModel is updated and the editor scrolls to show the latest content
+```
+
+```mdl
+feat F-5: AI Summary
+
+  req F-5.1:
+    when: op:summarizeTranscript is invoked
+    then: LLM returns a structured Markdown document with key points, quotes, and tasks
+
+  req F-5.2:
+    when: participant notes are provided
+    then: notes are correlated with transcript content in the summary
+
+  req F-5.3:
+    when: no actionable tasks are found
+    then: the tasks section is omitted from the summary
+```
+
+```mdl
+feat F-6: Blueprint Integration
+
+  req F-6.1: Blueprint exposes op:openTranscript and op:summarizeTranscript as AI tools.
+  req F-6.2: An AI assistant can open a transcript by Ref and return formatted text to the user.
+  req F-6.3: An AI assistant can invoke the summary operation and present the result as a message.
+```
+
+## Acceptance
+
+```mdl
+test T-1: Create transcript
+  given: a DXOS space is available
+  when: op:createTranscript is invoked
+  then:
+    - a new Feed is created in the space
+    - a Transcript object linked to the Feed is created in the space
+    - the returned Transcript.feed.target is not null
+```
+
+```mdl
+test T-2: Record and persist segment
+  given: a Transcript with a Feed, microphone capture active
+  when: the user speaks and silence is detected
+  then:
+    - Whisper returns at least one segment
+    - a Message is appended to the Feed
+    - Message.blocks contains at least one TranscriptSegment with non-empty text
+```
+
+```mdl
+test T-3: Deduplicate segments across chunk boundaries
+  given: two consecutive transcription tasks with overlapping Whisper output
+  when: the second task produces segments whose end timestamps fall before lastTimestamp
+  then:
+    - duplicate segments are filtered out
+    - only novel words (start >= lastTimestamp) are included
+```
+
+```mdl
+test T-4: Open transcript
+  given: a Transcript with three Messages in its Feed
+  when: op:openTranscript is invoked
+  then:
+    - content string contains three byline sections
+    - each section starts with "## <speaker> · <time>"
+```
+
+```mdl
+test T-5: Summarize transcript
+  given: a transcript string and optional notes string
+  when: op:summarizeTranscript is invoked
+  then:
+    - summary is a non-empty Markdown string
+    - speaker names appear in bold
+    - a tasks section is present if actionable tasks were detected
+```
+
+```mdl
+test T-6: Sentence normalization is pure
+  given: an array of raw word-level Messages
+  when: op:normalizeSentences is called
+  then:
+    - output messages contain complete sentences
+    - no ECHO writes occur
+    - input array is not mutated
+```
+
+```mdl
+test T-7: Toggle recording in UI
+  given: TranscriptionArticle is mounted with a Transcript
+  when: user clicks the Start Recording toolbar button
+  then:
+    - recording state becomes true
+    - toolbar button label changes to "Stop Recording"
+    - microphone capture begins
+```
+
+---
+
+## Appendix: Extension Definitions
+
+Extension block types used in this document are defined below using
+the core `ext` primitive — the only construct the base language provides.
+
+```mdl
+ext type
+  uri: org.dxos.mdl.type@1.0
+  desc: A named data structure with typed fields and optional literals.
+  fields:
+    desc?: Prose
+    fields?: FieldMap        # name[?]: TypeExpr  (# inline comment)
+    literals?: UnionList     # a | b | c
+    extends?: TypeRef[]
+```
+
+```mdl
+ext feat
+  uri: org.dxos.mdl.feat@1.0
+  desc: A named feature grouping one or more requirements.
+  fields:
+    desc?: Prose
+    req: RequirementList
+  nesting: self              # feat blocks may contain feat blocks
+```
+
+```mdl
+ext test
+  uri: org.dxos.mdl.test@1.0
+  desc: An acceptance scenario expressed as given / when / then steps.
+  fields:
+    given?: Step | Step[]
+    when?: Step | Step[]
+    then: Step | Step[]
+    tags?: TagList
+```
+
+```mdl
+ext component
+  uri: org.dxos.mdl.component@1.0
+  desc: A UI component with props, internal state, slots, actions, and events.
+  fields:
+    desc?: Prose
+    props?: FieldMap            # external inputs (immutable inside component)
+    state?: FieldMap            # internal reactive state
+    slots?: FieldMap            # named ReactNode injection points
+    actions?: ActionMap         # methods the component exposes or handles
+    emits?: EventMap            # events the component raises to its parent
+    layout?: CodeBlock          # ASCII sketch of visual structure (non-normative)
+```
+
+```mdl
+ext op
+  uri: org.dxos.mdl.op@1.0
+  desc: |
+    A named operation with typed inputs, outputs, and declared errors.
+    Pure ops have no effects or requires. Effectful ops declare both.
+  fields:
+    desc?: Prose
+    input?: FieldMap            # named input parameters
+    output?: TypeExpr           # return type
+    errors?: ErrorMap           # name: Prose  (when this error occurs)
+    effects?: EffectList        # echo:read | echo:write | http | fs | ...
+    requires?: ServiceList      # injected service dependencies
+    note?: Prose                # implementation guidance (non-normative)
+```

package/README.md

@@ -12,4 +12,4 @@ Transcription functionality
 
 Your ideas, issues, and code are most welcome. Please take a look at our [community code of conduct](https://github.com/dxos/dxos/blob/main/CODE_OF_CONDUCT.md), the [issue guide](https://github.com/dxos/dxos/blob/main/CONTRIBUTING.md#submitting-issues), and the [PR contribution guide](https://github.com/dxos/dxos/blob/main/CONTRIBUTING.md#submitting-prs).
 
-License: [MIT](./LICENSE) Copyright 2023 © DXOS
+License: [FSL-1.1-Apache-2.0](./LICENSE) Copyright 2023 © DXOS

package/dist/lib/neutral/TranscriptionArticle-L6XANBH5.mjs

@@ -0,0 +1,54 @@
+import {
+  renderByline
+} from "./chunk-KAEXRX3Y.mjs";
+import "./chunk-J5LGTIGS.mjs";
+
+// src/containers/TranscriptionArticle/TranscriptionArticle.tsx
+import React from "react";
+import { Filter, Obj, Query } from "@dxos/echo";
+import { useMembers, useQuery } from "@dxos/react-client/echo";
+import { Panel } from "@dxos/react-ui";
+import { Menu, MenuBuilder, useMenuBuilder } from "@dxos/react-ui-menu";
+import { Message } from "@dxos/types";
+import { Transcription } from "#components";
+import { useFeedModelAdapter, useTranscriptionRecording } from "#hooks";
+import { meta } from "#meta";
+var TranscriptionArticle = ({ role, subject: transcript, attendableId }) => {
+  const db = Obj.getDatabase(transcript);
+  const members = useMembers(db?.spaceId).map((member) => member.identity);
+  const feed = transcript.feed.target;
+  const messages = useQuery(db, feed ? Query.select(Filter.type(Message.Message)).from(feed) : Query.select(Filter.nothing()));
+  const model = useFeedModelAdapter(renderByline(members), messages);
+  const { recording, toggleRecording } = useTranscriptionRecording(transcript);
+  const menuActions = useMenuBuilder(() => MenuBuilder.make().action("toggle-recording", {
+    label: [
+      recording ? "stop-recording.label" : "start-recording.label",
+      {
+        ns: meta.id
+      }
+    ],
+    icon: recording ? "ph--stop-circle--regular" : "ph--microphone--regular",
+    disposition: "toolbar",
+    testId: "transcription.toggle-recording"
+  }, toggleRecording).build(), [
+    recording,
+    toggleRecording
+  ]);
+  return /* @__PURE__ */ React.createElement(Panel.Root, {
+    role
+  }, /* @__PURE__ */ React.createElement(Menu.Root, {
+    ...menuActions,
+    attendableId
+  }, /* @__PURE__ */ React.createElement(Panel.Toolbar, {
+    asChild: true
+  }, /* @__PURE__ */ React.createElement(Menu.Toolbar, null))), /* @__PURE__ */ React.createElement(Panel.Content, {
+    asChild: true
+  }, /* @__PURE__ */ React.createElement(Transcription, {
+    model,
+    transcript
+  })));
+};
+export {
+  TranscriptionArticle as default
+};
+//# sourceMappingURL=TranscriptionArticle-L6XANBH5.mjs.map

package/dist/lib/neutral/TranscriptionArticle-L6XANBH5.mjs.map

@@ -0,0 +1,7 @@
+{
+  "version": 3,
+  "sources": ["../../../src/containers/TranscriptionArticle/TranscriptionArticle.tsx"],
+  "sourcesContent": ["//\n// Copyright 2024 DXOS.org\n//\n\nimport React from 'react';\n\nimport { type AppSurface } from '@dxos/app-toolkit/ui';\nimport { Filter, Obj, Query } from '@dxos/echo';\nimport { useMembers, useQuery } from '@dxos/react-client/echo';\nimport { Panel } from '@dxos/react-ui';\nimport { Menu, MenuBuilder, useMenuBuilder } from '@dxos/react-ui-menu';\nimport { Message, type Transcript } from '@dxos/types';\n\nimport { Transcription } from '#components';\nimport { useFeedModelAdapter, useTranscriptionRecording } from '#hooks';\nimport { meta } from '#meta';\n\nimport { renderByline } from '../../util';\n\nexport type TranscriptionArticleProps = AppSurface.ObjectArticleProps<Transcript.Transcript>;\n\nexport const TranscriptionArticle = ({ role, subject: transcript, attendableId }: TranscriptionArticleProps) => {\n  const db = Obj.getDatabase(transcript);\n  const members = useMembers(db?.spaceId).map((member) => member.identity);\n  const feed = transcript.feed.target;\n  const messages = useQuery(\n    db,\n    feed ? Query.select(Filter.type(Message.Message)).from(feed) : Query.select(Filter.nothing()),\n  );\n  const model = useFeedModelAdapter(renderByline(members), messages);\n  const { recording, toggleRecording } = useTranscriptionRecording(transcript);\n\n  const menuActions = useMenuBuilder(\n    () =>\n      MenuBuilder.make()\n        .action(\n          'toggle-recording',\n          {\n            label: [recording ? 'stop-recording.label' : 'start-recording.label', { ns: meta.id }],\n            icon: recording ? 'ph--stop-circle--regular' : 'ph--microphone--regular',\n            disposition: 'toolbar',\n            testId: 'transcription.toggle-recording',\n          },\n          toggleRecording,\n        )\n        .build(),\n    [recording, toggleRecording],\n  );\n\n  return (\n    <Panel.Root role={role}>\n      <Menu.Root {...menuActions} attendableId={attendableId}>\n        <Panel.Toolbar asChild>\n          <Menu.Toolbar />\n        </Panel.Toolbar>\n      </Menu.Root>\n      <Panel.Content asChild>\n        <Transcription model={model} transcript={transcript} />\n      </Panel.Content>\n    </Panel.Root>\n  );\n};\n"],
+  "mappings": ";;;;;;AAIA,OAAOA,WAAW;AAGlB,SAASC,QAAQC,KAAKC,aAAa;AACnC,SAASC,YAAYC,gBAAgB;AACrC,SAASC,aAAa;AACtB,SAASC,MAAMC,aAAaC,sBAAsB;AAClD,SAASC,eAAgC;AAEzC,SAASC,qBAAqB;AAC9B,SAASC,qBAAqBC,iCAAiC;AAC/D,SAASC,YAAY;AAMd,IAAMC,uBAAuB,CAAC,EAAEC,MAAMC,SAASC,YAAYC,aAAY,MAA6B;AACzG,QAAMC,KAAKC,IAAIC,YAAYJ,UAAAA;AAC3B,QAAMK,UAAUC,WAAWJ,IAAIK,OAAAA,EAASC,IAAI,CAACC,WAAWA,OAAOC,QAAQ;AACvE,QAAMC,OAAOX,WAAWW,KAAKC;AAC7B,QAAMC,WAAWC,SACfZ,IACAS,OAAOI,MAAMC,OAAOC,OAAOC,KAAKC,QAAQA,OAAO,CAAA,EAAGC,KAAKT,IAAAA,IAAQI,MAAMC,OAAOC,OAAOI,QAAO,CAAA,CAAA;AAE5F,QAAMC,QAAQC,oBAAoBC,aAAanB,OAAAA,GAAUQ,QAAAA;AACzD,QAAM,EAAEY,WAAWC,gBAAe,IAAKC,0BAA0B3B,UAAAA;AAEjE,QAAM4B,cAAcC,eAClB,MACEC,YAAYC,KAAI,EACbC,OACC,oBACA;IACEC,OAAO;MAACR,YAAY,yBAAyB;MAAyB;QAAES,IAAIC,KAAKC;MAAG;;IACpFC,MAAMZ,YAAY,6BAA6B;IAC/Ca,aAAa;IACbC,QAAQ;EACV,GACAb,eAAAA,EAEDc,MAAK,GACV;IAACf;IAAWC;GAAgB;AAG9B,SACE,sBAAA,cAACe,MAAMC,MAAI;IAAC5C;KACV,sBAAA,cAAC6C,KAAKD,MAAI;IAAE,GAAGd;IAAa3B;KAC1B,sBAAA,cAACwC,MAAMG,SAAO;IAACC,SAAAA;KACb,sBAAA,cAACF,KAAKC,SAAO,IAAA,CAAA,CAAA,GAGjB,sBAAA,cAACH,MAAMK,SAAO;IAACD,SAAAA;KACb,sBAAA,cAACE,eAAAA;IAAczB;IAActB;;AAIrC;",
+  "names": ["React", "Filter", "Obj", "Query", "useMembers", "useQuery", "Panel", "Menu", "MenuBuilder", "useMenuBuilder", "Message", "Transcription", "useFeedModelAdapter", "useTranscriptionRecording", "meta", "TranscriptionArticle", "role", "subject", "transcript", "attendableId", "db", "Obj", "getDatabase", "members", "useMembers", "spaceId", "map", "member", "identity", "feed", "target", "messages", "useQuery", "Query", "select", "Filter", "type", "Message", "from", "nothing", "model", "useFeedModelAdapter", "renderByline", "recording", "toggleRecording", "useTranscriptionRecording", "menuActions", "useMenuBuilder", "MenuBuilder", "make", "action", "label", "ns", "meta", "id", "icon", "disposition", "testId", "build", "Panel", "Root", "Menu", "Toolbar", "asChild", "Content", "Transcription"]
+}

package/dist/lib/neutral/TranscriptionPlugin.mjs

@@ -0,0 +1,46 @@
+import "./chunk-J5LGTIGS.mjs";
+
+// src/TranscriptionPlugin.tsx
+import { Plugin } from "@dxos/app-framework";
+import { AppActivationEvents, AppPlugin } from "@dxos/app-toolkit";
+import { Transcript } from "@dxos/types";
+import { BlueprintDefinition, OperationHandler, ReactSurface, TextContent, Transcriber } from "#capabilities";
+import { meta } from "#meta";
+import { translations } from "#translations";
+
+// raw-loader:/__w/dxos/dxos/packages/plugins/plugin-transcription/PLUGIN.mdl?raw
+var PLUGIN_default = '---\nid: org.dxos.plugin.transcription\nname: TranscriptionPlugin\nversion: 0.1.0\n---\n\nA real-time voice-to-text transcription plugin for DXOS Composer.\nCaptures spoken audio from a microphone, converts it to timestamped text segments via the\nWhisper speech-recognition service, and stores the results in an ECHO `Transcript` object\nbacked by a `Feed`.\nThe plugin also exposes blueprint-compatible operations that let AI assistants open and\nsummarize transcripts inside agent workflows.\n\n## Extensions\n\nThe following extension dialects are used in this document.\nEach extension is defined in the Appendix or resolved via its URI.\n\n| Term        | URI                            |\n|-------------|--------------------------------|\n| `type`      | `org.dxos.mdl.type@1.0`       |\n| `feat`      | `org.dxos.mdl.feat@1.0`       |\n| `test`      | `org.dxos.mdl.test@1.0`       |\n| `component` | `org.dxos.mdl.component@1.0`  |\n| `op`        | `org.dxos.mdl.op@1.0`         |\n\n## Types\n\n```mdl\ntype TranscriptSegment\n  fields:\n    text: string              # transcribed text for this segment\n    started: string           # ISO-8601 wall-clock timestamp of segment start\n```\n\n```mdl\ntype Message\n  fields:\n    id: string\n    created: string           # ISO-8601 creation timestamp\n    blocks: TranscriptSegment[]\n    sender:\n      identityDid?: string    # DID of the speaker, if known\n```\n\n```mdl\ntype Transcript\n  fields:\n    id: string\n    feed: Ref<Feed>           # ECHO feed that stores Message objects\n    started?: string          # ISO-8601 timestamp when recording started\n```\n\n```mdl\ntype WhisperWord\n  fields:\n    word: string\n    start: number             # seconds from start of audio chunk\n    end: number\n```\n\n```mdl\ntype WhisperSegment\n  fields:\n    text: string\n    start: number\n    end: number\n    no_speech_prob: number    # probability of silence; used to filter noise\n    words: WhisperWord[]\n```\n\n## Components\n\n```mdl\ncomponent Transcription\n  desc: Read-only CodeMirror editor that renders transcript messages as Markdown with byline headings.\n  props:\n    transcript?: Transcript\n    model: SerializationModel<Message>\n  slots:\n    (none \u2014 self-contained editor viewport)\n  layout: |\n    \u250C\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n    \u2502 [CodeMirror editor \u2014 read-only] \u2502\n    \u2502  ## Speaker \xB7 HH:MM             \u2502\n    \u2502  Transcribed text\u2026              \u2502\n    \u2502  \u2026                              \u2502\n    \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n```mdl\ncomponent Oracle\n  desc: |\n    Ambient display surface with an SVG graph canvas and a centered text overlay.\n    Used to show the latest transcribed utterance alongside a live visualization.\n  props:\n    text?: string             # latest utterance to display\n    delay?: number            # per-character reveal delay in ms (default 12)\n  slots:\n    canvas?: ReactNode        # replaces default SVG.Graph when provided\n  layout: |\n    \u250C\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n    \u2502  [SVG canvas \u2014 absolutely fills]\u2502\n    \u2502         [Oracle.Text]           \u2502\n    \u2502   "latest utterance\u2026"           \u2502\n    \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n```mdl\ncomponent TranscriptionArticle\n  desc: Full-page article panel combining the Transcription editor with a start/stop recording toolbar action.\n  props:\n    subject: Transcript\n    role: string\n    attendableId?: string\n  state:\n    recording: boolean        # whether microphone capture is active\n  actions:\n    toggleRecording()\n  layout: |\n    \u250C\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n    \u2502 [Toolbar]  [\u{1F3A4} Start/Stop]      \u2502\n    \u251C\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n    \u2502 [Transcription editor]          \u2502\n    \u2502  \u2026                              \u2502\n    \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n## Operations\n\nPure operations \u2014 no side-effects.\n\n```mdl\nop openTranscript\n  desc: Loads a Transcript from ECHO, reads all Messages from its Feed, and returns the full text content rendered with speaker bylines.\n  input:\n    transcript: Ref<Transcript>\n  output: string\n  errors:\n    TranscriptNotFound: no Transcript object found for the given ref\n    FeedNotFound: the Transcript\'s feed ref cannot be resolved\n  requires: [Database, FeedService]\n```\n\nEffectful operations \u2014 produce output via AI or mutate persistent state.\n\n```mdl\nop createTranscript\n  desc: Creates a new Feed and a Transcript object linked to it, then writes both into the current ECHO space.\n  input:\n    name?: string\n    space: Space\n  output: Transcript\n  effects: [echo:write]\n```\n\n```mdl\nop summarizeTranscript\n  desc: Sends a transcript string and optional participant notes to an LLM and returns a Markdown summary with key points, verbatim quotes, and actionable tasks extracted from the conversation.\n  input:\n    transcript: string\n    notes?: string\n  output: string\n  effects: [http]\n  requires: [AiService]\n  note: |\n    Uses Claude Sonnet. Prompt instructs the model to bold speaker names,\n    include verbatim quotes, and emit only tasks that are directly actionable\n    and clearly assigned.\n```\n\n```mdl\nop normalizeSentences\n  desc: Post-processes an array of raw transcript messages, merging word-level tokens into grammatically complete sentences.\n  input:\n    messages: Message[]\n  output: Message[]\n  note: pure transformation; no effects or service dependencies\n```\n\n## Features\n\n```mdl\nfeat F-1: Recording Lifecycle\n\n  req F-1.1: User can start microphone capture from TranscriptionArticle toolbar.\n  req F-1.2: User can stop microphone capture; final buffered chunks are flushed and transcribed.\n  req F-1.3: TranscriptionManager restarts the Transcriber automatically when the audio track changes.\n```\n\n```mdl\nfeat F-2: Audio Chunking and Transcription\n\n  req F-2.1:\n    when: user is speaking and chunk count reaches transcribeAfterChunksAmount\n    then: Transcriber schedules a deferred transcription task\n\n  req F-2.2:\n    when: transcription task runs\n    then: buffered PCM chunks are merged into a WAV file and POSTed to the Whisper endpoint\n\n  req F-2.3:\n    when: Whisper response is received\n    then: segments are filtered by no_speech_prob and aligned to absolute wall-clock timestamps\n\n  req F-2.4:\n    when: user is not speaking\n    then: only prefixBufferChunksAmount most-recent chunks are retained; older chunks discarded\n```\n\n```mdl\nfeat F-3: Transcript Persistence\n\n  req F-3.1:\n    when: aligned segments are produced\n    then: a Message object with the segments is appended to the Transcript\'s ECHO Feed\n\n  req F-3.2:\n    when: a messageEnricher is configured\n    then: Message is passed through the enricher before being written to the Feed\n\n  req F-3.3:\n    when: a peer\'s ECHO space receives the Feed update\n    then: TranscriptionArticle re-renders with the new messages\n    tags: [collaborative]\n```\n\n```mdl\nfeat F-4: Transcript Display\n\n  req F-4.1:\n    when: a Transcript object is opened\n    then: all Messages are rendered in the Transcription editor with speaker bylines and timestamps\n\n  req F-4.2:\n    when: new messages arrive during live recording\n    then: the SerializationModel is updated and the editor scrolls to show the latest content\n```\n\n```mdl\nfeat F-5: AI Summary\n\n  req F-5.1:\n    when: op:summarizeTranscript is invoked\n    then: LLM returns a structured Markdown document with key points, quotes, and tasks\n\n  req F-5.2:\n    when: participant notes are provided\n    then: notes are correlated with transcript content in the summary\n\n  req F-5.3:\n    when: no actionable tasks are found\n    then: the tasks section is omitted from the summary\n```\n\n```mdl\nfeat F-6: Blueprint Integration\n\n  req F-6.1: Blueprint exposes op:openTranscript and op:summarizeTranscript as AI tools.\n  req F-6.2: An AI assistant can open a transcript by Ref and return formatted text to the user.\n  req F-6.3: An AI assistant can invoke the summary operation and present the result as a message.\n```\n\n## Acceptance\n\n```mdl\ntest T-1: Create transcript\n  given: a DXOS space is available\n  when: op:createTranscript is invoked\n  then:\n    - a new Feed is created in the space\n    - a Transcript object linked to the Feed is created in the space\n    - the returned Transcript.feed.target is not null\n```\n\n```mdl\ntest T-2: Record and persist segment\n  given: a Transcript with a Feed, microphone capture active\n  when: the user speaks and silence is detected\n  then:\n    - Whisper returns at least one segment\n    - a Message is appended to the Feed\n    - Message.blocks contains at least one TranscriptSegment with non-empty text\n```\n\n```mdl\ntest T-3: Deduplicate segments across chunk boundaries\n  given: two consecutive transcription tasks with overlapping Whisper output\n  when: the second task produces segments whose end timestamps fall before lastTimestamp\n  then:\n    - duplicate segments are filtered out\n    - only novel words (start >= lastTimestamp) are included\n```\n\n```mdl\ntest T-4: Open transcript\n  given: a Transcript with three Messages in its Feed\n  when: op:openTranscript is invoked\n  then:\n    - content string contains three byline sections\n    - each section starts with "## <speaker> \xB7 <time>"\n```\n\n```mdl\ntest T-5: Summarize transcript\n  given: a transcript string and optional notes string\n  when: op:summarizeTranscript is invoked\n  then:\n    - summary is a non-empty Markdown string\n    - speaker names appear in bold\n    - a tasks section is present if actionable tasks were detected\n```\n\n```mdl\ntest T-6: Sentence normalization is pure\n  given: an array of raw word-level Messages\n  when: op:normalizeSentences is called\n  then:\n    - output messages contain complete sentences\n    - no ECHO writes occur\n    - input array is not mutated\n```\n\n```mdl\ntest T-7: Toggle recording in UI\n  given: TranscriptionArticle is mounted with a Transcript\n  when: user clicks the Start Recording toolbar button\n  then:\n    - recording state becomes true\n    - toolbar button label changes to "Stop Recording"\n    - microphone capture begins\n```\n\n---\n\n## Appendix: Extension Definitions\n\nExtension block types used in this document are defined below using\nthe core `ext` primitive \u2014 the only construct the base language provides.\n\n```mdl\next type\n  uri: org.dxos.mdl.type@1.0\n  desc: A named data structure with typed fields and optional literals.\n  fields:\n    desc?: Prose\n    fields?: FieldMap        # name[?]: TypeExpr  (# inline comment)\n    literals?: UnionList     # a | b | c\n    extends?: TypeRef[]\n```\n\n```mdl\next feat\n  uri: org.dxos.mdl.feat@1.0\n  desc: A named feature grouping one or more requirements.\n  fields:\n    desc?: Prose\n    req: RequirementList\n  nesting: self              # feat blocks may contain feat blocks\n```\n\n```mdl\next test\n  uri: org.dxos.mdl.test@1.0\n  desc: An acceptance scenario expressed as given / when / then steps.\n  fields:\n    given?: Step | Step[]\n    when?: Step | Step[]\n    then: Step | Step[]\n    tags?: TagList\n```\n\n```mdl\next component\n  uri: org.dxos.mdl.component@1.0\n  desc: A UI component with props, internal state, slots, actions, and events.\n  fields:\n    desc?: Prose\n    props?: FieldMap            # external inputs (immutable inside component)\n    state?: FieldMap            # internal reactive state\n    slots?: FieldMap            # named ReactNode injection points\n    actions?: ActionMap         # methods the component exposes or handles\n    emits?: EventMap            # events the component raises to its parent\n    layout?: CodeBlock          # ASCII sketch of visual structure (non-normative)\n```\n\n```mdl\next op\n  uri: org.dxos.mdl.op@1.0\n  desc: |\n    A named operation with typed inputs, outputs, and declared errors.\n    Pure ops have no effects or requires. Effectful ops declare both.\n  fields:\n    desc?: Prose\n    input?: FieldMap            # named input parameters\n    output?: TypeExpr           # return type\n    errors?: ErrorMap           # name: Prose  (when this error occurs)\n    effects?: EffectList        # echo:read | echo:write | http | fs | ...\n    requires?: ServiceList      # injected service dependencies\n    note?: Prose                # implementation guidance (non-normative)\n```\n';
+
+// src/TranscriptionPlugin.tsx
+var TranscriptionPlugin = Plugin.define(meta).pipe(AppPlugin.addBlueprintDefinitionModule({
+  activate: BlueprintDefinition
+}), AppPlugin.addTextContentModule({
+  activate: TextContent
+}), AppPlugin.addOperationHandlerModule({
+  activate: OperationHandler
+}), AppPlugin.addSchemaModule({
+  schema: [
+    Transcript.Transcript
+  ]
+}), AppPlugin.addSurfaceModule({
+  activate: ReactSurface
+}), AppPlugin.addTranslationsModule({
+  translations
+}), Plugin.addModule({
+  id: "transcription",
+  activatesOn: AppActivationEvents.SetupAppGraph,
+  activate: Transcriber
+}), AppPlugin.addPluginAssetModule({
+  asset: {
+    pluginId: meta.id,
+    path: "PLUGIN.mdl",
+    content: PLUGIN_default,
+    mimeType: "application/x-mdl"
+  }
+}), Plugin.make);
+var TranscriptionPlugin_default = TranscriptionPlugin;
+export {
+  TranscriptionPlugin,
+  TranscriptionPlugin_default as default
+};
+//# sourceMappingURL=TranscriptionPlugin.mjs.map