alt-plugin-sdk 0.2.1 → 0.2.3

package/llms.txt

@@ -193,8 +193,8 @@ major `1` semantics until you opt in.
 ## 5. Permissions
 
 A plugin only gets exactly what its manifest declares. Calling a method
-without its permission throws `Plugin {id} lacks permission "{perm}"` at the
-IPC boundary and never touches the database, file system, or models.
+without its permission throws `Plugin permission required: {permission}` at
+the IPC boundary and never touches the database, file system, or models.
 
 | Permission          | Unlocks                                                                                                                  |
 | ------------------- | ------------------------------------------------------------------------------------------------------------------------ |
@@ -205,13 +205,14 @@ IPC boundary and never touches the database, file system, or models.
 | `notes:write`       | `alt.notes.create / update / delete / setMemo / setSummary / appendTranscriptLine / upsertComponent / deleteComponent`   |
 | `notes:select`      | `alt.notes.select(...)` — focuses a note in Alt's main window                                                            |
 | `folders:write`     | `alt.folders.create / rename / move / delete`                                                                            |
-| `ai:chat`           | `alt.ai.models.list / stream / chat.stream / complete / summarize`                                                       |
+| `ai:chat`           | `alt.ai.stream / chat.stream / complete / summarize`. (Note: `alt.ai.models.list` is NOT gated — any plugin can call it.) |
 | `recording:control` | `alt.recording.start / stop / getStatus`                                                                                 |
 | `transcription:run` | `alt.transcription.transcribeFile / transcribeNote`                                                                      |
 | `files:read`        | `alt.files.list / read`                                                                                                  |
 | `files:write`       | `alt.files.attach / delete`                                                                                              |
 | `settings:read`     | `alt.settings.get / list`                                                                                                |
 | `actions:notes`     | **Legacy.** Old plugins that declare this auto-get `notes:write` + `notes:select`. New plugins should declare the modern names. |
+| `settings:write`    | **Reserved.** Accepted by the manifest schema for forward compatibility, but no method currently requires it. Don't declare it. |
 
 Declare the **minimum** permissions you need. Users see the list before
 install and can later disable a plugin if its permissions feel off.
@@ -232,9 +233,12 @@ keeps working even if the host bridge is initialized after your code starts.
 
 ### 6.1 `alt.storage` — JSON KV per plugin
 
-Plugin-scoped, persisted to electron-store inside the host. Values are deep
-JSON: strings, numbers, booleans, `null`, arrays, and plain objects. No
-binary, no Map/Set, no circular refs.
+Plugin-scoped, persisted to Alt's SQLite database (in the `plugin_storage`
+table, one row per `(pluginId, key)`). Values are deep JSON: strings,
+numbers, booleans, `null`, arrays, and plain objects. No binary, no
+Map/Set, no circular refs. Large blobs belong in `alt.files.attach` —
+the storage row is JSON-encoded and meant for small structured state
+(preferences, indices, drafts).
 
 ```ts
 type Counter = { count: number; updatedAt: string };
@@ -300,7 +304,7 @@ Event types and their payload (`PluginHostEventPayload` union):
 | `folderCreated`          | `PluginFolderNode`                                                   | New folder                                       |
 | `folderUpdated`          | `PluginFolderNode`                                                   | Folder renamed or moved                         |
 | `folderDeleted`          | `{ folderId }`                                                       | Folder removed                                   |
-| `componentUpdated`       | `PluginNoteComponentSummary`                                         | Component (memo/summary/transcript/etc) changed |
+| `componentUpdated`       | `PluginNoteComponentSummary`                                         | Component (memo/summary/transcript/etc) created or updated. Note: NOT fired on `deleteComponent` — there's no `componentDeleted` event today; subscribe to `noteUpdated` or re-list if you need to track deletions. |
 | `settingChanged`         | `{ key, value }`                                                     | One of the curated app settings updated         |
 | `recordingLevel`         | `{ micDb, systemDb, timestamp }`                                     | Reserved — not currently emitted                 |
 
@@ -326,6 +330,23 @@ const recent = await alt.notes.list({ limit: 20 });
 const filtered = await alt.notes.list({ folderId: 7, query: 'midterm', limit: 50 });
 const content = await alt.notes.getContent(recent[0].id);
 // { id, title, transcript, memo, summary }
+//
+// `transcript`, `memo`, `summary` are all PLAIN STRINGS, already converted
+// from the host's internal storage format for easy LLM consumption:
+//
+//   transcript: LLM-formatted plaintext, one line per entry. Examples:
+//                 [0:00] Hello everyone, welcome to today's meeting.
+//                 [0:06 | Speaker 1] I have the numbers ready.
+//                 [1:23:45 | SPEAKER_0] And in chapter four...
+//               (`[m:ss]` or `[h:mm:ss]` for >= 1h. Speaker is appended after
+//               ` | ` when present. This is NOT structured JSON — if you need
+//               raw timing/speaker data, use getComponent on the transcript
+//               component instead, see below.)
+//
+//   memo:      Markdown, already converted from Alt's internal Plate-JSON
+//              rich-text format. Pass back to setMemo as-is.
+//
+//   summary:   Markdown, same treatment as memo.
 
 // Write
 const created = await alt.notes.create({
@@ -348,12 +369,61 @@ await alt.notes.appendTranscriptLine({
 // Focus a note in Alt's main window (requires notes:select)
 await alt.notes.select({ noteId });
 
-// Components — fine-grained content blocks attached to a note
+// Components — fine-grained content blocks attached to a note.
+// `getComponent().contentText` is the RAW value as stored by the host — it
+// is NOT pre-converted like getContent() is. The exact shape depends on
+// componentType:
+//
+//   transcript      JSON-stringified `TranscriptEntry[]`. Parse with
+//                   JSON.parse to get structured speaker/timing data.
+//                   This is the right source for anything that needs to
+//                   line up with the recording — getContent().transcript
+//                   has already been collapsed to plaintext.
+//   memo, summary   Plate-JSON (Alt's internal rich-text format). Treat
+//                   as opaque — there are no SDK helpers to manipulate
+//                   Plate nodes. To read these as markdown, use
+//                   getContent() / getMemo() instead. To write, use
+//                   setMemo() / setSummary() with markdown.
+//   slide_summary,  Plain text/markdown — exactly what was last upserted
+//   meeting_notes   via upsertComponent({ contentText, ... }).
+//   slides,         File-backed components. `contentText` is `null`. Use
+//   recording       alt.files.read({ fileId }) for the bytes; the fileId
+//                   comes from alt.files.list({ noteId }).
+
 const components = await alt.notes.listComponents({ noteId });
+
 const memoComponent = await alt.notes.getComponent({
   componentId: components.find(c => c.componentType === 'memo')!.id,
 });
-// memoComponent.contentText holds the markdown.
+// memoComponent.contentText is Plate JSON; use alt.notes.getMemo() or
+// alt.notes.getContent() if you want markdown instead.
+
+// Structured transcript (speaker + ms timing) — see recipe in §8 below.
+// The raw transcript contentText is JSON-stringified entries with this
+// internal shape (NOT the SDK's `PluginTranscriptionSegment`, which uses
+// `startMs`/`endMs` — the stored shape uses `start`/`end`):
+interface RawTranscriptSegment {
+  start: number;          // ms from recording start
+  end: number;            // ms from recording start
+  text: string;
+  speaker?: string;       // e.g. 'SPEAKER_0', 'Speaker 1'
+  translatedText?: string;
+}
+interface RawTranscriptEntry {
+  relativeStart?: number; // ms from recording start (entry-level)
+  speaker?: string;
+  segments?: RawTranscriptSegment[];
+  originalText?: string;
+  translatedText?: string;
+  createdAt: number;      // unix ms when the entry was first appended
+}
+
+const transcriptComp = components.find(c => c.componentType === 'transcript');
+if (transcriptComp) {
+  const raw = await alt.notes.getComponent({ componentId: transcriptComp.id });
+  const entries: RawTranscriptEntry[] = JSON.parse(raw.contentText ?? '[]');
+  // entries[i].relativeStart, entries[i].speaker, entries[i].segments[j].text
+}
 
 // Singleton component types (memo / summary / transcript) auto-upsert —
 // calling upsertComponent for type=memo on a note that already has a memo
@@ -379,8 +449,12 @@ Limits:
 - `markdown` body (memo/summary) — ≤ 500,000 chars.
 - `appendTranscriptLine.text` — 1–10,000 chars.
 - `upsertComponent.contentText` — ≤ 1,000,000 chars.
-- `setMemo` / `setSummary` / `appendTranscriptLine` and `upsertComponent`
-  return `{ componentId }` of the (possibly newly created) component.
+- `setMemo` / `setSummary` / `appendTranscriptLine` return `{ componentId }`
+  of the (possibly newly created) component.
+- `upsertComponent` returns the full `PluginNoteComponentSummary` (`id`,
+  `noteId`, `componentType`, `title`, `displayOrder`, `hasFile`,
+  `createdAt`, `updatedAt`) of the upserted component — not just
+  `{ componentId }`.
 
 ### 6.5 `alt.folders` — manage the note tree
 
@@ -465,11 +539,25 @@ fileHandle.cancel();
 
 Error codes (`PluginTranscriptionStreamErrorCode`):
 
-- `FORBIDDEN` — plugin lacks `transcription:run`
-- `INVALID_REQUEST` — bad params (e.g. unknown file path, invalid noteId)
-- `RECORDING_ACTIVE` — Alt is currently recording; can't run transcription
-- `FILE_UNAVAILABLE` — the file was deleted or unreadable
-- `PIPELINE_ERROR` — Whisper/diarization pipeline crashed
+- `FORBIDDEN` — plugin lacks `transcription:run`.
+- `INVALID_REQUEST` — request fails schema validation (missing/empty
+  `filePath`, non-positive `noteId`, etc.). Note: this is shape-only —
+  the host does NOT pre-check that a `filePath` exists; a non-existent
+  or unreadable path surfaces later as `PIPELINE_ERROR`.
+- `RECORDING_ACTIVE` — Alt is currently recording; can't run transcription.
+- `FILE_UNAVAILABLE` — only from `transcribeNote`. Fires when the target
+  note has no `recording` component or the file_inode can't be resolved.
+  A deleted file passed to `transcribeFile` does NOT fire this code —
+  it surfaces as `PIPELINE_ERROR` (the Whisper pipeline can't open it).
+- `PIPELINE_ERROR` — Whisper/diarization pipeline crashed, OR the file
+  couldn't be opened (see `FILE_UNAVAILABLE` note above).
+
+Also note the defaults the host applies when you omit optional fields:
+
+- `language` falls back to the user's `transcription.lectureLanguage`
+  setting (typically `'en'`). You don't need to pass it unless you want
+  to override.
+- `diarization` defaults to `false` when omitted.
 
 ### 6.8 `alt.ai` — chat completions and summarize
 
@@ -518,8 +606,11 @@ const handle = await alt.ai.chat.stream(
 );
 // handle.cancel() to abort.
 
-// Run Alt's own summarize prompt against a note. Pulls the transcript+memo,
-// runs through the configured model, returns markdown.
+// Run Alt's own summarize prompt against a note. The host composes the
+// prompt from up to three sources on the note — memo, transcript, and any
+// existing summary (so re-running it refines an earlier output rather than
+// starting from scratch). Each is included only if non-empty. Throws if
+// all three are empty. Runs through the chosen model, returns markdown.
 const sum = await alt.ai.summarize({
   noteId,
   style: 'Focus on definitions and worked examples.',
@@ -553,38 +644,48 @@ const bytes: ArrayBuffer = await someFile.arrayBuffer();
 const attached = await alt.files.attach({
   noteId,
   fileName: 'lecture12.pdf',
-  data: bytes,                  // max 256 MB
+  data: bytes,                  // max 256 MiB (256 * 1024 * 1024 bytes)
   mimeType: 'application/pdf',
   componentType: 'slides',      // 'slides' | 'recording'
   title: 'Lecture 12 — Slides',
   displayOrder: 0,
 });
 // attached: { componentId, fileId, fileName, mimeType, sizeBytes }
+//   mimeType is `string | null` — null when the caller omitted it.
 
 const list = await alt.files.list({ noteId });
 const blob = await alt.files.read({ fileId: list[0].fileId });
-// { fileName, mimeType, sizeBytes, data: ArrayBuffer }
+// { fileName, mimeType: string | null, sizeBytes, data: ArrayBuffer }
 
 await alt.files.delete({ componentId: attached.componentId });
 ```
 
 Constraints:
 
-- `fileName` is ≤ 260 chars and **must not contain** path separators or
-  control characters. Host re-validates with a strict regex; bad names
-  throw.
+- `fileName` is 1–260 chars and must match `^[^\\/\x00\n\r:*?"<>|]+$` —
+  blocks `\ / : * ? " < > |` and the NUL/LF/CR control bytes (Windows
+  reserved chars plus path separators). Other control bytes are
+  technically allowed by this regex but are still a bad idea. Host
+  re-validates; bad names throw.
 - `componentType` is restricted to `slides | recording` because the database
   only allows file-backed components for those types. Use
   `notes.upsertComponent` for text-based components instead.
-- `data` is an `ArrayBuffer` capped at 256 MiB.
+- `data` is an `ArrayBuffer` capped at 256 MiB (`256 * 1024 * 1024`).
 
 ### 6.10 `alt.settings` — read curated app settings
 
-Permission: `settings:read`. Read-only allowlist. There is **no
-settings:write**; if you need plugin-owned settings, use `alt.storage`.
+Permission: `settings:read`. Read-only allowlist — there is no
+write surface on the host today (the `settings:write` permission is
+reserved but unused). If you need plugin-owned settings, use
+`alt.storage`.
 
 ```ts
-const theme = await alt.settings.get('theme'); // 'system' | 'light' | 'dark' | null
+const theme = await alt.settings.get('theme');
+// theme is typed `PluginAppSettingValue` = `string | number | boolean | null`.
+// The SDK does NOT narrow per-key — you have to check at runtime.
+// Typical values seen here: 'system' | 'light' | 'dark'. `null` if unset.
+if (typeof theme === 'string') applyTheme(theme);
+
 const all = await alt.settings.list();
 // {
 //   theme: 'dark',
@@ -628,9 +729,13 @@ const res = await altFetch('https://alt-plugin.invalid/v1/chat/completions', {
 // Standard streaming Response — use res.body.getReader() or res.text().
 ```
 
-`Authorization` and `x-machine-id` headers are stripped before the request
-reaches the host, so SDKs that try to inject API keys won't accidentally
-exfiltrate them.
+Headers are filtered twice on the way to the cloud. First in the SDK,
+`Authorization` and `x-machine-id` are stripped so AI-SDK clients that
+auto-inject API keys can't exfiltrate them. Then the host applies a
+strict allowlist: only `accept` and `content-type` survive — everything
+else (custom routing hints, request IDs, telemetry headers) is silently
+dropped. Don't rely on passing arbitrary headers through `createAltFetch`;
+put any plugin-side metadata in the request body instead.
 
 ### `createAltProvider(options?)`
 
@@ -665,8 +770,11 @@ Use this whenever you'd normally instantiate `@ai-sdk/openai` etc. Plugins
 
 ### 8.1 Live transcript subscriber
 
-Watch active-note changes, fetch its transcript whenever it updates, render
-the last 10 lines.
+Watch active-note changes, fetch its transcript whenever it updates,
+render the last 10 transcript lines. Note `getContent().transcript` is
+the LLM-formatted plaintext (`[m:ss | Speaker] text` per line) — slicing
+by `\n` is fine here since each entry is exactly one line, but if you
+need structured timing data go through §8.5 instead.
 
 ```ts
 // permissions: ['notes:read', 'events:subscribe', 'appState:read']
@@ -683,21 +791,33 @@ const refresh = async (noteId: number) => {
   render(lines);
 };
 
-await alt.events.subscribe('activeNoteChanged', note => {
+// IMPORTANT: hold on to the unsubscribe handles. Leaked subscriptions
+// stay alive for the lifetime of the plugin window. Call them in your
+// teardown path (e.g. `beforeunload`, your framework's unmount hook).
+const unsubActive = await alt.events.subscribe('activeNoteChanged', note => {
   currentNoteId = note?.id ?? null;
   if (currentNoteId !== null) void refresh(currentNoteId);
   else render(['(no active note)']);
 });
 
-await alt.events.subscribe('transcriptUpdated', ({ noteId }) => {
-  if (noteId === currentNoteId) void refresh(noteId);
-});
+const unsubTranscript = await alt.events.subscribe(
+  'transcriptUpdated',
+  ({ noteId }) => {
+    if (noteId === currentNoteId) void refresh(noteId);
+  },
+);
 
 const active = await alt.state.getActiveNoteSummary();
 if (active) {
   currentNoteId = active.id;
   await refresh(active.id);
 }
+
+// On teardown:
+window.addEventListener('beforeunload', () => {
+  void unsubActive();
+  void unsubTranscript();
+});
 ```
 
 ### 8.2 Quiz from a note (AI + storage)
@@ -734,7 +854,9 @@ export async function buildFlashcards(noteId: number) {
 ### 8.3 Transcribe an arbitrary audio file, then save as a new note
 
 ```ts
-// permissions: ['notes:write', 'transcription:run', 'notes:select']
+// permissions: ['notes:write', 'transcription:run']
+// Note: no `notes:select` needed — the host auto-focuses freshly created
+// notes in the main window as part of `notes.create`.
 import { alt } from 'alt-plugin-sdk';
 
 export async function importAudio(filePath: string, title: string) {
@@ -757,7 +879,6 @@ export async function importAudio(filePath: string, title: string) {
           title: 'Imported transcript',
           contentText: segments.join('\n'),
         });
-        await alt.notes.select({ noteId });
       },
       onError: err => console.error(err.code, err.message),
     },
@@ -768,7 +889,7 @@ export async function importAudio(filePath: string, title: string) {
 ### 8.4 Attach a PDF chosen via `<input type="file">`
 
 ```ts
-// permissions: ['files:write']
+// permissions: ['files:write', 'appState:read']
 import { alt } from 'alt-plugin-sdk';
 
 document.querySelector<HTMLInputElement>('#picker')!.addEventListener(
@@ -790,6 +911,76 @@ document.querySelector<HTMLInputElement>('#picker')!.addEventListener(
 );
 ```
 
+### 8.5 Structured transcript with speaker + ms timing
+
+`alt.notes.getContent(noteId).transcript` returns LLM-formatted plaintext
+(`[0:06 | Speaker 1] ...`). That's the wrong source when you need to line
+up text with the recording, or render a speaker-labeled UI similar to
+Alt's own transcript view. For that, go through the raw transcript
+component:
+
+```ts
+// permissions: ['notes:read']
+import { alt } from 'alt-plugin-sdk';
+
+interface RawTranscriptSegment {
+  start: number;            // ms from recording start
+  end: number;              // ms from recording start
+  text: string;
+  speaker?: string;         // 'SPEAKER_0', 'Speaker 1', etc.
+  translatedText?: string;
+}
+interface RawTranscriptEntry {
+  relativeStart?: number;   // ms; entry-level start
+  speaker?: string;
+  segments?: RawTranscriptSegment[];
+  originalText?: string;
+  translatedText?: string;
+  createdAt: number;        // unix ms
+}
+
+async function getStructuredTranscript(
+  noteId: number,
+): Promise<RawTranscriptEntry[]> {
+  const components = await alt.notes.listComponents({ noteId });
+  const transcript = components.find(c => c.componentType === 'transcript');
+  if (!transcript) return [];
+
+  const raw = await alt.notes.getComponent({ componentId: transcript.id });
+  if (!raw.contentText) return [];
+
+  try {
+    const parsed = JSON.parse(raw.contentText);
+    return Array.isArray(parsed) ? (parsed as RawTranscriptEntry[]) : [];
+  } catch {
+    return [];
+  }
+}
+
+// Use it:
+const entries = await getStructuredTranscript(noteId);
+for (const entry of entries) {
+  for (const seg of entry.segments ?? []) {
+    console.log(
+      `${seg.start}ms - ${seg.end}ms [${seg.speaker ?? '?'}]:`,
+      seg.text,
+    );
+  }
+}
+```
+
+Two important caveats:
+
+- Diarization is opt-in. If the user recorded without diarization
+  enabled, `speaker` will be absent on most entries — fall back to a
+  single-speaker render.
+- The internal segment shape is `{ start, end }` (milliseconds), NOT the
+  SDK's `PluginTranscriptionSegment` (which uses `startMs` / `endMs`).
+  The plugin SDK type is for the streaming transcription API; the stored
+  format is older and predates that naming choice. Use the inline
+  `RawTranscriptSegment` type above (or copy it into your code) rather
+  than reusing the SDK type — the field names don't match.
+
 ---
 
 ## 9. Errors
@@ -866,13 +1057,31 @@ The npm package follows semver:
   also subscribe to. Polling `getStatus()` 10× a second is wasteful when
   `recordingStatusChanged` exists.
 - **Cap your storage.** `alt.storage` is plugin-scoped but lives in the
-  user's electron-store on disk. Anything over a few MB belongs in
+  user's SQLite database on disk. Anything over a few MB belongs in
   `alt.files.attach`.
 - **Plate JSON is not exposed.** Memo/summary content is round-tripped
   through Alt's internal Plate-rich-text codec; the SDK gives you the
   markdown view via `getContent` / `getMemo` and accepts markdown back via
   `setMemo` / `setSummary`. There is no API to manipulate Plate nodes
-  directly.
+  directly. If you call `getComponent` on a memo or summary you get the
+  raw Plate JSON in `contentText` — don't try to parse it, use the
+  markdown surfaces instead.
+- **`getContent().transcript` is plaintext, not JSON.** It's the same
+  LLM-friendly format Alt uses internally to feed transcripts to LLMs
+  (lines like `[0:06 | Speaker 1] text`). If you need structured
+  speaker/timing data (to render a transcript UI, jump to a timestamp,
+  group by speaker, etc.), don't try to parse `getContent().transcript`
+  — go through `listComponents` → find the `transcript` component →
+  `getComponent` → `JSON.parse(contentText)`. See §8.5 for a worked
+  example. The inline JSON shape is internal and differs from
+  `PluginTranscriptionSegment` (it uses `start`/`end` ms, not `startMs`/
+  `endMs`).
+- **Components have different `contentText` shapes per type.** The same
+  `getComponent` API gives you Plate JSON for `memo`/`summary`,
+  JSON-stringified entries for `transcript`, plain text for
+  `slide_summary`/`meeting_notes`, and `null` for file-backed types
+  (`slides`/`recording`). Always branch on `componentType` before
+  touching `contentText`.
 
 ---
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "alt-plugin-sdk",
-  "version": "0.2.1",
+  "version": "0.2.3",
   "description": "Type-safe browser SDK and runtime contracts for Alt plugins.",
   "license": "MIT",
   "author": "Alt (https://altalt.io)",