@mulmocast/slide 0.1.7 → 0.3.0

package/.claude/skills/extend/SKILL.md

@@ -0,0 +1,120 @@
+# /extend - MulmoScript to ExtendedScript Conversion
+
+Convert a MulmoScript JSON into an ExtendedScript by adding `scriptMeta` and `beats[].meta` metadata fields. The metadata is used by mulmocast-preprocessor's AI features (summarize, query).
+
+## Invocation
+
+```
+/extend <mulmo_script.json path> [--source <source file path>]
+```
+
+## Instructions
+
+### Step 1: Read Inputs
+
+1. Read the MulmoScript JSON at the specified path
+2. Read the ExtendedScript schema: `.claude/skills/extend/references/extended-script-schema.md`
+3. Check for extracted texts file:
+   - Look for `extracted_texts.json` in the same directory as the MulmoScript
+   - This file is generated by `mulmo-slide pdf` and contains per-page text extracted from the source PDF
+   - If found, read it — this provides the raw text content for each beat
+4. Locate the source file:
+   - If `--source` is specified, use that file
+   - Otherwise, infer from the MulmoScript path:
+     - `scripts/{basename}/mulmo_script.json` -> search for `samples/{basename}.*` (try `.md`, `.pptx`, `.pdf`, `.key`)
+   - If no source file found, work from the MulmoScript content and extracted texts alone
+5. If a source file is found, read it
+
+### Step 2: Analyze Content
+
+Analyze the MulmoScript beats and source file (if available) to understand:
+
+- The overall theme and purpose of the presentation
+- The target audience
+- The logical structure (sections, flow)
+- Content types in each slide (text, code, diagrams, tables, data)
+- URLs and external references mentioned
+- Key terminology and concepts
+
+For Markdown sources, also examine:
+- Header hierarchy and structure
+- Speaker notes (after `---` or in HTML comments)
+- Code blocks and their languages
+- Mermaid diagrams
+- Links and references
+
+### Step 3: Generate Metadata
+
+Based on the analysis, generate:
+
+**`scriptMeta`** (script-level):
+- `background`: 1-2 sentence overview of the presentation's theme
+- `audience`: Who this presentation is for
+- `goals`: 2-4 learning objectives or presentation goals
+- `keywords`: 5-10 main keywords for search/discovery
+- `references`: Extract URLs from slides, categorize as web/code/document/video
+- `author`: If identifiable from content
+- `faq`: 2-4 likely questions with answers based on the content
+
+**`beats[].meta`** (per-beat):
+- `section`: Logical section name (e.g., "opening", "introduction", "main-topic-1", "demo", "closing")
+  - Use consistent naming: consecutive beats in the same logical section share the same section value
+- `tags`: Content type and topic tags. Use from this vocabulary when applicable:
+  - Content type: `intro`, `overview`, `definition`, `example`, `code`, `diagram`, `data`, `table`, `demo`, `comparison`, `summary`, `conclusion`, `q-and-a`
+  - Add topic-specific tags as needed
+- `keywords`: 2-5 beat-specific important terms
+- `notes`: If `extracted_texts.json` exists, put the corresponding extracted text for that beat here. This preserves the raw source content as metadata.
+- `context`: Background information that would help an AI answer questions about this beat. Add supplementary information: related concepts, technical details, connections to other topics. This is the most valuable field - be substantive.
+- `expectedQuestions`: 1-3 questions the audience might ask about this specific beat's content
+
+**`beats[].text`** (narration):
+- If the beat's `text` field is empty and `extracted_texts.json` is available, generate a concise narration text based on the extracted text. The narration should be a natural spoken summary of the slide content, NOT a verbatim copy of the extracted text.
+- If the beat already has `text`, preserve it as-is.
+
+### Step 4: Build ExtendedScript
+
+1. Start with the original MulmoScript (preserve ALL existing fields exactly)
+2. Add `scriptMeta` at the top level
+3. Add `meta` to each beat
+4. Add `outputProfiles: {}` (empty, for user to configure later)
+5. If beats don't have `id` fields, add them (e.g., `"beat-1"`, `"beat-2"`, ...)
+
+### Step 5: Write and Validate Output
+
+1. Determine output path:
+   - Same directory as input: replace `mulmo_script.json` with `extended_script.json`
+   - Example: `scripts/simple_text/mulmo_script.json` -> `scripts/simple_text/extended_script.json`
+2. Write the JSON with 2-space indentation
+3. Run `mulmo-slide extend validate <output_path>` (or `yarn cli extend validate <output_path>`) to validate against the schema
+4. If validation fails, fix the errors and re-write the file, then validate again
+5. Generate MulmoScript from ExtendedScript using the preprocessor:
+   ```bash
+   npx mulmocast-preprocessor <extended_script.json> -o <mulmo_script.json>
+   ```
+   Note: The preprocessor may not fully strip `scriptMeta`. If `mulmo movie` fails with "Unrecognized key" errors, manually strip remaining fields with a node one-liner:
+   ```bash
+   node -e "const fs=require('fs');const d=JSON.parse(fs.readFileSync('<mulmo_script.json>','utf8'));delete d.scriptMeta;delete d.outputProfiles;d.beats.forEach(b=>{delete b.meta;delete b.variants});fs.writeFileSync('<mulmo_script.json>',JSON.stringify(d,null,2))"
+   ```
+6. Display a summary to the user:
+   - Number of beats processed
+   - Sections identified
+   - Key topics/keywords
+   - Output file path
+
+### Step 6: Offer Adjustments
+
+Ask the user if they want to adjust any of the generated metadata. Common adjustments:
+- Refine audience description
+- Add/remove keywords
+- Adjust FAQ entries
+- Modify section boundaries
+- Enhance context fields
+
+## Quality Guidelines
+
+- **context field**: This is the most important field for AI features. Don't just restate the slide text. Add supplementary information: related concepts, technical background, real-world examples, historical context, common misconceptions.
+- **section naming**: Use lowercase-kebab-case. Keep section names meaningful and consistent across beats.
+- **tags**: Be specific but not excessive. 2-4 tags per beat is typical.
+- **expectedQuestions**: Write natural questions a real audience member would ask, not generic ones.
+- **keywords**: Prefer specific technical terms over generic words.
+- **Preserve original content**: Never modify existing MulmoScript fields (text, image, speaker, etc.).

package/.claude/skills/extend/references/extended-script-schema.md

@@ -0,0 +1,153 @@
+# ExtendedScript Schema Reference
+
+This document defines the ExtendedScript format used by `mulmocast-preprocessor`.
+Canonical source: `@mulmocast/extended-types` npm package (`mulmocast-plus/packages/mulmocast-extended-types/src/index.ts`)
+
+## Type Definitions
+
+### BeatMeta
+
+Metadata for filtering and context, attached to each beat.
+
+```typescript
+{
+  tags?: string[];              // Content type/topic tags (e.g., "intro", "code", "diagram", "data", "demo", "conclusion")
+  section?: string;             // Logical section identifier (e.g., "opening", "chapter1", "closing")
+  context?: string;             // Background info not in the slide text (for AI summarize/query)
+  notes?: string;               // Raw extracted text from source document (e.g., PDF text)
+  keywords?: string[];          // Beat-specific search keywords
+  expectedQuestions?: string[]; // Questions the audience might ask about this beat
+}
+```
+
+### BeatVariant
+
+Profile-specific content overrides (not generated by `/extend`).
+
+```typescript
+{
+  text?: string;           // Override text for this profile
+  skip?: boolean;          // Skip this beat in this profile
+  image?: MulmoImageAsset; // Override image
+  imagePrompt?: string;    // Override image prompt
+}
+```
+
+### ExtendedBeat
+
+A beat with optional `variants` and `meta` fields (extends MulmoBeat).
+
+```typescript
+{
+  // ... all MulmoBeat fields preserved ...
+  variants?: Record<string, BeatVariant>;  // Profile-keyed overrides
+  meta?: BeatMeta;                         // Metadata for this beat
+}
+```
+
+### Reference
+
+External resource reference.
+
+```typescript
+{
+  type?: "web" | "code" | "document" | "video";
+  url: string;
+  title?: string;
+  description?: string;
+}
+```
+
+### FAQ
+
+Frequently asked question for quick Q&A matching.
+
+```typescript
+{
+  question: string;
+  answer: string;
+  relatedBeats?: string[];  // Beat IDs related to this FAQ
+}
+```
+
+### ScriptMeta
+
+Script-level metadata for AI features.
+
+```typescript
+{
+  audience?: string;              // Target audience description
+  prerequisites?: string[];       // Required knowledge
+  goals?: string[];               // Learning goals / presentation objectives
+  background?: string;            // Overview / theme of the presentation
+  faq?: FAQ[];                    // Frequently asked questions
+  keywords?: string[];            // Script-wide search keywords
+  references?: Reference[];       // External resource links
+  author?: string;                // Author name
+  version?: string;               // Version string
+}
+```
+
+### OutputProfile
+
+Profile display information (not generated by `/extend`).
+
+```typescript
+{
+  name: string;
+  description?: string;
+}
+```
+
+### ExtendedScript
+
+The top-level type (extends MulmoScript).
+
+```typescript
+{
+  // ... all MulmoScript fields preserved ...
+  beats: ExtendedBeat[];                              // Beats with meta
+  outputProfiles?: Record<string, OutputProfile>;      // Profile definitions
+  scriptMeta?: ScriptMeta;                             // Script-level metadata
+}
+```
+
+## Example
+
+```json
+{
+  "$mulmocast": { "version": "1.1" },
+  "title": "Example Presentation",
+  "lang": "en",
+  "speechParams": { "speakers": { "Presenter": { "voiceId": "alloy" } } },
+  "scriptMeta": {
+    "audience": "Software engineers interested in AI",
+    "goals": ["Understand LLM agent patterns", "Learn practical implementation approaches"],
+    "background": "LLM agents combine language models with tools and memory for autonomous task execution.",
+    "keywords": ["LLM", "agents", "AI"],
+    "references": [
+      {
+        "type": "document",
+        "url": "https://arxiv.org/abs/2210.03629",
+        "title": "ReAct Paper"
+      }
+    ],
+    "author": "AI Team"
+  },
+  "beats": [
+    {
+      "id": "intro-1",
+      "speaker": "Presenter",
+      "text": "Welcome to this presentation about LLM agents.",
+      "meta": {
+        "section": "opening",
+        "tags": ["intro"],
+        "keywords": ["LLM agents"],
+        "context": "This is the opening slide that sets the stage for the presentation.",
+        "expectedQuestions": ["What are LLM agents?"]
+      }
+    }
+  ],
+  "outputProfiles": {}
+}
+```

package/.claude/skills/narrate/SKILL.md

@@ -0,0 +1,145 @@
+# /narrate - Source File to Narrated ExtendedScript
+
+Convert any supported source file (PDF, PPTX, Markdown, Keynote) into a validated ExtendedScript with AI-generated narration and metadata. This is the main entry point for the full pipeline.
+
+## Invocation
+
+```
+/narrate <source file path>
+```
+
+Supported formats: `.pdf`, `.pptx`, `.md`, `.key`
+
+## Instructions
+
+### Step 1: Convert Source to MulmoScript
+
+Detect the file format and run the appropriate converter.
+
+**PDF:**
+```bash
+yarn cli pdf <file>
+```
+
+**PPTX:**
+```bash
+yarn cli pptx <file>
+```
+
+**Markdown:**
+```bash
+yarn cli markdown <file>
+```
+
+**Keynote (.key):**
+```bash
+yarn cli keynote <file>
+```
+
+This produces `scripts/{basename}/mulmo_script.json` and (for PDF) `scripts/{basename}/extracted_texts.json`.
+
+Confirm the output:
+- How many slides/beats were generated
+- Whether `extracted_texts.json` exists (PDF only)
+
+### Step 2: Read Inputs
+
+1. Read the generated `scripts/{basename}/mulmo_script.json`
+2. Read the ExtendedScript schema: `.claude/skills/narrate/references/extended-script-schema.md`
+3. Check for `scripts/{basename}/extracted_texts.json` (generated by PDF converter)
+   - If found, read it — this provides raw text content for each beat
+4. If the source is Markdown, also read the original `.md` file for speaker notes and structure
+
+### Step 3: Analyze Content
+
+Analyze the MulmoScript beats, extracted texts, and source file to understand:
+
+- The overall theme and purpose
+- The target audience
+- The logical structure (sections, flow)
+- Content types in each slide (text, code, diagrams, tables, data)
+- URLs and external references
+- Key terminology and concepts
+
+For slides with images, read the slide images to understand visual content that may not be in extracted text.
+
+### Step 4: Generate Narration and Metadata
+
+Based on the analysis, generate:
+
+**`scriptMeta`** (script-level):
+- `background`: 1-2 sentence overview of the theme
+- `audience`: Who this is for
+- `goals`: 2-4 learning objectives or goals
+- `keywords`: 5-10 main keywords
+- `references`: Extract URLs, categorize as web/code/document/video
+- `author`: If identifiable from content
+- `faq`: 2-4 likely questions with answers
+
+**`beats[].text`** (narration):
+- If the beat's `text` is empty, generate a concise narration based on the slide content and extracted text
+- The narration should be natural spoken language, NOT a verbatim copy of source text
+- Match the language specified in `lang` field of the MulmoScript
+- If the beat already has `text`, preserve it as-is
+
+**`beats[].meta`** (per-beat):
+- `section`: Logical section name (lowercase-kebab-case, e.g., "introduction", "main-topic-1", "conclusion")
+- `tags`: Content type tags from: `intro`, `overview`, `definition`, `example`, `code`, `diagram`, `data`, `table`, `demo`, `comparison`, `summary`, `conclusion`, `q-and-a`
+- `keywords`: 2-5 beat-specific terms
+- `notes`: If `extracted_texts.json` exists, put the raw extracted text here
+- `context`: Background info for AI query/summarize. Be substantive — don't just restate the slide
+- `expectedQuestions`: 1-3 natural audience questions
+
+### Step 5: Build and Write ExtendedScript
+
+1. Start with the original MulmoScript (preserve ALL existing fields)
+2. Add `scriptMeta` at the top level
+3. Add `meta` to each beat, set `text` for narration
+4. Add `outputProfiles: {}` (empty)
+5. If beats don't have `id` fields, add them (`"beat-1"`, `"beat-2"`, ...)
+6. Write to `scripts/{basename}/extended_script.json` with 2-space indentation
+
+### Step 6: Validate
+
+Run validation:
+```bash
+yarn cli extend validate scripts/{basename}/extended_script.json
+```
+
+If validation fails, fix the errors and re-write the file. Repeat until validation passes.
+
+### Step 7: Present Results and Next Steps
+
+Display a summary:
+- Number of beats processed
+- Sections identified
+- Key topics/keywords
+- Output file path
+
+Then show the user the next steps they can take:
+
+```
+ExtendedScript is ready! Here's what you can do next:
+
+## Query the content interactively
+npx mulmocast-preprocessor query scripts/{basename}/extended_script.json -i
+
+## Generate a summary
+npx mulmocast-preprocessor summarize scripts/{basename}/extended_script.json
+
+## Generate a narrated video
+npx mulmocast-preprocessor scripts/{basename}/extended_script.json -o scripts/{basename}/mulmo_script.json
+npx mulmo movie scripts/{basename}/mulmo_script.json
+```
+
+Ask the user if they want to adjust any narration or metadata before proceeding.
+
+## Quality Guidelines
+
+- **Narration**: Write as if presenting to an audience. Use clear, spoken language. Avoid reading raw data verbatim — summarize and explain instead.
+- **context field**: Most important for AI features. Add supplementary info: related concepts, technical background, real-world examples, historical context.
+- **section naming**: Use lowercase-kebab-case. Consecutive beats in the same logical section share the same section value.
+- **tags**: 2-4 tags per beat. Be specific but not excessive.
+- **expectedQuestions**: Natural questions a real audience member would ask.
+- **keywords**: Prefer specific technical terms over generic words.
+- **Preserve original content**: Never modify existing MulmoScript fields (image, speaker, etc.) except `text` when generating narration.

package/.claude/skills/narrate/references/extended-script-schema.md

@@ -0,0 +1,153 @@
+# ExtendedScript Schema Reference
+
+This document defines the ExtendedScript format used by `mulmocast-preprocessor`.
+Canonical source: `@mulmocast/extended-types` npm package (`mulmocast-plus/packages/mulmocast-extended-types/src/index.ts`)
+
+## Type Definitions
+
+### BeatMeta
+
+Metadata for filtering and context, attached to each beat.
+
+```typescript
+{
+  tags?: string[];              // Content type/topic tags (e.g., "intro", "code", "diagram", "data", "demo", "conclusion")
+  section?: string;             // Logical section identifier (e.g., "opening", "chapter1", "closing")
+  context?: string;             // Background info not in the slide text (for AI summarize/query)
+  notes?: string;               // Raw extracted text from source document (e.g., PDF text)
+  keywords?: string[];          // Beat-specific search keywords
+  expectedQuestions?: string[]; // Questions the audience might ask about this beat
+}
+```
+
+### BeatVariant
+
+Profile-specific content overrides (not generated by `/extend`).
+
+```typescript
+{
+  text?: string;           // Override text for this profile
+  skip?: boolean;          // Skip this beat in this profile
+  image?: MulmoImageAsset; // Override image
+  imagePrompt?: string;    // Override image prompt
+}
+```
+
+### ExtendedBeat
+
+A beat with optional `variants` and `meta` fields (extends MulmoBeat).
+
+```typescript
+{
+  // ... all MulmoBeat fields preserved ...
+  variants?: Record<string, BeatVariant>;  // Profile-keyed overrides
+  meta?: BeatMeta;                         // Metadata for this beat
+}
+```
+
+### Reference
+
+External resource reference.
+
+```typescript
+{
+  type?: "web" | "code" | "document" | "video";
+  url: string;
+  title?: string;
+  description?: string;
+}
+```
+
+### FAQ
+
+Frequently asked question for quick Q&A matching.
+
+```typescript
+{
+  question: string;
+  answer: string;
+  relatedBeats?: string[];  // Beat IDs related to this FAQ
+}
+```
+
+### ScriptMeta
+
+Script-level metadata for AI features.
+
+```typescript
+{
+  audience?: string;              // Target audience description
+  prerequisites?: string[];       // Required knowledge
+  goals?: string[];               // Learning goals / presentation objectives
+  background?: string;            // Overview / theme of the presentation
+  faq?: FAQ[];                    // Frequently asked questions
+  keywords?: string[];            // Script-wide search keywords
+  references?: Reference[];       // External resource links
+  author?: string;                // Author name
+  version?: string;               // Version string
+}
+```
+
+### OutputProfile
+
+Profile display information (not generated by `/extend`).
+
+```typescript
+{
+  name: string;
+  description?: string;
+}
+```
+
+### ExtendedScript
+
+The top-level type (extends MulmoScript).
+
+```typescript
+{
+  // ... all MulmoScript fields preserved ...
+  beats: ExtendedBeat[];                              // Beats with meta
+  outputProfiles?: Record<string, OutputProfile>;      // Profile definitions
+  scriptMeta?: ScriptMeta;                             // Script-level metadata
+}
+```
+
+## Example
+
+```json
+{
+  "$mulmocast": { "version": "1.1" },
+  "title": "Example Presentation",
+  "lang": "en",
+  "speechParams": { "speakers": { "Presenter": { "voiceId": "alloy" } } },
+  "scriptMeta": {
+    "audience": "Software engineers interested in AI",
+    "goals": ["Understand LLM agent patterns", "Learn practical implementation approaches"],
+    "background": "LLM agents combine language models with tools and memory for autonomous task execution.",
+    "keywords": ["LLM", "agents", "AI"],
+    "references": [
+      {
+        "type": "document",
+        "url": "https://arxiv.org/abs/2210.03629",
+        "title": "ReAct Paper"
+      }
+    ],
+    "author": "AI Team"
+  },
+  "beats": [
+    {
+      "id": "intro-1",
+      "speaker": "Presenter",
+      "text": "Welcome to this presentation about LLM agents.",
+      "meta": {
+        "section": "opening",
+        "tags": ["intro"],
+        "keywords": ["LLM agents"],
+        "context": "This is the opening slide that sets the stage for the presentation.",
+        "expectedQuestions": ["What are LLM agents?"]
+      }
+    }
+  ],
+  "outputProfiles": {}
+}
+```