mulmocast 0.0.5 → 0.0.7

package/README.md

@@ -1,13 +1,70 @@
-# mulmocast
+# MulmoCast: A Multi-Modal Presentation Tool for the AI-Native Era
 
-A CLI tool for generating podcast and video content from script files (MulmoScript). Automates the process of creating audio, images, and video from structured MulmoScript files.
+## Quick Start Guide
 
-## What is MulmoScript?
+If you want to try our beta version, follow the instruction in the release note below.
+
+- [MulmoCast 0.1.x Beta Release Notes](./docs/beta1_en.md)
+- [MulmoCast 0.1.x Beta版 リリースノート](./docs/beta1_ja.md)
+
+## What is MulmoCast?
+
+Note: The video version of ["What is MulmoCast?"](https://x.com/mulmocast/status/1924682345065623719) is available on X, which is created by MulmoCast itself.
+
+**MulmoCast** is a next-generation presentation platform, purpose-built for a world where AI and humans collaborate to create and share ideas.
+
+Unlike traditional tools like PowerPoint or Keynote—both designed in the pre-AI era for human creators—MulmoCast is AI-native: reimagined from the ground up to work hand-in-hand with generative AI that understands and produces natural language, images, audio, and video.
+
+As AI takes on an increasingly central role in content creation, it needs more than raw capability—it needs an environment designed for its strengths. MulmoCast provides that environment, empowering AI to generate rich, coherent presentations across multiple formats, while keeping humans in the creative loop.
+
+At the core of MulmoCast is **MulmoScript**, a JSON-based intermediate language that functions like a screenplay or web markup. Generated by large language models, MulmoScript defines everything from narrative structure to visuals, making it easy to produce presentations that adapt seamlessly across formats.
+
+```mermaid
+flowchart TD
+    S_TextEditor["Creator"] <--> S_AIChat2("LLM")
+    S_AIChat2 --> MulmoScript["MulmoScript"]
+    MulmoScript --> MulmoCast["MulmoCast"]
+    MulmoCast --> O_Video("Video") & O_Podcast("Podcast") & O_Slideshow("Slideshow") & O_PDF("PDF") & O_Manga("Manga") & O_SwipeAnime("Swipe Anime")
+
+    MulmoScript@{ shape: div-proc}
+    MulmoCast@{ shape: trap-t}
+     S_TextEditor:::StoryInput
+     S_AIChat2:::ScriptInput
+     O_Video:::ScriptOutput
+     O_Podcast:::ScriptOutput
+     O_Slideshow:::ScriptOutput
+     O_PDF:::ScriptOutput
+     O_Manga:::ScriptOutput
+     O_SwipeAnime:::ScriptOutput
+    classDef StoryInput stroke-width:1px, stroke-dasharray:none, stroke:#46EDC8, fill:#DEFFF8, color:#378E7A
+    classDef ScriptInput stroke-width:1px, stroke-dasharray:none, stroke:#374D7C, fill:#E2EBFF, color:#374D7C
+    classDef ScriptOutput stroke-width:1px, stroke-dasharray:none, stroke:#FBB35A, fill:#FFEFDB, color:#8F632D
+```
+
+In today’s world, people consume information through many channels—on Zoom calls, in Slack threads, via videos on the go, or through podcasts during commutes. Presentations can no longer be confined to a single format or device. That’s why MulmoCast is built to be truly multi-modal, enabling output not only as slide decks but also as videos, podcasts, manga-style comics, and PDF documents.
+
+Whether you're building a podcast from a research report or turning a business pitch into a narrated video, MulmoCast transforms source material into dynamic, multi-format content with the help of state-of-the-art AI like ChatGPT and Claude.
+
+MulmoCast isn't just a tool. It’s a new foundation for communication—one where humans and AI co-create across every mode of expression.
+
+## MulmoScript
 
 **MulmoScript** is a simple JSON/YAML format for describing multi-modal content.  
 You can define speakers, text, images, and layout — all in one script.
 
-A Hello World script is available in [./docs/scripts](./docs/scripts).   
+Here is the "Hello World" in MulmoScript.
+
+```JSON
+{
+  "$mulmocast": {
+    "version": "1.0"
+  },
+  "beats": [
+    { "text": "Hello World" }
+  ]
+}
+```
+
 See [MulmoScript Format](#mulmoscript-format) for details on the structure.
 
 ## Installation
@@ -38,40 +95,85 @@ OPENAI_API_KEY=your_openai_api_key
 DEFAULT_OPENAI_IMAGE_MODEL=gpt-image-1 # for the advanced image generation model
 GOOGLE_PROJECT_ID=your_google_project_id # for Google's image generation model
 NIJIVOICE_API_KEY=your_nijivoice_api_key # for Nijivoice's TTS model
-BROWSERLESS_API_TOKEN=your_browserless_api_token # to access web in mulmo-tool
+BROWSERLESS_API_TOKEN=your_browserless_api_token # to access web in mulmo tool
 ```
 
 ## Workflow
 
-1. Create a MulmoScript JSON file with `mulmo-tool scripting`
+1. Create a MulmoScript JSON file with `mulmo tool scripting`
 2. Generate audio with `mulmo audio`
 3. Generate images with `mulmo images` 
 4. Create final video with `mulmo movie`
 
 ## Quick Start
+### Step 1: Generate a MulmoScript with interactive mode
+- Step 1-1: Run the script generation command
+  ```bash
+  mulmo tool scripting -i -t children_book -o ./ -s story
+  ```
+  This will initiate the script creation process.
+  
+- Step 1-2: Interactive conversation with AI
+
+  After running the command, you'll engage in an interactive conversation with the AI to create your story script. The AI will guide you through the process and help shape your content based on your inputs. Once completed, a JSON file (like ⁠`story-1746600802426.json`) will be generated.
+
+### Step 2: Generate both audio and images, then combine into video
+```bash
+mulmo movie {generated_script_file}
+```
+Replace `{generated_script_file}` with the output file from STEP 1, such as `story-1746600802426.json`.
 
+Optionally, you can specify *__clipboard* as the script file name to paste the script from the clipboard, which is generated by other interactive environment such as ChatGPT or Claude.
+
+## Quick Start for Ghibli-Style Animation
+
+[![Watch Ghibli-Style Animation Example](https://github.com/receptron/mulmocast-cli/blob/main/docs/images/introducing-mulmocast-new-era-presentations-thumbnail.jpg?raw=true)](https://www.youtube.com/watch?v=gz_Oe5p2n3s)  
+*Click the image above to watch an example of what you can create*
+
+### Step 1: Check/Update Environment Setup
+Verify your `.env` file contains:
 ```bash
-# Generate script with interactive mode
-mulmo-tool scripting -i -t children_book -o ./ -s story
+OPENAI_API_KEY=your_openai_api_key
+DEFAULT_OPENAI_IMAGE_MODEL=gpt-image-1 # required for high-quality Ghibli-style images
 ```
-After running this command, you'll create a story script through an interactive conversation with the AI.
 
+> **Note:** Ensure your OpenAI organization is verified to access the gpt-image-1 model. Visit https://platform.openai.com/settings/organization/general and complete the "Verifications" section.
+
+### Step 2: Generate a Ghibli-style MulmoScript
 ```bash
-# Generate both audio and images, then combine into video
-mulmo movie {generated_script_file}
+mulmo tool scripting -i -t ghibli_strips -o ./ -s story
+```
+This will initiate an interactive conversation with the AI to create your Ghibli-inspired story. Once completed, a JSON file (e.g., `story-1747834931950.json`) will be generated.
+
+### Step 3: Translate the MulmoScript
+```bash
+mulmo translate {generated_script_file_from_step_2}
+```
+
+### Step 4: Generate video with Japanese subtitles
+```bash
+mulmo movie {generated_script_file_from_step_2} -c ja
+```
+The `-c ja` flag adds Japanese subtitles to your video.
+
+When the process completes, the CLI will display the path to your generated video file:
+```bash
+Video created successfully! 14.939 sec
+writing: /Users/username/path/to/output/story-1747834931950__ja.mp4
 ```
-Replace `{generated_script_file}` with the output file from the previous command, such as `story-1746600802426.json`.
 
 ## Generate MulmoScript
 
 ```bash
 # Generate script from web content (requires Browserless API KEY)
-mulmo-tool scripting -u https://example.com
+mulmo tool scripting -u https://example.com
 
 # Generate script with interactive mode
-mulmo-tool scripting -i
+mulmo tool scripting -i
 ```
 
+When using the `⁠sensei_and_taro` template, a Nijivoice API key is required.
+
 ## Generate content from MulmoScript
 
 Mulmo provides several commands to handle different aspects of content creation:
@@ -121,52 +223,150 @@ https://github.com/receptron/mulmocast-cli/tree/main/scripts
 
 CLI Usage
 
-### `mulmo -h` Output
-Use this command to generate content (audio, images, movie) from an existing MulmoScript.
 
-```bash
-mulmo -h
+
 ```
-```bash
-mulmo <action> <file>
+mulmo <command> [options]
 
-Run mulmocast
+Commands:
+  mulmo translate <file>  Translate Mulmo script
+  mulmo audio <file>      Generate audio files
+  mulmo images <file>     Generate image files
+  mulmo movie <file>      Generate movie file
+  mulmo pdf <file>        Generate PDF files
+  mulmo tool <command>    Generate Mulmo script and other tools
+
+Options:
+      --version  Show version number                                   [boolean]
+  -v, --verbose  verbose log               [boolean] [required] [default: false]
+  -h, --help     Show help                                             [boolean]
+```
+
+```
+mulmo translate <file>
+
+Translate Mulmo script
+
+Positionals:
+  file  Mulmo Script File                                    [string] [required]
+
+Options:
+      --version  Show version number                                   [boolean]
+  -v, --verbose  verbose log               [boolean] [required] [default: false]
+  -h, --help     Show help                                             [boolean]
+  -o, --outdir   output dir                                             [string]
+  -b, --basedir  base dir                                               [string]
+  -l, --lang     target language                  [string] [choices: "en", "ja"]
+  -f, --force    Force regenerate                     [boolean] [default: false]
+```
+
+```
+mulmo audio <file>
+
+Generate audio files
 
 Positionals:
-  action  action to perform
-       [string] [choices: "translate", "audio", "images", "movie", "preprocess"]
-  file    Mulmo Script File                                             [string]
+  file  Mulmo Script File                                    [string] [required]
 
 Options:
       --version   Show version number                                  [boolean]
   -v, --verbose   verbose log              [boolean] [required] [default: false]
+  -h, --help      Show help                                            [boolean]
   -o, --outdir    output dir                                            [string]
   -b, --basedir   base dir                                              [string]
-  -a, --audiodir  audio dir                                             [string]
-  -i, --imagedir  image dir                                             [string]
-  -f, --force     force generate                      [boolean] [default: false]
-      --help      Show help                                            [boolean]
+  -l, --lang      target language                 [string] [choices: "en", "ja"]
+  -f, --force     Force regenerate                    [boolean] [default: false]
+  -a, --audiodir  Audio output directory                                [string]
+```
 
 ```
+mulmo images <file>
 
-### `mulmo-tool -h` Output
-Use this command to generate a new MulmoScript from a URL or interactively.
+Generate image files
 
-```bash
-mulmo-tool -h
+Positionals:
+  file  Mulmo Script File                                    [string] [required]
+
+Options:
+      --version   Show version number                                  [boolean]
+  -v, --verbose   verbose log              [boolean] [required] [default: false]
+  -h, --help      Show help                                            [boolean]
+  -o, --outdir    output dir                                            [string]
+  -b, --basedir   base dir                                              [string]
+  -l, --lang      target language                 [string] [choices: "en", "ja"]
+  -f, --force     Force regenerate                    [boolean] [default: false]
+  -i, --imagedir  Image output directory                                [string]
 ```
 
-```bash
-mulmo-tool <action>
+```
+mulmo movie <file>
 
-Run mulmocast tool
+Generate movie file
 
 Positionals:
-  action  action to perform            [string] [choices: "scripting", "prompt"]
+  file  Mulmo Script File                                    [string] [required]
+
+Options:
+      --version   Show version number                                  [boolean]
+  -v, --verbose   verbose log              [boolean] [required] [default: false]
+  -h, --help      Show help                                            [boolean]
+  -o, --outdir    output dir                                            [string]
+  -b, --basedir   base dir                                              [string]
+  -l, --lang      target language                 [string] [choices: "en", "ja"]
+  -f, --force     Force regenerate                    [boolean] [default: false]
+  -a, --audiodir  Audio output directory                                [string]
+  -i, --imagedir  Image output directory                                [string]
+  -c, --caption   Video captions                  [string] [choices: "en", "ja"]
+```
+
+```
+mulmo pdf <file>
+
+Generate PDF files
+
+Positionals:
+  file  Mulmo Script File                                    [string] [required]
+
+Options:
+      --version   Show version number                                  [boolean]
+  -v, --verbose   verbose log              [boolean] [required] [default: false]
+  -h, --help      Show help                                            [boolean]
+  -o, --outdir    output dir                                            [string]
+  -b, --basedir   base dir                                              [string]
+  -l, --lang      target language                 [string] [choices: "en", "ja"]
+  -f, --force     Force regenerate                    [boolean] [default: false]
+  -i, --imagedir  Image output directory                                [string]
+      --pdf_mode  PDF mode
+               [string] [choices: "slide", "talk", "handout"] [default: "slide"]
+      --pdf_size  PDF paper size (default: letter)
+                                   [choices: "letter", "a4"] [default: "letter"]
+```
+
+```
+mulmo tool <command>
+
+Generate Mulmo script and other tools
+
+Commands:
+  mulmo tool scripting  Generate mulmocast script
+  mulmo tool prompt     Dump prompt from template
+  mulmo tool schema     Dump mulmocast schema
+
+Options:
+      --version  Show version number                                   [boolean]
+  -v, --verbose  verbose log               [boolean] [required] [default: false]
+  -h, --help     Show help                                             [boolean]
+```
+
+```
+mulmo tool scripting
+
+Generate mulmocast script
 
 Options:
       --version      Show version number                               [boolean]
   -v, --verbose      verbose log           [boolean] [required] [default: false]
+  -h, --help         Show help                                         [boolean]
   -o, --outdir       output dir                                         [string]
   -b, --basedir      base dir                                           [string]
   -u, --url          URLs to reference (required when not in interactive mode)
@@ -174,12 +374,67 @@ Options:
   -i, --interactive  Generate script in interactive mode with user prompts
                                                                        [boolean]
   -t, --template     Template name to use
-   [string] [choices: "business", "children_book", "coding", "podcast_standard",
-                                                              "sensei_and_taro"]
+       [string] [choices: "business", "children_book", "coding", "comic_strips",
+                         "ghibli_strips", "podcast_standard", "sensei_and_taro"]
   -c, --cache        cache dir                                          [string]
   -s, --script       script filename                [string] [default: "script"]
-      --help         Show help                                         [boolean]
+      --llm          llm
+              [string] [choices: "openAIAgent", "anthropicAgent", "geminiAgent",
+                                                                    "groqAgent"]
+      --llm_model    llm model                                          [string]
+```
+
 ```
+mulmo tool story_to_script <file>
+
+Generate Mulmo script from story
+
+Positionals:
+  file  story file path                                      [string] [required]
+
+Options:
+      --version          Show version number                           [boolean]
+  -v, --verbose          verbose log       [boolean] [required] [default: false]
+  -h, --help             Show help                                     [boolean]
+  -o, --outdir           output dir                                     [string]
+  -b, --basedir          base dir                                       [string]
+  -t, --template         Template name to use
+       [string] [choices: "business", "children_book", "coding", "comic_strips",
+                         "ghibli_strips", "podcast_standard", "sensei_and_taro"]
+  -s, --script           script filename            [string] [default: "script"]
+      --beats_per_scene  beats per scene                   [number] [default: 3]
+      --llm              llm
+                     [string] [choices: "openAI", "anthropic", "gemini", "groq"]
+      --llm_model        llm model                                      [string]
+      --mode             story to script generation mode
+              [string] [choices: "step_wise", "one_step"] [default: "step_wise"]
+```
+
+```
+mulmo tool prompt
+
+Dump prompt from template
+
+Options:
+      --version   Show version number                                  [boolean]
+  -v, --verbose   verbose log              [boolean] [required] [default: false]
+  -h, --help      Show help                                            [boolean]
+  -t, --template  Template name to use
+       [string] [choices: "business", "children_book", "coding", "comic_strips",
+                         "ghibli_strips", "podcast_standard", "sensei_and_taro"]
+```
+
+```
+mulmo tool schema
+
+Dump mulmocast schema
+
+Options:
+      --version  Show version number                                   [boolean]
+  -v, --verbose  verbose log               [boolean] [required] [default: false]
+  -h, --help     Show help                                             [boolean]
+```
+
 
 
 ## Contributing

package/assets/html/caption.html

@@ -0,0 +1,45 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <style>
+    body{
+      margin: 0;
+      background: transparent;
+    }
+    .container {
+      /* Box styling */
+      width: ${width}px;
+      height: ${height}px;
+      position: relative;
+      margin: 0;
+      box-sizing: border-box;
+      background: transparent;
+    }
+    .caption {
+      /* Text positioned at the bottom */
+      position: absolute;
+      bottom: 0px;
+      /* Enable text wrapping */
+      word-wrap: break-word;
+      overflow-wrap: break-word;
+      /* Optional styling */
+      font-family: Arial, sans-serif;
+      font-size: 32px;
+      text-align: center;
+      color: white;
+      text-shadow: 0px 0px 20px rgba(0, 0, 0, 1.0);
+      padding-left: 10%;
+      padding-right: 10%;
+      padding-top: 4px;
+      background: rgba(0, 0, 0, 0.4);
+    }
+  </style>
+</head>
+<body>
+  <div class="container">
+    <div class="caption">${caption}</div>
+  </div>
+</body>
+</html>

package/assets/html/chart.html

@@ -7,7 +7,7 @@
     <style>
         ${style}
         .chart-container {
-            width: ${width}px;
+            width: ${chart_width}px;
             margin: 0 auto;
         }
     </style>

package/assets/html/mermaid.html

@@ -23,7 +23,7 @@
         }
         .mermaid svg {
             max-width: 100%;
-            max-height: 100%;
+            max-height: 80%;
             height: auto;
             width: auto;
         }
@@ -38,6 +38,10 @@
                 useMaxWidth: true
             }
         });
+        document.addEventListener("DOMContentLoaded", () => {
+            mermaid.init(undefined, ".mermaid");
+            document.querySelector(".mermaid").dataset.ready = "true";
+        });
     </script>
 </head>
 <body>
@@ -48,4 +52,4 @@
         </div>
     </div>
 </body>
-</html> 
+</html> 

package/assets/html/tailwind.html

@@ -0,0 +1,13 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+  <title>${title}</title>
+  <!-- Tailwind CSS CDN -->
+  <script src="https://cdn.tailwindcss.com"></script>
+</head>
+<body class="bg-white text-gray-800 min-h-screen flex flex-col">
+  ${html_body}
+</body>
+</html>

package/assets/templates/business.json

@@ -1,132 +1,6 @@
 {
   "title": "Business presentation",
   "description": "Template for business presentation.",
-  "systemPrompt": "Generate a script for a business presentation of the given topic. Use textSlides, markdown, mermaid, or chart to show slides. Extract image links in the article to reuse them in the presentation. Use the JSON below as a template. chartData is the data for Chart.js",
-  "script": {
-    "$mulmocast": {
-      "version": "1.0",
-      "credit": "closing"
-    },
-    "title": "Sample Title",
-    "speechParams": {
-      "speakers": {
-        "Presenter": {
-          "voiceId": "shimmer",
-          "displayName": {
-            "en": "Presenter",
-            "ja": "語り手"
-          }
-        }
-      }
-    },
-    "lang": "en",
-    "beats": [
-      {
-        "speaker": "Presenter",
-        "text": "Today we're exploring a fascinating concept that has shaped some of the most innovative companies and leaders of our time: the Reality Distortion Field. This term has become synonymous with visionary leadership and the ability to accomplish seemingly impossible goals.",
-        "image": {
-          "type": "textSlide",
-          "slide": {
-            "title": "This is the title of this slide",
-            "bullets": []
-          }
-        },
-        "textSlideParams": {
-          "cssStyles": ["h1 { margin-top: 300px }"]
-        }
-      },
-      {
-        "speaker": "Presenter",
-        "text": "The evolution of humans is a complex journey that spans millions of years, shaped by biology, environment, and culture. Here's a high-level summary of the key stages in human evolution",
-        "image": {
-          "type": "textSlide",
-          "slide": {
-            "title": "Human Evolution",
-            "bullets": [
-              "Early Primates",
-              "Hominids and Hominins",
-              "Australopithecus",
-              "Genus Homo Emerges",
-              "Homo erectus and Migration",
-              "Neanderthals and Other Archaic Humans",
-              "Homo sapiens"
-            ]
-          }
-        }
-      },
-      {
-        "speaker": "Presenter",
-        "text": "Our story begins with tree-dwelling primates—small mammals with forward-facing eyes and grasping hands. These ancestors lived in forests and evolved traits useful for climbing and visual depth perception.",
-        "image": {
-          "type": "markdown",
-          "markdown": [
-            "# Markdown Table Example",
-            "| Item              | In Stock | Price |",
-            "| :---------------- | :------: | ----: |",
-            "| Python Hat        |   True   | 23.99 |",
-            "| SQL Hat           |   True   | 23.99 |",
-            "| Codecademy Tee    |  False   | 19.99 |",
-            "| Codecademy Hoodie |  False   | 42.99 |"
-          ]
-        }
-      },
-      {
-        "speaker": "Presenter",
-        "text": "This page shows the sales and profits of this company from January 2024 to June 2024.",
-        "image": {
-          "type": "chart",
-          "title": "Sales and Profits (from Jan to June)",
-          "chartData": {
-            "type": "bar",
-            "data": {
-              "labels": ["January", "February", "March", "April", "May", "June"],
-              "datasets": [
-                {
-                  "label": "Revenue ($1000s)",
-                  "data": [120, 135, 180, 155, 170, 190],
-                  "backgroundColor": "rgba(54, 162, 235, 0.5)",
-                  "borderColor": "rgba(54, 162, 235, 1)",
-                  "borderWidth": 1
-                },
-                {
-                  "label": "Profit ($1000s)",
-                  "data": [45, 52, 68, 53, 61, 73],
-                  "backgroundColor": "rgba(75, 192, 192, 0.5)",
-                  "borderColor": "rgba(75, 192, 192, 1)",
-                  "borderWidth": 1
-                }
-              ]
-            },
-            "options": {
-              "responsive": true,
-              "animation": false
-            }
-          }
-        }
-      },
-      {
-        "speaker": "Presenter",
-        "text": "Next, let's look at a diagram of our business process flow. This illustrates the key steps from product development to sales.",
-        "image": {
-          "type": "mermaid",
-          "title": "Business Process Flow",
-          "code": {
-            "kind": "text",
-            "text": "graph LR\n    A[Market Research] --> B[Product Planning]\n    B --> C[Development]\n    C --> D[Testing]\n    D --> E[Manufacturing]\n    E --> F[Marketing]\n    F --> G[Sales]\n    G --> H[Customer Support]\n    H --> A"
-          }
-        }
-      },
-      {
-        "speaker": "Presenter",
-        "text": "This is the image of a pingpong ball.",
-        "image": {
-          "type": "image",
-          "source": {
-            "kind": "url",
-            "url": "https://github.com/receptron/mulmocast-media/raw/refs/heads/main/test/pingpong.png"
-          }
-        }
-      }
-    ]
-  }
+  "systemPrompt": "Generate a script for a business presentation of the given topic. Use textSlides, markdown, mermaid, or chart to show slides. Extract image links in the article (from <img> tag) to reuse them in the presentation. Mention the reference in one of beats, if it exists. Use the JSON below as a template. chartData is the data for Chart.js",
+  "scriptName": "business.json"
 }