mulmocast 0.0.5 → 0.0.6

package/README.md

@@ -1,13 +1,59 @@
-# mulmocast
+# MulmoCast: A Multi-Modal Presentation Tool for the AI-Native Era
+
+**MulmoCast** is a next-generation presentation platform, purpose-built for a world where AI and humans collaborate to create and share ideas.
+
+Unlike traditional tools like PowerPoint or Keynote—both designed in the pre-AI era for human creators—MulmoCast is AI-native: reimagined from the ground up to work hand-in-hand with generative AI that understands and produces natural language, images, audio, and video.
+
+As AI takes on an increasingly central role in content creation, it needs more than raw capability—it needs an environment designed for its strengths. MulmoCast provides that environment, empowering AI to generate rich, coherent presentations across multiple formats, while keeping humans in the creative loop.
+
+At the core of MulmoCast is **MulmoScript**, a JSON-based intermediate language that functions like a screenplay or web markup. Generated by large language models, MulmoScript defines everything from narrative structure to visuals, making it easy to produce presentations that adapt seamlessly across formats.
+
+```mermaid
+flowchart TD
+    S_TextEditor["Creator"] <--> S_AIChat2("LLM")
+    S_AIChat2 --> MulmoScript["MulmoScript"]
+    MulmoScript --> MulmoCast["MulmoCast"]
+    MulmoCast --> O_Video("Video") & O_Podcast("Podcast") & O_Slideshow("Slideshow") & O_PDF("PDF") & O_Manga("Manga") & O_SwipeAnime("Swipe Anime")
+
+    MulmoScript@{ shape: div-proc}
+    MulmoCast@{ shape: trap-t}
+     S_TextEditor:::StoryInput
+     S_AIChat2:::ScriptInput
+     O_Video:::ScriptOutput
+     O_Podcast:::ScriptOutput
+     O_Slideshow:::ScriptOutput
+     O_PDF:::ScriptOutput
+     O_Manga:::ScriptOutput
+     O_SwipeAnime:::ScriptOutput
+    classDef StoryInput stroke-width:1px, stroke-dasharray:none, stroke:#46EDC8, fill:#DEFFF8, color:#378E7A
+    classDef ScriptInput stroke-width:1px, stroke-dasharray:none, stroke:#374D7C, fill:#E2EBFF, color:#374D7C
+    classDef ScriptOutput stroke-width:1px, stroke-dasharray:none, stroke:#FBB35A, fill:#FFEFDB, color:#8F632D
+```
+
+In today’s world, people consume information through many channels—on Zoom calls, in Slack threads, via videos on the go, or through podcasts during commutes. Presentations can no longer be confined to a single format or device. That’s why MulmoCast is built to be truly multi-modal, enabling output not only as slide decks but also as videos, podcasts, manga-style comics, and PDF documents.
 
-A CLI tool for generating podcast and video content from script files (MulmoScript). Automates the process of creating audio, images, and video from structured MulmoScript files.
+Whether you're building a podcast from a research report or turning a business pitch into a narrated video, MulmoCast transforms source material into dynamic, multi-format content with the help of state-of-the-art AI like ChatGPT and Claude.
 
-## What is MulmoScript?
+MulmoCast isn't just a tool. It’s a new foundation for communication—one where humans and AI co-create across every mode of expression.
+
+## MulmoScript
 
 **MulmoScript** is a simple JSON/YAML format for describing multi-modal content.  
 You can define speakers, text, images, and layout — all in one script.
 
-A Hello World script is available in [./docs/scripts](./docs/scripts).   
+Here is the "Hello World" in MulmoScript.
+
+```JSON
+{
+  "$mulmocast": {
+    "version": "1.0"
+  },
+  "beats": [
+    { "text": "Hello World" }
+  ]
+}
+```
+
 See [MulmoScript Format](#mulmoscript-format) for details on the structure.
 
 ## Installation
@@ -38,40 +84,85 @@ OPENAI_API_KEY=your_openai_api_key
 DEFAULT_OPENAI_IMAGE_MODEL=gpt-image-1 # for the advanced image generation model
 GOOGLE_PROJECT_ID=your_google_project_id # for Google's image generation model
 NIJIVOICE_API_KEY=your_nijivoice_api_key # for Nijivoice's TTS model
-BROWSERLESS_API_TOKEN=your_browserless_api_token # to access web in mulmo-tool
+BROWSERLESS_API_TOKEN=your_browserless_api_token # to access web in mulmo tool
 ```
 
 ## Workflow
 
-1. Create a MulmoScript JSON file with `mulmo-tool scripting`
+1. Create a MulmoScript JSON file with `mulmo tool scripting`
 2. Generate audio with `mulmo audio`
 3. Generate images with `mulmo images` 
 4. Create final video with `mulmo movie`
 
 ## Quick Start
+### Step 1: Generate a MulmoScript with interactive mode
+- Step 1-1: Run the script generation command
+  ```bash
+  mulmo tool scripting -i -t children_book -o ./ -s story
+  ```
+  This will initiate the script creation process.
+  
+- Step 1-2: Interactive conversation with AI
+
+  After running the command, you'll engage in an interactive conversation with the AI to create your story script. The AI will guide you through the process and help shape your content based on your inputs. Once completed, a JSON file (like ⁠`story-1746600802426.json`) will be generated.
+
+### Step 2: Generate both audio and images, then combine into video
+```bash
+mulmo movie {generated_script_file}
+```
+Replace `{generated_script_file}` with the output file from STEP 1, such as `story-1746600802426.json`.
+
+Optionally, you can specify *__clipboard* as the script file name to paste the script from the clipboard, which is generated by other interactive environment such as ChatGPT or Claude.
+
+## Quick Start for Ghibli-Style Animation
 
+[![Watch Ghibli-Style Animation Example](https://img.youtube.com/vi/gz_Oe5p2n3s/0.jpg)](https://www.youtube.com/watch?v=gz_Oe5p2n3s)  
+*Click the image above to watch an example of what you can create*
+
+### Step 1: Check/Update Environment Setup
+Verify your `.env` file contains:
 ```bash
-# Generate script with interactive mode
-mulmo-tool scripting -i -t children_book -o ./ -s story
+OPENAI_API_KEY=your_openai_api_key
+DEFAULT_OPENAI_IMAGE_MODEL=gpt-image-1 # required for high-quality Ghibli-style images
 ```
-After running this command, you'll create a story script through an interactive conversation with the AI.
 
+> **Note:** Ensure your OpenAI organization is verified to access the gpt-image-1 model. Visit https://platform.openai.com/settings/organization/general and complete the "Verifications" section.
+
+### Step 2: Generate a Ghibli-style MulmoScript
 ```bash
-# Generate both audio and images, then combine into video
-mulmo movie {generated_script_file}
+mulmo tool scripting -i -t ghibli_strips -o ./ -s story
+```
+This will initiate an interactive conversation with the AI to create your Ghibli-inspired story. Once completed, a JSON file (e.g., `story-1747834931950.json`) will be generated.
+
+### Step 3: Translate the MulmoScript
+```bash
+mulmo translate {generated_script_file_from_step_2}
+```
+
+### Step 4: Generate video with Japanese subtitles
+```bash
+mulmo movie {generated_script_file_from_step_2} -c ja
+```
+The `-c ja` flag adds Japanese subtitles to your video.
+
+When the process completes, the CLI will display the path to your generated video file:
+```bash
+Video created successfully! 14.939 sec
+writing: /Users/username/path/to/output/story-1747834931950__ja.mp4
 ```
-Replace `{generated_script_file}` with the output file from the previous command, such as `story-1746600802426.json`.
 
 ## Generate MulmoScript
 
 ```bash
 # Generate script from web content (requires Browserless API KEY)
-mulmo-tool scripting -u https://example.com
+mulmo tool scripting -u https://example.com
 
 # Generate script with interactive mode
-mulmo-tool scripting -i
+mulmo tool scripting -i
 ```
 
+When using the `⁠sensei_and_taro` template, a Nijivoice API key is required.
+
 ## Generate content from MulmoScript
 
 Mulmo provides several commands to handle different aspects of content creation:
@@ -121,52 +212,150 @@ https://github.com/receptron/mulmocast-cli/tree/main/scripts
 
 CLI Usage
 
-### `mulmo -h` Output
-Use this command to generate content (audio, images, movie) from an existing MulmoScript.
 
-```bash
-mulmo -h
+
 ```
-```bash
-mulmo <action> <file>
+mulmo <command> [options]
+
+Commands:
+  mulmo translate <file>  Translate Mulmo script
+  mulmo audio <file>      Generate audio files
+  mulmo images <file>     Generate image files
+  mulmo movie <file>      Generate movie file
+  mulmo pdf <file>        Generate PDF files
+  mulmo tool <command>    Generate Mulmo script and other tools
+
+Options:
+      --version  Show version number                                   [boolean]
+  -v, --verbose  verbose log               [boolean] [required] [default: false]
+  -h, --help     Show help                                             [boolean]
+```
+
+```
+mulmo translate <file>
+
+Translate Mulmo script
+
+Positionals:
+  file  Mulmo Script File                                    [string] [required]
+
+Options:
+      --version  Show version number                                   [boolean]
+  -v, --verbose  verbose log               [boolean] [required] [default: false]
+  -h, --help     Show help                                             [boolean]
+  -o, --outdir   output dir                                             [string]
+  -b, --basedir  base dir                                               [string]
+  -l, --lang     target language                  [string] [choices: "en", "ja"]
+  -f, --force    Force regenerate                     [boolean] [default: false]
+```
+
+```
+mulmo audio <file>
 
-Run mulmocast
+Generate audio files
 
 Positionals:
-  action  action to perform
-       [string] [choices: "translate", "audio", "images", "movie", "preprocess"]
-  file    Mulmo Script File                                             [string]
+  file  Mulmo Script File                                    [string] [required]
 
 Options:
       --version   Show version number                                  [boolean]
   -v, --verbose   verbose log              [boolean] [required] [default: false]
+  -h, --help      Show help                                            [boolean]
   -o, --outdir    output dir                                            [string]
   -b, --basedir   base dir                                              [string]
-  -a, --audiodir  audio dir                                             [string]
-  -i, --imagedir  image dir                                             [string]
-  -f, --force     force generate                      [boolean] [default: false]
-      --help      Show help                                            [boolean]
+  -l, --lang      target language                 [string] [choices: "en", "ja"]
+  -f, --force     Force regenerate                    [boolean] [default: false]
+  -a, --audiodir  Audio output directory                                [string]
+```
 
 ```
+mulmo images <file>
 
-### `mulmo-tool -h` Output
-Use this command to generate a new MulmoScript from a URL or interactively.
+Generate image files
 
-```bash
-mulmo-tool -h
+Positionals:
+  file  Mulmo Script File                                    [string] [required]
+
+Options:
+      --version   Show version number                                  [boolean]
+  -v, --verbose   verbose log              [boolean] [required] [default: false]
+  -h, --help      Show help                                            [boolean]
+  -o, --outdir    output dir                                            [string]
+  -b, --basedir   base dir                                              [string]
+  -l, --lang      target language                 [string] [choices: "en", "ja"]
+  -f, --force     Force regenerate                    [boolean] [default: false]
+  -i, --imagedir  Image output directory                                [string]
 ```
 
-```bash
-mulmo-tool <action>
+```
+mulmo movie <file>
 
-Run mulmocast tool
+Generate movie file
 
 Positionals:
-  action  action to perform            [string] [choices: "scripting", "prompt"]
+  file  Mulmo Script File                                    [string] [required]
+
+Options:
+      --version   Show version number                                  [boolean]
+  -v, --verbose   verbose log              [boolean] [required] [default: false]
+  -h, --help      Show help                                            [boolean]
+  -o, --outdir    output dir                                            [string]
+  -b, --basedir   base dir                                              [string]
+  -l, --lang      target language                 [string] [choices: "en", "ja"]
+  -f, --force     Force regenerate                    [boolean] [default: false]
+  -a, --audiodir  Audio output directory                                [string]
+  -i, --imagedir  Image output directory                                [string]
+  -c, --caption   Video captions                  [string] [choices: "en", "ja"]
+```
+
+```
+mulmo pdf <file>
+
+Generate PDF files
+
+Positionals:
+  file  Mulmo Script File                                    [string] [required]
+
+Options:
+      --version   Show version number                                  [boolean]
+  -v, --verbose   verbose log              [boolean] [required] [default: false]
+  -h, --help      Show help                                            [boolean]
+  -o, --outdir    output dir                                            [string]
+  -b, --basedir   base dir                                              [string]
+  -l, --lang      target language                 [string] [choices: "en", "ja"]
+  -f, --force     Force regenerate                    [boolean] [default: false]
+  -i, --imagedir  Image output directory                                [string]
+      --pdf_mode  PDF mode
+               [string] [choices: "slide", "talk", "handout"] [default: "slide"]
+      --pdf_size  PDF paper size (default: letter)
+                                   [choices: "letter", "a4"] [default: "letter"]
+```
+
+```
+mulmo tool <command>
+
+Generate Mulmo script and other tools
+
+Commands:
+  mulmo tool scripting  Generate mulmocast script
+  mulmo tool prompt     Dump prompt from template
+  mulmo tool schema     Dump mulmocast schema
+
+Options:
+      --version  Show version number                                   [boolean]
+  -v, --verbose  verbose log               [boolean] [required] [default: false]
+  -h, --help     Show help                                             [boolean]
+```
+
+```
+mulmo tool scripting
+
+Generate mulmocast script
 
 Options:
       --version      Show version number                               [boolean]
   -v, --verbose      verbose log           [boolean] [required] [default: false]
+  -h, --help         Show help                                         [boolean]
   -o, --outdir       output dir                                         [string]
   -b, --basedir      base dir                                           [string]
   -u, --url          URLs to reference (required when not in interactive mode)
@@ -174,14 +363,43 @@ Options:
   -i, --interactive  Generate script in interactive mode with user prompts
                                                                        [boolean]
   -t, --template     Template name to use
-   [string] [choices: "business", "children_book", "coding", "podcast_standard",
-                                                              "sensei_and_taro"]
+       [string] [choices: "business", "children_book", "coding", "comic_strips",
+                         "ghibli_strips", "podcast_standard", "sensei_and_taro"]
   -c, --cache        cache dir                                          [string]
   -s, --script       script filename                [string] [default: "script"]
-      --help         Show help                                         [boolean]
+      --llm_agent    llm agent
+              [string] [choices: "openAIAgent", "anthropicAgent", "geminiAgent",
+                                                                    "groqAgent"]
+      --llm_model    llm model                                          [string]
+```
+
+```
+mulmo tool prompt
+
+Dump prompt from template
+
+Options:
+      --version   Show version number                                  [boolean]
+  -v, --verbose   verbose log              [boolean] [required] [default: false]
+  -h, --help      Show help                                            [boolean]
+  -t, --template  Template name to use
+       [string] [choices: "business", "children_book", "coding", "comic_strips",
+                         "ghibli_strips", "podcast_standard", "sensei_and_taro"]
+```
+
+```
+mulmo tool schema
+
+Dump mulmocast schema
+
+Options:
+      --version  Show version number                                   [boolean]
+  -v, --verbose  verbose log               [boolean] [required] [default: false]
+  -h, --help     Show help                                             [boolean]
 ```
 
 
+
 ## Contributing
 
 For developers interested in contributing to this project, please see [CONTRIBUTING.md](./CONTRIBUTING.md).

package/assets/html/caption.html

@@ -0,0 +1,45 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <style>
+    body{
+      margin: 0;
+      background: transparent;
+    }
+    .container {
+      /* Box styling */
+      width: ${width}px;
+      height: ${height}px;
+      position: relative;
+      margin: 0;
+      box-sizing: border-box;
+      background: transparent;
+    }
+    .caption {
+      /* Text positioned at the bottom */
+      position: absolute;
+      bottom: 0px;
+      /* Enable text wrapping */
+      word-wrap: break-word;
+      overflow-wrap: break-word;
+      /* Optional styling */
+      font-family: Arial, sans-serif;
+      font-size: 32px;
+      text-align: center;
+      color: white;
+      text-shadow: 0px 0px 20px rgba(0, 0, 0, 1.0);
+      padding-left: 10%;
+      padding-right: 10%;
+      padding-top: 4px;
+      background: rgba(0, 0, 0, 0.4);
+    }
+  </style>
+</head>
+<body>
+  <div class="container">
+    <div class="caption">${caption}</div>
+  </div>
+</body>
+</html>

package/assets/html/chart.html

@@ -7,7 +7,7 @@
     <style>
         ${style}
         .chart-container {
-            width: ${width}px;
+            width: ${chart_width}px;
             margin: 0 auto;
         }
     </style>

package/assets/html/mermaid.html

@@ -23,7 +23,7 @@
         }
         .mermaid svg {
             max-width: 100%;
-            max-height: 100%;
+            max-height: 80%;
             height: auto;
             width: auto;
         }
@@ -38,6 +38,10 @@
                 useMaxWidth: true
             }
         });
+        document.addEventListener("DOMContentLoaded", () => {
+            mermaid.init(undefined, ".mermaid");
+            document.querySelector(".mermaid").dataset.ready = "true";
+        });
     </script>
 </head>
 <body>
@@ -48,4 +52,4 @@
         </div>
     </div>
 </body>
-</html> 
+</html> 

package/assets/html/tailwind.html

@@ -0,0 +1,13 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+  <title>${title}</title>
+  <!-- Tailwind CSS CDN -->
+  <script src="https://cdn.tailwindcss.com"></script>
+</head>
+<body class="bg-white text-gray-800 min-h-screen flex flex-col">
+  ${html_body}
+</body>
+</html>

package/assets/templates/business.json

@@ -1,7 +1,7 @@
 {
   "title": "Business presentation",
   "description": "Template for business presentation.",
-  "systemPrompt": "Generate a script for a business presentation of the given topic. Use textSlides, markdown, mermaid, or chart to show slides. Extract image links in the article to reuse them in the presentation. Use the JSON below as a template. chartData is the data for Chart.js",
+  "systemPrompt": "Generate a script for a business presentation of the given topic. Use textSlides, markdown, mermaid, or chart to show slides. Extract image links in the article to reuse them in the presentation. Mention the reference in one of beats, if it exists. Use the JSON below as a template. chartData is the data for Chart.js",
   "script": {
     "$mulmocast": {
       "version": "1.0",
@@ -19,6 +19,13 @@
         }
       }
     },
+    "references": [
+      {
+        "url": "https://www.somegreatwebsite.com/article/123",
+        "title": "Title of the article we are referencing",
+        "type": "article"
+      }
+    ],
     "lang": "en",
     "beats": [
       {
@@ -30,9 +37,6 @@
             "title": "This is the title of this slide",
             "bullets": []
           }
-        },
-        "textSlideParams": {
-          "cssStyles": ["h1 { margin-top: 300px }"]
         }
       },
       {
@@ -116,6 +120,55 @@
           }
         }
       },
+      {
+        "speaker": "Presenter",
+        "text": "This is a tailwind html format.",
+        "image": {
+          "type": "html_tailwind",
+          "html": [
+            "<main class=\"flex-grow\">",
+            "  <!-- Hero Section -->",
+            "  <section class=\"bg-blue-600 text-white py-20\">",
+            "    <div class=\"container mx-auto px-6 text-center\">",
+            "      <h1 class=\"text-4xl md:text-5xl font-bold mb-4\">Welcome to Mulmocast</h1>",
+            "      <p class=\"text-lg md:text-xl mb-8\">A modern web experience powered by Tailwind CSS</p>",
+            "      <a href=\"#features\" class=\"bg-white text-blue-600 px-6 py-3 rounded-lg font-semibold shadow hover:bg-gray-100 transition\">",
+            "        Learn More",
+            "      </a>",
+            "    </div>",
+            "  </section>",
+            "",
+            "  <!-- Features Section -->",
+            "  <section id=\"features\" class=\"py-16 bg-gray-100\">",
+            "    <div class=\"container mx-auto px-6\">",
+            "      <div class=\"grid grid-cols-1 md:grid-cols-3 gap-8 text-center\">",
+            "        <div>",
+            "          <div class=\"text-blue-600 text-4xl mb-2\">⚡</div>",
+            "          <h3 class=\"text-xl font-semibold mb-2\">Fast</h3>",
+            "          <p class=\"text-gray-600\">Built with performance in mind using modern tools.</p>",
+            "        </div>",
+            "        <div>",
+            "          <div class=\"text-blue-600 text-4xl mb-2\">🎨</div>",
+            "          <h3 class=\"text-xl font-semibold mb-2\">Beautiful</h3>",
+            "          <p class=\"text-gray-600\">Styled with Tailwind CSS for clean, responsive design.</p>",
+            "        </div>",
+            "        <div>",
+            "          <div class=\"text-blue-600 text-4xl mb-2\">🚀</div>",
+            "          <h3 class=\"text-xl font-semibold mb-2\">Launch Ready</h3>",
+            "          <p class=\"text-gray-600\">Easy to deploy and extend for your next big idea.</p>",
+            "        </div>",
+            "      </div>",
+            "    </div>",
+            "  </section>",
+            "</main>",
+            "",
+            "<!-- Footer -->",
+            "<footer class=\"bg-white text-gray-500 text-center py-6 border-t\">",
+            "  2025 Mulmocast.",
+            "</footer>"
+          ]
+        }
+      },
       {
         "speaker": "Presenter",
         "text": "This is the image of a pingpong ball.",

package/assets/templates/comic_strips.json

@@ -0,0 +1,35 @@
+{
+  "title": "American Comic Strips",
+  "description": "Template for Dilbert-style comic strips.",
+  "systemPrompt": "Generate a script for a presentation of the given topic. Another AI will generate comic strips for each beat based on the text description of that beat. Mention the reference in one of beats, if it exists. Use the JSON below as a template.",
+  "script": {
+    "$mulmocast": {
+      "version": "1.0",
+      "credit": "closing"
+    },
+    "title": "Sample Title",
+    "lang": "en",
+    "canvasSize": {
+      "width": 1536,
+      "height": 1024
+    },
+    "imageParams": {
+      "style": "<style>A multi panel comic strips. 1990s American workplace humor. Clean, minimalist line art with muted colors. One character is a nerdy office worker with glasses</style>"
+    },
+    "references": [
+      {
+        "url": "https://www.somegreatwebsite.com/article/123",
+        "title": "Title of the article we are referencing",
+        "type": "article"
+      }
+    ],
+    "beats": [
+      {
+        "text": "Today we're exploring a fascinating concept that has shaped some of the most innovative companies and leaders of our time. This story is based on an Awesome News articled, titled 'The reality distortion field'. This term has become synonymous with visionary leadership and the ability to accomplish seemingly impossible goals."
+      },
+      {
+        "text": "The Reality Distortion Field is a concept that was first introduced by Steve Jobs, the co-founder of Apple. It refers to the ability of a leader to convince people that something is possible, even if it seems impossible."
+      }
+    ]
+  }
+}

package/assets/templates/ghibli_strips.json

@@ -0,0 +1,35 @@
+{
+  "title": "American Comic Strips",
+  "description": "Template for Dilbert-style comic strips.",
+  "systemPrompt": "Generate a script for a presentation of the given topic. Another AI will generate comic strips for each beat based on the text description of that beat. Mention the reference in one of beats, if it exists. Use the JSON below as a template.",
+  "script": {
+    "$mulmocast": {
+      "version": "1.0",
+      "credit": "closing"
+    },
+    "title": "Sample Title",
+    "lang": "en",
+    "canvasSize": {
+      "width": 1536,
+      "height": 1024
+    },
+    "imageParams": {
+      "style": "<style>A multi panel comic strips. Ghibli style. The presenter is a young woman with a dark short hair with glasses.</style>"
+    },
+    "references": [
+      {
+        "url": "https://www.somegreatwebsite.com/article/123",
+        "title": "The Awesome News: The reality distortion field",
+        "type": "article"
+      }
+    ],
+    "beats": [
+      {
+        "text": "Today we're exploring a fascinating concept that has shaped some of the most innovative companies and leaders of our time. This story is based on an Awesome News articled, titled 'The reality distortion field'. This term has become synonymous with visionary leadership and the ability to accomplish seemingly impossible goals."
+      },
+      {
+        "text": "The Reality Distortion Field is a concept that was first introduced by Steve Jobs, the co-founder of Apple. It refers to the ability of a leader to convince people that something is possible, even if it seems impossible."
+      }
+    ]
+  }
+}