@aigne/doc-smith 0.9.7 → 0.9.8-beta

package/CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## [0.9.8-beta](https://github.com/AIGNE-io/aigne-doc-smith/compare/v0.9.7...v0.9.8-beta) (2025-12-05)
+
+
+### Features
+
+* use nano banana pro to generate doc diagrams ([#343](https://github.com/AIGNE-io/aigne-doc-smith/issues/343)) ([eaf9a06](https://github.com/AIGNE-io/aigne-doc-smith/commit/eaf9a06df8ecb57c1a39c3c338210f02b1b7ab94))
+
+
+### Bug Fixes
+
+* add global perspective for evaluate document ([#346](https://github.com/AIGNE-io/aigne-doc-smith/issues/346)) ([2f58699](https://github.com/AIGNE-io/aigne-doc-smith/commit/2f5869904db4aea9d8756bcbf0bf0710b8a14783))
+* add support for reading SVG content to generate media descriptions ([#347](https://github.com/AIGNE-io/aigne-doc-smith/issues/347)) ([a679604](https://github.com/AIGNE-io/aigne-doc-smith/commit/a6796040cfda58abe8a6c525b51605d8c5dcbd3d))
+
 ## [0.9.7](https://github.com/AIGNE-io/aigne-doc-smith/compare/v0.9.7-beta.3...v0.9.7) (2025-11-28)
 
 ## [0.9.7-beta.3](https://github.com/AIGNE-io/aigne-doc-smith/compare/v0.9.7-beta.2...v0.9.7-beta.3) (2025-11-28)

package/agents/create/analyze-diagram-type-llm.yaml

@@ -0,0 +1,160 @@
+name: analyzeDiagramTypeLLM
+description: Analyze document content using LLM to determine diagram type and select appropriate style
+model:
+  reasoning_effort: 1
+instructions: |
+  You are an AI assistant specialized in technical documentation visualization. Your task is to analyze a document segment and generate a structured visual plan for an image generator.
+
+  {% if feedback %}
+  **CRITICAL: User Feedback (HIGHEST PRIORITY)**
+  <feedback>
+  {{ feedback }}
+  </feedback>
+  
+  **IMPORTANT**: User feedback has the **HIGHEST PRIORITY** in all decision-making. Any explicit requests in the feedback (e.g., diagram type, style, colors, aspect ratio, size, layout preferences) must be respected and applied. Additionally, extract and note any other feedback information (such as color preferences, size requirements, layout specifications, etc.) that should be passed to subsequent image generation steps.
+  {% endif %}
+
+  Your responsibilities:
+
+  1. **Analyze Context**: Understand the document’s content, structure, and its purpose, especially around where the diagram will be inserted.
+
+  2. **Generate Document Summary**:
+    **CRITICAL**: The documentSummary will be the **only input** passed to the image generation model. Preserve as much information as possible, only removing content that is truly useless for diagram generation.
+
+    **What to PRESERVE (keep as much as possible):**
+    - **All structural elements**: Headings, sections, hierarchy, ordering, and document structure
+    - **All entities and components**: Names, roles, services, modules, actors, objects, and any elements that could appear as nodes
+    - **All relationships and connections**: How entities relate, data flows, dependencies, interactions, and any connections
+    - **All process flows and steps**: Sequential steps, decision points, workflows, logical order, and any process information
+    - **All labels and names**: All names, labels, identifiers, and terminology used in the document
+    - **Technical details**: Specifications, protocols, interfaces, configurations, and technical information
+    - **Examples and use cases**: Concrete examples, scenarios, and use cases that illustrate the concepts
+    - **Contextual information**: Explanatory text, background context, and descriptions that help understand relationships
+    - **All content that could inform diagram structure**: Any information that might be relevant for creating accurate diagrams
+
+    **What to REMOVE (only truly useless content):**
+    - **Verbatim duplicates**: Exact duplicate sentences or paragraphs that repeat the same information
+    - **Completely off-topic content**: Content that has no relation to the diagram subject matter
+    - **Pure marketing/promotional text**: Sales language that doesn't contain technical or structural information
+    - **Unrelated notes or comments**: Comments that are completely unrelated to the document's main content
+
+    **Summary Guidelines:**
+    - **Preserve the vast majority of content** - only remove content that is clearly redundant or completely unrelated
+    - Keep the original document structure, hierarchy, and organization
+    - Maintain all technical details, examples, and contextual information
+    - When in doubt, **keep the content** rather than removing it
+    - The summary should be comprehensive and contain all information that could be useful for diagram generation
+
+  3. **Determine Diagram Type**:
+    Choose one of the following types based on the content:
+    - **architecture**: Static system structure (components, containers, services)
+    - **flowchart**: Decision logic, workflows, process steps
+    - **guide**: Tutorials, step-by-step user journeys
+    - **intro**: Concept overviews, mind maps
+    - **sequence**: Time-based interactions between entities
+    - **network**: Logical or physical network topologies
+
+    **Decision Priority (in order):**
+    {% if feedback %}
+    0. **HIGHEST PRIORITY**: Analyze the user feedback carefully. If the feedback explicitly or implicitly specifies a diagram type (e.g., "architecture diagram", "flowchart", "sequence diagram", "流程图", "架构图") → **MUST use that type and override any other considerations**. Use your understanding of natural language to identify the user's intent. The feedback type takes absolute precedence.
+    {% endif %}
+    1. **Content Analysis**: If no type preference is found in feedback, analyze the document content structure and characteristics:
+       - If the document is an **overview** (e.g. titled `# Overview`, describes whole system/project) → use `"architecture"`.
+       - Sequential flow with time-based interactions → `sequence`
+       - Branching logic, decision points, workflows → `flowchart`
+       - User steps/tutorials, guided processes → `guide`
+       - Concept maps, high-level introductions → `intro`
+       - Infrastructure, network topologies → `network`
+
+  4. **Select Diagram Style**:
+    **Decision Priority (in order):**
+    {% if feedback %}
+    0. **HIGHEST PRIORITY**: Analyze the user feedback carefully. If the feedback explicitly or implicitly specifies a diagram style (e.g., "modern style", "hand-drawn", "anthropomorphic", "3d", "flat design", "现代风格", "手绘风格") → **MUST use that style and override any default style**. Use your understanding of natural language to identify the user's style preference. The feedback style takes absolute precedence.
+    {% endif %}
+    {% if defaultStyle %}
+    1. **Default Style**: If no style preference is found in feedback, use the configured default style: `{{ defaultStyle }}`. This is the user's preferred default style from configuration.
+    {% endif %}
+    2. **Content-Based Selection**: If no feedback style and no default style, choose a style appropriate for technical documentation tone based on the content characteristics. You can use any style name that best fits the content, including but not limited to:
+       - Common styles: `modern`, `standard`, `hand-drawn`, `anthropomorphic`, `flat`, `minimalist`, `3d`
+       - Other creative styles: `watercolor`, `sketch`, `vintage`, `cyberpunk`, `minimal`, `realistic`, `cartoon`, `isometric`, `neon`, `pastel`, etc.
+       - You are not limited to predefined styles - use your knowledge of visual styles to select the most appropriate one
+    3. **Available Styles Reference**: If `availableStyles` is provided and not empty, prefer styles from that list. However, if a better style is needed and not in the list, you can still use it. The `styleDescriptions` object provides descriptions of common styles for reference, but you are not restricted to only those styles.
+
+  5. **Recommend Aspect Ratio**:
+    {% if feedback %}
+    **HIGHEST PRIORITY**: If user feedback explicitly specifies an aspect ratio (e.g., "16:9", "4:3", "use landscape", "make it square") → **MUST use that aspect ratio**.
+    {% endif %}
+    
+    Otherwise, select the most suitable aspect ratio based on layout direction:
+    - `"1:1"`: Radial layouts, mind maps, central concepts
+    - `"5:4"` or `"4:3"`: Vertical flows (step-by-step, guides)
+    - `"3:2"`, `"16:9"`, `"21:9"`: Horizontal flows (timelines, architecture)
+
+    **Decision Logic:**
+    - Vertical flows → use `"4:3"` (default), or `"5:4"` for taller needs
+    - Horizontal flows → `"16:9"` (default), `"21:9"` for very wide, `"3:2"` for moderate width
+    - Central hub structures → use `"1:1"`
+
+    **Never** mismatch direction and ratio:
+    - Don't use portrait for horizontal content or vice versa
+    - Don't use `"1:1"` unless layout is truly radial
+
+  Document Content:
+  <document_content>
+  {{ documentContent }}
+  </document_content>
+  
+
+input_schema:
+  type: object
+  properties:
+    documentContent:
+      type: string
+      description: The document content to analyze
+    availableStyles:
+      type: array
+      description: List of available diagram styles
+      items:
+        type: string
+    styleDescriptions:
+      type: object
+      description: Style descriptions
+      additionalProperties:
+        type: string
+    locale:
+      type: string
+      description: Language for labels
+      default: en
+    feedback:
+      type: string
+      description: User feedback that may contain style, type, or other preferences. You should analyze this feedback carefully to extract any explicit or implicit preferences. If feedback specifies a style or type, it MUST override the defaultStyle.
+      default: ""
+    defaultStyle:
+      type: string
+      description: Default diagram style from configuration. Use this only if no style preference is found in feedback. If feedback specifies a style, it takes precedence over this default.
+      nullable: true
+  required:
+    - documentContent
+    - availableStyles
+output_schema:
+  type: object
+  properties:
+    documentSummary:
+      type: string
+      description: A comprehensive summary that preserves the vast majority of the original document content. Only remove verbatim duplicates, completely off-topic content, or pure marketing text. Keep all structural elements, entities, relationships, processes, technical details, examples, and contextual information. This summary will be the only content passed to the image generation model.
+    diagramType:
+      type: string
+      description: The selected diagram type
+    diagramStyle:
+      type: string
+      description: The selected diagram style. Can be any style name (e.g., 'modern', 'hand-drawn', 'watercolor', 'cyberpunk', 'isometric', etc.). Not limited to predefined styles - use your knowledge of visual styles to select the most appropriate one.
+    aspectRatio:
+      type: string
+      description: Recommended aspect ratio for the image based on content structure analysis. MUST match the primary flow direction (vertical→portrait, horizontal→landscape, radial→square)
+      enum: ["1:1", "5:4", "4:3", "3:2", "16:9", "21:9"]
+  required:
+    - documentSummary
+    - diagramType
+    - diagramStyle
+    - aspectRatio
+

package/agents/create/analyze-diagram-type.mjs

@@ -0,0 +1,297 @@
+import { DIAGRAM_STYLES } from "../../utils/constants/index.mjs";
+
+const DEFAULT_DIAGRAM_STYLE = "modern";
+const DEFAULT_DIAGRAM_TYPE = "flowchart";
+
+// Type-specific content requirements
+const TYPE_REQUIREMENTS = {
+  architecture: `- Accurately represent the system architecture, components, services, and their relationships
+  - Show clear component boundaries and service interactions
+  - Include all key architectural elements (layers, modules, services, databases, APIs)
+  - Display data flow and communication patterns between components
+  - Use clear labels for each component and connection`,
+  flowchart: `- Accurately represent the process flow, steps, decisions, and workflow
+  - Show clear step-by-step progression with decision points
+  - Use standard flowchart symbols: rectangles for processes, diamonds for decisions, arrows for flows
+  - Include all key steps and decision branches
+  - Maintain logical flow direction (top-to-bottom or left-to-right)`,
+  guide: `- Show user journey, tutorial flow, or guided process
+  - Display clear progression from start to completion
+  - Include key milestones, checkpoints, or decision points
+  - Use clear visual cues to guide the viewer through the process
+  - Make it easy to follow and understand the path`,
+  intro: `- Provide a high-level overview or conceptual explanation
+  - Show main concepts, relationships, and key ideas
+  - Use clear visual hierarchy to emphasize important elements
+  - Make it accessible and easy to understand for newcomers
+  - Focus on big picture rather than detailed implementation`,
+  sequence: `- Show interactions over time between components or actors
+  - Display clear message flow and timing
+  - Include all participating entities and their interactions
+  - Show chronological order of events
+  - Use clear labels for messages and interactions`,
+  network: `- Show network structure, nodes, and connections
+  - Display routing paths and network topology
+  - Include all network components (routers, switches, servers, clients)
+  - Show connection types and data flow directions
+  - Use clear labels for network elements`,
+};
+
+// Style-specific requirements
+const STYLE_REQUIREMENTS = {
+  modern: `- Modern, clean, professional diagram style
+  - Contemporary design elements with smooth lines
+  - Professional color scheme suitable for technical documentation
+  - Clear visual hierarchy and readable text
+  - Sleek and polished appearance`,
+  standard: `- Standard flowchart style with traditional symbols
+  - Conventional formatting and clear structure
+  - Rectangles for processes, diamonds for decisions, arrows for flows
+  - Clear, readable text labels
+  - Professional and familiar appearance`,
+  "hand-drawn": `- Hand-drawn, sketch-like style with natural, organic lines
+  - Slightly imperfect shapes for a casual, approachable appearance
+  - Natural line variations and hand-drawn aesthetics
+  - Friendly and informal visual style
+  - Avoid perfect geometric shapes`,
+  anthropomorphic: `- Anthropomorphic style with personified elements
+  - Vivid and lively imagery with characters or objects having human-like features
+  - Engaging and memorable visual elements
+  - Creative and expressive design
+  - Make abstract concepts more relatable through personification`,
+  flat: `- Flat design style with no shadows, gradients, or 3D effects
+  - Clean geometric shapes with bold colors
+  - Minimalist aesthetics with simple, flat surfaces
+  - Modern and clean appearance
+  - Avoid depth and dimensionality`,
+  minimalist: `- Minimalist style with the fewest possible elements
+  - Maximum clarity with simple shapes
+  - Ample white space and essential information only
+  - Clean and uncluttered appearance
+  - Focus on core message without distractions`,
+  "3d": `- 3D style with three-dimensional effects and perspective
+  - Depth, shadows, and realistic spatial relationships
+  - Three-dimensional appearance with volume and dimension
+  - Professional and modern 3D rendering
+  - Clear depth cues and perspective`,
+};
+
+/**
+ * Analyze document content to determine diagram type and select appropriate style
+ * Uses LLM analysis to determine diagram type and style
+ * Supports extracting style and type preferences from user feedback
+ */
+export default async function analyzeDiagramType(
+  {
+    documentContent,
+    availableStyles = [],
+    defaultStyle,
+    diagramming,
+    locale = "en",
+    feedback = "",
+  },
+  options,
+) {
+  // Extract defaultStyle from diagramming object if not provided directly
+  if (!defaultStyle && diagramming?.style) {
+    defaultStyle = diagramming.style;
+  }
+
+  // Step 1: Use LLM to analyze and make final decision (LLM will analyze feedback directly)
+  const llmAgent = options.context?.agents?.["analyzeDiagramTypeLLM"];
+  let llmResult = null;
+
+  if (llmAgent) {
+    try {
+      // Build styleDescriptions object for template
+      // Include predefined styles as reference, but allow LLM to use any style
+      const styleDescriptions = {};
+      const stylesToUse =
+        availableStyles.length > 0 ? availableStyles : Object.keys(DIAGRAM_STYLES);
+      for (const style of stylesToUse) {
+        if (DIAGRAM_STYLES[style]) {
+          styleDescriptions[style] =
+            DIAGRAM_STYLES[style].description || DIAGRAM_STYLES[style].name;
+        }
+      }
+      // Also include all predefined styles as reference even if not in availableStyles
+      // This helps LLM understand common style options but doesn't restrict it
+      for (const [style, styleInfo] of Object.entries(DIAGRAM_STYLES)) {
+        if (!styleDescriptions[style]) {
+          styleDescriptions[style] = styleInfo.description || styleInfo.name;
+        }
+      }
+
+      const llmInput = {
+        documentContent,
+        availableStyles: stylesToUse,
+        styleDescriptions,
+        locale,
+        feedback: feedback || "",
+        defaultStyle: defaultStyle || null,
+      };
+
+      llmResult = await options.context.invoke(llmAgent, llmInput);
+    } catch (error) {
+      console.warn(`⚠️  LLM analysis failed: ${error.message}`);
+    }
+  }
+
+  // Step 2: Determine diagram type
+  // Priority: LLM result (which already analyzed feedback) > default
+  const diagramType = llmResult?.diagramType || DEFAULT_DIAGRAM_TYPE;
+
+  // Step 3: Select style
+  // Trust LLM to always return a valid style (required in output_schema)
+  // LLM can return any style name, not limited to predefined styles
+  // Only use fallback if LLM completely failed
+  const diagramStyle = llmResult?.diagramStyle || defaultStyle || DEFAULT_DIAGRAM_STYLE;
+
+  // Note: We allow any style name from LLM, even if not in availableStyles
+  // This enables creative styles beyond predefined ones (e.g., 'watercolor', 'cyberpunk', 'isometric')
+  // If availableStyles is provided and not empty, it serves as a preference guide, not a strict restriction
+
+  // Step 4: Generate prompt requirements for image generation
+  const diagramTypeRequirements =
+    TYPE_REQUIREMENTS[diagramType] || TYPE_REQUIREMENTS[DEFAULT_DIAGRAM_TYPE];
+  const diagramStyleRequirements =
+    STYLE_REQUIREMENTS[diagramStyle] || STYLE_REQUIREMENTS[DEFAULT_DIAGRAM_STYLE];
+
+  // Generate negative prompt exclusions based on style
+  let negativePromptExclusions = "";
+  if (diagramStyle !== "anthropomorphic") {
+    negativePromptExclusions += ", anthropomorphic";
+  }
+  if (diagramStyle !== "hand-drawn") {
+    negativePromptExclusions += ", hand-drawn, sketch";
+  }
+
+  // Step 5: Extract document summary from LLM result
+  // The LLM creates a concise summary focusing on key elements for diagram generation
+  // This ensures both the analysis model and image generation model have consistent understanding
+  const documentSummary = llmResult?.documentSummary || documentContent;
+
+  // If LLM didn't provide a summary (fallback), use original content
+  // But prefer the LLM-generated summary as it's focused and aligned with the analysis
+
+  // Step 6: Determine aspect ratio from LLM result
+  // The LLM analyzes the content structure and recommends the best aspect ratio
+  // We trust the LLM's judgment as it has analyzed the actual content
+  // If LLM doesn't provide aspectRatio (shouldn't happen, but fallback for safety), use 4:3 as safe default
+  let aspectRatio = llmResult?.aspectRatio || "4:3";
+
+  // Validate that the aspectRatio is one of the supported values
+  const supportedRatios = ["1:1", "5:4", "4:3", "3:2", "16:9", "21:9"];
+  if (!supportedRatios.includes(aspectRatio)) {
+    console.warn(`Invalid aspectRatio "${aspectRatio}" from LLM, falling back to "4:3"`);
+    aspectRatio = "4:3";
+  }
+
+  // Step 7: Return document content and summary for image generation
+  return {
+    diagramType,
+    diagramStyle,
+    aspectRatio,
+    documentContent, // The full document content (kept for backward compatibility and additional context)
+    documentSummary, // The concise summary generated by LLM, focused on key elements for diagram generation
+    diagramTypeRequirements,
+    diagramStyleRequirements,
+    negativePromptExclusions,
+  };
+}
+
+analyzeDiagramType.input_schema = {
+  type: "object",
+  properties: {
+    documentContent: {
+      type: "string",
+      description: "The document content to analyze for diagram type and style selection",
+    },
+    availableStyles: {
+      type: "array",
+      description:
+        "List of available diagram styles configured by user (optional restriction). If empty, any style is allowed.",
+      items: {
+        type: "string",
+      },
+    },
+    defaultStyle: {
+      type: "string",
+      description:
+        "Default diagram style to use when no style is specified in feedback. Can be any style name, not limited to predefined styles.",
+    },
+    diagramming: {
+      type: "object",
+      description: "Diagramming configuration object (alternative way to pass style)",
+      properties: {
+        style: {
+          type: "string",
+          description: "Default diagram style",
+        },
+      },
+    },
+    locale: {
+      type: "string",
+      description: "Language for analysis",
+      default: "en",
+    },
+    feedback: {
+      type: "string",
+      description:
+        "User feedback that may contain style or type preferences (e.g., 'use anthropomorphic style', 'create architecture diagram')",
+      default: "",
+    },
+  },
+  required: ["documentContent"],
+};
+
+analyzeDiagramType.output_schema = {
+  type: "object",
+  properties: {
+    diagramType: {
+      type: "string",
+      description: "The detected diagram type",
+    },
+    diagramStyle: {
+      type: "string",
+      description: "The selected diagram style",
+    },
+    diagramTypeRequirements: {
+      type: "string",
+      description: "Content requirements for the diagram type",
+    },
+    diagramStyleRequirements: {
+      type: "string",
+      description: "Style requirements for the diagram style",
+    },
+    negativePromptExclusions: {
+      type: "string",
+      description: "Additional negative prompt exclusions based on style",
+    },
+    aspectRatio: {
+      type: "string",
+      description: "Aspect ratio for the diagram (must match content flow direction)",
+      enum: ["1:1", "5:4", "4:3", "3:2", "16:9", "21:9"],
+    },
+    documentContent: {
+      type: "string",
+      description:
+        "The full document content (kept for backward compatibility and additional context)",
+    },
+    documentSummary: {
+      type: "string",
+      description:
+        "A concise summary of the document content focusing on key elements needed for diagram generation. This summary is generated by the analysis LLM to ensure consistent understanding between analysis and image generation models.",
+    },
+  },
+  required: [
+    "diagramType",
+    "diagramStyle",
+    "aspectRatio",
+    "documentSummary",
+    "diagramTypeRequirements",
+    "diagramStyleRequirements",
+    "negativePromptExclusions",
+    "documentContent",
+  ],
+};

package/agents/create/generate-diagram-image.yaml

@@ -0,0 +1,60 @@
+type: image
+name: generateDiagramImage
+image_model:
+  model: google/gemini-3-pro-image-preview
+  #  The cues that come with thought patterns are actually not obvious
+  # thinkingConfig: 
+  #   includeThoughts: true      
+  # responseModalities: 
+  #   - 'TEXT'
+  #   - 'IMAGE'
+  imageConfig:
+    imageSize:
+      $get: size
+    aspectRatio:
+      $get: ratio
+
+instructions:
+  - role: system
+    url: ../../prompts/detail/diagram/generate-image-system.md
+  - role: user
+    url: ../../prompts/detail/diagram/generate-image-user.md
+
+input_schema:
+  type: object
+  properties:
+    documentContent:
+      type: string
+      description: The full original document content
+    documentSummary:
+      type: string
+      description: A comprehensive summary of the document content for diagram generation (preferred over documentContent if available)
+    diagramType:
+      type: string
+      description: The type of diagram to generate (architecture, flowchart, guide, intro, sequence, network)
+    diagramStyle:
+      type: string
+      description: The visual style for the diagram (modern, standard, hand-drawn, anthropomorphic, flat, minimalist, 3d)
+    locale:
+      type: string
+      description: Language for diagram labels
+      default: en
+    size:
+      type: string
+      description: Size of the generated image (e.g., "1K", "2K")
+      default: "1K"
+    ratio:
+      type: string
+      description: Aspect ratio of the generated image (must match content flow direction)
+      enum: ["1:1", "5:4", "4:3", "3:2", "16:9", "21:9"]
+    aspectRatio:
+      type: string
+      description: Aspect ratio of the generated image (alias for ratio, used in prompt templates)
+      enum: ["1:1", "5:4", "4:3", "3:2", "16:9", "21:9"]
+  required:
+    - documentContent
+    - diagramType
+    - diagramStyle
+    - ratio
+include_input_in_output: true
+