@gradio/imageeditor 0.12.14 → 0.13.0

package/shared/image/IMAGE.md

@@ -0,0 +1,135 @@
+# Image Tool Documentation
+
+## Overview
+
+The Image Tool is a component of the image editor that handles adding and managing background images on the canvas. It allows users to upload, paste, or capture images from a webcam and set them as the background of the editing canvas. The tool manages image sizing, positioning, and integration with the layer system.
+
+## File Structure
+
+- `js/imageeditor/shared/image/image.ts` - Main implementation of the image tool
+- `js/imageeditor/shared/image/Sources.svelte` - UI component for image source options
+- `js/imageeditor/shared/core/editor.ts` - Defines the `Tool` interface and `ImageEditorContext`
+- `js/imageeditor/shared/Toolbar.svelte` - Defines the tool types and subtool types
+
+## Implementation Details
+
+### Class: `ImageTool`
+
+The `ImageTool` class implements the `Tool` interface defined in `editor.ts`. It provides the following functionality:
+
+- Adding images to the canvas
+- Managing image dimensions and positioning
+- Integrating with the layer system
+
+### Key Components
+
+#### `ImageTool` Class
+
+The main class that implements the `Tool` interface with methods:
+- `setup(context, tool, subtool)` - Initializes the tool with the editor context
+- `cleanup()` - Cleans up resources
+- `add_image(image, fixed_canvas)` - Adds an image to the canvas
+- `set_tool(tool, subtool)` - Updates the current tool and subtool
+
+#### `AddImageCommand` Class
+
+Implements the command pattern for adding images, allowing for undo/redo functionality:
+- `start()` - Initializes the image sprite and calculates dimensions
+- `execute()` - Adds the image to the canvas and updates the editor state
+- `undo()` - Removes the image from the canvas
+
+#### Helper Functions
+
+- `fit_image_to_canvas(image_width, image_height, canvas_width, canvas_height)` - Calculates dimensions to fit an image within the canvas while maintaining aspect ratio
+- `add_bg_color(container, renderer, color, width, height, resize)` - Adds a solid color background to the canvas
+
+### Image Processing Flow
+
+1. **Image Acquisition**: The image is acquired as a Blob or File from one of the sources (upload, clipboard, webcam)
+2. **Image Processing**: 
+   - The image is converted to a bitmap and then to a PIXI.js Texture
+   - The dimensions are calculated based on whether fixed_canvas is true or false
+   - If fixed_canvas is true, the image is scaled to fit the canvas while maintaining aspect ratio
+   - If fixed_canvas is false, the canvas is resized to match the image dimensions
+
+3. **Canvas Integration**:
+   - The editor's image properties are updated with the new dimensions
+   - Existing layers are preserved and scaled to match the new dimensions
+   - A new background layer is created with the image sprite
+   - The image is centered in the viewport
+
+4. **Layer Management**:
+   - The image is added as a sprite to a background layer
+   - Existing layers are preserved and scaled to match the new dimensions
+   - If no layers exist, an initial drawing layer is created
+
+## Command Pattern Implementation
+
+The image tool uses the command pattern to implement undo/redo functionality:
+
+1. **Command Creation**: When adding an image, an `AddImageCommand` is created
+2. **Command Execution**: The command's `execute()` method is called to add the image
+3. **Command Registration**: The command is registered with the editor's command manager
+4. **Undo Support**: The command's `undo()` method can be called to remove the image
+
+## Integration with Editor
+
+The image tool integrates with the editor through the `ImageEditorContext` interface, which provides:
+
+- `app` - The PIXI.js Application instance
+- `layer_manager` - Manages the layers in the editor
+- `set_image_properties` - Updates the image dimensions and position
+- `set_background_image` - Sets the background image sprite
+- `execute_command` - Registers a command with the command manager
+
+## Usage Flow
+
+1. The user selects an image source (upload, clipboard, webcam)
+2. The image is acquired as a Blob or File
+3. The `add_image` method is called with the image and a flag indicating whether to maintain the canvas size
+4. An `AddImageCommand` is created and executed
+5. The image is added to the canvas as a background layer
+6. The editor's state is updated with the new dimensions and position
+
+## Implementation Notes
+
+### Image Scaling
+
+The tool provides two modes for handling image dimensions:
+
+1. **Fixed Canvas Mode** (fixed_canvas = true):
+   - The image is scaled to fit within the canvas dimensions
+   - The aspect ratio is maintained
+   - The canvas size remains unchanged
+
+2. **Flexible Canvas Mode** (fixed_canvas = false):
+   - The canvas is resized to match the image dimensions
+   - No scaling is applied to the image
+   - Existing layers are scaled to match the new dimensions
+
+### Layer Preservation
+
+When adding a new background image:
+
+1. Existing layers are preserved
+2. Layer textures are captured before modification
+3. New layers are created with the new dimensions
+4. Content from old layers is scaled and centered on the new layers
+5. If no layers exist, an initial drawing layer is created
+
+## Maintenance Notes
+
+When modifying the image tool, consider:
+
+1. **Command Pattern**: Ensure that all modifications to the canvas state are implemented as commands for proper undo/redo support
+2. **Layer Management**: Be careful with layer creation and destruction to avoid memory leaks
+3. **Image Scaling**: Ensure that aspect ratios are maintained when scaling images
+4. **Performance**: Large images may need to be downsampled for performance
+5. **Memory Management**: Properly destroy textures and sprites when they are no longer needed
+
+## Related Components
+
+- **Toolbar**: Controls tool selection
+- **ImageEditor**: Provides the context and manages the overall editor state
+- **LayerManager**: Manages image layers
+- **Sources.svelte**: Provides UI for selecting image sources 

package/shared/image/image.ts

@@ -0,0 +1,300 @@
+import {
+	Container,
+	Graphics,
+	Color,
+	type ColorSource,
+	RenderTexture,
+	Sprite,
+	type Renderer,
+	Texture
+} from "pixi.js";
+import { type ImageEditorContext, type Tool } from "../core/editor";
+import { type Tool as ToolbarTool, type Subtool } from "../Toolbar.svelte";
+import { type Command } from "../utils/commands";
+import { make_graphics } from "../utils/pixi";
+import { get } from "svelte/store";
+
+/**
+ * Handles adding images to the editor
+ * Only a single image can be added at a time
+ * Adding an image removes the current image is one exists
+ */
+export class ImageTool implements Tool {
+	name = "image";
+	context!: ImageEditorContext;
+	current_tool!: ToolbarTool;
+	current_subtool!: Subtool;
+	async setup(
+		context: ImageEditorContext,
+		tool: ToolbarTool,
+		subtool: Subtool
+	): Promise<void> {
+		this.context = context;
+		this.current_tool = tool;
+		this.current_subtool = subtool;
+	}
+
+	cleanup(): void {}
+
+	async add_image({
+		image,
+		fixed_canvas,
+		border_region = 0
+	}: {
+		image: Blob | File | Texture;
+		fixed_canvas: boolean;
+		border_region?: number;
+	}): Promise<void> {
+		const image_command = new AddImageCommand(
+			this.context,
+			image,
+			fixed_canvas,
+			border_region
+		);
+
+		await image_command.start();
+		this.context.execute_command(image_command);
+	}
+
+	set_tool(tool: ToolbarTool, subtool: Subtool): void {
+		this.current_tool = tool;
+		this.current_subtool = subtool;
+	}
+}
+
+interface BgImageCommand extends Command {
+	/**
+	 * Initial setup for the bg command
+	 * @returns
+	 */
+	start: () => Promise<[number, number]>;
+}
+
+export class AddImageCommand implements BgImageCommand {
+	sprite: Sprite;
+	fixed_canvas: boolean;
+	context: ImageEditorContext;
+	background: Blob | File | Texture;
+	current_canvas_size: { width: number; height: number };
+	current_scale: number;
+	current_position: { x: number; y: number };
+	border_region: number;
+
+	scaled_width!: number;
+	scaled_height!: number;
+
+	constructor(
+		context: ImageEditorContext,
+		background: Blob | File | Texture,
+		fixed_canvas: boolean,
+		border_region = 0
+	) {
+		this.context = context;
+		this.background = background;
+		this.fixed_canvas = fixed_canvas;
+		this.border_region = border_region;
+		this.current_canvas_size = get(this.context.dimensions);
+		this.current_scale = get(this.context.scale);
+		this.current_position = get(this.context.position);
+		this.sprite = new Sprite();
+	}
+
+	async start(): Promise<[number, number]> {
+		// Create the sprite from the blob
+		let image_texture: Texture;
+		if (this.background instanceof Texture) {
+			image_texture = this.background;
+		} else {
+			const img = await createImageBitmap(this.background);
+			image_texture = Texture.from(img);
+		}
+
+		this.sprite = new Sprite(image_texture);
+
+		return this.handle_image();
+	}
+
+	private handle_image(): [number, number] {
+		// Handle fixed canvas differently when border region is present
+		if (this.fixed_canvas) {
+			// Calculate the effective canvas size accounting for border region
+			const effectiveCanvasWidth = Math.max(
+				this.current_canvas_size.width - this.border_region * 2,
+				10
+			);
+			const effectiveCanvasHeight = Math.max(
+				this.current_canvas_size.height - this.border_region * 2,
+				10
+			);
+
+			// Fit the image to the canvas while maintaining aspect ratio
+
+			const { width, height, x, y } = fit_image_to_canvas(
+				this.sprite.width,
+				this.sprite.height,
+				effectiveCanvasWidth,
+				effectiveCanvasHeight
+			);
+
+			this.sprite.width = width;
+			this.sprite.height = height;
+
+			// Center the image in the canvas
+			this.sprite.x = x + this.border_region;
+			this.sprite.y = y + this.border_region;
+		} else {
+			// For non-fixed canvas, use natural dimensions plus border
+			const width = this.sprite.width;
+			const height = this.sprite.height;
+
+			// Position at the border's offset from origin
+			this.sprite.x = this.border_region;
+			this.sprite.y = this.border_region;
+
+			return [width + this.border_region * 2, height + this.border_region * 2];
+		}
+
+		return [this.current_canvas_size.width, this.current_canvas_size.height];
+	}
+
+	async execute(): Promise<void> {
+		// First ensure we have run start() to create the sprite and get dimensions
+		const [width, height] = await this.start();
+
+		// Update image properties with the original dimensions and center in viewport
+		this.context.set_image_properties({
+			scale: 1, // Start at 1:1 scale
+			position: {
+				x: this.context.app.screen.width / 2,
+				y: this.context.app.screen.height / 2
+			},
+			width: this.fixed_canvas ? this.current_canvas_size.width : width,
+			height: this.fixed_canvas ? this.current_canvas_size.height : height
+		});
+
+		// Create new background layer and add the sprite
+		const background_layer = this.context.layer_manager.create_background_layer(
+			this.fixed_canvas ? this.current_canvas_size.width : width,
+			this.fixed_canvas ? this.current_canvas_size.height : height
+		);
+		this.sprite.zIndex = 0;
+		background_layer.addChild(this.sprite);
+
+		// Resize and preserve content of existing layers
+		this.context.layer_manager.reset_layers(
+			this.fixed_canvas ? this.current_canvas_size.width : width,
+			this.fixed_canvas ? this.current_canvas_size.height : height,
+			true
+		);
+
+		this.context.set_background_image(this.sprite);
+
+		this.context.reset();
+	}
+
+	async undo(): Promise<void> {
+		if (this.sprite) {
+			this.sprite.destroy();
+		}
+		// The background layer will be automatically cleaned up when a new image is added
+	}
+}
+
+/**
+ * Command that sets a background
+ */
+interface BgColorCommand extends Command {
+	/**
+	 * Initial setup for the bg command
+	 * @returns
+	 */
+	start: () => [number, number];
+}
+
+/**
+ * Adds a background color to the canvas.
+ * @param container The container to add the image to.
+ * @param renderer The renderer to use for the image.
+ * @param color The background color to add.
+ * @param width The width of the background.
+ * @param height The height of the background.
+ * @param resize The function to resize the canvas.
+ * @returns A command that can be used to undo the action.
+ */
+export function add_bg_color(
+	container: Container,
+	renderer: Renderer,
+	color: ColorSource,
+	width: number,
+	height: number,
+	resize: (width: number, height: number) => void
+): BgColorCommand {
+	let sprite: Sprite;
+	return {
+		start() {
+			container.removeChildren();
+			const graphics = make_graphics(1);
+			const texture = RenderTexture.create({
+				width,
+				height
+			});
+
+			renderer.render(graphics, { renderTexture: texture });
+			sprite = new Sprite(texture);
+			return [sprite.width, sprite.height];
+		},
+		async execute() {
+			resize(sprite.width, sprite.height);
+			sprite.zIndex = 1;
+			container.addChild(sprite);
+		},
+		undo() {
+			container.removeChildren();
+		}
+	};
+}
+
+/**
+ * Calculates new dimensions and position for an image to fit within a canvas while maintaining aspect ratio
+ * @param image_width Original width of the image
+ * @param image_height Original height of the image
+ * @param canvas_width Width of the canvas
+ * @param canvas_height Height of the canvas
+ * @returns Object containing new dimensions and position
+ */
+export function fit_image_to_canvas(
+	image_width: number,
+	image_height: number,
+	canvas_width: number,
+	canvas_height: number
+): {
+	width: number;
+	height: number;
+	x: number;
+	y: number;
+} {
+	const image_aspect_ratio = image_width / image_height;
+	const canvas_aspect_ratio = canvas_width / canvas_height;
+
+	let new_width: number;
+	let new_height: number;
+
+	if (image_aspect_ratio > canvas_aspect_ratio) {
+		new_width = canvas_width;
+		new_height = canvas_width / image_aspect_ratio;
+	} else {
+		new_height = canvas_height;
+		new_width = canvas_height * image_aspect_ratio;
+	}
+
+	// center image
+	const x = Math.round((canvas_width - new_width) / 2);
+	const y = Math.round((canvas_height - new_height) / 2);
+
+	return {
+		width: Math.round(new_width),
+		height: Math.round(new_height),
+		x,
+		y
+	};
+}