@gradio/imageeditor 0.12.14 → 0.13.0

package/CHANGELOG.md

@@ -1,5 +1,19 @@
 # @gradio/imageeditor
 
+## 0.13.0
+
+### Features
+
+- [#10635](https://github.com/gradio-app/gradio/pull/10635) [`2f68c9d`](https://github.com/gradio-app/gradio/commit/2f68c9d988dcbc53a0b8e53bdb1de49c9c8c65d8) - Refactor and redesign `ImageEditor` component.  Thanks @pngwn!
+
+### Dependency updates
+
+- @gradio/statustracker@0.10.7
+- @gradio/atoms@0.15.0
+- @gradio/icons@0.11.0
+- @gradio/upload@0.16.0
+- @gradio/image@0.22.0
+
 ## 0.12.14
 
 ### Dependency updates

package/Example.svelte

@@ -1,6 +1,6 @@
 <script lang="ts">
 	import { BaseImage as Image } from "@gradio/image";
-	import type { EditorData } from "./shared/InteractiveImageEditor.svelte";
+	import type { EditorData } from "./InteractiveImageEditor.svelte";
 
 	export let value: EditorData;
 	export let type: "gallery" | "table";

package/IMAGE_EDITOR_OVERVIEW.md

@@ -0,0 +1,224 @@
+# Image Editor Overview
+
+## Introduction
+
+The Image Editor is a powerful, web-based tool built with PIXI.js and Svelte that allows users to edit images through a variety of operations including drawing, erasing, cropping, and resizing. It features a layered architecture, undo/redo functionality, and a modular tool system. This document provides a high-level overview of the editor's architecture and components to help developers understand the system before diving into specific implementations.
+
+## Architecture Overview
+
+The image editor follows a modular architecture with several key components:
+
+1. **Core Editor** - The central component that manages the canvas, tools, layers, and user interactions
+2. **Tool System** - A pluggable system for different editing operations (crop, draw, erase, etc.)
+3. **Layer Management** - Handles multiple layers for complex image compositions
+4. **Command Pattern** - Implements undo/redo functionality through commands
+5. **UI Components** - Svelte components that provide the user interface
+6. **Rendering Pipeline** - PIXI.js-based rendering system for the canvas
+
+### Component Hierarchy
+
+```
+InteractiveImageEditor.svelte
+└── ImageEditor.svelte
+    ├── Core Editor (editor.ts)
+    │   ├── Command Manager
+    │   └── Layer Manager
+    ├── Tools
+    │   ├── Image Tool (image.ts)
+    │   ├── Crop Tool (crop.ts)
+    │   ├── Brush Tool (brush.ts)
+    │   ├── Resize Tool (resize.ts)
+    │   └── Zoom Tool (zoom.ts)
+    └── UI Components
+        ├── Toolbar.svelte
+        ├── Controls.svelte
+        └── Tool-specific UI components
+```
+
+## Key Components
+
+### Core Editor
+
+The Core Editor (`editor.ts`) is the central component that initializes and manages the editor. It:
+
+- Sets up the PIXI.js application and containers
+- Manages tools and handles tool switching
+- Maintains the editor state (scale, position, dimensions)
+- Executes commands and manages undo/redo
+- Handles the rendering loop
+
+The editor uses Svelte stores for reactive state management and springs for smooth animations.
+
+### Tool System
+
+The editor implements a pluggable tool system where each tool follows the `Tool` interface:
+
+```typescript
+interface Tool {
+  name: string;
+  setup(context: ImageEditorContext, tool: ToolbarTool, subtool: Subtool): Promise<void>;
+  cleanup(): void;
+  set_tool(tool: ToolbarTool, subtool: Subtool): void;
+}
+```
+
+Each tool receives the editor context during setup, which provides access to the PIXI.js application, containers, and other utilities. Tools are responsible for handling their specific functionality and cleaning up resources when deactivated.
+
+#### Available Tools
+
+1. **Image Tool** - Handles adding and managing background images
+2. **Crop Tool** - Allows selecting a portion of the image to keep
+3. **Brush Tool** - Provides drawing functionality with customizable brushes
+4. **Erase Tool** - Allows erasing parts of the image
+5. **Resize Tool** - Enables resizing the canvas
+6. **Zoom Tool** - Handles zooming and panning the canvas
+
+### Layer Management
+
+The editor supports a layered approach to image editing:
+
+- **Background Layer** - Contains the background image
+- **Drawing Layers** - Contains user drawings and modifications
+- **UI Layer** - Contains UI elements that overlay the canvas
+
+Each layer has associated textures for rendering and can be manipulated independently.
+
+### Command Pattern
+
+The editor implements the Command pattern for undo/redo functionality:
+
+```typescript
+interface Command {
+  execute(): void;
+  undo(): void;
+}
+```
+
+Operations that modify the canvas state (like adding an image, drawing, or cropping) are implemented as commands. This allows for complex operations to be encapsulated and reversed.
+
+### UI Components
+
+The UI is built with Svelte components:
+
+- **ImageEditor.svelte** - The main editor component
+- **Toolbar.svelte** - Provides tool selection
+- **Controls.svelte** - Provides additional controls (save, undo, redo)
+- **Tool-specific components** - Provide UI for specific tools (BrushOptions, ColorPicker, etc.)
+
+### Rendering Pipeline
+
+The editor uses PIXI.js for rendering:
+
+1. **Layer Rendering** - Each layer renders its content to a texture
+2. **Container Composition** - Layers are composed in the image container
+3. **UI Overlay** - UI elements are rendered on top of the image
+4. **Scale and Position** - The image container is scaled and positioned based on user interactions
+
+## Data Flow
+
+1. **User Interaction** - User interacts with the UI or canvas
+2. **Tool Handling** - The active tool handles the interaction
+3. **Command Creation** - A command is created for operations that modify the canvas
+4. **Command Execution** - The command is executed and registered with the command manager
+5. **State Update** - The editor state is updated
+6. **Rendering** - The canvas is re-rendered to reflect the changes
+
+## Integration Points
+
+### Svelte Integration
+
+The editor is designed to work with Svelte:
+
+- **Stores** - Uses Svelte stores for reactive state management
+- **Springs** - Uses Svelte springs for smooth animations
+- **Component Integration** - Can be integrated with Svelte components
+
+### External Integration
+
+The editor can be integrated with external systems:
+
+- **File Upload** - Supports uploading images from various sources
+- **Export** - Can export the edited image in various formats
+- **History** - Can save and restore editing history
+
+## Design Considerations
+
+### Performance
+
+The editor uses several techniques to maintain performance:
+
+- **Texture Management** - Efficiently manages textures to minimize memory usage
+- **Layer Composition** - Composes layers efficiently to minimize rendering overhead
+- **Event Throttling** - Throttles events to avoid excessive updates
+- **Resolution Scaling** - Adjusts resolution based on device pixel ratio
+
+### Extensibility
+
+The editor is designed to be extensible:
+
+- **Tool Interface** - New tools can be added by implementing the Tool interface
+- **Command Pattern** - New operations can be added by implementing the Command interface
+- **Layer System** - The layer system can be extended to support new layer types
+
+### Usability
+
+The editor prioritizes usability:
+
+- **Responsive UI** - The UI adapts to different screen sizes
+- **Smooth Animations** - Uses springs for smooth transitions
+- **Intuitive Controls** - Provides familiar controls for common operations
+- **Visual Feedback** - Provides visual feedback for user actions
+
+## Implementation Notes
+
+When working with the image editor, consider the following:
+
+1. **Resource Management** - Always clean up resources (textures, sprites, event listeners) to prevent memory leaks
+2. **Event Handling** - Be careful with event propagation and stopping
+3. **Coordinate Systems** - Be aware of the different coordinate systems (global, local, scaled)
+4. **State Management** - Update state through the appropriate methods to ensure proper notification
+5. **Command Pattern** - Use the Command pattern for operations that should be undoable
+
+## Tool-Specific Considerations
+
+### Image Tool
+
+- Handles adding images to the canvas
+- Manages image dimensions and positioning
+- Integrates with the layer system
+- Supports fixed canvas mode and flexible canvas mode
+
+### Crop Tool
+
+- Provides an interactive crop area with draggable handles
+- Uses a mask to show only the selected area
+- Handles scaling and coordinate conversions
+- Enforces constraints on the crop area
+
+### Brush Tool
+
+- Supports drawing and erasing
+- Provides customizable brush settings (size, color, opacity)
+- Uses point interpolation for smooth lines
+- Shows a preview of the brush before drawing
+
+### Resize Tool
+
+- Allows resizing the canvas
+- Preserves content when resizing
+- Handles aspect ratio constraints
+- Updates the editor state after resizing
+
+## Future Improvements
+
+Potential areas for enhancement:
+
+1. **Performance Optimization** - Further optimize rendering for large canvases
+2. **Tool Extensions** - Add support for more tools and tool options
+3. **Layer Effects** - Add support for layer effects and blending modes
+4. **Selection Tools** - Enhance selection tools and operations
+5. **Export Options** - Add more export options and formats
+
+## Conclusion
+
+The Image Editor is a powerful, extensible system for image editing. Its modular architecture, command pattern implementation, and layer management system provide a solid foundation for a wide range of image editing operations. By understanding the high-level architecture and components, developers can more easily navigate and extend the codebase. 

package/ImageEditor.stories.svelte

@@ -55,7 +55,7 @@
 	}}
 />
 
-<Story
+<!-- <Story
 	name="Image Editor Undo/Redo Interactions"
 	args={{
 		value: {
@@ -108,15 +108,15 @@
 
 		await userEvent.click(canvas.getByLabelText("Redo"));
 	}}
-/>
+/> -->
 
 <Story
 	name="Static Image Display"
 	args={{
 		value: {
 			composite: {
-				path: "",
-				url: "/output-image.png",
+				path: "https://gradio-builds.s3.amazonaws.com/demo-files/ghepardo-primo-piano.jpg",
+				url: "https://gradio-builds.s3.amazonaws.com/demo-files/ghepardo-primo-piano.jpg",
 				size: null,
 				orig_name: null,
 				mime_type: null,

package/Index.svelte

@@ -1,20 +1,19 @@
 <svelte:options accessors={true} immutable={true} />
 
 <script lang="ts">
-	import type { Brush, Eraser } from "./shared/tools/Brush.svelte";
-	import type {
-		EditorData,
-		ImageBlobs
-	} from "./shared/InteractiveImageEditor.svelte";
+	import type { Brush, Eraser } from "./shared/brush/types";
+	import type { EditorData, ImageBlobs } from "./InteractiveImageEditor.svelte";
+
+	import { FileData } from "@gradio/client";
 
 	import type { Gradio, SelectData } from "@gradio/utils";
 	import { BaseStaticImage as StaticImage } from "@gradio/image";
-	import InteractiveImageEditor from "./shared/InteractiveImageEditor.svelte";
+	import InteractiveImageEditor from "./InteractiveImageEditor.svelte";
 	import { Block } from "@gradio/atoms";
 	import { StatusTracker } from "@gradio/statustracker";
 	import type { LoadingStatus } from "@gradio/statustracker";
 	import { tick } from "svelte";
-
+	import type { LayerOptions, Transform, Source } from "./shared/types";
 	export let elem_id = "";
 	export let elem_classes: string[] = [];
 	export let visible = true;
@@ -29,7 +28,7 @@
 	export let root: string;
 	export let value_is_output = false;
 
-	export let height: number | undefined;
+	export let height = 350;
 	export let width: number | undefined;
 
 	export let _selectable = false;
@@ -38,19 +37,13 @@
 	export let min_width: number | undefined = undefined;
 	export let loading_status: LoadingStatus;
 	export let show_share_button = false;
-	export let sources: ("clipboard" | "webcam" | "upload")[] = [
-		"upload",
-		"clipboard",
-		"webcam"
-	];
+	export let sources: Source[] = [];
 	export let interactive: boolean;
 	export let placeholder: string | undefined;
-
 	export let brush: Brush;
 	export let eraser: Eraser;
-	export let crop_size: [number, number] | `${string}:${string}` | null = null;
-	export let transforms: "crop"[] = ["crop"];
-	export let layers = true;
+	export let transforms: Transform[] = [];
+	export let layers: LayerOptions;
 	export let attached_events: string[] = [];
 	export let server: {
 		accept_blobs: (a: any) => void;
@@ -59,7 +52,6 @@
 	export let fixed_canvas = false;
 	export let show_fullscreen_button = true;
 	export let full_history: any = null;
-
 	export let gradio: Gradio<{
 		change: never;
 		error: string;
@@ -73,6 +65,7 @@
 		share: ShareData;
 		clear_status: LoadingStatus;
 	}>;
+	export let border_region = 0;
 
 	let editor_instance: InteractiveImageEditor;
 	let image_id: null | string = null;
@@ -89,8 +82,7 @@
 		return blobs;
 	}
 
-	let dragging: boolean;
-
+	let is_dragging: boolean;
 	$: value && handle_change();
 	const is_browser = typeof window !== "undefined";
 	const raf = is_browser
@@ -126,32 +118,30 @@
 		}
 	}
 
-	let dynamic_height: number | undefined = undefined;
-
-	// In case no height given, pick a height large enough for the entire canvas
-	// in pixi.ts, the max-height of the canvas is canvas height / pixel ratio
-
-	let safe_height_initial = Math.max(
-		canvas_size[1] / (is_browser ? window.devicePixelRatio : 1),
-		250
-	);
-
-	$: safe_height = Math.max((dynamic_height ?? safe_height_initial) + 100, 250);
-
 	$: has_value = value?.background || value?.layers?.length || value?.composite;
+
+	$: normalised_background = value?.background
+		? new FileData(value.background)
+		: null;
+	$: normalised_composite = value?.composite
+		? new FileData(value.composite)
+		: null;
+	$: normalised_layers =
+		value?.layers?.map((layer) => new FileData(layer)) || [];
 </script>
 
 {#if !interactive}
 	<Block
 		{visible}
 		variant={"solid"}
-		border_mode={dragging ? "focus" : "base"}
+		border_mode={is_dragging ? "focus" : "base"}
 		padding={false}
 		{elem_id}
 		{elem_classes}
 		{height}
 		{width}
-		allow_overflow={false}
+		allow_overflow={true}
+		overflow_behavior="visible"
 		{container}
 		{scale}
 		{min_width}
@@ -180,13 +170,14 @@
 	<Block
 		{visible}
 		variant={has_value ? "solid" : "dashed"}
-		border_mode={dragging ? "focus" : "base"}
+		border_mode={is_dragging ? "focus" : "base"}
 		padding={false}
 		{elem_id}
 		{elem_classes}
-		height={height || safe_height}
+		{height}
 		{width}
-		allow_overflow={false}
+		allow_overflow={true}
+		overflow_behavior="visible"
 		{container}
 		{scale}
 		{min_width}
@@ -199,25 +190,25 @@
 		/>
 
 		<InteractiveImageEditor
+			{border_region}
 			on:history={(e) => (full_history = e.detail)}
-			bind:dragging
+			bind:is_dragging
 			{canvas_size}
 			on:change={() => handle_history_change()}
 			bind:image_id
-			{crop_size}
-			{value}
+			layers={normalised_layers}
+			composite={normalised_composite}
+			background={normalised_background}
 			bind:this={editor_instance}
-			bind:dynamic_height
 			{root}
 			{sources}
 			{label}
 			{show_label}
-			{height}
 			{fixed_canvas}
 			on:save={(e) => handle_save()}
 			on:edit={() => gradio.dispatch("edit")}
 			on:clear={() => gradio.dispatch("clear")}
-			on:drag={({ detail }) => (dragging = detail)}
+			on:drag={({ detail }) => (is_dragging = detail)}
 			on:upload={() => gradio.dispatch("upload")}
 			on:share={({ detail }) => gradio.dispatch("share", detail)}
 			on:error={({ detail }) => {
@@ -240,10 +231,8 @@
 			i18n={gradio.i18n}
 			{transforms}
 			accept_blobs={server.accept_blobs}
-			{layers}
-			status={loading_status?.status}
+			layer_options={layers}
 			upload={(...args) => gradio.client.upload(...args)}
-			stream_handler={(...args) => gradio.client.stream(...args)}
 			{placeholder}
 			{full_history}
 		></InteractiveImageEditor>