@viamrobotics/motion-tools 1.21.0 → 1.23.0

package/README.md

@@ -1,119 +1,37 @@
-# motion-tools
+# Viam Visualization
 
-`motion-tools` aims to provide a visualization interface for any spatial information using Viam's APIs. This typically means motion-related monitoring, testing, and debugging.
+3D visualization interface for spatial data using Viam's APIs — frame systems, geometries, point clouds, drawings — for motion-related monitoring, testing, and debugging.
 
-## Getting started
+## Documentation
 
-To run the app, you will need to set your project up by installing the app dependencies and ensuring you can connect
-to your machines if required.
+📚 **[viamrobotics.github.io/visualization](https://viamrobotics.github.io/visualization/)** is the canonical guide. It covers:
 
-### Project setup
+- [Running locally](https://viamrobotics.github.io/visualization/guides/local-usage/) — set up the app and drive it from Go via `client/api`.
+- [Embedding `<MotionTools />`](https://viamrobotics.github.io/visualization/guides/embedding/) — drop the visualizer into your own Svelte app.
+- [Implementing WorldStateStoreService](https://viamrobotics.github.io/visualization/guides/worldstatestore/) — produce `Transform`s for a Viam WSS module with `draw`.
+- API reference for [`client/api`](https://viamrobotics.github.io/visualization/api/client-api/) and [`draw`](https://viamrobotics.github.io/visualization/api/draw/).
+- [v1 → v2 migration guide](https://viamrobotics.github.io/visualization/migration/v1-to-v2/).
+- A live [playground](https://viamrobotics.github.io/visualization/playground/) rendering a baked snapshot.
 
-The easiest way to get started is using our automated setup script:
+## Quick start
 
 ```bash
-make setup
+make setup     # one-time: install Node 22, pnpm, bun, Go, buf, project deps
+make up        # http://localhost:5173
 ```
 
-This single command will:
+For manual setup, machine configs, multiple instances, and troubleshooting see the [local-usage guide](https://viamrobotics.github.io/visualization/guides/local-usage/).
 
-1. Install **fnm** (Fast Node Manager) and **Node.js 22**
-2. Install **pnpm** package manager
-3. Install **bun** runtime
-4. Install **Go** and **buf** (for protobuf generation)
-5. Install all project dependencies
-6. Generate protobuf code
+## Contributing
 
-After setup completes, add the shell configuration it prints to your shell config file (`~/.zshrc` or `~/.bashrc`), then restart your terminal.
-
-#### Manual setup
-
-If the above does not work for you, or if you prefer to install dependencies manually:
-
-1. [Install fnm](https://github.com/Schniz/fnm#installation): `curl -fsSL https://fnm.vercel.app/install | bash`
-2. Install Node.js: `fnm install 22 && fnm use 22`
-3. [Install pnpm](https://pnpm.io/installation): `curl -fsSL https://get.pnpm.io/install.sh | sh -`
-4. [Install bun](https://bun.sh/docs/installation): `curl -fsSL https://bun.sh/install | bash`
-5. [Install Go](https://go.dev/doc/install)
-6. [Install buf](https://buf.build/docs/installation): download from GitHub releases
-7. Install Go tools: `go install google.golang.org/protobuf/cmd/protoc-gen-go@latest`
-8. Install dependencies: `pnpm install`
-9. Generate protobufs: `make proto`
-
-### Env files for machine configs
-
-To add a list of connection configs in an `.env.local` file, use the following format:
-
-```
-VITE_CONFIGS='
-{
-  "fleet-rover-01": {
-    "host": "fleet-rover-01-main.ve4ba7w5qr.viam.cloud",
-    "partId": "myPartID",
-    "apiKeyId": "myApiKeyId",
-    "apiKeyValue": "MyApiKeyValue",
-    "signalingAddress": "https://app.viam.com:443"
-  }
-}
-'
-```
-
-### Running the app locally
-
-After setup completes, you can start a local app server with:
+Run the dev server with HMR:
 
 ```bash
-make up
-```
-
-This starts the app as a static site. The build part of the process will only run if you have not built the app yet as a part of `make up`, or your build is out of date.
-
-#### Running multiple apps
-
-If you want to be able to run multiple versions of the app, you can configure how the servers run. The `make up` command can accept two options:
-
-1. `STATIC_PORT` is the port for the static file server, and defaults to `5173`
-2. `WS_PORT` is the port for the websocket server used to communicate with the draw client API
-
-> [!NOTE]
-> The `WS_PORT` is not fully configurable at the moment, so passing it will only affect where the frontend listens for the websocket server, but calls with the draw client API are currently hardcoded to point to `"http://localhost:3000/"`. If this is a feature you require, please submit a request to the viz team!
-
-To run two apps using the same web socket server, run:
-
-```
-# in one terminal
-make up
-
-# in another terminal
-make up STATIC_PORT=5174
-```
-
-The apps should be available on `http://localhost:5173/` and `http://localhost:5174/`, and calls to the draw client API should render in both.
-
-### Local development
-
-If you are contributing to `motion-tools`, you should just run the development web server with:
-
-```
 pnpm dev
 ```
 
-## Running the visualizer
-
-To visit the visualizer, go to `http://localhost:5173/`
-
-Open the machine config page (bottom right) and enter in connection details to visualize a specific machine. You can also add machine configs from an env file (see below).
-
-## Executing drawing commands
-
-The visualizer includes a golang package that allows executing commands to the visualizer.
-
-The list of available commands [can be found here](https://pkg.go.dev/github.com/viam-labs/motion-tools@v0.9.0/client/client).
+See [CLAUDE.md](CLAUDE.md) for contributor conventions.
 
 ## Programmatic camera control
 
-It is possible to programmatically move the viewer camera and even modify the camera settings during runtime.
-
-To do this, open the Javascript console while using the visualizer and call methods or set properties on the `cameraControls` object.
-
-The following APIs are available: https://github.com/yomotsu/camera-controls?tab=readme-ov-file#properties
+The visualizer exposes a `cameraControls` object on `window`. Open the browser console and call methods on it to move the camera or tweak its settings at runtime. Full API: <https://github.com/yomotsu/camera-controls#properties>.

package/dist/FrameConfigUpdater.svelte.d.ts

@@ -23,6 +23,5 @@ export declare class FrameConfigUpdater {
     setFrameParent: (entity: Entity, parentName: string) => void;
     deleteFrame: (entity: Entity) => void;
     setGeometryType: (entity: Entity, type: "none" | "box" | "sphere" | "capsule") => void;
-    private sanitizeFloatValue;
 }
 export {};

package/dist/FrameConfigUpdater.svelte.js

@@ -7,9 +7,7 @@ export class FrameConfigUpdater {
         this.removeFrame = removeFrame;
     }
     updateLocalPosition = (entity, position) => {
-        const x = this.sanitizeFloatValue(position.x);
-        const y = this.sanitizeFloatValue(position.y);
-        const z = this.sanitizeFloatValue(position.z);
+        const { x, y, z } = position;
         if (x === undefined && y === undefined && z === undefined)
             return;
         const change = {};
@@ -28,10 +26,7 @@ export class FrameConfigUpdater {
         }
     };
     updateLocalOrientation = (entity, orientation) => {
-        const oX = this.sanitizeFloatValue(orientation.oX);
-        const oY = this.sanitizeFloatValue(orientation.oY);
-        const oZ = this.sanitizeFloatValue(orientation.oZ);
-        const theta = this.sanitizeFloatValue(orientation.theta);
+        const { oX, oY, oZ, theta } = orientation;
         if (oX === undefined && oY === undefined && oZ === undefined && theta === undefined) {
             return;
         }
@@ -57,9 +52,7 @@ export class FrameConfigUpdater {
         const parent = entity.get(traits.Parent) ?? 'world';
         const pose = entity.get(traits.EditedPose);
         if (geometry?.type === 'box') {
-            const x = this.sanitizeFloatValue(geometry.x);
-            const y = this.sanitizeFloatValue(geometry.y);
-            const z = this.sanitizeFloatValue(geometry.z);
+            const { x, y, z } = geometry;
             if (x === undefined && y === undefined && z === undefined)
                 return;
             const change = {};
@@ -76,7 +69,7 @@ export class FrameConfigUpdater {
             }
         }
         else if (geometry?.type === 'sphere') {
-            const r = this.sanitizeFloatValue(geometry.r);
+            const { r } = geometry;
             if (r === undefined)
                 return;
             entity.set(traits.Sphere, { r });
@@ -86,8 +79,7 @@ export class FrameConfigUpdater {
             }
         }
         else if (geometry?.type === 'capsule') {
-            const r = this.sanitizeFloatValue(geometry.r);
-            const l = this.sanitizeFloatValue(geometry.l);
+            const { r, l } = geometry;
             if (r === undefined && l === undefined)
                 return;
             const change = {};
@@ -95,7 +87,7 @@ export class FrameConfigUpdater {
                 change.r = r;
             if (l !== undefined)
                 change.l = l;
-            entity.set(traits.Capsule, { r, l });
+            entity.set(traits.Capsule, change);
             const capsule = entity.get(traits.Capsule);
             if (name && capsule && pose) {
                 this.updateFrame(name, parent, pose, { type: 'capsule', ...capsule });
@@ -134,14 +126,4 @@ export class FrameConfigUpdater {
             this.updateFrame(name, parent, pose, { type: 'capsule', r: 20, l: 100 });
         }
     };
-    sanitizeFloatValue = (value) => {
-        if (value === undefined) {
-            return undefined;
-        }
-        const num = Number.parseFloat(value.toFixed(2));
-        if (Number.isNaN(num)) {
-            return undefined;
-        }
-        return value;
-    };
 }

package/dist/components/App.svelte

@@ -47,7 +47,7 @@
 
 	interface Props {
 		partID?: string
-		enableKeybindings?: boolean
+		inputBindingsEnabled?: boolean
 		localConfigProps?: LocalConfigProps
 		drawConnectionConfig?: DrawConnectionConfig
 
@@ -74,7 +74,7 @@
 
 	let {
 		partID = '',
-		enableKeybindings = true,
+		inputBindingsEnabled = true,
 		localConfigProps,
 		cameraPose,
 		drawConnectionConfig,
@@ -91,7 +91,7 @@
 	const { isPresenting } = useXR()
 
 	$effect(() => {
-		settings.current.enableKeybindings = enableKeybindings
+		environment.current.inputBindingsEnabled = inputBindingsEnabled
 	})
 
 	createPartIDContext(() => partID)

package/dist/components/App.svelte.d.ts

@@ -11,7 +11,7 @@ interface LocalConfigProps {
 }
 interface Props {
     partID?: string;
-    enableKeybindings?: boolean;
+    inputBindingsEnabled?: boolean;
     localConfigProps?: LocalConfigProps;
     drawConnectionConfig?: DrawConnectionConfig;
     /**

package/dist/components/CameraControls.svelte

@@ -4,15 +4,15 @@
 
 	import Button from './overlay/dashboard/Button.svelte'
 	import { useCameraControls, useTransformControls } from '../hooks/useControls.svelte'
-	import { useSettings } from '../hooks/useSettings.svelte'
+	import { useEnvironment } from '../hooks/useEnvironment.svelte'
 
-	import KeyboardControls from './KeyboardControls.svelte'
+	import InputBindings from './InputBindings.svelte'
 
 	const cameraControls = useCameraControls()
-	const settings = useSettings()
+	const environment = useEnvironment()
 	const transformControls = useTransformControls()
 
-	const enableKeybindings = $derived(settings.current.enableKeybindings)
+	const inputBindingsEnabled = $derived(environment.current.inputBindingsEnabled)
 </script>
 
 <Portal id="dashboard">
@@ -37,8 +37,8 @@
 	}}
 >
 	{#snippet children({ ref }: { ref: CameraControlsRef })}
-		{#if enableKeybindings}
-			<KeyboardControls cameraControls={ref} />
+		{#if inputBindingsEnabled}
+			<InputBindings cameraControls={ref} />
 		{/if}
 		<Gizmo placement="bottom-right" />
 	{/snippet}

package/dist/components/Entities/Pose.svelte

@@ -6,7 +6,7 @@
 	import { traits, useTrait } from '../../ecs'
 	import { usePartConfig } from '../../hooks/usePartConfig.svelte'
 	import { usePose } from '../../hooks/usePose.svelte'
-	import { matrixToPose, poseToMatrix } from '../../transform'
+	import { composeRenderedPose } from '../../transform'
 
 	interface Props {
 		entity: Entity
@@ -25,22 +25,27 @@
 		() => parent.current
 	)
 
-	const resolvedPose = $derived.by(() => {
-		if (pose.current === undefined || partConfig.hasPendingSave) {
-			return editedPose.current
-		}
+	$effect.pre(() => {
+		if (pose.current === undefined) return
 
-		if (!entityPose.current || !editedPose.current) {
-			return
+		if (entity.has(traits.LivePose)) {
+			entity.set(traits.LivePose, pose.current)
+		} else {
+			entity.add(traits.LivePose(pose.current))
 		}
+	})
 
-		const poseNetwork = poseToMatrix(entityPose.current)
-		const poseUsePose = poseToMatrix(pose.current)
-		const poseLocalEditedPose = poseToMatrix(editedPose.current)
+	// Always render through the live blend: live × network⁻¹ × edited. With
+	// `edited === network` (no edits) this collapses to `live`, so the rendered
+	// pose tracks the robot's kinematics-resolved position. With edits, the
+	// formula composes the staged delta on top of live. Input handlers that
+	// drive edits (gizmo onChange, Details panel) compute `edited` such that
+	// the blend renders to the user's intent.
+	const resolvedPose = $derived.by(() => {
+		if (pose.current === undefined || partConfig.hasPendingSave) return editedPose.current
+		if (!entityPose.current || !editedPose.current) return undefined
 
-		const poseNetworkInverse = poseNetwork.invert()
-		const resultMatrix = poseUsePose.multiply(poseNetworkInverse).multiply(poseLocalEditedPose)
-		return matrixToPose(resultMatrix)
+		return composeRenderedPose(pose.current, entityPose.current, editedPose.current)
 	})
 </script>
 

package/dist/components/FileDrop/useFileDrop.svelte.js

@@ -2,6 +2,9 @@ import { Extensions, parseFileName, Prefixes, readFile } from './file-names';
 import { pcdDropper } from './pcd-dropper';
 import { plyDropper } from './ply-dropper';
 import { snapshotDropper } from './snapshot-dropper';
+const hasDraggedFiles = (dataTransfer) => {
+    return dataTransfer?.types?.includes('Files') ?? false;
+};
 const createFileDropper = (extension, prefix) => {
     switch (prefix) {
         case Prefixes.Snapshot: {
@@ -22,11 +25,15 @@ export const useFileDrop = (onSuccess, onError) => {
     let dropState = $state('inactive');
     // prevent default to allow drop
     const ondragenter = (event) => {
+        if (!hasDraggedFiles(event.dataTransfer))
+            return;
         event.preventDefault();
         dropState = 'hovering';
     };
     // prevent default to allow drop
     const ondragover = (event) => {
+        if (!hasDraggedFiles(event.dataTransfer))
+            return;
         event.preventDefault();
     };
     const ondragleave = (event) => {
@@ -40,10 +47,17 @@ export const useFileDrop = (onSuccess, onError) => {
         dropState = 'inactive';
     };
     const ondrop = (event) => {
+        const { dataTransfer } = event;
+        if (dataTransfer === null || !hasDraggedFiles(dataTransfer)) {
+            dropState = 'inactive';
+            return;
+        }
         event.preventDefault();
-        if (event.dataTransfer === null)
+        const { files } = dataTransfer;
+        if (files.length === 0) {
+            dropState = 'inactive';
             return;
-        const { files } = event.dataTransfer;
+        }
         let completed = 0;
         for (const file of files) {
             const fileName = parseFileName(file.name);

package/dist/components/{KeyboardControls.svelte → InputBindings.svelte}

@@ -2,6 +2,7 @@
 	import type { CameraControlsRef } from '@threlte/extras'
 
 	import { isInstanceOf, useTask } from '@threlte/core'
+	import { useGamepad, useInputMap, useKeyboard } from '@threlte/extras'
 	import { PressedKeys } from 'runed'
 	import { MathUtils, Vector3 } from 'three'
 
@@ -22,19 +23,33 @@
 
 	const settings = useSettings()
 
-	const keys = new PressedKeys()
-	const meta = $derived(keys.has('meta'))
-	const w = $derived(keys.has('w'))
-	const s = $derived(keys.has('s'))
-	const a = $derived(keys.has('a'))
-	const d = $derived(keys.has('d'))
-	const r = $derived(keys.has('r'))
-	const f = $derived(keys.has('f'))
-	const up = $derived(keys.has('arrowup'))
-	const left = $derived(keys.has('arrowleft'))
-	const down = $derived(keys.has('arrowdown'))
-	const right = $derived(keys.has('arrowright'))
-	const anyKeysPressed = $derived(w || s || a || d || r || f || up || left || down || right)
+	const keyboard = useKeyboard()
+	const gamepad = useGamepad()
+	const input = useInputMap(
+		({ key, gamepadAxis, gamepadButton }) => ({
+			truckLeft: [key('a'), gamepadAxis('leftStick', 'x', -1)],
+			truckRight: [key('d'), gamepadAxis('leftStick', 'x', 1)],
+			forward: [key('w'), gamepadAxis('leftStick', 'y', -1)],
+			backward: [key('s'), gamepadAxis('leftStick', 'y', 1)],
+			dollyIn: [key('r'), gamepadButton('rightBumper')],
+			dollyOut: [key('f'), gamepadButton('leftBumper')],
+			rotateLeft: [key('arrowleft'), gamepadAxis('rightStick', 'x', -1)],
+			rotateRight: [key('arrowright'), gamepadAxis('rightStick', 'x', 1)],
+			tiltUp: [key('arrowup'), gamepadAxis('rightStick', 'y', 1)],
+			tiltDown: [key('arrowdown'), gamepadAxis('rightStick', 'y', -1)],
+		}),
+		{ keyboard, gamepad }
+	)
+
+	const truckAxis = $derived(input.axis('truckLeft', 'truckRight'))
+	const forwardAxis = $derived(input.axis('backward', 'forward'))
+	const dollyAxis = $derived(input.axis('dollyOut', 'dollyIn'))
+	const yawAxis = $derived(input.axis('rotateLeft', 'rotateRight'))
+	const pitchAxis = $derived(input.axis('tiltUp', 'tiltDown'))
+
+	const anyKeysPressed = $derived(
+		truckAxis !== 0 || forwardAxis !== 0 || dollyAxis !== 0 || yawAxis !== 0 || pitchAxis !== 0
+	)
 
 	const target = new Vector3()
 
@@ -70,7 +85,7 @@
 			const dt = delta * 1000
 
 			// Disallow keyboard navigation if the user is holding down the meta key
-			if (meta) {
+			if (keyboard.key('meta').pressed) {
 				return
 			}
 
@@ -80,64 +95,41 @@
 			const dollySpeed = 0.005 * dt
 			const zoomSpeed = 0.5 * dt
 
-			if (a) {
-				cameraControls.truck(-moveSpeed * dt, 0, true)
-			}
-
-			if (d) {
-				cameraControls.truck(moveSpeed * dt, 0, true)
-			}
-
-			if (w) {
-				cameraControls.forward(moveSpeed * dt, true)
-			}
-
-			if (s) {
-				cameraControls.forward(-moveSpeed * dt, true)
+			if (truckAxis !== 0) {
+				cameraControls.truck(truckAxis * moveSpeed * dt, 0, true)
 			}
 
-			if (r) {
-				if (isInstanceOf(cameraControls.camera, 'PerspectiveCamera')) {
-					cameraControls.dolly(dollySpeed, true)
-				} else {
-					cameraControls.zoom(zoomSpeed, true)
-				}
+			if (forwardAxis !== 0) {
+				cameraControls.forward(forwardAxis * moveSpeed * dt, true)
 			}
 
-			if (f) {
+			if (dollyAxis !== 0) {
 				if (isInstanceOf(cameraControls.camera, 'PerspectiveCamera')) {
-					cameraControls.dolly(-dollySpeed, true)
+					cameraControls.dolly(dollyAxis * dollySpeed, true)
 				} else {
-					cameraControls.zoom(-zoomSpeed, true)
+					cameraControls.zoom(dollyAxis * zoomSpeed, true)
 				}
 			}
 
-			if (left) {
-				cameraControls.rotate(-rotateSpeed, 0, true)
-			}
-
-			if (right) {
-				cameraControls.rotate(rotateSpeed, 0, true)
+			if (yawAxis !== 0) {
+				cameraControls.rotate(yawAxis * rotateSpeed, 0, true)
 			}
 
-			if (up) {
-				cameraControls.rotate(0, -tiltSpeed, true)
-			}
-
-			if (down) {
-				cameraControls.rotate(0, tiltSpeed, true)
+			if (pitchAxis !== 0) {
+				cameraControls.rotate(0, pitchAxis * tiltSpeed, true)
 			}
 		},
 		{
+			after: input.task,
 			running: () => anyKeysPressed,
 			autoInvalidate: false,
 		}
 	)
 
+	const keys = new PressedKeys()
+
 	keys.onKeys('escape', () => {
-		if (keys.has('escape')) {
-			focusedEntity.set()
-		}
+		focusedEntity.set()
 	})
 
 	keys.onKeys('c', () => {
@@ -157,30 +149,11 @@
 		settings.current.transformMode = 'scale'
 	})
 
-	keys.onKeys('x', () => {
-		settings.current.enableXR = !settings.current.enableXR
-	})
-
-	/**
-	 * Handler for any keybindings that need to access the event object
-	 */
-	const onkeydown = (event: KeyboardEvent) => {
-		const key = event.key.toLowerCase()
-
-		if (key === 'h') {
-			if (!entity) return
-
-			event.stopImmediatePropagation()
-
-			if (entity.has(traits.Invisible)) {
-				entity.remove(traits.Invisible)
-			} else {
-				entity.add(traits.Invisible)
-			}
-
-			return
+	keys.onKeys('h', () => {
+		if (entity?.has(traits.Invisible)) {
+			entity.remove(traits.Invisible)
+		} else {
+			entity?.add(traits.Invisible)
 		}
-	}
+	})
 </script>
-
-<svelte:window {onkeydown} />

package/dist/components/InputBindings.svelte.d.ts

@@ -0,0 +1,7 @@
+import type { CameraControlsRef } from '@threlte/extras';
+interface Props {
+    cameraControls: CameraControlsRef;
+}
+declare const InputBindings: import("svelte").Component<Props, {}, "">;
+type InputBindings = ReturnType<typeof InputBindings>;
+export default InputBindings;

package/dist/components/PointerMissBox.svelte

@@ -23,7 +23,7 @@
 	onpointerdown={() => {
 		cameraDown.copy(camera.current.position)
 	}}
-	onpointerup={() => {
+	onclick={() => {
 		if (transformControls.active) {
 			return
 		}

package/dist/components/Scene.svelte

@@ -10,6 +10,7 @@
 	import Entities from './Entities/Entities.svelte'
 	import Focus from './Focus.svelte'
 	import Selected from './Selected.svelte'
+	import SelectedTransformControls from './SelectedTransformControls.svelte'
 	import StaticGeometries from './StaticGeometries.svelte'
 	import { useFocusedObject3d } from '../hooks/useSelection.svelte'
 	import { useSettings } from '../hooks/useSettings.svelte'
@@ -77,6 +78,7 @@
 
 		<StaticGeometries />
 		<Selected />
+		<SelectedTransformControls />
 
 		{#if !$isPresenting && settings.current.grid}
 			<Grid

package/dist/components/SceneProviders.svelte

@@ -12,6 +12,7 @@
 	} from '../hooks/useControls.svelte'
 	import { provideDrawAPI } from '../hooks/useDrawAPI.svelte'
 	import { provideDrawService } from '../hooks/useDrawService.svelte'
+	import { provideFrameEditSession } from '../hooks/useFrameEditSession.svelte'
 	import { provideFramelessComponents } from '../hooks/useFramelessComponents.svelte'
 	import { provideFrames } from '../hooks/useFrames.svelte'
 	import { provideGeometries } from '../hooks/useGeometries.svelte'
@@ -47,6 +48,7 @@
 
 	provideResourceByName(() => partID.current)
 	provideConfigFrames()
+	provideFrameEditSession(() => partID.current)
 	provideFrames(() => partID.current)
 	provideGeometries(() => partID.current)
 	provide3DModels(() => partID.current)