@needle-tools/engine 4.16.0-next.73c93c0 → 4.16.0-next.bc4d35d

package/README.md

@@ -1,63 +1,31 @@
-# Needle Engine 
+# Needle Engine
 
-**Needle Engine** is a web engine for high quality 3D applications with performance in mind.
+**Needle Engine** is a web-first 3D framework built on [three.js](https://threejs.org/) for building games, configurators, AR/VR experiences, and interactive websites.
 
-Built on three.js and the glTF standard, Needle Engine delivers flexible, extensible web experiences with built-in collaboration and XR support.
+`Built-in: Rapier Physics | WebXR (incl. iOS) | Multiplayer & VOIP | Blender & Unity Integration`
 
-## Quick Links
-🏓 [Changelog](https://github.com/needle-tools/needle-engine-support/releases) • 📑 [Documentation](https://docs.needle.tools) • 🧠 [Sample Scenes](https://samples.needle.tools) • 💎 [Showcase](https://engine.needle.tools/samples/?overlay=showcase)
-
-## Key Features
-
-**🎮 Development Experience**
-- Component system with easy custom component creation
-- Unity and Blender integrations for familiar workflows
-- Multi-scene support with dynamic content loading
-
-**🌐 Web & XR Ready**
-- WebXR support for immersive experiences on Android and iOS ([yes! WebXR on iOS!](https://engine.needle.tools/docs/how-to-guides/xr/ios-webxr-app-clip.html))
-- Interactive QuickLook support for interactive AR on Vision Pro
-- Built-in networking and collaboration
+Built on three.js and the glTF standard, Needle Engine delivers flexible, extensible web experiences with built-in collaboration and XR support. Use it standalone with npm or with powerful editor integrations for Unity and Blender.
 
-**⚡ Performance Optimized**
-- Progressive texture and mesh loading
-- Automatic LOD generation for textures and meshes
-- Advanced PBR rendering with lightmap support
-
-**🎬 Animation & Effects**
-- Animation state machines and timeline animations
-- Physics simulation
-- Post-processing effects
-- Animate anything with ease
-
-[→ See all features](https://docs.needle.tools/features)
-
-## Editor Integrations
-
-**Powerful integrations for Unity and Blender** allow artists and developers to collaborate and manage web applications inside battle-tested 3d editors. Needle Engine integrations allow you to use editor features for exporting models, author materials, animate and sequence animations, bake lightmaps and more.   
-- 🎲 [Download Unity Integration](https://engine.needle.tools/downloads/unity)
-- 🐵 [Download Blender Integration](https://engine.needle.tools/downloads/blender)
-- 📜 [Use with Code](https://engine.needle.tools/docs/three/)
+## Quick Links
+[Changelog](https://github.com/needle-tools/needle-engine-support/releases) | [Documentation](https://docs.needle.tools) | [Samples](https://samples.needle.tools) | [Showcase](https://engine.needle.tools/samples/?overlay=showcase) | [API Reference](https://engine.needle.tools/docs/api/latest)
 
 ## Getting Started
 
-**Quick Start with npm:**
 ```bash
 npm install @needle-tools/engine
 ```
 
-**Or use our Editor Integrations:** Follow the [Getting Started Guide](https://docs.needle.tools/getting-started) to download and install Needle Engine with Unity or Blender.
-
-**Explore Examples:** [Try our interactive samples](https://engine.needle.tools/samples) to see what's possible ⚡
+[Try it now on StackBlitz](https://engine.needle.tools/new) | [Getting Started Guide](https://docs.needle.tools/getting-started)
 
-## Basic Usage
+## Code Examples
 
-### Creating a Custom Component
+### Custom Component
 
 ```typescript
 import { Behaviour, serializable } from "@needle-tools/engine";
 
 export class MyComponent extends Behaviour {
+
     @serializable()
     speed: number = 1;
 
@@ -71,7 +39,42 @@ export class MyComponent extends Behaviour {
 }
 ```
 
-### Using Lifecycle Hooks (without components)
+### Physics with Rapier (built-in)
+
+```typescript
+import { Behaviour, Rigidbody, BoxCollider } from "@needle-tools/engine";
+
+export class PhysicsBox extends Behaviour {
+    
+    awake() {
+        // Add a physics body — Rapier is built in, no extra install needed
+        const rb = this.gameObject.addComponent(Rigidbody);
+        rb.useGravity = true;
+
+        // Add a collider
+        this.gameObject.addComponent(BoxCollider);
+    }
+}
+```
+
+### Multiplayer with @syncField
+
+```typescript
+import { Behaviour, syncField } from "@needle-tools/engine";
+
+export class SyncedCounter extends Behaviour {
+    // Automatically synced across all connected clients
+    @syncField()
+    count: number = 0;
+
+    onPointerClick() {
+        // Reassign to trigger sync (this is required for sync to work)
+        this.count = this.count + 1;
+    }
+}
+```
+
+### Lifecycle Hooks (no component needed)
 
 ```typescript
 import { onStart, onUpdate } from "@needle-tools/engine";
@@ -85,11 +88,54 @@ onUpdate((context) => {
 });
 ```
 
-[→ See API Documentation](https://engine.needle.tools/docs/api/latest)
+## Key Features
+
+**WebXR & AR** — immersive experiences on Android and iOS
+- WebXR support including [WebXR on iOS](https://engine.needle.tools/docs/how-to-guides/xr/ios-webxr-app-clip.html)
+- `WebXRImageTracking` — AR image targets with full tracking lifecycle
+- `WebXRPlaneTracking` — AR surface detection
+- Interactive QuickLook for AR on Vision Pro
+
+**Scene & Asset Management**
+- `SceneSwitcher` — load different scenes by URL
+- `AssetReference` — runtime asset loading by URL
+- `NestedGltf` — lazy-load GLB files on demand within a scene
+- Multi-scene support with dynamic content loading
+
+**Physics & Interaction** — Built-in [Rapier](https://rapier.rs/) physics engine
+- `Rigidbody`, `BoxCollider`, `SphereCollider`, `MeshCollider` — full physics simulation
+- `CharacterController` — movement with gravity, slopes, and steps
+- `DragControls` — click-and-drag 3D objects with zero code
+- `SpatialTrigger` — proximity and enter-zone detection
+
+**Multiplayer & Networking** — real-time collaboration out of the box
+- `SyncedRoom` + `@syncField()` — automatic state synchronization
+- `Voip` — built-in WebRTC voice chat
+- `PlayerSync` — player object sync on join/leave
+- `SyncedCamera` — camera sync for observer sessions
+
+**Rendering & Effects**
+- Advanced PBR rendering with lightmap support
+- Post-processing (Bloom, DepthOfField, SSAO, ChromaticAberration, and more)
+- Progressive texture and mesh loading with automatic LOD generation
+
+**Animation & Media**
+- Animation state machines and timeline animations
+- `VideoPlayer` — full video playback component
+- `AudioSource` — 3D spatial audio
+- Animate anything via [KHR_animation_pointer](https://github.com/nicolo-ribaudo/glTF/tree/animation-pointer/extensions/2.0/Khronos/KHR_animation_pointer)
+
+**Framework Integration** — works with React, Vue, Svelte, or vanilla JS/TS
+
+[See all features](https://docs.needle.tools/features)
+
+## Editor Integrations
 
----
+Needle Engine works standalone with just npm — no editor required. For asset-heavy workflows, use our editor integrations:
 
-*Available under commercial and educational licenses*
+- [Download Unity Integration](https://engine.needle.tools/downloads/unity)
+- [Download Blender Integration](https://engine.needle.tools/downloads/blender)
+- [Code-only workflow docs](https://engine.needle.tools/docs/three/)
 
 ## Examples
 

package/SKILL.md

@@ -0,0 +1,229 @@
+---
+name: needle-engine
+description: >
+  Provides Needle Engine context for web-based 3D projects built on Three.js
+  with the @needle-tools/engine component system. Use this skill whenever the user
+  is working with Needle Engine components, GLB files exported from Unity or Blender,
+  Vite configs with needlePlugins, TypeScript classes extending Behaviour, or anything
+  involving @needle-tools/engine imports. Also trigger when the user mentions
+  "needle engine", "needle tools", serializable decorators (@serializable, @syncField,
+  @registerType), the <needle-engine> web component, or 3D scenes loaded from GLB
+  in a web context — even if they don't explicitly name the engine.
+compatibility:
+  - optional: needle_search MCP tool (search Needle Engine docs, forum posts, and community answers)
+---
+
+# Needle Engine
+
+You are an expert in Needle Engine — a web-first 3D engine built on Three.js with a component system and Unity/Blender-based workflow.
+
+## Quick Start
+
+```html
+<needle-engine src="assets/scene.glb"></needle-engine>
+<script type="module">
+  import "@needle-tools/engine";
+</script>
+```
+
+Minimal TypeScript component:
+```ts
+import { Behaviour, serializable, registerType } from "@needle-tools/engine";
+
+@registerType
+export class HelloWorld extends Behaviour {
+  @serializable() message: string = "Hello!";
+
+  start() {
+    console.log(this.message);
+  }
+}
+```
+
+> ⚠️ **TypeScript config required:** `tsconfig.json` must have `"experimentalDecorators": true` and `"useDefineForClassFields": false` for decorators to work. Without `useDefineForClassFields: false`, TypeScript overwrites `@serializable()` properties with their default values *after* the decorator runs, silently breaking deserialization.
+
+---
+
+## Key Concepts
+
+**Needle Engine** ships 3D scenes from Unity or Blender as GLB files and renders them in the browser using Three.js. TypeScript components attached to objects are serialized into the GLB and re-hydrated at runtime.
+
+- **Unity workflow:** C# MonoBehaviours → auto-generated TypeScript stubs → GLB export on play/build
+- **Blender workflow:** Components added via the Needle Engine Blender addon → GLB export with component data embedded
+- **Embedding:** `<needle-engine src="assets/scene.glb">` web component creates and manages a 3D context
+- **Context access:** use `onStart(ctx => { ... })` or `onInitialize(ctx => { ... })` lifecycle hooks (preferred); `document.querySelector("needle-engine").context` works but only from UI event handlers
+
+### `<needle-engine>` Attributes
+
+Boolean attributes can be disabled with `="0"` (e.g. `camera-controls="0"`).
+
+```html
+<needle-engine
+  src="assets/scene.glb"
+  camera-controls
+  auto-rotate
+  autoplay
+  background-color="#222"
+  environment-image="studio"
+  contactshadows
+></needle-engine>
+```
+
+| Attribute | Description |
+|---|---|
+| `src` | GLB/glTF file path(s) — string, array, or comma-separated |
+| `camera-controls` | Adds default OrbitControls with auto-fit if no `OrbitControls`/`ICameraController` exists in the root GLB. Disable with `="0"` for fully custom camera. To tweak defaults, get `OrbitControls` from the main camera in `onStart` |
+| `auto-rotate` | Auto-rotate the camera (requires `camera-controls`) |
+| `autoplay` | Auto-play animations in the loaded scene |
+| `background-color` | Hex or RGB background color (e.g. `#ff0000`) |
+| `background-image` | Skybox URL or preset: `studio`, `blurred-skybox`, `quicklook`, `quicklook-ar` |
+| `background-blurriness` | Blur intensity for background (0–1) |
+| `environment-image` | Environment lighting image URL or preset (same presets as `background-image`) |
+| `contactshadows` | Enable contact shadows |
+| `tone-mapping` | `none`, `linear`, `neutral`, `agx` |
+| `poster` | Placeholder image URL shown while loading |
+| `loadstart` / `progress` / `loadfinished` | Callback functions for loading lifecycle |
+
+---
+
+## Unity → Needle Cheat Sheet
+
+| Unity (C#) | Needle Engine (TypeScript) |
+|---|---|
+| `MonoBehaviour` | `Behaviour` |
+| `[SerializeField]` / public field | `@serializable()` (required for all serialized fields) |
+| `Instantiate(prefab)` | `instantiate(obj)` |
+| `Destroy(obj)` | `destroy(obj)` |
+| `GetComponent<T>()` | `this.gameObject.getComponent(T)` |
+| `AddComponent<T>()` | `this.gameObject.addComponent(T)` |
+| `FindObjectOfType<T>()` | `findObjectOfType(T, ctx)` |
+| `transform.position` | `this.gameObject.worldPosition` (world) / `this.gameObject.position` (local) |
+| `transform.rotation` | `this.gameObject.worldQuaternion` (world) / `this.gameObject.quaternion` (local) |
+| `transform.localScale` | `this.gameObject.worldScale` (world) / `this.gameObject.scale` (local) |
+| `Resources.Load<T>()` | No direct equivalent — use `@serializable(AssetReference)` to assign refs in editor, then `.instantiate()` or `.asset` at runtime |
+| `StartCoroutine()` | `this.startCoroutine()` (in a component; unlike Unity, coroutines stop when the component is disabled) |
+| `Time.deltaTime` | `this.context.time.deltaTime` |
+| `Camera.main` | `this.context.mainCamera` (THREE.Camera) / `this.context.mainCameraComponent` (Needle Camera component) |
+| `Debug.Log()` | `console.log()` |
+| `OnCollisionEnter()` | `onCollisionEnter(col: Collision)` |
+| `OnTriggerEnter()` | `onTriggerEnter(col: Collision)` |
+
+---
+
+## Three.js → Needle Cheat Sheet
+
+| Three.js | Needle Engine |
+|---|---|
+| `new Mesh(geo, mat)` | Created in Unity/Blender, exported as GLB; access via `Renderer.sharedMesh` / `Renderer.sharedMaterials` |
+| `scene.add(obj)` | `this.gameObject.add(obj)` or `instantiate(prefab)` |
+| `scene.remove(obj)` | `obj.removeFromParent()` (re-parent) or `destroy(obj)` (permanent) |
+| `obj.position` | `obj.position` (local) / `obj.worldPosition` (world — Needle extension) |
+| `obj.quaternion` | `obj.quaternion` (local) / `obj.worldQuaternion` (world — Needle extension) |
+| `obj.scale` | `obj.scale` (local) / `obj.worldScale` (world — Needle extension) |
+| `obj.getWorldPosition(v)` | `obj.worldPosition` (getter, no temp vec needed) |
+| `obj.traverse(cb)` | `obj.traverse(cb)` (same — it's Three.js underneath) |
+| `obj.children` | `obj.children` (same) |
+| `obj.parent` | `obj.parent` (same) |
+| `raycaster.intersectObjects()` | `this.context.physics.raycast()` (auto BVH, faster) |
+| `renderer.setAnimationLoop(cb)` | `update() {}` in a component, or `onUpdate(cb)` hook |
+| `clock.getDelta()` | `this.context.time.deltaTime` |
+| `new GLTFLoader().load(url)` | `AssetReference.getOrCreate(base, url)` then `.instantiate()`, or `loadAsset(url)` |
+
+Needle Engine extends `Object3D` with component methods (`getComponent`, `addComponent`, `worldPosition`, `worldQuaternion`, `worldScale`, `worldForward`, `worldRight`, `worldUp`, `contains`, etc.). `this.gameObject` is the `Object3D` a component is attached to. The underlying Three.js API still works directly.
+
+---
+
+## Creating a New Project
+
+```bash
+npm create needle my-app                  # Vite (default)
+npm create needle my-app -t react         # React + Vite
+npm create needle my-app -t vue           # Vue + Vite
+npm create needle my-app -t sveltekit     # SvelteKit
+npm create needle my-app -t nextjs        # Next.js
+npm create needle my-app -t react-three-fiber  # R3F
+```
+
+---
+
+## Vite Plugin System
+
+```ts
+import { defineConfig } from "vite";
+import { needlePlugins } from "@needle-tools/engine/vite";
+
+export default defineConfig(async ({ command }) => ({
+  plugins: [
+    ...(await needlePlugins(command, {}, {})),
+  ],
+}));
+```
+
+---
+
+## Deployment
+
+- **Needle Cloud** — `npx needle-cloud deploy`
+- **Vercel / Netlify** — standard Vite web app
+- **itch.io** — for games
+- **Any static host / FTP** — `npm run build` (or `npm run build:production`) produces a standard dist folder
+
+From Unity, built-in deployment components (e.g. `DeployToNetlify`) require a PRO license. Needle Cloud deployment works with the free tier.
+
+---
+
+## Progressive Loading (`@needle-tools/gltf-progressive`)
+
+```ts
+import { useNeedleProgressive } from "@needle-tools/gltf-progressive";
+useNeedleProgressive(gltfLoader, renderer);
+gltfLoader.load(url, (gltf) => scene.add(gltf.scene));
+```
+
+In Needle Engine projects this is built in — configure via **Compression & LOD Settings** in Unity.
+
+---
+
+## Searching the Documentation
+
+Use the `needle_search` MCP tool to find relevant docs, forum posts, and community answers:
+
+```
+needle_search("how to play animation clip from code")
+needle_search("SyncedTransform multiplayer")
+needle_search("deploy to Needle Cloud CI")
+```
+
+Use this *before* guessing at API details — the docs are the source of truth.
+
+---
+
+## Common Gotchas
+
+- `@registerType` is required or the component won't be instantiated from GLB. Unity/Blender export adds this automatically via codegen; hand-written components need it explicitly.
+- GLB assets go in `assets/`, static files (fonts, images, videos) in `public/` (configurable via `needle.config.json`)
+- `useDefineForClassFields: false` must be set in `tsconfig.json` — otherwise TypeScript overwrites decorated fields with their defaults after the decorator runs, silently breaking serialization
+- `@syncField()` only triggers on reassignment — mutating an array/object in place won't sync. Do `this.arr = this.arr` to force a sync event.
+- Physics callbacks (`onCollisionEnter` etc.) require a Needle `Collider` component (BoxCollider, SphereCollider ...) on the GameObject — they won't fire on mesh-only objects
+- `removeComponent()` does NOT call `onDestroy` — any cleanup logic in `onDestroy` (event listeners, timers, allocated resources) will be skipped. Use `destroy(obj)` for full cleanup.
+- Prefer the standalone `instantiate()` and `destroy()` functions over `GameObject.instantiate()` / `GameObject.destroy()` — the standalone versions are the current API
+- `loadAsset()` returns a model wrapper (not an Object3D) — use `.scene` to get the root Object3D
+- `AssetReference.getOrCreateFromUrl()` caches by URL — loading the same URL twice returns the same Object3D. Use `.instantiate()` or `loadAsset()` with `{ context }` for multiple independent copies
+
+---
+
+## References
+
+Read these **only when needed** — don't load them all upfront:
+
+- 📖 [Full API Reference](https://raw.githubusercontent.com/needle-tools/ai/refs/heads/main/providers/claude/plugin/skills/needle-engine/references/api.md) — read when writing component code (lifecycle, decorators, context API, animation, networking, XR, physics)
+- 🔗 [Framework Integration](https://raw.githubusercontent.com/needle-tools/ai/refs/heads/main/providers/claude/plugin/skills/needle-engine/references/integration.md) — read when integrating with React, Svelte, Vue, or vanilla JS
+- 🐛 [Troubleshooting](https://raw.githubusercontent.com/needle-tools/ai/refs/heads/main/providers/claude/plugin/skills/needle-engine/references/troubleshooting.md) — read when debugging errors or unexpected behavior
+- 🧩 [Component Template](https://raw.githubusercontent.com/needle-tools/ai/refs/heads/main/providers/claude/plugin/skills/needle-engine/templates/my-component.ts) — use as a starting point for new components
+
+## Important URLs
+
+- Docs: https://engine.needle.tools/docs/
+- Samples: https://engine.needle.tools/samples/
+- GitHub: https://github.com/needle-tools/needle-engine-support
+- npm: https://www.npmjs.com/package/@needle-tools/engine