mesh-avatar-element 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Yuki Shindo
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,250 @@
-# Temporary Holding Version
+# mesh-avatar-element
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Embed an animated 2D avatar with a `<mesh-avatar>` HTML element. Uses the [mesh-avatar](https://github.com/shinshin86/mesh-avatar-studio/tree/main/packages/runtime) WebGL runtime, with no framework required.
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/shinshin86/mesh-avatar-studio/main/docs/images/face-tracking.gif" width="400" alt="A Mesh Avatar Studio avatar following a face through a webcam">
+</p>
+
+## Getting started
+
+### CDN
+
+Add these two lines to your page to show the Miko sample:
+
+```html
+<script type="module" src="https://cdn.jsdelivr.net/npm/mesh-avatar-element@0/dist/standalone.js"></script>
+<mesh-avatar></mesh-avatar>
+```
+
+Without `src`, the element loads Miko from
+`https://shinshin86.github.io/mesh-avatar-studio/miko.mavatar`.
+
+### npm
+
+```sh
+npm install mesh-avatar-element
+```
+
+Import once in your app's JavaScript:
+
+```js
+import 'mesh-avatar-element';
+```
+
+Then add `<mesh-avatar></mesh-avatar>` to your HTML.
+
+### Your own avatar
+
+Open your project in [Mesh Avatar Studio](https://github.com/shinshin86/mesh-avatar-studio) and choose **Export .mavatar**. Put the exported file on your site and set `src`:
+
+```html
+<mesh-avatar src="/avatars/character.mavatar" expression="smile"></mesh-avatar>
+```
+
+An unpacked avatar folder URL also works. Cross-origin assets need CORS headers.
+
+### Size
+
+The element defaults to `display: block; aspect-ratio: 1 / 1`. Set its width in your page's CSS; you can also override its height or aspect ratio. The background is transparent.
+
+```html
+<style>
+  mesh-avatar { width: min(100%, 320px); }
+</style>
+<mesh-avatar></mesh-avatar>
+```
+
+## Attributes
+
+| Attribute | Values | Default |
+| --- | --- | --- |
+| `src` | A `.mavatar` URL or avatar folder URL | [Miko on GitHub Pages](https://shinshin86.github.io/mesh-avatar-studio/miko.mavatar) |
+| `fit` | `contain`, `cover` | `contain` |
+| `expression` | `neutral`, `smile`, `shy`, `surprise`, `halfLidded`, `angry`, `sad`, `wink` | `neutral` |
+| `emotion` | `happy`, `sad`, `angry`, `surprised`, `relaxed`, `neutral` | Unset |
+| `paused` | Boolean attribute; presence pauses animation | Absent |
+| `no-idle` | Boolean attribute; presence disables automatic idle motion | Absent |
+| `label` | Accessible canvas label | `Avatar` |
+
+Attributes can be changed after loading. Changing `src` or `fit` destroys the current avatar and loads a new one. After `load(source)`, changing `fit` reloads that source; changing `src` switches back to the attribute URL. A failed load leaves the element empty; listen for `error` to show a fallback. Unknown expression, emotion or fit values warn once per attribute/value pair rather than throwing.
+
+## Properties, methods and events
+
+Wait for the custom element to be registered before accessing it from a separate script:
+
+```js
+await customElements.whenDefined('mesh-avatar');
+const el = document.querySelector('mesh-avatar');
+await el.ready;
+el.avatar.setExpression('smile');
+```
+
+| Property or method | Description |
+| --- | --- |
+| `avatar` | Loaded runtime avatar, or `null`. All [runtime methods](https://github.com/shinshin86/mesh-avatar-studio/tree/main/packages/runtime#drive-the-avatar) are available through `el.avatar`. |
+| `load(source)` | Loads a File, Blob, ArrayBuffer, Uint8Array, URL or URL string without changing `src`. Returns the loading promise and emits `load` or `error`. The `AvatarSource` type is re-exported from `mesh-avatar`. |
+| `ready` | Promise resolving with the avatar for the current source, rejecting on loading failure. A new load after completion creates a new promise; overlapping pending loads resolve with the latest avatar. |
+| `src`, `fit`, `expression`, `emotion`, `paused` | Properties reflecting their corresponding attributes. Use attributes for `no-idle` and `label`. |
+| `setVoiceLevel(value)` | Mouth opening from loudness, 0–1. Retains the value until loading finishes. |
+| `speak(text, options?)` | Calls `speakKana` after loading. Japanese text animates the mouth without generating audio. Options: `speed`, `loop`. |
+| `play(id)` | Plays a runtime motion after loading. Available IDs are in `el.avatar.motions`. |
+
+`speak` and `play` return promises; handle rejected promises if loading can fail.
+
+| Event | `event.detail` | When |
+| --- | --- | --- |
+| `load` | `{ avatar }` | Avatar loaded and ready |
+| `error` | `{ error }` | Loading failed |
+| `motion` | `{ id }` (string or `null`) | The runtime reports a motion change |
+
+Attach listeners directly to the element with `el.addEventListener('load', event => { /* … */ })`.
+Moving the element between parents in the same turn preserves the avatar. Removing it destroys the avatar; reconnecting it reloads the last source, including a File passed to `load()`.
+
+To display a `.mavatar` chosen with a file input, pass the File directly rather than a `blob:` URL:
+
+```html
+<input id="avatar-file" type="file" accept=".mavatar">
+<mesh-avatar></mesh-avatar>
+<script type="module">
+await customElements.whenDefined('mesh-avatar');
+const el = document.querySelector('mesh-avatar');
+document.querySelector('#avatar-file').onchange = async event => {
+  const file = event.target.files[0];
+  if (!file) return;
+  try {
+    await el.load(file);
+  } catch (error) {
+    console.error('Could not load avatar:', error);
+  }
+};
+</script>
+```
+
+## Examples
+
+- [Basic embed](https://github.com/shinshin86/mesh-avatar-studio/blob/main/packages/element/examples/basic.html): two HTML lines and a CSS width.
+- [Playground](https://github.com/shinshin86/mesh-avatar-studio/blob/main/packages/element/examples/playground.html): expressions, text and audio lip sync, microphone input and your own avatar files.
+- [Interactive demo](https://shinshin86.github.io/mesh-avatar-studio/embed/) ([日本語](https://shinshin86.github.io/mesh-avatar-studio/ja/embed/)): try the controls in your browser.
+
+Download either HTML example and open it directly, or serve it on your site. Both load the element and Miko over the internet. Microphone access depends on browser permissions and secure-context support; use HTTPS or localhost if it is unavailable when opening a file directly. Examples are in the repository and are not included in the npm package.
+
+## Lip sync
+
+Each example below assumes the CDN script and a `<mesh-avatar>` element are on your page. Run JavaScript as a module.
+
+### Audio file or TTS audio
+
+Replace `/voice.mp3` with an audio file or a URL returned by your TTS service. Cross-origin audio needs CORS headers. Click the button to start playback.
+
+```html
+<button id="play-voice">Play voice</button>
+<script type="module">
+await customElements.whenDefined('mesh-avatar');
+const el = document.querySelector('mesh-avatar');
+await el.ready;
+const audio = new Audio();
+audio.crossOrigin = 'anonymous';
+audio.src = '/voice.mp3';
+const ctx = new AudioContext();
+const analyser = ctx.createAnalyser();
+ctx.createMediaElementSource(audio).connect(analyser);
+analyser.connect(ctx.destination);
+const data = new Float32Array(analyser.fftSize);
+function frame() {
+  if (audio.paused || audio.ended) {
+    el.avatar.setSpeaking(false);
+    el.setVoiceLevel(0);
+    return;
+  }
+  analyser.getFloatTimeDomainData(data);
+  el.setVoiceLevel(Math.min(1, Math.sqrt(data.reduce((s, v) => s + v * v, 0) / data.length) * 5));
+  requestAnimationFrame(frame);
+}
+document.querySelector('#play-voice').onclick = async () => {
+  await ctx.resume();
+  await audio.play();
+  el.avatar.setSpeaking(true);
+  frame();
+};
+</script>
+```
+
+### Microphone
+
+Start microphone capture from a user action. Microphone access requires HTTPS or localhost and the user's permission. The microphone is not connected to the speakers.
+
+```html
+<button id="mic">Start microphone</button>
+<script type="module">
+await customElements.whenDefined('mesh-avatar');
+const el = document.querySelector('mesh-avatar');
+await el.ready;
+let stop;
+document.querySelector('#mic').onclick = async event => {
+  if (stop) {
+    stop();
+    stop = null;
+    event.target.textContent = 'Start microphone';
+    return;
+  }
+  const ctx = new AudioContext();
+  await ctx.resume();
+  const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
+  const analyser = ctx.createAnalyser();
+  ctx.createMediaStreamSource(stream).connect(analyser);
+  const data = new Float32Array(analyser.fftSize);
+  let frameId;
+  el.avatar.setSpeaking(true);
+  const frame = () => {
+    analyser.getFloatTimeDomainData(data);
+    el.setVoiceLevel(Math.min(1, Math.sqrt(data.reduce((s, v) => s + v * v, 0) / data.length) * 5));
+    frameId = requestAnimationFrame(frame);
+  };
+  stop = () => {
+    cancelAnimationFrame(frameId);
+    stream.getTracks().forEach(t => t.stop());
+    void ctx.close();
+    el.avatar.setSpeaking(false);
+    el.setVoiceLevel(0);
+  };
+  frame();
+  event.target.textContent = 'Stop microphone';
+};
+</script>
+```
+
+### Text only
+
+This makes mouth shapes from Japanese text; it does not play or synthesize a voice.
+
+```html
+<button id="say">Animate speech</button>
+<script type="module">
+await customElements.whenDefined('mesh-avatar');
+const el = document.querySelector('mesh-avatar');
+await el.ready;
+document.querySelector('#say').onclick = () => el.speak('こんにちは');
+</script>
+```
+
+## Using the Miko sample
+
+Without `src`, Miko loads from Mesh Avatar Studio's GitHub Pages site. To host it yourself instead, save
+[the Miko archive](https://shinshin86.github.io/mesh-avatar-studio/miko.mavatar) as part of your site and point `src` at your copy.
+
+Miko © Yuki Shindo (AITuber OnAir). Follow the [Miko character guidelines](https://miko.aituberonair.com/#terms).
+Redistribution as part of an application or website is allowed; standalone distribution, asset collections and distribution primarily intended to provide the assets are not.
+Do not imply official affiliation or endorsement by Miko or AITuber OnAir, or claim ownership of the character or exclusive usage rights. See also the [sample asset terms](https://github.com/shinshin86/mesh-avatar-studio/blob/main/samples/miko-qipao/MIKO_ASSET_TERMS.md).
+
+## Notes
+
+- Requires WebGL2 in a browser.
+- Avoid placing many avatars on one page: browsers limit simultaneous WebGL contexts.
+- Animation pauses offscreen, while the tab is hidden, or when `paused` is present.
+- With `prefers-reduced-motion: reduce`, hair sway is reduced.
+- The npm entry point can be imported during server-side rendering, but rendering requires the browser.
+
+## License
+
+MIT. Miko's artwork is subject to the character guidelines above.

package/dist/attributes.d.ts

@@ -0,0 +1,26 @@
+import { type LiveExpression } from 'mesh-avatar';
+/** Shown when the element has no `src`: the Miko sample published with the Mesh Avatar Studio site. */
+export declare const DEFAULT_SRC = "https://shinshin86.github.io/mesh-avatar-studio/miko.mavatar";
+export declare const EMOTIONS: readonly ["happy", "sad", "angry", "surprised", "relaxed", "neutral"];
+export type Emotion = typeof EMOTIONS[number];
+export type Fit = 'contain' | 'cover';
+/** Returns the expression, 'neutral' when the attribute is missing, or undefined when it is unknown. */
+export declare function parseExpression(value: string | null): LiveExpression | undefined;
+/** Returns the emotion, null when the attribute is missing, or undefined when it is unknown. */
+export declare function parseEmotion(value: string | null): Emotion | null | undefined;
+/** Returns the fit, 'contain' when the attribute is missing, or undefined when it is unknown. */
+export declare function parseFit(value: string | null): Fit | undefined;
+/** Seconds since the previous frame, limited like the runtime's own loop. */
+export declare function frameSeconds(now: number, previous: number): number;
+/**
+ * A promise for the avatar that is loading now. A load that starts before the previous one
+ * finished keeps the same pending promise, so callers waiting on it get the newest avatar.
+ */
+export declare class ReadySignal<T> {
+    #private;
+    promise: Promise<T>;
+    /** Call when a load starts. */
+    start(): void;
+    resolve(value: T): void;
+    reject(error: unknown): void;
+}

package/dist/index.d.ts

@@ -0,0 +1,48 @@
+import { type AvatarSource, type MeshAvatar } from 'mesh-avatar';
+import { type Fit } from './attributes.js';
+export { DEFAULT_SRC, EMOTIONS } from './attributes.js';
+export type { Emotion, Fit } from './attributes.js';
+export type { AvatarSource, MeshAvatar, LiveExpression } from 'mesh-avatar';
+declare const Base: typeof HTMLElement;
+/**
+ * `<mesh-avatar>`: shows a Mesh Avatar Studio avatar. Without `src` it shows the Miko sample.
+ * Animation pauses while the element is off screen, the tab is hidden or `paused` is set.
+ */
+export declare class MeshAvatarElement extends Base {
+    #private;
+    static observedAttributes: string[];
+    constructor();
+    /** The loaded runtime avatar, or null while loading. Every `mesh-avatar` method is available on it. */
+    get avatar(): MeshAvatar | null;
+    /** Resolves with the avatar once the current source has loaded. */
+    get ready(): Promise<MeshAvatar>;
+    /** Loads a file, archive or URL without changing `src`. A changed `src` replaces this source. */
+    load(source: AvatarSource): Promise<MeshAvatar>;
+    get src(): string;
+    set src(value: string);
+    get fit(): Fit;
+    set fit(value: Fit);
+    get expression(): string;
+    set expression(value: string);
+    get emotion(): string | null;
+    set emotion(value: string | null);
+    get paused(): boolean;
+    set paused(value: boolean);
+    /** Mouth opening from the voice loudness, 0 to 1. Kept until the avatar loads. */
+    setVoiceLevel(value: number): void;
+    /** Moves the mouth to Japanese text without audio. Waits for the avatar to load. */
+    speak(text: string, options?: {
+        speed?: number;
+        loop?: boolean;
+    }): Promise<void>;
+    /** Plays a motion by id (see `avatar.motions`). Waits for the avatar to load. */
+    play(id: string): Promise<void>;
+    connectedCallback(): void;
+    disconnectedCallback(): void;
+    attributeChangedCallback(name: string, old: string | null, value: string | null): void;
+}
+declare global {
+    interface HTMLElementTagNameMap {
+        'mesh-avatar': MeshAvatarElement;
+    }
+}

package/dist/index.js

@@ -0,0 +1,180 @@
+import { LIVE_EXPRESSIONS as o, loadMeshAvatar as h } from "mesh-avatar";
+const a = "https://shinshin86.github.io/mesh-avatar-studio/miko.mavatar", c = ["happy", "sad", "angry", "surprised", "relaxed", "neutral"];
+function d(i) {
+  return i === null || i === "" ? "neutral" : o.includes(i) ? i : void 0;
+}
+function l(i) {
+  return i === null || i === "" ? null : c.includes(i) ? i : void 0;
+}
+function n(i) {
+  return i === null || i === "" ? "contain" : i === "contain" || i === "cover" ? i : void 0;
+}
+function u(i, t) {
+  return Math.min(0.05, Math.max(1e-3, (i - t) / 1e3));
+}
+class f {
+  promise;
+  #s;
+  #t;
+  #e = !0;
+  /** Call when a load starts. */
+  start() {
+    this.#e && (this.#e = !1, this.promise = new Promise((t, s) => {
+      this.#s = t, this.#t = s;
+    }), this.promise.catch(() => {
+    }));
+  }
+  resolve(t) {
+    this.#e || (this.#e = !0, this.#s(t));
+  }
+  reject(t) {
+    this.#e || (this.#e = !0, this.#t(t));
+  }
+}
+const p = `
+:host { display: block; position: relative; aspect-ratio: 1 / 1; }
+:host([hidden]) { display: none; }
+canvas { display: block; width: 100%; height: 100%; }
+`, m = typeof HTMLElement > "u" ? class {
+} : HTMLElement;
+class b extends m {
+  static observedAttributes = ["src", "fit", "expression", "emotion", "paused", "no-idle", "label"];
+  #s;
+  #t = null;
+  #e = new f();
+  #o = 0;
+  #r = !1;
+  #c;
+  #n = 0;
+  #d = 0;
+  #p = !0;
+  #l = null;
+  #u = null;
+  #m = /* @__PURE__ */ new Set();
+  constructor() {
+    super();
+    const t = this.attachShadow({ mode: "open" }), s = document.createElement("style");
+    s.textContent = p, this.#s = document.createElement("canvas"), this.#s.setAttribute("role", "img"), this.#s.setAttribute("aria-label", "Avatar"), t.append(s, this.#s), this.#e.start(), this.#s.addEventListener("webglcontextlost", (e) => {
+      e.preventDefault(), this.#a();
+    }), this.#s.addEventListener("webglcontextrestored", () => this.#h());
+  }
+  /** The loaded runtime avatar, or null while loading. Every `mesh-avatar` method is available on it. */
+  get avatar() {
+    return this.#t;
+  }
+  /** Resolves with the avatar once the current source has loaded. */
+  get ready() {
+    return this.#e.promise;
+  }
+  /** Loads a file, archive or URL without changing `src`. A changed `src` replaces this source. */
+  load(t) {
+    return this.#c = t, this.#h(), this.ready;
+  }
+  get src() {
+    return this.getAttribute("src") ?? "";
+  }
+  set src(t) {
+    this.setAttribute("src", t);
+  }
+  get fit() {
+    return n(this.getAttribute("fit")) ?? "contain";
+  }
+  set fit(t) {
+    this.setAttribute("fit", t);
+  }
+  get expression() {
+    return this.getAttribute("expression") ?? "neutral";
+  }
+  set expression(t) {
+    this.setAttribute("expression", t);
+  }
+  get emotion() {
+    return this.getAttribute("emotion");
+  }
+  set emotion(t) {
+    t === null ? this.removeAttribute("emotion") : this.setAttribute("emotion", t);
+  }
+  get paused() {
+    return this.hasAttribute("paused");
+  }
+  set paused(t) {
+    this.toggleAttribute("paused", t);
+  }
+  /** Mouth opening from the voice loudness, 0 to 1. Kept until the avatar loads. */
+  setVoiceLevel(t) {
+    this.#u = t, this.#t?.setVoiceLevel(t);
+  }
+  /** Moves the mouth to Japanese text without audio. Waits for the avatar to load. */
+  async speak(t, s) {
+    (await this.ready).speakKana(t, s);
+  }
+  /** Plays a motion by id (see `avatar.motions`). Waits for the avatar to load. */
+  async play(t) {
+    (await this.ready).play(t);
+  }
+  connectedCallback() {
+    this.#l ??= new IntersectionObserver((t) => {
+      this.#p = t.at(-1)?.isIntersecting ?? !0, this.#i();
+    }), this.#l.observe(this), document.addEventListener("visibilitychange", this.#i), !this.#t && !this.#r ? this.#h() : this.#i();
+  }
+  disconnectedCallback() {
+    this.#l?.disconnect(), document.removeEventListener("visibilitychange", this.#i), this.#a(), queueMicrotask(() => {
+      this.isConnected || this.#v();
+    });
+  }
+  attributeChangedCallback(t, s, e) {
+    s !== e && (t === "src" || t === "fit" ? (t === "src" && (this.#c = void 0), t === "fit" && n(e) === void 0 && this.#f(t, e), this.isConnected && this.#h()) : t === "label" ? this.#s.setAttribute("aria-label", e || "Avatar") : t === "paused" ? this.#i() : this.#t && this.#b(this.#t, t));
+  }
+  #b(t, s) {
+    if (s === "expression") {
+      const e = d(this.getAttribute("expression"));
+      e ? t.setExpression(e) : this.#f(s, this.getAttribute(s));
+    } else if (s === "emotion") {
+      const e = l(this.getAttribute("emotion"));
+      e !== void 0 ? t.setEmotion(e) : this.#f(s, this.getAttribute(s));
+    } else s === "no-idle" && t.setAutoIdle(!this.hasAttribute("no-idle"));
+  }
+  async #h(t = this.#c ?? (this.getAttribute("src") || a)) {
+    const s = ++this.#o;
+    this.#v(!1), this.#r = !0, this.#e.start();
+    try {
+      const e = await h(this.#s, t, { manual: !0, fit: this.fit });
+      if (s !== this.#o) {
+        e.destroy();
+        return;
+      }
+      this.#r = !1, this.#t = e;
+      for (const r of ["expression", "emotion", "no-idle"])
+        (r !== "emotion" || this.hasAttribute(r)) && this.#b(e, r);
+      this.#u !== null && e.setVoiceLevel(this.#u), matchMedia("(prefers-reduced-motion: reduce)").matches && e.setSwayGain(0.5), e.onMotion((r) => this.dispatchEvent(new CustomEvent("motion", { detail: { id: r } }))), e.advance(0), this.#e.resolve(e), this.dispatchEvent(new CustomEvent("load", { detail: { avatar: e } })), this.#i();
+    } catch (e) {
+      if (s !== this.#o) return;
+      this.#r = !1, this.#e.reject(e), this.dispatchEvent(new CustomEvent("error", { detail: { error: e } }));
+    }
+  }
+  /** Destroys the avatar. Without `cancel = false` it also abandons a load in progress. */
+  #v(t = !0) {
+    t && (this.#o++, this.#r = !1), this.#a(), this.#t?.destroy(), this.#t = null;
+  }
+  #i = () => {
+    const t = this.#t && this.isConnected && this.#p && !document.hidden && !this.paused;
+    t && !this.#n ? (this.#d = performance.now(), this.#n = requestAnimationFrame(this.#g)) : t || this.#a();
+  };
+  #g = (t) => {
+    const s = u(t, this.#d);
+    this.#d = t, this.#t?.advance(s, 1 / s), this.#n = requestAnimationFrame(this.#g);
+  };
+  #a() {
+    cancelAnimationFrame(this.#n), this.#n = 0;
+  }
+  #f(t, s) {
+    const e = `${t}=${s}`;
+    this.#m.has(e) || (this.#m.add(e), console.warn(`<mesh-avatar> ignores ${t}="${s}".`));
+  }
+}
+typeof customElements < "u" && !customElements.get("mesh-avatar") && customElements.define("mesh-avatar", b);
+export {
+  a as DEFAULT_SRC,
+  c as EMOTIONS,
+  b as MeshAvatarElement
+};