@naruya/gaussian-vrm 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2025 naruya
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,139 @@
1
+ # Instant Skinned Gaussian Avatars
2
+
3
+ [![Project Page](https://img.shields.io/badge/Project%20Page-Open-1f6feb?style=flat-square&logo=homepage)](https://gaussian-vrm.github.io/)
4
+ [![arXiv](https://img.shields.io/badge/arXiv-2501.12345-b31b1b?style=flat-square&logo=arxiv&logoColor=white)](https://arxiv.org/abs/2510.13978)
5
+ [![Demo](https://img.shields.io/badge/Demo-Live-00b140?style=flat-square&logo=googlechrome&logoColor=white)](http://naruya.github.io/gaussian-vrm)
6
+
7
+ <img src="https://i.gyazo.com/5f7bcfed3de6e51f98e84278f18c1897.gif" width=80%>
8
+
9
+ This is the official repository for **Instant Skinned Gaussian Avatars for Web, Mobile and VR Applications** (SUI 2025 Demo Track).<br>
10
+
11
+ Try our online demo at https://naruya.github.io/gaussian-vrm/ <br>
12
+ For more details, check out our project page at https://gaussian-vrm.github.io/
13
+
14
+ 🎉 We’ve released **sample avatars and original scan data**!
15
+ Check out the [Sample Avatars](#sample-avatars) section below for details.
16
+
17
+ # Gaussian-VRM
18
+
19
+ **Gaussian-VRM (GVRM)** is a three.js implementation of **Instant Skinned Gaussian Avatars**. GVRM is built on top of [three-vrm](https://github.com/pixiv/three-vrm) and [gaussian-splats-3d](https://github.com/mkkellogg/GaussianSplats3D). GVRM can handle avatars as standard three.js objects, allowing you to directly reuse VRM format avatar operations (such as movement and animations).
20
+
21
+ **For detailed usage instructions, please check 👉 [Gaussian-VRM Examples](https://github.com/naruya/gaussian-vrm/tree/main/examples) 👈**
22
+
23
+ ## Quick Example
24
+
25
+ ```javascript
26
+ // GVRM
27
+ const gvrm = await GVRM.load('./assets/author.gvrm', scene, camera, renderer); // 1/3
28
+ await gvrm.changeFBX('./assets/Idle.fbx'); // 2/3
29
+
30
+ renderer.setAnimationLoop(() => {
31
+ gvrm.update(); // 3/3
32
+ renderer.render(scene, camera);
33
+ });
34
+ ```
35
+
36
+ The three steps are:
37
+
38
+ 1. Load GVRM file
39
+ 2. Change animation
40
+ 3. Update every frame
41
+
42
+ That's all! (Super easy!) The full JavaScript code can be seen below:
43
+
44
+ ```javascript
45
+ import * as THREE from 'three';
46
+ import { GVRM } from 'gvrm';
47
+
48
+ const canvas = document.getElementById('canvas');
49
+ const renderer = new THREE.WebGLRenderer({ canvas });
50
+ renderer.setSize(640, 480);
51
+
52
+ const scene = new THREE.Scene();
53
+
54
+ const camera = new THREE.PerspectiveCamera(65, 640 / 480, 0.01, 100);
55
+ camera.position.set(0, 0.4, 1.5);
56
+
57
+ // GVRM
58
+ const gvrm = await GVRM.load('./assets/author.gvrm', scene, camera, renderer); // 1/3
59
+ await gvrm.changeFBX('./assets/Idle.fbx'); // 2/3
60
+
61
+ renderer.setAnimationLoop(() => {
62
+ gvrm.update(); // 3/3
63
+ renderer.render(scene, camera);
64
+ });
65
+ ```
66
+
67
+
68
+ **For more usage, please check 👉 [Gaussian-VRM Examples](https://github.com/naruya/gaussian-vrm/tree/main/examples) 👈**
69
+
70
+
71
+ ## Advanced Examples
72
+
73
+ ### Embed in Your Website
74
+
75
+ <img src="https://i.gyazo.com/250c99bd3bed7f30e3ca2ab064da88cc.png" width="400px">
76
+
77
+ ### Playable Avatar
78
+
79
+ <img src="https://i.gyazo.com/ec1ba83eb39d6ecf08605feaa880c1fd.png" width="400px">
80
+
81
+ ### Webcam control / Splat Effects
82
+
83
+ (coming soon!)
84
+
85
+ ## Sample Avatars
86
+
87
+ Six sample avatars and original scan data are available on Google Drive:
88
+ 🔗 [Sample Avatars (Google Drive)](https://drive.google.com/drive/folders/1R9QXjUiDZf0vo7E4GnmnvyB_KB9-N2F8?usp=drive_link)
89
+ These avatars are released under the **MIT License**,
90
+ **as long as they are not used in ways that violate public order or morality.**
91
+
92
+ <img width="300" height="876" alt="samples" src="https://i.gyazo.com/9df98a958283f7d5bcc539d61e6dfab4.png" />
93
+
94
+ ## Extra Animation Files
95
+
96
+ If you would like to use animation files with the sample avatars, the easiest way is to download them from [Mixamo](https://www.mixamo.com/).
97
+
98
+ **Recommended download settings:**
99
+ - **Format:** FBX ASCII (.fbx)
100
+ - **Skin:** Without Skin
101
+ - **Frames per Second:** 60
102
+ - **Keyframe Reduction:** None
103
+
104
+ ## License
105
+
106
+ - **Source Code**
107
+ This repository's source code is licensed under the [MIT License](./LICENSE).
108
+ 👉 Unlike other related research, this work does **not** use SMPL, any deep learning models,
109
+ or mesh optimizers with restrictive licenses — therefore, it can be released under the MIT License! 🎉🎉
110
+
111
+ - **Assets** (`./assets` and `./examples/assets/` directories)
112
+ The files under these directories are **not covered by the MIT License**.
113
+ They are provided solely for research purposes and **may not be used, modified, or redistributed**
114
+ without explicit permission.
115
+
116
+ Certain sample avatars are separately provided under the MIT License
117
+ — see the [Sample Avatars](#sample-avatars) section above for details.
118
+
119
+
120
+ ## Acknowledgements
121
+
122
+ This work was supported by the Ochiai Pavilion at the Osaka/Kansai Expo 2025.<br>
123
+ This work was supported by JSPS KAKENHI Grant Number [23KJ0284](https://kaken.nii.ac.jp/ja/grant/KAKENHI-PROJECT-23KJ0284/).
124
+
125
+ The VRM model in this repository is freely usable for any purpose, except standalone redistribution of the original, unmodified model. The model can be found at [JOKER's store](https://jokerconentsshop.booth.pm/). Thanks!
126
+
127
+ ## Bibtex
128
+
129
+ ```bibtex
130
+ @misc{kondo2025instantskinnedgaussianavatars,
131
+ title={Instant Skinned Gaussian Avatars for Web, Mobile and VR Applications},
132
+ author={Naruya Kondo and Yuto Asano and Yoichi Ochiai},
133
+ year={2025},
134
+ eprint={2510.13978},
135
+ archivePrefix={arXiv},
136
+ primaryClass={cs.CG},
137
+ url={https://arxiv.org/abs/2510.13978},
138
+ }
139
+ ```
@@ -0,0 +1,161 @@
1
+ var ae=Object.defineProperty;var le=(a,e)=>{for(var t in e)ae(a,t,{get:e[t],enumerable:!0})};import*as f from"three";var Q={};le(Q,{BONE_CONFIG:()=>$,addChannels:()=>C,addPMC:()=>J,applyBoneOperations:()=>X,colors:()=>j,createDataTexture:()=>P,getPointsMeshCapsules:()=>O,removePMC:()=>N,resetPose:()=>ce,setPose:()=>F,simpleAnim:()=>de,visualizeBoneAxes:()=>me,visualizePMC:()=>G,visualizeVRM:()=>W});import*as x from"three";var j=[[255,222,62],[138,119,199],[243,82,82],[16,157,123],[43,247,242],[120,84,254],[157,238,149],[80,105,17],[39,121,232],[88,149,76],[22,60,188],[243,67,171],[94,62,2],[162,192,4]];function X(a,e){for(let t of e){let i=t.boneName,s=a.humanoid.getRawBoneNode(i),n=a.humanoid.getNormalizedBoneNode(i);t.position&&(s.position.x+=t.position.x,s.position.y+=t.position.y,s.position.z+=t.position.z),t.rotation&&(n.rotation.x=t.rotation.x*Math.PI/180,n.rotation.y=t.rotation.y*Math.PI/180,n.rotation.z=t.rotation.z*Math.PI/180),t.scale&&s.scale.set(t.scale.x,t.scale.y,t.scale.z)}}function F(a,e){X(a.currentVrm,e),a.currentVrm.humanoid.update()}function ce(a,e){a.currentVrm.humanoid.resetRawPose(),a.currentVrm.humanoid.resetNormalizedPose(),F(a,e)}function W(a,e){let t=a.currentVrm.scene.children[a.skinnedMeshIndex],i=a.faceIndex?a.currentVrm.scene.children[a.faceIndex]:null;e===null?(t.material.forEach(s=>{s.colorWrite=!s.colorWrite,s.depthWrite=!s.depthWrite}),i&&(i.visible=!i.visible)):(t.material.forEach(s=>{s.colorWrite=e,s.depthWrite=e}),i&&(i.visible=e))}function G(a,e){let{points:t,mesh:i,capsules:s}=a;e===null?(t.visible=!t.visible,i.visible=!i.visible,s.children.forEach(n=>{n.visible=!n.visible})):(t.visible=e,i.visible=e,s.children.forEach(n=>{n.visible=e}))}function me(a,e){if(!a||!a.debugAxes||a.debugAxes.size===0)return;let t=a.debugAxes.values().next().value.visible;a.debugAxes.forEach(i=>{i.visible=!t})}function N(a,e){let{points:t,mesh:i,capsules:s}=e;t&&(a.remove(t),t.geometry.dispose(),t.material.dispose()),i&&(a.remove(i),i.geometry.dispose(),i.material.dispose()),s&&(a.remove(s),s.children.forEach(n=>{n.geometry.dispose(),n.material.dispose()}))}function J(a,e){let{points:t,mesh:i,capsules:s}=e;t&&a.add(t),i&&a.add(i),s&&a.add(s)}function C(a,e,t,i=1){for(let s=0;s<t;s++)e[s*4+0]=i>3?1:a[s*(4-i)+0],e[s*4+1]=i>2?1:a[s*(4-i)+1],e[s*4+2]=i>1?1:a[s*(4-i)+2],e[s*4+3]=i>0?1:a[s*(4-i)+3]}function P(...a){let e=new x.DataTexture(...a);return e.needsUpdate=!0,e}function de(a,e){let t=Math.PI*65/180*Math.sin(Math.PI*(e/60+.5)),i=.4*Math.PI*Math.sin(Math.PI*(e/60));a.currentVrm.humanoid.getNormalizedBoneNode("leftUpperArm").rotation.z=t,a.currentVrm.humanoid.getNormalizedBoneNode("leftUpperLeg").rotation.x=i,a.currentVrm.humanoid.getNormalizedBoneNode("leftLowerLeg").rotation.x=-Math.max(i,0),a.currentVrm.humanoid.getNormalizedBoneNode("rightLowerLeg").rotation.y=i}var $={arm:{names:["J_Bip_L_Hand","J_Bip_L_LowerArm","J_Bip_R_Hand","J_Bip_R_LowerArm"],radius:.06,scale:{x:1,z:1}},leg:{names:["J_Bip_L_LowerLeg","J_Bip_L_Foot","J_Bip_R_LowerLeg","J_Bip_R_Foot"],radius:.08,scale:{x:1,z:1}},torso:{names:["J_Bip_C_Neck","J_Bip_C_Spine","J_Bip_C_Chest","J_Bip_C_UpperChest"],radius:.03,scale:{x:6,z:4}},headTop:{names:["J_Bip_C_HeadTop_End"],radius:.06,scale:{x:1.5,z:2}},head:{names:["J_Bip_C_Head"],radius:.03,scale:{x:2,z:2}}};function O(a){let e=a.currentVrm.scene.children[a.skinnedMeshIndex],t=new x.PointsMaterial({color:16711680,size:.02,opacity:.3,transparent:!0}),i=new x.MeshBasicMaterial({color:65280,wireframe:!0,opacity:.2,transparent:!0}),s=new x.MeshBasicMaterial({wireframe:!0,opacity:.5,transparent:!0}),n=new x.BufferGeometry,l=[],o=e.geometry.getAttribute("position"),m=new x.Vector3,d=new x.Vector3;for(let h=0;h<o.count;h++)m.fromBufferAttribute(o,h),d=e.applyBoneTransform(h,m),d.applyMatrix4(a.currentVrm.scene.matrixWorld),l.push(d.x,d.y,d.z);n.setAttribute("position",new x.Float32BufferAttribute(l,3));let y=new x.Points(n,t),g=new x.BufferGeometry;g.setAttribute("position",new x.Float32BufferAttribute(l,3));let M=e.geometry.getIndex();g.setIndex(M);let p=new x.Mesh(g,i),R=new x.Group,T=[],E=0;function B(h,w=0){E++;let r=new x.Vector3().setFromMatrixPosition(h.matrixWorld);h.children.forEach(function(V){if(V.isBone){let I=new x.Vector3().setFromMatrixPosition(V.matrixWorld),b=r.distanceTo(I),L=new x.Vector3().addVectors(r,I).multiplyScalar(.5),H=null;for(let U of Object.values($))if(U.names.includes(V.name)){H=U;break}if(H){let[U,ee,te]=j[R.children.length],ie=U<<16|ee<<8|te,ne=new x.CapsuleGeometry(H.radius,b-H.radius*2,1,6),z=new x.Mesh(ne,s.clone());z.material.color.setHex(ie),z.scale.set(H.scale.x,1,H.scale.z),z.position.copy(L);let se=new x.Vector3().subVectors(I,r).normalize(),re=new x.Quaternion().setFromUnitVectors(new x.Vector3(0,1,0),se);z.setRotationFromQuaternion(re),z.updateMatrixWorld();let oe=e.skeleton.bones.indexOf(V);R.add(z),T.push(oe)}B(V,w+1)}})}let _=a.currentVrm.scene.children[0].children[0];return B(_,1),{pmc:{points:y,mesh:p,capsules:R},capsuleBoneIndex:T}}import*as v from"three";import{GLTFLoader as pe}from"three/addons/loaders/GLTFLoader.js";import{VRMLoaderPlugin as ue,VRMUtils as S}from"@pixiv/three-vrm";import{FBXLoader as he}from"three/addons/loaders/FBXLoader.js";var A=class{constructor(e,t=null,i=null,s=1,n=!1){this.modelUrl=t,this.animationUrl=i,this.currentVrm=void 0,this.currentMixer=void 0,this.currentAction=void 0,this.previousAction=null,this.transitionDuration=.5,this.scene=e,this.scale=s,this.center=n,this._isLoading=!0,this.clock=new v.Clock,this.loadVRM(this.modelUrl,this.animationUrl),this.place()}loadVRM(e=null,t=null,i=null){this._isLoading=!0,e&&(this.modelUrl=e),t&&(this.animationUrl=t),i&&(this.scale=i);let s=new pe;s.crossOrigin="anonymous";let n=new v.Group;n.renderOrder=1e4,n.clear(),s.register(l=>new ue(l,{helperRoot:n,autoUpdateHumanBones:!0})),this.loadingPromise=new Promise((l,c)=>{s.load(this.modelUrl,async o=>{let m=o.userData.vrm;S.removeUnnecessaryVertices(o.scene),S.removeUnnecessaryJoints(o.scene),this.currentVrm=m,m.scene.traverse(g=>{g.frustumCulled=!1});let d=new v.Vector3;new v.Box3().setFromObject(m.scene).getSize(d),this.ground=-d.y*.5*this.scale,this.animationUrl&&this.animationUrl!==""&&await this.loadFBX(this.animationUrl),m.scene.position.y+=this.ground,m.scene.scale.setScalar(this.scale);for(let g of m.springBoneManager.joints)g.settings.stiffness*=this.scale,g.settings.hitRadius*=this.scale;for(let g of m.springBoneManager.colliders){let M=g.shape;M.radius*=this.scale,M.tail&&M.tail.multiplyScalar(this.scale)}S.rotateVRM0(m),m.scene.updateMatrix(),m.scene.position0=m.scene.position.clone(),m.scene.rotation0=m.scene.rotation.clone(),m.scene.quaternion0=m.scene.quaternion.clone(),m.scene.matrix0=m.scene.matrix.clone(),m.hipPos0=m.humanoid.getNormalizedBoneNode("hips").position.clone(),this._isLoading=!1,l(o)},o=>{let m=parseFloat((100*(o.loaded/o.total)).toPrecision(3)),d=document.getElementById("loaddisplay");d&&(d.innerHTML=m+"%")},o=>c(o))})}async loadFBX(e=null){this._isLoading=!0,e&&(this.animationUrl=e),this.currentMixer||(this.currentMixer=new v.AnimationMixer(this.currentVrm.scene));let t=await xe(this.animationUrl,this.currentVrm,this.scale);this.previousAction=this.action,this.action=this.currentMixer.clipAction(t),this.currentAction=this.action,this.previousAction?(this.previousAction.fadeOut(this.transitionDuration),this.action.reset().setEffectiveTimeScale(1).setEffectiveWeight(1).fadeIn(this.transitionDuration).play()):(this.action.play(),this.currentMixer.update(0),this.currentVrm.update(0)),this._isLoading=!1}async place(){await this.loadingPromise,this.scene.add(this.currentVrm.scene),this.center||this.currentVrm.scene.position.set(0,0,0)}async leave(e){e.remove(this.currentVrm.scene),S.deepDispose(this.currentVrm.scene),this.currentVrm=void 0,this.currentMixer=void 0}async changeVRM(e,t,i=null){let s=this.currentVrm.scene.position.clone(),n=this.currentVrm.scene.rotation.clone(),l=this.currentVrm.scene.rotation0.clone();s.y=0,await this.leave(e),await this.loadVRM(t,null,i),await this.place();let c=this.currentVrm.scene.rotation0.clone(),o=m=>new v.Quaternion().setFromEuler(m);n=o(n).multiply(o(l).clone().invert()),n=o(c).multiply(n),n=new v.Euler().setFromQuaternion(n,"YZX"),this.currentVrm.scene.position.copy(s),this.currentVrm.scene.rotation.copy(n),this.center&&(this.currentVrm.scene.position.y+=this.ground)}async changeFBX(e){await this.loadFBX(e)}isLoading(){return this._isLoading}update(){if(this._isLoading)return;let e=this.clock.getDelta();this.currentVrm&&this.currentVrm.update(e),this.currentMixer&&this.currentMixer.update(e)}};function xe(a,e,t){return new he().loadAsync(a).then(s=>{let n=v.AnimationClip.findByName(s.animations,"mixamo.com"),l=[],c=new v.Quaternion,o=new v.Quaternion,m=new v.Quaternion,d=new v.Vector3,y=s.getObjectByName("mixamorigHips").position.y,M=Math.abs(e.hipPos0.y)*t/y;return n.tracks.forEach(p=>{let R=p.name.split("."),T=R[0],E=fe[T],B=e.humanoid?.getNormalizedBoneNode(E)?.name,_=s.getObjectByName(T);if(B!=null){let u=R[1];if(_.getWorldQuaternion(c).invert(),_.parent.getWorldQuaternion(o),p instanceof v.QuaternionKeyframeTrack){for(let h=0;h<p.values.length;h+=4){let w=p.values.slice(h,h+4);m.fromArray(w),m.premultiply(o).multiply(c),m.toArray(w),w.forEach((r,V)=>{p.values[V+h]=r})}l.push(new v.QuaternionKeyframeTrack(`${B}.${u}`,p.times,p.values.map((h,w)=>e.meta?.metaVersion==="0"&&w%2===0?-h:h)))}else if(p instanceof v.VectorKeyframeTrack){let h=p.values.map((w,r)=>(e.meta?.metaVersion==="0"&&r%3!==1?-w:w)*M);l.push(new v.VectorKeyframeTrack(`${B}.${u}`,p.times,h))}}}),new v.AnimationClip("vrmAnimation",n.duration,l)})}var fe={mixamorigHips:"hips",mixamorigSpine:"spine",mixamorigSpine1:"chest",mixamorigSpine2:"upperChest",mixamorigNeck:"neck",mixamorigHead:"head",mixamorigLeftShoulder:"leftShoulder",mixamorigLeftArm:"leftUpperArm",mixamorigLeftForeArm:"leftLowerArm",mixamorigLeftHand:"leftHand",mixamorigLeftHandThumb1:"leftThumbMetacarpal",mixamorigLeftHandThumb2:"leftThumbProximal",mixamorigLeftHandThumb3:"leftThumbDistal",mixamorigLeftHandIndex1:"leftIndexProximal",mixamorigLeftHandIndex2:"leftIndexIntermediate",mixamorigLeftHandIndex3:"leftIndexDistal",mixamorigLeftHandMiddle1:"leftMiddleProximal",mixamorigLeftHandMiddle2:"leftMiddleIntermediate",mixamorigLeftHandMiddle3:"leftMiddleDistal",mixamorigLeftHandRing1:"leftRingProximal",mixamorigLeftHandRing2:"leftRingIntermediate",mixamorigLeftHandRing3:"leftRingDistal",mixamorigLeftHandPinky1:"leftLittleProximal",mixamorigLeftHandPinky2:"leftLittleIntermediate",mixamorigLeftHandPinky3:"leftLittleDistal",mixamorigRightShoulder:"rightShoulder",mixamorigRightArm:"rightUpperArm",mixamorigRightForeArm:"rightLowerArm",mixamorigRightHand:"rightHand",mixamorigRightHandPinky1:"rightLittleProximal",mixamorigRightHandPinky2:"rightLittleIntermediate",mixamorigRightHandPinky3:"rightLittleDistal",mixamorigRightHandRing1:"rightRingProximal",mixamorigRightHandRing2:"rightRingIntermediate",mixamorigRightHandRing3:"rightRingDistal",mixamorigRightHandMiddle1:"rightMiddleProximal",mixamorigRightHandMiddle2:"rightMiddleIntermediate",mixamorigRightHandMiddle3:"rightMiddleDistal",mixamorigRightHandIndex1:"rightIndexProximal",mixamorigRightHandIndex2:"rightIndexIntermediate",mixamorigRightHandIndex3:"rightIndexDistal",mixamorigRightHandThumb1:"rightThumbMetacarpal",mixamorigRightHandThumb2:"rightThumbProximal",mixamorigRightHandThumb3:"rightThumbDistal",mixamorigLeftUpLeg:"leftUpperLeg",mixamorigLeftLeg:"leftLowerLeg",mixamorigLeftFoot:"leftFoot",mixamorigLeftToeBase:"leftToes",mixamorigRightUpLeg:"rightUpperLeg",mixamorigRightLeg:"rightLowerLeg",mixamorigRightFoot:"rightFoot",mixamorigRightToeBase:"rightToes"};import*as k from"three";import*as Y from"gaussian-splats-3d";var D=class extends k.Group{constructor(e,t,i,s){super(),this.loadGS(e,t,i,s)}loadGS(e,t,i=[0,0,0],s=[0,0,1,0]){Array.isArray(e)||(e=[e]),this.loadingPromise=new Promise(async(n,l)=>{let c=new Y.DropInViewer({sharedMemoryForWorkers:!1,dynamicScene:!0,sceneRevealMode:2,sphericalHarmonicsDegree:2}),o=e.map(m=>({path:m,scale:[t,t,t],position:i,rotation:s,splatAlphaRemovalThreshold:0}));await c.addSplatScenes(o,!1),this.add(c),this.viewer=c,this.position0=new k.Vector3(...i),this.quaternion0=new k.Quaternion(...s),this.rotation0=new k.Euler().setFromQuaternion(this.quaternion0),this.matrix0=new k.Matrix4().compose(this.position0,this.quaternion0,new k.Vector3(1,1,1)),n(this)},void 0,function(n){console.error(n)})}};var q=class{constructor(){this.header=null,this.vertexCount=0,this.properties=[],this.propertyTypes=new Map([["char",1],["uchar",1],["short",2],["ushort",2],["int",4],["uint",4],["float",4],["double",8]])}async parsePLY(e,t){let i;if(e.endsWith(".ply")||(t=!1),t)try{let u=await fetch(e,{method:"HEAD"});i=Number(u.headers.get("content-length"))}catch{let h=await fetch(e);i=Number(h.headers.get("content-length"))}else{let u=await fetch(e);i=Number(u.headers.get("content-length"))}let n=(await fetch(e)).body.getReader(),l=[],c=0;for(;;){let{done:u,value:h}=await n.read();if(u)break;l.push(h),c+=h.length;let w=document.getElementById("loaddisplay");if(w)if(i){let r=c/i*100;w.innerHTML=`${r.toFixed(1)}% (1/2)`}else w.innerHTML=`${c} bytes loaded (1/2)`}let o=document.getElementById("loaddisplay");o&&(o.innerHTML=`${100 .toFixed(1)}% (1/2)`);let m=new ArrayBuffer(c),d=new Uint8Array(m),y=0;for(let u of l)d.set(u,y),y+=u.length;let g=new DataView(m);y=0;let M="";for(;;){let u=g.getUint8(y++);if(M+=String.fromCharCode(u),M.includes(`end_header
2
+ `))break}let p=M.split(`
3
+ `);this.header=p.filter(u=>u.trim()!=="");let R="binary_little_endian";for(let u of this.header)if(u.startsWith("format"))R=u.split(" ")[1];else if(u.startsWith("element vertex"))this.vertexCount=parseInt(u.split(" ")[2]);else if(u.startsWith("property")){let h=u.split(" ");this.properties.push({type:h[1],name:h[2]})}let T=this.properties.reduce((u,h)=>u+this.propertyTypes.get(h.type),0),E=[],B=new Uint8Array(m.slice(y));for(let u=0;u<this.vertexCount;u++){let h={rawData:B.slice(u*T,(u+1)*T)},w=0;for(let r of this.properties){let V=this.propertyTypes.get(r.type),I;switch(r.type){case"float":I=g.getFloat32(y+w,!0);break}h[r.name]=I,w+=V}if(E.push(h),y+=T,u%1e4===0){let r=u/this.vertexCount*100,V=document.getElementById("loaddisplay");V&&await new Promise(I=>{requestAnimationFrame(()=>{V.innerHTML=`${r.toFixed(1)}% (2/2)`,I()})})}}let _=document.getElementById("loaddisplay");return _&&(_.innerHTML=`${100 .toFixed(1)}% (2/2)`),{header:this.header,vertices:E,vertexCount:this.vertexCount,vertexSize:T}}createPLYFile(e,t,i){let s=e.join(`
4
+ `)+`
5
+ `,l=new TextEncoder().encode(s),c=new Uint8Array(t.length*i);t.forEach((m,d)=>{c.set(m.rawData,d*i)});let o=new Uint8Array(l.length+c.length);return o.set(l,0),o.set(c,l.length),o}async splitPLY(e,t){let i=await this.parsePLY(e,!1),s=l=>i.header.map(c=>c.startsWith("element vertex")?`element vertex ${l}`:c),n=[];for(let[l,c]of Object.entries(t)){let o=c.map(y=>i.vertices[y]),m=this.createPLYFile(s(o.length),o,i.vertexSize),d=new Blob([m],{type:"application/octet-stream"});n.push(URL.createObjectURL(d))}return n}};import K from"jszip";var Z=class a extends f.Group{constructor(e,t){super(),this.character=e,this.gs=t,this.debugAxes=new Map,this.isReady=!1,this.t=0}static async initVRM(e,t,i,s,n,l){l||(l=(await(await fetch("./assets/default.json")).json()).boneOperations),n||(n=1);let c=new A(t,e,"",n,!0);await c.loadingPromise,c.skinnedMeshIndex=1,c.faceIndex=void 0,c.currentVrm.scene.children.length>4&&(c.skinnedMeshIndex=2,c.faceIndex=1);let o=c.currentVrm.scene.children[c.skinnedMeshIndex];if(W(c,!1),F(c,l),o.skeleton.update(),o.skeleton.computeBoneTexture(),o.geometry.computeVertexNormals(),c.skinnedMeshIndex===2){let p=c.currentVrm.humanoid.getRawBoneNode("head"),R=new f.Bone;R.name="J_Bip_C_HeadTop_End",R.position.set(0,.2,-.05),R.updateMatrixWorld(!0),p.add(R),o.skeleton.bones.push(R),o.bind(new f.Skeleton(o.skeleton.bones),o.matrixWorld)}s.render(t,i),o.bindMatrix0=o.bindMatrix.clone(),o.bindMatrixInverse0=o.bindMatrixInverse.clone();let m=o.skeleton.boneTexture.image.width,d=o.skeleton.boneTexture.image.height,y=o.skeleton.boneTexture.format,g=o.skeleton.boneTexture.type,M=o.skeleton.boneTexture.image.data.slice();return o.boneTexture0=new f.DataTexture(M,m,d,y,g),o.boneTexture0.needsUpdate=!0,c}static async initGS(e,t,i,s){let n=await new D(e,1,t,i);return await n.loadingPromise,s.add(n),n.splatMesh=n.viewer.splatMesh,n.centers=n.splatMesh.splatDataTextures.baseData.centers,n.colors=n.splatMesh.splatDataTextures.baseData.colors,n.covariances=n.splatMesh.splatDataTextures.baseData.covariances,n.splatCount=n.splatMesh.geometry.attributes.splatIndex.array.length,n.centers0=new Float32Array(n.centers),n.colors0=new Float32Array(n.colors),n.covariances0=new Float32Array(n.covariances),n.splatMesh.updateDataTexturesFromBaseData(0,n.splatCount-1),n}static async load(e,t,i,s,n){console.log("Loading GVRM:",e);let l=await fetch(e),c=await K.loadAsync(l.arrayBuffer()),o=await c.file("model.vrm").async("arraybuffer"),m=await c.file("model.ply").async("arraybuffer"),d=JSON.parse(await c.file("data.json").async("text")),y=new Blob([o],{type:"application/octet-stream"}),g=URL.createObjectURL(y),M=new Blob([m],{type:"application/octet-stream"}),p=URL.createObjectURL(M),R=d.modelScale,T=d.boneOperations;d.splatRelativePoses===void 0&&(d.splatRelativePoses=d.relativePoses);let E=await a.initVRM(g,t,i,s,R,T),{sceneSplatIndices:B,boneSceneMap:_}=a.sortSplatsByBones(d),h=await new q().splitPLY(p,B),w=await a.initGS(h,d.gsPosition,d.gsQuaternion,t),r=new a(E,w);r.modelScale=R,r.boneOperations=T,r.boneSceneMap=_,r.fileName=n,r.updatePMC(),J(t,r.pmc),G(r.pmc,!1),s.render(t,i),r.gs.splatVertexIndices=d.splatVertexIndices,r.gs.splatBoneIndices=d.splatBoneIndices,r.gs.splatRelativePoses=d.splatRelativePoses,a.gsCustomizeMaterial(E,w);for(let b=0;b<r.gs.splatCount;b++){let L=Math.sqrt(r.gs.splatRelativePoses[b*3+0]**2+r.gs.splatRelativePoses[b*3+1]**2+r.gs.splatRelativePoses[b*3+2]**2);(r.gs.splatBoneIndices[b]!==57&&L>.2||r.gs.splatBoneIndices[b]==21&&L>.1||r.gs.splatBoneIndices[b]==19&&L>.1||r.gs.splatBoneIndices[b]===57&&L>.3)&&(r.gs.colors[b*4+3]=0)}r.gs.splatMesh.updateDataTexturesFromBaseData(0,r.gs.splatCount-1);function V(b,L=0){b.children.forEach(function(H){H.isBone&&(["J_Bip_L_Hand","J_Bip_L_LowerArm","J_Bip_R_Hand","J_Bip_R_LowerArm","J_Bip_L_LowerLeg","J_Bip_L_Foot","J_Bip_R_LowerLeg","J_Bip_R_Foot","J_Bip_C_Neck","J_Bip_C_Spine","J_Bip_C_Chest","J_Bip_C_UpperChest","J_Bip_C_HeadTop_End","J_Bip_C_Head"].includes(H.name)&&(H.updateMatrix(),H.matrixWorld0=H.matrixWorld.clone()),V(H,L+1))})}let I=E.currentVrm.scene.children[0].children[0];return V(I,1),r.isReady=!0,r}static async save(e,t,i,s,n,l,c=!1){let o=await fetch(t).then(p=>p.arrayBuffer()),m=await fetch(i).then(p=>p.arrayBuffer()),d={modelScale:n,boneOperations:s,gsQuaternion:e.gs.viewer.splatMesh.scenes[0].quaternion.toArray(),gsPosition:e.gs.viewer.splatMesh.scenes[0].position.toArray(),splatVertexIndices:e.gs.splatVertexIndices,splatBoneIndices:e.gs.splatBoneIndices,splatRelativePoses:e.gs.splatRelativePoses},y=new K;y.file("model.vrm",o),y.file("model.ply",m),y.file("data.json",JSON.stringify(d,null,2));let g=await y.generateAsync({type:"blob"});if(!l&&i.endsWith(".ply")?l=i.split("/").pop().replace(".ply",".gvrm"):l||(l=i.split("/").pop()+".gvrm"),M(g,l),c){console.log("savePly!");let p=new Blob([m],{type:"application/octet-stream"}),R=l.replace(".gvrm","_processed.ply");M(p,R)}function M(p,R){let T=URL.createObjectURL(p),E=document.createElement("a");E.href=T,E.download=R,E.click(),p===g&&e.url&&URL.revokeObjectURL(e.url),p===g?e.url=T:URL.revokeObjectURL(T)}}static async remove(e,t){e.character&&(await e.character.leave(t),e.character=null),e.gs&&(await e.gs.viewer.dispose(),e.gs=null),e.pmc&&N(t,e.pmc)}async load(e,t,i,s,n=null){let l=await a.load(e,t,i,s,n);this.character=l.character,this.gs=l.gs,this.modelScale=l.modelScale,this.boneOperations=l.boneOperations,this.boneSceneMap=l.boneSceneMap,this.vertexSceneMap=l.vertexSceneMap,this.fileName=l.fileName,this.isReady=!0}async save(e,t,i,s,n,l=!1){await a.save(this,e,t,i,s,n,l)}async remove(e){this.isReady=!1,await a.remove(this,e)}async changeFBX(e){await this.character.changeFBX(e)}updatePMC(){let{pmc:e}=O(this.character);this.pmc=e}updateByBones(){let e=new f.Vector3,t=new f.Vector3,i=new f.Vector3,s=new f.Matrix4,n=new f.Quaternion,l=["J_Bip_C_Neck","J_Bip_C_Spine","J_Bip_C_Chest","J_Bip_C_UpperChest","J_Bip_C_HeadTop_End","J_Bip_C_Head"],c=this.character.currentVrm.scene.children[2].skeleton;c.bones.forEach(o=>{let m=o.children;m.length!==0&&m.forEach(d=>{let y=c.bones.indexOf(d),g=this.boneSceneMap[y];if(g===void 0)return;o.updateMatrixWorld(!0),d.updateMatrixWorld(!0),e.setFromMatrixPosition(o.matrixWorld),t.setFromMatrixPosition(d.matrixWorld),i.addVectors(e,t).multiplyScalar(.5),i.sub(this.character.currentVrm.scene.position).add(this.character.currentVrm.scene.position0),i.applyQuaternion(this.gs.viewer.quaternion.clone().invert()),s.extractRotation(d.matrixWorld.multiply(d.matrixWorld0.clone().invert())),n.setFromRotationMatrix(s),n.premultiply(this.gs.viewer.quaternion.clone().invert()),n.multiply(this.gs.quaternion0);let M=this.gs.viewer.getSplatScene(g);if(M){l.includes(d.name)||(M.position.copy(i),M.quaternion.copy(n));let p=this.debugAxes.get(g);p||(p=this.createDebugAxes(g)),p.position.copy(i),p.quaternion.copy(n)}})})}createDebugAxes(e){let t=new f.AxesHelper(.3);return t.visible=!1,this.gs.add(t),this.debugAxes.set(e,t),t}update(){if(!this.isReady)return;let e=this.character.currentVrm.scene.quaternion.clone(),t=this.character.currentVrm.scene.quaternion0.clone(),i=this.character.currentVrm.scene.position.clone(),s=this.character.currentVrm.scene.position0.clone();this.gs.viewer.quaternion.copy(e.multiply(t.invert())),this.gs.viewer.position.copy(i.sub(s)),this.updateByBones(),this.character.update()}static sortSplatsByBones(e){let t={},i=0,s={};for(let n=0;n<e.splatBoneIndices.length;n++){let l=e.splatBoneIndices[n];s[l]===void 0&&(s[l]=i,i++,t[s[l]]=[]),t[s[l]].push(n)}return a.updateExtraData(e,t),{sceneSplatIndices:t,boneSceneMap:s}}static updateExtraData(e,t){let i=[];for(let c=0;c<Object.keys(t).length;c++)i=i.concat(t[c]);let s=[],n=[],l=[];for(let c of Object.keys(t))for(let o of t[c])s.push(e.splatVertexIndices[o]),n.push(e.splatBoneIndices[o]),l.push(e.splatRelativePoses[o*3],e.splatRelativePoses[o*3+1],e.splatRelativePoses[o*3+2]);e.splatVertexIndices=s,e.splatBoneIndices=n,e.splatRelativePoses=l}static gsCustomizeMaterial(e,t){t.splatMesh.material=t.splatMesh.material.clone(),t.splatMesh.material.needsUpdate=!0;let i=e.currentVrm.scene.children[e.skinnedMeshIndex],s=i.geometry.attributes.position.count,n=i.geometry.attributes.position.array,l=i.geometry.attributes.normal.array,c=i.geometry.attributes.skinIndex.array,o=i.geometry.attributes.skinWeight.array,m=t.splatVertexIndices,d=t.splatRelativePoses,y=new Float32Array(4096*1024*4),g=new Float32Array(4096*1024*4),M=new Float32Array(4096*1024*4),p=new Float32Array(4096*1024*4),R=new Float32Array(4096*1024*4),T=new Float32Array(4096*1024*4);C(n,y,s,1),C(l,g,s,1),M.set(c),p.set(o),C(m,R,t.splatCount,3),C(d,T,t.splatCount,1);let E=P(y,4096,1024,f.RGBAFormat,f.FloatType),B=P(g,4096,1024,f.RGBAFormat,f.FloatType),_=P(M,4096,1024,f.RGBAFormat,f.FloatType),u=P(p,4096,1024,f.RGBAFormat,f.FloatType),h=P(R,4096,1024,f.RGBAFormat,f.FloatType),w=P(T,4096,1024,f.RGBAFormat,f.FloatType);t.splatMesh.material.onBeforeCompile=function(r){r.uniforms.meshPositionTexture={value:E},r.uniforms.meshNormalTexture={value:B},r.uniforms.meshSkinIndexTexture={value:_},r.uniforms.meshSkinWeightTexture={value:u},r.uniforms.gsMeshVertexIndexTexture={value:h},r.uniforms.gsMeshRelativePosTexture={value:w},r.uniforms.bindMatrix0={value:i.bindMatrix0},r.uniforms.bindMatrix={value:i.bindMatrix},r.uniforms.bindMatrixInverse0={value:i.bindMatrixInverse0},r.uniforms.bindMatrixInverse={value:i.bindMatrixInverse},r.uniforms.boneTexture0={value:i.boneTexture0},r.uniforms.boneTexture={value:i.skeleton.boneTexture},r.uniforms.meshMatrixWorld={value:e.currentVrm.scene.matrixWorld},r.uniforms.gsMatrix0={value:t.matrix0},r.uniforms.gsMatrix={value:t.viewer.matrixWorld},r.vertexShader=r.vertexShader.replace("#include <common>",`
6
+ #define USE_SKINNING
7
+
8
+ #include <common>
9
+ #include <skinning_pars_vertex> // boneTexture
10
+
11
+ uniform sampler2D meshPositionTexture;
12
+ uniform sampler2D meshNormalTexture;
13
+ uniform sampler2D meshSkinIndexTexture;
14
+ uniform sampler2D meshSkinWeightTexture;
15
+ uniform sampler2D gsMeshVertexIndexTexture;
16
+ uniform sampler2D gsMeshRelativePosTexture;
17
+ uniform mat4 meshMatrixWorld;
18
+ uniform mat4 gsMatrix0;
19
+ uniform mat4 gsMatrix;
20
+
21
+ uniform mat4 bindMatrix0;
22
+ uniform mat4 bindMatrixInverse0;
23
+ uniform highp sampler2D boneTexture0;
24
+
25
+ mat4 getBoneMatrix0( const in float i ) {
26
+ int size = textureSize( boneTexture0, 0 ).x;
27
+ int j = int( i ) * 4;
28
+ int x = j % size;
29
+ int y = j / size;
30
+ vec4 v1 = texelFetch( boneTexture0, ivec2( x, y ), 0 );
31
+ vec4 v2 = texelFetch( boneTexture0, ivec2( x + 1, y ), 0 );
32
+ vec4 v3 = texelFetch( boneTexture0, ivec2( x + 2, y ), 0 );
33
+ vec4 v4 = texelFetch( boneTexture0, ivec2( x + 3, y ), 0 );
34
+ return mat4( v1, v2, v3, v4 );
35
+ }
36
+
37
+ // TODO: check this
38
+ vec4 quatFromMat3(mat3 m) {
39
+ float trace = m[0][0] + m[1][1] + m[2][2];
40
+ vec4 q;
41
+
42
+ if (trace > 0.0) {
43
+ float s = 0.5 / sqrt(trace + 1.0);
44
+ q.w = 0.25 / s;
45
+ q.x = (m[2][1] - m[1][2]) * s;
46
+ q.y = (m[0][2] - m[2][0]) * s;
47
+ q.z = (m[1][0] - m[0][1]) * s;
48
+ } else if (m[0][0] > m[1][1] && m[0][0] > m[2][2]) {
49
+ float s = 2.0 * sqrt(1.0 + m[0][0] - m[1][1] - m[2][2]);
50
+ q.w = (m[2][1] - m[1][2]) / s;
51
+ q.x = 0.25 * s;
52
+ q.y = (m[0][1] + m[1][0]) / s;
53
+ q.z = (m[0][2] + m[2][0]) / s;
54
+ } else if (m[1][1] > m[2][2]) {
55
+ float s = 2.0 * sqrt(1.0 + m[1][1] - m[0][0] - m[2][2]);
56
+ q.w = (m[0][2] - m[2][0]) / s;
57
+ q.x = (m[0][1] + m[1][0]) / s;
58
+ q.y = 0.25 * s;
59
+ q.z = (m[1][2] + m[2][1]) / s;
60
+ } else {
61
+ float s = 2.0 * sqrt(1.0 + m[2][2] - m[0][0] - m[1][1]);
62
+ q.w = (m[1][0] - m[0][1]) / s;
63
+ q.x = (m[0][2] + m[2][0]) / s;
64
+ q.y = (m[1][2] + m[2][1]) / s;
65
+ q.z = 0.25 * s;
66
+ }
67
+ return q;
68
+ }
69
+
70
+ vec4 quatInverse(vec4 q) {
71
+ return vec4(-q.x, -q.y, -q.z, q.w) / dot(q, q);
72
+ }
73
+
74
+ vec4 quatMultiply(vec4 a, vec4 b) {
75
+ return vec4(
76
+ a.w * b.x + a.x * b.w + a.y * b.z - a.z * b.y,
77
+ a.w * b.y - a.x * b.z + a.y * b.w + a.z * b.x,
78
+ a.w * b.z + a.x * b.y - a.y * b.x + a.z * b.w,
79
+ a.w * b.w - a.x * b.x - a.y * b.y - a.z * b.z
80
+ );
81
+ }
82
+
83
+ mat3 mat3FromQuat(vec4 q) {
84
+ float x = q.x, y = q.y, z = q.z, w = q.w;
85
+ float x2 = x + x, y2 = y + y, z2 = z + z;
86
+ float xx = x * x2, xy = x * y2, xz = x * z2;
87
+ float yy = y * y2, yz = y * z2, zz = z * z2;
88
+ float wx = w * x2, wy = w * y2, wz = w * z2;
89
+
90
+ return mat3(
91
+ 1.0 - (yy + zz), xy - wz, xz + wy,
92
+ xy + wz, 1.0 - (xx + zz), yz - wx,
93
+ xz - wy, yz + wx, 1.0 - (xx + yy)
94
+ );
95
+ }
96
+ `),r.vertexShader=r.vertexShader.replace("mat4 transform = transforms[sceneIndex]","mat4 transform = gsMatrix * gsMatrix0;"),r.vertexShader=r.vertexShader.replace("vec3 splatCenter = uintBitsToFloat(uvec3(sampledCenterColor.gba));",`
97
+ vec2 samplerUV2 = vec2(0.0, 0.0);
98
+ float d2 = float(splatIndex) / 4096.0;
99
+ samplerUV2.y = float(floor(d2)) / 1024.0;
100
+ samplerUV2.x = fract(d2);
101
+ float meshVertexIndex = texture2D(gsMeshVertexIndexTexture, samplerUV2).r;
102
+ vec3 relativePos = texture2D(gsMeshRelativePosTexture, samplerUV2).rgb;
103
+
104
+ vec2 samplerUV3 = vec2(0.0, 0.0);
105
+ float d3 = float(meshVertexIndex) / 4096.0;
106
+ samplerUV3.y = float(floor(d3)) / 1024.0;
107
+ samplerUV3.x = fract(d3);
108
+ vec3 transformed = texture2D(meshPositionTexture, samplerUV3).rgb;
109
+ vec3 objectNormal = texture2D(meshNormalTexture, samplerUV3).rgb;
110
+ vec4 skinIndex = texture2D(meshSkinIndexTexture, samplerUV3);
111
+ vec4 skinWeight = texture2D(meshSkinWeightTexture, samplerUV3);
112
+
113
+ mat4 boneMatX0 = getBoneMatrix0( skinIndex.x );
114
+ mat4 boneMatY0 = getBoneMatrix0( skinIndex.y );
115
+ mat4 boneMatZ0 = getBoneMatrix0( skinIndex.z );
116
+ mat4 boneMatW0 = getBoneMatrix0( skinIndex.w );
117
+ mat4 skinMatrix0 = mat4( 0.0 );
118
+ skinMatrix0 += skinWeight.x * boneMatX0;
119
+ skinMatrix0 += skinWeight.y * boneMatY0;
120
+ skinMatrix0 += skinWeight.z * boneMatZ0;
121
+ skinMatrix0 += skinWeight.w * boneMatW0;
122
+ skinMatrix0 = bindMatrixInverse0 * skinMatrix0 * bindMatrix0;
123
+
124
+ #include <skinbase_vertex> // boneMat
125
+ #include <skinnormal_vertex> // skinMatrix, using normal
126
+ #include <defaultnormal_vertex> // ?
127
+ #include <skinning_vertex>
128
+
129
+ // vec3 splatCenter = ( vec4(transformed, 1.0) ).xyz;
130
+ // vec3 splatCenter = ( meshMatrixWorld * vec4(transformed, 1.0) ).xyz;
131
+ // vec3 splatCenter = ( meshMatrixWorld * vec4(transformed + relativePos, 1.0) ).xyz; // GOOD
132
+
133
+ vec3 skinnedRelativePos = vec4( skinMatrix * inverse(skinMatrix0) * vec4( relativePos, 0.0 ) ).xyz;
134
+ vec3 splatCenter = ( meshMatrixWorld * vec4(transformed + skinnedRelativePos, 1.0) ).xyz;
135
+ `),r.vertexShader=r.vertexShader.replace("vec4 viewCenter = transformModelViewMatrix * vec4(splatCenter, 1.0);",`
136
+ // The splatCenter is the coordinate system for inverse(transform).
137
+ splatCenter = (inverse(transform) * vec4(splatCenter, 1.0)).xyz;
138
+ vec4 viewCenter = transformModelViewMatrix * vec4(splatCenter, 1.0);
139
+ `),r.vertexShader=r.vertexShader.replace("mat3 cov2Dm = transpose(T) * Vrk * T;",`
140
+ // for debug
141
+ // Vrk[0][0] *= 25.0; Vrk[1][1] *= 0.1; Vrk[2][2] *= 0.1;
142
+ // Vrk[1][1] *= 25.0; Vrk[0][0] *= 0.1; Vrk[2][2] *= 0.1;
143
+ // Vrk[2][2] *= 25.0; Vrk[0][0] *= 0.1; Vrk[1][1] *= 0.1;
144
+
145
+ // via quat
146
+ mat3 gsRotation0 = mat3(gsMatrix0);
147
+ mat3 skinRotationMatrix = mat3(skinMatrix * inverse(skinMatrix0));
148
+ mat3 relativeRotation = transpose(gsRotation0) * skinRotationMatrix * gsRotation0;
149
+ vec4 tempQuat = quatFromMat3(relativeRotation);
150
+ tempQuat.y = -tempQuat.y; // Hardcode, maybe bug in quatFromMat3?
151
+ relativeRotation = mat3FromQuat(tempQuat);
152
+ mat3 rotatedVrk = transpose(relativeRotation) * Vrk * relativeRotation;
153
+ mat3 cov2Dm = transpose(T) * rotatedVrk * T;
154
+
155
+ // TODO: via mat
156
+ // mat3 gsRotation0 = mat3(gsMatrix0);
157
+ // mat3 skinRotationMatrix = mat3(skinMatrix * inverse(skinMatrix0));
158
+ // mat3 relativeRotation = transpose(gsRotation0) * skinRotationMatrix * gsRotation0;
159
+ // mat3 rotatedVrk = transpose(relativeRotation) * Vrk * relativeRotation;
160
+ // mat3 cov2Dm = transpose(T) * rotatedVrk * T;
161
+ `)},t.splatMesh.material.needsUpdate=!0}};export{Z as GVRM,Q as GVRMUtils};
package/package.json ADDED
@@ -0,0 +1,45 @@
1
+ {
2
+ "name": "@naruya/gaussian-vrm",
3
+ "version": "1.0.0",
4
+ "description": "3D Gaussian Splatting with VRM character animation",
5
+ "main": "lib/gaussian-vrm.min.js",
6
+ "module": "lib/gaussian-vrm.min.js",
7
+ "type": "module",
8
+ "exports": {
9
+ ".": {
10
+ "import": "./lib/gaussian-vrm.min.js"
11
+ }
12
+ },
13
+ "files": [
14
+ "lib/gaussian-vrm.min.js"
15
+ ],
16
+ "peerDependencies": {
17
+ "three": "^0.170.0",
18
+ "@pixiv/three-vrm": "^2.1.0",
19
+ "jszip": "^3.10.1"
20
+ },
21
+ "repository": {
22
+ "type": "git",
23
+ "url": "https://github.com/naruya/gaussian-vrm"
24
+ },
25
+ "homepage": "https://github.com/naruya/gaussian-vrm#readme",
26
+ "bugs": {
27
+ "url": "https://github.com/naruya/gaussian-vrm/issues"
28
+ },
29
+ "keywords": [
30
+ "gaussian-splatting",
31
+ "3dgs",
32
+ "vrm",
33
+ "vtuber",
34
+ "avatar",
35
+ "three.js",
36
+ "threejs",
37
+ "3d",
38
+ "animation"
39
+ ],
40
+ "author": "naruya",
41
+ "license": "MIT",
42
+ "publishConfig": {
43
+ "access": "public"
44
+ }
45
+ }