@ssafy-mhk/e-ver 1.0.0 → 1.0.2

package/README.md

@@ -1,10 +1,175 @@
-# E-ver 프로젝트 SDK
+# `@ssafy-mhk/e-ver` - Virtual Fitting React SDK 👗👕
 
-## 프로젝트 테스트
+S14P21M104 가상 피팅(WebAR) 프로젝트를 위한 공식 프론트엔드 React SDK입니다.
+복잡한 Three.js 카메라 및 조명 설정, MediaPipe AI 바디 트래킹 연동, SkinnedMesh 기반 의류 렌더링 로직을 추상화하여, **단순한 React 컴포넌트와 Hook**으로 증강현실(AR) 가상 피팅 기능을 빠르게 구현할 수 있도록 지원합니다.
 
-## 프로젝트 빌드 및 로컬 배포
+## 개발 스택 (Technology Stack)
+
+- **Framework**: React / Next.js 호환
+- **3D & AR Rendering**: Three.js, React Three Fiber (R3F), Drei, three-vrm
+- **AI Tracking**: MediaPipe Pose Landmarker (Lite)
+- **State Management**: Zustand
+
+---
+
+## 🛠 설치 방법 (Installation)
+
+이 패키지는 사내 또는 외부 NPM에 직접 배포되기 전 로컬 저장소를 통한 심볼릭 링크(Symbolic Link) 방식으로 연결하거나 파일 디렉토리를 참조하여 사용할 수 있습니다.
+
+### 로컬 프로젝트에서 참조하여 설치할 경우:
+
+여러분의 Next.js 프로젝트 최상위 폴더(`frontend`)에서 아래 명령어를 실행하세요.
 
 ```bash
-pnpm build
-pnpm link --global
+# pnpm 사용 시
+pnpm add @ssafy-mhk/e-ver
+
+# 또는 패키지 매니저에 따라
+npm install @ssafy-mhk/e-ver
+yarn add @ssafy-mhk/e-ver
+```
+
+_(의존성 설치가 잘 되었는지 꼭 `package.json` 코드를 확인하세요.)_
+
+---
+
+## 🚀 코어로직 한눈에 보기 (SDK 핵심 요소)
+
+SDK는 크게 **State(상태 관리)**, **Renderer(렌더링 UI)**, **Tracker(추적/측정 Hook)** 이렇게 3가지 핵심 축으로 구성됩니다.
+
+| 분류         | 모듈명            | 역할                                                                                 |
+| ------------ | ----------------- | ------------------------------------------------------------------------------------ |
+| **State**    | `useEverStore`    | 웹캠 활성화 여부, 렌더링 중인 옷/아바타 주소, 최종 핏 결과(체형 측정) 관리           |
+| **Renderer** | `EverCanvas`      | AR/3D 환경에 최적화된 조명과 카메라 세팅을 내장한 R3F 캔버스 래퍼                    |
+| **Renderer** | `AvatarModel`     | 사용자의 AI 기반 생성 3D 아바타(GLB)를 로딩하고, 뼈대(Bone) 구조를 추출              |
+| **Renderer** | `GarmentModel`    | 사용자가 선택한 옷(GLB)을 로딩하고, 옷의 SkinnedMesh를 대상 아바타에 입힘            |
+| **Tracker**  | `usePoseTracker`  | 브라우저 내에서 MediaPipe를 가동시켜 실시간 30-60Hz 웹캠 추적                        |
+| **Tracker**  | `usePoseMapper`   | 웹캠에서 얻은 2D/3D 점(Landmarks)을 필수 13본(Bone)의 회전값(Quaternion)으로 변환    |
+| **Tracker**  | `useBodyMeasurer` | AR 카메라 상에서 사용자 실루엣 기반 사이즈(어깨, 가슴, 허리)를 연산하여 옷의 핏 평가 |
+
+---
+
+## 📖 사용 가이드 (Usage Guide)
+
+아래는 Next.js(App Router) 환경에서 가상 피팅 서비스를 구축하는 기본 예제 코드입니다.
+
+### Step 1. Zustand로 글로벌 상태 세팅하기
+
+우선, 애플리케이션 외부 버튼 조작을 위해 스토어를 호출할 수 있습니다.
+
+```tsx
+// app/fitting/page.tsx
+"use client";
+
+import { useEverStore } from "@ssafy-mhk/e-ver";
+
+export default function FittingControls() {
+  const { setWebcamActive, setActiveGarmentUrl, fitResult } = useEverStore();
+
+  return (
+    <div className="absolute top-4 left-4 z-10 flex flex-col gap-2">
+      <button onClick={() => setWebcamActive(true)}>웹캠 가동 시작 🎥</button>
+      <button onClick={() => setActiveGarmentUrl("/path/to/clothes.glb")}>
+        새로운 옷 입어보기 👕
+      </button>
+
+      {fitResult && (
+        <div className="bg-white/80 p-2 rounded">
+          핏 타입: {fitResult.type} (적합도 {fitResult.suitability}%)
+        </div>
+      )}
+    </div>
+  );
+}
+```
+
+### Step 2. 3D AR 캔버스 및 모델 구축하기
+
+SDK에서 제공하는 `EverCanvas`, `AvatarModel`, `GarmentModel`을 활용하여 화면을 그립니다.
+
+```tsx
+// components/VirtualFittingScene.tsx
+"use client";
+
+import { useState } from "react";
+import {
+  EverCanvas,
+  AvatarModel,
+  GarmentModel,
+  useEverStore,
+} from "@ssafy-mhk/e-ver";
+
+export default function VirtualFittingScene() {
+  const [avatarBones, setAvatarBones] = useState<Record<string, THREE.Bone>>();
+  const { activeGarmentUrl } = useEverStore();
+
+  // 내 계정의 3D 아바타 파일 경로
+  const AVATAR_URL = "/models/my-avatar.glb";
+
+  return (
+    <div className="w-full h-screen bg-transparent relative">
+      {/* 1. 기본 AR 조명이 세팅된 캔버스 호출 */}
+      <EverCanvas showEnvironment={true}>
+        {/* 2. 사용자의 베이스 아바타 불러오기 */}
+        <AvatarModel
+          url={AVATAR_URL}
+          onBonesReady={(bones) => setAvatarBones(bones)}
+        />
+
+        {/* 3. 활성화된 옷이 있다면 아바타 위에 덧입히기 */}
+        {activeGarmentUrl && avatarBones && (
+          <GarmentModel
+            url={activeGarmentUrl}
+            targetAvatarBones={avatarBones}
+          />
+        )}
+      </EverCanvas>
+    </div>
+  );
+}
+```
+
+### Step 3. AI 바디 트래킹 연동하기
+
+웹캠 화면과 R3F 화면을 겹쳐주려면 `<video>` 태그를 `usePoseTracker`에 넘겨 연결해 주어야 합니다.
+
+```tsx
+// components/ArCameraView.tsx
+"use client";
+
+import { useRef } from "react";
+import { usePoseTracker, useEverStore } from "@ssafy-mhk/e-ver";
+
+export default function ArCameraView() {
+  const videoRef = useRef<HTMLVideoElement>(null);
+
+  // SDK 내부의 로직을 통해 실시간으로 프레임을 MediaPipe에 전송
+  const { isTrackerReady } = usePoseTracker(videoRef);
+  const { isWebcamActive } = useEverStore();
+
+  return (
+    <div className="absolute inset-0 z-0">
+      <video
+        ref={videoRef}
+        autoPlay
+        playsInline
+        muted
+        className="w-full h-full object-cover"
+        style={{ display: isWebcamActive ? "block" : "none" }}
+      />
+      {/* Tracker가 로딩 중일 때 로딩 오버레이 표시 가능 */}
+      {!isTrackerReady && isWebcamActive && <p>AI 모델 로딩 중...</p>}
+    </div>
+  );
+}
 ```
+
+---
+
+## 🙋‍♂️ Q&A (추가 기능 문의)
+
+**Q. 옷 핏 평가는 어떻게 동작하나요?**  
+-> 내부에 구축된 `useBodyMeasurer` 훅이 약 1초 단위(1Hz)로 캔버스의 실루엣을 통해 [어깨, 가슴, 허리] 사이즈를 분석하여 `useEverStore`의 `fitResult`에 자동 업데이트합니다.
+
+**Q. 물리 엔진(Rapier.js)이 적용되어 있나요?**  
+-> 현재 MVP 스코프에서 무거운 런타임 물리 물리엔진은 제외되었으며, 대신 **SkinnedMesh 본 바인딩(P0)**과 **Normal Map 애니메이션(P1)**을 통하여 주름과 디테일을 시뮬레이션합니다.

package/dist/api/EverClient.d.ts

@@ -0,0 +1,168 @@
+export type GarmentCategory = "top" | "bottom" | "dress" | "outer" | "accessory";
+export type GarmentAssetStatus = "uploaded" | "trellis_pending" | "trellis_complete" | "rigging_pending" | "rigging_in_progress" | "rigging_complete" | "rigging_failed" | "rigging_fallback" | "ready";
+export interface BodyPreset {
+    id: string;
+    preset_id: string;
+    base_gender: string;
+    size_band: string;
+    upper_body_silhouette: string;
+    torso_or_pelvis_profile: string;
+    model_url: string | null;
+    height_cm: number | null;
+    chest_cm: number | null;
+    waist_cm: number | null;
+    hip_cm: number | null;
+    shoulder_width_cm: number | null;
+    arm_length_cm: number | null;
+    inseam_cm: number | null;
+    torso_length_cm: number | null;
+    neck_to_hip_cm: number | null;
+}
+export interface BodyPresetMatchRequest {
+    gender: string;
+    height_cm?: number;
+    chest_cm?: number;
+    waist_cm?: number;
+    hip_cm?: number;
+    shoulder_width_cm?: number;
+}
+export interface GarmentData {
+    name: string;
+    category: GarmentCategory;
+    brand_id?: string;
+    description?: string;
+    material?: string;
+    price?: number | string;
+    thumbnail_url?: string;
+    model_url?: string;
+}
+export interface GarmentImage {
+    id: string;
+    image_url: string;
+    image_type: string;
+    sort_order: number;
+    is_primary: boolean;
+}
+export interface GarmentSize {
+    id: string;
+    garment_id: string;
+    size: string;
+    fit_type: string;
+    shoulder_width_cm: number | null;
+    chest_cm: number | null;
+    total_length_cm: number | null;
+    waist_cm: number | null;
+    hip_cm: number | null;
+    sleeve_length_cm: number | null;
+    inseam_cm: number | null;
+    rise_cm: number | null;
+    thigh_cm: number | null;
+    hem_cm: number | null;
+    created_at: string;
+    updated_at: string;
+}
+export interface GarmentVariant {
+    id: string;
+    color: string | null;
+    pattern: string | null;
+    is_default: boolean;
+    model_url: string | null;
+    size: string | null;
+    fit_type: string | null;
+    shoulder_width_cm: number | null;
+    chest_cm: number | null;
+    total_length_cm: number | null;
+    waist_cm: number | null;
+    hip_cm: number | null;
+    sleeve_length_cm: number | null;
+    inseam_cm: number | null;
+    rise_cm: number | null;
+    thigh_cm: number | null;
+    hem_cm: number | null;
+}
+export interface GarmentSummary {
+    id: string;
+    brand_id: string;
+    name: string;
+    category: string;
+    occlusion_hint: string | null;
+    description: string | null;
+    material: string | null;
+    price: number | string | null;
+    external_url: string | null;
+    thumbnail_url: string | null;
+    model_url: string | null;
+    asset_status: GarmentAssetStatus;
+    is_active: boolean;
+    created_at: string;
+    updated_at: string;
+}
+export interface GarmentDetail extends GarmentSummary {
+    brand: {
+        id: string;
+        name: string;
+        logo_url: string | null;
+        website_url: string | null;
+    } | null;
+    images: GarmentImage[];
+    sizes: GarmentSize[];
+    variants: GarmentVariant[];
+}
+export interface GarmentListResponse {
+    items: GarmentSummary[];
+    total: number;
+    page: number;
+    size: number;
+}
+export interface GarmentPipelineStatus {
+    garment_id: string;
+    asset_status: GarmentAssetStatus;
+    progress: number;
+    message: string;
+}
+export interface Generate3DResponse {
+    garment_id: string;
+    asset_status: GarmentAssetStatus;
+    status_url: string;
+    message: string;
+    include_measurements: boolean;
+    target_fit_types: string[];
+}
+export interface ListGarmentsParams {
+    category?: string;
+    search?: string;
+    page?: number;
+    size?: number;
+}
+export interface WaitForGarmentModelOptions {
+    intervalMs?: number;
+    timeoutMs?: number;
+}
+/**
+ * EverClient provides methods to interact with the e-ver backend API.
+ */
+export declare class EverClient {
+    private readonly baseUrl;
+    private readonly apiOrigin;
+    constructor(baseUrl?: string);
+    resolveAssetUrl(assetUrl: string | null): string | null;
+    listBodyPresets(gender?: string): Promise<BodyPreset[]>;
+    matchBodyPreset(body: BodyPresetMatchRequest): Promise<BodyPreset>;
+    listGarments(params?: ListGarmentsParams): Promise<GarmentListResponse>;
+    getGarment(garmentId: string): Promise<GarmentDetail>;
+    getPipelineStatus(garmentId: string): Promise<GarmentPipelineStatus>;
+    createGarment(data: GarmentData): Promise<GarmentSummary>;
+    uploadGarmentImage(garmentId: string, photo: Blob, type: "front" | "back"): Promise<GarmentImage[]>;
+    generateGarment3D(garmentId: string, options?: {
+        include_measurements?: boolean;
+        target_fit_types?: string[];
+    }): Promise<Generate3DResponse>;
+    waitForGarmentModel(garmentId: string, options?: WaitForGarmentModelOptions): Promise<GarmentDetail>;
+    pickGarmentModelUrl(garment: Pick<GarmentDetail, "model_url" | "variants">): string | null;
+    private normalizeBodyPreset;
+    private normalizeGarmentSummary;
+    private normalizeGarmentDetail;
+    private request;
+    private buildUrl;
+    private getResponseErrorMessage;
+}

package/dist/components/AvatarModel.d.ts

@@ -0,0 +1,11 @@
+import React from "react";
+import * as THREE from "three";
+export interface AvatarModelProps {
+    url: string | null;
+    onBonesReady?: (bones: Record<string, THREE.Bone>) => void;
+}
+/**
+ * Loads and renders the user's 3D avatar (GLB/VRM).
+ * Automatically registers bone references to the EverProvider for the pose mapping pipeline.
+ */
+export declare const AvatarModel: React.FC<AvatarModelProps>;

package/dist/components/CameraView.d.ts

@@ -0,0 +1,10 @@
+import React from "react";
+export interface CameraViewProps {
+    className?: string;
+    style?: React.CSSProperties;
+}
+/**
+ * CameraView handles the rendering of the user's webcam feed.
+ * It automatically registers the video element to the EverProvider for tracking.
+ */
+export declare const CameraView: React.FC<CameraViewProps>;

package/dist/components/EverCanvas.d.ts

@@ -0,0 +1,10 @@
+import React from "react";
+export interface EverCanvasProps {
+    children: React.ReactNode;
+    showEnvironment?: boolean;
+}
+/**
+ * A wrapper around R3F <Canvas> specifically configured for optimal
+ * 3D garment rendering and AR visualization.
+ */
+export declare const EverCanvas: React.FC<EverCanvasProps>;

package/dist/components/EverProvider.d.ts

@@ -0,0 +1,17 @@
+import React from "react";
+import * as THREE from "three";
+export interface EverContextType {
+    videoRef: React.RefObject<HTMLVideoElement | null>;
+    avatarBones: Record<string, THREE.Bone> | null;
+    setAvatarBones: (bones: Record<string, THREE.Bone> | null) => void;
+    isProviderReady: boolean;
+}
+export declare const useEver: () => EverContextType;
+export interface EverProviderProps {
+    children: React.ReactNode;
+}
+/**
+ * EverProvider coordinates the shared state between tracking and rendering.
+ * It manages the video reference for MediaPipe and the bone mappings for 3D models.
+ */
+export declare const EverProvider: React.FC<EverProviderProps>;

package/dist/components/GarmentModel.d.ts

@@ -0,0 +1,10 @@
+import React from "react";
+import * as THREE from "three";
+export interface GarmentModelProps {
+    url: string | null;
+    targetAvatarBones?: Record<string, THREE.Bone>;
+}
+/**
+ * Loads the clothing GLB and binds its SkinnedMeshes to the target avatar's bones.
+ */
+export declare const GarmentModel: React.FC<GarmentModelProps>;

package/dist/hooks/useBodyMeasurer.d.ts

@@ -0,0 +1,11 @@
+/**
+ * Extracts silhouette, calculates body dimensions periodically (e.g. 1Hz).
+ * Evaluates clothing fit.
+ */
+export declare const useBodyMeasurer: () => {
+    dimensions: {
+        shoulder: number;
+        chest: number;
+        waist: number;
+    } | null;
+};

package/dist/hooks/useEverGeneration.d.ts

@@ -0,0 +1,17 @@
+import { type BodyPresetMatchRequest, type GarmentData } from "../api/EverClient";
+export type GenerationStatus = "idle" | "uploading" | "processing" | "ready" | "error";
+/**
+ * Coordinates backend-backed model acquisition for the SDK demo flows.
+ *
+ * `generateAvatar` keeps the legacy call signature for compatibility, but the
+ * backend currently exposes body preset matching rather than photo-based avatar
+ * synthesis. Blob input therefore resolves to a default preset match.
+ */
+export declare function useEverGeneration(baseUrl?: string): {
+    status: GenerationStatus;
+    resultUrl: string | null;
+    error: string | null;
+    generateAvatar: (input?: Blob | BodyPresetMatchRequest) => Promise<string>;
+    createGarment: (data: GarmentData, frontImage: Blob, backImage?: Blob, dimensions?: Record<string, unknown>) => Promise<string>;
+    reset: () => void;
+};

package/dist/hooks/usePoseMapper.d.ts

@@ -0,0 +1,7 @@
+import * as THREE from "three";
+/**
+ * Custom hook to map MediaPipe 2D/3D landmarks to 3D bones (Quaternions).
+ */
+export declare const usePoseMapper: () => {
+    mapPoseToBones: (landmarks: any[], targetBones: Record<string, THREE.Bone>) => void;
+};

package/dist/hooks/usePoseMapper.test.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/hooks/usePoseTracker.d.ts

@@ -0,0 +1,7 @@
+/**
+ * Initializes MediaPipe Pose Landmarker (Lite).
+ * Connects to the user's webcam feed and outputs real-time skeletal landmarks to the store.
+ */
+export declare const usePoseTracker: (videoRef: React.RefObject<HTMLVideoElement | null>) => {
+    isTrackerReady: boolean;
+};

package/dist/index.d.ts

@@ -1 +1,11 @@
-export {}
+export * from "./store/useEverStore";
+export * from "./components/EverProvider";
+export * from "./components/EverCanvas";
+export * from "./components/CameraView";
+export * from "./components/AvatarModel";
+export * from "./components/GarmentModel";
+export * from "./hooks/usePoseTracker";
+export * from "./hooks/usePoseMapper";
+export * from "./hooks/useBodyMeasurer";
+export * from "./api/EverClient";
+export * from "./hooks/useEverGeneration";