neuromeka-vfm 0.1.2__py3-none-any.whl → 0.1.4__py3-none-any.whl

neuromeka_vfm/segmentation.py

@@ -26,6 +26,25 @@ class Segmentation:
             self.call_time = {"add_image_prompt": 0, "register_first_frame": 0, "get_next": 0}
             self.call_count = {"add_image_prompt": 0, "register_first_frame": 0, "get_next": 0}
 
+    def _is_success(self, response):
+        """
+        Normalize server success flag.
+        Some servers return {"result": "SUCCESS"}, others {"success": true}, and
+        the segmentation server returns {"status": "success"}.
+        """
+        # Try known keys in order of common usage.
+        for key in ("result", "success", "status"):
+            flag = response.get(key)
+            if flag is None:
+                continue
+            if isinstance(flag, str):
+                return flag.lower() == "success"
+            if isinstance(flag, bool):
+                return flag
+            return bool(flag)
+        # Fallback: anything truthy counts as success.
+        return bool(response)
+
     def switch_compression_strategy(self, compression_strategy):
         if compression_strategy in STRATEGIES:
             self.compression_strategy_name = compression_strategy
@@ -46,7 +65,7 @@ class Segmentation:
             start = time.time()
         data = {"operation": "add_image_prompt", "object_name": object_name, "object_image": object_image}
         response = self.client.send_data(data)
-        if response.get("result") == "SUCCESS":
+        if self._is_success(response):
             self.image_prompt_names.add(object_name)
         if self.benchmark:
             self.call_time["add_image_prompt"] += time.time() - start
@@ -72,7 +91,7 @@ class Segmentation:
             "compression_strategy": self.compression_strategy_name,
         }
         response = self.client.send_data(data)
-        if response.get("result") == "SUCCESS":
+        if self._is_success(response):
             self.first_frame_registered = True
             self.tracking_object_ids = response["data"]["obj_ids"]
             masks = {}
@@ -98,7 +117,7 @@ class Segmentation:
         if self.benchmark:
             start = time.time()
         response = self.client.send_data({"operation": "get_next", "frame": self.compression_strategy.encode(frame)})
-        if response.get("result") == "SUCCESS":
+        if self._is_success(response):
             masks = {}
             for i, obj_id in enumerate(self.tracking_object_ids):
                 mask = self.compression_strategy.decode(response["data"]["masks"][i])

neuromeka_vfm-0.1.4.dist-info/METADATA

@@ -0,0 +1,186 @@
+Metadata-Version: 2.4
+Name: neuromeka_vfm
+Version: 0.1.4
+Summary: Client utilities for Neuromeka VFM FoundationPose RPC (upload meshes, call server)
+Author: Neuromeka
+License: MIT License
+        
+        Copyright (c) 2025 Neuromeka Co., Ltd.
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: pyzmq
+Requires-Dist: paramiko
+Requires-Dist: av
+Dynamic: license-file
+
+# neuromeka_vfm
+
+클라이언트 PC에서 Segmentation (SAM2, Grounding DINO), Pose Estimation (NVIDIA FoundationPose) 서버(RPC, ZeroMQ)와 통신하고, SSH/SFTP로 호스트에 mesh를 업로드하는 간단한 유틸 패키지입니다.
+
+- Website: http://www.neuromeka.com
+- Source code: https://github.com/neuromeka-robotics/neuromeka_vfm
+- PyPI package: https://pypi.org/project/neuromeka_vfm/
+- Documents: https://docs.neuromeka.com
+
+## Web UI (VFM Tester)를 통해 사용 가능
+
+- VFM Tester (Web UI): https://gitlab.com/neuromeka-group/nrmkq/nrmk_vfm_tester
+
+
+## Installation
+```bash
+pip install neuromeka_vfm
+```
+
+## Python API (예제로 보는 사용법)
+
+* 내 PC: 어플리케이션을 구현하고 이 패키지 (neuromeka_vfm)이 설치된 PC
+* 서버PC (Host): Segmentation, Pose Estimation 도커 서버가 설치된 PC. 내 PC에 도커를 설치할 경우 localhost 사용.
+
+### Segmentation
+```python
+from neuromeka_vfm import Segmentation
+
+seg = Segmentation(
+    hostname="192.168.10.63", 
+    port=5432,
+    compression_strategy="png",    # none | png | jpeg | h264
+)
+
+# Image Prompt를 이용한 등록
+seg.add_image_prompt("drug_box", ref_rgb)
+seg.register_first_frame(frame=first_rgb, 
+                        prompt="drug_box", # ID str
+                        use_image_prompt=True)
+
+# Text Prompt를 이용한 등록
+seg.register_first_frame(frame=first_rgb, 
+                        prompt="box .",   # Text prompt (끝에 띄어쓰기 . 필수)
+                        use_image_prompt=False)
+
+# 등록된 mask에 대한 SAM2 tracking
+masks = seg.get_next(next_rgb)
+
+
+seg.close()
+```
+
+### Pose Estimation
+
+**Mesh 파일 업로드**: 등록/인식하고자 하는 mesh 파일 (stl)을 호스트PC의 '/opt/meshes/' 경로에 업로드 (직접 SSH 통해 파일을 옮겨도 됨)
+```python
+from neuromeka_vfm import upload_mesh
+upload_mesh(
+    host="192.168.10.63",
+    user="user",
+    password="pass",                  
+    local="mesh/my_mesh.stl",         # 내 PC mesh 경로
+    remote="/opt/meshes/my_mesh.stl", # 호스트PC mesh 경로 (도커 볼륨마운트)
+)
+```
+
+초기화
+```python
+from neuromeka_vfm import PoseEstimation
+
+pose = PoseEstimation(host="192.168.10.72", port=5557)  
+
+pose.init(
+    mesh_path="/app/modules/foundation_pose/mesh/my_mesh.stl",
+    apply_scale=1.0,  
+    track_refine_iter=3,
+    min_n_views=40,
+    inplane_step=60
+)
+```
+- mesh_path: 사용할 물체 메시 파일(STL/OBJ 등) 경로. 없으면 초기화 실패.
+- apply_scale: 메시를 로드한 뒤 전체를 배율 조정하는 스케일 값. 단위 없는 곱셈 계수.
+    - STL 모델이 미터 단위라면 1.0 (스케일 없음)
+    - STL 모델이 센티미터 단위라면 0.01 (1 cm → 0.01 m)
+    - STL 모델이 밀리미터 단위라면 0.001 (1 mm → 0.001 m)
+- force_apply_color: True일 때 메시에 단색 텍스처를 강제로 입힘. 메시가 색상을 안 가졌을 때 시각화 안정성을 위해 사용.
+- apply_color: force_apply_color가 True일 때 적용할 RGB 색상값(0~255) 튜플.
+- est_refine_iter: 초기 등록(register) 단계에서 포즈를 반복 정련하는 횟수. 값이 클수록 정확도 ↑, 연산 시간 ↑.
+- track_refine_iter: 추적(track) 단계에서 한 프레임당 포즈 정련 반복 횟수.
+- min_n_views: 초기 뷰 샘플링 시 생성할 최소 카메라 뷰 수(회전 후보 수에 영향).
+- inplane_step: in-plane 회전 샘플링 간격(도 단위). 값이 작을수록 더 많은 회전 후보를 생성.
+
+
+인식 및 추적
+```python
+# 초기 등록 (iteration 생략 시 서버 기본값, check_vram=True로 VRAM 사전 체크)
+register_resp = pose.register(rgb=rgb0, depth=depth0, mask=mask0, K=cam_K, check_vram=True)
+
+# 추적 (bbox_xywh로 탐색 범위 제한 가능)
+track_resp = pose.track(rgb=rgb1, depth=depth1, K=cam_K, bbox_xywh=bbox_xywh)
+pose.close()
+```
+- cam_K: camera intrinsic
+- RGB resolution이 크거나, min_n_views 값이 크거나, inplane_step이 작을 경우 GPU VRAM 초과 에러 발생. 
+- register check_vram=True 일 경우 VRAM 초과 사전 검사하여 shutdown 방지.
+
+
+## VFM (Vision Foundation Model) latency benchmark
+로컬 서버 구동 시 측정. 빈칸은 아직 미측정 항목입니다. 
+
+**RTX 5060**
+| Task | Prompt | None (s) | JPEG (s) | PNG (s) | h264 (s) |
+| --- | --- | --- | --- | --- | --- |
+| Grounding DINO | text (human . cup .) | 0.86 | 0.35 | 0.50 | 0.52 |
+| DINOv2 | image prompt | 0.85 | 0.49 | 0.65 | 0.63 |
+| SAM2 | - |  |  |  |  |
+| FoundationPose registration | - |  |  |  |  |
+| FoundationPose track | - |  |  |  |  |
+
+**RTX 5090**
+| Task | Prompt | None (s) | JPEG (s) | PNG (s) | h264 (s) |
+| --- | --- | --- | --- | --- | --- |
+| Grounding DINO | text (human . cup .) |  |  |  |  |
+| DINOv2 | image prompt |  |  |  |  |
+| SAM2 | - |  |  |  |  |
+| FoundationPose registration | - | 0.4 | - |  |  |
+| FoundationPose track | - | 0.03 |  |  |  |
+
+**Jetson Orin**
+| Task | Prompt | None (s) | JPEG (s) | PNG (s) | h264 (s) |
+| --- | --- | --- | --- | --- | --- |
+| Grounding DINO | text (human . cup .) |  |  |  |  |
+| DINOv2 | image prompt |  |  |  |  |
+| SAM2 | - |  |  |  |  |
+| FoundationPose registration | - | 0.4 | - |  |  |
+| FoundationPose track | - | 0.03 |  |  |  |
+
+## 릴리스 노트
+- 0.1.2: Segmentation 응답 성공 판정 개선(`result`/`success`/`status` 모두 지원), image prompt 등록/사용 오류 수정, PoseEstimation `register`에 `check_vram` 옵션 반영.
+- 0.1.1: PoseEstimation/Segmentation에서 리소스 정리 개선, iteration 미전달 시 서버 기본값 사용, pose 데모 예제 추가.
+- 0.1.0: 초기 공개 버전. FoundationPose RPC 클라이언트, 실시간 세그멘테이션 클라이언트, SSH 기반 mesh 업로드 CLI/API 포함.

{neuromeka_vfm-0.1.2.dist-info → neuromeka_vfm-0.1.4.dist-info}/RECORD

@@ -2,13 +2,13 @@ neuromeka_vfm/__init__.py,sha256=h5ODdWFgN7a9TBzF6Qfdyx5VxUr2hG0pFTwq57jEvDo,422
 neuromeka_vfm/compression.py,sha256=d2xOz4XBJZ60pPSXwQ5LPYwhpsaNORvNoY_0CUiAvt0,5191
 neuromeka_vfm/pickle_client.py,sha256=Iw2fpxdnKB20oEUgsd0rJlvzOd5JhetphpKkF9qQcX0,591
 neuromeka_vfm/pose_estimation.py,sha256=3MUVhL0nMcpHApZDAzutS7fINPHcb-tu_WoXvNGU33E,2625
-neuromeka_vfm/segmentation.py,sha256=PKOJiNoZz8tlwSX-eAgvqA2YWKkcCkKcoQfaiIEqCdU,5630
+neuromeka_vfm/segmentation.py,sha256=wae0_m225DUMD8Nm2A7iQm49QWeUas17B8PaoGGFt5w,6311
 neuromeka_vfm/upload_mesh.py,sha256=aW5G9aE5OeiDN5pEVKDzMeV538U-I2iRYZvVZTfGsr4,2728
 neuromeka_vfm/examples/__init__.py,sha256=dEhb0FqhpEGNmg0pMunmrTlViIcxvd95fYEjZ49IOTQ,37
 neuromeka_vfm/examples/pose_demo.py,sha256=zq1Z0_kxQc4CB-ltfwm_oMoC7JLoN5GyeE3C6jKGQKw,13658
-neuromeka_vfm-0.1.2.dist-info/licenses/LICENSE,sha256=40cBWxFahhu0p_EB0GhU8oVIifVNmH1o2fZtx0bIif8,1076
-neuromeka_vfm-0.1.2.dist-info/METADATA,sha256=MxiWRP0T5iZzHMsHqyIXSX9JZQF0Kv2dGBhzyOT6EMc,7372
-neuromeka_vfm-0.1.2.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
-neuromeka_vfm-0.1.2.dist-info/entry_points.txt,sha256=Wl4XqiUt_GLQ08oTJtsYjLW0iYxZ52ysVd1-cN0kYP4,72
-neuromeka_vfm-0.1.2.dist-info/top_level.txt,sha256=uAH_yXikUvxXTSEnUC0M8Zl5ggxbnkYtXlmTfEG8MUk,14
-neuromeka_vfm-0.1.2.dist-info/RECORD,,
+neuromeka_vfm-0.1.4.dist-info/licenses/LICENSE,sha256=40cBWxFahhu0p_EB0GhU8oVIifVNmH1o2fZtx0bIif8,1076
+neuromeka_vfm-0.1.4.dist-info/METADATA,sha256=EPZ5Y6uuSz00whHesar6bKAqrUDNpGAfquZq9Pvd24U,7829
+neuromeka_vfm-0.1.4.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+neuromeka_vfm-0.1.4.dist-info/entry_points.txt,sha256=Wl4XqiUt_GLQ08oTJtsYjLW0iYxZ52ysVd1-cN0kYP4,72
+neuromeka_vfm-0.1.4.dist-info/top_level.txt,sha256=uAH_yXikUvxXTSEnUC0M8Zl5ggxbnkYtXlmTfEG8MUk,14
+neuromeka_vfm-0.1.4.dist-info/RECORD,,

neuromeka_vfm-0.1.2.dist-info/METADATA

@@ -1,159 +0,0 @@
-Metadata-Version: 2.4
-Name: neuromeka_vfm
-Version: 0.1.2
-Summary: Client utilities for Neuromeka VFM FoundationPose RPC (upload meshes, call server)
-Author: Neuromeka
-License: MIT License
-        
-        Copyright (c) 2025 Neuromeka Co., Ltd.
-        
-        Permission is hereby granted, free of charge, to any person obtaining a copy
-        of this software and associated documentation files (the "Software"), to deal
-        in the Software without restriction, including without limitation the rights
-        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-        copies of the Software, and to permit persons to whom the Software is
-        furnished to do so, subject to the following conditions:
-        
-        The above copyright notice and this permission notice shall be included in all
-        copies or substantial portions of the Software.
-        
-        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-        SOFTWARE.
-        
-Classifier: Development Status :: 3 - Alpha
-Classifier: Intended Audience :: Developers
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Requires-Python: >=3.8
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: numpy
-Requires-Dist: pyzmq
-Requires-Dist: paramiko
-Requires-Dist: av
-Requires-Dist: opencv-python-headless
-Dynamic: license-file
-
-# neuromeka_vfm
-
-클라이언트 PC에서 Segmentation (SAM2, Grounding DINO), Pose Estimation (NVIDIA FoundationPose) 서버(RPC, ZeroMQ)와 통신하고, SSH/SFTP로 호스트에 mesh를 업로드하는 간단한 유틸 패키지입니다.
-
-- Website: http://www.neuromeka.com
-- Source code: https://github.com/neuromeka-robotics/neuromeka_vfm
-- PyPI package: https://pypi.org/project/neuromeka_vfm/
-- Documents: https://docs.neuromeka.com
-
-## VFM (Vision Foundation Model) latency benchmark
-로컬 서버 구동 시 측정. 빈칸은 아직 미측정 항목입니다. 
-
-**RTX 5060**
-| Task | Prompt | None (s) | JPEG (s) | PNG (s) | h264 (s) |
-| --- | --- | --- | --- | --- | --- |
-| Grounding DINO | text (human . cup .) | 0.86 | 0.35 | 0.50 | 0.52 |
-| DINOv2 | image prompt | 0.85 | 0.49 | 0.65 | 0.63 |
-| SAM2 | - |  |  |  |  |
-| FoundationPose registration | - |  |  |  |  |
-| FoundationPose track | - |  |  |  |  |
-
-**RTX 5090**
-| Task | Prompt | None (s) | JPEG (s) | PNG (s) | h264 (s) |
-| --- | --- | --- | --- | --- | --- |
-| Grounding DINO | text (human . cup .) |  |  |  |  |
-| DINOv2 | image prompt |  |  |  |  |
-| SAM2 | - |  |  |  |  |
-| FoundationPose registration | - |  |  |  |  |
-| FoundationPose track | - |  |  |  |  |
-
-
-## Installation
-```bash
-pip install neuromeka_vfm
-```
-
-## 사용 예
-### Python API
-```python
-from neuromeka_vfm import PoseEstimation, upload_mesh
-# (옵션) Realtime segmentation client도 포함됩니다.
-
-# 1) 서버로 mesh 업로드 (호스트 경로는 컨테이너에 -v로 마운트된 곳)
-upload_mesh(
-    host="192.168.10.72",
-    user="user",
-    password="pass",          # 또는 key="~/.ssh/id_rsa"
-    local="mesh/123.stl",
-    remote="/home/user/meshes/123.stl",
-)
-
-# 2) PoseEstimation 클라이언트
-pose = PoseEstimation(host="192.168.10.72", port=5557)
-pose.init(mesh_path="/app/modules/foundation_pose/mesh/123.stl")
-# ...
-pose.close()
-
-# 3) Realtime segmentation client (예)
-from neuromeka_vfm import Segmentation
-seg = Segmentation(
-    hostname="192.168.10.63",
-    port=5432,                     # 해당 도커/서버 포트
-    compression_strategy="png",    # none | png | jpeg | h264
-    benchmark=False,
-)
-# seg.register_first_frame(...)
-# seg.get_next(...)
-# seg.reset()
-# seg.finish()
-```
-
-## 주의
-- `remote`는 **호스트** 경로입니다. 컨테이너 실행 시 `-v /home/user/meshes:/app/modules/foundation_pose/mesh`처럼 마운트하면, 업로드 직후 컨테이너에서 접근 가능합니다.
-- RPC 포트(기본 5557)는 서버가 `-p 5557:5557`으로 노출되어 있어야 합니다.
-
-
-
-## API 레퍼런스 (Python)
-
-### PoseEstimation (FoundationPose RPC)
-- `PoseEstimation(host=None, port=None)`
-    - `host`: FoundationPose 도커 서버가 구동 중인 PC의 IP.
-    - `port`: 5557
-- `init(mesh_path, apply_scale=1.0, force_apply_color=False, apply_color=(160,160,160), est_refine_iter=10, track_refine_iter=3, min_n_views=40, inplane_step=60)`: 서버에 메쉬 등록 및 초기화.
-- `register(rgb, depth, mask, K, iteration=None, check_vram=True)`: 첫 프레임 등록. `iteration`을 생략하면 서버 기본 반복 횟수를 사용하며, `check_vram=False`로 두면 GPU 메모리 사전 체크를 건너뜁니다.
-- `track(rgb, depth, K, iteration=None, bbox_xywh=None)`: 추적/갱신. `bbox_xywh` 제공 시 해당 영역으로 탐색 범위를 좁힙니다.
-- `reset()`: 세션 리셋.
-- `reset_object()`: 캐시된 메쉬로 서버 측 `reset_object` 재호출.
-- `close()`: ZeroMQ 소켓/컨텍스트 정리 (사용 후 필수 호출 권장).
-
-### Segmentation (실시간 SAM2/GroundingDINO)
-- `Segmentation(hostname, port, compression_strategy="none", benchmark=False)`:
-    - `compression_strategy`: `none|png|jpeg|h264`
-    - `hostname`: 세그멘테이션 도커 서버가 구동 중인 PC의 IP.
-- `add_image_prompt(object_name, object_image)`: 이미지 프롬프트 등록.
-- `register_first_frame(frame, prompt, use_image_prompt=False) -> bool`: 첫 프레임 등록, 성공 시 `True` 반환. `use_image_prompt=True`면 모든 이름을 사전에 `add_image_prompt`로 등록해야 합니다(누락 시 `ValueError`).
-- `get_next(frame) -> dict[obj_id, mask] | None`: 다음 프레임 세그멘테이션/트래킹 결과.
-- `switch_compression_strategy(compression_strategy)`: 런타임 압축 방식 교체.
-- `reset()`: 내부 상태 및 벤치마크 타이머 리셋.
-- `finish()`: 로컬 상태 초기화.
-- `close()`: ZeroMQ 소켓/컨텍스트 정리 (사용 후 필수 호출 권장).
-
-### 업로드 CLI/API
-- `upload_mesh(host, user, port=22, password=None, key=None, local=None, remote=None)`: SSH/SFTP로 메쉬 전송, 비밀번호 또는 키 중 하나 필수.
-- CLI: `neuromeka-upload-mesh --host ... --user ... (--password ... | --key ...) --local ... --remote ...`
-
-### 예제
-- 실시간 Pose + Segmentation 데모: `python -m neuromeka_vfm.examples.pose_demo` (RealSense 필요, 서버 실행 상태에서 사용).
-
-
-## 릴리스 노트
-- 0.1.1: PoseEstimation/Segmentation에서 리소스 정리 개선, iteration 미전달 시 서버 기본값 사용, pose 데모 예제 추가.
-- 0.1.0: 초기 공개 버전. FoundationPose RPC 클라이언트, 실시간 세그멘테이션 클라이언트, SSH 기반 mesh 업로드 CLI/API 포함.