neuromeka-vfm 0.1.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2025 Neuromeka Co., Ltd.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,109 @@
1
+ Metadata-Version: 2.1
2
+ Name: neuromeka_vfm
3
+ Version: 0.1.1
4
+ Summary: Client utilities for Neuromeka VFM FoundationPose RPC (upload meshes, call server)
5
+ Author: Neuromeka
6
+ License: MIT License
7
+
8
+ Copyright (c) 2025 Neuromeka Co., Ltd.
9
+
10
+ Permission is hereby granted, free of charge, to any person obtaining a copy
11
+ of this software and associated documentation files (the "Software"), to deal
12
+ in the Software without restriction, including without limitation the rights
13
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
14
+ copies of the Software, and to permit persons to whom the Software is
15
+ furnished to do so, subject to the following conditions:
16
+
17
+ The above copyright notice and this permission notice shall be included in all
18
+ copies or substantial portions of the Software.
19
+
20
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
21
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
22
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
23
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
24
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
25
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
26
+ SOFTWARE.
27
+
28
+ Classifier: Development Status :: 3 - Alpha
29
+ Classifier: Intended Audience :: Developers
30
+ Classifier: License :: OSI Approved :: MIT License
31
+ Classifier: Programming Language :: Python :: 3
32
+ Classifier: Programming Language :: Python :: 3.8
33
+ Classifier: Programming Language :: Python :: 3.9
34
+ Classifier: Programming Language :: Python :: 3.10
35
+ Classifier: Programming Language :: Python :: 3.11
36
+ Classifier: Programming Language :: Python :: 3.12
37
+ Requires-Python: >=3.8
38
+ Description-Content-Type: text/markdown
39
+ License-File: LICENSE
40
+ Requires-Dist: numpy
41
+ Requires-Dist: pyzmq
42
+ Requires-Dist: paramiko
43
+ Requires-Dist: av
44
+ Requires-Dist: opencv-python-headless
45
+
46
+ # neuromeka_vfm
47
+
48
+ 클라이언트 PC에서 FoundationPose 서버(RPC, ZeroMQ)와 통신하고, SSH/SFTP로 호스트에 mesh를 업로드하는 간단한 유틸 패키지입니다.
49
+
50
+ ## 설치
51
+ ```bash
52
+ pip install neuromeka_vfm
53
+ ```
54
+
55
+ ### 로컬 개발
56
+ ```bash
57
+ pip install -e .
58
+ ```
59
+
60
+ ## 사용 예
61
+ ### Python API
62
+ ```python
63
+ from neuromeka_vfm import PoseEstimation, upload_mesh
64
+ # (옵션) Realtime segmentation client도 포함됩니다.
65
+
66
+ # 1) 서버로 mesh 업로드 (호스트 경로는 컨테이너에 -v로 마운트된 곳)
67
+ upload_mesh(
68
+ host="192.168.10.72",
69
+ user="user",
70
+ password="pass", # 또는 key="~/.ssh/id_rsa"
71
+ local="mesh/123.stl",
72
+ remote="/home/user/meshes/123.stl",
73
+ )
74
+
75
+ # 2) PoseEstimation 클라이언트
76
+ pose = PoseEstimation(host="192.168.10.72", port=5557)
77
+ pose.init(mesh_path="/app/modules/foundation_pose/mesh/123.stl")
78
+ # ...
79
+ pose.close()
80
+
81
+ # 3) Realtime segmentation client (예)
82
+ from neuromeka_vfm import Segmentation
83
+ seg = Segmentation(
84
+ hostname="192.168.10.72",
85
+ port=5432, # 해당 도커/서버 포트
86
+ compression_strategy="png", # none | png | jpeg | h264
87
+ benchmark=False,
88
+ )
89
+ # seg.register_first_frame(...), seg.get_next(...), seg.finish(), seg.reset()
90
+ ```
91
+
92
+ ### CLI 업로드
93
+ ```bash
94
+ neuromeka-upload-mesh --host 192.168.10.72 --user user --password pass \
95
+ --local mesh/123.stl --remote /home/user/meshes/123.stl
96
+ ```
97
+
98
+ ## 주의
99
+ - `remote`는 **호스트** 경로입니다. 컨테이너 실행 시 `-v /home/user/meshes:/app/modules/foundation_pose/mesh`처럼 마운트하면, 업로드 직후 컨테이너에서 접근 가능합니다.
100
+ - RPC 포트(기본 5557)는 서버가 `-p 5557:5557`으로 노출되어 있어야 합니다.
101
+
102
+ ## 링크
103
+ - Website: http://www.neuromeka.com
104
+ - Source code: https://github.com/neuromeka-robotics/neuromeka_vfm
105
+ - PyPI package: https://pypi.org/project/neuromeka_vfm/
106
+ - Documents: https://docs.neuromeka.com
107
+
108
+ ## 릴리스 노트
109
+ - 0.1.0: 초기 공개 버전. FoundationPose RPC 클라이언트, 실시간 세그멘테이션 클라이언트, SSH 기반 mesh 업로드 CLI/API 포함.
@@ -0,0 +1,64 @@
1
+ # neuromeka_vfm
2
+
3
+ 클라이언트 PC에서 FoundationPose 서버(RPC, ZeroMQ)와 통신하고, SSH/SFTP로 호스트에 mesh를 업로드하는 간단한 유틸 패키지입니다.
4
+
5
+ ## 설치
6
+ ```bash
7
+ pip install neuromeka_vfm
8
+ ```
9
+
10
+ ### 로컬 개발
11
+ ```bash
12
+ pip install -e .
13
+ ```
14
+
15
+ ## 사용 예
16
+ ### Python API
17
+ ```python
18
+ from neuromeka_vfm import PoseEstimation, upload_mesh
19
+ # (옵션) Realtime segmentation client도 포함됩니다.
20
+
21
+ # 1) 서버로 mesh 업로드 (호스트 경로는 컨테이너에 -v로 마운트된 곳)
22
+ upload_mesh(
23
+ host="192.168.10.72",
24
+ user="user",
25
+ password="pass", # 또는 key="~/.ssh/id_rsa"
26
+ local="mesh/123.stl",
27
+ remote="/home/user/meshes/123.stl",
28
+ )
29
+
30
+ # 2) PoseEstimation 클라이언트
31
+ pose = PoseEstimation(host="192.168.10.72", port=5557)
32
+ pose.init(mesh_path="/app/modules/foundation_pose/mesh/123.stl")
33
+ # ...
34
+ pose.close()
35
+
36
+ # 3) Realtime segmentation client (예)
37
+ from neuromeka_vfm import Segmentation
38
+ seg = Segmentation(
39
+ hostname="192.168.10.72",
40
+ port=5432, # 해당 도커/서버 포트
41
+ compression_strategy="png", # none | png | jpeg | h264
42
+ benchmark=False,
43
+ )
44
+ # seg.register_first_frame(...), seg.get_next(...), seg.finish(), seg.reset()
45
+ ```
46
+
47
+ ### CLI 업로드
48
+ ```bash
49
+ neuromeka-upload-mesh --host 192.168.10.72 --user user --password pass \
50
+ --local mesh/123.stl --remote /home/user/meshes/123.stl
51
+ ```
52
+
53
+ ## 주의
54
+ - `remote`는 **호스트** 경로입니다. 컨테이너 실행 시 `-v /home/user/meshes:/app/modules/foundation_pose/mesh`처럼 마운트하면, 업로드 직후 컨테이너에서 접근 가능합니다.
55
+ - RPC 포트(기본 5557)는 서버가 `-p 5557:5557`으로 노출되어 있어야 합니다.
56
+
57
+ ## 링크
58
+ - Website: http://www.neuromeka.com
59
+ - Source code: https://github.com/neuromeka-robotics/neuromeka_vfm
60
+ - PyPI package: https://pypi.org/project/neuromeka_vfm/
61
+ - Documents: https://docs.neuromeka.com
62
+
63
+ ## 릴리스 노트
64
+ - 0.1.0: 초기 공개 버전. FoundationPose RPC 클라이언트, 실시간 세그멘테이션 클라이언트, SSH 기반 mesh 업로드 CLI/API 포함.
@@ -0,0 +1,39 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "neuromeka_vfm"
7
+ version = "0.1.1"
8
+ description = "Client utilities for Neuromeka VFM FoundationPose RPC (upload meshes, call server)"
9
+ readme = "README.md"
10
+ requires-python = ">=3.8"
11
+ license = {file = "LICENSE"}
12
+ dependencies = [
13
+ "numpy",
14
+ "pyzmq",
15
+ "paramiko",
16
+ "av",
17
+ "opencv-python-headless",
18
+ ]
19
+ authors = [{name = "Neuromeka"}]
20
+ classifiers = [
21
+ "Development Status :: 3 - Alpha",
22
+ "Intended Audience :: Developers",
23
+ "License :: OSI Approved :: MIT License",
24
+ "Programming Language :: Python :: 3",
25
+ "Programming Language :: Python :: 3.8",
26
+ "Programming Language :: Python :: 3.9",
27
+ "Programming Language :: Python :: 3.10",
28
+ "Programming Language :: Python :: 3.11",
29
+ "Programming Language :: Python :: 3.12",
30
+ ]
31
+
32
+ [project.scripts]
33
+ neuromeka-upload-mesh = "neuromeka_vfm.upload_mesh:cli"
34
+
35
+ [tool.setuptools]
36
+ package-dir = {"" = "src"}
37
+
38
+ [tool.setuptools.packages.find]
39
+ where = ["src"]
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,13 @@
1
+ from .pose_estimation import PoseEstimation, FoundationPoseClient
2
+ from .upload_mesh import upload_mesh
3
+ from .segmentation import Segmentation, NrmkRealtimeSegmentation
4
+ from .compression import STRATEGIES as SEGMENTATION_COMPRESSION_STRATEGIES
5
+
6
+ __all__ = [
7
+ "PoseEstimation",
8
+ "FoundationPoseClient",
9
+ "upload_mesh",
10
+ "Segmentation",
11
+ "NrmkRealtimeSegmentation",
12
+ "SEGMENTATION_COMPRESSION_STRATEGIES",
13
+ ]
@@ -0,0 +1,167 @@
1
+ from abc import ABC, abstractmethod
2
+ from fractions import Fraction
3
+ import av
4
+ import numpy as np
5
+ import cv2
6
+
7
+
8
+ class CompressionStrategy(ABC):
9
+ def __init__(self, first_frame):
10
+ pass
11
+
12
+ @abstractmethod
13
+ def encode(self, frame):
14
+ pass
15
+
16
+ @abstractmethod
17
+ def decode(self, payload):
18
+ pass
19
+
20
+
21
+ class NoneStrategy(CompressionStrategy):
22
+ def encode(self, frame):
23
+ return frame
24
+
25
+ def decode(self, payload):
26
+ return payload
27
+
28
+
29
+ class H264LosslessCompression(CompressionStrategy):
30
+ def __init__(self, first_frame):
31
+ super().__init__(first_frame)
32
+ self.fps = 20
33
+
34
+ def _to_rgb3_and_flag(self, arr: np.ndarray):
35
+ if arr.dtype != np.uint8:
36
+ raise ValueError("Input must be uint8.")
37
+ if arr.ndim == 3 and arr.shape[2] == 1:
38
+ arr = arr[:, :, 0]
39
+ if arr.ndim == 2:
40
+ rgb = np.repeat(arr[:, :, None], 3, axis=2)
41
+ return rgb, True
42
+ if arr.ndim == 3 and arr.shape[2] == 3:
43
+ return arr, False
44
+ raise ValueError("Array must be HxW, HxWx1, or HxWx3 (uint8).")
45
+
46
+ def encode(self, arr: np.ndarray) -> bytes:
47
+ rgb, is_mask = self._to_rgb3_and_flag(arr)
48
+ h, w, _ = rgb.shape
49
+ enc = av.CodecContext.create("libx264rgb", "w")
50
+ enc.width = w
51
+ enc.height = h
52
+ enc.time_base = Fraction(1, self.fps)
53
+ enc.pix_fmt = "rgb24"
54
+ enc.options = {
55
+ "crf": "0",
56
+ "preset": "ultrafast",
57
+ "g": "1",
58
+ "keyint": "1",
59
+ }
60
+ enc.open()
61
+ frame = av.VideoFrame.from_ndarray(rgb, format="rgb24")
62
+ packets = []
63
+ packets.extend(enc.encode(frame))
64
+ packets.extend(enc.encode(None))
65
+ if not packets:
66
+ raise RuntimeError("H.264 encoder produced no packets.")
67
+ bitstream = b"".join(bytes(p) for p in packets)
68
+ header = b"\x01" if is_mask else b"\x00"
69
+ return header + bitstream
70
+
71
+ def decode(self, data: bytes) -> np.ndarray:
72
+ if len(data) < 2:
73
+ raise ValueError("Payload too short.")
74
+ header = data[0]
75
+ bitstream = data[1:]
76
+ is_mask = (header == 0x01)
77
+ dec = av.CodecContext.create("h264", "r")
78
+ packet = av.packet.Packet(bitstream)
79
+ frames = []
80
+ frames.extend(dec.decode(packet))
81
+ frames.extend(dec.decode(None))
82
+ if not frames:
83
+ raise RuntimeError("H.264 decoder produced no frames.")
84
+ img3 = frames[0].to_ndarray(format="rgb24")
85
+ if is_mask:
86
+ return img3[..., :1]
87
+ return img3
88
+
89
+
90
+ class PNGCompression(CompressionStrategy):
91
+ def __init__(self, first_frame, compression_level=1):
92
+ self.compression_level = compression_level
93
+
94
+ def encode(self, arr):
95
+ if arr.dtype != np.uint8:
96
+ raise ValueError("Input must be uint8.")
97
+ if arr.ndim == 3 and arr.shape[2] == 1:
98
+ arr = arr[:, :, 0]
99
+ if arr.ndim == 2:
100
+ img_to_encode = arr
101
+ elif arr.ndim == 3 and arr.shape[2] == 3:
102
+ img_to_encode = arr
103
+ else:
104
+ raise ValueError("Array must be HxW, HxWx1, or HxWx3 (uint8).")
105
+ ok, buf = cv2.imencode(
106
+ ".png",
107
+ img_to_encode,
108
+ [cv2.IMWRITE_PNG_COMPRESSION, int(self.compression_level)],
109
+ )
110
+ if not ok:
111
+ raise RuntimeError("PNG encode failed.")
112
+ return buf.tobytes()
113
+
114
+ def decode(self, data):
115
+ buf = np.frombuffer(data, dtype=np.uint8)
116
+ img = cv2.imdecode(buf, cv2.IMREAD_UNCHANGED)
117
+ if img is None:
118
+ raise RuntimeError("PNG decode failed.")
119
+ if img.ndim == 2:
120
+ return img[..., np.newaxis]
121
+ if img.ndim == 3 and img.shape[2] == 3:
122
+ return img
123
+ raise ValueError(f"Unexpected PNG shape: {img.shape}")
124
+
125
+
126
+ class JPEGCompression(CompressionStrategy):
127
+ def __init__(self, first_frame, quality=95):
128
+ self.quality = quality
129
+
130
+ def encode(self, arr):
131
+ if arr.dtype != np.uint8:
132
+ raise ValueError("Input must be uint8.")
133
+ if arr.ndim == 3 and arr.shape[2] == 1:
134
+ arr = arr[:, :, 0]
135
+ if arr.ndim == 2:
136
+ img_to_encode = arr
137
+ elif arr.ndim == 3 and arr.shape[2] == 3:
138
+ img_to_encode = arr
139
+ else:
140
+ raise ValueError("Array must be HxW, HxWx1, or HxWx3 (uint8).")
141
+ ok, buf = cv2.imencode(
142
+ ".jpg",
143
+ img_to_encode,
144
+ [cv2.IMWRITE_JPEG_QUALITY, int(self.quality)],
145
+ )
146
+ if not ok:
147
+ raise RuntimeError("JPEG encode failed.")
148
+ return buf.tobytes()
149
+
150
+ def decode(self, data):
151
+ buf = np.frombuffer(data, dtype=np.uint8)
152
+ img = cv2.imdecode(buf, cv2.IMREAD_UNCHANGED)
153
+ if img is None:
154
+ raise RuntimeError("JPEG decode failed.")
155
+ if img.ndim == 2:
156
+ return img[..., np.newaxis]
157
+ if img.ndim == 3 and img.shape[2] == 3:
158
+ return img
159
+ raise ValueError(f"Unexpected JPEG shape: {img.shape}")
160
+
161
+
162
+ STRATEGIES = {
163
+ "none": NoneStrategy,
164
+ "h264": H264LosslessCompression,
165
+ "png": PNGCompression,
166
+ "jpeg": JPEGCompression,
167
+ }
@@ -0,0 +1,22 @@
1
+ import pickle
2
+ import zmq
3
+
4
+
5
+ class PickleClient:
6
+ """Minimal ZeroMQ pickle-based RPC client."""
7
+
8
+ def __init__(self, hostname: str, port: int):
9
+ self.hostname = hostname
10
+ self.port = port
11
+ self.context = zmq.Context()
12
+ self.socket = self.context.socket(zmq.REQ)
13
+ self.socket.connect(f"tcp://{self.hostname}:{self.port}")
14
+
15
+ def send_data(self, data):
16
+ self.socket.send(pickle.dumps(data))
17
+ response = pickle.loads(self.socket.recv())
18
+ return response
19
+
20
+ def close(self):
21
+ self.socket.close()
22
+ self.context.term()
@@ -0,0 +1,80 @@
1
+ from typing import Optional, Sequence
2
+ import numpy as np
3
+
4
+ from .pickle_client import PickleClient
5
+
6
+
7
+ class PoseEstimation:
8
+ """
9
+ Client for FoundationPose pickle RPC server.
10
+ """
11
+
12
+ def __init__(self, host: Optional[str] = None, port: Optional[int] = None):
13
+ import os
14
+
15
+ host = host or os.environ.get("FPOSE_HOST", "localhost")
16
+ port = port or int(os.environ.get("FPOSE_PORT", "5557"))
17
+ self.client = PickleClient(host, port)
18
+
19
+ def init(
20
+ self,
21
+ mesh_path: str,
22
+ apply_scale: float = 1.0,
23
+ force_apply_color: bool = False,
24
+ apply_color: Sequence[float] = (160, 160, 160),
25
+ est_refine_iter: int = 10,
26
+ track_refine_iter: int = 3,
27
+ min_n_views: int = 40,
28
+ inplane_step: int = 60,
29
+ ):
30
+ return self.client.send_data(
31
+ {
32
+ "operation": "init",
33
+ "mesh_path": mesh_path,
34
+ "apply_scale": apply_scale,
35
+ "force_apply_color": force_apply_color,
36
+ "apply_color": list(apply_color),
37
+ "est_refine_iter": est_refine_iter,
38
+ "track_refine_iter": track_refine_iter,
39
+ "min_n_views": min_n_views,
40
+ "inplane_step": inplane_step,
41
+ }
42
+ )
43
+
44
+ def register(self, rgb: np.ndarray, depth: np.ndarray, mask: np.ndarray, K: np.ndarray, iteration: int = None):
45
+ return self.client.send_data(
46
+ {
47
+ "operation": "register",
48
+ "rgb": rgb,
49
+ "depth": depth,
50
+ "mask": mask,
51
+ "K": K,
52
+ "iteration": iteration,
53
+ }
54
+ )
55
+
56
+ def track(self, rgb: np.ndarray, depth: np.ndarray, K: np.ndarray, iteration: int = None, bbox_xywh=None):
57
+ return self.client.send_data(
58
+ {
59
+ "operation": "track",
60
+ "rgb": rgb,
61
+ "depth": depth,
62
+ "K": K,
63
+ "iteration": iteration,
64
+ "bbox_xywh": bbox_xywh,
65
+ }
66
+ )
67
+
68
+ def reset(self):
69
+ return self.client.send_data({"operation": "reset"})
70
+
71
+ def reset_object(self):
72
+ """Re-run server-side reset_object/make_rotation_grid with cached mesh from init."""
73
+ return self.client.send_data({"operation": "reset_object"})
74
+
75
+ def close(self):
76
+ self.client.close()
77
+
78
+
79
+ # Backward-compat alias
80
+ FoundationPoseClient = PoseEstimation
@@ -0,0 +1,116 @@
1
+ import time
2
+ from typing import Union, List
3
+ import numpy as np
4
+
5
+ from .pickle_client import PickleClient
6
+ from .compression import STRATEGIES
7
+
8
+
9
+ class Segmentation:
10
+ """
11
+ Client for realtime segmentation/tracking server (ZeroMQ pickle RPC).
12
+ """
13
+
14
+ def __init__(self, hostname, port, compression_strategy="none", benchmark=False):
15
+ self.first_frame_registered = False
16
+ self.client = PickleClient(hostname, port)
17
+ self.tracking_object_ids = []
18
+ self.current_frame_masks = {}
19
+ if compression_strategy in STRATEGIES:
20
+ self.compression_strategy_name = compression_strategy
21
+ else:
22
+ raise ValueError(f"Only valid compression strategies are {list(STRATEGIES.keys())}")
23
+ self.benchmark = benchmark
24
+ if self.benchmark:
25
+ self.call_time = {"add_image_prompt": 0, "register_first_frame": 0, "get_next": 0}
26
+ self.call_count = {"add_image_prompt": 0, "register_first_frame": 0, "get_next": 0}
27
+
28
+ def switch_compression_strategy(self, compression_strategy):
29
+ if compression_strategy in STRATEGIES:
30
+ self.compression_strategy_name = compression_strategy
31
+ else:
32
+ raise ValueError(f"Only valid compression strategies are {list(STRATEGIES.keys())}")
33
+
34
+ def reset(self):
35
+ self.first_frame_registered = False
36
+ self.tracking_object_ids = []
37
+ self.current_frame_masks = {}
38
+ self.encoder = None
39
+ if self.benchmark:
40
+ self.call_time = {"add_image_prompt": 0, "register_first_frame": 0, "get_next": 0}
41
+ self.call_count = {"add_image_prompt": 0, "register_first_frame": 0, "get_next": 0}
42
+
43
+ def add_image_prompt(self, object_name, object_image):
44
+ if self.benchmark:
45
+ start = time.time()
46
+ data = {"operation": "add_image_prompt", "object_name": object_name, "object_image": object_image}
47
+ response = self.client.send_data(data)
48
+ if self.benchmark:
49
+ self.call_time["add_image_prompt"] += time.time() - start
50
+ self.call_count["add_image_prompt"] += 1
51
+ return response
52
+
53
+ def register_first_frame(self, frame: np.ndarray, prompt: Union[str, List[str]], use_image_prompt: bool = False):
54
+ if self.benchmark:
55
+ start = time.time()
56
+ self.compression_strategy = STRATEGIES[self.compression_strategy_name](frame)
57
+ data = {
58
+ "operation": "start",
59
+ "prompt": prompt,
60
+ "frame": self.compression_strategy.encode(frame),
61
+ "use_image_prompt": use_image_prompt,
62
+ "compression_strategy": self.compression_strategy_name,
63
+ }
64
+ response = self.client.send_data(data)
65
+ if response.get("result") == "SUCCESS":
66
+ self.first_frame_registered = True
67
+ self.tracking_object_ids = response["data"]["obj_ids"]
68
+ masks = {}
69
+ for i, obj_id in enumerate(self.tracking_object_ids):
70
+ mask = self.compression_strategy.decode(response["data"]["masks"][i])
71
+ if np.any(mask):
72
+ masks[obj_id] = mask
73
+ self.current_frame_masks = masks
74
+ if self.benchmark:
75
+ self.call_time["register_first_frame"] += time.time() - start
76
+ self.call_count["register_first_frame"] += 1
77
+ return True
78
+ else:
79
+ if self.benchmark:
80
+ self.call_time["register_first_frame"] += time.time() - start
81
+ self.call_count["register_first_frame"] += 1
82
+ return False
83
+
84
+ def get_next(self, frame: np.ndarray):
85
+ if not self.first_frame_registered:
86
+ print("Segmentation: register_first_frame must be called first")
87
+ return None
88
+ if self.benchmark:
89
+ start = time.time()
90
+ response = self.client.send_data({"operation": "get_next", "frame": self.compression_strategy.encode(frame)})
91
+ if response.get("result") == "SUCCESS":
92
+ masks = {}
93
+ for i, obj_id in enumerate(self.tracking_object_ids):
94
+ mask = self.compression_strategy.decode(response["data"]["masks"][i])
95
+ if np.any(mask):
96
+ masks[obj_id] = mask
97
+ self.current_frame_masks = masks
98
+ if self.benchmark:
99
+ self.call_time["get_next"] += time.time() - start
100
+ self.call_count["get_next"] += 1
101
+ return masks
102
+ if self.benchmark:
103
+ self.call_time["get_next"] += time.time() - start
104
+ self.call_count["get_next"] += 1
105
+ return None
106
+
107
+ def finish(self):
108
+ if not self.first_frame_registered:
109
+ print("Warning: Segmentation: register_first_frame must be called first")
110
+ self.first_frame_registered = False
111
+ self.tracking_object_ids = []
112
+ self.current_frame_masks = {}
113
+
114
+
115
+ # Backward-compat alias
116
+ NrmkRealtimeSegmentation = Segmentation
@@ -0,0 +1,81 @@
1
+ """
2
+ Upload a mesh file from client PC to server host via SSH/SFTP.
3
+ Expose both API (upload_mesh) and CLI (neuromeka-upload-mesh).
4
+ """
5
+ import argparse
6
+ import os
7
+ import pathlib
8
+ import sys
9
+
10
+ import paramiko
11
+
12
+
13
+ def upload_mesh(host: str, user: str, port: int = 22, password: str = None, key: str = None, local: str = None, remote: str = None):
14
+ if not local or not remote:
15
+ raise ValueError("local and remote paths are required")
16
+ if not os.path.isfile(local):
17
+ raise FileNotFoundError(f"Local file not found: {local}")
18
+ if password is None and key is None:
19
+ raise ValueError("Either password or key must be provided")
20
+
21
+ ssh = paramiko.SSHClient()
22
+ ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
23
+ try:
24
+ if key:
25
+ ssh.connect(hostname=host, port=port, username=user, key_filename=key)
26
+ else:
27
+ ssh.connect(hostname=host, port=port, username=user, password=password)
28
+ sftp = ssh.open_sftp()
29
+ remote_dir = os.path.dirname(remote)
30
+ if remote_dir:
31
+ # ensure directories exist
32
+ parts = pathlib.Path(remote_dir).parts
33
+ cur = ""
34
+ for p in parts:
35
+ cur = os.path.join(cur, p)
36
+ try:
37
+ sftp.stat(cur)
38
+ except FileNotFoundError:
39
+ sftp.mkdir(cur)
40
+ sftp.put(local, remote)
41
+ print(f"Uploaded {local} -> {remote}")
42
+ finally:
43
+ try:
44
+ sftp.close()
45
+ except Exception:
46
+ pass
47
+ ssh.close()
48
+
49
+
50
+ def _parse_args():
51
+ ap = argparse.ArgumentParser(description="Upload mesh file to host via SSH (for FoundationPose docker volume).")
52
+ ap.add_argument("--host", required=True, help="Host IP or name")
53
+ ap.add_argument("--port", type=int, default=22, help="SSH port (default: 22)")
54
+ ap.add_argument("--user", required=True, help="SSH username")
55
+ auth = ap.add_mutually_exclusive_group(required=True)
56
+ auth.add_argument("--password", help="SSH password")
57
+ auth.add_argument("--key", help="SSH private key path")
58
+ ap.add_argument("--local", required=True, help="Local mesh file path (e.g., mesh/123.stl)")
59
+ ap.add_argument("--remote", required=True, help="Remote host path (mounted into container)")
60
+ return ap.parse_args()
61
+
62
+
63
+ def cli():
64
+ args = _parse_args()
65
+ try:
66
+ upload_mesh(
67
+ host=args.host,
68
+ port=args.port,
69
+ user=args.user,
70
+ password=args.password,
71
+ key=args.key,
72
+ local=args.local,
73
+ remote=args.remote,
74
+ )
75
+ except Exception as e:
76
+ print(f"Upload failed: {e}", file=sys.stderr)
77
+ sys.exit(1)
78
+
79
+
80
+ if __name__ == "__main__":
81
+ cli()
@@ -0,0 +1,109 @@
1
+ Metadata-Version: 2.1
2
+ Name: neuromeka_vfm
3
+ Version: 0.1.1
4
+ Summary: Client utilities for Neuromeka VFM FoundationPose RPC (upload meshes, call server)
5
+ Author: Neuromeka
6
+ License: MIT License
7
+
8
+ Copyright (c) 2025 Neuromeka Co., Ltd.
9
+
10
+ Permission is hereby granted, free of charge, to any person obtaining a copy
11
+ of this software and associated documentation files (the "Software"), to deal
12
+ in the Software without restriction, including without limitation the rights
13
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
14
+ copies of the Software, and to permit persons to whom the Software is
15
+ furnished to do so, subject to the following conditions:
16
+
17
+ The above copyright notice and this permission notice shall be included in all
18
+ copies or substantial portions of the Software.
19
+
20
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
21
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
22
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
23
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
24
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
25
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
26
+ SOFTWARE.
27
+
28
+ Classifier: Development Status :: 3 - Alpha
29
+ Classifier: Intended Audience :: Developers
30
+ Classifier: License :: OSI Approved :: MIT License
31
+ Classifier: Programming Language :: Python :: 3
32
+ Classifier: Programming Language :: Python :: 3.8
33
+ Classifier: Programming Language :: Python :: 3.9
34
+ Classifier: Programming Language :: Python :: 3.10
35
+ Classifier: Programming Language :: Python :: 3.11
36
+ Classifier: Programming Language :: Python :: 3.12
37
+ Requires-Python: >=3.8
38
+ Description-Content-Type: text/markdown
39
+ License-File: LICENSE
40
+ Requires-Dist: numpy
41
+ Requires-Dist: pyzmq
42
+ Requires-Dist: paramiko
43
+ Requires-Dist: av
44
+ Requires-Dist: opencv-python-headless
45
+
46
+ # neuromeka_vfm
47
+
48
+ 클라이언트 PC에서 FoundationPose 서버(RPC, ZeroMQ)와 통신하고, SSH/SFTP로 호스트에 mesh를 업로드하는 간단한 유틸 패키지입니다.
49
+
50
+ ## 설치
51
+ ```bash
52
+ pip install neuromeka_vfm
53
+ ```
54
+
55
+ ### 로컬 개발
56
+ ```bash
57
+ pip install -e .
58
+ ```
59
+
60
+ ## 사용 예
61
+ ### Python API
62
+ ```python
63
+ from neuromeka_vfm import PoseEstimation, upload_mesh
64
+ # (옵션) Realtime segmentation client도 포함됩니다.
65
+
66
+ # 1) 서버로 mesh 업로드 (호스트 경로는 컨테이너에 -v로 마운트된 곳)
67
+ upload_mesh(
68
+ host="192.168.10.72",
69
+ user="user",
70
+ password="pass", # 또는 key="~/.ssh/id_rsa"
71
+ local="mesh/123.stl",
72
+ remote="/home/user/meshes/123.stl",
73
+ )
74
+
75
+ # 2) PoseEstimation 클라이언트
76
+ pose = PoseEstimation(host="192.168.10.72", port=5557)
77
+ pose.init(mesh_path="/app/modules/foundation_pose/mesh/123.stl")
78
+ # ...
79
+ pose.close()
80
+
81
+ # 3) Realtime segmentation client (예)
82
+ from neuromeka_vfm import Segmentation
83
+ seg = Segmentation(
84
+ hostname="192.168.10.72",
85
+ port=5432, # 해당 도커/서버 포트
86
+ compression_strategy="png", # none | png | jpeg | h264
87
+ benchmark=False,
88
+ )
89
+ # seg.register_first_frame(...), seg.get_next(...), seg.finish(), seg.reset()
90
+ ```
91
+
92
+ ### CLI 업로드
93
+ ```bash
94
+ neuromeka-upload-mesh --host 192.168.10.72 --user user --password pass \
95
+ --local mesh/123.stl --remote /home/user/meshes/123.stl
96
+ ```
97
+
98
+ ## 주의
99
+ - `remote`는 **호스트** 경로입니다. 컨테이너 실행 시 `-v /home/user/meshes:/app/modules/foundation_pose/mesh`처럼 마운트하면, 업로드 직후 컨테이너에서 접근 가능합니다.
100
+ - RPC 포트(기본 5557)는 서버가 `-p 5557:5557`으로 노출되어 있어야 합니다.
101
+
102
+ ## 링크
103
+ - Website: http://www.neuromeka.com
104
+ - Source code: https://github.com/neuromeka-robotics/neuromeka_vfm
105
+ - PyPI package: https://pypi.org/project/neuromeka_vfm/
106
+ - Documents: https://docs.neuromeka.com
107
+
108
+ ## 릴리스 노트
109
+ - 0.1.0: 초기 공개 버전. FoundationPose RPC 클라이언트, 실시간 세그멘테이션 클라이언트, SSH 기반 mesh 업로드 CLI/API 포함.
@@ -0,0 +1,15 @@
1
+ LICENSE
2
+ README.md
3
+ pyproject.toml
4
+ src/neuromeka_vfm/__init__.py
5
+ src/neuromeka_vfm/compression.py
6
+ src/neuromeka_vfm/pickle_client.py
7
+ src/neuromeka_vfm/pose_estimation.py
8
+ src/neuromeka_vfm/segmentation.py
9
+ src/neuromeka_vfm/upload_mesh.py
10
+ src/neuromeka_vfm.egg-info/PKG-INFO
11
+ src/neuromeka_vfm.egg-info/SOURCES.txt
12
+ src/neuromeka_vfm.egg-info/dependency_links.txt
13
+ src/neuromeka_vfm.egg-info/entry_points.txt
14
+ src/neuromeka_vfm.egg-info/requires.txt
15
+ src/neuromeka_vfm.egg-info/top_level.txt
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ neuromeka-upload-mesh = neuromeka_vfm.upload_mesh:cli
@@ -0,0 +1,5 @@
1
+ numpy
2
+ pyzmq
3
+ paramiko
4
+ av
5
+ opencv-python-headless
@@ -0,0 +1 @@
1
+ neuromeka_vfm