neuromeka-grasp 0.1.0__tar.gz

neuromeka_grasp-0.1.0/PKG-INFO

@@ -0,0 +1,206 @@
+Metadata-Version: 2.4
+Name: neuromeka_grasp
+Version: 0.1.0
+Summary: Python client utilities for Neuromeka GraspGen Docker server
+Author-email: Neuromeka <support@neuromeka.com>
+License: Proprietary
+Project-URL: Homepage, https://github.com/neuromeka
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: numpy>=1.23
+Requires-Dist: pyzmq>=25.0.0
+Requires-Dist: opencv-python-headless>=4.5.0
+
+# neuromeka_grasp
+
+Python client utilities for the GraspGen Docker server (ZeroMQ + pickle RPC).
+Import and use it directly with `from neuromeka_grasp import GraspGeneration`.
+
+## Installation
+```bash
+pip install neuromeka_grasp
+```
+
+## Requirements
+- Python >= 3.9
+- A running GraspGen Docker server from `neuromeka-repo/nrmk_graspgen`
+- Server port is fixed to `5558`; the client defaults to this port and you usually do not need to change it
+- Dependencies: numpy, pyzmq, opencv-python-headless
+
+## Supported grippers (model presets)
+These names map to server-side checkpoints (see `nrmk_graspgen/modules/graspgen/app.py`):
+- `dh_ag_160_95` (default)
+- `robotiq_2f_140`
+- `franka_panda`
+- `single_suction_cup_30mm`
+
+You can also pass a `.yml` config path via `model` if the server has other checkpoints.
+
+## Quick start (depth + mask)
+```python
+from neuromeka_grasp import GraspGeneration
+
+client = GraspGeneration(hostname="localhost")
+client.init(fx, fy, cx, cy, model="dh_ag_160_95")
+
+resp = client.inference_from_depth_mask(
+    depth=depth_np,
+    mask=mask_np,
+    fx=fx, fy=fy, cx=cx, cy=cy,
+    enable_orientation_projection=True,
+    approach_axis_source="local_normal_avg",
+    enable_roll_projection=True,
+    target_roll_direction="auto",
+    enable_translation_projection=True,
+    translation_axis=-1,
+    desired_offset=0.03,
+)
+
+if resp["result"] == "SUCCESS":
+    grasps = resp["data"]["grasps"]
+    scores = resp["data"]["scores"]
+```
+
+
+## Point cloud example
+```python
+import numpy as np
+from neuromeka_grasp import GraspGeneration
+
+client = GraspGeneration(hostname="localhost")
+client.init(fx, fy, cx, cy, model="dh_ag_160_95")
+
+object_pc = np.asarray(object_points, dtype=np.float32)  # (N, 3)
+scene_pc = np.asarray(scene_points, dtype=np.float32)    # (M, 3)
+
+resp = client.inference_from_point_cloud(
+    object_pc=object_pc,
+    scene_pc=scene_pc,
+    collision_check=True,
+    grasp_threshold=0.8,
+    num_grasps=200,
+)
+```
+
+## Visualization (optional)
+```python
+from neuromeka_grasp import draw_grasps_overlay
+
+overlay = draw_grasps_overlay(
+    rgb=rgb_image,
+    grasps=grasps,   # (K, 4, 4)
+    scores=scores,   # (K,)
+    fx=fx, fy=fy, cx=cx, cy=cy,
+)
+```
+
+## API overview
+- `GraspGeneration`
+  - `init(fx, fy, cx, cy, model="dh_ag_160_95")`
+  - `inference_from_point_cloud(...)`
+  - `inference_from_depth_mask(...)`
+  - `point_cloud_outlier_removal(obj_pc, threshold=0.014, k=20)`
+  - `reset()`, `close()`
+- `draw_grasps_overlay(...)`: Project predicted grasps onto an RGB image.
+
+## init() reference
+**Signature**
+`init(fx, fy, cx, cy, model="dh_ag_160_95")`
+
+**Parameters**
+- `fx`, `fy`: Focal lengths in pixels.
+- `cx`, `cy`: Principal point in pixels.
+- `model`: Gripper preset name (see Supported grippers) or a `.yml` config path.
+  Config paths are resolved on the server side, so use a path visible to the server container.
+
+**Behavior**
+- Stores intrinsics in the client for future calls.
+- Initializes the server model; response includes `gripper_name`, `model_path`, and `intrinsics`.
+
+## inference_from_depth_mask() reference
+**Signature**
+`inference_from_depth_mask(depth, mask, fx=None, fy=None, cx=None, cy=None, ...)`
+
+**Core inputs**
+- `depth`: (H, W) depth image. Units are scaled by `depth_scale`.
+- `mask`: (H, W) or (H, W, 1) segmentation mask. If `target_object_id` is set, only that label is used.
+- `fx`, `fy`, `cx`, `cy`: Camera intrinsics. If omitted, values set by `init()` are used.
+
+**Object/scene point clouds**
+- `target_object_id`: If set, selects a single object label from `mask`.
+- `depth_scale`: Multiplier applied to `depth` before projection.
+- `max_object_points`: Random downsample limit for object points (None disables).
+- `max_scene_points`: Random downsample limit for scene points (used only when collision_check=True).
+
+**Sampling and filtering**
+- `collision_check`: If True, runs collision filtering with the scene point cloud.
+- `collision_threshold`: Distance (meters) for collision filtering.
+- `grasp_threshold`: Minimum score threshold for grasps.
+- `num_grasps`: Number of grasps to sample before filtering.
+- `return_topk`: If True, also returns top-k grasps (server-side).
+- `topk_num_grasps`: Number of top grasps to keep when `return_topk=True`.
+- `min_grasps`: Minimum number of grasps to return (server-side).
+- `max_tries`: Max sampling attempts (server-side).
+
+**Orientation projection**
+- `enable_orientation_projection`: Enable orientation constraint projection.
+- `approach_axis_index`: Gripper-frame approach axis index (0=x, 1=y, 2=z).
+- `approach_axis_source`: `"local_normal"`, `"local_normal_avg"`, or `"global_pca"`.
+- `normal_avg_k`: KNN count for averaging normals in `"local_normal_avg"`.
+- `target_approach_direction`: Target approach direction (3-vector, camera frame).
+- `step_strength_orient`: Orientation correction strength in [0, 1].
+- `use_surface_normal_for_approach`: Legacy switch; affects default behavior when `approach_axis_source` is not set.
+- `contact_proxy_offset`: Offset (meters) along approach axis for contact proxy.
+
+**Translation projection**
+- `enable_translation_projection`: Enable translation constraint projection.
+- `translation_axis`: PCA axis index (`-1` = principal axis).
+- `desired_offset`: Target offset (meters) along PCA axis.
+- `step_strength_trans`: Translation correction strength in [0, 1].
+
+**Roll projection**
+- `enable_roll_projection`: Enable roll projection around approach axis.
+- `jaw_axis_index`: Gripper-frame jaw axis index (0=x, 1=y, 2=z).
+- `target_roll_direction`: Target jaw direction (3-vector, camera frame) or `"auto"`.
+- `roll_target_axis`: PCA axis index when `target_roll_direction="auto"`.
+- `roll_target_axis_sign`: Optional sign override (+1 or -1) for the PCA axis.
+- `roll_use_local_tangent`: Use local tangent (2D PCA) for `"auto"` roll targets (server support required).
+- `step_strength_roll`: Roll correction strength in [0, 1].
+
+**PCA sign and debug**
+- `pca_axis_sign_ref`: PCA sign reference (`"camera_x"`, `"roll_ref"`, or a 3-vector).
+- `projection_debug`: Enable projection debug logging on the server.
+
+## Axis convention notes
+- The rotation matrix columns represent the gripper-frame axes expressed in the camera frame.
+- `approach_axis_index` and `jaw_axis_index` refer to those gripper-frame axes.
+- When using `"global_pca"`, some server builds allow `approach_axis_index` or `roll_target_axis`
+  values `-10/-11/-12` to select the opposite direction of PCA axis 0/1/2.
+
+## PCA sign note
+PCA eigenvectors can flip sign across runs. Use `pca_axis_sign_ref` to keep sign consistent
+and avoid ambiguous translation/roll directions.
+
+## Inputs and shapes
+- `depth`: HxW float or uint depth image.
+- `mask`: HxW or HxWx1 integer mask. If `target_object_id` is set, only that label is used.
+- `depth_scale`: Multiplier applied to `depth` before projection.
+- `object_pc`, `scene_pc`: (N, 3) and (M, 3) float32 point clouds in camera frame.
+- `grasps`: (K, 4, 4) homogeneous poses, `scores`: (K,) in [0, 1].
+
+## Projection parameters (summary)
+- Orientation: `enable_orientation_projection`, `approach_axis_index`,
+  `approach_axis_source` (`local_normal`, `local_normal_avg`, `global_pca`),
+  `normal_avg_k`, `contact_proxy_offset`
+- Translation: `enable_translation_projection`, `translation_axis`, `desired_offset`
+- Roll: `enable_roll_projection`, `target_roll_direction` (supports `"auto"`),
+  `roll_target_axis`, `roll_target_axis_sign`, `roll_use_local_tangent`
+- Sign control and debug: `pca_axis_sign_ref`, `projection_debug`
+
+For detailed parameter semantics, refer to the server README.
+
+## Notes
+- `inference_from_depth_mask` requires intrinsics set via `init()` or passed directly.
+- Server port is fixed at `5558`; the client default matches this and typically should not be changed.
+- `PickleClient` is synchronous and blocking; set timeouts or retry logic on the server side if needed.
+- Pickle is not secure against untrusted sources. Use only in trusted environments.

neuromeka_grasp-0.1.0/README.md

@@ -0,0 +1,193 @@
+# neuromeka_grasp
+
+Python client utilities for the GraspGen Docker server (ZeroMQ + pickle RPC).
+Import and use it directly with `from neuromeka_grasp import GraspGeneration`.
+
+## Installation
+```bash
+pip install neuromeka_grasp
+```
+
+## Requirements
+- Python >= 3.9
+- A running GraspGen Docker server from `neuromeka-repo/nrmk_graspgen`
+- Server port is fixed to `5558`; the client defaults to this port and you usually do not need to change it
+- Dependencies: numpy, pyzmq, opencv-python-headless
+
+## Supported grippers (model presets)
+These names map to server-side checkpoints (see `nrmk_graspgen/modules/graspgen/app.py`):
+- `dh_ag_160_95` (default)
+- `robotiq_2f_140`
+- `franka_panda`
+- `single_suction_cup_30mm`
+
+You can also pass a `.yml` config path via `model` if the server has other checkpoints.
+
+## Quick start (depth + mask)
+```python
+from neuromeka_grasp import GraspGeneration
+
+client = GraspGeneration(hostname="localhost")
+client.init(fx, fy, cx, cy, model="dh_ag_160_95")
+
+resp = client.inference_from_depth_mask(
+    depth=depth_np,
+    mask=mask_np,
+    fx=fx, fy=fy, cx=cx, cy=cy,
+    enable_orientation_projection=True,
+    approach_axis_source="local_normal_avg",
+    enable_roll_projection=True,
+    target_roll_direction="auto",
+    enable_translation_projection=True,
+    translation_axis=-1,
+    desired_offset=0.03,
+)
+
+if resp["result"] == "SUCCESS":
+    grasps = resp["data"]["grasps"]
+    scores = resp["data"]["scores"]
+```
+
+
+## Point cloud example
+```python
+import numpy as np
+from neuromeka_grasp import GraspGeneration
+
+client = GraspGeneration(hostname="localhost")
+client.init(fx, fy, cx, cy, model="dh_ag_160_95")
+
+object_pc = np.asarray(object_points, dtype=np.float32)  # (N, 3)
+scene_pc = np.asarray(scene_points, dtype=np.float32)    # (M, 3)
+
+resp = client.inference_from_point_cloud(
+    object_pc=object_pc,
+    scene_pc=scene_pc,
+    collision_check=True,
+    grasp_threshold=0.8,
+    num_grasps=200,
+)
+```
+
+## Visualization (optional)
+```python
+from neuromeka_grasp import draw_grasps_overlay
+
+overlay = draw_grasps_overlay(
+    rgb=rgb_image,
+    grasps=grasps,   # (K, 4, 4)
+    scores=scores,   # (K,)
+    fx=fx, fy=fy, cx=cx, cy=cy,
+)
+```
+
+## API overview
+- `GraspGeneration`
+  - `init(fx, fy, cx, cy, model="dh_ag_160_95")`
+  - `inference_from_point_cloud(...)`
+  - `inference_from_depth_mask(...)`
+  - `point_cloud_outlier_removal(obj_pc, threshold=0.014, k=20)`
+  - `reset()`, `close()`
+- `draw_grasps_overlay(...)`: Project predicted grasps onto an RGB image.
+
+## init() reference
+**Signature**
+`init(fx, fy, cx, cy, model="dh_ag_160_95")`
+
+**Parameters**
+- `fx`, `fy`: Focal lengths in pixels.
+- `cx`, `cy`: Principal point in pixels.
+- `model`: Gripper preset name (see Supported grippers) or a `.yml` config path.
+  Config paths are resolved on the server side, so use a path visible to the server container.
+
+**Behavior**
+- Stores intrinsics in the client for future calls.
+- Initializes the server model; response includes `gripper_name`, `model_path`, and `intrinsics`.
+
+## inference_from_depth_mask() reference
+**Signature**
+`inference_from_depth_mask(depth, mask, fx=None, fy=None, cx=None, cy=None, ...)`
+
+**Core inputs**
+- `depth`: (H, W) depth image. Units are scaled by `depth_scale`.
+- `mask`: (H, W) or (H, W, 1) segmentation mask. If `target_object_id` is set, only that label is used.
+- `fx`, `fy`, `cx`, `cy`: Camera intrinsics. If omitted, values set by `init()` are used.
+
+**Object/scene point clouds**
+- `target_object_id`: If set, selects a single object label from `mask`.
+- `depth_scale`: Multiplier applied to `depth` before projection.
+- `max_object_points`: Random downsample limit for object points (None disables).
+- `max_scene_points`: Random downsample limit for scene points (used only when collision_check=True).
+
+**Sampling and filtering**
+- `collision_check`: If True, runs collision filtering with the scene point cloud.
+- `collision_threshold`: Distance (meters) for collision filtering.
+- `grasp_threshold`: Minimum score threshold for grasps.
+- `num_grasps`: Number of grasps to sample before filtering.
+- `return_topk`: If True, also returns top-k grasps (server-side).
+- `topk_num_grasps`: Number of top grasps to keep when `return_topk=True`.
+- `min_grasps`: Minimum number of grasps to return (server-side).
+- `max_tries`: Max sampling attempts (server-side).
+
+**Orientation projection**
+- `enable_orientation_projection`: Enable orientation constraint projection.
+- `approach_axis_index`: Gripper-frame approach axis index (0=x, 1=y, 2=z).
+- `approach_axis_source`: `"local_normal"`, `"local_normal_avg"`, or `"global_pca"`.
+- `normal_avg_k`: KNN count for averaging normals in `"local_normal_avg"`.
+- `target_approach_direction`: Target approach direction (3-vector, camera frame).
+- `step_strength_orient`: Orientation correction strength in [0, 1].
+- `use_surface_normal_for_approach`: Legacy switch; affects default behavior when `approach_axis_source` is not set.
+- `contact_proxy_offset`: Offset (meters) along approach axis for contact proxy.
+
+**Translation projection**
+- `enable_translation_projection`: Enable translation constraint projection.
+- `translation_axis`: PCA axis index (`-1` = principal axis).
+- `desired_offset`: Target offset (meters) along PCA axis.
+- `step_strength_trans`: Translation correction strength in [0, 1].
+
+**Roll projection**
+- `enable_roll_projection`: Enable roll projection around approach axis.
+- `jaw_axis_index`: Gripper-frame jaw axis index (0=x, 1=y, 2=z).
+- `target_roll_direction`: Target jaw direction (3-vector, camera frame) or `"auto"`.
+- `roll_target_axis`: PCA axis index when `target_roll_direction="auto"`.
+- `roll_target_axis_sign`: Optional sign override (+1 or -1) for the PCA axis.
+- `roll_use_local_tangent`: Use local tangent (2D PCA) for `"auto"` roll targets (server support required).
+- `step_strength_roll`: Roll correction strength in [0, 1].
+
+**PCA sign and debug**
+- `pca_axis_sign_ref`: PCA sign reference (`"camera_x"`, `"roll_ref"`, or a 3-vector).
+- `projection_debug`: Enable projection debug logging on the server.
+
+## Axis convention notes
+- The rotation matrix columns represent the gripper-frame axes expressed in the camera frame.
+- `approach_axis_index` and `jaw_axis_index` refer to those gripper-frame axes.
+- When using `"global_pca"`, some server builds allow `approach_axis_index` or `roll_target_axis`
+  values `-10/-11/-12` to select the opposite direction of PCA axis 0/1/2.
+
+## PCA sign note
+PCA eigenvectors can flip sign across runs. Use `pca_axis_sign_ref` to keep sign consistent
+and avoid ambiguous translation/roll directions.
+
+## Inputs and shapes
+- `depth`: HxW float or uint depth image.
+- `mask`: HxW or HxWx1 integer mask. If `target_object_id` is set, only that label is used.
+- `depth_scale`: Multiplier applied to `depth` before projection.
+- `object_pc`, `scene_pc`: (N, 3) and (M, 3) float32 point clouds in camera frame.
+- `grasps`: (K, 4, 4) homogeneous poses, `scores`: (K,) in [0, 1].
+
+## Projection parameters (summary)
+- Orientation: `enable_orientation_projection`, `approach_axis_index`,
+  `approach_axis_source` (`local_normal`, `local_normal_avg`, `global_pca`),
+  `normal_avg_k`, `contact_proxy_offset`
+- Translation: `enable_translation_projection`, `translation_axis`, `desired_offset`
+- Roll: `enable_roll_projection`, `target_roll_direction` (supports `"auto"`),
+  `roll_target_axis`, `roll_target_axis_sign`, `roll_use_local_tangent`
+- Sign control and debug: `pca_axis_sign_ref`, `projection_debug`
+
+For detailed parameter semantics, refer to the server README.
+
+## Notes
+- `inference_from_depth_mask` requires intrinsics set via `init()` or passed directly.
+- Server port is fixed at `5558`; the client default matches this and typically should not be changed.
+- `PickleClient` is synchronous and blocking; set timeouts or retry logic on the server side if needed.
+- Pickle is not secure against untrusted sources. Use only in trusted environments.

neuromeka_grasp-0.1.0/neuromeka_grasp/__init__.py

@@ -0,0 +1,4 @@
+from .client import GraspGeneration, PickleClient
+from .overlay import draw_grasps_overlay
+
+__all__ = ["GraspGeneration", "PickleClient", "draw_grasps_overlay"]

neuromeka_grasp-0.1.0/neuromeka_grasp/client.py

@@ -0,0 +1,373 @@
+import pickle
+from pathlib import Path
+from typing import Optional, Sequence
+
+import numpy as np
+import zmq
+
+
+class PickleClient:
+    """ZeroMQ 기반 간단한 pickle RPC 클라이언트."""
+
+    def __init__(self, hostname: str, port: int = 5558):
+        self.hostname = hostname
+        self.port = port
+        self.context = zmq.Context()
+        self.socket = self.context.socket(zmq.REQ)
+        self.socket.connect(f"tcp://{self.hostname}:{self.port}")
+
+    def send_data(self, data: dict):
+        self.socket.send(pickle.dumps(data))
+        response = pickle.loads(self.socket.recv())
+        return response
+
+    def close(self):
+        self.socket.close()
+        self.context.term()
+
+
+class GraspGeneration:
+    """GraspGen Docker 서버용 클라이언트.
+
+    서버가 제공하는 projection 옵션을 모두 전달할 수 있도록 vfm_tester 및 nrmk_graspgen 양쪽을 통합했습니다.
+    """
+
+    def __init__(self, hostname: str, port: int = 5558):
+        self.client = PickleClient(hostname, port)
+        self.intrinsics: Optional[dict] = None
+        self.model: Optional[str] = None
+
+    # ---------- 기본 유틸 ----------
+    @staticmethod
+    def _to_2d_mask(mask):
+        if mask is None:
+            return None
+        if mask.ndim == 3 and mask.shape[-1] == 1:
+            return mask[:, :, 0]
+        return mask
+
+    @staticmethod
+    def _downsample_points(points: np.ndarray, max_points: Optional[int]):
+        if max_points is None or max_points <= 0:
+            return points
+        if points.shape[0] <= max_points:
+            return points
+        idx = np.random.choice(points.shape[0], max_points, replace=False)
+        return points[idx]
+
+    @staticmethod
+    def depth_to_points(
+        depth: np.ndarray,
+        fx: float,
+        fy: float,
+        cx: float,
+        cy: float,
+        mask: Optional[np.ndarray] = None,
+        depth_scale: float = 1.0,
+    ) -> np.ndarray:
+        depth = depth.astype(np.float32) * depth_scale
+        h, w = depth.shape
+        xs = np.arange(w)
+        ys = np.arange(h)
+        xmap, ymap = np.meshgrid(xs, ys)
+
+        valid = depth > 0
+        if mask is not None:
+            valid = valid & mask
+
+        z = depth[valid]
+        if z.size == 0:
+            return np.zeros((0, 3), dtype=np.float32)
+
+        x = (xmap[valid] - cx) * z / fx
+        y = (ymap[valid] - cy) * z / fy
+        return np.stack([x, y, z], axis=1).astype(np.float32)
+
+    def depth_mask_to_point_clouds(
+        self,
+        depth: np.ndarray,
+        mask: np.ndarray,
+        fx: float,
+        fy: float,
+        cx: float,
+        cy: float,
+        target_object_id: Optional[int] = None,
+        depth_scale: float = 1.0,
+        remove_object_from_scene: bool = True,
+    ) -> tuple[np.ndarray, np.ndarray]:
+        mask = self._to_2d_mask(mask)
+        if target_object_id is None:
+            obj_mask = mask > 0
+        else:
+            obj_mask = mask == target_object_id
+
+        object_pc = self.depth_to_points(
+            depth, fx, fy, cx, cy, mask=obj_mask, depth_scale=depth_scale
+        )
+
+        scene_mask = depth > 0
+        if remove_object_from_scene:
+            scene_mask = scene_mask & (~obj_mask)
+        scene_pc = self.depth_to_points(
+            depth, fx, fy, cx, cy, mask=scene_mask, depth_scale=depth_scale
+        )
+
+        return object_pc, scene_pc
+
+    @staticmethod
+    def _mean_knn_distances_numpy(points: np.ndarray, k_eff: int, chunk_size: int = 1024):
+        n = points.shape[0]
+        mean_dists = np.empty(n, dtype=np.float32)
+        for start in range(0, n, chunk_size):
+            end = min(start + chunk_size, n)
+            diff = points[start:end, None, :] - points[None, :, :]
+            dists = np.linalg.norm(diff, axis=-1)
+            if k_eff < n:
+                idx = np.argpartition(dists, kth=k_eff - 1, axis=1)[:, :k_eff]
+                row = np.take_along_axis(dists, idx, axis=1)
+            else:
+                row = np.sort(dists, axis=1)
+            if row.shape[1] > 1:
+                mean = row[:, 1:].mean(axis=1)
+            else:
+                mean = row[:, 0]
+            mean_dists[start:end] = mean.astype(np.float32)
+        return mean_dists
+
+    def point_cloud_outlier_removal(
+        self, obj_pc: np.ndarray, threshold: float = 0.014, k: int = 20
+    ) -> tuple[np.ndarray, np.ndarray]:
+        obj_pc = np.asarray(obj_pc, dtype=np.float32)
+        if obj_pc.ndim != 2 or obj_pc.shape[1] != 3:
+            raise ValueError("obj_pc must be Nx3")
+        if obj_pc.shape[0] == 0:
+            return obj_pc.copy(), obj_pc.copy()
+
+        k_eff = min(k + 1, obj_pc.shape[0])
+        mean_dists = None
+        try:
+            from scipy.spatial import cKDTree
+
+            tree = cKDTree(obj_pc)
+            dists, _ = tree.query(obj_pc, k=k_eff)
+            if k_eff > 1:
+                mean_dists = dists[:, 1:].mean(axis=1)
+            else:
+                mean_dists = dists[:, 0]
+        except Exception:
+            mean_dists = self._mean_knn_distances_numpy(obj_pc, k_eff)
+
+        mask = mean_dists < threshold
+        filtered_pc = obj_pc[mask]
+        removed_pc = obj_pc[~mask]
+        return filtered_pc, removed_pc
+
+    # ---------- RPC ----------
+    def init(self, fx: float, fy: float, cx: float, cy: float, model: str = "dh_ag_160_95"):
+        self.intrinsics = {"fx": fx, "fy": fy, "cx": cx, "cy": cy}
+        self.model = model
+        payload = {"operation": "init", "model": model, "intrinsics": self.intrinsics}
+        return self.client.send_data(payload)
+
+    def inference_from_point_cloud(
+        self,
+        object_pc: np.ndarray,
+        scene_pc: Optional[np.ndarray] = None,
+        collision_check: bool = False,
+        grasp_threshold: float = 0.8,
+        num_grasps: int = 200,
+        return_topk: bool = False,
+        topk_num_grasps: int = -1,
+        collision_threshold: float = 0.02,
+        min_grasps: Optional[int] = None,
+        max_tries: Optional[int] = None,
+        enable_orientation_projection: Optional[bool] = None,
+        enable_translation_projection: Optional[bool] = None,
+        approach_axis_index: Optional[int] = None,
+        approach_axis_source: Optional[str] = None,
+        normal_avg_k: Optional[int] = None,
+        target_approach_direction: Optional[Sequence[float]] = None,
+        translation_axis: Optional[int] = None,
+        desired_offset: Optional[float] = None,
+        step_strength_orient: Optional[float] = None,
+        step_strength_trans: Optional[float] = None,
+        enable_roll_projection: Optional[bool] = None,
+        jaw_axis_index: Optional[int] = None,
+        target_roll_direction: Optional[Sequence[float] | str] = None,
+        roll_target_axis: Optional[int] = None,
+        roll_target_axis_sign: Optional[int] = None,
+        roll_use_local_tangent: Optional[bool] = None,
+        step_strength_roll: Optional[float] = None,
+        use_surface_normal_for_approach: Optional[bool] = None,
+        contact_proxy_offset: Optional[float] = None,
+        pca_axis_sign_ref: Optional[Sequence[float] | str] = None,
+        projection_debug: Optional[bool] = None,
+    ):
+        payload = {
+            "operation": "inference",
+            "object_pc": object_pc,
+            "grasp_threshold": grasp_threshold,
+            "num_grasps": num_grasps,
+            "return_topk": return_topk,
+            "topk_num_grasps": topk_num_grasps,
+            "collision_check": collision_check,
+            "collision_threshold": collision_threshold,
+        }
+        if min_grasps is not None:
+            payload["min_grasps"] = min_grasps
+        if max_tries is not None:
+            payload["max_tries"] = max_tries
+        if enable_orientation_projection is not None:
+            payload["enable_orientation_projection"] = enable_orientation_projection
+        if enable_translation_projection is not None:
+            payload["enable_translation_projection"] = enable_translation_projection
+        if approach_axis_index is not None:
+            payload["approach_axis_index"] = approach_axis_index
+        if approach_axis_source is not None:
+            payload["approach_axis_source"] = approach_axis_source
+        if normal_avg_k is not None:
+            payload["normal_avg_k"] = normal_avg_k
+        if target_approach_direction is not None:
+            payload["target_approach_direction"] = target_approach_direction
+        if translation_axis is not None:
+            payload["translation_axis"] = translation_axis
+        if desired_offset is not None:
+            payload["desired_offset"] = desired_offset
+        if step_strength_orient is not None:
+            payload["step_strength_orient"] = step_strength_orient
+        if step_strength_trans is not None:
+            payload["step_strength_trans"] = step_strength_trans
+        if enable_roll_projection is not None:
+            payload["enable_roll_projection"] = enable_roll_projection
+        if jaw_axis_index is not None:
+            payload["jaw_axis_index"] = jaw_axis_index
+        if target_roll_direction is not None:
+            payload["target_roll_direction"] = target_roll_direction
+        if roll_target_axis is not None:
+            payload["roll_target_axis"] = roll_target_axis
+        if roll_target_axis_sign is not None:
+            payload["roll_target_axis_sign"] = roll_target_axis_sign
+        if roll_use_local_tangent is not None:
+            payload["roll_use_local_tangent"] = roll_use_local_tangent
+        if step_strength_roll is not None:
+            payload["step_strength_roll"] = step_strength_roll
+        if use_surface_normal_for_approach is not None:
+            payload["use_surface_normal_for_approach"] = use_surface_normal_for_approach
+        if contact_proxy_offset is not None:
+            payload["contact_proxy_offset"] = contact_proxy_offset
+        if pca_axis_sign_ref is not None:
+            payload["pca_axis_sign_ref"] = pca_axis_sign_ref
+        if projection_debug is not None:
+            payload["projection_debug"] = projection_debug
+        if collision_check and scene_pc is not None:
+            payload["scene_pc"] = scene_pc
+        return self.client.send_data(payload)
+
+    def inference_from_depth_mask(
+        self,
+        depth: np.ndarray,
+        mask: np.ndarray,
+        fx: Optional[float] = None,
+        fy: Optional[float] = None,
+        cx: Optional[float] = None,
+        cy: Optional[float] = None,
+        target_object_id: Optional[int] = None,
+        depth_scale: float = 1.0,
+        max_object_points: Optional[int] = None,
+        max_scene_points: int = 8192,
+        collision_check: bool = False,
+        grasp_threshold: float = 0.8,
+        num_grasps: int = 200,
+        return_topk: bool = False,
+        topk_num_grasps: int = -1,
+        collision_threshold: float = 0.02,
+        min_grasps: Optional[int] = None,
+        max_tries: Optional[int] = None,
+        enable_orientation_projection: Optional[bool] = None,
+        enable_translation_projection: Optional[bool] = None,
+        approach_axis_index: Optional[int] = None,
+        approach_axis_source: Optional[str] = None,
+        normal_avg_k: Optional[int] = None,
+        target_approach_direction: Optional[Sequence[float]] = None,
+        translation_axis: Optional[int] = None,
+        desired_offset: Optional[float] = None,
+        step_strength_orient: Optional[float] = None,
+        step_strength_trans: Optional[float] = None,
+        enable_roll_projection: Optional[bool] = None,
+        jaw_axis_index: Optional[int] = None,
+        target_roll_direction: Optional[Sequence[float] | str] = None,
+        roll_target_axis: Optional[int] = None,
+        roll_target_axis_sign: Optional[int] = None,
+        roll_use_local_tangent: Optional[bool] = None,
+        step_strength_roll: Optional[float] = None,
+        use_surface_normal_for_approach: Optional[bool] = None,
+        contact_proxy_offset: Optional[float] = None,
+        pca_axis_sign_ref: Optional[Sequence[float] | str] = None,
+        projection_debug: Optional[bool] = None,
+    ):
+        if self.intrinsics is None and None in (fx, fy, cx, cy):
+            raise ValueError("Camera intrinsics not set. Call init() or pass fx,fy,cx,cy.")
+
+        if fx is None:
+            fx = self.intrinsics["fx"]
+            fy = self.intrinsics["fy"]
+            cx = self.intrinsics["cx"]
+            cy = self.intrinsics["cy"]
+
+        object_pc, scene_pc = self.depth_mask_to_point_clouds(
+            depth,
+            mask,
+            fx,
+            fy,
+            cx,
+            cy,
+            target_object_id=target_object_id,
+            depth_scale=depth_scale,
+            remove_object_from_scene=True,
+        )
+        object_pc = self._downsample_points(object_pc, max_object_points)
+
+        if collision_check:
+            scene_pc = self._downsample_points(scene_pc, max_scene_points)
+        else:
+            scene_pc = None
+
+        return self.inference_from_point_cloud(
+            object_pc=object_pc,
+            scene_pc=scene_pc,
+            collision_check=collision_check,
+            grasp_threshold=grasp_threshold,
+            num_grasps=num_grasps,
+            return_topk=return_topk,
+            topk_num_grasps=topk_num_grasps,
+            collision_threshold=collision_threshold,
+            min_grasps=min_grasps,
+            max_tries=max_tries,
+            enable_orientation_projection=enable_orientation_projection,
+            enable_translation_projection=enable_translation_projection,
+            approach_axis_index=approach_axis_index,
+            approach_axis_source=approach_axis_source,
+            normal_avg_k=normal_avg_k,
+            target_approach_direction=target_approach_direction,
+            translation_axis=translation_axis,
+            desired_offset=desired_offset,
+            step_strength_orient=step_strength_orient,
+            step_strength_trans=step_strength_trans,
+            enable_roll_projection=enable_roll_projection,
+            jaw_axis_index=jaw_axis_index,
+            target_roll_direction=target_roll_direction,
+            roll_target_axis=roll_target_axis,
+            roll_target_axis_sign=roll_target_axis_sign,
+            roll_use_local_tangent=roll_use_local_tangent,
+            step_strength_roll=step_strength_roll,
+            use_surface_normal_for_approach=use_surface_normal_for_approach,
+            contact_proxy_offset=contact_proxy_offset,
+            pca_axis_sign_ref=pca_axis_sign_ref,
+            projection_debug=projection_debug,
+        )
+
+    def reset(self):
+        return self.client.send_data({"operation": "reset"})
+
+    def close(self):
+        self.client.close()

neuromeka_grasp-0.1.0/neuromeka_grasp/overlay.py

@@ -0,0 +1,103 @@
+import numpy as np
+import cv2
+
+
+def _project_points(points, fx, fy, cx, cy, image_shape):
+    z = points[:, 2]
+    valid = z > 1e-6
+    u = fx * points[:, 0] / z + cx
+    v = fy * points[:, 1] / z + cy
+    h, w = image_shape
+    valid = valid & (u >= 0) & (u < w) & (v >= 0) & (v < h)
+    return np.stack([u, v], axis=1), valid
+
+
+def _get_color_from_score(scores):
+    scores = np.clip(scores, 0.0, 1.0)
+    colors = np.stack([scores, 1.0 - scores, np.zeros_like(scores)], axis=1) * 255.0
+    return colors.astype(np.uint8)
+
+
+def draw_grasps_overlay(
+    rgb,
+    grasps,
+    scores,
+    fx,
+    fy,
+    cx,
+    cy,
+    max_grasps=50,
+    line_width=2,
+    labels=None,
+    label_color=(255, 255, 255),
+    label_scale=0.5,
+    label_thickness=1,
+    control_points=None,
+):
+    """
+    RGB 이미지에 grasp를 투영해 시각화합니다.
+
+    Args:
+        rgb: HxWx3 uint8 이미지
+        grasps: Kx4x4 grasp poses
+        scores: K 점수
+        control_points: (옵션) list of (M,3) arrays per grasp. 주어지면 해당 선분을 그림.
+    """
+    if rgb is None or grasps is None or len(grasps) == 0:
+        return None
+
+    frame = rgb.copy()
+    if frame.dtype != np.uint8:
+        frame = np.clip(frame, 0, 255).astype(np.uint8)
+
+    grasps = np.asarray(grasps)
+    scores = np.asarray(scores).reshape(-1)
+    image_shape = frame.shape[:2]
+    order = np.argsort(scores)[::-1]
+    if max_grasps > 0:
+        order = order[:max_grasps]
+    colors = _get_color_from_score(scores[order])
+
+    axis_length_m = 0.05
+    axis_colors = [(0, 0, 255), (0, 255, 0), (255, 0, 0)]  # x,y,z
+
+    for idx, grasp_idx in enumerate(order):
+        grasp = grasps[grasp_idx]
+        origin = grasp[:3, 3]
+        color = tuple(int(c) for c in colors[idx])
+        origin_uv, origin_valid = _project_points(origin[None, :], fx, fy, cx, cy, image_shape)
+        if not origin_valid[0]:
+            continue
+        ox, oy = int(origin_uv[0][0]), int(origin_uv[0][1])
+        cv2.circle(frame, (ox, oy), max(2, line_width + 1), color, line_width)
+
+        rot = grasp[:3, :3]
+        for axis_idx, axis_dir in enumerate([rot[:, 0], rot[:, 1], rot[:, 2]]):
+            end_pt = origin + axis_dir * axis_length_m
+            end_uv, end_valid = _project_points(end_pt[None, :], fx, fy, cx, cy, image_shape)
+            if end_valid[0]:
+                ex, ey = int(end_uv[0][0]), int(end_uv[0][1])
+                cv2.line(frame, (ox, oy), (ex, ey), axis_colors[axis_idx], line_width)
+
+        if control_points is not None and grasp_idx < len(control_points):
+            pts = np.asarray(control_points[grasp_idx], dtype=np.float32)
+            pts_cam = (grasp[:3, :3] @ pts.T).T + grasp[:3, 3]
+            uv, valid = _project_points(pts_cam, fx, fy, cx, cy, image_shape)
+            for i in range(len(uv) - 1):
+                if valid[i] and valid[i + 1]:
+                    p0 = (int(uv[i][0]), int(uv[i][1]))
+                    p1 = (int(uv[i + 1][0]), int(uv[i + 1][1]))
+                    cv2.line(frame, p0, p1, color, line_width)
+
+        if labels is not None and grasp_idx < len(labels):
+            cv2.putText(
+                frame,
+                str(labels[grasp_idx]),
+                (ox + 4, oy - 6),
+                cv2.FONT_HERSHEY_SIMPLEX,
+                label_scale,
+                label_color,
+                label_thickness,
+                cv2.LINE_AA,
+            )
+    return frame

neuromeka_grasp-0.1.0/neuromeka_grasp.egg-info/PKG-INFO

@@ -0,0 +1,206 @@
+Metadata-Version: 2.4
+Name: neuromeka_grasp
+Version: 0.1.0
+Summary: Python client utilities for Neuromeka GraspGen Docker server
+Author-email: Neuromeka <support@neuromeka.com>
+License: Proprietary
+Project-URL: Homepage, https://github.com/neuromeka
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: numpy>=1.23
+Requires-Dist: pyzmq>=25.0.0
+Requires-Dist: opencv-python-headless>=4.5.0
+
+# neuromeka_grasp
+
+Python client utilities for the GraspGen Docker server (ZeroMQ + pickle RPC).
+Import and use it directly with `from neuromeka_grasp import GraspGeneration`.
+
+## Installation
+```bash
+pip install neuromeka_grasp
+```
+
+## Requirements
+- Python >= 3.9
+- A running GraspGen Docker server from `neuromeka-repo/nrmk_graspgen`
+- Server port is fixed to `5558`; the client defaults to this port and you usually do not need to change it
+- Dependencies: numpy, pyzmq, opencv-python-headless
+
+## Supported grippers (model presets)
+These names map to server-side checkpoints (see `nrmk_graspgen/modules/graspgen/app.py`):
+- `dh_ag_160_95` (default)
+- `robotiq_2f_140`
+- `franka_panda`
+- `single_suction_cup_30mm`
+
+You can also pass a `.yml` config path via `model` if the server has other checkpoints.
+
+## Quick start (depth + mask)
+```python
+from neuromeka_grasp import GraspGeneration
+
+client = GraspGeneration(hostname="localhost")
+client.init(fx, fy, cx, cy, model="dh_ag_160_95")
+
+resp = client.inference_from_depth_mask(
+    depth=depth_np,
+    mask=mask_np,
+    fx=fx, fy=fy, cx=cx, cy=cy,
+    enable_orientation_projection=True,
+    approach_axis_source="local_normal_avg",
+    enable_roll_projection=True,
+    target_roll_direction="auto",
+    enable_translation_projection=True,
+    translation_axis=-1,
+    desired_offset=0.03,
+)
+
+if resp["result"] == "SUCCESS":
+    grasps = resp["data"]["grasps"]
+    scores = resp["data"]["scores"]
+```
+
+
+## Point cloud example
+```python
+import numpy as np
+from neuromeka_grasp import GraspGeneration
+
+client = GraspGeneration(hostname="localhost")
+client.init(fx, fy, cx, cy, model="dh_ag_160_95")
+
+object_pc = np.asarray(object_points, dtype=np.float32)  # (N, 3)
+scene_pc = np.asarray(scene_points, dtype=np.float32)    # (M, 3)
+
+resp = client.inference_from_point_cloud(
+    object_pc=object_pc,
+    scene_pc=scene_pc,
+    collision_check=True,
+    grasp_threshold=0.8,
+    num_grasps=200,
+)
+```
+
+## Visualization (optional)
+```python
+from neuromeka_grasp import draw_grasps_overlay
+
+overlay = draw_grasps_overlay(
+    rgb=rgb_image,
+    grasps=grasps,   # (K, 4, 4)
+    scores=scores,   # (K,)
+    fx=fx, fy=fy, cx=cx, cy=cy,
+)
+```
+
+## API overview
+- `GraspGeneration`
+  - `init(fx, fy, cx, cy, model="dh_ag_160_95")`
+  - `inference_from_point_cloud(...)`
+  - `inference_from_depth_mask(...)`
+  - `point_cloud_outlier_removal(obj_pc, threshold=0.014, k=20)`
+  - `reset()`, `close()`
+- `draw_grasps_overlay(...)`: Project predicted grasps onto an RGB image.
+
+## init() reference
+**Signature**
+`init(fx, fy, cx, cy, model="dh_ag_160_95")`
+
+**Parameters**
+- `fx`, `fy`: Focal lengths in pixels.
+- `cx`, `cy`: Principal point in pixels.
+- `model`: Gripper preset name (see Supported grippers) or a `.yml` config path.
+  Config paths are resolved on the server side, so use a path visible to the server container.
+
+**Behavior**
+- Stores intrinsics in the client for future calls.
+- Initializes the server model; response includes `gripper_name`, `model_path`, and `intrinsics`.
+
+## inference_from_depth_mask() reference
+**Signature**
+`inference_from_depth_mask(depth, mask, fx=None, fy=None, cx=None, cy=None, ...)`
+
+**Core inputs**
+- `depth`: (H, W) depth image. Units are scaled by `depth_scale`.
+- `mask`: (H, W) or (H, W, 1) segmentation mask. If `target_object_id` is set, only that label is used.
+- `fx`, `fy`, `cx`, `cy`: Camera intrinsics. If omitted, values set by `init()` are used.
+
+**Object/scene point clouds**
+- `target_object_id`: If set, selects a single object label from `mask`.
+- `depth_scale`: Multiplier applied to `depth` before projection.
+- `max_object_points`: Random downsample limit for object points (None disables).
+- `max_scene_points`: Random downsample limit for scene points (used only when collision_check=True).
+
+**Sampling and filtering**
+- `collision_check`: If True, runs collision filtering with the scene point cloud.
+- `collision_threshold`: Distance (meters) for collision filtering.
+- `grasp_threshold`: Minimum score threshold for grasps.
+- `num_grasps`: Number of grasps to sample before filtering.
+- `return_topk`: If True, also returns top-k grasps (server-side).
+- `topk_num_grasps`: Number of top grasps to keep when `return_topk=True`.
+- `min_grasps`: Minimum number of grasps to return (server-side).
+- `max_tries`: Max sampling attempts (server-side).
+
+**Orientation projection**
+- `enable_orientation_projection`: Enable orientation constraint projection.
+- `approach_axis_index`: Gripper-frame approach axis index (0=x, 1=y, 2=z).
+- `approach_axis_source`: `"local_normal"`, `"local_normal_avg"`, or `"global_pca"`.
+- `normal_avg_k`: KNN count for averaging normals in `"local_normal_avg"`.
+- `target_approach_direction`: Target approach direction (3-vector, camera frame).
+- `step_strength_orient`: Orientation correction strength in [0, 1].
+- `use_surface_normal_for_approach`: Legacy switch; affects default behavior when `approach_axis_source` is not set.
+- `contact_proxy_offset`: Offset (meters) along approach axis for contact proxy.
+
+**Translation projection**
+- `enable_translation_projection`: Enable translation constraint projection.
+- `translation_axis`: PCA axis index (`-1` = principal axis).
+- `desired_offset`: Target offset (meters) along PCA axis.
+- `step_strength_trans`: Translation correction strength in [0, 1].
+
+**Roll projection**
+- `enable_roll_projection`: Enable roll projection around approach axis.
+- `jaw_axis_index`: Gripper-frame jaw axis index (0=x, 1=y, 2=z).
+- `target_roll_direction`: Target jaw direction (3-vector, camera frame) or `"auto"`.
+- `roll_target_axis`: PCA axis index when `target_roll_direction="auto"`.
+- `roll_target_axis_sign`: Optional sign override (+1 or -1) for the PCA axis.
+- `roll_use_local_tangent`: Use local tangent (2D PCA) for `"auto"` roll targets (server support required).
+- `step_strength_roll`: Roll correction strength in [0, 1].
+
+**PCA sign and debug**
+- `pca_axis_sign_ref`: PCA sign reference (`"camera_x"`, `"roll_ref"`, or a 3-vector).
+- `projection_debug`: Enable projection debug logging on the server.
+
+## Axis convention notes
+- The rotation matrix columns represent the gripper-frame axes expressed in the camera frame.
+- `approach_axis_index` and `jaw_axis_index` refer to those gripper-frame axes.
+- When using `"global_pca"`, some server builds allow `approach_axis_index` or `roll_target_axis`
+  values `-10/-11/-12` to select the opposite direction of PCA axis 0/1/2.
+
+## PCA sign note
+PCA eigenvectors can flip sign across runs. Use `pca_axis_sign_ref` to keep sign consistent
+and avoid ambiguous translation/roll directions.
+
+## Inputs and shapes
+- `depth`: HxW float or uint depth image.
+- `mask`: HxW or HxWx1 integer mask. If `target_object_id` is set, only that label is used.
+- `depth_scale`: Multiplier applied to `depth` before projection.
+- `object_pc`, `scene_pc`: (N, 3) and (M, 3) float32 point clouds in camera frame.
+- `grasps`: (K, 4, 4) homogeneous poses, `scores`: (K,) in [0, 1].
+
+## Projection parameters (summary)
+- Orientation: `enable_orientation_projection`, `approach_axis_index`,
+  `approach_axis_source` (`local_normal`, `local_normal_avg`, `global_pca`),
+  `normal_avg_k`, `contact_proxy_offset`
+- Translation: `enable_translation_projection`, `translation_axis`, `desired_offset`
+- Roll: `enable_roll_projection`, `target_roll_direction` (supports `"auto"`),
+  `roll_target_axis`, `roll_target_axis_sign`, `roll_use_local_tangent`
+- Sign control and debug: `pca_axis_sign_ref`, `projection_debug`
+
+For detailed parameter semantics, refer to the server README.
+
+## Notes
+- `inference_from_depth_mask` requires intrinsics set via `init()` or passed directly.
+- Server port is fixed at `5558`; the client default matches this and typically should not be changed.
+- `PickleClient` is synchronous and blocking; set timeouts or retry logic on the server side if needed.
+- Pickle is not secure against untrusted sources. Use only in trusted environments.

neuromeka_grasp-0.1.0/neuromeka_grasp.egg-info/SOURCES.txt

@@ -0,0 +1,10 @@
+README.md
+pyproject.toml
+neuromeka_grasp/__init__.py
+neuromeka_grasp/client.py
+neuromeka_grasp/overlay.py
+neuromeka_grasp.egg-info/PKG-INFO
+neuromeka_grasp.egg-info/SOURCES.txt
+neuromeka_grasp.egg-info/dependency_links.txt
+neuromeka_grasp.egg-info/requires.txt
+neuromeka_grasp.egg-info/top_level.txt

neuromeka_grasp-0.1.0/neuromeka_grasp.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

neuromeka_grasp-0.1.0/neuromeka_grasp.egg-info/requires.txt

@@ -0,0 +1,3 @@
+numpy>=1.23
+pyzmq>=25.0.0
+opencv-python-headless>=4.5.0

neuromeka_grasp-0.1.0/neuromeka_grasp.egg-info/top_level.txt

@@ -0,0 +1 @@
+neuromeka_grasp

neuromeka_grasp-0.1.0/pyproject.toml

@@ -0,0 +1,20 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "neuromeka_grasp"
+version = "0.1.0"
+description = "Python client utilities for Neuromeka GraspGen Docker server"
+authors = [{ name = "Neuromeka", email = "support@neuromeka.com" }]
+readme = "README.md"
+requires-python = ">=3.9"
+dependencies = [
+    "numpy>=1.23",
+    "pyzmq>=25.0.0",
+    "opencv-python-headless>=4.5.0",
+]
+license = { text = "Proprietary" }
+
+[project.urls]
+Homepage = "https://github.com/neuromeka"

neuromeka_grasp-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+