mblt-vision-python 0.0.1__tar.gz → 0.0.3__tar.gz

mblt_vision_python-0.0.3/MANIFEST.in

@@ -0,0 +1,6 @@
+# Sample images under assets/ are development-only inputs for manual QA and
+# documented CLI examples. They are deliberately tracked in git but must never
+# reach a distributed artifact. setuptools already excludes them (only
+# `mblt_vision*` packages are built), so this prunes them explicitly to pin the
+# intent against future packaging changes rather than to fix a current leak.
+prune assets

{mblt_vision_python-0.0.1/mblt_vision_python.egg-info → mblt_vision_python-0.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mblt-vision-python
-Version: 0.0.1
+Version: 0.0.3
 Summary: Mobilint NPU vision models: classification, detection, segmentation, pose
 Author: Mobilint
 License: BSD-3-Clause
@@ -84,7 +84,8 @@ pip install "mblt-vision-python[onnxruntime-gpu]"
 
 ## Quick start
 
-Each model includes its matching preprocess and postprocess behavior:
+Most models include matching `preprocess`/`postprocess` behavior around a raw
+call (mask generation is the exception -- see below):
 
 ```python
 from mblt_vision import ResNet50
@@ -114,6 +115,20 @@ code should use the task subpackages (for example,
 
 `obb` is the canonical oriented-bounding-box task name.
 
+Mask generation is promptable, so `SAM2HieraLarge` takes point prompts and
+returns candidate masks from a single `predict()` call instead of the separate
+`preprocess`/raw-call/`postprocess` steps above:
+
+```python
+from mblt_vision.mask_generation import SAM2HieraLarge
+
+model = SAM2HieraLarge()  # framework="onnx" for ONNX Runtime inference
+result = model.predict("image.jpg", points=[[320, 240]], labels=[1])
+```
+
+See the [mask generation section](mblt_vision/README.md#mask-generation) for the
+prompt contract, the two-artifact layout, and SA-V validation.
+
 ## Model Zoo migration
 
 Vision is now maintained in this package. `mblt-model-zoo` retains
@@ -131,17 +146,24 @@ mblt-vision predict --source image.jpg --model resnet50
 ```
 
 `predict` is the single inference command for classification, depth estimation,
-object and face detection, instance and semantic segmentation, OBB, and pose
-estimation. The selected model determines its task and processing pipeline.
+object and face detection, instance and semantic segmentation, OBB, pose
+estimation, and point-prompted mask generation. The selected model determines
+its task and processing pipeline.
 By default it downloads the model artifact and saves a plotted result under
 `runs/vision/predict/`. Use `--output` to choose the result-image path,
 `--topk` for classification labels, and `--conf-thres`/`--iou-thres` for
 detection-style tasks. `--framework onnx` selects ONNX Runtime inference;
 `--target-device` and `--core-mode` select the MXQ board/runtime mode.
 
+Mask generation models require 1-3 point prompts (`--point X,Y,LABEL`, where
+LABEL is 1 for positive and 0 for negative) and load two artifacts, overridden
+with `--encoder-mxq-path`/`--decoder-mxq-path` (or the `--encoder-onnx-path`/
+`--decoder-onnx-path` pair) rather than `--model-path`/`--mxq-path`/`--onnx-path`.
+
 ```bash
 mblt-vision predict --source image.jpg --model yolo11m --conf-thres 0.4 --output result.jpg
 mblt-vision predict --source image.jpg --model yolo11m-pose --target-device regulus-ra --core-mode single
+mblt-vision predict --source image.jpg --model sam2-hiera-large --point 320,240,1
 ```
 
 The corresponding `mblt-model-zoo` commands use the same standalone handlers for

{mblt_vision_python-0.0.1 → mblt_vision_python-0.0.3}/README.md

@@ -40,7 +40,8 @@ pip install "mblt-vision-python[onnxruntime-gpu]"
 
 ## Quick start
 
-Each model includes its matching preprocess and postprocess behavior:
+Most models include matching `preprocess`/`postprocess` behavior around a raw
+call (mask generation is the exception -- see below):
 
 ```python
 from mblt_vision import ResNet50
@@ -70,6 +71,20 @@ code should use the task subpackages (for example,
 
 `obb` is the canonical oriented-bounding-box task name.
 
+Mask generation is promptable, so `SAM2HieraLarge` takes point prompts and
+returns candidate masks from a single `predict()` call instead of the separate
+`preprocess`/raw-call/`postprocess` steps above:
+
+```python
+from mblt_vision.mask_generation import SAM2HieraLarge
+
+model = SAM2HieraLarge()  # framework="onnx" for ONNX Runtime inference
+result = model.predict("image.jpg", points=[[320, 240]], labels=[1])
+```
+
+See the [mask generation section](mblt_vision/README.md#mask-generation) for the
+prompt contract, the two-artifact layout, and SA-V validation.
+
 ## Model Zoo migration
 
 Vision is now maintained in this package. `mblt-model-zoo` retains
@@ -87,17 +102,24 @@ mblt-vision predict --source image.jpg --model resnet50
 ```
 
 `predict` is the single inference command for classification, depth estimation,
-object and face detection, instance and semantic segmentation, OBB, and pose
-estimation. The selected model determines its task and processing pipeline.
+object and face detection, instance and semantic segmentation, OBB, pose
+estimation, and point-prompted mask generation. The selected model determines
+its task and processing pipeline.
 By default it downloads the model artifact and saves a plotted result under
 `runs/vision/predict/`. Use `--output` to choose the result-image path,
 `--topk` for classification labels, and `--conf-thres`/`--iou-thres` for
 detection-style tasks. `--framework onnx` selects ONNX Runtime inference;
 `--target-device` and `--core-mode` select the MXQ board/runtime mode.
 
+Mask generation models require 1-3 point prompts (`--point X,Y,LABEL`, where
+LABEL is 1 for positive and 0 for negative) and load two artifacts, overridden
+with `--encoder-mxq-path`/`--decoder-mxq-path` (or the `--encoder-onnx-path`/
+`--decoder-onnx-path` pair) rather than `--model-path`/`--mxq-path`/`--onnx-path`.
+
 ```bash
 mblt-vision predict --source image.jpg --model yolo11m --conf-thres 0.4 --output result.jpg
 mblt-vision predict --source image.jpg --model yolo11m-pose --target-device regulus-ra --core-mode single
+mblt-vision predict --source image.jpg --model sam2-hiera-large --point 320,240,1
 ```
 
 The corresponding `mblt-model-zoo` commands use the same standalone handlers for

{mblt_vision_python-0.0.1 → mblt_vision_python-0.0.3}/mblt_vision/__init__.py

@@ -11,6 +11,7 @@ from . import depth_estimation as depth_estimation
 from . import face_detection as face_detection
 from . import image_classification as image_classification
 from . import instance_segmentation as instance_segmentation
+from . import mask_generation as mask_generation
 from . import obb as obb
 from . import object_detection as object_detection
 from . import pose_estimation as pose_estimation
@@ -19,13 +20,14 @@ from ._api import list_models as list_models
 from ._api import list_tasks as list_tasks
 from .wrapper import MBLT_Engine as MBLT_Engine
 
-__version__ = "0.0.1"
+__version__ = "0.0.3"
 
 _TASK_MODULES = (
     face_detection,
     depth_estimation,
     image_classification,
     instance_segmentation,
+    mask_generation,
     object_detection,
     obb,
     pose_estimation,
@@ -49,6 +51,7 @@ _PUBLIC_EXPORTS = [
     "depth_estimation",
     "image_classification",
     "instance_segmentation",
+    "mask_generation",
     "object_detection",
     "obb",
     "pose_estimation",

{mblt_vision_python-0.0.1 → mblt_vision_python-0.0.3}/mblt_vision/_tasks.py

@@ -13,6 +13,7 @@ VISION_TASKS: tuple[str, ...] = (
     "obb",
     "pose_estimation",
     "face_detection",
+    "mask_generation",
 )
 
 

{mblt_vision_python-0.0.1 → mblt_vision_python-0.0.3}/mblt_vision/cli/_vision.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import argparse
+import math
 import os
 import sys
 from pathlib import Path
@@ -53,6 +54,39 @@ def parse_target_clusters(value: str | None) -> list[int] | None:
     return clusters or None
 
 
+def parse_point(value: str) -> tuple[float, float, int]:
+    """Parses an `X,Y,LABEL` point prompt for mask generation models."""
+
+    parts = value.split(",")
+    if len(parts) != 3:
+        raise argparse.ArgumentTypeError("expected a point as X,Y,LABEL")
+    try:
+        x, y = float(parts[0]), float(parts[1])
+        label = int(parts[2])
+    except ValueError as exc:
+        raise argparse.ArgumentTypeError(
+            "expected numeric X,Y coordinates and an integer LABEL"
+        ) from exc
+    # `float()` accepts "nan"/"inf", which would otherwise reach Fourier prompt
+    # encoding and contaminate the resulting tokens, masks, and IoU scores.
+    if not (math.isfinite(x) and math.isfinite(y)):
+        raise argparse.ArgumentTypeError("expected finite X,Y coordinates")
+    if label not in (0, 1):
+        raise argparse.ArgumentTypeError(
+            "point LABEL must be 1 (positive) or 0 (negative)"
+        )
+    return (x, y, label)
+
+
+def resolve_cli_task(args: argparse.Namespace) -> str:
+    """Resolves the selected model's task without constructing a runtime."""
+
+    from mblt_vision.wrapper import resolve_model_config
+
+    config = resolve_model_config(args.model, args.model_type)
+    return normalize_vision_task(config["post_cfg"]["task"])
+
+
 def add_common_vision_args(parser: argparse.ArgumentParser) -> None:
     """Adds arguments shared by all vision inference commands."""
 
@@ -275,6 +309,141 @@ def create_vision_engine(args: argparse.Namespace) -> Any:
     )
 
 
+def reject_single_artifact_paths(args: argparse.Namespace) -> None:
+    """Rejects single-artifact model paths for two-artifact mask generation models.
+
+    Raises:
+        SystemExit: If `--model-path`, `--mxq-path`, or `--onnx-path` is set.
+    """
+
+    if (
+        getattr(args, "model_path", "")
+        or getattr(args, "mxq_path", "")
+        or getattr(args, "onnx_path", "")
+    ):
+        raise SystemExit(
+            "Mask generation models load two artifacts; use "
+            "`--encoder-mxq-path`/`--decoder-mxq-path` (or "
+            "`--encoder-onnx-path`/`--decoder-onnx-path` with `--framework onnx`) "
+            "instead of `--model-path`/`--mxq-path`/`--onnx-path`."
+        )
+
+
+MASK_GENERATION_ONLY_OPTIONS: tuple[tuple[str, str], ...] = (
+    ("encoder_mxq_path", "--encoder-mxq-path"),
+    ("decoder_mxq_path", "--decoder-mxq-path"),
+    ("encoder_onnx_path", "--encoder-onnx-path"),
+    ("decoder_onnx_path", "--decoder-onnx-path"),
+    ("prompt_weights_path", "--prompt-weights-path"),
+)
+
+
+def reject_mask_generation_only_options(args: argparse.Namespace) -> None:
+    """Rejects two-artifact overrides for models that do not accept them.
+
+    The generic engine never receives these paths, so accepting them silently
+    downloads and runs the default single artifact instead of the requested
+    local one.
+
+    Raises:
+        SystemExit: If any mask-generation-only artifact override is set.
+    """
+
+    supplied = [
+        flag
+        for attribute, flag in MASK_GENERATION_ONLY_OPTIONS
+        if getattr(args, attribute, "")
+    ]
+    if supplied:
+        raise SystemExit(
+            f"{', '.join(supplied)} is only supported for mask generation models "
+            "such as SAM2HieraLarge. Use `--model-path`, `--mxq-path`, or "
+            "`--onnx-path` for this model."
+        )
+
+
+def create_mask_generation_engine(args: argparse.Namespace) -> Any:
+    """Creates a promptable mask generation engine from shared CLI model options.
+
+    Mask generation models load two artifacts (encoder + decoder): MXQ by
+    default, or ONNX with `--framework onnx`. The shared NPU options
+    (`--dev-no`, `--core-mode`, `--target-cores`, `--target-clusters`) apply
+    to both MXQ backends and are ignored for ONNX inference.
+
+    The single-artifact path options are rejected here rather than in one
+    command's handler, so every command that builds a mask generation engine
+    fails loudly instead of silently evaluating the downloaded default in
+    place of an explicitly requested local artifact.
+    """
+
+    reject_single_artifact_paths(args)
+
+    try:
+        import mblt_vision.mask_generation as mask_generation_module
+        from mblt_vision.wrapper import _model_name_aliasing
+    except ImportError as exc:
+        print(f"Missing dependencies for vision CLI: {exc}", file=sys.stderr)
+        raise SystemExit(2) from exc
+
+    class_name = Path(_model_name_aliasing(args.model)).stem
+    model_class = getattr(mask_generation_module, class_name, None)
+    if model_class is None:
+        raise SystemExit(
+            f"Mask generation model class '{class_name}' is not exported by "
+            "mblt_vision.mask_generation."
+        )
+    return model_class(
+        encoder_mxq_path=getattr(args, "encoder_mxq_path", None) or None,
+        decoder_mxq_path=getattr(args, "decoder_mxq_path", None) or None,
+        prompt_weights_path=getattr(args, "prompt_weights_path", None) or None,
+        encoder_dev_no=args.dev_no,
+        decoder_dev_no=args.dev_no,
+        encoder_core_mode=args.core_mode,
+        decoder_core_mode=args.core_mode,
+        encoder_target_cores=args.target_cores,
+        decoder_target_cores=args.target_cores,
+        encoder_target_clusters=args.target_clusters,
+        decoder_target_clusters=args.target_clusters,
+        target_device=args.target_device,
+        framework=args.framework,
+        encoder_onnx_path=getattr(args, "encoder_onnx_path", None) or None,
+        decoder_onnx_path=getattr(args, "decoder_onnx_path", None) or None,
+    )
+
+
+def run_mask_generation_inference(
+    args: argparse.Namespace,
+    *,
+    command: str,
+) -> Any:
+    """Runs point-prompted mask generation for a CLI command."""
+
+    point_prompts = getattr(args, "points", None) or []
+    if not 1 <= len(point_prompts) <= 3:
+        raise SystemExit(
+            "Mask generation requires 1 to 3 point prompts; pass `--point X,Y,LABEL` "
+            "(LABEL 1 positive, 0 negative) up to three times."
+        )
+    reject_single_artifact_paths(args)
+
+    points = [[x, y] for x, y, _ in point_prompts]
+    labels = [label for _, _, label in point_prompts]
+    model = create_mask_generation_engine(args)
+    try:
+        result = model.predict(args.source, points, labels)
+        save_path = resolve_output_path(args.output, command, args.source, args.model)
+        result.plot(source_path=args.source, save_path=save_path)
+        iou_text = ", ".join(f"{float(value):.4f}" for value in result.iou_predictions)
+        print(
+            f"Predicted IoU per mask candidate: [{iou_text}]; "
+            f"selected mask index: {result.selected}"
+        )
+        print(f"Saved result to {os.path.relpath(save_path)}")
+        return result
+    finally:
+        model.dispose()
+
+
 def run_vision_inference(
     args: argparse.Namespace,
     *,
@@ -283,6 +452,14 @@ def run_vision_inference(
     """Runs a complete vision inference pipeline for a CLI command."""
 
     require_source_file(args.source)
+    if resolve_cli_task(args) == "mask_generation":
+        return run_mask_generation_inference(args, command=command)
+    if getattr(args, "points", None):
+        raise SystemExit(
+            "`--point` is only supported for mask generation models such as "
+            "SAM2HieraLarge."
+        )
+    reject_mask_generation_only_options(args)
     model = create_vision_engine(args)
     try:
         actual_task = normalize_vision_task(model.post_cfg.get("task", ""))

{mblt_vision_python-0.0.1 → mblt_vision_python-0.0.3}/mblt_vision/cli/predict.py

@@ -8,6 +8,7 @@ from ._vision import (
     add_e2e_arg,
     add_threshold_args,
     add_vision_parser,
+    parse_point,
     run_vision_inference,
 )
 
@@ -29,7 +30,7 @@ def add_predict_parser(
         command="predict",
         help_text=(
             "Run vision inference for classification, depth estimation, detection, instance or semantic "
-            "segmentation, OBB, pose, and face detection."
+            "segmentation, OBB, pose, face detection, and point-prompted mask generation."
         ),
         description=(
             "Run a configured Vision model on one image. The selected model determines the task, "
@@ -37,18 +38,26 @@ def add_predict_parser(
         ),
         epilog="""Supported tasks:
   image classification, depth estimation, object and face detection, instance
-  and semantic segmentation, oriented bounding boxes (OBB), and pose estimation.
+  and semantic segmentation, oriented bounding boxes (OBB), pose estimation,
+  and point-prompted mask generation (SAM2).
 
 The command downloads the default MXQ artifact when no local model path is supplied,
 then writes a plotted result under runs/vision/predict/ by default. Use --output to
 choose the image destination. Use --framework onnx with --model-path or --onnx-path
 for ONNX Runtime inference; MXQ is the default framework.
 
+Mask generation models take 1-3 point prompts (--point X,Y,LABEL; LABEL 1 positive,
+0 negative) and load two artifacts: MXQ by default (overridable with
+--encoder-mxq-path/--decoder-mxq-path) or ONNX with --framework onnx (overridable
+with --encoder-onnx-path/--decoder-onnx-path) instead of --model-path/--mxq-path.
+
 Examples:
   mblt-vision predict --source image.jpg --model resnet50 --topk 3
   mblt-vision predict --source image.jpg --model yolo11m --conf-thres 0.4 --output result.jpg
   mblt-vision predict --source image.jpg --model yolo11m --framework onnx
   mblt-vision predict --source image.jpg --model yolo11m-pose --target-device regulus-ra --core-mode single
+  mblt-vision predict --source image.jpg --model sam2-hiera-large --point 320,240,1
+  mblt-vision predict --source image.jpg --model sam2-hiera-large --point 320,240,1 --framework onnx
 
 For export-style YOLO output, use --e2e false and optionally save it with --raw-output.""",
         handler=_cmd_predict,
@@ -60,5 +69,46 @@ For export-style YOLO output, use --e2e false and optionally save it with --raw-
         "--raw-output",
         help="Path to save raw export-style output with `--e2e false`.",
     )
+    parser.add_argument(
+        "--point",
+        action="append",
+        dest="points",
+        type=parse_point,
+        metavar="X,Y,LABEL",
+        help=(
+            "Point prompt for mask generation models: pixel X,Y in the source image "
+            "and LABEL (1 positive, 0 negative). Repeat up to 3 times."
+        ),
+    )
+    parser.add_argument(
+        "--encoder-mxq-path",
+        dest="encoder_mxq_path",
+        default="",
+        help="Optional local encoder MXQ path for mask generation models.",
+    )
+    parser.add_argument(
+        "--decoder-mxq-path",
+        dest="decoder_mxq_path",
+        default="",
+        help="Optional local decoder MXQ path for mask generation models.",
+    )
+    parser.add_argument(
+        "--encoder-onnx-path",
+        dest="encoder_onnx_path",
+        default="",
+        help="Optional local encoder ONNX path for mask generation models.",
+    )
+    parser.add_argument(
+        "--decoder-onnx-path",
+        dest="decoder_onnx_path",
+        default="",
+        help="Optional local decoder ONNX path for mask generation models.",
+    )
+    parser.add_argument(
+        "--prompt-weights-path",
+        dest="prompt_weights_path",
+        default="",
+        help="Optional local prompt-encoder weights path for mask generation models.",
+    )
     add_threshold_args(parser, conf_default=0.25, iou_default=None)
     add_e2e_arg(parser)