@pikaa-ai/pikaa 0.2.5 → 0.3.1

package/skills/matlab/references/python-integration.md

@@ -0,0 +1,248 @@
+# MATLAB and Python Integration
+
+This reference is pinned to MATLAB R2026a as reviewed on 2026-07-23.
+Calling Python from MATLAB and calling MATLAB from Python are different
+interfaces with different process, data, and license behavior.
+
+## Exact R2026a compatibility
+
+MathWorks' support table lists 64-bit CPython **3.9, 3.10, 3.11, 3.12,
+and 3.13** for:
+
+- MATLAB Interface to Python;
+- MATLAB Engine for Python;
+- MATLAB Compiler SDK for Python;
+- MATLAB Production Server Client Library.
+
+The current MathWorks-maintained PyPI release reviewed here is
+`matlabengine==26.1.12`, released 2026-05-08. It requires MATLAB R2026a.
+Do not install a floating latest package for a reproducible environment.
+
+```bash
+uv pip install "matlabengine==26.1.12"
+```
+
+Installation does not include MATLAB or grant a license. Engine requires an
+installed R2026a on the same machine; MATLAB Runtime alone is insufficient.
+The Python architecture must match MATLAB.
+
+R2026a also ships a preinstalled Engine distribution at the single named path:
+
+```text
+<matlabroot>/extern/engines/python/dist
+```
+
+Add only that confirmed path to the selected environment when using this
+method. Do not print or upload the full `PATH`, `PYTHONPATH`, environment,
+license configuration, home directory, or credentials.
+
+## Plan compatibility without execution
+
+```bash
+python scripts/plan_python_compatibility.py \
+  --matlab-release R2026a \
+  --python-version 3.13 \
+  --engine-version 26.1.12
+```
+
+The planner uses a bundled dated support table. It does not import
+`matlab.engine`, inspect the environment, locate installations, start MATLAB,
+or check out a license. `--include-launch-snippet` adds an explicitly labeled
+launch example to the JSON plan but still does not execute it.
+
+## Calling Python from MATLAB
+
+Review and pin one interpreter before any `py.*` access:
+
+```matlab
+environment = pyenv( ...
+    Version="/reviewed/venv/bin/python", ...
+    ExecutionMode="OutOfProcess");
+```
+
+On Windows, a registered version can be selected by version, but a full named
+executable is clearer. On macOS/Linux, use the full executable. The Python
+build architecture must match MATLAB. MATLAB does not support CPython from the
+Microsoft Store.
+
+Interpreter switching:
+
+- in-process: restart MATLAB before changing the loaded interpreter;
+- out-of-process: `terminate(pyenv)` can stop the external interpreter, after
+  which `pyenv` can be reconfigured.
+
+Out-of-process isolates interpreter crashes and allows reload, but it is not a
+security sandbox. Data plus transfer metadata are limited to 2 GiB per
+out-of-process transfer. Python modules can perform arbitrary process, file,
+network, native, and credential operations.
+
+Never run untrusted `py.*`, `pyrun`, `pyrunfile`, Python modules, wheels, or
+requirements files. `pyrun` and `pyrunfile` are dynamic code execution
+surfaces.
+
+## Calling MATLAB from Python
+
+Starting Engine is explicit execution:
+
+```python
+import matlab.engine
+
+engine = matlab.engine.start_matlab()
+try:
+    result = engine.sqrt(16.0)
+finally:
+    engine.quit()
+```
+
+`start_matlab()` creates a MATLAB process, can execute startup code, and can
+check out a MATLAB license. Never call it merely to probe availability.
+Review startup paths/actions and confirm entitlement first.
+
+`connect_matlab()` connects to a deliberately shared local MATLAB session;
+`find_matlab()` lists shared sessions. Sharing changes the trust boundary and
+must be explicitly approved. Do not connect to an unknown session.
+
+Only add narrow reviewed paths:
+
+```python
+engine.addpath("/reviewed/project/src", nargout=0)
+value = engine.analyzeSignal(
+    matlab.double([[1.0], [2.0], [3.0]]),
+    nargout=1,
+)
+```
+
+Do not use recursive path additions. Pass fixed function names, not text to
+`eval`, and set `nargout` deliberately. Redirect output to bounded
+`io.StringIO` only when needed; logs can contain paths or data.
+
+## MATLAB-to-Python conversion in R2026a
+
+Scalar automatic mappings include:
+
+| MATLAB | Python |
+|---|---|
+| real `double`/`single` | `float` by default; type hints can select `int` |
+| complex float | `complex` |
+| integer scalar | `int` |
+| logical scalar | `bool` |
+| string scalar/character vector | `str` |
+| missing string | `None`-like string conversion documented by MathWorks |
+| `dictionary`/scalar `struct` | `dict` |
+| `table`/`timetable` | pandas `DataFrame` |
+| `datetime` | `datetime.datetime` |
+| `duration` | `datetime.timedelta` |
+
+With NumPy available, numeric/logical MATLAB arrays convert to NumPy arrays
+with corresponding precision/sign/complex dtype. Without NumPy, numeric arrays
+use Python buffer/memoryview behavior. Since R2025a, array conversion behavior
+changed; test code migrated from older vector `array.array` assumptions.
+
+R2026a additions:
+
+- MATLAB string vectors automatically convert to Python lists;
+- `pystringarray` converts MATLAB string arrays to NumPy `StringDType`
+  arrays;
+- missing string entries need explicit round-trip tests.
+
+No automatic conversion is documented for multidimensional character/cell
+arrays or M-by-N string arrays where both dimensions exceed one. Sparse
+arrays, nonscalar structure arrays, categorical arrays, `containers.Map`,
+MATLAB objects, and metadata classes are unsupported in the MATLAB-to-Python
+interface.
+
+## Python-to-MATLAB conversion
+
+MATLAB automatically converts selected scalar Python returns. Other values use
+explicit conversion:
+
+| Python value | MATLAB conversion |
+|---|---|
+| `py.str` | `string` or `char` |
+| Python numeric scalar | `double`, `single`, or integer constructors |
+| `py.bytes` | `uint8` |
+| `py.numpy.ndarray` | matching MATLAB numeric class or `string` where supported |
+| `py.list`/`py.tuple` | numeric/logical/string/cell conversion when homogeneous/compatible |
+| mapping protocol / `py.dict` | `dictionary` or `struct` |
+| pandas `DataFrame` | `table`/`timetable` with documented conversion |
+| Python datetime/timedelta/NumPy time | MATLAB `datetime`/`duration` conversions |
+
+Always test:
+
+- rank and row/column orientation;
+- C-order versus column-major interpretation;
+- dtype width/sign and complex values;
+- `NaN`, `Inf`, `None`, `NaT`, missing strings, and categorical values;
+- table index versus timetable row times;
+- time zones and units;
+- dictionary key restrictions and column-name normalization;
+- copies versus shared/buffer-backed memory.
+
+Do not flatten an array to "fix" a shape mismatch without recording the
+ordering contract.
+
+## MATLAB Engine array classes
+
+The `matlab` Python package provides MATLAB array classes such as
+`matlab.double`, `matlab.single`, signed/unsigned integer classes, and
+`matlab.logical`. These are for Engine calls, not general NumPy replacements.
+
+```python
+column = matlab.double([[1.0], [2.0], [3.0]])
+matrix = matlab.double([[1.0, 2.0], [3.0, 4.0]])
+```
+
+Construct the intended dimensions explicitly. Function outputs can be Engine
+proxy/object types; convert only through documented APIs.
+
+## Serialization and exchange
+
+- MATLAB does not support saving Python objects into MAT files.
+- Never use Python pickle for MATLAB exchange. Pickle deserialization executes
+  attacker-controlled behavior.
+- For simple data, prefer schema-documented CSV/JSON/Parquet/HDF5.
+- For trusted MATLAB arrays, use MAT with explicit version and inventory it
+  before loading in another process.
+- SciPy `loadmat` is not used by this skill's inventory. It can deserialize
+  complex structures and is not a safety scanner.
+
+## Compiler SDK distinction
+
+MATLAB Compiler SDK for Python packages are not MATLAB Engine:
+
+- building requires MATLAB, MATLAB Compiler SDK, and source dependencies;
+- deployed components use a compatible MATLAB Runtime under applicable terms;
+- generated Python packages are not supported as Python modules called back
+  from MATLAB's Python interface;
+- Engine itself requires full installed MATLAB, not Runtime.
+
+Confirm product, target release, platform, package, runtime, and license terms.
+
+## Troubleshooting without broad disclosure
+
+Collect only named facts:
+
+- MATLAB release/update and architecture;
+- one Python executable path and `major.minor`;
+- Engine package version;
+- selected `pyenv` status/mode (not all environment values);
+- one failing function, input classes/shapes, and redacted traceback;
+- whether NumPy/pandas are required and their pinned versions.
+
+Do not ask for `env`, `set`, complete `PATH`, complete `sys.path`, license
+files, tokens, home-directory listings, or credentials.
+
+## Sources (verified 2026-07-23)
+
+- [Python Compatibility by MATLAB Release](https://www.mathworks.com/support/requirements/python-compatibility.html)
+- [Install MATLAB Engine API for Python](https://www.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html)
+- [MathWorks `matlabengine` 26.1.12 package](https://pypi.org/project/matlabengine/26.1.12/)
+- [Call MATLAB from Python](https://www.mathworks.com/help/matlab/matlab-engine-for-python.html)
+- [Get Started with MATLAB Engine](https://www.mathworks.com/help/matlab/matlab_external/get-started-with-matlab-engine-for-python.html)
+- [Configure Python for MATLAB](https://www.mathworks.com/help/matlab/matlab_external/install-supported-python-implementation.html)
+- [Call Python from MATLAB](https://www.mathworks.com/help/matlab/call-python-libraries.html)
+- [Pass Data from MATLAB to Python](https://www.mathworks.com/help/matlab/matlab_external/passing-data-to-python.html)
+- [Pass Data from Python to MATLAB](https://www.mathworks.com/help/matlab/matlab_external/pass-data-between-matlab-and-python-from-python.html)
+- [Python Interface Limitations](https://www.mathworks.com/help/matlab/matlab_external/limitations-to-python-support.html)
+- [MATLAB Engine Limitations](https://www.mathworks.com/help/matlab/matlab_external/limitations-to-the-matlab-engine-for-python.html)
+- [R2026a Release Highlights](https://www.mathworks.com/products/new_products/latest_features.html)

package/skills/matlab/scripts/_common.py

@@ -0,0 +1,263 @@
+#!/usr/bin/env python3
+"""Shared bounded local-only helpers for the MATLAB skill CLIs."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import re
+import sys
+from pathlib import Path
+from typing import Any, Iterable
+
+MAX_INPUT_BYTES = 64 * 1024 * 1024
+MAX_TEXT_BYTES = 4 * 1024 * 1024
+MAX_JSON_BYTES = 2 * 1024 * 1024
+MAX_FILES = 500
+MAX_JSON_ITEMS = 20_000
+MAX_JSON_DEPTH = 24
+MAX_PATH_CHARS = 4096
+MATLAB_IDENTIFIER = re.compile(r"^[A-Za-z][A-Za-z0-9_]{0,62}$")
+MATLAB_RELEASE = re.compile(r"^R(20[0-9]{2})([ab])$")
+URL_LIKE = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*://")
+
+
+class CliError(ValueError):
+    """Expected, user-actionable CLI failure."""
+
+
+def _raw_path(value: str) -> Path:
+    if not value or len(value) > MAX_PATH_CHARS:
+        raise CliError("path is empty or exceeds the length limit")
+    if "\x00" in value or URL_LIKE.match(value):
+        raise CliError("only local filesystem paths are accepted")
+    return Path(value)
+
+
+def _reject_symlink_chain(path: Path) -> None:
+    """Reject every existing symlink component without following it."""
+    absolute = path.absolute()
+    current = Path(absolute.anchor)
+    for part in absolute.parts[1:]:
+        current = current / part
+        if current.exists() and current.is_symlink():
+            raise CliError(f"symlink paths are not accepted: {current}")
+
+
+def checked_root(value: str | Path) -> Path:
+    raw = _raw_path(str(value))
+    _reject_symlink_chain(raw)
+    try:
+        root = raw.resolve(strict=True)
+    except OSError as exc:
+        raise CliError(f"root does not exist or is inaccessible: {raw}") from exc
+    if not root.is_dir():
+        raise CliError(f"root is not a directory: {root}")
+    return root
+
+
+def checked_input(
+    value: str | Path,
+    *,
+    root: Path,
+    kind: str = "file",
+    suffixes: Iterable[str] | None = None,
+    max_bytes: int = MAX_INPUT_BYTES,
+) -> Path:
+    raw = _raw_path(str(value))
+    candidate = raw if raw.is_absolute() else root / raw
+    _reject_symlink_chain(candidate)
+    try:
+        path = candidate.resolve(strict=True)
+        path.relative_to(root)
+    except (OSError, ValueError) as exc:
+        raise CliError(f"input must exist within root {root}") from exc
+    if kind == "file" and not path.is_file():
+        raise CliError(f"input is not a regular file: {path}")
+    if kind == "directory" and not path.is_dir():
+        raise CliError(f"input is not a directory: {path}")
+    if kind == "any" and not (path.is_file() or path.is_dir()):
+        raise CliError(f"input is not a regular file or directory: {path}")
+    if kind not in {"file", "directory", "any"}:
+        raise CliError(f"unsupported input kind: {kind}")
+    if suffixes is not None and path.is_file():
+        allowed = {suffix.casefold() for suffix in suffixes}
+        if path.suffix.casefold() not in allowed:
+            raise CliError(
+                f"unsupported suffix {path.suffix!r}; expected one of {sorted(allowed)}"
+            )
+    if path.is_file():
+        try:
+            size = path.stat().st_size
+        except OSError as exc:
+            raise CliError(f"cannot stat input: {path}") from exc
+        if size > max_bytes:
+            raise CliError(f"input is {size} bytes; limit is {max_bytes}")
+    return path
+
+
+def checked_output(
+    value: str | Path,
+    *,
+    root: Path,
+    suffixes: Iterable[str] | None = None,
+) -> Path:
+    raw = _raw_path(str(value))
+    candidate = raw if raw.is_absolute() else root / raw
+    _reject_symlink_chain(candidate)
+    try:
+        parent = candidate.parent.resolve(strict=True)
+        parent.relative_to(root)
+    except (OSError, ValueError) as exc:
+        raise CliError(f"output parent must exist within root {root}") from exc
+    path = parent / candidate.name
+    if path.exists() or path.is_symlink():
+        raise CliError(f"refusing to overwrite existing output: {path}")
+    if suffixes is not None:
+        allowed = {suffix.casefold() for suffix in suffixes}
+        if path.suffix.casefold() not in allowed:
+            raise CliError(
+                f"unsupported output suffix {path.suffix!r}; "
+                f"expected one of {sorted(allowed)}"
+            )
+    return path
+
+
+def read_bytes(path: Path, *, max_bytes: int = MAX_INPUT_BYTES) -> bytes:
+    try:
+        size = path.stat().st_size
+        if size > max_bytes:
+            raise CliError(f"input is {size} bytes; limit is {max_bytes}")
+        return path.read_bytes()
+    except CliError:
+        raise
+    except OSError as exc:
+        raise CliError(f"cannot read input: {path}") from exc
+
+
+def read_text(path: Path, *, max_bytes: int = MAX_TEXT_BYTES) -> str:
+    payload = read_bytes(path, max_bytes=max_bytes)
+    try:
+        return payload.decode("utf-8")
+    except UnicodeDecodeError as exc:
+        raise CliError(f"input is not UTF-8 text: {path}") from exc
+
+
+def _reject_duplicate_keys(pairs: list[tuple[str, Any]]) -> dict[str, Any]:
+    output: dict[str, Any] = {}
+    for key, value in pairs:
+        if key in output:
+            raise CliError(f"duplicate JSON key: {key!r}")
+        output[key] = value
+    return output
+
+
+def _validate_json_shape(value: Any, *, depth: int = 0) -> int:
+    if depth > MAX_JSON_DEPTH:
+        raise CliError(f"JSON nesting exceeds {MAX_JSON_DEPTH}")
+    if value is None or isinstance(value, (bool, int, float, str)):
+        if isinstance(value, str) and len(value) > 100_000:
+            raise CliError("JSON string exceeds 100000 characters")
+        return 1
+    if isinstance(value, list):
+        return 1 + sum(_validate_json_shape(item, depth=depth + 1) for item in value)
+    if isinstance(value, dict):
+        if not all(isinstance(key, str) for key in value):
+            raise CliError("JSON object keys must be strings")
+        return 1 + sum(
+            _validate_json_shape(item, depth=depth + 1) for item in value.values()
+        )
+    raise CliError(f"unsupported JSON value type: {type(value).__name__}")
+
+
+def parse_json_text(text: str) -> Any:
+    try:
+        value = json.loads(text, object_pairs_hook=_reject_duplicate_keys)
+    except CliError:
+        raise
+    except (json.JSONDecodeError, ValueError) as exc:
+        raise CliError(f"invalid JSON: {exc}") from exc
+    item_count = _validate_json_shape(value)
+    if item_count > MAX_JSON_ITEMS:
+        raise CliError(f"JSON has {item_count} values; limit is {MAX_JSON_ITEMS}")
+    return value
+
+
+def load_json(path: Path) -> Any:
+    return parse_json_text(read_text(path, max_bytes=MAX_JSON_BYTES))
+
+
+def bounded_int(
+    value: str | int,
+    *,
+    name: str,
+    minimum: int,
+    maximum: int,
+) -> int:
+    try:
+        number = int(value)
+    except (TypeError, ValueError) as exc:
+        raise CliError(f"{name} must be an integer") from exc
+    if number < minimum or number > maximum:
+        raise CliError(f"{name} must be between {minimum} and {maximum}")
+    return number
+
+
+def validate_identifier(value: str, *, name: str = "identifier") -> str:
+    if not MATLAB_IDENTIFIER.fullmatch(value):
+        raise CliError(
+            f"{name} must be a MATLAB identifier of at most 63 characters"
+        )
+    return value
+
+
+def validate_release(value: str) -> str:
+    if not MATLAB_RELEASE.fullmatch(value):
+        raise CliError("MATLAB release must look like R2026a")
+    return value
+
+
+def sha256_file(path: Path, *, max_bytes: int = MAX_INPUT_BYTES) -> str:
+    size = path.stat().st_size
+    if size > max_bytes:
+        raise CliError(f"input is {size} bytes; hash limit is {max_bytes}")
+    digest = hashlib.sha256()
+    try:
+        with path.open("rb") as handle:
+            while chunk := handle.read(64 * 1024):
+                digest.update(chunk)
+    except OSError as exc:
+        raise CliError(f"cannot hash input: {path}") from exc
+    return digest.hexdigest()
+
+
+def relative_id(path: Path, root: Path) -> str:
+    return path.relative_to(root).as_posix()
+
+
+def write_new_text(path: Path, text: str) -> None:
+    try:
+        with path.open("x", encoding="utf-8", newline="\n") as handle:
+            handle.write(text)
+    except FileExistsError as exc:
+        raise CliError(f"refusing to overwrite existing output: {path}") from exc
+    except OSError as exc:
+        raise CliError(f"cannot write output: {path}") from exc
+
+
+def emit_json(value: Any) -> None:
+    json.dump(value, sys.stdout, indent=2, sort_keys=True, ensure_ascii=False)
+    sys.stdout.write("\n")
+
+
+def fail_json(tool: str, exc: Exception) -> int:
+    emit_json(
+        {
+            "error": str(exc),
+            "executes_external_code": False,
+            "network_accessed": False,
+            "ok": False,
+            "tool": tool,
+        }
+    )
+    return 2

package/skills/matlab/scripts/generate_function_scaffold.py

@@ -0,0 +1,165 @@
+#!/usr/bin/env python3
+"""Dry-run or write deterministic MATLAB function and unit-test scaffolds."""
+
+from __future__ import annotations
+
+import argparse
+import re
+import sys
+from pathlib import Path
+from typing import Any
+
+from _common import (
+    CliError,
+    checked_input,
+    checked_root,
+    emit_json,
+    fail_json,
+    relative_id,
+    validate_identifier,
+    write_new_text,
+)
+
+TOOL = "generate_function_scaffold"
+SAFE_DIRECTORY = re.compile(r"^[A-Za-z][A-Za-z0-9_.-]{0,63}$")
+
+
+def _directory(root: Path, name: str, *, write: bool) -> Path:
+    if not SAFE_DIRECTORY.fullmatch(name):
+        raise CliError("source/test directory must be one simple local directory name")
+    path = root / name
+    if path.exists():
+        return checked_input(path, root=root, kind="directory")
+    if not write:
+        return path
+    try:
+        path.mkdir()
+    except FileExistsError:
+        return checked_input(path, root=root, kind="directory")
+    except OSError as exc:
+        raise CliError(f"cannot create directory: {path}") from exc
+    return path
+
+
+def _title_name(name: str) -> str:
+    return name[0].upper() + name[1:]
+
+
+def function_source(name: str) -> str:
+    upper = name.upper()
+    return f"""function result = {name}(data, options)
+%{upper} Scale a finite numeric matrix.
+%   RESULT = {upper}(DATA, Scale=VALUE) multiplies DATA by VALUE.
+
+arguments
+    data (:,:) double {{mustBeFinite}}
+    options.Scale (1,1) double {{mustBeFinite}} = 1
+end
+
+result = data .* options.Scale;
+end
+"""
+
+
+def test_source(name: str, class_name: str) -> str:
+    return f"""classdef {class_name} < matlab.unittest.TestCase
+    methods (Test)
+        function scalesFiniteMatrix(testCase)
+            actual = {name}([1 2; 3 4], Scale=2);
+            expected = [2 4; 6 8];
+            testCase.verifyEqual(actual, expected, AbsTol=1e-14);
+        end
+
+        function preservesShape(testCase)
+            actual = {name}(zeros(2, 3));
+            testCase.verifySize(actual, [2 3]);
+        end
+    end
+end
+"""
+
+
+def generate(args: argparse.Namespace) -> dict[str, Any]:
+    root = checked_root(args.root)
+    name = validate_identifier(args.name, name="function name")
+    class_name = validate_identifier(
+        "Test" + _title_name(name), name="test class name"
+    )
+    if args.source_dir == args.test_dir:
+        raise CliError("source and test directories must be distinct")
+    if not SAFE_DIRECTORY.fullmatch(args.source_dir) or not SAFE_DIRECTORY.fullmatch(
+        args.test_dir
+    ):
+        raise CliError(
+            "source/test directory must be one simple local directory name"
+        )
+    source_dir = _directory(root, args.source_dir, write=args.write)
+    test_dir = _directory(root, args.test_dir, write=args.write)
+    function_path = source_dir / f"{name}.m"
+    test_path = test_dir / f"{class_name}.m"
+    for path in (function_path, test_path):
+        if path.exists() or path.is_symlink():
+            raise CliError(f"refusing to overwrite existing scaffold: {path}")
+    function_text = function_source(name)
+    test_text = test_source(name, class_name)
+    if args.write:
+        write_new_text(function_path, function_text)
+        try:
+            write_new_text(test_path, test_text)
+        except Exception:
+            try:
+                function_path.unlink(missing_ok=True)
+            except OSError:
+                pass
+            raise
+    return {
+        "contents": {
+            "function": function_text,
+            "test": test_text,
+        },
+        "dry_run": not args.write,
+        "executes": False,
+        "files": {
+            "function": relative_id(function_path, root),
+            "test": relative_id(test_path, root),
+        },
+        "network_accessed": False,
+        "ok": True,
+        "requires_matlab_to_run_tests": True,
+        "target_release": "R2026a",
+        "tool": TOOL,
+        "wrote_files": bool(args.write),
+        "warning": (
+            "Generated text is a starting point. Review domain semantics, "
+            "tolerances, products, paths, and safety before execution."
+        ),
+    }
+
+
+def parser() -> argparse.ArgumentParser:
+    result = argparse.ArgumentParser(
+        description=(
+            "Generate a MATLAB R2026a function and matlab.unittest class. "
+            "Default is dry-run; --write refuses collisions."
+        )
+    )
+    result.add_argument("name", help="MATLAB function identifier")
+    result.add_argument("--root", default=".", help="allowed project root")
+    result.add_argument("--source-dir", default="src")
+    result.add_argument("--test-dir", default="tests")
+    result.add_argument(
+        "--write", action="store_true", help="create new files and directories"
+    )
+    return result
+
+
+def main() -> int:
+    try:
+        emit_json(generate(parser().parse_args()))
+        return 0
+    except CliError as exc:
+        return fail_json(TOOL, exc)
+
+
+if __name__ == "__main__":
+    sys.exit(main())