pycatalyst 0.0.7.dev0__tar.gz → 0.0.9__tar.gz

pycatalyst-0.0.9/.github/workflows/README.md

@@ -0,0 +1,87 @@
+# GitHub Actions in PyCatalyst
+
+## What should happen when you create a PR (develop → main)
+
+When you open a **pull request that targets `main` or `develop`**:
+
+1. The **Test** workflow should run (`.github/workflows/test.yml`).
+2. You get **3 matrix jobs** (Python 3.11, 3.12, 3.13), each running:
+   - Install deps (`pip install -e ".[dev]"`)
+   - `pytest`
+   - Ruff format + Ruff check (linting)
+   - Mypy (type checking)
+   - Node/npm for UI (if present): `npm ci`, `npm run type-check`
+3. The run appears under the **Checks** tab of the PR and in **Actions**.
+
+Trigger in `test.yml` is:
+
+```yaml
+on:
+  pull_request:
+    branches: [main, develop]
+```
+
+So the PR’s **base (target) branch** must be `main` or `develop`. A PR **develop → main** has base `main`, so it qualifies.
+
+---
+
+## Why the workflow might not run at all
+
+### 1. Workflow file not on the PR’s **source** branch (most common)
+
+GitHub runs the workflow that exists on the **head** of the PR (the branch you’re merging **from**).
+
+- PR: **develop → main**
+- GitHub uses the workflow from the **develop** branch.
+- If `.github/workflows/test.yml` is **not** on `develop` (e.g. you only added it on `main`), **no workflow run is created**.
+
+**Fix:** Ensure the workflow is on the branch you use for the PR:
+
+- Merge `main` into `develop` and push, then open (or update) the PR, or
+- Cherry-pick/add the workflow on `develop` and push.
+
+After that, new pushes to the PR or reopening it should trigger the Test workflow.
+
+### 2. GitHub Actions disabled or restricted
+
+- **Settings → Actions → General**
+  - “Actions permissions”: should allow “Allow all actions and reusable workflows” (or your intended choice).
+  - If “Disable actions” is selected, no workflows run.
+- **Fork PRs:** If the PR is from a **fork**, check “Fork pull request workflows from outside collaborators” (or equivalent); if disabled, workflows from fork PRs may not run.
+
+### 3. Default branch / branch name
+
+- The trigger only lists `main` and `develop`. If your default branch is something else (e.g. `master`) and you open a PR **into** that branch, the workflow will not run unless you add that branch to `branches:` in `test.yml`.
+
+### 4. Credits / billing (unlikely to prevent the run from starting)
+
+- **Public repos:** GitHub Actions minutes are free; you don’t run out of “credits” in a way that would stop the workflow from **starting**.
+- **Private repos:** Free tier has a monthly minutes limit. Exhausting it or having a **spending limit set to $0** can block new runs. You’d usually see a billing warning in the UI or email; the run might show as “canceled” or not start.
+- If **no run appears at all** (no “Test” workflow in the Actions tab for that PR), the usual cause is that the workflow didn’t trigger (e.g. workflow file not on the source branch or Actions disabled), not billing.
+
+---
+
+## Quick checklist when no runs appear for “develop → main”
+
+1. **Workflow on `develop`**
+   On GitHub, open the repo, switch to branch **develop**, and confirm `.github/workflows/test.yml` exists and has the `pull_request: branches: [main, develop]` trigger.
+
+2. **PR base branch**
+   The PR must target **main** (or **develop**). If it targets another branch, the current trigger won’t run.
+
+3. **Actions enabled**
+   **Settings → Actions → General** → Actions are not disabled.
+
+4. **Fork (if applicable)**
+   If the PR is from a fork, ensure “Fork pull request workflows” (or similar) is allowed.
+
+5. **Push after fixing**
+   If you just merged `main` into `develop` to get the workflow file onto `develop`, push that merge (or any new commit) to the PR branch so the workflow is re-evaluated.
+
+---
+
+## Summary
+
+- **Expected:** A PR that targets `main` or `develop` (e.g. develop → main) should trigger the **Test** workflow and run tests on Python 3.11, 3.12, and 3.13.
+- **Most likely reason it doesn’t:** The workflow file is not on the PR’s source branch (e.g. not on `develop`). Get `.github/workflows/test.yml` onto that branch (e.g. by merging `main` into `develop`), then push and re-open or update the PR.
+- **Then check:** Settings → Actions and, for fork PRs, fork workflow permissions. Credits are rarely the cause when no run appears at all.

{pycatalyst-0.0.7.dev0 → pycatalyst-0.0.9}/.github/workflows/publish.yml

@@ -1,4 +1,5 @@
-# Build and publish to PyPI when a version tag (v*) is pushed.
+# Build and publish to PyPI when a version tag (v*) is pushed or a
+# GitHub Release is published via the UI.
 # Also supports manual run (workflow_dispatch).
 # Version is derived from the git tag by setuptools-scm.
 
@@ -7,6 +8,8 @@ name: Build and Publish
 on:
   push:
     tags: ['v*']
+  release:
+    types: [published]
   workflow_dispatch:
 
 permissions:
@@ -22,6 +25,12 @@ jobs:
         uses: actions/checkout@v4
         with:
           fetch-depth: 0
+          # Build from the release tag so setuptools-scm produces a full release version (not .dev)
+          ref: ${{ github.event_name == 'release' && github.event.release.tag_name || github.ref }}
+
+      - name: Set version from tag
+        if: startsWith(github.ref, 'refs/tags/')
+        run: echo "SETUPTOOLS_SCM_PRETEND_VERSION=${GITHUB_REF_NAME#v}" >> $GITHUB_ENV
 
       - name: Set up Python
         uses: actions/setup-python@v5
@@ -75,13 +84,15 @@ jobs:
           python -m zipfile -l dist/pycatalyst-*.whl | head -30
           echo "Package built successfully"
 
+      # Only publish to PyPI when building from a tag (official release). Skip for workflow_dispatch so we don't publish dev versions as pre-release.
       - name: Publish to PyPI
+        if: startsWith(github.ref, 'refs/tags/') || github.event_name == 'release'
         uses: pypa/gh-action-pypi-publish@release/v1
         with:
           packages-dir: dist/
 
       - name: Create GitHub Release
-        if: startsWith(github.ref, 'refs/tags/')
+        if: startsWith(github.ref, 'refs/tags/') && github.event_name != 'release'
         uses: softprops/action-gh-release@v2.4.2
         with:
           tag_name: ${{ github.ref_name }}

{pycatalyst-0.0.7.dev0 → pycatalyst-0.0.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pycatalyst
-Version: 0.0.7.dev0
+Version: 0.0.9
 Summary: A schema-aware data generation and testing platform for Python
 Author-email: StatFYI <contact@statfyi.com>
 License-Expression: MIT
@@ -24,6 +24,7 @@ Requires-Python: >=3.11
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: pyyaml>=6.0.0
+Requires-Dist: exrex>=0.11.0
 Provides-Extra: inference
 Requires-Dist: pandas>=2.0.0; extra == "inference"
 Provides-Extra: faker
@@ -59,6 +60,24 @@ Requires-Dist: pycatalyst[api,sandbox]; extra == "workbench"
 Requires-Dist: ipython>=8.0.0; extra == "workbench"
 Provides-Extra: sandbox
 Requires-Dist: docker>=7.0.0; extra == "sandbox"
+Provides-Extra: ml-core
+Requires-Dist: torch>=2.0.0; extra == "ml-core"
+Requires-Dist: numpy>=1.24.0; extra == "ml-core"
+Requires-Dist: pandas>=2.0.0; extra == "ml-core"
+Requires-Dist: matplotlib>=3.7.0; extra == "ml-core"
+Requires-Dist: pydantic>=2.0.0; extra == "ml-core"
+Requires-Dist: joblib>=1.3.0; extra == "ml-core"
+Provides-Extra: ml-sklearn
+Requires-Dist: pycatalyst[ml-core]; extra == "ml-sklearn"
+Requires-Dist: scikit-learn>=1.3.0; extra == "ml-sklearn"
+Provides-Extra: ml-llm
+Requires-Dist: pycatalyst[ml-core]; extra == "ml-llm"
+Requires-Dist: transformers>=4.36.0; extra == "ml-llm"
+Requires-Dist: peft>=0.7.0; extra == "ml-llm"
+Requires-Dist: datasets>=2.16.0; extra == "ml-llm"
+Requires-Dist: accelerate>=0.25.0; extra == "ml-llm"
+Provides-Extra: ml
+Requires-Dist: pycatalyst[ml-core,ml-llm,ml-sklearn]; extra == "ml"
 Provides-Extra: db
 Requires-Dist: alembic>=1.13.0; extra == "db"
 Requires-Dist: sqlalchemy>=2.0.0; extra == "db"
@@ -68,7 +87,7 @@ Provides-Extra: docs
 Requires-Dist: mkdocs-material>=9.5.0; extra == "docs"
 Requires-Dist: mkdocstrings[python]>=0.24.0; extra == "docs"
 Provides-Extra: all
-Requires-Dist: pycatalyst[api,db,docs,faker,http,inference,parquet,postgres,sandbox,ui,workbench,worker]; extra == "all"
+Requires-Dist: pycatalyst[api,db,docs,faker,http,inference,ml,parquet,postgres,sandbox,ui,workbench,worker]; extra == "all"
 Provides-Extra: dev
 Requires-Dist: pycatalyst[all]; extra == "dev"
 Requires-Dist: pytest>=7.0.0; extra == "dev"

{pycatalyst-0.0.7.dev0 → pycatalyst-0.0.9}/pyproject.toml

@@ -12,6 +12,7 @@ requires-python = ">=3.11"
 license = "MIT"
 dependencies = [
     "pyyaml>=6.0.0",
+    "exrex>=0.11.0",
 ]
 authors = [
     { name = "StatFYI", email = "contact@statfyi.com" },
@@ -56,6 +57,10 @@ local_scheme = "no-local-version"
 #   pip install pycatalyst[faker]       -> + realistic fake data via Faker
 #   pip install pycatalyst[http]        -> + HTTP sink
 #   pip install pycatalyst[api]         -> + REST API server
+#   pip install pycatalyst[ml-core]     -> + PyTorch ML training engine
+#   pip install pycatalyst[ml-sklearn]  -> + scikit-learn ML engine
+#   pip install pycatalyst[ml-llm]      -> + HuggingFace LLM fine-tuning
+#   pip install pycatalyst[ml]          -> all ML engines
 #   pip install pycatalyst[all]         -> everything
 #   pip install pycatalyst[dev]         -> all + testing & dev tools
 # ---------------------------------------------------------------------------
@@ -120,6 +125,30 @@ sandbox = [
     "docker>=7.0.0",
 ]
 
+# --- ML training and serving ---
+ml-core = [
+    "torch>=2.0.0",
+    "numpy>=1.24.0",
+    "pandas>=2.0.0",
+    "matplotlib>=3.7.0",
+    "pydantic>=2.0.0",
+    "joblib>=1.3.0",
+]
+ml-sklearn = [
+    "pycatalyst[ml-core]",
+    "scikit-learn>=1.3.0",
+]
+ml-llm = [
+    "pycatalyst[ml-core]",
+    "transformers>=4.36.0",
+    "peft>=0.7.0",
+    "datasets>=2.16.0",
+    "accelerate>=0.25.0",
+]
+ml = [
+    "pycatalyst[ml-core,ml-sklearn,ml-llm]",
+]
+
 # --- Package-specific extras ---
 db = [
     "alembic>=1.13.0",
@@ -136,7 +165,7 @@ docs = [
 
 # --- Aggregate extras ---
 all = [
-    "pycatalyst[inference,faker,parquet,http,api,worker,ui,workbench,sandbox,db,postgres,docs]",
+    "pycatalyst[inference,faker,parquet,http,api,worker,ui,workbench,sandbox,db,postgres,docs,ml]",
 ]
 dev = [
     "pycatalyst[all]",

{pycatalyst-0.0.7.dev0 → pycatalyst-0.0.9}/src/pycatalyst/_types.py

@@ -112,6 +112,9 @@ class FieldConstraints:
         min_items: Minimum number of items for array fields.
         max_items: Maximum number of items for array fields.
         format: Named format hint (e.g., "email", "url", "phone").
+        exclusive_min: If True, generated value must be strictly greater than min.
+        exclusive_max: If True, generated value must be strictly less than max.
+        multiple_of: Generated value must be a multiple of this (int/float).
     """
 
     min: int | float | None = None
@@ -126,6 +129,9 @@ class FieldConstraints:
     min_items: int | None = None
     max_items: int | None = None
     format: str | None = None
+    exclusive_min: bool = False
+    exclusive_max: bool = False
+    multiple_of: int | float | None = None
 
     def __post_init__(self) -> None:
         if self.min is not None and self.max is not None and self.min > self.max:
@@ -166,6 +172,14 @@ class FieldConstraints:
         if distribution and isinstance(distribution, str):
             distribution = Distribution(distribution)
 
+        exclusive_min = config.get(
+            "exclusive_min", config.get("exclusiveMinimum", False)
+        )
+        exclusive_max = config.get(
+            "exclusive_max", config.get("exclusiveMaximum", False)
+        )
+        multiple_of = config.get("multiple_of", config.get("multipleOf"))
+
         return cls(
             min=config.get("min"),
             max=config.get("max"),
@@ -179,6 +193,9 @@ class FieldConstraints:
             min_items=config.get("min_items"),
             max_items=config.get("max_items"),
             format=config.get("format"),
+            exclusive_min=bool(exclusive_min),
+            exclusive_max=bool(exclusive_max),
+            multiple_of=multiple_of,
         )
 
 

{pycatalyst-0.0.7.dev0 → pycatalyst-0.0.9}/src/pycatalyst/api/main.py

@@ -19,7 +19,16 @@ from fastapi.responses import JSONResponse
 
 from pycatalyst import __version__ as catalyst_version
 from pycatalyst.api.dependencies.auth import get_current_user
-from pycatalyst.api.routes.v1 import auth, generate, infer, recipes, settings, workbench
+from pycatalyst.api.routes.v1 import (
+    auth,
+    generate,
+    infer,
+    ml,
+    recipes,
+    services,
+    settings,
+    workbench,
+)
 
 API_VERSION = "v1"
 API_PREFIX = f"/api/{API_VERSION}"
@@ -142,6 +151,20 @@ def create_application() -> FastAPI:
         tags=["recipes"],
         dependencies=auth_dep,
     )
+    app.include_router(
+        ml.router,
+        prefix=API_PREFIX,
+        tags=["ml"],
+        dependencies=auth_dep,
+    )
+
+    # Service containers (database services) — protected
+    app.include_router(
+        services.router,
+        prefix=API_PREFIX,
+        tags=["services"],
+        dependencies=auth_dep,
+    )
 
     # Workbench public (capabilities — no auth)
     app.include_router(

pycatalyst-0.0.9/src/pycatalyst/api/routes/v1/ml.py

@@ -0,0 +1,92 @@
+"""ML experiment endpoint — POST /api/v1/ml/run.
+
+Runs a single experiment from YAML config. Core pycatalyst.ml remains stateless;
+this route owns the HTTP contract and temporary file handling.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import tempfile
+from pathlib import Path
+
+from fastapi import APIRouter, HTTPException, status
+from pydantic import BaseModel, Field
+
+from pycatalyst.ml.run import run_experiment
+
+router = APIRouter()
+
+
+class MlRunRequest(BaseModel):
+    """Request body for running an ML experiment."""
+
+    config: str = Field(..., description="Experiment config as YAML string")
+
+
+class MlRunResponse(BaseModel):
+    """Response from running an ML experiment. Mirrors ExperimentResult."""
+
+    name: str
+    backend: str
+    metrics: dict[str, float] = Field(default_factory=dict)
+    artifacts: dict[str, str] = Field(default_factory=dict)
+    training_history: list[dict] = Field(default_factory=list)
+    duration_seconds: float = 0.0
+
+
+@router.post("/ml/run", response_model=MlRunResponse)
+async def run_ml_experiment(request: MlRunRequest) -> MlRunResponse:
+    """Run an ML experiment from YAML config and return the result.
+
+    Writes config to a temporary file and calls stateless run_experiment() in a
+    thread pool so the event loop is not blocked. Long-running training may
+    hit timeouts; consider increasing gateway/worker timeouts for large jobs.
+    """
+    if not request.config.strip():
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail="config must be a non-empty YAML string",
+        )
+
+    try:
+        with tempfile.NamedTemporaryFile(
+            mode="w",
+            suffix=".yaml",
+            delete=False,
+            encoding="utf-8",
+        ) as f:
+            f.write(request.config)
+            config_path = f.name
+
+        try:
+            result = await asyncio.to_thread(run_experiment, config_path)
+        finally:
+            Path(config_path).unlink(missing_ok=True)
+
+        return MlRunResponse(
+            name=result.name,
+            backend=result.backend,
+            metrics=dict(result.metrics),
+            artifacts=dict(result.artifacts),
+            training_history=list(result.training_history),
+            duration_seconds=result.duration_seconds,
+        )
+    except FileNotFoundError as exc:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail={"message": str(exc)},
+        ) from exc
+    except ValueError as exc:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail={"message": str(exc)},
+        ) from exc
+    except ImportError as exc:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail={
+                "message": str(exc),
+                "hint": "Install the required ML extra (e.g. pycatalyst[ml-core], pycatalyst[ml-sklearn], pycatalyst[ml-llm])",
+            },
+        ) from exc

pycatalyst-0.0.9/src/pycatalyst/api/routes/v1/services.py

@@ -0,0 +1,184 @@
+"""Service container endpoints — CRUD for dockerized database services.
+
+Endpoints for creating, listing, inspecting, and destroying database
+service containers (Postgres, MongoDB) tied to workbench environments.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import Any
+
+from fastapi import APIRouter, Depends, HTTPException, status
+from pydantic import BaseModel, Field, field_validator
+
+from pycatalyst.api.dependencies.auth import get_current_user
+from pycatalyst.errors import ServiceError
+from pycatalyst.services.service_containers import ServiceConfig, ServiceType
+from pycatalyst.services.workbench_manager import workbench_manager
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/workbench")
+
+ALLOWED_SERVICE_TYPES = frozenset(t.value for t in ServiceType)
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _owner_from_user(user: dict[str, Any]) -> str:
+    """Extract the owner identifier from the resolved user dict."""
+    return user.get("username", "anonymous")
+
+
+# ---------------------------------------------------------------------------
+# Request / Response models
+# ---------------------------------------------------------------------------
+
+
+class CreateServiceRequest(BaseModel):
+    """Request to create a database service container."""
+
+    service_type: str = Field(
+        description="Database type: postgres or mongodb",
+    )
+    database: str = Field(
+        default="pycatalyst_db",
+        description="Initial database name to create",
+    )
+    username: str = Field(
+        default="pycatalyst",
+        description="Database admin username",
+    )
+    password: str = Field(
+        default="pycatalyst_dev",
+        description="Database admin password",
+    )
+
+    @field_validator("service_type")
+    @classmethod
+    def _validate_service_type(cls, v: str) -> str:
+        val = v.strip().lower()
+        if val not in ALLOWED_SERVICE_TYPES:
+            allowed = ", ".join(sorted(ALLOWED_SERVICE_TYPES))
+            raise ValueError(f"Service type {v!r} is not supported. Allowed: {allowed}")
+        return val
+
+
+class ServiceInfoResponse(BaseModel):
+    """Public representation of a database service."""
+
+    service_id: str
+    env_id: str
+    service_type: str
+    status: str
+    host_url: str
+    container_url: str
+    host_port: int
+    container_name: str
+    container_id: str
+    database: str
+    username: str
+    created_at: float
+    error_message: str
+
+
+class ServiceListResponse(BaseModel):
+    """Wrapper for a list of services."""
+
+    services: list[ServiceInfoResponse]
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.post(
+    "/environments/{env_id}/services",
+    response_model=ServiceInfoResponse,
+    status_code=status.HTTP_201_CREATED,
+)
+async def create_service(
+    env_id: str,
+    body: CreateServiceRequest,
+    user: dict[str, Any] = Depends(get_current_user),
+) -> dict[str, Any]:
+    """Create a database service container for an environment."""
+    owner = _owner_from_user(user)
+
+    config = ServiceConfig(
+        service_type=ServiceType(body.service_type),
+        database=body.database,
+        username=body.username,
+        password=body.password,
+    )
+
+    try:
+        svc = await workbench_manager.create_service(env_id, config, owner=owner)
+    except ValueError as exc:
+        raise HTTPException(status_code=404, detail=str(exc)) from exc
+    except RuntimeError as exc:
+        raise HTTPException(status_code=400, detail=str(exc)) from exc
+    except ServiceError as exc:
+        raise HTTPException(status_code=400, detail=exc.message) from exc
+
+    return svc.to_dict()
+
+
+@router.get(
+    "/environments/{env_id}/services",
+    response_model=ServiceListResponse,
+)
+async def list_services(
+    env_id: str,
+    user: dict[str, Any] = Depends(get_current_user),
+) -> dict[str, Any]:
+    """List database services for an environment."""
+    owner = _owner_from_user(user)
+
+    env_info = workbench_manager.get_environment(env_id, owner=owner)
+    if env_info is None:
+        raise HTTPException(status_code=404, detail=f"Unknown environment: {env_id}")
+
+    services = workbench_manager.list_services(env_id, owner=owner)
+    return {"services": services}
+
+
+@router.get(
+    "/services/{service_id}",
+    response_model=ServiceInfoResponse,
+)
+async def get_service(
+    service_id: str,
+    user: dict[str, Any] = Depends(get_current_user),
+) -> dict[str, Any]:
+    """Get status of a database service."""
+    owner = _owner_from_user(user)
+
+    result = workbench_manager.get_service_status(service_id, owner=owner)
+    if result is None:
+        raise HTTPException(status_code=404, detail=f"Unknown service: {service_id}")
+    return result
+
+
+@router.delete(
+    "/services/{service_id}",
+    status_code=status.HTTP_204_NO_CONTENT,
+)
+async def destroy_service(
+    service_id: str,
+    user: dict[str, Any] = Depends(get_current_user),
+) -> None:
+    """Destroy a database service container."""
+    owner = _owner_from_user(user)
+
+    try:
+        await workbench_manager.destroy_service(service_id, owner=owner)
+    except ValueError as exc:
+        raise HTTPException(status_code=404, detail=str(exc)) from exc
+    except RuntimeError as exc:
+        raise HTTPException(status_code=400, detail=str(exc)) from exc