openretailscience 0.48.0__tar.gz → 0.48.1__tar.gz

{openretailscience-0.48.0 → openretailscience-0.48.1}/CONTRIBUTING.md

@@ -45,9 +45,10 @@ describe the public API — analyses, plots, options, and how to connect to data
 
 If you add, rename, or remove a public class, function, plot, or option, update
 the skill in the same PR. `tests/test_skills.py` guards this: it re-executes
-every import example in the skill and fails if one no longer resolves, and it
-checks that every referenced file exists. Run `uv run pytest tests/test_skills.py`
-before pushing.
+every import example and every bundled example script, and it requires the
+reference files under `references/` and the links pointing at them to match
+exactly, so a new reference file must also be linked from the skill. Run
+`uv run pytest tests/test_skills.py` before pushing.
 
 ### Getting Help
 

{openretailscience-0.48.0 → openretailscience-0.48.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: openretailscience
-Version: 0.48.0
+Version: 0.48.1
 Summary: Retail Data Science Tools
 Author-email: Murray Vanwyk <2493311+mvanwyk@users.noreply.github.com>
 License-Expression: Elastic-2.0

{openretailscience-0.48.0 → openretailscience-0.48.1}/docs/getting_started/agent_skills.md

@@ -25,17 +25,18 @@ the installed package never drift apart:
   `.claude/skills/using-openretailscience/` when Claude Code is detected (a
   `~/.claude` directory exists).
 - **Global mode** links into the equivalent directories under your home folder.
+  Not available on Databricks (see below).
 
 Each skill installs as a subfolder named after itself, so the bundled skill
 lands at `using-openretailscience/` inside whichever skills directory applies.
 
-Because the entries are symlinks back into the installed package, a
-`pip install -U openretailscience` updates the skill in place, so there is nothing
+Outside Databricks the entries are symlinks back into the installed package, so a
+`pip install -U openretailscience` updates the skill in place and there is nothing
 to reinstall.
 
 ```python
 install_skills()                       # project install
-install_skills(global_mode=True)       # install for all your projects
+install_skills(global_mode=True)       # install for all your projects (not on Databricks)
 ```
 
 The operation is **idempotent**: re-running it re-uses existing links and never
@@ -52,19 +53,22 @@ one left by an earlier install.
 
 ## Databricks
 
-Databricks works differently, and `install_skills()` adapts
+Databricks works differently, and the default `install_skills()` adapts
 automatically. A pip-installed package lives on the cluster's ephemeral storage,
 which is wiped on restart, and Databricks Genie reads skills from a persistent
 workspace location that cannot symlink back into that storage. So on Databricks
 the installer **copies** the skill into the Genie skills directory instead of
-linking it:
+linking it.
 
-- Project mode → `/Workspace/.assistant/skills/`
-- Global mode → `/Workspace/Users/<you>/.assistant/skills/` (falling back to the
-  shared workspace directory when the user cannot be determined)
+Skills go to `/Workspace/Users/<you>/.assistant/skills/`, your own workspace home.
+`global_mode=True` raises `NotImplementedError` there: the workspace-wide
+`/Workspace/.assistant/skills/` needs admin rights, and a single shared copy
+cannot match the package version each user has installed.
 
-Run it from a notebook, or from a cluster init script so it re-applies on every
-start:
+Run it from a notebook, as yourself. Your workspace home comes from the Spark
+session's `current_user()`, so cases like these raise rather than install: a
+context with no session (a cluster init script, a bare Python job), and an identity
+with no workspace home of its own, such as a job running as a service principal.
 
 ```python
 from openretailscience.skills import install_skills
@@ -73,11 +77,10 @@ install_skills()
 ```
 
 Because this is a copy rather than a link, it does **not** update itself when you
-upgrade the package. Re-run `install_skills()` after upgrading OpenRetailScience
-(a cluster init script is the easiest way to keep it current).
+upgrade the package: re-run `install_skills()` after upgrading OpenRetailScience.
 
 !!! warning "Managed directory"
-    The workspace `.assistant/skills/` directory is treated as installer-managed:
+    Your workspace `.assistant/skills/` directory is treated as installer-managed:
     a folder there whose name matches a bundled skill is refreshed on every run.
     If you hand-author your own skill under that directory, give it a different
     name so `install_skills()` does not overwrite it.
@@ -88,7 +91,7 @@ upgrade the package. Re-run `install_skills()` after upgrading OpenRetailScience
 | --- | --- | --- |
 | Local / project | symlink | Yes, automatically |
 | Global (home) | symlink | Yes, automatically |
-| Databricks | copy | No, re-run `install_skills()` |
+| Databricks (no global mode) | copy | No, re-run `install_skills()` |
 
 The skill's content is maintained alongside the codebase and validated in CI, so
 each release ships guidance that matches that version's public API.

{openretailscience-0.48.0 → openretailscience-0.48.1}/openretailscience/skills.py

@@ -20,8 +20,8 @@ Linking behaviour:
 * On Databricks the package lives on ephemeral compute that is wiped on cluster
   restart, and Genie reads skills from a persistent ``/Workspace`` location that
   cannot symlink into site-packages. There we always **copy** skills into the
-  Genie skills directory; re-run :func:`install_skills` after upgrading the
-  package to refresh the copy.
+  user's own ``/Workspace/Users/<user>/.assistant/skills`` directory; re-run
+  :func:`install_skills` after upgrading the package to refresh the copy.
 """
 
 from __future__ import annotations
@@ -39,14 +39,13 @@ SKILL_MARKER_FILENAME = "SKILL.md"
 AGENTS_DIR_NAME = ".agents"
 CLAUDE_DIR_NAME = ".claude"
 SKILLS_SUBDIR = "skills"
+# pip byte-compiles the package it installs, leaving these inside the bundled skill.
+# The Databricks workspace mount rejects them with OSError 95, part-way through a copy.
+BYTECODE_CACHE_DIR = "__pycache__"
 
 # Databricks Genie / Assistant reads skills from ``<root>/.assistant/skills``.
 DATABRICKS_ASSISTANT_DIR = ".assistant"
-DATABRICKS_USERS_DIR = "Users"
 _DATABRICKS_RUNTIME_ENV = "DATABRICKS_RUNTIME_VERSION"
-_DATABRICKS_USER_ENV = "DATABRICKS_USER"
-# Persistent workspace root on Databricks. Kept as a module constant so tests
-# can redirect it away from the real ``/Workspace`` mount.
 _DATABRICKS_WORKSPACE_ROOT = Path("/Workspace")
 
 
@@ -109,7 +108,7 @@ def _find_project_root(start: Path | None = None) -> Path:
     Returns:
         Path: The resolved project root.
     """
-    start_dir = (start or Path.cwd()).resolve()
+    start_dir = (start if start is not None else Path.cwd()).resolve()
     home = Path.home().resolve()
 
     git_root: Path | None = None
@@ -137,47 +136,37 @@ def _skills_dir(base: Path, harness_dir: str) -> Path:
     return base / harness_dir / SKILLS_SUBDIR
 
 
-def _get_target_dirs(base: Path) -> list[Path]:
-    """Return the ``.agents/skills`` (and ``.claude/skills``) targets under ``base``.
-
-    Always targets ``<base>/.agents/skills``; adds ``<base>/.claude/skills`` when
-    Claude Code is detected (a ``~/.claude`` directory exists).
-
-    Args:
-        base (Path): The resolved project root (project mode) or home directory
-            (global mode).
+def _get_databricks_target_dir() -> Path:
+    """Return the Genie skills directory in the caller's Databricks workspace home.
 
     Returns:
-        list[Path]: Target skill directories.
-    """
-    targets = [_skills_dir(base, AGENTS_DIR_NAME)]
-    if (Path.home() / CLAUDE_DIR_NAME).is_dir():
-        targets.append(_skills_dir(base, CLAUDE_DIR_NAME))
-    return targets
-
+        Path: The Genie skills directory to copy into.
 
-def _get_databricks_target_dirs(*, global_mode: bool) -> list[Path]:
-    """Return the persistent Databricks Genie skills directory.
+    Raises:
+        ImportError: When pyspark is unavailable, so the caller cannot be named.
+        RuntimeError: When no Spark session is running to name the caller, when
+            ``current_user()`` does not name a workspace user, or when that user's
+            workspace home does not exist.
+    """
+    from pyspark.sql import SparkSession  # noqa: PLC0415 - pyspark is not a dependency; Databricks provides it
 
-    Project mode installs to the shared workspace skills directory; global mode
-    installs to the per-user directory when the user is known, otherwise falls
-    back to the shared directory with a note.
+    spark = SparkSession.getActiveSession()
+    if spark is None:
+        msg = "Installing to your Databricks workspace home needs an active Spark session; run this from a notebook."
+        raise RuntimeError(msg)
 
-    Args:
-        global_mode (bool): Whether to install per-user rather than shared.
+    # A SELECT of a scalar function always yields exactly one row.
+    user = str(spark.sql("SELECT current_user()").collect()[0][0])
+    # An empty, "..", or otherwise path-like name would join back out to the admin-only workspace root.
+    if user in {"", ".."} or user != Path(user).name:
+        msg = f"Databricks current_user() did not name a workspace user: {user!r}"
+        raise RuntimeError(msg)
 
-    Returns:
-        list[Path]: The single Genie skills directory to copy into.
-    """
-    if global_mode:
-        user = os.environ.get(_DATABRICKS_USER_ENV) or None  # blank env var counts as unknown
-        if user is not None:
-            base = _DATABRICKS_WORKSPACE_ROOT / DATABRICKS_USERS_DIR / user
-            return [_skills_dir(base, DATABRICKS_ASSISTANT_DIR)]
-        print(  # noqa: T201 - user-facing installer output
-            "Could not determine the Databricks user; installing to the shared workspace skills directory instead."
-        )
-    return [_skills_dir(_DATABRICKS_WORKSPACE_ROOT, DATABRICKS_ASSISTANT_DIR)]
+    home = _DATABRICKS_WORKSPACE_ROOT / "Users" / user
+    if not home.is_dir():
+        msg = f"Databricks workspace home not found: {home}. current_user() returned {user!r}."
+        raise RuntimeError(msg)
+    return _skills_dir(home, DATABRICKS_ASSISTANT_DIR)
 
 
 def _relative_symlink_target(source_path: Path, target_path: Path) -> str:
@@ -219,6 +208,18 @@ def _try_symlink(source_path: Path, target_path: Path) -> bool:
     return True
 
 
+def _copied_paths(root: Path) -> list[Path]:
+    """List the paths under ``root`` that an install copies.
+
+    Args:
+        root (Path): A skill directory, bundled or installed.
+
+    Returns:
+        list[Path]: Sorted paths relative to ``root``, excluding bytecode caches.
+    """
+    return sorted(p.relative_to(root) for p in root.rglob("*") if BYTECODE_CACHE_DIR not in p.parts)
+
+
 def _skill_copy_matches(source_path: Path, target_path: Path) -> bool:
     """Return whether a copied skill directory matches the source byte-for-byte.
 
@@ -231,8 +232,8 @@ def _skill_copy_matches(source_path: Path, target_path: Path) -> bool:
     """
     if not target_path.is_dir():
         return False
-    source_files = sorted(p.relative_to(source_path) for p in source_path.rglob("*"))
-    if source_files != sorted(p.relative_to(target_path) for p in target_path.rglob("*")):
+    source_files = _copied_paths(source_path)
+    if source_files != _copied_paths(target_path):
         return False
     for rel in source_files:
         source_file = source_path / rel
@@ -249,11 +250,8 @@ def _is_owned_target(target_path: Path, bundled_names: set[str]) -> bool:
 
     A target is owned when it is a symlink whose name matches a bundled skill, or
     a real directory named after a bundled skill that itself contains a
-    ``SKILL.md`` (a prior copy install). Anything else is user content.
-
-    Ownership only permits replacement; whether an owned *real directory* is
-    actually replaced depends on the install method (see :func:`_prepare_target`):
-    symlink mode always leaves real directories in place, copy mode refreshes them.
+    ``SKILL.md`` (a prior copy install). Anything else is user content. Ownership
+    only permits replacement; :func:`_prepare_target` decides whether it happens.
 
     Args:
         target_path (Path): The candidate target.
@@ -287,7 +285,7 @@ def _prepare_target(
         use_copy (bool): Whether the install method is copy (vs. symlink).
 
     Returns:
-        str: One of ``"install"``, ``"up_to_date"``, or ``"skip"``.
+        Literal["install", "up_to_date", "skip"]: The disposition for this target.
     """
     if not target_path.exists() and not target_path.is_symlink():
         return "install"
@@ -303,11 +301,6 @@ def _prepare_target(
         target_path.unlink()  # os.unlink semantics: remove the link, not its target
         return "install"
 
-    # Owned real directory. A byte-identical tree is a current install (a prior
-    # copy-fallback run, or a Databricks copy), so report it up to date. Otherwise
-    # copy mode refreshes it, while symlink mode skips it: a non-matching real
-    # directory may be user-authored content that merely shares a bundled skill's
-    # name, so it must not be deleted.
     if _skill_copy_matches(source_path, target_path):
         return "up_to_date"
     if not use_copy:
@@ -354,37 +347,22 @@ def _install_one(
     # so a skipped skill never leaves an empty directory behind.
     target_dir.mkdir(parents=True, exist_ok=True)
     if use_copy or not _try_symlink(source_path, target_path):
-        shutil.copytree(source_path, target_path)
+        shutil.copytree(source_path, target_path, ignore=shutil.ignore_patterns(BYTECODE_CACHE_DIR))
     result.installed.append(label)
 
 
-def _print_result(result: SkillInstallResult) -> None:
-    """Print a summary of the installation outcome.
-
-    Args:
-        result (SkillInstallResult): The completed install result.
-    """
-    for label, paths in (
-        ("Installed", result.installed),
-        ("Already up to date", result.up_to_date),
-        ("Skipped (conflicting file)", result.skipped),
-    ):
-        for path in paths:
-            print(f"{label}: {path}")  # noqa: T201
-
-
 def install_skills(global_mode: bool = False) -> SkillInstallResult:
     """Install the package's bundled agent skills.
 
     In project mode (default) skills are linked into the current project's
     ``.agents/skills/`` (and ``.claude/skills/`` when Claude Code is detected). In
     global mode they are linked into the equivalent home directories. On
-    Databricks, skills are copied into the persistent Genie workspace skills
-    directory instead of linked. The operation is idempotent.
+    Databricks skills are copied into the caller's persistent workspace Genie
+    skills directory instead. The operation is idempotent.
 
     Args:
-        global_mode (bool): Install to user-level home directories instead of the
-            current project. Defaults to False.
+        global_mode (bool): Install into the user-level home directories rather
+            than the current project. Defaults to False. Unsupported on Databricks.
 
     Returns:
         SkillInstallResult: The skills that were installed, already up to date,
@@ -393,7 +371,9 @@ def install_skills(global_mode: bool = False) -> SkillInstallResult:
     Raises:
         FileNotFoundError: When the bundled skills directory is missing.
         RuntimeError: When the bundled skills directory contains no installable
-            skills.
+            skills, or when the Databricks workspace home cannot be resolved.
+        NotImplementedError: When ``global_mode`` is requested on Databricks.
+        ImportError: When pyspark is unavailable on Databricks.
     """
     source_dir = _get_source_skills_dir()
     if not source_dir.is_dir():
@@ -406,13 +386,19 @@ def install_skills(global_mode: bool = False) -> SkillInstallResult:
         raise RuntimeError(msg)
 
     if _DATABRICKS_RUNTIME_ENV in os.environ:
-        target_dirs = _get_databricks_target_dirs(global_mode=global_mode)
+        if global_mode:
+            msg = (
+                "global_mode is not supported on Databricks: the workspace-wide skills directory needs admin "
+                "rights, and one shared copy cannot match the package version each user has installed."
+            )
+            raise NotImplementedError(msg)
+        target_dirs = [_get_databricks_target_dir()]
         use_copy = True
-    elif global_mode:
-        target_dirs = _get_target_dirs(Path.home())
-        use_copy = False
     else:
-        target_dirs = _get_target_dirs(_find_project_root())
+        base = Path.home() if global_mode else _find_project_root()
+        target_dirs = [_skills_dir(base, AGENTS_DIR_NAME)]
+        if (Path.home() / CLAUDE_DIR_NAME).is_dir():  # Claude Code is installed
+            target_dirs.append(_skills_dir(base, CLAUDE_DIR_NAME))
         use_copy = False
 
     result = SkillInstallResult()
@@ -421,5 +407,11 @@ def install_skills(global_mode: bool = False) -> SkillInstallResult:
         for target_dir in target_dirs:
             _install_one(skill_name, source_dir, target_dir, result, bundled_names, use_copy=use_copy)
 
-    _print_result(result)
+    for label, paths in (
+        ("Installed", result.installed),
+        ("Already up to date", result.up_to_date),
+        ("Skipped (conflicting file)", result.skipped),
+    ):
+        for path in paths:
+            print(f"{label}: {path}")  # noqa: T201
     return result

{openretailscience-0.48.0 → openretailscience-0.48.1}/pyproject.toml

@@ -27,7 +27,7 @@ license = "Elastic-2.0"
 name = "openretailscience"
 readme = "README.md"
 requires-python = ">=3.10,<3.14"
-version = "0.48.0"
+version = "0.48.1"
 [[project.authors]]
 email = "2493311+mvanwyk@users.noreply.github.com"
 name  = "Murray Vanwyk"