func-adl-local 0.1.0__tar.gz

func_adl_local-0.1.0/.gitignore

@@ -0,0 +1 @@
+.venv

func_adl_local-0.1.0/PKG-INFO

@@ -0,0 +1,149 @@
+Metadata-Version: 2.4
+Name: func_adl_local
+Version: 0.1.0
+Summary: Run FuncADL queries on local files without ServiceX
+Project-URL: Homepage, https://github.com/RogerJanusiak/func_adl_local
+Author-email: Roger Janusiak <rogerjanusiak@gmail.com>
+License: MIT
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Physics
+Requires-Python: >=3.9
+Requires-Dist: func-adl-servicex-xaodr21
+Requires-Dist: func-adl-servicex-xaodr22
+Requires-Dist: func-adl-servicex-xaodr25
+Requires-Dist: func-adl-uproot
+Requires-Dist: func-adl>=3.2
+Requires-Dist: jinja2
+Requires-Dist: servicex-analysis-utils
+Requires-Dist: servicex-local
+Provides-Extra: test
+Requires-Dist: pytest; extra == 'test'
+Description-Content-Type: text/markdown
+
+# func_adl_local
+
+Run [FuncADL](https://github.com/iris-hep/func_adl) queries on local files without ServiceX.
+
+[FuncADL](https://github.com/iris-hep/func_adl) (Functional Analysis Description Language) is a declarative query language for HEP data analysis. Normally it is used with [ServiceX](https://github.com/ssl-hep/ServiceX) to query remote datasets. `func_adl_local` brings that same interface to files on your local machine — no ServiceX deployment required.
+
+## Installation
+
+```bash
+pip install func_adl_local
+```
+
+## Usage
+
+### xAOD files
+
+Use `xAODConfig` to configure the ATLAS release and runtime platform, then call `get_data` with your dataset path and FuncADL query:
+
+```python
+from func_adl_local import xAODConfig, get_data
+
+config = xAODConfig(
+    release=22,        # ATLAS release: 21, 22, or 25
+    platform="docker", # "docker", "singularity", or "wsl2"
+)
+
+# Build a query using the release-appropriate FuncADL dataset type
+query = config.FuncADLQueryPHYS().SelectMany(lambda e: e.Jets("AntiKt4EMTopoJets")).Select(
+    lambda j: {"pt": j.pt(), "eta": j.eta()}
+)
+
+result = get_data("path/to/file.root", query, config)
+```
+
+To get the result as an [awkward-array](https://awkward-array.org/), set `awk=True`:
+
+```python
+config = xAODConfig(release=22, awk=True)
+result = get_data("path/to/file.root", query, config)
+# result is now an awkward array
+```
+
+**Available query types:**
+
+| Method | Releases |
+|---|---|
+| `config.FuncADLQueryPHYS()` | 21, 22, 25 |
+| `config.FuncADLQueryPHYSLITE()` | 22, 25 |
+
+**Platforms:**
+
+| Value | Description |
+|---|---|
+| `"docker"` | Run the transformer in Docker (default) |
+| `"singularity"` | Run in Singularity/Apptainer |
+| `"wsl2"` | Run in WSL2 |
+
+**Docker image versions:**
+
+`xAODConfig` automatically selects the latest image for the chosen release. You can inspect available versions or pin to a specific one:
+
+```python
+config = xAODConfig(release=22)
+print(config.available_versions)   # all available tags
+print(config.latest_r22_version)   # e.g. "22.2.110"
+
+config_pinned = xAODConfig(release=22, version="22.2.107")
+```
+
+### Uproot / columnar files
+
+For ROOT files and other columnar formats, use the re-exported `UprootDataset` from [func_adl_uproot](https://github.com/iris-hep/func_adl_uproot):
+
+```python
+from func_adl_local import UprootDataset
+
+ds = UprootDataset("path/to/file.root", "treename")
+result = ds.Select(lambda e: {"pt": e["pt"]}).AsAwkwardArray().value()
+```
+
+## API Reference
+
+### `xAODConfig`
+
+```python
+@dataclass
+class xAODConfig:
+    release: int = 21           # ATLAS release year: 21, 22, or 25
+    version: str = "latest"     # Docker image tag, or "latest" to auto-select
+    platform: str = "docker"    # "docker", "singularity", or "wsl2"
+    ignore_cache: bool = False  # Bypass the local ServiceX cache
+    awk: bool = False           # Return results as awkward arrays
+```
+
+### `get_data(ds_name, query, config)`
+
+Runs a FuncADL query against a local xAOD file.
+
+- `ds_name` — path to the local dataset
+- `query` — a FuncADL `ObjectStream` built from `config.FuncADLQueryPHYS()` or `config.FuncADLQueryPHYSLITE()`
+- `config` — an `xAODConfig` instance
+
+Returns a dict of arrays (or an awkward array if `config.awk=True`).
+
+## Related Projects
+
+- [func_adl](https://github.com/iris-hep/func_adl) — core FuncADL library
+- [func_adl_uproot](https://github.com/iris-hep/func_adl_uproot) — Uproot backend for FuncADL
+- [servicex_local](https://github.com/iris-hep/servicex_local) — local ServiceX runtime
+- [ServiceX](https://github.com/ssl-hep/ServiceX) — remote data delivery service
+- [func_adl_servicex](https://github.com/iris-hep/func_adl_servicex) — FuncADL ServiceX backend
+
+## Contributing
+
+Contributions are welcome! Please open an issue or pull request on [GitHub](https://github.com/RogerJanusiak/func_adl_local).
+
+## License
+
+MIT License

func_adl_local-0.1.0/README.md

@@ -0,0 +1,119 @@
+# func_adl_local
+
+Run [FuncADL](https://github.com/iris-hep/func_adl) queries on local files without ServiceX.
+
+[FuncADL](https://github.com/iris-hep/func_adl) (Functional Analysis Description Language) is a declarative query language for HEP data analysis. Normally it is used with [ServiceX](https://github.com/ssl-hep/ServiceX) to query remote datasets. `func_adl_local` brings that same interface to files on your local machine — no ServiceX deployment required.
+
+## Installation
+
+```bash
+pip install func_adl_local
+```
+
+## Usage
+
+### xAOD files
+
+Use `xAODConfig` to configure the ATLAS release and runtime platform, then call `get_data` with your dataset path and FuncADL query:
+
+```python
+from func_adl_local import xAODConfig, get_data
+
+config = xAODConfig(
+    release=22,        # ATLAS release: 21, 22, or 25
+    platform="docker", # "docker", "singularity", or "wsl2"
+)
+
+# Build a query using the release-appropriate FuncADL dataset type
+query = config.FuncADLQueryPHYS().SelectMany(lambda e: e.Jets("AntiKt4EMTopoJets")).Select(
+    lambda j: {"pt": j.pt(), "eta": j.eta()}
+)
+
+result = get_data("path/to/file.root", query, config)
+```
+
+To get the result as an [awkward-array](https://awkward-array.org/), set `awk=True`:
+
+```python
+config = xAODConfig(release=22, awk=True)
+result = get_data("path/to/file.root", query, config)
+# result is now an awkward array
+```
+
+**Available query types:**
+
+| Method | Releases |
+|---|---|
+| `config.FuncADLQueryPHYS()` | 21, 22, 25 |
+| `config.FuncADLQueryPHYSLITE()` | 22, 25 |
+
+**Platforms:**
+
+| Value | Description |
+|---|---|
+| `"docker"` | Run the transformer in Docker (default) |
+| `"singularity"` | Run in Singularity/Apptainer |
+| `"wsl2"` | Run in WSL2 |
+
+**Docker image versions:**
+
+`xAODConfig` automatically selects the latest image for the chosen release. You can inspect available versions or pin to a specific one:
+
+```python
+config = xAODConfig(release=22)
+print(config.available_versions)   # all available tags
+print(config.latest_r22_version)   # e.g. "22.2.110"
+
+config_pinned = xAODConfig(release=22, version="22.2.107")
+```
+
+### Uproot / columnar files
+
+For ROOT files and other columnar formats, use the re-exported `UprootDataset` from [func_adl_uproot](https://github.com/iris-hep/func_adl_uproot):
+
+```python
+from func_adl_local import UprootDataset
+
+ds = UprootDataset("path/to/file.root", "treename")
+result = ds.Select(lambda e: {"pt": e["pt"]}).AsAwkwardArray().value()
+```
+
+## API Reference
+
+### `xAODConfig`
+
+```python
+@dataclass
+class xAODConfig:
+    release: int = 21           # ATLAS release year: 21, 22, or 25
+    version: str = "latest"     # Docker image tag, or "latest" to auto-select
+    platform: str = "docker"    # "docker", "singularity", or "wsl2"
+    ignore_cache: bool = False  # Bypass the local ServiceX cache
+    awk: bool = False           # Return results as awkward arrays
+```
+
+### `get_data(ds_name, query, config)`
+
+Runs a FuncADL query against a local xAOD file.
+
+- `ds_name` — path to the local dataset
+- `query` — a FuncADL `ObjectStream` built from `config.FuncADLQueryPHYS()` or `config.FuncADLQueryPHYSLITE()`
+- `config` — an `xAODConfig` instance
+
+Returns a dict of arrays (or an awkward array if `config.awk=True`).
+
+## Related Projects
+
+- [func_adl](https://github.com/iris-hep/func_adl) — core FuncADL library
+- [func_adl_uproot](https://github.com/iris-hep/func_adl_uproot) — Uproot backend for FuncADL
+- [servicex_local](https://github.com/iris-hep/servicex_local) — local ServiceX runtime
+- [ServiceX](https://github.com/ssl-hep/ServiceX) — remote data delivery service
+- [func_adl_servicex](https://github.com/iris-hep/func_adl_servicex) — FuncADL ServiceX backend
+
+## Contributing
+
+Contributions are welcome! Please open an issue or pull request on [GitHub](https://github.com/RogerJanusiak/func_adl_local).
+
+## License
+
+MIT License

func_adl_local-0.1.0/func_adl_local/__init__.py

@@ -0,0 +1,8 @@
+# func_adl_local — run FuncADL queries on local files without ServiceX
+
+__version__ = "0.1.0"
+
+# Allow function from UprootTo be used without importing the module
+from func_adl_uproot import UprootDataset
+
+from .functions import xAODConfig, get_data

func_adl_local-0.1.0/func_adl_local/functions.py

@@ -0,0 +1,140 @@
+import importlib
+import json
+import urllib.request
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Union
+
+from func_adl import ObjectStream
+from servicex import deliver
+from servicex.databinder_models import Sample, ServiceXSpec
+from servicex_analysis_utils import to_awk
+
+from enum import Enum
+
+
+class Platform(Enum):
+    """Options for which platform to use for the runtime environment."""
+
+    docker = "docker"
+    singularity = "singularity"
+    wsl2 = "wsl2"
+
+if TYPE_CHECKING:
+    from func_adl_servicex_xaodr21.sx_dataset import FuncADLQueryPHYS as _PHYS21
+    from func_adl_servicex_xaodr22.sx_dataset import FuncADLQueryPHYS as _PHYS22
+    from func_adl_servicex_xaodr22.sx_dataset import FuncADLQueryPHYSLITE as _PHYSLITE22
+    from func_adl_servicex_xaodr25.sx_dataset import FuncADLQueryPHYS as _PHYS25
+    from func_adl_servicex_xaodr25.sx_dataset import FuncADLQueryPHYSLITE as _PHYSLITE25
+
+_DOCKER_IMAGE = "sslhep/servicex_func_adl_xaod_transformer"
+
+
+_VALID_RELEASES = (21, 22, 25)
+
+
+@dataclass
+class xAODConfig:
+    """Configuration for xAOD datasets."""
+
+    release: int = 21
+    version: str = "latest"
+    platform: Union[Platform, str] = "docker"
+    ignore_cache: bool = False
+    awk: bool = False
+
+    def __post_init__(self):
+        if self.release not in _VALID_RELEASES:
+            raise ValueError(f"release must be one of {_VALID_RELEASES}, got {self.release}")
+        if self.version == "latest":
+            self.version = self._latest_for_release(self.release)
+        if isinstance(self.platform, str):
+            self.platform = Platform[self.platform]
+
+    def _latest_for_release(self, release: int) -> str:
+        versions = [t for t in self.available_versions if t.startswith(f"{release}.")]
+        if not versions:
+            raise ValueError(f"No versions found for release {release}")
+        return max(versions, key=lambda t: tuple(int(x) for x in t.split(".")))
+
+    @property
+    def available_versions(self) -> list[str]:
+        """Return available versions (21.x, 22.x, 25.x) of the xAOD transformer Docker image."""
+        tags = []
+        url = f"https://hub.docker.com/v2/repositories/{_DOCKER_IMAGE}/tags?page_size=100"
+        while url:
+            with urllib.request.urlopen(url) as response:
+                data = json.loads(response.read())
+            tags.extend(result["name"] for result in data["results"])
+            url = data.get("next")
+        return [t for t in tags if t.startswith(("21.", "22.", "25."))]
+
+    @property
+    def latest_r21_version(self) -> str:
+        """Return the latest 21.x version of the xAOD transformer Docker image."""
+        return self._latest_for_release(21)
+
+    @property
+    def latest_r22_version(self) -> str:
+        """Return the latest 22.x version of the xAOD transformer Docker image."""
+        return self._latest_for_release(22)
+
+    @property
+    def latest_r25_version(self) -> str:
+        """Return the latest 25.x version of the xAOD transformer Docker image."""
+        return self._latest_for_release(25)
+
+    def _release_module(self):
+        return importlib.import_module(f"func_adl_servicex_xaodr{self.release}")
+
+    def FuncADLQueryPHYS(self) -> "Union[_PHYS21, _PHYS22, _PHYS25]":
+        return self._release_module().FuncADLQueryPHYS()
+
+    def FuncADLQueryPHYSLITE(self) -> "Union[_PHYSLITE22, _PHYSLITE25]":
+        return self._release_module().FuncADLQueryPHYSLITE()
+
+
+def build_sx_spec(query, ds_name: str, config: xAODConfig):
+    """Build a ServiceX spec from the given query and dataset."""
+    from servicex_local.utils import find_dataset, install_sx_local
+    from servicex_local.utils import Platform as _SxPlatform
+
+    dataset, use_local = find_dataset(ds_name, prefer_local=True)
+
+    if not use_local:
+        raise ValueError(f"Unable to run dataset {ds_name} locally.")
+
+    image = f"docker://{_DOCKER_IMAGE}:{config.version}"
+    sx_platform = _SxPlatform(config.platform.value)
+    codegen_name, adaptor = install_sx_local(image, sx_platform)
+
+    spec = ServiceXSpec(
+        Sample=[
+            Sample(
+                Name="MySample",
+                Dataset=dataset,
+                Query=query,
+                Codegen=codegen_name,
+            ),
+        ],
+    )
+
+    return spec, adaptor
+
+
+def get_data(
+    ds_name: str,
+    query: ObjectStream,
+    config: xAODConfig,
+):
+    """Run a query against a dataset, either locally or remotely."""
+    from servicex_local.deliver import deliver as local_deliver
+
+    spec, adaptor = build_sx_spec(query, ds_name, config)
+
+    sx_result = local_deliver(
+        spec, adaptor=adaptor, ignore_local_cache=config.ignore_cache
+    )
+
+    if config.awk:
+        return to_awk(sx_result)["MySample"]
+    return sx_result["MySample"]

func_adl_local-0.1.0/pyproject.toml

@@ -0,0 +1,61 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "func_adl_local"
+description = "Run FuncADL queries on local files without ServiceX"
+readme = "README.md"
+authors = [{ name = "Roger Janusiak", email = "rogerjanusiak@gmail.com" }]
+license = { text = "MIT" }
+requires-python = ">=3.9"
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Physics",
+]
+dependencies = [
+    "func_adl>=3.2",
+    "func_adl_uproot",
+    "servicex_local",
+    "func_adl_servicex_xaodr21",
+    "func_adl_servicex_xaodr22",
+    "func_adl_servicex_xaodr25",
+    "jinja2",
+    "servicex_analysis_utils",
+]
+dynamic = ["version"]
+
+[project.optional-dependencies]
+test = ["pytest"]
+
+[tool.hatch.version]
+path = "func_adl_local/__init__.py"
+
+[tool.hatch.build.targets.sdist]
+include = ["README.md", "/func_adl_local"]
+
+[tool.hatch.build.targets.wheel]
+include = ["README.md", "/func_adl_local"]
+
+[project.urls]
+Homepage = "https://github.com/RogerJanusiak/func_adl_local"
+
+[tool.black]
+line-length = 99
+
+[tool.isort]
+profile = "black"
+
+[tool.flake8]
+max-line-length = 99
+
+[tool.ruff]
+line-length = 99