atlas-ftag-tools 0.0.7__tar.gz → 0.0.9__tar.gz

atlas-ftag-tools-0.0.9/PKG-INFO

@@ -0,0 +1,76 @@
+Metadata-Version: 2.1
+Name: atlas-ftag-tools
+Version: 0.0.9
+Summary: ATLAS Flavour Tagging Tools
+Author: Sam Van Stroud, Philipp Gadow
+License: MIT
+Project-URL: Homepage, https://github.com/umami-hep/atlas-ftag-tools/
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Provides-Extra: dev
+
+# ATLAS FTAG Python Tools
+
+This is a collection of Python tools for working with files produced with the FTAG [ntuple dumper](https://gitlab.cern.ch/atlas-flavor-tagging-tools/training-dataset-dumper/).
+The code is intended to be used a [library](https://iscinumpy.dev/post/app-vs-library/) for other projects.
+Please see the [example notebook](ftag/example.ipynb) for usage.
+
+## Installation
+
+To install the package you can install from pip using the [release on pypi](https://pypi.org/project/atlas-ftag-tools/) via
+
+```bash
+pip install atlas-ftag-tools
+```
+
+or you can clone the repository and install in editable mode with
+```bash
+python -m pip install -e .
+```
+
+To install optional development dependencies (for formatting and linting) you can instead install with either from pip
+```bash
+pip install atlas-ftag-tools[dev]
+```
+
+or from source
+```bash
+python -m pip install -e ".[dev]"
+```
+
+
+## Create virtual file
+
+This package contains a script to easily merge a set of H5 files.
+A virtual file is a fast and lightweight way to wrap a set of files.
+See the [h5py documentation](https://docs.h5py.org/en/stable/vds.html) for more information on virtual datasets.
+
+The script is `vds.py` and can be run after installing this package with
+
+```
+vds <pattern> <output path>
+```
+
+The `<pattern>` argument should be a quotes enclosed [glob pattern](https://en.wikipedia.org/wiki/Glob_(programming)), for example `"dsid/path/*.h5"`
+
+
+## Calculate WPs
+
+This package contains a script to calculate tagger working points (WPs).
+The script is `working_points.py` and can be run after installing this package with
+
+```
+wps \
+    --ttbar "path/to/ttbar/*.h5" \
+    --tagger GN120220509 \
+    --fx 0.1
+```
+
+Both the `--tagger` and `--fx` options accept a list if you want to get the WPs for multiple taggers.
+
+If you want to use the `ttbar` WPs get the efficiencies and rejections for the `zprime` sample, you can add `--zprime "path/to/zprime/*.h5"` to the command.
+Note that a default selection of $p_T > 250 ~GeV$ to jets in the `zprime` sample.
+
+By default the working points are printed to the terminal, but you can save the results to a YAML file with the `--outfile` option.
+
+Use `--help` for more options and information.

atlas-ftag-tools-0.0.9/README.md

@@ -0,0 +1,65 @@
+# ATLAS FTAG Python Tools
+
+This is a collection of Python tools for working with files produced with the FTAG [ntuple dumper](https://gitlab.cern.ch/atlas-flavor-tagging-tools/training-dataset-dumper/).
+The code is intended to be used a [library](https://iscinumpy.dev/post/app-vs-library/) for other projects.
+Please see the [example notebook](ftag/example.ipynb) for usage.
+
+## Installation
+
+To install the package you can install from pip using the [release on pypi](https://pypi.org/project/atlas-ftag-tools/) via
+
+```bash
+pip install atlas-ftag-tools
+```
+
+or you can clone the repository and install in editable mode with
+```bash
+python -m pip install -e .
+```
+
+To install optional development dependencies (for formatting and linting) you can instead install with either from pip
+```bash
+pip install atlas-ftag-tools[dev]
+```
+
+or from source
+```bash
+python -m pip install -e ".[dev]"
+```
+
+
+## Create virtual file
+
+This package contains a script to easily merge a set of H5 files.
+A virtual file is a fast and lightweight way to wrap a set of files.
+See the [h5py documentation](https://docs.h5py.org/en/stable/vds.html) for more information on virtual datasets.
+
+The script is `vds.py` and can be run after installing this package with
+
+```
+vds <pattern> <output path>
+```
+
+The `<pattern>` argument should be a quotes enclosed [glob pattern](https://en.wikipedia.org/wiki/Glob_(programming)), for example `"dsid/path/*.h5"`
+
+
+## Calculate WPs
+
+This package contains a script to calculate tagger working points (WPs).
+The script is `working_points.py` and can be run after installing this package with
+
+```
+wps \
+    --ttbar "path/to/ttbar/*.h5" \
+    --tagger GN120220509 \
+    --fx 0.1
+```
+
+Both the `--tagger` and `--fx` options accept a list if you want to get the WPs for multiple taggers.
+
+If you want to use the `ttbar` WPs get the efficiencies and rejections for the `zprime` sample, you can add `--zprime "path/to/zprime/*.h5"` to the command.
+Note that a default selection of $p_T > 250 ~GeV$ to jets in the `zprime` sample.
+
+By default the working points are printed to the terminal, but you can save the results to a YAML file with the `--outfile` option.
+
+Use `--help` for more options and information.

atlas-ftag-tools-0.0.9/atlas_ftag_tools.egg-info/PKG-INFO

@@ -0,0 +1,76 @@
+Metadata-Version: 2.1
+Name: atlas-ftag-tools
+Version: 0.0.9
+Summary: ATLAS Flavour Tagging Tools
+Author: Sam Van Stroud, Philipp Gadow
+License: MIT
+Project-URL: Homepage, https://github.com/umami-hep/atlas-ftag-tools/
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Provides-Extra: dev
+
+# ATLAS FTAG Python Tools
+
+This is a collection of Python tools for working with files produced with the FTAG [ntuple dumper](https://gitlab.cern.ch/atlas-flavor-tagging-tools/training-dataset-dumper/).
+The code is intended to be used a [library](https://iscinumpy.dev/post/app-vs-library/) for other projects.
+Please see the [example notebook](ftag/example.ipynb) for usage.
+
+## Installation
+
+To install the package you can install from pip using the [release on pypi](https://pypi.org/project/atlas-ftag-tools/) via
+
+```bash
+pip install atlas-ftag-tools
+```
+
+or you can clone the repository and install in editable mode with
+```bash
+python -m pip install -e .
+```
+
+To install optional development dependencies (for formatting and linting) you can instead install with either from pip
+```bash
+pip install atlas-ftag-tools[dev]
+```
+
+or from source
+```bash
+python -m pip install -e ".[dev]"
+```
+
+
+## Create virtual file
+
+This package contains a script to easily merge a set of H5 files.
+A virtual file is a fast and lightweight way to wrap a set of files.
+See the [h5py documentation](https://docs.h5py.org/en/stable/vds.html) for more information on virtual datasets.
+
+The script is `vds.py` and can be run after installing this package with
+
+```
+vds <pattern> <output path>
+```
+
+The `<pattern>` argument should be a quotes enclosed [glob pattern](https://en.wikipedia.org/wiki/Glob_(programming)), for example `"dsid/path/*.h5"`
+
+
+## Calculate WPs
+
+This package contains a script to calculate tagger working points (WPs).
+The script is `working_points.py` and can be run after installing this package with
+
+```
+wps \
+    --ttbar "path/to/ttbar/*.h5" \
+    --tagger GN120220509 \
+    --fx 0.1
+```
+
+Both the `--tagger` and `--fx` options accept a list if you want to get the WPs for multiple taggers.
+
+If you want to use the `ttbar` WPs get the efficiencies and rejections for the `zprime` sample, you can add `--zprime "path/to/zprime/*.h5"` to the command.
+Note that a default selection of $p_T > 250 ~GeV$ to jets in the `zprime` sample.
+
+By default the working points are printed to the terminal, but you can save the results to a YAML file with the `--outfile` option.
+
+Use `--help` for more options and information.

{atlas-ftag-tools-0.0.7 → atlas-ftag-tools-0.0.9}/atlas_ftag_tools.egg-info/SOURCES.txt

@@ -4,6 +4,7 @@ pyproject.toml
 atlas_ftag_tools.egg-info/PKG-INFO
 atlas_ftag_tools.egg-info/SOURCES.txt
 atlas_ftag_tools.egg-info/dependency_links.txt
+atlas_ftag_tools.egg-info/entry_points.txt
 atlas_ftag_tools.egg-info/requires.txt
 atlas_ftag_tools.egg-info/top_level.txt
 ftag/__init__.py

atlas-ftag-tools-0.0.9/atlas_ftag_tools.egg-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+vds = ftag.vds:main
+wps = ftag.wps.working_points:main

atlas-ftag-tools-0.0.9/ftag/__init__.py

@@ -0,0 +1,25 @@
+"""atlas-ftag-tools - Common tools for ATLAS flavour tagging software."""
+
+
+__version__ = "v0.0.9"
+
+
+import ftag.hdf5 as hdf5
+from ftag.cuts import Cuts
+from ftag.flavour import Flavour, Flavours
+from ftag.mock import get_mock_file
+from ftag.sample import Sample
+from ftag.wps.discriminant import get_discriminant
+from ftag.wps.working_points import get_working_points
+
+__all__ = [
+    "Cuts",
+    "Flavour",
+    "Flavours",
+    "Sample",
+    "hdf5",
+    "get_mock_file",
+    "get_discriminant",
+    "get_working_points",
+    "__version__",
+]

{atlas-ftag-tools-0.0.7 → atlas-ftag-tools-0.0.9}/ftag/flavour.py

@@ -2,6 +2,9 @@ from __future__ import annotations
 
 from collections.abc import Generator
 from dataclasses import dataclass
+from pathlib import Path
+
+import yaml
 
 from ftag.cuts import Cuts
 
@@ -74,3 +77,10 @@ class FlavourContainer:
             if flavour.cuts == cuts:
                 return flavour
         raise KeyError(f"Flavour with {cuts} not found")
+
+
+with open(Path(__file__).parent / "flavours.yaml") as f:
+    flavours_yaml = yaml.safe_load(f)
+flavours_dict = {f["name"]: Flavour(cuts=Cuts.from_list(f.pop("cuts")), **f) for f in flavours_yaml}
+assert len(flavours_dict) == len(flavours_yaml), "Duplicate flavour names detected"
+Flavours = FlavourContainer(flavours_dict)

{atlas-ftag-tools-0.0.7 → atlas-ftag-tools-0.0.9}/ftag/hdf5/h5reader.py

@@ -111,12 +111,33 @@ class H5SingleReader:
 
 @dataclass
 class H5Reader:
+    """Reads data from multiple HDF5 files.
+
+    Parameters
+    ----------
+    fname : Path | str | list[Path | str]
+        Path to the HDF5 file or list of paths
+    batch_size : int, optional
+        Number of jets to read at a time, by default 100_000
+    jets_name : str, optional
+        Name of the jets dataset, by default "jets"
+    precision : str | None, optional
+        Cast floats to given precision, by default None
+    shuffle : bool, optional
+        Read batches in a shuffled order, by default True
+    weights : list[float] | None, optional
+        Weights for different input datasets, by default None
+    do_remove_inf : bool, optional
+        Remove jets with inf values, by default False
+    """
+
     fname: Path | str | list[Path | str]
     batch_size: int = 100_000
     jets_name: str = "jets"
     precision: str | None = None
     shuffle: bool = True
     weights: list[float] | None = None
+    do_remove_inf: bool = False
 
     def __post_init__(self) -> None:
         if isinstance(self.fname, (str, Path)):
@@ -129,8 +150,8 @@ class H5Reader:
 
         # create readers
         self.readers = [
-            H5SingleReader(fname, batch_size, self.jets_name, self.precision, self.shuffle)
-            for fname, batch_size in zip(self.fname, self.batch_sizes)
+            H5SingleReader(f, b, self.jets_name, self.precision, self.shuffle, self.do_remove_inf)
+            for f, b in zip(self.fname, self.batch_sizes)
         ]
 
     @property

{atlas-ftag-tools-0.0.7 → atlas-ftag-tools-0.0.9}/ftag/vds.py

@@ -60,3 +60,26 @@ def create_virtual_file(
             f.create_virtual_dataset(group, layout)
 
     return out_fname
+
+
+def main():
+    import argparse
+
+    parser = argparse.ArgumentParser(
+        description="Create a lightweight wrapper around a set of h5 files"
+    )
+    parser.add_argument("pattern", type=Path, help="quotes-enclosed glob pattern of files to merge")
+    parser.add_argument("output", type=Path, help="path to output virtual file")
+    args = parser.parse_args()
+
+    print(f"Globbing {args.pattern}...")
+    create_virtual_file(args.pattern, args.output, overwrite=True)
+    with h5py.File(args.output) as f:
+        key = list(f.keys())[0]
+        num = len(f[key])
+    print(f"Virtual dataset '{key}' has {num:,} entries")
+    print(f"Saved virtual file to {args.output.resolve()}")
+
+
+if __name__ == "__main__":
+    main()

{atlas-ftag-tools-0.0.7 → atlas-ftag-tools-0.0.9}/pyproject.toml

@@ -26,6 +26,10 @@ dev = [
   "ipykernel==6.21.3",
 ]
 
+[project.scripts]
+vds = "ftag.vds:main"
+wps = "ftag.wps.working_points:main"
+
 [tool.setuptools]
 packages = ["ftag", "ftag.hdf5"]
 include-package-data = true

atlas-ftag-tools-0.0.7/PKG-INFO

@@ -1,39 +0,0 @@
-Metadata-Version: 2.1
-Name: atlas-ftag-tools
-Version: 0.0.7
-Summary: ATLAS Flavour Tagging Tools
-Author: Sam Van Stroud, Philipp Gadow
-License: MIT
-Project-URL: Homepage, https://github.com/umami-hep/atlas-ftag-tools/
-Requires-Python: >=3.8
-Description-Content-Type: text/markdown
-Provides-Extra: dev
-
-# ATLAS FTAG Python Tools
-
-This is a collection of Python tools for working with files produced with the FTAG [ntuple dumper](https://gitlab.cern.ch/atlas-flavor-tagging-tools/training-dataset-dumper/).
-The code is intended to be used a [library](https://iscinumpy.dev/post/app-vs-library/) for other projects.
-Please see the [example notebook](ftag/example.ipynb) for usage.
-
-## Installation
-
-To install the package you can install from pip using the [release on pypi](https://pypi.org/project/atlas-ftag-tools/) via
-
-```bash
-pip install atlas-ftag-tools
-```
-
-or you can clone the repository and install in editable mode with
-```bash
-python -m pip install -e .
-```
-
-To install optional development dependencies (for formatting and linting) you can instead install with either from pip
-```bash
-pip install atlas-ftag-tools[dev]
-```
-
-or from source
-```bash
-python -m pip install -e ".[dev]"
-```

atlas-ftag-tools-0.0.7/README.md

@@ -1,28 +0,0 @@
-# ATLAS FTAG Python Tools
-
-This is a collection of Python tools for working with files produced with the FTAG [ntuple dumper](https://gitlab.cern.ch/atlas-flavor-tagging-tools/training-dataset-dumper/).
-The code is intended to be used a [library](https://iscinumpy.dev/post/app-vs-library/) for other projects.
-Please see the [example notebook](ftag/example.ipynb) for usage.
-
-## Installation
-
-To install the package you can install from pip using the [release on pypi](https://pypi.org/project/atlas-ftag-tools/) via
-
-```bash
-pip install atlas-ftag-tools
-```
-
-or you can clone the repository and install in editable mode with
-```bash
-python -m pip install -e .
-```
-
-To install optional development dependencies (for formatting and linting) you can instead install with either from pip
-```bash
-pip install atlas-ftag-tools[dev]
-```
-
-or from source
-```bash
-python -m pip install -e ".[dev]"
-```

atlas-ftag-tools-0.0.7/atlas_ftag_tools.egg-info/PKG-INFO

@@ -1,39 +0,0 @@
-Metadata-Version: 2.1
-Name: atlas-ftag-tools
-Version: 0.0.7
-Summary: ATLAS Flavour Tagging Tools
-Author: Sam Van Stroud, Philipp Gadow
-License: MIT
-Project-URL: Homepage, https://github.com/umami-hep/atlas-ftag-tools/
-Requires-Python: >=3.8
-Description-Content-Type: text/markdown
-Provides-Extra: dev
-
-# ATLAS FTAG Python Tools
-
-This is a collection of Python tools for working with files produced with the FTAG [ntuple dumper](https://gitlab.cern.ch/atlas-flavor-tagging-tools/training-dataset-dumper/).
-The code is intended to be used a [library](https://iscinumpy.dev/post/app-vs-library/) for other projects.
-Please see the [example notebook](ftag/example.ipynb) for usage.
-
-## Installation
-
-To install the package you can install from pip using the [release on pypi](https://pypi.org/project/atlas-ftag-tools/) via
-
-```bash
-pip install atlas-ftag-tools
-```
-
-or you can clone the repository and install in editable mode with
-```bash
-python -m pip install -e .
-```
-
-To install optional development dependencies (for formatting and linting) you can instead install with either from pip
-```bash
-pip install atlas-ftag-tools[dev]
-```
-
-or from source
-```bash
-python -m pip install -e ".[dev]"
-```

atlas-ftag-tools-0.0.7/ftag/__init__.py

@@ -1,23 +0,0 @@
-"""atlas-ftag-tools - Common tools for ATLAS flavour tagging software."""
-
-
-__version__ = "v0.0.7"
-
-from pathlib import Path
-
-import yaml
-
-import ftag.hdf5 as hdf5
-from ftag.cuts import Cuts
-from ftag.flavour import Flavour, FlavourContainer
-from ftag.mock import get_mock_file
-from ftag.sample import Sample
-
-# load flavours
-with open(Path(__file__).parent / "flavours.yaml") as f:
-    flavours_yaml = yaml.safe_load(f)
-flavours_dict = {f["name"]: Flavour(cuts=Cuts.from_list(f.pop("cuts")), **f) for f in flavours_yaml}
-assert len(flavours_dict) == len(flavours_yaml), "Duplicate flavour names detected"
-Flavours = FlavourContainer(flavours_dict)
-
-__all__ = ["Cuts", "Flavours", "Sample", "hdf5", "get_mock_file", "__version__"]