hebb 0.7__tar.gz

hebb-0.7/LICENSE.md

@@ -0,0 +1,11 @@
+Copyright 2026 Andrea Negri
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

hebb-0.7/PKG-INFO

@@ -0,0 +1,79 @@
+Metadata-Version: 2.4
+Name: hebb
+Version: 0.7
+Summary: Halo Extreme Block Bootstrap package to estimate the largest halo mass given the field of view and redshift depth of a survey
+Author-email: Andrea Negri <anegri@us.es>
+License-Expression: BSD-3-Clause
+Project-URL: Homepage, https://github.com/NegriAndrea/hebb
+Project-URL: Issues, https://github.com/NegriAndrea/hebb/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: numpy
+Requires-Dist: astropy
+Requires-Dist: scipy
+Requires-Dist: h5py>=3.2.0
+Requires-Dist: scipy
+Requires-Dist: matplotlib
+Requires-Dist: mpi4py
+Dynamic: license-file
+
+# hebb
+Halo Extreme Block Bootstrap package to estimate the distribution of heaviest halo mass you can find in a survery at a given redshift.
+
+`hebb` uses as a basedata the Uchuu simulation halo catalogue (https://skiesanduniverses.org/Simulations/Uchuu) to perform a block bootstrap by shooting N boxes of a volume equal of the estimated volume of a survey, and recovering the largest halo formed at a particular z, with an uncertainty estimate. The volume of the survey is computed given the survey's field of view and redshift depth, or manually selected. Optionally, the code can dump the list of halo found in the search, which can be used to perform a trace back in time with the Uchuu merger tree.
+
+Please cite the following papers if you use `hebb` in your work: Negri & Belli (2026), [Ishiyama et al. (2021)](https://ui.adsabs.harvard.edu/abs/2021MNRAS.506.4210I) (Uchuu Data Release 1).
+
+## Installation
+There are 2 ways to install `hebb`: via `PyPI`
+```
+python3 -m pip install hebb
+```
+Or directly from the repository `https://github.com/NegriAndrea/hebb/` and install locally
+```
+git clone https://github.com/NegriAndrea/hebb
+cd hebb
+python3 -m pip install -e .
+```
+
+### Database Setup
+In addition to installing the Python package, you must download a reduced version of the Uchuu database. Two versions are available, with a different cut in mass; a light [one](https://uses0-my.sharepoint.com/:u:/g/personal/anegri_us_es/IQCDx5T1XlTnSbM2YmYcFoYMATIk_pI1XKPzp4ycUJ6N12M?e=pDdoDg) (2.3 GB) and a more complete one (~40 GB) from [here](https://uses0-my.sharepoint.com/:u:/g/personal/anegri_us_es/IQBrGP-3e0xHR6pV0YkfCb1hAYS1KLSNnjmDvVb3H6ytsUA?e=fcudjY). To set the database path you can define the following environment variable in your `~/.bash_profile`
+```
+# bash
+export HEBB_DB_PATH=/path/to/database
+```
+where you have to change `/path/to/database` to the path of the downloaded database. By default, the code looks first for the most complete catalugue, and if it is not found, it will look for the light one.
+
+NOTE: In order to keep the file size manageble the database contains only log10(M200), positions and merger tree IDs; the halo positions have been binned on a 40 ckpc gridsize and stored as `uint16` integer, which is precise enough for volumes that are usually way larger than 1 cMpc.
+
+### Test installation
+The easiest way to test if everything is correctly installed and set up is to run the command
+```
+hebb 100 0. -L 1000 -M 1e14
+```
+
+## Usage
+The simplest way to use `hebb` is via command line, `hebb -h` returns the user manual:
+
+
+
+
+
+@ARTICLE{2021MNRAS.506.4210I,
+       author = {{Ishiyama}, Tomoaki and {Prada}, Francisco and {Klypin}, Anatoly A. and {Sinha}, Manodeep and {Metcalf}, R. Benton and {Jullo}, Eric and {Altieri}, Bruno and {Cora}, Sof{\'\i}a A. and {Croton}, Darren and {de la Torre}, Sylvain and {Mill{\'a}n-Calero}, David E. and {Oogi}, Taira and {Ruedas}, Jos{\'e} and {Vega-Mart{\'\i}nez}, Cristian A.},
+        title = "{The Uchuu simulations: Data Release 1 and dark matter halo concentrations}",
+      journal = {\mnras},
+         year = 2021,
+        month = sep,
+       volume = {506},
+       number = {3},
+        pages = {4210-4231},
+          doi = {10.1093/mnras/stab1755},
+archivePrefix = {arXiv},
+       eprint = {2007.14720},
+ primaryClass = {astro-ph.CO},
+       adsurl = {https://ui.adsabs.harvard.edu/abs/2021MNRAS.506.4210I},
+}

hebb-0.7/README.md

@@ -0,0 +1,57 @@
+# hebb
+Halo Extreme Block Bootstrap package to estimate the distribution of heaviest halo mass you can find in a survery at a given redshift.
+
+`hebb` uses as a basedata the Uchuu simulation halo catalogue (https://skiesanduniverses.org/Simulations/Uchuu) to perform a block bootstrap by shooting N boxes of a volume equal of the estimated volume of a survey, and recovering the largest halo formed at a particular z, with an uncertainty estimate. The volume of the survey is computed given the survey's field of view and redshift depth, or manually selected. Optionally, the code can dump the list of halo found in the search, which can be used to perform a trace back in time with the Uchuu merger tree.
+
+Please cite the following papers if you use `hebb` in your work: Negri & Belli (2026), [Ishiyama et al. (2021)](https://ui.adsabs.harvard.edu/abs/2021MNRAS.506.4210I) (Uchuu Data Release 1).
+
+## Installation
+There are 2 ways to install `hebb`: via `PyPI`
+```
+python3 -m pip install hebb
+```
+Or directly from the repository `https://github.com/NegriAndrea/hebb/` and install locally
+```
+git clone https://github.com/NegriAndrea/hebb
+cd hebb
+python3 -m pip install -e .
+```
+
+### Database Setup
+In addition to installing the Python package, you must download a reduced version of the Uchuu database. Two versions are available, with a different cut in mass; a light [one](https://uses0-my.sharepoint.com/:u:/g/personal/anegri_us_es/IQCDx5T1XlTnSbM2YmYcFoYMATIk_pI1XKPzp4ycUJ6N12M?e=pDdoDg) (2.3 GB) and a more complete one (~40 GB) from [here](https://uses0-my.sharepoint.com/:u:/g/personal/anegri_us_es/IQBrGP-3e0xHR6pV0YkfCb1hAYS1KLSNnjmDvVb3H6ytsUA?e=fcudjY). To set the database path you can define the following environment variable in your `~/.bash_profile`
+```
+# bash
+export HEBB_DB_PATH=/path/to/database
+```
+where you have to change `/path/to/database` to the path of the downloaded database. By default, the code looks first for the most complete catalugue, and if it is not found, it will look for the light one.
+
+NOTE: In order to keep the file size manageble the database contains only log10(M200), positions and merger tree IDs; the halo positions have been binned on a 40 ckpc gridsize and stored as `uint16` integer, which is precise enough for volumes that are usually way larger than 1 cMpc.
+
+### Test installation
+The easiest way to test if everything is correctly installed and set up is to run the command
+```
+hebb 100 0. -L 1000 -M 1e14
+```
+
+## Usage
+The simplest way to use `hebb` is via command line, `hebb -h` returns the user manual:
+
+
+
+
+
+@ARTICLE{2021MNRAS.506.4210I,
+       author = {{Ishiyama}, Tomoaki and {Prada}, Francisco and {Klypin}, Anatoly A. and {Sinha}, Manodeep and {Metcalf}, R. Benton and {Jullo}, Eric and {Altieri}, Bruno and {Cora}, Sof{\'\i}a A. and {Croton}, Darren and {de la Torre}, Sylvain and {Mill{\'a}n-Calero}, David E. and {Oogi}, Taira and {Ruedas}, Jos{\'e} and {Vega-Mart{\'\i}nez}, Cristian A.},
+        title = "{The Uchuu simulations: Data Release 1 and dark matter halo concentrations}",
+      journal = {\mnras},
+         year = 2021,
+        month = sep,
+       volume = {506},
+       number = {3},
+        pages = {4210-4231},
+          doi = {10.1093/mnras/stab1755},
+archivePrefix = {arXiv},
+       eprint = {2007.14720},
+ primaryClass = {astro-ph.CO},
+       adsurl = {https://ui.adsabs.harvard.edu/abs/2021MNRAS.506.4210I},
+}

hebb-0.7/pyproject.toml

@@ -0,0 +1,52 @@
+[build-system]
+requires = ["setuptools >= 77.0.3", "wheel"]
+build-backend = "setuptools.build_meta"
+#[tool.setuptools-git-versioning]
+#enabled = true
+
+[project]
+name = "hebb"
+authors = [
+    { name = "Andrea Negri", email = "anegri@us.es" }
+]
+readme = "README.md"
+description = "Halo Extreme Block Bootstrap package to estimate the largest halo mass given the field of view and redshift depth of a survey"
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Operating System :: OS Independent",
+]
+license = "BSD-3-clause"
+license-files = ["LICEN[CS]E*"]
+version = "0.7"
+
+
+requires-python = ">=3.10"
+dependencies = [
+    "numpy",
+    "astropy",
+    "scipy",
+    "h5py>=3.2.0",
+    "scipy",
+    "matplotlib",
+    "mpi4py",
+    ]
+
+#dynamic = ["version"]
+
+[project.optional-dependencies]
+
+[project.urls]
+Homepage = "https://github.com/NegriAndrea/hebb"
+Issues = "https://github.com/NegriAndrea/hebb/issues"
+
+[project.scripts]
+hebb = "hebb.hebb_cli:hebb_CLI"
+hebb-tr = "hebb.hebb_cli:hebb_trace_CLI"
+hebb-estimate = "hebb.hebb_cli:hebb_estimate_CLI"
+
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"]

hebb-0.7/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

hebb-0.7/src/hebb/__init__.py

@@ -0,0 +1,2 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-

hebb-0.7/src/hebb/hebb_cli.py

@@ -0,0 +1,164 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+from .hebb_core import hebb, hebb_estimate
+from .hebb_trace import hebb_trace
+
+def hebb_CLI():
+    import numpy as np
+    import argparse
+    import os
+
+    try:
+        def_path = os.environ['HEBB_DB_PATH']
+    except KeyError:
+        raise ValueError('The environment variable HEBB_DB_PATH must be set')
+
+    description = (
+            'Compute the heaviest dark matter halo that you can find in a given '
+            'survey with [z_min, z_max] and field-of-view by performing a '
+            'non-parametric block bootstrap over the Uchuu (2/h cGpc)^3 run.'
+            )
+    parser = argparse.ArgumentParser(description=description)
+    parser.add_argument('Nboxes', type=int, help='Number of boxes for bootstrap')
+    parser.add_argument('z_target', type=float, help='Redshift of your target')
+
+    group = parser.add_argument_group( "Processing mode (required, mutually exclusive)")
+    group_container = group.add_mutually_exclusive_group(required=True)
+    group_container.add_argument('--survey', nargs=3, type=float,
+                                 help='Survey min z, max z, fov in arcmin^2'
+                                 ' (used to compute box volume)',
+                                 metavar=("z_min", "z_max", "fov"))
+    group_container.add_argument('-L', type=float, help='Size of the box in cMpc,'
+                                 ' alternative to the boxsize computation from the FOV'
+                                 ' and z-depth of the survey')
+
+    parser.add_argument('-n', type=int, help='Track the N most massive haloes'
+                        ' in each box (1 tracks only the most massive) [default: %(default)d]', default=1)
+    parser.add_argument('-v', action='count', default=0,
+                        help='Verbosity level [default: %(default)d]')
+    parser.add_argument('-t', action='store_true', help='Create a table with'
+                        ' the sampled haloes')
+    parser.add_argument('--plot', action='store_true', help='Show a plot of the M200 distribution')
+
+    parser.add_argument('-M', type=float, help='OPTIMIZATION: Database mass'
+                        ' cut, greatly speed up the search but you can incour'
+                        ' into empty boxes [default: None]')
+    parser.add_argument('--lf', type=int, help='OPTIMIZATION: Leafe size for'
+                        ' each node of the KDTree [default: %(default)d]',
+                        default=128)
+    parser.add_argument('--force-light', action='store_true', help='DEBUG: '
+                        'force the reading of the light catalogue first')
+
+
+    args = parser.parse_args()
+
+    if args.survey is None:
+        z1 = None
+        z2 = None
+        fov = None
+    else:
+        z1  = args.survey[0]
+        z2  = args.survey[1]
+        fov = args.survey[2]
+
+    if args.n < 1:
+        raise ValueError('The value in -n needs to be a positive integer')
+
+    Mmax, fileNrMax, subNrMax = hebb(args.z_target, args.Nboxes,
+                                     def_path, z1, z2, fov,
+                                     L=args.L, M=args.M, v=args.v, leafsize =
+                                     args.lf, force_light=args.force_light,
+                                     nn=args.n)
+
+    for i in range(Mmax.shape[0]):
+        M16, M50, M86 = np.quantile(10.**Mmax[i,:].astype(np.float64), [0.16, 0.5, 0.86])
+        # print(f"log10(M200) = {np.log10(M50)} _-{(M50-M16)/M50/np.log(10)} ^+{(M86-M50)/M50/np.log(10)}")
+        print(f"{i} log10(M200) = {np.log10(M50):.2f} _-{(M50-M16)/M50/np.log(10):.2f} ^+{(M86-M50)/M50/np.log(10):.2f}")
+
+    if args.plot:
+        import matplotlib.pyplot as plt
+        fig, ax = plt.subplots()
+        for i in range(Mmax.shape[0]):
+            hist, bins = np.histogram(Mmax[i,:], 50, range=[Mmax.min(), Mmax.max()])
+            ax.plot((bins[:-1]+bins[1:])/2, hist, label=f"{i}")
+        ax.set_ylabel('N halos')
+        ax.set_xlabel(r'$\log (M_{200}/M_\odot)$')
+        ax.legend(loc='best')
+        plt.show()
+
+    if args.t:
+        from astropy.table import Table
+        import astropy.units as u
+        # unroll them
+        ordering = np.repeat(np.arange(Mmax.shape[0],
+                                       dtype=np.min_scalar_type(args.n)),
+                             Mmax.shape[1])
+
+        # alter shape to force an error in case the code attempts to copy the
+        # arrays
+        fileNrMax.shape = fileNrMax.size
+        subNrMax.shape = subNrMax.size
+        ordering.shape = ordering.size
+        Mmax.shape = Mmax.size
+
+        t=Table([Mmax, ordering, fileNrMax, subNrMax],
+                names=['M200', 'IndMostMassive', 'fileNr', 'subNr'])
+        t['M200'].unit = u.dex(u.Msun)
+        t.meta = {'Description':'Hebb result table','Nboxes':args.Nboxes,
+                  'z_target':args.z_target, '-n':args.n}
+        t.sort(['fileNr', 'subNr', 'IndMostMassive'])
+        t.write('table_max.txt', format='ascii.ecsv', overwrite=True)
+
+def hebb_trace_CLI():
+    import argparse
+
+    description = ("Trace back galaxies found with hebb.")
+    parser = argparse.ArgumentParser(description=description)
+
+    parser.add_argument('hebbTable', type=str, help='Data table produced by hebb')
+    parser.add_argument('z_target', type=float, help='Target redshift')
+    parser.add_argument('treePath', type=str, help='Path of the merger tree file')
+
+
+    parser.add_argument('-v', action='count', default=0,
+                        help='verbosity level [%(default)d]')
+    parser.add_argument('-s', action='store_true',
+                        help='Use the code in serial, without the need of MPI')
+
+    args = parser.parse_args()
+
+    finalT, mpirank = hebb_trace(args.hebbTable, args.treePath, args.z_target,
+            args.v, args.s)
+
+
+    if mpirank == 0:
+        finalT.write('hebb_traceback.txt', format='ascii.ecsv', overwrite=True)
+
+def hebb_estimate_CLI():
+    import numpy as np
+    import argparse
+    import os
+
+    try:
+        def_path = os.environ['HEBB_DB_PATH']
+    except KeyError:
+        raise ValueError('The environment variable HEBB_DB_PATH must be set')
+
+    description = (
+            'Estimate the smallest L at a certain z with a certain mass cut'
+            )
+    parser = argparse.ArgumentParser(description=description)
+    parser.add_argument('z_target', type=float, help='Number of sampling')
+    parser.add_argument('L', type=float, help='Size of the box in cMpc')
+
+    parser.add_argument('-v', action='count', default=0,
+                        help='verbosity level [%(default)d]')
+
+    parser.add_argument('-M', type=float, help='OPTIMIZATION: Database mass'
+                        ' cut, greatly speed up the search but you can incour'
+                        ' into empty boxes [default: None]')
+
+
+    args = parser.parse_args()
+
+    hebb_estimate(args.z_target, def_path, args.L, M=args.M, v=args.v)

hebb-0.7/src/hebb/hebb_core.py

@@ -0,0 +1,327 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+import numpy as np
+import h5py
+from pathlib import PurePath, Path
+import astropy.units as u
+import astropy.cosmology.units as cu
+u.add_enabled_units(cu)
+import numpy as np
+# from numba import njit
+from scipy import spatial
+
+def comov_volume(area, z1, z2):
+    """
+    Area must be with an astropy unit, e.g. 300*(u.arcmin**2)
+    Returns size lenght of a box in cMpc
+
+    """
+    from astropy.cosmology import Planck15
+    Omega     = area.to(u.steradian).value # get rid of unit
+    d2        = Planck15.comoving_distance(z1)
+    d3        = Planck15.comoving_distance(z2)
+    V         = Omega/3 * (d3**3 - d2**3)
+    newL      = np.cbrt(V.to(u.Mpc**3))
+
+    return newL
+
+# @njit
+def dL(coord1, coord2_in, boxsize):
+    """
+    Computes the distance in a 3D periodic box between a set of coordinates and
+    a point, or another, broadcastable set of coordinates.
+
+    coord1: array of dimensions (n,3)
+
+    coord2: array of dimensions (1,3) or (3,) in the case of a single point, or
+                array of size (n,3) in case of multiple points
+
+    boxsize : scalar, the size of the box, each dimension has the same size
+
+    Returns: the distance squared
+
+
+    usage:
+
+    import numpy as np
+
+    box = 0.5
+
+    # generate fake data
+    rng = np.random.default_rng(12345)
+    c1 = rng.random((1000,3))
+
+    distancePeriodic(c1, np.array([0.,0.,0.], box))
+
+
+    c2 = rng.random((1000,3))
+    distancePeriodic(c1, c2, box))
+
+    """
+
+    message = 'coord1 must be a two dimensional array with shape[1]==3'
+    coord2=np.atleast_2d(coord2_in)
+
+    if coord1.ndim != 2:
+        raise ValueError(message)
+
+    if coord1.shape[1] != 3:
+        raise ValueError(message)
+
+    if coord2.ndim == 1:
+        if coord2.size != 3:
+            raise ValueError('I need 3 points as np.array([x,y,z])')
+    elif coord2.ndim == 2:
+        if coord2.shape[1] != 3:
+            raise ValueError('I need 3 points as np.array([x,y,z])')
+    else:
+        raise ValueError('I need 3 points as np.array([x,y,z])')
+
+    dx = coord1[:,0] - coord2[:,0]
+    dy = coord1[:,1] - coord2[:,1]
+    dz = coord1[:,2] - coord2[:,2]
+
+    dx[(dx > boxsize*0.5)] -= boxsize
+    dx[(dx < - boxsize*0.5)] += boxsize
+    dy[(dy > boxsize*0.5)] -= boxsize
+    dy[(dy < - boxsize*0.5)] += boxsize
+    dz[(dz > boxsize*0.5)] -= boxsize
+    dz[(dz < - boxsize*0.5)] += boxsize
+
+    return dx, dy, dz
+
+# @njit(parallel=True)
+def bootstrap_brute_force(Nboxes, BoxSize, newL, coords, centers, mass, fileNr, subNr):
+
+    Mmax = np.zeros(Nboxes)
+    fileNrMax = np.zeros(Nboxes, dtype=fileNr.dtype)
+    subNrMax = np.zeros(Nboxes, dtype=subNr.dtype)
+
+    for j in range(Nboxes):
+
+        dx, dy, dz = dL(coords, centers[j,:], BoxSize)
+        mask = (np.abs(dx) < newL) & (np.abs(dy) < newL) & (np.abs(dz) < newL)
+
+        if np.count_nonzero(mask) == 0:
+            raise ValueError('Hitting an empty region, in case you used -M try'
+                             ' to lower (or omit) the mass cut')
+        mass_tmp = mass[mask]
+        fileNr_tmp = fileNr[mask]
+        subNr_tmp = subNr[mask]
+
+        index = np.argmax(mass_tmp)
+
+        Mmax[j] = mass_tmp[index]
+        fileNrMax[j] = fileNr_tmp[index]
+        subNrMax[j] = subNr_tmp[index]
+
+    return Mmax, fileNrMax, subNrMax
+
+def bootstrap_kdtree_double(Nboxes, BoxSize, newL, coords, centers, mass, fileNr, subNr):
+    """
+    Do a search using a double KDTree and perform a block bootstrap.
+
+    """
+
+    Mmax = np.zeros(Nboxes)
+    fileNrMax = np.zeros(Nboxes, dtype=fileNr.dtype)
+    subNrMax = np.zeros(Nboxes, dtype=subNr.dtype)
+
+    tree_data = spatial.KDTree(coords, boxsize=BoxSize, leafsize=32)
+    tree_centres = spatial.KDTree(centers, boxsize=BoxSize, leafsize=32)
+    indexes = tree_centres.query_ball_tree(tree_data, newL, p=np.inf)
+
+    assert len(indexes) == Nboxes
+    for j in range(Nboxes):
+
+        if len(indexes[j]) == 0:
+            raise ValueError('Hitting an empty region, in case you used -M try'
+                             ' to lower (or omit) the mass cut')
+        ind = np.array(indexes[j],
+                             dtype=np.min_scalar_type(coords.shape[0]))
+        mass_tmp = mass[ind]
+        fileNr_tmp = fileNr[ind]
+        subNr_tmp = subNr[ind]
+
+        index = np.argmax(mass_tmp)
+        Mmax[j] = mass_tmp[index]
+        fileNrMax[j] = fileNr_tmp[index]
+        subNrMax[j] = subNr_tmp[index]
+
+    return Mmax, fileNrMax, subNrMax
+
+def bootstrap_kdtree_single(Nboxes, BoxSize, newL, coords, centers, mass,
+                            fileNr, subNr, leafsize=128, nn=1):
+    """
+    Do a search using a single KDTree and perform a block bootstrap. Better
+    than the double one since the centers are uniformly seeded in the volume,
+    and it can be parallelized
+
+    """
+
+    Mmax = np.zeros((nn,Nboxes), dtype=mass.dtype)
+    fileNrMax = np.zeros((nn,Nboxes), dtype=fileNr.dtype)
+    subNrMax = np.zeros((nn,Nboxes), dtype=subNr.dtype)
+
+    tree = spatial.KDTree(coords, boxsize=BoxSize, leafsize=leafsize)
+    dtype=np.min_scalar_type(coords.shape[0])
+
+    for j in range(Nboxes):
+        ind = np.array(tree.query_ball_point(centers[j,:], newL, p=np.inf, workers=-1), dtype=dtype)
+
+        if ind.size < nn:
+            raise ValueError('Hitting a region with less than '
+                             f"{nn} subhalos, in case you used -M try"
+                             ' to lower (or omit) the mass cut')
+
+        mass_tmp = mass[ind]
+        fileNr_tmp = fileNr[ind]
+        subNr_tmp = subNr[ind]
+
+        index = np.argmax(mass_tmp)
+        index = np.argsort(mass_tmp)[-nn:][::-1]
+        Mmax[:,j] = mass_tmp[index]
+        fileNrMax[:,j] = fileNr_tmp[index]
+        subNrMax[:,j] = subNr_tmp[index]
+
+    return Mmax, fileNrMax, subNrMax
+
+
+
+def hebb(z_target, Nboxes, path_data, z1, z2, fov, L=None, M=None, v=0,
+         leafsize=128, force_light=False, nn = 1):
+    from .uchuu_snaps_z import uchuu_snap_list
+
+    snapNr_list, z = uchuu_snap_list()
+    snapNr = snapNr_list[np.abs(z - z_target).argmin()]
+
+    # Boxsize in cMpc
+    BoxSize = 2000/0.6774
+
+    catFileName = Path(path_data)/f'catalogue_uchuu.hdf5'
+    if not catFileName.is_file() or force_light:
+        catFileName = Path(path_data)/f'catalogue_uchuu_light.hdf5'
+
+    if not catFileName.is_file():
+        raise IOError('I cannot locate the catalogue file, I have tried'
+                      f"{Path(path_data)/f'catalogue_uchuu.hdf5'}"
+                      f" and {Path(path_data)/f'catalogue_uchuu_light.hdf5'}"
+                      f" visit https://github.com/NegriAndrea/hebb")
+
+    if v>0:
+        print(f"Reading {catFileName}")
+
+    with h5py.File(catFileName, 'r') as ff:
+
+        # the catalogue is sorted in M200, with histograms to load only the
+        # haloes above a certain mass without having to read the full dataset
+        # if not needed
+        if M is None:
+            offset = 0
+        else:
+            m200_indexes = ff[f'S-{snapNr}/M200_indexes'][()]
+            m200_bins_edges = ff[f'S-{snapNr}/M200c_bins_edges'][()]
+            if np.log10(M) < m200_bins_edges[0]:
+                raise ValueError(f"The requested mass {M=:.2e} Msun is too low for "
+                                 f"z={z_target}, database min mass at this z "
+                                 f"is {10.**float(m200_bins_edges[0]):.2e} Msun")
+            tmp = np.searchsorted(m200_bins_edges, np.log10(M))
+            offset = int(m200_indexes[max(tmp-1,0)])
+
+        coords = ff[f'S-{snapNr}/Coordinates'][offset:]
+        mass= ff[f'S-{snapNr}/M200c'][offset:]
+        fileNr = ff[f'S-{snapNr}/fileNr'][offset:]
+        subNr = ff[f'/S-{snapNr}/SubNr'][offset:]
+        bin_size = BoxSize/65_535
+
+    coords=coords.astype(np.float32)*bin_size
+
+    if v>0:
+        print(f"READ log(M200/Msun): min={mass[0]:.2f},"
+              f" max={mass[-1]:.2f}")
+
+    if L is None:
+        # for surveys
+        area = fov*(u.arcmin**2)
+        newL = comov_volume(area,z1, z2).value/2 # in cMpc
+    else:
+        newL = L/2.
+
+    if v>0:
+        print(f"V={8*newL**3:3e} cMpc^3,  L={newL*2:3f} cMpc")
+
+    # shoot (Nboxes,3) random numbers between 0 and BoxSize
+    rng = np.random.default_rng()
+    centers=rng.random(size=(Nboxes,3), dtype=np.float32)
+    centers*=BoxSize
+
+
+    # Mmax, fileNrMax, subNrMax = bootstrap_brute_force(Nboxes, BoxSize, newL, coords,
+                                          # centers, mass, fileNr, subNr)
+    # Mmax, fileNrMax, subNrMax = bootstrap_kdtree_double(Nboxes, BoxSize, newL, coords,
+                                          # centers, mass, fileNr, subNr)
+    Mmax, fileNrMax, subNrMax = bootstrap_kdtree_single(Nboxes, BoxSize, newL, coords,
+                                                        centers, mass, fileNr, subNr,
+                                                        leafsize=leafsize,
+                                                        nn=nn)
+
+    return Mmax, fileNrMax, subNrMax
+
+def hebb_estimate(z_target, path_data, L, M=None, v=0):
+    from .uchuu_snaps_z import uchuu_snap_list
+
+    snapNr_list, z = uchuu_snap_list()
+    snapNr = snapNr_list[np.abs(z - z_target).argmin()]
+
+    # Boxsize in cMpc
+    BoxSize = 2000/0.6774
+
+    catFileName = Path(path_data)/f'catalogue_uchuu.hdf5'
+    if not catFileName.is_file():
+        catFileName = Path(path_data)/f'catalogue_uchuu_light.hdf5'
+
+    if not catFileName.is_file():
+        raise IOError('I cannot locate the catalogue file, I have tried'
+                      f"{Path(path_data)/f'catalogue_uchuu.hdf5'}"
+                      f" and {Path(path_data)/f'catalogue_uchuu_light.hdf5'}")
+
+    if v>0:
+        print(f"Reading {catFileName}")
+
+    with h5py.File(catFileName, 'r') as ff:
+
+        # the catalogue is sorted in M200, with histograms to load only the
+        # haloes above a certain mass without having to read the full dataset
+        # if not needed
+        if M is None:
+            offset = 0
+        else:
+            m200_indexes = ff[f'S-{snapNr}/M200_indexes'][()]
+            m200_bins_edges = ff[f'S-{snapNr}/M200c_bins_edges'][()]
+            if np.log10(M) < m200_bins_edges[0]:
+                raise ValueError(f"The requested mass {M=:.2e} Msun is too low for "
+                                 f"z={z_target}, database min mass at this z "
+                                 f"is {10.**float(m200_bins_edges[0]):.2e} Msun")
+            tmp = np.searchsorted(m200_bins_edges, np.log10(M))
+            offset = int(m200_indexes[max(tmp-1,0)])
+
+        coords = ff[f'S-{snapNr}/Coordinates'][offset:]
+        mass= ff[f'S-{snapNr}/M200c'][offset:]
+        fileNr = ff[f'S-{snapNr}/fileNr'][offset:]
+        subNr = ff[f'/S-{snapNr}/SubNr'][offset:]
+        bin_size = BoxSize/65_535
+
+    coords=coords.astype(np.float32)*bin_size
+
+    if v>0:
+        print(f"READ log(M200/Msun): min={mass[0]:.2f},"
+              f" max={mass[-1]:.2f}")
+
+
+    for n in range(32):
+        nbins = 2**n
+        H, bins = np.histogramdd(coords, bins=nbins)
+        stop_cond = np.any(H==0)
+        print(f"L={BoxSize/nbins} {stop_cond}")
+        if stop_cond:
+            return

hebb-0.7/src/hebb/hebb_trace.py

@@ -0,0 +1,136 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+from astropy.table import Table, vstack
+
+def unique_ordered(x):
+    """
+    Same output of np.unique when return_counts=True,
+    return_index=True, but 1000 faster since x is assumed sorted.
+    It works on 1d arrays, untested for other shapes.
+    """
+    import numpy as np
+
+    x = np.asarray(x)
+    if x.ndim != 1:
+        raise ValueError('x must be a 1D array-like')
+
+    # compare the elements and find where the elements change. This
+    # is better than using np.diff, since the comparison always
+    # work, while for some arrays (like recarrays) the difference
+    # may be not defined
+    flt =  np.flatnonzero(x[1:] != x[:-1])
+
+    # by construction, we to add 1 to flt
+    off = np.concatenate([[0], flt+1])
+    dim = np.concatenate([np.diff(off), [x.size-off[-1]]])
+    unique = x[off]
+
+    return unique, off, dim
+
+def hebb_trace(tableName, mergerTreePath, targetZ, v, serial):
+
+    if serial:
+        # do it serially
+        t = Table.read(tableName, format='ascii.ecsv')
+        return (hebb_trace_single(tableName, mergerTreePath, targetZ, v, 0,
+                len(t)),0)
+    else:
+        import helperspy as hpy
+        from mpi4py import MPI
+        comm = MPI.COMM_WORLD
+        rank = comm.Get_rank()
+        size = comm.Get_size()
+
+        t = Table.read(tableName, format='ascii.ecsv')
+        t.sort(['fileNr'])
+
+        offset, sizes = hpy.chopArray(len(t), size)
+
+        i_begin = offset[rank]
+        i_end = sizes[rank]+i_begin
+
+        local_t = hebb_trace_single(tableName,
+                mergerTreePath, targetZ, v, i_begin, i_end)
+
+        all_tables = comm.gather(local_t, root=0)
+
+        if rank == 0:
+            final_table = vstack(all_tables)
+            return final_table, rank
+        else:
+            return None, rank
+
+
+
+
+def hebb_trace_single(tableName, mergerTreePath, targetZ, v, i_begin, i_end):
+    """
+    Trace back galaxies found with hebb.
+
+    """
+    import numpy as np
+    from pathlib import PurePath
+    import astropy.units as u
+    import astropy.cosmology.units as cu
+    u.add_enabled_units(cu)
+    import helperspy as hpy
+
+    from .uchuu_snaps_z import uchuu_snap_list
+
+    snapNr_list, z = uchuu_snap_list()
+    targetSnapNr = snapNr_list[np.abs(z - targetZ).argmin()]
+
+    t = Table.read(tableName, format='ascii.ecsv')
+    t=t[i_begin:i_end]
+
+    t.sort(['fileNr', 'subNr'])
+
+    UfileNr, off, size = unique_ordered(t['fileNr'])
+
+    if v>1:
+        tmp = Table([UfileNr, size], names=['MergTreeID', 'Nhalos'])
+        print(tmp)
+        del tmp
+
+    fullT = []
+    fileNr_original = []
+    subNr_original = []
+
+    # I loop over the merger tree files (MT)
+    for jMT, o, s in zip(UfileNr, off, size):
+
+        # table containing only files related to 1 merger tree file
+        subt = t[o:o+s]
+
+        fileMT = PurePath(mergerTreePath) / f'mergertree_{jMT}.h5'
+
+        if v>0:
+            print(f'Read {fileMT}')
+        # in case I want the position, use newFields=['Pos']
+        tree = hpy.forestCT(fileMT, newFields=[])
+
+        for sNr in subt['subNr']:
+            if v>0:
+                print(f'Doing {sNr}')
+
+            try:
+                # depends what I want to find, in this case the progenitor
+                # do I need raiseError=True?
+                tHist = tree.subHistory(sNr, targetSnapNr=targetSnapNr,
+                                        t=True, raiseError=True)
+
+                # filter to get the target I want
+                fullT.append(tHist[tHist['SnapNr'] == targetSnapNr])
+                fileNr_original.append(jMT)
+                subNr_original.append(sNr)
+            except hpy.CTNoProgenitorError:
+                if v>1:
+                    print('NoProgenitor')
+
+    fullT = vstack(fullT)
+    fullT['fileNr_original'] = np.array(fileNr_original,
+            dtype=t['fileNr'].dtype)
+    fullT['subNr_original'] = np.array(subNr_original,
+            dtype=t['subNr'].dtype)
+
+    return fullT

hebb-0.7/src/hebb/uchuu_snaps_z.py

@@ -0,0 +1,110 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+def uchuu_snap_list():
+    import numpy as np
+    snapNr = np.array([
+        0  ,
+        1  ,
+        2  ,
+        3  ,
+        4  ,
+        5  ,
+        6  ,
+        7  ,
+        8  ,
+        9  ,
+        10 ,
+        11 ,
+        12 ,
+        13 ,
+        14 ,
+        15 ,
+        16 ,
+        17 ,
+        18 ,
+        19 ,
+        20 ,
+        21 ,
+        22 ,
+        23 ,
+        24 ,
+        25 ,
+        26 ,
+        27 ,
+        28 ,
+        29 ,
+        30 ,
+        31 ,
+        32 ,
+        33 ,
+        34 ,
+        35 ,
+        36 ,
+        37 ,
+        38 ,
+        39 ,
+        40 ,
+        41 ,
+        42 ,
+        43 ,
+        44 ,
+        45 ,
+        46 ,
+        47 ,
+        48 ,
+        49 ], dtype=np.uint8)
+
+    z = np.array([
+        13.956626    ,
+        12.687381    ,
+        11.509382    ,
+        10.4416485   ,
+        9.472301     ,
+        8.587728     ,
+        7.764242     ,
+        7.0256824    ,
+        6.347539     ,
+        5.725402     ,
+        5.1549826    ,
+        4.630631     ,
+        4.268704     ,
+        3.9307232    ,
+        3.6095695    ,
+        3.3151808    ,
+        3.12916      ,
+        2.9508514    ,
+        2.779575     ,
+        2.6155903    ,
+        2.4582927    ,
+        2.309505     ,
+        2.1655588    ,
+        2.0288346    ,
+        1.8969555    ,
+        1.7713115    ,
+        1.6509027    ,
+        1.5358186    ,
+        1.4257126    ,
+        1.3211012    ,
+        1.2202487    ,
+        1.1240892    ,
+        1.0319009    ,
+        0.9442769    ,
+        0.8595655    ,
+        0.77916944   ,
+        0.70206976   ,
+        0.62813413   ,
+        0.55789936   ,
+        0.48993552   ,
+        0.42559803   ,
+        0.36407042   ,
+        0.3046825    ,
+        0.2480967    ,
+        0.19424379   ,
+        0.14247847   ,
+        0.0928843    ,
+        0.04553294   ,
+        0.022683144  ,
+        2.002716e-05 ], dtype=np.float32)
+
+    return snapNr, z

hebb-0.7/src/hebb.egg-info/PKG-INFO

@@ -0,0 +1,79 @@
+Metadata-Version: 2.4
+Name: hebb
+Version: 0.7
+Summary: Halo Extreme Block Bootstrap package to estimate the largest halo mass given the field of view and redshift depth of a survey
+Author-email: Andrea Negri <anegri@us.es>
+License-Expression: BSD-3-Clause
+Project-URL: Homepage, https://github.com/NegriAndrea/hebb
+Project-URL: Issues, https://github.com/NegriAndrea/hebb/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: numpy
+Requires-Dist: astropy
+Requires-Dist: scipy
+Requires-Dist: h5py>=3.2.0
+Requires-Dist: scipy
+Requires-Dist: matplotlib
+Requires-Dist: mpi4py
+Dynamic: license-file
+
+# hebb
+Halo Extreme Block Bootstrap package to estimate the distribution of heaviest halo mass you can find in a survery at a given redshift.
+
+`hebb` uses as a basedata the Uchuu simulation halo catalogue (https://skiesanduniverses.org/Simulations/Uchuu) to perform a block bootstrap by shooting N boxes of a volume equal of the estimated volume of a survey, and recovering the largest halo formed at a particular z, with an uncertainty estimate. The volume of the survey is computed given the survey's field of view and redshift depth, or manually selected. Optionally, the code can dump the list of halo found in the search, which can be used to perform a trace back in time with the Uchuu merger tree.
+
+Please cite the following papers if you use `hebb` in your work: Negri & Belli (2026), [Ishiyama et al. (2021)](https://ui.adsabs.harvard.edu/abs/2021MNRAS.506.4210I) (Uchuu Data Release 1).
+
+## Installation
+There are 2 ways to install `hebb`: via `PyPI`
+```
+python3 -m pip install hebb
+```
+Or directly from the repository `https://github.com/NegriAndrea/hebb/` and install locally
+```
+git clone https://github.com/NegriAndrea/hebb
+cd hebb
+python3 -m pip install -e .
+```
+
+### Database Setup
+In addition to installing the Python package, you must download a reduced version of the Uchuu database. Two versions are available, with a different cut in mass; a light [one](https://uses0-my.sharepoint.com/:u:/g/personal/anegri_us_es/IQCDx5T1XlTnSbM2YmYcFoYMATIk_pI1XKPzp4ycUJ6N12M?e=pDdoDg) (2.3 GB) and a more complete one (~40 GB) from [here](https://uses0-my.sharepoint.com/:u:/g/personal/anegri_us_es/IQBrGP-3e0xHR6pV0YkfCb1hAYS1KLSNnjmDvVb3H6ytsUA?e=fcudjY). To set the database path you can define the following environment variable in your `~/.bash_profile`
+```
+# bash
+export HEBB_DB_PATH=/path/to/database
+```
+where you have to change `/path/to/database` to the path of the downloaded database. By default, the code looks first for the most complete catalugue, and if it is not found, it will look for the light one.
+
+NOTE: In order to keep the file size manageble the database contains only log10(M200), positions and merger tree IDs; the halo positions have been binned on a 40 ckpc gridsize and stored as `uint16` integer, which is precise enough for volumes that are usually way larger than 1 cMpc.
+
+### Test installation
+The easiest way to test if everything is correctly installed and set up is to run the command
+```
+hebb 100 0. -L 1000 -M 1e14
+```
+
+## Usage
+The simplest way to use `hebb` is via command line, `hebb -h` returns the user manual:
+
+
+
+
+
+@ARTICLE{2021MNRAS.506.4210I,
+       author = {{Ishiyama}, Tomoaki and {Prada}, Francisco and {Klypin}, Anatoly A. and {Sinha}, Manodeep and {Metcalf}, R. Benton and {Jullo}, Eric and {Altieri}, Bruno and {Cora}, Sof{\'\i}a A. and {Croton}, Darren and {de la Torre}, Sylvain and {Mill{\'a}n-Calero}, David E. and {Oogi}, Taira and {Ruedas}, Jos{\'e} and {Vega-Mart{\'\i}nez}, Cristian A.},
+        title = "{The Uchuu simulations: Data Release 1 and dark matter halo concentrations}",
+      journal = {\mnras},
+         year = 2021,
+        month = sep,
+       volume = {506},
+       number = {3},
+        pages = {4210-4231},
+          doi = {10.1093/mnras/stab1755},
+archivePrefix = {arXiv},
+       eprint = {2007.14720},
+ primaryClass = {astro-ph.CO},
+       adsurl = {https://ui.adsabs.harvard.edu/abs/2021MNRAS.506.4210I},
+}

hebb-0.7/src/hebb.egg-info/SOURCES.txt

@@ -0,0 +1,14 @@
+LICENSE.md
+README.md
+pyproject.toml
+src/hebb/__init__.py
+src/hebb/hebb_cli.py
+src/hebb/hebb_core.py
+src/hebb/hebb_trace.py
+src/hebb/uchuu_snaps_z.py
+src/hebb.egg-info/PKG-INFO
+src/hebb.egg-info/SOURCES.txt
+src/hebb.egg-info/dependency_links.txt
+src/hebb.egg-info/entry_points.txt
+src/hebb.egg-info/requires.txt
+src/hebb.egg-info/top_level.txt

hebb-0.7/src/hebb.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

hebb-0.7/src/hebb.egg-info/entry_points.txt

@@ -0,0 +1,4 @@
+[console_scripts]
+hebb = hebb.hebb_cli:hebb_CLI
+hebb-estimate = hebb.hebb_cli:hebb_estimate_CLI
+hebb-tr = hebb.hebb_cli:hebb_trace_CLI

hebb-0.7/src/hebb.egg-info/requires.txt

@@ -0,0 +1,7 @@
+numpy
+astropy
+scipy
+h5py>=3.2.0
+scipy
+matplotlib
+mpi4py

hebb-0.7/src/hebb.egg-info/top_level.txt

@@ -0,0 +1 @@
+hebb