dftracer-utils 0.0.13.post27.dev0__tar.gz → 0.0.14.post2.dev0__tar.gz

{dftracer_utils-0.0.13.post27.dev0 → dftracer_utils-0.0.14.post2.dev0}/CHANGELOG.md

@@ -9,6 +9,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
 
 ### Changed
 
+- Keep on PyPI only the newest `<tag>.postN.devX` prerelease of the previous release when a release is published, and delete the older ones, so the project stays under its PyPI storage limit.
+
+### Fixed
+
+- Grow an elastic runtime for a burst of tasks sent before its parked worker starts to run. The burst stayed on one worker for up to one `elastic_keepalive` (250 ms by default) before the pool grew.
+
+## [0.0.14] - 2026-10-10
+
+### Changed
+
 - Make the 2048-bin DDSketch of `dftracer_genesis_gen_dist` and the duql percentile folds grow its store with the key range it sees instead of holding all 2048 bins (8 KB) from the start: on 60 runs of 2000 call paths with 8 counters the tool peaks at 0.72 GB instead of 1.7 GB, runs that needed a 4 GB budget fit in 2 GB, and creating a sketch is 4.7 times faster. The sketch serializes to the same bytes. The genesis state pool now counts each growth of a store, and the 128-bin sketch of the aggregation engine keeps its inline array.
 - Make `dftracer_genesis_gen_dist` find a call's counter samples from where the previous call of its lane left each series, instead of a binary search over the whole series for every call and counter: 4 runs with 64 counter keys take 3.9 s instead of 6.2 s, with the same output.
 - Make `dftracer_genesis_gen_dist` write each run as soon as it is done, through its worker's own producer of a parallel gzip writer, instead of holding every finished run in memory until the runs before it in discovery order were written. One slow run early in the order no longer stalls the output and grows memory with every later run, and the writer now takes an eighth of `--memory-budget` instead of a budget of its own. The output holds the same lines as before, but runs, and the members of one run, appear in the order they finish.
@@ -1435,7 +1445,8 @@ Initial release.
 - Add gzip block-boundary-aware reading with prefix-based hashes and checkpointing.
 - Add Python packaging and Sphinx documentation with a C++ API reference.
 
-[Unreleased]: https://github.com/llnl-asr/dftracer-utils/compare/v0.0.13...HEAD
+[Unreleased]: https://github.com/llnl-asr/dftracer-utils/compare/v0.0.14...HEAD
+[0.0.14]: https://github.com/llnl-asr/dftracer-utils/compare/v0.0.13...v0.0.14
 [0.0.13]: https://github.com/llnl-asr/dftracer-utils/compare/v0.0.12...v0.0.13
 [0.0.12]: https://github.com/llnl-asr/dftracer-utils/compare/v0.0.11...v0.0.12
 [0.0.11]: https://github.com/llnl-asr/dftracer-utils/compare/v0.0.10...v0.0.11

{dftracer_utils-0.0.13.post27.dev0 → dftracer_utils-0.0.14.post2.dev0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: dftracer-utils
-Version: 0.0.13.post27.dev0
+Version: 0.0.14.post2.dev0
 Summary: Python binding for dftracer-utils
 Author-Email: Ray Andrew Sinurat <raydreww@gmail.com>, "Hariharan Devarajan (Hari)" <hariharandev1@llnl.gov>
 License: MIT

{dftracer_utils-0.0.13.post27.dev0 → dftracer_utils-0.0.14.post2.dev0}/python/dftracer/utils/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.0.13.post27.dev0'
-__version_tuple__ = version_tuple = (0, 0, 13, 'post27', 'dev0')
+__version__ = version = '0.0.14.post2.dev0'
+__version_tuple__ = version_tuple = (0, 0, 14, 'post2', 'dev0')
 
 __commit_id__ = commit_id = None

dftracer_utils-0.0.14.post2.dev0/scripts/ci/pypi_cleanup_prereleases.py

@@ -0,0 +1,117 @@
+#!/usr/bin/env python3
+"""Delete the PyPI prereleases <version>.postN.devX of one release but the newest.
+
+PyPI has no API to delete a release, so this logs in to the website through
+pypi-cleanup (https://github.com/arcivanov/pypi-cleanup). The account must own
+the project and have TOTP two-factor authentication.
+
+Modes: --query-only lists the plan with no login. The default also logs in and
+has pypi-cleanup list its own plan, but deletes nothing. --do-it deletes.
+
+Environment for a login: PYPI_CLEANUP_USERNAME, PYPI_CLEANUP_PASSWORD and
+PYPI_CLEANUP_TOTP_SECRET (the base32 secret of the TOTP device).
+"""
+
+import argparse
+import json
+import os
+import re
+import shutil
+import subprocess
+import sys
+import time
+import urllib.request
+
+
+def plan(versions, version):
+    """Return (keep, delete) among the <version>.postN.devX of versions.
+
+    keep is the one with the highest (N, X), or None when there is none.
+    """
+    pattern = re.compile(rf"{re.escape(version)}\.post(\d+)\.dev(\d+)")
+    found = []
+    for v in versions:
+        m = pattern.fullmatch(v)
+        if m:
+            found.append(((int(m.group(1)), int(m.group(2))), v))
+    ordered = [v for _, v in sorted(found)]
+    if not ordered:
+        return None, []
+    return ordered[-1], ordered[:-1]
+
+
+def delete_regex(delete):
+    return "^(" + "|".join(re.escape(v) for v in delete) + ")$"
+
+
+def pypi_versions(host, package):
+    request = urllib.request.Request(
+        f"{host}/simple/{package}/",
+        headers={"Accept": "application/vnd.pypi.simple.v1+json"},
+    )
+    with urllib.request.urlopen(request, timeout=60) as response:
+        return json.load(response)["versions"]
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    parser.add_argument("--package", required=True)
+    parser.add_argument(
+        "--version", required=True, help="release whose prereleases go, e.g. 0.0.14"
+    )
+    parser.add_argument("--host", default="https://pypi.org")
+    mode = parser.add_mutually_exclusive_group()
+    mode.add_argument("--query-only", action="store_true")
+    mode.add_argument("--do-it", action="store_true")
+    args = parser.parse_args()
+    host = args.host.rstrip("/")
+
+    keep, delete = plan(pypi_versions(host, args.package), args.version)
+    if keep is None:
+        print(f"no {args.version}.postN.devX prereleases on {host}")
+        return 0
+    print(f"keep {keep}")
+    for v in delete:
+        print(f"delete {v}")
+    if not delete or args.query_only:
+        return 0
+
+    tool = shutil.which("pypi-cleanup")
+    if tool is None:
+        sys.exit("pypi-cleanup is not on PATH")
+    for name in ("PYPI_CLEANUP_USERNAME", "PYPI_CLEANUP_PASSWORD", "PYPI_CLEANUP_TOTP_SECRET"):
+        if not os.environ.get(name):
+            sys.exit(f"{name} is not set")
+
+    import pyotp
+
+    cmd = [
+        tool,
+        "-t",
+        host + "/",
+        "-p",
+        args.package,
+        "-r",
+        delete_regex(delete),
+        "-u",
+        os.environ["PYPI_CLEANUP_USERNAME"],
+    ]
+    if args.do_it:
+        cmd.append("--do-it")
+    # pypi-cleanup reads the two-factor code from stdin after the password login.
+    code = pyotp.TOTP(os.environ["PYPI_CLEANUP_TOTP_SECRET"]).now()
+    result = subprocess.run(cmd, input=code + "\n", text=True, check=False)
+    if result.returncode != 0 or not args.do_it:
+        return result.returncode
+
+    # The simple index sits behind a CDN, so give a delete time to show.
+    for _ in range(10):
+        left, gone = plan(pypi_versions(host, args.package), args.version)
+        if left == keep and not gone:
+            return 0
+        time.sleep(30)
+    sys.exit(f"after the delete {host} still lists {gone + [left]}, expected only {keep}")
+
+
+if __name__ == "__main__":
+    sys.exit(main())

{dftracer_utils-0.0.13.post27.dev0 → dftracer_utils-0.0.14.post2.dev0}/src/dftracer/utils/core/pipeline/thread_pool_executor.cpp

@@ -621,9 +621,14 @@ void ThreadPoolExecutor::schedule_coroutine_resumption(
 void ThreadPoolExecutor::signal_global_work() {
     work_signal_.fetch_add(1, std::memory_order_acq_rel);
     wake_one_worker();
-    if (min_workers_ < num_threads_ &&
-        idle_workers_.load(std::memory_order_acquire) == 0 &&
-        live_workers_.load(std::memory_order_relaxed) < num_threads_) {
+    if (min_workers_ >= num_threads_ ||
+        live_workers_.load(std::memory_order_relaxed) >= num_threads_) {
+        return;
+    }
+    // A woken worker counts as idle until it runs, so a burst sent before it
+    // runs never sees zero idle workers; compare the backlog with them too.
+    const std::size_t idle = idle_workers_.load(std::memory_order_acquire);
+    if (idle == 0 || run_queue_.size_approx() > idle) {
         {
             std::lock_guard<std::mutex> lock(coord_mutex_);
             grow_pending_ = true;