requake 0.8__tar.gz → 0.8.2__tar.gz

{requake-0.8 → requake-0.8.2}/CHANGELOG.md

@@ -2,6 +2,39 @@
 
 Copyright (c) 2021-2026 Claudio Satriano <satriano@ipgp.fr>
 
+## [0.8.2] - 2026-06-03
+
+Note: you might want to run `requake update_config` to update your config file
+to the latest version.
+
+- Parallel processing of earthquake pairs in `scan_catalog`. On by default
+  with automatic detection of the number of worker processes. This can be
+  configured with the `catalog_scan_nprocs` config option.
+
+## [0.8.1] - 2026-05-31
+
+- Slimmed-down DB size by optimizing `event_pairs` storage. This allows storing
+  also invalid pairs, so that resuming catalog scan is faster.
+- Dramatic speedup of event pairs building by using a KD-tree
+  spatial index and approximate geodetic distance calculations.
+- Dramatic speedup of waveform fetching by improving the caching mechanism.
+  A new config option `catalog_waveform_cache_size` allows to set the maximum
+  number of waveforms to cache.
+- Dramatically faster reading of event pairs with cross-correlation
+  filters in large catalogs, thanks to the new SQLite-backed storage.
+- Add an optional disk cache, through the `catalog_waveform_disk_cache_enabled`
+  config option.
+- `scan_catalog`: when running as a batch job, log a progress
+  report every minute, instead of showing the progress bar.
+- `scan_catalog`: if event pairs already exist in the database,
+  ask whether to overwrite or continue from an interrupted run.
+  Add `--force` to restart from scratch and `--force-continue`
+  (short `-c`) to resume without prompting.
+- `print_families`: add horizontal and vertical distance min/max values
+  to both summary and detailed (`-d`) outputs.
+- `print_families`: add `-f stats` output format for summary family
+  statistics.
+
 ## [0.8] - 2026-05-22
 
 This release introduces a major storage change.
@@ -13,6 +46,9 @@ data access performance, especially for large catalogs.
 This change is not backward compatible at the data format level. The
 high-level behavior of the command-line interface remains unchanged.
 
+Note: further improvements based on this new storage system will be introduced
+throughout the 0.8.x release series.
+
 ## [0.7.3] - 2026-04-14
 
 This release is intended to validate the new release workflow.
@@ -151,6 +187,8 @@ Mostly a bugfix release with a slightly improved trace plotting.
 
 - Initial release, not yet feature complete
 
+[0.8.2]: https://github.com/SeismicSource/requake/compare/v0.8.1...v0.8.2
+[0.8.1]: https://github.com/SeismicSource/requake/compare/v0.8...v0.8.1
 [0.8]: https://github.com/SeismicSource/requake/compare/v0.7.3...v0.8
 [0.7.3]: https://github.com/SeismicSource/requake/compare/v0.7.2...v0.7.3
 [0.7.2]: https://github.com/SeismicSource/requake/compare/v0.7.1...v0.7.2

{requake-0.8 → requake-0.8.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: requake
-Version: 0.8
+Version: 0.8.2
 Summary: Repeating earthquakes search and analysis
 Author-email: Claudio Satriano <satriano@ipgp.fr>
 License-Expression: GPL-3.0-or-later
@@ -37,7 +37,7 @@ Dynamic: description
 Dynamic: description-content-type
 Dynamic: license-file
 
-<img src="https://cdn.jsdelivr.net/gh/SeismicSource/requake@e3ac4cce14f782d62a5f22f432a5391a2ccacfe7/imgs/Requake_logo.svg" width="400">
+<img src="https://cdn.jsdelivr.net/gh/SeismicSource/requake@9071fd7343802f3ef642a16f6bb02ff5ce1d88f5/imgs/Requake_logo.svg" width="400">
 
 # Requake
 
@@ -223,11 +223,14 @@ repeating earthquake families:
 
 ## Performances
 
-- `requake scan_catalog` took 53 minutes on my 2.7 GHz i7 MacBook Pro to
-process 14,100,705 earthquake pairs.
-Dowloaded traces are cached in memory to speed up execution. Processing is not
-yet parallel: some improvements might come in future versions, when
-parallelization will be implemented.
+- `requake scan_catalog` is designed to fully utilize available CPU cores by
+  processing earthquake pairs in parallel. On an M2 MacBook Air, scanning
+  95,034 earthquake pairs downloaded via FDSNWS took approximately 5 minutes
+  using 7 worker processes, yielding ~19,000 pairs per minute. When repeating
+  the same scan with a fully cached set of waveforms, runtime dropped to 84
+  seconds (~68,000 pairs per minute). These results indicate that overall
+  performance is typically dominated by waveform download latency rather than
+  computation.
 
 - `requake build_families` is fast™.
 

{requake-0.8 → requake-0.8.2}/README.md

@@ -184,11 +184,14 @@ repeating earthquake families:
 
 ## Performances
 
-- `requake scan_catalog` took 53 minutes on my 2.7 GHz i7 MacBook Pro to
-process 14,100,705 earthquake pairs.
-Dowloaded traces are cached in memory to speed up execution. Processing is not
-yet parallel: some improvements might come in future versions, when
-parallelization will be implemented.
+- `requake scan_catalog` is designed to fully utilize available CPU cores by
+  processing earthquake pairs in parallel. On an M2 MacBook Air, scanning
+  95,034 earthquake pairs downloaded via FDSNWS took approximately 5 minutes
+  using 7 worker processes, yielding ~19,000 pairs per minute. When repeating
+  the same scan with a fully cached set of waveforms, runtime dropped to 84
+  seconds (~68,000 pairs per minute). These results indicate that overall
+  performance is typically dominated by waveform download latency rather than
+  computation.
 
 - `requake build_families` is fast™.
 

{requake-0.8 → requake-0.8.2}/requake/__init__.py

@@ -9,5 +9,14 @@ Initialize requake package.
     GNU General Public License v3.0 or later
     (https://www.gnu.org/licenses/gpl-3.0-standalone.html)
 """
+import multiprocessing
+import signal
+
+# Child processes spawned for parallel scans may import requake before worker
+# initializers run. Ignore SIGINT early in children so Ctrl+C is managed by
+# the parent process only.
+if multiprocessing.current_process().name != 'MainProcess':
+    signal.signal(signal.SIGINT, signal.SIG_IGN)
+
 from . import _version
 __version__ = _version.get_versions()['version']

{requake-0.8 → requake-0.8.2}/requake/_version.py

@@ -8,11 +8,11 @@ import json
 
 version_json = '''
 {
- "date": "2026-05-22T08:53:09+0200",
+ "date": "2026-06-03T14:39:27+0200",
  "dirty": false,
  "error": null,
- "full-revisionid": "e3ac4cce14f782d62a5f22f432a5391a2ccacfe7",
- "version": "0.8"
+ "full-revisionid": "9071fd7343802f3ef642a16f6bb02ff5ce1d88f5",
+ "version": "0.8.2"
 }
 '''  # END VERSION_JSON
 

{requake-0.8 → requake-0.8.2}/requake/catalog/catalog.py

@@ -11,7 +11,6 @@ Classes and functions for downloading, reading and writing catalogs.
 """
 import numpy as np
 from obspy import UTCDateTime
-from ..config import config
 from ..formulas import float_or_none
 from ..waveforms import get_traceid_coords
 
@@ -230,7 +229,7 @@ def read_stored_catalog():
     """
     from ..database.catalog import read_catalog as read_catalog_from_db
 
-    cat = read_catalog_from_db(config)
+    cat = read_catalog_from_db()
     cat.sort()
     if not cat:
         raise ValueError('Empty catalog')
@@ -246,7 +245,7 @@ def fix_non_locatable_events(catalog):
     """
     if not any(ev.lat is None or ev.lon is None for ev in catalog):
         return
-    traceid_coords = get_traceid_coords(config)
+    traceid_coords = get_traceid_coords()
     mean_lat = np.mean([
         coords['latitude'] for coords in traceid_coords.values()])
     mean_lon = np.mean([

{requake-0.8 → requake-0.8.2}/requake/catalog/read_catalog.py

@@ -117,8 +117,8 @@ def read_catalog():
             catalog = read_stored_catalog()
             nevs_read = len(catalog)
             logger.info(
-                f'{nevs_read} events read from db file '
-                f'{get_db_path(config)}'
+                f'{nevs_read:n} events read from db file '
+                f'{get_db_path()}'
             )
     logger.info('Reading catalog...')
     input_cat_file = config.args.catalog_file
@@ -150,9 +150,9 @@ def read_catalog():
     # Sort catalog in increasing time order
     catalog.sort()
     # Write catalog to the output database
-    write_stored_catalog(catalog, config)
+    write_stored_catalog(catalog)
     nevs_written = len(catalog) - nevs_read
     logger.info(
-        f'{nevs_written} events written to db file '
-        f'{get_db_path(config)}'
+        f'{nevs_written:n} events written to db file '
+        f'{get_db_path()}'
     )

{requake-0.8 → requake-0.8.2}/requake/config/__init__.py

@@ -9,7 +9,7 @@ Configuration and initialization of the Requake package.
     GNU General Public License v3.0 or later
     (https://www.gnu.org/licenses/gpl-3.0-standalone.html)
 """
-from .rq_setup import rq_exit  # noqa
+from .rq_setup import rq_exit, wait_for_sigint_pause  # noqa
 from .generic_printer import generic_printer  # noqa
 # The config object is created in config.py and needs to be populated
 # when using requake from command line, this is done by the configure()

{requake-0.8 → requake-0.8.2}/requake/config/configspec.conf

@@ -68,6 +68,20 @@ catalog_search_range = float(default=30)
 ## If more than one trace_id is specified (separated by commas), the closest
 ## station will be used for a given event pair.
 catalog_trace_id = force_list(default=NET.STA.LOC.CHAN)
+## Maximum number of event waveforms kept in memory cache during
+## catalog-based scan. Higher values increase memory usage and can reduce
+## repeated waveform downloads.
+catalog_waveform_cache_size = integer(min=2, default=5000)
+## Per-worker waveform cache size for parallel catalog scan.
+## Set to 0 to derive it automatically from catalog_waveform_cache_size.
+catalog_waveform_cache_size_parallel = integer(min=0, default=0)
+## Keep a persistent copy of fetched event waveforms on disk to speed up
+## reruns. This on-disk cache lives in OUTDIR/waveform_cache and is not
+## limited by catalog_waveform_cache_size.
+catalog_waveform_disk_cache_enabled = boolean(default=true)
+## Number of worker processes for catalog scan.
+## 0 means automatic selection; 1 disables parallelism.
+catalog_scan_nprocs = integer(min=0, default=0)
 
 #### Template-based scan
 ### The following parameters are for template-based scan:

{requake-0.8 → requake-0.8.2}/requake/config/parse_arguments.py

@@ -49,6 +49,21 @@ class SubcommandHelpFormatter(argparse.RawDescriptionHelpFormatter):
         return parts
 
 
+def _nonnegative_int(value):
+    """Argparse type for integers >= 0."""
+    try:
+        int_value = int(value)
+    except ValueError as err:
+        raise argparse.ArgumentTypeError(
+            f'invalid non-negative integer value: {value}'
+        ) from err
+    if int_value < 0:
+        raise argparse.ArgumentTypeError(
+            f'invalid non-negative integer value: {value}'
+        )
+    return int_value
+
+
 def parse_arguments(progname='requake'):
     """
     Parse command line arguments.
@@ -160,11 +175,30 @@ def parse_arguments(progname='requake'):
     )
     # ---
     # --- scan_catalog
-    subparser.add_parser(
+    scan_catalog = subparser.add_parser(
         'scan_catalog',
-        parents=[force],
         help='scan an existing catalog for earthquake pairs'
     )
+    scan_catalog_mode = scan_catalog.add_mutually_exclusive_group()
+    scan_catalog_mode.add_argument(
+        '-f',
+        '--force',
+        action='store_true',
+        help='delete existing event pairs and restart the scan'
+    )
+    scan_catalog_mode.add_argument(
+        '-c',
+        '--force-continue',
+        action='store_true',
+        help='continue an interrupted scan and keep existing event pairs'
+    )
+    scan_catalog.add_argument(
+        '--nprocs',
+        type=_nonnegative_int,
+        default=None,
+        help='number of worker processes for scan_catalog '
+             '(0: auto, 1: disable parallelism)'
+    )
     # ---
     # --- print_pairs
     print_pairs = subparser.add_parser(
@@ -324,10 +358,15 @@ def parse_arguments(progname='requake'):
     print_families = subparser.add_parser(
         'print_families',
         parents=[
-            longerthan, shorterthan, minevents, family_numbers, print_format
+            longerthan, shorterthan, minevents, family_numbers
         ],
         help='print families to screen'
     )
+    print_families.add_argument(
+        '-f', '--format', type=str, default='simple',
+        choices=['simple', 'markdown', 'csv', 'stats'],
+        help='format for the output (default: %(default)s)'
+    )
     print_families.add_argument(
         '-d', '--detailed', action='store_true',
         help='print detailed information for each family, including a list '

{requake-0.8 → requake-0.8.2}/requake/config/rq_setup.py

@@ -10,11 +10,13 @@ Setup functions for Requake.
     (https://www.gnu.org/licenses/gpl-3.0-standalone.html)
 """
 import contextlib
+import multiprocessing
 import sys
 import os
 import shutil
 import logging
 import signal
+import time
 import tqdm
 from .._version import get_versions
 from ..database.db import get_db_path
@@ -31,6 +33,10 @@ PYTHON_VERSION_STR = None
 NUMPY_VERSION_STR = None
 SCIPY_VERSION_STR = None
 OBSPY_VERSION_STR = None
+SIGINT_CONFIRM_WINDOW_S = 2.0
+SIGINT_PAUSE_POLL_S = 0.05
+_LAST_SIGINT_TS = None
+_SIGINT_PAUSE_UNTIL_TS = None
 
 
 def _check_library_versions():
@@ -273,7 +279,7 @@ def configure(args):
     should_write_catalog = (
         args.action == 'read_catalog'
         and not args.append
-        and not write_ok(get_db_path(config), args.force)
+        and not write_ok(get_db_path(), args.force)
     )
     if should_write_catalog:
         print('Exiting now.')
@@ -330,8 +336,45 @@ def rq_exit(retval=0, abort=False, progname='requake'):
 
 
 def sigint_handler(_sig, _frame):
-    """Abort gracefully."""
-    rq_exit(1, abort=True)
+    """Ask for confirmation on first Ctrl+C and abort on second."""
+    global _LAST_SIGINT_TS  # pylint: disable=global-statement
+    global _SIGINT_PAUSE_UNTIL_TS  # pylint: disable=global-statement
 
+    if multiprocessing.current_process().name != 'MainProcess':
+        signal.default_int_handler(_sig, _frame)
+        return
+
+    now = time.monotonic()
+    if (
+        _LAST_SIGINT_TS is not None
+        and now - _LAST_SIGINT_TS <= SIGINT_CONFIRM_WINDOW_S
+    ):
+        rq_exit(1, abort=True)
+    _LAST_SIGINT_TS = now
+    _SIGINT_PAUSE_UNTIL_TS = now + SIGINT_CONFIRM_WINDOW_S
+    print(
+        '\nCtrl+C pressed. '
+        f'Press Ctrl+C again within {SIGINT_CONFIRM_WINDOW_S:g}s '
+        'to abort.'
+    )
 
-signal.signal(signal.SIGINT, sigint_handler)
+
+def wait_for_sigint_pause():
+    """Block while the first Ctrl+C confirmation window is active."""
+    global _SIGINT_PAUSE_UNTIL_TS  # pylint: disable=global-statement
+
+    if multiprocessing.current_process().name != 'MainProcess':
+        return
+    while _SIGINT_PAUSE_UNTIL_TS is not None:
+        if time.monotonic() >= _SIGINT_PAUSE_UNTIL_TS:
+            _SIGINT_PAUSE_UNTIL_TS = None
+            break
+        time.sleep(SIGINT_PAUSE_POLL_S)
+
+
+if multiprocessing.current_process().name == 'MainProcess':
+    signal.signal(signal.SIGINT, sigint_handler)
+else:
+    # Workers must ignore SIGINT from startup to avoid pool breakage during
+    # spawn, before the worker initializer runs.
+    signal.signal(signal.SIGINT, signal.SIG_IGN)

{requake-0.8 → requake-0.8.2}/requake/database/catalog.py

@@ -86,9 +86,9 @@ def _event_from_row(row):
     )
 
 
-def write_catalog(catalog, config, db_path=None):
+def write_catalog(catalog, db_path=None):
     """Write the stored catalog into SQLite."""
-    conn = get_db_connection(config, initdb=True, db_path=db_path)
+    conn = get_db_connection(initdb=True, db_path=db_path)
     try:
         cursor = conn.cursor()
         _ensure_catalog_table(cursor)
@@ -112,11 +112,11 @@ def write_catalog(catalog, config, db_path=None):
         conn.close()
 
 
-def read_catalog(config, db_path=None):
+def read_catalog(db_path=None):
     """Read the stored catalog from SQLite."""
     from ..catalog.catalog import RequakeCatalog
 
-    conn = get_db_connection(config, initdb=False, db_path=db_path)
+    conn = get_db_connection(initdb=False, db_path=db_path)
     try:
         rows = conn.execute(
             f'SELECT * FROM {CATALOG_TABLE} ORDER BY orig_time'
@@ -125,7 +125,7 @@ def read_catalog(config, db_path=None):
         if MISSING_CATALOG_TABLE in str(err):
             raise FileNotFoundError(
                 'Catalog not found in db file '
-                f'{get_db_path(config, db_path=db_path)}'
+                f'{get_db_path(db_path=db_path)}'
             ) from err
         raise
     finally:

{requake-0.8 → requake-0.8.2}/requake/database/db.py

@@ -14,6 +14,7 @@ import contextlib
 import random
 import sqlite3
 import time
+from ..config.config import config
 
 DB_VERSION = 1
 DB_FILENAME = 'requake.sqlite'
@@ -29,7 +30,11 @@ class DatabaseBusyError(RuntimeError):
     """Raised when retries for transient SQLite lock errors are exhausted."""
 
 
-def get_db_path(config, db_path=None):
+class DatabaseCorruptError(RuntimeError):
+    """Raised when the SQLite database file is corrupt or malformed."""
+
+
+def get_db_path(db_path=None):
     """Return the SQLite path."""
     if db_path is not None:
         return db_path
@@ -61,9 +66,9 @@ def get_db_path(config, db_path=None):
     return os.path.join(outdir, DB_FILENAME)
 
 
-def check_db_exists(config, initdb=False, db_path=None):
+def check_db_exists(initdb=False, db_path=None):
     """Validate the configured database path for read or init mode."""
-    db_path = get_db_path(config, db_path=db_path)
+    db_path = get_db_path(db_path=db_path)
     if initdb:
         _ensure_parent_dir(db_path)
         return db_path
@@ -138,15 +143,15 @@ def execute_with_retry(operation, operation_name):
             attempt += 1
 
 
-def initialize_database_if_needed(config, db_path=None):
+def initialize_database_if_needed(db_path=None):
     """Initialize database schema version and pragmas if missing."""
-    conn = get_db_connection(config, initdb=True, db_path=db_path)
+    conn = get_db_connection(initdb=True, db_path=db_path)
     conn.close()
 
 
-def get_db_connection(config, initdb=False, db_path=None):
+def get_db_connection(initdb=False, db_path=None):
     """Open a SQLite connection for the configured database file."""
-    db_path = check_db_exists(config, initdb=initdb, db_path=db_path)
+    db_path = check_db_exists(initdb=initdb, db_path=db_path)
     conn = sqlite3.connect(db_path)
     _configure_connection(conn)
     conn.row_factory = sqlite3.Row

{requake-0.8 → requake-0.8.2}/requake/database/families.py

@@ -64,9 +64,9 @@ def _family_row(family, event):
     )
 
 
-def write_families(families, config):
+def write_families(families):
     """Write catalog-scan families into SQLite."""
-    conn = get_db_connection(config, initdb=True)
+    conn = get_db_connection(initdb=True)
     try:
         cursor = conn.cursor()
         _ensure_families_table(cursor)
@@ -95,12 +95,12 @@ def write_families(families, config):
         conn.close()
 
 
-def read_families(config):
+def read_families():
     """Read catalog-scan families from SQLite."""
     from ..catalog import RequakeEvent
     from ..families.families import Family
 
-    conn = get_db_connection(config, initdb=False)
+    conn = get_db_connection(initdb=False)
     try:
         cursor = conn.cursor()
         try:
@@ -113,7 +113,7 @@ def read_families(config):
         except sqlite3.OperationalError as err:
             if MISSING_FAMILIES_TABLE in str(err):
                 raise FileNotFoundError(
-                    f'Families not found in db file {get_db_path(config)}'
+                    f'Families not found in db file {get_db_path()}'
                 ) from err
             raise
     finally:
@@ -147,9 +147,9 @@ def read_families(config):
     return families
 
 
-def update_family_valid(config, family_number, is_valid):
+def update_family_valid(family_number, is_valid):
     """Update the validity flag for all events in one family."""
-    conn = get_db_connection(config, initdb=False)
+    conn = get_db_connection(initdb=False)
     try:
         cursor = conn.cursor()
         try:
@@ -166,7 +166,7 @@ def update_family_valid(config, family_number, is_valid):
         except sqlite3.OperationalError as err:
             if MISSING_FAMILIES_TABLE in str(err):
                 raise FileNotFoundError(
-                    f'Families not found in db file {get_db_path(config)}'
+                    f'Families not found in db file {get_db_path()}'
                 ) from err
             raise
         conn.commit()