crossref-matcher 1.3.0__tar.gz → 1.3.2__tar.gz

{crossref_matcher-1.3.0/crossref_matcher.egg-info → crossref_matcher-1.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: crossref-matcher
-Version: 1.3.0
+Version: 1.3.2
 Summary: Crossref's matching API
 Home-page: https://gitlab.com/crossref/marple
 Author: Crossref

{crossref_matcher-1.3.0 → crossref_matcher-1.3.2}/README.md

@@ -185,3 +185,96 @@ Requirements are defined in `setup.py`. Requirements files with exact versions c
 * To update `requirements.txt` file: `pip-compile setup.py --extra strategies-core`
 
 * To update `requirements-tests.txt` file: `pip-compile setup.py --extra strategies-core --extra evaluation --extra test -o requirements-tests.txt`
+
+## Branching and deployment
+
+CI (see [.gitlab-ci.yml](.gitlab-ci.yml)) deploys from two long-lived branches:
+
+| Branch / trigger | Deploys to | Jobs |
+|---|---|---|
+| `dev` | **Research** environment | ECS deploy (`marple-research`), Airflow DAG data sync to `s3://airflow.crossref.org` (research-only bucket), `X.Y.Z-dev.N` prerelease tags |
+| `main` | **Staging** environment | ECS deploy (`marple-staging`), GitLab-registry image for the integration project, release tags via semantic-release |
+| Stable `vX.Y.Z` tags | PyPI | `crossref-matcher` published via OIDC trusted publishing (prerelease tags are not published) |
+
+Both branches are protected. `main` is merged to via MR only; `dev`'s
+protection must also allow Developers to push and merge directly
+(**Settings → Repository → Protected branches → `dev` → Allowed to push
+and merge**), since the workflow below pushes to it directly. Deploys are
+automatic on push and are keyed on the commit SHA (`$CI_COMMIT_SHORT_SHA`),
+independent of the semantic version.
+
+### Feature workflow
+
+1. **Branch off `main`** (not `dev`), so the feature can merge to `main`
+   cleanly regardless of what else is being tested on `dev`.
+2. **To test on the research environment**, merge the feature into `dev` and
+   push (no MR needed — review happens at the `main` MR):
+
+   ```sh
+   git checkout dev && git pull
+   git merge --no-ff my-feature
+   git push
+   ```
+
+3. **When validated, open an MR from the feature branch into `main`.**
+   Do not merge `dev` into `main` — `dev` may contain other unfinished work.
+   Use a [conventional commit](https://www.conventionalcommits.org/) message
+   (`feat:`/`fix:`) for the squash/merge commit; semantic-release derives the
+   release version from it.
+4. **After the MR merges, merge `main` back into `dev`** to keep the branches
+   converged. This isn't strictly required — semantic-release's prerelease
+   branches skip the "behind the release branch" check — but without it,
+   `dev`'s next prerelease can recompute a version `main` already released,
+   or a confusing `vX.Y.Z-dev.N` alongside an already-shipped `vX.Y.Z`:
+
+   ```sh
+   git checkout dev && git pull
+   git merge origin/main
+   git push
+   ```
+
+### Versioning
+
+Releases are cut by semantic-release from conventional commit messages
+(`fix:` → patch, `feat:` → minor, `BREAKING CHANGE` → major; other types cut
+no release). On `dev`, release-worthy commits produce prerelease tags like
+`v1.3.0-dev.1` (configured in [.releaserc.json](.releaserc.json)). Commits
+without a `feat:`/`fix:` still deploy — only the tag is skipped, and
+`VERSION.txt` keeps its `0.0.dev0` placeholder. To identify what is running
+in an environment, use the commit SHA (the ECS image tag), not the version
+string.
+
+### Resetting a polluted `dev` back to `main`
+
+`dev` is disposable: `main` never depends on its history. If `dev` has
+accumulated abandoned experiments, reset it to `main` and re-merge only the
+features still in flight:
+
+1. Temporarily enable force-push for `dev` in GitLab
+   (**Settings → Repository → Protected branches → `dev` → Allowed to force push**).
+2. Delete any `vX.Y.Z-dev.N` prerelease tags cut on the old `dev` — after the
+   reset their commits are unreachable from the branch, so semantic-release
+   won't see them and will try to recreate the same tag, failing every
+   subsequent release-worthy `dev` push until they're gone:
+
+   ```sh
+   git tag -l 'v*-dev.*' | xargs -r git push origin --delete
+   git tag -l 'v*-dev.*' | xargs -r git tag -d
+   ```
+
+3. Reset and push:
+
+   ```sh
+   git fetch origin
+   git checkout dev
+   git reset --hard origin/main
+   git push --force-with-lease origin dev
+   git merge --no-ff still-in-flight-feature   # re-merge anything still being tested
+   git push
+   ```
+
+4. Disable force-push again.
+
+The reset push re-deploys research and re-syncs the Airflow DAG data in
+`s3://airflow.crossref.org` to the reset state (the sync uses `--delete`, so
+files added by abandoned experiments are removed, not just left stale).

{crossref_matcher-1.3.0 → crossref_matcher-1.3.2}/crossref_matcher/evaluation/evaluation.py

@@ -176,15 +176,29 @@ class EvaluateSklearn:
         self.mlb.fit(y_true + y_pred)
         y_true_transformed = self.mlb.transform(y_true)
         y_pred_transformed = self.mlb.transform(y_pred)
-        at = y_true_transformed.toarray()
-        af = y_pred_transformed.toarray()
-        false_positives = int((af & (1 - at)).sum())
-        false_negatives = int((at & (1 - af)).sum())
+        # Stay in sparse representation: with large label spaces (e.g. grant
+        # IDs), densifying via .toarray() blows up memory/time since the
+        # dense matrix is n_samples x n_unique_labels.
+        true_positives = y_true_transformed.multiply(y_pred_transformed).sum()
+        false_positives = int(y_pred_transformed.sum() - true_positives)
+        false_negatives = int(y_true_transformed.sum() - true_positives)
+        # sklearn's type_of_target() requires >1 column to classify a target
+        # as "multilabel-indicator" (native sparse support); with exactly one
+        # label column it's classified as "binary" instead, which sklearn
+        # handles via column_or_1d(), and that rejects sparse input outright.
+        # Densify just for that narrow edge case (cheap, since there's only
+        # one column).
+        if y_true_transformed.shape[1] <= 1:
+            y_true_for_fscore = y_true_transformed.toarray()
+            y_pred_for_fscore = y_pred_transformed.toarray()
+        else:
+            y_true_for_fscore = y_true_transformed
+            y_pred_for_fscore = y_pred_transformed
         f_scores = []
         for beta in f_score_betas:
             precision, recall, fscore, _ = precision_recall_fscore_support(
-                at,
-                af,
+                y_true_for_fscore,
+                y_pred_for_fscore,
                 average="micro",
                 zero_division=0.0,
                 sample_weight=sample_weights,
@@ -352,7 +366,7 @@ def run_strategy_on_eval_data(
             extra_data = extra_fn(strategy, data_point, this_result)
             out_item["extra"].update(extra_data)
 
-        if hasattr(data_point, "alternates"):
+        if hasattr(data_point, "alternates") and len(data_point.alternates) > 0:
             dataset_has_alternates = True
             out_item["alternates"] = data_point.alternates
         this_weight = getattr(data_point, "weight", 1.0)

crossref_matcher-1.3.2/crossref_matcher/evaluation/sparse_metrics_verification/verify_sparse_metrics.py

@@ -0,0 +1,239 @@
+"""Reproduces the investigation behind the sparse-matrix rewrite of
+`EvaluateSklearn.run_one_evaluation` in `crossref_matcher/evaluation/evaluation.py`.
+
+Run directly to reproduce all findings:
+
+    python -m crossref_matcher.evaluation.sparse_metrics_verification.verify_sparse_metrics
+
+See README.md in this directory for the narrative writeup of what each
+check demonstrates.
+"""
+
+import random
+import time
+import tracemalloc
+
+import numpy as np
+from sklearn.metrics import precision_recall_fscore_support
+from sklearn.preprocessing import MultiLabelBinarizer
+from sklearn.utils.multiclass import type_of_target
+
+
+def old_dense_false_positives_negatives(at_sparse, af_sparse):
+    """The original implementation: densify, then bitwise AND."""
+    at = at_sparse.toarray()
+    af = af_sparse.toarray()
+    false_positives = int((af & (1 - at)).sum())
+    false_negatives = int((at & (1 - af)).sum())
+    return false_positives, false_negatives
+
+
+def new_sparse_false_positives_negatives(at_sparse, af_sparse):
+    """The replacement implementation: stay sparse throughout."""
+    true_positives = at_sparse.multiply(af_sparse).sum()
+    false_positives = int(af_sparse.sum() - true_positives)
+    false_negatives = int(at_sparse.sum() - true_positives)
+    return false_positives, false_negatives
+
+
+def fscore_inputs(at_sparse, af_sparse):
+    """Mirrors the shape[1] <= 1 densify branch added to run_one_evaluation."""
+    if at_sparse.shape[1] <= 1:
+        return at_sparse.toarray(), af_sparse.toarray()
+    return at_sparse, af_sparse
+
+
+def check_equivalence(trials: int = 500, seed: int = 42) -> int:
+    """Fuzz-tests old (dense bitwise) vs. new (sparse) false_positives /
+    false_negatives / precision / recall / f-score against each other over
+    randomized multi-label data, including empty label sets and duplicate
+    labels within a single sample.
+
+    Returns the number of mismatches found (0 means full equivalence).
+    """
+    rng = random.Random(seed)
+    mismatches = 0
+    checked = 0
+
+    for trial in range(trials):
+        n_samples = rng.randint(1, 8)
+        vocab = [f"id{i}" for i in range(rng.randint(1, 6))]
+        y_true, y_pred = [], []
+        for _ in range(n_samples):
+            yt = rng.sample(vocab, k=rng.randint(0, len(vocab)))
+            if yt and rng.random() < 0.3:
+                yt = yt + [yt[0]]  # duplicate label within a sample
+            yp = rng.sample(vocab, k=rng.randint(0, len(vocab)))
+            y_true.append(yt)
+            y_pred.append(yp)
+
+        if not any(y_true) and not any(y_pred):
+            continue  # matches the early-return "all empty" branch in evaluation.py
+
+        mlb = MultiLabelBinarizer(sparse_output=True)
+        mlb.fit(y_true + y_pred)
+        at_sparse = mlb.transform(y_true)
+        af_sparse = mlb.transform(y_pred)
+
+        checked += 1
+
+        fp_old, fn_old = old_dense_false_positives_negatives(at_sparse, af_sparse)
+        fp_new, fn_new = new_sparse_false_positives_negatives(at_sparse, af_sparse)
+        if (fp_old, fn_old) != (fp_new, fn_new):
+            mismatches += 1
+            print(f"  MISMATCH (fp/fn) trial={trial} y_true={y_true} y_pred={y_pred}")
+            print(f"    old: fp={fp_old} fn={fn_old}  new: fp={fp_new} fn={fn_new}")
+
+        at_dense, af_dense = at_sparse.toarray(), af_sparse.toarray()
+        p_old, r_old, f_old, _ = precision_recall_fscore_support(
+            at_dense, af_dense, average="micro", zero_division=0.0, beta=1.0
+        )
+        yt_new, yp_new = fscore_inputs(at_sparse, af_sparse)
+        p_new, r_new, f_new, _ = precision_recall_fscore_support(
+            yt_new, yp_new, average="micro", zero_division=0.0, beta=1.0
+        )
+        if not np.allclose([p_old, r_old, f_old], [p_new, r_new, f_new]):
+            mismatches += 1
+            print(f"  MISMATCH (fscore) trial={trial} y_true={y_true} y_pred={y_pred}")
+            print(f"    old: p={p_old} r={r_old} f={f_old}")
+            print(f"    new: p={p_new} r={r_new} f={f_new}")
+
+    print(
+        f"[equivalence] checked {checked} non-trivial trials, {mismatches} mismatches"
+    )
+    return mismatches
+
+
+def check_single_column_edge_case() -> None:
+    """Demonstrates why a densify fallback is needed when there is only one
+    unique label across a batch: sklearn's type_of_target() requires more
+    than one column to classify a target as "multilabel-indicator" (which
+    has native sparse support). With exactly one column it's classified as
+    "binary" instead, and sklearn's binary-handling path (column_or_1d)
+    rejects sparse input outright -- regardless of dense vs sparse, the
+    *classification* is the same; only the *acceptance* of sparse input
+    differs downstream.
+    """
+    mlb = MultiLabelBinarizer(sparse_output=True)
+    y_true = [["a"], []]
+    y_pred = [["a"], ["a"]]
+    mlb.fit(y_true + y_pred)
+    at_sparse = mlb.transform(y_true)
+    af_sparse = mlb.transform(y_pred)
+    at_dense, af_dense = at_sparse.toarray(), af_sparse.toarray()
+
+    print(f"[single-column] shape={at_sparse.shape}")
+    print(f"  type_of_target(sparse) = {type_of_target(at_sparse)!r}")
+    print(f"  type_of_target(dense)  = {type_of_target(at_dense)!r}")
+
+    try:
+        precision_recall_fscore_support(
+            at_sparse, af_sparse, average="micro", zero_division=0.0, beta=1.0
+        )
+        print("  sparse input: unexpectedly succeeded")
+    except TypeError as e:
+        print(f"  sparse input raises TypeError: {e}")
+
+    p, r, f, _ = precision_recall_fscore_support(
+        at_dense, af_dense, average="micro", zero_division=0.0, beta=1.0
+    )
+    print(f"  dense input succeeds: precision={p} recall={r} fscore={f}")
+
+
+def check_sample_weight_equivalence() -> None:
+    """Confirms sample_weight produces identical results for sparse vs. dense
+    once the label space has more than one column (the common case)."""
+    mlb = MultiLabelBinarizer(sparse_output=True)
+    y_true = [["a", "b"], ["c"]]
+    y_pred = [["a"], ["c", "d"]]
+    mlb.fit(y_true + y_pred)
+    at_sparse = mlb.transform(y_true)
+    af_sparse = mlb.transform(y_pred)
+    weights = [2.0, 3.0]
+
+    p1, r1, f1, _ = precision_recall_fscore_support(
+        at_sparse,
+        af_sparse,
+        average="micro",
+        zero_division=0.0,
+        beta=1.0,
+        sample_weight=weights,
+    )
+    p2, r2, f2, _ = precision_recall_fscore_support(
+        at_sparse.toarray(),
+        af_sparse.toarray(),
+        average="micro",
+        zero_division=0.0,
+        beta=1.0,
+        sample_weight=weights,
+    )
+    print(f"[sample_weight] sparse: precision={p1} recall={r1} fscore={f1}")
+    print(f"[sample_weight] dense:  precision={p2} recall={r2} fscore={f2}")
+    assert np.allclose([p1, r1, f1], [p2, r2, f2])
+
+
+def benchmark_dense_vs_sparse(n_samples: int = 2000, n_labels: int = 20000) -> None:
+    """Demonstrates the actual cost of `.toarray()` that motivated this fix:
+    a label-indicator matrix scales as n_samples x n_unique_labels, which for
+    a large ID space (e.g. grant DOIs) becomes very large very fast, even
+    though the underlying data is highly sparse (each sample only has a
+    handful of true/predicted labels)."""
+    rng = random.Random(7)
+    vocab = [f"id{i}" for i in range(n_labels)]
+    y_true = [rng.sample(vocab, k=rng.randint(0, 3)) for _ in range(n_samples)]
+    y_pred = [rng.sample(vocab, k=rng.randint(0, 3)) for _ in range(n_samples)]
+
+    mlb = MultiLabelBinarizer(sparse_output=True)
+    mlb.fit(y_true + y_pred)
+    at_sparse = mlb.transform(y_true)
+    af_sparse = mlb.transform(y_pred)
+    print(
+        f"[benchmark] n_samples={n_samples} n_labels={n_labels} "
+        f"matrix shape={at_sparse.shape}"
+    )
+
+    tracemalloc.start()
+    start = time.perf_counter()
+    old_dense_false_positives_negatives(at_sparse, af_sparse)
+    dense_time = time.perf_counter() - start
+    _, dense_peak = tracemalloc.get_traced_memory()
+    tracemalloc.stop()
+
+    tracemalloc.start()
+    start = time.perf_counter()
+    new_sparse_false_positives_negatives(at_sparse, af_sparse)
+    sparse_time = time.perf_counter() - start
+    _, sparse_peak = tracemalloc.get_traced_memory()
+    tracemalloc.stop()
+
+    print(
+        f"[benchmark] dense (.toarray) : {dense_time:.3f}s, "
+        f"peak extra memory {dense_peak / 1e6:.1f} MB"
+    )
+    print(
+        f"[benchmark] sparse (.multiply): {sparse_time:.3f}s, "
+        f"peak extra memory {sparse_peak / 1e6:.1f} MB"
+    )
+
+
+def main() -> None:
+    print(
+        "== 1. Old vs. new false_positives/false_negatives/precision/recall/f-score equivalence =="
+    )
+    mismatches = check_equivalence()
+
+    print("\n== 2. Single-label-column edge case (why the densify fallback exists) ==")
+    check_single_column_edge_case()
+
+    print("\n== 3. sample_weight equivalence (sparse vs. dense) ==")
+    check_sample_weight_equivalence()
+
+    print("\n== 4. Benchmark: dense .toarray() vs. sparse, large label space ==")
+    benchmark_dense_vs_sparse()
+
+    print("\n== Summary ==")
+    print(f"Equivalence mismatches: {mismatches} (expect 0)")
+
+
+if __name__ == "__main__":
+    main()

crossref_matcher-1.3.2/crossref_matcher/indexes/ror_organizations/opensearch_utils.py

@@ -0,0 +1,44 @@
+"""Shared OpenSearch client utilities for ROR index DAG tasks."""
+
+import base64
+import boto3
+
+REGION = "us-east-1"
+INDEX = "organization"
+
+
+def get_secret(secret_name: str) -> str | bytes:
+    """Retrieve a secret value from AWS Secrets Manager."""
+    session = boto3.session.Session()
+    client = session.client(service_name="secretsmanager", region_name=REGION)
+
+    get_secret_value_response = client.get_secret_value(SecretId=secret_name)
+    if "SecretString" in get_secret_value_response:
+        return get_secret_value_response["SecretString"]
+    else:
+        return base64.b64decode(get_secret_value_response["SecretBinary"])
+
+
+def get_es_client(host: str | None = None):
+    """Create an authenticated OpenSearch client using AWS credentials."""
+    from opensearchpy import (
+        OpenSearch,
+        RequestsHttpConnection,
+        AWSV4SignerAuth,
+    )
+
+    if host is None:
+        host = get_secret("es-host-matching")
+
+    credentials = boto3.Session().get_credentials()
+    auth = AWSV4SignerAuth(credentials, REGION, "es")
+
+    return OpenSearch(
+        hosts=[{"host": host, "port": 443}],
+        http_auth=auth,
+        use_ssl=True,
+        verify_certs=True,
+        connection_class=RequestsHttpConnection,
+        pool_maxsize=20,
+        timeout=120,
+    )

crossref_matcher-1.3.2/crossref_matcher/resources/VERSION.txt

@@ -0,0 +1 @@
+1.3.2

{crossref_matcher-1.3.0 → crossref_matcher-1.3.2/crossref_matcher.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: crossref-matcher
-Version: 1.3.0
+Version: 1.3.2
 Summary: Crossref's matching API
 Home-page: https://gitlab.com/crossref/marple
 Author: Crossref

{crossref_matcher-1.3.0 → crossref_matcher-1.3.2}/crossref_matcher.egg-info/SOURCES.txt

@@ -20,13 +20,15 @@ crossref_matcher/evaluation/evaluation.py
 crossref_matcher/evaluation/google_sheets_connector.py
 crossref_matcher/evaluation/ror_utils.py
 crossref_matcher/evaluation/schemas.py
+crossref_matcher/evaluation/sparse_metrics_verification/__init__.py
+crossref_matcher/evaluation/sparse_metrics_verification/verify_sparse_metrics.py
 crossref_matcher/indexes/__init__.py
 crossref_matcher/indexes/preprint_matching/__init__.py
 crossref_matcher/indexes/preprint_matching/indexer_service.py
 crossref_matcher/indexes/ror_organizations/__init__.py
 crossref_matcher/indexes/ror_organizations/create_index.py
 crossref_matcher/indexes/ror_organizations/index_data.py
-crossref_matcher/indexes/ror_organizations/ror_opensearch_update_dag.py
+crossref_matcher/indexes/ror_organizations/opensearch_utils.py
 crossref_matcher/indexes/ror_organizations/ror_zenodo_api.py
 crossref_matcher/matching/__init__.py
 crossref_matcher/matching/http_utils.py

crossref_matcher-1.3.0/crossref_matcher/indexes/ror_organizations/ror_opensearch_update_dag.py

@@ -1,124 +0,0 @@
-"""Airflow DAG to keep the ROR OpenSearch index up to date."""
-
-import base64
-import boto3
-
-from airflow.decorators import dag, task
-from datetime import datetime
-from datetime import timedelta
-from opensearchpy import (
-    OpenSearch,
-    RequestsHttpConnection,
-    AWSV4SignerAuth,
-    NotFoundError,
-)
-
-import logging
-from typing import Any
-
-from crossref_matcher.indexes.ror_organizations.ror_zenodo_api import (
-    get_ror_data_dump_metadata,
-)
-from crossref_matcher.indexes.ror_organizations.create_index import (
-    index_body,
-    initialize_index,
-)
-from crossref_matcher.indexes.ror_organizations.index_data import (
-    download_ror_data_and_update_index,
-)
-
-logger = logging.getLogger(__name__)
-
-REGION = "us-east-1"
-INDEX = "organization"
-INDEX_BATCH_SIZE = 400
-
-
-def get_secret(secret_name: str) -> str | bytes:
-    """Retrieve a secret value from AWS Secrets Manager."""
-    session = boto3.session.Session()
-    client = session.client(service_name="secretsmanager", region_name=REGION)
-
-    get_secret_value_response = client.get_secret_value(SecretId=secret_name)
-    if "SecretString" in get_secret_value_response:
-        return get_secret_value_response["SecretString"]
-    else:
-        return base64.b64decode(get_secret_value_response["SecretBinary"])
-
-
-def get_es_client() -> OpenSearch:
-    """Create an authenticated OpenSearch client using AWS credentials."""
-    credentials = boto3.Session().get_credentials()
-    auth = AWSV4SignerAuth(credentials, REGION, "es")
-
-    return OpenSearch(
-        hosts=[{"host": get_secret("es-host-matching"), "port": 443}],
-        http_auth=auth,
-        use_ssl=True,
-        verify_certs=True,
-        connection_class=RequestsHttpConnection,
-        pool_maxsize=20,
-    )
-
-
-@dag(
-    schedule_interval="@daily",
-    catchup=False,
-    dagrun_timeout=timedelta(hours=10),
-    start_date=datetime(2026, 1, 3),
-    tags=["experiment", "ror", "matching"],
-)
-def update_ror_index():
-    @task()
-    def fetch_ror_metadata() -> dict[str, Any]:
-        """Fetch metadata for the latest ROR data dump from Zenodo."""
-        return get_ror_data_dump_metadata()
-
-    @task.short_circuit()
-    def is_update_needed(metadata: dict[str, Any]) -> bool:
-        """Return True if the index checksum differs from the latest dump."""
-        ror_checksum = metadata["checksum"]
-        es_client = get_es_client()
-
-        # # FOR TESTING: RUN THIS TO FORCE UPDATE, OTHERWISE COMMENT OUT
-        # index_metadata = {
-        #     "target_data": "",
-        #     "ror_data_dump_checksum": "",
-        #     "ror_data_dump_doi": "",
-        # }
-        # body = {"_meta": index_metadata}
-        # es_client.indices.put_mapping(index=INDEX, body=body)
-
-        try:
-            mapping = es_client.indices.get_mapping(index=INDEX)
-        except NotFoundError:
-            return True
-        meta = mapping[INDEX]["mappings"].get("_meta", {})
-        current_checksum = meta.get("ror_data_dump_checksum", "")
-        return current_checksum != ror_checksum
-
-    @task()
-    def create_index_if_not_exists():
-        es_client = get_es_client()
-
-        index_exists = es_client.indices.exists(index=INDEX)
-
-        if not index_exists:
-            index_body_content = index_body()
-            response = initialize_index(INDEX, es_client, index_body_content)
-            logger.info(f"Created index '{INDEX}': {response}")
-
-    @task
-    def download_and_update(metadata: dict[str, Any]) -> None:
-        """Download the ROR data dump and update the index."""
-        es_client = get_es_client()
-        download_ror_data_and_update_index(metadata, es_client)
-
-    metadata = fetch_ror_metadata()
-    update_needed = is_update_needed(metadata)
-    # if update_needed is false, the DAG will stop here
-
-    update_needed >> create_index_if_not_exists() >> download_and_update(metadata)
-
-
-workflow = update_ror_index()

crossref_matcher-1.3.0/crossref_matcher/resources/VERSION.txt

@@ -1 +0,0 @@
-1.3.0