altar-enformer 0.3.0__tar.gz

altar_enformer-0.3.0/.gitignore

@@ -0,0 +1,33 @@
+*.pyc
+*~
+**/__pycache__/*
+*.swp
+.vscode/
+.idea/
+.DS_Store
+.env*
+.mypy_cache/
+.ruff_cache/
+.pytest_cache/
+.coverage
+htmlcov/
+coverage.xml
+pytest.xml
+.hypothesis/
+.python-version
+.venv
+.venv-*/
+*.egg-info/
+build/
+dist/
+site/
+*.sqlite3
+*.log
+
+# Generated variant indexes and datasets. The small canonical gene table is tracked.
+altar/altar/variants/data/ccres.dnatree
+altar/altar/variants/data/region_annotations.parquet
+altar/altar/variants/data/variants.pkl.gz
+altar/altar/variants/data/region_provenance.json
+altar/altar/variants/data/raw/
+

altar_enformer-0.3.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Riya Sinha
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

altar_enformer-0.3.0/PKG-INFO

@@ -0,0 +1,55 @@
+Metadata-Version: 2.5
+Name: altar-enformer
+Version: 0.3.0
+Summary: Enformer model binding for large-scale Altar variant scoring
+Project-URL: Documentation, https://kundajelab.github.io/altar/
+Author: Riya Sinha
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Classifier: Typing :: Typed
+Requires-Python: >=3.12
+Requires-Dist: altar<0.4,>=0.3.0
+Provides-Extra: prepare
+Requires-Dist: kagglehub<2,>=1; extra == 'prepare'
+Provides-Extra: test
+Requires-Dist: pytest-asyncio>=0.24; extra == 'test'
+Requires-Dist: pytest>=8; extra == 'test'
+Description-Content-Type: text/markdown
+
+# Altar Enformer binding
+
+`altar-enformer` is the lightweight control-plane binding for high-throughput Enformer variant scoring. It
+registers `ENFORMER` in Altar's canonical model-plugin group and has no TensorFlow dependency.
+
+One Altar model instance contains immutable SavedModel and target-metadata resources plus a fixed selection
+of Enformer's 5,313 human output tracks. A job is partitioned upstream into deterministic scoring batches;
+the binding emits one independently retryable container task per batch. The runtime loads the model once per
+task, microbatches reference/alternate sequence pairs, and writes:
+
+- one primary row per variant, containing the track with the largest absolute SAD; and
+- a `track_effects` detail row for every selected `(variant, track)` pair, preserving reference and alternate
+  activity, SAD, and SADR.
+
+SAD and SADR follow the definitions in the [Basenji variant scorer linked by DeepMind's Enformer
+release](https://github.com/calico/basenji/blob/738321c85f8925ae6ac318a6cd4901a42ea6bc3f/bin/basenji_sad.py):
+`SAD = sum(alt) - sum(ref)` and `SADR = log2(sum(alt) + 1) - log2(sum(ref) + 1)` across the model's 896 output
+bins. DeepMind's simpler [`EnformerScoreVariantsRaw`
+notebook](https://github.com/google-deepmind/deepmind-research/blob/master/enformer/enformer-usage.ipynb) divides SAD
+by the constant 896 by taking means; that does not change per-variant track ranking. `SAR` is a different Basenji
+statistic—the sum of per-bin log ratios—and is intentionally not emitted under the wrong name. Track selection,
+weight bytes, metadata bytes, runtime image, and hg38 reference identity are all durable inputs.
+
+Scoring semantics 2.0 selects the primary track by absolute SAD, the main published Enformer variant-effect
+score, and renames the former `sar`/`top_track_sar` fields to `sadr`/`top_track_sadr`. Existing primary scores
+must be regenerated. Existing complete `track_effects` rows are sufficient for a later storage-layer migration
+because their numeric `sar` values already contained SADR; Altar itself does not mutate deployed result stores.
+
+The corresponding TensorFlow 2.13 / Python 3.10 executor lives under `runtimes/enformer` with its own lockfile
+and immutable image lifecycle.

altar_enformer-0.3.0/README.md

@@ -0,0 +1,30 @@
+# Altar Enformer binding
+
+`altar-enformer` is the lightweight control-plane binding for high-throughput Enformer variant scoring. It
+registers `ENFORMER` in Altar's canonical model-plugin group and has no TensorFlow dependency.
+
+One Altar model instance contains immutable SavedModel and target-metadata resources plus a fixed selection
+of Enformer's 5,313 human output tracks. A job is partitioned upstream into deterministic scoring batches;
+the binding emits one independently retryable container task per batch. The runtime loads the model once per
+task, microbatches reference/alternate sequence pairs, and writes:
+
+- one primary row per variant, containing the track with the largest absolute SAD; and
+- a `track_effects` detail row for every selected `(variant, track)` pair, preserving reference and alternate
+  activity, SAD, and SADR.
+
+SAD and SADR follow the definitions in the [Basenji variant scorer linked by DeepMind's Enformer
+release](https://github.com/calico/basenji/blob/738321c85f8925ae6ac318a6cd4901a42ea6bc3f/bin/basenji_sad.py):
+`SAD = sum(alt) - sum(ref)` and `SADR = log2(sum(alt) + 1) - log2(sum(ref) + 1)` across the model's 896 output
+bins. DeepMind's simpler [`EnformerScoreVariantsRaw`
+notebook](https://github.com/google-deepmind/deepmind-research/blob/master/enformer/enformer-usage.ipynb) divides SAD
+by the constant 896 by taking means; that does not change per-variant track ranking. `SAR` is a different Basenji
+statistic—the sum of per-bin log ratios—and is intentionally not emitted under the wrong name. Track selection,
+weight bytes, metadata bytes, runtime image, and hg38 reference identity are all durable inputs.
+
+Scoring semantics 2.0 selects the primary track by absolute SAD, the main published Enformer variant-effect
+score, and renames the former `sar`/`top_track_sar` fields to `sadr`/`top_track_sadr`. Existing primary scores
+must be regenerated. Existing complete `track_effects` rows are sufficient for a later storage-layer migration
+because their numeric `sar` values already contained SADR; Altar itself does not mutate deployed result stores.
+
+The corresponding TensorFlow 2.13 / Python 3.10 executor lives under `runtimes/enformer` with its own lockfile
+and immutable image lifecycle.

altar_enformer-0.3.0/altar_enformer/__init__.py

@@ -0,0 +1,29 @@
+# Copyright 2026 Riya Sinha
+"""Enformer model binding for Altar."""
+
+from altar_enformer.plugin import (
+    DEFAULT_IMAGE,
+    HUMAN_TRACK_COUNT,
+    KIND_SCORING,
+    OUTPUT_BIN_SIZE,
+    OUTPUT_BINS,
+    SEQUENCE_LENGTH,
+    EnformerConfiguration,
+    EnformerPlugin,
+    EnformerRunInputs,
+    EnformerShardKind,
+)
+
+
+__all__ = [
+    "DEFAULT_IMAGE",
+    "HUMAN_TRACK_COUNT",
+    "KIND_SCORING",
+    "OUTPUT_BINS",
+    "OUTPUT_BIN_SIZE",
+    "SEQUENCE_LENGTH",
+    "EnformerConfiguration",
+    "EnformerPlugin",
+    "EnformerRunInputs",
+    "EnformerShardKind",
+]

altar_enformer-0.3.0/altar_enformer/plugin.py

@@ -0,0 +1,263 @@
+# Copyright 2026 Riya Sinha
+"""Backend-neutral Enformer binding for large-scale variant scoring.
+
+The binding emits one independent container task for the request's deterministic scoring batch. The runtime
+loads a content-addressed Enformer SavedModel once, microbatches reference/alternate sequence pairs, writes a
+compact primary row per variant, and preserves every selected track's official SAD/SADR effect as named detail.
+"""
+
+from __future__ import annotations
+import enum
+from typing import TYPE_CHECKING, Any, cast
+
+from altar.execution import GpuPool, ResourceRequest
+from altar.models import (
+    ApiVersionRange,
+    ContainerImageDigest,
+    ContainerScoringPlan,
+    DetailSchema,
+    ModelConfiguration,
+    ModelPlugin,
+    OutputSelection,
+    PluginCapability,
+    PluginKind,
+    PluginManifest,
+    PrioritizationPolicy,
+    ReferenceGenomeRunInputs,
+    ResourceReference,
+    ResultField,
+    ResultSchema,
+    RuntimeIdentityPolicy,
+    RuntimeKind,
+    RuntimeRequirement,
+    ScoringRequest,
+    VariantClass,
+    VariantEligibility,
+    require_resource_digests,
+    require_strictly_increasing,
+    resource_transfer,
+    single_container_plan,
+)
+from altar.predicates import Abs, Col, Ge, Lit, Predicate
+from pydantic import Field, model_validator
+
+from altar_enformer.resources import manifest_resources
+
+
+if TYPE_CHECKING:
+    import os
+
+
+SEQUENCE_LENGTH = 393_216
+OUTPUT_BINS = 896
+OUTPUT_BIN_SIZE = 128
+HUMAN_TRACK_COUNT = 5_313
+DEFAULT_IMAGE = "riyavsinha/altar-enformer@sha256:52aafa8c8d781763559814c4ab956ea4094fadb556e6c4903097dc3fab500fe5"
+
+
+class EnformerShardKind(enum.StrEnum):
+    """Stable execution kind emitted by :class:`EnformerPlugin`."""
+
+    SCORING = "enformer_scoring"
+
+
+KIND_SCORING = EnformerShardKind.SCORING
+_SCORE_RESOURCES = ResourceRequest(gpu=GpuPool.NORMAL_GPU, count=1, memory_gb=16)
+_SAD_PRIORITIZE_THRESHOLD = 0.5
+
+
+class EnformerConfiguration(ModelConfiguration):
+    """Immutable Enformer resources and selected human output tracks.
+
+    ``weights`` points to a tar archive containing one TensorFlow SavedModel. ``track_metadata`` is the
+    official ``targets_human.txt`` table whose ``index`` column maps output channels to stable identifiers.
+    Track indices are canonicalized by requiring strictly increasing values; equivalent selections therefore
+    cannot acquire different cache identities solely from ordering.
+    """
+
+    weights: ResourceReference
+    track_metadata: ResourceReference
+    track_indices: tuple[int, ...] = Field(min_length=1)
+    image: ContainerImageDigest = DEFAULT_IMAGE
+
+    @model_validator(mode="after")
+    def validate_resources_and_tracks(self) -> EnformerConfiguration:
+        require_resource_digests(self, owner="Enformer")
+        require_strictly_increasing(
+            self.track_indices, owner="Enformer", field="track_indices", minimum=0, maximum=HUMAN_TRACK_COUNT - 1
+        )
+        return self
+
+    @classmethod
+    def from_resource_manifest(
+        cls, path: str | os.PathLike[str], *, base_uri: str | None = None, **settings: Any
+    ) -> EnformerConfiguration:
+        """Build a configuration from the ``enformer.resources.json`` that ``altar-enformer-resources`` wrote.
+
+        ``settings`` are the remaining configuration fields: ``track_indices`` and ``image``. See
+        :func:`altar_enformer.resources.manifest_resources` for how ``base_uri`` locates the files.
+        """
+        resources = manifest_resources(path, base_uri=base_uri)
+        return cls(weights=resources.weights, track_metadata=resources.track_metadata, **settings)
+
+
+class EnformerRunInputs(ReferenceGenomeRunInputs):
+    """Per-run Enformer inputs, currently restricted to its training assembly."""
+
+    supported_genome_builds = frozenset({"hg38"})
+
+
+class EnformerPlugin(ModelPlugin):
+    """Predicts how a variant changes the chromatin, CAGE and TF tracks you select from Enformer's 5,313 (SAD)."""
+
+    model_type = "ENFORMER"
+    manifest = PluginManifest(
+        plugin_id="org.kundajelab.altar.model.enformer",
+        kind=PluginKind.MODEL,
+        capabilities=frozenset({PluginCapability.SCORE, PluginCapability.MATERIALIZE_RESULTS}),
+        architecture="Enformer",
+        plugin_version="0.0.1",
+        altar_api=ApiVersionRange("0.1.0", "0.2.0"),
+        configuration_schema_version="0.0.1",
+        result_schema=ResultSchema(
+            version="0.0.1",
+            fields=(
+                ResultField("top_track_index", "int", "Top Track Index", "Output index with largest absolute SAD"),
+                ResultField("top_track_id", "str", "Top Track", "Identifier of the output with largest absolute SAD"),
+                ResultField(
+                    "top_track_sad",
+                    "float",
+                    "Top Track SAD",
+                    "Alternate minus reference summed activity for the track with largest absolute SAD",
+                ),
+                ResultField(
+                    "top_track_sadr",
+                    "float",
+                    "Top Track SADR",
+                    "Log2 alternate/reference summed-activity ratio for the track with largest absolute SAD",
+                ),
+                ResultField("n_tracks_scored", "int", "Tracks Scored", "Number of selected human output tracks"),
+            ),
+        ),
+        detail_schemas=(
+            DetailSchema(
+                name="track_effects",
+                version="0.0.1",
+                description="Official Enformer SAD and SADR effects for every selected human output track.",
+                fields=(
+                    ResultField("track_index", "int", "Track Index", "Zero-based Enformer human output index", False),
+                    ResultField("track_id", "str", "Track ID", "Identifier from targets_human.txt", False),
+                    ResultField(
+                        "ref_signal", "float", "Reference Signal", "Reference activity summed over 896 bins", False
+                    ),
+                    ResultField(
+                        "alt_signal", "float", "Alternate Signal", "Alternate activity summed over 896 bins", False
+                    ),
+                    ResultField("sad", "float", "SAD", "Alternate minus reference summed activity", False),
+                    ResultField(
+                        "sadr",
+                        "float",
+                        "SADR",
+                        "Log2 alternate/reference summed-activity ratio with pseudocount 1",
+                        False,
+                    ),
+                ),
+                key_fields=("track_index",),
+            ),
+        ),
+        # The Enformer runtime substitutes one reference base; batch preparation filters out indels and MNVs.
+        variant_eligibility=VariantEligibility(frozenset({VariantClass.SNV})),
+        prioritization=PrioritizationPolicy(name="enformer-sad-effect-v2", version="0.0.1"),
+        runtimes=(
+            RuntimeRequirement(
+                name="model_runtime",
+                kind=RuntimeKind.CONTAINER_IMAGE,
+                policy=RuntimeIdentityPolicy.DIGEST_REQUIRED,
+            ),
+        ),
+    )
+    configuration_schema_id = "org.kundajelab.altar.model.enformer.configuration"
+    configuration_model = EnformerConfiguration
+    run_input_model = EnformerRunInputs
+    default_output = OutputSelection(outputs=("scores", "track_effects"), reducer="official_sad_sadr_v2")
+
+    def scoring_runtime_identities(self, configuration: ModelConfiguration) -> dict[str, str]:
+        """Record the configured Enformer image as the scoring plan's one runtime."""
+        return {"model_runtime": self.parse_configuration_as(configuration, EnformerConfiguration).image}
+
+    def build_scoring_plan(self, request: ScoringRequest) -> ContainerScoringPlan:
+        """Build one independently retryable task for one deterministic variant batch."""
+        configuration = self.parse_configuration_as(request.configuration, EnformerConfiguration)
+        run_inputs = self.parse_run_inputs_as(request.run_inputs, EnformerRunInputs)
+
+        layout, resolver = request.layout, request.resolver
+        weights_logical = layout.model_resource_file(request.model_id, "enformer.savedmodel.tar.gz")
+        metadata_logical = layout.model_resource_file(request.model_id, "targets_human.txt")
+        genome_logical = layout.genome_fasta(request.genome_label)
+        variants_logical = layout.model_variants_file(request.job_id, request.model_id, request.scoring_batch)
+        primary_logical = layout.scoring_result_file(request.job_id, request.model_id, request.scoring_batch)
+        details_logical = layout.detail_result_file(
+            request.job_id,
+            request.model_id,
+            "track_effects",
+            request.scoring_batch,
+        )
+
+        command = [
+            "enformer-score",
+            "score",
+            "--weights",
+            resolver.resolve(weights_logical),
+            "--weights-digest",
+            cast("str", configuration.weights.digest),
+            "--track-metadata",
+            resolver.resolve(metadata_logical),
+            "--track-metadata-digest",
+            cast("str", configuration.track_metadata.digest),
+        ]
+        for track_index in configuration.track_indices:
+            command += ["--track-index", str(track_index)]
+        command += [
+            "--genome",
+            resolver.resolve(genome_logical),
+            "--genome-digest",
+            cast("str", run_inputs.genome.digest),
+            "--variants",
+            resolver.resolve(variants_logical),
+            "--primary-output",
+            resolver.resolve(primary_logical),
+            "--detail-output",
+            resolver.resolve(details_logical),
+        ]
+
+        return single_container_plan(
+            self,
+            request,
+            kind=KIND_SCORING,
+            image=configuration.image,
+            command=command,
+            resources=_SCORE_RESOURCES,
+            inputs=[
+                resource_transfer(configuration.weights, weights_logical, locality_key=request.model_id),
+                resource_transfer(configuration.track_metadata, metadata_logical, locality_key=request.model_id),
+                resource_transfer(run_inputs.genome, genome_logical),
+            ],
+        )
+
+    def prioritize_predicate(self) -> Predicate:
+        # Placeholder effect threshold, not a calibrated biological cutoff.
+        return Ge(Abs(Col("top_track_sad")), Lit(_SAD_PRIORITIZE_THRESHOLD))
+
+
+__all__ = [
+    "DEFAULT_IMAGE",
+    "HUMAN_TRACK_COUNT",
+    "KIND_SCORING",
+    "OUTPUT_BINS",
+    "OUTPUT_BIN_SIZE",
+    "SEQUENCE_LENGTH",
+    "EnformerConfiguration",
+    "EnformerPlugin",
+    "EnformerRunInputs",
+    "EnformerShardKind",
+]

altar_enformer-0.3.0/altar_enformer/py.typed

@@ -0,0 +1 @@
+