arc-pie 0.1.6__py3-none-any.whl

arc_pie-0.1.6.dist-info/METADATA

@@ -0,0 +1,174 @@
+Metadata-Version: 2.4
+Name: arc-pie
+Version: 0.1.6
+Summary: PIE: predicting transcriptional responses to perturbations from biological knowledge sources
+Project-URL: Homepage, https://github.com/ArcInstitute/pie
+Project-URL: Repository, https://github.com/ArcInstitute/pie
+Project-URL: Issues, https://github.com/ArcInstitute/pie/issues
+Project-URL: Data and models, https://huggingface.co/collections/arcinstitute/pie
+Author-email: Rishi Verma <rishi.verma@arcinstitute.org>
+License-Expression: CC-BY-NC-SA-4.0
+License-File: LICENSE.md
+Keywords: biology,gene-expression,perceiver,perturbation,single-cell
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: <3.13,>=3.12
+Requires-Dist: anndata<0.14,>=0.13.2
+Requires-Dist: filelock<5,>=4.0.6
+Requires-Dist: h5py<4,>=3.16.0
+Requires-Dist: huggingface-hub<2,>=1.21.0
+Requires-Dist: hydra-core<1.4,>=1.3.3
+Requires-Dist: lightning<2.7,>=2.6.5
+Requires-Dist: numpy<3,>=2.4.6
+Requires-Dist: omegaconf<2.4,>=2.3.1
+Requires-Dist: pandas<4,>=3.0.3
+Requires-Dist: pyarrow<25,>=24.0.0
+Requires-Dist: pydantic<3,>=2.13.4
+Requires-Dist: safetensors<0.9,>=0.8.0
+Requires-Dist: scipy<2,>=1.18.0
+Requires-Dist: torch<2.11,>=2.10.0
+Requires-Dist: tqdm<5,>=4.68.3
+Requires-Dist: wandb<0.29,>=0.28.0
+Provides-Extra: process
+Requires-Dist: gpudge[fast]<0.10,>=0.9.1; extra == 'process'
+Provides-Extra: sources
+Requires-Dist: openai<3,>=2.44.0; extra == 'sources'
+Requires-Dist: rdkit<2027,>=2026.3.6; extra == 'sources'
+Requires-Dist: requests<3,>=2.34.2; extra == 'sources'
+Requires-Dist: tiktoken<0.14,>=0.13.0; extra == 'sources'
+Requires-Dist: transformers<6,>=5.12.1; extra == 'sources'
+Description-Content-Type: text/markdown
+
+# PIE: Generalizing perturbation effects across unseen perturbations, contexts and datasets
+
+PIE is a perturbation effect prediction model that generalizes to unseen biological context, perturbations and combinations of both. It is based on the Perceiver IO architecture (Jaegle et al., 2022), and leverages curated Knowledge Sources, mean of control cells on context query and perturbation response evidence from contexts and perturbations in the train set. 
+
+Preprint available at: [biorxiv](https://doi.org/10.64898/2026.10.02.756297)\
+Data and Knowledge Sources (🤗 Hugging Face): [PIE collection](https://huggingface.co/collections/arcinstitute/pie)
+
+## Installation
+
+```bash
+git clone https://github.com/ArcInstitute/pie.git && cd pie
+uv sync --frozen                    # add --extra sources / --extra process as needed
+```
+
+`uv sync --frozen` reproduces the paper environment exactly. Prefix every command with `uv run`
+(for example `uv run pie train ...`).
+
+### Settings
+
+PIE reads `WANDB_ENTITY`, `WANDB_PROJECT`, `PIE_DATA_ROOT`, `PIE_RUNS_ROOT` and `PIE_CACHE_DIR`
+from the environment, else from the first of `$PIE_ENV_FILE`, `./common.sh` and
+`~/.config/pie/common.sh`. Copy `common.sh.example` from the repository and fill it in.
+Set `logger.enabled=false` to train without wandb. Building text embeddings also needs
+`OPENAI_API_KEY`; export it in your own environment, never in `common.sh`.
+Run `pie` to list the commands and `pie <command> --help` for each one.
+
+## Published datasets and knowledge sources
+
+- **Ready to run:** the experiment configs pin full HF commits of every dataset, knowledge
+  source and split, and PIE downloads what a run needs. Rebuilding data is optional.
+- **On Hugging Face:** preprocessed datasets (with their `contexts.yaml`), `PIE_sources` (with
+  per-source `aliases.yaml`) and `PIE_splits` (train/val/test files), in the
+  [PIE collection](https://huggingface.co/collections/arcinstitute/pie).
+- **References:** `hf://datasets/<owner>/<repo>[@revision]/<directory>`; mix them freely with
+  local paths. Runs and checkpoints record the resolved commit.
+- **Your own splits:** `data.split_dir=<dir>` (local or `hf://`, with `train.json` and
+  `val.json`) for `pie train`; `split_path=<file>` for `pie eval`. A split file maps
+  `<dataset>.<context>` to a list of perturbations.
+- **Cache:** downloads go to `$PIE_DATA_ROOT/hf/`; `HF_HUB_OFFLINE=1` reuses them without
+  network. Use `hf auth login` or `HF_TOKEN` if a repository needs a login.
+
+## Optional: build Replogle data and sources locally
+
+Install the source-building and dataset-processing tools:
+
+```bash
+uv sync --frozen --extra sources --extra process
+```
+
+Download the Replogle-Nadig count matrix (K562, RPE1, Jurkat, HepG2) from [PLACEHOLDER] into `$PIE_DATA_ROOT/replogle/counts/`.
+
+Normalize the counts and compute per-cell-line differential expression (uses a GPU for running `gpudge`):
+
+```bash
+uv run pie process dataset=replogle input="$PIE_DATA_ROOT/replogle/counts/replogle_4cl.h5ad" \
+  normalize.output_dir="$PIE_DATA_ROOT/replogle/expression" \
+  de.output_dir="$PIE_DATA_ROOT/replogle/de"
+```
+
+Build the preprocessed dataset. `contexts=` copies the context map (context to Cellosaurus
+accession) into the dir; `pie sources` needs it for `context_text`. The published map is in the
+dataset repo:
+
+```bash
+uv run hf download arcinstitute/PIE_replogle_nadig_essential preprocessed/contexts.yaml \
+  --repo-type dataset --local-dir .
+uv run pie prep dataset=replogle label_format=pie_process \
+  labels="$PIE_DATA_ROOT/replogle/de/*.parquet" \
+  h5ad="$PIE_DATA_ROOT/replogle/expression/replogle_4cl.h5ad" \
+  contexts=preprocessed/contexts.yaml output_dir="$PIE_DATA_ROOT/replogle/preprocessed"
+```
+
+Build the knowledge sources:
+
+You need to download the CRISPRGeneEffect.csv file and point the `pie sources` tool to it. 
+
+```bash
+uv run pie sources \
+  "tools=[esm2,ncbi_text,string_space,depmap_gene_effect,context_text,perturbation_text,gene_text]" \
+  "preprocessed_dirs=[$PIE_DATA_ROOT/replogle/preprocessed]" \
+  options.depmap_csv=CRISPRGeneEffect.csv options.device=cuda output_root="$PIE_DATA_ROOT/sources"
+```
+
+To use these local assets for a canonical experiment, override `data.preprocessed_dirs`,
+`data.source_dirs`, and `data.gene_text_dir`; the published recipes select HF assets by default.
+
+## Train and evaluate: replogle_wdataset
+
+Each fold holds out one cell line and trains on the other three. For the K562 fold:
+
+```bash
+SPLITS=hf://datasets/arcinstitute/PIE_splits@396ab9563175ee887750c9eed7ccaea6f5fdbf50
+uv run pie train experiment=replogle_wdataset vars.fold=k562
+for setting in unseen_ctx unseen_pert unseen_ctx_pert; do
+  uv run pie eval experiment_name=replogle_wdataset/k562 ckpt=best_auprc \
+    split_path=$SPLITS/replogle_wdataset/$setting/k562/test.json row_set=$setting
+done
+```
+
+The three test settings are unseen cell line (`unseen_ctx`), unseen perturbation (`unseen_pert`)
+and both (`unseen_ctx_pert`). Loop over `hepg2`, `jurkat`, `k562` and `rpe1` to run all four
+folds. See [AGENTS.md](AGENTS.md) for the cross-dataset experiment, inference and every CLI option.
+
+## Train and evaluate: replogle_xdataset
+
+This experiment trains on Tahoe, Jiang, ARC VCC 25, and Orion and evaluates zero-shot on Replogle.
+It uses two nodes with four GPUs each and a run directory on shared storage. Run the following
+on each node with its rank, a shared rendezvous ID, and the node-0 hostname:
+
+```bash
+uv run torchrun --nnodes=2 --nproc-per-node=4 --node-rank=<0|1> \
+  --master-addr=<node-0 host> --master-port=29500 --rdzv-id="$RDZV_ID" \
+  --no-python pie train experiment=replogle_xdataset
+```
+
+Evaluate the best checkpoint on one GPU:
+
+```bash
+SPLITS=hf://datasets/arcinstitute/PIE_splits@396ab9563175ee887750c9eed7ccaea6f5fdbf50
+for rows in test_seen test_unseen; do
+  uv run pie eval experiment_name=replogle_xdataset ckpt=best_auprc \
+    split_path=$SPLITS/replogle_xdataset/$rows.json row_set=$rows
+done
+```
+
+## License
+
+PIE is released under the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0
+International](https://creativecommons.org/licenses/by-nc-sa/4.0/) license (CC BY-NC-SA 4.0).
+You may use, share and adapt it for non-commercial purposes with attribution, and adaptations
+must be shared under the same license. See [LICENSE.md](LICENSE.md) for the full text.

arc_pie-0.1.6.dist-info/RECORD

@@ -0,0 +1,84 @@
+pie/__init__.py,sha256=L6LLjwBj0QBa2XitZUzUfrFGveFjDqTimNqPE76CT5Y,313
+pie/assets.py,sha256=NYv9QuNT2b-dbh1Qk6ufStg_0lb_duYq6oNf4hYMkzM,11471
+pie/cli.py,sha256=btm9ysf10WB97IjXLGmBMDsfDkMYeesEHM_ZYCvLql4,8783
+pie/config.py,sha256=0N1nwPNDKMCe9izEF8FAngOyp8xpDfezPOe1zU-BlQg,10190
+pie/evaluate.py,sha256=Kb-cwPaJUPzSYNQBfOkGEPwOhzkqL4_eW59DkGmarmE,5195
+pie/infer.py,sha256=lB0AoA52FGl1OPYmiKVTEUvYs_YF9GqC4HNr0tPqfrI,1447
+pie/metrics.py,sha256=He1dTh-bvNSmKe6GghCMSr8IREiNChv8yDFx1Gw1mJA,17353
+pie/predict.py,sha256=_oQqYnVFmX38P-5f3maYsd5Y_gkv9mj-c0GAGIu0C94,12397
+pie/train.py,sha256=SLyiZXOdooF2usoyVU1vh-p5KvvZ2QWhmyr4oWVz5jI,19180
+pie/utils.py,sha256=G5kUcZQHn7pnLQts_TRQ_l_BATfl-Rnn-MdU1RqjBqE,12823
+pie/configs/eval.yaml,sha256=qdhkTCRes10z8pqVolzPlbQ8MkhluolV6_yRSQKdB0U,284
+pie/configs/infer.yaml,sha256=X1BMPGoSiufTIhB6EC7fjpBBAz4aR9SaE-z9rb55uVY,332
+pie/configs/prep.yaml,sha256=ReF8Z1TVei-pKyEYUv5zxQhMM8069SxiMyiCc1E9ikc,969
+pie/configs/process.yaml,sha256=3ps3anu3pNec1apQBhsVKfLQy5KcA5Tl_4oCEiuODEM,920
+pie/configs/sources.yaml,sha256=oRFaCQzZaEw_GuYFxBSpD75sYZIQ3aRqYHpFmDFcCCM,691
+pie/configs/train.yaml,sha256=aokhldcqiwkXJyDMI0xVvLfv6MevKpG9XQGOL_T6ll0,1602
+pie/configs/experiment/replogle_wdataset.yaml,sha256=LyFuc9_6_cHVvAYsijXEPIvdXokThwJC3bCGpeMw9Hk,1750
+pie/configs/experiment/replogle_xdataset.yaml,sha256=kF6hNbb30doQGOS37guuH9x1OX5rx1Gxh0cqb-JDJtQ,2567
+pie/configs/prep/dataset/arc_vcc_25.yaml,sha256=ngg4BRoEvAdslR_ew-9pXIXOW2eDkye-JXHoKdl5wGQ,394
+pie/configs/prep/dataset/jiang.yaml,sha256=eYQuVcqxNkerowGi2-132KBa4cJr4RE1NjLFxD8mgPo,143
+pie/configs/prep/dataset/orion.yaml,sha256=sAzB75pZK1ySq4Jnwi96bVpbO-Ug_tr1sESHFifTdkY,142
+pie/configs/prep/dataset/replogle.yaml,sha256=Kcc6j3PF1jiE3luj-8eHB72HTmZtSgnTbn5RXyfRiLk,387
+pie/configs/prep/dataset/tahoe.yaml,sha256=gVz7ed9Q1r1WIGq7zGEyPYGw7Tu9mCLzf093KoST6_4,256
+pie/configs/prep/label_format/pie_process.yaml,sha256=wZjK0U8QqevoFazxO7VCxrNX-MBh6lAfNQYS4p4XVxA,267
+pie/configs/prep/label_format/table.yaml,sha256=vOJjrOc-n4A2HMl6QFSK9YxEtiD57Z3kJeHDHQEdOeM,253
+pie/configs/process/dataset/arc_vcc_25.yaml,sha256=osx-Oeqm36xRwCrJBJHv23E7Wn8AQiyHEGZktyGYutk,308
+pie/configs/process/dataset/jiang.yaml,sha256=fznejh7HWyWTyYvXJnqEomqxjgm_GjZT-lax6d0I7r0,262
+pie/configs/process/dataset/orion.yaml,sha256=IdXwW0xghs-CzeXq-UwHtjXdNOMn5eUb6WgvLMXiQVI,253
+pie/configs/process/dataset/replogle.yaml,sha256=dWjUkkUggUssteEYKxL4Z4ehyExc92I4cO8uBJqkdas,236
+pie/configs/process/dataset/tahoe.yaml,sha256=bVEnZAHSyqx9Y9cUgWhZfG3TXWK-WRRmZBRwCrGy3q4,212
+pie/data/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+pie/data/datamodule.py,sha256=752aaPHM3FckjlSTNO1JfdJGSQE2AZDucLhxTPEb4Vo,19373
+pie/data/dataset.py,sha256=d5Bv0q_lN9ibYZvNlVa2TFwP4ydhw_cVg96xMXYebAI,11410
+pie/data/delta_p.py,sha256=U5wJXGEnWWsL2X-lD9-V7QqJhO-2q0Ge3Vu_I9tgSJY,4491
+pie/data/evidence.py,sha256=bpGUhtY4Qbo0Bkq1hq-vklBEUy8LsDj-JREdKlkDZs0,41107
+pie/data/preprocessed.py,sha256=fVPsDt8QV3T18thzDyK0KhbVQ7U6vi0XFjt91Fz4zro,11207
+pie/data/samplers.py,sha256=E7Q0FUMhTNo7uw_hYSfblQWbTw6O3qrqSYh-DW3owtM,6574
+pie/data/splits.py,sha256=3i8vKLJz4Q5S1rS6h-7jlpL5OSAdKW3oJ5y2EnvOeDM,4990
+pie/model/__init__.py,sha256=u4Bs3jZS1OoKlyNAL0CbG1Bn_zu7tOU_f2g8YJMlYHs,435
+pie/model/evidence.py,sha256=igeOZnKi02rFzOz5StyIQgOK_zCi7bhc1M7cFou8lFk,3103
+pie/model/heads.py,sha256=iYodqWyYn4kGQUyae3whmoFae13NIXjdzfUyUK3MmJ4,1224
+pie/model/layers.py,sha256=SriJx3oRrumMMvR13xIWVjRrIU5E7tiEHr3uyJmA-II,1981
+pie/model/losses.py,sha256=JLnjQUhmJ6rxemoag_QYzETHxckoZN_rjS-eE5WQV1k,6343
+pie/model/pie.py,sha256=WoxNiqy9MPJSXDtW7e26uEZdbDwT9Bz0RkecCFSQ-cA,11879
+pie/model/trunk.py,sha256=9NBYlCZszNXzguluGu2pk43_NmAwLvmacKA3hF9LHLw,4038
+pie/prep/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+pie/prep/config.py,sha256=I9pxyPjpSVhgzLtXUDb23MY-5wwztZAr87yuRWfFOEI,2630
+pie/prep/labels.py,sha256=ENcq9TlSxfq2AAClZozd1EFzf63WmJi99J045-hTqvQ,29023
+pie/process/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+pie/process/config.py,sha256=IMsAgXf9FjUDFyoNgJr-Z-5Ew2WbfL16ZebGL0dvlCk,2919
+pie/process/de.py,sha256=Ur_JEKl7JP0QokUkLImEP2mkE5crX8hxvx4L4xQTPtc,5521
+pie/process/io.py,sha256=kXSIViQprDMwlQjQDwLbJGc1GoQhoESLATzCVxRqQFI,4037
+pie/process/knockdown.py,sha256=dUVHy_lVQF4C6Sfd6kaXxh2MOp6y-N0RRu1kmff-_ZA,12140
+pie/process/normalize.py,sha256=mOMQkb-4oHGpZSeoUkaI4nmMmNP2RkiIi914iaXnIDM,6970
+pie/process/run.py,sha256=opfQg-VavKZjLJDNKKhlNUtaIBv6qi81S24KXiScN-4,2757
+pie/sources/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+pie/sources/chem_profiles.py,sha256=XhRQrvGv8Y1D2Dk4AJloxM3LurCrsm4gpjlRAfJLf1o,20419
+pie/sources/common.py,sha256=MOQE6o04Tvd9NFpT1MTqPFE5zXPRXQhJAkjySIEwX4g,4910
+pie/sources/config.py,sha256=3exyfpoFKEBCg4LobY_jH3FGWjZrnk80HoSPOTBVDjA,2376
+pie/sources/contract.py,sha256=78RaB2ljMdSJan8hZpUu2BXQpcB_p1_bwopeyn7Z69I,12575
+pie/sources/curated_aliases.yaml,sha256=F1Bcs8d6smU0WtiYKoTj0DE6zG99GO44MpKWiUAVuOU,770
+pie/sources/depmap.py,sha256=GAWT4Le-8VksL9Z9FZFJdyAjvIkDavutWxGwB3H_HUg,3084
+pie/sources/registry.py,sha256=0H8vuECOWzd9b5gmt03nAI-XUcKwJkzh4tbMGOSUJp4,7964
+pie/sources/string.py,sha256=Cn7qg8XBymiDP18e-jYn7KlOKvOwOFstdKd_lyevtfU,3572
+pie/sources/uniprot.py,sha256=k9x2FALUM887NBw5QCZcLnifr8irLwTm5muUSVLKm6w,5227
+pie/sources/verify.py,sha256=uYRMh9tVYELVceb580FEMCNVgw9FWAmzWLWZGJK6OQk,6094
+pie/sources/embed/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+pie/sources/embed/chemberta.py,sha256=aRK-5xAY7aQ2kv6Qcoav6CeIhhCJqEC1-dWLquGwnDg,6088
+pie/sources/embed/esm2.py,sha256=TEN0ek50UHZXtXiXxwpCuwP2GVWNZj2b2tGfqMaWMZk,5195
+pie/sources/embed/openai.py,sha256=RLaAL9tAHCFK4XxZ9BuB-48U8ZJHfzIZ2v06G0hJjh8,4330
+pie/sources/embed/qwen.py,sha256=sNfZbvhpEqEoD-TKMvADCBWmX13WMnS-XMSGA1818MY,5277
+pie/sources/text/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+pie/sources/text/_http.py,sha256=3GCYQOjC5ocV2KQDquxRiKEyEaDq-1vmYe_uRY_fnsA,2325
+pie/sources/text/context_file.py,sha256=J7IJZ6LS1wHfc0pm_vUowkNJPHBiE_CjttLNbIp-S3M,1569
+pie/sources/text/contexts.py,sha256=PTB94yDJhmQz7A6U4LSwmyhenQdqxwdsRnjRiVy3hlc,23384
+pie/sources/text/drugs.py,sha256=_gfU3HhId1vsEbObVJlHtzQ9shr33Cr0Z__GWBWGTFI,21083
+pie/sources/text/genes.py,sha256=mgRFSe6_EUULBuTHoCyBlR9oxu52My-zvXqCzJKVoRA,21868
+pie/sources/text/ncbi.py,sha256=Hdt07FYuGwIEmvX8iVYdKs6CzA54hgN2_mdCbWfoUwE,19032
+pie/sources/text/tools.py,sha256=2p1AUe_Wp2v-ZMf-BYDZF4hFW2ZHrNxEARlu7YcxCe8,10109
+arc_pie-0.1.6.dist-info/METADATA,sha256=fP_6O9bxY4fNlwqLUdDTB5ZVZGlQ-ikf67ghZauOAGM,8179
+arc_pie-0.1.6.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+arc_pie-0.1.6.dist-info/entry_points.txt,sha256=GiZMOYuKChpcM9SB0M-xBahVhlPXaHI2IfivhTWaEcU,233
+arc_pie-0.1.6.dist-info/licenses/LICENSE.md,sha256=ja0peRnIxQjFyuKKrG-gWeNNr2YHHvDzDz0VKLJiyGo,18859
+arc_pie-0.1.6.dist-info/RECORD,,

arc_pie-0.1.6.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.27.0
+Root-Is-Purelib: true
+Tag: py3-none-any

arc_pie-0.1.6.dist-info/entry_points.txt

@@ -0,0 +1,8 @@
+[console_scripts]
+pie = pie.cli:main
+pie-eval = pie.cli:eval_alias
+pie-infer = pie.cli:infer_alias
+pie-prep = pie.cli:prep_alias
+pie-process = pie.cli:process_alias
+pie-sources = pie.cli:sources_alias
+pie-train = pie.cli:train_alias