encode-toolkit 0.3.2__tar.gz → 0.3.4__tar.gz

encode_toolkit-0.3.4/CHANGELOG.md

@@ -0,0 +1,329 @@
+# Changelog
+
+All notable changes to the ENCODE Toolkit will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.3.4] - 2026-09-20
+
+Bug-fix release for the MCP server, with the pipeline skills, their images and conda environments, the validation scripts and all documentation checked against the code.
+
+### Fixed
+
+- **`encode_track_experiment` failed for every experiment.** The tool passed the experiment's
+  assemblies to the tracker as a list, which SQLite cannot store, so tracking stopped with
+  `Error binding parameter 13: type 'list' is not supported`. The tests did not see it because
+  they mocked the tracker. Assemblies are now stored as joined text, two experiments count as
+  compatible when they share an assembly, and a test runs the tool against a real tracker.
+- **Search results lost half their fields.** `encode_search_experiments` asked the API for
+  `frame=object`, which returns linked objects as paths, so every hit had an empty organism,
+  organ and assembly list, and targets and labs came back as slugs (`H3K4me1-human`,
+  `bradley-bernstein`) that the tool's own `target` and `lab` filters do not accept. The search
+  now names the fields it needs. Checked against a live ENCODE response.
+- **Publications without a PMID overwrote each other.** ENCODE lists some papers by PMCID only;
+  they collided on the `(experiment, pmid)` key, so one survived while the count reported all of
+  them. They are now kept apart, and storing them again does not duplicate them (including a
+  row that an older version stored with an empty PMID).
+- **`encode_search_files` with an organism ignored `offset`** and reported the number of files
+  it had collected as the total, so every page was the first page and `has_more` was false.
+  `encode_batch_download` always took that path. Pages now advance and `total` is a lower bound
+  that exceeds the page when more files exist. The search also stopped after the first 200
+  experiments, and after the first 200 files of each experiment, so later files were never
+  found; it now reads further pages of both until the requested page is full (at most 1,000
+  experiments, and it says so when it stops there, also in `encode_batch_download`'s replies).
+- A negative `offset` in `encode_search_experiments` and `encode_search_files` is treated as 0,
+  in the request, in the reported `offset` and in `has_more` / `next_offset`, instead of hiding
+  or repeating pages.
+- **`encode_get_experiment` always reported zero audit flags.** It asked ENCODE for
+  `frame=embedded`, which leaves out the `audit` property, so every experiment looked free of
+  errors and warnings (the search tool was not affected). It now asks for `frame=page`. Checked
+  against the portal: ENCSR133RZO has 8 warnings and 2 internal-action flags, not 0.
+- BibTeX export joined author names with commas, which BibTeX reads as a single author; they
+  are joined with `and`.
+- **`encode_list_tracked` and `encode_export_data` ignored a filter that matched nothing** and
+  returned every tracked experiment. They now return none.
+- `encode_batch_download` reported `next_offset` but had no `offset` parameter to continue with;
+  it has one now.
+- The description of `encode_get_experiment` promised quality metrics. It returns audit counts,
+  replicate counts and possible controls, and now says so.
+- The value lists behind `encode_get_metadata` named three values ENCODE does not have (file
+  format `dat`, output type `stable peaks`, output category `quality metric`) and lacked eleven
+  file formats (among them `h5ad`) and eight assemblies. File formats, assemblies and output
+  categories now match ENCODE's file schema, and the output types include the single-cell
+  sparse gene count matrices.
+- `encode_connector.__version__` was a hard-coded `0.2.1`; it now reports the installed version.
+- **Four pipelines failed at their last step.** MultiQC names its report after `--title`, so
+  CUT&RUN, DNase-seq, Hi-C and WGBS never produced the `multiqc_report.html` they declare. They
+  now pass `--filename multiqc_report` (checked against MultiQC 1.21).
+- **WGBS**: `MethylDackel extract` was given `--maxDepth`, which 0.6.1 does not accept, so
+  extraction could not start. The `--nOT/--nOB` values excluded the 3' ends of reads rather than
+  the end-repair bias they were documented to remove (Trim Galore's `--clip_R2 10` already does
+  that), so they are gone. The M-bias report captured stdout while MethylDackel prints its
+  suggested bounds on stderr. The final sorted BAM was never published. `--min_coverage` now
+  filters the bedMethyl files only, so the coverage statistics describe every covered CpG.
+- **RNA-seq**: plus- and minus-strand signal tracks were swapped for reverse-stranded (dUTP)
+  libraries, the default. Track naming now follows `--strandedness` the way ENCODE's
+  `STAR_RSEM.sh` does, and unstranded libraries get a single track. `--strandedness` is
+  validated. The kallisto index and the RSeQC gene model were hard-wired to file names in the
+  launch directory; they are now `--kallisto_index` and `--rseqc_bed`.
+- **DNase-seq**: insert sizes used Picard `CollectInsertSizeMetrics`, which needs R for its
+  mandatory histogram and the image has none. It now uses `samtools stats`.
+- **ChIP-seq**: `--chrom_sizes` is required up front instead of failing on a missing
+  `chrom.sizes` file, the BWA index directory is a parameter (`--bwa_index`), and duplication
+  metrics are published (the `publishDir` pattern never matched the file name).
+- **ATAC-seq**: the Bowtie2 index directory is a parameter (`--bowtie2_index`), the
+  mitochondrial statistics are published, and two parameters that nothing read are removed.
+- **CUT&RUN** publishes the fragment BED and the spike-in counts. **Hi-C** publishes the
+  pairtools parse statistics and passes the normalization to HiCCUPS explicitly.
+- **Cloud profiles**: every pipeline takes `--gcp_project`/`--gcp_workdir` and
+  `--aws_queue`/`--aws_workdir`, and stops with a clear message when `-profile gcp` or `aws` is
+  used without them. All seven configs now come from one template, so resource limits, SLURM
+  queue/account and report settings no longer differ between pipelines.
+- `install-nextflow.sh` accepted any Nextflow already on the PATH, which skipped both the pinned
+  version and the checksum. It now accepts only the pinned release.
+- Trimmed-read FastQC reports are kept and passed to MultiQC in every pipeline that trims.
+- Conda environments: `idr` added to ATAC-seq, `picard` added to DNase-seq, BWA 0.7.18.
+
+- **The four validation scripts that ship in the aggregation skills had no tests and several
+  bugs.** `validate_methylation.py` rescaled every methylation value at or below 1 by 100, so in
+  an ENCODE bedMethyl (percentages) a CpG at 1% was reported as 100% methylated and every real
+  file triggered a "mixed formats" warning; the scale is now decided once per file (`--scale`).
+  Reversed intervals entered the statistics in the histone and Hi-C scripts, 9- and 10-column
+  bedMethyl files were read with the wrong columns, percentages were taken over malformed lines
+  too, gzipped inputs crashed, and an empty file passed. All fixed, with 37 tests that run the
+  scripts on synthetic files. The skills now say how to run them.
+
+### Changed
+
+- **One version of every tool.** The ChIP-seq, ATAC-seq and RNA-seq images move to the versions
+  the other four images and all seven conda environments already pinned: samtools 1.19, Picard
+  3.1.1 (Java 17), MultiQC 1.21, cutadapt 4.6, Trim Galore 0.6.10, FastQC 0.12.1, STAR 2.7.11b,
+  kallisto 0.50.1, RSeQC 5.0.3. Bowtie2 is 2.5.4 in every image. A kallisto index has to be
+  built with 0.50.1; older index files are rejected.
+- **FRiP is computed** by the ChIP-seq, ATAC-seq and CUT&RUN workflows
+  (`qc/<sample>.frip_mqc.tsv`, shown as a table in MultiQC).
+- **IDR compares every pair of replicates** instead of only the first two. Outputs are named
+  `peaks/idr/<sampleA>_vs_<sampleB>.idr_peaks.txt`.
+- **Hi-C honours `--resolutions`**: the cooler base bin is the smallest requested resolution,
+  and the new `--hiccups_resolutions` selects the loop-calling resolutions. Both lists are
+  validated before anything runs (positive integers with no empty fields, multiples of the
+  smallest), and HiCCUPS
+  gets one peak width, window width and merge radius per resolution (juicer_tools exits when
+  the `-d` list has a different length).
+- The conda environments pin every tool with `==` (a single `=` is a prefix match in conda) and
+  add cutadapt 4.6 and OpenJDK 17, as in the images; they resolved to cutadapt 5.2, samtools
+  1.19.2 and Java 22 before. Every pin now resolves as written. cutadapt 4.6 has no
+  Python 3.11 build, so all seven environments use Python 3.10. The Hi-C and WGBS images
+  install `openjdk-17-jre-headless` like the other five instead of Ubuntu's `default-jre`
+  (Java 11), and the image smoke tests assert Java 17.
+- ATAC-seq publishes `samtools idxstats` (which MultiQC reads) in place of a text line that
+  needed `bc` and failed on genomes without the mitochondrial contig.
+- WGBS coverage statistics also report the run's `--min_coverage`; deduplication reports reach
+  MultiQC. RNA-seq publishes the unique-mapper signal bedGraphs and kallisto's HDF5 output.
+- In WGBS, Hi-C, DNase-seq and CUT&RUN every process asks for more memory on a retry.
+
+### Documentation
+
+- Every pipeline skill was reconciled with its workflow: parameters, example commands, output
+  layouts, step lists and QC tables now describe what the workflow does. Analyses it does not
+  run (for example FRiP, NSC/RSC, TSS enrichment, TAD and compartment calling, bisulfite
+  conversion rate) are labelled as manual steps instead of being listed as outputs.
+- `encode_*` tool calls in 40 skills used 135 parameter names the server does not have (for
+  example `encode_list_files(accession=...)` instead of `experiment_accession=`). All are
+  corrected.
+- Reference commands were checked against the pinned tool versions (Hotspot2, MethylDackel,
+  SEACR, Juicer tools, pairtools, RGT, MPRAflow) and corrected.
+- **`docs/`, the README, `agents/` and `commands/` were never checked against the code.** The
+  API reference, showcase, walkthrough and vignettes showed invented response shapes (facets as
+  `key`/`doc_count`, a nested `audit` object, `href`, `files_found`, wrong compatibility verdicts);
+  all are corrected to what the server returns, and the API reference documents all 20 tools
+  instead of 16. The seven pipeline agents described steps the workflows do not run (Arrowhead
+  TAD calling, a WGBS conversion-rate stage, RNA-SeQC, Trimmomatic) and flags they do not use.
+  The doc checker now covers all of these folders, and CI also compares `agents/` and
+  `commands/` with their copies under `plugin/`.
+- The pipeline vignettes and the pipeline-execution walkthrough told users to run
+  `nextflow run ENCODE-DCC/chip-seq-pipeline2` (the official ENCODE pipelines are WDL, not
+  Nextflow), to pull images that were never verified, and to pass options that no longer exist
+  (`--motif_db`, `--restriction_site`, `--gtf`, `--lambda_genome`, `--single_end`). They now
+  build the image from the skill's Dockerfile and run the toolkit's own workflows with real
+  parameters, outputs and versions; DNase-seq is documented as paired-end only, and WGBS
+  conversion rate as a manual check.
+- Examples used values ENCODE does not have: the assay title `RNA-seq` (it is `total RNA-seq`,
+  `polyA plus RNA-seq`, ...), and the output types `chromatin interactions` (it is `loops`) and
+  `filtered feature barcode matrix`. Four `encode_link_reference` calls omitted the experiment.
+  The WGBS CpG-coverage one-liner piped BAM output into `awk`; it uses `bedtools coverage` now.
+
+### CI
+
+- The seven `nextflow.config` files are generated by `.github/scripts/gen-pipeline-configs.py`, and CI fails
+  when a config differs from what the generator writes.
+- `check-tool-versions.py` fails the build when a pipeline image and its conda environment pin
+  different versions of the same tool, when the image pins a package the environment does not,
+  or when both install a tool the environment leaves unpinned. It compares Java too and rejects
+  an unversioned `default-jre`. Dockerfile comments do not count as evidence, and the few
+  deliberate exceptions (packages from the Ubuntu base) are printed.
+- `check-skill-docs.py` fails the build when a skill shows a tool call the server would reject
+  or a `nextflow run` example with a parameter or profile the pipeline does not declare, and
+  when an example output uses a field the server never emits (model fields, dict keys and
+  SQLite columns, read from the source with `ast`). It also rejects a call that omits a required
+  argument, and filter values ENCODE does not use (`assay_title="RNA-seq"`), in calls to the
+  tools that query the portal and in example outputs (single values, lists and facet terms).
+  Example outputs are checked against the fields that the documented tool itself can return,
+  followed through the code it calls, its models and the SQL rows it reads, not against one
+  list for the whole server.
+- Image smoke tests start FastQC and Trim Galore; the preview suite covers the new parameters
+  and the cloud-profile checks.
+
+## [0.3.3] - 2026-09-20
+
+Pipeline skills release. The Python package (MCP server) is functionally identical to 0.3.2.
+
+### Fixed
+
+- **All seven Nextflow pipelines now run on current Nextflow (validated on 26.04.6).** The
+  workflows mixed top-level statements with process definitions and four configs defined a
+  function, both of which the strict parser rejects. Validation and channel setup moved into the
+  `workflow` block, and `check_max` was replaced by `process.resourceLimits`.
+- **ChIP-seq**: the workflow called each process twice (samples, then controls), which Nextflow
+  does not allow, so it could not start. Controls now go through the same calls and are split
+  off before peak calling, where they are pooled. `--control` is optional. Signal tracks now
+  receive the sample ID they were missing.
+- **CUT&RUN**: SEACR was given the control BAM instead of a control bedGraph; spike-in scale
+  factors were computed but never applied to the signal track; `--seacr_mode` was ignored;
+  MACS2 peak calling always failed on a no-op `mv`; fragments were extracted from a
+  coordinate-sorted BAM, which drops most read pairs; `--control` and chromosome sizes were
+  not staged into tasks.
+- **DNase-seq**: Hotspot2 was called with options it does not have and without its mandatory
+  center-sites file; the image pinned a Hotspot2 tag that does not exist and lacked `modwt`
+  and `bc`. Footprinting could pair a BAM with another sample's peaks. `--hotspot_index` is
+  replaced by `--hotspot_center_sites` and `--hotspot_mappable`.
+  HINT footprinting needs an RGT data directory that the container's unprivileged user could
+  never find; it is now an explicit input, `--rgt_data`, checked before the run starts.
+- **WGBS**: the bedMethyl conversion divided by zero on MethylDackel's header line, so
+  extraction always failed; only CpG was converted although CHG and CHH were promised;
+  `--no_overlap` toggled `--mergeContext`, which is unrelated to mate overlap (renamed
+  `--merge_context`). bedMethyl score and strand now follow the ENCODE format.
+- **RNA-seq**: the RSEM reference is a file prefix but was required to be a directory.
+- **ATAC-seq**: the BAM index was not passed to the Tn5 shift step, which `alignmentSieve`
+  requires; the mitochondrial fraction used `bc`, which the image lacked, and silently wrote an
+  empty value; duplication metrics were never published. The workflow now states that it needs
+  paired-end reads instead of filtering every single-end read away.
+- **Hi-C**: `pairtools sort` was given a temporary directory that was never created. HiCCUPS
+  now runs its CPU mode by default, because the image has no CUDA runtime (`--hiccups_gpu`).
+- **IDR** (ChIP-seq, ATAC-seq) picked two peak files in arbitrary order and crashed with a
+  single replicate. The pair is now sorted, and IDR is skipped below two replicates.
+- **Pipeline images had never been built.** Beyond missing `build-essential`, `unzip`, `bc`, and
+  Boost: `idr`, `trim-galore`, and `phantompeakqualtools` are not PyPI packages; `deeptools
+  3.5.4` was never published; BWA 0.7.17 does not link with current GCC (now 0.7.18); Picard 3
+  needs Java 17; MethylDackel needs libBigWig; SEACR could not find its R script through a
+  symlink; RGT 0.13.2 and pairtools 1.0.3 no longer install (now 1.0.2 and 1.1.2). Index
+  prefixes are resolved from staged files, so cloud executors work.
+- **Conda environment files** pinned packages that do not exist (`hotspot2`, `hint`, `f-seq2`)
+  or cannot be installed together. All seven now solve, and the Anaconda `defaults` channel is
+  no longer used.
+- Pipelines referenced container images that do not exist. Each config now uses an image built
+  from the skill's own Dockerfile, with a fixed tag and a `--container` override.
+- `gcp` profiles used the retired `google-lifesciences` executor; they now use `google-batch`.
+- QC references: `samtools view` needs `-L` for a BED file; the WGBS coverage one-liner never
+  counted bases at 5x or more.
+
+### Changed
+
+- **Removed parameters that had no effect**: `--aligner` and `--lambda_genome` (WGBS),
+  `--motif_db` (DNase-seq), `--restriction_site` (Hi-C), `--gtf` (RNA-seq). The skills now
+  describe what the workflows actually do.
+- `install-nextflow.sh` installs a pinned Nextflow release and verifies its SHA-256 before use,
+  instead of piping a remote script into a shell. It no longer fails when the install
+  directory is not on the `PATH`.
+- `install-python-packages.sh` installs against `constraints.txt`, a lock file with exact
+  versions for Python 3.10+, generated from `requirements.in`.
+- The pipeline guide now separates the official ENCODE WDL pipelines from this toolkit's own
+  Nextflow implementations, and no longer cites repositories or images that do not exist.
+  Pipeline images are no longer labelled as maintained by the ENCODE DCC.
+
+### Added
+
+- `Pipelines` CI workflow: `nextflow lint`, `nextflow run -preview` across parameter
+  combinations, profile resolution, a Docker build of every pipeline image with checks that
+  the tools each workflow calls are present and actually start, a dry-run solve of every conda
+  environment, shellcheck, and a `skills/` vs `plugin/skills/` identity check.
+
+## [0.3.2] - 2026-09-20
+
+Maintenance release. The Python package is functionally identical to 0.3.1.
+
+### Fixed
+
+- `bioinformatics-installer` skill: the ChIP-seq walkthrough pointed at `scripts/chipseq-env.yml`
+  (the file lives in `environments/`) and at an `annotation-env.yml` that did not exist. The first
+  path is corrected and the second is replaced with an explicit `conda create` command.
+
+### Changed
+
+- Removed local tooling configuration files from the repository and ignored them going forward.
+- `CONTRIBUTING.md` and `docs/SHOWCASE.md` now use the ENCODE Toolkit name and the current skill
+  count (47).
+
+## [0.3.1] - 2026-09-20
+
+### Fixed
+
+- **Server failed to start on fresh installs.** The `mcp` dependency had no upper bound, so new
+  environments resolved `mcp` 2.x, which removed `mcp.server.fastmcp`. Startup then crashed with
+  `ModuleNotFoundError: No module named 'mcp.server.fastmcp'`. The dependency is now capped at
+  `mcp[cli]>=1.0,<2`. Existing installs that already had `mcp` 1.x were not affected.
+
+  If you hit this error, uv may have cached the broken environment. Refresh it once with
+  `uvx --refresh encode-toolkit` (or `uv cache clean encode-toolkit`); pip users can run
+  `pip install --upgrade encode-toolkit`.
+
+### Security
+
+- The ChIP-seq, ATAC-seq, and RNA-seq pipeline Dockerfiles now download the UCSC
+  `bedGraphToBigWig` executable over HTTPS instead of plain HTTP.
+
+### Changed
+
+- The source distribution now contains only the Python package, tests, and project documents.
+  It previously bundled the whole repository, including editor configuration and a duplicate
+  copy of the plugin tree (1.6 MB down to 118 KB). The wheel is unchanged.
+- Updated dead GREAT links in the `peak-annotation` and `multi-omics-integration` skills.
+
+### Added
+
+- Packaging regression test that fails if the `mcp` dependency loses its upper bound.
+
+## [0.3.0-beta.1] - 2026-03-08
+
+Initial public beta release.
+
+### Features
+
+- **20 MCP tools** for searching, downloading, and tracking ENCODE data
+  - Search experiments and files with comprehensive filters and pagination
+  - Download files with MD5 verification, concurrent downloads, and directory organization
+  - Local experiment tracking with SQLite (publications, pipelines, quality metrics)
+  - Cross-reference with PubMed, bioRxiv, ClinicalTrials.gov, GEO
+  - Citation export (BibTeX, RIS) for reference managers
+  - Data provenance chain for derived files
+  - Batch download with dry-run preview
+
+- **47 skills** across 10 categories
+  - Core: setup, search, download, track, cross-reference
+  - Analysis: quality assessment, integrative analysis, regulatory elements, epigenome profiling, compare biosamples, visualization, motif analysis, peak annotation, batch analysis
+  - Functional genomics: CRISPR/MPRA/STARR-seq screen analysis
+  - Data aggregation: histone, accessibility, Hi-C, methylation
+  - External databases: UCSC, GEO, gnomAD, Ensembl, GTEx, ClinVar, CELLxGENE, GWAS Catalog, JASPAR
+  - Workflows: provenance, citations, variant annotation, pipelines, single-cell, disease research, publication trust, bioinformatics installer, scientific writing, liftover coordinates
+  - Pipeline execution: ChIP-seq, ATAC-seq, RNA-seq, WGBS, Hi-C, DNase-seq, CUT&RUN (Nextflow + Docker)
+  - Meta-analysis: scRNA-seq meta-analysis, multi-omics integration
+
+- **Async ENCODE API client** with retry logic, 1-hour TTL cache, and rate limiting
+- **OS keyring credential management** with Fernet-encrypted file fallback
+- **Thread-safe SQLite tracker** with full transaction safety
+- **Streaming downloads** with 64KB chunks and SSRF-safe redirect validation
+- **568 tests** with 98% code coverage
+- **34 literature reference documents** (~320 papers cataloged with DOI, PMID, key findings)
+- **9 scientist-facing vignettes** with real ENCODE API output
+- **GitHub Actions CI/CD** (pytest across Python 3.10–3.13, ruff lint, plugin validation)

{encode_toolkit-0.3.2 → encode_toolkit-0.3.4}/CITATION.cff

@@ -8,7 +8,7 @@ authors:
     given-names: Alex M.
     orcid: "https://orcid.org/0000-0003-0907-464X"
     affiliation: "Independent Researcher"
-version: 0.3.2
+version: 0.3.4
 date-released: "2026-09-20"
 license: AGPL-3.0-only
 repository-code: "https://github.com/ammawla/encode-toolkit"

{encode_toolkit-0.3.2 → encode_toolkit-0.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: encode-toolkit
-Version: 0.3.2
+Version: 0.3.4
 Summary: MCP server for querying and downloading ENCODE Project genomics data directly from Claude
 Project-URL: Homepage, https://github.com/ammawla/encode-toolkit
 Project-URL: Repository, https://github.com/ammawla/encode-toolkit
@@ -37,7 +37,7 @@ Description-Content-Type: text/markdown
 
 [![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-green.svg)](LICENSE)
 [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
-[![Version](https://img.shields.io/badge/version-0.3.2-green)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-0.3.4-green)](CHANGELOG.md)
 [![Status](https://img.shields.io/badge/status-beta-yellow)]()
 [![Skills](https://img.shields.io/badge/skills-47-orange)](docs/skill-vignettes/)
 [![Tools](https://img.shields.io/badge/MCP_tools-20-purple)](src/encode_connector/server/main.py)
@@ -331,7 +331,7 @@ Search ENCODE experiments with 20+ filters.
 
 | Parameter | Type | Description |
 |-----------|------|-------------|
-| `assay_title` | string | Assay type: "Histone ChIP-seq", "ATAC-seq", "RNA-seq", "Hi-C", etc. |
+| `assay_title` | string | Assay type: "Histone ChIP-seq", "ATAC-seq", "total RNA-seq", "Hi-C", etc. |
 | `organism` | string | Species (default: "Homo sapiens") |
 | `organ` | string | Organ: "pancreas", "brain", "liver", "heart", "kidney", etc. |
 | `biosample_type` | string | "tissue", "cell line", "primary cell", "organoid" |
@@ -496,7 +496,7 @@ Get grouped statistics of your tracked experiment collection.
 </details>
 
 <details>
-<summary><strong>Provenance and export tools (4)</strong></summary>
+<summary><strong>Provenance and export tools (5)</strong></summary>
 
 ### `encode_log_derived_file`
 
@@ -624,7 +624,7 @@ When installed as a Claude Code plugin, ENCODE Toolkit includes 47 literature-ba
 </details>
 
 <details>
-<summary><strong>Workflow skills (7)</strong></summary>
+<summary><strong>Workflow skills (10)</strong></summary>
 
 | Skill | Description |
 |-------|-------------|
@@ -681,7 +681,7 @@ Each pipeline includes a SKILL.md overview, 5-stage reference files (preprocessi
 | File | Description |
 |-------|-------------|
 | `skills/histone-aggregation/references/histone-marks-reference.md` | Comprehensive chromatin biology catalog (1,442 lines) — 21 histone marks with writers/erasers/readers, 5 novel acylation marks, ChromHMM state models (5 to 51 states), TF co-binding patterns, chromatin remodeling complexes, DNA methylation-chromatin interplay, nucleosome dynamics, 3D genome organization, chromatin in disease. 74 primary references |
-| `skills/*/references/literature.md` | 33 per-skill literature reference documents — ~250 papers cataloged with DOI, PMID, citation counts, and skill-relevant key findings |
+| `skills/*/references/literature.md` | 47 per-skill literature reference documents — about 240 papers cataloged with DOI, PMID, citation counts, and skill-relevant key findings |
 
 </details>
 
@@ -710,12 +710,12 @@ Most genomics tools give you one thing. ENCODE Toolkit gives you the full resear
 | Category | Assays |
 |----------|--------|
 | **Histone/Chromatin** | Histone ChIP-seq, TF ChIP-seq, ATAC-seq, DNase-seq, CUT&RUN, CUT&Tag, MNase-seq |
-| **Transcription** | RNA-seq, total RNA-seq, small RNA-seq, long read RNA-seq, CAGE, RAMPAGE, PRO-seq, GRO-seq |
+| **Transcription** | total RNA-seq, polyA plus RNA-seq, small RNA-seq, long read RNA-seq, CAGE, RAMPAGE, PRO-seq, GRO-seq |
 | **3D Genome** | Hi-C, intact Hi-C, Micro-C, ChIA-PET, HiChIP, PLAC-seq, 5C |
 | **DNA Methylation** | WGBS, RRBS, MeDIP-seq, MRE-seq |
 | **Functional** | STARR-seq, MPRA, CRISPR screen, eCLIP, iCLIP |
-| **Single Cell** | scRNA-seq, snATAC-seq, 10x multiome, SHARE-seq, Parse SPLiT-seq |
-| **Perturbation** | CRISPRi + RNA-seq, shRNA + RNA-seq, siRNA + RNA-seq |
+| **Single Cell** | scRNA-seq, snATAC-seq, snRNA-seq, long read scRNA-seq |
+| **Perturbation** | CRISPRi RNA-seq, shRNA RNA-seq, siRNA RNA-seq, CRISPR RNA-seq |
 
 **Supported file formats**: `fastq` `bam` `bed` `bigWig` `bigBed` `tsv` `csv` `hic` `tagAlign` `bedpe` `pairs` `fasta` `vcf` `tar`
 

{encode_toolkit-0.3.2 → encode_toolkit-0.3.4}/README.md

@@ -4,7 +4,7 @@
 
 [![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-green.svg)](LICENSE)
 [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
-[![Version](https://img.shields.io/badge/version-0.3.2-green)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-0.3.4-green)](CHANGELOG.md)
 [![Status](https://img.shields.io/badge/status-beta-yellow)]()
 [![Skills](https://img.shields.io/badge/skills-47-orange)](docs/skill-vignettes/)
 [![Tools](https://img.shields.io/badge/MCP_tools-20-purple)](src/encode_connector/server/main.py)
@@ -298,7 +298,7 @@ Search ENCODE experiments with 20+ filters.
 
 | Parameter | Type | Description |
 |-----------|------|-------------|
-| `assay_title` | string | Assay type: "Histone ChIP-seq", "ATAC-seq", "RNA-seq", "Hi-C", etc. |
+| `assay_title` | string | Assay type: "Histone ChIP-seq", "ATAC-seq", "total RNA-seq", "Hi-C", etc. |
 | `organism` | string | Species (default: "Homo sapiens") |
 | `organ` | string | Organ: "pancreas", "brain", "liver", "heart", "kidney", etc. |
 | `biosample_type` | string | "tissue", "cell line", "primary cell", "organoid" |
@@ -463,7 +463,7 @@ Get grouped statistics of your tracked experiment collection.
 </details>
 
 <details>
-<summary><strong>Provenance and export tools (4)</strong></summary>
+<summary><strong>Provenance and export tools (5)</strong></summary>
 
 ### `encode_log_derived_file`
 
@@ -591,7 +591,7 @@ When installed as a Claude Code plugin, ENCODE Toolkit includes 47 literature-ba
 </details>
 
 <details>
-<summary><strong>Workflow skills (7)</strong></summary>
+<summary><strong>Workflow skills (10)</strong></summary>
 
 | Skill | Description |
 |-------|-------------|
@@ -648,7 +648,7 @@ Each pipeline includes a SKILL.md overview, 5-stage reference files (preprocessi
 | File | Description |
 |-------|-------------|
 | `skills/histone-aggregation/references/histone-marks-reference.md` | Comprehensive chromatin biology catalog (1,442 lines) — 21 histone marks with writers/erasers/readers, 5 novel acylation marks, ChromHMM state models (5 to 51 states), TF co-binding patterns, chromatin remodeling complexes, DNA methylation-chromatin interplay, nucleosome dynamics, 3D genome organization, chromatin in disease. 74 primary references |
-| `skills/*/references/literature.md` | 33 per-skill literature reference documents — ~250 papers cataloged with DOI, PMID, citation counts, and skill-relevant key findings |
+| `skills/*/references/literature.md` | 47 per-skill literature reference documents — about 240 papers cataloged with DOI, PMID, citation counts, and skill-relevant key findings |
 
 </details>
 
@@ -677,12 +677,12 @@ Most genomics tools give you one thing. ENCODE Toolkit gives you the full resear
 | Category | Assays |
 |----------|--------|
 | **Histone/Chromatin** | Histone ChIP-seq, TF ChIP-seq, ATAC-seq, DNase-seq, CUT&RUN, CUT&Tag, MNase-seq |
-| **Transcription** | RNA-seq, total RNA-seq, small RNA-seq, long read RNA-seq, CAGE, RAMPAGE, PRO-seq, GRO-seq |
+| **Transcription** | total RNA-seq, polyA plus RNA-seq, small RNA-seq, long read RNA-seq, CAGE, RAMPAGE, PRO-seq, GRO-seq |
 | **3D Genome** | Hi-C, intact Hi-C, Micro-C, ChIA-PET, HiChIP, PLAC-seq, 5C |
 | **DNA Methylation** | WGBS, RRBS, MeDIP-seq, MRE-seq |
 | **Functional** | STARR-seq, MPRA, CRISPR screen, eCLIP, iCLIP |
-| **Single Cell** | scRNA-seq, snATAC-seq, 10x multiome, SHARE-seq, Parse SPLiT-seq |
-| **Perturbation** | CRISPRi + RNA-seq, shRNA + RNA-seq, siRNA + RNA-seq |
+| **Single Cell** | scRNA-seq, snATAC-seq, snRNA-seq, long read scRNA-seq |
+| **Perturbation** | CRISPRi RNA-seq, shRNA RNA-seq, siRNA RNA-seq, CRISPR RNA-seq |
 
 **Supported file formats**: `fastq` `bam` `bed` `bigWig` `bigBed` `tsv` `csv` `hic` `tagAlign` `bedpe` `pairs` `fasta` `vcf` `tar`
 

{encode_toolkit-0.3.2 → encode_toolkit-0.3.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "encode-toolkit"
-version = "0.3.2"
+version = "0.3.4"
 description = "MCP server for querying and downloading ENCODE Project genomics data directly from Claude"
 readme = "README.md"
 license = "AGPL-3.0-only"

encode_toolkit-0.3.4/src/encode_connector/__init__.py

@@ -0,0 +1,9 @@
+"""ENCODE Project connector - MCP server and Python client."""
+
+from importlib.metadata import PackageNotFoundError, version
+
+try:
+    __version__ = version("encode-toolkit")
+except PackageNotFoundError:  # a source tree that is not installed
+    __version__ = "0+unknown"
+__author__ = "Dr. Alex M. Mawla, PhD"

{encode_toolkit-0.3.2 → encode_toolkit-0.3.4}/src/encode_connector/client/constants.py

@@ -12,12 +12,21 @@ DOWNLOAD_CONCURRENCY = 3
 DEFAULT_TIMEOUT = 30.0
 DOWNLOAD_TIMEOUT = 300.0
 DEFAULT_LIMIT = 25
+# Frame for a single experiment. "page" is "embedded" plus the "audit" property; with
+# "embedded" ENCODE omits the audits and every experiment looks free of errors and warnings.
+EXPERIMENT_FRAME = "page"
+# experiments read per request while a file search walks the experiments of one organism
+EXPERIMENT_PAGE_SIZE = 200
+# ...and how many experiments it reads at most: each one costs a request to ENCODE
+MAX_EXPERIMENTS_SCANNED = 1000
+# files of one experiment read per request during that walk
+FILES_PAGE_SIZE = 200
 try:
     import importlib.metadata
 
     _version = importlib.metadata.version("encode-toolkit")
 except importlib.metadata.PackageNotFoundError:
-    _version = "0.3.2"
+    _version = "0.3.4"
 USER_AGENT = f"encode-toolkit/{_version} (MCP; +https://github.com/ammawla/encode-toolkit)"
 
 # Keyring service name for credential storage
@@ -239,8 +248,18 @@ FILE_FORMATS = [
     "vcf",
     "bigInteract",
     "idx",
-    "dat",
     "txt",
+    "h5ad",
+    "hdf5",
+    "sam",
+    "wig",
+    "starch",
+    "chain",
+    "PWM",
+    "btr",
+    "cndb",
+    "nucle3d",
+    "yaml",
 ]
 
 OUTPUT_TYPES = [
@@ -277,7 +296,6 @@ OUTPUT_TYPES = [
     "pseudoreplicated peaks",
     "pseudoreplicated IDR thresholded peaks",
     "replicated peaks",
-    "stable peaks",
     "hotspots",
     "footprints",
     "peaks and background as input for IDR",
@@ -289,6 +307,10 @@ OUTPUT_TYPES = [
     "filtered peaks",
     # Quantifications
     "gene quantifications",
+    "sparse gene count matrix of unique reads",
+    "sparse gene count matrix of all reads",
+    "unfiltered sparse gene count matrix of unique reads",
+    "unfiltered sparse gene count matrix of all reads",
     "transcript quantifications",
     "exon quantifications",
     "microRNA quantifications",
@@ -347,7 +369,6 @@ OUTPUT_CATEGORIES = [
     "annotation",
     "quantification",
     "reference",
-    "quality metric",
 ]
 
 FILE_STATUSES = [
@@ -381,6 +402,14 @@ ASSEMBLIES = [
     "dm3",
     "ce11",
     "ce10",
+    "GRCh38-minimal",
+    "mm10-minimal",
+    "T2T-CHM13",
+    "J02459.1",
+    "ENC001.1",
+    "ENC002.1",
+    "ENC003.1",
+    "ENC004.1",
 ]
 
 LIFE_STAGES = [
@@ -416,6 +445,28 @@ METADATA_MAP = {
 }
 
 # ENCODE API parameter name mapping (user-friendly -> API param)
+# Fields requested from /search/?type=Experiment. They are what ExperimentSummary.from_api
+# reads; naming them makes the API embed labels instead of returning object paths.
+EXPERIMENT_SEARCH_FIELDS = (
+    "accession",
+    "assay_title",
+    "target.label",
+    "biosample_summary",
+    "biosample_ontology.classification",
+    "biosample_ontology.organ_slims",
+    "replicates.library.biosample.organism.scientific_name",
+    "status",
+    "date_released",
+    "description",
+    "lab.title",
+    "files.@id",
+    "replication_type",
+    "life_stage_age",
+    "assembly",
+    "audit",
+    "dbxrefs",
+)
+
 EXPERIMENT_FILTER_MAP = {
     "assay_title": "assay_title",
     "organism": "replicates.library.biosample.donor.organism.scientific_name",