chromosomix 0.15.0 → 0.17.0

package/README.md

@@ -6,7 +6,7 @@
 [![npm](https://img.shields.io/npm/v/chromosomix?logo=npm)](https://www.npmjs.com/package/chromosomix)
 [![Release](https://img.shields.io/github/v/release/js2264/chromosomix?logo=github)](https://github.com/js2264/chromosomix/releases)
 [![Validation](https://img.shields.io/badge/validation-vs%20reference%20tools-2ea44f)](https://chromosomix.app/validation/)
-[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
+[![License: GPL-3.0-or-later](https://img.shields.io/badge/license-GPL--3.0--or--later-blue)](LICENSE)
 
 **A unified genome browser for multi-dimensional genomic data.** Chromatin contact maps, genome tracks and 3D models, side by side, straight from your files.
 
@@ -20,8 +20,8 @@ chromosomix launches in about a second and reads your data where it lives, on yo
 
 - Support for all genomic data types: 
   - Supports 1D genomic formats: bigWig, bigBed, BED, bedGraph, wig, GTF/GFF3, FASTA (+`.fai`), `.2bit`, BAM, VCF, barChart/bigBarChart, genePred/refFlat/bigGenePred, bedMethyl/bigMethyl, chrom.sizes, ...
-  - Supports 2D genomic formats: `.*cool`, `.hic`, HiC-Pro, `pairs(.gz)`, ..., at every resolution the file stores. Also supports interaction formats: BEDPE, and UCSC's `interact`/`bigInteract` and `longrange`/longTabix — drawn as arcs over the contact map and as their own lane in the 1D browser.
-  - Supports 3D genomic formats: WashU's g3d (text or binary), `.3dg`, and `bed3d`, drawn in 3D and synchronized with the maps and tracks view. 
+  - Supports 2D genomic formats: `.*cool`, `.hic`, HiC-Pro, `pairs(.gz)`, ..., at every resolution the file stores. A `.pairs` is binned on import, and the same pass counts what `pairtools stats` counts — mapped, duplicates, cis and trans, pair types, strand convergence, library size — shown in the General panel and saved as a `.stats` file MultiQC reads. Also supports interaction formats: BEDPE, and UCSC's `interact`/`bigInteract` and `longrange`/longTabix — drawn as arcs over the contact map and as their own lane in the 1D browser.
+  - Supports 3D genomic formats: WashU's g3d (text or binary), `.3dg`, and `bed3d`, drawn in 3D and synchronized with the maps and tracks view. Every model also leaves as a PDB for PyMOL, ChimeraX and the other molecular viewers, a chain per chromosome and bonds along the fibre, and reads back here. 
 
 - Perform analyses directly in the browser:
   - 1D analysis tasks: peak calling from a coverage track (yapc, with IDR between replicates), motif search, base composition and GC skew, the origin and terminus of replication, metaplots, track correlation, FRiP over a feature set, signal by group, interval overlap with a shuffle null, ...
@@ -30,8 +30,6 @@ chromosomix launches in about a second and reads your data where it lives, on yo
 
 - Serverless browser & CLI-based: there is no upload, no required preprocessing step and no backend of any kind — the viewer is static files that read standard genome-wide formats directly. Every analysis can be run **interactively**, in the browser, or **programmatically**, with the `chromosomix` command line in a script or a pipeline: the same tasks, with the same parameters, producing the same results. A bottom Console panel in the browser records every run and reports the `chromosomix` command that repeats it.
 
-- Side by side: contact maps, 1D browsers and 3D models open as the cells of a deck, panned and zoomed together, and several sessions stay open at once, one tab each.
-
 - Dedicated browser panels, four around the picture:
   - General panel: the file itself — its attributes, its chromosomes (or a metagenome's genomes and contigs), a minimap of the genome and the metadata it stores.
   - Aesthetics panel: to customize scales, color map, normalization, track layout, grouping, ...
@@ -42,13 +40,17 @@ chromosomix launches in about a second and reads your data where it lives, on yo
   - Cloud-based file access: Open files directly from any object store that serves byte ranges with CORS (e.g. S3, Cloudflare R2, Google Cloud Storage, or any HTTP server that supports range requests), from Dropbox and Google Drive share links, and private Google Drive folder or Dropbox.
   - Stores: several pre-configured remote data stores are available — the lab's own figures, IGV's genomes, 4DN open data, ENCODE, DNA Zoo and Roadmap — and so is any UCSC track hub with a `hub.txt` file. The user can also setup their own store by providing a structured JSON file describing the store and its data.
 
+- Side by side: contact maps, 1D browsers and 3D models open as the cells of a deck, panned and zoomed together, and several sessions stay open at once, one tab each.
+
 ## Philosophy
 
 1. **No backend or install of any kind.** No tile server, no specific process on a host, no accounts. The viewer is static files; the data are objects on a bucket, or a file on your disk. This is what makes the app free to run and free to archive.
 
-1. **Nothing is uploaded.** A local file is handed to an air-gapped worker by reference and read in place through the Emscripten filesystem. Nothing crosses the network, which is why there is no account system, no storage, no hosting cost and nothing to review for GDPR. You can open unpublished patient-derived genomic data on a laptop with the network cable out.
+2. **Nothing is uploaded.** A local file is handed to an air-gapped worker by reference and read in place through the Emscripten filesystem. Nothing of it crosses the network, which is why there is no account system, no storage and no hosting cost. You can open unpublished patient-derived genomic data on a laptop with the network cable out.
+
+   The website does send anonymous telemetry about how it is used — which kinds of file are opened, which analyses run, which errors are met — as words from fixed lists, with no cookie and nothing that identifies you: never a file name, an address or a place on the genome. **About → telemetry** turns it off, and a browser sending Global Privacy Control sends nothing. The desktop app, the static HTML file, the npm package and embeds never send any ([SPECS §31](SPECS.md#31-telemetry)).
 
-1. **No pre-parsing of genomic files is required.** Contact maps and 1D genomic files are read in bounded reads, and the rest is pure canvas transforms with **zero** further I/O. That is the central difference from HiGlass, which needs a tile server, a tile cache and a tile compositor to stream contact maps. 
+3. **No pre-parsing of genomic files is required.** Contact maps and 1D genomic files are read in bounded reads, and the rest is pure canvas transforms with **zero** further I/O. That is the central difference from HiGlass, which needs a tile server, a tile cache and a tile compositor to stream contact maps. 
 
 ## Using chromosomix
 
@@ -96,17 +98,57 @@ A minimal version of the viewer can be added to any HTML page as an `<iframe>`.
 
 Paste it into a page and it works (not rendered here though because GitHub does not allow `<iframe>`). The `src` URL is the same as the one you would use to open the viewer in a browser, with the addition of `?embed=1` to tell it to hide the header and tab strip. 
 
-The text below is a live version of the above `<iframe>` (it will not work on GitHub, but it would work on any HTML page):
+The page holding the frame can drive it too: `postMessage` moves the view, adds tracks beside the map, and is told where the reader has taken it (SPECS §21.4).
 
-<iframe
-  src="https://chromosomix.app/?embed=1#uri=https://rsglab-data.s3.eu-west-3.amazonaws.com/demo/yeast/Yeast_microC_G2M_1kb.cool&view=region&region=IV:1000000-1400000&transform=log10&balance=true"
-  width="100%"
-  height="640"
-></iframe>
+## In a Jupyter notebook
+
+The same viewer is a Jupyter widget: the app itself in a cell, driven from Python and telling Python where the reader has taken it. It works in JupyterLab, Notebook 7, VS Code, Colab and marimo.
+
+```bash
+pip install chromosomix          # from a clone: pip install -e widgets/python
+```
+
+```python
+from chromosomix import Chromosomix
+
+view = Chromosomix(
+    'sample.mcool::/resolutions/5000',      # a path on the kernel's disk, or any address the app opens
+    region='chr2:10000000-14000000',
+    tracks=['ATAC.bw', 'genes.gtf.gz'],
+    transform='log10',
+)
+view                                        # the map, its tracks, its side panels
+
+view.goto('chr2:11000000-12000000')         # frame a region (or a locus, in the 1D browser)
+view.navigate(transform='log2', oe=True)    # any key the address bar carries
+view.add('model.g3d.gz')                    # the header's +: a track, a map, a 3D model beside the map
+view.observe(lambda change: print(change['new']['region']), names='state')
+```
+
+Files on the kernel's disk are served to the viewer by byte range from a server on `127.0.0.1`, with their sidecars beside them; nothing is uploaded. [`widgets/python/demo.ipynb`](widgets/python/demo.ipynb) walks through it on the yeast demo, and [`widgets/python/README.md`](widgets/python/README.md) says which viewer build the frame holds and how to pin one.
+
+## In R: R Markdown, Quarto, RStudio and Shiny
+
+The same viewer is an htmlwidget, in the R package under [`widgets/r`](widgets/r/README.md):
+
+```r
+install.packages("widgets/r", repos = NULL, type = "source")   # from a clone
+
+library(chromosomix)
+chromosomix("sample.mcool::/resolutions/5000", region = "chr2:10000000-14000000",
+            tracks = c("ATAC.bw", "genes.gtf.gz"), transform = "log10") |>
+  chromosomix_goto("chr2:11000000-12000000") |>
+  chromosomix_navigate(oe = TRUE) |>
+  chromosomix_add("model.g3d.gz")
+```
+
+In Shiny, `chromosomixOutput()`, `renderChromosomix()` and `chromosomixProxy()` drive a view on screen,
+and `input$<id>_state` follows the reader. `system.file("examples", package = "chromosomix")` holds an
+R Markdown page of the yeast demo and a Shiny app driving it.
 
 ## Using the chromosomix analysis CLI
 
-Thirty-one analysis tasks can be executed programatically through the CLI, with the same parameters and producing the same results as the Analysis panel interactively. This can be used in a pipeline or a script.
+Over thirty analysis tasks can be executed programatically through the CLI, with the same parameters and producing the same results as the Analysis panel interactively. This can be used in a pipeline or a script.
 
 ### Installing the CLI
 
@@ -165,6 +207,14 @@ chromosomix tasks
 ##       --matrix (default true)  draw the square
 ##       --no-balance  read the raw counts (balanced by default where the file carries weights)
 ## 
+## cellqc
+##     Per-cell contact QC
+##       --cells (default "all")  cells
+##       --nearFrom (default 23000)  near band, from
+##       --nearTo (default 2000000)  near band, to
+##       --mitoticTo (default 12000000)  mitotic band, to
+##       --no-balance  read the raw counts (balanced by default where the file carries weights)
+## 
 ## expected
 ##     Contact probability P(s)
 ##       --scope (default "contig")  scope
@@ -173,10 +223,15 @@ chromosomix tasks
 ##       --factor (default 1.1)  log bins per step
 ##       --minSeparation (default 2)  minimum separation
 ##       --derivative (default true)  log-log slope
-##       --stranded (default false)  four orientations
-##       --pairsUri  .pairs file
 ##       --no-balance  read the raw counts (balanced by default where the file carries weights)
 ## 
+## pairsqc
+##     Pairs QC
+##       --pairs  .pairs file
+##       --reference  reference
+##       --enzyme (default "none")  enzyme
+##       --sites (default "")  sites
+## 
 ## scalogram
 ##     Scalogram
 ##       --quantiles (default "standard")  quantiles
@@ -251,6 +306,18 @@ chromosomix tasks
 ##       --armWindow (default 50)  window along the arms
 ##       --no-balance  read the raw counts (balanced by default where the file carries weights)
 ## 
+## loops
+##     Loop calling
+##       --sequences  sequences
+##       --outer (default 5)  outer half-width
+##       --inner (default 2)  inner half-width
+##       --maxSeparation (default 1000)  furthest from the diagonal
+##       --fdr (default 0.1)  false discovery rate
+##       --lambdaBins (default 40)  λ-chunks
+##       --clusteringRadius (default 2)  clustering radius
+##       --filter (default true)  enrichment filter
+##       --no-balance  read the raw counts (balanced by default where the file carries weights)
+## 
 ## chromosight
 ##     Pattern detection
 ##       --pattern (default "loops")  pattern
@@ -267,6 +334,14 @@ chromosomix tasks
 ##       --maxQvalue (default 1)  report down to a q-value of
 ##       --no-balance  read the raw counts (balanced by default where the file carries weights)
 ## 
+## hicrep
+##     Reproducibility
+##       --against <file>  The other maps to compare with; repeat per map
+##       --sequences  sequences
+##       --maxDistance (default 500)  furthest separation compared
+##       --smooth (default 1)  smoothing half-width
+##       --no-balance  read the raw counts (balanced by default where the file carries weights)
+## 
 ## apa
 ##     Aggregate peak analysis
 ##       --anchors (default "pairs")  anchors
@@ -304,6 +379,7 @@ chromosomix tasks
 ##       --region  sequences
 ##       --alpha (default 3)  exponent α
 ##       --weights (default "contacts")  weight pairs by
+##       --seed (default 0)  seed
 ##       --iterations (default 150)  refinement steps
 ##       --no-balance  read the raw counts (balanced by default where the file carries weights)
 ## 
@@ -329,6 +405,13 @@ chromosomix tasks
 ##       --mismatches (default 0)  mismatches
 ##       --limit (default 500000)  most sites
 ## 
+## restriction
+##     Restriction sites
+##       --sequence  reference
+##       --sequences  sequences
+##       --enzyme (default "none")  enzyme
+##       --sites (default "")  sites
+## 
 ## peaks
 ##     Peak calling
 ##       --signal  track
@@ -457,8 +540,16 @@ chromosomix tasks
 
 Each analysis is held to the package it reproduces — cooltools, coolpuppy, chromosight, deepTools,
 bedtools and a dozen others — on public data, command for command, in a notebook rendered with every
-release: <https://chromosomix.app/validation/>, and how long each command takes beside its
-reference, <https://chromosomix.app/timings/>.
+release: <https://chromosomix.app/validation/>, with every number its figures draw in one table beside
+it, <https://chromosomix.app/validation/validation.txt>; and how long each command takes beside its
+reference, <https://chromosomix.app/timings/>. A third notebook holds the app to this command: ten
+tasks run in the app, in a headless browser, then from the line its console wrote for each, and every
+file the two wrote compared, <https://chromosomix.app/parity/>. And what the commands cost on a machine
+like yours — the time and the peak memory of 23 analyses run genome-wide, up to a 1.08-billion-contact
+human map — is <https://chromosomix.app/laptop/>, rendered on a laptop and committed as
+`notebooks/laptop.html`; render `notebooks/laptop.qmd` with Quarto from a clone to measure your own: it
+downloads the data, runs each analysis once and tabulates them. The app lists all four in
+**About → Notebooks**, each with the one line of what it found.
 
 Every task writes its results where `--out` points, each named after the file it came from up to
 its first dot — `sample.coverage.coverage-total.tsv`. Two files of one command that would share a
@@ -471,11 +562,14 @@ value after it: `--stranded` turns it on, `--stranded=false` or `--no-stranded`
 is none of its values is refused before any file is opened, with exit 2 and the values there are,
 and so is a number that is not one.
 
-Files are read from disk. A file named by an http or https address — the map, or what `--against`,
-`--sequence`, `--tracks`, `--features` or `--pairs` names — is downloaded first, whole, because the
-reader's range requests are synchronous, which a browser worker has and Node does not: it goes into
-`$TMPDIR`, with its progress on stderr, and is removed when the command ends. For several commands
-over one large remote file, download it once yourself and name the copy.
+Files are read from disk or from where they are published. A map named by an http or https address —
+a cooler, an `.mcool` rung, a `.hic`, an `.scool` or a `.pairs`, as the positional or after
+`--against` — and a bigWig, a `.2bit` or an indexed FASTA after `--tracks` or `--sequence` is read
+**by range**, as the app reads it: one rung of a seven-gigabyte `.mcool` costs that rung, and nothing
+is written to disk; a bedGraph or a wig after `--tracks` is read in one request. What is read whole
+from a copy — a `--features` or `--pairs` set, a HiC-Pro matrix, a workspace — and any file whose host
+will not serve a range is downloaded into `$TMPDIR` instead, with its progress on stderr, and removed
+when the command ends.
 
 ```bash
 chromosomix insulation *.mcool --sequences II --out results/
@@ -500,6 +594,13 @@ chromosomix apa sample.cool --features domains.bed --anchors features \
 chromosomix motif sample.cool --sequence ref.fa.gz --pattern GGTCANNNTGACC --mismatches 1 --out results/
 chromosomix overlap sample.cool --features peaks.bed --features boundaries.bed --a peaks.bed --b boundaries.bed
 
+# A library's quality from its contact list: the pairtools counts as figures,
+# the four orientations' P(s), and — with a reference and an enzyme — pair
+# classes, distance to the cut, fragments seen and coverage against sites.
+# Restriction sites alone digests the reference into a lane of cuts.
+chromosomix pairsqc sample.pairs.gz --bin-size 10000 --sequence ref.fa.gz --enzyme DpnII --out qc/
+chromosomix restriction sample.cool --sequence ref.fa.gz --enzyme Arima --out results/
+
 # The tasks that read tracks take them as --tracks: a bigWig, a bedGraph, a
 # wig, or a FASTA read as GC content. Repeat the flag or separate with commas.
 chromosomix metaplot sample.cool --tracks ATAC.bw --features peaks.narrowPeak --flank 2000 --out results/
@@ -534,7 +635,8 @@ chromosomix circular ecoli.mcool::/resolutions/5000
 chromosomix interarm ecoli.mcool::/resolutions/5000 --circular NC_000913.3 \
   --origin NC_000913.3:3925744 --out results/
 
-# A 3D model built from the map, written as a bed3d the viewer opens beside it.
+# A 3D model built from the map, written as a bed3d the viewer opens beside it,
+# and as a PDB for PyMOL.
 chromosomix reconstruct3d ecoli.mcool::/resolutions/20000 --out results/
 
 # What the session set from the picture, as flags: the score column a
@@ -548,6 +650,10 @@ chromosomix insulation sample.metacool --value balanced_by_depth --circular pCEN
 chromosomix coverage Hum53.pairs.gz --bin-size 5000 --rungs 4
 
 chromosomix replay workspace.chromosomix.json --over data/*.hic
+
+# What `pairtools stats` writes for a contact list, counted by the pass the
+# app's import makes, in pairtools' own format — for MultiQC.
+chromosomix stats sample.pairs.gz --out qc/
 ```
 
 ### Recovering the commands you ran
@@ -558,15 +664,25 @@ The log is kept in your own browser for seven days or five hundred lines, whiche
 
 ## Licence
 
-chromosomix is released under the MIT licence (`LICENSE`).
+Copyright © 2026 Jacques Serizay.
+
+chromosomix is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. It is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License, in [`LICENSE`](LICENSE), for more details.
+
+It is under the GPL because its peak calling is ported from yapc (GPL-3.0) and the IDR test it runs across replicates from IDR (GPL-2.0 or later), and a translation of GPL code can only be distributed under the GPL.
+
+Every build also carries the work of other projects, each under its own licence, and the app names them in **About → Licences**:
+
+- **Libraries compiled into the bundles:** three.js (MIT); echarts (Apache-2.0) with zrender (BSD-3-Clause) and tslib (0BSD); h5wasm (NIST's notice, with the BSD-style licence of the HDF5 library inside it); and `@gmod/bbi` with its dependencies `generic-filehandle2`, `@gmod/abortable-promise-cache` and `@jbrowse/quick-lru` (all MIT).
+- **Code ported from reference implementations**, where an analysis follows its reference function for function so that it gives the same numbers: yapc (GPL-3.0) and IDR (GPL-2.0 or later) for peak calling; cooltools (MIT) for loop calling and insulation's peak prominence; cooler (BSD-3-Clause) for region strings; chromosight (MIT) for pattern detection; SciPy (BSD-3-Clause) with the cephes functions its `xsf` carries for the Poisson tail and `fminbound`; NumPy (BSD-3-Clause, and NCSA or BSD-3-Clause for `numpy.random`) for its pairwise sums and its binomial and hypergeometric samplers; scikit-learn (BSD-3-Clause) for BIRCH; Boost-HiC (MIT); and fdlibm (Sun Microsystems' notice) for the error function.
+- **Material copied into the source by a generator:** Lucide (ISC), Simple Icons (CC0), chromosight's kernels (MIT), matplotlib's colour tables (the Matplotlib licence) and ColorBrewer's schemes (Apache-2.0).
 
-Every build also carries the work of other projects, each under its own licence. The libraries compiled into the bundles are echarts (Apache-2.0), zrender (BSD-3-Clause), three.js (MIT), h5wasm (NIST's notice, with the BSD-style licence of the HDF5 library inside it), and `@gmod/bbi` with its dependencies (MIT). The source also holds material copied from Lucide (ISC), Simple Icons (CC0), chromosight (MIT), matplotlib (the Matplotlib licence) and ColorBrewer (Apache-2.0). Each build lists what it carries and reproduces every one of those licences in full in `THIRD-PARTY-NOTICES.txt`, beside its `index.html`:
+All of them are compatible with version 3 of the GPL. Each build lists what it carries and reproduces every one of those licences in full, after chromosomix's own, in `THIRD-PARTY-NOTICES.txt`, beside its `index.html`:
 
 - the web app: <https://chromosomix.app/THIRD-PARTY-NOTICES.txt>;
 - the npm package: `dist/` for the viewer, `dist-cli/` for the command;
-- the desktop app: inside the application;
+- the desktop app: inside the application, beside the licences of Electron and Chromium that every installer ships;
 - the standalone HTML page: in a comment at the end of the file.
 
-The upstream texts that are not in an npm package are kept under [`licenses/`](licenses/).
+The upstream texts that are not in an npm package are kept under [`licenses/`](licenses/), and so is the MIT licence `generic-filehandle2` declares but does not ship.
 
 **Trademarks.** The logos on the data store tiles (IGV, 4DN, ENCODE, DNA Zoo, UCSC and Roadmap Epigenomics) are those projects' own published marks, and the GitHub, Google Drive and Dropbox marks in the interface are their owners' trademarks. Each is shown only to name the store or service it stands for. None of them is covered by chromosomix's licence or by the notices above, and their use implies no endorsement of chromosomix by their owners.