@pikaa-ai/pikaa 0.3.1 → 0.3.4

package/skills/exploratory-data-analysis/references/microscopy_imaging_formats.md

@@ -1,189 +0,0 @@
-# Microscopy and Scientific Imaging Formats
-
-**Reviewed:** 2026-07-23
-**Executable scope:** Metadata-only PNG/JPEG and TIFF/OME-TIFF inspection.
-Pixels are never decoded by bundled tools.
-
-## Exact capability matrix
-
-| Format | Bundled inspection | Depth |
-|---|---|---|
-| `.png`, `.jpg`, `.jpeg` | Optional, `pillow==12.3.0` | Width, height, mode, frame count, format, and metadata-entry count |
-| `.tif`, `.tiff` | Optional, `tifffile==2026.7.14` | Bounded page/series structure, axes, shape, dtype class, BigTIFF/OME flags |
-| `.ome.tif`, `.ome.tiff` | Optional, `tifffile==2026.7.14` | Same structural metadata; OME-XML values are not emitted or semantically validated |
-| ND2/CZI/LIF and other vendor microscopy | No | Reference-only vendor/Bio-Formats workflow |
-| DICOM/NIfTI/MRC | No | Reference-only medical/neuro/EM workflow |
-| SVS/NDPI and other whole-slide formats | No | Reference-only WSI workflow |
-| OME-Zarr/Zarr | No | Directory/store formats are outside the regular-file boundary |
-
-No bundled script supports “all Pillow formats” or “all tifffile formats.”
-Only the registered suffixes above are accepted. Unknown formats fail closed.
-
-## Metadata-only safety model
-
-Images and metadata can contain protected health information, accession
-numbers, specimen labels, GPS/EXIF fields, user comments, XML, external
-references, or adversarial text. The inspectors:
-
-- accept only bounded local regular files inside `--root`;
-- reject URLs, traversal, symlinks, special files, and suffix/signature
-  mismatches;
-- reject declared element counts above 100,000,000 and excessive TIFF
-  pages/series;
-- make Pillow decompression-bomb warnings fatal;
-- never call `load()`, `asarray()`, `imread()`, image codecs, or thumbnail
-  generation;
-- report metadata counts and structural facts, not EXIF/tag/OME-XML values;
-- never follow metadata links or embedded instructions; and
-- do not claim full corruption, codec, or semantic validation.
-
-Metadata-only access reduces decompression risk but is not a sandbox. Keep
-libraries pinned and inspect untrusted images in an isolated, resource-limited
-process when risk warrants it.
-
-## PNG and JPEG
-
-Pillow's `Image.open()` is lazy: it identifies the container and reads enough
-header information to construct an image object. The bundled inspector closes
-the object without decoding pixels.
-
-### Interpret carefully
-
-- PNG may be palette, grayscale, RGB/RGBA, 8/16-bit, multi-frame/APNG, or carry
-  textual/profile chunks.
-- JPEG is lossy and normally unsuitable as a quantitative raw measurement
-  source. Repeated saves change pixels.
-- Width/height/mode do not establish bit-depth fidelity, calibration, channel
-  identity, linearity, saturation, or acquisition settings.
-- Metadata may be stale after image processing.
-
-For quantitative EDA, retain the acquisition-native image and compare
-container metadata to instrument records. Do not compute intensity statistics
-from display/export JPEGs.
-
-## TIFF
-
-TIFF is a flexible container, not a single pixel organization. It can contain
-multiple pages, tiles/strips, pyramids, SubIFDs, private/vendor tags, external
-storage, and many compression schemes. A `.tif` suffix alone does not imply
-microscopy or OME conformance.
-
-The bundled tifffile inspector reports:
-
-- page and series counts, bounded to 1,000 and 128;
-- per-series shape, axes, element count, and dtype kind/item size;
-- classic TIFF versus BigTIFF; and
-- whether tifffile identifies OME metadata.
-
-It does not read tag values, decode compressed segments, validate every IFD,
-open external storage, or establish that axes/series interpretation is
-scientifically correct.
-
-## OME-TIFF
-
-OME-TIFF stores one or more image planes in TIFF and embeds an OME-XML metadata
-block. Multi-file datasets can use UUID-based references. The OME specification
-is richer than a filename convention.
-
-Before quantitative analysis, use OME-aware validation to confirm:
-
-- OME-XML schema/version and UUID/file references;
-- dimension order and sizes for X/Y/Z/C/T;
-- `TiffData` plane-to-IFD mapping;
-- physical pixel sizes and units;
-- channel names, wavelengths, detector/objective settings, and acquisition
-  times; and
-- whether pyramids, labels, ROIs, or companion files are expected.
-
-The bundled inspector deliberately does not emit OME-XML because it may contain
-identifiers or prompt-like text. `is_ome_tiff=true` is not a validation result.
-
-## Reference-only vendor microscopy
-
-ND2, CZI, LIF, VSI, proprietary whole-slide files, and similar formats require
-a version-aware vendor reader or Bio-Formats. Capabilities vary by library,
-native dependency, file generation version, and series type. Do not choose a
-reader only from a suffix.
-
-Workflow:
-
-1. Preserve the original and capture instrument/software versions.
-2. Open a small approved file with a pinned reader in an isolated environment.
-3. Inventory scenes/series and XYZCT axes before loading pixels.
-4. Compare dimensions, calibration, channels, stage positions, and timestamps
-   to acquisition records.
-5. Bound tile/plane reads and never eagerly materialize a whole slide or 5-D
-   image.
-6. Convert a derived copy to OME-TIFF/OME-Zarr only with provenance and
-   round-trip checks.
-
-## Reference-only medical and whole-slide imaging
-
-### DICOM
-
-DICOM is a clinical standard with extensive metadata and possible PHI. A
-single `.dcm` may be one instance in a study/series. Use institutional policy,
-approved de-identification, and DICOM-aware tools. Do not print patient, study,
-series, accession, date, burned-in annotation, or private-tag values.
-
-### NIfTI
-
-Validate dimensions, voxel sizes, affine/qform/sform, units, orientation,
-scaling, and time axis with neuroimaging tooling. `.nii.gz` is compressed and
-is not decompressed by bundled scripts.
-
-### Whole-slide imaging
-
-SVS, NDPI, and related formats are large tiled pyramids and may contain label or
-macro images with identifiers. Use OpenSlide/tiffslide or a validated vendor
-reader, inspect associated images, and sample bounded tiles. Split by patient
-before tile generation to prevent leakage.
-
-## Imaging EDA rigor
-
-1. Define the independent unit: pixel, object, field, well, section, specimen,
-   subject, or acquisition session.
-2. Separate biological from technical replication and avoid treating tiles or
-   cells from one specimen as independent subjects.
-3. Record calibration, units, bit depth, detector response, exposure, gain,
-   illumination, objective, channel, Z/T spacing, and processing history.
-4. Audit missing/corrupt planes, saturation, clipping, background, focus,
-   illumination, registration, segmentation, and batch/site effects.
-5. Preserve raw pixels. Do not automatically rescale, denoise, background
-   subtract, discard fields, or remove objects.
-6. Fit normalization, segmentation thresholds, feature selection, and models
-   on training specimens only; split subjects/specimens before tiling.
-7. Report object/field/specimen-level sensitivity, not only pooled pixels.
-8. Do not infer biological mechanism, diagnosis, or treatment effect from
-   descriptive image patterns.
-
-## Pinned optional snapshot
-
-```bash
-uv pip install \
-  "pillow==12.3.0" \
-  "tifffile==2026.7.14" \
-  "numpy==2.5.1"
-```
-
-Pillow 12.3.0 was released 2026-07-01 and requires Python 3.10+.
-tifffile 2026.7.14 was released 2026-07-14 and requires Python 3.12+.
-Imagecodecs is not installed or invoked by the metadata-only inspector.
-
-## Authoritative sources
-
-All links accessed 2026-07-23.
-
-- Pillow, [`Image` module and decompression-bomb protection](https://pillow.readthedocs.io/en/stable/reference/Image.html).
-- [Pillow PyPI](https://pypi.org/project/pillow/), version 12.3.0,
-  released 2026-07-01.
-- [tifffile PyPI](https://pypi.org/project/tifffile/), version 2026.7.14,
-  released 2026-07-14; upstream notes that codecs are required for decoding
-  compressed segments.
-- Library of Congress, [TIFF, Revision 6.0 format description](https://www.loc.gov/preservation/digital/formats/fdd/fdd000022.shtml)
-  and the ITU-hosted [TIFF 6.0 specification](https://www.itu.int/itudoc/itu-t/com16/tiff-fx/docs/tiff6.pdf).
-- OME, [OME-TIFF specification](https://ome-model.readthedocs.io/en/stable/ome-tiff/specification.html).
-- OME, [OME Data Model and File Formats](https://ome-model.readthedocs.io/en/stable/).
-- DICOM Standards Committee, [current DICOM standard](https://www.dicomstandard.org/current).
-- OpenSlide, [supported formats and Python API](https://openslide.org/api/python/).
-- National Academies (2019), [reproducibility and provenance](https://doi.org/10.17226/25303).

package/skills/exploratory-data-analysis/references/proteomics_metabolomics_formats.md

@@ -1,217 +0,0 @@
-# Proteomics and Metabolomics Formats
-
-**Reviewed:** 2026-07-23
-**Executable scope:** No omics-native standard is parsed by bundled scripts.
-Rectangular CSV/TSV result exports can use the general tabular CLIs after the
-schema, units, and missing/censoring codes are confirmed.
-
-## Exact capability boundary
-
-| Format | Bundled native inspection | Status |
-|---|---|---|
-| mzML/mzXML, vendor RAW | No | Reference-only MS tooling; see `spectroscopy_analytical_formats.md` |
-| mzIdentML (`.mzid`, `.mzIdentML`) | No | Reference-only PSI schema/CV-aware tooling |
-| mzTab 1.0 / mzTab-M 2.0 | No | Reference-only version-aware validator; generic TSV parsing is insufficient |
-| pepXML/protXML | No | Reference-only search/inference-aware parser |
-| featureXML/consensusXML/idXML | No | Reference-only OpenMS tooling |
-| Rectangular `.csv`/`.tsv` feature or abundance table | General scripts | Bounded aggregate tabular EDA, no omics semantics |
-| `.h5`/`.hdf5` | Generic metadata only | No payload values or convention validation |
-| `.h5ad`, `.loom` | No semantic support | See bioinformatics reference |
-| Pickled models/results | **Never** | Request non-executable export |
-
-Unknown formats fail closed. No format is identified from free-text metadata or
-content guessing.
-
-## mzML and raw spectra
-
-mzML is a HUPO-PSI standard for spectra/chromatograms; use PSI-aware tooling.
-Vendor RAW extensions are ambiguous and often require vendor libraries or
-conversion. Preserve originals and record converter, version, options, and
-checksums.
-
-For spectral EDA, inventory:
-
-- acquisition method, instrument, polarity, MS levels, scan modes, precursor
-  isolation/activation, resolution, and centroid/profile status;
-- run order, batches, blanks, pooled QC, standards, carryover, drift, and
-  calibration;
-- spectrum/chromatogram counts, retention/mobility ranges, m/z coverage, TIC/
-  BPC, peak counts, and missing/corrupt scans; and
-- processing history, controlled-vocabulary terms, source files, and units.
-
-Do not automatically centroid, denoise, recalibrate, align, peak-pick, or
-discard spectra.
-
-## Identification formats
-
-### mzIdentML
-
-mzIdentML represents peptide/protein identification results, scores, search
-parameters, databases, modifications, and links to spectra using controlled
-vocabularies. Validate the schema and CV mapping with PSI-aware tooling.
-
-Check:
-
-- search engine/version, sequence database/version, decoy strategy, enzyme,
-  tolerances, fixed/variable modifications, and spectrum references;
-- score direction/meaning, rank, charge, mass error, peptide-spectrum matches,
-  peptides, proteins, and protein groups;
-- target/decoy and FDR method at each reported level; and
-- ambiguity from shared peptides, indistinguishable proteins, and inference.
-
-A score threshold is not automatically a validated FDR threshold. Do not
-recompute or reinterpret confidence without the method and decoy design.
-
-### pepXML/protXML
-
-These formats are Trans-Proteomic Pipeline conventions. Use Pyteomics or TPP
-tools with the generating software/version known. Preserve search-engine,
-PeptideProphet/ProteinProphet, modification, decoy, and inference context.
-
-## mzTab and mzTab-M
-
-HUPO-PSI lists:
-
-- mzTab 1.0.0 as the final proteomics release (accepted June 2014); and
-- mzTab-M 2.0.0 as the final metabolomics/small-molecule release (accepted
-  March 2019).
-
-mzTab-M 2.1.0 is listed as draft, not a final standard. Do not silently treat
-it as 2.0.
-
-Although mzTab is tab-delimited, it has section-specific row types, metadata,
-controlled vocabulary, optional columns, and null conventions. The generic
-rectangular TSV scanner is not a validator and will reject legitimate
-non-rectangular section structure. Use the PSI specification/reference
-validator, then export a controlled rectangular analysis table if needed.
-
-## Rectangular quantitative tables
-
-Common outputs contain features/peptides/proteins/metabolites in rows and
-samples in columns, or long-form measurements. Before using general CLIs,
-create a data dictionary that records:
-
-- row entity and identifier namespace/version;
-- sample/subject/specimen, condition, batch, injection order, and QC role;
-- abundance scale (raw intensity, area, count, ratio, normalized/logged);
-- zero, missing, censored, filtered, not-identified, and not-quantified codes;
-- normalization, transformation, imputation, roll-up, and batch correction
-  already applied;
-- internal standards, dilution, LOD/LOQ, blank subtraction, and detection
-  frequency; and
-- peptide-to-protein or feature-to-metabolite ambiguity.
-
-Do not assume zeros are measured zeros. Missingness is often abundance-,
-feature-, batch-, or identification-dependent and may be non-random.
-
-### Safe commands
-
-```bash
-python scripts/tabular_profile.py abundance.csv \
-  --root /approved/project \
-  --missing-token NA \
-  --max-rows 100000
-
-python scripts/missingness_leakage_audit.py abundance.csv \
-  --root /approved/project \
-  --group-column condition \
-  --entity-column subject_id \
-  --split-column split \
-  --time-column acquisition_time
-
-python scripts/distribution_sensitivity.py abundance.csv \
-  --root /approved/project \
-  --column intensity
-```
-
-The column arguments are exact local identifiers; output tokenizes them unless
-`--reveal-identifiers` is explicit. Values and subject/sample identifiers are
-not emitted.
-
-## Missingness, censoring, and limits
-
-Separate at least:
-
-- structurally absent/not applicable;
-- not detected;
-- detected below quantitation;
-- failed identification or confidence filter;
-- failed extraction/integration;
-- filtered during preprocessing;
-- saturated/above range; and
-- genuinely missing metadata.
-
-Preserve flags and limits in separate columns. Do not automatically replace
-non-detects with zero, half-minimum, LOD/2, or a random draw. Report missing/
-censored fractions by feature, sample, condition, batch, and run order, and
-compare conclusions across scientifically justified handling strategies.
-
-## Distribution and outlier sensitivity
-
-For abundance tables:
-
-- inspect sample totals/detection rates and feature detection frequency;
-- compare raw-scale and scientifically justified log/variance-stabilizing
-  diagnostics without overwriting raw data;
-- compare mean/SD with median/IQR/MAD and leave-one-sample/batch sensitivity;
-- investigate outliers against blank/QC/internal-standard performance,
-  acquisition order, contamination, carryover, and sample handling; and
-- preserve excluded samples/features with reasons and show sensitivity.
-
-PCA/clustering can reveal structure but is not proof of batch, identity, or
-biological separation. Fit transformations and feature selection on training
-data only.
-
-## Design, leakage, and inference
-
-1. Define the independent experimental unit; technical injections, spectra,
-   peptides, or features are usually not independent subjects.
-2. Preserve subject/sample pairing, repeated measures, batches, sites, and
-   acquisition order.
-3. Split by subject/specimen/batch/time before normalization, imputation,
-   feature selection, PCA, or model tuning.
-4. Ensure spectra/peptides/features derived from one sample do not cross
-   train/test boundaries.
-5. Distinguish QC, blank, pooled, calibrator, and biological samples.
-6. Treat identification/feature discovery and differential testing as separate
-   selection stages when assessing error rates.
-7. Define the hypothesis family (features, contrasts, endpoints) and report
-   effect sizes/uncertainty plus an appropriate FWER/FDR method.
-8. Label discoveries from EDA as exploratory and confirm on independent data.
-9. Do not make biomarker, diagnostic, mechanism, exposure, or causal claims
-   from descriptive patterns.
-
-## HDF5 and related containers
-
-The generic HDF5 inspector reports only bounded hierarchy/dataset metadata. It
-does not:
-
-- read spectra, abundance matrices, annotations, or attributes;
-- follow soft/external links or external dataset storage;
-- validate mzMLb, H5AD, Loom, or vendor schemas; or
-- invoke filter plugins for dataset decompression.
-
-Use the convention's official reader/validator for semantics. NumPy object
-arrays and all pickle-based objects are rejected.
-
-## Authoritative sources
-
-All links accessed 2026-07-23.
-
-- HUPO-PSI, [mzML specification/status](https://www.psidev.info/mzml)
-  (mzML 1.1.0 long-term stable).
-- HUPO-PSI, [mzIdentML](https://www.psidev.info/mzidentml).
-- HUPO-PSI, [mzTab specifications](https://www.psidev.info/mztab-specifications)
-  (page updated 2024-04-19; mzTab 1.0.0 final, mzTab-M 2.0.0 final,
-  mzTab-M 2.1.0 draft).
-- HUPO-PSI, [mzTab repository and released specifications](https://github.com/HUPO-PSI/mzTab).
-- Hoffmann et al. (2019), [mzTab-M 2.0](https://doi.org/10.1021/acs.analchem.8b04310),
-  published 2019-01-28.
-- Pyteomics, [formats documentation](https://pyteomics.readthedocs.io/en/latest/).
-- OpenMS, [recognized file types](https://openms.de/documentation/structOpenMS_1_1FileTypes.html).
-- US EPA, [Detection Limits Best Practices Guide](https://www.epa.gov/system/files/documents/2025-09/wqxdetectionlimitsbestpracticesguide_final.pdf),
-  dated August 2025.
-- FDA/ICH E9(R1), [sensitivity analysis guidance](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/e9r1-statistical-principles-clinical-trials-addendum-estimands-and-sensitivity-analysis-clinical),
-  final May 2021.
-- Benjamini and Hochberg (1995), [FDR control](https://academic.oup.com/jrsssb/article/57/1/289/7035855).
-- scikit-learn, [data leakage guidance](https://scikit-learn.org/stable/common_pitfalls.html).

package/skills/exploratory-data-analysis/references/spectroscopy_analytical_formats.md

@@ -1,191 +0,0 @@
-# Spectroscopy and Analytical Chemistry Formats
-
-**Reviewed:** 2026-07-23
-**Executable scope:** No spectroscopy-native parser is bundled. General
-CSV/TSV/JSON/NumPy/HDF5 inspectors apply only when a file is truly one of those
-registered formats and do not add spectroscopy semantics.
-
-## Capability boundary
-
-| Format | Bundled native inspection | Status |
-|---|---|---|
-| mzML/mzXML, MGF | No | Reference-only MS tooling |
-| JCAMP-DX (`.jdx`, `.dx`) | No | Reference-only technique/version-aware parser |
-| SPC and vendor spectroscopy binaries | No | Reference-only producer-specific parser |
-| Vendor `.raw`, `.d`, `.fid`, `.dat`, `.out` | No | Ambiguous suffix/path; producer and format must be confirmed |
-| CSV/TSV exports | General tabular scripts | Bounded aggregates only after delimiter, units, axes, and missing codes are confirmed |
-| NPY/NPZ/HDF5 exports | General container scripts | Structural/bounded numeric inspection only; no instrument semantics |
-
-Unknown formats fail closed. Directory-based acquisitions are rejected by the
-regular-file CLIs. No archive or compressed stream is unpacked.
-
-## mzML and related mass-spectrometry formats
-
-HUPO-PSI identifies mzML 1.1.0 as the long-term stable format; its index schema
-and controlled vocabulary continue to receive compatible updates. mzML is XML
-with encoded binary arrays and controlled-vocabulary metadata. A generic XML
-parser is not sufficient.
-
-Use pinned pymzML, Pyteomics, OpenMS, or ProteoWizard tooling and inspect:
-
-- schema/version, controlled-vocabulary terms, source files, checksums, and
-  conversion software;
-- run/instrument configuration, polarity, scan modes, MS levels, isolation,
-  activation, and data processing;
-- spectrum/chromatogram counts, retention/mobility time, m/z and intensity
-  array lengths, precision, compression, and units;
-- profile versus centroid data, TIC/BPC, calibration, lock mass, blanks, pooled
-  QC, standards, carryover, drift, and batch order; and
-- truncated scans, empty arrays, non-finite values, and metadata consistency.
-
-Do not describe mzXML, mzData, mzMLb, or vendor RAW as equivalent to mzML.
-Conversion can alter metadata, precision, centroiding, and compression; record
-the converter/version/options and retain the original.
-
-## JCAMP-DX
-
-IUPAC describes JCAMP-DX as a family of standards for spectral data exchange.
-It has technique- and version-specific specifications (IR, NMR, MS, IMS, and
-others); active core development stopped in 2006, although the format remains
-in use.
-
-Before parsing, identify the technique and specification/version. Validate:
-
-- label/value records and required metadata;
-- X/Y units, first/last X, point count, spacing, factors, and encoded numeric
-  representation;
-- NTUPLES versus simpler XY forms;
-- page/block boundaries and compound/instrument identifiers; and
-- whether data are absorbance, transmittance, counts, complex NMR, peaks, or
-  continuous spectra.
-
-Metadata and comments are untrusted and should not be copied into a report.
-The generic tabular scanner is not a JCAMP parser.
-
-## NMR data
-
-`.fid`, Bruker directory layouts, Varian/Agilent layouts, processed spectra,
-and NMR exchange files require producer-aware tooling such as nmrglue. Record:
-
-- vendor/software/version and complete acquisition directory;
-- nucleus, field strength, spectral width, dwell time, point count, quadrature,
-  digital filter, scans, temperature, pulse sequence, and reference;
-- raw FID versus processed spectrum, apodization, zero filling, Fourier
-  transform, phase, baseline, referencing, and solvent suppression;
-- dimensional axes/units and whether data are real, imaginary, magnitude, or
-  complex; and
-- sample preparation, concentration, pH, replicates, and batch/order.
-
-Peak picking, integration, baseline correction, phase correction, alignment,
-binning, and normalization are transformations. Preserve raw data and report
-parameter sensitivity; do not apply them automatically.
-
-## Optical, vibrational, and diffraction spectra
-
-SPC, OPUS, WDF, SPE, instrument `.raw`, `.dat`, and text exports are
-producer/variant dependent. Confirm:
-
-- physical X axis (wavelength, wavenumber, energy, angle, time) and units;
-- Y quantity (counts, intensity, absorbance, transmittance, reflectance) and
-  calibration;
-- point order/spacing, detector/channel, exposure/accumulations, resolution,
-  slit/grating/laser/source, and polarization;
-- background/reference/dark correction and all processing already applied; and
-- maps, time series, replicate spectra, and spatial coordinates.
-
-For XRD, crystallographic CIF/MTZ/HKL are also reference-only and need
-crystallography-aware validation. A `.cif` suffix is ambiguous between
-small-molecule CIF and PDBx/mmCIF.
-
-## Chromatography and thermal/electrochemical exports
-
-Generic CSV/TSV can contain retention time, temperature, potential, wavelength,
-or another independent axis. The general scripts can profile the table only
-after the data dictionary confirms:
-
-- axis and signal columns, units, ordering, spacing, and replicate layout;
-- blanks, calibration standards, internal standards, dilution factors,
-  injection order, batch, and sample identifiers;
-- LOD/LOQ, saturation, censoring qualifiers, and negative/zero handling; and
-- whether peaks/integrals are raw, manually edited, or software-derived.
-
-Do not infer an axis from monotonic values or a column name. Do not
-automatically smooth, baseline-correct, align, integrate, normalize, subtract
-blanks, or delete peaks.
-
-## Safe bounded tabular workflow
-
-For an approved values-only export:
-
-```bash
-python scripts/tabular_profile.py spectrum.csv \
-  --root /approved/project \
-  --max-rows 100000
-
-python scripts/missingness_leakage_audit.py spectrum.csv \
-  --root /approved/project \
-  --group-column sample_group \
-  --entity-column sample_id \
-  --split-column split \
-  --time-column acquisition_time
-
-python scripts/distribution_sensitivity.py spectrum.csv \
-  --root /approved/project \
-  --column intensity
-```
-
-Use only pseudonymous column roles in shared commands/logs. The outputs contain
-aggregates and tokens, not spectra or identifiers.
-
-## Analytical EDA rigor
-
-1. Define the independent unit: scan, injection, spectrum, sample, batch,
-   subject, instrument, site, or experiment.
-2. Preserve raw acquisition files and processing audit trails.
-3. Record calibration, units, standards, blanks, internal standards,
-   acquisition order, maintenance, software, and method versions.
-4. Keep non-detects, below-LOQ values, saturation, missing scans, failed QC, and
-   true zeros distinct. Preserve qualifier and limit fields.
-5. Compare raw and processed summaries and sensitivity to baseline, smoothing,
-   peak picking, alignment, integration, normalization, and transformations.
-6. Investigate outliers against calibration, instrument state, carryover, and
-   sample handling; do not delete automatically.
-7. Split independent samples/batches/time before learned preprocessing. Never
-   fit normalization or feature selection on test data.
-8. Account for repeated spectra, technical replicates, correlated wavelengths/
-   peaks, and many comparisons.
-9. Label discovered peaks/patterns as exploratory and confirm independently.
-10. Do not make identity, purity, mechanism, exposure, diagnostic, or causal
-    claims from EDA alone.
-
-## Detection limits and censoring
-
-EPA guidance treats non-detects/over-detects as censored observations carrying
-partial information and recommends preserving detection condition and limit
-type rather than forcing a numeric result. Apply the same principle to
-instrumental assays:
-
-- keep measured value, qualifier, limit type, and limit value in distinct
-  fields;
-- do not replace censored values automatically with zero, LOD/2, or LOQ;
-- summarize the censoring fraction by group/batch/time;
-- choose a model appropriate to censoring and scientific design; and
-- report sensitivity to plausible assumptions.
-
-## Authoritative sources
-
-All links accessed 2026-07-23.
-
-- HUPO-PSI, [mzML specification/status](https://www.psidev.info/mzml)
-  (mzML 1.1.0 long-term stable; current schema/CV links and 2026 IM-MS/DIA
-  proposal status).
-- HUPO-PSI, [mzML GitHub specification repository](https://github.com/HUPO-PSI/mzML).
-- IUPAC, [JCAMP-DX digital standard family](https://iupac.org/what-we-do/digital-standards/jcamp-dx/)
-  (page dated 2021-08-03; finalized technique-specific standards).
-- IUPAC, [JCAMP-DX 5.01 recommendation](https://doi.org/10.1351/pac199971081549).
-- nmrglue, [current documentation](https://nmrglue.readthedocs.io/en/latest/).
-- US EPA, [Detection Limits Best Practices Guide](https://www.epa.gov/system/files/documents/2025-09/wqxdetectionlimitsbestpracticesguide_final.pdf),
-  dated August 2025.
-- NIST/SEMATECH, [Exploratory Data Analysis](https://www.itl.nist.gov/div898/handbook/eda/eda.htm).
-- FDA/ICH E9(R1), [estimands and sensitivity analysis](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/e9r1-statistical-principles-clinical-trials-addendum-estimands-and-sensitivity-analysis-clinical),
-  final guidance May 2021.

package/skills/exploratory-data-analysis/scripts/__init__.py

@@ -1 +0,0 @@
-"""Bounded, local-only helper CLIs for exploratory-data-analysis."""