ipaapi 1.0.0__py3-none-any.whl

ipaapi-1.0.0.dist-info/METADATA

@@ -0,0 +1,833 @@
+Metadata-Version: 2.5
+Name: ipaapi
+Version: 1.0.0
+Summary: Command-line client for QIAGEN Ingenuity Pathway Analysis: map arbitrary column layouts, submit datasets in bulk, and resume cleanly when the analysis allowance runs out.
+Author-email: Ken Jones <Ken.Jones@bioinformaticsolutions.com>
+License: MIT License
+        
+        Copyright (c) 2026 Ken Jones
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Keywords: bioinformatics,ingenuity,ipa,pathway,qiagen
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.9
+Requires-Dist: pandas>=1.5
+Requires-Dist: requests-oauthlib>=1.3
+Requires-Dist: requests>=2.28
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Provides-Extra: progress
+Requires-Dist: tqdm>=4.64; extra == 'progress'
+Description-Content-Type: text/markdown
+
+# ipaapi
+
+A Python package and command-line tool for QIAGEN Ingenuity Pathway Analysis
+(IPA). Upload datasets into an IPA project using an explicit column mapping,
+submit them for analysis, and track the results — one file or several hundred.
+
+Free software (MIT). Built on QIAGEN's `python-api-demo` example code — **not
+an official QIAGEN product**, and not endorsed by QIAGEN.
+
+```bash
+ipaapi submit ~/data --ID 1:hugo --FC 4:logratio --skip-rows 1 \
+    --reference-set ipkb --project MyStudy --pattern _DEG
+```
+
+---
+
+## Contents
+
+- [Why this exists](#why-this-exists)
+- [Installation](#installation)
+- [Quick start](#quick-start)
+- [How the mapping works](#how-the-mapping-works)
+- [Command-line reference](#command-line-reference)
+- [Recipes](#recipes)
+- [Working with IPA](#working-with-ipa) — the undocumented parts
+- [Authentication](#authentication)
+- [Python API](#python-api)
+- [Troubleshooting](#troubleshooting)
+- [How a submission is encoded](#how-a-submission-is-encoded)
+- [Development](#development)
+- [Contributing](#contributing)
+- [Licence](#licence)
+
+---
+
+## Why this exists
+
+QIAGEN's demo script works, but assumes a rigid file layout: the gene ID in
+column 0, then `n_observations × n_measurements` value columns in strict
+repeating order, every observation carrying the same measurement types in the
+same positions. Real analysis output rarely looks like that.
+
+This package replaces that assumption with a declaration. You name the
+identifier column and describe each observation as a set of
+`(column, measurement type)` pairs. Columns may be in any order, named
+anything, and interleaved with columns the analysis should ignore.
+
+It also fixes a number of things the demo got wrong or left out — see
+[Differences from the demo](#differences-from-the-demo).
+
+---
+
+## Installation
+
+```bash
+git clone <this-repo> ipaapi && cd ipaapi
+pip install -e .
+```
+
+Or build and install a wheel:
+
+```bash
+python3 -m pip wheel . --no-deps -w dist
+python3 -m pip install dist/ipaapi-*.whl
+```
+
+Requires Python 3.9+, `requests`, `requests-oauthlib`, `pandas`.
+
+Confirm what you're running — this reports the version, the install location,
+and whether it's an editable checkout rather than a built wheel:
+
+```bash
+$ ipaapi --version
+ipaapi 1.0.0
+installed at /usr/lib/python3.11/site-packages/ipaapi
+python 3.11.5 (/usr/bin/python3)
+```
+
+---
+
+## Quick start
+
+Say your file looks like this — a comment line, then a header, then data:
+
+```
+# generated by pipeline v3
+Gene,Common_name,Control_mean,Treatment_mean,Fold_change,P-value,Q-value
+ENSG00000229807,XIST,4.21,2.88,-1.33,0.001,0.02
+```
+
+Column positions are **0-based** and counted from the *header* row:
+
+```
+0 Gene   1 Common_name   2 Control_mean   3 Treatment_mean   4 Fold_change   5 P-value   6 Q-value
+```
+
+Check the mapping without contacting IPA:
+
+```bash
+ipaapi validate results.csv --ID 1:hugo --FC 4:logratio --skip-rows 1
+```
+
+```
+results: 2,338 rows
+gene id: 'Common_name' (hugo)
+observations: 1
+  results:
+    'Fold_change' -> Log Ratio
+
+     Common_name  Fold_change
+0           XIST        -1.33
+...
+1 file valid. Nothing was uploaded.
+```
+
+When that looks right, submit:
+
+```bash
+ipaapi submit results.csv --ID 1:hugo --FC 4:logratio --skip-rows 1 \
+    --reference-set ipkb --project MyStudy
+```
+
+```
+submitted results: 43595871
+
+Submitted 1 analysis.
+Analyses are running in IPA. Check on them with:
+  ipaapi status 43595871
+  ipaapi report 43595871
+Recorded in ~/.local/state/ipaapi/submissions.tsv -- see 'ipaapi history'.
+```
+
+---
+
+## How the mapping works
+
+Three ideas, and they mirror how IPA thinks about a dataset.
+
+**Measurement** — one value column: which column, what kind of number it holds,
+and an optional cutoff.
+
+**Observation** — a named sample or contrast, and the measurement columns
+belonging to it. One analysis is created per observation.
+
+**ColumnMapping** — the identifier column, its type, and the observations.
+
+```python
+ColumnMapping(
+    gene_id_column="Common_name",
+    gene_id_type="hugo",
+    observations=[
+        Observation("drug A vs ctrl", [
+            Measurement("A_log2fc", MeasurementType.LOG_RATIO),
+            Measurement("A_padj",   MeasurementType.FALSE_DISCOVERY, cutoff=0.05),
+        ]),
+        Observation("drug B vs ctrl", [
+            # declared in a different order on purpose -- this is fine
+            Measurement("B_padj",   MeasurementType.FALSE_DISCOVERY, cutoff=0.05),
+            Measurement("B_log2fc", MeasurementType.LOG_RATIO),
+        ]),
+    ],
+)
+```
+
+**One constraint is imposed by IPA, not by this package.** The wire format
+declares `expvaltype`, `expvaltype2`, … and `cutoff`, `cutoff2`, … *once for the
+whole submission*, then supplies per-observation column names against those
+slots. So every observation must contribute exactly one column per measurement
+type, and a given type carries one cutoff throughout. Both are checked before
+anything is uploaded, with an error that explains why.
+
+Within those limits, order and naming are free — observations declared in
+different column orders are normalised automatically.
+
+Everything is validated against the actual data before upload: columns exist,
+none is claimed twice, and values fall in the range IPA expects for their type.
+That last check matters more than it looks — see
+[measurement types](#measurement-types).
+
+---
+
+## Command-line reference
+
+```
+ipaapi validate   check a mapping against file(s) without uploading
+ipaapi submit     upload into a project and start analyses
+ipaapi status     check the state of existing analyses
+ipaapi report     print IPA Interpret links
+ipaapi history    list analyses submitted through this tool
+```
+
+### Mapping arguments
+
+Used by `validate` and `submit`.
+
+| Flag | Form | Meaning |
+| --- | --- | --- |
+| `PATH` | positional | a data file, or a directory to search |
+| `--ID` | `COLUMN:TYPE` | 0-based identifier column and its IPA gene ID type. May be given twice — see [two identifier columns](#two-identifier-columns) |
+| `--FC` | `COLUMN:TYPE[:CUTOFF]` | 0-based value column, [measurement type](#measurement-types), optional cutoff |
+| `--skip-rows` | `N` | discard N lines above the header row |
+| `--sep` | `CHAR` | field delimiter (sniffed from the header line by default) |
+| `--pattern` | `TEXT` | when PATH is a directory: substring or glob selecting files |
+| `--recursive` | flag | search subdirectories too |
+| `--observation` | `NAME` | observation name in IPA (default: the filename). Single file only |
+| `--no-range-check` | flag | skip the value-range validation |
+| `--list-id-types` | flag | print all 33 gene ID types and exit |
+
+### `submit`
+
+| Flag | Default | Meaning |
+| --- | --- | --- |
+| `--project` | *required* | destination IPA project. **Created if it doesn't exist**, so a typo silently makes a new one |
+| `--reference-set` | `omit` | `ipkb`, `dataset`, or `omit`. See [the reference set](#the-reference-set) |
+| `--wait` | off | poll until analyses finish and print report links |
+| `--interval` / `--timeout` | 30s / 3600s | polling, only with `--wait` |
+| `--dry-run` | off | validate and stop before login |
+| `--analysis-name` / `--dataset-name` | filename | single file only |
+| `--log-file` | `~/.local/state/ipaapi/submissions.tsv` | submission log |
+
+### Authentication arguments
+
+Used by every command that contacts IPA.
+
+| Flag | Meaning |
+| --- | --- |
+| `--no-cache` | ignore any cached token |
+| `--token-file` | token cache path (default `~/.cache/ipaapi/token.json`) |
+| `--application-name` | `applicationname` IPA scopes the session to (default `PythonAPI`) |
+| `--browser` | browser to launch for login, e.g. `firefox` |
+
+### `history`
+
+| Flag | Meaning |
+| --- | --- |
+| `--project` / `--since` / `--limit` | filters |
+| `--status` | look up each analysis's current state (requires login) |
+| `--log-file` | read a different log |
+
+### Environment variables
+
+| Variable | Purpose |
+| --- | --- |
+| `IPAAPI_TOKEN_FILE` | token cache location — set this if `$HOME` isn't writable |
+| `IPAAPI_LOG_FILE` | submission log location |
+
+---
+
+## Recipes
+
+### Many files, one analysis each
+
+```bash
+ipaapi submit ~/data --pattern _DEG --ID 1:hugo --FC 4:logratio \
+    --skip-rows 1 --reference-set ipkb --project Study1
+```
+
+`--pattern` takes plain text or a glob. Text with no `*`, `?` or `[` matches as
+a **substring**, so `--pattern SampleA` finds `SampleA_DEG.txt` and
+`SampleA_raw.tsv`. With no `--pattern`, `*.txt`/`*.tsv`/`*.csv` are searched.
+Hidden files are skipped and results sorted, so run order is predictable.
+
+Every matched file must fit the same `--ID`/`--FC` positions.
+
+### Files are filed as they're processed
+
+When `PATH` is a directory, each file moves as its outcome becomes known:
+
+| Outcome | Destination |
+| --- | --- |
+| IPA accepted it | `submitted/` |
+| The file is at fault | `failed/`, with a `.error.txt` note beside it |
+| Allowance exhausted, or IPA declined | left in place for the next run |
+
+```
+submitted SampleA_DEG: 43595001
+submitted SampleB_DEG: 43595002
+
+Allowance exhausted while submitting SampleC_DEG:
+REJECTED: the analysis allowance appears to be exhausted.
+IPA said: 'Unable to run analysis: Analysis limit exceeded'
+
+2 file(s) moved to submitted/
+2 file(s) left in place for the next run
+Re-run the same command later; the files left in place are exactly the ones
+still to do.
+```
+
+The source directory shrinks to exactly the work outstanding, and re-running the
+identical command resumes. `submitted/` and `failed/` are excluded from
+discovery, so a run can't re-ingest its own output.
+
+Nothing is moved when the *command* is at fault — a bad `--ID` type or a mapping
+that fails every file leaves the directory untouched, because that's a mistake
+to fix rather than data to quarantine. Single-file submits are never moved.
+
+### Draining a backlog against a daily allowance
+
+Because a stopped run resumes cleanly, this is safe to leave unattended:
+
+```cron
+0 6 * * * cd ~/data && ipaapi submit ./ --pattern _DEG --ID 1:hugo \
+    --FC 4:logratio --skip-rows 1 --reference-set ipkb --project Study1 \
+    >> ~/ipaapi-cron.log 2>&1
+```
+
+It submits until the allowance runs out, files what succeeded, leaves the rest.
+Check the log after the first few runs — a cron job whose *refresh* token has
+expired fails into that file rather than prompting anyone.
+
+### Finding analysis IDs later
+
+IPA's API cannot list the analyses on an account, so the package keeps its own
+log — every submission appends a timestamped row.
+
+```bash
+ipaapi history
+ipaapi history --project Study1 --since 2026-08-01
+ipaapi history --status
+```
+
+```
+2026-08-05T08:35:53-06:00  43595039  Study1  SampleA_DEG
+2026-08-05T08:35:53-06:00  43595041  Study1  SampleB_DEG
+
+2 submission(s). Report links: ipaapi report 43595039 43595041
+```
+
+Plain TSV — grep it, open it in a spreadsheet. It only covers submissions made
+through this tool; anything submitted from the IPA client won't appear.
+
+### Comment lines above the header
+
+```
+# generated by pipeline v3, 2026-08-05
+EnsemblID	log2FC	pval
+```
+
+`--skip-rows 1` discards the preamble. Column numbers count from the header, so
+they don't change when you add it.
+
+Skipping also fixes delimiter detection: the delimiter is sniffed from the
+header line, and a comment line is a bad thing to sniff — the one above has
+commas but no tabs, so without `--skip-rows` the file would be read as CSV and
+collapse into nonsense. Rather than let that through, a header that looks like a
+comment is rejected with a message pointing at this flag.
+
+### Two identifier columns
+
+`--ID` may be given twice. The first is the primary; the second fills rows where
+the primary is blank (`.`, `NA`, empty, and similar are all treated as missing).
+
+```bash
+ipaapi submit data.csv --ID 0:ensembl --ID 1:hugo --FC 4:logratio --project S1
+```
+
+> **Read this before relying on it.** IPA accepts one `geneidtype` per
+> submission. Rows filled from the second column are still uploaded under the
+> *primary's* type, so they may fail to map. The fill count is always reported:
+>
+> ```
+> Warning: 344 of 2,338 rows took their identifier from the fallback column
+> 'Common_name' (hugo). IPA is told a single gene ID type for the submission --
+> 'ensembl' -- so those rows are uploaded under that declaration and may not map.
+> ```
+>
+> If a large fraction is being filled, using the fallback column as the *only*
+> identifier is usually better than mixing.
+
+---
+
+## Working with IPA
+
+Most of this is either undocumented or documented somewhere hard to find. It's
+recorded here because getting it wrong is expensive — analyses consume a
+metered allowance.
+
+### Gene ID types
+
+`--ID COLUMN:TYPE` takes any value from IPA's `geneidtype` list (Integration
+Module §3.1). `ipaapi submit --list-id-types` prints all 33.
+
+Common ones: `ensembl`, `hugo`, `entrezgene`, `refseq`, `swissprot`,
+`affymetrix`, `illumina`, `agilent`.
+
+Two things are not guessable:
+
+- **Human gene symbols are `hugo`.** Not `genesymbol`, not `hgnc`, and not the
+  desktop client's own label `Gene Symbol` — all three are rejected outright.
+- **Species rides on the identifier type.** There is no species parameter:
+  `hugo` human, `mousesymeg` mouse, `ratsymeg` rat.
+
+A type outside the documented list produces a warning with a near-match
+suggestion but is still sent, since IPA is the authority and the list will age.
+An unrecognised value fails before anything is uploaded, and IPA names it.
+
+### The reference set
+
+The background enrichment is scored against — the denominator of the Fisher's
+exact test behind every p-value.
+
+| Value | Background |
+| --- | --- |
+| `ipkb` | Ingenuity Knowledge Base (Genes Only, or + Endogenous Chemicals if chemicals are present) |
+| `dataset` | the genes you uploaded |
+| `omit` (default) | IPA chooses |
+
+Which to use depends on **what you uploaded**:
+
+- Uploading a **complete measured transcriptome** with a cutoff? `dataset` is
+  the better science — the background is what your assay could actually detect,
+  which controls for detection bias.
+- Uploading a **pre-filtered hit list**? `dataset` makes the background nearly
+  identical to the foreground. Use `ipkb`.
+
+§4.1.3.1 states that with the parameter omitted IPA picks by size — `ipkb` below
+2000 identifiers, `dataset` at 2000 or more. **In practice this has not been
+observed to hold**: files of 1,804–6,245 rows all came back as
+`Ingenuity Knowledge Base (Genes Only)`. Since the behaviour is unpredictable,
+set it explicitly for anything you intend to compare against itself.
+
+Verify after the fact — the setting is recorded in every IPA export:
+
+```bash
+grep -h "^Reference set" *_IPA_output.txt | sort | uniq -c
+```
+
+Array platforms can also be named as reference sets, paired with a
+`referencesettype`. Not exposed here; see §4.1.3.
+
+### Measurement types
+
+| Value | Meaning | Valid range |
+| --- | --- | --- |
+| `ratio` | Ratio | `[0, +∞)` |
+| `foldchange` | Fold Change | `(-∞, -1]` and `[1, +∞)` |
+| `logratio` | Log Ratio | `(-∞, +∞)` |
+| `pvalue` | p-value | `[0, 1]` |
+| `falsediscovery` | FDR / q-value | `[0, 100]` |
+| `intensity` | Intensity | `[0, +∞)` |
+| `other` | Other (normalised around zero) | `(-∞, +∞)` |
+| `gain_loss` | Variant Gain/Loss | `-2, -1, 0, 1, 2` |
+| `classification` | Variant ACMG Classification | `-2, -1, 0, 1, 2` |
+
+> **Out-of-range values are silently discarded by IPA.** §3.1: "analysis will
+> still proceed without errors or warning diagnostics" — offending entries are
+> simply dropped. This is why the range check exists and why it refuses rather
+> than warns. Declaring log2 fold changes as `foldchange`, for instance, would
+> quietly discard every gene between −1 and 1, which in a typical scRNA-seq
+> table is most of them.
+
+The package helps in both directions:
+
+- Values declared `foldchange` that cluster inside (−1, 1) → suggests `logratio`.
+- A column declared `logratio` with *no* values in (−1, 1) → warns that it looks
+  like signed fold change, since a real log ratio is centred on zero.
+
+A column called `Fold_change` may hold either. Check the data, not the name.
+
+### What the API cannot do
+
+- **List your projects.** `--project` creates one if the name doesn't exist, so
+  a typo silently makes a new project rather than erroring.
+- **List your analyses.** Every endpoint needs an ID you already hold — hence
+  the local submission log.
+- **Tell you your remaining allowance.** You discover the limit by hitting it.
+
+### Errors IPA actually returns
+
+IPA answers a rejected submission with an **HTML error page**, not plain text.
+The reason is at the *end*, after support boilerplate. This package strips the
+boilerplate and the page footer, and classifies what's left:
+
+| IPA's message | Class | What the tool does |
+| --- | --- | --- |
+| `Unknown GeneId Type (X)` | `MalformedRequestError` | stops; names the flag; moves nothing |
+| `Unable to run analysis: Analysis limit exceeded` | `QuotaExceededError` | stops; leaves remaining files for the next run |
+| `Unable to run analysis: …` (other) | `AnalysisRefusedError` | as above — reached the analysis logic, so not a parameter fault |
+| anything else | `SubmissionError` | files that one under `failed/` |
+
+Quota matching is deliberately broad (`ipaapi.client.QUOTA_PATTERNS` plus HTTP
+429): a false positive only leaves a file for the next run, while a false
+negative would quarantine a retryable submission. The raw response is always
+printed, so a misclassification is visible.
+
+### Interpret links
+
+`ipaapi report <id>` fetches the IPA Interpret URL for a finished analysis. It
+checks status first, so an unfinished analysis says so rather than surfacing a
+bare HTTP 500.
+
+**These have been observed to return HTTP 500 even for succeeded analyses.**
+The cause is unconfirmed — possibly the commercial add-on licence, possibly a
+stale endpoint path inherited from the demo. `examples/probe_interpret.py`
+prints the raw response for diagnosis. Analyses open fine in IPA itself.
+
+---
+
+## Authentication
+
+Browser-based OAuth 2.0 with PKCE. Your password never reaches this package.
+
+1. A short-lived HTTP server binds `127.0.0.1:8000`.
+2. Your browser opens QIAGEN's authorization page; you log in there.
+3. QIAGEN redirects back to `localhost:8000` with a one-time code. The `state`
+   parameter is verified, then the code plus the PKCE verifier is exchanged for
+   a token.
+4. The token is used as `Authorization: Bearer …` and the server shuts down.
+
+Whichever account you log in as owns the datasets and projects.
+
+The client ID is the public one any IPA user may use — it is not a secret.
+
+### Token caching and refresh
+
+Tokens are cached at `~/.cache/ipaapi/token.json`, owner-only (0600). **Access
+tokens are short-lived**, but a refresh token comes with them and is spent
+automatically: an expired cache is renewed over HTTP with no browser and no
+prompt. A browser login is only needed when the refresh token itself is
+rejected.
+
+Deleting the cache is effectively logging out. `--no-cache` forces a fresh
+login. Be aware the token is plaintext on disk — anyone who can read your home
+directory can use it until it expires.
+
+### Headless servers
+
+Because refresh is automatic, a token copied from a machine with a browser keeps
+renewing itself indefinitely:
+
+```bash
+# once, on a machine with a browser
+ipaapi submit ... # or any command that logs in
+
+scp ~/.cache/ipaapi/token.json server:~/.cache/ipaapi/token.json
+ssh server chmod 600 ~/.cache/ipaapi/token.json
+```
+
+**If `$HOME` isn't writable**, the cache can't be saved and every run needs a
+fresh login — crippling on a headless box. Point it somewhere writable:
+
+```bash
+export IPAAPI_TOKEN_FILE=$HOME/ipaapi-token.json
+export IPAAPI_LOG_FILE=$HOME/ipaapi-submissions.tsv
+```
+
+Both failures are reported loudly rather than swallowed, because a cache that
+never writes looks exactly like a token that expires instantly.
+
+When an interactive login is genuinely needed, X forwarding is the cleanest
+route — the server-side browser renders locally *and* `localhost:8000` resolves
+server-side where the callback listens, so no port forwarding is required:
+
+```bash
+ssh -X you@server        # ssh -Y from macOS, with XQuartz running
+```
+
+Failing that, forward the callback port and use your own browser:
+
+```bash
+ssh -L 8000:localhost:8000 you@server
+```
+
+The error message distinguishes `DISPLAY` unset from no browser found.
+
+> The redirect URI is pinned to `http://localhost:8000` by the OAuth client
+> registration, so the port is not configurable in practice.
+
+### Using a token obtained elsewhere
+
+```python
+import os
+from ipaapi import Credentials, IPAClient
+
+client = IPAClient(Credentials.from_token(os.environ["IPA_TOKEN"]))
+```
+
+---
+
+## Python API
+
+```python
+from ipaapi import (
+    ColumnMapping, Dataset, IPAClient, Measurement, MeasurementType,
+    Observation, ReferenceSet, TokenCache,
+)
+
+mapping = ColumnMapping(
+    gene_id_column="Common_name",
+    gene_id_type="hugo",
+    observations=[
+        Observation("HIV vs NEG", [
+            Measurement("Fold_change", MeasurementType.LOG_RATIO),
+        ]),
+    ],
+)
+
+dataset = Dataset.from_file("results.csv", mapping, skip_rows=1)
+print(dataset.describe())          # confirm before uploading
+
+client = IPAClient.login(cache=TokenCache())
+ids = client.submit(dataset, project="MyStudy", reference_set=ReferenceSet.IPKB)
+
+for analysis_id, status in client.wait_for(ids).items():
+    if status.succeeded:
+        print(client.report_url(analysis_id))
+```
+
+Key objects:
+
+| Object | Purpose |
+| --- | --- |
+| `ColumnMapping`, `Observation`, `Measurement` | describe the file |
+| `Dataset.from_file` / `.from_frame` | load and validate |
+| `IPAClient.login()` | OAuth, with caching and refresh |
+| `.submit()` `.status()` `.wait_for()` `.results()` `.report_url()` | the API |
+| `GENE_ID_TYPES` | all 33 identifier types and what they mean |
+| `ipaapi.history` | the submission log |
+| `ipaapi.errors` | everything derives from `IPAError` |
+
+### Results
+
+```python
+results = client.results(analysis_id)
+print(results.canonical_pathways.head())
+cp, ur, df = results                  # unpacks like the demo's ipa_results()
+```
+
+> Programmatic result retrieval is a **commercial IPA add-on**. Without it these
+> calls raise `ResultsUnavailableError`. Submission, status polling and report
+> links are unaffected.
+
+---
+
+## Troubleshooting
+
+| Symptom | Cause | Fix |
+| --- | --- | --- |
+| `REJECTED: IPA does not recognise the gene ID type 'X'` | not in IPA's vocabulary | `--list-id-types`; human symbols are `hugo` |
+| `declared 'foldchange' but holds N out-of-range value(s)` | log2 values declared as linear fold change | `--FC N:logratio` |
+| `Could not find a header row … looks like a comment` | preamble above the header | `--skip-rows N` |
+| `--FC refers to column N, but the file has only M column(s)` | 1-based counting, or wrong `--skip-rows` | positions are 0-based, from the header |
+| `Every row is missing an identifier` | wrong column, or no header | check with `head -1 file \| tr '\t' '\n' \| nl -v0` |
+| `the analysis allowance appears to be exhausted` | daily/period limit | re-run later; files left in place resume |
+| Login prompt on every run | token cache not writable | `export IPAAPI_TOKEN_FILE=...`; check for a root-owned cache |
+| `Could not open a browser automatically` | headless | `ssh -X`, or copy a token across |
+| `report` returns HTTP 500 on a succeeded analysis | unconfirmed; possibly add-on licence | open the analysis in IPA; see `examples/probe_interpret.py` |
+| Analyses have z-scores but no p-values | reference set equals the gene list | `--reference-set ipkb` |
+| Half of all pathways significant | list too large for the background | apply a cutoff, or upload unfiltered data with a cutoff |
+
+Useful first move for any column problem:
+
+```bash
+head -1 yourfile.csv | tr ',\t' '\n' | nl -v0
+```
+
+---
+
+## How a submission is encoded
+
+Worth knowing when debugging. `--ID 1:hugo` becomes three separate things:
+
+| From `--ID` | Wire parameter | Sent |
+| --- | --- | --- |
+| the type | `geneidtype=hugo` | once |
+| the column, resolved from position to header name | `genecolname=Common_name` | once |
+| that column's values | `geneid=XIST`, `geneid=UTY`, … | once per row |
+
+The column *number* never leaves your machine.
+
+The whole dataset travels in one `application/x-www-form-urlencoded` POST to
+`/pa/api/v2/multiobsanalysis`, which both creates the dataset in the project and
+starts one analysis per observation. Parameter naming is positional and
+irregular — for measurement slot *k* and observation *i*, both zero-based:
+
+| Parameter | Meaning |
+| --- | --- |
+| `expvaltype`, `expvaltypeK+1` | measurement type for slot *k* (global) |
+| `cutoff`, `cutoffK+1` | cutoff for slot *k* (global, optional) |
+| `obsI+1name` | observation name |
+| `expvalname`, `expvalK+1name` | column label, first observation |
+| `obsI+1expvalname`, `obsI+1expvalK+1name` | column label, later observations |
+| `geneid` | one per data row |
+| `expvalue`, `expvalK+1` | one per slot per observation, per row |
+
+Per-row value parameters carry no observation prefix — they cycle through the
+slots of observation 1, then observation 2, and so on. Order is load-bearing.
+
+The body is properly percent-encoded. The demo concatenated it by hand, so any
+value containing a space, `&`, `=`, `+` or `%` corrupted the request — including
+the `Group Max Intensity` column in the demo's own sample dataset.
+
+---
+
+## Development
+
+```
+src/ipaapi/
+  __init__.py    public API and the version (single source of truth)
+  models.py      MeasurementType, AnalysisStatus, ReferenceSet, GENE_ID_TYPES
+  mapping.py     Measurement, Observation, ColumnMapping
+  dataset.py     Dataset, load_table
+  _payload.py    multiobsanalysis body construction
+  auth.py        OAuth 2.0 + PKCE, Credentials, TokenCache, refresh
+  client.py      IPAClient, error classification
+  history.py     the submission log
+  triage.py      submitted/ and failed/ filing
+  cli.py         the ipaapi console script
+  errors.py      exception hierarchy
+tests/           offline; no network required
+examples/        runnable scripts and diagnostics
+```
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+The suite is fully offline — mapping validation, the exact parameter layout of
+the submission body, encoding of hostile characters, error classification,
+triage behaviour, token cache and refresh logic.
+
+**Versioning.** The version lives only in `src/ipaapi/__init__.py`;
+`pyproject.toml` reads it at build time. Bump it there and nowhere else, and add
+a `CHANGELOG.md` entry. `ipaapi --version` reports the install path too, which
+is what actually answers "am I running the wheel I think I am".
+
+### Differences from the demo
+
+- Column mapping by name in any order, validated before upload.
+- Request bodies are percent-encoded.
+- OAuth: no CPU-spinning wait loop, `state` is verified, logins time out, the
+  callback server is shut down, error redirects are handled, tokens are cached
+  and refreshed.
+- Submissions are never retried automatically — a retried POST could create a
+  duplicate analysis. GETs retry with backoff.
+- Typed exceptions; access tokens excluded from `repr()`.
+- No `install_dependencies()` shelling out to `pip3`.
+
+---
+
+## Contributing
+
+Issues and pull requests are welcome. The most useful contributions are
+**corrections to the [Working with IPA](#working-with-ipa) section** — much of
+it was established by trial against a live account, and a few points have
+already had to be corrected more than once. If IPA behaves differently for you,
+that is worth reporting even without a code change.
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+Tests are fully offline; none of them contact IPA.
+
+---
+
+## Status
+
+**1.0** — stable and in production use against live IPA. The command line and
+the Python API are settled; breaking changes from here mean a major version
+bump. See `CHANGELOG.md`.
+
+Known open questions, none of which affect submission:
+
+- Interpret links (`ipaapi report`) have returned HTTP 500 for analyses that
+  succeeded. Cause unconfirmed; possibly the commercial add-on licence.
+- Programmatic result retrieval (`client.results()`) requires that same add-on
+  and is largely untested here.
+- The documented reference-set size rule does not match observed behaviour;
+  set `--reference-set` explicitly.
+
+---
+
+## Licence
+
+MIT — see [LICENSE](LICENSE). Free to use, modify and redistribute.
+
+Not affiliated with, endorsed by, or supported by QIAGEN. IPA is QIAGEN's
+product; this is an independent client for its public API, built on the
+`python-api-demo` example code QIAGEN publishes. For questions about the API
+itself, QIAGEN's contact is `AdvancedGenomicsSupport@qiagen.com` — please don't
+send them bug reports about this package.