ipaapi 1.0.0__tar.gz → 1.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {ipaapi-1.0.0 → ipaapi-1.2.0}/CHANGELOG.md +99 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/PKG-INFO +149 -2
- {ipaapi-1.0.0 → ipaapi-1.2.0}/README.md +148 -1
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/__init__.py +1 -1
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/cli.py +460 -7
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/client.py +43 -2
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/errors.py +13 -0
- ipaapi-1.2.0/tests/test_cli_observation_name.py +218 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/tests/test_triage.py +162 -1
- {ipaapi-1.0.0 → ipaapi-1.2.0}/.gitignore +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/LICENSE +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/examples/probe_geneidtype.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/examples/probe_interpret.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/examples/quickstart.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/ipaapi-1.0.0-py3-none-any.whl +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/pyproject.toml +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/_payload.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/auth.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/dataset.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/history.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/mapping.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/models.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/src/ipaapi/triage.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/tests/conftest.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/tests/test_auth.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/tests/test_batch.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/tests/test_cli.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/tests/test_client.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/tests/test_history.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/tests/test_mapping.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/tests/test_payload.py +0 -0
- {ipaapi-1.0.0 → ipaapi-1.2.0}/tests/test_skiprows.py +0 -0
|
@@ -7,6 +7,105 @@ minor, fixes bump the patch.
|
|
|
7
7
|
Check what you're running with `ipaapi --version`, which reports the version,
|
|
8
8
|
the install location, and whether it's an editable checkout rather than a wheel.
|
|
9
9
|
|
|
10
|
+
## 1.2.0 — 2026-08-24
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- **Long observation names are shortened automatically.** IPA rejects an
|
|
15
|
+
observation name past roughly 65 characters and reports it as *"The page you
|
|
16
|
+
are looking for is currently unavailable"* — the same page it returns for a
|
|
17
|
+
duplicate dataset name, and for a genuine outage. Because the observation
|
|
18
|
+
name defaults to the filename, descriptive pipeline output names tripped it,
|
|
19
|
+
and a batch died on its first file looking like a total service failure.
|
|
20
|
+
|
|
21
|
+
Established by A/B on one file, holding project, reference set and data
|
|
22
|
+
constant: a 25-character observation name with an 82-character dataset name
|
|
23
|
+
was accepted (it reached IPA's allowance check); a 25-character *dataset*
|
|
24
|
+
name with an 82-character observation name was rejected. So the limit is
|
|
25
|
+
specific to the observation — a long dataset name is fine.
|
|
26
|
+
|
|
27
|
+
Names are now brought under 60 characters automatically, removing what
|
|
28
|
+
carries the least meaning first. Two parts of a pipeline filename matter —
|
|
29
|
+
the contrast (`Estrus_vs_2dpp`) and the cell type the comparison came from
|
|
30
|
+
(`Immature_cortical_ovarian_stroma`) — and everything appended about how the
|
|
31
|
+
pipeline ran does not.
|
|
32
|
+
|
|
33
|
+
Paralome output is cut on its own structure rather than by heuristic. It
|
|
34
|
+
names files
|
|
35
|
+
`<contrast>_<celltype>_<method>_<test>_significant_<threshold>_<assay>`, so
|
|
36
|
+
the `significant` literal anchors the cut exactly: drop it and everything
|
|
37
|
+
after, drop the test immediately before it, drop the aggregation method. No
|
|
38
|
+
list of test names is needed — the test is whatever token precedes the
|
|
39
|
+
anchor, so `wilcox` works as well as `t`. Methods are matched as whole
|
|
40
|
+
phrases in that one position only, since a cell type of `Naive_T_cell`
|
|
41
|
+
shares both words with the `naive_cell` method.
|
|
42
|
+
|
|
43
|
+
Files from anything else fall back to generic metadata removal, then any
|
|
44
|
+
suffix the batch shares, then the shared prefix, and only as a last resort a
|
|
45
|
+
two-ended cut marked with `..`.
|
|
46
|
+
|
|
47
|
+
Comparison is token by token, so `Mature` is never read as a prefix of
|
|
48
|
+
`Immature` and `cell_type` is never left as `cell_t`. Any step is abandoned
|
|
49
|
+
if it would make two names identical or leave one unreadable, since
|
|
50
|
+
observation names are what IPA lists side by side in a comparison analysis.
|
|
51
|
+
Names already within the limit are untouched, and the dataset and analysis
|
|
52
|
+
always keep the full filename.
|
|
53
|
+
|
|
54
|
+
- Rejection guidance now names the observation length first, since it is the
|
|
55
|
+
cause hardest to guess from what IPA returns.
|
|
56
|
+
|
|
57
|
+
### Added
|
|
58
|
+
|
|
59
|
+
- **`--strip TEXT`**, repeatable, removes text from observation names before
|
|
60
|
+
shortening — for a pipeline whose suffix the built-in list does not cover.
|
|
61
|
+
Ignored as a whole if applying it would leave the names empty, unreadable, or
|
|
62
|
+
no longer distinct.
|
|
63
|
+
|
|
64
|
+
### Notes
|
|
65
|
+
|
|
66
|
+
- This is a *second*, independent cause of the same misleading page — 1.1.0
|
|
67
|
+
fixed duplicate dataset names. Having already attributed that page once made
|
|
68
|
+
this one harder to see, not easier. If a batch dies on its first file with
|
|
69
|
+
that wording, both causes are worth ruling out.
|
|
70
|
+
|
|
71
|
+
## 1.1.0 — 2026-08-05
|
|
72
|
+
|
|
73
|
+
### Added
|
|
74
|
+
|
|
75
|
+
- **Duplicate dataset names are detected before submitting.** IPA refuses a
|
|
76
|
+
dataset whose name already exists in a project, and reports it as *"The page
|
|
77
|
+
you are looking for is currently unavailable"* — wording that reads as an
|
|
78
|
+
outage. Because dataset names come from filenames, re-running a batch retried
|
|
79
|
+
names an earlier run had created, so the run died on its first file and
|
|
80
|
+
looked like a total service failure. It cost a day to find.
|
|
81
|
+
|
|
82
|
+
Established by experiment: the identical 2 KB request succeeded and then
|
|
83
|
+
failed twice; with unique dataset names three consecutive submissions all
|
|
84
|
+
succeeded. Not size, not rate limiting, not parameters, not an outage.
|
|
85
|
+
|
|
86
|
+
`submit` now checks the submission log for that project and dataset name
|
|
87
|
+
first, skips the file with an explanation, and files it under `submitted/`.
|
|
88
|
+
`--force` overrides.
|
|
89
|
+
|
|
90
|
+
### Notes
|
|
91
|
+
|
|
92
|
+
- The guard covers submissions made through this tool with the same log file.
|
|
93
|
+
A collision caused by another user or the IPA client still surfaces as the
|
|
94
|
+
misleading outage page; the troubleshooting table now says so.
|
|
95
|
+
|
|
96
|
+
## 1.0.1 — 2026-08-05
|
|
97
|
+
|
|
98
|
+
### Fixed
|
|
99
|
+
|
|
100
|
+
- **An IPA outage is no longer reported as a bad parameter.** IPA's maintenance
|
|
101
|
+
page ("currently unavailable", "experiencing technical difficulties", "try
|
|
102
|
+
again later") is HTML, so it fell through to `MalformedRequestError` and the
|
|
103
|
+
message told the user to check `--reference-set` and `--ID` — sending them to
|
|
104
|
+
rewrite a command that was correct. New `ServiceUnavailableError`, also
|
|
105
|
+
raised on 502/503/504, says plainly that IPA is down, that nothing about the
|
|
106
|
+
command needs changing, and that re-running later resumes. Files are left in
|
|
107
|
+
place, as before.
|
|
108
|
+
|
|
10
109
|
## 1.0.0 — 2026-08-05
|
|
11
110
|
|
|
12
111
|
First stable release. No code changes from 0.5.0 — the version marks that the
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: ipaapi
|
|
3
|
-
Version: 1.
|
|
3
|
+
Version: 1.2.0
|
|
4
4
|
Summary: Command-line client for QIAGEN Ingenuity Pathway Analysis: map arbitrary column layouts, submit datasets in bulk, and resume cleanly when the analysis allowance runs out.
|
|
5
5
|
Author-email: Ken Jones <Ken.Jones@bioinformaticsolutions.com>
|
|
6
6
|
License: MIT License
|
|
@@ -251,7 +251,8 @@ Used by `validate` and `submit`.
|
|
|
251
251
|
| `--sep` | `CHAR` | field delimiter (sniffed from the header line by default) |
|
|
252
252
|
| `--pattern` | `TEXT` | when PATH is a directory: substring or glob selecting files |
|
|
253
253
|
| `--recursive` | flag | search subdirectories too |
|
|
254
|
-
| `--observation` | `NAME` | observation name in IPA (default: the filename). Single file only |
|
|
254
|
+
| `--observation` | `NAME` | observation name in IPA (default: the filename). Single file only. Shortened to 60 characters if needed — see [long observation names](#long-observation-names) |
|
|
255
|
+
| `--strip` | `TEXT` | remove TEXT from observation names before shortening. Repeatable |
|
|
255
256
|
| `--no-range-check` | flag | skip the value-range validation |
|
|
256
257
|
| `--list-id-types` | flag | print all 33 gene ID types and exit |
|
|
257
258
|
|
|
@@ -443,6 +444,147 @@ A type outside the documented list produces a warning with a near-match
|
|
|
443
444
|
suggestion but is still sent, since IPA is the authority and the list will age.
|
|
444
445
|
An unrecognised value fails before anything is uploaded, and IPA names it.
|
|
445
446
|
|
|
447
|
+
### Duplicate dataset names
|
|
448
|
+
|
|
449
|
+
**IPA refuses to create a dataset whose name already exists in a project — and
|
|
450
|
+
reports it as "The page you are looking for is currently unavailable."** The
|
|
451
|
+
wording says outage; the cause is a name collision. This is the single most
|
|
452
|
+
misleading response the API produces, and it cost a full day to identify.
|
|
453
|
+
|
|
454
|
+
Verified: the same 2 KB request, byte for byte, succeeded and then failed twice
|
|
455
|
+
in a row. With unique dataset names, three consecutive submissions all
|
|
456
|
+
succeeded.
|
|
457
|
+
|
|
458
|
+
Since dataset names come from filenames, re-running a batch retries names an
|
|
459
|
+
earlier run already created — so the run dies on its *first* file and looks
|
|
460
|
+
like a total outage.
|
|
461
|
+
|
|
462
|
+
The package prevents this using the submission log. Before uploading, it checks
|
|
463
|
+
whether that dataset name already went to that project:
|
|
464
|
+
|
|
465
|
+
```
|
|
466
|
+
skipping SampleA_DEG: already submitted to 'Study1' on 2026-08-05T08:35:53-06:00
|
|
467
|
+
as analysis 43595039. IPA would reject a second dataset of the same name.
|
|
468
|
+
Use --force to submit it again anyway.
|
|
469
|
+
```
|
|
470
|
+
|
|
471
|
+
Skipped files are filed under `submitted/`, because they are. `--force`
|
|
472
|
+
overrides, though IPA will normally reject it.
|
|
473
|
+
|
|
474
|
+
The guard only knows about submissions made through this tool with the same log
|
|
475
|
+
file. If you hit the collision anyway — a colleague's upload, or the IPA client
|
|
476
|
+
— the fix is a different `--project`, a different `--dataset-name`, or deleting
|
|
477
|
+
the existing dataset in IPA.
|
|
478
|
+
|
|
479
|
+
### Long observation names
|
|
480
|
+
|
|
481
|
+
**IPA rejects a long observation name, and reports it as "The page you are
|
|
482
|
+
looking for is currently unavailable."** The same misleading page as a
|
|
483
|
+
duplicate dataset name, from an entirely unrelated cause — which is what made
|
|
484
|
+
this one expensive to find: the obvious explanation had already been used up.
|
|
485
|
+
|
|
486
|
+
The observation name defaults to the filename, so descriptive pipeline output
|
|
487
|
+
names run past the limit without anyone choosing a long name. A long *dataset*
|
|
488
|
+
name is fine; only the observation is affected.
|
|
489
|
+
|
|
490
|
+
Established by A/B on one file, holding project, reference set and data
|
|
491
|
+
constant:
|
|
492
|
+
|
|
493
|
+
| dataset name | observation name | result |
|
|
494
|
+
| --- | --- | --- |
|
|
495
|
+
| short (25) | **long (82)** | rejected as an "outage" |
|
|
496
|
+
| **long (82)** | short (25) | accepted — reached IPA's allowance check |
|
|
497
|
+
|
|
498
|
+
65 characters is known good and 82 known bad; the exact limit is undocumented.
|
|
499
|
+
Since 1.2.0 the observation name is brought under 60 characters automatically.
|
|
500
|
+
|
|
501
|
+
**What gets removed is chosen by what it means, not by where it sits.** Two
|
|
502
|
+
parts of a Paralome filename carry information: the contrast (`Estrus_vs_2dpp`)
|
|
503
|
+
and the cell type the comparison was computed from
|
|
504
|
+
(`Immature_cortical_ovarian_stroma`). Everything the pipeline appends about how
|
|
505
|
+
it ran — `_naive_cell_t_significant_p0.05_rna`, `_pseudobulk_t_significant_p0.05_rna` —
|
|
506
|
+
is disposable. So that is what goes first:
|
|
507
|
+
|
|
508
|
+
```
|
|
509
|
+
Estrus_vs_2dpp_Immature_cortical_ovarian_stroma_naive_cell_t_significant_p0.05_rna
|
|
510
|
+
-> Estrus_vs_2dpp_Immature_cortical_ovarian_stroma
|
|
511
|
+
|
|
512
|
+
Estrus_vs_2dpp_Glandular_epithelium_pseudobulk_t_significant_p0.05_rna
|
|
513
|
+
-> Estrus_vs_2dpp_Glandular_epithelium
|
|
514
|
+
```
|
|
515
|
+
|
|
516
|
+
**Paralome output is cut on its own structure, not by guesswork.** Paralome
|
|
517
|
+
names its files
|
|
518
|
+
`<contrast>_<celltype>_<method>_<test>_significant_<threshold>_<assay>`, and
|
|
519
|
+
the `significant` literal is a reliable anchor:
|
|
520
|
+
|
|
521
|
+
```
|
|
522
|
+
Estrus_vs_2dpp_Immature_cortical_ovarian_stroma | naive_cell | t | significant_p0.05_rna
|
|
523
|
+
Estrus_vs_2dpp_Glandular_epithelium | pseudobulk | t | significant_p0.05_rna
|
|
524
|
+
```
|
|
525
|
+
|
|
526
|
+
Working backwards from the anchor: drop it and everything after, drop the
|
|
527
|
+
statistical test immediately before it, then drop the aggregation method. The
|
|
528
|
+
test needs no list of names — it is simply the token before the anchor, so
|
|
529
|
+
`wilcox` works as well as `t` without anyone maintaining a vocabulary. Methods
|
|
530
|
+
are matched as **whole phrases in that one position only**, because a cell type
|
|
531
|
+
of `Naive_T_cell` shares both words with the `naive_cell` method and loose
|
|
532
|
+
matching cut it to `Naive_T`.
|
|
533
|
+
|
|
534
|
+
Removal is graded, least damaging first, and stops as soon as the names fit:
|
|
535
|
+
|
|
536
|
+
1. anything named with `--strip`
|
|
537
|
+
2. the Paralome tail, cut at its anchor as above
|
|
538
|
+
3. trailing pipeline metadata, for files from anything else — tokens like
|
|
539
|
+
`significant`, `deseq2`, `filtered`, and cutoffs in any of the shapes
|
|
540
|
+
`p0.05`, `fdr0.01`, `padj0.05`, `0.05`, read right to left and stopping at
|
|
541
|
+
the first token that isn't recognisable as a setting
|
|
542
|
+
4. the suffix every file in the batch happens to share
|
|
543
|
+
5. the prefix every file shares — this costs the contrast, so it is late
|
|
544
|
+
6. a two-ended cut with `..` marking the gap, on a token boundary
|
|
545
|
+
|
|
546
|
+
Steps 2 to 4 always run together rather than stopping the moment the names
|
|
547
|
+
merely fit, since half a removed suffix reads worse than either whole. Comparison
|
|
548
|
+
is **whole tokens at a time**, so `Mature` is never treated as a prefix of
|
|
549
|
+
`Immature` and `cell_type` is never left as `cell_t`. Names already within the
|
|
550
|
+
limit are returned untouched.
|
|
551
|
+
|
|
552
|
+
Any step is abandoned if it would make two names identical, or leave one under
|
|
553
|
+
four characters or without a letter — reducing files to `1` and `2` keeps them
|
|
554
|
+
distinct and makes the analysis unreadable. Distinctness is the property being
|
|
555
|
+
protected: observation names are what IPA lists side by side in a comparison
|
|
556
|
+
analysis, so two files reduced to the same label would be worse than a long name.
|
|
557
|
+
|
|
558
|
+
Everything removed is printed:
|
|
559
|
+
|
|
560
|
+
```
|
|
561
|
+
note: shortened 2 observation names. IPA rejects a long observation name and
|
|
562
|
+
reports it as an outage, so this is not optional.
|
|
563
|
+
Datasets and analyses keep the full filename; only the observation label
|
|
564
|
+
inside the analysis is shorter.
|
|
565
|
+
Estrus_vs_2dpp_Glandular_epithelium_pseudobulk_t_significant_p0.05_rna
|
|
566
|
+
-> Estrus_vs_2dpp_Glandular_epithelium
|
|
567
|
+
```
|
|
568
|
+
|
|
569
|
+
If your pipeline's suffix is not recognised, name it explicitly. `--strip` is
|
|
570
|
+
repeatable, applied before anything else, and ignored as a whole if it would
|
|
571
|
+
leave the names unusable:
|
|
572
|
+
|
|
573
|
+
```bash
|
|
574
|
+
ipaapi submit ~/data --ID 1:mousesymeg --FC 4:foldchange --skip-rows 1 \
|
|
575
|
+
--project Study --strip _significant_p0.05_rna
|
|
576
|
+
```
|
|
577
|
+
|
|
578
|
+
Because steps 4 and 5 depend on the other files in the run, **the same file
|
|
579
|
+
submitted in a different batch can get a different observation label**. Steps 1 to 3
|
|
580
|
+
do not — they read only the name in front of them, which is why a
|
|
581
|
+
single-file submit shortens as well as a batch does. The dataset and analysis
|
|
582
|
+
always keep the full filename, so `ipaapi history` and the IPA project view are
|
|
583
|
+
unaffected.
|
|
584
|
+
|
|
585
|
+
`--observation` sets the name explicitly for a single file and is shortened the
|
|
586
|
+
same way if it needs to be.
|
|
587
|
+
|
|
446
588
|
### The reference set
|
|
447
589
|
|
|
448
590
|
The background enrichment is scored against — the denominator of the Fisher's
|
|
@@ -525,6 +667,7 @@ boilerplate and the page footer, and classifies what's left:
|
|
|
525
667
|
| `Unknown GeneId Type (X)` | `MalformedRequestError` | stops; names the flag; moves nothing |
|
|
526
668
|
| `Unable to run analysis: Analysis limit exceeded` | `QuotaExceededError` | stops; leaves remaining files for the next run |
|
|
527
669
|
| `Unable to run analysis: …` (other) | `AnalysisRefusedError` | as above — reached the analysis logic, so not a parameter fault |
|
|
670
|
+
| `currently unavailable` / `technical difficulties`, or 502/503/504 | `ServiceUnavailableError` | IPA is down; stops, moves nothing, says the command is fine |
|
|
528
671
|
| anything else | `SubmissionError` | files that one under `failed/` |
|
|
529
672
|
|
|
530
673
|
Quota matching is deliberately broad (`ipaapi.client.QUOTA_PATTERNS` plus HTTP
|
|
@@ -691,6 +834,10 @@ cp, ur, df = results # unpacks like the demo's ipa_results()
|
|
|
691
834
|
| `--FC refers to column N, but the file has only M column(s)` | 1-based counting, or wrong `--skip-rows` | positions are 0-based, from the header |
|
|
692
835
|
| `Every row is missing an identifier` | wrong column, or no header | check with `head -1 file \| tr '\t' '\n' \| nl -v0` |
|
|
693
836
|
| `the analysis allowance appears to be exhausted` | daily/period limit | re-run later; files left in place resume |
|
|
837
|
+
| `IPA appears to be down or having trouble` | IPA outage — **or a duplicate dataset name, or a long observation name**, all reported identically | `ipaapi --version` (1.2.0+ handles the name length); check `ipaapi history --project X` for the dataset name; otherwise wait |
|
|
838
|
+
| Batch dies on the first file, names come from long filenames | observation name too long | upgrade to 1.2.0+, or pass a short `--observation` |
|
|
839
|
+
| Batch dies on the *first* file after an earlier run | dataset names already exist in the project | expected — 1.1.0 skips them automatically; before that, use a new `--project` |
|
|
840
|
+
| `Cannot listen on 127.0.0.1:8000` | stale login process, or another user mid-login | `ss -ltnp 'sport = :8000'`, then kill it if it's yours |
|
|
694
841
|
| Login prompt on every run | token cache not writable | `export IPAAPI_TOKEN_FILE=...`; check for a root-owned cache |
|
|
695
842
|
| `Could not open a browser automatically` | headless | `ssh -X`, or copy a token across |
|
|
696
843
|
| `report` returns HTTP 500 on a succeeded analysis | unconfirmed; possibly add-on licence | open the analysis in IPA; see `examples/probe_interpret.py` |
|
|
@@ -202,7 +202,8 @@ Used by `validate` and `submit`.
|
|
|
202
202
|
| `--sep` | `CHAR` | field delimiter (sniffed from the header line by default) |
|
|
203
203
|
| `--pattern` | `TEXT` | when PATH is a directory: substring or glob selecting files |
|
|
204
204
|
| `--recursive` | flag | search subdirectories too |
|
|
205
|
-
| `--observation` | `NAME` | observation name in IPA (default: the filename). Single file only |
|
|
205
|
+
| `--observation` | `NAME` | observation name in IPA (default: the filename). Single file only. Shortened to 60 characters if needed — see [long observation names](#long-observation-names) |
|
|
206
|
+
| `--strip` | `TEXT` | remove TEXT from observation names before shortening. Repeatable |
|
|
206
207
|
| `--no-range-check` | flag | skip the value-range validation |
|
|
207
208
|
| `--list-id-types` | flag | print all 33 gene ID types and exit |
|
|
208
209
|
|
|
@@ -394,6 +395,147 @@ A type outside the documented list produces a warning with a near-match
|
|
|
394
395
|
suggestion but is still sent, since IPA is the authority and the list will age.
|
|
395
396
|
An unrecognised value fails before anything is uploaded, and IPA names it.
|
|
396
397
|
|
|
398
|
+
### Duplicate dataset names
|
|
399
|
+
|
|
400
|
+
**IPA refuses to create a dataset whose name already exists in a project — and
|
|
401
|
+
reports it as "The page you are looking for is currently unavailable."** The
|
|
402
|
+
wording says outage; the cause is a name collision. This is the single most
|
|
403
|
+
misleading response the API produces, and it cost a full day to identify.
|
|
404
|
+
|
|
405
|
+
Verified: the same 2 KB request, byte for byte, succeeded and then failed twice
|
|
406
|
+
in a row. With unique dataset names, three consecutive submissions all
|
|
407
|
+
succeeded.
|
|
408
|
+
|
|
409
|
+
Since dataset names come from filenames, re-running a batch retries names an
|
|
410
|
+
earlier run already created — so the run dies on its *first* file and looks
|
|
411
|
+
like a total outage.
|
|
412
|
+
|
|
413
|
+
The package prevents this using the submission log. Before uploading, it checks
|
|
414
|
+
whether that dataset name already went to that project:
|
|
415
|
+
|
|
416
|
+
```
|
|
417
|
+
skipping SampleA_DEG: already submitted to 'Study1' on 2026-08-05T08:35:53-06:00
|
|
418
|
+
as analysis 43595039. IPA would reject a second dataset of the same name.
|
|
419
|
+
Use --force to submit it again anyway.
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
Skipped files are filed under `submitted/`, because they are. `--force`
|
|
423
|
+
overrides, though IPA will normally reject it.
|
|
424
|
+
|
|
425
|
+
The guard only knows about submissions made through this tool with the same log
|
|
426
|
+
file. If you hit the collision anyway — a colleague's upload, or the IPA client
|
|
427
|
+
— the fix is a different `--project`, a different `--dataset-name`, or deleting
|
|
428
|
+
the existing dataset in IPA.
|
|
429
|
+
|
|
430
|
+
### Long observation names
|
|
431
|
+
|
|
432
|
+
**IPA rejects a long observation name, and reports it as "The page you are
|
|
433
|
+
looking for is currently unavailable."** The same misleading page as a
|
|
434
|
+
duplicate dataset name, from an entirely unrelated cause — which is what made
|
|
435
|
+
this one expensive to find: the obvious explanation had already been used up.
|
|
436
|
+
|
|
437
|
+
The observation name defaults to the filename, so descriptive pipeline output
|
|
438
|
+
names run past the limit without anyone choosing a long name. A long *dataset*
|
|
439
|
+
name is fine; only the observation is affected.
|
|
440
|
+
|
|
441
|
+
Established by A/B on one file, holding project, reference set and data
|
|
442
|
+
constant:
|
|
443
|
+
|
|
444
|
+
| dataset name | observation name | result |
|
|
445
|
+
| --- | --- | --- |
|
|
446
|
+
| short (25) | **long (82)** | rejected as an "outage" |
|
|
447
|
+
| **long (82)** | short (25) | accepted — reached IPA's allowance check |
|
|
448
|
+
|
|
449
|
+
65 characters is known good and 82 known bad; the exact limit is undocumented.
|
|
450
|
+
Since 1.2.0 the observation name is brought under 60 characters automatically.
|
|
451
|
+
|
|
452
|
+
**What gets removed is chosen by what it means, not by where it sits.** Two
|
|
453
|
+
parts of a Paralome filename carry information: the contrast (`Estrus_vs_2dpp`)
|
|
454
|
+
and the cell type the comparison was computed from
|
|
455
|
+
(`Immature_cortical_ovarian_stroma`). Everything the pipeline appends about how
|
|
456
|
+
it ran — `_naive_cell_t_significant_p0.05_rna`, `_pseudobulk_t_significant_p0.05_rna` —
|
|
457
|
+
is disposable. So that is what goes first:
|
|
458
|
+
|
|
459
|
+
```
|
|
460
|
+
Estrus_vs_2dpp_Immature_cortical_ovarian_stroma_naive_cell_t_significant_p0.05_rna
|
|
461
|
+
-> Estrus_vs_2dpp_Immature_cortical_ovarian_stroma
|
|
462
|
+
|
|
463
|
+
Estrus_vs_2dpp_Glandular_epithelium_pseudobulk_t_significant_p0.05_rna
|
|
464
|
+
-> Estrus_vs_2dpp_Glandular_epithelium
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
**Paralome output is cut on its own structure, not by guesswork.** Paralome
|
|
468
|
+
names its files
|
|
469
|
+
`<contrast>_<celltype>_<method>_<test>_significant_<threshold>_<assay>`, and
|
|
470
|
+
the `significant` literal is a reliable anchor:
|
|
471
|
+
|
|
472
|
+
```
|
|
473
|
+
Estrus_vs_2dpp_Immature_cortical_ovarian_stroma | naive_cell | t | significant_p0.05_rna
|
|
474
|
+
Estrus_vs_2dpp_Glandular_epithelium | pseudobulk | t | significant_p0.05_rna
|
|
475
|
+
```
|
|
476
|
+
|
|
477
|
+
Working backwards from the anchor: drop it and everything after, drop the
|
|
478
|
+
statistical test immediately before it, then drop the aggregation method. The
|
|
479
|
+
test needs no list of names — it is simply the token before the anchor, so
|
|
480
|
+
`wilcox` works as well as `t` without anyone maintaining a vocabulary. Methods
|
|
481
|
+
are matched as **whole phrases in that one position only**, because a cell type
|
|
482
|
+
of `Naive_T_cell` shares both words with the `naive_cell` method and loose
|
|
483
|
+
matching cut it to `Naive_T`.
|
|
484
|
+
|
|
485
|
+
Removal is graded, least damaging first, and stops as soon as the names fit:
|
|
486
|
+
|
|
487
|
+
1. anything named with `--strip`
|
|
488
|
+
2. the Paralome tail, cut at its anchor as above
|
|
489
|
+
3. trailing pipeline metadata, for files from anything else — tokens like
|
|
490
|
+
`significant`, `deseq2`, `filtered`, and cutoffs in any of the shapes
|
|
491
|
+
`p0.05`, `fdr0.01`, `padj0.05`, `0.05`, read right to left and stopping at
|
|
492
|
+
the first token that isn't recognisable as a setting
|
|
493
|
+
4. the suffix every file in the batch happens to share
|
|
494
|
+
5. the prefix every file shares — this costs the contrast, so it is late
|
|
495
|
+
6. a two-ended cut with `..` marking the gap, on a token boundary
|
|
496
|
+
|
|
497
|
+
Steps 2 to 4 always run together rather than stopping the moment the names
|
|
498
|
+
merely fit, since half a removed suffix reads worse than either whole. Comparison
|
|
499
|
+
is **whole tokens at a time**, so `Mature` is never treated as a prefix of
|
|
500
|
+
`Immature` and `cell_type` is never left as `cell_t`. Names already within the
|
|
501
|
+
limit are returned untouched.
|
|
502
|
+
|
|
503
|
+
Any step is abandoned if it would make two names identical, or leave one under
|
|
504
|
+
four characters or without a letter — reducing files to `1` and `2` keeps them
|
|
505
|
+
distinct and makes the analysis unreadable. Distinctness is the property being
|
|
506
|
+
protected: observation names are what IPA lists side by side in a comparison
|
|
507
|
+
analysis, so two files reduced to the same label would be worse than a long name.
|
|
508
|
+
|
|
509
|
+
Everything removed is printed:
|
|
510
|
+
|
|
511
|
+
```
|
|
512
|
+
note: shortened 2 observation names. IPA rejects a long observation name and
|
|
513
|
+
reports it as an outage, so this is not optional.
|
|
514
|
+
Datasets and analyses keep the full filename; only the observation label
|
|
515
|
+
inside the analysis is shorter.
|
|
516
|
+
Estrus_vs_2dpp_Glandular_epithelium_pseudobulk_t_significant_p0.05_rna
|
|
517
|
+
-> Estrus_vs_2dpp_Glandular_epithelium
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
If your pipeline's suffix is not recognised, name it explicitly. `--strip` is
|
|
521
|
+
repeatable, applied before anything else, and ignored as a whole if it would
|
|
522
|
+
leave the names unusable:
|
|
523
|
+
|
|
524
|
+
```bash
|
|
525
|
+
ipaapi submit ~/data --ID 1:mousesymeg --FC 4:foldchange --skip-rows 1 \
|
|
526
|
+
--project Study --strip _significant_p0.05_rna
|
|
527
|
+
```
|
|
528
|
+
|
|
529
|
+
Because steps 4 and 5 depend on the other files in the run, **the same file
|
|
530
|
+
submitted in a different batch can get a different observation label**. Steps 1 to 3
|
|
531
|
+
do not — they read only the name in front of them, which is why a
|
|
532
|
+
single-file submit shortens as well as a batch does. The dataset and analysis
|
|
533
|
+
always keep the full filename, so `ipaapi history` and the IPA project view are
|
|
534
|
+
unaffected.
|
|
535
|
+
|
|
536
|
+
`--observation` sets the name explicitly for a single file and is shortened the
|
|
537
|
+
same way if it needs to be.
|
|
538
|
+
|
|
397
539
|
### The reference set
|
|
398
540
|
|
|
399
541
|
The background enrichment is scored against — the denominator of the Fisher's
|
|
@@ -476,6 +618,7 @@ boilerplate and the page footer, and classifies what's left:
|
|
|
476
618
|
| `Unknown GeneId Type (X)` | `MalformedRequestError` | stops; names the flag; moves nothing |
|
|
477
619
|
| `Unable to run analysis: Analysis limit exceeded` | `QuotaExceededError` | stops; leaves remaining files for the next run |
|
|
478
620
|
| `Unable to run analysis: …` (other) | `AnalysisRefusedError` | as above — reached the analysis logic, so not a parameter fault |
|
|
621
|
+
| `currently unavailable` / `technical difficulties`, or 502/503/504 | `ServiceUnavailableError` | IPA is down; stops, moves nothing, says the command is fine |
|
|
479
622
|
| anything else | `SubmissionError` | files that one under `failed/` |
|
|
480
623
|
|
|
481
624
|
Quota matching is deliberately broad (`ipaapi.client.QUOTA_PATTERNS` plus HTTP
|
|
@@ -642,6 +785,10 @@ cp, ur, df = results # unpacks like the demo's ipa_results()
|
|
|
642
785
|
| `--FC refers to column N, but the file has only M column(s)` | 1-based counting, or wrong `--skip-rows` | positions are 0-based, from the header |
|
|
643
786
|
| `Every row is missing an identifier` | wrong column, or no header | check with `head -1 file \| tr '\t' '\n' \| nl -v0` |
|
|
644
787
|
| `the analysis allowance appears to be exhausted` | daily/period limit | re-run later; files left in place resume |
|
|
788
|
+
| `IPA appears to be down or having trouble` | IPA outage — **or a duplicate dataset name, or a long observation name**, all reported identically | `ipaapi --version` (1.2.0+ handles the name length); check `ipaapi history --project X` for the dataset name; otherwise wait |
|
|
789
|
+
| Batch dies on the first file, names come from long filenames | observation name too long | upgrade to 1.2.0+, or pass a short `--observation` |
|
|
790
|
+
| Batch dies on the *first* file after an earlier run | dataset names already exist in the project | expected — 1.1.0 skips them automatically; before that, use a new `--project` |
|
|
791
|
+
| `Cannot listen on 127.0.0.1:8000` | stale login process, or another user mid-login | `ss -ltnp 'sport = :8000'`, then kill it if it's yours |
|
|
645
792
|
| Login prompt on every run | token cache not writable | `export IPAAPI_TOKEN_FILE=...`; check for a root-owned cache |
|
|
646
793
|
| `Could not open a browser automatically` | headless | `ssh -X`, or copy a token across |
|
|
647
794
|
| `report` returns HTTP 500 on a succeeded analysis | unconfirmed; possibly add-on licence | open the analysis in IPA; see `examples/probe_interpret.py` |
|
|
@@ -56,7 +56,7 @@ from .models import GENE_ID_TYPES, AnalysisStatus, MeasurementType, ReferenceSet
|
|
|
56
56
|
|
|
57
57
|
#: Single source of truth for the package version; pyproject.toml reads it from
|
|
58
58
|
#: here at build time. Bump it in this file and nowhere else.
|
|
59
|
-
__version__ = "1.
|
|
59
|
+
__version__ = "1.2.0"
|
|
60
60
|
|
|
61
61
|
__all__ = [
|
|
62
62
|
"__version__",
|