@extantape/onephylogeny-acquire 0.0.0-stage → 0.1.0-alpha.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +75 -0
- package/README.md +30 -2
- package/dist/index.d.ts +474 -0
- package/dist/index.js +792 -0
- package/package.json +43 -3
package/LICENSE
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
Required Notice: Copyright 2026 EXTANT APE (https://github.com/ExtantApe)
|
|
2
|
+
|
|
3
|
+
# PolyForm Noncommercial License 1.0.0
|
|
4
|
+
|
|
5
|
+
<https://polyformproject.org/licenses/noncommercial/1.0.0>
|
|
6
|
+
|
|
7
|
+
## Acceptance
|
|
8
|
+
|
|
9
|
+
In order to get any license under these terms, you must agree to them as both strict obligations and conditions to all your licenses.
|
|
10
|
+
|
|
11
|
+
## Copyright License
|
|
12
|
+
|
|
13
|
+
The licensor grants you a copyright license for the software to do everything you might do with the software that would otherwise infringe the licensor's copyright in it for any permitted purpose. However, you may only distribute the software according to [Distribution License](#distribution-license) and make changes or new works based on the software according to [Changes and New Works License](#changes-and-new-works-license).
|
|
14
|
+
|
|
15
|
+
## Distribution License
|
|
16
|
+
|
|
17
|
+
The licensor grants you an additional copyright license to distribute copies of the software. Your license to distribute covers distributing the software with changes and new works permitted by [Changes and New Works License](#changes-and-new-works-license).
|
|
18
|
+
|
|
19
|
+
## Notices
|
|
20
|
+
|
|
21
|
+
You must ensure that anyone who gets a copy of any part of the software from you also gets a copy of these terms or the URL for them above, as well as copies of any plain-text lines beginning with `Required Notice:` that the licensor provided with the software. For example:
|
|
22
|
+
|
|
23
|
+
> Required Notice: Copyright Yoyodyne, Inc. (http://example.com)
|
|
24
|
+
|
|
25
|
+
## Changes and New Works License
|
|
26
|
+
|
|
27
|
+
The licensor grants you an additional copyright license to make changes and new works based on the software for any permitted purpose.
|
|
28
|
+
|
|
29
|
+
## Patent License
|
|
30
|
+
|
|
31
|
+
The licensor grants you a patent license for the software that covers patent claims the licensor can license, or becomes able to license, that you would infringe by using the software.
|
|
32
|
+
|
|
33
|
+
## Noncommercial Purposes
|
|
34
|
+
|
|
35
|
+
Any noncommercial purpose is a permitted purpose.
|
|
36
|
+
|
|
37
|
+
## Personal Uses
|
|
38
|
+
|
|
39
|
+
Personal use for research, experiment, and testing for the benefit of public knowledge, personal study, private entertainment, hobby projects, amateur pursuits, or religious observance, without any anticipated commercial application, is use for a permitted purpose.
|
|
40
|
+
|
|
41
|
+
## Noncommercial Organizations
|
|
42
|
+
|
|
43
|
+
Use by any charitable organization, educational institution, public research organization, public safety or health organization, environmental protection organization, or government institution is use for a permitted purpose regardless of the source of funding or obligations resulting from the funding.
|
|
44
|
+
|
|
45
|
+
## Fair Use
|
|
46
|
+
|
|
47
|
+
You may have "fair use" rights for the software under the law. These terms do not limit them.
|
|
48
|
+
|
|
49
|
+
## No Other Rights
|
|
50
|
+
|
|
51
|
+
These terms do not allow you to sublicense or transfer any of your licenses to anyone else, or prevent the licensor from granting licenses to anyone else. These terms do not imply any other licenses.
|
|
52
|
+
|
|
53
|
+
## Patent Defense
|
|
54
|
+
|
|
55
|
+
If you make any written claim that the software infringes or contributes to infringement of any patent, your patent license for the software granted under these terms ends immediately. If your company makes such a claim, your patent license ends immediately for work on behalf of your company.
|
|
56
|
+
|
|
57
|
+
## Violations
|
|
58
|
+
|
|
59
|
+
The first time you are notified in writing that you have violated any of these terms, or done anything with the software not covered by your licenses, your licenses can nonetheless continue if you come into full compliance with these terms, and take practical steps to correct past violations, within 32 days of receiving notice. Otherwise, all your licenses end immediately.
|
|
60
|
+
|
|
61
|
+
## No Liability
|
|
62
|
+
|
|
63
|
+
***As far as the law allows, the software comes as is, without any warranty or condition, and the licensor will not be liable to you for any damages arising out of these terms or the use or nature of the software, under any kind of legal claim.***
|
|
64
|
+
|
|
65
|
+
## Definitions
|
|
66
|
+
|
|
67
|
+
The **licensor** is the individual or entity offering these terms, and the **software** is the software the licensor makes available under these terms.
|
|
68
|
+
|
|
69
|
+
**You** refers to the individual or entity agreeing to these terms.
|
|
70
|
+
|
|
71
|
+
**Your company** is any legal entity, sole proprietorship, or other kind of organization that you work for, plus all organizations that have control over, are under the control of, or are under common control with that organization. **Control** means ownership of substantially all the assets of an entity, or the power to direct its management and policies by vote, contract, or otherwise. Control can be direct or indirect.
|
|
72
|
+
|
|
73
|
+
**Your licenses** are all the licenses granted to you for the software under these terms.
|
|
74
|
+
|
|
75
|
+
**Use** means anything you do with the software requiring one of your licenses.
|
package/README.md
CHANGED
|
@@ -1,3 +1,31 @@
|
|
|
1
|
-
#
|
|
1
|
+
# @extantape/onephylogeny-acquire
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
OnePhylogeny acquire: fetches sequence records by taxon and marker with safe retries and no duplicates, keeps raw files with hashes, and suggests outgroups outward through the taxonomy.
|
|
4
|
+
|
|
5
|
+
See the [repository](https://github.com/ExtantApe/OnePhylogeny) for documentation.
|
|
6
|
+
|
|
7
|
+
## More sources
|
|
8
|
+
|
|
9
|
+
Listings and counts from other services (roadmap stage 6.2). Nothing here downloads reads, genomes or individual occurrence records. Each result carries `source`, `retrieved` (ISO time), `query` (a URL a person can open) and `requests` (every request made).
|
|
10
|
+
|
|
11
|
+
| Function | Service | Returns |
|
|
12
|
+
| --- | --- | --- |
|
|
13
|
+
| `listSraRuns(client, taxon)` | NCBI SRA (E-utilities) | One row per run: run, experiment, study, BioProject, BioSample, library strategy, source, selection, layout, platform, instrument, bases, spots. `experiments.total` against `experiments.listed` shows a list cut short by `limit` (500 by default). |
|
|
14
|
+
| `biosampleFor(client, accessions)` | NCBI BioSample (E-utilities) | Organism, taxon and a short list of attributes (isolate, sex, tissue, `geo_loc_name`, collection date and a few more). Accessions not found are in `missing`. |
|
|
15
|
+
| `datasetsGenomes(fetch, taxon)` | NCBI Datasets v2 | Genome assemblies: accession, GenBank or RefSeq, paired accession, assembly level, length, release date, organelles listed, and `mitogenome: 'listed' \| 'not listed'`. |
|
|
16
|
+
| `occurrenceCoverage(fetch, name)` | GBIF and iDigBio | Record counts by country and by basis of record. GBIF also gives counts by licence and the number of datasets. |
|
|
17
|
+
| `boldCoverage(fetch, name)` | BOLD portal | Record counts by country or ocean, and by marker. |
|
|
18
|
+
|
|
19
|
+
A taxon for the NCBI functions is a Taxonomy ID (digits) or a scientific name, subtaxa included. GBIF, iDigBio and BOLD take a scientific name.
|
|
20
|
+
|
|
21
|
+
What it won't do:
|
|
22
|
+
|
|
23
|
+
- **No coordinates.** BioSample attributes named like coordinates (`lat_lon`, `decimal_latitude` and so on), and any value shaped like one, are dropped as they're parsed and never returned or kept. `withheld` lists the attribute names left out, never their values. The occurrence counts never ask for a record, so no coordinate is fetched. Contacts, submitters and owners are never read.
|
|
24
|
+
- **No guessed matches.** A name GBIF only matches at a higher rank is refused rather than counted for the genus. A fuzzy match is counted and says so in `notes`. A name BOLD can't resolve is refused.
|
|
25
|
+
- **No hidden gaps.** `occurrenceCoverage` lists a service that failed in `failed`, so a missing count never reads as zero. GBIF and iDigBio overlap, because iDigBio publishes to GBIF: don't add them together. iDigBio doesn't index a licence per record, so its `licences` is `null`. BOLD's summary gives no total or licence, so neither is reported.
|
|
26
|
+
|
|
27
|
+
Etiquette: Datasets, GBIF, iDigBio and BOLD calls are paced inside each function, one request at a time and at least a second apart, with a User-Agent naming this tool and its repository and never a person. A refusal (HTTP 403 or 429) is reported and not retried. The E-utilities calls go through `NcbiClient`, which keeps to NCBI's three requests a second and retries a few times with growing waits before giving up. The reported `query` never includes an email or API key.
|
|
28
|
+
|
|
29
|
+
`publicListings({ fetch, client }, taxon, { only })` runs the listings above for one taxon, one after another, with one retrieval time. A Taxonomy ID is named by NCBI Taxonomy first, since GBIF, iDigBio and BOLD take names; if it can't be named, they're listed as failed rather than asked about the digits. A service that fails is in `failed` by name and its slot is `null`. The `onephylogeny sources` command and the Mac app's Public data section use it.
|
|
30
|
+
|
|
31
|
+
Dryad, Figshare and Zenodo aren't searched: they stay flagged for a person to review.
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1,474 @@
|
|
|
1
|
+
import { StudyPaper, Selection, GenBankRecord, LinkedRecord, FieldSource, Marker } from '@extantape/onephylogeny-core';
|
|
2
|
+
|
|
3
|
+
/** A paper as a search found it. Nothing joins a Study until someone adds it. */
|
|
4
|
+
type FoundPaper = StudyPaper & {
|
|
5
|
+
/** Another listed entry whose title matches. A person decides, so records never attach to the wrong paper. */
|
|
6
|
+
possibleSameAs?: string[];
|
|
7
|
+
};
|
|
8
|
+
type PaperSearch = {
|
|
9
|
+
papers: FoundPaper[];
|
|
10
|
+
counts: Record<string, number>;
|
|
11
|
+
/** How many each source has in all. Where it's more than the count, the list was cut short. */
|
|
12
|
+
totals: Record<string, number>;
|
|
13
|
+
sources: string[];
|
|
14
|
+
failed: string[];
|
|
15
|
+
};
|
|
16
|
+
type Listed = {
|
|
17
|
+
papers: FoundPaper[];
|
|
18
|
+
total: number;
|
|
19
|
+
};
|
|
20
|
+
declare function searchEuropePmc(fetch: FetchLike, query: string, pageSize?: number): Promise<Listed>;
|
|
21
|
+
declare function searchPubMed(client: NcbiClient, query: string, retmax?: number): Promise<Listed>;
|
|
22
|
+
/** Records under one publication title, for the many that have no PubMed link. One submission can share a paper's title. */
|
|
23
|
+
declare function studiesFromRecords(records: GenBankRecord[], describe: (r: GenBankRecord) => LinkedRecord): FoundPaper[];
|
|
24
|
+
/**
|
|
25
|
+
* Joins lists on exact PMID or DOI. Matching titles are only marked: a person decides those, so records never attach to the wrong paper.
|
|
26
|
+
*/
|
|
27
|
+
declare function mergePapers(lists: FoundPaper[][]): FoundPaper[];
|
|
28
|
+
/** Searches every source and reports any that failed, because a partial list mustn't look complete. */
|
|
29
|
+
declare function searchPapers(client: NcbiClient, fetch: FetchLike, query: string, records?: FoundPaper[]): Promise<PaperSearch>;
|
|
30
|
+
/** Nucleotide records PubMed links to a paper. Most records have no such link, so an empty list doesn't mean the paper has none. */
|
|
31
|
+
declare function linkedAccessions(client: NcbiClient, pmid: string): Promise<string[]>;
|
|
32
|
+
/** Which accessions are of the target taxa, and which hold the marker. Whole mitochondrial genomes hold it even when unannotated. */
|
|
33
|
+
declare function classifyAccessions(client: NcbiClient, accessions: string[], selection: Selection): Promise<{
|
|
34
|
+
inTarget: Set<string>;
|
|
35
|
+
hasMarker: Set<string>;
|
|
36
|
+
}>;
|
|
37
|
+
|
|
38
|
+
/**
|
|
39
|
+
* Paper lookup (roadmap stage 3): for a study known only by its GenBank title, find the published paper's DOI and
|
|
40
|
+
* PubMed IDs in Crossref and Europe PMC, and the paper's open supplements. Every result is a proposal with its source
|
|
41
|
+
* and how well the title matched; nothing is attached until a person approves it, and the approval records who and when.
|
|
42
|
+
*
|
|
43
|
+
* Both services are free and keyless. We keep to their published etiquette: one request at a time, at least a second
|
|
44
|
+
* apart by default, a User-Agent naming this tool and its repository (no personal email), and no retries on refusal.
|
|
45
|
+
*/
|
|
46
|
+
type LinkProposal = {
|
|
47
|
+
service: 'crossref' | 'europepmc';
|
|
48
|
+
title: string;
|
|
49
|
+
year?: number;
|
|
50
|
+
journal?: string;
|
|
51
|
+
doi?: string;
|
|
52
|
+
pmid?: string;
|
|
53
|
+
pmcid?: string;
|
|
54
|
+
/** Shared title words over all title words (0–1), and whether the titles are the same after normalising. */
|
|
55
|
+
titleMatch: number;
|
|
56
|
+
exactTitle: boolean;
|
|
57
|
+
/** Europe PMC's list of a paper's supplementary files, for open-access papers with a PMC ID. A link, not a download. */
|
|
58
|
+
supplementsUrl?: string;
|
|
59
|
+
source: FieldSource;
|
|
60
|
+
};
|
|
61
|
+
type ApprovedLink = Pick<LinkProposal, 'doi' | 'pmid'> & {
|
|
62
|
+
source: FieldSource & {
|
|
63
|
+
approvedBy?: string;
|
|
64
|
+
approvedAt: string;
|
|
65
|
+
};
|
|
66
|
+
};
|
|
67
|
+
declare const LOOKUP_USER_AGENT = "OnePhylogeny (https://github.com/ExtantApe/OnePhylogeny)";
|
|
68
|
+
declare function titleMatch(a: string, b: string): {
|
|
69
|
+
score: number;
|
|
70
|
+
exact: boolean;
|
|
71
|
+
};
|
|
72
|
+
/** Sends requests one at a time, at least `gapMs` apart. */
|
|
73
|
+
declare function paced(fetch: FetchLike, gapMs?: number, sleep?: (ms: number) => Promise<void>, now?: () => number): FetchLike;
|
|
74
|
+
declare function lookupCrossref(fetch: FetchLike, paper: Pick<StudyPaper, 'title' | 'authors'>, at: string, rows?: number): Promise<LinkProposal[]>;
|
|
75
|
+
declare function lookupEuropePmc(fetch: FetchLike, paper: Pick<StudyPaper, 'title'>, at: string, pageSize?: number): Promise<LinkProposal[]>;
|
|
76
|
+
/**
|
|
77
|
+
* Both services, best title match first. A service that fails is listed in `failed`, so a short list never looks
|
|
78
|
+
* complete. Proposals below `minMatch` are dropped as noise.
|
|
79
|
+
*/
|
|
80
|
+
declare function lookupPaper(fetch: FetchLike, paper: Pick<StudyPaper, 'title' | 'authors'>, options?: {
|
|
81
|
+
at?: string;
|
|
82
|
+
minMatch?: number;
|
|
83
|
+
}): Promise<{
|
|
84
|
+
proposals: LinkProposal[];
|
|
85
|
+
failed: string[];
|
|
86
|
+
}>;
|
|
87
|
+
/** A person's approval of one proposal: the paper gains its DOI and PMID, each with where it came from, who and when. */
|
|
88
|
+
declare function approveLink(paper: StudyPaper, proposal: LinkProposal, meta: {
|
|
89
|
+
by?: string;
|
|
90
|
+
at: string;
|
|
91
|
+
}): StudyPaper & {
|
|
92
|
+
linkSource: ApprovedLink;
|
|
93
|
+
};
|
|
94
|
+
|
|
95
|
+
/**
|
|
96
|
+
* More record sources (roadmap stage 6.2): listings and counts only. Nothing here downloads reads, genomes or
|
|
97
|
+
* individual occurrence records, and nothing returns or keeps coordinates.
|
|
98
|
+
*
|
|
99
|
+
* Etiquette is the same as the paper lookup's: one request at a time, at least a second apart (NCBI E-utilities go
|
|
100
|
+
* through `NcbiClient`, which keeps to NCBI's documented three a second), a User-Agent naming this tool and its
|
|
101
|
+
* repository and never a person, and a refusal (HTTP 403 or 429) is reported, never retried.
|
|
102
|
+
*/
|
|
103
|
+
/** Where a listing came from: the service, when, and the request that produced it. */
|
|
104
|
+
type Provenance = {
|
|
105
|
+
source: string;
|
|
106
|
+
/** ISO 8601 time the listing was retrieved. */
|
|
107
|
+
retrieved: string;
|
|
108
|
+
/** The main request, as a URL a person can open. Never carries an email or API key. */
|
|
109
|
+
query: string;
|
|
110
|
+
/** Every request made, in order. */
|
|
111
|
+
requests: string[];
|
|
112
|
+
};
|
|
113
|
+
type SourceOptions = {
|
|
114
|
+
/** Retrieval time to record. Defaults to now. */
|
|
115
|
+
at?: string;
|
|
116
|
+
/** Pause between this call's requests. A second unless set higher. */
|
|
117
|
+
gapMs?: number;
|
|
118
|
+
sleep?: (ms: number) => Promise<void>;
|
|
119
|
+
now?: () => number;
|
|
120
|
+
};
|
|
121
|
+
/** A taxon for NCBI: digits are a Taxonomy ID, anything else a scientific name. Subtaxa are included. */
|
|
122
|
+
declare function ncbiTaxonTerm(taxon: string): string;
|
|
123
|
+
type SraRun = {
|
|
124
|
+
run: string;
|
|
125
|
+
experiment?: string;
|
|
126
|
+
study?: string;
|
|
127
|
+
bioProject?: string;
|
|
128
|
+
bioSample?: string;
|
|
129
|
+
organism?: string;
|
|
130
|
+
taxid?: string;
|
|
131
|
+
/** WGS, RNA-Seq, AMPLICON, RAD-Seq and so on, as SRA has it. */
|
|
132
|
+
libraryStrategy?: string;
|
|
133
|
+
librarySource?: string;
|
|
134
|
+
librarySelection?: string;
|
|
135
|
+
layout?: 'PAIRED' | 'SINGLE';
|
|
136
|
+
platform?: string;
|
|
137
|
+
instrument?: string;
|
|
138
|
+
bases?: number;
|
|
139
|
+
spots?: number;
|
|
140
|
+
/** As SRA gives it, YYYY/MM/DD. */
|
|
141
|
+
published?: string;
|
|
142
|
+
};
|
|
143
|
+
type SraListing = Provenance & {
|
|
144
|
+
taxon: string;
|
|
145
|
+
/** Experiments SRA has for the search, and how many were read. Where `listed` is less, the list was cut short. */
|
|
146
|
+
experiments: {
|
|
147
|
+
total: number;
|
|
148
|
+
listed: number;
|
|
149
|
+
};
|
|
150
|
+
runs: SraRun[];
|
|
151
|
+
};
|
|
152
|
+
/**
|
|
153
|
+
* Parses one SRA esummary document into its runs. Submitter, contact, titles and library names and protocols are
|
|
154
|
+
* never read, so they can't be passed on.
|
|
155
|
+
*/
|
|
156
|
+
declare function parseSraSummary(doc: {
|
|
157
|
+
expxml?: string;
|
|
158
|
+
runs?: string;
|
|
159
|
+
createdate?: string;
|
|
160
|
+
}): SraRun[];
|
|
161
|
+
/**
|
|
162
|
+
* SRA runs for a taxon and its subtaxa: run accession, library strategy, layout, platform, bases, BioProject and
|
|
163
|
+
* BioSample. A listing only; no reads are downloaded. `limit` caps the experiments read, and the result says how many
|
|
164
|
+
* there were in all.
|
|
165
|
+
*/
|
|
166
|
+
declare function listSraRuns(client: NcbiClient, taxon: string, options?: {
|
|
167
|
+
limit?: number;
|
|
168
|
+
at?: string;
|
|
169
|
+
}): Promise<SraListing>;
|
|
170
|
+
/** Attributes passed on. Anything else (contacts, owners, providers, free-text notes, coordinates) is never read. */
|
|
171
|
+
declare const BIOSAMPLE_ATTRIBUTES: readonly ["isolate", "breed", "sex", "dev_stage", "tissue", "sample_type", "specimen_voucher", "geo_loc_name", "country", "collection_date", "isolation_source", "host"];
|
|
172
|
+
declare function looksLikeCoordinates(value: string): boolean;
|
|
173
|
+
type BioSample = {
|
|
174
|
+
accession: string;
|
|
175
|
+
organism?: string;
|
|
176
|
+
taxid?: string;
|
|
177
|
+
/** As BioSample gives it, YYYY/MM/DD. */
|
|
178
|
+
published?: string;
|
|
179
|
+
package?: string;
|
|
180
|
+
/** Only the names in `BIOSAMPLE_ATTRIBUTES`, keyed by harmonized name. */
|
|
181
|
+
attributes: Partial<Record<(typeof BIOSAMPLE_ATTRIBUTES)[number], string>>;
|
|
182
|
+
/** Attributes left out because they hold coordinates. Names only; the values are never kept. */
|
|
183
|
+
withheld: string[];
|
|
184
|
+
};
|
|
185
|
+
type BioSampleListing = Provenance & {
|
|
186
|
+
samples: BioSample[];
|
|
187
|
+
/** Accessions asked for that BioSample didn't return. */
|
|
188
|
+
missing: string[];
|
|
189
|
+
};
|
|
190
|
+
/** Parses one BioSample esummary document. Coordinates are dropped here, so nothing after this can see them. */
|
|
191
|
+
declare function parseBioSample(doc: Record<string, any>): BioSample;
|
|
192
|
+
/**
|
|
193
|
+
* BioSamples by accession (SAMN, SAME, SAMD), with a short list of descriptive attributes. Coordinates (`lat_lon` and
|
|
194
|
+
* the like, or any value shaped like one) are dropped at parse time and never returned or stored; `withheld` names
|
|
195
|
+
* what was left out. Contacts and owners are never read.
|
|
196
|
+
*/
|
|
197
|
+
declare function biosampleFor(client: NcbiClient, accessions: string[], options?: {
|
|
198
|
+
at?: string;
|
|
199
|
+
}): Promise<BioSampleListing>;
|
|
200
|
+
type GenomeSummary = {
|
|
201
|
+
accession: string;
|
|
202
|
+
/** GenBank (GCA) or RefSeq (GCF). */
|
|
203
|
+
database: 'genbank' | 'refseq' | 'unknown';
|
|
204
|
+
/** The same assembly in the other database, if any. */
|
|
205
|
+
pairedAccession?: string;
|
|
206
|
+
organism?: string;
|
|
207
|
+
taxid?: string;
|
|
208
|
+
assemblyName?: string;
|
|
209
|
+
/** Contig, Scaffold, Chromosome or Complete Genome. */
|
|
210
|
+
assemblyLevel?: string;
|
|
211
|
+
status?: string;
|
|
212
|
+
/** Total sequence length in bases. */
|
|
213
|
+
length?: number;
|
|
214
|
+
released?: string;
|
|
215
|
+
/** Organelles the report lists, such as Mitochondrion. */
|
|
216
|
+
organelles: string[];
|
|
217
|
+
/**
|
|
218
|
+
* Whether this report lists a mitochondrial sequence. `not listed` doesn't mean the assembly has none: one of a
|
|
219
|
+
* GenBank and RefSeq pair often lists it when the other doesn't, so check `pairedAccession` too.
|
|
220
|
+
*/
|
|
221
|
+
mitogenome: 'listed' | 'not listed';
|
|
222
|
+
};
|
|
223
|
+
type GenomeListing = Provenance & {
|
|
224
|
+
taxon: string;
|
|
225
|
+
total: number;
|
|
226
|
+
genomes: GenomeSummary[];
|
|
227
|
+
};
|
|
228
|
+
/** One NCBI Datasets genome report. Only these fields are read; the embedded BioSample (with any coordinates) is not. */
|
|
229
|
+
declare function parseGenomeReport(r: Record<string, any>): GenomeSummary;
|
|
230
|
+
/**
|
|
231
|
+
* Genome assemblies NCBI Datasets has for a taxon (an NCBI Taxonomy ID or a name), with subtaxa: accession, level,
|
|
232
|
+
* size and whether a mitogenome is listed. A listing only; nothing is downloaded. Reads up to `maxPages` pages.
|
|
233
|
+
*/
|
|
234
|
+
declare function datasetsGenomes(fetch: FetchLike, taxon: string, options?: SourceOptions & {
|
|
235
|
+
pageSize?: number;
|
|
236
|
+
maxPages?: number;
|
|
237
|
+
}): Promise<GenomeListing>;
|
|
238
|
+
type Count = {
|
|
239
|
+
name: string;
|
|
240
|
+
count: number;
|
|
241
|
+
};
|
|
242
|
+
type Coverage = Provenance & {
|
|
243
|
+
/** The taxon as the service matched it. */
|
|
244
|
+
matched: {
|
|
245
|
+
name: string;
|
|
246
|
+
key?: string;
|
|
247
|
+
rank?: string;
|
|
248
|
+
matchType?: string;
|
|
249
|
+
};
|
|
250
|
+
total: number;
|
|
251
|
+
/** Country as the service gives it: ISO codes for GBIF, verbatim lowercase text for iDigBio, names for BOLD. */
|
|
252
|
+
byCountry: Count[];
|
|
253
|
+
/** Records with no country, where the service makes that knowable. */
|
|
254
|
+
noCountry?: number;
|
|
255
|
+
byBasisOfRecord: Count[];
|
|
256
|
+
/** Licences of the records counted, with counts. `null` where the service doesn't say. */
|
|
257
|
+
licences: Count[] | null;
|
|
258
|
+
datasets?: number;
|
|
259
|
+
notes: string[];
|
|
260
|
+
};
|
|
261
|
+
type OccurrenceCoverage = {
|
|
262
|
+
taxon: string;
|
|
263
|
+
gbif: Coverage | null;
|
|
264
|
+
idigbio: Coverage | null;
|
|
265
|
+
failed: string[];
|
|
266
|
+
};
|
|
267
|
+
/**
|
|
268
|
+
* GBIF occurrence counts for a scientific name: by country, by basis of record and by licence, with how many datasets
|
|
269
|
+
* they come from. Two requests, the name match and one search with `limit=0`, so no record (and no coordinate) is
|
|
270
|
+
* ever fetched. A name GBIF can't match exactly to a species or lower is refused, never counted at a higher rank.
|
|
271
|
+
*/
|
|
272
|
+
declare function gbifCoverage(fetch: FetchLike, name: string, options?: SourceOptions): Promise<Coverage>;
|
|
273
|
+
/**
|
|
274
|
+
* iDigBio record counts for a scientific name, by country and basis of record, from its summary endpoint: no record
|
|
275
|
+
* is fetched. iDigBio doesn't index a licence per record, so `licences` is `null` rather than guessed.
|
|
276
|
+
*/
|
|
277
|
+
declare function idigbioCoverage(fetch: FetchLike, name: string, options?: SourceOptions): Promise<Coverage>;
|
|
278
|
+
/**
|
|
279
|
+
* Where a taxon has been recorded: GBIF and, unless turned off, iDigBio. Counts only, never coordinates or records.
|
|
280
|
+
* A service that fails is listed in `failed`, so a missing count never reads as zero. iDigBio publishes to GBIF, so
|
|
281
|
+
* the two overlap and shouldn't be added together.
|
|
282
|
+
*/
|
|
283
|
+
declare function occurrenceCoverage(fetch: FetchLike, name: string, options?: SourceOptions & {
|
|
284
|
+
idigbio?: boolean;
|
|
285
|
+
}): Promise<OccurrenceCoverage>;
|
|
286
|
+
type BoldCoverage = Provenance & {
|
|
287
|
+
matched: string;
|
|
288
|
+
/** Records by country or ocean, as BOLD names them, including its own "Unrecoverable". */
|
|
289
|
+
byCountry: Count[];
|
|
290
|
+
/** Records by marker, such as COI-5P. A record can carry more than one. */
|
|
291
|
+
byMarker: Count[];
|
|
292
|
+
notes: string[];
|
|
293
|
+
};
|
|
294
|
+
/**
|
|
295
|
+
* BOLD record counts for a taxon by country and by marker, from the public portal's documented summary endpoint. Two
|
|
296
|
+
* requests: resolving the name, then the summary. No records or coordinates are fetched.
|
|
297
|
+
*/
|
|
298
|
+
declare function boldCoverage(fetch: FetchLike, name: string, options?: SourceOptions): Promise<BoldCoverage>;
|
|
299
|
+
|
|
300
|
+
/**
|
|
301
|
+
* Every record-source listing for one taxon in one call, for the CLI's `sources` command and the Mac app's Public data
|
|
302
|
+
* section. It only calls the functions in `sources.ts`, one after another, and changes none of what they do. A service
|
|
303
|
+
* that fails is named in `failed`, and its slot stays `null`, so a missing count never reads as zero.
|
|
304
|
+
*/
|
|
305
|
+
type ListingKind = 'sra' | 'genomes' | 'occurrences' | 'bold';
|
|
306
|
+
declare const LISTING_KINDS: readonly ListingKind[];
|
|
307
|
+
type ListingFailure = {
|
|
308
|
+
service: string;
|
|
309
|
+
message: string;
|
|
310
|
+
};
|
|
311
|
+
type PublicListings = {
|
|
312
|
+
/** The taxon as given: an NCBI Taxonomy ID or a scientific name. */
|
|
313
|
+
taxon: string;
|
|
314
|
+
/**
|
|
315
|
+
* The scientific name GBIF, iDigBio and BOLD were asked about: as given, or NCBI Taxonomy's name for a Taxonomy ID.
|
|
316
|
+
* `null` when none of them was asked, or the ID couldn't be named.
|
|
317
|
+
*/
|
|
318
|
+
name: string | null;
|
|
319
|
+
nameFrom: 'given' | 'ncbi-taxonomy' | null;
|
|
320
|
+
/** What was asked for, in order. A kind not asked for stays `null` and isn't a failure. */
|
|
321
|
+
asked: ListingKind[];
|
|
322
|
+
retrieved: string;
|
|
323
|
+
sra: SraListing | null;
|
|
324
|
+
genomes: GenomeListing | null;
|
|
325
|
+
gbif: Coverage | null;
|
|
326
|
+
idigbio: Coverage | null;
|
|
327
|
+
bold: BoldCoverage | null;
|
|
328
|
+
failed: ListingFailure[];
|
|
329
|
+
};
|
|
330
|
+
declare function publicListings(services: {
|
|
331
|
+
fetch: FetchLike;
|
|
332
|
+
client: NcbiClient;
|
|
333
|
+
}, taxon: string, options?: SourceOptions & {
|
|
334
|
+
only?: readonly ListingKind[];
|
|
335
|
+
sraLimit?: number;
|
|
336
|
+
}): Promise<PublicListings>;
|
|
337
|
+
|
|
338
|
+
type FetchLike = (url: string, init?: {
|
|
339
|
+
method?: string;
|
|
340
|
+
headers?: Record<string, string>;
|
|
341
|
+
body?: string;
|
|
342
|
+
}) => Promise<{
|
|
343
|
+
ok: boolean;
|
|
344
|
+
status: number;
|
|
345
|
+
text(): Promise<string>;
|
|
346
|
+
}>;
|
|
347
|
+
type NcbiOptions = {
|
|
348
|
+
fetch: FetchLike;
|
|
349
|
+
/** Sent to NCBI so it can identify the client. */
|
|
350
|
+
tool?: string;
|
|
351
|
+
email?: string;
|
|
352
|
+
apiKey?: string;
|
|
353
|
+
/** Pause between requests. NCBI asks for at most 3 a second without a key. */
|
|
354
|
+
delayMs?: number;
|
|
355
|
+
retries?: number;
|
|
356
|
+
sleep?: (ms: number) => Promise<void>;
|
|
357
|
+
log?: (level: 'info' | 'warning', message: string, data?: Record<string, unknown>) => void;
|
|
358
|
+
};
|
|
359
|
+
declare class AcquireError extends Error {
|
|
360
|
+
}
|
|
361
|
+
/** A small NCBI E-utilities client. Failed requests retry, then throw: nothing is skipped silently. */
|
|
362
|
+
declare class NcbiClient {
|
|
363
|
+
private readonly o;
|
|
364
|
+
constructor(options: NcbiOptions);
|
|
365
|
+
request(path: string, params: Record<string, string>): Promise<string>;
|
|
366
|
+
private json;
|
|
367
|
+
/** Accession versions for a search, every page of them. A short read is an error. */
|
|
368
|
+
searchAccessions(term: string, page?: number): Promise<string[]>;
|
|
369
|
+
count(term: string): Promise<number>;
|
|
370
|
+
/** Raw GenBank text for these accessions, checked to contain exactly what was asked for. */
|
|
371
|
+
fetchGenBankText(ids: string[]): Promise<{
|
|
372
|
+
text: string;
|
|
373
|
+
records: GenBankRecord[];
|
|
374
|
+
}>;
|
|
375
|
+
summaries(term: string, retmax?: number): Promise<{
|
|
376
|
+
accession: string;
|
|
377
|
+
organism: string;
|
|
378
|
+
length: number;
|
|
379
|
+
title: string;
|
|
380
|
+
taxid?: string;
|
|
381
|
+
}[]>;
|
|
382
|
+
/** Finds a taxon by scientific name. More than one result is returned for a person to choose from. */
|
|
383
|
+
findTaxon(name: string): Promise<{
|
|
384
|
+
id: string;
|
|
385
|
+
scientificName: string;
|
|
386
|
+
}[]>;
|
|
387
|
+
/** A taxon's ancestors, nearest first. */
|
|
388
|
+
lineage(taxid: string): Promise<{
|
|
389
|
+
scientificName: string;
|
|
390
|
+
ancestors: TaxonStep[];
|
|
391
|
+
mitoGeneticCode?: number;
|
|
392
|
+
}>;
|
|
393
|
+
}
|
|
394
|
+
type TaxonStep = {
|
|
395
|
+
id: string;
|
|
396
|
+
name: string;
|
|
397
|
+
rank: string;
|
|
398
|
+
};
|
|
399
|
+
/** The part of a query that names a marker. Comes entirely from configuration. */
|
|
400
|
+
declare function markerTerm(marker: Marker): string;
|
|
401
|
+
/**
|
|
402
|
+
* Mitochondrial genomes of the same taxa, whatever they annotate. A genome can hold the marker with no feature naming it,
|
|
403
|
+
* so the marker search can't see it. Only for a mitochondrial gene, where a whole genome would contain the marker.
|
|
404
|
+
*/
|
|
405
|
+
declare function genomesHoldingMarkerQuery(selection: Selection): string | null;
|
|
406
|
+
declare function selectionQuery(selection: Selection): string;
|
|
407
|
+
type OutgroupCandidate = {
|
|
408
|
+
accession: string;
|
|
409
|
+
organism: string;
|
|
410
|
+
length: number;
|
|
411
|
+
taxid?: string;
|
|
412
|
+
};
|
|
413
|
+
type OutgroupStep = {
|
|
414
|
+
taxon: TaxonStep;
|
|
415
|
+
/** Records for this marker inside this ancestor, excluding the previous step. */
|
|
416
|
+
records: number;
|
|
417
|
+
species: string[];
|
|
418
|
+
candidates: OutgroupCandidate[];
|
|
419
|
+
sufficient: boolean;
|
|
420
|
+
};
|
|
421
|
+
type OutgroupLadder = {
|
|
422
|
+
ingroup: string;
|
|
423
|
+
steps: OutgroupStep[];
|
|
424
|
+
suggested: OutgroupStep | null;
|
|
425
|
+
caveat: string;
|
|
426
|
+
};
|
|
427
|
+
/**
|
|
428
|
+
* Walks outward from the ingroup through the taxonomy, one ancestor at a time, until enough species have this
|
|
429
|
+
* marker. Taxonomy isn't phylogeny, so a rank step suggests where to look; the finished tree must confirm it.
|
|
430
|
+
*/
|
|
431
|
+
declare function suggestOutgroups(client: NcbiClient, selection: Selection, options?: {
|
|
432
|
+
minSpecies?: number;
|
|
433
|
+
maxSteps?: number;
|
|
434
|
+
minLength?: number;
|
|
435
|
+
perSpecies?: number;
|
|
436
|
+
}): Promise<OutgroupLadder>;
|
|
437
|
+
|
|
438
|
+
type ReferenceCandidate = {
|
|
439
|
+
accession: string;
|
|
440
|
+
organism: string;
|
|
441
|
+
length: number;
|
|
442
|
+
title: string;
|
|
443
|
+
};
|
|
444
|
+
type ReferenceSuggestion = {
|
|
445
|
+
/** The nearest level with a RefSeq mitochondrial genome: the taxon itself, or an ancestor. */
|
|
446
|
+
from: {
|
|
447
|
+
id: string;
|
|
448
|
+
name: string;
|
|
449
|
+
rank: string;
|
|
450
|
+
sameTaxon: boolean;
|
|
451
|
+
} | null;
|
|
452
|
+
candidates: ReferenceCandidate[];
|
|
453
|
+
/** From NCBI Taxonomy. Only codes the frame check supports are offered; others say why not. */
|
|
454
|
+
geneticCode: {
|
|
455
|
+
code: number;
|
|
456
|
+
supported: true;
|
|
457
|
+
label: string;
|
|
458
|
+
} | {
|
|
459
|
+
code?: number;
|
|
460
|
+
supported: false;
|
|
461
|
+
reason: string;
|
|
462
|
+
};
|
|
463
|
+
notes: string[];
|
|
464
|
+
};
|
|
465
|
+
/**
|
|
466
|
+
* Guided setup (roadmap stage 3): a reference a newcomer can start from. The nearest RefSeq complete mitochondrial genome,
|
|
467
|
+
* from the taxon itself or up to `maxSteps` ancestors, and the taxon's mitochondrial genetic code. Suggestions only: a
|
|
468
|
+
* person chooses, and can change both.
|
|
469
|
+
*/
|
|
470
|
+
declare function suggestReference(client: NcbiClient, selection: Selection, options?: {
|
|
471
|
+
maxSteps?: number;
|
|
472
|
+
}): Promise<ReferenceSuggestion>;
|
|
473
|
+
|
|
474
|
+
export { AcquireError, type ApprovedLink, BIOSAMPLE_ATTRIBUTES, type BioSample, type BioSampleListing, type BoldCoverage, type Count, type Coverage, type FetchLike, type FoundPaper, type GenomeListing, type GenomeSummary, LISTING_KINDS, LOOKUP_USER_AGENT, type LinkProposal, type ListingFailure, type ListingKind, NcbiClient, type NcbiOptions, type OccurrenceCoverage, type OutgroupCandidate, type OutgroupLadder, type OutgroupStep, type PaperSearch, type Provenance, type PublicListings, type ReferenceCandidate, type ReferenceSuggestion, type SourceOptions, type SraListing, type SraRun, type TaxonStep, approveLink, biosampleFor, boldCoverage, classifyAccessions, datasetsGenomes, gbifCoverage, genomesHoldingMarkerQuery, idigbioCoverage, linkedAccessions, listSraRuns, looksLikeCoordinates, lookupCrossref, lookupEuropePmc, lookupPaper, markerTerm, mergePapers, ncbiTaxonTerm, occurrenceCoverage, paced, parseBioSample, parseGenomeReport, parseSraSummary, publicListings, searchEuropePmc, searchPapers, searchPubMed, selectionQuery, studiesFromRecords, suggestOutgroups, suggestReference, titleMatch };
|