@captiveorigins/onephylogeny-acquire 0.0.0-stage → 0.1.0-alpha.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,75 @@
1
+ Required Notice: Copyright 2026 Captive Origins (https://captiveorigins.com)
2
+
3
+ # PolyForm Noncommercial License 1.0.0
4
+
5
+ <https://polyformproject.org/licenses/noncommercial/1.0.0>
6
+
7
+ ## Acceptance
8
+
9
+ In order to get any license under these terms, you must agree to them as both strict obligations and conditions to all your licenses.
10
+
11
+ ## Copyright License
12
+
13
+ The licensor grants you a copyright license for the software to do everything you might do with the software that would otherwise infringe the licensor's copyright in it for any permitted purpose. However, you may only distribute the software according to [Distribution License](#distribution-license) and make changes or new works based on the software according to [Changes and New Works License](#changes-and-new-works-license).
14
+
15
+ ## Distribution License
16
+
17
+ The licensor grants you an additional copyright license to distribute copies of the software. Your license to distribute covers distributing the software with changes and new works permitted by [Changes and New Works License](#changes-and-new-works-license).
18
+
19
+ ## Notices
20
+
21
+ You must ensure that anyone who gets a copy of any part of the software from you also gets a copy of these terms or the URL for them above, as well as copies of any plain-text lines beginning with `Required Notice:` that the licensor provided with the software. For example:
22
+
23
+ > Required Notice: Copyright Yoyodyne, Inc. (http://example.com)
24
+
25
+ ## Changes and New Works License
26
+
27
+ The licensor grants you an additional copyright license to make changes and new works based on the software for any permitted purpose.
28
+
29
+ ## Patent License
30
+
31
+ The licensor grants you a patent license for the software that covers patent claims the licensor can license, or becomes able to license, that you would infringe by using the software.
32
+
33
+ ## Noncommercial Purposes
34
+
35
+ Any noncommercial purpose is a permitted purpose.
36
+
37
+ ## Personal Uses
38
+
39
+ Personal use for research, experiment, and testing for the benefit of public knowledge, personal study, private entertainment, hobby projects, amateur pursuits, or religious observance, without any anticipated commercial application, is use for a permitted purpose.
40
+
41
+ ## Noncommercial Organizations
42
+
43
+ Use by any charitable organization, educational institution, public research organization, public safety or health organization, environmental protection organization, or government institution is use for a permitted purpose regardless of the source of funding or obligations resulting from the funding.
44
+
45
+ ## Fair Use
46
+
47
+ You may have "fair use" rights for the software under the law. These terms do not limit them.
48
+
49
+ ## No Other Rights
50
+
51
+ These terms do not allow you to sublicense or transfer any of your licenses to anyone else, or prevent the licensor from granting licenses to anyone else. These terms do not imply any other licenses.
52
+
53
+ ## Patent Defense
54
+
55
+ If you make any written claim that the software infringes or contributes to infringement of any patent, your patent license for the software granted under these terms ends immediately. If your company makes such a claim, your patent license ends immediately for work on behalf of your company.
56
+
57
+ ## Violations
58
+
59
+ The first time you are notified in writing that you have violated any of these terms, or done anything with the software not covered by your licenses, your licenses can nonetheless continue if you come into full compliance with these terms, and take practical steps to correct past violations, within 32 days of receiving notice. Otherwise, all your licenses end immediately.
60
+
61
+ ## No Liability
62
+
63
+ ***As far as the law allows, the software comes as is, without any warranty or condition, and the licensor will not be liable to you for any damages arising out of these terms or the use or nature of the software, under any kind of legal claim.***
64
+
65
+ ## Definitions
66
+
67
+ The **licensor** is the individual or entity offering these terms, and the **software** is the software the licensor makes available under these terms.
68
+
69
+ **You** refers to the individual or entity agreeing to these terms.
70
+
71
+ **Your company** is any legal entity, sole proprietorship, or other kind of organization that you work for, plus all organizations that have control over, are under the control of, or are under common control with that organization. **Control** means ownership of substantially all the assets of an entity, or the power to direct its management and policies by vote, contract, or otherwise. Control can be direct or indirect.
72
+
73
+ **Your licenses** are all the licenses granted to you for the software under these terms.
74
+
75
+ **Use** means anything you do with the software requiring one of your licenses.
package/README.md CHANGED
@@ -1,3 +1,5 @@
1
- # Temporary Holding Version
1
+ # @captiveorigins/onephylogeny-acquire
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ OnePhylogeny acquire: fetches sequence records by taxon and marker with safe retries and no duplicates, keeps raw files with hashes, and suggests outgroups outward through the taxonomy.
4
+
5
+ See the [repository](https://github.com/CaptiveOrigins/onephylogeny) for documentation.
@@ -0,0 +1,137 @@
1
+ import { StudyPaper, Selection, GenBankRecord, LinkedRecord, Marker } from '@captiveorigins/onephylogeny-core';
2
+
3
+ /** A paper as a search found it. Nothing joins a Study until someone adds it. */
4
+ type FoundPaper = StudyPaper & {
5
+ /** Another listed entry whose title matches. A person decides, so records never attach to the wrong paper. */
6
+ possibleSameAs?: string[];
7
+ };
8
+ type PaperSearch = {
9
+ papers: FoundPaper[];
10
+ counts: Record<string, number>;
11
+ /** How many each source has in all. Where it's more than the count, the list was cut short. */
12
+ totals: Record<string, number>;
13
+ sources: string[];
14
+ failed: string[];
15
+ };
16
+ type Listed = {
17
+ papers: FoundPaper[];
18
+ total: number;
19
+ };
20
+ declare function searchEuropePmc(fetch: FetchLike, query: string, pageSize?: number): Promise<Listed>;
21
+ declare function searchPubMed(client: NcbiClient, query: string, retmax?: number): Promise<Listed>;
22
+ /** Records under one publication title, for the many that have no PubMed link. One submission can share a paper's title. */
23
+ declare function studiesFromRecords(records: GenBankRecord[], describe: (r: GenBankRecord) => LinkedRecord): FoundPaper[];
24
+ /**
25
+ * Joins lists on exact PMID or DOI. Matching titles are only marked: a person decides those, so records never attach to the wrong paper.
26
+ */
27
+ declare function mergePapers(lists: FoundPaper[][]): FoundPaper[];
28
+ /** Searches every source and reports any that failed, because a partial list mustn't look complete. */
29
+ declare function searchPapers(client: NcbiClient, fetch: FetchLike, query: string, records?: FoundPaper[]): Promise<PaperSearch>;
30
+ /** Nucleotide records PubMed links to a paper. Most records have no such link, so an empty list doesn't mean the paper has none. */
31
+ declare function linkedAccessions(client: NcbiClient, pmid: string): Promise<string[]>;
32
+ /** Which accessions are of the target taxa, and which hold the marker. Whole mitochondrial genomes hold it even when unannotated. */
33
+ declare function classifyAccessions(client: NcbiClient, accessions: string[], selection: Selection): Promise<{
34
+ inTarget: Set<string>;
35
+ hasMarker: Set<string>;
36
+ }>;
37
+
38
+ type FetchLike = (url: string, init?: {
39
+ method?: string;
40
+ headers?: Record<string, string>;
41
+ body?: string;
42
+ }) => Promise<{
43
+ ok: boolean;
44
+ status: number;
45
+ text(): Promise<string>;
46
+ }>;
47
+ type NcbiOptions = {
48
+ fetch: FetchLike;
49
+ /** Sent to NCBI so it can identify the client. */
50
+ tool?: string;
51
+ email?: string;
52
+ apiKey?: string;
53
+ /** Pause between requests. NCBI asks for at most 3 a second without a key. */
54
+ delayMs?: number;
55
+ retries?: number;
56
+ sleep?: (ms: number) => Promise<void>;
57
+ log?: (level: 'info' | 'warning', message: string, data?: Record<string, unknown>) => void;
58
+ };
59
+ declare class AcquireError extends Error {
60
+ }
61
+ /** A small NCBI E-utilities client. Failed requests retry, then throw: nothing is skipped silently. */
62
+ declare class NcbiClient {
63
+ private readonly o;
64
+ constructor(options: NcbiOptions);
65
+ request(path: string, params: Record<string, string>): Promise<string>;
66
+ private json;
67
+ /** Accession versions for a search, every page of them. A short read is an error. */
68
+ searchAccessions(term: string, page?: number): Promise<string[]>;
69
+ count(term: string): Promise<number>;
70
+ /** Raw GenBank text for these accessions, checked to contain exactly what was asked for. */
71
+ fetchGenBankText(ids: string[]): Promise<{
72
+ text: string;
73
+ records: GenBankRecord[];
74
+ }>;
75
+ summaries(term: string, retmax?: number): Promise<{
76
+ accession: string;
77
+ organism: string;
78
+ length: number;
79
+ title: string;
80
+ taxid?: string;
81
+ }[]>;
82
+ /** Finds a taxon by scientific name. More than one result is returned for a person to choose from. */
83
+ findTaxon(name: string): Promise<{
84
+ id: string;
85
+ scientificName: string;
86
+ }[]>;
87
+ /** A taxon's ancestors, nearest first. */
88
+ lineage(taxid: string): Promise<{
89
+ scientificName: string;
90
+ ancestors: TaxonStep[];
91
+ }>;
92
+ }
93
+ type TaxonStep = {
94
+ id: string;
95
+ name: string;
96
+ rank: string;
97
+ };
98
+ /** The part of a query that names a marker. Comes entirely from configuration. */
99
+ declare function markerTerm(marker: Marker): string;
100
+ /**
101
+ * Mitochondrial genomes of the same taxa, whatever they annotate. A genome can hold the marker with no feature naming it,
102
+ * so the marker search can't see it. Only for a mitochondrial gene, where a whole genome would contain the marker.
103
+ */
104
+ declare function genomesHoldingMarkerQuery(selection: Selection): string | null;
105
+ declare function selectionQuery(selection: Selection): string;
106
+ type OutgroupCandidate = {
107
+ accession: string;
108
+ organism: string;
109
+ length: number;
110
+ taxid?: string;
111
+ };
112
+ type OutgroupStep = {
113
+ taxon: TaxonStep;
114
+ /** Records for this marker inside this ancestor, excluding the previous step. */
115
+ records: number;
116
+ species: string[];
117
+ candidates: OutgroupCandidate[];
118
+ sufficient: boolean;
119
+ };
120
+ type OutgroupLadder = {
121
+ ingroup: string;
122
+ steps: OutgroupStep[];
123
+ suggested: OutgroupStep | null;
124
+ caveat: string;
125
+ };
126
+ /**
127
+ * Walks outward from the ingroup through the taxonomy, one ancestor at a time, until enough species have this
128
+ * marker. Taxonomy isn't phylogeny, so a rank step suggests where to look; the finished tree must confirm it.
129
+ */
130
+ declare function suggestOutgroups(client: NcbiClient, selection: Selection, options?: {
131
+ minSpecies?: number;
132
+ maxSteps?: number;
133
+ minLength?: number;
134
+ perSpecies?: number;
135
+ }): Promise<OutgroupLadder>;
136
+
137
+ export { AcquireError, type FetchLike, type FoundPaper, NcbiClient, type NcbiOptions, type OutgroupCandidate, type OutgroupLadder, type OutgroupStep, type PaperSearch, type TaxonStep, classifyAccessions, genomesHoldingMarkerQuery, linkedAccessions, markerTerm, mergePapers, searchEuropePmc, searchPapers, searchPubMed, selectionQuery, studiesFromRecords, suggestOutgroups };
package/dist/index.js ADDED
@@ -0,0 +1,315 @@
1
+ // src/index.ts
2
+ import { parseGenBank } from "@captiveorigins/onephylogeny-core";
3
+
4
+ // src/papers.ts
5
+ import "@captiveorigins/onephylogeny-core";
6
+ var EUROPE_PMC = "https://www.ebi.ac.uk/europepmc/webservices/rest/search";
7
+ var NOTICE = /retract|correct|erratum|expression of concern/i;
8
+ var normalTitle = (t) => t.toLowerCase().replace(/<[^>]+>/g, "").replace(/[^a-z0-9]+/g, " ").trim();
9
+ var cleanTitle = (t) => t.replace(/<[^>]+>/g, "").replace(/\.$/, "").trim();
10
+ var firstAuthor = (authors) => authors?.split(/[,;]| and /)[0]?.trim().split(/\s+/)[0] ?? "";
11
+ var shortLabel = (authors, year, title) => [firstAuthor(authors) || title.split(/\s+/).slice(0, 3).join(" "), year].filter(Boolean).join(" ");
12
+ var blankPaper = (id, title) => ({ id, label: "", colorIndex: 0, title, sources: [], linked: [], linkLookup: "not-done" });
13
+ var withLabel = (p) => ({ ...p, label: shortLabel(p.authors, p.year, p.title) });
14
+ async function searchEuropePmc(fetch, query, pageSize = 100) {
15
+ const url = `${EUROPE_PMC}?${new URLSearchParams({ query, format: "json", resultType: "core", pageSize: String(pageSize) })}`;
16
+ const res = await fetch(url);
17
+ if (!res.ok) throw new AcquireError(`Europe PMC answered HTTP ${res.status}`);
18
+ const body = JSON.parse(await res.text());
19
+ const results = body.resultList?.result ?? [];
20
+ const papers = results.filter((r) => r.title).map((r) => {
21
+ const types = r.pubTypeList?.pubType ?? [];
22
+ const notices = types.filter((t) => NOTICE.test(t));
23
+ return withLabel({
24
+ ...blankPaper(r.pmid ? `pmid:${r.pmid}` : r.doi ? `doi:${String(r.doi).toLowerCase()}` : `epmc:${r.id}`, cleanTitle(r.title)),
25
+ sources: ["europepmc"],
26
+ ...r.authorString ? { authors: r.authorString } : {},
27
+ ...r.journalInfo?.journal?.title ? { journal: r.journalInfo.journal.title } : {},
28
+ ...r.pubYear ? { year: Number(r.pubYear) } : {},
29
+ ...r.doi ? { doi: String(r.doi).toLowerCase() } : {},
30
+ ...r.pmid ? { pmid: String(r.pmid) } : {},
31
+ openAccess: r.isOpenAccess === "Y",
32
+ ...notices.length ? { notices } : {}
33
+ });
34
+ });
35
+ return { papers, total: Number(body.hitCount ?? papers.length) };
36
+ }
37
+ async function searchPubMed(client, query, retmax = 100) {
38
+ const search = JSON.parse(await client.request("esearch.fcgi", { db: "pubmed", term: query, retmax: String(retmax), retmode: "json" }));
39
+ const ids = search.esearchresult?.idlist ?? [];
40
+ const total = Number(search.esearchresult?.count ?? ids.length);
41
+ if (!ids.length) return { papers: [], total };
42
+ const summary = JSON.parse(await client.request("esummary.fcgi", { db: "pubmed", id: ids.join(","), retmode: "json" })).result ?? {};
43
+ const papers = ids.filter((id) => summary[id]?.title).map((id) => {
44
+ const r = summary[id];
45
+ const authors = (r.authors ?? []).map((a) => a.name).join(", ");
46
+ const doi = (r.articleids ?? []).find((a) => a.idtype === "doi")?.value;
47
+ const year = Number(String(r.pubdate ?? "").slice(0, 4));
48
+ const notices = (r.pubtype ?? []).filter((t) => NOTICE.test(t));
49
+ return withLabel({
50
+ ...blankPaper(`pmid:${id}`, cleanTitle(r.title)),
51
+ sources: ["pubmed"],
52
+ pmid: id,
53
+ ...authors ? { authors } : {},
54
+ ...r.fulljournalname ? { journal: r.fulljournalname } : {},
55
+ ...year ? { year } : {},
56
+ ...doi ? { doi: String(doi).toLowerCase() } : {},
57
+ ...notices.length ? { notices } : {}
58
+ });
59
+ });
60
+ return { papers, total };
61
+ }
62
+ function studiesFromRecords(records, describe) {
63
+ const byKey = /* @__PURE__ */ new Map();
64
+ for (const r of records) {
65
+ const pub = r.references.find((p) => p.title && p.title !== "Direct Submission");
66
+ const key = pub?.pubmedId ? `pmid:${pub.pubmedId}` : pub ? `genbank:${normalTitle(pub.title)}` : "genbank:no-publication";
67
+ let entry = byKey.get(key);
68
+ if (!entry) {
69
+ const year = Number((pub?.journal ?? "").match(/\b(?:19|20)\d{2}\b/g)?.at(-1));
70
+ const title = pub ? cleanTitle(pub.title) : "Records with no publication listed in GenBank";
71
+ entry = { linked: [], paper: withLabel({
72
+ ...blankPaper(key, title),
73
+ sources: ["genbank"],
74
+ ...pub?.authors ? { authors: pub.authors } : {},
75
+ ...pub?.journal ? { journal: pub.journal } : {},
76
+ ...year ? { year } : {},
77
+ ...pub?.pubmedId ? { pmid: pub.pubmedId } : {},
78
+ ...pub ? {} : { unpublished: true }
79
+ }) };
80
+ if (!pub) entry.paper.label = "No publication";
81
+ byKey.set(key, entry);
82
+ }
83
+ entry.linked.push(describe(r));
84
+ }
85
+ return [...byKey.values()].map(({ paper, linked }) => ({ ...paper, linked, linkLookup: "done" }));
86
+ }
87
+ function mergePapers(lists) {
88
+ const out = [];
89
+ for (const paper of lists.flat()) {
90
+ const same = out.find((p) => paper.pmid && p.pmid === paper.pmid || paper.doi && p.doi === paper.doi);
91
+ if (!same) {
92
+ out.push(paper);
93
+ continue;
94
+ }
95
+ same.sources = [.../* @__PURE__ */ new Set([...same.sources, ...paper.sources])];
96
+ same.linked = same.linked.length ? same.linked : paper.linked;
97
+ if (same.linkLookup !== "done") same.linkLookup = paper.linkLookup;
98
+ same.authors ??= paper.authors;
99
+ same.doi ??= paper.doi;
100
+ same.pmid ??= paper.pmid;
101
+ same.journal ??= paper.journal;
102
+ same.year ??= paper.year;
103
+ if (paper.notices) same.notices = [.../* @__PURE__ */ new Set([...same.notices ?? [], ...paper.notices])];
104
+ if (paper.openAccess) same.openAccess = true;
105
+ }
106
+ for (const p of out) {
107
+ const twins = out.filter((q) => q !== p && !p.unpublished && !q.unpublished && normalTitle(q.title) === normalTitle(p.title)).map((q) => q.id);
108
+ if (twins.length) p.possibleSameAs = twins;
109
+ }
110
+ return out;
111
+ }
112
+ async function searchPapers(client, fetch, query, records = []) {
113
+ const failed = [];
114
+ const attempt = async (name, work) => {
115
+ try {
116
+ return await work();
117
+ } catch (e) {
118
+ failed.push(`${name}: ${e instanceof Error ? e.message : String(e)}`);
119
+ return { papers: [], total: 0 };
120
+ }
121
+ };
122
+ const empty = !query.trim();
123
+ const pubmed = empty ? { papers: [], total: 0 } : await attempt("PubMed", () => searchPubMed(client, query));
124
+ const europe = empty ? { papers: [], total: 0 } : await attempt("Europe PMC", () => searchEuropePmc(fetch, query));
125
+ return {
126
+ papers: mergePapers([records, pubmed.papers, europe.papers]),
127
+ counts: { PubMed: pubmed.papers.length, "Europe PMC": europe.papers.length, "GenBank studies": records.length },
128
+ totals: { PubMed: pubmed.total, "Europe PMC": europe.total, "GenBank studies": records.length },
129
+ sources: [...empty ? [] : ["PubMed", "Europe PMC"], ...records.length ? ["GenBank"] : []],
130
+ failed
131
+ };
132
+ }
133
+ async function linkedAccessions(client, pmid) {
134
+ const r = JSON.parse(await client.request("elink.fcgi", { dbfrom: "pubmed", db: "nuccore", linkname: "pubmed_nuccore", id: pmid, retmode: "json" }));
135
+ const ids = r.linksets?.[0]?.linksetdbs?.[0]?.links ?? [];
136
+ if (!ids.length) return [];
137
+ const out = [];
138
+ for (let i = 0; i < ids.length; i += 200) {
139
+ const s = JSON.parse(await client.request("esummary.fcgi", { db: "nuccore", id: ids.slice(i, i + 200).join(","), retmode: "json" })).result ?? {};
140
+ for (const id of s.uids ?? []) if (s[id]?.accessionversion) out.push(s[id].accessionversion);
141
+ }
142
+ return out;
143
+ }
144
+ var taxaTerm = (selection) => selection.taxa.filter((t) => t.source === "ncbi-taxonomy").map((t) => `txid${t.id}[Organism:exp]`).join(" OR ");
145
+ async function classifyAccessions(client, accessions, selection) {
146
+ const inTarget = /* @__PURE__ */ new Set();
147
+ const hasMarker = /* @__PURE__ */ new Set();
148
+ const genomeTerm = genomesHoldingMarkerQuery(selection) ? " OR (mitochondrion[Filter] AND 10000:25000[SLEN])" : "";
149
+ for (let i = 0; i < accessions.length; i += 150) {
150
+ const chunk = accessions.slice(i, i + 150);
151
+ const list = `(${chunk.map((a) => `${a}[Accession]`).join(" OR ")})`;
152
+ for (const a of await client.searchAccessions(`${list} AND (${taxaTerm(selection)})`)) inTarget.add(a);
153
+ for (const a of await client.searchAccessions(`${list} AND ((${markerTerm(selection.marker)})${genomeTerm})`)) hasMarker.add(a);
154
+ }
155
+ return { inTarget, hasMarker };
156
+ }
157
+
158
+ // src/index.ts
159
+ var AcquireError = class extends Error {
160
+ };
161
+ var BASE = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/";
162
+ var NcbiClient = class {
163
+ o;
164
+ constructor(options) {
165
+ this.o = { delayMs: options.apiKey ? 120 : 380, retries: 4, sleep: (ms) => new Promise((r) => setTimeout(r, ms)), ...options };
166
+ }
167
+ async request(path, params) {
168
+ const body = new URLSearchParams({ ...params, tool: this.o.tool ?? "onephylogeny", ...this.o.email ? { email: this.o.email } : {}, ...this.o.apiKey ? { api_key: this.o.apiKey } : {} }).toString();
169
+ let lastError = "";
170
+ for (let attempt = 0; attempt <= this.o.retries; attempt++) {
171
+ if (attempt) await this.o.sleep(Math.min(3e4, 1e3 * 2 ** attempt));
172
+ await this.o.sleep(this.o.delayMs);
173
+ try {
174
+ const res = await this.o.fetch(BASE + path, { method: "POST", headers: { "content-type": "application/x-www-form-urlencoded" }, body });
175
+ if (res.ok) return await res.text();
176
+ lastError = `HTTP ${res.status}`;
177
+ if (res.status < 500 && res.status !== 429) break;
178
+ } catch (e) {
179
+ lastError = e instanceof Error ? e.message : String(e);
180
+ }
181
+ this.o.log?.("warning", `${path} failed (${lastError}), attempt ${attempt + 1} of ${this.o.retries + 1}`);
182
+ }
183
+ throw new AcquireError(`${path} failed: ${lastError}`);
184
+ }
185
+ async json(path, params) {
186
+ const text = await this.request(path, { ...params, retmode: "json" });
187
+ try {
188
+ return JSON.parse(text);
189
+ } catch {
190
+ throw new AcquireError(`${path} did not return JSON: ${text.slice(0, 120)}`);
191
+ }
192
+ }
193
+ /** Accession versions for a search, every page of them. A short read is an error. */
194
+ async searchAccessions(term, page = 5e3) {
195
+ const out = [];
196
+ let total = 0;
197
+ do {
198
+ const r = await this.json("esearch.fcgi", { db: "nuccore", term, idtype: "acc", retmax: String(page), retstart: String(out.length) });
199
+ const res = r.esearchresult;
200
+ if (!res || res.ERROR) throw new AcquireError(`Search failed: ${String(res?.ERROR ?? "no result")}`);
201
+ total = Number(res.count);
202
+ const ids = res.idlist ?? [];
203
+ if (!ids.length && out.length < total) throw new AcquireError(`Search stopped at ${out.length} of ${total} records`);
204
+ out.push(...ids);
205
+ } while (out.length < total);
206
+ return [...new Set(out)];
207
+ }
208
+ async count(term) {
209
+ const r = await this.json("esearch.fcgi", { db: "nuccore", term, retmax: "0" });
210
+ return Number(r.esearchresult?.count ?? 0);
211
+ }
212
+ /** Raw GenBank text for these accessions, checked to contain exactly what was asked for. */
213
+ async fetchGenBankText(ids) {
214
+ const text = await this.request("efetch.fcgi", { db: "nuccore", id: ids.join(","), rettype: "gbwithparts", retmode: "text" });
215
+ const records = parseGenBank(text);
216
+ const got = new Set(records.flatMap((r) => [r.version, r.accession]));
217
+ const missing = ids.filter((id) => !got.has(id) && !got.has(id.split(".")[0]));
218
+ if (missing.length) throw new AcquireError(`NCBI did not return ${missing.length} of ${ids.length} records, including ${missing.slice(0, 3).join(", ")}`);
219
+ return { text, records };
220
+ }
221
+ async summaries(term, retmax = 300) {
222
+ const search = await this.json("esearch.fcgi", { db: "nuccore", term, retmax: String(retmax) });
223
+ const ids = search.esearchresult?.idlist ?? [];
224
+ if (!ids.length) return [];
225
+ const r = await this.json("esummary.fcgi", { db: "nuccore", id: ids.join(",") });
226
+ return ids.map((id) => r.result?.[id]).filter(Boolean).map((s) => ({ accession: s.accessionversion, organism: s.organism, length: Number(s.slen), title: s.title, taxid: s.taxid ? String(s.taxid) : void 0 }));
227
+ }
228
+ /** Finds a taxon by scientific name. More than one result is returned for a person to choose from. */
229
+ async findTaxon(name) {
230
+ const r = await this.json("esearch.fcgi", { db: "taxonomy", term: `${quote(name)}[Scientific Name]`, retmax: "5" });
231
+ const ids = r.esearchresult?.idlist ?? [];
232
+ const out = [];
233
+ for (const id of ids) out.push({ id, scientificName: (await this.lineage(id)).scientificName });
234
+ return out;
235
+ }
236
+ /** A taxon's ancestors, nearest first. */
237
+ async lineage(taxid) {
238
+ const xml = await this.request("efetch.fcgi", { db: "taxonomy", id: taxid, retmode: "xml" });
239
+ const name = /<ScientificName>([^<]+)<\/ScientificName>/.exec(xml)?.[1];
240
+ if (!name) throw new AcquireError(`Taxon ${taxid} was not found`);
241
+ const lineage = /<LineageEx>([\s\S]*?)<\/LineageEx>/.exec(xml)?.[1] ?? "";
242
+ const ancestors = [...lineage.matchAll(/<Taxon>\s*<TaxId>(\d+)<\/TaxId>\s*<ScientificName>([^<]+)<\/ScientificName>\s*<Rank>([^<]*)<\/Rank>/g)].map((m) => ({ id: m[1], name: m[2], rank: m[3] })).reverse();
243
+ return { scientificName: name, ancestors };
244
+ }
245
+ };
246
+ var quote = (s) => `"${s.replace(/"/g, "")}"`;
247
+ function markerTerm(marker) {
248
+ if (marker.kind === "mitogenome") return '(mitochondrion[Filter] AND ("complete genome"[Title] OR "partial genome"[Title]))';
249
+ if (marker.kind === "snp") throw new AcquireError("SNP data has no NCBI nucleotide search. Import a prepared dataset instead");
250
+ const names = [marker.gene, ...marker.synonyms ?? []].filter(Boolean);
251
+ const fields = [...names.map((n) => `${quote(n)}[Gene]`), ...(marker.products ?? []).map((p) => `${quote(p)}[Protein Name]`)];
252
+ if (!fields.length) throw new AcquireError("A gene marker must name its gene");
253
+ const organelle = marker.organelle === "mitochondrion" ? "gene_in_mitochondrion[PROP] AND " : marker.organelle === "plastid" ? "gene_in_plastid[PROP] AND " : "";
254
+ return `${organelle}(${fields.join(" OR ")})`;
255
+ }
256
+ var ncbiTaxa = (taxa) => {
257
+ const ids = taxa.filter((t) => t.source === "ncbi-taxonomy").map((t) => t.id);
258
+ if (!ids.length) throw new AcquireError("NCBI searches need taxa from the ncbi-taxonomy source");
259
+ return ids;
260
+ };
261
+ function genomesHoldingMarkerQuery(selection) {
262
+ const { marker } = selection;
263
+ if (marker.kind !== "gene" || marker.organelle !== "mitochondrion") return null;
264
+ const taxa = ncbiTaxa(selection.taxa).map((id) => `txid${id}[Organism:exp]`).join(" OR ");
265
+ return `(${taxa}) AND mitochondrion[Filter] AND 10000:25000[SLEN] NOT refseq[Filter]`;
266
+ }
267
+ function selectionQuery(selection) {
268
+ const taxa = ncbiTaxa(selection.taxa).map((id) => `txid${id}[Organism:exp]`).join(" OR ");
269
+ return `(${taxa}) AND ${markerTerm(selection.marker)} NOT refseq[Filter]`;
270
+ }
271
+ async function suggestOutgroups(client, selection, options = {}) {
272
+ const { minSpecies = 2, maxSteps = 8, minLength = 0, perSpecies = 1 } = options;
273
+ const taxid = ncbiTaxa(selection.taxa)[0];
274
+ const { scientificName, ancestors } = await client.lineage(taxid);
275
+ const marker = markerTerm(selection.marker);
276
+ const steps = [];
277
+ let suggested = null;
278
+ let inner = taxid;
279
+ for (const taxon of ancestors.slice(0, maxSteps)) {
280
+ const term = `txid${taxon.id}[Organism:exp] NOT txid${inner}[Organism:exp] AND ${marker} NOT refseq[Filter]`;
281
+ const records = await client.count(term);
282
+ const summaries = records ? await client.summaries(term) : [];
283
+ const bySpecies = /* @__PURE__ */ new Map();
284
+ for (const s of summaries) {
285
+ if (s.length < minLength) continue;
286
+ const list = bySpecies.get(s.organism);
287
+ if (list) list.push(s);
288
+ else bySpecies.set(s.organism, [s]);
289
+ }
290
+ const candidates = [...bySpecies.values()].flatMap((list) => list.sort((a, b) => b.length - a.length).slice(0, perSpecies));
291
+ const step = { taxon, records, species: [...bySpecies.keys()].sort(), candidates: candidates.sort((a, b) => b.length - a.length), sufficient: bySpecies.size >= minSpecies };
292
+ steps.push(step);
293
+ inner = taxon.id;
294
+ if (step.sufficient) {
295
+ suggested = step;
296
+ break;
297
+ }
298
+ }
299
+ return { ingroup: scientificName, steps, suggested, caveat: "Taxonomic rank is not evidence of ancestry. Check that the ingroup is monophyletic and these outgroups fall outside it in the finished tree." };
300
+ }
301
+ export {
302
+ AcquireError,
303
+ NcbiClient,
304
+ classifyAccessions,
305
+ genomesHoldingMarkerQuery,
306
+ linkedAccessions,
307
+ markerTerm,
308
+ mergePapers,
309
+ searchEuropePmc,
310
+ searchPapers,
311
+ searchPubMed,
312
+ selectionQuery,
313
+ studiesFromRecords,
314
+ suggestOutgroups
315
+ };
package/package.json CHANGED
@@ -1,6 +1,46 @@
1
1
  {
2
2
  "name": "@captiveorigins/onephylogeny-acquire",
3
- "version": "0.0.0-stage",
4
- "stub": true,
5
- "description": "Temporary package placeholder for staged publishing"
3
+ "version": "0.1.0-alpha.1",
4
+ "description": "OnePhylogeny acquire: fetches sequence records by taxon and marker with safe retries and no duplicates, keeps raw files with hashes, and suggests outgroups outward through the taxonomy.",
5
+ "keywords": [
6
+ "phylogenetics",
7
+ "reference-panel",
8
+ "genbank"
9
+ ],
10
+ "homepage": "https://github.com/CaptiveOrigins/onephylogeny#readme",
11
+ "bugs": {
12
+ "url": "https://github.com/CaptiveOrigins/onephylogeny/issues"
13
+ },
14
+ "license": "PolyForm-Noncommercial-1.0.0",
15
+ "repository": {
16
+ "type": "git",
17
+ "url": "git+https://github.com/CaptiveOrigins/onephylogeny.git",
18
+ "directory": "packages/acquire"
19
+ },
20
+ "type": "module",
21
+ "sideEffects": false,
22
+ "exports": {
23
+ ".": {
24
+ "types": "./dist/index.d.ts",
25
+ "import": "./dist/index.js"
26
+ }
27
+ },
28
+ "main": "./dist/index.js",
29
+ "types": "./dist/index.d.ts",
30
+ "files": [
31
+ "dist",
32
+ "README.md",
33
+ "LICENSE"
34
+ ],
35
+ "engines": {
36
+ "node": ">=22"
37
+ },
38
+ "dependencies": {
39
+ "@captiveorigins/onephylogeny-core": "^0.1.0-alpha.1"
40
+ },
41
+ "scripts": {
42
+ "build": "tsup src/index.ts --format esm --dts --clean --target es2022",
43
+ "typecheck": "tsc --noEmit",
44
+ "test": "vitest run --passWithNoTests"
45
+ }
6
46
  }