@yangfei_93sky/biocli 0.2.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 youngfly93
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,197 @@
+# biocli
+
+Query biological databases from the terminal. Agent-first design.
+
+```
+biocli v0.2.0
+NCBI · UniProt · KEGG · STRING · Ensembl · Enrichr
+42 commands · 6 database backends · 8 workflow commands · 4 download commands
+```
+
+## Install
+
+```bash
+npm install -g @biocli/cli
+```
+
+Requires Node.js >= 20. No API keys needed (optional NCBI key increases rate limit).
+
+## Why biocli
+
+biocli is the only CLI that takes you from a **research question** to an **analysis-ready working directory** — scout datasets, download data, fetch annotations, all in one pipeline.
+
+```bash
+# Scout relevant datasets for your research question
+biocli aggregate workflow-scout "TP53 breast cancer RNA-seq" --gene TP53
+
+# Prepare a working directory with data + annotations + manifest
+biocli aggregate workflow-prepare GSE315149 --gene TP53 --outdir ./project
+```
+
+Designed for **AI agents** (Claude Code, Codex CLI, etc.) — structured JSON output, per-command schema, self-describing help, batch input, local cache.
+
+## How biocli compares
+
+|  | biocli | gget | BioMCP | EDirect |
+|--|--------|------|--------|---------|
+| Query biological databases | ✅ | ✅ | ✅ | ✅ |
+| Structured JSON output | ✅ | ✅ | ✅ | ❌ |
+| Cross-database aggregation | ✅ | ❌ | ✅ | ❌ |
+| Download GEO/SRA data files | ✅ | ❌ | ❌ | ❌ |
+| Dataset discovery (scout) | ✅ | ❌ | ❌ | ❌ |
+| Working directory prep (prepare) | ✅ | ❌ | ❌ | ❌ |
+| Agent command self-description | ✅ | ❌ | ⚠️ | ❌ |
+| Safe preview (--plan/--dry-run) | ✅ | ❌ | ❌ | ❌ |
+| Per-command JSON Schema | ✅ | ❌ | ❌ | ❌ |
+| Local response cache | ✅ | ❌ | ❌ | ❌ |
+| Batch input (--input) | ✅ | ❌ | ✅ | ✅ |
+
+> **gget** excels at sequence analysis (BLAST, AlphaFold, MUSCLE). **BioMCP** covers more biomedical entities (drugs, trials, diseases). **EDirect** has the deepest NCBI Entrez integration. **biocli** is the only one that combines query + download + data preparation into agent-orchestrated workflows.
+
+### Benchmark: Agent-First Biological Workflow Tasks (2026-04-04)
+
+12 tasks across gene intelligence, variant interpretation, literature search, and data preparation. Task scores are automated from raw output; cross-cutting scores are manual audit with published justifications. [Full methodology →](benchmarks/README.md)
+
+<p align="center">
+  <img src="benchmarks/results/2026-04-04/plots/total_scores.png" width="420" alt="Overall benchmark scores">
+</p>
+
+| Tool | Version | Task Success | Agent Readiness | Workflow Depth | Safety | Reproducibility | **Total** |
+|------|---------|:---:|:---:|:---:|:---:|:---:|:---:|
+| **biocli** | 0.2.0 | 47/49 | 10/10 | 10/10 | 9/10 | 10/10 | **96/100** |
+| BioMCP | 0.8.19 | 20/49 | 6/10 | 4/10 | 3/10 | 2/10 | 44/100 |
+| gget | 0.30.3 | 8/49 | 3/10 | 2/10 | 2/10 | 1/10 | 24/100 |
+
+<details>
+<summary>Detailed breakdown by dimension and category</summary>
+
+<p align="center">
+  <img src="benchmarks/results/2026-04-04/plots/dimensions.png" width="560" alt="Cross-cutting quality dimensions">
+</p>
+
+<p align="center">
+  <img src="benchmarks/results/2026-04-04/plots/task_categories.png" width="500" alt="Task success by category">
+</p>
+
+</details>
+
+> All three tools were installed (`npm install -g @biocli/cli`, `pip install gget==0.30.3`, `uv tool install biomcp-cli==0.8.19`) and executed on the same machine with the same inputs. Raw stdout/stderr, scoring scripts, and runner scripts are in [`benchmarks/`](benchmarks/). BioMCP excels at biomedical entity breadth (drugs, trials, diseases) not covered by this task set; gget excels at sequence analysis (BLAST, AlphaFold) not covered here.
+
+## Quick start
+
+**One command replaces 4 browser tabs:**
+
+```bash
+biocli aggregate gene-dossier TP53 -f json
+```
+
+Returns a unified JSON with gene summary, protein function, KEGG pathways, GO terms, protein interactions, recent literature, and clinical variants — sourced from NCBI, UniProt, KEGG, STRING, PubMed, and ClinVar in parallel.
+
+```bash
+# Gene intelligence (NCBI + UniProt + KEGG + STRING + PubMed + ClinVar)
+biocli aggregate gene-dossier TP53
+
+# Variant interpretation (dbSNP + ClinVar + Ensembl VEP)
+biocli aggregate variant-dossier rs334
+
+# Literature review with abstracts
+biocli aggregate literature-brief "CRISPR cancer immunotherapy" --limit 10
+
+# Pathway enrichment (Enrichr + STRING)
+biocli aggregate enrichment TP53,BRCA1,EGFR,MYC,CDK2
+
+# Gene profile (NCBI + UniProt + KEGG + STRING)
+biocli aggregate gene-profile TP53
+```
+
+## All commands
+
+### Workflow commands (agent-optimized)
+
+| Command | Sources | Use case |
+|---------|---------|----------|
+| `aggregate gene-dossier <gene>` | NCBI+UniProt+KEGG+STRING+PubMed+ClinVar | Complete gene intelligence report |
+| `aggregate variant-dossier <variant>` | dbSNP+ClinVar+Ensembl VEP | Variant interpretation |
+| `aggregate variant-interpret <variant>` | dbSNP+ClinVar+VEP+UniProt | Variant interpretation with clinical context |
+| `aggregate literature-brief <query>` | PubMed | Literature summary with abstracts |
+| `aggregate enrichment <genes>` | Enrichr+STRING | Pathway/GO enrichment analysis |
+| `aggregate gene-profile <gene>` | NCBI+UniProt+KEGG+STRING | Gene profile (no literature) |
+| `aggregate workflow-scout <query>` | GEO+SRA | Scout datasets for a research question |
+| `aggregate workflow-prepare <dataset>` | GEO+NCBI+UniProt+KEGG | Prepare research-ready directory with data + annotations |
+
+### Database commands (atomic)
+
+| Database | Commands |
+|----------|----------|
+| **PubMed** | `pubmed search`, `fetch`, `abstract`, `cited-by`, `related`, `info` |
+| **Gene** | `gene search`, `info`, `fetch` (FASTA download) |
+| **GEO** | `geo search`, `dataset`, `samples`, `download` |
+| **SRA** | `sra search`, `run`, `download` (FASTQ via ENA/sra-tools) |
+| **ClinVar** | `clinvar search`, `variant` |
+| **SNP** | `snp lookup` |
+| **Taxonomy** | `taxonomy lookup` |
+| **UniProt** | `uniprot search`, `fetch`, `sequence` (FASTA download) |
+| **KEGG** | `kegg pathway`, `link`, `disease`, `convert` |
+| **STRING** | `string partners`, `network`, `enrichment` |
+| **Ensembl** | `ensembl lookup`, `vep`, `xrefs` |
+| **Enrichr** | `enrichr analyze` |
+
+## Output formats
+
+```bash
+biocli gene info 7157 -f json    # JSON (default for workflow commands)
+biocli gene info 7157 -f table   # Table (default for atomic commands)
+biocli gene info 7157 -f yaml    # YAML
+biocli gene info 7157 -f csv     # CSV
+biocli gene info 7157 -f plain   # Plain text
+```
+
+## Agent-first result schema
+
+All workflow commands (`aggregate *`) return a standard `BiocliResult` envelope:
+
+```json
+{
+  "data": { ... },
+  "ids": { "ncbiGeneId": "7157", "uniprotAccession": "P04637", ... },
+  "sources": ["NCBI Gene", "UniProt", "KEGG", "STRING"],
+  "warnings": [],
+  "queriedAt": "2026-04-03T10:00:00.000Z",
+  "organism": "Homo sapiens",
+  "query": "TP53"
+}
+```
+
+- `data` — the actual result payload
+- `ids` — cross-database identifiers for the queried entity
+- `sources` — which databases contributed data
+- `warnings` — partial failures, ambiguous matches (never silently hidden)
+- `queriedAt` — ISO timestamp for reproducibility
+- `organism` — species context
+
+## Configuration
+
+```bash
+biocli config set api_key YOUR_NCBI_KEY   # Optional: increases NCBI rate limit 3→10 req/s
+biocli config set email you@example.com
+biocli config show
+```
+
+Config stored at `~/.biocli/config.yaml`.
+
+## Rate limits
+
+| Database | Rate | Auth |
+|----------|------|------|
+| NCBI | 3/s (10/s with API key) | Optional API key |
+| UniProt | 50/s | None |
+| KEGG | 10/s | None |
+| STRING | 1/s | None |
+| Ensembl | 15/s | None |
+| Enrichr | 5/s | None |
+
+All rate limits are enforced automatically per-database.
+
+## License
+
+MIT

package/dist/batch.d.ts

@@ -0,0 +1,20 @@
+/**
+ * Batch input support for biocli commands.
+ *
+ * Resolves a list of IDs/queries from:
+ *   1. --input <file>  (one ID per line)
+ *   2. --input -       (stdin, one per line)
+ *   3. Comma-separated positional arg (e.g. "TP53,BRCA1,EGFR")
+ *
+ * Returns null if no batch mode is detected (single-value execution).
+ */
+/**
+ * Parse a batch input source into an array of individual values.
+ * Returns null if the input is a single non-batch value.
+ */
+export declare function parseBatchInput(positionalValue: string | undefined, inputFlag: string | undefined): string[] | null;
+/**
+ * Merge batch results into a flat array.
+ * Handles both plain arrays and ResultWithMeta objects.
+ */
+export declare function mergeBatchResults(results: unknown[]): unknown[];

package/dist/batch.js

@@ -0,0 +1,69 @@
+/**
+ * Batch input support for biocli commands.
+ *
+ * Resolves a list of IDs/queries from:
+ *   1. --input <file>  (one ID per line)
+ *   2. --input -       (stdin, one per line)
+ *   3. Comma-separated positional arg (e.g. "TP53,BRCA1,EGFR")
+ *
+ * Returns null if no batch mode is detected (single-value execution).
+ */
+import { readFileSync } from 'node:fs';
+/**
+ * Parse a batch input source into an array of individual values.
+ * Returns null if the input is a single non-batch value.
+ */
+export function parseBatchInput(positionalValue, inputFlag) {
+    // Priority 1: --input flag (file or stdin)
+    if (inputFlag) {
+        let raw;
+        if (inputFlag === '-') {
+            // Read from stdin (synchronous — assumes piped input, not interactive)
+            raw = readFileSync(0, 'utf-8');
+        }
+        else {
+            raw = readFileSync(inputFlag, 'utf-8');
+        }
+        const items = raw
+            .split(/[\n\r]+/)
+            .map(line => line.trim())
+            .filter(line => line && !line.startsWith('#'));
+        return items.length > 0 ? items : null;
+    }
+    // Priority 2: Comma-separated positional arg
+    if (positionalValue && positionalValue.includes(',')) {
+        const items = positionalValue
+            .split(',')
+            .map(s => s.trim())
+            .filter(Boolean);
+        if (items.length > 1)
+            return items;
+    }
+    return null;
+}
+/**
+ * Merge batch results into a flat array.
+ * Handles both plain arrays and ResultWithMeta objects.
+ */
+export function mergeBatchResults(results) {
+    const merged = [];
+    for (const result of results) {
+        if (result === null || result === undefined)
+            continue;
+        if (Array.isArray(result)) {
+            merged.push(...result);
+        }
+        else if (typeof result === 'object' && 'rows' in result) {
+            // ResultWithMeta
+            const rows = result.rows;
+            if (Array.isArray(rows))
+                merged.push(...rows);
+            else
+                merged.push(result);
+        }
+        else {
+            merged.push(result);
+        }
+    }
+    return merged;
+}

package/dist/build-manifest.d.ts

@@ -0,0 +1,38 @@
+#!/usr/bin/env node
+/**
+ * Build-time CLI manifest compiler.
+ *
+ * Scans all YAML/TS CLI definitions and pre-compiles them into a single
+ * manifest.json for instant cold-start registration (no runtime YAML parsing).
+ *
+ * Usage: npx tsx src/build-manifest.ts
+ * Output: dist/cli-manifest.json
+ */
+export interface ManifestEntry {
+    site: string;
+    name: string;
+    aliases?: string[];
+    description: string;
+    database?: string;
+    strategy: string;
+    args: Array<{
+        name: string;
+        type?: string;
+        default?: unknown;
+        required?: boolean;
+        positional?: boolean;
+        help?: string;
+        choices?: string[];
+    }>;
+    columns?: string[];
+    pipeline?: Record<string, unknown>[];
+    timeout?: number;
+    deprecated?: boolean | string;
+    replacedBy?: string;
+    /** 'yaml' or 'ts' — determines how executeCommand loads the handler */
+    type: 'yaml' | 'ts';
+    /** Relative path from clis/ dir, e.g. 'pubmed/search.yaml' or 'gene/info.js' */
+    modulePath?: string;
+}
+export declare function loadTsManifestEntries(filePath: string, site: string, importer?: (moduleHref: string) => Promise<unknown>): Promise<ManifestEntry[]>;
+export declare function buildManifest(): Promise<void>;

package/dist/build-manifest.js

@@ -0,0 +1,186 @@
+#!/usr/bin/env node
+/**
+ * Build-time CLI manifest compiler.
+ *
+ * Scans all YAML/TS CLI definitions and pre-compiles them into a single
+ * manifest.json for instant cold-start registration (no runtime YAML parsing).
+ *
+ * Usage: npx tsx src/build-manifest.ts
+ * Output: dist/cli-manifest.json
+ */
+import * as fs from 'node:fs';
+import * as path from 'node:path';
+import { fileURLToPath, pathToFileURL } from 'node:url';
+import yaml from 'js-yaml';
+import { getErrorMessage } from './errors.js';
+import { fullName, getRegistry } from './registry.js';
+import { parseYamlArgs } from './yaml-schema.js';
+import { isRecord } from './utils.js';
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const CLIS_DIR = path.resolve(__dirname, 'clis');
+const OUTPUT = path.resolve(__dirname, '..', 'dist', 'cli-manifest.json');
+// ── Helpers ─────────────────────────────────────────────────────────────────
+const CLI_MODULE_PATTERN = /\bcli\s*\(/;
+function toManifestArgs(args) {
+    return args.map(arg => ({
+        name: arg.name,
+        type: arg.type ?? 'str',
+        default: arg.default,
+        required: !!arg.required,
+        positional: arg.positional || undefined,
+        help: arg.help ?? '',
+        choices: arg.choices,
+    }));
+}
+function toTsModulePath(filePath, site) {
+    const baseName = path.basename(filePath, path.extname(filePath));
+    return `${site}/${baseName}.js`;
+}
+function isCliCommandValue(value, site) {
+    return isRecord(value)
+        && typeof value.site === 'string'
+        && value.site === site
+        && typeof value.name === 'string'
+        && Array.isArray(value.args);
+}
+function toManifestEntry(cmd, modulePath) {
+    return {
+        site: cmd.site,
+        name: cmd.name,
+        aliases: cmd.aliases,
+        description: cmd.description ?? '',
+        database: cmd.database,
+        strategy: (cmd.strategy ?? 'public').toString().toLowerCase(),
+        args: toManifestArgs(cmd.args),
+        columns: cmd.columns,
+        timeout: cmd.timeoutSeconds,
+        deprecated: cmd.deprecated,
+        replacedBy: cmd.replacedBy,
+        type: 'ts',
+        modulePath,
+    };
+}
+// ── YAML scanner ────────────────────────────────────────────────────────────
+function scanYaml(filePath, site) {
+    try {
+        const raw = fs.readFileSync(filePath, 'utf-8');
+        const def = yaml.load(raw);
+        if (!isRecord(def))
+            return null;
+        const cliDef = def;
+        const strategyStr = cliDef.strategy ?? 'public';
+        const strategy = strategyStr.toLowerCase();
+        const args = parseYamlArgs(cliDef.args);
+        return {
+            site: cliDef.site ?? site,
+            name: cliDef.name ?? path.basename(filePath, path.extname(filePath)),
+            description: cliDef.description ?? '',
+            database: cliDef.database,
+            strategy,
+            aliases: isRecord(cliDef) && Array.isArray(cliDef.aliases)
+                ? cliDef.aliases.filter((value) => typeof value === 'string')
+                : undefined,
+            args,
+            columns: cliDef.columns,
+            pipeline: cliDef.pipeline,
+            timeout: cliDef.timeout,
+            deprecated: cliDef.deprecated,
+            replacedBy: cliDef.replacedBy,
+            type: 'yaml',
+        };
+    }
+    catch (err) {
+        process.stderr.write(`Warning: failed to parse ${filePath}: ${getErrorMessage(err)}\n`);
+        return null;
+    }
+}
+// ── TS scanner ──────────────────────────────────────────────────────────────
+export async function loadTsManifestEntries(filePath, site, importer = moduleHref => import(moduleHref)) {
+    try {
+        const src = fs.readFileSync(filePath, 'utf-8');
+        // Helper/test modules should not appear as CLI commands in the manifest.
+        if (!CLI_MODULE_PATTERN.test(src))
+            return [];
+        // Snapshot registry keys before the import.
+        const before = new Set(getRegistry().keys());
+        // Import the module — its top-level cli() calls register commands.
+        const moduleExports = await importer(pathToFileURL(filePath).href);
+        // Collect newly registered commands.
+        const entries = [];
+        const modulePath = toTsModulePath(filePath, site);
+        // Strategy 1: Check exports for CliCommand objects.
+        if (moduleExports && typeof moduleExports === 'object') {
+            for (const value of Object.values(moduleExports)) {
+                if (isCliCommandValue(value, site)) {
+                    entries.push(toManifestEntry(value, modulePath));
+                }
+            }
+        }
+        // Strategy 2: Check newly registered commands in the registry.
+        if (entries.length === 0) {
+            for (const [key, cmd] of getRegistry()) {
+                if (!before.has(key) && key === fullName(cmd) && cmd.site === site) {
+                    entries.push(toManifestEntry(cmd, modulePath));
+                }
+            }
+        }
+        return entries;
+    }
+    catch (err) {
+        process.stderr.write(`Warning: failed to load TS adapter ${filePath}: ${getErrorMessage(err)}\n`);
+        return [];
+    }
+}
+// ── Main build function ─────────────────────────────────────────────────────
+export async function buildManifest() {
+    const manifest = [];
+    // Check that CLIS_DIR exists
+    if (!fs.existsSync(CLIS_DIR)) {
+        process.stderr.write(`Warning: CLIs directory not found at ${CLIS_DIR}\n`);
+        // Write empty manifest
+        const outputDir = path.dirname(OUTPUT);
+        if (!fs.existsSync(outputDir))
+            fs.mkdirSync(outputDir, { recursive: true });
+        fs.writeFileSync(OUTPUT, JSON.stringify([], null, 2) + '\n', 'utf-8');
+        return;
+    }
+    const siteDirs = fs.readdirSync(CLIS_DIR, { withFileTypes: true })
+        .filter(d => d.isDirectory() && !d.name.startsWith('.') && !d.name.startsWith('_'));
+    for (const siteDir of siteDirs) {
+        const site = siteDir.name;
+        const sitePath = path.join(CLIS_DIR, site);
+        const files = fs.readdirSync(sitePath);
+        for (const file of files) {
+            if (file.startsWith('.'))
+                continue; // skip hidden/AppleDouble files
+            const filePath = path.join(sitePath, file);
+            if (file.endsWith('.yaml') || file.endsWith('.yml')) {
+                const entry = scanYaml(filePath, site);
+                if (entry)
+                    manifest.push(entry);
+            }
+            else if ((file.endsWith('.js') && !file.endsWith('.d.js')) ||
+                (file.endsWith('.ts') && !file.endsWith('.d.ts') && !file.endsWith('.test.ts'))) {
+                const entries = await loadTsManifestEntries(filePath, site);
+                manifest.push(...entries);
+            }
+        }
+    }
+    // Write manifest
+    const outputDir = path.dirname(OUTPUT);
+    if (!fs.existsSync(outputDir))
+        fs.mkdirSync(outputDir, { recursive: true });
+    fs.writeFileSync(OUTPUT, JSON.stringify(manifest, null, 2) + '\n', 'utf-8');
+    process.stdout.write(`Manifest compiled: ${manifest.length} commands → ${OUTPUT}\n`);
+}
+// ── Run directly ────────────────────────────────────────────────────────────
+// ESM equivalent of if (require.main === module)
+const isMain = process.argv[1] &&
+    (process.argv[1] === fileURLToPath(import.meta.url) ||
+        process.argv[1].endsWith('/build-manifest.js'));
+if (isMain) {
+    buildManifest().catch((err) => {
+        console.error('Manifest build failed:', err);
+        process.exit(1);
+    });
+}

package/dist/cache.d.ts

@@ -0,0 +1,28 @@
+/**
+ * Local file-based cache for biocli API responses.
+ *
+ * Cache layout: ~/.biocli/cache/{database}/{command}/{sha256}.json
+ * Each entry stores: { data, cachedAt, ttlMs, key }
+ *
+ * TTL default: 24 hours. Configurable via `biocli config set cache.ttl <hours>`.
+ * Disable per-request with --no-cache global flag.
+ */
+export interface CacheStats {
+    totalEntries: number;
+    totalSizeBytes: number;
+    databases: Record<string, number>;
+    oldestEntry: string | null;
+    newestEntry: string | null;
+}
+/** Build a stable cache key from command args. */
+export declare function buildCacheKey(database: string, command: string, args: Record<string, unknown>): string;
+/** Get a cached result if it exists and hasn't expired. */
+export declare function getCached(database: string, command: string, argsKey: string, ttlMs?: number): unknown | null;
+/** Store a result in the cache. */
+export declare function setCached(database: string, command: string, argsKey: string, data: unknown, ttlMs?: number): void;
+/** Get cache statistics. */
+export declare function getStats(): CacheStats;
+/** Clear all cache entries. Returns number of entries deleted. */
+export declare function clearCache(): number;
+/** Get the default TTL in milliseconds. */
+export declare function getDefaultTtlMs(): number;