open-research 0.1.26 → 1.0.0

package/README.md

@@ -15,11 +15,6 @@
 
 ## Install
 
-```bash
-# curl
-curl -fsSL https://raw.githubusercontent.com/gangj277/open-research/main/install.sh | bash
-```
-
 ```bash
 # npm
 npm install -g open-research
@@ -64,170 +59,228 @@ Then ask anything:
   and identify gaps in the literature
 ```
 
-The agent searches arXiv, Semantic Scholar, and OpenAlex — reads papers, runs analysis scripts, writes source-grounded notes, and drafts artifacts in your local workspace.
+The agent searches arXiv, Semantic Scholar, and OpenAlex — reads papers (including PDFs), extracts evidence for and against your research target, runs analysis scripts, writes source-grounded notes, and drafts artifacts in your local workspace.
 
 ## How is this different from Cursor / Claude Code?
 
 Those are coding agents. Open Research is a **research agent**.
 
-It has tools that coding agents don't: federated academic paper search, PDF extraction, source-grounded synthesis, sub-agent delegation, and pluggable research skills (novelty checker, experiment designer, reviewer response manager, etc.).
+It has tools that coding agents don't: federated academic paper search with target extraction, web search with evidence analysis, PDF parsing from URLs, a research knowledge graph (ontology), sub-agent delegation, and pluggable research skills.
 
 Everything stays local. Your workspace is a directory with `sources/`, `notes/`, `papers/`, `experiments/`. The agent reads and writes to it. Risky edits go to a review queue.
 
-## Agent Modes
+## Research Ontology
 
-Open Research operates in three modes. Cycle with `Shift+Tab`:
+The agent automatically builds a **structured knowledge graph** as you research. Every paper read, claim made, finding extracted, and method discovered gets captured as typed, connected notes.
 
-### Manual Review (default)
+### How it works
 
-The agent proposes changes. You review and accept (`a`) or reject (`r`) each one. Best for sensitive work where every edit matters.
+You don't manage the ontology manually — it emerges from conversation:
 
-### Auto-Approve
+1. **After each turn**, a background ontology manager extracts knowledge from the conversation and tool outputs
+2. **Before each turn**, a relevance agent selects notes related to your current question and injects them as context
+3. **During a turn**, the agent can query the ontology for evidence, contradictions, and connections
 
-All file writes are applied immediately without review. Best for exploratory work where speed matters more than control.
+### Note types
 
-### Auto-Research
+| Kind | What it captures |
+|------|-----------------|
+| `source` | Citable origin — paper, URL, dataset, book |
+| `finding` | Specific result extracted from a source |
+| `claim` | Argument or assertion in the research |
+| `question` | Open gap, uncertainty, research question |
+| `method` | Methodology or analytical technique |
+| `insight` | Synthesis connecting multiple findings |
 
-The most powerful mode. A two-phase autonomous research workflow:
+### Connections
 
-**Phase 1 — Planning.** The agent enters read-only planning mode. It reads your workspace, searches academic databases, and asks you clarifying questions. It then produces a **Research Charter** — a structured contract defining:
+Notes are linked with typed edges: `supports`, `contradicts`, `derived-from`, `relates-to` — each with a strength (strong/moderate/weak) and a context explaining *why* the connection exists.
 
-- The research question (precisely stated)
-- Success criteria (what "done" looks like)
-- Scope boundaries (what's explicitly out of scope)
-- Known starting points (papers, data, leads)
-- Proposed investigation steps
+### Slash commands
 
-You review the charter and either approve it, send it back for revision, or cancel.
+```
+/ontology                 Overview — note counts, contradictions, open questions
+/ontology claims          List all claims with evidence counts
+/ontology conflicts       Show all contradiction pairs
+/ontology around <term>   Find notes related to a topic with their edges
+/ontology delete <id>     Remove a note and its edges
+```
 
-**Phase 2 — Execution.** Once approved, the agent executes the charter autonomously — searching papers, reading sources, running analysis code, writing notes, and producing artifacts. It runs until the success criteria are met or it hits a dead end and reports what it found.
+### Agent tools
 
-## Sub-Agents
+| Tool | What it does |
+|------|-------------|
+| `query_ontology` | Ask research questions — a sub-agent traverses the graph and returns a synthesized answer |
+| `ontology_status` | Get a snapshot: note counts, contradictions, unsupported claims, open questions |
+
+## Search with Target Extraction
 
-The main agent can delegate exploration tasks to lightweight sub-agents that run on their own context window. This keeps the main agent's context clean and improves token efficiency.
+Both search tools use a **target extraction pipeline**: discover sources → fetch content (PDFs, HTML, abstracts) → extract evidence with gpt-5.4-mini → return structured findings. The main agent never sees raw page content.
+
+### Academic search
+
+```
+search_external_sources(
+  target: "What speedups do efficient attention methods achieve",
+  searches: [{ query: "transformer attention efficiency" }]
+)
+```
+
+Returns structured findings per paper:
+- **Supports**: Evidence supporting your target
+- **Contradicts**: Evidence challenging your target
+- **Related**: Relevant context (methods, definitions, frameworks)
+- **Summary**: One-paragraph synthesis
+- **Relevance score**: 0-10
+
+The pipeline handles PDFs from URLs (arXiv, open access journals) — downloads, parses via pdfjs, extracts text from the first 5 pages. arXiv papers use the abstract directly (zero network cost).
+
+### Web search
 
 ```
-launch_subagent(type: "explore", goal: "Find all files related to the auth flow...")
+web_search(
+  target: "Best practices for PyTorch DataLoader multi-GPU",
+  query: "pytorch dataloader num_workers multi gpu"
+)
 ```
 
-The **explore** sub-agent runs on `gpt-5.4-mini` with high reasoning effort. It has read-only tools (`read_file`, `list_directory`, `search_workspace`) and returns a concise, conclusion-oriented summary. The main agent gets the answer without burning its context on raw file reads.
+Same extraction pipeline, different discovery backend:
+- **Default**: DuckDuckGo HTML scraping (zero config, no API key)
+- **Upgrade**: Brave Search API for better results — set via `/api-keys brave <key>` (~1,000 free queries/month)
+
+## Agent Modes
+
+Three modes. Cycle with `Shift+Tab`:
+
+- **Manual Review** (default) — agent proposes changes, you accept (`a`) or reject (`r`)
+- **Auto-Approve** — all file writes applied immediately
+- **Auto-Research** — two-phase: planning (produces a Research Charter) → autonomous execution
+
+## Sub-Agents
+
+The main agent delegates exploration to lightweight sub-agents running on their own context window.
+
+The **explore** sub-agent (gpt-5.4-mini, high reasoning) has read-only tools and returns concise findings. The main agent gets answers without burning its context on raw file reads.
+
+## Task Tracking
 
-Sub-agents are extensible — new types can be added as config entries without changing the tool schema.
+For multi-step research, the agent creates a visible task checklist:
+
+```
+  ⠋ Searching for chain-of-thought papers...
+  ○ Read and extract from top papers
+  ○ Build comparison table
+  ✓ 1 completed
+```
+
+Tasks are injected into the agent's context on every turn — it always knows what it's done and what's next. Toggle with `Ctrl+T`.
 
 ## Research Skills
 
-Skills are pluggable research methodologies — detailed workflow prompts that guide the agent through a specific research task. Type `/<skill-name>` to activate.
+Skills are pluggable research methodologies. Type `/<skill-name>` to activate.
 
 ### Ideation & Discovery
 
 | Skill | What it does |
 |---|---|
-| **`/novelty-checker`** | Quick "has this been done?" assessment. Decomposes ideas into technique/domain/claim components, runs 5-8 search variations, and delivers a verdict: Novel, Partially novel, Incremental, or Already done — with closest existing work, white space map, and pivot recommendations. |
-| **`/source-scout`** | Systematically finds papers the workspace is missing. Searches with multiple query variations, evaluates relevance by citation count and venue, fetches key papers, produces a prioritized scout report with gap analysis. |
-| **`/paper-explainer`** | Two modes: (1) Single paper deep read with structured breakdown including methodological red flags, or (2) Multi-paper comparison table with structured extraction across 6-10 dimensions (Elicit-style) and cross-paper synthesis. |
+| **`/novelty-checker`** | Quick "has this been done?" assessment with verdict: Novel, Partially novel, Incremental, or Already done. |
+| **`/source-scout`** | Finds papers the workspace is missing with gap analysis and prioritized scout report. |
+| **`/paper-explainer`** | Single paper deep read with red flags, or multi-paper comparison table (Elicit-style). |
 
 ### Critical Evaluation
 
 | Skill | What it does |
 |---|---|
-| **`/devils-advocate`** | Stress-tests every claim in the workspace. Attacks each one through six lenses: evidence gap, logical gap, scope overclaim, alternative explanation, replication concern, and statistical concern. Actively searches for counter-evidence. Rates each weakness as Critical/Significant/Minor. |
-| **`/methodology-critic`** | Reviews study design, sample selection, controls, measurement validity, statistical methods, and reporting completeness. If code is available, reproduces the analysis to verify results. Rates each study Rigorous/Acceptable/Concerning/Flawed. |
-| **`/evidence-adjudicator`** | Judges conflicting claims using a formal evidence hierarchy (meta-analysis → RCT → cohort → case study → opinion). Checks for bias and conflicts of interest. Delivers a clear verdict with evidence ratings: Strong/Moderate/Weak/Insufficient. |
+| **`/devils-advocate`** | Stress-tests claims through six lenses. Actively searches for counter-evidence. |
+| **`/methodology-critic`** | Reviews study design, statistical methods, reproducibility. Rates Rigorous to Flawed. |
+| **`/evidence-adjudicator`** | Judges conflicting claims using formal evidence hierarchy. Delivers verdict with ratings. |
 
 ### Analysis & Experimentation
 
 | Skill | What it does |
 |---|---|
-| **`/experiment-designer`** | Autonomous proof engine. Takes a hypothesis and runs the full loop: formalize → design minimal experiment → write code → run it → analyze results → iterate (up to 5x) until proven or disproven. All artifacts saved to `experiments/` with versioned scripts. |
-| **`/data-analyst`** | End-to-end statistical analysis: explore data (distributions, missing values) → clean (with documented decisions) → analyze (appropriate tests, mandatory effect sizes and confidence intervals) → visualize (matplotlib/seaborn) → interpret with honest caveats. |
+| **`/experiment-designer`** | Autonomous proof engine: hypothesis → experiment → code → run → iterate. |
+| **`/data-analyst`** | End-to-end statistical analysis with mandatory effect sizes and confidence intervals. |
 
 ### Writing & Revision
 
 | Skill | What it does |
 |---|---|
-| **`/draft-paper`** | Drafts a publication-quality LaTeX paper: gathers workspace evidence → outlines the argument → writes each section (intro through conclusion) → generates BibTeX from sources → self-reviews for unsupported claims and argument flow. |
-| **`/reviewer-response`** | Parses peer review comments into numbered items (R1.1, R1.2...), classifies as Major/Minor/Praise/Question, flags contradictions between reviewers, generates a point-by-point response letter with verbatim quotes and specific change locations, and maintains a revision completion checklist. |
+| **`/draft-paper`** | Drafts publication-quality LaTeX with BibTeX from workspace sources. |
+| **`/reviewer-response`** | Parses peer review, generates point-by-point response letter with revision tracking. |
 
 ### Meta
 
 | Skill | What it does |
 |---|---|
-| **`/skill-creator`** | Create custom skills in `~/.open-research/skills/`. Full guidance on the SKILL.md format, directory structure, prompt design, and validation — with quality guidelines for writing effective workflow prompts. |
+| **`/skill-creator`** | Create custom skills with full format guide and validation. |
 
 ## Memory
 
-The agent learns about you automatically. After each conversation, a background process identifies facts worth remembering — your research field, preferred tools, current projects, methodological preferences.
+The agent learns about you automatically — research field, preferred tools, methodological preferences.
 
-Memories are stored at two levels:
-- **Global** (`~/.open-research/memory.json`) — your profile, preferences, expertise
+Two levels:
+- **Global** (`~/.open-research/memory.json`) — your profile, preferences
 - **Project** (`<workspace>/.open-research/memory.json`) — project-specific context
 
-Only relevant memories are injected each turn based on query similarity, keeping the context window efficient.
-
 ```
-/memory              View all stored memories
+/memory              View stored memories
 /memory clear        Delete everything
-/memory delete <id>  Remove a specific memory
+/memory delete <id>  Remove one
 ```
 
 ## Live LaTeX Preview
 
-When the agent drafts a paper, preview it instantly:
-
 ```
 /preview papers/draft.tex
 ```
 
-Opens a localhost server in your browser with:
-- Sections, math (KaTeX), citations, lists rendered as styled HTML
-- Auto-reload — the page refreshes every time the file changes
-- Dark theme matching the CLI aesthetic
-- No LaTeX installation required for preview
-
-For final PDF output, the agent compiles with `pdflatex` or `tectonic` via `run_command`.
+Opens a localhost server with KaTeX math, auto-reload on file changes, and dark theme. No LaTeX installation required.
 
 ## Tools
 
-The agent has 14 tools with full filesystem and shell access:
-
 | Tool | Description |
 |---|---|
 | `read_file` | Read any file — streaming, binary detection, `~` expansion |
 | `read_pdf` | Extract text from PDFs with page-range selection |
 | `run_command` | Shell execution — Python, R, LaTeX, curl, git, anything |
 | `list_directory` | Explore directory trees with depth control |
-| `search_external_sources` | Federated search: arXiv + Semantic Scholar + OpenAlex |
-| `fetch_url` | Fetch web pages and APIs, HTML auto-converted to text via cheerio |
+| `search_external_sources` | Academic search with target extraction (arXiv + Semantic Scholar + OpenAlex) |
+| `web_search` | Web search with target extraction (DuckDuckGo or Brave) |
+| `fetch_url` | Fetch a specific URL, HTML auto-converted to text |
 | `write_new_file` | Create workspace files |
 | `update_existing_file` | Edit existing files with review policy |
-| `ask_user` | Pause and ask the user a question with selectable options |
+| `ask_user` | Pause and ask the user a question |
 | `search_workspace` | Full-text search across workspace files |
 | `create_paper` | Create LaTeX paper drafts |
 | `load_skill` | Activate a research skill |
-| `read_skill_reference` | Read reference materials from active skills |
-| `launch_subagent` | Delegate tasks to lightweight sub-agents with isolated context |
+| `launch_subagent` | Delegate tasks to lightweight sub-agents |
+| `create_tasks` | Create a research task checklist |
+| `update_task` | Update task status and details |
+| `query_ontology` | Query the research knowledge graph |
+| `ontology_status` | Get ontology overview — notes, contradictions, gaps |
 
 ## Commands
 
 | Command | Description |
 |---|---|
 | `/auth` | Connect OpenAI account via browser |
-| `/auth-codex` | Import existing Codex CLI auth |
 | `/init` | Initialize workspace in current directory |
 | `/skills` | List available research skills |
+| `/ontology` | View or manage the research ontology |
 | `/preview <file>` | Live-preview a LaTeX file in browser |
 | `/memory` | View or manage stored memories |
-| `/api-keys` | Set API keys for Semantic Scholar, OpenAlex |
-| `/config` | View or change settings (model, theme, mode, apikey) |
-| `/compact` | Manually compress conversation to save context |
-| `/cost` | Show token usage and cost for the session |
-| `/context` | Show context window usage — how full it is |
-| `/btw` | Ask a side question without affecting the main conversation |
+| `/api-keys` | Set API keys (Semantic Scholar, OpenAlex, Brave) |
+| `/config` | Settings (model, theme, mode, apikey) |
+| `/compact` | Compress conversation to save context |
+| `/cost` | Token usage for the session |
+| `/context` | Context window usage |
+| `/btw` | Side question without affecting main conversation |
 | `/export` | Export conversation as markdown |
-| `/diff` | Show files the agent has changed this session |
-| `/doctor` | Diagnose auth, connectivity, and tool availability |
+| `/diff` | Files changed this session |
+| `/doctor` | Diagnose auth, connectivity, tools |
 | `/resume` | Resume a previous session |
-| `/clear` | Start a new conversation |
+| `/clear` | Start fresh |
 | `/help` | Show all commands |
 
 ## Workspace
@@ -239,24 +292,30 @@ my-research/
   artifacts/       # Generated outputs
   papers/          # LaTeX paper drafts
   experiments/     # Analysis scripts, results, hypotheses
-  .open-research/  # Workspace metadata, sessions, project memory
-    AGENTS.md      # Auto-generated project context (injected into system prompt)
+  .open-research/
+    AGENTS.md      # Auto-generated project context
+    ontology.json  # Research knowledge graph
+    tasks.json     # Task tracking state
+    memory.json    # Project-scoped memories
+    sessions/      # Chat history
 ```
 
 ## Features
 
-- **Senior research director persona** — concise, conclusion-oriented responses. Findings first, evidence second.
-- **Sub-agent delegation** — explore agent handles codebase navigation on its own context, returns summaries
-- **Terminal markdown** — bold, italic, code blocks, headings rendered natively with chalk
-- **Autocomplete** — slash commands, skills, and @file mentions in a scrollable arrow-key dropdown
-- **Condensed tool activity** — grouped summary per turn instead of per-tool spam, with live progress in footer
-- **Shift+Enter** — multi-line input
+- **Research ontology** — automatic knowledge graph that captures sources, findings, claims, contradictions, and connections as you work
+- **Target extraction search** — academic and web search that returns structured evidence (supports/contradicts/related), not raw pages
+- **PDF parsing from URLs** — fetches and extracts text from academic PDFs directly during search
+- **Task tracking** — visible checklist for multi-step work, injected into agent context every turn
+- **Sub-agent delegation** — explore agent navigates the workspace on its own context, returns summaries
+- **Init banner** — version, model, context window, workspace info at launch
+- **Terminal markdown** — bold, italic, code blocks, headings rendered natively
+- **Autocomplete** — commands, skills, and @file mentions in a scrollable dropdown
+- **Condensed tool activity** — grouped summary per turn, Ctrl+O to expand
 - **Slash command highlighting** — commands appear in blue as you type
 - **Context management** — automatic two-phase compaction at 90% of context window
-- **Token tracking** — context usage visible in the status bar (input/output/reasoning/cache breakdown)
-- **AGENTS.md** — auto-generated project context file, updated after each turn, injected into system prompt
-- **Two-tier memory** — global + project-level, with selective retrieval based on query relevance
-- **Update notifications** — checks for new versions on launch
+- **Token tracking** — context usage in the status bar
+- **Two-tier memory** — global + project-level, selective retrieval per turn
+- **AGENTS.md** — auto-generated project context, injected into system prompt
 
 ## Development
 

package/dist/chunk-3RG5ZIWI.js

@@ -0,0 +1,10 @@
+var __require = /* @__PURE__ */ ((x) => typeof require !== "undefined" ? require : typeof Proxy !== "undefined" ? new Proxy(x, {
+  get: (a, b) => (typeof require !== "undefined" ? require : a)[b]
+}) : x)(function(x) {
+  if (typeof require !== "undefined") return require.apply(this, arguments);
+  throw Error('Dynamic require of "' + x + '" is not supported');
+});
+
+export {
+  __require
+};

package/dist/chunk-3WM33M3O.js

@@ -0,0 +1,38 @@
+// src/lib/ontology/store.ts
+import fs from "fs/promises";
+import path from "path";
+function getOntologyPath(workspaceDir) {
+  return path.join(workspaceDir, ".open-research", "ontology.json");
+}
+var EMPTY_ONTOLOGY = { version: 1, notes: [] };
+async function loadOntology(workspaceDir) {
+  try {
+    const raw = await fs.readFile(getOntologyPath(workspaceDir), "utf8");
+    const parsed = JSON.parse(raw);
+    if (!parsed.notes || !Array.isArray(parsed.notes)) return { ...EMPTY_ONTOLOGY };
+    return parsed;
+  } catch {
+    return { ...EMPTY_ONTOLOGY };
+  }
+}
+async function saveOntology(ontology, workspaceDir) {
+  const filePath = getOntologyPath(workspaceDir);
+  const tmpPath = filePath + ".tmp";
+  await fs.mkdir(path.dirname(filePath), { recursive: true });
+  await fs.writeFile(tmpPath, JSON.stringify(ontology, null, 2), "utf8");
+  await fs.rename(tmpPath, filePath);
+}
+async function cleanupStaleTmp(workspaceDir) {
+  const tmpPath = getOntologyPath(workspaceDir) + ".tmp";
+  try {
+    await fs.unlink(tmpPath);
+  } catch {
+  }
+}
+
+export {
+  getOntologyPath,
+  loadOntology,
+  saveOntology,
+  cleanupStaleTmp
+};

package/dist/chunk-I5NVYKG7.js

@@ -0,0 +1,37 @@
+// src/lib/fs/paths.ts
+import os from "os";
+import path from "path";
+function resolveHomeDir(options) {
+  return options?.homeDir ?? os.homedir();
+}
+function getOpenResearchRoot(options) {
+  return path.join(resolveHomeDir(options), ".open-research");
+}
+function getOpenResearchAuthFile(options) {
+  return path.join(getOpenResearchRoot(options), "auth.json");
+}
+function getOpenResearchConfigFile(options) {
+  return path.join(getOpenResearchRoot(options), "config.json");
+}
+function getOpenResearchSkillsDir(options) {
+  return path.join(getOpenResearchRoot(options), "skills");
+}
+function getWorkspaceMetaDir(workspaceDir) {
+  return path.join(workspaceDir, ".open-research");
+}
+function getWorkspaceProjectFile(workspaceDir) {
+  return path.join(getWorkspaceMetaDir(workspaceDir), "project.json");
+}
+function getWorkspaceSessionsDir(workspaceDir) {
+  return path.join(getWorkspaceMetaDir(workspaceDir), "sessions");
+}
+
+export {
+  getOpenResearchRoot,
+  getOpenResearchAuthFile,
+  getOpenResearchConfigFile,
+  getOpenResearchSkillsDir,
+  getWorkspaceMetaDir,
+  getWorkspaceProjectFile,
+  getWorkspaceSessionsDir
+};

package/dist/chunk-IOR7G25X.js

@@ -0,0 +1,215 @@
+// src/lib/ontology/read-tools.ts
+var STOP_WORDS = /* @__PURE__ */ new Set([
+  "the",
+  "a",
+  "an",
+  "is",
+  "are",
+  "was",
+  "were",
+  "be",
+  "been",
+  "being",
+  "have",
+  "has",
+  "had",
+  "do",
+  "does",
+  "did",
+  "will",
+  "would",
+  "could",
+  "should",
+  "may",
+  "might",
+  "can",
+  "shall",
+  "to",
+  "of",
+  "in",
+  "for",
+  "on",
+  "with",
+  "at",
+  "by",
+  "from",
+  "as",
+  "into",
+  "through",
+  "about",
+  "and",
+  "but",
+  "or",
+  "not",
+  "no",
+  "if",
+  "then",
+  "than",
+  "so",
+  "that",
+  "this",
+  "it",
+  "its",
+  "i",
+  "me",
+  "my",
+  "we",
+  "our",
+  "you",
+  "your",
+  "what",
+  "which",
+  "who",
+  "how",
+  "when",
+  "where",
+  "why"
+]);
+function tokenize(text) {
+  return text.toLowerCase().replace(/[^a-z0-9\s-]/g, " ").split(/\s+/).filter((w) => w.length > 2 && !STOP_WORDS.has(w));
+}
+function getNote(ontology, noteId) {
+  return ontology.notes.find((n) => n.id === noteId) ?? null;
+}
+function hasMutualIncoming(ontology, noteId, relation) {
+  return ontology.notes.some(
+    (other) => other.id !== noteId && other.edges.some(
+      (e) => e.targetId === noteId && e.relation === relation && e.direction === "mutual"
+    )
+  );
+}
+function searchNotes(ontology, params) {
+  const { queries, kind, confidence, hasEdge, missingEdge, limit = 10 } = params;
+  let candidates = ontology.notes;
+  if (kind) {
+    candidates = candidates.filter((n) => n.kind === kind);
+  }
+  if (confidence) {
+    candidates = candidates.filter((n) => n.confidence === confidence);
+  }
+  if (hasEdge) {
+    candidates = candidates.filter(
+      (n) => n.edges.some((e) => e.relation === hasEdge) || hasMutualIncoming(ontology, n.id, hasEdge)
+    );
+  }
+  if (missingEdge) {
+    candidates = candidates.filter(
+      (n) => !n.edges.some((e) => e.relation === missingEdge) && !hasMutualIncoming(ontology, n.id, missingEdge)
+    );
+  }
+  if (!queries || queries.length === 0) {
+    return candidates.sort((a, b2) => b2.updatedAt.localeCompare(a.updatedAt)).slice(0, limit);
+  }
+  const queryTokenSets = queries.map((q) => tokenize(q));
+  const N = candidates.length;
+  if (N === 0) return [];
+  const docTokensCache = /* @__PURE__ */ new Map();
+  let totalDocLen = 0;
+  for (const note of candidates) {
+    const tokens = tokenize(note.content);
+    docTokensCache.set(note.id, tokens);
+    totalDocLen += tokens.length;
+  }
+  const avgDocLen = totalDocLen / N;
+  const df = /* @__PURE__ */ new Map();
+  for (const note of candidates) {
+    const uniqueTokens = new Set(docTokensCache.get(note.id));
+    for (const token of uniqueTokens) {
+      df.set(token, (df.get(token) ?? 0) + 1);
+    }
+  }
+  const k1 = 1.2;
+  const b = 0.75;
+  const scored = candidates.map((note) => {
+    const noteTokens = docTokensCache.get(note.id);
+    const docLen = noteTokens.length;
+    const tf = /* @__PURE__ */ new Map();
+    for (const token of noteTokens) {
+      tf.set(token, (tf.get(token) ?? 0) + 1);
+    }
+    let bestBM25 = 0;
+    for (const queryTokens of queryTokenSets) {
+      let score = 0;
+      for (const qt of queryTokens) {
+        const termFreq = tf.get(qt) ?? 0;
+        const docFreq = df.get(qt) ?? 0;
+        if (termFreq === 0) continue;
+        const idf = Math.log((N - docFreq + 0.5) / (docFreq + 0.5) + 1);
+        const tfNorm = termFreq * (k1 + 1) / (termFreq + k1 * (1 - b + b * docLen / avgDocLen));
+        score += idf * tfNorm;
+      }
+      bestBM25 = Math.max(bestBM25, score);
+    }
+    let metaBonus = 0;
+    if (note.kind === "source" && note.meta) {
+      const metaText = [
+        note.meta.authors,
+        note.meta.venue,
+        note.meta.year?.toString()
+      ].filter(Boolean).join(" ");
+      const metaTokens = new Set(tokenize(metaText));
+      for (const queryTokens of queryTokenSets) {
+        const hits = queryTokens.filter((qt) => metaTokens.has(qt)).length;
+        metaBonus = Math.max(metaBonus, hits * 0.5);
+      }
+    }
+    return { note, score: bestBM25 + metaBonus };
+  });
+  return scored.filter((s) => s.score > 0).sort((a, b2) => b2.score - a.score).slice(0, limit).map((s) => s.note);
+}
+function getConnections(ontology, noteId, depth = 1) {
+  const clampedDepth = Math.min(Math.max(depth, 1), 3);
+  const root = getNote(ontology, noteId);
+  if (!root) return { root: null, connected: [] };
+  const visited = /* @__PURE__ */ new Set([noteId]);
+  let frontier = [noteId];
+  for (let d = 0; d < clampedDepth; d++) {
+    const nextFrontier = [];
+    for (const currentId of frontier) {
+      const current = getNote(ontology, currentId);
+      if (!current) continue;
+      for (const edge of current.edges) {
+        if (!visited.has(edge.targetId)) {
+          visited.add(edge.targetId);
+          nextFrontier.push(edge.targetId);
+        }
+      }
+      for (const other of ontology.notes) {
+        if (visited.has(other.id)) continue;
+        const hasMutual = other.edges.some(
+          (e) => e.targetId === currentId && e.direction === "mutual"
+        );
+        if (hasMutual) {
+          visited.add(other.id);
+          nextFrontier.push(other.id);
+        }
+      }
+    }
+    frontier = nextFrontier;
+    if (frontier.length === 0) break;
+  }
+  visited.delete(noteId);
+  const connected = [...visited].map((id) => getNote(ontology, id)).filter((n) => n !== null);
+  return { root, connected };
+}
+function normalizeTitle(text) {
+  return text.toLowerCase().replace(/[^a-z0-9\s]/g, "").replace(/\s+/g, " ").trim();
+}
+function findExistingSource(ontology, meta) {
+  for (const note of ontology.notes) {
+    if (note.kind !== "source" || !note.meta) continue;
+    if (meta.doi && note.meta.doi && meta.doi === note.meta.doi) return note;
+    if (meta.url && note.meta.url && meta.url === note.meta.url) return note;
+    if (meta.authors && meta.year && note.meta.authors && note.meta.year && meta.year === note.meta.year && normalizeTitle(meta.authors) === normalizeTitle(note.meta.authors)) {
+      return note;
+    }
+  }
+  return null;
+}
+
+export {
+  getNote,
+  searchNotes,
+  getConnections,
+  findExistingSource
+};