gsurgeon 1.0.0__tar.gz

gsurgeon-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 JoM
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

gsurgeon-1.0.0/PKG-INFO

@@ -0,0 +1,111 @@
+Metadata-Version: 2.4
+Name: gsurgeon
+Version: 1.0.0
+Summary: A tool to dissect biology of model organisms using genomic information
+License-Expression: MIT
+License-File: LICENSE
+Author: Johannes Medagbe
+Author-email: johanmedagbe@gmail.com
+Requires-Python: >=3.11,<3.15
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: bio (>=1.8.3,<2.0.0)
+Requires-Dist: botocore (>=1.43.11,<2.0.0)
+Requires-Dist: dotenv (>=0.9.9,<0.10.0)
+Requires-Dist: dspy (>=3.2.1,<4.0.0)
+Requires-Dist: httpx (>=0.28.1,<0.29.0)
+Requires-Dist: langchain-core (>=1.4.0,<2.0.0)
+Requires-Dist: langgraph (>=1.2.0,<2.0.0)
+Requires-Dist: pydantic (>=2.13.4,<3.0.0)
+Requires-Dist: sparqlwrapper (>=2.0.0,<3.0.0)
+Requires-Dist: typing-extensions (>=4.15.0,<5.0.0)
+Project-URL: Homepage, https://github.com/johanmed/gsurgeon
+Description-Content-Type: text/markdown
+
+# GSurgeon: the genomic surgeon
+
+## What is GSurgeon?
+
+**GSurgeon** is an AI tool to dissect biology of model organisms through genomic information. It leverages LLM capabilities to send dynamic requests in natural language to genomic databases and extract any biological information.
+
+## What questions can you ask GSurgeon?
+
+**GSurgeon** has been tested on questions related to model organisms involving markers, genes and traits.
+
+As such it has good performance on queries such as:
+
+- Which genes on chromosome 1 of the mouse genome are related to inflammation and diabetes at the same time?
+- List traits measured in GeneNetwork that are related to diabetes.
+
+Other queries in the realm of biology and genomics are also possible.
+
+## How to install and run GSurgeon?
+
+#### 1. Get the source code
+
+You can clone this repository.
+
+```bash
+git clone https://github.com/johanmed/gsurgeon.git
+```
+
+#### 2. Set tool parameters
+
+**GSurgeon** expects a number of parameters to be defined for the surgery:
+
+- N_ITERATIONS: number of operations
+- MODEL_NAME: DSPy model identifier
+- API_KEY: provider key
+- EMAIL: email address for NCBI authentication
+
+We recommend creating them in an environment file. For more details, see file `env_example`.
+
+#### 3. Add gsurgeon path to your search path
+
+```bash
+export PATH="$PATH:/path/to/project"
+```
+
+Replace the path above by yours. You can also add it to your file `~/.bashrc`.
+
+#### 4. Run your query
+
+```bash
+gsurgeon --env-file env_example "Which genes on chromosome 1 of the mouse genome are related to inflammation and diabetes at the same time?"
+```
+
+Replace the query above by yours.
+
+## Why use GSurgeon?
+
+#### 1. Access to up-to-date biological information
+
+Accessing genomic information is a pain. It requires knowledge of right databases to query but also skills to dig deep and find relevant information. **GSurgeon** makes the process easier for the community by providing a simpler, yet powerful interface to interact in real time with biological databases.
+
+In the research ecosystem, this can be used for a variety of applications:
+
+- literature review
+- cross-checking of research findings against current knowledge
+- hypothesis exploration
+- biological link discovery
+- advanced bioinformatic analyses
+
+#### 2. Prevent hallucination, trust a bit more language models used in biology
+
+Despite advances in language AI, hallucination remains a serious concern in biological research. **GSurgeon** offers a scalable solution by grounding generation in true information from biological databases. Current databases supported include:
+- [GeneNetwork](https://genenetwork.org/): database service to explore biology of model organisms with bioinformatic tools
+- [NCBI](https://www.ncbi.nlm.nih.gov/): database service for access and analysis of biological information
+
+#### 3. Empower your LLM to handle with surgical precision the hard work for you with no limits
+
+**GSurgeon** exploits reasoning capabilities of LLM to orchestrate the search of biological information. Using its knowledge of biological databases, it finds dynamically the best approach of answering or completing the task you have in mind.
+
+The execution logic is abstracted to give full control to the agents. No need for extra coding!
+
+The tool footprint is lightweight. Most of the computational resources required to run the system are handled by the provider. No need to have monstruous specs to get started!
+
+**GSurgeon** can be executed on the command-line on any model, provided sufficient training.
+

gsurgeon-1.0.0/README.md

@@ -0,0 +1,83 @@
+# GSurgeon: the genomic surgeon
+
+## What is GSurgeon?
+
+**GSurgeon** is an AI tool to dissect biology of model organisms through genomic information. It leverages LLM capabilities to send dynamic requests in natural language to genomic databases and extract any biological information.
+
+## What questions can you ask GSurgeon?
+
+**GSurgeon** has been tested on questions related to model organisms involving markers, genes and traits.
+
+As such it has good performance on queries such as:
+
+- Which genes on chromosome 1 of the mouse genome are related to inflammation and diabetes at the same time?
+- List traits measured in GeneNetwork that are related to diabetes.
+
+Other queries in the realm of biology and genomics are also possible.
+
+## How to install and run GSurgeon?
+
+#### 1. Get the source code
+
+You can clone this repository.
+
+```bash
+git clone https://github.com/johanmed/gsurgeon.git
+```
+
+#### 2. Set tool parameters
+
+**GSurgeon** expects a number of parameters to be defined for the surgery:
+
+- N_ITERATIONS: number of operations
+- MODEL_NAME: DSPy model identifier
+- API_KEY: provider key
+- EMAIL: email address for NCBI authentication
+
+We recommend creating them in an environment file. For more details, see file `env_example`.
+
+#### 3. Add gsurgeon path to your search path
+
+```bash
+export PATH="$PATH:/path/to/project"
+```
+
+Replace the path above by yours. You can also add it to your file `~/.bashrc`.
+
+#### 4. Run your query
+
+```bash
+gsurgeon --env-file env_example "Which genes on chromosome 1 of the mouse genome are related to inflammation and diabetes at the same time?"
+```
+
+Replace the query above by yours.
+
+## Why use GSurgeon?
+
+#### 1. Access to up-to-date biological information
+
+Accessing genomic information is a pain. It requires knowledge of right databases to query but also skills to dig deep and find relevant information. **GSurgeon** makes the process easier for the community by providing a simpler, yet powerful interface to interact in real time with biological databases.
+
+In the research ecosystem, this can be used for a variety of applications:
+
+- literature review
+- cross-checking of research findings against current knowledge
+- hypothesis exploration
+- biological link discovery
+- advanced bioinformatic analyses
+
+#### 2. Prevent hallucination, trust a bit more language models used in biology
+
+Despite advances in language AI, hallucination remains a serious concern in biological research. **GSurgeon** offers a scalable solution by grounding generation in true information from biological databases. Current databases supported include:
+- [GeneNetwork](https://genenetwork.org/): database service to explore biology of model organisms with bioinformatic tools
+- [NCBI](https://www.ncbi.nlm.nih.gov/): database service for access and analysis of biological information
+
+#### 3. Empower your LLM to handle with surgical precision the hard work for you with no limits
+
+**GSurgeon** exploits reasoning capabilities of LLM to orchestrate the search of biological information. Using its knowledge of biological databases, it finds dynamically the best approach of answering or completing the task you have in mind.
+
+The execution logic is abstracted to give full control to the agents. No need for extra coding!
+
+The tool footprint is lightweight. Most of the computational resources required to run the system are handled by the provider. No need to have monstruous specs to get started!
+
+**GSurgeon** can be executed on the command-line on any model, provided sufficient training.

gsurgeon-1.0.0/pyproject.toml

@@ -0,0 +1,29 @@
+[project]
+name = "gsurgeon"
+version = "1.0.0"
+description = "A tool to dissect biology of model organisms using genomic information"
+authors = [
+    {name = "Johannes Medagbe",email = "johanmedagbe@gmail.com"}
+]
+license = "MIT"
+readme = "README.md"
+requires-python = ">=3.11,<3.15"
+dependencies = [
+    "langchain-core (>=1.4.0,<2.0.0)",
+    "langgraph (>=1.2.0,<2.0.0)",
+    "bio (>=1.8.3,<2.0.0)",
+    "httpx (>=0.28.1,<0.29.0)",
+    "pydantic (>=2.13.4,<3.0.0)",
+    "typing-extensions (>=4.15.0,<5.0.0)",
+    "dotenv (>=0.9.9,<0.10.0)",
+    "dspy (>=3.2.1,<4.0.0)",
+    "botocore (>=1.43.11,<2.0.0)",
+    "sparqlwrapper (>=2.0.0,<3.0.0)",
+]
+
+[project.urls]
+Homepage = "https://github.com/johanmed/gsurgeon"
+
+[build-system]
+requires = ["poetry-core>=2.0.0,<3.0.0"]
+build-backend = "poetry.core.masonry.api"

gsurgeon-1.0.0/src/gsurgeon/metrics/bootstrapping.py

@@ -0,0 +1,24 @@
+"""Module with bootstrapping logic"""
+
+from collections import Counter
+
+
+def bootstrap(element_ranks: dict[str, list[int]]) -> dict[tuple, float]:
+    """
+    Compute bootstrap rank for each element
+    Args:
+        element_ranks: dictionary of elements and ranks (list)
+    Output:
+        Dictionary of elements and bootstrap ranks (single value)
+    """
+    bootstrap_ranks = {}
+    for element, ranks in element_ranks.items():
+        sorted_ranks = sorted(ranks)
+        total = len(sorted_ranks)
+        top = sorted(
+            Counter(sorted_ranks).items(), key=lambda item: item[1], reverse=True
+        )[0]
+        top_rank, frequency = top
+        bootstrap_ranks[(element, top_rank)] = frequency / total
+    bootstrap_ranks = dict(sorted(bootstrap_ranks.items(), key=lambda item: item[0][1]))
+    return bootstrap_ranks

gsurgeon-1.0.0/src/gsurgeon/metrics/finemapping.py

@@ -0,0 +1,27 @@
+"""Module with constructs to compute metrics for finemapping"""
+
+import dspy
+from gsurgeon.metrics.bootstrapping import bootstrap
+
+
+class ExtractGene(dspy.Signature):
+    """
+    Construct a list of genes consistent across answers.
+    Scan answers and extract rank for each gene in the previous list.
+    Build a dictionary with gene as key and list of ranks as value.
+    """
+
+    query: str = dspy.InputField(desc="Finemapping query")
+    answers: list[str] = dspy.InputField(
+        desc="List of answers to the query, each reporting ranked list of genes"
+    )
+    gene_ranks: dict[str, list[int]] = dspy.OutputField(
+        desc="Dictionary of genes and assigned ranks. Example: {'gene A': [1, 1, 2], 'gene B': [2, 3, 2]}"
+    )
+
+
+def bootstrap_rank(query: str, answers: list[str]) -> dict[tuple, float]:
+    """Compute a bootstrap rank for each gene"""
+    extract = dspy.Predict(ExtractGene)
+    gene_ranks = extract(query=query, answers=answers).get("gene_ranks")
+    return bootstrap(gene_ranks)

gsurgeon-1.0.0/src/gsurgeon/metrics/network.py

@@ -0,0 +1,28 @@
+"""Module with constructs to compute metrics for network analysis"""
+
+import dspy
+from gsurgeon.metrics.bootstrapping import bootstrap
+
+
+class ExtractEdge(dspy.Signature):
+    """
+    Construct a list of connections consistent across answers.
+    Scan answers and extract rank for each connection in the previous list.
+    Build a dictionary with edge as key and list of ranks as value.
+    Example: {"gene A -> gene C": [1, 1, 2], "gene B -> gene D": [2, 3, 2]}
+    """
+
+    query: str = dspy.InputField(desc="Network analysis query")
+    answers: list[str] = dspy.InputField(
+        desc="List of answers to the query, each reporting a ranked of list of connections between genes"
+    )
+    edge_ranks: dict[str, list[int]] = dspy.OutputField(
+        desc="Dictionary of connections and assigned ranks assigned"
+    )
+
+
+def bootstrap_rank(query: str, answers: list[str]) -> dict[tuple, float]:
+    """Compute a bootstrap rank for each edge"""
+    extract = dspy.Predict(ExtractEdge)
+    edge_ranks = extract(query=query, answers=answers).get("edge_ranks")
+    return bootstrap(edge_ranks)

gsurgeon-1.0.0/src/gsurgeon/operations/standard.py

@@ -0,0 +1,80 @@
+"""Module with standard operation constructs for GSurgeon"""
+
+import asyncio
+
+import dspy
+from gsurgeon.procedures.standard import Reproduce
+from gsurgeon.surgeon.agent import GSurgeon
+
+
+async def operate(query: str, n_steps: int = 5) -> str:
+    """Execute operation or analysis with GSurgeon"""
+    surgeon = GSurgeon(max_steps=n_steps)
+    return await surgeon.handle(query)
+
+
+async def reoperate(query: str, n_steps: int = 5, n_iterations: int = 5) -> str:
+    """
+    Execute operation or analysis a given number of times for reproducibility
+    Args:
+        query: inquiry
+        n_steps: max number of steps allowed during operation
+        n_iterations: number of operation repetitions
+    Output:
+        Consensus resulting from different runs
+    """
+    print(f"Bootstrapping operation {n_iterations} times for query...")
+    results = await asyncio.gather(
+        *[operate(query, n_steps) for n in range(n_iterations)]
+    )
+    reproduce = dspy.Predict(Reproduce)
+    print("Bootstrapped run completed")
+    return reproduce(query=query, results=results).get("consensus")
+
+
+async def serialize(
+    queries: list, n_steps: int = 5, n_iterations: int = 5
+) -> dict:
+    """
+    Execute operation a given number of times for a set/series of queries
+    Args:
+        queries: list of queries to investigate
+        n_steps: max number of steps allowed during operation
+        n_iterations: number of operation repetitions
+    Output:
+        Dictionary of query and responses
+    """
+
+    base = dspy.ChainOfThought("query -> answer: str")
+    base_results = [base(query=query).get("answer") for query in queries]
+
+    surgeon_results = await asyncio.gather(
+        *[reoperate(query, n_steps, n_iterations) for query in queries]
+    )
+
+    collection = {}
+    for query, base_result, surgeon_result in zip(
+        queries, base_results, surgeon_results
+    ):
+        collection[f"Query was '{query}'"] = [
+            f"Base response was '{base_result}'",
+            f"Surgeon response was: '{surgeon_result}'",
+        ]
+
+    return collection
+
+
+async def meta_analyze(
+    query: str, n_steps: int = 5, n_iterations: int = 5, n_bootstraps: int = 10
+) -> list[str]:
+    """
+    Perform a meta-analysis of an operation or genomic task with n samples
+    Args:
+        query: genomic task
+        n_steps: max number of steps allowed during operation
+        n_iterations: number of operation repetitions
+        n_bootstraps: number of repetition sampling for statistical support
+    """
+    return await asyncio.gather(
+        *[reoperate(query, n_steps, n_iterations) for n in range(n_bootstraps)]
+    )

gsurgeon-1.0.0/src/gsurgeon/procedures/react.py

@@ -0,0 +1,81 @@
+"""Module with ReAct procedures (dspy programs) for GSurgeon"""
+
+import dspy
+from gsurgeon.tools.general import checker, reformulator, splitter
+from gsurgeon.tools.gn import make_sparql_tool
+from gsurgeon.tools.ncbi import ncbi_searcher, record_fetcher, record_synthesizer
+from langchain_core.messages import BaseMessage
+
+
+class ReactSig(dspy.Signature):
+    query: list[BaseMessage] = dspy.InputField()
+    reasoning: str = dspy.OutputField(desc="Concise explanation of solution")
+    solution: str = dspy.OutputField(
+        desc="Final answer to the query in 2000 words max making use of all relevant information in accumulated context."
+    )
+
+
+class Research(dspy.Module):
+    """
+    Address a query or plan to completion using GeneNetwork resources only.
+    For efficiency, only call a tool when it is strictly necessary in completing the next task.
+    Use splitter when input query is too complex to be handled in a single step.
+    Harness the reformulator to clarify a request when it seems ambiguous.
+    To get a specific information, call the fetcher. It has access to data and can extract any information.
+    Once an information is extracted, check its relevance with the checker before proceeding.
+    For reproducibility, seed your reasoning on the following master thoughts:
+        1. GeneNetwork has the data requested by the user and it can be obtained with targeted SPARQL queries.
+        2. Reasoning does not need to be complicated. Prefer the simplest approach.
+    """
+
+    def __init__(self):
+        super().__init__()
+        fetcher = make_sparql_tool("https://sparql.genenetwork.org/sparql")
+        self.tools = [splitter, checker, reformulator, fetcher]
+
+        self.react = dspy.ReAct(
+            signature=ReactSig,
+            tools=self.tools,
+            max_iters=10,  # maximum number of steps for reasoning and tool calling
+        )
+
+    def forward(self, query: list[BaseMessage]):
+        return self.react(query=query)
+
+
+class Consult(dspy.Module):
+    """
+    Address a query or plan to completion using NCBI resources only.
+    For effficiency, only call a tool when it is strictly necessary in completing the next task.
+    Use splitter when input query is too complex to be handled in a single step.
+    Harness the reformulator to clarify a request when it seems ambiguous.
+    Extract answers from NCBI by performing first a search with ncbi_searcher. The search terms must be as specific as possible.
+    When search results contain records, fetch information with record_fetcher.
+    For records with a lot of data specifically, take some time to synthesize informations.
+    Check relevance of generated information with the checker before proceeding.
+    Every information you extracted regarding genes and/or functions must be verified with another search using proper terms with ncbi_searcher.
+    You must ascertain that all informations in the final answer are true at all cost.
+    For reproducibility, seed your reasoning on the following master thoughts:
+        1. NCBI has the data requested by the user and it can be obtained by using a combination of terms with AND or OR keywords.
+        2. Reasoning does not need to be complicated. Prefer the simplest approach.
+    """
+
+    def __init__(self):
+        super().__init__()
+        self.tools = [
+            splitter,
+            checker,
+            reformulator,
+            ncbi_searcher,
+            record_fetcher,
+            record_synthesizer,
+        ]
+
+        self.react = dspy.ReAct(
+            signature=ReactSig,
+            tools=self.tools,
+            max_iters=20,
+        )
+
+    def forward(self, query: list[BaseMessage]):
+        return self.react(query=query)

gsurgeon-1.0.0/src/gsurgeon/procedures/standard.py

@@ -0,0 +1,75 @@
+"""Module with standard procedures (dspy modules) for GSurgeon"""
+
+import dspy
+from langchain_core.messages import BaseMessage
+from typing_extensions import Literal
+
+
+class Plan(dspy.Signature):
+    """
+    Generate plan to solve query in background.
+    For reproducibility, seed the plan generation on the following master thought:
+    The query submitted by the user can be addressed in less than 5 straightforward and targeted steps. No need to complicate things.
+    """
+
+    background: list[BaseMessage] = dspy.InputField()
+    answer: str = dspy.OutputField(desc="The plan to solve the task")
+    reasoning: str = dspy.OutputField(
+        desc="Concise explanation of the output in 50 words"
+    )
+
+
+class Tune(dspy.Signature):
+    """Make recommendations to improve user satisfaction to answer generated so far to query"""
+
+    background: list[BaseMessage] = dspy.InputField()
+    answer: str = dspy.OutputField(desc="The new questions")
+    reasoning: str = dspy.OutputField(
+        desc="Concise explanation of the output in 50 words"
+    )
+
+
+class Supervise(dspy.Signature):
+    """
+    Decide the next action the system should take.
+    To select the next step, you must take into account the query and the curent context.
+    If the query is not related to GeneNetwork traits, do not call gn_researcher. ncbi_expert should be the main actor.
+    Similarly, do not call the ncbi_expert if the query is GeneNetwork specific.
+    When the query is related to genes, finemapping or network analysis, you must call the ncbi_expert and not the gn_researcher.
+    Call the reflector only to improve generation from gn_researcher and ncbi_expert.
+    Act on suggestions proposed by reflector using the most appropriate actor between gn_researcher and ncbi_expert depending on the query.
+    End execution if there is nothing else to do.
+    """
+
+    background: list[BaseMessage] = dspy.InputField()
+    next_decision: Literal["gn_researcher", "ncbi_expert", "reflector", "end"] = (
+        dspy.OutputField(desc="The next step to take based on instructions")
+    )
+    reasoning: str = dspy.OutputField(
+        desc="Concise explanation of the decision in 50 words"
+    )
+
+
+class Finalize(dspy.Signature):
+    """Build the final synthesis to send back to the user in less than 500 words"""
+
+    messages: list[BaseMessage] = dspy.InputField()
+    feedback: str = dspy.OutputField(
+        desc="Detailed and comprehensive final feedback combining AI outputs in the list of messages and linking them when necessary"
+    )
+
+
+class Reproduce(dspy.Signature):
+    """
+    Extract answers that are consistent across results.
+    Do not include information that is missing in some results for reproducibility.
+    Synthesize a coherent and detailed solution to the query using answer consensus.
+    """
+
+    query: str = dspy.InputField()
+    results: list = dspy.InputField(
+        desc="List of results generated to the same query by the system"
+    )
+    consensus: str = dspy.OutputField(
+        desc="Final output built from consistent answers across results"
+    )

gsurgeon-1.0.0/src/gsurgeon/surgeon/agent.py

@@ -0,0 +1,176 @@
+"""GSurgeon: Multi-agent system to dissect genomic information"""
+
+import asyncio
+import json
+import logging
+import os
+import time
+from dataclasses import dataclass, field
+from typing import Any
+
+import dspy
+from gsurgeon.procedures.react import Consult, Research
+from gsurgeon.procedures.standard import Finalize, Plan, Supervise, Tune
+from gsurgeon.surgeon.prompts import (
+    expert_prompt,
+    planner_prompt,
+    reflector_prompt,
+    researcher_prompt,
+    supervisor_prompt1,
+    supervisor_prompt2,
+)
+from langchain_core.messages import AIMessage, BaseMessage, HumanMessage, SystemMessage
+from langgraph.graph import END, START, StateGraph
+from langgraph.graph.message import add_messages
+from langgraph.prebuilt import ToolNode, tools_condition
+from pydantic import BaseModel
+from typing_extensions import Annotated, Literal
+
+
+class AgentState(BaseModel):
+    """
+    Represent agent state
+    Avail 02 attributes to allow communication between agents
+    """
+
+    messages: Annotated[list[BaseMessage], add_messages]
+    next_decision: Literal[
+        "gn_researcher", "planner", "reflector", "ncbi_expert", "end"
+    ]
+
+
+@dataclass
+class GSurgeon:
+    """
+    Represent Search Agent
+    Input:
+        max_steps: maximum number of steps allowed
+    Operations:
+        Initialization of multi-agent graph
+        Run of query through system
+    """
+
+    max_steps: int = 5
+    _graph: Any = field(init=False)
+
+    def __post_init__(self):
+        self._graph = self._build_graph()
+
+    async def _researcher(self, state: AgentState) -> dict:
+        """Answer query with GeneNetwork information"""
+        print("Calling the researcher...")
+        if len(state.messages) < 3:  # handle first call to researcher
+            input_text = state.messages[0]  # use original query
+        else:
+            input_text = state.messages[-1]  # use reflection insights
+        input_text = [researcher_prompt, input_text.content]
+        research = Research()
+        result = await asyncio.to_thread(research, query=input_text)
+        print("Researcher performed analysis")
+        return {
+            "messages": [AIMessage(result.get("solution"))],
+        }
+
+    async def _expert(self, state: AgentState) -> dict:
+        """Answer query with NCBI information"""
+        print("Calling the expert...")
+        if len(state.messages) < 4:  # handle first call to expert
+            input_text = state.messages[1] + state.messages[0]  # use plan and query
+        else:
+            input_text = state.messages[-2]  # use reflection insights
+        input_text = [expert_prompt, input_text]
+        consult = Consult()
+        result = await asyncio.to_thread(consult, query=input_text)
+        print("Expert produced answers")
+        return {
+            "messages": [AIMessage(result.get("solution"))],
+        }
+
+    async def _planner(self, state: AgentState) -> dict:
+        """Generate a plan to solve query"""
+        print("Generating a plan to solve the problem...")
+        plan = dspy.Predict(Plan)
+        input_text = [planner_prompt] + state.messages
+        result = await asyncio.to_thread(plan, background=input_text)
+        print("Plan acquired")
+        return {
+            "messages": [AIMessage(result.get("answer"))],
+        }
+
+    async def _reflector(self, state: AgentState) -> dict:
+        """Propose improvements to answer"""
+        print("Calling the reflector...")
+        tune = dspy.Predict(Tune)
+        trans_map = {AIMessage: HumanMessage, HumanMessage: AIMessage}
+        translated_messages = [reflector_prompt, state.messages[0]] + [
+            trans_map[msg.__class__](content=msg.content) for msg in state.messages[1:]
+        ]
+        result = await asyncio.to_thread(tune, background=translated_messages)
+        print("Reflector made suggestions")
+        return {
+            "messages": [
+                HumanMessage(
+                    f"Progress has been made. Use now all the resources to addess this new suggestion: {result.get('answer')}"
+                )
+            ],
+        }
+
+    async def _supervisor(self, state: AgentState) -> dict:
+        """Orchestrate agentic system"""
+        print("Getting guidance from the supervisor...")
+        supervise = dspy.Predict(Supervise)
+        messages = [
+            supervisor_prompt1,
+            *state.messages,
+            supervisor_prompt2,
+        ]
+        if len(messages) > self.max_steps:
+            return {"next_decision": "end"}
+        result = await asyncio.to_thread(supervise, background=messages)
+        print("Supervisor selected the next worker")
+        return {
+            "next_decision": result.get("next_decision"),
+        }
+
+    def _build_graph(self) -> Any:
+        graph_builder = StateGraph(AgentState)
+        graph_builder.add_node("gn_researcher", self._researcher)
+        graph_builder.add_node("planner", self._planner)
+        graph_builder.add_node("reflector", self._reflector)
+        graph_builder.add_node("supervisor", self._supervisor)
+        graph_builder.add_node("ncbi_expert", self._expert)
+        graph_builder.add_edge(START, "planner")
+        graph_builder.add_edge("planner", "supervisor")
+        graph_builder.add_edge("gn_researcher", "supervisor")
+        graph_builder.add_edge("ncbi_expert", "supervisor")
+        graph_builder.add_edge("reflector", "supervisor")
+        graph_builder.add_conditional_edges(
+            "supervisor",
+            lambda state: state.next_decision,
+            {
+                "reflector": "reflector",
+                "gn_researcher": "gn_researcher",
+                "ncbi_expert": "ncbi_expert",
+                "end": END,
+            },
+        )
+        return graph_builder.compile()
+
+    async def _run_graph(self, query: str) -> Any:
+        initial_state = {
+            "messages": [HumanMessage(query)],
+            "next_decision": "planner",  # always plan first
+        }
+        return await self._graph.ainvoke(initial_state)
+
+    async def handle(self, query: str) -> str:
+        """Run query through the system"""
+        print("Starting operation...")
+        result = await self._run_graph(query)
+        unprocessed_result = result.get("messages")[2].content
+        finalize = dspy.Predict(Finalize)
+        processed_result = await asyncio.to_thread(
+            lambda: finalize(messages=result.get("messages")).get("feedback")
+        )
+        print("Operation complete")
+        return f"Raw feedback: {unprocessed_result}\nProcessed feedback: {processed_result}"

gsurgeon-1.0.0/src/gsurgeon/surgeon/prompts.py

@@ -0,0 +1,41 @@
+"""Module with system prompts"""
+
+from langchain_core.messages import SystemMessage
+
+researcher_prompt = SystemMessage(
+    """
+    You are a researcher who have access to a variety of tools built on GeneNetwork RDF knowledge base. You can investigate any question users might have. Leverage the tools at your disposal to efficiently extract the answer to the query. While carrying out your job, you must follow the plan proposed by the planner. Do not leave anything out. Verify the final answer before reporting it.
+    """
+)
+
+supervisor_prompt1 = SystemMessage(
+    """
+    You are a supervisor for a genomic analysis. You tasked with managing a conversation between the following workers: [gn_researcher, ncbi_expert, reflector]. Given the following user request, respond with the worker to act next. Each worker will perform a task and respond with its results.
+    Follow the plan made by the planner to decide the next node. Do not finish before completing the plan. When finished, respond with end. Make sure to reflect right before finishing.
+    """
+)
+
+supervisor_prompt2 = SystemMessage(
+    """
+    Given the conversation above, who should act next? Or should we end? Select one of: [gn_researcher, ncbi_expert, reflector, end]. You must help in making progress towards executing and completing the plan. Look at the messages. Do not repeat the same step consecutively. For example, do not call the expert two times consecutively.
+    """
+)
+
+planner_prompt = SystemMessage(
+    """
+    You are an experienced and powerful task planner for genomic analysis. Generate a list of clear and relevant steps to take to solve the query below.
+    """
+)
+
+reflector_prompt = SystemMessage(
+    """
+    You have been doing research for almost 50 years and have a very deep knowledge of biology, genomics and bioinformatics. You always have relevant follow questions. Improve the system answer by providing follow up questions.
+    """
+)
+
+expert_prompt = SystemMessage(
+    """
+    You are a powerful system that have access to specialized tools to fetch relevant information and help you achieve your task. With those tools, you can extract any information in biology and specifically in genetics and genomics. Regardless of the organism, you can find the information that is requested.
+    Follow and execute step-by-step the plan below to solve the query further below using your knowledge and the tools at your disposal. Make sure to return the final solution alongside with intermediary results. Be accurate and thorough. Always countercheck your results and their relevance to ensure satisfaction.
+    """
+)

gsurgeon-1.0.0/src/gsurgeon/tools/general.py

@@ -0,0 +1,95 @@
+"""Modules with general tools for GSurgeon"""
+
+import dspy
+
+
+class Split(dspy.Signature):
+    """Split query into multiple atomic subqueries easier to handle for better satisfaction"""
+
+    query: str = dspy.InputField()
+    answer: list[str] = dspy.OutputField(desc="The list of smaller tasks")
+
+
+def split_query(query: str) -> list[str]:
+    split = dspy.Predict(Split)
+    return split(query=query).get("answer")
+
+
+splitter = dspy.Tool(
+    name="splitter",
+    desc="Process a query by splitting into atomic subqueries for efficiency",
+    args={
+        "query": {
+            "type": "string",
+            "desc": "Query to process",
+        },
+    },
+    func=split_query,
+)
+
+
+class Check(dspy.Signature):
+    """Check if info is relevant to query"""
+
+    query: str = dspy.InputField()
+    info: str = dspy.InputField()
+    decision: str = dspy.OutputField(desc="Say 'yes' or 'no'")
+
+
+def check_relevance(query: str, info: str) -> str:
+    check = dspy.Predict(Check)
+    return check(query=query, info=info).get("decision")
+
+
+checker = dspy.Tool(
+    name="checker",
+    desc="Check if information previously extracted is relevant for the query",
+    args={
+        "query": {
+            "type": "string",
+            "desc": "Query to address",
+        },
+        "info": {
+            "type": "string",
+            "desc": "Information extracted in attempt to provide answer to query",
+        },
+    },
+    func=check_relevance,
+)
+
+
+class Rephrase(dspy.Signature):
+    """Reformulate query given target and context accumulated so far"""
+
+    query: str = dspy.InputField()
+    target: str = dspy.InputField()
+    background: str = dspy.InputField()
+    reformulation: str = dspy.OutputField(desc="Reformulated query")
+
+
+def rephrase_query(query: str, target: str, background: str) -> str:
+    rephrase = dspy.Predict(Rephrase)
+    return rephrase(query=query, target=target, background=background).get(
+        "reformulation"
+    )
+
+
+reformulator = dspy.Tool(
+    name="reformulator",
+    desc="Reformulate the query to be next processed in light of the context accumulated so far (background) and the target",
+    args={
+        "query": {
+            "type": "string",
+            "desc": "Query to be reformulated",
+        },
+        "target": {
+            "type": "string",
+            "desc": "Original query or target",
+        },
+        "background": {
+            "type": "string",
+            "desc": "Accumulated context in effort to achieve the target",
+        },
+    },
+    func=rephrase_query,
+)

gsurgeon-1.0.0/src/gsurgeon/tools/gn.py

@@ -0,0 +1,219 @@
+"""Module with GeneNetwork SPARQL tools for GSurgeon"""
+
+import asyncio
+import concurrent.futures
+from typing import Any
+
+import dspy
+import httpx
+from SPARQLWrapper import JSON, SPARQLWrapper
+
+
+class QueryTranslation(dspy.Signature):
+    """Compare object snapshot in schema hint to keywords in the original query to find best semantic matches.
+    Use matches to generate valid SPARQL SELECT queries that can retrieve relevant information for the query.
+    CRITICAL:
+    1. Every query MUST start with the PREFIX declarations. Only use declared prefixes.
+    2. Leverage as many schema hints as possible.
+
+    When querying SPARQL, prefer fast, efficient SPARQL SELECT queries
+    that avoid Virtuoso timeouts (504 errors).
+
+    PERFORMANCE RULES:
+    1. Always add `LIMIT` - start with `LIMIT 50`, increase only if needed. Never omit `LIMIT`.
+    2. Never use `SELECT *` - list only the variables you actually need.
+    3. Avoid expensive operations: no Cartesian products, no cross joins, no full graph scans.
+    4. Use specific FILTER patterns that leverage indexes:
+    - Prefer `STRSTARTS(?label, "prefix")` over `CONTAINS` or regex.
+    - Avoid `FILTER regex(...)` - it disables indexes.
+    - Use `FILTER(?value = "exact")` or `IN` with small lists.
+    5. Prefer property paths over multiple joins when traversing a chain.
+    6. Use VALUES blocks for small sets of constants instead of UNION or OPTIONAL.
+    7. Avoid ORDER BY on large result sets - if needed, combine with `LIMIT` and a narrow `WHERE` clause.
+    8. Never use nested subqueries unless absolutely necessary; flatten them.
+    9. Use `OPTIONAL` only for truly optional patterns – otherwise, use a simple triple pattern.
+    """
+
+    original_query: str = dspy.InputField(desc="User query")
+    schema_hint: str = dspy.InputField(desc="GeneNetwork schema from Virtuoso")
+    translated_queries: list[str] = dspy.OutputField(
+        desc="Top 10 valid SPARQL SELECT query with PREFIX declarations."
+    )
+
+
+def make_sparql_tool(sparql_uri: str) -> dspy.Tool:
+
+    def sparql_fetcher(query: str) -> Any:
+
+        def build_schema_hint(sparql_uri: str) -> str:
+            """Build a compact schema hint from the live Virtuoso endpoint."""
+            _PREFIX_MAP = {
+                "http://rdf.genenetwork.org/v1/term/": "gnt",
+                "http://rdf.genenetwork.org/v1/category/": "gnc",
+                "http://rdf.genenetwork.org/v1/id/": "gn",
+                "http://purl.org/dc/terms/": "dct",
+                "http://www.w3.org/ns/dcat#": "dcat",
+                "http://www.w3.org/2000/01/rdf-schema#": "rdfs",
+                "http://www.w3.org/2004/02/skos/core#": "skos",
+                "http://www.w3.org/1999/02/22-rdf-syntax-ns#": "rdf",
+                "http://www.w3.org/2002/07/owl#": "owl",
+                "http://purl.org/linked-data/cube#": "qb",
+                "http://purl.org/linked-data/sdmx/2009/measure#": "sdmx-measure",
+                "http://rdf-vocabulary.ddialliance.org/xkos#": "xkos",
+                "https://schema.org/": "schema",
+                "http://rdf.ncbi.nlm.nih.gov/pubmed/": "pubmed",
+                "http://xmlns.com/foaf/0.1/": "foaf",
+                "http://purl.org/spar/fabio/": "fabio",
+                "http://prismstandard.org/namespaces/basic/2.0/": "prism",
+            }
+
+            def uri_to_qname(uri: str) -> str:
+                """Convert a full URI to a prefixed name, or return the URI in angle brackets."""
+                for ns, prefix in sorted(_PREFIX_MAP.items(), key=lambda x: -len(x[0])):
+                    if uri.startswith(ns):
+                        return f"{prefix}:{uri[len(ns):]}"
+                return f"<{uri}>"
+
+            def fetch_schema(sparql_uri: str) -> tuple[set[str], set[str], set[str]]:
+                """Fetch literal and object properties from the live Virtuoso endpoint.
+                Return (literal_props, iri_props) where each is a set of full URIs.
+                """
+                sparql = SPARQLWrapper(sparql_uri)
+                sparql.setReturnFormat(JSON)
+
+                literal_query = """
+                SELECT DISTINCT ?p
+                WHERE { ?s ?p ?o . FILTER isLiteral(?o) }
+                """
+                sparql.setQuery(literal_query)
+                lit_result = sparql.queryAndConvert()
+                literal_props = {
+                    b["p"]["value"]
+                    for b in lit_result.get("results", {}).get("bindings", [])
+                    if b.get("p")
+                }
+
+                iri_query = """
+                SELECT DISTINCT ?p
+                WHERE { ?s ?p ?o . FILTER isIRI(?o) }
+                """
+                sparql.setQuery(iri_query)
+                iri_result = sparql.queryAndConvert()
+                iri_props = {
+                    b["p"]["value"]
+                    for b in iri_result.get("results", {}).get("bindings", [])
+                    if b.get("p")
+                }
+
+                return literal_props, iri_props
+
+            literal_props, iri_props = fetch_schema(sparql_uri)
+            return f"""=== GENENETWORK SCHEMA (from Virtuoso) ===
+            PREFIX dcat: <http://www.w3.org/ns/dcat#>
+            PREFIX gn: <http://rdf.genenetwork.org/v1/id/>
+            PREFIX dct: <http://purl.org/dc/terms/>
+            PREFIX gnc: <http://rdf.genenetwork.org/v1/category/>
+            PREFIX gnt: <http://rdf.genenetwork.org/v1/term/>
+            PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
+            PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
+            PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+            PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+
+            LITERAL PROPERTIES (object is a string/number/date):
+            {" ,".join([uri_to_qname(uri) for uri in literal_props])}
+
+            IRI PROPERTIES (object is a URI / another resource):
+            {" ,".join([uri_to_qname(uri) for uri in iri_props])}
+
+            SPECIAL HINTS FOR TRIPLE GENERATION:
+            1. To check if a trait is mapped, use: `?trait a gnt:mappedTrait .`
+            2. To get trait id, use: `?trait gnt:traitId ?trait_id .`
+            3. To fetch trait description, use: `?trait dct:description ?trait_description .`
+            4. To extract lod score at a specific locus, use: `?trait gnt:locus ?locus; gnt:lodScore ?lod_score .`
+            5. To fetch information related to QTL for a trait, use:
+            `?trait gnt:qtlChr ?chromosome; gnt:qtlStart ?start; gnt:qtlStop ?stop; gnt:qtlLOD ?lod_score .`
+
+            CRITICAL RULES:
+            1. Only use properties listed above. Do NOT invent new ones.
+            2. Literal properties give strings/numbers — use FILTER, not ?o a ...
+            3. Object properties link to other resources — you can chain ?o a <Class>.
+            4. Do NOT use taxon: for species. Use gn:Mus_musculus, gn:Rattus_norvegicus, gn:Homo_sapiens, etc.
+            5. gnt:has_trait_page gives the URL directly. Never build trait URLs manually.
+            """
+
+        schema_hint = build_schema_hint(sparql_uri)
+        translate_sparql = dspy.Predict(QueryTranslation)
+        sparql_queries = translate_sparql(
+            original_query=query, schema_hint=schema_hint
+        ).get("translated_queries")
+
+        async def run_sparql(
+            sparql_uri: str,
+            query: str,
+            max_retries: int = 3,
+            base_delay: float = 2,
+        ) -> dict:
+            """Execute a single SPARQL query with retry + exponential jitter via httpx."""
+            client = httpx.AsyncClient(timeout=5000)
+            for attempt in range(max_retries):
+                try:
+                    resp = await client.post(
+                        sparql_uri,
+                        data={"query": query},
+                        headers={"Accept": "application/sparql-results+json"},
+                    )
+                    resp.raise_for_status()
+                    return resp.json()
+                except httpx.HTTPStatusError as e:
+                    if (
+                        e.response.status_code in (504, 503, 502)
+                        and attempt < max_retries - 1
+                    ):
+                        await asyncio.sleep(
+                            base_delay * (2**attempt) + random.uniform(0, 1)
+                        )
+                        continue
+                    raise
+            return {}
+
+        async def sparql_fetch(
+            sparql_queries: list[str],
+            sparql_uri: str,
+            max_retries: int = 3,
+            base_delay: float = 0.5,
+        ) -> str:
+            """Execute *sparql_queries* concurrently against *sparql_uri*."""
+            if not sparql_queries:
+                return "No SPARQL queries to run."
+
+            async def _fetch_one(query: str, idx: int) -> str:
+                try:
+                    result = await run_sparql(
+                        sparql_uri, query, max_retries, base_delay
+                    )
+                    bindings = result.get("results", {}).get("bindings", [])
+                    return f"Query {idx} succeeded ({len(bindings)} rows): {bindings}"
+                except Exception as e:
+                    return f"Query {idx} failed: {e}\nQuery was:\n{query}"
+
+            tasks = [_fetch_one(q, i) for i, q in enumerate(sparql_queries)]
+            results = await asyncio.gather(*tasks)
+            return "\n\n".join(results)
+
+        with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
+            future = executor.submit(
+                asyncio.run, sparql_fetch(sparql_queries, sparql_uri)
+            )
+            return future.result()
+
+    return dspy.Tool(
+        name="sparql_fetcher",
+        desc="Fetch RDF data around GeneNetwork data through SPARQL",
+        args={
+            "query": {
+                "type": "string",
+                "desc": "SPARQL query to run to fetch relevant data",
+            },
+        },
+        func=sparql_fetcher,
+    )

gsurgeon-1.0.0/src/gsurgeon/tools/ncbi.py

@@ -0,0 +1,81 @@
+"""Modules with NCBI tools for GSurgeon"""
+
+import json
+
+import dspy
+from Bio.Entrez import efetch, esearch, esummary, read
+
+
+def search_ncbi(database: str, term: str, max_results: int = 10) -> str:
+    handle = esearch(db=database, term=term, retmax=max_results)
+    records = read(handle)
+    handle.close()
+    # Order records for determinism
+    if isinstance(records, dict) and "IdList" in records:
+        records["IdList"] = sorted(records["IdList"])
+    return json.dumps(records, sort_keys=True)
+
+
+ncbi_searcher = dspy.Tool(
+    name="ncbi_searcher",
+    desc="Search an NCBI database (e.g., nucleotide, protein, pubmed) for a term",
+    args={
+        "database": {
+            "type": "string",
+            "desc": "Database name like 'nucleotide' or 'pubmed'",
+        },
+        "term": {"type": "string", "desc": "Search term or query"},
+        "max_results": {
+            "type": "integer",
+            "desc": "Max results (default 2000)",
+            "default": 2000,
+        },
+    },
+    func=search_ncbi,
+)
+
+
+def fetch_record(database: str, record_id: str, rettype: str) -> str:
+    handle = efetch(db=database, id=record_id, rettype=rettype, retmode="text")
+    result = handle.readline().strip()
+    handle.close()
+    return result
+
+
+record_fetcher = dspy.Tool(
+    name="record_fetcher",
+    desc="Fetch a record from an NCBI database (e.g., nucleotide, protein, pubmed)",
+    args={
+        "database": {
+            "type": "string",
+            "desc": "Database name like 'nucleotide' or 'pubmed'",
+        },
+        "record_id": {"type": "string", "desc": "Identifier of record"},
+        "rettype": {"type": "string", "desc": "Return type compatible with database"},
+    },
+    func=fetch_record,
+)
+
+
+def summarize_record(database: str, record_id: str) -> str:
+    handle = esummary(db=database, id=record_id)
+    result = read(handle)
+    handle.close()
+    # If a list of summaries, sort by Id for determinism
+    if isinstance(result, list):
+        result = sorted(result, key=lambda x: x.get("Id", ""))
+    return json.dumps(result, sort_keys=True)
+
+
+record_synthesizer = dspy.Tool(
+    name="record_synthesiser",
+    desc="Get summary on a record from an NCBI database (e.g., nucleotide, protein, pubmed)",
+    args={
+        "database": {
+            "type": "string",
+            "desc": "Database name like 'nucleotide' or 'pubmed'",
+        },
+        "record_id": {"type": "string", "desc": "Identifier of record"},
+    },
+    func=summarize_record,
+)