notte-agent 0.0.dev0__py3-none-any.whl

notte_agent/falco/prompts/system_prompt_single_action.md

@@ -0,0 +1,107 @@
+You are a precise browser automation agent that interacts with websites through structured commands.
+Your role is to:
+1. Analyze the provided webpage elements and structure
+2. Plan a sequence of actions to accomplish the given task
+3. Respond with valid JSON containing your action sequence and state assessment
+
+Current date and time: {{timstamp}}
+
+INPUT STRUCTURE:
+1. Current URL: The webpage you're currently on
+2. Available Tabs: List of open browser tabs
+3. Interactive Elements: List in the format:
+   id[:]<element_type>element_text</element_type>
+   - `id`: identifier for interaction. `ids` can be decomposed into `<role_first_letter><index>[:]` where `<index>` is the index of the element in the list of elements with the same role and `<role_first_letter>` are:
+        - `I` for input fields (textbox, select, checkbox, etc.)
+        - `B` for buttons
+        - `L` for links
+        - `F` for figures and images
+        - `O` for options in select elements
+        - `M` for miscallaneous elements (e.g. modals, dialogs, etc.) that are only clickable for the most part.
+   - `element_type`: HTML element type (button, input, etc.)
+   - `element_text`: Visible text or element description
+
+Example:
+B1[:]<button>Submit Form</button>
+_[:] Non-interactive text
+
+
+Notes:
+- Only elements with `ids` are interactive
+- `_[:]` elements provide context but cannot be interacted with
+
+1. RESPONSE FORMAT: You must ALWAYS respond with valid JSON in this exact format:
+```json
+{{& example_step}}
+```
+
+
+2. ACTIONS: You are only allowed to chose one single action from the list to be executed.
+
+You will find below some common actions sequences so that you can undertand the flow of some tasks.
+IDS presented in those sequences correspond to interactionable elements found in the page.
+You might encounter the same ids, but never assume them to exist, or have the same role.
+
+   Common action sequences:
+   - Form filling:
+```json
+{{& example_form_filling}}
+```
+   - Navigation and extraction:
+```json
+{{& example_navigation_and_extraction}}
+```
+
+REMEMBER: You are NEVER allowed to specify multiple actions in the list of actions.
+
+
+3. ELEMENT INTERACTION:
+   - Only use `ids` that exist in the provided element list
+   - Each element has a unique `id` (e.g., `I2[:]<button>`)
+   - Elements marked with `_[:]` are non-interactive (for context only)
+
+4. NAVIGATION & ERROR HANDLING:
+   - If no suitable elements exist, use other functions to complete the task
+   - If stuck, try alternative approaches
+   - Handle popups/cookies by accepting or closing them
+   - Use scroll to find elements you are looking for
+
+5. TASK COMPLETION:
+   - Use the `{{completion_action_name}}` action as the last action as soon as the task is complete
+   - Don't hallucinate actions
+   - If the task requires specific information - make sure to include everything in the `{{completion_action_name}}` function. This is what the user will see.
+   - If you are running out of steps (current step), think about speeding it up, and ALWAYS use the `{{completion_action_name}}` action as the last action.
+   - Note that the `{{completion_action_name}}` can fail because an external validator failed to validate the output. If this happens, you should reflect on why the output is invalid and try to fix it.
+
+   - Example of sucessfuly `{{completion_action_name}}` action:
+```json
+{{& completion_example}}
+```
+
+6. VISUAL CONTEXT:
+   - When an image is provided, use it to understand the page layout
+   - Bounding boxes with labels correspond to element indexes
+   - Each bounding box and its label have the same color
+   - Most often the label is inside the bounding box, on the top right
+   - Visual context helps verify element locations and relationships
+   - sometimes labels overlap, so use the context to verify the correct element
+
+7. Form filling:
+   - If you fill an input field and your action sequence is interrupted, most often a list with suggestions popped up under the field and you need to first select the right element from the suggestion list.
+
+8. ACTION SEQUENCING:
+   - Actions are executed in the order they appear in the list
+   - Each action should logically follow from the previous one
+   - If the page changes after an action, the sequence is interrupted and you get the new state.
+   - If content only disappears the sequence continues.
+   - Only provide the action sequence until you think the page will change.
+   - Try to be efficient, e.g. fill forms at once, or chain actions where nothing changes on the page like saving, extracting, checkboxes...
+   - NEVER use multiple actions in a single step (otherwise ONLY the first action will be executed)
+
+9. Long tasks:
+- If the task is long keep track of the status in the memory. If the ultimate task requires multiple subinformation, keep track of the status in the memory
+
+Functions:
+{{& action_description}}
+
+Remember: Your responses must be valid JSON matching the specified format. Each action in the sequence must be valid.

notte_agent/falco/trajectory_history.py

@@ -0,0 +1,42 @@
+from pydantic import Field
+from typing_extensions import override
+
+from notte_agent.common.trajectory_history import (
+    TrajectoryHistory,
+    TrajectoryStep,
+)
+from notte_agent.falco.types import StepAgentOutput
+
+
+class FalcoTrajectoryHistory(TrajectoryHistory[StepAgentOutput]):
+    steps: list[TrajectoryStep[StepAgentOutput]] = Field(default_factory=list)
+    max_error_length: int | None = None
+
+    @override
+    def perceive_step(
+        self,
+        step: TrajectoryStep[StepAgentOutput],
+        step_idx: int = 0,
+        include_ids: bool = False,
+        include_data: bool = True,
+    ) -> str:
+        action_msg = "\n".join(["  - " + result.input.dump_str() for result in step.results])
+        status_msg = "\n".join(
+            ["  - " + self.perceive_step_result(result, include_ids, include_data) for result in step.results]
+        )
+        return f"""
+# Execution step {step_idx}
+* state:
+    - page_summary: {step.agent_response.state.page_summary}
+    - previous_goal_status: {step.agent_response.state.previous_goal_status}
+    - previous_goal_eval: {step.agent_response.state.previous_goal_eval}
+    - memory: {step.agent_response.state.memory}
+    - next_goal: {step.agent_response.state.next_goal}
+* selected actions:
+{action_msg}
+* execution results:
+{status_msg}"""
+
+    @override
+    def add_output(self, output: StepAgentOutput) -> None:
+        self.steps.append(TrajectoryStep(agent_response=output, results=[]))

notte_agent/falco/types.py

@@ -0,0 +1,132 @@
+from typing import Any, Literal, TypeVar
+
+from loguru import logger
+from notte_core.controller.actions import BaseAction, ClickAction, CompletionAction
+from notte_core.controller.space import ActionSpace
+from pydantic import BaseModel, Field, create_model, field_serializer
+
+
+class RelevantInteraction(BaseModel):
+    """Interaction ids that can be relevant to the next actions"""
+
+    id: str
+    reason: str
+
+
+class AgentState(BaseModel):
+    """Current state of the agent"""
+
+    previous_goal_status: Literal["success", "failure", "unknown"]
+    previous_goal_eval: str
+    page_summary: str
+    relevant_interactions: list[RelevantInteraction]
+    memory: str
+    next_goal: str
+
+
+# TODO: for later when we do a refactoring
+class BetterAgentAction(BaseModel):
+    """Base class for agent actions with explicit action handling"""
+
+    action_name: str
+    parameters: dict[str, str | int | bool | None]
+
+    @classmethod
+    def from_action(cls, action: BaseAction) -> "BetterAgentAction":
+        return cls(action_name=action.name(), parameters=action.model_dump(exclude={"category", "id"}))
+
+    def to_action(self, space: ActionSpace) -> BaseAction:
+        action_cls = space.action_map.get(self.action_name)
+        if not action_cls:
+            raise ValueError(f"Unknown action type: {self.action_name}")
+        return action_cls(**self.parameters)  # type: ignore[arg-type]
+
+
+class AgentAction(BaseModel):
+    def to_action(self) -> BaseAction:
+        field_sets = self.model_fields_set
+        if len(field_sets) != 1:
+            raise ValueError(f"Multiple actions found in {self.model_dump_json()}")
+        action_name = list(field_sets)[0]
+        return getattr(self, action_name)
+
+
+def create_agent_action_model() -> type[AgentAction]:
+    """Creates a Pydantic model from registered actions"""
+    space = ActionSpace(description="does not matter")
+    fields = {
+        name: (
+            ActionModel | None,
+            Field(default=None, description=ActionModel.model_json_schema()["properties"]["description"]["default"]),
+        )
+        for name, ActionModel in space.action_map.items()
+    }
+    return create_model(AgentAction.__name__, __base__=AgentAction, **fields)  # type: ignore[call-overload]
+
+
+TAgentAction = TypeVar("TAgentAction", bound=AgentAction)
+
+_AgentAction: type[AgentAction] = create_agent_action_model()
+
+
+class StepAgentOutput(BaseModel):
+    state: AgentState
+    actions: list[_AgentAction] = Field(min_length=1)  # type: ignore[type-arg]
+
+    @field_serializer("actions")
+    def serialize_actions(self, actions: list[AgentAction], _info: Any) -> list[dict[str, Any]]:
+        return [action.to_action().dump_dict() for action in actions]
+
+    @property
+    def output(self) -> CompletionAction | None:
+        last_action: CompletionAction | None = getattr(self.actions[-1], CompletionAction.name())  # type: ignore[attr-defined]
+        if last_action is not None:
+            return CompletionAction(success=last_action.success, answer=last_action.answer)
+        return None
+
+    def get_actions(self, max_actions: int | None = None) -> list[BaseAction]:
+        actions: list[BaseAction] = []
+        # compute valid list of actions
+        raw_actions: list[AgentAction] = self.actions  # type: ignore[type-assignment]
+        for i, _action in enumerate(raw_actions):
+            is_last = i == len(raw_actions) - 1
+            actions.append(_action.to_action())
+            if not is_last and max_actions is not None and i >= max_actions:
+                logger.warning(f"Max actions reached: {max_actions}. Skipping remaining actions.")
+                break
+            if not is_last and actions[-1].name() == ClickAction.name() and actions[-1].id.startswith("L"):
+                logger.warning(f"Removing all actions after link click: {actions[-1].id}")
+                # all actions after a link `L` should be removed from the list
+                break
+        return actions
+
+    def pretty_string(self, colors: bool = True) -> str:
+        status = self.state.previous_goal_status
+        status_emoji: str
+        match status:
+            case "unknown":
+                status_emoji = "❓"
+            case "success":
+                status_emoji = "✅"
+            case "failure":
+                status_emoji = "❌"
+
+        def surround_tags(s: str, tags: tuple[str, ...] = ("b", "blue")) -> str:
+            if not colors:
+                return s
+
+            start = "".join(f"<{tag}>" for tag in tags)
+            end = "".join(f"</{tag}>" for tag in reversed(tags))
+            return f"{start}{s}{end}"
+
+        action_str = ""
+        actions: list[AgentAction] = self.actions  # type: ignore[reportUnkownMemberType]
+        for action in actions:
+            action_base: BaseAction = action.to_action()
+            action_str += f"   ▶ {action_base.name()} with id {action_base.id}"
+        return f"""📝 {surround_tags("Current page:")} {self.state.page_summary}
+🔬 {surround_tags("Previous goal:")} {status_emoji} {self.state.previous_goal_eval}
+🧠 {surround_tags("Memory:")} {self.state.memory}
+🎯 {surround_tags("Next goal:")} {self.state.next_goal}
+⚡ {surround_tags("Taking action:")}
+{action_str}"""

notte_agent/gufo/agent.py

@@ -0,0 +1,180 @@
+from collections.abc import Callable
+
+from loguru import logger
+from notte_browser.dom.locate import locate_element
+from notte_browser.resolution import NodeResolutionPipe
+from notte_browser.session import NotteSession, NotteSessionConfig
+from notte_browser.vault import VaultScreetsScreenshotMask
+from notte_browser.window import BrowserWindow
+from notte_core.browser.observation import Observation
+from notte_core.common.tracer import LlmUsageDictTracer
+from notte_core.controller.actions import CompletionAction, InteractionAction
+from notte_core.credentials.base import BaseVault
+from notte_core.llms.engine import LLMEngine
+from patchright.async_api import Locator
+from typing_extensions import override
+
+from notte_agent.common.base import BaseAgent
+from notte_agent.common.config import AgentConfig
+from notte_agent.common.conversation import Conversation
+from notte_agent.common.parser import NotteStepAgentOutput
+from notte_agent.common.types import AgentResponse
+from notte_agent.falco.agent import FalcoAgent
+from notte_agent.gufo.parser import GufoParser
+from notte_agent.gufo.perception import GufoPerception
+from notte_agent.gufo.prompt import GufoPrompt
+
+
+class GufoAgentConfig(AgentConfig):
+    @classmethod
+    @override
+    def default_session(cls) -> NotteSessionConfig:
+        return NotteSessionConfig().use_llm()
+
+
+class GufoAgent(BaseAgent):
+    """
+    A base agent implementation that coordinates between an LLM and the Notte environment.
+
+    This class demonstrates how to build an agent that can:
+    1. Maintain a conversation with an LLM
+    2. Execute actions in the Notte environment
+    3. Parse and format responses between the LLM and Notte
+
+    To customize this agent:
+    1. Implement your own Parser class to format observations and actions
+    2. Modify the conversation flow in the run() method
+    3. Adjust the think() method to handle LLM interactions
+    4. Customize the ask_notte() method for your specific needs
+
+    Args:
+        task (str): The task description for the agent
+        model (str): The LLM model identifier
+        max_steps (int): Maximum number of steps before terminating
+        headless (bool): Whether to run browser in headless mode
+        parser (Parser | None): Custom parser for formatting interactions
+    """
+
+    def __init__(
+        self,
+        config: AgentConfig,
+        window: BrowserWindow | None = None,
+        vault: BaseVault | None = None,
+        step_callback: Callable[[str, NotteStepAgentOutput], None] | None = None,
+    ) -> None:
+        super().__init__(session=NotteSession(config=config.session, window=window))
+        self.step_callback: Callable[[str, NotteStepAgentOutput], None] | None = step_callback
+        self.tracer: LlmUsageDictTracer = LlmUsageDictTracer()
+        self.config: AgentConfig = config
+        self.vault: BaseVault | None = vault
+        self.llm: LLMEngine = LLMEngine(
+            model=config.reasoning_model,
+            tracer=self.tracer,
+            structured_output_retries=config.session.structured_output_retries,
+            verbose=self.config.verbose,
+        )
+        # Users should implement their own parser to customize how observations
+        # and actions are formatted for their specific LLM and use case
+        self.parser: GufoParser = GufoParser()
+        self.prompt: GufoPrompt = GufoPrompt(self.parser)
+        self.perception: GufoPerception = GufoPerception()
+        self.conv: Conversation = Conversation()
+
+        if self.vault is not None:
+            # hide vault leaked credentials within llm inputs
+            self.llm.structured_completion = self.vault.patch_structured_completion(0, self.vault.get_replacement_map)(
+                self.llm.structured_completion
+            )
+
+    async def reset(self):
+        await self.session.reset()
+        self.conv.reset()
+
+    def output(self, answer: str, success: bool) -> AgentResponse:
+        return AgentResponse(
+            answer=answer,
+            success=success,
+            session_trajectory=self.session.trajectory,
+            agent_trajectory=[],
+            llm_usage=self.tracer.usage,
+        )
+
+    async def step(self, task: str) -> CompletionAction | None:
+        # Processes the conversation history through the LLM to decide the next action.
+        # logger.info(f"🤖 LLM prompt:\n{self.conv.messages()}")
+        response: str = self.llm.single_completion(self.conv.messages())
+        self.conv.add_assistant_message(content=response)
+        logger.info(f"🤖 LLM response:\n{response}")
+        # Ask Notte to perform the selected action
+        parsed_response = self.parser.parse(response)
+
+        if parsed_response is None or parsed_response.action is None:
+            self.conv.add_user_message(content=self.prompt.env_rules())
+            return None
+
+        if self.step_callback is not None:
+            self.step_callback(task, parsed_response)
+
+        if parsed_response.completion is not None:
+            return parsed_response.completion
+        action = parsed_response.action
+        # Replace credentials if needed using the vault
+        if self.vault is not None and self.vault.contains_credentials(action):
+            action_with_selector = await NodeResolutionPipe.forward(action, self.session.snapshot)
+
+            if isinstance(action_with_selector, InteractionAction) and action_with_selector.selector is not None:
+                locator: Locator = await locate_element(self.session.window.page, action_with_selector.selector)
+                attrs = await FalcoAgent.compute_locator_attributes(locator)
+
+                assert isinstance(action_with_selector, InteractionAction) and action_with_selector.selector is not None
+
+                action = self.vault.replace_credentials(
+                    action,
+                    attrs,
+                    self.session.snapshot,
+                )
+        # Execute the action
+        obs: Observation = await self.session.act(action)
+        text_obs = self.perception.perceive(obs)
+        self.conv.add_user_message(
+            content=f"""
+{text_obs}
+{self.prompt.select_action_rules()}
+{self.prompt.completion_rules()}
+""",
+            image=obs.screenshot if self.config.include_screenshot else None,
+        )
+        logger.info(f"🌌 Action successfully executed:\n{text_obs}")
+        return None
+
+    @override
+    async def run(self, task: str, url: str | None = None) -> AgentResponse:
+        """
+        Main execution loop that coordinates between the LLM and Notte environment.
+
+        This method shows a basic conversation flow. Consider customizing:
+        1. The initial system prompt
+        2. How observations are added to the conversation
+        3. When and how to determine task completion
+        4. Error handling and recovery strategies
+        """
+        logger.info(f"🚀 starting agent with task: {task} and url: {url}")
+        system_msg = self.prompt.system(task, url)
+        if self.vault is not None:
+            system_msg += "\n" + self.vault.instructions()
+        self.conv.add_system_message(content=system_msg)
+        self.conv.add_user_message(self.prompt.env_rules())
+        async with self.session:
+            if self.vault is not None:
+                self.session.window.screenshot_mask = VaultScreetsScreenshotMask(vault=self.vault)
+            for i in range(self.config.session.max_steps):
+                logger.info(f"> step {i}: looping in")
+                output = await self.step(task=task)
+                if output is not None:
+                    status = "😎 task completed sucessfully" if output.success else "👿 task failed"
+                    logger.info(f"{status} with answer: {output.answer}")
+                    return self.output(output.answer, output.success)
+            # If the task is not done, raise an error
+            error_msg = f"Failed to solve task in {self.config.session.max_steps} steps"
+            logger.info(f"🚨 {error_msg}")
+            return self.output(error_msg, False)

notte_agent/gufo/parser.py

@@ -0,0 +1,79 @@
+from typing import ClassVar, Literal
+
+from notte_core.actions.base import ActionParameterValue, ExecutableAction
+from notte_core.controller.actions import CompletionAction, GotoAction, ScrapeAction
+from typing_extensions import override
+
+from notte_agent.common.parser import BaseParser, NotteStepAgentOutput
+
+
+class GufoParser(BaseParser):
+    observe_tag: ClassVar[str] = "observe"
+    step_tag: ClassVar[str] = "execute-action"
+    scrape_tag: ClassVar[str] = "scrape-data"
+    done_tag: ClassVar[str] = "done"
+
+    @override
+    def example_format(self, endpoint: Literal["observe", "step", "scrape", "done", "error"]) -> str | None:
+        match endpoint:
+            case "observe":
+                return f"""
+<{self.observe_tag}>
+{GotoAction(url="https://www.example.com").dump_str(name=False)}
+</{self.observe_tag}>
+"""
+            case "step":
+                return f"""
+<{self.step_tag}>
+{
+                    ExecutableAction(
+                        id="<YOUR_ACTION_ID>",
+                        params_values=[ActionParameterValue(name="<YOUR_PARAM_NAME>", value="<YOUR_PARAM_VALUE>")],
+                    ).dump_str(name=False)
+                }
+</{self.step_tag}>
+"""
+            case "scrape":
+                return f"""
+<{self.scrape_tag}>
+{ScrapeAction(instructions="<YOUR_SCRAPING_INSTRUCTIONS | null to scrape the whole page>").dump_str(name=False)}
+</{self.scrape_tag}>
+"""
+            case "done":
+                return f"""
+<{self.done_tag}>
+{CompletionAction(success=True, answer="<YOUR_ANSWER>").dump_str(name=False)}
+</{self.done_tag}>
+"""
+            case "error":
+                return f"""
+<{self.done_tag}>
+{CompletionAction(success=False, answer="<REASON_FOR_FAILURE>").dump_str(name=False)}
+</{self.done_tag}>
+"""
+
+    @override
+    def parse(self, text: str) -> NotteStepAgentOutput | None:
+        url = self.search_pattern(text, GufoParser.observe_tag)
+        action = self.search_pattern(text, GufoParser.step_tag)
+        scrape = self.search_pattern(text, GufoParser.scrape_tag)
+        output = self.search_pattern(text, GufoParser.done_tag)
+        match (bool(url), bool(action), bool(scrape), bool(output)):
+            case (True, False, False, False):
+                return NotteStepAgentOutput(
+                    observe=GotoAction.model_validate(self.parse_json(text, GufoParser.observe_tag))
+                )
+            case (False, True, False, False):
+                return NotteStepAgentOutput(
+                    step=ExecutableAction.model_validate(self.parse_json(text, GufoParser.step_tag)),
+                )
+            case (False, False, True, False):
+                return NotteStepAgentOutput(
+                    scrape=ScrapeAction.model_validate(self.parse_json(text, GufoParser.scrape_tag))
+                )
+            case (False, False, False, True):
+                return NotteStepAgentOutput(
+                    completion=CompletionAction.model_validate(self.parse_json(text, GufoParser.done_tag))
+                )
+            case _:
+                return None

notte_agent/gufo/perception.py

@@ -0,0 +1,53 @@
+from typing import final
+
+from notte_core.browser.observation import Observation
+from typing_extensions import override
+
+from notte_agent.common.perception import BasePerception
+
+
+@final
+class GufoPerception(BasePerception):
+    @override
+    def perceive_metadata(self, obs: Observation) -> str:
+        space_description = obs.space.description
+        category: str = obs.space.category.value if obs.space.category is not None else ""
+        return f"""
+Webpage information:
+- URL: {obs.metadata.url}
+- Title: {obs.metadata.title}
+- Description: {space_description or "No description available"}
+- Timestamp: {obs.metadata.timestamp.strftime("%Y-%m-%d %H:%M:%S")}
+- Page category: {category or "No category available"}
+"""
+
+    @override
+    def perceive_data(
+        self,
+        obs: Observation,
+    ) -> str:
+        if not obs.has_data():
+            raise ValueError("No scraping data found")
+        return f"""
+Here is some data that has been extracted from this page:
+<data>
+{obs.data.markdown if obs.data is not None else "No data available"}
+</data>
+"""
+
+    @override
+    def perceive_actions(self, obs: Observation) -> str:
+        return f"""
+Here are the available actions you can take on this page:
+<actions>
+{obs.space.markdown()}
+</actions>
+"""
+
+    @override
+    def perceive(self, obs: Observation) -> str:
+        return f"""
+{self.perceive_metadata(obs).strip()}
+{self.perceive_data(obs).strip() if obs.has_data() else ""}
+{self.perceive_actions(obs).strip()}
+"""

notte_agent/gufo/prompt.py

@@ -0,0 +1,61 @@
+from pathlib import Path
+
+import chevron
+
+from notte_agent.gufo.parser import GufoParser
+
+system_prompt_file = Path(__file__).parent / "system.md"
+
+
+class GufoPrompt:
+    def __init__(self, parser: GufoParser):
+        self.parser: GufoParser = parser
+        self.system_prompt: str = system_prompt_file.read_text()
+
+    def system(self, task: str, url: str | None = None) -> str:
+        return chevron.render(self.system_prompt, {"task": task, "url": url or "the web"}, warn=True)
+
+    def env_rules(self) -> str:
+        return f"""
+Hi there! I am the Notte web environment, and will help you navigate the internet.
+# How it works:
+* Provide me with a URL. I will respond with the actions you can take on that page.
+* You are NOT allowed to provide me with more than one URL.
+* Important: Make sure to use the **exact format** below when sending me a URL:
+{self.parser.example_format("observe")}
+> So, where would you like to go?
+"""
+
+    def completion_rules(self) -> str:
+        return f"""
+# How to format your answer when you're done
+## Success answer
+* If you're done, include you final answer in <{self.parser.done_tag}> tags.
+* Don't forget to justify why your answer is correct and solves the task.
+* Don't assume anything, just provide factual information backuped by the page you're on.
+Format your answer as follows:
+{self.parser.example_format("done")}
+
+## Error answer
+* If you feel stuck, remember that you are also allowed to use `Special Browser Actions` at any time to:
+    * Go to a different url
+    * Go back to the previous page
+    * Refresh the current page
+    * Scrape data from the page
+    * Etc
+* If you want to stop or you're unable to pursue your goal, format your answer as follows:
+{self.parser.example_format("error")}
+"""
+
+    def select_action_rules(self) -> str:
+        return f"""
+# Next Action Selection
+* Provide me with the ID of the action you want to take next.
+* You are allowed to take only exactly ONE action from the list.
+* You are ONLY allowed to pick actions from the latest list of actions!
+* You are NOT allowed to pick actions from list of actions in previous messages!
+* If the action is parameterized, provide the value for each parameter.
+Use the exact following format:
+
+{self.parser.example_format("step")}
+"""

notte_agent/gufo/system.md

@@ -0,0 +1,8 @@
+You are a helpful web agent.
+Now you are given the task: {{task}}.
+Please interact with : {{url}} to get the answer.
+
+Instructions:
+- At every step, you will be provided with a list of actions you can take.
+- If you are asked to accept cookies to continue, please accept them. Accepting cookies is MANDATORY.
+- If you see one action about cookie management, you should stop thinking about the goal and accept cookies DIRECTLY.