devseeker 0.0.1__tar.gz

devseeker-0.0.1/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2023 Ibrahim Rayamah
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

devseeker-0.0.1/PKG-INFO

@@ -0,0 +1,120 @@
+Metadata-Version: 2.3
+Name: devseeker
+Version: 0.0.1
+Summary: Coolest coding agent
+License: MIT
+Author: Ibrahim Rayamah
+Author-email: ibz04.dev@gmail.com
+Requires-Python: >=3.10,<3.13
+Classifier: Development Status :: 4 - Beta
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Provides-Extra: doc
+Provides-Extra: test
+Requires-Dist: black (==23.3.0)
+Requires-Dist: dataclasses-json (==0.5.7)
+Requires-Dist: datasets (>=2.17.1,<3.0.0)
+Requires-Dist: langchain (>=0.1.2)
+Requires-Dist: langchain-anthropic (>=0.1.1,<0.2.0)
+Requires-Dist: langchain-community (>=0.2.0,<0.3.0)
+Requires-Dist: langchain_openai
+Requires-Dist: openai (>=1.0,<2.0)
+Requires-Dist: pillow (>=10.2.0,<11.0.0)
+Requires-Dist: pyperclip (>=1.8.2,<2.0.0)
+Requires-Dist: python-dotenv (>=0.21.0)
+Requires-Dist: regex (>=2023.12.25,<2024.0.0)
+Requires-Dist: rudder-sdk-python (>=2.0.2)
+Requires-Dist: tabulate (==0.9.0)
+Requires-Dist: termcolor (==2.3.0)
+Requires-Dist: tiktoken (>=0.0.4)
+Requires-Dist: toml (>=0.10.2)
+Requires-Dist: tomlkit (>=0.12.4,<0.13.0)
+Requires-Dist: typer (>=0.3.2)
+Project-URL: Documentation, https://github.com/iBz-04/Devseeker/blob/main/README.md
+Project-URL: Homepage, https://github.com/iBz-04/Devseeker
+Project-URL: Repository, https://github.com/iBz-04/Devseekerr
+Description-Content-Type: text/markdown
+
+# devseeker
+
+[![PyPI version](https://img.shields.io/pypi/v/devseeker)](https://pypi.org/project/devseeker)
+[![License](https://img.shields.io/github/license/devseeker-org/devseeker)](LICENSE)
+[![Tests](https://github.com/devseeker-org/devseeker/actions/workflows/ci.yaml/badge.svg)](https://github.com/devseeker-org/devseeker/actions/workflows/ci.yaml)
+
+**devseeker is an CLI coding agent for generating and improving code through prompts**
+
+## Installation
+
+### Stable release
+
+```sh
+pip install devseeker
+```
+
+### Development installation
+
+```sh
+git clone https://github.com/iBz-04/devseeker.git
+cd devseeker
+poetry install
+poetry shell
+```
+
+## Configuration
+
+devseeker requires an OpenAI API key. Set it as an environment variable or in a `.env` file:
+
+```sh
+export OPENAI_API_KEY=your_api_key
+```
+
+or create a `.env` file:
+
+```
+OPENAI_API_KEY=your_api_key
+```
+
+## Usage
+
+### Creating a new project
+
+1. Create an empty directory for your project.
+2. Inside the directory, create a file named `prompt` containing your instructions.
+3. Run:
+
+```sh
+devseeker projects/my-new-project
+```
+
+### Improving existing code
+
+```sh
+devseeker projects/my-existing-project -i
+```
+
+### Benchmarking AI agents
+
+```sh
+bench run --help
+```
+
+## Commands
+
+- `devseeker` (alias `ds`, `dste`) runs the main CLI application.
+- `bench` runs benchmarks on AI agents.
+
+
+- Windows users can refer to [WINDOWS_README.md](WINDOWS_README.md).
+
+## Contributing
+
+Contributions are welcome! 
+
+## License
+
+This project is licensed under the MIT License. See [LICENSE](LICENSE) for details.
+

devseeker-0.0.1/README.md

@@ -0,0 +1,78 @@
+# devseeker
+
+[![PyPI version](https://img.shields.io/pypi/v/devseeker)](https://pypi.org/project/devseeker)
+[![License](https://img.shields.io/github/license/devseeker-org/devseeker)](LICENSE)
+[![Tests](https://github.com/devseeker-org/devseeker/actions/workflows/ci.yaml/badge.svg)](https://github.com/devseeker-org/devseeker/actions/workflows/ci.yaml)
+
+**devseeker is an CLI coding agent for generating and improving code through prompts**
+
+## Installation
+
+### Stable release
+
+```sh
+pip install devseeker
+```
+
+### Development installation
+
+```sh
+git clone https://github.com/iBz-04/devseeker.git
+cd devseeker
+poetry install
+poetry shell
+```
+
+## Configuration
+
+devseeker requires an OpenAI API key. Set it as an environment variable or in a `.env` file:
+
+```sh
+export OPENAI_API_KEY=your_api_key
+```
+
+or create a `.env` file:
+
+```
+OPENAI_API_KEY=your_api_key
+```
+
+## Usage
+
+### Creating a new project
+
+1. Create an empty directory for your project.
+2. Inside the directory, create a file named `prompt` containing your instructions.
+3. Run:
+
+```sh
+devseeker projects/my-new-project
+```
+
+### Improving existing code
+
+```sh
+devseeker projects/my-existing-project -i
+```
+
+### Benchmarking AI agents
+
+```sh
+bench run --help
+```
+
+## Commands
+
+- `devseeker` (alias `ds`, `dste`) runs the main CLI application.
+- `bench` runs benchmarks on AI agents.
+
+
+- Windows users can refer to [WINDOWS_README.md](WINDOWS_README.md).
+
+## Contributing
+
+Contributions are welcome! 
+
+## License
+
+This project is licensed under the MIT License. See [LICENSE](LICENSE) for details.

devseeker-0.0.1/devseeker/__init__.py

@@ -0,0 +1,4 @@
+# Adding convenience imports to the package
+
+# from devseeker.tools import code_vector_repository
+# from devseeker.core.default import on_disk_repository

devseeker-0.0.1/devseeker/applications/cli/cli_agent.py

@@ -0,0 +1,232 @@
+"""
+This module provides the CliAgent class which manages the lifecycle of code generation and improvement
+using an AI model. It includes functionalities to initialize code generation, improve existing code,
+and process the code through various steps defined in the step bundle.
+"""
+
+from typing import Callable, Optional, TypeVar
+
+# from devseeker.core.default.git_version_manager import GitVersionManager
+from devseeker.core.ai import AI
+from devseeker.core.base_agent import BaseAgent
+from devseeker.core.base_execution_env import BaseExecutionEnv
+from devseeker.core.base_memory import BaseMemory
+from devseeker.core.default.disk_execution_env import DiskExecutionEnv
+from devseeker.core.default.disk_memory import DiskMemory
+from devseeker.core.default.paths import PREPROMPTS_PATH
+from devseeker.core.default.steps import (
+    execute_entrypoint,
+    gen_code,
+    gen_entrypoint,
+    improve_fn,
+)
+from devseeker.core.files_dict import FilesDict
+from devseeker.core.preprompts_holder import PrepromptsHolder
+from devseeker.core.prompt import Prompt
+
+CodeGenType = TypeVar("CodeGenType", bound=Callable[[AI, str, BaseMemory], FilesDict])
+CodeProcessor = TypeVar(
+    "CodeProcessor", bound=Callable[[AI, BaseExecutionEnv, FilesDict], FilesDict]
+)
+ImproveType = TypeVar(
+    "ImproveType", bound=Callable[[AI, str, FilesDict, BaseMemory], FilesDict]
+)
+
+
+class CliAgent(BaseAgent):
+    """
+    The `CliAgent` class is responsible for managing the lifecycle of code generation and improvement
+    using an AI model. It orchestrates the generation of new code and the improvement of existing code
+    based on given prompts and utilizes a memory system and execution environment for processing.
+
+    Parameters
+    ----------
+    memory : BaseMemory
+        An instance of a class that adheres to the BaseMemory interface, used for storing and retrieving
+        information during the code generation process.
+    execution_env : BaseExecutionEnv
+        An instance of a class that adheres to the BaseExecutionEnv interface, used for executing code
+        and managing the execution environment.
+    ai : AI, optional
+        An instance of the AI class that manages calls to the language model. If not provided, a default
+        instance is created.
+    code_gen_fn : CodeGenType, optional
+        A callable that takes an AI instance, a prompt, and a memory instance to generate code. Defaults
+        to the `gen_code` function.
+    improve_fn : ImproveType, optional
+        A callable that takes an AI instance, a prompt, a FilesDict instance, and a memory instance to
+        improve code. Defaults to the `improve` function.
+    process_code_fn : CodeProcessor, optional
+        A callable that takes an AI instance, an execution environment, and a FilesDict instance to
+        process code. Defaults to the `execute_entrypoint` function.
+    preprompts_holder : PrepromptsHolder, optional
+        An instance of PrepromptsHolder that manages preprompt templates. If not provided, a default
+        instance is created using the PREPROMPTS_PATH.
+
+    Attributes
+    ----------
+    memory : BaseMemory
+        The memory instance where the agent stores and retrieves information.
+    execution_env : BaseExecutionEnv
+        The execution environment instance where the agent executes and manages code.
+    ai : AI
+        The AI instance used for interacting with the language model.
+    code_gen_fn : CodeGenType
+        The function used for generating code.
+    improve_fn : ImproveType
+        The function used for improving code.
+    process_code_fn : CodeProcessor
+        The function used for processing code.
+    preprompts_holder : PrepromptsHolder
+        The holder for preprompt templates.
+    """
+
+    def __init__(
+        self,
+        memory: BaseMemory,
+        execution_env: BaseExecutionEnv,
+        ai: AI = None,
+        code_gen_fn: CodeGenType = gen_code,
+        improve_fn: ImproveType = improve_fn,
+        process_code_fn: CodeProcessor = execute_entrypoint,
+        preprompts_holder: PrepromptsHolder = None,
+    ):
+        self.memory = memory
+        self.execution_env = execution_env
+        self.ai = ai or AI()
+        self.code_gen_fn = code_gen_fn
+        self.process_code_fn = process_code_fn
+        self.improve_fn = improve_fn
+        self.preprompts_holder = preprompts_holder or PrepromptsHolder(PREPROMPTS_PATH)
+
+    @classmethod
+    def with_default_config(
+        cls,
+        memory: DiskMemory,
+        execution_env: DiskExecutionEnv,
+        ai: AI = None,
+        code_gen_fn: CodeGenType = gen_code,
+        improve_fn: ImproveType = improve_fn,
+        process_code_fn: CodeProcessor = execute_entrypoint,
+        preprompts_holder: PrepromptsHolder = None,
+        diff_timeout=3,
+    ):
+        """
+        Creates a new instance of CliAgent with default configurations for memory, execution environment,
+        AI, and other functional parameters.
+
+        Parameters
+        ----------
+        memory : DiskMemory
+            An instance of DiskMemory for storing and retrieving information.
+        execution_env : DiskExecutionEnv
+            An instance of DiskExecutionEnv for executing code.
+        ai : AI, optional
+            An instance of AI for interacting with the language model. Defaults to None, which will create
+            a new AI instance.
+        code_gen_fn : CodeGenType, optional
+            A function for generating code. Defaults to `gen_code`.
+        improve_fn : ImproveType, optional
+            A function for improving code. Defaults to `improve`.
+        process_code_fn : CodeProcessor, optional
+            A function for processing code. Defaults to `execute_entrypoint`.
+        preprompts_holder : PrepromptsHolder, optional
+            An instance of PrepromptsHolder for managing preprompt templates. Defaults to None, which will
+            create a new PrepromptsHolder instance using PREPROMPTS_PATH.
+
+        Returns
+        -------
+        CliAgent
+            An instance of CliAgent configured with the provided or default parameters.
+        """
+        return cls(
+            memory=memory,
+            execution_env=execution_env,
+            ai=ai,
+            code_gen_fn=code_gen_fn,
+            process_code_fn=process_code_fn,
+            improve_fn=improve_fn,
+            preprompts_holder=preprompts_holder or PrepromptsHolder(PREPROMPTS_PATH),
+        )
+
+    def init(self, prompt: Prompt) -> FilesDict:
+        """
+        Generates a new piece of code using the AI and step bundle based on the provided prompt.
+
+        Parameters
+        ----------
+        prompt : str
+            A string prompt that guides the code generation process.
+
+        Returns
+        -------
+        FilesDict
+            An instance of the `FilesDict` class containing the generated code.
+        """
+
+        files_dict = self.code_gen_fn(
+            self.ai, prompt, self.memory, self.preprompts_holder
+        )
+        entrypoint = gen_entrypoint(
+            self.ai, prompt, files_dict, self.memory, self.preprompts_holder
+        )
+        combined_dict = {**files_dict, **entrypoint}
+        files_dict = FilesDict(combined_dict)
+        files_dict = self.process_code_fn(
+            self.ai,
+            self.execution_env,
+            files_dict,
+            preprompts_holder=self.preprompts_holder,
+            prompt=prompt,
+            memory=self.memory,
+        )
+        return files_dict
+
+    def improve(
+        self,
+        files_dict: FilesDict,
+        prompt: Prompt,
+        execution_command: Optional[str] = None,
+        diff_timeout=3,
+    ) -> FilesDict:
+        """
+        Improves an existing piece of code using the AI and step bundle based on the provided prompt.
+
+        Parameters
+        ----------
+        files_dict : FilesDict
+            An instance of `FilesDict` containing the code to be improved.
+        prompt : str
+            A string prompt that guides the code improvement process.
+        execution_command : str, optional
+            An optional command to execute the code. If not provided, the default execution command is used.
+
+        Returns
+        -------
+        FilesDict
+            An instance of the `FilesDict` class containing the improved code.
+        """
+
+        files_dict = self.improve_fn(
+            self.ai,
+            prompt,
+            files_dict,
+            self.memory,
+            self.preprompts_holder,
+            diff_timeout=diff_timeout,
+        )
+        # entrypoint = gen_entrypoint(
+        #     self.ai, prompt, files_dict, self.memory, self.preprompts_holder
+        # )
+        # combined_dict = {**files_dict, **entrypoint}
+        # files_dict = FilesDict(combined_dict)
+        # files_dict = self.process_code_fn(
+        #     self.ai,
+        #     self.execution_env,
+        #     files_dict,
+        #     preprompts_holder=self.preprompts_holder,
+        #     prompt=prompt,
+        #     memory=self.memory,
+        # )
+
+        return files_dict

devseeker-0.0.1/devseeker/applications/cli/collect.py

@@ -0,0 +1,177 @@
+"""
+Module `collect` - Data Handling and RudderStack Integration
+
+This module provides functionalities to handle and send learning data to RudderStack
+for the purpose of analysis and to improve the gpt-engineer system. The data is sent
+only when the user gives consent to share.
+
+Functions:
+    send_learning(learning): Sends learning data to RudderStack.
+    collect_learnings(prompt, model, temperature, config, memory, review): Processes and sends learning data.
+    collect_and_send_human_review(prompt, model, temperature, config, memory): Collects human feedback and sends it.
+
+Dependencies:
+    hashlib: For generating SHA-256 hash.
+    typing: For type annotations.
+    devseeker.core: Core functionalities of gpt-engineer.
+    devseeker.cli.learning: Handles the extraction of learning data.
+
+Notes:
+    Data sent to RudderStack is not shared with third parties and is used solely to
+    improve gpt-engineer and allow it to handle a broader range of use cases.
+    Consent logic is in devseeker/learning.py.
+"""
+
+from typing import Tuple
+
+from devseeker.applications.cli.learning import (
+    Learning,
+    Review,
+    extract_learning,
+    human_review_input,
+)
+from devseeker.core.default.disk_memory import DiskMemory
+from devseeker.core.prompt import Prompt
+
+
+def send_learning(learning: Learning):
+    """
+    Send the learning data to RudderStack for analysis.
+
+    Parameters
+    ----------
+    learning : Learning
+        An instance of the Learning class containing the data to be sent.
+
+    Notes
+    -----
+    This function is only called if consent is given to share data.
+    Data is not shared to a third party. It is used with the sole purpose of
+    improving gpt-engineer, and letting it handle more use cases.
+    Consent logic is in devseeker/learning.py.
+    """
+    import rudderstack.analytics as rudder_analytics
+
+    rudder_analytics.write_key = "2Re4kqwL61GDp7S8ewe6K5dbogG"
+    rudder_analytics.dataPlaneUrl = "https://gptengineerezm.dataplane.rudderstack.com"
+
+    rudder_analytics.track(
+        user_id=learning.session,
+        event="learning",
+        properties=learning.to_dict(),  # type: ignore
+    )
+
+
+def collect_learnings(
+    prompt: Prompt,
+    model: str,
+    temperature: float,
+    config: any,
+    memory: DiskMemory,
+    review: Review,
+):
+    """
+    Collect the learning data and send it to RudderStack for analysis.
+
+    Parameters
+    ----------
+    prompt : str
+        The initial prompt or question that was provided to the model.
+    model : str
+        The name of the model used for generating the response.
+    temperature : float
+        The temperature setting used in the model's response generation.
+    config : any
+        Configuration parameters used for the learning session.
+    memory : DiskMemory
+        An instance of DiskMemory for storing and retrieving data.
+    review : Review
+        An instance of Review containing human feedback on the model's response.
+
+    Notes
+    -----
+    This function attempts to send the learning data to RudderStack. If the data size exceeds
+    the maximum allowed size, it trims the data and retries sending it.
+    """
+    learnings = extract_learning(prompt, model, temperature, config, memory, review)
+    try:
+        send_learning(learnings)
+    except RuntimeError:
+        # try to remove some parts of learning that might be too big
+        # rudderstack max event size is 32kb
+        max_size = 32 << 10  # 32KB in bytes
+        current_size = len(learnings.to_json().encode("utf-8"))  # get size in bytes
+
+        overflow = current_size - max_size
+
+        # Add some extra characters for the "[REMOVED...]" string and for safety margin
+        remove_length = overflow + len(f"[REMOVED {overflow} CHARACTERS]") + 100
+
+        learnings.logs = (
+            learnings.logs[:-remove_length]
+            + f"\n\n[REMOVED {remove_length} CHARACTERS]"
+        )
+
+        print(
+            "WARNING: learning too big, removing some parts. "
+            "Please report if this results in a crash."
+        )
+        try:
+            send_learning(learnings)
+        except RuntimeError:
+            print(
+                "Sending learnings crashed despite truncation. Progressing without saving learnings."
+            )
+
+
+# def steps_file_hash():
+#     """
+#     Compute the SHA-256 hash of the steps file.
+#
+#     Returns
+#     -------
+#     str
+#         The SHA-256 hash of the steps file.
+#     """
+#     with open(steps.__file__, "r") as f:
+#         content = f.read()
+#         return hashlib.sha256(content.encode("utf-8")).hexdigest()
+
+
+def collect_and_send_human_review(
+    prompt: Prompt,
+    model: str,
+    temperature: float,
+    config: Tuple[str, ...],
+    memory: DiskMemory,
+):
+    """
+    Collects human feedback on the code and sends it for analysis.
+
+    Parameters
+    ----------
+    prompt : str
+        The initial prompt or question that was provided to the model.
+    model : str
+        The name of the model used for generating the response.
+    temperature : float
+        The temperature setting used in the model's response generation.
+    config : Tuple[str, ...]
+        Configuration parameters used for the learning session.
+    memory : DiskMemory
+        An instance of DiskMemory for storing and retrieving data.
+
+    Returns
+    -------
+    None
+
+    Notes
+    -----
+    This function prompts the user for a review of the generated or improved code using the
+    `human_review_input` function. If a valid review is provided, it's serialized to JSON format
+    and stored within the database's memory under the "review" key.
+    """
+
+    review = human_review_input()
+    if review:
+        collect_learnings(prompt, model, temperature, config, memory, review)