wcgw 2.7.0__tar.gz → 2.7.2__tar.gz

{wcgw-2.7.0 → wcgw-2.7.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: wcgw
-Version: 2.7.0
+Version: 2.7.2
 Summary: Shell and coding agent on claude and chatgpt
 Project-URL: Homepage, https://github.com/rusiaaman/wcgw
 Author-email: Aman Rusia <gapypi@arcfu.com>
@@ -33,7 +33,7 @@ Description-Content-Type: text/markdown
 - Chatgpt - Allows custom gpt to talk to your shell via a relay server. (linux or mac)
 
 
-⚠️ Warning: do not use this repo if you aren't scared of "Autonomous shell command execution"
+⚠️ Warning: do not allow BashCommand tool without reviewing the command, it may result in data loss.
 
 [![Tests](https://github.com/rusiaaman/wcgw/actions/workflows/python-tests.yml/badge.svg?branch=main)](https://github.com/rusiaaman/wcgw/actions/workflows/python-tests.yml)
 [![Mypy strict](https://github.com/rusiaaman/wcgw/actions/workflows/python-types.yml/badge.svg?branch=main)](https://github.com/rusiaaman/wcgw/actions/workflows/python-types.yml)
@@ -41,6 +41,8 @@ Description-Content-Type: text/markdown
 [![codecov](https://codecov.io/gh/rusiaaman/wcgw/graph/badge.svg)](https://codecov.io/gh/rusiaaman/wcgw)
 
 ## Updates
+- [8 Jan 2025] Context saving tool for saving relevant file paths along with a description in a single file. Can be used as a task checkpoint or for knowledge transfer.
+
 - [29 Dec 2024] Syntax checking on file writing and edits is now stable. Made `initialize` tool call useful; sending smart repo structure to claude if any repo is referenced. Large file handling is also now improved.
 
 - [9 Dec 2024] [Vscode extension to paste context on Claude app](https://marketplace.visualstudio.com/items?itemName=AmanRusia.wcgw)  
@@ -55,7 +57,18 @@ Description-Content-Type: text/markdown
 - ⚡ **Large file edit**: Supports large file incremental edits to avoid token limit issues. Faster than full file write.
 - ⚡ **Syntax checking on edits**: Reports feedback to the LLM if its edits have any syntax errors, so that it can redo it.
 - ⚡ **Interactive Command Handling**: Supports interactive commands using arrow keys, interrupt, and ansi escape sequences.
-- ⚡ **Full Shell Access**: No restrictions, complete control.
+- ⚡ **File protections**: 
+  - The AI needs to read a file at least once before it's allowed to edit or rewrite it. This avoids accidental overwrites. 
+  - Avoids context filling up while reading very large files. Files get chunked based on token length. 
+  - On initialisation the provided workspace's directory structure is returned after selecting important files (based on .gitignore as well as a statistical approach)
+  - File edit based on search-replace tries to find correct search block if it has multiple matches based on previous search blocks. Fails otherwise (for correctness).
+  - File edit has spacing tolerant matching, with warning on issues like indentation mismatch. If there's no match, the closest match is returned to the AI to fix its mistakes.
+  - Using Aider-like search and replace, which has better performance than tool call based search and replace.
+- ⚡ **Shell optimisations**: 
+  - Only one command is allowed to be run at a time, simplifying management and avoiding rogue processes. There's only single shell instance at any point of time.
+  - Current working directory is always returned after any shell command to prevent AI from getting lost. 
+  - Command polling exits after a quick timeout to avoid slow feedback. However, status checking has wait tolerance based on fresh output streaming from a command. Both of these approach combined provides a good shell interaction experience.
+- ⚡ **Saving repo context in a single file**: Task checkpointing using "ContextSave" tool saves detailed context in a single file. Tasks can later be resumed in a new chat asking "Resume `task id`". The saved file can be used to do other kinds of knowledge transfer, such as taking help from another AI.
 
 ## Top use cases examples
 
@@ -129,6 +142,11 @@ over here
 
 Then ask claude to execute shell commands, read files, edit files, run your code, etc.
 
+#### Task checkpoint or knowledge transfer
+- You can do a task checkpoint or a knowledge transfer by attaching "KnowledgeTransfer" prompt using "Attach from MCP" button.
+- On running "KnowledgeTransfer" prompt, the "ContextSave" tool will be called saving the task description and all file content together in a single file. An id for the task will be generated.
+- You can in a new chat say "Resume '<task id>'", the AI should then call "Initialize" with the task id and load the context from there.
+- Or you can directly open the file generated and share it with another AI for help.
 
 ### [Optional] Vs code extension 
 https://marketplace.visualstudio.com/items?itemName=AmanRusia.wcgw

{wcgw-2.7.0 → wcgw-2.7.2}/README.md

@@ -4,7 +4,7 @@
 - Chatgpt - Allows custom gpt to talk to your shell via a relay server. (linux or mac)
 
 
-⚠️ Warning: do not use this repo if you aren't scared of "Autonomous shell command execution"
+⚠️ Warning: do not allow BashCommand tool without reviewing the command, it may result in data loss.
 
 [![Tests](https://github.com/rusiaaman/wcgw/actions/workflows/python-tests.yml/badge.svg?branch=main)](https://github.com/rusiaaman/wcgw/actions/workflows/python-tests.yml)
 [![Mypy strict](https://github.com/rusiaaman/wcgw/actions/workflows/python-types.yml/badge.svg?branch=main)](https://github.com/rusiaaman/wcgw/actions/workflows/python-types.yml)
@@ -12,6 +12,8 @@
 [![codecov](https://codecov.io/gh/rusiaaman/wcgw/graph/badge.svg)](https://codecov.io/gh/rusiaaman/wcgw)
 
 ## Updates
+- [8 Jan 2025] Context saving tool for saving relevant file paths along with a description in a single file. Can be used as a task checkpoint or for knowledge transfer.
+
 - [29 Dec 2024] Syntax checking on file writing and edits is now stable. Made `initialize` tool call useful; sending smart repo structure to claude if any repo is referenced. Large file handling is also now improved.
 
 - [9 Dec 2024] [Vscode extension to paste context on Claude app](https://marketplace.visualstudio.com/items?itemName=AmanRusia.wcgw)  
@@ -26,7 +28,18 @@
 - ⚡ **Large file edit**: Supports large file incremental edits to avoid token limit issues. Faster than full file write.
 - ⚡ **Syntax checking on edits**: Reports feedback to the LLM if its edits have any syntax errors, so that it can redo it.
 - ⚡ **Interactive Command Handling**: Supports interactive commands using arrow keys, interrupt, and ansi escape sequences.
-- ⚡ **Full Shell Access**: No restrictions, complete control.
+- ⚡ **File protections**: 
+  - The AI needs to read a file at least once before it's allowed to edit or rewrite it. This avoids accidental overwrites. 
+  - Avoids context filling up while reading very large files. Files get chunked based on token length. 
+  - On initialisation the provided workspace's directory structure is returned after selecting important files (based on .gitignore as well as a statistical approach)
+  - File edit based on search-replace tries to find correct search block if it has multiple matches based on previous search blocks. Fails otherwise (for correctness).
+  - File edit has spacing tolerant matching, with warning on issues like indentation mismatch. If there's no match, the closest match is returned to the AI to fix its mistakes.
+  - Using Aider-like search and replace, which has better performance than tool call based search and replace.
+- ⚡ **Shell optimisations**: 
+  - Only one command is allowed to be run at a time, simplifying management and avoiding rogue processes. There's only single shell instance at any point of time.
+  - Current working directory is always returned after any shell command to prevent AI from getting lost. 
+  - Command polling exits after a quick timeout to avoid slow feedback. However, status checking has wait tolerance based on fresh output streaming from a command. Both of these approach combined provides a good shell interaction experience.
+- ⚡ **Saving repo context in a single file**: Task checkpointing using "ContextSave" tool saves detailed context in a single file. Tasks can later be resumed in a new chat asking "Resume `task id`". The saved file can be used to do other kinds of knowledge transfer, such as taking help from another AI.
 
 ## Top use cases examples
 
@@ -100,6 +113,11 @@ over here
 
 Then ask claude to execute shell commands, read files, edit files, run your code, etc.
 
+#### Task checkpoint or knowledge transfer
+- You can do a task checkpoint or a knowledge transfer by attaching "KnowledgeTransfer" prompt using "Attach from MCP" button.
+- On running "KnowledgeTransfer" prompt, the "ContextSave" tool will be called saving the task description and all file content together in a single file. An id for the task will be generated.
+- You can in a new chat say "Resume '<task id>'", the AI should then call "Initialize" with the task id and load the context from there.
+- Or you can directly open the file generated and share it with another AI for help.
 
 ### [Optional] Vs code extension 
 https://marketplace.visualstudio.com/items?itemName=AmanRusia.wcgw

{wcgw-2.7.0 → wcgw-2.7.2}/gpt_action_json_schema.json

@@ -290,16 +290,16 @@
         }
       }
     },
-    "/v1/knowledge_transfer": {
+    "/v1/context_save": {
       "post": {
         "x-openai-isConsequential": false,
-        "summary": "Knowledge Transfer",
-        "operationId": "knowledge_transfer_v1_knowledge_transfer_post",
+        "summary": "Context Save",
+        "operationId": "context_save_v1_context_save_post",
         "requestBody": {
           "content": {
             "application/json": {
               "schema": {
-                "$ref": "#/components/schemas/KTWithUUID"
+                "$ref": "#/components/schemas/ContextSaveWithUUID"
               }
             }
           },
@@ -312,7 +312,7 @@
               "application/json": {
                 "schema": {
                   "type": "string",
-                  "title": "Response Knowledge Transfer V1 Knowledge Transfer Post"
+                  "title": "Response Context Save V1 Context Save Post"
                 }
               }
             }
@@ -427,6 +427,44 @@
         ],
         "title": "CommandWithUUID"
       },
+      "ContextSaveWithUUID": {
+        "properties": {
+          "id": {
+            "type": "string",
+            "title": "Id"
+          },
+          "project_root_path": {
+            "type": "string",
+            "title": "Project Root Path"
+          },
+          "description": {
+            "type": "string",
+            "title": "Description"
+          },
+          "relevant_file_globs": {
+            "items": {
+              "type": "string"
+            },
+            "type": "array",
+            "title": "Relevant File Globs"
+          },
+          "user_id": {
+            "type": "string",
+            "format": "uuid",
+            "title": "User Id"
+          }
+        },
+        "additionalProperties": false,
+        "type": "object",
+        "required": [
+          "id",
+          "project_root_path",
+          "description",
+          "relevant_file_globs",
+          "user_id"
+        ],
+        "title": "ContextSaveWithUUID"
+      },
       "FileEditWithUUID": {
         "properties": {
           "file_path": {
@@ -498,64 +536,6 @@
         ],
         "title": "InitializeWithUUID"
       },
-      "KTWithUUID": {
-        "properties": {
-          "id": {
-            "type": "string",
-            "title": "Id"
-          },
-          "project_root_path": {
-            "type": "string",
-            "title": "Project Root Path"
-          },
-          "objective": {
-            "type": "string",
-            "title": "Objective"
-          },
-          "all_user_instructions": {
-            "type": "string",
-            "title": "All User Instructions"
-          },
-          "current_status_of_the_task": {
-            "type": "string",
-            "title": "Current Status Of The Task"
-          },
-          "all_issues_snippets": {
-            "type": "string",
-            "title": "All Issues Snippets"
-          },
-          "relevant_file_paths": {
-            "items": {
-              "type": "string"
-            },
-            "type": "array",
-            "title": "Relevant File Paths"
-          },
-          "build_and_development_instructions": {
-            "type": "string",
-            "title": "Build And Development Instructions"
-          },
-          "user_id": {
-            "type": "string",
-            "format": "uuid",
-            "title": "User Id"
-          }
-        },
-        "additionalProperties": false,
-        "type": "object",
-        "required": [
-          "id",
-          "project_root_path",
-          "objective",
-          "all_user_instructions",
-          "current_status_of_the_task",
-          "all_issues_snippets",
-          "relevant_file_paths",
-          "build_and_development_instructions",
-          "user_id"
-        ],
-        "title": "KTWithUUID"
-      },
       "ReadFileWithUUID": {
         "properties": {
           "file_paths": {

{wcgw-2.7.0 → wcgw-2.7.2}/gpt_instructions.txt

@@ -44,6 +44,11 @@ Instructions for `BashInteraction`
 Instructions for `ResetShell`
 - Resets the shell. Use only if all interrupts and prompt reset attempts have failed repeatedly.
 
+Instructions for `ContextSave`
+- Saves provided description and file contents of all the relevant file paths or globs in a single text file.
+  - Provide random unqiue id or whatever user provided.
+  - Leave project path as empty string if no project path
+
 Instructions for `FileEdit`:
     - Use absolute file path only.
     - Use SEARCH/REPLACE blocks to edit the file.

{wcgw-2.7.0 → wcgw-2.7.2}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 authors = [{ name = "Aman Rusia", email = "gapypi@arcfu.com" }]
 name = "wcgw"
-version = "2.7.0"
+version = "2.7.2"
 description = "Shell and coding agent on claude and chatgpt"
 readme = "README.md"
 requires-python = ">=3.11, <3.13"

{wcgw-2.7.0 → wcgw-2.7.2}/src/wcgw/client/anthropic_client.py

@@ -25,10 +25,10 @@ from typer import Typer
 from ..types_ import (
     BashCommand,
     BashInteraction,
+    ContextSave,
     FileEdit,
     GetScreenInfo,
     Keyboard,
-    KnowledgeTransfer,
     Mouse,
     ReadFiles,
     ReadImage,
@@ -130,7 +130,12 @@ def loop(
     memory = None
     if resume:
         try:
-            memory = load_memory(resume)
+            _, memory = load_memory(
+                resume,
+                8000,
+                lambda x: default_enc.encode(x).ids,
+                lambda x: default_enc.decode(x),
+            )
         except OSError:
             if resume == "latest":
                 resume_path = sorted(Path(".wcgw").iterdir(), key=os.path.getmtime)[-1]
@@ -215,22 +220,12 @@ def loop(
 """,
         ),
         ToolParam(
-            input_schema=KnowledgeTransfer.model_json_schema(),
-            name="KnowledgeTransfer",
+            input_schema=ContextSave.model_json_schema(),
+            name="ContextSave",
             description="""
-Write detailed description in order to do a KT, if the user asks for it.
-Save all information necessary for a person to understand the task and the problems.
-
-- `all_user_instructions` should contain all instructions user shared in the conversation.
-- `current_status_of_the_task` should contain only what is already achieved, not what's remaining.
-- `all_issues_snippets` should only contain snippets of error, traceback, file snippets, commands, etc., no comments or solutions (important!).
-- Be very verbose in `all_issues_snippets` providing as much error context as possible.
-- Provide an id if the user hasn't provided one. 
-- This tool will return a text file path where the information is saved.
-- After the tool completes succesfully, tell the user the task id and the generate file path. (important!)
-- Leave arguments as empty string if they aren't relevant.
-- This tool marks end of your conversation, do not run any further tools after calling this.
-- Provide absolute file paths only in `relevant_file_paths` containing all relevant files.
+Saves provided description and file contents of all the relevant file paths or globs in a single text file.
+- Provide random unqiue id or whatever user provided.
+- Leave project path as empty string if no project path
 """,
         ),
     ]

{wcgw-2.7.0 → wcgw-2.7.2}/src/wcgw/client/mcp_server/server.py

@@ -14,6 +14,7 @@ from mcp_wcgw.types import Tool as ToolParam
 from ...types_ import (
     BashCommand,
     BashInteraction,
+    ContextSave,
     FileEdit,
     GetScreenInfo,
     Initialize,
@@ -46,14 +47,47 @@ async def handle_read_resource(uri: AnyUrl) -> str:
 
 @server.list_prompts()  # type: ignore
 async def handle_list_prompts() -> list[types.Prompt]:
-    return []
+    return [
+        types.Prompt(
+            name="KnowledgeTransfer",
+            description="Prompt for invoking ContextSave tool in order to do a comprehensive knowledge transfer of a coding task. Prompts to save detailed error log and instructions.",
+        )
+    ]
 
 
 @server.get_prompt()  # type: ignore
 async def handle_get_prompt(
     name: str, arguments: dict[str, str] | None
 ) -> types.GetPromptResult:
-    return types.GetPromptResult(messages=[])
+    messages = []
+    if name == "KnowledgeTransfer":
+        messages = [
+            types.PromptMessage(
+                role="user",
+                content=types.TextContent(
+                    type="text",
+                    text="""Use `ContextSave` tool to do a knowledge transfer of the task in hand.
+Write detailed description in order to do a KT.
+Save all information necessary for a person to understand the task and the problems.
+
+Format the `description` field using Markdown with the following sections.
+- "# Objective" section containing project and task objective.
+- "# All user instructions" section should be provided containing all instructions user shared in the conversation.
+- "# Current status of the task" should be provided containing only what is already achieved, not what's remaining.
+- "# All issues with snippets" section containing snippets of error, traceback, file snippets, commands, etc. But no comments or solutions.
+- Be very verbose in the all issues with snippets section providing as much error context as possible.
+- "# Build and development instructions" section containing instructions to build or run project or run tests, or envrionment related information. Only include what's known. Leave empty if unknown.
+- After the tool completes succesfully, tell me the task id and the file path the tool generated (important!)
+- This tool marks end of your conversation, do not run any further tools after calling this.
+
+Provide all relevant file paths in order to understand and solve the the task. Err towards providing more file paths than fewer.
+
+(Note to self: this conversation can then be resumed later asking "Resume `<generated id>`" which should call Initialize tool)
+""",
+                ),
+            )
+        ]
+    return types.GetPromptResult(messages=messages)
 
 
 @server.list_tools()  # type: ignore
@@ -153,24 +187,14 @@ async def handle_list_tools() -> list[types.Tool]:
 """
             + diffinstructions,
         ),
-        #         ToolParam(
-        #             inputSchema=KnowledgeTransfer.model_json_schema(),
-        #             name="KnowledgeTransfer",
-        #             description="""
-        # Write detailed description in order to do a KT, if the user asks for it.
-        # Save all information necessary for a person to understand the task and the problems.
-        # - `all_user_instructions` should contain all instructions user shared in the conversation.
-        # - `current_status_of_the_task` should contain only what is already achieved, not what's remaining.
-        # - `all_issues_snippets` should only contain snippets of error, traceback, file snippets, commands, etc., no comments or solutions (important!).
-        # - Be very verbose in `all_issues_snippets` providing as much error context as possible.
-        # - Provide an id if the user hasn't provided one.
-        # - This tool will return a text file path where the information is saved.
-        # - After the tool completes succesfully, tell the user the task id and the generate file path. (important!)
-        # - Leave arguments as empty string if they aren't relevant.
-        # - This tool marks end of your conversation, do not run any further tools after calling this.
-        # - Provide absolute file paths only in `relevant_file_paths` containing all relevant files.
-        # """,
-        #         ),
+        ToolParam(
+            inputSchema=ContextSave.model_json_schema(),
+            name="ContextSave",
+            description="""
+Saves provided description and file contents of all the relevant file paths or globs in a single text file.
+- Provide random unqiue id or whatever user provided.
+- Leave project path as empty string if no project path""",
+        ),
     ]
     if COMPUTER_USE_ON_DOCKER_ENABLED:
         tools += [

wcgw-2.7.2/src/wcgw/client/memory.py

@@ -0,0 +1,78 @@
+import os
+import re
+import shlex
+from typing import Callable, Optional
+
+from ..types_ import ContextSave
+
+
+def get_app_dir_xdg() -> str:
+    xdg_data_dir = os.environ.get("XDG_DATA_HOME", os.path.expanduser("~/.local/share"))
+    return os.path.join(xdg_data_dir, "wcgw")
+
+
+def format_memory(task_memory: ContextSave, relevant_files: str) -> str:
+    memory_data = ""
+    if task_memory.project_root_path:
+        memory_data += (
+            f"# PROJECT ROOT = {shlex.quote(task_memory.project_root_path)}\n"
+        )
+    memory_data += task_memory.description
+
+    memory_data += (
+        "\n\n"
+        + "# Relevant file paths\n"
+        + ", ".join(map(shlex.quote, task_memory.relevant_file_globs))
+    )
+
+    memory_data += "\n\n# Relevant Files:\n" + relevant_files
+
+    return memory_data
+
+
+def save_memory(task_memory: ContextSave, relevant_files: str) -> str:
+    app_dir = get_app_dir_xdg()
+    memory_dir = os.path.join(app_dir, "memory")
+    os.makedirs(memory_dir, exist_ok=True)
+
+    task_id = task_memory.id
+    if not task_id:
+        raise Exception("Task id can not be empty")
+    memory_data = format_memory(task_memory, relevant_files)
+
+    memory_file_full = os.path.join(memory_dir, f"{task_id}.txt")
+
+    with open(memory_file_full, "w") as f:
+        f.write(memory_data)
+
+    return memory_file_full
+
+
+def load_memory[T](
+    task_id: str,
+    max_tokens: Optional[int],
+    encoder: Callable[[str], list[T]],
+    decoder: Callable[[list[T]], str],
+) -> tuple[str, str]:
+    app_dir = get_app_dir_xdg()
+    memory_dir = os.path.join(app_dir, "memory")
+    memory_file = os.path.join(memory_dir, f"{task_id}.txt")
+
+    with open(memory_file, "r") as f:
+        data = f.read()
+
+    if max_tokens:
+        toks = encoder(data)
+        if len(toks) > max_tokens:
+            toks = toks[: max(0, max_tokens - 10)]
+            data = decoder(toks)
+            data += "\n(... truncated)"
+
+    project_root_match = re.search(r"# PROJECT ROOT = \s*(.*?)\s*$", data, re.MULTILINE)
+    project_root_path = ""
+    if project_root_match:
+        matched_path = project_root_match.group(1)
+        parsed_ = shlex.split(matched_path)
+        if parsed_ and len(parsed_) == 1:
+            project_root_path = parsed_[0]
+    return project_root_path, data

{wcgw-2.7.0 → wcgw-2.7.2}/src/wcgw/client/openai_client.py

@@ -26,8 +26,8 @@ from typer import Typer
 from ..types_ import (
     BashCommand,
     BashInteraction,
+    ContextSave,
     FileEdit,
-    KnowledgeTransfer,
     ReadFiles,
     ReadImage,
     ResetShell,
@@ -39,6 +39,7 @@ from .openai_utils import get_input_cost, get_output_cost
 from .tools import (
     DoneFlag,
     ImageData,
+    default_enc,
     get_tool_output,
     initialize,
     which_tool,
@@ -124,7 +125,12 @@ def loop(
     memory = None
     if resume:
         try:
-            memory = load_memory(resume)
+            _, memory = load_memory(
+                resume,
+                8000,
+                lambda x: default_enc.encode(x).ids,
+                lambda x: default_enc.decode(x),
+            )
         except OSError:
             if resume == "latest":
                 resume_path = sorted(Path(".wcgw").iterdir(), key=os.path.getmtime)[-1]
@@ -211,22 +217,12 @@ def loop(
             description="Resets the shell. Use only if all interrupts and prompt reset attempts have failed repeatedly.",
         ),
         openai.pydantic_function_tool(
-            KnowledgeTransfer,
+            ContextSave,
             description="""
-Write detailed description in order to do a KT, if the user asks for it.
-Save all information necessary for a person to understand the task and the problems.
-
-- `all_user_instructions` should contain all instructions user shared in the conversation.
-- `current_status_of_the_task` should contain only what is already achieved, not what's remaining.
-- `all_issues_snippets` should only contain snippets of error, traceback, file snippets, commands, etc., no comments or solutions (important!).
-- Be very verbose in `all_issues_snippets` providing as much error context as possible.
-- Provide an id if the user hasn't provided one. 
-- This tool will return a text file path where the information is saved.
-- After the tool completes succesfully, tell the user the task id and the generate file path. (important!)
-- Leave arguments as empty string if they aren't relevant.
-- This tool marks end of your conversation, do not run any further tools after calling this.
-- Provide absolute file paths only in `relevant_file_paths` containing all relevant files.
-""",
+
+Saves provided description and file contents of all the relevant file paths or globs in a single text file.
+- Provide random unqiue id or whatever user provided.
+- Leave project path as empty string if no project path""",
         ),
     ]