editcodewithai 0.1.0 → 1.0.0

package/README.md

@@ -1,22 +1,17 @@
 # editcodewithai
 
-Edit Code With AI
+A lightweight, flexible library for AI-powered code editing.
 
-## A Simple Open Source AI Code Editor
+## Overview
 
-The premise of this project is to build and iterate an open source AI code editing system. The core idea is that you can feed this system a set of source code files and high level instructions, and it will return the edited code after some time. It aims to be compatible with many LLMs, and flexible enough to adapt to future models as they come out.
+`editcodewithai` is a JavaScript/TypeScript library that enables AI-powered code editing in your applications. It provides a simple interface to send code files and instructions to an LLM (Large Language Model) and receive edited code in return.
+
+The library is designed to be model-agnostic, allowing you to use any LLM provider while handling the prompt engineering, file parsing, and response processing for you.
 
 ## Installation
 
 ```bash
-# Clone the repository
-git clone https://github.com/yourusername/editcodewithai.git
-
-# Navigate to project directory
-cd editcodewithai
-
-# Install dependencies
-npm install
+npm install editcodewithai
 ```
 
 ## Usage
@@ -41,7 +36,7 @@ const myLlmFunction = async (prompt: string) => {
         model: "openai/gpt-4",
         messages: [{ role: "user", content: prompt }],
       }),
-    }
+    },
   );
 
   const data = await response.json();
@@ -66,35 +61,58 @@ const result = await performAiEdit({
   llmFunction: myLlmFunction,
   apiKey: "your-openrouter-api-key",
 });
+
+console.log(result.changedFiles);
 ```
 
-### Parameters
+## API Reference
+
+### performAiEdit(params)
 
-The `performAiEdit` function accepts the following parameters:
+The main function that processes files with an AI model and returns edited code.
 
-- `prompt`: A string containing instructions for the AI
-- `files`: A `VizFiles` object (map of file IDs to file objects)
-- `llmFunction`: A function that takes a prompt string and returns a Promise with the LLM response
-- `apiKey`: Your OpenRouter API key for retrieving cost metadata
+#### Parameters
 
-Each file in the `files` object should contain:
+| Parameter     | Type          | Description                                                       |
+| ------------- | ------------- | ----------------------------------------------------------------- |
+| `prompt`      | `string`      | Instructions for the AI on how to modify the code                 |
+| `files`       | `VizFiles`    | Object containing file information (see below)                    |
+| `llmFunction` | `LlmFunction` | Function that sends the prompt to an LLM and returns the response |
+| `apiKey`      | `string`      | OpenRouter API key for retrieving cost metadata                   |
 
-- `name`: The filename (e.g. "index.js")
+The `VizFiles` type is a map of file IDs to file objects, where each file object has:
+
+- `name`: The filename (e.g., "index.js")
 - `text`: The file contents as a string
 
-### Return Value
+The `LlmFunction` type is a function that takes a prompt string and returns a Promise with:
+
+- `content`: The LLM's response text
+- `generationId`: A unique ID for the generation (used for cost tracking)
+
+#### Return Value
 
 The function returns an object with:
 
-- `changedFiles`: Updated files with modifications
-- `openRouterGenerationId`: ID of the generation
-- `upstreamCostCents`: Cost in cents
-- `provider`: The AI provider used
-- `inputTokens`: Number of input tokens
-- `outputTokens`: Number of output tokens
-- `promptTemplateVersion`: Version of the prompt template used
+| Property                 | Type       | Description                                |
+| ------------------------ | ---------- | ------------------------------------------ |
+| `changedFiles`           | `VizFiles` | Updated files with AI modifications        |
+| `openRouterGenerationId` | `string`   | ID of the generation from the LLM provider |
+| `upstreamCostCents`      | `number`   | Cost of the API call in cents              |
+| `provider`               | `string`   | The AI provider used (e.g., "openai")      |
+| `inputTokens`            | `number`   | Number of input tokens used                |
+| `outputTokens`           | `number`   | Number of output tokens generated          |
+| `promptTemplateVersion`  | `number`   | Version of the prompt template used        |
+
+### File Operations
+
+The library handles several file operations automatically:
 
-## Similar open source projects
+- **Updating existing files**: When the AI modifies a file's content
+- **Creating new files**: When the AI suggests new files to add
+- **Deleting files**: When the AI returns empty content for a file
+
+## Similar Projects
 
 - **Aider**: An AI pair programming tool that integrates with your terminal to assist in code editing within your local git repository. [https://aider.chat/](https://aider.chat/)
 
@@ -109,3 +127,39 @@ The function returns an object with:
 - **Void**: An open-source alternative to proprietary AI code editors, offering AI-assisted coding features while prioritizing user privacy and control. [https://void.dev/](https://void.dev/)
 
 - **Cody**: An advanced AI coding assistant developed by Sourcegraph, integrating seamlessly with popular IDEs to provide features like AI-driven chat, code autocompletion, and inline editing. [https://github.com/sourcegraph/cody](https://github.com/sourcegraph/cody)
+
+## Contributing
+
+To contribute to this project:
+
+```bash
+# Clone the repository
+git clone https://github.com/yourusername/editcodewithai.git
+
+# Navigate to project directory
+cd editcodewithai
+
+# Install dependencies
+npm install
+```
+
+Run tests to ensure everything is working correctly:
+
+```bash
+npm test
+```
+
+Please submit pull requests with clear descriptions of changes and ensure all tests pass. Protocol for wrapping up a PR:
+
+```
+npm test
+npm run typecheck
+npm run prettier
+# Verify the README is up to date
+```
+
+Please create an issue first before creating a PR to discuss the changes you want to make. This helps ensure that your contributions align with the project's goals and vision.
+
+## License
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.

package/dist/fileUtils.d.ts

@@ -1,4 +1,4 @@
-import { VizFiles, VizFile } from "@vizhub/viz-types";
+import { VizFiles, VizFile, FileCollection } from "@vizhub/viz-types";
 /**
  * If the LLM outputs empty text for a file, we interpret this
  * as a request to delete the file.
@@ -7,14 +7,8 @@ export declare function shouldDeleteFile(file?: VizFile): boolean;
 /**
  * Processes files for the prompt by truncating large files
  */
-export declare function prepareFilesForPrompt(files: VizFiles): {
-    name: string;
-    text: string;
-}[];
+export declare function prepareFilesForPrompt(files: VizFiles): FileCollection;
 /**
  * Merges original files with changes from the LLM
  */
-export declare function mergeFileChanges(originalFiles: VizFiles, parsedFiles: {
-    name: string;
-    text: string;
-}[]): VizFiles;
+export declare function mergeFileChanges(originalFiles: VizFiles, parsedFiles: FileCollection): VizFiles;

package/dist/fileUtils.js

@@ -17,15 +17,16 @@ function shouldDeleteFile(file) {
  * Processes files for the prompt by truncating large files
  */
 function prepareFilesForPrompt(files) {
-    return Object.values(files).map((file) => ({
-        name: file.name,
+    const result = {};
+    Object.values(files).forEach((file) => {
         // Example: truncate large files, etc.
-        text: file.text
+        result[file.name] = file.text
             .split("\n")
             .slice(0, file.name.endsWith(".csv") || file.name.endsWith(".json") ? 50 : 500)
             .map((line) => line.slice(0, 200))
-            .join("\n"),
-    }));
+            .join("\n");
+    });
+    return result;
 }
 /**
  * Merges original files with changes from the LLM
@@ -34,7 +35,10 @@ function mergeFileChanges(originalFiles, parsedFiles) {
     // Start with existing files
     let changedFiles = Object.keys(originalFiles).reduce((acc, fileId) => {
         const original = originalFiles[fileId];
-        const changedFile = parsedFiles.find((f) => f.name === original.name);
+        const changedText = parsedFiles[original.name];
+        const changedFile = changedText !== undefined
+            ? { name: original.name, text: changedText }
+            : undefined;
         if (shouldDeleteFile(changedFile)) {
             // Exclude from new set
             return acc;
@@ -47,14 +51,14 @@ function mergeFileChanges(originalFiles, parsedFiles) {
         return acc;
     }, {});
     // Handle newly-created files
-    parsedFiles.forEach((changedFile) => {
-        const existingFile = Object.values(changedFiles).find((file) => file.name === changedFile.name);
+    Object.entries(parsedFiles).forEach(([fileName, fileText]) => {
+        const existingFile = Object.values(changedFiles).find((file) => file.name === fileName);
         // If no existing file and not empty => it's a new file
-        if (!existingFile && !shouldDeleteFile(changedFile)) {
+        if (!existingFile && fileText.trim() !== "") {
             const newFileId = (0, viz_utils_1.generateVizFileId)();
             changedFiles[newFileId] = {
-                name: changedFile.name,
-                text: changedFile.text,
+                name: fileName,
+                text: fileText,
             };
         }
     });

package/dist/index.js

@@ -14,9 +14,9 @@ const debug = false;
  * - Retrieving cost metadata
  */
 async function performAiEdit({ prompt, files, llmFunction, apiKey, }) {
-    // 1. Serialize the existing files into the "markdown code block" format
+    // 1. Format the existing files into the "markdown code block" format
     const preparedFiles = (0, fileUtils_1.prepareFilesForPrompt)(files);
-    const filesContext = (0, llm_code_format_1.serializeMarkdownFiles)(preparedFiles);
+    const filesContext = (0, llm_code_format_1.formatMarkdownFiles)(preparedFiles);
     // 2. Assemble the final prompt
     const fullPrompt = (0, prompt_1.assembleFullPrompt)({ filesContext, prompt });
     debug && console.log("[performAiEdit] fullPrompt:", fullPrompt);

package/dist/types.d.ts

@@ -8,14 +8,15 @@ export interface PerformAiEditParams {
     files: VizFiles;
     llmFunction: LlmFunction;
     apiKey?: string;
+    baseURL?: string;
 }
 export interface PerformAiEditResult {
     changedFiles: VizFiles;
-    openRouterGenerationId: string;
-    upstreamCostCents: number;
-    provider: string;
-    inputTokens: number;
-    outputTokens: number;
-    promptTemplateVersion: number;
-    rawResponse: string;
+    openRouterGenerationId?: string;
+    upstreamCostCents?: number;
+    provider?: string;
+    inputTokens?: number;
+    outputTokens?: number;
+    promptTemplateVersion?: number;
+    rawResponse?: string;
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "editcodewithai",
-  "version": "0.1.0",
+  "version": "1.0.0",
   "description": "Edit Code With AI",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
@@ -13,7 +13,11 @@
     "build": "tsc",
     "prepublishOnly": "npm run build",
     "typecheck": "tsc --noEmit",
-    "benchmark": "ts-node src/benchmarks/index.ts"
+    "benchmark": "ts-node src/benchmarks/cli.ts run; cp -r ./benchmarks grader-app/public",
+    "grade": "ts-node src/benchmarks/cli.ts grade",
+    "benchmark:help": "ts-node src/benchmarks/cli.ts help",
+    "upgrade": "ncu -u",
+    "prettier": "prettier {*.*,**/*.*} --write"
   },
   "repository": {
     "type": "git",
@@ -37,15 +41,23 @@
   "homepage": "https://github.com/vizhub-core/editcodewithai#readme",
   "dependencies": {
     "@langchain/core": "^0.3.43",
-    "@langchain/openai": "^0.5.2",
-    "@vizhub/viz-types": "^0.0.2",
-    "@vizhub/viz-utils": "^0.0.1",
+    "@langchain/openai": "^0.5.4",
+    "@types/d3": "^7.4.3",
+    "@vizhub/viz-types": "^0.1.0",
+    "@vizhub/viz-utils": "^0.1.0",
+    "d3": "^7.9.0",
     "dotenv": "^16.4.7",
-    "llm-code-format": "^0.2.2"
+    "langchain": "^0.3.20",
+    "llm-code-format": "^2.0.1"
   },
   "devDependencies": {
+    "cors": "^2.8.5",
+    "express": "^5.1.0",
+    "npm-check-updates": "^17.1.16",
+    "prettier": "^3.5.3",
+    "puppeteer": "^24.6.0",
     "ts-node": "^10.9.2",
-    "typescript": "^5.8.2",
+    "typescript": "^5.8.3",
     "vitest": "^3.1.1"
   }
 }

package/dist/benchmark.d.ts

@@ -1,37 +0,0 @@
-import { LlmFunction } from "./types";
-/**
- * Creates an OpenRouter LLM function with the provided API key and model using LangChain
- * @param apiKey OpenRouter API key
- * @param model Model to use (defaults to "anthropic/claude-3.5-sonnet")
- * @returns LLM function that can be used with performAiEdit
- */
-export declare function createOpenRouterLlmFunction(apiKey: string, model?: string): LlmFunction;
-/**
- * Runs a benchmark test using OpenRouter to implement an "add" function
- * @param apiKey OpenRouter API key
- * @param model Optional model to use (defaults to "openai/gpt-4")
- * @returns Benchmark results including the implementation and performance metrics
- */
-export declare function runAddFunctionBenchmark(apiKey: string, model?: string): Promise<{
-    success: boolean;
-    implementation: any;
-    isCorrect: boolean;
-    testOutput: any;
-    elapsedTime: number;
-    costCents: any;
-    provider: any;
-    inputTokens: any;
-    outputTokens: any;
-    error?: undefined;
-} | {
-    success: boolean;
-    error: string;
-    implementation?: undefined;
-    isCorrect?: undefined;
-    testOutput?: undefined;
-    elapsedTime?: undefined;
-    costCents?: undefined;
-    provider?: undefined;
-    inputTokens?: undefined;
-    outputTokens?: undefined;
-}>;

package/dist/benchmark.js

@@ -1,123 +0,0 @@
-import { performAiEdit } from "./index";
-import { ChatOpenAI } from "@langchain/openai";
-import { StringOutputParser } from "@langchain/core/output_parsers";
-/**
- * Creates an OpenRouter LLM function with the provided API key and model using LangChain
- * @param apiKey OpenRouter API key
- * @param model Model to use (defaults to "anthropic/claude-3.5-sonnet")
- * @returns LLM function that can be used with performAiEdit
- */
-export function createOpenRouterLlmFunction(apiKey, model = "anthropic/claude-3.5-sonnet") {
-    return async (prompt) => {
-        try {
-            const options = {
-                modelName: model,
-                configuration: {
-                    apiKey,
-                    baseURL: "https://openrouter.ai/api/v1",
-                },
-                streaming: false,
-            };
-            const chatModel = new ChatOpenAI(options);
-            const result = await chatModel.invoke(prompt);
-            const parser = new StringOutputParser();
-            const resultString = await parser.invoke(result);
-            return {
-                content: resultString,
-                generationId: Date.now().toString(), // OpenRouter doesn't return an ID through LangChain
-            };
-        }
-        catch (error) {
-            if (error instanceof Error) {
-                throw new Error(`OpenRouter API error: ${error.message}`);
-            }
-            else {
-                throw new Error(`OpenRouter API error: ${String(error)}`);
-            }
-        }
-    };
-}
-/**
- * Runs a benchmark test using OpenRouter to implement an "add" function
- * @param apiKey OpenRouter API key
- * @param model Optional model to use (defaults to "openai/gpt-4")
- * @returns Benchmark results including the implementation and performance metrics
- */
-export async function runAddFunctionBenchmark(apiKey, model = "openai/gpt-4") {
-    console.log(`Running benchmark with model: ${model}`);
-    // Create test file with empty add function
-    const files = {
-        file1: {
-            name: "index.js",
-            text: "function add(a, b) {\n  // TODO: Implement this function\n}\n\nmodule.exports = { add };",
-        },
-    };
-    const prompt = "Implement the 'add' function to add two numbers together and return the result.";
-    // Create LLM function
-    const llmFunction = createOpenRouterLlmFunction(apiKey, model);
-    console.log("Sending request to OpenRouter...");
-    const startTime = Date.now();
-    try {
-        // Perform the AI edit
-        const result = await performAiEdit({
-            prompt,
-            files,
-            llmFunction,
-            apiKey,
-        });
-        const endTime = Date.now();
-        const elapsedTime = (endTime - startTime) / 1000;
-        // Extract the implementation
-        const implementation = result.changedFiles.file1?.text || "";
-        // Validate the implementation
-        let isCorrect = false;
-        let testOutput = null;
-        try {
-            // Create a function from the implementation to test it
-            const funcStr = implementation + "\nreturn add(3, 4);";
-            const testFunc = new Function(funcStr);
-            testOutput = testFunc();
-            isCorrect = testOutput === 7;
-        }
-        catch (error) {
-            console.error("Error testing implementation:", error);
-        }
-        return {
-            success: true,
-            implementation,
-            isCorrect,
-            testOutput,
-            elapsedTime,
-            costCents: result.upstreamCostCents,
-            provider: result.provider,
-            inputTokens: result.inputTokens,
-            outputTokens: result.outputTokens,
-        };
-    }
-    catch (error) {
-        console.error("Benchmark failed:", error);
-        return {
-            success: false,
-            error: error instanceof Error ? error.message : String(error),
-        };
-    }
-}
-/**
- * Example usage of the benchmark function
- */
-// Check if this file is being executed directly (CommonJS approach)
-if (require.main === module) {
-    // This code runs when the file is executed directly
-    const apiKey = process.env.OPENROUTER_API_KEY;
-    if (!apiKey) {
-        console.error("Please set the OPENROUTER_API_KEY environment variable");
-        process.exit(1);
-    }
-    runAddFunctionBenchmark(apiKey)
-        .then((result) => {
-        console.log("Benchmark result:", JSON.stringify(result, null, 2));
-    })
-        .catch((error) => {
-        console.error("Error running benchmark:", error);
-    });
-}

package/dist/benchmarks/benchmarkRunner.d.ts

@@ -1,5 +0,0 @@
-/**
- * Main function: runs the challenges on different models, calls the AI to fill them in,
- * executes each test, and writes pass/fail results to CSV.
- */
-export declare function runAllChallenges(): Promise<void>;

package/dist/benchmarks/benchmarkRunner.js

@@ -1,85 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.runAllChallenges = runAllChallenges;
-const file_system_1 = require("langchain/cache/file_system");
-const index_1 = require("../index");
-const llm_1 = require("./llm");
-const fileUtils_1 = require("./fileUtils");
-const runner_1 = require("./runner");
-const challenges_1 = require("./challenges");
-const models_1 = require("./models");
-/**
- * Main function: runs the challenges on different models, calls the AI to fill them in,
- * executes each test, and writes pass/fail results to CSV.
- */
-async function runAllChallenges() {
-    const apiKey = process.env.OPENROUTER_API_KEY || "";
-    if (!apiKey) {
-        throw new Error("Please set OPENROUTER_API_KEY in your .env");
-    }
-    // Setup cache
-    const cacheDirBase = "benchmarks/llmResponseCache";
-    (0, fileUtils_1.ensureCacheDir)(cacheDirBase);
-    const cache = await file_system_1.LocalFileCache.create(cacheDirBase);
-    // Store results
-    const results = [];
-    // Run each model against each challenge
-    for (const model of models_1.models) {
-        try {
-            await runModelChallenges(model, apiKey, cache, results);
-        }
-        catch (error) {
-            console.error(`Error running model ${model}:`, error);
-            // Add failure results for remaining challenges
-            for (const challenge of challenges_1.challenges) {
-                results.push({
-                    challenge: challenge.name,
-                    model,
-                    passFail: "error",
-                });
-            }
-        }
-    }
-    // Write results to CSV
-    const csv = (0, fileUtils_1.writeResultsToCsv)(results, "benchmarks/results.csv");
-    console.log("\nAll challenges completed. See 'benchmarks/results.csv' for summary.\n");
-    console.log(csv);
-}
-/**
- * Runs all challenges for a specific model
- */
-async function runModelChallenges(model, apiKey, cache, results) {
-    // Create the LLM function for this model
-    const llmFunction = (0, llm_1.createOpenRouterLlmFunction)(model, apiKey, cache);
-    // Run each challenge for this model
-    for (const challenge of challenges_1.challenges) {
-        console.log(`\n=== Challenge: ${challenge.name} | Model: ${model} ===`);
-        try {
-            // Ask AI to fill out the placeholder TODOs
-            const aiResult = await (0, index_1.performAiEdit)({
-                prompt: challenge.prompt,
-                files: challenge.files,
-                llmFunction,
-            });
-            // Write returned files to disk, including LLM response
-            const challengeDir = (0, fileUtils_1.writeChallengeFiles)(challenge.name, model, aiResult.changedFiles, aiResult.rawResponse);
-            // Run index.js in a child process
-            const exitCode = (0, runner_1.runNodeTest)(challengeDir);
-            const passFail = exitCode === 0 ? "pass" : "fail";
-            // Record the result
-            results.push({
-                challenge: challenge.name,
-                model,
-                passFail,
-            });
-        }
-        catch (error) {
-            console.error(`Error running challenge ${challenge.name}:`, error);
-            results.push({
-                challenge: challenge.name,
-                model,
-                passFail: "error",
-            });
-        }
-    }
-}

package/dist/benchmarks/challenges.d.ts

@@ -1,2 +0,0 @@
-import { Challenge } from "./types";
-export declare const challenges: Challenge[];

package/dist/benchmarks/challenges.js

@@ -1,158 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.challenges = void 0;
-// Challenges definition. Each has multiple files (some with TODO).
-// "index.mjs" includes the test code that calls process.exit(1) if it fails.
-exports.challenges = [
-    {
-        name: "add",
-        prompt: "Implement the 'add' function to correctly add two numbers (a+b) and pass the test in index.mjs.",
-        files: {
-            file1: {
-                name: "index.mjs",
-                text: `
-import { add } from "./functions.mjs";
-
-// A simple test:
-const result = add(3, 4);
-if (result !== 7) {
-  console.error("Test failed: expected 7, got", result);
-  process.exit(1);
-}
-console.log("Add test passed");
-process.exit(0);
-        `,
-            },
-            file2: {
-                name: "functions.mjs",
-                text: `
-// TODO: Implement the add function
-export function add(a, b) {
-  // TODO
-}
-        `,
-            },
-        },
-    },
-    {
-        name: "multiply",
-        prompt: "Implement the 'multiply' function to correctly multiply two numbers and pass the unit test in index.mjs.",
-        files: {
-            file1: {
-                name: "index.mjs",
-                text: `
-import { multiply } from "./functions.mjs";
-
-const result = multiply(6, 7);
-if (result !== 42) {
-  console.error("Test failed: expected 42, got", result);
-  process.exit(1);
-}
-console.log("Multiply test passed");
-process.exit(0);
-      `,
-            },
-            file2: {
-                name: "functions.mjs",
-                text: `
-// TODO: Implement the multiply function
-export function multiply(a, b) {
-  // TODO
-}
-      `,
-            },
-        },
-    },
-    {
-        name: "square",
-        prompt: "Implement the 'square' function that returns x*x.",
-        files: {
-            file1: {
-                name: "index.mjs",
-                text: `
-import { square } from "./functions.mjs";
-
-const input = 5;
-const result = square(input);
-if (result !== 25) {
-  console.error("Test failed: expected 25, got", result);
-  process.exit(1);
-}
-console.log("Square test passed");
-process.exit(0);
-      `,
-            },
-            file2: {
-                name: "functions.mjs",
-                text: `
-// TODO: Implement the square function
-export function square(x) {
-  // TODO
-}
-      `,
-            },
-        },
-    },
-    {
-        name: "toUpperCase",
-        prompt: "Implement the toUpperCase function that returns the given string in uppercase.",
-        files: {
-            file1: {
-                name: "index.mjs",
-                text: `
-import { toUpperCase } from "./functions.mjs";
-
-const input = "hello";
-const result = toUpperCase(input);
-if (result !== "HELLO") {
-  console.error("Test failed: expected 'HELLO', got", result);
-  process.exit(1);
-}
-console.log("toUpperCase test passed");
-process.exit(0);
-      `,
-            },
-            file2: {
-                name: "functions.mjs",
-                text: `
-// TODO: Implement the toUpperCase function
-export function toUpperCase(str) {
-  // TODO
-}
-      `,
-            },
-        },
-    },
-    {
-        name: "reverseString",
-        prompt: "Implement the reverseString function that reverses the given string.",
-        files: {
-            file1: {
-                name: "index.mjs",
-                text: `
-import { reverseString } from "./functions.mjs";
-
-const input = "OpenAI";
-const expected = "IAnepO";
-
-const result = reverseString(input);
-if (result !== expected) {
-  console.error("Test failed: expected", expected, "but got", result);
-  process.exit(1);
-}
-console.log("reverseString test passed");
-process.exit(0);
-      `,
-            },
-            file2: {
-                name: "functions.mjs",
-                text: `
-// TODO: Implement the reverseString function
-export function reverseString(str) {
-  // TODO
-}
-      `,
-            },
-        },
-    },
-];

package/dist/benchmarks/fileUtils.d.ts

@@ -1,15 +0,0 @@
-import { VizFiles } from "@vizhub/viz-types";
-import { ChallengeResult } from "./types";
-/**
- * Writes the files for a given challenge into a folder named after challenge.name + model
- * e.g. "benchmarks/challenges/add/gpt4" to keep them separate per model.
- */
-export declare function writeChallengeFiles(challengeName: string, model: string, changedFiles: VizFiles, llmResponse?: string): string;
-/**
- * Writes results to a CSV file
- */
-export declare function writeResultsToCsv(results: ChallengeResult[], filePath?: string): string;
-/**
- * Ensures a cache directory exists
- */
-export declare function ensureCacheDir(cacheDirBase: string): void;

package/dist/benchmarks/fileUtils.js

@@ -1,55 +0,0 @@
-"use strict";
-var __importDefault = (this && this.__importDefault) || function (mod) {
-    return (mod && mod.__esModule) ? mod : { "default": mod };
-};
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.writeChallengeFiles = writeChallengeFiles;
-exports.writeResultsToCsv = writeResultsToCsv;
-exports.ensureCacheDir = ensureCacheDir;
-const fs_1 = __importDefault(require("fs"));
-const path_1 = __importDefault(require("path"));
-/**
- * Writes the files for a given challenge into a folder named after challenge.name + model
- * e.g. "benchmarks/challenges/add/gpt4" to keep them separate per model.
- */
-function writeChallengeFiles(challengeName, model, changedFiles, llmResponse) {
-    const challengeDir = path_1.default.join("benchmarks", "challenges", challengeName, model);
-    if (!fs_1.default.existsSync(challengeDir)) {
-        fs_1.default.mkdirSync(challengeDir, { recursive: true });
-    }
-    Object.keys(changedFiles).forEach((fileKey) => {
-        const { name, text } = changedFiles[fileKey];
-        const filePath = path_1.default.join(challengeDir, name);
-        fs_1.default.writeFileSync(filePath, text, "utf-8");
-    });
-    // Write the LLM response to a file if provided
-    if (llmResponse) {
-        const responsePath = path_1.default.join(challengeDir, "llmResponse.md");
-        fs_1.default.writeFileSync(responsePath, llmResponse, "utf-8");
-    }
-    return challengeDir;
-}
-/**
- * Writes results to a CSV file
- */
-function writeResultsToCsv(results, filePath = "benchmarks/results.csv") {
-    let csv = "challenge,model,passFail\n";
-    for (const r of results) {
-        csv += `${r.challenge},${r.model},${r.passFail}\n`;
-    }
-    // Ensure benchmarks directory exists
-    const dir = path_1.default.dirname(filePath);
-    if (!fs_1.default.existsSync(dir)) {
-        fs_1.default.mkdirSync(dir, { recursive: true });
-    }
-    fs_1.default.writeFileSync(filePath, csv, "utf-8");
-    return csv;
-}
-/**
- * Ensures a cache directory exists
- */
-function ensureCacheDir(cacheDirBase) {
-    if (!fs_1.default.existsSync(cacheDirBase)) {
-        fs_1.default.mkdirSync(cacheDirBase, { recursive: true });
-    }
-}

package/dist/benchmarks/index.d.ts

@@ -1,5 +0,0 @@
-export { runAllChallenges } from "./benchmarkRunner";
-export { challenges } from "./challenges";
-export { createOpenRouterLlmFunction } from "./llm";
-export { runNodeTest } from "./runner";
-export { writeChallengeFiles, writeResultsToCsv } from "./fileUtils";

package/dist/benchmarks/index.js

@@ -1,44 +0,0 @@
-"use strict";
-var __importDefault = (this && this.__importDefault) || function (mod) {
-    return (mod && mod.__esModule) ? mod : { "default": mod };
-};
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.writeResultsToCsv = exports.writeChallengeFiles = exports.runNodeTest = exports.createOpenRouterLlmFunction = exports.challenges = exports.runAllChallenges = void 0;
-/**********************************************************************
- * benchmark.ts
- * --------------------------------------------------------------------
- * Requires environment variables to be set in a .env file
- * --------------------------------------------------------------------
- * Defines 5 challenges, each with index.js (ES module) that performs
- * a simple unit test. The code calls process.exit(0) on success or
- * process.exit(1) on failure.
- *
- * Runs all challenges on 5 different models:
- *    ["gpt3", "gpt4", "gpt5", "deepseek", "qwen32"]
- *
- * Writes final results to "results.csv" with columns:
- *    challenge, model, passFail, exitCode
- **********************************************************************/
-const dotenv_1 = __importDefault(require("dotenv"));
-const benchmarkRunner_1 = require("./benchmarkRunner");
-// Load environment variables from .env file
-dotenv_1.default.config();
-// Run if invoked directly
-if (require.main === module) {
-    (0, benchmarkRunner_1.runAllChallenges)().catch((error) => {
-        console.error("Error running challenges:", error);
-        process.exit(1);
-    });
-}
-// Re-export for programmatic usage
-var benchmarkRunner_2 = require("./benchmarkRunner");
-Object.defineProperty(exports, "runAllChallenges", { enumerable: true, get: function () { return benchmarkRunner_2.runAllChallenges; } });
-var challenges_1 = require("./challenges");
-Object.defineProperty(exports, "challenges", { enumerable: true, get: function () { return challenges_1.challenges; } });
-var llm_1 = require("./llm");
-Object.defineProperty(exports, "createOpenRouterLlmFunction", { enumerable: true, get: function () { return llm_1.createOpenRouterLlmFunction; } });
-var runner_1 = require("./runner");
-Object.defineProperty(exports, "runNodeTest", { enumerable: true, get: function () { return runner_1.runNodeTest; } });
-var fileUtils_1 = require("./fileUtils");
-Object.defineProperty(exports, "writeChallengeFiles", { enumerable: true, get: function () { return fileUtils_1.writeChallengeFiles; } });
-Object.defineProperty(exports, "writeResultsToCsv", { enumerable: true, get: function () { return fileUtils_1.writeResultsToCsv; } });

package/dist/benchmarks/llm.d.ts

@@ -1,10 +0,0 @@
-import { LlmFunction } from "../types";
-import { LocalFileCache } from "langchain/cache/file_system";
-/**
- * Creates an LLM function that connects to OpenRouter
- * @param model The model identifier to use
- * @param apiKey OpenRouter API key
- * @param cache Optional local file cache to avoid duplicate requests
- * @returns A function that can be used with performAiEdit
- */
-export declare function createOpenRouterLlmFunction(model: string, apiKey: string, cache?: LocalFileCache): LlmFunction;

package/dist/benchmarks/llm.js

@@ -1,29 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.createOpenRouterLlmFunction = createOpenRouterLlmFunction;
-const openai_1 = require("@langchain/openai");
-const output_parsers_1 = require("@langchain/core/output_parsers");
-/**
- * Creates an LLM function that connects to OpenRouter
- * @param model The model identifier to use
- * @param apiKey OpenRouter API key
- * @param cache Optional local file cache to avoid duplicate requests
- * @returns A function that can be used with performAiEdit
- */
-function createOpenRouterLlmFunction(model, apiKey, cache) {
-    return async (prompt) => {
-        // Create OpenAI chat model with OpenRouter configuration
-        const chatModel = new openai_1.ChatOpenAI({
-            modelName: model,
-            configuration: { apiKey, baseURL: "https://openrouter.ai/api/v1" },
-            streaming: false,
-            cache,
-        });
-        // Invoke the model
-        const result = await chatModel.invoke(prompt);
-        // Parse to string
-        const parser = new output_parsers_1.StringOutputParser();
-        const resultString = await parser.invoke(result);
-        return { content: resultString, generationId: result.lc_kwargs.id };
-    };
-}

package/dist/benchmarks/models.d.ts

@@ -1 +0,0 @@
-export declare const models: string[];

package/dist/benchmarks/models.js

@@ -1,21 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.models = void 0;
-exports.models = [
-    "anthropic/claude-3.7-sonnet",
-    "anthropic/claude-3.7-sonnet:thinking",
-    "google/gemini-2.0-flash-001",
-    "anthropic/claude-3.5-sonnet",
-    "deepseek/deepseek-r1",
-    "openai/o3-mini-high",
-    "openai/o3-mini",
-    "deepseek/deepseek-chat",
-    "qwen/qwen-2.5-72b-instruct",
-    "qwen/qwq-32b",
-    "anthropic/claude-3.5-haiku",
-    "qwen/qwen-2.5-7b-instruct",
-    "amazon/nova-micro-v1",
-    "meta-llama/llama-3.1-8b-instruct",
-    "meta-llama/llama-3.2-3b-instruct",
-    "meta-llama/llama-3.2-1b-instruct",
-];

package/dist/benchmarks/runner.d.ts

@@ -1,4 +0,0 @@
-/**
- * Runs "index.mjs" for a given challenge directory with Node and returns exit code.
- */
-export declare function runNodeTest(challengeDir: string): number;

package/dist/benchmarks/runner.js

@@ -1,35 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.runNodeTest = runNodeTest;
-const child_process_1 = require("child_process");
-/**
- * Runs "index.mjs" for a given challenge directory with Node and returns exit code.
- */
-function runNodeTest(challengeDir) {
-    const result = runNodeProcess(challengeDir);
-    logTestResult(challengeDir, result);
-    return result.status ?? 1; // If status is null, consider that a failure
-}
-/**
- * Executes the Node.js process for a test
- */
-function runNodeProcess(challengeDir) {
-    const child = (0, child_process_1.spawnSync)("node", ["index.mjs"], {
-        cwd: challengeDir,
-        encoding: "utf-8",
-    });
-    return {
-        status: child.status,
-        stdout: child.stdout,
-        stderr: child.stderr
-    };
-}
-/**
- * Logs the test result to the console
- */
-function logTestResult(challengeDir, result) {
-    console.log(`--- Running test in ${challengeDir} ---`);
-    console.log("stdout:\n", result.stdout.trim());
-    console.log("stderr:\n", result.stderr.trim());
-    console.log("exit code:", result.status, "\n");
-}

package/dist/benchmarks/types.d.ts

@@ -1,16 +0,0 @@
-import { VizFiles } from "@vizhub/viz-types";
-export interface Challenge {
-    name: string;
-    prompt: string;
-    files: VizFiles;
-}
-export interface ChallengeResult {
-    challenge: string;
-    model: string;
-    passFail: "pass" | "fail" | "error";
-}
-export interface TestRunResult {
-    status: number | null;
-    stdout: string;
-    stderr: string;
-}

package/dist/benchmarks/types.js

@@ -1,2 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });

package/dist/cli.d.ts

@@ -1,2 +0,0 @@
-#!/usr/bin/env node
-export {};

package/dist/cli.js

@@ -1,157 +0,0 @@
-#!/usr/bin/env node
-"use strict";
-var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
-    if (k2 === undefined) k2 = k;
-    var desc = Object.getOwnPropertyDescriptor(m, k);
-    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
-      desc = { enumerable: true, get: function() { return m[k]; } };
-    }
-    Object.defineProperty(o, k2, desc);
-}) : (function(o, m, k, k2) {
-    if (k2 === undefined) k2 = k;
-    o[k2] = m[k];
-}));
-var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
-    Object.defineProperty(o, "default", { enumerable: true, value: v });
-}) : function(o, v) {
-    o["default"] = v;
-});
-var __importStar = (this && this.__importStar) || (function () {
-    var ownKeys = function(o) {
-        ownKeys = Object.getOwnPropertyNames || function (o) {
-            var ar = [];
-            for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
-            return ar;
-        };
-        return ownKeys(o);
-    };
-    return function (mod) {
-        if (mod && mod.__esModule) return mod;
-        var result = {};
-        if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
-        __setModuleDefault(result, mod);
-        return result;
-    };
-})();
-var __importDefault = (this && this.__importDefault) || function (mod) {
-    return (mod && mod.__esModule) ? mod : { "default": mod };
-};
-Object.defineProperty(exports, "__esModule", { value: true });
-const commander_1 = require("commander");
-const path = __importStar(require("path"));
-const readline = __importStar(require("readline"));
-const index_1 = require("./index");
-const llm_1 = require("./benchmarks/llm");
-const computeInitialDocument_1 = require("./computeInitialDocument");
-const dotenv_1 = __importDefault(require("dotenv"));
-const fs = __importStar(require("fs/promises"));
-// Load environment variables
-dotenv_1.default.config();
-const program = new commander_1.Command();
-program
-    .name("editcode")
-    .description("Edit code files using AI")
-    .option("-p, --prompt <prompt>", "The instruction for editing the code")
-    .option("-d, --dir <directory>", "Directory to process", process.cwd())
-    .option("--dry-run", "Show changes without writing them", false)
-    .option("--model <model>", "OpenRouter model to use", "anthropic/claude-3.7-sonnet")
-    .action(async (options) => {
-    try {
-        // Get prompt from user if not provided as argument
-        if (!options.prompt) {
-            const rl = readline.createInterface({
-                input: process.stdin,
-                output: process.stdout
-            });
-            options.prompt = await new Promise((resolve) => {
-                rl.question("> ", (answer) => {
-                    rl.close();
-                    resolve(answer);
-                });
-            });
-        }
-        if (!options.prompt.trim()) {
-            console.error("Error: Prompt cannot be empty");
-            process.exit(1);
-        }
-        // Debug: Log environment
-        console.log('Environment variables:', {
-            OPENROUTER_API_KEY: process.env.OPENROUTER_API_KEY ? '[PRESENT]' : '[MISSING]',
-            NODE_ENV: process.env.NODE_ENV,
-            PATH: process.env.PATH?.substring(0, 50) + '...'
-        });
-        // Validate API key
-        const apiKey = process.env.OPENROUTER_API_KEY;
-        if (!apiKey) {
-            console.error("Error: OPENROUTER_API_KEY environment variable is required");
-            process.exit(1);
-        }
-        // Get absolute path
-        const fullPath = path.resolve(options.dir);
-        console.log(`Processing directory: ${fullPath}`);
-        // Use computeInitialDocument to get files (respects .ignore files)
-        const initialDocument = (0, computeInitialDocument_1.computeInitialDocument)({ fullPath });
-        // Convert to VizFiles format
-        const vizFiles = {};
-        Object.entries(initialDocument.files).forEach(([id, file]) => {
-            if (file.text !== null) {
-                // Skip directories
-                vizFiles[id] = {
-                    name: path.relative(process.cwd(), path.join(fullPath, file.name)),
-                    text: file.text,
-                };
-            }
-        });
-        const fileCount = Object.keys(vizFiles).length;
-        if (fileCount === 0) {
-            console.error("No files found to process");
-            process.exit(1);
-        }
-        console.log(`Found ${fileCount} files to process`);
-        // Create LLM function
-        const llmFunction = (0, llm_1.createOpenRouterLlmFunction)(options.model, apiKey);
-        // Perform the edit
-        console.log("\nRequesting changes from AI...");
-        const result = await (0, index_1.performAiEdit)({
-            prompt: options.prompt,
-            files: vizFiles,
-            llmFunction,
-            apiKey,
-        });
-        // Show changes
-        console.log("\nProposed changes:");
-        for (const [id, file] of Object.entries(result.changedFiles)) {
-            const original = vizFiles[id];
-            if (!original) {
-                console.log(`\nNew file: ${file.name}`);
-                console.log(file.text);
-            }
-            else if (original.text !== file.text) {
-                console.log(`\nModified: ${file.name}`);
-                console.log(file.text);
-            }
-        }
-        // Write changes if not dry run
-        if (!options.dryRun) {
-            console.log("\nWriting changes...");
-            for (const [_, file] of Object.entries(result.changedFiles)) {
-                const fullPath = path.resolve(file.name);
-                await fs.mkdir(path.dirname(fullPath), { recursive: true });
-                await fs.writeFile(fullPath, file.text);
-                console.log(`Updated ${fullPath}`);
-            }
-        }
-        // Show metadata
-        console.log("\nMetadata:");
-        console.log(`Model: ${options.model}`);
-        console.log(`Provider: ${result.provider}`);
-        console.log(`Cost: ${result.upstreamCostCents / 100} USD`);
-        console.log(`Input tokens: ${result.inputTokens}`);
-        console.log(`Output tokens: ${result.outputTokens}`);
-    }
-    catch (error) {
-        console.error("Error:", error);
-        process.exit(1);
-    }
-});
-program.parse();

package/dist/computeInitialDocument.d.ts

@@ -1,13 +0,0 @@
-interface InitialDocument {
-    files: {
-        [key: string]: {
-            text: string | null;
-            name: string;
-        };
-    };
-}
-export declare const isDirectory: (path: string) => boolean;
-export declare const computeInitialDocument: ({ fullPath }: {
-    fullPath: string;
-}) => InitialDocument;
-export {};

package/dist/computeInitialDocument.js

@@ -1,135 +0,0 @@
-"use strict";
-var __importDefault = (this && this.__importDefault) || function (mod) {
-    return (mod && mod.__esModule) ? mod : { "default": mod };
-};
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.computeInitialDocument = exports.isDirectory = void 0;
-const fs_1 = __importDefault(require("fs"));
-const path_1 = __importDefault(require("path"));
-const ignore_1 = __importDefault(require("ignore"));
-// Configuration constants
-const ignoreFilePattern = '**/{.ignore,.gitignore}';
-const baseIgnore = [
-    '.git',
-    'node_modules',
-    'dist',
-    '*.log',
-    '.DS_Store',
-    'coverage',
-    '.env',
-    '.env.*',
-];
-const debugIgnore = false;
-const debugDirectories = false;
-const enableDirectories = true;
-/**
- * @param {string} fullPath - absolute path of the workspace root
- * @param {string} currentDirectoryPath - path where the ignore file is found, relative to fullPath
- * @param {string} fileName - name of the ignore file
- * @returns {string[]} parsed lines
- */
-const parseIgnoreFile = (fullPath, currentDirectory, fileName) => {
-    const filePath = path_1.default.join(fullPath, currentDirectory, fileName);
-    const content = fs_1.default.readFileSync(filePath, 'utf8');
-    const globs = content
-        .split(/[\n\r]+/)
-        .filter(
-    // remove blank line and comments
-    (line) => line.length > 0 && !line.startsWith('#'))
-        .map((line) => {
-        const { bang, slash, glob } = line.match(/^(?<bang>!?)(?<slash>\/?)(?<glob>.*)$/).groups;
-        const hasSlash = Boolean(slash) || /\/.*\S/.test(glob);
-        const relativeGlob = path_1.default.posix.join(currentDirectory.replace(
-        // escape characters with special meaning in glob expressions
-        /[*?!# \[\]\\]/g, (char) => '\\' + char), 
-        // a pattern that doesn't include a slash (not counting a trailing one) matches files in any descendant directory of the current one
-        hasSlash ? '' : '**', glob);
-        // preserve leading `!` and `/` characters
-        return bang + slash + relativeGlob;
-    });
-    if (debugIgnore) {
-        console.debug('at', currentDirectory, 'parsing', fileName, 'obtained globs', globs);
-    }
-    return globs;
-};
-const isDirectory = (path) => path.endsWith('/');
-exports.isDirectory = isDirectory;
-// Lists files from the file system,
-// converts them into the internal data structure.
-const computeInitialDocument = ({ fullPath }) => {
-    // Initialize the document using our data structure for representing files.
-    const initialDocument = {
-        files: {},
-    };
-    /**
-     * Stack for recursively traversing directories.
-     * @type {string[]}
-     */
-    let files = [];
-    const ignoreFileMatcher = (0, ignore_1.default)().add(ignoreFilePattern);
-    const isIgnoreFile = (fileName) => ignoreFileMatcher.ignores(fileName);
-    const unsearchedDirectories = [
-        {
-            currentDirectory: '.',
-            ignore: (0, ignore_1.default)().add(baseIgnore),
-        },
-    ];
-    while (unsearchedDirectories.length !== 0) {
-        const { currentDirectory, ignore: parentIgnore } = unsearchedDirectories.pop();
-        const currentDirectoryPath = path_1.default.join(fullPath, currentDirectory);
-        const dirEntries = fs_1.default
-            .readdirSync(currentDirectoryPath, {
-            withFileTypes: true,
-        })
-            .filter((dirent) => enableDirectories ? true : dirent.isFile());
-        // find .ignore or .gitignore files in the current directory
-        const ignoreFiles = dirEntries
-            .filter((dirent) => dirent.isFile() && isIgnoreFile(dirent.name))
-            .map((file) => file.name);
-        let ignore = parentIgnore;
-        if (ignoreFiles.length > 0) {
-            const globs = ignoreFiles.flatMap((fileName) => parseIgnoreFile(fullPath, currentDirectory, fileName));
-            ignore = (0, ignore_1.default)().add(parentIgnore).add(globs);
-        }
-        const newFiles = dirEntries
-            .filter((dirent) => {
-            const relativePath = path_1.default.posix.join(currentDirectory, dirent.name) +
-                (dirent.isDirectory() ? '/' : '');
-            const keep = !ignore.ignores(relativePath);
-            if (debugIgnore && !keep) {
-                console.debug('at', currentDirectory, 'ignoring', relativePath);
-            }
-            return keep;
-        })
-            // Add a trailing slash for directories
-            .map((dirent) => {
-            const relativePath = path_1.default.posix.join(currentDirectory, dirent.name);
-            if (!dirent.isDirectory()) {
-                return relativePath;
-            }
-            unsearchedDirectories.push({
-                currentDirectory: relativePath,
-                ignore,
-            });
-            return relativePath + '/';
-        });
-        files.push(...newFiles);
-    }
-    files.forEach((file) => {
-        const id = Math.floor(Math.random() * 10000000).toString();
-        initialDocument.files[id] = {
-            text: (0, exports.isDirectory)(file)
-                ? null
-                : fs_1.default.readFileSync(path_1.default.join(fullPath, file), 'utf-8'),
-            name: file,
-        };
-    });
-    if (debugDirectories) {
-        console.log('files:');
-        console.log(files);
-        console.log('initialDocument:');
-        console.log(JSON.stringify(initialDocument, null, 2));
-    }
-    return initialDocument;
-};
-exports.computeInitialDocument = computeInitialDocument;