editcodewithai 0.1.0 → 1.1.0

package/README.md

@@ -1,22 +1,19 @@
 # editcodewithai
 
-Edit Code With AI
+A lightweight, flexible library for AI-powered code editing.
 
-## A Simple Open Source AI Code Editor
+See also [vizhub-benchmarks](https://github.com/vizhub-core/vizhub-benchmarks).
 
-The premise of this project is to build and iterate an open source AI code editing system. The core idea is that you can feed this system a set of source code files and high level instructions, and it will return the edited code after some time. It aims to be compatible with many LLMs, and flexible enough to adapt to future models as they come out.
+## Overview
 
-## Installation
+`editcodewithai` is a JavaScript/TypeScript library that enables AI-powered code editing in your applications. It provides a simple interface to send code files and instructions to an LLM (Large Language Model) and receive edited code in return.
 
-```bash
-# Clone the repository
-git clone https://github.com/yourusername/editcodewithai.git
+The library is designed to be model-agnostic, allowing you to use any LLM provider while handling the prompt engineering, file parsing, and response processing for you.
 
-# Navigate to project directory
-cd editcodewithai
+## Installation
 
-# Install dependencies
-npm install
+```bash
+npm install editcodewithai
 ```
 
 ## Usage
@@ -38,10 +35,10 @@ const myLlmFunction = async (prompt: string) => {
         Authorization: `Bearer ${apiKey}`,
       },
       body: JSON.stringify({
-        model: "openai/gpt-4",
+        model: "anthropic/claude-3.5-sonnet",
         messages: [{ role: "user", content: prompt }],
       }),
-    }
+    },
   );
 
   const data = await response.json();
@@ -66,35 +63,58 @@ const result = await performAiEdit({
   llmFunction: myLlmFunction,
   apiKey: "your-openrouter-api-key",
 });
+
+console.log(result.changedFiles);
 ```
 
-### Parameters
+## API Reference
 
-The `performAiEdit` function accepts the following parameters:
+### performAiEdit(params)
 
-- `prompt`: A string containing instructions for the AI
-- `files`: A `VizFiles` object (map of file IDs to file objects)
-- `llmFunction`: A function that takes a prompt string and returns a Promise with the LLM response
-- `apiKey`: Your OpenRouter API key for retrieving cost metadata
+The main function that processes files with an AI model and returns edited code.
 
-Each file in the `files` object should contain:
+#### Parameters
 
-- `name`: The filename (e.g. "index.js")
+| Parameter     | Type          | Description                                                       |
+| ------------- | ------------- | ----------------------------------------------------------------- |
+| `prompt`      | `string`      | Instructions for the AI on how to modify the code                 |
+| `files`       | `VizFiles`    | Object containing file information (see below)                    |
+| `llmFunction` | `LlmFunction` | Function that sends the prompt to an LLM and returns the response |
+| `apiKey`      | `string`      | OpenRouter API key for retrieving cost metadata                   |
+
+The `VizFiles` type is a map of file IDs to file objects, where each file object has:
+
+- `name`: The filename (e.g., "index.js")
 - `text`: The file contents as a string
 
-### Return Value
+The `LlmFunction` type is a function that takes a prompt string and returns a Promise with:
+
+- `content`: The LLM's response text
+- `generationId`: A unique ID for the generation (used for cost tracking)
+
+#### Return Value
 
 The function returns an object with:
 
-- `changedFiles`: Updated files with modifications
-- `openRouterGenerationId`: ID of the generation
-- `upstreamCostCents`: Cost in cents
-- `provider`: The AI provider used
-- `inputTokens`: Number of input tokens
-- `outputTokens`: Number of output tokens
-- `promptTemplateVersion`: Version of the prompt template used
+| Property                 | Type       | Description                                |
+| ------------------------ | ---------- | ------------------------------------------ |
+| `changedFiles`           | `VizFiles` | Updated files with AI modifications        |
+| `openRouterGenerationId` | `string`   | ID of the generation from the LLM provider |
+| `upstreamCostCents`      | `number`   | Cost of the API call in cents              |
+| `provider`               | `string`   | The AI provider used (e.g., "openai")      |
+| `inputTokens`            | `number`   | Number of input tokens used                |
+| `outputTokens`           | `number`   | Number of output tokens generated          |
+| `promptTemplateVersion`  | `number`   | Version of the prompt template used        |
+
+### File Operations
 
-## Similar open source projects
+The library handles several file operations automatically:
+
+- **Updating existing files**: When the AI modifies a file's content
+- **Creating new files**: When the AI suggests new files to add
+- **Deleting files**: When the AI returns empty content for a file
+
+## Similar Projects
 
 - **Aider**: An AI pair programming tool that integrates with your terminal to assist in code editing within your local git repository. [https://aider.chat/](https://aider.chat/)
 
@@ -109,3 +129,39 @@ The function returns an object with:
 - **Void**: An open-source alternative to proprietary AI code editors, offering AI-assisted coding features while prioritizing user privacy and control. [https://void.dev/](https://void.dev/)
 
 - **Cody**: An advanced AI coding assistant developed by Sourcegraph, integrating seamlessly with popular IDEs to provide features like AI-driven chat, code autocompletion, and inline editing. [https://github.com/sourcegraph/cody](https://github.com/sourcegraph/cody)
+
+## Contributing
+
+To contribute to this project:
+
+```bash
+# Clone the repository
+git clone https://github.com/yourusername/editcodewithai.git
+
+# Navigate to project directory
+cd editcodewithai
+
+# Install dependencies
+npm install
+```
+
+Run tests to ensure everything is working correctly:
+
+```bash
+npm test
+```
+
+Please submit pull requests with clear descriptions of changes and ensure all tests pass. Protocol for wrapping up a PR:
+
+```
+npm test
+npm run typecheck
+npm run prettier
+# Verify the README is up to date
+```
+
+Please create an issue first before creating a PR to discuss the changes you want to make. This helps ensure that your contributions align with the project's goals and vision.
+
+## License
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.

package/dist/fileUtils.d.ts

@@ -1,4 +1,4 @@
-import { VizFiles, VizFile } from "@vizhub/viz-types";
+import { VizFiles, VizFile, FileCollection } from "@vizhub/viz-types";
 /**
  * If the LLM outputs empty text for a file, we interpret this
  * as a request to delete the file.
@@ -7,14 +7,8 @@ export declare function shouldDeleteFile(file?: VizFile): boolean;
 /**
  * Processes files for the prompt by truncating large files
  */
-export declare function prepareFilesForPrompt(files: VizFiles): {
-    name: string;
-    text: string;
-}[];
+export declare function prepareFilesForPrompt(files: VizFiles): FileCollection;
 /**
  * Merges original files with changes from the LLM
  */
-export declare function mergeFileChanges(originalFiles: VizFiles, parsedFiles: {
-    name: string;
-    text: string;
-}[]): VizFiles;
+export declare function mergeFileChanges(originalFiles: VizFiles, parsedFiles: FileCollection): VizFiles;

package/dist/fileUtils.js

@@ -17,15 +17,16 @@ function shouldDeleteFile(file) {
  * Processes files for the prompt by truncating large files
  */
 function prepareFilesForPrompt(files) {
-    return Object.values(files).map((file) => ({
-        name: file.name,
+    const result = {};
+    Object.values(files).forEach((file) => {
         // Example: truncate large files, etc.
-        text: file.text
+        result[file.name] = file.text
             .split("\n")
             .slice(0, file.name.endsWith(".csv") || file.name.endsWith(".json") ? 50 : 500)
             .map((line) => line.slice(0, 200))
-            .join("\n"),
-    }));
+            .join("\n");
+    });
+    return result;
 }
 /**
  * Merges original files with changes from the LLM
@@ -34,7 +35,10 @@ function mergeFileChanges(originalFiles, parsedFiles) {
     // Start with existing files
     let changedFiles = Object.keys(originalFiles).reduce((acc, fileId) => {
         const original = originalFiles[fileId];
-        const changedFile = parsedFiles.find((f) => f.name === original.name);
+        const changedText = parsedFiles[original.name];
+        const changedFile = changedText !== undefined
+            ? { name: original.name, text: changedText }
+            : undefined;
         if (shouldDeleteFile(changedFile)) {
             // Exclude from new set
             return acc;
@@ -47,14 +51,14 @@ function mergeFileChanges(originalFiles, parsedFiles) {
         return acc;
     }, {});
     // Handle newly-created files
-    parsedFiles.forEach((changedFile) => {
-        const existingFile = Object.values(changedFiles).find((file) => file.name === changedFile.name);
+    Object.entries(parsedFiles).forEach(([fileName, fileText]) => {
+        const existingFile = Object.values(changedFiles).find((file) => file.name === fileName);
         // If no existing file and not empty => it's a new file
-        if (!existingFile && !shouldDeleteFile(changedFile)) {
+        if (!existingFile && fileText.trim() !== "") {
             const newFileId = (0, viz_utils_1.generateVizFileId)();
             changedFiles[newFileId] = {
-                name: changedFile.name,
-                text: changedFile.text,
+                name: fileName,
+                text: fileText,
             };
         }
     });

package/dist/index.d.ts

@@ -1,4 +1,5 @@
-import { PerformAiEditParams, PerformAiEditResult } from "./types";
+import type { PerformAiEditParams, PerformAiEditResult } from "./types";
+export type { LlmFunction, PerformAiEditParams, PerformAiEditResult, } from "./types";
 /**
  * Core AI logic for:
  * - Building the prompt (including context)

package/dist/index.js

@@ -14,9 +14,9 @@ const debug = false;
  * - Retrieving cost metadata
  */
 async function performAiEdit({ prompt, files, llmFunction, apiKey, }) {
-    // 1. Serialize the existing files into the "markdown code block" format
+    // 1. Format the existing files into the "markdown code block" format
     const preparedFiles = (0, fileUtils_1.prepareFilesForPrompt)(files);
-    const filesContext = (0, llm_code_format_1.serializeMarkdownFiles)(preparedFiles);
+    const filesContext = (0, llm_code_format_1.formatMarkdownFiles)(preparedFiles);
     // 2. Assemble the final prompt
     const fullPrompt = (0, prompt_1.assembleFullPrompt)({ filesContext, prompt });
     debug && console.log("[performAiEdit] fullPrompt:", fullPrompt);

package/dist/types.d.ts

@@ -8,14 +8,15 @@ export interface PerformAiEditParams {
     files: VizFiles;
     llmFunction: LlmFunction;
     apiKey?: string;
+    baseURL?: string;
 }
 export interface PerformAiEditResult {
     changedFiles: VizFiles;
-    openRouterGenerationId: string;
-    upstreamCostCents: number;
-    provider: string;
-    inputTokens: number;
-    outputTokens: number;
-    promptTemplateVersion: number;
-    rawResponse: string;
+    openRouterGenerationId?: string;
+    upstreamCostCents?: number;
+    provider?: string;
+    inputTokens?: number;
+    outputTokens?: number;
+    promptTemplateVersion?: number;
+    rawResponse?: string;
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "editcodewithai",
-  "version": "0.1.0",
+  "version": "1.1.0",
   "description": "Edit Code With AI",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
@@ -13,7 +13,11 @@
     "build": "tsc",
     "prepublishOnly": "npm run build",
     "typecheck": "tsc --noEmit",
-    "benchmark": "ts-node src/benchmarks/index.ts"
+    "benchmark": "vite-node src/benchmarks/cli.ts run; cp -r ./benchmarks grader-app/public",
+    "grade": "ts-node src/benchmarks/cli.ts grade",
+    "benchmark:help": "ts-node src/benchmarks/cli.ts help",
+    "upgrade": "ncu -u",
+    "prettier": "prettier {*.*,**/*.*} --write"
   },
   "repository": {
     "type": "git",
@@ -36,16 +40,22 @@
   },
   "homepage": "https://github.com/vizhub-core/editcodewithai#readme",
   "dependencies": {
-    "@langchain/core": "^0.3.43",
-    "@langchain/openai": "^0.5.2",
-    "@vizhub/viz-types": "^0.0.2",
-    "@vizhub/viz-utils": "^0.0.1",
-    "dotenv": "^16.4.7",
-    "llm-code-format": "^0.2.2"
+    "@vizhub/viz-types": "^0.1.0",
+    "@vizhub/viz-utils": "^1.0.1",
+    "dotenv": "^16.5.0",
+    "llm-code-format": "^2.0.1"
   },
   "devDependencies": {
+    "@langchain/core": "^0.3.44",
+    "@langchain/openai": "^0.5.5",
+    "langchain": "^0.3.21",
+    "cors": "^2.8.5",
+    "express": "^5.1.0",
+    "npm-check-updates": "^17.1.18",
+    "prettier": "^3.5.3",
+    "puppeteer": "^24.6.1",
     "ts-node": "^10.9.2",
-    "typescript": "^5.8.2",
+    "typescript": "^5.8.3",
     "vitest": "^3.1.1"
   }
 }

package/dist/benchmark.d.ts

@@ -1,37 +0,0 @@
-import { LlmFunction } from "./types";
-/**
- * Creates an OpenRouter LLM function with the provided API key and model using LangChain
- * @param apiKey OpenRouter API key
- * @param model Model to use (defaults to "anthropic/claude-3.5-sonnet")
- * @returns LLM function that can be used with performAiEdit
- */
-export declare function createOpenRouterLlmFunction(apiKey: string, model?: string): LlmFunction;
-/**
- * Runs a benchmark test using OpenRouter to implement an "add" function
- * @param apiKey OpenRouter API key
- * @param model Optional model to use (defaults to "openai/gpt-4")
- * @returns Benchmark results including the implementation and performance metrics
- */
-export declare function runAddFunctionBenchmark(apiKey: string, model?: string): Promise<{
-    success: boolean;
-    implementation: any;
-    isCorrect: boolean;
-    testOutput: any;
-    elapsedTime: number;
-    costCents: any;
-    provider: any;
-    inputTokens: any;
-    outputTokens: any;
-    error?: undefined;
-} | {
-    success: boolean;
-    error: string;
-    implementation?: undefined;
-    isCorrect?: undefined;
-    testOutput?: undefined;
-    elapsedTime?: undefined;
-    costCents?: undefined;
-    provider?: undefined;
-    inputTokens?: undefined;
-    outputTokens?: undefined;
-}>;

package/dist/benchmark.js

@@ -1,123 +0,0 @@
-import { performAiEdit } from "./index";
-import { ChatOpenAI } from "@langchain/openai";
-import { StringOutputParser } from "@langchain/core/output_parsers";
-/**
- * Creates an OpenRouter LLM function with the provided API key and model using LangChain
- * @param apiKey OpenRouter API key
- * @param model Model to use (defaults to "anthropic/claude-3.5-sonnet")
- * @returns LLM function that can be used with performAiEdit
- */
-export function createOpenRouterLlmFunction(apiKey, model = "anthropic/claude-3.5-sonnet") {
-    return async (prompt) => {
-        try {
-            const options = {
-                modelName: model,
-                configuration: {
-                    apiKey,
-                    baseURL: "https://openrouter.ai/api/v1",
-                },
-                streaming: false,
-            };
-            const chatModel = new ChatOpenAI(options);
-            const result = await chatModel.invoke(prompt);
-            const parser = new StringOutputParser();
-            const resultString = await parser.invoke(result);
-            return {
-                content: resultString,
-                generationId: Date.now().toString(), // OpenRouter doesn't return an ID through LangChain
-            };
-        }
-        catch (error) {
-            if (error instanceof Error) {
-                throw new Error(`OpenRouter API error: ${error.message}`);
-            }
-            else {
-                throw new Error(`OpenRouter API error: ${String(error)}`);
-            }
-        }
-    };
-}
-/**
- * Runs a benchmark test using OpenRouter to implement an "add" function
- * @param apiKey OpenRouter API key
- * @param model Optional model to use (defaults to "openai/gpt-4")
- * @returns Benchmark results including the implementation and performance metrics
- */
-export async function runAddFunctionBenchmark(apiKey, model = "openai/gpt-4") {
-    console.log(`Running benchmark with model: ${model}`);
-    // Create test file with empty add function
-    const files = {
-        file1: {
-            name: "index.js",
-            text: "function add(a, b) {\n  // TODO: Implement this function\n}\n\nmodule.exports = { add };",
-        },
-    };
-    const prompt = "Implement the 'add' function to add two numbers together and return the result.";
-    // Create LLM function
-    const llmFunction = createOpenRouterLlmFunction(apiKey, model);
-    console.log("Sending request to OpenRouter...");
-    const startTime = Date.now();
-    try {
-        // Perform the AI edit
-        const result = await performAiEdit({
-            prompt,
-            files,
-            llmFunction,
-            apiKey,
-        });
-        const endTime = Date.now();
-        const elapsedTime = (endTime - startTime) / 1000;
-        // Extract the implementation
-        const implementation = result.changedFiles.file1?.text || "";
-        // Validate the implementation
-        let isCorrect = false;
-        let testOutput = null;
-        try {
-            // Create a function from the implementation to test it
-            const funcStr = implementation + "\nreturn add(3, 4);";
-            const testFunc = new Function(funcStr);
-            testOutput = testFunc();
-            isCorrect = testOutput === 7;
-        }
-        catch (error) {
-            console.error("Error testing implementation:", error);
-        }
-        return {
-            success: true,
-            implementation,
-            isCorrect,
-            testOutput,
-            elapsedTime,
-            costCents: result.upstreamCostCents,
-            provider: result.provider,
-            inputTokens: result.inputTokens,
-            outputTokens: result.outputTokens,
-        };
-    }
-    catch (error) {
-        console.error("Benchmark failed:", error);
-        return {
-            success: false,
-            error: error instanceof Error ? error.message : String(error),
-        };
-    }
-}
-/**
- * Example usage of the benchmark function
- */
-// Check if this file is being executed directly (CommonJS approach)
-if (require.main === module) {
-    // This code runs when the file is executed directly
-    const apiKey = process.env.OPENROUTER_API_KEY;
-    if (!apiKey) {
-        console.error("Please set the OPENROUTER_API_KEY environment variable");
-        process.exit(1);
-    }
-    runAddFunctionBenchmark(apiKey)
-        .then((result) => {
-        console.log("Benchmark result:", JSON.stringify(result, null, 2));
-    })
-        .catch((error) => {
-        console.error("Error running benchmark:", error);
-    });
-}

package/dist/benchmarks/benchmarkRunner.d.ts

@@ -1,5 +0,0 @@
-/**
- * Main function: runs the challenges on different models, calls the AI to fill them in,
- * executes each test, and writes pass/fail results to CSV.
- */
-export declare function runAllChallenges(): Promise<void>;

package/dist/benchmarks/benchmarkRunner.js

@@ -1,85 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.runAllChallenges = runAllChallenges;
-const file_system_1 = require("langchain/cache/file_system");
-const index_1 = require("../index");
-const llm_1 = require("./llm");
-const fileUtils_1 = require("./fileUtils");
-const runner_1 = require("./runner");
-const challenges_1 = require("./challenges");
-const models_1 = require("./models");
-/**
- * Main function: runs the challenges on different models, calls the AI to fill them in,
- * executes each test, and writes pass/fail results to CSV.
- */
-async function runAllChallenges() {
-    const apiKey = process.env.OPENROUTER_API_KEY || "";
-    if (!apiKey) {
-        throw new Error("Please set OPENROUTER_API_KEY in your .env");
-    }
-    // Setup cache
-    const cacheDirBase = "benchmarks/llmResponseCache";
-    (0, fileUtils_1.ensureCacheDir)(cacheDirBase);
-    const cache = await file_system_1.LocalFileCache.create(cacheDirBase);
-    // Store results
-    const results = [];
-    // Run each model against each challenge
-    for (const model of models_1.models) {
-        try {
-            await runModelChallenges(model, apiKey, cache, results);
-        }
-        catch (error) {
-            console.error(`Error running model ${model}:`, error);
-            // Add failure results for remaining challenges
-            for (const challenge of challenges_1.challenges) {
-                results.push({
-                    challenge: challenge.name,
-                    model,
-                    passFail: "error",
-                });
-            }
-        }
-    }
-    // Write results to CSV
-    const csv = (0, fileUtils_1.writeResultsToCsv)(results, "benchmarks/results.csv");
-    console.log("\nAll challenges completed. See 'benchmarks/results.csv' for summary.\n");
-    console.log(csv);
-}
-/**
- * Runs all challenges for a specific model
- */
-async function runModelChallenges(model, apiKey, cache, results) {
-    // Create the LLM function for this model
-    const llmFunction = (0, llm_1.createOpenRouterLlmFunction)(model, apiKey, cache);
-    // Run each challenge for this model
-    for (const challenge of challenges_1.challenges) {
-        console.log(`\n=== Challenge: ${challenge.name} | Model: ${model} ===`);
-        try {
-            // Ask AI to fill out the placeholder TODOs
-            const aiResult = await (0, index_1.performAiEdit)({
-                prompt: challenge.prompt,
-                files: challenge.files,
-                llmFunction,
-            });
-            // Write returned files to disk, including LLM response
-            const challengeDir = (0, fileUtils_1.writeChallengeFiles)(challenge.name, model, aiResult.changedFiles, aiResult.rawResponse);
-            // Run index.js in a child process
-            const exitCode = (0, runner_1.runNodeTest)(challengeDir);
-            const passFail = exitCode === 0 ? "pass" : "fail";
-            // Record the result
-            results.push({
-                challenge: challenge.name,
-                model,
-                passFail,
-            });
-        }
-        catch (error) {
-            console.error(`Error running challenge ${challenge.name}:`, error);
-            results.push({
-                challenge: challenge.name,
-                model,
-                passFail: "error",
-            });
-        }
-    }
-}

package/dist/benchmarks/challenges.d.ts

@@ -1,2 +0,0 @@
-import { Challenge } from "./types";
-export declare const challenges: Challenge[];