naisys 1.0.2 → 1.0.3

package/README.md

@@ -1,15 +1,15 @@
 ## NAISYS (Node.js Autonomous Intelligence System)
 
-NAISYS is acts as a proxy shell between a LLM and a real shell. The goal is to see how far a LLM can
+NAISYS acts as a proxy shell between LLM(s) and a real shell. The goal is to see how far a LLM can
 get into writing a website from scratch as well as work with other LLM agents on the same project. Trying to figure
-out what works and what doesn't when it comes to 'cognitive architectures'. NAISYS isn't
+out what works and what doesn't when it comes to 'cognitive architectures' for autonomy. NAISYS isn't
 limited to websites, but it seemed like a good place to start.
 
-Since the LLM has a limited context, NAISYS should take this into account and help the LLM
+Since the LLM has a limited context, NAISYS takes this into account and helps the LLM
 perform 'context friendly' operations. For example reading/writing a file can't use a typical editor like
 vim or nano so point the LLM to use cat to read/write files in a single operation.
 
-[NPM](https://www.npmjs.com/package/naisys) | [Website](https://naisys.org) | [Discord](https://discord.gg/JBUPWSbaEt)
+[NPM](https://www.npmjs.com/package/naisys) | [Website](https://naisys.org) | [Discord](https://discord.gg/JBUPWSbaEt) | [Demo Video](https://www.youtube.com/watch?v=Ttya3ixjumo)
 
 #### Node.js is used to create a simple proxy shell environment for the LLM that
 
@@ -76,6 +76,11 @@ agentPrompt: |
   You can use PHP as a way to share layout across pages and reduce duplication.
   Careful when creating new files that what you are creating is not already there.
 
+# The number of tokens you want to limit a session to, independent of the LLM token max itself
+# A lower max relies more on the LLM ending the session with good enough notes to not get lost when the session restarts
+# A higher max allows the LLM to do more without losing track, but is more expensive
+tokenMax: 5000
+
 # The number of seconds to pause after each console interaction for debugging and rate limiting
 # No value or zero means wait indefinitely (debug driven)
 debugPauseSeconds: 5
@@ -94,14 +99,14 @@ spendLimitDollars: 2.00
   - If a yaml file is passed, naisys will start a single agent
   - If a directory is passed, naisys will start a tmux session with the screen split for each agent
 
-### Creating a persistent agent run website (on Digital Ocean for example)
+#### Creating a persistent agent run website (on Digital Ocean for example)
 
 - Create new VM using the [LAMP stack droplet template](https://marketplace.digitalocean.com/apps/lamp)
 - Login to the droplet using the web console
 - Run `apt install npm`
 - Install `nvm` using the `curl` url from these [instructions](https://github.com/nvm-sh/nvm?tab=readme-ov-file#installing-and-updating)
   - Run `nvm install/use 20` to set node version to 20
-  - Follow the general install instructions above
+- Follow the general install instructions above
 
 ## Using NAISYS
 
@@ -128,12 +133,12 @@ spendLimitDollars: 2.00
 - NAISYS tries to be light, acting as a helpful proxy between the LLM and a real shell, most commands should pass right though to the shell
 - Debug Commands
   - `cost` - Prints the current total LLM cost
-  - `context` - Prints the current context 
+  - `context` - Prints the current context
   - `exit` - Exits NAISYS. If the LLM tries to use `exit`, it is directed to use `endsession` instead
   - `talk` - Communicate with the local agent to give hints or ask questions (the agent itself does not know about talk and is directed to use `comment` or `llmail` for communication)
 - Special Commands usable by the LLM as well as by the debug prompt
-  - `comment <notes>` - The LLM is directed to use this for 'thinking out loud' which avoids 'invalid command' errors
-  - `endsession <notes>` - Clear the context and start a new session.
+  - `comment "<note>"` - The LLM is directed to use this for 'thinking out loud' which avoids 'invalid command' errors
+  - `endsession "<note>"` - Clear the context and start a new session.
     - The LLM is directed to track it's context size and to end the session with a note before running over the context limit
   - `pause <seconds>` - Can be used by the debug agent or the LLM to pause execution indefinitely, or until a new message is received from another agent, or for a set number of seconds
 - NAISYS apps
@@ -154,6 +159,8 @@ spendLimitDollars: 2.00
 
 #### Notes for Windows users
 
+- To use NAISYS on Windows you need to run it locally from source (or from within WSL)
+- Use the above instructions to install locally, and then continue with the instructions below
 - Install WSL (Windows Subsystem for Linux)
 - The `NAISYS_FOLDER` and `WEBSITE_FOLDER` should be set to the WSL path
   - So `C:\var\naisys` should be `/mnt/c/var/naisys` in the `.env` file

package/dist/apps/llmail.js

@@ -7,7 +7,7 @@ import { naisysToHostPath } from "../utils/utilities.js";
 const _dbFilePath = naisysToHostPath(`${config.naisysFolder}/lib/llmail.db`);
 let _myUserId = -1;
 // Implement maxes so that LLMs actively manage threads, archive, and create new ones
-const _threadTokenMax = config.tokenMax / 2; // So 4000, would be 2000 thread max
+const _threadTokenMax = config.agent.tokenMax / 2; // So 4000, would be 2000 thread max
 const _messageTokenMax = _threadTokenMax / 5; // Given the above a 400 token max, and 5 big messages per thread
 /** The 'non-simple' version of this is a thread first mail system. Where agents can create threads, add users, and reply to threads, etc..
  * The problem with this was the agents were too chatty with so many mail commands, wasting context replying, reading threads, etc..

package/dist/apps/llmynx.js

@@ -15,7 +15,7 @@ let _nextGlobalLinkNum = 1;
 export async function handleCommand(cmdArgs) {
     outputInDebugMode("LLMYNX DEBUG MODE IS ON");
     const argParams = cmdArgs.split(" ");
-    const defualtTokenMax = config.tokenMax / 8;
+    const defualtTokenMax = config.agent.tokenMax / 8;
     if (!argParams[0]) {
         argParams[0] = "help";
     }
@@ -28,7 +28,7 @@ export async function handleCommand(cmdArgs) {
   links <url> <page>: Lists only the links for the given url. Use the page number to get more links`;
         case "search": {
             const query = argParams.slice(1).join(" ");
-            return await loadUrl("https://www.google.com/search?q=" + encodeURIComponent(query), config.tokenMax / 2, // Prevent form being reduced as google results are usually short anyways and we want to maintainq the links
+            return await loadUrl("https://www.google.com/search?q=" + encodeURIComponent(query), config.agent.tokenMax / 2, // Prevent form being reduced as google results are usually short anyways and we want to maintainq the links
             true, true);
         }
         case "open": {

package/dist/command/commandHandler.js

@@ -10,6 +10,7 @@ import { InputMode } from "../utils/inputMode.js";
 import * as logService from "../utils/logService.js";
 import * as output from "../utils/output.js";
 import { OutputColor } from "../utils/output.js";
+import * as utilities from "../utils/utilities.js";
 import * as promptBuilder from "./promptBuilder.js";
 import * as shellCommand from "./shellCommand.js";
 export var NextCommandAction;
@@ -23,46 +24,19 @@ export async function consoleInput(prompt, consoleInput) {
     // We process the lines one at a time so we can support multiple commands with line breaks
     let firstLine = true;
     let processNextLLMpromptBlock = true;
-    const userHostPrompt = promptBuilder.getUserHostPrompt();
     let nextCommandAction = NextCommandAction.Continue;
-    let nextInput = consoleInput.trim();
-    while (processNextLLMpromptBlock && nextInput) {
-        let input = "";
-        // if the prompt exists in the input, save if for the next run
-        const nextPromptPos = nextInput.indexOf(userHostPrompt);
-        const newLinePos = nextInput.indexOf("\n");
-        if (nextPromptPos == 0) {
-            const pathPrompt = await promptBuilder.getUserHostPathPrompt();
-            // check working directory is the same
-            if (nextInput.startsWith(pathPrompt)) {
-                // slice nextInput after $
-                const endPrompt = nextInput.indexOf("$", pathPrompt.length);
-                nextInput = nextInput.slice(endPrompt + 1).trim();
-                continue;
-            }
-            // else prompt did not match, stop processing input
-            else {
-                break;
-            }
-        }
-        // we can't validate that the working directory in the prompt is good until the commands are processed
-        else if (nextPromptPos > 0) {
-            input = nextInput.slice(0, nextPromptPos);
-            nextInput = nextInput.slice(nextPromptPos).trim();
-        }
-        // Else for single line custom NAISYS commands, only process the first line as there may be follow up shell commands
-        else if (newLinePos > 0 && nextInput.startsWith("comment ")) {
-            input = nextInput.slice(0, newLinePos);
-            nextInput = nextInput.slice(newLinePos).trim();
+    consoleInput = consoleInput.trim();
+    while (processNextLLMpromptBlock && consoleInput) {
+        const { input, nextInput, splitResult } = await splitMultipleInputCommands(consoleInput);
+        consoleInput = nextInput;
+        if (splitResult == SplitResult.InputIsPrompt) {
+            continue;
         }
-        else {
-            input = nextInput;
-            nextInput = "";
-        }
-        if (!input.trim()) {
+        else if (splitResult == SplitResult.InputPromptMismatch ||
+            !input.trim()) {
             break;
         }
-        // first line is special because we want to append the output to the context without a line break
+        // First line is special because we want to append the output to the context without a line break
         if (inputMode.current == InputMode.LLM) {
             if (firstLine) {
                 firstLine = false;
@@ -83,7 +57,12 @@ export async function consoleInput(prompt, consoleInput) {
                 break;
             }
             case "endsession": {
-                previousSessionNotes = cmdArgs;
+                // Don't need to check end line as this is the last command in the context, just read to the end
+                previousSessionNotes = utilities.trimChars(cmdArgs, '"');
+                if (!previousSessionNotes) {
+                    await contextManager.append(`End session notes are required. Use endsession "<notes>"`);
+                    break;
+                }
                 await output.commentAndLog("------------------------------------------------------");
                 nextCommandAction = NextCommandAction.EndSession;
                 processNextLLMpromptBlock = false;
@@ -144,8 +123,8 @@ export async function consoleInput(prompt, consoleInput) {
         }
     }
     // display unprocessed lines to aid in debugging
-    if (nextInput.trim()) {
-        await output.errorAndLog(`Unprocessed LLM response:\n${nextInput}`);
+    if (consoleInput.trim()) {
+        await output.errorAndLog(`Unprocessed LLM response:\n${consoleInput}`);
     }
     return {
         nextCommandAction,
@@ -153,4 +132,68 @@ export async function consoleInput(prompt, consoleInput) {
         wakeOnMessage: config.agent.wakeOnMessage,
     };
 }
+var SplitResult;
+(function (SplitResult) {
+    SplitResult[SplitResult["InputIsPrompt"] = 0] = "InputIsPrompt";
+    SplitResult[SplitResult["InputPromptMismatch"] = 1] = "InputPromptMismatch";
+})(SplitResult || (SplitResult = {}));
+async function splitMultipleInputCommands(nextInput) {
+    let input = "";
+    let splitResult;
+    // If the prompt exists in the input, save if for the next run
+    const userHostPrompt = promptBuilder.getUserHostPrompt();
+    const nextPromptPos = nextInput.indexOf(userHostPrompt);
+    const newLinePos = nextInput.indexOf("\n");
+    if (nextPromptPos == 0) {
+        const pathPrompt = await promptBuilder.getUserHostPathPrompt();
+        // Check working directory is the same
+        if (nextInput.startsWith(pathPrompt)) {
+            // Slice nextInput after $
+            const endPrompt = nextInput.indexOf("$", pathPrompt.length);
+            nextInput = nextInput.slice(endPrompt + 1).trim();
+            splitResult = SplitResult.InputIsPrompt;
+        }
+        // Else prompt did not match, stop processing input
+        else {
+            splitResult = SplitResult.InputPromptMismatch;
+        }
+    }
+    // We can't validate that the working directory in the prompt is good until the commands are processed
+    else if (nextPromptPos > 0) {
+        input = nextInput.slice(0, nextPromptPos);
+        nextInput = nextInput.slice(nextPromptPos).trim();
+    }
+    // Most custom NAISYS commands are single line, but comment in quotes can span multiple lines so we need to handle that
+    // because often the LLM puts shell commands after the comment
+    else if (nextInput.startsWith(`comment "`)) {
+        // Find next double quote in nextInput that isn't escaped
+        let endQuote = nextInput.indexOf(`"`, 9);
+        while (endQuote > 0 && nextInput[endQuote - 1] === "\\") {
+            endQuote = nextInput.indexOf(`"`, endQuote + 1);
+        }
+        if (endQuote > 0) {
+            input = nextInput.slice(0, endQuote + 1);
+            nextInput = nextInput.slice(endQuote + 1).trim();
+        }
+        else {
+            input = nextInput;
+            nextInput = "";
+        }
+    }
+    // If the LLM forgets the quote on the comment, treat it as a single line comment
+    else if (newLinePos > 0 && nextInput.startsWith("comment ")) {
+        input = nextInput.slice(0, newLinePos);
+        nextInput = nextInput.slice(newLinePos).trim();
+    }
+    // Else process the entire input now
+    else {
+        input = nextInput;
+        nextInput = "";
+    }
+    return { input, nextInput, splitResult };
+}
+export const exportedForTesting = {
+    splitMultipleInputCommands,
+    SplitResult,
+};
 //# sourceMappingURL=commandHandler.js.map

package/dist/command/commandLoop.js

@@ -150,7 +150,7 @@ async function displayNewMail() {
     // Check that token max for session will not be exceeded
     const newMsgTokenCount = newMessages.reduce((acc, msg) => acc + utilities.getTokenCount(msg), 0);
     const sessionTokens = contextManager.getTokenCount();
-    const tokenMax = config.tokenMax;
+    const tokenMax = config.agent.tokenMax;
     // Show full messages unless we are close to the token limit of the session
     // or in simple mode, which means non-threaded messages
     if (sessionTokens + newMsgTokenCount < tokenMax * 0.75) {
@@ -176,7 +176,7 @@ async function displayNewMail() {
 }
 async function displayContextWarning() {
     const tokenCount = contextManager.getTokenCount();
-    const tokenMax = config.tokenMax;
+    const tokenMax = config.agent.tokenMax;
     if (tokenCount > tokenMax) {
         await contextManager.append(`The token limit for this session has been exceeded.
 Use \`endsession <note>\` to clear the console and reset the session.

package/dist/command/promptBuilder.js

@@ -31,7 +31,7 @@ _readlineInterface.on("close", () => {
 });
 export async function getPrompt(pauseSeconds, wakeOnMessage) {
     const promptSuffix = inputMode.current == InputMode.Debug ? "#" : "$";
-    const tokenMax = config.tokenMax;
+    const tokenMax = config.agent.tokenMax;
     const usedTokens = contextManager.getTokenCount();
     const tokenSuffix = ` [Tokens: ${usedTokens}/${tokenMax}]`;
     let pause = "";

package/dist/config.js

@@ -7,9 +7,7 @@ program.argument("<agent-path>", "Path to agent configuration file").parse();
 dotenv.config();
 /** The system name that shows after the @ in the command prompt */
 export const hostname = "naisys";
-/** The number of tokens you want to limit a session to, independent of the LLM token max */
-export const tokenMax = 4000;
-/* .env is used for global configs across naisys, while agent configs for the specific agent */
+/* .env is used for global configs across naisys, while agent configs are for the specific agent */
 export const naisysFolder = getEnv("NAISYS_FOLDER", true);
 export const websiteFolder = getEnv("WEBSITE_FOLDER");
 export const localLlmUrl = getEnv("LOCAL_LLM_URL");
@@ -36,6 +34,7 @@ function loadAgentConfig() {
         "webModel",
         "agentPrompt",
         "spendLimitDollars",
+        "tokenMax",
         // debugPauseSeconds and wakeOnMessage can be undefined
     ]) {
         if (!valueFromString(checkAgentConfig, key)) {
@@ -44,4 +43,19 @@ function loadAgentConfig() {
     }
     return checkAgentConfig;
 }
+export const packageVersion = await getVersion();
+/** Can only get version from env variable when naisys is started with npm,
+ * otherwise need to rip it from the package ourselves relative to where this file is located */
+async function getVersion() {
+    try {
+        const packageJsonPath = new URL("../package.json", import.meta.url);
+        const packageJson = await import(packageJsonPath.href, {
+            assert: { type: "json" },
+        });
+        return packageJson.default.version;
+    }
+    catch (e) {
+        return "0.1";
+    }
+}
 //# sourceMappingURL=config.js.map

package/dist/llm/contextManager.js

@@ -34,7 +34,7 @@ For example when you run 'cat' or 'ls', don't write what you think the output wi
 Your role is that of the user. The system will provide responses and next command prompt. Don't output your own command prompt.
 Be careful when writing files through the command prompt with cat. Make sure to close and escape quotes properly.
 
-NAISYS ${process.env.npm_package_version} Shell
+NAISYS ${config.packageVersion} Shell
 Welcome back ${config.agent.username}!
 MOTD:
 Date: ${new Date().toLocaleString()}
@@ -46,9 +46,9 @@ Commands:
 Special Commands: (Don't mix with standard commands on the same prompt)
   llmail: A local mail system for communicating with your team
   llmynx: A context optimized web browser. Enter 'llmynx help' to learn how to use it
-  comment <thought>: Any non-command output like thinking out loud, prefix with the 'comment' command
+  comment "<thought>": Any non-command output like thinking out loud, prefix with the 'comment' command
   pause <seconds>: Pause for <seconds> or indeterminite if no argument is provided. Auto wake up on new mail message
-  endsession <note>: Ends this session, clears the console log and context.
+  endsession "<note>": Ends this session, clears the console log and context.
     The note should help you find your bearings in the next session. 
     The note should contain your next goal, and important things should you remember.
     Try to keep the note around 400 tokens.

package/dist/utils/utilities.js

@@ -37,4 +37,7 @@ export function ensureFileDirExists(filePath) {
         fs.mkdirSync(dir, { recursive: true });
     }
 }
+export function trimChars(text, charList) {
+    return text.replace(new RegExp(`^[${charList}]+|[${charList}]+$`, "g"), "");
+}
 //# sourceMappingURL=utilities.js.map

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "naisys",
   "description": "Node.js Autonomous Intelligence System",
-  "version": "1.0.2",
+  "version": "1.0.3",
   "type": "module",
   "main": "dist/naisys.js",
   "preferGlobal": true,
@@ -40,10 +40,10 @@
   "devDependencies": {
     "@types/escape-html": "1.0.4",
     "@types/js-yaml": "4.0.9",
-    "@types/node": "20.11.22",
+    "@types/node": "20.11.25",
     "@types/text-table": "0.2.5",
-    "@typescript-eslint/eslint-plugin": "7.1.0",
-    "@typescript-eslint/parser": "7.1.0",
+    "@typescript-eslint/eslint-plugin": "7.1.1",
+    "@typescript-eslint/parser": "7.1.1",
     "eslint": "8.57.0",
     "jest": "29.7.0",
     "prettier": "3.2.5",

package/dist/__tests__/utils/output.test.js

@@ -1,22 +0,0 @@
-import { describe, expect, it, jest } from "@jest/globals";
-jest.unstable_mockModule("../../config.js", () => ({}));
-const mockLogServiceWrite = jest
-    .fn()
-    .mockResolvedValue(1);
-jest.unstable_mockModule("../../utils/logService.js", () => ({
-    write: mockLogServiceWrite,
-}));
-const output = await import("../../utils/output.js");
-describe("commentAndLog function", () => {
-    it("should call writeDbLog with the correct arguments", async () => {
-        // Assuming you've refactored commentAndLog to take logService or its functionality as a parameter
-        await output.commentAndLog("Test message");
-        // Verify the mock was called correctly
-        expect(mockLogServiceWrite).toHaveBeenCalledWith({
-            content: "Test message",
-            role: "user",
-            type: "comment",
-        });
-    });
-});
-//# sourceMappingURL=output.test.js.map

package/dist/__tests__/utils/utilities.test.js

@@ -1,42 +0,0 @@
-import { describe, expect, test } from "@jest/globals";
-import { valueFromString } from "../../utils/utilities.js";
-describe("valueFromString", () => {
-    const obj = {
-        user: {
-            name: "John Doe",
-            contact: {
-                email: "john@example.com",
-                phone: {
-                    home: "123456",
-                    work: "789101",
-                },
-            },
-        },
-    };
-    test("retrieves a nested value successfully", () => {
-        expect(valueFromString(obj, "user.name")).toBe("John Doe");
-        expect(valueFromString(obj, "user.contact.email")).toBe("john@example.com");
-        expect(valueFromString(obj, "user.contact.phone.home")).toBe("123456");
-    });
-    test("returns undefined for non-existent path", () => {
-        expect(valueFromString(obj, "user.address")).toBeUndefined();
-    });
-    test("returns default value for non-existent path when specified", () => {
-        const defaultValue = "N/A";
-        expect(valueFromString(obj, "user.age", defaultValue)).toBe(defaultValue);
-    });
-    test("handles non-object inputs gracefully", () => {
-        expect(valueFromString(null, "user.name")).toBeUndefined();
-        expect(valueFromString(undefined, "user.name")).toBeUndefined();
-        expect(valueFromString("not-an-object", "user.name")).toBeUndefined();
-    });
-    test("deals with edge cases for paths", () => {
-        expect(valueFromString(obj, "")).toEqual(obj);
-        expect(valueFromString(obj, ".", "default")).toBe("default");
-    });
-    test("handles empty object and non-matching paths", () => {
-        expect(valueFromString({}, "user.name")).toBeUndefined();
-        expect(valueFromString(obj, "user.nonexistent.prop", "default")).toBe("default");
-    });
-});
-//# sourceMappingURL=utilities.test.js.map