openkbs 0.0.31 → 0.0.33

package/MODIFY.md

@@ -1,12 +1,12 @@
-#### HUMAN-DEFINED REQUIREMENTS
+#### USER-DEFINED REQUIREMENTS 
 
 Define any additional features, integrations, or behavior modifications
 
-#### HUMAN-DEFINED REQUIREMENTS END
-
+#### USER-DEFINED REQUIREMENTS END
 
 # OpenKBS Framework Technical Guide (For AI Coding Agents)
 
+OpenKBS is claud AI platform designed to build, deploy and integrate AI agents and applications.
 This guide provides a brief overview of developing backend and frontend logic for your OpenKBS app/agent.
 
 ## Project Structure Overview
@@ -14,8 +14,8 @@ This guide provides a brief overview of developing backend and frontend logic fo
 A typical OpenKBS project has the following key directories:
 
 *   **`src/`**: Contains your custom source code.
-    *   **`Events/`**: Houses backend event handlers.
-        *   `actions.js`: Often used for shared logic/tool definitions (does not have own NPM dependencies).
+    *   **`Events/`**: Houses backend LLM event handlers.
+        *   `actions.js`: Example file commonly used for shared backend logic/tools definitions.
         *   `onRequest.js`: Handles incoming user messages before LLM processing.
         *   `onRequest.json`: NPM dependencies for `onRequest.js`.
         *   `onResponse.js`: Handles LLM responses before sending to the user.
@@ -32,52 +32,101 @@ A typical OpenKBS project has the following key directories:
     *   `instructions.txt`: Instructions for the LLM.
     *   `icon.png`: Application icon.
 
+#### Backend Handlers
+- Files with names starting with "on" (like onRequest.js) are called handler
+- Each handler functions as NodeJS execution entry point executed by the platform upon certain events.
+- use ES6, user import, do not use require()
+
 #### onRequest and onResponse Handlers
 
-The core of the OpenKBS backend framework revolves around the `onRequest` and `onResponse` event handlers.  
-These handlers act as middleware, intercepting messages before and after they are processed by the LLM.
+The core of the OpenKBS backend framework revolves around the `onRequest` and `onResponse` Node.js backend event handlers.
+These handlers act as middleware, intercepting user messages and messages generated by the LLM.
+
+- onRequest handler is invoked every time a user sends a message to the chat. It provides an opportunity to process the user's input, extract commands and perform actions based on the user's message.
+- onResponse handler is invoked after the LLM generates a response. It allows processing of the LLM's output, execution of commands based on the LLM's message.
 
-* **`onRequest` Handler:** This handler is invoked every time a user sends a message to the chat. It provides an opportunity to pre-process the user's input, extract commands and perform actions based on the user's message.
+#### NPM Dependencies for Backend Handlers
 
-* **`onResponse` Handler:** This handler is invoked after the LLM generates a response. It allows post-processing of the LLM's output, execution of commands based on the LLM's intentions.
+1. If a file imports an NPM dependency and is then imported by a handler, this dependency must be defined in the handler's corresponding json file
+2. Example: If actions.js imports got and onRequest.js imports actions.js, then got must be in onRequest.json
 
 
 ## Backend Dependencies
 
 To use external NPM packages in your backend event handlers, you must declare them in the corresponding `.json` file.
-OpenKBS also provides a secure way to handle sensitive information using the `{{secrets.your_secret_name}}` syntax.
 
-**Example: Using axios with an API key**
+**Example: Using https module with an API key**
 
-1. **Declare dependencies** in both `src/Events/onRequest.json` and `src/Events/onResponse.json` (as each handler have separate build):
+1. **Declare dependencies** in both `src/Events/onRequest.json` and `src/Events/onResponse.json` (as each handler have separate Node.js build):
     ```json
     {
       "dependencies": {
-        "axios": "^1.6.2"
+        "got": "^11.8.5"
       }
     }
     ```
 
-2.  **Implement and use in `src/Events/actions.js`**:
-    ```javascript
-    import axios from 'axios';
-
-    export const getActions = (meta) => {
-        return [
-            [/\/?getNews\("(.*)"\)/, async (match) => {
-                const topic = match[1];
-                const response = await axios.get('https://newsapi.org/v2/everything', {
-                    params: {
-                        q: topic,
-                        apiKey: '{{secrets.news_api_key}}' // Securely injected at runtime
-                    }
-                });
-                return { result: response.data.articles, ...meta };
-            }],
-            // ... other actions
-        ];
-    };
-    ```
+2.  **Implement in any JavaScript file** (actions.js is just an example name):
+
+```javascript
+import got from 'got';
+export const getActions = (meta) => {
+    return [
+        [/\/?getNews\("(.*)"\)/, async (match) => {
+            const { body } = await got(`https://newsapi.org/v2/everything?q=${match[1]}&apiKey={{secrets.news_api_key}}`, 
+                { responseType: 'json' });
+            
+            return { result: body.articles, ...meta };
+        }],
+        // ... other actions
+    ];
+};
+```
+
 ## Secrets Management
-Define `news_api_key` in your application's secrets manager on the OpenKBS platform.
-The platform will inject the actual value at runtime, keeping your credentials secure while enabling you to make API calls with authenticated services.
+OpenKBS provides a secure way to handle sensitive information using the `{{secrets.your_secret_name}}` syntax.
+Never hardcode secrets in the code, if any secrets are provided by the user replace them with placeholders syntax above.
+The user will later insert the secrets using the secrets manager
+
+#### LLM Instructions
+`app/instructions.txt`
+This file contains the instructions for the agent
+
+**Example Instructions:**
+
+```
+You are an AI assistant.
+
+You can execute the following commands:
+
+/getNews("query")
+Description: """
+Get the latest news
+"""
+```
+
+#### Important Coding guidelines
+- Organize new features across multiple files (like Events/utils.js) rather than adding everything to actions.js
+- Similar for Frontend: keep newly implemented React components outside contentRender.js for maintainability.
+- Keep in mind onRenderChatMessage is not a React component but simple JS function
+
+
+#### onRenderChatMessage injected functions
+These functions are automatically injected by the framework to onRenderChatMessage, providing access to system features
+```
+// Where functions are injected
+const onRenderChatMessage = async (params) => {
+    const { APIResponseComponent, theme, setBlockingLoading, setSystemAlert, RequestChatAPI,
+    kbUserData, generateMsgId, messages, msgIndex } = params;
+    ...
+}
+```
+
+```
+// Shows toast notifications in top-right corner
+setSystemAlert({
+    msg: 'Payment successful',
+    type: 'success', //'error'|'success'|'info'
+    duration: 5000
+});
+```

package/README.md

@@ -597,6 +597,8 @@ The `openkbs` object provides a set of utility functions and services to interac
 
 * **`openkbs.googleSearch(q, params)`:** Performs a Google search using the provided query and parameters.
 
+* **`openkbs.sendMail(email, subject, content)`:** Sends an email to the specified recipient
+
 * **`openkbs.documentToText(documentURL, params)`:** Extracts text from various document formats.
 
 * **`openkbs.imageToText(imageUrl, params)`:** Extracts text from an image.
@@ -619,6 +621,8 @@ The `openkbs` object provides a set of utility functions and services to interac
 
 * **`openkbs.clientHeaders`:** Exposes client headers for accessing information like IP address, location, etc. (e.g., `openkbs.clientHeaders['x-forwarded-for']`).
 
+* **`openkbs.createEmbeddings(input, model)`:** Create embeddings from input text
+
 **Example SDK Usage:**
 
 ```javascript

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "openkbs",
-  "version": "0.0.31",
+  "version": "0.0.33",
   "description": "OpenKBS - Command Line Interface",
   "main": "src/index.js",
   "scripts": {
@@ -9,6 +9,7 @@
     "deploy:linux": "aws s3 cp build/openkbs-linux-x64 s3://openkbs-downloads/cli/linux/openkbs && aws s3 cp build/openkbs-linux-arm64 s3://openkbs-downloads/cli/linux/openkbs-arm64",
     "deploy:macos": "aws s3 cp build/openkbs-macos-arm64 s3://openkbs-downloads/cli/macos/openkbs && aws s3 cp build/openkbs-macos-x64 s3://openkbs-downloads/cli/macos/openkbs-x64",
     "deploy:win": "aws s3 cp build/openkbs-win-x64.exe s3://openkbs-downloads/cli/windows/openkbs.exe && aws s3 cp build/openkbs-win-arm64.exe s3://openkbs-downloads/cli/windows/openkbs-arm64.exe",
+    "deploy:templates": "node scripts/deploy-templates.js",
     "postinstall": "node src/install.js"
   },
   "bin": {

package/scripts/deploy-templates.js

@@ -0,0 +1,43 @@
+#!/usr/bin/env node
+
+const fs = require('fs-extra');
+const path = require('path');
+const { S3Client, PutObjectCommand } = require('@aws-sdk/client-s3');
+
+const s3Client = new S3Client({ region: 'us-east-1' });
+const BUCKET = 'openkbs-downloads';
+const TEMPLATES_DIR = path.join(__dirname, '../templates');
+
+async function uploadDirectory(localPath, s3Prefix) {
+    const items = await fs.readdir(localPath, { withFileTypes: true });
+    
+    for (const item of items) {
+        const itemPath = path.join(localPath, item.name);
+        const s3Key = `${s3Prefix}/${item.name}`;
+        
+        if (item.isDirectory()) {
+            await uploadDirectory(itemPath, s3Key);
+        } else {
+            const fileContent = await fs.readFile(itemPath);
+            await s3Client.send(new PutObjectCommand({
+                Bucket: BUCKET,
+                Key: s3Key,
+                Body: fileContent
+            }));
+            console.log(`Uploaded: ${s3Key}`);
+        }
+    }
+}
+
+async function deployTemplates() {
+    try {
+        console.log('Deploying templates to S3...');
+        await uploadDirectory(TEMPLATES_DIR, 'templates');
+        console.log('Templates deployed successfully!');
+    } catch (error) {
+        console.error('Error deploying templates:', error);
+        process.exit(1);
+    }
+}
+
+deployTemplates();

package/src/actions.js

@@ -10,10 +10,10 @@ const {
     fetchAndSaveSettings, downloadFiles, downloadIcon, updateKB, uploadFiles, generateKey, generateMnemonic,
     reset, bold, red, yellow, green, createKB, saveLocalKBData, listKBs, deleteKBFile,
     deleteKB, buildPackage, replacePlaceholderInFiles, buildNodePackage, initByTemplateAction, modifyKB,
-    listKBsSharedWithMe
+    listKBsSharedWithMe, downloadTemplates
 } = require("./utils");
 
-const TEMPLATE_DIR = path.join(__dirname, '../templates');
+const TEMPLATE_DIR = path.join(os.homedir(), '.openkbs', 'templates');
 const jwtPath = path.join(os.homedir(), '.openkbs', 'clientJWT');
 const generateTransactionId = () => `${+new Date()}-${Math.floor(100000 + Math.random() * 900000)}`;
 
@@ -247,12 +247,31 @@ async function pushAction(location = 'origin', targetFile, options) {
 }
 
 async function cloneAction(kbId) {
+    // Store the original directory so we can restore it if needed
+    const originalDir = process.cwd();
+    
     try {
+        // Create a subdirectory with the name of the kbId
+        const targetDir = path.join(originalDir, kbId);
+        
+        // Check if directory already exists
+        if (fs.existsSync(targetDir)) {
+            console.red(`Directory ${kbId} already exists.`);
+            return;
+        }
+        
+        // Create the subdirectory
+        fs.mkdirSync(targetDir);
+        
+        // Change to the new directory
+        process.chdir(targetDir);
+        
         const localKBData = await fetchLocalKBData({forceInit: true});
 
         if (localKBData?.kbId) {
             console.red(`KB ${localKBData?.kbId} already saved in settings.json.`);
             console.yellow(`To pull the changes from OpenKBS remote use "openkbs pull"`);
+            process.chdir(originalDir); // Change back to original directory
             return;
         }
 
@@ -262,14 +281,26 @@ async function cloneAction(kbId) {
         await fetchAndSaveSettings({ kbId }, kbId, kbToken);
         await downloadIcon(kbId);
         await downloadFiles(['functions', 'frontend'], kbId, kbToken);
-        console.green('Cloning complete!');
+        console.green(`Cloning complete! Files created in directory: ${kbId}`);
     } catch (error) {
         console.error('Error during clone operation:', error.message);
+        // Make sure we return to the original directory in case of error
+        if (process.cwd() !== originalDir) {
+            process.chdir(originalDir);
+        }
+    } finally {
+        // Always ensure we return to the original directory
+        if (process.cwd() !== originalDir) {
+            process.chdir(originalDir);
+        }
     }
 }
 
 async function createByTemplateAction(name) {
     try {
+        // Download templates from S3 first
+        await downloadTemplates();
+        
         const targetDir = path.join(process.cwd(), name);
 
         if (fs.existsSync(targetDir)) {

package/src/utils.js

@@ -10,7 +10,7 @@ const { exec } = require('child_process');
 const readline = require('readline');
 const Spinner = require('cli-spinner').Spinner;
 
-const TEMPLATE_DIR = path.join(__dirname, '../templates');
+const TEMPLATE_DIR = path.join(os.homedir(), '.openkbs', 'templates');
 
 /**
  * Encrypts the given text using AES encryption with a passphrase.
@@ -1021,6 +1021,65 @@ async function buildLocalFilesMap(localDir, namespaces) {
     return filesMap;
 }
 
+async function downloadTemplates() {
+    try {
+        const templatesDir = TEMPLATE_DIR;
+        await fs.ensureDir(templatesDir);
+        
+        // Clear existing templates
+        await fs.emptyDir(templatesDir);
+        
+        // Download templates from S3
+        await downloadTemplatesFromS3(templatesDir);
+        
+        console.log('Templates downloaded successfully');
+    } catch (error) {
+        console.error('Error downloading templates:', error);
+        throw error;
+    }
+}
+
+async function downloadTemplatesFromS3(targetDir) {
+    const s3Client = new S3Client({ region: 'us-east-1' });
+    const { ListObjectsV2Command, GetObjectCommand } = require('@aws-sdk/client-s3');
+    
+    const bucket = 'openkbs-downloads';
+    const prefix = 'templates/';
+    
+    // List all objects in templates folder
+    const listResponse = await s3Client.send(new ListObjectsV2Command({
+        Bucket: bucket,
+        Prefix: prefix
+    }));
+    
+    // Download all files in parallel
+    const downloadPromises = (listResponse.Contents || []).map(async (obj) => {
+        const key = obj.Key;
+        const relativePath = key.substring(prefix.length);
+        
+        // Skip if it's a directory marker
+        if (relativePath.endsWith('/')) return;
+        
+        const localPath = path.join(targetDir, relativePath);
+        
+        // Ensure directory exists
+        await fs.ensureDir(path.dirname(localPath));
+        
+        // Download file
+        const response = await s3Client.send(new GetObjectCommand({
+            Bucket: bucket,
+            Key: key
+        }));
+        
+        const fileContent = await streamToBuffer(response.Body);
+        await fs.writeFile(localPath, fileContent);
+        console.log(`Downloaded: ${relativePath}`);
+    });
+    
+    // Wait for all downloads to complete
+    await Promise.all(downloadPromises);
+}
+
 const generateKey = (passphrase) => {
     const salt = 'salt';
     const iterations = 1000;
@@ -1054,6 +1113,9 @@ const replacePlaceholderInFiles = (dir, name) => {
 
 async function initByTemplateAction(params) {
     try {
+        // Download templates from S3 first
+        await downloadTemplates();
+        
         const targetDir = process.cwd();
 
         // Copy all files and folders, skipping existing ones
@@ -1077,5 +1139,6 @@ module.exports = {
     KB_API_URL, AUTH_API_URL, decryptKBFields, fetchLocalKBData, fetchKBJWT, createAccountIdFromPublicKey, signPayload,
     listFiles, getUserProfile, getKB, fetchAndSaveSettings, downloadIcon, downloadFiles, updateKB, uploadFiles, generateKey,
     generateMnemonic, reset, bold, red, yellow, green, cyan, createKB, getClientJWT, saveLocalKBData, listKBs, deleteKBFile,
-    deleteKB, buildPackage, replacePlaceholderInFiles, buildNodePackage, initByTemplateAction, modifyKB, listKBsSharedWithMe
+    deleteKB, buildPackage, replacePlaceholderInFiles, buildNodePackage, initByTemplateAction, modifyKB, listKBsSharedWithMe,
+    downloadTemplates
 }