openkbs 0.0.32 → 0.0.34

package/MODIFY.md

@@ -1,12 +1,12 @@
-#### HUMAN-DEFINED REQUIREMENTS
+#### USER-DEFINED REQUIREMENTS 
 
 Define any additional features, integrations, or behavior modifications
 
-#### HUMAN-DEFINED REQUIREMENTS END
-
+#### USER-DEFINED REQUIREMENTS END
 
 # OpenKBS Framework Technical Guide (For AI Coding Agents)
 
+OpenKBS is claud AI platform designed to build, deploy and integrate AI agents and applications.
 This guide provides a brief overview of developing backend and frontend logic for your OpenKBS app/agent.
 
 ## Project Structure Overview
@@ -14,8 +14,8 @@ This guide provides a brief overview of developing backend and frontend logic fo
 A typical OpenKBS project has the following key directories:
 
 *   **`src/`**: Contains your custom source code.
-    *   **`Events/`**: Houses backend event handlers.
-        *   `actions.js`: Example file commonly used for shared logic/tools definitions.
+    *   **`Events/`**: Houses backend LLM event handlers.
+        *   `actions.js`: Example file commonly used for shared backend logic/tools definitions.
         *   `onRequest.js`: Handles incoming user messages before LLM processing.
         *   `onRequest.json`: NPM dependencies for `onRequest.js`.
         *   `onResponse.js`: Handles LLM responses before sending to the user.
@@ -32,25 +32,23 @@ A typical OpenKBS project has the following key directories:
     *   `instructions.txt`: Instructions for the LLM.
     *   `icon.png`: Application icon.
 
-#### Important Coding guidelines
-Organize new features across multiple files (like Events/utils.js) rather than adding everything to actions.js; export functionality from separate modules and import them in actions.js as needed.
+#### Backend Handlers
+- Files with names starting with "on" (like onRequest.js) are called handler
+- Each handler functions as NodeJS execution entry point executed by the platform upon certain events.
+- use ES6, user import, do not use require()
 
 #### onRequest and onResponse Handlers
 
-The core of the OpenKBS backend framework revolves around the `onRequest` and `onResponse` event handlers.  
-These handlers act as middleware, intercepting messages before and after they are processed by the LLM.
-
-* **`onRequest` Handler:** This handler is invoked every time a user sends a message to the chat. It provides an opportunity to pre-process the user's input, extract commands and perform actions based on the user's message. The `onRequest.js` file must export a function that receives `request` and `metadata` parameters and returns a modified request object.
-
-* **`onResponse` Handler:** This handler is invoked after the LLM generates a response. It allows post-processing of the LLM's output, execution of commands based on the LLM's intentions. The `onResponse.js` file must export a function that receives `response` and `metadata` parameters and returns a modified response object.
+The core of the OpenKBS backend framework revolves around the `onRequest` and `onResponse` Node.js backend event handlers.
+These handlers act as middleware, intercepting user messages and messages generated by the LLM.
 
-#### NPM Dependencies for Handlers
+- onRequest handler is invoked every time a user sends a message to the chat. It provides an opportunity to process the user's input, extract commands and perform actions based on the user's message.
+- onResponse handler is invoked after the LLM generates a response. It allows processing of the LLM's output, execution of commands based on the LLM's message.
 
-**Important**: Files with names starting with "on" (like onRequest.js) are entry points for Node.js builds:
+#### NPM Dependencies for Backend Handlers
 
-1. Each handler has its own build process
-2. If a file imports node-fetch, then node-fetch must be in that handler's JSON file
-3. Example: If utils.js uses node-fetch and is imported by onRequest.js, then node-fetch must be in onRequest.json
+1. If a file imports an NPM dependency and is then imported by a handler, this dependency must be defined in the handler's corresponding json file
+2. Example: If actions.js imports got and onRequest.js imports actions.js, then got must be in onRequest.json
 
 
 ## Backend Dependencies
@@ -59,7 +57,7 @@ To use external NPM packages in your backend event handlers, you must declare th
 
 **Example: Using https module with an API key**
 
-1. **Declare dependencies** in both `src/Events/onRequest.json` and `src/Events/onResponse.json` (as each handler have separate build):
+1. **Declare dependencies** in both `src/Events/onRequest.json` and `src/Events/onResponse.json` (as each handler have separate Node.js build):
     ```json
     {
       "dependencies": {
@@ -71,6 +69,7 @@ To use external NPM packages in your backend event handlers, you must declare th
 2.  **Implement in any JavaScript file** (actions.js is just an example name):
 
 ```javascript
+import got from 'got';
 export const getActions = (meta) => {
     return [
         [/\/?getNews\("(.*)"\)/, async (match) => {
@@ -83,7 +82,7 @@ export const getActions = (meta) => {
     ];
 };
 ```
-    
+
 ## Secrets Management
 OpenKBS provides a secure way to handle sensitive information using the `{{secrets.your_secret_name}}` syntax.
 Never hardcode secrets in the code, if any secrets are provided by the user replace them with placeholders syntax above.
@@ -104,4 +103,30 @@ You can execute the following commands:
 Description: """
 Get the latest news
 """
+```
+
+#### Important Coding guidelines
+- Organize new features across multiple files (like Events/utils.js) rather than adding everything to actions.js
+- Similar for Frontend: keep newly implemented React components outside contentRender.js for maintainability.
+- Keep in mind onRenderChatMessage is not a React component but simple JS function
+
+
+#### onRenderChatMessage injected functions
+These functions are automatically injected by the framework to onRenderChatMessage, providing access to system features
+```
+// Where functions are injected
+const onRenderChatMessage = async (params) => {
+    const { APIResponseComponent, theme, setBlockingLoading, setSystemAlert, RequestChatAPI,
+    kbUserData, generateMsgId, messages, msgIndex } = params;
+    ...
+}
+```
+
+```
+// Shows toast notifications in top-right corner
+setSystemAlert({
+    msg: 'Payment successful',
+    type: 'success', //'error'|'success'|'info'
+    duration: 5000
+});
 ```

package/README.md

@@ -597,6 +597,8 @@ The `openkbs` object provides a set of utility functions and services to interac
 
 * **`openkbs.googleSearch(q, params)`:** Performs a Google search using the provided query and parameters.
 
+* **`openkbs.sendMail(email, subject, content)`:** Sends an email to the specified recipient
+
 * **`openkbs.documentToText(documentURL, params)`:** Extracts text from various document formats.
 
 * **`openkbs.imageToText(imageUrl, params)`:** Extracts text from an image.
@@ -619,6 +621,8 @@ The `openkbs` object provides a set of utility functions and services to interac
 
 * **`openkbs.clientHeaders`:** Exposes client headers for accessing information like IP address, location, etc. (e.g., `openkbs.clientHeaders['x-forwarded-for']`).
 
+* **`openkbs.createEmbeddings(input, model)`:** Create embeddings from input text
+
 **Example SDK Usage:**
 
 ```javascript

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "openkbs",
-  "version": "0.0.32",
+  "version": "0.0.34",
   "description": "OpenKBS - Command Line Interface",
   "main": "src/index.js",
   "scripts": {
@@ -9,6 +9,7 @@
     "deploy:linux": "aws s3 cp build/openkbs-linux-x64 s3://openkbs-downloads/cli/linux/openkbs && aws s3 cp build/openkbs-linux-arm64 s3://openkbs-downloads/cli/linux/openkbs-arm64",
     "deploy:macos": "aws s3 cp build/openkbs-macos-arm64 s3://openkbs-downloads/cli/macos/openkbs && aws s3 cp build/openkbs-macos-x64 s3://openkbs-downloads/cli/macos/openkbs-x64",
     "deploy:win": "aws s3 cp build/openkbs-win-x64.exe s3://openkbs-downloads/cli/windows/openkbs.exe && aws s3 cp build/openkbs-win-arm64.exe s3://openkbs-downloads/cli/windows/openkbs-arm64.exe",
+    "deploy:templates": "node scripts/deploy-templates.js",
     "postinstall": "node src/install.js"
   },
   "bin": {

package/scripts/deploy-templates.js

@@ -0,0 +1,43 @@
+#!/usr/bin/env node
+
+const fs = require('fs-extra');
+const path = require('path');
+const { S3Client, PutObjectCommand } = require('@aws-sdk/client-s3');
+
+const s3Client = new S3Client({ region: 'us-east-1' });
+const BUCKET = 'openkbs-downloads';
+const TEMPLATES_DIR = path.join(__dirname, '../templates');
+
+async function uploadDirectory(localPath, s3Prefix) {
+    const items = await fs.readdir(localPath, { withFileTypes: true });
+    
+    for (const item of items) {
+        const itemPath = path.join(localPath, item.name);
+        const s3Key = `${s3Prefix}/${item.name}`;
+        
+        if (item.isDirectory()) {
+            await uploadDirectory(itemPath, s3Key);
+        } else {
+            const fileContent = await fs.readFile(itemPath);
+            await s3Client.send(new PutObjectCommand({
+                Bucket: BUCKET,
+                Key: s3Key,
+                Body: fileContent
+            }));
+            console.log(`Uploaded: ${s3Key}`);
+        }
+    }
+}
+
+async function deployTemplates() {
+    try {
+        console.log('Deploying templates to S3...');
+        await uploadDirectory(TEMPLATES_DIR, 'templates');
+        console.log('Templates deployed successfully!');
+    } catch (error) {
+        console.error('Error deploying templates:', error);
+        process.exit(1);
+    }
+}
+
+deployTemplates();

package/src/actions.js

@@ -10,10 +10,10 @@ const {
     fetchAndSaveSettings, downloadFiles, downloadIcon, updateKB, uploadFiles, generateKey, generateMnemonic,
     reset, bold, red, yellow, green, createKB, saveLocalKBData, listKBs, deleteKBFile,
     deleteKB, buildPackage, replacePlaceholderInFiles, buildNodePackage, initByTemplateAction, modifyKB,
-    listKBsSharedWithMe
+    listKBsSharedWithMe, downloadTemplates
 } = require("./utils");
 
-const TEMPLATE_DIR = path.join(__dirname, '../templates');
+const TEMPLATE_DIR = path.join(os.homedir(), '.openkbs', 'templates');
 const jwtPath = path.join(os.homedir(), '.openkbs', 'clientJWT');
 const generateTransactionId = () => `${+new Date()}-${Math.floor(100000 + Math.random() * 900000)}`;
 
@@ -298,6 +298,9 @@ async function cloneAction(kbId) {
 
 async function createByTemplateAction(name) {
     try {
+        // Download templates from S3 first
+        await downloadTemplates();
+        
         const targetDir = path.join(process.cwd(), name);
 
         if (fs.existsSync(targetDir)) {

package/src/utils.js

@@ -10,7 +10,7 @@ const { exec } = require('child_process');
 const readline = require('readline');
 const Spinner = require('cli-spinner').Spinner;
 
-const TEMPLATE_DIR = path.join(__dirname, '../templates');
+const TEMPLATE_DIR = path.join(os.homedir(), '.openkbs', 'templates');
 
 /**
  * Encrypts the given text using AES encryption with a passphrase.
@@ -1021,6 +1021,64 @@ async function buildLocalFilesMap(localDir, namespaces) {
     return filesMap;
 }
 
+async function downloadTemplates() {
+    try {
+        const templatesDir = TEMPLATE_DIR;
+        await fs.ensureDir(templatesDir);
+        
+        // Clear existing templates
+        await fs.emptyDir(templatesDir);
+        
+        // Download templates from S3
+        await downloadTemplatesFromS3(templatesDir);
+        
+    } catch (error) {
+        console.error('Error downloading templates:', error);
+        throw error;
+    }
+}
+
+async function downloadTemplatesFromS3(targetDir) {
+    const s3Client = new S3Client({ region: 'us-east-1' });
+    const { ListObjectsV2Command, GetObjectCommand } = require('@aws-sdk/client-s3');
+    
+    const bucket = 'openkbs-downloads';
+    const prefix = 'templates/';
+    
+    // List all objects in templates folder
+    const listResponse = await s3Client.send(new ListObjectsV2Command({
+        Bucket: bucket,
+        Prefix: prefix
+    }));
+    
+    // Download all files in parallel
+    const downloadPromises = (listResponse.Contents || []).map(async (obj) => {
+        const key = obj.Key;
+        const relativePath = key.substring(prefix.length);
+        
+        // Skip if it's a directory marker
+        if (relativePath.endsWith('/')) return;
+        
+        const localPath = path.join(targetDir, relativePath);
+        
+        // Ensure directory exists
+        await fs.ensureDir(path.dirname(localPath));
+        
+        // Download file
+        const response = await s3Client.send(new GetObjectCommand({
+            Bucket: bucket,
+            Key: key
+        }));
+        
+        const fileContent = await streamToBuffer(response.Body);
+        await fs.writeFile(localPath, fileContent);
+        console.log(relativePath);
+    });
+    
+    // Wait for all downloads to complete
+    await Promise.all(downloadPromises);
+}
+
 const generateKey = (passphrase) => {
     const salt = 'salt';
     const iterations = 1000;
@@ -1054,6 +1112,9 @@ const replacePlaceholderInFiles = (dir, name) => {
 
 async function initByTemplateAction(params) {
     try {
+        // Download templates from S3 first
+        await downloadTemplates();
+        
         const targetDir = process.cwd();
 
         // Copy all files and folders, skipping existing ones
@@ -1077,5 +1138,6 @@ module.exports = {
     KB_API_URL, AUTH_API_URL, decryptKBFields, fetchLocalKBData, fetchKBJWT, createAccountIdFromPublicKey, signPayload,
     listFiles, getUserProfile, getKB, fetchAndSaveSettings, downloadIcon, downloadFiles, updateKB, uploadFiles, generateKey,
     generateMnemonic, reset, bold, red, yellow, green, cyan, createKB, getClientJWT, saveLocalKBData, listKBs, deleteKBFile,
-    deleteKB, buildPackage, replacePlaceholderInFiles, buildNodePackage, initByTemplateAction, modifyKB, listKBsSharedWithMe
+    deleteKB, buildPackage, replacePlaceholderInFiles, buildNodePackage, initByTemplateAction, modifyKB, listKBsSharedWithMe,
+    downloadTemplates
 }