viho-llm 0.1.4 → 0.1.6

package/README.md

@@ -1,6 +1,10 @@
-# viho-llm
+<p align="center">
+  <img src="https://static-small.vincentqiao.com/viho/logo.png" alt="viho logo" width="200"/>
+</p>
 
-Utility library for working with Google Gemini AI, providing common tools and helpers for AI interactions.
+<h1 align="center">viho-llm</h1>
+
+<p align="center">Utility library for working with Google Gemini AI, providing common tools and helpers for AI interactions.</p>
 
 ## Installation
 
@@ -10,17 +14,26 @@ npm install viho-llm
 
 ## Prerequisites
 
-You need to have a Google AI API key. Get one from [Google AI Studio](https://makersuite.google.com/app/apikey).
+This library supports two ways to access Google Gemini AI:
+
+1. **Google AI Studio (GeminiAPI)** - For personal development and prototyping
+   - Get an API key from [Google AI Studio](https://makersuite.google.com/app/apikey)
+
+2. **Vertex AI (GeminiVertex)** - For enterprise applications with advanced features
+   - Requires a Google Cloud project with Vertex AI enabled
+   - Supports context caching for cost optimization
 
 ## Usage
 
-### Basic Example
+### Basic Example with GeminiAPI
+
+Using Google AI Studio API Key (recommended for development):
 
 ```javascript
-import { Gemini } from 'viho-llm';
+import { GeminiAPI } from 'viho-llm';
 
-// Initialize Gemini client
-const gemini = Gemini({
+// Initialize Gemini client with API Key
+const gemini = GeminiAPI({
   apiKey: 'your-google-api-key',
   modelName: 'gemini-pro',
 });
@@ -38,17 +51,38 @@ const response = await gemini.chat({
 console.log(response);
 ```
 
-### Streaming Example
+### Basic Example with GeminiVertex
+
+Using Vertex AI (recommended for production):
 
 ```javascript
-import { Gemini } from 'viho-llm';
+import { GeminiVertex } from 'viho-llm';
 
-// Initialize Gemini client
-const gemini = Gemini({
-  apiKey: 'your-google-api-key',
+// Initialize Gemini client with Vertex AI
+const gemini = GeminiVertex({
+  projectId: 'your-gcp-project-id',
+  location: 'us-east1',
   modelName: 'gemini-pro',
 });
 
+// Send a chat message
+const response = await gemini.chat({
+  contents: [
+    {
+      role: 'user',
+      parts: [{ text: 'Hello, how are you?' }],
+    },
+  ],
+});
+
+console.log(response);
+```
+
+### Streaming Example
+
+Both GeminiAPI and GeminiVertex support streaming responses:
+
+```javascript
 // Send a chat message with streaming
 await gemini.chatWithStreaming(
   {
@@ -79,11 +113,44 @@ await gemini.chatWithStreaming(
 );
 ```
 
+### Context Caching Example (Vertex AI Only)
+
+GeminiVertex supports context caching to reduce costs and latency when using large contexts:
+
+```javascript
+import { GeminiVertex } from 'viho-llm';
+
+const gemini = GeminiVertex({
+  projectId: 'your-gcp-project-id',
+  location: 'us-east1',
+  modelName: 'gemini-1.5-flash-002',
+});
+
+// Add a new cache
+const cache = await gemini.cacheAdd({
+  gsPath: 'gs://your-bucket/large-document.pdf',
+  systemPrompt: 'You are an expert at analyzing technical documents.',
+  cacheName: 'my-document-cache',
+  cacheTTL: '3600s', // 1 hour
+});
+
+console.log('Cache created:', cache.name);
+
+// List all caches
+const caches = await gemini.cacheList();
+console.log('Available caches:', caches);
+
+// Update cache TTL
+await gemini.cacheUpdate(cache.name, {
+  ttl: '7200s', // Extend to 2 hours
+});
+```
+
 ## API Reference
 
-### `Gemini(options)`
+### `GeminiAPI(options)`
 
-Creates a new Gemini client instance.
+Creates a new Gemini client instance using Google AI Studio API.
 
 #### Parameters
 
@@ -172,6 +239,98 @@ await gemini.chatWithStreaming(
 );
 ```
 
+---
+
+### `GeminiVertex(options)`
+
+Creates a new Gemini client instance using Vertex AI. Includes all features of GeminiAPI plus context caching support.
+
+#### Parameters
+
+- `options` (Object) - Configuration options
+  - `projectId` (string) **required** - Your Google Cloud project ID
+  - `location` (string) **required** - GCP region (e.g., 'us-east1', 'us-central1')
+  - `modelName` (string) **required** - Model name (e.g., 'gemini-1.5-flash-002', 'gemini-1.5-pro-002')
+
+#### Returns
+
+Returns a Gemini client object with the following methods:
+
+##### `client.chat(chatOptions)`
+
+Same as GeminiAPI.chat(). See above for details.
+
+##### `client.chatWithStreaming(chatOptions, callbackOptions)`
+
+Same as GeminiAPI.chatWithStreaming(). See above for details.
+
+##### `client.cacheAdd(cacheOptions)`
+
+Creates a new context cache for frequently used content.
+
+**Parameters:**
+
+- `cacheOptions` (Object)
+  - `gsPath` (string) **required** - Google Cloud Storage path (e.g., 'gs://bucket/file.pdf')
+  - `systemPrompt` (string) **required** - System instruction for the cached context
+  - `cacheName` (string) **required** - Display name for the cache
+  - `cacheTTL` (string) **required** - Time-to-live (e.g., '3600s' for 1 hour)
+
+**Returns:**
+
+- (Promise\<Object\>) - Cache object with name and metadata
+
+**Example:**
+
+```javascript
+const cache = await gemini.cacheAdd({
+  gsPath: 'gs://my-bucket/documentation.pdf',
+  systemPrompt: 'You are a helpful documentation assistant.',
+  cacheName: 'docs-cache',
+  cacheTTL: '3600s',
+});
+```
+
+##### `client.cacheList()`
+
+Lists all available caches in the project.
+
+**Parameters:** None
+
+**Returns:**
+
+- (Promise\<Array\>) - Array of cache objects with `name` and `displayName` properties
+
+**Example:**
+
+```javascript
+const caches = await gemini.cacheList();
+console.log(caches);
+// [{ name: 'projects/.../cachedContents/...', displayName: 'docs-cache' }]
+```
+
+##### `client.cacheUpdate(cacheName, cacheOptions)`
+
+Updates an existing cache configuration.
+
+**Parameters:**
+
+- `cacheName` (string) **required** - The cache name to update
+- `cacheOptions` (Object) **required** - Update configuration
+  - `ttl` (string) - New time-to-live value (e.g., '7200s')
+
+**Returns:**
+
+- (Promise\<Object\>) - Updated cache object
+
+**Example:**
+
+```javascript
+await gemini.cacheUpdate('projects/.../cachedContents/abc123', {
+  ttl: '7200s', // Extend to 2 hours
+});
+```
+
 ## License
 
 MIT

package/index.js

@@ -5,94 +5,88 @@ var mime = require('mime-types');
 var qiao_log_js = require('qiao.log.js');
 
 // gemini
-const logger = qiao_log_js.Logger('viho-llm');
+const logger$2 = qiao_log_js.Logger('gemini-util.js');
 
 /**
- * Gemini
- * @param {*} options
+ * chat
+ * @param {*} client
+ * @param {*} modelName
+ * @param {*} chatOptions
  * @returns
  */
-const Gemini = (options) => {
-  const methodName = 'Gemini';
+const chat = async (client, modelName, chatOptions) => {
+  const methodName = 'chat';
 
   // check
-  if (!options) {
-    logger.error(methodName, 'need options');
-    return;
-  }
-  if (!options.apiKey) {
-    logger.error(methodName, 'need options.apiKey');
+  if (!client) {
+    logger$2.error(methodName, 'need client');
     return;
   }
-  if (!options.modelName) {
-    logger.error(methodName, 'need options.modelName');
+  if (!modelName) {
+    logger$2.error(methodName, 'need modelName');
     return;
   }
-
-  // gemini
-  const gemini = {};
-  gemini.client = new genai.GoogleGenAI({
-    vertexai: true,
-    apiKey: options.apiKey,
-  });
-
-  // chat
-  gemini.chat = async (chatOptions) => {
-    return await chat(gemini.client, options.modelName, chatOptions);
-  };
-  gemini.chatWithStreaming = async (chatOptions, callbackOptions) => {
-    return await chatWithStreaming(gemini.client, options.modelName, chatOptions, callbackOptions);
-  };
-
-  // cache
-  gemini.cacheAdd = async (cacheOptions) => {
-    return await cacheAdd(gemini.client, options.modelName, cacheOptions);
-  };
-
-  // r
-  return gemini;
-};
-
-// chat
-async function chat(client, modelName, chatOptions) {
-  const methodName = 'Gemini - chat';
-
-  // check
   if (!chatOptions) {
-    logger.error(methodName, 'need chatOptions');
+    logger$2.error(methodName, 'need chatOptions');
     return;
   }
   if (!chatOptions.contents) {
-    logger.error(methodName, 'need chatOptions.contents');
+    logger$2.error(methodName, 'need chatOptions.contents');
     return;
   }
 
   try {
-    const response = await client.models.generateContent({
-      model: modelName,
-      contents: chatOptions.contents,
-    });
+    // options
+    const options = Object.assign(
+      {
+        model: modelName,
+      },
+      chatOptions,
+    );
+
+    // gen
+    const response = await client.models.generateContent(options);
     if (!response || !response.text) {
-      logger.error(methodName, 'invalid response');
+      logger$2.error(methodName, 'invalid response');
       return;
     }
 
     return response.text;
   } catch (error) {
-    logger.error(methodName, 'error', error);
+    logger$2.error(methodName, 'error', error);
   }
-}
+};
 
-async function chatWithStreaming(client, modelName, chatOptions, callbackOptions) {
-  const methodName = 'Gemini - chatWithStreaming';
+/**
+ * chatWithStreaming
+ * @param {*} client
+ * @param {*} modelName
+ * @param {*} chatOptions
+ * @param {*} callbackOptions
+ * @returns
+ */
+const chatWithStreaming = async (client, modelName, chatOptions, callbackOptions) => {
+  const methodName = 'chatWithStreaming';
 
   // check
+  if (!client) {
+    logger$2.error(methodName, 'need client');
+    return;
+  }
+  if (!modelName) {
+    logger$2.error(methodName, 'need modelName');
+    return;
+  }
   if (!chatOptions) {
-    logger.error(methodName, 'need chatOptions');
+    logger$2.error(methodName, 'need chatOptions');
     return;
   }
   if (!chatOptions.contents) {
-    logger.error(methodName, 'need chatOptions.contents');
+    logger$2.error(methodName, 'need chatOptions.contents');
+    return;
+  }
+  if (!callbackOptions) {
+    logger$2.error(methodName, 'need callbackOptions');
     return;
   }
 
@@ -104,11 +98,19 @@ async function chatWithStreaming(client, modelName, chatOptions, callbackOptions
   const firstContentCallback = callbackOptions.firstContentCallback;
 
   try {
+    // begin
     if (beginCallback) beginCallback();
-    const response = await client.models.generateContentStream({
-      model: modelName,
-      contents: chatOptions.contents,
-    });
+
+    // options
+    const options = Object.assign(
+      {
+        model: modelName,
+      },
+      chatOptions,
+    );
+
+    // gen
+    const response = await client.models.generateContentStream(options);
 
     // go
     let firstContent = true;
@@ -130,52 +132,59 @@ async function chatWithStreaming(client, modelName, chatOptions, callbackOptions
   } catch (error) {
     if (errorCallback) errorCallback(error);
   }
-}
+};
 
-// cache add
-async function cacheAdd(client, modelName, cacheOptions) {
-  const methodName = 'Gemini - cacheAdd';
+/**
+ * cacheAdd
+ * @param {*} client
+ * @param {*} modelName
+ * @param {*} cacheOptions
+ * @returns
+ */
+const cacheAdd = async (client, modelName, cacheOptions) => {
+  const methodName = 'cacheAdd';
 
   // check
+  if (!client) {
+    logger$2.error(methodName, 'need client');
+    return;
+  }
+  if (!modelName) {
+    logger$2.error(methodName, 'need modelName');
+    return;
+  }
   if (!cacheOptions) {
-    logger.error(methodName, 'need cacheOptions');
+    logger$2.error(methodName, 'need cacheOptions');
     return;
   }
-  if (!cacheOptions.filePath) {
-    logger.error(methodName, 'need cacheOptions.filePath');
+  if (!cacheOptions.gsPath) {
+    logger$2.error(methodName, 'need cacheOptions.gsPath');
     return;
   }
   if (!cacheOptions.systemPrompt) {
-    logger.error(methodName, 'need cacheOptions.systemPrompt');
+    logger$2.error(methodName, 'need cacheOptions.systemPrompt');
     return;
   }
   if (!cacheOptions.cacheName) {
-    logger.error(methodName, 'need cacheOptions.cacheName');
+    logger$2.error(methodName, 'need cacheOptions.cacheName');
     return;
   }
   if (!cacheOptions.cacheTTL) {
-    logger.error(methodName, 'need cacheOptions.cacheTTL');
+    logger$2.error(methodName, 'need cacheOptions.cacheTTL');
     return;
   }
 
   // const
-  const mimeType = mime.lookup(cacheOptions.filePath);
-  logger.info(methodName, 'cacheOptions', cacheOptions);
-  logger.info(methodName, 'mimeType', mimeType);
+  const mimeType = mime.lookup(cacheOptions.gsPath);
+  logger$2.info(methodName, 'cacheOptions', cacheOptions);
+  logger$2.info(methodName, 'mimeType', mimeType);
 
   try {
-    // upload doc
-    const doc = await client.files.upload({
-      file: cacheOptions.filePath,
-      config: { mimeType: mimeType },
-    });
-    logger.info(methodName, 'doc.name', doc.name);
-
     // cache add
     const cache = await client.caches.create({
       model: modelName,
       config: {
-        contents: genai.createUserContent(genai.createPartFromUri(doc.uri, doc.mimeType)),
+        contents: genai.createUserContent(genai.createPartFromUri(cacheOptions.gsPath, mimeType)),
         systemInstruction: cacheOptions.systemPrompt,
         displayName: cacheOptions.cacheName,
         ttl: cacheOptions.cacheTTL,
@@ -184,8 +193,177 @@ async function cacheAdd(client, modelName, cacheOptions) {
 
     return cache;
   } catch (error) {
-    logger.error(methodName, 'error', error);
+    logger$2.error(methodName, 'error', error);
+  }
+};
+
+/**
+ * cacheList
+ * @param {*} client
+ * @returns
+ */
+const cacheList = async (client) => {
+  const methodName = 'cacheList';
+
+  // check
+  if (!client) {
+    logger$2.error(methodName, 'need client');
+    return;
+  }
+
+  // cache list
+  try {
+    const cacheList = await client.caches.list();
+    const cacheObjs = cacheList?.pageInternal?.map((contentCache) => ({
+      name: contentCache.name,
+      displayName: contentCache.displayName,
+    }));
+
+    return cacheObjs;
+  } catch (error) {
+    logger$2.error(methodName, 'error', error);
   }
-}
+};
+
+/**
+ * cacheUpdate
+ * @param {*} client
+ * @param {*} cacheName
+ * @param {*} cacheOptions
+ * @returns
+ */
+const cacheUpdate = async (client, cacheName, cacheOptions) => {
+  const methodName = 'cacheUpdate';
+
+  // check
+  if (!client) {
+    logger$2.error(methodName, 'need client');
+    return;
+  }
+  if (!cacheName) {
+    logger$2.error(methodName, 'need cacheName');
+    return;
+  }
+  if (!cacheOptions) {
+    logger$2.error(methodName, 'need cacheOptions');
+    return;
+  }
+
+  // cache update
+  try {
+    const res = await client.caches.update({
+      name: cacheName,
+      config: cacheOptions,
+    });
+
+    return res;
+  } catch (error) {
+    logger$2.error(methodName, 'error', error);
+  }
+};
+
+// gemini
+const logger$1 = qiao_log_js.Logger('gemini-api.js');
+
+/**
+ * GeminiAPI
+ * @param {*} options
+ * @returns
+ */
+const GeminiAPI = (options) => {
+  const methodName = 'GeminiAPI';
+
+  // check
+  if (!options) {
+    logger$1.error(methodName, 'need options');
+    return;
+  }
+  if (!options.apiKey) {
+    logger$1.error(methodName, 'need options.apiKey');
+    return;
+  }
+  if (!options.modelName) {
+    logger$1.error(methodName, 'need options.modelName');
+    return;
+  }
+
+  // gemini
+  const gemini = {};
+  gemini.client = new genai.GoogleGenAI({
+    apiKey: options.apiKey,
+  });
+
+  // chat
+  gemini.chat = async (chatOptions) => {
+    return await chat(gemini.client, options.modelName, chatOptions);
+  };
+  gemini.chatWithStreaming = async (chatOptions, callbackOptions) => {
+    return await chatWithStreaming(gemini.client, options.modelName, chatOptions, callbackOptions);
+  };
+
+  // r
+  return gemini;
+};
+
+// gemini
+const logger = qiao_log_js.Logger('viho-llm');
+
+/**
+ * GeminiVertex
+ * @param {*} options
+ * @returns
+ */
+const GeminiVertex = (options) => {
+  const methodName = 'GeminiVertex';
+
+  // check
+  if (!options) {
+    logger.error(methodName, 'need options');
+    return;
+  }
+  if (!options.projectId) {
+    logger.error(methodName, 'need options.projectId');
+    return;
+  }
+  if (!options.location) {
+    logger.error(methodName, 'need options.location');
+    return;
+  }
+  if (!options.modelName) {
+    logger.error(methodName, 'need options.modelName');
+    return;
+  }
+
+  // gemini
+  const gemini = {};
+  gemini.client = new genai.GoogleGenAI({
+    vertexai: true,
+    project: options.projectId,
+    location: options.location,
+  });
+
+  // chat
+  gemini.chat = async (chatOptions) => {
+    return await chat(gemini.client, options.modelName, chatOptions);
+  };
+  gemini.chatWithStreaming = async (chatOptions, callbackOptions) => {
+    return await chatWithStreaming(gemini.client, options.modelName, chatOptions, callbackOptions);
+  };
+
+  // cache
+  gemini.cacheAdd = async (cacheOptions) => {
+    return await cacheAdd(gemini.client, options.modelName, cacheOptions);
+  };
+  gemini.cacheList = async () => {
+    return await cacheList(gemini.client);
+  };
+  gemini.cacheUpdate = async (cacheName, cacheOptions) => {
+    return await cacheUpdate(gemini.client, cacheName, cacheOptions);
+  };
+
+  // r
+  return gemini;
+};
 
-exports.Gemini = Gemini;
+exports.GeminiAPI = GeminiAPI;
+exports.GeminiVertex = GeminiVertex;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "viho-llm",
-  "version": "0.1.4",
+  "version": "0.1.6",
   "description": "Utility library for working with Google Gemini AI, providing common tools and helpers for AI interactions",
   "keywords": [
     "llm",
@@ -61,5 +61,5 @@
       }
     }
   },
-  "gitHead": "38e28266271d8b64e56c5f6d27deca78ce8cdb1b"
+  "gitHead": "d77d2ba692eac3cd5262f55ade7cf74b84172385"
 }

package/src/index.js

@@ -1 +1,2 @@
-export * from './gemini.js';
+export * from './models/gemini-api.js';
+export * from './models/gemini-vertex.js';

package/src/models/gemini-api.js

@@ -0,0 +1,49 @@
+// gemini
+import { GoogleGenAI } from '@google/genai';
+
+// util
+import { chat, chatWithStreaming } from './gemini-util.js';
+
+// Logger
+import { Logger } from 'qiao.log.js';
+const logger = Logger('gemini-api.js');
+
+/**
+ * GeminiAPI
+ * @param {*} options
+ * @returns
+ */
+export const GeminiAPI = (options) => {
+  const methodName = 'GeminiAPI';
+
+  // check
+  if (!options) {
+    logger.error(methodName, 'need options');
+    return;
+  }
+  if (!options.apiKey) {
+    logger.error(methodName, 'need options.apiKey');
+    return;
+  }
+  if (!options.modelName) {
+    logger.error(methodName, 'need options.modelName');
+    return;
+  }
+
+  // gemini
+  const gemini = {};
+  gemini.client = new GoogleGenAI({
+    apiKey: options.apiKey,
+  });
+
+  // chat
+  gemini.chat = async (chatOptions) => {
+    return await chat(gemini.client, options.modelName, chatOptions);
+  };
+  gemini.chatWithStreaming = async (chatOptions, callbackOptions) => {
+    return await chatWithStreaming(gemini.client, options.modelName, chatOptions, callbackOptions);
+  };
+
+  // r
+  return gemini;
+};

package/src/models/gemini-util.js

@@ -0,0 +1,264 @@
+// gemini
+import { createUserContent, createPartFromUri } from '@google/genai';
+
+// mime
+import mime from 'mime-types';
+
+// Logger
+import { Logger } from 'qiao.log.js';
+const logger = Logger('gemini-util.js');
+
+/**
+ * chat
+ * @param {*} client
+ * @param {*} modelName
+ * @param {*} chatOptions
+ * @returns
+ */
+export const chat = async (client, modelName, chatOptions) => {
+  const methodName = 'chat';
+
+  // check
+  if (!client) {
+    logger.error(methodName, 'need client');
+    return;
+  }
+  if (!modelName) {
+    logger.error(methodName, 'need modelName');
+    return;
+  }
+  if (!chatOptions) {
+    logger.error(methodName, 'need chatOptions');
+    return;
+  }
+  if (!chatOptions.contents) {
+    logger.error(methodName, 'need chatOptions.contents');
+    return;
+  }
+
+  try {
+    // options
+    const options = Object.assign(
+      {
+        model: modelName,
+      },
+      chatOptions,
+    );
+
+    // gen
+    const response = await client.models.generateContent(options);
+    if (!response || !response.text) {
+      logger.error(methodName, 'invalid response');
+      return;
+    }
+
+    return response.text;
+  } catch (error) {
+    logger.error(methodName, 'error', error);
+  }
+};
+
+/**
+ * chatWithStreaming
+ * @param {*} client
+ * @param {*} modelName
+ * @param {*} chatOptions
+ * @param {*} callbackOptions
+ * @returns
+ */
+export const chatWithStreaming = async (client, modelName, chatOptions, callbackOptions) => {
+  const methodName = 'chatWithStreaming';
+
+  // check
+  if (!client) {
+    logger.error(methodName, 'need client');
+    return;
+  }
+  if (!modelName) {
+    logger.error(methodName, 'need modelName');
+    return;
+  }
+  if (!chatOptions) {
+    logger.error(methodName, 'need chatOptions');
+    return;
+  }
+  if (!chatOptions.contents) {
+    logger.error(methodName, 'need chatOptions.contents');
+    return;
+  }
+  if (!callbackOptions) {
+    logger.error(methodName, 'need callbackOptions');
+    return;
+  }
+
+  // callback
+  const beginCallback = callbackOptions.beginCallback;
+  const endCallback = callbackOptions.endCallback;
+  const errorCallback = callbackOptions.errorCallback;
+  const contentCallback = callbackOptions.contentCallback;
+  const firstContentCallback = callbackOptions.firstContentCallback;
+
+  try {
+    // begin
+    if (beginCallback) beginCallback();
+
+    // options
+    const options = Object.assign(
+      {
+        model: modelName,
+      },
+      chatOptions,
+    );
+
+    // gen
+    const response = await client.models.generateContentStream(options);
+
+    // go
+    let firstContent = true;
+    for await (const chunk of response) {
+      // content
+      const content = chunk.text;
+      if (content && contentCallback) {
+        if (firstContent && firstContentCallback) {
+          firstContent = false;
+          firstContentCallback();
+        }
+
+        contentCallback(content);
+      }
+    }
+
+    // end
+    if (endCallback) endCallback();
+  } catch (error) {
+    if (errorCallback) errorCallback(error);
+  }
+};
+
+/**
+ * cacheAdd
+ * @param {*} client
+ * @param {*} modelName
+ * @param {*} cacheOptions
+ * @returns
+ */
+export const cacheAdd = async (client, modelName, cacheOptions) => {
+  const methodName = 'cacheAdd';
+
+  // check
+  if (!client) {
+    logger.error(methodName, 'need client');
+    return;
+  }
+  if (!modelName) {
+    logger.error(methodName, 'need modelName');
+    return;
+  }
+  if (!cacheOptions) {
+    logger.error(methodName, 'need cacheOptions');
+    return;
+  }
+  if (!cacheOptions.gsPath) {
+    logger.error(methodName, 'need cacheOptions.gsPath');
+    return;
+  }
+  if (!cacheOptions.systemPrompt) {
+    logger.error(methodName, 'need cacheOptions.systemPrompt');
+    return;
+  }
+  if (!cacheOptions.cacheName) {
+    logger.error(methodName, 'need cacheOptions.cacheName');
+    return;
+  }
+  if (!cacheOptions.cacheTTL) {
+    logger.error(methodName, 'need cacheOptions.cacheTTL');
+    return;
+  }
+
+  // const
+  const mimeType = mime.lookup(cacheOptions.gsPath);
+  logger.info(methodName, 'cacheOptions', cacheOptions);
+  logger.info(methodName, 'mimeType', mimeType);
+
+  try {
+    // cache add
+    const cache = await client.caches.create({
+      model: modelName,
+      config: {
+        contents: createUserContent(createPartFromUri(cacheOptions.gsPath, mimeType)),
+        systemInstruction: cacheOptions.systemPrompt,
+        displayName: cacheOptions.cacheName,
+        ttl: cacheOptions.cacheTTL,
+      },
+    });
+
+    return cache;
+  } catch (error) {
+    logger.error(methodName, 'error', error);
+  }
+};
+
+/**
+ * cacheList
+ * @param {*} client
+ * @returns
+ */
+export const cacheList = async (client) => {
+  const methodName = 'cacheList';
+
+  // check
+  if (!client) {
+    logger.error(methodName, 'need client');
+    return;
+  }
+
+  // cache list
+  try {
+    const cacheList = await client.caches.list();
+    const cacheObjs = cacheList?.pageInternal?.map((contentCache) => ({
+      name: contentCache.name,
+      displayName: contentCache.displayName,
+    }));
+
+    return cacheObjs;
+  } catch (error) {
+    logger.error(methodName, 'error', error);
+  }
+};
+
+/**
+ * cacheUpdate
+ * @param {*} client
+ * @param {*} cacheName
+ * @param {*} cacheOptions
+ * @returns
+ */
+export const cacheUpdate = async (client, cacheName, cacheOptions) => {
+  const methodName = 'cacheUpdate';
+
+  // check
+  if (!client) {
+    logger.error(methodName, 'need client');
+    return;
+  }
+  if (!cacheName) {
+    logger.error(methodName, 'need cacheName');
+    return;
+  }
+  if (!cacheOptions) {
+    logger.error(methodName, 'need cacheOptions');
+    return;
+  }
+
+  // cache update
+  try {
+    const res = await client.caches.update({
+      name: cacheName,
+      config: cacheOptions,
+    });
+
+    return res;
+  } catch (error) {
+    logger.error(methodName, 'error', error);
+  }
+};

package/src/models/gemini-vertex.js

@@ -0,0 +1,66 @@
+// gemini
+import { GoogleGenAI } from '@google/genai';
+
+// util
+import { chat, chatWithStreaming, cacheAdd, cacheList, cacheUpdate } from './gemini-util.js';
+
+// Logger
+import { Logger } from 'qiao.log.js';
+const logger = Logger('viho-llm');
+
+/**
+ * GeminiVertex
+ * @param {*} options
+ * @returns
+ */
+export const GeminiVertex = (options) => {
+  const methodName = 'GeminiVertex';
+
+  // check
+  if (!options) {
+    logger.error(methodName, 'need options');
+    return;
+  }
+  if (!options.projectId) {
+    logger.error(methodName, 'need options.projectId');
+    return;
+  }
+  if (!options.location) {
+    logger.error(methodName, 'need options.location');
+    return;
+  }
+  if (!options.modelName) {
+    logger.error(methodName, 'need options.modelName');
+    return;
+  }
+
+  // gemini
+  const gemini = {};
+  gemini.client = new GoogleGenAI({
+    vertexai: true,
+    project: options.projectId,
+    location: options.location,
+  });
+
+  // chat
+  gemini.chat = async (chatOptions) => {
+    return await chat(gemini.client, options.modelName, chatOptions);
+  };
+  gemini.chatWithStreaming = async (chatOptions, callbackOptions) => {
+    return await chatWithStreaming(gemini.client, options.modelName, chatOptions, callbackOptions);
+  };
+
+  // cache
+  gemini.cacheAdd = async (cacheOptions) => {
+    return await cacheAdd(gemini.client, options.modelName, cacheOptions);
+  };
+  gemini.cacheList = async () => {
+    return await cacheList(gemini.client);
+  };
+  gemini.cacheUpdate = async (cacheName, cacheOptions) => {
+    return await cacheUpdate(gemini.client, cacheName, cacheOptions);
+  };
+
+  // r
+  return gemini;
+};

package/src/gemini.js

@@ -1,190 +0,0 @@
-// gemini
-import { GoogleGenAI, createUserContent, createPartFromUri } from '@google/genai';
-
-// mime
-import mime from 'mime-types';
-
-// Logger
-import { Logger } from 'qiao.log.js';
-const logger = Logger('viho-llm');
-
-/**
- * Gemini
- * @param {*} options
- * @returns
- */
-export const Gemini = (options) => {
-  const methodName = 'Gemini';
-
-  // check
-  if (!options) {
-    logger.error(methodName, 'need options');
-    return;
-  }
-  if (!options.apiKey) {
-    logger.error(methodName, 'need options.apiKey');
-    return;
-  }
-  if (!options.modelName) {
-    logger.error(methodName, 'need options.modelName');
-    return;
-  }
-
-  // gemini
-  const gemini = {};
-  gemini.client = new GoogleGenAI({
-    vertexai: true,
-    apiKey: options.apiKey,
-  });
-
-  // chat
-  gemini.chat = async (chatOptions) => {
-    return await chat(gemini.client, options.modelName, chatOptions);
-  };
-  gemini.chatWithStreaming = async (chatOptions, callbackOptions) => {
-    return await chatWithStreaming(gemini.client, options.modelName, chatOptions, callbackOptions);
-  };
-
-  // cache
-  gemini.cacheAdd = async (cacheOptions) => {
-    return await cacheAdd(gemini.client, options.modelName, cacheOptions);
-  };
-
-  // r
-  return gemini;
-};
-
-// chat
-async function chat(client, modelName, chatOptions) {
-  const methodName = 'Gemini - chat';
-
-  // check
-  if (!chatOptions) {
-    logger.error(methodName, 'need chatOptions');
-    return;
-  }
-  if (!chatOptions.contents) {
-    logger.error(methodName, 'need chatOptions.contents');
-    return;
-  }
-
-  try {
-    const response = await client.models.generateContent({
-      model: modelName,
-      contents: chatOptions.contents,
-    });
-    if (!response || !response.text) {
-      logger.error(methodName, 'invalid response');
-      return;
-    }
-
-    return response.text;
-  } catch (error) {
-    logger.error(methodName, 'error', error);
-  }
-}
-
-async function chatWithStreaming(client, modelName, chatOptions, callbackOptions) {
-  const methodName = 'Gemini - chatWithStreaming';
-
-  // check
-  if (!chatOptions) {
-    logger.error(methodName, 'need chatOptions');
-    return;
-  }
-  if (!chatOptions.contents) {
-    logger.error(methodName, 'need chatOptions.contents');
-    return;
-  }
-
-  // callback
-  const beginCallback = callbackOptions.beginCallback;
-  const endCallback = callbackOptions.endCallback;
-  const errorCallback = callbackOptions.errorCallback;
-  const contentCallback = callbackOptions.contentCallback;
-  const firstContentCallback = callbackOptions.firstContentCallback;
-
-  try {
-    if (beginCallback) beginCallback();
-    const response = await client.models.generateContentStream({
-      model: modelName,
-      contents: chatOptions.contents,
-    });
-
-    // go
-    let firstContent = true;
-    for await (const chunk of response) {
-      // content
-      const content = chunk.text;
-      if (content && contentCallback) {
-        if (firstContent && firstContentCallback) {
-          firstContent = false;
-          firstContentCallback();
-        }
-
-        contentCallback(content);
-      }
-    }
-
-    // end
-    if (endCallback) endCallback();
-  } catch (error) {
-    if (errorCallback) errorCallback(error);
-  }
-}
-
-// cache add
-async function cacheAdd(client, modelName, cacheOptions) {
-  const methodName = 'Gemini - cacheAdd';
-
-  // check
-  if (!cacheOptions) {
-    logger.error(methodName, 'need cacheOptions');
-    return;
-  }
-  if (!cacheOptions.filePath) {
-    logger.error(methodName, 'need cacheOptions.filePath');
-    return;
-  }
-  if (!cacheOptions.systemPrompt) {
-    logger.error(methodName, 'need cacheOptions.systemPrompt');
-    return;
-  }
-  if (!cacheOptions.cacheName) {
-    logger.error(methodName, 'need cacheOptions.cacheName');
-    return;
-  }
-  if (!cacheOptions.cacheTTL) {
-    logger.error(methodName, 'need cacheOptions.cacheTTL');
-    return;
-  }
-
-  // const
-  const mimeType = mime.lookup(cacheOptions.filePath);
-  logger.info(methodName, 'cacheOptions', cacheOptions);
-  logger.info(methodName, 'mimeType', mimeType);
-
-  try {
-    // upload doc
-    const doc = await client.files.upload({
-      file: cacheOptions.filePath,
-      config: { mimeType: mimeType },
-    });
-    logger.info(methodName, 'doc.name', doc.name);
-
-    // cache add
-    const cache = await client.caches.create({
-      model: modelName,
-      config: {
-        contents: createUserContent(createPartFromUri(doc.uri, doc.mimeType)),
-        systemInstruction: cacheOptions.systemPrompt,
-        displayName: cacheOptions.cacheName,
-        ttl: cacheOptions.cacheTTL,
-      },
-    });
-
-    return cache;
-  } catch (error) {
-    logger.error(methodName, 'error', error);
-  }
-}