@adaptic/lumic-utils 1.0.1

package/README.md

@@ -0,0 +1,637 @@
+# lumic-utility-functions
+
+lumic-utility-functions is a versatile npm package that provides a comprehensive collection of utility functions for Node.js projects. It simplifies common tasks including:
+- Making LLM API calls (unified interface for OpenAI models)
+- Managing AWS services (Lambda, S3)
+- Sending Slack messages
+- Handling file operations (ZIP files, S3 storage)
+- Working with JSON data
+- Interacting with Google Sheets
+
+## Installation
+
+```bash
+npm install lumic-utility-functions
+```
+
+## Usage
+
+Import the lumic namespace:
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+// Access functions through their namespaces
+const openaiResult = await lumic.llm.call("Your prompt", "text", { model: "GPT-5-mini" }); // OpenAI model
+const deepseekResult = await lumic.llm.seek("Your prompt", "json"); // Deepseek model
+const s3Result = await lumic.s3.upload("bucket", "file.txt");
+const slackResult = await lumic.slack.sendMessage("#channel", "message");
+const fixedJson = lumic.json.fix(brokenJsonString);
+const gsResult = await lumic.gs.addRow({ spreadsheetId: "id", sheetName: "Sheet1" }, ["Value1", "Value2"]);
+```
+
+## Required Environment Variables
+
+The following environment variables are recommended for full functionality:
+
+* `MY_AWS_ACCESS_KEY_ID` - For AWS services
+* `MY_AWS_REGION` - For AWS services
+* `MY_AWS_SECRET_ACCESS_KEY` - For AWS services
+* `OPENAI_API_KEY` - For OpenAI models
+* `DEEPSEEK_API_KEY` - For Deepseek models
+* `SLACK_BOT_TOKEN` - For Slack messaging
+* `GOOGLE_SHEETS_CLIENT_EMAIL` - For Google Sheets service account
+* `GOOGLE_SHEETS_PRIVATE_KEY` - For Google Sheets service account
+* `AI_MODEL` - Default model for LLM calls (optional, defaults to 'GPT-5-mini')
+
+Most functions also accept explicit authentication parameters if you prefer not to use environment variables.
+
+### Table of Contents
+- [Installation](#installation)
+- [Usage](#usage)
+- [LLM Interface](#llm-interface)
+- [sendMessageToSlack](#sendmessagetoslack)
+- [callPerplexityAPI](#callperplexityapi)
+- [invokeLambdaFunction](#invokelambdafunction)
+- [Zip File Utilities](#zip-file-utilities)
+- [AWS S3 Utilities](#aws-s3-utilities)
+- [JSON Utilities](#json-utilities)
+- [Google Sheets Utilities](#google-sheets-utilities)
+- [Performance Timer](#performance-timer)
+
+## LLM Interface
+
+The package provides a unified interface for interacting with various LLM providers through the `lumic.llm.call` function.
+
+### Supported Models
+
+#### OpenAI Models
+- `GPT-5-mini`: Optimized GPT-4 model for general use (default)
+- `GPT-5`: Full GPT-4 model with enhanced capabilities
+- `o1-mini`: Compact O1 model
+- `o1`: Full O1 model
+- `o3-mini`: Compact O3 model
+- `gpt-4.1`: GPT-4.1 model
+- `gpt-4.1-mini`: Compact GPT-4.1 model
+- `o4-mini`: Compact O4 model
+
+#### Deepseek Models
+- `deepseek-chat`: Versatile chat model with JSON output and function calling support (via `lumic.llm.seek`)
+- `deepseek-reasoner`: Advanced reasoning model for complex tasks (via `lumic.llm.seek`)
+
+### Input Parameters
+
+- `content` (string | ContentPart[]): The content to send to the LLM. Can be either:
+  - A string: Traditional text prompt
+  - An array of content parts: For multi-modal inputs, supporting:
+    - Text content: `{ type: 'text', text: string }`
+    - Image content: `{ type: 'image_url', image_url: { url: string, detail?: 'auto' | 'low' | 'high' } }`
+
+- `responseFormat` (string | object): The desired format of the response. Can be:
+  - "text": Returns plain text response
+  - "json": Returns parsed JSON object with advanced parsing strategies
+  - JSON schema object: Returns parsed object conforming to the provided schema
+
+- `options` (object, optional): Additional options for the API call
+  - `model` (string, optional): The model to use (default: "GPT-5-mini" for OpenAI, "deepseek-chat" for Deepseek)
+  - `developerPrompt` (string, optional): Custom developer context prompt
+  - `apiKey` (string, optional): Custom API key
+  - `reasoning_effort` (string, optional): Specify reasoning depth ('low' | 'medium' | 'high')
+  - `temperature` (number, optional): Controls randomness of the output
+  - `top_p` (number, optional): Controls diversity of token selection
+  - `frequency_penalty` (number, optional): Reduces repetition of tokens
+  - `presence_penalty` (number, optional): Encourages discussing new topics
+  - `max_completion_tokens` (number, optional): Limits response length
+  - `store` (boolean, optional): Option to store conversation
+  - `metadata` (object, optional): Additional metadata for stored conversations
+  - `tools` (Tool[], optional): Function calling tools
+
+### Response Format
+
+All responses follow a consistent format:
+```typescript
+interface LLMResponse<T> {
+  response: T;                 // The model's response
+  usage: {
+    prompt_tokens: number;     // Number of tokens in the prompt
+    completion_tokens: number; // Number of tokens in the completion
+    reasoning_tokens?: number; // Number of tokens for reasoning (Deepseek models)
+    provider: string;         // 'openai' or 'deepseek'
+    model: string;            // The model name (e.g., 'GPT-5-mini', 'deepseek-chat')
+    cache_hit_tokens?: number; // Cache hit tokens for Deepseek models
+    cost: number;             // Cost of the API call
+  };
+  tool_calls?: ToolCall[];    // Optional tool calls if tools were provided
+}
+```
+
+### Examples
+
+1. Basic text completion with OpenAI:
+```javascript
+const result = await lumic.llm.call(
+  "What is the capital of France?",
+  "text",
+  { model: "GPT-5-mini" }
+);
+```
+
+2. JSON output:
+```javascript
+const result = await lumic.llm.call(
+  "List three famous scientists in JSON format",
+  "json"
+);
+```
+
+3. Using function calling:
+```javascript
+const weatherTool = {
+  type: 'function',
+  function: {
+    name: 'get_weather',
+    description: 'Get the current weather for a location',
+    parameters: {
+      type: 'object',
+      properties: {
+        location: {
+          type: 'string',
+          description: 'The city name',
+        },
+      },
+      required: ['location'],
+    },
+  },
+};
+
+const result = await lumic.llm.call(
+  "What's the weather like in Tokyo?",
+  "text",
+  {
+    model: "GPT-5-mini",
+    tools: [weatherTool],
+  }
+);
+```
+
+4. Using Deepseek models:
+```javascript
+// Using Deepseek chat model for JSON output
+const result = await lumic.llm.seek(
+  "List three famous scientists in JSON format",
+  "json"
+);
+
+// Using Deepseek reasoner model for complex reasoning tasks
+const result = await lumic.llm.seek(
+  "Explain quantum entanglement in simple terms",
+  "text",
+  { model: "deepseek-reasoner" }
+);
+```
+
+### Model Features and Compatibility
+
+| Feature | OpenAI Models | Deepseek Chat | Deepseek Reasoner |
+|---------|---------------|---------------|-------------------|
+| Text output | ✅ | ✅ | ✅ |
+| JSON output | ✅ | ✅ | ❌ |
+| Function calling | ✅ | ✅ | ❌ |
+| Context length | Up to 128K | 64K | 64K |
+| Cost | See [OpenAI pricing](https://openai.com/pricing) | $0.27/1M input tokens, $1.10/1M output tokens | $0.55/1M input tokens, $2.19/1M output tokens |
+
+Note: When attempting to use unsupported features with a model (e.g., JSON output with Deepseek Reasoner), the API will return a graceful error response with details about the incompatibility.
+
+## sendMessageToSlack
+
+The `sendMessageToSlack` function is designed to send a message to a specified Slack channel.
+
+### Input Parameters
+
+- `channel` (string): The Slack channel to send the message to.
+- `message` (string): The message content to send.
+
+### Usage Example
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+const success = await lumic.slack.sendMessage("#general", "Hello, Slack!");
+console.log(success); // true if message was sent successfully
+```
+
+## callPerplexityAPI
+
+The `callPerplexityAPI` function is designed to send a query to the Perplexity API and return the response.
+
+### Input Parameters
+
+- `query` (string): The text query to send to the Perplexity API.
+- `maxTokens` (number, optional): The maximum number of tokens for the response (default: 4000).
+- `apiKey` (string, optional): The API key to use for the Perplexity API (defaults to the configured PERPLEXITY_API_KEY).
+
+### Usage Example
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+const result = await lumic.perplexity.call("What is the capital of France?");
+console.log(result); // "The capital of France is Paris."
+```
+
+## invokeLambdaFunction
+
+The `invokeLambdaFunction` function is designed to invoke an AWS Lambda function.
+
+### Input Parameters
+
+- `functionName` (string): The name of the Lambda function to invoke.
+- `payload` (object, optional): The payload to send to the Lambda function (default: empty object).
+- `invocationType` (string, optional): The type of invocation (default: 'RequestResponse').
+- `auth` (object, optional): AWS credentials for Lambda invocation. If not provided, falls back to environment variables.
+
+### Usage Example
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+// Using environment variables
+const result = await lumic.lambda.invoke(
+  "myLambdaFunction",
+  { key: "value" }
+);
+
+// Using explicit auth credentials
+const resultWithAuth = await lumic.lambda.invoke(
+  "myLambdaFunction",
+  { key: "value" },
+  'RequestResponse',
+  {
+    AWS_ACCESS_KEY_ID: 'your-access-key',
+    AWS_SECRET_ACCESS_KEY: 'your-secret-key',
+    AWS_REGION: 'us-west-2'
+  }
+);
+```
+
+## Zip File Utilities
+
+The package includes a set of file management utilities to work with zip files, along with other general utility functions.
+
+### createZipFile
+
+The `createZipFile` function compresses a directory into a zip file.
+
+### Input Parameters
+
+- `sourceDir` (string): The path of the directory to compress. It must start with `/tmp/` for security purposes.
+
+### Usage Example
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+const zipFilePath = await lumic.zip.create("/tmp/myDirectory");
+console.log(`Zip file created at: ${zipFilePath}`);
+```
+
+### extractZipFile
+
+The `extractZipFile` function extracts a zip archive into a specified directory.
+
+### Input Parameters
+
+- `zipPath` (string): The path to the zip file. It must start with `/tmp/`.
+- `destPath` (string): The directory where the zip content will be extracted. It must start with `/tmp/`.
+
+### Usage Example
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+const destPath = await lumic.zip.extract("/tmp/myZipFile.zip", "/tmp/extractedFolder");
+console.log(`Files extracted to: ${destPath}`);
+```
+
+### addToZipFile
+
+The `addToZipFile` function allows adding new files or directories to an existing zip file.
+
+### Input Parameters
+
+- `zipFilePath` (string): The path to the existing zip file (must start with `/tmp/`).
+- `itemPath` (string): The file or directory to add to the zip (must start with `/tmp/`).
+- `destPath` (string, optional): The destination folder inside the zip.
+
+### Usage Example
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+const updatedZip = await lumic.zip.addTo("/tmp/myZipFile.zip", "/tmp/newFile.txt");
+console.log(`File added to zip: ${updatedZip}`);
+```
+
+## AWS S3 Utilities
+
+The package includes comprehensive S3 management functions:
+
+### Core Functions
+- `create(bucketName, auth?)`: Creates a new S3 bucket
+- `destroy(bucketName, auth?)`: Deletes a bucket and all its contents
+- `generateBucketName(prefix?)`: Generates a valid S3 bucket name
+- `ls(options?)`: Lists buckets or objects in a bucket, sorted by most recent first. Options include:
+  - `bucket?`: Optional bucket name. If provided, lists objects in the bucket. If not provided, lists all buckets.
+  - `auth?`: Optional AWS credentials
+  - `limit?`: Maximum number of items to return (default: 10)
+
+### File Operations
+- `upload(bucketName, sourcePath, destinationPath?, auth?, isPublic?)`: Uploads files/directories with optional public access
+- `download(bucketName, s3Path?, localPath?, auth?)`: Downloads from S3
+- `readFile(bucket, key, auth?)`: Reads a single file as a stream
+- `saveTo(bucket, content, filename?, auth?, isPublic?)`: Saves content directly to S3 with optional public access
+- `deleteFile(s3FileUrl, auth?)`: Deletes a file using its S3 URL
+
+### Utility Functions
+- `generateBucketName(prefix?)`: Creates a valid bucket name with optional prefix
+- `checkBucketForFileNamePart(bucketName, fileNamePart, auth?)`: Searches for files in a bucket
+
+### Usage Examples
+
+#### Creating and Managing Buckets
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+// Generate a unique bucket name with prefix
+const bucketName = await lumic.s3.generateBucketName("myproject");
+
+// Create the bucket
+await lumic.s3.create(bucketName);
+
+// Clean up when done
+await lumic.s3.destroy(bucketName);
+
+// List all buckets in the account (up to 10, most recent first)
+const buckets = await lumic.s3.ls();
+console.log(buckets.objects); // Array of { key: string, createdAt: number }
+
+// List objects in a specific bucket with custom limit
+const objects = await lumic.s3.ls({
+  bucket: "my-bucket",
+  limit: 5
+});
+console.log(objects.objects); // Array of { key: string, createdAt: number }
+```
+
+#### Uploading and Downloading Files
+```javascript
+// Upload a file (private by default)
+const uploadResult = await lumic.s3.upload(
+  "my-bucket",
+  "/tmp/myfile.txt"
+);
+
+// Upload a publicly accessible file
+const publicUploadResult = await lumic.s3.upload(
+  "my-bucket",
+  "/tmp/myfile.txt",
+  {
+    isPublic: true,
+  }
+);
+
+// Download files
+const downloadResult = await lumic.s3.download(
+  "my-bucket",
+  "uploads/",
+  "/tmp/downloads"
+);
+
+// Stream a file
+const fileStream = await lumic.s3.readFile(
+  "my-bucket",
+  "uploads/myfile.txt"
+);
+```
+
+#### Working with Content
+```javascript
+// Save JSON directly to S3 (private by default)
+const jsonContent = { name: "John", age: 30 };
+const saveResult = await lumic.s3.saveTo(
+  "my-bucket",
+  jsonContent,
+  "data.json"
+);
+
+// Save JSON as a publicly accessible file
+const publicSaveResult = await lumic.s3.saveTo(
+  "my-bucket",
+  jsonContent,
+  "public-data.json",
+  null,  // using default AWS credentials
+  true   // make the file public
+);
+
+// Check if a file exists
+const searchResult = await lumic.s3.checkBucketForFileNamePart(
+  "my-bucket",
+  "data-2024"
+);
+
+// Delete a file using its URL
+await lumic.s3.deleteFile(
+  "https://my-bucket.s3.region.amazonaws.com/uploads/myfile.txt"
+);
+```
+
+## JSON Utilities
+
+The package includes utility functions for working with JSON data, particularly for handling and fixing malformed JSON.
+
+### fixBrokenJson
+
+The `fixBrokenJson` function is a robust JSON parser that attempts to repair and parse malformed JSON strings.
+
+### Input Parameters
+
+- `jsonStr` (string): The potentially malformed JSON string to fix and parse
+
+### Usage Example
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+// Fix malformed JSON
+const brokenJson = `{
+  name: "John",
+  age: 30,
+  hobbies: ['reading' "writing", coding],
+  address: {
+    street: '123 Main St,
+    city: "Springfield"
+  },
+}`;
+
+const fixed = lumic.json.fix(brokenJson);
+console.log(fixed);
+```
+
+### parseResponse
+
+The `parseResponse` function is a sophisticated JSON parser that attempts to extract and parse JSON content from various formats, including code blocks.
+
+### Input Parameters
+
+- `content` (string): The content string that may contain JSON
+- `responseFormat` ('text' | 'json'): The expected format of the response. Use 'json' to attempt JSON parsing.
+
+### Usage Example
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+// Parse JSON from a string that might contain code blocks or extra text
+const content = `
+\`\`\`json
+{
+  "name": "John",
+  "age": 30
+}
+\`\`\`
+`;
+
+const parsed = await lumic.json.parse(content, 'json');
+console.log(parsed); // { name: "John", age: 30 }
+```
+
+The function employs multiple parsing strategies:
+1. Direct JSON parsing
+2. Extracting JSON between first {} or []
+3. Removing leading/trailing text
+4. AI-assisted JSON fixing (using GPT-4)
+
+### isValidJson
+
+The `isValidJson` function checks if a string contains valid JSON.
+
+### Usage Example
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+const validJson = '{"name": "John", "age": 30}';
+const invalidJson = '{name: John, age: 30}';
+
+console.log(lumic.json.isValid(validJson));                         // true
+console.log(lumic.json.isValid(invalidJson));                       // false
+```
+
+## Google Sheets Utilities
+
+The package provides utilities for working with Google Sheets through the `lumic.gs` namespace:
+
+### Functions
+
+- `addRow(config, values, startColumn?)`: Add a row of values to a Google Sheet
+- `addSheet(spreadsheetId, sheetTitle)`: Add a new sheet to an existing spreadsheet
+- `writeCell(config, cell, value)`: Write a value to a specific cell
+- `getCell(config, cell)`: Get the value from a specific cell
+- `convertA1ToRowCol(a1Notation)`: Convert A1 notation (e.g., 'A1') to row and column numbers
+- `writeArray(config, data, startCell?)`: Write a 2D array of data to a sheet
+
+### Usage Example
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+// Configure the sheet
+const config = {
+  spreadsheetId: "your-spreadsheet-id",
+  sheetName: "Sheet1"
+};
+
+// Add a row
+await lumic.gs.addRow(config, ["Value1", "Value2", "Value3"]);
+
+// Write to a specific cell
+await lumic.gs.writeCell(config, { row: 1, column: 1 }, "Hello World");
+
+// Write an array of data
+const data = [
+  ["Header1", "Header2"],
+  ["Value1", "Value2"],
+  ["Value3", "Value4"]
+];
+await lumic.gs.writeArray(config, data, "A1");
+```
+
+## Performance Timer
+
+The `PerformanceTimer` utility helps track execution time and performance metrics across your application. It's accessible through `lumic.utils.Timer`.
+
+### Usage
+
+```javascript
+import { lumic } from "lumic-utility-functions";
+
+// Get the global Timer instance
+const timer = lumic.utils.Timer;
+
+// Timer starts automatically upon instantiation
+timer.checkpoint("start-processing"); // Record a checkpoint
+
+// Do some work...
+await someAsyncOperation();
+timer.checkpoint("after-async-operation");
+
+// Get current elapsed time in seconds
+console.log(`Elapsed time: ${timer.getElapsedSeconds()} seconds`);
+
+// Do more work...
+await anotherOperation();
+timer.checkpoint("after-another-operation");
+
+// Stop the timer when done
+timer.stop();
+
+// Generate a performance report
+const report = timer.generateReport();
+console.log(report);
+// Output:
+// {
+//   totalTime: 1234.56, // Total time in milliseconds
+//   phases: [
+//     { label: "start-processing", duration: 100.5 },
+//     { label: "after-async-operation", duration: 534.2 },
+//     { label: "after-another-operation", duration: 599.86 }
+//   ]
+// }
+```
+
+The PerformanceTimer provides:
+- Automatic timing start upon instantiation
+- Checkpoint recording with custom labels
+- Total execution time tracking
+- Quick elapsed time in seconds via `getElapsedSeconds()`
+- Detailed performance report with phase-by-phase timing
+
+Note: Use the global Timer instance across your application to track the complete execution flow, rather than creating new instances.
+
+## TypeScript Support
+
+This package includes TypeScript definitions. When using TypeScript, you'll get full type support and autocompletion for all functions and their parameters.
+
+```typescript
+import { lumic } from "lumic-utility-functions";
+
+// TypeScript will provide type checking and autocompletion
+const result = await lumic.llm.call(
+  "Your prompt",
+  "json",
+  { model: "GPT-5-mini" }
+);
+```
+
+## Author
+
+The utility-functions project is a product of [Lumic.ai](https://lumic.ai).