@jeromeetienne/openai-cache 1.0.0 → 1.0.3

package/README.md

@@ -1,24 +1,47 @@
 # Cache OpenAI
-This is a simple caching layer for OpenAI API requests using local file storage. 
-It helps to reduce redundant API calls by storing responses in a cache directory `./.openai_cache`.
+A simple caching layer for [OpenAI API](https://www.npmjs.com/package/openai), designed to reduce redundant API calls and save time and costs. It works by intercepting API requests and storing their responses in a cache. When the same request is made again, the cached response is returned instead of making a new API call.
+
+It is based on the [cacheable](https://cacheable.org/docs/) library, which provides a simple interface for caching data with support for various storage backends (like in-memory, Redis, SQLite, etc). This allows you to easily integrate caching into your OpenAI API usage without having to manage the caching logic yourself.
+
+You can use any Keyv storage backend (like Redis, filesystem, etc) to store the cached responses. 
+See the [Keyv documentation](https://keyv.org/docs/) for more details on available storage options and how to set them up.
+In the example below, we use a SQLite database to persist the cache.
+
+# Installation
+
+```bash
+npm install @jeromeetienne/openai-cache
+```
+
+If you want to use the SQLite storage backend, you also need to install the `@keyv/sqlite` package:
+
+```bash
+npm install @keyv/sqlite
+```
 
 ## Usage
 
 ```ts
 import OpenAI from "openai";
-import { OpenAICache } from "./src/openai_cache"; // for repo-local usage
-// If published to npm, replace with your package import.
+import OpenAICache from "@jeromeetienne/openai-cache"; 
+import KeyvSqlite from '@keyv/sqlite';
+import { Cacheable } from "cacheable";
 
-const openaiCache = new OpenAICache({
-  cacheDir: ".cache/openai",
-  debug: true,
-});
+// init a cacheable instance
+// - here it is backed by a sqlite database, but you can use any Keyv storage backend (redis, filesystem, etc)
+const sqlitePath = `sqlite://${__dirname}/.openai_cache.sqlite`;
+const sqliteCache = new Cacheable({ secondary: new KeyvSqlite(sqlitePath) });
+
+// init the OpenAICache with the cacheable instance
+const openaiCache = new OpenAICache(sqliteCache);
 
+// init the OpenAI client with the cache's fetch function
 const client = new OpenAI({
   apiKey: process.env.OPENAI_API_KEY,
   fetch: openaiCache.getFetchFn(),
 });
 
+// now use it normally - responses will be cached in the sqlite database
 const response = await client.responses.create({
   model: "gpt-4.1-mini",
   input: "Say hello in one short sentence.",
@@ -31,13 +54,27 @@ console.log(response.output_text);
 - **PRO**: Reduces redundant API calls, saving time and costs.
 data.
 - **NOTE**: When `temperature === 0`, caching works optimally as responses are deterministic. However, with `temperature > 0`, caching may reduce variety across multiple calls since identical prompts will return cached results instead of generating new varied responses.
+- **NOTE**: Only successful responses (`2xx`) are cached. Error responses (`4xx`/`5xx`) are returned normally but are not persisted.
 
 
 ## Possible improvements
 - dont cache if temporature > 0 or top_p < 1, You’ll freeze randomness if cached
-- implement with keyv https://keyv.org/, instead of custom file system
-  - various storage backends (filesystem, redis, etc)
-  - built in TTL support
-- check errors, and dont cache if error
-  - 429/500 errors should not be cached, but other errors (like invalid request) could be cached to prevent repeated bad requests
-  - tools requests errors should not be cached
+  - NOTE: do that on options
+- add configurable cache policy for errors (for example, cache selected deterministic `4xx` while never caching `429`/`5xx`)
+- tools requests errors should not be cached
+
+## Developper Notes
+
+### Q. how to publish the package to npm?
+A. Do the following steps:
+
+```bash
+npm run version:patch && npm run publish:all
+```
+
+Lots of trouble with the 2fa system
+
+Revevant Documentation:
+- https://docs.npmjs.com/requiring-2fa-for-package-publishing-and-settings-modification
+- https://docs.npmjs.com/trusted-publishers
+- https://docs.npmjs.com/creating-and-viewing-access-tokens

package/dist/openai_cache.js

@@ -73,12 +73,14 @@ class OpenAICache {
         // Collect headers and normalize them
         const headers = Array.from(clonedResponse.headers.entries());
         const normalizedHeaders = OpenAICache._normalizeHeaders(headers, responseBuffer.length);
-        await this._cache.set(cacheKey, {
-            status: clonedResponse.status,
-            headers: normalizedHeaders,
-            body: responseBuffer.toString("base64"),
-            bodyEncoding: "base64",
-        });
+        if (response.ok) {
+            await this._cache.set(cacheKey, {
+                status: clonedResponse.status,
+                headers: normalizedHeaders,
+                body: responseBuffer.toString("base64"),
+                bodyEncoding: "base64",
+            });
+        }
         // Return live response (body already buffered)
         return new Response(responseBuffer, { status: response.status, headers: normalizedHeaders });
     }

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@jeromeetienne/openai-cache",
-	"version": "1.0.0",
+	"version": "1.0.3",
 	"main": "dist/openai_cache.js",
 	"types": "dist/openai_cache.d.ts",
 	"exports": {
@@ -25,6 +25,7 @@
 		"prepublishOnly": "npm run test && npm run build",
 		"test": "tsx --test ./test/*_test.ts",
 		"test:watch": "tsx --test --watch ./test/*_test.ts",
+		"publish:all": "npm run build && npm publish --access public",
 		"version:patch": "npm version patch",
 		"version:minor": "npm version minor",
 		"version:major": "npm version major",
@@ -46,12 +47,12 @@
 	},
 	"repository": {
 		"type": "git",
-		"url": "git+https://github.com/jeromeetienne/democracy.ai.git",
+		"url": "git+https://github.com/jeromeetienne/openai-cache.git",
 		"directory": "packages/openai_cache"
 	},
-	"homepage": "https://github.com/jeromeetienne/democracy.ai/tree/main/packages/openai_cache",
+	"homepage": "https://github.com/jeromeetienne/openai-cache/tree/main/#cache-openai",
 	"bugs": {
-		"url": "https://github.com/jeromeetienne/democracy.ai/issues"
+		"url": "https://github.com/jeromeetienne/openai-cache/issues"
 	},
 	"devDependencies": {
 		"@keyv/sqlite": "^4.0.8",
@@ -66,4 +67,4 @@
 	"dependencies": {
 		"cacheable": "^2.3.3"
 	}
-}
+}