@jeromeetienne/openai-cache 1.0.2 → 1.0.3

package/README.md

@@ -7,11 +7,23 @@ You can use any Keyv storage backend (like Redis, filesystem, etc) to store the
 See the [Keyv documentation](https://keyv.org/docs/) for more details on available storage options and how to set them up.
 In the example below, we use a SQLite database to persist the cache.
 
+# Installation
+
+```bash
+npm install @jeromeetienne/openai-cache
+```
+
+If you want to use the SQLite storage backend, you also need to install the `@keyv/sqlite` package:
+
+```bash
+npm install @keyv/sqlite
+```
+
 ## Usage
 
 ```ts
 import OpenAI from "openai";
-import { OpenAICache } from "@jeromeetienne/openai-cache"; 
+import OpenAICache from "@jeromeetienne/openai-cache"; 
 import KeyvSqlite from '@keyv/sqlite';
 import { Cacheable } from "cacheable";
 
@@ -42,14 +54,14 @@ console.log(response.output_text);
 - **PRO**: Reduces redundant API calls, saving time and costs.
 data.
 - **NOTE**: When `temperature === 0`, caching works optimally as responses are deterministic. However, with `temperature > 0`, caching may reduce variety across multiple calls since identical prompts will return cached results instead of generating new varied responses.
+- **NOTE**: Only successful responses (`2xx`) are cached. Error responses (`4xx`/`5xx`) are returned normally but are not persisted.
 
 
 ## Possible improvements
 - dont cache if temporature > 0 or top_p < 1, You’ll freeze randomness if cached
   - NOTE: do that on options
-- check errors, and dont cache if error
-  - 429/500 errors should not be cached, but other errors (like invalid request) could be cached to prevent repeated bad requests
-  - tools requests errors should not be cached
+- add configurable cache policy for errors (for example, cache selected deterministic `4xx` while never caching `429`/`5xx`)
+- tools requests errors should not be cached
 
 ## Developper Notes
 
@@ -60,4 +72,9 @@ A. Do the following steps:
 npm run version:patch && npm run publish:all
 ```
 
-Lots of trouble with the 2fa system
+Lots of trouble with the 2fa system
+
+Revevant Documentation:
+- https://docs.npmjs.com/requiring-2fa-for-package-publishing-and-settings-modification
+- https://docs.npmjs.com/trusted-publishers
+- https://docs.npmjs.com/creating-and-viewing-access-tokens

package/dist/openai_cache.js

@@ -73,12 +73,14 @@ class OpenAICache {
         // Collect headers and normalize them
         const headers = Array.from(clonedResponse.headers.entries());
         const normalizedHeaders = OpenAICache._normalizeHeaders(headers, responseBuffer.length);
-        await this._cache.set(cacheKey, {
-            status: clonedResponse.status,
-            headers: normalizedHeaders,
-            body: responseBuffer.toString("base64"),
-            bodyEncoding: "base64",
-        });
+        if (response.ok) {
+            await this._cache.set(cacheKey, {
+                status: clonedResponse.status,
+                headers: normalizedHeaders,
+                body: responseBuffer.toString("base64"),
+                bodyEncoding: "base64",
+            });
+        }
         // Return live response (body already buffered)
         return new Response(responseBuffer, { status: response.status, headers: normalizedHeaders });
     }

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@jeromeetienne/openai-cache",
-	"version": "1.0.2",
+	"version": "1.0.3",
 	"main": "dist/openai_cache.js",
 	"types": "dist/openai_cache.d.ts",
 	"exports": {