openai-cache 1.0.14

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jerome Etienne
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,112 @@
+# Cache OpenAI
+A simple caching layer for [OpenAI API](https://www.npmjs.com/package/openai), designed to reduce redundant API calls and save time and costs. It works by intercepting API requests and storing their responses in a cache. When the same request is made again, the cached response is returned instead of making a new API call.
+
+It is based on the [cacheable](https://cacheable.org/docs/) library, which provides a simple interface for caching data with support for various storage backends (like in-memory, Redis, SQLite, etc). This allows you to easily integrate caching into your OpenAI API usage without having to manage the caching logic yourself.
+
+You can use any Keyv storage backend (like Redis, filesystem, etc) to store the cached responses. 
+See the [Keyv documentation](https://keyv.org/docs/) for more details on available storage options and how to set them up.
+In the example below, we use a SQLite database to persist the cache.
+
+# Installation
+
+```bash
+npm install @jeromeetienne/openai-cache
+```
+
+If you want to use the SQLite storage backend, you also need to install the `@keyv/sqlite` package:
+
+```bash
+npm install @keyv/sqlite
+```
+
+## Usage
+
+```ts
+import OpenAI from "openai";
+import OpenAICache from "@jeromeetienne/openai-cache"; 
+import KeyvSqlite from '@keyv/sqlite';
+import { Cacheable } from "cacheable";
+
+// init a cacheable instance
+// - here it is backed by a sqlite database, but you can use any Keyv storage backend (redis, filesystem, etc)
+const sqlitePath = `sqlite://${__dirname}/.openai_cache.sqlite`;
+const sqliteCache = new Cacheable({ secondary: new KeyvSqlite(sqlitePath) });
+
+// init the OpenAICache with the cacheable instance
+const openaiCache = new OpenAICache(sqliteCache);
+
+// init the OpenAI client with the cache's fetch function
+const client = new OpenAI({
+  apiKey: process.env.OPENAI_API_KEY,
+  fetch: openaiCache.getFetchFn(),
+});
+
+// now use it normally - responses will be cached in the sqlite database
+const response = await client.responses.create({
+  model: "gpt-4.1-mini",
+  input: "Say hello in one short sentence.",
+});
+
+console.log(response.output_text);
+```
+
+## PRO/CON
+- **PRO**: Reduces redundant API calls, saving time and costs.
+data.
+- **NOTE**: When `temperature === 0`, caching works optimally as responses are deterministic. However, with `temperature > 0`, caching may reduce variety across multiple calls since identical prompts will return cached results instead of generating new varied responses.
+- **NOTE**: Only successful responses (`2xx`) are cached. Error responses (`4xx`/`5xx`) are returned normally but are not persisted.
+
+
+## Possible improvements
+- dont cache if temporature > 0 or top_p < 1, You’ll freeze randomness if cached
+  - NOTE: do that on options
+- add configurable cache policy for errors (for example, cache selected deterministic `4xx` while never caching `429`/`5xx`)
+- tools requests errors should not be cached
+
+## Developper Notes
+
+### Q. How to disable the cache ?
+A. Set the `OPENAI_CACHE` environment variable to `disabled`:
+
+```bash
+OPENAI_CACHE=disabled node your_app.js
+```
+
+It will still write in the cache but will ignore the cached responses and always call the OpenAI API. This is useful for testing or debugging purposes when you want to bypass the cache without changing your code.
+
+### Q. How to know if a given call was a cache hit or miss?
+A. You can enable the `markResponseEnabled` option when initializing the `OpenAICache`. When this option is enabled, the cache will add a custom property
+to the response object to indicate whether it was a cache hit or miss. 
+
+
+```ts
+const openaiCache = new OpenAICache(sqliteCache, {
+    markResponseEnabled: true, // default is false
+});
+
+// later, when you make a call, you can check the custom property to see if it was a cache hit or miss
+const response = await client.responses.create({
+    model: "gpt-4.1-mini",
+    input: "Say hello in one short sentence.",
+});
+
+if (response.X_FROM_OPENAI_CACHE) {
+    console.log("Cache hit!");
+} else {
+    console.log("Cache miss!");
+}
+```
+
+### Q. how to publish the package to npm?
+A. Do the following steps:
+
+```bash
+npm run version:patch && npm run publish:all
+```
+
+Lots of trouble with the 2fa system
+
+Revevant Documentation:
+- https://docs.npmjs.com/requiring-2fa-for-package-publishing-and-settings-modification
+- https://docs.npmjs.com/trusted-publishers
+- https://docs.npmjs.com/creating-and-viewing-access-tokens

package/dist/openai_cache.d.ts

@@ -0,0 +1,54 @@
+import { Cacheable } from "cacheable";
+type FetchFn = typeof globalThis.fetch;
+type FetchInput = Parameters<FetchFn>[0];
+type FetchInit = Parameters<FetchFn>[1];
+type FetchResponse = Awaited<ReturnType<FetchFn>>;
+/**
+ * OpenAICachingCacheable is a wrapper around the Fetch API that adds caching capabilities for OpenAI requests.
+ * It uses a Cacheable instance to store and retrieve cached responses based on a hash of the request details.
+ */
+export default class OpenAICache {
+    private readonly _cache;
+    private readonly _markResponseEnabled;
+    static readonly MarkResponseName = "X_FROM_OPENAI_CACHE";
+    /**
+     * Creates a new instance of OpenAICache.
+     *
+     * @param cache cacheable instance
+     * @param options.markResponseEnabled whether to mark cached responses with an additional property in the JSON body (default: true).
+     * This can be useful for downstream logic that needs to differentiate between live and cached responses, but it does modify
+     * the original response body so it is optional. so the response is { X_FROM_OPENAI_CACHE: true, ...originalResponseBody }
+     */
+    constructor(cache?: Cacheable, { markResponseEnabled }?: {
+        markResponseEnabled?: boolean;
+    });
+    /**
+     * Cleans the OpenAI cache by deleting all cached values.
+     */
+    cleanCache(): Promise<void>;
+    /**
+     * return a fetch function that can be passed to OpenAI client for caching support
+     *
+     * ```js
+     * const openai = new OpenAI({
+     *   fetch: openaiCache.getFetchFn()
+     * });
+     * ```
+     */
+    getFetchFn(): (input: FetchInput, init?: FetchInit) => Promise<FetchResponse>;
+    /**
+     * This is the fetch() implementation that adds caching for OpenAI requests.
+     *
+     * @param input The resource that you wish to fetch.
+     * @param init An options object containing any custom settings that you want to apply to the request.
+     * @returns A Promise that resolves to the Response to that request.
+     */
+    private _fetch;
+    /**
+     * Remove transfer/content encodings that no longer apply once the body is materialized
+     * and optionally set a correct content-length for the cached payload.
+     */
+    private static _normalizeHeaders;
+    private static _serializeBodyForHash;
+}
+export {};

package/dist/openai_cache.js

@@ -0,0 +1,157 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+// node imports
+const node_crypto_1 = __importDefault(require("node:crypto"));
+const node_buffer_1 = require("node:buffer");
+const cacheable_1 = require("cacheable");
+///////////////////////////////////////////////////////////////////////////////
+///////////////////////////////////////////////////////////////////////////////
+//	OpenAICache
+///////////////////////////////////////////////////////////////////////////////
+///////////////////////////////////////////////////////////////////////////////
+/**
+ * OpenAICachingCacheable is a wrapper around the Fetch API that adds caching capabilities for OpenAI requests.
+ * It uses a Cacheable instance to store and retrieve cached responses based on a hash of the request details.
+ */
+class OpenAICache {
+    /**
+     * Creates a new instance of OpenAICache.
+     *
+     * @param cache cacheable instance
+     * @param options.markResponseEnabled whether to mark cached responses with an additional property in the JSON body (default: true).
+     * This can be useful for downstream logic that needs to differentiate between live and cached responses, but it does modify
+     * the original response body so it is optional. so the response is { X_FROM_OPENAI_CACHE: true, ...originalResponseBody }
+     */
+    constructor(cache, { markResponseEnabled = false } = {}) {
+        this._cache = cache !== null && cache !== void 0 ? cache : new cacheable_1.Cacheable();
+        this._markResponseEnabled = markResponseEnabled;
+    }
+    /**
+     * Cleans the OpenAI cache by deleting all cached values.
+     */
+    async cleanCache() {
+        await this._cache.clear();
+    }
+    /**
+     * return a fetch function that can be passed to OpenAI client for caching support
+     *
+     * ```js
+     * const openai = new OpenAI({
+     *   fetch: openaiCache.getFetchFn()
+     * });
+     * ```
+     */
+    getFetchFn() {
+        return this._fetch.bind(this);
+    }
+    /**
+     * This is the fetch() implementation that adds caching for OpenAI requests.
+     *
+     * @param input The resource that you wish to fetch.
+     * @param init An options object containing any custom settings that you want to apply to the request.
+     * @returns A Promise that resolves to the Response to that request.
+     */
+    async _fetch(input, init) {
+        var _a, _b;
+        // Extract the URL from the input (string or Request)
+        const url = typeof input === "string" ? input : input instanceof Request ? input.url : input.toString();
+        // Normalize HTTP method
+        const method = ((init === null || init === void 0 ? void 0 : init.method) || "GET").toUpperCase();
+        // Generate body hash payload
+        const bodyForHash = OpenAICache._serializeBodyForHash(init === null || init === void 0 ? void 0 : init.body);
+        // If body type unsupported, skip caching
+        if (bodyForHash === null)
+            return fetch(input, init);
+        // Build cache key and file path
+        const cacheKey = node_crypto_1.default.createHash("sha256")
+            .update(`${method}:${url}:${bodyForHash}`)
+            .digest("hex");
+        const cached = (await this._cache.get(cacheKey));
+        if (cached !== undefined && process.env.OPENAI_CACHE !== "disabled") {
+            const bodyEncoding = (_a = cached.bodyEncoding) !== null && _a !== void 0 ? _a : "utf8";
+            const cachedBodyBuffer = node_buffer_1.Buffer.from(cached.body, bodyEncoding);
+            // Return cached response
+            let newResponse = new Response(cachedBodyBuffer, {
+                status: cached.status,
+                headers: cached.headers,
+            });
+            // honor this._markResponseEnabled option to indicate cache hit
+            const contentTypeIsJson = ((_b = newResponse.headers.get("content-type")) === null || _b === void 0 ? void 0 : _b.includes("application/json")) ? true : false;
+            if (this._markResponseEnabled && contentTypeIsJson) {
+                try {
+                    // decode JSON from cachedBodyBuffer
+                    const bodyJson = JSON.parse(cachedBodyBuffer.toString());
+                    // Set the magic property to indicate this response is from cache
+                    bodyJson.X_FROM_OPENAI_CACHE = true;
+                    // Rebuild response with modified body
+                    const modifiedBodyBuffer = node_buffer_1.Buffer.from(JSON.stringify(bodyJson));
+                    newResponse = new Response(modifiedBodyBuffer, { status: cached.status, headers: cached.headers, });
+                }
+                catch (error) {
+                    // If parsing fails, return the original cached response without modification
+                    console.warn("Failed to parse cached response body as JSON for header modification:", error);
+                }
+            }
+            // Return cached response (body already buffered)
+            return newResponse;
+        }
+        // Perform network fetch
+        const response = await fetch(input, init);
+        const clonedResponse = response.clone();
+        // Materialize response body for caching
+        const responseBuffer = node_buffer_1.Buffer.from(await clonedResponse.arrayBuffer());
+        // Collect headers and normalize them
+        const headers = Array.from(clonedResponse.headers.entries());
+        const normalizedHeaders = OpenAICache._normalizeHeaders(headers, responseBuffer.length);
+        if (response.ok) {
+            await this._cache.set(cacheKey, {
+                status: clonedResponse.status,
+                headers: normalizedHeaders,
+                body: responseBuffer.toString("base64"),
+                bodyEncoding: "base64",
+            });
+        }
+        // Return live response (body already buffered)
+        return new Response(responseBuffer, { status: response.status, headers: normalizedHeaders });
+    }
+    ///////////////////////////////////////////////////////////////////////////////
+    ///////////////////////////////////////////////////////////////////////////////
+    //	Private functions
+    ///////////////////////////////////////////////////////////////////////////////
+    ///////////////////////////////////////////////////////////////////////////////
+    /**
+     * Remove transfer/content encodings that no longer apply once the body is materialized
+     * and optionally set a correct content-length for the cached payload.
+     */
+    static _normalizeHeaders(headers, bodyLength) {
+        const drop = new Set([
+            "content-encoding", // body is already decoded by fetch()
+            "transfer-encoding",
+            "content-length", // will be recalculated
+        ]);
+        const filtered = headers.filter(([name]) => drop.has(name.toLowerCase()) === false);
+        if (bodyLength !== undefined) {
+            filtered.push(["content-length", String(bodyLength)]);
+        }
+        return filtered;
+    }
+    // Serialize body into a deterministic string for hashing
+    static _serializeBodyForHash(body) {
+        if (body === undefined || body === null)
+            return "";
+        if (typeof body === "string")
+            return body;
+        if (node_buffer_1.Buffer.isBuffer(body))
+            return body.toString("base64");
+        if (body instanceof ArrayBuffer)
+            return node_buffer_1.Buffer.from(body).toString("base64");
+        if (ArrayBuffer.isView(body))
+            return node_buffer_1.Buffer.from(body.buffer, body.byteOffset, body.byteLength).toString("base64");
+        return null; // unsupported body type
+    }
+}
+OpenAICache.MarkResponseName = "X_FROM_OPENAI_CACHE";
+exports.default = OpenAICache;

package/package.json

@@ -0,0 +1,70 @@
+{
+	"name": "openai-cache",
+	"version": "1.0.14",
+	"main": "dist/openai_cache.js",
+	"types": "dist/openai_cache.d.ts",
+	"exports": {
+		".": {
+			"types": "./dist/openai_cache.d.ts",
+			"default": "./dist/openai_cache.js"
+		}
+	},
+	"files": [
+		"dist",
+		"README.md",
+		"LICENSE"
+	],
+	"scripts": {
+		"example:openai_cache": "tsx examples/openai_cache_example.ts",
+		"example:tts": "tsx examples/tts_example.ts",
+		"example:image_generation": "tsx examples/image_generation_example.ts",
+		"example:image_understanding": "tsx examples/image_understanding_example.ts",
+		"clean_cache": "rm -f ./examples/.openai_cache.sqlite",
+		"build": "tsc -p tsconfig.json",
+		"clean": "rm -rf dist",
+		"prepublishOnly": "npm run test && npm run build",
+		"test": "tsx --test ./test/*_test.ts",
+		"test:watch": "tsx --test --watch ./test/*_test.ts",
+		"publish:all": "npm run build && npm run version:patch && npm publish --access public",
+		"version:patch": "npm version patch",
+		"version:minor": "npm version minor",
+		"version:major": "npm version major",
+		"version:prerelease": "npm version prerelease --preid=rc"
+	},
+	"keywords": [
+		"openai",
+		"cache",
+		"caching",
+		"keyv",
+		"sqlite",
+		"typescript"
+	],
+	"author": "jerome etienne <jerome.etienne@example.com>",
+	"license": "MIT",
+	"description": "A caching layer for OpenAI API calls, designed to improve performance and reduce costs.",
+	"publishConfig": {
+		"access": "public"
+	},
+	"repository": {
+		"type": "git",
+		"url": "git+https://github.com/jeromeetienne/openai-cache.git",
+		"directory": "packages/openai_cache"
+	},
+	"homepage": "https://github.com/jeromeetienne/openai-cache/tree/main/#cache-openai",
+	"bugs": {
+		"url": "https://github.com/jeromeetienne/openai-cache/issues"
+	},
+	"devDependencies": {
+		"@keyv/sqlite": "^4.0.8",
+		"@types/node": "^25.1.0",
+		"keyv": "^5.6.0",
+		"openai": "^6.17.0",
+		"ts-node": "^10.9.2",
+		"tsx": "^4.21.0",
+		"typescript": "^5.9.3",
+		"zod": "^4.3.6"
+	},
+	"dependencies": {
+		"cacheable": "^2.3.3"
+	}
+}