polystore 0.19.0 → 0.21.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/index.d.ts +1 -0
- package/index.js +115 -26
- package/package.json +32 -8
- package/readme.md +237 -46
- package/src/integrations/express.d.ts +307 -0
- package/src/integrations/express.js +42 -0
- package/src/integrations/hono-sessions.d.ts +299 -0
- package/src/integrations/hono-sessions.js +36 -0
- package/src/express.js +0 -47
package/readme.md
CHANGED
|
@@ -63,7 +63,7 @@ MyApi({ cache: env.KV_NAMESPACE }); // OR
|
|
|
63
63
|
|
|
64
64
|
First, install `polystore` and whatever [supported client](#clients) that you prefer. Let's see Redis as an example here:
|
|
65
65
|
|
|
66
|
-
```
|
|
66
|
+
```sh
|
|
67
67
|
npm i polystore redis
|
|
68
68
|
```
|
|
69
69
|
|
|
@@ -93,7 +93,7 @@ await store.del(key);
|
|
|
93
93
|
|
|
94
94
|
## API
|
|
95
95
|
|
|
96
|
-
The base `kv()` initialization is shared across clients ([see full clients list](#clients));
|
|
96
|
+
The base `kv()` initialization is shared across clients ([see full clients list](#clients)); an argument that receives the client or a string representing the client and then the options:
|
|
97
97
|
|
|
98
98
|
```js
|
|
99
99
|
import kv from "polystore";
|
|
@@ -107,7 +107,7 @@ const store = kv(MyClientInstance, { expires: null, prefix: "" });
|
|
|
107
107
|
> [!IMPORTANT]
|
|
108
108
|
> The library delivers excellent performance for item-level operations (GET, SET, ADD, HAS, DEL). For other methods or detailed guidance, check the performance considerations and consult your specific client’s documentation.
|
|
109
109
|
|
|
110
|
-
You can enforce **types** for
|
|
110
|
+
You can enforce **types** for the values either at store creation or at the method level:
|
|
111
111
|
|
|
112
112
|
```ts
|
|
113
113
|
const store = kv<number>(new Map());
|
|
@@ -631,6 +631,25 @@ A client is the library that manages the low-level store operations. For example
|
|
|
631
631
|
|
|
632
632
|
Polystore provides a unified API you can use `Promises`, `expires` and `.prefix()` even with those stores that do not support these operations natively.
|
|
633
633
|
|
|
634
|
+
Quick overview:
|
|
635
|
+
|
|
636
|
+
| Client | Runtime | Persistence | Native expiration | Notes |
|
|
637
|
+
|---|---|---|---|---|
|
|
638
|
+
| [Memory](#memory) | Node.js + Browser | ❌ | ❌ | Great for tests and ephemeral caches |
|
|
639
|
+
| [Local Storage](#local-storage) | Browser | ✅ | ❌ | Persistent browser storage |
|
|
640
|
+
| [Session Storage](#session-storage) | Browser | ❌ | ❌ | Cleared when tab/session ends |
|
|
641
|
+
| [Cookies](#cookies) | Browser | ✅ | ✅ | Browser-side cookies |
|
|
642
|
+
| [Local Forage](#local-forage) | Browser | ✅ | ❓ | Better capacity than localStorage |
|
|
643
|
+
| [Redis](#redis) | Node.js | ✅ | ✅ | Good distributed cache backend |
|
|
644
|
+
| [SQLite](#sqlite) | Node.js | ✅ | ❌ | Simple local persistence |
|
|
645
|
+
| [Fetch API](#fetch-api) | Any with `fetch` | ❓ | ❓ | Bring your own KV HTTP API |
|
|
646
|
+
| [File](#file) | Node.js | ✅ | ❌ | Single JSON file store |
|
|
647
|
+
| [Folder](#folder) | Node.js | ✅ | ❌ | One-file-per-key store |
|
|
648
|
+
| [Cloudflare KV](#cloudflare-kv) | Cloudflare | ✅ | ✅ | Edge-native KV |
|
|
649
|
+
| [Level](#level) | Node.js | ✅ | ❌ | Uses Level ecosystem |
|
|
650
|
+
| [Etcd](#etcd) | Node.js | ✅ | ✅ | Distributed KV |
|
|
651
|
+
| [Postgres](#postgres) | Node.js | ✅ | ❌ | Table-backed KV |
|
|
652
|
+
|
|
634
653
|
While you can keep a reference to the client and access it directly, we strongly recommend to only access it through `polystore`, since we might add custom serialization and extra properties for e.g. expiration time:
|
|
635
654
|
|
|
636
655
|
```js
|
|
@@ -1164,28 +1183,32 @@ console.log(await store.get("key1"));
|
|
|
1164
1183
|
// "Hello world"
|
|
1165
1184
|
```
|
|
1166
1185
|
|
|
1167
|
-
|
|
1186
|
+
Polystore will initialize the schema automatically: it creates the `kv` table and expiration index if they do not exist yet, and does not fail if they already exist.
|
|
1168
1187
|
|
|
1169
|
-
|
|
1188
|
+
Required schema (auto-created by Polystore):
|
|
1170
1189
|
|
|
1171
1190
|
```sql
|
|
1172
|
-
CREATE TABLE kv (
|
|
1191
|
+
CREATE TABLE IF NOT EXISTS kv (
|
|
1173
1192
|
id TEXT PRIMARY KEY,
|
|
1174
1193
|
value TEXT NOT NULL,
|
|
1175
1194
|
"expiresAt" TIMESTAMP
|
|
1176
1195
|
);
|
|
1196
|
+
|
|
1197
|
+
CREATE INDEX IF NOT EXISTS idx_kv_expiresAt
|
|
1198
|
+
ON kv ("expiresAt");
|
|
1177
1199
|
```
|
|
1178
1200
|
|
|
1179
|
-
The default table name is `kv
|
|
1201
|
+
The default table name is `kv`. Key prefixes still work as normal key namespaces:
|
|
1180
1202
|
|
|
1181
1203
|
```js
|
|
1182
|
-
const sessions = store.prefix("session:");
|
|
1183
|
-
const cache = store.prefix("cache:");
|
|
1204
|
+
const sessions = store.prefix("session:");
|
|
1205
|
+
const cache = store.prefix("cache:");
|
|
1184
1206
|
|
|
1185
1207
|
await sessions.set("user123", { name: "Alice" });
|
|
1208
|
+
// Stored key in Postgres: "session:user123"
|
|
1186
1209
|
```
|
|
1187
1210
|
|
|
1188
|
-
This
|
|
1211
|
+
This keeps a single table while preserving namespace-style grouping through prefixed keys.
|
|
1189
1212
|
|
|
1190
1213
|
<details>
|
|
1191
1214
|
<summary>Why use polystore with Postgres?</summary>
|
|
@@ -1202,7 +1225,158 @@ This maps prefixes to table names for better performance on group operations.
|
|
|
1202
1225
|
|
|
1203
1226
|
Please see the [creating a store](#creating-a-store) section for all the details!
|
|
1204
1227
|
|
|
1205
|
-
##
|
|
1228
|
+
## Integrations
|
|
1229
|
+
|
|
1230
|
+
Polystore has some easy integrations for you to use it as a simple connector.
|
|
1231
|
+
|
|
1232
|
+
### Express
|
|
1233
|
+
|
|
1234
|
+
Use any Polystore-compatible store as an [express-session](https://github.com/expressjs/session) store:
|
|
1235
|
+
|
|
1236
|
+
```js
|
|
1237
|
+
import session from "express-session";
|
|
1238
|
+
import expressStore from "polystore/express";
|
|
1239
|
+
|
|
1240
|
+
app.use(session({
|
|
1241
|
+
secret: "my-secret",
|
|
1242
|
+
store: expressStore(),
|
|
1243
|
+
}));
|
|
1244
|
+
```
|
|
1245
|
+
|
|
1246
|
+
By default it uses an in-memory `Map`, which is fine for development. For production, pass any Polystore client:
|
|
1247
|
+
|
|
1248
|
+
```js
|
|
1249
|
+
import { createClient } from "redis";
|
|
1250
|
+
// `npm install polystore`
|
|
1251
|
+
import expressStore from "polystore/express";
|
|
1252
|
+
|
|
1253
|
+
const store = expressStore(createClient().connect());
|
|
1254
|
+
|
|
1255
|
+
app.use(session({ secret: "my-secret", store }));
|
|
1256
|
+
```
|
|
1257
|
+
|
|
1258
|
+
Any client works — Redis, Postgres, SQLite, file-based, etc. Session TTL is read automatically from `cookie.originalMaxAge` so you don't need to configure it separately.
|
|
1259
|
+
|
|
1260
|
+
Use `.prefix()` to namespace sessions, for example in a multi-tenant app:
|
|
1261
|
+
|
|
1262
|
+
```js
|
|
1263
|
+
const store = expressStore(createClient().connect());
|
|
1264
|
+
|
|
1265
|
+
app.use((req, res, next) => {
|
|
1266
|
+
req.sessionStore = store.prefix(`tenant:${req.params.tenant}:`);
|
|
1267
|
+
next();
|
|
1268
|
+
});
|
|
1269
|
+
```
|
|
1270
|
+
|
|
1271
|
+
### Hono Sessions
|
|
1272
|
+
|
|
1273
|
+
Use any Polystore-compatible store as a [hono-sessions](https://github.com/jcs224/hono_sessions) store:
|
|
1274
|
+
|
|
1275
|
+
```js
|
|
1276
|
+
import { Hono } from "hono";
|
|
1277
|
+
import { sessionMiddleware } from "hono-sessions";
|
|
1278
|
+
import honoStore from "polystore/hono-sessions";
|
|
1279
|
+
|
|
1280
|
+
const app = new Hono();
|
|
1281
|
+
|
|
1282
|
+
app.use("*", sessionMiddleware({
|
|
1283
|
+
store: honoStore(),
|
|
1284
|
+
encryptionKey: process.env.SESSION_KEY,
|
|
1285
|
+
expireAfterSeconds: 3600,
|
|
1286
|
+
}));
|
|
1287
|
+
```
|
|
1288
|
+
|
|
1289
|
+
By default it uses an in-memory `Map`. For production, pass any Polystore client:
|
|
1290
|
+
|
|
1291
|
+
```js
|
|
1292
|
+
import { createClient } from "redis";
|
|
1293
|
+
import honoStore from "polystore/hono-sessions";
|
|
1294
|
+
|
|
1295
|
+
app.use("*", sessionMiddleware({
|
|
1296
|
+
store: honoStore(createClient().connect()),
|
|
1297
|
+
encryptionKey: process.env.SESSION_KEY,
|
|
1298
|
+
expireAfterSeconds: 3600,
|
|
1299
|
+
}));
|
|
1300
|
+
```
|
|
1301
|
+
|
|
1302
|
+
Session TTL is derived automatically from `expireAfterSeconds` — hono-sessions writes it to `_expire` on the session data, and Polystore uses it to set the underlying store TTL for automatic cleanup.
|
|
1303
|
+
|
|
1304
|
+
Use `.prefix()` to namespace sessions per tenant:
|
|
1305
|
+
|
|
1306
|
+
```js
|
|
1307
|
+
const store = honoStore(createClient().connect());
|
|
1308
|
+
|
|
1309
|
+
app.use("*", (c, next) => {
|
|
1310
|
+
const tenant = c.req.param("tenant");
|
|
1311
|
+
return sessionMiddleware({
|
|
1312
|
+
store: store.prefix(`tenant:${tenant}:`),
|
|
1313
|
+
encryptionKey: process.env.SESSION_KEY,
|
|
1314
|
+
})(c, next);
|
|
1315
|
+
});
|
|
1316
|
+
```
|
|
1317
|
+
|
|
1318
|
+
|
|
1319
|
+
### fch
|
|
1320
|
+
|
|
1321
|
+
[Fch](https://www.npmjs.com/package/fch) is a lightweight fetch wrapper that uses Polystore natively for caching. Pass any Polystore store as the `cache` option and GET responses are cached automatically:
|
|
1322
|
+
|
|
1323
|
+
```js
|
|
1324
|
+
import fch from "fch";
|
|
1325
|
+
import kv from "polystore";
|
|
1326
|
+
|
|
1327
|
+
const api = fch.create({
|
|
1328
|
+
baseUrl: "https://api.example.com",
|
|
1329
|
+
cache: kv(new Map(), { expires: "1h" }),
|
|
1330
|
+
});
|
|
1331
|
+
|
|
1332
|
+
await api.get("/users"); // fetched from network, stored in cache
|
|
1333
|
+
await api.get("/users"); // served from cache
|
|
1334
|
+
```
|
|
1335
|
+
|
|
1336
|
+
Swap the backend without changing anything else:
|
|
1337
|
+
|
|
1338
|
+
```js
|
|
1339
|
+
import { createClient } from "redis";
|
|
1340
|
+
|
|
1341
|
+
const api = fch.create({
|
|
1342
|
+
cache: kv(createClient().connect(), { expires: "10min" }),
|
|
1343
|
+
});
|
|
1344
|
+
```
|
|
1345
|
+
|
|
1346
|
+
You can override or skip the cache per request:
|
|
1347
|
+
|
|
1348
|
+
```js
|
|
1349
|
+
const shortCache = kv(new Map(), { expires: "30s" });
|
|
1350
|
+
|
|
1351
|
+
api.get("/realtime", { cache: null }); // skip cache
|
|
1352
|
+
api.get("/prices", { cache: shortCache }); // use a different cache
|
|
1353
|
+
```
|
|
1354
|
+
|
|
1355
|
+
|
|
1356
|
+
### @server/next
|
|
1357
|
+
|
|
1358
|
+
> [!WARNING]
|
|
1359
|
+
> @server/next is still experimental, but it's the main reason I created Polystore and so I wanted to document it as well
|
|
1360
|
+
|
|
1361
|
+
Server.js supports Polystore directly:
|
|
1362
|
+
|
|
1363
|
+
```ts
|
|
1364
|
+
import kv from "polystore";
|
|
1365
|
+
import server from "@server/next";
|
|
1366
|
+
|
|
1367
|
+
const session = kv(new Map());
|
|
1368
|
+
|
|
1369
|
+
export default server({ session }).get("/", (ctx) => {
|
|
1370
|
+
if (!ctx.session.counter) ctx.session.counter = 0;
|
|
1371
|
+
ctx.session.counter++;
|
|
1372
|
+
return `User visited ${ctx.session.counter} times`;
|
|
1373
|
+
});
|
|
1374
|
+
```
|
|
1375
|
+
|
|
1376
|
+
|
|
1377
|
+
## Guides
|
|
1378
|
+
|
|
1379
|
+
### Performance
|
|
1206
1380
|
|
|
1207
1381
|
> TL;DR: if you only use the item operations (add, set, get, has, del) and your client supports expiration natively, you have nothing to worry about! Otherwise, please read on.
|
|
1208
1382
|
|
|
@@ -1214,7 +1388,7 @@ While all of our stores support `expires`, `.prefix()` and group operations, the
|
|
|
1214
1388
|
|
|
1215
1389
|
**Substores** when dealing with a `.prefix()` substore, the same applies. Item operations should see no performance degradation from `.prefix()`, but group operations follow the above performance considerations. Some engines might have native prefix support, so performance in those is better for group operations in a substore than the whole store. But in general you should consider `.prefix()` as a convenient way of classifying your keys and not as a performance fix for group operations.
|
|
1216
1390
|
|
|
1217
|
-
|
|
1391
|
+
### Expirations
|
|
1218
1392
|
|
|
1219
1393
|
> Warning: if a client doesn't support expiration natively, we will hide expired keys on the API calls for a nice DX, but _old data might not be evicted automatically_. See [the notes in Performance](#performance) for details on how to work around this.
|
|
1220
1394
|
|
|
@@ -1258,7 +1432,7 @@ These are all the units available:
|
|
|
1258
1432
|
|
|
1259
1433
|
This is great because with polystore we do ensure that if a key has expired, it doesn't show up in `.keys()`, `.entries()`, `.values()`, `.has()` or `.get()`.
|
|
1260
1434
|
|
|
1261
|
-
|
|
1435
|
+
#### Eviction
|
|
1262
1436
|
|
|
1263
1437
|
However, in some stores this does come with some potential performance disadvantages. For example, both the in-memory example above and localStorage _don't_ have a native expiration/eviction process, so we have to store that information as metadata, meaning that even to check if a key exists we need to read and decode its value. For one or few keys it's not a problem, but for large sets this can become an issue.
|
|
1264
1438
|
|
|
@@ -1266,7 +1440,7 @@ For other stores like Redis this is not a problem, because the low-level operati
|
|
|
1266
1440
|
|
|
1267
1441
|
These details are explained in the respective client information.
|
|
1268
1442
|
|
|
1269
|
-
|
|
1443
|
+
### Substores
|
|
1270
1444
|
|
|
1271
1445
|
> There's some [basic `.prefix()` API info](#prefix) for everyday usage, this section is the in-depth explanation.
|
|
1272
1446
|
|
|
@@ -1280,7 +1454,46 @@ When dealing with large or complex amounts of data in a KV store, sometimes it's
|
|
|
1280
1454
|
|
|
1281
1455
|
For these and more situations, you can use `.prefix()` to simplify your life further.
|
|
1282
1456
|
|
|
1283
|
-
|
|
1457
|
+
### Error Handling
|
|
1458
|
+
|
|
1459
|
+
Polystore methods return promises and surface errors from the underlying client. A good rule of thumb is to treat errors in three categories:
|
|
1460
|
+
|
|
1461
|
+
1. **Connectivity/runtime errors**
|
|
1462
|
+
Network/database/filesystem/client runtime failures (for example Redis unavailable, failed fetch, permission denied on files).
|
|
1463
|
+
|
|
1464
|
+
2. **Data/serialization errors**
|
|
1465
|
+
Invalid JSON payloads, invalid value encoding, or data that was written outside Polystore and cannot be decoded with its metadata expectations.
|
|
1466
|
+
|
|
1467
|
+
3. **Usage/configuration errors**
|
|
1468
|
+
Invalid client setup, invalid URLs/paths, or unsupported operations in a specific runtime.
|
|
1469
|
+
|
|
1470
|
+
Recommended patterns:
|
|
1471
|
+
|
|
1472
|
+
- Use `try/catch` around all write/read operations in production paths.
|
|
1473
|
+
- Prefer returning safe fallbacks for cache-like usage (`null`, stale response, or refetch).
|
|
1474
|
+
- Log enough context (`client type`, `key`, operation name) without logging sensitive values.
|
|
1475
|
+
- For remote clients, consider retry/backoff only for transient failures.
|
|
1476
|
+
- Call `.close()` during shutdown when the client supports it.
|
|
1477
|
+
|
|
1478
|
+
Example:
|
|
1479
|
+
|
|
1480
|
+
```js
|
|
1481
|
+
const key = `user:${userId}`;
|
|
1482
|
+
|
|
1483
|
+
try {
|
|
1484
|
+
const cached = await store.get(key);
|
|
1485
|
+
if (cached) return cached;
|
|
1486
|
+
|
|
1487
|
+
const fresh = await fetchUserFromAPI(userId);
|
|
1488
|
+
await store.set(key, fresh, { expires: "10min" });
|
|
1489
|
+
return fresh;
|
|
1490
|
+
} catch (err) {
|
|
1491
|
+
console.error("polystore error", { key, err });
|
|
1492
|
+
return fetchUserFromAPI(userId);
|
|
1493
|
+
}
|
|
1494
|
+
```
|
|
1495
|
+
|
|
1496
|
+
### Creating a store
|
|
1284
1497
|
|
|
1285
1498
|
To create a store, you define a class with these properties and methods:
|
|
1286
1499
|
|
|
@@ -1341,6 +1554,10 @@ client.keys = (prefix) => {
|
|
|
1341
1554
|
|
|
1342
1555
|
While the signatures are different, you can check each entries on the output of Polystore API to see what is expected for the methods of the client to do, e.g. `.clear()` will remove all of the items that match the prefix (or everything if there's no prefix).
|
|
1343
1556
|
|
|
1557
|
+
|
|
1558
|
+
|
|
1559
|
+
## Examples
|
|
1560
|
+
|
|
1344
1561
|
### Plain Object client
|
|
1345
1562
|
|
|
1346
1563
|
This is a good example of how simple a store can be, however do not use it literally since it behaves the same as the already-supported `new Map()`, only use it as the base for your own clients:
|
|
@@ -1371,7 +1588,7 @@ class MyClient {
|
|
|
1371
1588
|
|
|
1372
1589
|
We don't set `HAS_EXPIRATION` to true since plain objects do NOT support expiration natively. So by not adding the `HAS_EXPIRATION` property, it's the same as setting it to `false`, and polystore will manage all the expirations as a layer on top of the data. We could be more explicit and set it to `HAS_EXPIRATION = false`, but it's not needed in this case.
|
|
1373
1590
|
|
|
1374
|
-
###
|
|
1591
|
+
### Custom ID generation
|
|
1375
1592
|
|
|
1376
1593
|
You might want to provide your custom key generation algorithm, which I'm going to call `customId()` for example purposes. The only place where `polystore` generates IDs is in `add`, so you can provide your client with a custom generator:
|
|
1377
1594
|
|
|
@@ -1404,7 +1621,7 @@ const id2 = await store.prefix("hello:").add({ hello: "world" });
|
|
|
1404
1621
|
// this is `hello:{your own custom id}`
|
|
1405
1622
|
```
|
|
1406
1623
|
|
|
1407
|
-
###
|
|
1624
|
+
### Serializing the data
|
|
1408
1625
|
|
|
1409
1626
|
If you need to serialize the data before storing it, you can do it within your custom client. Here's an example of how you can handle data serialization when setting values:
|
|
1410
1627
|
|
|
@@ -1429,7 +1646,7 @@ class MyClient {
|
|
|
1429
1646
|
}
|
|
1430
1647
|
```
|
|
1431
1648
|
|
|
1432
|
-
###
|
|
1649
|
+
### Cloudflare API calls
|
|
1433
1650
|
|
|
1434
1651
|
In this example on one of my projects, I needed to use Cloudflare's REST API since I didn't have access to any KV store I was happy with on Netlify's Edge Functions. So I created it like this:
|
|
1435
1652
|
|
|
@@ -1501,9 +1718,6 @@ const store = kv(CloudflareCustom);
|
|
|
1501
1718
|
|
|
1502
1719
|
It's lacking a few things, so make sure to adapt to your needs, but it worked for my very simple cache needs.
|
|
1503
1720
|
|
|
1504
|
-
|
|
1505
|
-
## Examples
|
|
1506
|
-
|
|
1507
1721
|
### Simple cache
|
|
1508
1722
|
|
|
1509
1723
|
I've used Polystore in many projects as a simple cache. With `fetch()`, it's fairly easy:
|
|
@@ -1514,12 +1728,9 @@ async function getProductInfo(id: string) {
|
|
|
1514
1728
|
if (data) return data;
|
|
1515
1729
|
|
|
1516
1730
|
const res = await fetch(`https://some-url.com/products/${id}`);
|
|
1517
|
-
const
|
|
1518
|
-
|
|
1519
|
-
// Some processing here
|
|
1520
|
-
const clean = raw??;
|
|
1731
|
+
const data = await res.json();
|
|
1521
1732
|
|
|
1522
|
-
await store.set(id,
|
|
1733
|
+
await store.set(id, data, { expires: "10days" });
|
|
1523
1734
|
return clean;
|
|
1524
1735
|
}
|
|
1525
1736
|
```
|
|
@@ -1548,23 +1759,3 @@ if (process.env.REDIS_URL) {
|
|
|
1548
1759
|
|
|
1549
1760
|
export default store;
|
|
1550
1761
|
```
|
|
1551
|
-
|
|
1552
|
-
### @server/next
|
|
1553
|
-
|
|
1554
|
-
> [!info]
|
|
1555
|
-
> @server/next is still experimental, but it's the main reason I created Polystore and so I wanted to document it as well
|
|
1556
|
-
|
|
1557
|
-
Server.js supports Polystore directly:
|
|
1558
|
-
|
|
1559
|
-
```ts
|
|
1560
|
-
import kv from "polystore";
|
|
1561
|
-
import server from "../../";
|
|
1562
|
-
|
|
1563
|
-
const session = kv(new Map());
|
|
1564
|
-
|
|
1565
|
-
export default server({ session }).get("/", (ctx) => {
|
|
1566
|
-
if (!ctx.session.counter) ctx.session.counter = 0;
|
|
1567
|
-
ctx.session.counter++;
|
|
1568
|
-
return `User visited ${ctx.session.counter} times`;
|
|
1569
|
-
});
|
|
1570
|
-
```
|