tersejson 0.2.1 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +192 -357
- package/dist/mongodb.d.mts +107 -0
- package/dist/mongodb.d.ts +107 -0
- package/dist/mongodb.js +472 -0
- package/dist/mongodb.js.map +1 -0
- package/dist/mongodb.mjs +465 -0
- package/dist/mongodb.mjs.map +1 -0
- package/dist/server-memory.d.mts +167 -0
- package/dist/server-memory.d.ts +167 -0
- package/dist/server-memory.js +455 -0
- package/dist/server-memory.js.map +1 -0
- package/dist/server-memory.mjs +451 -0
- package/dist/server-memory.mjs.map +1 -0
- package/package.json +27 -10
package/README.md
CHANGED
|
@@ -1,41 +1,71 @@
|
|
|
1
1
|
# TerseJSON
|
|
2
2
|
|
|
3
|
-
**
|
|
3
|
+
**Memory-efficient JSON processing. Lazy Proxy expansion uses 70% less RAM than JSON.parse.**
|
|
4
|
+
|
|
5
|
+
> TerseJSON does **LESS work** than JSON.parse, not more. The Proxy skips full deserialization - only accessed fields allocate memory. Plus 30-80% smaller payloads.
|
|
4
6
|
|
|
5
7
|
[](https://www.npmjs.com/package/tersejson)
|
|
6
8
|
[](https://opensource.org/licenses/MIT)
|
|
7
9
|
|
|
8
10
|
## The Problem
|
|
9
11
|
|
|
10
|
-
|
|
12
|
+
Your CMS API returns 21 fields per article. Your list view renders 3.
|
|
11
13
|
|
|
12
|
-
```
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
14
|
+
```javascript
|
|
15
|
+
// Standard JSON.parse workflow:
|
|
16
|
+
const articles = await fetch('/api/articles').then(r => r.json());
|
|
17
|
+
// Result: 1000 objects x 21 fields = 21,000 properties allocated in memory
|
|
18
|
+
// You use: title, slug, excerpt (3 fields)
|
|
19
|
+
// Wasted: 18,000 properties that need garbage collection
|
|
18
20
|
```
|
|
19
21
|
|
|
20
|
-
|
|
22
|
+
**Full deserialization wastes memory.** Every field gets allocated whether you access it or not. Binary formats (Protobuf, MessagePack) have the same problem - they require complete deserialization.
|
|
21
23
|
|
|
22
24
|
## The Solution
|
|
23
25
|
|
|
24
|
-
TerseJSON
|
|
26
|
+
TerseJSON's Proxy wraps compressed data and translates keys **on-demand**:
|
|
25
27
|
|
|
28
|
+
```javascript
|
|
29
|
+
// TerseJSON workflow:
|
|
30
|
+
const articles = await terseFetch('/api/articles');
|
|
31
|
+
// Result: Compressed payload + Proxy wrapper
|
|
32
|
+
// Access: article.title → translates key, returns value
|
|
33
|
+
// Never accessed: 18 other fields stay compressed, never allocate
|
|
26
34
|
```
|
|
27
|
-
Over the wire (compressed):
|
|
28
|
-
{
|
|
29
|
-
"k": { "a": "firstName", "b": "lastName", "c": "emailAddress" },
|
|
30
|
-
"d": [
|
|
31
|
-
{ "a": "John", "b": "Doe", "c": "john@example.com" },
|
|
32
|
-
{ "a": "Jane", "b": "Doe", "c": "jane@example.com" }
|
|
33
|
-
]
|
|
34
|
-
}
|
|
35
35
|
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
36
|
+
**Memory Benchmarks (1000 records, 21 fields each):**
|
|
37
|
+
|
|
38
|
+
| Fields Accessed | Normal JSON | TerseJSON Proxy | Memory Saved |
|
|
39
|
+
|-----------------|-------------|-----------------|--------------|
|
|
40
|
+
| 1 field | 6.35 MB | 4.40 MB | **31%** |
|
|
41
|
+
| 3 fields (list view) | 3.07 MB | ~0 MB | **~100%** |
|
|
42
|
+
| 6 fields (card view) | 3.07 MB | ~0 MB | **~100%** |
|
|
43
|
+
| All 21 fields | 4.53 MB | 1.36 MB | **70%** |
|
|
44
|
+
|
|
45
|
+
*Run the benchmark yourself: `node --expose-gc demo/memory-analysis.js`*
|
|
46
|
+
|
|
47
|
+
## "Doesn't This Add Overhead?"
|
|
48
|
+
|
|
49
|
+
This is the most common misconception. Let's trace the actual operations:
|
|
50
|
+
|
|
51
|
+
**Standard JSON.parse workflow:**
|
|
52
|
+
1. Parse 890KB string → allocate 1000 objects x 21 fields = **21,000 properties**
|
|
53
|
+
2. Access 3 fields per object
|
|
54
|
+
3. GC eventually collects 18,000 unused properties
|
|
55
|
+
|
|
56
|
+
**TerseJSON workflow:**
|
|
57
|
+
1. Parse 180KB string (smaller = faster) → allocate 1000 objects x 21 SHORT keys
|
|
58
|
+
2. Wrap in Proxy (O(1), ~0.1ms, no allocation)
|
|
59
|
+
3. Access 3 fields → **3,000 properties CREATED**
|
|
60
|
+
4. 18,000 properties **NEVER EXIST**
|
|
61
|
+
|
|
62
|
+
**The math:**
|
|
63
|
+
- Parse time: Smaller string (180KB vs 890KB) = **faster**
|
|
64
|
+
- Allocations: 3,000 vs 21,000 = **86% fewer**
|
|
65
|
+
- GC pressure: Only 3,000 objects to collect vs 21,000
|
|
66
|
+
- Proxy lookup: O(1) Map access, ~0.001ms per field
|
|
67
|
+
|
|
68
|
+
**Result:** LESS total work, not more. The Proxy doesn't add overhead - it **skips** work.
|
|
39
69
|
|
|
40
70
|
## Quick Start
|
|
41
71
|
|
|
@@ -68,153 +98,165 @@ import { fetch } from 'tersejson/client';
|
|
|
68
98
|
// Use exactly like regular fetch
|
|
69
99
|
const users = await fetch('/api/users').then(r => r.json());
|
|
70
100
|
|
|
71
|
-
// Access properties normally -
|
|
101
|
+
// Access properties normally - Proxy handles key translation
|
|
72
102
|
console.log(users[0].firstName); // Works transparently!
|
|
73
103
|
console.log(users[0].emailAddress); // Works transparently!
|
|
74
104
|
```
|
|
75
105
|
|
|
76
106
|
## How It Works
|
|
77
107
|
|
|
78
|
-
### Compression Flow
|
|
79
|
-
|
|
80
108
|
```
|
|
81
109
|
┌─────────────────────────────────────────────────────────────┐
|
|
110
|
+
│ SERVER │
|
|
82
111
|
│ 1. Your Express route calls res.json(data) │
|
|
83
|
-
│
|
|
84
|
-
│
|
|
85
|
-
│
|
|
86
|
-
│ 3. Detects array of objects with repeated keys │
|
|
87
|
-
│ ↓ │
|
|
88
|
-
│ 4. Creates key map: { "a": "firstName", "b": "lastName" } │
|
|
89
|
-
│ ↓ │
|
|
90
|
-
│ 5. Replaces keys in data with short aliases │
|
|
91
|
-
│ ↓ │
|
|
92
|
-
│ 6. Sends compressed payload + header │
|
|
112
|
+
│ 2. TerseJSON middleware intercepts │
|
|
113
|
+
│ 3. Compresses keys: { "a": "firstName", "b": "lastName" } │
|
|
114
|
+
│ 4. Sends smaller payload (180KB vs 890KB) │
|
|
93
115
|
└─────────────────────────────────────────────────────────────┘
|
|
94
|
-
↓
|
|
116
|
+
↓ Network (smaller, faster)
|
|
95
117
|
┌─────────────────────────────────────────────────────────────┐
|
|
96
|
-
│
|
|
97
|
-
│
|
|
98
|
-
│
|
|
99
|
-
│
|
|
100
|
-
│
|
|
101
|
-
│ ↓ │
|
|
102
|
-
│ 10. Your code accesses data.firstName → mapped to data.a │
|
|
118
|
+
│ CLIENT │
|
|
119
|
+
│ 5. JSON.parse smaller string (faster) │
|
|
120
|
+
│ 6. Wrap in Proxy (O(1), near-zero cost) │
|
|
121
|
+
│ 7. Access data.firstName → Proxy translates to data.a │
|
|
122
|
+
│ 8. Unused fields never materialize in memory │
|
|
103
123
|
└─────────────────────────────────────────────────────────────┘
|
|
104
124
|
```
|
|
105
125
|
|
|
106
|
-
|
|
126
|
+
## Perfect For
|
|
127
|
+
|
|
128
|
+
- **CMS list views** - title + slug + excerpt from 20+ field objects
|
|
129
|
+
- **Dashboards** - large datasets, aggregate calculations on subsets
|
|
130
|
+
- **Mobile apps** - memory constrained, infinite scroll
|
|
131
|
+
- **E-commerce** - product grids (name + price + image from 30+ field objects)
|
|
132
|
+
- **Long-running SPAs** - memory accumulation over hours (support tools, dashboards)
|
|
107
133
|
|
|
108
|
-
|
|
134
|
+
## Network Savings (Bonus)
|
|
109
135
|
|
|
110
|
-
|
|
111
|
-
|----------|----------|----------------|---------|
|
|
112
|
-
| 100 users, 10 fields | 45 KB | 12 KB | **73%** |
|
|
113
|
-
| 1000 products, 15 fields | 890 KB | 180 KB | **80%** |
|
|
114
|
-
| 10000 logs, 8 fields | 2.1 MB | 450 KB | **79%** |
|
|
136
|
+
Memory efficiency is the headline. Smaller payloads are the bonus:
|
|
115
137
|
|
|
116
|
-
|
|
138
|
+
| Compression Method | Reduction | Use Case |
|
|
139
|
+
|--------------------|-----------|----------|
|
|
140
|
+
| TerseJSON alone | **30-39%** | Sites without Gzip (68% of web) |
|
|
141
|
+
| Gzip alone | ~75% | Large payloads (>32KB) |
|
|
142
|
+
| **TerseJSON + Gzip** | **~85%** | Recommended for production |
|
|
143
|
+
| **TerseJSON + Brotli** | **~93%** | Maximum compression |
|
|
117
144
|
|
|
118
|
-
**
|
|
145
|
+
**Network speed impact (1000-record payload):**
|
|
119
146
|
|
|
120
|
-
|
|
|
121
|
-
|
|
122
|
-
|
|
|
123
|
-
|
|
|
124
|
-
|
|
|
147
|
+
| Network | Normal JSON | TerseJSON + Gzip | Saved |
|
|
148
|
+
|---------|-------------|------------------|-------|
|
|
149
|
+
| 4G (20 Mbps) | 200ms | 30ms | **170ms** |
|
|
150
|
+
| 3G (2 Mbps) | 2,000ms | 300ms | **1,700ms** |
|
|
151
|
+
| Slow 3G | 10,000ms | 1,500ms | **8,500ms** |
|
|
125
152
|
|
|
126
|
-
|
|
153
|
+
## Why Gzip Isn't Enough
|
|
127
154
|
|
|
128
|
-
**
|
|
155
|
+
**"Just use gzip"** misses two points:
|
|
129
156
|
|
|
130
|
-
|
|
131
|
-
|---------|-----------------|---------------|-----------------|
|
|
132
|
-
| 1M requests/day | 40 KB | **40 GB** | **1.2 TB** |
|
|
133
|
-
| 10M requests/day | 40 KB | **400 GB** | **12 TB** |
|
|
134
|
-
| 100M requests/day | 40 KB | **4 TB** | **120 TB** |
|
|
157
|
+
1. **68% of websites don't have Gzip enabled** ([W3Techs](https://w3techs.com/technologies/details/ce-gzipcompression)). Proxy defaults are hostile - nginx, Traefik, Kubernetes all ship with compression off.
|
|
135
158
|
|
|
136
|
-
|
|
159
|
+
2. **Gzip doesn't help memory.** Even with perfect compression over the wire, JSON.parse still allocates every field. TerseJSON's Proxy keeps unused fields compressed in memory.
|
|
137
160
|
|
|
138
|
-
|
|
161
|
+
**TerseJSON works at the application layer:**
|
|
162
|
+
- No proxy config needed
|
|
163
|
+
- No DevOps tickets
|
|
164
|
+
- Stacks with gzip/brotli for maximum savings
|
|
165
|
+
- **Plus** memory benefits that gzip can't provide
|
|
166
|
+
|
|
167
|
+
## vs Binary Formats (Protobuf, MessagePack)
|
|
168
|
+
|
|
169
|
+
| | TerseJSON | Protobuf/MessagePack |
|
|
170
|
+
|---|-----------|---------------------|
|
|
171
|
+
| Wire compression | 30-80% | 80-90% |
|
|
172
|
+
| **Memory on partial access** | **Only accessed fields** | Full deserialization required |
|
|
173
|
+
| Schema required | No | Yes |
|
|
174
|
+
| Human-readable | Yes (JSON in DevTools) | No (binary) |
|
|
175
|
+
| Migration effort | 2 minutes | Days/weeks |
|
|
176
|
+
| Debugging | Easy | Need special tools |
|
|
139
177
|
|
|
140
|
-
**
|
|
178
|
+
**Binary formats win on wire size. TerseJSON wins on memory.**
|
|
141
179
|
|
|
142
|
-
|
|
180
|
+
If you access 3 fields from a 21-field object:
|
|
181
|
+
- Protobuf: All 21 fields deserialized into memory
|
|
182
|
+
- TerseJSON: Only 3 fields materialize
|
|
143
183
|
|
|
144
|
-
|
|
145
|
-
- **60%** of HTTP responses have no text-based compression (HTTP Archive)
|
|
184
|
+
## MongoDB Integration (Zero-Config)
|
|
146
185
|
|
|
147
|
-
|
|
186
|
+
**NEW:** Automatic memory-efficient queries with MongoDB native driver.
|
|
148
187
|
|
|
149
|
-
|
|
188
|
+
```typescript
|
|
189
|
+
import { terseMongo } from 'tersejson/mongodb';
|
|
190
|
+
import { MongoClient } from 'mongodb';
|
|
191
|
+
|
|
192
|
+
// Call once at app startup
|
|
193
|
+
await terseMongo();
|
|
150
194
|
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
- Official Docker nginx image ships with `#gzip on;` (commented out)
|
|
195
|
+
// All queries automatically return Proxy-wrapped results
|
|
196
|
+
const client = new MongoClient(uri);
|
|
197
|
+
const users = await client.db('mydb').collection('users').find().toArray();
|
|
155
198
|
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
- Must explicitly add labels to every service:
|
|
159
|
-
```yaml
|
|
160
|
-
traefik.http.middlewares.compress.compress=true
|
|
161
|
-
traefik.http.routers.myrouter.middlewares=compress
|
|
199
|
+
// Access properties normally - 70% less memory
|
|
200
|
+
console.log(users[0].firstName); // Works transparently!
|
|
162
201
|
```
|
|
163
202
|
|
|
164
|
-
**
|
|
165
|
-
- `
|
|
166
|
-
-
|
|
203
|
+
**What gets patched:**
|
|
204
|
+
- `find().toArray()` - arrays of documents
|
|
205
|
+
- `find().next()` - single document iteration
|
|
206
|
+
- `for await (const doc of cursor)` - async iteration
|
|
207
|
+
- `findOne()` - single document queries
|
|
208
|
+
- `aggregate().toArray()` - aggregation results
|
|
167
209
|
|
|
168
|
-
|
|
210
|
+
**Options:**
|
|
211
|
+
```typescript
|
|
212
|
+
await terseMongo({
|
|
213
|
+
minArrayLength: 5, // Only compress arrays with 5+ items
|
|
214
|
+
skipSingleDocs: true, // Don't wrap findOne results
|
|
215
|
+
minKeyLength: 4, // Only compress keys with 4+ chars
|
|
216
|
+
});
|
|
169
217
|
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
gzip_proxied any;
|
|
174
|
-
gzip_http_version 1.0;
|
|
175
|
-
gzip_types text/plain application/json application/javascript text/css;
|
|
218
|
+
// Restore original behavior
|
|
219
|
+
import { unterse } from 'tersejson/mongodb';
|
|
220
|
+
await unterse();
|
|
176
221
|
```
|
|
177
222
|
|
|
178
|
-
|
|
223
|
+
## Server-Side Memory Optimization
|
|
179
224
|
|
|
180
|
-
|
|
225
|
+
TerseJSON includes utilities for memory-efficient server-side data handling:
|
|
181
226
|
|
|
182
227
|
```typescript
|
|
183
|
-
|
|
184
|
-
```
|
|
228
|
+
import { TerseCache, compressStream } from 'tersejson/server-memory';
|
|
185
229
|
|
|
186
|
-
|
|
230
|
+
// Memory-efficient caching - stores compressed, expands on access
|
|
231
|
+
const cache = new TerseCache();
|
|
232
|
+
cache.set('users', largeUserArray);
|
|
233
|
+
const users = cache.get('users'); // Returns Proxy-wrapped data
|
|
187
234
|
|
|
188
|
-
|
|
189
|
-
|
|
235
|
+
// Streaming compression for database cursors
|
|
236
|
+
const cursor = db.collection('users').find().stream();
|
|
237
|
+
for await (const batch of compressStream(cursor, { batchSize: 100 })) {
|
|
238
|
+
// Process compressed batches without loading entire result set
|
|
239
|
+
}
|
|
190
240
|
|
|
191
|
-
|
|
241
|
+
// Inter-service communication - pass compressed data without intermediate expansion
|
|
242
|
+
import { createTerseServiceClient } from 'tersejson/server-memory';
|
|
243
|
+
const serviceB = createTerseServiceClient({ baseUrl: 'http://service-b' });
|
|
244
|
+
const data = await serviceB.get('/api/users'); // Returns Proxy-wrapped
|
|
245
|
+
```
|
|
192
246
|
|
|
193
247
|
## API Reference
|
|
194
248
|
|
|
195
249
|
### Express Middleware
|
|
196
250
|
|
|
197
251
|
```typescript
|
|
198
|
-
import { terse
|
|
199
|
-
|
|
200
|
-
// Basic usage
|
|
201
|
-
app.use(terse());
|
|
252
|
+
import { terse } from 'tersejson/express';
|
|
202
253
|
|
|
203
|
-
// With options
|
|
204
254
|
app.use(terse({
|
|
205
255
|
minArrayLength: 5, // Only compress arrays with 5+ items
|
|
206
256
|
minKeyLength: 4, // Only compress keys with 4+ characters
|
|
207
257
|
maxDepth: 5, // Max nesting depth to traverse
|
|
208
258
|
debug: true, // Log compression stats
|
|
209
|
-
headerName: 'x-terse', // Custom header name
|
|
210
|
-
shouldCompress: (data, req) => {
|
|
211
|
-
// Custom logic to skip compression
|
|
212
|
-
return !req.path.includes('/admin');
|
|
213
|
-
},
|
|
214
259
|
}));
|
|
215
|
-
|
|
216
|
-
// Enable via query parameter (?terse=true)
|
|
217
|
-
app.use(terseQueryParam());
|
|
218
260
|
```
|
|
219
261
|
|
|
220
262
|
### Client Library
|
|
@@ -226,23 +268,11 @@ import {
|
|
|
226
268
|
expand, // Fully expand a terse payload
|
|
227
269
|
proxy, // Wrap payload with Proxy (default)
|
|
228
270
|
process, // Auto-detect and expand/proxy
|
|
229
|
-
axiosInterceptor // Axios support
|
|
230
271
|
} from 'tersejson/client';
|
|
231
272
|
|
|
232
273
|
// Drop-in fetch replacement
|
|
233
274
|
const data = await fetch('/api/users').then(r => r.json());
|
|
234
275
|
|
|
235
|
-
// Custom fetch instance
|
|
236
|
-
const customFetch = createFetch({
|
|
237
|
-
debug: true,
|
|
238
|
-
autoExpand: true,
|
|
239
|
-
});
|
|
240
|
-
|
|
241
|
-
// Axios integration
|
|
242
|
-
import axios from 'axios';
|
|
243
|
-
axios.interceptors.request.use(axiosInterceptor.request);
|
|
244
|
-
axios.interceptors.response.use(axiosInterceptor.response);
|
|
245
|
-
|
|
246
276
|
// Manual processing
|
|
247
277
|
import { process } from 'tersejson/client';
|
|
248
278
|
const response = await regularFetch('/api/users');
|
|
@@ -254,51 +284,21 @@ const data = process(await response.json());
|
|
|
254
284
|
```typescript
|
|
255
285
|
import {
|
|
256
286
|
compress, // Compress an array of objects
|
|
257
|
-
expand, // Expand a terse payload
|
|
258
|
-
|
|
287
|
+
expand, // Expand a terse payload (full deserialization)
|
|
288
|
+
wrapWithProxy, // Wrap payload with Proxy (lazy expansion - recommended)
|
|
259
289
|
isTersePayload, // Check if data is a terse payload
|
|
260
|
-
createTerseProxy, // Create a Proxy for transparent access
|
|
261
290
|
} from 'tersejson';
|
|
262
291
|
|
|
263
292
|
// Manual compression
|
|
264
293
|
const compressed = compress(users, { minKeyLength: 3 });
|
|
265
294
|
|
|
266
|
-
//
|
|
267
|
-
const
|
|
268
|
-
|
|
269
|
-
// Type checking
|
|
270
|
-
if (isTersePayload(data)) {
|
|
271
|
-
const expanded = expand(data);
|
|
272
|
-
}
|
|
273
|
-
```
|
|
274
|
-
|
|
275
|
-
## TypeScript Support
|
|
276
|
-
|
|
277
|
-
TerseJSON is written in TypeScript and provides full type definitions:
|
|
278
|
-
|
|
279
|
-
```typescript
|
|
280
|
-
import type {
|
|
281
|
-
TersePayload,
|
|
282
|
-
TerseMiddlewareOptions,
|
|
283
|
-
TerseClientOptions,
|
|
284
|
-
Tersed,
|
|
285
|
-
} from 'tersejson';
|
|
286
|
-
|
|
287
|
-
interface User {
|
|
288
|
-
firstName: string;
|
|
289
|
-
lastName: string;
|
|
290
|
-
email: string;
|
|
291
|
-
}
|
|
292
|
-
|
|
293
|
-
// Types flow through compression
|
|
294
|
-
const users: User[] = await fetch('/api/users').then(r => r.json());
|
|
295
|
-
users[0].firstName; // TypeScript knows this is a string
|
|
295
|
+
// Two expansion strategies:
|
|
296
|
+
const expanded = expand(compressed); // Full expansion - all fields allocated
|
|
297
|
+
const proxied = wrapWithProxy(compressed); // Lazy expansion - only accessed fields
|
|
296
298
|
```
|
|
297
299
|
|
|
298
300
|
## Framework Integrations
|
|
299
301
|
|
|
300
|
-
TerseJSON provides ready-to-use integrations for popular HTTP clients and frameworks.
|
|
301
|
-
|
|
302
302
|
### Axios
|
|
303
303
|
|
|
304
304
|
```typescript
|
|
@@ -308,95 +308,6 @@ import { createAxiosInterceptors } from 'tersejson/integrations';
|
|
|
308
308
|
const { request, response } = createAxiosInterceptors();
|
|
309
309
|
axios.interceptors.request.use(request);
|
|
310
310
|
axios.interceptors.response.use(response);
|
|
311
|
-
|
|
312
|
-
// Now all axios requests automatically handle TerseJSON!
|
|
313
|
-
const { data } = await axios.get('/api/users');
|
|
314
|
-
console.log(data[0].firstName); // Works transparently!
|
|
315
|
-
```
|
|
316
|
-
|
|
317
|
-
### Angular (HttpClient)
|
|
318
|
-
|
|
319
|
-
```typescript
|
|
320
|
-
// terse.interceptor.ts
|
|
321
|
-
import { Injectable } from '@angular/core';
|
|
322
|
-
import {
|
|
323
|
-
HttpInterceptor,
|
|
324
|
-
HttpRequest,
|
|
325
|
-
HttpHandler,
|
|
326
|
-
HttpEvent,
|
|
327
|
-
HttpResponse
|
|
328
|
-
} from '@angular/common/http';
|
|
329
|
-
import { Observable } from 'rxjs';
|
|
330
|
-
import { map } from 'rxjs/operators';
|
|
331
|
-
import { isTersePayload, wrapWithProxy } from 'tersejson';
|
|
332
|
-
|
|
333
|
-
@Injectable()
|
|
334
|
-
export class TerseInterceptor implements HttpInterceptor {
|
|
335
|
-
intercept(req: HttpRequest<any>, next: HttpHandler): Observable<HttpEvent<any>> {
|
|
336
|
-
// Add accept-terse header
|
|
337
|
-
const terseReq = req.clone({
|
|
338
|
-
setHeaders: { 'accept-terse': 'true' }
|
|
339
|
-
});
|
|
340
|
-
|
|
341
|
-
return next.handle(terseReq).pipe(
|
|
342
|
-
map(event => {
|
|
343
|
-
if (event instanceof HttpResponse && event.body) {
|
|
344
|
-
const isTerse = event.headers.get('x-terse-json') === 'true';
|
|
345
|
-
if (isTerse && isTersePayload(event.body)) {
|
|
346
|
-
return event.clone({ body: wrapWithProxy(event.body) });
|
|
347
|
-
}
|
|
348
|
-
}
|
|
349
|
-
return event;
|
|
350
|
-
})
|
|
351
|
-
);
|
|
352
|
-
}
|
|
353
|
-
}
|
|
354
|
-
|
|
355
|
-
// app.module.ts
|
|
356
|
-
@NgModule({
|
|
357
|
-
providers: [
|
|
358
|
-
{ provide: HTTP_INTERCEPTORS, useClass: TerseInterceptor, multi: true }
|
|
359
|
-
]
|
|
360
|
-
})
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
### AngularJS (1.x)
|
|
364
|
-
|
|
365
|
-
```javascript
|
|
366
|
-
angular.module('myApp', [])
|
|
367
|
-
.factory('terseInterceptor', function() {
|
|
368
|
-
return {
|
|
369
|
-
request: function(config) {
|
|
370
|
-
config.headers = config.headers || {};
|
|
371
|
-
config.headers['accept-terse'] = 'true';
|
|
372
|
-
return config;
|
|
373
|
-
},
|
|
374
|
-
response: function(response) {
|
|
375
|
-
var isTerse = response.headers('x-terse-json') === 'true';
|
|
376
|
-
if (isTerse && response.data && response.data.__terse__) {
|
|
377
|
-
response.data = tersejson.process(response.data);
|
|
378
|
-
}
|
|
379
|
-
return response;
|
|
380
|
-
}
|
|
381
|
-
};
|
|
382
|
-
})
|
|
383
|
-
.config(['$httpProvider', function($httpProvider) {
|
|
384
|
-
$httpProvider.interceptors.push('terseInterceptor');
|
|
385
|
-
}]);
|
|
386
|
-
```
|
|
387
|
-
|
|
388
|
-
### jQuery
|
|
389
|
-
|
|
390
|
-
```javascript
|
|
391
|
-
import { setupJQueryAjax } from 'tersejson/integrations';
|
|
392
|
-
|
|
393
|
-
// One-time setup
|
|
394
|
-
setupJQueryAjax($);
|
|
395
|
-
|
|
396
|
-
// All jQuery AJAX calls now support TerseJSON
|
|
397
|
-
$.get('/api/users', function(data) {
|
|
398
|
-
console.log(data[0].firstName); // Works!
|
|
399
|
-
});
|
|
400
311
|
```
|
|
401
312
|
|
|
402
313
|
### SWR (React)
|
|
@@ -408,18 +319,8 @@ import { createSWRFetcher } from 'tersejson/integrations';
|
|
|
408
319
|
const fetcher = createSWRFetcher();
|
|
409
320
|
|
|
410
321
|
function UserList() {
|
|
411
|
-
const { data
|
|
412
|
-
|
|
413
|
-
if (error) return <div>Error loading</div>;
|
|
414
|
-
if (!data) return <div>Loading...</div>;
|
|
415
|
-
|
|
416
|
-
return (
|
|
417
|
-
<ul>
|
|
418
|
-
{data.map(user => (
|
|
419
|
-
<li key={user.id}>{user.firstName}</li>
|
|
420
|
-
))}
|
|
421
|
-
</ul>
|
|
422
|
-
);
|
|
322
|
+
const { data } = useSWR('/api/users', fetcher);
|
|
323
|
+
return <ul>{data?.map(user => <li>{user.firstName}</li>)}</ul>;
|
|
423
324
|
}
|
|
424
325
|
```
|
|
425
326
|
|
|
@@ -436,130 +337,64 @@ function UserList() {
|
|
|
436
337
|
queryKey: ['users'],
|
|
437
338
|
queryFn: () => queryFn('/api/users')
|
|
438
339
|
});
|
|
439
|
-
|
|
440
340
|
return <div>{data?.[0].firstName}</div>;
|
|
441
341
|
}
|
|
442
342
|
```
|
|
443
343
|
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
TerseJSON includes optional analytics to track your compression savings.
|
|
447
|
-
|
|
448
|
-
### Local Analytics
|
|
449
|
-
|
|
450
|
-
Track compression stats without sending data anywhere:
|
|
344
|
+
### GraphQL (Apollo)
|
|
451
345
|
|
|
452
346
|
```typescript
|
|
453
|
-
|
|
454
|
-
import {
|
|
455
|
-
|
|
456
|
-
// Enable local-only analytics
|
|
457
|
-
app.use(terse({ analytics: true }));
|
|
458
|
-
|
|
459
|
-
// Or with custom callbacks
|
|
460
|
-
app.use(terse({
|
|
461
|
-
analytics: {
|
|
462
|
-
enabled: true,
|
|
463
|
-
onEvent: (event) => {
|
|
464
|
-
console.log(`Saved ${event.originalSize - event.compressedSize} bytes`);
|
|
465
|
-
},
|
|
466
|
-
},
|
|
467
|
-
}));
|
|
347
|
+
// Server
|
|
348
|
+
import { terseGraphQL } from 'tersejson/graphql';
|
|
349
|
+
app.use('/graphql', terseGraphQL(graphqlHTTP({ schema })));
|
|
468
350
|
|
|
469
|
-
//
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
|
|
351
|
+
// Client
|
|
352
|
+
import { createTerseLink } from 'tersejson/graphql-client';
|
|
353
|
+
const client = new ApolloClient({
|
|
354
|
+
link: from([createTerseLink(), httpLink]),
|
|
355
|
+
cache: new InMemoryCache(),
|
|
356
|
+
});
|
|
474
357
|
```
|
|
475
358
|
|
|
476
|
-
|
|
359
|
+
## TypeScript Support
|
|
477
360
|
|
|
478
|
-
|
|
361
|
+
Full type definitions included:
|
|
479
362
|
|
|
480
363
|
```typescript
|
|
481
|
-
|
|
482
|
-
analytics: {
|
|
483
|
-
apiKey: 'your-api-key', // Get one at tersejson.com/dashboard
|
|
484
|
-
projectId: 'my-app',
|
|
485
|
-
reportToCloud: true,
|
|
486
|
-
},
|
|
487
|
-
}));
|
|
488
|
-
```
|
|
489
|
-
|
|
490
|
-
Dashboard features:
|
|
491
|
-
- Real-time compression stats
|
|
492
|
-
- Bandwidth savings over time
|
|
493
|
-
- Per-endpoint analytics
|
|
494
|
-
- Team sharing
|
|
364
|
+
import type { TersePayload, Tersed } from 'tersejson';
|
|
495
365
|
|
|
496
|
-
|
|
366
|
+
interface User {
|
|
367
|
+
firstName: string;
|
|
368
|
+
lastName: string;
|
|
369
|
+
}
|
|
497
370
|
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
- Only aggregate stats are reported
|
|
371
|
+
const users: User[] = await fetch('/api/users').then(r => r.json());
|
|
372
|
+
users[0].firstName; // TypeScript knows this is a string
|
|
373
|
+
```
|
|
502
374
|
|
|
503
375
|
## FAQ
|
|
504
376
|
|
|
505
|
-
### Does this
|
|
506
|
-
|
|
507
|
-
Yes! TerseJSON recursively compresses nested objects and arrays:
|
|
508
|
-
|
|
509
|
-
```javascript
|
|
510
|
-
// This works
|
|
511
|
-
const data = [
|
|
512
|
-
{
|
|
513
|
-
user: { firstName: "John", lastName: "Doe" },
|
|
514
|
-
orders: [
|
|
515
|
-
{ productName: "Widget", quantity: 5 }
|
|
516
|
-
]
|
|
517
|
-
}
|
|
518
|
-
];
|
|
519
|
-
```
|
|
377
|
+
### Does this break JSON.stringify?
|
|
520
378
|
|
|
521
|
-
|
|
379
|
+
No! The Proxy is transparent. `JSON.stringify(data)` outputs original key names.
|
|
522
380
|
|
|
523
|
-
|
|
381
|
+
### What about nested objects?
|
|
524
382
|
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
No! The Proxy is transparent. `JSON.stringify(data)` works and outputs the original key names.
|
|
383
|
+
Fully supported. TerseJSON recursively compresses nested objects and arrays.
|
|
528
384
|
|
|
529
385
|
### What's the performance overhead?
|
|
530
386
|
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
### Can I use this with GraphQL?
|
|
534
|
-
|
|
535
|
-
Yes! TerseJSON supports GraphQL via `express-graphql` and Apollo Client:
|
|
536
|
-
|
|
537
|
-
```typescript
|
|
538
|
-
// Server (express-graphql)
|
|
539
|
-
import { graphqlHTTP } from 'express-graphql';
|
|
540
|
-
import { terseGraphQL } from 'tersejson/graphql';
|
|
541
|
-
|
|
542
|
-
app.use('/graphql', terseGraphQL(graphqlHTTP({
|
|
543
|
-
schema: mySchema,
|
|
544
|
-
graphiql: true,
|
|
545
|
-
})));
|
|
387
|
+
Proxy mode adds **<5% CPU overhead** vs JSON.parse(). But with smaller payloads and fewer allocations, **net total work is LESS**. Memory is significantly lower.
|
|
546
388
|
|
|
547
|
-
|
|
548
|
-
import { createTerseLink } from 'tersejson/graphql-client';
|
|
549
|
-
|
|
550
|
-
const client = new ApolloClient({
|
|
551
|
-
link: from([createTerseLink(), httpLink]),
|
|
552
|
-
cache: new InMemoryCache(),
|
|
553
|
-
});
|
|
554
|
-
```
|
|
389
|
+
### When should I use expand() vs wrapWithProxy()?
|
|
555
390
|
|
|
556
|
-
|
|
391
|
+
- **wrapWithProxy()** (default): Best for most cases. Lazy expansion, lower memory.
|
|
392
|
+
- **expand()**: When you need a plain object (serialization to storage, passing to libraries that don't support Proxy).
|
|
557
393
|
|
|
558
394
|
## Browser Support
|
|
559
395
|
|
|
560
|
-
Works in all modern browsers
|
|
561
|
-
-
|
|
562
|
-
- `fetch` - Or use a polyfill
|
|
396
|
+
Works in all modern browsers supporting `Proxy` (ES6):
|
|
397
|
+
- Chrome 49+, Firefox 18+, Safari 10+, Edge 12+
|
|
563
398
|
|
|
564
399
|
## Contributing
|
|
565
400
|
|
|
@@ -571,4 +406,4 @@ MIT - see [LICENSE](LICENSE)
|
|
|
571
406
|
|
|
572
407
|
---
|
|
573
408
|
|
|
574
|
-
**[tersejson.com](https://tersejson.com)** |
|
|
409
|
+
**[tersejson.com](https://tersejson.com)** | Memory-efficient JSON for high-volume APIs
|