forge-sql-orm 2.0.29 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/README.md +1090 -81
  2. package/dist/ForgeSQLORM.js +1090 -69
  3. package/dist/ForgeSQLORM.js.map +1 -1
  4. package/dist/ForgeSQLORM.mjs +1073 -69
  5. package/dist/ForgeSQLORM.mjs.map +1 -1
  6. package/dist/core/ForgeSQLAnalyseOperations.d.ts +1 -1
  7. package/dist/core/ForgeSQLAnalyseOperations.d.ts.map +1 -1
  8. package/dist/core/ForgeSQLCacheOperations.d.ts +119 -0
  9. package/dist/core/ForgeSQLCacheOperations.d.ts.map +1 -0
  10. package/dist/core/ForgeSQLCrudOperations.d.ts +38 -22
  11. package/dist/core/ForgeSQLCrudOperations.d.ts.map +1 -1
  12. package/dist/core/ForgeSQLORM.d.ts +104 -13
  13. package/dist/core/ForgeSQLORM.d.ts.map +1 -1
  14. package/dist/core/ForgeSQLQueryBuilder.d.ts +243 -15
  15. package/dist/core/ForgeSQLQueryBuilder.d.ts.map +1 -1
  16. package/dist/index.d.ts +1 -1
  17. package/dist/index.d.ts.map +1 -1
  18. package/dist/lib/drizzle/extensions/additionalActions.d.ts +42 -0
  19. package/dist/lib/drizzle/extensions/additionalActions.d.ts.map +1 -0
  20. package/dist/utils/cacheContextUtils.d.ts +123 -0
  21. package/dist/utils/cacheContextUtils.d.ts.map +1 -0
  22. package/dist/utils/cacheUtils.d.ts +56 -0
  23. package/dist/utils/cacheUtils.d.ts.map +1 -0
  24. package/dist/utils/sqlUtils.d.ts +8 -0
  25. package/dist/utils/sqlUtils.d.ts.map +1 -1
  26. package/dist/webtriggers/clearCacheSchedulerTrigger.d.ts +46 -0
  27. package/dist/webtriggers/clearCacheSchedulerTrigger.d.ts.map +1 -0
  28. package/dist/webtriggers/index.d.ts +1 -0
  29. package/dist/webtriggers/index.d.ts.map +1 -1
  30. package/package.json +15 -12
  31. package/src/core/ForgeSQLAnalyseOperations.ts +1 -1
  32. package/src/core/ForgeSQLCacheOperations.ts +195 -0
  33. package/src/core/ForgeSQLCrudOperations.ts +49 -40
  34. package/src/core/ForgeSQLORM.ts +443 -34
  35. package/src/core/ForgeSQLQueryBuilder.ts +291 -20
  36. package/src/index.ts +1 -1
  37. package/src/lib/drizzle/extensions/additionalActions.ts +548 -0
  38. package/src/lib/drizzle/extensions/types.d.ts +68 -10
  39. package/src/utils/cacheContextUtils.ts +210 -0
  40. package/src/utils/cacheUtils.ts +403 -0
  41. package/src/utils/sqlUtils.ts +29 -12
  42. package/src/webtriggers/clearCacheSchedulerTrigger.ts +79 -0
  43. package/src/webtriggers/index.ts +1 -0
  44. package/dist/lib/drizzle/extensions/selectAliased.d.ts +0 -9
  45. package/dist/lib/drizzle/extensions/selectAliased.d.ts.map +0 -1
  46. package/src/lib/drizzle/extensions/selectAliased.ts +0 -72
package/README.md CHANGED
@@ -17,6 +17,8 @@
17
17
 
18
18
  ## Key Features
19
19
  - ✅ **Custom Drizzle Driver** for direct integration with @forge/sql
20
+ - ✅ **Local Cache System (Level 1)** for in-memory query optimization within single resolver invocation scope
21
+ - ✅ **Global Cache System (Level 2)** with cross-invocation caching, automatic cache invalidation and context-aware operations (using [@forge/kvs](https://developer.atlassian.com/platform/forge/storage-reference/storage-api-custom-entities/) )
20
22
  - ✅ **Type-Safe Query Building**: Write SQL queries with full TypeScript support
21
23
  - ✅ **Supports complex SQL queries** with joins and filtering using Drizzle ORM
22
24
  - ✅ **Schema migration support**, allowing automatic schema evolution
@@ -26,24 +28,115 @@
26
28
  - ✅ **Schema Fetching** Development-only web trigger to retrieve current database schema and generate SQL statements for schema recreation
27
29
  - ✅ **Ready-to-use Migration Triggers** Built-in web triggers for applying migrations, dropping tables (development-only), and fetching schema (development-only) with proper error handling and security controls
28
30
  - ✅ **Optimistic Locking** Ensures data consistency by preventing conflicts when multiple users update the same record
29
- - ✅ **Query Plan Analysis**: Detailed execution plan analysis and optimization insights (Performance analysis and Troubleshooting only)
31
+ - ✅ **Query Plan Analysis**: Detailed execution plan analysis and optimization insights
32
+
33
+ ## Table of Contents
34
+
35
+ ### 🚀 Getting Started
36
+ - [Key Features](#key-features)
37
+ - [Usage Approaches](#usage-approaches)
38
+ - [Installation](#installation)
39
+ - [CLI Commands](#cli-commands) | [CLI Documentation](forge-sql-orm-cli/README.md)
40
+ - [Quick Start](#quick-start)
41
+
42
+ ### 📖 Core Features
43
+ - [Field Name Collision Prevention](#field-name-collision-prevention-in-complex-queries)
44
+ - [Drizzle Usage with forge-sql-orm](#drizzle-usage-with-forge-sql-orm)
45
+ - [Direct Drizzle Usage with Custom Driver](#direct-drizzle-usage-with-custom-driver)
46
+
47
+ ### 🗄️ Database Operations
48
+ - [Fetch Data](#fetch-data)
49
+ - [Modify Operations](#modify-operations)
50
+ - [SQL Utilities](#sql-utilities)
51
+
52
+ ### ⚡ Caching System
53
+ - [Setting Up Caching with @forge/kvs](#setting-up-caching-with-forgekvs-optional)
54
+ - [Global Cache System (Level 2)](#global-cache-system-level-2)
55
+ - [Cache Context Operations](#cache-context-operations)
56
+ - [Local Cache Operations (Level 1)](#local-cache-operations-level-1)
57
+ - [Cache-Aware Query Operations](#cache-aware-query-operations)
58
+ - [Manual Cache Management](#manual-cache-management)
59
+
60
+ ### 🔒 Advanced Features
61
+ - [Optimistic Locking](#optimistic-locking)
62
+ - [Query Analysis and Performance Optimization](#query-analysis-and-performance-optimization)
63
+ - [Date and Time Types](#date-and-time-types)
64
+
65
+ ### 🛠️ Development Tools
66
+ - [CLI Commands](#cli-commands) | [CLI Documentation](forge-sql-orm-cli/README.md)
67
+ - [Web Triggers for Migrations](#web-triggers-for-migrations)
68
+ - [Step-by-Step Migration Workflow](#step-by-step-migration-workflow)
69
+ - [Drop Migrations](#drop-migrations)
70
+
71
+ ### 📚 Examples
72
+ - [Simple Example](examples/forge-sql-orm-example-simple)
73
+ - [Drizzle Driver Example](examples/forge-sql-orm-example-drizzle-driver-simple)
74
+ - [Optimistic Locking Example](examples/forge-sql-orm-example-optimistic-locking)
75
+ - [Dynamic Queries Example](examples/forge-sql-orm-example-dynamic)
76
+ - [Query Analysis Example](examples/forge-sql-orm-example-query-analyses)
77
+ - [Organization Tracker Example](examples/forge-sql-orm-example-org-tracker)
78
+ - [Checklist Example](examples/forge-sql-orm-example-checklist)
79
+
80
+ ### 📚 Reference
81
+ - [ForgeSqlOrmOptions](#forgesqlormoptions)
82
+ - [Migration Guide](#migration-guide)
83
+
84
+ ## 🚀 Quick Navigation
85
+
86
+ **New to Forge-SQL-ORM?** Start here:
87
+ - [Quick Start](#quick-start) - Get up and running in 5 minutes
88
+ - [Installation](#installation) - Complete setup guide
89
+ - [Basic Usage Examples](#fetch-data) - Simple query examples
90
+
91
+ **Looking for specific features?**
92
+ - [Global Cache System (Level 2)](#global-cache-system-level-2) - Cross-invocation persistent caching
93
+ - [Local Cache System (Level 1)](#local-cache-operations-level-1) - In-memory invocation caching
94
+ - [Optimistic Locking](#optimistic-locking) - Data consistency
95
+ - [Migration Tools](#web-triggers-for-migrations) - Database migrations
96
+ - [Query Analysis](#query-analysis-and-performance-optimization) - Performance optimization
97
+
98
+ **Looking for practical examples?**
99
+ - [Simple Example](examples/forge-sql-orm-example-simple) - Basic ORM usage
100
+ - [Optimistic Locking Example](examples/forge-sql-orm-example-optimistic-locking) - Real-world conflict handling
101
+ - [Organization Tracker Example](examples/forge-sql-orm-example-org-tracker) - Complex relationships
102
+ - [Checklist Example](examples/forge-sql-orm-example-checklist) - Jira integration
30
103
 
31
104
  ## Usage Approaches
32
105
 
33
- ### 1. Direct Drizzle Usage
106
+
107
+ ### 1. Full Forge-SQL-ORM Usage
108
+ ```typescript
109
+ import ForgeSQL from "forge-sql-orm";
110
+ const forgeSQL = new ForgeSQL();
111
+ ```
112
+ Best for: Advanced features like optimistic locking, automatic versioning, and automatic field name collision prevention in complex queries.
113
+
114
+ ### 2. Direct Drizzle Usage
34
115
  ```typescript
35
116
  import { drizzle } from "drizzle-orm/mysql-proxy";
36
117
  import { forgeDriver } from "forge-sql-orm";
37
118
  const db = drizzle(forgeDriver);
38
119
  ```
39
- Best for: Simple CRUD operations without optimistic locking. Note that you need to manually patch drizzle `patchDbWithSelectAliased` for select fields to prevent field name collisions in Atlassian Forge SQL.
120
+ Best for: Simple Modify operations without optimistic locking. Note that you need to manually patch drizzle `patchDbWithSelectAliased` for select fields to prevent field name collisions in Atlassian Forge SQL.
121
+
40
122
 
41
- ### 2. Full Forge-SQL-ORM Usage
123
+ ### 3. Local Cache Optimization
42
124
  ```typescript
43
125
  import ForgeSQL from "forge-sql-orm";
44
126
  const forgeSQL = new ForgeSQL();
127
+
128
+ // Optimize repeated queries within a single invocation
129
+ await forgeSQL.executeWithLocalContext(async () => {
130
+ // Multiple queries here will benefit from local caching
131
+ const users = await forgeSQL.select({ id: users.id, name: users.name })
132
+ .from(users).where(eq(users.active, true));
133
+
134
+ // This query will use local cache (no database call)
135
+ const cachedUsers = await forgeSQL.select({ id: users.id, name: users.name })
136
+ .from(users).where(eq(users.active, true));
137
+ });
45
138
  ```
46
- Best for: Advanced features like optimistic locking, automatic versioning, and automatic field name collision prevention in complex queries.
139
+ Best for: Performance optimization of repeated queries within resolvers or single invocation contexts.
47
140
 
48
141
  ## Field Name Collision Prevention in Complex Queries
49
142
 
@@ -91,18 +184,103 @@ Forge-SQL-ORM is designed to work with @forge/sql and requires some additional s
91
184
 
92
185
  ✅ Step 1: Install Dependencies
93
186
 
187
+ **Basic installation (without caching):**
94
188
  ```sh
95
- npm install forge-sql-orm @forge/sql drizzle-orm momment -S
96
- npm install forge-sql-orm-cli -D
189
+ npm install forge-sql-orm @forge/sql drizzle-orm -S
190
+ ```
191
+
192
+ **With caching support:**
193
+ ```sh
194
+ npm install forge-sql-orm @forge/sql @forge/kvs drizzle-orm -S
97
195
  ```
98
196
 
99
197
  This will:
100
198
  - Install Forge-SQL-ORM (the ORM for @forge/sql)
101
199
  - Install @forge/sql, the Forge database layer
200
+ - Install @forge/kvs, the Forge Key-Value Store for caching (optional, only needed for caching features)
102
201
  - Install Drizzle ORM and its MySQL driver
103
202
  - Install TypeScript types for MySQL
104
203
  - Install forge-sql-orm-cli A command-line interface tool for managing Atlassian Forge SQL migrations and model generation with Drizzle ORM integration.
105
204
 
205
+ ## Quick Start
206
+
207
+ ### 1. Basic Setup
208
+ ```typescript
209
+ import ForgeSQL from "forge-sql-orm";
210
+
211
+ // Initialize ForgeSQL
212
+ const forgeSQL = new ForgeSQL();
213
+
214
+ // Simple query
215
+ const users = await forgeSQL.select().from(users);
216
+ ```
217
+
218
+ ### 2. With Caching (Optional)
219
+ ```typescript
220
+ import ForgeSQL from "forge-sql-orm";
221
+
222
+ // Initialize with caching
223
+ const forgeSQL = new ForgeSQL({
224
+ cacheEntityName: "cache",
225
+ cacheTTL: 300
226
+ });
227
+
228
+ // Cached query
229
+ const users = await forgeSQL.selectCacheable({ id: users.id, name: users.name })
230
+ .from(users).where(eq(users.active, true));
231
+ ```
232
+
233
+ ### 3. Local Cache Optimization
234
+ ```typescript
235
+ // Optimize repeated queries within a single invocation
236
+ await forgeSQL.executeWithLocalContext(async () => {
237
+ const users = await forgeSQL.select({ id: users.id, name: users.name })
238
+ .from(users).where(eq(users.active, true));
239
+
240
+ // This query will use local cache (no database call)
241
+ const cachedUsers = await forgeSQL.select({ id: users.id, name: users.name })
242
+ .from(users).where(eq(users.active, true));
243
+ });
244
+ ```
245
+
246
+ ### 4. Next Steps
247
+ - [Full Installation Guide](#installation) - Complete setup instructions
248
+ - [Core Features](#core-features) - Learn about key capabilities
249
+ - [Global Cache System (Level 2)](#global-cache-system-level-2) - Cross-invocation caching features
250
+ - [Local Cache System (Level 1)](#local-cache-operations-level-1) - In-memory caching features
251
+ - [API Reference](#reference) - Complete API documentation
252
+
253
+ ## Drizzle Usage with forge-sql-orm
254
+
255
+ If you prefer to use Drizzle ORM with the additional features of Forge-SQL-ORM (like optimistic locking and caching), you can use the enhanced API:
256
+
257
+ ```typescript
258
+ import ForgeSQL from "forge-sql-orm";
259
+ const forgeSQL = new ForgeSQL();
260
+
261
+ // Versioned operations with cache management (recommended)
262
+ await forgeSQL.modifyWithVersioningAndEvictCache().insert(Users, [userData]);
263
+ await forgeSQL.modifyWithVersioningAndEvictCache().updateById(updateData, Users);
264
+
265
+ // Versioned operations without cache management
266
+ await forgeSQL.modifyWithVersioning().insert(Users, [userData]);
267
+ await forgeSQL.modifyWithVersioning().updateById(updateData, Users);
268
+
269
+ // Non-versioned operations with cache management
270
+ await forgeSQL.insertAndEvictCache(Users).values(userData);
271
+ await forgeSQL.updateAndEvictCache(Users).set(updateData).where(eq(Users.id, 1));
272
+
273
+ // Basic Drizzle operations (cache context aware)
274
+ await forgeSQL.insert(Users).values(userData);
275
+ await forgeSQL.update(Users).set(updateData).where(eq(Users.id, 1));
276
+
277
+ // Direct Drizzle access
278
+ const db = forgeSQL.getDrizzleQueryBuilder();
279
+ const users = await db.select().from(users);
280
+ ```
281
+
282
+ This approach gives you direct access to all Drizzle ORM features while still using the @forge/sql backend with enhanced caching and versioning capabilities.
283
+
106
284
  ## Direct Drizzle Usage with Custom Driver
107
285
 
108
286
  If you prefer to use Drizzle ORM directly without the additional features of Forge-SQL-ORM (like optimistic locking), you can use the custom driver:
@@ -116,24 +294,373 @@ const db = patchDbWithSelectAliased(drizzle(forgeDriver));
116
294
 
117
295
  // Use drizzle directly
118
296
  const users = await db.select().from(users);
297
+ const users = await db.selectAliased(getTableColumns(users)).from(users);
298
+ const users = await db.selectAliasedDistinct(getTableColumns(users)).from(users);
299
+ await db.insert(users)...;
300
+ await db.update(users)...;
301
+ await db.delete(users)...;
302
+ // Use drizzle with kvs cache
303
+ const users = await db.selectAliasedCacheable(getTableColumns(users)).from(users);
304
+ const users = await db.selectAliasedDistinctCacheable(getTableColumns(users)).from(users);
305
+ await db.insertAndEvictCache(users)...;
306
+ await db.updateAndEvictCache(users)...;
307
+ await db.deleteAndEvictCache(users)...;
308
+
309
+ // Use drizzle with kvs cache context
310
+ await forgeSQL.executeWithCacheContext(async () => {
311
+ await db.insertWithCacheContext(users)...;
312
+ await db.updateWithCacheContext(users)...;
313
+ await db.deleteWithCacheContext(users)...;
314
+ // invoke without cache
315
+ const users = await db.selectAliasedCacheable(getTableColumns(users)).from(users);
316
+ // Cache is cleared only once at the end for all affected tables
317
+ });
318
+ ```
319
+
320
+ ## Setting Up Caching with @forge/kvs (Optional)
321
+
322
+ The caching system is optional and only needed if you want to use cache-related features. To enable the caching system, you need to install the required dependency and configure your manifest.
323
+
324
+ ### How Caching Works
325
+
326
+ To use caching, you need to use Forge-SQL-ORM methods that support cache management:
327
+
328
+ **Methods that perform cache eviction after execution and in cache context (batch eviction):**
329
+ - `forgeSQL.insertAndEvictCache()`
330
+ - `forgeSQL.updateAndEvictCache()`
331
+ - `forgeSQL.deleteAndEvictCache()`
332
+ - `forgeSQL.modifyWithVersioningAndEvictCache()`
333
+ - `forgeSQL.getDrizzleQueryBuilder().insertAndEvictCache()`
334
+ - `forgeSQL.getDrizzleQueryBuilder().updateAndEvictCache()`
335
+ - `forgeSQL.getDrizzleQueryBuilder().deleteAndEvictCache()`
336
+
337
+ **Methods that participate in cache context only (batch eviction):**
338
+ - All methods except the default Drizzle methods:
339
+ - `forgeSQL.insert()`
340
+ - `forgeSQL.update()`
341
+ - `forgeSQL.delete()`
342
+ - `forgeSQL.modifyWithVersioning()`
343
+ - `forgeSQL.getDrizzleQueryBuilder().insertWithCacheContext()`
344
+ - `forgeSQL.getDrizzleQueryBuilder().updateWithCacheContext()`
345
+ - `forgeSQL.getDrizzleQueryBuilder().deleteWithCacheContext()`
346
+
347
+ **Methods do not do evict cache, better do not use with cache feature:**
348
+ - `forgeSQL.getDrizzleQueryBuilder().insert()`
349
+ - `forgeSQL.getDrizzleQueryBuilder().update()`
350
+ - `forgeSQL.getDrizzleQueryBuilder().delete()`
351
+
352
+ **Cacheable methods:**
353
+ - `forgeSQL.selectCacheable()`
354
+ - `forgeSQL.selectDistinctCacheable()`
355
+ - `forgeSQL.getDrizzleQueryBuilder().selectAliasedCacheable()`
356
+ - `forgeSQL.getDrizzleQueryBuilder().selectAliasedDistinctCacheable()`
357
+
358
+ **Cache context example:**
359
+ ```typescript
360
+ await forgeSQL.executeWithCacheContext(async () => {
361
+ // These methods participate in batch cache clearing
362
+ await forgeSQL.insert(Users).values(userData);
363
+ await forgeSQL.update(Users).set(updateData).where(eq(Users.id, 1));
364
+ await forgeSQL.delete(Users).where(eq(Users.id, 1));
365
+ // Cache is cleared only once at the end for all affected tables
366
+ });
367
+ ```
368
+
369
+
370
+ The diagram below shows the lifecycle of a cacheable query in Forge-SQL-ORM:
371
+
372
+ 1. Resolver calls forge-sql-orm with a SQL query and parameters.
373
+ 2. forge-sql-orm generates a cache key = hash(sql, parameters).
374
+ 3. It asks @forge/kvs for an existing cached result.
375
+ - Cache hit → result is returned immediately.
376
+ - Cache miss / expired → query is executed against @forge/sql.
377
+ 4. Fresh result is stored in @forge/kvs with TTL and returned to the caller.
378
+
379
+ ![img.png](img/umlCache1.png)
380
+
381
+
382
+ The diagram below shows how Evict Cache works in Forge-SQL-ORM:
383
+
384
+ 1. **Data modification** is executed through `@forge/sql` (e.g., `UPDATE users ...`).
385
+ 2. After a successful update, **forge-sql-orm** queries the `cache` entity by using the **`sql` field** with `filter.contains("users")` to find affected cached queries.
386
+ 3. The returned cache entries are deleted in **batches** (up to 25 per transaction).
387
+ 4. Once eviction is complete, the update result is returned to the resolver.
388
+ 5. **Note:** Expired entries are not processed here — they are cleaned up separately by the scheduled cache cleanup trigger using the `expiration` index.
389
+
390
+ ![img.png](img/umlCacheEvict1.png)
391
+
392
+ The diagram below shows how Scheduled Expiration Cleanup works:
393
+
394
+ 1. A periodic scheduler (Forge trigger) runs cache cleanup independently of data modifications.
395
+ 2. forge-sql-orm queries the cache entity by the expiration index to find entries with expiration < now.
396
+ 3. Entries are deleted in batches (up to 25 per transaction) until the page is empty; pagination is done with a cursor (e.g., 100 per page).
397
+ 4. This keeps the cache footprint small and prevents stale data accumulation.
398
+
399
+ ![img.png](img/umlCacheEvictScheduler1.png)
400
+
401
+ The diagram below shows how Cache Context works:
402
+
403
+ `executeWithCacheContext(fn)` lets you group multiple data modifications and perform **one consolidated cache eviction** at the end:
404
+
405
+ 1. The context starts with an empty `affectedTables` set.
406
+ 2. Each successful `INSERT/UPDATE/DELETE` inside the context registers its table name in `affectedTables`.
407
+ 3. **Reads inside the same context** that target tables present in `affectedTables` will **bypass the cache** (read-through to SQL) to avoid serving stale data. These reads also **do not write** back to cache until eviction completes.
408
+ 4. On context completion, `affectedTables` is de-duplicated and used to build **one combined KVS query** over the `sql` field with
409
+ `filter.or(filter.contains("<t1>"), filter.contains("<t2>"), ...)`, returning all impacted cache entries in a single scan (paged by cursor, e.g., 100/page).
410
+ 5. Matching cache entries are deleted in **batches** (≤25 per transaction) until the page is exhausted; then the next page is fetched via the cursor.
411
+ 6. Expiration is handled separately by the scheduled cleanup and is **not part of** the context flow.
412
+
413
+ ![img.png](img/umlCacheEvictCacheContext1.png)
414
+
415
+
416
+ ### Important Considerations
417
+
418
+ **@forge/kvs Limits:**
419
+ Please review the [official @forge/kvs quotas and limits](https://developer.atlassian.com/platform/forge/platform-quotas-and-limits/#kvs-and-custom-entity-store-quotas) before implementing caching.
420
+
421
+ **Caching Guidelines:**
422
+ - Don't cache everything - be selective about what to cache
423
+ - Don't cache simple and fast queries - sometimes direct query is faster than cache
424
+ - Consider data size and frequency of changes
425
+ - Monitor cache usage to stay within quotas
426
+ - Use appropriate TTL values
427
+
428
+ ### Step 1: Install Dependencies
429
+
430
+ ```bash
431
+ npm install @forge/kvs -S
432
+ ```
433
+
434
+ ### Step 2: Configure Manifest
435
+
436
+ Add the storage entity configuration and scheduler trigger to your `manifest.yml`:
437
+
438
+ ```yaml
439
+ modules:
440
+ scheduledTrigger:
441
+ - key: clear-cache-trigger
442
+ function: clearCache
443
+ interval: fiveMinute
444
+ storage:
445
+ entities:
446
+ - name: cache
447
+ attributes:
448
+ sql:
449
+ type: string
450
+ expiration:
451
+ type: integer
452
+ data:
453
+ type: string
454
+ indexes:
455
+ - sql
456
+ - expiration
457
+ sql:
458
+ - key: main
459
+ engine: mysql
460
+ function:
461
+ - key: clearCache
462
+ handler: index.clearCache
463
+ ```
464
+ ```typescript
465
+ // Example usage in your Forge app
466
+ import { clearCacheSchedulerTrigger } from "forge-sql-orm";
467
+
468
+ export const clearCache = () => {
469
+ return clearCacheSchedulerTrigger({
470
+ cacheEntityName: "cache",
471
+ });
472
+ };
473
+ ```
474
+
475
+
476
+ ### Step 3: Configure ORM Options
477
+
478
+ Set the cache entity name in your ForgeSQL configuration:
479
+
480
+ ```typescript
481
+ const options = {
482
+ cacheEntityName: "cache", // Must match the entity name in manifest.yml
483
+ cacheTTL: 300, // Default cache TTL in seconds (5 minutes)
484
+ cacheWrapTable: true, // Wrap table names with backticks in cache keys
485
+ // ... other options
486
+ };
487
+
488
+ const forgeSQL = new ForgeSQL(options);
119
489
  ```
120
490
 
121
- ## Drizzle Usage with forge-sql-orm
491
+ **Important Notes:**
492
+ - The `cacheEntityName` must exactly match the `name` in your manifest storage entities
493
+ - The entity attributes (`sql`, `expiration`, `data`) are required for proper cache functionality
494
+ - Indexes on `sql` and `expiration` improve cache lookup performance
495
+ - Cache data is automatically cleaned up based on TTL settings
496
+ - No additional permissions are required beyond standard Forge app permissions
497
+
498
+ ### Complete Setup Examples
122
499
 
123
- If you prefer to use Drizzle ORM with the additional features of Forge-SQL-ORM (like optimistic locking), you can use the custom driver:
500
+ **Basic setup (without caching):**
124
501
 
502
+ **package.json:**
503
+ ```shell
504
+ npm install forge-sql-orm @forge/sql drizzle-orm -S
505
+ ```
506
+
507
+ **manifest.yml:**
508
+ ```yaml
509
+ modules:
510
+ sql:
511
+ - key: main
512
+ engine: mysql
513
+ ```
514
+
515
+ **index.ts:**
125
516
  ```typescript
126
517
  import ForgeSQL from "forge-sql-orm";
518
+
127
519
  const forgeSQL = new ForgeSQL();
128
- forgeSQL.crud().insert(...);
129
- forgeSQL.crud().updateById(...);
130
- const db = forgeSQL.getDrizzleQueryBuilder();
131
520
 
132
- // Use drizzle
133
- const users = await db.select().from(users);
521
+ // simple insert
522
+ await forgeSQL.insert(Users, [userData]);
523
+ // Use versioned operations without caching
524
+ await forgeSQL.modifyWithVersioning().insert(Users, [userData]);
525
+ const users = await forgeSQL.select({id: Users.id});
526
+
527
+
134
528
  ```
135
529
 
136
- This approach gives you direct access to all Drizzle ORM features while still using the @forge/sql backend.
530
+ **With caching support:**
531
+ ```shell
532
+ npm install forge-sql-orm @forge/sql @forge/kvs drizzle-orm -S
533
+ ```
534
+
535
+ **manifest.yml:**
536
+ ```yaml
537
+ modules:
538
+ scheduledTrigger:
539
+ - key: clear-cache-trigger
540
+ function: clearCache
541
+ interval: fiveMinute
542
+ storage:
543
+ entities:
544
+ - name: cache
545
+ attributes:
546
+ sql:
547
+ type: string
548
+ expiration:
549
+ type: integer
550
+ data:
551
+ type: string
552
+ indexes:
553
+ - sql
554
+ - expiration
555
+ sql:
556
+ - key: main
557
+ engine: mysql
558
+ function:
559
+ - key: clearCache
560
+ handler: index.clearCache
561
+ ```
562
+
563
+ **index.ts:**
564
+
565
+ ```typescript
566
+ import ForgeSQL from "forge-sql-orm";
567
+
568
+ const forgeSQL = new ForgeSQL({
569
+ cacheEntityName: "cache"
570
+ });
571
+
572
+ import {clearCacheSchedulerTrigger} from "forge-sql-orm";
573
+ import {getTableColumns} from "drizzle-orm";
574
+
575
+ export const clearCache = () => {
576
+ return clearCacheSchedulerTrigger({
577
+ cacheEntityName: "cache",
578
+ });
579
+ };
580
+
581
+
582
+ // Now you can use caching features
583
+ const usersData = await forgeSQL.selectCacheable(getTableColumns(users)).from(users).where(eq(users.active, true))
584
+
585
+ // simple insert
586
+ await forgeSQL.insertAndEvictCache(users, [userData]);
587
+ // Use versioned operations with caching
588
+ await forgeSQL.modifyWithVersioningAndEvictCache().insert(users, [userData]);
589
+
590
+ // use Cache Context
591
+ const data = await forgeSQL.executeWithCacheContextAndReturnValue(async () => {
592
+ // after insert mark users to evict
593
+ await forgeSQL.insert(users, [userData]);
594
+ // after insertAndEvictCache mark orders to evict
595
+ await forgeSQL.insertAndEvictCache(orders, [order1, order2]);
596
+ // execute query and put result to local cache
597
+ await forgeSQL.selectCacheable({userId: users.id, userName: users.name, orderId: orders.id, orderName: orders.name})
598
+ .from(users)
599
+ .innerJoin(orders, eq(orders.userId, users.id)).where(eq(users.active, true))
600
+ // use local cache without @forge/kvs and @forge/sql
601
+ return await forgeSQL.selectCacheable({userId: users.id, userName: users.name, orderId: orders.id, orderName: orders.name})
602
+ .from(users)
603
+ .innerJoin(orders, eq(orders.userId, users.id)).where(eq(users.active, true))
604
+ })
605
+ // execute query and put result to kvs cache
606
+ await forgeSQL.selectCacheable({userId: users.id, userName: users.name, orderId: orders.id, orderName: orders.name})
607
+ .from(users)
608
+ .innerJoin(orders, eq(orders.userId, users.id)).where(eq(users.active, true))
609
+
610
+ // get result from @foge/kvs cache without real @forge/sql call
611
+ await forgeSQL.selectCacheable({userId: users.id, userName: users.name, orderId: orders.id, orderName: orders.name})
612
+ .from(users)
613
+ .innerJoin(orders, eq(orders.userId, users.id)).where(eq(users.active, true))
614
+
615
+ // use Local Cache for performance optimization
616
+ const optimizedData = await forgeSQL.executeWithLocalCacheContextAndReturnValue(async () => {
617
+ // First query - hits database and caches result
618
+ const users = await forgeSQL.select({id: users.id, name: users.name})
619
+ .from(users).where(eq(users.active, true));
620
+
621
+ // Second query - uses local cache (no database call)
622
+ const cachedUsers = await forgeSQL.select({id: users.id, name: users.name})
623
+ .from(users).where(eq(users.active, true));
624
+
625
+ // Insert operation - evicts local cache
626
+ await forgeSQL.insert(users).values({name: 'New User', active: true});
627
+
628
+ // Third query - hits database again and caches new result
629
+ const updatedUsers = await forgeSQL.select({id: users.id, name: users.name})
630
+ .from(users).where(eq(users.active, true));
631
+
632
+ return { users, cachedUsers, updatedUsers };
633
+ });
634
+
635
+ ```
636
+
637
+ ## Choosing the Right Method - ForgeSQL ORM
638
+
639
+ ### When to Use Each Approach
640
+
641
+ | Method | Use Case | Versioning | Cache Management |
642
+ |--------|----------|------------|------------------|
643
+ | `modifyWithVersioningAndEvictCache()` | High-concurrency scenarios with Cache support| ✅ Yes | ✅ Yes |
644
+ | `modifyWithVersioning()` | High-concurrency scenarios | ✅ Yes | Cache Context |
645
+ | `insertAndEvictCache()` | Simple inserts | ❌ No | ✅ Yes |
646
+ | `updateAndEvictCache()` | Simple updates | ❌ No | ✅ Yes |
647
+ | `deleteAndEvictCache()` | Simple deletes | ❌ No | ✅ Yes |
648
+ | `insert/update/delete` | Basic Drizzle operations | ❌ No | Cache Context |
649
+
650
+
651
+ ## Choosing the Right Method - Direct Drizzle
652
+
653
+ ### When to Use Each Approach
654
+
655
+ | Method | Use Case | Versioning | Cache Management |
656
+ |--------|----------|------------|------------------|
657
+ | `insertWithCacheContext/insertWithCacheContext/updateWithCacheContext` | Basic Drizzle operations | ❌ No | Cache Context |
658
+ | `insertAndEvictCache()` | Simple inserts without conflicts | ❌ No | ✅ Yes |
659
+ | `updateAndEvictCache()` | Simple updates without conflicts | ❌ No | ✅ Yes |
660
+ | `deleteAndEvictCache()` | Simple deletes without conflicts | ❌ No | ✅ Yes |
661
+ | `insert/update/delete` | Basic Drizzle operations | ❌ No | ❌ No |
662
+ where Cache context - allows you to batch cache invalidation events and bypass cache reads for affected tables.
663
+
137
664
 
138
665
  ## Step-by-Step Migration Workflow
139
666
 
@@ -344,6 +871,21 @@ const users = await db.select().from(users);
344
871
  ### Basic Fetch Operations
345
872
 
346
873
  ```js
874
+ // Using forgeSQL.select()
875
+ const user = await forgeSQL
876
+ .select({user: users})
877
+ .from(users);
878
+
879
+ // Using forgeSQL.selectDistinct()
880
+ const user = await forgeSQL
881
+ .selectDistinct({user: users})
882
+ .from(users);
883
+
884
+ // Using forgeSQL.selectCacheable()
885
+ const user = await forgeSQL
886
+ .selectCacheable({user: users})
887
+ .from(users);
888
+
347
889
  // Using forgeSQL.getDrizzleQueryBuilder()
348
890
  const user = await forgeSQL
349
891
  .getDrizzleQueryBuilder()
@@ -442,30 +984,101 @@ const users = await forgeSQL
442
984
  .executeRawSQL<Users>("SELECT * FROM users");
443
985
  ```
444
986
 
445
- ## CRUD Operations
987
+ ## Modify Operations
988
+
989
+ Forge-SQL-ORM provides multiple approaches for Modify operations, each with different characteristics:
990
+
991
+ ### 1. Basic Drizzle Operations (Cache Context Aware)
992
+
993
+ These operations work like standard Drizzle methods but participate in cache context when used within `executeWithCacheContext()`:
994
+
995
+ ```js
996
+ // Basic insert (participates in cache context when used within executeWithCacheContext)
997
+ await forgeSQL.insert(Users).values({ id: 1, name: "Smith" });
998
+
999
+ // Basic update (participates in cache context when used within executeWithCacheContext)
1000
+ await forgeSQL.update(Users)
1001
+ .set({ name: "Smith Updated" })
1002
+ .where(eq(Users.id, 1));
1003
+
1004
+ // Basic delete (participates in cache context when used within executeWithCacheContext)
1005
+ await forgeSQL.delete(Users)
1006
+ .where(eq(Users.id, 1));
1007
+ ```
1008
+
1009
+ ### 2. Non-Versioned Operations with Cache Management
446
1010
 
447
- ### Insert Operations
1011
+ These operations don't use optimistic locking but provide cache invalidation:
448
1012
 
449
- ```js
450
- // Single insert
451
- const userId = await forgeSQL.crud().insert(Users, [{ id: 1, name: "Smith" }]);
1013
+ ```js
1014
+ // Insert without versioning but with cache invalidation
1015
+ await forgeSQL.insertAndEvictCache(Users).values({ id: 1, name: "Smith" });
1016
+
1017
+ // Update without versioning but with cache invalidation
1018
+ await forgeSQL.updateAndEvictCache(Users)
1019
+ .set({ name: "Smith Updated" })
1020
+ .where(eq(Users.id, 1));
1021
+
1022
+ // Delete without versioning but with cache invalidation
1023
+ await forgeSQL.deleteAndEvictCache(Users)
1024
+ .where(eq(Users.id, 1));
1025
+ ```
452
1026
 
453
- // Bulk insert
454
- await forgeSQL.crud().insert(Users, [
1027
+ ### 3. Versioned Operations with Cache Management (Recommended)
1028
+
1029
+ These operations use optimistic locking and automatic cache invalidation:
1030
+
1031
+ ```js
1032
+ // Insert with versioning and cache management
1033
+ const userId = await forgeSQL.modifyWithVersioningAndEvictCache().insert(Users, [{ id: 1, name: "Smith" }]);
1034
+
1035
+ // Bulk insert with versioning
1036
+ await forgeSQL.modifyWithVersioningAndEvictCache().insert(Users, [
455
1037
  { id: 2, name: "Smith" },
456
1038
  { id: 3, name: "Vasyl" },
457
1039
  ]);
458
1040
 
459
- // Insert with duplicate handling
460
- await forgeSQL.crud().insert(
461
- Users,
462
- [
463
- { id: 4, name: "Smith" },
464
- { id: 4, name: "Vasyl" },
465
- ],
466
- true
467
- );
1041
+ // Update by ID with optimistic locking and cache invalidation
1042
+ await forgeSQL.modifyWithVersioningAndEvictCache().updateById({ id: 1, name: "Smith Updated" }, Users);
1043
+
1044
+ // Delete by ID with versioning and cache invalidation
1045
+ await forgeSQL.modifyWithVersioningAndEvictCache().deleteById(1, Users);
1046
+ ```
1047
+
1048
+ ### 4. Versioned Operations without Cache Management
468
1049
 
1050
+ These operations use optimistic locking but don't manage cache:
1051
+
1052
+ ```js
1053
+ // Insert with versioning only (no cache management)
1054
+ const userId = await forgeSQL.modifyWithVersioning().insert(Users, [{ id: 1, name: "Smith" }]);
1055
+
1056
+ // Update with versioning only
1057
+ await forgeSQL.modifyWithVersioning().updateById({ id: 1, name: "Smith Updated" }, Users);
1058
+
1059
+ // Delete with versioning only
1060
+ await forgeSQL.modifyWithVersioning().deleteById(1, Users);
1061
+ ```
1062
+
1063
+ ### 5. Legacy Modify Operations (Removed in 2.1.x)
1064
+
1065
+ ⚠️ **BREAKING CHANGE**: The `crud()` and `modify()` methods have been completely removed in version 2.1.x.
1066
+
1067
+ ```js
1068
+ // ❌ These methods no longer exist in 2.1.x
1069
+ // const userId = await forgeSQL.crud().insert(Users, [{ id: 1, name: "Smith" }]);
1070
+ // await forgeSQL.crud().updateById({ id: 1, name: "Smith Updated" }, Users);
1071
+ // await forgeSQL.crud().deleteById(1, Users);
1072
+
1073
+ // ✅ Use the new methods instead
1074
+ const userId = await forgeSQL.modifyWithVersioning().insert(Users, [{ id: 1, name: "Smith" }]);
1075
+ await forgeSQL.modifyWithVersioning().updateById({ id: 1, name: "Smith Updated" }, Users);
1076
+ await forgeSQL.modifyWithVersioning().deleteById(1, Users);
1077
+ ```
1078
+
1079
+ ### Advanced Operations
1080
+
1081
+ ```js
469
1082
  // Insert with sequence (nextVal)
470
1083
  import { nextVal } from "forge-sql-orm";
471
1084
 
@@ -474,38 +1087,24 @@ const user = {
474
1087
  name: "user test",
475
1088
  organization_id: 1
476
1089
  };
477
- const id = await forgeSQL.modify().insert(appUser, [user]);
478
-
479
- // The generated SQL will be:
480
- // INSERT INTO app_user (id, name, organization_id)
481
- // VALUES (NEXTVAL(user_id_seq), ?, ?) -- params: ["user test", 1]
482
- ```
483
-
484
- ### Update Operations
485
-
486
- ```js
487
- // Update by ID with optimistic locking
488
- await forgeSQL.crud().updateById({ id: 1, name: "Smith Updated" }, Users);
489
-
490
- // Update specific fields
491
- await forgeSQL.crud().updateById(
492
- { id: 1, age: 35 },
493
- Users
494
- );
1090
+ const id = await forgeSQL.modifyWithVersioning().insert(appUser, [user]);
495
1091
 
496
1092
  // Update with custom WHERE condition
497
- await forgeSQL.crud().updateFields(
1093
+ await forgeSQL.modifyWithVersioning().updateFields(
498
1094
  { name: "New Name", age: 35 },
499
1095
  Users,
500
1096
  eq(Users.email, "smith@example.com")
501
1097
  );
502
- ```
503
-
504
- ### Delete Operations
505
1098
 
506
- ```js
507
- // Delete by ID
508
- await forgeSQL.crud().deleteById(1, Users);
1099
+ // Insert with duplicate handling
1100
+ await forgeSQL.modifyWithVersioning().insert(
1101
+ Users,
1102
+ [
1103
+ { id: 4, name: "Smith" },
1104
+ { id: 4, name: "Vasyl" },
1105
+ ],
1106
+ true
1107
+ );
509
1108
  ```
510
1109
 
511
1110
  ## SQL Utilities
@@ -543,8 +1142,267 @@ const result = await forgeSQL
543
1142
  - This prevents SQL injection by ensuring only numeric values are inserted
544
1143
  - Always use this function instead of string concatenation for LIMIT and OFFSET values
545
1144
 
1145
+ ## Global Cache System (Level 2)
1146
+
1147
+ [↑ Back to Top](#table-of-contents)
1148
+
1149
+ Forge-SQL-ORM includes a sophisticated global caching system that provides **cross-invocation caching** - the ability to share cached data between different resolver invocations. The global cache system is built on top of [@forge/kvs Custom entity store](https://developer.atlassian.com/platform/forge/storage-reference/storage-api-custom-entities/) and provides persistent cross-invocation caching with automatic serialization/deserialization of complex data structures.
1150
+
1151
+ ### Cache Levels Overview
1152
+
1153
+ Forge-SQL-ORM implements a two-level caching architecture:
1154
+
1155
+ - **Level 1 (Local Cache)**: In-memory caching within a single resolver invocation scope
1156
+ - **Level 2 (Global Cache)**: Cross-invocation persistent caching using KVS storage
1157
+
1158
+ This multi-level approach provides optimal performance by checking the fastest cache first, then falling back to cross-invocation persistent storage.
1159
+
1160
+ ### Cache Configuration
1161
+
1162
+ The caching system uses Atlassian Forge's Custom entity store to persist cache data. Each cache entry is stored as a custom entity with automatic TTL management and efficient key-based retrieval.
1163
+
1164
+ ```typescript
1165
+ const options = {
1166
+ cacheEntityName: "cache", // KVS Custom entity name for cache storage
1167
+ cacheTTL: 300, // Default cache TTL in seconds (5 minutes)
1168
+ cacheWrapTable: true, // Wrap table names with backticks in cache keys
1169
+ additionalMetadata: {
1170
+ users: {
1171
+ tableName: "users",
1172
+ versionField: {
1173
+ fieldName: "updatedAt",
1174
+ }
1175
+ }
1176
+ }
1177
+ };
1178
+
1179
+ const forgeSQL = new ForgeSQL(options);
1180
+ ```
1181
+
1182
+ ### How Caching Works with @forge/kvs
1183
+
1184
+ The caching system leverages Forge's Custom entity store to provide:
1185
+
1186
+ - **Persistent Storage**: Cache data survives app restarts and deployments
1187
+ - **Automatic TTL**: Built-in expiration handling through Forge's entity lifecycle
1188
+ - **Efficient Retrieval**: Fast key-based lookups using Forge's optimized storage
1189
+ - **Data Serialization**: Automatic handling of complex objects and query results
1190
+ - **Batch Operations**: Efficient bulk cache operations for better performance
1191
+
1192
+ ```typescript
1193
+ // Cache entries are stored as custom entities in Forge's KVS
1194
+ // Example cache key structure:
1195
+ // Key: "CachedQuery_8d74bdd9d85064b72fb2ee072ca948e5"
1196
+ // Value: { data: [...], expiration: 1234567890, sql: "select * from 1" }
1197
+ ```
1198
+
1199
+
1200
+ ### Cache Context Operations
1201
+
1202
+ The cache context allows you to batch cache invalidation events and bypass cache reads for affected tables:
1203
+
1204
+ ```typescript
1205
+ // Execute operations within a cache context
1206
+ await forgeSQL.executeWithCacheContext(async () => {
1207
+ // All cache invalidation events are collected and executed in batch
1208
+ await forgeSQL.modifyWithVersioningAndEvictCache().insert(Users, [userData]);
1209
+ await forgeSQL.modifyWithVersioningAndEvictCache().updateById(updateData, Users);
1210
+ // Cache is cleared only once at the end for all affected tables
1211
+ });
1212
+
1213
+ // Execute with return value
1214
+ const result = await forgeSQL.executeWithCacheContextAndReturnValue(async () => {
1215
+ const user = await forgeSQL.modifyWithVersioningAndEvictCache().insert(Users, [userData]);
1216
+ return user;
1217
+ });
1218
+
1219
+ // Basic operations also participate in cache context
1220
+ await forgeSQL.executeWithCacheContext(async () => {
1221
+ // These operations will participate in batch cache clearing
1222
+ await forgeSQL.insert(Users).values(userData);
1223
+ await forgeSQL.update(Users).set(updateData).where(eq(Users.id, 1));
1224
+ await forgeSQL.delete(Users).where(eq(Users.id, 1));
1225
+ // Cache is cleared only once at the end for all affected tables
1226
+ });
1227
+ ```
1228
+
1229
+ ### Local Cache Operations (Level 1)
1230
+
1231
+ Forge-SQL-ORM provides a local cache system (Level 1 cache) that stores query results in memory for the duration of a single resolver invocation. This is particularly useful for optimizing repeated queries within the same execution context(resolver invocation).
1232
+
1233
+ #### What is Local Cache?
1234
+
1235
+ Local cache is an in-memory caching layer that operates within a single resolver invocation scope. Unlike the global KVS cache, local cache:
1236
+
1237
+ - **Stores data in memory** using Node.js `AsyncLocalStorage`
1238
+ - **Automatically clears** when the invocation completes (Resolver call)
1239
+ - **Provides instant access** to previously executed queries in resolver invocation
1240
+ - **Reduces database load** for repeated operations within the same invocation
1241
+ - **Works alongside** the global KVS cache system
1242
+
1243
+ #### Key Features of Local Cache
1244
+
1245
+ - **In-Memory Storage**: Query results are cached in memory using Node.js `AsyncLocalStorage`
1246
+ - **Invocation-Scoped**: Cache is automatically cleared when the invocation completes
1247
+ - **Automatic Eviction**: Cache is cleared when insert/update/delete operations are performed
1248
+ - **No Persistence**: Data is not stored between Invocations (unlike global KVS cache)
1249
+ - **Performance Optimization**: Reduces database queries for repeated operations
1250
+ - **Simple Configuration**: Works out of the box with simple setup
1251
+
1252
+ #### Usage Examples
1253
+
1254
+ ##### Basic Local Cache Usage
1255
+
1256
+ ```typescript
1257
+ // Execute operations within a local cache context
1258
+ await forgeSQL.executeWithLocalContext(async () => {
1259
+ // First call - executes query and caches result
1260
+ const users = await forgeSQL.select({ id: users.id, name: users.name })
1261
+ .from(users).where(eq(users.active, true));
1262
+
1263
+ // Second call - gets result from local cache (no database query)
1264
+ const cachedUsers = await forgeSQL.select({ id: users.id, name: users.name })
1265
+ .from(users).where(eq(users.active, true));
1266
+
1267
+ // Insert operation - evicts local cache for users table
1268
+ await forgeSQL.insert(users).values({ name: 'New User', active: true });
1269
+
1270
+ // Third call - executes query again and caches new result
1271
+ const updatedUsers = await forgeSQL.select({ id: users.id, name: users.name })
1272
+ .from(users).where(eq(users.active, true));
1273
+ });
1274
+
1275
+ // Execute with return value
1276
+ const result = await forgeSQL.executeWithLocalCacheContextAndReturnValue(async () => {
1277
+ // First call - executes query and caches result
1278
+ const users = await forgeSQL.select({ id: users.id, name: users.name })
1279
+ .from(users).where(eq(users.active, true));
1280
+
1281
+ // Second call - gets result from local cache (no database query)
1282
+ const cachedUsers = await forgeSQL.select({ id: users.id, name: users.name })
1283
+ .from(users).where(eq(users.active, true));
1284
+
1285
+ return { users, cachedUsers };
1286
+ });
1287
+ ```
1288
+
1289
+ ##### Real-World Resolver Example
1290
+
1291
+ ```typescript
1292
+ // Atlassian forge resolver with local cache optimization
1293
+ const userResolver = async (req) => {
1294
+ return await forgeSQL.executeWithLocalCacheContextAndReturnValue(async () => {
1295
+ // Get user details
1296
+ const user = await forgeSQL.select({ id: users.id, name: users.name, email: users.email })
1297
+ .from(users).where(eq(users.id, args.userId));
1298
+
1299
+ // Get user's orders (this query will be cached if called again)
1300
+ const orders = await forgeSQL.select({
1301
+ id: orders.id,
1302
+ product: orders.product,
1303
+ amount: orders.amount
1304
+ }).from(orders).where(eq(orders.userId, args.userId));
1305
+
1306
+ // Get user's profile (this query will be cached if called again)
1307
+ const profile = await forgeSQL.select({
1308
+ id: profiles.id,
1309
+ bio: profiles.bio,
1310
+ avatar: profiles.avatar
1311
+ }).from(profiles).where(eq(profiles.userId, args.userId));
1312
+
1313
+ // If any of these queries are repeated within the same resolver,
1314
+ // they will use the local cache instead of hitting the database
1315
+
1316
+ return {
1317
+ ...user[0],
1318
+ orders,
1319
+ profile: profile[0]
1320
+ };
1321
+ });
1322
+ };
1323
+ ```
1324
+
1325
+
1326
+ #### Local Cache (Level 1) vs Global Cache (Level 2)
1327
+
1328
+ | Feature | Local Cache (Level 1) | Global Cache (Level 2) |
1329
+ |---------|----------------------|------------------------|
1330
+ | **Storage** | In-memory (Node.js process) | Persistent (KVS Custom Entities) |
1331
+ | **Scope** | Single forge invocation | Cross-invocation (between calls) |
1332
+ | **Persistence** | No (cleared on invocation end) | Yes (survives app redeploy) |
1333
+ | **Performance** | Very fast (memory access) | Fast (KVS optimized storage) |
1334
+ | **Memory Usage** | Low (invocation-scoped) | Higher (persistent storage) |
1335
+ | **Use Case** | Invocation optimization | Cross-invocation data sharing |
1336
+ | **Configuration** | None required | Requires KVS setup |
1337
+ | **TTL Support** | No (invocation-scoped) | Yes (automatic expiration) |
1338
+ | **Cache Eviction** | Automatic on DML operations | Manual or scheduled cleanup |
1339
+ | **Best For** | Repeated queries in single invocation | Frequently accessed data across invocations |
1340
+
1341
+ #### Integration with Global Cache (Level 2)
1342
+
1343
+ Local cache (Level 1) works alongside the global cache (Level 2) system:
1344
+
1345
+ ```typescript
1346
+ // Multi-level cache checking: Level 1 → Level 2 → Database
1347
+ await forgeSQL.executeWithLocalContext(async () => {
1348
+ // This will check:
1349
+ // 1. Local cache (Level 1 - in-memory)
1350
+ // 2. Global cache (Level 2 - KVS)
1351
+ // 3. Database query
1352
+ const users = await forgeSQL.selectCacheable({ id: users.id, name: users.name })
1353
+ .from(users).where(eq(users.active, true));
1354
+ });
1355
+ ```
1356
+
1357
+ #### Local Cache Flow Diagram
1358
+
1359
+ The diagram below shows how local cache works in Forge-SQL-ORM:
1360
+
1361
+ 1. **Request Start**: Local cache context is initialized with empty cache
1362
+ 2. **First Query**: Cache miss → Global cache miss → Database query → Save to local cache
1363
+ 3. **Repeated Query**: Cache hit → Return cached result (no database call)
1364
+ 4. **Data Modification**: Insert/Update/Delete → Evict local cache for affected table
1365
+ 5. **Query After Modification**: Cache miss (was evicted) → Database query → Save to local cache
1366
+ 6. **Request End**: Local cache context is destroyed, all data cleared
1367
+
1368
+ ![Local Cache Flow](img/localCacheFlow.txt)
1369
+
1370
+ ### Cache-Aware Query Operations
1371
+
1372
+ ```typescript
1373
+ // Execute queries with caching
1374
+ const users = await forgeSQL.modifyWithVersioningAndEvictCache().executeQuery(
1375
+ forgeSQL.select().from(Users).where(eq(Users.active, true)),
1376
+ 600 // Custom TTL in seconds
1377
+ );
1378
+
1379
+ // Execute single result queries with caching
1380
+ const user = await forgeSQL.modifyWithVersioningAndEvictCache().executeQueryOnlyOne(
1381
+ forgeSQL.select().from(Users).where(eq(Users.id, 1))
1382
+ );
1383
+
1384
+ // Execute raw SQL with caching
1385
+ const results = await forgeSQL.modifyWithVersioningAndEvictCache().executeRawSQL(
1386
+ "SELECT * FROM users WHERE active = ?",
1387
+ [true],
1388
+ 300 // TTL in seconds
1389
+ );
1390
+ ```
1391
+
1392
+ ### Manual Cache Management
1393
+
1394
+ ```typescript
1395
+ // Clear cache for specific tables
1396
+ await forgeSQL.modifyWithVersioningAndEvictCache().evictCache(["users", "orders"]);
1397
+
1398
+ // Clear cache for specific entities
1399
+ await forgeSQL.modifyWithVersioningAndEvictCache().evictCacheEntities([Users, Orders]);
1400
+ ```
1401
+
546
1402
  ## Optimistic Locking
547
1403
 
1404
+ [↑ Back to Top](#table-of-contents)
1405
+
548
1406
  Optimistic locking is a concurrency control mechanism that prevents data conflicts when multiple transactions attempt to update the same record concurrently. Instead of using locks, this technique relies on a version field in your entity models.
549
1407
 
550
1408
  ### Supported Version Field Types
@@ -575,7 +1433,19 @@ const forgeSQL = new ForgeSQL(options);
575
1433
 
576
1434
  ```typescript
577
1435
  // The version field will be automatically handled
578
- await forgeSQL.crud().updateById(
1436
+ await forgeSQL.modifyWithVersioning().updateById(
1437
+ {
1438
+ id: 1,
1439
+ name: "Updated Name",
1440
+ updatedAt: new Date() // Will be automatically set if not provided
1441
+ },
1442
+ Users
1443
+ );
1444
+ ```
1445
+ or with cache support
1446
+ ```typescript
1447
+ // The version field will be automatically handled
1448
+ await forgeSQL.modifyWithVersioningAndEvictCache().updateById(
579
1449
  {
580
1450
  id: 1,
581
1451
  name: "Updated Name",
@@ -594,10 +1464,46 @@ The `ForgeSqlOrmOptions` object allows customization of ORM behavior:
594
1464
  | `logRawSqlQuery` | `boolean` | Enables logging of raw SQL queries in the Atlassian Forge Developer Console. Useful for debugging and monitoring. Defaults to `false`. |
595
1465
  | `disableOptimisticLocking` | `boolean` | Disables optimistic locking. When set to `true`, no additional condition (e.g., a version check) is added during record updates, which can improve performance. However, this may lead to conflicts when multiple transactions attempt to update the same record concurrently. |
596
1466
  | `additionalMetadata` | `object` | Allows adding custom metadata to all entities. This is useful for tracking common fields across all tables (e.g., `createdAt`, `updatedAt`, `createdBy`, etc.). The metadata will be automatically added to all generated entities. |
1467
+ | `cacheEntityName` | `string` | KVS Custom entity name for cache storage. Must match the `name` in your `manifest.yml` storage entities configuration. Required for caching functionality. Defaults to `"cache"`. |
1468
+ | `cacheTTL` | `number` | Default cache TTL in seconds. Defaults to `120` (2 minutes). |
1469
+ | `cacheWrapTable` | `boolean` | Whether to wrap table names with backticks in cache keys. Defaults to `true`. |
1470
+ | `hints` | `object` | SQL hints for query optimization. Optional configuration for advanced query tuning. |
597
1471
 
598
1472
  ## CLI Commands
599
1473
 
600
- Documentation [here](forge-sql-orm-cli/README.md)
1474
+ Forge-SQL-ORM provides a command-line interface for managing database migrations and model generation.
1475
+
1476
+ **📖 [Full CLI Documentation](forge-sql-orm-cli/README.md)** - Complete CLI reference with all commands and options.
1477
+
1478
+ ### Quick CLI Reference
1479
+
1480
+ The CLI tool provides the following main commands:
1481
+
1482
+ - `generate:model` - Generate Drizzle ORM models from your database schema
1483
+ - `migrations:create` - Create new migration files
1484
+ - `migrations:update` - Update existing migrations with schema changes
1485
+ - `migrations:drop` - Create migration to drop tables
1486
+
1487
+ ### Installation
1488
+
1489
+ ```bash
1490
+ npm install -g forge-sql-orm-cli
1491
+ ```
1492
+
1493
+ ### Basic Usage
1494
+
1495
+ ```bash
1496
+ # Generate models from database
1497
+ forge-sql-orm-cli generate:model --dbName myapp --output ./database/entities
1498
+
1499
+ # Create migration
1500
+ forge-sql-orm-cli migrations:create --dbName myapp --entitiesPath ./database/entities
1501
+
1502
+ # Update migration
1503
+ forge-sql-orm-cli migrations:update --dbName myapp --entitiesPath ./database/entities
1504
+ ```
1505
+
1506
+ For detailed information about all available options and advanced usage, see the [Full CLI Documentation](forge-sql-orm-cli/README.md).
601
1507
 
602
1508
  ## Web Triggers for Migrations
603
1509
 
@@ -718,6 +1624,41 @@ CREATE TABLE IF NOT EXISTS orders (...);
718
1624
  SET foreign_key_checks = 1;
719
1625
  ```
720
1626
 
1627
+ ### 4. Clear Cache Scheduler Trigger
1628
+
1629
+ This trigger automatically cleans up expired cache entries based on their TTL (Time To Live). It's useful for:
1630
+ - Automatic cache maintenance
1631
+ - Preventing cache storage from growing indefinitely
1632
+ - Ensuring optimal cache performance
1633
+ - Reducing storage costs
1634
+
1635
+ ```typescript
1636
+ // Example usage in your Forge app
1637
+ import { clearCacheSchedulerTrigger } from "forge-sql-orm";
1638
+
1639
+ export const clearCache = () => {
1640
+ return clearCacheSchedulerTrigger({
1641
+ cacheEntityName: "cache",
1642
+ });
1643
+ };
1644
+ ```
1645
+
1646
+ Configure in `manifest.yml`:
1647
+ ```yaml
1648
+ scheduledTrigger:
1649
+ - key: clear-cache-trigger
1650
+ function: clearCache
1651
+ interval: fiveMinute
1652
+ function:
1653
+ - key: clearCache
1654
+ handler: index.clearCache
1655
+ ```
1656
+
1657
+ **Available Intervals**:
1658
+ - `fiveMinute` - Every 5 minutes
1659
+ - `hour` - Every hour
1660
+ - `day` - Every day
1661
+
721
1662
  ### Important Notes
722
1663
 
723
1664
  **Security Considerations**:
@@ -733,34 +1674,20 @@ SET foreign_key_checks = 1;
733
1674
 
734
1675
  ## Query Analysis and Performance Optimization
735
1676
 
736
- ⚠️ **IMPORTANT NOTE**: The query analysis features described below are experimental and should be used only for troubleshooting purposes. These features rely on TiDB's `information_schema` and `performance_schema` which may change in future updates. As of April 2025, these features are available but their future availability is not guaranteed.
1677
+ [↑ Back to Top](#table-of-contents)
1678
+
1679
+ Forge-SQL-ORM provides comprehensive query analysis tools to help you optimize your database queries and identify performance bottlenecks.
737
1680
 
738
1681
  ### About Atlassian's Built-in Analysis Tools
739
1682
 
740
- Atlassian already provides comprehensive query analysis tools in the development console, including:
1683
+ Atlassian provides comprehensive query analysis tools in the development console, including:
741
1684
  - Basic query performance metrics
742
1685
  - Slow query tracking (queries over 500ms)
743
1686
  - Basic execution statistics
744
1687
  - Query history and patterns
745
1688
 
746
- Our analysis tools are designed to complement these built-in features by providing additional insights directly from TiDB's system schemas. However, they should be used with caution and only for troubleshooting purposes.
747
-
748
- ### Usage Guidelines
1689
+ Our analysis tools complement these built-in features by providing additional insights directly from TiDB's system schemas.
749
1690
 
750
- 1. **Development and Troubleshooting Only**
751
- - These tools should not be used in production code
752
- - Intended only for development and debugging
753
- - Use for identifying and fixing performance issues
754
-
755
- 2. **Schema Stability**
756
- - Features rely on TiDB's `information_schema` and `performance_schema`
757
- - Schema structure may change in future TiDB updates
758
- - No guarantee of long-term availability
759
-
760
- 3. **Current Availability (April 2025)**
761
- - `information_schema` based analysis is currently functional
762
- - Query plan analysis is available
763
- - Performance metrics collection is working
764
1691
 
765
1692
  ### Available Analysis Tools
766
1693
 
@@ -773,10 +1700,10 @@ const analyzeForgeSql = forgeSQL.analyze();
773
1700
 
774
1701
  #### Query Plan Analysis
775
1702
 
776
- ⚠️ **For Troubleshooting Only**: This feature should only be used during development and debugging sessions.
1703
+ Query plan analysis helps you understand how your queries are executed and identify optimization opportunities.
777
1704
 
778
1705
  ```typescript
779
- // Example usage for troubleshooting a specific query
1706
+ // Example usage for analyzing a specific query
780
1707
  const forgeSQL = new ForgeSQL();
781
1708
  const analyzeForgeSql = forgeSQL.analyze();
782
1709
 
@@ -803,13 +1730,95 @@ const rawPlan = await analyzeForgeSql.explainRaw(
803
1730
  );
804
1731
  ```
805
1732
 
806
- This analysis helps you understand:
1733
+ This analysis provides insights into:
807
1734
  - How the database executes your query
808
1735
  - Which indexes are being used
809
1736
  - Estimated vs actual row counts
810
1737
  - Resource usage at each step
811
- - Potential performance bottlenecks
1738
+ - Performance optimization opportunities
1739
+
1740
+
1741
+ ## Migration Guide
1742
+
1743
+ ### Migrating from 2.0.x to 2.1.x
1744
+
1745
+ This section covers the breaking changes introduced in version 2.1.x and how to migrate your existing code.
1746
+
1747
+ #### 1. Method Renaming (BREAKING CHANGES)
1748
+
1749
+ **Removed Methods:**
1750
+ - `forgeSQL.modify()` → **REMOVED** (use `forgeSQL.modifyWithVersioning()`)
1751
+ - `forgeSQL.crud()` → **REMOVED** (use `forgeSQL.modifyWithVersioning()`)
1752
+
1753
+ **Migration Steps:**
1754
+
1755
+ 1. **Replace `modify()` calls:**
1756
+ ```typescript
1757
+ // ❌ Old (2.0.x) - NO LONGER WORKS
1758
+ await forgeSQL.modify().insert(Users, [userData]);
1759
+ await forgeSQL.modify().updateById(updateData, Users);
1760
+ await forgeSQL.modify().deleteById(1, Users);
1761
+
1762
+ // ✅ New (2.1.x) - REQUIRED
1763
+ await forgeSQL.modifyWithVersioning().insert(Users, [userData]);
1764
+ await forgeSQL.modifyWithVersioning().updateById(updateData, Users);
1765
+ await forgeSQL.modifyWithVersioning().deleteById(1, Users);
1766
+ ```
1767
+
1768
+ 2. **Replace `crud()` calls:**
1769
+ ```typescript
1770
+ // ❌ Old (2.0.x) - NO LONGER WORKS
1771
+ await forgeSQL.crud().insert(Users, [userData]);
1772
+ await forgeSQL.crud().updateById(updateData, Users);
1773
+ await forgeSQL.crud().deleteById(1, Users);
1774
+
1775
+ // ✅ New (2.1.x) - REQUIRED
1776
+ await forgeSQL.modifyWithVersioning().insert(Users, [userData]);
1777
+ await forgeSQL.modifyWithVersioning().updateById(updateData, Users);
1778
+ await forgeSQL.modifyWithVersioning().deleteById(1, Users);
1779
+ ```
1780
+
1781
+ #### 2. New API Methods
1782
+
1783
+ **New Methods Available:**
1784
+ - `forgeSQL.insert()` - Basic Drizzle operations
1785
+ - `forgeSQL.update()` - Basic Drizzle operations
1786
+ - `forgeSQL.delete()` - Basic Drizzle operations
1787
+ - `forgeSQL.insertAndEvictCache()` - Basic Drizzle operations with evict cache after execution
1788
+ - `forgeSQL.updateAndEvictCache()` - Basic Drizzle operations with evict cache after execution
1789
+ - `forgeSQL.deleteAndEvictCache()` - Basic Drizzle operations with evict cache after execution
1790
+
1791
+ **Optional Migration:**
1792
+ You can optionally migrate to the new API methods for better performance and cache management:
1793
+
1794
+ ```typescript
1795
+ // ❌ Old approach (still works)
1796
+ await forgeSQL.modifyWithVersioning().insert(Users, [userData]);
1797
+
1798
+ // ✅ New approach (recommended for new code)
1799
+ await forgeSQL.insert(Users).values(userData);
1800
+ // or for versioned operations with cache management
1801
+ await forgeSQL.modifyWithVersioningAndEvictCache().insert(Users, [userData]);
1802
+ ```
1803
+
1804
+ #### 3. Automatic Migration Script
1805
+
1806
+ You can use a simple find-and-replace to migrate your code:
1807
+
1808
+ ```bash
1809
+ # Replace modify() calls
1810
+ find . -name "*.ts" -o -name "*.js" | xargs sed -i 's/forgeSQL\.modify()/forgeSQL.modifyWithVersioning()/g'
1811
+
1812
+ # Replace crud() calls
1813
+ find . -name "*.ts" -o -name "*.js" | xargs sed -i 's/forgeSQL\.crud()/forgeSQL.modifyWithVersioning()/g'
1814
+ ```
1815
+
1816
+ #### 4. Breaking Changes
1817
+
1818
+ **Important:** The old methods (`modify()` and `crud()`) have been completely removed in version 2.1.x.
812
1819
 
1820
+ - ❌ **2.1.x**: Old methods are no longer available
1821
+ - ✅ **Migration Required**: You must update your code to use the new methods
813
1822
 
814
1823
  ## License
815
1824
  This project is licensed under the **MIT License**.