forge-sql-orm 2.1.24 → 2.1.25
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +45 -23
- package/dist/utils/cacheUtils.d.ts +13 -1
- package/dist/utils/cacheUtils.d.ts.map +1 -1
- package/dist/utils/cacheUtils.js +60 -47
- package/dist/utils/cacheUtils.js.map +1 -1
- package/dist/webtriggers/clearCacheSchedulerTrigger.d.ts +24 -4
- package/dist/webtriggers/clearCacheSchedulerTrigger.d.ts.map +1 -1
- package/dist/webtriggers/clearCacheSchedulerTrigger.js +24 -4
- package/dist/webtriggers/clearCacheSchedulerTrigger.js.map +1 -1
- package/package.json +9 -9
- package/src/utils/cacheUtils.ts +70 -47
- package/src/webtriggers/clearCacheSchedulerTrigger.ts +24 -4
package/README.md
CHANGED
|
@@ -708,10 +708,12 @@ The diagram below shows how Evict Cache works in Forge-SQL-ORM:
|
|
|
708
708
|
|
|
709
709
|
The diagram below shows how Scheduled Expiration Cleanup works:
|
|
710
710
|
|
|
711
|
+
**Note:** forge-sql-orm uses Forge KVS TTL feature (`{ ttl: { unit: "SECONDS", value: number } }`) to mark entries as expired. However, **actual deletion is asynchronous and may take up to 48 hours**. During this window, read operations may still return expired results. The scheduler trigger proactively cleans up expired entries to prevent cache growth from impacting INSERT/UPDATE performance.
|
|
712
|
+
|
|
711
713
|
1. A periodic scheduler (Forge trigger) runs cache cleanup independently of data modifications.
|
|
712
714
|
2. forge-sql-orm queries the cache entity by the expiration index to find entries with expiration < now.
|
|
713
715
|
3. Entries are deleted in batches (up to 25 per transaction) until the page is empty; pagination is done with a cursor (e.g., 100 per page).
|
|
714
|
-
4. This keeps the cache footprint small and prevents stale data accumulation.
|
|
716
|
+
4. This keeps the cache footprint small and prevents stale data accumulation, especially important when cache size impacts data modification performance.
|
|
715
717
|
|
|
716
718
|

|
|
717
719
|
|
|
@@ -734,6 +736,12 @@ The diagram below shows how Cache Context works:
|
|
|
734
736
|
**@forge/kvs Limits:**
|
|
735
737
|
Please review the [official @forge/kvs quotas and limits](https://developer.atlassian.com/platform/forge/platform-quotas-and-limits/#kvs-and-custom-entity-store-quotas) before implementing caching.
|
|
736
738
|
|
|
739
|
+
**TTL Limitations:**
|
|
740
|
+
|
|
741
|
+
- **Maximum TTL**: The maximum supported TTL is 1 year from the time the expiry is set.
|
|
742
|
+
- **Asynchronous deletion**: Expired data is not removed immediately upon expiry. Deletion may take up to 48 hours. During this window, read operations may still return expired results.
|
|
743
|
+
- **Performance impact**: If cache grows large, expired entries can impact INSERT/UPDATE performance. Use the Clear Cache Scheduler Trigger to proactively clean up expired entries.
|
|
744
|
+
|
|
737
745
|
**Caching Guidelines:**
|
|
738
746
|
|
|
739
747
|
- Don't cache everything - be selective about what to cache
|
|
@@ -741,6 +749,7 @@ Please review the [official @forge/kvs quotas and limits](https://developer.atla
|
|
|
741
749
|
- Consider data size and frequency of changes
|
|
742
750
|
- Monitor cache usage to stay within quotas
|
|
743
751
|
- Use appropriate TTL values
|
|
752
|
+
- If cache growth impacts performance, configure the Clear Cache Scheduler Trigger
|
|
744
753
|
|
|
745
754
|
**⚠️ Important Cache Limitations:**
|
|
746
755
|
|
|
@@ -754,10 +763,11 @@ npm install @forge/kvs -S
|
|
|
754
763
|
|
|
755
764
|
### Step 2: Configure Manifest
|
|
756
765
|
|
|
757
|
-
Add the storage entity configuration
|
|
766
|
+
Add the storage entity configuration to your `manifest.yml`. The `scheduledTrigger` is **optional** - only configure it if your cache grows large and impacts INSERT/UPDATE performance:
|
|
758
767
|
|
|
759
768
|
```yaml
|
|
760
769
|
modules:
|
|
770
|
+
# Optional: Only needed if cache growth impacts INSERT/UPDATE performance
|
|
761
771
|
scheduledTrigger:
|
|
762
772
|
- key: clear-cache-trigger
|
|
763
773
|
function: clearCache
|
|
@@ -814,7 +824,7 @@ const forgeSQL = new ForgeSQL(options);
|
|
|
814
824
|
- The `cacheEntityName` must exactly match the `name` in your manifest storage entities
|
|
815
825
|
- The entity attributes (`sql`, `expiration`, `data`) are required for proper cache functionality
|
|
816
826
|
- Indexes on `sql` and `expiration` improve cache lookup performance
|
|
817
|
-
- Cache data is
|
|
827
|
+
- Cache data uses Forge KVS TTL for expiration (deletion is asynchronous, may take up to 48 hours)
|
|
818
828
|
- No additional permissions are required beyond standard Forge app permissions
|
|
819
829
|
|
|
820
830
|
### Complete Setup Examples
|
|
@@ -862,6 +872,7 @@ npm install forge-sql-orm @forge/sql @forge/kvs drizzle-orm -S
|
|
|
862
872
|
|
|
863
873
|
```yaml
|
|
864
874
|
modules:
|
|
875
|
+
# Optional: Only needed if cache growth impacts INSERT/UPDATE performance
|
|
865
876
|
scheduledTrigger:
|
|
866
877
|
- key: clear-cache-trigger
|
|
867
878
|
function: clearCache
|
|
@@ -1681,7 +1692,7 @@ This multi-level approach provides optimal performance by checking the fastest c
|
|
|
1681
1692
|
|
|
1682
1693
|
### Cache Configuration
|
|
1683
1694
|
|
|
1684
|
-
The caching system uses Atlassian Forge's Custom entity store to persist cache data. Each cache entry is stored as a custom entity with
|
|
1695
|
+
The caching system uses Atlassian Forge's Custom entity store to persist cache data. Each cache entry is stored as a custom entity with TTL management via Forge KVS. Note that expired data deletion is asynchronous and may take up to 48 hours. If cache growth impacts INSERT/UPDATE performance, use the Clear Cache Scheduler Trigger for proactive cleanup.
|
|
1685
1696
|
|
|
1686
1697
|
```typescript
|
|
1687
1698
|
const options = {
|
|
@@ -1706,7 +1717,7 @@ const forgeSQL = new ForgeSQL(options);
|
|
|
1706
1717
|
The caching system leverages Forge's Custom entity store to provide:
|
|
1707
1718
|
|
|
1708
1719
|
- **Persistent Storage**: Cache data survives app restarts and deployments
|
|
1709
|
-
- **
|
|
1720
|
+
- **TTL Support**: Uses Forge KVS TTL feature for expiration (deletion is asynchronous, may take up to 48 hours)
|
|
1710
1721
|
- **Efficient Retrieval**: Fast key-based lookups using Forge's optimized storage
|
|
1711
1722
|
- **Data Serialization**: Automatic handling of complex objects and query results
|
|
1712
1723
|
- **Batch Operations**: Efficient bulk cache operations for better performance
|
|
@@ -1905,18 +1916,18 @@ const userResolver = async (req) => {
|
|
|
1905
1916
|
|
|
1906
1917
|
#### Local Cache (Level 1) vs Global Cache (Level 2)
|
|
1907
1918
|
|
|
1908
|
-
| Feature | Local Cache (Level 1) | Global Cache (Level 2)
|
|
1909
|
-
| ------------------ | ------------------------------------- |
|
|
1910
|
-
| **Storage** | In-memory (Node.js process) | Persistent (KVS Custom Entities)
|
|
1911
|
-
| **Scope** | Single forge invocation | Cross-invocation (between calls)
|
|
1912
|
-
| **Persistence** | No (cleared on invocation end) | Yes (survives app redeploy)
|
|
1913
|
-
| **Performance** | Very fast (memory access) | Fast (KVS optimized storage)
|
|
1914
|
-
| **Memory Usage** | Low (invocation-scoped) | Higher (persistent storage)
|
|
1915
|
-
| **Use Case** | Invocation optimization | Cross-invocation data sharing
|
|
1916
|
-
| **Configuration** | None required | Requires KVS setup
|
|
1917
|
-
| **TTL Support** | No (invocation-scoped) | Yes (
|
|
1918
|
-
| **Cache Eviction** | Automatic on DML operations | Manual or scheduled cleanup
|
|
1919
|
-
| **Best For** | Repeated queries in single invocation | Frequently accessed data across invocations
|
|
1919
|
+
| Feature | Local Cache (Level 1) | Global Cache (Level 2) |
|
|
1920
|
+
| ------------------ | ------------------------------------- | ------------------------------------------------------------------- |
|
|
1921
|
+
| **Storage** | In-memory (Node.js process) | Persistent (KVS Custom Entities) |
|
|
1922
|
+
| **Scope** | Single forge invocation | Cross-invocation (between calls) |
|
|
1923
|
+
| **Persistence** | No (cleared on invocation end) | Yes (survives app redeploy) |
|
|
1924
|
+
| **Performance** | Very fast (memory access) | Fast (KVS optimized storage) |
|
|
1925
|
+
| **Memory Usage** | Low (invocation-scoped) | Higher (persistent storage) |
|
|
1926
|
+
| **Use Case** | Invocation optimization | Cross-invocation data sharing |
|
|
1927
|
+
| **Configuration** | None required | Requires KVS setup |
|
|
1928
|
+
| **TTL Support** | No (invocation-scoped) | Yes (TTL via Forge KVS, async deletion up to 48h) |
|
|
1929
|
+
| **Cache Eviction** | Automatic on DML operations | Manual or scheduled cleanup (optional if cache impacts performance) |
|
|
1930
|
+
| **Best For** | Repeated queries in single invocation | Frequently accessed data across invocations |
|
|
1920
1931
|
|
|
1921
1932
|
#### Integration with Global Cache (Level 2)
|
|
1922
1933
|
|
|
@@ -2519,12 +2530,22 @@ SET foreign_key_checks = 1;
|
|
|
2519
2530
|
|
|
2520
2531
|
### 4. Clear Cache Scheduler Trigger
|
|
2521
2532
|
|
|
2522
|
-
This trigger automatically cleans up expired cache entries based on their TTL (Time To Live).
|
|
2533
|
+
This trigger automatically cleans up expired cache entries based on their TTL (Time To Live).
|
|
2534
|
+
|
|
2535
|
+
**⚠️ Important:** While Forge KVS uses TTL to mark entries as expired, **actual deletion is asynchronous and may take up to 48 hours**. During this window, read operations may still return expired results. If your cache grows large and impacts INSERT/UPDATE performance, you should use this scheduler trigger to proactively clean up expired entries.
|
|
2523
2536
|
|
|
2524
|
-
|
|
2525
|
-
|
|
2526
|
-
-
|
|
2527
|
-
-
|
|
2537
|
+
**When to use:**
|
|
2538
|
+
|
|
2539
|
+
- Your cache grows large over time
|
|
2540
|
+
- INSERT/UPDATE operations are slowing down due to cache size
|
|
2541
|
+
- You need strict expiry semantics (immediate cleanup)
|
|
2542
|
+
- You want to reduce storage costs proactively
|
|
2543
|
+
|
|
2544
|
+
**When optional:**
|
|
2545
|
+
|
|
2546
|
+
- Small cache footprint
|
|
2547
|
+
- No performance impact on data modifications
|
|
2548
|
+
- You can tolerate expired entries being returned for up to 48 hours
|
|
2528
2549
|
|
|
2529
2550
|
```typescript
|
|
2530
2551
|
// Example usage in your Forge app
|
|
@@ -2537,9 +2558,10 @@ export const clearCache = () => {
|
|
|
2537
2558
|
};
|
|
2538
2559
|
```
|
|
2539
2560
|
|
|
2540
|
-
Configure in `manifest.yml
|
|
2561
|
+
Configure in `manifest.yml` (optional - only if cache growth impacts INSERT/UPDATE performance):
|
|
2541
2562
|
|
|
2542
2563
|
```yaml
|
|
2564
|
+
# Optional: Only needed if cache growth impacts INSERT/UPDATE performance
|
|
2543
2565
|
scheduledTrigger:
|
|
2544
2566
|
- key: clear-cache-trigger
|
|
2545
2567
|
function: clearCache
|
|
@@ -25,6 +25,7 @@ export declare function clearCache<T extends AnyMySqlTable>(schema: T, options:
|
|
|
25
25
|
*/
|
|
26
26
|
export declare function clearTablesCache(tables: string[], options: ForgeSqlOrmOptions): Promise<void>;
|
|
27
27
|
/**
|
|
28
|
+
* since https://developer.atlassian.com/platform/forge/changelog/#CHANGE-3038
|
|
28
29
|
* Clears expired cache entries with retry logic and performance logging.
|
|
29
30
|
*
|
|
30
31
|
* @param options - ForgeSQL ORM options
|
|
@@ -34,6 +35,10 @@ export declare function clearExpiredCache(options: ForgeSqlOrmOptions): Promise<
|
|
|
34
35
|
/**
|
|
35
36
|
* Retrieves data from cache if it exists and is not expired.
|
|
36
37
|
*
|
|
38
|
+
* Note: Due to Forge KVS asynchronous deletion (up to 48 hours), expired entries may still
|
|
39
|
+
* be returned. This function checks the expiration timestamp to filter out expired entries.
|
|
40
|
+
* If cache growth impacts performance, use the Clear Cache Scheduler Trigger.
|
|
41
|
+
*
|
|
37
42
|
* @param query - Query object with toSQL method
|
|
38
43
|
* @param options - ForgeSQL ORM options
|
|
39
44
|
* @returns Cached data if found and valid, undefined otherwise
|
|
@@ -44,11 +49,18 @@ export declare function getFromCache<T>(query: {
|
|
|
44
49
|
/**
|
|
45
50
|
* Stores query results in cache with specified TTL.
|
|
46
51
|
*
|
|
52
|
+
* Uses Forge KVS TTL feature to set expiration. Note that expired data deletion is asynchronous:
|
|
53
|
+
* expired data is not removed immediately upon expiry. Deletion may take up to 48 hours.
|
|
54
|
+
* During this window, read operations may still return expired results. If your app requires
|
|
55
|
+
* strict expiry semantics, consider using the Clear Cache Scheduler Trigger to proactively
|
|
56
|
+
* clean up expired entries, especially if cache growth impacts INSERT/UPDATE performance.
|
|
57
|
+
*
|
|
47
58
|
* @param query - Query object with toSQL method
|
|
48
59
|
* @param options - ForgeSQL ORM options
|
|
49
60
|
* @param results - Data to cache
|
|
50
|
-
* @param cacheTtl - Time to live in seconds
|
|
61
|
+
* @param cacheTtl - Time to live in seconds (maximum TTL is 1 year from write time)
|
|
51
62
|
* @returns Promise that resolves when data is stored in cache
|
|
63
|
+
* @see https://developer.atlassian.com/platform/forge/runtime-reference/storage-api-basic-api/#ttl
|
|
52
64
|
*/
|
|
53
65
|
export declare function setCacheResult(query: {
|
|
54
66
|
toSQL: () => Query;
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cacheUtils.d.ts","sourceRoot":"","sources":["../../src/utils/cacheUtils.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,KAAK,EAAE,MAAM,aAAa,CAAC;AACpC,OAAO,EAAE,aAAa,EAAE,MAAM,wBAAwB,CAAC;AAGvD,OAAO,EAAE,kBAAkB,EAAE,MAAM,8BAA8B,CAAC;
|
|
1
|
+
{"version":3,"file":"cacheUtils.d.ts","sourceRoot":"","sources":["../../src/utils/cacheUtils.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,KAAK,EAAE,MAAM,aAAa,CAAC;AACpC,OAAO,EAAE,aAAa,EAAE,MAAM,wBAAwB,CAAC;AAGvD,OAAO,EAAE,kBAAkB,EAAE,MAAM,8BAA8B,CAAC;AAoElE;;;;;GAKG;AACH,wBAAgB,OAAO,CAAC,KAAK,EAAE,KAAK,GAAG,MAAM,CAK5C;AAiKD;;;;;;GAMG;AACH,wBAAsB,UAAU,CAAC,CAAC,SAAS,aAAa,EACtD,MAAM,EAAE,CAAC,EACT,OAAO,EAAE,kBAAkB,GAC1B,OAAO,CAAC,IAAI,CAAC,CAOf;AAED;;;;;;GAMG;AACH,wBAAsB,gBAAgB,CACpC,MAAM,EAAE,MAAM,EAAE,EAChB,OAAO,EAAE,kBAAkB,GAC1B,OAAO,CAAC,IAAI,CAAC,CAiBf;AACD;;;;;;GAMG;AACH,wBAAsB,iBAAiB,CAAC,OAAO,EAAE,kBAAkB,GAAG,OAAO,CAAC,IAAI,CAAC,CAiBlF;AAED;;;;;;;;;;GAUG;AACH,wBAAsB,YAAY,CAAC,CAAC,EAClC,KAAK,EAAE;IAAE,KAAK,EAAE,MAAM,KAAK,CAAA;CAAE,EAC7B,OAAO,EAAE,kBAAkB,GAC1B,OAAO,CAAC,CAAC,GAAG,SAAS,CAAC,CA2CxB;AAED;;;;;;;;;;;;;;;GAeG;AACH,wBAAsB,cAAc,CAClC,KAAK,EAAE;IAAE,KAAK,EAAE,MAAM,KAAK,CAAA;CAAE,EAC7B,OAAO,EAAE,kBAAkB,EAC3B,OAAO,EAAE,OAAO,EAChB,QAAQ,EAAE,MAAM,GACf,OAAO,CAAC,IAAI,CAAC,CAyCf"}
|
package/dist/utils/cacheUtils.js
CHANGED
|
@@ -77,6 +77,29 @@ function nowPlusSeconds(secondsToAdd) {
|
|
|
77
77
|
const dt = luxon_1.DateTime.now().plus({ seconds: secondsToAdd });
|
|
78
78
|
return Math.floor(dt.toSeconds());
|
|
79
79
|
}
|
|
80
|
+
/**
|
|
81
|
+
* Logs a message to console.debug when options.logCache is enabled.
|
|
82
|
+
*
|
|
83
|
+
* @param message - Message to log
|
|
84
|
+
* @param options - ForgeSQL ORM options (optional)
|
|
85
|
+
*/
|
|
86
|
+
function debugLog(message, options) {
|
|
87
|
+
if (options?.logCache) {
|
|
88
|
+
// eslint-disable-next-line no-console
|
|
89
|
+
console.debug(message);
|
|
90
|
+
}
|
|
91
|
+
} /**
|
|
92
|
+
* Logs a message to console.debug when options.logCache is enabled.
|
|
93
|
+
*
|
|
94
|
+
* @param message - Message to log
|
|
95
|
+
* @param options - ForgeSQL ORM options (optional)
|
|
96
|
+
*/
|
|
97
|
+
function warnLog(message, options) {
|
|
98
|
+
if (options?.logCache) {
|
|
99
|
+
// eslint-disable-next-line no-console
|
|
100
|
+
console.warn(message);
|
|
101
|
+
}
|
|
102
|
+
}
|
|
80
103
|
/**
|
|
81
104
|
* Generates a hash key for a query based on its SQL and parameters.
|
|
82
105
|
*
|
|
@@ -94,16 +117,14 @@ function hashKey(query) {
|
|
|
94
117
|
*
|
|
95
118
|
* @param results - Array of cache entries to delete
|
|
96
119
|
* @param cacheEntityName - Name of the cache entity
|
|
120
|
+
* @param options - Forge SQL ORM properties
|
|
97
121
|
* @returns Promise that resolves when all deletions are complete
|
|
98
122
|
*/
|
|
99
|
-
async function deleteCacheEntriesInBatches(results, cacheEntityName) {
|
|
123
|
+
async function deleteCacheEntriesInBatches(results, cacheEntityName, options) {
|
|
100
124
|
for (let i = 0; i < results.length; i += CACHE_CONSTANTS.BATCH_SIZE) {
|
|
101
125
|
const batch = results.slice(i, i + CACHE_CONSTANTS.BATCH_SIZE);
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
transactionBuilder = transactionBuilder.delete(result.key, { entityName: cacheEntityName });
|
|
105
|
-
}
|
|
106
|
-
await transactionBuilder.execute();
|
|
126
|
+
const batchResult = await kvs_1.kvs.batchDelete(batch.map((result) => ({ key: result.key, entityName: cacheEntityName })));
|
|
127
|
+
batchResult.failedKeys.forEach((failedKey) => warnLog(JSON.stringify(failedKey), options));
|
|
107
128
|
}
|
|
108
129
|
}
|
|
109
130
|
/**
|
|
@@ -134,11 +155,8 @@ async function clearCursorCache(tables, cursor, options) {
|
|
|
134
155
|
entityQueryBuilder = entityQueryBuilder.cursor(cursor);
|
|
135
156
|
}
|
|
136
157
|
const listResult = await entityQueryBuilder.limit(100).getMany();
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
console.warn(`clear cache Records: ${JSON.stringify(listResult.results.map((r) => r.key))}`);
|
|
140
|
-
}
|
|
141
|
-
await deleteCacheEntriesInBatches(listResult.results, cacheEntityName);
|
|
158
|
+
debugLog(`clear cache Records: ${JSON.stringify(listResult.results.map((r) => r.key))}`, options);
|
|
159
|
+
await deleteCacheEntriesInBatches(listResult.results, cacheEntityName, options);
|
|
142
160
|
if (listResult.nextCursor) {
|
|
143
161
|
return (listResult.results.length + (await clearCursorCache(tables, listResult.nextCursor, options)));
|
|
144
162
|
}
|
|
@@ -168,10 +186,7 @@ async function clearExpirationCursorCache(cursor, options) {
|
|
|
168
186
|
entityQueryBuilder = entityQueryBuilder.cursor(cursor);
|
|
169
187
|
}
|
|
170
188
|
const listResult = await entityQueryBuilder.limit(100).getMany();
|
|
171
|
-
|
|
172
|
-
// eslint-disable-next-line no-console
|
|
173
|
-
console.warn(`clear expired Records: ${JSON.stringify(listResult.results.map((r) => r.key))}`);
|
|
174
|
-
}
|
|
189
|
+
debugLog(`clear expired Records: ${JSON.stringify(listResult.results.map((r) => r.key))}`, options);
|
|
175
190
|
await deleteCacheEntriesInBatches(listResult.results, cacheEntityName);
|
|
176
191
|
if (listResult.nextCursor) {
|
|
177
192
|
return (listResult.results.length + (await clearExpirationCursorCache(listResult.nextCursor, options)));
|
|
@@ -243,14 +258,12 @@ async function clearTablesCache(tables, options) {
|
|
|
243
258
|
totalRecords = await executeWithRetry(() => clearCursorCache(tables, "", options), "clearing cache");
|
|
244
259
|
}
|
|
245
260
|
finally {
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
// eslint-disable-next-line no-console
|
|
249
|
-
console.info(`Cleared ${totalRecords} cache records in ${duration} seconds`);
|
|
250
|
-
}
|
|
261
|
+
const duration = luxon_1.DateTime.now().toSeconds() - startTime.toSeconds();
|
|
262
|
+
debugLog(`Cleared ${totalRecords} cache records in ${duration} seconds`, options);
|
|
251
263
|
}
|
|
252
264
|
}
|
|
253
265
|
/**
|
|
266
|
+
* since https://developer.atlassian.com/platform/forge/changelog/#CHANGE-3038
|
|
254
267
|
* Clears expired cache entries with retry logic and performance logging.
|
|
255
268
|
*
|
|
256
269
|
* @param options - ForgeSQL ORM options
|
|
@@ -267,15 +280,16 @@ async function clearExpiredCache(options) {
|
|
|
267
280
|
}
|
|
268
281
|
finally {
|
|
269
282
|
const duration = luxon_1.DateTime.now().toSeconds() - startTime.toSeconds();
|
|
270
|
-
|
|
271
|
-
// eslint-disable-next-line no-console
|
|
272
|
-
console.debug(`Cleared ${totalRecords} expired cache records in ${duration} seconds`);
|
|
273
|
-
}
|
|
283
|
+
debugLog(`Cleared ${totalRecords} expired cache records in ${duration} seconds`, options);
|
|
274
284
|
}
|
|
275
285
|
}
|
|
276
286
|
/**
|
|
277
287
|
* Retrieves data from cache if it exists and is not expired.
|
|
278
288
|
*
|
|
289
|
+
* Note: Due to Forge KVS asynchronous deletion (up to 48 hours), expired entries may still
|
|
290
|
+
* be returned. This function checks the expiration timestamp to filter out expired entries.
|
|
291
|
+
* If cache growth impacts performance, use the Clear Cache Scheduler Trigger.
|
|
292
|
+
*
|
|
279
293
|
* @param query - Query object with toSQL method
|
|
280
294
|
* @param options - ForgeSQL ORM options
|
|
281
295
|
* @returns Cached data if found and valid, undefined otherwise
|
|
@@ -291,23 +305,21 @@ async function getFromCache(query, options) {
|
|
|
291
305
|
const key = hashKey(sqlQuery);
|
|
292
306
|
// Skip cache if table is in cache context (will be cleared)
|
|
293
307
|
if (await (0, cacheContextUtils_1.isTableContainsTableInCacheContext)(sqlQuery.sql, options)) {
|
|
294
|
-
|
|
295
|
-
// eslint-disable-next-line no-console
|
|
296
|
-
console.warn(`Context contains value to clear. Skip getting from cache`);
|
|
297
|
-
}
|
|
308
|
+
debugLog("Context contains value to clear. Skip getting from cache", options);
|
|
298
309
|
return undefined;
|
|
299
310
|
}
|
|
300
311
|
try {
|
|
301
312
|
const cacheResult = await kvs_1.kvs.entity(options.cacheEntityName).get(key);
|
|
302
|
-
if (cacheResult
|
|
303
|
-
cacheResult[expirationName] >= getCurrentTime() &&
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
313
|
+
if (cacheResult) {
|
|
314
|
+
if (cacheResult[expirationName] >= getCurrentTime() &&
|
|
315
|
+
(0, cacheTableUtils_1.extractBacktickedValues)(sqlQuery.sql, options) === cacheResult[entityQueryName]) {
|
|
316
|
+
debugLog(`Get value from cache, cacheKey: ${key}`, options);
|
|
317
|
+
const results = cacheResult[dataName];
|
|
318
|
+
return JSON.parse(results);
|
|
319
|
+
}
|
|
320
|
+
else {
|
|
321
|
+
debugLog(`Expired cache entry still exists (will be automatically removed within 48 hours per Forge KVS TTL documentation), cacheKey: ${key}`, options);
|
|
308
322
|
}
|
|
309
|
-
const results = cacheResult[dataName];
|
|
310
|
-
return JSON.parse(results);
|
|
311
323
|
}
|
|
312
324
|
}
|
|
313
325
|
catch (error) {
|
|
@@ -319,11 +331,18 @@ async function getFromCache(query, options) {
|
|
|
319
331
|
/**
|
|
320
332
|
* Stores query results in cache with specified TTL.
|
|
321
333
|
*
|
|
334
|
+
* Uses Forge KVS TTL feature to set expiration. Note that expired data deletion is asynchronous:
|
|
335
|
+
* expired data is not removed immediately upon expiry. Deletion may take up to 48 hours.
|
|
336
|
+
* During this window, read operations may still return expired results. If your app requires
|
|
337
|
+
* strict expiry semantics, consider using the Clear Cache Scheduler Trigger to proactively
|
|
338
|
+
* clean up expired entries, especially if cache growth impacts INSERT/UPDATE performance.
|
|
339
|
+
*
|
|
322
340
|
* @param query - Query object with toSQL method
|
|
323
341
|
* @param options - ForgeSQL ORM options
|
|
324
342
|
* @param results - Data to cache
|
|
325
|
-
* @param cacheTtl - Time to live in seconds
|
|
343
|
+
* @param cacheTtl - Time to live in seconds (maximum TTL is 1 year from write time)
|
|
326
344
|
* @returns Promise that resolves when data is stored in cache
|
|
345
|
+
* @see https://developer.atlassian.com/platform/forge/runtime-reference/storage-api-basic-api/#ttl
|
|
327
346
|
*/
|
|
328
347
|
async function setCacheResult(query, options, results, cacheTtl) {
|
|
329
348
|
if (!options.cacheEntityName) {
|
|
@@ -336,10 +355,7 @@ async function setCacheResult(query, options, results, cacheTtl) {
|
|
|
336
355
|
const sqlQuery = query.toSQL();
|
|
337
356
|
// Skip cache if table is in cache context (will be cleared)
|
|
338
357
|
if (await (0, cacheContextUtils_1.isTableContainsTableInCacheContext)(sqlQuery.sql, options)) {
|
|
339
|
-
|
|
340
|
-
// eslint-disable-next-line no-console
|
|
341
|
-
console.warn(`Context contains value to clear. Skip setting from cache`);
|
|
342
|
-
}
|
|
358
|
+
debugLog("Context contains value to clear. Skip setting from cache", options);
|
|
343
359
|
return;
|
|
344
360
|
}
|
|
345
361
|
const key = hashKey(sqlQuery);
|
|
@@ -347,14 +363,11 @@ async function setCacheResult(query, options, results, cacheTtl) {
|
|
|
347
363
|
.transact()
|
|
348
364
|
.set(key, {
|
|
349
365
|
[entityQueryName]: (0, cacheTableUtils_1.extractBacktickedValues)(sqlQuery.sql, options),
|
|
350
|
-
[expirationName]: nowPlusSeconds(cacheTtl),
|
|
366
|
+
[expirationName]: nowPlusSeconds(cacheTtl + 2),
|
|
351
367
|
[dataName]: JSON.stringify(results),
|
|
352
|
-
}, { entityName: options.cacheEntityName })
|
|
368
|
+
}, { entityName: options.cacheEntityName }, { ttl: { value: cacheTtl, unit: "SECONDS" } })
|
|
353
369
|
.execute();
|
|
354
|
-
|
|
355
|
-
// eslint-disable-next-line no-console
|
|
356
|
-
console.warn(`Store value to cache, cacheKey: ${key}`);
|
|
357
|
-
}
|
|
370
|
+
debugLog(`Store value to cache, cacheKey: ${key}`, options);
|
|
358
371
|
}
|
|
359
372
|
catch (error) {
|
|
360
373
|
// eslint-disable-next-line no-console
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cacheUtils.js","sourceRoot":"","sources":["../../src/utils/cacheUtils.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
1
|
+
{"version":3,"file":"cacheUtils.js","sourceRoot":"","sources":["../../src/utils/cacheUtils.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAgFA,0BAKC;AAwKD,gCAUC;AASD,4CAoBC;AAQD,8CAiBC;AAaD,oCA8CC;AAkBD,wCA8CC;AAxbD,iCAAiC;AACjC,oDAAsC;AAGtC,6CAAiD;AACjD,oCAA4E;AAE5E,2DAAkG;AAClG,uDAA4D;AAE5D,uCAAuC;AACvC,MAAM,eAAe,GAAG;IACtB,UAAU,EAAE,EAAE;IACd,kBAAkB,EAAE,CAAC;IACrB,mBAAmB,EAAE,IAAI;IACzB,sBAAsB,EAAE,CAAC;IACzB,yBAAyB,EAAE,KAAK;IAChC,uBAAuB,EAAE,YAAY;IACrC,iBAAiB,EAAE,MAAM;IACzB,WAAW,EAAE,EAAE;CACP,CAAC;AAOX;;;;GAIG;AACH,SAAS,cAAc;IACrB,MAAM,EAAE,GAAG,gBAAQ,CAAC,GAAG,EAAE,CAAC;IAC1B,OAAO,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,CAAC;AACpC,CAAC;AAED;;;;;;;GAOG;AACH,SAAS,cAAc,CAAC,YAAoB;IAC1C,MAAM,EAAE,GAAG,gBAAQ,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,EAAE,OAAO,EAAE,YAAY,EAAE,CAAC,CAAC;IAC1D,OAAO,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,CAAC;AACpC,CAAC;AAED;;;;;GAKG;AACH,SAAS,QAAQ,CAAC,OAAe,EAAE,OAA4B;IAC7D,IAAI,OAAO,EAAE,QAAQ,EAAE,CAAC;QACtB,sCAAsC;QACtC,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;IACzB,CAAC;AACH,CAAC,CAAC;;;;;GAKC;AACH,SAAS,OAAO,CAAC,OAAe,EAAE,OAA4B;IAC5D,IAAI,OAAO,EAAE,QAAQ,EAAE,CAAC;QACtB,sCAAsC;QACtC,OAAO,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;IACxB,CAAC;AACH,CAAC;AAED;;;;;GAKG;AACH,SAAgB,OAAO,CAAC,KAAY;IAClC,MAAM,CAAC,GAAG,MAAM,CAAC,UAAU,CAAC,QAAQ,CAAC,CAAC;IACtC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,WAAW,EAAE,CAAC,CAAC;IAClC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC;IACvC,OAAO,cAAc,GAAG,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,eAAe,CAAC,WAAW,CAAC,CAAC;AAChF,CAAC;AAED;;;;;;;GAOG;AACH,KAAK,UAAU,2BAA2B,CACxC,OAA+B,EAC/B,eAAuB,EACvB,OAA4B;IAE5B,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,IAAI,eAAe,CAAC,UAAU,EAAE,CAAC;QACpE,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,GAAG,eAAe,CAAC,UAAU,CAAC,CAAC;QAC/D,MAAM,WAAW,GAAG,MAAM,SAAG,CAAC,WAAW,CACvC,KAAK,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,EAAE,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,UAAU,EAAE,eAAe,EAAE,CAAC,CAAC,CAC1E,CAAC;QACF,WAAW,CAAC,UAAU,CAAC,OAAO,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,SAAS,CAAC,SAAS,CAAC,EAAE,OAAO,CAAC,CAAC,CAAC;IAC7F,CAAC;AACH,CAAC;AAED;;;;;;;GAOG;AACH,KAAK,UAAU,gBAAgB,CAC7B,MAAgB,EAChB,MAAc,EACd,OAA2B;IAE3B,MAAM,eAAe,GAAG,OAAO,CAAC,eAAe,CAAC;IAChD,IAAI,CAAC,eAAe,EAAE,CAAC;QACrB,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;IACvD,CAAC;IAED,MAAM,eAAe,GAAG,OAAO,CAAC,oBAAoB,IAAI,eAAe,CAAC,yBAAyB,CAAC;IAClG,IAAI,OAAO,GAAG,IAAI,YAAM,EAEpB,CAAC;IAEL,KAAK,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;QAC3B,MAAM,YAAY,GAAG,OAAO,CAAC,cAAc,CAAC,CAAC,CAAC,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,KAAK,CAAC;QACrE,OAAO,CAAC,EAAE,CAAC,eAAe,EAAE,sBAAgB,CAAC,QAAQ,CAAC,YAAY,EAAE,WAAW,EAAE,CAAC,CAAC,CAAC;IACtF,CAAC;IAED,IAAI,kBAAkB,GAAG,SAAG;SACzB,MAAM,CAEJ,eAAe,CAAC;SAClB,KAAK,EAAE;SACP,KAAK,CAAC,eAAe,CAAC;SACtB,OAAO,CAAC,OAAO,CAAC,CAAC;IAEpB,IAAI,MAAM,EAAE,CAAC;QACX,kBAAkB,GAAG,kBAAkB,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;IACzD,CAAC;IAED,MAAM,UAAU,GAAG,MAAM,kBAAkB,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,CAAC;IAEjE,QAAQ,CAAC,wBAAwB,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC;IAElG,MAAM,2BAA2B,CAAC,UAAU,CAAC,OAAO,EAAE,eAAe,EAAE,OAAO,CAAC,CAAC;IAEhF,IAAI,UAAU,CAAC,UAAU,EAAE,CAAC;QAC1B,OAAO,CACL,UAAU,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,MAAM,gBAAgB,CAAC,MAAM,EAAE,UAAU,CAAC,UAAU,EAAE,OAAO,CAAC,CAAC,CAC7F,CAAC;IACJ,CAAC;SAAM,CAAC;QACN,OAAO,UAAU,CAAC,OAAO,CAAC,MAAM,CAAC;IACnC,CAAC;AACH,CAAC;AAED;;;;;;GAMG;AACH,KAAK,UAAU,0BAA0B,CACvC,MAAc,EACd,OAA2B;IAE3B,MAAM,eAAe,GAAG,OAAO,CAAC,eAAe,CAAC;IAChD,IAAI,CAAC,eAAe,EAAE,CAAC;QACrB,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;IACvD,CAAC;IAED,MAAM,oBAAoB,GACxB,OAAO,CAAC,yBAAyB,IAAI,eAAe,CAAC,uBAAuB,CAAC;IAC/E,IAAI,kBAAkB,GAAG,SAAG;SACzB,MAAM,CAEJ,eAAe,CAAC;SAClB,KAAK,EAAE;SACP,KAAK,CAAC,oBAAoB,CAAC;SAC3B,KAAK,CAAC,qBAAe,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,gBAAQ,CAAC,GAAG,EAAE,CAAC,SAAS,EAAE,CAAC,CAAC,CAAC,CAAC;IAE3E,IAAI,MAAM,EAAE,CAAC;QACX,kBAAkB,GAAG,kBAAkB,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;IACzD,CAAC;IAED,MAAM,UAAU,GAAG,MAAM,kBAAkB,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,CAAC;IAEjE,QAAQ,CACN,0BAA0B,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,EAChF,OAAO,CACR,CAAC;IAEF,MAAM,2BAA2B,CAAC,UAAU,CAAC,OAAO,EAAE,eAAe,CAAC,CAAC;IAEvE,IAAI,UAAU,CAAC,UAAU,EAAE,CAAC;QAC1B,OAAO,CACL,UAAU,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,MAAM,0BAA0B,CAAC,UAAU,CAAC,UAAU,EAAE,OAAO,CAAC,CAAC,CAC/F,CAAC;IACJ,CAAC;SAAM,CAAC;QACN,OAAO,UAAU,CAAC,OAAO,CAAC,MAAM,CAAC;IACnC,CAAC;AACH,CAAC;AAED;;;;;;;GAOG;AACH,KAAK,UAAU,gBAAgB,CAAI,SAA2B,EAAE,aAAqB;IACnF,IAAI,OAAO,GAAG,CAAC,CAAC;IAChB,IAAI,KAAK,GAAG,eAAe,CAAC,mBAAmB,CAAC;IAEhD,OAAO,OAAO,GAAG,eAAe,CAAC,kBAAkB,EAAE,CAAC;QACpD,IAAI,CAAC;YACH,OAAO,MAAM,SAAS,EAAE,CAAC;QAC3B,CAAC;QAAC,OAAO,GAAQ,EAAE,CAAC;YAClB,sCAAsC;YACtC,OAAO,CAAC,IAAI,CAAC,gBAAgB,aAAa,KAAK,GAAG,CAAC,OAAO,WAAW,OAAO,EAAE,EAAE,GAAG,CAAC,CAAC;YACrF,OAAO,EAAE,CAAC;YAEV,IAAI,OAAO,IAAI,eAAe,CAAC,kBAAkB,EAAE,CAAC;gBAClD,sCAAsC;gBACtC,OAAO,CAAC,KAAK,CAAC,gBAAgB,aAAa,KAAK,GAAG,CAAC,OAAO,EAAE,EAAE,GAAG,CAAC,CAAC;gBACpE,MAAM,GAAG,CAAC;YACZ,CAAC;YAED,MAAM,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,UAAU,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC,CAAC;YAC3D,KAAK,IAAI,eAAe,CAAC,sBAAsB,CAAC;QAClD,CAAC;IACH,CAAC;IAED,MAAM,IAAI,KAAK,CAAC,uCAAuC,aAAa,EAAE,CAAC,CAAC;AAC1E,CAAC;AAED;;;;;;GAMG;AACI,KAAK,UAAU,UAAU,CAC9B,MAAS,EACT,OAA2B;IAE3B,MAAM,SAAS,GAAG,IAAA,oBAAY,EAAC,MAAM,CAAC,CAAC;IACvC,IAAI,2CAAuB,CAAC,QAAQ,EAAE,EAAE,CAAC;QACvC,2CAAuB,CAAC,QAAQ,EAAE,EAAE,MAAM,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC;IAC5D,CAAC;SAAM,CAAC;QACN,MAAM,gBAAgB,CAAC,CAAC,SAAS,CAAC,EAAE,OAAO,CAAC,CAAC;IAC/C,CAAC;AACH,CAAC;AAED;;;;;;GAMG;AACI,KAAK,UAAU,gBAAgB,CACpC,MAAgB,EAChB,OAA2B;IAE3B,IAAI,CAAC,OAAO,CAAC,eAAe,EAAE,CAAC;QAC7B,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;IACvD,CAAC;IAED,MAAM,SAAS,GAAG,gBAAQ,CAAC,GAAG,EAAE,CAAC;IACjC,IAAI,YAAY,GAAG,CAAC,CAAC;IAErB,IAAI,CAAC;QACH,YAAY,GAAG,MAAM,gBAAgB,CACnC,GAAG,EAAE,CAAC,gBAAgB,CAAC,MAAM,EAAE,EAAE,EAAE,OAAO,CAAC,EAC3C,gBAAgB,CACjB,CAAC;IACJ,CAAC;YAAS,CAAC;QACT,MAAM,QAAQ,GAAG,gBAAQ,CAAC,GAAG,EAAE,CAAC,SAAS,EAAE,GAAG,SAAS,CAAC,SAAS,EAAE,CAAC;QACpE,QAAQ,CAAC,WAAW,YAAY,qBAAqB,QAAQ,UAAU,EAAE,OAAO,CAAC,CAAC;IACpF,CAAC;AACH,CAAC;AACD;;;;;;GAMG;AACI,KAAK,UAAU,iBAAiB,CAAC,OAA2B;IACjE,IAAI,CAAC,OAAO,CAAC,eAAe,EAAE,CAAC;QAC7B,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;IACvD,CAAC;IAED,MAAM,SAAS,GAAG,gBAAQ,CAAC,GAAG,EAAE,CAAC;IACjC,IAAI,YAAY,GAAG,CAAC,CAAC;IAErB,IAAI,CAAC;QACH,YAAY,GAAG,MAAM,gBAAgB,CACnC,GAAG,EAAE,CAAC,0BAA0B,CAAC,EAAE,EAAE,OAAO,CAAC,EAC7C,wBAAwB,CACzB,CAAC;IACJ,CAAC;YAAS,CAAC;QACT,MAAM,QAAQ,GAAG,gBAAQ,CAAC,GAAG,EAAE,CAAC,SAAS,EAAE,GAAG,SAAS,CAAC,SAAS,EAAE,CAAC;QACpE,QAAQ,CAAC,WAAW,YAAY,6BAA6B,QAAQ,UAAU,EAAE,OAAO,CAAC,CAAC;IAC5F,CAAC;AACH,CAAC;AAED;;;;;;;;;;GAUG;AACI,KAAK,UAAU,YAAY,CAChC,KAA6B,EAC7B,OAA2B;IAE3B,IAAI,CAAC,OAAO,CAAC,eAAe,EAAE,CAAC;QAC7B,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;IACvD,CAAC;IAED,MAAM,eAAe,GAAG,OAAO,CAAC,oBAAoB,IAAI,eAAe,CAAC,yBAAyB,CAAC;IAClG,MAAM,cAAc,GAClB,OAAO,CAAC,yBAAyB,IAAI,eAAe,CAAC,uBAAuB,CAAC;IAC/E,MAAM,QAAQ,GAAG,OAAO,CAAC,mBAAmB,IAAI,eAAe,CAAC,iBAAiB,CAAC;IAElF,MAAM,QAAQ,GAAG,KAAK,CAAC,KAAK,EAAE,CAAC;IAC/B,MAAM,GAAG,GAAG,OAAO,CAAC,QAAQ,CAAC,CAAC;IAE9B,4DAA4D;IAC5D,IAAI,MAAM,IAAA,sDAAkC,EAAC,QAAQ,CAAC,GAAG,EAAE,OAAO,CAAC,EAAE,CAAC;QACpE,QAAQ,CAAC,0DAA0D,EAAE,OAAO,CAAC,CAAC;QAC9E,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,CAAC;QACH,MAAM,WAAW,GAAG,MAAM,SAAG,CAAC,MAAM,CAAc,OAAO,CAAC,eAAe,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QAEpF,IAAI,WAAW,EAAE,CAAC;YAChB,IACG,WAAW,CAAC,cAAc,CAAY,IAAI,cAAc,EAAE;gBAC3D,IAAA,yCAAuB,EAAC,QAAQ,CAAC,GAAG,EAAE,OAAO,CAAC,KAAK,WAAW,CAAC,eAAe,CAAC,EAC/E,CAAC;gBACD,QAAQ,CAAC,mCAAmC,GAAG,EAAE,EAAE,OAAO,CAAC,CAAC;gBAC5D,MAAM,OAAO,GAAG,WAAW,CAAC,QAAQ,CAAC,CAAC;gBACtC,OAAO,IAAI,CAAC,KAAK,CAAC,OAAiB,CAAC,CAAC;YACvC,CAAC;iBAAM,CAAC;gBACN,QAAQ,CACN,+HAA+H,GAAG,EAAE,EACpI,OAAO,CACR,CAAC;YACJ,CAAC;QACH,CAAC;IACH,CAAC;IAAC,OAAO,KAAU,EAAE,CAAC;QACpB,sCAAsC;QACtC,OAAO,CAAC,KAAK,CAAC,6BAA6B,KAAK,CAAC,OAAO,EAAE,EAAE,KAAK,CAAC,CAAC;IACrE,CAAC;IAED,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;;;;;;;;;;;;;GAeG;AACI,KAAK,UAAU,cAAc,CAClC,KAA6B,EAC7B,OAA2B,EAC3B,OAAgB,EAChB,QAAgB;IAEhB,IAAI,CAAC,OAAO,CAAC,eAAe,EAAE,CAAC;QAC7B,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;IACvD,CAAC;IAED,IAAI,CAAC;QACH,MAAM,eAAe,GACnB,OAAO,CAAC,oBAAoB,IAAI,eAAe,CAAC,yBAAyB,CAAC;QAC5E,MAAM,cAAc,GAClB,OAAO,CAAC,yBAAyB,IAAI,eAAe,CAAC,uBAAuB,CAAC;QAC/E,MAAM,QAAQ,GAAG,OAAO,CAAC,mBAAmB,IAAI,eAAe,CAAC,iBAAiB,CAAC;QAElF,MAAM,QAAQ,GAAG,KAAK,CAAC,KAAK,EAAE,CAAC;QAE/B,4DAA4D;QAC5D,IAAI,MAAM,IAAA,sDAAkC,EAAC,QAAQ,CAAC,GAAG,EAAE,OAAO,CAAC,EAAE,CAAC;YACpE,QAAQ,CAAC,0DAA0D,EAAE,OAAO,CAAC,CAAC;YAC9E,OAAO;QACT,CAAC;QAED,MAAM,GAAG,GAAG,OAAO,CAAC,QAAQ,CAAC,CAAC;QAE9B,MAAM,SAAG;aACN,QAAQ,EAAE;aACV,GAAG,CACF,GAAG,EACH;YACE,CAAC,eAAe,CAAC,EAAE,IAAA,yCAAuB,EAAC,QAAQ,CAAC,GAAG,EAAE,OAAO,CAAC;YACjE,CAAC,cAAc,CAAC,EAAE,cAAc,CAAC,QAAQ,GAAG,CAAC,CAAC;YAC9C,CAAC,QAAQ,CAAC,EAAE,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC;SACpC,EACD,EAAE,UAAU,EAAE,OAAO,CAAC,eAAe,EAAE,EACvC,EAAE,GAAG,EAAE,EAAE,KAAK,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,EAAE,CAC9C;aACA,OAAO,EAAE,CAAC;QAEb,QAAQ,CAAC,mCAAmC,GAAG,EAAE,EAAE,OAAO,CAAC,CAAC;IAC9D,CAAC;IAAC,OAAO,KAAU,EAAE,CAAC;QACpB,sCAAsC;QACtC,OAAO,CAAC,KAAK,CAAC,wBAAwB,KAAK,CAAC,OAAO,EAAE,EAAE,KAAK,CAAC,CAAC;IAChE,CAAC;AACH,CAAC"}
|
|
@@ -1,9 +1,27 @@
|
|
|
1
1
|
import { ForgeSqlOrmOptions } from "../core/ForgeSQLQueryBuilder";
|
|
2
2
|
/**
|
|
3
|
-
* Scheduler trigger for clearing expired cache entries.
|
|
3
|
+
* Scheduler trigger for proactively clearing expired cache entries.
|
|
4
4
|
*
|
|
5
|
-
*
|
|
6
|
-
*
|
|
5
|
+
* **Why this trigger is needed:**
|
|
6
|
+
*
|
|
7
|
+
* While forge-sql-orm uses Forge KVS TTL feature to mark entries as expired, **actual deletion
|
|
8
|
+
* is asynchronous and may take up to 48 hours**. During this window, expired entries remain in
|
|
9
|
+
* storage and can impact INSERT/UPDATE performance if the cache grows large.
|
|
10
|
+
*
|
|
11
|
+
* This scheduler trigger proactively cleans up expired entries by querying the expiration index
|
|
12
|
+
* and deleting entries where expiration < now, preventing cache growth from impacting data
|
|
13
|
+
* modification operations.
|
|
14
|
+
*
|
|
15
|
+
* **When to use:**
|
|
16
|
+
* - Your cache grows large over time
|
|
17
|
+
* - INSERT/UPDATE operations are slowing down due to cache size
|
|
18
|
+
* - You need strict expiry semantics (immediate cleanup)
|
|
19
|
+
* - You want to reduce storage costs proactively
|
|
20
|
+
*
|
|
21
|
+
* **When optional:**
|
|
22
|
+
* - Small cache footprint
|
|
23
|
+
* - No performance impact on data modifications
|
|
24
|
+
* - You can tolerate expired entries being returned for up to 48 hours
|
|
7
25
|
*
|
|
8
26
|
* @note This function is automatically disabled in production environments and will return a 500 error if called.
|
|
9
27
|
*
|
|
@@ -26,7 +44,7 @@ import { ForgeSqlOrmOptions } from "../core/ForgeSQLQueryBuilder";
|
|
|
26
44
|
*
|
|
27
45
|
* @example
|
|
28
46
|
* ```yaml
|
|
29
|
-
* # In manifest.yml
|
|
47
|
+
* # In manifest.yml (optional - only if cache growth impacts INSERT/UPDATE performance)
|
|
30
48
|
* scheduledTrigger:
|
|
31
49
|
* - key: clear-cache-trigger
|
|
32
50
|
* function: clearCache
|
|
@@ -36,6 +54,8 @@ import { ForgeSqlOrmOptions } from "../core/ForgeSQLQueryBuilder";
|
|
|
36
54
|
* - key: clearCache
|
|
37
55
|
* handler: index.clearCache
|
|
38
56
|
* ```
|
|
57
|
+
*
|
|
58
|
+
* @see https://developer.atlassian.com/platform/forge/runtime-reference/storage-api-basic-api/#ttl
|
|
39
59
|
*/
|
|
40
60
|
export declare const clearCacheSchedulerTrigger: (options?: ForgeSqlOrmOptions) => Promise<{
|
|
41
61
|
headers: {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"clearCacheSchedulerTrigger.d.ts","sourceRoot":"","sources":["../../src/webtriggers/clearCacheSchedulerTrigger.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,kBAAkB,EAAE,MAAM,8BAA8B,CAAC;AAElE
|
|
1
|
+
{"version":3,"file":"clearCacheSchedulerTrigger.d.ts","sourceRoot":"","sources":["../../src/webtriggers/clearCacheSchedulerTrigger.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,kBAAkB,EAAE,MAAM,8BAA8B,CAAC;AAElE;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAyDG;AACH,eAAO,MAAM,0BAA0B,GAAU,UAAU,kBAAkB;;;;;;;EAwC5E,CAAC"}
|
|
@@ -3,10 +3,28 @@ Object.defineProperty(exports, "__esModule", { value: true });
|
|
|
3
3
|
exports.clearCacheSchedulerTrigger = void 0;
|
|
4
4
|
const cacheUtils_1 = require("../utils/cacheUtils");
|
|
5
5
|
/**
|
|
6
|
-
* Scheduler trigger for clearing expired cache entries.
|
|
6
|
+
* Scheduler trigger for proactively clearing expired cache entries.
|
|
7
7
|
*
|
|
8
|
-
*
|
|
9
|
-
*
|
|
8
|
+
* **Why this trigger is needed:**
|
|
9
|
+
*
|
|
10
|
+
* While forge-sql-orm uses Forge KVS TTL feature to mark entries as expired, **actual deletion
|
|
11
|
+
* is asynchronous and may take up to 48 hours**. During this window, expired entries remain in
|
|
12
|
+
* storage and can impact INSERT/UPDATE performance if the cache grows large.
|
|
13
|
+
*
|
|
14
|
+
* This scheduler trigger proactively cleans up expired entries by querying the expiration index
|
|
15
|
+
* and deleting entries where expiration < now, preventing cache growth from impacting data
|
|
16
|
+
* modification operations.
|
|
17
|
+
*
|
|
18
|
+
* **When to use:**
|
|
19
|
+
* - Your cache grows large over time
|
|
20
|
+
* - INSERT/UPDATE operations are slowing down due to cache size
|
|
21
|
+
* - You need strict expiry semantics (immediate cleanup)
|
|
22
|
+
* - You want to reduce storage costs proactively
|
|
23
|
+
*
|
|
24
|
+
* **When optional:**
|
|
25
|
+
* - Small cache footprint
|
|
26
|
+
* - No performance impact on data modifications
|
|
27
|
+
* - You can tolerate expired entries being returned for up to 48 hours
|
|
10
28
|
*
|
|
11
29
|
* @note This function is automatically disabled in production environments and will return a 500 error if called.
|
|
12
30
|
*
|
|
@@ -29,7 +47,7 @@ const cacheUtils_1 = require("../utils/cacheUtils");
|
|
|
29
47
|
*
|
|
30
48
|
* @example
|
|
31
49
|
* ```yaml
|
|
32
|
-
* # In manifest.yml
|
|
50
|
+
* # In manifest.yml (optional - only if cache growth impacts INSERT/UPDATE performance)
|
|
33
51
|
* scheduledTrigger:
|
|
34
52
|
* - key: clear-cache-trigger
|
|
35
53
|
* function: clearCache
|
|
@@ -39,6 +57,8 @@ const cacheUtils_1 = require("../utils/cacheUtils");
|
|
|
39
57
|
* - key: clearCache
|
|
40
58
|
* handler: index.clearCache
|
|
41
59
|
* ```
|
|
60
|
+
*
|
|
61
|
+
* @see https://developer.atlassian.com/platform/forge/runtime-reference/storage-api-basic-api/#ttl
|
|
42
62
|
*/
|
|
43
63
|
const clearCacheSchedulerTrigger = async (options) => {
|
|
44
64
|
try {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"clearCacheSchedulerTrigger.js","sourceRoot":"","sources":["../../src/webtriggers/clearCacheSchedulerTrigger.ts"],"names":[],"mappings":";;;AAAA,oDAAwD;AAGxD
|
|
1
|
+
{"version":3,"file":"clearCacheSchedulerTrigger.js","sourceRoot":"","sources":["../../src/webtriggers/clearCacheSchedulerTrigger.ts"],"names":[],"mappings":";;;AAAA,oDAAwD;AAGxD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAyDG;AACI,MAAM,0BAA0B,GAAG,KAAK,EAAE,OAA4B,EAAE,EAAE;IAC/E,IAAI,CAAC;QACH,MAAM,UAAU,GAAuB,OAAO,IAAI;YAChD,cAAc,EAAE,KAAK;YACrB,wBAAwB,EAAE,KAAK;YAC/B,QAAQ,EAAE,GAAG;YACb,eAAe,EAAE,OAAO;YACxB,oBAAoB,EAAE,KAAK;YAC3B,yBAAyB,EAAE,YAAY;YACvC,mBAAmB,EAAE,MAAM;SAC5B,CAAC;QACF,IAAI,CAAC,UAAU,CAAC,eAAe,EAAE,CAAC;YAChC,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;QACvD,CAAC;QACD,MAAM,IAAA,8BAAiB,EAAC,UAAU,CAAC,CAAC;QAEpC,OAAO;YACL,OAAO,EAAE,EAAE,cAAc,EAAE,CAAC,kBAAkB,CAAC,EAAE;YACjD,UAAU,EAAE,GAAG;YACf,UAAU,EAAE,IAAI;YAChB,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;gBACnB,OAAO,EAAE,IAAI;gBACb,OAAO,EAAE,sCAAsC;gBAC/C,SAAS,EAAE,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE;aACpC,CAAC;SACH,CAAC;IACJ,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,sCAAsC;QACtC,OAAO,CAAC,KAAK,CAAC,8BAA8B,EAAE,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC,CAAC;QACrE,OAAO;YACL,OAAO,EAAE,EAAE,cAAc,EAAE,CAAC,kBAAkB,CAAC,EAAE;YACjD,UAAU,EAAE,GAAG;YACf,UAAU,EAAE,uBAAuB;YACnC,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;gBACnB,OAAO,EAAE,KAAK;gBACd,KAAK,EAAE,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,oCAAoC;gBACpF,SAAS,EAAE,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE;aACpC,CAAC;SACH,CAAC;IACJ,CAAC;AACH,CAAC,CAAC;AAxCW,QAAA,0BAA0B,8BAwCrC"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "forge-sql-orm",
|
|
3
|
-
"version": "2.1.
|
|
3
|
+
"version": "2.1.25",
|
|
4
4
|
"description": "Drizzle ORM integration for Atlassian @forge/sql. Provides a custom driver, schema migration, two levels of caching (local and global via @forge/kvs), optimistic locking, and query analysis.",
|
|
5
5
|
"main": "dist/index.js",
|
|
6
6
|
"homepage": "https://github.com/forge-sql-orm/forge-sql-orm#readme",
|
|
@@ -27,9 +27,9 @@
|
|
|
27
27
|
"devDependencies": {
|
|
28
28
|
"@eslint/js": "^9.39.2",
|
|
29
29
|
"@types/luxon": "^3.7.1",
|
|
30
|
-
"@types/node": "^25.
|
|
31
|
-
"@typescript-eslint/eslint-plugin": "^8.
|
|
32
|
-
"@typescript-eslint/parser": "^8.
|
|
30
|
+
"@types/node": "^25.4.0",
|
|
31
|
+
"@typescript-eslint/eslint-plugin": "^8.57.0",
|
|
32
|
+
"@typescript-eslint/parser": "^8.57.0",
|
|
33
33
|
"@vitest/coverage-v8": "^4.0.18",
|
|
34
34
|
"@vitest/ui": "^4.0.18",
|
|
35
35
|
"eslint": "^9.39.2",
|
|
@@ -37,7 +37,7 @@
|
|
|
37
37
|
"eslint-plugin-import": "^2.32.0",
|
|
38
38
|
"eslint-plugin-vitest": "^0.5.4",
|
|
39
39
|
"husky": "^9.1.7",
|
|
40
|
-
"knip": "^5.
|
|
40
|
+
"knip": "^5.86.0",
|
|
41
41
|
"patch-package": "^8.0.1",
|
|
42
42
|
"prettier": "^3.8.1",
|
|
43
43
|
"ts-node": "^10.9.2",
|
|
@@ -71,15 +71,15 @@
|
|
|
71
71
|
"README.md"
|
|
72
72
|
],
|
|
73
73
|
"peerDependencies": {
|
|
74
|
-
"@forge/sql": "^3.0.
|
|
74
|
+
"@forge/sql": "^3.0.19",
|
|
75
75
|
"drizzle-orm": "^0.45.1"
|
|
76
76
|
},
|
|
77
77
|
"optionalDependencies": {
|
|
78
|
-
"@forge/kvs": "^1.
|
|
78
|
+
"@forge/kvs": "^1.4.0"
|
|
79
79
|
},
|
|
80
80
|
"dependencies": {
|
|
81
|
-
"@forge/api": "^7.0
|
|
82
|
-
"@forge/events": "^2.0
|
|
81
|
+
"@forge/api": "^7.1.0",
|
|
82
|
+
"@forge/events": "^2.1.0",
|
|
83
83
|
"luxon": "^3.7.2",
|
|
84
84
|
"node-sql-parser": "^5.4.0"
|
|
85
85
|
},
|
package/src/utils/cacheUtils.ts
CHANGED
|
@@ -48,6 +48,30 @@ function nowPlusSeconds(secondsToAdd: number): number {
|
|
|
48
48
|
return Math.floor(dt.toSeconds());
|
|
49
49
|
}
|
|
50
50
|
|
|
51
|
+
/**
|
|
52
|
+
* Logs a message to console.debug when options.logCache is enabled.
|
|
53
|
+
*
|
|
54
|
+
* @param message - Message to log
|
|
55
|
+
* @param options - ForgeSQL ORM options (optional)
|
|
56
|
+
*/
|
|
57
|
+
function debugLog(message: string, options?: ForgeSqlOrmOptions): void {
|
|
58
|
+
if (options?.logCache) {
|
|
59
|
+
// eslint-disable-next-line no-console
|
|
60
|
+
console.debug(message);
|
|
61
|
+
}
|
|
62
|
+
} /**
|
|
63
|
+
* Logs a message to console.debug when options.logCache is enabled.
|
|
64
|
+
*
|
|
65
|
+
* @param message - Message to log
|
|
66
|
+
* @param options - ForgeSQL ORM options (optional)
|
|
67
|
+
*/
|
|
68
|
+
function warnLog(message: string, options?: ForgeSqlOrmOptions): void {
|
|
69
|
+
if (options?.logCache) {
|
|
70
|
+
// eslint-disable-next-line no-console
|
|
71
|
+
console.warn(message);
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
|
|
51
75
|
/**
|
|
52
76
|
* Generates a hash key for a query based on its SQL and parameters.
|
|
53
77
|
*
|
|
@@ -66,19 +90,20 @@ export function hashKey(query: Query): string {
|
|
|
66
90
|
*
|
|
67
91
|
* @param results - Array of cache entries to delete
|
|
68
92
|
* @param cacheEntityName - Name of the cache entity
|
|
93
|
+
* @param options - Forge SQL ORM properties
|
|
69
94
|
* @returns Promise that resolves when all deletions are complete
|
|
70
95
|
*/
|
|
71
96
|
async function deleteCacheEntriesInBatches(
|
|
72
97
|
results: Array<{ key: string }>,
|
|
73
98
|
cacheEntityName: string,
|
|
99
|
+
options?: ForgeSqlOrmOptions,
|
|
74
100
|
): Promise<void> {
|
|
75
101
|
for (let i = 0; i < results.length; i += CACHE_CONSTANTS.BATCH_SIZE) {
|
|
76
102
|
const batch = results.slice(i, i + CACHE_CONSTANTS.BATCH_SIZE);
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
await transactionBuilder.execute();
|
|
103
|
+
const batchResult = await kvs.batchDelete(
|
|
104
|
+
batch.map((result) => ({ key: result.key, entityName: cacheEntityName })),
|
|
105
|
+
);
|
|
106
|
+
batchResult.failedKeys.forEach((failedKey) => warnLog(JSON.stringify(failedKey), options));
|
|
82
107
|
}
|
|
83
108
|
}
|
|
84
109
|
|
|
@@ -124,12 +149,9 @@ async function clearCursorCache(
|
|
|
124
149
|
|
|
125
150
|
const listResult = await entityQueryBuilder.limit(100).getMany();
|
|
126
151
|
|
|
127
|
-
|
|
128
|
-
// eslint-disable-next-line no-console
|
|
129
|
-
console.warn(`clear cache Records: ${JSON.stringify(listResult.results.map((r) => r.key))}`);
|
|
130
|
-
}
|
|
152
|
+
debugLog(`clear cache Records: ${JSON.stringify(listResult.results.map((r) => r.key))}`, options);
|
|
131
153
|
|
|
132
|
-
await deleteCacheEntriesInBatches(listResult.results, cacheEntityName);
|
|
154
|
+
await deleteCacheEntriesInBatches(listResult.results, cacheEntityName, options);
|
|
133
155
|
|
|
134
156
|
if (listResult.nextCursor) {
|
|
135
157
|
return (
|
|
@@ -172,10 +194,10 @@ async function clearExpirationCursorCache(
|
|
|
172
194
|
|
|
173
195
|
const listResult = await entityQueryBuilder.limit(100).getMany();
|
|
174
196
|
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
197
|
+
debugLog(
|
|
198
|
+
`clear expired Records: ${JSON.stringify(listResult.results.map((r) => r.key))}`,
|
|
199
|
+
options,
|
|
200
|
+
);
|
|
179
201
|
|
|
180
202
|
await deleteCacheEntriesInBatches(listResult.results, cacheEntityName);
|
|
181
203
|
|
|
@@ -265,14 +287,12 @@ export async function clearTablesCache(
|
|
|
265
287
|
"clearing cache",
|
|
266
288
|
);
|
|
267
289
|
} finally {
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
// eslint-disable-next-line no-console
|
|
271
|
-
console.info(`Cleared ${totalRecords} cache records in ${duration} seconds`);
|
|
272
|
-
}
|
|
290
|
+
const duration = DateTime.now().toSeconds() - startTime.toSeconds();
|
|
291
|
+
debugLog(`Cleared ${totalRecords} cache records in ${duration} seconds`, options);
|
|
273
292
|
}
|
|
274
293
|
}
|
|
275
294
|
/**
|
|
295
|
+
* since https://developer.atlassian.com/platform/forge/changelog/#CHANGE-3038
|
|
276
296
|
* Clears expired cache entries with retry logic and performance logging.
|
|
277
297
|
*
|
|
278
298
|
* @param options - ForgeSQL ORM options
|
|
@@ -293,16 +313,17 @@ export async function clearExpiredCache(options: ForgeSqlOrmOptions): Promise<vo
|
|
|
293
313
|
);
|
|
294
314
|
} finally {
|
|
295
315
|
const duration = DateTime.now().toSeconds() - startTime.toSeconds();
|
|
296
|
-
|
|
297
|
-
// eslint-disable-next-line no-console
|
|
298
|
-
console.debug(`Cleared ${totalRecords} expired cache records in ${duration} seconds`);
|
|
299
|
-
}
|
|
316
|
+
debugLog(`Cleared ${totalRecords} expired cache records in ${duration} seconds`, options);
|
|
300
317
|
}
|
|
301
318
|
}
|
|
302
319
|
|
|
303
320
|
/**
|
|
304
321
|
* Retrieves data from cache if it exists and is not expired.
|
|
305
322
|
*
|
|
323
|
+
* Note: Due to Forge KVS asynchronous deletion (up to 48 hours), expired entries may still
|
|
324
|
+
* be returned. This function checks the expiration timestamp to filter out expired entries.
|
|
325
|
+
* If cache growth impacts performance, use the Clear Cache Scheduler Trigger.
|
|
326
|
+
*
|
|
306
327
|
* @param query - Query object with toSQL method
|
|
307
328
|
* @param options - ForgeSQL ORM options
|
|
308
329
|
* @returns Cached data if found and valid, undefined otherwise
|
|
@@ -325,27 +346,27 @@ export async function getFromCache<T>(
|
|
|
325
346
|
|
|
326
347
|
// Skip cache if table is in cache context (will be cleared)
|
|
327
348
|
if (await isTableContainsTableInCacheContext(sqlQuery.sql, options)) {
|
|
328
|
-
|
|
329
|
-
// eslint-disable-next-line no-console
|
|
330
|
-
console.warn(`Context contains value to clear. Skip getting from cache`);
|
|
331
|
-
}
|
|
349
|
+
debugLog("Context contains value to clear. Skip getting from cache", options);
|
|
332
350
|
return undefined;
|
|
333
351
|
}
|
|
334
352
|
|
|
335
353
|
try {
|
|
336
354
|
const cacheResult = await kvs.entity<CacheEntity>(options.cacheEntityName).get(key);
|
|
337
355
|
|
|
338
|
-
if (
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
356
|
+
if (cacheResult) {
|
|
357
|
+
if (
|
|
358
|
+
(cacheResult[expirationName] as number) >= getCurrentTime() &&
|
|
359
|
+
extractBacktickedValues(sqlQuery.sql, options) === cacheResult[entityQueryName]
|
|
360
|
+
) {
|
|
361
|
+
debugLog(`Get value from cache, cacheKey: ${key}`, options);
|
|
362
|
+
const results = cacheResult[dataName];
|
|
363
|
+
return JSON.parse(results as string);
|
|
364
|
+
} else {
|
|
365
|
+
debugLog(
|
|
366
|
+
`Expired cache entry still exists (will be automatically removed within 48 hours per Forge KVS TTL documentation), cacheKey: ${key}`,
|
|
367
|
+
options,
|
|
368
|
+
);
|
|
346
369
|
}
|
|
347
|
-
const results = cacheResult[dataName];
|
|
348
|
-
return JSON.parse(results as string);
|
|
349
370
|
}
|
|
350
371
|
} catch (error: any) {
|
|
351
372
|
// eslint-disable-next-line no-console
|
|
@@ -358,11 +379,18 @@ export async function getFromCache<T>(
|
|
|
358
379
|
/**
|
|
359
380
|
* Stores query results in cache with specified TTL.
|
|
360
381
|
*
|
|
382
|
+
* Uses Forge KVS TTL feature to set expiration. Note that expired data deletion is asynchronous:
|
|
383
|
+
* expired data is not removed immediately upon expiry. Deletion may take up to 48 hours.
|
|
384
|
+
* During this window, read operations may still return expired results. If your app requires
|
|
385
|
+
* strict expiry semantics, consider using the Clear Cache Scheduler Trigger to proactively
|
|
386
|
+
* clean up expired entries, especially if cache growth impacts INSERT/UPDATE performance.
|
|
387
|
+
*
|
|
361
388
|
* @param query - Query object with toSQL method
|
|
362
389
|
* @param options - ForgeSQL ORM options
|
|
363
390
|
* @param results - Data to cache
|
|
364
|
-
* @param cacheTtl - Time to live in seconds
|
|
391
|
+
* @param cacheTtl - Time to live in seconds (maximum TTL is 1 year from write time)
|
|
365
392
|
* @returns Promise that resolves when data is stored in cache
|
|
393
|
+
* @see https://developer.atlassian.com/platform/forge/runtime-reference/storage-api-basic-api/#ttl
|
|
366
394
|
*/
|
|
367
395
|
export async function setCacheResult(
|
|
368
396
|
query: { toSQL: () => Query },
|
|
@@ -385,10 +413,7 @@ export async function setCacheResult(
|
|
|
385
413
|
|
|
386
414
|
// Skip cache if table is in cache context (will be cleared)
|
|
387
415
|
if (await isTableContainsTableInCacheContext(sqlQuery.sql, options)) {
|
|
388
|
-
|
|
389
|
-
// eslint-disable-next-line no-console
|
|
390
|
-
console.warn(`Context contains value to clear. Skip setting from cache`);
|
|
391
|
-
}
|
|
416
|
+
debugLog("Context contains value to clear. Skip setting from cache", options);
|
|
392
417
|
return;
|
|
393
418
|
}
|
|
394
419
|
|
|
@@ -400,17 +425,15 @@ export async function setCacheResult(
|
|
|
400
425
|
key,
|
|
401
426
|
{
|
|
402
427
|
[entityQueryName]: extractBacktickedValues(sqlQuery.sql, options),
|
|
403
|
-
[expirationName]: nowPlusSeconds(cacheTtl),
|
|
428
|
+
[expirationName]: nowPlusSeconds(cacheTtl + 2),
|
|
404
429
|
[dataName]: JSON.stringify(results),
|
|
405
430
|
},
|
|
406
431
|
{ entityName: options.cacheEntityName },
|
|
432
|
+
{ ttl: { value: cacheTtl, unit: "SECONDS" } },
|
|
407
433
|
)
|
|
408
434
|
.execute();
|
|
409
435
|
|
|
410
|
-
|
|
411
|
-
// eslint-disable-next-line no-console
|
|
412
|
-
console.warn(`Store value to cache, cacheKey: ${key}`);
|
|
413
|
-
}
|
|
436
|
+
debugLog(`Store value to cache, cacheKey: ${key}`, options);
|
|
414
437
|
} catch (error: any) {
|
|
415
438
|
// eslint-disable-next-line no-console
|
|
416
439
|
console.error(`Error setting cache: ${error.message}`, error);
|
|
@@ -2,10 +2,28 @@ import { clearExpiredCache } from "../utils/cacheUtils";
|
|
|
2
2
|
import { ForgeSqlOrmOptions } from "../core/ForgeSQLQueryBuilder";
|
|
3
3
|
|
|
4
4
|
/**
|
|
5
|
-
* Scheduler trigger for clearing expired cache entries.
|
|
5
|
+
* Scheduler trigger for proactively clearing expired cache entries.
|
|
6
6
|
*
|
|
7
|
-
*
|
|
8
|
-
*
|
|
7
|
+
* **Why this trigger is needed:**
|
|
8
|
+
*
|
|
9
|
+
* While forge-sql-orm uses Forge KVS TTL feature to mark entries as expired, **actual deletion
|
|
10
|
+
* is asynchronous and may take up to 48 hours**. During this window, expired entries remain in
|
|
11
|
+
* storage and can impact INSERT/UPDATE performance if the cache grows large.
|
|
12
|
+
*
|
|
13
|
+
* This scheduler trigger proactively cleans up expired entries by querying the expiration index
|
|
14
|
+
* and deleting entries where expiration < now, preventing cache growth from impacting data
|
|
15
|
+
* modification operations.
|
|
16
|
+
*
|
|
17
|
+
* **When to use:**
|
|
18
|
+
* - Your cache grows large over time
|
|
19
|
+
* - INSERT/UPDATE operations are slowing down due to cache size
|
|
20
|
+
* - You need strict expiry semantics (immediate cleanup)
|
|
21
|
+
* - You want to reduce storage costs proactively
|
|
22
|
+
*
|
|
23
|
+
* **When optional:**
|
|
24
|
+
* - Small cache footprint
|
|
25
|
+
* - No performance impact on data modifications
|
|
26
|
+
* - You can tolerate expired entries being returned for up to 48 hours
|
|
9
27
|
*
|
|
10
28
|
* @note This function is automatically disabled in production environments and will return a 500 error if called.
|
|
11
29
|
*
|
|
@@ -28,7 +46,7 @@ import { ForgeSqlOrmOptions } from "../core/ForgeSQLQueryBuilder";
|
|
|
28
46
|
*
|
|
29
47
|
* @example
|
|
30
48
|
* ```yaml
|
|
31
|
-
* # In manifest.yml
|
|
49
|
+
* # In manifest.yml (optional - only if cache growth impacts INSERT/UPDATE performance)
|
|
32
50
|
* scheduledTrigger:
|
|
33
51
|
* - key: clear-cache-trigger
|
|
34
52
|
* function: clearCache
|
|
@@ -38,6 +56,8 @@ import { ForgeSqlOrmOptions } from "../core/ForgeSQLQueryBuilder";
|
|
|
38
56
|
* - key: clearCache
|
|
39
57
|
* handler: index.clearCache
|
|
40
58
|
* ```
|
|
59
|
+
*
|
|
60
|
+
* @see https://developer.atlassian.com/platform/forge/runtime-reference/storage-api-basic-api/#ttl
|
|
41
61
|
*/
|
|
42
62
|
export const clearCacheSchedulerTrigger = async (options?: ForgeSqlOrmOptions) => {
|
|
43
63
|
try {
|