forge-sql-orm 2.1.9 → 2.1.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +202 -254
- package/dist/ForgeSQLORM.js +3238 -3231
- package/dist/ForgeSQLORM.js.map +1 -1
- package/dist/ForgeSQLORM.mjs +3236 -3229
- package/dist/ForgeSQLORM.mjs.map +1 -1
- package/dist/core/ForgeSQLORM.d.ts +70 -16
- package/dist/core/ForgeSQLORM.d.ts.map +1 -1
- package/dist/core/ForgeSQLQueryBuilder.d.ts +95 -16
- package/dist/core/ForgeSQLQueryBuilder.d.ts.map +1 -1
- package/dist/core/SystemTables.d.ts +82 -82
- package/dist/lib/drizzle/extensions/additionalActions.d.ts +30 -6
- package/dist/lib/drizzle/extensions/additionalActions.d.ts.map +1 -1
- package/dist/utils/cacheUtils.d.ts.map +1 -1
- package/dist/utils/forgeDriver.d.ts.map +1 -1
- package/dist/utils/forgeDriverProxy.d.ts +6 -2
- package/dist/utils/forgeDriverProxy.d.ts.map +1 -1
- package/dist/utils/metadataContextUtils.d.ts +5 -2
- package/dist/utils/metadataContextUtils.d.ts.map +1 -1
- package/dist/utils/sqlUtils.d.ts +72 -1
- package/dist/utils/sqlUtils.d.ts.map +1 -1
- package/dist/webtriggers/index.d.ts +1 -1
- package/dist/webtriggers/index.d.ts.map +1 -1
- package/dist/webtriggers/slowQuerySchedulerTrigger.d.ts +67 -0
- package/dist/webtriggers/slowQuerySchedulerTrigger.d.ts.map +1 -0
- package/package.json +14 -14
- package/src/core/ForgeSQLORM.ts +165 -34
- package/src/core/ForgeSQLQueryBuilder.ts +118 -19
- package/src/core/SystemTables.ts +1 -1
- package/src/lib/drizzle/extensions/additionalActions.ts +231 -18
- package/src/utils/cacheUtils.ts +3 -1
- package/src/utils/forgeDriver.ts +10 -42
- package/src/utils/forgeDriverProxy.ts +58 -6
- package/src/utils/metadataContextUtils.ts +21 -6
- package/src/utils/sqlUtils.ts +229 -2
- package/src/webtriggers/index.ts +1 -1
- package/src/webtriggers/slowQuerySchedulerTrigger.ts +82 -0
- package/dist/webtriggers/topSlowestStatementLastHourTrigger.d.ts +0 -114
- package/dist/webtriggers/topSlowestStatementLastHourTrigger.d.ts.map +0 -1
- package/src/webtriggers/topSlowestStatementLastHourTrigger.ts +0 -563
package/README.md
CHANGED
|
@@ -9,7 +9,6 @@
|
|
|
9
9
|
|
|
10
10
|
[](https://github.com/vzakharchenko/forge-sql-orm/actions/workflows/node.js.yml)
|
|
11
11
|
[](https://coveralls.io/github/vzakharchenko/forge-sql-orm?branch=master)
|
|
12
|
-
[](https://sonarcloud.io/summary/new_code?id=vzakharchenko_forge-sql-orm)
|
|
13
12
|
[](https://deepscan.io/dashboard#view=project&tid=26652&pid=29272&bid=940614)
|
|
14
13
|
|
|
15
14
|
|
|
@@ -19,11 +18,11 @@
|
|
|
19
18
|
- ✅ **Custom Drizzle Driver** for direct integration with @forge/sql
|
|
20
19
|
- ✅ **Local Cache System (Level 1)** for in-memory query optimization within single resolver invocation scope
|
|
21
20
|
- ✅ **Global Cache System (Level 2)** with cross-invocation caching, automatic cache invalidation and context-aware operations (using [@forge/kvs](https://developer.atlassian.com/platform/forge/storage-reference/storage-api-custom-entities/) )
|
|
22
|
-
- ✅ **Performance Monitoring**:
|
|
21
|
+
- ✅ **Performance Monitoring**: Query execution metrics and analysis capabilities with automatic error analysis for timeout and OOM errors
|
|
23
22
|
- ✅ **Type-Safe Query Building**: Write SQL queries with full TypeScript support
|
|
24
23
|
- ✅ **Supports complex SQL queries** with joins and filtering using Drizzle ORM
|
|
25
24
|
- ✅ **Advanced Query Methods**: `selectFrom()`, `selectDistinctFrom()`, `selectCacheableFrom()`, `selectDistinctCacheableFrom()` for all-column queries with field aliasing
|
|
26
|
-
- ✅ **Query Execution with Metadata**: `executeWithMetadata()` method for capturing detailed execution metrics including database execution time, response size, and
|
|
25
|
+
- ✅ **Query Execution with Metadata**: `executeWithMetadata()` method for capturing detailed execution metrics including database execution time, response size, and query analysis capabilities with performance monitoring
|
|
27
26
|
- ✅ **Raw SQL Execution**: `execute()`, `executeCacheable()`, `executeDDL()`, and `executeDDLActions()` methods for direct SQL queries with local and global caching
|
|
28
27
|
- ✅ **Common Table Expressions (CTEs)**: `with()` method for complex queries with subqueries
|
|
29
28
|
- ✅ **Schema migration support**, allowing automatic schema evolution
|
|
@@ -82,7 +81,7 @@
|
|
|
82
81
|
- [Query Analysis Example](examples/forge-sql-orm-example-query-analyses)
|
|
83
82
|
- [Organization Tracker Example](examples/forge-sql-orm-example-org-tracker)
|
|
84
83
|
- [Checklist Example](examples/forge-sql-orm-example-checklist)
|
|
85
|
-
- [Cache Example](examples/forge-sql-orm-example-cache) - Advanced caching capabilities with
|
|
84
|
+
- [Cache Example](examples/forge-sql-orm-example-cache) - Advanced caching capabilities with performance monitoring
|
|
86
85
|
|
|
87
86
|
### 📚 Reference
|
|
88
87
|
- [ForgeSqlOrmOptions](#forgesqlormoptions)
|
|
@@ -99,7 +98,6 @@
|
|
|
99
98
|
- [Global Cache System (Level 2)](#global-cache-system-level-2) - Cross-invocation persistent caching
|
|
100
99
|
- [Local Cache System (Level 1)](#local-cache-operations-level-1) - In-memory invocation caching
|
|
101
100
|
- [Optimistic Locking](#optimistic-locking) - Data consistency
|
|
102
|
-
- [Memory Usage Monitoring](#memory-usage-monitoring) - Memory-intensive query detection
|
|
103
101
|
- [Migration Tools](#web-triggers-for-migrations) - Database migrations
|
|
104
102
|
- [Query Analysis](#query-analysis-and-performance-optimization) - Performance optimization
|
|
105
103
|
|
|
@@ -108,7 +106,7 @@
|
|
|
108
106
|
- [Optimistic Locking Example](examples/forge-sql-orm-example-optimistic-locking) - Real-world conflict handling
|
|
109
107
|
- [Organization Tracker Example](examples/forge-sql-orm-example-org-tracker) - Complex relationships
|
|
110
108
|
- [Checklist Example](examples/forge-sql-orm-example-checklist) - Jira integration
|
|
111
|
-
- [Cache Example](examples/forge-sql-orm-example-cache) - Advanced caching
|
|
109
|
+
- [Cache Example](examples/forge-sql-orm-example-cache) - Advanced caching capabilities
|
|
112
110
|
|
|
113
111
|
## Usage Approaches
|
|
114
112
|
|
|
@@ -276,7 +274,41 @@ await forgeSQL.executeWithLocalContext(async () => {
|
|
|
276
274
|
});
|
|
277
275
|
```
|
|
278
276
|
|
|
279
|
-
### 4.
|
|
277
|
+
### 4. Resolver Performance Monitoring
|
|
278
|
+
```typescript
|
|
279
|
+
// Resolver with performance monitoring
|
|
280
|
+
resolver.define("fetch", async (req: Request) => {
|
|
281
|
+
try {
|
|
282
|
+
return await forgeSQL.executeWithMetadata(
|
|
283
|
+
async () => {
|
|
284
|
+
// Resolver logic with multiple queries
|
|
285
|
+
const users = await forgeSQL.selectFrom(demoUsers);
|
|
286
|
+
const orders = await forgeSQL.selectFrom(demoOrders)
|
|
287
|
+
.where(eq(demoOrders.userId, demoUsers.id));
|
|
288
|
+
return { users, orders };
|
|
289
|
+
},
|
|
290
|
+
async (totalDbExecutionTime, totalResponseSize, printQueriesWithPlan) => {
|
|
291
|
+
const threshold = 500; // ms baseline for this resolver
|
|
292
|
+
|
|
293
|
+
if (totalDbExecutionTime > threshold * 1.5) {
|
|
294
|
+
console.warn(`[Performance Warning fetch] Resolver exceeded DB time: ${totalDbExecutionTime} ms`);
|
|
295
|
+
await printQueriesWithPlan(); // Optionally log or capture diagnostics for further analysis
|
|
296
|
+
} else if (totalDbExecutionTime > threshold) {
|
|
297
|
+
console.debug(`[Performance Debug] High DB time: ${totalDbExecutionTime} ms`);
|
|
298
|
+
}
|
|
299
|
+
|
|
300
|
+
console.log(`DB response size: ${totalResponseSize} bytes`);
|
|
301
|
+
}
|
|
302
|
+
);
|
|
303
|
+
} catch (e) {
|
|
304
|
+
const error = e?.cause?.debug?.sqlMessage ?? e?.cause;
|
|
305
|
+
console.error(error, e);
|
|
306
|
+
throw error;
|
|
307
|
+
}
|
|
308
|
+
});
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
### 5. Next Steps
|
|
280
312
|
- [Full Installation Guide](#installation) - Complete setup instructions
|
|
281
313
|
- [Core Features](#core-features) - Learn about key capabilities
|
|
282
314
|
- [Global Cache System (Level 2)](#global-cache-system-level-2) - Cross-invocation caching features
|
|
@@ -334,13 +366,24 @@ const cachedRawUsers = await forgeSQL.executeCacheable(
|
|
|
334
366
|
300
|
|
335
367
|
);
|
|
336
368
|
|
|
337
|
-
// Raw SQL with execution metadata
|
|
369
|
+
// Raw SQL with execution metadata and performance monitoring
|
|
338
370
|
const usersWithMetadata = await forgeSQL.executeWithMetadata(
|
|
339
|
-
async () =>
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
371
|
+
async () => {
|
|
372
|
+
const users = await forgeSQL.selectFrom(usersTable);
|
|
373
|
+
const orders = await forgeSQL.selectFrom(ordersTable).where(eq(ordersTable.userId, usersTable.id));
|
|
374
|
+
return { users, orders };
|
|
375
|
+
},
|
|
376
|
+
(totalDbExecutionTime, totalResponseSize, printQueriesWithPlan) => {
|
|
377
|
+
const threshold = 500; // ms baseline for this resolver
|
|
378
|
+
|
|
379
|
+
if (totalDbExecutionTime > threshold * 1.5) {
|
|
380
|
+
console.warn(`[Performance Warning] Resolver exceeded DB time: ${totalDbExecutionTime} ms`);
|
|
381
|
+
await printQueriesWithPlan(); // Analyze and print query execution plans
|
|
382
|
+
} else if (totalDbExecutionTime > threshold) {
|
|
383
|
+
console.debug(`[Performance Debug] High DB time: ${totalDbExecutionTime} ms`);
|
|
384
|
+
}
|
|
385
|
+
|
|
386
|
+
console.log(`DB response size: ${totalResponseSize} bytes`);
|
|
344
387
|
}
|
|
345
388
|
);
|
|
346
389
|
|
|
@@ -444,13 +487,24 @@ const cachedRawUsers = await forgeSQL.executeCacheable(
|
|
|
444
487
|
300
|
|
445
488
|
);
|
|
446
489
|
|
|
447
|
-
// Raw SQL with execution metadata
|
|
490
|
+
// Raw SQL with execution metadata and performance monitoring
|
|
448
491
|
const usersWithMetadata = await forgeSQL.executeWithMetadata(
|
|
449
|
-
async () =>
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
492
|
+
async () => {
|
|
493
|
+
const users = await forgeSQL.selectFrom(usersTable);
|
|
494
|
+
const orders = await forgeSQL.selectFrom(ordersTable).where(eq(ordersTable.userId, usersTable.id));
|
|
495
|
+
return { users, orders };
|
|
496
|
+
},
|
|
497
|
+
(totalDbExecutionTime, totalResponseSize, printQueriesWithPlan) => {
|
|
498
|
+
const threshold = 500; // ms baseline for this resolver
|
|
499
|
+
|
|
500
|
+
if (totalDbExecutionTime > threshold * 1.5) {
|
|
501
|
+
console.warn(`[Performance Warning] Resolver exceeded DB time: ${totalDbExecutionTime} ms`);
|
|
502
|
+
await printQueriesWithPlan(); // Analyze and print query execution plans
|
|
503
|
+
} else if (totalDbExecutionTime > threshold) {
|
|
504
|
+
console.debug(`[Performance Debug] High DB time: ${totalDbExecutionTime} ms`);
|
|
505
|
+
}
|
|
506
|
+
|
|
507
|
+
console.log(`DB response size: ${totalResponseSize} bytes`);
|
|
454
508
|
}
|
|
455
509
|
);
|
|
456
510
|
```
|
|
@@ -1215,16 +1269,26 @@ const users = await forgeSQL
|
|
|
1215
1269
|
const users = await forgeSQL
|
|
1216
1270
|
.executeCacheable("SELECT * FROM users WHERE active = ?", [true], 300);
|
|
1217
1271
|
|
|
1218
|
-
// Using executeWithMetadata() for capturing execution metrics
|
|
1219
|
-
const usersWithMetadata = await forgeSQL
|
|
1220
|
-
|
|
1221
|
-
|
|
1222
|
-
|
|
1223
|
-
|
|
1224
|
-
|
|
1225
|
-
|
|
1226
|
-
|
|
1227
|
-
|
|
1272
|
+
// Using executeWithMetadata() for capturing execution metrics and performance monitoring
|
|
1273
|
+
const usersWithMetadata = await forgeSQL.executeWithMetadata(
|
|
1274
|
+
async () => {
|
|
1275
|
+
const users = await forgeSQL.selectFrom(usersTable);
|
|
1276
|
+
const orders = await forgeSQL.selectFrom(ordersTable).where(eq(ordersTable.userId, usersTable.id));
|
|
1277
|
+
return { users, orders };
|
|
1278
|
+
},
|
|
1279
|
+
(totalDbExecutionTime, totalResponseSize, printQueriesWithPlan) => {
|
|
1280
|
+
const threshold = 500; // ms baseline for this resolver
|
|
1281
|
+
|
|
1282
|
+
if (totalDbExecutionTime > threshold * 1.5) {
|
|
1283
|
+
console.warn(`[Performance Warning] Resolver exceeded DB time: ${totalDbExecutionTime} ms`);
|
|
1284
|
+
await printQueriesWithPlan(); // Analyze and print query execution plans
|
|
1285
|
+
} else if (totalDbExecutionTime > threshold) {
|
|
1286
|
+
console.debug(`[Performance Debug] High DB time: ${totalDbExecutionTime} ms`);
|
|
1287
|
+
}
|
|
1288
|
+
|
|
1289
|
+
console.log(`DB response size: ${totalResponseSize} bytes`);
|
|
1290
|
+
}
|
|
1291
|
+
);
|
|
1228
1292
|
|
|
1229
1293
|
// Using executeDDL() for DDL operations (CREATE, ALTER, DROP, etc.)
|
|
1230
1294
|
await forgeSQL.executeDDL(`
|
|
@@ -1575,13 +1639,24 @@ await forgeSQL.executeWithLocalContext(async () => {
|
|
|
1575
1639
|
[true]
|
|
1576
1640
|
);
|
|
1577
1641
|
|
|
1578
|
-
// Raw SQL with execution metadata and
|
|
1642
|
+
// Raw SQL with execution metadata and performance monitoring
|
|
1579
1643
|
const usersWithMetadata = await forgeSQL.executeWithMetadata(
|
|
1580
|
-
async () =>
|
|
1581
|
-
|
|
1582
|
-
|
|
1583
|
-
|
|
1584
|
-
|
|
1644
|
+
async () => {
|
|
1645
|
+
const users = await forgeSQL.selectFrom(usersTable);
|
|
1646
|
+
const orders = await forgeSQL.selectFrom(ordersTable).where(eq(ordersTable.userId, usersTable.id));
|
|
1647
|
+
return { users, orders };
|
|
1648
|
+
},
|
|
1649
|
+
(totalDbExecutionTime, totalResponseSize, printQueriesWithPlan) => {
|
|
1650
|
+
const threshold = 500; // ms baseline for this resolver
|
|
1651
|
+
|
|
1652
|
+
if (totalDbExecutionTime > threshold * 1.5) {
|
|
1653
|
+
console.warn(`[Performance Warning] Resolver exceeded DB time: ${totalDbExecutionTime} ms`);
|
|
1654
|
+
await printQueriesWithPlan(); // Analyze and print query execution plans
|
|
1655
|
+
} else if (totalDbExecutionTime > threshold) {
|
|
1656
|
+
console.debug(`[Performance Debug] High DB time: ${totalDbExecutionTime} ms`);
|
|
1657
|
+
}
|
|
1658
|
+
|
|
1659
|
+
console.log(`DB response size: ${totalResponseSize} bytes`);
|
|
1585
1660
|
}
|
|
1586
1661
|
);
|
|
1587
1662
|
|
|
@@ -1754,13 +1829,24 @@ const userStats = await forgeSQL
|
|
|
1754
1829
|
.from(sql`activeUsers au`)
|
|
1755
1830
|
.leftJoin(sql`completedOrders co`, eq(sql`au.id`, sql`co.userId`));
|
|
1756
1831
|
|
|
1757
|
-
// Using executeWithMetadata() for capturing execution metrics with
|
|
1832
|
+
// Using executeWithMetadata() for capturing execution metrics with performance monitoring
|
|
1758
1833
|
const usersWithMetadata = await forgeSQL.executeWithMetadata(
|
|
1759
|
-
async () =>
|
|
1760
|
-
|
|
1761
|
-
|
|
1762
|
-
|
|
1763
|
-
|
|
1834
|
+
async () => {
|
|
1835
|
+
const users = await forgeSQL.selectFrom(usersTable);
|
|
1836
|
+
const orders = await forgeSQL.selectFrom(ordersTable).where(eq(ordersTable.userId, usersTable.id));
|
|
1837
|
+
return { users, orders };
|
|
1838
|
+
},
|
|
1839
|
+
(totalDbExecutionTime, totalResponseSize, printQueriesWithPlan) => {
|
|
1840
|
+
const threshold = 500; // ms baseline for this resolver
|
|
1841
|
+
|
|
1842
|
+
if (totalDbExecutionTime > threshold * 1.5) {
|
|
1843
|
+
console.warn(`[Performance Warning] Resolver exceeded DB time: ${totalDbExecutionTime} ms`);
|
|
1844
|
+
await printQueriesWithPlan(); // Analyze and print query execution plans
|
|
1845
|
+
} else if (totalDbExecutionTime > threshold) {
|
|
1846
|
+
console.debug(`[Performance Debug] High DB time: ${totalDbExecutionTime} ms`);
|
|
1847
|
+
}
|
|
1848
|
+
|
|
1849
|
+
console.log(`DB response size: ${totalResponseSize} bytes`);
|
|
1764
1850
|
}
|
|
1765
1851
|
);
|
|
1766
1852
|
```
|
|
@@ -2065,6 +2151,64 @@ Atlassian provides comprehensive query analysis tools in the development console
|
|
|
2065
2151
|
|
|
2066
2152
|
Our analysis tools complement these built-in features by providing additional insights directly from TiDB's system schemas.
|
|
2067
2153
|
|
|
2154
|
+
### Automatic Error Analysis
|
|
2155
|
+
|
|
2156
|
+
Forge-SQL-ORM automatically intercepts and analyzes critical query errors to help you diagnose performance issues. When a query fails due to **timeout** or **out-of-memory** errors, the library automatically:
|
|
2157
|
+
|
|
2158
|
+
1. **Detects the error type** (SQL_QUERY_TIMEOUT or Out of Memory)
|
|
2159
|
+
2. **Logs detailed error information** to the Forge Developer Console
|
|
2160
|
+
3. **Waits for system tables to populate** (200ms delay)
|
|
2161
|
+
4. **Retrieves and logs the execution plan** for the failed query
|
|
2162
|
+
5. **Provides performance metrics** including memory usage, execution time, and query details
|
|
2163
|
+
|
|
2164
|
+
This automatic analysis happens transparently - no additional code is required on your part.
|
|
2165
|
+
|
|
2166
|
+
#### Supported Error Types
|
|
2167
|
+
|
|
2168
|
+
- **SQL_QUERY_TIMEOUT**: Queries that exceed the execution time limit
|
|
2169
|
+
- **Out of Memory (OOM)**: Queries that exceed the 16 MiB memory limit (errno: 8175)
|
|
2170
|
+
|
|
2171
|
+
#### Example Console Output
|
|
2172
|
+
|
|
2173
|
+
When a query fails, you'll see output like this in the Forge Developer Console:
|
|
2174
|
+
|
|
2175
|
+
```
|
|
2176
|
+
❌ TIMEOUT detected - Query exceeded time limit
|
|
2177
|
+
⏳ Waiting 200ms for CLUSTER_STATEMENTS_SUMMARY to populate...
|
|
2178
|
+
📊 Analyzing query performance and execution plan...
|
|
2179
|
+
⏱️ Query duration: 10500ms
|
|
2180
|
+
|
|
2181
|
+
SQL: SELECT * FROM users u INNER JOIN orders o ON u.id = o.user_id WHERE u.active = ? | Memory: 12.45 MB | Time: 10500.00 ms | stmtType: Select | Executions: 1
|
|
2182
|
+
Plan:
|
|
2183
|
+
id task estRows operator info actRows execution info memory disk
|
|
2184
|
+
Projection_7 root 1000.00 forge_38dd1c6156b94bb59c2c9a45582bbfc7.users.id, ... 1000 time:10.5s, loops:1 12.45 MB N/A
|
|
2185
|
+
└─IndexHashJoin_14 root 1000.00 inner join, ... 1000 time:10.2s, loops:1 11.98 MB N/A
|
|
2186
|
+
```
|
|
2187
|
+
|
|
2188
|
+
#### How It Works
|
|
2189
|
+
|
|
2190
|
+
The error analysis mechanism:
|
|
2191
|
+
|
|
2192
|
+
1. **Error Detection**: When a query fails, the driver proxy checks the error code/errno
|
|
2193
|
+
2. **Error Logging**: Logs the specific error type to console.error
|
|
2194
|
+
3. **Data Population Wait**: Waits 200ms for TiDB's `CLUSTER_STATEMENTS_SUMMARY` table to be populated with the failed query's metadata
|
|
2195
|
+
4. **Query Analysis**: Automatically calls `printQueriesWithPlan()` to retrieve and display:
|
|
2196
|
+
- SQL query text
|
|
2197
|
+
- Memory consumption (average and max in MB)
|
|
2198
|
+
- Execution time (average in ms)
|
|
2199
|
+
- Statement type
|
|
2200
|
+
- Number of executions
|
|
2201
|
+
- Detailed execution plan
|
|
2202
|
+
|
|
2203
|
+
#### Benefits
|
|
2204
|
+
|
|
2205
|
+
- **Zero Configuration**: Works automatically - no setup required
|
|
2206
|
+
- **Immediate Insights**: Get execution plans for failed queries instantly
|
|
2207
|
+
- **Performance Debugging**: Identify bottlenecks without manual investigation
|
|
2208
|
+
- **Development Console Integration**: All logs appear in Atlassian Forge Developer Console
|
|
2209
|
+
- **No Code Changes**: Existing code automatically benefits from error analysis
|
|
2210
|
+
|
|
2211
|
+
> **💡 Tip**: The automatic error analysis only triggers for timeout and OOM errors. Other errors are logged normally without plan analysis.
|
|
2068
2212
|
|
|
2069
2213
|
### Available Analysis Tools
|
|
2070
2214
|
|
|
@@ -2138,213 +2282,6 @@ This analysis provides insights into:
|
|
|
2138
2282
|
- Resource usage at each step
|
|
2139
2283
|
- Performance optimization opportunities
|
|
2140
2284
|
|
|
2141
|
-
## Performance Monitoring
|
|
2142
|
-
|
|
2143
|
-
[↑ Back to Top](#table-of-contents)
|
|
2144
|
-
|
|
2145
|
-
Forge-SQL-ORM provides automated performance monitoring capabilities to help you identify and track memory-intensive queries in your Forge SQL instance. This feature is **essential for Atlassian Forge applications** as it helps you stay within the **16 MiB per query memory limit** and provides detailed insights for optimization.
|
|
2146
|
-
|
|
2147
|
-
### Why Performance Monitoring is Critical
|
|
2148
|
-
|
|
2149
|
-
Atlassian Forge SQL has a strict **16 MiB memory limit per query**. Unlike slow query detection (which is available in the Forge Developer Console), there's **no built-in way to monitor memory usage** of your queries. This monitoring system fills that gap by:
|
|
2150
|
-
|
|
2151
|
-
- **Detecting memory-intensive queries** before they hit the 16 MiB limit
|
|
2152
|
-
- **Providing detailed memory metrics** including average and maximum memory usage
|
|
2153
|
-
- **Showing execution plans** to help optimize memory consumption
|
|
2154
|
-
- **Configurable thresholds** to match your application's memory requirements
|
|
2155
|
-
- **Scheduled monitoring** via Forge scheduler triggers
|
|
2156
|
-
|
|
2157
|
-
### Overview
|
|
2158
|
-
|
|
2159
|
-
The performance monitoring system:
|
|
2160
|
-
- **Automatically detects memory-intensive queries** based on configurable memory thresholds
|
|
2161
|
-
- **Provides detailed memory metrics** including execution time, memory usage, and execution plans
|
|
2162
|
-
- **Logs memory issues** to the Forge Developer Console for easy debugging
|
|
2163
|
-
- **Supports scheduled monitoring** via Forge scheduler triggers
|
|
2164
|
-
- **Filters out system queries** to focus on your application's performance
|
|
2165
|
-
|
|
2166
|
-
### Key Features
|
|
2167
|
-
|
|
2168
|
-
- **Memory-Focused Performance Monitoring**: Primary focus on memory usage with configurable thresholds
|
|
2169
|
-
- **Atlassian 16 MiB Limit Awareness**: Designed specifically for Forge SQL's memory constraints
|
|
2170
|
-
- **Execution Plan Analysis**: Shows detailed query plans to help optimize memory consumption
|
|
2171
|
-
- **Configurable Thresholds**: Set custom memory usage thresholds (default: 4MB warning)
|
|
2172
|
-
- **Automatic Filtering**: Excludes system queries (`Use`, `Set`, `Show`) and empty queries
|
|
2173
|
-
- **Scheduled Monitoring**: Run automatically on configurable intervals
|
|
2174
|
-
|
|
2175
|
-
### Basic Usage
|
|
2176
|
-
|
|
2177
|
-
#### 1. Import the Trigger
|
|
2178
|
-
|
|
2179
|
-
```typescript
|
|
2180
|
-
import ForgeSQL, { topSlowestStatementLastHourTrigger } from "forge-sql-orm";
|
|
2181
|
-
```
|
|
2182
|
-
|
|
2183
|
-
#### 2. Create a Scheduler Function
|
|
2184
|
-
|
|
2185
|
-
```typescript
|
|
2186
|
-
import ForgeSQL, { topSlowestStatementLastHourTrigger } from 'forge-sql-orm';
|
|
2187
|
-
|
|
2188
|
-
// Initialize ForgeSQL ORM instance
|
|
2189
|
-
const forgeSQL = new ForgeSQL();
|
|
2190
|
-
|
|
2191
|
-
// Basic usage with default thresholds (300ms latency, 8MB memory warning)
|
|
2192
|
-
export const performanceTrigger = () =>
|
|
2193
|
-
topSlowestStatementLastHourTrigger(forgeSQL);
|
|
2194
|
-
|
|
2195
|
-
// Conservative performance monitoring: 4MB warning (well below 16MB limit)
|
|
2196
|
-
export const conservativeMemoryTrigger = () =>
|
|
2197
|
-
topSlowestStatementLastHourTrigger(forgeSQL, { memoryThresholdBytes: 4 * 1024 * 1024 });
|
|
2198
|
-
|
|
2199
|
-
// Aggressive performance monitoring: 12MB warning (75% of 16MB limit)
|
|
2200
|
-
export const aggressiveMemoryTrigger = () =>
|
|
2201
|
-
topSlowestStatementLastHourTrigger(forgeSQL, { memoryThresholdBytes: 12 * 1024 * 1024 });
|
|
2202
|
-
|
|
2203
|
-
// Memory-only performance monitoring: Only trigger on memory usage (latency effectively disabled)
|
|
2204
|
-
export const memoryOnlyTrigger = () =>
|
|
2205
|
-
topSlowestStatementLastHourTrigger(forgeSQL, { warnThresholdMs: 10000, memoryThresholdBytes: 4 * 1024 * 1024 });
|
|
2206
|
-
|
|
2207
|
-
// Latency-only monitoring: Only trigger on slow queries (memory effectively disabled)
|
|
2208
|
-
export const latencyOnlyTrigger = () =>
|
|
2209
|
-
topSlowestStatementLastHourTrigger(forgeSQL, { warnThresholdMs: 500, memoryThresholdBytes: 16 * 1024 * 1024 });
|
|
2210
|
-
|
|
2211
|
-
// With execution plan in logs
|
|
2212
|
-
export const performanceWithPlanTrigger = () =>
|
|
2213
|
-
topSlowestStatementLastHourTrigger(forgeSQL, { showPlan: true });
|
|
2214
|
-
```
|
|
2215
|
-
|
|
2216
|
-
#### 3. Configure in manifest.yml
|
|
2217
|
-
|
|
2218
|
-
**As Scheduler Trigger (Recommended for Production):**
|
|
2219
|
-
```yaml
|
|
2220
|
-
scheduledTrigger:
|
|
2221
|
-
- key: performance-trigger
|
|
2222
|
-
function: perfTrigger
|
|
2223
|
-
interval: hour # Required: only hour interval is supported
|
|
2224
|
-
|
|
2225
|
-
function:
|
|
2226
|
-
- key: perfTrigger
|
|
2227
|
-
handler: index.performanceTrigger
|
|
2228
|
-
```
|
|
2229
|
-
|
|
2230
|
-
**As Web Trigger (Development Only):**
|
|
2231
|
-
```yaml
|
|
2232
|
-
webtrigger:
|
|
2233
|
-
- key: print-slowest-queries
|
|
2234
|
-
function: perfTrigger
|
|
2235
|
-
|
|
2236
|
-
function:
|
|
2237
|
-
- key: perfTrigger
|
|
2238
|
-
handler: index.performanceTrigger
|
|
2239
|
-
```
|
|
2240
|
-
|
|
2241
|
-
> **⚠️ Important**: Web triggers are not recommended for production as they violate the "run-on-atlassian" principle. Use scheduler triggers for production monitoring.
|
|
2242
|
-
|
|
2243
|
-
### How It Works
|
|
2244
|
-
|
|
2245
|
-
The performance monitoring trigger works differently depending on how it's configured:
|
|
2246
|
-
|
|
2247
|
-
#### Scheduler Trigger Mode (Production)
|
|
2248
|
-
|
|
2249
|
-
When used as a **scheduler trigger**, the system:
|
|
2250
|
-
- **Runs automatically** on the configured interval (hour only)
|
|
2251
|
-
- **Logs to Forge Developer Console** only when thresholds are exceeded
|
|
2252
|
-
- **No HTTP response** - operates silently in the background
|
|
2253
|
-
- **Perfect for production** monitoring without violating "run-on-atlassian"
|
|
2254
|
-
|
|
2255
|
-
**Example Console Log Output:**
|
|
2256
|
-
```
|
|
2257
|
-
1. Select avg=3006.03ms max=3006.03ms mem≈0.08MB(max 0.08MB) exec=1
|
|
2258
|
-
digest=28344800f90f6c929484e83337404df7e55a660c5f4ce922c4b298ab5e90c425
|
|
2259
|
-
sql=select `demo_users` . `id` as `a_userid_id` , `demo_users` . `name` as `a_username_name` , `demo_orders` . `product` as `a_product_product` , `demo_orders` . `id` as `a_productid_id` , `sleep` ( ? ) from `demo_users` inner join `demo_orders` on `demo_orders` . `user_id` = `demo_users` . `id`
|
|
2260
|
-
|
|
2261
|
-
full plan:
|
|
2262
|
-
id task estRows operator info actRows execution info memory disk
|
|
2263
|
-
Projection_7 root 2.50 forge_38dd1c6156b94bb59c2c9a45582bbfc7.demo_users.id, forge_38dd1c6156b94bb59c2c9a45582bbfc7.demo_users.name, forge_38dd1c6156b94bb59c2c9a45582bbfc7.demo_orders.product, forge_38dd1c6156b94bb59c2c9a45582bbfc7.demo_orders.id, sleep(?)->Column#7 3 time:3s, loops:2, Concurrency:OFF 1.98 KB N/A
|
|
2264
|
-
└─IndexHashJoin_14 root 2.50 inner join, inner:IndexLookUp_11, outer key:forge_38dd1c6156b94bb59c2c9a45582bbfc7.demo_users.id, inner key:forge_38dd1c6156b94bb59c2c9a45582bbfc7.demo_orders.user_id, equal cond:eq(forge_38dd1c6156b94bb59c2c9a45582bbfc7.demo_users.id, forge_38dd1c6156b94bb59c2c9a45582bbfc7.demo_orders.user_id) 3 time:2.11ms, loops:2, inner:{total:1.31ms, concurrency:5, task:1, construct:6.62µs, fetch:1.3ms, build:4.68µs, join:6.14µs} 57.9 KB N/A
|
|
2265
|
-
```
|
|
2266
|
-
|
|
2267
|
-
#### Web Trigger Mode (Development)
|
|
2268
|
-
|
|
2269
|
-
When used as a **web trigger**, the system:
|
|
2270
|
-
- **Runs on-demand** when the web endpoint is called
|
|
2271
|
-
- **Returns JSON response** with detailed metrics
|
|
2272
|
-
- **Logs to console** AND provides structured data
|
|
2273
|
-
- **Useful for development** and debugging
|
|
2274
|
-
|
|
2275
|
-
**Example JSON Response:**
|
|
2276
|
-
```json
|
|
2277
|
-
{
|
|
2278
|
-
"success": true,
|
|
2279
|
-
"window": "last_1h",
|
|
2280
|
-
"top": 1,
|
|
2281
|
-
"warnThresholdMs": 300,
|
|
2282
|
-
"memoryThresholdBytes": 8388608,
|
|
2283
|
-
"rows": [
|
|
2284
|
-
{
|
|
2285
|
-
"rank": 1,
|
|
2286
|
-
"digest": "abc123...",
|
|
2287
|
-
"stmtType": "Select",
|
|
2288
|
-
"schemaName": "myapp",
|
|
2289
|
-
"execCount": 150,
|
|
2290
|
-
"avgLatencyMs": 450.25,
|
|
2291
|
-
"maxLatencyMs": 1200.50,
|
|
2292
|
-
"minLatencyMs": 200.10,
|
|
2293
|
-
"avgProcessTimeMs": 400.20,
|
|
2294
|
-
"avgWaitTimeMs": 50.05,
|
|
2295
|
-
"avgBackoffTimeMs": 0.00,
|
|
2296
|
-
"avgMemMB": 2.5,
|
|
2297
|
-
"maxMemMB": 8.2,
|
|
2298
|
-
"avgMemBytes": 2621440,
|
|
2299
|
-
"maxMemBytes": 8598323,
|
|
2300
|
-
"avgTotalKeys": 1000,
|
|
2301
|
-
"firstSeen": "2024-01-15 10:30:00",
|
|
2302
|
-
"lastSeen": "2024-01-15 11:30:00",
|
|
2303
|
-
"planInCache": true,
|
|
2304
|
-
"planCacheHits": 120,
|
|
2305
|
-
"digestText": "SELECT * FROM users WHERE active = ?",
|
|
2306
|
-
"plan": "IndexScan(users, idx_active)..."
|
|
2307
|
-
}
|
|
2308
|
-
],
|
|
2309
|
-
"generatedAt": "2024-01-15T11:30:00.000Z"
|
|
2310
|
-
}
|
|
2311
|
-
```
|
|
2312
|
-
|
|
2313
|
-
### Configuration Options
|
|
2314
|
-
|
|
2315
|
-
#### Threshold Parameters
|
|
2316
|
-
|
|
2317
|
-
| Parameter | Type | Default | Description |
|
|
2318
|
-
|-----------|------|---------|-------------|
|
|
2319
|
-
| `warnThresholdMs` | `number` | `300` | Latency threshold in milliseconds (secondary) |
|
|
2320
|
-
| `memoryThresholdBytes` | `number` | `8 * 1024 * 1024` | **Memory usage threshold in bytes (primary focus)** |
|
|
2321
|
-
| `showPlan` | `boolean` | `false` | Whether to include execution plan in logs |
|
|
2322
|
-
| `logCache` | `boolean` | `false` | Whether to log cache operations |
|
|
2323
|
-
|
|
2324
|
-
**⚠️ Important: OR Logic**
|
|
2325
|
-
The monitoring uses **OR logic** - if **either** threshold is exceeded, the query will be logged/returned:
|
|
2326
|
-
- Query exceeds `warnThresholdMs` **OR** `memoryThresholdBytes` → Included in results
|
|
2327
|
-
- This means you can set different thresholds for different monitoring priorities
|
|
2328
|
-
- No need to exceed both thresholds simultaneously
|
|
2329
|
-
|
|
2330
|
-
**💡 Pro Tips:**
|
|
2331
|
-
- **Memory-only performance monitoring**: Set `warnThresholdMs` to a very high value (e.g., 10000ms) to trigger only on memory usage
|
|
2332
|
-
- **Latency-only monitoring**: Set `memoryThresholdBytes` to 16MB (16 * 1024 * 1024) to trigger only on latency
|
|
2333
|
-
- **Combined monitoring**: Use both thresholds for comprehensive monitoring
|
|
2334
|
-
- **Execution plan analysis**: Set `showPlan: true` to include detailed execution plans in logs (useful for debugging)
|
|
2335
|
-
- **Cache debugging**: Set `logCache: true` to log cache operations and debug caching issues
|
|
2336
|
-
|
|
2337
|
-
**Memory Threshold Guidelines:**
|
|
2338
|
-
- **Conservative**: 4MB (25% of 16MB limit)
|
|
2339
|
-
- **Default**: 8MB (50% of 16MB limit)
|
|
2340
|
-
- **Aggressive**: 12MB (75% of 16MB limit)
|
|
2341
|
-
- **Critical**: 14MB (87.5% of 16MB limit)
|
|
2342
|
-
|
|
2343
|
-
#### Available Intervals
|
|
2344
|
-
|
|
2345
|
-
- `hour` - **Every hour (Required)** - Statistics are available for approximately 12 hours
|
|
2346
|
-
|
|
2347
|
-
> **⚠️ Important**: Due to Forge SQL's statistics retention policy (approximately 12 hours), **only `hour` interval is supported**. Using `day` or `week` intervals will result in incomplete or missing data.
|
|
2348
2285
|
|
|
2349
2286
|
## Migration Guide
|
|
2350
2287
|
|
|
@@ -2439,14 +2376,25 @@ const cachedRawUsers = await forgeSQL.executeCacheable(
|
|
|
2439
2376
|
300
|
|
2440
2377
|
);
|
|
2441
2378
|
|
|
2442
|
-
// ✅ Raw SQL execution with metadata capture
|
|
2379
|
+
// ✅ Raw SQL execution with metadata capture and performance monitoring
|
|
2443
2380
|
const usersWithMetadata = await forgeSQL.executeWithMetadata(
|
|
2444
|
-
|
|
2445
|
-
|
|
2446
|
-
|
|
2447
|
-
|
|
2448
|
-
|
|
2381
|
+
async () => {
|
|
2382
|
+
const users = await forgeSQL.selectFrom(usersTable);
|
|
2383
|
+
const orders = await forgeSQL.selectFrom(ordersTable).where(eq(ordersTable.userId, usersTable.id));
|
|
2384
|
+
return { users, orders };
|
|
2385
|
+
},
|
|
2386
|
+
(totalDbExecutionTime, totalResponseSize, printQueriesWithPlan) => {
|
|
2387
|
+
const threshold = 500; // ms baseline for this resolver
|
|
2388
|
+
|
|
2389
|
+
if (totalDbExecutionTime > threshold * 1.5) {
|
|
2390
|
+
console.warn(`[Performance Warning] Resolver exceeded DB time: ${totalDbExecutionTime} ms`);
|
|
2391
|
+
await printQueriesWithPlan(); // Analyze and print query execution plans
|
|
2392
|
+
} else if (totalDbExecutionTime > threshold) {
|
|
2393
|
+
console.debug(`[Performance Debug] High DB time: ${totalDbExecutionTime} ms`);
|
|
2449
2394
|
}
|
|
2395
|
+
|
|
2396
|
+
console.log(`DB response size: ${totalResponseSize} bytes`);
|
|
2397
|
+
}
|
|
2450
2398
|
);
|
|
2451
2399
|
|
|
2452
2400
|
// ✅ DDL operations for schema modifications
|