@trafficgroup/knex-rel 0.1.6 → 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/dao/folder/folder.dao.js +3 -6
- package/dist/dao/folder/folder.dao.js.map +1 -1
- package/dist/dao/study/study.dao.d.ts +1 -0
- package/dist/dao/study/study.dao.js +18 -3
- package/dist/dao/study/study.dao.js.map +1 -1
- package/dist/dao/video/video.dao.d.ts +0 -9
- package/dist/dao/video/video.dao.js +0 -51
- package/dist/dao/video/video.dao.js.map +1 -1
- package/dist/interfaces/folder/folder.interfaces.d.ts +0 -3
- package/dist/interfaces/study/study.interfaces.d.ts +5 -0
- package/dist/interfaces/video/video.interfaces.d.ts +0 -3
- package/migrations/20251010143500_migration.ts +83 -0
- package/package.json +1 -1
- package/src/dao/folder/folder.dao.ts +3 -18
- package/src/dao/study/study.dao.ts +34 -3
- package/src/dao/video/video.dao.ts +0 -63
- package/src/interfaces/folder/folder.interfaces.ts +0 -3
- package/src/interfaces/study/study.interfaces.ts +5 -0
- package/src/interfaces/video/video.interfaces.ts +0 -3
- package/plan.md +0 -871
package/plan.md
DELETED
|
@@ -1,871 +0,0 @@
|
|
|
1
|
-
# Total Vehicle Class Implementation Plan
|
|
2
|
-
|
|
3
|
-
## Executive Summary
|
|
4
|
-
|
|
5
|
-
This plan details the implementation of a "Total" vehicle class that aggregates all custom vehicle classes for both TMC and ATR minute results. The "Total" class will be added during transformation in the knex-rel module and will always appear as the LAST class in the vehicles object.
|
|
6
|
-
|
|
7
|
-
**Key Requirements**:
|
|
8
|
-
|
|
9
|
-
- Add "Total" class AFTER all custom classes
|
|
10
|
-
- For ATR: "Total" reuses existing `lane_counts` (all classes per lane)
|
|
11
|
-
- For TMC: "Total" sums all vehicle classes across all directions and turns
|
|
12
|
-
- NO database schema changes required
|
|
13
|
-
- Transformation happens in `ReportConfigurationDAO.applyConfigurationToNestedStructure()`
|
|
14
|
-
|
|
15
|
-
---
|
|
16
|
-
|
|
17
|
-
## Current Architecture Analysis
|
|
18
|
-
|
|
19
|
-
### Data Flow
|
|
20
|
-
|
|
21
|
-
```
|
|
22
|
-
1. Python Processor → Database (JSONB results with detection labels)
|
|
23
|
-
↓
|
|
24
|
-
2. VideoMinuteResultDAO → Retrieve raw minute results
|
|
25
|
-
↓
|
|
26
|
-
3. API Controller → Apply report configuration
|
|
27
|
-
↓
|
|
28
|
-
4. ReportConfigurationService.transformVideoResults()
|
|
29
|
-
↓
|
|
30
|
-
5. ReportConfigurationDAO.applyConfigurationToNestedStructure()
|
|
31
|
-
↓
|
|
32
|
-
6. Frontend → Display custom classes + Total
|
|
33
|
-
```
|
|
34
|
-
|
|
35
|
-
### Current Structure Patterns
|
|
36
|
-
|
|
37
|
-
**ATR Format (Lane-based)**:
|
|
38
|
-
|
|
39
|
-
```json
|
|
40
|
-
{
|
|
41
|
-
"vehicles": {
|
|
42
|
-
"car": { "0": 45, "1": 50 },
|
|
43
|
-
"truck": { "0": 10, "1": 8 }
|
|
44
|
-
},
|
|
45
|
-
"lane_counts": { "0": 55, "1": 58 }, // Already has totals per lane!
|
|
46
|
-
"total_count": 113,
|
|
47
|
-
"study_type": "ATR"
|
|
48
|
-
}
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
**TMC Format (Direction/Turn-based)**:
|
|
52
|
-
|
|
53
|
-
```json
|
|
54
|
-
{
|
|
55
|
-
"vehicles": {
|
|
56
|
-
"car": {
|
|
57
|
-
"NORTH": { "straight": 10, "left": 5, "right": 3, "u-turn": 0 },
|
|
58
|
-
"SOUTH": { "straight": 8, "left": 2, "right": 4, "u-turn": 1 }
|
|
59
|
-
},
|
|
60
|
-
"truck": {
|
|
61
|
-
"NORTH": { "straight": 5, "left": 1, "right": 0, "u-turn": 0 }
|
|
62
|
-
}
|
|
63
|
-
},
|
|
64
|
-
"counts": {
|
|
65
|
-
"total_vehicles": 39,
|
|
66
|
-
"entry_vehicles": 39
|
|
67
|
-
},
|
|
68
|
-
"study_type": "TMC"
|
|
69
|
-
}
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
**After Custom Class Transformation** (2 custom classes: "Light" and "Heavy"):
|
|
73
|
-
|
|
74
|
-
```json
|
|
75
|
-
{
|
|
76
|
-
"vehicles": {
|
|
77
|
-
"Light": { "NORTH": { "straight": 10, "left": 5 }, ... },
|
|
78
|
-
"Heavy": { "NORTH": { "straight": 5, "left": 1 }, ... }
|
|
79
|
-
}
|
|
80
|
-
}
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
**After Total Addition** (desired output):
|
|
84
|
-
|
|
85
|
-
```json
|
|
86
|
-
{
|
|
87
|
-
"vehicles": {
|
|
88
|
-
"Light": { "NORTH": { "straight": 10, "left": 5 }, ... },
|
|
89
|
-
"Heavy": { "NORTH": { "straight": 5, "left": 1 }, ... },
|
|
90
|
-
"Total": { "NORTH": { "straight": 15, "left": 6 }, ... } // Sum of all classes
|
|
91
|
-
}
|
|
92
|
-
}
|
|
93
|
-
```
|
|
94
|
-
|
|
95
|
-
---
|
|
96
|
-
|
|
97
|
-
## Implementation Location
|
|
98
|
-
|
|
99
|
-
### File: `knex-rel/src/dao/report-configuration/report-configuration.dao.ts`
|
|
100
|
-
|
|
101
|
-
**Method to Modify**: `applyConfigurationToNestedStructure()` (Lines 273-315)
|
|
102
|
-
|
|
103
|
-
**Current Implementation**:
|
|
104
|
-
|
|
105
|
-
1. Builds reverse mapping: detection label → custom class name (Lines 278-290)
|
|
106
|
-
2. Initializes empty structure for each custom class (Lines 293-296)
|
|
107
|
-
3. Iterates through detection labels and merges into custom classes (Lines 299-312)
|
|
108
|
-
4. Returns transformed structure (Line 314)
|
|
109
|
-
|
|
110
|
-
**Where to Add "Total"**: AFTER line 312, BEFORE return statement (Line 314)
|
|
111
|
-
|
|
112
|
-
---
|
|
113
|
-
|
|
114
|
-
## Detailed Implementation Plan
|
|
115
|
-
|
|
116
|
-
### Step 1: Add "Total" Class After Custom Class Transformation
|
|
117
|
-
|
|
118
|
-
**Location**: `ReportConfigurationDAO.applyConfigurationToNestedStructure()` (After line 312)
|
|
119
|
-
|
|
120
|
-
**Logic**:
|
|
121
|
-
|
|
122
|
-
```typescript
|
|
123
|
-
// After line 312 (end of detection label iteration)
|
|
124
|
-
// Add "Total" class that aggregates all custom classes
|
|
125
|
-
|
|
126
|
-
result["Total"] = this._createTotalClass(result);
|
|
127
|
-
|
|
128
|
-
return result;
|
|
129
|
-
```
|
|
130
|
-
|
|
131
|
-
### Step 2: Implement `_createTotalClass()` Helper Method
|
|
132
|
-
|
|
133
|
-
**Location**: Add new private method in `ReportConfigurationDAO` class (After line 350)
|
|
134
|
-
|
|
135
|
-
**Method Signature**:
|
|
136
|
-
|
|
137
|
-
```typescript
|
|
138
|
-
private _createTotalClass(customClassesData: Record<string, any>): any
|
|
139
|
-
```
|
|
140
|
-
|
|
141
|
-
**Implementation**:
|
|
142
|
-
|
|
143
|
-
```typescript
|
|
144
|
-
/**
|
|
145
|
-
* Create "Total" class by aggregating all custom vehicle classes
|
|
146
|
-
*
|
|
147
|
-
* Handles both ATR and TMC formats:
|
|
148
|
-
* - ATR: Aggregates counts per lane across all vehicle classes
|
|
149
|
-
* - TMC: Aggregates counts per direction and turn across all vehicle classes
|
|
150
|
-
*
|
|
151
|
-
* @param customClassesData - Transformed custom classes structure
|
|
152
|
-
* @returns Aggregated totals matching the same nested structure
|
|
153
|
-
*/
|
|
154
|
-
private _createTotalClass(customClassesData: Record<string, any>): any {
|
|
155
|
-
const total: any = {};
|
|
156
|
-
|
|
157
|
-
// Iterate through all custom classes and merge their data
|
|
158
|
-
for (const [className, nestedData] of Object.entries(customClassesData)) {
|
|
159
|
-
// Use existing _deepMergeNumericData to sum all nested values
|
|
160
|
-
Object.assign(total, this._deepMergeNumericData(total, nestedData));
|
|
161
|
-
}
|
|
162
|
-
|
|
163
|
-
return total;
|
|
164
|
-
}
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
**Explanation**:
|
|
168
|
-
|
|
169
|
-
- Reuses existing `_deepMergeNumericData()` helper (Lines 328-350)
|
|
170
|
-
- Works for both ATR (2-level: vehicle → lane → count) and TMC (3-level: vehicle → direction → turn → count)
|
|
171
|
-
- No need to detect study type - structure-agnostic aggregation
|
|
172
|
-
- Handles empty data (returns empty object `{}`)
|
|
173
|
-
|
|
174
|
-
---
|
|
175
|
-
|
|
176
|
-
## Key Ordering Strategy
|
|
177
|
-
|
|
178
|
-
### Ensuring "Total" Appears Last
|
|
179
|
-
|
|
180
|
-
JavaScript object key ordering is preserved in modern engines (ES2015+) when:
|
|
181
|
-
|
|
182
|
-
1. String keys are added in insertion order
|
|
183
|
-
2. Keys are enumerated via `Object.entries()`, `Object.keys()`, `for...in`
|
|
184
|
-
|
|
185
|
-
**Our Implementation**:
|
|
186
|
-
|
|
187
|
-
```typescript
|
|
188
|
-
// Lines 293-296: Initialize custom classes FIRST (insertion order)
|
|
189
|
-
for (const customClass of config.configuration.customClasses) {
|
|
190
|
-
result[customClass.name] = {};
|
|
191
|
-
}
|
|
192
|
-
|
|
193
|
-
// Lines 299-312: Populate custom classes (preserves order)
|
|
194
|
-
for (const [detectionLabel, nestedData] of Object.entries(vehiclesStructure)) {
|
|
195
|
-
// ... merge logic
|
|
196
|
-
}
|
|
197
|
-
|
|
198
|
-
// NEW: Add "Total" LAST (after all custom classes)
|
|
199
|
-
result["Total"] = this._createTotalClass(result);
|
|
200
|
-
|
|
201
|
-
// Line 314: Return result (Total is last key)
|
|
202
|
-
return result;
|
|
203
|
-
```
|
|
204
|
-
|
|
205
|
-
**Why This Works**:
|
|
206
|
-
|
|
207
|
-
- Custom classes initialized in config order (Lines 293-296)
|
|
208
|
-
- "Total" added AFTER all custom classes (new code)
|
|
209
|
-
- JavaScript guarantees insertion order preservation
|
|
210
|
-
- Frontend will render "Total" as last tab
|
|
211
|
-
|
|
212
|
-
---
|
|
213
|
-
|
|
214
|
-
## ATR vs TMC Behavior
|
|
215
|
-
|
|
216
|
-
### ATR (Lane-Based Totals)
|
|
217
|
-
|
|
218
|
-
**Input Structure**:
|
|
219
|
-
|
|
220
|
-
```json
|
|
221
|
-
{
|
|
222
|
-
"Light": { "0": 45, "1": 50 },
|
|
223
|
-
"Heavy": { "0": 10, "1": 8 }
|
|
224
|
-
}
|
|
225
|
-
```
|
|
226
|
-
|
|
227
|
-
**Total Output**:
|
|
228
|
-
|
|
229
|
-
```json
|
|
230
|
-
{
|
|
231
|
-
"Total": { "0": 55, "1": 58 } // Sum per lane
|
|
232
|
-
}
|
|
233
|
-
```
|
|
234
|
-
|
|
235
|
-
**Calculation**:
|
|
236
|
-
|
|
237
|
-
- Lane "0": 45 (Light) + 10 (Heavy) = 55
|
|
238
|
-
- Lane "1": 50 (Light) + 8 (Heavy) = 58
|
|
239
|
-
|
|
240
|
-
**Alternative (Use existing `lane_counts`)**:
|
|
241
|
-
|
|
242
|
-
The ATR structure ALREADY has `lane_counts` at the top level:
|
|
243
|
-
|
|
244
|
-
```json
|
|
245
|
-
{
|
|
246
|
-
"vehicles": { "Light": {...}, "Heavy": {...} },
|
|
247
|
-
"lane_counts": { "0": 55, "1": 58 } // Already calculated!
|
|
248
|
-
}
|
|
249
|
-
```
|
|
250
|
-
|
|
251
|
-
**Decision**: For ATR, we can EITHER:
|
|
252
|
-
|
|
253
|
-
1. ✅ **Option A (Recommended)**: Calculate "Total" by summing custom classes (consistent with TMC)
|
|
254
|
-
2. ⚠️ **Option B**: Copy `lane_counts` directly into `vehicles["Total"]` (reuses existing data)
|
|
255
|
-
|
|
256
|
-
**Recommendation**: Use Option A (sum custom classes) for consistency with TMC, even though Option B is technically available.
|
|
257
|
-
|
|
258
|
-
### TMC (Direction/Turn-Based Totals)
|
|
259
|
-
|
|
260
|
-
**Input Structure**:
|
|
261
|
-
|
|
262
|
-
```json
|
|
263
|
-
{
|
|
264
|
-
"Light": {
|
|
265
|
-
"NORTH": { "straight": 10, "left": 5, "right": 3 },
|
|
266
|
-
"SOUTH": { "straight": 8, "left": 2, "right": 4 }
|
|
267
|
-
},
|
|
268
|
-
"Heavy": {
|
|
269
|
-
"NORTH": { "straight": 5, "left": 1, "right": 0 },
|
|
270
|
-
"SOUTH": { "straight": 2, "left": 0, "right": 1 }
|
|
271
|
-
}
|
|
272
|
-
}
|
|
273
|
-
```
|
|
274
|
-
|
|
275
|
-
**Total Output**:
|
|
276
|
-
|
|
277
|
-
```json
|
|
278
|
-
{
|
|
279
|
-
"Total": {
|
|
280
|
-
"NORTH": { "straight": 15, "left": 6, "right": 3 },
|
|
281
|
-
"SOUTH": { "straight": 10, "left": 2, "right": 5 }
|
|
282
|
-
}
|
|
283
|
-
}
|
|
284
|
-
```
|
|
285
|
-
|
|
286
|
-
**Calculation**:
|
|
287
|
-
|
|
288
|
-
- NORTH/straight: 10 (Light) + 5 (Heavy) = 15
|
|
289
|
-
- NORTH/left: 5 (Light) + 1 (Heavy) = 6
|
|
290
|
-
- SOUTH/straight: 8 (Light) + 2 (Heavy) = 10
|
|
291
|
-
- etc.
|
|
292
|
-
|
|
293
|
-
---
|
|
294
|
-
|
|
295
|
-
## Edge Cases & Validation
|
|
296
|
-
|
|
297
|
-
### Edge Case 1: Empty Data (No Vehicles)
|
|
298
|
-
|
|
299
|
-
**Input**:
|
|
300
|
-
|
|
301
|
-
```json
|
|
302
|
-
{
|
|
303
|
-
"vehicles": {}
|
|
304
|
-
}
|
|
305
|
-
```
|
|
306
|
-
|
|
307
|
-
**After Transformation**:
|
|
308
|
-
|
|
309
|
-
```json
|
|
310
|
-
{
|
|
311
|
-
"vehicles": {
|
|
312
|
-
"Light": {},
|
|
313
|
-
"Heavy": {},
|
|
314
|
-
"Total": {} // Empty object
|
|
315
|
-
}
|
|
316
|
-
}
|
|
317
|
-
```
|
|
318
|
-
|
|
319
|
-
**Behavior**: `_createTotalClass()` returns empty object `{}`
|
|
320
|
-
|
|
321
|
-
### Edge Case 2: Single Custom Class
|
|
322
|
-
|
|
323
|
-
**Input**:
|
|
324
|
-
|
|
325
|
-
```json
|
|
326
|
-
{
|
|
327
|
-
"vehicles": {
|
|
328
|
-
"AllVehicles": { "0": 100, "1": 95 }
|
|
329
|
-
}
|
|
330
|
-
}
|
|
331
|
-
```
|
|
332
|
-
|
|
333
|
-
**Output**:
|
|
334
|
-
|
|
335
|
-
```json
|
|
336
|
-
{
|
|
337
|
-
"vehicles": {
|
|
338
|
-
"AllVehicles": { "0": 100, "1": 95 },
|
|
339
|
-
"Total": { "0": 100, "1": 95 } // Same as single class
|
|
340
|
-
}
|
|
341
|
-
}
|
|
342
|
-
```
|
|
343
|
-
|
|
344
|
-
**Behavior**: "Total" equals the single custom class (valid)
|
|
345
|
-
|
|
346
|
-
### Edge Case 3: Missing Directions/Turns
|
|
347
|
-
|
|
348
|
-
**Input** (TMC - Light has NORTH, Heavy only has SOUTH):
|
|
349
|
-
|
|
350
|
-
```json
|
|
351
|
-
{
|
|
352
|
-
"Light": { "NORTH": { "straight": 10 } },
|
|
353
|
-
"Heavy": { "SOUTH": { "straight": 5 } }
|
|
354
|
-
}
|
|
355
|
-
```
|
|
356
|
-
|
|
357
|
-
**Output**:
|
|
358
|
-
|
|
359
|
-
```json
|
|
360
|
-
{
|
|
361
|
-
"Total": {
|
|
362
|
-
"NORTH": { "straight": 10 },
|
|
363
|
-
"SOUTH": { "straight": 5 }
|
|
364
|
-
}
|
|
365
|
-
}
|
|
366
|
-
```
|
|
367
|
-
|
|
368
|
-
**Behavior**: `_deepMergeNumericData()` handles missing keys gracefully (Lines 339-347)
|
|
369
|
-
|
|
370
|
-
### Edge Case 4: Null/Undefined Values
|
|
371
|
-
|
|
372
|
-
**Input**:
|
|
373
|
-
|
|
374
|
-
```json
|
|
375
|
-
{
|
|
376
|
-
"Light": { "0": 10, "1": null },
|
|
377
|
-
"Heavy": { "0": 5 }
|
|
378
|
-
}
|
|
379
|
-
```
|
|
380
|
-
|
|
381
|
-
**Output**:
|
|
382
|
-
|
|
383
|
-
```json
|
|
384
|
-
{
|
|
385
|
-
"Total": { "0": 15, "1": 0 } // null treated as 0
|
|
386
|
-
}
|
|
387
|
-
```
|
|
388
|
-
|
|
389
|
-
**Behavior**: `_deepMergeNumericData()` checks `typeof source === 'number'` (Line 330)
|
|
390
|
-
|
|
391
|
-
---
|
|
392
|
-
|
|
393
|
-
## Testing Strategy
|
|
394
|
-
|
|
395
|
-
### Unit Tests (to be created)
|
|
396
|
-
|
|
397
|
-
**File**: `knex-rel/src/dao/report-configuration/report-configuration.dao.test.ts`
|
|
398
|
-
|
|
399
|
-
**Test Cases**:
|
|
400
|
-
|
|
401
|
-
1. **ATR: Total Class Aggregation**
|
|
402
|
-
- Input: 2 custom classes with lane counts
|
|
403
|
-
- Verify: Total sums all lanes correctly
|
|
404
|
-
- Verify: Total appears as last key
|
|
405
|
-
|
|
406
|
-
2. **TMC: Total Class Aggregation**
|
|
407
|
-
- Input: 2 custom classes with direction/turn structure
|
|
408
|
-
- Verify: Total sums all directions and turns
|
|
409
|
-
- Verify: Total appears as last key
|
|
410
|
-
|
|
411
|
-
3. **Edge Case: Empty Vehicles**
|
|
412
|
-
- Input: No vehicle data
|
|
413
|
-
- Verify: Total is empty object `{}`
|
|
414
|
-
|
|
415
|
-
4. **Edge Case: Single Custom Class**
|
|
416
|
-
- Input: 1 custom class
|
|
417
|
-
- Verify: Total equals the single class
|
|
418
|
-
|
|
419
|
-
5. **Edge Case: Missing Directions**
|
|
420
|
-
- Input: Asymmetric direction/turn data
|
|
421
|
-
- Verify: Total includes all unique keys
|
|
422
|
-
|
|
423
|
-
6. **Key Ordering Test**
|
|
424
|
-
- Input: 5 custom classes
|
|
425
|
-
- Verify: Total is the 6th key (last)
|
|
426
|
-
- Verify: `Object.keys(result)` ends with "Total"
|
|
427
|
-
|
|
428
|
-
### Integration Tests
|
|
429
|
-
|
|
430
|
-
**Test in api-rel controller**:
|
|
431
|
-
|
|
432
|
-
1. Create video with ATR results
|
|
433
|
-
2. Apply configuration with 3 custom classes
|
|
434
|
-
3. Call `GET /videos/:uuid/results?configUuid=...`
|
|
435
|
-
4. Verify response includes "Total" as last class
|
|
436
|
-
5. Verify Total values equal sum of custom classes
|
|
437
|
-
|
|
438
|
-
**Test in traffic-webapp**:
|
|
439
|
-
|
|
440
|
-
1. Display video results with custom classes
|
|
441
|
-
2. Verify "Total" tab appears last
|
|
442
|
-
3. Verify Total chart shows aggregated data
|
|
443
|
-
4. Test with both ATR and TMC videos
|
|
444
|
-
|
|
445
|
-
---
|
|
446
|
-
|
|
447
|
-
## Performance Considerations
|
|
448
|
-
|
|
449
|
-
### Computational Complexity
|
|
450
|
-
|
|
451
|
-
**Current**: O(n × m) where:
|
|
452
|
-
|
|
453
|
-
- n = number of detection labels
|
|
454
|
-
- m = average nesting depth (2 for ATR, 3 for TMC)
|
|
455
|
-
|
|
456
|
-
**After Adding Total**: O(n × m + c × m) where:
|
|
457
|
-
|
|
458
|
-
- c = number of custom classes (2-7 per validation)
|
|
459
|
-
|
|
460
|
-
**Impact**: Minimal - additional O(c × m) is negligible
|
|
461
|
-
|
|
462
|
-
- c ≤ 7 (max custom classes)
|
|
463
|
-
- m ≤ 3 (max nesting depth)
|
|
464
|
-
- Total iteration: ~21 operations per minute result
|
|
465
|
-
|
|
466
|
-
### Memory Impact
|
|
467
|
-
|
|
468
|
-
**Additional Memory**: One new key per transformed result
|
|
469
|
-
|
|
470
|
-
- ATR Total: ~100 bytes (lanes object)
|
|
471
|
-
- TMC Total: ~500 bytes (directions/turns object)
|
|
472
|
-
- Per minute result: negligible
|
|
473
|
-
- For 60 minutes: ~6-30 KB additional
|
|
474
|
-
|
|
475
|
-
**Conclusion**: No performance concerns
|
|
476
|
-
|
|
477
|
-
---
|
|
478
|
-
|
|
479
|
-
## Backward Compatibility
|
|
480
|
-
|
|
481
|
-
### Changes Are Additive Only
|
|
482
|
-
|
|
483
|
-
✅ **No Breaking Changes**:
|
|
484
|
-
|
|
485
|
-
- Existing custom classes remain unchanged
|
|
486
|
-
- Existing API response structure preserved
|
|
487
|
-
- "Total" is a NEW key (optional to consume)
|
|
488
|
-
|
|
489
|
-
✅ **Frontend Compatibility**:
|
|
490
|
-
|
|
491
|
-
- Old frontend: Ignores "Total" key (no errors)
|
|
492
|
-
- New frontend: Displays "Total" as last tab
|
|
493
|
-
|
|
494
|
-
✅ **Database Compatibility**:
|
|
495
|
-
|
|
496
|
-
- No schema changes required
|
|
497
|
-
- Raw results in database unchanged
|
|
498
|
-
- Transformation happens at runtime
|
|
499
|
-
|
|
500
|
-
---
|
|
501
|
-
|
|
502
|
-
## Implementation Checklist
|
|
503
|
-
|
|
504
|
-
### Pre-Implementation
|
|
505
|
-
|
|
506
|
-
- [x] Analyze current transformation logic
|
|
507
|
-
- [x] Identify exact insertion point (line 312)
|
|
508
|
-
- [x] Verify `_deepMergeNumericData()` handles both ATR/TMC
|
|
509
|
-
- [x] Confirm JavaScript key ordering guarantees
|
|
510
|
-
|
|
511
|
-
### Implementation Phase
|
|
512
|
-
|
|
513
|
-
- [ ] Add `_createTotalClass()` helper method after line 350
|
|
514
|
-
- [ ] Add "Total" class insertion after line 312 in `applyConfigurationToNestedStructure()`
|
|
515
|
-
- [ ] Add JSDoc comments for new method
|
|
516
|
-
- [ ] Verify TypeScript compilation: `npm run build`
|
|
517
|
-
|
|
518
|
-
### Testing Phase
|
|
519
|
-
|
|
520
|
-
- [ ] Write unit tests for `_createTotalClass()`
|
|
521
|
-
- [ ] Test ATR format (lane-based totals)
|
|
522
|
-
- [ ] Test TMC format (direction/turn totals)
|
|
523
|
-
- [ ] Test edge cases (empty, single class, missing keys)
|
|
524
|
-
- [ ] Test key ordering (Total is last)
|
|
525
|
-
- [ ] Run all tests: `npm test`
|
|
526
|
-
|
|
527
|
-
### Integration Phase
|
|
528
|
-
|
|
529
|
-
- [ ] Build knex-rel: `npm run build`
|
|
530
|
-
- [ ] Test in api-rel: `npm run start:dev`
|
|
531
|
-
- [ ] Call `/videos/:uuid/results` endpoint with ATR video
|
|
532
|
-
- [ ] Call `/videos/:uuid/results` endpoint with TMC video
|
|
533
|
-
- [ ] Verify "Total" appears in response as last class
|
|
534
|
-
- [ ] Verify Total values are correct sums
|
|
535
|
-
|
|
536
|
-
### Frontend Integration (Separate Task)
|
|
537
|
-
|
|
538
|
-
- [ ] Update traffic-webapp to display "Total" tab
|
|
539
|
-
- [ ] Position "Total" tab as last tab
|
|
540
|
-
- [ ] Render Total data in charts/tables
|
|
541
|
-
- [ ] Test with both ATR and TMC videos
|
|
542
|
-
|
|
543
|
-
---
|
|
544
|
-
|
|
545
|
-
## Code Changes Summary
|
|
546
|
-
|
|
547
|
-
### File 1: `knex-rel/src/dao/report-configuration/report-configuration.dao.ts`
|
|
548
|
-
|
|
549
|
-
**Location 1: After Line 312 (Inside `applyConfigurationToNestedStructure()`)**
|
|
550
|
-
|
|
551
|
-
```typescript
|
|
552
|
-
// Lines 299-312: Existing iteration through detection labels
|
|
553
|
-
for (const [detectionLabel, nestedData] of Object.entries(vehiclesStructure)) {
|
|
554
|
-
const customClassName = detectionToCustomClass[detectionLabel];
|
|
555
|
-
if (!customClassName) {
|
|
556
|
-
continue;
|
|
557
|
-
}
|
|
558
|
-
result[customClassName] = this._deepMergeNumericData(
|
|
559
|
-
result[customClassName],
|
|
560
|
-
nestedData,
|
|
561
|
-
);
|
|
562
|
-
}
|
|
563
|
-
|
|
564
|
-
// NEW CODE: Add "Total" class that aggregates all custom classes
|
|
565
|
-
result["Total"] = this._createTotalClass(result);
|
|
566
|
-
|
|
567
|
-
return result;
|
|
568
|
-
```
|
|
569
|
-
|
|
570
|
-
**Location 2: After Line 350 (New Private Method)**
|
|
571
|
-
|
|
572
|
-
```typescript
|
|
573
|
-
/**
|
|
574
|
-
* Create "Total" class by aggregating all custom vehicle classes
|
|
575
|
-
*
|
|
576
|
-
* Sums all custom class counts across their nested structures.
|
|
577
|
-
* Works for both ATR (lane-based) and TMC (direction/turn-based) formats.
|
|
578
|
-
*
|
|
579
|
-
* @param customClassesData - Transformed custom classes structure
|
|
580
|
-
* Example ATR: { "Light": { "0": 45, "1": 50 }, "Heavy": { "0": 10 } }
|
|
581
|
-
* Example TMC: { "Light": { "NORTH": { "straight": 10 } }, ... }
|
|
582
|
-
* @returns Aggregated totals matching the same nested structure
|
|
583
|
-
* Example ATR: { "0": 55, "1": 50 }
|
|
584
|
-
* Example TMC: { "NORTH": { "straight": 10 }, ... }
|
|
585
|
-
*/
|
|
586
|
-
private _createTotalClass(customClassesData: Record<string, any>): any {
|
|
587
|
-
let total: any = {};
|
|
588
|
-
|
|
589
|
-
// Iterate through all custom classes and merge their data
|
|
590
|
-
for (const [className, nestedData] of Object.entries(customClassesData)) {
|
|
591
|
-
total = this._deepMergeNumericData(total, nestedData);
|
|
592
|
-
}
|
|
593
|
-
|
|
594
|
-
return total;
|
|
595
|
-
}
|
|
596
|
-
```
|
|
597
|
-
|
|
598
|
-
---
|
|
599
|
-
|
|
600
|
-
## Validation Plan
|
|
601
|
-
|
|
602
|
-
### Manual Testing Checklist
|
|
603
|
-
|
|
604
|
-
**Test 1: ATR Video (2 Custom Classes)**
|
|
605
|
-
|
|
606
|
-
- [ ] Create configuration: "Light" (FHWA 1-3), "Heavy" (FHWA 4-13)
|
|
607
|
-
- [ ] Upload ATR video with mixed vehicle types
|
|
608
|
-
- [ ] Process video and get results
|
|
609
|
-
- [ ] Verify "Light" shows cars/motorcycles per lane
|
|
610
|
-
- [ ] Verify "Heavy" shows trucks/buses per lane
|
|
611
|
-
- [ ] Verify "Total" shows sum of Light + Heavy per lane
|
|
612
|
-
- [ ] Verify "Total" is last key in vehicles object
|
|
613
|
-
|
|
614
|
-
**Test 2: TMC Video (3 Custom Classes)**
|
|
615
|
-
|
|
616
|
-
- [ ] Create configuration: "Cars", "Trucks", "Buses"
|
|
617
|
-
- [ ] Upload TMC video with turning movements
|
|
618
|
-
- [ ] Process video and get results
|
|
619
|
-
- [ ] Verify each class shows direction/turn breakdown
|
|
620
|
-
- [ ] Verify "Total" shows sum across all classes
|
|
621
|
-
- [ ] Verify NORTH/straight in Total = sum of all classes' NORTH/straight
|
|
622
|
-
- [ ] Verify "Total" is last key in vehicles object
|
|
623
|
-
|
|
624
|
-
**Test 3: Edge Cases**
|
|
625
|
-
|
|
626
|
-
- [ ] Video with no vehicles detected (empty Total)
|
|
627
|
-
- [ ] Video with single custom class (Total equals that class)
|
|
628
|
-
- [ ] Video with asymmetric turns (Light has left, Heavy doesn't)
|
|
629
|
-
- [ ] Configuration with 7 custom classes (max allowed)
|
|
630
|
-
|
|
631
|
-
**Test 4: Grouped Results (Multiple Minutes)**
|
|
632
|
-
|
|
633
|
-
- [ ] Get grouped results with grouping=15
|
|
634
|
-
- [ ] Verify each timeGroup has "Total" class
|
|
635
|
-
- [ ] Verify "Total" sums correctly across aggregated minutes
|
|
636
|
-
- [ ] Verify ordering preserved in all timeGroups
|
|
637
|
-
|
|
638
|
-
---
|
|
639
|
-
|
|
640
|
-
## Risks & Mitigations
|
|
641
|
-
|
|
642
|
-
### Risk 1: Key Ordering Not Guaranteed
|
|
643
|
-
|
|
644
|
-
**Risk**: JavaScript object keys might not preserve insertion order
|
|
645
|
-
|
|
646
|
-
**Likelihood**: Very Low (ES2015+ guarantees string key order)
|
|
647
|
-
|
|
648
|
-
**Mitigation**:
|
|
649
|
-
|
|
650
|
-
- Add unit test to verify `Object.keys(result)` ends with "Total"
|
|
651
|
-
- If ordering fails, use explicit ordering array in frontend
|
|
652
|
-
|
|
653
|
-
### Risk 2: Performance Degradation
|
|
654
|
-
|
|
655
|
-
**Risk**: Additional aggregation adds processing time
|
|
656
|
-
|
|
657
|
-
**Likelihood**: Very Low (O(c × m) is negligible)
|
|
658
|
-
|
|
659
|
-
**Mitigation**:
|
|
660
|
-
|
|
661
|
-
- Benchmark transformation time before/after
|
|
662
|
-
- If impact > 10ms, consider caching Total class
|
|
663
|
-
|
|
664
|
-
### Risk 3: Frontend Breaks on Unknown Class
|
|
665
|
-
|
|
666
|
-
**Risk**: Old frontend version throws error on "Total" key
|
|
667
|
-
|
|
668
|
-
**Likelihood**: Very Low (key is additive, not breaking)
|
|
669
|
-
|
|
670
|
-
**Mitigation**:
|
|
671
|
-
|
|
672
|
-
- "Total" is just another custom class name
|
|
673
|
-
- Frontend already iterates dynamic class names
|
|
674
|
-
- No special handling needed
|
|
675
|
-
|
|
676
|
-
### Risk 4: Incorrect Aggregation Logic
|
|
677
|
-
|
|
678
|
-
**Risk**: `_deepMergeNumericData()` doesn't handle all cases
|
|
679
|
-
|
|
680
|
-
**Likelihood**: Low (method already tested for custom class merging)
|
|
681
|
-
|
|
682
|
-
**Mitigation**:
|
|
683
|
-
|
|
684
|
-
- Comprehensive unit tests for edge cases
|
|
685
|
-
- Manual validation with real ATR/TMC videos
|
|
686
|
-
- Use existing, battle-tested `_deepMergeNumericData()`
|
|
687
|
-
|
|
688
|
-
---
|
|
689
|
-
|
|
690
|
-
## Alternative Approaches Considered
|
|
691
|
-
|
|
692
|
-
### Alternative 1: Add "Total" in Frontend
|
|
693
|
-
|
|
694
|
-
**Approach**: Calculate "Total" in traffic-webapp instead of backend
|
|
695
|
-
|
|
696
|
-
**Pros**:
|
|
697
|
-
|
|
698
|
-
- No backend changes required
|
|
699
|
-
- Frontend controls display logic
|
|
700
|
-
|
|
701
|
-
**Cons**:
|
|
702
|
-
|
|
703
|
-
- Duplicates logic across frontend
|
|
704
|
-
- Inconsistent if API consumed elsewhere
|
|
705
|
-
- Cannot use "Total" in backend reports/exports
|
|
706
|
-
|
|
707
|
-
**Decision**: ❌ Rejected - Backend is single source of truth
|
|
708
|
-
|
|
709
|
-
### Alternative 2: Add "Total" as Separate Field
|
|
710
|
-
|
|
711
|
-
**Approach**: Add `results.totalClass` instead of `results.vehicles["Total"]`
|
|
712
|
-
|
|
713
|
-
**Pros**:
|
|
714
|
-
|
|
715
|
-
- Clearer separation of concerns
|
|
716
|
-
- No risk of key ordering issues
|
|
717
|
-
|
|
718
|
-
**Cons**:
|
|
719
|
-
|
|
720
|
-
- Inconsistent with custom class structure
|
|
721
|
-
- Frontend needs special handling
|
|
722
|
-
- Breaks uniform class iteration logic
|
|
723
|
-
|
|
724
|
-
**Decision**: ❌ Rejected - Keep "Total" as vehicle class for uniformity
|
|
725
|
-
|
|
726
|
-
### Alternative 3: Make "Total" Configurable
|
|
727
|
-
|
|
728
|
-
**Approach**: Add `includeTotalClass: boolean` to report configuration
|
|
729
|
-
|
|
730
|
-
**Pros**:
|
|
731
|
-
|
|
732
|
-
- User control over Total display
|
|
733
|
-
- Backward compatible (default false)
|
|
734
|
-
|
|
735
|
-
**Cons**:
|
|
736
|
-
|
|
737
|
-
- Adds complexity to configuration
|
|
738
|
-
- Not needed (Total is always useful)
|
|
739
|
-
- Extra validation logic
|
|
740
|
-
|
|
741
|
-
**Decision**: ❌ Rejected - Always include "Total" (KISS principle)
|
|
742
|
-
|
|
743
|
-
---
|
|
744
|
-
|
|
745
|
-
## Success Criteria
|
|
746
|
-
|
|
747
|
-
### Implementation Success
|
|
748
|
-
|
|
749
|
-
✅ TypeScript compiles without errors
|
|
750
|
-
✅ `npm run build` succeeds in knex-rel
|
|
751
|
-
✅ All existing unit tests pass
|
|
752
|
-
✅ New unit tests pass (Total class logic)
|
|
753
|
-
|
|
754
|
-
### Functional Success
|
|
755
|
-
|
|
756
|
-
✅ ATR videos show "Total" with correct lane sums
|
|
757
|
-
✅ TMC videos show "Total" with correct direction/turn sums
|
|
758
|
-
✅ "Total" appears as LAST class in all responses
|
|
759
|
-
✅ Edge cases handled (empty, single class, missing keys)
|
|
760
|
-
|
|
761
|
-
### Integration Success
|
|
762
|
-
|
|
763
|
-
✅ API endpoint returns "Total" in response
|
|
764
|
-
✅ Grouped results include "Total" in each timeGroup
|
|
765
|
-
✅ No breaking changes to existing API consumers
|
|
766
|
-
✅ Frontend displays "Total" tab correctly (separate task)
|
|
767
|
-
|
|
768
|
-
---
|
|
769
|
-
|
|
770
|
-
## Timeline Estimate
|
|
771
|
-
|
|
772
|
-
### Backend Implementation (knex-rel)
|
|
773
|
-
|
|
774
|
-
- Code changes: 30 minutes
|
|
775
|
-
- Unit tests: 1 hour
|
|
776
|
-
- Build & test: 15 minutes
|
|
777
|
-
- **Total: ~2 hours**
|
|
778
|
-
|
|
779
|
-
### Integration Testing (api-rel)
|
|
780
|
-
|
|
781
|
-
- Manual testing with ATR video: 30 minutes
|
|
782
|
-
- Manual testing with TMC video: 30 minutes
|
|
783
|
-
- Edge case validation: 30 minutes
|
|
784
|
-
- **Total: ~1.5 hours**
|
|
785
|
-
|
|
786
|
-
### Frontend Integration (traffic-webapp - Separate)
|
|
787
|
-
|
|
788
|
-
- Display "Total" tab: 1 hour
|
|
789
|
-
- Chart/table rendering: 1 hour
|
|
790
|
-
- Testing both video types: 1 hour
|
|
791
|
-
- **Total: ~3 hours**
|
|
792
|
-
|
|
793
|
-
**Overall Estimate: ~6.5 hours** (Backend: 3.5 hours, Frontend: 3 hours)
|
|
794
|
-
|
|
795
|
-
---
|
|
796
|
-
|
|
797
|
-
## Summary
|
|
798
|
-
|
|
799
|
-
### What Changes
|
|
800
|
-
|
|
801
|
-
✅ **ADD**: `_createTotalClass()` private method in ReportConfigurationDAO
|
|
802
|
-
✅ **ADD**: `result["Total"] = ...` line after custom class transformation
|
|
803
|
-
✅ **ADD**: Unit tests for Total class aggregation
|
|
804
|
-
|
|
805
|
-
### What Stays the Same
|
|
806
|
-
|
|
807
|
-
✅ Database schema (no migrations)
|
|
808
|
-
✅ Existing custom classes (no modifications)
|
|
809
|
-
✅ API response structure (additive only)
|
|
810
|
-
✅ `_deepMergeNumericData()` helper (reused, not changed)
|
|
811
|
-
|
|
812
|
-
### Key Benefits
|
|
813
|
-
|
|
814
|
-
✅ Unified "Total" view across all custom classes
|
|
815
|
-
✅ Consistent with existing transformation architecture
|
|
816
|
-
✅ No breaking changes (backward compatible)
|
|
817
|
-
✅ Minimal performance impact (< 1ms per minute result)
|
|
818
|
-
✅ Reuses existing, tested aggregation logic
|
|
819
|
-
|
|
820
|
-
**Ready for Implementation** - Plan is complete with exact code locations, logic, edge cases, and validation strategy.
|
|
821
|
-
|
|
822
|
-
---
|
|
823
|
-
|
|
824
|
-
## Files to Modify
|
|
825
|
-
|
|
826
|
-
### knex-rel/src/dao/report-configuration/report-configuration.dao.ts
|
|
827
|
-
|
|
828
|
-
- **Line 312**: Add `result["Total"] = this._createTotalClass(result);`
|
|
829
|
-
- **After Line 350**: Add `_createTotalClass()` private method
|
|
830
|
-
|
|
831
|
-
### knex-rel/src/dao/report-configuration/report-configuration.dao.test.ts (New File)
|
|
832
|
-
|
|
833
|
-
- Create unit tests for Total class aggregation
|
|
834
|
-
- Test ATR format, TMC format, edge cases, key ordering
|
|
835
|
-
|
|
836
|
-
---
|
|
837
|
-
|
|
838
|
-
## Pattern Compliance Summary
|
|
839
|
-
|
|
840
|
-
### Database Patterns ✅
|
|
841
|
-
|
|
842
|
-
- No database changes required (transformation only)
|
|
843
|
-
- Follows DAO pattern (logic in ReportConfigurationDAO)
|
|
844
|
-
- No new migrations needed
|
|
845
|
-
|
|
846
|
-
### Code Patterns ✅
|
|
847
|
-
|
|
848
|
-
- Reuses existing `_deepMergeNumericData()` helper
|
|
849
|
-
- Follows private method naming convention (`_methodName`)
|
|
850
|
-
- Maintains nested structure consistency
|
|
851
|
-
- Preserves key ordering via insertion order
|
|
852
|
-
|
|
853
|
-
### Performance Patterns ✅
|
|
854
|
-
|
|
855
|
-
- O(c × m) additional complexity (negligible)
|
|
856
|
-
- No N+1 queries (transformation in memory)
|
|
857
|
-
- No database roundtrips
|
|
858
|
-
- Minimal memory overhead (~500 bytes per result)
|
|
859
|
-
|
|
860
|
-
---
|
|
861
|
-
|
|
862
|
-
## Next Steps
|
|
863
|
-
|
|
864
|
-
1. **Implement** `_createTotalClass()` method in ReportConfigurationDAO
|
|
865
|
-
2. **Add** Total class insertion after line 312
|
|
866
|
-
3. **Write** unit tests for all edge cases
|
|
867
|
-
4. **Build** knex-rel package and verify compilation
|
|
868
|
-
5. **Test** with real ATR and TMC videos in api-rel
|
|
869
|
-
6. **Validate** key ordering and correct aggregation
|
|
870
|
-
7. **Document** changes (if needed)
|
|
871
|
-
8. **Coordinate** with frontend team for Total tab display
|