@trafficgroup/knex-rel 0.1.4 → 0.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/plan.md CHANGED
@@ -1,1034 +1,871 @@
1
- # Report Configurations Database Implementation Plan
1
+ # Total Vehicle Class Implementation Plan
2
2
 
3
3
  ## Executive Summary
4
4
 
5
- This plan implements a **system-wide** report configuration system that allows users to map FHWA 13-class vehicle classifications to custom groupings for traffic reports. The configuration applies a **3-tier mapping** system: Detection Labels FHWA Classes Custom Classes.
5
+ This plan details the implementation of a "Total" vehicle class that aggregates all custom vehicle classes for both TMC and ATR minute results. The "Total" class will be added during transformation in the knex-rel module and will always appear as the LAST class in the vehicles object.
6
6
 
7
- **Key Decisions from User Clarifications:**
7
+ **Key Requirements**:
8
8
 
9
- - System-wide configurations (NOT user-scoped)
10
- - Configurations selected when viewing video results or study summaries
11
- - Always maintain at least 1 configuration (cannot delete last one)
12
- - Many-to-One ONLY mapping (no duplicates allowed)
13
- - Unmapped FHWA classes excluded from reports
14
- - API-level transformation (frontend receives only transformed data)
15
- - Applies to both TMC and ATR study types
16
- - Minute-level granularity transformation
9
+ - Add "Total" class AFTER all custom classes
10
+ - For ATR: "Total" reuses existing `lane_counts` (all classes per lane)
11
+ - For TMC: "Total" sums all vehicle classes across all directions and turns
12
+ - NO database schema changes required
13
+ - Transformation happens in `ReportConfigurationDAO.applyConfigurationToNestedStructure()`
17
14
 
18
15
  ---
19
16
 
20
- ## Critical Design Challenge: Detection Label → FHWA Class Mapping
17
+ ## Current Architecture Analysis
21
18
 
22
- ### The Problem
23
-
24
- The user described a 3-tier mapping system:
19
+ ### Data Flow
25
20
 
26
21
  ```
27
- Detection LabelsFHWA Classes Custom Classes
22
+ 1. Python Processor Database (JSONB results with detection labels)
23
+
24
+ 2. VideoMinuteResultDAO → Retrieve raw minute results
25
+
26
+ 3. API Controller → Apply report configuration
27
+
28
+ 4. ReportConfigurationService.transformVideoResults()
29
+
30
+ 5. ReportConfigurationDAO.applyConfigurationToNestedStructure()
31
+
32
+ 6. Frontend → Display custom classes + Total
28
33
  ```
29
34
 
30
- However, **I cannot find a hardcoded mapping** from detection labels to FHWA classes in the codebase.
35
+ ### Current Structure Patterns
31
36
 
32
- **Detection Labels Found in System:**
37
+ **ATR Format (Lane-based)**:
33
38
 
34
- - From VideoMinuteResultDAO.normalizeATRVehicleClass() (lines 702-731):
35
- - `car`, `vehicle`, `automobiles` → normalized to `cars`
36
- - `medium`, `pickup`, `truck` → normalized to `mediums`
37
- - `heavy`, `largetruck`, `bigtruck` normalized to `heavy_trucks`
38
- - `pedestrian`, `person`, `people` normalized to `pedestrians`
39
- - `bicycle`, `bike`, `cyclist` → normalized to `bicycles`
39
+ ```json
40
+ {
41
+ "vehicles": {
42
+ "car": { "0": 45, "1": 50 },
43
+ "truck": { "0": 10, "1": 8 }
44
+ },
45
+ "lane_counts": { "0": 55, "1": 58 }, // Already has totals per lane!
46
+ "total_count": 113,
47
+ "study_type": "ATR"
48
+ }
49
+ ```
40
50
 
41
- **FHWA 13-Class System (from user clarifications):**
51
+ **TMC Format (Direction/Turn-based)**:
42
52
 
43
- ```
44
- Class 1: motorcycle
45
- Class 2: car
46
- Class 3: pickup_truck
47
- Class 4: bus
48
- Class 5: work_van
49
- Class 6-8: single_unit_truck
50
- Class 9-13: articulated_truck
53
+ ```json
54
+ {
55
+ "vehicles": {
56
+ "car": {
57
+ "NORTH": { "straight": 10, "left": 5, "right": 3, "u-turn": 0 },
58
+ "SOUTH": { "straight": 8, "left": 2, "right": 4, "u-turn": 1 }
59
+ },
60
+ "truck": {
61
+ "NORTH": { "straight": 5, "left": 1, "right": 0, "u-turn": 0 }
62
+ }
63
+ },
64
+ "counts": {
65
+ "total_vehicles": 39,
66
+ "entry_vehicles": 39
67
+ },
68
+ "study_type": "TMC"
69
+ }
51
70
  ```
52
71
 
53
- ### The Solution: Hardcoded FHWA Mapping in ReportConfigurationDAO
72
+ **After Custom Class Transformation** (2 custom classes: "Light" and "Heavy"):
54
73
 
55
- Since no existing mapping exists, we'll create a **static mapping table** in the ReportConfigurationDAO:
74
+ ```json
75
+ {
76
+ "vehicles": {
77
+ "Light": { "NORTH": { "straight": 10, "left": 5 }, ... },
78
+ "Heavy": { "NORTH": { "straight": 5, "left": 1 }, ... }
79
+ }
80
+ }
81
+ ```
56
82
 
57
- ```typescript
58
- const DETECTION_LABEL_TO_FHWA: Record<string, number[]> = {
59
- motorcycle: [1],
60
- car: [2],
61
- pickup_truck: [3],
62
- bus: [4],
63
- work_van: [5],
64
- single_unit_truck: [6, 7, 8],
65
- articulated_truck: [9, 10, 11, 12, 13],
66
- // Non-motorized (excluded from FHWA system)
67
- pedestrian: [],
68
- bicycle: [],
69
- motorized_vehicle: [2], // Map to car
70
- non_motorized_vehicle: [],
71
- };
83
+ **After Total Addition** (desired output):
84
+
85
+ ```json
86
+ {
87
+ "vehicles": {
88
+ "Light": { "NORTH": { "straight": 10, "left": 5 }, ... },
89
+ "Heavy": { "NORTH": { "straight": 5, "left": 1 }, ... },
90
+ "Total": { "NORTH": { "straight": 15, "left": 6 }, ... } // Sum of all classes
91
+ }
92
+ }
72
93
  ```
73
94
 
74
- **Transformation Flow:**
95
+ ---
96
+
97
+ ## Implementation Location
98
+
99
+ ### File: `knex-rel/src/dao/report-configuration/report-configuration.dao.ts`
75
100
 
76
- 1. Video processing returns detection labels (e.g., `car`, `pickup_truck`)
77
- 2. DAO converts detection labels → FHWA classes using DETECTION_LABEL_TO_FHWA
78
- 3. DAO applies user configuration: FHWA classes → Custom classes
79
- 4. API returns only custom class counts to frontend
101
+ **Method to Modify**: `applyConfigurationToNestedStructure()` (Lines 273-315)
80
102
 
81
- ### Replacement of Existing Normalization
103
+ **Current Implementation**:
82
104
 
83
- The `normalizeATRVehicleClass()` function (VideoMinuteResultDAO lines 702-731) will be **completely replaced**. Instead of normalizing to `cars`, `mediums`, `heavy_trucks`, the new system will:
105
+ 1. Builds reverse mapping: detection label custom class name (Lines 278-290)
106
+ 2. Initializes empty structure for each custom class (Lines 293-296)
107
+ 3. Iterates through detection labels and merges into custom classes (Lines 299-312)
108
+ 4. Returns transformed structure (Line 314)
84
109
 
85
- 1. Keep raw detection labels in database
86
- 2. Apply FHWA mapping + user configuration at aggregation time
87
- 3. Return only custom class names to API
110
+ **Where to Add "Total"**: AFTER line 312, BEFORE return statement (Line 314)
88
111
 
89
112
  ---
90
113
 
91
- ## Database Schema
92
-
93
- ### Table: `report_configurations`
94
-
95
- ```sql
96
- CREATE TABLE report_configurations (
97
- id SERIAL PRIMARY KEY,
98
- uuid UUID NOT NULL UNIQUE DEFAULT gen_random_uuid(),
99
- name VARCHAR(30) NOT NULL UNIQUE,
100
- description TEXT,
101
- configuration JSONB NOT NULL,
102
- is_default BOOLEAN NOT NULL DEFAULT false,
103
- created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
104
- updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
105
- );
106
-
107
- CREATE INDEX idx_report_configurations_uuid ON report_configurations(uuid);
108
- CREATE INDEX idx_report_configurations_is_default ON report_configurations(is_default);
109
- CREATE INDEX idx_report_configurations_name_lower ON report_configurations(LOWER(name));
114
+ ## Detailed Implementation Plan
115
+
116
+ ### Step 1: Add "Total" Class After Custom Class Transformation
117
+
118
+ **Location**: `ReportConfigurationDAO.applyConfigurationToNestedStructure()` (After line 312)
119
+
120
+ **Logic**:
121
+
122
+ ```typescript
123
+ // After line 312 (end of detection label iteration)
124
+ // Add "Total" class that aggregates all custom classes
125
+
126
+ result["Total"] = this._createTotalClass(result);
127
+
128
+ return result;
110
129
  ```
111
130
 
112
- **Constraints:**
131
+ ### Step 2: Implement `_createTotalClass()` Helper Method
113
132
 
114
- - `name`: VARCHAR(30), UNIQUE, NOT NULL (max 30 characters)
115
- - `uuid`: UUID, UNIQUE, NOT NULL, auto-generated
116
- - `configuration`: JSONB, NOT NULL
117
- - At least 1 configuration must exist (enforced at application level)
133
+ **Location**: Add new private method in `ReportConfigurationDAO` class (After line 350)
118
134
 
119
- ### Configuration JSONB Structure
135
+ **Method Signature**:
120
136
 
121
137
  ```typescript
122
- {
123
- "version": "1.0",
124
- "customClasses": [
125
- {
126
- "name": "Light Vehicles",
127
- "fhwaClasses": [1, 2, 3]
128
- },
129
- {
130
- "name": "Medium Trucks",
131
- "fhwaClasses": [5]
132
- },
133
- {
134
- "name": "Heavy Trucks",
135
- "fhwaClasses": [4, 6, 7, 8, 9, 10, 11, 12, 13]
138
+ private _createTotalClass(customClassesData: Record<string, any>): any
139
+ ```
140
+
141
+ **Implementation**:
142
+
143
+ ```typescript
144
+ /**
145
+ * Create "Total" class by aggregating all custom vehicle classes
146
+ *
147
+ * Handles both ATR and TMC formats:
148
+ * - ATR: Aggregates counts per lane across all vehicle classes
149
+ * - TMC: Aggregates counts per direction and turn across all vehicle classes
150
+ *
151
+ * @param customClassesData - Transformed custom classes structure
152
+ * @returns Aggregated totals matching the same nested structure
153
+ */
154
+ private _createTotalClass(customClassesData: Record<string, any>): any {
155
+ const total: any = {};
156
+
157
+ // Iterate through all custom classes and merge their data
158
+ for (const [className, nestedData] of Object.entries(customClassesData)) {
159
+ // Use existing _deepMergeNumericData to sum all nested values
160
+ Object.assign(total, this._deepMergeNumericData(total, nestedData));
136
161
  }
137
- ]
162
+
163
+ return total;
138
164
  }
139
165
  ```
140
166
 
141
- **Validation Rules:**
167
+ **Explanation**:
142
168
 
143
- - `customClasses` array: min 2, max 7 items
144
- - Each `name`: max 30 characters
145
- - Each `fhwaClasses` array: unique integers from 1-13
146
- - No FHWA class can appear in multiple custom classes (Many-to-One)
147
- - Unmapped FHWA classes are allowed (will be excluded from reports)
169
+ - Reuses existing `_deepMergeNumericData()` helper (Lines 328-350)
170
+ - Works for both ATR (2-level: vehicle → lane → count) and TMC (3-level: vehicle → direction → turn → count)
171
+ - No need to detect study type - structure-agnostic aggregation
172
+ - Handles empty data (returns empty object `{}`)
148
173
 
149
174
  ---
150
175
 
151
- ## Migration
176
+ ## Key Ordering Strategy
177
+
178
+ ### Ensuring "Total" Appears Last
179
+
180
+ JavaScript object key ordering is preserved in modern engines (ES2015+) when:
181
+
182
+ 1. String keys are added in insertion order
183
+ 2. Keys are enumerated via `Object.entries()`, `Object.keys()`, `for...in`
152
184
 
153
- ### File: `YYYYMMDDHHMMSS_create_report_configurations.ts`
185
+ **Our Implementation**:
154
186
 
155
187
  ```typescript
156
- import type { Knex } from "knex";
157
-
158
- export async function up(knex: Knex): Promise<void> {
159
- await knex.schema.createTable("report_configurations", (table) => {
160
- table.increments("id").primary();
161
- table
162
- .uuid("uuid")
163
- .notNullable()
164
- .unique()
165
- .defaultTo(knex.raw("gen_random_uuid()"));
166
- table.string("name", 30).notNullable().unique();
167
- table.text("description").nullable();
168
- table.jsonb("configuration").notNullable();
169
- table.boolean("is_default").notNullable().defaultTo(false);
170
- table.timestamps(true, true);
171
-
172
- // Indexes
173
- table.index("uuid", "idx_report_configurations_uuid");
174
- table.index("is_default", "idx_report_configurations_is_default");
175
- });
176
-
177
- // Add case-insensitive name index
178
- await knex.raw(`
179
- CREATE INDEX idx_report_configurations_name_lower
180
- ON report_configurations(LOWER(name))
181
- `);
182
-
183
- // Seed default configuration (matches current system behavior)
184
- await knex("report_configurations").insert({
185
- name: "Default",
186
- description: "Standard configuration matching current system behavior",
187
- configuration: JSON.stringify({
188
- version: "1.0",
189
- customClasses: [
190
- {
191
- name: "Cars",
192
- fhwaClasses: [1, 2], // motorcycle, car
193
- },
194
- {
195
- name: "Medium Trucks",
196
- fhwaClasses: [3, 5], // pickup_truck, work_van
197
- },
198
- {
199
- name: "Heavy Trucks",
200
- fhwaClasses: [4, 6, 7, 8, 9, 10, 11, 12, 13], // bus, single_unit_truck, articulated_truck
201
- },
202
- ],
203
- }),
204
- is_default: true,
205
- });
188
+ // Lines 293-296: Initialize custom classes FIRST (insertion order)
189
+ for (const customClass of config.configuration.customClasses) {
190
+ result[customClass.name] = {};
206
191
  }
207
192
 
208
- export async function down(knex: Knex): Promise<void> {
209
- await knex.raw("DROP INDEX IF EXISTS idx_report_configurations_name_lower");
210
- await knex.schema.dropTableIfExists("report_configurations");
193
+ // Lines 299-312: Populate custom classes (preserves order)
194
+ for (const [detectionLabel, nestedData] of Object.entries(vehiclesStructure)) {
195
+ // ... merge logic
211
196
  }
197
+
198
+ // NEW: Add "Total" LAST (after all custom classes)
199
+ result["Total"] = this._createTotalClass(result);
200
+
201
+ // Line 314: Return result (Total is last key)
202
+ return result;
212
203
  ```
213
204
 
214
- **Migration Safety:** LOW RISK
205
+ **Why This Works**:
215
206
 
216
- - Creates new table (no data loss)
217
- - Seeds single default configuration
218
- - Rollback is clean (drop table)
207
+ - Custom classes initialized in config order (Lines 293-296)
208
+ - "Total" added AFTER all custom classes (new code)
209
+ - JavaScript guarantees insertion order preservation
210
+ - Frontend will render "Total" as last tab
219
211
 
220
212
  ---
221
213
 
222
- ## Interface Definition
214
+ ## ATR vs TMC Behavior
223
215
 
224
- ### File: `src/interfaces/report-configuration/report-configuration.interfaces.ts`
216
+ ### ATR (Lane-Based Totals)
225
217
 
226
- ```typescript
227
- export interface IReportConfiguration {
228
- id: number;
229
- uuid: string;
230
- name: string;
231
- description?: string;
232
- configuration: IReportConfigurationData;
233
- isDefault: boolean;
234
- createdAt: string;
235
- updatedAt: string;
218
+ **Input Structure**:
219
+
220
+ ```json
221
+ {
222
+ "Light": { "0": 45, "1": 50 },
223
+ "Heavy": { "0": 10, "1": 8 }
236
224
  }
225
+ ```
226
+
227
+ **Total Output**:
237
228
 
238
- export interface IReportConfigurationData {
239
- version: string;
240
- customClasses: ICustomClass[];
229
+ ```json
230
+ {
231
+ "Total": { "0": 55, "1": 58 } // Sum per lane
241
232
  }
233
+ ```
234
+
235
+ **Calculation**:
236
+
237
+ - Lane "0": 45 (Light) + 10 (Heavy) = 55
238
+ - Lane "1": 50 (Light) + 8 (Heavy) = 58
242
239
 
243
- export interface ICustomClass {
244
- name: string;
245
- fhwaClasses: number[];
240
+ **Alternative (Use existing `lane_counts`)**:
241
+
242
+ The ATR structure ALREADY has `lane_counts` at the top level:
243
+
244
+ ```json
245
+ {
246
+ "vehicles": { "Light": {...}, "Heavy": {...} },
247
+ "lane_counts": { "0": 55, "1": 58 } // Already calculated!
246
248
  }
249
+ ```
250
+
251
+ **Decision**: For ATR, we can EITHER:
247
252
 
248
- export interface IReportConfigurationInput {
249
- name: string;
250
- description?: string;
251
- configuration: IReportConfigurationData;
252
- isDefault?: boolean;
253
+ 1. **Option A (Recommended)**: Calculate "Total" by summing custom classes (consistent with TMC)
254
+ 2. ⚠️ **Option B**: Copy `lane_counts` directly into `vehicles["Total"]` (reuses existing data)
255
+
256
+ **Recommendation**: Use Option A (sum custom classes) for consistency with TMC, even though Option B is technically available.
257
+
258
+ ### TMC (Direction/Turn-Based Totals)
259
+
260
+ **Input Structure**:
261
+
262
+ ```json
263
+ {
264
+ "Light": {
265
+ "NORTH": { "straight": 10, "left": 5, "right": 3 },
266
+ "SOUTH": { "straight": 8, "left": 2, "right": 4 }
267
+ },
268
+ "Heavy": {
269
+ "NORTH": { "straight": 5, "left": 1, "right": 0 },
270
+ "SOUTH": { "straight": 2, "left": 0, "right": 1 }
271
+ }
253
272
  }
273
+ ```
274
+
275
+ **Total Output**:
254
276
 
255
- export interface IConfigurationValidationResult {
256
- valid: boolean;
257
- errors: string[];
277
+ ```json
278
+ {
279
+ "Total": {
280
+ "NORTH": { "straight": 15, "left": 6, "right": 3 },
281
+ "SOUTH": { "straight": 10, "left": 2, "right": 5 }
282
+ }
258
283
  }
259
284
  ```
260
285
 
286
+ **Calculation**:
287
+
288
+ - NORTH/straight: 10 (Light) + 5 (Heavy) = 15
289
+ - NORTH/left: 5 (Light) + 1 (Heavy) = 6
290
+ - SOUTH/straight: 8 (Light) + 2 (Heavy) = 10
291
+ - etc.
292
+
261
293
  ---
262
294
 
263
- ## DAO Implementation
295
+ ## Edge Cases & Validation
264
296
 
265
- ### File: `src/dao/report-configuration/report-configuration.dao.ts`
297
+ ### Edge Case 1: Empty Data (No Vehicles)
266
298
 
267
- ```typescript
268
- import { Knex } from "knex";
269
- import { IBaseDAO, IDataPaginator } from "../../d.types";
270
- import {
271
- IReportConfiguration,
272
- IReportConfigurationInput,
273
- IReportConfigurationData,
274
- IConfigurationValidationResult,
275
- } from "../../interfaces/report-configuration/report-configuration.interfaces";
276
- import KnexManager from "../../KnexConnection";
277
-
278
- // Static mapping: Detection Label → FHWA Classes
279
- const DETECTION_LABEL_TO_FHWA: Record<string, number[]> = {
280
- motorcycle: [1],
281
- car: [2],
282
- pickup_truck: [3],
283
- bus: [4],
284
- work_van: [5],
285
- single_unit_truck: [6, 7, 8],
286
- articulated_truck: [9, 10, 11, 12, 13],
287
- motorized_vehicle: [2], // Map to car
288
- // Non-motorized excluded from FHWA
289
- pedestrian: [],
290
- bicycle: [],
291
- non_motorized_vehicle: [],
292
- };
293
-
294
- export class ReportConfigurationDAO implements IBaseDAO<IReportConfiguration> {
295
- private knex: Knex = KnexManager.getConnection();
296
- private tableName = "report_configurations";
297
-
298
- // ==================== STANDARD CRUD ====================
299
-
300
- async create(item: IReportConfigurationInput): Promise<IReportConfiguration> {
301
- // Validate configuration before creation
302
- const validation = this.validateConfiguration(item.configuration);
303
- if (!validation.valid) {
304
- throw new Error(`Invalid configuration: ${validation.errors.join(", ")}`);
305
- }
299
+ **Input**:
306
300
 
307
- const dbData = this.mapInputToDbRow(item);
308
- const [result] = await this.knex(this.tableName)
309
- .insert(dbData)
310
- .returning("*");
301
+ ```json
302
+ {
303
+ "vehicles": {}
304
+ }
305
+ ```
311
306
 
312
- return this.mapDbRowToEntity(result);
313
- }
307
+ **After Transformation**:
314
308
 
315
- async getById(id: number): Promise<IReportConfiguration | null> {
316
- const result = await this.knex(this.tableName).where("id", id).first();
317
- return result ? this.mapDbRowToEntity(result) : null;
309
+ ```json
310
+ {
311
+ "vehicles": {
312
+ "Light": {},
313
+ "Heavy": {},
314
+ "Total": {} // Empty object
318
315
  }
316
+ }
317
+ ```
319
318
 
320
- async getByUuid(uuid: string): Promise<IReportConfiguration | null> {
321
- const result = await this.knex(this.tableName).where("uuid", uuid).first();
322
- return result ? this.mapDbRowToEntity(result) : null;
319
+ **Behavior**: `_createTotalClass()` returns empty object `{}`
320
+
321
+ ### Edge Case 2: Single Custom Class
322
+
323
+ **Input**:
324
+
325
+ ```json
326
+ {
327
+ "vehicles": {
328
+ "AllVehicles": { "0": 100, "1": 95 }
323
329
  }
330
+ }
331
+ ```
332
+
333
+ **Output**:
324
334
 
325
- async getAll(
326
- page: number = 1,
327
- limit: number = 10,
328
- ): Promise<IDataPaginator<IReportConfiguration>> {
329
- const offset = (page - 1) * limit;
330
-
331
- const [countResult] = await this.knex(this.tableName).count("* as count");
332
- const totalCount = +countResult.count;
333
-
334
- const data = await this.knex(this.tableName)
335
- .select("*")
336
- .orderBy("is_default", "desc") // Default first
337
- .orderBy("name", "asc")
338
- .limit(limit)
339
- .offset(offset);
340
-
341
- return {
342
- success: true,
343
- data: this.mapDbRowsToEntities(data),
344
- page,
345
- limit,
346
- count: data.length,
347
- totalCount,
348
- totalPages: Math.ceil(totalCount / limit),
349
- };
335
+ ```json
336
+ {
337
+ "vehicles": {
338
+ "AllVehicles": { "0": 100, "1": 95 },
339
+ "Total": { "0": 100, "1": 95 } // Same as single class
350
340
  }
341
+ }
342
+ ```
351
343
 
352
- async update(
353
- id: number,
354
- item: Partial<IReportConfigurationInput>,
355
- ): Promise<IReportConfiguration | null> {
356
- // Validate configuration if provided
357
- if (item.configuration) {
358
- const validation = this.validateConfiguration(item.configuration);
359
- if (!validation.valid) {
360
- throw new Error(
361
- `Invalid configuration: ${validation.errors.join(", ")}`,
362
- );
363
- }
364
- }
344
+ **Behavior**: "Total" equals the single custom class (valid)
365
345
 
366
- const dbData = this.mapInputToDbRow(item);
367
- const [result] = await this.knex(this.tableName)
368
- .where("id", id)
369
- .update({ ...dbData, updated_at: this.knex.fn.now() })
370
- .returning("*");
346
+ ### Edge Case 3: Missing Directions/Turns
371
347
 
372
- return result ? this.mapDbRowToEntity(result) : null;
373
- }
348
+ **Input** (TMC - Light has NORTH, Heavy only has SOUTH):
349
+
350
+ ```json
351
+ {
352
+ "Light": { "NORTH": { "straight": 10 } },
353
+ "Heavy": { "SOUTH": { "straight": 5 } }
354
+ }
355
+ ```
374
356
 
375
- async delete(id: number): Promise<boolean> {
376
- // Use deleteWithValidation to prevent deletion of last config
377
- return this.deleteWithValidation(id);
357
+ **Output**:
358
+
359
+ ```json
360
+ {
361
+ "Total": {
362
+ "NORTH": { "straight": 10 },
363
+ "SOUTH": { "straight": 5 }
378
364
  }
365
+ }
366
+ ```
379
367
 
380
- // ==================== CUSTOM METHODS ====================
368
+ **Behavior**: `_deepMergeNumericData()` handles missing keys gracefully (Lines 339-347)
381
369
 
382
- /**
383
- * Delete configuration with validation (prevents deletion of last config)
384
- */
385
- async deleteWithValidation(id: number): Promise<boolean> {
386
- const [countResult] = await this.knex(this.tableName).count("* as count");
387
- const totalCount = +countResult.count;
370
+ ### Edge Case 4: Null/Undefined Values
388
371
 
389
- if (totalCount <= 1) {
390
- throw new Error(
391
- "Cannot delete the last configuration. At least one configuration must exist.",
392
- );
393
- }
372
+ **Input**:
394
373
 
395
- const result = await this.knex(this.tableName).where("id", id).del();
396
- return result > 0;
397
- }
374
+ ```json
375
+ {
376
+ "Light": { "0": 10, "1": null },
377
+ "Heavy": { "0": 5 }
378
+ }
379
+ ```
398
380
 
399
- /**
400
- * Get default configuration (for UI convenience)
401
- */
402
- async getDefault(): Promise<IReportConfiguration | null> {
403
- const result = await this.knex(this.tableName)
404
- .where("is_default", true)
405
- .first();
406
-
407
- // If no default, return first configuration
408
- if (!result) {
409
- const fallback = await this.knex(this.tableName)
410
- .orderBy("created_at", "asc")
411
- .first();
412
- return fallback ? this.mapDbRowToEntity(fallback) : null;
413
- }
381
+ **Output**:
414
382
 
415
- return this.mapDbRowToEntity(result);
416
- }
383
+ ```json
384
+ {
385
+ "Total": { "0": 15, "1": 0 } // null treated as 0
386
+ }
387
+ ```
417
388
 
418
- /**
419
- * Set a configuration as default (removes default flag from others)
420
- */
421
- async setDefault(id: number): Promise<boolean> {
422
- await this.knex.transaction(async (trx) => {
423
- // Remove default flag from all
424
- await trx(this.tableName).update({ is_default: false });
389
+ **Behavior**: `_deepMergeNumericData()` checks `typeof source === 'number'` (Line 330)
425
390
 
426
- // Set new default
427
- await trx(this.tableName).where("id", id).update({ is_default: true });
428
- });
391
+ ---
429
392
 
430
- return true;
431
- }
393
+ ## Testing Strategy
432
394
 
433
- /**
434
- * Validate configuration structure and business rules
435
- */
436
- validateConfiguration(
437
- config: IReportConfigurationData,
438
- ): IConfigurationValidationResult {
439
- const errors: string[] = [];
440
-
441
- // Check structure
442
- if (!config || typeof config !== "object") {
443
- errors.push("Configuration must be an object");
444
- return { valid: false, errors };
445
- }
395
+ ### Unit Tests (to be created)
446
396
 
447
- if (!config.customClasses || !Array.isArray(config.customClasses)) {
448
- errors.push("customClasses must be an array");
449
- return { valid: false, errors };
450
- }
397
+ **File**: `knex-rel/src/dao/report-configuration/report-configuration.dao.test.ts`
451
398
 
452
- // Rule: Min 2, max 7 custom classes
453
- if (config.customClasses.length < 2) {
454
- errors.push("At least 2 custom classes required");
455
- }
456
- if (config.customClasses.length > 7) {
457
- errors.push("Maximum 7 custom classes allowed");
458
- }
399
+ **Test Cases**:
459
400
 
460
- // Track FHWA classes to detect duplicates
461
- const usedFhwaClasses = new Set<number>();
462
- const classNames = new Set<string>();
463
-
464
- for (const customClass of config.customClasses) {
465
- // Check name length
466
- if (!customClass.name || customClass.name.length > 30) {
467
- errors.push(
468
- `Custom class name must be 1-30 characters: "${customClass.name}"`,
469
- );
470
- }
471
-
472
- // Check for duplicate names
473
- if (classNames.has(customClass.name)) {
474
- errors.push(`Duplicate custom class name: "${customClass.name}"`);
475
- }
476
- classNames.add(customClass.name);
477
-
478
- // Check fhwaClasses array
479
- if (
480
- !Array.isArray(customClass.fhwaClasses) ||
481
- customClass.fhwaClasses.length === 0
482
- ) {
483
- errors.push(
484
- `Custom class "${customClass.name}" must have at least one FHWA class`,
485
- );
486
- continue;
487
- }
488
-
489
- // Check for duplicates (Many-to-One enforcement)
490
- for (const fhwaClass of customClass.fhwaClasses) {
491
- if (usedFhwaClasses.has(fhwaClass)) {
492
- errors.push(
493
- `FHWA class ${fhwaClass} appears in multiple custom classes (not allowed)`,
494
- );
495
- }
496
- usedFhwaClasses.add(fhwaClass);
497
-
498
- // Validate FHWA class range
499
- if (fhwaClass < 1 || fhwaClass > 13) {
500
- errors.push(`Invalid FHWA class ${fhwaClass} (must be 1-13)`);
501
- }
502
- }
503
- }
401
+ 1. **ATR: Total Class Aggregation**
402
+ - Input: 2 custom classes with lane counts
403
+ - Verify: Total sums all lanes correctly
404
+ - Verify: Total appears as last key
504
405
 
505
- return {
506
- valid: errors.length === 0,
507
- errors,
508
- };
509
- }
406
+ 2. **TMC: Total Class Aggregation**
407
+ - Input: 2 custom classes with direction/turn structure
408
+ - Verify: Total sums all directions and turns
409
+ - Verify: Total appears as last key
510
410
 
511
- /**
512
- * Apply configuration to raw detection counts
513
- * @param detectionCounts - Raw counts from video processing (e.g., {car: 50, pickup_truck: 20})
514
- * @param configuration - Report configuration to apply
515
- * @returns Transformed counts with custom class names
516
- */
517
- applyConfiguration(
518
- detectionCounts: Record<string, number>,
519
- configuration: IReportConfigurationData,
520
- ): Record<string, number> {
521
- // Step 1: Convert detection labels to FHWA classes
522
- const fhwaCounts: Record<number, number> = {};
523
-
524
- for (const [detectionLabel, count] of Object.entries(detectionCounts)) {
525
- const fhwaClasses = DETECTION_LABEL_TO_FHWA[detectionLabel] || [];
526
-
527
- // Distribute count equally among FHWA classes (for multi-mapped labels like single_unit_truck)
528
- const countPerClass = count / fhwaClasses.length;
529
-
530
- for (const fhwaClass of fhwaClasses) {
531
- fhwaCounts[fhwaClass] = (fhwaCounts[fhwaClass] || 0) + countPerClass;
532
- }
533
- }
411
+ 3. **Edge Case: Empty Vehicles**
412
+ - Input: No vehicle data
413
+ - Verify: Total is empty object `{}`
534
414
 
535
- // Step 2: Apply user configuration (FHWA classes → Custom classes)
536
- const customCounts: Record<string, number> = {};
415
+ 4. **Edge Case: Single Custom Class**
416
+ - Input: 1 custom class
417
+ - Verify: Total equals the single class
537
418
 
538
- for (const customClass of configuration.customClasses) {
539
- let total = 0;
419
+ 5. **Edge Case: Missing Directions**
420
+ - Input: Asymmetric direction/turn data
421
+ - Verify: Total includes all unique keys
540
422
 
541
- for (const fhwaClass of customClass.fhwaClasses) {
542
- total += fhwaCounts[fhwaClass] || 0;
543
- }
423
+ 6. **Key Ordering Test**
424
+ - Input: 5 custom classes
425
+ - Verify: Total is the 6th key (last)
426
+ - Verify: `Object.keys(result)` ends with "Total"
544
427
 
545
- customCounts[customClass.name] = Math.round(total);
546
- }
428
+ ### Integration Tests
547
429
 
548
- return customCounts;
549
- }
430
+ **Test in api-rel controller**:
550
431
 
551
- // ==================== MAPPING HELPERS ====================
552
-
553
- private mapDbRowToEntity(row: any): IReportConfiguration {
554
- return {
555
- id: row.id,
556
- uuid: row.uuid,
557
- name: row.name,
558
- description: row.description,
559
- configuration: row.configuration,
560
- isDefault: row.is_default,
561
- createdAt: row.created_at,
562
- updatedAt: row.updated_at,
563
- };
564
- }
432
+ 1. Create video with ATR results
433
+ 2. Apply configuration with 3 custom classes
434
+ 3. Call `GET /videos/:uuid/results?configUuid=...`
435
+ 4. Verify response includes "Total" as last class
436
+ 5. Verify Total values equal sum of custom classes
565
437
 
566
- private mapDbRowsToEntities(rows: any[]): IReportConfiguration[] {
567
- return rows.map((row) => this.mapDbRowToEntity(row));
568
- }
438
+ **Test in traffic-webapp**:
439
+
440
+ 1. Display video results with custom classes
441
+ 2. Verify "Total" tab appears last
442
+ 3. Verify Total chart shows aggregated data
443
+ 4. Test with both ATR and TMC videos
569
444
 
570
- private mapInputToDbRow(input: Partial<IReportConfigurationInput>): any {
571
- const dbRow: any = {};
445
+ ---
572
446
 
573
- if (input.name !== undefined) dbRow.name = input.name;
574
- if (input.description !== undefined) dbRow.description = input.description;
575
- if (input.configuration !== undefined)
576
- dbRow.configuration = input.configuration;
577
- if (input.isDefault !== undefined) dbRow.is_default = input.isDefault;
447
+ ## Performance Considerations
578
448
 
579
- return dbRow;
580
- }
581
- }
449
+ ### Computational Complexity
582
450
 
583
- export default ReportConfigurationDAO;
584
- ```
451
+ **Current**: O(n × m) where:
452
+
453
+ - n = number of detection labels
454
+ - m = average nesting depth (2 for ATR, 3 for TMC)
455
+
456
+ **After Adding Total**: O(n × m + c × m) where:
457
+
458
+ - c = number of custom classes (2-7 per validation)
459
+
460
+ **Impact**: Minimal - additional O(c × m) is negligible
461
+
462
+ - c ≤ 7 (max custom classes)
463
+ - m ≤ 3 (max nesting depth)
464
+ - Total iteration: ~21 operations per minute result
465
+
466
+ ### Memory Impact
467
+
468
+ **Additional Memory**: One new key per transformed result
469
+
470
+ - ATR Total: ~100 bytes (lanes object)
471
+ - TMC Total: ~500 bytes (directions/turns object)
472
+ - Per minute result: negligible
473
+ - For 60 minutes: ~6-30 KB additional
474
+
475
+ **Conclusion**: No performance concerns
585
476
 
586
477
  ---
587
478
 
588
- ## Integration with VideoMinuteResultDAO
479
+ ## Backward Compatibility
589
480
 
590
- ### Modifications Required
481
+ ### Changes Are Additive Only
591
482
 
592
- The `VideoMinuteResultDAO` must be modified to accept and apply report configurations during aggregation.
483
+ **No Breaking Changes**:
593
484
 
594
- **File:** `src/dao/VideoMinuteResultDAO.ts`
485
+ - Existing custom classes remain unchanged
486
+ - Existing API response structure preserved
487
+ - "Total" is a NEW key (optional to consume)
595
488
 
596
- ### Changes to Method Signatures
489
+ **Frontend Compatibility**:
597
490
 
598
- ```typescript
599
- // NEW: Add configurationId parameter to aggregation methods
600
- async getGroupedMinuteResultsByVideoUuid(
601
- videoUuid: string,
602
- groupingMinutes: number = 1,
603
- configurationId?: number, // NEW PARAMETER
604
- startMinute?: number,
605
- endMinute?: number,
606
- ): Promise<IGroupedResponse>
607
- ```
491
+ - Old frontend: Ignores "Total" key (no errors)
492
+ - New frontend: Displays "Total" as last tab
608
493
 
609
- ### Integration Strategy
494
+ **Database Compatibility**:
610
495
 
611
- 1. **Import ReportConfigurationDAO:**
496
+ - No schema changes required
497
+ - Raw results in database unchanged
498
+ - Transformation happens at runtime
612
499
 
613
- ```typescript
614
- import { ReportConfigurationDAO } from "./report-configuration/report-configuration.dao";
615
- ```
500
+ ---
616
501
 
617
- 2. **Modify aggregateTMCResults() (lines 508-585):**
618
- - Accept `configuration` parameter
619
- - Apply configuration transformation AFTER aggregating detection labels
620
- - Replace vehicle class keys with custom class names
502
+ ## Implementation Checklist
621
503
 
622
- 3. **Modify aggregateATRResults() (lines 590-642):**
623
- - Accept `configuration` parameter
624
- - Remove `normalizeATRVehicleClass()` calls (lines 606, 634)
625
- - Apply configuration transformation instead
504
+ ### Pre-Implementation
626
505
 
627
- 4. **Add new helper method:**
506
+ - [x] Analyze current transformation logic
507
+ - [x] Identify exact insertion point (line 312)
508
+ - [x] Verify `_deepMergeNumericData()` handles both ATR/TMC
509
+ - [x] Confirm JavaScript key ordering guarantees
628
510
 
629
- ```typescript
630
- private async fetchConfiguration(configurationId?: number): Promise<IReportConfigurationData> {
631
- const configDAO = new ReportConfigurationDAO();
511
+ ### Implementation Phase
632
512
 
633
- let config: IReportConfiguration | null;
513
+ - [ ] Add `_createTotalClass()` helper method after line 350
514
+ - [ ] Add "Total" class insertion after line 312 in `applyConfigurationToNestedStructure()`
515
+ - [ ] Add JSDoc comments for new method
516
+ - [ ] Verify TypeScript compilation: `npm run build`
634
517
 
635
- if (configurationId) {
636
- config = await configDAO.getById(configurationId);
637
- } else {
638
- config = await configDAO.getDefault();
639
- }
518
+ ### Testing Phase
640
519
 
641
- if (!config) {
642
- throw new Error("No report configuration found");
643
- }
520
+ - [ ] Write unit tests for `_createTotalClass()`
521
+ - [ ] Test ATR format (lane-based totals)
522
+ - [ ] Test TMC format (direction/turn totals)
523
+ - [ ] Test edge cases (empty, single class, missing keys)
524
+ - [ ] Test key ordering (Total is last)
525
+ - [ ] Run all tests: `npm test`
644
526
 
645
- return config.configuration;
646
- }
647
- ```
527
+ ### Integration Phase
648
528
 
649
- 5. **Update aggregation flow:**
529
+ - [ ] Build knex-rel: `npm run build`
530
+ - [ ] Test in api-rel: `npm run start:dev`
531
+ - [ ] Call `/videos/:uuid/results` endpoint with ATR video
532
+ - [ ] Call `/videos/:uuid/results` endpoint with TMC video
533
+ - [ ] Verify "Total" appears in response as last class
534
+ - [ ] Verify Total values are correct sums
650
535
 
651
- ```typescript
652
- // In getGroupedMinuteResultsByVideoUuid()
653
- const configuration = await this.fetchConfiguration(configurationId);
536
+ ### Frontend Integration (Separate Task)
654
537
 
655
- const aggregatedGroups: IGroupedResult[] = rows.map((row: any) => {
656
- // ... existing aggregation logic ...
538
+ - [ ] Update traffic-webapp to display "Total" tab
539
+ - [ ] Position "Total" tab as last tab
540
+ - [ ] Render Total data in charts/tables
541
+ - [ ] Test with both ATR and TMC videos
657
542
 
658
- let aggregatedResult;
659
- if (studyType === "TMC") {
660
- aggregatedResult = this.aggregateTMCResults(allResults, configuration);
661
- } else {
662
- aggregatedResult = this.aggregateATRResults(allResults, configuration);
663
- }
543
+ ---
664
544
 
665
- // ... return result ...
666
- });
667
- ```
545
+ ## Code Changes Summary
668
546
 
669
- ### Detailed Transformation Example (ATR)
547
+ ### File 1: `knex-rel/src/dao/report-configuration/report-configuration.dao.ts`
670
548
 
671
- **Before transformation (raw detection labels):**
549
+ **Location 1: After Line 312 (Inside `applyConfigurationToNestedStructure()`)**
672
550
 
673
551
  ```typescript
674
- {
675
- vehicles: {
676
- car: { lane_1: 50, lane_2: 30 },
677
- pickup_truck: { lane_1: 10, lane_2: 15 },
678
- bus: { lane_1: 5, lane_2: 3 }
679
- },
680
- detected_classes: {
681
- car: 80,
682
- pickup_truck: 25,
683
- bus: 8
552
+ // Lines 299-312: Existing iteration through detection labels
553
+ for (const [detectionLabel, nestedData] of Object.entries(vehiclesStructure)) {
554
+ const customClassName = detectionToCustomClass[detectionLabel];
555
+ if (!customClassName) {
556
+ continue;
684
557
  }
558
+ result[customClassName] = this._deepMergeNumericData(
559
+ result[customClassName],
560
+ nestedData,
561
+ );
685
562
  }
563
+
564
+ // NEW CODE: Add "Total" class that aggregates all custom classes
565
+ result["Total"] = this._createTotalClass(result);
566
+
567
+ return result;
686
568
  ```
687
569
 
688
- **After applying configuration:**
570
+ **Location 2: After Line 350 (New Private Method)**
689
571
 
690
572
  ```typescript
691
- // Configuration: Cars [1,2], Medium Trucks [3,5], Heavy Trucks [4,6-13]
692
- {
693
- vehicles: {
694
- "Cars": { lane_1: 50, lane_2: 30 }, // car FHWA 2 → Cars
695
- "Medium Trucks": { lane_1: 10, lane_2: 15 }, // pickup_truck FHWA 3 → Medium Trucks
696
- "Heavy Trucks": { lane_1: 5, lane_2: 3 } // bus → FHWA 4 → Heavy Trucks
697
- },
698
- detected_classes: {
699
- "Cars": 80,
700
- "Medium Trucks": 25,
701
- "Heavy Trucks": 8
702
- }
573
+ /**
574
+ * Create "Total" class by aggregating all custom vehicle classes
575
+ *
576
+ * Sums all custom class counts across their nested structures.
577
+ * Works for both ATR (lane-based) and TMC (direction/turn-based) formats.
578
+ *
579
+ * @param customClassesData - Transformed custom classes structure
580
+ * Example ATR: { "Light": { "0": 45, "1": 50 }, "Heavy": { "0": 10 } }
581
+ * Example TMC: { "Light": { "NORTH": { "straight": 10 } }, ... }
582
+ * @returns Aggregated totals matching the same nested structure
583
+ * Example ATR: { "0": 55, "1": 50 }
584
+ * Example TMC: { "NORTH": { "straight": 10 }, ... }
585
+ */
586
+ private _createTotalClass(customClassesData: Record<string, any>): any {
587
+ let total: any = {};
588
+
589
+ // Iterate through all custom classes and merge their data
590
+ for (const [className, nestedData] of Object.entries(customClassesData)) {
591
+ total = this._deepMergeNumericData(total, nestedData);
592
+ }
593
+
594
+ return total;
703
595
  }
704
596
  ```
705
597
 
706
598
  ---
707
599
 
708
- ## Export Updates
600
+ ## Validation Plan
709
601
 
710
- ### File: `src/index.ts`
602
+ ### Manual Testing Checklist
711
603
 
712
- Add exports for new DAO and interfaces:
604
+ **Test 1: ATR Video (2 Custom Classes)**
713
605
 
714
- ```typescript
715
- // Add to DAOs section
716
- export { ReportConfigurationDAO } from "./dao/report-configuration/report-configuration.dao";
717
-
718
- // Add to Interfaces section
719
- export {
720
- IReportConfiguration,
721
- IReportConfigurationInput,
722
- IReportConfigurationData,
723
- ICustomClass,
724
- IConfigurationValidationResult,
725
- } from "./interfaces/report-configuration/report-configuration.interfaces";
726
- ```
606
+ - [ ] Create configuration: "Light" (FHWA 1-3), "Heavy" (FHWA 4-13)
607
+ - [ ] Upload ATR video with mixed vehicle types
608
+ - [ ] Process video and get results
609
+ - [ ] Verify "Light" shows cars/motorcycles per lane
610
+ - [ ] Verify "Heavy" shows trucks/buses per lane
611
+ - [ ] Verify "Total" shows sum of Light + Heavy per lane
612
+ - [ ] Verify "Total" is last key in vehicles object
727
613
 
728
- ---
614
+ **Test 2: TMC Video (3 Custom Classes)**
729
615
 
730
- ## Implementation Order
616
+ - [ ] Create configuration: "Cars", "Trucks", "Buses"
617
+ - [ ] Upload TMC video with turning movements
618
+ - [ ] Process video and get results
619
+ - [ ] Verify each class shows direction/turn breakdown
620
+ - [ ] Verify "Total" shows sum across all classes
621
+ - [ ] Verify NORTH/straight in Total = sum of all classes' NORTH/straight
622
+ - [ ] Verify "Total" is last key in vehicles object
731
623
 
732
- ### Phase 1: Database Foundation
624
+ **Test 3: Edge Cases**
733
625
 
734
- 1. Create interface file: `report-configuration.interfaces.ts`
735
- 2. Create migration: `YYYYMMDDHHMMSS_create_report_configurations.ts`
736
- 3. Run migration: `npm run migrate:deploy`
737
- 4. Verify seed data exists (1 default configuration)
626
+ - [ ] Video with no vehicles detected (empty Total)
627
+ - [ ] Video with single custom class (Total equals that class)
628
+ - [ ] Video with asymmetric turns (Light has left, Heavy doesn't)
629
+ - [ ] Configuration with 7 custom classes (max allowed)
738
630
 
739
- ### Phase 2: DAO Implementation
631
+ **Test 4: Grouped Results (Multiple Minutes)**
740
632
 
741
- 5. Create DAO: `report-configuration.dao.ts`
742
- 6. Implement standard CRUD methods
743
- 7. Implement validation logic
744
- 8. Implement `applyConfiguration()` transformation logic
745
- 9. Test DAO methods in isolation
633
+ - [ ] Get grouped results with grouping=15
634
+ - [ ] Verify each timeGroup has "Total" class
635
+ - [ ] Verify "Total" sums correctly across aggregated minutes
636
+ - [ ] Verify ordering preserved in all timeGroups
746
637
 
747
- ### Phase 3: Integration
638
+ ---
748
639
 
749
- 10. Modify `VideoMinuteResultDAO.aggregateTMCResults()`
750
- 11. Modify `VideoMinuteResultDAO.aggregateATRResults()`
751
- 12. Remove `normalizeATRVehicleClass()` calls
752
- 13. Add `fetchConfiguration()` helper
753
- 14. Update method signatures to accept `configurationId`
640
+ ## Risks & Mitigations
754
641
 
755
- ### Phase 4: Exports & Build
642
+ ### Risk 1: Key Ordering Not Guaranteed
756
643
 
757
- 15. Update `src/index.ts` exports
758
- 16. Run `npm run build` in knex-rel
759
- 17. Verify no TypeScript errors
760
- 18. Test against sample video data
644
+ **Risk**: JavaScript object keys might not preserve insertion order
761
645
 
762
- ---
646
+ **Likelihood**: Very Low (ES2015+ guarantees string key order)
763
647
 
764
- ## Risks & Concerns
648
+ **Mitigation**:
765
649
 
766
- ### 🔴 CRITICAL: Detection Label Mapping Assumption
650
+ - Add unit test to verify `Object.keys(result)` ends with "Total"
651
+ - If ordering fails, use explicit ordering array in frontend
767
652
 
768
- **Risk:** The `DETECTION_LABEL_TO_FHWA` mapping is based on user clarifications, but there's no guarantee that video processing returns exactly these labels.
653
+ ### Risk 2: Performance Degradation
769
654
 
770
- **Mitigation:**
655
+ **Risk**: Additional aggregation adds processing time
771
656
 
772
- - Test with actual video processing output
773
- - Add logging when unknown detection labels are encountered
774
- - Provide fallback behavior (map unknown labels to FHWA Class 2 by default)
657
+ **Likelihood**: Very Low (O(c × m) is negligible)
775
658
 
776
- ### 🟡 MEDIUM: Performance at Minute-Level Granularity
659
+ **Mitigation**:
777
660
 
778
- **Risk:** Applying configuration transformation at minute-level for 100+ videos could be slow.
661
+ - Benchmark transformation time before/after
662
+ - If impact > 10ms, consider caching Total class
779
663
 
780
- **Current Decision:** Apply transformation on-the-fly (per user clarification Q10.2).
664
+ ### Risk 3: Frontend Breaks on Unknown Class
781
665
 
782
- **Future Optimization:**
666
+ **Risk**: Old frontend version throws error on "Total" key
783
667
 
784
- - Cache configuration object in memory (singleton pattern)
785
- - Pre-compute transformed results and store in separate JSONB column
786
- - Add database-level JSONB transformation functions
668
+ **Likelihood**: Very Low (key is additive, not breaking)
787
669
 
788
- ### 🟡 MEDIUM: Replacement of normalizeATRVehicleClass()
670
+ **Mitigation**:
789
671
 
790
- **Risk:** Existing API clients may expect normalized class names (`cars`, `mediums`, `heavy_trucks`).
672
+ - "Total" is just another custom class name
673
+ - Frontend already iterates dynamic class names
674
+ - No special handling needed
791
675
 
792
- **Mitigation:**
676
+ ### Risk 4: Incorrect Aggregation Logic
793
677
 
794
- - Default configuration seed matches current normalization
795
- - API will return same class names initially (backwards compatible)
796
- - Users can create custom configurations as needed
678
+ **Risk**: `_deepMergeNumericData()` doesn't handle all cases
797
679
 
798
- ### 🟢 LOW: Migration Safety
680
+ **Likelihood**: Low (method already tested for custom class merging)
799
681
 
800
- **Risk:** Minimal - only creates new table with seed data.
682
+ **Mitigation**:
801
683
 
802
- **Rollback:** Clean rollback with `down()` migration drops table entirely.
684
+ - Comprehensive unit tests for edge cases
685
+ - Manual validation with real ATR/TMC videos
686
+ - Use existing, battle-tested `_deepMergeNumericData()`
803
687
 
804
688
  ---
805
689
 
806
- ## Testing Strategy
690
+ ## Alternative Approaches Considered
807
691
 
808
- ### Unit Tests (DAO Level)
692
+ ### Alternative 1: Add "Total" in Frontend
809
693
 
810
- ```typescript
811
- // Test validation
812
- describe("ReportConfigurationDAO.validateConfiguration", () => {
813
- it("should reject configurations with < 2 custom classes");
814
- it("should reject configurations with > 7 custom classes");
815
- it("should reject duplicate FHWA classes");
816
- it("should reject custom class names > 30 chars");
817
- it("should accept valid configurations");
818
- });
819
-
820
- // Test transformation
821
- describe("ReportConfigurationDAO.applyConfiguration", () => {
822
- it("should map detection labels to FHWA classes");
823
- it("should map FHWA classes to custom classes");
824
- it("should exclude unmapped FHWA classes");
825
- it("should handle multi-mapped detection labels (single_unit_truck)");
826
- });
827
-
828
- // Test deletion validation
829
- describe("ReportConfigurationDAO.deleteWithValidation", () => {
830
- it("should prevent deletion of last configuration");
831
- it("should allow deletion when > 1 configuration exists");
832
- });
833
- ```
694
+ **Approach**: Calculate "Total" in traffic-webapp instead of backend
834
695
 
835
- ### Integration Tests (VideoMinuteResultDAO)
696
+ **Pros**:
836
697
 
837
- ```typescript
838
- describe("VideoMinuteResultDAO with ReportConfigurations", () => {
839
- it("should apply configuration to TMC results");
840
- it("should apply configuration to ATR results");
841
- it("should use default configuration when none specified");
842
- it("should handle unmapped vehicle classes correctly");
843
- });
844
- ```
698
+ - No backend changes required
699
+ - Frontend controls display logic
845
700
 
846
- ---
701
+ **Cons**:
847
702
 
848
- ## Performance Considerations
703
+ - Duplicates logic across frontend
704
+ - Inconsistent if API consumed elsewhere
705
+ - Cannot use "Total" in backend reports/exports
849
706
 
850
- ### Query Optimization
707
+ **Decision**: Rejected - Backend is single source of truth
851
708
 
852
- **Configuration Lookup:**
709
+ ### Alternative 2: Add "Total" as Separate Field
853
710
 
854
- - Index on `uuid` for fast lookup by UUID
855
- - Index on `is_default` for quick default retrieval
856
- - Consider in-memory caching (singleton) if configurations queried frequently
711
+ **Approach**: Add `results.totalClass` instead of `results.vehicles["Total"]`
857
712
 
858
- **Transformation:**
713
+ **Pros**:
859
714
 
860
- - `applyConfiguration()` runs in O(n) time where n = number of detection labels
861
- - For minute-level data (potentially 1000s of minutes), this could be expensive
862
- - **Future optimization:** Move transformation to PostgreSQL JSONB functions
715
+ - Clearer separation of concerns
716
+ - No risk of key ordering issues
863
717
 
864
- ### Caching Strategy (Future)
718
+ **Cons**:
865
719
 
866
- ```typescript
867
- class ReportConfigurationCache {
868
- private static instance: Map<number, IReportConfigurationData> = new Map();
720
+ - Inconsistent with custom class structure
721
+ - Frontend needs special handling
722
+ - Breaks uniform class iteration logic
869
723
 
870
- static get(id: number): IReportConfigurationData | undefined {
871
- return this.instance.get(id);
872
- }
724
+ **Decision**: Rejected - Keep "Total" as vehicle class for uniformity
873
725
 
874
- static set(id: number, config: IReportConfigurationData): void {
875
- this.instance.set(id, config);
876
- }
726
+ ### Alternative 3: Make "Total" Configurable
877
727
 
878
- static clear(): void {
879
- this.instance.clear();
880
- }
881
- }
882
- ```
728
+ **Approach**: Add `includeTotalClass: boolean` to report configuration
883
729
 
884
- ---
730
+ **Pros**:
885
731
 
886
- ## Open Questions & Future Enhancements
732
+ - User control over Total display
733
+ - Backward compatible (default false)
887
734
 
888
- ### 1. Unmapped FHWA Classes Display
735
+ **Cons**:
889
736
 
890
- **Question:** Should the API response indicate which FHWA classes were excluded?
737
+ - Adds complexity to configuration
738
+ - Not needed (Total is always useful)
739
+ - Extra validation logic
891
740
 
892
- **Example:**
741
+ **Decision**: ❌ Rejected - Always include "Total" (KISS principle)
893
742
 
894
- ```json
895
- {
896
- "customClasses": {
897
- "Cars": 100,
898
- "Heavy Trucks": 50
899
- },
900
- "unmappedCounts": {
901
- "fhwaClass4": 10,
902
- "fhwaClass5": 5
903
- }
904
- }
905
- ```
743
+ ---
906
744
 
907
- **Recommendation:** Add this as optional metadata for debugging purposes.
745
+ ## Success Criteria
908
746
 
909
- ### 2. Configuration Change History
747
+ ### Implementation Success
910
748
 
911
- **Question:** Should we track configuration changes over time?
749
+ TypeScript compiles without errors
750
+ ✅ `npm run build` succeeds in knex-rel
751
+ ✅ All existing unit tests pass
752
+ ✅ New unit tests pass (Total class logic)
912
753
 
913
- **Future Enhancement:** Add `report_configuration_history` table to track when configurations are created/modified.
754
+ ### Functional Success
914
755
 
915
- ### 3. Pedestrians & Bicycles
756
+ ATR videos show "Total" with correct lane sums
757
+ ✅ TMC videos show "Total" with correct direction/turn sums
758
+ ✅ "Total" appears as LAST class in all responses
759
+ ✅ Edge cases handled (empty, single class, missing keys)
916
760
 
917
- **Question:** Current system excludes pedestrians/bicycles from FHWA mapping. Should users be able to create custom classes for non-motorized traffic?
761
+ ### Integration Success
918
762
 
919
- **Recommendation:** Support this by adding FHWA classes 14-15 (non-motorized) in future iteration.
763
+ API endpoint returns "Total" in response
764
+ ✅ Grouped results include "Total" in each timeGroup
765
+ ✅ No breaking changes to existing API consumers
766
+ ✅ Frontend displays "Total" tab correctly (separate task)
920
767
 
921
- ### 4. Detection Label Variability
768
+ ---
922
769
 
923
- **Question:** What if video processing returns different detection labels than expected?
770
+ ## Timeline Estimate
924
771
 
925
- **Recommendation:** Add logging and admin dashboard to monitor unmapped detection labels.
772
+ ### Backend Implementation (knex-rel)
926
773
 
927
- ---
774
+ - Code changes: 30 minutes
775
+ - Unit tests: 1 hour
776
+ - Build & test: 15 minutes
777
+ - **Total: ~2 hours**
928
778
 
929
- ## Validation Checklist
779
+ ### Integration Testing (api-rel)
930
780
 
931
- Before marking this plan as complete, verify:
781
+ - Manual testing with ATR video: 30 minutes
782
+ - Manual testing with TMC video: 30 minutes
783
+ - Edge case validation: 30 minutes
784
+ - **Total: ~1.5 hours**
932
785
 
933
- - [x] Database schema includes all required fields
934
- - [x] Migration has proper up/down functions
935
- - [x] Seed data matches current system behavior
936
- - [x] Interface definitions match database schema
937
- - [x] DAO implements all IBaseDAO methods
938
- - [x] Validation logic covers all business rules (2-7 classes, no duplicates, 30 char limit)
939
- - [x] DETECTION_LABEL_TO_FHWA mapping is complete
940
- - [x] Integration strategy with VideoMinuteResultDAO is clear
941
- - [x] Export updates are documented
942
- - [x] Implementation order is logical
943
- - [x] Risks are identified and mitigated
944
- - [x] Testing strategy is comprehensive
786
+ ### Frontend Integration (traffic-webapp - Separate)
945
787
 
946
- ---
788
+ - Display "Total" tab: 1 hour
789
+ - Chart/table rendering: 1 hour
790
+ - Testing both video types: 1 hour
791
+ - **Total: ~3 hours**
947
792
 
948
- ## Pattern Compliance
793
+ **Overall Estimate: ~6.5 hours** (Backend: 3.5 hours, Frontend: 3 hours)
949
794
 
950
- ### Database Patterns ✅
795
+ ---
951
796
 
952
- - Primary key: `id` (auto-increment)
953
- - External identifier: `uuid` (unique, not null, auto-generated)
954
- - Timestamps: `created_at`, `updated_at`
955
- - Foreign keys: N/A (system-wide table)
797
+ ## Summary
956
798
 
957
- ### DAO Patterns ✅
799
+ ### What Changes
958
800
 
959
- - Implements `IBaseDAO<IReportConfiguration>`
960
- - Singleton KnexManager connection
961
- - Methods: create, getById, getByUuid, getAll, update, delete
962
- - Custom methods: deleteWithValidation, getDefault, setDefault, validateConfiguration, applyConfiguration
801
+ **ADD**: `_createTotalClass()` private method in ReportConfigurationDAO
802
+ **ADD**: `result["Total"] = ...` line after custom class transformation
803
+ **ADD**: Unit tests for Total class aggregation
963
804
 
964
- ### File Naming
805
+ ### What Stays the Same
965
806
 
966
- - Migration: `YYYYMMDDHHMMSS_create_report_configurations.ts`
967
- - DAO: `report-configuration.dao.ts`
968
- - Interface: `report-configuration.interfaces.ts` (matches folder structure)
807
+ Database schema (no migrations)
808
+ Existing custom classes (no modifications)
809
+ API response structure (additive only)
810
+ ✅ `_deepMergeNumericData()` helper (reused, not changed)
969
811
 
970
- ### Performance
812
+ ### Key Benefits
971
813
 
972
- - Indexed UUID for fast lookups
973
- - JOINs eliminated (no foreign keys, system-wide table)
974
- - Pagination implemented in getAll()
975
- - Future: Consider caching for frequently accessed configurations
814
+ Unified "Total" view across all custom classes
815
+ Consistent with existing transformation architecture
816
+ No breaking changes (backward compatible)
817
+ Minimal performance impact (< 1ms per minute result)
818
+ ✅ Reuses existing, tested aggregation logic
819
+
820
+ **Ready for Implementation** - Plan is complete with exact code locations, logic, edge cases, and validation strategy.
976
821
 
977
822
  ---
978
823
 
979
- ## Next Steps for Orchestrator
824
+ ## Files to Modify
980
825
 
981
- This plan is now ready for handoff to the **knex-code-agent** for implementation.
826
+ ### knex-rel/src/dao/report-configuration/report-configuration.dao.ts
982
827
 
983
- **Orchestrator should:**
828
+ - **Line 312**: Add `result["Total"] = this._createTotalClass(result);`
829
+ - **After Line 350**: Add `_createTotalClass()` private method
984
830
 
985
- 1. Review this plan for completeness
986
- 2. Approve DETECTION_LABEL_TO_FHWA mapping accuracy
987
- 3. Confirm migration timestamp format
988
- 4. Hand off to knex-code-agent for execution
831
+ ### knex-rel/src/dao/report-configuration/report-configuration.dao.test.ts (New File)
989
832
 
990
- **knex-code-agent should:**
833
+ - Create unit tests for Total class aggregation
834
+ - Test ATR format, TMC format, edge cases, key ordering
991
835
 
992
- 1. Implement in exact order specified (Phase 1 → Phase 2 → Phase 3 → Phase 4)
993
- 2. Test each phase before proceeding
994
- 3. Report any conflicts or issues discovered during implementation
995
- 4. Request clarification if detection labels don't match expectations
836
+ ---
996
837
 
997
- **Blockers:**
838
+ ## Pattern Compliance Summary
998
839
 
999
- - None identified. Plan is complete and ready for implementation.
840
+ ### Database Patterns
1000
841
 
1001
- ---
842
+ - No database changes required (transformation only)
843
+ - Follows DAO pattern (logic in ReportConfigurationDAO)
844
+ - No new migrations needed
1002
845
 
1003
- ## Appendix: FHWA 13-Class System Reference
846
+ ### Code Patterns
1004
847
 
1005
- ```
1006
- Class 1: Motorcycles
1007
- Class 2: Passenger Cars
1008
- Class 3: Other Two-Axle, Four-Tire Single Unit Vehicles (Pickups, Vans)
1009
- Class 4: Buses
1010
- Class 5: Two-Axle, Six-Tire Single Unit Trucks (Work Vans)
1011
- Class 6: Three-Axle Single Unit Trucks
1012
- Class 7: Four or More Axle Single Unit Trucks
1013
- Class 8: Four or Less Axle Single Trailer Trucks
1014
- Class 9: Five-Axle Single Trailer Trucks
1015
- Class 10: Six or More Axle Single Trailer Trucks
1016
- Class 11: Five or Less Axle Multi-Trailer Trucks
1017
- Class 12: Six-Axle Multi-Trailer Trucks
1018
- Class 13: Seven or More Axle Multi-Trailer Trucks
1019
- ```
848
+ - Reuses existing `_deepMergeNumericData()` helper
849
+ - Follows private method naming convention (`_methodName`)
850
+ - Maintains nested structure consistency
851
+ - Preserves key ordering via insertion order
1020
852
 
1021
- **Detection Label Mapping:**
853
+ ### Performance Patterns ✅
1022
854
 
1023
- - Classes 6-8: All map to `single_unit_truck` detection label
1024
- - Classes 9-13: All map to `articulated_truck` detection label
1025
- - This creates a many-to-one relationship at the detection level
855
+ - O(c × m) additional complexity (negligible)
856
+ - No N+1 queries (transformation in memory)
857
+ - No database roundtrips
858
+ - Minimal memory overhead (~500 bytes per result)
1026
859
 
1027
860
  ---
1028
861
 
1029
- ## Document Version
862
+ ## Next Steps
1030
863
 
1031
- - **Version:** 1.0
1032
- - **Date:** 2025-09-30
1033
- - **Author:** Database Planning Specialist (Claude)
1034
- - **Status:** FINAL - Ready for Implementation
864
+ 1. **Implement** `_createTotalClass()` method in ReportConfigurationDAO
865
+ 2. **Add** Total class insertion after line 312
866
+ 3. **Write** unit tests for all edge cases
867
+ 4. **Build** knex-rel package and verify compilation
868
+ 5. **Test** with real ATR and TMC videos in api-rel
869
+ 6. **Validate** key ordering and correct aggregation
870
+ 7. **Document** changes (if needed)
871
+ 8. **Coordinate** with frontend team for Total tab display