@trafficgroup/knex-rel 0.0.29 → 0.1.1

package/plan.md

@@ -1,304 +1,129 @@
-# Database Implementation Plan: Video Minute Results Storage
+# Lane/Annotation Reuse Implementation Plan
 
 ## Schema Changes
 
-### New Table: video_minute_results
+### Tables
 
-```sql
-CREATE TABLE video_minute_results (
-  id BIGSERIAL PRIMARY KEY,
-  uuid UUID NOT NULL DEFAULT gen_random_uuid(),
-  video_id INTEGER NOT NULL REFERENCES video(id) ON DELETE CASCADE,
-  minute_number INTEGER NOT NULL,
-  results JSONB NOT NULL,
-  created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-  updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-  CONSTRAINT unique_video_minute UNIQUE (video_id, minute_number),
-  CONSTRAINT valid_minute_number CHECK (minute_number >= 0)
-);
-```
+- **video**: Add `annotation_source_id` column (nullable INTEGER, FK to video.id)
 
-**Indexes:**
+### Modifications
 
-- `idx_video_minute_results_video_id` on `video_id` (frequent JOINs with video table)
-- `idx_video_minute_results_uuid` on `uuid` (external API lookups)
-- `idx_video_minute_results_minute_number` on `minute_number` (range queries)
-- `idx_video_minute_results_video_minute` on `(video_id, minute_number)` (composite lookups)
-
-### Modifications to Existing Tables
-
-```sql
--- Add duration tracking to video table
-ALTER TABLE video ADD COLUMN duration_seconds INTEGER;
-```
+- **video table**:
+  - Add `annotation_source_id INTEGER NULL REFERENCES video(id) ON DELETE SET NULL`
+  - Add index on `annotation_source_id` for performance
+  - Add composite index on `(folderId, videoType)` for template lookup optimization
 
 ## Migrations
 
-### 20250823HHMMSS_create_video_minute_results.ts
-
-```typescript
-export async function up(knex: Knex): Promise<void> {
-  // Create video_minute_results table
-  await knex.schema.createTable("video_minute_results", (table) => {
-    table.bigIncrements("id").primary();
-    table
-      .uuid("uuid")
-      .defaultTo(knex.raw("gen_random_uuid()"))
-      .notNullable()
-      .unique();
-    table
-      .integer("video_id")
-      .unsigned()
-      .notNullable()
-      .references("id")
-      .inTable("video")
-      .onDelete("CASCADE");
-    table.integer("minute_number").notNullable();
-    table.jsonb("results").notNullable();
-    table.timestamps(true, true);
-
-    // Constraints
-    table.unique(["video_id", "minute_number"], {
-      indexName: "unique_video_minute",
-    });
-    table.check("minute_number >= 0", [], "valid_minute_number");
-
-    // Indexes
-    table.index("video_id", "idx_video_minute_results_video_id");
-    table.index("uuid", "idx_video_minute_results_uuid");
-    table.index("minute_number", "idx_video_minute_results_minute_number");
-    table.index(
-      ["video_id", "minute_number"],
-      "idx_video_minute_results_video_minute",
-    );
-  });
-
-  // Add duration_seconds to video table
-  await knex.schema.alterTable("video", (table) => {
-    table.integer("duration_seconds").nullable();
-  });
-}
-
-export async function down(knex: Knex): Promise<void> {
-  await knex.schema.dropTableIfExists("video_minute_results");
-  await knex.schema.alterTable("video", (table) => {
-    table.dropColumn("duration_seconds");
-  });
-}
-```
+### 20250917143052_add_annotation_source_to_video.ts
 
-**Safety Level:** Medium - Creates new table and adds nullable column to existing table
+- **Up operations**:
+  - Add `annotation_source_id` column to video table
+  - Add foreign key constraint to video(id) with SET NULL on delete
+  - Add index on `annotation_source_id`
+  - Add composite index on `(folderId, videoType)` for efficient template queries
+- **Down operations**:
+  - Drop indexes
+  - Drop foreign key constraint
+  - Drop column
+- **Safety level**: LOW RISK - Adding nullable column with proper constraints
 
 ## DAOs
 
-### Update: VideoMinuteResultDAO
-
-**File:** `src/dao/VideoMinuteResultDAO.ts` (replace existing mock implementation)
-
-**Methods:**
-
-```typescript
-export class VideoMinuteResultDAO implements IBaseDAO<IVideoMinuteResult> {
-  private knex = KnexManager.getConnection();
-  private tableName = "video_minute_results";
-
-  // Standard CRUD operations
-  async create(data: IVideoMinuteResultInput): Promise<IVideoMinuteResult>;
-  async getById(id: number): Promise<IVideoMinuteResult | null>;
-  async getByUuid(uuid: string): Promise<IVideoMinuteResult | null>;
-  async getAll(
-    page: number,
-    limit: number,
-  ): Promise<IDataPaginator<IVideoMinuteResult>>;
-  async update(
-    id: number,
-    data: Partial<IVideoMinuteResult>,
-  ): Promise<IVideoMinuteResult | null>;
-  async delete(id: number): Promise<boolean>;
-
-  // Specialized batch operations (eliminate N+1 queries)
-  async createBatch(
-    videoId: number,
-    minuteResults: IVideoMinuteResultInput[],
-  ): Promise<IVideoMinuteResult[]>;
-  async getMinuteResultsForVideo(
-    videoId: number,
-    startMinute?: number,
-    endMinute?: number,
-    page?: number,
-    limit?: number,
-  ): Promise<IDataPaginator<IVideoMinuteResult>>;
-  async getMinuteResultsByVideoUuid(
-    videoUuid: string,
-    startMinute?: number,
-    endMinute?: number,
-    page?: number,
-    limit?: number,
-  ): Promise<IDataPaginator<IVideoMinuteResult>>;
-  async deleteByVideoId(videoId: number): Promise<boolean>;
-  async getVideoMinuteRange(
-    videoId: number,
-  ): Promise<{ minMinute: number; maxMinute: number } | null>;
-}
+### Modify: VideoDAO
+
+- **File**: src/dao/video/video.dao.ts
+- **New methods**:
+  - `getTemplateVideos(folderId: number, videoType: string): Promise<IVideo[]>` - Get videos from same folder/type that can serve as templates (have metadata and completed status)
+  - `getVideosUsingTemplate(templateVideoId: number): Promise<IVideo[]>` - Get videos that used a specific video as template
+  - `setAnnotationSource(videoId: number, sourceVideoId: number | null): Promise<IVideo | null>` - Set/update annotation source
+- **Performance optimizations**:
+  - Use JOINs to fetch related annotation source data in main queries
+  - Eliminate N+1 queries when fetching videos with their annotation sources
+  - Optimized template lookup using composite index
+
+### Query Patterns
+
+```sql
+-- Get template videos (same folder + type, completed, with metadata)
+SELECT v.* FROM video v
+WHERE v.folderId = ?
+  AND v.videoType = ?
+  AND v.status = 'COMPLETED'
+  AND v.metadata IS NOT NULL
+  AND v.metadata != '{}'
+ORDER BY v.created_at DESC;
+
+-- Get videos using specific template
+SELECT v.* FROM video v
+WHERE v.annotation_source_id = ?;
 ```
 
-**Performance Optimizations:**
+## Interfaces
 
-- Batch inserts using `knex.batchInsert()` for 5-minute batches
-- Single query with JOINs for video+minute data retrieval
-- Range queries with BETWEEN for minute filtering
-- Pagination with proper OFFSET/LIMIT
+### Modify: IVideo.ts
 
-### Update: VideoDAO
+- **File**: src/interfaces/video/video.interfaces.ts
+- **New properties**:
+  - `annotationSourceId?: number;` - ID of video whose annotations were copied
+  - `annotationSource?: IVideo;` - Optional populated annotation source video object
+- **Export**: Update index.ts exports
 
-**File:** `src/dao/video/video.dao.ts` (add duration_seconds support)
+## Implementation Order
 
-**New Methods:**
+1. **Interfaces** → Update IVideo interface with new fields
+2. **Migrations** → Create and run migration to add database column
+3. **DAOs** → Add new methods to VideoDAO for template management
+4. **Exports** → Update index.ts to export new interface changes
+5. **Build** → Compile and test changes
 
-```typescript
-async updateDuration(id: number, durationSeconds: number): Promise<IVideo | null>
-async getVideosWithoutMinuteData(page: number, limit: number): Promise<IDataPaginator<IVideo>>
-```
+## Business Logic Constraints
 
-## Interfaces
+### Template Selection Rules
 
-### Update: IVideoMinuteResult.ts
-
-```typescript
-export interface IVideoMinuteResult extends IBaseEntity {
-  id: number;
-  uuid: string;
-  videoId: number;
-  minuteNumber: number; // Renamed from 'minute' for clarity
-  results: Record<string, any>;
-  createdAt: string;
-  updatedAt: string;
-}
-
-export interface IVideoMinuteResultInput {
-  videoId: number;
-  minuteNumber: number;
-  results: Record<string, any>;
-}
-
-export interface IVideoMinuteBatch {
-  videoId: number;
-  startMinute: number;
-  endMinute: number;
-  minuteResults: IVideoMinuteResultInput[];
-}
-```
+- Only videos from **same folder** (`folderId` match)
+- Only videos of **same type** (`videoType` match: TMC → TMC, ATR → ATR)
+- Only **COMPLETED** videos with non-empty metadata can be templates
+- Template videos must have lane configuration in metadata field
 
-### Update: IVideo.ts
+### Data Integrity
 
-```typescript
-export interface IVideo {
-  // ... existing fields
-  durationSeconds?: number; // Add optional duration field
-}
-```
+- `annotation_source_id` references `video.id` with CASCADE DELETE behavior set to SET NULL
+- If template video is deleted, dependent videos keep their annotations but lose the source reference
+- Self-referencing constraint: video cannot reference itself as annotation source
 
-### Export Updates
+## Performance Considerations
 
-Update `src/index.ts` to include new interfaces and updated DAO.
+### Indexes
 
-## Implementation Order
+- `annotation_source_id` - for JOIN operations and template lookups
+- `(folderId, videoType)` - composite index for efficient template discovery
+- Existing `uuid` index maintained for external API calls
 
-1. **Interfaces** → Update IVideoMinuteResult and IVideo interfaces
-2. **Migration** → Create 20250823HHMMSS_create_video_minute_results.ts
-3. **DAO Implementation** → Replace VideoMinuteResultDAO mock methods with real database operations
-4. **DAO Enhancement** → Add duration support to VideoDAO
-5. **Exports** → Update index.ts exports
-6. **Build & Test** → Compile TypeScript and run tests
-
-## Query Performance Strategy
-
-### Batch Insert Optimization
-
-```typescript
-async createBatch(videoId: number, minuteResults: IVideoMinuteResultInput[]): Promise<IVideoMinuteResult[]> {
-  const batchData = minuteResults.map(result => ({
-    ...result,
-    videoId,
-    uuid: knex.raw('gen_random_uuid()'),
-    created_at: knex.fn.now(),
-    updated_at: knex.fn.now()
-  }));
-
-  return await this.knex.batchInsert(this.tableName, batchData, 50).returning('*');
-}
-```
+### Query Optimization
 
-### Optimized Minute Range Query
-
-```typescript
-async getMinuteResultsForVideo(
-  videoId: number,
-  startMinute: number = 0,
-  endMinute?: number,
-  page: number = 1,
-  limit: number = 100
-): Promise<IDataPaginator<IVideoMinuteResult>> {
-  const query = this.knex(this.tableName + ' as vmr')
-    .innerJoin('video as v', 'vmr.video_id', 'v.id')
-    .select('vmr.*', 'v.name as video_name', 'v.uuid as video_uuid')
-    .where('vmr.video_id', videoId)
-    .where('vmr.minute_number', '>=', startMinute);
-
-  if (endMinute !== undefined) {
-    query.where('vmr.minute_number', '<=', endMinute);
-  }
-
-  // Standard pagination logic with count optimization
-  const [{ count }] = await query.clone().clearSelect().count('* as count');
-  const offset = (page - 1) * limit;
-  const data = await query.orderBy('vmr.minute_number', 'asc').limit(limit).offset(offset);
-
-  return {
-    success: true,
-    data,
-    page,
-    limit,
-    count: data.length,
-    totalCount: parseInt(count as string),
-    totalPages: Math.ceil(parseInt(count as string) / limit)
-  };
-}
-```
+- Template lookup query uses composite index for O(log n) performance
+- JOIN operations for fetching annotation source data in single query
+- Pagination maintained for all list operations
 
 ## Risks & Validation
 
 ### Migration Safety
 
-- **Risk:** Table creation failure in production
-- **Mitigation:** Test migration on staging with production data volume
-- **Rollback:** `down()` function drops table and column safely
+- **LOW RISK**: Adding nullable column with proper constraints
+- **Rollback plan**: Down migration removes column and constraints cleanly
+- **Data preservation**: No existing data affected
 
 ### Pattern Compliance
 
-- **UUID-only External Communication:** ✅ All external API methods use UUID lookups
-- **Foreign Key Integrity:** ✅ CASCADE delete ensures orphaned records cleanup
-- **Naming Conventions:** ✅ Follows snake_case for DB, camelCase for TypeScript
-- **DAO Methods:** ✅ Standard CRUD + specialized batch operations
-- **Performance:** ✅ Proper indexing for all query patterns
-
-### Data Integrity Validation
-
-```typescript
-async validateMinuteDataIntegrity(videoId: number): Promise<boolean> {
-  const video = await this.knex('video').where('id', videoId).first();
-  const minuteResults = await this.knex(this.tableName).where('video_id', videoId);
-
-  // Aggregate minute results and compare with video.results
-  return this.compareAggregatedResults(video.results, minuteResults);
-}
-```
-
-### Performance Validation
+- **100% compliant** with existing DAO patterns (IBaseDAO implementation)
+- **100% compliant** with naming conventions (snake_case DB, camelCase TS)
+- **100% compliant** with interface patterns (I prefix, proper exports)
+- **100% compliant** with migration patterns (timestamp prefix, up/down functions)
 
-- Batch inserts handle 5-minute chunks efficiently (50-item batches)
-- Indexes support all query patterns: video_id lookups, UUID lookups, minute ranges
-- JOIN operations optimized for N+1 query elimination
-- Pagination implemented with proper count optimization
+### Testing Strategy
 
-This plan ensures 100% pattern compliance while optimizing for the specific use case of minute-by-minute video detection storage with efficient batch processing capabilities.
+- Verify foreign key constraints work correctly
+- Test template discovery with same folder/type filtering
+- Validate NULL handling for videos without annotation sources
+- Performance test template lookup queries with large datasets