@things-factory/dataset 9.1.18 → 9.1.19

package/EXPLORATION_INDEX.md

@@ -0,0 +1,252 @@
+# Frontend Architecture Exploration - Document Index
+
+## Overview
+This package contains comprehensive documentation of the Things-Factory dataset client-side frontend architecture, specifically focused on understanding how data entry forms are structured and how to implement media input components (image, video, audio).
+
+## Generated Documents
+
+### 1. FRONTEND_ARCHITECTURE.md (COMPREHENSIVE)
+- **Purpose**: Complete technical reference
+- **Contents**:
+  - Input component architecture and patterns
+  - Data entry form system deep dive
+  - DataItemType enumeration and structures
+  - Component hierarchy and integration
+  - File upload handling (client + server)
+  - Media validation utilities
+  - Component registration and bootstrap
+  - Subgroup form system (Grist tables)
+  - File/media options schema with examples
+  - Implementation guide for new media components
+  - Complete usage flow with diagrams
+  - Directory structure summary
+- **Best for**: Understanding the complete architecture, implementing new components, architecture decisions
+
+### 2. QUICK_REFERENCE.md (CONCISE)
+- **Purpose**: Quick lookup and checklists
+- **Contents**:
+  - File location quick lookup
+  - 5-step data entry flow
+  - Type mapping reference
+  - Key functions and classes
+  - Component creation checklist
+  - Value extraction mechanisms
+  - DataItem configuration examples
+  - GraphQL context explanation
+  - Common gotchas and pitfalls
+  - Testing checklist
+- **Best for**: Quick lookups during implementation, finding specific functions, remembering patterns
+
+### 3. EXPLORATION_SUMMARY.txt (THIS FILE)
+- **Purpose**: High-level findings and absolute paths
+- **Contents**:
+  - Key findings summary
+  - Critical file paths with absolute paths
+  - Component architecture flow
+  - Type-to-component mapping
+  - Key methods to understand
+  - DataItemType enum definition
+  - Media validation function reference
+  - File upload flow end-to-end
+  - Implementation requirements
+- **Best for**: Getting oriented, understanding what needs to be implemented, seeing the big picture
+
+## File Locations Reference
+
+### In This Package
+```
+/Users/super/Documents/GitHub/things-factory/packages/dataset/
+├── FRONTEND_ARCHITECTURE.md      (Comprehensive reference)
+├── QUICK_REFERENCE.md            (Quick lookup guide)
+├── EXPLORATION_INDEX.md          (This file)
+├── client/
+│   ├── components/
+│   │   ├── data-entry-form.ts
+│   │   └── checklist-entry-form.ts
+│   ├── activities/
+│   │   └── activity-data-collect-edit.ts
+│   └── bootstrap.ts
+└── server/
+    ├── service/data-set/
+    │   └── data-item-type.ts
+    ├── utils/
+    │   └── media-validation.ts
+    └── controllers/
+        └── create-data-sample.ts
+```
+
+### Node Modules (Dependencies)
+```
+node_modules/@operato/dataset/dist/src/
+├── ox-data-entry-form.js
+├── ox-data-entry-subgroup-form.js
+└── ox-data-input-factory.js
+
+node_modules/@operato/input/dist/src/
+├── ox-form-field.d.ts|.js
+├── ox-input-file.d.ts|.js
+├── ox-input-image.d.ts|.js
+└── ox-input-signature.d.ts|.js
+```
+
+## How to Use These Documents
+
+### For Understanding the Architecture
+1. Start with: **EXPLORATION_SUMMARY.txt** - High-level overview
+2. Then read: **FRONTEND_ARCHITECTURE.md** - Deep technical details
+3. Reference: **QUICK_REFERENCE.md** - For specific lookups
+
+### For Implementation
+1. Check: **QUICK_REFERENCE.md** - "Adding New Media Components - Checklist"
+2. Reference: **FRONTEND_ARCHITECTURE.md** - "Implementation Guide: Adding Media Components"
+3. Look at: **QUICK_REFERENCE.md** - DataItem configuration examples
+
+### For Debugging
+1. Consult: **QUICK_REFERENCE.md** - "Common Gotchas" section
+2. Trace: **FRONTEND_ARCHITECTURE.md** - "Current Usage Flow" section
+3. Check: **QUICK_REFERENCE.md** - "Testing Checklist"
+
+## Key Findings Summary
+
+### Current State
+- Input components for file and signature types exist
+- Media types (image, video, audio) are defined in enum but not rendered
+- OxDataInputFactory uses switch statement to map types to components
+- Factory is missing 'image', 'video', and 'audio' cases
+- Server-side validation and file handling fully implemented
+
+### What's Missing
+- OxInputImage component
+- OxInputVideo component
+- OxInputAudio component
+- Factory mappings for these three types
+- Component imports in appropriate locations
+
+### Implementation Effort
+- Create three components: 2-3 hours
+- Update factory: 30 minutes
+- Add imports: 15 minutes
+- End-to-end testing: 1-2 hours
+- **Total**: ~4-6 hours for complete implementation
+
+## Architecture Patterns
+
+### Component Creation Pattern
+All input components:
+1. Extend OxFormField (from @operato/input)
+2. Use @customElement decorator for registration
+3. Implement required methods: render, updated, firstUpdated, _onChangeValue, _notifyChange
+4. Set `name` attribute on native input for form extraction
+5. Handle FormDataEvent for form submission integration
+
+### Data Flow Pattern
+1. Activity receives dataSetId
+2. DataSet fetched with dataItems
+3. OxDataEntryForm renders per dataItems
+4. OxDataInputFactory.createInputElement() maps type to component
+5. Form collects values via name attribute
+6. GraphQL mutation with hasUpload context
+7. Server validates and creates attachments
+8. File paths stored in DataSample.data
+
+### Factory Pattern
+OxDataInputFactory uses static methods:
+1. createInputElement(type, tag, value, options, idx) - Routes to correct component
+2. mapGristType(type) - Maps to Grist column types
+3. getGristRecordOptions(type, options) - Grid cell configuration
+
+## Critical Code Snippets
+
+### How Form Extracts Values
+```typescript
+// Location: ox-data-entry-form.js line ~89
+const editors = Array.from(
+  this.renderRoot.querySelectorAll(`[name=${tag}]`)
+)
+// Looks for elements with name="product_image", etc.
+```
+
+### How Types are Mapped
+```javascript
+// Location: ox-data-input-factory.js
+static createInputElement(type, tag, value, options = {}, idx = 0) {
+  switch (type) {
+    case 'image':
+      // TO BE ADDED
+    case 'video':
+      // TO BE ADDED
+    case 'audio':
+      // TO BE ADDED
+    // ... existing cases ...
+  }
+}
+```
+
+### Server File Upload Handling
+```typescript
+// Location: create-data-sample.ts line ~122
+if (isFileOrMediaType(dataItem.type)) {
+  const filesArray = data[tag]
+  // Process and validate files
+  // Create attachments
+  // Replace data[tag] with path info
+}
+```
+
+## Next Steps
+
+### For Implementation
+1. Read **QUICK_REFERENCE.md** - "Adding New Media Components - Checklist"
+2. Create three component files based on OxInputFile pattern
+3. Update OxDataInputFactory in @operato/dataset (or create wrapper)
+4. Add component imports to wrapper components
+5. Test end-to-end with example DataSet
+
+### For Maintenance
+1. Keep documentation in sync with code changes
+2. Update type mappings when new types added
+3. Document new media options in data-item-type.ts
+4. Maintain backward compatibility with existing components
+
+### For Learning
+1. Study OxInputFile implementation for pattern
+2. Review ox-data-entry-form.js rendering logic
+3. Trace complete user flow in activity component
+4. Understand FormDataEvent integration
+5. Review GraphQL multipart/form-data setup
+
+## Questions and Clarifications
+
+### Q: Why does OxInputImage exist but isn't used?
+A: OxInputImage exists in @operato/input but the factory doesn't route to it. Need to add the mapping in OxDataInputFactory.
+
+### Q: How are values extracted from complex components?
+A: Via the `name` attribute in DOM queries. All OxInput* components must set name on their native input.
+
+### Q: Why `hasUpload: true` in GraphQL context?
+A: Tells Apollo Client to use multipart/form-data encoding instead of JSON for file streaming.
+
+### Q: Where do file paths get stored?
+A: In DataSample.data[tag] as array of { id, mimetype, name, fullpath } objects, not the actual files.
+
+### Q: How are media validations handled?
+A: Server-side in create-data-sample.ts using validateMediaFile() utility before attachment creation.
+
+## Document Versions
+
+- **Created**: 2024-11-09
+- **Exploration Scope**: Very Thorough
+- **Focus Area**: Things-Factory Dataset Client Frontend Architecture
+- **Specific Goal**: Understanding media component implementation requirements
+
+## Additional Resources
+
+- @operato/input package documentation
+- @operato/dataset package documentation
+- Lit web components documentation
+- GraphQL upload specifications
+- Material Design components (md-icon usage)
+
+---
+
+**Navigation**: Start with the executive summary, then dive into detailed docs as needed. Use QUICK_REFERENCE.md during active development.