@things-factory/dataset 9.1.18 → 10.0.0-beta.1

package/QUICK_REFERENCE.md

@@ -0,0 +1,365 @@
+# Quick Reference: Frontend Component Architecture
+
+## File Locations - Quick Lookup
+
+### Client-Side Components (Your Repo)
+```
+/Users/super/Documents/GitHub/things-factory/packages/dataset/client/
+├── components/
+│   ├── data-entry-form.ts          - Form wrapper using ox-data-entry-form
+│   └── checklist-entry-form.ts     - Checklist wrapper using ox-checklist-entry-form
+└── activities/
+    └── activity-data-collect-edit.ts - Workflow activity entry point
+```
+
+### Server-Side Types & Utils (Your Repo)
+```
+/Users/super/Documents/GitHub/things-factory/packages/dataset/server/
+├── service/data-set/
+│   └── data-item-type.ts           - DataItemType enum + media options schema
+└── utils/
+    └── media-validation.ts         - Media file validation utilities
+```
+
+### Core Components (Dependency - Node Modules)
+```
+@operato/dataset (in node_modules)
+├── ox-data-entry-form.js          - Main form component
+├── ox-data-entry-subgroup-form.js - Grouped items in Grist table
+└── ox-data-input-factory.js        - Type-to-component mapping factory
+
+@operato/input (in node_modules)
+├── ox-input-file.js               - File upload component
+├── ox-input-image.js              - Image upload component
+├── ox-input-signature.js           - Digital signature component
+└── ox-form-field.js               - Base class for all input components
+```
+
+---
+
+## How Data Entry Works - 5 Step Flow
+
+### Step 1: User Opens Activity
+File: `activity-data-collect-edit.ts`
+- Gets `dataSetId` from workflow
+- Fetches DataSet with dataItems via GraphQL
+
+### Step 2: OxDataEntryForm Renders
+File: `@operato/dataset/ox-data-entry-form.js`
+- Iterates through `dataItems.filter(item => item.active)`
+- For grouped items: renders `ox-data-entry-subgroup-form`
+- For non-grouped: calls `OxDataInputFactory.createInputElement()`
+
+### Step 3: OxDataInputFactory Creates Input
+File: `@operato/dataset/ox-data-input-factory.js`
+- Switch statement on `dataItem.type`
+- Currently supports: text, number, boolean, date, datetime, select, radio, file, signature
+- **MISSING:** image, video, audio
+
+### Step 4: User Submits Form
+- Form collects values from all inputs by `name` attribute
+- `buildValue()` extracts values into object: `{ [tag]: any }`
+- GraphQL mutation sends with `context: { hasUpload: true }`
+
+### Step 5: Server Processes Files
+File: `create-data-sample.ts`
+- Iterates `dataItems` looking for `isFileOrMediaType()`
+- If media type: validates with `validateMediaFile()`
+- Creates attachments and stores file paths in data
+
+---
+
+## Type Mapping (OxDataInputFactory)
+
+### Current Types
+```
+'select'    → <select name=${tag}> with options
+'radio'     → <label><input type="radio"> with options
+'boolean'   → <input type="checkbox">
+'number'    → <input type="number">
+'date'      → <input type="date">
+'datetime'  → <input type="datetime-local">
+'file'      → <ox-input-file multiple>
+'signature' → <ox-input-signature>
+'text'      → <input type="text"> (default)
+```
+
+### Missing Types (Need Implementation)
+```
+'image'     → <ox-input-image>      (TO BE CREATED)
+'video'     → <ox-input-video>      (TO BE CREATED)
+'audio'     → <ox-input-audio>      (TO BE CREATED)
+```
+
+---
+
+## Key Functions & Classes
+
+### OxFormField (Base Class)
+**Properties:**
+- `name: string` - Form field identifier
+- `value: T` - Field value (generic)
+- `disabled: boolean` - Disabled state
+
+**Methods:**
+- `connectedCallback()` - Register with parent form
+- `appendFormData(e: FormDataEvent)` - Add to form submission
+
+### OxDataEntryForm
+**Key Methods:**
+- `buildInputs()` - Generate all input elements
+- `buildValue()` - Extract form values as object
+- `extractValuesFromInputs()` - Query DOM by name attribute
+
+### OxDataInputFactory
+**Static Methods:**
+- `createInputElement(type, tag, value, options, idx)` - Route to correct component
+- `mapGristType(type)` - Convert to Grist column type
+- `getGristRecordOptions(type, options)` - Grid cell configuration
+
+### Media Validation Utils
+**Functions:**
+- `validateMediaFile(file, mediaType, options)` - Main validator
+- `isValidMimeType(mediaType, mimetype)` - Check MIME type
+- `isAcceptedFormat(filename, acceptFormats)` - Check extension
+- `isValidFileSize(fileSize, maxSize)` - Check size
+- `isMediaType(type)` - Check if image/video/audio
+
+---
+
+## Adding New Media Components - Checklist
+
+### 1. Create Component File
+```
+Location: packages/dataset/client/components/ox-input-image.ts
+Extend: OxFormField
+Pattern: Similar to ox-input-file
+Methods needed:
+  - render()
+  - updated()
+  - firstUpdated()
+  - _onChangeValue()
+  - _notifyChange()
+```
+
+### 2. Update OxDataInputFactory
+File: Update @operato/dataset source (or create wrapper)
+Add to switch:
+```javascript
+case 'image': return html`<ox-input-image ...>`;
+case 'video': return html`<ox-input-video ...>`;
+case 'audio': return html`<ox-input-audio ...>`;
+```
+
+### 3. Add to mapGristType()
+```javascript
+case 'image': return 'image';
+case 'video': return 'video';
+case 'audio': return 'audio';
+```
+
+### 4. Add to getGristRecordOptions()
+```javascript
+case 'image':
+case 'video':
+case 'audio':
+  return { multiple: options.multiple || false };
+```
+
+### 5. Import in Components
+```typescript
+import '@operato/dataset/ox-input-image.js'
+import '@operato/dataset/ox-input-video.js'
+import '@operato/dataset/ox-input-audio.js'
+```
+
+---
+
+## Component Value Extraction (Important!)
+
+### How Form Gets Values
+```typescript
+// In ox-data-entry-form.js, line ~89
+const editors = Array.from(
+  this.renderRoot.querySelectorAll(`[name=${tag}]`)
+)
+// Extracts value from: name="product_image"
+```
+
+### Why Name Attribute Matters
+- Form values keyed by `name` attribute
+- Must match `dataItem.tag` value
+- All OxInput* components must set name on native input
+
+### File Upload Special Handling
+Files passed as: `[{ file: FileUpload }, ...]`
+Backend converts to: `[{ id, mimetype, name, fullpath }, ...]`
+
+---
+
+## DataItem Configuration Examples
+
+### Image Field Config
+```typescript
+{
+  name: 'Product Image',
+  tag: 'product_image',        // ← Used as name attribute
+  type: 'image',               // ← Routes to OxInputImage
+  description: 'Upload product photo',
+  unit: 'JPG/PNG',
+  quota: 3,                    // Allow up to 3 images
+  options: {
+    accept: ['jpeg', 'png'],
+    maxSize: 10485760,         // 10MB
+    maxWidth: 4096,
+    maxHeight: 4096,
+    multiple: true
+  }
+}
+```
+
+### Video Field Config
+```typescript
+{
+  name: 'Demo Video',
+  tag: 'demo_video',
+  type: 'video',
+  quota: 1,
+  options: {
+    accept: ['mp4', 'webm'],
+    maxSize: 104857600,        // 100MB
+    maxDuration: 300,          // 5 minutes
+    multiple: false
+  }
+}
+```
+
+### Audio Field Config
+```typescript
+{
+  name: 'Voice Note',
+  tag: 'voice_note',
+  type: 'audio',
+  quota: 1,
+  options: {
+    accept: ['mp3', 'wav'],
+    maxSize: 52428800,         // 50MB
+    maxDuration: 600,          // 10 minutes
+    multiple: false
+  }
+}
+```
+
+---
+
+## File Upload GraphQL Context
+
+### Why `hasUpload: true`?
+```typescript
+// In data-entry-form.ts
+context: {
+  hasUpload: true
+}
+```
+This tells Apollo Client to:
+1. Use `multipart/form-data` encoding (not JSON)
+2. Include files in GraphQL variables
+3. Stream files to server
+
+### GraphQL Variable Structure
+```typescript
+{
+  dataSample: {
+    dataSet: { id: '...' },
+    data: {
+      product_image: [{file: FileUpload}, ...],
+      demo_video: [{file: FileUpload}],
+      ...
+    }
+  }
+}
+```
+
+---
+
+## Common Gotchas
+
+### 1. OxInputImage vs OxInputFile
+- **File**: Generic file upload
+- **Image**: Image-specific with preview/validation
+- Note: OxInputImage exists in @operato/input but not used in factory
+
+### 2. Name Attribute is Critical
+```typescript
+// CORRECT - matches dataItem.tag
+<ox-input-image name="product_image" ...></ox-input-image>
+
+// WRONG - value won't be extracted
+<ox-input-image ...></ox-input-image>
+```
+
+### 3. Quota vs Multiple
+```typescript
+quota: 3, options: { multiple: true }
+// Allows user to select 3 times, each time multiple files
+
+quota: 1, options: { multiple: false }
+// Single file selection
+```
+
+### 4. Group vs Subgroup
+```typescript
+group: 'measurements'     // Groups items in table (Grist)
+subgroup: 'length'        // Secondary grouping
+// Non-grouped items show in main form
+```
+
+### 5. Active vs Hidden
+```typescript
+active: true   // Rendered in form
+active: false  // Skipped entirely
+hidden: true   // Rendered but CSS hidden (for validation specs)
+```
+
+---
+
+## File Paths in Codebase
+
+### Where to Make Changes
+1. **New Components**: `packages/dataset/client/components/`
+2. **Type Updates**: Update @operato/dataset OR create wrapper
+3. **Media Options**: Already defined in `data-item-type.ts`
+4. **Validation**: Already implemented in `media-validation.ts`
+
+### Dependencies Already Available
+```json
+{
+  "@operato/input": "^9.0.0",
+  "@operato/dataset": "^9.0.0",
+  "lit": "^2.x.x"
+}
+```
+
+### Key Imports for New Components
+```typescript
+import { css, html } from 'lit'
+import { customElement, property, query } from 'lit/decorators.js'
+import { OxFormField } from '@operato/input'
+```
+
+---
+
+## Testing Checklist
+
+Before deploying new media components:
+
+- [ ] Component renders with correct icon/label
+- [ ] File selection works (click and drag-drop)
+- [ ] Value extracted correctly by name attribute
+- [ ] Multiple/single file toggle works
+- [ ] GraphQL mutation includes files
+- [ ] Server-side validation passes
+- [ ] Media validation errors displayed to user
+- [ ] Attachment created in database
+- [ ] File paths stored in DataSample.data
+- [ ] Grist grid column type correct for subgroup items

package/dist-client/pages/data-archive/data-archive-list-page.d.ts

@@ -4,13 +4,7 @@ import '@operato/context/ox-context-page-toolbar.js';
 import { FetchOption } from '@operato/data-grist';
 import { PageView } from '@operato/shell';
 import './data-archive-request-popup.js';
-declare const DataArchiveListPage_base: (new (...args: any[]) => {
-    _storeUnsubscribe: import("redux").Unsubscribe;
-    connectedCallback(): void;
-    disconnectedCallback(): void;
-    stateChanged(_state: unknown): void;
-    readonly isConnected: boolean;
-}) & (new (...args: any[]) => import("lit").LitElement) & typeof PageView;
+declare const DataArchiveListPage_base: (new (...args: any[]) => import("lit").LitElement) & typeof PageView;
 export declare class DataArchiveListPage extends DataArchiveListPage_base {
     static styles: import("lit").CSSResult[];
     gristConfig: any;

package/dist-client/pages/data-archive/data-archive-list-page.js

@@ -5,16 +5,15 @@ import '@operato/context/ox-context-page-toolbar.js';
 import gql from 'graphql-tag';
 import { css, html } from 'lit';
 import { customElement, query, state } from 'lit/decorators.js';
-import { connect } from 'pwa-helpers/connect-mixin';
 import { DataGrist } from '@operato/data-grist';
 import { client } from '@operato/graphql';
 import { i18next, localize } from '@operato/i18n';
 import { openPopup } from '@operato/layout';
-import { PageView, store } from '@operato/shell';
+import { PageView } from '@operato/shell';
 import { CommonHeaderStyles, ScrollbarStyles } from '@operato/styles';
 import { isMobileDevice } from '@operato/utils';
 import './data-archive-request-popup.js';
-let DataArchiveListPage = class DataArchiveListPage extends connect(store)(localize(i18next)(PageView)) {
+let DataArchiveListPage = class DataArchiveListPage extends localize(i18next)(PageView) {
     constructor() {
         super(...arguments);
         this.mode = isMobileDevice() ? 'CARD' : 'GRID';
@@ -23,51 +22,41 @@ let DataArchiveListPage = class DataArchiveListPage extends connect(store)(local
         ScrollbarStyles,
         CommonHeaderStyles,
         css `
-      :host {
-        display: flex;
+      :host {         display: flex;
         flex-direction: column;
 
         overflow: hidden;
-      }
+ }
 
-      ox-grist {
-        overflow-y: auto;
+      ox-grist {         overflow-y: auto;
         flex: 1;
-      }
+ }
 
-      .header {
-        grid-template-areas: 'filters actions';
-      }
+      .header {         grid-template-areas: 'filters actions';
+ }
     `
     ]; }
     get context() {
-        return {
-            title: i18next.t('title.data-archive list'),
-            search: {
-                handler: (search) => {
+        return { title: i18next.t('title.data-archive list'),
+            search: { handler: (search) => {
                     this.grist.searchText = search;
                 },
-                value: this.grist?.searchText || ''
-            },
-            filter: {
-                handler: () => {
+                value: this.grist?.searchText || '' },
+            filter: { handler: () => {
                     this.grist.toggleHeadroom();
                 }
             },
             help: 'dataset/data-archive',
             actions: [
-                {
-                    title: i18next.t('button.request-archive'),
+                { title: i18next.t('button.request-archive'),
                     action: this.openArchivePopup.bind(this),
                     icon: 'archive'
                 }
             ],
-            exportable: {
-                name: i18next.t('title.data-archive list'),
+            exportable: { name: i18next.t('title.data-archive list'),
                 data: this._exportableData.bind(this)
             },
-            toolbar: false
-        };
+            toolbar: false };
     }
     render() {
         const mode = this.mode || (isMobileDevice() ? 'LIST' : 'GRID');
@@ -89,78 +78,65 @@ let DataArchiveListPage = class DataArchiveListPage extends connect(store)(local
     `;
     }
     async pageInitialized(lifecycle) {
-        this.gristConfig = {
-            list: { fields: ['updater', 'updatedAt'] },
+        this.gristConfig = { list: { fields: ['updater', 'updatedAt'] },
             columns: [
                 { type: 'gutter', gutterName: 'sequence' },
                 // { type: 'gutter', gutterName: 'row-selector', multiple: true },
-                {
-                    type: 'resource-object',
+                { type: 'resource-object',
                     name: 'creator',
                     header: i18next.t('field.creator'),
                     sortable: true,
                     width: 120,
                     imex: true
                 },
-                {
-                    type: 'datetime',
+                { type: 'datetime',
                     name: 'createdAt',
                     header: i18next.t('field.created_at'),
                     sortable: true,
                     width: 180,
                     imex: true
                 },
-                {
-                    type: 'datetime',
+                { type: 'datetime',
                     name: 'updatedAt',
                     header: i18next.t('field.updated_at'),
                     sortable: true,
                     width: 180,
                     imex: true
                 },
-                {
-                    type: 'json5',
+                { type: 'json5',
                     name: 'requestParams',
                     header: i18next.t('field.request-params'),
-                    record: {
-                        editable: false
+                    record: { editable: false
                     },
                     width: 200,
                     imex: true
                 },
-                {
-                    type: 'string',
+                { type: 'string',
                     name: 'downloadUrl',
                     label: true,
                     header: i18next.t('field.download-url'),
-                    record: {
-                        editable: false
+                    record: { editable: false
                     },
                     width: 240,
                     imex: true
                 },
-                {
-                    type: 'string',
+                { type: 'string',
                     name: 'status',
                     label: true,
                     header: i18next.t('field.status'),
-                    record: {
-                        editable: false
+                    record: { editable: false
                     },
                     sortable: true,
                     width: 120,
                     imex: true
                 }
             ],
-            rows: {
-                appendable: false,
-                selectable: {
-                    multiple: true
+            rows: { appendable: false,
+                selectable: { multiple: true
                 }
             },
             sorters: [
-                {
-                    name: 'createdAt',
+                { name: 'createdAt',
                     desc: true
                 }
             ]
@@ -174,46 +150,37 @@ let DataArchiveListPage = class DataArchiveListPage extends connect(store)(local
         this.popup = openPopup(html ` <data-archive-request-popup
         @requested=${this.closePopupAndRefesh.bind(this)}
         @created=${this.closePopupAndRefesh.bind(this)}
-      ></data-archive-request-popup>`, {
-            backdrop: true,
+      ></data-archive-request-popup>`, { backdrop: true,
             size: 'small',
             title: i18next.t('title.data-archive request popup')
         });
     }
     async fetchHandler({ page, limit, sortings = [], filters = [] }) {
-        const response = await client.query({
-            query: gql `
-        query ($filters: [Filter!], $pagination: Pagination, $sortings: [Sorting!]) {
-          responses: dataArchives(filters: $filters, pagination: $pagination, sortings: $sortings) {
-            items {
-              id
+        const response = await client.query({ query: gql `
+        query ($filters: [Filter!], $pagination: Pagination, $sortings: [Sorting!]) {           responses: dataArchives(filters: $filters, pagination: $pagination, sortings: $sortings) {             items {               id
               type
               requestParams
               downloadUrl
               status
-              creator {
-                id
+              creator {                 id
                 name
-              }
-              updater {
-                id
+ }
+              updater {                 id
                 name
-              }
+ }
               updatedAt
               createdAt
-            }
+ }
             total
-          }
-        }
+ }
+ }
       `,
-            variables: {
-                filters,
+            variables: { filters,
                 pagination: { page, limit },
                 sortings
             }
         });
-        return {
-            total: response.data.responses.total || 0,
+        return { total: response.data.responses.total || 0,
             records: response.data.responses.items || []
         };
     }
@@ -229,8 +196,7 @@ let DataArchiveListPage = class DataArchiveListPage extends connect(store)(local
             .filter(column => column.type !== 'gutter' && column.record !== undefined && column.imex !== undefined)
             .map(column => {
             return column.imex === true
-                ? {
-                    header: column.header.renderer(column),
+                ? { header: column.header.renderer(column),
                     key: column.name,
                     width: column.width,
                     type: column.type
@@ -238,8 +204,7 @@ let DataArchiveListPage = class DataArchiveListPage extends connect(store)(local
                 : column.imex;
         });
         var data = records.map(item => {
-            return {
-                id: item.id,
+            return { id: item.id,
                 ...this.gristConfig.columns
                     .filter(column => column.type !== 'gutter' && column.record !== undefined && column.imex !== undefined)
                     .reduce((record, column) => {