json-object-editor 0.10.657 → 0.10.660

package/CHANGELOG.md

@@ -1,4 +1,4 @@
-## 0.10.657
+## 0.10.660
 
 - MCP as an optional layer on all AI surfaces
   - `ai_prompt`: added MCP config block (`mcp_enabled`, `mcp_toolset`, `mcp_selected_tools`, `mcp_instructions_mode`) so prompts can opt into MCP tools with named toolsets (`read-only`, `minimal`, `all`, `custom`) and auto‑generated tool instructions.
@@ -10,6 +10,10 @@
   - `runWithTools` now applies `attachFilesToResponsesPayload` to both the initial tool‑planning call and the final Responses call (including the token‑limit retry), so MCP runs consistently see file attachments.
 - Storage / history robustness
   - Hardened `JOE.Storage.save` history diffing by sanitizing cached vs current documents before `$c.changes(...)` to avoid a `craydent-object.equals` edge case where comparing `null`/`undefined` could throw on `.toString()`. This only affects `_history.changes`, not saved documents.
+ - Object chat unification
+   - The legacy assistants-based `objectChat` flow has been replaced with a floating `<joe-ai-widget>` shell (with `<joe-ai-assistant-picker>`), so object chat uses the same widget + MCP stack as AIHub.
+   - `ai_widget_conversation` now supports scoped chats via `scope_itemtype`/`scope_id` and tracks attached OpenAI Files via `attached_openai_file_ids` and `attached_files_meta`.
+   - On the first turn of an object-scoped chat, the server preloads a slimmed `understandObject` snapshot for the scoped object and merges object files + assistant files into the `openai_file_ids` passed to `runWithTools`, so assistants immediately understand which record they’re talking about and can reason over its files.
 
 ## 0.10.654
 

package/docs/React_Form_Integration_Strategy.md

@@ -0,0 +1,398 @@
+# React Form Integration Strategy for JOE
+
+## Executive Summary
+
+This document outlines strategy options for integrating:
+1. **JSON-based form definitions** (from `form-qs.json` format)
+2. **React Single Page Application** as a page type in JOE
+3. **Form submission storage** into JOE's submission system
+
+---
+
+## Current State Analysis
+
+### JOE Form System (Current)
+- **Schema**: `form` schema with `sections` (objectList), `questions` (references to question schema)
+- **Rendering**: `formBuilder` plugin renders HTML forms server-side
+- **Submission**: POST to `/API/plugin/formBuilder/submission` with `{formid, submission: {...}}`
+- **Storage**: Creates `submission` items with `data`, `form_name`, `form` (reference), `visitor`, `submission_date`
+
+### JOE Page System (Current)
+- **Content Types**: `wysiwyg`, `code`, `plugin`, `module`
+- **Module Type**: Server-side JS function that returns HTML string
+- **Includes**: Can load JS/CSS files via `include` schema (served at `/_include/{id}`)
+- **Layouts**: Templates with `${this.PAGE.content}` placeholder
+
+### Target Requirements
+1. Use JSON file (`form-qs.json`) for form structure and rules
+2. Embed React SPA as a page in JOE
+3. Submit to JOE submission storage on form completion
+
+---
+
+## Strategy Options
+
+### **Option 1: React SPA as Code Page with API Bridge** ⭐ **RECOMMENDED**
+
+#### Architecture
+- **Page Type**: `content_type: 'code'`
+- **React App**: Compiled/bundled React app loaded via includes or inline
+- **JSON Loading**: React app fetches form definition JSON via API endpoint
+- **Submission**: React app posts to JOE submission API endpoint
+
+#### Implementation Details
+
+**1. Create Form Definition API Endpoint**
+```javascript
+// server/plugins/formDefinition.js
+this.get = function(req, res) {
+    var formId = req.query.formid || req.params.formid;
+    // Load form-qs.json format, return as JSON
+    // Could be stored as include or in a new form_definition schema
+}
+```
+
+**2. Create Submission API Endpoint** (extend existing or new)
+```javascript
+// Extend server/plugins/formBuilder.js or create formSubmission.js
+this.submit = function(data, req) {
+    // Accept JSON form submission, map to JOE submission format
+    // Create submission record with form reference
+}
+```
+
+**3. React App Integration**
+- React app bundled (webpack/vite) and stored as an `include` (JS file)
+- Page content: `<div id="react-form-root"></div><script src="/_include/{react-bundle-id}"></script>`
+- React app:
+  - Fetches form definition from `/API/plugin/formDefinition/get?formid={id}`
+  - Renders form using `react-form-spa-ex.js` logic
+  - Submits to `/API/plugin/formBuilder/submission` (or new endpoint)
+
+#### Pros
+- ✅ Leverages existing JOE infrastructure
+- ✅ React app is independent and can be versioned/updated separately
+- ✅ Works with existing submission storage
+- ✅ Minimal changes to JOE core
+- ✅ Can reuse existing form validation/storage logic
+
+#### Cons
+- ⚠️ Requires bundling React app separately
+- ⚠️ Two API endpoints to maintain (form definition + submission)
+- ⚠️ React app needs to know JOE API structure
+
+#### Estimated Effort
+- **Low-Medium**: ~2-3 days
+  - API endpoint for form definitions: 4-6 hours
+  - Submission endpoint extension: 2-4 hours
+  - React app integration/hooking: 4-6 hours
+  - Testing: 4-6 hours
+
+---
+
+### **Option 2: New "spa" Content Type with Form Plugin**
+
+#### Architecture
+- **Page Type**: New `content_type: 'spa'` option
+- **Form Plugin**: New `reactForm` plugin that handles React app bootstrapping
+- **Configuration**: Page stores React component path and form definition reference
+
+#### Implementation Details
+
+**1. Extend Page Schema**
+```javascript
+// server/schemas/page.js
+{ name:'content_type', type:'string', enumValues:['wysiwyg','code','plugin','module','spa'] }
+// Add fields: spa_component, form_definition_id
+```
+
+**2. Create React Form Plugin**
+```javascript
+// server/plugins/reactForm.js
+this.render = function(data, req) {
+    var page = data.page;
+    var formDef = loadFormDefinition(page.form_definition_id);
+    // Return HTML with:
+    // - React libraries (CDN or includes)
+    // - Form definition as JSON in <script> tag
+    // - React component mount point
+    // - Submission handler that calls JOE API
+}
+```
+
+**3. Submission Handling**
+- Plugin generates submission handler that posts to existing formBuilder endpoint
+- Maps JSON form format to JOE submission format
+
+#### Pros
+- ✅ Native JOE integration
+- ✅ Form definitions can be stored as JOE items
+- ✅ Consistent with other page content types
+- ✅ Can be configured through JOE UI
+
+#### Cons
+- ⚠️ Requires schema changes to page
+- ⚠️ More complex implementation
+- ⚠️ Need to handle React bundling/delivery
+
+#### Estimated Effort
+- **Medium**: ~4-5 days
+  - Schema extensions: 2-4 hours
+  - React form plugin: 1-2 days
+  - React bundling strategy: 4-8 hours
+  - UI for configuring SPA pages: 4-6 hours
+  - Testing: 1 day
+
+---
+
+### **Option 3: Module Type with Server-Side React Rendering**
+
+#### Architecture
+- **Page Type**: `content_type: 'module'`
+- **SSR**: Use React server-side rendering (via `react-dom/server`)
+- **Hydration**: Client-side React app for interactivity
+
+#### Implementation Details
+
+**1. Module Function**
+```javascript
+// Page content (module type)
+module.exports = async function(content) {
+    var React = require('react');
+    var ReactDOMServer = require('react-dom/server');
+    var FormApp = require('./path/to/react-form-spa-ex.js');
+    var formDef = loadFormDefinition(content.PAGE.meta.form_definition_id);
+    
+    var html = ReactDOMServer.renderToString(
+        React.createElement(FormApp, { formDefinition: formDef })
+    );
+    
+    return `<div id="root">${html}</div>
+            <script>window.__FORM_DEF__ = ${JSON.stringify(formDef)};</script>
+            <script src="/_include/{react-bundle-id}"></script>`;
+}
+```
+
+**2. Client Hydration**
+- React app mounts and hydrates the server-rendered HTML
+- Handles form submission to JOE API
+
+#### Pros
+- ✅ SEO-friendly (server-rendered)
+- ✅ Faster initial render
+- ✅ Uses existing module type
+
+#### Cons
+- ⚠️ Requires React SSR dependencies on server
+- ⚠️ More complex build/deployment
+- ⚠️ Node.js React dependencies add overhead
+- ⚠️ Hydration complexity
+
+#### Estimated Effort
+- **Medium-High**: ~5-7 days
+  - SSR setup: 1-2 days
+  - Hydration logic: 1 day
+  - Build configuration: 1 day
+  - Testing/debugging: 2-3 days
+
+---
+
+### **Option 4: Hybrid - Form Definition Schema + React Include**
+
+#### Architecture
+- **New Schema**: `form_definition` schema to store JSON form definitions
+- **Form Mapping**: Bridge between JSON form format and JOE form/question schemas
+- **Page**: Code or module type that loads React app and form definition
+
+#### Implementation Details
+
+**1. Create Form Definition Schema**
+```javascript
+// server/schemas/form_definition.js
+{
+    name: 'form_definition',
+    fields: [
+        { name: 'form_id', type: 'string' }, // matches form-qs.json formId
+        { name: 'definition', type: 'object' }, // full form-qs.json structure
+        { name: 'form', type: 'string', isReference: true } // link to JOE form
+    ]
+}
+```
+
+**2. Form Bridge Service**
+```javascript
+// server/plugins/formBridge.js
+// Convert form-qs.json format to JOE form/questions
+// Or: Store JSON format, map on-the-fly for submissions
+```
+
+**3. React App**
+- Loads form definition from API
+- Renders form
+- Submits in format compatible with JOE submissions
+
+#### Pros
+- ✅ JSON form definitions stored as first-class JOE items
+- ✅ Can sync/import from external JSON files
+- ✅ Separation of concerns (definition vs. rendering)
+- ✅ Can potentially auto-generate JOE forms from JSON
+
+#### Cons
+- ⚠️ Additional schema to maintain
+- ⚠️ Need mapping/bridge logic
+- ⚠️ More moving parts
+
+#### Estimated Effort
+- **Medium**: ~4-5 days
+  - Form definition schema: 4-6 hours
+  - Bridge/mapping logic: 1-2 days
+  - React integration: 1 day
+  - Testing: 1 day
+
+---
+
+## Recommended Approach: **Option 1 with Option 4 Enhancements**
+
+### Phase 1: Quick Win (Option 1)
+1. Create API endpoint to serve form definitions (store JSON in includes or files)
+2. Extend formBuilder submission endpoint to accept JSON format submissions
+3. Create React app that:
+   - Loads form definition from API
+   - Renders form (based on react-form-spa-ex.js)
+   - Submits to JOE endpoint
+4. Embed React app in a JOE page via includes
+
+### Phase 2: Enhanced (Option 4 additions)
+1. Create `form_definition` schema to store JSON form definitions in JOE
+2. Add UI to import/manage form definitions
+3. Optionally: Auto-create JOE forms from JSON definitions for reporting/admin
+
+### Why This Approach?
+- ✅ **Fastest to implement**: Leverages existing infrastructure
+- ✅ **Low risk**: Minimal changes to core JOE systems
+- ✅ **Flexible**: Can evolve to Option 4 later without breaking changes
+- ✅ **Testable**: Can test React app independently
+
+---
+
+## Technical Implementation Details
+
+### JSON Form Format Mapping
+
+The `form-qs.json` format has:
+- `formId`, `formName`, `version`
+- `sections[]` with `fields[]`
+- `visibility` rules with `whenAll`/`whenAny` conditions
+- `dataExport` settings
+
+**Mapping to JOE Submissions:**
+```javascript
+{
+    itemtype: 'submission',
+    form: {joe_form_id}, // reference to JOE form
+    form_name: formDef.formName,
+    data: {
+        // Flattened field values
+        first_name: "...",
+        last_name: "...",
+        // ... all field values
+    },
+    submission_date: new Date().toISOString(),
+    visitor: visitor_id, // from form if configured
+    source: req.headers.referer
+}
+```
+
+### React App Submission Handler
+
+```javascript
+async function submitForm(formData, formId) {
+    const response = await fetch('/API/plugin/formBuilder/submission', {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({
+            formid: formId, // JOE form ID
+            submission: formData // { first_name: "...", ... }
+        })
+    });
+    return response.json();
+}
+```
+
+### Form Definition API
+
+```javascript
+// GET /API/plugin/formDefinition/get?formid={id}
+// Returns: form-qs.json format
+{
+    formId: "...",
+    formName: "...",
+    sections: [...],
+    settings: {...}
+}
+```
+
+---
+
+## Migration Path
+
+1. **Week 1**: Implement Option 1 (API endpoints + React integration)
+2. **Week 2**: Test with example form, refine submission mapping
+3. **Week 3** (Optional): Implement Option 4 enhancements (form_definition schema)
+4. **Week 4**: Documentation and production deployment
+
+---
+
+## Open Questions
+
+1. **Form Definition Storage**: Store JSON in includes, files, or new schema?
+   - **Recommendation**: Start with includes/files, move to schema in Phase 2
+
+2. **React Bundling**: Use CDN, bundle via build process, or serve from includes?
+   - **Recommendation**: Bundle React app, store as include for versioning
+
+3. **Form ID Mapping**: How to map `form-qs.json` `formId` to JOE `form._id`?
+   - **Recommendation**: Add `form_json_id` field to form schema, or store mapping in form_definition
+
+4. **Visitor Tracking**: How to identify visitors in React SPA context?
+   - **Recommendation**: Use existing visitor_id_field mechanism, pass via form definition
+
+5. **Validation**: Use React-side validation only, or also JOE-side?
+   - **Recommendation**: Client-side for UX, server-side for security (both)
+
+---
+
+## Next Steps
+
+1. **Decision**: Review options and select approach
+2. **Prototype**: Build minimal POC of selected option
+3. **Refine**: Iterate based on prototype feedback
+4. **Implement**: Full implementation
+5. **Test**: Comprehensive testing with example forms
+6. **Deploy**: Production deployment
+
+---
+
+## Appendix: File Structure
+
+```
+server/
+  plugins/
+    formDefinition.js  (new - serves JSON form definitions)
+    formBuilder.js     (extend - handle JSON submissions)
+    
+schemas/
+  form_definition.js   (optional - Phase 2)
+
+includes/
+  react-form-bundle.js (bundled React app)
+
+pages/
+  react-form-example   (page with React form)
+```
+
+---
+
+**Document Version**: 1.0  
+**Date**: 2026-01-04  
+**Author**: AI Assistant + User Collaboration