json-object-editor 0.10.653 → 0.10.657

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "json-object-editor",
-  "version": "0.10.653",
+  "version": "0.10.657",
   "description": "JOE the Json Object Editor | Platform Edition",
   "main": "app.js",
   "scripts": {

package/readme.md

@@ -5,6 +5,27 @@ JOE is software that allows you to manage data models via JSON objects. There ar
 
 
 
+## What’s new in 0.10.657 (brief)
+- MCP everywhere (prompts, autofill, widget):
+  - **Prompts** (`ai_prompt`): new MCP config block (`mcp_enabled`, `mcp_toolset`, `mcp_selected_tools`, `mcp_instructions_mode`) lets you turn MCP tools on per‑prompt, pick a toolset (`read-only`, `minimal`, `all`, or `custom`), and auto‑generate short tool instructions.
+  - **Autofill fields**: the same MCP keys are now supported under a field’s `ai` config so autofill runs can optionally call MCP tools with the same toolset/playlist model.
+  - **Audit**: `ai_response` records MCP config and actual tool calls (`mcp_tools_used[]`) plus `used_openai_file_ids[]` so you can see which tools and files were used for any run.
+- Uploader file roles + AI‑aware attachments:
+  - Uploader fields can define `file_roles` (e.g. `{ value:'transcript', label:'Intake Transcript', default:true }`) and JOE renders a per‑file role `<select>` that saves to `file_role` on each file object.
+  - `executeJOEAiPrompt` now sends a compact `uploaded_files[]` header (including `itemtype`, `field`, `filename`, `file_role`, and `openai_file_id`) alongside Responses input so prompts can reason about “transcript vs summary” sources while the OpenAI Files integration still handles raw content.
+  - Responses+tools (`runWithTools`) now attaches files on both the initial tool‑planning call and the final answer call, so MCP runs see the same attachments end‑to‑end.
+- History safety:
+  - Hardened `JOE.Storage.save` history diffing to avoid a `craydent-object` edge case where comparing `null`/`undefined` values could throw on `.toString()`. This only affects `_history.changes`, not what is saved.
+
+## What’s new in 0.10.654 (brief)
+- OpenAI Files mirrored on S3 upload; uploader tiles show the `openai_file_id`. Retry upload is available per file.
+- Responses integration improvements:
+  - Per‑prompt `attachments_mode` on `ai_prompt` (`direct` vs `file_search`). Direct sends `input_file` parts; file search auto‑creates a vector store and attaches it.
+  - Safe retry if a model rejects `temperature/top_p` (we strip and retry once).
+  - Select Prompt lists prompts by active status where either `datasets[]` or `content_items[].itemtype` matches the current object.
+  - `ai_response` now shows `used_openai_file_ids` and correctly records `referenced_objects` for Select Prompt runs.
+- UX: “Run AI Prompt” and “Run Thought Agent” buttons disable and pulse while running to avoid double‑submits.
+
 ## Architecture & Mental Model (Server)
 
 - Global JOE
@@ -176,18 +197,67 @@ JOE is software that allows you to manage data models via JSON objects. There ar
 - Each Thought run persists an `ai_response` with `response_type:'thought_generation'`, `referenced_objects:[scope_id]`, and `generated_thoughts[]` containing the ids of created Thought records.
 - In any schema UI you can include core fields `proposeThought` and `ai_responses` to (a) trigger a Thought run for the current object and (b) list all related `ai_response` records for audit and reuse.
 
-## File uploads (S3)
+## File uploads (S3 + OpenAI Files)
 - Uploader field options:
   - `allowmultiple: true|false` — allow selecting multiple files.
   - `url_field: 'image_url'` — on success, sets this property to the remote URL and rerenders that field.
   - `ACL: 'public-read'` — optional per-field ACL. When omitted, server currently defaults to `public-read` (temporary during migration).
 - Flow:
   - Client posts `{ Key, base64, contentType, ACL? }` to `/API/plugin/awsConnect`.
-  - Server uploads with AWS SDK v3 and returns `{ url, Key, bucket, etag }` (HTTP 200).
-  - Client uses `response.url`; if `url_field` is set, it assigns and rerenders that field.
+  - Server uploads to S3 (AWS SDK v3) and, if `OPENAI_API_KEY` is configured, also uploads the same bytes to OpenAI Files (purpose=`assistants`).
+  - Response shape: `{ url, Key, bucket, etag, openai_file_id?, openai_purpose?, openai_error? }`.
+  - Client:
+    - Sets the `url` on the file object; if `url_field` is set on the schema field, it assigns that property and rerenders.
+    - Persists OpenAI metadata on the file object: `openai_file_id`, `openai_purpose`, `openai_status`, `openai_error`.
+    - Renders the OpenAI file id under the filename on each uploader tile. The “OpenAI: OK” banner has been removed.
+    - Shows a per‑file “Upload to OpenAI” / “Retry OpenAI” action when no id is present or when an error occurred. This calls `POST /API/plugin/chatgpt/filesRetryFromUrl` with `{ url, filename, contentType }` and updates the file metadata.
 - Errors:
   - If bucket or region config is missing, server returns 400 with a clear message.
   - If the bucket has ACLs disabled, server returns 400: “Bucket has ACLs disabled… remove ACL or switch to presigned/proxy access.”
+  - If OpenAI upload fails, the uploader shows `OpenAI error: <message>` inline; you can retry from the file row.
+
+- Using OpenAI file ids:
+  - File ids are private; there is no public URL to view them.
+  - Use the OpenAI Files API (with your API key) to retrieve metadata or download content:
+    - Metadata: `GET /v1/files/{file_id}`
+    - Content: `GET /v1/files/{file_id}/content`
+  - Node example:
+    ```js
+    const OpenAI = require('openai');
+    const fs = require('fs');
+    const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
+    const meta = await client.files.retrieve('file_abc123');
+    const stream = await client.files.content('file_abc123');
+    const buf = Buffer.from(await stream.arrayBuffer());
+    fs.writeFileSync('downloaded.bin', buf);
+    ```
+
+### File roles on uploader fields
+- **Schema configuration**:
+  - Any uploader field can declare `file_roles` as an array of `{ value, label?, default? }` objects, for example:
+    - `{ value:'transcript', label:'Intake Transcript', default:true }`
+    - `{ value:'summary', label:'Intake Summary' }`
+  - `label` is optional; it falls back to `value`. At most one role should have `default:true`.
+- **Runtime behavior**:
+  - JOE renders a role `<select>` next to each uploaded file with:
+    - A blank option, and one option per configured role.
+    - The select updates the file object’s `file_role` property in the parent object (e.g. `client.files[].file_role`).
+    - Existing uploads show the role selector on first render as long as `file_roles` is configured on the field.
+  - When OpenAI Files are enabled, uploader files still receive `openai_file_id`, `openai_purpose`, `openai_status`, and `openai_error` as before; `file_role` is an additional, JOE‑level label.
+- **AI integration**:
+  - When running an AI prompt via `executeJOEAiPrompt`, JOE inspects referenced objects for uploader fields and builds an `uploaded_files[]` header:
+    - Each entry includes `{ itemtype, field, name, role, openai_file_id }`.
+    - This header is merged into the user input so prompts can explicitly reason about which files are “transcripts”, “summaries”, etc., while the actual file bytes are attached via Responses `input_file` parts / `file_search` tool resources.
+
+### Related endpoints (server/plugins)
+
+- `POST /API/plugin/awsConnect` – S3 upload (and OpenAI mirror when configured)
+  - Input: `{ Key, base64, contentType, ACL? }`
+  - Output: `{ url, Key, bucket, etag, openai_file_id?, openai_purpose?, openai_error? }`
+
+- `POST /API/plugin/chatgpt/filesRetryFromUrl` – (Re)upload an existing S3 file to OpenAI
+  - Input: `{ url, filename?, contentType? }`
+  - Output: `{ success, openai_file_id?, openai_purpose?, error? }`
 
 ## SERVER/PLATFORM mode
 	check port 2099
@@ -690,4 +760,4 @@ To help you develop and debug the widget + plugin in your instance, JOE exposes
 - Added a Responses‑based tool runner for `<joe-ai-widget>` that wires `ai_assistant.tools` into MCP functions via `chatgpt.runWithTools`.
 - Enhanced widget UX: assistant/user bubble theming (using `assistant_color` and user `color`), inline “tools used this turn” meta messages, and markdown rendering for assistant replies.
 - Expanded the AI widget test page with an assistant picker, live tool JSON viewer, a clickable conversation history list (resume existing `ai_widget_conversation` threads), and safer user handling (widget conversations now store user id/name/color explicitly and OAuth token‑exchange errors from Google are surfaced clearly during login).
-- Added field-level AI autofill support: schemas can declare `ai` config on a field (e.g. `{ name:'ai_summary', type:'rendering', ai:{ prompt:'Summarize the project in a few sentences.' } }`), which renders an inline “AI” button that calls `_joe.Ai.populateField('ai_summary')` and posts to `/API/plugin/chatgpt/autofill` to compute a JSON `patch` and update the UI (with confirmation before overwriting non-empty values).
+- Added field-level AI autofill support: schemas can declare `ai` config on a field (e.g. `{ name:'ai_summary', type:'rendering', ai:{ prompt:'Summarize the project in a few sentences.' } }`), which renders an inline “AI” button that calls `_joe.Ai.populateField('ai_summary')` and posts to `/API/plugin/chatgpt/autofill` to compute a JSON `patch` and update the UI (with confirmation before overwriting non-empty values).

package/server/fields/core.js

@@ -143,7 +143,7 @@ var fields = {
         }
         return 'new item';
     }},
-    status:{type:'select',rerender:'status',icon:'status',
+    status:{type:'select',rerender:'status',icon:'status',reloadable:true,
         after:function(item){
             if(item.joeUpdated){
                 var cont =`
@@ -523,6 +523,7 @@ var fields = {
         type: "select",
         display: "Ai Model",
         values: [
+            {value:"gpt-5.2", name: "GPT-5.2 (Strong, 128k)" },
             { value:"gpt-5.1", name: "GPT-5.1 (Strong, 128k)" },
             { value:"gpt-5", name: "GPT-5 (Strong, 128K)" },
             { value:"gpt-5-mini", name: "GPT-5-mini (Cheap, 1M)" },
@@ -533,11 +534,13 @@ var fields = {
             { value: "gpt-4.1-nano", name: "4.1-nano (Fastest, light tasks)" }
           ],
           tooltip:`Ai Model Guide -
-            GPT-4o is the default for fast, responsive tasks and supports up to 128k tokens. It’s ideal for short completions, summaries, and dynamic UI tools.
+          GPT-5.2 is the default for strong, 128k token tasks. It’s ideal for complex analysis, large datasets, and detailed reasoning.
+          GPT-5-mini is the default for cheap, 1M token tasks. It’s ideal for quick completions, summaries, and dynamic UI tools.
+            GPT-4o is , responsive tasks and supports up to 128k tokens. It’s ideal for short completions, summaries, and dynamic UI tools.
             GPT-4.1 and 4.1-mini support a massive 1 million token context, making them perfect for large inputs like full business profiles, long strategy texts, and multi-object analysis.
             4.1-mini is significantly cheaper than full 4.1, with great balance for most structured AI workflows.
             4.1-nano is best for lightweight classification or routing logic where speed and cost matter more than depth.`,
-        default: "gpt-4o",
+        default: "gpt-5-mini",
       },
     objectChat:{
         type:'button',
@@ -560,10 +563,50 @@ var fields = {
             return _joe.schemas.ai_response.methods.listResponses(obj);
         }
     },
+    select_prompt:{
+        display:'Run AI Prompt',
+        type:'content',
+        reloadable:true,
+        icon:'ai_prompt',
+        run:function(obj){
+            if(!obj || !obj._id){
+                return '<joe-text>Save this item before running AI prompts.</joe-text>';
+            }
+            var itemtype = obj.itemtype || (_joe.current && _joe.current.schema && _joe.current.schema.name) || null;
+            // Active ai_prompt statuses
+            var activeStatuses = (_joe.getDataset('status')||[]).filter(function(s){
+                return Array.isArray(s.datasets) && s.datasets.includes('ai_prompt') && s.active;
+            }).map(function(s){ return s._id; });
+            // Filter prompts by dataset match (datasets[] OR content_items[].itemtype) and active
+            var prompts = (_joe.getDataset('ai_prompt')||[]).filter(function(p){
+                var okStatus = !p.status || activeStatuses.indexOf(p.status) !== -1;
+                var matchByContentItems = (p.content_items||[]).some(function(ci){ return ci && ci.itemtype === itemtype; });
+                var matchByDatasets = Array.isArray(p.datasets) && p.datasets.indexOf(itemtype) !== -1;
+                var okDataset = matchByContentItems || matchByDatasets;
+                return okStatus && okDataset;
+            });
+            var selId = 'select_prompt_'+obj._id;
+            var filesSelId = 'select_prompt_files_'+obj._id;
+            var html = '';
+            html += '<div class="joe-field-comment">Select prompt</div>';
+            html += '<select id="'+selId+'" style="width:100%;">';
+            prompts.forEach(function(p){
+                var name = (p && p.name) || '';
+                html += '<option value="'+p._id+'">'+name+'</option>';
+            });
+            html += '</select>';
+            html += '<div class="joe-field-comment" style="margin-top:8px;">Attach files (optional)</div>';
+            html += '<select id="'+filesSelId+'" multiple class="joe-prompt-select"></select>';
+            html += '<script>(function(){ try{ _joe && _joe.Ai && _joe.Ai.renderFilesSelector && _joe.Ai.renderFilesSelector("'+filesSelId+'",{ cap:10, disableWithoutOpenAI:true }); }catch(e){} })();</script>';
+			html += '<joe-button class="joe-button joe-ai-button joe-iconed-button" onclick="_joe.Ai.runPromptSelection(this,\''+obj._id+'\',\''+selId+'\',\''+filesSelId+'\')">Run AI Prompt</joe-button>';
+            return html;
+        }
+    },
     proposeThought:{
         display:'Propose Thought',
         type:'content',
         reloadable:true,
+        icon:'ai_thought',
         run:function(obj){
             if (!obj || !obj._id) {
                 return '<joe-text>Save this item before proposing Thoughts.</joe-text>';
@@ -584,9 +627,14 @@ var fields = {
                 'Avoid meta-thoughts about prompts or schemas.'
             );
             var taId = 'propose_thought_prompt_' + obj._id;
+            var selId = 'propose_thought_files_' + obj._id;
             var html = '';
             html += '<div class="joe-field-comment">Thought prompt</div>';
-            html += '<textarea id="'+taId+'" style="width:100%;min-height:80px;">'+defaultPrompt+'</textarea>';
+            html += '<textarea id="'+taId+'" class="joe-prompt-textarea">'+defaultPrompt+'</textarea>';
+            // Attach files selector (optional)
+            html += '<div class="joe-field-comment" style="margin-top:8px;">Attach files (optional)</div>';
+            html += '<select id="'+selId+'" class="joe-prompt-select" multiple></select>';
+            html += '<script>(function(){ try{ _joe && _joe.Ai && _joe.Ai.renderFilesSelector && _joe.Ai.renderFilesSelector("'+selId+'",{ cap:10, disableWithoutOpenAI:true }); }catch(e){} })();</script>';
             // For now, use the generic Thought agent; scope_id is the current object id.
             var args = "'" + obj._id + "','" + taId + "'";
             if (fieldDef && fieldDef.model) {
@@ -594,8 +642,8 @@ var fields = {
                 var m = String(fieldDef.model).replace(/'/g, "\\'");
                 args += ",'" + m + "'";
             }
-            html += '<joe-button class="joe-button joe-ai-button joe-iconed-button" ';
-            html += 'onclick="_joe.Ai.runProposeThought('+ args +')">Run Thought Agent</joe-button>';
+			html += '<joe-button class="joe-button joe-ai-button joe-iconed-button" ';
+			html += 'onclick="_joe.Ai.runProposeThought(this,'+ args +')">Run Thought Agent</joe-button>';
             return html;
         }
     },