@agent-native/core 0.79.27 → 0.80.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/corpus/README.md +2 -2
- package/corpus/core/CHANGELOG.md +26 -0
- package/corpus/core/docs/content/locales/ar-SA/template-design.mdx +41 -0
- package/corpus/core/docs/content/locales/de-DE/template-design.mdx +51 -0
- package/corpus/core/docs/content/locales/es-ES/template-design.mdx +50 -0
- package/corpus/core/docs/content/locales/fr-FR/template-design.mdx +49 -0
- package/corpus/core/docs/content/locales/hi-IN/template-design.mdx +44 -0
- package/corpus/core/docs/content/locales/ja-JP/template-design.mdx +46 -0
- package/corpus/core/docs/content/locales/ko-KR/template-design.mdx +44 -0
- package/corpus/core/docs/content/locales/pt-BR/template-design.mdx +48 -0
- package/corpus/core/docs/content/locales/zh-CN/template-design.mdx +36 -0
- package/corpus/core/docs/content/locales/zh-TW/template-design.mdx +38 -0
- package/corpus/core/docs/content/template-design.mdx +45 -0
- package/corpus/core/package.json +2 -1
- package/corpus/core/src/agent/engine/index.ts +1 -0
- package/corpus/core/src/agent/engine/registry.ts +40 -0
- package/corpus/core/src/agent/production-agent.ts +6 -2
- package/corpus/core/src/cli/design-connect.ts +490 -0
- package/corpus/core/src/cli/index.ts +16 -0
- package/corpus/core/src/cli/skills.ts +126 -3
- package/corpus/core/src/client/AssistantChat.tsx +43 -7
- package/corpus/core/src/client/agent-chat.ts +27 -3
- package/corpus/core/src/client/blocks/library/MermaidBlock.tsx +1 -1
- package/corpus/core/src/client/blocks/library/diagram.tsx +2 -2
- package/corpus/core/src/client/blocks/library/wireframe.tsx +1 -1
- package/corpus/core/src/client/chat/message-components.tsx +1 -1
- package/corpus/core/src/client/chat-model-groups.ts +1 -2
- package/corpus/core/src/client/components/LiveCursorOverlay.tsx +33 -66
- package/corpus/core/src/client/composer/TiptapComposer.tsx +99 -24
- package/corpus/core/src/client/composer/useVoiceDictation.ts +429 -260
- package/corpus/core/src/client/frame-protocol.ts +17 -2
- package/corpus/core/src/client/rich-markdown-editor/RegistryBlockNode.tsx +2 -2
- package/corpus/core/src/client/sharing/ShareButton.tsx +20 -15
- package/corpus/core/src/client/use-agent-engine-configured.ts +80 -28
- package/corpus/core/src/server/transcribe-voice.ts +57 -22
- package/corpus/core/src/shared/mcp-embed-headers.ts +23 -1
- package/corpus/core/src/styles/agent-native.css +56 -7
- package/corpus/core/src/templates/workspace-core/.agents/skills/voice-transcription/SKILL.md +10 -0
- package/corpus/core/src/vite/client.ts +2 -0
- package/corpus/core/src/voice/index.ts +18 -0
- package/corpus/core/src/voice/voice-cleanup-prompt.ts +38 -0
- package/corpus/core/src/voice/voice-context.ts +217 -0
- package/corpus/core/src/voice/voice-replacements.ts +75 -0
- package/corpus/templates/analytics/AGENTS.md +4 -3
- package/corpus/templates/analytics/actions/list-session-recordings.ts +6 -1
- package/corpus/templates/analytics/app/components/layout/Sidebar.tsx +557 -400
- package/corpus/templates/analytics/app/global.css +0 -4
- package/corpus/templates/analytics/app/i18n/zh-TW.ts +2 -0
- package/corpus/templates/analytics/app/i18n-data.ts +20 -0
- package/corpus/templates/analytics/app/pages/adhoc/sql-dashboard/index.tsx +33 -8
- package/corpus/templates/analytics/app/pages/sessions/SessionDetailPage.tsx +7 -0
- package/corpus/templates/analytics/app/pages/sessions/SessionsPage.tsx +7 -0
- package/corpus/templates/analytics/changelog/2026-06-28-ask-tab-now-matches-the-darker-app-background.md +6 -0
- package/corpus/templates/analytics/changelog/2026-06-28-dashboard-email-reports-now-use-cleaner-light-mode-snapshots.md +6 -0
- package/corpus/templates/analytics/changelog/2026-06-28-left-sidebar-collapses-into-an-animated-icon-rail.md +6 -0
- package/corpus/templates/analytics/changelog/2026-06-28-sessions-now-show-only-signed-in-recordings-with-playable-re.md +6 -0
- package/corpus/templates/analytics/seeds/dashboards/agent-native-templates-first-party.json +3 -3
- package/corpus/templates/analytics/server/lib/dashboard-report.ts +29 -7
- package/corpus/templates/analytics/server/lib/first-party-metric-catalog.ts +5 -2
- package/corpus/templates/analytics/server/lib/session-replay.ts +24 -4
- package/corpus/templates/assets/app/components/layout/Sidebar.tsx +4 -4
- package/corpus/templates/assets/changelog/2026-06-28-left-sidebar-collapse-motion-and-footer-chrome-are-quieter.md +6 -0
- package/corpus/templates/brain/app/components/layout/Layout.tsx +29 -2
- package/corpus/templates/brain/app/components/layout/Sidebar.tsx +131 -42
- package/corpus/templates/brain/app/i18n/zh-TW.ts +2 -0
- package/corpus/templates/brain/app/i18n-data.ts +20 -0
- package/corpus/templates/brain/changelog/2026-06-28-left-sidebar-collapses-into-an-animated-icon-rail.md +6 -0
- package/corpus/templates/calendar/app/components/layout/AppLayout.tsx +29 -1
- package/corpus/templates/calendar/app/components/layout/Sidebar.tsx +417 -299
- package/corpus/templates/calendar/app/i18n/zh-TW.ts +2 -0
- package/corpus/templates/calendar/app/i18n-data.ts +20 -0
- package/corpus/templates/calendar/changelog/2026-06-28-left-sidebar-collapses-into-an-animated-icon-rail.md +6 -0
- package/corpus/templates/chat/app/components/layout/Sidebar.tsx +6 -19
- package/corpus/templates/chat/changelog/2026-06-28-left-sidebar-collapse-motion-and-footer-chrome-are-quieter.md +6 -0
- package/corpus/templates/clips/AGENTS.md +4 -0
- package/corpus/templates/clips/actions/cleanup-transcript.ts +39 -9
- package/corpus/templates/clips/actions/finalize-meeting.ts +81 -12
- package/corpus/templates/clips/actions/lib/agents-md-context.ts +4 -1
- package/corpus/templates/clips/app/components/library/library-layout.tsx +3 -3
- package/corpus/templates/clips/app/components/meetings/share-meeting-dialog.tsx +1 -1
- package/corpus/templates/clips/app/components/player/share-dialog.tsx +2 -2
- package/corpus/templates/clips/changelog/2026-06-28-left-sidebar-footer-controls-use-less-divider-chrome.md +6 -0
- package/corpus/templates/clips/desktop/src/lib/personal-vocabulary.ts +13 -8
- package/corpus/templates/clips/desktop/src/lib/voice-dictation.ts +60 -1
- package/corpus/templates/content/app/components/editor/DocumentDatabase.tsx +1 -1
- package/corpus/templates/content/app/components/editor/database/DatabaseView.tsx +1 -1
- package/corpus/templates/content/app/components/sidebar/DocumentSidebar.tsx +5 -5
- package/corpus/templates/content/changelog/2026-06-28-left-sidebar-collapse-motion-and-footer-chrome-are-quieter.md +6 -0
- package/corpus/templates/design/.agents/skills/visual-edit/SKILL.md +137 -0
- package/corpus/templates/design/AGENTS.md +95 -5
- package/corpus/templates/design/DESIGN.md +82 -0
- package/corpus/templates/design/DEVELOPING.md +43 -0
- package/corpus/templates/design/actions/add-localhost-screens.ts +426 -0
- package/corpus/templates/design/actions/apply-shader.ts +277 -0
- package/corpus/templates/design/actions/apply-visual-edit.ts +303 -0
- package/corpus/templates/design/actions/connect-localhost.ts +183 -0
- package/corpus/templates/design/actions/edit-design.ts +195 -42
- package/corpus/templates/design/actions/generate-design.ts +71 -1
- package/corpus/templates/design/actions/generate-screens.ts +193 -0
- package/corpus/templates/design/actions/get-code-layer-projection.ts +191 -0
- package/corpus/templates/design/actions/get-shader.ts +119 -0
- package/corpus/templates/design/actions/list-localhost-connections.ts +95 -0
- package/corpus/templates/design/actions/navigate.ts +55 -2
- package/corpus/templates/design/actions/view-screen.ts +68 -1
- package/corpus/templates/design/agent-native.app-skill.json +10 -0
- package/corpus/templates/design/app/components/design/CanvasContextMenu.tsx +761 -0
- package/corpus/templates/design/app/components/design/DesignCanvas.tsx +2057 -103
- package/corpus/templates/design/app/components/design/DesignEditorSkeleton.tsx +74 -38
- package/corpus/templates/design/app/components/design/DesignExtensionsPanel.tsx +399 -0
- package/corpus/templates/design/app/components/design/EditPanel.tsx +3185 -439
- package/corpus/templates/design/app/components/design/LayersPanel.tsx +1124 -0
- package/corpus/templates/design/app/components/design/MultiScreenCanvas.tsx +3116 -147
- package/corpus/templates/design/app/components/design/QuestionFlow.tsx +700 -13
- package/corpus/templates/design/app/components/design/TweaksPanel.tsx +107 -63
- package/corpus/templates/design/app/components/design/index.ts +0 -5
- package/corpus/templates/design/app/components/design/inspector/AlignmentMatrix.tsx +286 -0
- package/corpus/templates/design/app/components/design/inspector/AutoLayoutMatrix.tsx +699 -0
- package/corpus/templates/design/app/components/design/inspector/ConstraintsWidget.tsx +461 -0
- package/corpus/templates/design/app/components/design/inspector/ExportSettingsPanel.tsx +166 -0
- package/corpus/templates/design/app/components/design/inspector/FigmaColorPicker.tsx +1420 -0
- package/corpus/templates/design/app/components/design/inspector/SHADER_INTEGRATION.md +420 -0
- package/corpus/templates/design/app/components/design/inspector/ScrubInput.tsx +215 -0
- package/corpus/templates/design/app/components/design/inspector/ShaderControls.stories.tsx +71 -0
- package/corpus/templates/design/app/components/design/inspector/ShaderControls.tsx +518 -0
- package/corpus/templates/design/app/components/design/inspector/ShaderEffectRow.tsx +106 -0
- package/corpus/templates/design/app/components/design/inspector/figma-icons.tsx +361 -0
- package/corpus/templates/design/app/components/design/inspector/index.ts +60 -0
- package/corpus/templates/design/app/components/design/inspector/scrub-input-utils.ts +201 -0
- package/corpus/templates/design/app/components/design/types.ts +23 -0
- package/corpus/templates/design/app/components/layout/Layout.tsx +24 -13
- package/corpus/templates/design/app/components/layout/Sidebar.tsx +4 -4
- package/corpus/templates/design/app/components/visual-editor/CanvasCommentPins.tsx +474 -36
- package/corpus/templates/design/app/components/visual-editor/DrawOverlay.tsx +22 -5
- package/corpus/templates/design/app/components/visual-editor/SaveStatusIndicator.tsx +24 -11
- package/corpus/templates/design/app/global.css +121 -3
- package/corpus/templates/design/app/hooks/use-agent-generating.ts +4 -0
- package/corpus/templates/design/app/hooks/use-navigation-state.ts +130 -8
- package/corpus/templates/design/app/hooks/useDesignHotkeys.ts +329 -0
- package/corpus/templates/design/app/i18n/zh-TW.ts +264 -0
- package/corpus/templates/design/app/i18n-data.ts +2790 -1
- package/corpus/templates/design/app/pages/DesignEditor.tsx +5108 -851
- package/corpus/templates/design/changelog/2026-06-28-annotate-mode-now-combines-drawing-and-comment-pins-in-one-c.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-design-editor-panel-borders-are-cleaner-so-the-canvas-edge-s.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-drawing-prompts-now-reliably-open-and-submit-to-the-design-a.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-editor-app-menus-now-use-an-outline-agent-native-mark-with-b.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-figma-style-code-design-mode.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-layer-selection-now-updates-chat-context-without-opening-the.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-layer-selections-now-preserve-hidden-and-locked-canvas-state.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-left-sidebar-collapse-motion-and-footer-chrome-are-quieter.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-live-cursors-now-keep-names-beside-a-cleaner-pointer-marker.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-mode-switching-now-lives-in-a-compact-bottom-right-icon-dock.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-screen-overview-now-handles-zooming-drawing-opening-and-dele.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-screen-overview-now-selects-screens-on-click-and-opens-them-.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-screen-overview-zooms-into-the-active-screen-with-smoother-e.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-selected-element-chips-in-chat-are-shorter-and-easier-to-sca.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-the-design-editor-toolbar-is-more-compact-and-the-app-menu-m.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-tweaks-now-live-at-the-top-of-the-design-sidebar-instead-of-.md +6 -0
- package/corpus/templates/design/changelog/2026-06-28-tweaks-now-live-in-the-right-inspector-beside-the-design-tab.md +6 -0
- package/corpus/templates/design/changelog/2026-06-29-active-sidebar-rows-use-soft-highlights.md +6 -0
- package/corpus/templates/design/changelog/2026-06-29-agents-can-now-open-screen-overview-focus-screens-and-add.md +6 -0
- package/corpus/templates/design/changelog/2026-06-29-design-agents-now-edit-broad-copy-changes-more-reliably.md +6 -0
- package/corpus/templates/design/changelog/2026-06-29-design-editor-extensions-now-run-inline-beside-the-artboard.md +6 -0
- package/corpus/templates/design/changelog/2026-06-29-sidebars-use-darker-chrome-and-chat-warns-immediately.md +6 -0
- package/corpus/templates/design/changelog/2026-06-29-visual-edit-can-open-localhost-routes-as-url-backed-screens.md +6 -0
- package/corpus/templates/design/package.json +1 -0
- package/corpus/templates/design/server/db/schema.ts +26 -0
- package/corpus/templates/design/server/lib/design-snapshot.ts +1 -1
- package/corpus/templates/design/server/plugins/agent-chat.ts +3 -0
- package/corpus/templates/design/shared/api.ts +27 -0
- package/corpus/templates/design/shared/canvas-frames.ts +103 -0
- package/corpus/templates/design/shared/canvas-math.ts +1058 -0
- package/corpus/templates/design/shared/code-layer.ts +2212 -0
- package/corpus/templates/design/shared/color-utils.ts +180 -0
- package/corpus/templates/design/shared/generation-session.ts +61 -0
- package/corpus/templates/design/shared/shader-presets.ts +782 -0
- package/corpus/templates/design/shared/shader-safety.ts +137 -0
- package/corpus/templates/design/shared/source-mode.ts +237 -0
- package/corpus/templates/forms/app/components/layout/Sidebar.tsx +149 -6
- package/corpus/templates/forms/app/i18n/ar-SA.ts +2 -0
- package/corpus/templates/forms/app/i18n/de-DE.ts +2 -0
- package/corpus/templates/forms/app/i18n/en-US.ts +2 -0
- package/corpus/templates/forms/app/i18n/es-ES.ts +2 -0
- package/corpus/templates/forms/app/i18n/fr-FR.ts +2 -0
- package/corpus/templates/forms/app/i18n/hi-IN.ts +2 -0
- package/corpus/templates/forms/app/i18n/ja-JP.ts +2 -0
- package/corpus/templates/forms/app/i18n/ko-KR.ts +2 -0
- package/corpus/templates/forms/app/i18n/pt-BR.ts +2 -0
- package/corpus/templates/forms/app/i18n/zh-CN.ts +2 -0
- package/corpus/templates/forms/app/i18n/zh-TW.ts +2 -0
- package/corpus/templates/forms/changelog/2026-06-28-left-sidebar-collapses-into-an-animated-icon-rail.md +6 -0
- package/corpus/templates/macros/app/components/layout/AppLayout.tsx +2 -2
- package/corpus/templates/macros/changelog/2026-06-28-left-sidebar-footer-controls-use-less-divider-chrome.md +6 -0
- package/corpus/templates/mail/app/components/layout/AppLayout.tsx +408 -247
- package/corpus/templates/mail/app/i18n/ar-SA.ts +4 -0
- package/corpus/templates/mail/app/i18n/de-DE.ts +4 -0
- package/corpus/templates/mail/app/i18n/en-US.ts +4 -0
- package/corpus/templates/mail/app/i18n/es-ES.ts +4 -0
- package/corpus/templates/mail/app/i18n/fr-FR.ts +4 -0
- package/corpus/templates/mail/app/i18n/hi-IN.ts +4 -0
- package/corpus/templates/mail/app/i18n/ja-JP.ts +4 -0
- package/corpus/templates/mail/app/i18n/ko-KR.ts +4 -0
- package/corpus/templates/mail/app/i18n/pt-BR.ts +4 -0
- package/corpus/templates/mail/app/i18n/zh-CN.ts +4 -0
- package/corpus/templates/mail/app/i18n/zh-TW.ts +4 -0
- package/corpus/templates/mail/changelog/2026-06-28-pinned-sidebar-collapses-into-an-animated-icon-rail.md +6 -0
- package/corpus/templates/plan/app/components/layout/Sidebar.tsx +6 -8
- package/corpus/templates/plan/app/components/plan/wireframe/Wireframe.tsx +1 -1
- package/corpus/templates/plan/app/global.css +61 -78
- package/corpus/templates/plan/app/pages/PlansPage.tsx +8 -0
- package/corpus/templates/plan/changelog/2026-06-28-left-sidebar-collapse-motion-and-footer-chrome-are-quieter.md +6 -0
- package/corpus/templates/plan/changelog/2026-06-28-plans-use-the-shared-neutral-light-and-dark-theme.md +6 -0
- package/corpus/templates/plan/changelog/2026-06-28-sending-open-feedback-from-a-plan-now-opens-the-inline-plan-.md +6 -0
- package/corpus/templates/slides/app/components/layout/Sidebar.tsx +5 -5
- package/corpus/templates/slides/changelog/2026-06-28-left-sidebar-collapse-motion-and-footer-chrome-are-quieter.md +6 -0
- package/corpus/templates/videos/app/components/Sidebar.tsx +2 -2
- package/corpus/templates/videos/app/components/layout/Layout.tsx +29 -2
- package/corpus/templates/videos/app/components/layout/NavSidebar.tsx +113 -30
- package/corpus/templates/videos/app/i18n/ar-SA.ts +2 -0
- package/corpus/templates/videos/app/i18n/de-DE.ts +2 -0
- package/corpus/templates/videos/app/i18n/en-US.ts +2 -0
- package/corpus/templates/videos/app/i18n/es-ES.ts +2 -0
- package/corpus/templates/videos/app/i18n/fr-FR.ts +2 -0
- package/corpus/templates/videos/app/i18n/hi-IN.ts +2 -0
- package/corpus/templates/videos/app/i18n/ja-JP.ts +2 -0
- package/corpus/templates/videos/app/i18n/ko-KR.ts +2 -0
- package/corpus/templates/videos/app/i18n/pt-BR.ts +2 -0
- package/corpus/templates/videos/app/i18n/zh-CN.ts +2 -0
- package/corpus/templates/videos/app/i18n/zh-TW.ts +2 -0
- package/corpus/templates/videos/changelog/2026-06-28-left-sidebar-collapses-into-an-animated-icon-rail.md +6 -0
- package/dist/agent/engine/index.d.ts +1 -1
- package/dist/agent/engine/index.d.ts.map +1 -1
- package/dist/agent/engine/index.js +1 -1
- package/dist/agent/engine/index.js.map +1 -1
- package/dist/agent/engine/registry.d.ts +9 -0
- package/dist/agent/engine/registry.d.ts.map +1 -1
- package/dist/agent/engine/registry.js +37 -0
- package/dist/agent/engine/registry.js.map +1 -1
- package/dist/agent/production-agent.d.ts.map +1 -1
- package/dist/agent/production-agent.js +4 -3
- package/dist/agent/production-agent.js.map +1 -1
- package/dist/cli/design-connect.d.ts +52 -0
- package/dist/cli/design-connect.d.ts.map +1 -0
- package/dist/cli/design-connect.js +409 -0
- package/dist/cli/design-connect.js.map +1 -0
- package/dist/cli/index.js +15 -0
- package/dist/cli/index.js.map +1 -1
- package/dist/cli/skills.d.ts +3 -0
- package/dist/cli/skills.d.ts.map +1 -1
- package/dist/cli/skills.js +121 -3
- package/dist/cli/skills.js.map +1 -1
- package/dist/client/AssistantChat.d.ts.map +1 -1
- package/dist/client/AssistantChat.js +38 -8
- package/dist/client/AssistantChat.js.map +1 -1
- package/dist/client/agent-chat.d.ts +6 -0
- package/dist/client/agent-chat.d.ts.map +1 -1
- package/dist/client/agent-chat.js +17 -3
- package/dist/client/agent-chat.js.map +1 -1
- package/dist/client/blocks/library/MermaidBlock.js +1 -1
- package/dist/client/blocks/library/MermaidBlock.js.map +1 -1
- package/dist/client/blocks/library/diagram.js +1 -1
- package/dist/client/blocks/library/diagram.js.map +1 -1
- package/dist/client/blocks/library/wireframe.js +1 -1
- package/dist/client/blocks/library/wireframe.js.map +1 -1
- package/dist/client/chat/message-components.js +1 -1
- package/dist/client/chat/message-components.js.map +1 -1
- package/dist/client/chat-model-groups.d.ts.map +1 -1
- package/dist/client/chat-model-groups.js +1 -2
- package/dist/client/chat-model-groups.js.map +1 -1
- package/dist/client/components/LiveCursorOverlay.d.ts +1 -2
- package/dist/client/components/LiveCursorOverlay.d.ts.map +1 -1
- package/dist/client/components/LiveCursorOverlay.js +19 -40
- package/dist/client/components/LiveCursorOverlay.js.map +1 -1
- package/dist/client/composer/TiptapComposer.d.ts +3 -1
- package/dist/client/composer/TiptapComposer.d.ts.map +1 -1
- package/dist/client/composer/TiptapComposer.js +86 -24
- package/dist/client/composer/TiptapComposer.js.map +1 -1
- package/dist/client/composer/useVoiceDictation.d.ts +9 -0
- package/dist/client/composer/useVoiceDictation.d.ts.map +1 -1
- package/dist/client/composer/useVoiceDictation.js +154 -32
- package/dist/client/composer/useVoiceDictation.js.map +1 -1
- package/dist/client/frame-protocol.d.ts +17 -2
- package/dist/client/frame-protocol.d.ts.map +1 -1
- package/dist/client/frame-protocol.js.map +1 -1
- package/dist/client/rich-markdown-editor/RegistryBlockNode.js +2 -2
- package/dist/client/rich-markdown-editor/RegistryBlockNode.js.map +1 -1
- package/dist/client/sharing/ShareButton.d.ts +2 -0
- package/dist/client/sharing/ShareButton.d.ts.map +1 -1
- package/dist/client/sharing/ShareButton.js +7 -6
- package/dist/client/sharing/ShareButton.js.map +1 -1
- package/dist/client/use-agent-engine-configured.d.ts +5 -0
- package/dist/client/use-agent-engine-configured.d.ts.map +1 -1
- package/dist/client/use-agent-engine-configured.js +51 -25
- package/dist/client/use-agent-engine-configured.js.map +1 -1
- package/dist/collab/awareness.d.ts +2 -2
- package/dist/collab/awareness.d.ts.map +1 -1
- package/dist/collab/routes.d.ts +1 -1
- package/dist/file-upload/actions/upload-image.d.ts +2 -2
- package/dist/notifications/routes.d.ts +3 -3
- package/dist/observability/routes.d.ts +8 -8
- package/dist/progress/routes.d.ts +1 -1
- package/dist/server/agent-engine-api-key-route.d.ts +1 -1
- package/dist/server/transcribe-voice.d.ts.map +1 -1
- package/dist/server/transcribe-voice.js +40 -24
- package/dist/server/transcribe-voice.js.map +1 -1
- package/dist/shared/mcp-embed-headers.d.ts +1 -0
- package/dist/shared/mcp-embed-headers.d.ts.map +1 -1
- package/dist/shared/mcp-embed-headers.js +20 -1
- package/dist/shared/mcp-embed-headers.js.map +1 -1
- package/dist/styles/agent-native.css +56 -7
- package/dist/templates/workspace-core/.agents/skills/voice-transcription/SKILL.md +10 -0
- package/dist/vite/client.d.ts.map +1 -1
- package/dist/vite/client.js +2 -0
- package/dist/vite/client.js.map +1 -1
- package/dist/voice/index.d.ts +5 -0
- package/dist/voice/index.d.ts.map +1 -0
- package/dist/voice/index.js +4 -0
- package/dist/voice/index.js.map +1 -0
- package/dist/voice/voice-cleanup-prompt.d.ts +6 -0
- package/dist/voice/voice-cleanup-prompt.d.ts.map +1 -0
- package/dist/voice/voice-cleanup-prompt.js +26 -0
- package/dist/voice/voice-cleanup-prompt.js.map +1 -0
- package/dist/voice/voice-context.d.ts +25 -0
- package/dist/voice/voice-context.d.ts.map +1 -0
- package/dist/voice/voice-context.js +156 -0
- package/dist/voice/voice-context.js.map +1 -0
- package/dist/voice/voice-replacements.d.ts +4 -0
- package/dist/voice/voice-replacements.d.ts.map +1 -0
- package/dist/voice/voice-replacements.js +52 -0
- package/dist/voice/voice-replacements.js.map +1 -0
- package/docs/content/locales/ar-SA/template-design.mdx +41 -0
- package/docs/content/locales/de-DE/template-design.mdx +51 -0
- package/docs/content/locales/es-ES/template-design.mdx +50 -0
- package/docs/content/locales/fr-FR/template-design.mdx +49 -0
- package/docs/content/locales/hi-IN/template-design.mdx +44 -0
- package/docs/content/locales/ja-JP/template-design.mdx +46 -0
- package/docs/content/locales/ko-KR/template-design.mdx +44 -0
- package/docs/content/locales/pt-BR/template-design.mdx +48 -0
- package/docs/content/locales/zh-CN/template-design.mdx +36 -0
- package/docs/content/locales/zh-TW/template-design.mdx +38 -0
- package/docs/content/template-design.mdx +45 -0
- package/package.json +2 -1
- package/src/templates/workspace-core/.agents/skills/voice-transcription/SKILL.md +10 -0
- package/corpus/templates/design/app/components/design/DesignToolbar.tsx +0 -339
- package/corpus/templates/design/app/components/design/PresentMode.tsx +0 -68
- package/corpus/templates/design/app/components/design/ViewportTabs.tsx +0 -149
- package/corpus/templates/design/app/components/design/ZoomControls.tsx +0 -157
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import type { VoiceContextPack } from "@agent-native/core/voice";
|
|
1
2
|
import { invoke } from "@tauri-apps/api/core";
|
|
2
3
|
import { emit, listen } from "@tauri-apps/api/event";
|
|
3
4
|
|
|
@@ -5,6 +6,7 @@ import { applyBacktrack } from "./backtrack";
|
|
|
5
6
|
import {
|
|
6
7
|
configureVocabularyClient,
|
|
7
8
|
loadVocabulary,
|
|
9
|
+
loadVocabularyEntries,
|
|
8
10
|
recordPasteForLearn,
|
|
9
11
|
} from "./personal-vocabulary";
|
|
10
12
|
import {
|
|
@@ -312,6 +314,7 @@ async function transcribe(
|
|
|
312
314
|
providerPref: ServerVoiceProvider,
|
|
313
315
|
controller: AbortController,
|
|
314
316
|
instructions?: string,
|
|
317
|
+
contextPack?: VoiceContextPack,
|
|
315
318
|
): Promise<string> {
|
|
316
319
|
const audioBlob = new Blob(chunks, { type: mimeType });
|
|
317
320
|
const form = new FormData();
|
|
@@ -329,6 +332,7 @@ async function transcribe(
|
|
|
329
332
|
if (trimmedInstructions) {
|
|
330
333
|
form.append("instructions", trimmedInstructions);
|
|
331
334
|
}
|
|
335
|
+
appendVoiceContext(form, contextPack);
|
|
332
336
|
// Aggressive timeout — short clips should transcribe in well under
|
|
333
337
|
// 2 seconds with Gemini Flash Lite or Whisper. If the server hasn't
|
|
334
338
|
// come back in 8s it's hanging; abort and let the bar dismiss with an
|
|
@@ -367,6 +371,7 @@ async function cleanupTranscript(
|
|
|
367
371
|
providerPref: ServerVoiceProvider,
|
|
368
372
|
controller: AbortController,
|
|
369
373
|
instructions?: string,
|
|
374
|
+
contextPack?: VoiceContextPack,
|
|
370
375
|
): Promise<string> {
|
|
371
376
|
const form = new FormData();
|
|
372
377
|
form.append("text", text);
|
|
@@ -375,6 +380,7 @@ async function cleanupTranscript(
|
|
|
375
380
|
if (trimmedInstructions) {
|
|
376
381
|
form.append("instructions", trimmedInstructions);
|
|
377
382
|
}
|
|
383
|
+
appendVoiceContext(form, contextPack);
|
|
378
384
|
|
|
379
385
|
const timeout = window.setTimeout(() => controller.abort(), 8_000);
|
|
380
386
|
try {
|
|
@@ -400,6 +406,18 @@ async function cleanupTranscript(
|
|
|
400
406
|
}
|
|
401
407
|
}
|
|
402
408
|
|
|
409
|
+
function appendVoiceContext(
|
|
410
|
+
form: FormData,
|
|
411
|
+
contextPack: VoiceContextPack | undefined,
|
|
412
|
+
): void {
|
|
413
|
+
if (!contextPack) return;
|
|
414
|
+
try {
|
|
415
|
+
form.append("voiceContext", JSON.stringify(contextPack));
|
|
416
|
+
} catch {
|
|
417
|
+
/* best effort */
|
|
418
|
+
}
|
|
419
|
+
}
|
|
420
|
+
|
|
403
421
|
export function installDesktopVoiceDictation(
|
|
404
422
|
options: DesktopVoiceDictationOptions,
|
|
405
423
|
): () => void {
|
|
@@ -640,6 +658,7 @@ export function installDesktopVoiceDictation(
|
|
|
640
658
|
const controller = new AbortController();
|
|
641
659
|
target.transcribeAbort = controller;
|
|
642
660
|
try {
|
|
661
|
+
const contextPack = await buildDesktopVoiceContextPack();
|
|
643
662
|
text =
|
|
644
663
|
(await cleanupTranscript(
|
|
645
664
|
serverUrl,
|
|
@@ -647,6 +666,7 @@ export function installDesktopVoiceDictation(
|
|
|
647
666
|
target.cleanupProvider,
|
|
648
667
|
controller,
|
|
649
668
|
instructions,
|
|
669
|
+
contextPack,
|
|
650
670
|
)) || original;
|
|
651
671
|
} catch (err) {
|
|
652
672
|
if ((err as { name?: string })?.name !== "AbortError") {
|
|
@@ -709,6 +729,42 @@ export function installDesktopVoiceDictation(
|
|
|
709
729
|
micDeviceLabel: micDeviceLabel || null,
|
|
710
730
|
});
|
|
711
731
|
|
|
732
|
+
const buildDesktopVoiceContextPack = async (): Promise<
|
|
733
|
+
VoiceContextPack | undefined
|
|
734
|
+
> => {
|
|
735
|
+
const vocabulary = await loadVocabularyEntries().catch(() => []);
|
|
736
|
+
const terms = vocabulary.map((entry) => ({
|
|
737
|
+
term: entry.term,
|
|
738
|
+
replacement: entry.replacement,
|
|
739
|
+
confidence: entry.confidence,
|
|
740
|
+
source: "learned-correction",
|
|
741
|
+
scope: "user",
|
|
742
|
+
}));
|
|
743
|
+
const snippets: NonNullable<VoiceContextPack["snippets"]> = [
|
|
744
|
+
{ label: "Target surface", value: "Desktop dictation paste" },
|
|
745
|
+
];
|
|
746
|
+
if (micDeviceLabel) {
|
|
747
|
+
snippets.push({ label: "Microphone", value: micDeviceLabel });
|
|
748
|
+
}
|
|
749
|
+
if (instructions.trim()) {
|
|
750
|
+
snippets.push({
|
|
751
|
+
label: "Saved voice instructions",
|
|
752
|
+
value: instructions.trim().slice(0, 1200),
|
|
753
|
+
});
|
|
754
|
+
}
|
|
755
|
+
return {
|
|
756
|
+
surface: "clips-desktop",
|
|
757
|
+
mode: "dictation",
|
|
758
|
+
snippets,
|
|
759
|
+
terms,
|
|
760
|
+
metadata: {
|
|
761
|
+
provider,
|
|
762
|
+
locale: navigator.language || "en-US",
|
|
763
|
+
micDeviceLabel: micDeviceLabel || null,
|
|
764
|
+
},
|
|
765
|
+
};
|
|
766
|
+
};
|
|
767
|
+
|
|
712
768
|
/**
|
|
713
769
|
* Server-path: capture audio with MediaRecorder, POST it to the
|
|
714
770
|
* transcribe-voice endpoint on Fn-up, paste the response text. The
|
|
@@ -790,6 +846,7 @@ export function installDesktopVoiceDictation(
|
|
|
790
846
|
const controller = new AbortController();
|
|
791
847
|
next.transcribeAbort = controller;
|
|
792
848
|
try {
|
|
849
|
+
const contextPack = await buildDesktopVoiceContextPack();
|
|
793
850
|
const text = await transcribe(
|
|
794
851
|
serverUrl,
|
|
795
852
|
next.chunks,
|
|
@@ -797,6 +854,7 @@ export function installDesktopVoiceDictation(
|
|
|
797
854
|
providerPref,
|
|
798
855
|
controller,
|
|
799
856
|
instructions,
|
|
857
|
+
contextPack,
|
|
800
858
|
);
|
|
801
859
|
if (next.cancelled) {
|
|
802
860
|
cleanup(next);
|
|
@@ -920,7 +978,8 @@ export function installDesktopVoiceDictation(
|
|
|
920
978
|
// metadata to `native_speech_start` without also carrying vocabulary.
|
|
921
979
|
// Best-effort — if the load failed we just stage an empty list and
|
|
922
980
|
// the recognizer behaves as before.
|
|
923
|
-
const
|
|
981
|
+
const vocabularyEntries = await loadVocabularyEntries().catch(() => []);
|
|
982
|
+
const contextualStrings = vocabularyEntries.map((v) => v.replacement);
|
|
924
983
|
await invoke("native_speech_set_vocabulary", {
|
|
925
984
|
strings: contextualStrings,
|
|
926
985
|
}).catch(() => {});
|
|
@@ -7824,7 +7824,7 @@ function DatabaseTimelineView({
|
|
|
7824
7824
|
<div
|
|
7825
7825
|
key={`${dateKey}-header`}
|
|
7826
7826
|
className={cn(
|
|
7827
|
-
"sticky top-0 z-10 grid gap-0.5 border-r border-b border-border bg-background
|
|
7827
|
+
"sticky top-0 z-10 grid gap-0.5 border-r border-b border-border bg-background px-2 py-2 last:border-r-0",
|
|
7828
7828
|
!inMonth && "bg-muted/70",
|
|
7829
7829
|
)}
|
|
7830
7830
|
style={{ gridColumn: index + 1, gridRow: 1 }}
|
|
@@ -7767,7 +7767,7 @@ function DatabaseTimelineView({
|
|
|
7767
7767
|
<div
|
|
7768
7768
|
key={`${dateKey}-header`}
|
|
7769
7769
|
className={cn(
|
|
7770
|
-
"sticky top-0 z-10 grid gap-0.5 border-r border-b border-border bg-background
|
|
7770
|
+
"sticky top-0 z-10 grid gap-0.5 border-r border-b border-border bg-background px-2 py-2 last:border-r-0",
|
|
7771
7771
|
!inMonth && "bg-muted/70",
|
|
7772
7772
|
)}
|
|
7773
7773
|
style={{ gridColumn: index + 1, gridRow: 1 }}
|
|
@@ -701,7 +701,7 @@ export function DocumentSidebar({
|
|
|
701
701
|
|
|
702
702
|
if (collapsed) {
|
|
703
703
|
return (
|
|
704
|
-
<div className="flex
|
|
704
|
+
<div className="flex h-full w-12 flex-col items-center gap-1 border-e border-border bg-muted/30 py-3 transition-[width] duration-200 ease-out">
|
|
705
705
|
<Tooltip>
|
|
706
706
|
<TooltipTrigger asChild>
|
|
707
707
|
<button
|
|
@@ -763,7 +763,7 @@ export function DocumentSidebar({
|
|
|
763
763
|
return (
|
|
764
764
|
<div
|
|
765
765
|
className={cn(
|
|
766
|
-
"agent-layout-left-drawer relative flex h-full min-h-0 flex-col border-e border-border bg-muted/30",
|
|
766
|
+
"agent-layout-left-drawer relative flex h-full min-h-0 flex-col border-e border-border bg-muted/30 transition-[width] duration-200 ease-out",
|
|
767
767
|
width === undefined && "w-full",
|
|
768
768
|
)}
|
|
769
769
|
style={width === undefined ? undefined : { width, flexShrink: 0 }}
|
|
@@ -962,14 +962,14 @@ export function DocumentSidebar({
|
|
|
962
962
|
</div>
|
|
963
963
|
</ScrollArea>
|
|
964
964
|
|
|
965
|
-
<div className="shrink-0
|
|
965
|
+
<div className="shrink-0 px-3 py-2">
|
|
966
966
|
<div className="space-y-1">
|
|
967
967
|
{renderLocalFilesNavButton()}
|
|
968
968
|
{renderSettingsNavButton()}
|
|
969
969
|
</div>
|
|
970
970
|
</div>
|
|
971
971
|
|
|
972
|
-
<div className="shrink-0
|
|
972
|
+
<div className="shrink-0">
|
|
973
973
|
<ExtensionSlot
|
|
974
974
|
id="content.sidebar.bottom"
|
|
975
975
|
context={{
|
|
@@ -985,7 +985,7 @@ export function DocumentSidebar({
|
|
|
985
985
|
</div>
|
|
986
986
|
|
|
987
987
|
{/* Footer */}
|
|
988
|
-
<div className="shrink-0 space-y-2
|
|
988
|
+
<div className="shrink-0 space-y-2 px-3 py-2">
|
|
989
989
|
<OrgSwitcher />
|
|
990
990
|
{isCodeMode ? <DevDatabaseLink /> : null}
|
|
991
991
|
<div className="flex items-center gap-1">
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: visual-edit
|
|
3
|
+
description: >-
|
|
4
|
+
Open a running local app in Design overview mode as URL-backed iframe screens
|
|
5
|
+
for visual editing, flow review, duplication, and route-state exploration.
|
|
6
|
+
metadata:
|
|
7
|
+
visibility: exported
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Visual Edit
|
|
11
|
+
|
|
12
|
+
Use `/visual-edit` when the user wants to inspect or edit a real local app
|
|
13
|
+
visually instead of generating standalone Alpine HTML. The source of truth is
|
|
14
|
+
the running localhost app plus its route URLs. Design shows those routes as
|
|
15
|
+
iframe-backed screens on the infinite canvas.
|
|
16
|
+
|
|
17
|
+
## Core Model
|
|
18
|
+
|
|
19
|
+
- Each screen is a URL-backed iframe, not copied HTML.
|
|
20
|
+
- Each screen keeps its own URL metadata: `connectionId`, `routeId`, `path`,
|
|
21
|
+
`url`, `bridgeUrl`, title, and viewport size.
|
|
22
|
+
- Start in Design's screen overview mode. The user can edit/select/drop on any
|
|
23
|
+
visible screen from the overview canvas.
|
|
24
|
+
- In overview, screens are static like Figma frames: no iframe scrolling or app
|
|
25
|
+
interaction. Full-screen focus is for scrolling and app interaction.
|
|
26
|
+
- Alt-drag duplicates a screen. For localhost screens, duplication copies the
|
|
27
|
+
iframe frame and URL metadata; change the copy's path/query when you need a
|
|
28
|
+
new flow state.
|
|
29
|
+
- Flow visualization is just multiple URL states: `/checkout?step=shipping`,
|
|
30
|
+
`/checkout?step=payment`, `/checkout?step=done`, etc.
|
|
31
|
+
|
|
32
|
+
## Required Local Bridge
|
|
33
|
+
|
|
34
|
+
From the target app repo, make sure its dev server is running, then run the
|
|
35
|
+
Design bridge with `npx`:
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
npx @agent-native/core@latest design connect --url http://localhost:5173 --root .
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Use the app's real port. The command starts a local bridge on
|
|
42
|
+
`http://127.0.0.1:7331` by default and exposes:
|
|
43
|
+
|
|
44
|
+
- `GET /manifest.json` — dev server URL, bridge URL, discovered routes, root.
|
|
45
|
+
- `GET /routes.json` — route manifest only.
|
|
46
|
+
- `GET /health` — bridge liveness.
|
|
47
|
+
|
|
48
|
+
For one-shot agent setup, ask for JSON and keep the long-running bridge open in
|
|
49
|
+
a second terminal if the user needs live updates:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
npx @agent-native/core@latest design connect --url http://localhost:5173 --root . --json
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
## Action Flow
|
|
56
|
+
|
|
57
|
+
1. Register or refresh the bridge in Design:
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
pnpm action connect-localhost '{
|
|
61
|
+
"devServerUrl": "http://localhost:5173",
|
|
62
|
+
"bridgeUrl": "http://127.0.0.1:7331",
|
|
63
|
+
"rootPath": ".",
|
|
64
|
+
"routeManifest": { "version": 1, "sourceType": "localhost", "routes": [] }
|
|
65
|
+
}'
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Prefer passing the actual `/manifest.json` result as `routeManifest` and
|
|
69
|
+
`capabilities`. Keep any local filesystem paths out of user-facing summaries
|
|
70
|
+
unless the user asks.
|
|
71
|
+
|
|
72
|
+
2. Create or reuse a Design project:
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
pnpm action create-design --title "Local app visual edit" --projectType prototype
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
3. Place URL-backed screens on the overview canvas:
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
pnpm action add-localhost-screens '{
|
|
82
|
+
"designId": "<design-id>",
|
|
83
|
+
"connectionId": "<connection-id>",
|
|
84
|
+
"routes": [
|
|
85
|
+
{ "path": "/", "title": "Home", "width": 1280, "height": 900 },
|
|
86
|
+
{ "path": "/pricing", "title": "Pricing", "width": 1280, "height": 900 },
|
|
87
|
+
{ "path": "/checkout?step=payment", "title": "Checkout payment", "width": 1280, "height": 900 }
|
|
88
|
+
],
|
|
89
|
+
"startX": 0,
|
|
90
|
+
"startY": 0,
|
|
91
|
+
"gap": 160
|
|
92
|
+
}'
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
If no `routes` or `paths` are supplied, `add-localhost-screens` uses every route
|
|
96
|
+
from the latest localhost manifest. Use `paths` for a concise flow:
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
pnpm action add-localhost-screens '{
|
|
100
|
+
"designId": "<design-id>",
|
|
101
|
+
"paths": ["/", "/pricing", "/checkout?step=payment"]
|
|
102
|
+
}'
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
4. Navigate the user to overview mode:
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
pnpm action navigate --view editor --designId "<design-id>" --editorView overview
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
## Editing URLs
|
|
112
|
+
|
|
113
|
+
To change a localhost screen's URL, update that screen through
|
|
114
|
+
`add-localhost-screens` again using the same route-derived filename/path or use
|
|
115
|
+
normal Design screen duplication followed by a route/path update. Keep the file
|
|
116
|
+
content as the absolute URL and keep `screenMetadata[fileId]` aligned:
|
|
117
|
+
|
|
118
|
+
```json
|
|
119
|
+
{
|
|
120
|
+
"sourceType": "localhost",
|
|
121
|
+
"previewState": "live",
|
|
122
|
+
"url": "http://localhost:5173/checkout?step=done",
|
|
123
|
+
"previewUrl": "http://localhost:5173/checkout?step=done",
|
|
124
|
+
"path": "/checkout?step=done"
|
|
125
|
+
}
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
Do not replace localhost screens with copied `srcdoc` HTML unless the user
|
|
129
|
+
explicitly asks to freeze a snapshot.
|
|
130
|
+
|
|
131
|
+
## Verification
|
|
132
|
+
|
|
133
|
+
- `list-localhost-connections` returns the expected connection and routes.
|
|
134
|
+
- The Design editor opens in overview mode.
|
|
135
|
+
- Every requested screen renders the intended localhost URL.
|
|
136
|
+
- Alt-dragging a screen copies the URL-backed frame, not an inline HTML clone.
|
|
137
|
+
- A query/path edit changes only the target screen's URL metadata and iframe.
|
|
@@ -50,6 +50,19 @@ patterns live in `.agents/skills/`.
|
|
|
50
50
|
is backed by a CSS custom property the rendered file actually uses. If source
|
|
51
51
|
edits are needed, use `get-design-snapshot` first and persist the complete
|
|
52
52
|
updated tweak definition list through `generate-design`.
|
|
53
|
+
- Design editor extensions render in the right inspector slot
|
|
54
|
+
`design.editor.inspector`. When creating an extension for that panel, call
|
|
55
|
+
`create-extension`, then `add-extension-slot-target` with that slot id, then
|
|
56
|
+
`install-extension` so it appears inline. Extensions receive
|
|
57
|
+
`window.slotContext` with the current design id, active screen, selected
|
|
58
|
+
element, zoom, mode, tool, and tweak values. For AI-driven style/artboard
|
|
59
|
+
changes, extension HTML should call `agentNative.chat.send(...)` with the
|
|
60
|
+
selected element selector/sourceId and request; the agent should call
|
|
61
|
+
`view-screen` first, prefer `apply-visual-edit` for element style/class/text
|
|
62
|
+
changes, and use `update-design` or `generate-design` with `canvasFrames` for
|
|
63
|
+
overview artboard placement changes. If creating the extension opens the
|
|
64
|
+
standalone extension editor, return to the same design with `navigate` and
|
|
65
|
+
`inspectorTab: "extensions"` after installing it.
|
|
53
66
|
- Follow linked design-system tokens and `customInstructions` whenever present;
|
|
54
67
|
explicit user instructions in the current turn still win.
|
|
55
68
|
- Persist useful work early: create/update the design and files as soon as a
|
|
@@ -57,31 +70,108 @@ patterns live in `.agents/skills/`.
|
|
|
57
70
|
- For non-trivial new design prompts, ask before generating: create/open the
|
|
58
71
|
design shell, call `show-design-questions`, stop while the main canvas shows
|
|
59
72
|
the questions, then continue from the user's answers.
|
|
73
|
+
- For multiple screens/states, call `generate-screens` first. It opens the
|
|
74
|
+
infinite screen overview and returns target filenames plus `canvasFrame`
|
|
75
|
+
placements. Then call `generate-design` with those files and pass the matching
|
|
76
|
+
`canvasFrames` entries so screens appear in the overview instead of only in
|
|
77
|
+
the file list.
|
|
78
|
+
- After generation or broad updates, leave the user in the screen overview when
|
|
79
|
+
the work involves multiple screens or artboard placement. Overview is the
|
|
80
|
+
primary editing surface: users can select screens, move/resize/drop static
|
|
81
|
+
frames and canvas primitives, edit layers in place, and use the frame's
|
|
82
|
+
full-view button to enter focused editing.
|
|
83
|
+
- To move the user's editor, call `navigate` with `view: "editor"` and
|
|
84
|
+
`editorView: "overview"` for the screen overview, or `editorView: "single"`
|
|
85
|
+
plus `fileId`, `filename`, or `screen` to focus one screen.
|
|
86
|
+
- Single-screen mode is for scrolling, interacting with prototype behavior, and
|
|
87
|
+
editing the DOM/code layers inside one screen. Do not use it as the default
|
|
88
|
+
landing mode for multi-screen generation unless the user asks to focus a
|
|
89
|
+
specific screen.
|
|
90
|
+
- The layers panel shows screens as top-level frames and nests DOM/code layers
|
|
91
|
+
beneath the active screen. Rename DOM/code layers by safely editing source to
|
|
92
|
+
set `data-agent-native-layer-name`; this is the stable human-readable layer
|
|
93
|
+
name. `data-code-layer-id` and similar ids are for selection stability, not
|
|
94
|
+
display naming.
|
|
95
|
+
- For inline/Alpine screens, stamp and preserve unique
|
|
96
|
+
`data-agent-native-node-id` attributes on selectable DOM nodes. Treat
|
|
97
|
+
generated CSS selectors as a compatibility fallback only. For localhost React
|
|
98
|
+
screens, resolve through build-time source/debug metadata (stable generated
|
|
99
|
+
ids, component name, file, and line) before falling back to selectors.
|
|
60
100
|
- For multi-variant work, write candidates incrementally so the UI can preview
|
|
61
101
|
progress. External MCP hosts should use `present-design-variants` so the same
|
|
62
102
|
picker opens inline instead of writing `application_state` directly.
|
|
63
103
|
- Use framework sharing actions for design and design-system visibility/grants.
|
|
64
104
|
- When the user asks to download/export, use export actions or point to the
|
|
65
105
|
editor download menu.
|
|
106
|
+
- Design source modes are `inline`, `localhost`, and `fusion`. Inline is the
|
|
107
|
+
current SQL-backed prototype mode. Localhost connections come from
|
|
108
|
+
`npx @agent-native/core@latest design connect` and are persisted with
|
|
109
|
+
`connect-localhost`; list them with `list-localhost-connections` before
|
|
110
|
+
creating or resolving local-code artboards. Fusion is a future source type and
|
|
111
|
+
should be preserved when present but not invented for inline/local work.
|
|
112
|
+
- Localhost route manifests are scaffolding for URL-backed Flow Canvas
|
|
113
|
+
artboards. Use `add-localhost-screens` to place routes or path/query states as
|
|
114
|
+
iframe screens in overview mode. Preserve route ids, paths, `sourceType`,
|
|
115
|
+
bridge URL, and snapshot/state references when moving between actions so later
|
|
116
|
+
flow-edge derivation has stable anchors.
|
|
66
117
|
|
|
67
118
|
## Application State
|
|
68
119
|
|
|
69
120
|
- `navigation` tells you the current view, design id, file id, and related UI
|
|
70
121
|
state.
|
|
71
122
|
- `navigate` moves the UI and is auto-deleted after the client consumes it.
|
|
123
|
+
- `design-selection` includes active screen, selected element, overview mode,
|
|
124
|
+
inspector tab, zoom, and screen list for the current tab.
|
|
125
|
+
- `design-generation-session:<designId>` contains visible multi-screen
|
|
126
|
+
generation frames created by `generate-screens`.
|
|
72
127
|
- `show-design-questions` opens focused pre-generation questions in the main
|
|
73
128
|
design canvas (`show-questions` application state).
|
|
74
129
|
- `design-variants` contains in-progress candidates for the variant picker.
|
|
75
130
|
|
|
131
|
+
## Code Layers
|
|
132
|
+
|
|
133
|
+
- `get-code-layer-projection` reads inline HTML/JSX and returns selectable layer
|
|
134
|
+
nodes, selectors, names, and edit intents for agent and UI workflows.
|
|
135
|
+
- `apply-visual-edit` supports deterministic local edits for HTML-backed code
|
|
136
|
+
layers: text, classes, styles, attributes, source order, and small structural
|
|
137
|
+
changes. Use it for selected-element edits before falling back to full
|
|
138
|
+
`update-design` / `generate-design` rewrites.
|
|
139
|
+
- Prefer `data-agent-native-layer-name="Readable name"` on meaningful elements.
|
|
140
|
+
The projection uses it before semantic/text fallbacks, and layer renames should
|
|
141
|
+
persist by updating that attribute.
|
|
142
|
+
- Current limitation: visual code-layer edits are HTML-only. Localhost source
|
|
143
|
+
mode can list and resolve local routes now, but file reads/writes remain a
|
|
144
|
+
bridge contract until explicit permission controls are hardened.
|
|
145
|
+
|
|
146
|
+
## Localhost Source Actions
|
|
147
|
+
|
|
148
|
+
- `connect-localhost`: registers or refreshes a localhost source connection
|
|
149
|
+
emitted by `npx @agent-native/core@latest design connect`. Pass
|
|
150
|
+
`devServerUrl`, optional `bridgeUrl`, optional `rootPath`, and either `routes`
|
|
151
|
+
or a full `routeManifest`.
|
|
152
|
+
- `list-localhost-connections`: lists the current user's saved localhost
|
|
153
|
+
connections and route manifests. Use this before referring to local-code
|
|
154
|
+
artboards.
|
|
155
|
+
- `add-localhost-screens`: creates or refreshes URL-backed iframe screens from
|
|
156
|
+
the latest localhost connection or a specific `connectionId`. Pass `routes`
|
|
157
|
+
with `path`/`url` when visualizing a flow; pass `paths` for a concise route
|
|
158
|
+
list. Then call `navigate --view editor --designId <id> --editorView overview`.
|
|
159
|
+
|
|
76
160
|
## App-Backed Skill Distribution
|
|
77
161
|
|
|
78
162
|
- The preferred hosted install path is
|
|
79
|
-
`npx @agent-native/core@latest skills add design-exploration
|
|
80
|
-
|
|
81
|
-
|
|
163
|
+
`npx @agent-native/core@latest skills add design-exploration`,
|
|
164
|
+
`npx @agent-native/core@latest skills add visual-edit`, or `design` for the
|
|
165
|
+
full Design bundle. It installs the exported Design instructions and registers
|
|
166
|
+
the hosted Design MCP connector together.
|
|
82
167
|
- The open Skills CLI path
|
|
83
|
-
`npx skills@latest add BuilderIO/agent-native --skill
|
|
84
|
-
|
|
168
|
+
`npx skills@latest add BuilderIO/agent-native --skill visual-edit` installs
|
|
169
|
+
exported instructions only.
|
|
170
|
+
- For local-code visual editing, `/visual-edit` should run the target app dev
|
|
171
|
+
server, run
|
|
172
|
+
`npx @agent-native/core@latest design connect --url http://localhost:<port> --root .`,
|
|
173
|
+
register that manifest with `connect-localhost`, call `add-localhost-screens`,
|
|
174
|
+
and open the editor in overview mode.
|
|
85
175
|
- For human-in-the-loop UI exploration, create a design shell, call
|
|
86
176
|
`present-design-variants` with 2-5 complete HTML directions (three by
|
|
87
177
|
default), wait for the user to pick one, then use `get-design-snapshot` and
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# Design System Contract
|
|
2
|
+
|
|
3
|
+
This file is the source-control scaffold for Design projects that point at real
|
|
4
|
+
code. The app can keep inline prototypes in SQL today, connect to localhost next,
|
|
5
|
+
and later blend hosted/fusion sources without changing the artboard model.
|
|
6
|
+
|
|
7
|
+
## Editor Contract
|
|
8
|
+
|
|
9
|
+
Design is overview-first for multi-screen work. Generation and broad updates
|
|
10
|
+
should land in the screen overview: an infinite canvas where each screen is a
|
|
11
|
+
static frame that can be selected, moved, resized, dropped, or duplicated.
|
|
12
|
+
Use the frame's full-view button when a screen needs focused scrolling or
|
|
13
|
+
prototype interaction.
|
|
14
|
+
|
|
15
|
+
Single-screen mode is the scroll/interact/edit surface for one rendered file. It
|
|
16
|
+
is where prototype behavior runs, iframe scrolling happens, and local DOM/code
|
|
17
|
+
layer edits are applied.
|
|
18
|
+
|
|
19
|
+
The layers panel mirrors that model: screens are top-level frames, with nested
|
|
20
|
+
DOM/code layers for the active screen. Stable human-readable layer names should
|
|
21
|
+
be stored in `data-agent-native-layer-name`; selector ids such as
|
|
22
|
+
`data-code-layer-id` remain implementation anchors.
|
|
23
|
+
|
|
24
|
+
## Source Modes
|
|
25
|
+
|
|
26
|
+
- `inline`: the current SQL-backed Design files and deterministic HTML swaps.
|
|
27
|
+
- `localhost`: a running app dev server exposed through
|
|
28
|
+
`npx @agent-native/core@latest design connect`.
|
|
29
|
+
- `fusion`: future hosted or hybrid sources that can provide snapshots and code
|
|
30
|
+
context through the same bridge contract.
|
|
31
|
+
|
|
32
|
+
Every artboard should carry `sourceType`, a source descriptor, stable route/path
|
|
33
|
+
identity, and a snapshot reference when available. Flow edges should be durable
|
|
34
|
+
objects even when they are derived later from captured state.
|
|
35
|
+
|
|
36
|
+
## Localhost Bridge
|
|
37
|
+
|
|
38
|
+
Run this from the app/repo root while the app dev server is running:
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
npx @agent-native/core@latest design connect --url http://localhost:5173 --root .
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
The command starts a local bridge and prints a manifest URL. It also creates
|
|
45
|
+
`.agent-native/design-routes.json` only when that file does not already exist.
|
|
46
|
+
That manifest is non-authoritative scaffolding; the live bridge response and the
|
|
47
|
+
Design `connect-localhost` action are the durable app contract.
|
|
48
|
+
|
|
49
|
+
For agent-launched visual editing, install or invoke `/visual-edit`. The skill
|
|
50
|
+
registers the bridge with `connect-localhost`, uses `add-localhost-screens` to
|
|
51
|
+
place each route or path/query state as a URL-backed iframe screen, then
|
|
52
|
+
navigates the editor to overview mode. Keep these screens as URL sources; do not
|
|
53
|
+
copy localhost HTML into inline files unless intentionally freezing a snapshot.
|
|
54
|
+
|
|
55
|
+
Current bridge operations:
|
|
56
|
+
|
|
57
|
+
- `select`
|
|
58
|
+
- `resolveNodeToFile`
|
|
59
|
+
- `readFile`
|
|
60
|
+
- `applyEdit`
|
|
61
|
+
- `writeFile`
|
|
62
|
+
- `captureSnapshot`
|
|
63
|
+
- `captureState`
|
|
64
|
+
|
|
65
|
+
`select`, route listing, and capture contracts are foundations now. Local file
|
|
66
|
+
reads/writes and LLM-backed instructions remain planned until the bridge has
|
|
67
|
+
permission hardening and explicit user controls.
|
|
68
|
+
|
|
69
|
+
## Code-Layer Editing
|
|
70
|
+
|
|
71
|
+
Inline HTML supports local code-layer projection and deterministic visual edits:
|
|
72
|
+
text, classes, styles, attributes, source order, and small structural changes.
|
|
73
|
+
`data-agent-native-layer-name` is the preferred naming surface because renames
|
|
74
|
+
can persist as safe source edits without changing selection ids.
|
|
75
|
+
Use durable `data-agent-native-node-id` values for Alpine/inline HTML whenever a
|
|
76
|
+
layer may be selected, reordered, duplicated, or patched; CSS selectors are only
|
|
77
|
+
a fallback. React/localhost targets should prefer build-time source metadata
|
|
78
|
+
(component/file/line and generated stable ids) over DOM path selectors.
|
|
79
|
+
|
|
80
|
+
Current limitation: visual code-layer edits only support HTML files. Localhost
|
|
81
|
+
artboards can be registered and resolved today, but local file reads/writes stay
|
|
82
|
+
behind the bridge permission model until explicit user controls are in place.
|
|
@@ -15,6 +15,12 @@ This guide is for development-mode agents editing this app's source code. For ap
|
|
|
15
15
|
- **Dev:** `pnpm dev` (Vite dev server with both React Router + Nitro plugins)
|
|
16
16
|
- **Build:** `pnpm build` (React Router build — client + SSR + Nitro server)
|
|
17
17
|
- **Start:** `node .output/server/index.mjs` (production)
|
|
18
|
+
- **Design local bridge:** `npx @agent-native/core@latest design connect --url http://localhost:5173 --root .`
|
|
19
|
+
exposes a local manifest for Design's localhost source mode and scaffolds
|
|
20
|
+
`.agent-native/design-routes.json` without overwriting an existing file.
|
|
21
|
+
- **Visual edit local apps:** `/visual-edit` registers that bridge with
|
|
22
|
+
`connect-localhost`, adds URL-backed iframe screens with
|
|
23
|
+
`add-localhost-screens`, and opens the editor in overview mode.
|
|
18
24
|
|
|
19
25
|
## Directory Structure
|
|
20
26
|
|
|
@@ -92,6 +98,43 @@ export default function MyPageRoute() {
|
|
|
92
98
|
|
|
93
99
|
Normal app data starts as an action, not a custom route. Add `actions/<verb>-<resource>.ts` with `defineAction`, mark reads with `http: { method: "GET" }`, and call reads/writes from React with `useActionQuery` / `useActionMutation` from `@agent-native/core/client`. This keeps the UI and agent on one contract and lets mutating actions refresh action-backed queries automatically.
|
|
94
100
|
|
|
101
|
+
## Design Source Modes
|
|
102
|
+
|
|
103
|
+
The shared source model lives in `shared/source-mode.ts`. Use `inline` for
|
|
104
|
+
SQL-backed prototype files, `localhost` for local dev-server artboards, and
|
|
105
|
+
`fusion` only for future hybrid/hosted sources. Bridge operations are named
|
|
106
|
+
`select`, `resolveNodeToFile`, `readFile`, `applyEdit`, `writeFile`,
|
|
107
|
+
`captureSnapshot`, and `captureState`; current localhost writes are contract-only
|
|
108
|
+
until the bridge adds explicit permission hardening.
|
|
109
|
+
|
|
110
|
+
## Editor Model
|
|
111
|
+
|
|
112
|
+
Design has two editor views:
|
|
113
|
+
|
|
114
|
+
- **Overview** is the default multi-screen canvas after generation or broad
|
|
115
|
+
updates. It renders every screen as a static, movable frame on an infinite
|
|
116
|
+
surface. Users can select screens, move/resize/drop frames and primitives,
|
|
117
|
+
edit layers in place, and use a frame's full-view button to focus it.
|
|
118
|
+
- **Single screen** renders one file in the iframe editor. Use it for scrolling,
|
|
119
|
+
interacting with prototype behavior, text/DOM selection, and local visual
|
|
120
|
+
edits inside that screen.
|
|
121
|
+
|
|
122
|
+
The layers panel treats screens/files as top-level frames and nests projected
|
|
123
|
+
DOM/code layers under the active screen. Layer display names come from
|
|
124
|
+
`data-agent-native-layer-name` first, then semantic/text fallbacks. Persist user
|
|
125
|
+
renames by editing that attribute in the source; ids such as
|
|
126
|
+
`data-code-layer-id` are for stable selection targets.
|
|
127
|
+
Inline/Alpine HTML should also carry durable `data-agent-native-node-id`
|
|
128
|
+
attributes for any layer the editor can select, reorder, duplicate, or patch.
|
|
129
|
+
Selectors are fallback aliases, not the primary identity. Localhost React mode
|
|
130
|
+
should resolve through build-time source/debug metadata (generated stable id,
|
|
131
|
+
component, file, and line) before using selector fallbacks.
|
|
132
|
+
|
|
133
|
+
Inline source mode supports local code-layer projection and deterministic HTML
|
|
134
|
+
edits through `get-code-layer-projection` and `apply-visual-edit`. Current
|
|
135
|
+
limitations: visual code-layer edits are HTML-only, and localhost reads/writes
|
|
136
|
+
are still bridge-contract scaffolding until permission controls are hardened.
|
|
137
|
+
|
|
95
138
|
## Adding a Route-Only Endpoint
|
|
96
139
|
|
|
97
140
|
Use `server/routes/api/` only for protocols that cannot be modeled as JSON actions: multipart uploads, streaming/SSE/WebSocket, webhooks, OAuth callbacks/redirects, public SEO/OG endpoints, or binary/static asset serving. Do not add `/api/*` routes for normal CRUD, data queries, or pass-through wrappers around actions; the action endpoint already exists at `/_agent-native/actions/:name`.
|