@neuralis/brain-core 0.3.2 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +80 -1410
- package/app/README.md +1 -1
- package/app/cards/FilesystemCardShell.tsx +42 -4
- package/app/cards/FilesystemChangeCard.tsx +34 -11
- package/app/cards/README.md +6 -5
- package/app/cards/diffUtils.ts +99 -1
- package/app/filesystem/FilesystemHost.tsx +17 -2
- package/app/filesystem/api/README.md +3 -3
- package/app/filesystem/api/index.ts +35 -11
- package/app/filesystem/api/projectPackages.ts +13 -102
- package/app/filesystem/domain/types.ts +30 -10
- package/app/filesystem/hooks/README.md +3 -3
- package/app/filesystem/hooks/useDiscoverableSources.ts +19 -30
- package/app/filesystem/hooks/useFilesystemSSE.ts +1 -1
- package/app/filesystem/host/README.md +5 -3
- package/app/filesystem/host/buildFilesystemHostValue.ts +19 -0
- package/app/filesystem/host/installFilesystemHostComponents.tsx +1 -1
- package/app/filesystem/lib/sourceKindForm.ts +118 -39
- package/app/filesystem/lib/treeVisualState.tsx +98 -4
- package/app/filesystem/store/README.md +1 -0
- package/app/filesystem/store/filesystemStore.ts +86 -5
- package/app/filesystem/store/gitSlice.ts +5 -0
- package/app/filesystem/store/helpers.ts +17 -0
- package/app/filesystem/store/listSlice.ts +8 -0
- package/app/filesystem/ui/CodeEditor.tsx +18 -3
- package/app/filesystem/ui/FileEditor.tsx +15 -10
- package/app/filesystem/ui/FileTree.tsx +41 -16
- package/app/filesystem/ui/FolderPicker.tsx +251 -0
- package/app/filesystem/ui/GitControlPanel.tsx +5 -0
- package/app/filesystem/ui/PackagesPanel.tsx +52 -374
- package/app/filesystem/ui/PolicyEditor.tsx +109 -24
- package/app/filesystem/ui/README.md +17 -11
- package/app/filesystem/ui/RawMediaPreview.tsx +58 -0
- package/app/filesystem/ui/SourceBranchHeader.tsx +4 -4
- package/app/filesystem/ui/SourcesPanel.tsx +420 -360
- package/app/filesystem/ui/Toolbar.tsx +1 -2
- package/app/shared/filesystem.png +0 -0
- package/dist/app/host.js +53 -48
- package/dist/scripts/sourceRootPairV1.d.ts +47 -0
- package/dist/scripts/sourceRootPairV1.d.ts.map +1 -0
- package/dist/scripts/sourceRootPairV1.js +147 -0
- package/dist/scripts/sourceRootPairV1.js.map +1 -0
- package/dist/scripts/sourceToFolderV1.d.ts +56 -0
- package/dist/scripts/sourceToFolderV1.d.ts.map +1 -0
- package/dist/scripts/sourceToFolderV1.js +152 -0
- package/dist/scripts/sourceToFolderV1.js.map +1 -0
- package/dist/scripts/sourceToHostV1.d.ts +56 -0
- package/dist/scripts/sourceToHostV1.d.ts.map +1 -0
- package/dist/scripts/sourceToHostV1.js +132 -0
- package/dist/scripts/sourceToHostV1.js.map +1 -0
- package/dist/src/config/env.d.ts +8 -0
- package/dist/src/config/env.d.ts.map +1 -1
- package/dist/src/config/env.js +17 -6
- package/dist/src/config/env.js.map +1 -1
- package/dist/src/connectors/AppSourceConfigStore.d.ts +17 -3
- package/dist/src/connectors/AppSourceConfigStore.d.ts.map +1 -1
- package/dist/src/connectors/AppSourceConfigStore.js +46 -6
- package/dist/src/connectors/AppSourceConfigStore.js.map +1 -1
- package/dist/src/connectors/HostConnector.d.ts +201 -0
- package/dist/src/connectors/HostConnector.d.ts.map +1 -0
- package/dist/src/connectors/HostConnector.js +903 -0
- package/dist/src/connectors/HostConnector.js.map +1 -0
- package/dist/src/connectors/inProcessTransport.d.ts +131 -0
- package/dist/src/connectors/inProcessTransport.d.ts.map +1 -0
- package/dist/src/connectors/inProcessTransport.js +447 -0
- package/dist/src/connectors/inProcessTransport.js.map +1 -0
- package/dist/src/domain/BrainError.d.ts +14 -0
- package/dist/src/domain/BrainError.d.ts.map +1 -1
- package/dist/src/domain/BrainError.js +21 -0
- package/dist/src/domain/BrainError.js.map +1 -1
- package/dist/src/domain/access/callerPermissions.d.ts +29 -0
- package/dist/src/domain/access/callerPermissions.d.ts.map +1 -1
- package/dist/src/domain/access/callerPermissions.js +85 -5
- package/dist/src/domain/access/callerPermissions.js.map +1 -1
- package/dist/src/domain/access/nativeLocalFence.d.ts +65 -0
- package/dist/src/domain/access/nativeLocalFence.d.ts.map +1 -0
- package/dist/src/domain/access/nativeLocalFence.js +171 -0
- package/dist/src/domain/access/nativeLocalFence.js.map +1 -0
- package/dist/src/domain/access/uriPermission.js +3 -3
- package/dist/src/domain/access/uriPermission.js.map +1 -1
- package/dist/src/domain/byteRange.d.ts +15 -0
- package/dist/src/domain/byteRange.d.ts.map +1 -0
- package/dist/src/domain/byteRange.js +30 -0
- package/dist/src/domain/byteRange.js.map +1 -0
- package/dist/src/domain/defaultPermissions.d.ts.map +1 -1
- package/dist/src/domain/defaultPermissions.js +29 -0
- package/dist/src/domain/defaultPermissions.js.map +1 -1
- package/dist/src/domain/mime.d.ts +8 -16
- package/dist/src/domain/mime.d.ts.map +1 -1
- package/dist/src/domain/mime.js +10 -99
- package/dist/src/domain/mime.js.map +1 -1
- package/dist/src/domain/paths.d.ts.map +1 -1
- package/dist/src/domain/paths.js +5 -3
- package/dist/src/domain/paths.js.map +1 -1
- package/dist/src/domain/sourceConfig.d.ts +52 -19
- package/dist/src/domain/sourceConfig.d.ts.map +1 -1
- package/dist/src/domain/sourceConfig.js +103 -19
- package/dist/src/domain/sourceConfig.js.map +1 -1
- package/dist/src/domain/sourcePackages.d.ts +8 -8
- package/dist/src/domain/sourcePackages.d.ts.map +1 -1
- package/dist/src/domain/sourceRootPair.d.ts +88 -0
- package/dist/src/domain/sourceRootPair.d.ts.map +1 -0
- package/dist/src/domain/sourceRootPair.js +108 -0
- package/dist/src/domain/sourceRootPair.js.map +1 -0
- package/dist/src/domain/syncTypes.d.ts +8 -1
- package/dist/src/domain/syncTypes.d.ts.map +1 -1
- package/dist/src/domain/syncTypes.js +1 -0
- package/dist/src/domain/syncTypes.js.map +1 -1
- package/dist/src/domain/types.d.ts +29 -1
- package/dist/src/domain/types.d.ts.map +1 -1
- package/dist/src/domain/uri.d.ts.map +1 -1
- package/dist/src/domain/uri.js +3 -2
- package/dist/src/domain/uri.js.map +1 -1
- package/dist/src/embeddings/clients/GeminiEmbedder.d.ts.map +1 -1
- package/dist/src/embeddings/clients/GeminiEmbedder.js +2 -5
- package/dist/src/embeddings/clients/GeminiEmbedder.js.map +1 -1
- package/dist/src/embeddings/clients/OpenAIEmbedder.d.ts.map +1 -1
- package/dist/src/embeddings/clients/OpenAIEmbedder.js +2 -3
- package/dist/src/embeddings/clients/OpenAIEmbedder.js.map +1 -1
- package/dist/src/embeddings/clients/QwenEmbedder.d.ts +2 -0
- package/dist/src/embeddings/clients/QwenEmbedder.d.ts.map +1 -1
- package/dist/src/embeddings/clients/QwenEmbedder.js +10 -2
- package/dist/src/embeddings/clients/QwenEmbedder.js.map +1 -1
- package/dist/src/embeddings/clients/VoyageEmbedder.d.ts +2 -0
- package/dist/src/embeddings/clients/VoyageEmbedder.d.ts.map +1 -1
- package/dist/src/embeddings/clients/VoyageEmbedder.js +8 -1
- package/dist/src/embeddings/clients/VoyageEmbedder.js.map +1 -1
- package/dist/src/embeddings/factory.d.ts +5 -3
- package/dist/src/embeddings/factory.d.ts.map +1 -1
- package/dist/src/embeddings/factory.js +7 -3
- package/dist/src/embeddings/factory.js.map +1 -1
- package/dist/src/embeddings/models.d.ts +2 -3
- package/dist/src/embeddings/models.d.ts.map +1 -1
- package/dist/src/embeddings/models.js +1 -2
- package/dist/src/embeddings/models.js.map +1 -1
- package/dist/src/embeddings/retry.js +1 -1
- package/dist/src/embeddings/retry.js.map +1 -1
- package/dist/src/embeddings/types.d.ts +6 -0
- package/dist/src/embeddings/types.d.ts.map +1 -1
- package/dist/src/embeddings/types.js +11 -0
- package/dist/src/embeddings/types.js.map +1 -1
- package/dist/src/git/GitRunner.d.ts +110 -30
- package/dist/src/git/GitRunner.d.ts.map +1 -1
- package/dist/src/git/GitRunner.js +511 -199
- package/dist/src/git/GitRunner.js.map +1 -1
- package/dist/src/git/GitWatchRegistry.d.ts +41 -35
- package/dist/src/git/GitWatchRegistry.d.ts.map +1 -1
- package/dist/src/git/GitWatchRegistry.js +61 -79
- package/dist/src/git/GitWatchRegistry.js.map +1 -1
- package/dist/src/git/gitPlace.d.ts +61 -0
- package/dist/src/git/gitPlace.d.ts.map +1 -0
- package/dist/src/git/gitPlace.js +74 -0
- package/dist/src/git/gitPlace.js.map +1 -0
- package/dist/src/git/sourceGitControl.d.ts +31 -19
- package/dist/src/git/sourceGitControl.d.ts.map +1 -1
- package/dist/src/git/sourceGitControl.js +208 -92
- package/dist/src/git/sourceGitControl.js.map +1 -1
- package/dist/src/git/sourceGitStatus.js +1 -1
- package/dist/src/git/sourceGitStatus.js.map +1 -1
- package/dist/src/git/untrackedStats.js +1 -1
- package/dist/src/git/untrackedStats.js.map +1 -1
- package/dist/src/index.d.ts +2 -1
- package/dist/src/index.d.ts.map +1 -1
- package/dist/src/index.js +2 -1
- package/dist/src/index.js.map +1 -1
- package/dist/src/lifecycle.d.ts +12 -16
- package/dist/src/lifecycle.d.ts.map +1 -1
- package/dist/src/lifecycle.js +2 -1
- package/dist/src/lifecycle.js.map +1 -1
- package/dist/src/routes/approve.d.ts.map +1 -1
- package/dist/src/routes/approve.js +3 -3
- package/dist/src/routes/approve.js.map +1 -1
- package/dist/src/routes/git.d.ts.map +1 -1
- package/dist/src/routes/git.js +183 -54
- package/dist/src/routes/git.js.map +1 -1
- package/dist/src/routes/health.d.ts.map +1 -1
- package/dist/src/routes/health.js +8 -5
- package/dist/src/routes/health.js.map +1 -1
- package/dist/src/routes/helpers/uriGuard.d.ts +1 -1
- package/dist/src/routes/helpers/uriGuard.d.ts.map +1 -1
- package/dist/src/routes/helpers/uriGuard.js +3 -2
- package/dist/src/routes/helpers/uriGuard.js.map +1 -1
- package/dist/src/routes/helpers.d.ts +20 -4
- package/dist/src/routes/helpers.d.ts.map +1 -1
- package/dist/src/routes/helpers.js +38 -5
- package/dist/src/routes/helpers.js.map +1 -1
- package/dist/src/routes/list.d.ts.map +1 -1
- package/dist/src/routes/list.js +5 -6
- package/dist/src/routes/list.js.map +1 -1
- package/dist/src/routes/pending.d.ts.map +1 -1
- package/dist/src/routes/pending.js +5 -5
- package/dist/src/routes/pending.js.map +1 -1
- package/dist/src/routes/policies.d.ts +6 -4
- package/dist/src/routes/policies.d.ts.map +1 -1
- package/dist/src/routes/policies.js +22 -16
- package/dist/src/routes/policies.js.map +1 -1
- package/dist/src/routes/raw.d.ts.map +1 -1
- package/dist/src/routes/raw.js +68 -14
- package/dist/src/routes/raw.js.map +1 -1
- package/dist/src/routes/read.d.ts.map +1 -1
- package/dist/src/routes/read.js +3 -3
- package/dist/src/routes/read.js.map +1 -1
- package/dist/src/routes/search.d.ts.map +1 -1
- package/dist/src/routes/search.js +3 -3
- package/dist/src/routes/search.js.map +1 -1
- package/dist/src/routes/sources.d.ts +60 -18
- package/dist/src/routes/sources.d.ts.map +1 -1
- package/dist/src/routes/sources.js +420 -308
- package/dist/src/routes/sources.js.map +1 -1
- package/dist/src/routes/upload.d.ts.map +1 -1
- package/dist/src/routes/upload.js +3 -3
- package/dist/src/routes/upload.js.map +1 -1
- package/dist/src/services/BridgeService.d.ts +15 -35
- package/dist/src/services/BridgeService.d.ts.map +1 -1
- package/dist/src/services/BridgeService.js +30 -88
- package/dist/src/services/BridgeService.js.map +1 -1
- package/dist/src/services/FilesystemService.d.ts +3 -2
- package/dist/src/services/FilesystemService.d.ts.map +1 -1
- package/dist/src/services/FilesystemService.js +116 -32
- package/dist/src/services/FilesystemService.js.map +1 -1
- package/dist/src/services/FilesystemUiService.d.ts +4 -1
- package/dist/src/services/FilesystemUiService.d.ts.map +1 -1
- package/dist/src/services/FilesystemUiService.js +71 -27
- package/dist/src/services/FilesystemUiService.js.map +1 -1
- package/dist/src/services/SyncScheduler.d.ts +12 -4
- package/dist/src/services/SyncScheduler.d.ts.map +1 -1
- package/dist/src/services/SyncScheduler.js +45 -10
- package/dist/src/services/SyncScheduler.js.map +1 -1
- package/dist/src/services/baselineRepair.d.ts +13 -3
- package/dist/src/services/baselineRepair.d.ts.map +1 -1
- package/dist/src/services/baselineRepair.js +13 -3
- package/dist/src/services/baselineRepair.js.map +1 -1
- package/dist/src/services/declaredSources.d.ts +125 -42
- package/dist/src/services/declaredSources.d.ts.map +1 -1
- package/dist/src/services/declaredSources.js +117 -45
- package/dist/src/services/declaredSources.js.map +1 -1
- package/dist/src/store/dataFormats.d.ts +17 -2
- package/dist/src/store/dataFormats.d.ts.map +1 -1
- package/dist/src/store/dataFormats.js +18 -3
- package/dist/src/store/dataFormats.js.map +1 -1
- package/dist/src/sync/DeltaSyncRunner.d.ts.map +1 -1
- package/dist/src/sync/DeltaSyncRunner.js +144 -4
- package/dist/src/sync/DeltaSyncRunner.js.map +1 -1
- package/dist/src/sync/SyncJobQueue.d.ts +14 -2
- package/dist/src/sync/SyncJobQueue.d.ts.map +1 -1
- package/dist/src/sync/SyncJobQueue.js +56 -10
- package/dist/src/sync/SyncJobQueue.js.map +1 -1
- package/dist/src/sync/SyncWatchService.d.ts +21 -4
- package/dist/src/sync/SyncWatchService.d.ts.map +1 -1
- package/dist/src/sync/SyncWatchService.js +115 -10
- package/dist/src/sync/SyncWatchService.js.map +1 -1
- package/dist/src/sync/categoryInference.d.ts +1 -1
- package/dist/src/sync/categoryInference.js +1 -1
- package/dist/src/tools/execCtx.d.ts +6 -0
- package/dist/src/tools/execCtx.d.ts.map +1 -1
- package/dist/src/tools/execCtx.js +11 -0
- package/dist/src/tools/execCtx.js.map +1 -1
- package/dist/src/tools/fs_list.js +2 -2
- package/dist/src/tools/fs_list.js.map +1 -1
- package/dist/src/tools/fs_read.js +2 -2
- package/dist/src/tools/fs_read.js.map +1 -1
- package/dist/src/tools/fs_search.js +2 -2
- package/dist/src/tools/fs_search.js.map +1 -1
- package/dist/src/tools/fs_write.js +2 -2
- package/dist/src/tools/fs_write.js.map +1 -1
- package/dist/src/usecases/admitSourceRoot.d.ts +65 -0
- package/dist/src/usecases/admitSourceRoot.d.ts.map +1 -0
- package/dist/src/usecases/admitSourceRoot.js +85 -0
- package/dist/src/usecases/admitSourceRoot.js.map +1 -0
- package/dist/src/usecases/browseSourceFolders.d.ts +85 -0
- package/dist/src/usecases/browseSourceFolders.d.ts.map +1 -0
- package/dist/src/usecases/browseSourceFolders.js +210 -0
- package/dist/src/usecases/browseSourceFolders.js.map +1 -0
- package/dist/src/usecases/createDeclaredSources.d.ts +13 -2
- package/dist/src/usecases/createDeclaredSources.d.ts.map +1 -1
- package/dist/src/usecases/createDeclaredSources.js +39 -0
- package/dist/src/usecases/createDeclaredSources.js.map +1 -1
- package/dist/src/usecases/deprovisionProject.d.ts +4 -5
- package/dist/src/usecases/deprovisionProject.d.ts.map +1 -1
- package/dist/src/usecases/deprovisionProject.js +3 -8
- package/dist/src/usecases/deprovisionProject.js.map +1 -1
- package/dist/src/usecases/list.d.ts.map +1 -1
- package/dist/src/usecases/list.js +83 -26
- package/dist/src/usecases/list.js.map +1 -1
- package/dist/src/usecases/listAttachableDevices.d.ts +13 -0
- package/dist/src/usecases/listAttachableDevices.d.ts.map +1 -0
- package/dist/src/usecases/listAttachableDevices.js +10 -0
- package/dist/src/usecases/listAttachableDevices.js.map +1 -0
- package/dist/src/usecases/listSourcePackages.d.ts.map +1 -1
- package/dist/src/usecases/listSourcePackages.js +95 -79
- package/dist/src/usecases/listSourcePackages.js.map +1 -1
- package/dist/src/usecases/purgeExcludedPoints.d.ts.map +1 -1
- package/dist/src/usecases/purgeExcludedPoints.js +3 -2
- package/dist/src/usecases/purgeExcludedPoints.js.map +1 -1
- package/dist/src/usecases/readFiles.d.ts.map +1 -1
- package/dist/src/usecases/readFiles.js +17 -1
- package/dist/src/usecases/readFiles.js.map +1 -1
- package/dist/src/usecases/reconcileSourcePackageRoots.d.ts +1 -1
- package/dist/src/usecases/reconcileSourcePackageRoots.d.ts.map +1 -1
- package/dist/src/usecases/reconcileSourcePackageRoots.js +2 -1
- package/dist/src/usecases/reconcileSourcePackageRoots.js.map +1 -1
- package/dist/src/usecases/revokePrincipal.d.ts +7 -5
- package/dist/src/usecases/revokePrincipal.d.ts.map +1 -1
- package/dist/src/usecases/revokePrincipal.js +29 -7
- package/dist/src/usecases/revokePrincipal.js.map +1 -1
- package/dist/src/usecases/scheduleSourceSync.d.ts +7 -0
- package/dist/src/usecases/scheduleSourceSync.d.ts.map +1 -1
- package/dist/src/usecases/scheduleSourceSync.js +6 -5
- package/dist/src/usecases/scheduleSourceSync.js.map +1 -1
- package/dist/src/usecases/searchFiles.d.ts.map +1 -1
- package/dist/src/usecases/searchFiles.js +13 -3
- package/dist/src/usecases/searchFiles.js.map +1 -1
- package/dist/src/usecases/sourcePackageInventory.d.ts +10 -2
- package/dist/src/usecases/sourcePackageInventory.d.ts.map +1 -1
- package/dist/src/usecases/sourcePackageInventory.js +24 -7
- package/dist/src/usecases/sourcePackageInventory.js.map +1 -1
- package/dist/src/usecases/triggerSourceSync.d.ts +46 -0
- package/dist/src/usecases/triggerSourceSync.d.ts.map +1 -0
- package/dist/src/usecases/triggerSourceSync.js +75 -0
- package/dist/src/usecases/triggerSourceSync.js.map +1 -0
- package/dist/src/usecases/uploadBinary.d.ts +2 -1
- package/dist/src/usecases/uploadBinary.d.ts.map +1 -1
- package/dist/src/usecases/uploadBinary.js +2 -2
- package/dist/src/usecases/uploadBinary.js.map +1 -1
- package/dist/src/usecases/vector/endpointGuard.d.ts +1 -1
- package/dist/src/usecases/vector/endpointGuard.js +1 -1
- package/dist/src/usecases/vectorActivityWatch.d.ts +20 -13
- package/dist/src/usecases/vectorActivityWatch.d.ts.map +1 -1
- package/dist/src/usecases/vectorActivityWatch.js +61 -36
- package/dist/src/usecases/vectorActivityWatch.js.map +1 -1
- package/dist/src/usecases/vectorMaintenance.d.ts.map +1 -1
- package/dist/src/usecases/vectorMaintenance.js +16 -1
- package/dist/src/usecases/vectorMaintenance.js.map +1 -1
- package/dist/src/usecases/write/index.d.ts.map +1 -1
- package/dist/src/usecases/write/index.js +17 -2
- package/dist/src/usecases/write/index.js.map +1 -1
- package/dist/src/usecases/write/occupancy.d.ts +5 -1
- package/dist/src/usecases/write/occupancy.d.ts.map +1 -1
- package/dist/src/usecases/write/occupancy.js +13 -4
- package/dist/src/usecases/write/occupancy.js.map +1 -1
- package/dist/src/usecases/write/structure.d.ts.map +1 -1
- package/dist/src/usecases/write/structure.js +20 -5
- package/dist/src/usecases/write/structure.js.map +1 -1
- package/dist/src/vector/FilesystemVectorIndex.d.ts +72 -2
- package/dist/src/vector/FilesystemVectorIndex.d.ts.map +1 -1
- package/dist/src/vector/FilesystemVectorIndex.js +211 -27
- package/dist/src/vector/FilesystemVectorIndex.js.map +1 -1
- package/dist/src/vector/InMemoryVectorIndex.d.ts.map +1 -1
- package/dist/src/vector/InMemoryVectorIndex.js +55 -7
- package/dist/src/vector/InMemoryVectorIndex.js.map +1 -1
- package/dist/src/vector/qdrantClient.d.ts +18 -1
- package/dist/src/vector/qdrantClient.d.ts.map +1 -1
- package/dist/src/vector/qdrantClient.js +48 -6
- package/dist/src/vector/qdrantClient.js.map +1 -1
- package/dist/src/wiring.d.ts +10 -12
- package/dist/src/wiring.d.ts.map +1 -1
- package/dist/src/wiring.js +101 -61
- package/dist/src/wiring.js.map +1 -1
- package/package.json +166 -54
- package/skills/connect-business-data/SKILL.md +2 -2
- package/skills/create-package/SKILL.md +18 -18
- package/skills/create-package/references/config-and-credentials.md +16 -13
- package/skills/create-package/references/connectors-and-uri-policies.md +36 -27
- package/skills/create-package/references/manifest-and-discovery.md +12 -12
- package/skills/create-package/references/runtime-and-build.md +18 -18
- package/skills/create-package/references/skills-and-context.md +35 -19
- package/skills/create-package/references/surfaces-and-design.md +16 -16
- package/skills/create-package/references/team-and-workflows.md +2 -2
- package/skills/create-package/references/test-and-ship.md +17 -17
- package/skills/create-package/references/tools-and-schemas.md +5 -5
- package/skills/create-package/scripts/{check-package.sh → check-package.mjs} +78 -64
- package/skills/create-package/scripts/{verify-loaded.sh → verify-loaded.mjs} +156 -162
- package/skills/git/SKILL.md +31 -29
- package/skills/manage-files/SKILL.md +11 -14
- package/skills/manage-sources/SKILL.md +85 -45
- package/skills/manage-uri-policy/SKILL.md +9 -6
- package/skills/sync/SKILL.md +2 -2
- package/dist/src/connectors/LocalDiskConnector.d.ts +0 -171
- package/dist/src/connectors/LocalDiskConnector.d.ts.map +0 -1
- package/dist/src/connectors/LocalDiskConnector.js +0 -1233
- package/dist/src/connectors/LocalDiskConnector.js.map +0 -1
- package/skills/manage-files/scripts/copy.sh +0 -7
- package/skills/manage-files/scripts/create.sh +0 -7
- package/skills/manage-files/scripts/mkdir.sh +0 -7
- package/skills/manage-files/scripts/modify.sh +0 -7
- package/skills/manage-files/scripts/upload.sh +0 -16
- package/skills/manage-files/scripts/write.sh +0 -29
- package/skills/manage-sources/scripts/reembed.sh +0 -23
package/README.md
CHANGED
|
@@ -11,6 +11,30 @@ The brain of Neuralis: any system a connector can reach — local directories, r
|
|
|
11
11
|
- Expose the Filesystem package routes (`/api/packages/brain-core/*`) and the agent tools (`fs_read`, `fs_write`, `fs_list`, `fs_search`).
|
|
12
12
|
- Render the workspace Files widget and the `filesystem.change` agent-card.
|
|
13
13
|
|
|
14
|
+
## Where things live
|
|
15
|
+
|
|
16
|
+
This README is the package-level map; each boundary below has its own README with the
|
|
17
|
+
mechanism, and the package root covers the contribution folders (see Public Surface).
|
|
18
|
+
|
|
19
|
+
| Path | Owns |
|
|
20
|
+
|---|---|
|
|
21
|
+
| `src/` (`src/README.md`) | lifecycle (`src/lifecycle.ts`), wiring, the cross-cutting invariants |
|
|
22
|
+
| `src/domain/` | URI, source-config and sync types; the shared access predicates; the sync floor (`DEFAULT_QDRANT_SYNC_EXCLUDES`) |
|
|
23
|
+
| `src/connectors/` | the `host` and `brain` connectors, `HostConnector` transports, the source-config stores |
|
|
24
|
+
| `src/services/` | `FilesystemService`, `BridgeService` (source CRUD, baselines, declared sources, delete + purge), `VectorHealthService` |
|
|
25
|
+
| `src/usecases/` | the connector-first / index-first pipelines: write engine, search, list, source-package discovery, rebuild |
|
|
26
|
+
| `src/sync/` | the ONE `SyncJobQueue`, job modes, watcher, scheduler |
|
|
27
|
+
| `src/vector/` | `VectorIndexPort`, Qdrant + in-memory backends, payload indexes, Vector Health actions |
|
|
28
|
+
| `src/embeddings/` (+ `clients/`) | the embedding catalog (`src/embeddings/models.ts`), the lock, spend meter, clients |
|
|
29
|
+
| `src/config/` | embedding resolution and the boot-time Qdrant env |
|
|
30
|
+
| `src/routes/` | every HTTP route (`git/*`, `sources/*`, `health/*`, `pending`, `policies/uri`, `raw`, …) with its feature gate |
|
|
31
|
+
| `src/tools/` | the four `fs_*` handlers, paired with `tools/*.json` |
|
|
32
|
+
| `src/search/` · `src/logging/` | the `grep` engine and binary resolution · the logger adapters |
|
|
33
|
+
| `app/` (`app/README.md`) | the Files widget, dock and `filesystem.change` card; `app/filesystem/{store,ui,hooks,api,domain,host,lib,ports}` and `app/cards` each have a README |
|
|
34
|
+
|
|
35
|
+
Maintainer rules, coupling traps and failure fingerprints are in the private `neuralis-brain-core`
|
|
36
|
+
skill; the security floors are in `neuralis-security-invariants`.
|
|
37
|
+
|
|
14
38
|
## URI vocabulary (the four-tuple coordinate)
|
|
15
39
|
|
|
16
40
|
Every file artifact in Neuralis lives at the coordinate
|
|
@@ -19,7 +43,7 @@ Every file artifact in Neuralis lives at the coordinate
|
|
|
19
43
|
- `uri` — `<source>://<path>` where `<source>` is the source slug (URI scheme).
|
|
20
44
|
- `connector` — the `ConnectorPort` bound to that source slug at boot.
|
|
21
45
|
- `os_uri` — optional full host-OS path, produced by `connector.resolveOsUri(uri)`.
|
|
22
|
-
`
|
|
46
|
+
`HostConnector.resolveOsUri` throws `OsUriUnresolvableError`
|
|
23
47
|
(`code: 'os_uri_host_pair_missing'`) when running in Docker without a
|
|
24
48
|
`hostRoot` configured — vectors and UI persist `os_uri: undefined`
|
|
25
49
|
rather than fake paths. The shared helper `domain/paths.ts:resolveOsUri`
|
|
@@ -27,676 +51,59 @@ Every file artifact in Neuralis lives at the coordinate
|
|
|
27
51
|
- `container_dir` — optional container-path counterpart for skill / shell
|
|
28
52
|
execution, threaded through `runtimeStack`.
|
|
29
53
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
(Qdrant-backed internal memory; reads go through `FilesystemService`, not the
|
|
35
|
-
placeholder connector). Other packages contribute more kinds — `webtop` from
|
|
36
|
-
`machine-core`, and `host` from `agent-core` (a read-WRITE view of a paired
|
|
37
|
-
device — the server or a person's own machine — through its broker over the device
|
|
38
|
-
link; 13 declared capabilities including `write`/`delete`/`move`/`mkdir`/`rmdir`
|
|
39
|
-
and `rawRead`; containment comes from the device owner's ceiling, never from a
|
|
40
|
-
missing capability. See the
|
|
41
|
-
host-plane rules below).
|
|
54
|
+
A folder's root is a `root`/`hostRoot` pair over one data home, and a `host` folder runs over the
|
|
55
|
+
in-process transport or, when its stored config names a `device`, over that device's broker
|
|
56
|
+
(`src/connectors/README.md`). The structural ops (create, rename, move, copy, delete, `mkdir`)
|
|
57
|
+
and the occupancy rule behind them are in `src/usecases/README.md`.
|
|
42
58
|
|
|
43
59
|
A connector's **manifest id is not its kind**, and only the KIND addresses the
|
|
44
|
-
registry
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
the registry kind — `local` — and answers `404 Unknown kind "local-disk"` for
|
|
48
|
-
every caller, at every role, when handed the id instead. Enumerate the live set
|
|
49
|
-
with `GET /sources/kinds` rather than deriving it from a connector id or a
|
|
50
|
-
UI label.
|
|
51
|
-
|
|
52
|
-
**A mutation against a connector that cannot perform it is a 400, never a
|
|
53
|
-
silent success.** `applySourceMutation` checks the connector's declared
|
|
54
|
-
capability SET before every `write`/`delete`/`move` — an emulated move needs
|
|
55
|
-
both `write` and `delete`. Optional-chaining these calls (the previous shape)
|
|
56
|
-
made a read-only connector report a write that never happened, while the
|
|
57
|
-
journal and the vector index recorded it as landed, so the indexed content and
|
|
58
|
-
the real file diverged with nothing to signal it.
|
|
59
|
-
|
|
60
|
-
The Files widget offers VS Code-style structural ops — create file/folder,
|
|
61
|
-
rename, move (incl. **cross-source**), copy/paste, delete (files + recursive
|
|
62
|
-
folders) — each a package capability the agent ALSO has: `fs_write` carries
|
|
63
|
-
`mkdir` and `copy` as verb keys (and recursive folder `delete` + cross-source
|
|
64
|
-
`move`), all behind the one `write` route. A rename or
|
|
65
|
-
move onto a uri that already holds a file or folder is **refused** (`DESTINATION_EXISTS`,
|
|
66
|
-
still HTTP 200 — the outcome is in `ok`): a relocation never overwrites, on any connector
|
|
67
|
-
kind. Create, copy and upload land BESIDE an existing file under a suffixed name. The
|
|
68
|
-
occupancy answer is ONE primitive (`uriOccupied` in `usecases/write/occupancy.ts`: the index
|
|
69
|
-
at any status ∪ the connector — the disk decides, never the user-scoped index alone). A HUMAN's
|
|
70
|
-
structural op commits immediately; an AGENT's stays `pending_review` (the
|
|
71
|
-
`interactive` marker — see `src/routes/README.md` + `app/filesystem/ui/README.md`).
|
|
72
|
-
Pending pages expose visible-id continuation and complete/partial/restart state. Full
|
|
73
|
-
projected vector reconstruction plus canonical URI twins occurs before slicing, so page
|
|
74
|
-
size bounds output rather than scan cost. Journal retention defaults to 32 MiB; oversize
|
|
75
|
-
journals rebuild per operation. Repair defaults to 50 visible records per batch with fresh
|
|
76
|
-
per-record authorization.
|
|
60
|
+
registry (`GET /sources/kinds/:kind/schema` answers `404 Unknown kind "<x>"` for an
|
|
61
|
+
id or a UI label, for every caller, at every role). Enumerate the live set with
|
|
62
|
+
`GET /sources/kinds` rather than deriving it from a connector id or a UI label.
|
|
77
63
|
|
|
78
64
|
URI schemes you will see in production:
|
|
79
65
|
|
|
80
66
|
| Scheme | Connector kind | Source of slug | Notes |
|
|
81
67
|
|---|---|---|---|
|
|
82
68
|
| `brain://` | `brain` | builtin | Vector-only memory, agent-scoped by default |
|
|
83
|
-
| `data://` | `
|
|
84
|
-
| `data.X://` | `
|
|
85
|
-
| `packages://` | `
|
|
86
|
-
| `app://` | `
|
|
87
|
-
| `neuralis://` | `
|
|
88
|
-
| `neuralis-skills://` | `
|
|
89
|
-
| `machine://` | `webtop` | machine-core | Webtop sandbox; declared by `machine-core`, not brain-core.
|
|
90
|
-
| `os://<key>` | — | **decorative only** | Renders inside the system prompt's `<runtime_stack>` block as a label for `NEURALIS_MOUNT_<KEY>` mounts. **No connector handles `os://`.** Agents addressing files inside a mount use the `
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
`
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
`
|
|
103
|
-
|
|
104
|
-
`
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
enable-semantics land later.
|
|
114
|
-
|
|
115
|
-
### Git engine (`src/git/GitRunner.ts`)
|
|
116
|
-
|
|
117
|
-
`GitRunner` wraps `simple-git` over the container's baked-in git binary (reused
|
|
118
|
-
via `resolveSearchBinaries`, the same path as `gitGrep`). It is keyed on a
|
|
119
|
-
**resolved checkout root** (`resolveLocalRoot().resolvedRoot`) — the SAME root the
|
|
120
|
-
`local` connector mounts and the shell uri-policy gate keys on — and enforces four
|
|
121
|
-
containment floors: (1) `cwd` = root + a sub-path guard + a
|
|
122
|
-
`git rev-parse --show-toplevel --absolute-git-dir` clamp, so a `.git` ABOVE the
|
|
123
|
-
source **and** a `.git` FILE/symlink whose git-dir redirects OUTSIDE the source
|
|
124
|
-
are both rejected (`GitOutsideSourceError`, one memoized spawn — the
|
|
125
|
-
containment close; the git-dir is compared against the CANONICALIZED root, so a
|
|
126
|
-
legitimately symlinked mount path never false-rejects. Limit: a standalone-mounted
|
|
127
|
-
LINKED WORKTREE, whose git-dir lives in the main repo outside the source, fails
|
|
128
|
-
soft — no badges); (2) reject leading-dash refs/paths (option-injection)
|
|
129
|
-
and require `--` before path operands; (3) `-c core.hooksPath=/dev/null
|
|
130
|
-
-c core.fsmonitor=false -c core.pager=cat` on every call, defeating a malicious
|
|
131
|
-
repo-local `.git/config` (closes the malicious-checkout RCE class); (4) strip EVERY
|
|
132
|
-
ambient `GIT_*` env var, then set `GIT_CONFIG_GLOBAL=/dev/null`,
|
|
133
|
-
`GIT_CONFIG_SYSTEM=/dev/null`, `GIT_TERMINAL_PROMPT=0` and `GIT_OPTIONAL_LOCKS=0`.
|
|
134
|
-
Simple-git v4 requires top-level `allowEnvironment` naming exactly those four
|
|
135
|
-
fixed defensive overrides, together with the existing explicit `unsafe:{…}`
|
|
136
|
-
opt-ins for floors (3)/(4). It blocks those same flags/env as attacker-injection
|
|
137
|
-
vectors — stripping ALL ambient `GIT_*` (not a denylist) remains the robust rule. It exports the
|
|
138
|
-
pure helpers `classifyGitOp(op, args) → 'read'|'mutating'|'destructive'` (for the
|
|
139
|
-
later approval gate) and `normalizeStatus`. Read primitives are fully implemented;
|
|
140
|
-
local-mutating ops are implemented but NOT yet tool-exposed (they wait on the
|
|
141
|
-
uri-policy write gate + approval guard).
|
|
142
|
-
|
|
143
|
-
**Remote ops.** `fetch`/`push`/`pull`/`mergeFetched`
|
|
144
|
-
run through a SEPARATE `#spawnGitRemote` variant — raw `node:child_process`, NEVER
|
|
145
|
-
simple-git (the remote `-c` set — `credential.helper`, `core.sshCommand`,
|
|
146
|
-
`protocol.*` — is exactly what simple-git's vulnerability check blocks, so routing
|
|
147
|
-
it through the simple-git ctor `config:` would be boot-fatal). In addition to the
|
|
148
|
-
three local hardening flags it prepends the full **protected-config-aware
|
|
149
|
-
remote-hardening `-c` set** (only 4 git config keys are protected; everything else
|
|
150
|
-
is honored from a hostile repo-local `.git/config`, and `-c` = command scope wins):
|
|
151
|
-
`credential.helper=` (empty RESET first — a hostile repo-local helper is BOTH a
|
|
152
|
-
`!shell` RCE AND receives the pushed token on git's unconditional `store`
|
|
153
|
-
broadcast) then `credential.helper='!<constant helper>'`; `core.askPass=` +
|
|
154
|
-
`credential.interactive=false` (a repo-local `core.askPass` is consulted before the
|
|
155
|
-
`GIT_TERMINAL_PROMPT=0` gate); `protocol.{file,ext,git}.allow=never`; pinned
|
|
156
|
-
`remote.<n>.{upload,receive}pack`; `core.sshCommand=`/`core.gitProxy=`. The
|
|
157
|
-
credential helper snippet is a **CONSTANT string** (no interpolation of ANY dynamic
|
|
158
|
-
value — username/host/token arrive via child-env `NRS_GIT_{USER,HOST,TOKEN}`, which
|
|
159
|
-
the shell reads but never re-evaluates as code); it reads `protocol=`/`host=` from
|
|
160
|
-
stdin and releases the token ONLY when `protocol` is `https` AND `host` equals the
|
|
161
|
-
BINDING host (`NRS_GIT_HOST`) — this stdin host-match, NOT any URL derivation, is
|
|
162
|
-
the redirect-class defense (`insteadOf` is un-`-c`-clearable). **HTTPS-only** (SSH
|
|
163
|
-
fails closed). The token is present ONLY on the network-only invocations
|
|
164
|
-
(`fetch`/`push`); **`pull` = `fetch` (token in env) + `mergeFetched` (`merge
|
|
165
|
-
--ff-only`, NO token in env)** so the token never rides a working-tree checkout/merge
|
|
166
|
-
(a hostile `filter`/`textconv`/`merge` driver is attacker-NAMED ⇒ un-`-c`-clearable,
|
|
167
|
-
closed only by keeping the token out of that process). **No clone, no force-push.**
|
|
168
|
-
The push token is the CALLER'S OWN `git.<host>.pat`/`.oauth.accessToken`, resolved
|
|
169
|
-
per-request via `src/git/tokenResolve.ts` (`resolveGitToken`) over the
|
|
170
|
-
kernel-injected `ctx.credentials` cascade at the `ensureAccess`-validated scope, and
|
|
171
|
-
never cached on `GitRunner`. `status(subPath?)` is **pathspec-scoped** when given a folder (the per-file
|
|
172
|
-
badge path) — `git status -- <subPath>` so it costs `O(subtree)`, never `O(repo)`
|
|
173
|
-
— and an optional `GitRunnerOptions.timeoutMs` caps a slow scan (kills the child →
|
|
174
|
-
`GitTimeoutError`). That timeout is an INACTIVITY timer — it resets on every output chunk,
|
|
175
|
-
capping silence, not wall-clock — so the potentially-large read primitives (`diff`,
|
|
176
|
-
`logGraph`, `show`, `cat-file` type/size) run on a direct-spawn path with an engine-level
|
|
177
|
-
**output accumulation bound** (~4 MB combined stdout+stderr; the child is KILLED past it →
|
|
178
|
-
`GitOutputLimitError`): every byte is buffered in-process before any route cap can trim it,
|
|
179
|
-
and a continuously-streaming command trips no inactivity timer, so the byte bound is the real
|
|
180
|
-
memory floor beneath the route caps. `blobType`/`blobSize` (`git cat-file -t`/`-s`) let a
|
|
181
|
-
caller refuse a non-blob or oversized object BEFORE buffering its content; the `<rev>:<path>`
|
|
182
|
-
colon form is composed inside the engine (the literal rev `index` is the sentinel for the
|
|
183
|
-
staged blob, `:0:<path>`) — caller text never enters it raw. The per-commit / outgoing file
|
|
184
|
-
lists ride ONE two-tree form: `treeDiffFiles(from, to)` = `git diff-tree -r
|
|
185
|
-
--name-status -M -z <from> <to>` (NUL-safe `parseNameStatusZ`; a rename record is
|
|
186
|
-
`R<score> NUL <old> NUL <new>`), with `commitFiles(sha, {hasParent})` composing
|
|
187
|
-
`<sha>^ ↔ <sha>` (first-parent, merge-correct) or `EMPTY_TREE_SHA ↔ <sha>` for a root
|
|
188
|
-
commit — the caller answers the parent question via `commitParents(sha)` (`rev-list
|
|
189
|
-
--parents -n 1`), never by catching errors. **The single-arg `diff-tree -r <sha>` and the
|
|
190
|
-
`<sha>^!` forms are EMPTY on merge commits (live-proven) — never use them, and never
|
|
191
|
-
`--root` (it prepends a header record the `-z` parse would have to skip).** The outgoing
|
|
192
|
-
readers are `outgoingShas(cap=500)` (`rev-list @{u}..HEAD`, `null` — never a throw — when
|
|
193
|
-
no upstream/any failure) and `resolveUpstreamHead()` (`{baseSha|null, headSha}`, server-side
|
|
194
|
-
ref resolution so no route ever takes caller refs; **`baseSha` is `merge-base(@{u}, HEAD)`,
|
|
195
|
-
NOT `@{u}`** — a two-tree diff against a DIVERGED remote tip would phantom remote-only files
|
|
196
|
-
as Deleted, disagreeing with the two-dot `@{u}..HEAD` commit set; on an ahead-only branch the
|
|
197
|
-
merge base equals `@{u}`, on unrelated histories it degrades to `EMPTY_TREE_SHA`); both are
|
|
198
|
-
LOCAL-ref reads — zero network; the network reach is the remote-ops surface (fetch/pull/push, below).
|
|
199
|
-
|
|
200
|
-
### Git-aware file tree — per-file badges
|
|
201
|
-
|
|
202
|
-
The first consumer of `GitRunner`: `FilesystemService.list` stamps a
|
|
203
|
-
**git working-tree letter** (`M`/`A`/`D`/`R`/`?`, the 2-char porcelain XY code) onto
|
|
204
|
-
dirty files under a git-backed **local** source, surfaced as a new `git?` field on each
|
|
205
|
-
`ListFolderResponse` file. Both the `fs_list` tool (a bracketed `[M]` glyph in the
|
|
206
|
-
tree/flat markdown + legend) and the Files-widget tree (a colour-coded `GitStatusBadge`)
|
|
207
|
-
render it as a **third, orthogonal decoration axis** — a file is simultaneously `synced`
|
|
208
|
-
(sync tone) AND `M` (git) — never folded into the sync tone / `getNodeVisualState`
|
|
209
|
-
precedence. Computation lives in `src/git/sourceGitStatus.ts`. **Folders** with dirty
|
|
210
|
-
descendants get a rollup dot (`rollupFolderGit` picks the most-significant descendant), so
|
|
211
|
-
changes are visible from the root without drilling in. **Both client list paths carry the
|
|
212
|
-
`git` field** — the root load (`listSlice`) AND the lazy folder-expand
|
|
213
|
-
(`filesystemStore.ensureFolderLoaded`, what the tree uses when you open a folder); a
|
|
214
|
-
server-sourced `FileNode` field is dropped if only one is wired.
|
|
215
|
-
|
|
216
|
-
**Scaling (million-file safe).** The filesystem serves huge sources by paying only for
|
|
217
|
-
the folder being viewed, so git enrichment does too: `sourceGitStatus` runs `git status`
|
|
218
|
-
**pathspec-scoped to the listed folder** (never the whole repo), behind an **inactivity
|
|
219
|
-
timeout** (bounds git's silent whole-tree scan — the huge-repo `lstat` storm — so a
|
|
220
|
-
too-slow scan is skipped, not blocked), a short **per-`(root,folder)` TTL cache**, and a
|
|
221
|
-
cheap `.git` stat pre-gate (non-git sources — the majority — pay one `stat` and skip). It
|
|
222
|
-
is strictly additive and **fail-soft**: any problem yields no badges, never an error and
|
|
223
|
-
never a slowdown of the vector/connector list paths.
|
|
224
|
-
|
|
225
|
-
**Security + limits.** Badges are attached AFTER the source-level access filter, so a
|
|
226
|
-
badge never lands on a file from a source the caller cannot see. The status is read-only
|
|
227
|
-
(no creds/network/mutation); the listed folder passes `GitRunner`'s existing path guard.
|
|
228
|
-
**UX limit:** per-file badges require the source ROOT to BE the git checkout toplevel — a
|
|
229
|
-
source that is a *subdir* of a larger repo shows no badges (the toplevel clamp fails soft).
|
|
230
|
-
Files INSIDE the nested checkouts of a **multi-repo source** (see the multi-repo model below) **now DO get
|
|
231
|
-
per-file badges/rollups**: when the source root has no `.git` the badge path
|
|
232
|
-
enumerates the discovered nested roots (the injected `gitRepos` port =
|
|
233
|
-
`discoverSourceRepoRoots`) and runs a pathspec-scoped `git status` against each in the
|
|
234
|
-
listed scope, rebasing the match keys by the `<rel>/` prefix — up to the ≤25-repo
|
|
235
|
-
discovery cap of nested `git status` on a root view (each timed + cached, the documented
|
|
236
|
-
first-view cost). Untracked files list individually (`--untracked-files=all`, VS-Code
|
|
237
|
-
parity; `.gitignore` still excludes ignored trees).
|
|
238
|
-
|
|
239
|
-
### File tree — lazy hydration and convergence
|
|
240
|
-
|
|
241
|
-
The tree loads in two independent stages.
|
|
242
|
-
|
|
243
|
-
**Stage 1 — source headers, no walk.** The root call asks for the source REGISTRY
|
|
244
|
-
only (`sources_only`), so every header paints before a single connector is touched.
|
|
245
|
-
Nothing about one source's reachability can delay another's row. Counts stay opt-in
|
|
246
|
-
and belong to the Sources panel.
|
|
247
|
-
|
|
248
|
-
**Stage 2 — content on expand.** Opening a source branch (or a folder, or following
|
|
249
|
-
a deep link) lists exactly that folder, non-recursively. Each source carries its own
|
|
250
|
-
fetch state — `idle` / `loading` / `ready` / `error` — shown as a dim + an in-place
|
|
251
|
-
spinner on its header. That state is the CLIENT's; it is orthogonal to the source's
|
|
252
|
-
connector liveness (`live`), which dims the same row for a different reason and says
|
|
253
|
-
so in the tooltip. A listing that failed is not retried automatically; the header's
|
|
254
|
-
refresh control retries it.
|
|
255
|
-
|
|
256
|
-
**Converging on server truth.** Refresh is deliberately more than a re-fetch. The
|
|
257
|
-
plain source/root load is upsert-only — it merges what came back and leaves the rest
|
|
258
|
-
alone, which is right for a partial or truncated response but means a row the server
|
|
259
|
-
no longer has can never disappear through it. So the toolbar **Refresh** re-lists the
|
|
260
|
-
visible folders of every hydrated source and applies absence semantics: an
|
|
261
|
-
external-only file that is gone is pruned; a tracked file that is gone becomes a
|
|
262
|
-
struck-through, still-revertible tombstone; a folder is pruned only when it is
|
|
263
|
-
external-only and parents nothing live. A **truncated** listing never prunes (absence
|
|
264
|
-
is not deletion there) and instead flags the source as possibly-stale on its header.
|
|
265
|
-
Every mutation converges the same way, but scoped to the parents it actually touched.
|
|
266
|
-
|
|
267
|
-
**Per-source refresh.** Each source header carries its own refresh control, in the
|
|
268
|
-
same slot the spinner uses. It works on a collapsed source too — an explicit click is
|
|
269
|
-
a user intent, unlike the automatic event-driven loads, which never list a collapsed
|
|
270
|
-
source — and it opens the branch so the result is visible.
|
|
271
|
-
|
|
272
|
-
**When events go missing.** The filesystem event channel is live-only: no replay, no
|
|
273
|
-
cursor. Anything emitted while the connection was down is gone for good. The widget
|
|
274
|
-
therefore re-converges automatically after a connection flip, which is what keeps
|
|
275
|
-
Refresh a deliberate action rather than routine repair.
|
|
276
|
-
|
|
277
|
-
### Search view
|
|
278
|
-
|
|
279
|
-
The **Search** sidebar view replaces the tree with a search surface. The query
|
|
280
|
-
input rides INLINE in the sidebar header (title flips to "Search"; the toggle —
|
|
281
|
-
or Escape — clears the query and closes even with text present). Three modes:
|
|
282
|
-
**Keyword** (default — the indexed `filter` engine, term-frequency ranked),
|
|
283
|
-
**Semantic** (embedding/hybrid), **Grep** (literal scan of a live connector
|
|
284
|
-
scope; disabled when only `brain://` is reachable). A separate filter row
|
|
285
|
-
(filename glob, category, status, tags, created range, scope-to-folder) combines
|
|
286
|
-
with Keyword AND Semantic; grep uses the filename + scope filters. Results are a
|
|
287
|
-
single list with match badges `[H]/[S]/[K]`, scores, grep `uri:line`, and
|
|
288
|
-
per-mode loading/error states; backend error rows (e.g. grep scope guidance)
|
|
289
|
-
render inline. The wire body is the tool vocabulary (`search_mode`) — pinned by
|
|
290
|
-
`test/ui/searchApi.wire.test.ts`.
|
|
291
|
-
|
|
292
|
-
### Git Control panel — source-control view
|
|
293
|
-
|
|
294
|
-
The **Git Control** sidebar view (a top-level sibling toggle beside *Explorer* and
|
|
295
|
-
*Search* — `sidebarView: 'tree' | 'search' | 'git'` on the Files store) replaces the file
|
|
296
|
-
tree in place with a source-control surface for one focused git repository. It
|
|
297
|
-
shows a fixed header zone — a **repo-switcher** over the caller's accessible git
|
|
298
|
-
REPOSITORIES (a root-repo source contributes one entry; a multi-repo source lists every
|
|
299
|
-
nested checkout, lazy-fetched from `GET /git/repos` — the multi-repo model below; header + rows
|
|
300
|
-
render the two-part **`slug:// Name`** label from `git.repoName` / the nested repo's name
|
|
301
|
-
— dim-mono source coordinate + the repo's real name; slug-only fallback for older cached
|
|
302
|
-
payloads), the current **branch** (a quick-pick to switch / create / delete), a
|
|
303
|
-
**collapsible commit box** behind a `canWrite`-gated **Commit** toggle in the branch bar
|
|
304
|
-
(default-collapsed so the change lists get the vertical space, staged-count pill on
|
|
305
|
-
the toggle, open/closed state persisted component-local like the section prefs,
|
|
306
|
-
`Cmd/Ctrl+Enter` in the textarea commits), and an always-visible **tree/list
|
|
307
|
-
toggle** — over a VS-Code-style **pane stack**
|
|
308
|
-
(the generic `app/filesystem/ui/StackedSections.tsx`): collapsible,
|
|
309
|
-
drag-resizable sections with always-visible fixed-height headers (uppercase title +
|
|
310
|
-
chevron + count + hover-revealed bulk **stage / unstage / discard** actions; trailing
|
|
311
|
-
collapsed headers pile at the bottom, and each expanded body scrolls itself). Resizing
|
|
312
|
-
is VS-Code-sash style: **the section HEADER itself drags** the boundary above it
|
|
313
|
-
(pointer travel past a 4px threshold = resize; a short press stays a click-to-toggle —
|
|
314
|
-
the toggle fires from pointerup, NOT the button's onClick, because pointer capture
|
|
315
|
-
retargets the post-drag click; keyboard activation `detail===0` keeps the button path),
|
|
316
|
-
plus a thin dedicated separator strip above every section whose predecessor is expanded —
|
|
317
|
-
**including a COLLAPSED one**, so a collapsed pane shows a grabbable divider. A section's
|
|
318
|
-
header (and that strip) is a drag handle whenever the section **directly above it is
|
|
319
|
-
expanded**: an expanded section resizes the boundary, and a collapsed section **OPENS +
|
|
320
|
-
sizes itself in the same gesture** — so a collapsed pane is resizable without the
|
|
321
|
-
open-resize-close dance (owner fixes, 2026-07-14). Two layout invariants keep collapsed
|
|
322
|
-
headers where the user expects: expanded-body `flex-grow` factors are **normalized so their
|
|
323
|
-
sum is always ≥ 1** (`growScaleForSum`) — a CSS quirk otherwise leaves a lone small weight
|
|
324
|
-
filling only a fraction of the pane and stranding collapsed headers mid-stack instead of at
|
|
325
|
-
the bottom — and re-expanding a section dragged to the min-body floor RESCUES its starved
|
|
326
|
-
weight to the default (`MIN_USABLE_WEIGHT`). Gestures starting on the hover actions never
|
|
327
|
-
drag (`data-section-actions` guard). Each change **row** is the VS-Code layout: a
|
|
328
|
-
shared **file-type icon** LEFT (the same `pickFileIcon` table as the Files tree, extracted
|
|
329
|
-
to `app/filesystem/lib/fileIcon.tsx`), the filename (always visible), the dimmed dir path
|
|
330
|
-
(truncates FIRST), and the **status letter RIGHT** — the per-row Stage/Unstage/Discard
|
|
331
|
-
cluster is a hover-revealed ABSOLUTE overlay that reserves no width. The THREE
|
|
332
|
-
sections: **Changes** (always renders, empty-state text at 0 — and INCLUDES untracked
|
|
333
|
-
files as a green **`U`**, VS-Code parity: untracked rows get Stage only, no Discard),
|
|
334
|
-
**Staged Changes** (vanishes when empty), and **Graph** (always present,
|
|
335
|
-
collapsed by default, `defaultWeight: 2` so the first expand is usable without an
|
|
336
|
-
immediate resize) — the commit DAG renders INSIDE the panel; the former FileEditor
|
|
337
|
-
Graph tab (`'git-graph'` / `gitGraphSource`) is REMOVED. Section `{collapsed, weight}`
|
|
338
|
-
prefs persist to component-local localStorage (`neuralis.filesystem.gitPanel.sections`,
|
|
339
|
-
same guards as the sidebar width); collapsing a section unmounts its body, so the graph
|
|
340
|
-
only fetches while open. The graph itself is **scroll-paged and virtualized**:
|
|
341
|
-
200-commit pages append as an IntersectionObserver sentinel nears the bottom (`skip` +
|
|
342
|
-
the exact `hasMore` probe, de-duped by sha so a concurrent commit shifting the pages
|
|
343
|
-
produces a dropped duplicate, never a gap; a load that leaves the sentinel still
|
|
344
|
-
visible re-pages on its own — the observer only reports status changes — and a FAILED
|
|
345
|
-
fetch keeps the already-loaded pages behind an error strip instead of wiping the
|
|
346
|
-
graph), and rows render through
|
|
347
|
-
`@tanstack/react-virtual` as fixed-height per-row SVG slices — the pure
|
|
348
|
-
`gitGraphLayout` pre-splits every lane line and fork/merge curve at the row edges so
|
|
349
|
-
the DOM stays bounded on unbounded histories. It also **live-refreshes** off the
|
|
350
|
-
existing git-badge SSE with no new channel: the debounced badge path bumps a store
|
|
351
|
-
`gitLogNonce` for the FOCUSED repo only, and the graph re-fetches its head window and
|
|
352
|
-
REPLACES the loaded prefix (decorations move on checkout, so prepending new SHAs would
|
|
353
|
-
go stale); the graph's own header refresh button forces the same head-window path and is
|
|
354
|
-
THE graph refresh (the old Graph section-header remount button was dropped as a duplicate).
|
|
355
|
-
Graph rows are **commit drill-ins**: the rows render **flush (no card
|
|
356
|
-
background)** to fill the width; **hovering** a collapsed row floats a **full-message
|
|
357
|
-
card** (subject + the full commit **body** — `%b`, carried on `GitGraphCommit.body`, a
|
|
358
|
-
NUL-terminated `logGraph` record so a multi-line body is parsed safely, server-capped —
|
|
359
|
-
plus sha/author/relative-date), anchored to the graph ROOT and `pointer-events-none` so it
|
|
360
|
-
never steals the hover. Clicking a row expands it inline (the virtualizer switches that
|
|
361
|
-
row to dynamic `measureElement` height; collapsed rows keep the fixed-height fast path)
|
|
362
|
-
with the body + sha/author/relative-date and the commit's file list — lazy-fetched ONCE
|
|
363
|
-
per sha from `GET /git/commit` and cached client-side; a file row opens the commit↔parent
|
|
364
|
-
side-by-side diff, and a hover **Open Changes** action (an absolute overlay) opens the
|
|
365
|
-
multi-file stacked tab (`GitCommitChangesView`, one lazy diff card per file — the
|
|
366
|
-
PendingChangesPanel pattern). **Outgoing indication** (LOCAL refs — the network
|
|
367
|
-
reach is the remote-ops popover below): **BLUE is reserved as the UNPUSHED language** — commits not yet on the
|
|
368
|
-
upstream render a HOLLOW **blue** dot (`OUTGOING_COLOR`; the HEAD ring follows the dot color;
|
|
369
|
-
pushed dots keep their lane color, and the lane palette moved blue to its LAST slot so an
|
|
370
|
-
early fork lane can't collide with it), an expandable **"Outgoing changes" divider row**
|
|
371
|
-
(blue family) sits above the first outgoing run (expanding shows the
|
|
372
|
-
aggregated merge-base↔HEAD file list from `GET /git/outgoing`; its Open Changes opens the same
|
|
373
|
-
stacked tab over the range), ref chips are explicitly classed (HEAD/local = **violet** filled —
|
|
374
|
-
violet stays the local-ref language, `origin/*` = outline + cloud glyph, tags = amber), and the
|
|
375
|
-
branch bar shows `↑ahead ↓behind` chips whenever `/git/status` carries an upstream — per-pair
|
|
376
|
-
arrow/number spans, the ahead pair blue (it counts exactly the outgoing commits), the behind
|
|
377
|
-
pair neutral. For a **writer** (`canWrite`) the chips ride inside the **fetch/pull/push
|
|
378
|
-
trigger** button (a sync glyph when there is no upstream); clicking it opens the **remote-ops
|
|
379
|
-
popover** — Pull / Push / Fetch when the source grants uri-policy
|
|
380
|
-
`exec` (`canExec`), otherwise a precise **"grant exec"** affordance. A read-only caller sees
|
|
381
|
-
the display-only chips. No upstream ⇒ the chips don't render (the trigger stays for a writer). Clicking a change row
|
|
382
|
-
opens a **side-by-side diff** (below). The focused repo is the store PAIR
|
|
383
|
-
`(activeSource, activeRepoRel)` — the source slug plus the nested checkout's dir name
|
|
384
|
-
(`''` = the source root — the multi-repo model below) — never a second slug concept; the graph derives its
|
|
385
|
-
`source`/`repo` props from it and remounts on the pair, so pages/decorations never leak
|
|
386
|
-
across checkouts. (Agent-facing git tools + the `PendingInteraction` approval ladder are a
|
|
387
|
-
SEPARATE later increment — the panel here is the HUMAN face.)
|
|
388
|
-
|
|
389
|
-
**Source-level `git` field.** Source rows now carry an optional
|
|
390
|
-
`git?: { isRepo: boolean; repoName?: string; repoNames?: string[]; branch?: string;
|
|
391
|
-
canWrite?: boolean; containsRepos?: boolean }`, present when the source's LOCAL root is a git
|
|
392
|
-
checkout — or, for a multi-repo source, stamped `{ isRepo: false, containsRepos: true, repoNames? }` when a NON-repo
|
|
393
|
-
root contains nested depth-1 checkouts (that row carries NO branch/canWrite — per-repo state
|
|
394
|
-
lives on `GET /git/repos`). It drives a small `GitBranch` overlay + a repository-NAME hint on
|
|
395
|
-
the source header in BOTH the file tree (`SourceBranchHeader`) and the Sources config panel
|
|
396
|
-
(`SourcesPanel`), and populates the Git Control repo-switcher. `repoName` is the
|
|
397
|
-
repository's REAL name — the git **remote origin's basename** (`repoNameFromOrigin`,
|
|
398
|
-
mount-independent: a checkout mounted at `/mounts/repo` still reads `Neuralis`, not `repo`),
|
|
399
|
-
falling back to the checkout dir basename (`repoNameFromRoot`) for a no-remote repo / linked
|
|
400
|
-
worktree; a NAME only, never the URL/host/org (`scrubHostPath` discipline). The **file tree
|
|
401
|
-
shows the repository NAME(s) beside the source name, not the branch** (a source can hold several
|
|
402
|
-
repos; the branch lives in the Git Control panel): a root-repo source its `repoName`, a
|
|
403
|
-
`containsRepos` source its nested `repoNames` (the checkout DIRS, first two + `+N`). Those nested names are
|
|
404
|
-
**per-path READ-gated server-side** in BOTH summary builders (`filterVisibleRepoNames`,
|
|
405
|
-
`resolvePermissions(<source>://<rel>/).read` — "per-file uri-policy wherever file NAMES are
|
|
406
|
-
revealed"; a nested checkout the caller cannot read is absent). The **`/git/repos`
|
|
407
|
-
route applies the SAME `filterVisibleRepoNames` per-path gate** (tree and route are unified —
|
|
408
|
-
no name residual is left), keeping only the root `rel:''` row unconditionally. `repoName` also
|
|
409
|
-
sits behind the two-part `slug:// Name` label on the panel header, switcher rows, and graph
|
|
410
|
-
header; absent (older cached payloads) ⇒ slug-only fallback. It is **orthogonal to the sync
|
|
411
|
-
tone** — a source is simultaneously synced (blue) AND a git repo — rendered alongside, never
|
|
412
|
-
replacing it. The field is CHEAP and bounded O(1)/source (a `.git` stat + an O(1) `.git/HEAD`
|
|
413
|
-
branch read via `getSourceGitSummary`, TTL-cached — never a whole-repo status on the list path)
|
|
414
|
-
and is stamped in the **two** source-summary builders: `FilesystemService.list`'s
|
|
415
|
-
`filteredSources` (the tree) and `GET /sources` (the config panel), both AFTER the
|
|
416
|
-
source-level access filter so a non-member never infers a project's git repos. `dirty`
|
|
417
|
-
stays deferred off the list path; ahead/behind now ride the ON-DEMAND panel status
|
|
418
|
-
(below) — the source-ROW summary itself stays O(1) and tracking-free. Refreshing the
|
|
419
|
-
remote-tracking refs (fetch + credentials) is the remote-ops surface above.
|
|
420
|
-
|
|
421
|
-
**Multi-repo sources — nested checkouts.** A source does not have to BE a checkout to
|
|
422
|
-
be a git surface. The model: **root `.git` wins** — a source whose root IS a repo behaves
|
|
423
|
-
exactly as before (one repo, no scan); only when the root is NOT a repo does brain-core run
|
|
424
|
-
a **depth-1 discovery** (`discoverSourceRepoRoots`, `src/git/sourceGitControl.ts`): direct
|
|
425
|
-
children with their own `.git`, found by a STREAMING `fs.opendir` scan (≤ 500 dirents
|
|
426
|
-
examined — a 100k-child source never pays a full `readdir`; ≤ 25 repos reported,
|
|
427
|
-
name-sorted), zero git spawn, its own 30 s TTL (separate from the 2.5 s summary TTL —
|
|
428
|
-
discovery carries no branch payload and checkout creation is rare; deletion is covered by
|
|
429
|
-
the route's fresh re-check). This ONE discovery cache feeds three consumers — the source
|
|
430
|
-
summary's `containsRepos` + `repoNames`, the `GET /git/repos` route, and the watcher
|
|
431
|
-
expansion. **Containment is enforced AT DISCOVERY, not only in the route
|
|
432
|
-
ladder** (the load-bearing rule): `hasDotGit` uses `stat`, which FOLLOWS SYMLINKS, and two
|
|
433
|
-
consumers — the `GET /git/repos` listing and the watcher expansion below — consume the
|
|
434
|
-
discovered set WITHOUT ever passing the route ladder, so a symlinked child pointing at a
|
|
435
|
-
foreign repo would otherwise leak that repo's existence + branch and plant a host-side
|
|
436
|
-
watcher on a foreign `.git`. Every candidate must therefore `realpath`-resolve inside the
|
|
437
|
-
source root — the child dir AND its `.git` entry (a contained child whose `.git` is itself
|
|
438
|
-
a symlink onto a foreign repo is dropped the same way; a regular `.git` dir or a
|
|
439
|
-
linked-worktree `.git` FILE stays admitted). The `containsRepos` summary probe applies the
|
|
440
|
-
SAME predicate with a stop-at-first-find short-circuit — the ONLY new list-path cost,
|
|
441
|
-
fail-soft. Residual: a `.git` FILE whose `gitdir:` line points outside the root would need
|
|
442
|
-
a git spawn to detect (banned on this zero-spawn path) — a known, logged residual.
|
|
443
|
-
|
|
444
|
-
Every `git/*` route then accepts an optional **`repo` selector** (query for GET, JSON body
|
|
445
|
-
for POST) naming ONE discovered child checkout; a root-repo source rejects any `repo`.
|
|
446
|
-
The shared resolver validates it as belt AND braces — syntactic whitelist (one path
|
|
447
|
-
segment, 1–128 chars, no `.`/`..`/`.git`/leading `~`; a malformed value is a request-shape
|
|
448
|
-
400 BEFORE any access check, uniform whether or not the source exists) → membership in the
|
|
449
|
-
TTL-cached discovered set (no arbitrary-subdir probing, even syntactically valid names) →
|
|
450
|
-
a FRESH `.git` stat + realpath containment re-check on the nested root AND its `.git`
|
|
451
|
-
target (closes discovery-TTL races and case-aliasing) — and every failure past the belt
|
|
452
|
-
maps onto ONE constant 404, never revealing whether the child exists. With `repo` set the git ops re-root at the
|
|
453
|
-
nested checkout and **every policy URI carries the `<repo>/` prefix**, composed ONCE in
|
|
454
|
-
the resolver (`policyPrefix`) so no gate site can drift: root-gated routes gate
|
|
455
|
-
`<slug>://<repo>/` (trailing slash — the exact root form a `<repo>/**` path rule matches,
|
|
456
|
-
pinned by test), per-file routes gate `<slug>://<repo>/<path>`, and `guardWritePaths`
|
|
457
|
-
prefixes EVERY gated write path — the whole-file `paths[]` AND the hunk-extracted ones (a
|
|
458
|
-
`+++ b/<path>` patch header gates as `<repo>/<path>` through the nested form). Without
|
|
459
|
-
`repo` everything is byte-identical to the single-repo behavior.
|
|
460
|
-
|
|
461
|
-
`GET /git/repos?source=<slug>` is the repo-picker listing: `{ rel, name, branch?,
|
|
462
|
-
canWrite, canExec }` rows — the realpath-FILTERED discovery, branch-enriched per nested
|
|
463
|
-
root, plus a **per-repo `canWrite`** (the two-plane gate evaluated at `<slug>://<repo>/` —
|
|
464
|
-
root-level write capability is the wrong granularity in both directions: a `<repo>/**`
|
|
465
|
-
write-deny flips that row read-only while siblings stay writable, and a per-repo allow is
|
|
466
|
-
not hidden by a stricter root) and a **per-repo `canExec`** (the uri-policy `exec`
|
|
467
|
-
capability at the same granularity — whether the remote push/pull/fetch ops are enabled,
|
|
468
|
-
so the panel renders the popover or the grant-exec affordance without a click-then-403).
|
|
469
|
-
Names-only ⇒ **per-path READ-gated** (`filterVisibleRepoNames`, `<slug>://<rel>/`.read — keyed on the DIR, never the display `name`, which is the origin basename when the checkout has one, the same order as `repoName`): a nested repo whose name is read-denied is DROPPED from the listing — the SAME gate
|
|
470
|
-
the file-tree repo-name stamp applies, unified onto this route (`resolveGitSource` runs
|
|
471
|
-
with `ignoreRepo`, so the listing spans all nested repos of the source, each then
|
|
472
|
-
name-gated). The root repo (`rel:''`) is ALWAYS kept and source-root-gated (a `<slug>:///`
|
|
473
|
-
form would mis-gate it), so the ONLY residual is a fully read-denied ROOT-level repo still
|
|
474
|
-
listing its own name + branch (source-root name-level exposure, the same family as
|
|
475
|
-
`/git/status`). Rows carry `rel`/`name` only; the server-side nested root (a host path) is
|
|
476
|
-
never serialized.
|
|
477
|
-
|
|
478
|
-
The `.git` badge **watchers expand per nested repo**: `SyncWatchService` derives its
|
|
479
|
-
git-watch desired set from the FILTERED discovery — one `.git` watcher per nested checkout
|
|
480
|
-
of a `containsRepos` source (root-repo sources keep exactly one), so external commits in
|
|
481
|
-
ANY nested repo refresh the panel/graph live. The badge event stays SOURCE-scoped (an
|
|
482
|
-
in-source over-refresh is fine, never a leak), and cache invalidation is already exact
|
|
483
|
-
(the TTL caches key by resolved root, so a nested-root write invalidates only that repo).
|
|
484
|
-
|
|
485
|
-
**`git.canWrite` — the write-UI visibility gate.** `canWrite` is computed SERVER-side
|
|
486
|
-
**per-caller** as the SAME two-plane gate the write routes enforce: the caller HAS the
|
|
487
|
-
`files.write` FEATURE **and** the caller's RESOLVED uri-policy `write` on the source root
|
|
488
|
-
`<slug>://` is allowed (`resolveGitCanWrite`, deny-by-default). The panel shows the
|
|
489
|
-
Stage/Unstage/Discard/Commit/branch-write affordances IFF `git.canWrite === true` — or, with
|
|
490
|
-
a focused NESTED repo, IFF that repo's `/git/repos` row carries `canWrite: true` (the
|
|
491
|
-
per-repo granularity above; the root row's flag is not consulted). This must NOT
|
|
492
|
-
be derived from `SourceInfo.permissions.write` (the source's DEFAULT permission block) — that
|
|
493
|
-
hid the owner's write UI on a `default write:false` source where the owner gains write via
|
|
494
|
-
`byRole`/`byUser`, and would also show buttons to a `files.read`-only role. The server route is
|
|
495
|
-
the deny-by-default floor regardless of this UI gate.
|
|
496
|
-
|
|
497
|
-
**The `git/*` routes** (`src/routes/git.ts`, file-level `feature='files.read'`):
|
|
498
|
-
- **GET (read, `files.read`):** `status?source=<slug>` (whole-repo change-set + current
|
|
499
|
-
branch, plus `upstream`/`ahead`/`behind` when the branch tracks an upstream — all
|
|
500
|
-
three absent otherwise), `branch?source=<slug>` (local branch list, for the switcher),
|
|
501
|
-
`diff?source=<slug>&path=<repoRelPath>&(staged=<0|1> | commit=<sha> | base=<sha>&head=<sha>)`
|
|
502
|
-
(the unified diff for ONE file — three MUTUALLY-EXCLUSIVE modes: worktree WT↔Index /
|
|
503
|
-
Index↔HEAD, commit↔FIRST-parent (a root commit diffs from the empty tree), or an explicit
|
|
504
|
-
sha pair (the outgoing per-file form — the commit mode composes to it internally); 400 on
|
|
505
|
-
any combination or half pair, every sha whitelisted before the engine),
|
|
506
|
-
`log?source=<slug>&maxCount=&skip=&ref=` (ONE page of the commit DAG with parent SHAs for
|
|
507
|
-
the SVG graph — `maxCount` is the page size, default 200, clamped to 1999 (one below the
|
|
508
|
-
2000 engine hard cap so the server's `maxCount+1` probe always fits); `skip` (≤ 100 000)
|
|
509
|
-
offsets the page from the tip — a `skip` PAST that window answers `hasMore:false` (the
|
|
510
|
-
clamp is the END of the pageable window, never an infinite repeat of the deepest page);
|
|
511
|
-
the response's `hasMore` is otherwise an EXACT next-page probe, not a
|
|
512
|
-
heuristic — it replaced `truncated` on this route only; plus `outgoing?: string[]`, the
|
|
513
|
-
not-yet-pushed `@{u}..HEAD` sha set, newest first, capped at 500, ABSENT — not `[]` —
|
|
514
|
-
when the branch has no upstream),
|
|
515
|
-
`commit?source=<slug>&sha=<sha>` (the files ONE commit changed as
|
|
516
|
-
`{path, status, renamedFrom?}` rows, FIRST-PARENT semantics; an unknown sha is a typed
|
|
517
|
-
`{ok:false, code:'unavailable'}`, never a fake empty list),
|
|
518
|
-
`outgoing?source=<slug>` (the aggregated committed-but-not-pushed file list +
|
|
519
|
-
the resolved `{baseSha, headSha}` pair; the server resolves BOTH refs ITSELF, the route
|
|
520
|
-
takes NO refs, and `baseSha` is `merge-base(@{u}, HEAD)` — never `@{u}`, which would
|
|
521
|
-
phantom remote-only files as deletions on a diverged branch; no upstream →
|
|
522
|
-
`{baseSha:null, headSha, files:[]}` fail-soft; LOCAL refs only, zero network — staleness
|
|
523
|
-
is "since the last fetch", and the network reach is the `fetch`/`pull`/`push` remote POSTs), and
|
|
524
|
-
`show?source=<slug>&path=<repoRelPath>&rev=<sha|index>` (the blob CONTENT of one file at a
|
|
525
|
-
commit sha or the index — the diff-expander data source; blob-ONLY with typed refusals,
|
|
526
|
-
below), and
|
|
527
|
-
`repos?source=<slug>` (the in-source repo-picker listing `{rel, name, branch?,
|
|
528
|
-
canWrite}`; names-only, source-root read gate + per-nested-name filter —
|
|
529
|
-
see the multi-repo model above).
|
|
530
|
-
- **POST (write, additionally `files.write`-gated):** `stage` (`{paths[]}` whole-file `add`
|
|
531
|
-
OR `{hunk}` `git apply --cached`), `unstage` (`{paths[]}` `reset -- <p>` OR `{hunk}`
|
|
532
|
-
reverse-apply), `discard` (`{paths[]}` restore WT from index; typed `rescue?` returned),
|
|
533
|
-
`commit` (`{message, amend?}`), `branch` (`{action:'create'|'delete', name, startPoint?,
|
|
534
|
-
force?}`; delete returns a typed `rescue?`), `checkout` (`{ref}` — PLAIN switch, no `-f`;
|
|
535
|
-
git refuses a clobber → 409), and `restore` (`{rescue:{kind,ref,name?}}` — one-click
|
|
536
|
-
UNDO: `git stash apply` / `git branch <name> <ref>`, re-snapshots first).
|
|
537
|
-
|
|
538
|
-
Every route resolves the source deny-by-default (non-accessible → 404, never a leak) and
|
|
539
|
-
returns **repo-relative paths only** (never host / `os_uri` paths); git op-error messages are
|
|
540
|
-
host-path-scrubbed. Responses are size-capped for MEMORY/response bounding, with a `truncated`
|
|
541
|
-
flag on `/git/status` + `/git/branch` + `/git/diff` (`/git/log` instead PAGES — `skip` +
|
|
542
|
-
`hasMore`, above) — a TWO-cap split: the human-panel `/git/diff` cap is 2 MB (`MAX_DIFF_BYTES_UI`, whole-line
|
|
543
|
-
truncation; the confidentiality boundary for these routes is the per-path read policy, not the
|
|
544
|
-
cap), while the `boundToolResult` 20 KB floor stays the MODEL-facing tool-result invariant for
|
|
545
|
-
the planned hosted git tools. A diff past the engine's ~4 MB output accumulation bound is KILLED
|
|
546
|
-
inside `GitRunner` (there is no serveable prefix), so `/git/diff` returns a typed
|
|
547
|
-
`{ok:false, code:'too_large'}` refusal on a 200 (the `/git/show` dialect) instead of an empty
|
|
548
|
-
diff that would render as a false "no changes" for a file the panel lists as Modified.
|
|
549
|
-
`diff` (every mode — historical content is content)/`show`/`stage`/`unstage`/`discard` gate
|
|
550
|
-
the uri-policy check on the SPECIFIC file (`<slug>://<path>`) so a per-path deny is
|
|
551
|
-
honored, where `status`/`branch`/`log`/GET `commit`/`outgoing`/POST `commit`/`checkout` gate
|
|
552
|
-
the checkout root (the GET `commit`/`outgoing` file lists are NAMES-only — the names-only dialect,
|
|
553
|
-
consistent with `/git/status`). With the multi-repo `repo` selector BOTH forms carry the `<repo>/`
|
|
554
|
-
policy prefix (root-gated → `<slug>://<repo>/`, per-file → `<slug>://<repo>/<path>` — see
|
|
555
|
-
the multi-repo model above). **Names-only residual (documented, not yet closed):**
|
|
556
|
-
a per-path read-DENIED file's NAME — including historical names — appears in the root-gated
|
|
557
|
-
listings (`status`, the `commit`/`outgoing` file lists, `log` decorations) while its CONTENT
|
|
558
|
-
403s on the per-path-gated `diff`/`show`; closing the names must close
|
|
559
|
-
status+commit+log-decorations together. **Diff direction:**
|
|
560
|
-
a *Changes* row is WT↔Index (`staged=0`), a *Staged Changes* row is Index↔HEAD (`staged=1`);
|
|
561
|
-
an UNTRACKED (brand-new) file has no index entry so its WT↔Index diff is empty — it is rendered
|
|
562
|
-
fully-added via `git diff --no-index -- /dev/null <path>` so the whole file shows as added.
|
|
563
|
-
Whole-repo status/log are `O(repo)` so they fire ON DEMAND only (panel open / repo select /
|
|
564
|
-
graph open), TTL-cached, for the SELECTED repo — never the `fs_list` hot path. The panel
|
|
565
|
-
readers split their inactivity timeouts two-tier: status/branches keep 5 s, while the HEAVY
|
|
566
|
-
readers (diff / log / show / commit files / outgoing) get 15 s so a 2 MB diff or a
|
|
567
|
-
2000-commit log on a slow volume isn't killed early (both tiers cap silence, not wall-clock —
|
|
568
|
-
see the engine section above).
|
|
569
|
-
`GIT_OPTIONAL_LOCKS=0` (set in `GitRunner.sanitizedGitEnv()`) makes read queries truly
|
|
570
|
-
index-read-only, closing the badge → re-list → `git status` → index-rewrite watcher feedback
|
|
571
|
-
loop (see the live-badge note below).
|
|
572
|
-
|
|
573
|
-
**`GET /git/show` — blob-only, refuse-don't-truncate.** The diff-expander data source
|
|
574
|
-
returns the CONTENT of one file at a rev. `rev` is whitelisted at the route BEFORE the
|
|
575
|
-
engine (the literal `index` or a 7–40 lowercase-hex commit sha; everything else is a 400 —
|
|
576
|
-
defense-in-depth ahead of the engine's own ref guard). It never truncates — refusals come
|
|
577
|
-
back typed (`ok:false`) on a 200, fail-soft, so the client degrades expanders instead of
|
|
578
|
-
erroring: `not_a_blob` (a directory target would print a tree LISTING — an embedded
|
|
579
|
-
name-listing side door around the per-path read policy), `too_large` (> 2 MB, size-checked
|
|
580
|
-
via `cat-file -s` BEFORE buffering the content), `binary` (NUL in the head), `unavailable`
|
|
581
|
-
(non-repo / path missing at the rev). The result is deliberately UNcached — a stale blob
|
|
582
|
-
would corrupt expanded diff context.
|
|
583
|
-
|
|
584
|
-
**Write model — human consent + SERVER unconditional auto-snapshot.** The write POST
|
|
585
|
-
routes are the HUMAN face of the git engine: the caller's authenticated session IS the consent
|
|
586
|
-
(`files.write` feature + uri-policy `write`), NOT the `PendingInteraction` agent-approval
|
|
587
|
-
guard (that is the additive AGENT face, a later increment; both faces drive the same `GitRunner`
|
|
588
|
-
mutating methods). Before ANY **destructive** op (`discard`, `branch` delete) the SERVER
|
|
589
|
-
**UNCONDITIONALLY** `snapshot()`s a reflog-recoverable rescue point (a `refs/nrs-rescue/<label>`
|
|
590
|
-
anchor + a `git stash create` of working-tree+index content) — **never gated on a client
|
|
591
|
-
`confirmed` flag** (the UI confirm dialog is UX-only; a client can POST `/git/discard`
|
|
592
|
-
directly). The op returns a TYPED, CONDITIONAL `rescue` carrier — `{kind:'stash', ref}` on a
|
|
593
|
-
dirty-tree discard, `{kind:'branch', ref, name}` on a resolved-tip branch delete (absent when
|
|
594
|
-
there is nothing to recover) — which drives the **one-click UNDO**: `POST /git/restore
|
|
595
|
-
{rescue}` re-snapshots first (a restore is itself a mutation) then `git stash apply` /
|
|
596
|
-
`git branch <name> <ref>` (non-force → 409 on an existing name), same `files.write` +
|
|
597
|
-
uri-policy `write` floor, no `PendingInteraction`. The panel's rescue banner grows an **Undo**
|
|
598
|
-
button beside Dismiss. Every successful write publishes the IN-APP `changeType:'badge'`
|
|
599
|
-
event with the actor's **REAL** `(userId, projectId, agentId)` via `publishInAppGitBadge` (the
|
|
600
|
-
ONLY in-app publish path — an empty `userId` would broaden delivery) so the panel + badges
|
|
601
|
-
update live through the ordinary user/agent-equality SSE filter.
|
|
602
|
-
|
|
603
|
-
**Side-by-side diff-on-click — full content + hidden-region expanders.** Clicking a
|
|
604
|
-
change-set row (or a file inside an expanded commit / the outgoing divider) opens a diff as
|
|
605
|
-
a **FileEditor view** (tab id `git-diff`, driven by the `gitDiffTarget` store descriptor —
|
|
606
|
-
a discriminated union mirroring the route's three modes: `{source, path, staged}` working
|
|
607
|
-
tree, `{source, path, commit}` commit↔first-parent, or `{source, path, base, head}` the
|
|
608
|
-
explicit outgoing pair — because a bare `activeWorkspaceTab` id can't carry the target).
|
|
609
|
-
`GitDiffView` is a THIN adapter: the fetch-and-render core is the prop-driven
|
|
610
|
-
**`GitFileDiff`** (`app/filesystem/ui/GitFileDiff.tsx`, mode vocabulary in the pure
|
|
611
|
-
`gitFileDiffMode.ts`), shared with the multi-file card view below so a diff fix lands in
|
|
612
|
-
both surfaces at once. It
|
|
613
|
-
fetches `GET /git/diff`, parses the unified diff with **`react-diff-view`** (MIT, no network
|
|
614
|
-
— it only parses the diff string), and renders a split (side-by-side) view of the WHOLE
|
|
615
|
-
change: the diff is size-capped by the SERVER alone (the 2 MB `MAX_DIFF_BYTES_UI` route cap
|
|
616
|
-
— the ONE cap owner; the client keeps no line/byte re-cap and only renders the server
|
|
617
|
-
`truncated` flag as a banner). Unchanged regions between hunks collapse into clickable
|
|
618
|
-
**"⋯ N hidden lines" expander rows** (per-gap, plus an "Expand all" header button); the
|
|
619
|
-
omit-gutter stripe is transparent — the expander rows ARE the affordance. Expansion lazily
|
|
620
|
-
fetches the NEW-side full source on first click (unchanged gaps are identical on both
|
|
621
|
-
sides), routed per mode: an unstaged diff reads the working-tree file via the read API, a
|
|
622
|
-
staged diff fetches the index blob via `GET /git/show?rev=index`, a commit diff fetches
|
|
623
|
-
`rev=<sha>` and a range diff `rev=<head>` (never `<sha>^` — the show route rejects it); the
|
|
624
|
-
result is cached per `(source, path, rev)`
|
|
625
|
-
and any refusal (`not_a_blob`/`too_large`/`binary`/missing file) degrades the expanders to
|
|
626
|
-
static "N hidden lines" text — fail-soft, never an error banner. Staging renders in the
|
|
627
|
-
WORKING variant ONLY — a commit/range (historical) diff never shows stage buttons; the gate
|
|
628
|
-
is the mode variant, never `staged` truthiness. **Hunk staging rides a
|
|
629
|
-
DUAL-MODEL** (`app/filesystem/ui/gitDiffExpand.ts`): the raw server diff is parsed ONCE into
|
|
630
|
-
ORIGINAL hunks — the sole staging source (the same raw text feeds `gitDiffPatch`) — while
|
|
631
|
-
expansion produces a separate RENDER-ONLY hunk list (`expandFromRawCode` MERGES and REWRITES
|
|
632
|
-
hunks, so rendered indexes stop mapping 1:1 onto the originals); stage/unstage buttons
|
|
633
|
-
anchor by ORIGINAL-hunk identity (old-range containment — a rendered hunk that swallowed N
|
|
634
|
-
originals renders N buttons), so the staged payload is identical before and after any
|
|
635
|
-
expansion. On a server-TRUNCATED diff both staging AND the expanders are disabled — a cut
|
|
636
|
-
final hunk would apply wrong, and gaps computed from the VISIBLE hunk headers would
|
|
637
|
-
fabricate "unchanged" rows over regions the dropped hunks own. The server's typed
|
|
638
|
-
`too_large` refusal (>4 MB engine-killed diff) renders as a dedicated too-large state, never
|
|
639
|
-
the "No … changes" empty state; the flag states reset on every target switch so a failed
|
|
640
|
-
fetch can't inherit the previous target's banner. `FileEditor`'s `showFilePane` guard
|
|
641
|
-
EXCLUDES the git views (`&& !showGitDiff && !showGitCommit`) — otherwise the file-content
|
|
642
|
-
layer renders ON TOP and blocks the pane (the same overlay trap the Pending view hit).
|
|
643
|
-
|
|
644
|
-
**Multi-file "Open Changes" tab (`GitCommitChangesView`).** The second git
|
|
645
|
-
FileEditor view (tab id `git-commit`, descriptor `gitCommitTarget` — `{kind:'commit', sha,
|
|
646
|
-
subject}` from a graph row's Open Changes action, or `{kind:'outgoing', base, head}` from
|
|
647
|
-
the outgoing divider's) renders one commit's — or the whole outgoing range's — change-set
|
|
648
|
-
as a vertical stack of collapsible
|
|
649
|
-
per-file diff cards (the PendingChangesPanel pattern): the file list comes from
|
|
650
|
-
`GET /git/commit` / `GET /git/outgoing`, and each card mounts the SAME `GitFileDiff` (card
|
|
651
|
-
variant) on expand only — conditional render, so a collapsed card fetches nothing.
|
|
652
|
-
Every change-set starts logically expanded; only visible expanded cards mount and
|
|
653
|
-
fetch through the shared content lanes (three at a time, answers cached per scroller), including histories above 200 files. A per-path read-DENIED file renders a fail-soft LOCK row inside its
|
|
654
|
-
own card (the per-file diff gate is per-file; the names-only list is root-gated — the names-only
|
|
655
|
-
dialect above), never a view-level error. Clearing the descriptor (pill close) falls the
|
|
656
|
-
active tab back to `file` — the store centralises the fallback so a `git-commit` tab id is
|
|
657
|
-
never stranded without a target — and BOTH git targets reset on an agent/project switch.
|
|
658
|
-
|
|
659
|
-
**Live `changeType:'badge'` SSE channel + `.git/HEAD` watcher.** The previously-dead
|
|
660
|
-
`changeType:'badge'` filesystem-event channel is now LIT so git badges + the panel +
|
|
661
|
-
the branch indicator update live. A per-checkout **`GitWatchRegistry`**
|
|
662
|
-
(`src/git/GitWatchRegistry.ts`) — reconciled over the DISCOVERED checkout set:
|
|
663
|
-
one watcher per root-repo source, one per nested repo of a multi-repo source (see the
|
|
664
|
-
multi-repo model above) — watches each checkout's `.git/` root for external git ops
|
|
665
|
-
(a terminal `git commit`/`add`/`checkout` the sync watcher can't see) and publishes a
|
|
666
|
-
**source-scoped** badge on the per-project event bus (git-state only — it NEVER feeds the
|
|
667
|
-
vector index / sync pipeline; the `.git/`-sync-ignore floor is unchanged). It acquires the
|
|
668
|
-
`.git` directory as a **SEPARATE `FsWatchHub` root** from the source root (a second
|
|
669
|
-
`ignored` predicate on a shared root is inherited, not applied — so the source watcher's
|
|
670
|
-
`.git`-ignore would otherwise blind it), reacts NARROWLY to `HEAD`/`packed-refs`/`refs/**`/
|
|
671
|
-
`index`, and prunes the high-churn `objects/`+`logs/` subtrees; it is reconciled + torn
|
|
672
|
-
down by `SyncWatchService`'s per-source lifecycle. On the client the
|
|
673
|
-
`changeType:'badge'` hook branch (`useFilesystemSSE.ts`) is an EARLY short-circuit (before
|
|
674
|
-
`incrementFolderBadges`/`upsertFileNodeFromEvent`) that debounces + re-lists the source
|
|
675
|
-
subtree (recomputing `FileNode.git`) and refetches the panel status + `SourceInfo.git`.
|
|
676
|
-
**Changes-section live stream.** The badge watcher above only fires on `.git`
|
|
677
|
-
ref/index moves — so **Staged** streams (staging writes `.git/index`) and the **graph**
|
|
678
|
-
streams (commits write `.git/HEAD`), but a plain WORKING-TREE edit (a new untracked file, an
|
|
679
|
-
unstaged modification) touches neither and emitted NO badge, leaving the **Changes** section
|
|
680
|
-
stale until a manual Refresh. Now `useFilesystemSSE` also refetches the FOCUSED repo's
|
|
681
|
-
`/git/status` from the two working-tree event branches (the `external:true` reconcile branch +
|
|
682
|
-
the per-file `created/updated/deleted/reverted` switch), focused-source-gated + per-source
|
|
683
|
-
trailing-debounced (`gitStatusSeq` drops out-of-order responses) — so Changes streams too.
|
|
684
|
-
**Badge delivery is an AUTHORIZATION decision, not a filter tweak** — the external watcher's
|
|
685
|
-
badge carries NO acting user, so `subscribeFilesystemEventsImpl` has a dedicated
|
|
686
|
-
`changeType:'badge'` branch that authorizes delivery by **SOURCE-ACCESS** (project gate +
|
|
687
|
-
`payload.source` ∈ the subscriber's accessible-source set, precomputed once at
|
|
688
|
-
subscription), deny-by-default, NEVER a blanket project broadcast — a user/agent-scoped
|
|
689
|
-
source's badge stays scoped to its owner. IN-APP badges (published by the write routes via
|
|
690
|
-
`publishInAppGitBadge` with the actor's REAL identity) skip that branch and reach the actor
|
|
691
|
-
through the ordinary user/agent-equality filter, unchanged.
|
|
69
|
+
| `data://` | `host` | builtin | Project data zone (`{neuralisHome}/projects/{pid}/data/`) |
|
|
70
|
+
| `data.X://` | `host` | user | Additional folder mount with a named slug |
|
|
71
|
+
| `packages://` | `host` | builtin | Project's `_packages/` directory |
|
|
72
|
+
| `app://` | `host` | builtin (privileged mount — `sources.privileged`) | Platform app zone (`~/.neuralis/app/`); writes from agents are denied |
|
|
73
|
+
| `neuralis://` | `host` | user (`pnpm neuralis:mount add … --neuralis`) | The deployment's OWN `neuralis/` folder, live from the host. Surfaces as a `host` source — there is **no** dedicated `neuralis` connector kind. The switch is `--neuralis`, not `--slug neuralis` (that is the ordinary data mount at `/mounts/neuralis`, without the two anonymous volumes that keep `_runtime/` and `node_modules/` from being shadowed). |
|
|
74
|
+
| `neuralis-skills://` | `host` | user (same overlay, rooted at `<root>/skills`) | The narrower default: only the shipped operator skills. Preferred over `neuralis://` for giving an agent the operator-command vocabulary — the overlay root holds `.env`, and a source root is a READ boundary in a way a sync `include` is not. |
|
|
75
|
+
| `<source-slug>://` (e.g. `machine-ops://`) | `webtop` | machine-core | Webtop sandbox; declared by `machine-core`, not brain-core. The scheme is the source's slug — a user-scoped machine source lists as `machine-ops://`, never `machine://` or `machine-user://` (the connector's `urlScheme` `machine` is only the slug base). The registry KIND is `webtop`, and only the kind addresses `GET /sources/kinds/:kind/schema`. |
|
|
76
|
+
| `os://<key>` | — | **decorative only** | Renders inside the system prompt's `<runtime_stack>` block as a label for `NEURALIS_MOUNT_<KEY>` mounts. **No connector handles `os://`.** Agents addressing files inside a mount use the `host` source slug attached to that mount. |
|
|
77
|
+
|
|
78
|
+
## Git
|
|
79
|
+
|
|
80
|
+
A `host` source that is, or contains, a git checkout gets a governed source-control surface:
|
|
81
|
+
per-file tree badges, the Git Control panel and the `git/*` routes (the `git` skill drives the
|
|
82
|
+
same routes). The package also declares a `type:'mcp'` `git` connector (in-process, trust
|
|
83
|
+
`>= trusted`); it is metadata only and hosts no tools yet.
|
|
84
|
+
|
|
85
|
+
| Code | Role |
|
|
86
|
+
|---|---|
|
|
87
|
+
| `src/git/GitRunner.ts` | every git op as ONE argv on either `host` transport, the containment floors, remote ops (`#spawnGitRemote`); helpers `classifyGitOp`, `parseStatusPorcelainV2` (the ONE status reader), `parseBranchList`; `GitTimeoutError` / `GitOutputLimitError` |
|
|
88
|
+
| `src/git/tokenResolve.ts` | `resolveGitToken`: the caller's own push token, per request, never cached |
|
|
89
|
+
| `src/git/sourceGitStatus.ts` | the per-file badges: pathspec-scoped, timed, cached; `rollupFolderGit` |
|
|
90
|
+
| `src/git/sourceGitControl.ts` · `gitPlace.ts` | multi-repo discovery (`discoverSourceRepoRoots`) · where a checkout lives and the spawn-free reads |
|
|
91
|
+
| `src/git/GitWatchRegistry.ts` | the per-checkout `.git` watcher behind the source-scoped badge event |
|
|
92
|
+
| `src/git/untrackedStats.ts` | line counts for untracked files without a git spawn per file |
|
|
93
|
+
| `src/routes/git.ts` | the `git/*` routes, caps and gates → `src/routes/README.md` |
|
|
94
|
+
| `app/filesystem/ui/GitControlPanel.tsx`, `StackedSections.tsx` (`growScaleForSum`, `MIN_USABLE_WEIGHT`), `GitCommitGraph.tsx`, `GitFileDiff.tsx` + `gitDiffExpand.ts` | the panel → `app/filesystem/ui/README.md` |
|
|
95
|
+
|
|
96
|
+
The engine and route floors (argv-only spawn, repo-local config neutralised, the constant
|
|
97
|
+
credential helper, HTTPS-only, no clone and no force-push) and the panel traps are maintained
|
|
98
|
+
in the `neuralis-brain-core` skill; the git-plane security floors in `neuralis-security-invariants`.
|
|
692
99
|
|
|
693
100
|
## Public Surface
|
|
694
101
|
|
|
695
102
|
Lifecycle exports (`src/lifecycle.ts`):
|
|
696
103
|
|
|
697
|
-
- `init`, `start`, `onBootstrapComplete`, `drain`, `stop`, `api`, `onCredentialChanged`, `health`, `onPrincipalRevoked`, `provisionProject` (`usecases/provisionProject.ts` — a NEW project's declared `seed:'auto'` sources, written complete through the same create path as the boot repair; see
|
|
104
|
+
- `init`, `start`, `onBootstrapComplete`, `drain`, `stop`, `api`, `onCredentialChanged`, `health`, `onPrincipalRevoked`, `provisionProject` (`usecases/provisionProject.ts` — a NEW project's declared `seed:'auto'` sources, written complete through the same create path as the boot repair; see `src/services/README.md`), `deprovisionProject` (`usecases/deprovisionProject.ts` — on a PERMANENT project delete, ALL of the project's vector points leave the shared index in ONE bounded `deleteByFilter({ must: [{ field: 'projectId', equals: id }] })` — chunks + folder-ancestor points covered by the same keyword-index filter. MUST-HAVE: a failed delete throws — an unavailable vector store included (the typed `vector_store_unavailable` refusal) — and the host then stops the purge before any file is removed. It runs BEFORE the fs tree is removed, so resolving the brain infra cannot re-materialize the project dir).
|
|
698
105
|
- `init` state exposes `services["package-source-roots"]` for kernel registration; its `listPackageSourceRoots` delegate is the same function as the direct API method, and its `subscribePackageSourceRoots(listener)` calls `listener(projectId)` after every committed source-config write of the app root (`SourceConfigMutation.onChanged` — fired outside the write lock, a no-op write never fires, a throwing listener is logged and never reaches the writer), so the host re-derives that project's flagged package roots at once.
|
|
699
|
-
- `BrainCoreApi` — `getProjectServices`, `getProjectInfra`, `getFilesystem`, `upsertSourceConfig`, `listShellSourceRoots(projectId)` (every shell-capable source with its persisted permissions; feeds agent-core's shell uri-policy gate and the terminal's host-tab resolver; NOT access-filtered — consumers evaluate the policy against the caller's own `SessionContext`. Rows carry an explicit **`plane`**: `container` sources contribute their `containerRoot`, resolved by the canonical `resolveLocalRoot`; **
|
|
106
|
+
- `BrainCoreApi` — `getProjectServices`, `getProjectInfra`, `getFilesystem`, `upsertSourceConfig`, `listShellSourceRoots(projectId)` (every shell-capable source with its persisted permissions; feeds agent-core's shell uri-policy gate and the terminal's host-tab resolver; NOT access-filtered — consumers evaluate the policy against the caller's own `SessionContext`. Rows carry an explicit **`plane`**: `container` sources contribute their `containerRoot`, resolved by the canonical `resolveLocalRoot`; **device-plane** sources contribute a raw `hostRoot` and an intentionally EMPTY `containerRoot` — no container path exists for them, and the empty value is load-bearing, because the consumer's local-plane filter drops rootless entries so a folder on a device can never leak into a local-plane lookup even if `plane` were ignored. The plane is the kernel's `sourcePlane` over the STORED connection (a `host` folder naming a `device` is the device plane), never the slug string — an ordinary folder may legitimately be named `host-notes`. Kinds that host no shell at all (brain vectors, webtop) contribute nothing), and `listSourceRuntimeSummaries(projectId, { userId, agentId, role?, grantedFeatures? })` (the READ sibling of `listShellSourceRoots`, feeding agent-core's `<runtime_stack>` source rendering — **all** connector kinds, **access-filtered** (device-plane rows carry `containerDir: null` — their root is a path in the OPERATOR's filesystem with no container equivalent, and the local-shaped derivation would otherwise print a container path that does not exist, baked into every agent's cached system prompt) via `canAccessScope` (with the source's plane: a user-scope device folder lists only for its owner) → `files.readOthers`/deny-by-default, each row carrying connector, `os_uri`/`container_dir`, the caller-effective uri-policy, `description`, and the cached index snapshot; reads persisted source-config JSON ONLY — no live walk / no vector count — so it is safe to call once per stream. Agent-core renders the stable structure in the cached `<runtime_stack>` prefix and the volatile counts + on/off in `<workspace_live>`. Disabled sources are INCLUDED, marked `enabledEffective:false`). `listPackageSourceRoots(projectId)` (enumerate every project-associated source as a runtime package-source candidate `{ slug, uriRoot, containerRoot, scope, plane, packageFolders, enabled, kind }` — `plane` is `sourcePlane` of the stored connection, the host's only source of it; identity-free like `listShellSourceRoots`, ALL candidates returned so the host applies the floor — package folders + enabled + a root this process opens; `containerRoot` of a device folder is agent-core's `devices.localFolderRoot` answer, asked only on a native host for a folder with package folders — the app's own loopback device inside the owner's ceiling — and `''` for any other device; the host's `projectPackages` consumes it to scan + hot-reload each package folder alongside `_packages/`). Filesystem events reach the `/brain-core/events` SSE route and the discovered `filesystem` channel of the host `/api/events` hub through ONE authorization adapter: it takes the real session, authorizes EACH requested agent deny-by-default (a foreign agent yields zero events, never an existence-confirming error), and filters events by user/project/types and the authorized agent or source.
|
|
700
107
|
|
|
701
108
|
**Cross-package writes go through `FilesystemService` itself, and its permission check is
|
|
702
109
|
the ONLY gate on that path.** `getProjectServices(projectId).filesystem` hands back the live
|
|
@@ -723,246 +130,6 @@ on `AgentCoreApi`. A member agent-core renames is now a brain-core compile error
|
|
|
723
130
|
silent `undefined`. Type-only use means no brain-core declaration surfaces an agent-core
|
|
724
131
|
type — `dist/**/*.d.ts` carries no `@neuralis/agent-core` reference.
|
|
725
132
|
|
|
726
|
-
### Source-package discovery (`listSourcePackages`)
|
|
727
|
-
|
|
728
|
-
`FilesystemService.listSourcePackages(ctx)` (reached via `getProjectServices(projectId).filesystem`)
|
|
729
|
-
lists **source-contributed packages** from the vector index — the discovery half of the
|
|
730
|
-
source-package pipeline (agent-core consumes it per stream). The service is a thin
|
|
731
|
-
delegate: the pass itself is `src/usecases/listSourcePackages.ts`, and the raw inventory it
|
|
732
|
-
shares is `src/usecases/sourcePackageInventory.ts`. Four invariants:
|
|
733
|
-
|
|
734
|
-
1. **No connector I/O and no OS walk.** The pass is index-only: pass 1 scrolls the bounded
|
|
735
|
-
*structural trigger categories* (`skill`, `instruction`, `rule`, `agent`, `package`)
|
|
736
|
-
project-wide; pass 2 fetches `docs` for every detected package root and the skill-bundle
|
|
737
|
-
siblings of every detected skill through the keyword-indexed `folder_ancestors` field.
|
|
738
|
-
Content hydration is the caller's job (per-stream, after gates).
|
|
739
|
-
|
|
740
|
-
Pass 2 is **batched, not per root**: one paged `folder_ancestors matchAny(active roots)`
|
|
741
|
-
query answers every root at once, and a root whose raw cap is full leaves the active set
|
|
742
|
-
so later pages stop carrying it. Three properties of that shape are load-bearing — the
|
|
743
|
-
raw cap stays **per root** and **before** the privacy/policy filters, exactly where the
|
|
744
|
-
per-root query used to stop paging; a point under two NESTED roots is offered to BOTH
|
|
745
|
-
(the nearest-root rule then decides); and the cursor is a point ID scanned in ID order,
|
|
746
|
-
which is the only reason narrowing the filter between pages is safe.
|
|
747
|
-
|
|
748
|
-
Every pass **projects** its payloads (`uri`, `category`, `agent_owner`,
|
|
749
|
-
`manifest_usable`, `frontmatter`, `os_uri`, `content_hash`, plus `folder_ancestors` in
|
|
750
|
-
pass 2) — never `content`. A manifest's display name comes from its own bounded batch,
|
|
751
|
-
for the manifests the caller may actually read. Measured on 70 skills across 8 roots:
|
|
752
|
-
79 backend round-trips → 5, and 1.44 MB of payload → 88 KB.
|
|
753
|
-
2. **Category-driven grouping.** The sync indexer assigns the trigger categories (the ONE
|
|
754
|
-
shared `categorizeContributionPath` table in `@neuralis/package-system/source`, kept only
|
|
755
|
-
for files the ONE shared `admitsContribution` rule admits — a filename-identity form, or
|
|
756
|
-
frontmatter `id`/`name`; an undeclared `agents/references/*.md` is a plain `file`); the
|
|
757
|
-
query groups the trigger nodes with the ONE shared `detectPackageRoots` rules module
|
|
758
|
-
(usable-manifest anchor → marker dot-dir (`.claude`, `.cursor`, …) → category-dirs root
|
|
759
|
-
→ bare-skill dir → RESTRICTED implicit source-root). Grouping is query-time and
|
|
760
|
-
in-memory — layout changes self-heal on the next call, no payload migration.
|
|
761
|
-
3. **All caller scoping is post-filter.** The vector filter is `projectId`-only — index
|
|
762
|
-
points carry the SYNCING identity, a `userId equals` filter would under-report. The
|
|
763
|
-
caller is scoped by: `#listAccessibleSources` (enabled + scope + source-level read),
|
|
764
|
-
the source's persisted **uri-policy evaluated per package root AND per file**
|
|
765
|
-
(denied ⇒ absent — member non-leak extends to file and bundle-member names), and the
|
|
766
|
-
`agent_owner` privacy marker (`data://agent-core/<agentId>/…` subtrees belong to that
|
|
767
|
-
agent only). A source without a persisted config is skipped (fail closed). An absent
|
|
768
|
-
`ctx.role` returns `[]` (weakest interpretation — never a `'member'` default).
|
|
769
|
-
4. **The shared inventory holds RAW index points, never a caller's answer.** Root detection
|
|
770
|
-
runs AFTER the `agent_owner` filter and the accessible-source set, so the roots, the raw
|
|
771
|
-
caps and the package tree a privileged caller produces are not the ones a member
|
|
772
|
-
produces — freezing any of that into a shared entry would hand one caller's view to
|
|
773
|
-
another. Every pass therefore re-reads the persisted source config and re-evaluates
|
|
774
|
-
source enable, `enabled:'auto'`, scope, role/priority and every uri-policy row; a
|
|
775
|
-
revocation bites on the NEXT request, including one issued while a scan is in flight.
|
|
776
|
-
Discovery deliberately **bypasses** the `brainAccessibleCacheTtlMs` window that the hot
|
|
777
|
-
list/read paths use, in both directions — it neither reads that cache nor seeds it.
|
|
778
|
-
|
|
779
|
-
CONTENT freshness is a generation fence instead. `vector/discoveryInvalidation.ts` holds
|
|
780
|
-
a per-project counter that the ONE write boundary raises at the START and the END of any
|
|
781
|
-
write that could move a package candidate — success, throw or timeout alike, because a
|
|
782
|
-
failed write is not evidence that nothing landed. A `deleteByFilter` is relevant unless
|
|
783
|
-
its filter pins `category === 'chunk'`, and its return value is never consulted: a
|
|
784
|
-
returned `0` does not mean nothing was deleted. Chunk writes are irrelevant by rule, so
|
|
785
|
-
the high-rate path costs nothing; a non-chunk write outside every enumerated dir is
|
|
786
|
-
irrelevant only while a registration is on record, because a project whose inventory is
|
|
787
|
-
being built right now has none — with nothing to judge a write against, it counts. What
|
|
788
|
-
the registry watches is derived from the PUBLISHED inventory, never from the roots of the
|
|
789
|
-
caller whose pass happened to build it: an entry is the union of every caller's keys, so
|
|
790
|
-
a registration taken from the narrower of two callers would leave the wider entry's
|
|
791
|
-
bundle folders unwatched. A pass that
|
|
792
|
-
straddles a write does not publish; it still serves its own caller, marks the project
|
|
793
|
-
dirty and schedules at most ONE bounded rebuild — never a retry loop. Two package-declared
|
|
794
|
-
bounds tune it: `brainDiscoveryInventoryTtlMs` (the recovery window for writes made to
|
|
795
|
-
the same vector store by something else — anchored to the BUILD, never slid forward by a
|
|
796
|
-
read) and `brainDiscoveryInventoryMaxProjects`.
|
|
797
|
-
|
|
798
|
-
Sync-time enrichment (`DeltaSyncRunner`): ADMITTED trigger `.md` nodes get a whitelisted
|
|
799
|
-
`frontmatter` payload (`id`/`name`/`title`/`description`/`required_features`/`credentials`/
|
|
800
|
-
`metadata`/`allowed_tools`); `package`-category nodes get `manifest_usable: true` when their content
|
|
801
|
-
proves a neuralis block / known plugin manifest (content is already in hand — no extra
|
|
802
|
-
read) — that flag comes from the kernel's `isUsableManifestContent`, a deliberately
|
|
803
|
-
permissive GROUPING heuristic and **not** the fail-closed `validateManifest` gate; the
|
|
804
|
-
asymmetry, and why narrowing it would be a capability cut, is documented once in
|
|
805
|
-
`packages/package-system/README.md` (§ Source-contributed packages). `agent-core/<id>/…`
|
|
806
|
-
nodes get `agent_owner`. Unchanged (hash-matched) points
|
|
807
|
-
receive a **payload-only refresh** (existing vector reused, no re-embed) so one sync pass
|
|
808
|
-
after a deploy migrates a source fully — and because the changed-set loop only reaches
|
|
809
|
-
files the delta scan reports, the run ends with an **enrichment refresh sweep** over the
|
|
810
|
-
remaining indexed file points: unchanged pre-deploy points AND locally authored `active`
|
|
811
|
-
points (an agent's own `fs_write`) pick up their category/frontmatter/`agent_owner` on the
|
|
812
|
-
NEXT regular sync, no full reindex needed. The sweep is **stamp-gated**, not a walk of the
|
|
813
|
-
whole index: `enrichment_revision` (`src/sync/categoryInference.ts`) marks a payload whose
|
|
814
|
-
derived fields are what the current rules give for its own uri + content, a stamped point
|
|
815
|
-
is skipped before anything reads its content back, and only the unstamped candidates are
|
|
816
|
-
hydrated — payload-only upserts, existing vector reused, no re-embed. On a settled source
|
|
817
|
-
the sweep WRITES nothing. A point holding no inline text (a media point) never becomes
|
|
818
|
-
stamped, so it stays a candidate and is re-hydrated each full pass, but it is never
|
|
819
|
-
re-written unless a URI-derived field drifted. The
|
|
820
|
-
stamp is written by ONE boundary,
|
|
821
|
-
`src/vector/FilesystemVectorIndex.ts`, on every index write from every path; a payload it
|
|
822
|
-
cannot derive from (a chunk, a tombstone, a `pending_review` draft, a node holding no
|
|
823
|
-
inline text) has its stamp REMOVED instead, so a writer that spread an older payload
|
|
824
|
-
cannot carry a "done" marker it did not earn. Category resolution on every index write is the ONE
|
|
825
|
-
`resolveSyncCategory` rule (`src/sync/categoryInference.ts`): a caller's free-form tag
|
|
826
|
-
(`note`, `config`, …) and the runtime-data values survive; a trigger category
|
|
827
|
-
(`skill`/`agent`/…) and the generic `file` are always re-derived from path + content — so a
|
|
828
|
-
pre-deploy `agent` label on undeclared prose heals through the refresh sweep, and a caller
|
|
829
|
-
cannot label a file into a contribution category through `fs_write`/upload/
|
|
830
|
-
approve (the write paths call the same helper; the write answer echoes the stored `category`). A
|
|
831
|
-
point whose payload holds no content keeps its stored category on the sweep; a changed file
|
|
832
|
-
whose embed FAILS (exhausted credits, a provider 429) still gets its existing point's derived
|
|
833
|
-
metadata refreshed from the content that point already carries, payload-only, before the
|
|
834
|
-
failure is reported. Enrichment never propagates to chunk points. Port contract: a
|
|
835
|
-
`VectorIndexPort.upsert` WITHOUT `vector` is a payload-only write that PRESERVES the
|
|
836
|
-
stored vector — both impls (Qdrant, InMemory) honor it.
|
|
837
|
-
Caps: pass-1 scroll 2000 points, 64 files per category per package, 64-entry skill
|
|
838
|
-
bundles — truncation is logged on the sync channel.
|
|
839
|
-
|
|
840
|
-
Sync reconcile is **documents-only**: `DeltaSyncRunner.#listIndexedPoints` (the index
|
|
841
|
-
snapshot feeding `existingByUri` + the cleanup/delete loop) excludes chunks server-side
|
|
842
|
-
(`mustNot:[category='chunk']`, mirroring `#pointsByUris`). Chunks share the parent's `uri`
|
|
843
|
-
and carry a hardcoded `status:'active'`, so before this they (a) could shadow the real
|
|
844
|
-
document point in the uri-keyed `existingByUri` map and (b) each hit the tombstone when a
|
|
845
|
-
file left disk — inflating `filesRemoved` by the chunk multiplier and leaving orphaned
|
|
846
|
-
`status:'deleted'` tombstone chunks. Cascade deletion is unaffected: a parent's tombstone
|
|
847
|
-
(`usecases/syncPointWrites.ts`) removes its chunks via `deleteChunksForParent` (a Qdrant
|
|
848
|
-
`parent_id` query, independent of the snapshot array).
|
|
849
|
-
|
|
850
|
-
**A sync write costs what changed.** The parent vector is keyed on its embedding TEXT
|
|
851
|
-
(`filename` + the first `brainSyncMaxIndexTextChars` of content): when the stored point's own
|
|
852
|
-
inline text gives the same string — an append past that cap, a move that keeps the filename —
|
|
853
|
-
the parent write is payload-only. A renamed file is paired with the point that left disk BEFORE
|
|
854
|
-
anything is embedded (a 1:1 content-hash match, `sync/moveDetector.ts`) and that point moves: it
|
|
855
|
-
keeps its id at the new uri, so its chunks keep their ids and vectors and only their location is
|
|
856
|
-
rewritten, payload-only; the parent is re-embedded once because the filename is in its text. A
|
|
857
|
-
rename split across two watcher batches still costs a delete plus a create. A tombstone is
|
|
858
|
-
payload-only (the vector is neither read back nor re-uploaded), and a parent with no chunks
|
|
859
|
-
issues no chunk delete. (Test-double note: `InMemoryVectorIndex.scroll` derives
|
|
860
|
-
`nextCursor` from the last **scanned** index, not the output count — a `mustNot` that
|
|
861
|
-
excludes leading keys must not re-yield a matching point across pages, the way Qdrant
|
|
862
|
-
returns each point once.)
|
|
863
|
-
|
|
864
|
-
`upsertSourceConfig(scope, patch)` is the only sanctioned cross-package source-config write path. It re-reads the existing config, merges `permissions` and `sync` from the patch, runs `validateSourceConfig`, and persists through `AppSourceConfigStore`. Admin and other host callers must NOT touch the source JSON files directly.
|
|
865
|
-
|
|
866
|
-
`SourceConfig.description?` is a human one-line summary of the source's purpose (settable via the sources create/PATCH route, the `manage-sources` skill, the Files-UI **Sources** panel — a collapsible *Description* section per source plus an add-source field — and the admin **Config → Source Configs** editor); it is loaded into the agent `<runtime_stack>` source table. Leaving it EMPTY is a supported state, not a gap: the read-time chain below resolves the package's own text instead, and every editing surface shows that resolved value labelled as coming from the package. An editor must therefore send `description` only when it CHANGED — writing the field unconditionally on save turned a resolved description into a persisted empty string, and the declared-source backfill is version-stamped, so it never came back. `SyncConfig.stats?` (`SourceIndexStats` — `{ indexedCount, unindexedCount, filesCount, computedAt, truncated?, filesScanCap? }`) is the **absolute** post-sync index snapshot, distinct from the delta-of-last-run `lastSyncResult`. It is written ONLY by the `SyncJobQueue` completion handler on a successful run (the totals are derived inside `DeltaSyncRunner` from data already scanned — no extra walk), and read cheaply by `listSourceRuntimeSummaries` so the runtime-stack counts never trigger a live walk. A permissions/sync edit preserves an existing `stats` snapshot.
|
|
867
|
-
|
|
868
|
-
**Scan cap (`maxScanFiles` / `brainMaxScanFiles`).** Both the sync walk (`DeltaSyncRunner` → `scanDelta`) and the display count (`fs_list` → `connector.count`) stop at an effective per-source cap resolved by `resolveScanCap`: per-source `SyncConfig.maxScanFiles` (Files-UI Sources panel → *Max scan files*) ?? platform `brainMaxScanFiles` setting ?? `DEFAULT_MAX_SCAN_FILES` (200 000). The connector applies its own `DEFAULT_MAX_SCAN_FILES` as a safety bound when no explicit cap is passed (`walk()`/`peekCount` cache-miss paths). The display count is pushed the source's exclude set too, so `on_disk` prunes `node_modules`/`.git` (matching the sync walk) and `unindexed = on_disk − indexed` stays honest. When a walk stops at the cap the result carries `truncated: true` (with `filesScanCap`), and every surface renders the count as **`N+`** (Sources panel "N+ connector", runtime-stack `files=N+`) rather than presenting the cap value as the true total — there is no extra walk to compute the real number.
|
|
869
|
-
|
|
870
|
-
Tools (`tools/*.json` schemas paired with `src/tools/*.ts` handlers):
|
|
871
|
-
|
|
872
|
-
| Tool | Risk | Operation |
|
|
873
|
-
|---|---|---|
|
|
874
|
-
| `fs_read` | read | `filesystem.read` — batch read by URI/UUID. Each entry is a bare reference or `{uri, line_start?, line_end?}` with its OWN window (it wins over the batch window); baseline selection unchanged. Output is line-numbered under a `lines X-Y / total` header, and the snippet cap on a whole-file read states the total size plus the next window instead of a bare `…(truncated)`. |
|
|
875
|
-
| `fs_list` | read | `filesystem.list` — bounded recursive walk merging Qdrant index with live connector entries. Enforces the per-uri `read` policy on the enumerated NAMES (not just the source scope), the same axis `fs_search` applies — a `read:false` path's name is dropped from the tree, not just content-403'd. `sources_only` returns the source registry with no walk at all, and `limit` floors at 1. |
|
|
876
|
-
| `fs_search` | read | `filesystem.search` — `semantic`, `filter`, or `grep` modes. `filter` (the indexed keyword mode) paginates the scroll and ranks rows by term frequency (score normalized 0..1); `glob` narrows filenames in every mode, including grep (include-glob mapping), and grep takes `path_glob` (a real include glob that wins over `glob`), `case_sensitive` and `context`. **Every mode answers with `truncated` + `total_seen`** — a set cut at its limit says so, because a partial list that looks complete is what produces a false "it is not there". Enforces the per-uri `read` policy on every result across all three modes — a `read:false` path is never search-visible — and `grep` is the uri-policy-gated path to literal search without `execute`/shell. |
|
|
877
|
-
| `fs_write` | write | `filesystem.write` — THE write tool. Verb KEYS, not an action enum: the content verbs (`create`, `overwrite`, `edit[]`, `insert`, `lines`, `fields`) compose in ONE call, applied create/overwrite → edit → insert → lines → fields, into one write with one revert baseline; the structural verbs (`move`, `copy`, `mkdir`, `revert`, `delete`) each stand alone. `create` never overwrites — an occupied name lands beside it and the answer gives the uri used. The asking posture is DECLARED per key (`x-neuralis.askOnKeys: ["delete"]`), so `balanced` asks before a delete and stays quiet for an edit. |
|
|
878
|
-
|
|
879
|
-
The package ships FOUR agent tools. `fs_create` and `fs_modify` were the older write
|
|
880
|
-
pair; `fs_write` replaced every one of their functions and both are DELETED. Their names
|
|
881
|
-
survive in two places on purpose — stored transcripts still hold 1 800+ of their result
|
|
882
|
-
rows, so the `filesystem-change` card's manifest `match.tools` and agent-core's
|
|
883
|
-
`REVERTIBLE_TOOLS` keep naming them so that history renders and rewinds.
|
|
884
|
-
|
|
885
|
-
Each tool declares its feature gate on itself (`x-neuralis.requires.features`): `fs_read`/`fs_list` → `files.read`, `fs_write` → `files.write`,
|
|
886
|
-
`fs_search` → `files.search`. This is the ONE source for the tool gate — the
|
|
887
|
-
former manifest `policies[] kind:'access'` block was removed; only the `fs_*`
|
|
888
|
-
timeout `safety` policy remains. The HTTP routes below carry the SAME features
|
|
889
|
-
independently (route-layer `requireFeature`), so both the agent-tool path and
|
|
890
|
-
the direct-HTTP path are gated.
|
|
891
|
-
|
|
892
|
-
Routes (`src/routes/*.ts`, exported `pattern` + `feature`):
|
|
893
|
-
|
|
894
|
-
- `read` (POST, `files.read`), `list` (POST, `files.read`), `search` (POST, `files.search` — body key `search_mode`, the same vocabulary as the `fs_search` tool; an unknown value or the legacy `mode` key is a hard 400, never a silent semantic fallthrough).
|
|
895
|
-
- `write` (POST, `files.write` — the ONE write route, every `fs_write` verb key; a copy-only call is guarded as `read` on the source), `approve` (POST, `files.write`), `upload` (POST, `files.write`).
|
|
896
|
-
- `raw` (GET, `files.read`), `events` (GET SSE, `files.read` — a single-agent caller of the `subscribeFilesystemEventsImpl` authorization adapter, which re-authorizes the agentId deny-by-default), `health/*` (GET/POST, `files.read` — but the `gc` sub-path is **POST-only** and escalates in-handler to **`files.write`**, and the GET `hot-uris` sub-path (the write monitor) takes the same `files.write` tier and then filters per uri, both declared via `export const escalations`; see the Vector health section).
|
|
897
|
-
|
|
898
|
-
**The `raw` byte lane never asks a browser to RENDER what a member wrote.** Response headers come from one owner, `buildRawResponseHeaders` (`domain/mime.ts`), and the route builds none of its own. `files.write` is a member default on the seeded `data` source, so anything the lane renders on the app origin runs with the READER's session — an SVG carrying `<script>` was a stored-XSS primitive while the route echoed the file's own mime with `content-disposition: inline`. Three classes, and the split preserves every capability: safe previews (image except SVG, video, audio, PDF, text, JSON) stay `inline`; the ACTIVE-DOCUMENT class (SVG, HTML, XHTML, XML and every `+xml` subtype) is served with its TRUE type and `content-disposition: attachment`, so the file downloads normally and simply never becomes a live document; everything else downloads too, **keeping its real type** — the protection comes from the disposition, so a `.docx` or a `.wasm` is not flattened to `application/octet-stream` (only an absent or unparseable type is). A non-inline response always carries the header even without a filename, and filenames ride as a sanitized ASCII form plus the RFC 5987 `filename*` form so non-ASCII names survive. The store side is closed with it: `upload` no longer persists the caller-supplied part `Content-Type` verbatim (`normalizeStoredMimeType` — extension first, declared value only for unknown extensions, magic-byte check on the raster formats). Defense in depth, not a substitute for either: the host adds `Content-Security-Policy: sandbox` to every `/api/packages/*` response, which forces any navigated document on the lane into an opaque origin.
|
|
899
|
-
- `git/*` (`src/routes/git.ts` — the Git Control panel surface): **GET** (`files.read`) `status` (whole-repo change-set + branch + `upstream`/`ahead`/`behind` when tracking exists) + `branch` (local branch list) + `diff` (per-file unified diff — `?path=&staged=` WT↔Index vs Index↔HEAD, `?commit=<sha>` commit↔first-parent, or `?base=&head=` explicit sha pair; modes mutually exclusive, shas whitelisted, untracked rendered fully-added via `--no-index`, per-path uri-policy gate in every mode) + `log` (`?maxCount=&skip=&ref=` — ONE page of the commit DAG for the graph; page ≤ 1999, `skip` ≤ 100 000 (past the window ⇒ `hasMore:false`), exact `hasMore` probe replaced `truncated` here; each entry carries `subject` + the full `body` (both server-capped) for the hover card; plus `outgoing?: string[]` not-pushed shas, absent without an upstream) + `commit` (`?sha=` — the commit's file list, first-parent two-tree form, root commit = all-adds vs the empty tree; names-only ⇒ root-gated) + `outgoing` (server-resolved merge-base(`@{u}`,`HEAD`)↔`HEAD` aggregate file list + sha pair; NO caller refs; local refs only, fail-soft without an upstream) + `show` (`?path=&rev=<sha|index>` — blob content at a rev for the diff expanders; per-path uri-policy gate, blob-only + 2 MB/binary guards, typed `ok:false` refusals instead of truncation) + `log-stats` (per-commit `+/-` totals for one log page, totals only, never a path) + `repos` (the in-source repo picker — and the only route that tells a caller whether a git plane exists at all: `rel:""` is the source root, any other `rel` is a nested checkout a root-gated read will not see). **The four root-gated summary reads — `status`/`branch`/`log`/`log-stats` — fail SOFT to an empty `ok:true` on a non-repo, a containment refusal or any git error, which is byte-identical to a clean repository; `commit`/`outgoing`/`show` answer the typed `{ok:false,'unavailable'}` dialect instead.** **POST** (additionally `files.write` + uri-policy `write`) `stage`/`unstage` (whole-file OR `{hunk}`) + `discard` + `commit` + `branch` (create/delete) + `checkout` — HUMAN consent (the session, NOT the agent-approval guard), SERVER unconditional auto-snapshot before a destructive op (returns `rescue`), and an in-app badge with the actor's REAL identity on success. All per-source (`?source=` GET / body `source` POST), deny-by-default (non-accessible → 404), repo-relative paths only + host-path-scrubbed errors, size-capped (diff 2 MB — the human-panel memory bound; the 20 KB `boundToolResult` floor stays the model-facing invariant for the planned hosted git tools).
|
|
900
|
-
- `policies/uri` (GET read = `files.read`; POST patch escalates in-handler to **`sources.policy`** — a feature replacing the former owner/admin role-name check; not granted below the admin tier — owner via `'*'`, admin via its enumerated default grants — and grantable per-role; declared via `export const escalations`). The brain-local `hasRouteFeature` is now a thin wrapper over the shared `hasFeature` (one-predicate invariant). In-handler escalation features (`sources.mount`, `sources.privileged`, `sources.sync`, `files.readOthers`/`files.writeOthers`, `sources.policy`) gate path/scope-dependent operations on top of the route's declared baseline — the same two-tier pattern machine-core uses; they are LIVE, not dead. **`sources.privileged`** (not granted below the admin tier — owner via `'*'`, admin via its enumerated default grants — and grantable per-role) gates SEEing in `discoverable`, ADDing (create), EDITING (PATCH — permission widening on an already-attached protected-rooted source), and — since warded-wren — BROWSING (`sources/browse`) a source rooted in a high-blast-radius zone (the app zone, the Neuralis tree, or the whole host `/`); the add/edit arm is kind-agnostic on the container plane (any non-host-plane kind whose config carries a string `root`); enforced by canonical path containment in `sources.ts` (browse matches IS-or-UNDER only and omits protected children from the listing, and it requires the feature for the filesystem ROOT and for ANY path while the runtime stack is unavailable — an empty protected-root table must never read as *unrestricted*) — see `src/routes/README.md`. A source on the **host plane** — any connector kind that resolves OUTSIDE the container (`host`, `host-*`), i.e. a paired device — is attached or edited only by **that device's owner holding `device.own`**, answered by agent-core's device gate; no role feature opens it for anyone else, at any root, and the check runs BEFORE the container-plane protected-root logic with no runtime-stack precondition (fail closed). **Editing** one needs the same: `PATCH` cannot change `connection`, but it CAN widen `read` → `write`, so a plain-`sources.mount` manager must not be able to escalate an existing host source in place. Beyond that, the device's own allowlist (`neuralis device grant`) bounds what the owner's source can actually reach (see the agent-core README's host-access section). **`packages.author`** gates the package-authoring *affordances* — the Packages-panel scaffold launcher and the install/build actions — granted to `admin` and `manager` via `defaultRoleGrants` (owner passes on the wildcard). It does NOT gate the authoring SKILL: `skills/create-package` is ungated on purpose, so a caller who cannot yet write or install can still read the contract and learn exactly which capability they are missing instead of failing blind. The downstream gates are unchanged — scaffolding into `packages://` still needs `files.write`, and install/build still need `packages.author`.
|
|
901
|
-
- `sources/*` (CRUD + `sync` (202 + queue job snapshot) + `cancel` (per-source `{source}` or all-visible; sources.sync) + `reindex` (202) + `reset` + `sync-status` (GET job snapshots, sources.sync) + `browse` (**`sources.mount`**; the filesystem root and any protected host zone additionally need `sources.privileged`) + `preflight` (POST-only create validation/authorization, no write) + `kinds` + `kinds/:kind/schema` + `enabled` (light toggle, `'auto'` machine-only) + `discoverable` (**`sources.mount`** — well-known unattached source roots from the runtime stack, e.g. the "Neuralis Tree" `neuralis://` mount when `pnpm neuralis:mount add … --neuralis` is configured), mixed `drive.*`). A suggestion may pre-fill **`suggestedSync`** (`include` + `trigger`, both riding the contract — the add form no longer hard-codes `'manual'`) and **`suggestedPolicies`** (uri-policy path rules, `$source`-tokenised so they survive a slug rename). The two are not interchangeable: `include` narrows what is **indexed**, `suggestedPolicies` narrows what can be **read**. The `--neuralis` overlay therefore offers TWO proposals, narrowest first — `neuralis-skills://` rooted at `<root>/skills` (root = boundary, `existsSync`-guarded so a `skills`-less overlay advertises nothing) and `neuralis://` over the whole folder, which pre-fills `include: ['skills/**']` plus a `.env*` read-deny because that root holds `.env`. `POST sources/sync` classifies its two pre-flight refusals rather than redacting them: **404** (plain shape) when the slug does not resolve for the caller, **400** when it resolves but carries no `sync` block. **PATCH replaces `permissions` wholesale — it never merges.** A body carrying only `permissions.default` answers 200 and drops every `paths[]` rule the source held; only `floor:true` rules are protected (409), so a 200 is not evidence the rest survived. Every caller — skill, UI, test — must round-trip the source's current `permissions` object. **A connector-declared floor self-heals on ANY scheme; the manifest-baseline union self-heals on `data`/`packages` only; nothing else self-heals at all.** The two halves run on different gates. `reconcileImmutablePolicyFloors` runs on EVERY boot, past the version stamp, for every source whose connector declares `floor:true` rows — so a machine/webtop source restores its floors too, and a floor row added by a later package version reaches EXISTING configs without a version bump. The version-gated half is the baseline UNION, and a PATCH no longer re-arms it: `updateSourceConfig` rebuilds the record from an explicit FIELD ALLOWLIST (every sibling writer spreads `existing`), and `baselineRepairVersion` was missing from that list, so any PATCH — a no-op included — silently un-stamped the config and the next boot's `repairSourceBaselines` re-unioned the manifest baselines. Since `mergeBaselines`'s `paths` semantics are UNION, that RESURRECTED a non-floor rule a `removeRule` had deliberately taken out. The allowlist now preserves the stamp. On every other scheme there is no baseline union to re-run. Hand-added rules never come back on any scheme.
|
|
902
|
-
|
|
903
|
-
**A non-project-scoped source never keeps the `default` you send — and the two non-project kinds seed DIFFERENTLY.** `applyScopeSeedPermissions` forces `default: {read:false,write:false,exec:false}` and `byRole: {owner:{read:true}, admin:{read:true}}` for BOTH `user` and `agent` scope, but the principal grant branches on the kind:
|
|
904
|
-
|
|
905
|
-
- **`user` scope** — `byUser[creator] = {read:true, write:true, exec:true}`; `byAgent` is not touched.
|
|
906
|
-
- **`agent` scope** — `byAgent[agentId] = {read:true, write:true}` (no `exec`), and **no `byUser` entry is written at all**, so the human who attached it reaches the source only THROUGH that agent.
|
|
907
|
-
|
|
908
|
-
Both scope seeds also write `byRole: { owner: {read:true}, admin: {read:true} }`. Since 2026-08-19 those
|
|
909
|
-
literals are **priority LABELS, not names the evaluator compares**: `resolveRoleLayer` resolves a caller
|
|
910
|
-
at the same declared priority to the same entry, so a custom role declared at the owner anchor reads the
|
|
911
|
-
source and a `manager` still does not (the grant clause matches strength EXACTLY; the cascade clause is
|
|
912
|
-
restrict-only and can never hand a weaker role a stronger role's grant). Deleting these rows is NOT
|
|
913
|
-
neutral — it would leave `default:{false,false,false}` with nothing to lift it.
|
|
914
|
-
|
|
915
|
-
Manifest baselines (the `uriPolicies` provenance) are unioned for the `data`/`packages` schemes ONLY. So on a non-project source (machine/webtop, the app zone) the ALLOW axis is decided at the SOURCE level rather than per path — **with one exception, and it is a deny-only one**: `defaultPermissions.paths` rows marked `floor: true` DO reach the config, on any scheme, through `pickConnectorFloorRules` (`BridgeService.#applyConnectorFloors` on create, `reconcileImmutablePolicyFloors` on every boot). The connector's non-floor ALLOW rows still never reach it. Read the persisted `sources/<project>/<slug>.json`, not the manifest, and check which BRANCH the source's `scope.kind` takes before reasoning about who can read what.
|
|
916
|
-
|
|
917
|
-
Browser consumers import the pure pending record/preview types and shared diff split/work preflight through `@neuralis/brain-core/domain/pending-change`; its packed compiled leaf remains resolvable when `app/` is relocated. Public API (`src/index.ts`) re-exports the URI/source domain helpers, the `VectorIndexPort` and `EmbedderPort` ports, the embedding model registry, the connector kernel re-exports (`LocalDiskConnector`, `BrainInternalConnector`, `AppSourceConfigStore`, `InMemorySourceConfigStore`), the wiring entry points (`buildSharedInfra`, `buildCommunityFilesystemInfra`, `CommunityInfraFactory`), and the lifecycle hooks.
|
|
918
|
-
|
|
919
|
-
The embedding model registry (`src/embeddings/models.ts`, `EMBEDDING_MODELS`) is a declarative one-row-per-model catalog across six providers, plus the operator's own OpenAI-compatible endpoints (`embeddingEndpoints` — URL, auth, `public`/`private` network, models with their dimension, optional price and modalities; written through the host's endpoint route and rendered by the admin Vector section) as dynamic rows `endpoint:<endpointId>:<model>`. An endpoint model the index depends on — the ACTIVE model, a rebuild's target, or the configured target — cannot be taken away: brain-core registers a write guard for `embeddingEndpoints` (`usecases/vector/endpointGuard.ts`, through the kernel's config-write guard port), and the host's endpoint route answers 409 `endpoint_in_use` with the reason for a delete, a disable, a model removal or a rename that would do it:
|
|
920
|
-
|
|
921
|
-
- **Deterministic** — `sha256-384` (dev/test only, no network).
|
|
922
|
-
- **Ollama** (local, free, no credential) — `nomic-embed-text` **(platform DEFAULT @768)**, `mxbai-embed-large`, `all-minilm`, `snowflake-arctic-embed-l`, `embeddinggemma` (Google EmbeddingGemma 300M), `qwen3-embedding` (pinned `qwen3-embedding:0.6b`).
|
|
923
|
-
- **OpenAI** (`embedding.openai`) — `text-embedding-3-small`, `text-embedding-3-large`.
|
|
924
|
-
- **Gemini** (`embedding.gemini`) — `gemini-embedding-001` (text only; shuts down 2028-05-14) and `gemini-embedding-2` (GA, the first-party **multimodal** embedder: text/image/PDF/audio/video; conditioned by in-text prefixes, its own request recipe). Both take any dimension 128–3072. Use a paid key — on the free tier Google may use the content to improve its products.
|
|
925
|
-
- **Qwen** (`embedding.qwen`, DashScope-intl; its own `qwenEmbeddingBaseUrl`, independent of the chat provider's) — `text-embedding-v4`, `text-embedding-v3` (legacy).
|
|
926
|
-
- **Voyage** (`embedding.voyage`) — `voyage-3-large`, `voyage-3.5`, `voyage-3.5-lite`, `voyage-3-lite` (superseded), and `voyage-multimodal-3.5` (text + images in one space, spoken over `/v1/multimodalembeddings`).
|
|
927
|
-
|
|
928
|
-
Every cloud provider has its OWN platform credential (`embedding.<provider>`), never the chat provider's `llm.*` key — so an embed call is counted by the credential-use rules and priced by the embedding spend meter on its own; an operator endpoint's key is `embedding.endpoint.<endpointId>`. Every call to a public provider or endpoint is address-checked and IP-pinned by the kernel's SSRF floor on every call. Many rows expose MRL / Matryoshka selectable dimensions (`supportedDimensions`).
|
|
929
|
-
|
|
930
|
-
**Active vs target, and the switch.** The v3 serving lock records metadata mode or a certified ACTIVE dense generation (model, dimension, recipe, collection). `embeddingEnabled:false` makes payload sync/writes and keyword/filter search provider-free; semantic requests fall back with `requested_mode`, `effective_mode` and `embedding_degraded`. Result rows expose current/stale/missing/unknown embedding status, so old vectors never imply fresh content. Existing no-lock collections preserve unknown vectors in metadata mode without inferring a model. Dense work uses the certified active generation. The configured `embeddingModelId`/`embeddingDimension` are the TARGET; a target that differs is a pending switch the admin Vector section shows with a sampled estimate — nothing switches by itself. The switch is a generation rebuild (`health/collection` `rebuild-generation`): the target re-embeds eligible points from their stored text into a new collection while the old one keeps serving, every write that lands meanwhile is mirrored onto it, and the swap moves the lock one generation on — `brain://` content and version history survive, and the dimension may change freely. It can be cancelled (the shadow is dropped, nothing else changes) and resumed after a restart (a restart pauses it). Separately, `sources/reindex` with `force` re-embeds a file, a folder or a source with the ACTIVE model, estimate first — an unchanged file keeps its payload (`updatedAt`, version history), so a re-embed never reads as an edit. Already recovery-only points with unavailable input retain their complete payload without vectors in the target; verified available input restores ordinary embedding. Unmarked missing input still pauses the rebuild. Recovery retention, cleanup and generation eligibility follow the constraints under Manifest-baseline merge below.
|
|
931
|
-
|
|
932
|
-
**Media — images, PDFs, video, audio.** With a multimodal ACTIVE model (`gemini-embedding-2` or `voyage-multimodal-3.5` — the registry `modalities` as far as the model's client can send them: an operator endpoint speaks the text-only `/v1/embeddings` wire, so its row's declared media modalities are kept but not embedded) the sync embeds media files from their BYTES: an image is one point, a PDF is embedded page by page through the connector's PDF renderer (the document point carries page 1, pages 2..`brainSyncMaxPdfPages` are page chunks), each point marked `modality` and carrying no stored bytes — a rebuild or a forced re-embed reads the source again. Each source chooses the kinds in its sync settings (`sync.media`): images and PDFs on, video and audio off by default; `brainSyncMaxMediaMb` bounds one file, and a kind switched off is purged by the next full pass. A text-only model or disabled embedding still indexes enabled media metadata and PDF page inventory without provider calls, reporting embedding debt; it never invents OCR text or semantic vectors. Invoked PNG inputs receive bounded structural signature/chunk/CRC checks before provider work; an invalid skip preserves existing indexed and recovery payloads and is counted separately. This preflight does not decode pixels. A semantic search returns media hits with their `modality` (and a PDF's `page`); full-text search never matches them, they carry no text.
|
|
933
|
-
|
|
934
|
-
**Embedding spend.** Every priced or keyed embed call leaves one row in `<appRoot>/data/brain-core/embedding-usage.jsonl` (`{ts, model, tokens, estUsd, calls}`; an unpriced model is counted, never priced) and adds to today's aggregate the Vector section shows; `embeddingDailyUsdCap` (0 = off) stops embedding for the rest of the UTC day once reached — payload sync continues with embedding debt and stops later provider calls for that job; semantic search falls back to full-text.
|
|
935
|
-
|
|
936
|
-
**Credential-use limits (v1).** Embedding spend is governed by the platform's per-credential
|
|
937
|
-
calls/period rules through `embeddings/usageGate.ts`: brain-core's `init` stashes the
|
|
938
|
-
host-injected `ctx.credentialUse` port, and the factory threads a per-provider-HTTP-call
|
|
939
|
-
guard (`guardCall`) into every cloud client — awaited at the top of each `#post`, so a
|
|
940
|
-
batch that splits into several posts counts one use per post. This per-call hook is the
|
|
941
|
-
REAL enforcement for the embed class (not telemetry): the key is construction-captured and
|
|
942
|
-
`LazyEmbedder` memoizes the client, so the host's resolve-time counter sees roughly one
|
|
943
|
-
grant per process. A deny throws the typed `CredentialUseLimitedError`
|
|
944
|
-
(`code: 'credential_use_limited'`): **sync** continues payload indexing and records
|
|
945
|
-
`embeddingDegraded`; later provider calls in that job stop, while already running calls finish,
|
|
946
|
-
**semantic search** soft-degrades to the full-text filter search with a `warning` on the
|
|
947
|
-
response envelope — as it does for the embedding daily cap, a missing `embedding.*` key
|
|
948
|
-
(`EmbeddingCredentialMissingError`) and an active model the platform no longer defines;
|
|
949
|
-
Typed provider/network/timeout/quota unavailability degrades; validation, parse and local authorization errors propagate before the payload lands, storage errors propagate, and after the durable payload every embedding error — typed or not — is named debt in the answer (`usage.metadata.embeddingDeferred`), never a failed write. A sync against an out-of-credit embedder COMPLETES with `lastSyncResult.embeddingDegraded` (at most the concurrency's worth of provider calls, then no more that job) and writes its checkpoint, so the source is not due again until its own interval; only the embedding-only pass pauses, with the reason.
|
|
950
|
-
|
|
951
|
-
since 2026-08-18 the search degrade covers a SECOND typed code:
|
|
952
|
-
`ProviderQuotaExhaustedError` (`code: 'provider_quota_exhausted'`,
|
|
953
|
-
`embeddings/providerQuota.ts`), raised when the provider's own 429 body proves the account
|
|
954
|
-
is out of credit (`insufficient_quota` / `credit_balance_exhausted` / …). That 429 is
|
|
955
|
-
thrown immediately instead of burning the retry budget on a wait that cannot help, and
|
|
956
|
-
semantic search answers from full-text rather than a 500. The classification **fails toward
|
|
957
|
-
retryable** — an unparseable or unrecognised body keeps the old backoff, because an
|
|
958
|
-
OpenAI-compatible custom `baseUrl` may return anything — and a genuine, retry-exhausted rate
|
|
959
|
-
limit becomes typed availability after its retry budget, retaining the safe degraded reason. Every degrade code lives on
|
|
960
|
-
the USE-CASE, so the `fs_search` tool inherits it, not just the route.
|
|
961
|
-
Deterministic/Ollama models carry no credential and are never gated. Rules are set on the
|
|
962
|
-
admin Credentials tab (exact-id). The `embedding.*` keys are ordinary counted credentials —
|
|
963
|
-
the LLM keys' exclusion from use rules (USD spend limits govern those) no longer reaches an
|
|
964
|
-
embed call, because no embed call uses an LLM key.
|
|
965
|
-
|
|
966
133
|
The per-category contribution folders are the agent-facing knowledge surface for brain-core:
|
|
967
134
|
|
|
968
135
|
- `rules/brain-rules.md` (always on) and `instructions/brain-model.md` (on demand) explain URI identity, connector-first read/list behavior, search freshness, the **fs write policy** (modify-don't-recreate, read-before-modify, `revert` as 1-level undo — moved here from agent-core's `tool-use` so the filesystem owner owns writes), source scope and the independent gates; each `fs_*` tool description carries its own mechanics. brain-core ships no `docs/` and no MCP `resources[]`.
|
|
@@ -976,529 +143,28 @@ There is intentionally no README inside the contribution folders; this section i
|
|
|
976
143
|
## How it plugs into the package-system kernel
|
|
977
144
|
|
|
978
145
|
- `init(ctx: PackageInitContext)` builds shared infra (Qdrant + lazy embedder), the per-project `CommunityInfraFactory`, the global `AppSourceConfigStore`, the per-package sync/vector loggers, and the `VectorHealthService`. It returns a `BrainCoreState` and never starts the sync scheduler.
|
|
979
|
-
- `start(state)` begins the background vector activity watch (`startQdrantWatch` wires `usecases/vectorActivityWatch.ts`, every `brainQdrantWatchIntervalMs` — 30 s — `
|
|
146
|
+
- `start(state)` begins the background vector activity watch (`startQdrantWatch` wires `usecases/vectorActivityWatch.ts`, every `brainQdrantWatchIntervalMs` — 30 s — whenever the backend is Qdrant, `local` and `real` alike). Qdrant is wired at init whenever it is intended, reachable or not; the watch is what drives its availability gate: one bounded probe at a time (a probe-only client bounded by `brainQdrantProbeTimeoutMs`, 5 s, boot-read), one immediate probe when a call finds the store gone, and on the first answer the boot-once steps (server-version read, embedding-generation boot with its collection ensure) run once before the gate opens — no restart, ever. It logs `vector.qdrant.lost` and `vector.qdrant.recovered` (held sync jobs then run) and reports `degraded` / `store_unreachable` to health while the gate is closed. Every Qdrant request is bounded by `brainQdrantRequestTimeoutMs` (default 300 000 — the client's own default, so no long index build is newly aborted; boot-read). While Qdrant is reachable the same tick reads the server's optimizer tracker (one `GET /collections/<c>/optimizations?with=completed`, at most 50 finished runs) — Qdrant does not log its own background optimization — and writes `vector.optimizer.completed {optimizer, status, points, segments, durationMs}` once per finished run (the SERVER's start/finish times, so a run shorter than the tick is not lost), `vector.optimizer.long` once for a run still going after 10 minutes, and `vector.optimizer.overflow {returned}` when every returned run is new while earlier ones were seen (the tracker rolled past runs between two reads). The first read only seeds: runs finished before the process started are not re-logged at every boot. These lines go to the console (`docker logs`) AND `vector.jsonl`; the per-probe `vector.qdrant.health` line stays file-only. The lifecycle `health(state)` returns structured backend/connectivity checks to the generic runtime health aggregate; no vector-status package API is exposed. The interval is cleared in `stop()`.
|
|
980
147
|
- `onBootstrapComplete(state, ctx: BootstrapCompleteContext)` is the single deferred-wiring entry point: it reads the published registries through `ctx.platform.connectorRegistries()` and starts the `SyncScheduler` from `ctx.platform.hostIdentity()`. The factory resolves those registries lazily through its init context. There are no imperative setters (`setHostIdentity`, `setConnectorRegistries`).
|
|
981
148
|
- `init` delegates to `wiring.buildBrainState`, wiring the declared `ctx.config` reader and GLOBAL-only `ctx.credentials.resolveCredential` into the embedding provider before shared infrastructure is built. Platform facts are read lazily through `ctx.platform`; no config setter or source-running bootstrap push is exposed. Credential changes reach the bounded lifecycle hook, and runtime health reads structured vector checks.
|
|
982
|
-
- **Data formats.** brain-core's three durable record kinds — `brain-core/embedding-lock` (v3), `brain-core/source-config` (
|
|
149
|
+
- **Data formats.** brain-core's three durable record kinds — `brain-core/embedding-lock` (v3), `brain-core/source-config` (v7) and `brain-core/pending-changes` (v1, the `_pending/changes.jsonl` journal) — are declared on the kernel's data-format ledger at module load (`src/store/dataFormats.ts`) and claimed at the top of `init`, before the first read: a kind a newer binary raised stops the init by name. Per record, the lock's `version` and a source config's or pending change's `formatVersion` (absent = 1) are checked before every rewrite — the check reads the STORED record, never the incoming one, so a config built fresh over a newer file is refused too — and a newer record is never rewritten (the journal's appends stay open; its compaction refuses). Every source-config write stamps the declared version (`SOURCE_CONFIG_FORMAT.version`, 7); an identical re-upsert still writes nothing. `embedding-lock` v3 and `source-config` v7 are `upgrade: offline` kinds: the app refuses to start over older data naming `neuralis upgrade`, which raises them (`scripts/embeddingLockV3.ts`, exported as `@neuralis/brain-core/upgrade`; `source-config` is a chain — the host's `package-identity-v2` step, then `scripts/sourceRootPairV1.ts`, exported as `@neuralis/brain-core/upgrade/source-root-pair`, which fills the root pairs, then `scripts/sourceToFolderV1.ts`, exported as `@neuralis/brain-core/upgrade/source-to-folder`, which merges the old `local` and `host` records into ONE `folder` kind and turns `recognizesPackages` into `packageFolders`, then `scripts/sourceToHostV1.ts`, exported as `@neuralis/brain-core/upgrade/source-to-host`, which spells that kind `host`, then the host's `source-uri-subject-v1` step, which moves owner/admin grant rows to `priority` (the caller's role priority, n or stronger), then the host's `source-use-grants-v1` step, which places agent-core's private-agent rows right after a project's first agent-folder write row and appends one read row per user who already used an agent they did not create — each step passes a record a later step already raised through untouched, so a resumed chain never re-reads one in an older vocabulary). Source configs are written durably, and one that fails validation is skipped AND logged (`source-config.invalid-skipped`), never silently gone.
|
|
983
150
|
- **Declared credentials + dead-sweep.** brain-core's manifest declares the four embedding credentials — `embedding.openai`, `embedding.gemini`, `embedding.qwen`, `embedding.voyage` — in `neuralis.credentials[]` (`category: 'vector'`), so they appear in the admin catalog for an owner to set; an operator endpoint's `embedding.endpoint.<id>` is a dynamic family, never declared. `qdrant.apiKey` is deliberately NOT declared: `getQdrantClient()` is synchronous and boot-critical, so it structurally cannot read the credential store; the value comes from `QDRANT_API_KEY` in the boot env (Qdrant is infrastructure topology, not a per-scope secret — see `src/config/README.md`; setup writes the line itself and the server enforces the key on every request, with a mismatch surfacing as the structured `vector.qdrant.unauthorized` log line + a degraded Vector tab, not a boot death). `src/config/env.ts` was swept of dead credential plumbing: the old `resolveCredential()` helper, `BRAIN_CREDENTIAL_MAP`, and the `s3.*` / `media.apiKey` env reads (declared nowhere, consumed nowhere) are all removed. Only `getNeuralisEnv().qdrant.*` (URL/apiKey/collection — boot-critical infra topology from `process.env`) and the registry-driven embedding resolvers remain.
|
|
984
|
-
- **Declared config settings.** brain-core's manifest declares its
|
|
151
|
+
- **Declared config settings.** brain-core's manifest declares its platform-config keys in `neuralis.configSettings[]` (`PackageConfigSetting[]`) — sync/scan/chunking/media/binary-sniff/git-panel/watch-recovery/vector-maintenance/vector-write-retry/pending-orphan-probe/search caps and timeouts across three categories: `brain`, `vector` and `debug` (the manifest array is the count). The originals: the `brain` category (`brainAutoSyncMode`, `brainMaxScanFiles`, `inlineVideoMaxMb`, `brainSyncConcurrency`, `brainOllamaEmbedConcurrency`, `brainDebug` — `brainEmbedConcurrency` was DELETED: every embedder implements `embedBatch`, so the only fan-out fallback was dead code), the `vector` category (`embeddingModelId`, `embeddingDimension`, `ollamaUrl`, `openaiBaseUrl`, and `embeddingEndpoints` — written only through the host's endpoint route — rendered by the admin Vector tab, not the flat list), the embedding keys in `brain` (`embeddingEnabled`, `embeddingRequestTimeoutMs`, `qwenEmbeddingBaseUrl`, `embeddingDailyUsdCap`, `vectorKeepPreviousGeneration`), the media lane's `brainSyncMaxMediaMb` and `brainSyncMaxPdfPages`, and the `debug` category (`brainVectorLogConsole` — the stdout tee described in `src/vector/README.md` §Observability). The host registers them into `PlatformConfigStore` at bootstrap via the agent-core collector (native trust-gated); values reach brain-core through the injected `BrainConfigProvider`; the shared `getBrainConfigNumber(key, env, default)` live reader (config/env.ts) backs `brainSyncConcurrency` (DeltaSyncRunner) and `brainOllamaEmbedConcurrency` (the OllamaEmbedder semaphore, embeddings/factory.ts — applies on restart/embedder rebuild). The agent-core `configDeclarationDrift` test pins brain-core's consumed ⊆ declared set bidirectionally.
|
|
985
152
|
- The `CommunityInfraFactory` uses the kernel’s bounded `ProjectServiceFactory` internally. Routes and tools resolve services via `state.infraFactory.getServices(projectId)` — no global default-project state.
|
|
986
|
-
- Source connectors are declared in package manifests via `neuralis.connectors[]`. Connector **kinds** register into the host's `SourceKindRegistry` at boot (the factory templates); Brain-core itself contributes the built-in `
|
|
153
|
+
- Source connectors are declared in package manifests via `neuralis.connectors[]`. Connector **kinds** register into the host's `SourceKindRegistry` at boot (the factory templates); Brain-core itself contributes the built-in `host` and `brain` kinds — `host` through `createHostConnector`, which builds `HostConnector` (the in-process transport) or asks agent-core's device API for the device transport, and `brain` through `BrainInternalConnector`. Connector **instances** are bound PER PROJECT — `buildCommunityFilesystemInfra` constructs a fresh `ConnectorRegistry` inside each project's `FilesystemInfra` and `registerForSource`s that project's source slugs into it. Two projects with the same local slugs (`data`/`packages`/`brain`/`app`) therefore never collide; the previously-shared host registry was last-writer-wins and leaked one tenant's filesystem (and poisoned the shared vector index) into another. The host-shared registry handed in at bootstrap is retained only as the boot-readiness gate, no longer a source-binding store.
|
|
987
154
|
- Brain-core declares **no hooks**. The former `hooks.json` / `host.ts` `onFileCreated` / `onFileModified` PostToolUse callbacks were deleted: they were intentional no-ops (the create/modify use cases update the vector index inline), and their registration produced a noisy `[hooks] Callback resolution failed` warning whenever `dist/src/host.js` drifted. The `./host` package export was removed with them.
|
|
988
155
|
|
|
989
156
|
## Security & scope
|
|
990
157
|
|
|
991
158
|
- Every route helper (`src/routes/helpers.ts`) extracts `(userId, projectId, agentId, role, grantedFeatures, agentAccess, agentOwnership)` from the canonical `req.session` (`SessionContext`) and rejects sentinel identities. `projectId`/`userId` come straight from the session; `agentId` resolves `body ?? query ?? req.session.agentId`, so a session-ticket caller (skill-script curl) inherits the stream's agent without passing `AGENT_ID` — `ensureAccess` only 400s when no agent is present anywhere. A body/query `projectId` that disagrees with the session is rejected (403). `getServices(state, projectId)` resolves project-scoped infra; sentinel `__system__` ids never reach a real Qdrant collection.
|
|
992
|
-
- Source scope evaluation uses `canAccessScope` from `@neuralis/package-system/contracts`. Project-scoped sources are visible to every project member; user-scoped sources require ownership or the `files.readOthers` / `files.writeOthers` feature. The grant alone decides — `canAccessScope` carries no role-name conjunct, so a custom role explicitly granted the feature observes exactly what a wildcard holder does. Default grants: `files.readOthers` AND `files.writeOthers` are both enumerated for `admin`; `owner` reaches them through the `'*'` wildcard; `manager`/`member`/`viewer` hold neither and reach a foreign scope only through an explicit grant. Since `canAccessScope` lost its role-name conjunct, these grants are what carry the decision — they are not inert. `allowedScopes` declared on a connector kind rejects mismatched scope at create time. Every route-level source-config read (`BridgeService.getSourceConfig` / `listSourceConfigs`) MUST pass the caller's access context (`toScopeAccessCtx(ctx)` from `routes/helpers.ts`) — omitting it downgrades the caller to a feature-less `member` and hides user/agent-scoped sources from their own owner (the "Source not found on edit" regression). The `FilesystemService` accessible-sources cache is keyed on the full access context (user, project, agent, role, grantedFeatures), never on `userId:projectId` alone — a narrower key lets one agent's set poison another agent's view for the TTL window.
|
|
159
|
+
- Source scope evaluation uses `canAccessScope` from `@neuralis/package-system/contracts`. Project-scoped sources are visible to every project member; user-scoped sources require ownership or the `files.readOthers` / `files.writeOthers` feature — except a user-scope folder on a DEVICE, which serves its owner alone: every gate that reads a stored source scope passes the source's plane (`sourceScopeOrigin`), and the kernel refuses every cross-user grant, `'*'` included, on the device plane. The grant alone decides — `canAccessScope` carries no role-name conjunct, so a custom role explicitly granted the feature observes exactly what a wildcard holder does. Default grants: `files.readOthers` AND `files.writeOthers` are both enumerated for `admin`; `owner` reaches them through the `'*'` wildcard; `manager`/`member`/`viewer` hold neither and reach a foreign scope only through an explicit grant. Since `canAccessScope` lost its role-name conjunct, these grants are what carry the decision — they are not inert. `allowedScopes` declared on a connector kind rejects mismatched scope at create time. Every route-level source-config read (`BridgeService.getSourceConfig` / `listSourceConfigs`) MUST pass the caller's access context (`toScopeAccessCtx(ctx)` from `routes/helpers.ts`) — omitting it downgrades the caller to a feature-less `member` and hides user/agent-scoped sources from their own owner (the "Source not found on edit" regression). The `FilesystemService` accessible-sources cache is keyed on the full access context (user, project, agent, role, grantedFeatures), never on `userId:projectId` alone — a narrower key lets one agent's set poison another agent's view for the TTL window.
|
|
993
160
|
- A delete asks through the GUARD, not through a hand-written gate: `fs_write` declares `x-neuralis.askOnKeys: ["delete"]`, so the presence of that root key makes `balanced` open the approval while an edit stays quiet, and `auto` runs neither. Approval replays the original `toolUseId` and runs the mutation; denial leaves connector and index state untouched. The old tool-side consent (`maybeRequestDeleteApproval`) is gone — a second approval gate outside `evaluateGuard` is exactly what `auto` could not short-circuit.
|
|
994
161
|
- **Pending-resolution security floor (2026-07-17).** Every pending-change verb (list / approve / revert / discard) resolves its vector target with an exact tenant-scoped `(projectId, id)` lookup — never a collection-global `retrieve(id)` — so a same-id point in another project is invisible: the previously tenant-blind lookups are hardened and the bounded uri-echo disclosure (a foreign tenant's point donating its uri to a discard/revert response) is closed. Chunk deletes/relocations are project-scoped the same way. The canonical check order on every mutating verb: session identity → `files.write` (in-handler, before any lookup) → projectId tenant boundary → per-record owner (`canAccessScope` + `recordOwnerScope`) → source scope + uri-policy on the RESOLVED uri(s) (including a move destination and a move-revert's baseline uri; id-only modifies are gated too) → only then side effects. Cross-user resolution widens ONLY for a `files.writeOthers` HOLDER — the grant decides, not the role name (`canAccessScope` carries no role-name conjunct) — and preserves the record author's provenance; approving a delete tombstone deletes the point + chunks instead of resurrecting an active ghost. The owner floor also precedes the idempotent already-terminal early-return on discard/dismiss — a foreign caller probing a terminal record id gets `FORBIDDEN` (403) with no uri echo; terminal state is no oracle. The route maps every single-record resolution verb (revert / discard / dismiss) through ONE status mapping: `FORBIDDEN` → 403, `NOT_FOUND` → 404, retryable → 503, else 409.
|
|
995
|
-
- `
|
|
162
|
+
- `HostConnector` blocks symlink traversal, `..` escape, and explicit access to the platform app zone (`~/.neuralis/app`) or credential files. On a case-insensitive volume (Windows NTFS, macOS APFS) a path has more spellings than the five the policy canonicalizes: letter case is decided by the policy (a rule's deny bits match every case, its grants only the exact spelling), and every other alias — an 8.3 short name, an NTFS stream (`meta.json::$DATA`, `dir::$INDEX_ALLOCATION`), another Unicode normalization — is refused by the in-process transport exactly like a missing path, a create naming a stream included. Residual: a Linux case-insensitive mount (drvfs, CIFS, ext4 casefold) answers the requested spelling, so only the policy fold applies there.
|
|
996
163
|
- The brain source is per-agent inside the same project — `searchFiles` isolates brain rows by `agentId` unless `include_siblings` is set: an explicit `brain://` scope gets the vector pre-filter, an UNSCOPED (no `folder_uri`) search drops foreign brain rows per-node via `isVisibleSearchNode` while non-brain rows stay project-shared (the old global pre-filter starved cross-source search); cross-agent observation requires the corresponding feature.
|
|
997
164
|
|
|
998
|
-
|
|
999
|
-
|
|
1000
|
-
|
|
1001
|
-
gates whether `fs_*` tools and route handlers can reach a source at all —
|
|
1002
|
-
orthogonal to `uriPolicies` (the policy says *what* URIs you can touch;
|
|
1003
|
-
`enabled` says *whether* the source is live in this scope). `'auto'` derives
|
|
1004
|
-
from the configured connector's `isRuntimeActive()` member, bound to the
|
|
1005
|
-
project and source by its factory context; the probe never starts a runtime. Disabled or
|
|
1006
|
-
`'auto' && !isRunning` sources surface a structured `SourceDisabledError`
|
|
1007
|
-
(`code: 'source_disabled'`, `nextStep: 'enable_source'`) which the
|
|
1008
|
-
`fs_*` tools forward to the use→result card body. Toggle endpoint:
|
|
1009
|
-
`POST /sources/enabled` with `{ source, scope, enabled }`.
|
|
1010
|
-
|
|
1011
|
-
**A deleted user's own sources are disabled, never removed**
|
|
1012
|
-
(`usecases/revokePrincipal.ts`, the package's `onPrincipalRevoked`). On the
|
|
1013
|
-
host's `deleted` revocation every source that user OWNS — user scope, or agent
|
|
1014
|
-
scope under them — has its sync cancelled and is switched off through the same
|
|
1015
|
-
`BridgeService.setSourceEnabled` the toggle uses, whose sources-changed signal
|
|
1016
|
-
drops the connector's OS-watch subscription. The config, its permissions, the
|
|
1017
|
-
files behind it and its vectors all stay; nothing removes a source root. Every
|
|
1018
|
-
other revocation reason is a no-op here: the user is already refused by the
|
|
1019
|
-
host's session producer, and their filesystem subscriptions close with the
|
|
1020
|
-
`/api/events` hub connection.
|
|
1021
|
-
|
|
1022
|
-
A source slug that has **no config at all** in this project is a different
|
|
1023
|
-
answer: `SourceNotConfiguredError` → **404** `code: 'source_not_configured'`,
|
|
1024
|
-
`retryable: false`, `nextStep: 'check_configured_sources'`, plus an
|
|
1025
|
-
`availableSources` list of the caller's OWN visible sources so one round trip
|
|
1026
|
-
fixes a wrong guess. Before this it fell into the catch-all 500, where the
|
|
1027
|
-
dispatcher redacts the message away.
|
|
1028
|
-
|
|
1029
|
-
A source the caller cannot SEE — another user's or agent's, without
|
|
1030
|
-
`files.readOthers` — answers exactly like an absent one, on every plane:
|
|
1031
|
-
the same 404 body (its slug never in `availableSources`; a disabled hidden source
|
|
1032
|
-
never answers `source_disabled`, which named its owner), the same 403 at the uri
|
|
1033
|
-
guard. (Creating a source under a taken slug is still refused — the per-project
|
|
1034
|
-
slug namespace needs that, and only a `sources.mount` holder reaches it.) A source the caller CAN see keeps its real
|
|
1035
|
-
answers: `403 uri_policy_denied` for a denied path, `409 source_disabled` when it
|
|
1036
|
-
is off — never the 404 and its suggestion list.
|
|
1037
|
-
|
|
1038
|
-
Which layer answers is measured, not assumed: the route-level uri guard runs
|
|
1039
|
-
first and denies an unconfigured source as `403 uri_policy_denied`, so the 404
|
|
1040
|
-
surfaces on `list` (the one fs route with no pre-guard — and where all 67
|
|
1041
|
-
observed failures were) and, for scheme-less uris, on `write`/`upload`.
|
|
1042
|
-
The per-route matrix lives in `src/routes/README.md`; **read it before adding
|
|
1043
|
-
an arm to a route that cannot reach one.**
|
|
1044
|
-
|
|
1045
|
-
The identity-free shell-root export applies this lifecycle gate before returning
|
|
1046
|
-
roots (including the `'auto'` runtime probe) and carries each source's scope.
|
|
1047
|
-
Agent-core then applies the kernel `canAccessScope` predicate to the real caller
|
|
1048
|
-
before uri-policy evaluation. A disabled or foreign user-scoped source therefore
|
|
1049
|
-
cannot be recovered by guessing its slug/path through shell or terminal routes.
|
|
1050
|
-
|
|
1051
|
-
**`live` / `live_reason` — connector reachability, tone-only (distinct from
|
|
1052
|
-
`enabled`).** A **universal** signal: whether a source's connector can currently
|
|
1053
|
-
reach its backing store. Resolved per `list` in `FilesystemService.#resolveLiveness`
|
|
1054
|
-
via two mechanisms, in precedence order:
|
|
1055
|
-
1. **connector-owned** — the connector's optional `probeConnection()`. `LocalDiskConnector`
|
|
1056
|
-
implements it (one cheap `fs.access` — the exact symptom of a missing bind mount,
|
|
1057
|
-
e.g. an absent `/mounts/repo`, becomes `live:false`).
|
|
1058
|
-
2. **host-runtime fallback** — for `runtimeGated` kinds (e.g. `machine-core`'s `webtop`
|
|
1059
|
-
container, which the connector doesn't own) via the injected `isSourceRunning` probe.
|
|
1060
|
-
|
|
1061
|
-
`live` is `undefined` for always-on connectors with no probe → never dimmed.
|
|
1062
|
-
`live_reason` is the fixed `SourceLiveReason` enum (`runtime_offline | fs_unreachable
|
|
1063
|
-
| fs_permission | store_unreachable`) — never a raw fs path (path-disclosure guard);
|
|
1064
|
-
the display string is mapped in the UI. `BrainInternalConnector` deliberately has **no**
|
|
1065
|
-
`probeConnection` (the in-memory degraded fallback always serves, so brain is never
|
|
1066
|
-
truly offline today — revisit if that changes). Unlike `enabled`/`enabled_effective`,
|
|
1067
|
-
`live` **never** feeds the `accessible` set or the file filter — it is purely the tone
|
|
1068
|
-
signal (file tree + Sources panel dim the icon **and** name; `<runtime_stack>` shows an
|
|
1069
|
-
`offline (<reason>)` marker in the volatile `<workspace_live>` block). Net effect: a
|
|
1070
|
-
source whose connector is disconnected dims instead of lying as Healthy, yet its
|
|
1071
|
-
already-synced files stay readable; full tone returns once the connector reconnects.
|
|
1072
|
-
|
|
1073
|
-
### `fs_read` vision blocks
|
|
1074
|
-
|
|
1075
|
-
`fs_read` now emits image / video / audio / PDF content as MCP content
|
|
1076
|
-
blocks alongside the markdown summary, so a capable model can ingest a
|
|
1077
|
-
URI the same way it ingests `machine_use screenshot`.
|
|
1078
|
-
|
|
1079
|
-
- **Connector seam.** `ConnectorReadResult` carries an optional
|
|
1080
|
-
`bytes?: string` (base64). `LocalDiskConnector.read()` populates it
|
|
1081
|
-
when the file MIME is `image/*` (cap 4 MB), `video/*` (cap = the
|
|
1082
|
-
**admin-tunable `inlineVideoMaxMb` platform setting**, default 16 MB,
|
|
1083
|
-
env fallback `NEURALIS_INLINE_VIDEO_MB`, read LIVE per call via the
|
|
1084
|
-
brain config provider; the bytes are transit-only,
|
|
1085
|
-
the stream delivery boundary decides each model's final form),
|
|
1086
|
-
`audio/*` (cap 8 MB), or `application/pdf` (cap 4 MB). Above the cap,
|
|
1087
|
-
`bytes` stays unset and `text` carries
|
|
1088
|
-
`[binary file too large to inline: <mime> <size>]`.
|
|
1089
|
-
- **PDFs.** Default path runs `pdftotext` (when available) and returns
|
|
1090
|
-
the first 200 KB of extracted text. With `NEURALIS_FS_READ_PDF_RENDER=1`
|
|
1091
|
-
set, the connector additionally renders the first page via `pdftoppm`
|
|
1092
|
-
and emits the rendered PNG bytes — useful for vision models that
|
|
1093
|
-
outperform on layout-rich PDFs. When neither binary is installed a
|
|
1094
|
-
clean fallback marker is returned. For the render case the connector
|
|
1095
|
-
also sets `ConnectorReadResult.sourceMimeType: 'application/pdf'` (the
|
|
1096
|
-
pre-render source MIME, distinct from the emitted `image/png`); the
|
|
1097
|
-
use-case forwards it as `files[].rendered: true`, and `fs_read`
|
|
1098
|
-
surfaces it on `structuredContent.files[].rendered`. This is the signal
|
|
1099
|
-
the agent-core stream journal uses to persist a rendered-PDF png INLINE
|
|
1100
|
-
rather than as a re-resolvable `ref{uri}` — re-reading the `.pdf` uri on
|
|
1101
|
-
replay would re-render non-deterministically (or, with rendering off,
|
|
1102
|
-
return text → an empty image → a provider 400).
|
|
1103
|
-
- **Unconditional producer.** `readFilesUseCase` emits media whenever the connector supplies bytes and a recognized media MIME. Both direct-URI and indexed reads use the existing binary-extension table when a byte-bearing connector omits MIME. Unknown valid UTF-8 remains text; invalid UTF-8 and opaque files keep their source URI and an honest marker. Upload/write paths preserve original bytes. The agent-core per-step delivery boundary alone decides model acceptance, image integrity and explicitly declared subtype limits; converters only translate shapes.
|
|
1104
|
-
- **Per-batch budget.** The use-case sums approximate decoded sizes of
|
|
1105
|
-
every attached block; once the running total crosses 12 MB, remaining
|
|
1106
|
-
blocks downgrade to a `[batch image budget exceeded — request fewer
|
|
1107
|
-
files]` marker so the response stays inside the runtime tool-result
|
|
1108
|
-
cap.
|
|
1109
|
-
- **Tool output.** `fs_read.ts` now returns a multi-block `content`
|
|
1110
|
-
array: text summary first, then per-file anchor header + image/resource
|
|
1111
|
-
block for every file with a vision payload. The `structuredContent`
|
|
1112
|
-
preserves the file list shape (no blocks — `block` is content-only) so
|
|
1113
|
-
downstream consumers don't double-cost. For each file that produced a
|
|
1114
|
-
block, `structuredContent.files[]` additionally carries `mediaUri: f.uri`
|
|
1115
|
-
(and `rendered: true` for a rendered PDF). The blocks in `content[]` and
|
|
1116
|
-
the `mediaUri` entries in `structuredContent.files[]` are emitted in the
|
|
1117
|
-
SAME order, so a consumer can pair the Nth media block with the Nth
|
|
1118
|
-
`mediaUri` positionally (no MIME heuristic) — this is what lets the
|
|
1119
|
-
agent-core journal persist a re-resolvable `ref{uri}` for a real image
|
|
1120
|
-
while keeping a rendered PDF inline.
|
|
1121
|
-
|
|
1122
|
-
Review uses a fresh `POST /pending` `{action:"preview",id}` receipt, separating current connector bytes from the recorded snapshot and saved baseline. The preview reports missing/unavailable/binary/too-large content and missing/stored/stale/recovery-only index states within `pendingPreviewMaxBytes` (default 262144). Approve (`POST /approve`) and Revert/Dismiss (`POST /pending`) carry `expectedFingerprint`; states requiring acknowledgement additionally need `acknowledgeCurrentState:true` after review. Freshness, tenant, owner, source scope and URI READ/WRITE gates run again at apply. Missing state is never fabricated, backend errors never become absence, and acknowledgement grants no permission. Approve/Dismiss preserve the stored snapshot and index state, including a genuinely absent index; recovery-only metadata uses explicit vector-clearing retention. Revert uses the saved baseline, refuses missing baselines and occupied destinations, and gates both move coordinates. Legacy unsafe pending `/write revert:true` requests refuse; ordinary non-pending one-level undo remains available. Repeated terminal Approve/Dismiss is owner-gated and idempotent; the response omits `uri` if current READ or WRITE access is unavailable.
|
|
1123
|
-
|
|
1124
|
-
### Source delete + vector purge
|
|
1125
|
-
|
|
1126
|
-
`POST /sources/purge` cleans cached index data without deleting the source or connector files. Body: `{source,selector:{kind:"source"}|{kind:"file"|"folder",uri},mode:"estimate"|"apply",obsoleteOnly?:boolean}`. Canonical selectors stay within the source; both sources.mount and sources.sync plus source-management and per-path READ gates apply. `obsoleteOnly:true` removes only current pattern/media exclusions; false/omitted explicitly clears selected eligible cached data too, while retaining recovery payloads. Estimate returns counts without provider work; confirm before apply. Unknown records remain unresolved. The projected inventory reuses caller READ configuration until its revision changes; selected points recheck fresh READ and source-management authority before hydration or mutation.
|
|
1127
|
-
|
|
1128
|
-
`DELETE /sources/:slug` always removes the source-config JSON and the
|
|
1129
|
-
in-memory connector binding. **The host folder is never touched** — the
|
|
1130
|
-
brain has no business deleting user content; it only owns the wiring.
|
|
1131
|
-
|
|
1132
|
-
By default vector index points scoped to the deleted source are also
|
|
1133
|
-
left alone — they become orphans whose parent file is no longer
|
|
1134
|
-
synced. Pass `?purge=1` (or `&purge=true`) to additionally purge those
|
|
1135
|
-
points. Chunks denormalize `folder_ancestors` from their parent, so ONE
|
|
1136
|
-
filter covers parents and chunks alike: `BridgeService.deleteSource`
|
|
1137
|
-
enqueues an abortable `mode: 'purge'` job that issues a single bounded
|
|
1138
|
-
server-side `deleteByFilter` on `{ projectId, folder_ancestors:
|
|
1139
|
-
"<source>://" }` — never a project-wide scroll with client-side matching.
|
|
1140
|
-
Because the purge is queued, the route answers **202** with
|
|
1141
|
-
`{ ok: true, purgeJob }` (a `SyncQueueJobSnapshot`); the removed-point
|
|
1142
|
-
count arrives on the job, pollable via `GET /sources/sync-status`.
|
|
1143
|
-
Without `?purge` it is a 200 with `{ ok: true }`.
|
|
1144
|
-
|
|
1145
|
-
The Sources panel "Delete" button surfaces this as an inline confirm
|
|
1146
|
-
popover with a `Purge vector index entries` checkbox (default off,
|
|
1147
|
-
conservative). Without the purge, leftover orphan points can be mopped
|
|
1148
|
-
up later via Vector Health → Repair.
|
|
1149
|
-
|
|
1150
|
-
CLI / Docker: `pnpm neuralis:mount remove <slug>` is intentionally
|
|
1151
|
-
process-isolated from the running app — it only edits config JSON and
|
|
1152
|
-
`docker-compose.yml`. It does **not** call back into the brain to
|
|
1153
|
-
purge vectors. Use the UI Delete dialog before removing the mount, or
|
|
1154
|
-
run `Repair` afterwards.
|
|
1155
|
-
|
|
1156
|
-
### Vector Health (janitor)
|
|
1157
|
-
|
|
1158
|
-
`VectorHealthService` is the project-scoped janitor. It does NOT do
|
|
1159
|
-
per-source mass-delete (that lives on the source-delete path above).
|
|
1160
|
-
|
|
1161
|
-
It exposes two cost tiers, so opening the Files UI never triggers a
|
|
1162
|
-
collection walk:
|
|
1163
|
-
|
|
1164
|
-
- `getStats(scope)` — **cheap, count-only** (the passive `GET/POST /health`).
|
|
1165
|
-
total / document / chunk / deleted / per-source counts via `countByFilter`
|
|
1166
|
-
on the indexed `category` / `status` / `folder_ancestors` fields — **no
|
|
1167
|
-
scroll**, instant at millions of points. The document-vs-chunk discriminator
|
|
1168
|
-
is the indexed `category==='chunk'`; per-source counts are config-derived
|
|
1169
|
-
(`folder_ancestors='<slug>://'`). The `VectorHealthPanel` fetches this on
|
|
1170
|
-
expand (it is mounted only when the section is open).
|
|
1171
|
-
- `scanIntegrity(scope)` — **expensive, on-demand** (the explicit
|
|
1172
|
-
`POST /health/scan`, the panel's "Scan for problems" button). The ONE path
|
|
1173
|
-
that walks every point: orphan-chunk + duplicate-document detection.
|
|
1174
|
-
|
|
1175
|
-
The GC actions — **`POST /health/gc` only, and gated one tier above the rest of
|
|
1176
|
-
the route.** The module feature is `files.read`, which every role holds down to
|
|
1177
|
-
`viewer`, and that is right for the two read arms above; `gc` and
|
|
1178
|
-
`repair-orphans` are mutations whose `dryRun` defaults to **false**, so the
|
|
1179
|
-
sub-path escalates in-handler to **`files.write`** and answers a caller without
|
|
1180
|
-
it `403 Missing feature: files.write (vector health repair)`. `deduplicate` is
|
|
1181
|
-
read-only but rides the same gate — its rows enumerate other agents' `data://`
|
|
1182
|
-
paths. `GET /health/gc` answers **405**, so a link or a prefetch can never start
|
|
1183
|
-
a deletion; it declares `Allow: POST` (the host catch-all forwards route headers
|
|
1184
|
-
only on content-type-bearing responses, so it is not on the wire yet). The arm is
|
|
1185
|
-
fenced, not removed: a `files.write` holder (manager tier and up) runs every
|
|
1186
|
-
action exactly as before, and the Vector Health panel already posts.
|
|
1187
|
-
`action:"repair-pending"` (dry-run supported in the panel) works on the CALLER's own
|
|
1188
|
-
visible pending records, never the project scope: it counts orphaned and
|
|
1189
|
-
older unknown-original records and, applied, discards the orphans through the
|
|
1190
|
-
per-record discard (WRITE re-authorized per record); the unknown-original ones
|
|
1191
|
-
are only counted — they are confirmed in the Changes panel, and nothing is
|
|
1192
|
-
reindexed (a sync would index an unapproved draft as active).
|
|
1193
|
-
|
|
1194
|
-
The two destructive passes live in `usecases/vectorMaintenance.ts`; the service
|
|
1195
|
-
methods are their one entry, for the route and the schedule alike, and every run
|
|
1196
|
-
logs `vector.gc.completed` / `vector.repair-orphans.completed` with its
|
|
1197
|
-
`principal` and `trigger` (`route` = the caller, `schedule` = the host's system
|
|
1198
|
-
identity).
|
|
1199
|
-
|
|
1200
|
-
- `garbageCollect(scope, { retentionHours?, dryRun?, run })` — drops records
|
|
1201
|
-
marked `status: 'deleted'` that are older than the retention window
|
|
1202
|
-
(default 24 h, `vectorGcRetentionHours`) in ONE server-side filter delete per
|
|
1203
|
-
project: the filter re-checks `status` at delete time, so a tombstone a revert
|
|
1204
|
-
revived in the meantime stays. A tombstone an **unresolved pending change**
|
|
1205
|
-
references (by point id or by uri) is kept, because revert reads it back — the
|
|
1206
|
-
answer reports it as `skippedPending`, and a pending journal that cannot be
|
|
1207
|
-
read refuses the pass instead of deleting without the exclusion. `removed` is
|
|
1208
|
-
counted just before the delete (the port's delete returns no usable count), so
|
|
1209
|
-
a tombstone revived in that same moment is kept but still counted. Two
|
|
1210
|
-
consequences worth knowing: a file deleted just now is **not** GC-able yet —
|
|
1211
|
-
the immediate way to resolve it is the Changes panel — and GC emits
|
|
1212
|
-
**no filesystem event**, so the file tree drops the struck-through rows on its
|
|
1213
|
-
next reconcile or Refresh, not the moment the GC finishes.
|
|
1214
|
-
- **The scheduled GC** runs the same pass for every project every
|
|
1215
|
-
`vectorGcIntervalHours` (default 24; 0 turns the schedule off, the manual
|
|
1216
|
-
action stays) as explicit **system work** — on the sync scheduler's existing
|
|
1217
|
-
tick, which fires even when auto-sync is `off` (GC is not sync), under the
|
|
1218
|
-
host's system identity, never a user's; one project failing is logged
|
|
1219
|
-
(`vector.gc.failed`) and the next still runs; a project whose GC is already
|
|
1220
|
-
running (a manual pass) is skipped with `vector.gc.skipped`, and then the
|
|
1221
|
-
interval is NOT consumed — the next scheduler tick runs the pass again. The interval clock is
|
|
1222
|
-
in-process and restarts on every boot, so the first scheduled pass runs ONE
|
|
1223
|
-
INTERVAL after boot, never at boot — a newly deployed schedule leaves time to
|
|
1224
|
-
change the key before it deletes anything; the manual action is immediate.
|
|
1225
|
-
- `repairOrphans(scope, { dryRun?, run })` — drops chunks whose `parent_id`
|
|
1226
|
-
is missing from the index or points at a parent flagged deleted. It reads the
|
|
1227
|
-
chunks' `parent_id` only, looks the parents up and deletes in id batches of
|
|
1228
|
-
`vectorHealthBatchSize` — never one call per point.
|
|
1229
|
-
- `findDuplicates(scope)` — read-only; returns groups of documents
|
|
1230
|
-
sharing a `content_hash`. Nothing is removed.
|
|
1231
|
-
|
|
1232
|
-
GC and orphan repair never run twice at once for the same project: a second
|
|
1233
|
-
request for the pass that is already running on that project answers 409
|
|
1234
|
-
`action_in_progress`. Another project, the other pass, and a collection action
|
|
1235
|
-
are independent (a long backfill does not block GC).
|
|
1236
|
-
|
|
1237
|
-
The UI panel (`VectorHealthPanel.tsx`) ships with `dryRun=false` by
|
|
1238
|
-
default plus an inline 2-click confirm on destructive buttons; the
|
|
1239
|
-
previous `dryRun=true` default made every click a no-op.
|
|
1240
|
-
Before/after deltas appear on the Deleted / Orphans stat cards after
|
|
1241
|
-
GC / Repair completes.
|
|
1242
|
-
|
|
1243
|
-
**The write monitor — which files the index keeps rewriting.** Every point write
|
|
1244
|
-
passes the one write boundary (`vector/FilesystemVectorIndex`), which reports it
|
|
1245
|
-
to `usecases/writeMonitor.ts`: a rolling one-hour count per uri and project —
|
|
1246
|
-
`writes` (document-point writes: a file write, a tombstone, a payload refresh)
|
|
1247
|
-
and `points` (everything written, chunks included). It is bounded to
|
|
1248
|
-
`vectorWriteMonitorMaxUris` uris per project (least recently written dropped
|
|
1249
|
-
first). A uri reaching `vectorWriteHotUriThreshold` writes in the hour logs ONE
|
|
1250
|
-
server-side `vector.write.hot-uri` warn per hour. `GET /health/hot-uris` lists the
|
|
1251
|
-
top `vectorWriteHotUriTopN` for the caller: it takes the same `files.write` tier as
|
|
1252
|
-
`gc` (the list names files across the project's users and agents) and is then
|
|
1253
|
-
filtered per uri exactly like a file listing — the caller's source scope, the
|
|
1254
|
-
uri's READ policy, and agent isolation for `brain://` — before it is ranked and
|
|
1255
|
-
cut, so a hot uri is visible to whoever may read it and to no one else. The
|
|
1256
|
-
counts are in-process: they start empty after a restart.
|
|
1257
|
-
|
|
1258
|
-
**Collection maintenance — the operator window (`GET|POST /health/collection`).**
|
|
1259
|
-
One Qdrant collection serves every project, so its collection-level work takes
|
|
1260
|
-
a third tier: the platform feature **`platform.vector.maintain`** (no default
|
|
1261
|
-
grant — the owner holds it through `'*'`; grantable to a custom role), checked
|
|
1262
|
-
in-handler like `gc`. `GET` returns the report — status, points, segments, the
|
|
1263
|
-
declared indexes the server is missing, the fields whose ECHOED params differ
|
|
1264
|
-
from their declaration, retired indexes still present, the quantization config,
|
|
1265
|
-
and whether the ranked-text vector exists and how many text-bearing points still
|
|
1266
|
-
lack it. `POST {action}` runs ONE action, reading its settings live
|
|
1267
|
-
(`usecases/vectorMaintenance.ts` `runCollectionAction`):
|
|
1268
|
-
|
|
1269
|
-
- `migrate-indexes` — delete + recreate every declared payload index with its
|
|
1270
|
-
full params (`enable_hnsw` per row: per-value HNSW subgraphs only on
|
|
1271
|
-
`projectId` (`is_tenant`), `userId` and `folder_ancestors`, the fields a vector
|
|
1272
|
-
search pre-filters on) and drop the retired `projectName` / `agentName`
|
|
1273
|
-
indexes. Each field is unindexed for the seconds its own recreate takes; the
|
|
1274
|
-
graph edges follow at the next segment rebuild. Boot never does this — it
|
|
1275
|
-
creates only missing indexes, still delete+recreates a TEXT index whose
|
|
1276
|
-
tokenizer params drifted (the existing reconcile), and for every other
|
|
1277
|
-
difference only warns `vector.payload-index.params-unapplied`.
|
|
1278
|
-
- `apply-tuning` — collection PATCH from `vectorDefaultSegmentNumber`,
|
|
1279
|
-
`vectorMaxSegmentSizeKb`, `vectorMaxIndexingThreads` (0 = leave the server's
|
|
1280
|
-
value).
|
|
1281
|
-
- `apply-quantization` — `vectorQuantizationMode` (`off` | `scalar-int8` |
|
|
1282
|
-
`binary-1bit` | `binary-1.5bit` | `binary-2bit` | `turbo-4bit`); any mode but
|
|
1283
|
-
`off` also moves the original vectors on disk, and every dense search then
|
|
1284
|
-
re-scores its candidates with them; `off` is the full rollback — quantization
|
|
1285
|
-
disabled and the originals back in RAM. The server rebuilds every segment.
|
|
1286
|
-
Only one collection action runs at a time: a second request while one runs
|
|
1287
|
-
answers 409 `action_in_progress`.
|
|
1288
|
-
- `enable-text-ranking` — adds the sparse BM25 vector (`bm25`, server-side
|
|
1289
|
-
model `qdrant/bm25`) to an OLDER collection. A collection created by this
|
|
1290
|
-
version on a server that runs BM25 inference (Qdrant ≥ 1.15.2, from the
|
|
1291
|
-
version boot already reads — no extra request) carries it from the first point
|
|
1292
|
-
(every write then costs one server-side BM25 inference beside the dense
|
|
1293
|
-
upsert), so on a fresh install this action answers `created: false`; on an
|
|
1294
|
-
older or unreadable server version the collection is created without it. From then on every write
|
|
1295
|
-
carries it and the hybrid search's text leg is RANKED by it; a point without
|
|
1296
|
-
it (not backfilled yet, or no inline content) still matches through the old
|
|
1297
|
-
filter leg, so nothing drops out while the backfill runs.
|
|
1298
|
-
- `backfill-text-ranking {cursor?, batches?}` — up to `batches` (≤ 50) bounded
|
|
1299
|
-
batches of `vectorHealthBatchSize` points, sparse vector only, answering
|
|
1300
|
-
`nextCursor` until `done`.
|
|
1301
|
-
|
|
1302
|
-
Validation errors are 400, an in-memory backend (no collection) or a backfill
|
|
1303
|
-
before `enable-text-ranking` is 409, a backend failure is a fixed 502 (detail in
|
|
1304
|
-
the vector log). Boot also reads the Qdrant server version and warns
|
|
1305
|
-
`vector.qdrant.version-gap` when the pinned client is more than one minor away —
|
|
1306
|
-
the admin Vector tab shows the same comparison; neither ever switches backend.
|
|
1307
|
-
|
|
1308
|
-
### Manifest-baseline merge
|
|
1309
|
-
|
|
1310
|
-
`BridgeService.createSource` unions every loaded package's manifest
|
|
1311
|
-
`uriPolicies` (resolved against `<self>` and filtered to the source URI's
|
|
1312
|
-
scheme) into the persisted source-config JSON at first attach. From then
|
|
1313
|
-
on, the source-config JSON is authoritative — the manifest is reseeded
|
|
1314
|
-
only when a source is added or replaced. Brain-core's
|
|
1315
|
-
`defaultPermissions.ts` is generic (`{ default { read: true, write: true,
|
|
1316
|
-
exec: false }, paths: [] }`); foreign-package patterns no longer live
|
|
1317
|
-
inside brain-core.
|
|
1318
|
-
|
|
1319
|
-
**Immutable policy floor (trusted-manifest-only).** A manifest path rule may carry
|
|
1320
|
-
the restrict-only kernel marker `floor:true` (`read/write/exec:false` only). Its
|
|
1321
|
-
provenance is trusted manifests ONLY: `createSource` rejects a caller-supplied
|
|
1322
|
-
`floor` (`CallerPolicyFloorError`) before the baseline merge, the policy editor and
|
|
1323
|
-
the full-permissions PATCH refuse to create/remove/alter one
|
|
1324
|
-
(`ImmutablePolicyFloorError`), and `repairSourceBaselines` v3 re-asserts the exact
|
|
1325
|
-
trusted set on every boot (above). `permissionResolver` DELEGATES to the kernel
|
|
1326
|
-
evaluator (it is an adapter, not a second implementation — retired 2026-08-19), so
|
|
1327
|
-
the floor-last pass is the kernel's own and a floor deny survives any later
|
|
1328
|
-
`override:true` by construction rather than by mirroring. Agent-core uses
|
|
1329
|
-
this to protect the conversation raw files (`messages.jsonl`, `meta.json`,
|
|
1330
|
-
`SUMMARY.md`, legacy `summary.md`, `_summary-revisions/**`) from every `fs_*`/route
|
|
1331
|
-
write — only its internal store may write them. Separately, `_summary-revisions/**`
|
|
1332
|
-
is in the runtime **sync floor** (`DEFAULT_QDRANT_SYNC_EXCLUDES`): the immutable
|
|
1333
|
-
authoritative summary revisions are never vectorized or source-package-scanned,
|
|
1334
|
-
while the active `SUMMARY.md` projection stays indexable. The same floor carries
|
|
1335
|
-
the package-manager content stores — `node_modules`, `.pnpm-store`, `.yarn/cache`,
|
|
1336
|
-
root and nested — because an in-container `pnpm install` on a bind-mounted
|
|
1337
|
-
repository lands `.pnpm-store/` at the repo root (5 124 files on one mounted
|
|
1338
|
-
package, 2026-08-27), and a source's `sync.exclude` is ADDITIVE over this floor,
|
|
1339
|
-
so every existing source inherits a new floor entry at its next sync. Pattern/media exclusions remove obsolete parent/chunk search data. Recovery-bearing parents — pending/deleted records, saved baselines and sole-copy `brain://` content — retain their complete payload with `recovery_only:true`, no embedding stamp and no dense or BM25 vectors. Retrieval, browsing and review remain policy-gated; search, enrichment and source-package discovery exclude retained recovery points. Verified source input permits ordinary payload indexing again; missing/stale embedding debt remains explicit until real dense completion. Enabled/auto inactivity never purges an index, and an immutable READ floor alone never authorizes cleanup. Generation rebuilding classifies current source policy before provider work and again at its config-coordinated swap: unknown provenance or unavailable/invalid configuration pauses without swapping. The dollar estimate samples only eligible content and exposes protected/excluded/unresolved counts; it remains a range, not a hard spending bound. Full sync runs this cleanup at its existing end; targeted sync never purges.
|
|
1340
|
-
|
|
1341
|
-
**The READ FLOOR is the INDEX FLOOR.** Since 2026-08-05 the immutable `floor: true`
|
|
1342
|
-
rows govern a THIRD axis: `isUriSyncIndexable` evaluates them alongside the
|
|
1343
|
-
exclude/include patterns, so a path no principal may READ is never newly
|
|
1344
|
-
EMBEDDED. This is derived, not enumerated — every floored path of every package
|
|
1345
|
-
is covered with zero new contract surface — and it deliberately replaces the
|
|
1346
|
-
package-declared "sync-exclude floor" that was once proposed: a sync-exclude is a
|
|
1347
|
-
user PREFERENCE (`syncExcludes` stays editable and un-gated); the uri policy
|
|
1348
|
-
is where security lives. Four rules keep it honest, each measured before it was
|
|
1349
|
-
written: **floors only**, never the effective per-role permission (a per-role deny
|
|
1350
|
-
means somebody CAN read it, so it stays indexed and the query-time per-caller
|
|
1351
|
-
filter decides — evaluating effective permissions would strip ~42 % of the live
|
|
1352
|
-
index for every non-owner, the owner's own search included); the **`read` bit
|
|
1353
|
-
only** (every agent-core `data` floor is `{write:false}` and covers 27 047 live
|
|
1354
|
-
points, all of which must stay searchable); **`$self` rows are skipped** as
|
|
1355
|
-
principal-relative (the index is one shared collection) and the skip is logged;
|
|
1356
|
-
and **fail closed** — an unresolvable source config is not indexed. It applies to
|
|
1357
|
-
the sync walk in BOTH modes, to every indexing `write` verb, and — as an
|
|
1358
|
-
outright refusal on a read-floored upload — to `writeUpload`. Pattern-excluded uploads remain journaled and return `indexed:false`.
|
|
1359
|
-
Unlike the patterns (which a full pass purges on), the floor governs only what is
|
|
1360
|
-
NEWLY embedded: a point already in Qdrant when a floor arrives is not purged,
|
|
1361
|
-
though the query-time read filters hide it from every caller. See `src/domain/README.md` for the mechanism and the
|
|
1362
|
-
per-source measurement behind it.
|
|
1363
|
-
|
|
1364
|
-
### Declarative source instances + `origin_package` attribution
|
|
1365
|
-
|
|
1366
|
-
A package declares the concrete source INSTANCES it owns in its manifest
|
|
1367
|
-
`neuralis.sources[]` (`PackageDefinition.sources?: PackageSourceDeclaration[]`,
|
|
1368
|
-
distinct from the singular provenance `source` and from `connectors`, which
|
|
1369
|
-
register source KINDS). Agent-core aggregates every loaded package's `sources[]`
|
|
1370
|
-
into `ctx.platform.packageSourceDeclarations()` keyed by the declaring
|
|
1371
|
-
package id, exactly like `packageUriPolicies`; brain-core reads it lazily where used,
|
|
1372
|
-
without copying it into state. A package may declare a source on a kind it does NOT own
|
|
1373
|
-
— admin declares the platform app zone on brain-core's `local` kind.
|
|
1374
|
-
|
|
1375
|
-
**`origin_package` is contract-backed, unspoofable attribution** — the
|
|
1376
|
-
DECLARING package id, resolved at runtime by `matchDeclaredSource`
|
|
1377
|
-
(`src/services/declaredSources.ts`). For ROOTED `local` kinds it requires the
|
|
1378
|
-
declared slug AND the RESOLVED root to match (a user mount merely named `app`
|
|
1379
|
-
but NOT rooted at the platform app zone is NOT attributed — it stays
|
|
1380
|
-
unattributed); for ROOTLESS sandbox kinds (`brain` / `webtop`) it matches by
|
|
1381
|
-
KIND alone, since their slugs are dynamic (e.g. per-user `machine-<id>`). This
|
|
1382
|
-
SUBSUMES the old kind-registry `contributedByPackage` heuristic. `origin_package`
|
|
1383
|
-
is exposed on the `GET /sources` route and on every `SourceRuntimeSummary` (and
|
|
1384
|
-
mirrored in agent-core's `BrainSourceRuntimeSummary`), where `renderRuntimeStack`
|
|
1385
|
-
renders it as the `[pkg: …]` tag in the `<runtime_stack>` source table.
|
|
1386
|
-
|
|
1387
|
-
The source **description** follows ONE fallback chain — the authored text, else
|
|
1388
|
-
what the owning package DECLARED for that source, else what the connector KIND
|
|
1389
|
-
says it is, else nothing — implemented once as `resolveSourceDescription`
|
|
1390
|
-
(`domain/sourceConfig.ts`) and called by every reader: both `<runtime_stack>`
|
|
1391
|
-
rows (the config-backed sources and the implicit `brain://` row, which used to
|
|
1392
|
-
skip the config and the kind entirely), `GET /sources`, the single-source GET,
|
|
1393
|
-
and — through `listSourceRuntimeSummaries` — the admin Config editor. **Blank or
|
|
1394
|
-
whitespace counts as ABSENT at every level**, so clearing the field falls back to
|
|
1395
|
-
the package's wording rather than showing the agent an empty row.
|
|
1396
|
-
|
|
1397
|
-
It is **resolved per read, never persisted** (the derived value rides as
|
|
1398
|
-
`description_effective`; `description` stays exactly what is on disk). A copied
|
|
1399
|
-
default would freeze: upgrade the package that owns the text and every source
|
|
1400
|
-
that copied the old wording keeps it forever. The one deliberate exception is
|
|
1401
|
-
`repairDeclaredSources`, which backfills a blank description on a DECLARED source
|
|
1402
|
-
once per repair version — so a first-party source's persisted config is authored;
|
|
1403
|
-
the chain covers the remainder, i.e. hand-added sources and any description a
|
|
1404
|
-
human has cleared.
|
|
1405
|
-
|
|
1406
|
-
**Three-way repair split.** Three independent, version-stamped passes touch a
|
|
1407
|
-
source config, and order matters because the last two full-object
|
|
1408
|
-
read-modify-write the SAME JSON:
|
|
1409
|
-
|
|
1410
|
-
1. **first-attach baselines** — `BridgeService.createSource` unions the manifest
|
|
1411
|
-
`uriPolicies` into a NEW config at attach time (above).
|
|
1412
|
-
2. **`repairSourceBaselines`** (`SOURCE_BASELINE_REPAIR_VERSION = 5` — v4 carried
|
|
1413
|
-
the log-read rows, v5 agent-core's creator-only conversation rows; both are
|
|
1414
|
-
role- or requester-scoped and therefore NON-floor, so only the version-gated
|
|
1415
|
-
union can carry them to existing projects) —
|
|
1416
|
-
re-unions live manifest baselines into EXISTING `data`/`packages` configs so a
|
|
1417
|
-
new security-floor pattern reaches pre-existing projects. It also runs the kernel
|
|
1418
|
-
`reconcileImmutablePolicyFloors` on EVERY boot even after the version stamp is
|
|
1419
|
-
current: a persisted `floor:true` marker that has gone missing or been altered is
|
|
1420
|
-
restored to the exact trusted-manifest set (mutable/non-floor rules stay
|
|
1421
|
-
byte-untouched), so an immutable floor (e.g. the agent-core conversation raw-file
|
|
1422
|
-
write-deny) is never a "ran once, assume present" hole.
|
|
1423
|
-
|
|
1424
|
-
**The required floor set is a UNION of two sources, assembled before ONE
|
|
1425
|
-
reconcile call.** `reconcileImmutablePolicyFloors` REPLACES the whole persisted
|
|
1426
|
-
floor class with what its `trustedBaselines` argument names — it does not merge —
|
|
1427
|
-
so calling it once per source of truth deletes the other one's floors, silently
|
|
1428
|
-
(this pass is `void`-fired and its catch only warns) and persistently. The two:
|
|
1429
|
-
(a) the scheme-keyed manifest `uriPolicies`, `data`/`packages` only, and
|
|
1430
|
-
(b) the source's CONNECTOR-declared `floor:true` rows, keyed on
|
|
1431
|
-
`connection.kind` and scheme-INDEPENDENT — which is what v3 added, and the only
|
|
1432
|
-
channel a connector on a foreign scheme (`machine://`) has, since manifest
|
|
1433
|
-
`uriPolicies` are hard-wired to `data`/`packages` at four gates. Both are derived
|
|
1434
|
-
through the same `pickConnectorFloorRules` / `pickManifestBaselinesForSource` the
|
|
1435
|
-
create path uses, so the floor signatures match and the repair is a no-op on the
|
|
1436
|
-
second boot. If the connector-floor lookup cannot be resolved, the source is
|
|
1437
|
-
skipped WITHOUT a stamp so the next boot retries; and a boot where NEITHER
|
|
1438
|
-
source yields anything leaves the persisted floors alone rather than stripping
|
|
1439
|
-
them.
|
|
1440
|
-
3. **`repairDeclaredSources`** — one-time backfill for existing projects: fill a
|
|
1441
|
-
blank `description` from the matched declaration (NEVER overwrites a user edit)
|
|
1442
|
-
and create missing `seed:'auto'` project sources. A `data` source it creates —
|
|
1443
|
-
identified as brain-core's OWN declaration of the `data` root, never by slug —
|
|
1444
|
-
also carries the kernel's machine-written seed (below).
|
|
1445
|
-
|
|
1446
|
-
At `onBootstrapComplete` the passes run as ONE **SERIALIZED** lane,
|
|
1447
|
-
`runBootSourceConfigRepairs` (`services/declaredSources.ts`): pass 3 FIRST, then
|
|
1448
|
-
pass 2, then the exec-grant pass — never a second `void`-fire, because a
|
|
1449
|
-
parallel fire would lose-update the shared configs. A source pass 3 CREATES is
|
|
1450
|
-
written complete: before its first write it goes through the kernel's
|
|
1451
|
-
`applyTrustedSourceBaselines` — the computation pass 2 applies — and is stamped,
|
|
1452
|
-
so it is never persisted without its baselines and floors, and pass 2 leaves it
|
|
1453
|
-
alone. If its connector floors cannot be resolved, or a `data`/`packages` source
|
|
1454
|
-
finds no manifest baseline at all, it is not created and the next boot retries.
|
|
1455
|
-
Each pass carries its own version stamp (`SourceConfig.declaredRepairVersion`
|
|
1456
|
-
/ `DECLARED_SOURCE_REPAIR_VERSION` for pass 3); none is fatal to bootstrap. A
|
|
1457
|
-
project created from the app gets the same sources from brain-core's
|
|
1458
|
-
`provisionProject` hook, through the same create path
|
|
1459
|
-
(`createMissingDeclaredSources`) — complete in its first write, with missing
|
|
1460
|
-
connectors bound into cached, pending or pinned project infra before completion.
|
|
1461
|
-
Cold projects remain lazy and build from the persisted config on first access.
|
|
1462
|
-
Existing configs are preserved on binding retry. Boot awaits this repair lane
|
|
1463
|
-
before embedding, pending purge and scheduling; failures remain logged and
|
|
1464
|
-
nonfatal. A new project is refused and rolled back on persistence or binding failure; the host names no source. A fresh install's first project reaches pass 3:
|
|
1465
|
-
the setup wizard writes only its `packages` and `brain` sources, and the first
|
|
1466
|
-
boot creates `data`.
|
|
1467
|
-
|
|
1468
|
-
**The machine-written seed.** A package declares the files it writes on its own
|
|
1469
|
-
in `dataLayout.machineWritten[]` (`{ path, class }`, zone-relative, validated by
|
|
1470
|
-
the kernel). A NEW project's `data` source appends the anchored globs of the
|
|
1471
|
-
classes the kernel names in `SEED_EXCLUDED_MACHINE_WRITTEN_CLASSES` — today
|
|
1472
|
-
`metrics` only (the agent usage directory and `codex-usage.json`) — to its
|
|
1473
|
-
declared `sync.exclude`, once: `provisionProject` and `repairDeclaredSources`
|
|
1474
|
-
both derive them through `ctx.platform.machineWrittenPaths()` — the package set
|
|
1475
|
-
as of boot, never refreshed by a later install — for a `data` source they
|
|
1476
|
-
create.
|
|
1477
|
-
Logs keep the declared `**/logs/**` / `**/audit/**` preferences; runtime text
|
|
1478
|
-
(conversation titles, shell output, workflow events) stays indexed. It is a
|
|
1479
|
-
preference seed, never a floor: an existing config is never rewritten, and the
|
|
1480
|
-
owner edits the list like any other exclude. For an attested own `data` source without a sync block, Sources offers the same metrics exclusions when syncing is enabled; they persist only on Save. Custom sources and existing explicit-empty lists are unchanged.
|
|
1481
|
-
|
|
1482
|
-
### Discoverable sources — declared ∪ host-mount
|
|
1483
|
-
|
|
1484
|
-
`GET /sources/discoverable` returns a UNION of two suggestion families:
|
|
1485
|
-
|
|
1486
|
-
- **declared `seed:'discoverable'` LOCAL sources** — a package surfaces an
|
|
1487
|
-
on-demand source on a registered local kind. Admin's app zone is the first
|
|
1488
|
-
(suggestion id `discover:decl:app`, replacing the former hand-built
|
|
1489
|
-
`discover:app` block); the host resolves its `${appRoot}` root token. The
|
|
1490
|
-
declared `label`/`description` prefill the add form. These carry a real
|
|
1491
|
-
`origin_package`.
|
|
1492
|
-
- **host-mount suggestions** — the Neuralis tree (`discover:neuralis`), the
|
|
1493
|
-
projects zone, the whole host, and generic `NEURALIS_MOUNT_*` binds. These
|
|
1494
|
-
depend on a runtime mount, NOT a package, so `origin_package` is null.
|
|
1495
|
-
|
|
1496
|
-
The protected-root SEE filter and the `handleSourcesCreate`
|
|
1497
|
-
ADD gate remain AUTHORITATIVE for both families; a declaration's
|
|
1498
|
-
`visibility.seeRequires` is defense-in-depth, unioned into the SEE filter, never
|
|
1499
|
-
the boundary.
|
|
1500
|
-
|
|
1501
|
-
The package declares the optional singleton `package-source-roots` service, delegating to its existing source-root enumerator with scope, enabled state and package-recognition flags intact, plus a change subscription off the one source-config commit funnel. Project infrastructure uses the loader's bounded project factory; running jobs and event subscriptions pin the same infrastructure through idle eviction. Permanent purge fences new queue work and waits for existing writes before vector deletion, subscription disposal and cache invalidation. Shutdown `drain` stops background admission and awaits completion bookkeeping under the runtime bound.
|
|
165
|
+
The source lifecycle (the `enabled` flag, live probes, delete + purge, manifest baselines and
|
|
166
|
+
`floor:true` rows, declared and discoverable sources) is in `src/services/README.md`; Vector
|
|
167
|
+
Health is in `src/vector/README.md`; `fs_read` vision blocks are in `src/tools/README.md`.
|
|
1502
168
|
|
|
1503
169
|
## Gotchas
|
|
1504
170
|
|
|
@@ -1555,10 +221,14 @@ embedding clients (`test/embeddings`), search modes including `grep`
|
|
|
1555
221
|
(`test/tools`), the list pipeline (`test/list`), `permissionResolver`
|
|
1556
222
|
(`test/permissionResolver.test.ts`), use cases (`test/usecases`), service
|
|
1557
223
|
shapes (`test/services`), bug-fix regression cases (`test/bugfix`), and the
|
|
1558
|
-
UI store (`test/ui`). Run with `pnpm --filter @neuralis/brain-core test
|
|
224
|
+
UI store (`test/ui`). Run with `pnpm --filter @neuralis/brain-core test`;
|
|
225
|
+
vitest does not typecheck, so `pnpm typecheck:test` (`test/tsconfig.json`) is the
|
|
226
|
+
tree's type gate.
|
|
1559
227
|
`test/services/freshInstallFirstBoot.test.ts` imports the setup wizard's writer
|
|
1560
228
|
from `neuralis/scripts/setup/records.mts` (a monorepo-only coupling): moving or
|
|
1561
|
-
renaming that script breaks this test.
|
|
229
|
+
renaming that script breaks this test. That import pulls the host's own types
|
|
230
|
+
(its `@/` alias, its build scripts, the `NODE_ENV` it declares) into any program
|
|
231
|
+
that holds it, so `test/tsconfig.json` leaves this one file out.
|
|
1562
232
|
|
|
1563
233
|
## Team
|
|
1564
234
|
|
|
@@ -1575,7 +245,7 @@ predictably and read like a stable contract:
|
|
|
1575
245
|
resume state not already owned by `<session>` or memory. The fresh seed's
|
|
1576
246
|
governed-data onboarding reminder is removed once completed or declined.
|
|
1577
247
|
|
|
1578
|
-
**Neither shipped team member pins `config.settings.maxSteps`.** A per-agent pin can only LOWER the live platform ceiling (`AgentService.getRuntimeSettings` → `Math.min(pin, maxAgentSteps)`), never raise it, and an unpinned agent runs AT the ceiling —
|
|
248
|
+
**Neither shipped team member pins `config.settings.maxSteps`.** A per-agent pin can only LOWER the live platform ceiling (`AgentService.getRuntimeSettings` → `Math.min(pin, maxAgentSteps)`), never raise it, and an unpinned agent runs AT the ceiling — 2000 out of the box, operator-adjustable through `maxAgentSteps`. Admin and Coder are the two agents that are meant to be able to finish long work, so they follow the platform rather than shadowing it; the old 50/60 literals capped exactly the runs that needed the room. Pinned by `packages/agent-core/packages/__tests__/teamTemplateContracts.test.ts`. An ALREADY-PROVISIONED agent keeps the value baked at provision time, and `manage-agents`' template resync does NOT remove it (measured): a settings patch merges key-by-key, so the template's now-empty `settings` cannot delete a stored key. Clearing the pin is the Agent Config card's max-steps field emptied — which sends an explicit `null`, the clear verb `AgentStore.mergeSettings` reads. The operational consequence to state, never discover: raising `maxAgentSteps` does NOT reach an agent that carries a provision-time pin — the pin must be cleared first, or the raise frees every agent except the one it was meant for.
|
|
1579
249
|
|
|
1580
250
|
Identity files are writable through `fs_write` against
|
|
1581
251
|
`neuralis://packages/brain-core/team/coder/identity/...` (when the
|