@harperfast/harper-pro 5.0.25 → 5.1.0-beta.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (927) hide show
  1. package/analytics/profile.ts +4 -0
  2. package/core/AGENTS.md +92 -6
  3. package/core/DESIGN.md +24 -0
  4. package/core/README.md +11 -10
  5. package/core/agent/agent.ts +203 -0
  6. package/core/agent/loop.ts +205 -0
  7. package/core/agent/operations.ts +148 -0
  8. package/core/agent/session.ts +187 -0
  9. package/core/agent/tools/fsTools.ts +276 -0
  10. package/core/agent/tools/httpFetchTool.ts +112 -0
  11. package/core/agent/tools/scheduleTool.ts +68 -0
  12. package/core/agent/toolset.ts +43 -0
  13. package/core/agent/types.ts +85 -0
  14. package/core/benchmarks/hnsw-search.js +157 -0
  15. package/core/benchmarks/ycsb/README.md +144 -0
  16. package/core/benchmarks/ycsb/analyze-profile.mts +120 -0
  17. package/core/benchmarks/ycsb/app/config.yaml +3 -0
  18. package/core/benchmarks/ycsb/app/schema.graphql +17 -0
  19. package/core/benchmarks/ycsb/harness.mts +328 -0
  20. package/core/benchmarks/ycsb/restClient.mts +120 -0
  21. package/core/benchmarks/ycsb/run-single-node.mts +97 -0
  22. package/core/benchmarks/ycsb/to-benchmark-json.mts +58 -0
  23. package/core/benchmarks/ycsb/workload.mts +499 -0
  24. package/core/benchmarks/ycsb/workload.test.mts +174 -0
  25. package/core/bin/{BinObjects.js → BinObjects.ts} +4 -5
  26. package/core/bin/cliCredentials.ts +133 -0
  27. package/core/bin/cliOperations.ts +339 -0
  28. package/core/bin/copyDb.ts +10 -10
  29. package/core/bin/deployRenderer.ts +196 -0
  30. package/core/bin/{harper.js → harper.ts} +48 -24
  31. package/core/bin/{install.js → install.ts} +3 -3
  32. package/core/bin/lite.ts +2 -0
  33. package/core/bin/login.ts +168 -0
  34. package/core/bin/logout.ts +11 -0
  35. package/core/bin/mcp/client.ts +407 -0
  36. package/core/bin/mcp/doctor.ts +189 -0
  37. package/core/bin/mcp/index.ts +80 -0
  38. package/core/bin/mcp/options.ts +122 -0
  39. package/core/bin/mcp/printConfig.ts +89 -0
  40. package/core/bin/multipartBuilder.ts +74 -0
  41. package/core/bin/{restart.js → restart.ts} +31 -32
  42. package/core/bin/{run.js → run.ts} +57 -46
  43. package/core/bin/sseConsumer.ts +126 -0
  44. package/core/bin/{status.js → status.ts} +10 -10
  45. package/core/bin/stop.ts +21 -0
  46. package/core/bin/upgrade.js +6 -6
  47. package/core/components/Application.ts +144 -18
  48. package/core/components/ApplicationScope.ts +2 -2
  49. package/core/components/ComponentV1.ts +2 -2
  50. package/core/components/EntryHandler.ts +159 -9
  51. package/core/components/OptionsWatcher.ts +75 -11
  52. package/core/components/Scope.ts +125 -15
  53. package/core/components/anthropic/index.ts +547 -0
  54. package/core/components/bedrock/index.ts +823 -0
  55. package/core/components/componentLoader.ts +63 -32
  56. package/core/components/deployLifecycle.ts +161 -0
  57. package/core/components/deploymentOperations.ts +173 -0
  58. package/core/components/deploymentRecorder.ts +402 -0
  59. package/core/components/deriveURLPath.ts +4 -4
  60. package/core/components/mcp/adapters/fastify.ts +87 -0
  61. package/core/components/mcp/adapters/harperHttp.ts +103 -0
  62. package/core/components/mcp/audit.ts +75 -0
  63. package/core/components/mcp/index.ts +134 -0
  64. package/core/components/mcp/jsonrpc.ts +134 -0
  65. package/core/components/mcp/lifecycle.ts +105 -0
  66. package/core/components/mcp/listChanged.ts +270 -0
  67. package/core/components/mcp/rateLimit.ts +217 -0
  68. package/core/components/mcp/resources.ts +593 -0
  69. package/core/components/mcp/session.ts +151 -0
  70. package/core/components/mcp/sessionRegistry.ts +140 -0
  71. package/core/components/mcp/toolRegistry.ts +292 -0
  72. package/core/components/mcp/tools/application.ts +603 -0
  73. package/core/components/mcp/tools/operations.ts +283 -0
  74. package/core/components/mcp/tools/schemas/derive.ts +256 -0
  75. package/core/components/mcp/tools/schemas/operations.ts +245 -0
  76. package/core/components/mcp/transport.ts +517 -0
  77. package/core/components/ollama/index.ts +316 -0
  78. package/core/components/openai/index.ts +563 -0
  79. package/core/components/operations.js +211 -60
  80. package/core/components/operationsValidation.js +3 -3
  81. package/core/components/packageComponent.ts +97 -29
  82. package/core/components/requestRestart.ts +17 -2
  83. package/core/components/status/crossThread.ts +14 -5
  84. package/core/components/status/errors.ts +1 -1
  85. package/core/config/RootConfigWatcher.ts +56 -2
  86. package/core/config/configUtils.js +29 -8
  87. package/core/config/harperConfigEnvVars.ts +1 -1
  88. package/core/dataLayer/{CreateAttributeObject.js → CreateAttributeObject.ts} +4 -3
  89. package/core/dataLayer/{CreateTableObject.js → CreateTableObject.ts} +2 -1
  90. package/core/dataLayer/{DataLayerObjects.js → DataLayerObjects.ts} +17 -9
  91. package/core/dataLayer/{DeleteBeforeObject.js → DeleteBeforeObject.ts} +2 -1
  92. package/core/dataLayer/{DeleteObject.js → DeleteObject.ts} +3 -2
  93. package/core/dataLayer/{DropAttributeObject.js → DropAttributeObject.ts} +2 -1
  94. package/core/dataLayer/{GetBackupObject.js → GetBackupObject.ts} +3 -2
  95. package/core/dataLayer/{InsertObject.js → InsertObject.ts} +3 -2
  96. package/core/dataLayer/{ReadAuditLogObject.js → ReadAuditLogObject.ts} +3 -2
  97. package/core/dataLayer/{SQLSearch.js → SQLSearch.ts} +97 -43
  98. package/core/dataLayer/{SearchByConditionsObject.js → SearchByConditionsObject.ts} +5 -6
  99. package/core/dataLayer/{SearchByHashObject.js → SearchByHashObject.ts} +2 -1
  100. package/core/dataLayer/{SearchObject.js → SearchObject.ts} +2 -1
  101. package/core/dataLayer/{SqlSearchObject.js → SqlSearchObject.ts} +2 -1
  102. package/core/dataLayer/{UpdateObject.js → UpdateObject.ts} +3 -2
  103. package/core/dataLayer/{UpsertObject.js → UpsertObject.ts} +3 -2
  104. package/core/dataLayer/{bulkLoad.js → bulkLoad.ts} +40 -49
  105. package/core/dataLayer/{delete.js → delete.ts} +21 -26
  106. package/core/dataLayer/{export.js → export.ts} +22 -26
  107. package/core/dataLayer/{getBackup.js → getBackup.ts} +7 -9
  108. package/core/dataLayer/harperBridge/BridgeMethods.ts +102 -0
  109. package/core/dataLayer/harperBridge/ResourceBridge.ts +27 -26
  110. package/core/dataLayer/harperBridge/TableSizeObject.ts +1 -0
  111. package/core/dataLayer/harperBridge/bridgeUtility/insertUpdateValidate.js +4 -4
  112. package/core/dataLayer/harperBridge/{harperBridge.js → harperBridge.ts} +3 -3
  113. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateAttribute.js +8 -6
  114. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateRecords.js +4 -4
  115. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateSchema.js +1 -1
  116. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateTable.js +6 -4
  117. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDeleteAuditLogsBefore.js +5 -4
  118. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDeleteRecords.js +4 -4
  119. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDropAttribute.js +6 -5
  120. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDropSchema.js +5 -4
  121. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDropTable.js +5 -5
  122. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbFlush.js +1 -1
  123. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbGetBackup.js +3 -3
  124. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbGetDataByHash.js +1 -1
  125. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbGetDataByValue.js +3 -2
  126. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbReadAuditLog.js +5 -5
  127. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbSearchByConditions.js +10 -8
  128. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbSearchByHash.js +1 -1
  129. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbSearchByValue.js +4 -3
  130. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbUpdateRecords.js +3 -3
  131. package/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbUpsertRecords.js +6 -5
  132. package/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/LMDBCreateAttributeObject.js +2 -1
  133. package/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/initializeHashSearch.js +3 -2
  134. package/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/initializePaths.js +2 -2
  135. package/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbCheckForNewAttributes.js +5 -4
  136. package/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbCreateTransactionsAuditEnvironment.js +6 -3
  137. package/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbGetTableSize.ts +1 -1
  138. package/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbProcessRows.js +4 -4
  139. package/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbSearch.js +5 -5
  140. package/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbWriteTransaction.js +4 -4
  141. package/core/dataLayer/{hdbInfoController.js → hdbInfoController.ts} +20 -26
  142. package/core/dataLayer/{insert.js → insert.ts} +24 -24
  143. package/core/dataLayer/{readAuditLog.js → readAuditLog.ts} +8 -10
  144. package/core/dataLayer/{schema.js → schema.ts} +32 -44
  145. package/core/dataLayer/{schemaDescribe.js → schemaDescribe.ts} +23 -26
  146. package/core/dataLayer/{search.js → search.ts} +9 -15
  147. package/core/dataLayer/{transaction.js → transaction.ts} +2 -5
  148. package/core/dataLayer/{update.js → update.ts} +19 -22
  149. package/core/index.ts +5 -0
  150. package/core/json/systemSchema.json +65 -0
  151. package/core/launchServiceScripts/utility/checkNodeVersion.js +2 -0
  152. package/core/package-lock.json +9051 -4318
  153. package/core/resources/DESIGN.md +98 -0
  154. package/core/resources/DatabaseTransaction.ts +58 -30
  155. package/core/resources/ErrorResource.ts +2 -1
  156. package/core/resources/LMDBTransaction.ts +38 -29
  157. package/core/resources/RecordEncoder.ts +12 -4
  158. package/core/resources/RequestTarget.ts +2 -0
  159. package/core/resources/Resource.ts +89 -56
  160. package/core/resources/ResourceInterface.ts +44 -21
  161. package/core/resources/Resources.ts +8 -8
  162. package/core/resources/RocksIndexStore.ts +3 -0
  163. package/core/resources/RocksTransactionLogStore.ts +47 -28
  164. package/core/resources/Table.ts +386 -214
  165. package/core/resources/analytics/metadata.ts +1 -0
  166. package/core/resources/analytics/read.ts +24 -6
  167. package/core/resources/analytics/write.ts +240 -17
  168. package/core/resources/auditStore.ts +28 -19
  169. package/core/resources/blob.ts +53 -53
  170. package/core/resources/dataLoader.ts +4 -4
  171. package/core/resources/databases.ts +190 -71
  172. package/core/resources/graphql.ts +227 -164
  173. package/core/resources/indexes/HierarchicalNavigableSmallWorld.ts +294 -65
  174. package/core/resources/indexes/vector.ts +17 -0
  175. package/core/resources/loadEnv.ts +21 -17
  176. package/core/resources/login.ts +5 -3
  177. package/core/resources/models/Models.ts +304 -0
  178. package/core/resources/models/TestBackend.ts +83 -0
  179. package/core/resources/models/agentLoop.ts +895 -0
  180. package/core/resources/models/analyticsTable.ts +199 -0
  181. package/core/resources/models/backendHelpers.ts +116 -0
  182. package/core/resources/models/backendRegistry.ts +66 -0
  183. package/core/resources/models/bootstrap.ts +135 -0
  184. package/core/resources/models/embedHook.ts +138 -0
  185. package/core/resources/models/types.ts +296 -0
  186. package/core/resources/replayLogs.ts +15 -7
  187. package/core/resources/roles.ts +62 -67
  188. package/core/resources/search.ts +355 -135
  189. package/core/resources/tracked.ts +8 -8
  190. package/core/resources/transaction.ts +8 -8
  191. package/core/resources/transactionBroadcast.ts +3 -3
  192. package/core/security/auth.ts +35 -26
  193. package/core/security/certificateVerification/crlVerification.ts +11 -4
  194. package/core/security/{cryptoHash.js → cryptoHash.ts} +3 -8
  195. package/core/security/data_objects/{PermissionAttributeResponseObject.js → PermissionAttributeResponseObject.ts} +4 -4
  196. package/core/security/data_objects/{PermissionResponseObject.js → PermissionResponseObject.ts} +12 -11
  197. package/core/security/data_objects/{PermissionTableResponseObject.js → PermissionTableResponseObject.ts} +6 -4
  198. package/core/security/{fastifyAuth.js → fastifyAuth.ts} +93 -20
  199. package/core/security/impersonation.ts +3 -3
  200. package/core/security/jsLoader.ts +22 -8
  201. package/core/security/{keys.js → keys.ts} +113 -121
  202. package/core/security/permissionsTranslator.js +2 -2
  203. package/core/security/{role.js → role.ts} +26 -33
  204. package/core/security/tokenAuthentication.ts +34 -7
  205. package/core/security/user.ts +26 -22
  206. package/core/server/DESIGN.md +139 -0
  207. package/core/server/DurableSubscriptionsSession.ts +67 -50
  208. package/core/server/REST.ts +120 -107
  209. package/core/server/Server.ts +31 -12
  210. package/core/server/fastifyRoutes/helpers/getCORSOptions.js +1 -1
  211. package/core/server/fastifyRoutes/helpers/getHeaderTimeoutConfig.js +1 -1
  212. package/core/server/fastifyRoutes/helpers/getServerOptions.js +1 -1
  213. package/core/server/fastifyRoutes.ts +37 -33
  214. package/core/server/graphqlQuerying.ts +6 -5
  215. package/core/server/http.ts +517 -26
  216. package/core/server/itc/serverHandlers.js +75 -14
  217. package/core/server/jobs/{JobObject.js → JobObject.ts} +13 -6
  218. package/core/server/jobs/{jobProcess.js → jobProcess.ts} +20 -16
  219. package/core/server/jobs/{jobRunner.js → jobRunner.ts} +20 -21
  220. package/core/server/jobs/{jobs.js → jobs.ts} +41 -44
  221. package/core/server/loadRootComponents.js +1 -1
  222. package/core/server/middlewareChain.ts +270 -0
  223. package/core/server/mqtt.ts +35 -26
  224. package/core/server/nodeName.ts +2 -1
  225. package/core/server/operationsServer.ts +85 -10
  226. package/core/server/serverHelpers/Headers.ts +10 -8
  227. package/core/server/serverHelpers/JSONStream.ts +15 -5
  228. package/core/server/serverHelpers/Request.ts +370 -13
  229. package/core/server/serverHelpers/contentTypes.ts +42 -11
  230. package/core/server/serverHelpers/multipartParser.ts +152 -0
  231. package/core/server/serverHelpers/progressEmitter.ts +110 -0
  232. package/core/server/serverHelpers/serverHandlers.js +43 -7
  233. package/core/server/serverHelpers/serverUtilities.ts +40 -30
  234. package/core/server/static.ts +9 -6
  235. package/core/server/status/index.ts +2 -2
  236. package/core/server/storageReclamation.ts +39 -2
  237. package/core/server/threads/itc.js +7 -4
  238. package/core/server/threads/manageThreads.js +100 -26
  239. package/core/server/threads/socketRouter.ts +12 -275
  240. package/core/server/threads/threadServer.js +345 -105
  241. package/core/server/threads/workerProcessGuard.ts +93 -0
  242. package/core/server/throttle.ts +18 -0
  243. package/core/sqlTranslator/{SelectValidator.js → SelectValidator.ts} +41 -49
  244. package/core/sqlTranslator/{alasqlFunctionImporter.js → alasqlFunctionImporter.ts} +5 -5
  245. package/core/sqlTranslator/{deleteTranslator.js → deleteTranslator.ts} +13 -18
  246. package/core/sqlTranslator/{index.js → index.ts} +30 -33
  247. package/core/sqlTranslator/{sql_statement_bucket.js → sql_statement_bucket.ts} +49 -20
  248. package/core/static/README.md +10 -9
  249. package/core/system/000004.log +0 -0
  250. package/core/system/CURRENT +1 -0
  251. package/core/system/IDENTITY +1 -0
  252. package/core/system/LOCK +0 -0
  253. package/core/system/LOG +1351 -0
  254. package/core/system/MANIFEST-000005 +0 -0
  255. package/core/system/OPTIONS-000013 +607 -0
  256. package/core/system/OPTIONS-000015 +734 -0
  257. package/core/upgrade/{UpgradeObjects.js → UpgradeObjects.ts} +3 -6
  258. package/core/upgrade/directives/5-2-0.ts +49 -0
  259. package/core/upgrade/directives/{directivesController.js → directivesController.ts} +16 -16
  260. package/core/upgrade/{directivesManager.js → directivesManager.ts} +7 -11
  261. package/core/upgrade/{upgradePrompt.js → upgradePrompt.ts} +8 -14
  262. package/core/upgrade/{upgradeUtilities.js → upgradeUtilities.ts} +3 -7
  263. package/core/utility/{OperationFunctionCaller.js → OperationFunctionCaller.ts} +7 -7
  264. package/core/utility/{assignCmdEnvVariables.js → assignCmdEnvVariables.ts} +6 -8
  265. package/core/utility/{common_utils.js → common_utils.ts} +113 -139
  266. package/core/utility/environment/{environmentManager.js → environmentManager.ts} +34 -33
  267. package/core/utility/environment/systemInformation.ts +18 -4
  268. package/core/utility/errors/{commonErrors.js → commonErrors.ts} +9 -9
  269. package/core/utility/errors/{hdbError.js → hdbError.ts} +39 -45
  270. package/core/utility/expandEnvVar.ts +110 -0
  271. package/core/utility/functions/geo.js +2 -2
  272. package/core/utility/functions/sql/alaSQLExtension.js +1 -1
  273. package/core/utility/globalSchema.ts +30 -0
  274. package/core/utility/hdbTerms.ts +56 -0
  275. package/core/utility/install/checkJWTTokensExist.js +1 -1
  276. package/core/utility/install/{installer.js → installer.ts} +58 -59
  277. package/core/utility/installation.ts +2 -2
  278. package/core/utility/lmdb/{DBIDefinition.js → DBIDefinition.ts} +4 -1
  279. package/core/utility/lmdb/{DeleteRecordsResponseObject.js → DeleteRecordsResponseObject.ts} +2 -1
  280. package/core/utility/lmdb/{InsertRecordsResponseObject.js → InsertRecordsResponseObject.ts} +2 -1
  281. package/core/utility/lmdb/OpenDBIObject.ts +43 -0
  282. package/core/utility/lmdb/{OpenEnvironmentObject.js → OpenEnvironmentObject.ts} +19 -6
  283. package/core/utility/lmdb/{UpdateRecordsResponseObject.js → UpdateRecordsResponseObject.ts} +2 -1
  284. package/core/utility/lmdb/{UpsertRecordsResponseObject.js → UpsertRecordsResponseObject.ts} +2 -1
  285. package/core/utility/lmdb/{cleanLMDBMap.js → cleanLMDBMap.ts} +5 -5
  286. package/core/utility/lmdb/{commonUtility.js → commonUtility.ts} +13 -21
  287. package/core/utility/lmdb/{deleteUtility.js → deleteUtility.ts} +8 -12
  288. package/core/utility/lmdb/{environmentUtility.js → environmentUtility.ts} +43 -52
  289. package/core/utility/lmdb/{searchCursorFunctions.js → searchCursorFunctions.ts} +12 -26
  290. package/core/utility/lmdb/{searchUtility.js → searchUtility.ts} +75 -64
  291. package/core/utility/lmdb/{terms.js → terms.ts} +10 -23
  292. package/core/utility/lmdb/{writeUtility.js → writeUtility.ts} +37 -22
  293. package/core/utility/logging/{harper_logger.js → harper_logger.ts} +136 -89
  294. package/core/utility/logging/{logRotator.js → logRotator.ts} +13 -13
  295. package/core/utility/logging/logger.ts +1 -1
  296. package/core/utility/logging/{readLog.js → readLog.ts} +19 -19
  297. package/core/utility/logging/{transactionLog.js → transactionLog.ts} +10 -14
  298. package/core/utility/{mount_hdb.js → mount_hdb.ts} +15 -16
  299. package/core/utility/{npmUtilities.js → npmUtilities.ts} +14 -17
  300. package/core/utility/{operation_authorization.js → operation_authorization.ts} +173 -124
  301. package/core/utility/packageUtils.js +7 -16
  302. package/core/utility/password.ts +1 -1
  303. package/core/utility/processManagement/processManagement.js +2 -2
  304. package/core/utility/processManagement/servicesConfig.js +1 -1
  305. package/core/utility/{signalling.js → signalling.ts} +6 -11
  306. package/core/utility/watcherFallback.ts +74 -0
  307. package/core/validation/analyticsValidator.ts +43 -0
  308. package/core/validation/{bulkDeleteValidator.js → bulkDeleteValidator.ts} +5 -5
  309. package/core/validation/{check_permissions.js → check_permissions.ts} +3 -3
  310. package/core/validation/{common_validators.js → common_validators.ts} +12 -24
  311. package/core/validation/{configValidator.js → configValidator.ts} +114 -18
  312. package/core/validation/{deleteValidator.js → deleteValidator.ts} +5 -5
  313. package/core/validation/{fileLoadValidator.js → fileLoadValidator.ts} +12 -19
  314. package/core/validation/{insertValidator.js → insertValidator.ts} +5 -5
  315. package/core/validation/{installValidator.js → installValidator.ts} +8 -8
  316. package/core/validation/{readLogValidator.js → readLogValidator.ts} +10 -10
  317. package/core/validation/{role_validation.js → role_validation.ts} +26 -32
  318. package/core/validation/{schemaMetadataValidator.js → schemaMetadataValidator.ts} +5 -11
  319. package/core/validation/{searchValidator.js → searchValidator.ts} +12 -11
  320. package/core/validation/statusValidator.ts +1 -1
  321. package/core/validation/{transactionLogValidator.js → transactionLogValidator.ts} +4 -9
  322. package/core/validation/{user_validation.js → user_validation.ts} +4 -10
  323. package/core/validation/{validationWrapper.js → validationWrapper.ts} +3 -9
  324. package/dist/analytics/profile.js +4 -0
  325. package/dist/analytics/profile.js.map +1 -1
  326. package/dist/cloneNode/cloneNode.js +224 -12
  327. package/dist/cloneNode/cloneNode.js.map +1 -1
  328. package/dist/core/agent/agent.js +175 -0
  329. package/dist/core/agent/agent.js.map +1 -0
  330. package/dist/core/agent/loop.js +176 -0
  331. package/dist/core/agent/loop.js.map +1 -0
  332. package/dist/core/agent/operations.js +137 -0
  333. package/dist/core/agent/operations.js.map +1 -0
  334. package/dist/core/agent/session.js +182 -0
  335. package/dist/core/agent/session.js.map +1 -0
  336. package/dist/core/agent/tools/fsTools.js +286 -0
  337. package/dist/core/agent/tools/fsTools.js.map +1 -0
  338. package/dist/core/agent/tools/httpFetchTool.js +116 -0
  339. package/dist/core/agent/tools/httpFetchTool.js.map +1 -0
  340. package/dist/core/agent/tools/scheduleTool.js +54 -0
  341. package/dist/core/agent/tools/scheduleTool.js.map +1 -0
  342. package/dist/core/agent/toolset.js +33 -0
  343. package/dist/core/agent/toolset.js.map +1 -0
  344. package/dist/core/agent/types.js +10 -0
  345. package/dist/core/agent/types.js.map +1 -0
  346. package/dist/core/bin/BinObjects.js +6 -3
  347. package/dist/core/bin/BinObjects.js.map +1 -1
  348. package/dist/core/bin/cliCredentials.js +130 -0
  349. package/dist/core/bin/cliCredentials.js.map +1 -0
  350. package/dist/core/bin/cliOperations.js +254 -40
  351. package/dist/core/bin/cliOperations.js.map +1 -1
  352. package/dist/core/bin/copyDb.js +16 -16
  353. package/dist/core/bin/copyDb.js.map +1 -1
  354. package/dist/core/bin/deployRenderer.js +185 -0
  355. package/dist/core/bin/deployRenderer.js.map +1 -0
  356. package/dist/core/bin/harper.js +92 -31
  357. package/dist/core/bin/harper.js.map +1 -1
  358. package/dist/core/bin/install.js +41 -4
  359. package/dist/core/bin/install.js.map +1 -1
  360. package/dist/core/bin/lite.js +3 -4
  361. package/dist/core/bin/lite.js.map +1 -1
  362. package/dist/core/bin/login.js +158 -0
  363. package/dist/core/bin/login.js.map +1 -0
  364. package/dist/core/bin/logout.js +16 -0
  365. package/dist/core/bin/logout.js.map +1 -0
  366. package/dist/core/bin/mcp/client.js +395 -0
  367. package/dist/core/bin/mcp/client.js.map +1 -0
  368. package/dist/core/bin/mcp/doctor.js +193 -0
  369. package/dist/core/bin/mcp/doctor.js.map +1 -0
  370. package/dist/core/bin/mcp/index.js +81 -0
  371. package/dist/core/bin/mcp/index.js.map +1 -0
  372. package/dist/core/bin/mcp/options.js +113 -0
  373. package/dist/core/bin/mcp/options.js.map +1 -0
  374. package/dist/core/bin/mcp/printConfig.js +85 -0
  375. package/dist/core/bin/mcp/printConfig.js.map +1 -0
  376. package/dist/core/bin/multipartBuilder.js +55 -0
  377. package/dist/core/bin/multipartBuilder.js.map +1 -0
  378. package/dist/core/bin/restart.js +85 -48
  379. package/dist/core/bin/restart.js.map +1 -1
  380. package/dist/core/bin/run.js +123 -77
  381. package/dist/core/bin/run.js.map +1 -1
  382. package/dist/core/bin/sseConsumer.js +127 -0
  383. package/dist/core/bin/sseConsumer.js.map +1 -0
  384. package/dist/core/bin/status.js +48 -11
  385. package/dist/core/bin/status.js.map +1 -1
  386. package/dist/core/bin/stop.js +44 -7
  387. package/dist/core/bin/stop.js.map +1 -1
  388. package/dist/core/bin/upgrade.js +6 -6
  389. package/dist/core/components/Application.js +134 -28
  390. package/dist/core/components/Application.js.map +1 -1
  391. package/dist/core/components/ApplicationScope.js +2 -2
  392. package/dist/core/components/ComponentV1.js +5 -5
  393. package/dist/core/components/ComponentV1.js.map +1 -1
  394. package/dist/core/components/EntryHandler.js +153 -13
  395. package/dist/core/components/EntryHandler.js.map +1 -1
  396. package/dist/core/components/OptionsWatcher.js +72 -10
  397. package/dist/core/components/OptionsWatcher.js.map +1 -1
  398. package/dist/core/components/Scope.js +105 -9
  399. package/dist/core/components/Scope.js.map +1 -1
  400. package/dist/core/components/anthropic/index.js +428 -0
  401. package/dist/core/components/anthropic/index.js.map +1 -0
  402. package/dist/core/components/bedrock/index.js +734 -0
  403. package/dist/core/components/bedrock/index.js.map +1 -0
  404. package/dist/core/components/componentLoader.js +63 -38
  405. package/dist/core/components/componentLoader.js.map +1 -1
  406. package/dist/core/components/deployLifecycle.js +156 -0
  407. package/dist/core/components/deployLifecycle.js.map +1 -0
  408. package/dist/core/components/deploymentOperations.js +185 -0
  409. package/dist/core/components/deploymentOperations.js.map +1 -0
  410. package/dist/core/components/deploymentRecorder.js +401 -0
  411. package/dist/core/components/deploymentRecorder.js.map +1 -0
  412. package/dist/core/components/deriveURLPath.js +2 -2
  413. package/dist/core/components/deriveURLPath.js.map +1 -1
  414. package/dist/core/components/mcp/adapters/fastify.js +66 -0
  415. package/dist/core/components/mcp/adapters/fastify.js.map +1 -0
  416. package/dist/core/components/mcp/adapters/harperHttp.js +78 -0
  417. package/dist/core/components/mcp/adapters/harperHttp.js.map +1 -0
  418. package/dist/core/components/mcp/audit.js +73 -0
  419. package/dist/core/components/mcp/audit.js.map +1 -0
  420. package/dist/core/components/mcp/index.js +109 -0
  421. package/dist/core/components/mcp/index.js.map +1 -0
  422. package/dist/core/components/mcp/jsonrpc.js +93 -0
  423. package/dist/core/components/mcp/jsonrpc.js.map +1 -0
  424. package/dist/core/components/mcp/lifecycle.js +79 -0
  425. package/dist/core/components/mcp/lifecycle.js.map +1 -0
  426. package/dist/core/components/mcp/listChanged.js +257 -0
  427. package/dist/core/components/mcp/listChanged.js.map +1 -0
  428. package/dist/core/components/mcp/rateLimit.js +226 -0
  429. package/dist/core/components/mcp/rateLimit.js.map +1 -0
  430. package/dist/core/components/mcp/resources.js +515 -0
  431. package/dist/core/components/mcp/resources.js.map +1 -0
  432. package/dist/core/components/mcp/session.js +170 -0
  433. package/dist/core/components/mcp/session.js.map +1 -0
  434. package/dist/core/components/mcp/sessionRegistry.js +124 -0
  435. package/dist/core/components/mcp/sessionRegistry.js.map +1 -0
  436. package/dist/core/components/mcp/toolRegistry.js +176 -0
  437. package/dist/core/components/mcp/toolRegistry.js.map +1 -0
  438. package/dist/core/components/mcp/tools/application.js +549 -0
  439. package/dist/core/components/mcp/tools/application.js.map +1 -0
  440. package/dist/core/components/mcp/tools/operations.js +303 -0
  441. package/dist/core/components/mcp/tools/operations.js.map +1 -0
  442. package/dist/core/components/mcp/tools/schemas/derive.js +216 -0
  443. package/dist/core/components/mcp/tools/schemas/derive.js.map +1 -0
  444. package/dist/core/components/mcp/tools/schemas/operations.js +243 -0
  445. package/dist/core/components/mcp/tools/schemas/operations.js.map +1 -0
  446. package/dist/core/components/mcp/transport.js +467 -0
  447. package/dist/core/components/mcp/transport.js.map +1 -0
  448. package/dist/core/components/ollama/index.js +239 -0
  449. package/dist/core/components/ollama/index.js.map +1 -0
  450. package/dist/core/components/openai/index.js +475 -0
  451. package/dist/core/components/openai/index.js.map +1 -0
  452. package/dist/core/components/operations.js +198 -52
  453. package/dist/core/components/operations.js.map +1 -1
  454. package/dist/core/components/operationsValidation.js +3 -3
  455. package/dist/core/components/packageComponent.js +87 -26
  456. package/dist/core/components/packageComponent.js.map +1 -1
  457. package/dist/core/components/requestRestart.js +12 -1
  458. package/dist/core/components/requestRestart.js.map +1 -1
  459. package/dist/core/components/status/crossThread.js +12 -5
  460. package/dist/core/components/status/crossThread.js.map +1 -1
  461. package/dist/core/components/status/errors.js +7 -7
  462. package/dist/core/config/RootConfigWatcher.js +52 -1
  463. package/dist/core/config/RootConfigWatcher.js.map +1 -1
  464. package/dist/core/config/configUtils.js +31 -8
  465. package/dist/core/config/configUtils.js.map +1 -1
  466. package/dist/core/config/harperConfigEnvVars.js +1 -1
  467. package/dist/core/config/harperConfigEnvVars.js.map +1 -1
  468. package/dist/core/dataLayer/CreateAttributeObject.js +4 -3
  469. package/dist/core/dataLayer/CreateAttributeObject.js.map +1 -1
  470. package/dist/core/dataLayer/CreateTableObject.js +2 -1
  471. package/dist/core/dataLayer/CreateTableObject.js.map +1 -1
  472. package/dist/core/dataLayer/DataLayerObjects.js +19 -5
  473. package/dist/core/dataLayer/DataLayerObjects.js.map +1 -1
  474. package/dist/core/dataLayer/DeleteBeforeObject.js +2 -1
  475. package/dist/core/dataLayer/DeleteBeforeObject.js.map +1 -1
  476. package/dist/core/dataLayer/DeleteObject.js +4 -3
  477. package/dist/core/dataLayer/DeleteObject.js.map +1 -1
  478. package/dist/core/dataLayer/DropAttributeObject.js +2 -1
  479. package/dist/core/dataLayer/DropAttributeObject.js.map +1 -1
  480. package/dist/core/dataLayer/GetBackupObject.js +4 -3
  481. package/dist/core/dataLayer/GetBackupObject.js.map +1 -1
  482. package/dist/core/dataLayer/InsertObject.js +4 -3
  483. package/dist/core/dataLayer/InsertObject.js.map +1 -1
  484. package/dist/core/dataLayer/ReadAuditLogObject.js +4 -3
  485. package/dist/core/dataLayer/ReadAuditLogObject.js.map +1 -1
  486. package/dist/core/dataLayer/SQLSearch.js +140 -78
  487. package/dist/core/dataLayer/SQLSearch.js.map +1 -1
  488. package/dist/core/dataLayer/SearchByConditionsObject.js +5 -7
  489. package/dist/core/dataLayer/SearchByConditionsObject.js.map +1 -1
  490. package/dist/core/dataLayer/SearchByHashObject.js +2 -1
  491. package/dist/core/dataLayer/SearchByHashObject.js.map +1 -1
  492. package/dist/core/dataLayer/SearchObject.js +2 -1
  493. package/dist/core/dataLayer/SearchObject.js.map +1 -1
  494. package/dist/core/dataLayer/SqlSearchObject.js +2 -1
  495. package/dist/core/dataLayer/SqlSearchObject.js.map +1 -1
  496. package/dist/core/dataLayer/UpdateObject.js +4 -3
  497. package/dist/core/dataLayer/UpdateObject.js.map +1 -1
  498. package/dist/core/dataLayer/UpsertObject.js +4 -3
  499. package/dist/core/dataLayer/UpsertObject.js.map +1 -1
  500. package/dist/core/dataLayer/bulkLoad.js +122 -88
  501. package/dist/core/dataLayer/bulkLoad.js.map +1 -1
  502. package/dist/core/dataLayer/delete.js +74 -39
  503. package/dist/core/dataLayer/delete.js.map +1 -1
  504. package/dist/core/dataLayer/export.js +90 -55
  505. package/dist/core/dataLayer/export.js.map +1 -1
  506. package/dist/core/dataLayer/getBackup.js +43 -11
  507. package/dist/core/dataLayer/getBackup.js.map +1 -1
  508. package/dist/core/dataLayer/harperBridge/BridgeMethods.js +41 -20
  509. package/dist/core/dataLayer/harperBridge/BridgeMethods.js.map +1 -1
  510. package/dist/core/dataLayer/harperBridge/ResourceBridge.js +26 -25
  511. package/dist/core/dataLayer/harperBridge/ResourceBridge.js.map +1 -1
  512. package/dist/core/dataLayer/harperBridge/TableSizeObject.js.map +1 -1
  513. package/dist/core/dataLayer/harperBridge/bridgeUtility/insertUpdateValidate.js +4 -4
  514. package/dist/core/dataLayer/harperBridge/harperBridge.js +38 -4
  515. package/dist/core/dataLayer/harperBridge/harperBridge.js.map +1 -1
  516. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateAttribute.js +7 -6
  517. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateAttribute.js.map +1 -1
  518. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateRecords.js +4 -4
  519. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateRecords.js.map +1 -1
  520. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateSchema.js +1 -1
  521. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateSchema.js.map +1 -1
  522. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateTable.js +5 -4
  523. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbCreateTable.js.map +1 -1
  524. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDeleteAuditLogsBefore.js +4 -4
  525. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDeleteAuditLogsBefore.js.map +1 -1
  526. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDeleteRecords.js +4 -4
  527. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDropAttribute.js +5 -5
  528. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDropAttribute.js.map +1 -1
  529. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDropSchema.js +4 -4
  530. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDropSchema.js.map +1 -1
  531. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDropTable.js +5 -5
  532. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbDropTable.js.map +1 -1
  533. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbFlush.js +1 -1
  534. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbGetBackup.js +3 -3
  535. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbGetDataByHash.js +1 -1
  536. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbGetDataByValue.js +2 -2
  537. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbGetDataByValue.js.map +1 -1
  538. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbReadAuditLog.js +5 -5
  539. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbSearchByConditions.js +8 -8
  540. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbSearchByConditions.js.map +1 -1
  541. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbSearchByHash.js +1 -1
  542. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbSearchByValue.js +3 -3
  543. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbSearchByValue.js.map +1 -1
  544. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbUpdateRecords.js +3 -3
  545. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbUpsertRecords.js +5 -5
  546. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbUpsertRecords.js.map +1 -1
  547. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/LMDBCreateAttributeObject.js +1 -1
  548. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/LMDBCreateAttributeObject.js.map +1 -1
  549. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/initializeHashSearch.js +2 -2
  550. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/initializeHashSearch.js.map +1 -1
  551. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/initializePaths.js +2 -2
  552. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbCheckForNewAttributes.js +4 -4
  553. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbCheckForNewAttributes.js.map +1 -1
  554. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbCreateTransactionsAuditEnvironment.js +5 -3
  555. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbCreateTransactionsAuditEnvironment.js.map +1 -1
  556. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbGetTableSize.js +2 -2
  557. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbProcessRows.js +4 -4
  558. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbProcessRows.js.map +1 -1
  559. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbSearch.js +5 -5
  560. package/dist/core/dataLayer/harperBridge/lmdbBridge/lmdbUtility/lmdbWriteTransaction.js +4 -4
  561. package/dist/core/dataLayer/hdbInfoController.js +66 -31
  562. package/dist/core/dataLayer/hdbInfoController.js.map +1 -1
  563. package/dist/core/dataLayer/insert.js +63 -28
  564. package/dist/core/dataLayer/insert.js.map +1 -1
  565. package/dist/core/dataLayer/readAuditLog.js +45 -13
  566. package/dist/core/dataLayer/readAuditLog.js.map +1 -1
  567. package/dist/core/dataLayer/schema.js +124 -89
  568. package/dist/core/dataLayer/schema.js.map +1 -1
  569. package/dist/core/dataLayer/schemaDescribe.js +78 -41
  570. package/dist/core/dataLayer/schemaDescribe.js.map +1 -1
  571. package/dist/core/dataLayer/search.js +12 -13
  572. package/dist/core/dataLayer/search.js.map +1 -1
  573. package/dist/core/dataLayer/transaction.js +3 -4
  574. package/dist/core/dataLayer/transaction.js.map +1 -1
  575. package/dist/core/dataLayer/update.js +53 -18
  576. package/dist/core/dataLayer/update.js.map +1 -1
  577. package/dist/core/globals.js +1 -0
  578. package/dist/core/globals.js.map +1 -1
  579. package/dist/core/index.js +4 -1
  580. package/dist/core/index.js.map +1 -1
  581. package/dist/core/json/systemSchema.json +65 -0
  582. package/dist/core/launchServiceScripts/utility/checkNodeVersion.js +3 -0
  583. package/dist/core/launchServiceScripts/utility/checkNodeVersion.js.map +1 -1
  584. package/dist/core/resources/DatabaseTransaction.js +39 -15
  585. package/dist/core/resources/DatabaseTransaction.js.map +1 -1
  586. package/dist/core/resources/ErrorResource.js +3 -1
  587. package/dist/core/resources/ErrorResource.js.map +1 -1
  588. package/dist/core/resources/LMDBTransaction.js +18 -7
  589. package/dist/core/resources/LMDBTransaction.js.map +1 -1
  590. package/dist/core/resources/RecordEncoder.js +5 -2
  591. package/dist/core/resources/RecordEncoder.js.map +1 -1
  592. package/dist/core/resources/RequestTarget.js.map +1 -1
  593. package/dist/core/resources/Resource.js +37 -10
  594. package/dist/core/resources/Resource.js.map +1 -1
  595. package/dist/core/resources/ResourceInterface.js +20 -0
  596. package/dist/core/resources/ResourceInterface.js.map +1 -1
  597. package/dist/core/resources/Resources.js +7 -6
  598. package/dist/core/resources/Resources.js.map +1 -1
  599. package/dist/core/resources/RocksIndexStore.js +3 -0
  600. package/dist/core/resources/RocksIndexStore.js.map +1 -1
  601. package/dist/core/resources/RocksTransactionLogStore.js +46 -27
  602. package/dist/core/resources/RocksTransactionLogStore.js.map +1 -1
  603. package/dist/core/resources/Table.js +267 -107
  604. package/dist/core/resources/Table.js.map +1 -1
  605. package/dist/core/resources/analytics/metadata.js +1 -0
  606. package/dist/core/resources/analytics/metadata.js.map +1 -1
  607. package/dist/core/resources/analytics/read.js +16 -5
  608. package/dist/core/resources/analytics/read.js.map +1 -1
  609. package/dist/core/resources/analytics/write.js +232 -20
  610. package/dist/core/resources/analytics/write.js.map +1 -1
  611. package/dist/core/resources/auditStore.js +13 -8
  612. package/dist/core/resources/auditStore.js.map +1 -1
  613. package/dist/core/resources/blob.js +8 -8
  614. package/dist/core/resources/blob.js.map +1 -1
  615. package/dist/core/resources/dataLoader.js +13 -13
  616. package/dist/core/resources/databases.js +160 -52
  617. package/dist/core/resources/databases.js.map +1 -1
  618. package/dist/core/resources/graphql.js +224 -174
  619. package/dist/core/resources/graphql.js.map +1 -1
  620. package/dist/core/resources/indexes/HierarchicalNavigableSmallWorld.js +308 -59
  621. package/dist/core/resources/indexes/HierarchicalNavigableSmallWorld.js.map +1 -1
  622. package/dist/core/resources/indexes/vector.js +14 -0
  623. package/dist/core/resources/indexes/vector.js.map +1 -1
  624. package/dist/core/resources/loadEnv.js +21 -18
  625. package/dist/core/resources/loadEnv.js.map +1 -1
  626. package/dist/core/resources/login.js +5 -4
  627. package/dist/core/resources/login.js.map +1 -1
  628. package/dist/core/resources/models/Models.js +265 -0
  629. package/dist/core/resources/models/Models.js.map +1 -0
  630. package/dist/core/resources/models/TestBackend.js +71 -0
  631. package/dist/core/resources/models/TestBackend.js.map +1 -0
  632. package/dist/core/resources/models/agentLoop.js +746 -0
  633. package/dist/core/resources/models/agentLoop.js.map +1 -0
  634. package/dist/core/resources/models/analyticsTable.js +166 -0
  635. package/dist/core/resources/models/analyticsTable.js.map +1 -0
  636. package/dist/core/resources/models/backendHelpers.js +109 -0
  637. package/dist/core/resources/models/backendHelpers.js.map +1 -0
  638. package/dist/core/resources/models/backendRegistry.js +54 -0
  639. package/dist/core/resources/models/backendRegistry.js.map +1 -0
  640. package/dist/core/resources/models/bootstrap.js +112 -0
  641. package/dist/core/resources/models/bootstrap.js.map +1 -0
  642. package/dist/core/resources/models/embedHook.js +127 -0
  643. package/dist/core/resources/models/embedHook.js.map +1 -0
  644. package/dist/core/resources/models/types.js +11 -0
  645. package/dist/core/resources/models/types.js.map +1 -0
  646. package/dist/core/resources/replayLogs.js +7 -1
  647. package/dist/core/resources/replayLogs.js.map +1 -1
  648. package/dist/core/resources/roles.js +67 -71
  649. package/dist/core/resources/roles.js.map +1 -1
  650. package/dist/core/resources/search.js +349 -140
  651. package/dist/core/resources/search.js.map +1 -1
  652. package/dist/core/resources/tracked.js +14 -14
  653. package/dist/core/resources/tracked.js.map +1 -1
  654. package/dist/core/resources/transaction.js +1 -1
  655. package/dist/core/resources/transaction.js.map +1 -1
  656. package/dist/core/resources/transactionBroadcast.js.map +1 -1
  657. package/dist/core/security/auth.js +34 -25
  658. package/dist/core/security/auth.js.map +1 -1
  659. package/dist/core/security/certificateVerification/crlVerification.js +7 -1
  660. package/dist/core/security/certificateVerification/crlVerification.js.map +1 -1
  661. package/dist/core/security/cryptoHash.js +37 -5
  662. package/dist/core/security/cryptoHash.js.map +1 -1
  663. package/dist/core/security/data_objects/PermissionAttributeResponseObject.js +4 -1
  664. package/dist/core/security/data_objects/PermissionAttributeResponseObject.js.map +1 -1
  665. package/dist/core/security/data_objects/PermissionResponseObject.js +15 -8
  666. package/dist/core/security/data_objects/PermissionResponseObject.js.map +1 -1
  667. package/dist/core/security/data_objects/PermissionTableResponseObject.js +6 -1
  668. package/dist/core/security/data_objects/PermissionTableResponseObject.js.map +1 -1
  669. package/dist/core/security/fastifyAuth.js +131 -22
  670. package/dist/core/security/fastifyAuth.js.map +1 -1
  671. package/dist/core/security/impersonation.js +15 -15
  672. package/dist/core/security/jsLoader.js +18 -5
  673. package/dist/core/security/jsLoader.js.map +1 -1
  674. package/dist/core/security/keys.js +160 -114
  675. package/dist/core/security/keys.js.map +1 -1
  676. package/dist/core/security/permissionsTranslator.js +2 -2
  677. package/dist/core/security/role.js +70 -35
  678. package/dist/core/security/role.js.map +1 -1
  679. package/dist/core/security/tokenAuthentication.js +57 -27
  680. package/dist/core/security/tokenAuthentication.js.map +1 -1
  681. package/dist/core/security/user.js +74 -38
  682. package/dist/core/security/user.js.map +1 -1
  683. package/dist/core/server/DurableSubscriptionsSession.js +71 -55
  684. package/dist/core/server/DurableSubscriptionsSession.js.map +1 -1
  685. package/dist/core/server/REST.js +17 -16
  686. package/dist/core/server/REST.js.map +1 -1
  687. package/dist/core/server/Server.js +1 -1
  688. package/dist/core/server/Server.js.map +1 -1
  689. package/dist/core/server/fastifyRoutes/helpers/getCORSOptions.js +1 -1
  690. package/dist/core/server/fastifyRoutes/helpers/getHeaderTimeoutConfig.js +1 -1
  691. package/dist/core/server/fastifyRoutes/helpers/getServerOptions.js +1 -1
  692. package/dist/core/server/fastifyRoutes.js +34 -30
  693. package/dist/core/server/fastifyRoutes.js.map +1 -1
  694. package/dist/core/server/graphqlQuerying.js +8 -7
  695. package/dist/core/server/graphqlQuerying.js.map +1 -1
  696. package/dist/core/server/http.js +523 -47
  697. package/dist/core/server/http.js.map +1 -1
  698. package/dist/core/server/itc/serverHandlers.js +66 -15
  699. package/dist/core/server/itc/serverHandlers.js.map +1 -1
  700. package/dist/core/server/jobs/JobObject.js +53 -7
  701. package/dist/core/server/jobs/JobObject.js.map +1 -1
  702. package/dist/core/server/jobs/jobProcess.js +64 -24
  703. package/dist/core/server/jobs/jobProcess.js.map +1 -1
  704. package/dist/core/server/jobs/jobRunner.js +68 -30
  705. package/dist/core/server/jobs/jobRunner.js.map +1 -1
  706. package/dist/core/server/jobs/jobs.js +93 -61
  707. package/dist/core/server/jobs/jobs.js.map +1 -1
  708. package/dist/core/server/loadRootComponents.js +1 -1
  709. package/dist/core/server/middlewareChain.js +252 -0
  710. package/dist/core/server/middlewareChain.js.map +1 -0
  711. package/dist/core/server/mqtt.js +22 -17
  712. package/dist/core/server/mqtt.js.map +1 -1
  713. package/dist/core/server/nodeName.js +44 -13
  714. package/dist/core/server/nodeName.js.map +1 -1
  715. package/dist/core/server/operationsServer.js +106 -33
  716. package/dist/core/server/operationsServer.js.map +1 -1
  717. package/dist/core/server/serverHelpers/Headers.js +2 -0
  718. package/dist/core/server/serverHelpers/Headers.js.map +1 -1
  719. package/dist/core/server/serverHelpers/JSONStream.js +12 -3
  720. package/dist/core/server/serverHelpers/JSONStream.js.map +1 -1
  721. package/dist/core/server/serverHelpers/Request.js +370 -9
  722. package/dist/core/server/serverHelpers/Request.js.map +1 -1
  723. package/dist/core/server/serverHelpers/contentTypes.js +36 -7
  724. package/dist/core/server/serverHelpers/contentTypes.js.map +1 -1
  725. package/dist/core/server/serverHelpers/multipartParser.js +142 -0
  726. package/dist/core/server/serverHelpers/multipartParser.js.map +1 -0
  727. package/dist/core/server/serverHelpers/progressEmitter.js +103 -0
  728. package/dist/core/server/serverHelpers/progressEmitter.js.map +1 -0
  729. package/dist/core/server/serverHelpers/serverHandlers.js +38 -7
  730. package/dist/core/server/serverHelpers/serverHandlers.js.map +1 -1
  731. package/dist/core/server/serverHelpers/serverUtilities.js +97 -93
  732. package/dist/core/server/serverHelpers/serverUtilities.js.map +1 -1
  733. package/dist/core/server/static.js +8 -5
  734. package/dist/core/server/static.js.map +1 -1
  735. package/dist/core/server/status/index.js +3 -3
  736. package/dist/core/server/storageReclamation.js +68 -9
  737. package/dist/core/server/storageReclamation.js.map +1 -1
  738. package/dist/core/server/threads/itc.js +7 -4
  739. package/dist/core/server/threads/itc.js.map +1 -1
  740. package/dist/core/server/threads/manageThreads.js +110 -26
  741. package/dist/core/server/threads/manageThreads.js.map +1 -1
  742. package/dist/core/server/threads/socketRouter.js +8 -271
  743. package/dist/core/server/threads/socketRouter.js.map +1 -1
  744. package/dist/core/server/threads/threadServer.js +360 -118
  745. package/dist/core/server/threads/threadServer.js.map +1 -1
  746. package/dist/core/server/threads/workerProcessGuard.js +114 -0
  747. package/dist/core/server/threads/workerProcessGuard.js.map +1 -0
  748. package/dist/core/server/throttle.js +17 -0
  749. package/dist/core/server/throttle.js.map +1 -1
  750. package/dist/core/sqlTranslator/SelectValidator.js +86 -47
  751. package/dist/core/sqlTranslator/SelectValidator.js.map +1 -1
  752. package/dist/core/sqlTranslator/alasqlFunctionImporter.js +40 -3
  753. package/dist/core/sqlTranslator/alasqlFunctionImporter.js.map +1 -1
  754. package/dist/core/sqlTranslator/deleteTranslator.js +48 -14
  755. package/dist/core/sqlTranslator/deleteTranslator.js.map +1 -1
  756. package/dist/core/sqlTranslator/index.js +69 -30
  757. package/dist/core/sqlTranslator/index.js.map +1 -1
  758. package/dist/core/sqlTranslator/sql_statement_bucket.js +55 -13
  759. package/dist/core/sqlTranslator/sql_statement_bucket.js.map +1 -1
  760. package/dist/core/upgrade/UpgradeObjects.js +37 -4
  761. package/dist/core/upgrade/UpgradeObjects.js.map +1 -1
  762. package/dist/core/upgrade/directives/5-2-0.js +77 -0
  763. package/dist/core/upgrade/directives/5-2-0.js.map +1 -0
  764. package/dist/core/upgrade/directives/directivesController.js +52 -11
  765. package/dist/core/upgrade/directives/directivesController.js.map +1 -1
  766. package/dist/core/upgrade/directivesManager.js +53 -18
  767. package/dist/core/upgrade/directivesManager.js.map +1 -1
  768. package/dist/core/upgrade/upgradePrompt.js +65 -30
  769. package/dist/core/upgrade/upgradePrompt.js.map +1 -1
  770. package/dist/core/upgrade/upgradeUtilities.js +37 -5
  771. package/dist/core/upgrade/upgradeUtilities.js.map +1 -1
  772. package/dist/core/utility/OperationFunctionCaller.js +45 -10
  773. package/dist/core/utility/OperationFunctionCaller.js.map +1 -1
  774. package/dist/core/utility/assignCmdEnvVariables.js +8 -4
  775. package/dist/core/utility/assignCmdEnvVariables.js.map +1 -1
  776. package/dist/core/utility/common_utils.js +140 -79
  777. package/dist/core/utility/common_utils.js.map +1 -1
  778. package/dist/core/utility/environment/environmentManager.js +75 -29
  779. package/dist/core/utility/environment/environmentManager.js.map +1 -1
  780. package/dist/core/utility/environment/systemInformation.js +27 -16
  781. package/dist/core/utility/environment/systemInformation.js.map +1 -1
  782. package/dist/core/utility/errors/commonErrors.js +49 -18
  783. package/dist/core/utility/errors/commonErrors.js.map +1 -1
  784. package/dist/core/utility/errors/hdbError.js +65 -26
  785. package/dist/core/utility/errors/hdbError.js.map +1 -1
  786. package/dist/core/utility/expandEnvVar.js +113 -0
  787. package/dist/core/utility/expandEnvVar.js.map +1 -0
  788. package/dist/core/utility/functions/geo.js +2 -2
  789. package/dist/core/utility/functions/sql/alaSQLExtension.js +1 -1
  790. package/dist/core/utility/globalSchema.js +14 -11
  791. package/dist/core/utility/globalSchema.js.map +1 -1
  792. package/dist/core/utility/hdbTerms.js +56 -0
  793. package/dist/core/utility/hdbTerms.js.map +1 -1
  794. package/dist/core/utility/install/checkJWTTokensExist.js +1 -1
  795. package/dist/core/utility/install/installer.js +106 -70
  796. package/dist/core/utility/install/installer.js.map +1 -1
  797. package/dist/core/utility/installation.js +3 -3
  798. package/dist/core/utility/lmdb/DBIDefinition.js +5 -1
  799. package/dist/core/utility/lmdb/DBIDefinition.js.map +1 -1
  800. package/dist/core/utility/lmdb/DeleteRecordsResponseObject.js +2 -1
  801. package/dist/core/utility/lmdb/DeleteRecordsResponseObject.js.map +1 -1
  802. package/dist/core/utility/lmdb/InsertRecordsResponseObject.js +2 -1
  803. package/dist/core/utility/lmdb/InsertRecordsResponseObject.js.map +1 -1
  804. package/dist/core/utility/lmdb/OpenDBIObject.js +54 -6
  805. package/dist/core/utility/lmdb/OpenDBIObject.js.map +1 -1
  806. package/dist/core/utility/lmdb/OpenEnvironmentObject.js +52 -4
  807. package/dist/core/utility/lmdb/OpenEnvironmentObject.js.map +1 -1
  808. package/dist/core/utility/lmdb/UpdateRecordsResponseObject.js +2 -1
  809. package/dist/core/utility/lmdb/UpdateRecordsResponseObject.js.map +1 -1
  810. package/dist/core/utility/lmdb/UpsertRecordsResponseObject.js +2 -1
  811. package/dist/core/utility/lmdb/UpsertRecordsResponseObject.js.map +1 -1
  812. package/dist/core/utility/lmdb/cleanLMDBMap.js +44 -7
  813. package/dist/core/utility/lmdb/cleanLMDBMap.js.map +1 -1
  814. package/dist/core/utility/lmdb/commonUtility.js +46 -17
  815. package/dist/core/utility/lmdb/commonUtility.js.map +1 -1
  816. package/dist/core/utility/lmdb/deleteUtility.js +51 -16
  817. package/dist/core/utility/lmdb/deleteUtility.js.map +1 -1
  818. package/dist/core/utility/lmdb/environmentUtility.js +91 -51
  819. package/dist/core/utility/lmdb/environmentUtility.js.map +1 -1
  820. package/dist/core/utility/lmdb/searchCursorFunctions.js +46 -14
  821. package/dist/core/utility/lmdb/searchCursorFunctions.js.map +1 -1
  822. package/dist/core/utility/lmdb/searchUtility.js +91 -55
  823. package/dist/core/utility/lmdb/searchUtility.js.map +1 -1
  824. package/dist/core/utility/lmdb/terms.js +12 -22
  825. package/dist/core/utility/lmdb/terms.js.map +1 -1
  826. package/dist/core/utility/lmdb/writeUtility.js +61 -28
  827. package/dist/core/utility/lmdb/writeUtility.js.map +1 -1
  828. package/dist/core/utility/logging/harper_logger.js +176 -81
  829. package/dist/core/utility/logging/harper_logger.js.map +1 -1
  830. package/dist/core/utility/logging/logRotator.js +65 -28
  831. package/dist/core/utility/logging/logRotator.js.map +1 -1
  832. package/dist/core/utility/logging/logger.js +4 -4
  833. package/dist/core/utility/logging/readLog.js +54 -17
  834. package/dist/core/utility/logging/readLog.js.map +1 -1
  835. package/dist/core/utility/logging/transactionLog.js +51 -16
  836. package/dist/core/utility/logging/transactionLog.js.map +1 -1
  837. package/dist/core/utility/mount_hdb.js +54 -17
  838. package/dist/core/utility/mount_hdb.js.map +1 -1
  839. package/dist/core/utility/npmUtilities.js +54 -19
  840. package/dist/core/utility/npmUtilities.js.map +1 -1
  841. package/dist/core/utility/operation_authorization.js +135 -86
  842. package/dist/core/utility/operation_authorization.js.map +1 -1
  843. package/dist/core/utility/packageUtils.js +7 -17
  844. package/dist/core/utility/packageUtils.js.map +1 -1
  845. package/dist/core/utility/password.js +2 -2
  846. package/dist/core/utility/processManagement/processManagement.js +2 -2
  847. package/dist/core/utility/processManagement/servicesConfig.js +1 -1
  848. package/dist/core/utility/signalling.js +51 -16
  849. package/dist/core/utility/signalling.js.map +1 -1
  850. package/dist/core/utility/watcherFallback.js +73 -0
  851. package/dist/core/utility/watcherFallback.js.map +1 -0
  852. package/dist/core/validation/analyticsValidator.js +79 -0
  853. package/dist/core/validation/analyticsValidator.js.map +1 -0
  854. package/dist/core/validation/bulkDeleteValidator.js +49 -11
  855. package/dist/core/validation/bulkDeleteValidator.js.map +1 -1
  856. package/dist/core/validation/check_permissions.js +38 -3
  857. package/dist/core/validation/check_permissions.js.map +1 -1
  858. package/dist/core/validation/common_validators.js +62 -31
  859. package/dist/core/validation/common_validators.js.map +1 -1
  860. package/dist/core/validation/configValidator.js +189 -54
  861. package/dist/core/validation/configValidator.js.map +1 -1
  862. package/dist/core/validation/deleteValidator.js +49 -11
  863. package/dist/core/validation/deleteValidator.js.map +1 -1
  864. package/dist/core/validation/fileLoadValidator.js +67 -32
  865. package/dist/core/validation/fileLoadValidator.js.map +1 -1
  866. package/dist/core/validation/insertValidator.js +48 -10
  867. package/dist/core/validation/insertValidator.js.map +1 -1
  868. package/dist/core/validation/installValidator.js +47 -10
  869. package/dist/core/validation/installValidator.js.map +1 -1
  870. package/dist/core/validation/readLogValidator.js +60 -22
  871. package/dist/core/validation/readLogValidator.js.map +1 -1
  872. package/dist/core/validation/role_validation.js +55 -19
  873. package/dist/core/validation/role_validation.js.map +1 -1
  874. package/dist/core/validation/schemaMetadataValidator.js +11 -12
  875. package/dist/core/validation/schemaMetadataValidator.js.map +1 -1
  876. package/dist/core/validation/searchValidator.js +82 -43
  877. package/dist/core/validation/searchValidator.js.map +1 -1
  878. package/dist/core/validation/transactionLogValidator.js +52 -17
  879. package/dist/core/validation/transactionLogValidator.js.map +1 -1
  880. package/dist/core/validation/user_validation.js +38 -6
  881. package/dist/core/validation/user_validation.js.map +1 -1
  882. package/dist/core/validation/validationWrapper.js +4 -5
  883. package/dist/core/validation/validationWrapper.js.map +1 -1
  884. package/dist/licensing/usageLicensing.js +30 -21
  885. package/dist/licensing/usageLicensing.js.map +1 -1
  886. package/dist/replication/knownNodes.js +173 -39
  887. package/dist/replication/knownNodes.js.map +1 -1
  888. package/dist/replication/replicationConnection.js +441 -85
  889. package/dist/replication/replicationConnection.js.map +1 -1
  890. package/dist/replication/replicator.js +44 -26
  891. package/dist/replication/replicator.js.map +1 -1
  892. package/dist/replication/setNode.js +22 -2
  893. package/dist/replication/setNode.js.map +1 -1
  894. package/dist/replication/subscriptionManager.js +121 -9
  895. package/dist/replication/subscriptionManager.js.map +1 -1
  896. package/dist/security/certificate.js +41 -6
  897. package/dist/security/certificate.js.map +1 -1
  898. package/dist/security/sshKeyOperations.js +35 -2
  899. package/dist/security/sshKeyOperations.js.map +1 -1
  900. package/licensing/usageLicensing.ts +32 -37
  901. package/npm-shrinkwrap.json +8994 -4305
  902. package/package.json +16 -11
  903. package/replication/DESIGN.md +139 -0
  904. package/replication/knownNodes.ts +166 -43
  905. package/replication/replicationConnection.ts +475 -92
  906. package/replication/replicator.ts +42 -25
  907. package/replication/setNode.ts +28 -9
  908. package/replication/subscriptionManager.ts +138 -14
  909. package/security/certificate.ts +8 -4
  910. package/security/sshKeyOperations.ts +1 -1
  911. package/static/defaultConfig.yaml +1 -0
  912. package/studio/web/assets/{index-CmtPP0YO.js → index-COfIkCT-.js} +5 -5
  913. package/studio/web/assets/index-COfIkCT-.js.map +1 -0
  914. package/studio/web/assets/{index.lazy-C8jvGtlu.js → index.lazy-CIvl7Fj9.js} +2 -2
  915. package/studio/web/assets/{index.lazy-C8jvGtlu.js.map → index.lazy-CIvl7Fj9.js.map} +1 -1
  916. package/studio/web/assets/{profile-O0DYlJUv.js → profile-B2ZVB--r.js} +2 -2
  917. package/studio/web/assets/{profile-O0DYlJUv.js.map → profile-B2ZVB--r.js.map} +1 -1
  918. package/studio/web/assets/{status-BIlJkJby.js → status-Db6WBmhf.js} +2 -2
  919. package/studio/web/assets/{status-BIlJkJby.js.map → status-Db6WBmhf.js.map} +1 -1
  920. package/studio/web/index.html +1 -1
  921. package/core/bin/cliOperations.js +0 -159
  922. package/core/bin/lite.js +0 -5
  923. package/core/bin/stop.js +0 -21
  924. package/core/dataLayer/harperBridge/BridgeMethods.js +0 -85
  925. package/core/utility/globalSchema.js +0 -35
  926. package/core/utility/lmdb/OpenDBIObject.js +0 -31
  927. package/studio/web/assets/index-CmtPP0YO.js.map +0 -1
@@ -0,0 +1,112 @@
1
+ /**
2
+ * `http_fetch` for the built-in agent (#626). Wraps the platform `fetch`
3
+ * with a size cap, an inactivity timeout, and a metadata/loopback blocklist
4
+ * so the agent can probe its own deployed components and pull lightweight
5
+ * web pages for context without becoming an SSRF vector against cloud
6
+ * instance-metadata endpoints or unrelated internal services.
7
+ */
8
+
9
+ import { isIP } from 'node:net';
10
+ import type { AgentTool, AgentToolContext } from '../types.ts';
11
+
12
+ const MAX_BYTES = 2 * 1024 * 1024; // 2 MiB cap on response bodies
13
+ const DEFAULT_TIMEOUT_MS = 30_000;
14
+ // Hard-blocked literal hosts. Cloud-metadata services live on these IPs and exposing them
15
+ // to a prompt-controlled fetch is a credential-leak vector. Loopback to the local Harper
16
+ // instance is allowed via `localhost`/`127.0.0.1` for self-testing — those are NOT blocked.
17
+ const BLOCKED_HOSTS = new Set([
18
+ '169.254.169.254', // AWS / GCP / Azure IMDS
19
+ 'fd00:ec2::254', // AWS IMDSv2 IPv6
20
+ 'metadata.google.internal',
21
+ 'metadata.goog',
22
+ ]);
23
+
24
+ export const httpFetchTool: AgentTool = {
25
+ def: {
26
+ name: 'http_fetch',
27
+ description:
28
+ "Issue an HTTP request from the Harper server. Useful for hitting the agent's own components on localhost and pulling reference pages.",
29
+ parameters: {
30
+ type: 'object',
31
+ properties: {
32
+ url: { type: 'string', description: 'Absolute URL.' },
33
+ method: { type: 'string', enum: ['GET', 'POST', 'PUT', 'PATCH', 'DELETE', 'HEAD'] },
34
+ headers: { type: 'object', additionalProperties: { type: 'string' } },
35
+ body: { type: 'string', description: 'Request body as a string (JSON or form-encoded).' },
36
+ timeoutMs: { type: 'integer', minimum: 1, maximum: 120_000 },
37
+ },
38
+ required: ['url'],
39
+ },
40
+ },
41
+ handler: async (args: any, ctx: AgentToolContext) => {
42
+ const url = String(args.url ?? '');
43
+ if (!/^https?:\/\//i.test(url)) throw new Error('http_fetch requires an http(s) URL');
44
+ let parsed: URL;
45
+ try {
46
+ parsed = new URL(url);
47
+ } catch {
48
+ throw new Error(`http_fetch could not parse URL: ${url}`);
49
+ }
50
+ const host = parsed.hostname.toLowerCase();
51
+ if (BLOCKED_HOSTS.has(host)) {
52
+ throw new Error(`http_fetch blocked by metadata-host policy: ${host}`);
53
+ }
54
+ // IPv4 link-local (169.254.0.0/16) covers IMDS variants beyond the canonical 169.254.169.254.
55
+ if (isIP(host) === 4 && host.startsWith('169.254.')) {
56
+ throw new Error(`http_fetch blocked by link-local policy: ${host}`);
57
+ }
58
+ const timeoutMs = Math.min(args.timeoutMs ?? DEFAULT_TIMEOUT_MS, 120_000);
59
+ const localAbort = new AbortController();
60
+ const timer = setTimeout(() => localAbort.abort(new Error(`http_fetch timed out after ${timeoutMs}ms`)), timeoutMs);
61
+ const signal = combineSignals(ctx.signal, localAbort.signal);
62
+ try {
63
+ const response = await fetch(url, {
64
+ method: args.method ?? 'GET',
65
+ headers: args.headers,
66
+ body: args.body,
67
+ signal,
68
+ });
69
+ const buffer = await readCapped(response, MAX_BYTES);
70
+ return {
71
+ status: response.status,
72
+ headers: Object.fromEntries(response.headers.entries()),
73
+ body: buffer.toString('utf8'),
74
+ truncated: buffer.length === MAX_BYTES,
75
+ };
76
+ } finally {
77
+ clearTimeout(timer);
78
+ }
79
+ },
80
+ };
81
+
82
+ async function readCapped(response: Response, cap: number): Promise<Buffer> {
83
+ const reader = response.body?.getReader();
84
+ if (!reader) return Buffer.alloc(0);
85
+ const chunks: Buffer[] = [];
86
+ let total = 0;
87
+ while (total < cap) {
88
+ const { value, done } = await reader.read();
89
+ if (done) break;
90
+ const chunk = Buffer.from(value);
91
+ const room = cap - total;
92
+ if (chunk.length > room) {
93
+ chunks.push(chunk.subarray(0, room));
94
+ total += room;
95
+ await reader.cancel().catch(() => {});
96
+ break;
97
+ }
98
+ chunks.push(chunk);
99
+ total += chunk.length;
100
+ }
101
+ return Buffer.concat(chunks, total);
102
+ }
103
+
104
+ function combineSignals(...signals: Array<AbortSignal | undefined>): AbortSignal | undefined {
105
+ const present = signals.filter((s): s is AbortSignal => Boolean(s));
106
+ if (present.length === 0) return undefined;
107
+ if (present.length === 1) return present[0];
108
+ // `AbortSignal.any` (Node 20+) manages listener cleanup internally; the manual
109
+ // `addEventListener` approach leaked listeners on `ctx.signal` for the lifetime of the agent
110
+ // run when a fetch completed normally.
111
+ return AbortSignal.any(present);
112
+ }
@@ -0,0 +1,68 @@
1
+ /**
2
+ * `schedule_followup` for the built-in agent (#626).
3
+ *
4
+ * Lives on the main thread so it survives worker-thread restarts that happen
5
+ * on code reload. The followup callback is injected from the entry point —
6
+ * this module only schedules the timer and tracks pending entries.
7
+ */
8
+
9
+ import { randomUUID } from 'node:crypto';
10
+ import type { AgentTool, AgentToolContext } from '../types.ts';
11
+
12
+ const MAX_DELAY_MS = 24 * 60 * 60 * 1000; // 24h
13
+ const MIN_DELAY_MS = 1_000;
14
+
15
+ export interface ScheduledFollowup {
16
+ id: string;
17
+ sessionId: string;
18
+ prompt: string;
19
+ fireAt: number;
20
+ timer: NodeJS.Timeout;
21
+ }
22
+
23
+ export interface ScheduleToolDeps {
24
+ onFollowup: (sessionId: string, prompt: string) => Promise<void> | void;
25
+ }
26
+
27
+ export function buildScheduleTool(deps: ScheduleToolDeps): {
28
+ tool: AgentTool;
29
+ pending: Map<string, ScheduledFollowup>;
30
+ } {
31
+ const pending = new Map<string, ScheduledFollowup>();
32
+ const tool: AgentTool = {
33
+ def: {
34
+ name: 'schedule_followup',
35
+ description: 'Re-invoke the agent on the current session after the given delay with a new prompt.',
36
+ parameters: {
37
+ type: 'object',
38
+ properties: {
39
+ delayMs: { type: 'integer', minimum: MIN_DELAY_MS, maximum: MAX_DELAY_MS },
40
+ prompt: { type: 'string', description: 'Prompt the agent should re-enter on with.' },
41
+ },
42
+ required: ['delayMs', 'prompt'],
43
+ },
44
+ },
45
+ handler: async (args: any, ctx: AgentToolContext) => {
46
+ const delay = Number(args.delayMs);
47
+ if (!Number.isFinite(delay) || delay < MIN_DELAY_MS || delay > MAX_DELAY_MS) {
48
+ throw new Error(`delayMs must be between ${MIN_DELAY_MS} and ${MAX_DELAY_MS}`);
49
+ }
50
+ const prompt = String(args.prompt ?? '').trim();
51
+ if (!prompt) throw new Error('prompt is required');
52
+ const id = randomUUID();
53
+ const fireAt = Date.now() + delay;
54
+ const timer = setTimeout(() => {
55
+ pending.delete(id);
56
+ Promise.resolve()
57
+ .then(() => deps.onFollowup(ctx.sessionId, prompt))
58
+ .catch(() => {
59
+ /* swallow — caller logs */
60
+ });
61
+ }, delay);
62
+ timer.unref?.();
63
+ pending.set(id, { id, sessionId: ctx.sessionId, prompt, fireAt, timer });
64
+ return { id, fireAt };
65
+ },
66
+ };
67
+ return { tool, pending };
68
+ }
@@ -0,0 +1,43 @@
1
+ /**
2
+ * Tool composer for the built-in agent (#626).
3
+ *
4
+ * Operator-only tools (FS, schedule, fetch) are inline today. Once the
5
+ * unified MCP tool registry (#615) lands with the Operations (#617) and
6
+ * Application (#618) profiles, this is the seam where RBAC-filtered
7
+ * registry tools get folded in for the agent's configured user. The shape
8
+ * of {@link composeToolset} won't change — only the body gains a registry
9
+ * lookup.
10
+ */
11
+
12
+ import { fsTools } from './tools/fsTools.ts';
13
+ import { httpFetchTool } from './tools/httpFetchTool.ts';
14
+ import { buildScheduleTool, type ScheduleToolDeps, type ScheduledFollowup } from './tools/scheduleTool.ts';
15
+ import type { AgentTool } from './types.ts';
16
+
17
+ export interface ComposeToolsetOpts extends ScheduleToolDeps {
18
+ /** When `false`, destructive tools are filtered out at composition time. */
19
+ allowDestructive?: boolean;
20
+ /** Operator-injected extras (tests, custom plugins). */
21
+ extraTools?: AgentTool[];
22
+ }
23
+
24
+ export interface ComposedToolset {
25
+ tools: AgentTool[];
26
+ scheduled: Map<string, ScheduledFollowup>;
27
+ }
28
+
29
+ export function composeToolset(opts: ComposeToolsetOpts): ComposedToolset {
30
+ const schedule = buildScheduleTool(opts);
31
+ const all: AgentTool[] = [...fsTools, httpFetchTool, schedule.tool, ...(opts.extraTools ?? [])];
32
+ const tools = opts.allowDestructive === false ? all.filter((t) => !t.destructive) : all;
33
+ return { tools, scheduled: schedule.pending };
34
+ }
35
+
36
+ export function toolMapByName(tools: AgentTool[]): Map<string, AgentTool> {
37
+ const map = new Map<string, AgentTool>();
38
+ for (const tool of tools) {
39
+ if (map.has(tool.def.name)) throw new Error(`Duplicate tool registered: ${tool.def.name}`);
40
+ map.set(tool.def.name, tool);
41
+ }
42
+ return map;
43
+ }
@@ -0,0 +1,85 @@
1
+ /**
2
+ * Shared types for the built-in Harper Agent component (#626).
3
+ *
4
+ * `AgentMessage` mirrors `resources/models/types.ts:Message` rather than
5
+ * re-exporting it: the session row is durable storage and shouldn't trail
6
+ * shape changes in the model-access surface.
7
+ */
8
+
9
+ import type { ToolCall, ToolDef } from '../resources/models/types.ts';
10
+
11
+ export type AgentRunStatus = 'idle' | 'running' | 'awaiting_approval' | 'completed' | 'aborted' | 'error';
12
+
13
+ export interface AgentMessage {
14
+ role: 'system' | 'user' | 'assistant' | 'tool';
15
+ content: string;
16
+ toolCalls?: ToolCall[];
17
+ toolCallId?: string;
18
+ /** Wall-clock timestamp of when this item entered the transcript. */
19
+ createdAt: number;
20
+ }
21
+
22
+ export interface ApprovalRequest {
23
+ id: string;
24
+ toolName: string;
25
+ arguments: object;
26
+ /** Tool-call id from the assistant message that produced this request. Used to resume execution. */
27
+ toolCallId: string;
28
+ /** Human-readable rationale for why approval is required (e.g. 'destructive', 'first_use'). */
29
+ reason: string;
30
+ createdAt: number;
31
+ resolved?: boolean;
32
+ approved?: boolean;
33
+ resolvedAt?: number;
34
+ /** Set to true once the loop has consumed the resolved approval (executed or refused). */
35
+ consumed?: boolean;
36
+ }
37
+
38
+ export interface AgentSessionRow {
39
+ session_id: string;
40
+ user: string;
41
+ status: AgentRunStatus;
42
+ messages: AgentMessage[];
43
+ pendingApprovals: ApprovalRequest[];
44
+ model?: string;
45
+ provider?: string;
46
+ createdAt: number;
47
+ updatedAt: number;
48
+ lastError?: string;
49
+ }
50
+
51
+ export interface AgentToolHandler {
52
+ (args: object, ctx: AgentToolContext): Promise<unknown>;
53
+ }
54
+
55
+ export interface AgentTool {
56
+ def: ToolDef;
57
+ handler: AgentToolHandler;
58
+ /** When true, invocations of this tool require an approval gate unless `autoApprove` is set. */
59
+ destructive?: boolean;
60
+ }
61
+
62
+ export interface AgentToolContext {
63
+ sessionId: string;
64
+ signal?: AbortSignal;
65
+ /** Filesystem scope roots accessible to FS tools. Read-only after composition. */
66
+ scopes: AgentScopes;
67
+ }
68
+
69
+ export interface AgentScopes {
70
+ componentsRoot: string;
71
+ logDir: string;
72
+ configDir: string;
73
+ }
74
+
75
+ export interface AgentConfig {
76
+ enabled: boolean;
77
+ provider?: string;
78
+ model?: string;
79
+ maxTurns: number;
80
+ maxCostUsd: number;
81
+ autoApprove: boolean;
82
+ allowDestructive: boolean;
83
+ user: string;
84
+ componentsScope?: string;
85
+ }
@@ -0,0 +1,157 @@
1
+ 'use strict';
2
+ /**
3
+ * HNSW searchLayer performance benchmark — measures search latency, throughput,
4
+ * nodes visited per query, and recall@K against brute-force ground truth.
5
+ *
6
+ * Run (from core/ directory, after npm run build):
7
+ * node benchmarks/hnsw-search.js [N_VECTORS] [DIMS] [N_QUERIES]
8
+ *
9
+ * Examples:
10
+ * node benchmarks/hnsw-search.js # defaults: 2000 x 384 x 50
11
+ * node benchmarks/hnsw-search.js 5000 768 # realistic embedding workload
12
+ */
13
+
14
+ const { performance } = require('node:perf_hooks');
15
+ const { HierarchicalNavigableSmallWorld } = require('#src/resources/indexes/HierarchicalNavigableSmallWorld');
16
+ const { cosineDistance } = require('#src/resources/indexes/vector');
17
+
18
+ const N_VECTORS = parseInt(process.argv[2]) || 2_000;
19
+ const DIMS = parseInt(process.argv[3]) || 384;
20
+ const N_QUERIES = parseInt(process.argv[4]) || 50;
21
+ const TOP_K = 10;
22
+
23
+ // ---------------------------------------------------------------------------
24
+ // Minimal in-memory store — enough interface for HierarchicalNavigableSmallWorld
25
+ // ---------------------------------------------------------------------------
26
+
27
+ class MemoryStore {
28
+ constructor() {
29
+ this._map = new Map();
30
+ this.encoder = { useFloat32: null };
31
+ }
32
+ _key(k) {
33
+ if (Array.isArray(k)) {
34
+ return k.map((v) => (typeof v === 'symbol' ? (Symbol.keyFor(v) ?? String(v)) : v)).join('\x00');
35
+ }
36
+ return k;
37
+ }
38
+ getSync(key) {
39
+ return this._map.get(this._key(key));
40
+ }
41
+ put(key, value) {
42
+ this._map.set(this._key(key), value);
43
+ }
44
+ remove(key) {
45
+ this._map.delete(this._key(key));
46
+ }
47
+ *getKeys({ reverse = false, limit = Infinity, start = -Infinity, end = Infinity } = {}) {
48
+ const keys = [];
49
+ for (const k of this._map.keys()) {
50
+ if (typeof k === 'number' && k >= start && k <= end) keys.push(k);
51
+ }
52
+ keys.sort((a, b) => (reverse ? b - a : a - b));
53
+ let n = 0;
54
+ for (const k of keys) {
55
+ if (n++ >= limit) break;
56
+ yield k;
57
+ }
58
+ }
59
+ *getRange({ start = -Infinity, end = Infinity } = {}) {
60
+ for (const [k, v] of this._map) {
61
+ if (typeof k === 'number' && k >= start && k <= end) yield { key: k, value: v };
62
+ }
63
+ }
64
+ getUserSharedBuffer(_name, buf) {
65
+ return buf;
66
+ }
67
+ getStats() {
68
+ let n = 0;
69
+ for (const k of this._map.keys()) if (typeof k === 'number') n++;
70
+ return { entryCount: n };
71
+ }
72
+ }
73
+
74
+ // ---------------------------------------------------------------------------
75
+ // Helpers
76
+ // ---------------------------------------------------------------------------
77
+
78
+ function randomVector(dims) {
79
+ const v = new Array(dims);
80
+ for (let i = 0; i < dims; i++) v[i] = Math.random() * 2 - 1;
81
+ return v;
82
+ }
83
+
84
+ function fmtNs(ns) {
85
+ if (ns < 1_000) return `${ns.toFixed(1)} ns`;
86
+ if (ns < 1_000_000) return `${(ns / 1_000).toFixed(2)} µs`;
87
+ return `${(ns / 1_000_000).toFixed(2)} ms`;
88
+ }
89
+
90
+ // ---------------------------------------------------------------------------
91
+ // Build index
92
+ // ---------------------------------------------------------------------------
93
+
94
+ console.log(`\nHNSW benchmark — ${N_VECTORS.toLocaleString()} vectors × ${DIMS} dims, ${N_QUERIES} queries\n`);
95
+
96
+ const vectors = Array.from({ length: N_VECTORS }, () => randomVector(DIMS));
97
+ const store = new MemoryStore();
98
+ const hnsw = new HierarchicalNavigableSmallWorld(store, { distance: 'cosine' });
99
+
100
+ process.stdout.write('Building index...');
101
+ const buildStart = performance.now();
102
+ for (let i = 0; i < N_VECTORS; i++) {
103
+ hnsw.index('r' + i, vectors[i], undefined, {});
104
+ }
105
+ const buildMs = performance.now() - buildStart;
106
+ console.log(` done in ${buildMs.toFixed(0)} ms (${(buildMs / N_VECTORS).toFixed(2)} ms/insert)\n`);
107
+
108
+ // ---------------------------------------------------------------------------
109
+ // Brute-force ground truth for recall@K
110
+ // ---------------------------------------------------------------------------
111
+
112
+ const queries = Array.from({ length: N_QUERIES }, () => randomVector(DIMS));
113
+
114
+ process.stdout.write('Computing brute-force ground truth...');
115
+ const gtStart = performance.now();
116
+ const groundTruth = queries.map((query) => {
117
+ const scored = vectors.map((v, i) => ({ i, d: cosineDistance(query, v) }));
118
+ scored.sort((a, b) => a.d - b.d);
119
+ return new Set(scored.slice(0, TOP_K).map((x) => x.i));
120
+ });
121
+ console.log(` done in ${(performance.now() - gtStart).toFixed(0)} ms\n`);
122
+
123
+ // ---------------------------------------------------------------------------
124
+ // Search benchmark
125
+ // ---------------------------------------------------------------------------
126
+
127
+ // Warmup
128
+ for (let i = 0; i < 10; i++) {
129
+ hnsw.search({ target: queries[i % N_QUERIES], comparator: 'sort', descending: false }, {});
130
+ }
131
+ hnsw.nodesVisitedCount = 0;
132
+
133
+ const searchStart = performance.now();
134
+ const allResults = queries.map((query) => hnsw.search({ target: query, comparator: 'sort', descending: false }, {}));
135
+ const searchMs = performance.now() - searchStart;
136
+
137
+ // ---------------------------------------------------------------------------
138
+ // Recall@K
139
+ // ---------------------------------------------------------------------------
140
+
141
+ let recallSum = 0;
142
+ for (let i = 0; i < N_QUERIES; i++) {
143
+ const resultIndices = new Set(allResults[i].slice(0, TOP_K).map((r) => parseInt(r.key.slice(1))));
144
+ let hits = 0;
145
+ for (const idx of groundTruth[i]) if (resultIndices.has(idx)) hits++;
146
+ recallSum += hits / TOP_K;
147
+ }
148
+ const recall = recallSum / N_QUERIES;
149
+
150
+ const avgNs = (searchMs * 1_000_000) / N_QUERIES;
151
+ const qps = (N_QUERIES / searchMs) * 1_000;
152
+
153
+ console.log('Search results:');
154
+ console.log(` Latency: ${fmtNs(avgNs)}/query avg`);
155
+ console.log(` Throughput: ${qps.toFixed(0)} qps`);
156
+ console.log(` Nodes visited/query: ${(hnsw.nodesVisitedCount / N_QUERIES).toFixed(1)}`);
157
+ console.log(` Recall@${TOP_K}: ${(recall * 100).toFixed(1)}%\n`);
@@ -0,0 +1,144 @@
1
+ # YCSB-style CRUD load test
2
+
3
+ A [YCSB](https://github.com/brianfrankcooper/YCSB)-style load test that drives
4
+ CRUD operations against Harper over the **HTTP REST interface**. It boots a real
5
+ Harper instance using [`@harperfast/integration-testing`](https://github.com/HarperFast/integration-testing-framework),
6
+ loads a dataset, then runs the standard YCSB workloads and reports throughput and
7
+ latency percentiles. Results are written as JSON for tracking over time (e.g. the
8
+ nightly workflow in `.github/workflows/ycsb-nightly.yml`).
9
+
10
+ ## What it measures
11
+
12
+ A record is a string primary key plus N string fields (YCSB default: 10 × 100 B).
13
+ Operations map onto REST as:
14
+
15
+ | Operation | REST |
16
+ | --------- | -------------------------------------------- |
17
+ | read | `GET /usertable/<key>` |
18
+ | insert | `PUT /usertable/<key>` (full record) |
19
+ | update | `PUT /usertable/<key>` (full record replace) |
20
+ | rmw | `GET` then `PUT` |
21
+ | scan | `GET /usertable/?id>=<key>&limit(<n>)` |
22
+
23
+ The runner is **closed-loop**: a fixed number of in-flight requests
24
+ (`--concurrency`) each issue their next request as soon as the previous completes,
25
+ so throughput reflects the server rather than a fixed injection rate.
26
+
27
+ ### Workloads
28
+
29
+ | Workload | Mix | Distribution |
30
+ | -------- | ---------------------------------- | ------------ |
31
+ | A | 50% read / 50% update | zipfian |
32
+ | B | 95% read / 5% update | zipfian |
33
+ | C | 100% read | zipfian |
34
+ | D | 95% read / 5% insert (read latest) | latest |
35
+ | E | 95% scan / 5% insert | zipfian |
36
+ | F | 50% read / 50% read-modify-write | zipfian |
37
+
38
+ Key distributions: `uniform`, `zipfian` (scrambled hotspot, YCSB default), and
39
+ `latest` (biased toward recently inserted keys). Override per-run with
40
+ `--distribution`.
41
+
42
+ ## Prerequisites
43
+
44
+ ```sh
45
+ npm run build # the test spawns dist/bin/harper.js
46
+ ```
47
+
48
+ On macOS/Windows, configure loopback addresses once (Linux has them by default):
49
+
50
+ ```sh
51
+ npx harper-integration-test-setup-loopback
52
+ ```
53
+
54
+ ## Running (single node)
55
+
56
+ ```sh
57
+ node benchmarks/ycsb/run-single-node.mts --scale=standard
58
+ node benchmarks/ycsb/run-single-node.mts --scale=quick --workloads=C,A
59
+ node benchmarks/ycsb/run-single-node.mts --engine=lmdb --threads=4
60
+ ```
61
+
62
+ ### Scales
63
+
64
+ | Scale | records | ops/workload | concurrency |
65
+ | ---------- | ------- | ------------ | ----------- |
66
+ | `quick` | 50k | 100k | 32 |
67
+ | `standard` | 200k | 500k | 64 |
68
+ | `heavy` | 1M | 4M | 128 |
69
+
70
+ ### Flags
71
+
72
+ `--scale` `--records` `--ops` `--concurrency` `--load-concurrency` `--fields`
73
+ `--field-length` `--scan-max` `--warmup` `--workloads` `--distribution`
74
+ `--engine` (`rocksdb`|`lmdb`) `--threads` `--out` `--startup-timeout`.
75
+ Individual flags override the scale preset.
76
+
77
+ ## Output
78
+
79
+ - Console: a per-workload table of throughput and p50/p95/p99/max latency.
80
+ - JSON: `benchmarks/ycsb/results/ycsb-single-node-<timestamp>.json` plus a stable
81
+ `ycsb-single-node-latest.json`. The `results/` directory is git-ignored.
82
+
83
+ The JSON includes `meta` (git commit/branch, node version, platform),
84
+ `config` (scale, threads, engine), `load`, and per-workload throughput +
85
+ latency stats — enough to diff runs across nights.
86
+
87
+ ### Trend tracking & regression alerts
88
+
89
+ The nightly workflow feeds results to
90
+ [`github-action-benchmark`](https://github.com/benchmark-action/github-action-benchmark)
91
+ via `to-benchmark-json.mts` (throughput as bigger-is-better, p99 latency as
92
+ smaller-is-better). History is pushed to the `gh-pages` branch and a regression
93
+ beyond the configured threshold comments on the commit and `@`-mentions the
94
+ maintainer. **Enable GitHub Pages on `gh-pages` to view the trend dashboard.**
95
+ Thresholds are deliberately generous to absorb shared-runner variance — tighten
96
+ them once the benchmark runs on a fixed/self-hosted runner.
97
+
98
+ ## Notes
99
+
100
+ - `threads.count` is set via `HARPER_SET_CONFIG` (defaults to 4), which takes
101
+ precedence over the framework's default of 1.
102
+ - Instances run with `AUTHENTICATION_AUTHORIZELOCAL=true`, so loopback requests
103
+ are authorized without per-request auth — the benchmark measures CRUD, not auth.
104
+ - Only the primary key is indexed, matching the YCSB access model.
105
+ - The schema declares `field0..field9`; keep `--fields` ≤ 10 (or edit
106
+ `app/schema.graphql`). Higher counts are stored as dynamic attributes, which
107
+ changes the load-phase write path and isn't directly comparable.
108
+
109
+ ### Reading the numbers (vs. reference YCSB)
110
+
111
+ These are trend signals on a fixed host, not a 1:1 reproduction of the Java YCSB:
112
+
113
+ - **Throughput** is _successful_ ops/sec over wall-clock time; errors are
114
+ reported separately. Clean runs (the expectation) report 0 errors, so the two
115
+ definitions coincide.
116
+ - **Latency percentiles** use the nearest-rank method (slightly conservative vs.
117
+ interpolated / HdrHistogram), consistent across runs — don't compare them 1:1
118
+ to HdrHistogram p99s.
119
+ - The **`latest`** distribution (workload D) is approximate: a scrambled-zipfian
120
+ value mod the keyspace, biased toward newest keys but with a uniform tail.
121
+ Faithful enough for trends, not identical to YCSB's `LatestGenerator`.
122
+
123
+ ## Profiling
124
+
125
+ `--profile` runs the HTTP server on the main thread (`threads.count=0`) and
126
+ sets `--cpu-prof`, so a single V8 CPU profile captures request handling.
127
+ (Harper's worker threads use a fixed `execArgv` that `--cpu-prof` can't reach,
128
+ hence the single-thread mode for profiling.)
129
+
130
+ ```sh
131
+ node benchmarks/ycsb/run-single-node.mts --profile --records=100000 --ops=300000 --workloads=A
132
+ node benchmarks/ycsb/analyze-profile.mts benchmarks/ycsb/results/profile
133
+ ```
134
+
135
+ The analyzer prints self-time by module/category and the hottest functions. The
136
+ profile spans startup + load + the run phase, so one-time costs (module loading,
137
+ cert/key generation) appear alongside steady-state request handling — discount
138
+ those when reading the results.
139
+
140
+ ## Cluster
141
+
142
+ The 3-node cluster variant (harper-pro) reuses `workload.mts`, `restClient.mts`,
143
+ and `harness.mts` from here, starting three connected nodes and round-robining
144
+ load across them.