@aifabrix/builder 2.42.1 → 2.44.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (392) hide show
  1. package/.cursor/rules/anchor-docs.mdc +15 -0
  2. package/README.md +2 -2
  3. package/anchor-docs/README.md +10 -0
  4. package/anchor-docs/_TEMPLATE +24 -0
  5. package/bin/aifabrix.js +13 -4
  6. package/integration/hubspot-test/README.md +157 -0
  7. package/integration/{hubspot → hubspot-test}/application.json +6 -6
  8. package/integration/{hubspot → hubspot-test}/create-hubspot.js +10 -10
  9. package/integration/hubspot-test/env.template +4 -0
  10. package/integration/hubspot-test/hubspot-test-datasource-company.json +138 -0
  11. package/integration/hubspot-test/hubspot-test-datasource-contact.json +146 -0
  12. package/integration/hubspot-test/hubspot-test-datasource-deal.json +146 -0
  13. package/integration/hubspot-test/hubspot-test-datasource-users.json +76 -0
  14. package/integration/{hubspot/hubspot-deploy.json → hubspot-test/hubspot-test-deploy.json} +201 -24
  15. package/integration/{hubspot/hubspot-system.json → hubspot-test/hubspot-test-system.json} +8 -7
  16. package/integration/hubspot-test/rbac.json +166 -0
  17. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-hubspot-credential-real.yaml +3 -3
  18. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-hubspot-env-vars.yaml +2 -2
  19. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-add-datasource.yaml +1 -1
  20. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-credential-create.yaml +1 -1
  21. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-credential-select.yaml +1 -1
  22. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-known-platform.yaml +1 -1
  23. package/integration/hubspot-test/test-artifacts/wizard-invalid-missing-source.yaml +2 -0
  24. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-mode.yaml +1 -1
  25. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-openapi-file.yaml +1 -1
  26. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-openapi-url.yaml +1 -1
  27. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-source.yaml +1 -1
  28. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-valid-for-dimension-array-test.yaml +1 -1
  29. package/integration/hubspot-test/test-artifacts/wizard-valid-for-dimension-key-test.yaml +5 -0
  30. package/integration/hubspot-test/test-artifacts/wizard-valid-for-dimension-path-test.yaml +5 -0
  31. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-valid-for-dimension-test.yaml +1 -1
  32. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-valid-for-rbac-test.yaml +1 -1
  33. package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-valid-for-rbac-yaml-test.yaml +1 -1
  34. package/integration/{hubspot → hubspot-test}/test-dataplane-down-tests.js +1 -7
  35. package/integration/{hubspot → hubspot-test}/test-dataplane-down.js +3 -3
  36. package/integration/{hubspot → hubspot-test}/test.js +137 -102
  37. package/integration/{hubspot → hubspot-test}/wizard-hubspot-e2e.yaml +2 -2
  38. package/integration/{hubspot → hubspot-test}/wizard-hubspot-platform.yaml +1 -1
  39. package/integration/hubspot-test/wizard-hubspot-test-headless.yaml +23 -0
  40. package/integration/roundtrip-test-local/README.md +144 -0
  41. package/integration/roundtrip-test-local/application.yaml +13 -0
  42. package/integration/roundtrip-test-local/env.template +15 -0
  43. package/integration/roundtrip-test-local/roundtrip-test-local-datasource-roundtrip-test-company.yaml +14 -0
  44. package/integration/roundtrip-test-local/roundtrip-test-local-deploy.json +61 -0
  45. package/integration/roundtrip-test-local/roundtrip-test-local-system.yaml +25 -0
  46. package/integration/roundtrip-test-local2/README.md +144 -0
  47. package/integration/roundtrip-test-local2/application.yaml +13 -0
  48. package/integration/roundtrip-test-local2/env.template +15 -0
  49. package/integration/roundtrip-test-local2/roundtrip-test-local2-datasource-company.yaml +31 -0
  50. package/integration/roundtrip-test-local2/roundtrip-test-local2-deploy.json +86 -0
  51. package/integration/roundtrip-test-local2/roundtrip-test-local2-system.yaml +25 -0
  52. package/integration/test/wizard.yaml +8 -0
  53. package/jest.config.default.js +10 -0
  54. package/jest.config.integration.fixtures.js +22 -0
  55. package/jest.config.integration.js +21 -18
  56. package/jest.config.isolated.js +10 -0
  57. package/jest.projects.js +288 -0
  58. package/lib/api/datasources-core.api.js +3 -3
  59. package/lib/api/dev-mtls-request.js +110 -0
  60. package/lib/api/dev-server-https.js +145 -0
  61. package/lib/api/dev.api.js +133 -144
  62. package/lib/api/index.js +0 -1
  63. package/lib/api/pipeline.api.js +67 -20
  64. package/lib/api/service-users.api.js +111 -2
  65. package/lib/api/types/dev.types.js +4 -3
  66. package/lib/api/types/pipeline.types.js +8 -5
  67. package/lib/api/types/service-users.types.js +41 -0
  68. package/lib/api/types/validation-run.types.js +56 -0
  69. package/lib/api/validation-run.api.js +99 -0
  70. package/lib/api/validation-runner.js +99 -0
  71. package/lib/app/config.js +1 -1
  72. package/lib/app/deploy-status-display.js +2 -2
  73. package/lib/app/deploy.js +7 -6
  74. package/lib/app/display.js +2 -1
  75. package/lib/app/dockerfile.js +3 -2
  76. package/lib/app/down.js +2 -1
  77. package/lib/app/helpers.js +6 -5
  78. package/lib/app/index.js +27 -8
  79. package/lib/app/list.js +7 -6
  80. package/lib/app/push.js +4 -3
  81. package/lib/app/register.js +19 -8
  82. package/lib/app/rotate-secret.js +17 -13
  83. package/lib/app/run-container-start.js +184 -0
  84. package/lib/app/run-docker-fallback.js +108 -0
  85. package/lib/app/run-env-compose.js +30 -42
  86. package/lib/app/run-helpers.js +49 -126
  87. package/lib/app/run-infra-requirements.js +30 -0
  88. package/lib/app/run-resolve-image.js +21 -0
  89. package/lib/app/run.js +74 -21
  90. package/lib/app/show-display.js +1 -1
  91. package/lib/app/show.js +1 -1
  92. package/lib/build/index.js +13 -10
  93. package/lib/cli/index.js +2 -0
  94. package/lib/cli/setup-app.help.js +67 -0
  95. package/lib/cli/setup-app.js +59 -123
  96. package/lib/cli/setup-app.test-commands.js +179 -0
  97. package/lib/cli/setup-auth.js +36 -14
  98. package/lib/cli/setup-credential-deployment.js +22 -8
  99. package/lib/cli/setup-dev-path-commands.js +124 -0
  100. package/lib/cli/setup-dev.js +190 -103
  101. package/lib/cli/setup-environment.js +11 -20
  102. package/lib/cli/setup-external-system.js +62 -22
  103. package/lib/cli/setup-infra.js +139 -47
  104. package/lib/cli/setup-parameters.js +32 -0
  105. package/lib/cli/setup-secrets.js +147 -10
  106. package/lib/cli/setup-service-user.js +146 -20
  107. package/lib/cli/setup-utility.js +47 -19
  108. package/lib/commands/app-down.js +5 -7
  109. package/lib/commands/app-install.js +14 -7
  110. package/lib/commands/app-logs.js +13 -10
  111. package/lib/commands/app-shell.js +4 -1
  112. package/lib/commands/app-test.js +25 -19
  113. package/lib/commands/app.js +22 -10
  114. package/lib/commands/auth-config.js +10 -14
  115. package/lib/commands/auth-status.js +4 -3
  116. package/lib/commands/credential-env.js +4 -3
  117. package/lib/commands/credential-list.js +5 -4
  118. package/lib/commands/credential-push.js +4 -3
  119. package/lib/commands/datasource-unified-test-cli.js +495 -0
  120. package/lib/commands/datasource-unified-test-cli.options.js +149 -0
  121. package/lib/commands/datasource-validation-cli.js +129 -0
  122. package/lib/commands/datasource.js +123 -71
  123. package/lib/commands/deployment-list.js +6 -5
  124. package/lib/commands/dev-cli-handlers.js +122 -18
  125. package/lib/commands/dev-down.js +4 -3
  126. package/lib/commands/dev-init.js +231 -116
  127. package/lib/commands/dev-show-display.js +473 -0
  128. package/lib/commands/login-credentials.js +3 -2
  129. package/lib/commands/login-device.js +4 -3
  130. package/lib/commands/login.js +5 -4
  131. package/lib/commands/logout.js +8 -7
  132. package/lib/commands/parameters-validate.js +54 -0
  133. package/lib/commands/repair-datasource.js +314 -68
  134. package/lib/commands/repair-env-template.js +16 -10
  135. package/lib/commands/repair-rbac.js +25 -19
  136. package/lib/commands/repair.js +116 -32
  137. package/lib/commands/secrets-list.js +23 -12
  138. package/lib/commands/secrets-remove-all.js +220 -0
  139. package/lib/commands/secrets-remove.js +22 -13
  140. package/lib/commands/secrets-set.js +21 -12
  141. package/lib/commands/secrets-validate.js +20 -7
  142. package/lib/commands/secure.js +10 -9
  143. package/lib/commands/service-user.js +243 -13
  144. package/lib/commands/test-e2e-external.js +27 -1
  145. package/lib/commands/up-common.js +28 -2
  146. package/lib/commands/up-dataplane.js +31 -18
  147. package/lib/commands/up-miso.js +19 -29
  148. package/lib/commands/upload.js +138 -39
  149. package/lib/commands/wizard-core-helpers.js +1 -1
  150. package/lib/commands/wizard-dataplane.js +4 -3
  151. package/lib/commands/wizard-helpers.js +3 -3
  152. package/lib/commands/wizard.js +2 -2
  153. package/lib/core/admin-secrets.js +16 -5
  154. package/lib/core/audit-logger.js +12 -4
  155. package/lib/core/config-attach-extensions.js +46 -0
  156. package/lib/core/config-runtime-paths.js +29 -0
  157. package/lib/core/config.js +59 -58
  158. package/lib/core/diff.js +3 -2
  159. package/lib/core/ensure-encryption-key.js +2 -4
  160. package/lib/core/secrets-ensure-infra.js +77 -0
  161. package/lib/core/secrets-ensure.js +120 -64
  162. package/lib/core/secrets-env-write.js +35 -7
  163. package/lib/core/secrets-infra-placeholder-sync.js +61 -0
  164. package/lib/core/secrets.js +228 -42
  165. package/lib/core/templates-env.js +4 -3
  166. package/lib/core/templates.js +1 -1
  167. package/lib/datasource/abac-validator.js +148 -0
  168. package/lib/datasource/deploy.js +75 -53
  169. package/lib/datasource/field-reference-validator.js +77 -36
  170. package/lib/datasource/integration-context.js +63 -0
  171. package/lib/datasource/list.js +8 -7
  172. package/lib/datasource/log-viewer.js +252 -0
  173. package/lib/datasource/resolve-app.js +109 -0
  174. package/lib/datasource/test-e2e.js +95 -155
  175. package/lib/datasource/test-integration.js +121 -109
  176. package/lib/datasource/unified-validation-run-body.js +65 -0
  177. package/lib/datasource/unified-validation-run-post.js +23 -0
  178. package/lib/datasource/unified-validation-run-resolve.js +43 -0
  179. package/lib/datasource/unified-validation-run.js +92 -0
  180. package/lib/datasource/validate.js +162 -15
  181. package/lib/deployment/deployer.js +4 -3
  182. package/lib/deployment/environment.js +7 -6
  183. package/lib/deployment/push.js +17 -8
  184. package/lib/external-system/delete.js +4 -3
  185. package/lib/external-system/deploy.js +131 -53
  186. package/lib/external-system/download-helpers.js +1 -1
  187. package/lib/external-system/download.js +7 -6
  188. package/lib/external-system/generator.js +104 -14
  189. package/lib/external-system/integration-test-dispatch.js +26 -0
  190. package/lib/external-system/test-execution.js +5 -1
  191. package/lib/external-system/test-helpers.js +0 -4
  192. package/lib/external-system/test-system-level-helpers.js +110 -0
  193. package/lib/external-system/test-system-level.js +83 -44
  194. package/lib/external-system/test.js +59 -8
  195. package/lib/generator/builders.js +23 -11
  196. package/lib/generator/deploy-manifest-azure-kv.js +81 -0
  197. package/lib/generator/external-controller-manifest.js +3 -3
  198. package/lib/generator/external.js +23 -11
  199. package/lib/generator/helpers.js +71 -12
  200. package/lib/generator/index.js +8 -4
  201. package/lib/generator/split-readme.js +12 -7
  202. package/lib/generator/split-variables.js +2 -1
  203. package/lib/generator/split.js +46 -11
  204. package/lib/generator/wizard-readme.js +3 -3
  205. package/lib/generator/wizard.js +16 -13
  206. package/lib/infrastructure/compose.js +60 -6
  207. package/lib/infrastructure/helpers.js +238 -51
  208. package/lib/infrastructure/index.js +64 -37
  209. package/lib/infrastructure/services.js +21 -15
  210. package/lib/internal/fs-real-sync.js +104 -0
  211. package/lib/internal/node-fs.js +98 -0
  212. package/lib/parameters/database-secret-values.js +173 -0
  213. package/lib/parameters/infra-kv-discovery.js +121 -0
  214. package/lib/parameters/infra-parameter-catalog.js +458 -0
  215. package/lib/parameters/infra-parameter-validate.js +64 -0
  216. package/lib/schema/application-schema.json +37 -17
  217. package/lib/schema/datasource-test-run.schema.json +493 -0
  218. package/lib/schema/deployment-rules.yaml +102 -63
  219. package/lib/schema/external-datasource.schema.json +1201 -433
  220. package/lib/schema/external-system.schema.json +181 -5
  221. package/lib/schema/flag-map-validation-run.json +31 -0
  222. package/lib/schema/infra-parameter.schema.json +106 -0
  223. package/lib/schema/infra.parameter.yaml +421 -0
  224. package/lib/schema/type/credential-auth-templates.json +40 -0
  225. package/lib/schema/type/document-storage.json +213 -0
  226. package/lib/schema/type/message-service.json +123 -0
  227. package/lib/schema/type/vector-store.json +88 -0
  228. package/lib/utils/aifabrix-runtime-config-dir.js +132 -0
  229. package/lib/utils/api-error-handler.js +2 -2
  230. package/lib/utils/api.js +49 -14
  231. package/lib/utils/app-config-resolver.js +23 -1
  232. package/lib/utils/app-register-api.js +3 -2
  233. package/lib/utils/app-register-auth.js +1 -1
  234. package/lib/utils/app-register-config.js +4 -4
  235. package/lib/utils/app-register-display.js +3 -2
  236. package/lib/utils/app-register-validator.js +3 -2
  237. package/lib/utils/app-run-containers.js +26 -22
  238. package/lib/utils/app-scoped-config.js +31 -0
  239. package/lib/utils/app-service-env-from-builder.js +164 -0
  240. package/lib/utils/build-copy.js +1 -1
  241. package/lib/utils/build-helpers.js +20 -20
  242. package/lib/utils/build-resolve-image.js +165 -0
  243. package/lib/utils/cli-layout-chalk.js +8 -0
  244. package/lib/utils/cli-test-layout-chalk.js +267 -0
  245. package/lib/utils/cli-utils.js +88 -11
  246. package/lib/utils/compose-db-passwords.js +138 -0
  247. package/lib/utils/compose-generate-docker-compose.js +216 -0
  248. package/lib/utils/compose-generator.js +197 -291
  249. package/lib/utils/compose-miso-env.js +18 -0
  250. package/lib/utils/compose-traefik-ingress-base.js +158 -0
  251. package/lib/utils/config-paths.js +209 -6
  252. package/lib/utils/config-scoped-resources-preference.js +41 -0
  253. package/lib/utils/controller-deployment-outcome.js +68 -0
  254. package/lib/utils/credential-display.js +2 -2
  255. package/lib/utils/credential-secrets-env.js +16 -1
  256. package/lib/utils/dataplane-pipeline-warning.js +4 -3
  257. package/lib/utils/datasource-test-run-capability-scope.js +43 -0
  258. package/lib/utils/datasource-test-run-debug-display.js +137 -0
  259. package/lib/utils/datasource-test-run-debug-slice.js +93 -0
  260. package/lib/utils/datasource-test-run-display.js +442 -0
  261. package/lib/utils/datasource-test-run-exit.js +58 -0
  262. package/lib/utils/datasource-test-run-legacy-adapter.js +93 -0
  263. package/lib/utils/datasource-test-run-report-version.js +51 -0
  264. package/lib/utils/datasource-test-run-schema-sync.js +59 -0
  265. package/lib/utils/datasource-test-run-tty-log.js +81 -0
  266. package/lib/utils/datasource-validation-watch.js +266 -0
  267. package/lib/utils/declarative-url-ports.js +47 -0
  268. package/lib/utils/derive-env-key-from-client-id.js +41 -0
  269. package/lib/utils/dev-ca-install.js +185 -23
  270. package/lib/utils/dev-cert-helper.js +266 -17
  271. package/lib/utils/dev-hosts-helper.js +307 -0
  272. package/lib/utils/dev-init-cert-hints.js +37 -0
  273. package/lib/utils/dev-init-health-messages.js +52 -0
  274. package/lib/utils/dev-init-resolve.js +86 -0
  275. package/lib/utils/dev-init-ssh-merge.js +65 -0
  276. package/lib/utils/dev-ssh-config-helper.js +196 -0
  277. package/lib/utils/dev-user-groups.js +93 -0
  278. package/lib/utils/docker-build.js +42 -17
  279. package/lib/utils/docker-exec.js +28 -0
  280. package/lib/utils/docker-manifest-public-port.js +116 -0
  281. package/lib/utils/docker-not-running-hint.js +52 -0
  282. package/lib/utils/docker.js +98 -11
  283. package/lib/utils/ensure-dev-certs-for-remote-docker.js +192 -0
  284. package/lib/utils/env-config-loader.js +10 -91
  285. package/lib/utils/env-copy.js +19 -10
  286. package/lib/utils/env-map.js +42 -11
  287. package/lib/utils/env-template.js +2 -2
  288. package/lib/utils/environment-scoped-resources.js +144 -0
  289. package/lib/utils/error-formatter.js +125 -9
  290. package/lib/utils/error-formatters/http-status-errors.js +6 -5
  291. package/lib/utils/error-formatters/network-errors.js +2 -1
  292. package/lib/utils/error-formatters/permission-errors.js +2 -1
  293. package/lib/utils/error-formatters/validation-errors.js +2 -1
  294. package/lib/utils/external-env-template.js +180 -0
  295. package/lib/utils/external-readme.js +8 -1
  296. package/lib/utils/external-system-display.js +277 -136
  297. package/lib/utils/external-system-local-test-tty.js +389 -0
  298. package/lib/utils/external-system-readiness-core.js +377 -0
  299. package/lib/utils/external-system-readiness-deploy-display.js +270 -0
  300. package/lib/utils/external-system-readiness-display-internals.js +150 -0
  301. package/lib/utils/external-system-readiness-display.js +186 -0
  302. package/lib/utils/external-system-test-helpers.js +24 -6
  303. package/lib/utils/external-system-validators.js +32 -14
  304. package/lib/utils/health-check-url.js +119 -0
  305. package/lib/utils/health-check.js +59 -25
  306. package/lib/utils/help-builder.js +14 -13
  307. package/lib/utils/image-version.js +4 -8
  308. package/lib/utils/infra-containers.js +4 -7
  309. package/lib/utils/infra-env-defaults.js +162 -0
  310. package/lib/utils/infra-status-display.js +167 -0
  311. package/lib/utils/infra-status.js +16 -8
  312. package/lib/utils/local-secrets.js +29 -7
  313. package/lib/utils/paths.js +136 -48
  314. package/lib/utils/port-resolver.js +10 -23
  315. package/lib/utils/redis-env-scope.js +62 -0
  316. package/lib/utils/register-aifabrix-shell-env.js +204 -0
  317. package/lib/utils/remote-builder-validation.js +99 -0
  318. package/lib/utils/remote-dev-auth.js +117 -21
  319. package/lib/utils/remote-docker-env.js +67 -15
  320. package/lib/utils/remote-secrets-loader.js +13 -4
  321. package/lib/utils/resolve-docker-image-ref.js +124 -0
  322. package/lib/utils/schema-loader.js +22 -9
  323. package/lib/utils/secrets-bash-kv.js +25 -0
  324. package/lib/utils/secrets-generator.js +171 -51
  325. package/lib/utils/secrets-helpers.js +70 -59
  326. package/lib/utils/secrets-kv-scope.js +60 -0
  327. package/lib/utils/secrets-utils.js +35 -37
  328. package/lib/utils/secrets-validation.js +3 -1
  329. package/lib/utils/secrets-yaml-preserve.js +109 -0
  330. package/lib/utils/secure-file-permissions.js +91 -0
  331. package/lib/utils/ssh-key-helper.js +4 -2
  332. package/lib/utils/template-helpers.js +2 -2
  333. package/lib/utils/test-log-writer.js +3 -3
  334. package/lib/utils/token-manager.js +37 -5
  335. package/lib/utils/url-declarative-public-base.js +188 -0
  336. package/lib/utils/url-declarative-resolve-build.js +493 -0
  337. package/lib/utils/url-declarative-resolve-load-doc.js +51 -0
  338. package/lib/utils/url-declarative-resolve.js +220 -0
  339. package/lib/utils/url-declarative-token-parse.js +74 -0
  340. package/lib/utils/url-declarative-url-flags.js +50 -0
  341. package/lib/utils/url-declarative-vdir-inactive-env.js +99 -0
  342. package/lib/utils/url-public-path-prefix.js +34 -0
  343. package/lib/utils/urls-local-registry.js +220 -0
  344. package/lib/utils/validation-report-tty-kit.js +77 -0
  345. package/lib/utils/validation-run-poll.js +89 -0
  346. package/lib/utils/validation-run-post-retry.js +73 -0
  347. package/lib/utils/validation-run-request.js +98 -0
  348. package/lib/utils/variable-transformer.js +21 -4
  349. package/lib/utils/yaml-preserve.js +78 -1
  350. package/lib/validation/datasource-warnings.js +56 -0
  351. package/lib/validation/env-template-auth.js +50 -2
  352. package/lib/validation/external-manifest-validator.js +35 -7
  353. package/lib/validation/validate-display.js +37 -31
  354. package/lib/validation/validate.js +9 -10
  355. package/lib/validation/validator-unresolved-placeholders.js +98 -0
  356. package/lib/validation/validator.js +32 -78
  357. package/lib/validation/wizard-config-validator.js +2 -1
  358. package/package.json +11 -3
  359. package/scripts/check-datasource-test-run-schema-sync.js +34 -0
  360. package/scripts/diagnose-cli.js +150 -0
  361. package/scripts/install-local.js +304 -55
  362. package/templates/README.md +15 -2
  363. package/templates/applications/dataplane/application.yaml +52 -2
  364. package/templates/applications/dataplane/env.template +80 -18
  365. package/templates/applications/dataplane/rbac.yaml +8 -0
  366. package/templates/applications/keycloak/application.yaml +9 -1
  367. package/templates/applications/keycloak/env.template +15 -6
  368. package/templates/applications/miso-controller/application.yaml +10 -2
  369. package/templates/applications/miso-controller/env.template +55 -14
  370. package/templates/applications/miso-controller/rbac.yaml +5 -0
  371. package/templates/external-system/README.md.hbs +20 -7
  372. package/templates/external-system/deploy.js.hbs +5 -5
  373. package/templates/external-system/env.template.hbs +22 -0
  374. package/templates/external-system/external-datasource.yaml.hbs +197 -118
  375. package/templates/infra/compose.yaml.hbs +20 -4
  376. package/templates/python/docker-compose.hbs +16 -0
  377. package/templates/typescript/docker-compose.hbs +16 -0
  378. package/integration/hubspot/README.md +0 -102
  379. package/integration/hubspot/env.template +0 -4
  380. package/integration/hubspot/hubspot-datasource-company.json +0 -541
  381. package/integration/hubspot/hubspot-datasource-contact.json +0 -639
  382. package/integration/hubspot/hubspot-datasource-deal.json +0 -588
  383. package/integration/hubspot/hubspot-datasource-users.json +0 -116
  384. package/integration/hubspot/test-artifacts/wizard-invalid-missing-source.yaml +0 -2
  385. package/integration/hubspot/test-artifacts/wizard-valid-for-dimension-key-test.yaml +0 -5
  386. package/integration/hubspot/test-artifacts/wizard-valid-for-dimension-path-test.yaml +0 -5
  387. package/lib/api/external-test.api.js +0 -111
  388. package/lib/schema/env-config.yaml +0 -43
  389. /package/integration/{hubspot → hubspot-test}/companies.json +0 -0
  390. /package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-app-name.yaml +0 -0
  391. /package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-missing-app.yaml +0 -0
  392. /package/integration/{hubspot → hubspot-test}/test-dataplane-down-helpers.js +0 -0
@@ -1,47 +1,89 @@
1
+ # External datasource definition. Validate against lib/schema/external-datasource.schema.json (e.g. aifabrix validate / CI).
2
+ # Normative usage (metadata, FKs, dimensions, expressions, execution vs query) aligns with schema v2.4.1 and the Datasource Rules baseline (Plan 346).
3
+ # key: stable identifier (typically <systemKey>-<entity>); referenced by credentials, deploy, and datasource-chaining CIP steps.
1
4
  key: "{{fullDatasourceKey}}"
5
+ # Short name and description shown in tooling and docs.
2
6
  displayName: "{{datasourceDisplayName}}"
3
7
  description: "{{datasourceDescription}}"
8
+ # Parent external system this datasource belongs to (matches external-system key).
4
9
  systemKey: "{{systemKey}}"
10
+ # entityType: recordStorage = structured rows in DB; documentStorage = same family + binary/large content outside metadata_json;
11
+ # vectorStore = retrieval/embedding external store; messageService = channels/notifications; none = orchestration/API composition ONLY.
12
+ # If none: do NOT add metadataSchema, primaryKey, labelKey, dimensions, fieldMappings, sync, documentStorage, etc. (schema forbids them).
5
13
  entityType: "{{schemaEntityType}}"
14
+ # Fine-grained resource classification (e.g. customer, document); used for UX and policies.
6
15
  resourceType: "{{resourceType}}"
16
+ enabled: true
17
+ # File / config version for this YAML (distinct from execution.cip.version when you add CIP below).
18
+ version: "1.0.0"
19
+ {{#unless (eq schemaEntityType "none")}}
20
+ # primaryKey: normalized attribute names used for get/update/delete and storage keys (must exist in metadataSchema + fieldMappings).
7
21
  primaryKey:
8
22
  {{#each primaryKey}} - "{{this}}"
9
23
  {{/each}}
10
- enabled: true
11
- version: "1.0.0"
12
- fieldMappings:
13
- {{#if dimensions}}
14
- dimensions:
15
- {{#each dimensions}}
16
- {{@key}}: "{{this}}"
24
+ # labelKey: attributes used for display labels (search, pickers); often includes primaryKey + human-readable fields.
25
+ labelKey:
26
+ {{#each labelKey}} - "{{this}}"
27
+ {{/each}}
28
+ # metadataSchema: canonical normalized fields (camelCase). Only index=true / filter=true scalars materialize as DB columns; full shape lives in metadata_json.
29
+ # Allowed on fields: type, format, index, filter, nullable, description, string/number validation keywords. Forbidden: anyOf/oneOf/allOf/not/enum/const in author trees.
30
+ # Datetime: type string + format date-time, UTC. Max structural depth per product rules (~2); avoid deep JSON for filter/join paths.
31
+ metadataSchema:
32
+ type: object
33
+ required:
34
+ {{#each attributeKeysForMetadata}} - "{{this}}"
35
+ {{/each}}
36
+ properties:
37
+ {{#each attributeKeysForMetadata}}
38
+ {{this}}:
39
+ type: string
40
+ {{#if (eq this "externalId")}} index: true
41
+ {{else}}{{#if (eq this "id")}} index: true
42
+ {{else}}{{#if @first}} index: true
43
+ {{else}} filter: true
44
+ {{/if}}{{/if}}{{/if}}
45
+ {{/each}}
46
+ {{#if dimensionBindingEntries}}
47
+ # dimensions: top-level only. type local binds to an indexed metadata field; type fk uses via: [{ fk, dimension }]. Optional for global reference data; expected when ABAC/FK joins apply (Plan 346 section 7).
48
+ # Actors: displayName|email|userId → operator eq; groups|roles → in. If actor is omitted, operator should be omitted (operator has no effect without actor).
49
+ dimensions:
50
+ {{#each dimensionBindingEntries}}
51
+ {{dimKey}}:
52
+ type: local
53
+ field: "{{field}}"
54
+ actor: displayName
55
+ operator: eq
17
56
  {{/each}}
18
- {{else}}
19
- dimensions: {}
20
- # Optional: add country, department, organization for ABAC
21
57
  {{/if}}
58
+ # fieldMappings: normalization-only. Each attribute allows only { expression }. Roots MUST be raw.*, fk.*, or dimension.* (e.g. raw.id, fk.company.metadata.name).
59
+ # Heavy transforms belong upstream, in sync, or in entityType none orchestration — not here. Output keys MUST match metadataSchema properties.
60
+ fieldMappings:
22
61
  attributes:
23
62
  {{#if attributes}}
24
63
  {{#each attributes}}
25
64
  {{@key}}:
26
65
  expression: "{{this.expression}}"
27
- type: {{this.type}}
28
- indexed: {{#if this.indexed}}true{{else}}false{{/if}}
29
66
  {{/each}}
30
67
  {{else}}
31
68
  id:
32
69
  expression: "{{raw.id}}"
33
- type: string
34
- indexed: true
35
70
  name:
36
71
  expression: "{{raw.name}}"
37
- type: string
38
- indexed: false
39
72
  {{/if}}
73
+ # foreignKeys (optional): declare join graph to other storage datasources. name + fields[] (indexed metadata fields) + targetDatasource; acyclic; types must match (Plan 346 sections 6 and 8).
74
+ # foreignKeys:
75
+ # - name: company
76
+ # fields: [companyId]
77
+ # targetDatasource: my-system-company
78
+ {{/unless}}
40
79
  {{#if (eq systemType "openapi")}}
80
+ # openapi: links this datasource to the generated OpenAPI document (same systemKey-api artifact). CIP fetch.openapiRef uses keys below: list, get, …
41
81
  openapi:
42
82
  enabled: true
83
+ # Must match the OpenAPI bundle key published for this system (wizard default: <systemKey>-api).
43
84
  documentKey: "{{systemKey}}-api"
44
85
  operations:
86
+ # Standard names list/get/create/update/delete; operationId and path should match your spec. CIP uses openapiRef: list | get.
45
87
  list:
46
88
  operationId: "list{{entityKey}}"
47
89
  method: GET
@@ -50,168 +92,205 @@ openapi:
50
92
  operationId: "get{{entityKey}}"
51
93
  method: GET
52
94
  path: "/{{entityKey}}/{id}"
95
+ # When true, runtime can derive RBAC hints from OpenAPI metadata where supported.
53
96
  autoRbac: true
54
97
  {{/if}}
55
98
 
56
99
  # --- Optional sections: uncomment or delete as needed ---
57
- # CIP (Custom Integration Pipeline) supports: fetch, paginate, map, filter, output, pythonInline steps.
58
- # Operations: list, get, create, update, delete. Pagination: cursor | page | offset.
100
+ #
101
+ # Query-time (v2.4): public list/get/filter are served from persisted Dataplane DB not live external HTTP on every read (Plan 346 section 11).
102
+ # External APIs: sync jobs and create/update/delete execution paths. CIP fetch→map is for ingestion, writes, or orchestration — populate storage before expecting query APIs.
103
+ #
104
+ # CIP (Composable Integration Pipeline): fetch → [paginate] → [map] → [filter] → output
105
+ # Sources: openapi (openapi.operations.<name>), http (method + path), datasource (another datasource key + operation name). Operation names: ^[a-z][a-zA-Z0-9]*$
106
+ # execution.cip requires version "1.0" and operations.<name>.steps ($defs.cipDefinition). Custom operation keys must match the same pattern as capabilities[].
107
+ #
59
108
  {{#if (eq schemaEntityType "recordStorage")}}
109
+ # sync: external APIs run during sync (Plan 346 section 11). Minimal contract: mode/schedule/batchSize per schema. FK fields on a record should update atomically in one logical write.
60
110
  # sync:
61
111
  # pull:
62
112
  # enabled: true
63
113
  # schedule: "0 * * * *"
114
+ # batchSize: 500
115
+ # validation: structural / pre-persistence checks; quality: post-mapping acceptance (e.g. rejectIf) — see schema when you enable them.
116
+ # capabilities: standard names list, get, create, update, delete + custom names matching ^[a-z][a-zA-Z0-9]*$; must match execution.cip.operations / openapi.operations you implement.
64
117
  # capabilities: [list, get]
65
118
  {{/if}}
66
119
  {{#if (eq schemaEntityType "documentStorage")}}
120
+ # documentStorage: binary/attachment flow — which operation fetches bytes and which metadata field holds extractable text for embeddings.
67
121
  # documentStorage:
68
122
  # enabled: true
69
123
  # binaryOperationRef: get
70
124
  # embeddingField: content
71
125
  {{/if}}
72
126
  {{#if (eq schemaEntityType "vectorStore")}}
127
+ # vectorStore: vector index settings (model id, dimensions, etc. per schema). Uncomment and set to match your embedding deployment.
73
128
  # vectorStore:
74
129
  # enabled: true
75
130
  # embeddingModel: "text-embedding-ada-002"
76
131
  {{/if}}
77
132
  {{#if (eq schemaEntityType "messageService")}}
133
+ # messageService: channel-oriented messaging integration; channels list drives subscription/send surfaces.
78
134
  # messageService:
79
135
  # enabled: true
80
136
  # channels: []
81
137
  {{/if}}
82
138
 
83
- # --- execution: CIP pipeline ---
84
- # Steps: fetch (openapi/http) → paginate (cursor|page|offset) → map (useFieldMappings, inputPath) → filter (enforceAbac, expression) → output (mode: records)
85
- # Pagination: cursor=cursorField+cursorParam | page=pageParam+pageSizeParam | offset=offsetParam+pageSizeParam
86
- # Adjust inputPath to your API ($.results[*], $.items[*], $.value[*], etc.)
87
- {{#if (eq schemaEntityType "recordStorage")}}
139
+ {{#unless (eq schemaEntityType "none")}}
140
+ {{#if (eq systemType "openapi")}}
141
+ # Example CIP (commented): matches openapi.operations list/get for "{{entityKey}}" above — copy, uncomment, tune inputPath/query.
142
+ # Alternative engines: execution.engine python (custom handler) or datasource (chain); see schema execution.* properties.
143
+ #
88
144
  # execution:
89
145
  # engine: cip
90
146
  # cip:
147
+ # version: "1.0"
148
+ # # Optional: idempotency / runtimeContext blocks live alongside version at this level (schema $defs.idempotencyConfig, cipRuntimeContext).
91
149
  # operations:
92
150
  # list:
93
151
  # enabled: true
94
- # description: "List all records"
152
+ # description: "List {{entityKey}} records"
95
153
  # steps:
96
- # - fetch: { source: openapi, openapiRef: list, query: {} }
97
- # - paginate: { strategy: cursor, cursorField: "$.paging.next.after", cursorParam: after, pageSize: 100, maxPages: 100 }
98
- # # Alternative: page → pageParam, pageSizeParam | offset → offsetParam, pageSizeParam
99
- # - map: { useFieldMappings: true, inputPath: "$.results[*]" }
100
- # - filter: { enforceAbac: true }
101
- # # Optional: filter.expression for SQL or JSON filter, e.g. expression: "status = 'active'"
102
- # - output: { mode: records }
154
+ # - fetch:
155
+ # source: openapi
156
+ # openapiRef: list
157
+ # query: {}
158
+ # # expectedItemsPath: "$.results"
159
+ # # - paginate:
160
+ # # strategy: cursor
161
+ # # cursorField: "next_cursor"
162
+ # # cursorParam: "cursor"
163
+ # # pageSize: 100
164
+ # # maxPages: 50
165
+ # - map:
166
+ # useFieldMappings: true
167
+ # # inputPath: JSONPath-like slice over the fetch body; must match list payload (array of raw objects before fieldMappings).
168
+ # inputPath: "$.results[*]"
169
+ # # Try $.items[*], $.data[*], etc. if your list lives under a different key (schema default hint is $.items[*]).
170
+ # # inline: { customField: "\{{raw.custom}}" } # optional field override beside useFieldMappings
171
+ # - filter:
172
+ # enforceAbac: true
173
+ # # expression "@": pass records through JMESPath after ABAC; replace with a predicate when you need server-side row filtering.
174
+ # expression: "@"
175
+ # expressionLanguage: jmespath
176
+ # # abac: { mode: mandatory, policyScope: datasource }
177
+ # - output:
178
+ # mode: records
179
+ # # limit: soft cap on list size; sync jobs may override.
180
+ # # limit: 500
181
+ # # includeConfidence: true # optional per-record scoring when supported
103
182
  # get:
104
183
  # enabled: true
105
- # description: "Get record by ID"
106
- # steps:
107
- # - fetch: { source: openapi, openapiRef: get, query: {} }
108
- # - map: { useFieldMappings: true, inputPath: "$" }
109
- # - filter: { enforceAbac: true }
110
- # - output: { mode: records }
111
- # create:
112
- # steps:
113
- # - fetch: { source: openapi, openapiRef: create, bodyTemplate: "{{body}}" }
114
- # - map: { useFieldMappings: true, inputPath: "$" }
115
- # - output: { mode: records }
116
- # update:
117
- # steps:
118
- # - fetch: { source: openapi, openapiRef: update, bodyTemplate: "{{body}}" }
119
- # - map: { useFieldMappings: true, inputPath: "$" }
120
- # - output: { mode: records }
121
- # delete:
184
+ # description: "Single {{entityKey}} by id (path /{{entityKey}}/{id} on openapi.get)"
122
185
  # steps:
123
- # - fetch: { source: openapi, openapiRef: delete }
124
- # - output: { mode: records }
125
- {{/if}}
126
- {{#if (eq schemaEntityType "documentStorage")}}
186
+ # - fetch:
187
+ # source: openapi
188
+ # openapiRef: get
189
+ # query:
190
+ # id: "{{raw.id}}"
191
+ # # If your API uses a different param name, rename the key; path params are resolved per your OpenAPI operation.
192
+ # # onError: { retry: { maxAttempts: 3 } } # optional per-step error policy (schema $defs.cipOnErrorConfig)
193
+ # - map:
194
+ # useFieldMappings: true
195
+ # # Single-object body: "$" or a path to the entity object inside a wrapper response.
196
+ # inputPath: "$"
197
+ # # Wrapped entity: use "$.data" or "$.result"
198
+ # - filter:
199
+ # enforceAbac: true
200
+ # expression: "@"
201
+ # expressionLanguage: jmespath
202
+ # - output:
203
+ # mode: records
204
+ # # Optional: delegate to another datasource's CIP
205
+ # # related:
206
+ # # enabled: true
207
+ # # steps:
208
+ # # - fetch: { source: datasource, datasource: "other-system-entity", operation: list, parameters: {} }
209
+ # # - output: { mode: records }
210
+ {{else}}
211
+ # Example CIP (commented): no openapi block in this file — use http fetch (or add openapi: first, then use the openapi example pattern).
212
+ # http fetch: method + path are required; query holds static query params (runtime may merge ABAC params).
213
+ #
127
214
  # execution:
128
215
  # engine: cip
129
216
  # cip:
217
+ # version: "1.0"
130
218
  # operations:
131
219
  # list:
132
220
  # enabled: true
133
- # description: "List documents"
134
221
  # steps:
135
- # - fetch: { source: openapi, openapiRef: list, query: {} }
136
- # - paginate: { strategy: offset, offsetParam: skip, pageSizeParam: top, pageSize: 100, maxPages: 100 }
137
- # - map: { useFieldMappings: true, inputPath: "$.value[*]" }
138
- # - filter: { enforceAbac: true }
139
- # - output: { mode: records }
222
+ # - fetch:
223
+ # source: http
224
+ # method: GET
225
+ # path: "/{{entityKey}}"
226
+ # query: {}
227
+ # # headers: { Accept: "application/json" }
228
+ # - map:
229
+ # useFieldMappings: true
230
+ # inputPath: "$.items[*]"
231
+ # - filter:
232
+ # enforceAbac: true
233
+ # expression: "@"
234
+ # expressionLanguage: jmespath
235
+ # - output:
236
+ # mode: records
140
237
  # get:
141
238
  # enabled: true
142
- # description: "Get document by ID"
143
- # steps:
144
- # - fetch: { source: openapi, openapiRef: get, query: {} }
145
- # - map: { useFieldMappings: true, inputPath: "$" }
146
- # - filter: { enforceAbac: true }
147
- # - output: { mode: records }
148
- # create:
149
- # enabled: true
150
- # description: "Upload document"
151
239
  # steps:
152
- # - fetch: { source: openapi, openapiRef: create, bodyTemplate: "{{fileContent}}" }
153
- # - map: { useFieldMappings: true, inputPath: "$" }
154
- # - output: { mode: records }
155
- {{/if}}
156
- {{#if (eq schemaEntityType "vectorStore")}}
157
- # execution:
158
- # engine: cip
159
- # cip:
160
- # operations:
161
- # list:
162
- # steps:
163
- # - fetch: { source: openapi, openapiRef: list, query: {} }
164
- # - map: { useFieldMappings: true, inputPath: "$" }
165
- # - output: { mode: records }
166
- # get:
167
- # steps:
168
- # - fetch: { source: openapi, openapiRef: get, query: {} }
169
- # - map: { useFieldMappings: true, inputPath: "$" }
170
- # - output: { mode: records }
171
- {{/if}}
172
- {{#if (eq schemaEntityType "messageService")}}
173
- # execution:
174
- # engine: cip
175
- # cip:
176
- # operations:
177
- # list:
178
- # steps:
179
- # - fetch: { source: openapi, openapiRef: list, query: {} }
180
- # - map: { useFieldMappings: true, inputPath: "$" }
181
- # - output: { mode: records }
240
+ # - fetch:
241
+ # source: http
242
+ # method: GET
243
+ # # Path template must match your API; {id} is illustrative — align with real route and param names.
244
+ # path: "/{{entityKey}}/{id}"
245
+ # query: {}
246
+ # - map:
247
+ # useFieldMappings: true
248
+ # inputPath: "$"
249
+ # - filter:
250
+ # enforceAbac: true
251
+ # expression: "@"
252
+ # expressionLanguage: jmespath
253
+ # - output:
254
+ # mode: records
182
255
  {{/if}}
256
+ {{/unless}}
183
257
  {{#if (eq schemaEntityType "none")}}
184
- # execution:
258
+ # execution (entityType none — orchestration / API composition only; no storage contract on this datasource):
259
+ # Cross-datasource access without FK is not a query-time join: use FK graph, bridge recordStorage, pre-normalize, or fan-out here (Plan 346 section 5).
185
260
  # engine: cip
186
261
  # cip:
262
+ # version: "1.0"
187
263
  # operations:
188
264
  # list:
189
265
  # steps:
190
266
  # - fetch: { source: openapi, openapiRef: list, query: {} }
191
- # - map: { useFieldMappings: true, inputPath: "$" }
192
- # - output: { mode: records }
193
- # get:
194
- # steps:
195
- # - fetch: { source: openapi, openapiRef: get, query: {} }
196
- # - map: { useFieldMappings: true, inputPath: "$" }
197
267
  # - output: { mode: records }
268
+ # # Chain another datasource (no fieldMappings on this datasource):
269
+ # # chain:
270
+ # # steps:
271
+ # # - fetch: { source: datasource, datasource: "upstream-system-entity", operation: list, parameters: {} }
272
+ # # - output: { mode: records }
198
273
  {{/if}}
199
274
 
200
- # config: ABAC cross-system filters (SQL or JSON)
275
+ # config.extensions: vendor-specific knobs; property names must match ^x[A-Z]… (e.g. xHubSpotFeature). Keeps core schema stable.
201
276
  # config:
202
- # abac:
203
- # crossSystemSql: "country = '{{actor.country}}'"
204
- # # crossSystemJson: { "dimensions.country": { "eq": "{{actor.country}}" } }
277
+ # extensions:
278
+ # xHubSpotFeature: {}
205
279
 
206
- # capabilities: [list, get, create, update, delete]
207
-
208
- # exposed: attributes to expose via MCP/OpenAPI
280
+ # exposed.schema: single public response shape for REST/MCP; values are expressions (metadata.*, dimension.*, fk.*.metadata.*). No implicit fields; use exposed.readonly[] / exposed.omit[] for writes (Plan 346 section 13). exposed.attributes is deprecated.
209
281
  # exposed:
210
- # attributes: [id, name]
282
+ # schema:
283
+ # id: metadata.id
284
+ # name: metadata.name
285
+ # # nested objects and fk.* paths are allowed when foreignKeys exist and join rules pass join rules (section 8)
286
+ # # readonly: [status]
287
+ # # omit: [internalScore]
211
288
 
212
- # metadataSchema: JSON Schema for raw metadata validation
213
- # metadataSchema:
214
- # type: object
215
- # properties:
216
- # id: { type: string }
217
- # name: { type: string }
289
+ # testPayload (optional): fixtures for static expression resolution (raw/fk/dimension); root keys are closed in schema — no typos at top level (Plan 346 section 16).
290
+ # testPayload:
291
+ # mode: smoke
292
+ # payloadTemplate:
293
+ # raw: { id: "1", name: "Example" }
294
+
295
+ # capabilities: root-level array preferred (v2.2+); names must match ^[a-z][a-zA-Z0-9]*$.
296
+ # capabilities: [list, get, create, update, delete]
@@ -17,7 +17,9 @@ services:
17
17
  - "{{postgresPort}}:5432"
18
18
  volumes:
19
19
  - {{#if (eq devId 0)}}postgres_data{{else}}dev{{devId}}_postgres_data{{/if}}:/var/lib/postgresql/data
20
- - ./init-scripts:/docker-entrypoint-initdb.d
20
+ - type: bind
21
+ source: "{{initScriptsBind}}"
22
+ target: /docker-entrypoint-initdb.d
21
23
  networks:
22
24
  - {{networkName}}
23
25
  healthcheck:
@@ -61,7 +63,10 @@ services:
61
63
  - "{{pgadminPort}}:80"
62
64
  volumes:
63
65
  - {{#if (eq devId 0)}}pgadmin_data{{else}}dev{{devId}}_pgadmin_data{{/if}}:/var/lib/pgadmin
64
- - {{infraDir}}:/host-config:ro
66
+ - type: bind
67
+ source: "{{infraDirBind}}"
68
+ target: /host-config
69
+ read_only: true
65
70
  command: >
66
71
  sh -c "
67
72
  if [ -f /host-config/servers.json ]; then
@@ -110,8 +115,13 @@ services:
110
115
  - "--providers.docker=true"
111
116
  - "--providers.docker.exposedbydefault=false"
112
117
  - "--providers.docker.network={{networkName}}"
118
+ - "--providers.docker.allowEmptyServices=true"
119
+ - "--log.level=INFO"
113
120
  - "--entrypoints.web.address=:80"
114
121
  - "--entrypoints.websecure.address=:443"
122
+ {{#if traefik.trustForwardedHeaders}}
123
+ - "--entrypoints.web.forwardedHeaders.insecure=true"
124
+ {{/if}}
115
125
  {{#if traefik.certStore}}
116
126
  {{#if traefik.certFile}}
117
127
  - "--certificatesstores.{{traefik.certStore}}.defaultcertificate.certfile={{traefik.certFile}}"
@@ -126,10 +136,16 @@ services:
126
136
  volumes:
127
137
  - /var/run/docker.sock:/var/run/docker.sock:ro
128
138
  {{#if traefik.certFile}}
129
- - {{traefik.certFile}}:{{traefik.certFile}}:ro
139
+ - type: bind
140
+ source: "{{traefik.certFile}}"
141
+ target: "{{traefik.certFile}}"
142
+ read_only: true
130
143
  {{/if}}
131
144
  {{#if traefik.keyFile}}
132
- - {{traefik.keyFile}}:{{traefik.keyFile}}:ro
145
+ - type: bind
146
+ source: "{{traefik.keyFile}}"
147
+ target: "{{traefik.keyFile}}"
148
+ read_only: true
133
149
  {{/if}}
134
150
  networks:
135
151
  - {{networkName}}
@@ -13,16 +13,28 @@ services:
13
13
  {{#if traefik.tls}}
14
14
  - "traefik.http.routers.{{app.key}}.entrypoints=websecure"
15
15
  - "traefik.http.routers.{{app.key}}.tls=true"
16
+ - "traefik.http.routers.{{app.key}}.service={{app.key}}"
16
17
  {{#if traefik.certStore}}
17
18
  - "traefik.http.routers.{{app.key}}.tls.certstore={{traefik.certStore}}"
18
19
  {{/if}}
20
+ - "traefik.http.routers.{{app.key}}-http.rule=Host(`{{traefik.host}}`) && PathPrefix(`{{traefik.path}}`)"
21
+ - "traefik.http.routers.{{app.key}}-http.entrypoints=web"
22
+ {{#unless (eq traefik.path "/")}}
23
+ {{#if traefik.stripPathPrefix}}
24
+ - "traefik.http.routers.{{app.key}}-http.middlewares={{app.key}}-stripprefix"
25
+ {{/if}}
26
+ {{/unless}}
27
+ - "traefik.http.routers.{{app.key}}-http.service={{app.key}}"
19
28
  {{else}}
20
29
  - "traefik.http.routers.{{app.key}}.entrypoints=web"
30
+ - "traefik.http.routers.{{app.key}}.service={{app.key}}"
21
31
  {{/if}}
22
32
  - "traefik.http.services.{{app.key}}.loadbalancer.server.port={{containerPort}}"
23
33
  {{#unless (eq traefik.path "/")}}
34
+ {{#if traefik.stripPathPrefix}}
24
35
  - "traefik.http.middlewares.{{app.key}}-stripprefix.stripprefix.prefixes={{traefik.path}}"
25
36
  - "traefik.http.routers.{{app.key}}.middlewares={{app.key}}-stripprefix"
37
+ {{/if}}
26
38
  {{/unless}}
27
39
  {{/if}}
28
40
  environment:
@@ -59,7 +71,11 @@ services:
59
71
  {{/if}}
60
72
  {{/if}}
61
73
  healthcheck:
74
+ {{#if healthCheck.bashProbe}}
75
+ test: ["CMD-SHELL", "bash -c 'exec 3<>/dev/tcp/127.0.0.1/{{port}} && printf \"GET {{healthCheck.path}} HTTP/1.1\\r\\nHost: localhost\\r\\nConnection: close\\r\\n\\r\\n\" >&3 && head -n1 <&3 | grep -q \"200 OK\"'"]
76
+ {{else}}
62
77
  test: ["CMD", "curl", "-f", "http://localhost:{{port}}{{healthCheck.path}}"]
78
+ {{/if}}
63
79
  interval: {{healthCheck.interval}}s
64
80
  timeout: 10s
65
81
  retries: 3
@@ -13,16 +13,28 @@ services:
13
13
  {{#if traefik.tls}}
14
14
  - "traefik.http.routers.{{app.key}}.entrypoints=websecure"
15
15
  - "traefik.http.routers.{{app.key}}.tls=true"
16
+ - "traefik.http.routers.{{app.key}}.service={{app.key}}"
16
17
  {{#if traefik.certStore}}
17
18
  - "traefik.http.routers.{{app.key}}.tls.certstore={{traefik.certStore}}"
18
19
  {{/if}}
20
+ - "traefik.http.routers.{{app.key}}-http.rule=Host(`{{traefik.host}}`) && PathPrefix(`{{traefik.path}}`)"
21
+ - "traefik.http.routers.{{app.key}}-http.entrypoints=web"
22
+ {{#unless (eq traefik.path "/")}}
23
+ {{#if traefik.stripPathPrefix}}
24
+ - "traefik.http.routers.{{app.key}}-http.middlewares={{app.key}}-stripprefix"
25
+ {{/if}}
26
+ {{/unless}}
27
+ - "traefik.http.routers.{{app.key}}-http.service={{app.key}}"
19
28
  {{else}}
20
29
  - "traefik.http.routers.{{app.key}}.entrypoints=web"
30
+ - "traefik.http.routers.{{app.key}}.service={{app.key}}"
21
31
  {{/if}}
22
32
  - "traefik.http.services.{{app.key}}.loadbalancer.server.port={{containerPort}}"
23
33
  {{#unless (eq traefik.path "/")}}
34
+ {{#if traefik.stripPathPrefix}}
24
35
  - "traefik.http.middlewares.{{app.key}}-stripprefix.stripprefix.prefixes={{traefik.path}}"
25
36
  - "traefik.http.routers.{{app.key}}.middlewares={{app.key}}-stripprefix"
37
+ {{/if}}
26
38
  {{/unless}}
27
39
  {{/if}}
28
40
  environment:
@@ -59,7 +71,11 @@ services:
59
71
  {{/if}}
60
72
  {{/if}}
61
73
  healthcheck:
74
+ {{#if healthCheck.bashProbe}}
75
+ test: ["CMD-SHELL", "bash -c 'exec 3<>/dev/tcp/127.0.0.1/{{port}} && printf \"GET {{healthCheck.path}} HTTP/1.1\\r\\nHost: localhost\\r\\nConnection: close\\r\\n\\r\\n\" >&3 && head -n1 <&3 | grep -q \"200 OK\"'"]
76
+ {{else}}
62
77
  test: ["CMD", "curl", "-f", "http://localhost:{{port}}{{healthCheck.path}}"]
78
+ {{/if}}
63
79
  interval: {{healthCheck.interval}}s
64
80
  timeout: 10s
65
81
  retries: 3
@@ -1,102 +0,0 @@
1
- # HubSpot CRM
2
-
3
- HubSpot CRM integration with OpenAPI support for companies, contacts, and deals
4
-
5
- ## System Information
6
-
7
- - **System Key**: `hubspot`
8
- - **System Type**: `openapi`
9
- - **Datasources**: 4
10
-
11
- ## Files
12
-
13
- - `application.yaml` – Application configuration with `app` and `externalIntegration` blocks
14
- - `hubspot-system.yaml` – External system definition (authentication, OpenAPI/MCP, RBAC)
15
- - `hubspot-datasource-deal.yaml` – Datasource: HubSpot Deal
16
- - `hubspot-datasource-contact.yaml` – Datasource: HubSpot Contact
17
- - `hubspot-datasource-users-datasource.yaml` – Datasource: HubSpot Users
18
- - `hubspot-datasource-company.yaml` – Datasource: HubSpot Company
19
- - `env.template` – Environment variables template (secrets, API keys)
20
- - `hubspot-deploy.json` – Deployment manifest (generated by `aifabrix json`)
21
-
22
- Optional: `rbac.yaml` – Roles and permissions merged into the system when present.
23
-
24
- ## Quick Start
25
-
26
- ### 1. Create External System
27
-
28
- ```bash
29
- aifabrix create hubspot
30
- ```
31
-
32
- (External is the default type. Use `aifabrix create <name> --type webapp` for a builder app.)
33
-
34
- Or use the interactive wizard:
35
-
36
- ```bash
37
- aifabrix wizard --app hubspot
38
- ```
39
-
40
- ### 2. Configure Authentication and Datasources
41
-
42
- Edit files in `integration/hubspot/`:
43
-
44
- - **Authentication**: `hubspot-system.yaml` (auth type, credentials placeholders)
45
- - **Field mappings**: `hubspot-datasource-*.yaml` (dimensions, attributes, operations)
46
-
47
- ### 3. Validate Configuration
48
-
49
- ```bash
50
- aifabrix validate hubspot --type external
51
- ```
52
-
53
- ### 4. Generate Deployment Manifest
54
-
55
- ```bash
56
- aifabrix json hubspot --type external
57
- ```
58
-
59
- This creates `hubspot-deploy.json` in `integration/hubspot/`.
60
-
61
- ### 5. Deploy
62
-
63
- Controller URL and environment are read from config. Configure and log in first:
64
-
65
- ```bash
66
- aifabrix auth config --set-controller <url> --set-environment dev
67
- aifabrix login
68
- ```
69
-
70
- Then deploy:
71
-
72
- ```bash
73
- aifabrix deploy hubspot
74
- ```
75
-
76
- ## Testing
77
-
78
- ### Unit Tests (Local Validation, No API)
79
-
80
- ```bash
81
- aifabrix test hubspot
82
- ```
83
-
84
- ### Integration Tests (Via Dataplane)
85
-
86
- ```bash
87
- aifabrix test-integration hubspot
88
- ```
89
-
90
- ## Deployment
91
-
92
- Deploy via miso-controller pipeline (same as regular apps). Auth and controller come from `aifabrix login` and `aifabrix auth config`:
93
-
94
- ```bash
95
- aifabrix deploy hubspot
96
- ```
97
-
98
- ## Troubleshooting
99
-
100
- - **Validation errors**: Run `aifabrix validate hubspot --type external` to see schema and manifest errors.
101
- - **Deployment / auth**: Run `aifabrix auth config --set-controller <url> --set-environment <env>` and `aifabrix login` before `aifabrix deploy`.
102
- - **File not found**: Run commands from the project root (where `package.json` and `integration/` live).
@@ -1,4 +0,0 @@
1
- HUBSPOT_API_VERSION=v3
2
- MAX_PAGE_SIZE=100
3
- KV_HUBSPOT_CLIENTID=kv://hubspot/clientid
4
- KV_HUBSPOT_CLIENTSECRET=kv://hubspot/clientsecret