@aifabrix/builder 2.42.1 → 2.44.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.cursor/rules/anchor-docs.mdc +15 -0
- package/README.md +2 -2
- package/anchor-docs/README.md +10 -0
- package/anchor-docs/_TEMPLATE +24 -0
- package/bin/aifabrix.js +13 -4
- package/integration/hubspot-test/README.md +157 -0
- package/integration/{hubspot → hubspot-test}/application.json +6 -6
- package/integration/{hubspot → hubspot-test}/create-hubspot.js +10 -10
- package/integration/hubspot-test/env.template +4 -0
- package/integration/hubspot-test/hubspot-test-datasource-company.json +138 -0
- package/integration/hubspot-test/hubspot-test-datasource-contact.json +146 -0
- package/integration/hubspot-test/hubspot-test-datasource-deal.json +146 -0
- package/integration/hubspot-test/hubspot-test-datasource-users.json +76 -0
- package/integration/{hubspot/hubspot-deploy.json → hubspot-test/hubspot-test-deploy.json} +201 -24
- package/integration/{hubspot/hubspot-system.json → hubspot-test/hubspot-test-system.json} +8 -7
- package/integration/hubspot-test/rbac.json +166 -0
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-hubspot-credential-real.yaml +3 -3
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-hubspot-env-vars.yaml +2 -2
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-add-datasource.yaml +1 -1
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-credential-create.yaml +1 -1
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-credential-select.yaml +1 -1
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-known-platform.yaml +1 -1
- package/integration/hubspot-test/test-artifacts/wizard-invalid-missing-source.yaml +2 -0
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-mode.yaml +1 -1
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-openapi-file.yaml +1 -1
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-openapi-url.yaml +1 -1
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-source.yaml +1 -1
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-valid-for-dimension-array-test.yaml +1 -1
- package/integration/hubspot-test/test-artifacts/wizard-valid-for-dimension-key-test.yaml +5 -0
- package/integration/hubspot-test/test-artifacts/wizard-valid-for-dimension-path-test.yaml +5 -0
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-valid-for-dimension-test.yaml +1 -1
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-valid-for-rbac-test.yaml +1 -1
- package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-valid-for-rbac-yaml-test.yaml +1 -1
- package/integration/{hubspot → hubspot-test}/test-dataplane-down-tests.js +1 -7
- package/integration/{hubspot → hubspot-test}/test-dataplane-down.js +3 -3
- package/integration/{hubspot → hubspot-test}/test.js +137 -102
- package/integration/{hubspot → hubspot-test}/wizard-hubspot-e2e.yaml +2 -2
- package/integration/{hubspot → hubspot-test}/wizard-hubspot-platform.yaml +1 -1
- package/integration/hubspot-test/wizard-hubspot-test-headless.yaml +23 -0
- package/integration/roundtrip-test-local/README.md +144 -0
- package/integration/roundtrip-test-local/application.yaml +13 -0
- package/integration/roundtrip-test-local/env.template +15 -0
- package/integration/roundtrip-test-local/roundtrip-test-local-datasource-roundtrip-test-company.yaml +14 -0
- package/integration/roundtrip-test-local/roundtrip-test-local-deploy.json +61 -0
- package/integration/roundtrip-test-local/roundtrip-test-local-system.yaml +25 -0
- package/integration/roundtrip-test-local2/README.md +144 -0
- package/integration/roundtrip-test-local2/application.yaml +13 -0
- package/integration/roundtrip-test-local2/env.template +15 -0
- package/integration/roundtrip-test-local2/roundtrip-test-local2-datasource-company.yaml +31 -0
- package/integration/roundtrip-test-local2/roundtrip-test-local2-deploy.json +86 -0
- package/integration/roundtrip-test-local2/roundtrip-test-local2-system.yaml +25 -0
- package/integration/test/wizard.yaml +8 -0
- package/jest.config.default.js +10 -0
- package/jest.config.integration.fixtures.js +22 -0
- package/jest.config.integration.js +21 -18
- package/jest.config.isolated.js +10 -0
- package/jest.projects.js +288 -0
- package/lib/api/datasources-core.api.js +3 -3
- package/lib/api/dev-mtls-request.js +110 -0
- package/lib/api/dev-server-https.js +145 -0
- package/lib/api/dev.api.js +133 -144
- package/lib/api/index.js +0 -1
- package/lib/api/pipeline.api.js +67 -20
- package/lib/api/service-users.api.js +111 -2
- package/lib/api/types/dev.types.js +4 -3
- package/lib/api/types/pipeline.types.js +8 -5
- package/lib/api/types/service-users.types.js +41 -0
- package/lib/api/types/validation-run.types.js +56 -0
- package/lib/api/validation-run.api.js +99 -0
- package/lib/api/validation-runner.js +99 -0
- package/lib/app/config.js +1 -1
- package/lib/app/deploy-status-display.js +2 -2
- package/lib/app/deploy.js +7 -6
- package/lib/app/display.js +2 -1
- package/lib/app/dockerfile.js +3 -2
- package/lib/app/down.js +2 -1
- package/lib/app/helpers.js +6 -5
- package/lib/app/index.js +27 -8
- package/lib/app/list.js +7 -6
- package/lib/app/push.js +4 -3
- package/lib/app/register.js +19 -8
- package/lib/app/rotate-secret.js +17 -13
- package/lib/app/run-container-start.js +184 -0
- package/lib/app/run-docker-fallback.js +108 -0
- package/lib/app/run-env-compose.js +30 -42
- package/lib/app/run-helpers.js +49 -126
- package/lib/app/run-infra-requirements.js +30 -0
- package/lib/app/run-resolve-image.js +21 -0
- package/lib/app/run.js +74 -21
- package/lib/app/show-display.js +1 -1
- package/lib/app/show.js +1 -1
- package/lib/build/index.js +13 -10
- package/lib/cli/index.js +2 -0
- package/lib/cli/setup-app.help.js +67 -0
- package/lib/cli/setup-app.js +59 -123
- package/lib/cli/setup-app.test-commands.js +179 -0
- package/lib/cli/setup-auth.js +36 -14
- package/lib/cli/setup-credential-deployment.js +22 -8
- package/lib/cli/setup-dev-path-commands.js +124 -0
- package/lib/cli/setup-dev.js +190 -103
- package/lib/cli/setup-environment.js +11 -20
- package/lib/cli/setup-external-system.js +62 -22
- package/lib/cli/setup-infra.js +139 -47
- package/lib/cli/setup-parameters.js +32 -0
- package/lib/cli/setup-secrets.js +147 -10
- package/lib/cli/setup-service-user.js +146 -20
- package/lib/cli/setup-utility.js +47 -19
- package/lib/commands/app-down.js +5 -7
- package/lib/commands/app-install.js +14 -7
- package/lib/commands/app-logs.js +13 -10
- package/lib/commands/app-shell.js +4 -1
- package/lib/commands/app-test.js +25 -19
- package/lib/commands/app.js +22 -10
- package/lib/commands/auth-config.js +10 -14
- package/lib/commands/auth-status.js +4 -3
- package/lib/commands/credential-env.js +4 -3
- package/lib/commands/credential-list.js +5 -4
- package/lib/commands/credential-push.js +4 -3
- package/lib/commands/datasource-unified-test-cli.js +495 -0
- package/lib/commands/datasource-unified-test-cli.options.js +149 -0
- package/lib/commands/datasource-validation-cli.js +129 -0
- package/lib/commands/datasource.js +123 -71
- package/lib/commands/deployment-list.js +6 -5
- package/lib/commands/dev-cli-handlers.js +122 -18
- package/lib/commands/dev-down.js +4 -3
- package/lib/commands/dev-init.js +231 -116
- package/lib/commands/dev-show-display.js +473 -0
- package/lib/commands/login-credentials.js +3 -2
- package/lib/commands/login-device.js +4 -3
- package/lib/commands/login.js +5 -4
- package/lib/commands/logout.js +8 -7
- package/lib/commands/parameters-validate.js +54 -0
- package/lib/commands/repair-datasource.js +314 -68
- package/lib/commands/repair-env-template.js +16 -10
- package/lib/commands/repair-rbac.js +25 -19
- package/lib/commands/repair.js +116 -32
- package/lib/commands/secrets-list.js +23 -12
- package/lib/commands/secrets-remove-all.js +220 -0
- package/lib/commands/secrets-remove.js +22 -13
- package/lib/commands/secrets-set.js +21 -12
- package/lib/commands/secrets-validate.js +20 -7
- package/lib/commands/secure.js +10 -9
- package/lib/commands/service-user.js +243 -13
- package/lib/commands/test-e2e-external.js +27 -1
- package/lib/commands/up-common.js +28 -2
- package/lib/commands/up-dataplane.js +31 -18
- package/lib/commands/up-miso.js +19 -29
- package/lib/commands/upload.js +138 -39
- package/lib/commands/wizard-core-helpers.js +1 -1
- package/lib/commands/wizard-dataplane.js +4 -3
- package/lib/commands/wizard-helpers.js +3 -3
- package/lib/commands/wizard.js +2 -2
- package/lib/core/admin-secrets.js +16 -5
- package/lib/core/audit-logger.js +12 -4
- package/lib/core/config-attach-extensions.js +46 -0
- package/lib/core/config-runtime-paths.js +29 -0
- package/lib/core/config.js +59 -58
- package/lib/core/diff.js +3 -2
- package/lib/core/ensure-encryption-key.js +2 -4
- package/lib/core/secrets-ensure-infra.js +77 -0
- package/lib/core/secrets-ensure.js +120 -64
- package/lib/core/secrets-env-write.js +35 -7
- package/lib/core/secrets-infra-placeholder-sync.js +61 -0
- package/lib/core/secrets.js +228 -42
- package/lib/core/templates-env.js +4 -3
- package/lib/core/templates.js +1 -1
- package/lib/datasource/abac-validator.js +148 -0
- package/lib/datasource/deploy.js +75 -53
- package/lib/datasource/field-reference-validator.js +77 -36
- package/lib/datasource/integration-context.js +63 -0
- package/lib/datasource/list.js +8 -7
- package/lib/datasource/log-viewer.js +252 -0
- package/lib/datasource/resolve-app.js +109 -0
- package/lib/datasource/test-e2e.js +95 -155
- package/lib/datasource/test-integration.js +121 -109
- package/lib/datasource/unified-validation-run-body.js +65 -0
- package/lib/datasource/unified-validation-run-post.js +23 -0
- package/lib/datasource/unified-validation-run-resolve.js +43 -0
- package/lib/datasource/unified-validation-run.js +92 -0
- package/lib/datasource/validate.js +162 -15
- package/lib/deployment/deployer.js +4 -3
- package/lib/deployment/environment.js +7 -6
- package/lib/deployment/push.js +17 -8
- package/lib/external-system/delete.js +4 -3
- package/lib/external-system/deploy.js +131 -53
- package/lib/external-system/download-helpers.js +1 -1
- package/lib/external-system/download.js +7 -6
- package/lib/external-system/generator.js +104 -14
- package/lib/external-system/integration-test-dispatch.js +26 -0
- package/lib/external-system/test-execution.js +5 -1
- package/lib/external-system/test-helpers.js +0 -4
- package/lib/external-system/test-system-level-helpers.js +110 -0
- package/lib/external-system/test-system-level.js +83 -44
- package/lib/external-system/test.js +59 -8
- package/lib/generator/builders.js +23 -11
- package/lib/generator/deploy-manifest-azure-kv.js +81 -0
- package/lib/generator/external-controller-manifest.js +3 -3
- package/lib/generator/external.js +23 -11
- package/lib/generator/helpers.js +71 -12
- package/lib/generator/index.js +8 -4
- package/lib/generator/split-readme.js +12 -7
- package/lib/generator/split-variables.js +2 -1
- package/lib/generator/split.js +46 -11
- package/lib/generator/wizard-readme.js +3 -3
- package/lib/generator/wizard.js +16 -13
- package/lib/infrastructure/compose.js +60 -6
- package/lib/infrastructure/helpers.js +238 -51
- package/lib/infrastructure/index.js +64 -37
- package/lib/infrastructure/services.js +21 -15
- package/lib/internal/fs-real-sync.js +104 -0
- package/lib/internal/node-fs.js +98 -0
- package/lib/parameters/database-secret-values.js +173 -0
- package/lib/parameters/infra-kv-discovery.js +121 -0
- package/lib/parameters/infra-parameter-catalog.js +458 -0
- package/lib/parameters/infra-parameter-validate.js +64 -0
- package/lib/schema/application-schema.json +37 -17
- package/lib/schema/datasource-test-run.schema.json +493 -0
- package/lib/schema/deployment-rules.yaml +102 -63
- package/lib/schema/external-datasource.schema.json +1201 -433
- package/lib/schema/external-system.schema.json +181 -5
- package/lib/schema/flag-map-validation-run.json +31 -0
- package/lib/schema/infra-parameter.schema.json +106 -0
- package/lib/schema/infra.parameter.yaml +421 -0
- package/lib/schema/type/credential-auth-templates.json +40 -0
- package/lib/schema/type/document-storage.json +213 -0
- package/lib/schema/type/message-service.json +123 -0
- package/lib/schema/type/vector-store.json +88 -0
- package/lib/utils/aifabrix-runtime-config-dir.js +132 -0
- package/lib/utils/api-error-handler.js +2 -2
- package/lib/utils/api.js +49 -14
- package/lib/utils/app-config-resolver.js +23 -1
- package/lib/utils/app-register-api.js +3 -2
- package/lib/utils/app-register-auth.js +1 -1
- package/lib/utils/app-register-config.js +4 -4
- package/lib/utils/app-register-display.js +3 -2
- package/lib/utils/app-register-validator.js +3 -2
- package/lib/utils/app-run-containers.js +26 -22
- package/lib/utils/app-scoped-config.js +31 -0
- package/lib/utils/app-service-env-from-builder.js +164 -0
- package/lib/utils/build-copy.js +1 -1
- package/lib/utils/build-helpers.js +20 -20
- package/lib/utils/build-resolve-image.js +165 -0
- package/lib/utils/cli-layout-chalk.js +8 -0
- package/lib/utils/cli-test-layout-chalk.js +267 -0
- package/lib/utils/cli-utils.js +88 -11
- package/lib/utils/compose-db-passwords.js +138 -0
- package/lib/utils/compose-generate-docker-compose.js +216 -0
- package/lib/utils/compose-generator.js +197 -291
- package/lib/utils/compose-miso-env.js +18 -0
- package/lib/utils/compose-traefik-ingress-base.js +158 -0
- package/lib/utils/config-paths.js +209 -6
- package/lib/utils/config-scoped-resources-preference.js +41 -0
- package/lib/utils/controller-deployment-outcome.js +68 -0
- package/lib/utils/credential-display.js +2 -2
- package/lib/utils/credential-secrets-env.js +16 -1
- package/lib/utils/dataplane-pipeline-warning.js +4 -3
- package/lib/utils/datasource-test-run-capability-scope.js +43 -0
- package/lib/utils/datasource-test-run-debug-display.js +137 -0
- package/lib/utils/datasource-test-run-debug-slice.js +93 -0
- package/lib/utils/datasource-test-run-display.js +442 -0
- package/lib/utils/datasource-test-run-exit.js +58 -0
- package/lib/utils/datasource-test-run-legacy-adapter.js +93 -0
- package/lib/utils/datasource-test-run-report-version.js +51 -0
- package/lib/utils/datasource-test-run-schema-sync.js +59 -0
- package/lib/utils/datasource-test-run-tty-log.js +81 -0
- package/lib/utils/datasource-validation-watch.js +266 -0
- package/lib/utils/declarative-url-ports.js +47 -0
- package/lib/utils/derive-env-key-from-client-id.js +41 -0
- package/lib/utils/dev-ca-install.js +185 -23
- package/lib/utils/dev-cert-helper.js +266 -17
- package/lib/utils/dev-hosts-helper.js +307 -0
- package/lib/utils/dev-init-cert-hints.js +37 -0
- package/lib/utils/dev-init-health-messages.js +52 -0
- package/lib/utils/dev-init-resolve.js +86 -0
- package/lib/utils/dev-init-ssh-merge.js +65 -0
- package/lib/utils/dev-ssh-config-helper.js +196 -0
- package/lib/utils/dev-user-groups.js +93 -0
- package/lib/utils/docker-build.js +42 -17
- package/lib/utils/docker-exec.js +28 -0
- package/lib/utils/docker-manifest-public-port.js +116 -0
- package/lib/utils/docker-not-running-hint.js +52 -0
- package/lib/utils/docker.js +98 -11
- package/lib/utils/ensure-dev-certs-for-remote-docker.js +192 -0
- package/lib/utils/env-config-loader.js +10 -91
- package/lib/utils/env-copy.js +19 -10
- package/lib/utils/env-map.js +42 -11
- package/lib/utils/env-template.js +2 -2
- package/lib/utils/environment-scoped-resources.js +144 -0
- package/lib/utils/error-formatter.js +125 -9
- package/lib/utils/error-formatters/http-status-errors.js +6 -5
- package/lib/utils/error-formatters/network-errors.js +2 -1
- package/lib/utils/error-formatters/permission-errors.js +2 -1
- package/lib/utils/error-formatters/validation-errors.js +2 -1
- package/lib/utils/external-env-template.js +180 -0
- package/lib/utils/external-readme.js +8 -1
- package/lib/utils/external-system-display.js +277 -136
- package/lib/utils/external-system-local-test-tty.js +389 -0
- package/lib/utils/external-system-readiness-core.js +377 -0
- package/lib/utils/external-system-readiness-deploy-display.js +270 -0
- package/lib/utils/external-system-readiness-display-internals.js +150 -0
- package/lib/utils/external-system-readiness-display.js +186 -0
- package/lib/utils/external-system-test-helpers.js +24 -6
- package/lib/utils/external-system-validators.js +32 -14
- package/lib/utils/health-check-url.js +119 -0
- package/lib/utils/health-check.js +59 -25
- package/lib/utils/help-builder.js +14 -13
- package/lib/utils/image-version.js +4 -8
- package/lib/utils/infra-containers.js +4 -7
- package/lib/utils/infra-env-defaults.js +162 -0
- package/lib/utils/infra-status-display.js +167 -0
- package/lib/utils/infra-status.js +16 -8
- package/lib/utils/local-secrets.js +29 -7
- package/lib/utils/paths.js +136 -48
- package/lib/utils/port-resolver.js +10 -23
- package/lib/utils/redis-env-scope.js +62 -0
- package/lib/utils/register-aifabrix-shell-env.js +204 -0
- package/lib/utils/remote-builder-validation.js +99 -0
- package/lib/utils/remote-dev-auth.js +117 -21
- package/lib/utils/remote-docker-env.js +67 -15
- package/lib/utils/remote-secrets-loader.js +13 -4
- package/lib/utils/resolve-docker-image-ref.js +124 -0
- package/lib/utils/schema-loader.js +22 -9
- package/lib/utils/secrets-bash-kv.js +25 -0
- package/lib/utils/secrets-generator.js +171 -51
- package/lib/utils/secrets-helpers.js +70 -59
- package/lib/utils/secrets-kv-scope.js +60 -0
- package/lib/utils/secrets-utils.js +35 -37
- package/lib/utils/secrets-validation.js +3 -1
- package/lib/utils/secrets-yaml-preserve.js +109 -0
- package/lib/utils/secure-file-permissions.js +91 -0
- package/lib/utils/ssh-key-helper.js +4 -2
- package/lib/utils/template-helpers.js +2 -2
- package/lib/utils/test-log-writer.js +3 -3
- package/lib/utils/token-manager.js +37 -5
- package/lib/utils/url-declarative-public-base.js +188 -0
- package/lib/utils/url-declarative-resolve-build.js +493 -0
- package/lib/utils/url-declarative-resolve-load-doc.js +51 -0
- package/lib/utils/url-declarative-resolve.js +220 -0
- package/lib/utils/url-declarative-token-parse.js +74 -0
- package/lib/utils/url-declarative-url-flags.js +50 -0
- package/lib/utils/url-declarative-vdir-inactive-env.js +99 -0
- package/lib/utils/url-public-path-prefix.js +34 -0
- package/lib/utils/urls-local-registry.js +220 -0
- package/lib/utils/validation-report-tty-kit.js +77 -0
- package/lib/utils/validation-run-poll.js +89 -0
- package/lib/utils/validation-run-post-retry.js +73 -0
- package/lib/utils/validation-run-request.js +98 -0
- package/lib/utils/variable-transformer.js +21 -4
- package/lib/utils/yaml-preserve.js +78 -1
- package/lib/validation/datasource-warnings.js +56 -0
- package/lib/validation/env-template-auth.js +50 -2
- package/lib/validation/external-manifest-validator.js +35 -7
- package/lib/validation/validate-display.js +37 -31
- package/lib/validation/validate.js +9 -10
- package/lib/validation/validator-unresolved-placeholders.js +98 -0
- package/lib/validation/validator.js +32 -78
- package/lib/validation/wizard-config-validator.js +2 -1
- package/package.json +11 -3
- package/scripts/check-datasource-test-run-schema-sync.js +34 -0
- package/scripts/diagnose-cli.js +150 -0
- package/scripts/install-local.js +304 -55
- package/templates/README.md +15 -2
- package/templates/applications/dataplane/application.yaml +52 -2
- package/templates/applications/dataplane/env.template +80 -18
- package/templates/applications/dataplane/rbac.yaml +8 -0
- package/templates/applications/keycloak/application.yaml +9 -1
- package/templates/applications/keycloak/env.template +15 -6
- package/templates/applications/miso-controller/application.yaml +10 -2
- package/templates/applications/miso-controller/env.template +55 -14
- package/templates/applications/miso-controller/rbac.yaml +5 -0
- package/templates/external-system/README.md.hbs +20 -7
- package/templates/external-system/deploy.js.hbs +5 -5
- package/templates/external-system/env.template.hbs +22 -0
- package/templates/external-system/external-datasource.yaml.hbs +197 -118
- package/templates/infra/compose.yaml.hbs +20 -4
- package/templates/python/docker-compose.hbs +16 -0
- package/templates/typescript/docker-compose.hbs +16 -0
- package/integration/hubspot/README.md +0 -102
- package/integration/hubspot/env.template +0 -4
- package/integration/hubspot/hubspot-datasource-company.json +0 -541
- package/integration/hubspot/hubspot-datasource-contact.json +0 -639
- package/integration/hubspot/hubspot-datasource-deal.json +0 -588
- package/integration/hubspot/hubspot-datasource-users.json +0 -116
- package/integration/hubspot/test-artifacts/wizard-invalid-missing-source.yaml +0 -2
- package/integration/hubspot/test-artifacts/wizard-valid-for-dimension-key-test.yaml +0 -5
- package/integration/hubspot/test-artifacts/wizard-valid-for-dimension-path-test.yaml +0 -5
- package/lib/api/external-test.api.js +0 -111
- package/lib/schema/env-config.yaml +0 -43
- /package/integration/{hubspot → hubspot-test}/companies.json +0 -0
- /package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-app-name.yaml +0 -0
- /package/integration/{hubspot → hubspot-test}/test-artifacts/wizard-invalid-missing-app.yaml +0 -0
- /package/integration/{hubspot → hubspot-test}/test-dataplane-down-helpers.js +0 -0
|
@@ -1,47 +1,89 @@
|
|
|
1
|
+
# External datasource definition. Validate against lib/schema/external-datasource.schema.json (e.g. aifabrix validate / CI).
|
|
2
|
+
# Normative usage (metadata, FKs, dimensions, expressions, execution vs query) aligns with schema v2.4.1 and the Datasource Rules baseline (Plan 346).
|
|
3
|
+
# key: stable identifier (typically <systemKey>-<entity>); referenced by credentials, deploy, and datasource-chaining CIP steps.
|
|
1
4
|
key: "{{fullDatasourceKey}}"
|
|
5
|
+
# Short name and description shown in tooling and docs.
|
|
2
6
|
displayName: "{{datasourceDisplayName}}"
|
|
3
7
|
description: "{{datasourceDescription}}"
|
|
8
|
+
# Parent external system this datasource belongs to (matches external-system key).
|
|
4
9
|
systemKey: "{{systemKey}}"
|
|
10
|
+
# entityType: recordStorage = structured rows in DB; documentStorage = same family + binary/large content outside metadata_json;
|
|
11
|
+
# vectorStore = retrieval/embedding external store; messageService = channels/notifications; none = orchestration/API composition ONLY.
|
|
12
|
+
# If none: do NOT add metadataSchema, primaryKey, labelKey, dimensions, fieldMappings, sync, documentStorage, etc. (schema forbids them).
|
|
5
13
|
entityType: "{{schemaEntityType}}"
|
|
14
|
+
# Fine-grained resource classification (e.g. customer, document); used for UX and policies.
|
|
6
15
|
resourceType: "{{resourceType}}"
|
|
16
|
+
enabled: true
|
|
17
|
+
# File / config version for this YAML (distinct from execution.cip.version when you add CIP below).
|
|
18
|
+
version: "1.0.0"
|
|
19
|
+
{{#unless (eq schemaEntityType "none")}}
|
|
20
|
+
# primaryKey: normalized attribute names used for get/update/delete and storage keys (must exist in metadataSchema + fieldMappings).
|
|
7
21
|
primaryKey:
|
|
8
22
|
{{#each primaryKey}} - "{{this}}"
|
|
9
23
|
{{/each}}
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
{{
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
24
|
+
# labelKey: attributes used for display labels (search, pickers); often includes primaryKey + human-readable fields.
|
|
25
|
+
labelKey:
|
|
26
|
+
{{#each labelKey}} - "{{this}}"
|
|
27
|
+
{{/each}}
|
|
28
|
+
# metadataSchema: canonical normalized fields (camelCase). Only index=true / filter=true scalars materialize as DB columns; full shape lives in metadata_json.
|
|
29
|
+
# Allowed on fields: type, format, index, filter, nullable, description, string/number validation keywords. Forbidden: anyOf/oneOf/allOf/not/enum/const in author trees.
|
|
30
|
+
# Datetime: type string + format date-time, UTC. Max structural depth per product rules (~2); avoid deep JSON for filter/join paths.
|
|
31
|
+
metadataSchema:
|
|
32
|
+
type: object
|
|
33
|
+
required:
|
|
34
|
+
{{#each attributeKeysForMetadata}} - "{{this}}"
|
|
35
|
+
{{/each}}
|
|
36
|
+
properties:
|
|
37
|
+
{{#each attributeKeysForMetadata}}
|
|
38
|
+
{{this}}:
|
|
39
|
+
type: string
|
|
40
|
+
{{#if (eq this "externalId")}} index: true
|
|
41
|
+
{{else}}{{#if (eq this "id")}} index: true
|
|
42
|
+
{{else}}{{#if @first}} index: true
|
|
43
|
+
{{else}} filter: true
|
|
44
|
+
{{/if}}{{/if}}{{/if}}
|
|
45
|
+
{{/each}}
|
|
46
|
+
{{#if dimensionBindingEntries}}
|
|
47
|
+
# dimensions: top-level only. type local binds to an indexed metadata field; type fk uses via: [{ fk, dimension }]. Optional for global reference data; expected when ABAC/FK joins apply (Plan 346 section 7).
|
|
48
|
+
# Actors: displayName|email|userId → operator eq; groups|roles → in. If actor is omitted, operator should be omitted (operator has no effect without actor).
|
|
49
|
+
dimensions:
|
|
50
|
+
{{#each dimensionBindingEntries}}
|
|
51
|
+
{{dimKey}}:
|
|
52
|
+
type: local
|
|
53
|
+
field: "{{field}}"
|
|
54
|
+
actor: displayName
|
|
55
|
+
operator: eq
|
|
17
56
|
{{/each}}
|
|
18
|
-
{{else}}
|
|
19
|
-
dimensions: {}
|
|
20
|
-
# Optional: add country, department, organization for ABAC
|
|
21
57
|
{{/if}}
|
|
58
|
+
# fieldMappings: normalization-only. Each attribute allows only { expression }. Roots MUST be raw.*, fk.*, or dimension.* (e.g. raw.id, fk.company.metadata.name).
|
|
59
|
+
# Heavy transforms belong upstream, in sync, or in entityType none orchestration — not here. Output keys MUST match metadataSchema properties.
|
|
60
|
+
fieldMappings:
|
|
22
61
|
attributes:
|
|
23
62
|
{{#if attributes}}
|
|
24
63
|
{{#each attributes}}
|
|
25
64
|
{{@key}}:
|
|
26
65
|
expression: "{{this.expression}}"
|
|
27
|
-
type: {{this.type}}
|
|
28
|
-
indexed: {{#if this.indexed}}true{{else}}false{{/if}}
|
|
29
66
|
{{/each}}
|
|
30
67
|
{{else}}
|
|
31
68
|
id:
|
|
32
69
|
expression: "{{raw.id}}"
|
|
33
|
-
type: string
|
|
34
|
-
indexed: true
|
|
35
70
|
name:
|
|
36
71
|
expression: "{{raw.name}}"
|
|
37
|
-
type: string
|
|
38
|
-
indexed: false
|
|
39
72
|
{{/if}}
|
|
73
|
+
# foreignKeys (optional): declare join graph to other storage datasources. name + fields[] (indexed metadata fields) + targetDatasource; acyclic; types must match (Plan 346 sections 6 and 8).
|
|
74
|
+
# foreignKeys:
|
|
75
|
+
# - name: company
|
|
76
|
+
# fields: [companyId]
|
|
77
|
+
# targetDatasource: my-system-company
|
|
78
|
+
{{/unless}}
|
|
40
79
|
{{#if (eq systemType "openapi")}}
|
|
80
|
+
# openapi: links this datasource to the generated OpenAPI document (same systemKey-api artifact). CIP fetch.openapiRef uses keys below: list, get, …
|
|
41
81
|
openapi:
|
|
42
82
|
enabled: true
|
|
83
|
+
# Must match the OpenAPI bundle key published for this system (wizard default: <systemKey>-api).
|
|
43
84
|
documentKey: "{{systemKey}}-api"
|
|
44
85
|
operations:
|
|
86
|
+
# Standard names list/get/create/update/delete; operationId and path should match your spec. CIP uses openapiRef: list | get.
|
|
45
87
|
list:
|
|
46
88
|
operationId: "list{{entityKey}}"
|
|
47
89
|
method: GET
|
|
@@ -50,168 +92,205 @@ openapi:
|
|
|
50
92
|
operationId: "get{{entityKey}}"
|
|
51
93
|
method: GET
|
|
52
94
|
path: "/{{entityKey}}/{id}"
|
|
95
|
+
# When true, runtime can derive RBAC hints from OpenAPI metadata where supported.
|
|
53
96
|
autoRbac: true
|
|
54
97
|
{{/if}}
|
|
55
98
|
|
|
56
99
|
# --- Optional sections: uncomment or delete as needed ---
|
|
57
|
-
#
|
|
58
|
-
#
|
|
100
|
+
#
|
|
101
|
+
# Query-time (v2.4): public list/get/filter are served from persisted Dataplane DB — not live external HTTP on every read (Plan 346 section 11).
|
|
102
|
+
# External APIs: sync jobs and create/update/delete execution paths. CIP fetch→map is for ingestion, writes, or orchestration — populate storage before expecting query APIs.
|
|
103
|
+
#
|
|
104
|
+
# CIP (Composable Integration Pipeline): fetch → [paginate] → [map] → [filter] → output
|
|
105
|
+
# Sources: openapi (openapi.operations.<name>), http (method + path), datasource (another datasource key + operation name). Operation names: ^[a-z][a-zA-Z0-9]*$
|
|
106
|
+
# execution.cip requires version "1.0" and operations.<name>.steps ($defs.cipDefinition). Custom operation keys must match the same pattern as capabilities[].
|
|
107
|
+
#
|
|
59
108
|
{{#if (eq schemaEntityType "recordStorage")}}
|
|
109
|
+
# sync: external APIs run during sync (Plan 346 section 11). Minimal contract: mode/schedule/batchSize per schema. FK fields on a record should update atomically in one logical write.
|
|
60
110
|
# sync:
|
|
61
111
|
# pull:
|
|
62
112
|
# enabled: true
|
|
63
113
|
# schedule: "0 * * * *"
|
|
114
|
+
# batchSize: 500
|
|
115
|
+
# validation: structural / pre-persistence checks; quality: post-mapping acceptance (e.g. rejectIf) — see schema when you enable them.
|
|
116
|
+
# capabilities: standard names list, get, create, update, delete + custom names matching ^[a-z][a-zA-Z0-9]*$; must match execution.cip.operations / openapi.operations you implement.
|
|
64
117
|
# capabilities: [list, get]
|
|
65
118
|
{{/if}}
|
|
66
119
|
{{#if (eq schemaEntityType "documentStorage")}}
|
|
120
|
+
# documentStorage: binary/attachment flow — which operation fetches bytes and which metadata field holds extractable text for embeddings.
|
|
67
121
|
# documentStorage:
|
|
68
122
|
# enabled: true
|
|
69
123
|
# binaryOperationRef: get
|
|
70
124
|
# embeddingField: content
|
|
71
125
|
{{/if}}
|
|
72
126
|
{{#if (eq schemaEntityType "vectorStore")}}
|
|
127
|
+
# vectorStore: vector index settings (model id, dimensions, etc. per schema). Uncomment and set to match your embedding deployment.
|
|
73
128
|
# vectorStore:
|
|
74
129
|
# enabled: true
|
|
75
130
|
# embeddingModel: "text-embedding-ada-002"
|
|
76
131
|
{{/if}}
|
|
77
132
|
{{#if (eq schemaEntityType "messageService")}}
|
|
133
|
+
# messageService: channel-oriented messaging integration; channels list drives subscription/send surfaces.
|
|
78
134
|
# messageService:
|
|
79
135
|
# enabled: true
|
|
80
136
|
# channels: []
|
|
81
137
|
{{/if}}
|
|
82
138
|
|
|
83
|
-
#
|
|
84
|
-
#
|
|
85
|
-
#
|
|
86
|
-
#
|
|
87
|
-
|
|
139
|
+
{{#unless (eq schemaEntityType "none")}}
|
|
140
|
+
{{#if (eq systemType "openapi")}}
|
|
141
|
+
# Example CIP (commented): matches openapi.operations list/get for "{{entityKey}}" above — copy, uncomment, tune inputPath/query.
|
|
142
|
+
# Alternative engines: execution.engine python (custom handler) or datasource (chain); see schema execution.* properties.
|
|
143
|
+
#
|
|
88
144
|
# execution:
|
|
89
145
|
# engine: cip
|
|
90
146
|
# cip:
|
|
147
|
+
# version: "1.0"
|
|
148
|
+
# # Optional: idempotency / runtimeContext blocks live alongside version at this level (schema $defs.idempotencyConfig, cipRuntimeContext).
|
|
91
149
|
# operations:
|
|
92
150
|
# list:
|
|
93
151
|
# enabled: true
|
|
94
|
-
# description: "List
|
|
152
|
+
# description: "List {{entityKey}} records"
|
|
95
153
|
# steps:
|
|
96
|
-
# - fetch:
|
|
97
|
-
#
|
|
98
|
-
#
|
|
99
|
-
#
|
|
100
|
-
#
|
|
101
|
-
# #
|
|
102
|
-
#
|
|
154
|
+
# - fetch:
|
|
155
|
+
# source: openapi
|
|
156
|
+
# openapiRef: list
|
|
157
|
+
# query: {}
|
|
158
|
+
# # expectedItemsPath: "$.results"
|
|
159
|
+
# # - paginate:
|
|
160
|
+
# # strategy: cursor
|
|
161
|
+
# # cursorField: "next_cursor"
|
|
162
|
+
# # cursorParam: "cursor"
|
|
163
|
+
# # pageSize: 100
|
|
164
|
+
# # maxPages: 50
|
|
165
|
+
# - map:
|
|
166
|
+
# useFieldMappings: true
|
|
167
|
+
# # inputPath: JSONPath-like slice over the fetch body; must match list payload (array of raw objects before fieldMappings).
|
|
168
|
+
# inputPath: "$.results[*]"
|
|
169
|
+
# # Try $.items[*], $.data[*], etc. if your list lives under a different key (schema default hint is $.items[*]).
|
|
170
|
+
# # inline: { customField: "\{{raw.custom}}" } # optional field override beside useFieldMappings
|
|
171
|
+
# - filter:
|
|
172
|
+
# enforceAbac: true
|
|
173
|
+
# # expression "@": pass records through JMESPath after ABAC; replace with a predicate when you need server-side row filtering.
|
|
174
|
+
# expression: "@"
|
|
175
|
+
# expressionLanguage: jmespath
|
|
176
|
+
# # abac: { mode: mandatory, policyScope: datasource }
|
|
177
|
+
# - output:
|
|
178
|
+
# mode: records
|
|
179
|
+
# # limit: soft cap on list size; sync jobs may override.
|
|
180
|
+
# # limit: 500
|
|
181
|
+
# # includeConfidence: true # optional per-record scoring when supported
|
|
103
182
|
# get:
|
|
104
183
|
# enabled: true
|
|
105
|
-
# description: "
|
|
106
|
-
# steps:
|
|
107
|
-
# - fetch: { source: openapi, openapiRef: get, query: {} }
|
|
108
|
-
# - map: { useFieldMappings: true, inputPath: "$" }
|
|
109
|
-
# - filter: { enforceAbac: true }
|
|
110
|
-
# - output: { mode: records }
|
|
111
|
-
# create:
|
|
112
|
-
# steps:
|
|
113
|
-
# - fetch: { source: openapi, openapiRef: create, bodyTemplate: "{{body}}" }
|
|
114
|
-
# - map: { useFieldMappings: true, inputPath: "$" }
|
|
115
|
-
# - output: { mode: records }
|
|
116
|
-
# update:
|
|
117
|
-
# steps:
|
|
118
|
-
# - fetch: { source: openapi, openapiRef: update, bodyTemplate: "{{body}}" }
|
|
119
|
-
# - map: { useFieldMappings: true, inputPath: "$" }
|
|
120
|
-
# - output: { mode: records }
|
|
121
|
-
# delete:
|
|
184
|
+
# description: "Single {{entityKey}} by id (path /{{entityKey}}/{id} on openapi.get)"
|
|
122
185
|
# steps:
|
|
123
|
-
# - fetch:
|
|
124
|
-
#
|
|
125
|
-
|
|
126
|
-
|
|
186
|
+
# - fetch:
|
|
187
|
+
# source: openapi
|
|
188
|
+
# openapiRef: get
|
|
189
|
+
# query:
|
|
190
|
+
# id: "{{raw.id}}"
|
|
191
|
+
# # If your API uses a different param name, rename the key; path params are resolved per your OpenAPI operation.
|
|
192
|
+
# # onError: { retry: { maxAttempts: 3 } } # optional per-step error policy (schema $defs.cipOnErrorConfig)
|
|
193
|
+
# - map:
|
|
194
|
+
# useFieldMappings: true
|
|
195
|
+
# # Single-object body: "$" or a path to the entity object inside a wrapper response.
|
|
196
|
+
# inputPath: "$"
|
|
197
|
+
# # Wrapped entity: use "$.data" or "$.result"
|
|
198
|
+
# - filter:
|
|
199
|
+
# enforceAbac: true
|
|
200
|
+
# expression: "@"
|
|
201
|
+
# expressionLanguage: jmespath
|
|
202
|
+
# - output:
|
|
203
|
+
# mode: records
|
|
204
|
+
# # Optional: delegate to another datasource's CIP
|
|
205
|
+
# # related:
|
|
206
|
+
# # enabled: true
|
|
207
|
+
# # steps:
|
|
208
|
+
# # - fetch: { source: datasource, datasource: "other-system-entity", operation: list, parameters: {} }
|
|
209
|
+
# # - output: { mode: records }
|
|
210
|
+
{{else}}
|
|
211
|
+
# Example CIP (commented): no openapi block in this file — use http fetch (or add openapi: first, then use the openapi example pattern).
|
|
212
|
+
# http fetch: method + path are required; query holds static query params (runtime may merge ABAC params).
|
|
213
|
+
#
|
|
127
214
|
# execution:
|
|
128
215
|
# engine: cip
|
|
129
216
|
# cip:
|
|
217
|
+
# version: "1.0"
|
|
130
218
|
# operations:
|
|
131
219
|
# list:
|
|
132
220
|
# enabled: true
|
|
133
|
-
# description: "List documents"
|
|
134
221
|
# steps:
|
|
135
|
-
# - fetch:
|
|
136
|
-
#
|
|
137
|
-
#
|
|
138
|
-
#
|
|
139
|
-
#
|
|
222
|
+
# - fetch:
|
|
223
|
+
# source: http
|
|
224
|
+
# method: GET
|
|
225
|
+
# path: "/{{entityKey}}"
|
|
226
|
+
# query: {}
|
|
227
|
+
# # headers: { Accept: "application/json" }
|
|
228
|
+
# - map:
|
|
229
|
+
# useFieldMappings: true
|
|
230
|
+
# inputPath: "$.items[*]"
|
|
231
|
+
# - filter:
|
|
232
|
+
# enforceAbac: true
|
|
233
|
+
# expression: "@"
|
|
234
|
+
# expressionLanguage: jmespath
|
|
235
|
+
# - output:
|
|
236
|
+
# mode: records
|
|
140
237
|
# get:
|
|
141
238
|
# enabled: true
|
|
142
|
-
# description: "Get document by ID"
|
|
143
|
-
# steps:
|
|
144
|
-
# - fetch: { source: openapi, openapiRef: get, query: {} }
|
|
145
|
-
# - map: { useFieldMappings: true, inputPath: "$" }
|
|
146
|
-
# - filter: { enforceAbac: true }
|
|
147
|
-
# - output: { mode: records }
|
|
148
|
-
# create:
|
|
149
|
-
# enabled: true
|
|
150
|
-
# description: "Upload document"
|
|
151
239
|
# steps:
|
|
152
|
-
# - fetch:
|
|
153
|
-
#
|
|
154
|
-
#
|
|
155
|
-
{
|
|
156
|
-
|
|
157
|
-
#
|
|
158
|
-
#
|
|
159
|
-
#
|
|
160
|
-
#
|
|
161
|
-
#
|
|
162
|
-
#
|
|
163
|
-
#
|
|
164
|
-
#
|
|
165
|
-
# - output:
|
|
166
|
-
#
|
|
167
|
-
# steps:
|
|
168
|
-
# - fetch: { source: openapi, openapiRef: get, query: {} }
|
|
169
|
-
# - map: { useFieldMappings: true, inputPath: "$" }
|
|
170
|
-
# - output: { mode: records }
|
|
171
|
-
{{/if}}
|
|
172
|
-
{{#if (eq schemaEntityType "messageService")}}
|
|
173
|
-
# execution:
|
|
174
|
-
# engine: cip
|
|
175
|
-
# cip:
|
|
176
|
-
# operations:
|
|
177
|
-
# list:
|
|
178
|
-
# steps:
|
|
179
|
-
# - fetch: { source: openapi, openapiRef: list, query: {} }
|
|
180
|
-
# - map: { useFieldMappings: true, inputPath: "$" }
|
|
181
|
-
# - output: { mode: records }
|
|
240
|
+
# - fetch:
|
|
241
|
+
# source: http
|
|
242
|
+
# method: GET
|
|
243
|
+
# # Path template must match your API; {id} is illustrative — align with real route and param names.
|
|
244
|
+
# path: "/{{entityKey}}/{id}"
|
|
245
|
+
# query: {}
|
|
246
|
+
# - map:
|
|
247
|
+
# useFieldMappings: true
|
|
248
|
+
# inputPath: "$"
|
|
249
|
+
# - filter:
|
|
250
|
+
# enforceAbac: true
|
|
251
|
+
# expression: "@"
|
|
252
|
+
# expressionLanguage: jmespath
|
|
253
|
+
# - output:
|
|
254
|
+
# mode: records
|
|
182
255
|
{{/if}}
|
|
256
|
+
{{/unless}}
|
|
183
257
|
{{#if (eq schemaEntityType "none")}}
|
|
184
|
-
# execution:
|
|
258
|
+
# execution (entityType none — orchestration / API composition only; no storage contract on this datasource):
|
|
259
|
+
# Cross-datasource access without FK is not a query-time join: use FK graph, bridge recordStorage, pre-normalize, or fan-out here (Plan 346 section 5).
|
|
185
260
|
# engine: cip
|
|
186
261
|
# cip:
|
|
262
|
+
# version: "1.0"
|
|
187
263
|
# operations:
|
|
188
264
|
# list:
|
|
189
265
|
# steps:
|
|
190
266
|
# - fetch: { source: openapi, openapiRef: list, query: {} }
|
|
191
|
-
# - map: { useFieldMappings: true, inputPath: "$" }
|
|
192
|
-
# - output: { mode: records }
|
|
193
|
-
# get:
|
|
194
|
-
# steps:
|
|
195
|
-
# - fetch: { source: openapi, openapiRef: get, query: {} }
|
|
196
|
-
# - map: { useFieldMappings: true, inputPath: "$" }
|
|
197
267
|
# - output: { mode: records }
|
|
268
|
+
# # Chain another datasource (no fieldMappings on this datasource):
|
|
269
|
+
# # chain:
|
|
270
|
+
# # steps:
|
|
271
|
+
# # - fetch: { source: datasource, datasource: "upstream-system-entity", operation: list, parameters: {} }
|
|
272
|
+
# # - output: { mode: records }
|
|
198
273
|
{{/if}}
|
|
199
274
|
|
|
200
|
-
# config:
|
|
275
|
+
# config.extensions: vendor-specific knobs; property names must match ^x[A-Z]… (e.g. xHubSpotFeature). Keeps core schema stable.
|
|
201
276
|
# config:
|
|
202
|
-
#
|
|
203
|
-
#
|
|
204
|
-
# # crossSystemJson: { "dimensions.country": { "eq": "{{actor.country}}" } }
|
|
277
|
+
# extensions:
|
|
278
|
+
# xHubSpotFeature: {}
|
|
205
279
|
|
|
206
|
-
#
|
|
207
|
-
|
|
208
|
-
# exposed: attributes to expose via MCP/OpenAPI
|
|
280
|
+
# exposed.schema: single public response shape for REST/MCP; values are expressions (metadata.*, dimension.*, fk.*.metadata.*). No implicit fields; use exposed.readonly[] / exposed.omit[] for writes (Plan 346 section 13). exposed.attributes is deprecated.
|
|
209
281
|
# exposed:
|
|
210
|
-
#
|
|
282
|
+
# schema:
|
|
283
|
+
# id: metadata.id
|
|
284
|
+
# name: metadata.name
|
|
285
|
+
# # nested objects and fk.* paths are allowed when foreignKeys exist and join rules pass join rules (section 8)
|
|
286
|
+
# # readonly: [status]
|
|
287
|
+
# # omit: [internalScore]
|
|
211
288
|
|
|
212
|
-
#
|
|
213
|
-
#
|
|
214
|
-
#
|
|
215
|
-
#
|
|
216
|
-
#
|
|
217
|
-
|
|
289
|
+
# testPayload (optional): fixtures for static expression resolution (raw/fk/dimension); root keys are closed in schema — no typos at top level (Plan 346 section 16).
|
|
290
|
+
# testPayload:
|
|
291
|
+
# mode: smoke
|
|
292
|
+
# payloadTemplate:
|
|
293
|
+
# raw: { id: "1", name: "Example" }
|
|
294
|
+
|
|
295
|
+
# capabilities: root-level array preferred (v2.2+); names must match ^[a-z][a-zA-Z0-9]*$.
|
|
296
|
+
# capabilities: [list, get, create, update, delete]
|
|
@@ -17,7 +17,9 @@ services:
|
|
|
17
17
|
- "{{postgresPort}}:5432"
|
|
18
18
|
volumes:
|
|
19
19
|
- {{#if (eq devId 0)}}postgres_data{{else}}dev{{devId}}_postgres_data{{/if}}:/var/lib/postgresql/data
|
|
20
|
-
-
|
|
20
|
+
- type: bind
|
|
21
|
+
source: "{{initScriptsBind}}"
|
|
22
|
+
target: /docker-entrypoint-initdb.d
|
|
21
23
|
networks:
|
|
22
24
|
- {{networkName}}
|
|
23
25
|
healthcheck:
|
|
@@ -61,7 +63,10 @@ services:
|
|
|
61
63
|
- "{{pgadminPort}}:80"
|
|
62
64
|
volumes:
|
|
63
65
|
- {{#if (eq devId 0)}}pgadmin_data{{else}}dev{{devId}}_pgadmin_data{{/if}}:/var/lib/pgadmin
|
|
64
|
-
-
|
|
66
|
+
- type: bind
|
|
67
|
+
source: "{{infraDirBind}}"
|
|
68
|
+
target: /host-config
|
|
69
|
+
read_only: true
|
|
65
70
|
command: >
|
|
66
71
|
sh -c "
|
|
67
72
|
if [ -f /host-config/servers.json ]; then
|
|
@@ -110,8 +115,13 @@ services:
|
|
|
110
115
|
- "--providers.docker=true"
|
|
111
116
|
- "--providers.docker.exposedbydefault=false"
|
|
112
117
|
- "--providers.docker.network={{networkName}}"
|
|
118
|
+
- "--providers.docker.allowEmptyServices=true"
|
|
119
|
+
- "--log.level=INFO"
|
|
113
120
|
- "--entrypoints.web.address=:80"
|
|
114
121
|
- "--entrypoints.websecure.address=:443"
|
|
122
|
+
{{#if traefik.trustForwardedHeaders}}
|
|
123
|
+
- "--entrypoints.web.forwardedHeaders.insecure=true"
|
|
124
|
+
{{/if}}
|
|
115
125
|
{{#if traefik.certStore}}
|
|
116
126
|
{{#if traefik.certFile}}
|
|
117
127
|
- "--certificatesstores.{{traefik.certStore}}.defaultcertificate.certfile={{traefik.certFile}}"
|
|
@@ -126,10 +136,16 @@ services:
|
|
|
126
136
|
volumes:
|
|
127
137
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
|
128
138
|
{{#if traefik.certFile}}
|
|
129
|
-
-
|
|
139
|
+
- type: bind
|
|
140
|
+
source: "{{traefik.certFile}}"
|
|
141
|
+
target: "{{traefik.certFile}}"
|
|
142
|
+
read_only: true
|
|
130
143
|
{{/if}}
|
|
131
144
|
{{#if traefik.keyFile}}
|
|
132
|
-
-
|
|
145
|
+
- type: bind
|
|
146
|
+
source: "{{traefik.keyFile}}"
|
|
147
|
+
target: "{{traefik.keyFile}}"
|
|
148
|
+
read_only: true
|
|
133
149
|
{{/if}}
|
|
134
150
|
networks:
|
|
135
151
|
- {{networkName}}
|
|
@@ -13,16 +13,28 @@ services:
|
|
|
13
13
|
{{#if traefik.tls}}
|
|
14
14
|
- "traefik.http.routers.{{app.key}}.entrypoints=websecure"
|
|
15
15
|
- "traefik.http.routers.{{app.key}}.tls=true"
|
|
16
|
+
- "traefik.http.routers.{{app.key}}.service={{app.key}}"
|
|
16
17
|
{{#if traefik.certStore}}
|
|
17
18
|
- "traefik.http.routers.{{app.key}}.tls.certstore={{traefik.certStore}}"
|
|
18
19
|
{{/if}}
|
|
20
|
+
- "traefik.http.routers.{{app.key}}-http.rule=Host(`{{traefik.host}}`) && PathPrefix(`{{traefik.path}}`)"
|
|
21
|
+
- "traefik.http.routers.{{app.key}}-http.entrypoints=web"
|
|
22
|
+
{{#unless (eq traefik.path "/")}}
|
|
23
|
+
{{#if traefik.stripPathPrefix}}
|
|
24
|
+
- "traefik.http.routers.{{app.key}}-http.middlewares={{app.key}}-stripprefix"
|
|
25
|
+
{{/if}}
|
|
26
|
+
{{/unless}}
|
|
27
|
+
- "traefik.http.routers.{{app.key}}-http.service={{app.key}}"
|
|
19
28
|
{{else}}
|
|
20
29
|
- "traefik.http.routers.{{app.key}}.entrypoints=web"
|
|
30
|
+
- "traefik.http.routers.{{app.key}}.service={{app.key}}"
|
|
21
31
|
{{/if}}
|
|
22
32
|
- "traefik.http.services.{{app.key}}.loadbalancer.server.port={{containerPort}}"
|
|
23
33
|
{{#unless (eq traefik.path "/")}}
|
|
34
|
+
{{#if traefik.stripPathPrefix}}
|
|
24
35
|
- "traefik.http.middlewares.{{app.key}}-stripprefix.stripprefix.prefixes={{traefik.path}}"
|
|
25
36
|
- "traefik.http.routers.{{app.key}}.middlewares={{app.key}}-stripprefix"
|
|
37
|
+
{{/if}}
|
|
26
38
|
{{/unless}}
|
|
27
39
|
{{/if}}
|
|
28
40
|
environment:
|
|
@@ -59,7 +71,11 @@ services:
|
|
|
59
71
|
{{/if}}
|
|
60
72
|
{{/if}}
|
|
61
73
|
healthcheck:
|
|
74
|
+
{{#if healthCheck.bashProbe}}
|
|
75
|
+
test: ["CMD-SHELL", "bash -c 'exec 3<>/dev/tcp/127.0.0.1/{{port}} && printf \"GET {{healthCheck.path}} HTTP/1.1\\r\\nHost: localhost\\r\\nConnection: close\\r\\n\\r\\n\" >&3 && head -n1 <&3 | grep -q \"200 OK\"'"]
|
|
76
|
+
{{else}}
|
|
62
77
|
test: ["CMD", "curl", "-f", "http://localhost:{{port}}{{healthCheck.path}}"]
|
|
78
|
+
{{/if}}
|
|
63
79
|
interval: {{healthCheck.interval}}s
|
|
64
80
|
timeout: 10s
|
|
65
81
|
retries: 3
|
|
@@ -13,16 +13,28 @@ services:
|
|
|
13
13
|
{{#if traefik.tls}}
|
|
14
14
|
- "traefik.http.routers.{{app.key}}.entrypoints=websecure"
|
|
15
15
|
- "traefik.http.routers.{{app.key}}.tls=true"
|
|
16
|
+
- "traefik.http.routers.{{app.key}}.service={{app.key}}"
|
|
16
17
|
{{#if traefik.certStore}}
|
|
17
18
|
- "traefik.http.routers.{{app.key}}.tls.certstore={{traefik.certStore}}"
|
|
18
19
|
{{/if}}
|
|
20
|
+
- "traefik.http.routers.{{app.key}}-http.rule=Host(`{{traefik.host}}`) && PathPrefix(`{{traefik.path}}`)"
|
|
21
|
+
- "traefik.http.routers.{{app.key}}-http.entrypoints=web"
|
|
22
|
+
{{#unless (eq traefik.path "/")}}
|
|
23
|
+
{{#if traefik.stripPathPrefix}}
|
|
24
|
+
- "traefik.http.routers.{{app.key}}-http.middlewares={{app.key}}-stripprefix"
|
|
25
|
+
{{/if}}
|
|
26
|
+
{{/unless}}
|
|
27
|
+
- "traefik.http.routers.{{app.key}}-http.service={{app.key}}"
|
|
19
28
|
{{else}}
|
|
20
29
|
- "traefik.http.routers.{{app.key}}.entrypoints=web"
|
|
30
|
+
- "traefik.http.routers.{{app.key}}.service={{app.key}}"
|
|
21
31
|
{{/if}}
|
|
22
32
|
- "traefik.http.services.{{app.key}}.loadbalancer.server.port={{containerPort}}"
|
|
23
33
|
{{#unless (eq traefik.path "/")}}
|
|
34
|
+
{{#if traefik.stripPathPrefix}}
|
|
24
35
|
- "traefik.http.middlewares.{{app.key}}-stripprefix.stripprefix.prefixes={{traefik.path}}"
|
|
25
36
|
- "traefik.http.routers.{{app.key}}.middlewares={{app.key}}-stripprefix"
|
|
37
|
+
{{/if}}
|
|
26
38
|
{{/unless}}
|
|
27
39
|
{{/if}}
|
|
28
40
|
environment:
|
|
@@ -59,7 +71,11 @@ services:
|
|
|
59
71
|
{{/if}}
|
|
60
72
|
{{/if}}
|
|
61
73
|
healthcheck:
|
|
74
|
+
{{#if healthCheck.bashProbe}}
|
|
75
|
+
test: ["CMD-SHELL", "bash -c 'exec 3<>/dev/tcp/127.0.0.1/{{port}} && printf \"GET {{healthCheck.path}} HTTP/1.1\\r\\nHost: localhost\\r\\nConnection: close\\r\\n\\r\\n\" >&3 && head -n1 <&3 | grep -q \"200 OK\"'"]
|
|
76
|
+
{{else}}
|
|
62
77
|
test: ["CMD", "curl", "-f", "http://localhost:{{port}}{{healthCheck.path}}"]
|
|
78
|
+
{{/if}}
|
|
63
79
|
interval: {{healthCheck.interval}}s
|
|
64
80
|
timeout: 10s
|
|
65
81
|
retries: 3
|
|
@@ -1,102 +0,0 @@
|
|
|
1
|
-
# HubSpot CRM
|
|
2
|
-
|
|
3
|
-
HubSpot CRM integration with OpenAPI support for companies, contacts, and deals
|
|
4
|
-
|
|
5
|
-
## System Information
|
|
6
|
-
|
|
7
|
-
- **System Key**: `hubspot`
|
|
8
|
-
- **System Type**: `openapi`
|
|
9
|
-
- **Datasources**: 4
|
|
10
|
-
|
|
11
|
-
## Files
|
|
12
|
-
|
|
13
|
-
- `application.yaml` – Application configuration with `app` and `externalIntegration` blocks
|
|
14
|
-
- `hubspot-system.yaml` – External system definition (authentication, OpenAPI/MCP, RBAC)
|
|
15
|
-
- `hubspot-datasource-deal.yaml` – Datasource: HubSpot Deal
|
|
16
|
-
- `hubspot-datasource-contact.yaml` – Datasource: HubSpot Contact
|
|
17
|
-
- `hubspot-datasource-users-datasource.yaml` – Datasource: HubSpot Users
|
|
18
|
-
- `hubspot-datasource-company.yaml` – Datasource: HubSpot Company
|
|
19
|
-
- `env.template` – Environment variables template (secrets, API keys)
|
|
20
|
-
- `hubspot-deploy.json` – Deployment manifest (generated by `aifabrix json`)
|
|
21
|
-
|
|
22
|
-
Optional: `rbac.yaml` – Roles and permissions merged into the system when present.
|
|
23
|
-
|
|
24
|
-
## Quick Start
|
|
25
|
-
|
|
26
|
-
### 1. Create External System
|
|
27
|
-
|
|
28
|
-
```bash
|
|
29
|
-
aifabrix create hubspot
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
(External is the default type. Use `aifabrix create <name> --type webapp` for a builder app.)
|
|
33
|
-
|
|
34
|
-
Or use the interactive wizard:
|
|
35
|
-
|
|
36
|
-
```bash
|
|
37
|
-
aifabrix wizard --app hubspot
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
### 2. Configure Authentication and Datasources
|
|
41
|
-
|
|
42
|
-
Edit files in `integration/hubspot/`:
|
|
43
|
-
|
|
44
|
-
- **Authentication**: `hubspot-system.yaml` (auth type, credentials placeholders)
|
|
45
|
-
- **Field mappings**: `hubspot-datasource-*.yaml` (dimensions, attributes, operations)
|
|
46
|
-
|
|
47
|
-
### 3. Validate Configuration
|
|
48
|
-
|
|
49
|
-
```bash
|
|
50
|
-
aifabrix validate hubspot --type external
|
|
51
|
-
```
|
|
52
|
-
|
|
53
|
-
### 4. Generate Deployment Manifest
|
|
54
|
-
|
|
55
|
-
```bash
|
|
56
|
-
aifabrix json hubspot --type external
|
|
57
|
-
```
|
|
58
|
-
|
|
59
|
-
This creates `hubspot-deploy.json` in `integration/hubspot/`.
|
|
60
|
-
|
|
61
|
-
### 5. Deploy
|
|
62
|
-
|
|
63
|
-
Controller URL and environment are read from config. Configure and log in first:
|
|
64
|
-
|
|
65
|
-
```bash
|
|
66
|
-
aifabrix auth config --set-controller <url> --set-environment dev
|
|
67
|
-
aifabrix login
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
Then deploy:
|
|
71
|
-
|
|
72
|
-
```bash
|
|
73
|
-
aifabrix deploy hubspot
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
## Testing
|
|
77
|
-
|
|
78
|
-
### Unit Tests (Local Validation, No API)
|
|
79
|
-
|
|
80
|
-
```bash
|
|
81
|
-
aifabrix test hubspot
|
|
82
|
-
```
|
|
83
|
-
|
|
84
|
-
### Integration Tests (Via Dataplane)
|
|
85
|
-
|
|
86
|
-
```bash
|
|
87
|
-
aifabrix test-integration hubspot
|
|
88
|
-
```
|
|
89
|
-
|
|
90
|
-
## Deployment
|
|
91
|
-
|
|
92
|
-
Deploy via miso-controller pipeline (same as regular apps). Auth and controller come from `aifabrix login` and `aifabrix auth config`:
|
|
93
|
-
|
|
94
|
-
```bash
|
|
95
|
-
aifabrix deploy hubspot
|
|
96
|
-
```
|
|
97
|
-
|
|
98
|
-
## Troubleshooting
|
|
99
|
-
|
|
100
|
-
- **Validation errors**: Run `aifabrix validate hubspot --type external` to see schema and manifest errors.
|
|
101
|
-
- **Deployment / auth**: Run `aifabrix auth config --set-controller <url> --set-environment <env>` and `aifabrix login` before `aifabrix deploy`.
|
|
102
|
-
- **File not found**: Run commands from the project root (where `package.json` and `integration/` live).
|