hubspot-cms-sync 0.1.0

package/docs/PLAN_REVIEW.md

@@ -0,0 +1,42 @@
+# Plan Review Notes
+
+This captures the read-only Claude review of the initial extraction plan.
+
+## Critical Findings
+
+1. **Move the tests with the engine.** The website repo unit/integration tests
+   import `sync/**` directly. If the website deletes local sync code before the
+   tests move or repoint to package exports, `npm run test:unit` will fail.
+
+2. **Move the corpus scanner.** `hcms corpus` depends on
+   `scripts/corpus-scan.mjs`, which lives outside `sync/` and was missing from
+   the initial inventory.
+
+3. **Make the read-only guard config-driven.** The current single hardcoded
+   `READ_ONLY_PORTAL = '529456'` must become `config.readOnlyPortalIds` while
+   preserving fail-closed behavior.
+
+4. **Thread config explicitly.** Several modules resolve paths from `__dirname`;
+   that will point into the npm package once installed. Build a resolved config
+   object once in the CLI and pass it into commands/libs/adapters instead of using
+   package-relative roots or global mutable config.
+
+5. **Scrub publish artifacts.** Docs/examples currently mention private portal
+   IDs. Before npm publication, scrub all shipped docs/examples or exclude them
+   from `package.json#files`.
+
+## Plan Updates Made
+
+- Added `scripts/corpus-scan.mjs`, `test/unit/**`, and `test/integration/**` to
+  the migration inventory.
+- Added explicit root/config threading requirements and named the affected
+  modules.
+- Added acceptance gates for config loader behavior, known portal IDs, and
+  read-only portal arrays.
+- Added v1 limitations for CTA/menu producers, legacy asset hosts, and global
+  transaction limits.
+- Aligned the example config with the documented schema.
+- Added `prepublishOnly` placeholder guard in `package.json`.
+- Updated the skill plan to keep the deterministic engine in npm and use the
+  skill only as an agent workflow wrapper.
+

package/docs/SKILL_DISTRIBUTION.md

@@ -0,0 +1,79 @@
+# Skill Distribution Plan
+
+The Codex skill should be a companion to the npm CLI, not a replacement for it.
+
+## Skill Purpose
+
+Help an agent operate `hubspot-cms-sync` correctly inside a project:
+
+- inspect config and manifest
+- run preflight and corpus checks
+- perform pull/push/republish flows
+- interpret common failures
+- manage preview/deployment PR gates
+- capture and compare screenshots through the consuming repo's configured tests
+
+The npm CLI owns deterministic behavior. The skill owns agent procedure,
+interpretation, and safe sequencing.
+
+## Proposed Skill Layout
+
+```text
+hubspot-cms-sync/
+├── SKILL.md
+└── references/
+    ├── commands.md
+    ├── config.md
+    ├── failures.md
+    ├── github-actions.md
+    └── screenshots-and-fidelity.md
+```
+
+Do not bundle the full sync implementation into the skill. The deterministic
+engine lives in npm.
+
+`SKILL.md` with YAML frontmatter is the portable core. Agent-specific metadata
+files such as `agents/openai.yaml` can be generated later for marketplaces that
+need them, but they should not be the primary source of truth.
+
+## SKILL.md Scope
+
+The skill body should stay short:
+
+1. Confirm `hubspot-cms-sync` / `hcms` is installed.
+2. Read `hubspot-cms-sync.config.mjs`.
+3. Run `hcms doctor` before risky operations.
+4. For pull: run pull, corpus, git diff, and summarize.
+5. For push: run corpus, preflight, plan, push, republish if needed, verify.
+6. For CI failures: inspect logs, classify failure, rerun the minimum command.
+7. Never bypass read-only portal guards.
+8. Surface v1 engine limits clearly: surviving `@cta:*` or `@menu:*` refs fail
+   closed until producer adapters exist; global HubSpot writes are rerun-to-
+   convergence, not transactional rollback.
+
+## References
+
+- `commands.md`: command matrix and expected outputs.
+- `config.md`: config schema and examples.
+- `failures.md`: HubSpot API failures and remediation text.
+- `github-actions.md`: environment secrets, preview, deploy, PR gates.
+- `screenshots-and-fidelity.md`: Playwright screenshot workflows and baseline
+  update rules.
+
+## Example User Prompts
+
+- "Pull HubSpot prod into git and summarize the diff."
+- "Deploy this PR to the HubSpot preview sandbox and run fidelity checks."
+- "Why did `hcms preflight dev` fail?"
+- "Republish all live pages after this template change."
+- "Set up HubSpot CMS sync in this repo."
+
+## Skill Guardrails
+
+- The skill must not tell the agent to edit `.sync-state` manually.
+- The skill must not suggest pushing to a read-only portal.
+- The skill should prefer `hcms push --dry-run` and `hcms preflight` before any write.
+- The skill should report verification gaps clearly if credentials or a live
+  preview URL are unavailable.
+- The skill should source common failure explanations from CLI output and
+  package references, not invent parallel remediation text.

package/examples/github-actions/ci.yml

@@ -0,0 +1,56 @@
+name: HubSpot CMS Sync CI
+
+on:
+  pull_request:
+    branches: [main]
+  push:
+    branches: [main]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: hcms-ci-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  HUBSPOT_KEY_DIR: ${{ runner.temp }}/hubspot-keys
+  HCMS_TARGET: dev
+
+jobs:
+  hcms-ci:
+    name: Validate HubSpot CMS sync inputs
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Set up Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+
+      - name: Install project dependencies
+        run: npm ci
+
+      - name: Hydrate read-only HubSpot credentials
+        env:
+          DEV_TOKEN: ${{ secrets.HUBSPOT_DEV_PRIVATE_APP_TOKEN }}
+        run: |
+          mkdir -p "$HUBSPOT_KEY_DIR"
+          token_path="$HUBSPOT_KEY_DIR/dev.private-app-token"
+          printf '%s' "$DEV_TOKEN" > "$token_path"
+          chmod 600 "$token_path"
+
+      - name: Run sync doctor
+        run: npx --yes hubspot-cms-sync@latest doctor --root .
+
+      - name: Validate local corpus
+        run: npx --yes hubspot-cms-sync@latest corpus --root .
+
+      - name: Run read-only preflight
+        run: |
+          npx --yes hubspot-cms-sync@latest preflight \
+            "$HCMS_TARGET" --root . --read-only

package/examples/github-actions/preview.yml

@@ -0,0 +1,71 @@
+name: HubSpot CMS Sync Preview
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]
+    branches: [main]
+
+permissions:
+  contents: read
+  pull-requests: write
+
+concurrency:
+  group: hcms-preview-pr-${{ github.event.pull_request.number }}
+  cancel-in-progress: true
+
+env:
+  HUBSPOT_KEY_DIR: ${{ runner.temp }}/hubspot-keys
+  HCMS_TARGET: preview
+  SITE_BASE_URL: ${{ secrets.PREVIEW_SITE_BASE_URL }}
+
+jobs:
+  preview:
+    name: Deploy PR preview
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft }}
+
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Set up Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+
+      - name: Install project dependencies
+        run: npm ci
+
+      - name: Hydrate preview HubSpot credentials
+        env:
+          PREVIEW_TOKEN: ${{ secrets.HUBSPOT_PREVIEW_PRIVATE_APP_TOKEN }}
+        run: |
+          mkdir -p "$HUBSPOT_KEY_DIR"
+          token_path="$HUBSPOT_KEY_DIR/preview.private-app-token"
+          printf '%s' "$PREVIEW_TOKEN" > "$token_path"
+          chmod 600 "$token_path"
+
+      - name: Validate preview deployment plan
+        run: |
+          npx --yes hubspot-cms-sync@latest doctor --root .
+          npx --yes hubspot-cms-sync@latest corpus --root .
+          npx --yes hubspot-cms-sync@latest preflight "$HCMS_TARGET" --root .
+          npx --yes hubspot-cms-sync@latest plan \
+            "$HCMS_TARGET" --root . --out .hcms/preview-plan.md
+
+      - name: Push preview content
+        run: |
+          npx --yes hubspot-cms-sync@latest push \
+            "$HCMS_TARGET" --root . --publish
+
+      - name: Verify preview
+        run: npx --yes hubspot-cms-sync@latest verify "$HCMS_TARGET" --root .
+
+      - name: Upload preview plan
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: hcms-preview-plan
+          path: .hcms/preview-plan.md
+          if-no-files-found: ignore

package/examples/github-actions/publish.yml

@@ -0,0 +1,82 @@
+name: HubSpot CMS Sync Publish
+
+on:
+  workflow_dispatch:
+    inputs:
+      target:
+        description: HubSpot sync target from the consuming repo config
+        required: true
+        default: prod
+        type: choice
+        options:
+          - prod
+          - staging
+      dry_run:
+        description: Build and validate the plan without writing to HubSpot
+        required: true
+        default: true
+        type: boolean
+
+permissions:
+  contents: read
+  deployments: write
+
+concurrency:
+  group: hcms-publish-${{ inputs.target }}
+  cancel-in-progress: false
+
+env:
+  HUBSPOT_KEY_DIR: ${{ runner.temp }}/hubspot-keys
+  HCMS_TARGET: ${{ inputs.target }}
+  SITE_BASE_URL: ${{ secrets.SITE_BASE_URL }}
+
+jobs:
+  publish:
+    name: Publish HubSpot CMS target
+    runs-on: ubuntu-latest
+    environment: ${{ inputs.target }}
+
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Set up Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+
+      - name: Install project dependencies
+        run: npm ci
+
+      - name: Hydrate target HubSpot credentials
+        env:
+          PROD_TOKEN: ${{ secrets.HUBSPOT_PROD_PRIVATE_APP_TOKEN }}
+        run: |
+          mkdir -p "$HUBSPOT_KEY_DIR"
+          token_path="$HUBSPOT_KEY_DIR/prod.private-app-token"
+          printf '%s' "$PROD_TOKEN" > "$token_path"
+          chmod 600 "$token_path"
+
+      - name: Build publish plan
+        run: |
+          npx --yes hubspot-cms-sync@latest doctor --root .
+          npx --yes hubspot-cms-sync@latest corpus --root .
+          npx --yes hubspot-cms-sync@latest preflight "$HCMS_TARGET" --root .
+          npx --yes hubspot-cms-sync@latest plan \
+            "$HCMS_TARGET" --root . --out .hcms/publish-plan.md
+
+      - name: Publish to HubSpot
+        if: ${{ !inputs.dry_run }}
+        run: |
+          npx --yes hubspot-cms-sync@latest push \
+            "$HCMS_TARGET" --root . --publish
+          npx --yes hubspot-cms-sync@latest verify "$HCMS_TARGET" --root .
+
+      - name: Upload publish plan
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: hcms-publish-plan-${{ inputs.target }}
+          path: .hcms/publish-plan.md
+          if-no-files-found: ignore

package/examples/hubspot-cms-sync.config.mjs

@@ -0,0 +1,45 @@
+export default {
+  accountsFile: 'sync/accounts.json',
+  keyDirEnv: 'HUBSPOT_KEY_DIR',
+  contentDir: 'content',
+  syncStateDir: '.sync-state',
+  manifestPath: 'site.manifest.json',
+  readOnlyPortalIds: [],
+  knownPortalIds: [],
+  assetHosts: {
+    canonicalizeHostPatterns: [
+      'hubfs',
+      'hubspotusercontent',
+      'cdn\\d*\\.hubspot\\.net'
+    ],
+    legacySiteHosts: []
+  },
+  adapters: {
+    externalDirs: []
+  },
+  theme: {
+    name: 'example-theme',
+    dirs: ['templates', 'modules', 'css', 'js', 'images'],
+    files: ['theme.json', 'fields.json']
+  },
+  blog: {
+    slug: 'blog',
+    itemTemplate: 'example-theme/templates/blog-post.html',
+    listingTemplate: 'example-theme/templates/blog.html'
+  },
+  uiGated: [
+    'blogContainerCreate',
+    'domainConnect',
+    'homepageDesignation',
+    'themeSettingsValues',
+    'nativeMenus'
+  ],
+  verification: {
+    baseUrlEnv: 'SITE_BASE_URL',
+    commands: {
+      unit: 'npm run test:unit',
+      corpus: 'hcms corpus',
+      playwright: 'npx playwright test verify/fidelity.spec.mjs verify/forms.spec.mjs verify/links.spec.mjs'
+    }
+  }
+};

package/examples/site.manifest.json

@@ -0,0 +1,19 @@
+{
+  "theme": {
+    "name": "example-theme"
+  },
+  "blog": {
+    "slug": "blog",
+    "itemTemplate": "example-theme/templates/blog-post.html",
+    "listingTemplate": "example-theme/templates/blog.html"
+  },
+  "forms": [],
+  "pages": [],
+  "uiGated": [
+    "blogContainerCreate",
+    "domainConnect",
+    "homepageDesignation",
+    "themeSettingsValues",
+    "nativeMenus"
+  ]
+}

package/package.json

@@ -0,0 +1,41 @@
+{
+  "name": "hubspot-cms-sync",
+  "version": "0.1.0",
+  "description": "Git-backed bidirectional sync for HubSpot CMS themes, content, blogs, forms, and assets.",
+  "type": "module",
+  "license": "MIT",
+  "engines": {
+    "node": ">=20.10.0",
+    "npm": ">=10"
+  },
+  "bin": {
+    "hubspot-cms-sync": "./bin/hubspot-cms-sync.mjs",
+    "hcms": "./bin/hubspot-cms-sync.mjs"
+  },
+  "exports": {
+    ".": "./src/index.mjs",
+    "./adapters/*": "./src/adapters/*.mjs",
+    "./lib/*": "./src/lib/*.mjs",
+    "./manifest": "./src/manifest.mjs",
+    "./preflight": "./src/preflight.mjs",
+    "./cta-inventory": "./src/cta-inventory.mjs",
+    "./corpus-scan": "./src/corpus-scan.mjs"
+  },
+  "files": [
+    "bin/",
+    "src/",
+    "docs/",
+    "examples/",
+    "skill/",
+    "README.md",
+    "LICENSE"
+  ],
+  "scripts": {
+    "test": "node --test test/**/*.test.mjs",
+    "test:unit": "node --test test/unit/*.test.mjs test/integration/corpus.test.mjs",
+    "test:integration": "RUN_INTEGRATION=1 node --test test/integration/roundtrip.test.mjs",
+    "lint": "node --check bin/hubspot-cms-sync.mjs && node bin/hubspot-cms-sync.mjs --help >/dev/null",
+    "prepare": "npm run lint",
+    "prepublishOnly": "npm test && npm run lint && npm pack --dry-run"
+  }
+}

package/skill/SKILL.md

@@ -0,0 +1,54 @@
+---
+name: hubspot-cms-sync
+description: Use when operating the hubspot-cms-sync CLI in a repository: inspect config and manifest files, run doctor/corpus/preflight/pull/push/republish/verify flows, debug GitHub Actions failures, and safely manage HubSpot CMS preview or publish workflows.
+---
+
+# HubSpot CMS Sync
+
+Use this skill to operate `hubspot-cms-sync` or `hcms` inside a consuming repo.
+The CLI owns deterministic sync behavior; the agent owns sequencing,
+interpretation, and clear reporting.
+
+## First Checks
+
+1. Confirm the repo has `hubspot-cms-sync.config.mjs` and `site.manifest.json`.
+2. Confirm `hubspot-cms-sync` or `hcms` is available. If not, use the package
+   command documented by the repo.
+3. Read the config before choosing targets, credential environment variables, or
+   verification commands.
+4. Run `hcms doctor` before risky operations.
+5. Never bypass read-only portal guards or write to a production portal from a
+   pull request workflow.
+
+## Common Flows
+
+- Pull: run `hcms pull <target>`, `hcms corpus`, inspect `git diff`, then
+  summarize content changes and verification gaps.
+- Push: run `hcms corpus`, `hcms preflight <target>`, `hcms push <target> --dry-run`,
+  then `hcms push <target> --publish` only after the checks match intent.
+- Republish: run `hcms preflight <target>`, then `hcms republish <target>` with
+  the narrowest flags needed.
+- Verify: use the consuming repo's configured tests after the CLI checks pass.
+- CI failure: inspect logs, classify the failure, rerun the smallest local
+  command that reproduces it, and avoid broad retries until the cause is known.
+
+## References
+
+Load only the reference needed for the task:
+
+- `references/commands.md`: command matrix and expected sequencing.
+- `references/config.md`: config and manifest fields to inspect.
+- `references/failures.md`: common failure classes and remediation.
+- `references/github-actions.md`: CI, preview, and publish workflow policy.
+- `references/screenshots-and-fidelity.md`: visual verification workflows.
+
+## Guardrails
+
+- Do not edit `.sync-state` by hand.
+- Do not suggest pushing to a configured read-only portal.
+- Prefer `hcms push --dry-run` and `hcms preflight` before any write.
+- Report missing credentials or preview URLs as verification gaps.
+- Treat surviving `@cta:*` or `@menu:*` references as closed failures until
+  producer adapters exist.
+- Treat HubSpot writes as rerun-to-convergence operations, not transactional
+  rollbacks.

package/skill/references/commands.md

@@ -0,0 +1,54 @@
+# Command Reference
+
+Commands are shown with the short alias `hcms`. Use `hubspot-cms-sync` if the
+repo has not installed the alias.
+
+## Inspection
+
+- `hcms doctor`: validate config, credentials, target definitions, and local
+  filesystem assumptions.
+- `hcms corpus`: validate the local content corpus and adapter references.
+- `hcms push <target> --dry-run`: produce a write plan without applying it.
+
+## Pull
+
+Use pull when HubSpot is the source of truth for the current task.
+
+```bash
+hcms doctor
+hcms pull <target>
+hcms corpus
+git diff --stat
+git diff
+```
+
+Summarize changed files, content objects, and any verification that could not be
+run.
+
+## Push
+
+Use push when git is the source of truth and the target is write-capable.
+
+```bash
+hcms doctor
+hcms corpus
+hcms preflight <target>
+hcms push <target> --dry-run
+hcms push <target> --publish
+the consuming repo verification commands
+```
+
+Stop before `push` if preflight or plan output indicates a read-only portal,
+unresolved dependency, missing credential, or unexpected destructive change.
+
+## Republish
+
+Use the narrowest republish scope that satisfies the change.
+
+```bash
+hcms preflight <target>
+hcms republish <target> --all
+the consuming repo verification commands
+```
+
+Prefer explicit page, blog, or template scopes when the CLI supports them.

package/skill/references/config.md

@@ -0,0 +1,25 @@
+# Config And Manifest Reference
+
+Read `hubspot-cms-sync.config.mjs` before selecting targets or commands.
+
+Key fields:
+
+- `accountsFile`: maps target names to HubSpot account details.
+- `keyDirEnv`: environment variable pointing at hydrated credentials.
+- `contentDir`: local CMS content directory.
+- `syncStateDir`: local sync state directory; do not edit by hand.
+- `manifestPath`: path to `site.manifest.json`.
+- `readOnlyPortalIds`: portals that must never receive writes.
+- `knownPortalIds`: expected portal allowlist.
+- `assetHosts`: host canonicalization policy for HubSpot assets.
+- `adapters.externalDirs`: optional consumer-owned adapters.
+- `theme`: theme directory and file layout.
+- `blog`: blog slug and template mapping.
+- `uiGated`: operations that require HubSpot UI action.
+- `verification`: base URL environment variable and repo-specific test commands.
+
+`site.manifest.json` is the deployment surface for theme, blog, forms, pages,
+and UI-gated operations. If the manifest and config disagree, stop and ask for
+the intended source of truth.
+
+Use paths relative to the repo root unless the config explicitly says otherwise.

package/skill/references/failures.md

@@ -0,0 +1,58 @@
+# Failure Reference
+
+Prefer the CLI's error text and remediation output. Use this file to classify
+failures and choose the next diagnostic step.
+
+## Configuration
+
+Symptoms:
+
+- Missing `hubspot-cms-sync.config.mjs`.
+- Invalid config shape.
+- Target name is not present in `accountsFile`.
+- `keyDirEnv` is unset or points at missing credentials.
+
+Action: fix config or CI secret hydration before retrying sync commands.
+
+## Read-Only Portal
+
+Symptoms:
+
+- Target portal ID is listed in `readOnlyPortalIds`.
+- Preflight blocks write operations.
+
+Action: do not bypass the guard. Choose a write-capable preview target or ask
+for an explicit config change.
+
+## Dependency References
+
+Symptoms:
+
+- Surviving `@cta:*` or `@menu:*` references.
+- Missing producer adapter output.
+- Manifest objects refer to absent content.
+
+Action: fail closed until the producer adapter or source content exists.
+
+## HubSpot API
+
+Symptoms:
+
+- Authentication failure.
+- Rate limit or transient 5xx response.
+- Validation error from a HubSpot object endpoint.
+
+Action: distinguish credential/config failures from transient API failures. For
+transient failures, retry the smallest failed operation. For validation errors,
+fix the source object or adapter output.
+
+## Verification
+
+Symptoms:
+
+- Missing `SITE_BASE_URL` or configured base URL env var.
+- Playwright or link checks cannot reach the preview.
+- Screenshots differ after a template or module change.
+
+Action: report unavailable verification separately from failed verification.
+When screenshots differ, include the affected page and artifact path.

package/skill/references/github-actions.md

@@ -0,0 +1,56 @@
+# GitHub Actions Reference
+
+Use the repository's workflow files first. If adding new workflows, start from
+the package examples:
+
+- `examples/github-actions/ci.yml`
+- `examples/github-actions/preview.yml`
+- `examples/github-actions/publish.yml`
+
+## CI
+
+CI should be read-only. It can run:
+
+```bash
+hcms doctor
+hcms corpus
+hcms preflight <non-prod-target> --read-only
+```
+
+Do not expose production credentials to `pull_request` events.
+
+## Preview
+
+Preview workflows should use a non-prod HubSpot target and PR-level
+concurrency.
+
+Typical sequence:
+
+```bash
+hcms doctor
+hcms corpus
+hcms preflight preview
+hcms push preview --dry-run
+hcms push preview --publish
+the preview verification commands
+```
+
+Upload the plan and verification artifacts when available.
+
+## Publish
+
+Publish workflows should be manual or release-driven and protected by a GitHub
+Environment.
+
+Typical sequence:
+
+```bash
+hcms doctor
+hcms corpus
+hcms preflight prod
+hcms push prod --dry-run
+hcms push prod --publish
+the production verification commands
+```
+
+Keep a dry-run option for validating the plan without writing to HubSpot.