hubspot-cms-sync 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +1 -1
- package/README.md +52 -12
- package/bin/hubspot-cms-sync.mjs +123 -94
- package/docs/CONFIGURATION.md +5 -0
- package/docs/CONTENT_LAYOUT.md +146 -0
- package/docs/GITHUB_ACTIONS.md +3 -1
- package/docs/HUBSPOT-SYNC-NOTES.md +123 -0
- package/examples/hubspot-cms-sync.config.mjs +1 -0
- package/examples/minimal-site/content/assets/hero.svg +9 -0
- package/examples/minimal-site/content/blog/container.json +6 -0
- package/examples/minimal-site/content/blog/posts/blog__hello-world.json +13 -0
- package/examples/minimal-site/content/forms/contact.json +18 -0
- package/examples/minimal-site/content/forms/properties.json +7 -0
- package/examples/minimal-site/content/pages/about.json +9 -0
- package/examples/minimal-site/content/pages/home.json +9 -0
- package/examples/minimal-site/content/pages/home.widgets.json +17 -0
- package/examples/minimal-site/css/main.css +3 -0
- package/examples/minimal-site/fields.json +11 -0
- package/examples/minimal-site/hubspot-cms-sync.config.mjs +27 -0
- package/examples/minimal-site/js/hs-forms.js +1 -0
- package/examples/minimal-site/modules/hero.module/fields.json +14 -0
- package/examples/minimal-site/modules/hero.module/module.html +4 -0
- package/examples/minimal-site/site.manifest.json +29 -0
- package/examples/minimal-site/sync/accounts.json +10 -0
- package/examples/minimal-site/sync/redirects.csv +2 -0
- package/examples/minimal-site/templates/blog-post.html +8 -0
- package/examples/minimal-site/templates/blog.html +9 -0
- package/examples/minimal-site/templates/home.html +5 -0
- package/examples/minimal-site/templates/page.html +7 -0
- package/examples/minimal-site/theme.json +4 -0
- package/package.json +17 -3
- package/skill/references/commands.md +4 -0
- package/skill/references/config.md +6 -2
- package/src/config.mjs +5 -0
- package/src/index.mjs +1 -0
- package/src/lib/content-view.mjs +168 -0
- package/src/lib/posts-format.mjs +90 -0
- package/src/lib/render.mjs +153 -0
- package/src/preflight.mjs +97 -8
- package/src/redirects.mjs +283 -0
package/LICENSE
CHANGED
package/README.md
CHANGED
|
@@ -3,10 +3,16 @@
|
|
|
3
3
|
Git-backed bidirectional HubSpot CMS sync for themes, site pages, page module
|
|
4
4
|
content, blogs, forms, and assets.
|
|
5
5
|
|
|
6
|
-
The package provides the `
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
6
|
+
The package provides the `hcms` CLI. The long binary name
|
|
7
|
+
`hubspot-cms-sync` is also installed, but examples use `hcms` for consistency.
|
|
8
|
+
The tool reads account and site settings from `hubspot-cms-sync.config.mjs`,
|
|
9
|
+
stores per-account identity in a gitignored `.sync-state/` directory, and
|
|
10
|
+
refuses to write to configured read-only portal ids.
|
|
11
|
+
|
|
12
|
+
See [`docs/CONTENT_LAYOUT.md`](docs/CONTENT_LAYOUT.md) for the repository
|
|
13
|
+
layout, supported push surfaces, and a minimal example tree. A runnable fixture
|
|
14
|
+
lives in [`examples/minimal-site/`](examples/minimal-site/) and is validated by
|
|
15
|
+
the unit tests.
|
|
10
16
|
|
|
11
17
|
## Install
|
|
12
18
|
|
|
@@ -24,21 +30,55 @@ npm install --save-dev ../hubspot-cms-sync
|
|
|
24
30
|
|
|
25
31
|
```bash
|
|
26
32
|
hcms doctor
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
hubspot-cms-sync push dev --publish
|
|
33
|
+
hcms pull dev
|
|
34
|
+
hcms preflight dev
|
|
30
35
|
hcms push dev --dry-run
|
|
31
|
-
|
|
32
|
-
|
|
36
|
+
hcms push dev --publish
|
|
37
|
+
hcms redirects dev
|
|
38
|
+
hcms redirects dev --apply
|
|
39
|
+
hcms republish dev --all --blog
|
|
40
|
+
hcms corpus
|
|
33
41
|
hcms manifest validate
|
|
34
42
|
```
|
|
35
43
|
|
|
44
|
+
`hcms redirects` is dry-run by default. Pass `--apply` to create or update
|
|
45
|
+
HubSpot URL redirects from the configured `redirectsFile`, or pass `--file` to
|
|
46
|
+
use a specific repo-stored CSV or JSON spec. Managed redirects
|
|
47
|
+
default to `301` and `isOnlyAfterNotFound: false`, so they can intentionally
|
|
48
|
+
take precedence over an existing live HubSpot page during a cutover.
|
|
49
|
+
|
|
36
50
|
## Configuration
|
|
37
51
|
|
|
38
52
|
Copy `examples/hubspot-cms-sync.config.mjs` into the consuming repo, then set
|
|
39
|
-
the theme name, manifest path, read-only portal ids, and
|
|
40
|
-
Account keys live outside git at
|
|
41
|
-
`~/.hubspot/<portalId>.key`.
|
|
53
|
+
the theme name, manifest path, redirect spec path, read-only portal ids, and
|
|
54
|
+
account registry path. Account keys live outside git at
|
|
55
|
+
`$HUBSPOT_KEY_DIR/<portalId>.key` or `~/.hubspot/<portalId>.key`.
|
|
56
|
+
|
|
57
|
+
```js
|
|
58
|
+
export default {
|
|
59
|
+
accountsFile: 'sync/accounts.json',
|
|
60
|
+
contentDir: 'content',
|
|
61
|
+
manifestPath: 'site.manifest.json',
|
|
62
|
+
redirectsFile: 'content/redirects.csv',
|
|
63
|
+
readOnlyPortalIds: ['529456'],
|
|
64
|
+
theme: {
|
|
65
|
+
name: 'seventh-sense-theme',
|
|
66
|
+
dirs: ['templates', 'modules', 'css', 'js', 'images'],
|
|
67
|
+
files: ['theme.json', 'fields.json']
|
|
68
|
+
}
|
|
69
|
+
};
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
CSV redirects need at least these columns:
|
|
73
|
+
|
|
74
|
+
```csv
|
|
75
|
+
routePrefix,destination,redirectStyle
|
|
76
|
+
/old-page,/new-page,301
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Optional columns accepted by the HubSpot URL Redirects API include
|
|
80
|
+
`isOnlyAfterNotFound`, `isMatchFullUrl`, `isMatchQueryString`, `isPattern`,
|
|
81
|
+
`isProtocolAgnostic`, `isTrailingSlashOptional`, and `precedence`.
|
|
42
82
|
|
|
43
83
|
## Tests
|
|
44
84
|
|
package/bin/hubspot-cms-sync.mjs
CHANGED
|
@@ -1,39 +1,15 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
|
|
3
3
|
import { spawn } from 'node:child_process';
|
|
4
|
+
import { Command } from 'commander';
|
|
5
|
+
|
|
4
6
|
import { loadConfig } from '../src/config.mjs';
|
|
5
7
|
import { pull } from '../src/pull.mjs';
|
|
6
|
-
import { push } from '../src/push.mjs';
|
|
8
|
+
import { push, preflightRefs } from '../src/push.mjs';
|
|
7
9
|
import { main as preflightMain } from '../src/preflight.mjs';
|
|
8
10
|
import { main as republishMain } from '../src/republish.mjs';
|
|
9
11
|
import { main as manifestMain } from '../src/manifest.mjs';
|
|
10
|
-
|
|
11
|
-
function usage() {
|
|
12
|
-
console.log(`usage: hcms [--root <path>] <command> [args]
|
|
13
|
-
|
|
14
|
-
commands:
|
|
15
|
-
pull <account>
|
|
16
|
-
push <account> [--publish] [--dry-run]
|
|
17
|
-
preflight <account>
|
|
18
|
-
republish <account|portalId> [--all] [--blog] [slugs...]
|
|
19
|
-
corpus [paths...]
|
|
20
|
-
manifest [args...]
|
|
21
|
-
doctor
|
|
22
|
-
`);
|
|
23
|
-
}
|
|
24
|
-
|
|
25
|
-
function parseGlobal(argv) {
|
|
26
|
-
const out = { root: process.cwd(), args: [] };
|
|
27
|
-
for (let i = 0; i < argv.length; i += 1) {
|
|
28
|
-
const a = argv[i];
|
|
29
|
-
if (a === '--root') {
|
|
30
|
-
out.root = argv[++i];
|
|
31
|
-
} else {
|
|
32
|
-
out.args.push(a);
|
|
33
|
-
}
|
|
34
|
-
}
|
|
35
|
-
return out;
|
|
36
|
-
}
|
|
12
|
+
import { renderRedirectReport, syncRedirects } from '../src/redirects.mjs';
|
|
37
13
|
|
|
38
14
|
function runNodeScript(script, args, { cwd }) {
|
|
39
15
|
return new Promise((resolve) => {
|
|
@@ -42,74 +18,127 @@ function runNodeScript(script, args, { cwd }) {
|
|
|
42
18
|
});
|
|
43
19
|
}
|
|
44
20
|
|
|
45
|
-
async function
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
21
|
+
async function withConfig(opts) {
|
|
22
|
+
return loadConfig({ root: opts.root });
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
async function main(argv = process.argv) {
|
|
26
|
+
const program = new Command();
|
|
27
|
+
|
|
28
|
+
program
|
|
29
|
+
.name('hcms')
|
|
30
|
+
.description('Git-backed HubSpot CMS sync')
|
|
31
|
+
.option('--root <path>', 'repo root', process.cwd())
|
|
32
|
+
.showHelpAfterError();
|
|
33
|
+
|
|
34
|
+
program
|
|
35
|
+
.command('doctor')
|
|
36
|
+
.description('print resolved configuration')
|
|
37
|
+
.action(async () => {
|
|
38
|
+
const config = await withConfig(program.opts());
|
|
39
|
+
console.log(`root: ${config.root}`);
|
|
40
|
+
console.log(`accounts: ${config.accountsPath}`);
|
|
41
|
+
console.log(`content: ${config.contentDirPath}`);
|
|
42
|
+
console.log(`manifest: ${config.manifestFilePath}`);
|
|
43
|
+
console.log(`redirects: ${config.redirectsFilePath || '(none)'}`);
|
|
44
|
+
console.log(`sync state: ${config.syncStateDirPath}`);
|
|
45
|
+
console.log(`theme: ${config.theme.name}`);
|
|
46
|
+
});
|
|
47
|
+
|
|
48
|
+
program
|
|
49
|
+
.command('pull')
|
|
50
|
+
.description('pull HubSpot content into the repo')
|
|
51
|
+
.argument('<account>')
|
|
52
|
+
.action(async (account) => {
|
|
53
|
+
const config = await withConfig(program.opts());
|
|
54
|
+
await pull(account, { config });
|
|
55
|
+
});
|
|
56
|
+
|
|
57
|
+
program
|
|
58
|
+
.command('push')
|
|
59
|
+
.description('push repo content to HubSpot')
|
|
60
|
+
.argument('<account>')
|
|
61
|
+
.option('--publish', 'publish/schedule pushed content')
|
|
62
|
+
.option('--dry-run', 'run local push preflight only')
|
|
63
|
+
.action(async (account, options) => {
|
|
64
|
+
const config = await withConfig(program.opts());
|
|
65
|
+
if (options.dryRun) {
|
|
66
|
+
preflightRefs(config.contentDirPath);
|
|
67
|
+
console.log(`dry-run push preflight passed for ${account}`);
|
|
68
|
+
return;
|
|
69
|
+
}
|
|
70
|
+
await push(account, { publish: !!options.publish, config });
|
|
71
|
+
});
|
|
72
|
+
|
|
73
|
+
program
|
|
74
|
+
.command('preflight')
|
|
75
|
+
.description('check account readiness before a push')
|
|
76
|
+
.argument('<account>')
|
|
77
|
+
.option('--allow-repairable', 'allow source-repairable portal drift before the push')
|
|
78
|
+
.action(async (account, options) => {
|
|
79
|
+
const config = await withConfig(program.opts());
|
|
80
|
+
const args = [account];
|
|
81
|
+
if (options.allowRepairable) args.push('--allow-repairable');
|
|
82
|
+
const code = await preflightMain(args, { config });
|
|
83
|
+
if (code) process.exitCode = code;
|
|
84
|
+
});
|
|
85
|
+
|
|
86
|
+
program
|
|
87
|
+
.command('redirects')
|
|
88
|
+
.description('plan or apply managed URL redirects')
|
|
89
|
+
.argument('<account>')
|
|
90
|
+
.option('--file <path>', 'redirect spec CSV or JSON; defaults to config.redirectsFile')
|
|
91
|
+
.option('--apply', 'write creates/updates to HubSpot')
|
|
92
|
+
.action(async (account, options) => {
|
|
93
|
+
const config = await withConfig(program.opts());
|
|
94
|
+
const result = await syncRedirects(account, {
|
|
95
|
+
file: options.file,
|
|
96
|
+
apply: !!options.apply,
|
|
97
|
+
config,
|
|
98
|
+
});
|
|
99
|
+
console.log(renderRedirectReport(result));
|
|
100
|
+
});
|
|
101
|
+
|
|
102
|
+
program
|
|
103
|
+
.command('republish')
|
|
104
|
+
.description('republish live pages and/or blog posts')
|
|
105
|
+
.allowUnknownOption()
|
|
106
|
+
.allowExcessArguments()
|
|
107
|
+
.argument('[args...]')
|
|
108
|
+
.action(async (args) => {
|
|
109
|
+
const config = await withConfig(program.opts());
|
|
110
|
+
const code = await republishMain(args, { config });
|
|
111
|
+
if (code) process.exitCode = code;
|
|
112
|
+
});
|
|
113
|
+
|
|
114
|
+
program
|
|
115
|
+
.command('corpus')
|
|
116
|
+
.description('scan repo content for unsafe refs and HubSpot artifacts')
|
|
117
|
+
.allowUnknownOption()
|
|
118
|
+
.argument('[paths...]')
|
|
119
|
+
.action(async (paths) => {
|
|
120
|
+
const config = await withConfig(program.opts());
|
|
121
|
+
const script = new URL('../src/corpus-scan.mjs', import.meta.url).pathname;
|
|
122
|
+
const code = await runNodeScript(script, paths, { cwd: config.root });
|
|
123
|
+
if (code) process.exitCode = code;
|
|
124
|
+
});
|
|
125
|
+
|
|
126
|
+
program
|
|
127
|
+
.command('manifest')
|
|
128
|
+
.description('manifest utilities')
|
|
129
|
+
.allowUnknownOption()
|
|
130
|
+
.allowExcessArguments()
|
|
131
|
+
.argument('[args...]')
|
|
132
|
+
.action(async (args) => {
|
|
133
|
+
const config = await withConfig(program.opts());
|
|
134
|
+
const code = await manifestMain(args, { config });
|
|
135
|
+
if (code) process.exitCode = code;
|
|
136
|
+
});
|
|
137
|
+
|
|
138
|
+
await program.parseAsync(argv);
|
|
108
139
|
}
|
|
109
140
|
|
|
110
|
-
main().
|
|
111
|
-
process.exitCode = code;
|
|
112
|
-
}).catch((e) => {
|
|
141
|
+
main().catch((e) => {
|
|
113
142
|
console.error(`hcms failed: ${e.message}`);
|
|
114
143
|
process.exitCode = 1;
|
|
115
144
|
});
|
package/docs/CONFIGURATION.md
CHANGED
|
@@ -12,6 +12,7 @@ export default {
|
|
|
12
12
|
contentDir: 'content',
|
|
13
13
|
syncStateDir: '.sync-state',
|
|
14
14
|
manifestPath: 'site.manifest.json',
|
|
15
|
+
redirectsFile: 'content/redirects.csv',
|
|
15
16
|
readOnlyPortalIds: ['529456'],
|
|
16
17
|
knownPortalIds: ['529456', '246389711'],
|
|
17
18
|
assetHosts: {
|
|
@@ -63,8 +64,12 @@ export default {
|
|
|
63
64
|
- The config loader validates shape and prints remediation, not stack traces.
|
|
64
65
|
- `site.manifest.json` remains the deploy surface for content, pages, forms, and
|
|
65
66
|
blog.
|
|
67
|
+
- Repo-stored redirects are a separate deploy surface. `redirectsFile` points to
|
|
68
|
+
a CSV or JSON spec consumed by `hcms redirects <account> [--apply]`.
|
|
66
69
|
- `hubspot-cms-sync.config.mjs` controls environment policy and filesystem
|
|
67
70
|
layout.
|
|
71
|
+
- Managed redirects default to `isOnlyAfterNotFound: false`, so a release can
|
|
72
|
+
intentionally redirect an old live page path without a manual archive step.
|
|
68
73
|
|
|
69
74
|
## Open Config Questions
|
|
70
75
|
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# Content Layout
|
|
2
|
+
|
|
3
|
+
`hcms` syncs a normal repository tree into HubSpot CMS records. The exact paths
|
|
4
|
+
are configurable in `hubspot-cms-sync.config.mjs`, but the default layout is:
|
|
5
|
+
|
|
6
|
+
```text
|
|
7
|
+
my-site/
|
|
8
|
+
|-- hubspot-cms-sync.config.mjs
|
|
9
|
+
|-- site.manifest.json
|
|
10
|
+
|-- sync/
|
|
11
|
+
| |-- accounts.json
|
|
12
|
+
| `-- redirects.csv
|
|
13
|
+
|-- content/
|
|
14
|
+
| |-- pages/
|
|
15
|
+
| | |-- home.json
|
|
16
|
+
| | |-- home.widgets.json
|
|
17
|
+
| | `-- about.json
|
|
18
|
+
| |-- forms/
|
|
19
|
+
| | |-- contact.json
|
|
20
|
+
| | `-- properties.json
|
|
21
|
+
| |-- assets/
|
|
22
|
+
| | `-- hero.svg
|
|
23
|
+
| `-- blog/
|
|
24
|
+
| |-- container.json
|
|
25
|
+
| `-- posts/
|
|
26
|
+
| `-- blog__hello-world.json
|
|
27
|
+
|-- templates/
|
|
28
|
+
| |-- home.html
|
|
29
|
+
| |-- page.html
|
|
30
|
+
| |-- blog.html
|
|
31
|
+
| `-- blog-post.html
|
|
32
|
+
|-- modules/
|
|
33
|
+
| `-- hero.module/
|
|
34
|
+
| |-- fields.json
|
|
35
|
+
| `-- module.html
|
|
36
|
+
|-- css/
|
|
37
|
+
| `-- main.css
|
|
38
|
+
|-- js/
|
|
39
|
+
| `-- hs-forms.js
|
|
40
|
+
|-- theme.json
|
|
41
|
+
`-- fields.json
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
A complete minimal fixture lives at [`examples/minimal-site/`](../examples/minimal-site/).
|
|
45
|
+
The unit test suite loads that fixture, validates its manifest and redirects,
|
|
46
|
+
runs the local push ref preflight, reads its canonical forms, and scans it with
|
|
47
|
+
the corpus scanner.
|
|
48
|
+
|
|
49
|
+
## What Can Be Pushed
|
|
50
|
+
|
|
51
|
+
`hcms push <account>` writes the content surfaces below. It is idempotent by
|
|
52
|
+
portable identity: page slug, form name/key, blog slug/post slug, theme path, and
|
|
53
|
+
redirect route.
|
|
54
|
+
|
|
55
|
+
| Surface | Repo source | HubSpot target |
|
|
56
|
+
| --- | --- | --- |
|
|
57
|
+
| Theme code | `templates/`, `modules/`, `css/`, `js/`, `images/`, `theme.json`, `fields.json` | CMS Source Code API |
|
|
58
|
+
| Site pages | `content/pages/*.json` plus `site.manifest.json` | CMS Pages API |
|
|
59
|
+
| Page module values | `content/pages/*.widgets.json` | CMS Pages draft widget carrier |
|
|
60
|
+
| Forms | `content/forms/<key>.json` | Forms API |
|
|
61
|
+
| Contact properties | `content/forms/properties.json` | CRM Properties API |
|
|
62
|
+
| File assets | `content/assets/**` and `content/blog/assets/**` | File Manager API |
|
|
63
|
+
| Blog container and posts | `content/blog/container.json`, `content/blog/posts/*.json` | Blog APIs |
|
|
64
|
+
| URL redirects | `sync/redirects.csv` or configured `redirectsFile` | CMS URL Redirects API |
|
|
65
|
+
|
|
66
|
+
## Deployment Surface
|
|
67
|
+
|
|
68
|
+
`site.manifest.json` is the allowlist for pages, forms, blog, theme, and
|
|
69
|
+
UI-gated prerequisites. A page file under `content/pages/` is not enough by
|
|
70
|
+
itself; the page must also be listed in the manifest.
|
|
71
|
+
|
|
72
|
+
```json
|
|
73
|
+
{
|
|
74
|
+
"theme": { "name": "example-theme" },
|
|
75
|
+
"pages": [
|
|
76
|
+
{
|
|
77
|
+
"slug": "",
|
|
78
|
+
"templatePath": "example-theme/templates/home.html",
|
|
79
|
+
"desiredState": "publish"
|
|
80
|
+
},
|
|
81
|
+
{
|
|
82
|
+
"slug": "about",
|
|
83
|
+
"templatePath": "example-theme/templates/page.html",
|
|
84
|
+
"desiredState": "draft"
|
|
85
|
+
}
|
|
86
|
+
],
|
|
87
|
+
"blog": {
|
|
88
|
+
"slug": "blog",
|
|
89
|
+
"itemTemplate": "example-theme/templates/blog-post.html",
|
|
90
|
+
"listingTemplate": "example-theme/templates/blog.html"
|
|
91
|
+
},
|
|
92
|
+
"forms": ["contact"],
|
|
93
|
+
"uiGated": ["blogContainerCreate", "domainConnect"]
|
|
94
|
+
}
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
`desiredState` may be `publish`, `draft`, `archive`, or `ignore`.
|
|
98
|
+
|
|
99
|
+
## Portable References
|
|
100
|
+
|
|
101
|
+
Committed content should not contain raw portal IDs, form GUIDs, CTA GUIDs, or
|
|
102
|
+
HubSpot-hosted asset URLs. Use logical refs instead:
|
|
103
|
+
|
|
104
|
+
| Logical ref | Producer source |
|
|
105
|
+
| --- | --- |
|
|
106
|
+
| `@portal` | The target account's portal ID |
|
|
107
|
+
| `@form:contact` | `content/forms/contact.json` or `content/forms/guids.json` |
|
|
108
|
+
| `@asset:hero.svg` | `content/assets/hero.svg` or `content/blog/assets/hero.svg` |
|
|
109
|
+
|
|
110
|
+
`hcms push` runs a local preflight before network writes. If content references
|
|
111
|
+
`@form:contact`, the form producer source must exist. If content references
|
|
112
|
+
`@asset:hero.png`, committed bytes must exist. `@cta:*` and `@menu:*` currently
|
|
113
|
+
fail closed because there are no producer adapters for them yet.
|
|
114
|
+
|
|
115
|
+
## Redirects
|
|
116
|
+
|
|
117
|
+
Redirects are separate from the page manifest. Configure their path with
|
|
118
|
+
`redirectsFile`, then run:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
hcms redirects dev # dry-run
|
|
122
|
+
hcms redirects dev --apply # create/update HubSpot redirects
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
CSV redirects need at least:
|
|
126
|
+
|
|
127
|
+
```csv
|
|
128
|
+
routePrefix,destination,redirectStyle,isOnlyAfterNotFound
|
|
129
|
+
/old-about,/about,301,false
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
`isOnlyAfterNotFound=false` makes a redirect take precedence over an existing
|
|
133
|
+
live page at the same route. Use it deliberately during cutovers.
|
|
134
|
+
|
|
135
|
+
## Not Fully Automated
|
|
136
|
+
|
|
137
|
+
Some portal state is still UI-gated or depends on HubSpot account setup:
|
|
138
|
+
|
|
139
|
+
- connecting domains;
|
|
140
|
+
- choosing system pages such as the default 404;
|
|
141
|
+
- creating the initial blog container in some portals;
|
|
142
|
+
- native menus until a menu producer exists;
|
|
143
|
+
- theme settings values if the theme relies on HubSpot UI-managed settings.
|
|
144
|
+
|
|
145
|
+
Track those prerequisites in `uiGated` and check them with
|
|
146
|
+
`hcms preflight <account>` before writing content.
|
package/docs/GITHUB_ACTIONS.md
CHANGED
|
@@ -54,7 +54,9 @@ The examples prefer this sequence for write-capable operations:
|
|
|
54
54
|
3. `hcms preflight <target>`
|
|
55
55
|
4. `hcms push <target> --dry-run`
|
|
56
56
|
5. `hcms push <target> --publish`
|
|
57
|
-
6. `
|
|
57
|
+
6. `hcms redirects <target>`
|
|
58
|
+
7. `hcms redirects <target> --apply`
|
|
59
|
+
8. `the consuming repo verification commands`
|
|
58
60
|
|
|
59
61
|
For read-only CI, omit write steps and keep production credentials unavailable.
|
|
60
62
|
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
# HubSpot Sync — Gotchas & Operational Notes
|
|
2
|
+
|
|
3
|
+
_HubSpot CMS API gotchas hit while building the [seventh-sense](https://github.com/telepathdata/7thsense-website) reference site with this engine. Examples reference that site; the behaviors are general._
|
|
4
|
+
|
|
5
|
+
Hard-won notes from migrating the Seventh Sense site/blog onto a CMS sandbox.
|
|
6
|
+
**Read this before pointing `sync/*` at production.** Every item here cost real
|
|
7
|
+
debugging time; they are the difference between a sync script that works and one
|
|
8
|
+
that silently does the wrong thing.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 1. Credentials & scopes — the #1 time sink
|
|
13
|
+
|
|
14
|
+
There are three credential types and they are **not** interchangeable:
|
|
15
|
+
|
|
16
|
+
| Type | Looks like | Use it for | Gotcha |
|
|
17
|
+
|---|---|---|---|
|
|
18
|
+
| **Personal Access Key (PAK)** | `CiRu…` (base64) or surfaced as a long token | the `hs` CLI (theme upload) | **Cannot grant the `content` scope** — so PAKs *cannot* create CMS pages/blog posts. PAK scopes are also **immutable**: "editing" = generating a new key. |
|
|
19
|
+
| **Service Key** (BETA) | `pat-na2-…` | our `sync/*` scripts (direct Bearer) | **Editable** scopes (no regen), but scope changes have **propagation lag** (~30–60s) — a call can still 401 right after you save. Retry before assuming failure. |
|
|
20
|
+
| **Private App token** | `pat-na1-…` | alternative to service key | Also grants `content`; heavier to manage. |
|
|
21
|
+
|
|
22
|
+
- Use service keys as `Authorization: Bearer <key>` directly — **no exchange**. (A `CiRu…` value is a PAK refresh token; it 401s as a Bearer and must be exchanged at `POST /localdevauth/v1/auth/refresh`.)
|
|
23
|
+
- **Read vs write are separate scopes.** `crm.schemas.contacts.write` does **not** include `.read`. The forms adapter must work write-only, so it create-or-patches instead of relying on list-then-decide behavior. Design sync scripts to not *require* the read scope.
|
|
24
|
+
- Per-operation scopes we actually needed:
|
|
25
|
+
- Pages create/publish: **`content`**
|
|
26
|
+
- Blog posts create/publish: **`content`**
|
|
27
|
+
- File Manager (image re-host / recovery): **`files`**
|
|
28
|
+
- Create HubSpot forms: **`forms`** (`forms-write`)
|
|
29
|
+
- Create/patch custom contact properties: **`crm.schemas.contacts.write`**
|
|
30
|
+
- Domains / business-units reads: a domains scope we never got on the sandbox key (blog-create needs it; see §4).
|
|
31
|
+
|
|
32
|
+
## 2. Account types — not all can host content
|
|
33
|
+
|
|
34
|
+
- **`DEVELOPER_TEST`** accounts (created from an app-developer account) include Design Manager (theme dev) but **cannot host CMS pages** — the `content`/`cms.pages.site_pages.write` scope is not grantable. Theme upload works, page creation never will.
|
|
35
|
+
- A **free CMS Developer Sandbox** (`accountType: STANDARD`, signup at `app.hubspot.com/signup-hubspot/cms-developers`) **can** host pages/blog. This is the right target for a staging mirror.
|
|
36
|
+
- Check `accountType` (via the PAK token exchange response `accountType`, or `/account-info/v3/details`) before assuming a portal can take content.
|
|
37
|
+
|
|
38
|
+
## 3. Publishing — the two big quirks
|
|
39
|
+
|
|
40
|
+
- **`POST …/draft/push-live` silently no-ops on first publish.** It returns `204` but the content stays `DRAFT`. **Use the schedule endpoint with a near-future date instead:**
|
|
41
|
+
`POST /cms/v3/pages/site-pages/schedule` (and `/cms/v3/blogs/posts/schedule`) with `{"id", "publishDate": <now + 90s>}`. It fires ~75–90s later and the page goes live.
|
|
42
|
+
- **`publishDate` must be in the future** — "now" is rejected with `publishDate must be in the future`.
|
|
43
|
+
- **Pages/posts require a non-empty title** to publish (`CONTENT_TITLE_MISSING`). Set `htmlTitle`/`name` before scheduling.
|
|
44
|
+
- **Template changes do not re-render already-published content.** After editing a template, you must **re-publish every affected page/post** (schedule them again) to pick up the new template. This is why a dynamic blog-post template still showed old hardcoded content until all 68 posts were re-scheduled.
|
|
45
|
+
- **⚠️ Re-scheduling a post resets its `publishDate` to the scheduled time** — bulk re-scheduling 68 posts clobbered their original 2017–2026 dates to "today," wrecking chronological order. Fix: `PATCH /cms/v3/blogs/posts/{id}` with the original `publishDate` (keeps it `PUBLISHED`, restores the date). The blog adapter always sends `publishDate` from the snapshot so a re-push restores dates.
|
|
46
|
+
- **Invalidate a cached *blog listing*** by re-PUTting the blog (`PUT /content/api/v2/blogs/{id}`) — that flips the edge-cache tag from the old listing page (`CT-…`) to the blog (`B-…`) and serves the current `listing_template_path`.
|
|
47
|
+
- **Edge cache is aggressive:** blog listing/pages serve with `s-maxage=36000` (10h). Cache-busting query params do **not** bypass it. Publishing invalidates by `edge-cache-tag`; otherwise expect lag.
|
|
48
|
+
|
|
49
|
+
## 4. Blog specifics
|
|
50
|
+
|
|
51
|
+
- **Creating a blog is UI-gated.** `POST /content/api/v2/blogs` fails with `BLOG_HS_SITE_DOMAIN_WRITE_SCOPE_MISSING` ("publish blog on domain ''") even with `content` + a domain in the body — binding a blog to the `hs-sites` system domain needs UI-level permission a service key can't get. **Create the blog once in Settings → Website → Blog**, then automate everything else.
|
|
52
|
+
- **Updating a blog DOES work via API:** `PUT /content/api/v2/blogs/{id}` — use it to fix `slug`, `item_template_path`, `listing_template_path` after the UI creates it with wrong defaults (it defaults to `@hubspot/elevate` templates and a `seventh-sense-blog` slug).
|
|
53
|
+
- **`listing_page_id` overrides `listing_template_path`.** A blog auto-creates a listing *page* that renders `/blog` with its own (default) template, ignoring `listing_template_path`. That listing page is **not** reachable via the site-pages or legacy-pages API (404). **Fix that worked:** `PUT /content/api/v2/blogs/{id}` with `{"listing_page_id": 0}` to clear the override, then re-PUT the blog to bust the edge cache → `/blog` then renders `listing_template_path`.
|
|
54
|
+
- **Post slugs are full paths** (e.g. `blog/spf-dkim-…`), not relative to the blog root. Push them as-is.
|
|
55
|
+
- The legacy CMS blogs API can have a **stale "Old" blog** + many `DRAFT`/`SCHEDULED`/`-temporary-slug-` junk posts. Filter to `state == PUBLISHED`, non-`Old` blog, real slug, body length > 500 before migrating (see `blog-sync` pruning).
|
|
56
|
+
|
|
57
|
+
## 5. Images & File Manager
|
|
58
|
+
|
|
59
|
+
- Blog bodies reference images on **legacy hosts** (`cdn2.hubspot.net`, `f.hubspotusercontent00.net`) whose public URLs are often **404 / dead** — these images were deleted years ago and 404 on the live prod blog too. Don't expect 100% recovery (we got 50/172).
|
|
60
|
+
- Recover what's live via the current host; for dead URLs, fall back to **File Manager search** (`GET /files/v3/files/search?name=<stem>`) — needs the **`files`** scope.
|
|
61
|
+
- On push, **re-host** each image to the target File Manager (`POST /files/v3/files`, multipart, `options.access=PUBLIC_INDEXABLE`) and **rewrite** body/featured URLs to the new hosted URL. Never leave prod hotlinks.
|
|
62
|
+
|
|
63
|
+
## 6. Forms
|
|
64
|
+
|
|
65
|
+
- All redesign forms shipped as **static mockups** (`onsubmit="preventDefault(); alert(...)"`). Fix = keep the styled markup, POST to the **Forms Submission API**: `POST https://api.hsforms.com/submissions/v3/integration/submit/{portalId}/{formGuid}` (shared `js/hs-forms.js`). Don't use `{% form %}` embeds — they replace the custom design.
|
|
66
|
+
- **The submission API enforces the form's `required` fields.** A lighter entry point (e.g. the audit-cta email-only form) submitting to a fuller form gets `REQUIRED_FIELD` errors. Pattern: make forms **email-only-required at the API layer**, enforce richer UX with per-page HTML5 `required`. One form can then back multiple entry points.
|
|
67
|
+
- Custom fields must exist as **contact properties** first, and be on the form. The forms adapter upserts both.
|
|
68
|
+
|
|
69
|
+
## 7. Module fields & repeater defaults (refactor blocker)
|
|
70
|
+
|
|
71
|
+
- A field's **`name` cannot be a reserved word**: `name`, `label`, `id`, `type`, `body`, `children`, `default`, `required`, `locked`, etc. Upload fails with `field name cannot be 'X'`. Rename (e.g. `label`→`caption`, `name`→`title`, `body`→`body_html`) and update the `{{ item.X }}` HubL refs.
|
|
72
|
+
- **Theme deploys are per-file (Source Code API PUT), not transactional across files** — a mid-deploy failure (e.g. a bad field name) leaves some module files updated and others not, so a republish renders a *mixed/broken* page. Always re-run `sync:push` to completion (and re-verify fidelity) after fixing.
|
|
73
|
+
- **⚠️ HubSpot does NOT render repeater (group + `occurrence`) field DEFAULTS for `{% module %}` instances in coded templates** — neither for existing pages nor newly-created ones. Simple field defaults (text/richtext) render; **repeater loops come up empty**, so a page that relied on the repeater default loses all its repeated content (stats, cards, FAQ, pricing rows). This broke home-page fidelity in the Phase-2 module refactor.
|
|
74
|
+
- **Fixes:** (a) write the content as **field VALUES on the page instance** via the Pages API `widgets` map (the correct content/theme model — content lives on the page, template is generic); or (b) pass the content as **template-level tag params** `{% module "x" path=… group=[{…}] %}` (HubSpot honors these at render, unlike `fields.json` repeater defaults); or (c) a conditional `{% if module.items %}…{% else %}<orig>{% endif %}` fallback. Do **not** rely on repeater `default` arrays alone.
|
|
75
|
+
- **⚠️ Tag-param serialization (b) only works for SIMPLE content.** It worked for `big-stats` (`stats=[{"num":"+44%","caption":"…"}]`) but **broke for rich content** — repeater items containing inline SVG (`width="24"`) or quoted phrases (`"who's getting what"`) fail HubL param parsing (`Error parsing … encountered '24'`), because HubL string quoting in tag params doesn't cleanly accept JSON-escaped `\"`/apostrophes. **For rich/HTML repeater content, use the `widgets` API (page-instance JSON values), not tag params.** Pattern A is fine for plain-text repeaters; the `widgets` API is the universal path.
|
|
76
|
+
|
|
77
|
+
## 8. Transport / tooling environment
|
|
78
|
+
|
|
79
|
+
- **No CLI.** The entire sync runs on the HubSpot REST API via `fetch` with a service-key Bearer token (`~/.hubspot/<portalId>.key`). The theme uses the **CMS Source Code v3 API** (`GET/PUT /cms/v3/source-code/published/{metadata,content}/<theme>/<path>`); pages/blog/forms/content use their respective v3 APIs.
|
|
80
|
+
- We previously used the `hs` CLI for theme pull/push and **dropped it**: the whole-tree `hs cms upload` silently no-op'd (it mis-named the theme after the build dir), the per-dir form was unreliable, and the CLI dragged in `rollup` (which periodically broke on the npm optional-dependency bug). The per-file Source Code API PUT is deterministic and idempotent (create-or-replace by path).
|
|
81
|
+
- The CLI had **no page/blog content commands** anyway — content was always API-only; `hs cms` only managed theme assets. So every adapter is now uniform: one transport, one auth model.
|
|
82
|
+
|
|
83
|
+
## 9. Operational checklist for the unified sync
|
|
84
|
+
|
|
85
|
+
Production is currently read-only. The supported deployment target is the dev
|
|
86
|
+
sandbox `246389711`; the production portal `529456` is hard-blocked in `hcms push`.
|
|
87
|
+
|
|
88
|
+
1. Create service keys with the needed scopes:
|
|
89
|
+
- Prod pull key at `~/.hubspot/529456.key` (`content` plus read scopes needed for the resources you pull).
|
|
90
|
+
- Dev push key at `~/.hubspot/246389711.key` (`content`, `files`, `forms`, `crm.schemas.contacts.write`).
|
|
91
|
+
2. Confirm the target portal can host CMS content (§2).
|
|
92
|
+
3. Create the UI-gated resources once (§4): blog container, homepage designation,
|
|
93
|
+
domain setup, theme setting values, and native menus.
|
|
94
|
+
4. Pull production into git:
|
|
95
|
+
```sh
|
|
96
|
+
hcms pull -- prod
|
|
97
|
+
```
|
|
98
|
+
5. Review the diff and prove portability:
|
|
99
|
+
```sh
|
|
100
|
+
git diff
|
|
101
|
+
hcms corpus
|
|
102
|
+
```
|
|
103
|
+
6. Preflight the dev sandbox:
|
|
104
|
+
```sh
|
|
105
|
+
hcms preflight -- dev
|
|
106
|
+
```
|
|
107
|
+
7. Push and publish to dev:
|
|
108
|
+
```sh
|
|
109
|
+
hcms push -- dev --publish
|
|
110
|
+
```
|
|
111
|
+
8. Re-publish live pages/posts after template changes (§3):
|
|
112
|
+
```sh
|
|
113
|
+
hcms republish --portal 246389711 --all
|
|
114
|
+
hcms republish --portal 246389711 --all --blog
|
|
115
|
+
```
|
|
116
|
+
9. Verify the rendered site:
|
|
117
|
+
```sh
|
|
118
|
+
npm test
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
For the complete operator flow and GitHub Actions deployment path, use
|
|
122
|
+
[`sync-runbook.md`](./sync-runbook.md). For the architectural rationale, use
|
|
123
|
+
[`deployment-architecture.md`](./deployment-architecture.md).
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="630" viewBox="0 0 1200 630" role="img" aria-labelledby="title desc">
|
|
2
|
+
<title id="title">Example hero image</title>
|
|
3
|
+
<desc id="desc">A simple placeholder image for the minimal hcms site.</desc>
|
|
4
|
+
<rect width="1200" height="630" fill="#f5f7fb"/>
|
|
5
|
+
<rect x="96" y="96" width="1008" height="438" rx="24" fill="#ffffff" stroke="#d8dee9"/>
|
|
6
|
+
<circle cx="210" cy="220" r="58" fill="#1f7a8c"/>
|
|
7
|
+
<path d="M320 190h520v34H320zM320 252h420v28H320zM320 310h600v22H320zM320 352h480v22H320z" fill="#233142"/>
|
|
8
|
+
<path d="M160 430h220v58H160z" fill="#f25f5c"/>
|
|
9
|
+
</svg>
|