wordpress-to-kirby 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" width="401.42" height="207.25" viewBox="0 0 401.42 207.25"><path d="M98.43,5.2C44.07,5.2,0,49.26,0,103.63s44.07,98.43,98.43,98.43,98.43-44.07,98.43-98.43S152.79,5.2,98.43,5.2ZM98.43,187.06c-46,0-83.43-37.43-83.43-83.43S52.43,20.2,98.43,20.2s83.43,37.43,83.43,83.43-37.43,83.43-83.43,83.43ZM311.67,0l-89.74,51.81v103.63l89.74,51.81,89.74-51.81V51.81L311.67,0ZM386.42,146.78l-74.74,43.15-74.74-43.15V60.47l74.74-43.15,74.74,43.15v86.31ZM339.73,117.54l15.64-10.65v35.93l-15.64-9.09h-57.78l-15.3,9.65v-35.49l15.3,9.65h57.78ZM310.64,111.91l-44-24.64v-18.96c2.15,1.7,4.82,3.94,8.04,6.71,3.22,2.77,6.65,5.82,10.31,9.15,3.66,3.33,7.28,6.75,10.87,10.26,2.72,2.67,5.26,5.26,7.62,7.77,8.64-5.37,16.97-11.24,24.98-17.64,8.47-6.76,16.62-13.66,24.45-20.68.81,1.63,1.65,3.31,2.5,5.05.85,1.74,1.27,3.75,1.27,6.04,0,3.55-1.02,6.99-3.05,10.31-2.03,3.33-4.97,6.29-8.82,8.87l-31.16,20.85-3.03-3.08ZM158.26,68.75l-8.87,15.42-28.39,73.3h-3.77l-19.53-49.53-17.07,49.53h-3.77l-28.83-73.42-9.43-15.3h27.95l.22,15.3,14.88,39.99,15.15-18.39-8.52-21.6-7.32-15.3h20.52l21.53,58.61,13.18-43.3-2.33-15.3h24.4Z"/></svg>
package/README.md CHANGED
@@ -1,5 +1,9 @@
1
1
  # wordpress-to-kirby
2
2
 
3
+ <img src="./.github/wordpress-to-kirby.svg" height='60'>
4
+
5
+ <a href="https://www.npmjs.org/package/wordpress-to-kirby"><img src="https://img.shields.io/npm/v/wordpress-to-kirby.svg?style=flat-square&labelColor=EEEEEE&color=black" alt="npm version"></a>
6
+
3
7
  A Node.js-based command-line utility to convert a WordPress XML export to a directory of Kirby CMS files.
4
8
 
5
9
  ## Getting started
@@ -22,37 +26,53 @@ Use `wordpress-to-kirby --help` for information on usage and available commands
22
26
 
23
27
  ### `wordpress-to-kirby generate`
24
28
 
25
- Generate a Kirby CMS file structure from the provided XML file. Usage:
29
+ Generate a Kirby CMS file structure from the input file. Usage:
26
30
 
27
31
  ```bash
28
- wordpress-to-kirby generate [options] <wordpress-xml-file>
32
+ wordpress-to-kirby generate [options] <input-file>
29
33
  ```
30
34
 
35
+ The input must correspond to a WordPress export XML file. When `-` is provided, the content is read from STDIN.
36
+
31
37
  ### `wordpress-to-kirby download`
32
38
 
33
- Download the attachments found in the provided XML file. Usage:
39
+ Download the attachments found in the input file. Usage:
34
40
 
35
41
  ```bash
36
- wordpress-to-kirby download [options] <wordpress-xml-file>
42
+ wordpress-to-kirby download [options] <input-file>
37
43
  ```
38
44
 
39
- ## Options
45
+ The input must correspond to either a WordPress export XML file or a metadata JSON file (generated with the `generate` command). When `-` is provided, the content is read from STDIN.
40
46
 
41
- ### General options
47
+ ## Options
42
48
 
43
- #### `-h, --help`
49
+ ### `-h, --help`
44
50
 
45
51
  Prints out this help information.
46
52
 
47
- #### `-v, --version`
53
+ ```bash
54
+ wordpress-to-kirby --help
55
+ ```
56
+
57
+ ### `-v, --version`
48
58
 
49
59
  Shows the program version.
50
60
 
51
- #### `-o, --outdir`
61
+ ```bash
62
+ wordpress-to-kirby --version
63
+ ```
64
+
65
+ ### `-o, --outdir=<path>`
52
66
 
53
67
  Root directory to which to write the output, relative to the current working directory. Default: `wordpress-to-kirby`.
54
68
 
55
- #### `--contentdir`
69
+ ```bash
70
+ wordpress-to-kirby generate my-blog.xml -o my-blog
71
+ ```
72
+
73
+ When using the `--only-metadata` option is provided, the `--outdir` value can be `-`, which will cause the `metadata.json` file to be output to STDOUT.
74
+
75
+ ### `--contentdir=<path>`
56
76
 
57
77
  For the `download` command, point to the location of a `wp-content` directory from which to download attachments.
58
78
 
@@ -61,9 +81,15 @@ The path can be:
61
81
  * an absolute or relative local path;
62
82
  * a fully-qualified URL, including the `wp-content` segment, e.g. `https://mywebsite.local/wp-content/`
63
83
 
64
- #### `--parse=<rules>`
84
+ ```bash
85
+ wordpress-to-kirby download my-blog.xml -o my-blog --contentdir=path/to/downloaded-wp-content
86
+ ```
87
+
88
+ ### `--parse=<rules>`
65
89
 
66
90
  Instructions to parse some fields, specified as `<key>=<parser>,<key>=<parser>,…`.
91
+
92
+ The `--parse` option can be used multiple times.
67
93
 
68
94
  Supported parsers:
69
95
 
@@ -72,10 +98,16 @@ Supported parsers:
72
98
  * `json`: parse as JSON;
73
99
  * `html`: parse as HTML.
74
100
 
75
- #### `--serialize=<rules>`
101
+ ```bash
102
+ wordpress-to-kirby generate my-blog.xml --parse=content=html
103
+ ```
104
+
105
+ ### `--serialize=<rules>`
76
106
 
77
107
  Instructions to serialize some fields, specified as `<key>=<serializer>,<key>=<serializer>,…`.
78
108
 
109
+ The `--serialize` option can be used multiple times.
110
+
79
111
  Supported serializers:
80
112
 
81
113
  * `php`: serialize as PHP object;
@@ -83,23 +115,53 @@ Supported serializers:
83
115
  * `json`: serialize as JSON;
84
116
  * `html`: serialize as HTML.
85
117
 
118
+ ```bash
119
+ wordpress-to-kirby generate my-blog.xml --serialize=content=html
120
+ ```
121
+
86
122
  ### `--exclude-fields=<fields>`
87
123
 
88
124
  Exclude certain fields from the output, specified as a comma-separated list of fields.
89
125
 
126
+ The `--exclude-fields` option can be used multiple times.
127
+
128
+ ```bash
129
+ wordpress-to-kirby generate my-blog.xml --exclude-fields=guid_isPermalink,comment_status,ping_status
130
+ ```
131
+
90
132
  ### `--include-fields=<fields>`
91
133
 
92
134
  Only include certain fields from the output, specified as a comma-separated list of fields. When `--include-fields` is specified, the `--exclude-fields` option is ignored.
93
135
 
136
+ The `--include-fields` option can be used multiple times.
137
+
138
+ ```bash
139
+ wordpress-to-kirby generate my-blog.xml --include-fields=title,content,uuid
140
+ ```
141
+
94
142
  ### `--exclude-types=<types>`
95
143
 
96
144
  Exclude certain post types from the output, specified as a comma-separated list of post types.
97
145
 
146
+ The `--exclude-types` option can be used multiple times.
147
+
148
+ ```bash
149
+ wordpress-to-kirby generate my-blog.xml --exclude-types=nav_menu_item
150
+ ```
151
+
98
152
  ### `--include-types=<types>`
99
153
 
100
- Only include certain post types from the output, specified as a comma-separated list of post types.
154
+ Only include certain post types from the output, specified as a comma-separated list of post types. When `--include-types` is specified, the `--exclude-types` option is ignored.
155
+
156
+ The `--include-types` option can be used multiple times.
157
+
158
+ ```bash
159
+ wordpress-to-kirby generate my-blog.xml --include-types=post,page,film,venue
160
+ ```
161
+
162
+ ### `--only-metadata`
101
163
 
102
- When `--include-types` is specified, the `--exclude-types` option is ignored.
164
+ For the `generate` command, only generate the `metadata.json` file.
103
165
 
104
166
  ## Keys extracted from the XML
105
167
 
package/index.js CHANGED
@@ -1,14 +1,16 @@
1
1
  #!/usr/bin/env node
2
2
  import { readFile, writeFile, copyFile, mkdir } from 'node:fs/promises';
3
3
  import { join, dirname, extname, relative, sep } from 'node:path';
4
+ import { parseArgs } from 'node:util';
4
5
 
5
- import opsh from 'opsh';
6
6
  import { JSDOM } from 'jsdom';
7
7
  import { load, dump } from 'js-yaml';
8
8
  import { serialize, unserialize } from 'php-serialize';
9
9
 
10
10
  import { serialize as serialize_item } from './src/kirby-txt.js';
11
11
  import { uuid } from './src/uuid.js';
12
+ import inside from './src/inside.js';
13
+ import slurp from './src/slurp.js';
12
14
 
13
15
  async function getPackage() {
14
16
  return JSON.parse(
@@ -16,31 +18,80 @@ async function getPackage() {
16
18
  );
17
19
  }
18
20
 
19
- const { operands, options } = opsh(
20
- process.argv.slice(2),
21
- ['help', 'h', 'version', 'v']
22
- );
21
+ const { positionals: operands, values: options } = parseArgs({
22
+ options: {
23
+ help: {
24
+ type: 'boolean',
25
+ short: 'h'
26
+ },
27
+ version: {
28
+ type: 'boolean',
29
+ short: 'v'
30
+ },
31
+ parse: {
32
+ type: 'string',
33
+ multiple: true
34
+ },
35
+ serialize: {
36
+ type: 'string',
37
+ multiple: true
38
+ },
39
+ outdir: {
40
+ type: 'string',
41
+ short: 'o',
42
+ default: 'wordpress-to-kirby'
43
+ },
44
+ contentdir: {
45
+ type: 'string'
46
+ },
47
+ 'include-fields': {
48
+ type: 'string',
49
+ multiple: true
50
+ },
51
+ 'exclude-fields': {
52
+ type: 'string',
53
+ multiple: true
54
+ },
55
+ 'include-types': {
56
+ type: 'string',
57
+ multiple: true
58
+ },
59
+ 'exclude-types': {
60
+ type: 'string',
61
+ multiple: true
62
+ },
63
+ 'only-metadata': {
64
+ type: 'boolean'
65
+ }
66
+ },
67
+ allowPositionals: true
68
+ });
23
69
 
24
- if (options.h || options.help) {
25
- await printHelp();
70
+ if (options.help) {
71
+ printHelp();
26
72
  process.exit();
27
73
  }
28
74
 
29
- if (options.v || options.version) {
75
+ if (options.version) {
30
76
  const pkg = await getPackage();
31
- console.log(pkg.version);
77
+ process.stdout.write(pkg.version + '\n');
32
78
  process.exit();
33
79
  }
34
80
 
35
81
  const [command, ...files ] = operands;
36
82
 
37
83
  if (command !== 'generate' && command !== 'download') {
38
- console.error(`Supported commands: generate, download; ${command} provided. Exiting.`);
84
+ process.stderr.write(`Supported commands: generate, download; ${command} provided. Exiting.\n`);
39
85
  process.exit(1);
40
86
  }
41
87
 
42
88
  if (files.length !== 1) {
43
- console.error(`Exactly one file is supported, ${files.length} provided. Exiting.`);
89
+ process.stderr.write(`Exactly one file is supported, ${files.length} provided. Exiting.\n`);
90
+ process.exit(1);
91
+ }
92
+
93
+ if (options.outdir === '-' && !options['only-metadata']) {
94
+ process.stderr.write('Outputting to STDOUT is only supported with the --only-metadata option. Exiting.\n');
44
95
  process.exit(1);
45
96
  }
46
97
 
@@ -51,8 +102,11 @@ const serializers = {
51
102
  'postmeta_wp_attachment_metadata': 'json'
52
103
  };
53
104
 
54
- function commasep(val) {
55
- return (val || '').split(/\s*,\s*/).filter(Boolean);
105
+ function commasep(arr = []) {
106
+ return arr
107
+ .map(val => (val || '').split(/\s*,\s*/))
108
+ .flat()
109
+ .filter(Boolean);
56
110
  }
57
111
 
58
112
  commasep(options.parse).forEach(pair => {
@@ -91,12 +145,22 @@ function process_item(item) {
91
145
  return Object.fromEntries(
92
146
  entries.map(entry => [
93
147
  entry[0],
94
- serialize_value(entry[0], entry[1])
148
+ serialize_value(entry[0], entry[1], item.post_id)
95
149
  ])
96
150
  );
97
151
  }
98
152
 
99
- function parse_value(key, value) {
153
+ function abbr(value, len = 64) {
154
+ if (typeof value !== 'string') {
155
+ return value;
156
+ }
157
+ if (value.length <= len) {
158
+ return value;
159
+ }
160
+ return value.slice(0, len) + '…';
161
+ }
162
+
163
+ function parse_value(key, value, id) {
100
164
  if (!value) {
101
165
  return value;
102
166
  }
@@ -105,21 +169,21 @@ function parse_value(key, value) {
105
169
  try {
106
170
  return unserialize(value);
107
171
  } catch(err) {
108
- console.error(`Warn: Can’t serialize PHP value, skipping: ${value}`);
172
+ process.stderr.write(`Warn: Can’t parse PHP value in post ${id}, skipping: ${abbr(value)}\n`);
109
173
  return value;
110
174
  }
111
175
  case 'yaml':
112
176
  try {
113
177
  return load(value);
114
178
  } catch (err) {
115
- console.error(`Warn: Can’t serialize YAML value, skipping: ${value}`);
179
+ process.stderr.write(`Warn: Can’t parse YAML value in post ${id}, skipping: ${abbr(value)}\n`);
116
180
  return value;
117
181
  }
118
182
  case 'json':
119
183
  try {
120
184
  return JSON.parse(value);
121
185
  } catch(err) {
122
- console.error(`Warn: Can’t serialize JSON value, skipping: ${value}`);
186
+ process.stderr.write(`Warn: Can’t parse JSON value in post ${id}, skipping: ${abbr(value)}\n`);
123
187
  return value;
124
188
  }
125
189
  case 'html':
@@ -128,7 +192,7 @@ function parse_value(key, value) {
128
192
  `<!doctype html><html><head></head><body>${value}</body></html>`
129
193
  ).window.document;
130
194
  } catch(err) {
131
- console.error(`Warn: Can’t serialize JSON value, skipping: ${value}`);
195
+ process.stderr.write(`Warn: Can’t parse HTML value in post ${id}, skipping: ${abbr(value)}\n`);
132
196
  return value;
133
197
  }
134
198
  default:
@@ -136,7 +200,7 @@ function parse_value(key, value) {
136
200
  }
137
201
  }
138
202
 
139
- function serialize_value(key, value) {
203
+ function serialize_value(key, value, id) {
140
204
  if (!value) {
141
205
  return value;
142
206
  }
@@ -145,28 +209,28 @@ function serialize_value(key, value) {
145
209
  try {
146
210
  return serialize(value);
147
211
  } catch(err) {
148
- console.error(`Warn: Can’t stringify PHP value, skipping: ${value}`);
212
+ process.stderr.write(`Warn: Can’t serialize PHP value in post ${id}, skipping: ${abbr(value)}\n`);
149
213
  return value;
150
214
  }
151
215
  case 'yaml':
152
216
  try {
153
217
  return dump(value);
154
218
  } catch (err) {
155
- console.error(`Warn: Can’t stringify YAML value, skipping: ${value}`);
219
+ process.stderr.write(`Warn: Can’t serialize YAML value in post ${id}, skipping: ${abbr(value)}\n`);
156
220
  return value;
157
221
  }
158
222
  case 'json':
159
223
  try {
160
224
  return JSON.stringify(value);
161
225
  } catch(err) {
162
- console.error(`Warn: Can’t stringify JSON value, skipping: ${value}`);
226
+ process.stderr.write(`Warn: Can’t serialize JSON value in post ${id}, skipping: ${abbr(value)}\n`);
163
227
  return value;
164
228
  }
165
229
  case 'html':
166
230
  try {
167
231
  return value.body.innerHTML;
168
232
  } catch(err) {
169
- console.error(`Warn: Can’t stringify HTML value, skipping: ${value}`);
233
+ process.stderr.write(`Warn: Can’t serialize HTML value in post ${id}, skipping: ${abbr(value)}\n`);
170
234
  return value;
171
235
  }
172
236
  default:
@@ -174,24 +238,6 @@ function serialize_value(key, value) {
174
238
  }
175
239
  }
176
240
 
177
- const document = new JSDOM(
178
- await readFile(files[0], 'utf8'),
179
- {
180
- contentType: 'application/xml'
181
- }
182
- ).window.document;
183
-
184
- const items = Array.from(document.querySelectorAll('item'))
185
- .map(extract)
186
- .filter(item => {
187
- if (types_to_include.size) {
188
- return types_to_include.has(item.post_type);
189
- } else if (types_to_exclude.size) {
190
- return !types_to_exclude.has(item.post_type);
191
- }
192
- return true;
193
- });
194
-
195
241
  function extract(item) {
196
242
  const select = (sel, ns, all = false, el) => {
197
243
  if (ns) {
@@ -202,37 +248,39 @@ function extract(item) {
202
248
  return (el ?? item).querySelector(sel);
203
249
  }
204
250
 
205
- const post_type = parse_value('post_type', select('post_type', 'wp').textContent);
251
+ const post_id = parse_value('post_id', select('post_id', 'wp').textContent);
252
+
253
+ const post_type = parse_value('post_type', select('post_type', 'wp').textContent, post_id);
206
254
 
207
255
  return {
208
- title: parse_value('title', select('title').textContent),
209
- link: parse_value('link', select('link').textContent),
210
- pubDate: parse_value('pubDate', select('pubDate').textContent),
211
- creator: parse_value('creator', select('creator', 'dc').textContent),
212
- guid: parse_value('guid', select('guid').textContent),
213
- guid_isPermaLink: parse_value('guid_isPermaLink', select('guid').getAttribute('isPermaLink')),
214
- description: parse_value('description', select('description').textContent),
215
- content: parse_value('content', select('encoded', 'content').textContent),
216
- excerpt: parse_value('excerpt', select('encoded', 'excerpt').textContent),
217
- post_id: parse_value('post_id', select('post_id', 'wp').textContent),
218
- post_date: parse_value('post_date', select('post_date', 'wp').textContent),
219
- post_date_gmt: parse_value('post_date_gmt', select('post_date_gmt', 'wp').textContent),
220
- post_modified: parse_value('post_modified', select('post_modified', 'wp').textContent),
221
- post_modified_gmt: parse_value('post_modified_gmt', select('post_modified_gmt', 'wp').textContent),
222
- comment_status: parse_value('comment_status', select('comment_status', 'wp').textContent),
223
- ping_status: parse_value('ping_status', select('ping_status', 'wp').textContent),
224
- post_name: parse_value('post_name', select('post_name', 'wp').textContent),
225
- status: parse_value('status', select('status', 'wp').textContent),
226
- post_parent: parse_value('post_parent', select('post_parent', 'wp').textContent),
227
- menu_order: parse_value('menu_order', select('menu_order', 'wp').textContent),
256
+ post_id,
228
257
  post_type,
229
- post_password: parse_value('post_password', select('post_password', 'wp').textContent),
230
- attachment_url: parse_value('attachment_url', select('attachment_url', 'wp')?.textContent),
231
- is_sticky: parse_value('is_sticky', select('is_sticky', 'wp').textContent),
258
+ title: parse_value('title', select('title').textContent, post_id),
259
+ link: parse_value('link', select('link').textContent, post_id),
260
+ pubDate: parse_value('pubDate', select('pubDate').textContent, post_id),
261
+ creator: parse_value('creator', select('creator', 'dc').textContent, post_id),
262
+ guid: parse_value('guid', select('guid').textContent, post_id),
263
+ guid_isPermaLink: parse_value('guid_isPermaLink', select('guid').getAttribute('isPermaLink'), post_id),
264
+ description: parse_value('description', select('description').textContent, post_id),
265
+ content: parse_value('content', select('encoded', 'content').textContent, post_id),
266
+ excerpt: parse_value('excerpt', select('encoded', 'excerpt').textContent, post_id),
267
+ post_date: parse_value('post_date', select('post_date', 'wp').textContent, post_id),
268
+ post_date_gmt: parse_value('post_date_gmt', select('post_date_gmt', 'wp').textContent, post_id),
269
+ post_modified: parse_value('post_modified', select('post_modified', 'wp').textContent, post_id),
270
+ post_modified_gmt: parse_value('post_modified_gmt', select('post_modified_gmt', 'wp').textContent, post_id),
271
+ comment_status: parse_value('comment_status', select('comment_status', 'wp').textContent, post_id),
272
+ ping_status: parse_value('ping_status', select('ping_status', 'wp').textContent, post_id),
273
+ post_name: parse_value('post_name', select('post_name', 'wp').textContent, post_id),
274
+ status: parse_value('status', select('status', 'wp').textContent, post_id),
275
+ post_parent: parse_value('post_parent', select('post_parent', 'wp').textContent, post_id),
276
+ menu_order: parse_value('menu_order', select('menu_order', 'wp').textContent, post_id),
277
+ post_password: parse_value('post_password', select('post_password', 'wp').textContent, post_id),
278
+ attachment_url: parse_value('attachment_url', select('attachment_url', 'wp')?.textContent, post_id),
279
+ is_sticky: parse_value('is_sticky', select('is_sticky', 'wp').textContent, post_id),
232
280
  ...select('postmeta', 'wp', true).reduce((acc, it) => {
233
281
  const key = select('meta_key', 'wp', false, it).textContent;
234
282
  const value = select('meta_value', 'wp', false, it).textContent;
235
- acc['postmeta' + key] = parse_value('postmeta' + key, value);
283
+ acc['postmeta' + key] = parse_value('postmeta' + key, value, post_id);
236
284
  return acc;
237
285
  }, {}),
238
286
  uuid: uuid(),
@@ -240,7 +288,7 @@ function extract(item) {
240
288
  }
241
289
  }
242
290
 
243
- const outdir = options.o || options.outdir || 'wordpress-to-kirby';
291
+ const outdir = options.outdir;
244
292
 
245
293
  function outpath(item) {
246
294
  let parts = (new URL(item.link)).pathname.split('/').filter(Boolean);
@@ -254,119 +302,179 @@ function outpath(item) {
254
302
  return join(outdir, 'content', ...parts, item.post_type + '.txt');
255
303
  }
256
304
 
305
+ function readFromExportXML(file) {
306
+ const document = new JSDOM(file, {
307
+ contentType: 'application/xml'
308
+ }).window.document;
309
+
310
+ return Array.from(document.querySelectorAll('item'))
311
+ .map(extract)
312
+ .filter(item => {
313
+ if (types_to_include.size) {
314
+ return types_to_include.has(item.post_type);
315
+ } else if (types_to_exclude.size) {
316
+ return !types_to_exclude.has(item.post_type);
317
+ }
318
+ return true;
319
+ });
320
+ }
321
+
257
322
  if (command === 'generate') {
258
- for await (let item of items) {
259
- const destination = outpath(item);
260
- await mkdir(dirname(destination), { recursive: true });
261
- await writeFile(destination, serialize_item(process_item(item)));
262
- };
323
+
324
+ const items = readFromExportXML(
325
+ files[0] === '-' ?
326
+ await slurp(process.stdin) :
327
+ await readFile(files[0], 'utf8')
328
+ );
329
+ const processed_items = items.map(process_item);
263
330
 
264
331
  const metadata = {
265
332
  output: [],
266
333
  types: {}
267
334
  };
268
335
 
269
- items.forEach(item => {
336
+ processed_items.forEach((item, idx) => {
337
+ /*
338
+ Some of these fields may have been omitted from the object,
339
+ but we still need them for metadata purposes.
340
+ */
341
+ const full_item = items[idx];
270
342
  metadata.output.push({
271
- post_id: +item.post_id,
272
- post_type: item.post_type,
273
- url: item.attachment_url ?? item.link,
274
- uuid: item.uuid,
275
- path: relative(outdir, outpath(item))
343
+ post_id: +full_item.post_id,
344
+ post_type: full_item.post_type,
345
+ url: full_item.attachment_url ?? full_item.link,
346
+ uuid: full_item.uuid,
347
+ path: relative(outdir, outpath(full_item))
276
348
  });
277
- metadata.types[item.post_type] = (metadata.types[item.post_type] || [])
278
- .concat(Object.keys(item));
349
+ metadata.types[full_item.post_type] =
350
+ (metadata.types[full_item.post_type] || [])
351
+ .concat(Object.keys(item));
279
352
  });
280
353
 
281
- const blueprints_path = join(outdir, 'site', 'blueprints');
282
- await mkdir(join(blueprints_path, 'pages'), { recursive: true });
283
- await mkdir(join(blueprints_path, 'files'), { recursive: true });
284
-
285
354
  Object.keys(metadata.types).forEach(key => {
286
355
  metadata.types[key] = Array.from(new Set(metadata.types[key])).sort();
356
+ });
357
+
358
+ if (outdir === '-') {
359
+ process.stdout.write(JSON.stringify(metadata, null, '\t'));
360
+ } else {
361
+ await mkdir(outdir, { recursive: true });
362
+ await writeFile(join(outdir, 'metadata.json'), JSON.stringify(metadata, null, '\t'));
363
+ }
364
+
365
+ if (!options['only-metadata']) {
366
+
367
+ for (let item of processed_items) {
368
+ const destination = outpath(item);
369
+ if (inside(outdir, destination)) {
370
+ await mkdir(dirname(destination), { recursive: true });
371
+ await writeFile(destination, serialize_item(item));
372
+ } else {
373
+ process.stderr.write(`Path not inside output directory, skipping: ${destination}\n`);
374
+ }
375
+ };
376
+
377
+ const blueprints_path = join(outdir, 'site', 'blueprints');
378
+ await mkdir(join(blueprints_path, 'pages'), { recursive: true });
379
+ await mkdir(join(blueprints_path, 'files'), { recursive: true });
380
+
287
381
  // Make blueprints
288
- const blueprint_content = metadata.types[key].reduce((acc, curr) => {
289
- acc[curr] = {
290
- type: 'text'
291
- };
292
- return acc;
293
- }, {});
382
+ Object.keys(metadata.types).forEach(key => {
383
+ const blueprint_content = metadata.types[key].reduce((acc, curr) => {
384
+ acc[curr] = {
385
+ type: 'text'
386
+ };
387
+ return acc;
388
+ }, {});
389
+
390
+ if (key === 'attachment') {
391
+ const blueprint_dest = join(
392
+ blueprints_path,
393
+ 'files',
394
+ key + '.yml'
395
+ );
294
396
 
295
- if (key === 'attachment') {
296
- const blueprint_dest = join(
297
- blueprints_path,
298
- 'files',
299
- key + '.yml'
300
- );
397
+ if (inside(outdir, blueprint_dest)) {
398
+ await writeFile(blueprint_dest, dump({
399
+ fields: blueprint_content
400
+ }));
401
+ } else {
402
+ process.stderr.write(`Path not inside output directory, skipping: ${blueprint_dest}\n`);
403
+ }
301
404
 
302
- writeFile(blueprint_dest, dump({
303
- fields: blueprint_content
304
- }));
405
+ } else {
406
+ const blueprint_dest = join(
407
+ blueprints_path,
408
+ 'pages',
409
+ key + '.yml'
410
+ );
305
411
 
306
- } else {
307
- const blueprint_dest = join(
308
- blueprints_path,
309
- 'pages',
310
- key + '.yml'
311
- );
312
-
313
- writeFile(blueprint_dest, dump({
314
- tabs: {
315
- fields: {
316
- label: 'Fields',
317
- sections: {
412
+ if (inside(outdir, blueprint_dest)) {
413
+ await writeFile(blueprint_dest, dump({
414
+ tabs: {
318
415
  fields: {
319
- type: 'fields',
320
- fields: blueprint_content
321
- }
322
- }
323
- },
324
- pages: {
325
- label: 'Pages',
326
- sections: {
416
+ label: 'Fields',
417
+ sections: {
418
+ fields: {
419
+ type: 'fields',
420
+ fields: blueprint_content
421
+ }
422
+ }
423
+ },
327
424
  pages: {
328
- type: 'pages'
329
- }
330
- }
331
- },
332
- files: {
333
- label: 'Files',
334
- sections: {
425
+ label: 'Pages',
426
+ sections: {
427
+ pages: {
428
+ type: 'pages'
429
+ }
430
+ }
431
+ },
335
432
  files: {
336
- type: 'files'
433
+ label: 'Files',
434
+ sections: {
435
+ files: {
436
+ type: 'files'
437
+ }
438
+ }
337
439
  }
338
440
  }
339
- }
441
+ }));
442
+ } else {
443
+ process.stderr.write(`Path not inside output directory, skipping: ${blueprint_dest}\n`);
340
444
  }
341
- }));
342
- }
343
- });
344
-
445
+ }
446
+ });
345
447
 
346
- const default_blueprint = {
347
- tabs: {
348
- pages: {
349
- label: 'Pages',
350
- sections: {
351
- pages: {
352
- type: 'pages'
448
+ const default_blueprint = {
449
+ tabs: {
450
+ pages: {
451
+ label: 'Pages',
452
+ sections: {
453
+ pages: {
454
+ type: 'pages'
455
+ }
353
456
  }
354
- }
355
- },
356
- files: {
357
- label: 'Files',
358
- sections: {
359
- files: {
360
- type: 'files'
457
+ },
458
+ files: {
459
+ label: 'Files',
460
+ sections: {
461
+ files: {
462
+ type: 'files'
463
+ }
361
464
  }
362
465
  }
363
466
  }
364
- }
365
- };
366
- await writeFile(join(blueprints_path, 'pages', 'default.yml'), dump(default_blueprint));
367
- await writeFile(join(blueprints_path, 'site.yml'), dump(default_blueprint));
467
+ };
468
+ await writeFile(join(blueprints_path, 'pages', 'default.yml'), dump(default_blueprint));
469
+ await writeFile(join(blueprints_path, 'site.yml'), dump(default_blueprint));
470
+ }
471
+ }
368
472
 
369
- await writeFile(join(outdir, 'metadata.json'), JSON.stringify(metadata, null, '\t'));
473
+ function readFromMetadataJSON(file) {
474
+ try {
475
+ return JSON.parse(file);
476
+ } catch(err) {}
477
+ return null;
370
478
  }
371
479
 
372
480
  if (command === 'download') {
@@ -377,45 +485,75 @@ if (command === 'download') {
377
485
  options.contentdir
378
486
  ) : null;
379
487
 
488
+ const file = files[0] === '-' ?
489
+ await slurp(process.stdin) :
490
+ await readFile(files[0], 'utf8');
491
+
492
+ const items = readFromMetadataJSON(file)?.output
493
+ .filter(item => item.post_type === 'attachment') ??
494
+ readFromExportXML(file)
495
+ .filter(item => item.post_type === 'attachment')
496
+ .map(process_item)
497
+ .map(item => ({
498
+ url: item.attachment_url,
499
+ path: relative(outdir, outpath(item))
500
+ }));
501
+
380
502
  const fetched = new Set();
381
503
 
382
- for await (let item of items.filter(it => it.post_type === 'attachment')) {
383
- const attachment = process_item(item);
384
- const dest = outpath(attachment).replace(/\.txt$/, '');
504
+ for (let item of items) {
505
+ const dest = join(outdir, item.path.replace(/\.txt$/, ''));
385
506
  if (contentdir) {
386
- const pathname = new URL(attachment.attachment_url).pathname
507
+ const pathname = new URL(item.url).pathname
387
508
  .split('/')
388
509
  .filter(p => p && p !== 'wp-content')
389
510
  .join(sep);
390
511
  if (typeof contentdir === 'string') {
391
- await copyFile(join(contentdir, pathname), dest);
512
+ if (inside(outdir, dest)) {
513
+ await mkdir(dirname(dest), { recursive: true });
514
+ await copyFile(join(contentdir, pathname), dest);
515
+ } else {
516
+ process.stderr.write(`Path not inside output directory, skipping: ${dest}\n`);
517
+ }
392
518
  } else {
393
519
  const url = new URL(pathname, contentdir);
394
- if (fetched.has(url)) {
395
- console.log(`Already fetched, skipping: ${url}`);
520
+ if (fetched.has(url.toString())) {
521
+ process.stderr.write(`Already fetched, skipping: ${url}\n`);
396
522
  } else {
397
- console.log(`Fetching: ${url}`);
398
- await mkdir(dirname(dest), { recursive: true });
399
- await writeFile(
400
- dest,
401
- (await fetch(url)).body
402
- );
403
- console.log(`Fetched: ${url}`);
404
- fetched.add(url);
523
+ if (inside(outdir, dest)) {
524
+ process.stderr.write(`Fetching: ${url}\n`);
525
+ const res = await fetch(url);
526
+ if (res.ok) {
527
+ await mkdir(dirname(dest), { recursive: true });
528
+ await writeFile(dest, res.body);
529
+ process.stderr.write(`Fetched: ${url}\n`);
530
+ } else {
531
+ process.stderr.write(`Error fetching: ${url}\n`);
532
+ }
533
+ fetched.add(url.toString());
534
+ } else {
535
+ process.stderr.write(`Path not inside output directory, skipping: ${dest}\n`);
536
+ }
405
537
  }
406
538
  }
407
539
  } else {
408
- if (fetched.has(attachment.attachment_url)) {
409
- console.log(`Already fetched, skipping: ${attachment.attachment_url}`);
540
+ if (fetched.has(item.url)) {
541
+ process.stderr.write(`Already fetched, skipping: ${item.url}\n`);
410
542
  } else {
411
- console.log(`Fetching: ${attachment.attachment_url}`);
412
- await mkdir(dirname(dest), { recursive: true });
413
- await writeFile(
414
- dest,
415
- (await fetch(attachment.attachment_url)).body
416
- );
417
- console.log(`Fetched: ${attachment.attachment_url}`);
418
- fetched.add(attachment.attachment_url);
543
+ if (inside(outdir, dest)) {
544
+ process.stderr.write(`Fetching: ${item.url}\n`);
545
+ const res = await fetch(item.url);
546
+ if (res.ok) {
547
+ await mkdir(dirname(dest), { recursive: true });
548
+ await writeFile(dest, res.body);
549
+ process.stderr.write(`Fetched: ${item.url}\n`);
550
+ } else {
551
+ process.stderr.write(`Error fetching: ${item.url}\n`);
552
+ }
553
+ fetched.add(item.url);
554
+ } else {
555
+ process.stderr.write(`Path not inside output directory, skipping: ${dest}\n`);
556
+ }
419
557
  }
420
558
  }
421
559
 
@@ -423,22 +561,26 @@ if (command === 'download') {
423
561
  }
424
562
 
425
563
  function printHelp() {
426
- console.log(`wordpress-to-kirby:
564
+ process.stdout.write(`wordpress-to-kirby:
427
565
 
428
566
  transform a WordPress XML export into a directory of Kirby CMS files.
429
567
 
430
568
  Usage:
431
569
 
432
- wordpress-to-kirby generate [options] <wordpress-xml-file>
433
- wordpress-to-kirby download [options] <wordpress-xml-file>
570
+ wordpress-to-kirby generate [options] <input-file>
571
+ wordpress-to-kirby download [options] <input-file>
434
572
 
435
573
  Commands available:
436
574
 
437
575
  generate
576
+
438
577
  Generate a Kirby CMS file structure from the provided XML file.
439
578
 
440
579
  download
441
- Download attachments.
580
+
581
+ Download attachments from the input file, which can be either
582
+ a WordPress XML export file or a JSON metadata file produced
583
+ by the 'generate' command.
442
584
 
443
585
  Options available:
444
586
 
@@ -450,14 +592,14 @@ Options available:
450
592
 
451
593
  Shows the program version.
452
594
 
453
- -o, --outdir
595
+ -o <path>, --outdir=<path>
454
596
 
455
597
  Root directory to which to write the output,
456
598
  relative to the current working directory.
457
599
 
458
600
  Default: 'wordpress-to-kirby'
459
601
 
460
- --contentdir
602
+ --contentdir=<path>
461
603
 
462
604
  For the 'download' command, point to the location
463
605
  of a 'wp-content' directory from which to download attachments.
@@ -471,7 +613,9 @@ Options available:
471
613
  --parse=<rules>
472
614
 
473
615
  Instructions to parse some fields, specified as:
474
- "<key>=<parser>,<key>=<parser>,…"
616
+ "<key>=<parser>,<key>=<parser>,…".
617
+
618
+ The option can be used multiple times.
475
619
 
476
620
  Supported parsers:
477
621
  * php: deserialize PHP object.
@@ -484,6 +628,8 @@ Options available:
484
628
  Instructions to serialize some fields, specified as:
485
629
  "<key>=<serializer>,<key>=<serializer>,…"
486
630
 
631
+ The option can be used multiple times.
632
+
487
633
  Supported serializers:
488
634
  * php: serialize as PHP object.
489
635
  * yaml: serialize as YAML.
@@ -493,12 +639,12 @@ Options available:
493
639
  --exclude-fields=<fields>
494
640
 
495
641
  Exclude certain fields from the output, specified as a
496
- comma-separated list of fields.
642
+ comma-separated list of fields. Can be used multiple times.
497
643
 
498
644
  --include-fields=<fields>
499
645
 
500
646
  Only include certain fields from the output, specified as a
501
- comma-separated list of fields.
647
+ comma-separated list of fields. Can be used multiple times.
502
648
 
503
649
  When '--include-fields' is specified, the '--exclude-fields'
504
650
  option is ignored.
@@ -506,15 +652,20 @@ Options available:
506
652
  --exclude-types=<types>
507
653
 
508
654
  Exclude certain post types from the output, specified as a
509
- comma-separated list of post types.
655
+ comma-separated list of post types. Can be used multiple times.
510
656
 
511
657
  --include-types=<types>
658
+
512
659
  Only include certain post types from the output, specified as a
513
- comma-separated list of post types.
660
+ comma-separated list of post types. Can be used multiple times.
514
661
 
515
662
  When '--include-types' is specified, the '--exclude-types'
516
663
  option is ignored.
517
664
 
665
+ --only-metadata
666
+
667
+ For the 'generate' command, only generate the 'metadata.json' file.
668
+
518
669
  Keys extracted from the XML:
519
670
 
520
671
  Top-level:
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "wordpress-to-kirby",
3
- "version": "0.2.0",
3
+ "version": "0.4.0",
4
4
  "description": "A CLI to convert WordPress XML to Kirby CMS files",
5
5
  "homepage": "https://github.com/tocmai/wordpress-to-kirby#readme",
6
6
  "bugs": {
@@ -21,7 +21,9 @@
21
21
  "dependencies": {
22
22
  "js-yaml": "^5.5.0",
23
23
  "jsdom": "^31.0.1",
24
- "opsh": "^1.1.0",
25
24
  "php-serialize": "^5.1.3"
25
+ },
26
+ "engines": {
27
+ "node": ">=22"
26
28
  }
27
29
  }
package/src/inside.js ADDED
@@ -0,0 +1,6 @@
1
+ import { relative, isAbsolute, sep } from 'node:path';
2
+ // See: https://github.com/sindresorhus/is-path-inside
3
+ export default function inside(dir, file) {
4
+ const rel = relative(dir, file);
5
+ return rel && rel.split(sep).shift() !== '..' && !isAbsolute(rel);
6
+ }
package/src/slurp.js ADDED
@@ -0,0 +1,5 @@
1
+ export default async function slurp(stream) {
2
+ // Switch to text mode, default is binary
3
+ stream.setEncoding('utf8');
4
+ return (await Array.fromAsync(stream)).join('');
5
+ }
@@ -0,0 +1,11 @@
1
+ import test from 'node:test';
2
+ import { equal } from 'node:assert';
3
+
4
+ import inside from '../src/inside.js';
5
+
6
+ test('inside()', () => {
7
+ equal(inside('a', ''), false);
8
+ equal(inside('a', 'a/.'), false);
9
+ equal(inside('a', 'a/b'), true);
10
+ equal(inside('a', 'a/../a/b'), true);
11
+ });