storyblok-backup 0.0.4 → 0.1.0

package/.env.example

@@ -0,0 +1,3 @@
+STORYBLOK_OAUTH_TOKEN=a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4y5z6
+STORYBLOK_SPACE_ID=123456
+STORYBLOK_REGION=eu

package/README.md

@@ -5,13 +5,17 @@
 
 A npx CLI tool to create a full backup of a space of the [Storyblok CMS](https://www.storyblok.com).
 
-The script will fetch the following resources of a Storyblok space using the Management API and archive them in a zip file:
+A restore tool to restore (create or update) resources is also included.
+
+The backup script will fetch the following resources of a Storyblok space using the Management API and archive them in a zip file:
 
 - Stories
+- Collaborators
 - Components
 - Component groups
 - Assets (optionally incl. original files)
 - Asset folders
+- Internal Tags
 - Datasources (incl. entries)
 - Space
 - Space Roles
@@ -19,17 +23,31 @@ The script will fetch the following resources of a Storyblok space using the Man
 - Activities
 - Presets
 - Field types
+- Webhooks
+- Workflows
 - Workflow stages
 - Workflow stage changes
-- Custom workflows
 - Releases
+- Pipeline Branches
+- Access Tokens
+
+The restore script is able to individually restore the the resources from the backup files (via update or create) with the following exceptions:
+
+- Assets: Creating assets is not supported
+- Tasks: Currently not supported due to missing fields returned from management API
+- Field types: Currently not supported
+- Workflow stage changes: No update possible
+- Access Tokens: Creating access tokens from backup makes no sense, since it will result in a new token-string.
 
 ## Installation
 
 ```shell
 
 # simply auto-download and run via npx
+## for backup:
 $ npx storyblok-backup
+## for restore:
+$ npx -p storyblok-backup storyblok-restore
 
 # or install globally
 $ npm install -g storyblok-backup
@@ -46,21 +64,26 @@ $ pnpm add storyblok-backup
 
 ## Usage
 
+### Backup
+
 Call `npx storyblok-backup` with the following options:
 
-### Options
+#### Backup options
 
 ```text
 --token <token>     (required) Personal OAuth access token created
                     in the account settings of a Stoyblok user.
                     (NOT the Access Token of a Space!)
+                    Alternatively, you can set the STORYBLOK_OAUTH_TOKEN environment variable.
 --space <space_id>  (required) ID of the space to backup
+                    Alternatively, you can set the STORYBLOK_SPACE_ID environment variable.
 --region <region>   Region of the space. Possible values are:
                     - 'eu' (default): EU
                     - 'us': US
                     - 'ap': Australia
                     - 'ca': Canada
                     - 'cn': China
+                    Alternatively, you can set the STORYBLOK_REGION environment variable.
 --with-asset-files  Downloads all files (assets) of the space. Defaults to false.
 --output-dir <dir>  Directory to write the backup to. Defaults to ./.output
                     (ATTENTION: Will fail if the directory already exists!)
@@ -72,7 +95,9 @@ Call `npx storyblok-backup` with the following options:
 --help              Show this help
 ```
 
-### Minimal example
+OAuth token, space-id and region can be set via environment variables. You can also use a `.env` file in your project root for this (see `.env.example`).
+
+#### Minimal backup example
 
 ```shell
 npx storyblok-backup --token 1234567890abcdef --space 12345
@@ -80,7 +105,7 @@ npx storyblok-backup --token 1234567890abcdef --space 12345
 
 This will create the folder `./.output/backup` and fetch all resources sorted into folders.
 
-### Maximal example
+#### Maximal backup example
 
 ```shell
 npx storyblok-backup \
@@ -97,7 +122,7 @@ npx storyblok-backup \
 
 This will create the folder `./my-dir/backup`, fetch all resources (incl. the original file assets) sorted into folders, zip them to `./my-dir/daily-Y-m-d-H-i-s.zip`, and log every written file to console.
 
-## Continuous Integration
+#### Continuous backup integration
 
 You can e.g. use this script to create periodic backups of Storyblok spaces using GitHub Actions and artifacts.
 
@@ -158,6 +183,79 @@ If you create multiple workflows for daily, weekly and monthly backups, by chang
 
 Also keep in mind, that there is a limit on artifact storage and runner minutes ([see GitHub docs](https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions#included-storage-and-minutes)).
 
+### Restore
+
+Call `npx -p storyblok-backup storyblok-restore` with the following options:
+
+#### Restore options
+
+```text
+--token <token>     (required) Personal OAuth access token created
+                    in the account settings of a Stoyblok user.
+                    (NOT the Access Token of a Space!)
+                    Alternatively, you can set the STORYBLOK_OAUTH_TOKEN environment variable.
+--space <space_id>  (required) ID of the space to backup
+                    Alternatively, you can set the STORYBLOK_SPACE_ID environment variable.
+--region <region>   Region of the space. Possible values are:
+                    - 'eu' (default): EU
+                    - 'us': US
+                    - 'ap': Australia
+                    - 'ca': Canada
+                    - 'cn': China
+                    Alternatively, you can set the STORYBLOK_REGION environment variable.
+--type <type>       (required) Type of resource to restore. Possible values are:
+                    - 'story'
+                    - 'collaborator'
+                    - 'component'
+                    - 'component-group'
+                    - 'asset'
+                    - 'asset-folder'
+                    - 'internal-tag'
+                    - 'datasource'
+                    - 'datasource-entries'
+                    - 'space'
+                    - 'space-role'
+                    - 'preset'
+                    - 'webhook'
+                    - 'workflow'
+                    - 'workflow-stage'
+                    - 'release'
+                    - 'pipeline-branch'
+                    - 'access-token
+--file <file>       (required) File of resource to restore.
+--publish           Perform a publish after restore of a story (default=false).
+--create            Create a new resource instead of updating (default=false).
+                    Not supported for assets.
+--id <file>         (required if type=datasource-entries and create is set)
+                    ID of datasource the entries belong to.
+--verbose           Will show detailed result of the restore process.
+--help              Show this help
+```
+
+#### Minimal restore example
+
+```shell
+npx -p storyblok-backup storyblok-restore --token 1234567890abcdef --space 12345 --type story --file ./.output/backup/123456789.json
+```
+
+This will restore the story from the stated file by updating it.
+
+#### Maximal restore example
+
+```shell
+npx -p storyblok-backup storyblok-restore \
+  --token 1234567890abcdef \
+  --space 12345 \
+  --region ap \
+  --type story \
+  --file ./.output/backup/123456789.json \
+  --publish \
+  --create \
+  --verbose
+```
+
+This will restore the story by creating a new story, immediately publish it, and log the API result to console.
+
 ## License
 
 This package is open-sourced software licensed under the [MIT license](https://github.com/webflorist/storyblok-backup/blob/main/LICENSE.).

package/bin/storyblok-backup.mjs

@@ -7,9 +7,12 @@ import zipLib from 'zip-lib'
 import minimist from 'minimist'
 import StoryblokClient from 'storyblok-js-client'
 import { performance } from 'perf_hooks'
+import dotenvx from '@dotenvx/dotenvx'
 
 const startTime = performance.now()
 
+dotenvx.config({ quiet: true })
+
 const args = minimist(process.argv.slice(2))
 
 if ('help' in args) {
@@ -20,13 +23,16 @@ OPTIONS
   --token <token>     (required) Personal OAuth access token created
                       in the account settings of a Stoyblok user.
                       (NOT the Access Token of a Space!)
+                      Alternatively, you can set the STORYBLOK_OAUTH_TOKEN environment variable.
   --space <space_id>  (required) ID of the space to backup
+                      Alternatively, you can set the STORYBLOK_SPACE_ID environment variable.
   --region <region>   Region of the space. Possible values are:
                       - 'eu' (default): EU
                       - 'us': US
                       - 'ap': Australia
                       - 'ca': Canada
                       - 'cn': China
+                      Alternatively, you can set the STORYBLOK_REGION environment variable.
   --with-asset-files  Downloads all files (assets) of the space. Defaults to false.
   --output-dir <dir>  Directory to write the backup to. Defaults to ./.output
                       (ATTENTION: Will fail if the directory already exists!)
@@ -55,21 +61,30 @@ MAXIMAL EXAMPLE
 	process.exit(0)
 }
 
-if (!('token' in args)) {
+if (!('token' in args) && !process.env.STORYBLOK_OAUTH_TOKEN) {
 	console.log(
-		'Error: State your oauth token via the --token argument. Use --help to find out more.'
+		'Error: State your oauth token via the --token argument or the environment variable STORYBLOK_OAUTH_TOKEN. Use --help to find out more.'
 	)
 	process.exit(1)
 }
+const oauthToken = args.token || process.env.STORYBLOK_OAUTH_TOKEN
 
-if (!('space' in args)) {
-	console.log('Error: State your space id via the --space argument. Use --help to find out more.')
+if (!('space' in args) && !process.env.STORYBLOK_SPACE_ID) {
+	console.log(
+		'Error: State your space id via the --space argument or the environment variable STORYBLOK_SPACE_ID. Use --help to find out more.'
+	)
 	process.exit(1)
 }
+const spaceId = args.space || process.env.STORYBLOK_SPACE_ID
 
-if ('region' in args && !['eu', 'us', 'ap', 'ca', 'cn'].includes(args.region)) {
-	console.log('Error: Invalid region parameter stated. Use --help to find out more.')
-	process.exit(1)
+let region = 'eu'
+if ('region' in args || process.env.STORYBLOK_REGION) {
+	region = args.region || process.env.STORYBLOK_REGION
+
+	if (!['eu', 'us', 'ap', 'ca', 'cn'].includes(region)) {
+		console.log('Error: Invalid region parameter stated. Use --help to find out more.')
+		process.exit(1)
+	}
 }
 
 const verbose = 'verbose' in args
@@ -85,10 +100,6 @@ if (fs.existsSync(outputDir) && !('force' in args)) {
 	process.exit(1)
 }
 
-const spaceId = args.space
-
-const region = args['region'] || 'eu'
-
 const filePrefix = args['zip-prefix'] || 'backup'
 
 const fileName =
@@ -112,7 +123,7 @@ if ('create-zip' in args) {
 
 // Init Management API
 const StoryblokMAPI = new StoryblokClient({
-	oauthToken: args.token,
+	oauthToken: oauthToken,
 	region: region,
 })
 
@@ -126,20 +137,25 @@ fs.mkdirSync(backupDir, { recursive: true })
 
 const resources = [
 	'stories',
+	'collaborators',
 	'components',
 	'component-groups',
 	'assets',
 	'asset-folders',
+	'internal-tags',
 	'datasources',
 	'space-roles',
 	'tasks',
 	'activities',
 	'presets',
 	'field-types',
+	'webhooks',
 	'workflow-stages',
 	'workflow-stage-changes',
 	'workflows',
 	'releases',
+	'pipeline-branches',
+	'access-tokens',
 ]
 resources.forEach((resource) => fs.mkdirSync(`${backupDir}/${resource}`))
 
@@ -170,7 +186,7 @@ const writeJson = (folder, file, content) => {
 		outputFile += `/${folder}`
 	}
 	outputFile += `/${file}.json`
-	fs.writeFile(outputFile, JSON.stringify(content), (error) => {
+	fs.writeFileSync(outputFile, JSON.stringify(content, null, 2), (error) => {
 		if (error) {
 			throw error
 		}
@@ -203,7 +219,10 @@ await StoryblokMAPI.getAll(`spaces/${spaceId}/stories`)
 	.then(async (stories) => {
 		for (const story of stories) {
 			await StoryblokMAPI.get(`spaces/${spaceId}/stories/${story.id}`)
-				.then((response) => writeJson('stories', story.id, response.data.story))
+				.then((response) => {
+					delete response.data.story.preview_token
+					writeJson('stories', story.id, response.data.story)
+				})
 				.catch((error) => {
 					throw error
 				})
@@ -213,6 +232,10 @@ await StoryblokMAPI.getAll(`spaces/${spaceId}/stories`)
 		throw error
 	})
 
+// Fetch all collaborators
+console.log(`Fetching collaborators`)
+await defaultFetch('collaborators', 'collaborators', 'user_id')
+
 // Fetch all components
 console.log(`Fetching components`)
 await defaultFetch('components', 'components', 'name')
@@ -242,6 +265,10 @@ await StoryblokMAPI.getAll(`spaces/${spaceId}/assets`)
 console.log(`Fetching asset-folders`)
 await defaultFetch('asset_folders', 'asset-folders', 'id')
 
+// Fetch all internal-tags
+console.log(`Fetching internal-tags`)
+await defaultFetch('internal_tags', 'internal-tags', 'id')
+
 // Fetch all datasources (including entries)
 console.log(`Fetching datasources`)
 await StoryblokMAPI.getAll(`spaces/${spaceId}/datasources`)
@@ -283,6 +310,10 @@ await defaultFetch('presets', 'presets', 'id')
 console.log(`Fetching field-types`)
 await defaultFetch('field_types', 'field-types', 'name')
 
+// Fetch all webhooks
+console.log(`Fetching webhooks`)
+await defaultFetch('webhook_endpoints', 'webhooks', 'id')
+
 // Fetch all workflow-stages
 console.log(`Fetching workflow-stages`)
 await defaultFetch('workflow_stages', 'workflow-stages', 'id')
@@ -299,6 +330,14 @@ await defaultFetch('workflows', 'workflows', 'id')
 console.log(`Fetching releases`)
 await defaultFetch('releases', 'releases', 'id')
 
+// Fetch all pipeline branches
+console.log(`Fetching pipeline branches`)
+await defaultFetch('branches', 'pipeline-branches', 'id')
+
+// Fetch all access tokens
+console.log(`Fetching access tokens`)
+await defaultFetch('api_keys', 'access-tokens', 'id')
+
 // Create zip file
 if ('create-zip' in args) {
 	console.log(`Creating zip file`)

package/bin/storyblok-restore.mjs

@@ -0,0 +1,293 @@
+#!/usr/bin/env node
+/* eslint-disable no-console */
+import fs from 'fs'
+import minimist from 'minimist'
+import StoryblokClient from 'storyblok-js-client'
+import { performance } from 'perf_hooks'
+import dotenvx from '@dotenvx/dotenvx'
+
+const startTime = performance.now()
+
+dotenvx.config({ quiet: true })
+
+const resourceTypes = [
+	'story',
+	'collaborator',
+	'component',
+	'component-group',
+	'asset',
+	'asset-folder',
+	'internal-tag',
+	'datasource',
+	'datasource-entries',
+	'space',
+	'space-role',
+	//'task',  // Currently not supported due to missing fields returned from management API
+	'preset',
+	// 'field-type',
+	'webhook',
+	'workflow',
+	'workflow-stage',
+	'release',
+	'pipeline-branch',
+	'access-token',
+]
+
+const args = minimist(process.argv.slice(2))
+
+if ('help' in args) {
+	console.log(`USAGE
+  $ npx -p storyblok-backup storyblok-restore
+  
+OPTIONS
+  --token <token>     (required) Personal OAuth access token created
+                      in the account settings of a Stoyblok user.
+                      (NOT the Access Token of a Space!)
+                      Alternatively, you can set the STORYBLOK_OAUTH_TOKEN environment variable.
+  --space <space_id>  (required) ID of the space to backup
+                      Alternatively, you can set the STORYBLOK_SPACE_ID environment variable.
+  --region <region>   Region of the space. Possible values are:
+                      - 'eu' (default): EU
+                      - 'us': US
+                      - 'ap': Australia
+                      - 'ca': Canada
+                      - 'cn': China
+                      Alternatively, you can set the STORYBLOK_REGION environment variable.
+  --type <type>       (required) Type of resource to restore. Possible values are:
+                      - '${resourceTypes.join("'\n                      - '")}
+  --file <file>       (required) File of resource to restore.
+  --publish           Perform a publish after restore of a story (default=false).
+  --create            Create a new resource instead of updating (default=false).
+                      Not supported for assets.
+  --id <file>         (required if type=datasource-entries and create is set)
+                      ID of datasource the entries belong to.
+  --verbose           Will show detailed result of the restore process.
+  --help              Show this help
+
+MINIMAL EXAMPLE
+  $ npx -p storyblok-backup storyblok-restore --token 1234567890abcdef --space 12345 --type story --file ./.output/backup/123456789.json
+
+MAXIMAL EXAMPLE
+  $ npx -p storyblok-backup storyblok-restore \\
+      --token 1234567890abcdef \\
+      --space 12345 \\
+      --region ap \\
+      --type story \\
+      --file ./.output/backup/123456789.json \\
+      --publish \\
+      --create \\
+      --verbose
+`)
+	process.exit(0)
+}
+
+if (!('token' in args) && !process.env.STORYBLOK_OAUTH_TOKEN) {
+	console.log(
+		'Error: State your oauth token via the --token argument or the environment variable STORYBLOK_OAUTH_TOKEN. Use --help to find out more.'
+	)
+	process.exit(1)
+}
+const oauthToken = args.token || process.env.STORYBLOK_OAUTH_TOKEN
+
+if (!('space' in args) && !process.env.STORYBLOK_SPACE_ID) {
+	console.log(
+		'Error: State your space id via the --space argument or the environment variable STORYBLOK_SPACE_ID. Use --help to find out more.'
+	)
+	process.exit(1)
+}
+const spaceId = args.space || process.env.STORYBLOK_SPACE_ID
+
+let region = 'eu'
+if ('region' in args || process.env.STORYBLOK_REGION) {
+	region = args.region || process.env.STORYBLOK_REGION
+
+	if (!['eu', 'us', 'ap', 'ca', 'cn'].includes(region)) {
+		console.log('Error: Invalid region parameter stated. Use --help to find out more.')
+		process.exit(1)
+	}
+}
+
+if (!('type' in args)) {
+	console.log(
+		'Error: State the resource type to restore via the --type argument. Use --help to find out more.'
+	)
+	process.exit(1)
+}
+
+if (!resourceTypes.includes(args.type)) {
+	console.log(`Error: Invalid resource type "${args.type}". Use --help to find out more.`)
+	process.exit(1)
+}
+
+if (!('file' in args)) {
+	console.log(
+		'Error: State the resource file to restore via the --file argument. Use --help to find out more.'
+	)
+	process.exit(1)
+}
+
+if (!fs.existsSync(args.file)) {
+	console.log(`Error: Stated file "${args.file}" does not exist.`)
+	process.exit(1)
+}
+
+const verbose = 'verbose' in args
+
+const publish = 'publish' in args
+
+const create = 'create' in args
+
+// Init Management API
+const StoryblokMAPI = new StoryblokClient({
+	oauthToken: oauthToken,
+	region: region,
+})
+
+// Function to perform a default single resource restore
+const defaultSingleRestore = async (type, id, params) => {
+	if (publish) {
+		params.publish = 1
+	}
+	let url = `spaces`
+
+	if (!(create && args.type === 'space')) {
+		url = `${url}/${spaceId}`
+	}
+
+	if (type) {
+		url = `${url}/${type}`
+	}
+
+	if (create) {
+		await StoryblokMAPI.post(url, params)
+			.then((response) => {
+				console.log(`Created "${type}" resource.`)
+				if (verbose) {
+					console.log('Result:')
+					console.log(response.data)
+				}
+			})
+			.catch((error) => {
+				throw error
+			})
+	} else {
+		if (id) {
+			url = `${url}/${id}`
+		}
+		await StoryblokMAPI.put(url, params)
+			.then((response) => {
+				console.log(`Updated "${type}" resource with id "${id}".`)
+				if (verbose) {
+					console.log('Result:')
+					console.log(response.data)
+				}
+			})
+			.catch((error) => {
+				throw error
+			})
+	}
+}
+
+const resource = JSON.parse(fs.readFileSync(args.file, 'utf8'))
+
+switch (args.type) {
+	case 'story':
+		delete resource.updated_at
+		await defaultSingleRestore('stories', resource.id, { story: resource })
+		break
+	case 'collaborator':
+		await defaultSingleRestore(
+			'collaborators',
+			resource.id,
+			create
+				? {
+						email: resource.user.userid,
+						role: resource.role,
+						space_id: resource.space_id,
+						permissions: resource.permissions,
+						space_role_ids: resource.space_role_ids,
+						allow_multiple_roles_creation: resource.role === 'multi',
+					}
+				: { collaborator: resource }
+		)
+		break
+	case 'component':
+		await defaultSingleRestore('components', resource.id, { component: resource })
+		break
+	case 'component-group':
+		await defaultSingleRestore('component_groups', resource.id, { component_group: resource })
+		break
+	case 'asset':
+		if (create) {
+			console.log('Error: Creating assets is not supported.')
+			process.exit(1)
+		}
+		await defaultSingleRestore('assets', resource.id, { asset: resource })
+		break
+	case 'asset-folder':
+		await defaultSingleRestore('asset_folders', resource.id, { asset_folder: resource })
+		break
+	case 'internal-tag':
+		await defaultSingleRestore('internal_tags', resource.id, { internal_tag: resource })
+		break
+	case 'datasource':
+		await defaultSingleRestore('datasources', resource.id, { datasource: resource })
+		break
+	case 'datasource-entries':
+		if (create && !('id' in args)) {
+			console.log(
+				'Error: State the datasource ID via the --id argument. Use --help to find out more.'
+			)
+			process.exit(1)
+		}
+		for (const entry of resource) {
+			if (create) {
+				entry.datasource_id = args.id
+			}
+			await defaultSingleRestore('datasource_entries', entry.id, {
+				datasource_entry: entry,
+			})
+		}
+		break
+	case 'space':
+		await defaultSingleRestore(null, null, { space: resource })
+		break
+	case 'space-role':
+		await defaultSingleRestore('space_roles', resource.id, { space_role: resource })
+		break
+	case 'task':
+		await defaultSingleRestore('tasks', resource.id, { task: resource })
+		break
+	case 'preset':
+		await defaultSingleRestore('presets', resource.id, { preset: resource })
+		break
+	case 'webhook':
+		await defaultSingleRestore('webhook_endpoints', resource.id, { webhook_endpoint: resource })
+		break
+	case 'workflow':
+		await defaultSingleRestore('workflows', resource.id, { workflow: resource })
+		break
+	case 'workflow-stage':
+		await defaultSingleRestore('workflow_stages', resource.id, { workflow_stage: resource })
+		break
+	case 'release':
+		await defaultSingleRestore('releases', resource.id, { release: resource })
+		break
+	case 'pipeline-branch':
+		await defaultSingleRestore('branches', resource.id, { branch: resource })
+		break
+	case 'access-token':
+		if (create) {
+			console.log(
+				'Error: Creating access-tokens from backup is not possible, since it will result in a new token.'
+			)
+			process.exit(1)
+		}
+		await defaultSingleRestore('api_keys', resource.id, { api_key: resource })
+		break
+}
+
+const endTime = performance.now()
+
+console.log(`Restore successful in ${Math.round((endTime - startTime) / 1000)} seconds.`)
+process.exit(0)

package/package.json

@@ -1,7 +1,7 @@
 {
 	"name": "storyblok-backup",
-	"version": "0.0.4",
-	"description": "npx CLI tool to create a full backup of a Storyblok space",
+	"version": "0.1.0",
+	"description": "npx CLI tool to create a full backup of a Storyblok space and restore single resources from it.",
 	"scripts": {
 		"upgrade": "npx npm-check-updates -i -u && pnpm install",
 		"lint:js": "eslint --ext .vue,.js,.jsx,.cjs,.mjs,.ts,.tsx,.cts,.mts --ignore-path .lintignore .",
@@ -12,7 +12,8 @@
 		"lintfix": "pnpm lintfix:js && pnpm lintfix:prettier"
 	},
 	"bin": {
-		"storyblok-backup": "bin/storyblok-backup.mjs"
+		"storyblok-backup": "bin/storyblok-backup.mjs",
+		"storyblok-restore": "bin/storyblok-restore.mjs"
 	},
 	"repository": {
 		"type": "git",
@@ -40,6 +41,7 @@
 		"prettier": "^3.3.2"
 	},
 	"dependencies": {
+		"@dotenvx/dotenvx": "^1.6.2",
 		"minimist": "^1.2.8",
 		"storyblok-js-client": "^6.7.3",
 		"zip-lib": "^1.0.4"