@automattic/vip 2.32.0 → 2.32.2

package/.prettierignore

@@ -1,6 +1,4 @@
-*.generated.d.ts
 /CHANGELOG.md
 /__fixtures__/
 /dist/
 /flow-typed/
-/src/graphqlTypes.d.ts

package/CHANGELOG.md

@@ -1,5 +1,19 @@
 ## Changelog
 
+### 2.32.2
+
+- #1443 On-demand Database Backup: Use a less error-prone way of detecting if we're rate limited
+- #1434 Add more clarifying steps for making a release
+- #1435 Remove enums from GraphQL codegen and make sure that prettier actually runs
+- #1433 Redeploy codegen for GraphQL queries
+- #1432 Revert "Add codegen for GraphQL queries (#1429)"
+- #1304 Generate docs
+
+### 2.32.1
+
+- #1445 chore(deps): update vulnerable dependencies to fix ReDoS in semver
+- #1444 chore(deps): update Lando
+
 ### 2.32.0
 
 - #1411 feature(export): Add ability to generate a new backup when exporting SQL

package/CONTRIBUTING.md

@@ -97,26 +97,23 @@ We use a custom pre-publish [script](https://github.com/Automattic/vip/blob/trun
 
 Further checks can be added to this flow as needed.
 
-### Pre-publish Tasks
-
-As part of the publish flow, we run the `prepareConfig:publish` task on `prepack`. This copies over "production" config values to your working copy to make sure the release includes those instead of development values.
-
-We use `prepack` because:
-
-- `prepareConfig:local` runs on `npm build` and we want to make sure those values are overriden.
-- This is the latest npm event that we can run on before publishing. (Note: we tried `prepublishOnly` but files added during that step [don't get included in the build](https://github.com/Automattic/vip/commit/c7dabe1b0f73ec9e6e8c05ccff0c41281e4cd5e8)).
-
 ### New Releases
 
 Prepare the release by making sure that:
 
 1. All relevant PRs have been merged.
 1. The release has been tested across macOS, Windows, and Linux.
-1. The [changelog](https://github.com/Automattic/vip/blob/trunk/CHANGELOG.md) has been updated on `trunk`.
-1. All tests pass and your working directory is clean (we have pre-publish checks to catch this, just-in-case).
+1. All tests pass and your working directory is clean (we have pre-publish checks to catch this,
+   just-in-case).
+1. Make sure not to merge anymore changes into `develop` while all the release steps below are in
+   progress.
 
 #### Changelog Generator Hint:
 
+In the first step, you'll need to generate a changelog.
+
+Run the following and copy the output somewhere.
+
 ```
 export LAST_RELEASE_DATE=2021-08-25T13:40:00+02
 gh pr list --search "is:merged sort:updated-desc closed:>$LAST_RELEASE_DATE" | sed -e 's/\s\+\S\+\tMERGED.*$//' -e 's/^/- #/'
@@ -124,15 +121,28 @@ gh pr list --search "is:merged sort:updated-desc closed:>$LAST_RELEASE_DATE" | s
 
 Then, let's publish:
 
-1. Make sure trunk branch is up to date `git pull`
+1. Create a pull request that adds the next version's changelog into `develop`. Use the Changelog
+   Generate Hint above to generate the changelog, and refer to previous releases to ensure that your
+   format matches.
+1. Create a pull request that merges `develop` to `trunk`.
+1. Merge it after approval.
+1. Make sure trunk branch is up-to-date `git pull`.
+1. Make sure to clean all of your repositories of extra files. Run a dangerous, destructive
+   command `git clean -xfd` to do so.
+1. Run `npm install`.
 1. Set the version (via `npm version minor` or `npm version major` or `npm version patch`)
 1. For most regular releases, this will be `npm version minor`.
 1. Push the tag to GitHub (`git push --tags`)
 1. Push the trunk branch `git push`
 1. Make sure you're part of the Automattic organization in npm
-1. Publish the release to npm (`npm publish --access public`) the script will do some extra checks (node version, branch, etc) to ensure everything is correct. If all looks good, the new version will be published and you can proceed.
-1. Edit [the release on GitHub](https://github.com/Automattic/vip/releases) to include a description of the changes and publish (this can just copy the details from the changelog).
-1. Push `trunk` changes (mostly the version bump) to `develop` (`git checkout develop && git merge trunk` )
+1. Publish the release to npm (`npm publish --access public`) the script will do some extra checks (
+   node version, branch, etc) to ensure everything is correct. If all looks good, the new version
+   will be published and you can proceed.
+1. Edit [the release on GitHub](https://github.com/Automattic/vip/releases) to include a description
+   of the changes and publish (this can just copy the details from the changelog).
+1. Push `trunk` changes (mostly the version bump)
+   to `develop` (`git checkout develop && git merge trunk` ). There's no need to use a pull request
+   to do so.
 
 Once released, it's worth running `npm i -g @automattic/vip` to install / upgrade the released version to make sure everything looks good.
 
@@ -161,6 +171,20 @@ Then, repeat for any additional versions that we need to patch.
 
 ### go-search-replace binaries
 
-Some unit tests require some go-search-replace executable binary files to run. Binaries files for several OS architectures can be downloaded from https://github.com/Automattic/go-search-replace/releases/
+Some unit tests require some go-search-replace executable binary files to run. Binaries files for
+several OS architectures can be downloaded
+from https://github.com/Automattic/go-search-replace/releases/
+
+If, for some reason, you need to compile these binaries yourself, please follow instructions
+at https://github.com/Automattic/go-search-replace
+
+### Generating the types
+
+If you're an employee of Automattic, you can follow these steps to regenerate the GraphQL types
+used.
 
-If, for some reason, you need to compile these binaries yourself, please follow instructions at https://github.com/Automattic/go-search-replace
+1. Get a hold of `schema.gql` and paste it in project root - this is the schema of the endpoint that
+   we communicate with.
+2. Run `npm run typescript:codegen:install-dependencies` - this will install the codegen
+   dependencies without updating `package.json`
+3. Run `npm run typescript:codegen:generate` - this will regenerate the types

package/dist/commands/backup-db.js

@@ -34,8 +34,6 @@ const CREATE_DB_BACKUP_JOB_MUTATION = (0, _graphqlTag.default)`
 		}
 	}
 `;
-
-// TODO: Replace this with the codegen
 exports.CREATE_DB_BACKUP_JOB_MUTATION = CREATE_DB_BACKUP_JOB_MUTATION;
 const DB_BACKUP_JOB_STATUS_QUERY = (0, _graphqlTag.default)`
 	query AppBackupJobStatus($appId: Int!, $envId: Int!) {
@@ -63,6 +61,7 @@ const DB_BACKUP_JOB_STATUS_QUERY = (0, _graphqlTag.default)`
 `;
 exports.DB_BACKUP_JOB_STATUS_QUERY = DB_BACKUP_JOB_STATUS_QUERY;
 async function getBackupJob(appId, envId) {
+  var _app$environments, _app$environments$, _app$environments$$jo;
   const api = await (0, _api.default)();
   const response = await api.query({
     query: DB_BACKUP_JOB_STATUS_QUERY,
@@ -74,13 +73,10 @@ async function getBackupJob(appId, envId) {
   });
   const {
     data: {
-      app: {
-        environments
-      }
+      app
     }
   } = response;
-  const job = environments[0].jobs[0];
-  return job || null;
+  return app === null || app === void 0 ? void 0 : (_app$environments = app.environments) === null || _app$environments === void 0 ? void 0 : (_app$environments$ = _app$environments[0]) === null || _app$environments$ === void 0 ? void 0 : (_app$environments$$jo = _app$environments$.jobs) === null || _app$environments$$jo === void 0 ? void 0 : _app$environments$$jo[0];
 }
 async function createBackupJob(appId, envId) {
   // Disable global error handling so that we can handle errors ourselves
@@ -138,10 +134,10 @@ class BackupDBCommand {
     this.progressTracker.stopPrinting();
   }
   async loadBackupJob() {
-    var _this$job, _this$job$metadata$fi, _this$job2, _this$job2$progress, _this$job3;
-    this.job = await getBackupJob(this.app.id, this.env.id);
-    this.backupName = ((_this$job = this.job) === null || _this$job === void 0 ? void 0 : (_this$job$metadata$fi = _this$job.metadata.find(meta => meta.name === 'backupName')) === null || _this$job$metadata$fi === void 0 ? void 0 : _this$job$metadata$fi.value) || 'Unknown';
-    this.jobStatus = (_this$job2 = this.job) === null || _this$job2 === void 0 ? void 0 : (_this$job2$progress = _this$job2.progress) === null || _this$job2$progress === void 0 ? void 0 : _this$job2$progress.status;
+    var _this$app$id, _this$env$id, _this$job$metadata$fi, _this$job, _this$job$metadata, _this$job$metadata$fi2, _this$job$progress$st, _this$job2, _this$job2$progress, _this$job3;
+    this.job = await getBackupJob((_this$app$id = this.app.id) !== null && _this$app$id !== void 0 ? _this$app$id : 0, (_this$env$id = this.env.id) !== null && _this$env$id !== void 0 ? _this$env$id : 0);
+    this.backupName = (_this$job$metadata$fi = (_this$job = this.job) === null || _this$job === void 0 ? void 0 : (_this$job$metadata = _this$job.metadata) === null || _this$job$metadata === void 0 ? void 0 : (_this$job$metadata$fi2 = _this$job$metadata.find(meta => (meta === null || meta === void 0 ? void 0 : meta.name) === 'backupName')) === null || _this$job$metadata$fi2 === void 0 ? void 0 : _this$job$metadata$fi2.value) !== null && _this$job$metadata$fi !== void 0 ? _this$job$metadata$fi : 'Unknown';
+    this.jobStatus = (_this$job$progress$st = (_this$job2 = this.job) === null || _this$job2 === void 0 ? void 0 : (_this$job2$progress = _this$job2.progress) === null || _this$job2$progress === void 0 ? void 0 : _this$job2$progress.status) !== null && _this$job$progress$st !== void 0 ? _this$job$progress$st : undefined;
     if ((_this$job3 = this.job) !== null && _this$job3 !== void 0 && _this$job3.completedAt) {
       this.jobAge = (new Date().getTime() - new Date(this.job.completedAt).getTime()) / 1000 / 60;
     } else {
@@ -161,49 +157,53 @@ class BackupDBCommand {
       this.log('Database backup already in progress...');
     } else {
       try {
+        var _this$app$id2, _this$env$id2;
         this.log('Generating a new database backup...');
         this.progressTracker.stepRunning(this.steps.PREPARE);
         this.progressTracker.startPrinting();
-        await createBackupJob(this.app.id, this.env.id);
-      } catch (e) {
-        var _err$message;
-        const err = e;
+        await createBackupJob((_this$app$id2 = this.app.id) !== null && _this$app$id2 !== void 0 ? _this$app$id2 : 0, (_this$env$id2 = this.env.id) !== null && _this$env$id2 !== void 0 ? _this$env$id2 : 0);
+      } catch (err) {
+        var _error$graphQLErrors, _graphQLError$extensi;
+        const error = err;
+        const graphQLError = (_error$graphQLErrors = error.graphQLErrors) === null || _error$graphQLErrors === void 0 ? void 0 : _error$graphQLErrors[0];
         this.progressTracker.stepFailed(this.steps.PREPARE);
         this.stopProgressTracker();
-        if ((_err$message = err.message) !== null && _err$message !== void 0 && _err$message.includes('Database backups limit reached')) {
+        if ((graphQLError === null || graphQLError === void 0 ? void 0 : (_graphQLError$extensi = graphQLError.extensions) === null || _graphQLError$extensi === void 0 ? void 0 : _graphQLError$extensi.errorHttpCode) === 429) {
+          const retryAfter = graphQLError.extensions.retryAfter;
           await this.track('error', {
             error_type: 'rate_limit_exceeded',
-            error_message: `Couldn't create a new database backup job: ${err === null || err === void 0 ? void 0 : err.message}`,
-            stack: err === null || err === void 0 ? void 0 : err.stack
+            error_message: `Couldn't create a new database backup job: ${error.message}`,
+            stack: error.stack
           });
-          let errMessage = err.message.replace('Database backups limit reached', 'A new database backup was not generated because a recently generated backup already exists.');
-          errMessage = errMessage.replace('Retry after', '\nIf you would like to run the same command, you can retry on or after:');
-          errMessage += `\nAlternatively, you can export the latest existing database backup by running: ${_chalk.default.green('vip @app.env export sql')}, right away.`;
-          errMessage += '\nLearn more about limitations around generating database backups: https://docs.wpvip.com/technical-references/vip-dashboard/backups/#0-limitations \n';
+          const errMessage = `A new database backup was not generated because a recently generated backup already exists.
+If you would like to run the same command, you can retry on or after: ${retryAfter}
+Alternatively, you can export the latest existing database backup by running: ${_chalk.default.green('vip @app.env export sql')}, right away.
+Learn more about limitations around generating database backups: https://docs.wpvip.com/technical-references/vip-dashboard/backups/#0-limitations
+					`;
           exit.withError(errMessage);
         }
         await this.track('error', {
           error_type: 'db_backup_job_creation_failed',
-          error_message: `Database Backup job creation failed: ${err === null || err === void 0 ? void 0 : err.message}`,
-          stack: err === null || err === void 0 ? void 0 : err.stack
+          error_message: `Database Backup job creation failed: ${error.message}`,
+          stack: error.stack
         });
-        exit.withError(`Couldn't create a new database backup job: ${err === null || err === void 0 ? void 0 : err.message}`);
+        exit.withError(`Couldn't create a new database backup job: ${error.message}`);
       }
     }
     this.progressTracker.stepSuccess(this.steps.PREPARE);
     this.progressTracker.stepRunning(this.steps.GENERATE);
     try {
       await (0, _utils.pollUntil)(this.loadBackupJob.bind(this), DB_BACKUP_PROGRESS_POLL_INTERVAL, this.isDone.bind(this));
-    } catch (e) {
-      const err = e;
+    } catch (err) {
+      const error = err;
       this.progressTracker.stepFailed(this.steps.GENERATE);
       this.stopProgressTracker();
       await this.track('error', {
         error_type: 'db_backup_job_failed',
-        error_message: `Database Backup job failed: ${err === null || err === void 0 ? void 0 : err.message}`,
-        stack: err === null || err === void 0 ? void 0 : err.stack
+        error_message: `Database Backup job failed: ${error.message}`,
+        stack: error.stack
       });
-      exit.withError(`Failed to create new database backup: ${err === null || err === void 0 ? void 0 : err.message}`);
+      exit.withError(`Failed to create new database backup: ${error.message}`);
     }
     this.progressTracker.stepSuccess(this.steps.GENERATE);
     this.stopProgressTracker();

package/dist/lib/api/http.js

@@ -51,6 +51,7 @@ var _default = async (path, options = {}) => {
       'Content-Type': 'application/json',
       ...((_options$headers = options.headers) !== null && _options$headers !== void 0 ? _options$headers : {})
     },
+    // eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
     body: typeof options.body === 'object' ? JSON.stringify(options.body) : options.body
   });
 };

package/dist/lib/types/graphql/rate-limit-exceeded-error.js

@@ -0,0 +1 @@
+"use strict";

package/helpers/generate-docs.js

@@ -0,0 +1,151 @@
+const { spawn } = require( 'child_process' );
+const { once } = require( 'events' );
+
+async function runCommand( subcommands ) {
+	const childProcess = spawn( 'vip', subcommands );
+
+	let output = '';
+
+	for await ( const data of childProcess.stdout ) {
+		output += data.toString();
+	}
+	for await ( const data of childProcess.stderr ) {
+		output += data.toString();
+	}
+
+	const [ exitCode ] = await once( childProcess, 'exit' );
+
+	if ( exitCode !== 0 ) {
+		console.log( 'o', output );
+		throw new Error( `Script exited with code ${ exitCode }` );
+	}
+
+	return output.trim();
+}
+
+const USAGE_REGEXP = /Usage: (.*)/;
+const COMMAND_REGEXP = /(\S+)\s+(.*)/;
+const OPTION_REGEXP = /(-\S, --\S+)\s+(.*)/;
+
+const SECTION_COMMAND = 'commands';
+const SECTION_OPTIONS = 'options';
+const SECTION_EXAMPLES = 'examples';
+
+const parseOutput = output => {
+	const result = {};
+
+	const lines = output.split( '\n' );
+	let currentSection = '';
+
+	for ( let lineIx = 0; lineIx < lines.length; lineIx++ ) {
+		const line = lines[ lineIx ].trim();
+		if ( ! line ) {
+			continue;
+		}
+		if ( line.startsWith( 'Usage:' ) ) {
+			result.usage = line.match( USAGE_REGEXP )[ 1 ];
+			continue;
+		}
+
+		if ( line.startsWith( 'Commands:' ) ) {
+			result.commands = [];
+			currentSection = SECTION_COMMAND;
+			continue;
+		}
+		if ( line.startsWith( 'Options:' ) ) {
+			result.options = [];
+			currentSection = SECTION_OPTIONS;
+			continue;
+		}
+		if ( line.startsWith( 'Examples:' ) ) {
+			result.examples = [];
+			result.examplesRaw = '';
+			currentSection = SECTION_EXAMPLES;
+			continue;
+		}
+
+		if ( currentSection === SECTION_COMMAND ) {
+			const [ , command, description ] = line.match( COMMAND_REGEXP );
+			result.commands.push( {
+				command,
+				description,
+			} );
+			continue;
+		}
+		if ( currentSection === SECTION_OPTIONS ) {
+			if ( line.match( OPTION_REGEXP ) ) {
+				const [ , option, description ] = line.match( OPTION_REGEXP );
+				result.options.push( {
+					option,
+					description,
+				} );
+			} else {
+				console.log( 'Unknown option', line );
+			}
+			continue;
+		}
+		if ( currentSection === SECTION_EXAMPLES ) {
+			let description = '';
+			while ( ! lines[ lineIx ].trim().startsWith( '$' ) ) {
+				const descriptionLine = lines[ lineIx ].trim();
+				if ( description ) {
+					description += '\n';
+				}
+				if ( result.examplesRaw ) {
+					result.examplesRaw += '\n';
+				}
+				result.examplesRaw += descriptionLine;
+				description += descriptionLine;
+				lineIx++;
+			}
+			const usage = lines[ lineIx ] && lines[ lineIx ].trim();
+			result.examplesRaw += '\n' + usage;
+			result.examples.push( {
+				description,
+				usage,
+			} );
+		}
+	}
+
+	return result;
+};
+
+const processCommand = async subcommands => {
+	const fullCommand = subcommands.join( ' ' );
+	console.error( 'Processing', fullCommand, '...' );
+
+	const output = await runCommand( subcommands.concat( [ '--help' ] ) );
+	const parsedOutput = parseOutput( output );
+
+	const commandCount = parsedOutput.commands?.length || 0;
+	const commandOutputs = [];
+	for ( let commandIx = 0; commandIx < commandCount; commandIx++ ) {
+		const element = parsedOutput.commands[ commandIx ];
+		// otherwise the parallel run will randomly fail on too many requests
+		// eslint-disable-next-line no-await-in-loop
+		commandOutputs.push( await processCommand( subcommands.concat( [ element.command ] ) ) );
+	}
+	console.error( `Processed ${ fullCommand } -> ${ commandOutputs.length } subcommands` );
+	for ( let commandIx = 0; commandIx < commandCount; commandIx++ ) {
+		const element = parsedOutput.commands[ commandIx ];
+		const commandOutput = commandOutputs[ commandIx ];
+		commandOutput.name = element.command;
+		commandOutput.description = element.description;
+		parsedOutput.commands[ commandIx ] = commandOutput;
+	}
+
+	return parsedOutput;
+};
+
+const main = async () => {
+	const version = await runCommand( [ '--version' ] );
+
+	console.error( 'triggering command processing...' );
+	const result = await processCommand( [] );
+	console.error( 'command processing done' );
+	result.version = version;
+
+	console.log( JSON.stringify( result, null, 2 ) );
+};
+
+main();