libdragon 10.2.1 → 10.3.1

package/CHANGELOG.md

@@ -1,5 +1,48 @@
 # Change Log
 
+## [10.3.1] - 2022-01-25
+
+### Fixed
+
+- Do not try to parse arguments after exec/make. They used to get evaluated as
+  libdragon paramaters previously, preventing passing down -v to the container
+  make for example.
+- Docker image update issues
+  - Attempt an image update whenever the image flag is provided. Previously this
+    was only done if a different image name is provided, preventing the update of
+    the latest image. Previously not providing an image name was behaving the same
+    so this is not a breaking change.
+  - Start action used to update the image if provided but this bug was already
+    prevented by previous fixes by not accepting the image flag for actions other
+    than init/update/install. Start action no longer tries to update the image
+    regardless.
+
+### Changed
+
+- Only accept he image flag for init, install, and update actions as documented.
+- Improve documentation for the `init` and `install` actions.
+- Do not attempt an `install` when running `exec`, just start the container. If
+  there is a half-baked container, a manual `install` will potentially restore it.
+- Update libdragon.
+
+### Added
+
+- Extra information for skipping the image flag when doing init for an already
+  initialized project.
+
+## [10.3.0] - 2022-01-20
+
+### Changed
+
+- Update dependencies.
+- Detailed help output.
+- Move action descriptions to `libdragon help`.
+- Update libdragon to latest version for local build.
+
+### Added
+
+- Shorthand flag support.
+
 ## [10.2.1] - 2021-10-14
 
 ### Changed
@@ -16,42 +59,42 @@
 
 - Container discovery. The tool can now find a container even if `.git` is lost.
 - A few additional error messages for some potentially confusing cases such as
-already having a file with `libdragon` like name.
+  already having a file with `libdragon` like name.
 
 ### Changed
 
 - Show more output for downloading the container and initial git operations.
 - Remove an extra log during initialization.
 - The submodule was always being initialized when a container is started. This
-was making some actions inconsistent. For example `install` or `make` action
-was trying to re-initialize the submodule unnecessarily. This is fixed by only
-initializing it with the `init` action. If any of those need to re-init the
-container, now they assume there is an intact libdragon folder to use.
+  was making some actions inconsistent. For example `install` or `make` action
+  was trying to re-initialize the submodule unnecessarily. This is fixed by only
+  initializing it with the `init` action. If any of those need to re-init the
+  container, now they assume there is an intact libdragon folder to use.
 - Similarly a git repository is not initialized unnecessarily anymore.
 - `update` and `install` are now able to start containers if necessary.
 - Always try to copy skeleton files, they won't overwrite anything already.
 - Do not re-initialize if there is a `.libdragon` folder. We now only try to
-start it in this case. If it is not a complete container, it can probably be
-recovered by a `libdragon install` or `libdragon update`.
+  start it in this case. If it is not a complete container, it can probably be
+  recovered by a `libdragon install` or `libdragon update`.
 
 ### Fixed
 
 - Fix wording for libdragon install on the container.
 - Improve image name persitence such that the tool finds and updates it more
-consistently.
+  consistently.
 
 ## [10.1.0] - 2021-10-07
 
 ### Added
 
 - `exec` action to execute arbitrary commands in the container with TTY support.
-This also improves the color output support as docker now knows it is using TTY.
+  This also improves the color output support as docker now knows it is using TTY.
 - Add more verbose logging for skeleton project copy.
 
 ### Changed
 
 - Removed partially not working `--byte-swap` option. It does not already work
-with the new libdragon build system so there is no need keeping it in the tool.
+  with the new libdragon build system so there is no need keeping it in the tool.
 
 ### Fixed
 
@@ -62,14 +105,14 @@ with the new libdragon build system so there is no need keeping it in the tool.
 ### Changed
 
 - A complete re-write of the tool. Check documentation for the new usage. It is
-much more straightforward to use now. `libdragon make` behaves almost the same.
+  much more straightforward to use now. `libdragon make` behaves almost the same.
 
 ## [9.0.0] - 2021-09-06
 
 ### Changed
 
 - Updated libdragon. We will be changing the update mechanism to be based on a
-git pull, so listing them here will not be useful anymore, let's not bother now.
+  git pull, so listing them here will not be useful anymore, let's not bother now.
 - Start using the new build system.
 - Update node engine spec to be at least 14.
 - Added minimum docker version to readme.
@@ -119,7 +162,6 @@ git pull, so listing them here will not be useful anymore, let's not bother now.
 - Support for RTC status/read/write commands (https://github.com/DragonMinded/libdragon/pull/152)
 - Generic libdragon NPM script
 
-
 ## [7.0.0] - 2021-06-06
 
 ### Changed

package/README.md

@@ -36,53 +36,15 @@ You can invoke libdragon as follows;
 
     libdragon [flags] <action>
 
-### Available actions
-
-__`init`__
-
-Creates a libdragon project in the current directory. Every libdragon project will have its own docker container instance. If you are in a git repository or an NPM project, libdragon will be initialized at their root also marking there with a `.libdragon` folder.
-
-A git repository and a submodule at `./libdragon` will also be created. Do not remove the `.libdragon` folder and commit its contents if you are using git, as it keeps persistent libdragon project information.
-
-If this is the first time you are creating a libdragon project at that location, this action will also create skeleton project files to kickstart things.
-
-__`make`__
-
-Runs the libdragon build system in the current directory. It will mirror your current working directory to the container.
-
-This action is a shortcut to the `exec` action under the hood.
-
-__`exec`__
-
-Executes the given command in the container passing down any arguments provided. If you change your host working directory, the command will be executed in the corresponding folder in the container as well.
-
-This action will first try to execute the command in the container and if the container is not accessible, it will attempt a complete `start` cycle.
-
-This will properly passthrough your TTY if you have one. So by running `libdragon exec bash` you can start an interactive bash session with full TTY support.
-
-__`install`__
-
-Attempts to build and install everything libdragon related into the container. This includes all the tools and third parties used by libdragon except for the toolchain. If you have made changes to libdragon, you can execute this action to build everything based on your changes. Requires you to have an intact `libdragon` at the root of the project. If you are not working on libdragon, you can just use the `update` action instead.
-
-__`update`__
-
-This action will update the submodule from the remote branch (`trunk`) with a merge strategy and then perform a `libdragon install`. You can use the `install` action to only update all libdragon related artifacts in the container.
-
-__`start`__
-
-Start the container assigned to the current libdragon project. Will first attempt to start an existing container if found, followed by a new container run and installation similar to the `install` action. Will always print out the container id on success.
-
-__`stop`__
-
-Stop the container assigned to the current libdragon project.
+Run `libdragon help [action]` for more details on individual actions.
 
 ### Available flags
 
-__`--image <docker-image>`__
+**`--image <docker-image>`**
 
 Use this flag to provide a custom image to use instead of the default. It should include the toolchain at `/n64_toolchain`. It will be effective for `init`, `install` and `update` actions and will cause a re-initialization of the container if an image different from what was written to project configuration is provided.
 
-__`--verbose`__
+**`--verbose`**
 
 Be verbose. This will print all commands dispatched and their outputs as well.
 

package/index.js

@@ -3,86 +3,93 @@
 const chalk = require('chalk');
 const { readProjectInfo, CommandError, globals } = require('./modules/helpers');
 const actions = require('./modules/actions');
+const { printUsage } = require('./modules/usage');
 
 const STATUS_OK = 0;
 const STATUS_ERROR = 1;
 const STATUS_BAD_PARAM = 2;
 
-// Command line options
-const options = {
-  DOCKER_IMAGE: undefined,
-  VERBOSE: false,
-
-  ACTION: undefined,
-  PARAMS: undefined,
-};
-
-// Allow standard io here
-/* eslint-disable no-console */
+let options = {},
+  currentAction;
 
 for (let i = 2; i < process.argv.length; i++) {
   const val = process.argv[i];
 
-  if (val === '--verbose') {
+  // Allow console here
+  /* eslint-disable no-console */
+
+  if (['--verbose', '-v'].includes(val)) {
     options.VERBOSE = true;
     globals.verbose = true;
     continue;
   }
 
-  if (val === '--image') {
+  if (['--image', '-i'].includes(val)) {
     options.DOCKER_IMAGE = process.argv[++i];
     continue;
+  } else if (val.indexOf('--image=') === 0) {
+    options.DOCKER_IMAGE = val.split('=')[1];
+    continue;
   }
 
   if (val.indexOf('--') >= 0) {
     console.error(chalk.red(`Invalid flag \`${val}\``));
-    actions.help.fn();
+    printUsage();
     process.exit(STATUS_BAD_PARAM);
   }
 
-  if (options.ACTION) {
+  if (currentAction) {
     console.error(
       chalk.red(`Expected only a single action, found: \`${val}\``)
     );
-    actions.help.fn();
+    printUsage();
     process.exit(STATUS_BAD_PARAM);
   }
 
-  options.ACTION = actions[val];
+  currentAction = actions[val];
 
-  if (!options.ACTION) {
+  if (!currentAction) {
     console.error(chalk.red(`Invalid action \`${val}\``));
-    actions.help.fn();
+    printUsage();
     process.exit(STATUS_BAD_PARAM);
   }
 
-  if (options.ACTION.forwardsRestParams) {
+  if (currentAction.forwardsRestParams) {
     options.PARAMS = process.argv.slice(i + 1);
     break;
   }
 }
 
-if (!options.ACTION) {
+if (!currentAction) {
   console.error(chalk.red('No action provided'));
-  actions.help.fn();
+  printUsage();
   process.exit(STATUS_BAD_PARAM);
 }
 
-if (options.ACTION === actions.exec && options.PARAMS.length === 0) {
+if (currentAction === actions.exec && options.PARAMS.length === 0) {
   console.error(chalk.red('You should provide a command to exec'));
-  actions.help.fn();
+  printUsage(undefined, [currentAction.name]);
+  process.exit(STATUS_BAD_PARAM);
+}
+
+if (
+  ![actions.init, actions.install, actions.update].includes(currentAction) &&
+  options.DOCKER_IMAGE
+) {
+  console.error(chalk.red('Invalid flag: image'));
+  printUsage(undefined, [currentAction.name]);
   process.exit(STATUS_BAD_PARAM);
 }
 
 readProjectInfo()
   .then((info) =>
-    options.ACTION.fn(
+    currentAction.fn(
       {
         ...info,
         options,
-        ...options.ACTION,
+        ...currentAction,
       },
-      options.PARAMS
+      options.PARAMS ?? []
     )
   )
   .catch((e) => {

package/modules/actions.js

@@ -2,6 +2,7 @@ const fs = require('fs');
 const path = require('path');
 const chalk = require('chalk');
 const _ = require('lodash');
+
 const {
   LIBDRAGON_SUBMODULE,
   LIBDRAGON_BRANCH,
@@ -9,6 +10,7 @@ const {
   CONTAINER_TARGET_PATH,
   LIBDRAGON_PROJECT_MANIFEST,
 } = require('./constants');
+
 const {
   spawnProcess,
   checkContainerAndClean,
@@ -27,6 +29,8 @@ const {
   tryCacheContainerId,
 } = require('./helpers');
 
+const { printUsage } = require('./usage');
+
 const destroyContainer = async (libdragonInfo) => {
   if (libdragonInfo.containerId) {
     await spawnProcess('docker', [
@@ -60,27 +64,60 @@ const initSubmodule = async (libdragonInfo) => {
 };
 
 /**
- * Will download image and create a new container
+ * Downloads the given docker image. Returns false if the local image is the
+ * same, new image name otherwise.
+ * @param libdragonInfo
+ * @param newImageName
+ * @returns false | string
  */
-const initContainer = async (libdragonInfo) => {
-  let newId;
-  try {
-    const imageName =
-      libdragonInfo.imageName ?? libdragonInfo.options.DOCKER_IMAGE;
-    await updateImageName({
-      ...libdragonInfo,
-      imageName,
-    });
+const updateImage = async (libdragonInfo, newImageName) => {
+  // Will not take too much time if already have the same
+  const download = async () => {
+    libdragonInfo.showStatus &&
+      log(`Downloading docker image: ${newImageName}`);
+    await spawnProcess(
+      'docker',
+      ['pull', newImageName],
+      false,
+      libdragonInfo.showStatus
+    );
+  };
 
-    // Download image
-    libdragonInfo.showStatus && log(`Downloading docker image: ${imageName}`);
+  const getDigest = async () =>
     await spawnProcess(
       'docker',
-      ['pull', imageName],
+      ['images', '-q', '--no-trunc', newImageName],
       false,
       libdragonInfo.showStatus
     );
 
+  // Attempt to compare digests if the new image name is the same
+  // Even if they are not the same tag, it is possible to have a different
+  // image but we already attempt a download in any case. It would just take
+  // less time as we already have the layers.
+  if (libdragonInfo.imageName === newImageName) {
+    const existingDigest = await getDigest();
+    await download();
+    const newDigest = await getDigest();
+
+    if (existingDigest === newDigest) {
+      libdragonInfo.showStatus && log(`Image is the same: ${newImageName}`);
+      return false;
+    }
+  } else {
+    await download();
+  }
+
+  libdragonInfo.showStatus && log(`Image is different: ${newImageName}`);
+  return newImageName;
+};
+
+/**
+ * Create a new container
+ */
+const initContainer = async (libdragonInfo) => {
+  let newId;
+  try {
     // Create a new container
     libdragonInfo.showStatus && log('Creating new container...');
     newId = (
@@ -93,7 +130,7 @@ const initContainer = async (libdragonInfo) => {
           ',target=' +
           CONTAINER_TARGET_PATH, // Mount files
         '-w=' + CONTAINER_TARGET_PATH, // Set working directory
-        imageName,
+        libdragonInfo.imageName,
         'tail',
         '-f',
         '/dev/null',
@@ -103,7 +140,6 @@ const initContainer = async (libdragonInfo) => {
     const newInfo = {
       ...libdragonInfo,
       containerId: newId,
-      imageName,
     };
 
     // chown the installation folder once on init
@@ -140,6 +176,9 @@ const initContainer = async (libdragonInfo) => {
     log(
       chalk.green(`Successfully initialized docker container: ${name.trim()}`)
     );
+
+  // Update the image name only after everything is OK
+  await updateImageName(libdragonInfo);
   return newId;
 };
 
@@ -183,6 +222,7 @@ function copyDirContents(src, dst) {
 
 /**
  * Initialize a new libdragon project in current working directory
+ * Also downloads the image
  */
 async function init(libdragonInfo) {
   log(`Initializing a libdragon project at ${libdragonInfo.root}`);
@@ -198,9 +238,14 @@ async function init(libdragonInfo) {
       `${path.join(
         libdragonInfo.root,
         manifestFile
-      )} exists. This is probably already a libdragon project, starting it...`
+      )} exists. This is already a libdragon project, starting it...`
     );
-    await startAndInstall(libdragonInfo);
+    if (libdragonInfo.options.DOCKER_IMAGE) {
+      log(
+        `Not changing docker image. Use the install action if you want to override the image.`
+      );
+    }
+    await install(libdragonInfo);
     return;
   }
 
@@ -218,7 +263,13 @@ async function init(libdragonInfo) {
   }
 
   await createManifestIfNotExist(libdragonInfo);
-  const newId = await start(libdragonInfo);
+  // Download image and start it
+  const newId = await start({
+    ...libdragonInfo,
+    imageName:
+      (await updateImage(libdragonInfo, libdragonInfo.imageName)) ||
+      libdragonInfo.imageName,
+  });
   const newInfo = {
     ...libdragonInfo,
     containerId: newId,
@@ -301,7 +352,7 @@ const exec = async (libdragonInfo, commandAndParams) => {
   let started = false;
   const startOnceAndCmd = async () => {
     if (!started) {
-      const newId = await startAndInstall(libdragonInfo);
+      const newId = await start(libdragonInfo);
       started = true;
       await tryCmd({
         ...libdragonInfo,
@@ -440,17 +491,6 @@ const installDependencies = async (libdragonInfo) => {
   }
 };
 
-async function startAndInstall(libdragonInfo) {
-  const containerId = await start(libdragonInfo);
-
-  // TODO: skip if unnecessary
-  await installDependencies({
-    ...libdragonInfo,
-    containerId,
-  });
-  return containerId;
-}
-
 const update = async (libdragonInfo) => {
   const containerId = await start(libdragonInfo);
 
@@ -484,13 +524,26 @@ const update = async (libdragonInfo) => {
   await install(newInfo);
 };
 
+/**
+ * Updates the image if flag is provided and install vendors onto the container.
+ * We should probably remove the image installation responsibility from this
+ * action but it might be a breaking change. Maybe we can keep it backward
+ * compatible with additional flags.
+ * @param libdragonInfo
+ */
 const install = async (libdragonInfo) => {
   let containerId;
   const oldImageName = libdragonInfo.imageName;
-  const imageName = libdragonInfo.options.DOCKER_IMAGE ?? oldImageName;
-  if (oldImageName !== imageName) {
+  const imageName = libdragonInfo.options.DOCKER_IMAGE;
+  // If an image is provided, always attempt to install it
+  // See https://github.com/anacierdem/libdragon-docker/issues/47
+  if (imageName) {
     log(`Changing image from \`${oldImageName}\` to \`${imageName}\``);
-    await destroyContainer(libdragonInfo);
+
+    // Download the new image and if it is different, re-create the container
+    if (await updateImage(libdragonInfo, imageName)) {
+      await destroyContainer(libdragonInfo);
+    }
 
     containerId = await start({
       ...libdragonInfo,
@@ -502,6 +555,7 @@ const install = async (libdragonInfo) => {
   }
 
   // Re-install vendors on new image
+  // TODO: skip this if unnecessary
   await installDependencies({
     ...libdragonInfo,
     imageName,
@@ -509,59 +563,49 @@ const install = async (libdragonInfo) => {
   });
 };
 
-const help = () => {
-  log(`${chalk.green('Usage:')}
-  libdragon [flags] <action>
-
-${chalk.green('Available Commands:')}
-  help        Display this help information.
-  init        Create a libdragon project in the current directory.
-  make        Run the libdragon build system in the current directory.
-  exec        Execute given command in the current directory.
-  start       Start the container for current project.
-  stop        Stop the container for current project.
-  install     Vendor libdragon as is.
-  update      Update libdragon and do an install.
-
-${chalk.green('Flags:')}
-  --image <docker-image>  Provide a custom image.
-  --verbose               Be verbose
-`);
-};
-
+// TODO: separate into files
 module.exports = {
   start: {
+    name: 'start',
     fn: start,
     showStatus: false, // This will only print out the id
   },
   init: {
+    name: 'init',
     fn: init,
     showStatus: true,
   },
   make: {
+    name: 'make',
     fn: make,
     forwardsRestParams: true,
     showStatus: true,
   },
   exec: {
+    name: 'exec',
     fn: exec,
     forwardsRestParams: true,
     showStatus: true,
   },
   stop: {
+    name: 'stop',
     fn: stop,
     showStatus: false, // This will only print out the id
   },
   install: {
+    name: 'install',
     fn: install,
     showStatus: true,
   },
   update: {
+    name: 'update',
     fn: update,
     showStatus: true,
   },
   help: {
-    fn: help,
+    name: 'help',
+    fn: printUsage,
     showStatus: true,
+    forwardsRestParams: true,
   },
 };

package/modules/usage.js

@@ -0,0 +1,129 @@
+const chalk = require('chalk');
+const commandLineUsage = require('command-line-usage');
+
+const { log } = require('./helpers');
+
+const printUsage = (_, actionArr) => {
+  const globalOptionDefinitions = [
+    {
+      name: 'verbose',
+      description: 'Be verbose',
+      alias: 'v',
+      group: 'global',
+    },
+  ];
+
+  const optionDefinitions = [
+    {
+      name: 'image',
+      description: 'Provide a custom image.',
+      alias: 'i',
+      typeLabel: '<docker-image>',
+      group: 'docker',
+    },
+  ];
+
+  const actions = {
+    help: {
+      name: 'help [action]',
+      summary: 'Display this help information or details for the given action.',
+    },
+    init: {
+      name: 'init',
+      summary: 'Create a libdragon project in the current directory.',
+      description: `Creates a libdragon project in the current directory. Every libdragon project will have its own docker container instance. If you are in a git repository or an NPM project, libdragon will be initialized at their root also marking there with a \`.libdragon\` folder.
+
+      A git repository and a submodule at \`./libdragon\` will also be created. Do not remove the \`.libdragon\` folder and commit its contents if you are using git, as it keeps persistent libdragon project information.
+
+      If this is the first time you are creating a libdragon project at that location, this action will also create skeleton project files to kickstart things with the given image, if provided. For subsequent runs, it will act like \`start\` thus can be used to revive an existing project without modifying it.`,
+      group: ['docker'],
+    },
+    make: {
+      name: 'make [params]',
+      summary: 'Run the libdragon build system in the current directory.',
+      description: `Runs the libdragon build system in the current directory. It will mirror your current working directory to the container.
+
+      This action is a shortcut to the \`exec\` action under the hood.`,
+    },
+    exec: {
+      name: 'exec <command>',
+      summary: 'Execute given command in the current directory.',
+      description: `Executes the given command in the container passing down any arguments provided. If you change your host working directory, the command will be executed in the corresponding folder in the container as well.
+
+      This action will first try to execute the command in the container and if the container is not accessible, it will attempt a complete \`start\` cycle.
+
+      This will properly passthrough your TTY if you have one. So by running \`libdragon exec bash\` you can start an interactive bash session with full TTY support.`,
+    },
+    start: {
+      name: 'start',
+      summary: 'Start the container for current project.',
+      description:
+        'Start the container assigned to the current libdragon project. Will first attempt to start an existing container if found, followed by a new container run and installation similar to the `install` action. Will always print out the container id on success.',
+    },
+    name: {
+      name: 'stop',
+      summary: 'Stop the container for current project.',
+      description:
+        'Stop the container assigned to the current libdragon project.',
+    },
+    install: {
+      name: 'install',
+      summary: 'Vendor libdragon as is.',
+      group: ['docker'],
+      description: `Attempts to build and install everything libdragon related into the container. This includes all the tools and third parties used by libdragon except for the toolchain. If you have made changes to libdragon, you can execute this action to build everything based on your changes. Requires you to have an intact \`libdragon\` at the root of the project. If you are not working on libdragon, you can just use the \`update\` action instead.
+
+        This can be useful to recover from a half-baked container.`,
+    },
+    update: {
+      name: 'update',
+      summary: 'Update libdragon and do an install.',
+      description:
+        'This action will update the submodule from the remote branch (`trunk`) with a merge strategy and then perform a `libdragon install`. You can use the `install` action to only update all libdragon related artifacts in the container.',
+      group: ['docker'],
+    },
+  };
+
+  const actionsToShow = actionArr
+    ?.filter((action) => Object.keys(actions).includes(action))
+    .filter((action) => !['help'].includes(action));
+
+  const sections = [
+    {
+      header: chalk.green('Usage:'),
+      content: 'libdragon [flags] <action>',
+    },
+    ...(actionsToShow?.length
+      ? actionsToShow.flatMap((action) => [
+          {
+            header: chalk.green(`${action} action:`),
+            content: actions[action].description,
+          },
+          actions[action].group
+            ? {
+                header: `accepted flags:`,
+                optionList: optionDefinitions,
+                group: actions[action].group,
+              }
+            : {},
+        ])
+      : [
+          {
+            header: chalk.green('Available Commands:'),
+            content: Object.values(actions).map((action) => ({
+              name: action.name,
+              summary: action.summary,
+            })),
+          },
+        ]),
+    {
+      header: chalk.green('Global flags:'),
+      optionList: globalOptionDefinitions,
+    },
+  ];
+  const usage = commandLineUsage(sections);
+  log(usage);
+};
+
+module.exports = {
+  printUsage,
+};

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "libdragon",
-  "version": "10.2.1",
+  "version": "10.3.1",
   "description": "This is a docker wrapper for libdragon",
   "main": "index.js",
   "engines": {
@@ -35,6 +35,7 @@
   "homepage": "https://github.com/anacierdem/libdragon-docker#readme",
   "dependencies": {
     "chalk": "^4.1.0",
+    "command-line-usage": "^6.1.1",
     "lodash": "^4.17.20"
   },
   "devDependencies": {