libdragon 10.8.0 → 10.8.2

package/README.md

@@ -1,12 +1,65 @@
 # Docker Libdragon
 
-[![Build](https://github.com/anacierdem/libdragon-docker/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/anacierdem/libdragon-docker/actions/workflows/ci.yml)
+[![Build](https://github.com/anacierdem/libdragon-docker/actions/workflows/release.yml/badge.svg?branch=master)](https://github.com/anacierdem/libdragon-docker/actions/workflows/release.yml)
 
-This is a wrapper for a docker container to make managing the libdragon toolchain easier.
+This is a wrapper for a docker container to make managing the libdragon toolchain easier. It has the additional advantage that libdragon toolchain and library can be installed on a per-project basis instead of managing system-wide installations.
 
-## Quick Install
+## Prerequisites
 
-On a system with [docker](https://www.docker.com/products/docker-desktop) (`>= 18`), grab a [pre-built executable](https://github.com/anacierdem/libdragon-docker/releases/latest) and put it somewhere on your PATH. Then navigate to the folder you want to initialize your project and invoke libdragon;
+You should have [docker](https://www.docker.com/products/docker-desktop) (`>= 18`) installed on your system.
+
+`git` is not strictly required to use the tool. Still, it is highly recommended to have git on your host machine as it will be used instead of the one in the container.
+
+## Installation
+
+This is primarily a node.js script which is also packaged as an executable. You have two options:
+
+### pre-built
+
+Download the [pre-built executable](https://github.com/anacierdem/libdragon-docker/releases/latest) for your system and put it somewhere on your PATH. It is discouraged to put it in your project folder.
+
+<details>
+  <summary>Windows instructions</summary>
+
+  - Download the Windows executable and copy it to `C:\bin`
+  - Press `Windows + R` key combination and then enter `rundll32 sysdm.cpl,EditEnvironmentVariables`
+  - In the `Environment Variables` window find the `Path` variable under `User variables for <your user name>`
+  - Double click it and add a new entry as `C:\bin`
+  - Restart your computer.
+  - You should now be able to use the `libdragon` on a command line or PowerShell.
+  - To update it with a new version, just replace the file in `C:\bin`
+
+</details>
+
+<details>
+  <summary>MacOS instructions</summary>
+
+  - Download the MacOS executable and copy it to `/usr/local/bin`
+  - Right click it and choose `Open`.
+  - It will show a warning, approve it by clicking `Open` again. You can close the newly opened terminal window.
+  - You should now be able to use the `libdragon` command.
+  - To update it with a new version, replace the file in `/usr/local/bin` and repeat the other steps.
+
+</details>
+
+<details>
+  <summary>Linux instructions</summary>
+
+  - You should already know this :)
+
+</details>
+
+### via NPM
+
+Install [node.js](https://nodejs.org/en/download/) (`>= 14`) and install `libdragon` as a global NPM package;
+
+    npm install -g libdragon
+
+To update the tool to the latest, do `npm i -g libdragon@latest`.
+
+## Quick Guide
+
+Navigate to the folder you want to initialize your project and invoke libdragon;
 
     libdragon init
 
@@ -22,21 +75,60 @@ To update the library and rebuild/install all the artifacts;
 
     libdragon update
 
-`git` is not strictly required to use this tool as docker's git will be used instead. Still, it is highly recommended to have git on your host machine.
+In general, you can invoke libdragon as follows;
 
-## Overall usage
+    libdragon [flags] <action>
 
-This is primarily a node.js script which is also packaged as an executable. If you have [node.js](https://nodejs.org/en/download/) (`>= 14`), you can do a global install and use it as usual. To install `libdragon` as a global NPM package;
+Run `libdragon help [action]` for more details on individual actions.
 
-    npm install -g libdragon
+## Recipes
 
-To update the tool to the latest, do `npm i -g libdragon@latest`.
+### Using a different branch of libdragon
 
-You can invoke libdragon as follows;
+Initialize your project as usual:
 
-    libdragon [flags] <action>
+    libdragon init
 
-Run `libdragon help [action]` for more details on individual actions.
+Then switch the submodule to the desired branch:
+
+    cd ./libdragon
+    git checkout opengl
+    cd ..
+    libdragon install
+
+If your changes are on a different remote, then you will need to manage your git remotes as usual.
+
+### Testing changes on libdragon
+
+As libdragon is an actively developed library, you may find yourself at a position where you want to change a few things on it and see how it works. In general, if you modify the files in `libdragon` folder of your project, you can install that version to the docker container by simply running:
+
+    libdragon install
+
+This will update all the artifacts in your container and your new code will start linking against the new version when you re-build it via `libdragon make`. The build system should pick up the change in the library and re-compile the dependent files.
+
+Instead of depending on the above command, you can re-build the library by making it a make dependency in your project:
+
+```makefile
+libdragon-install: libdragon
+	$(MAKE) -C ./libdragon install
+
+libdragon:
+	$(MAKE) -C ./libdragon
+```
+
+If your build now depends on `libdragon-install`, it will force an install (which should be pretty quick if you don't have changes) and force the build system to rebuild your project when necessary.
+
+If you clone this repository, this setup is pretty much ready for you. Make sure you have a working libdragon setup and you get the submodules (e.g `git submodule update --init`). Then you can run `libdragon make bench` to execute the code in `./src` with your library changes. Also see [test bench](#local-test-bench).
+
+When managing your changes to the library, you have a few options:
+
+#### **Using `submodule` vendor strategy.**
+
+To be able to share your project with the library change, you would need to push it somewhere public and make sure it is cloned properly by your contributors. This is not recommended for keeping track of your changes but is very useful if you plan to contribute it back to upstream. In the latter case, you can push your submodule branch to your fork and easily open a PR.
+
+#### **Using `subtree` vendor strategy.**
+
+To be able to share your project with the library change, you just commit your changes. This is very useful for keeping track of your changes specific to your project. On the other hand this is not recommended if you plan to contribute it back to upstream because you will need to make sure your libdragon commits are isolated and do the juggling when pushing it somewhere via `git subtree push`.
 
 ## Working on this repository
 
@@ -86,15 +178,40 @@ This repository also uses [ed64](https://github.com/anacierdem/ed64), so you can
 
 There are also additional vscode launch configurations to build libdragon examples and tests based on the currently built and installed libdragon in the docker container. Most of these will always rebuild so that they will use the latest if you made and installed an alternative libdragon. The test bench itself and a few examples (that use the new build system) will already rebuild and reinstall libdragon automatically. These will always produce a rom image using the latest libdragon code in the active repository via its make dependencies. You can clean everything with the `clean` task (open the command palette and choose `Run Task -> clean`).
 
+### Developing the tool itself
+
+For a quick development loop it really helps linking the code in this repository as the global libdragon installation. To do this run;
+
+    npm link
+
+in the root of the repository. Once you do this, running `libdragon` will use the code here rather than the actual npm installation. Then you can test your changes in the libdragon project here or elsewhere on your computer.
+
+When you are happy with your changes, you can verify you conform to the coding standards via:
+
+    npm run format-check
+    npm run lint-check
+
+You can auto-fix applicable errors by running `format` and `lint` scripts instead. Additionally, typescript is used as the type system. To be able to get away with transpiling the code during development, jsDoc flavor of types are used instead of inline ones. To check your types, run:
+
+    npm run tsc
+
+This repository uses [`semantic-release`](https://github.com/semantic-release/semantic-release) and manages releases from specially formatted commit messages. To simplify creating them you can use:
+
+    npx cz
+
+It will create a `semantic-release` compatible commit from your current staged changes.
+
 ## As an NPM dependency
 
 You can install libdragon as an NPM dependency by `npm install libdragon --save` in order to use docker in your N64 projects. A `libdragon` command similar to global intallation is provided that can be used in your NPM scripts as follows;
 
-    "scripts": {
-        "prepare": "libdragon init"
-        "build": "libdragon make",
-        "clean": "libdragon make clean"
-    }
+```json
+"scripts": {
+    "prepare": "libdragon init"
+    "build": "libdragon make",
+    "clean": "libdragon make clean"
+}
+```
 
 See [here](https://github.com/anacierdem/ed64-example) for a full example.
 
@@ -104,29 +221,27 @@ You can make an NPM package that a `libdragon` project can depend on. Just inclu
 
 For example this `package.json` in the dependent project;
 
-    {
-        "name": "libdragonDependentProject",
-        "version": "1.0.0",
-        "description": "...",
-        "scripts": {
-            "build": "libdragon make",
-            "clean": "libdragon make clean",
-            "prepare": "libdragon init"
-        },
-        "dependencies": {
-            "libdragon": <version>,
-            "ed64": <version>
-        }
+```json
+{
+    "name": "libdragonDependentProject",
+    "version": "1.0.0",
+    "description": "...",
+    "scripts": {
+        "build": "libdragon make",
+        "clean": "libdragon make clean",
+        "prepare": "libdragon init"
+    },
+    "dependencies": {
+        "libdragon": <version>,
+        "ed64": <version>
     }
+}
+```
 
 will init the container for this project and run `make && make install` for `ed64` upon running `npm install`. To develop a dependency [this](https://github.com/anacierdem/libdragon-dependency) is a good starting point.
 
 This is an experimental dependency management.
 
-## TODOS
-
-- [ ] Skip CI checks for irrelevant changes.
-
 ## Funding
 
 If this tool helped you, consider supporting its development by sponsoring it!

package/index.js

@@ -18,9 +18,30 @@ const {
 const { parseParameters } = require('./modules/parameters');
 const { readProjectInfo, writeProjectInfo } = require('./modules/project-info');
 
+// Note: it is not possible to merge these type definitions in a single comment
+/**
+ * @template {any} [U=any]
+ * @typedef {(U extends any ? (k: U)=>void : never) extends ((k: infer I)=>void) ? I : never} UnionToIntersection
+ */
+/**
+ * @template {any} [K=any]
+ * never does not break the call `info.options.CURRENT_ACTION.fn`
+ * @typedef {[K] extends [UnionToIntersection<K>] ? any : unknown} NoUnion
+ * @typedef {NoUnion<Exclude<Parameters<import('./modules/parameters').Actions[import('./modules/project-info').ActionsNoProject]['fn']>[0], undefined>>} EitherCLIOrLibdragonInfo
+ */
+
 parseParameters(process.argv)
   .then(readProjectInfo)
-  .then((info) => info.options.CURRENT_ACTION.fn(info))
+  .then((info) => {
+    return info.options.CURRENT_ACTION.fn(
+      /** @type {EitherCLIOrLibdragonInfo} */ (info)
+    );
+    // This type make sure a similar restriction to this code block is enforced
+    // without adding unnecessary javascript.
+    // return isProjectAction(info)
+    //   ? info.options.CURRENT_ACTION.fn(info)
+    //   : info.options.CURRENT_ACTION.fn(info);
+  })
   .catch((e) => {
     if (e instanceof ParameterError) {
       log(chalk.red(e.message));

package/modules/actions/destroy.js

@@ -6,6 +6,9 @@ const { CONFIG_FILE, LIBDRAGON_PROJECT_MANIFEST } = require('../constants');
 const { fileExists, dirExists, log } = require('../helpers');
 const chalk = require('chalk');
 
+/**
+ * @param {import('../project-info').LibdragonInfo} libdragonInfo
+ */
 const destroy = async (libdragonInfo) => {
   await destroyContainer(libdragonInfo);
 
@@ -22,13 +25,13 @@ const destroy = async (libdragonInfo) => {
   log(chalk.green('Done cleanup.'));
 };
 
-module.exports = {
+module.exports = /** @type {const} */ ({
   name: 'destroy',
   fn: destroy,
-  mustHaveProject: false,
+  forwardsRestParams: false,
   usage: {
     name: 'destroy',
     summary: 'Do clean-up for current project.',
     description: `Removes libdragon configuration from current project and removes any known containers but will not touch previously vendored files. \`libdragon\` will not work anymore for this project.`,
   },
-};
+});

package/modules/actions/disasm.js

@@ -10,6 +10,11 @@ const {
   ParameterError,
 } = require('../helpers');
 
+/**
+ * @param {string} stop
+ * @param {string} start
+ * @returns {Promise<string>}
+ */
 const findElf = async (stop, start = '.') => {
   start = path.resolve(start);
 
@@ -43,6 +48,9 @@ const findElf = async (stop, start = '.') => {
   }
 };
 
+/**
+ * @param {import('../project-info').LibdragonInfo} info
+ */
 const disasm = async (info) => {
   let elfPath;
   if (info.options.FILE) {
@@ -89,7 +97,7 @@ const disasm = async (info) => {
   });
 };
 
-module.exports = {
+module.exports = /** @type {const} */ ({
   name: 'disasm',
   fn: disasm,
   forwardsRestParams: true,
@@ -118,4 +126,4 @@ module.exports = {
       },
     ],
   },
-};
+});

package/modules/actions/docker-utils.js

@@ -1,3 +1,7 @@
+/**
+ *
+ * @param {import('../project-info').LibdragonInfo} libdragonInfo
+ */
 function dockerHostUserParams(libdragonInfo) {
   const { uid, gid } = libdragonInfo.userInfo;
   return ['-u', `${uid >= 0 ? uid : ''}:${gid >= 0 ? gid : ''}`];

package/modules/actions/exec.js

@@ -8,12 +8,16 @@ const {
   toPosixPath,
   fileExists,
   dirExists,
+  CommandError,
 } = require('../helpers');
 
 const { start } = require('./start');
 const { dockerHostUserParams } = require('./docker-utils');
 const { installDependencies } = require('./utils');
 
+/**
+ * @param {import('../project-info').LibdragonInfo} libdragonInfo
+ */
 function dockerRelativeWorkdir(libdragonInfo) {
   return (
     CONTAINER_TARGET_PATH +
@@ -22,10 +26,16 @@ function dockerRelativeWorkdir(libdragonInfo) {
   );
 }
 
+/**
+ * @param {import('../project-info').LibdragonInfo} libdragonInfo
+ */
 function dockerRelativeWorkdirParams(libdragonInfo) {
   return ['--workdir', dockerRelativeWorkdir(libdragonInfo)];
 }
 
+/**
+ * @param {import('../project-info').LibdragonInfo} info
+ */
 const exec = async (info) => {
   const parameters = info.options.EXTRA_PARAMS.slice(1);
   log(
@@ -37,21 +47,32 @@ const exec = async (info) => {
 
   const stdin = new PassThrough();
 
-  const paramsWithConvertedPaths = parameters.map((item) => {
-    if (item.startsWith('-')) {
+  /** @type {string[]} */
+  const paramsWithConvertedPaths = await Promise.all(
+    parameters.map(async (item) => {
+      if (item.startsWith('-')) {
+        return item;
+      }
+      if (
+        item.includes(path.sep) &&
+        ((await fileExists(item)) || (await dirExists(item)))
+      ) {
+        return toPosixPath(
+          path.isAbsolute(item) ? path.relative(process.cwd(), item) : item
+        );
+      }
       return item;
-    }
-    if (item.includes(path.sep) && (fileExists(item) || dirExists(item))) {
-      return toPosixPath(
-        path.isAbsolute(item) ? path.relative(process.cwd(), item) : item
-      );
-    }
-    return item;
-  });
+    })
+  );
 
+  /**
+   *
+   * @param {import('../project-info').LibdragonInfo} libdragonInfo
+   * @param {import('../helpers').SpawnOptions} opts
+   * @returns
+   */
   const tryCmd = (libdragonInfo, opts = {}) => {
     const enableTTY = Boolean(process.stdout.isTTY && process.stdin.isTTY);
-
     return (
       libdragonInfo.containerId &&
       dockerExec(
@@ -80,6 +101,9 @@ const exec = async (info) => {
   };
 
   let started = false;
+  /**
+   * @param {import('fs').ReadStream=} stdin
+   */
   const startOnceAndCmd = async (stdin) => {
     if (!started) {
       const newId = await start(info);
@@ -121,22 +145,30 @@ const exec = async (info) => {
       inheritStderr: false,
       // In the first run, pass the stdin to the process if it is not a TTY
       // o/w we loose a user input unnecesarily somehow.
-      stdin: !process.stdin.isTTY && process.stdin,
+      stdin:
+        (!process.stdin.isTTY || undefined) &&
+        /** @type {import('fs').ReadStream} */ (
+          /** @type {unknown} */ (process.stdin)
+        ),
     });
   } catch (e) {
+    if (!(e instanceof CommandError)) {
+      throw e;
+    }
     if (
-      !e.out ||
       // TODO: is there a better way?
       !e.out.toString().includes(info.containerId)
     ) {
       throw e;
     }
-    await startOnceAndCmd(stdin);
+    await startOnceAndCmd(
+      /** @type {import('fs').ReadStream} */ (/** @type {unknown} */ (stdin))
+    );
   }
   return info;
 };
 
-module.exports = {
+module.exports = /** @type {const} */ ({
   name: 'exec',
   fn: exec,
   forwardsRestParams: true,
@@ -152,4 +184,4 @@ module.exports = {
     
     Must be run in an initialized libdragon project.`,
   },
-};
+});

package/modules/actions/help.js

@@ -3,7 +3,10 @@ const commandLineUsage = require('command-line-usage');
 
 const { print } = require('../helpers');
 
-const printUsage = (info) => {
+/**
+ * @param {import('../project-info').CLIInfo} info
+ */
+const printUsage = async (info) => {
   const actions = require('./');
   const globalOptionDefinitions = [
     {
@@ -48,11 +51,12 @@ const printUsage = (info) => {
     },
   ];
 
-  const actionsToShow =
+  const actionsToShow = /** @type {(keyof actions)[]} */ (
     info?.options.EXTRA_PARAMS?.filter(
       (action) =>
         Object.keys(actions).includes(action) && !['help'].includes(action)
-    ) ?? (info ? [info.options.CURRENT_ACTION.name] : []);
+    ) ?? (info ? [info.options.CURRENT_ACTION.name] : [])
+  );
 
   const sections = [
     {
@@ -96,13 +100,12 @@ const printUsage = (info) => {
   print(usage);
 };
 
-module.exports = {
+module.exports = /** @type {const} */ ({
   name: 'help',
   fn: printUsage,
   forwardsRestParams: true,
-  mustHaveProject: false,
   usage: {
     name: 'help [action]',
     summary: 'Display this help information or details for the given action.',
   },
-};
+});

package/modules/actions/init.js

@@ -3,7 +3,6 @@ const path = require('path');
 
 const chalk = require('chalk').stderr;
 
-const { fn: install } = require('./install');
 const { start } = require('./start');
 const {
   installDependencies,
@@ -29,6 +28,10 @@ const {
 } = require('../helpers');
 const { syncImageAndStart } = require('./update-and-start');
 
+/**
+ * @param {import('../project-info').LibdragonInfo} info
+ * @returns {Promise<"submodule" | "subtree" | undefined>}
+ */
 const autoDetect = async (info) => {
   const vendorTarget = path.relative(
     info.root,
@@ -60,15 +63,25 @@ const autoDetect = async (info) => {
       inheritStdin: false,
       inheritStdout: false,
       inheritStderr: false,
+    }).catch((e) => {
+      if (!(e instanceof CommandError)) {
+        throw e;
+      }
     });
 
-    if (gitLogs.includes(`git-subtree-dir: ${info.vendorDirectory}`)) {
+    if (
+      gitLogs &&
+      gitLogs.includes(`git-subtree-dir: ${info.vendorDirectory}`)
+    ) {
       log(`${info.vendorDirectory} is a subtree.`);
       return 'subtree';
     }
   }
 };
 
+/**
+ * @param {import('../project-info').LibdragonInfo} info
+ */
 const autoVendor = async (info) => {
   // Update the strategy information for the project if the flag is provided
   if (info.options.VENDOR_STRAT) {
@@ -96,6 +109,8 @@ const autoVendor = async (info) => {
   }
 
   await runGitMaybeHost(info, ['init']);
+
+  // TODO: TS thinks this is already defined
   const detectedStrategy = await autoDetect(info);
 
   if (
@@ -114,22 +129,32 @@ const autoVendor = async (info) => {
     );
     return {
       ...info,
-      vendorStrategy: detectedStrategy,
+      vendorStrategy: /** @type {import('../parameters').VendorStrategy} */ (
+        detectedStrategy
+      ),
     };
   }
 
   if (info.vendorStrategy === 'submodule') {
-    await runGitMaybeHost(info, [
-      'submodule',
-      'add',
-      '--force',
-      '--name',
-      LIBDRAGON_SUBMODULE,
-      '--branch',
-      LIBDRAGON_BRANCH,
-      LIBDRAGON_GIT,
-      info.vendorDirectory,
-    ]);
+    try {
+      await runGitMaybeHost(info, [
+        'submodule',
+        'add',
+        '--force',
+        '--name',
+        LIBDRAGON_SUBMODULE,
+        '--branch',
+        LIBDRAGON_BRANCH,
+        LIBDRAGON_GIT,
+        info.vendorDirectory,
+      ]);
+    } catch (e) {
+      if (!(e instanceof CommandError)) throw e;
+      // We speculate this is caused by the user, so replace it with a more useful error message.
+      e.message = `Unable to create submodule. If you have copied the executable in your project folder or you have a file named ${info.vendorDirectory}, removing it might solve this issue. Original error:\n${e.message}`;
+      throw e;
+    }
+
     return info;
   }
 
@@ -163,11 +188,14 @@ const autoVendor = async (info) => {
     ]);
     return info;
   }
+
+  return info;
 };
 
 /**
  * Initialize a new libdragon project in current working directory
  * Also downloads the image
+ * @param {import('../project-info').LibdragonInfo} info
  */
 async function init(info) {
   log(`Initializing a libdragon project at ${info.root}`);
@@ -176,7 +204,7 @@ async function init(info) {
   const manifestPath = path.join(info.root, LIBDRAGON_PROJECT_MANIFEST);
   const manifestStats = await fs.stat(manifestPath).catch((e) => {
     if (e.code !== 'ENOENT') throw e;
-    return false;
+    return /** @type {const} */ (false);
   });
 
   if (manifestStats && !manifestStats.isDirectory()) {
@@ -216,7 +244,9 @@ async function init(info) {
   info.containerId = await start(info);
 
   // We have created a new container, save the new info ASAP
-  await initGitAndCacheContainerId(info);
+  await initGitAndCacheContainerId(
+    /** @type Parameters<initGitAndCacheContainerId>[0] */ (info)
+  );
 
   info = await autoVendor(info);
 
@@ -233,10 +263,10 @@ async function init(info) {
   return info;
 }
 
-module.exports = {
+module.exports = /** @type {const} */ ({
   name: 'init',
   fn: init,
-  mustHaveProject: false,
+  forwardsRestParams: false,
   usage: {
     name: 'init',
     summary: 'Create a libdragon project in the current directory.',
@@ -249,4 +279,4 @@ module.exports = {
     If you have an existing project with an already vendored submodule or subtree libdragon copy, \`init\` will automatically detect it at the provided \`--directory\`.`,
     group: ['docker', 'vendoring'],
   },
-};
+});