nw-builder 4.5.3 → 4.6.0

package/README.md

@@ -15,80 +15,296 @@ For version 3, please go to the [corresponding branch](https://github.com/nwutil
 - Downloading from mirrors
 - Node Native Addon support
 
-Check out the [documentation](https://nwutils.io/nw-builder/) if you wish to give `nw-builder` a try.
+## Table of Contents
 
-> Please note that the documentation assumes you know [how to write NW.js applications](https://nwjs.readthedocs.io/en/latest/For%20Users/Getting%20Started/).
+- [Installation](https://github.com/nwutils/nw-builder#install)
+- [Usage](https://github.com/nwutils/nw-builder#usage)
+- [Concepts](https://github.com/nwutils/nw-builder#concepts)
+- [API Reference](https://github.com/nwutils/nw-builder#api-reference)
+- [Guides](https://github.com/nwutils/nw-builder#guides)
+- [Contributing](https://github.com/nwutils/nw-builder#contributing)
+- [Roadmap](https://github.com/nwutils/nw-builder#roadmap)
+- [FAQ](https://github.com/nwutils/nw-builder#faq)
+- [License](https://github.com/nwutils/nw-builder#license)
 
-## Installation
+## Install
 
-With npm:
+Every NW.js release includes a modified Node.js binary at a specific version. It is recommended to [install](https://nodejs.org/en/download/package-manager) exactly that version on the host system. Not doing so may download ABI incompatible Node modules. Consult the NW.js [versions manifest](https://nwjs.io/versions) for what Node.js version to install. It is recommended to use a Node version manager (such as [volta](https://volta.sh), n, nvm, or nvm-windows) to be able to easily install and switch between Node versions.
+
+For example, NW.js v0.83.0 comes with Node.js v21.1.0.
 
 ```shell
-npm install nw-builder -D
+$: node --version
+v21.1.0
+```
+
+## Usage
+
+This package can be used via a command line interface, be imported as a JavaScript module, or configured via the Node manifest as a JSON object.
+
+ESM import:
+
+```javascript
+import nwbuild from "nw-builder";
+```
+
+CJS import:
+
+```javascript
+let nwbuild;
+import("nwbuild")
+  .then((object) => {
+    nwbuild = obj;
+  })
+  .catch((error) => {
+    console.error(error);
+  });
+```
+
+Node manifest usage:
+
+```json
+{
+    "nwbuild": {
+        // user specified options
+    }
+}
+```
+
+> From here on we will show `nw-builder` functionality by using the JavaScript module. Please note that the same method applies when using a command line or Node manifest.
+
+## Concepts
+
+`nw-builder` can get, run and build NW.js applications. We refer to them as get, run and build modes.
+
+### Get Mode
+
+By default you get the normal build of the latest NW.js release for your specific platform and arch. For more information, please refer to the API reference.
+
+```javascript
+nwbuild({
+  mode: "get"
+});
+```
+
+Get the community built FFmeg which contains proprietary codecs. This options is disabled by default. Please read the [license's constraints](https://nwjs.readthedocs.io/en/latest/For%20Developers/Enable%20Proprietary%20Codecs/#get-ffmpeg-binaries-from-the-community) before enabling this option.
+
+```javascript
+nwbuild({
+  mode: "get",
+  ffmpeg: true
+});
+```
+
+Get Node headers if you have to rebuild Node addons.
+
+```javascript
+nwbuild({
+  mode: "get",
+  nativeAddon: "gyp"
+});
+```
+
+### Run Mode
+
+```javascript
+nwbuild({
+  mode: "run",
+  srcDir: "./app",
+  glob: false,
+});
+```
+
+### Build Mode
+
+Build with defaults:
+
+```javascript
+nwbuild({
+  mode: "build",
+});
+```
+
+Managed Manifest
+
+You can let `nw-builder` manage your node modules. The `managedManifest` options accepts a `boolean`, `string` or `object` type. It will then remove `devDependencies`, autodetect and download `dependencies` via the relevant `packageManager`. If none is specified, it uses `npm` as default.
+
+Setting it to `true` will parse the first Node manifest it encounters as the NW manifest.
+
+```javascript
+nwbuild({
+  mode: "build",
+  managedManifest: true,
+});
+```
+
+Setting it to a `string` implies that you are passing the file path to the NW manifest.
+
+```javascript
+nwbuild({
+  mode: "build",
+  managedManifest: "./nw.js",
+});
 ```
 
-With yarn:
+Setting it to a `object` implies you are directly passing the NW manifest as a JavaScript object.
+
+```javascript
+nwbuild({
+  mode: "build",
+  managedManifest: {
+    name: "nwdemo",
+    main: "index.html"
+  },
+});
+```
+
+Rebuild Node addons
+
+Currently this feature is quite limited. It only builds node addons which have a `binding.gyp` file in the `srcDir`. There are plans to support nan, cmake, ffi and gn and auto rebuild native addons which are installed as node modules.
+
+```javascript
+nwbuild({
+  mode: "build",
+  nodeAddon: "gyp",
+});
+```
+
+We recommend rebuilding Node addons for NW.js via `node-gyp` if you are using NW.js v0.83.0 or above.
 
 ```shell
-yarn add nw-builder -D
+node-gyp rebuild --target=21.1.0 --nodedir=/path/to/nw/node/headers
 ```
 
-With pnpm:
+NW.js's Node version should match the Node version on the host machine due to [ABI differences in V8](https://github.com/nwjs/nw.js/issues/5025).
+
+## API Reference
+
+Options
+
+| Name | Type    | Default   | Description |
+| ---- | ------- | --------- | ----------- |
+| mode | `"get" \| "run" \| "build"` | `"build"` | Choose between get, run or build mode |
+| version | `string \| "latest" \| "stable"` | `"latest"` | Runtime version |
+| flavor | `"normal" \| "sdk"` | `"normal"` | Runtime flavor |
+| platform | `"linux" \| "osx" \| "win"` | | Host platform |
+| arch | `"ia32" \| "x64" \| "arm64"` | | Host architecture |
+| downloadUrl | `"https://dl.nwjs.io" \| "https://npm.taobao.org/mirrors/nwjs" \| https://npmmirror.com/mirrors/nwjs \| "https://github.com/corwin-of-amber/nw.js/releases/"` | `"https://dl.nwjs.io"` | Download server |
+| manifestUrl | `"https://nwjs.io/versions" \| "https://raw.githubusercontent.com/nwutils/nw-builder/main/src/util/osx.arm.versions.json"` | `"https://nwjs.io/versions"` | Versions manifest |
+| cacheDir | `string` | `"./cache"` | Directory to cache NW binaries |
+| srcDir | `string` | `"./"` | File paths to application code |
+| outDir | `string` | `"./out"` | Directory to store build artifacts |
+| managedManifest | `boolean \| string \| object` | `false` | Managed manifest |
+| nodeAddon | `false \| "gyp"` | `false` | Rebuild Node native addons |
+| cache | `boolean` | `true`| If true the existing cache is used. Otherwise it removes and redownloads it. |
+| ffmpeg | `boolean` | `false`| If true the chromium ffmpeg is replaced by community version with proprietary codecs. |
+| glob | `boolean` | `true`| If true file globbing is enabled when parsing `srcDir`. |
+| logLevel | `"error" \| "warn" \| "info" \| "debug"` | `"info"`| Specify level of logging. |
+| zip | `boolean \| "zip" \| "tar" \| "tgz"` | `false`| If true, "zip", "tar" or "tgz" the `outDir` directory is compressed. |
+
+## Guides
+
+### Get unofficial MacOS builds
+
+If you're running older Apple machines, you can download the unofficial builds.
+
+> Note: You will have to manually remove quarantine flag.
 
 ```shell
-pnpm add nw-builder -D
+sudo xattr -r -d com.apple.quarantine /path/to/nwjs.app
 ```
 
-## Usage
+```javascript
+nwbuild({
+  mode: "get",
+  platform: "osx",
+  arch: "arm64",
+  downloadUrl: "https://github.com/corwin-of-amber/nw.js/releases/download",
+  manifestUrl: "https://raw.githubusercontent.com/nwutils/nw-builder/main/src/util/osx.arm.versions.json",
+});
+```
 
-Here is two way to use nw-build to build your nwjs applications
-
-### CLI
-
-1. To get nwjs cache
-    ```bash
-    nwbuild --mode=get
-    ```
-2. To run nwjs application
-    ```bash
-    nwbuild --mode=run
-    ```
-3. To build nwjs application
-    ```bash
-    nwbuild --mode=build
-    ```
-
-### JavaScript API
-1. Define an npm script
-    ```json
-    {
-      "scripts": {
-        "build": "node scripts/build.js"
-      }
-    }
-    ```
-2. Create a build script
-    ```javascript
-    // scripts/build.js
-    const { nwbuild } = require("nw-builder");
-    await nwbuild({
-      srcDir: "./nwapp/**/* ./other/**/*.js",
-      mode: "build",
-      version: "latest",
-      flavor: "normal",
-      platform: "linux",
-      arch: "x64",
-      outDir: "./build",
-      cache: false,
-      app: { ... },
-    });
-    ```
-3. Run the script
-    ```bash
-    npm run build
-    ```
-
-## Alternatives
-
-- [nw-builder-platforms](https://github.com/naviapps/nw-builder-platforms) - Fork of this repo with platform specific build options
-- [nwjs-builder-phoenix](https://github.com/evshiron/nwjs-builder-phoenix) - Previously the most used build tool, however it is no longer maintained
+> Note: Community FFmpeg binaries may not be available for unofficial builds. You will have to rebuild them yourself.
+
+### Get binaries via mirrors
+
+China mirror:
+
+```javascript
+nwbuild({
+  mode: "get",
+  downloadUrl: "https://npm.taobao.org/mirrors/nwjs",
+});
+```
+
+Singapore mirror:
+
+```javascript
+nwbuild({
+  mode: "get",
+  downloadUrl: "https://cnpmjs.org/mirrors/nwjs/",
+});
+```
+
+### Let `nw-builder` manage your native addons
+
+> Note: this behaviour is buggy and quite limited. This guide is to show what will be possible in the coming minor releases.
+
+```javascript
+nwbuild({
+  mode: "build",
+  managedManifest: true,
+  nativeAddon: "gyp",
+});
+```
+
+## Contributing
+
+### External contributor
+
+- We use [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style of commit messages.
+- When making changes, try to follow [SOLID](https://en.wikipedia.org/wiki/SOLID) and [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) principles.
+- Pull requests are squashed and merged onto the `main` branch.
+- Lint your code before commiting your change.
+- Add tests whenever possible.
+
+### Maintainer guidelines
+
+## Roadmap
+
+### Bugs
+
+- MacOS fails to unzip MacOS NW.js binaries consistently
+- Add back error, info, warn and debug logs
+
+### Features
+
+- feat(get): support canary releases
+- feat(bld): rename MacOS Helper apps
+- feat(pkg): add `AppImage` installer
+- feat(pkg): add `NSIS` installer
+- feat(pkg): add `DMG` installer
+- feat(get): add Linux ARM unofficial support
+- feat(bld): add source code protection
+- feat(pkg): add code signing
+
+### Chores
+
+- chore(cicd): use `google-github-actions/release-please-action` to automate publishing to npm, updating changelog and creating releases
+- chore(cli): migrate from `yargs` to `commander`
+- chore(get): verify sha checksum for downloads
+- chore(util): factor out file paths as constant variables
+- chore(bld): factor out core build step
+- chore(bld): factor out linux config
+- chore(bld): factor out macos config
+- chore(bld): factor out windows config
+- chore(bld): factor out native addon
+- chore(bld): factor out compressing
+- chore(bld): factor out managed manifest
+- chore(bld): move `.desktop` entry file logic to `create-desktop-shortcuts` package
+
+## FAQ
+
+## License
+
+MIT License.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "nw-builder",
-  "version": "4.5.3",
+  "version": "4.6.0",
   "description": "Build NW.js desktop applications for MacOS, Windows and Linux.",
   "keywords": [
     "NW.js",
@@ -33,7 +33,6 @@
   "type": "module",
   "files": [
     "LICENSE",
-    "patches",
     "src"
   ],
   "homepage": "https://github.com/nwutils/nw-builder",
@@ -42,29 +41,39 @@
     "url": "https://github.com/nwutils/nw-builder.git"
   },
   "scripts": {
-    "lint": "eslint src/get.js",
-    "docs": "jsdoc -d docs ./src/get.js ./src/run.js ./src/bld.js",
-    "test": "node --test test/get.test.js",
+    "lint": "eslint ./src/**/*.js ./test/**/*.js",
+    "lint:fix": "eslint --fix ./**/*.{js,md} && markdownlint --fix ./README.md",
+    "docs": "jsdoc -d docs ./README.md ./src/index.js ./src/get.js ./src/run.js ./src/bld.js",
+    "test": "vitest",
     "demo": "cd test/fixture && node demo.js"
   },
   "devDependencies": {
+    "@stylistic/eslint-plugin-js": "^1.5.4",
     "eslint": "^8.56.0",
-    "eslint-plugin-jsdoc": "^46.9.1",
+    "eslint-config-tjw-jsdoc": "^1.0.5",
+    "eslint-plugin-jsdoc": "^48.0.2",
+    "eslint-plugin-markdown": "^3.0.1",
     "jsdoc": "^4.0.2",
-    "selenium-webdriver": "^4.16.0"
+    "markdownlint": "^0.33.0",
+    "markdownlint-cli": "^0.38.0",
+    "selenium-webdriver": "^4.16.0",
+    "vitest": "^1.2.1"
   },
   "dependencies": {
+    "axios": "^1.6.7",
     "cli-progress": "^3.12.0",
     "compressing": "^1.10.0",
+    "fs-extra": "^11.2.0",
     "glob": "^10.3.10",
     "node-gyp": "^10.0.1",
     "plist": "^3.1.0",
     "rcedit": "^4.0.1",
+    "semver": "^7.5.4",
     "tar": "^6.2.0",
-    "unzipper": "^0.10.14",
-    "yargs": "^17.7.2"
+    "yargs": "^17.7.2",
+    "yauzl-promise": "^4.0.0"
   },
-  "packageManager": "npm@10.2.4",
+  "packageManager": "npm@10.3.0",
   "engines": {
     "node": ">=14"
   },
@@ -79,11 +88,18 @@
     },
     "extends": [
       "eslint:recommended",
-      "plugin:jsdoc/recommended"
+      "plugin:markdown/recommended",
+      "tjw-jsdoc"
+    ],
+    "plugins": [
+      "@stylistic/js"
     ],
     "rules": {
-      "jsdoc/tag-lines": "off",
-      "jsdoc/check-property-names": "off"
+      "@stylistic/js/indent": [
+        "error",
+        2
+      ],
+      "jsdoc/require-file-overview": "off"
     }
   }
 }

package/src/bld.js

@@ -83,72 +83,26 @@ import util from "./util.js"
 
 /**
  * @typedef {object} BuildOptions
- * @property {string | "latest" | "stable" | "lts"} [options.version = "latest"]                  Runtime version
- * @property {"normal" | "sdk"}                     [options.flavor = "normal"]                   Build flavor
- * @property {"linux" | "osx" | "win"}              [options.platform]                            Target platform
- * @property {"ia32" | "x64" | "arm64"}             [options.arch]                                Target arch
- * @property {string}                               [options.manifestUrl = "https://nwjs.io/versions"] Manifest URL
- * @property {string}                               [options.srcDir = "./src"]                    Source directory
- * @property {string}                               [options.cacheDir = "./cache"]                Cache directory
- * @property {string}                               [options.outDir = "./out"]                    Out directory
- * @property {LinuxRc | WinRc | OsxRc}              [options.app]                                 Platform specific rc
- * @property {boolean}                              [options.glob = true]                         File globbing
- * @property {boolean | string | object}            [options.managedManifest = false]             Manage manifest
- * @property {false | "gyp"}                        [options.nativeAddon = false]                 Rebuild native modules
- * @property {false | "zip" | "tar" | "tgz"}        [options.zip = false]                         Compress built artifacts
+ * @property {string | "latest" | "stable" | "lts"} [options.version = "latest"]                        Runtime version
+ * @property {"normal" | "sdk"}                     [options.flavor = "normal"]                         Build flavor
+ * @property {"linux" | "osx" | "win"}              [options.platform]                                  Target platform
+ * @property {"ia32" | "x64" | "arm64"}             [options.arch]                                      Target arch
+ * @property {string}                               [options.manifestUrl = "https://nwjs.io/versions"]  Manifest URL
+ * @property {string}                               [options.srcDir = "./src"]                          Source directory
+ * @property {string}                               [options.cacheDir = "./cache"]                      Cache directory
+ * @property {string}                               [options.outDir = "./out"]                          Out directory
+ * @property {LinuxRc | WinRc | OsxRc}              [options.app]                                       Platform specific rc
+ * @property {boolean}                              [options.glob = true]                               File globbing
+ * @property {boolean | string | object}            [options.managedManifest = false]                   Manage manifest
+ * @property {false | "gyp"}                        [options.nativeAddon = false]                       Rebuild native modules
+ * @property {false | "zip" | "tar" | "tgz"}        [options.zip = false]                               Compress built artifacts
  */
 
 /**
  * @async
  * @function
- * @param {BuildOptions} options
+ * @param  {BuildOptions}  options  - Build options  
  * @return {Promise<void>}
- * 
- * @example
- * // Minimal Usage (uses default values)
- * nwbuild({
- *   mode: "build",
- * });
- * 
- * @example
- * // Managed Manifest mode
- * // Parse first Node manifest it encounters as NW manifest
- * // Remove development dependencies
- * // Auto detect and download dependencies via relevant package manager 
- * nwbuild({
- *   mode: "build",
- *   managedManifest: true
- * });
- * 
- * @example
- * // Managed Manifest JSON
- * // Use JSON object provided as NW manifest
- * nwbuild({
- *   mode: "build",
- *   managedManifest: { name: "demo", "main": "index.html" }
- * });
- * 
- * @example
- * // Managed Manifest File
- * // Use file path provided as NW manifest
- * nwbuild({
- *   mode: "build",
- *   managedManifest: "./manifest.json"
- * });
- * 
- * @example
- * // Rebuild Node native modules
- * // This behaviour is currently disabled.
- * // Tracking issue: https://github.com/nwutils/nw-builder/pull/993
- * nwbuild({
- *   mode: "build",
- *   nodeAddon: "gyp"
- * });
- * 
- * @example
- * // For MacOS ARM unofficial builds (<= v0.75), remove quarantine flag post build.
- * sudo xattr -r -d com.apple.quarantine /path/to/nwjs.app
- * 
  */
 async function bld({
   version = "latest",
@@ -218,7 +172,7 @@ async function bld({
     typeof managedManifest === "object" ||
     typeof managedManifest === "string"
   ) {
-    manageManifest({ manifest, managedManifest, outDir, platform });
+    await manageManifest({ manifest, managedManifest, outDir, platform });
   }
 
   if (platform === "linux") {
@@ -280,11 +234,11 @@ const manageManifest = async ({ nwPkg, managedManifest, outDir, platform }) => {
   );
 
   if (manifest.packageManager.startsWith("npm")) {
-    child_process.exec(`npm install`);
+    child_process.execSync(`npm install`);
   } else if (manifest.packageManager.startsWith("yarn")) {
-    child_process.exec(`yarn install`);
+    child_process.execSync(`yarn install`);
   } else if (manifest.packageManager.startsWith("pnpm")) {
-    child_process.exec(`pnpm install`);
+    child_process.execSync(`pnpm install`);
   }
 };
 
@@ -370,10 +324,12 @@ const setWinConfig = async ({ app, outDir }) => {
     await fsm.rename(path.resolve(outDir, "nw.exe"), outDirAppExe);
     await rcedit(outDirAppExe, rcEditOptions);
   } catch (error) {
-    console.warn(
-      "Renaming EXE failed or unable to modify EXE. If it's the latter, ensure WINE is installed or build your application Windows platform",
-    );
-    console.error(error);
+    if (process.platform !== "win32") {
+      console.warn(
+        "Ensure WINE is installed or build your application on Windows platform",
+      );
+    }
+    throw error;
   }
 };
 
@@ -453,14 +409,7 @@ const buildNativeAddon = ({ cacheDir, version, platform, arch, outDir, nodeVersi
     ),
   );
 
-  child_process.exec(
-    `node-gyp rebuild --target=${nodeVersion} --nodedir=${nodePath}`,
-    (error) => {
-      if (error !== null) {
-        console.error(error);
-      }
-    },
-  );
+  child_process.execSync(`node-gyp rebuild --target=${nodeVersion} --nodedir=${nodePath}`);
 };
 
 const compress = async ({

package/src/get/decompress.js

@@ -0,0 +1,96 @@
+import fs from "node:fs";
+import path from "node:path";
+import stream from "node:stream";
+
+import tar from "tar";
+import yauzl from "yauzl-promise";
+import {ensureSymlink} from "fs-extra";
+
+/**
+ * Decompresses a file at `filePath` to `cacheDir` directory.
+ *
+ * @param {string} filePath  - file path to compressed binary
+ * @param {string} cacheDir  - directory to decompress into
+ */
+export default async function decompress(filePath, cacheDir) {
+  if (filePath.endsWith(".zip")) {
+    await unzip(filePath, cacheDir);
+  } else {
+    await tar.extract({
+      file: filePath,
+      C: cacheDir
+    });
+  }
+}
+
+/**
+ * Wrapper for unzipping using `yauzl-promise`.
+ *
+ * @async
+ * @function
+ * @param  {string}        zippedFile  - file path to .zip file
+ * @param  {string}        cacheDir    - directory to unzip in
+ * @return {Promise<void>}
+ */
+export async function unzip(zippedFile, cacheDir) {
+  await unzipInternal(zippedFile, cacheDir, false).then(() => {
+    unzipInternal(zippedFile, cacheDir, true);
+  })
+}
+
+/**
+ * Method for unzip with symlink in theoretical
+ *
+ * @async
+ * @function
+ * @param                  unzipSymlink
+ * @param  {string}        zippedFile    - file path to .zip file
+ * @param  {string}        cacheDir      - directory to unzip in
+ * @param  {boolean}       unzipSymlink  - Using or not symlink
+ * @return {Promise<void>}
+ */
+async function unzipInternal(zippedFile, cacheDir, unzipSymlink )  {
+  const zip = await yauzl.open(zippedFile);
+
+  let entry = await zip.readEntry();
+
+  while (entry !== null) {
+    // console.log(entry)
+    let entryPathAbs = path.join(cacheDir, entry.filename);
+    // Create the directory beforehand to prevent `ENOENT: no such file or directory` errors.
+    await fs.promises.mkdir(path.dirname(entryPathAbs), {recursive: true});
+    const readStream = await entry.openReadStream();
+
+    try {
+      if (!unzipSymlink) {
+        // Regular method and silent error at this point
+        const writeStream = fs.createWriteStream(entryPathAbs);
+        await stream.promises.pipeline(readStream, writeStream);
+      } else {
+        // Need check before if file is a symlink or not at this point
+        const pathContent = await fs.promises.lstat(entryPathAbs);
+
+        if (pathContent.isSymbolicLink()) {
+          const chunks = [];
+          readStream.on('data', (chunk) => chunks.push(chunk));
+          await stream.promises.finished(readStream);
+          // need fetch value of current symlink here
+          const linkTarget = Buffer.concat(chunks).toString('utf8').trim();
+          await ensureSymlink(entryPathAbs, path.join(path.dirname(entryPathAbs), linkTarget));
+        }else{
+          // Regular method and silent error at this point
+          const writeStream = fs.createWriteStream(entryPathAbs);
+          await stream.promises.pipeline(readStream, writeStream);
+        }
+      }
+    } catch (error) {
+      if (unzipSymlink) {
+        console.error(error);
+      }
+    }
+
+    entry = await zip.readEntry();
+  }
+
+  await zip.close();
+}