@cyclonedx/cdxgen 9.4.0 → 9.5.0

package/README.md

@@ -2,13 +2,13 @@
 
 ![cdxgen logo](cdxgen.png)
 
-cdxgen is a cli tool and a library to create a valid and compliant [CycloneDX][cyclonedx-homepage] Software Bill-of-Materials (SBOM) containing an aggregate of all project dependencies for c/c++, node.js, php, python, ruby, rust, java, .Net, dart, haskell, elixir, and Go projects in JSON format. CycloneDX 1.5 is a lightweight SBOM specification that is easily created, human and machine-readable, and simple to parse.
+cdxgen is a cli tool, library, [REPL](./ADVANCED.md) and server to create a valid and compliant [CycloneDX][cyclonedx-homepage] Software Bill-of-Materials (SBOM) containing an aggregate of all project dependencies for c/c++, node.js, php, python, ruby, rust, java, .Net, dart, haskell, elixir, and Go projects in JSON format. CycloneDX 1.5 is a lightweight SBOM specification that is easily created, human and machine-readable, and simple to parse.
 
-When used with plugins, cdxgen could generate an SBoM for Linux docker images and even VMs running Linux or Windows operating system.
+When used with plugins, cdxgen could generate an SBoM for Linux docker images and even VMs running Linux or Windows operating system. cdxgen also includes a tool called `evinse` that can generate component evidences for some languages.
 
 NOTE:
 
-CycloneDX 1.5 specification is brand new and unsupported by many downstream tools. Use version 8.6.0 for 1.4 compatibility or pass the argument `--spec-version 1.4`.
+CycloneDX 1.5 specification is new and unsupported by many downstream tools. Use version 8.6.0 for 1.4 compatibility or pass the argument `--spec-version 1.4`.
 
 ## Why cdxgen?
 
@@ -93,7 +93,7 @@ sudo npm install -g @cyclonedx/cdxgen@8.6.0
 Deno install is also supported.
 
 ```shell
-deno install --allow-read --allow-env --allow-run --allow-sys=uid,systemMemoryInfo --allow-write --allow-net -n cdxgen "npm:@cyclonedx/cdxgen"
+deno install --allow-read --allow-env --allow-run --allow-sys=uid,systemMemoryInfo --allow-write --allow-net -n cdxgen "npm:@cyclonedx/cdxgen/cdxgen"
 ```
 
 You can also use the cdxgen container image
@@ -373,6 +373,10 @@ cdxgen -t os
 
 This feature is powered by osquery which is [installed](https://github.com/cyclonedx/cdxgen-plugins-bin/blob/main/build.sh#L8) along with the binary plugins. cdxgen would opportunistically try to detect as many components, apps and extensions as possible using the [default queries](queries.json). The process would take several minutes and result in an SBoM file with thousands of components.
 
+## Generating component evidence
+
+See [evinse mode](./ADVANCED.md) in the advanced documentation.
+
 ## SBoM signing
 
 cdxgen can sign the generated SBoM json file to increase authenticity and non-repudiation capabilities. To enable this, set the following environment variables.
@@ -385,7 +389,16 @@ To generate test public/private key pairs, you can run cdxgen by passing the arg
 
 ![SBoM signing](sbom-sign.jpg)
 
-### Verifying the signature (Node.js example)
+### Verifying the signature
+
+Use the bundled `cdx-verify` command which supports verifying a single signature added at the bom level.
+
+```shell
+npm install -g @cyclonedx/cdxgen
+cdx-verify -i bom.json --public-key public.key
+```
+
+### Custom verification tool (Node.js example)
 
 There are many [libraries](https://jwt.io/#libraries-io) available to validate JSON Web Tokens. Below is a javascript example.
 
@@ -410,7 +423,7 @@ if (validationResult) {
 
 ## Automatic services detection
 
-cdxgen could automatically detect names of services from YAML manifests such as docker-compose or Kubernetes or Skaffold manifests. These would be populated under the `services` attribute in the generated SBoM. Please help improve this feature by filing issues for any inaccurate detection.
+cdxgen can automatically detect names of services from YAML manifests such as docker-compose or Kubernetes or Skaffold manifests. These would be populated under the `services` attribute in the generated SBoM. With [evinse](./ADVANCED.md), additional services could be detected by parsing common annotations from the source code.
 
 ## Conversion to SPDX format
 
@@ -443,50 +456,6 @@ const bomNSData = await createBom(filePath, options);
 const dbody = await submitBom(args, bomNSData.bomJson);
 ```
 
-## Interactive mode
-
-`cdxi` is a new interactive REPL server to interactively create, import and search an SBoM. All the exported functions from cdxgen and node.js could be used in this mode. In addition, several custom commands are defined.
-
-### Custom commands
-
-| Command   | Description                                                                                                                                                                                                    |
-| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| .create   | Create an SBoM from a path                                                                                                                                                                                     |
-| .import   | Import an existing SBoM from a path. Any SBoM in CycloneDX format is supported.                                                                                                                                |
-| .search   | Search the given string in the components name, group, purl and description                                                                                                                                    |
-| .sort     | Sort the components based on the given attribute. Eg: .sort name to sort by name. Accepts full jsonata [order by](http://docs.jsonata.org/path-operators#order-by-) clause too. Eg: `.sort components^(>name)` |
-| .query    | Pass a raw query in [jsonata](http://docs.jsonata.org/) format                                                                                                                                                 |
-| .print    | Print the SBoM as a table                                                                                                                                                                                      |
-| .tree     | Print the dependency tree if available                                                                                                                                                                         |
-| .validate | Validate the SBoM                                                                                                                                                                                              |
-| .exit     | To exit the shell                                                                                                                                                                                              |
-| .save     | To save the modified SBoM to a new file                                                                                                                                                                        |
-| .update   | Update components based on query expression. Use syntax `\| query \| new object \|`. See example.                                                                                                              |
-
-### Sample REPL usage
-
-Start the REPL server.
-
-```shell
-cdxi
-```
-
-Below are some example commands to create an SBoM for a spring application and perform searches and queries.
-
-```
-.create /mnt/work/vuln-spring
-.print
-.search spring
-.query components[name ~> /spring/ and scope = "required"]
-.sort name
-.sort components^(>name)
-.update | components[name ~> /spring/] | {'publisher': "foo"} |
-```
-
-### REPL History
-
-Repl history will get persisted under `$HOME/.config/.cdxgen` directory. To override this location, use the environment variable `CDXGEN_REPL_HISTORY`.
-
 ## Node.js >= 20 permission model
 
 Refer to the [permissions document](./docs/PERMISSIONS.md)

package/analyzer.js

@@ -1,5 +1,5 @@
 import { parse } from "@babel/parser";
-import babelTraverse from "@babel/traverse";
+import traverse from "@babel/traverse";
 import { join } from "path";
 import { readdirSync, statSync, readFileSync } from "fs";
 import { basename, resolve, isAbsolute } from "path";
@@ -23,7 +23,7 @@ const IGNORE_DIRS = [
 ];
 
 const IGNORE_FILE_PATTERN = new RegExp(
-  "(conf|test|spec|mock|\\.d)\\.(js|ts|tsx)$",
+  "(conf|config|test|spec|mock|\\.d)\\.(js|ts|tsx)$",
   "i"
 );
 
@@ -64,6 +64,7 @@ const babelParserOptions = {
   sourceType: "unambiguous",
   allowImportExportEverywhere: true,
   allowAwaitOutsideFunction: true,
+  allowNewTargetOutsideFunction: true,
   allowReturnOutsideFunction: true,
   allowSuperOutsideMethod: true,
   errorRecovery: true,
@@ -123,8 +124,32 @@ const setFileRef = (allImports, file, pathway) => {
  */
 const parseFileASTTree = (file, allImports) => {
   const ast = parse(readFileSync(file, "utf-8"), babelParserOptions);
-  babelTraverse(ast, {
-    // Used for all ES6 import statements
+  traverse.default(ast, {
+    Import: (path) => {
+      if (path && path.node) {
+        setFileRef(allImports, file, path.node.source.value);
+      }
+    },
+    ImportDefaultSpecifier: (path) => {
+      if (path && path.node) {
+        setFileRef(allImports, file, path.node.source.value);
+      }
+    },
+    ImportNamespaceSpecifier: (path) => {
+      if (path && path.node) {
+        setFileRef(allImports, file, path.node.source.value);
+      }
+    },
+    ImportAttribute: (path) => {
+      if (path && path.node) {
+        setFileRef(allImports, file, path.node.source.value);
+      }
+    },
+    ImportOrExportDeclaration: (path) => {
+      if (path && path.node) {
+        setFileRef(allImports, file, path.node.source.value);
+      }
+    },
     ImportDeclaration: (path) => {
       if (path && path.node) {
         setFileRef(allImports, file, path.node.source.value);

package/bin/evinse.js

@@ -0,0 +1,121 @@
+// Evinse (Evinse Verification Is Nearly SBoM Evidence)
+import yargs from "yargs";
+import { hideBin } from "yargs/helpers";
+import { join } from "node:path";
+import fs from "node:fs";
+import { homedir, platform as _platform } from "node:os";
+import process from "node:process";
+import { analyzeProject, createEvinseFile, prepareDB } from "../evinser.js";
+import { validateBom } from "../validator.js";
+import { printCallStack, printOccurrences, printServices } from "../display.js";
+
+const isWin = _platform() === "win32";
+const isMac = _platform() === "darwin";
+let ATOM_DB = join(homedir(), ".local", "share", ".atomdb");
+if (isWin) {
+  ATOM_DB = join(homedir(), "AppData", "Local", ".atomdb");
+} else if (isMac) {
+  ATOM_DB = join(homedir(), "Library", "Application Support", ".atomdb");
+}
+
+if (!process.env.ATOM_DB && !fs.existsSync(ATOM_DB)) {
+  try {
+    fs.mkdirSync(ATOM_DB, { recursive: true });
+  } catch (e) {
+    // ignore
+  }
+}
+const args = yargs(hideBin(process.argv))
+  .option("input", {
+    alias: "i",
+    description: "Input SBoM file. Default bom.json",
+    default: "bom.json"
+  })
+  .option("output", {
+    alias: "o",
+    description: "Output file. Default bom.evinse.json",
+    default: "bom.evinse.json"
+  })
+  .option("language", {
+    alias: "l",
+    description: "Application language",
+    default: "java",
+    choices: ["java", "jar", "javascript", "python", "android", "cpp"]
+  })
+  .option("db-path", {
+    description: `Atom slices DB path. Default ${ATOM_DB}`,
+    default: process.env.ATOM_DB || ATOM_DB
+  })
+  .option("force", {
+    description: "Force creation of the database",
+    default: false,
+    type: "boolean"
+  })
+  .option("skip-maven-collector", {
+    description:
+      "Skip collecting jars from maven and gradle caches. Can speedup re-runs if the data was cached previously.",
+    default: false,
+    type: "boolean"
+  })
+  .option("with-deep-jar-collector", {
+    description:
+      "Enable collection of all jars from maven cache directory. Useful to improve the recall for callstack evidence.",
+    default: false,
+    type: "boolean"
+  })
+  .option("annotate", {
+    description: "Include contents of atom slices as annotations",
+    default: true,
+    type: "boolean"
+  })
+  .option("with-data-flow", {
+    description: "Enable inter-procedural data-flow slicing.",
+    default: false,
+    type: "boolean"
+  })
+  .option("usages-slices-file", {
+    description: "Use an existing usages slices file.",
+    default: "usages.slices.json"
+  })
+  .option("data-flow-slices-file", {
+    description: "Use an existing data-flow slices file.",
+    default: "data-flow.slices.json"
+  })
+  .option("print", {
+    alias: "p",
+    type: "boolean",
+    description: "Print the evidences as table"
+  })
+  .scriptName("evinse")
+  .version()
+  .help("h").argv;
+
+const evinseArt = `
+███████╗██╗   ██╗██╗███╗   ██╗███████╗███████╗
+██╔════╝██║   ██║██║████╗  ██║██╔════╝██╔════╝
+█████╗  ██║   ██║██║██╔██╗ ██║███████╗█████╗  
+██╔══╝  ╚██╗ ██╔╝██║██║╚██╗██║╚════██║██╔══╝  
+███████╗ ╚████╔╝ ██║██║ ╚████║███████║███████╗
+╚══════╝  ╚═══╝  ╚═╝╚═╝  ╚═══╝╚══════╝╚══════╝
+`;
+
+console.log(evinseArt);
+(async () => {
+  // First, prepare the database by cataloging jars and other libraries
+  const dbObjMap = await prepareDB(args);
+  if (dbObjMap) {
+    // Analyze the project using atom. Convert package namespaces to purl using the db
+    const sliceArtefacts = await analyzeProject(dbObjMap, args);
+    // Create the SBoM with Evidence
+    const bomJson = createEvinseFile(sliceArtefacts, args);
+    // Validate our final SBoM
+    if (!validateBom(bomJson)) {
+      process.exit(1);
+    }
+    if (args.print) {
+      printOccurrences(bomJson);
+      printCallStack(bomJson);
+      printServices(bomJson);
+    }
+  }
+})();

package/bin/repl.js

@@ -7,18 +7,28 @@ import process from "node:process";
 
 import { createBom } from "../index.js";
 import { validateBom } from "../validator.js";
-import { printTable, printDependencyTree } from "../display.js";
+import {
+  printCallStack,
+  printOccurrences,
+  printTable,
+  printDependencyTree,
+  printServices
+} from "../display.js";
 
 const options = {
   useColors: true,
   breakEvalOnSigint: true,
   preview: true,
-  prompt: "↝ ",
+  prompt: "cdx ↝ ",
   ignoreUndefined: true,
   useGlobal: true
 };
 
-const cdxArt = ` ██████╗██████╗ ██╗  ██╗
+// Use canonical terminal settings to support custom readlines
+process.env.NODE_NO_READLINE = 1;
+
+const cdxArt = `
+ ██████╗██████╗ ██╗  ██╗
 ██╔════╝██╔══██╗╚██╗██╔╝
 ██║     ██║  ██║ ╚███╔╝ 
 ██║     ██║  ██║ ██╔██╗ 
@@ -26,7 +36,7 @@ const cdxArt = ` ██████╗██████╗ ██╗  ██╗
  ╚═════╝╚═════╝ ╚═╝  ╚═╝
 `;
 
-console.log("\n" + cdxArt);
+console.log(cdxArt);
 
 // The current sbom is stored here
 let sbom = undefined;
@@ -133,7 +143,7 @@ cdxgenRepl.defineCommand("sbom", {
   }
 });
 cdxgenRepl.defineCommand("search", {
-  help: "search the current sbom",
+  help: "search the current sbom. performs case insensitive search on various attributes.",
   async action(searchStr) {
     if (sbom) {
       if (searchStr) {
@@ -199,7 +209,7 @@ cdxgenRepl.defineCommand("sort", {
   }
 });
 cdxgenRepl.defineCommand("query", {
-  help: "query the current sbom",
+  help: "query the current sbom using jsonata expression",
   async action(querySpec) {
     if (sbom) {
       if (querySpec) {
@@ -249,7 +259,7 @@ cdxgenRepl.defineCommand("tree", {
   }
 });
 cdxgenRepl.defineCommand("validate", {
-  help: "validate the sbom",
+  help: "validate the sbom using jsonschema",
   action() {
     if (sbom) {
       const result = validateBom(sbom);
@@ -309,3 +319,107 @@ cdxgenRepl.defineCommand("update", {
     this.displayPrompt();
   }
 });
+cdxgenRepl.defineCommand("occurrences", {
+  help: "view components with evidence.occurrences",
+  async action() {
+    if (sbom) {
+      try {
+        const expression = jsonata(
+          "components[$count(evidence.occurrences) > 0]"
+        );
+        let components = await expression.evaluate(sbom);
+        if (!components) {
+          console.log(
+            "No results found. Use evinse command to generate an SBoM with evidence."
+          );
+        } else {
+          if (!Array.isArray(components)) {
+            components = [components];
+          }
+          printOccurrences({ components });
+        }
+      } catch (e) {
+        console.log(e);
+      }
+    } else {
+      console.log(
+        "⚠ No SBoM is loaded. Use .import command to import an evinse SBoM"
+      );
+    }
+    this.displayPrompt();
+  }
+});
+cdxgenRepl.defineCommand("discord", {
+  help: "display the discord invite link for support",
+  async action() {
+    console.log("Head to https://discord.gg/pF4BYWEJcS for support");
+    this.displayPrompt();
+  }
+});
+cdxgenRepl.defineCommand("sponsor", {
+  help: "display the sponsorship link to fund this project",
+  async action() {
+    console.log(
+      "Hey, thanks a lot for considering! https://github.com/sponsors/prabhu"
+    );
+    this.displayPrompt();
+  }
+});
+cdxgenRepl.defineCommand("callstack", {
+  help: "view components with evidence.callstack",
+  async action() {
+    if (sbom) {
+      try {
+        const expression = jsonata(
+          "components[$count(evidence.callstack.frames) > 0]"
+        );
+        let components = await expression.evaluate(sbom);
+        if (!components) {
+          console.log(
+            "callstack evidence was not found. Use evinse command to generate an SBoM with evidence."
+          );
+        } else {
+          if (!Array.isArray(components)) {
+            components = [components];
+          }
+          printCallStack({ components });
+        }
+      } catch (e) {
+        console.log(e);
+      }
+    } else {
+      console.log(
+        "⚠ No SBoM is loaded. Use .import command to import an evinse SBoM"
+      );
+    }
+    this.displayPrompt();
+  }
+});
+cdxgenRepl.defineCommand("services", {
+  help: "view services",
+  async action() {
+    if (sbom) {
+      try {
+        const expression = jsonata("services");
+        let services = await expression.evaluate(sbom);
+        if (!services) {
+          console.log(
+            "No services found. Use evinse command to generate an SBoM with evidence."
+          );
+        } else {
+          if (!Array.isArray(services)) {
+            services = [services];
+          }
+          printServices({ services });
+        }
+      } catch (e) {
+        console.log(e);
+      }
+    } else {
+      console.log(
+        "⚠ No SBoM is loaded. Use .import command to import an evinse SBoM"
+      );
+    }
+    this.displayPrompt();
+  }
+});

package/bin/verify.js

@@ -0,0 +1,39 @@
+#!/usr/bin/env node
+
+import yargs from "yargs";
+import { hideBin } from "yargs/helpers";
+import fs from "node:fs";
+import jws from "jws";
+import process from "node:process";
+
+const args = yargs(hideBin(process.argv))
+  .option("input", {
+    alias: "i",
+    default: "bom.json",
+    description: "Input json to validate. Default bom.json"
+  })
+  .option("public-key", {
+    default: "public.key",
+    description: "Public key in PEM format. Default public.key"
+  })
+  .scriptName("cdx-verify")
+  .version()
+  .help("h").argv;
+
+const bomJson = JSON.parse(fs.readFileSync(args.input, "utf8"));
+const bomSignature = bomJson?.signature?.value;
+if (!bomSignature) {
+  console.log("No signature was found!");
+} else {
+  const validationResult = jws.verify(
+    bomSignature,
+    bomJson.signature.algorithm,
+    fs.readFileSync(args.publicKey, "utf8")
+  );
+  if (validationResult) {
+    console.log("Signature is valid!");
+  } else {
+    console.log("SBoM signature is invalid!");
+    process.exit(1);
+  }
+}

package/db.js

@@ -0,0 +1,80 @@
+import path from "node:path";
+import { Sequelize, DataTypes, Model } from "sequelize";
+import SQLite from "sqlite3";
+
+class Namespaces extends Model {}
+class Usages extends Model {}
+class DataFlows extends Model {}
+
+export const createOrLoad = async (dbName, dbPath, logging = false) => {
+  const sequelize = new Sequelize({
+    define: {
+      freezeTableName: true
+    },
+    dialect: "sqlite",
+    dialectOptions: {
+      mode:
+        SQLite.OPEN_READWRITE |
+        SQLite.OPEN_CREATE |
+        SQLite.OPEN_NOMUTEX |
+        SQLite.OPEN_SHAREDCACHE
+    },
+    storage: dbPath.includes("memory") ? dbPath : path.join(dbPath, dbName),
+    logging,
+    pool: {
+      max: 5,
+      min: 0,
+      acquire: 30000,
+      idle: 10000
+    }
+  });
+  Namespaces.init(
+    {
+      purl: {
+        type: DataTypes.STRING,
+        allowNull: false,
+        primaryKey: true
+      },
+      data: {
+        type: DataTypes.JSON,
+        allowNull: false
+      }
+    },
+    { sequelize, modelName: "Namespaces" }
+  );
+  Usages.init(
+    {
+      purl: {
+        type: DataTypes.STRING,
+        allowNull: false,
+        primaryKey: true
+      },
+      data: {
+        type: DataTypes.JSON,
+        allowNull: false
+      }
+    },
+    { sequelize, modelName: "Usages" }
+  );
+  DataFlows.init(
+    {
+      purl: {
+        type: DataTypes.STRING,
+        allowNull: false,
+        primaryKey: true
+      },
+      data: {
+        type: DataTypes.JSON,
+        allowNull: false
+      }
+    },
+    { sequelize, modelName: "DataFlows" }
+  );
+  await sequelize.sync();
+  return {
+    sequelize,
+    Namespaces,
+    Usages,
+    DataFlows
+  };
+};