owasp-depscan 5.1.4__tar.gz → 5.1.5__tar.gz

{owasp-depscan-5.1.4/owasp_depscan.egg-info → owasp-depscan-5.1.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: owasp-depscan
-Version: 5.1.4
+Version: 5.1.5
 Summary: Fully open-source security audit for project dependencies based on known vulnerabilities and advisories.
 Author-email: Team AppThreat <cloud@appthreat.com>
 License: MIT
@@ -20,7 +20,7 @@ Classifier: Topic :: Utilities
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: appthreat-vulnerability-db>=5.5.7
+Requires-Dist: appthreat-vulnerability-db>=5.5.8
 Requires-Dist: defusedxml
 Requires-Dist: oras
 Requires-Dist: PyYAML
@@ -30,6 +30,8 @@ Requires-Dist: PyGithub
 Requires-Dist: toml
 Requires-Dist: pdfkit
 Requires-Dist: Jinja2
+Requires-Dist: packageurl-python
+Requires-Dist: cvss
 Provides-Extra: dev
 Requires-Dist: black; extra == "dev"
 Requires-Dist: flake8; extra == "dev"
@@ -47,35 +49,36 @@ OWASP dep-scan is a next-generation security and risk audit tool based on known
 [![Discord](https://img.shields.io/badge/-Discord-lime?style=for-the-badge&logo=discord&logoColor=white&color=black)](https://discord.gg/pF4BYWEJcS)
 
 ## Contents
-- [Features](#features)
-    - [Vulnerability Data sources](#vulnerability-data-sources)
-    - [Linux distros](#linux-distros)
-- [Usage](#usage)
-    - [OCI Artifacts via ORAS cli](#oci-artifacts-via-oras-cli)
-    - [Single binary executables](#single-binary-executables)
-    - [Server mode](#server-mode)
-    - [Scanning projects locally (Python version)](#scanning-projects-locally-python-version)
-    - [Scanning containers locally (Python version)](#scanning-containers-locally-python-version)
-    - [Scanning projects locally (Docker container)](#scanning-projects-locally-docker-container)
-- [Supported languages and package format](#supported-languages-and-package-format)
-- [Reachability analysis](#reachability-analysis)
-    - [Example analysis for a Java project](#example-analysis-for-a-java-project)
-    - [Example analysis for a JavaScript project](#example-analysis-for-a-javascript-project)
-- [Customization through environment variables](#customization-through-environment-variables)
-- [GitHub Security Advisory](#github-security-advisory)
-- [Suggest mode](#suggest-mode)
-- [Package Risk audit](#package-risk-audit)
-    - [Automatic adjustment](#automatic-adjustment)
-    - [Configuring weights](#configuring-weights)
-- [Live OS scan](#live-os-scan)
-- [License scan](#license-scan)
-- [Kubernetes and Cloud apps](#kubernetes-and-cloud-apps)
-- [PDF reports](#pdf-reports)
-- [Custom reports](#custom-reports)
-- [Performance tuning](#performance-tuning)
-    - [Use nydus to speed up the initial vdb download](#use-nydus-to-speed-up-the-initial-vdb-download)
-- [Discord support](#discord-support)
-- [License](#license)
+
+-   [Features](#features)
+    -   [Vulnerability Data sources](#vulnerability-data-sources)
+    -   [Linux distros](#linux-distros)
+-   [Usage](#usage)
+    -   [OCI Artifacts via ORAS cli](#oci-artifacts-via-oras-cli)
+    -   [Single binary executables](#single-binary-executables)
+    -   [Server mode](#server-mode)
+    -   [Scanning projects locally (Python version)](#scanning-projects-locally-python-version)
+    -   [Scanning containers locally (Python version)](#scanning-containers-locally-python-version)
+    -   [Scanning projects locally (Docker container)](#scanning-projects-locally-docker-container)
+-   [Supported languages and package format](#supported-languages-and-package-format)
+-   [Reachability analysis](#reachability-analysis)
+    -   [Example analysis for a Java project](#example-analysis-for-a-java-project)
+    -   [Example analysis for a JavaScript project](#example-analysis-for-a-javascript-project)
+-   [Customization through environment variables](#customization-through-environment-variables)
+-   [GitHub Security Advisory](#github-security-advisory)
+-   [Suggest mode](#suggest-mode)
+-   [Package Risk audit](#package-risk-audit)
+    -   [Automatic adjustment](#automatic-adjustment)
+    -   [Configuring weights](#configuring-weights)
+-   [Live OS scan](#live-os-scan)
+-   [License scan](#license-scan)
+-   [Kubernetes and Cloud apps](#kubernetes-and-cloud-apps)
+-   [PDF reports](#pdf-reports)
+-   [Custom reports](#custom-reports)
+-   [Performance tuning](#performance-tuning)
+    -   [Use nydus to speed up the initial vdb download](#use-nydus-to-speed-up-the-initial-vdb-download)
+-   [Discord support](#discord-support)
+-   [License](#license)
 
 ## Features
 
@@ -129,15 +132,18 @@ Use [ORAS cli](https://oras.land/docs/) to download the vulnerability database f
 export VDB_HOME=depscan
 mkdir -p $VDB_HOME
 oras pull ghcr.io/appthreat/vdb:v5 -o $VDB_HOME
+# oras pull ghcr.io/appthreat/vdb-10y:v5 -o $VDB_HOME
 oras pull ghcr.io/owasp-dep-scan/depscan:v4 -o $VDB_HOME
 ```
 
+Use `vdb-10y` which is a larger database with vulnerability data spanning the last 10 years from 2014. In contrast, vdb with a starting year of 2018 is appropriate for most users.
+
 ### Single binary executables
 
 Download the executable binary for your operating system from the [releases page](https://github.com/owasp-dep-scan/depscan-bin/releases). These binary bundle the following:
 
--   dep-scan with Python 3.10
--   cdxgen with Node.js 18
+-   dep-scan with Python 3.11
+-   cdxgen with Node.js 21
 -   cdxgen binary plugins
 
 ```bash
@@ -386,6 +392,8 @@ depscan --profile research -t js -i <source directory> --reports-dir <reports di
 The following environment variables can be used to customise the behaviour.
 
 -   VDB_HOME - Directory to use for caching database. For docker based execution, this directory should get mounted as a volume from the host
+-   VDB_DATABASE_URL - Vulnerability DB URL. Defaults to: ghcr.io/appthreat/vdb:v5
+-   USE_VDB_10Y - Set to true to use the larger 10 year vulnerability database. Default download url: ghcr.io/appthreat/vdb-10y:v5
 
 ## GitHub Security Advisory
 

{owasp-depscan-5.1.4 → owasp-depscan-5.1.5}/README.md

@@ -8,35 +8,36 @@ OWASP dep-scan is a next-generation security and risk audit tool based on known
 [![Discord](https://img.shields.io/badge/-Discord-lime?style=for-the-badge&logo=discord&logoColor=white&color=black)](https://discord.gg/pF4BYWEJcS)
 
 ## Contents
-- [Features](#features)
-    - [Vulnerability Data sources](#vulnerability-data-sources)
-    - [Linux distros](#linux-distros)
-- [Usage](#usage)
-    - [OCI Artifacts via ORAS cli](#oci-artifacts-via-oras-cli)
-    - [Single binary executables](#single-binary-executables)
-    - [Server mode](#server-mode)
-    - [Scanning projects locally (Python version)](#scanning-projects-locally-python-version)
-    - [Scanning containers locally (Python version)](#scanning-containers-locally-python-version)
-    - [Scanning projects locally (Docker container)](#scanning-projects-locally-docker-container)
-- [Supported languages and package format](#supported-languages-and-package-format)
-- [Reachability analysis](#reachability-analysis)
-    - [Example analysis for a Java project](#example-analysis-for-a-java-project)
-    - [Example analysis for a JavaScript project](#example-analysis-for-a-javascript-project)
-- [Customization through environment variables](#customization-through-environment-variables)
-- [GitHub Security Advisory](#github-security-advisory)
-- [Suggest mode](#suggest-mode)
-- [Package Risk audit](#package-risk-audit)
-    - [Automatic adjustment](#automatic-adjustment)
-    - [Configuring weights](#configuring-weights)
-- [Live OS scan](#live-os-scan)
-- [License scan](#license-scan)
-- [Kubernetes and Cloud apps](#kubernetes-and-cloud-apps)
-- [PDF reports](#pdf-reports)
-- [Custom reports](#custom-reports)
-- [Performance tuning](#performance-tuning)
-    - [Use nydus to speed up the initial vdb download](#use-nydus-to-speed-up-the-initial-vdb-download)
-- [Discord support](#discord-support)
-- [License](#license)
+
+-   [Features](#features)
+    -   [Vulnerability Data sources](#vulnerability-data-sources)
+    -   [Linux distros](#linux-distros)
+-   [Usage](#usage)
+    -   [OCI Artifacts via ORAS cli](#oci-artifacts-via-oras-cli)
+    -   [Single binary executables](#single-binary-executables)
+    -   [Server mode](#server-mode)
+    -   [Scanning projects locally (Python version)](#scanning-projects-locally-python-version)
+    -   [Scanning containers locally (Python version)](#scanning-containers-locally-python-version)
+    -   [Scanning projects locally (Docker container)](#scanning-projects-locally-docker-container)
+-   [Supported languages and package format](#supported-languages-and-package-format)
+-   [Reachability analysis](#reachability-analysis)
+    -   [Example analysis for a Java project](#example-analysis-for-a-java-project)
+    -   [Example analysis for a JavaScript project](#example-analysis-for-a-javascript-project)
+-   [Customization through environment variables](#customization-through-environment-variables)
+-   [GitHub Security Advisory](#github-security-advisory)
+-   [Suggest mode](#suggest-mode)
+-   [Package Risk audit](#package-risk-audit)
+    -   [Automatic adjustment](#automatic-adjustment)
+    -   [Configuring weights](#configuring-weights)
+-   [Live OS scan](#live-os-scan)
+-   [License scan](#license-scan)
+-   [Kubernetes and Cloud apps](#kubernetes-and-cloud-apps)
+-   [PDF reports](#pdf-reports)
+-   [Custom reports](#custom-reports)
+-   [Performance tuning](#performance-tuning)
+    -   [Use nydus to speed up the initial vdb download](#use-nydus-to-speed-up-the-initial-vdb-download)
+-   [Discord support](#discord-support)
+-   [License](#license)
 
 ## Features
 
@@ -90,15 +91,18 @@ Use [ORAS cli](https://oras.land/docs/) to download the vulnerability database f
 export VDB_HOME=depscan
 mkdir -p $VDB_HOME
 oras pull ghcr.io/appthreat/vdb:v5 -o $VDB_HOME
+# oras pull ghcr.io/appthreat/vdb-10y:v5 -o $VDB_HOME
 oras pull ghcr.io/owasp-dep-scan/depscan:v4 -o $VDB_HOME
 ```
 
+Use `vdb-10y` which is a larger database with vulnerability data spanning the last 10 years from 2014. In contrast, vdb with a starting year of 2018 is appropriate for most users.
+
 ### Single binary executables
 
 Download the executable binary for your operating system from the [releases page](https://github.com/owasp-dep-scan/depscan-bin/releases). These binary bundle the following:
 
--   dep-scan with Python 3.10
--   cdxgen with Node.js 18
+-   dep-scan with Python 3.11
+-   cdxgen with Node.js 21
 -   cdxgen binary plugins
 
 ```bash
@@ -347,6 +351,8 @@ depscan --profile research -t js -i <source directory> --reports-dir <reports di
 The following environment variables can be used to customise the behaviour.
 
 -   VDB_HOME - Directory to use for caching database. For docker based execution, this directory should get mounted as a volume from the host
+-   VDB_DATABASE_URL - Vulnerability DB URL. Defaults to: ghcr.io/appthreat/vdb:v5
+-   USE_VDB_10Y - Set to true to use the larger 10 year vulnerability database. Default download url: ghcr.io/appthreat/vdb-10y:v5
 
 ## GitHub Security Advisory
 

{owasp-depscan-5.1.4 → owasp-depscan-5.1.5}/depscan/cli.py

@@ -2,22 +2,17 @@
 # -*- coding: utf-8 -*-
 
 import argparse
-from defusedxml.ElementTree import parse
 import json
 import os
-import shutil
-import subprocess
 import sys
-import tarfile
 import tempfile
 
-import oras.client
+from defusedxml.ElementTree import parse
 from quart import Quart, request
 from rich.panel import Panel
 from rich.terminal_theme import DEFAULT_TERMINAL_THEME, MONOKAI
 from vdb.lib import config
 from vdb.lib import db as db_lib
-from vdb.lib.config import data_dir
 from vdb.lib.gha import GitHubSource
 from vdb.lib.nvd import NvdSource
 from vdb.lib.osv import OSVSource
@@ -45,12 +40,11 @@ from depscan.lib.config import (
     UNIVERSAL_SCAN_TYPE,
     license_data_dir,
     spdx_license_list,
-    vdb_database_url,
-    vdb_rafs_database_url,
 )
 from depscan.lib.csaf import export_csaf, write_toml
 from depscan.lib.license import build_license_data, bulk_lookup
 from depscan.lib.logger import DEBUG, LOG, console
+from depscan.lib.orasclient import download_image
 
 try:
     os.environ["PYTHONIOENCODING"] = "utf-8"
@@ -497,69 +491,6 @@ def summarise(
     return summary, vdr_file, pkg_vulnerabilities, pkg_group_rows
 
 
-def download_rafs_based_image():
-    rafs_image_downloaded, paths_list = False, None
-    nydus_image_command = shutil.which("nydus-image", mode=os.X_OK)
-    if nydus_image_command is not None:
-        LOG.info(
-            "About to download the vulnerability database from %s. This might take a while ...",
-            vdb_rafs_database_url,
-        )
-
-        try:
-            oras_client = oras.client.OrasClient()
-            rafs_data_dir = tempfile.TemporaryDirectory()
-            paths_list = oras_client.pull(
-                target=vdb_rafs_database_url, outdir=rafs_data_dir.name
-            )
-
-            if (
-                paths_list
-                and os.path.exists(
-                    os.path.join(rafs_data_dir.name, "data.rafs")
-                )
-                and os.path.exists(
-                    os.path.join(rafs_data_dir.name, "meta.rafs")
-                )
-            ):
-                nydus_download_command = [
-                    f"{nydus_image_command}",
-                    "unpack",
-                    "--blob",
-                    os.path.join(rafs_data_dir.name, "data.rafs"),
-                    "--output",
-                    os.path.join(data_dir, "vdb.tar"),
-                    "--bootstrap",
-                    os.path.join(rafs_data_dir.name, "meta.rafs"),
-                ]
-                _ = subprocess.run(
-                    nydus_download_command,
-                    check=True,
-                    stdout=subprocess.DEVNULL,
-                    stderr=subprocess.DEVNULL,
-                )
-                if os.path.exists(os.path.join(data_dir, "vdb.tar")):
-                    rafs_image_downloaded = True
-                    with tarfile.open(
-                        os.path.join(data_dir, "vdb.tar"), "r"
-                    ) as tar:
-                        tar.extractall(path=data_dir)
-                    os.remove(os.path.join(data_dir, "vdb.tar"))
-                else:
-                    raise FileNotFoundError("vdb.tar not found")
-            else:
-                raise FileNotFoundError("data.rafs or meta.rafs not found")
-
-        except Exception:
-            LOG.info(
-                "Unable to pull the vulnerability database (rafs image) from %s. Trying to pull the non-rafs-based VDB image.",
-                vdb_rafs_database_url,
-            )
-            rafs_image_downloaded = False
-
-    return rafs_image_downloaded, data_dir
-
-
 @app.get("/")
 async def index():
     """
@@ -577,18 +508,17 @@ async def cache():
     """
     db = db_lib.get()
     if not db_lib.index_count(db["index_file"]):
-        rafs_image_downloaded, _ = download_rafs_based_image()
-        if not rafs_image_downloaded:
-            LOG.info(
-                "About to download the vulnerability database from %s. This might take a while ...",
-                vdb_database_url,
-            )
-            oras_client = oras.client.OrasClient()
-            oras_client.pull(target=vdb_database_url, outdir=data_dir)
+        paths_list = download_image()
+        if paths_list:
             return {
                 "error": "false",
                 "message": "vulnerability database cached successfully",
             }
+        else:
+            return {
+                "error": "true",
+                "message": "vulnerability database was not cached",
+            }
     return {
         "error": "false",
         "message": "vulnerability database already exists",
@@ -1036,17 +966,7 @@ def main():
             except Exception:
                 pass
         if run_cacher:
-            rafs_image_downloaded, paths_list = download_rafs_based_image()
-            if not rafs_image_downloaded:
-                LOG.info(
-                    "About to download the vulnerability database from %s. This might take a while ...",
-                    vdb_database_url,
-                )
-                oras_client = oras.client.OrasClient()
-                paths_list = oras_client.pull(
-                    target=vdb_database_url, outdir=data_dir
-                )
-
+            paths_list = download_image()
             LOG.debug("VDB data is stored at: %s", paths_list)
             run_cacher = False
             db = db_lib.get()
@@ -1101,12 +1021,10 @@ def main():
         # CSAF VEX export
         if args.csaf:
             export_csaf(
-                results,
+                pkg_vulnerabilities,
                 src_dir,
                 reports_dir,
-                vdr_file,
-                direct_purls=direct_purls,
-                reached_purls=reached_purls,
+                bom_file,
             )
     console.save_html(
         html_file,