codeaudit 1.5.0__tar.gz → 1.6.0__tar.gz

{codeaudit-1.5.0 → codeaudit-1.6.0}/CHANGELOG.md

@@ -1,5 +1,29 @@
 # Change Log
 
+## Version 1.6.0:
+
+**Added**:
+
+- Option to suppress security weaknesses in files that have a marker, like `#nosec`. Available for both CLI and API functions.
+
+- Extra check for use of subprocess methods: `subprocess.check_call`, `subprocess.check_output`, `subprocess.getoutput` and `subprocess.getstatusoutput`
+
+
+**Changed**:
+
+- Variable name updated to prevent false positives in secrets scan for egress risk.
+
+- Renamed API function get_construct_counts to get_weakness_counts to better reflect its purpose. This API function now also supports suppressing weaknesses that are marked.
+
+**Fixed**:
+
+- Gracefully catch errors when the directory for a custom output report does not exist. Python Code Audit will not create directories due to security constraints.
+
+**Documentation**:
+
+- Various text fixes and improvements in the manual.
+
+
 ## Version 1.5.0: 
 Added:
 

{codeaudit-1.5.0 → codeaudit-1.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codeaudit
-Version: 1.5.0
+Version: 1.6.0
 Summary: Simplified static security checks for Python 
 Project-URL: Documentation, https://github.com/nocomplexity/codeaudit#readme
 Project-URL: Issues, https://github.com/nocomplexity/codeaudit/issues

{codeaudit-1.5.0 → codeaudit-1.6.0}/docs/CLIcommands.ipynb

@@ -167,7 +167,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.13.5"
+   "version": "3.13.11"
   }
  },
  "nbformat": 4,

{codeaudit-1.5.0 → codeaudit-1.6.0}/docs/_config.yml

@@ -86,7 +86,7 @@ sphinx:
   - sphinx.ext.todo
 
   local_extensions          :   # A list of local extensions to load by sphinx specified by "name: path" items  
-#     simplifiednlp :  '/home/maikel/projects/pythondev/simplifiednlp'
+
   config:
     html_show_copyright: false
     html_last_updated_fmt: ""     

{codeaudit-1.5.0 → codeaudit-1.6.0}/docs/_toc.yml

@@ -46,7 +46,7 @@ parts:
     - file: checks/base64_check
     - file: checks/httpserver_check
     - file: checks/multiprocessing_check
-    - file: checks/pickle_check
+    - file: checks/pickle_check    
     - file: checks/random_check
     - file: checks/shelve_check
     - file: checks/syscalls_check
@@ -58,7 +58,8 @@ parts:
 - caption: Architecture 
   chapters:     
   #- file: astlines
-  # - file: astlines2
+  # - file: astlines2  
+  - file: architecture
   - file: makeitbetter
   - file: project_philosophy  
   - file: codeauditcommands  

codeaudit-1.6.0/docs/architecture.md

@@ -0,0 +1,83 @@
+# Architecture Overview
+
+Good security software, especially open-source security software, **should** have a written architecture document. Having an architecture document is vital from a security perspective.
+
+Documenting architecture is crucial for **security, understanding, and maintenance** of the solution by various incidental contributors.
+
+An architecture document:
+
+* Helps developers **understand** how and where to make changes, whether they are new to a project or not.  
+* Has a strong emphasis on the ‘**what**’ and provides boundaries for the ‘**how**’ (the implementation). A detailed implementation description is not part of the architecture document.
+
+Within the IT industry, there is continuous debate about what architecture is and what a design is. Being a TOGAF certified practitioner myself, I like to keep things simple and always make sure that the purpose of any document is clear. Therefore, this document is **not a** detailed design; instead, it covers crucial architecture and design decisions that steer the implementation. The code of Python Code Audit itself is **rich in** comments **outlining** detailed implementation choices where needed.
+
+In my opinion, open-source software for security **should** have an open architecture document. This architecture is:
+
+* **Available to** everyone. This architecture document is part of the FOSS product, Python Code Audit, and is released under a Creative Commons licence.  
+* Part of an open process in which everyone can participate to improve the architecture. See section [CONTRIBUTE](https://nocomplexity.com/documents/codeaudit/CONTRIBUTE.html).
+
+If you are interested in learning more about architecture—and especially open architecture, see:
+- [The Architecture Playbook](https://nocomplexity.com/documents/arplaybook/introduction.html) and
+- [Open Architectures do not work: The need for real open Architectures](http://www.slideshare.net/maikelm/open-architectures-do-not-work-the-need-for-real-open-architectures). 
+
+
+
+## High-Level Design
+
+![Architecture Overview](images/architecture_overview.png)
+
+
+## Architecture Principles
+
+Python Code Audit is built using the following guiding principles:
+
+* **Better safe than sorry:** Python Code Audit adopts a defensive security approach.  
+* **Local-first:** No data leakage and no reliance on third-party services. Security and security data should never be outsourced to a ‘black box’ environment.  
+* **Simple to use:** Python Code Audit is designed for ease of use by anyone, regardless of experience level.  
+* **Simple to extend:** The program must be easy to adapt and build upon for future needs.  
+* **Simple to maintain:** We follow [0Complexity design principles](https://nocomplexity.com/documents/0complexity/abstract.html). E.g. simplicity enhances security. This means minimising dependencies and keeping both design and implementation straightforward and transparent.  
+* **Transparent:** All code is released under the FOSS (Free and Open Source Software) [GPLv3 licence](https://nocomplexity.com/documents/codeaudit/license.html). Transparency builds trust.  
+* **Clear scope:** No tool can do everything well, so we make strong, opinionated choices regarding the functionality we support.
+
+**Implications:**
+
+To maintain this focus, the following are intentionally **out of scope**:
+
+* **No complex checks:** Tasks such as SQL injection detection, TLS certificate validation, or cryptographic misuse analysis are excluded. These areas are difficult to automate reliably and require significant continuous maintenance. We aim to avoid providing a false sense of security.  
+* **No multi-language programming support:** Security validation of other languages (such as PHP, C/C++, or Go) is not provided. This requires different expertise; a SAST tool that claims to support many languages is often of limited use.  
+* **No linting:** We do not perform PEP8 or code quality checking. Linters are designed for these tasks, but they are generally very weak at identifying security weaknesses. You should never rely on linters for security validation.  
+* **No SBOM validation:** Software Composition Analysis (SCA), including SBOM generation and dependency vulnerability checks, is excluded. Validating SBOMs for Python packages can become complex very quickly. We focus on finding weaknesses in the code itself—a distinct aspect of security that offers advantages that SBOM validation cannot provide.
+* **No Web Validations:** This Python Code Audit SAST tool prioritises finding weaknesses in core logic and standard library usage. Modern Python frameworks should be secure by design; utilising their standard APIs effectively mitigates risks such as SQL injection. As security flaws frequently arise from bypassing these built-in protections, web applications should be verified using Dynamic Application Security Testing (DAST) and fuzzing to validate crucial business logic.
+
+
+We focus on delivering a simple, trustworthy security tool that performs its defined tasks exceptionally well—without compromise.
+
+## Design Choices
+
+The following design choices have been made for Python Code Audit:
+
+* **The Python AST library is used for complex validations.**  
+  * **Rationale:** As we are creating a Python-specific security checker, using the `ast` module provides **significant** advantages:  
+    1. Code is not executed during examination, which is a major benefit when validating potentially malicious code.  
+    2. Implementing basic checks using complex regex patterns would make the code and its maintenance unnecessarily difficult.  
+    3. Users can add extra validations in a simple, straightforward manner.  
+* **Python Code Audit is not designed for identifying weaknesses in web applications.**  
+  * **Rationale:** We check Python source code, but do not perform XSS or SQL injection checks. Every Python web application **should** use a battle-tested FOSS framework that prevents these vulnerabilities by design. Testing for these would require building a fuzzer rather than a static code scanner. There are other tools that **must** always be used for validating web applications.  
+* **Postpone code performance optimisations until truly necessary.**  
+  * **Rationale:** We aim for a loosely coupled architecture of key functions; performance optimisations can be introduced at a later stage if required. In practice, performance optimisations are rarely needed. Most time will be spent by humans analysing results before deciding whether to use a Python package, or making security improvements to their own code. The baseline performance for scanning a 10MB Python package should be the priority.  
+  * **Implication:** Ensure that performance optimisations can be applied later to specific functional blocks if they are found to be causing bottlenecks for users.
+
+
+
+## Use of AI 
+
+I love new technology. I also advocate for Free and Open Machine Learning/AI. I think FOSS AI/ML is crucial for everyone. See [FOSS AI/ML Guide](https://nocomplexity.com/documents/fossml/abstract.html).
+
+AI/Machine learning is an exciting and powerful technology. The continuous use and growth of AI and machine learning technology opens new opportunities. It also enables opportunities for solving complex problems in a more simple way.
+
+For Python Code Audit we make use of AI/ML capabilities in a secure, safe and most ethical way possible. 
+
+In the view below is outlined how AI/ML technology is used for the development of Python Code Audit.   
+Truth is: Most AI tools turned out to be of limited value for real trustworthy cybersecurity aspects. Human knowledge work, especially on design and security aspects is currently still vital for developing and maintaining a trustworthy Python security code analyzer!
+
+![AI-use](images/ai_use.png)

codeaudit-1.6.0/docs/checks/subprocess_check.md

@@ -0,0 +1,104 @@
+# Subprocess Statement
+
+Codeaudit checks the use of Subprocesses in a Python file.
+
+Using the default Python library `subprocess` for Subprocess management is very powerfull.
+
+:::{caution} 
+When using the `subprocess` library the Python code can invoke an external executable. 
+:::
+
+
+Python `codeaudit` does not check on `Popen.communicate` This method cannot be directly used on a "foreign" process. It can only work on a launched by subprocess that is created by `Popen` in the same Python script. Popen.communicate() is a method of the Popen object. 
+
+It is good to known that Python's `subprocess` module doesn't provide a mechanism to "attach" to standard I/O streams of an already running process. 
+
+Note that when *old* subprocess functions are used, the severity is high. So avoid using the functions and migratie to `.run` for:
+* `subprocess.call`
+* `subprocess.Popen` 
+* `subprocess.call` 
+* `subprocess.check_call` 
+* `subprocess.check_output` 
+* `subprocess.getoutput` 
+* `subprocess.getstatusoutput` 
+* `subprocess.run` 
+
+## Rationale
+
+The subprocess module allows Python code to execute external system commands. This is powerful — and dangerous — because it creates a bridge between untrusted input and the operating system shell.
+
+Security vulnerabilities typically arise when:
+
+- User-controlled input is interpolated into command strings
+
+- The shell is invoked (shell=True)
+
+- Arguments are passed as strings instead of lists
+
+- Output or exit codes are trusted without validation
+
+- Errors are ignored or mishandled
+
+If an attacker can influence any part of the command, they may achieve command injection, privilege escalation, data exfiltration, or remote code execution (RCE).
+
+All of the listed APIs are safe by default only if used correctly — but very easy to misuse!
+
+
+## Preventive measures
+
+Applies to all subprocess APIs listed:
+
+1. Avoid shell=True
+
+- This is the single biggest risk factor.
+
+- Shell metacharacters (;, &&, |, $(), `) become exploitable.
+
+2. Pass arguments as lists, never strings
+
+- `["ls", "-l"]` instead of `ls -l`
+
+3. Never concatenate user input into commands
+
+- Especially paths, filenames, flags, or filters.
+
+4. Validate and sanitize all inputs
+
+- Use allowlists, not blocklists.
+
+5. Check return codes explicitly
+
+- Silent failures can mask partial command execution.
+
+6. Use the least-privileged user possible
+
+- Never run subprocesses as root unless unavoidable.
+
+7. Prefer high-level Python APIs
+
+- Many shell commands have safe Python equivalents (os, pathlib, shutil).
+
+## Example (by API)
+
+### `subprocess.call`
+
+**Why it’s risky**
+
+- Executes commands without raising exceptions
+- Return codes are often ignored
+- Vulnerable when `shell=True` or when strings are used
+
+**Bad example**
+```python
+subprocess.call("rm -rf " + user_input, shell=True)
+```
+
+Risk:
+```bash
+user_input = "/tmp/data; rm -rf /"
+```
+
+
+## More information
+
+* https://docs.python.org/3/library/subprocess.html

{codeaudit-1.5.0 → codeaudit-1.6.0}/docs/codeauditchecks.md

@@ -11,7 +11,7 @@ codeaudit checks
 
 This will generate a HTML report, including the version of Python Code Audit that is installed.
 
-Example output as HTML [click to see](examples/checks.html), or see below as embedded HTML:
+All implemented static security validations can be found [here](examples/codeauditchecks.html).
 
 :::{attention} 
 Always use the latest version to have the latest security validations!

{codeaudit-1.5.0 → codeaudit-1.6.0}/docs/codeauditcommands.md

@@ -133,28 +133,33 @@ errors defaults to 'strict'.
 ## codeaudit filescan
 ```text
 Scans Python source code or PyPI packages for security weaknesses.
-
 This function performs static application security testing (SAST) on a
-given input, which can be:
+specified input. The input can be one of the following:
 
-- A local directory containing Python source code
-- A single local Python file 
-- A package name hosted on PyPI.org
+* A local directory containing Python source code
+* A single local Python file
+* The name of a package hosted on PyPI
 
-codeaudit filescan <pythonfile|package-name|directory> [reportname.html]
+codeaudit filescan <pythonfile|package-name|directory> [reportname.html] [--nosec]
 
-Depending on the input type, the function analyzes the source code for
-potential security issues, generates an HTML report summarizing the
-findings, and writes the report to a static HTML file.
+Based on the input type, the function analyzes the source code for potential
+security issues, generates an HTML report summarizing the findings, and
+writes the report to disk.
 
 If a PyPI package name is provided, the function downloads the source
-distribution (sdist), scans the extracted source code, and removes all
-temporary files after the scan completes.
+distribution (sdist), extracts it to a temporary directory, scans the
+extracted source code, and cleans up all temporary files after the scan
+completes.
+
+ Examples:
 
-Example:
     Scan a local directory and write the report to ``report.html``::
 
-        codeaudit filescan_/shitwork/custompythonmodule/ 
+        codeaudit filescan /path/to/custompythonmodule report.html
+
+    Scan a local directory::
+
+        codeaudit filescan /path/to/project
 
     Scan a single Python file::
 
@@ -162,22 +167,53 @@ Example:
 
     Scan a package hosted on PyPI::
 
-        codeaudit filescan linkaudit  #A nice project to check broken links in markdown files
+        codeaudit filescan linkaudit
 
         codeaudit filescan requests
 
+
+    Specify an output report file::
+
+        codeaudit filescan /path/to/project report.html
+
+    Enable filtering of issues marked with ``#nosec`` or another marker on potential code weaknesses that mitigated or known  ::
+
+        codeaudit filescan myexample.py --nosec
+
+POSITIONAL ARGUMENTS
+INPUT_PATH
+    Path to a local Python file or directory, or the name of a package available on PyPI.
+
+
+FLAGS
+-f, --filename=FILENAME
+    Default: 'codeaudit-report.html'
+-n, --nosec=NOSEC
+    Default: False
+
+            
 Args:
+
+-f, --filename=FILENAME
+    Default: 'codeaudit-report.html'
+    Name (and optional path) of the HTML file to write the scan report to. The filename should use the ``.html`` extension. Defaults to ``DEFAULT_OUTPUT_FILE``.
+-n, --nosec=NOSEC
+    Default: False
+    Whether to filter out issues marked as reviewed or ignored in the source code. Defaults to ``False``, no filtering.
+
     input_path (str): Path to a local Python file or directory, or the name
-        of a package available on PyPI.org.
+        of a package available on PyPI.
     filename (str, optional): Name (and optional path) of the HTML file to
         write the scan report to. The filename should use the ``.html``
         extension. Defaults to ``DEFAULT_OUTPUT_FILE``.
+    nosec (bool, optional): Whether to filter out issues marked as reviewed
+        or ignored in the source code. Defaults to ``False``, no filtering.
 
 Returns:
-    None. The function writes a static HTML security report to disk.
+    None: The function writes a static HTML security report to disk.
 
 Raises:
-    None explicitly. Errors and invalid inputs are reported to stdout.    
+    None: Errors and invalid inputs are reported to stdout.    
 str(object='') -> str
 str(bytes_or_buffer[, encoding[, errors]]) -> str
 

{codeaudit-1.5.0 → codeaudit-1.6.0}/docs/codeauditoverview.md

@@ -69,21 +69,49 @@ Example of an overview plot:
 
 ```text
 NAME
-    codeaudit overview - Reports Complexity and statistics per Python file from a directory.
+    codeaudit overview - Generates an overview report of code complexity and security indicators.
 
 SYNOPSIS
     codeaudit overview DIRECTORY <flags>
 
 DESCRIPTION
-    Reports Complexity and statistics per Python file from a directory.
+    This function analyzes a Python project to produce a high-level overview of
+    complexity and security-related metrics. The input may be either:
+
+    - A local directory containing Python source files
+    - The name of a package hosted on PyPI.org
+
+    So:
+    codeaudit overview <package-name|directory> [reportname.html]
+
+    For PyPI packages, the source distribution (sdist) is downloaded,
+    extracted to a temporary directory, scanned, and removed after the report
+    is generated.
+
+    The report includes summary statistics, security risk indicators based on
+    complexity and total lines of code, a list of discovered modules, per-file
+    metrics, and a visual overview. Results are written to a static HTML file.
+
+    Examples:
+        Generate an overview report for a local project directory::
+
+            codeaudit overview /projects/mycolleaguesproject
+
+        Generate an overview report for a PyPI package::
+
+            codeaudit overview linkaudit #A nice project on PyPI.org
+
+            codeaudit overview pydantic  #A complex project on PyPI.org from a security perspective?
 
 POSITIONAL ARGUMENTS
     DIRECTORY
-        Path to the directory to scan.
+        Path to a local directory containing Python source files or the name of a package available on PyPI.org.
 
 FLAGS
     -f, --filename=FILENAME
         Default: 'codeaudit-report.html'
-        Output filename for the HTML report.
+        Name (and optional path) of the HTML file to write the overview report to. The filename should use the ``.html`` extension. Defaults to ``DEFAULT_OUTPUT_FILE``.
 
+NOTES
+    You can also use flags syntax for POSITIONAL ARGUMENTS
 ```

codeaudit-1.6.0/docs/examples/ca_checks.ipynb

@@ -0,0 +1,53 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "67936457-a2c4-40e3-bf53-93c8b5186fb6",
+   "metadata": {},
+   "source": [
+    "# Python Code Audit Implemented validations"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "d930de74-74c9-457a-b46e-a444ccb941c3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from codeaudit import codeaudit"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "edf3050b-a0df-4d6b-b9af-5470d89a8c8d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!codeaudit checks \"codeauditchecks.html\""
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.13.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}