devskill 2.0.7 → 2.0.8

package/skills/codeql/references/compiled-languages.md

@@ -0,0 +1,284 @@
+# CodeQL Build Modes for Compiled Languages
+
+Detailed reference for how CodeQL handles compiled language analysis, including build modes, autobuild behavior, runner requirements, and hardware specifications.
+
+## Build Modes Overview
+
+CodeQL offers three build modes for compiled languages:
+
+| Mode | Description | When to Use |
+|---|---|---|
+| `none` | Analyze source without building. Dependencies inferred heuristically. | Default setup; quick scans; interpreted-like analysis |
+| `autobuild` | Automatically detect and run the build system. | When `none` produces inaccurate results; when Kotlin code is present |
+| `manual` | User provides explicit build commands. | Complex build systems; autobuild failures; custom build requirements |
+
+## C/C++
+
+### Supported Build Modes
+`none`, `autobuild`, `manual`
+
+**Default setup mode:** `none`
+
+### No Build (`none`)
+- Infers compilation units through source file extensions
+- Compilation flags and include paths inferred by inspecting the codebase
+- No working build command needed
+
+**Accuracy considerations:**
+- May be less accurate if code depends heavily on custom macros/defines not in existing headers
+- May miss accuracy when codebase has many external dependencies
+
+**Improving accuracy:**
+- Place custom macros/defines in header files included by source files
+- Ensure external dependencies (headers) are available in system include directories or workspace
+- Run extraction on the target platform (e.g., Windows runner for Windows projects)
+
+### Autobuild
+
+**Windows autodetection:**
+1. Invoke `MSBuild.exe` on `.sln` or `.vcxproj` closest to root
+2. If multiple files at same depth, attempts to build all
+3. Falls back to build scripts: `build.bat`, `build.cmd`, `build.exe`
+
+**Linux/macOS autodetection:**
+1. Look for build system in root directory
+2. If not found, search subdirectories for unique build system
+3. Run appropriate configure/build command
+
+**Supported build systems:** MSBuild, Autoconf, Make, CMake, qmake, Meson, Waf, SCons, Linux Kbuild, build scripts
+
+### Runner Requirements (C/C++)
+- **Ubuntu:** `gcc` compiler; may need `clang` or `msvc`. Build tools: `msbuild`, `make`, `cmake`, `bazel`. Utilities: `python`, `perl`, `lex`, `yacc`.
+- **Auto-install dependencies:** Set `CODEQL_EXTRACTOR_CPP_AUTOINSTALL_DEPENDENCIES=true` (enabled by default on GitHub-hosted; disabled on self-hosted). Requires Ubuntu with passwordless `sudo apt-get`.
+- **Windows:** `powershell.exe` in PATH
+
+## C\#
+
+### Supported Build Modes
+`none`, `autobuild`, `manual`
+
+**Default setup mode:** `none`
+
+### No Build (`none`)
+- Restores dependencies using heuristics from: `*.csproj`, `*.sln`, `nuget.config`, `packages.config`, `global.json`, `project.assets.json`
+- Uses private NuGet feeds if configured for the organization
+- Generates additional source files for accuracy:
+  - Global `using` directives (implicit `using` feature)
+  - ASP.NET Core `.cshtml` → `.cs` conversion
+
+**Accuracy considerations:**
+- Requires internet access or private NuGet feed
+- Multiple versions of same NuGet dependency may cause issues (CodeQL picks newer version)
+- Multiple .NET framework versions may affect accuracy
+- Colliding class names cause missing method call targets
+
+### Autobuild
+
+**Windows autodetection:**
+1. `dotnet build` on `.sln` or `.csproj` closest to root
+2. `MSBuild.exe` on solution/project files
+3. Build scripts: `build.bat`, `build.cmd`, `build.exe`
+
+**Linux/macOS autodetection:**
+1. `dotnet build` on `.sln` or `.csproj` closest to root
+2. `MSbuild` on solution/project files
+3. Build scripts: `build`, `build.sh`
+
+### Injected Compiler Flags (Manual Builds)
+
+The CodeQL tracer injects these flags into C# compiler invocations:
+
+| Flag | Purpose |
+|---|---|
+| `/p:MvcBuildViews=true` | Precompile ASP.NET MVC views for security analysis |
+| `/p:UseSharedCompilation=false` | Disable shared compilation server (required for tracer inspection) |
+| `/p:EmitCompilerGeneratedFiles=true` | Write generated source files to disk for extraction |
+
+> `/p:EmitCompilerGeneratedFiles=true` may cause issues with legacy projects or `.sqlproj` files.
+
+### Runner Requirements (C#)
+- **.NET Core:** .NET SDK (for `dotnet`)
+- **.NET Framework (Windows):** Microsoft Build Tools + NuGet CLI
+- **.NET Framework (Linux/macOS):** Mono Runtime (`mono`, `msbuild`, `nuget`)
+- **`build-mode: none`:** Requires internet access or private NuGet feed
+
+## Go
+
+### Supported Build Modes
+`autobuild`, `manual` (no `none` mode)
+
+**Default setup mode:** `autobuild`
+
+### Autobuild
+
+Autodetection sequence:
+1. Invoke `make`, `ninja`, `./build`, or `./build.sh` until one succeeds and `go list ./...` works
+2. If none succeed, look for `go.mod` (`go get`), `Gopkg.toml` (`dep ensure -v`), or `glide.yaml` (`glide install`)
+3. If no dependency managers found, rearrange directory for `GOPATH` and use `go get`
+4. Extract all Go code (similar to `go build ./...`)
+
+**Default setup** automatically detects `go.mod` and installs compatible Go version.
+
+### Extractor Options
+
+| Environment Variable | Default | Description |
+|---|---|---|
+| `CODEQL_EXTRACTOR_GO_OPTION_EXTRACT_TESTS` | `false` | Include `_test.go` files in analysis |
+| `CODEQL_EXTRACTOR_GO_OPTION_EXTRACT_VENDOR_DIRS` | `false` | Include `vendor/` directories |
+
+## Java/Kotlin
+
+### Supported Build Modes
+- **Java:** `none`, `autobuild`, `manual`
+- **Kotlin:** `autobuild`, `manual` (no `none` mode)
+
+**Default setup mode:**
+- Java only: `none`
+- Kotlin or Java+Kotlin: `autobuild`
+
+> If Kotlin code is added to a repo using `none` mode, disable and re-enable default setup to switch to `autobuild`.
+
+### No Build (`none`) — Java Only
+- Runs Gradle or Maven for dependency information (not actual build)
+- Queries each root build file; prefers newer dependency versions on clash
+- Uses private Maven registries if configured
+
+**Accuracy considerations:**
+- Build scripts that can't be queried for dependencies may cause inaccurate guesses
+- Code generated during normal build process will be missed
+- Multiple versions of same dependency (CodeQL picks newer)
+- Multiple JDK versions — CodeQL uses highest found; lower-version files may be partially analyzed
+- Colliding class names cause missing method call targets
+
+### Autobuild
+
+**Autodetection sequence:**
+1. Search root directory for Gradle, Maven, Ant build files
+2. Run first found (Gradle preferred over Maven)
+3. Otherwise, search for build scripts
+
+**Build systems:** Gradle, Maven, Ant
+
+### Runner Requirements (Java)
+- JDK (appropriate version for the project)
+- Gradle and/or Maven
+- Internet access or private artifact repository (for `none` mode)
+
+## Rust
+
+### Supported Build Modes
+`none`, `autobuild`, `manual`
+
+**Default setup mode:** `none`
+
+## Swift
+
+### Supported Build Modes
+`autobuild`, `manual` (no `none` mode)
+
+**Default setup mode:** `autobuild`
+
+**Runner requirement:** macOS runners only. Not supported on Actions Runner Controller (ARC) — Linux only.
+
+> macOS runners are more expensive; consider scanning only the build step to optimize cost.
+
+## Multi-Language Matrix Examples
+
+### Mixed Build Modes
+
+```yaml
+strategy:
+  fail-fast: false
+  matrix:
+    include:
+      - language: c-cpp
+        build-mode: manual
+      - language: csharp
+        build-mode: autobuild
+      - language: java-kotlin
+        build-mode: none
+```
+
+### Conditional Manual Build Steps
+
+```yaml
+steps:
+  - name: Checkout
+    uses: actions/checkout@v4
+
+  - name: Initialize CodeQL
+    uses: github/codeql-action/init@v4
+    with:
+      languages: ${{ matrix.language }}
+      build-mode: ${{ matrix.build-mode }}
+
+  - if: matrix.build-mode == 'manual'
+    name: Build C/C++ code
+    run: |
+      make bootstrap
+      make release
+
+  - name: Perform CodeQL Analysis
+    uses: github/codeql-action/analyze@v4
+    with:
+      category: "/language:${{ matrix.language }}"
+```
+
+### OS-Specific Runners
+
+```yaml
+strategy:
+  fail-fast: false
+  matrix:
+    include:
+      - language: javascript-typescript
+        build-mode: none
+        runner: ubuntu-latest
+      - language: swift
+        build-mode: autobuild
+        runner: macos-latest
+      - language: csharp
+        build-mode: autobuild
+        runner: windows-latest
+
+jobs:
+  analyze:
+    runs-on: ${{ matrix.runner }}
+```
+
+## Hardware Requirements
+
+### Recommended Specifications (Self-Hosted Runners)
+
+| Codebase Size | Lines of Code | RAM | CPU Cores | Disk |
+|---|---|---|---|---|
+| Small | < 100K | 8 GB+ | 2 | SSD, ≥14 GB |
+| Medium | 100K – 1M | 16 GB+ | 4–8 | SSD, ≥14 GB |
+| Large | > 1M | 64 GB+ | 8 | SSD, ≥14 GB |
+
+### Performance Tips
+- Use SSD storage for all codebase sizes
+- Ensure enough disk space for checkout + build + CodeQL data
+- Use `--threads=0` to use all available CPU cores
+- Enable dependency caching to reduce analysis time
+- Consider `none` build mode where accuracy is acceptable — significantly faster than `autobuild`
+
+## Dependency Caching
+
+### Advanced Setup Workflows
+
+```yaml
+- uses: github/codeql-action/init@v4
+  with:
+    languages: java-kotlin
+    dependency-caching: true
+```
+
+| Value | Behavior |
+|---|---|
+| `false` / `none` / `off` | Disabled (default for advanced setup) |
+| `restore` | Restore existing caches only |
+| `store` | Store new caches only |
+| `true` / `full` / `on` | Restore and store caches |
+
+Default setup on GitHub-hosted runners has caching enabled automatically.

package/skills/codeql/references/sarif-output.md

@@ -0,0 +1,265 @@
+# CodeQL SARIF Output Reference
+
+Detailed reference for the SARIF v2.1.0 output produced by CodeQL analysis. Use this when interpreting or processing CodeQL scan results.
+
+## About SARIF
+
+SARIF (Static Analysis Results Interchange Format) is a standardized JSON format for representing static analysis tool output. CodeQL produces SARIF v2.1.0 (specification: `sarifv2.1.0`).
+
+- Specification: [OASIS SARIF v2.1.0](https://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html)
+- Schema: [sarif-schema-2.1.0.json](https://docs.oasis-open.org/sarif/sarif/v2.1.0/errata01/os/schemas/sarif-schema-2.1.0.json)
+- Format type: `sarifv2.1.0` (passed to `--format` flag)
+
+## Top-Level Structure
+
+### `sarifLog` Object
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `$schema` | ✅ | Link to the SARIF schema |
+| `version` | ✅ | SARIF specification version (`"2.1.0"`) |
+| `runs` | ✅ | Array containing a single `run` object per language |
+
+### `run` Object
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `tool` | ✅ | Tool information (`toolComponent`) |
+| `artifacts` | ✅ | Array of artifact objects for every file referenced in a result |
+| `results` | ✅ | Array of `result` objects |
+| `newLineSequences` | ✅ | Newline character sequences |
+| `columnKind` | ✅ | Column counting method |
+| `properties` | ✅ | Contains `semmle.formatSpecifier` identifying the format |
+
+## Tool Information
+
+### `tool` Object
+
+Contains a single `driver` property.
+
+### `toolComponent` Object (Driver)
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `name` | ✅ | `"CodeQL command-line toolchain"` |
+| `organization` | ✅ | `"GitHub"` |
+| `version` | ✅ | CodeQL release version (e.g., `"2.19.0"`) |
+| `rules` | ✅ | Array of `reportingDescriptor` objects for available/run rules |
+
+## Rules
+
+### `reportingDescriptor` Object (Rule)
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `id` | ✅ | Rule identifier from `@id` query property (e.g., `cpp/unsafe-format-string`). Uses `@opaqueid` if defined. |
+| `name` | ✅ | Same as `@id` property from the query |
+| `shortDescription` | ✅ | From `@name` query property |
+| `fullDescription` | ✅ | From `@description` query property |
+| `defaultConfiguration` | ❌ | `reportingConfiguration` with `enabled` (true/false) and `level` based on `@severity`. Omitted if no `@severity` specified. |
+
+### Severity Mapping
+
+| CodeQL `@severity` | SARIF `level` |
+|---|---|
+| `error` | `error` |
+| `warning` | `warning` |
+| `recommendation` | `note` |
+
+## Results
+
+### `result` Object
+
+By default, results are grouped by unique message format string and primary location. Two results at the same location with the same message appear as a single result. Disable grouping with `--ungroup-results`.
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `ruleId` | ✅ | Rule identifier (matches `reportingDescriptor.id`) |
+| `ruleIndex` | ✅ | Index into the `rules` array |
+| `message` | ✅ | Problem description. May contain SARIF "Message with placeholder" linking to `relatedLocations`. |
+| `locations` | ✅ | Array containing a single `location` object |
+| `partialFingerprints` | ✅ | Dictionary with at least `primaryLocationLineHash` for deduplication |
+| `codeFlows` | ❌ | Populated for `@kind path-problem` queries with one or more `codeFlow` objects |
+| `relatedLocations` | ❌ | Populated when message has placeholder options; each unique location included once |
+| `suppressions` | ❌ | If suppressed: single `suppression` object with `@kind: IN_SOURCE`. If not suppressed but other results are: empty array. Otherwise: not set. |
+
+### Fingerprints
+
+`partialFingerprints` contains:
+- `primaryLocationLineHash` — fingerprint based on the context of the primary location
+
+Used by GitHub to track alerts across commits and avoid duplicate notifications.
+
+## Locations
+
+### `location` Object
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `physicalLocation` | ✅ | Physical file location |
+| `id` | ❌ | Present in `relatedLocations` array |
+| `message` | ❌ | Present in `relatedLocations` and `threadFlowLocation.location` |
+
+### `physicalLocation` Object
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `artifactLocation` | ✅ | File reference |
+| `region` | ❌ | Present for text file locations |
+| `contextRegion` | ❌ | Present when location has an associated snippet |
+
+### `region` Object
+
+Two types of regions may be produced:
+
+**Line/Column Offset Regions:**
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `startLine` | ✅ | Starting line number |
+| `startColumn` | ❌ | Omitted if equal to default value of 1 |
+| `endLine` | ❌ | Omitted if identical to `startLine` |
+| `endColumn` | ✅ | Ending column number |
+| `snippet` | ❌ | Source code snippet |
+
+**Character Offset Regions:**
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `charOffset` | ✅ | Character offset from start of file |
+| `charLength` | ✅ | Length in characters |
+| `snippet` | ❌ | Source code snippet |
+
+> Consumers should handle both region types robustly.
+
+## Artifacts
+
+### `artifact` Object
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `location` | ✅ | `artifactLocation` object |
+| `index` | ✅ | Index of the artifact |
+| `contents` | ❌ | Populated with `artifactContent` when using `--sarif-add-file-contents` |
+
+### `artifactLocation` Object
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `uri` | ✅ | File path (relative or absolute) |
+| `index` | ✅ | Index reference |
+| `uriBaseId` | ❌ | Set when file is relative to a known abstract location (e.g., source root) |
+
+## Code Flows (Path Problems)
+
+For queries of `@kind path-problem`, results include code flow information showing the data flow path.
+
+### `codeFlow` Object
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `threadFlows` | ✅ | Array of `threadFlow` objects |
+
+### `threadFlow` Object
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `locations` | ✅ | Array of `threadFlowLocation` objects |
+
+### `threadFlowLocation` Object
+
+| Property | Always Generated | Description |
+|---|:---:|---|
+| `location` | ✅ | A `location` object for this step in the flow |
+
+## Automation Details
+
+The `category` value from `github/codeql-action/analyze` appears as `<run>.automationDetails.id` in the SARIF output.
+
+Example:
+```json
+{
+  "automationDetails": {
+    "id": "/language:javascript-typescript"
+  }
+}
+```
+
+## Key CLI Flags for SARIF
+
+| Flag | Effect |
+|---|---|
+| `--format=sarif-latest` | Produce SARIF v2.1.0 output |
+| `--sarif-category=<cat>` | Set `automationDetails.id` for result categorization |
+| `--sarif-add-file-contents` | Include source file content in `artifact.contents` |
+| `--ungroup-results` | Report every occurrence separately (no deduplication by location + message) |
+| `--output=<file>` | Write SARIF to specified file |
+
+## Third-Party SARIF Support
+
+When uploading SARIF from non-CodeQL tools, ensure these properties are populated for best results on GitHub.
+
+### Recommended `reportingDescriptor` Properties
+
+| Property | Required | Description |
+|---|:---:|---|
+| `id` | ✅ | Unique rule identifier |
+| `name` | ❌ | Rule name (max 255 chars) |
+| `shortDescription.text` | ✅ | Concise description (max 1024 chars) |
+| `fullDescription.text` | ✅ | Full description (max 1024 chars) |
+| `defaultConfiguration.level` | ❌ | Default severity: `note`, `warning`, `error` |
+| `help.text` | ✅ | Documentation in text format |
+| `help.markdown` | ❌ | Documentation in Markdown (displayed if available) |
+| `properties.tags[]` | ❌ | Tags for filtering (e.g., `security`) |
+| `properties.precision` | ❌ | `very-high`, `high`, `medium`, `low` — affects display ordering |
+| `properties.problem.severity` | ❌ | Non-security severity: `error`, `warning`, `recommendation` |
+| `properties.security-severity` | ❌ | Score 0.0–10.0 for security queries. Maps to: >9.0=critical, 7.0–8.9=high, 4.0–6.9=medium, 0.1–3.9=low |
+
+### Source File Location Requirements
+
+- Use relative paths (relative to repository root) when possible
+- Absolute URIs are converted to relative using the source root
+- Source root can be set via:
+  - `checkout_path` input to `github/codeql-action/analyze`
+  - `checkout_uri` parameter to SARIF upload API
+  - `invocations[0].workingDirectory.uri` in the SARIF file
+- Consistent file paths are required across runs for fingerprint stability
+- Symlinked files must use resolved (non-symlink) URIs
+
+### Fingerprint Requirements
+
+- `partialFingerprints` with `primaryLocationLineHash` prevents duplicate alerts across commits
+- CodeQL SARIF automatically includes fingerprints
+- Third-party SARIF: the `upload-sarif` action computes fingerprints if missing
+- API uploads without fingerprints may produce duplicate alerts
+
+## Upload Limits
+
+### File Size
+- Maximum: **10 MB** (gzip-compressed)
+- If too large: reduce query scope, remove `--sarif-add-file-contents`, or split into multiple uploads
+
+### Object Count Limits
+
+| Object | Maximum |
+|---|---|
+| Runs per file | 20 |
+| Results per run | 25,000 |
+| Rules per run | 25,000 |
+| Tool extensions per run | 100 |
+| Thread flow locations per result | 10,000 |
+| Locations per result | 1,000 |
+| Tags per rule | 20 |
+
+Files exceeding these limits are rejected. Split analysis across multiple SARIF uploads with different `--sarif-category` values.
+
+### Validation
+
+Validate SARIF files before upload using the [Microsoft SARIF validator](https://sarifweb.azurewebsites.net/).
+
+## Backwards Compatibility
+
+- Fields marked "always generated" will never be removed in future versions
+- Fields not always generated may change circumstances under which they appear
+- New fields may be added without breaking changes
+- Consumers should be robust to both presence and absence of optional fields