cli-jaw 0.1.7 → 0.1.9

package/skills_ref/static-analysis/skills/codeql/references/diagnostic-query-templates.md

@@ -1,339 +0,0 @@
-# Diagnostic Query Templates
-
-Language-specific QL queries for enumerating sources and sinks recognized by CodeQL. Used by the [create-data-extensions workflow](../workflows/create-data-extensions.md).
-
-## Source Enumeration Query
-
-All languages use the class `RemoteFlowSource`. The import differs per language.
-
-### Import Reference
-
-| Language | Imports | Class |
-|----------|---------|-------|
-| Python | `import python` + `import semmle.python.dataflow.new.RemoteFlowSources` | `RemoteFlowSource` |
-| JavaScript | `import javascript` | `RemoteFlowSource` |
-| Java | `import java` + `import semmle.code.java.dataflow.FlowSources` | `RemoteFlowSource` |
-| Go | `import go` | `RemoteFlowSource` |
-| C/C++ | `import cpp` + `import semmle.code.cpp.security.FlowSources` | `RemoteFlowSource` |
-| C# | `import csharp` + `import semmle.code.csharp.security.dataflow.flowsources.Remote` | `RemoteFlowSource` |
-| Ruby | `import ruby` + `import codeql.ruby.dataflow.RemoteFlowSources` | `RemoteFlowSource` |
-
-### Template (Python — swap imports per table above)
-
-```ql
-/**
- * @name List recognized dataflow sources
- * @description Enumerates all locations CodeQL recognizes as dataflow sources
- * @kind problem
- * @id custom/list-sources
- */
-import python
-import semmle.python.dataflow.new.RemoteFlowSources
-
-from RemoteFlowSource src
-select src,
-  src.getSourceType()
-    + " | " + src.getLocation().getFile().getRelativePath()
-    + ":" + src.getLocation().getStartLine().toString()
-```
-
-**Note:** `getSourceType()` is available on Python, Java, and C#. For Go, JavaScript, Ruby, and C++ replace the select with:
-```ql
-select src,
-  src.getLocation().getFile().getRelativePath()
-    + ":" + src.getLocation().getStartLine().toString()
-```
-
----
-
-## Sink Enumeration Queries
-
-The Concepts API differs significantly across languages. Use the correct template.
-
-### Concept Class Reference
-
-| Concept | Python | JavaScript | Go | Ruby |
-|---------|--------|------------|-----|------|
-| SQL | `SqlExecution.getSql()` | `DatabaseAccess.getAQueryArgument()` | `SQL::QueryString` (is-a Node) | `SqlExecution.getSql()` |
-| Command exec | `SystemCommandExecution.getCommand()` | `SystemCommandExecution.getACommandArgument()` | `SystemCommandExecution.getCommandName()` | `SystemCommandExecution.getAnArgument()` |
-| File access | `FileSystemAccess.getAPathArgument()` | `FileSystemAccess.getAPathArgument()` | `FileSystemAccess.getAPathArgument()` | `FileSystemAccess.getAPathArgument()` |
-| HTTP client | `Http::Client::Request.getAUrlPart()` | — | — | — |
-| Decoding | `Decoding.getAnInput()` | — | — | — |
-| XML parsing | — | — | — | `XmlParserCall.getAnInput()` |
-
-### Python
-
-```ql
-/**
- * @name List recognized dataflow sinks
- * @description Enumerates security-relevant sinks CodeQL recognizes
- * @kind problem
- * @id custom/list-sinks
- */
-import python
-import semmle.python.Concepts
-
-from DataFlow::Node sink, string kind
-where
-  exists(SqlExecution e | sink = e.getSql() and kind = "sql-execution")
-  or
-  exists(SystemCommandExecution e |
-    sink = e.getCommand() and kind = "command-execution"
-  )
-  or
-  exists(FileSystemAccess e |
-    sink = e.getAPathArgument() and kind = "file-access"
-  )
-  or
-  exists(Http::Client::Request r |
-    sink = r.getAUrlPart() and kind = "http-request"
-  )
-  or
-  exists(Decoding d | sink = d.getAnInput() and kind = "decoding")
-  or
-  exists(CodeExecution e | sink = e.getCode() and kind = "code-execution")
-select sink,
-  kind
-    + " | " + sink.getLocation().getFile().getRelativePath()
-    + ":" + sink.getLocation().getStartLine().toString()
-```
-
-### JavaScript / TypeScript
-
-```ql
-/**
- * @name List recognized dataflow sinks
- * @description Enumerates security-relevant sinks CodeQL recognizes
- * @kind problem
- * @id custom/list-sinks-js
- */
-import javascript
-
-from DataFlow::Node sink, string kind
-where
-  exists(DatabaseAccess e |
-    sink = e.getAQueryArgument() and kind = "database-access"
-  )
-  or
-  exists(SystemCommandExecution e |
-    sink = e.getACommandArgument() and kind = "command-execution"
-  )
-  or
-  exists(FileSystemAccess e |
-    sink = e.getAPathArgument() and kind = "file-access"
-  )
-select sink,
-  kind
-    + " | " + sink.getLocation().getFile().getRelativePath()
-    + ":" + sink.getLocation().getStartLine().toString()
-```
-
-### Go
-
-```ql
-/**
- * @name List recognized dataflow sinks
- * @description Enumerates security-relevant sinks CodeQL recognizes
- * @kind problem
- * @id custom/list-sinks-go
- */
-import go
-import semmle.go.frameworks.SQL
-
-from DataFlow::Node sink, string kind
-where
-  sink instanceof SQL::QueryString and kind = "sql-query"
-  or
-  exists(SystemCommandExecution e |
-    sink = e.getCommandName() and kind = "command-execution"
-  )
-  or
-  exists(FileSystemAccess e |
-    sink = e.getAPathArgument() and kind = "file-access"
-  )
-select sink,
-  kind
-    + " | " + sink.getLocation().getFile().getRelativePath()
-    + ":" + sink.getLocation().getStartLine().toString()
-```
-
-### Ruby
-
-```ql
-/**
- * @name List recognized dataflow sinks
- * @description Enumerates security-relevant sinks CodeQL recognizes
- * @kind problem
- * @id custom/list-sinks-ruby
- */
-import ruby
-import codeql.ruby.Concepts
-
-from DataFlow::Node sink, string kind
-where
-  exists(SqlExecution e | sink = e.getSql() and kind = "sql-execution")
-  or
-  exists(SystemCommandExecution e |
-    sink = e.getAnArgument() and kind = "command-execution"
-  )
-  or
-  exists(FileSystemAccess e |
-    sink = e.getAPathArgument() and kind = "file-access"
-  )
-  or
-  exists(CodeExecution e | sink = e.getCode() and kind = "code-execution")
-select sink,
-  kind
-    + " | " + sink.getLocation().getFile().getRelativePath()
-    + ":" + sink.getLocation().getStartLine().toString()
-```
-
-### Java
-
-Java lacks a unified Concepts module. Use language-specific sink classes. The diagnostics query needs its own `qlpack.yml` with a `codeql/java-all` dependency — create it alongside the `.ql` files:
-
-```yaml
-# $DIAG_DIR/qlpack.yml
-name: custom/diagnostics
-version: 0.0.1
-dependencies:
-  codeql/java-all: "*"
-```
-
-Then run `codeql pack install` in the diagnostics directory before executing queries.
-
-```ql
-/**
- * @name List recognized dataflow sinks
- * @description Enumerates security-relevant sinks CodeQL recognizes
- * @kind problem
- * @id custom/list-sinks
- */
-import java
-import semmle.code.java.dataflow.DataFlow
-import semmle.code.java.security.QueryInjection
-import semmle.code.java.security.CommandLineQuery
-import semmle.code.java.security.TaintedPathQuery
-import semmle.code.java.security.XSS
-import semmle.code.java.security.RequestForgery
-import semmle.code.java.security.Xxe
-
-from DataFlow::Node sink, string kind
-where
-  sink instanceof QueryInjectionSink and kind = "sql-injection"
-  or
-  sink instanceof CommandInjectionSink and kind = "command-injection"
-  or
-  sink instanceof TaintedPathSink and kind = "path-injection"
-  or
-  sink instanceof XssSink and kind = "xss"
-  or
-  sink instanceof RequestForgerySink and kind = "ssrf"
-  or
-  sink instanceof XxeSink and kind = "xxe"
-select sink,
-  kind
-    + " | " + sink.getLocation().getFile().getRelativePath()
-    + ":" + sink.getLocation().getStartLine().toString()
-```
-
-### C / C++
-
-C++ uses a similar per-vulnerability-class pattern. Requires a `qlpack.yml` with `codeql/cpp-all` dependency (same approach as Java):
-
-```yaml
-# $DIAG_DIR/qlpack.yml
-name: custom/diagnostics
-version: 0.0.1
-dependencies:
-  codeql/cpp-all: "*"
-```
-
-Then run `codeql pack install` in the diagnostics directory before executing queries.
-
-```ql
-/**
- * @name List recognized dataflow sinks
- * @description Enumerates security-relevant sinks CodeQL recognizes
- * @kind problem
- * @id custom/list-sinks-cpp
- */
-import cpp
-import semmle.code.cpp.dataflow.DataFlow
-import semmle.code.cpp.security.CommandExecution
-import semmle.code.cpp.security.FileAccess
-import semmle.code.cpp.security.BufferWrite
-
-from DataFlow::Node sink, string kind
-where
-  exists(FunctionCall call |
-    sink.asExpr() = call.getAnArgument() and
-    call.getTarget().hasGlobalOrStdName("system") and
-    kind = "command-injection"
-  )
-  or
-  exists(FunctionCall call |
-    sink.asExpr() = call.getAnArgument() and
-    call.getTarget().hasGlobalOrStdName(["fopen", "open", "freopen"]) and
-    kind = "file-access"
-  )
-  or
-  exists(FunctionCall call |
-    sink.asExpr() = call.getAnArgument() and
-    call.getTarget().hasGlobalOrStdName(["sprintf", "strcpy", "strcat", "gets"]) and
-    kind = "buffer-write"
-  )
-  or
-  exists(FunctionCall call |
-    sink.asExpr() = call.getAnArgument() and
-    call.getTarget().hasGlobalOrStdName(["execl", "execle", "execlp", "execv", "execvp", "execvpe", "popen"]) and
-    kind = "command-execution"
-  )
-select sink,
-  kind
-    + " | " + sink.getLocation().getFile().getRelativePath()
-    + ":" + sink.getLocation().getStartLine().toString()
-```
-
-### C\#
-
-C# uses per-vulnerability sink classes. Requires a `qlpack.yml` with `codeql/csharp-all` dependency:
-
-```yaml
-# $DIAG_DIR/qlpack.yml
-name: custom/diagnostics
-version: 0.0.1
-dependencies:
-  codeql/csharp-all: "*"
-```
-
-Then run `codeql pack install` in the diagnostics directory before executing queries.
-
-```ql
-/**
- * @name List recognized dataflow sinks
- * @description Enumerates security-relevant sinks CodeQL recognizes
- * @kind problem
- * @id custom/list-sinks-csharp
- */
-import csharp
-import semmle.code.csharp.dataflow.DataFlow
-import semmle.code.csharp.security.dataflow.SqlInjectionQuery
-import semmle.code.csharp.security.dataflow.CommandInjectionQuery
-import semmle.code.csharp.security.dataflow.TaintedPathQuery
-import semmle.code.csharp.security.dataflow.XSSQuery
-
-from DataFlow::Node sink, string kind
-where
-  sink instanceof SqlInjection::Sink and kind = "sql-injection"
-  or
-  sink instanceof CommandInjection::Sink and kind = "command-injection"
-  or
-  sink instanceof TaintedPath::Sink and kind = "path-injection"
-  or
-  sink instanceof XSS::Sink and kind = "xss"
-select sink,
-  kind
-    + " | " + sink.getLocation().getFile().getRelativePath()
-    + ":" + sink.getLocation().getStartLine().toString()
-```

package/skills_ref/static-analysis/skills/codeql/references/language-details.md

@@ -1,207 +0,0 @@
-# Language-Specific Guidance
-
-## No Build Required
-
-### Python
-
-```bash
-codeql database create codeql.db --language=python --source-root=.
-```
-
-**Framework Support:**
-- Django, Flask, FastAPI: Built-in models
-- Tornado, Pyramid: Partial support
-- Custom frameworks: May need data extensions
-
-**Common Issues:**
-| Issue | Fix |
-|-------|-----|
-| Missing Django models | Ensure `settings.py` is at expected location |
-| Virtual env included | Use `paths-ignore` in config |
-| Type stubs missing | Install `types-*` packages before extraction |
-
-### JavaScript/TypeScript
-
-```bash
-codeql database create codeql.db --language=javascript --source-root=.
-```
-
-**Framework Support:**
-- React, Vue, Angular: Built-in models
-- Express, Koa, Fastify: HTTP source/sink models
-- Next.js, Nuxt: Partial SSR support
-
-**Common Issues:**
-| Issue | Fix |
-|-------|-----|
-| node_modules bloat | Already excluded by default |
-| TypeScript not parsed | Ensure `tsconfig.json` is valid |
-| Monorepo issues | Use `--source-root` for specific package |
-
-### Go
-
-```bash
-codeql database create codeql.db --language=go --source-root=.
-```
-
-**Framework Support:**
-- net/http, Gin, Echo, Chi: Built-in models
-- gRPC: Partial support
-- Custom routers: May need data extensions
-
-**Common Issues:**
-| Issue | Fix |
-|-------|-----|
-| Missing dependencies | Run `go mod download` first |
-| Vendor directory | CodeQL handles automatically |
-| CGO code | Requires `--command='go build'` with CGO enabled |
-
-### Ruby
-
-```bash
-codeql database create codeql.db --language=ruby --source-root=.
-```
-
-**Framework Support:**
-- Rails: Full support (controllers, models, views)
-- Sinatra: Built-in support
-- Hanami: Partial support
-
-**Common Issues:**
-| Issue | Fix |
-|-------|-----|
-| Bundler issues | Run `bundle install` first |
-| Rails engines | May need multiple database passes |
-
-## Build Required
-
-### C/C++
-
-```bash
-# Make
-codeql database create codeql.db --language=cpp --command='make -j8'
-
-# CMake
-codeql database create codeql.db --language=cpp \
-  --source-root=/path/to/src \
-  --command='cmake --build build'
-
-# Ninja
-codeql database create codeql.db --language=cpp \
-  --command='ninja -C build'
-```
-
-**Build System Tips:**
-| Build System | Command |
-|--------------|---------|
-| Make | `make clean && make -j$(nproc)` |
-| CMake | `cmake -B build && cmake --build build` |
-| Meson | `meson setup build && ninja -C build` |
-| Bazel | `bazel build //...` |
-
-**Common Issues:**
-| Issue | Fix |
-|-------|-----|
-| Partial extraction | Ensure `make clean` before CodeQL build |
-| Header-only libraries | Use `--extractor-option cpp_trap_headers=true` |
-| Cross-compilation | Set `CODEQL_EXTRACTOR_CPP_TARGET_ARCH` |
-
-### Java/Kotlin
-
-```bash
-# Gradle
-codeql database create codeql.db --language=java --command='./gradlew build -x test'
-
-# Maven
-codeql database create codeql.db --language=java --command='mvn compile -DskipTests'
-```
-
-**Framework Support:**
-- Spring Boot: Full support
-- Jakarta EE: Built-in models
-- Android: Requires Android SDK
-
-**Common Issues:**
-| Issue | Fix |
-|-------|-----|
-| Missing dependencies | Run `./gradlew dependencies` first |
-| Kotlin mixed projects | Use `--language=java` (covers both) |
-| Annotation processors | Ensure they run during CodeQL build |
-
-### Rust
-
-```bash
-codeql database create codeql.db --language=rust --command='cargo build'
-```
-
-**Common Issues:**
-| Issue | Fix |
-|-------|-----|
-| Proc macros | May require special handling |
-| Workspace projects | Use `--source-root` for specific crate |
-| Build script failures | Ensure native dependencies are available |
-
-### C#
-
-```bash
-# .NET Core
-codeql database create codeql.db --language=csharp --command='dotnet build'
-
-# MSBuild
-codeql database create codeql.db --language=csharp --command='msbuild /t:rebuild'
-```
-
-**Framework Support:**
-- ASP.NET Core: Full support
-- Entity Framework: Database query models
-- Blazor: Partial support
-
-**Common Issues:**
-| Issue | Fix |
-|-------|-----|
-| NuGet restore | Run `dotnet restore` first |
-| Multiple solutions | Specify solution file in command |
-
-### Swift
-
-```bash
-# Xcode project
-codeql database create codeql.db --language=swift \
-  --command='xcodebuild -project MyApp.xcodeproj -scheme MyApp build'
-
-# Swift Package Manager
-codeql database create codeql.db --language=swift --command='swift build'
-```
-
-**Requirements:**
-- macOS only
-- Xcode Command Line Tools
-
-**Common Issues:**
-| Issue | Fix |
-|-------|-----|
-| Code signing | Add `CODE_SIGN_IDENTITY=- CODE_SIGNING_REQUIRED=NO` |
-| Simulator target | Add `-sdk iphonesimulator` |
-
-## Extractor Options
-
-Set via environment variables: `CODEQL_EXTRACTOR_<LANG>_OPTION_<NAME>=<VALUE>`
-
-### C/C++ Options
-
-| Option | Description |
-|--------|-------------|
-| `trap_headers=true` | Include header file analysis |
-| `target_arch=x86_64` | Target architecture |
-
-### Java Options
-
-| Option | Description |
-|--------|-------------|
-| `jdk_version=17` | JDK version for analysis |
-
-### Python Options
-
-| Option | Description |
-|--------|-------------|
-| `python_executable=/path/to/python` | Specific Python interpreter |

package/skills_ref/static-analysis/skills/codeql/references/performance-tuning.md

@@ -1,111 +0,0 @@
-# Performance Tuning
-
-## Memory Configuration
-
-### CODEQL_RAM Environment Variable
-
-Control maximum heap memory (in MB):
-
-```bash
-# 48GB for large codebases
-CODEQL_RAM=48000 codeql database analyze codeql.db ...
-
-# 16GB for medium codebases
-CODEQL_RAM=16000 codeql database analyze codeql.db ...
-```
-
-**Guidelines:**
-| Codebase Size | Recommended RAM |
-|---------------|-----------------|
-| Small (<100K LOC) | 4-8 GB |
-| Medium (100K-1M LOC) | 8-16 GB |
-| Large (1M+ LOC) | 32-64 GB |
-
-## Thread Configuration
-
-### Analysis Threads
-
-```bash
-# Use all available cores
-codeql database analyze codeql.db --threads=0 ...
-
-# Use specific number
-codeql database analyze codeql.db --threads=8 ...
-```
-
-**Note:** `--threads=0` uses all available cores. For shared machines, use explicit count.
-
-## Query-Level Timeouts
-
-Prevent individual queries from running indefinitely:
-
-```bash
-# Set per-query timeout (in milliseconds)
-codeql database analyze codeql.db --timeout=600000 ...
-```
-
-A 10-minute timeout (`600000`) catches runaway queries without killing legitimate complex analysis. Taint-tracking queries on large codebases may need longer.
-
-## Evaluator Diagnostics
-
-When analysis is slow, use `--evaluator-log` to identify which queries consume the most time:
-
-```bash
-codeql database analyze codeql.db \
-  --evaluator-log=evaluator.log \
-  --format=sarif-latest \
-  --output=results.sarif \
-  -- codeql/python-queries:codeql-suites/python-security-extended.qls
-
-# Summarize the log
-codeql generate log-summary evaluator.log --format=text
-```
-
-The summary shows per-query timing and tuple counts. Queries producing millions of tuples are likely the bottleneck.
-
-## Disk Space
-
-| Phase | Typical Size | Notes |
-|-------|-------------|-------|
-| Database creation | 2-10x source size | Compiled languages are larger due to build tracing |
-| Analysis cache | 1-5 GB | Stored in database directory |
-| SARIF output | 1-50 MB | Depends on finding count |
-
-Check available space before starting:
-
-```bash
-df -h .
-du -sh codeql_*.db 2>/dev/null
-```
-
-## Caching Behavior
-
-CodeQL caches query evaluation results inside the database directory. Subsequent runs of the same queries skip re-evaluation.
-
-| Scenario | Cache Effect |
-|----------|-------------|
-| Re-run same packs | Fast — uses cached results |
-| Add new query pack | Only new queries evaluate |
-| `codeql database cleanup` | Clears cache — forces full re-evaluation |
-| `--rerun` flag | Ignores cache for this run |
-
-**When to clear cache:**
-- After deploying new data extensions (cache may hold stale results)
-- When investigating unexpected zero-finding results
-- Before benchmark comparisons (ensures consistent timing)
-
-```bash
-# Clear evaluation cache
-codeql database cleanup codeql_1.db
-```
-
-## Troubleshooting Performance
-
-| Symptom | Likely Cause | Solution |
-|---------|--------------|----------|
-| OOM during analysis | Not enough RAM | Increase `CODEQL_RAM` |
-| Slow database creation | Complex build | Use `--threads`, simplify build |
-| Slow query execution | Large codebase | Reduce query scope, add RAM |
-| Database too large | Too many files | Use exclusion config (see [build-database workflow](../workflows/build-database.md#1b-create-exclusion-config-interpreted-languages-only)) |
-| Single query hangs | Runaway evaluation | Use `--timeout` and check `--evaluator-log` |
-| Repeated runs still slow | Cache not used | Check you're using same database path |