@gpzhang2001/sharpkit-skills 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +201 -0
- package/README.md +12 -0
- package/THIRD_PARTY_NOTICES.md +48 -0
- package/lib/index.d.ts +2027 -0
- package/lib/index.d.ts.map +1 -0
- package/lib/index.js +70 -0
- package/lib/index.js.map +1 -0
- package/package.json +46 -0
- package/skills/analysis/counterevidence.md +185 -0
- package/skills/analysis/fix_verification.md +129 -0
- package/skills/analysis/severity_calibration.md +130 -0
- package/skills/analysis/source_aware_discovery.md +211 -0
- package/skills/cloud/aws.md +231 -0
- package/skills/cloud/azure.md +262 -0
- package/skills/cloud/gcp.md +194 -0
- package/skills/cloud/kubernetes.md +223 -0
- package/skills/coordination/root_agent.md +105 -0
- package/skills/coordination/source_aware_whitebox.md +47 -0
- package/skills/custom/api_spec_testing.md +61 -0
- package/skills/custom/dependency_cve_scanning.md +341 -0
- package/skills/custom/npx_confusion.md +233 -0
- package/skills/custom/source_aware_sast.md +192 -0
- package/skills/frameworks/django.md +214 -0
- package/skills/frameworks/fastapi.md +191 -0
- package/skills/frameworks/nestjs.md +225 -0
- package/skills/frameworks/nextjs.md +228 -0
- package/skills/protocols/graphql.md +276 -0
- package/skills/protocols/oauth.md +185 -0
- package/skills/reconnaissance/asset_discovery.md +150 -0
- package/skills/reconnaissance/infrastructure_lifecycle.md +226 -0
- package/skills/scan_modes/deep.md +164 -0
- package/skills/scan_modes/diff.md +86 -0
- package/skills/scan_modes/quick.md +68 -0
- package/skills/scan_modes/standard.md +99 -0
- package/skills/technologies/active_directory.md +233 -0
- package/skills/technologies/auth0.md +188 -0
- package/skills/technologies/electron_desktop_apps.md +181 -0
- package/skills/technologies/firebase.md +263 -0
- package/skills/technologies/grafana_prometheus.md +189 -0
- package/skills/technologies/llm_applications.md +257 -0
- package/skills/technologies/supabase.md +268 -0
- package/skills/tooling/agent_browser.md +551 -0
- package/skills/tooling/ffuf.md +72 -0
- package/skills/tooling/httpx.md +82 -0
- package/skills/tooling/hurl.md +99 -0
- package/skills/tooling/hypothesis.md +100 -0
- package/skills/tooling/katana.md +102 -0
- package/skills/tooling/naabu.md +68 -0
- package/skills/tooling/nmap.md +66 -0
- package/skills/tooling/nuclei.md +67 -0
- package/skills/tooling/python.md +109 -0
- package/skills/tooling/semgrep.md +72 -0
- package/skills/tooling/sqlmap.md +67 -0
- package/skills/tooling/subfinder.md +66 -0
- package/skills/vulnerabilities/agentic_system_security.md +207 -0
- package/skills/vulnerabilities/argument_injection.md +157 -0
- package/skills/vulnerabilities/authentication_jwt.md +166 -0
- package/skills/vulnerabilities/broken_function_level_authorization.md +154 -0
- package/skills/vulnerabilities/browser_security.md +192 -0
- package/skills/vulnerabilities/business_logic.md +178 -0
- package/skills/vulnerabilities/csrf.md +198 -0
- package/skills/vulnerabilities/header_injection.md +216 -0
- package/skills/vulnerabilities/http_request_smuggling.md +255 -0
- package/skills/vulnerabilities/idor.md +217 -0
- package/skills/vulnerabilities/information_disclosure.md +187 -0
- package/skills/vulnerabilities/insecure_deserialization.md +210 -0
- package/skills/vulnerabilities/insecure_file_uploads.md +194 -0
- package/skills/vulnerabilities/llm_prompt_injection.md +187 -0
- package/skills/vulnerabilities/mass_assignment.md +153 -0
- package/skills/vulnerabilities/nosql_injection.md +288 -0
- package/skills/vulnerabilities/open_redirect.md +165 -0
- package/skills/vulnerabilities/path_traversal_lfi_rfi.md +218 -0
- package/skills/vulnerabilities/prototype_pollution.md +142 -0
- package/skills/vulnerabilities/race_conditions.md +181 -0
- package/skills/vulnerabilities/rce.md +250 -0
- package/skills/vulnerabilities/semantic_confusion.md +189 -0
- package/skills/vulnerabilities/sql_injection.md +190 -0
- package/skills/vulnerabilities/ssrf.md +186 -0
- package/skills/vulnerabilities/ssti.md +270 -0
- package/skills/vulnerabilities/subdomain_takeover.md +167 -0
- package/skills/vulnerabilities/weak_password_detection.md +200 -0
- package/skills/vulnerabilities/xss.md +206 -0
- package/skills/vulnerabilities/xxe.md +223 -0
- package/src/index.ts +89 -0
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: httpx
|
|
3
|
+
description: ProjectDiscovery httpx probing syntax, exact probe flags, and automation-safe output patterns.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# httpx CLI Playbook
|
|
7
|
+
|
|
8
|
+
Official docs:
|
|
9
|
+
- https://docs.projectdiscovery.io/opensource/httpx/usage
|
|
10
|
+
- https://docs.projectdiscovery.io/opensource/httpx/running
|
|
11
|
+
- https://github.com/projectdiscovery/httpx
|
|
12
|
+
|
|
13
|
+
Canonical syntax:
|
|
14
|
+
`httpx [flags]`
|
|
15
|
+
|
|
16
|
+
High-signal flags:
|
|
17
|
+
- `-u, -target <url>` single target
|
|
18
|
+
- `-l, -list <file>` target list
|
|
19
|
+
- `-nf, -no-fallback` probe both HTTP and HTTPS
|
|
20
|
+
- `-nfs, -no-fallback-scheme` do not auto-switch schemes
|
|
21
|
+
- `-sc` status code
|
|
22
|
+
- `-title` page title
|
|
23
|
+
- `-server, -web-server` server header
|
|
24
|
+
- `-td, -tech-detect` technology detection
|
|
25
|
+
- `-fr, -follow-redirects` follow redirects
|
|
26
|
+
- `-mc <codes>` / `-fc <codes>` match or filter status codes
|
|
27
|
+
- `-path <path_or_file>` probe specific paths
|
|
28
|
+
- `-p, -ports <ports>` probe custom ports
|
|
29
|
+
- `-proxy, -http-proxy <url>` proxy target requests
|
|
30
|
+
- `-tlsi, -tls-impersonate` experimental TLS impersonation
|
|
31
|
+
- `-j, -json` JSONL output
|
|
32
|
+
- `-sr, -store-response` store request/response artifacts
|
|
33
|
+
- `-srd, -store-response-dir <dir>` custom directory for stored artifacts
|
|
34
|
+
- `-silent` compact output
|
|
35
|
+
- `-rl <n>` requests/second cap
|
|
36
|
+
- `-t <n>` threads
|
|
37
|
+
- `-timeout <seconds>` request timeout
|
|
38
|
+
- `-retries <n>` retry attempts
|
|
39
|
+
- `-o <file>` output file
|
|
40
|
+
|
|
41
|
+
Agent-safe baseline for automation:
|
|
42
|
+
`httpx -l hosts.txt -sc -title -server -td -fr -timeout 10 -retries 1 -rl 50 -t 25 -silent -j -o httpx.jsonl`
|
|
43
|
+
|
|
44
|
+
Common patterns:
|
|
45
|
+
- Quick live+fingerprint check:
|
|
46
|
+
`httpx -l hosts.txt -sc -title -server -td -silent -o httpx.txt`
|
|
47
|
+
- Probe known admin paths:
|
|
48
|
+
`httpx -l hosts.txt -path /,/login,/admin -sc -title -silent -j -o httpx_paths.jsonl`
|
|
49
|
+
- Probe both schemes explicitly:
|
|
50
|
+
`httpx -l hosts.txt -nf -sc -title -silent`
|
|
51
|
+
- Vhost detection pass:
|
|
52
|
+
`httpx -l hosts.txt -vhost -sc -title -silent -j -o httpx_vhost.jsonl`
|
|
53
|
+
- Proxy-instrumented probing:
|
|
54
|
+
`httpx -l hosts.txt -sc -title -proxy http://127.0.0.1:48080 -silent -j -o httpx_proxy.jsonl`
|
|
55
|
+
- Response-storage pass for downstream content parsing:
|
|
56
|
+
`httpx -l hosts.txt -fr -sr -srd recon/httpx_store -sc -title -server -cl -ct -location -probe -silent`
|
|
57
|
+
|
|
58
|
+
Critical correctness rules:
|
|
59
|
+
- For machine parsing, prefer `-j -o <file>`.
|
|
60
|
+
- Keep `-rl` and `-t` explicit for reproducible throughput.
|
|
61
|
+
- Use `-nf` when you need dual-scheme probing from host-only input.
|
|
62
|
+
- When using `-path` or `-ports`, keep scope tight to avoid accidental scan inflation.
|
|
63
|
+
- Use `-sr -srd <dir>` when later steps need raw response artifacts (JS/route extraction, grepping, replay).
|
|
64
|
+
|
|
65
|
+
Usage rules:
|
|
66
|
+
- Use `-silent` for pipeline-friendly output.
|
|
67
|
+
- Use `-mc/-fc` when downstream steps depend on specific response classes.
|
|
68
|
+
- Prefer `-proxy` flag over global proxy env vars when only httpx traffic should be proxied.
|
|
69
|
+
- Do not use `-h`/`--help` for routine runs unless absolutely necessary.
|
|
70
|
+
|
|
71
|
+
Failure recovery:
|
|
72
|
+
- If too many timeouts occur, reduce `-rl/-t` and/or increase `-timeout`.
|
|
73
|
+
- If output is noisy, add `-fc` filters or `-fd` duplicate filtering.
|
|
74
|
+
- If HTTPS-only probing misses HTTP services, rerun with `-nf` (and avoid `-nfs`).
|
|
75
|
+
|
|
76
|
+
If uncertain, query web_search with:
|
|
77
|
+
`site:docs.projectdiscovery.io httpx <flag> usage`
|
|
78
|
+
|
|
79
|
+
Companion: `wafw00f <url>` fingerprints the WAF/CDN in front of a target
|
|
80
|
+
(Cloudflare, Akamai, AWS WAF, etc.). Run it once after httpx confirms the
|
|
81
|
+
host is live — the WAF identity decides whether to throttle fuzzing,
|
|
82
|
+
swap to evasion payload sets, or assume blocking and route differently.
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: hurl
|
|
3
|
+
description: Reproducible, reviewable HTTP request chains and response assertions with Hurl for authorized multi-step security validation, vulnerable-versus-fixed regression cases, captured values, and low-rate semantic oracles
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Hurl Security Regression Playbook
|
|
7
|
+
|
|
8
|
+
Use [Hurl](https://hurl.dev/) when a security proof requires an ordered HTTP session whose requests, captured values, and assertions should be code-reviewed and replayed. It is well suited to authentication flows, redirects, cookies, CSRF tokens, upload lifecycles, patch regression, and paired semantic-differential cases.
|
|
9
|
+
|
|
10
|
+
Hurl sends exactly what the file describes. It does not make state-changing requests safe. Review scope, methods, targets, and captured secrets before every run.
|
|
11
|
+
|
|
12
|
+
## Install
|
|
13
|
+
|
|
14
|
+
Prefer an official release binary or package. On macOS:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
brew install hurl
|
|
18
|
+
hurl --version
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Official alternatives include release packages and `cargo install --locked hurl`; see [installation](https://hurl.dev/docs/installation.html). Record the tool version with results.
|
|
22
|
+
|
|
23
|
+
## Minimal Chain
|
|
24
|
+
|
|
25
|
+
```hurl
|
|
26
|
+
# lab-regression.hurl
|
|
27
|
+
GET {{base_url}}/session
|
|
28
|
+
HTTP 200
|
|
29
|
+
[Captures]
|
|
30
|
+
csrf: xpath "string(//input[@name='csrf']/@value)"
|
|
31
|
+
[Asserts]
|
|
32
|
+
header "Content-Type" startsWith "text/html"
|
|
33
|
+
|
|
34
|
+
POST {{base_url}}/action
|
|
35
|
+
Content-Type: application/x-www-form-urlencoded
|
|
36
|
+
[FormParams]
|
|
37
|
+
csrf: {{csrf}}
|
|
38
|
+
operation: noop
|
|
39
|
+
HTTP 204
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Hurl keeps cookies across requests in the same file, so an explicit `Cookie` header is unnecessary here.
|
|
43
|
+
|
|
44
|
+
Run one reviewed case against one authorized target first:
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
hurl --test --jobs 1 --connect-timeout 5s --max-time 15s \
|
|
48
|
+
--variable base_url=https://lab.example lab-regression.hurl
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
When credentials are required, pass them with `--secrets-file local-secrets.env`, keep that file outside version control, and avoid verbose/debug output that could expose headers or bodies. Use `--variables-file` only for non-secret environment values.
|
|
52
|
+
|
|
53
|
+
## Designing a Security Regression
|
|
54
|
+
|
|
55
|
+
- Assert the security invariant, not only a status code: denied identity, final normalized location, absence/presence of a structural field, unchanged object state, or exact benign result.
|
|
56
|
+
- Capture only values needed by later requests. Do not write tokens, personal data, or response bodies into committed reports.
|
|
57
|
+
- Encode a malformed but non-triggering control alongside the suspected case.
|
|
58
|
+
- Run the same file against vulnerable and fixed builds through `base_url` or other explicit variables.
|
|
59
|
+
- Keep state-changing methods in a clearly labeled lab/staging file; prefer no-op actions, inert markers, and cleanup requests.
|
|
60
|
+
- Check every redirect step when the vulnerability crosses routing, origin, or authentication boundaries. Blindly following redirects can hide the relevant transition.
|
|
61
|
+
- Use unique canaries so cached or pre-existing state cannot create a false positive.
|
|
62
|
+
|
|
63
|
+
## Chain Structure
|
|
64
|
+
|
|
65
|
+
Organize longer files around capability transitions:
|
|
66
|
+
|
|
67
|
+
```text
|
|
68
|
+
fingerprint -> establish session -> reach boundary -> prove primitive -> verify state -> cleanup
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
At each response, assert the condition required by the next request. A final success assertion cannot explain which earlier assumption failed.
|
|
72
|
+
|
|
73
|
+
Useful Hurl features include:
|
|
74
|
+
|
|
75
|
+
- captures from headers, cookies, JSONPath, XPath, and regex queries
|
|
76
|
+
- assertions over status, headers, body, JSON/XML, redirects, and timing
|
|
77
|
+
- request-local options and variables
|
|
78
|
+
- `--test` plus JSON, JUnit, TAP, or HTML reports
|
|
79
|
+
|
|
80
|
+
Consult the [Hurl manual](https://hurl.dev/docs/manual.html) for version-specific syntax instead of guessing an option.
|
|
81
|
+
|
|
82
|
+
## Safety Rules
|
|
83
|
+
|
|
84
|
+
- Use an explicit `base_url`; never derive the destination from untrusted response data without validating scheme, host, and port.
|
|
85
|
+
- Review POST/PUT/PATCH/DELETE requests and server-side side effects before replay.
|
|
86
|
+
- Set bounded timeouts and retries for the target; do not use polling as an unbounded brute-force loop.
|
|
87
|
+
- Do not use Hurl for raw HTTP parser/smuggling cases when its HTTP stack normalizes the bytes being tested; use an appropriate raw harness in an isolated lab.
|
|
88
|
+
- Use `--path-as-is` when literal `/../` or `/./` path segments are the behavior under test; otherwise Hurl's underlying URL handling can normalize them.
|
|
89
|
+
- Redact reports. HTML/JSON/JUnit artifacts may contain request URLs, headers, captured variables, and response snippets.
|
|
90
|
+
- Keep authentication material in local secret storage and use dedicated test accounts with minimum privilege.
|
|
91
|
+
|
|
92
|
+
## Validation Deliverable
|
|
93
|
+
|
|
94
|
+
1. reviewed `.hurl` file with variableized target and no embedded secrets
|
|
95
|
+
2. vulnerable, fixed, and negative-control environment descriptions
|
|
96
|
+
3. assertion at every capability transition
|
|
97
|
+
4. deterministic results with tool version and timestamps
|
|
98
|
+
5. side effects, cleanup, and residual-state check
|
|
99
|
+
6. redacted report appropriate for sharing
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: hypothesis
|
|
3
|
+
description: Property-based local differential testing with Hypothesis for parsers, canonicalizers, serializers, validators, routers, and other pure functions, emphasizing explicit invariants, shrinking, reproducibility, and bounded resource use
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Hypothesis Differential Testing
|
|
7
|
+
|
|
8
|
+
Use [Hypothesis](https://hypothesis.readthedocs.io/) when a security property can be expressed over local code and failures are likely to hide in combinations of encoding, normalization, structure, or parser recovery. It is especially useful for comparing two implementations or checking that validation and consumption preserve the same meaning.
|
|
9
|
+
|
|
10
|
+
Do not point unrestricted generators at a live service. Hypothesis is safest and most useful against pure local adapters with no network, subprocess, filesystem, or persistent-state side effects.
|
|
11
|
+
|
|
12
|
+
## Install
|
|
13
|
+
|
|
14
|
+
Use an isolated virtual environment and install a reviewed pinned version:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
python -m pip install 'hypothesis==<reviewed-version>'
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Official project: [Hypothesis](https://github.com/HypothesisWorks/hypothesis)
|
|
21
|
+
|
|
22
|
+
## Start From an Invariant
|
|
23
|
+
|
|
24
|
+
Write the security relationship before writing strategies. Examples:
|
|
25
|
+
|
|
26
|
+
```text
|
|
27
|
+
allowlist(raw) implies sink(canonicalize(raw)) remains inside the allowed origin/path
|
|
28
|
+
validator(raw) accepts implies consumer(raw) assigns the same media type/structure
|
|
29
|
+
parse_A(raw) and parse_B(raw) agree on message boundaries and authoritative fields
|
|
30
|
+
serialize(parse(raw)) cannot introduce a delimiter, wildcard, traversal, or new field
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
A test that only checks “does not crash” can find robustness bugs but does not establish a security differential.
|
|
34
|
+
|
|
35
|
+
## Minimal Differential Harness
|
|
36
|
+
|
|
37
|
+
```python
|
|
38
|
+
from hypothesis import given, settings, strategies as st
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
def outcome(parser, raw):
|
|
42
|
+
try:
|
|
43
|
+
return ("accept", parser(raw))
|
|
44
|
+
except ExpectedParseError as exc:
|
|
45
|
+
return ("reject", type(exc).__name__)
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+
@settings(max_examples=250, deadline=500)
|
|
49
|
+
@given(st.text(max_size=128))
|
|
50
|
+
def test_security_boundary(raw: str) -> None:
|
|
51
|
+
checked = outcome(security_parser, raw)
|
|
52
|
+
consumed = outcome(sink_parser, raw)
|
|
53
|
+
assert equivalent_security_meaning(checked, consumed)
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
- Bound string/list/binary sizes, recursion, examples, and deadline.
|
|
57
|
+
- Build structured inputs from relevant tokens rather than generating unrestricted noise.
|
|
58
|
+
- Normalize expected accept/reject/error outcomes explicitly so ordinary parser rejection is not mistaken for a property-test failure.
|
|
59
|
+
- Use `st.one_of`, `st.sampled_from`, `st.lists`, `st.binary`, `st.text`, and composite strategies to represent the actual grammar.
|
|
60
|
+
- Add explicit edge seeds with `@example` for known delimiters and regressions.
|
|
61
|
+
- Let Hypothesis shrink failures; the minimal counterexample is often the clearest explanation of the parser disagreement.
|
|
62
|
+
|
|
63
|
+
## High-Value Strategy Axes
|
|
64
|
+
|
|
65
|
+
- percent and double encoding, malformed escapes, mixed separators
|
|
66
|
+
- Unicode normalization, replacement characters, surrogates, case folding, IDNA
|
|
67
|
+
- dot segments, slash/backslash, absolute/relative paths, sibling-prefix collisions
|
|
68
|
+
- duplicate, empty, first/last, comma-joined, or differently cased fields
|
|
69
|
+
- declared length versus actual bytes, truncation, padding, and terminators
|
|
70
|
+
- nested objects, parser depth, ordering, unknown keys, and error recovery
|
|
71
|
+
- serialize/deserialize round trips and version-to-version behavior
|
|
72
|
+
|
|
73
|
+
Generate only axes supported by the target's transformation graph. Cartesian payload spraying obscures causality.
|
|
74
|
+
|
|
75
|
+
## Reproducibility
|
|
76
|
+
|
|
77
|
+
- Keep the minimized failing example as a normal regression test.
|
|
78
|
+
- Preserve code revision, dependency lock, locale, platform, and parser/library versions.
|
|
79
|
+
- Keep Hypothesis's example database in a task-specific artifact directory when replay across runs matters.
|
|
80
|
+
- For CI, rely on stored explicit regressions for critical cases; randomized discovery supplements them.
|
|
81
|
+
- Classify nondeterminism before suppressing health checks. Timing, global state, environment, and shared caches can create flaky false differentials.
|
|
82
|
+
|
|
83
|
+
## Safety and Resource Controls
|
|
84
|
+
|
|
85
|
+
- Adapt target functions so tests cannot reach the network or execute commands.
|
|
86
|
+
- Use temporary directories and non-secret corpora for parsers that require files.
|
|
87
|
+
- Put native parsers in a disposable, networkless process/container with CPU, memory, file-size, and process ceilings.
|
|
88
|
+
- Do not disable deadlines globally to hide hangs; isolate and bound intentionally slow examples.
|
|
89
|
+
- A crash, timeout, or excessive allocation is a robustness result. Prove a security boundary or exploitability separately.
|
|
90
|
+
- Never reuse captured credentials, customer content, or production requests as generative corpora without sanitization.
|
|
91
|
+
|
|
92
|
+
## Validation Deliverable
|
|
93
|
+
|
|
94
|
+
1. stated invariant and why it protects a security boundary
|
|
95
|
+
2. adapters and exact component/version pair compared
|
|
96
|
+
3. bounded strategies and resource settings
|
|
97
|
+
4. minimized counterexample and both interpretations
|
|
98
|
+
5. stable explicit regression test
|
|
99
|
+
6. impact trace from disagreement to privileged consumer
|
|
100
|
+
7. fixed-version or corrected-invariant result
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: katana
|
|
3
|
+
description: Katana crawler syntax, depth/js/known-files behavior, and stable concurrency controls.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Katana CLI Playbook
|
|
7
|
+
|
|
8
|
+
Official docs:
|
|
9
|
+
- https://docs.projectdiscovery.io/opensource/katana/usage
|
|
10
|
+
- https://docs.projectdiscovery.io/opensource/katana/running
|
|
11
|
+
- https://github.com/projectdiscovery/katana
|
|
12
|
+
|
|
13
|
+
Canonical syntax:
|
|
14
|
+
`katana [flags]`
|
|
15
|
+
|
|
16
|
+
High-signal flags:
|
|
17
|
+
- `-u, -list <url|file>` target URL(s)
|
|
18
|
+
- `-d, -depth <n>` crawl depth
|
|
19
|
+
- `-jc, -js-crawl` parse JavaScript-discovered endpoints
|
|
20
|
+
- `-jsl, -jsluice` deeper JS parsing (memory intensive)
|
|
21
|
+
- `-kf, -known-files <all|robotstxt|sitemapxml>` known-file crawling mode
|
|
22
|
+
- `-proxy <http|socks5 proxy>` explicit proxy setting
|
|
23
|
+
- `-c, -concurrency <n>` concurrent fetchers
|
|
24
|
+
- `-p, -parallelism <n>` concurrent input targets
|
|
25
|
+
- `-rl, -rate-limit <n>` request rate limit
|
|
26
|
+
- `-timeout <seconds>` request timeout
|
|
27
|
+
- `-ct, -crawl-duration <s|m|h|d>` maximum time to crawl the target
|
|
28
|
+
- `-retry <n>` retry count
|
|
29
|
+
- `-mdp, -max-domain-pages <n>` cap pages crawled per domain (default: unlimited)
|
|
30
|
+
- `-fsu, -filter-similar` collapse similar URLs (e.g. /users/123 and /users/456)
|
|
31
|
+
- `-fs, -field-scope <dn|rdn|fqdn|regex>` crawl scope (default `rdn` = root domain + ALL subdomains)
|
|
32
|
+
- `-f, -field <url|path|...>` emit only one field (e.g. `-f url` for a plain URL list)
|
|
33
|
+
- `-or, -omit-raw` omit raw request/response from JSONL output
|
|
34
|
+
- `-ob, -omit-body` omit response body from JSONL output
|
|
35
|
+
- `-mrs, -max-response-size <bytes>` cap per-response bytes read (default 4194304)
|
|
36
|
+
- `-ef, -extension-filter <list>` extension exclusions
|
|
37
|
+
- `-tlsi, -tls-impersonate` experimental JA3/TLS impersonation
|
|
38
|
+
- `-hl, -headless` enable hybrid headless crawling
|
|
39
|
+
- `-sc, -system-chrome` use local Chrome for headless mode
|
|
40
|
+
- `-ho, -headless-options <csv>` extra Chrome options (for example proxy-server)
|
|
41
|
+
- `-nos, -no-sandbox` run Chrome headless with no-sandbox
|
|
42
|
+
- `-noi, -no-incognito` disable incognito in headless mode
|
|
43
|
+
- `-cdd, -chrome-data-dir <dir>` persist browser profile/session
|
|
44
|
+
- `-xhr, -xhr-extraction` include XHR endpoints in JSONL output
|
|
45
|
+
- `-silent`, `-j, -jsonl`, `-o <file>` output controls
|
|
46
|
+
|
|
47
|
+
Agent-safe baseline for automation:
|
|
48
|
+
`mkdir -p crawl && katana -u https://target.tld -d 3 -ct 10m -mdp 2000 -fsu -jc -kf robotstxt -c 10 -p 10 -rl 50 -timeout 10 -retry 1 -ef png,jpg,jpeg,gif,svg,css,woff,woff2,ttf,eot,map -silent -j -o crawl/katana.jsonl`
|
|
49
|
+
|
|
50
|
+
Common patterns:
|
|
51
|
+
- Fast crawl baseline:
|
|
52
|
+
`katana -u https://target.tld -d 3 -jc -silent`
|
|
53
|
+
- Deeper JS-aware crawl (narrowed target; keep it time-bounded):
|
|
54
|
+
`katana -u https://target.tld -d 5 -ct 15m -jc -jsl -kf all -c 10 -p 10 -rl 50 -o katana_urls.txt`
|
|
55
|
+
- Multi-target run with JSONL output:
|
|
56
|
+
`katana -list urls.txt -d 3 -jc -silent -j -o katana.jsonl`
|
|
57
|
+
- Headless crawl with local Chrome:
|
|
58
|
+
`katana -u https://target.tld -hl -sc -nos -xhr -j -o crawl/katana_headless.jsonl`
|
|
59
|
+
- Headless crawl through proxy:
|
|
60
|
+
`katana -u https://target.tld -hl -sc -ho proxy-server=http://127.0.0.1:48080 -j -o crawl/katana_proxy.jsonl`
|
|
61
|
+
|
|
62
|
+
Critical correctness rules:
|
|
63
|
+
- `-kf` must be followed by one of `all`, `robotstxt`, or `sitemapxml`.
|
|
64
|
+
- Use documented `-hl` for headless mode.
|
|
65
|
+
- `-proxy` expects a single proxy URL string (for example `http://127.0.0.1:8080`).
|
|
66
|
+
- `-ho` expects comma-separated Chrome options (example: `-ho --disable-gpu,proxy-server=http://127.0.0.1:8080`).
|
|
67
|
+
- For `-kf`, keep depth at least `-d 3` so known files are fully covered.
|
|
68
|
+
- If writing to a file, ensure parent directory exists before `-o`.
|
|
69
|
+
|
|
70
|
+
Keeping output small (katana has NO default page cap, so plan for volume):
|
|
71
|
+
- Bound scope and volume: `-fs fqdn` (or `-cs`/`-cos` regex) so the crawl doesn't wander across every subdomain, `-mdp <n>` to cap pages per domain, `-fsu` to collapse near-identical URLs, and `-ct`/`-d` to bound time and depth.
|
|
72
|
+
- Shrink each record: default JSONL is verbose. If you only need endpoints, emit a plain URL list with `-f url` instead of `-j`. If you need JSONL, drop the heavy parts with `-or` (omit raw) and `-ob` (omit body), and lower `-mrs` to cap per-response bytes.
|
|
73
|
+
- Reserve `-jsl` / `-kf all` / higher `-d` for a specific narrowed target — they multiply output fast on large sites.
|
|
74
|
+
- Reduce, then delete: once the crawl finishes, extract just what you need (e.g. `katana ... -f url -o urls.txt` or `sort -u` a URL list, or a short note of interesting paths) and remove the raw crawl file/dir. Don't keep large raw crawls around after you've distilled them.
|
|
75
|
+
- Sanity-check size (`du -sh <out>`); if it's outsized for the scope, tighten `-fs`/`-mdp`/`-fsu`/`-d`/`-ct` and re-run rather than keeping it.
|
|
76
|
+
|
|
77
|
+
Usage rules:
|
|
78
|
+
- Keep `-d`, `-c`, `-p`, and `-rl` explicit for reproducible runs.
|
|
79
|
+
- Use `-ef` early to reduce static-file noise before fuzzing.
|
|
80
|
+
- Prefer `-proxy` over environment proxy variables when proxying only Katana traffic.
|
|
81
|
+
- Use `-hc` only for one-time diagnostics, not routine crawling loops.
|
|
82
|
+
- Do not use `-h`/`--help` for routine runs unless absolutely necessary.
|
|
83
|
+
|
|
84
|
+
Failure recovery:
|
|
85
|
+
- If crawl runs too long, lower `-d` and optionally add `-ct`.
|
|
86
|
+
- If memory spikes, disable `-jsl` and lower `-c/-p`.
|
|
87
|
+
- If headless fails with Chrome errors, drop `-sc` or install system Chrome.
|
|
88
|
+
- If output is noisy, tighten scope and add `-ef` filters.
|
|
89
|
+
|
|
90
|
+
If uncertain, query web_search with:
|
|
91
|
+
`site:docs.projectdiscovery.io katana <flag> usage`
|
|
92
|
+
|
|
93
|
+
Complementary crawlers / JS endpoint extractors in the sandbox:
|
|
94
|
+
- `gospider -s https://target.tld -d 3 -c 10 -t 20` — alternate crawler;
|
|
95
|
+
picks up things Katana misses on weird sites; use it as a second
|
|
96
|
+
pass when Katana output looks thin.
|
|
97
|
+
- `~/tools/JS-Snooper/js_snooper.sh <domain>` and
|
|
98
|
+
`~/tools/jsniper.sh/jsniper.sh <domain>` — both take a bare domain and
|
|
99
|
+
run their own JS-file discovery internally (jsniper drives httpx +
|
|
100
|
+
katana + nuclei file templates). Reach for them when you want a quick
|
|
101
|
+
"find endpoints/keys/secrets in any JS this domain serves" sweep
|
|
102
|
+
without wiring it up yourself.
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: naabu
|
|
3
|
+
description: Naabu port-scanning syntax with host input, scan-type, verification, and rate controls.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Naabu CLI Playbook
|
|
7
|
+
|
|
8
|
+
Official docs:
|
|
9
|
+
- https://docs.projectdiscovery.io/opensource/naabu/usage
|
|
10
|
+
- https://docs.projectdiscovery.io/opensource/naabu/running
|
|
11
|
+
- https://github.com/projectdiscovery/naabu
|
|
12
|
+
|
|
13
|
+
Canonical syntax:
|
|
14
|
+
`naabu [flags]`
|
|
15
|
+
|
|
16
|
+
High-signal flags:
|
|
17
|
+
- `-host <host>` single host
|
|
18
|
+
- `-list, -l <file>` hosts list
|
|
19
|
+
- `-p <ports>` explicit ports (supports ranges)
|
|
20
|
+
- `-top-ports <n|full>` top ports profile
|
|
21
|
+
- `-exclude-ports <ports>` exclusions
|
|
22
|
+
- `-scan-type <s|c|syn|connect>` SYN or CONNECT scan
|
|
23
|
+
- `-Pn` skip host discovery
|
|
24
|
+
- `-rate <n>` packets per second
|
|
25
|
+
- `-c <n>` worker count
|
|
26
|
+
- `-timeout <ms>` per-probe timeout in milliseconds
|
|
27
|
+
- `-retries <n>` retry attempts
|
|
28
|
+
- `-proxy <socks5://host:port>` SOCKS5 proxy
|
|
29
|
+
- `-verify` verify discovered open ports
|
|
30
|
+
- `-j, -json` JSONL output
|
|
31
|
+
- `-silent` compact output
|
|
32
|
+
- `-o <file>` output file
|
|
33
|
+
|
|
34
|
+
Agent-safe baseline for automation:
|
|
35
|
+
`naabu -list hosts.txt -top-ports 100 -scan-type c -Pn -rate 300 -c 25 -timeout 1000 -retries 1 -verify -silent -j -o naabu.jsonl`
|
|
36
|
+
|
|
37
|
+
Common patterns:
|
|
38
|
+
- Top ports with controlled rate:
|
|
39
|
+
`naabu -list hosts.txt -top-ports 100 -scan-type c -rate 300 -c 25 -timeout 1000 -retries 1 -verify -silent -o naabu.txt`
|
|
40
|
+
- Focused web-ports sweep:
|
|
41
|
+
`naabu -list hosts.txt -p 80,443,8080,8443 -scan-type c -rate 300 -c 25 -timeout 1000 -retries 1 -verify -silent`
|
|
42
|
+
- Single-host quick check:
|
|
43
|
+
`naabu -host target.tld -p 22,80,443 -scan-type c -rate 300 -c 25 -timeout 1000 -retries 1 -verify`
|
|
44
|
+
- Root SYN mode (if available):
|
|
45
|
+
`sudo naabu -list hosts.txt -top-ports 100 -scan-type syn -rate 500 -c 25 -timeout 1000 -retries 1 -verify -silent`
|
|
46
|
+
|
|
47
|
+
Critical correctness rules:
|
|
48
|
+
- Use `-scan-type connect` when running without root/privileged raw socket access.
|
|
49
|
+
- Always set `-timeout` explicitly; it is in milliseconds.
|
|
50
|
+
- Set `-rate` explicitly to avoid unstable or noisy scans.
|
|
51
|
+
- `-timeout` is in milliseconds, not seconds.
|
|
52
|
+
- Keep port scope tight: prefer explicit important ports or a small `-top-ports` value unless broader coverage is explicitly required.
|
|
53
|
+
- Do not spam traffic; start with the smallest useful port set and conservative rate/worker settings.
|
|
54
|
+
- Prefer `-verify` before handing ports to follow-up scanners.
|
|
55
|
+
|
|
56
|
+
Usage rules:
|
|
57
|
+
- Keep host discovery behavior explicit (`-Pn` or default discovery).
|
|
58
|
+
- Use `-j -o <file>` for automation pipelines.
|
|
59
|
+
- Prefer `-p 22,80,443,8080,8443` or `-top-ports 100` before considering larger sweeps.
|
|
60
|
+
- Do not use `-h`/`--help` for normal flow unless absolutely necessary.
|
|
61
|
+
|
|
62
|
+
Failure recovery:
|
|
63
|
+
- If privileged socket errors occur, switch to `-scan-type c`.
|
|
64
|
+
- If scans are slow or lossy, lower `-rate`, lower `-c`, and tighten `-p`/`-top-ports`.
|
|
65
|
+
- If many hosts appear down, compare runs with and without `-Pn`.
|
|
66
|
+
|
|
67
|
+
If uncertain, query web_search with:
|
|
68
|
+
`site:docs.projectdiscovery.io naabu <flag> usage`
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: nmap
|
|
3
|
+
description: Canonical Nmap CLI syntax, two-pass scanning workflow, and sandbox-safe bounded scan patterns.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Nmap CLI Playbook
|
|
7
|
+
|
|
8
|
+
Official docs:
|
|
9
|
+
- https://nmap.org/book/man-briefoptions.html
|
|
10
|
+
- https://nmap.org/book/man.html
|
|
11
|
+
- https://nmap.org/book/man-performance.html
|
|
12
|
+
|
|
13
|
+
Canonical syntax:
|
|
14
|
+
`nmap [Scan Type(s)] [Options] {target specification}`
|
|
15
|
+
|
|
16
|
+
High-signal flags:
|
|
17
|
+
- `-n` skip DNS resolution
|
|
18
|
+
- `-Pn` skip host discovery when ICMP/ping is filtered
|
|
19
|
+
- `-sS` SYN scan (root/privileged)
|
|
20
|
+
- `-sT` TCP connect scan (no raw-socket privilege)
|
|
21
|
+
- `-sV` detect service versions
|
|
22
|
+
- `-sC` run default NSE scripts
|
|
23
|
+
- `-p <ports>` explicit ports (`-p-` for all TCP ports)
|
|
24
|
+
- `--top-ports <n>` quick common-port sweep
|
|
25
|
+
- `--open` show only hosts with open ports
|
|
26
|
+
- `-T<0-5>` timing template (`-T4` common)
|
|
27
|
+
- `--max-retries <n>` cap retransmissions
|
|
28
|
+
- `--host-timeout <time>` give up on very slow hosts
|
|
29
|
+
- `--script-timeout <time>` bound NSE script runtime
|
|
30
|
+
- `-oA <prefix>` output in normal/XML/grepable formats
|
|
31
|
+
|
|
32
|
+
Agent-safe baseline for automation:
|
|
33
|
+
`nmap -n -Pn --open --top-ports 100 -T4 --max-retries 1 --host-timeout 90s -oA nmap_quick <host>`
|
|
34
|
+
|
|
35
|
+
Common patterns:
|
|
36
|
+
- Fast first pass:
|
|
37
|
+
`nmap -n -Pn --top-ports 100 --open -T4 --max-retries 1 --host-timeout 90s <host>`
|
|
38
|
+
- Very small important-port pass:
|
|
39
|
+
`nmap -n -Pn -p 22,80,443,8080,8443 --open -T4 --max-retries 1 --host-timeout 90s <host>`
|
|
40
|
+
- Service/script enrichment on discovered ports:
|
|
41
|
+
`nmap -n -Pn -sV -sC -p <comma_ports> --script-timeout 30s --host-timeout 3m -oA nmap_services <host>`
|
|
42
|
+
- No-root fallback:
|
|
43
|
+
`nmap -n -Pn -sT --top-ports 100 --open --host-timeout 90s <host>`
|
|
44
|
+
|
|
45
|
+
Critical correctness rules:
|
|
46
|
+
- Always set target scope explicitly.
|
|
47
|
+
- Prefer two-pass scanning: discovery pass, then enrichment pass.
|
|
48
|
+
- Always set a timeout boundary with `--host-timeout`; add `--script-timeout` whenever NSE scripts are involved.
|
|
49
|
+
- Keep discovery scans tight: use explicit important ports or a small `--top-ports` profile unless broader coverage is explicitly required.
|
|
50
|
+
- In sandboxed runs, avoid exhaustive sweeps (`-p-`, very high `--top-ports`, or wide host ranges) unless explicitly required.
|
|
51
|
+
- Do not spam traffic; start with the smallest port set that can answer the question.
|
|
52
|
+
- Prefer `naabu` for broad port discovery; use `nmap` for scoped verification/enrichment.
|
|
53
|
+
|
|
54
|
+
Usage rules:
|
|
55
|
+
- Add `-n` by default in automation to avoid DNS delays.
|
|
56
|
+
- Use `-oA` for reusable artifacts.
|
|
57
|
+
- Prefer `-p 22,80,443,8080,8443` or `--top-ports 100` before considering larger sweeps.
|
|
58
|
+
- Do not use `-h`/`--help` for routine usage unless absolutely necessary.
|
|
59
|
+
|
|
60
|
+
Failure recovery:
|
|
61
|
+
- If host appears down unexpectedly, rerun with `-Pn`.
|
|
62
|
+
- If scan stalls, tighten scope (`-p` or smaller `--top-ports`) and lower retries.
|
|
63
|
+
- If scripts run too long, add `--script-timeout`.
|
|
64
|
+
|
|
65
|
+
If uncertain, query web_search with:
|
|
66
|
+
`site:nmap.org/book nmap <flag>`
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: nuclei
|
|
3
|
+
description: Exact Nuclei command structure, template selection, and bounded high-throughput execution controls.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Nuclei CLI Playbook
|
|
7
|
+
|
|
8
|
+
Official docs:
|
|
9
|
+
- https://docs.projectdiscovery.io/opensource/nuclei/running
|
|
10
|
+
- https://docs.projectdiscovery.io/opensource/nuclei/mass-scanning-cli
|
|
11
|
+
- https://github.com/projectdiscovery/nuclei
|
|
12
|
+
|
|
13
|
+
Canonical syntax:
|
|
14
|
+
`nuclei [flags]`
|
|
15
|
+
|
|
16
|
+
High-signal flags:
|
|
17
|
+
- `-u, -target <url>` single target
|
|
18
|
+
- `-l, -list <file>` targets file
|
|
19
|
+
- `-im, -input-mode <mode>` list/burp/jsonl/yaml/openapi/swagger
|
|
20
|
+
- `-t, -templates <path|tag>` explicit template path(s)
|
|
21
|
+
- `-tags <tag1,tag2>` run by tag
|
|
22
|
+
- `-s, -severity <critical,high,...>` severity filter
|
|
23
|
+
- `-as, -automatic-scan` tech-mapped automatic scan
|
|
24
|
+
- `-ni, -no-interactsh` disable OAST/interactsh requests
|
|
25
|
+
- `-rl, -rate-limit <n>` global request rate cap
|
|
26
|
+
- `-c, -concurrency <n>` template concurrency
|
|
27
|
+
- `-bs, -bulk-size <n>` hosts in parallel per template
|
|
28
|
+
- `-timeout <seconds>` request timeout
|
|
29
|
+
- `-retries <n>` retries
|
|
30
|
+
- `-stats` periodic scan stats output
|
|
31
|
+
- `-silent` findings-only output
|
|
32
|
+
- `-j, -jsonl` JSONL output
|
|
33
|
+
- `-o <file>` output file
|
|
34
|
+
|
|
35
|
+
Agent-safe baseline for automation:
|
|
36
|
+
`nuclei -l targets.txt -as -s critical,high -rl 50 -c 20 -bs 20 -timeout 10 -retries 1 -silent -j -o nuclei.jsonl`
|
|
37
|
+
|
|
38
|
+
Common patterns:
|
|
39
|
+
- Focused severity scan:
|
|
40
|
+
`nuclei -u https://target.tld -s critical,high -silent -o nuclei_high.txt`
|
|
41
|
+
- List-driven controlled scan:
|
|
42
|
+
`nuclei -l targets.txt -as -rl 50 -c 20 -bs 20 -timeout 10 -retries 1 -j -o nuclei.jsonl`
|
|
43
|
+
- Tag-driven run:
|
|
44
|
+
`nuclei -l targets.txt -tags cve,misconfig -s critical,high,medium -silent`
|
|
45
|
+
- Explicit templates:
|
|
46
|
+
`nuclei -l targets.txt -t http/cves/ -t dns/ -rl 30 -c 10 -bs 10 -j -o nuclei_templates.jsonl`
|
|
47
|
+
- Deterministic non-OAST run:
|
|
48
|
+
`nuclei -l targets.txt -as -s critical,high -ni -stats -rl 30 -c 10 -bs 10 -timeout 10 -retries 1 -j -o nuclei_no_oast.jsonl`
|
|
49
|
+
|
|
50
|
+
Critical correctness rules:
|
|
51
|
+
- Provide a template selection method (`-as`, `-t`, or `-tags`); avoid unscoped broad runs.
|
|
52
|
+
- Keep `-rl`, `-c`, and `-bs` explicit for predictable resource use.
|
|
53
|
+
- Use `-ni` when outbound interactsh/OAST traffic is not expected or not allowed.
|
|
54
|
+
- Use structured output (`-j -o <file>`) for automation.
|
|
55
|
+
|
|
56
|
+
Usage rules:
|
|
57
|
+
- Start with severity/tags/templates filters to keep runs explainable.
|
|
58
|
+
- Keep retries conservative (`-retries 1`) unless transport instability is proven.
|
|
59
|
+
- Do not use `-h`/`--help` for routine operation unless absolutely necessary.
|
|
60
|
+
|
|
61
|
+
Failure recovery:
|
|
62
|
+
- If performance degrades, lower `-c/-bs` before lowering `-rl`.
|
|
63
|
+
- If findings are unexpectedly empty, verify template selection (`-as` vs explicit `-t/-tags`).
|
|
64
|
+
- If scan duration grows, reduce target set and enforce stricter template/severity filters.
|
|
65
|
+
|
|
66
|
+
If uncertain, query web_search with:
|
|
67
|
+
`site:docs.projectdiscovery.io nuclei <flag> running`
|