augure 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +136 -0
- data/LICENSE +21 -0
- data/README.md +217 -0
- data/exe/augure +181 -0
- data/exe/augure-benchmark +67 -0
- data/home/runner/work/augure/augure/docs/agent-snippet.md +31 -0
- data/home/runner/work/augure/augure/docs/auditability.md +160 -0
- data/home/runner/work/augure/augure/docs/build-format.md +119 -0
- data/home/runner/work/augure/augure/docs/explanation-architecture.md +74 -0
- data/home/runner/work/augure/augure/docs/how-to-swap-solver.md +73 -0
- data/home/runner/work/augure/augure/docs/how-to-write-rules.md +71 -0
- data/home/runner/work/augure/augure/docs/pack-format.md +137 -0
- data/home/runner/work/augure/augure/docs/reference.md +293 -0
- data/home/runner/work/augure/augure/docs/tutorial.md +108 -0
- data/home/runner/work/augure/augure/examples/ret2win.facts +11 -0
- data/home/runner/work/augure/augure/examples/split.facts +12 -0
- data/home/runner/work/augure/augure/exe/augure +181 -0
- data/home/runner/work/augure/augure/exe/augure-benchmark +67 -0
- data/home/runner/work/augure/augure/lib/augure/bandit.rb +118 -0
- data/home/runner/work/augure/augure/lib/augure/coverage.rb +56 -0
- data/home/runner/work/augure/augure/lib/augure/ctf_corpus.json +773 -0
- data/home/runner/work/augure/augure/lib/augure/engine.rb +128 -0
- data/home/runner/work/augure/augure/lib/augure/facts.rb +194 -0
- data/home/runner/work/augure/augure/lib/augure/kb_corpus.json +345 -0
- data/home/runner/work/augure/augure/lib/augure/knowledge_base.rb +109 -0
- data/home/runner/work/augure/augure/lib/augure/mcts.rb +178 -0
- data/home/runner/work/augure/augure/lib/augure/pack.rb +244 -0
- data/home/runner/work/augure/augure/lib/augure/pack_loader.rb +93 -0
- data/home/runner/work/augure/augure/lib/augure/parity.rb +69 -0
- data/home/runner/work/augure/augure/lib/augure/pipeline.rb +107 -0
- data/home/runner/work/augure/augure/lib/augure/priors.rb +32 -0
- data/home/runner/work/augure/augure/lib/augure/rules.rb +300 -0
- data/home/runner/work/augure/augure/lib/augure/smt_process.rb +48 -0
- data/home/runner/work/augure/augure/lib/augure/verifier.rb +40 -0
- data/home/runner/work/augure/augure/lib/augure/version.rb +5 -0
- data/home/runner/work/augure/augure/lib/augure.rb +25 -0
- data/home/runner/work/augure/augure/packs/banner_leak.yml +46 -0
- data/home/runner/work/augure/augure/packs/brop.yml +46 -0
- data/home/runner/work/augure/augure/packs/fastbin_hook.yml +45 -0
- data/home/runner/work/augure/augure/packs/got_partial_overwrite.yml +45 -0
- data/home/runner/work/augure/augure/packs/got_partial_overwrite_deref.yml +39 -0
- data/home/runner/work/augure/augure/packs/house_botcake.yml +38 -0
- data/home/runner/work/augure/augure/packs/house_force.yml +37 -0
- data/home/runner/work/augure/augure/packs/house_of_apple2.yml +45 -0
- data/home/runner/work/augure/augure/packs/house_orange.yml +38 -0
- data/home/runner/work/augure/augure/packs/jop.yml +39 -0
- data/home/runner/work/augure/augure/packs/large_bin_attack.yml +39 -0
- data/home/runner/work/augure/augure/packs/one_gadget.yml +38 -0
- data/home/runner/work/augure/augure/packs/orw.yml +55 -0
- data/home/runner/work/augure/augure/packs/ret2csu_v2.yml +55 -0
- data/home/runner/work/augure/augure/packs/ret2dlresolve_x86.yml +39 -0
- data/home/runner/work/augure/augure/packs/ret2partial_overwrite.yml +38 -0
- data/home/runner/work/augure/augure/packs/ret2vdso.yml +39 -0
- data/home/runner/work/augure/augure/packs/setcontext_srop.yml +46 -0
- data/home/runner/work/augure/augure/packs/uaf_tcache.yml +45 -0
- data/home/runner/work/augure/augure/packs/unsorted_bin_attack.yml +38 -0
- metadata +103 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: b0439bafc6950f3d38a680b8187ebbba52acc7c8d0a19e12cfff0bad82555b11
|
|
4
|
+
data.tar.gz: f9b6f194a5652122823106fa65dd5fc19f0630a2095d831da32300c5c7cc350c
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: a97621f43eccee16165e6ab913620e8d853b9ec7cdff3c381f2dadda689cecceb269c022b9a631eb31a3e98efb8e4f6e5561bd1232ac475785bab930163c9960
|
|
7
|
+
data.tar.gz: b428108f2757244c6da8316ef9047fb1944adf257317244bf230a4ab0acfb8c78ed9a655cf92ea218605bc6aeebcd81300da915b5f21a27b7ddfad7dd18ce28e
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
- The `cve` fact (the CVE/KEV bridge): an agent or the operator proposes the CVE id as a fact (`cve("CVE-2024-1234").`) - the technique packs consume it like any observation. The schema + the atom grammar treat it as any string fact.
|
|
11
|
+
|
|
12
|
+
- The pack DSL gains an optional `build:` section (the pack teaching the BUILDER its assembly
|
|
13
|
+
order): armor check 8 validates it (own technique, slot vocabulary), `pack.build` exposes it -
|
|
14
|
+
lictor's PayloadBuilder consumes it (packs teach, the builder follows, for real this time).
|
|
15
|
+
- ADRs: docs/adr/ records the architectural decisions and renouncements - the arch abstraction
|
|
16
|
+
before any port (0001), ARM64 deferred behind it (0002), PE/Mach-O out of scope for v0.x (0003).
|
|
17
|
+
- The general gadget hunter: every ret-terminated chain in the executable sections, semantically
|
|
18
|
+
classified - write4 yields seventeen gadgets the byte patterns never saw (pop_r14_r15_ret, the
|
|
19
|
+
multi-pop families).
|
|
20
|
+
- The parametrized shellcode: execve of a chosen path, and the orw chain (open/read/write, no
|
|
21
|
+
execve) for the seccomp answer - the x64 stub reads and prints a real flag file in the suite.
|
|
22
|
+
- The ghost-writing pools: the zeroing, the call-number and the camouflage equivalents picked per
|
|
23
|
+
seed - ten seeds, ten living stubs, deterministic.
|
|
24
|
+
- The encoders: the keyed self-decoder with the backward-call get-pc (zero-free by construction),
|
|
25
|
+
the xor and the polymorphic flavours, both live-tested through a 00/41/0a/0d channel; a payload
|
|
26
|
+
wider than 120 bytes on a 0x00 channel is refused honestly.
|
|
27
|
+
- The dynamic layer: the write-site proof (the watchpoint catches the instruction that plants the
|
|
28
|
+
overflow), the runtime map (libc, stack), and the crash identification (vuln_function, sink,
|
|
29
|
+
crash_site).
|
|
30
|
+
- Static parity on the real binaries: a CI job downloads the eight sha256-pinned ROP Emporium x64
|
|
31
|
+
binaries, profiles binary + shipped libraries, and asserts every statically-derivable corpus
|
|
32
|
+
claim (96 checked; write-up-only facts are reported as skipped, never silently dropped). The
|
|
33
|
+
profiler learned the mov [r14], r15 encoding, and the corpus lost two copy-paste defects (a
|
|
34
|
+
duplicated format1 entry, badchars claiming write4's mov-deref-write gadget).
|
|
35
|
+
- The profiler emits `plt_addr(name, addr)` and `got_addr(name, addr)` for the JMP_SLOT imports
|
|
36
|
+
(PLT stub from the section layout, GOT entry from the relocation offset): a ROP chain resolves
|
|
37
|
+
addresses from facts instead of hand-parsed banners. `plt(name)` stays the rule vocabulary.
|
|
38
|
+
- Technique packs: complete techniques as YAML data (rules, KB, MCTS transitions, priors, mandatory
|
|
39
|
+
provenance), validated by the armor (constrained vocabulary, existing predicates only,
|
|
40
|
+
stratification, new heads only) and the corpus guard (a pack may not reorder an existing
|
|
41
|
+
technique or take a documented target's top-1).
|
|
42
|
+
- The network fact vocabulary: service/software_version/remote emitted from nmap scans;
|
|
43
|
+
`verified("remote")` = the differential confidence level (modeled-from-ELF vs remote-verified).
|
|
44
|
+
- The profiler detects win/flag symbols (the target_function fact).
|
|
45
|
+
- The shipped corpus (lib/augure/ctf_corpus.json) enables the corpus guard at consumer runtime.
|
|
46
|
+
- augure-mcp: the read-only MCP surface (analyze_target, list_rules, explain_technique).
|
|
47
|
+
|
|
48
|
+
### Changed
|
|
49
|
+
|
|
50
|
+
- `shellcode` carries the seccomp guard too: the technique as-modeled ends in execve - dead under a
|
|
51
|
+
filter, the orw pack owns the seccomp case. The frozen `pwnable/orw` entry moves to an EMPTY
|
|
52
|
+
built-in verdict (the pack layer's technique, deliberately not in the built-in table) with the
|
|
53
|
+
justification in the corpus; the benchmark counts a reproduced empty verdict as concordance
|
|
54
|
+
(33/33 holds).
|
|
55
|
+
- `printf` leaves UNSAFE_SYMBOLS: a constant format is not a vuln, and the hint derived
|
|
56
|
+
`vuln(fmtstr)` for every binary that prints - the lab's menu binaries all became "format string
|
|
57
|
+
bugs". `sprintf` stays flagged (unbounded by design); the controlled-format knowledge belongs to
|
|
58
|
+
the operator's suggest flow.
|
|
59
|
+
- seccomp demotes the shell techniques: `ret2libc`, `rop` and `srop` rules carry `[not_fact,
|
|
60
|
+
seccomp, "true"]` - a filter that kills execve kills system/execve chains, and the decision layer
|
|
61
|
+
now says so. The not_fact passes when the fact is absent, so the frozen corpus is untouched
|
|
62
|
+
(33/33); the lab's orw target demonstrated the gap (the demo picked ret2libc on a seccomp'd
|
|
63
|
+
binary).
|
|
64
|
+
- The shipped `augure` gem now carries the packs, the example fact bases, the docs and the
|
|
65
|
+
`augure-benchmark` executable (previously checkout-only: consumers could not reproduce the
|
|
66
|
+
headline concordance claim from an installed gem). `augure-profiler` gets its first real version
|
|
67
|
+
(0.0.1), a bounded `augure ~> 0.1` dependency and a changelog_uri. Doc drift fixed: 31 input
|
|
68
|
+
facts, 33 corpus targets, `leaked_address` in the schema table, `#ranking_of` documented.
|
|
69
|
+
- `GadgetHunt.hunt` was O(n^2) in memory traffic: at every offset it copied the remaining section
|
|
70
|
+
bytes into a fresh EncodedData (~512 GB cumulative for a 1 MB .text). One EncodedData per section
|
|
71
|
+
now, decoding moves `edata.ptr` instead of copying; the per-decode rescue narrows to
|
|
72
|
+
DecodeError/ParseError so metasm internal bugs surface instead of masquerading as "not a gadget".
|
|
73
|
+
Same gadget sets: ROP Emporium parity 8/8, write4 multi-instruction gadgets intact.
|
|
74
|
+
- `Pipeline.analyze` no longer accepts `solver:`: the parameter was accepted and never used, so the
|
|
75
|
+
API pretended the decision ran an SMT solver while `verify` is the arithmetic feasibility check.
|
|
76
|
+
The SMT path stays a tested seam (Verifier.payload_fits_smtlib + SmtProcess) wired by consumers
|
|
77
|
+
who need it; the docstring, reference.md and the new regression spec say so.
|
|
78
|
+
- `Mcts.terminal?` hardcoded `shell`: a pack declaring `terminal: true` (orw provides `flag_read`)
|
|
79
|
+
could never terminate a plan, so pack-authored terminal semantics - validated by the armor,
|
|
80
|
+
stored by `merged_model` - were ignored and plans ran on toward a shell the technique never
|
|
81
|
+
delivers. The model's terminal stages now own the semantics; the built-in model (all terminals
|
|
82
|
+
provide `shell`) behaves identically, 33/33 concordance.
|
|
83
|
+
- The engine refused to re-derive an existing head: sibling rules sharing a head (the documented
|
|
84
|
+
way to express OR) lost every reason after the first, so `explain.provenance` contradicted
|
|
85
|
+
`how-to-write-rules.md` ("the provenance tells which reason fired"). All firings now merge into
|
|
86
|
+
the head, each attributed to its own rule; verdicts unchanged (33/33 concordance).
|
|
87
|
+
- `plt` facts described every identifier-shaped symbol (`main`, `_start`, `_DYNAMIC`, a local
|
|
88
|
+
function named `system`...): a local `system` satisfied `app_ret2plt` and the parity subset check
|
|
89
|
+
could pass for the wrong reason. `plt` now means real dynamic imports only (shndx == UNDEF, FUNC
|
|
90
|
+
type), regression-tested against a local-system binary; ROP Emporium parity stays 8/8.
|
|
91
|
+
|
|
92
|
+
### Fixed
|
|
93
|
+
|
|
94
|
+
- `Shellcode.generate` on Ia32 with a seeded rng crashed (`zero_esi` pool had no x86 key while
|
|
95
|
+
every x86 stub zeroes `ecx`); the pool is now `zero_ecx`, and the x86 seeds are regression-tested
|
|
96
|
+
(assembly + decode validity; the live-run proof stays x64 - a 32-bit runner needs gcc multilib).
|
|
97
|
+
- Facts accepted any string atom (`nx("maybe")`, `relro("sorta")`, `canary("ture")`): the fact
|
|
98
|
+
parsed, the closed-world default was disabled, and every rule of the predicate silently never
|
|
99
|
+
fired. Closed predicates now validate their value domain (`VALUE_DOMAINS`) at parse and at `add`,
|
|
100
|
+
loud `MalformedFact` both ways.
|
|
101
|
+
- The pack armor let an invalid `match` pattern (`"("`) and a non-integer or unknown-operator `cmp`
|
|
102
|
+
(`"26"`, `"gtr"`) through to decision time, where they died as
|
|
103
|
+
`RegexpError`/`ArgumentError`/`KeyError` - violating the fail-loudly-at-load doctrine. Armor
|
|
104
|
+
check 7: match patterns precompile and cmp operands/operators type-check at load, all as
|
|
105
|
+
`PackError`.
|
|
106
|
+
- The fact-atom grammar is enforced at `Facts#add` too (the wall goes both ways): a symbol name
|
|
107
|
+
from an untrusted binary or a version string from a hostile banner can no longer carry quotes,
|
|
108
|
+
parens or newlines into a re-parsed facts file - the injected-facts decision manipulation is
|
|
109
|
+
closed. The profiler's win regex is anchored.
|
|
110
|
+
- The sub-gemspecs (augure-profiler, augure-mcp) declare relative `bindir`/`require_paths` again:
|
|
111
|
+
absolute values break RubyGems `full_require_paths` (the require path gets joined against the gem
|
|
112
|
+
dir), so the gems were unrequirable in CI and in the `augure-profile` subprocess. The `files`
|
|
113
|
+
globs stay anchored to `__dir__` so a gem builds identically from any working directory.
|
|
114
|
+
|
|
115
|
+
## [0.1.0] - 2026-10-02
|
|
116
|
+
|
|
117
|
+
### Added
|
|
118
|
+
|
|
119
|
+
- Project scaffold: gemspec, RSpec, Standard linting, CI skeleton.
|
|
120
|
+
- Core decision layer: Facts (strict parser), Rules-as-data + Engine with
|
|
121
|
+
provenance, Bandit (seeded Thompson sampling), MCTS planner, KnowledgeBase
|
|
122
|
+
(TF-IDF + prior calibration), Verifier with SMT-LIB subprocess boundary,
|
|
123
|
+
Pipeline facade and `augure analyze` CLI.
|
|
124
|
+
- CTF benchmark corpus: 34 targets across ROP Emporium, Protostar, Phoenix
|
|
125
|
+
and pwnable-style suites; 5 MCTS planning scenarios; byte-identical parity
|
|
126
|
+
with the frozen Python engine (24/24 classification, 5/5 plans).
|
|
127
|
+
- Extended technique vocabulary, all documented by public suites:
|
|
128
|
+
fmtstr_leak, ret2dlresolve, ret2csu, stack_pivot, got_overwrite.
|
|
129
|
+
- `augure-benchmark`: per-suite concordance harness with exit-code gate;
|
|
130
|
+
dedicated CI job.
|
|
131
|
+
- `augure-profiler` gem (metasm isolated, LGPL): ELF in, facts out -
|
|
132
|
+
checksec, PLT imports, unsafe-symbol hints, semantic gadget
|
|
133
|
+
classification; `augure-profile` CLI; compiled-binary test suite.
|
|
134
|
+
- Documentation: commercial README (mermaid pipeline), Diataxis suite
|
|
135
|
+
(tutorial, how-tos, reference, architecture), auditability page with
|
|
136
|
+
red/blue/purple team sections.
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Gregory Epp
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,217 @@
|
|
|
1
|
+
# augure
|
|
2
|
+
|
|
3
|
+

|
|
4
|
+
|
|
5
|
+
**The auditable exploitation decision layer.**
|
|
6
|
+
|
|
7
|
+
> *Augur (n.): a Roman official who read the signs, announced the strategy,
|
|
8
|
+
> and kept the record. Before any legion moved, someone had to decide -
|
|
9
|
+
> and account for the decision.*
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
[](https://github.com/log0u7/augure/actions/workflows/ci.yml)
|
|
14
|
+
[](https://rubygems.org/gems/augure)
|
|
15
|
+
[](https://www.ruby-lang.org/)
|
|
16
|
+
[](LICENSE)
|
|
17
|
+
|
|
18
|
+
Every exploitation framework answers *"how do I run this technique?"*.
|
|
19
|
+
**Augure answers the question nobody automates: *"which technique should run,
|
|
20
|
+
and why - provably?"***
|
|
21
|
+
|
|
22
|
+
An LLM will confidently recommend `ret2libc` against a target with no libc,
|
|
23
|
+
because it generates text, not proofs. Augure generates proofs, not text:
|
|
24
|
+
|
|
25
|
+
- **Propose** - Datalog rules over the target's fact base select every
|
|
26
|
+
applicable technique. Decidable, terminating, readable.
|
|
27
|
+
- **Verify** - arithmetic and SMT-LIB checks confirm the selected technique
|
|
28
|
+
is *numerically feasible*, not merely logically applicable.
|
|
29
|
+
- **Explain** - every decision ships with its provenance: every rule that
|
|
30
|
+
fired (siblings included, each attributed), the facts that justified it,
|
|
31
|
+
the Beta prior and the seeded draw that ranked it.
|
|
32
|
+
Same input + same seed = same decision. Always.
|
|
33
|
+
|
|
34
|
+
## Why it exists
|
|
35
|
+
|
|
36
|
+
When an exploit developer picks a technique - *"this is a ret2libc situation"* -
|
|
37
|
+
that pick is a **heuristic**, and heuristics are local optima: fast, usually
|
|
38
|
+
right, never systematically tested. Augure is the test rig for those local
|
|
39
|
+
optima. Run your intuition against the frozen CTF corpus and watch the
|
|
40
|
+
weight inversions: humans overweight shellcode and `ret2libc`; the data
|
|
41
|
+
underweights them. `ret2plt` - the boring, reliable one - was massively
|
|
42
|
+
underrated.
|
|
43
|
+
|
|
44
|
+
That is the product. Not autonomy. **Accountability.**
|
|
45
|
+
|
|
46
|
+
## Quickstart
|
|
47
|
+
|
|
48
|
+
```sh
|
|
49
|
+
gem install augure
|
|
50
|
+
augure doctor # verify the environment
|
|
51
|
+
augure analyze examples/ret2win.facts # your first decision
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
(The `examples/` ship in the gem; in a repo checkout the same paths work
|
|
55
|
+
from the repo root. Not published yet? The sibling lictor checkout
|
|
56
|
+
resolves augure by path - its Gemfile says so.)
|
|
57
|
+
|
|
58
|
+
Profile a real binary into facts, then decide on them:
|
|
59
|
+
|
|
60
|
+
```sh
|
|
61
|
+
augure-profile app.elf -o target.facts # gem augure-profiler (metasm)
|
|
62
|
+
augure analyze target.facts
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
```sh
|
|
66
|
+
$ augure analyze target.facts
|
|
67
|
+
applicable: ret2plt, rop
|
|
68
|
+
selected: ret2plt
|
|
69
|
+
plan: ret2plt
|
|
70
|
+
ret2plt <- app_ret2plt (vuln=sof, nx=true, pie=false, plt=system)
|
|
71
|
+
rop <- app_rop (vuln=sof, nx=true, enough_gadgets=true)
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Or the machine-readable form:
|
|
75
|
+
|
|
76
|
+
```console
|
|
77
|
+
$ augure analyze target.facts --json
|
|
78
|
+
{
|
|
79
|
+
"schema": "augure/decision@1",
|
|
80
|
+
"applicable": ["ret2plt", "rop"],
|
|
81
|
+
"ranking": ["ret2plt", "rop"],
|
|
82
|
+
"verified": { "ret2plt": { "status": "sat", "checks": { "payload_fits": "sat" } },
|
|
83
|
+
"rop": { "status": "sat", "checks": { "payload_fits": "sat" } } },
|
|
84
|
+
"selected": "ret2plt",
|
|
85
|
+
"plan": ["ret2plt"],
|
|
86
|
+
"explain": {
|
|
87
|
+
"provenance": {
|
|
88
|
+
"ret2plt": [{ "rule": "app_ret2plt", "origin": null,
|
|
89
|
+
"evidence": [["vuln", "sof"], ["nx", "true"], ["pie", "false"], ["plt", "system"]] }]
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
}
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
From Ruby:
|
|
96
|
+
|
|
97
|
+
```ruby
|
|
98
|
+
require "augure"
|
|
99
|
+
|
|
100
|
+
result = Augure::Pipeline.analyze(facts: File.read("target.facts"), seed: 42)
|
|
101
|
+
|
|
102
|
+
result[:selected] # => "ret2plt"
|
|
103
|
+
result[:explain] # => { provenance: { "ret2plt" => [{ rule:, origin:, evidence: [...] }] } }
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
## What is in the box
|
|
107
|
+
|
|
108
|
+
| Layer | Module | What it does |
|
|
109
|
+
|---|---|---|
|
|
110
|
+
| Facts | [`Augure::Facts`](docs/reference.md#fact-format) | Strict, schema-checked fact parser. Malformed input fails loudly, never silently. |
|
|
111
|
+
| Rules | [`Augure::Rules`](docs/how-to-write-rules.md) | Rules-as-data: the single source of truth for applicability. |
|
|
112
|
+
| Engine | [`Augure::Engine`](docs/reference.md#augureengine) | Forward-chaining evaluation with per-decision provenance. |
|
|
113
|
+
| Verifier | [`Augure::Verifier`](docs/reference.md#augureverifier) | Arithmetic fast path + SMT-LIB emission. |
|
|
114
|
+
| Solvers | [`Augure::SmtProcess`](docs/how-to-swap-solver.md) | Subprocess boundary to z3 / bitwuzla / cvc5. Timeout = `:unknown`, never a crash. |
|
|
115
|
+
| Knowledge | [`Augure::KnowledgeBase`](docs/reference.md#augureknowledgebase) | 22 documented exploitation patterns; TF-IDF retrieval; Beta prior calibration. |
|
|
116
|
+
| Bandit | [`Augure::Bandit`](docs/reference.md#augurebandit) | Thompson sampling over Beta posteriors, seeded and auditable. |
|
|
117
|
+
| Planner | [`Augure::Mcts`](docs/reference.md#auguremcts) | Multi-step planning: a leak is worth what it unlocks. |
|
|
118
|
+
| CLI | [`exe/augure`](docs/tutorial.md) | Facts in, decision out, JSON if you want it. |
|
|
119
|
+
|
|
120
|
+
**Zero runtime dependencies.** Every external capability (SMT
|
|
121
|
+
solvers, LLM providers) enters through a subprocess or HTTP boundary.
|
|
122
|
+
|
|
123
|
+
## How the decision flows
|
|
124
|
+
|
|
125
|
+
```mermaid
|
|
126
|
+
graph LR
|
|
127
|
+
B["binary<br/>(augure-profiler)"] --> F["Augure::Facts<br/>strict schema<br/>+ value domains"]
|
|
128
|
+
F --> E["Augure::Engine<br/>Datalog rules-as-data"]
|
|
129
|
+
P["technique packs<br/>(build: / detection: / mitre:)"] -- "armored + corpus-guarded" --> E
|
|
130
|
+
E -- provenance --> D
|
|
131
|
+
E --> V["Augure::Verifier<br/>arithmetic / SMT-LIB"]
|
|
132
|
+
V --> B1["Augure::Bandit<br/>Beta posteriors"]
|
|
133
|
+
B1 --> M["Augure::Mcts<br/>multi-step plan (model-owned terminals)"]
|
|
134
|
+
M --> D["decision + explain<br/>augure/decision@1"]
|
|
135
|
+
KB["KnowledgeBase<br/>22 documented patterns"] -. priors .-> B1
|
|
136
|
+
SOLV["z3 / bitwuzla / cvc5<br/>(subprocess)"] -. SMT-LIB text .-> V
|
|
137
|
+
D --> COV["augure coverage<br/>the corpus read backwards"]
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
## Honest numbers
|
|
141
|
+
|
|
142
|
+
Augure's decision engine reproduces the frozen CTF benchmark corpus
|
|
143
|
+
byte-for-byte - **33/33** technique classifications across ROP Emporium,
|
|
144
|
+
Protostar, Phoenix and pwnable-style targets (16 built-in technique
|
|
145
|
+
heads plus the 20 taught technique packs),
|
|
146
|
+
and 5/5 multi-step plans including an externally documented kernel chain.
|
|
147
|
+
CI runs [`augure-benchmark`](docs/reference.md#cli) and fails the build below 100%.
|
|
148
|
+
|
|
149
|
+
Read that carefully: these are **predictions against frozen ground truth**,
|
|
150
|
+
not field results. The success probabilities in the prior table are
|
|
151
|
+
*authored, not measured*. Augure is an auditable decision layer, and the
|
|
152
|
+
audit begins with this sentence.
|
|
153
|
+
|
|
154
|
+
| Claim | Evidence |
|
|
155
|
+
|---|---|
|
|
156
|
+
| 33/33 classification parity | `lib/augure/ctf_corpus.json`, `spec/augure/engine_spec.rb` |
|
|
157
|
+
| 5/5 plan parity (incl. PinTheft kernel chain) | `spec/augure/mcts_spec.rb` |
|
|
158
|
+
| Ruby engine == frozen Python engine | corpus conformance specs, CI-enforced |
|
|
159
|
+
| MEASURED vs a typed-decision classifier (Laya, open weights): 5/5 vs 1/5 on the discriminantes, 8.5 ms vs ~3-5 s on CPU | `pappers/laya-bench.py`, the measurement story in the augure-lictor draft |
|
|
160
|
+
|
|
161
|
+
## Why not just ask an LLM?
|
|
162
|
+
|
|
163
|
+
| | LLM alone | Augure |
|
|
164
|
+
|---|---|---|
|
|
165
|
+
| recommends `ret2libc` with no libc | confidently | structurally impossible (a rule forbids it) |
|
|
166
|
+
| the 5 discriminantes (the fact text as state, the 16 heads as choices, measured) | a typed-decision model answered **1/5**, confident-and-wrong (0.93-0.98 on its errors) | **5/5**, 8.5 ms warm with the MCTS |
|
|
167
|
+
| explains *why* | plausible prose | rule ID + facts + checks + seeded draw |
|
|
168
|
+
| reproducible | no | same seed, same decision |
|
|
169
|
+
| terminates | usually | provably (Datalog) |
|
|
170
|
+
| audit trail | chat log | machine-checkable provenance |
|
|
171
|
+
|
|
172
|
+
They compose: an LLM proposes hypotheses, Augure disposes. That is the
|
|
173
|
+
generate-then-verify pattern the planning literature names LLM-Modulo
|
|
174
|
+
(Kambhampati et al., ICML 2024) and that CHECKMATE (Wang et al., 2025)
|
|
175
|
+
measured at +20% penetration success. The shipped **augure-mcp** gem
|
|
176
|
+
exposes exactly that read-only surface (analyze_target, list_rules,
|
|
177
|
+
explain_technique) to agents.
|
|
178
|
+
|
|
179
|
+
## Acceptable use
|
|
180
|
+
|
|
181
|
+
Augure is a **decision layer**. It contains no exploit code, executes
|
|
182
|
+
nothing, and touches no target. It encodes technique *metadata* - rules,
|
|
183
|
+
facts, feasibility checks - the same class of knowledge as a public
|
|
184
|
+
write-up. Use it for:
|
|
185
|
+
|
|
186
|
+
- security research, CTF preparation and training;
|
|
187
|
+
- red-team engagement planning where you are authorized to operate;
|
|
188
|
+
- defensive analysis: the same model read backwards tells you which
|
|
189
|
+
techniques your estate makes impossible.
|
|
190
|
+
|
|
191
|
+
Do not use it to attack systems you do not own or are not explicitly
|
|
192
|
+
authorized to test.
|
|
193
|
+
|
|
194
|
+
## License
|
|
195
|
+
|
|
196
|
+
MIT. The `augure-profiler` gem depends on metasm, LGPL-2.1, used as an
|
|
197
|
+
external library in the readline style.
|
|
198
|
+
|
|
199
|
+
## Documentation
|
|
200
|
+
|
|
201
|
+
The architectural renouncements and deferrals are recorded as ADRs in
|
|
202
|
+
[docs/adr/](docs/adr/): the arch abstraction before any port (0001), ARM64
|
|
203
|
+
deferred behind it (0002), PE/Windows out of scope for v0.x (0003).
|
|
204
|
+
|
|
205
|
+
| I want to... | Read |
|
|
206
|
+
|---|---|
|
|
207
|
+
| get my first decision in 10 minutes | [the tutorial](docs/tutorial.md) |
|
|
208
|
+
| see what hardening kills (the blue scorecard) | `augure coverage --flip canary:true` |
|
|
209
|
+
| align with ATT&CK (purple) | `augure rules --mitre --packs packs --json` |
|
|
210
|
+
| write my own technique rules | [rule-writing how-to](docs/how-to-write-rules.md) |
|
|
211
|
+
| teach augure a technique (packs) | [the pack format](docs/pack-format.md) |
|
|
212
|
+
| read the published lineage and where this project stands on it | [build-format](docs/build-format.md) |
|
|
213
|
+
| plug a real SMT solver | [solver how-to](docs/how-to-swap-solver.md) |
|
|
214
|
+
| look up the fact format or API | [the reference](docs/reference.md) |
|
|
215
|
+
| understand auditability (red/blue/purple) | [auditability](docs/auditability.md) |
|
|
216
|
+
| understand why rules-as-data | [architecture](docs/explanation-architecture.md) |
|
|
217
|
+
| brief an LLM agent on the pair | [the agent snippet](docs/agent-snippet.md) |
|
data/exe/augure
ADDED
|
@@ -0,0 +1,181 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
require "json"
|
|
5
|
+
require "optparse"
|
|
6
|
+
require_relative "../lib/augure"
|
|
7
|
+
|
|
8
|
+
VERSION_LINE = "augure #{Augure::VERSION}"
|
|
9
|
+
|
|
10
|
+
options = {json: false, seed: nil}
|
|
11
|
+
parser = OptionParser.new do |opts|
|
|
12
|
+
opts.banner = <<~USAGE
|
|
13
|
+
|
|
14
|
+
#{VERSION_LINE} - the auditable exploitation decision layer
|
|
15
|
+
|
|
16
|
+
Usage:
|
|
17
|
+
augure analyze <facts-file | -> [options] decide on facts
|
|
18
|
+
augure coverage [options] the corpus read backwards (defender side)
|
|
19
|
+
augure doctor verify the environment
|
|
20
|
+
augure rules print the rules-as-data table
|
|
21
|
+
augure explain <technique> rules + knowledge for one technique
|
|
22
|
+
augure --version print the version
|
|
23
|
+
|
|
24
|
+
analyze options:
|
|
25
|
+
USAGE
|
|
26
|
+
opts.on("-j", "--json", "Emit the decision as JSON (schema: augure/decision@1)") { options[:json] = true }
|
|
27
|
+
opts.on("-s", "--seed N", Integer, "Seed for deterministic bandit/MCTS") { |n| options[:seed] = n }
|
|
28
|
+
opts.on("--packs DIR", "Load technique packs (armor + corpus guard) for analyze/explain") { |v| options[:packs] = v }
|
|
29
|
+
opts.on("--flip NAME:VALUE", "Coverage: a hardening flip (nx:true, canary:true, relro:full; NAME:nil removes)") do |v|
|
|
30
|
+
name, value = v.split(":", 2)
|
|
31
|
+
(options[:flips] ||= {})[name] = ((value == "nil") ? nil : value)
|
|
32
|
+
end
|
|
33
|
+
opts.on("--facts FILE", "Coverage: one facts entry instead of the corpus") { |v| options[:facts_entry] = v }
|
|
34
|
+
opts.on("--mitre", "rules: export the ATT&CK navigator layer (from the packs' mitre: fields)") { options[:mitre] = true }
|
|
35
|
+
opts.on("--version", "Print the version") do
|
|
36
|
+
puts VERSION_LINE
|
|
37
|
+
exit 0
|
|
38
|
+
end
|
|
39
|
+
opts.on("-h", "--help", "Show this help") do
|
|
40
|
+
puts opts
|
|
41
|
+
exit 0
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
parser.parse!
|
|
45
|
+
|
|
46
|
+
command = ARGV.shift
|
|
47
|
+
|
|
48
|
+
case command
|
|
49
|
+
when "--version", "-v"
|
|
50
|
+
puts VERSION_LINE
|
|
51
|
+
when "doctor"
|
|
52
|
+
checks = []
|
|
53
|
+
checks << ["ruby >= 3.4", (RUBY_VERSION >= "3.4") ? nil : "upgrade ruby (mise install ruby@3.4)"]
|
|
54
|
+
checks << [VERSION_LINE, nil]
|
|
55
|
+
|
|
56
|
+
# The corpus self-verifies: parse a shipped example, decide, compare.
|
|
57
|
+
begin
|
|
58
|
+
example = File.join(__dir__, "../examples/ret2win.facts")
|
|
59
|
+
result = Augure::Pipeline.analyze(facts: File.read(example))
|
|
60
|
+
checks << if result[:ranking].first == "ret2func"
|
|
61
|
+
["corpus self-test (selected: ret2func)", nil]
|
|
62
|
+
else
|
|
63
|
+
["corpus self-test", "corpus example decided differently - report a bug"]
|
|
64
|
+
end
|
|
65
|
+
rescue => e
|
|
66
|
+
checks << ["corpus self-test", e.message]
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
solvers = %w[z3 bitwuzla cvc5].select do |s|
|
|
70
|
+
system("sh", "-c", "command -v '#{s}' >/dev/null 2>&1")
|
|
71
|
+
end
|
|
72
|
+
checks << ["SMT solvers",
|
|
73
|
+
solvers.empty? ? "optional - none found; the arithmetic path works without (apt install z3 for the SMT path)" : "optional, found: #{solvers.join(", ")}"]
|
|
74
|
+
|
|
75
|
+
failed = false
|
|
76
|
+
checks.each do |name, problem|
|
|
77
|
+
if problem&.start_with?("optional")
|
|
78
|
+
puts " opt #{name} - #{problem}"
|
|
79
|
+
elsif problem
|
|
80
|
+
failed = true
|
|
81
|
+
puts " FAIL #{name} - fix: #{problem}"
|
|
82
|
+
else
|
|
83
|
+
puts " ok #{name}"
|
|
84
|
+
end
|
|
85
|
+
end
|
|
86
|
+
if failed
|
|
87
|
+
n = checks.count { |_, p| p && !p.start_with?("optional") }
|
|
88
|
+
abort("augure: doctor found #{n} issue(s) - fixes above")
|
|
89
|
+
end
|
|
90
|
+
when "rules"
|
|
91
|
+
if options[:mitre]
|
|
92
|
+
mitre_by_pack = {}
|
|
93
|
+
packs_dir = options[:packs] || "packs"
|
|
94
|
+
Dir.glob(File.expand_path(File.join(packs_dir, "*.yml"))).each do |path|
|
|
95
|
+
pack = Augure::Pack.load_file(path)
|
|
96
|
+
(mitre_by_pack[pack.technique] = pack.mitre) if pack.mitre
|
|
97
|
+
rescue
|
|
98
|
+
nil
|
|
99
|
+
end
|
|
100
|
+
if options[:json]
|
|
101
|
+
puts JSON.pretty_generate("schema" => "augure/mitre-navigator@1", "layers" => [{
|
|
102
|
+
"name" => "augure techniques", "versions" => {"attack" => "14", "navigator" => "4.9"},
|
|
103
|
+
"techniques" => mitre_by_pack.map { |tech, codes| codes.map { |c| {"techniqueID" => c, "score" => 1, "comment" => tech} } }.flatten
|
|
104
|
+
}])
|
|
105
|
+
exit 0
|
|
106
|
+
end
|
|
107
|
+
mitre_by_pack.each { |tech, codes| puts "#{tech}\t#{codes.join(", ")}" }
|
|
108
|
+
exit 0
|
|
109
|
+
end
|
|
110
|
+
Augure::Rules.all.each do |rule|
|
|
111
|
+
puts rule.id
|
|
112
|
+
puts " head: #{rule.head.join("=")}"
|
|
113
|
+
puts " why: #{rule.source}"
|
|
114
|
+
end
|
|
115
|
+
when "explain"
|
|
116
|
+
technique = ARGV.shift or abort(parser.to_s)
|
|
117
|
+
packs = options[:packs] && Augure::PackLoader.load_dir(options[:packs]).tap { |ps| Augure::PackLoader.corpus_guard(ps) }
|
|
118
|
+
rules = Augure::Rules.all + packs.to_a.flat_map(&:rules)
|
|
119
|
+
rules = rules.select { |r| r.head == [Augure::Rules::APPLICABLE, technique] }
|
|
120
|
+
entries = Augure::KnowledgeBase.corpus_with(packs.to_a.flat_map(&:kb))
|
|
121
|
+
.select { |e| e["technique"] == technique }
|
|
122
|
+
if rules.empty? && entries.empty?
|
|
123
|
+
known = Augure::Rules.applicable_rules.map { |r| r.head[1] }.uniq.sort
|
|
124
|
+
abort("augure: unknown technique '#{technique}' - known: #{known.join(", ")}")
|
|
125
|
+
end
|
|
126
|
+
puts "technique: #{technique}"
|
|
127
|
+
rules.each do |rule|
|
|
128
|
+
puts " rule: #{rule.id} - #{rule.source}"
|
|
129
|
+
end
|
|
130
|
+
entries.each do |e|
|
|
131
|
+
puts " kb: #{e["id"]} (success_rate #{e["success_rate"]})"
|
|
132
|
+
puts " #{e["description"]}"
|
|
133
|
+
end
|
|
134
|
+
when "coverage"
|
|
135
|
+
entries = if options[:facts_entry]
|
|
136
|
+
[{"suite" => "facts", "name" => File.basename(options[:facts_entry]),
|
|
137
|
+
"facts" => File.read(options[:facts_entry])}]
|
|
138
|
+
else
|
|
139
|
+
corpus = JSON.parse(File.read(File.expand_path("../lib/augure/ctf_corpus.json", __dir__)))
|
|
140
|
+
corpus["suites"].values.flatten
|
|
141
|
+
end
|
|
142
|
+
unless options[:flips]
|
|
143
|
+
abort("augure: coverage needs at least one --flip NAME:VALUE (nx:true, canary:true, relro:full)")
|
|
144
|
+
end
|
|
145
|
+
report = Augure::Coverage.report(entries, flips: options[:flips])
|
|
146
|
+
if options[:json]
|
|
147
|
+
puts JSON.pretty_generate("schema" => "augure/coverage@1", **report)
|
|
148
|
+
exit 0
|
|
149
|
+
end
|
|
150
|
+
entries.size
|
|
151
|
+
report[:entries].count { |e| e[:lost].empty? && e[:gained].empty? }
|
|
152
|
+
puts "the corpus read backwards: #{entries.size} targets, flips #{options[:flips].inspect}"
|
|
153
|
+
puts "targets that change their applicable set: #{report[:entries].count { |e| e[:lost] || e[:gained] } - report[:entries].count { |e| e[:lost].empty? && e[:gained].empty? }}"
|
|
154
|
+
report[:died].each { |tech, n| puts " died: #{tech} (#{n} target(s))" }
|
|
155
|
+
report[:gained].each { |tech, n| puts " gained: #{tech} (#{n} target(s))" }
|
|
156
|
+
puts "untouched targets: #{report[:survived_count]}"
|
|
157
|
+
puts "the remaining applicable techniques are the attacker's next moves - harden accordingly."
|
|
158
|
+
when "analyze"
|
|
159
|
+
path = ARGV.shift or abort(parser.to_s)
|
|
160
|
+
input = (path == "-") ? $stdin.read : File.read(path)
|
|
161
|
+
begin
|
|
162
|
+
packs = options[:packs] && Augure::PackLoader.load_dir(options[:packs]).tap { |ps| Augure::PackLoader.corpus_guard(ps) }
|
|
163
|
+
result = Augure::Pipeline.analyze(facts: input, seed: options[:seed], packs: packs)
|
|
164
|
+
if options[:json]
|
|
165
|
+
puts JSON.pretty_generate({schema: "augure/decision@1"}.merge(result))
|
|
166
|
+
else
|
|
167
|
+
puts "applicable: #{result[:applicable].join(", ")}"
|
|
168
|
+
puts "selected: #{result[:selected]}"
|
|
169
|
+
puts "plan: #{result[:plan].join(" -> ")}"
|
|
170
|
+
result[:explain][:provenance].each do |tech, provs|
|
|
171
|
+
provs.each do |p|
|
|
172
|
+
puts " #{tech} <- #{p[:rule]} (#{p[:evidence].map { |e| e.join("=") }.join(", ")})"
|
|
173
|
+
end
|
|
174
|
+
end
|
|
175
|
+
end
|
|
176
|
+
rescue Augure::MalformedFact, Augure::UnknownPredicate => e
|
|
177
|
+
abort("augure: #{e.message}")
|
|
178
|
+
end
|
|
179
|
+
else
|
|
180
|
+
abort(parser.to_s)
|
|
181
|
+
end
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
require "json"
|
|
5
|
+
require "optparse"
|
|
6
|
+
require_relative "../lib/augure"
|
|
7
|
+
|
|
8
|
+
# Concordance benchmark against the frozen CTF corpus, in the spirit of the
|
|
9
|
+
# Python prototype's benchmark harness: predictions vs documented truth,
|
|
10
|
+
# per suite, with an explicit exit code gate.
|
|
11
|
+
|
|
12
|
+
threshold = 1.00
|
|
13
|
+
OptionParser.new do |opts|
|
|
14
|
+
opts.banner = "Usage: augure-benchmark <corpus.json> [--min-top1 N]"
|
|
15
|
+
opts.on("--min-top1 N", Float, "Fail below this top-1 concordance (default 1.00)") { |v| threshold = v }
|
|
16
|
+
opts.on("--version", "Print the version") do
|
|
17
|
+
puts "augure #{Augure::VERSION} (benchmark)"
|
|
18
|
+
exit 0
|
|
19
|
+
end
|
|
20
|
+
end.parse!(ARGV)
|
|
21
|
+
|
|
22
|
+
path = ARGV.shift or abort("Usage: augure-benchmark <corpus.json> [--min-top1 N]")
|
|
23
|
+
corpus = JSON.parse(File.read(path))
|
|
24
|
+
meta_priors = corpus.dig("meta", "priors")
|
|
25
|
+
|
|
26
|
+
puts " augure-benchmark: prediction vs documented ground truth"
|
|
27
|
+
puts
|
|
28
|
+
|
|
29
|
+
correct_top1 = 0
|
|
30
|
+
correct_top3 = 0
|
|
31
|
+
n = 0
|
|
32
|
+
|
|
33
|
+
corpus["suites"].each do |suite, entries|
|
|
34
|
+
suite_ok1 = 0
|
|
35
|
+
suite_ok3 = 0
|
|
36
|
+
entries.each do |entry|
|
|
37
|
+
result = Augure::Pipeline.analyze(facts: entry["facts"], priors: meta_priors)
|
|
38
|
+
top3 = result[:ranking].first(3)
|
|
39
|
+
# An empty frozen verdict is a verdict: concordance = the engine
|
|
40
|
+
# reproduces it exactly (the pack layer owns the technique; the
|
|
41
|
+
# built-in table intentionally says nothing).
|
|
42
|
+
if entry["applicable"].empty?
|
|
43
|
+
ok = result[:applicable].empty?
|
|
44
|
+
suite_ok1 += 1 if ok
|
|
45
|
+
suite_ok3 += 1 if ok
|
|
46
|
+
next
|
|
47
|
+
end
|
|
48
|
+
suite_ok1 += 1 if result[:ranking].first == entry["truth"]
|
|
49
|
+
suite_ok3 += 1 if top3.include?(entry["truth"])
|
|
50
|
+
end
|
|
51
|
+
count = entries.size
|
|
52
|
+
n += count
|
|
53
|
+
correct_top1 += suite_ok1
|
|
54
|
+
correct_top3 += suite_ok3
|
|
55
|
+
puts format(" %-12s %d/%d top-1 %d/%d top-3",
|
|
56
|
+
suite, suite_ok1, count, suite_ok3, count)
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
puts
|
|
60
|
+
pct = correct_top1.to_f / n
|
|
61
|
+
puts format(" Top-1 concordance: %d/%d (%.0f%%)", correct_top1, n, pct * 100)
|
|
62
|
+
puts format(" Top-3 concordance: %d/%d (%.0f%%)", correct_top3, n, (correct_top3.to_f / n) * 100)
|
|
63
|
+
|
|
64
|
+
if pct < threshold
|
|
65
|
+
puts format(" BELOW THRESHOLD (%.0f%% < %.0f%%)", pct * 100, threshold * 100)
|
|
66
|
+
exit 1
|
|
67
|
+
end
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Agent snippet: paste into your project's AGENTS.md
|
|
2
|
+
|
|
3
|
+
```markdown
|
|
4
|
+
## augure/lictor - the auditable exploitation decision layer
|
|
5
|
+
|
|
6
|
+
- augure DECIDES (verified techniques + provenance); lictor ORCHESTRATES
|
|
7
|
+
(authorization + trail). I propose; augure disposes. I never claim a technique
|
|
8
|
+
augure did not rank, and I quote decisions as predictions with
|
|
9
|
+
provenance, never as proven results.
|
|
10
|
+
- Loop (all recorded in the trail): `lictor doctor` ->
|
|
11
|
+
`lictor scan <host>` -> `lictor acquire <target>
|
|
12
|
+
--from <url>|-bin <path>` -> [optional discovery: `lictor fuzz`,
|
|
13
|
+
`lictor verify`] -> `lictor suggest <facts> "plt(\"system\")."`
|
|
14
|
+
(propose observations; the delta comes back) -> `lictor plan
|
|
15
|
+
<binary-or-facts> [--json]` -> `lictor build <binary>
|
|
16
|
+
--facts <facts>` (offset measured, payload file) ->
|
|
17
|
+
`lictor exploit scaffold <technique> <binary> --facts <facts>
|
|
18
|
+
--repo-dir DIR` (no exploit in the repos? the draft lands, review it)
|
|
19
|
+
-> `lictor draft <facts> [binary]` (nothing applies? a staged pack)
|
|
20
|
+
`lictor run <target> --facts <f> --executor ronin`
|
|
21
|
+
-> `lictor outcome <run-id> --success|--failure --technique <t>` ->
|
|
22
|
+
`lictor report`. Full skill: augure/skills/augure-operator/SKILL.md.
|
|
23
|
+
- New technique = a PACK (YAML data, never code): copy
|
|
24
|
+
`augure/packs/ret2csu_v2.yml`, validate with `lictor rule validate`,
|
|
25
|
+
install only with operator confirmation. The corpus guard refuses
|
|
26
|
+
packs that move frozen verdicts - read the refusal, fix the data.
|
|
27
|
+
- Authorized use only: the operator decides the scope, the trail
|
|
28
|
+
records it. No
|
|
29
|
+
exceptions. The trail is never edited.
|
|
30
|
+
- Payload contract: `--json` emits schema "augure/decision@1".
|
|
31
|
+
```
|