twinflame 0.1.0b1__tar.gz

twinflame-0.1.0b1/PKG-INFO

@@ -0,0 +1,274 @@
+Metadata-Version: 2.4
+Name: twinflame
+Version: 0.1.0b1
+Summary: DEX-level Android APK class-diffing engine (SimHash + LSH + abstract-opcode comparison)
+Author: twinflame contributors
+License: MIT
+Project-URL: Homepage, https://github.com/ankorio/twinflame
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Topic :: Security
+Classifier: Topic :: Software Development :: Disassemblers
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: androguard>=4.1
+Requires-Dist: numpy>=1.26
+Requires-Dist: rapidfuzz>=3.14
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == "test"
+Requires-Dist: pytest-cov>=5; extra == "test"
+Requires-Dist: hypothesis>=6; extra == "test"
+
+# twinflame
+
+DEX/APK **bytecode-level** diffing engine for Android reverse engineers. Matches classes and methods across two builds by *structure* — surviving R8/ProGuard renaming — and emits a ranked, method-localized **change report** (what was added / removed / modified) plus a ProGuard `mapping.txt` to carry recovered names into your decompiler. Implements the four-stage architecture from Quarkslab's _"Android Application Diffing: Engine Overview"_ (Czayka & Thomas, 2019).
+
+Use cases:
+
+- **Vulnerability patch identification** — find the class a vendor changed to fix a flaw.
+- **Repackaging / mod analysis** — find injected code that replaces vendor logic with no-ops or hostile callbacks.
+- **Malware triage / kinship** — find shared code across samples and flag when both implement the same permission-gated component (AccessibilityService, NotificationListener, DeviceAdmin).
+- **Deobfuscation correlation** — pair classes across Proguard-renamed builds.
+
+Inputs can be APKs, single `.dex` files, or directories of dumped `.dex` (memory dumps included). For pipelines that compare a sample against many, `twinflame prepare` fingerprints a sample once into a reusable record so later comparisons skip the expensive parse.
+
+## Documentation
+
+- **[docs/using-twinflame.md](docs/using-twinflame.md)** — task-oriented guide: the three use cases, reading the change report, feeding the mapping to jadx/retrace.
+- **[docs/change-report-schema.md](docs/change-report-schema.md)** — the versioned JSON output format (for downstream tooling).
+- **[docs/how-it-works.md](docs/how-it-works.md)** — how the pipeline works internally, with diagrams.
+
+## Pipeline
+
+1. **Multi-DEX merge** — every `classes*.dex` is unioned into one logical view (first-wins on descriptor, matching ART). The loader also records each method's call targets + incoming-xref count and each class's string constants, which feed anchoring.
+2. **Stage 1 — clustering.** Classes group into pools keyed by package name; matching pools across the two APKs compare in parallel. Obfuscated packages (Shannon entropy + length heuristic) fall into a single fallback pool.
+3. **Stage B — anchoring.** Within each pool, R8-invariant seeds are locked in *before* structural scoring: classes sharing a rare string constant (IDF-weighted) or a distinctive set of `android/*`/`androidx/*`/`java/*`/`kotlin/*` framework calls are paired with high confidence even when SimHash can't tell them apart. Disable with `--no-anchors`.
+4. **Stage 2 — bulk SimHash + bucketed LSH.** Each remaining class gets a 128-bit signature built from four 32-bit partial hashes — `cls`, `fld`, `mth`, `code` — over **structure-only** features (no names, no strings). Nearest neighbors are found via Dullien-style bit-permutation LSH instead of all-pairs.
+5. **Stage 3 — accurate, method-level comparison.** Top-k Stage-2 neighbors are re-scored at **method granularity**: methods are assigned 1-to-1 within a class pair (abstract-opcode Levenshtein over 13 semantic categories + prototype + xref), yielding per-method `matched / modified / added / deleted` verdicts. The class score is the instruction-weighted roll-up of those verdicts blended with structural features. Greedy 1-to-1 assignment resolves class-level ties.
+
+Optional Redex pre-pass strips junk-instruction obfuscation (`LocalDcePass` + `RegAllocPass`) before loading.
+
+## Installation
+
+### pip (recommended)
+
+Requires Python ≥ 3.11. Runtime deps (androguard, numpy, rapidfuzz) install automatically.
+
+```sh
+git clone https://github.com/ankorio/twinflame && cd twinflame
+python -m venv .venv && . .venv/bin/activate
+pip install .            # or: pip install -e '.[test]' for a dev checkout
+twinflame --help           # console entry point is installed
+```
+
+That's everything for the core tool. Two features need external programs that aren't Python
+packages: `--normalize` needs **Redex** on `PATH` (optional; only for junk-instruction
+normalization), and applying a recovered `mapping.txt` needs your own decompiler (e.g. **JADX**).
+
+### Nix flake (reproducible environment)
+
+For a fully-pinned environment (Python + all deps + Redex built from source + JADX), use the
+[Nix flake](https://nixos.wiki/wiki/Flakes) — everything is locked in [flake.lock](flake.lock):
+
+```sh
+# Install Nix with flakes (Determinate Systems installer is easiest):
+curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
+git clone https://github.com/ankorio/twinflame && cd twinflame
+nix develop            # drops you in a shell with twinflame + Redex + JADX
+```
+
+## Running
+
+### One-shot CLI via `nix run`
+
+`nix run` builds the tool on demand and invokes it. The first run compiles Redex from source (~5 min); after that everything is cached.
+
+```sh
+# diff two APKs, scope to a vendor package, write a JSON report
+nix run . -- old.apk new.apk \
+    --auto-package \
+    --threshold 0.8 \
+    -o report.json
+
+# real-world example: vuln/patched pair, Redex-normalized to defeat
+# junk-instruction obfuscation, with the obfuscated-package fallback
+# so Proguard-renamed packages cluster together for comparison
+nix run . -- vuln.apk patched.apk --normalize --find-obfuscated
+```
+
+### Inputs: APK, `.dex`, or dumped DEX (no APK)
+
+Each side accepts an APK, a single `.dex`, or a directory of `.dex` files — so
+you can diff **dumped/extracted DEX** (e.g. pulled from memory or an unpacked
+payload) with no surrounding APK, with no flag: the input kind is detected.
+Partial dumps are tolerated — an unparseable or corrupt `.dex` in a directory is
+skipped (with a warning) rather than aborting the load. A DEX-only input has no
+manifest, so `--auto-package` is unavailable — pass `--app-package <prefix>` for
+app-vs-library ranking instead.
+
+```sh
+# directory of dumped classes*.dex on each side
+twinflame dump_old/ dump_new/ --no-cluster
+
+# write the machine report as CSV instead of the default JSON, only the
+# classes that actually differ
+twinflame a/ b/ --no-cluster --app-package com.target.app -s changed -f csv -o changes.csv
+```
+
+By default, stdout shows a ranked **change summary** (modified / added / removed /
+cosmetic / unchanged, app code first) and any sensitive components both builds
+share, while the full machine report is written to a file (`-f json`, default;
+`csv`/`xml` also available). Pass `-m`/`--matches` for the raw per-class match
+list instead:
+
+```
+[+] com.acme.foo: Login - Login.java | com.acme.foo: Login - SourceFile -> 0.8523
+      ~ validateToken (0.6100)                              # method modified
+      - legacyHash                                          # method removed
+      + verifyNonce                                         # method added
+[-] com.acme.foo: DeletedClass - DeletedClass.java          # in lhs only
+[*] com.acme.foo: AddedClass - SourceFile                   # in rhs only
+```
+
+Each `[+]` paired line is followed by the methods that actually changed, so you
+see *which* method moved the class score. The machine report carries the full
+per-method verdict list, plus each paired class's superclass/interfaces and any
+sensitive component it implements.
+
+Timings go to stderr; the report to stdout (so you can `| less` or redirect freely).
+
+#### Worked example
+
+Diffing a vuln/patched APK pair (a real Android wallet app, ~8 k classes per side, the new build is Proguard'd):
+
+```sh
+$ nix run . -- old.apk new.apk --normalize --find-obfuscated
+
+```
+
+Reading the output:
+
+- The `load:` and `diff:` lines on **stderr** report timings and class counts so you can see what was actually compared after `--normalize` (Redex strips junk instructions on both sides) and post-filtering.
+- `[+]` lines are paired matches with their similarity score (`distance < 1.0`). The two `androidx.activity` examples above pair clear-text class names on the left (lhs preserved source files) against the same logical classes on the right (rhs lost source files to `SourceFile` — Proguard's default). Source-file name was dropped, but the structure + bytecode similarity is high (~0.85–0.95).
+- `[-]` is a class only present in lhs; `[*]` only in rhs. Counts at the top tell you the global picture (e.g. _11 042 matches: 602 paired below 1.0, 5 027 deleted, 5 664 added_ — the bulk being unmatched is the price of heavy obfuscation; tweak `--find-obfuscated` and `--threshold` to trade quality for coverage).
+- Drop the `--normalize` and re-run to see what _uncleaned_ bytecode looks like — if the average paired similarity drops from ~0.9 to ~0.95+ everywhere, that's the fingerprint of junk-instruction obfuscation that Redex defeats.
+
+### Build a standalone binary
+
+```sh
+nix build              # produces ./result/bin/twinflame
+./result/bin/twinflame --help
+```
+
+### Dev shell
+
+For interactive use (REPL, hacking on the code, running tests):
+
+```sh
+nix develop            # enter shell with python + redex + jadx on PATH
+pytest tests/          # 86 tests, ~8s
+python -m twinflame.cli --help
+```
+
+### Python API
+
+```python
+from twinflame import load, filter, diff
+
+lhs_app = load("app-1.6.1.apk")
+rhs_app = load("app-1.6.3.apk")
+
+lhs = filter(lhs_app.classes, {"package_filtering": "com.vendor.app"})
+rhs = filter(rhs_app.classes, {"package_filtering": "com.vendor.app"})
+
+matches = diff(lhs, rhs, 0.8, {
+    "synthetic_skipping": True,
+    "min_inst_size_threshold": 5,
+    "top_match_threshold": 3,
+})
+for m in matches:
+    if m.is_paired and m.distance < 1.0:
+        print(f"[+] {m.lhs.info} | {m.rhs.info} -> {m.distance:.4f}")
+```
+
+### Cross-version deobfuscation map
+
+`--deobfuscation-map OUT` writes a ProGuard-format `mapping.txt` for **apk2** (the obfuscated/stripped build) by propagating class names recovered from the matched classes in **apk1** (the donor build that still carries the DEX `SourceFile` attribute). Load it into JADX/Ghidra and the rename propagates to every reference automatically — twinflame never rewrites the DEX.
+
+```sh
+# apk1 = older build that kept SourceFile; apk2 = the one you want to read
+twinflame app-old.apk app-new.apk \
+    --find-obfuscated --deobfuscation-map app-new.map --map-min-confidence 0.8
+```
+
+Load into JADX with `-Prename-mappings.invert=yes`:
+
+```sh
+jadx --mappings-path app-new.map -Prename-mappings.format=PROGUARD_FILE -Prename-mappings.invert=yes \
+    -d out_deobf app-new.apk
+```
+
+`invert=yes` is required: our file follows the standard ProGuard convention (`<original> -> <obfuscated>:`, what `retrace` consumes), but jadx's `PROGUARD_FILE` reader treats the left side as the name *currently in the binary* — the opposite. Without the flag it silently renames nothing (no error). Same applies in jadx-gui: check "Invert" alongside the mapping path.
+
+- Recovers the class **simple name** (`ContextCompat.java` → `x6.q` becomes `x6.ContextCompat`); the obfuscated **package** and inner-class structure are kept (source files carry no package).
+- Anchored / exact (`distance == 1.0`) matches are trusted; lower-confidence ones are still emitted but flagged with a `# low-confidence` comment. Tune the floor with `--map-min-confidence`.
+- This is the cross-version superpower over single-APK source-file deobfuscation: it names classes in a build that **stripped** `SourceFile`, using the build that didn't. Method/field name propagation is a planned next step.
+
+### Workflow recipes
+
+- **Patch identification.** Scope with `--package PREFIX` or `--auto-package` (drastically shrinks the comparison set), use `--threshold 0.8` since bug fixes are minor changes. Skip `distance == 1.0` matches.
+- **Modded / repackaged app.** Drop `--package` (mods often live in bundled SDKs, not the vendor package). If you see a sea of ~0.95 "everything changed" matches, run with `--normalize` — that's the signature of junk-instruction obfuscation.
+- **Heavily Proguard'd inputs.** Add `--find-obfuscated` so single-letter package names (`a.b.c`) collapse into a fallback pool rather than failing to pair with their clear-text counterparts.
+- **Deobfuscating a stripped build.** Diff it against an older build that kept `SourceFile`, add `--find-obfuscated --deobfuscation-map out.map`, then load `out.map` in JADX.
+
+### Evaluation harness (`eval/`, plan M3.2)
+
+Grades twinflame's own class matches against real ground truth instead of eyeballing them. Build an OSS Android app **twice** — R8 off, then R8 on at full optimization — the `mapping.txt` R8 writes for the R8-on build is a free, perfect oracle for the renaming-only case (directly grades M1.1 + M1.2; inlining/outlining is a future difficulty-graded slice per M3.1).
+
+```sh
+python -m eval.cli app-r8-off.apk app-r8-on.apk app-r8-on/mapping.txt --package com.vendor.app
+```
+
+Prints true/false positive & negative class-match counts plus precision/recall/F1. `eval/mapping.py` parses the ProGuard-format oracle (same format `deobf.py` emits); `eval/score.py` exposes `score_class_matches(matches, ground_truth) -> ScoreResult` for use as a library, independent of the CLI. Kept out of the `src/twinflame` package on purpose — it's tooling to grade the engine, not part of it.
+
+## Layout
+
+| Module                                               | Role                                                                 |
+| ---------------------------------------------------- | -------------------------------------------------------------------- |
+| [src/twinflame/model.py](src/twinflame/model.py)         | Dataclasses: `App`, `Class`, `Method`, `Field`, `Signature`, `Match`, `MethodMatch` |
+| [src/twinflame/loader.py](src/twinflame/loader.py)       | androguard wrapping + multi-DEX merge; captures calls/strings/xrefs  |
+| [src/twinflame/manifest.py](src/twinflame/manifest.py)   | `AndroidManifest.xml` → `ManifestInfo`; dev-package suggestion       |
+| [src/twinflame/normalize.py](src/twinflame/normalize.py) | Redex subprocess wrapper                                             |
+| [src/twinflame/cluster.py](src/twinflame/cluster.py)     | Package-based pools + entropy/length obfuscation fallback            |
+| [src/twinflame/anchor.py](src/twinflame/anchor.py)       | Stage B: string-IDF + framework-call seed matches                    |
+| [src/twinflame/signature.py](src/twinflame/signature.py) | 128-bit SimHash + LSH bucket index                                   |
+| [src/twinflame/accurate.py](src/twinflame/accurate.py)   | Abstract opcodes, method-level + class scoring, greedy 1-to-1 assignment |
+| [src/twinflame/opcodes.py](src/twinflame/opcodes.py)     | Dalvik opcode → 13-category lookup table                             |
+| [src/twinflame/\_hot.py](src/twinflame/_hot.py)          | Hot loops (popcount, Hamming, rapidfuzz-backed Levenshtein)          |
+| [src/twinflame/deobf.py](src/twinflame/deobf.py)         | Cross-version deobfuscation: recovered names → ProGuard mapping.txt   |
+| [src/twinflame/api.py](src/twinflame/api.py)             | Public `load / filter / diff` entry points                           |
+| [src/twinflame/cli.py](src/twinflame/cli.py)             | `twinflame` console script                                             |
+
+## Tests
+
+```sh
+nix develop --command pytest tests/
+```
+
+Currently: **86 tests passing**, 0 skipped. Redex is built and on PATH inside the dev shell.
+
+Test fixtures are **pure-synthetic**: `tests/fixtures/synthetic.py` builds `Class` objects directly via Python, no binaries committed to git. The integration tests exercise the full diff pipeline (cluster → anchor → signature → accurate → api) on these objects.
+
+The on-disk DEX/APK round-trip (loader.py end-to-end) currently has a smoke-only test; a real binary DEX emitter under [tests/fixtures/build_dex.py](tests/fixtures/build_dex.py) is stubbed for future work.
+
+## Known limitations
+
+- **Identical-structure classes collide.** Pools where many classes share signatures (trivial getter/setter classes, generated stubs) may produce false pairings via greedy assignment. `--min-instr 5` filters trivial classes; raising `--neighbors` widens the candidate pool.
+- **No inheritance context.** Matching is per-class (now with method-level detail *inside* a paired class), but ignores first-level parent/child signals (cf. LibPecker); these can be added later without disturbing the core.
+- **No cross-boundary / optimization resilience yet.** Matching assumes a 1:1 class and method correspondence. R8 *optimizations* that break that assumption — inlining, outlining, class merging — are a planned future track (see `twinflame-next-dev-plan.md`, M3.1), not yet implemented.
+- **JNI / native-code changes are invisible.** The diff is purely Dalvik-level — native library mutations require a complementary native-code diff.
+- **LSH is approximate by design.** `--buckets N` tunes the accuracy/speed knob.
+
+## License
+
+MIT.

twinflame-0.1.0b1/README.md

@@ -0,0 +1,253 @@
+# twinflame
+
+DEX/APK **bytecode-level** diffing engine for Android reverse engineers. Matches classes and methods across two builds by *structure* — surviving R8/ProGuard renaming — and emits a ranked, method-localized **change report** (what was added / removed / modified) plus a ProGuard `mapping.txt` to carry recovered names into your decompiler. Implements the four-stage architecture from Quarkslab's _"Android Application Diffing: Engine Overview"_ (Czayka & Thomas, 2019).
+
+Use cases:
+
+- **Vulnerability patch identification** — find the class a vendor changed to fix a flaw.
+- **Repackaging / mod analysis** — find injected code that replaces vendor logic with no-ops or hostile callbacks.
+- **Malware triage / kinship** — find shared code across samples and flag when both implement the same permission-gated component (AccessibilityService, NotificationListener, DeviceAdmin).
+- **Deobfuscation correlation** — pair classes across Proguard-renamed builds.
+
+Inputs can be APKs, single `.dex` files, or directories of dumped `.dex` (memory dumps included). For pipelines that compare a sample against many, `twinflame prepare` fingerprints a sample once into a reusable record so later comparisons skip the expensive parse.
+
+## Documentation
+
+- **[docs/using-twinflame.md](docs/using-twinflame.md)** — task-oriented guide: the three use cases, reading the change report, feeding the mapping to jadx/retrace.
+- **[docs/change-report-schema.md](docs/change-report-schema.md)** — the versioned JSON output format (for downstream tooling).
+- **[docs/how-it-works.md](docs/how-it-works.md)** — how the pipeline works internally, with diagrams.
+
+## Pipeline
+
+1. **Multi-DEX merge** — every `classes*.dex` is unioned into one logical view (first-wins on descriptor, matching ART). The loader also records each method's call targets + incoming-xref count and each class's string constants, which feed anchoring.
+2. **Stage 1 — clustering.** Classes group into pools keyed by package name; matching pools across the two APKs compare in parallel. Obfuscated packages (Shannon entropy + length heuristic) fall into a single fallback pool.
+3. **Stage B — anchoring.** Within each pool, R8-invariant seeds are locked in *before* structural scoring: classes sharing a rare string constant (IDF-weighted) or a distinctive set of `android/*`/`androidx/*`/`java/*`/`kotlin/*` framework calls are paired with high confidence even when SimHash can't tell them apart. Disable with `--no-anchors`.
+4. **Stage 2 — bulk SimHash + bucketed LSH.** Each remaining class gets a 128-bit signature built from four 32-bit partial hashes — `cls`, `fld`, `mth`, `code` — over **structure-only** features (no names, no strings). Nearest neighbors are found via Dullien-style bit-permutation LSH instead of all-pairs.
+5. **Stage 3 — accurate, method-level comparison.** Top-k Stage-2 neighbors are re-scored at **method granularity**: methods are assigned 1-to-1 within a class pair (abstract-opcode Levenshtein over 13 semantic categories + prototype + xref), yielding per-method `matched / modified / added / deleted` verdicts. The class score is the instruction-weighted roll-up of those verdicts blended with structural features. Greedy 1-to-1 assignment resolves class-level ties.
+
+Optional Redex pre-pass strips junk-instruction obfuscation (`LocalDcePass` + `RegAllocPass`) before loading.
+
+## Installation
+
+### pip (recommended)
+
+Requires Python ≥ 3.11. Runtime deps (androguard, numpy, rapidfuzz) install automatically.
+
+```sh
+git clone https://github.com/ankorio/twinflame && cd twinflame
+python -m venv .venv && . .venv/bin/activate
+pip install .            # or: pip install -e '.[test]' for a dev checkout
+twinflame --help           # console entry point is installed
+```
+
+That's everything for the core tool. Two features need external programs that aren't Python
+packages: `--normalize` needs **Redex** on `PATH` (optional; only for junk-instruction
+normalization), and applying a recovered `mapping.txt` needs your own decompiler (e.g. **JADX**).
+
+### Nix flake (reproducible environment)
+
+For a fully-pinned environment (Python + all deps + Redex built from source + JADX), use the
+[Nix flake](https://nixos.wiki/wiki/Flakes) — everything is locked in [flake.lock](flake.lock):
+
+```sh
+# Install Nix with flakes (Determinate Systems installer is easiest):
+curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
+git clone https://github.com/ankorio/twinflame && cd twinflame
+nix develop            # drops you in a shell with twinflame + Redex + JADX
+```
+
+## Running
+
+### One-shot CLI via `nix run`
+
+`nix run` builds the tool on demand and invokes it. The first run compiles Redex from source (~5 min); after that everything is cached.
+
+```sh
+# diff two APKs, scope to a vendor package, write a JSON report
+nix run . -- old.apk new.apk \
+    --auto-package \
+    --threshold 0.8 \
+    -o report.json
+
+# real-world example: vuln/patched pair, Redex-normalized to defeat
+# junk-instruction obfuscation, with the obfuscated-package fallback
+# so Proguard-renamed packages cluster together for comparison
+nix run . -- vuln.apk patched.apk --normalize --find-obfuscated
+```
+
+### Inputs: APK, `.dex`, or dumped DEX (no APK)
+
+Each side accepts an APK, a single `.dex`, or a directory of `.dex` files — so
+you can diff **dumped/extracted DEX** (e.g. pulled from memory or an unpacked
+payload) with no surrounding APK, with no flag: the input kind is detected.
+Partial dumps are tolerated — an unparseable or corrupt `.dex` in a directory is
+skipped (with a warning) rather than aborting the load. A DEX-only input has no
+manifest, so `--auto-package` is unavailable — pass `--app-package <prefix>` for
+app-vs-library ranking instead.
+
+```sh
+# directory of dumped classes*.dex on each side
+twinflame dump_old/ dump_new/ --no-cluster
+
+# write the machine report as CSV instead of the default JSON, only the
+# classes that actually differ
+twinflame a/ b/ --no-cluster --app-package com.target.app -s changed -f csv -o changes.csv
+```
+
+By default, stdout shows a ranked **change summary** (modified / added / removed /
+cosmetic / unchanged, app code first) and any sensitive components both builds
+share, while the full machine report is written to a file (`-f json`, default;
+`csv`/`xml` also available). Pass `-m`/`--matches` for the raw per-class match
+list instead:
+
+```
+[+] com.acme.foo: Login - Login.java | com.acme.foo: Login - SourceFile -> 0.8523
+      ~ validateToken (0.6100)                              # method modified
+      - legacyHash                                          # method removed
+      + verifyNonce                                         # method added
+[-] com.acme.foo: DeletedClass - DeletedClass.java          # in lhs only
+[*] com.acme.foo: AddedClass - SourceFile                   # in rhs only
+```
+
+Each `[+]` paired line is followed by the methods that actually changed, so you
+see *which* method moved the class score. The machine report carries the full
+per-method verdict list, plus each paired class's superclass/interfaces and any
+sensitive component it implements.
+
+Timings go to stderr; the report to stdout (so you can `| less` or redirect freely).
+
+#### Worked example
+
+Diffing a vuln/patched APK pair (a real Android wallet app, ~8 k classes per side, the new build is Proguard'd):
+
+```sh
+$ nix run . -- old.apk new.apk --normalize --find-obfuscated
+
+```
+
+Reading the output:
+
+- The `load:` and `diff:` lines on **stderr** report timings and class counts so you can see what was actually compared after `--normalize` (Redex strips junk instructions on both sides) and post-filtering.
+- `[+]` lines are paired matches with their similarity score (`distance < 1.0`). The two `androidx.activity` examples above pair clear-text class names on the left (lhs preserved source files) against the same logical classes on the right (rhs lost source files to `SourceFile` — Proguard's default). Source-file name was dropped, but the structure + bytecode similarity is high (~0.85–0.95).
+- `[-]` is a class only present in lhs; `[*]` only in rhs. Counts at the top tell you the global picture (e.g. _11 042 matches: 602 paired below 1.0, 5 027 deleted, 5 664 added_ — the bulk being unmatched is the price of heavy obfuscation; tweak `--find-obfuscated` and `--threshold` to trade quality for coverage).
+- Drop the `--normalize` and re-run to see what _uncleaned_ bytecode looks like — if the average paired similarity drops from ~0.9 to ~0.95+ everywhere, that's the fingerprint of junk-instruction obfuscation that Redex defeats.
+
+### Build a standalone binary
+
+```sh
+nix build              # produces ./result/bin/twinflame
+./result/bin/twinflame --help
+```
+
+### Dev shell
+
+For interactive use (REPL, hacking on the code, running tests):
+
+```sh
+nix develop            # enter shell with python + redex + jadx on PATH
+pytest tests/          # 86 tests, ~8s
+python -m twinflame.cli --help
+```
+
+### Python API
+
+```python
+from twinflame import load, filter, diff
+
+lhs_app = load("app-1.6.1.apk")
+rhs_app = load("app-1.6.3.apk")
+
+lhs = filter(lhs_app.classes, {"package_filtering": "com.vendor.app"})
+rhs = filter(rhs_app.classes, {"package_filtering": "com.vendor.app"})
+
+matches = diff(lhs, rhs, 0.8, {
+    "synthetic_skipping": True,
+    "min_inst_size_threshold": 5,
+    "top_match_threshold": 3,
+})
+for m in matches:
+    if m.is_paired and m.distance < 1.0:
+        print(f"[+] {m.lhs.info} | {m.rhs.info} -> {m.distance:.4f}")
+```
+
+### Cross-version deobfuscation map
+
+`--deobfuscation-map OUT` writes a ProGuard-format `mapping.txt` for **apk2** (the obfuscated/stripped build) by propagating class names recovered from the matched classes in **apk1** (the donor build that still carries the DEX `SourceFile` attribute). Load it into JADX/Ghidra and the rename propagates to every reference automatically — twinflame never rewrites the DEX.
+
+```sh
+# apk1 = older build that kept SourceFile; apk2 = the one you want to read
+twinflame app-old.apk app-new.apk \
+    --find-obfuscated --deobfuscation-map app-new.map --map-min-confidence 0.8
+```
+
+Load into JADX with `-Prename-mappings.invert=yes`:
+
+```sh
+jadx --mappings-path app-new.map -Prename-mappings.format=PROGUARD_FILE -Prename-mappings.invert=yes \
+    -d out_deobf app-new.apk
+```
+
+`invert=yes` is required: our file follows the standard ProGuard convention (`<original> -> <obfuscated>:`, what `retrace` consumes), but jadx's `PROGUARD_FILE` reader treats the left side as the name *currently in the binary* — the opposite. Without the flag it silently renames nothing (no error). Same applies in jadx-gui: check "Invert" alongside the mapping path.
+
+- Recovers the class **simple name** (`ContextCompat.java` → `x6.q` becomes `x6.ContextCompat`); the obfuscated **package** and inner-class structure are kept (source files carry no package).
+- Anchored / exact (`distance == 1.0`) matches are trusted; lower-confidence ones are still emitted but flagged with a `# low-confidence` comment. Tune the floor with `--map-min-confidence`.
+- This is the cross-version superpower over single-APK source-file deobfuscation: it names classes in a build that **stripped** `SourceFile`, using the build that didn't. Method/field name propagation is a planned next step.
+
+### Workflow recipes
+
+- **Patch identification.** Scope with `--package PREFIX` or `--auto-package` (drastically shrinks the comparison set), use `--threshold 0.8` since bug fixes are minor changes. Skip `distance == 1.0` matches.
+- **Modded / repackaged app.** Drop `--package` (mods often live in bundled SDKs, not the vendor package). If you see a sea of ~0.95 "everything changed" matches, run with `--normalize` — that's the signature of junk-instruction obfuscation.
+- **Heavily Proguard'd inputs.** Add `--find-obfuscated` so single-letter package names (`a.b.c`) collapse into a fallback pool rather than failing to pair with their clear-text counterparts.
+- **Deobfuscating a stripped build.** Diff it against an older build that kept `SourceFile`, add `--find-obfuscated --deobfuscation-map out.map`, then load `out.map` in JADX.
+
+### Evaluation harness (`eval/`, plan M3.2)
+
+Grades twinflame's own class matches against real ground truth instead of eyeballing them. Build an OSS Android app **twice** — R8 off, then R8 on at full optimization — the `mapping.txt` R8 writes for the R8-on build is a free, perfect oracle for the renaming-only case (directly grades M1.1 + M1.2; inlining/outlining is a future difficulty-graded slice per M3.1).
+
+```sh
+python -m eval.cli app-r8-off.apk app-r8-on.apk app-r8-on/mapping.txt --package com.vendor.app
+```
+
+Prints true/false positive & negative class-match counts plus precision/recall/F1. `eval/mapping.py` parses the ProGuard-format oracle (same format `deobf.py` emits); `eval/score.py` exposes `score_class_matches(matches, ground_truth) -> ScoreResult` for use as a library, independent of the CLI. Kept out of the `src/twinflame` package on purpose — it's tooling to grade the engine, not part of it.
+
+## Layout
+
+| Module                                               | Role                                                                 |
+| ---------------------------------------------------- | -------------------------------------------------------------------- |
+| [src/twinflame/model.py](src/twinflame/model.py)         | Dataclasses: `App`, `Class`, `Method`, `Field`, `Signature`, `Match`, `MethodMatch` |
+| [src/twinflame/loader.py](src/twinflame/loader.py)       | androguard wrapping + multi-DEX merge; captures calls/strings/xrefs  |
+| [src/twinflame/manifest.py](src/twinflame/manifest.py)   | `AndroidManifest.xml` → `ManifestInfo`; dev-package suggestion       |
+| [src/twinflame/normalize.py](src/twinflame/normalize.py) | Redex subprocess wrapper                                             |
+| [src/twinflame/cluster.py](src/twinflame/cluster.py)     | Package-based pools + entropy/length obfuscation fallback            |
+| [src/twinflame/anchor.py](src/twinflame/anchor.py)       | Stage B: string-IDF + framework-call seed matches                    |
+| [src/twinflame/signature.py](src/twinflame/signature.py) | 128-bit SimHash + LSH bucket index                                   |
+| [src/twinflame/accurate.py](src/twinflame/accurate.py)   | Abstract opcodes, method-level + class scoring, greedy 1-to-1 assignment |
+| [src/twinflame/opcodes.py](src/twinflame/opcodes.py)     | Dalvik opcode → 13-category lookup table                             |
+| [src/twinflame/\_hot.py](src/twinflame/_hot.py)          | Hot loops (popcount, Hamming, rapidfuzz-backed Levenshtein)          |
+| [src/twinflame/deobf.py](src/twinflame/deobf.py)         | Cross-version deobfuscation: recovered names → ProGuard mapping.txt   |
+| [src/twinflame/api.py](src/twinflame/api.py)             | Public `load / filter / diff` entry points                           |
+| [src/twinflame/cli.py](src/twinflame/cli.py)             | `twinflame` console script                                             |
+
+## Tests
+
+```sh
+nix develop --command pytest tests/
+```
+
+Currently: **86 tests passing**, 0 skipped. Redex is built and on PATH inside the dev shell.
+
+Test fixtures are **pure-synthetic**: `tests/fixtures/synthetic.py` builds `Class` objects directly via Python, no binaries committed to git. The integration tests exercise the full diff pipeline (cluster → anchor → signature → accurate → api) on these objects.
+
+The on-disk DEX/APK round-trip (loader.py end-to-end) currently has a smoke-only test; a real binary DEX emitter under [tests/fixtures/build_dex.py](tests/fixtures/build_dex.py) is stubbed for future work.
+
+## Known limitations
+
+- **Identical-structure classes collide.** Pools where many classes share signatures (trivial getter/setter classes, generated stubs) may produce false pairings via greedy assignment. `--min-instr 5` filters trivial classes; raising `--neighbors` widens the candidate pool.
+- **No inheritance context.** Matching is per-class (now with method-level detail *inside* a paired class), but ignores first-level parent/child signals (cf. LibPecker); these can be added later without disturbing the core.
+- **No cross-boundary / optimization resilience yet.** Matching assumes a 1:1 class and method correspondence. R8 *optimizations* that break that assumption — inlining, outlining, class merging — are a planned future track (see `twinflame-next-dev-plan.md`, M3.1), not yet implemented.
+- **JNI / native-code changes are invisible.** The diff is purely Dalvik-level — native library mutations require a complementary native-code diff.
+- **LSH is approximate by design.** `--buckets N` tunes the accuracy/speed knob.
+
+## License
+
+MIT.

twinflame-0.1.0b1/pyproject.toml

@@ -0,0 +1,52 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "twinflame"
+version = "0.1.0b1"
+description = "DEX-level Android APK class-diffing engine (SimHash + LSH + abstract-opcode comparison)"
+readme = "README.md"
+requires-python = ">=3.11"
+license = { text = "MIT" }
+authors = [{ name = "twinflame contributors" }]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Topic :: Security",
+    "Topic :: Software Development :: Disassemblers",
+]
+dependencies = [
+    "androguard>=4.1",
+    "numpy>=1.26",
+    "rapidfuzz>=3.14",
+]
+
+[project.optional-dependencies]
+test = [
+    "pytest>=8",
+    "pytest-cov>=5",
+    "hypothesis>=6",
+]
+
+[project.scripts]
+twinflame = "twinflame.cli:main"
+
+[project.urls]
+Homepage = "https://github.com/ankorio/twinflame"
+
+[tool.setuptools]
+package-dir = { "" = "src" }
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+twinflame = ["py.typed"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-ra"
+filterwarnings = [
+    "ignore::DeprecationWarning:androguard.*",
+]

twinflame-0.1.0b1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

twinflame-0.1.0b1/src/twinflame/__init__.py

@@ -0,0 +1,46 @@
+"""twinflame — DEX-level Android APK class-diffing engine."""
+
+from .api import diff, filter, load
+from .model import (
+    App,
+    Class,
+    DiffOptions,
+    Field,
+    Match,
+    Method,
+    MethodMatch,
+    Pool,
+    Signature,
+)
+from .prepare import PreparedRecord, load_record, prepare, save
+
+__all__ = [
+    "App",
+    "Class",
+    "DiffOptions",
+    "Field",
+    "Match",
+    "Method",
+    "MethodMatch",
+    "Pool",
+    "PreparedRecord",
+    "Signature",
+    "diff",
+    "filter",
+    "load",
+    "load_record",
+    "prepare",
+    "save",
+]
+
+# Single source of truth is the [project].version in pyproject.toml; read it
+# back from the installed package metadata so the two never drift. Falls back
+# when running from an uninstalled source tree.
+from importlib.metadata import PackageNotFoundError, version as _pkg_version  # noqa: E402
+
+try:
+    __version__ = _pkg_version("twinflame")
+except PackageNotFoundError:  # source checkout without an install
+    __version__ = "0.0.0+unknown"
+
+del _pkg_version, PackageNotFoundError