traceforge-osint 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. traceforge_osint-1.0.0/LICENSE +21 -0
  2. traceforge_osint-1.0.0/NOTICE +20 -0
  3. traceforge_osint-1.0.0/PKG-INFO +309 -0
  4. traceforge_osint-1.0.0/README.md +258 -0
  5. traceforge_osint-1.0.0/pyproject.toml +85 -0
  6. traceforge_osint-1.0.0/setup.cfg +4 -0
  7. traceforge_osint-1.0.0/tests/test_suite.py +295 -0
  8. traceforge_osint-1.0.0/traceforge/__init__.py +6 -0
  9. traceforge_osint-1.0.0/traceforge/__main__.py +5 -0
  10. traceforge_osint-1.0.0/traceforge/case.py +295 -0
  11. traceforge_osint-1.0.0/traceforge/catalog.py +202 -0
  12. traceforge_osint-1.0.0/traceforge/cli.py +912 -0
  13. traceforge_osint-1.0.0/traceforge/config.py +138 -0
  14. traceforge_osint-1.0.0/traceforge/data/VERSION +1 -0
  15. traceforge_osint-1.0.0/traceforge/data/tools.tsv +153 -0
  16. traceforge_osint-1.0.0/traceforge/exporters.py +429 -0
  17. traceforge_osint-1.0.0/traceforge/modules/__init__.py +3 -0
  18. traceforge_osint-1.0.0/traceforge/modules/documents.py +69 -0
  19. traceforge_osint-1.0.0/traceforge/modules/domain.py +65 -0
  20. traceforge_osint-1.0.0/traceforge/modules/email.py +66 -0
  21. traceforge_osint-1.0.0/traceforge/modules/identity.py +64 -0
  22. traceforge_osint-1.0.0/traceforge/modules/image.py +74 -0
  23. traceforge_osint-1.0.0/traceforge/modules/network.py +60 -0
  24. traceforge_osint-1.0.0/traceforge/modules/opsec.py +56 -0
  25. traceforge_osint-1.0.0/traceforge/platform_detect.py +289 -0
  26. traceforge_osint-1.0.0/traceforge/runners.py +282 -0
  27. traceforge_osint-1.0.0/traceforge/tools.py +760 -0
  28. traceforge_osint-1.0.0/traceforge_osint.egg-info/PKG-INFO +309 -0
  29. traceforge_osint-1.0.0/traceforge_osint.egg-info/SOURCES.txt +31 -0
  30. traceforge_osint-1.0.0/traceforge_osint.egg-info/dependency_links.txt +1 -0
  31. traceforge_osint-1.0.0/traceforge_osint.egg-info/entry_points.txt +2 -0
  32. traceforge_osint-1.0.0/traceforge_osint.egg-info/requires.txt +23 -0
  33. traceforge_osint-1.0.0/traceforge_osint.egg-info/top_level.txt +1 -0
traceforge_osint-1.0.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Aman Kumar Pandey
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
traceforge_osint-1.0.0/NOTICE ADDED
@@ -0,0 +1,20 @@
1
+ TraceForge
2
+ Copyright (c) 2026 Aman Kumar Pandey
3
+
4
+ Lead Architect & Core Developer: Aman Kumar Pandey
5
+ Contributors & Credits: Thanks to the Global Open-Source Security Community
6
+
7
+ This project is an open-source command-line toolkit for OSINT, DFIR and security
8
+ investigation, evidence handling, and first-party investigation tools.
9
+
10
+ -------------------------------------------------------------------------------
11
+ Third-Party Software & Tools Attribution Notice
12
+ -------------------------------------------------------------------------------
13
+
14
+ TraceForge installs and launches external command-line utilities.
15
+ It does not vendor or redistribute third-party tool binaries. All third-party
16
+ utilities installed or invoked through this suite remain the property of their
17
+ respective authors and are governed by their respective open-source licenses.
18
+
19
+ For a detailed index of cataloged tools, upstream repository links, and
20
+ licenses, refer to THIRD_PARTY_NOTICES.md and catalog/TOOLS.md.
traceforge_osint-1.0.0/PKG-INFO ADDED
@@ -0,0 +1,309 @@
1
+ Metadata-Version: 2.4
2
+ Name: traceforge-osint
3
+ Version: 1.0.0
4
+ Summary: TraceForge — Open-Source Intelligence & Digital Forensics Platform
5
+ Author-email: Aman Kumar Pandey <paman7647@proton.me>
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/paman7647/TraceForge
8
+ Project-URL: Documentation, https://traceforge.readthedocs.io
9
+ Project-URL: Repository, https://github.com/paman7647/TraceForge.git
10
+ Project-URL: Issues, https://github.com/paman7647/TraceForge/issues
11
+ Project-URL: Changelog, https://github.com/paman7647/TraceForge/blob/master/CHANGELOG.md
12
+ Keywords: osint,dfir,digital-forensics,incident-response,security,threat-intelligence,pcap,stix,misp,ioc-extractor
13
+ Classifier: Development Status :: 5 - Production/Stable
14
+ Classifier: Environment :: Console
15
+ Classifier: Intended Audience :: Information Technology
16
+ Classifier: Intended Audience :: System Administrators
17
+ Classifier: License :: OSI Approved :: MIT License
18
+ Classifier: Operating System :: MacOS
19
+ Classifier: Operating System :: POSIX :: Linux
20
+ Classifier: Programming Language :: Python :: 3
21
+ Classifier: Programming Language :: Python :: 3.9
22
+ Classifier: Programming Language :: Python :: 3.10
23
+ Classifier: Programming Language :: Python :: 3.11
24
+ Classifier: Programming Language :: Python :: 3.12
25
+ Classifier: Topic :: Security
26
+ Classifier: Topic :: System :: Networking
27
+ Requires-Python: >=3.9
28
+ Description-Content-Type: text/markdown
29
+ License-File: LICENSE
30
+ License-File: NOTICE
31
+ Provides-Extra: reporting
32
+ Requires-Dist: openpyxl>=3.1.0; extra == "reporting"
33
+ Requires-Dist: python-docx>=1.0.0; extra == "reporting"
34
+ Provides-Extra: docs
35
+ Requires-Dist: sphinx>=7.0.0; extra == "docs"
36
+ Requires-Dist: myst-parser>=3.0.0; extra == "docs"
37
+ Requires-Dist: sphinx-rtd-theme>=2.0.0; extra == "docs"
38
+ Provides-Extra: dev
39
+ Requires-Dist: build>=1.0.0; extra == "dev"
40
+ Requires-Dist: twine>=5.0.0; extra == "dev"
41
+ Requires-Dist: pytest>=7.0.0; extra == "dev"
42
+ Provides-Extra: all
43
+ Requires-Dist: openpyxl>=3.1.0; extra == "all"
44
+ Requires-Dist: python-docx>=1.0.0; extra == "all"
45
+ Requires-Dist: sphinx>=7.0.0; extra == "all"
46
+ Requires-Dist: myst-parser>=3.0.0; extra == "all"
47
+ Requires-Dist: sphinx-rtd-theme>=2.0.0; extra == "all"
48
+ Requires-Dist: build>=1.0.0; extra == "all"
49
+ Requires-Dist: twine>=5.0.0; extra == "all"
50
+ Dynamic: license-file
51
+
52
+ # TraceForge
53
+
54
+ **Open-source OSINT, DFIR and security investigation toolkit.**
55
+
56
+ [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
57
+ [![Platform](https://img.shields.io/badge/Platform-macOS%20%7C%20Linux%20(Debian%2C%20Ubuntu%2C%20Kali)-informational.svg)](docs/INSTALLATION.md)
58
+ [![Shell](https://img.shields.io/badge/Shell-Bash%203.2%2B-4EAA25.svg)](https://www.gnu.org/software/bash/)
59
+ [![Catalog](https://img.shields.io/badge/Catalog-152%20Tools-blueviolet.svg)](catalog/TOOLS.md)
60
+ [![Python](https://img.shields.io/badge/Python-%3E%3D3.9-3776AB.svg)](pyproject.toml)
61
+ [![CI](https://github.com/paman7647/TraceForge/actions/workflows/ci.yml/badge.svg)](.github/workflows/ci.yml)
62
+ [![Version](https://img.shields.io/badge/Version-1.0.0-green.svg)](VERSION)
63
+
64
+ ---
65
+
66
+ ## What It Does
67
+
68
+ TraceForge is an open-source command-line toolkit for OSINT, digital forensics, security investigation, evidence handling, and first-party investigation tools:
69
+
70
+ * **First-Party Native Tools (`traceforge` / `go/`)**: High-performance Go-native and Python utilities providing asset relationship graphing, snapshot diffing, streaming IOC extraction/defanging, forensic evidence indexing, UTC timeline normalization, PCAP network flow analysis, automated log triage, and filesystem baselining (see [docs/FIRST_PARTY_TOOLS.md](docs/FIRST_PARTY_TOOLS.md)).
71
+ * **Tool Catalog & Installer**: A searchable catalog of 152 tools and an automated installer that handles dependencies across Homebrew (macOS), APT (Linux), `pipx` (Python virtual environments), Go, Cargo, and RubyGems.
72
+ * **Investigation Modules**: 7 built-in analysis scripts covering image metadata, network PCAPs, usernames, email breach records, domain DNS records, document metadata, and local OPSEC hygiene.
73
+ * **Case Management**: Workspaces stored under `workspace/` with unique case IDs, SHA-256 evidence hashing, and append-only chain-of-custody logging.
74
+ * **Export Pipeline**: Generates reports in Markdown, standalone dark-mode HTML, CSV (with formula injection protection), TSV, JSON, JSONL streams, STIX 2.1, MISP, GeoJSON, KML, Excel XLSX, Word DOCX, and signed ZIP packages.
75
+ * **Redaction**: Built-in `--redact` flag masks sensitive IP addresses and email addresses in generated reports.
76
+
77
+ ---
78
+
79
+ ## Architecture
80
+
81
+ TraceForge does not ship binary copies of third-party tools inside the repository. It installs and launches external tools via your system's package managers:
82
+
83
+ ```text
84
+ +-------------------------------------------------------------------+
85
+ | TraceForge |
86
+ | (CLI Menu, Tool Catalog, Case Workspaces, Report Exporter) |
87
+ | [ MIT License ] |
88
+ +---------------------------------+---------------------------------+
89
+ |
90
+ | Installs / Launches / Collects Output
91
+ v
92
+ +-------------------------------------------------------------------+
93
+ | Third-Party Command-Line Tools |
94
+ | (ExifTool, TShark, Binwalk, Nmap, Sherlock, Maigret, etc.) |
95
+ | [ Retain Respective Upstream Copyright & Licenses ] |
96
+ +-------------------------------------------------------------------+
97
+ ```
98
+
99
+ All third-party tools remain the property of their respective creators and are licensed under their own open-source terms. See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for details.
100
+
101
+ ---
102
+
103
+ ## Supported Systems
104
+
105
+ | Platform | Tier | Architecture | Package Manager | Guide & Notes |
106
+ |---|---|---|---|---|
107
+ | **macOS Apple Silicon** | Supported | `arm64` (M1/M2/M3/M4) | Homebrew | Full native support |
108
+ | **macOS Intel** | Supported | `x86_64` | Homebrew | Full native support |
109
+ | **Debian / Ubuntu** | Supported | `x86_64`, `arm64` | APT | Full native support |
110
+ | **Kali Linux** | Supported | `x86_64`, `arm64` | APT | Full native support |
111
+ | **Termux / Android** | Supported with limitations | `arm64`, `armv7`, `x86_64` | `pkg` | Non-root userland. Offline forensics & PCAP analysis. [Termux Guide](docs/platforms/termux.md) |
112
+
113
+ ---
114
+
115
+ ## Installation & Quick Start
116
+
117
+ ### 1. Clone the Repository
118
+ ```bash
119
+ git clone https://github.com/paman7647/TraceForge.git
120
+ cd TraceForge
121
+ chmod +x install_all.sh main.sh modules/*.sh scripts/*.sh tests/*.sh
122
+ ```
123
+
124
+ ### 2. Run the Installer
125
+ ```bash
126
+ # Preview packages without installing (dry-run)
127
+ ./install_all.sh --profile recommended --dry-run
128
+
129
+ # Core Profile: Minimal tools for the 7 modules (~15 packages)
130
+ ./install_all.sh --profile core
131
+
132
+ # Recommended Profile (Default): Standard investigative toolset (~50 packages)
133
+ ./install_all.sh --profile recommended
134
+
135
+ # Full Profile: All 152 installable catalog tools
136
+ ./install_all.sh --profile full
137
+ ```
138
+
139
+ ### 3. Run TraceForge
140
+ ```bash
141
+ # Interactive TTY Menu
142
+ ./main.sh
143
+ # or via Python CLI
144
+ python3 -m traceforge
145
+
146
+ # Run system diagnostics
147
+ ./main.sh doctor
148
+
149
+ # List existing cases
150
+ ./main.sh list-cases
151
+
152
+ # Search the catalog
153
+ ./main.sh search "steganography"
154
+
155
+ # Run first-party tools
156
+ traceforge ioc extract /path/to/intel.txt
157
+ traceforge asset graph /path/to/entities.jsonl --html graph.html
158
+
159
+ # Run a module directly
160
+ ./main.sh module 1 /path/to/evidence.png CASE-20260825-ABC123
161
+
162
+ # Export a case
163
+ ./main.sh export CASE-20260825-ABC123 --all --package zip
164
+ ```
165
+
166
+ ---
167
+
168
+ ## Investigation Modules
169
+
170
+ | Module | Script | Target & Tools Used |
171
+ |---|---|---|
172
+ | **01. Media & Image Forensics** | `modules/01_image_forensics.sh` | Images and videos: EXIF/IPTC metadata, GPS coordinates, strings, steganography (`exiftool`, `binwalk`, `zsteg`, `xxd`). |
173
+ | **02. Network & PCAP Forensics** | `modules/02_network_recon.sh` | Packet captures: DNS queries, HTTP URIs, TLS SNI headers, conversations (`tshark`, `capinfos`, `aircrack-ng`). |
174
+ | **03. Identity & Social Research** | `modules/03_identity_social.sh` | Usernames and aliases: account discovery across web platforms (`sherlock`, `maigret`, `blackbird`, `socialscan`). |
175
+ | **04. Email & Breach Intelligence** | `modules/04_email_breach.sh` | Email addresses: account registrations, breach dumps, SPF/DMARC posture (`holehe`, `h8mail`, `emailrep`, `theHarvester`, `checkdmarc`). |
176
+ | **05. Domain & DNS Intelligence** | `modules/05_domain_dns.sh` | Domains: DNS records, passive subdomains, HTTP probing, typosquats (`dig`, `whois`, `subfinder`, `amass`, `dnsx`, `httpx`, `dnstwist`). |
177
+ | **06. Document & Metadata Harvesting** | `modules/06_document_harvesting.sh` | Documents (PDF, DOCX, XLSX): properties, text, embedded objects, macros, secret keys (`poppler`, `oletools`, `exiftool`, `qpdf`, `ripgrep`). |
178
+ | **07. OPSEC & Environment Audit** | `modules/07_opsec_anonymization.sh` | Local audit: public IP detection, DNS leak test, Tor/proxy status, encryption tools (`mat2`, `tor`, `proxychains`, `gnupg`, `age`). |
179
+
180
+ ---
181
+
182
+ ## Directory Structure
183
+
184
+ ```text
185
+ TraceForge/
186
+ ├── main.sh # Interactive console and CLI dispatcher
187
+ ├── install_all.sh # Multi-ecosystem package installer
188
+ ├── pyproject.toml # Modern Python package configuration
189
+ ├── go.mod # Go module declaration (zero external dependencies)
190
+ ├── VERSION # Project version (1.0.0)
191
+ ├── LICENSE # MIT License
192
+ ├── NOTICE # Copyright and attribution
193
+ ├── THIRD_PARTY_NOTICES.md # Upstream tool licenses and repository links
194
+ ├── README.md # Main project documentation
195
+ ├── SECURITY.md # Vulnerability reporting guidelines
196
+ ├── CONTRIBUTING.md # Contribution guidelines
197
+ ├── CODE_OF_CONDUCT.md # Contributor Covenant 2.1
198
+ ├── CHANGELOG.md # Version release history
199
+ ├── RELEASE_NOTES.md # 1.0.0 release notes
200
+ │
201
+ ├── go/ # Go-native high-performance engine
202
+ ├── traceforge/ # Pure Python implementation and package
203
+ │
204
+ ├── catalog/
205
+ │ ├── README.md # Catalog schema specification
206
+ │ ├── tools.tsv # 15-column registry of 152 tools
207
+ │ └── TOOLS.md # Formatted markdown tool index
208
+ │
209
+ ├── lib/
210
+ │ ├── common.sh # Terminal UI, colors, logging, string helpers
211
+ │ ├── platform.sh # OS and architecture detection
212
+ │ ├── packages.sh # Package managers (Homebrew, APT, pipx, Go, Gem, Cargo)
213
+ │ ├── catalog.sh # Catalog search and filter functions
214
+ │ ├── case.sh # Case lifecycle, evidence, findings, IOCs, timeline
215
+ │ ├── export.sh # Export coordinator, packaging, and hashing
216
+ │ └── report.sh # Markdown, HTML, and PDF report generators
217
+ │
218
+ ├── modules/ # 7 investigation modules (01 to 07)
219
+ ├── scripts/
220
+ │ ├── doctor.sh # System and dependency diagnostic checker
221
+ │ ├── export_case.sh # CLI export wrapper
222
+ │ ├── export_engine.py # Multi-format export engine
223
+ │ ├── install_tool.sh # Single tool installer
224
+ │ ├── generate_catalog_docs.sh # Markdown documentation generator
225
+ │ └── release_check.sh # Pre-flight release verification test
226
+ │
227
+ ├── docs/ # In-depth operator, architecture, and OPSEC guides
228
+ ├── tests/ # Automated test suite and regression tests
229
+ └── workspace/ # Local investigation cases
230
+ └── .gitkeep
231
+ ```
232
+
233
+ ---
234
+
235
+ ## Development & Branching Workflow
236
+
237
+ TraceForge maintains two long-lived branches:
238
+ * **`master`**: Stable, production-ready public release branch.
239
+ * **`beta`**: Active integration branch for new features, bug fixes, platform improvements, and release candidates.
240
+
241
+ ```text
242
+ master (Stable Releases)
243
+ ▲
244
+ │ Release PR (Validated & Tagged)
245
+ │
246
+ beta (Active Development & RC)
247
+ ▲ ▲ ▲
248
+ │ │ │ Pull Requests
249
+ feature fix docs (Short-Lived Working Branches)
250
+ ```
251
+
252
+ Pull requests and contributions must target the **`beta`** branch. See [docs/BRANCHING.md](docs/BRANCHING.md) for the branching model, [docs/BUG_REPORTING.md](docs/BUG_REPORTING.md) for issue guidelines, and [CONTRIBUTING.md](CONTRIBUTING.md) for pull request workflows.
253
+
254
+ ---
255
+
256
+ ## Automated Testing
257
+
258
+ Run the test suite before submitting pull requests or publishing releases:
259
+
260
+ ```bash
261
+ # Run Python & Platform test suites
262
+ python3 -m unittest discover -s tests
263
+
264
+ # Run master shell & integration test harness
265
+ ./tests/test.sh
266
+
267
+ # Run pre-flight release check
268
+ ./scripts/release_check.sh beta
269
+ ```
270
+
271
+ All shell scripts are validated with `bash -n` and `shellcheck -x`.
272
+
273
+ ---
274
+
275
+ ## Responsible Use
276
+
277
+ TraceForge is intended for lawful OSINT, digital forensics, incident response, security research, authorized testing, education, and lab work.
278
+
279
+ You are responsible for ensuring that your use of the software is authorized and complies with applicable law, regulation, contracts, platform terms, privacy requirements, and organizational policy.
280
+
281
+ TraceForge does not grant permission to access any system, account, network, service, or data.
282
+
283
+ See [DISCLAIMER.md](DISCLAIMER.md) and [RESPONSIBLE_USE.md](RESPONSIBLE_USE.md) for details.
284
+
285
+ ---
286
+
287
+ ## Documentation & Policies
288
+
289
+ * [docs/BRANCHING.md](docs/BRANCHING.md) — Two-branch Git model, release lifecycle, and PR rules.
290
+ * [docs/BUG_REPORTING.md](docs/BUG_REPORTING.md) — Bug reporting protocol, reproduction requirements, and fix lifecycle.
291
+ * [docs/RUNTIME_PROFILES.md](docs/RUNTIME_PROFILES.md) — Architecture and configuration of the 6 runtime profiles.
292
+ * [docs/FIRST_PARTY_TOOLS.md](docs/FIRST_PARTY_TOOLS.md) — Documentation for first-party analytical tools.
293
+ * [docs/platforms/termux.md](docs/platforms/termux.md) — Termux and Android platform guide and storage configuration.
294
+ * [DISCLAIMER.md](DISCLAIMER.md) — Legal disclaimer, liability boundaries, accuracy notices, and jurisdictional advice.
295
+ * [RESPONSIBLE_USE.md](RESPONSIBLE_USE.md) — Responsible use standards, authorized scope, and prohibited activities.
296
+ * [PRIVACY.md](PRIVACY.md) — Privacy principles, local storage layout, data minimization, and redaction.
297
+ * [SECURITY.md](SECURITY.md) — Vulnerability reporting policy and defensive engineering standards.
298
+ * [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) — Upstream tool licenses, attribution, and license boundaries.
299
+ * [CONTRIBUTING.md](CONTRIBUTING.md) — Contribution workflow, code formatting, and review guidelines.
300
+ * [LICENSE](LICENSE) — Official MIT License terms and conditions.
301
+
302
+ ---
303
+
304
+ ## Authors & Credits
305
+
306
+ * **Lead Architect & Core Developer**: Aman Kumar Pandey
307
+ * **Credits**: Thanks to the Global Open-Source Security Community
308
+ * **License**: [MIT License](LICENSE)
309
+ * **Copyright**: Copyright (c) 2026 Aman Kumar Pandey
traceforge_osint-1.0.0/README.md ADDED
@@ -0,0 +1,258 @@
1
+ # TraceForge
2
+
3
+ **Open-source OSINT, DFIR and security investigation toolkit.**
4
+
5
+ [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
6
+ [![Platform](https://img.shields.io/badge/Platform-macOS%20%7C%20Linux%20(Debian%2C%20Ubuntu%2C%20Kali)-informational.svg)](docs/INSTALLATION.md)
7
+ [![Shell](https://img.shields.io/badge/Shell-Bash%203.2%2B-4EAA25.svg)](https://www.gnu.org/software/bash/)
8
+ [![Catalog](https://img.shields.io/badge/Catalog-152%20Tools-blueviolet.svg)](catalog/TOOLS.md)
9
+ [![Python](https://img.shields.io/badge/Python-%3E%3D3.9-3776AB.svg)](pyproject.toml)
10
+ [![CI](https://github.com/paman7647/TraceForge/actions/workflows/ci.yml/badge.svg)](.github/workflows/ci.yml)
11
+ [![Version](https://img.shields.io/badge/Version-1.0.0-green.svg)](VERSION)
12
+
13
+ ---
14
+
15
+ ## What It Does
16
+
17
+ TraceForge is an open-source command-line toolkit for OSINT, digital forensics, security investigation, evidence handling, and first-party investigation tools:
18
+
19
+ * **First-Party Native Tools (`traceforge` / `go/`)**: High-performance Go-native and Python utilities providing asset relationship graphing, snapshot diffing, streaming IOC extraction/defanging, forensic evidence indexing, UTC timeline normalization, PCAP network flow analysis, automated log triage, and filesystem baselining (see [docs/FIRST_PARTY_TOOLS.md](docs/FIRST_PARTY_TOOLS.md)).
20
+ * **Tool Catalog & Installer**: A searchable catalog of 152 tools and an automated installer that handles dependencies across Homebrew (macOS), APT (Linux), `pipx` (Python virtual environments), Go, Cargo, and RubyGems.
21
+ * **Investigation Modules**: 7 built-in analysis scripts covering image metadata, network PCAPs, usernames, email breach records, domain DNS records, document metadata, and local OPSEC hygiene.
22
+ * **Case Management**: Workspaces stored under `workspace/` with unique case IDs, SHA-256 evidence hashing, and append-only chain-of-custody logging.
23
+ * **Export Pipeline**: Generates reports in Markdown, standalone dark-mode HTML, CSV (with formula injection protection), TSV, JSON, JSONL streams, STIX 2.1, MISP, GeoJSON, KML, Excel XLSX, Word DOCX, and signed ZIP packages.
24
+ * **Redaction**: Built-in `--redact` flag masks sensitive IP addresses and email addresses in generated reports.
25
+
26
+ ---
27
+
28
+ ## Architecture
29
+
30
+ TraceForge does not ship binary copies of third-party tools inside the repository. It installs and launches external tools via your system's package managers:
31
+
32
+ ```text
33
+ +-------------------------------------------------------------------+
34
+ | TraceForge |
35
+ | (CLI Menu, Tool Catalog, Case Workspaces, Report Exporter) |
36
+ | [ MIT License ] |
37
+ +---------------------------------+---------------------------------+
38
+ |
39
+ | Installs / Launches / Collects Output
40
+ v
41
+ +-------------------------------------------------------------------+
42
+ | Third-Party Command-Line Tools |
43
+ | (ExifTool, TShark, Binwalk, Nmap, Sherlock, Maigret, etc.) |
44
+ | [ Retain Respective Upstream Copyright & Licenses ] |
45
+ +-------------------------------------------------------------------+
46
+ ```
47
+
48
+ All third-party tools remain the property of their respective creators and are licensed under their own open-source terms. See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for details.
49
+
50
+ ---
51
+
52
+ ## Supported Systems
53
+
54
+ | Platform | Tier | Architecture | Package Manager | Guide & Notes |
55
+ |---|---|---|---|---|
56
+ | **macOS Apple Silicon** | Supported | `arm64` (M1/M2/M3/M4) | Homebrew | Full native support |
57
+ | **macOS Intel** | Supported | `x86_64` | Homebrew | Full native support |
58
+ | **Debian / Ubuntu** | Supported | `x86_64`, `arm64` | APT | Full native support |
59
+ | **Kali Linux** | Supported | `x86_64`, `arm64` | APT | Full native support |
60
+ | **Termux / Android** | Supported with limitations | `arm64`, `armv7`, `x86_64` | `pkg` | Non-root userland. Offline forensics & PCAP analysis. [Termux Guide](docs/platforms/termux.md) |
61
+
62
+ ---
63
+
64
+ ## Installation & Quick Start
65
+
66
+ ### 1. Clone the Repository
67
+ ```bash
68
+ git clone https://github.com/paman7647/TraceForge.git
69
+ cd TraceForge
70
+ chmod +x install_all.sh main.sh modules/*.sh scripts/*.sh tests/*.sh
71
+ ```
72
+
73
+ ### 2. Run the Installer
74
+ ```bash
75
+ # Preview packages without installing (dry-run)
76
+ ./install_all.sh --profile recommended --dry-run
77
+
78
+ # Core Profile: Minimal tools for the 7 modules (~15 packages)
79
+ ./install_all.sh --profile core
80
+
81
+ # Recommended Profile (Default): Standard investigative toolset (~50 packages)
82
+ ./install_all.sh --profile recommended
83
+
84
+ # Full Profile: All 152 installable catalog tools
85
+ ./install_all.sh --profile full
86
+ ```
87
+
88
+ ### 3. Run TraceForge
89
+ ```bash
90
+ # Interactive TTY Menu
91
+ ./main.sh
92
+ # or via Python CLI
93
+ python3 -m traceforge
94
+
95
+ # Run system diagnostics
96
+ ./main.sh doctor
97
+
98
+ # List existing cases
99
+ ./main.sh list-cases
100
+
101
+ # Search the catalog
102
+ ./main.sh search "steganography"
103
+
104
+ # Run first-party tools
105
+ traceforge ioc extract /path/to/intel.txt
106
+ traceforge asset graph /path/to/entities.jsonl --html graph.html
107
+
108
+ # Run a module directly
109
+ ./main.sh module 1 /path/to/evidence.png CASE-20260825-ABC123
110
+
111
+ # Export a case
112
+ ./main.sh export CASE-20260825-ABC123 --all --package zip
113
+ ```
114
+
115
+ ---
116
+
117
+ ## Investigation Modules
118
+
119
+ | Module | Script | Target & Tools Used |
120
+ |---|---|---|
121
+ | **01. Media & Image Forensics** | `modules/01_image_forensics.sh` | Images and videos: EXIF/IPTC metadata, GPS coordinates, strings, steganography (`exiftool`, `binwalk`, `zsteg`, `xxd`). |
122
+ | **02. Network & PCAP Forensics** | `modules/02_network_recon.sh` | Packet captures: DNS queries, HTTP URIs, TLS SNI headers, conversations (`tshark`, `capinfos`, `aircrack-ng`). |
123
+ | **03. Identity & Social Research** | `modules/03_identity_social.sh` | Usernames and aliases: account discovery across web platforms (`sherlock`, `maigret`, `blackbird`, `socialscan`). |
124
+ | **04. Email & Breach Intelligence** | `modules/04_email_breach.sh` | Email addresses: account registrations, breach dumps, SPF/DMARC posture (`holehe`, `h8mail`, `emailrep`, `theHarvester`, `checkdmarc`). |
125
+ | **05. Domain & DNS Intelligence** | `modules/05_domain_dns.sh` | Domains: DNS records, passive subdomains, HTTP probing, typosquats (`dig`, `whois`, `subfinder`, `amass`, `dnsx`, `httpx`, `dnstwist`). |
126
+ | **06. Document & Metadata Harvesting** | `modules/06_document_harvesting.sh` | Documents (PDF, DOCX, XLSX): properties, text, embedded objects, macros, secret keys (`poppler`, `oletools`, `exiftool`, `qpdf`, `ripgrep`). |
127
+ | **07. OPSEC & Environment Audit** | `modules/07_opsec_anonymization.sh` | Local audit: public IP detection, DNS leak test, Tor/proxy status, encryption tools (`mat2`, `tor`, `proxychains`, `gnupg`, `age`). |
128
+
129
+ ---
130
+
131
+ ## Directory Structure
132
+
133
+ ```text
134
+ TraceForge/
135
+ ├── main.sh # Interactive console and CLI dispatcher
136
+ ├── install_all.sh # Multi-ecosystem package installer
137
+ ├── pyproject.toml # Modern Python package configuration
138
+ ├── go.mod # Go module declaration (zero external dependencies)
139
+ ├── VERSION # Project version (1.0.0)
140
+ ├── LICENSE # MIT License
141
+ ├── NOTICE # Copyright and attribution
142
+ ├── THIRD_PARTY_NOTICES.md # Upstream tool licenses and repository links
143
+ ├── README.md # Main project documentation
144
+ ├── SECURITY.md # Vulnerability reporting guidelines
145
+ ├── CONTRIBUTING.md # Contribution guidelines
146
+ ├── CODE_OF_CONDUCT.md # Contributor Covenant 2.1
147
+ ├── CHANGELOG.md # Version release history
148
+ ├── RELEASE_NOTES.md # 1.0.0 release notes
149
+ │
150
+ ├── go/ # Go-native high-performance engine
151
+ ├── traceforge/ # Pure Python implementation and package
152
+ │
153
+ ├── catalog/
154
+ │ ├── README.md # Catalog schema specification
155
+ │ ├── tools.tsv # 15-column registry of 152 tools
156
+ │ └── TOOLS.md # Formatted markdown tool index
157
+ │
158
+ ├── lib/
159
+ │ ├── common.sh # Terminal UI, colors, logging, string helpers
160
+ │ ├── platform.sh # OS and architecture detection
161
+ │ ├── packages.sh # Package managers (Homebrew, APT, pipx, Go, Gem, Cargo)
162
+ │ ├── catalog.sh # Catalog search and filter functions
163
+ │ ├── case.sh # Case lifecycle, evidence, findings, IOCs, timeline
164
+ │ ├── export.sh # Export coordinator, packaging, and hashing
165
+ │ └── report.sh # Markdown, HTML, and PDF report generators
166
+ │
167
+ ├── modules/ # 7 investigation modules (01 to 07)
168
+ ├── scripts/
169
+ │ ├── doctor.sh # System and dependency diagnostic checker
170
+ │ ├── export_case.sh # CLI export wrapper
171
+ │ ├── export_engine.py # Multi-format export engine
172
+ │ ├── install_tool.sh # Single tool installer
173
+ │ ├── generate_catalog_docs.sh # Markdown documentation generator
174
+ │ └── release_check.sh # Pre-flight release verification test
175
+ │
176
+ ├── docs/ # In-depth operator, architecture, and OPSEC guides
177
+ ├── tests/ # Automated test suite and regression tests
178
+ └── workspace/ # Local investigation cases
179
+ └── .gitkeep
180
+ ```
181
+
182
+ ---
183
+
184
+ ## Development & Branching Workflow
185
+
186
+ TraceForge maintains two long-lived branches:
187
+ * **`master`**: Stable, production-ready public release branch.
188
+ * **`beta`**: Active integration branch for new features, bug fixes, platform improvements, and release candidates.
189
+
190
+ ```text
191
+ master (Stable Releases)
192
+ ▲
193
+ │ Release PR (Validated & Tagged)
194
+ │
195
+ beta (Active Development & RC)
196
+ ▲ ▲ ▲
197
+ │ │ │ Pull Requests
198
+ feature fix docs (Short-Lived Working Branches)
199
+ ```
200
+
201
+ Pull requests and contributions must target the **`beta`** branch. See [docs/BRANCHING.md](docs/BRANCHING.md) for the branching model, [docs/BUG_REPORTING.md](docs/BUG_REPORTING.md) for issue guidelines, and [CONTRIBUTING.md](CONTRIBUTING.md) for pull request workflows.
202
+
203
+ ---
204
+
205
+ ## Automated Testing
206
+
207
+ Run the test suite before submitting pull requests or publishing releases:
208
+
209
+ ```bash
210
+ # Run Python & Platform test suites
211
+ python3 -m unittest discover -s tests
212
+
213
+ # Run master shell & integration test harness
214
+ ./tests/test.sh
215
+
216
+ # Run pre-flight release check
217
+ ./scripts/release_check.sh beta
218
+ ```
219
+
220
+ All shell scripts are validated with `bash -n` and `shellcheck -x`.
221
+
222
+ ---
223
+
224
+ ## Responsible Use
225
+
226
+ TraceForge is intended for lawful OSINT, digital forensics, incident response, security research, authorized testing, education, and lab work.
227
+
228
+ You are responsible for ensuring that your use of the software is authorized and complies with applicable law, regulation, contracts, platform terms, privacy requirements, and organizational policy.
229
+
230
+ TraceForge does not grant permission to access any system, account, network, service, or data.
231
+
232
+ See [DISCLAIMER.md](DISCLAIMER.md) and [RESPONSIBLE_USE.md](RESPONSIBLE_USE.md) for details.
233
+
234
+ ---
235
+
236
+ ## Documentation & Policies
237
+
238
+ * [docs/BRANCHING.md](docs/BRANCHING.md) — Two-branch Git model, release lifecycle, and PR rules.
239
+ * [docs/BUG_REPORTING.md](docs/BUG_REPORTING.md) — Bug reporting protocol, reproduction requirements, and fix lifecycle.
240
+ * [docs/RUNTIME_PROFILES.md](docs/RUNTIME_PROFILES.md) — Architecture and configuration of the 6 runtime profiles.
241
+ * [docs/FIRST_PARTY_TOOLS.md](docs/FIRST_PARTY_TOOLS.md) — Documentation for first-party analytical tools.
242
+ * [docs/platforms/termux.md](docs/platforms/termux.md) — Termux and Android platform guide and storage configuration.
243
+ * [DISCLAIMER.md](DISCLAIMER.md) — Legal disclaimer, liability boundaries, accuracy notices, and jurisdictional advice.
244
+ * [RESPONSIBLE_USE.md](RESPONSIBLE_USE.md) — Responsible use standards, authorized scope, and prohibited activities.
245
+ * [PRIVACY.md](PRIVACY.md) — Privacy principles, local storage layout, data minimization, and redaction.
246
+ * [SECURITY.md](SECURITY.md) — Vulnerability reporting policy and defensive engineering standards.
247
+ * [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) — Upstream tool licenses, attribution, and license boundaries.
248
+ * [CONTRIBUTING.md](CONTRIBUTING.md) — Contribution workflow, code formatting, and review guidelines.
249
+ * [LICENSE](LICENSE) — Official MIT License terms and conditions.
250
+
251
+ ---
252
+
253
+ ## Authors & Credits
254
+
255
+ * **Lead Architect & Core Developer**: Aman Kumar Pandey
256
+ * **Credits**: Thanks to the Global Open-Source Security Community
257
+ * **License**: [MIT License](LICENSE)
258
+ * **Copyright**: Copyright (c) 2026 Aman Kumar Pandey