watchpost-cli 0.0.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- watchpost_cli-0.0.1/CHANGELOG.md +37 -0
- watchpost_cli-0.0.1/LICENSE +21 -0
- watchpost_cli-0.0.1/MANIFEST.in +13 -0
- watchpost_cli-0.0.1/PKG-INFO +218 -0
- watchpost_cli-0.0.1/README.md +204 -0
- watchpost_cli-0.0.1/docs/brief-v0.1.md +74 -0
- watchpost_cli-0.0.1/docs/compare-milestone.md +57 -0
- watchpost_cli-0.0.1/docs/publishing.md +107 -0
- watchpost_cli-0.0.1/docs/release-v0.0.1.md +138 -0
- watchpost_cli-0.0.1/docs/reporting-milestone.md +54 -0
- watchpost_cli-0.0.1/docs/scenarios.md +208 -0
- watchpost_cli-0.0.1/examples/after.xml +28 -0
- watchpost_cli-0.0.1/examples/baseline.xml +28 -0
- watchpost_cli-0.0.1/examples/incomplete.xml +22 -0
- watchpost_cli-0.0.1/examples/report.md +92 -0
- watchpost_cli-0.0.1/pyproject.toml +34 -0
- watchpost_cli-0.0.1/setup.cfg +4 -0
- watchpost_cli-0.0.1/tests/test_compare.py +361 -0
- watchpost_cli-0.0.1/tests/test_packaging.py +27 -0
- watchpost_cli-0.0.1/tests/test_reporting.py +397 -0
- watchpost_cli-0.0.1/tests/test_watchpost.py +233 -0
- watchpost_cli-0.0.1/uv.lock +121 -0
- watchpost_cli-0.0.1/watchpost.py +304 -0
- watchpost_cli-0.0.1/watchpost_cli.egg-info/PKG-INFO +218 -0
- watchpost_cli-0.0.1/watchpost_cli.egg-info/SOURCES.txt +28 -0
- watchpost_cli-0.0.1/watchpost_cli.egg-info/dependency_links.txt +1 -0
- watchpost_cli-0.0.1/watchpost_cli.egg-info/entry_points.txt +2 -0
- watchpost_cli-0.0.1/watchpost_cli.egg-info/requires.txt +1 -0
- watchpost_cli-0.0.1/watchpost_cli.egg-info/top_level.txt +2 -0
- watchpost_cli-0.0.1/watchpost_report.py +205 -0
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.0.1
|
|
4
|
+
|
|
5
|
+
Initial version for local-first review of saved Nmap XML.
|
|
6
|
+
Delivers the scope of the [v0.1 project brief](docs/brief-v0.1.md).
|
|
7
|
+
See [GitHub Releases](https://github.com/ejames-dev/watchpost/releases) for publication dates and assets.
|
|
8
|
+
This changelog entry alone does not establish tag or registry publication.
|
|
9
|
+
|
|
10
|
+
### Included
|
|
11
|
+
|
|
12
|
+
- Inspect explicit IPv4/TCP observations in one saved scan.
|
|
13
|
+
- Compare completed TCP connect or SYN scans with matching Nmap versions and declared port coverage.
|
|
14
|
+
- Require human confirmation of scan location, intended targets, and remaining settings.
|
|
15
|
+
- Preserve missing observations as `not observed`, never assumed closed.
|
|
16
|
+
- Generate deterministic Markdown reports with evidence, limitations, suggested checks, and blank review fields.
|
|
17
|
+
- Protect existing report files and notes with non-replacing output.
|
|
18
|
+
- Reject unsafe XML, unsupported inputs, incompatible scans, and reported host timeouts.
|
|
19
|
+
- Provide three synthetic walkthroughs and a [user wiki](https://github.com/ejames-dev/watchpost/wiki).
|
|
20
|
+
- Distribute Watchpost under the MIT license.
|
|
21
|
+
- Use `watchpost-cli` as the distribution name, retaining the `watchpost` command and module.
|
|
22
|
+
- Provide a manual, environment-approved Trusted Publishing workflow for TestPyPI and PyPI.
|
|
23
|
+
|
|
24
|
+
### Boundaries
|
|
25
|
+
|
|
26
|
+
Python 3.11+ on Linux or Ubuntu WSL. CI covers Python 3.11, 3.12, and 3.13.
|
|
27
|
+
Inputs are limited to 10 MiB per snapshot. Report output requires a trusted directory with hard-link support.
|
|
28
|
+
Watchpost does not scan, upload data, or make runtime network calls.
|
|
29
|
+
It does not infer device/application identity or declare a network secure or compromised.
|
|
30
|
+
|
|
31
|
+
No live scanning, scheduling, dashboard, packet capture, UDP/IPv6 support, AI verdicts, or automatic remediation.
|
|
32
|
+
|
|
33
|
+
### Verification
|
|
34
|
+
|
|
35
|
+
See [the release-readiness checklist](docs/release-v0.0.1.md).
|
|
36
|
+
The package version alone does not prove publication.
|
|
37
|
+
Published releases, when available, appear on [GitHub Releases](https://github.com/ejames-dev/watchpost/releases).
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Evan Newman
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
include LICENSE
|
|
2
|
+
include CHANGELOG.md
|
|
3
|
+
include uv.lock
|
|
4
|
+
include examples/baseline.xml
|
|
5
|
+
include examples/after.xml
|
|
6
|
+
include examples/incomplete.xml
|
|
7
|
+
include examples/report.md
|
|
8
|
+
include docs/brief-v0.1.md
|
|
9
|
+
include docs/compare-milestone.md
|
|
10
|
+
include docs/reporting-milestone.md
|
|
11
|
+
include docs/scenarios.md
|
|
12
|
+
include docs/release-v0.0.1.md
|
|
13
|
+
include docs/publishing.md
|
|
@@ -0,0 +1,218 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: watchpost-cli
|
|
3
|
+
Version: 0.0.1
|
|
4
|
+
Summary: Local-first, evidence-backed reviews of changes between saved Nmap scans.
|
|
5
|
+
License-Expression: MIT
|
|
6
|
+
Project-URL: Repository, https://github.com/ejames-dev/watchpost
|
|
7
|
+
Project-URL: Documentation, https://github.com/ejames-dev/watchpost/wiki
|
|
8
|
+
Project-URL: Issues, https://github.com/ejames-dev/watchpost/issues
|
|
9
|
+
Requires-Python: >=3.11
|
|
10
|
+
Description-Content-Type: text/markdown
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Requires-Dist: defusedxml<0.8,>=0.7.1
|
|
13
|
+
Dynamic: license-file
|
|
14
|
+
|
|
15
|
+
# Watchpost
|
|
16
|
+
|
|
17
|
+
Local-first, evidence-backed reviews of changes between saved Nmap scans.
|
|
18
|
+
|
|
19
|
+
**Status: inspection, comparison, Markdown reporting, and synthetic walkthroughs implemented.**
|
|
20
|
+
Two saved XML files can produce a change-review report. The [three walkthroughs](docs/scenarios.md)
|
|
21
|
+
cover expected deployment, unintended backend exposure, and an incomplete scan.
|
|
22
|
+
[Published releases](https://github.com/ejames-dev/watchpost/releases) are listed on GitHub.
|
|
23
|
+
A source checkout can contain unreleased changes.
|
|
24
|
+
See the [release notes](CHANGELOG.md) and [release-readiness checklist](docs/release-v0.0.1.md).
|
|
25
|
+
|
|
26
|
+
**[User wiki](https://github.com/ejames-dev/watchpost/wiki)** — setup, report interpretation,
|
|
27
|
+
scenarios, troubleshooting, and safety.
|
|
28
|
+
|
|
29
|
+
## Package name
|
|
30
|
+
|
|
31
|
+
The distribution name for PyPI and TestPyPI is **`watchpost-cli`**.
|
|
32
|
+
The command and Python module remain `watchpost`. The GitHub repository remains `ejames-dev/watchpost`.
|
|
33
|
+
The registry package named `watchpost` belongs to an unrelated project. Do not install it for this tool.
|
|
34
|
+
Until registry publication is confirmed, use the source-checkout instructions below.
|
|
35
|
+
See [publishing setup](docs/publishing.md) for the maintainer workflow.
|
|
36
|
+
|
|
37
|
+
## Scope
|
|
38
|
+
|
|
39
|
+
Watchpost is for students and home-lab operators working on authorized networks.
|
|
40
|
+
Inspect a saved Nmap XML file or compare two snapshots of explicit IPv4/TCP observations.
|
|
41
|
+
Watchpost does not launch Nmap, discover devices, or make network requests.
|
|
42
|
+
|
|
43
|
+
The approved target is in [the v0.1 project brief](docs/brief-v0.1.md).
|
|
44
|
+
|
|
45
|
+
## Try the synthetic demo
|
|
46
|
+
|
|
47
|
+
With Python 3.11+ and [uv](https://docs.astral.sh/uv/) installed:
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
git clone https://github.com/ejames-dev/watchpost.git
|
|
51
|
+
cd watchpost
|
|
52
|
+
uv sync --locked --dev --python 3.11
|
|
53
|
+
uv run --offline watchpost inspect examples/baseline.xml
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
```text
|
|
57
|
+
IPv4/TCP observations (not a security verdict)
|
|
58
|
+
192.0.2.10
|
|
59
|
+
22/tcp open
|
|
60
|
+
80/tcp closed
|
|
61
|
+
443/tcp open
|
|
62
|
+
3000/tcp closed
|
|
63
|
+
192.0.2.20
|
|
64
|
+
22/tcp open
|
|
65
|
+
Only explicit port observations are listed; missing ports are not classified.
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Nmap is not required. These addresses and states are fictional.
|
|
69
|
+
For an existing checkout, run the final two commands from its root directory.
|
|
70
|
+
|
|
71
|
+
## Compare two snapshots
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
uv run --offline watchpost compare examples/baseline.xml examples/after.xml \
|
|
75
|
+
--confirm-same-context
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
The synthetic second scan is one minute later. Its only explicit state change is:
|
|
79
|
+
|
|
80
|
+
```text
|
|
81
|
+
STATE CHANGE 192.0.2.10 3000/tcp: closed -> open
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
The output includes both source paths, UTC scan windows, Nmap version, scan method,
|
|
85
|
+
declared port count, differences, and limitations. Without `--output`, the command prints
|
|
86
|
+
only to the terminal.
|
|
87
|
+
|
|
88
|
+
**Before using real inputs, confirm the same scan location, intended targets, and remaining
|
|
89
|
+
Nmap settings.** The flag records your confirmation. It does not let Watchpost verify the
|
|
90
|
+
network location, reconstruct the full command line, or override failed metadata checks.
|
|
91
|
+
Do not use it to force a comparison you know is incompatible.
|
|
92
|
+
|
|
93
|
+
Comparison currently requires:
|
|
94
|
+
|
|
95
|
+
- One TCP connect or SYN scan per snapshot, with the same method and Nmap version.
|
|
96
|
+
- Identical declared TCP port coverage. Equivalent lists and ranges are normalized.
|
|
97
|
+
- A declared port count that matches that coverage, with all explicit ports inside it.
|
|
98
|
+
- Successful scan completion and valid start/end timestamps.
|
|
99
|
+
- Earlier and later scan windows that do not overlap. Equal windows with identical supported
|
|
100
|
+
observations are allowed, so comparing a snapshot with itself produces no observed changes.
|
|
101
|
+
|
|
102
|
+
| Evidence | Output |
|
|
103
|
+
|---|---|
|
|
104
|
+
| Explicit states differ | `STATE CHANGE ... closed -> open` |
|
|
105
|
+
| A port has no earlier explicit record | `PORT OBSERVATION ... not observed -> open` |
|
|
106
|
+
| A port has no later explicit record | `PORT OBSERVATION ... open -> not observed` |
|
|
107
|
+
| An address appears in only one snapshot | `ADDRESS OBSERVATION`, not a device addition/removal claim |
|
|
108
|
+
| Supported explicit observations match | `No observed changes in explicit IPv4/TCP records.` |
|
|
109
|
+
|
|
110
|
+
A newly observed port is not proof that a service just started. A missing port is not proof of
|
|
111
|
+
closure. Grouped records and uncertain states do not become guessed open/closed results.
|
|
112
|
+
See the [comparison milestone](docs/compare-milestone.md) for boundaries and verification.
|
|
113
|
+
|
|
114
|
+
## Save a Markdown report
|
|
115
|
+
|
|
116
|
+
Create the output directory, then generate a report:
|
|
117
|
+
|
|
118
|
+
```bash
|
|
119
|
+
mkdir -p reports
|
|
120
|
+
uv run --offline watchpost compare examples/baseline.xml examples/after.xml \
|
|
121
|
+
--confirm-same-context --output reports/review.md
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Open `reports/review.md` in a text editor or Markdown viewer.
|
|
125
|
+
See [the synthetic sample report](examples/report.md) for the expected result.
|
|
126
|
+
|
|
127
|
+
Each report includes:
|
|
128
|
+
|
|
129
|
+
- Source paths, UTC scan windows, scan metadata, and user-confirmed assumptions.
|
|
130
|
+
- Separate counts for changed states, newly observed records, and records not observed later.
|
|
131
|
+
- Each finding's address, numeric TCP port ID where applicable, and earlier/later source evidence.
|
|
132
|
+
- Fixed-rule explanations and suggested checks. No AI verdicts or vulnerability scores.
|
|
133
|
+
- Evidence limitations and blank fields for investigation notes and conclusions.
|
|
134
|
+
|
|
135
|
+
**Existing files and notes are never overwritten.** If `reports/review.md` exists, choose a
|
|
136
|
+
new filename, such as `reports/review-02.md`. There is no force-overwrite option.
|
|
137
|
+
The same input paths and contents produce the same report content.
|
|
138
|
+
|
|
139
|
+
Validation finishes before any report is created. Watchpost writes a private temporary file,
|
|
140
|
+
then publishes the complete report with a non-replacing hard link. This also protects a destination
|
|
141
|
+
created by another writer. The output directory must exist and its filesystem must support hard links.
|
|
142
|
+
If it does not, the command fails instead of using an unsafe overwrite fallback.
|
|
143
|
+
This workflow targets Linux and Ubuntu WSL. Use a trusted output directory.
|
|
144
|
+
|
|
145
|
+
Keep the original XML files with your report. Reports reference the supplied files but do not embed
|
|
146
|
+
or authenticate them. Paths appear as escaped literals so filename markup stays plain text.
|
|
147
|
+
Review real reports before sharing: they contain network details and are not anonymous.
|
|
148
|
+
See [the reporting milestone](docs/reporting-milestone.md) for the safety checks and boundaries.
|
|
149
|
+
|
|
150
|
+
## Practice the three scenarios
|
|
151
|
+
|
|
152
|
+
Follow the [synthetic review walkthroughs](docs/scenarios.md). No live scan is required.
|
|
153
|
+
|
|
154
|
+
1. **Expected deployment:** compare an observed change with an approved lab deployment plan.
|
|
155
|
+
2. **Unintended backend exposure:** interpret the same evidence against a different access policy.
|
|
156
|
+
3. **Incomplete scan:** confirm that a reported host timeout prevents report creation.
|
|
157
|
+
|
|
158
|
+
The first two scenarios use the same XML pair deliberately. Watchpost reports evidence, not intent.
|
|
159
|
+
The walkthroughs supply fictional context and clearly labeled example review notes.
|
|
160
|
+
|
|
161
|
+
## How inspection works
|
|
162
|
+
|
|
163
|
+
1. Read the selected local file and reject inputs larger than 10 MiB.
|
|
164
|
+
2. Parse XML with entity expansion and external entities disabled.
|
|
165
|
+
3. Check the Nmap root, completion marker, TCP scan metadata, and explicit IPv4/TCP records.
|
|
166
|
+
4. Print addresses and ports in numerical order.
|
|
167
|
+
|
|
168
|
+
The command rejects malformed XML, unsupported encodings, failed scans, reported host timeouts,
|
|
169
|
+
UDP/IPv6 data, invalid addresses or ports, and duplicate observations.
|
|
170
|
+
Errors exit with code 2, without partial inventory output.
|
|
171
|
+
|
|
172
|
+
Only explicit `<port>` records are listed. This milestone does not expand grouped
|
|
173
|
+
`<extraports>` records, even when additional metadata is present. Hostnames, service names,
|
|
174
|
+
script output, and other free-text scan fields are not printed.
|
|
175
|
+
|
|
176
|
+
This is not full Nmap schema validation. `inspect` focuses on individual records;
|
|
177
|
+
`compare` adds the stronger metadata checks described above. Host status, service names,
|
|
178
|
+
script output, and grouped states are not compared. Matching metadata does not prove identical
|
|
179
|
+
scan conditions, and no-change output is not a safety verdict.
|
|
180
|
+
|
|
181
|
+
## Development
|
|
182
|
+
|
|
183
|
+
Python 3.11+ and [uv](https://docs.astral.sh/uv/) are required for these commands.
|
|
184
|
+
Dependency installation can need internet access. The application and tests run offline afterward.
|
|
185
|
+
|
|
186
|
+
```bash
|
|
187
|
+
uv sync --dev --python 3.11
|
|
188
|
+
uv run python -m unittest discover -s tests -v
|
|
189
|
+
uv run ruff format --check .
|
|
190
|
+
uv run ruff check .
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
The runtime uses `defusedxml` to reject XML entities instead of maintaining a custom XML parser.
|
|
194
|
+
The tests use the standard library's `unittest`. Pytest is an optional development runner.
|
|
195
|
+
|
|
196
|
+
## Data handling
|
|
197
|
+
|
|
198
|
+
- Only inspect data from networks you own or have permission to assess.
|
|
199
|
+
- All bundled XML examples and the sample report are synthetic and use documentation-only IP addresses.
|
|
200
|
+
- Keep real inputs in `scans/` and reports in `reports/`. Both directories are ignored by Git.
|
|
201
|
+
- Reports are not automatically anonymous. Review files before publishing them.
|
|
202
|
+
- Missing observations do not prove closed ports or removed devices.
|
|
203
|
+
- IP addresses are not device identities. Port numbers are not application identities.
|
|
204
|
+
- An observed change does not prove vulnerability or compromise.
|
|
205
|
+
|
|
206
|
+
## License
|
|
207
|
+
|
|
208
|
+
Watchpost is licensed under the [MIT License](LICENSE).
|
|
209
|
+
Third-party dependencies retain their own licenses.
|
|
210
|
+
|
|
211
|
+
## Roadmap
|
|
212
|
+
|
|
213
|
+
- [x] Safely inspect one saved IPv4/TCP scan.
|
|
214
|
+
- [x] Validate and compare two snapshots without inventing missing evidence.
|
|
215
|
+
- [x] Produce Markdown reports with evidence, limitations, and human review notes.
|
|
216
|
+
- [x] Document the three scenarios in the brief.
|
|
217
|
+
|
|
218
|
+
No dashboard, live scanning, scheduling, AI verdicts, or automatic remediation is planned in the v0.1 brief.
|
|
@@ -0,0 +1,204 @@
|
|
|
1
|
+
# Watchpost
|
|
2
|
+
|
|
3
|
+
Local-first, evidence-backed reviews of changes between saved Nmap scans.
|
|
4
|
+
|
|
5
|
+
**Status: inspection, comparison, Markdown reporting, and synthetic walkthroughs implemented.**
|
|
6
|
+
Two saved XML files can produce a change-review report. The [three walkthroughs](docs/scenarios.md)
|
|
7
|
+
cover expected deployment, unintended backend exposure, and an incomplete scan.
|
|
8
|
+
[Published releases](https://github.com/ejames-dev/watchpost/releases) are listed on GitHub.
|
|
9
|
+
A source checkout can contain unreleased changes.
|
|
10
|
+
See the [release notes](CHANGELOG.md) and [release-readiness checklist](docs/release-v0.0.1.md).
|
|
11
|
+
|
|
12
|
+
**[User wiki](https://github.com/ejames-dev/watchpost/wiki)** — setup, report interpretation,
|
|
13
|
+
scenarios, troubleshooting, and safety.
|
|
14
|
+
|
|
15
|
+
## Package name
|
|
16
|
+
|
|
17
|
+
The distribution name for PyPI and TestPyPI is **`watchpost-cli`**.
|
|
18
|
+
The command and Python module remain `watchpost`. The GitHub repository remains `ejames-dev/watchpost`.
|
|
19
|
+
The registry package named `watchpost` belongs to an unrelated project. Do not install it for this tool.
|
|
20
|
+
Until registry publication is confirmed, use the source-checkout instructions below.
|
|
21
|
+
See [publishing setup](docs/publishing.md) for the maintainer workflow.
|
|
22
|
+
|
|
23
|
+
## Scope
|
|
24
|
+
|
|
25
|
+
Watchpost is for students and home-lab operators working on authorized networks.
|
|
26
|
+
Inspect a saved Nmap XML file or compare two snapshots of explicit IPv4/TCP observations.
|
|
27
|
+
Watchpost does not launch Nmap, discover devices, or make network requests.
|
|
28
|
+
|
|
29
|
+
The approved target is in [the v0.1 project brief](docs/brief-v0.1.md).
|
|
30
|
+
|
|
31
|
+
## Try the synthetic demo
|
|
32
|
+
|
|
33
|
+
With Python 3.11+ and [uv](https://docs.astral.sh/uv/) installed:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
git clone https://github.com/ejames-dev/watchpost.git
|
|
37
|
+
cd watchpost
|
|
38
|
+
uv sync --locked --dev --python 3.11
|
|
39
|
+
uv run --offline watchpost inspect examples/baseline.xml
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
```text
|
|
43
|
+
IPv4/TCP observations (not a security verdict)
|
|
44
|
+
192.0.2.10
|
|
45
|
+
22/tcp open
|
|
46
|
+
80/tcp closed
|
|
47
|
+
443/tcp open
|
|
48
|
+
3000/tcp closed
|
|
49
|
+
192.0.2.20
|
|
50
|
+
22/tcp open
|
|
51
|
+
Only explicit port observations are listed; missing ports are not classified.
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Nmap is not required. These addresses and states are fictional.
|
|
55
|
+
For an existing checkout, run the final two commands from its root directory.
|
|
56
|
+
|
|
57
|
+
## Compare two snapshots
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
uv run --offline watchpost compare examples/baseline.xml examples/after.xml \
|
|
61
|
+
--confirm-same-context
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
The synthetic second scan is one minute later. Its only explicit state change is:
|
|
65
|
+
|
|
66
|
+
```text
|
|
67
|
+
STATE CHANGE 192.0.2.10 3000/tcp: closed -> open
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
The output includes both source paths, UTC scan windows, Nmap version, scan method,
|
|
71
|
+
declared port count, differences, and limitations. Without `--output`, the command prints
|
|
72
|
+
only to the terminal.
|
|
73
|
+
|
|
74
|
+
**Before using real inputs, confirm the same scan location, intended targets, and remaining
|
|
75
|
+
Nmap settings.** The flag records your confirmation. It does not let Watchpost verify the
|
|
76
|
+
network location, reconstruct the full command line, or override failed metadata checks.
|
|
77
|
+
Do not use it to force a comparison you know is incompatible.
|
|
78
|
+
|
|
79
|
+
Comparison currently requires:
|
|
80
|
+
|
|
81
|
+
- One TCP connect or SYN scan per snapshot, with the same method and Nmap version.
|
|
82
|
+
- Identical declared TCP port coverage. Equivalent lists and ranges are normalized.
|
|
83
|
+
- A declared port count that matches that coverage, with all explicit ports inside it.
|
|
84
|
+
- Successful scan completion and valid start/end timestamps.
|
|
85
|
+
- Earlier and later scan windows that do not overlap. Equal windows with identical supported
|
|
86
|
+
observations are allowed, so comparing a snapshot with itself produces no observed changes.
|
|
87
|
+
|
|
88
|
+
| Evidence | Output |
|
|
89
|
+
|---|---|
|
|
90
|
+
| Explicit states differ | `STATE CHANGE ... closed -> open` |
|
|
91
|
+
| A port has no earlier explicit record | `PORT OBSERVATION ... not observed -> open` |
|
|
92
|
+
| A port has no later explicit record | `PORT OBSERVATION ... open -> not observed` |
|
|
93
|
+
| An address appears in only one snapshot | `ADDRESS OBSERVATION`, not a device addition/removal claim |
|
|
94
|
+
| Supported explicit observations match | `No observed changes in explicit IPv4/TCP records.` |
|
|
95
|
+
|
|
96
|
+
A newly observed port is not proof that a service just started. A missing port is not proof of
|
|
97
|
+
closure. Grouped records and uncertain states do not become guessed open/closed results.
|
|
98
|
+
See the [comparison milestone](docs/compare-milestone.md) for boundaries and verification.
|
|
99
|
+
|
|
100
|
+
## Save a Markdown report
|
|
101
|
+
|
|
102
|
+
Create the output directory, then generate a report:
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
mkdir -p reports
|
|
106
|
+
uv run --offline watchpost compare examples/baseline.xml examples/after.xml \
|
|
107
|
+
--confirm-same-context --output reports/review.md
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Open `reports/review.md` in a text editor or Markdown viewer.
|
|
111
|
+
See [the synthetic sample report](examples/report.md) for the expected result.
|
|
112
|
+
|
|
113
|
+
Each report includes:
|
|
114
|
+
|
|
115
|
+
- Source paths, UTC scan windows, scan metadata, and user-confirmed assumptions.
|
|
116
|
+
- Separate counts for changed states, newly observed records, and records not observed later.
|
|
117
|
+
- Each finding's address, numeric TCP port ID where applicable, and earlier/later source evidence.
|
|
118
|
+
- Fixed-rule explanations and suggested checks. No AI verdicts or vulnerability scores.
|
|
119
|
+
- Evidence limitations and blank fields for investigation notes and conclusions.
|
|
120
|
+
|
|
121
|
+
**Existing files and notes are never overwritten.** If `reports/review.md` exists, choose a
|
|
122
|
+
new filename, such as `reports/review-02.md`. There is no force-overwrite option.
|
|
123
|
+
The same input paths and contents produce the same report content.
|
|
124
|
+
|
|
125
|
+
Validation finishes before any report is created. Watchpost writes a private temporary file,
|
|
126
|
+
then publishes the complete report with a non-replacing hard link. This also protects a destination
|
|
127
|
+
created by another writer. The output directory must exist and its filesystem must support hard links.
|
|
128
|
+
If it does not, the command fails instead of using an unsafe overwrite fallback.
|
|
129
|
+
This workflow targets Linux and Ubuntu WSL. Use a trusted output directory.
|
|
130
|
+
|
|
131
|
+
Keep the original XML files with your report. Reports reference the supplied files but do not embed
|
|
132
|
+
or authenticate them. Paths appear as escaped literals so filename markup stays plain text.
|
|
133
|
+
Review real reports before sharing: they contain network details and are not anonymous.
|
|
134
|
+
See [the reporting milestone](docs/reporting-milestone.md) for the safety checks and boundaries.
|
|
135
|
+
|
|
136
|
+
## Practice the three scenarios
|
|
137
|
+
|
|
138
|
+
Follow the [synthetic review walkthroughs](docs/scenarios.md). No live scan is required.
|
|
139
|
+
|
|
140
|
+
1. **Expected deployment:** compare an observed change with an approved lab deployment plan.
|
|
141
|
+
2. **Unintended backend exposure:** interpret the same evidence against a different access policy.
|
|
142
|
+
3. **Incomplete scan:** confirm that a reported host timeout prevents report creation.
|
|
143
|
+
|
|
144
|
+
The first two scenarios use the same XML pair deliberately. Watchpost reports evidence, not intent.
|
|
145
|
+
The walkthroughs supply fictional context and clearly labeled example review notes.
|
|
146
|
+
|
|
147
|
+
## How inspection works
|
|
148
|
+
|
|
149
|
+
1. Read the selected local file and reject inputs larger than 10 MiB.
|
|
150
|
+
2. Parse XML with entity expansion and external entities disabled.
|
|
151
|
+
3. Check the Nmap root, completion marker, TCP scan metadata, and explicit IPv4/TCP records.
|
|
152
|
+
4. Print addresses and ports in numerical order.
|
|
153
|
+
|
|
154
|
+
The command rejects malformed XML, unsupported encodings, failed scans, reported host timeouts,
|
|
155
|
+
UDP/IPv6 data, invalid addresses or ports, and duplicate observations.
|
|
156
|
+
Errors exit with code 2, without partial inventory output.
|
|
157
|
+
|
|
158
|
+
Only explicit `<port>` records are listed. This milestone does not expand grouped
|
|
159
|
+
`<extraports>` records, even when additional metadata is present. Hostnames, service names,
|
|
160
|
+
script output, and other free-text scan fields are not printed.
|
|
161
|
+
|
|
162
|
+
This is not full Nmap schema validation. `inspect` focuses on individual records;
|
|
163
|
+
`compare` adds the stronger metadata checks described above. Host status, service names,
|
|
164
|
+
script output, and grouped states are not compared. Matching metadata does not prove identical
|
|
165
|
+
scan conditions, and no-change output is not a safety verdict.
|
|
166
|
+
|
|
167
|
+
## Development
|
|
168
|
+
|
|
169
|
+
Python 3.11+ and [uv](https://docs.astral.sh/uv/) are required for these commands.
|
|
170
|
+
Dependency installation can need internet access. The application and tests run offline afterward.
|
|
171
|
+
|
|
172
|
+
```bash
|
|
173
|
+
uv sync --dev --python 3.11
|
|
174
|
+
uv run python -m unittest discover -s tests -v
|
|
175
|
+
uv run ruff format --check .
|
|
176
|
+
uv run ruff check .
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
The runtime uses `defusedxml` to reject XML entities instead of maintaining a custom XML parser.
|
|
180
|
+
The tests use the standard library's `unittest`. Pytest is an optional development runner.
|
|
181
|
+
|
|
182
|
+
## Data handling
|
|
183
|
+
|
|
184
|
+
- Only inspect data from networks you own or have permission to assess.
|
|
185
|
+
- All bundled XML examples and the sample report are synthetic and use documentation-only IP addresses.
|
|
186
|
+
- Keep real inputs in `scans/` and reports in `reports/`. Both directories are ignored by Git.
|
|
187
|
+
- Reports are not automatically anonymous. Review files before publishing them.
|
|
188
|
+
- Missing observations do not prove closed ports or removed devices.
|
|
189
|
+
- IP addresses are not device identities. Port numbers are not application identities.
|
|
190
|
+
- An observed change does not prove vulnerability or compromise.
|
|
191
|
+
|
|
192
|
+
## License
|
|
193
|
+
|
|
194
|
+
Watchpost is licensed under the [MIT License](LICENSE).
|
|
195
|
+
Third-party dependencies retain their own licenses.
|
|
196
|
+
|
|
197
|
+
## Roadmap
|
|
198
|
+
|
|
199
|
+
- [x] Safely inspect one saved IPv4/TCP scan.
|
|
200
|
+
- [x] Validate and compare two snapshots without inventing missing evidence.
|
|
201
|
+
- [x] Produce Markdown reports with evidence, limitations, and human review notes.
|
|
202
|
+
- [x] Document the three scenarios in the brief.
|
|
203
|
+
|
|
204
|
+
No dashboard, live scanning, scheduling, AI verdicts, or automatic remediation is planned in the v0.1 brief.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# Watchpost v0.1 project brief
|
|
2
|
+
|
|
3
|
+
Status: approved scope. Implementation starts with single-snapshot inspection.
|
|
4
|
+
Watchpost is the working name. This document describes the target, not completed features.
|
|
5
|
+
|
|
6
|
+
## Purpose
|
|
7
|
+
|
|
8
|
+
Turn two saved network scans into one clear, evidence-backed change report.
|
|
9
|
+
Serve students and home-lab operators working on networks they own or have permission to assess.
|
|
10
|
+
Explain what changed, what evidence supports the finding, and what to check next.
|
|
11
|
+
Do not decide that a network is secure or compromised.
|
|
12
|
+
|
|
13
|
+
## Target workflow
|
|
14
|
+
|
|
15
|
+
Earlier Nmap XML + later Nmap XML → validate → compare → Markdown report → human review.
|
|
16
|
+
Use Python 3.11+ on Linux or Ubuntu WSL. Support IPv4 addresses and TCP ports first.
|
|
17
|
+
No account, API key, or cloud service is required at runtime.
|
|
18
|
+
|
|
19
|
+
## Inputs
|
|
20
|
+
|
|
21
|
+
Accept two saved Nmap XML files with results and scan metadata.
|
|
22
|
+
Validate completion status and relevant settings, including port coverage.
|
|
23
|
+
Reject malformed, failed, incomplete, or incompatible inputs with an actionable explanation.
|
|
24
|
+
If required comparison metadata is missing, do not claim a reliable comparison.
|
|
25
|
+
The user confirms the same scan location and intended target scope.
|
|
26
|
+
XML alone cannot establish these facts.
|
|
27
|
+
Use synthetic bundled files so the demo does not need Nmap installed.
|
|
28
|
+
|
|
29
|
+
## Report
|
|
30
|
+
|
|
31
|
+
- Source files, scan times, and comparison assumptions.
|
|
32
|
+
- Counts of observed changes and missing observations.
|
|
33
|
+
- Address, TCP port, earlier state, later state, and source evidence per finding.
|
|
34
|
+
- Fixed-rule explanations and suggested checks, not AI verdicts.
|
|
35
|
+
- Limitations of the available evidence.
|
|
36
|
+
- Blank fields for human investigation notes and conclusions.
|
|
37
|
+
|
|
38
|
+
## Non-negotiable rules
|
|
39
|
+
|
|
40
|
+
- Missing observations stay unknown or not observed, not confirmed closed or removed.
|
|
41
|
+
- An IP address is not proof of a device identity.
|
|
42
|
+
- Changes are not proof of vulnerability or compromise.
|
|
43
|
+
- Port numbers alone do not identify applications.
|
|
44
|
+
- Input stays local. No scans, uploads, telemetry, or external XML requests.
|
|
45
|
+
- Reports are not automatically anonymous. Public examples must be synthetic.
|
|
46
|
+
- Never silently overwrite existing reports or investigation notes.
|
|
47
|
+
|
|
48
|
+
## Completion criteria
|
|
49
|
+
|
|
50
|
+
- A documented command generates a report from example files entirely offline after installation.
|
|
51
|
+
- Identical snapshots produce "no observed changes", not a safety verdict.
|
|
52
|
+
- Tests cover newly observed ports and explicit, supported port-state changes.
|
|
53
|
+
- Missing observations never become invented closures.
|
|
54
|
+
- Unsuitable inputs fail with clear explanations.
|
|
55
|
+
- Three scenarios document an expected deployment, unintended backend exposure, and an incomplete scan.
|
|
56
|
+
- The same inputs produce repeatable findings.
|
|
57
|
+
|
|
58
|
+
Planned and current test command: `python -m unittest discover -s tests -v`.
|
|
59
|
+
This command alone does not prove all target features exist. Check the README milestone status.
|
|
60
|
+
|
|
61
|
+
## Outside v0.1
|
|
62
|
+
|
|
63
|
+
Dashboard, live scans, scheduling, packet capture, database, AI analysis, vulnerability scoring,
|
|
64
|
+
automatic fixes, UDP, and IPv6.
|
|
65
|
+
|
|
66
|
+
## First milestone
|
|
67
|
+
|
|
68
|
+
1. Safely read one synthetic XML file.
|
|
69
|
+
2. Validate that it is a completed IPv4/TCP Nmap scan.
|
|
70
|
+
3. List explicit address and port-state observations in a stable order.
|
|
71
|
+
4. Test normal input, unsafe XML, unsupported input, and incomplete scans.
|
|
72
|
+
5. Document an offline demo and limitations before publishing.
|
|
73
|
+
|
|
74
|
+
Comparison and Markdown reporting follow this milestone.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# Snapshot comparison milestone
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
|
|
5
|
+
Add `watchpost compare BEFORE AFTER --confirm-same-context` without changing the existing inspect output.
|
|
6
|
+
Print a deterministic, evidence-linked terminal comparison.
|
|
7
|
+
The later [reporting milestone](reporting-milestone.md) adds optional Markdown output.
|
|
8
|
+
|
|
9
|
+
## Impact and implementation
|
|
10
|
+
|
|
11
|
+
- Reuse the existing safe XML parser and inventory validation in `watchpost.py`.
|
|
12
|
+
- Preserve `read_inventory(Path)` for its existing CLI and test callers.
|
|
13
|
+
- Add typed snapshot metadata and change records with standard-library dataclasses.
|
|
14
|
+
- Add a second synthetic example and comparison tests. No new dependencies or network calls.
|
|
15
|
+
- Extend the current CI demo and source archive to include the second example.
|
|
16
|
+
|
|
17
|
+
## Comparison gate
|
|
18
|
+
|
|
19
|
+
- Require explicit confirmation of the same scan location, intended target scope, and remaining Nmap settings.
|
|
20
|
+
- Support one TCP connect or SYN scan per snapshot for this milestone.
|
|
21
|
+
- Require matching Nmap versions, scan methods, and normalized TCP port sets.
|
|
22
|
+
- Require a valid declared port count and explicit observations within the declared coverage.
|
|
23
|
+
- Require valid UTC start/end times with start <= end.
|
|
24
|
+
- Reject reversed or overlapping scan windows. Equal windows with identical supported observations are allowed for repeat-input checks.
|
|
25
|
+
- Retain the existing failures for malformed XML, unsafe entities, unsupported addresses/protocols, timeouts, duplicates, and unsuccessful scans.
|
|
26
|
+
|
|
27
|
+
## Difference semantics
|
|
28
|
+
|
|
29
|
+
- Compare addresses by IP, not by claimed device identity.
|
|
30
|
+
- Report addresses appearing in only one snapshot as newly observed or not observed.
|
|
31
|
+
- Compare the union of explicit TCP port records for each address.
|
|
32
|
+
- Report an explicit state change only when both snapshots contain a state for that port.
|
|
33
|
+
- Missing or grouped records remain not observed. Never infer a closure from absence.
|
|
34
|
+
- Preserve uncertain states such as open|filtered without relabeling them as open.
|
|
35
|
+
- Sort addresses and ports numerically. Display source paths safely and scan times in UTC.
|
|
36
|
+
- For identical supported observations, say "No observed changes in explicit IPv4/TCP records", not "safe".
|
|
37
|
+
|
|
38
|
+
## Verification
|
|
39
|
+
|
|
40
|
+
`uv run --offline python -m unittest discover -s tests -v`
|
|
41
|
+
|
|
42
|
+
Cover metadata parsing and rejection, port ranges/counts, chronological ordering, identical inputs,
|
|
43
|
+
explicit state changes, missing ports/addresses, newly observed ports/addresses, uncertain states,
|
|
44
|
+
mandatory confirmation, deterministic output, safe source-path display, and no partial output on failure.
|
|
45
|
+
Run the existing suite, Ruff, package checks, and Python 3.11/3.12/3.13 CI.
|
|
46
|
+
|
|
47
|
+
## Limits and prior art
|
|
48
|
+
|
|
49
|
+
This milestone does not parse or compare the entire Nmap command line. Human confirmation covers
|
|
50
|
+
settings and environment that the automated checks do not establish. Equal metadata is not proof
|
|
51
|
+
of an equivalent measurement. Hostnames, application identities, grouped port states, and script
|
|
52
|
+
output remain outside the diff. No automatic scans or security verdicts.
|
|
53
|
+
Comparison writes only to the terminal unless `--output` selects a new report file.
|
|
54
|
+
|
|
55
|
+
Nmap already provides scanning and Ndiff provides raw comparisons. This project adds conservative
|
|
56
|
+
validation, explicit limitations, and a review workflow rather than a replacement scanner.
|
|
57
|
+
Reference: https://nmap.org/book/output-formats-xml-output.html
|