devshield-scanner 0.1.1__tar.gz → 0.2.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.1}/PKG-INFO +12 -2
- devshield_scanner-0.1.1/devshield_scanner.egg-info/PKG-INFO → devshield_scanner-0.2.1/README.md +55 -59
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.1}/devshield_linux.py +26 -11
- devshield_scanner-0.2.1/devshield_processes.py +173 -0
- devshield_scanner-0.1.1/README.md → devshield_scanner-0.2.1/devshield_scanner.egg-info/PKG-INFO +69 -45
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.1}/devshield_scanner.egg-info/SOURCES.txt +1 -0
- devshield_scanner-0.2.1/devshield_scanner.egg-info/top_level.txt +2 -0
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.1}/pyproject.toml +3 -3
- devshield_scanner-0.1.1/devshield_scanner.egg-info/top_level.txt +0 -1
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.1}/LICENSE +0 -0
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.1}/devshield_scanner.egg-info/dependency_links.txt +0 -0
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.1}/devshield_scanner.egg-info/entry_points.txt +0 -0
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.1}/setup.cfg +0 -0
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: devshield-scanner
|
|
3
|
-
Version: 0.
|
|
4
|
-
Summary:
|
|
3
|
+
Version: 0.2.1
|
|
4
|
+
Summary: Linux host/Docker inspection with opt-in response for detected process executables
|
|
5
5
|
License-Expression: MIT
|
|
6
6
|
Classifier: Development Status :: 3 - Alpha
|
|
7
7
|
Classifier: Environment :: Console
|
|
@@ -14,6 +14,16 @@ Dynamic: license-file
|
|
|
14
14
|
|
|
15
15
|
# DevShield for Linux
|
|
16
16
|
|
|
17
|
+
0.2.1 fixes custom scan-limit propagation: `--max-filesize-mb`, `--max-scansize-mb`, and `--max-files` now apply to running-process snapshots as well as filesystem/container scans. Use `--remediate-running --skip-host --skip-docker` to run only the process stage. This remains an opt-in mode that can interrupt services.
|
|
18
|
+
|
|
19
|
+
## Process-first response (0.2.0)
|
|
20
|
+
|
|
21
|
+
`sudo /opt/devshield/bin/devshield --remediate-running` first snapshots and scans running executables, sends SIGKILL to signature-detected processes, and attempts to move matching host executables into `/var/lib/devshield/quarantine`. It then scans host files and Docker data. Without this flag, scanning remains read-only. This is an experimental opt-in response mode and can interrupt production services. Process response currently requires root and Python/Linux pidfd support (normally Python 3.9+ and Linux 5.3+).
|
|
22
|
+
|
|
23
|
+
Heuristic, PUA, encrypted-content, scan-limit and EICAR test alerts never trigger automatic process response. Evidence copies and action records are saved before termination. PID 1, DevShield and its ancestors are excluded. Original-file moves require a matching inode and a same-filesystem rename; failures are reported rather than falling back to destructive copy/delete. Container executables are not moved: rebuild affected containers from trusted images. Supervisors may restart terminated processes. Script interpreters, injected code, memory-only malware and persistence are not comprehensively inspected, and files found during the subsequent host/container scan are not automatically removed. Quarantine samples must not be executed. Quarantine is excluded from later scans.
|
|
24
|
+
|
|
25
|
+
`--max-files 200000` raises the archive member-count limit when needed. Increasing limits raises resource use. A privileged attacker can still evade or interfere with live-host inspection. There is no guarantee of complete malware removal.
|
|
26
|
+
|
|
17
27
|
An installable Python CLI package is included. The Python distribution is `devshield-scanner`; the command is `devshield`. See **INSTALL.md** for isolated installation under `/opt/devshield`. `devshield --version` displays the CLI version.
|
|
18
28
|
|
|
19
29
|
Version 0.1.1 enables ClamAV scan-limit alerts and marks zero-data scans, missing summaries and logged warnings/errors as incomplete coverage (exit 2), even when ClamAV returns 0. Limit alerts are not proof of malware. Optional `--max-filesize-mb` and `--max-scansize-mb` override engine limits in MB. Increasing limits can increase memory, CPU and runtime; engine hard limits still apply. A positive data count alone does not prove every archive entry was inspected.
|
devshield_scanner-0.1.1/devshield_scanner.egg-info/PKG-INFO → devshield_scanner-0.2.1/README.md
RENAMED
|
@@ -1,59 +1,55 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
sudo
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
Exit status: 0 = no signature detections reported and no recorded orchestration errors; 1 = detections; 2 = incomplete/error (detections may also exist; always inspect the report). No deletion or quarantine is requested. Reports persist if Docker scanning fails or the scan is interrupted with Ctrl+C. A forced kill or power loss may lose the active stage, but earlier checkpoints remain.
|
|
58
|
-
|
|
59
|
-
Validation on the development Windows machine covers syntax and mocked orchestration only. A real Linux/Docker/ClamAV integration scan must be validated on your server before operational use.
|
|
1
|
+
# DevShield for Linux
|
|
2
|
+
|
|
3
|
+
0.2.1 fixes custom scan-limit propagation: `--max-filesize-mb`, `--max-scansize-mb`, and `--max-files` now apply to running-process snapshots as well as filesystem/container scans. Use `--remediate-running --skip-host --skip-docker` to run only the process stage. This remains an opt-in mode that can interrupt services.
|
|
4
|
+
|
|
5
|
+
## Process-first response (0.2.0)
|
|
6
|
+
|
|
7
|
+
`sudo /opt/devshield/bin/devshield --remediate-running` first snapshots and scans running executables, sends SIGKILL to signature-detected processes, and attempts to move matching host executables into `/var/lib/devshield/quarantine`. It then scans host files and Docker data. Without this flag, scanning remains read-only. This is an experimental opt-in response mode and can interrupt production services. Process response currently requires root and Python/Linux pidfd support (normally Python 3.9+ and Linux 5.3+).
|
|
8
|
+
|
|
9
|
+
Heuristic, PUA, encrypted-content, scan-limit and EICAR test alerts never trigger automatic process response. Evidence copies and action records are saved before termination. PID 1, DevShield and its ancestors are excluded. Original-file moves require a matching inode and a same-filesystem rename; failures are reported rather than falling back to destructive copy/delete. Container executables are not moved: rebuild affected containers from trusted images. Supervisors may restart terminated processes. Script interpreters, injected code, memory-only malware and persistence are not comprehensively inspected, and files found during the subsequent host/container scan are not automatically removed. Quarantine samples must not be executed. Quarantine is excluded from later scans.
|
|
10
|
+
|
|
11
|
+
`--max-files 200000` raises the archive member-count limit when needed. Increasing limits raises resource use. A privileged attacker can still evade or interfere with live-host inspection. There is no guarantee of complete malware removal.
|
|
12
|
+
|
|
13
|
+
An installable Python CLI package is included. The Python distribution is `devshield-scanner`; the command is `devshield`. See **INSTALL.md** for isolated installation under `/opt/devshield`. `devshield --version` displays the CLI version.
|
|
14
|
+
|
|
15
|
+
Version 0.1.1 enables ClamAV scan-limit alerts and marks zero-data scans, missing summaries and logged warnings/errors as incomplete coverage (exit 2), even when ClamAV returns 0. Limit alerts are not proof of malware. Optional `--max-filesize-mb` and `--max-scansize-mb` override engine limits in MB. Increasing limits can increase memory, CPU and runtime; engine hard limits still apply. A positive data count alone does not prove every archive entry was inspected.
|
|
16
|
+
|
|
17
|
+
Read-only host and Docker filesystem malware scanning using the locally installed ClamAV signature engine. Requires Linux, Python 3.8+, `clamscan`, updated ClamAV databases, and the Docker CLI for container scanning. Run on the Docker host with a local Unix socket context. Root access normally provides the required file/socket access.
|
|
18
|
+
|
|
19
|
+
On Debian/Ubuntu, install prerequisites using your normal package-management policy:
|
|
20
|
+
|
|
21
|
+
```sh
|
|
22
|
+
sudo apt-get update
|
|
23
|
+
sudo apt-get install python3 clamav clamav-freshclam
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Ensure the ClamAV databases have downloaded before scanning. Use your distribution's freshclam service, or `sudo freshclam` if that service is not already managing updates. The scanner does not install software, change services, or update databases automatically. Docker must already be installed if you use it.
|
|
27
|
+
|
|
28
|
+
Copy this folder onto the server, then run:
|
|
29
|
+
|
|
30
|
+
```sh
|
|
31
|
+
sudo python3 devshield_linux.py
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Console messages identify each stage and show a heartbeat every 10 seconds during long scans. Verbose file-level progress is saved to each stage's log; inspect a log with `tail -f /path/printed/by/script.log`. JSON results are checkpointed after every completed scan and on interruption, under `/var/log/devshield/<run>/report.json`.
|
|
35
|
+
|
|
36
|
+
```sh
|
|
37
|
+
# Check a small directory first
|
|
38
|
+
sudo python3 devshield_linux.py --host-root /srv/project --skip-docker
|
|
39
|
+
|
|
40
|
+
# Containers and volumes only
|
|
41
|
+
sudo python3 devshield_linux.py --skip-host
|
|
42
|
+
|
|
43
|
+
# Use a scratch disk with room for the largest container filesystem export
|
|
44
|
+
sudo python3 devshield_linux.py --temp-dir /mnt/scratch --heartbeat 5
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Host scanning starts at `/` and traverses mounted filesystems, including potentially slow network mounts. `/proc`, `/sys`, `/dev`, `/run`, raw default Docker/containerd storage, the report directory and scratch directory are excluded. No symlinks are followed. Custom Docker storage may be scanned twice. Logs can be large and contain private file paths; keep them private.
|
|
48
|
+
|
|
49
|
+
For each running container the script uses `docker export`, scans the resulting TAR as an archive, then deletes only its own temporary export. It does not execute programs inside containers, extract archives, stop containers or change their files. It separately scans bind-mount sources and discovered Docker volumes, including unused volumes. Remote daemons are rejected. Volume plugins/network-backed volumes may need separate mounting and scanning. If using a local socket proxy to a remote daemon, do not use this script: mount paths must refer to this machine.
|
|
50
|
+
|
|
51
|
+
Stopped containers, unused image layers, RAM, kernel state and rootkits are not covered. ClamAV's configured/default size, archive and recursion limits still apply, including to large container TAR exports; a successful scan can miss content past those limits. Review ClamAV summaries and warnings in every log. This is a best-effort filesystem scan, not a comprehensive compromise assessment. Live files can change during scanning. Use a trusted rescue environment for suspected root-level compromise.
|
|
52
|
+
|
|
53
|
+
Exit status: 0 = no signature detections reported and no recorded orchestration errors; 1 = detections; 2 = incomplete/error (detections may also exist; always inspect the report). No deletion or quarantine is requested. Reports persist if Docker scanning fails or the scan is interrupted with Ctrl+C. A forced kill or power loss may lose the active stage, but earlier checkpoints remain.
|
|
54
|
+
|
|
55
|
+
Validation on the development Windows machine covers syntax and mocked orchestration only. A real Linux/Docker/ClamAV integration scan must be validated on your server before operational use.
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
#!/usr/bin/env python3
|
|
2
|
-
"""
|
|
2
|
+
"""Linux host/Docker inspection with opt-in response for detected running executables."""
|
|
3
3
|
import argparse
|
|
4
4
|
import contextlib
|
|
5
5
|
import datetime as dt
|
|
@@ -12,6 +12,18 @@ import subprocess
|
|
|
12
12
|
import sys
|
|
13
13
|
import tempfile
|
|
14
14
|
import time
|
|
15
|
+
from devshield_processes import inspect_running
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
def clamav_limits(args):
|
|
19
|
+
options = []
|
|
20
|
+
if args.max_filesize_mb is not None:
|
|
21
|
+
options.append(f'--max-filesize={args.max_filesize_mb}M')
|
|
22
|
+
if args.max_scansize_mb is not None:
|
|
23
|
+
options.append(f'--max-scansize={args.max_scansize_mb}M')
|
|
24
|
+
if args.max_files is not None:
|
|
25
|
+
options.append(f'--max-files={args.max_files}')
|
|
26
|
+
return options
|
|
15
27
|
|
|
16
28
|
|
|
17
29
|
def coverage_warnings(log_path):
|
|
@@ -35,7 +47,10 @@ def coverage_warnings(log_path):
|
|
|
35
47
|
|
|
36
48
|
def main():
|
|
37
49
|
parser = argparse.ArgumentParser(description=__doc__)
|
|
38
|
-
parser.add_argument('--version', action='version', version='DevShield 0.
|
|
50
|
+
parser.add_argument('--version', action='version', version='DevShield 0.2.1')
|
|
51
|
+
parser.add_argument('--remediate-running', action='store_true', help='FIRST scan process executables; kill signature-detected processes and quarantine host binaries. Can interrupt services.')
|
|
52
|
+
parser.add_argument('--quarantine', default='/var/lib/devshield/quarantine', help='Root-owned 0700 quarantine directory')
|
|
53
|
+
parser.add_argument('--max-files', type=int, help='ClamAV maximum archive member count')
|
|
39
54
|
parser.add_argument('--max-filesize-mb', type=int, help='ClamAV per-file limit in MB (default: engine setting)')
|
|
40
55
|
parser.add_argument('--max-scansize-mb', type=int, help='ClamAV archive scan limit in MB (default: engine setting)')
|
|
41
56
|
parser.add_argument('--host-root', default='/', help='Host directory to scan (default /)')
|
|
@@ -45,13 +60,14 @@ def main():
|
|
|
45
60
|
parser.add_argument('--temp-dir', help='Scratch filesystem with enough space for container exports')
|
|
46
61
|
parser.add_argument('--heartbeat', type=int, default=10)
|
|
47
62
|
args = parser.parse_args()
|
|
63
|
+
scan_options = clamav_limits(args)
|
|
48
64
|
if not sys.platform.startswith('linux'):
|
|
49
65
|
parser.error('Run this script on Linux.')
|
|
50
66
|
if args.heartbeat < 1:
|
|
51
67
|
parser.error('--heartbeat must be positive')
|
|
52
|
-
for value in (args.max_filesize_mb, args.max_scansize_mb):
|
|
68
|
+
for value in (args.max_filesize_mb, args.max_scansize_mb, args.max_files):
|
|
53
69
|
if value is not None and value < 1:
|
|
54
|
-
parser.error('Scan
|
|
70
|
+
parser.error('Scan limits must be positive')
|
|
55
71
|
if not shutil.which('clamscan'):
|
|
56
72
|
parser.error('clamscan is missing. Install ClamAV and update its databases first; see README.md.')
|
|
57
73
|
report_dir = Path(args.reports).resolve() / (dt.datetime.now(dt.timezone.utc).strftime('%Y%m%dT%H%M%SZ') + '-' + str(os.getpid()))
|
|
@@ -106,10 +122,7 @@ def main():
|
|
|
106
122
|
def scan(path, label, excludes=()):
|
|
107
123
|
command = ['clamscan', '--recursive=yes', '--infected', '--verbose', '--scan-archive=yes',
|
|
108
124
|
'--cross-fs=yes', '--follow-dir-symlinks=0', '--follow-file-symlinks=0', '--alert-exceeds-max=yes']
|
|
109
|
-
|
|
110
|
-
command.append(f'--max-filesize={args.max_filesize_mb}M')
|
|
111
|
-
if args.max_scansize_mb is not None:
|
|
112
|
-
command.append(f'--max-scansize={args.max_scansize_mb}M')
|
|
125
|
+
command.extend(scan_options)
|
|
113
126
|
command.extend('--exclude-dir=^' + re.escape(str(Path(p).resolve())) + r'(/|$)' for p in excludes)
|
|
114
127
|
rc, log = run([*command, '--', str(path)], label)
|
|
115
128
|
warnings = coverage_warnings(log)
|
|
@@ -129,12 +142,14 @@ def main():
|
|
|
129
142
|
version = subprocess.run(['clamscan', '--version'], capture_output=True, text=True)
|
|
130
143
|
report['engine'] = version.stdout.strip()
|
|
131
144
|
save()
|
|
145
|
+
if args.remediate_running:
|
|
146
|
+
inspect_running(run, issue, save, report, scratch.name, args.quarantine, coverage_warnings, scan_options)
|
|
132
147
|
if not args.skip_host:
|
|
133
148
|
if not Path(args.host_root).is_dir():
|
|
134
149
|
issue('Host root is not a directory: ' + args.host_root)
|
|
135
150
|
else:
|
|
136
151
|
scan(Path(args.host_root).resolve(), 'Host filesystem',
|
|
137
|
-
['/proc', '/sys', '/dev', '/run', '/var/lib/docker', '/var/lib/containerd', report_dir.parent, scratch.name])
|
|
152
|
+
['/proc', '/sys', '/dev', '/run', '/var/lib/docker', '/var/lib/containerd', report_dir.parent, scratch.name, args.quarantine])
|
|
138
153
|
if not args.skip_docker:
|
|
139
154
|
if not shutil.which('docker'):
|
|
140
155
|
issue('Docker CLI missing; containers and volumes were not inspected.')
|
|
@@ -176,7 +191,7 @@ def main():
|
|
|
176
191
|
issue(f'Volume {name}: {exc}')
|
|
177
192
|
for source in sorted(mounts):
|
|
178
193
|
if Path(source).exists():
|
|
179
|
-
scan(source, 'Docker mount ' + source, ['/proc', '/sys', '/dev', '/run', report_dir.parent, scratch.name])
|
|
194
|
+
scan(source, 'Docker mount ' + source, ['/proc', '/sys', '/dev', '/run', report_dir.parent, scratch.name, args.quarantine])
|
|
180
195
|
else:
|
|
181
196
|
issue('Docker mount source inaccessible: ' + source)
|
|
182
197
|
except (OSError, RuntimeError, ValueError, KeyError, subprocess.SubprocessError) as exc:
|
|
@@ -192,7 +207,7 @@ def main():
|
|
|
192
207
|
print(f'Report: {report_dir / "report.json"}', flush=True)
|
|
193
208
|
if report['issues']:
|
|
194
209
|
return 2
|
|
195
|
-
return 1 if any(s['exit_code'] == 1 for s in report['scans']) else 0
|
|
210
|
+
return 1 if report.get('process_response') or any(s['exit_code'] == 1 for s in report['scans']) else 0
|
|
196
211
|
|
|
197
212
|
|
|
198
213
|
if __name__ == '__main__':
|
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
"""Conservative process-executable response; never infer a verdict from a PID/name."""
|
|
2
|
+
import hashlib
|
|
3
|
+
import os
|
|
4
|
+
from pathlib import Path
|
|
5
|
+
import re
|
|
6
|
+
import signal
|
|
7
|
+
import stat
|
|
8
|
+
|
|
9
|
+
|
|
10
|
+
def process_scan_command(snapshot, scan_options=()):
|
|
11
|
+
return ['clamscan', '--infected', '--alert-exceeds-max=yes', *scan_options, '--', str(snapshot)]
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
def signature_names(log_path):
|
|
15
|
+
names = []
|
|
16
|
+
with open(log_path, encoding='utf-8', errors='replace') as stream:
|
|
17
|
+
for line in stream:
|
|
18
|
+
match = re.search(r': ([^\r\n]+) FOUND\s*$', line)
|
|
19
|
+
if match:
|
|
20
|
+
name = match.group(1)
|
|
21
|
+
# Coverage, heuristic/PUA and test alerts are insufficient for automatic response.
|
|
22
|
+
if not name.startswith(('Heuristics.', 'PUA.')) and not re.search(r'eicar|test[.-]signature', name, re.I):
|
|
23
|
+
names.append(name)
|
|
24
|
+
return names
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
def digest(stream):
|
|
28
|
+
stream.seek(0)
|
|
29
|
+
result = hashlib.sha256()
|
|
30
|
+
for block in iter(lambda: stream.read(1024 * 1024), b''):
|
|
31
|
+
result.update(block)
|
|
32
|
+
return result.hexdigest()
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
def protected_pids():
|
|
36
|
+
protected = {1}
|
|
37
|
+
pid = os.getpid()
|
|
38
|
+
while pid > 0 and pid not in protected:
|
|
39
|
+
protected.add(pid)
|
|
40
|
+
try:
|
|
41
|
+
text = Path(f'/proc/{pid}/stat').read_text()
|
|
42
|
+
pid = int(text[text.rfind(')') + 2:].split()[1])
|
|
43
|
+
except (OSError, ValueError, IndexError):
|
|
44
|
+
break
|
|
45
|
+
return protected
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+
def move_original(source, before, destination):
|
|
49
|
+
"""Same-filesystem rename only; never copy-and-delete across devices."""
|
|
50
|
+
parentfd = os.open(source.parent, os.O_RDONLY | os.O_DIRECTORY)
|
|
51
|
+
try:
|
|
52
|
+
current = os.stat(source.name, dir_fd=parentfd, follow_symlinks=False)
|
|
53
|
+
if not stat.S_ISREG(current.st_mode) or (current.st_dev, current.st_ino) != (before.st_dev, before.st_ino):
|
|
54
|
+
raise RuntimeError('Original path changed; not moved')
|
|
55
|
+
if destination.exists():
|
|
56
|
+
raise RuntimeError('Original quarantine destination already exists')
|
|
57
|
+
os.rename(source.name, str(destination), src_dir_fd=parentfd)
|
|
58
|
+
os.chmod(destination, 0o600)
|
|
59
|
+
finally:
|
|
60
|
+
os.close(parentfd)
|
|
61
|
+
|
|
62
|
+
|
|
63
|
+
def inspect_running(run, issue, save, report, scratch, quarantine, coverage_warnings, scan_options=()):
|
|
64
|
+
"""Scan immutable copies, pin process identities, preserve evidence before SIGKILL."""
|
|
65
|
+
if os.geteuid() != 0:
|
|
66
|
+
issue('Running-process response requires root; stage skipped.')
|
|
67
|
+
return
|
|
68
|
+
if not hasattr(os, 'pidfd_open') or not hasattr(signal, 'pidfd_send_signal'):
|
|
69
|
+
issue('Running-process response requires Python/Linux pidfd support; no unsafe PID-only fallback is used.')
|
|
70
|
+
return
|
|
71
|
+
quarantine = Path(quarantine).absolute()
|
|
72
|
+
quarantine.mkdir(parents=True, mode=0o700, exist_ok=True)
|
|
73
|
+
metadata = quarantine.lstat()
|
|
74
|
+
if not stat.S_ISDIR(metadata.st_mode) or metadata.st_uid != 0 or metadata.st_mode & 0o077:
|
|
75
|
+
issue('Quarantine must be a root-owned directory with mode 0700; stage skipped.')
|
|
76
|
+
return
|
|
77
|
+
protected = protected_pids()
|
|
78
|
+
results = report.setdefault('process_response', [])
|
|
79
|
+
cache = {}
|
|
80
|
+
host_namespace = os.readlink('/proc/self/ns/mnt')
|
|
81
|
+
pids = sorted(int(p.name) for p in Path('/proc').iterdir() if p.name.isdigit())
|
|
82
|
+
print(f'[Processes] Inspecting executables for {len(pids)} processes. Scripts, RAM and persistence are not covered.', flush=True)
|
|
83
|
+
for number, pid in enumerate(pids, 1):
|
|
84
|
+
if pid in protected:
|
|
85
|
+
continue
|
|
86
|
+
pidfd = None
|
|
87
|
+
try:
|
|
88
|
+
pidfd = os.pidfd_open(pid)
|
|
89
|
+
exe_link = Path(f'/proc/{pid}/exe')
|
|
90
|
+
with exe_link.open('rb') as executable:
|
|
91
|
+
before = os.fstat(executable.fileno())
|
|
92
|
+
if not stat.S_ISREG(before.st_mode):
|
|
93
|
+
continue
|
|
94
|
+
key = (before.st_dev, before.st_ino, before.st_size, before.st_mtime_ns, before.st_ctime_ns)
|
|
95
|
+
source_name = os.readlink(exe_link)
|
|
96
|
+
print(f'[Processes {number}/{len(pids)}] PID {pid}: {source_name}', flush=True)
|
|
97
|
+
if key not in cache:
|
|
98
|
+
snapshot = Path(scratch) / f'process-{pid}.bin'
|
|
99
|
+
with snapshot.open('xb') as target:
|
|
100
|
+
for block in iter(lambda: executable.read(1024 * 1024), b''):
|
|
101
|
+
target.write(block)
|
|
102
|
+
with snapshot.open('rb') as target:
|
|
103
|
+
snapshot_hash = digest(target)
|
|
104
|
+
rc, log = run(process_scan_command(snapshot, scan_options), f'Process executable PID {pid}')
|
|
105
|
+
for warning in coverage_warnings(log):
|
|
106
|
+
issue(f'PID {pid}: {warning} See {log}')
|
|
107
|
+
names = signature_names(log) if rc == 1 else []
|
|
108
|
+
# Never kill when the executable changed while its snapshot was scanned.
|
|
109
|
+
if digest(executable) != snapshot_hash:
|
|
110
|
+
issue(f'PID {pid}: executable changed during inspection; response skipped.')
|
|
111
|
+
snapshot.unlink()
|
|
112
|
+
continue
|
|
113
|
+
if rc not in (0, 1):
|
|
114
|
+
issue(f'PID {pid}: executable scan failed; see {log}')
|
|
115
|
+
if rc == 1 and not names:
|
|
116
|
+
issue(f'PID {pid}: non-actionable alert; no process killed; see {log}')
|
|
117
|
+
if names:
|
|
118
|
+
evidence = quarantine / (snapshot_hash + '.sample')
|
|
119
|
+
# Create-only: never overwrite a prior sample or follow a pre-existing symlink.
|
|
120
|
+
try:
|
|
121
|
+
with evidence.open('xb') as target, snapshot.open('rb') as original:
|
|
122
|
+
os.chmod(evidence, 0o600)
|
|
123
|
+
for block in iter(lambda: original.read(1024 * 1024), b''):
|
|
124
|
+
target.write(block)
|
|
125
|
+
except FileExistsError:
|
|
126
|
+
if evidence.is_symlink():
|
|
127
|
+
raise RuntimeError('Unsafe quarantine sample link')
|
|
128
|
+
with evidence.open('rb') as existing:
|
|
129
|
+
if digest(existing) != snapshot_hash:
|
|
130
|
+
raise RuntimeError('Quarantine sample hash mismatch')
|
|
131
|
+
cache[key] = (names, snapshot_hash, log, str(evidence))
|
|
132
|
+
else:
|
|
133
|
+
cache[key] = ([], snapshot_hash, log, None)
|
|
134
|
+
snapshot.unlink()
|
|
135
|
+
names, snapshot_hash, log, evidence = cache[key]
|
|
136
|
+
if not names:
|
|
137
|
+
continue
|
|
138
|
+
current = exe_link.stat()
|
|
139
|
+
if (current.st_dev, current.st_ino) != (before.st_dev, before.st_ino) or digest(executable) != snapshot_hash:
|
|
140
|
+
issue(f'PID {pid}: executable identity changed; response skipped.')
|
|
141
|
+
continue
|
|
142
|
+
action = {'pid': pid, 'executable': source_name, 'signatures': names, 'sha256': snapshot_hash,
|
|
143
|
+
'log': log, 'evidence': evidence, 'termination': 'pending', 'file_action': 'not moved'}
|
|
144
|
+
results.append(action)
|
|
145
|
+
container_namespace = os.readlink(f'/proc/{pid}/ns/mnt') != host_namespace
|
|
146
|
+
save() # Preserve evidence and intent before touching the process.
|
|
147
|
+
signal.pidfd_send_signal(pidfd, signal.SIGKILL)
|
|
148
|
+
action['termination'] = 'SIGKILL sent via pidfd'
|
|
149
|
+
print(f'[RESPONSE] PID {pid}: SIGKILL sent; signatures: {names}', flush=True)
|
|
150
|
+
save()
|
|
151
|
+
# Never resolve a container path against the host filesystem.
|
|
152
|
+
if container_namespace:
|
|
153
|
+
action['file_action'] = 'container namespace: evidence saved, original not moved; rebuild container'
|
|
154
|
+
issue(f'PID {pid}: container executable requires container rebuild; supervisors may restart it.')
|
|
155
|
+
elif source_name.endswith(' (deleted)'):
|
|
156
|
+
action['file_action'] = 'original already unlinked; evidence retained'
|
|
157
|
+
else:
|
|
158
|
+
source = Path(source_name)
|
|
159
|
+
# Pin the parent directory to avoid a changed parent/symlink redirect.
|
|
160
|
+
destination = quarantine / f'{snapshot_hash}-{pid}.original'
|
|
161
|
+
move_original(source, before, destination)
|
|
162
|
+
action['file_action'] = 'moved to ' + str(destination)
|
|
163
|
+
save()
|
|
164
|
+
except (FileNotFoundError, ProcessLookupError):
|
|
165
|
+
# Most often a process exited between enumeration and inspection.
|
|
166
|
+
if 'action' in locals() and action.get('pid') == pid:
|
|
167
|
+
action['file_action'] = 'process/path exited during response; inspect source manually'
|
|
168
|
+
issue(f'PID {pid}: process/path disappeared during response; original removal unverified.')
|
|
169
|
+
except (OSError, RuntimeError) as exc:
|
|
170
|
+
issue(f'PID {pid}: {exc}; consult process_response for actions already performed.')
|
|
171
|
+
finally:
|
|
172
|
+
if pidfd is not None:
|
|
173
|
+
os.close(pidfd)
|
devshield_scanner-0.1.1/README.md → devshield_scanner-0.2.1/devshield_scanner.egg-info/PKG-INFO
RENAMED
|
@@ -1,45 +1,69 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
sudo
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: devshield-scanner
|
|
3
|
+
Version: 0.2.1
|
|
4
|
+
Summary: Linux host/Docker inspection with opt-in response for detected process executables
|
|
5
|
+
License-Expression: MIT
|
|
6
|
+
Classifier: Development Status :: 3 - Alpha
|
|
7
|
+
Classifier: Environment :: Console
|
|
8
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
9
|
+
Classifier: Programming Language :: Python :: 3
|
|
10
|
+
Requires-Python: >=3.8
|
|
11
|
+
Description-Content-Type: text/markdown
|
|
12
|
+
License-File: LICENSE
|
|
13
|
+
Dynamic: license-file
|
|
14
|
+
|
|
15
|
+
# DevShield for Linux
|
|
16
|
+
|
|
17
|
+
0.2.1 fixes custom scan-limit propagation: `--max-filesize-mb`, `--max-scansize-mb`, and `--max-files` now apply to running-process snapshots as well as filesystem/container scans. Use `--remediate-running --skip-host --skip-docker` to run only the process stage. This remains an opt-in mode that can interrupt services.
|
|
18
|
+
|
|
19
|
+
## Process-first response (0.2.0)
|
|
20
|
+
|
|
21
|
+
`sudo /opt/devshield/bin/devshield --remediate-running` first snapshots and scans running executables, sends SIGKILL to signature-detected processes, and attempts to move matching host executables into `/var/lib/devshield/quarantine`. It then scans host files and Docker data. Without this flag, scanning remains read-only. This is an experimental opt-in response mode and can interrupt production services. Process response currently requires root and Python/Linux pidfd support (normally Python 3.9+ and Linux 5.3+).
|
|
22
|
+
|
|
23
|
+
Heuristic, PUA, encrypted-content, scan-limit and EICAR test alerts never trigger automatic process response. Evidence copies and action records are saved before termination. PID 1, DevShield and its ancestors are excluded. Original-file moves require a matching inode and a same-filesystem rename; failures are reported rather than falling back to destructive copy/delete. Container executables are not moved: rebuild affected containers from trusted images. Supervisors may restart terminated processes. Script interpreters, injected code, memory-only malware and persistence are not comprehensively inspected, and files found during the subsequent host/container scan are not automatically removed. Quarantine samples must not be executed. Quarantine is excluded from later scans.
|
|
24
|
+
|
|
25
|
+
`--max-files 200000` raises the archive member-count limit when needed. Increasing limits raises resource use. A privileged attacker can still evade or interfere with live-host inspection. There is no guarantee of complete malware removal.
|
|
26
|
+
|
|
27
|
+
An installable Python CLI package is included. The Python distribution is `devshield-scanner`; the command is `devshield`. See **INSTALL.md** for isolated installation under `/opt/devshield`. `devshield --version` displays the CLI version.
|
|
28
|
+
|
|
29
|
+
Version 0.1.1 enables ClamAV scan-limit alerts and marks zero-data scans, missing summaries and logged warnings/errors as incomplete coverage (exit 2), even when ClamAV returns 0. Limit alerts are not proof of malware. Optional `--max-filesize-mb` and `--max-scansize-mb` override engine limits in MB. Increasing limits can increase memory, CPU and runtime; engine hard limits still apply. A positive data count alone does not prove every archive entry was inspected.
|
|
30
|
+
|
|
31
|
+
Read-only host and Docker filesystem malware scanning using the locally installed ClamAV signature engine. Requires Linux, Python 3.8+, `clamscan`, updated ClamAV databases, and the Docker CLI for container scanning. Run on the Docker host with a local Unix socket context. Root access normally provides the required file/socket access.
|
|
32
|
+
|
|
33
|
+
On Debian/Ubuntu, install prerequisites using your normal package-management policy:
|
|
34
|
+
|
|
35
|
+
```sh
|
|
36
|
+
sudo apt-get update
|
|
37
|
+
sudo apt-get install python3 clamav clamav-freshclam
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Ensure the ClamAV databases have downloaded before scanning. Use your distribution's freshclam service, or `sudo freshclam` if that service is not already managing updates. The scanner does not install software, change services, or update databases automatically. Docker must already be installed if you use it.
|
|
41
|
+
|
|
42
|
+
Copy this folder onto the server, then run:
|
|
43
|
+
|
|
44
|
+
```sh
|
|
45
|
+
sudo python3 devshield_linux.py
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Console messages identify each stage and show a heartbeat every 10 seconds during long scans. Verbose file-level progress is saved to each stage's log; inspect a log with `tail -f /path/printed/by/script.log`. JSON results are checkpointed after every completed scan and on interruption, under `/var/log/devshield/<run>/report.json`.
|
|
49
|
+
|
|
50
|
+
```sh
|
|
51
|
+
# Check a small directory first
|
|
52
|
+
sudo python3 devshield_linux.py --host-root /srv/project --skip-docker
|
|
53
|
+
|
|
54
|
+
# Containers and volumes only
|
|
55
|
+
sudo python3 devshield_linux.py --skip-host
|
|
56
|
+
|
|
57
|
+
# Use a scratch disk with room for the largest container filesystem export
|
|
58
|
+
sudo python3 devshield_linux.py --temp-dir /mnt/scratch --heartbeat 5
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
Host scanning starts at `/` and traverses mounted filesystems, including potentially slow network mounts. `/proc`, `/sys`, `/dev`, `/run`, raw default Docker/containerd storage, the report directory and scratch directory are excluded. No symlinks are followed. Custom Docker storage may be scanned twice. Logs can be large and contain private file paths; keep them private.
|
|
62
|
+
|
|
63
|
+
For each running container the script uses `docker export`, scans the resulting TAR as an archive, then deletes only its own temporary export. It does not execute programs inside containers, extract archives, stop containers or change their files. It separately scans bind-mount sources and discovered Docker volumes, including unused volumes. Remote daemons are rejected. Volume plugins/network-backed volumes may need separate mounting and scanning. If using a local socket proxy to a remote daemon, do not use this script: mount paths must refer to this machine.
|
|
64
|
+
|
|
65
|
+
Stopped containers, unused image layers, RAM, kernel state and rootkits are not covered. ClamAV's configured/default size, archive and recursion limits still apply, including to large container TAR exports; a successful scan can miss content past those limits. Review ClamAV summaries and warnings in every log. This is a best-effort filesystem scan, not a comprehensive compromise assessment. Live files can change during scanning. Use a trusted rescue environment for suspected root-level compromise.
|
|
66
|
+
|
|
67
|
+
Exit status: 0 = no signature detections reported and no recorded orchestration errors; 1 = detections; 2 = incomplete/error (detections may also exist; always inspect the report). No deletion or quarantine is requested. Reports persist if Docker scanning fails or the scan is interrupted with Ctrl+C. A forced kill or power loss may lose the active stage, but earlier checkpoints remain.
|
|
68
|
+
|
|
69
|
+
Validation on the development Windows machine covers syntax and mocked orchestration only. A real Linux/Docker/ClamAV integration scan must be validated on your server before operational use.
|
|
@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "devshield-scanner"
|
|
7
|
-
version = "0.
|
|
8
|
-
description = "
|
|
7
|
+
version = "0.2.1"
|
|
8
|
+
description = "Linux host/Docker inspection with opt-in response for detected process executables"
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.8"
|
|
11
11
|
dependencies = []
|
|
@@ -22,4 +22,4 @@ classifiers = [
|
|
|
22
22
|
devshield = "devshield_linux:main"
|
|
23
23
|
|
|
24
24
|
[tool.setuptools]
|
|
25
|
-
py-modules = ["devshield_linux"]
|
|
25
|
+
py-modules = ["devshield_linux", "devshield_processes"]
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
devshield_linux
|
|
File without changes
|
{devshield_scanner-0.1.1 → devshield_scanner-0.2.1}/devshield_scanner.egg-info/dependency_links.txt
RENAMED
|
File without changes
|
{devshield_scanner-0.1.1 → devshield_scanner-0.2.1}/devshield_scanner.egg-info/entry_points.txt
RENAMED
|
File without changes
|
|
File without changes
|