devshield-scanner 0.1.1__tar.gz → 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.0}/PKG-INFO +10 -2
- devshield_scanner-0.1.1/devshield_scanner.egg-info/PKG-INFO → devshield_scanner-0.2.0/README.md +53 -59
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.0}/devshield_linux.py +15 -7
- devshield_scanner-0.2.0/devshield_processes.py +169 -0
- devshield_scanner-0.1.1/README.md → devshield_scanner-0.2.0/devshield_scanner.egg-info/PKG-INFO +67 -45
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.0}/devshield_scanner.egg-info/SOURCES.txt +1 -0
- devshield_scanner-0.2.0/devshield_scanner.egg-info/top_level.txt +2 -0
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.0}/pyproject.toml +3 -3
- devshield_scanner-0.1.1/devshield_scanner.egg-info/top_level.txt +0 -1
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.0}/LICENSE +0 -0
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.0}/devshield_scanner.egg-info/dependency_links.txt +0 -0
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.0}/devshield_scanner.egg-info/entry_points.txt +0 -0
- {devshield_scanner-0.1.1 → devshield_scanner-0.2.0}/setup.cfg +0 -0
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: devshield-scanner
|
|
3
|
-
Version: 0.
|
|
4
|
-
Summary:
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Linux host/Docker inspection with opt-in response for detected process executables
|
|
5
5
|
License-Expression: MIT
|
|
6
6
|
Classifier: Development Status :: 3 - Alpha
|
|
7
7
|
Classifier: Environment :: Console
|
|
@@ -14,6 +14,14 @@ Dynamic: license-file
|
|
|
14
14
|
|
|
15
15
|
# DevShield for Linux
|
|
16
16
|
|
|
17
|
+
## Process-first response (0.2.0)
|
|
18
|
+
|
|
19
|
+
`sudo /opt/devshield/bin/devshield --remediate-running` first snapshots and scans running executables, sends SIGKILL to signature-detected processes, and attempts to move matching host executables into `/var/lib/devshield/quarantine`. It then scans host files and Docker data. Without this flag, scanning remains read-only. This is an experimental opt-in response mode and can interrupt production services. Process response currently requires root and Python/Linux pidfd support (normally Python 3.9+ and Linux 5.3+).
|
|
20
|
+
|
|
21
|
+
Heuristic, PUA, encrypted-content, scan-limit and EICAR test alerts never trigger automatic process response. Evidence copies and action records are saved before termination. PID 1, DevShield and its ancestors are excluded. Original-file moves require a matching inode and a same-filesystem rename; failures are reported rather than falling back to destructive copy/delete. Container executables are not moved: rebuild affected containers from trusted images. Supervisors may restart terminated processes. Script interpreters, injected code, memory-only malware and persistence are not comprehensively inspected, and files found during the subsequent host/container scan are not automatically removed. Quarantine samples must not be executed. Quarantine is excluded from later scans.
|
|
22
|
+
|
|
23
|
+
`--max-files 200000` raises the archive member-count limit when needed. Example for the large container export discussed during validation: `devshield --remediate-running --max-files 200000 --max-filesize-mb 1536 --max-scansize-mb 2048`. Limits do not apply to process snapshots, which retain ClamAV defaults with limit alerts. Increasing limits raises resource use. A privileged attacker can still evade or interfere with live-host inspection. There is no guarantee of complete malware removal.
|
|
24
|
+
|
|
17
25
|
An installable Python CLI package is included. The Python distribution is `devshield-scanner`; the command is `devshield`. See **INSTALL.md** for isolated installation under `/opt/devshield`. `devshield --version` displays the CLI version.
|
|
18
26
|
|
|
19
27
|
Version 0.1.1 enables ClamAV scan-limit alerts and marks zero-data scans, missing summaries and logged warnings/errors as incomplete coverage (exit 2), even when ClamAV returns 0. Limit alerts are not proof of malware. Optional `--max-filesize-mb` and `--max-scansize-mb` override engine limits in MB. Increasing limits can increase memory, CPU and runtime; engine hard limits still apply. A positive data count alone does not prove every archive entry was inspected.
|
devshield_scanner-0.1.1/devshield_scanner.egg-info/PKG-INFO → devshield_scanner-0.2.0/README.md
RENAMED
|
@@ -1,59 +1,53 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
```
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
```sh
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
#
|
|
42
|
-
sudo python3 devshield_linux.py --
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
Stopped containers, unused image layers, RAM, kernel state and rootkits are not covered. ClamAV's configured/default size, archive and recursion limits still apply, including to large container TAR exports; a successful scan can miss content past those limits. Review ClamAV summaries and warnings in every log. This is a best-effort filesystem scan, not a comprehensive compromise assessment. Live files can change during scanning. Use a trusted rescue environment for suspected root-level compromise.
|
|
56
|
-
|
|
57
|
-
Exit status: 0 = no signature detections reported and no recorded orchestration errors; 1 = detections; 2 = incomplete/error (detections may also exist; always inspect the report). No deletion or quarantine is requested. Reports persist if Docker scanning fails or the scan is interrupted with Ctrl+C. A forced kill or power loss may lose the active stage, but earlier checkpoints remain.
|
|
58
|
-
|
|
59
|
-
Validation on the development Windows machine covers syntax and mocked orchestration only. A real Linux/Docker/ClamAV integration scan must be validated on your server before operational use.
|
|
1
|
+
# DevShield for Linux
|
|
2
|
+
|
|
3
|
+
## Process-first response (0.2.0)
|
|
4
|
+
|
|
5
|
+
`sudo /opt/devshield/bin/devshield --remediate-running` first snapshots and scans running executables, sends SIGKILL to signature-detected processes, and attempts to move matching host executables into `/var/lib/devshield/quarantine`. It then scans host files and Docker data. Without this flag, scanning remains read-only. This is an experimental opt-in response mode and can interrupt production services. Process response currently requires root and Python/Linux pidfd support (normally Python 3.9+ and Linux 5.3+).
|
|
6
|
+
|
|
7
|
+
Heuristic, PUA, encrypted-content, scan-limit and EICAR test alerts never trigger automatic process response. Evidence copies and action records are saved before termination. PID 1, DevShield and its ancestors are excluded. Original-file moves require a matching inode and a same-filesystem rename; failures are reported rather than falling back to destructive copy/delete. Container executables are not moved: rebuild affected containers from trusted images. Supervisors may restart terminated processes. Script interpreters, injected code, memory-only malware and persistence are not comprehensively inspected, and files found during the subsequent host/container scan are not automatically removed. Quarantine samples must not be executed. Quarantine is excluded from later scans.
|
|
8
|
+
|
|
9
|
+
`--max-files 200000` raises the archive member-count limit when needed. Example for the large container export discussed during validation: `devshield --remediate-running --max-files 200000 --max-filesize-mb 1536 --max-scansize-mb 2048`. Limits do not apply to process snapshots, which retain ClamAV defaults with limit alerts. Increasing limits raises resource use. A privileged attacker can still evade or interfere with live-host inspection. There is no guarantee of complete malware removal.
|
|
10
|
+
|
|
11
|
+
An installable Python CLI package is included. The Python distribution is `devshield-scanner`; the command is `devshield`. See **INSTALL.md** for isolated installation under `/opt/devshield`. `devshield --version` displays the CLI version.
|
|
12
|
+
|
|
13
|
+
Version 0.1.1 enables ClamAV scan-limit alerts and marks zero-data scans, missing summaries and logged warnings/errors as incomplete coverage (exit 2), even when ClamAV returns 0. Limit alerts are not proof of malware. Optional `--max-filesize-mb` and `--max-scansize-mb` override engine limits in MB. Increasing limits can increase memory, CPU and runtime; engine hard limits still apply. A positive data count alone does not prove every archive entry was inspected.
|
|
14
|
+
|
|
15
|
+
Read-only host and Docker filesystem malware scanning using the locally installed ClamAV signature engine. Requires Linux, Python 3.8+, `clamscan`, updated ClamAV databases, and the Docker CLI for container scanning. Run on the Docker host with a local Unix socket context. Root access normally provides the required file/socket access.
|
|
16
|
+
|
|
17
|
+
On Debian/Ubuntu, install prerequisites using your normal package-management policy:
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
sudo apt-get update
|
|
21
|
+
sudo apt-get install python3 clamav clamav-freshclam
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Ensure the ClamAV databases have downloaded before scanning. Use your distribution's freshclam service, or `sudo freshclam` if that service is not already managing updates. The scanner does not install software, change services, or update databases automatically. Docker must already be installed if you use it.
|
|
25
|
+
|
|
26
|
+
Copy this folder onto the server, then run:
|
|
27
|
+
|
|
28
|
+
```sh
|
|
29
|
+
sudo python3 devshield_linux.py
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Console messages identify each stage and show a heartbeat every 10 seconds during long scans. Verbose file-level progress is saved to each stage's log; inspect a log with `tail -f /path/printed/by/script.log`. JSON results are checkpointed after every completed scan and on interruption, under `/var/log/devshield/<run>/report.json`.
|
|
33
|
+
|
|
34
|
+
```sh
|
|
35
|
+
# Check a small directory first
|
|
36
|
+
sudo python3 devshield_linux.py --host-root /srv/project --skip-docker
|
|
37
|
+
|
|
38
|
+
# Containers and volumes only
|
|
39
|
+
sudo python3 devshield_linux.py --skip-host
|
|
40
|
+
|
|
41
|
+
# Use a scratch disk with room for the largest container filesystem export
|
|
42
|
+
sudo python3 devshield_linux.py --temp-dir /mnt/scratch --heartbeat 5
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Host scanning starts at `/` and traverses mounted filesystems, including potentially slow network mounts. `/proc`, `/sys`, `/dev`, `/run`, raw default Docker/containerd storage, the report directory and scratch directory are excluded. No symlinks are followed. Custom Docker storage may be scanned twice. Logs can be large and contain private file paths; keep them private.
|
|
46
|
+
|
|
47
|
+
For each running container the script uses `docker export`, scans the resulting TAR as an archive, then deletes only its own temporary export. It does not execute programs inside containers, extract archives, stop containers or change their files. It separately scans bind-mount sources and discovered Docker volumes, including unused volumes. Remote daemons are rejected. Volume plugins/network-backed volumes may need separate mounting and scanning. If using a local socket proxy to a remote daemon, do not use this script: mount paths must refer to this machine.
|
|
48
|
+
|
|
49
|
+
Stopped containers, unused image layers, RAM, kernel state and rootkits are not covered. ClamAV's configured/default size, archive and recursion limits still apply, including to large container TAR exports; a successful scan can miss content past those limits. Review ClamAV summaries and warnings in every log. This is a best-effort filesystem scan, not a comprehensive compromise assessment. Live files can change during scanning. Use a trusted rescue environment for suspected root-level compromise.
|
|
50
|
+
|
|
51
|
+
Exit status: 0 = no signature detections reported and no recorded orchestration errors; 1 = detections; 2 = incomplete/error (detections may also exist; always inspect the report). No deletion or quarantine is requested. Reports persist if Docker scanning fails or the scan is interrupted with Ctrl+C. A forced kill or power loss may lose the active stage, but earlier checkpoints remain.
|
|
52
|
+
|
|
53
|
+
Validation on the development Windows machine covers syntax and mocked orchestration only. A real Linux/Docker/ClamAV integration scan must be validated on your server before operational use.
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
#!/usr/bin/env python3
|
|
2
|
-
"""
|
|
2
|
+
"""Linux host/Docker inspection with opt-in response for detected running executables."""
|
|
3
3
|
import argparse
|
|
4
4
|
import contextlib
|
|
5
5
|
import datetime as dt
|
|
@@ -12,6 +12,7 @@ import subprocess
|
|
|
12
12
|
import sys
|
|
13
13
|
import tempfile
|
|
14
14
|
import time
|
|
15
|
+
from devshield_processes import inspect_running
|
|
15
16
|
|
|
16
17
|
|
|
17
18
|
def coverage_warnings(log_path):
|
|
@@ -35,7 +36,10 @@ def coverage_warnings(log_path):
|
|
|
35
36
|
|
|
36
37
|
def main():
|
|
37
38
|
parser = argparse.ArgumentParser(description=__doc__)
|
|
38
|
-
parser.add_argument('--version', action='version', version='DevShield 0.
|
|
39
|
+
parser.add_argument('--version', action='version', version='DevShield 0.2.0')
|
|
40
|
+
parser.add_argument('--remediate-running', action='store_true', help='FIRST scan process executables; kill signature-detected processes and quarantine host binaries. Can interrupt services.')
|
|
41
|
+
parser.add_argument('--quarantine', default='/var/lib/devshield/quarantine', help='Root-owned 0700 quarantine directory')
|
|
42
|
+
parser.add_argument('--max-files', type=int, help='ClamAV maximum archive member count')
|
|
39
43
|
parser.add_argument('--max-filesize-mb', type=int, help='ClamAV per-file limit in MB (default: engine setting)')
|
|
40
44
|
parser.add_argument('--max-scansize-mb', type=int, help='ClamAV archive scan limit in MB (default: engine setting)')
|
|
41
45
|
parser.add_argument('--host-root', default='/', help='Host directory to scan (default /)')
|
|
@@ -49,9 +53,9 @@ def main():
|
|
|
49
53
|
parser.error('Run this script on Linux.')
|
|
50
54
|
if args.heartbeat < 1:
|
|
51
55
|
parser.error('--heartbeat must be positive')
|
|
52
|
-
for value in (args.max_filesize_mb, args.max_scansize_mb):
|
|
56
|
+
for value in (args.max_filesize_mb, args.max_scansize_mb, args.max_files):
|
|
53
57
|
if value is not None and value < 1:
|
|
54
|
-
parser.error('Scan
|
|
58
|
+
parser.error('Scan limits must be positive')
|
|
55
59
|
if not shutil.which('clamscan'):
|
|
56
60
|
parser.error('clamscan is missing. Install ClamAV and update its databases first; see README.md.')
|
|
57
61
|
report_dir = Path(args.reports).resolve() / (dt.datetime.now(dt.timezone.utc).strftime('%Y%m%dT%H%M%SZ') + '-' + str(os.getpid()))
|
|
@@ -110,6 +114,8 @@ def main():
|
|
|
110
114
|
command.append(f'--max-filesize={args.max_filesize_mb}M')
|
|
111
115
|
if args.max_scansize_mb is not None:
|
|
112
116
|
command.append(f'--max-scansize={args.max_scansize_mb}M')
|
|
117
|
+
if args.max_files is not None:
|
|
118
|
+
command.append(f'--max-files={args.max_files}')
|
|
113
119
|
command.extend('--exclude-dir=^' + re.escape(str(Path(p).resolve())) + r'(/|$)' for p in excludes)
|
|
114
120
|
rc, log = run([*command, '--', str(path)], label)
|
|
115
121
|
warnings = coverage_warnings(log)
|
|
@@ -129,12 +135,14 @@ def main():
|
|
|
129
135
|
version = subprocess.run(['clamscan', '--version'], capture_output=True, text=True)
|
|
130
136
|
report['engine'] = version.stdout.strip()
|
|
131
137
|
save()
|
|
138
|
+
if args.remediate_running:
|
|
139
|
+
inspect_running(run, issue, save, report, scratch.name, args.quarantine, coverage_warnings)
|
|
132
140
|
if not args.skip_host:
|
|
133
141
|
if not Path(args.host_root).is_dir():
|
|
134
142
|
issue('Host root is not a directory: ' + args.host_root)
|
|
135
143
|
else:
|
|
136
144
|
scan(Path(args.host_root).resolve(), 'Host filesystem',
|
|
137
|
-
['/proc', '/sys', '/dev', '/run', '/var/lib/docker', '/var/lib/containerd', report_dir.parent, scratch.name])
|
|
145
|
+
['/proc', '/sys', '/dev', '/run', '/var/lib/docker', '/var/lib/containerd', report_dir.parent, scratch.name, args.quarantine])
|
|
138
146
|
if not args.skip_docker:
|
|
139
147
|
if not shutil.which('docker'):
|
|
140
148
|
issue('Docker CLI missing; containers and volumes were not inspected.')
|
|
@@ -176,7 +184,7 @@ def main():
|
|
|
176
184
|
issue(f'Volume {name}: {exc}')
|
|
177
185
|
for source in sorted(mounts):
|
|
178
186
|
if Path(source).exists():
|
|
179
|
-
scan(source, 'Docker mount ' + source, ['/proc', '/sys', '/dev', '/run', report_dir.parent, scratch.name])
|
|
187
|
+
scan(source, 'Docker mount ' + source, ['/proc', '/sys', '/dev', '/run', report_dir.parent, scratch.name, args.quarantine])
|
|
180
188
|
else:
|
|
181
189
|
issue('Docker mount source inaccessible: ' + source)
|
|
182
190
|
except (OSError, RuntimeError, ValueError, KeyError, subprocess.SubprocessError) as exc:
|
|
@@ -192,7 +200,7 @@ def main():
|
|
|
192
200
|
print(f'Report: {report_dir / "report.json"}', flush=True)
|
|
193
201
|
if report['issues']:
|
|
194
202
|
return 2
|
|
195
|
-
return 1 if any(s['exit_code'] == 1 for s in report['scans']) else 0
|
|
203
|
+
return 1 if report.get('process_response') or any(s['exit_code'] == 1 for s in report['scans']) else 0
|
|
196
204
|
|
|
197
205
|
|
|
198
206
|
if __name__ == '__main__':
|
|
@@ -0,0 +1,169 @@
|
|
|
1
|
+
"""Conservative process-executable response; never infer a verdict from a PID/name."""
|
|
2
|
+
import hashlib
|
|
3
|
+
import os
|
|
4
|
+
from pathlib import Path
|
|
5
|
+
import re
|
|
6
|
+
import signal
|
|
7
|
+
import stat
|
|
8
|
+
|
|
9
|
+
|
|
10
|
+
def signature_names(log_path):
|
|
11
|
+
names = []
|
|
12
|
+
with open(log_path, encoding='utf-8', errors='replace') as stream:
|
|
13
|
+
for line in stream:
|
|
14
|
+
match = re.search(r': ([^\r\n]+) FOUND\s*$', line)
|
|
15
|
+
if match:
|
|
16
|
+
name = match.group(1)
|
|
17
|
+
# Coverage, heuristic/PUA and test alerts are insufficient for automatic response.
|
|
18
|
+
if not name.startswith(('Heuristics.', 'PUA.')) and not re.search(r'eicar|test[.-]signature', name, re.I):
|
|
19
|
+
names.append(name)
|
|
20
|
+
return names
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def digest(stream):
|
|
24
|
+
stream.seek(0)
|
|
25
|
+
result = hashlib.sha256()
|
|
26
|
+
for block in iter(lambda: stream.read(1024 * 1024), b''):
|
|
27
|
+
result.update(block)
|
|
28
|
+
return result.hexdigest()
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
def protected_pids():
|
|
32
|
+
protected = {1}
|
|
33
|
+
pid = os.getpid()
|
|
34
|
+
while pid > 0 and pid not in protected:
|
|
35
|
+
protected.add(pid)
|
|
36
|
+
try:
|
|
37
|
+
text = Path(f'/proc/{pid}/stat').read_text()
|
|
38
|
+
pid = int(text[text.rfind(')') + 2:].split()[1])
|
|
39
|
+
except (OSError, ValueError, IndexError):
|
|
40
|
+
break
|
|
41
|
+
return protected
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
def move_original(source, before, destination):
|
|
45
|
+
"""Same-filesystem rename only; never copy-and-delete across devices."""
|
|
46
|
+
parentfd = os.open(source.parent, os.O_RDONLY | os.O_DIRECTORY)
|
|
47
|
+
try:
|
|
48
|
+
current = os.stat(source.name, dir_fd=parentfd, follow_symlinks=False)
|
|
49
|
+
if not stat.S_ISREG(current.st_mode) or (current.st_dev, current.st_ino) != (before.st_dev, before.st_ino):
|
|
50
|
+
raise RuntimeError('Original path changed; not moved')
|
|
51
|
+
if destination.exists():
|
|
52
|
+
raise RuntimeError('Original quarantine destination already exists')
|
|
53
|
+
os.rename(source.name, str(destination), src_dir_fd=parentfd)
|
|
54
|
+
os.chmod(destination, 0o600)
|
|
55
|
+
finally:
|
|
56
|
+
os.close(parentfd)
|
|
57
|
+
|
|
58
|
+
|
|
59
|
+
def inspect_running(run, issue, save, report, scratch, quarantine, coverage_warnings):
|
|
60
|
+
"""Scan immutable copies, pin process identities, preserve evidence before SIGKILL."""
|
|
61
|
+
if os.geteuid() != 0:
|
|
62
|
+
issue('Running-process response requires root; stage skipped.')
|
|
63
|
+
return
|
|
64
|
+
if not hasattr(os, 'pidfd_open') or not hasattr(signal, 'pidfd_send_signal'):
|
|
65
|
+
issue('Running-process response requires Python/Linux pidfd support; no unsafe PID-only fallback is used.')
|
|
66
|
+
return
|
|
67
|
+
quarantine = Path(quarantine).absolute()
|
|
68
|
+
quarantine.mkdir(parents=True, mode=0o700, exist_ok=True)
|
|
69
|
+
metadata = quarantine.lstat()
|
|
70
|
+
if not stat.S_ISDIR(metadata.st_mode) or metadata.st_uid != 0 or metadata.st_mode & 0o077:
|
|
71
|
+
issue('Quarantine must be a root-owned directory with mode 0700; stage skipped.')
|
|
72
|
+
return
|
|
73
|
+
protected = protected_pids()
|
|
74
|
+
results = report.setdefault('process_response', [])
|
|
75
|
+
cache = {}
|
|
76
|
+
host_namespace = os.readlink('/proc/self/ns/mnt')
|
|
77
|
+
pids = sorted(int(p.name) for p in Path('/proc').iterdir() if p.name.isdigit())
|
|
78
|
+
print(f'[Processes] Inspecting executables for {len(pids)} processes. Scripts, RAM and persistence are not covered.', flush=True)
|
|
79
|
+
for number, pid in enumerate(pids, 1):
|
|
80
|
+
if pid in protected:
|
|
81
|
+
continue
|
|
82
|
+
pidfd = None
|
|
83
|
+
try:
|
|
84
|
+
pidfd = os.pidfd_open(pid)
|
|
85
|
+
exe_link = Path(f'/proc/{pid}/exe')
|
|
86
|
+
with exe_link.open('rb') as executable:
|
|
87
|
+
before = os.fstat(executable.fileno())
|
|
88
|
+
if not stat.S_ISREG(before.st_mode):
|
|
89
|
+
continue
|
|
90
|
+
key = (before.st_dev, before.st_ino, before.st_size, before.st_mtime_ns, before.st_ctime_ns)
|
|
91
|
+
source_name = os.readlink(exe_link)
|
|
92
|
+
print(f'[Processes {number}/{len(pids)}] PID {pid}: {source_name}', flush=True)
|
|
93
|
+
if key not in cache:
|
|
94
|
+
snapshot = Path(scratch) / f'process-{pid}.bin'
|
|
95
|
+
with snapshot.open('xb') as target:
|
|
96
|
+
for block in iter(lambda: executable.read(1024 * 1024), b''):
|
|
97
|
+
target.write(block)
|
|
98
|
+
with snapshot.open('rb') as target:
|
|
99
|
+
snapshot_hash = digest(target)
|
|
100
|
+
rc, log = run(['clamscan', '--infected', '--alert-exceeds-max=yes', '--', str(snapshot)], f'Process executable PID {pid}')
|
|
101
|
+
for warning in coverage_warnings(log):
|
|
102
|
+
issue(f'PID {pid}: {warning} See {log}')
|
|
103
|
+
names = signature_names(log) if rc == 1 else []
|
|
104
|
+
# Never kill when the executable changed while its snapshot was scanned.
|
|
105
|
+
if digest(executable) != snapshot_hash:
|
|
106
|
+
issue(f'PID {pid}: executable changed during inspection; response skipped.')
|
|
107
|
+
snapshot.unlink()
|
|
108
|
+
continue
|
|
109
|
+
if rc not in (0, 1):
|
|
110
|
+
issue(f'PID {pid}: executable scan failed; see {log}')
|
|
111
|
+
if rc == 1 and not names:
|
|
112
|
+
issue(f'PID {pid}: non-actionable alert; no process killed; see {log}')
|
|
113
|
+
if names:
|
|
114
|
+
evidence = quarantine / (snapshot_hash + '.sample')
|
|
115
|
+
# Create-only: never overwrite a prior sample or follow a pre-existing symlink.
|
|
116
|
+
try:
|
|
117
|
+
with evidence.open('xb') as target, snapshot.open('rb') as original:
|
|
118
|
+
os.chmod(evidence, 0o600)
|
|
119
|
+
for block in iter(lambda: original.read(1024 * 1024), b''):
|
|
120
|
+
target.write(block)
|
|
121
|
+
except FileExistsError:
|
|
122
|
+
if evidence.is_symlink():
|
|
123
|
+
raise RuntimeError('Unsafe quarantine sample link')
|
|
124
|
+
with evidence.open('rb') as existing:
|
|
125
|
+
if digest(existing) != snapshot_hash:
|
|
126
|
+
raise RuntimeError('Quarantine sample hash mismatch')
|
|
127
|
+
cache[key] = (names, snapshot_hash, log, str(evidence))
|
|
128
|
+
else:
|
|
129
|
+
cache[key] = ([], snapshot_hash, log, None)
|
|
130
|
+
snapshot.unlink()
|
|
131
|
+
names, snapshot_hash, log, evidence = cache[key]
|
|
132
|
+
if not names:
|
|
133
|
+
continue
|
|
134
|
+
current = exe_link.stat()
|
|
135
|
+
if (current.st_dev, current.st_ino) != (before.st_dev, before.st_ino) or digest(executable) != snapshot_hash:
|
|
136
|
+
issue(f'PID {pid}: executable identity changed; response skipped.')
|
|
137
|
+
continue
|
|
138
|
+
action = {'pid': pid, 'executable': source_name, 'signatures': names, 'sha256': snapshot_hash,
|
|
139
|
+
'log': log, 'evidence': evidence, 'termination': 'pending', 'file_action': 'not moved'}
|
|
140
|
+
results.append(action)
|
|
141
|
+
container_namespace = os.readlink(f'/proc/{pid}/ns/mnt') != host_namespace
|
|
142
|
+
save() # Preserve evidence and intent before touching the process.
|
|
143
|
+
signal.pidfd_send_signal(pidfd, signal.SIGKILL)
|
|
144
|
+
action['termination'] = 'SIGKILL sent via pidfd'
|
|
145
|
+
print(f'[RESPONSE] PID {pid}: SIGKILL sent; signatures: {names}', flush=True)
|
|
146
|
+
save()
|
|
147
|
+
# Never resolve a container path against the host filesystem.
|
|
148
|
+
if container_namespace:
|
|
149
|
+
action['file_action'] = 'container namespace: evidence saved, original not moved; rebuild container'
|
|
150
|
+
issue(f'PID {pid}: container executable requires container rebuild; supervisors may restart it.')
|
|
151
|
+
elif source_name.endswith(' (deleted)'):
|
|
152
|
+
action['file_action'] = 'original already unlinked; evidence retained'
|
|
153
|
+
else:
|
|
154
|
+
source = Path(source_name)
|
|
155
|
+
# Pin the parent directory to avoid a changed parent/symlink redirect.
|
|
156
|
+
destination = quarantine / f'{snapshot_hash}-{pid}.original'
|
|
157
|
+
move_original(source, before, destination)
|
|
158
|
+
action['file_action'] = 'moved to ' + str(destination)
|
|
159
|
+
save()
|
|
160
|
+
except (FileNotFoundError, ProcessLookupError):
|
|
161
|
+
# Most often a process exited between enumeration and inspection.
|
|
162
|
+
if 'action' in locals() and action.get('pid') == pid:
|
|
163
|
+
action['file_action'] = 'process/path exited during response; inspect source manually'
|
|
164
|
+
issue(f'PID {pid}: process/path disappeared during response; original removal unverified.')
|
|
165
|
+
except (OSError, RuntimeError) as exc:
|
|
166
|
+
issue(f'PID {pid}: {exc}; consult process_response for actions already performed.')
|
|
167
|
+
finally:
|
|
168
|
+
if pidfd is not None:
|
|
169
|
+
os.close(pidfd)
|
devshield_scanner-0.1.1/README.md → devshield_scanner-0.2.0/devshield_scanner.egg-info/PKG-INFO
RENAMED
|
@@ -1,45 +1,67 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
sudo
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: devshield-scanner
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Linux host/Docker inspection with opt-in response for detected process executables
|
|
5
|
+
License-Expression: MIT
|
|
6
|
+
Classifier: Development Status :: 3 - Alpha
|
|
7
|
+
Classifier: Environment :: Console
|
|
8
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
9
|
+
Classifier: Programming Language :: Python :: 3
|
|
10
|
+
Requires-Python: >=3.8
|
|
11
|
+
Description-Content-Type: text/markdown
|
|
12
|
+
License-File: LICENSE
|
|
13
|
+
Dynamic: license-file
|
|
14
|
+
|
|
15
|
+
# DevShield for Linux
|
|
16
|
+
|
|
17
|
+
## Process-first response (0.2.0)
|
|
18
|
+
|
|
19
|
+
`sudo /opt/devshield/bin/devshield --remediate-running` first snapshots and scans running executables, sends SIGKILL to signature-detected processes, and attempts to move matching host executables into `/var/lib/devshield/quarantine`. It then scans host files and Docker data. Without this flag, scanning remains read-only. This is an experimental opt-in response mode and can interrupt production services. Process response currently requires root and Python/Linux pidfd support (normally Python 3.9+ and Linux 5.3+).
|
|
20
|
+
|
|
21
|
+
Heuristic, PUA, encrypted-content, scan-limit and EICAR test alerts never trigger automatic process response. Evidence copies and action records are saved before termination. PID 1, DevShield and its ancestors are excluded. Original-file moves require a matching inode and a same-filesystem rename; failures are reported rather than falling back to destructive copy/delete. Container executables are not moved: rebuild affected containers from trusted images. Supervisors may restart terminated processes. Script interpreters, injected code, memory-only malware and persistence are not comprehensively inspected, and files found during the subsequent host/container scan are not automatically removed. Quarantine samples must not be executed. Quarantine is excluded from later scans.
|
|
22
|
+
|
|
23
|
+
`--max-files 200000` raises the archive member-count limit when needed. Example for the large container export discussed during validation: `devshield --remediate-running --max-files 200000 --max-filesize-mb 1536 --max-scansize-mb 2048`. Limits do not apply to process snapshots, which retain ClamAV defaults with limit alerts. Increasing limits raises resource use. A privileged attacker can still evade or interfere with live-host inspection. There is no guarantee of complete malware removal.
|
|
24
|
+
|
|
25
|
+
An installable Python CLI package is included. The Python distribution is `devshield-scanner`; the command is `devshield`. See **INSTALL.md** for isolated installation under `/opt/devshield`. `devshield --version` displays the CLI version.
|
|
26
|
+
|
|
27
|
+
Version 0.1.1 enables ClamAV scan-limit alerts and marks zero-data scans, missing summaries and logged warnings/errors as incomplete coverage (exit 2), even when ClamAV returns 0. Limit alerts are not proof of malware. Optional `--max-filesize-mb` and `--max-scansize-mb` override engine limits in MB. Increasing limits can increase memory, CPU and runtime; engine hard limits still apply. A positive data count alone does not prove every archive entry was inspected.
|
|
28
|
+
|
|
29
|
+
Read-only host and Docker filesystem malware scanning using the locally installed ClamAV signature engine. Requires Linux, Python 3.8+, `clamscan`, updated ClamAV databases, and the Docker CLI for container scanning. Run on the Docker host with a local Unix socket context. Root access normally provides the required file/socket access.
|
|
30
|
+
|
|
31
|
+
On Debian/Ubuntu, install prerequisites using your normal package-management policy:
|
|
32
|
+
|
|
33
|
+
```sh
|
|
34
|
+
sudo apt-get update
|
|
35
|
+
sudo apt-get install python3 clamav clamav-freshclam
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Ensure the ClamAV databases have downloaded before scanning. Use your distribution's freshclam service, or `sudo freshclam` if that service is not already managing updates. The scanner does not install software, change services, or update databases automatically. Docker must already be installed if you use it.
|
|
39
|
+
|
|
40
|
+
Copy this folder onto the server, then run:
|
|
41
|
+
|
|
42
|
+
```sh
|
|
43
|
+
sudo python3 devshield_linux.py
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
Console messages identify each stage and show a heartbeat every 10 seconds during long scans. Verbose file-level progress is saved to each stage's log; inspect a log with `tail -f /path/printed/by/script.log`. JSON results are checkpointed after every completed scan and on interruption, under `/var/log/devshield/<run>/report.json`.
|
|
47
|
+
|
|
48
|
+
```sh
|
|
49
|
+
# Check a small directory first
|
|
50
|
+
sudo python3 devshield_linux.py --host-root /srv/project --skip-docker
|
|
51
|
+
|
|
52
|
+
# Containers and volumes only
|
|
53
|
+
sudo python3 devshield_linux.py --skip-host
|
|
54
|
+
|
|
55
|
+
# Use a scratch disk with room for the largest container filesystem export
|
|
56
|
+
sudo python3 devshield_linux.py --temp-dir /mnt/scratch --heartbeat 5
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Host scanning starts at `/` and traverses mounted filesystems, including potentially slow network mounts. `/proc`, `/sys`, `/dev`, `/run`, raw default Docker/containerd storage, the report directory and scratch directory are excluded. No symlinks are followed. Custom Docker storage may be scanned twice. Logs can be large and contain private file paths; keep them private.
|
|
60
|
+
|
|
61
|
+
For each running container the script uses `docker export`, scans the resulting TAR as an archive, then deletes only its own temporary export. It does not execute programs inside containers, extract archives, stop containers or change their files. It separately scans bind-mount sources and discovered Docker volumes, including unused volumes. Remote daemons are rejected. Volume plugins/network-backed volumes may need separate mounting and scanning. If using a local socket proxy to a remote daemon, do not use this script: mount paths must refer to this machine.
|
|
62
|
+
|
|
63
|
+
Stopped containers, unused image layers, RAM, kernel state and rootkits are not covered. ClamAV's configured/default size, archive and recursion limits still apply, including to large container TAR exports; a successful scan can miss content past those limits. Review ClamAV summaries and warnings in every log. This is a best-effort filesystem scan, not a comprehensive compromise assessment. Live files can change during scanning. Use a trusted rescue environment for suspected root-level compromise.
|
|
64
|
+
|
|
65
|
+
Exit status: 0 = no signature detections reported and no recorded orchestration errors; 1 = detections; 2 = incomplete/error (detections may also exist; always inspect the report). No deletion or quarantine is requested. Reports persist if Docker scanning fails or the scan is interrupted with Ctrl+C. A forced kill or power loss may lose the active stage, but earlier checkpoints remain.
|
|
66
|
+
|
|
67
|
+
Validation on the development Windows machine covers syntax and mocked orchestration only. A real Linux/Docker/ClamAV integration scan must be validated on your server before operational use.
|
|
@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "devshield-scanner"
|
|
7
|
-
version = "0.
|
|
8
|
-
description = "
|
|
7
|
+
version = "0.2.0"
|
|
8
|
+
description = "Linux host/Docker inspection with opt-in response for detected process executables"
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.8"
|
|
11
11
|
dependencies = []
|
|
@@ -22,4 +22,4 @@ classifiers = [
|
|
|
22
22
|
devshield = "devshield_linux:main"
|
|
23
23
|
|
|
24
24
|
[tool.setuptools]
|
|
25
|
-
py-modules = ["devshield_linux"]
|
|
25
|
+
py-modules = ["devshield_linux", "devshield_processes"]
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
devshield_linux
|
|
File without changes
|
{devshield_scanner-0.1.1 → devshield_scanner-0.2.0}/devshield_scanner.egg-info/dependency_links.txt
RENAMED
|
File without changes
|
{devshield_scanner-0.1.1 → devshield_scanner-0.2.0}/devshield_scanner.egg-info/entry_points.txt
RENAMED
|
File without changes
|
|
File without changes
|