firewatcher 1.55__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (14) hide show
  1. firewatcher-1.55/MANIFEST.in +3 -0
  2. firewatcher-1.55/PKG-INFO +116 -0
  3. firewatcher-1.55/README.md +91 -0
  4. firewatcher-1.55/examples/nvme_failure.regex +1 -0
  5. firewatcher-1.55/examples/sys_msg.txt +15 -0
  6. firewatcher-1.55/firewatcher.egg-info/PKG-INFO +116 -0
  7. firewatcher-1.55/firewatcher.egg-info/SOURCES.txt +12 -0
  8. firewatcher-1.55/firewatcher.egg-info/dependency_links.txt +1 -0
  9. firewatcher-1.55/firewatcher.egg-info/entry_points.txt +3 -0
  10. firewatcher-1.55/firewatcher.egg-info/top_level.txt +1 -0
  11. firewatcher-1.55/firewatcher.py +630 -0
  12. firewatcher-1.55/setup.cfg +4 -0
  13. firewatcher-1.55/setup.py +29 -0
  14. firewatcher-1.55/test_firewatcher.py +871 -0
firewatcher-1.55/MANIFEST.in ADDED
@@ -0,0 +1,3 @@
1
+ include README.md
2
+ include examples/*
3
+ include test_firewatcher.py
firewatcher-1.55/PKG-INFO ADDED
@@ -0,0 +1,116 @@
1
+ Metadata-Version: 2.4
2
+ Name: firewatcher
3
+ Version: 1.55
4
+ Summary: Watch logs for incident patterns and persist surrounding context off-box
5
+ Home-page: https://github.com/yufei-pan/firewatcher
6
+ Author: Yufei Pan
7
+ Author-email: pan@zopyr.us
8
+ License: GPLv3+
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
11
+ Classifier: Operating System :: POSIX :: Linux
12
+ Classifier: Topic :: System :: Logging
13
+ Classifier: Topic :: System :: Monitoring
14
+ Requires-Python: >=3.6
15
+ Description-Content-Type: text/markdown
16
+ Dynamic: author
17
+ Dynamic: author-email
18
+ Dynamic: classifier
19
+ Dynamic: description
20
+ Dynamic: description-content-type
21
+ Dynamic: home-page
22
+ Dynamic: license
23
+ Dynamic: requires-python
24
+ Dynamic: summary
25
+
26
+ # firewatcher
27
+
28
+ Watch a log stream (journald, syslog, or any file) for incident patterns, then persist a window of surrounding messages to a durable location — including a network filesystem — so the record survives even if the machine later dies.
29
+
30
+ This is the PyPI name for the internal `firewatch` tool (in use since 2023). The `firewatch` command remains as an alias. The PyPI name `firewatch` is a different, unrelated project.
31
+
32
+ The watcher prefers `journalctl --follow` when `journalctl` is on PATH. On hosts without journald it falls back to `/var/log/syslog`, then `/var/log/messages`, or `--log-file`. Running the daemon does not require systemd; only `--install-service` / `--uninstall-service` do.
33
+
34
+ ## Install
35
+
36
+ ```bash
37
+ pip install firewatcher
38
+ ```
39
+
40
+ From a clone of this repository:
41
+
42
+ ```bash
43
+ pip install .
44
+ ```
45
+
46
+ Requires **Python 3.6+**. No third-party runtime dependencies.
47
+
48
+ ## Quick start
49
+
50
+ ```bash
51
+ firewatcher /etc/firewatcher/patterns.d
52
+ firewatch -t 300 -o /var/log/captured_messages/ /etc/firewatcher/patterns.d
53
+ ```
54
+
55
+ A directory argument loads every non-hidden pattern file inside it. Files ending in `.regex` are compiled as regular expressions. Any other pattern file is treated as fnmatch (bash-like) substrings, with `*` added on both ends.
56
+
57
+ `--install-service` seeds `/etc/firewatcher/patterns.d` with example patterns when that directory is empty. The `examples/` directory in the source tree is not installed onto `PATH` by `pip install`; copy those files from a clone, or let `--install-service` write them.
58
+
59
+ On a match, `firewatcher` writes:
60
+
61
+ - a per-incident capture under `{output-folder}/{YYYY-MM}/…`
62
+ - a line in `{output-folder}/journal.log`
63
+
64
+ Live capture is written as lines arrive, so a copy can already be on another filesystem if the host then disappears.
65
+
66
+ ## systemd service
67
+
68
+ ```bash
69
+ sudo firewatcher --install-service
70
+ sudo firewatcher --print-unit
71
+ sudo firewatcher --uninstall-service
72
+ ```
73
+
74
+ `--install-service` writes `/etc/systemd/system/firewatcher.service`, creates the output directory, seeds `/etc/firewatcher/patterns.d/` with example patterns **only if that directory is empty**, then `systemctl daemon-reload && systemctl enable --now firewatcher`.
75
+
76
+ If systemd is not available, `--install-service` and `--uninstall-service` print a warning and exit. `--print-unit` still works so you can copy the unit elsewhere.
77
+
78
+ Useful flags:
79
+
80
+ | Flag | Description |
81
+ |------|-------------|
82
+ | `--unit-name NAME` | Unit name (default: `firewatcher`) |
83
+ | `--requires-mounts-for PATH` | Add `RequiresMountsFor=` (repeat for NFS/remote output) |
84
+ | `--no-enable` | Write the unit and reload, but do not enable or start it |
85
+
86
+ ```bash
87
+ sudo firewatcher --install-service -o /mnt/logs/captured_messages \
88
+ --requires-mounts-for /mnt/logs/captured_messages \
89
+ /etc/firewatcher/patterns.d
90
+ ```
91
+
92
+ ## Options
93
+
94
+ | Flag | Description |
95
+ |------|-------------|
96
+ | `pattern_file …` | Pattern files or directories |
97
+ | `--log-file` | Log file to follow (default: journalctl / syslog / messages) |
98
+ | `-t`, `--capture-time` | Seconds of context after a match (default: 300) |
99
+ | `-o`, `--output-folder` | Where to write captures (default: `/var/log/captured_messages/`) |
100
+ | `--tail_lines` | Lines to start with (default: 20; `+N` from line N, not with journalctl) |
101
+ | `--filter_only` | Filter existing logs only (`cat` instead of `tail -F`, or `journalctl` without `--follow`) |
102
+ | `--compress-after-months` | Tar.xz monthly dirs after N months (default: 3) |
103
+ | `--delete-after-months` | Delete `YYYY-MM` dirs after N months (default: 0 = never) |
104
+ | `--capture_line_count_max` | Max lines before/after a match (default: 10000) |
105
+ | `--install-service` | Install and enable a systemd unit |
106
+ | `--print-unit` | Print that unit and exit |
107
+ | `--uninstall-service` | Disable and remove the unit |
108
+ | `-V`, `--version` | Show version and exit |
109
+
110
+ ## Author
111
+
112
+ Yufei Pan (pan@zopyr.us)
113
+
114
+ ## License
115
+
116
+ GPL-3.0-or-later
firewatcher-1.55/README.md ADDED
@@ -0,0 +1,91 @@
1
+ # firewatcher
2
+
3
+ Watch a log stream (journald, syslog, or any file) for incident patterns, then persist a window of surrounding messages to a durable location — including a network filesystem — so the record survives even if the machine later dies.
4
+
5
+ This is the PyPI name for the internal `firewatch` tool (in use since 2023). The `firewatch` command remains as an alias. The PyPI name `firewatch` is a different, unrelated project.
6
+
7
+ The watcher prefers `journalctl --follow` when `journalctl` is on PATH. On hosts without journald it falls back to `/var/log/syslog`, then `/var/log/messages`, or `--log-file`. Running the daemon does not require systemd; only `--install-service` / `--uninstall-service` do.
8
+
9
+ ## Install
10
+
11
+ ```bash
12
+ pip install firewatcher
13
+ ```
14
+
15
+ From a clone of this repository:
16
+
17
+ ```bash
18
+ pip install .
19
+ ```
20
+
21
+ Requires **Python 3.6+**. No third-party runtime dependencies.
22
+
23
+ ## Quick start
24
+
25
+ ```bash
26
+ firewatcher /etc/firewatcher/patterns.d
27
+ firewatch -t 300 -o /var/log/captured_messages/ /etc/firewatcher/patterns.d
28
+ ```
29
+
30
+ A directory argument loads every non-hidden pattern file inside it. Files ending in `.regex` are compiled as regular expressions. Any other pattern file is treated as fnmatch (bash-like) substrings, with `*` added on both ends.
31
+
32
+ `--install-service` seeds `/etc/firewatcher/patterns.d` with example patterns when that directory is empty. The `examples/` directory in the source tree is not installed onto `PATH` by `pip install`; copy those files from a clone, or let `--install-service` write them.
33
+
34
+ On a match, `firewatcher` writes:
35
+
36
+ - a per-incident capture under `{output-folder}/{YYYY-MM}/…`
37
+ - a line in `{output-folder}/journal.log`
38
+
39
+ Live capture is written as lines arrive, so a copy can already be on another filesystem if the host then disappears.
40
+
41
+ ## systemd service
42
+
43
+ ```bash
44
+ sudo firewatcher --install-service
45
+ sudo firewatcher --print-unit
46
+ sudo firewatcher --uninstall-service
47
+ ```
48
+
49
+ `--install-service` writes `/etc/systemd/system/firewatcher.service`, creates the output directory, seeds `/etc/firewatcher/patterns.d/` with example patterns **only if that directory is empty**, then `systemctl daemon-reload && systemctl enable --now firewatcher`.
50
+
51
+ If systemd is not available, `--install-service` and `--uninstall-service` print a warning and exit. `--print-unit` still works so you can copy the unit elsewhere.
52
+
53
+ Useful flags:
54
+
55
+ | Flag | Description |
56
+ |------|-------------|
57
+ | `--unit-name NAME` | Unit name (default: `firewatcher`) |
58
+ | `--requires-mounts-for PATH` | Add `RequiresMountsFor=` (repeat for NFS/remote output) |
59
+ | `--no-enable` | Write the unit and reload, but do not enable or start it |
60
+
61
+ ```bash
62
+ sudo firewatcher --install-service -o /mnt/logs/captured_messages \
63
+ --requires-mounts-for /mnt/logs/captured_messages \
64
+ /etc/firewatcher/patterns.d
65
+ ```
66
+
67
+ ## Options
68
+
69
+ | Flag | Description |
70
+ |------|-------------|
71
+ | `pattern_file …` | Pattern files or directories |
72
+ | `--log-file` | Log file to follow (default: journalctl / syslog / messages) |
73
+ | `-t`, `--capture-time` | Seconds of context after a match (default: 300) |
74
+ | `-o`, `--output-folder` | Where to write captures (default: `/var/log/captured_messages/`) |
75
+ | `--tail_lines` | Lines to start with (default: 20; `+N` from line N, not with journalctl) |
76
+ | `--filter_only` | Filter existing logs only (`cat` instead of `tail -F`, or `journalctl` without `--follow`) |
77
+ | `--compress-after-months` | Tar.xz monthly dirs after N months (default: 3) |
78
+ | `--delete-after-months` | Delete `YYYY-MM` dirs after N months (default: 0 = never) |
79
+ | `--capture_line_count_max` | Max lines before/after a match (default: 10000) |
80
+ | `--install-service` | Install and enable a systemd unit |
81
+ | `--print-unit` | Print that unit and exit |
82
+ | `--uninstall-service` | Disable and remove the unit |
83
+ | `-V`, `--version` | Show version and exit |
84
+
85
+ ## Author
86
+
87
+ Yufei Pan (pan@zopyr.us)
88
+
89
+ ## License
90
+
91
+ GPL-3.0-or-later
firewatcher-1.55/examples/nvme_failure.regex ADDED
@@ -0,0 +1 @@
1
+ (?i)(?:^|[\s:])nvme(?:\d+n\d+)?\W.*(timeout|abort|restart|reset|unable|cannot|invalid|froze|fail|down|above|cancel|dead|large|bogus|could not|deprecate)
firewatcher-1.55/examples/sys_msg.txt ADDED
@@ -0,0 +1,15 @@
1
+ hard resetting link
2
+ Initializing cgroup subsys cpuset
3
+ Linux version
4
+ Out of Memory
5
+ Call Trace
6
+ I/O error
7
+ bad sector
8
+ panic
9
+ Critical
10
+ controller is down
11
+ timeout, aborting
12
+ timeout, reset controller
13
+ timeout, disable controller
14
+ iotest: 1% high is too high compared to 1% low!
15
+ Hardware error
firewatcher-1.55/firewatcher.egg-info/PKG-INFO ADDED
@@ -0,0 +1,116 @@
1
+ Metadata-Version: 2.4
2
+ Name: firewatcher
3
+ Version: 1.55
4
+ Summary: Watch logs for incident patterns and persist surrounding context off-box
5
+ Home-page: https://github.com/yufei-pan/firewatcher
6
+ Author: Yufei Pan
7
+ Author-email: pan@zopyr.us
8
+ License: GPLv3+
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
11
+ Classifier: Operating System :: POSIX :: Linux
12
+ Classifier: Topic :: System :: Logging
13
+ Classifier: Topic :: System :: Monitoring
14
+ Requires-Python: >=3.6
15
+ Description-Content-Type: text/markdown
16
+ Dynamic: author
17
+ Dynamic: author-email
18
+ Dynamic: classifier
19
+ Dynamic: description
20
+ Dynamic: description-content-type
21
+ Dynamic: home-page
22
+ Dynamic: license
23
+ Dynamic: requires-python
24
+ Dynamic: summary
25
+
26
+ # firewatcher
27
+
28
+ Watch a log stream (journald, syslog, or any file) for incident patterns, then persist a window of surrounding messages to a durable location — including a network filesystem — so the record survives even if the machine later dies.
29
+
30
+ This is the PyPI name for the internal `firewatch` tool (in use since 2023). The `firewatch` command remains as an alias. The PyPI name `firewatch` is a different, unrelated project.
31
+
32
+ The watcher prefers `journalctl --follow` when `journalctl` is on PATH. On hosts without journald it falls back to `/var/log/syslog`, then `/var/log/messages`, or `--log-file`. Running the daemon does not require systemd; only `--install-service` / `--uninstall-service` do.
33
+
34
+ ## Install
35
+
36
+ ```bash
37
+ pip install firewatcher
38
+ ```
39
+
40
+ From a clone of this repository:
41
+
42
+ ```bash
43
+ pip install .
44
+ ```
45
+
46
+ Requires **Python 3.6+**. No third-party runtime dependencies.
47
+
48
+ ## Quick start
49
+
50
+ ```bash
51
+ firewatcher /etc/firewatcher/patterns.d
52
+ firewatch -t 300 -o /var/log/captured_messages/ /etc/firewatcher/patterns.d
53
+ ```
54
+
55
+ A directory argument loads every non-hidden pattern file inside it. Files ending in `.regex` are compiled as regular expressions. Any other pattern file is treated as fnmatch (bash-like) substrings, with `*` added on both ends.
56
+
57
+ `--install-service` seeds `/etc/firewatcher/patterns.d` with example patterns when that directory is empty. The `examples/` directory in the source tree is not installed onto `PATH` by `pip install`; copy those files from a clone, or let `--install-service` write them.
58
+
59
+ On a match, `firewatcher` writes:
60
+
61
+ - a per-incident capture under `{output-folder}/{YYYY-MM}/…`
62
+ - a line in `{output-folder}/journal.log`
63
+
64
+ Live capture is written as lines arrive, so a copy can already be on another filesystem if the host then disappears.
65
+
66
+ ## systemd service
67
+
68
+ ```bash
69
+ sudo firewatcher --install-service
70
+ sudo firewatcher --print-unit
71
+ sudo firewatcher --uninstall-service
72
+ ```
73
+
74
+ `--install-service` writes `/etc/systemd/system/firewatcher.service`, creates the output directory, seeds `/etc/firewatcher/patterns.d/` with example patterns **only if that directory is empty**, then `systemctl daemon-reload && systemctl enable --now firewatcher`.
75
+
76
+ If systemd is not available, `--install-service` and `--uninstall-service` print a warning and exit. `--print-unit` still works so you can copy the unit elsewhere.
77
+
78
+ Useful flags:
79
+
80
+ | Flag | Description |
81
+ |------|-------------|
82
+ | `--unit-name NAME` | Unit name (default: `firewatcher`) |
83
+ | `--requires-mounts-for PATH` | Add `RequiresMountsFor=` (repeat for NFS/remote output) |
84
+ | `--no-enable` | Write the unit and reload, but do not enable or start it |
85
+
86
+ ```bash
87
+ sudo firewatcher --install-service -o /mnt/logs/captured_messages \
88
+ --requires-mounts-for /mnt/logs/captured_messages \
89
+ /etc/firewatcher/patterns.d
90
+ ```
91
+
92
+ ## Options
93
+
94
+ | Flag | Description |
95
+ |------|-------------|
96
+ | `pattern_file …` | Pattern files or directories |
97
+ | `--log-file` | Log file to follow (default: journalctl / syslog / messages) |
98
+ | `-t`, `--capture-time` | Seconds of context after a match (default: 300) |
99
+ | `-o`, `--output-folder` | Where to write captures (default: `/var/log/captured_messages/`) |
100
+ | `--tail_lines` | Lines to start with (default: 20; `+N` from line N, not with journalctl) |
101
+ | `--filter_only` | Filter existing logs only (`cat` instead of `tail -F`, or `journalctl` without `--follow`) |
102
+ | `--compress-after-months` | Tar.xz monthly dirs after N months (default: 3) |
103
+ | `--delete-after-months` | Delete `YYYY-MM` dirs after N months (default: 0 = never) |
104
+ | `--capture_line_count_max` | Max lines before/after a match (default: 10000) |
105
+ | `--install-service` | Install and enable a systemd unit |
106
+ | `--print-unit` | Print that unit and exit |
107
+ | `--uninstall-service` | Disable and remove the unit |
108
+ | `-V`, `--version` | Show version and exit |
109
+
110
+ ## Author
111
+
112
+ Yufei Pan (pan@zopyr.us)
113
+
114
+ ## License
115
+
116
+ GPL-3.0-or-later
firewatcher-1.55/firewatcher.egg-info/SOURCES.txt ADDED
@@ -0,0 +1,12 @@
1
+ MANIFEST.in
2
+ README.md
3
+ firewatcher.py
4
+ setup.py
5
+ test_firewatcher.py
6
+ examples/nvme_failure.regex
7
+ examples/sys_msg.txt
8
+ firewatcher.egg-info/PKG-INFO
9
+ firewatcher.egg-info/SOURCES.txt
10
+ firewatcher.egg-info/dependency_links.txt
11
+ firewatcher.egg-info/entry_points.txt
12
+ firewatcher.egg-info/top_level.txt
firewatcher-1.55/firewatcher.egg-info/entry_points.txt ADDED
@@ -0,0 +1,3 @@
1
+ [console_scripts]
2
+ firewatch = firewatcher:main
3
+ firewatcher = firewatcher:main
firewatcher-1.55/firewatcher.egg-info/top_level.txt ADDED
@@ -0,0 +1 @@
1
+ firewatcher