codefilterph-logcollector 0.1.1__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. codefilterph_logcollector-0.1.1.dist-info/METADATA +56 -0
  2. codefilterph_logcollector-0.1.1.dist-info/RECORD +55 -0
  3. codefilterph_logcollector-0.1.1.dist-info/WHEEL +5 -0
  4. codefilterph_logcollector-0.1.1.dist-info/licenses/LICENSE +21 -0
  5. codefilterph_logcollector-0.1.1.dist-info/top_level.txt +2 -0
  6. crowdsec_integration/__init__.py +1 -0
  7. crowdsec_integration/apps.py +6 -0
  8. crowdsec_integration/migrations/0001_initial.py +22 -0
  9. crowdsec_integration/migrations/__init__.py +1 -0
  10. crowdsec_integration/models.py +14 -0
  11. crowdsec_integration/services.py +70 -0
  12. log_collector/__init__.py +1 -0
  13. log_collector/apps.py +13 -0
  14. log_collector/consumers.py +56 -0
  15. log_collector/docs/api.md +27 -0
  16. log_collector/docs/configuration.md +82 -0
  17. log_collector/docs/installation.md +40 -0
  18. log_collector/docs/operations-guide.md +52 -0
  19. log_collector/forms.py +160 -0
  20. log_collector/integrations.py +54 -0
  21. log_collector/management/__init__.py +0 -0
  22. log_collector/management/commands/__init__.py +0 -0
  23. log_collector/management/commands/backfill_security_incidents.py +30 -0
  24. log_collector/management/commands/manage_security_block.py +58 -0
  25. log_collector/management/commands/retry_security_sms.py +33 -0
  26. log_collector/middleware.py +57 -0
  27. log_collector/migrations/0001_initial.py +127 -0
  28. log_collector/migrations/0002_remove_logcollectorentity_allowed_api_key_names_and_more.py +67 -0
  29. log_collector/migrations/0003_blockedipaddress_securityincident_ipblockauditevent_and_more.py +169 -0
  30. log_collector/migrations/0004_app_appearance.py +31 -0
  31. log_collector/migrations/__init__.py +1 -0
  32. log_collector/models.py +601 -0
  33. log_collector/permissions.py +283 -0
  34. log_collector/routing.py +76 -0
  35. log_collector/security_events.py +35 -0
  36. log_collector/services/__init__.py +1 -0
  37. log_collector/services/blocking.py +309 -0
  38. log_collector/services/client_ip.py +75 -0
  39. log_collector/services/incidents.py +326 -0
  40. log_collector/services/nginx.py +54 -0
  41. log_collector/services/remote.py +32 -0
  42. log_collector/services/response.py +67 -0
  43. log_collector/services/security.py +379 -0
  44. log_collector/services/security_sms.py +98 -0
  45. log_collector/services/sync.py +103 -0
  46. log_collector/services/trusted_ips.py +55 -0
  47. log_collector/static/log_collector/appearance.css +46 -0
  48. log_collector/tasks.py +261 -0
  49. log_collector/templates/log_collector/dashboard.html +1544 -0
  50. log_collector/templates/log_collector/entity_form.html +45 -0
  51. log_collector/templates/log_collector/entity_manage.html +425 -0
  52. log_collector/templates/log_collector/entity_selection.html +306 -0
  53. log_collector/templates/log_collector/security_report.html +99 -0
  54. log_collector/urls.py +67 -0
  55. log_collector/views.py +1889 -0
@@ -0,0 +1,56 @@
1
+ Metadata-Version: 2.4
2
+ Name: codefilterph-logcollector
3
+ Version: 0.1.1
4
+ Summary: Django Nginx log collection, security monitoring, and remote collector dashboards
5
+ Author: CodefilterPH | Euge
6
+ License-Expression: MIT
7
+ Project-URL: Documentation, https://github.com/codefilterPH/logcollector/tree/main/log_collector/docs
8
+ Project-URL: Source, https://github.com/codefilterPH/logcollector
9
+ Classifier: Framework :: Django
10
+ Classifier: Programming Language :: Python :: 3.12
11
+ Classifier: Programming Language :: Python :: 3.13
12
+ Classifier: Topic :: System :: Logging
13
+ Requires-Python: <3.14,>=3.12
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: Django<6.1,>=6.0
17
+ Requires-Dist: channels<5,>=4.2
18
+ Requires-Dist: celery<6,>=5.5
19
+ Requires-Dist: djangorestframework<4,>=3.16
20
+ Requires-Dist: requests<3,>=2.32
21
+ Requires-Dist: cryptography<47,>=44
22
+ Provides-Extra: redis
23
+ Requires-Dist: channels-redis<5,>=4.2; extra == "redis"
24
+ Provides-Extra: asgi
25
+ Requires-Dist: daphne<5,>=4.2; extra == "asgi"
26
+ Dynamic: license-file
27
+
28
+ # LOG Collector
29
+
30
+ Developed by: CodefilterPH | Euge
31
+
32
+ LOG Collector is a Django application for local Nginx access-log collection,
33
+ security incidents, API access, WebSocket updates, Celery processing, and
34
+ read-only connections to other LOG Collector installations. CrowdSec connection
35
+ management is included in the distribution.
36
+
37
+ Install with `pip install codefilterph-logcollector`. Add `log_collector` and
38
+ `crowdsec_integration` to `INSTALLED_APPS`, mount `log_collector.urls`, and run
39
+ migrations. Nginx file access, ASGI, a channel layer, and Celery services are
40
+ configured by the host project; installing the wheel does not provision them.
41
+
42
+ The developer guides are installed in `log_collector/docs/` inside the Python
43
+ package and are also available in the source repository:
44
+
45
+ - [Installation](https://github.com/codefilterPH/logcollector/blob/main/log_collector/docs/installation.md)
46
+ - [Configuration](https://github.com/codefilterPH/logcollector/blob/main/log_collector/docs/configuration.md)
47
+ - [API integration](https://github.com/codefilterPH/logcollector/blob/main/log_collector/docs/api.md)
48
+ - [Operations](https://github.com/codefilterPH/logcollector/blob/main/log_collector/docs/operations-guide.md)
49
+
50
+ The app title defaults to **LOG Collector**. A Django superuser can set the
51
+ displayed title and shared theme on the hosting entity Manage page. The developer
52
+ credit is fixed in the application and cannot be edited through its settings.
53
+
54
+ License: [MIT](LICENSE).
55
+
56
+ Release owner: [first-release account setup](https://github.com/codefilterPH/logcollector/blob/main/RELEASE_SETUP.md).
@@ -0,0 +1,55 @@
1
+ codefilterph_logcollector-0.1.1.dist-info/licenses/LICENSE,sha256=chffrjMT2T1CXDMOfiCwn8vsuqK7RiE2seq-T9y2de8,1076
2
+ crowdsec_integration/__init__.py,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
3
+ crowdsec_integration/apps.py,sha256=wWdHYhnPUIWY0Hi-bhlau0dcbVKanspW3yFDWc8vEXA,171
4
+ crowdsec_integration/models.py,sha256=szYmtSg3PrLIiG9n_MefUP0ZaM-Uu-2-cdOGue8kbNc,465
5
+ crowdsec_integration/services.py,sha256=_Ut0nYvielvj3il9cj47S2CI7zc8DHKZXpsWCg0kRvQ,2463
6
+ crowdsec_integration/migrations/0001_initial.py,sha256=zbm074Cr9eCGt1GQwfdCN-_eWW2yJi7D9cPmJUeiA_g,856
7
+ crowdsec_integration/migrations/__init__.py,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
8
+ log_collector/__init__.py,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
9
+ log_collector/apps.py,sha256=AdWJAgeU6DKycMnWSKejH52l_XpbL7t4d57Pfwen9KI,342
10
+ log_collector/consumers.py,sha256=A4AO91tAsNOBmf3tCwzH9BZiLRD1fj7GFkL5KJw5F0w,2410
11
+ log_collector/forms.py,sha256=uDrys5qc5ucM66MLLDK_vEPzeW1jUOYRm-kw0szB5NU,5737
12
+ log_collector/integrations.py,sha256=HcbYzb42uqKmTZ6fiimbU9JEJ-AeNzYvou4B2_N3oPI,1765
13
+ log_collector/middleware.py,sha256=_R52ieXiNCRjBzPVd0ZhwtNJ1p1H5CpENYmeasG_EGk,1998
14
+ log_collector/models.py,sha256=w_9Xon3jx9t68UeOCdi0mc0cZ5j3f2xDrV0tMWvN_4g,21577
15
+ log_collector/permissions.py,sha256=H3Tyt-bcSYgjFfjiYl3AjACVteVy2R4NaZ5If21_OSQ,9518
16
+ log_collector/routing.py,sha256=o95rg-lnEvS4dgGN5svxV-2JiCwJ5ka1yYg6RMIjahY,4199
17
+ log_collector/security_events.py,sha256=TbOsS58Gzpkk1cvgCfNik-EwjWZ9MKw5wK3OD0Z5was,1610
18
+ log_collector/tasks.py,sha256=wVbsbbkSjszdJbaWVF9IGiqLFlkLrdT7K4eYtrDJby4,10494
19
+ log_collector/urls.py,sha256=GVporqY0Eg3PGJ7whjiUMXvRwUlLyStGaSySyslGqWY,3114
20
+ log_collector/views.py,sha256=iQF_zSv_I0vIax7mb3zEFZpJ9aRGntCHTaeY1AYjpcw,81382
21
+ log_collector/docs/api.md,sha256=ylUXNCPFVzVkAfbXCH3FsAVdbjTPZ6pOiQ8J_0LPycg,1593
22
+ log_collector/docs/configuration.md,sha256=5KsaSW2EtuYcATgVx_Iens_xBIABWtyjSOI5ZdOSsd8,3761
23
+ log_collector/docs/installation.md,sha256=MdOg_mVxcRmAXizn-Ab1TuKyJclPOHUMPHaHmBxoEJQ,1460
24
+ log_collector/docs/operations-guide.md,sha256=i1Ib_c07T1uOjvf7vpPnDiqGN54qTja2kjl-vogsytg,3118
25
+ log_collector/management/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
26
+ log_collector/management/commands/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
27
+ log_collector/management/commands/backfill_security_incidents.py,sha256=N_45cF5QMjkTrst7fNYD3nmYVRULg8bi6rQR6CQah_I,1492
28
+ log_collector/management/commands/manage_security_block.py,sha256=ReW78tWLjMsJl2dvPC7tjadHtCNerp-T1xLegjpK7Qg,2833
29
+ log_collector/management/commands/retry_security_sms.py,sha256=Gyd3rDGGGKxfSs1T-VyWQWDUbwkhjRdbQ67ohMFeUTg,1363
30
+ log_collector/migrations/0001_initial.py,sha256=kY_ocOXBxvm149C3LI8MaRHk7HUUEABtYgKFzG40Lps,7415
31
+ log_collector/migrations/0002_remove_logcollectorentity_allowed_api_key_names_and_more.py,sha256=XR4EX-mbg5P3hiTHP8j3BONCD_2U37JF-Qd-dx8tlLI,4054
32
+ log_collector/migrations/0003_blockedipaddress_securityincident_ipblockauditevent_and_more.py,sha256=gOuFGdShB2ZwA8rt-xAG7U5oeMND080OnrFEm9qTqTE,11466
33
+ log_collector/migrations/0004_app_appearance.py,sha256=zq1MZjPGhvkOi3Kkxy9I5aT5yPChfLs4Q9V4d5wxJkM,947
34
+ log_collector/migrations/__init__.py,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
35
+ log_collector/services/__init__.py,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
36
+ log_collector/services/blocking.py,sha256=jL9OvyK-Xs6gdV-3xlS1WuUitkFjWbaA2L4REwk-Jyg,15867
37
+ log_collector/services/client_ip.py,sha256=SGd-XtjiapVBlkDBJJEuUvvDDICwcIuSX3KPIwmURJ4,2463
38
+ log_collector/services/incidents.py,sha256=MJPDREA9FGVfE0h3Gk27hz862pET__ZpKWY_5FVIbLQ,16539
39
+ log_collector/services/nginx.py,sha256=3vt2bIIfA6Xf_p7iV3okqoAd2KowFxUc4PWGlTgHiK4,1727
40
+ log_collector/services/remote.py,sha256=f55fgBSESu6wbg_cXuB0fghQzg2KkVcznIgNIvYZWzQ,1030
41
+ log_collector/services/response.py,sha256=HwQ-FSaGf1d67KIMGlhdlmbNQ-fctRfl9jMHgYL5skw,3449
42
+ log_collector/services/security.py,sha256=eegvTfLRJp2pL2ZDQ412y_5W7SlIVbgUPLRWU-JnXqw,15962
43
+ log_collector/services/security_sms.py,sha256=q2rdnP8maLrnjcxNEbZzKfIqtSy6AfBYaXUbQmJCK5c,3545
44
+ log_collector/services/sync.py,sha256=-o3fh5w9QIrjpnGgmfolKLT-Q_U8i5QqZrK1NDbCwUc,4187
45
+ log_collector/services/trusted_ips.py,sha256=Y0HKq4O7M3_bq6r6yGNuXNOqzP0evBkrlZMJXrXmARw,1717
46
+ log_collector/static/log_collector/appearance.css,sha256=ZqoaaK0yg2zKuWBGM7SKBCRHUsqIlWcQ5kbkIjNZbZU,1948
47
+ log_collector/templates/log_collector/dashboard.html,sha256=5kARivEs_S5nXPgiOd-eMjxeyWfCrpxUxFbSO23fWI4,76757
48
+ log_collector/templates/log_collector/entity_form.html,sha256=L7dzHqUgty8fkoQevLVhf08xah0ucdsHT3wcuNlK4TU,2648
49
+ log_collector/templates/log_collector/entity_manage.html,sha256=ZJTHtc5SJhNAtXEhCzmqblkH2k_3KA1tSI66D_Tf9AA,21626
50
+ log_collector/templates/log_collector/entity_selection.html,sha256=o4w3V-GFtpimIVwmDxsIP4MNxsXahDtyQccc1Q8F3C0,8433
51
+ log_collector/templates/log_collector/security_report.html,sha256=cGMMMRGpynd4LEMorQodpCLZXg_Br-BIBTJcSX1xHBQ,7114
52
+ codefilterph_logcollector-0.1.1.dist-info/METADATA,sha256=kNe3JdRHv4bAdAlF_KmjNvSXwjSbMolaJYQqKtWSEc8,2613
53
+ codefilterph_logcollector-0.1.1.dist-info/WHEEL,sha256=mOdXR9C3wMVov7cSY8PY-XCzo3SmmWCHvGQr5csJoao,91
54
+ codefilterph_logcollector-0.1.1.dist-info/top_level.txt,sha256=9w5wgE9-NeX0_DLqK78SHfg4OJrBFCjn6li-h0TiyvQ,35
55
+ codefilterph_logcollector-0.1.1.dist-info/RECORD,,
@@ -0,0 +1,5 @@
1
+ Wheel-Version: 1.0
2
+ Generator: setuptools (84.2.1)
3
+ Root-Is-Purelib: true
4
+ Tag: py3-none-any
5
+
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 CodefilterPH | Euge
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,2 @@
1
+ crowdsec_integration
2
+ log_collector
@@ -0,0 +1 @@
1
+
@@ -0,0 +1,6 @@
1
+ from django.apps import AppConfig
2
+
3
+
4
+ class CrowdSecIntegrationConfig(AppConfig):
5
+ default_auto_field = "django.db.models.BigAutoField"
6
+ name = "crowdsec_integration"
@@ -0,0 +1,22 @@
1
+ """Initial CrowdSec connection schema for standalone installations."""
2
+
3
+ from django.db import migrations, models
4
+
5
+
6
+ class Migration(migrations.Migration):
7
+ initial = True
8
+ dependencies = []
9
+ operations = [
10
+ migrations.CreateModel(
11
+ name="CrowdSecConnection",
12
+ fields=[
13
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
14
+ ("entity_id", models.PositiveBigIntegerField(db_index=True, unique=True)),
15
+ ("api_url", models.URLField(max_length=500)),
16
+ ("machine_id", models.CharField(max_length=255)),
17
+ ("encrypted_password", models.TextField()),
18
+ ("updated_at", models.DateTimeField(auto_now=True)),
19
+ ],
20
+ options={"ordering": ("-updated_at",)},
21
+ ),
22
+ ]
@@ -0,0 +1,14 @@
1
+ from django.db import models
2
+
3
+
4
+ class CrowdSecConnection(models.Model):
5
+ """CrowdSec LAPI credentials for the local Log Collector entity."""
6
+
7
+ entity_id = models.PositiveBigIntegerField(unique=True, db_index=True)
8
+ api_url = models.URLField(max_length=500)
9
+ machine_id = models.CharField(max_length=255)
10
+ encrypted_password = models.TextField()
11
+ updated_at = models.DateTimeField(auto_now=True)
12
+
13
+ class Meta:
14
+ ordering = ("-updated_at",)
@@ -0,0 +1,70 @@
1
+ import base64
2
+ import hashlib
3
+ from urllib.parse import urlsplit
4
+
5
+ import requests
6
+ from cryptography.fernet import Fernet, InvalidToken
7
+ from django.conf import settings
8
+
9
+ from .models import CrowdSecConnection
10
+
11
+
12
+ class CrowdSecError(Exception):
13
+ pass
14
+
15
+
16
+ def _fernet():
17
+ key = base64.urlsafe_b64encode(hashlib.sha256(settings.SECRET_KEY.encode()).digest())
18
+ return Fernet(key)
19
+
20
+
21
+ def encrypt_password(password):
22
+ return _fernet().encrypt(password.encode()).decode()
23
+
24
+
25
+ def _password(connection):
26
+ try:
27
+ return _fernet().decrypt(connection.encrypted_password.encode()).decode()
28
+ except (InvalidToken, ValueError) as exc:
29
+ raise CrowdSecError("Saved credential cannot be decrypted. Re-enter the CrowdSec credential.") from exc
30
+
31
+
32
+ def _url(api_url, path):
33
+ parsed = urlsplit(api_url)
34
+ if parsed.scheme not in {"https", "http"} or not parsed.hostname or parsed.username or parsed.password:
35
+ raise CrowdSecError("Enter a valid CrowdSec Local API URL.")
36
+ if parsed.scheme != "https" and parsed.hostname not in {"localhost", "127.0.0.1", "::1"}:
37
+ raise CrowdSecError("CrowdSec connections must use HTTPS except for loopback development.")
38
+ return f"{api_url.rstrip('/')}/{path.lstrip('/')}"
39
+
40
+
41
+ def request(connection, method, path, **kwargs):
42
+ try:
43
+ login = requests.post(
44
+ _url(connection.api_url, "v1/watchers/login"),
45
+ json={"machine_id": connection.machine_id, "password": _password(connection)},
46
+ timeout=8,
47
+ )
48
+ login.raise_for_status()
49
+ token = login.json().get("token")
50
+ if not token:
51
+ raise CrowdSecError("CrowdSec returned no session token.")
52
+ response = requests.request(
53
+ method, _url(connection.api_url, path),
54
+ headers={"Authorization": f"Bearer {token}"}, timeout=8, **kwargs,
55
+ )
56
+ response.raise_for_status()
57
+ return response.json() if response.content else {}
58
+ except CrowdSecError:
59
+ raise
60
+ except (requests.RequestException, ValueError) as exc:
61
+ raise CrowdSecError("CrowdSec connection failed. Check the URL, credentials, and LAPI permissions.") from exc
62
+
63
+
64
+ def validate_connection(api_url, machine_id, password):
65
+ class Candidate:
66
+ pass
67
+ candidate = Candidate()
68
+ candidate.api_url, candidate.machine_id = api_url, machine_id
69
+ candidate.encrypted_password = encrypt_password(password)
70
+ request(candidate, "GET", "v1/alerts?limit=1")
@@ -0,0 +1 @@
1
+
log_collector/apps.py ADDED
@@ -0,0 +1,13 @@
1
+ from django.apps import AppConfig
2
+
3
+
4
+ class LogCollectorConfig(AppConfig):
5
+ default_auto_field = "django.db.models.BigAutoField"
6
+ name = "log_collector"
7
+
8
+ def ready(self):
9
+ from django.core.checks import register
10
+
11
+ from log_collector.services.client_ip import check_security_cidrs
12
+
13
+ register(check_security_cidrs)
@@ -0,0 +1,56 @@
1
+ """Authenticated, read-only security incident and block notifications."""
2
+
3
+ import json
4
+ from urllib.parse import parse_qsl
5
+
6
+ from channels.db import database_sync_to_async
7
+ from channels.generic.websocket import AsyncWebsocketConsumer
8
+
9
+ from log_collector.models import LogCollectorEntity
10
+ from log_collector.security_events import sanitize_security_event, security_group_name
11
+
12
+
13
+ class SecurityEventsConsumer(AsyncWebsocketConsumer):
14
+ async def connect(self):
15
+ self.security_group = None
16
+ user = self.scope.get("user")
17
+ credentials = {"token", "api_key", "api_token", "access_token", "authorization", "key"}
18
+ query = self.scope.get("query_string", b"").decode("utf-8", errors="replace")
19
+ if (not user or not user.is_authenticated or not user.is_active
20
+ or any(key.lower() in credentials for key, _ in parse_qsl(query, keep_blank_values=True))):
21
+ await self.close(code=4401)
22
+ return
23
+ if not await self.is_data_center_member(user):
24
+ await self.close(code=4403)
25
+ return
26
+ self.entity_id = await self.active_entity_id(self.scope["url_route"]["kwargs"]["slug"])
27
+ if self.entity_id is None:
28
+ await self.close(code=4404)
29
+ return
30
+ self.security_group = security_group_name(self.entity_id)
31
+ await self.channel_layer.group_add(self.security_group, self.channel_name)
32
+ await self.accept()
33
+
34
+ @database_sync_to_async
35
+ def is_data_center_member(self, user):
36
+ return user.groups.filter(name="DATA_CENTER").exists()
37
+
38
+ @database_sync_to_async
39
+ def active_entity_id(self, slug):
40
+ return LogCollectorEntity.objects.filter(slug=slug, is_active=True).values_list("pk", flat=True).first()
41
+
42
+ async def disconnect(self, close_code):
43
+ if self.security_group:
44
+ await self.channel_layer.group_discard(self.security_group, self.channel_name)
45
+
46
+ async def receive(self, text_data=None, bytes_data=None):
47
+ # No client protocol: text, JSON actions and binary input have no effects.
48
+ pass
49
+
50
+ async def security_event(self, message):
51
+ payload = message.get("event", {})
52
+ if payload.get("entity_id") != self.entity_id:
53
+ return
54
+ event = sanitize_security_event(self.entity_id, payload.get("type"), payload)
55
+ if event is not None:
56
+ await self.send(text_data=json.dumps(event))
@@ -0,0 +1,27 @@
1
+ # API integration
2
+
3
+ Mounting `log_collector.urls` at `/log-collector/` exposes read-only entity
4
+ resources under `/log-collector/api/entities/<slug>/`. Resources include
5
+ `metrics/`, `access-logs/`, `geolocation/`, `user-sessions/`, `endpoints/`, and
6
+ `audit-events/`, subject to the entity's feature flags and caller permissions.
7
+ Table resources support pagination; supported resources accept `export=csv`.
8
+
9
+ Generate an entity-scoped collector token as a superuser on the entity Manage
10
+ page. The raw token is shown once. Server-to-server callers send it in the
11
+ `X-Api-Key` header. The server stores a hash, not the raw token. Tokens may
12
+ have expiry, caller IP/CIDR, and origin restrictions. Do not embed a collector
13
+ token in browser or mobile application code.
14
+
15
+ To view another installation, create an integration entity with its HTTPS base
16
+ URL, remote entity slug, and remote collector token. Remote calls are read-only
17
+ and fetched on demand; they do not copy remote raw logs into the local database.
18
+ The remote service must expose the LOG Collector API. Arbitrary logging APIs
19
+ need a separate adapter.
20
+
21
+ The available entity routes also include `security-summary/`,
22
+ `security-incidents/`, `security-blocks/`, and CrowdSec status and decisions.
23
+ State-changing security endpoints require an authorized same-origin session,
24
+ CSRF protection, and the required operator role; a read-only API token cannot
25
+ perform those actions. An expired, revoked, mismatched, or caller-restricted
26
+ token is denied. Inspect recent token usage on the entity Manage page to
27
+ diagnose denials without logging the raw key.
@@ -0,0 +1,82 @@
1
+ # Configuration
2
+
3
+ ## Nginx collection
4
+
5
+ Set `LOG_COLLECTOR_NGINX_ACCESS_LOG_PATHS` to the Nginx access-log files that
6
+ the Django and worker processes can read. The default path is
7
+ `/var/log/nginx/access.log`; set `[]` when no local file is available. The
8
+ reader accepts Nginx common and combined access-log entries. Configure the
9
+ hosting entity's paths, cooldown, maximum lines per sync, and retention on its
10
+ Manage page. Access-log rotation must leave new files readable by workers.
11
+
12
+ ## ASGI and WebSockets
13
+
14
+ Set `ASGI_APPLICATION` and a production `CHANNEL_LAYERS` backend, such as
15
+ `channels_redis`, in the host project. Include the collector route under the
16
+ host's authenticated ASGI WebSocket application:
17
+
18
+ ```python
19
+ from channels.auth import AuthMiddlewareStack
20
+ from channels.routing import ProtocolTypeRouter, URLRouter
21
+ from django.core.asgi import get_asgi_application
22
+
23
+ django_asgi_app = get_asgi_application()
24
+ from log_collector.routing import websocket_urlpatterns
25
+
26
+ application = ProtocolTypeRouter({
27
+ "http": django_asgi_app,
28
+ "websocket": AuthMiddlewareStack(URLRouter(websocket_urlpatterns)),
29
+ })
30
+ ```
31
+
32
+ The route is `/ws/log-collector/entities/<slug>/security/`. Its origin
33
+ validator is part of the route. Set the host's `ALLOWED_HOSTS` correctly.
34
+ The browser Origin must match the WebSocket Host and port; when
35
+ `LOG_COLLECTOR_SECURITY_WEBSOCKET_ALLOWED_ORIGINS` is set, that exact origin
36
+ must also appear in the list.
37
+
38
+ ## Celery
39
+
40
+ Configure a Celery broker, worker, and beat scheduler in the host project.
41
+ The default queue handles `log_collector.tasks.collect_analyze_and_respond`,
42
+ `expire_security_blocks`, and `retry_security_edge_actions`. Schedule the
43
+ first task at the desired polling interval (five seconds in the original
44
+ deployment) and the latter two at one-minute intervals. An `sms` queue worker
45
+ is required when security SMS is enabled; route `queue_security_incident_sms_alert`,
46
+ `send_security_incident_sms_chunk`, and `retry_security_incident_sms_alert` there.
47
+ Collection can also run on dashboard/API requests, but continuous collection
48
+ requires beat and a worker.
49
+
50
+ ## Optional host integrations
51
+
52
+ The local geolocation and user-session APIs need models supplied by the host:
53
+
54
+ ```python
55
+ LOG_COLLECTOR_GEOLOCATION_MODEL = "your_geo_app.UserLocation"
56
+ LOG_COLLECTOR_USER_SESSION_MODEL = "your_sessions_app.UserSession"
57
+ ```
58
+
59
+ The geolocation model must expose `user`, `captured_at`, `ip_address`,
60
+ `latitude`, `longitude`, `country`, and `city`; the session model must expose
61
+ `user`, `session_key`, `ip`, `created_at`, `last_seen_at`, and `user_agent`.
62
+ Disable the entity's geolocation or user-session feature when no compatible
63
+ model exists. With an enabled feature but no model configured, its local API
64
+ returns HTTP 503 naming the missing setting; security summaries still work
65
+ without session data. Remote entities use the remote collector API instead.
66
+
67
+
68
+ The package does not assume a particular user-profile or SMS provider. Set
69
+ the following to importable dotted Python callables when using those actions:
70
+
71
+ - `LOG_COLLECTOR_ACCOUNT_CONTAINMENT_FUNCTION`: accepts a user and returns an
72
+ object with `as_dict()` describing revoked authentication state.
73
+ - `LOG_COLLECTOR_SMS_RECIPIENTS_FUNCTION`: returns iterable
74
+ `SecuritySMSRecipient(user, phone_number)` values from
75
+ `log_collector.integrations`.
76
+ - `LOG_COLLECTOR_SMS_SENDER_FUNCTION`: accepts `mobile_number`, `message`, and
77
+ `context` keyword arguments.
78
+
79
+ `LOG_COLLECTOR_SECURITY_SMS_ENABLED` defaults to false. Enable it only after
80
+ the recipient and sender functions, provider, and `sms` worker are configured.
81
+ The optional Nginx edge-block helper is supplied and reviewed by the host;
82
+ the wheel does not install privileged executables. See [Operations](operations-guide.md).
@@ -0,0 +1,40 @@
1
+ # Installation
2
+
3
+ LOG Collector supports Python 3.12–3.13 and Django 6.0.x. Use a virtual
4
+ environment in the host Django project:
5
+
6
+ ```shell
7
+ python -m pip install 'codefilterph-logcollector[redis,asgi]'
8
+ ```
9
+
10
+ Add the apps and mount the HTTP URLs:
11
+
12
+ ```python
13
+ INSTALLED_APPS += [
14
+ "channels",
15
+ "rest_framework",
16
+ "crowdsec_integration.apps.CrowdSecIntegrationConfig",
17
+ "log_collector.apps.LogCollectorConfig",
18
+ ]
19
+
20
+ # In the project's urls.py:
21
+ urlpatterns += [path("log-collector/", include("log_collector.urls"))]
22
+ ```
23
+
24
+ Keep Django's `auth`, `contenttypes`, and `sessions` apps installed. Apply the
25
+ database schema with `python manage.py migrate`. The collector uses
26
+ `AUTH_USER_MODEL`, so configure a custom user model before its first migration
27
+ if your project needs one.
28
+
29
+ Add `log_collector.middleware.SecurityIPBlockMiddleware` after Django's
30
+ `AuthenticationMiddleware` if application-level IP blocking is wanted. Give
31
+ authorized dashboard users membership in the `DATA_CENTER` group. A user also
32
+ needs `is_superuser=True` to open the Manage page or change configuration.
33
+
34
+ The default hosting entity is created on first access and has slug
35
+ `hosting-app`. Open `/log-collector/entities/hosting-app/manage/` as a
36
+ superuser to set the displayed app name, shared theme, and local collection
37
+ paths. An unset name displays **LOG Collector**.
38
+
39
+ See [Configuration](configuration.md) before enabling collection, WebSockets,
40
+ Celery, SMS, or Nginx edge blocking.
@@ -0,0 +1,52 @@
1
+ # Operations
2
+
3
+ After installation, run `python manage.py check` and `python manage.py migrate`
4
+ with the host's settings. Confirm the web and Celery worker accounts can read
5
+ each configured Nginx log. Mount log files read-only where possible; grant the
6
+ smallest filesystem permission needed. Check the collector's Manage page for
7
+ local connection health and recent token usage.
8
+
9
+ Run an ASGI server and a channel layer for live security updates. Run Celery
10
+ beat and workers consuming `default`; add a worker consuming `sms` when SMS
11
+ alerts are enabled. A disconnected WebSocket does not stop HTTP refresh or
12
+ scheduled collection. Retain monitoring for worker failures, log rotation,
13
+ unreadable paths, and stale last-sync timestamps.
14
+
15
+ Application-level IP blocking is supplied by the optional collector middleware.
16
+ Edge blocking needs a separately installed and reviewed fixed-path helper in
17
+ `LOG_COLLECTOR_NGINX_BLOCK_HELPER`. Never configure a shell command or grant
18
+ the web account broad Nginx privileges. The helper must validate its inputs
19
+ and verify the resulting Nginx rule before reporting success.
20
+
21
+ Before upgrading, back up the database and apply the package migrations on a
22
+ staging copy. Keep the same Django app labels and migration names. Roll out
23
+ the new package version to web and worker processes together, then verify
24
+ collection, API reads, WebSocket connection, and queued tasks. Rotate API
25
+ tokens and SMS provider credentials through the host's normal secret process;
26
+ do not put secrets in package configuration files.
27
+
28
+ ## Publishing a release
29
+
30
+ Publish from the standalone `codefilterPH/logcollector` repository. Review its
31
+ source and archives for host-specific names, paths, credentials, and unrelated
32
+ apps. Confirm the public license and increment the version in `pyproject.toml`.
33
+ Check that the normalized `codefilterph-logcollector` name is still available on both
34
+ package indexes immediately before the first release.
35
+
36
+ In TestPyPI, register a Trusted Publisher for GitHub owner `codefilterPH`,
37
+ repository `logcollector`, workflow `publish.yml`, and environment `testpypi`.
38
+ Protect that GitHub environment. Tag the reviewed version as `vX.Y.Z`, then
39
+ run **Publish package** against that tag with target `testpypi`. Check the
40
+ uploaded files and README. In a fresh environment, install dependencies from
41
+ PyPI, then install the exact candidate from TestPyPI with
42
+ `python -m pip install --no-deps --index-url https://test.pypi.org/simple/ codefilterph-logcollector==X.Y.Z`.
43
+ Run the installation and configuration checks on that candidate.
44
+
45
+ For production, register a separate PyPI Trusted Publisher with the same
46
+ owner, repository, and workflow but environment `pypi`. Require review for
47
+ that GitHub environment. Run the same workflow against the reviewed tag with
48
+ target `pypi`. The workflow uses short-lived OIDC credentials; no PyPI API
49
+ token belongs in the repository. Verify `python -m pip install
50
+ codefilterph-logcollector==X.Y.Z` in a fresh environment, inspect the installed guides,
51
+ and record release notes. A package version cannot be overwritten on PyPI;
52
+ fixes require a new version and tag.