utilvo-edge-shield 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE +21 -0
- data/README.md +95 -0
- data/lib/utilvo_edge_shield.rb +33 -0
- metadata +49 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 78a483ac9d1d3ac64d44ef05734ba87e50579b3cf4ee90eb13e69316c45937ae
|
|
4
|
+
data.tar.gz: b868a75c32e5326b2793c1f09ded6f899bbb1aee83dabee26de43b3f06799279
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: bac2d2239144591c803258920ffd0611c6b571a5a6cfc0860b0f5cdb7b8605e7ce03d13cb9a0b32c4febce59b963c34a04f317cf9a4c412f6203b149c9a4e391
|
|
7
|
+
data.tar.gz: d0118c5318bcc06b3e7a2d926d36d9e98bc23aba8813d60d49deab07f804d64871af3942ff64cc1977629b8edfbc5376919734ae25aebff005c240cbcaefc6be
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Utilvo
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# utilvo-edge-shield
|
|
2
|
+
|
|
3
|
+
**Utilvo EdgeShield** (`v1.1.0`) is a zero-dependency, multi-language server firewall rule generator, CIDR/ASN threat intelligence library, Safe-Harbor bot classifier, and Google Analytics 4 (`gtag.js`) headless browser detector developed by **[Utilvo](https://utilvo.com/edge-shield)**.
|
|
4
|
+
|
|
5
|
+
- **Official Documentation, Interactive `access.log` Scanner & Rule Builder:** [https://utilvo.com/edge-shield](https://utilvo.com/edge-shield)
|
|
6
|
+
- **Self-Updating Live Threat Intelligence API (JSON):** [https://utilvo.com/api/edge-shield/threats](https://utilvo.com/api/edge-shield/threats)
|
|
7
|
+
- **Live CIDR Blocklist Feed (Fail2Ban / CrowdSec / UFW):** [https://utilvo.com/api/edge-shield/threats?format=cidr](https://utilvo.com/api/edge-shield/threats?format=cidr)
|
|
8
|
+
- **Zero-Conflict Nginx `http {}` Map Config:** [https://utilvo.com/edge-shield/nginx-bot-shield.conf](https://utilvo.com/edge-shield/nginx-bot-shield.conf)
|
|
9
|
+
- **Safe 1-Line Linux Nginx Installer (Auto-Backup & Daily Sync):** [https://utilvo.com/edge-shield/install.sh](https://utilvo.com/edge-shield/install.sh)
|
|
10
|
+
- **Client-Side GA4 WebGL Bot Guard:** [https://utilvo.com/edge-shield/ga4-bot-guard.min.js](https://utilvo.com/edge-shield/ga4-bot-guard.min.js)
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## Overview
|
|
15
|
+
|
|
16
|
+
Modern headless Chromium scrapers hosted on **Tencent Cloud (`AS132203`)**, **Alibaba Cloud (`AS45102`)**, and **Huawei Cloud (`AS136907`)** rotate outbound IPv4 addresses across `/10` and `/16` CIDR blocks (`43.128.0.0/10`, `47.74.0.0/16`, `101.32.0.0/15`) on every HTTP request, spoof legacy mobile User-Agents (`CPU iPhone OS 13_2_3`), hammer Next.js `? _rsc=` payloads, and pollute Google Analytics 4 with synthetic sessions.
|
|
17
|
+
|
|
18
|
+
`utilvo-edge-shield` provides programmatic request inspection, CIDR/ASN matching, and WAF rule generation while strictly whitelisting verified search engine crawlers (`Googlebot`, `Bingbot`, `Applebot`, `DuckDuckBot`, `YandexBot`).
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Installation
|
|
23
|
+
|
|
24
|
+
### Python (PyPI)
|
|
25
|
+
```bash
|
|
26
|
+
pip install utilvo-edge-shield
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
### Deno / TypeScript (JSR)
|
|
30
|
+
```bash
|
|
31
|
+
deno add jsr:@utilvo/edge-shield
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
### Rust (Crates.io)
|
|
35
|
+
```bash
|
|
36
|
+
cargo add utilvo-edge-shield
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
### Ruby (RubyGems)
|
|
40
|
+
```bash
|
|
41
|
+
gem install utilvo-edge-shield
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
### Elixir (Hex.pm)
|
|
45
|
+
```elixir
|
|
46
|
+
{:utilvo_edge_shield, "~> 1.1.0"}
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
### Dart / Flutter (Pub.dev)
|
|
50
|
+
```bash
|
|
51
|
+
dart pub add utilvo_edge_shield
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
### Lua / OpenResty / Nginx (LuaRocks)
|
|
55
|
+
```bash
|
|
56
|
+
luarocks install utilvo-edge-shield
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
### Go
|
|
60
|
+
```bash
|
|
61
|
+
go get github.com/utilvo/utilvo-edge-shield@v1.1.0
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
### 1-Line Linux / Nginx Automated Deployment
|
|
65
|
+
```bash
|
|
66
|
+
curl -sSL https://utilvo.com/edge-shield/install.sh | sudo bash
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## Quick Start
|
|
72
|
+
|
|
73
|
+
### Python
|
|
74
|
+
```python
|
|
75
|
+
from utilvo_edge_shield import inspect_request, generate_cloudflare_waf_expression, EDGE_SHIELD_FEEDS
|
|
76
|
+
|
|
77
|
+
verdict = inspect_request(
|
|
78
|
+
ip="43.173.182.197",
|
|
79
|
+
user_agent="Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X)"
|
|
80
|
+
)
|
|
81
|
+
print(verdict)
|
|
82
|
+
# {'is_bot': True, 'is_seo_crawler': False, 'reasons': ['CLOUD_SCRAPER_SUBNET', 'ROGUE_SCRAPER_USER_AGENT']}
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
### TypeScript / JavaScript
|
|
86
|
+
```ts
|
|
87
|
+
import { inspectServerRequest, generateCloudflareWafExpression, EDGE_SHIELD_FEEDS } from "jsr:@utilvo/edge-shield";
|
|
88
|
+
|
|
89
|
+
const res = inspectServerRequest("43.173.182.197", "HeadlessChrome/120.0");
|
|
90
|
+
console.log(res.isBot, res.reasons);
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## License
|
|
94
|
+
|
|
95
|
+
MIT © [Utilvo](https://utilvo.com/edge-shield)
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module UtilvoEdgeShield
|
|
4
|
+
VERSION = "1.1.0"
|
|
5
|
+
DOCS_URL = "https://utilvo.com/edge-shield"
|
|
6
|
+
THREAT_API_URL = "https://utilvo.com/api/edge-shield/threats"
|
|
7
|
+
CIDR_FEED_URL = "https://utilvo.com/edge-shield/ai-scrapers-cidr.txt"
|
|
8
|
+
|
|
9
|
+
SEO_BOT_RE = /(Googlebot|Google-InspectionTool|bingbot|adidxbot|MicrosoftPreview|Applebot|DuckDuckBot|YandexBot)/i
|
|
10
|
+
SUBNET_RE_1 = /^43\.(119|12[89]|1[3-6][0-9]|17[0-5])\./
|
|
11
|
+
SUBNET_RE_2 = /^(47\.(74|8[289])|8\.21[79]|101\.3[23]|119\.28|124\.156|129\.226|150\.109|170\.106|149\.232|159\.138|46\.250)\./
|
|
12
|
+
BAD_UA_RE = /(CPU iPhone OS 13_2_3|TokenFoundryBot|QlyzeBot|Bytespider|PetalBot|ClaudeBot|CCBot|GPTBot|HeadlessChrome)/i
|
|
13
|
+
|
|
14
|
+
def self.inspect_request(ip, user_agent)
|
|
15
|
+
ua = user_agent.to_s
|
|
16
|
+
return { is_bot: false, is_seo_crawler: true, reasons: [] } if ua.match?(SEO_BOT_RE)
|
|
17
|
+
|
|
18
|
+
clean_ip = ip.to_s.split(",").first.to_s.strip
|
|
19
|
+
reasons = []
|
|
20
|
+
reasons << "CLOUD_SCRAPER_SUBNET" if clean_ip.match?(SUBNET_RE_1) || clean_ip.match?(SUBNET_RE_2)
|
|
21
|
+
reasons << "ROGUE_SCRAPER_USER_AGENT" if ua.match?(BAD_UA_RE)
|
|
22
|
+
|
|
23
|
+
{
|
|
24
|
+
is_bot: !reasons.empty?,
|
|
25
|
+
is_seo_crawler: false,
|
|
26
|
+
reasons: reasons
|
|
27
|
+
}
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def self.generate_cloudflare_waf_expression
|
|
31
|
+
'(not cf.client.bot and (ip.src.asnum in {132203 45090 45102 37963 136907} or http.user_agent contains "CPU iPhone OS 13_2_3" or http.user_agent contains "TokenFoundryBot" or http.user_agent contains "QlyzeBot"))'
|
|
32
|
+
end
|
|
33
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: utilvo-edge-shield
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 1.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Utilvo
|
|
8
|
+
autorequire:
|
|
9
|
+
bindir: bin
|
|
10
|
+
cert_chain: []
|
|
11
|
+
date: 2026-10-20 00:00:00.000000000 Z
|
|
12
|
+
dependencies: []
|
|
13
|
+
description: Open-source server firewall generator, CIDR/ASN threat intelligence feed (AS132203, AS45102, AS136907), Safe-Harbor bot classifier, and GA4 headless bot detector.
|
|
14
|
+
email: utilvo.com@gmail.com
|
|
15
|
+
executables: []
|
|
16
|
+
extensions: []
|
|
17
|
+
extra_rdoc_files: []
|
|
18
|
+
files:
|
|
19
|
+
- lib/utilvo_edge_shield.rb
|
|
20
|
+
- README.md
|
|
21
|
+
- LICENSE
|
|
22
|
+
homepage: https://utilvo.com/edge-shield
|
|
23
|
+
licenses:
|
|
24
|
+
- MIT
|
|
25
|
+
metadata:
|
|
26
|
+
homepage_uri: https://utilvo.com/edge-shield
|
|
27
|
+
documentation_uri: https://utilvo.com/edge-shield
|
|
28
|
+
source_code_uri: https://github.com/utilvo/utilvo-edge-shield
|
|
29
|
+
bug_tracker_uri: https://github.com/utilvo/utilvo-edge-shield/issues
|
|
30
|
+
post_install_message:
|
|
31
|
+
rdoc_options: []
|
|
32
|
+
require_paths:
|
|
33
|
+
- lib
|
|
34
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
35
|
+
requirements:
|
|
36
|
+
- - ">="
|
|
37
|
+
- !ruby/object:Gem::Version
|
|
38
|
+
version: 2.7.0
|
|
39
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
40
|
+
requirements:
|
|
41
|
+
- - ">="
|
|
42
|
+
- !ruby/object:Gem::Version
|
|
43
|
+
version: '0'
|
|
44
|
+
requirements: []
|
|
45
|
+
rubygems_version: 3.4.20
|
|
46
|
+
signing_key:
|
|
47
|
+
specification_version: 4
|
|
48
|
+
summary: Server firewall generator, CIDR/ASN threat intelligence feed, and GA4 bot detector.
|
|
49
|
+
test_files: []
|