LightBPF 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- lightbpf-1.0.0/PKG-INFO +542 -0
- lightbpf-1.0.0/README.md +528 -0
- lightbpf-1.0.0/pyproject.toml +21 -0
- lightbpf-1.0.0/setup.cfg +4 -0
- lightbpf-1.0.0/src/LightBPF/Compiler.py +1300 -0
- lightbpf-1.0.0/src/LightBPF/Lexer.py +200 -0
- lightbpf-1.0.0/src/LightBPF/Parser.py +414 -0
- lightbpf-1.0.0/src/LightBPF/__init__.py +3 -0
- lightbpf-1.0.0/src/LightBPF.egg-info/PKG-INFO +542 -0
- lightbpf-1.0.0/src/LightBPF.egg-info/SOURCES.txt +10 -0
- lightbpf-1.0.0/src/LightBPF.egg-info/dependency_links.txt +1 -0
- lightbpf-1.0.0/src/LightBPF.egg-info/top_level.txt +1 -0
lightbpf-1.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,542 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: LightBPF
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: A flexible pure python cBPF Compiler
|
|
5
|
+
Author-email: Adam Boulaaz <adamboulaaz92@gmail.com>
|
|
6
|
+
License: MPL-2.0
|
|
7
|
+
Project-URL: Homepage, https://github.com/Light-Projects/LightBPF
|
|
8
|
+
Project-URL: Bug Tracker, https://github.com/Light-Projects/LightBPF/issues
|
|
9
|
+
Classifier: Programming Language :: Python :: 3
|
|
10
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Requires-Python: >=3.8
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
|
|
15
|
+
## LightBPF is part of LightPacket Library so for imports in this example you will find it always starts with LightPacket
|
|
16
|
+
|
|
17
|
+
# LightBPF: Pure-Python Berkeley Packet Filter (cBPF) Engine
|
|
18
|
+
|
|
19
|
+
**LightBPF** is a standalone, pure-Python Berkeley Packet Filter engine for packet sniffing, inspection, and network security applications. It tokenizes, parses, compiles, disassembles, and emulates classic BPF (cBPF / pcap-filter) bytecode with zero native C dependencies.
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
+------------------+ +------------------+ +----------------------+
|
|
23
|
+
| Filter Text | | Token Stream | | Abstract Syntax Tree |
|
|
24
|
+
| "tcp port 80" | -----> | [IP-PROTO, tcp], | -----> | KeywordFilterNode |
|
|
25
|
+
| | (Lexer) | [KEYWORD, port] | (Parser)| (port=80, tcp) |
|
|
26
|
+
+------------------+ +------------------+ +----------------------+
|
|
27
|
+
|
|
|
28
|
+
| (Compiler)
|
|
29
|
+
v
|
|
30
|
+
+----------------------+
|
|
31
|
+
| cBPF Machine Code |
|
|
32
|
+
| (List[Instruction]) |
|
|
33
|
+
+----------------------+
|
|
34
|
+
/ \
|
|
35
|
+
/ \
|
|
36
|
+
v v
|
|
37
|
+
+------------------+ +-------------------+
|
|
38
|
+
| Linux Raw Socket | | Pure-Python VM |
|
|
39
|
+
| SO_ATTACH_FILTER | | (BpfEmulator) |
|
|
40
|
+
+------------------+ +-------------------+
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Table of Contents
|
|
46
|
+
|
|
47
|
+
1. [Key Features](#key-features)
|
|
48
|
+
2. [Quick Start Guide](#quick-start-guide)
|
|
49
|
+
- [Compiling a Filter](#compiling-a-filter)
|
|
50
|
+
- [Offline Packet Filtering (BpfEmulator)](#offline-packet-filtering-bpfemulator)
|
|
51
|
+
- [Attaching to Linux Raw Sockets](#attaching-to-linux-raw-sockets)
|
|
52
|
+
- [Disassembling Bytecode](#disassembling-bytecode)
|
|
53
|
+
3. [Supported Filter Syntax Cheat Sheet](#supported-filter-syntax-cheat-sheet)
|
|
54
|
+
4. [Architecture & Pipeline](#architecture--pipeline)
|
|
55
|
+
- [The Classic BPF (cBPF) Virtual Machine](#the-classic-bpf-cbpf-virtual-machine)
|
|
56
|
+
- [Instruction Structure (struct bpf_insn)](#instruction-structure-struct-bpf_insn)
|
|
57
|
+
5. [Developer Deep-Dive: Stage 1 — Lexer (`Lexer.py`)](#developer-deep-dive-stage-1--lexer-lexerpy)
|
|
58
|
+
- [Tokenizer Mechanics](#tokenizer-mechanics)
|
|
59
|
+
- [Token Reference Table](#token-reference-table)
|
|
60
|
+
6. [Developer Deep-Dive: Stage 2 — Parser (`Parser.py`)](#developer-deep-dive-stage-2--parser-parserpy)
|
|
61
|
+
- [Grammar & Operator Precedence](#grammar--operator-precedence)
|
|
62
|
+
- [Abstract Syntax Tree (AST) Reference](#abstract-syntax-tree-ast-reference)
|
|
63
|
+
7. [Developer Deep-Dive: Stage 3 — Compiler (`Compiler.py`)](#developer-deep-dive-stage-3--compiler-compilerpy)
|
|
64
|
+
- [Short-Circuit Boolean Control Flow](#short-circuit-boolean-control-flow)
|
|
65
|
+
- [Relative Jump Resolution & Jump Threading](#relative-jump-resolution--jump-threading)
|
|
66
|
+
- [Register Spilling via Scratch Memory (M[0..15])](#register-spilling-via-scratch-memory-m015)
|
|
67
|
+
- [Dynamic IPv4 IHL Tracking (BPF_MSH)](#dynamic-ipv4-ihl-tracking-bpf_msh)
|
|
68
|
+
- [VLAN 802.1Q Handling](#vlan-8021q-handling)
|
|
69
|
+
- [Bitmask Optimization (BPF_JSET)](#bitmask-optimization-bpf_jset)
|
|
70
|
+
8. [Worked Examples & Disassembly Traces](#worked-examples--disassembly-traces)
|
|
71
|
+
- [Example 1: `tcp and port 80`](#example-1-tcp-and-port-80)
|
|
72
|
+
- [Example 2: `tcp[13] & 2 != 0` (TCP SYN)](#example-2-tcp13--2--0-tcp-syn)
|
|
73
|
+
- [Example 3: `host 192.168.1.1`](#example-3-host-19216811)
|
|
74
|
+
- [Example 4: `vlan 100`](#example-4-vlan-100)
|
|
75
|
+
- [Example 5: `(ip[0] & 0xf) * 4 > 20`](#example-5-ip0--0xf--4--20)
|
|
76
|
+
9. [Verification & Testing](#verification--testing)
|
|
77
|
+
10. [Opcode & Constant Reference](#opcode--constant-reference)
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## Key Features
|
|
82
|
+
|
|
83
|
+
- **100% Pure Python**: Runs everywhere without needing `libpcap`, `gcc`, or external tools installed.
|
|
84
|
+
- **Full cBPF Instruction Coverage**: Complete support for classes, addressing modes, arithmetic/bitwise operations, and jump branches.
|
|
85
|
+
- **Dual-Stack Networking**: Handles both IPv4 (with dynamic IHL header offsets) and IPv6 (multi-word 128-bit checks).
|
|
86
|
+
- **Rich Protocol & Filter Support**: Protocols (`tcp`, `udp`, `icmp`, `arp`, `vlan`, `mpls`), network filters (`host`, `net`, `port`, `portrange`), byte access (`tcp[13]`, `ip[2:2]`), and pseudo-fields (`tcpflags`, `icmptype`).
|
|
87
|
+
- **Scratch Memory Spilling**: Uses cBPF registers `M[0..15]` to compile nested mathematical expressions on both sides of comparisons.
|
|
88
|
+
- **Peephole Jump Optimization**: Automatically flattens jump chains via jump threading.
|
|
89
|
+
- **Integrated VM Emulator**: Execute BPF filters on raw byte packets in unit tests or user-space applications without root privileges.
|
|
90
|
+
- **Standard Serialization**: Export to Linux kernel `struct sock_fprog` or 8-byte raw machine instruction streams.
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## Quick Start Guide
|
|
95
|
+
|
|
96
|
+
### Compiling a Filter
|
|
97
|
+
|
|
98
|
+
```python
|
|
99
|
+
from LightPacket.LightBPF.Compiler import BpfCompiler
|
|
100
|
+
|
|
101
|
+
# Compile a classic filter string directly into cBPF bytecode
|
|
102
|
+
instructions = BpfCompiler.compile_str("tcp port 80 and src host 192.168.1.10")
|
|
103
|
+
|
|
104
|
+
print(f"Compiled into {len(instructions)} BPF instructions:")
|
|
105
|
+
for i, insn in enumerate(instructions):
|
|
106
|
+
print(f" [{i:02d}] {insn}")
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
### Offline Packet Filtering (`BpfEmulator`)
|
|
110
|
+
|
|
111
|
+
You can execute filters directly on raw bytes in Python:
|
|
112
|
+
|
|
113
|
+
```python
|
|
114
|
+
from LightPacket.LightBPF.Compiler import BpfCompiler, BpfEmulator
|
|
115
|
+
|
|
116
|
+
filter_code = BpfCompiler.compile_str("tcp[tcpflags] & tcp-syn != 0")
|
|
117
|
+
|
|
118
|
+
raw_packet = b"..." # raw Ethernet frame bytes
|
|
119
|
+
|
|
120
|
+
# Returns captured length (> 0) if matched, 0 if rejected
|
|
121
|
+
match_len = BpfEmulator.execute(filter_code, raw_packet)
|
|
122
|
+
|
|
123
|
+
if match_len > 0:
|
|
124
|
+
print(f"Matched! Captured {match_len} bytes.")
|
|
125
|
+
else:
|
|
126
|
+
print("Rejected.")
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
### Attaching to Linux Raw Sockets
|
|
130
|
+
|
|
131
|
+
Attach compiled filters straight to a Linux AF_PACKET raw socket with `SO_ATTACH_FILTER`:
|
|
132
|
+
|
|
133
|
+
```python
|
|
134
|
+
import socket
|
|
135
|
+
from LightPacket.LightBPF.Compiler import BpfCompiler
|
|
136
|
+
|
|
137
|
+
# 1. Open raw socket
|
|
138
|
+
sock = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(0x0003))
|
|
139
|
+
sock.bind(("eth0", 0))
|
|
140
|
+
|
|
141
|
+
# 2. Compile and package into ctypes sock_fprog
|
|
142
|
+
instructions = BpfCompiler.compile_str("tcp and port 443")
|
|
143
|
+
fprog = BpfCompiler.to_fprog(instructions)
|
|
144
|
+
|
|
145
|
+
# 3. Attach in kernel space
|
|
146
|
+
SO_ATTACH_FILTER = 26
|
|
147
|
+
sock.setsockopt(socket.SOL_SOCKET, SO_ATTACH_FILTER, fprog)
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
### Disassembling Bytecode
|
|
151
|
+
|
|
152
|
+
Print human-readable cBPF assembly formatted like `tcpdump -d`:
|
|
153
|
+
|
|
154
|
+
```python
|
|
155
|
+
from LightPacket.LightBPF.Compiler import BpfCompiler, disassemble
|
|
156
|
+
|
|
157
|
+
insns = BpfCompiler.compile_str("ip and (port 80 or port 443)")
|
|
158
|
+
print(disassemble(insns))
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
---
|
|
162
|
+
|
|
163
|
+
## Supported Filter Syntax Cheat Sheet
|
|
164
|
+
|
|
165
|
+
| Category | Filter Examples | Description |
|
|
166
|
+
|---|---|---|
|
|
167
|
+
| **Protocols** | `ip`, `ip6`, `tcp`, `udp`, `icmp`, `arp`, `vlan`, `mpls`, `ether` | Standalone protocol verification |
|
|
168
|
+
| **Hosts** | `host 192.168.1.1`<br>`src host 10.0.0.1`<br>`dst host fe80::1`<br>`ether host aa:bb:cc:dd:ee:ff` | Match IPv4, IPv6 (4 words), or MAC address |
|
|
169
|
+
| **Networks** | `net 10.0.0.0/8`<br>`src net 192.168.1.0/24`<br>`net 10.0.0.0 mask 255.0.0.0`<br>`ip6 net 2001:db8::/32` | CIDR and netmask matching |
|
|
170
|
+
| **Ports** | `port 80`<br>`tcp dst port 443`<br>`udp port 53`<br>`src port http`<br>`portrange 1000-2000` | Port, port range, and service name resolution |
|
|
171
|
+
| **Byte Access** | `tcp[13] & 2 != 0`<br>`ip[2:2] > 500`<br>`ether[12:2] == 0x0800`<br>`tcp[tcpflags] & tcp-syn != 0` | Arbitrary byte/half/word extraction |
|
|
172
|
+
| **Header Length** | `(ip[0] & 0xf) * 4 > 20` | Dynamic IPv4 IHL calculations |
|
|
173
|
+
| **Packet Size** | `len > 100`, `less 64`, `greater 1500` | Frame size checks via `BPF_LEN` |
|
|
174
|
+
| **Broadcast / Multicast** | `broadcast`<br>`multicast` | Ethernet broadcast and multicast bit checks |
|
|
175
|
+
| **VLAN / MPLS** | `vlan`, `vlan 100`<br>`mpls`, `mpls 500` | Tag presence and ID/label extraction |
|
|
176
|
+
| **Protocol Numbers** | `ip proto 6`, `ip proto tcp`, `ether proto 0x0800` | Layer 3/4 protocol inspection |
|
|
177
|
+
| **Boolean Logic** | `tcp and port 80`<br>`tcp or udp`<br>`not port 22`<br>`(tcp or udp) and not dst port 53` | Short-circuit boolean combinations |
|
|
178
|
+
|
|
179
|
+
---
|
|
180
|
+
|
|
181
|
+
## Architecture & Pipeline
|
|
182
|
+
|
|
183
|
+
### The Classic BPF (cBPF) Virtual Machine
|
|
184
|
+
|
|
185
|
+
cBPF is a 32-bit register machine designed specifically for in-kernel packet filtering:
|
|
186
|
+
|
|
187
|
+
```
|
|
188
|
+
+----------------------------------------------------------------+
|
|
189
|
+
| cBPF Virtual Machine Registers |
|
|
190
|
+
| |
|
|
191
|
+
| [ Register A ] (32-bit Accumulator) |
|
|
192
|
+
| - Holds results of packet loads, arithmetic, and tests |
|
|
193
|
+
| |
|
|
194
|
+
| [ Register X ] (32-bit Index Register) |
|
|
195
|
+
| - Used for indexed packet addressing: packet[X + k] |
|
|
196
|
+
| - Second operand for ALU operations |
|
|
197
|
+
| |
|
|
198
|
+
| [ Memory M[0..15] ] (16 Scratch Words) |
|
|
199
|
+
| - Register spill space for complex nested expressions |
|
|
200
|
+
| |
|
|
201
|
+
| [ Program Counter (PC) ] |
|
|
202
|
+
+----------------------------------------------------------------+
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
### Instruction Structure (`struct bpf_insn`)
|
|
206
|
+
|
|
207
|
+
Every instruction is exactly 8 bytes:
|
|
208
|
+
|
|
209
|
+
```
|
|
210
|
+
0 1 2 3
|
|
211
|
+
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
|
|
212
|
+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
|
213
|
+
| code (16) | jt (8) | jf (8) |
|
|
214
|
+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
|
215
|
+
| k (32) |
|
|
216
|
+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
- **`code`** (16 bits): Opcode encoding class, size modifier, and addressing mode/ALU operator.
|
|
220
|
+
- **`jt`** (8 bits): Jump-true relative offset (number of instructions to skip forward if comparison succeeds).
|
|
221
|
+
- **`jf`** (8 bits): Jump-false relative offset (number of instructions to skip forward if comparison fails).
|
|
222
|
+
- **`k`** (32 bits): Multi-purpose value (packet offset, immediate constant, or scratch index).
|
|
223
|
+
|
|
224
|
+
---
|
|
225
|
+
|
|
226
|
+
## Developer Deep-Dive: Stage 1 — Lexer (`Lexer.py`)
|
|
227
|
+
|
|
228
|
+
The `BpfLexer` turns a raw filter string into a list of typed `Token` tuples: `(type, value)`.
|
|
229
|
+
|
|
230
|
+
### Tokenizer Mechanics
|
|
231
|
+
- **Regex Ordering**: The tokenizer runs a single compiled regex with named groups. Specific tokens (such as MAC addresses and IPv6 with colons) take precedence over general words and symbols.
|
|
232
|
+
- **Normalization**: Operators like `&&`, `||`, and `!` are normalized directly into `and`, `or`, and `not`.
|
|
233
|
+
- **Named Constants**: TCP flags (`tcp-syn`, `tcp-ack`, `tcp-fin`) and ICMP types (`icmp-echo`, `icmp-echoreply`) are resolved directly into integer `NUMBER` tokens at lex time.
|
|
234
|
+
|
|
235
|
+
### Token Reference Table
|
|
236
|
+
|
|
237
|
+
| Token Type | Examples | Produced From |
|
|
238
|
+
|---|---|---|
|
|
239
|
+
| `NUMBER` | `'80'`, `'2048'`, `'2'` | Integer digits, hex (`0x0800`), named constants (`tcp-syn` $\rightarrow$ `2`) |
|
|
240
|
+
| `IPV4` | `'192.168.1.1'` | Standard dotted-quad IPv4 |
|
|
241
|
+
| `IPV4_CIDR` | `'10.0.0.0/8'` | Dotted-quad with prefix |
|
|
242
|
+
| `IPV6` | `'fe80::1'` | Colon-delimited hex address |
|
|
243
|
+
| `IPV6_CIDR` | `'2001:db8::/32'` | Colon-hex with prefix |
|
|
244
|
+
| `MAC` | `'aa:bb:cc:dd:ee:ff'` | 6 hex pairs with colons |
|
|
245
|
+
| `RANGE` | `'1000-2000'` | Port range syntax |
|
|
246
|
+
| `IP-PROTOCOL` | `'tcp'`, `'udp'`, `'icmp'`, `'gre'` | IP protocol names |
|
|
247
|
+
| `ETHERTYPE` | `'ip'`, `'ip6'`, `'arp'`, `'vlan'` | Link-layer protocol names |
|
|
248
|
+
| `KEYWORD` | `'host'`, `'port'`, `'net'`, `'src'`, `'dst'` | Filter keywords |
|
|
249
|
+
| `FIELD` | `'tcpflags'`, `'icmptype'` | Named protocol fields |
|
|
250
|
+
| `REL_OP` | `'=='`, `'!='`, `'>'`, `'>='`, `'<'`, `'<='` | Comparison operators |
|
|
251
|
+
| `MATH_OP` | `'+'`, `'-'`, `'*'`, `'/'`, `'%'`, `'&'`, `'\|'`, `'^'`, `'<<'`, `'>>'` | Arithmetic and bitwise operators |
|
|
252
|
+
|
|
253
|
+
---
|
|
254
|
+
|
|
255
|
+
## Developer Deep-Dive: Stage 2 — Parser (`Parser.py`)
|
|
256
|
+
|
|
257
|
+
The `BpfParser` implements a **recursive descent** parser that turns the token stream into an Abstract Syntax Tree (AST).
|
|
258
|
+
|
|
259
|
+
### Grammar & Operator Precedence
|
|
260
|
+
|
|
261
|
+
Precedence is enforced through tiered grammar methods (lowest to highest):
|
|
262
|
+
|
|
263
|
+
```
|
|
264
|
+
expr_or := expr_and ( 'or' expr_and )*
|
|
265
|
+
expr_and := expr_not ( 'and' expr_not )*
|
|
266
|
+
expr_not := 'not' expr_not | expr_primary
|
|
267
|
+
expr_primary := '(' expr_or ')' | qualifier_rule | expr_relation
|
|
268
|
+
expr_relation := expr_bitor ( REL_OP expr_bitor )?
|
|
269
|
+
expr_bitor := expr_bitxor ( '|' expr_bitxor )*
|
|
270
|
+
expr_bitxor := expr_bitand ( '^' expr_bitand )*
|
|
271
|
+
expr_bitand := expr_shift ( '&' expr_shift )*
|
|
272
|
+
expr_shift := expr_add ( ('<<'|'>>') expr_add )*
|
|
273
|
+
expr_add := expr_mult ( ('+'|'-') expr_mult )*
|
|
274
|
+
expr_mult := expr_atom ( ('*'|'/'|'%') expr_atom )*
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
### Abstract Syntax Tree (AST) Reference
|
|
278
|
+
|
|
279
|
+
```python
|
|
280
|
+
# Boolean Operations
|
|
281
|
+
BinaryOpNode(left: ASTNode, op: str, right: ASTNode)
|
|
282
|
+
UnaryOpNode(op: str, expr: ASTNode)
|
|
283
|
+
|
|
284
|
+
# Protocols & Qualifiers
|
|
285
|
+
ProtocolNode(name: str, proto_num: Optional[int] = None)
|
|
286
|
+
KeywordFilterNode(
|
|
287
|
+
keyword: str,
|
|
288
|
+
value: Any = None,
|
|
289
|
+
direction: Optional[str] = None, # 'src' | 'dst' | 'src or dst' | 'src and dst'
|
|
290
|
+
protocol: Optional[str] = None, # 'tcp' | 'ip6' | 'ether' ...
|
|
291
|
+
value_type: Optional[str] = None, # 'ipv4' | 'ipv6' | 'mac' | 'cidr4' | 'cidr6' | 'number' | 'range' | 'name'
|
|
292
|
+
mask: Optional[str] = None # for 'net X mask Y'
|
|
293
|
+
)
|
|
294
|
+
|
|
295
|
+
# Packet Access & Values
|
|
296
|
+
ByteAccessNode(protocol: str, offset: ASTNode, size: int = 1)
|
|
297
|
+
LengthNode() # 'len' operand
|
|
298
|
+
NumberNode(value: int)
|
|
299
|
+
LiteralNode(value: str, kind: str)
|
|
300
|
+
FieldNode(name: str) # 'tcpflags', 'icmptype'
|
|
301
|
+
```
|
|
302
|
+
|
|
303
|
+
---
|
|
304
|
+
|
|
305
|
+
## Developer Deep-Dive: Stage 3 — Compiler (`Compiler.py`)
|
|
306
|
+
|
|
307
|
+
The `BpfCompiler` walks the AST and emits classic BPF instructions.
|
|
308
|
+
|
|
309
|
+
### Short-Circuit Boolean Control Flow
|
|
310
|
+
|
|
311
|
+
All boolean sub-expressions compile with two symbolic jump targets: `true_target` and `false_target`.
|
|
312
|
+
|
|
313
|
+
- **`A and B`**: `A` compiles with `true_target = label_B` and `false_target = overall_false`. If `A` passes, it falls into `B`. If `A` fails, it immediately jumps to `overall_false`.
|
|
314
|
+
- **`A or B`**: `A` compiles with `true_target = overall_true` and `false_target = label_B`. If `A` passes, it immediately short-circuits to `overall_true`.
|
|
315
|
+
- **`not A`**: Swaps the true and false branch targets of `A`.
|
|
316
|
+
|
|
317
|
+
### Relative Jump Resolution & Jump Threading
|
|
318
|
+
|
|
319
|
+
In cBPF, relative jump offsets specify how many instructions to skip **past the next instruction**:
|
|
320
|
+
|
|
321
|
+
$$\text{offset} = \text{target\_index} - (\text{current\_index} + 1)$$
|
|
322
|
+
|
|
323
|
+
Before computing final numeric offsets, the compiler executes **Jump Threading (Peephole Optimization)**:
|
|
324
|
+
If a jump target points to an unconditional jump (`ja target_2`), the compiler rewrites the jump to target `target_2` directly, eliminating wasted jump hops.
|
|
325
|
+
|
|
326
|
+
### Register Spilling via Scratch Memory (`M[0..15]`)
|
|
327
|
+
|
|
328
|
+
cBPF only has two general registers: Accumulator `A` and Index `X`. When compiling complex binary operations where both the left and right operands are expressions (e.g. `(ip[0] & 0xf) * 4 == (tcp[12] >> 4) * 4`):
|
|
329
|
+
|
|
330
|
+
1. Right operand compiles into Register `A`.
|
|
331
|
+
2. Emits `st M[scratch]` to store the right operand in scratch memory.
|
|
332
|
+
3. Left operand compiles into Register `A`.
|
|
333
|
+
4. Emits `ldx M[scratch]` to load the right operand into Register `X`.
|
|
334
|
+
5. Emits the ALU or comparison operation using Register `X` (`BPF_X`).
|
|
335
|
+
|
|
336
|
+
### Dynamic IPv4 IHL Tracking (`BPF_MSH`)
|
|
337
|
+
|
|
338
|
+
IPv4 headers can vary in length between 20 and 60 bytes due to optional fields. To locate transport-layer headers (TCP/UDP) reliably without hardcoding an offset:
|
|
339
|
+
|
|
340
|
+
```python
|
|
341
|
+
self._emit(Instruction(BPF_LDX | BPF_B | BPF_MSH, 0, 0, 14))
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
The `BPF_MSH` opcode computes:
|
|
345
|
+
$$X = 4 \times (\text{packet}[14] \ \& \ \text{0x0F})$$
|
|
346
|
+
storing the exact byte length of the IPv4 header into Register `X`.
|
|
347
|
+
Subsequent transport-layer loads use indexed addressing (`BPF_IND`):
|
|
348
|
+
- Source Port: `ldh [x + 14]`
|
|
349
|
+
- Destination Port: `ldh [x + 16]`
|
|
350
|
+
|
|
351
|
+
### VLAN 802.1Q Handling
|
|
352
|
+
|
|
353
|
+
An 802.1Q tagged frame has the following layout:
|
|
354
|
+
|
|
355
|
+
```
|
|
356
|
+
Byte 12..13: TPID (0x8100)
|
|
357
|
+
Byte 14..15: TCI (Bits 15-13: Priority, Bit 12: DEI, Bits 11-0: 12-bit VLAN ID)
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
For `vlan 100`:
|
|
361
|
+
1. `ldh [12]` $\rightarrow$ Verify EtherType is `0x8100`.
|
|
362
|
+
2. `ldh [14]` $\rightarrow$ Load 16-bit TCI.
|
|
363
|
+
3. `and #0x0fff` $\rightarrow$ Mask out Priority and DEI to isolate the 12-bit VLAN ID.
|
|
364
|
+
4. `jeq #100` $\rightarrow$ Match target VLAN ID.
|
|
365
|
+
|
|
366
|
+
### Bitmask Optimization (`BPF_JSET`)
|
|
367
|
+
|
|
368
|
+
When testing specific bitflags (e.g. `tcp[13] & 2 != 0` or `tcp[tcpflags] & tcp-syn != 0`), the compiler optimizes this pattern into a single `BPF_JSET` jump instruction:
|
|
369
|
+
|
|
370
|
+
```
|
|
371
|
+
ldb [x + 27] ; Load TCP flags byte
|
|
372
|
+
jset #2 jt ACCEPT jf REJECT
|
|
373
|
+
```
|
|
374
|
+
|
|
375
|
+
This tests `(A & 2) != 0` in a single hardware cycle.
|
|
376
|
+
|
|
377
|
+
---
|
|
378
|
+
|
|
379
|
+
## Worked Examples & Disassembly Traces
|
|
380
|
+
|
|
381
|
+
### Example 1: `tcp and port 80`
|
|
382
|
+
|
|
383
|
+
```python
|
|
384
|
+
insns = BpfCompiler.compile_str("tcp and port 80")
|
|
385
|
+
print(disassemble(insns))
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
```
|
|
389
|
+
(000) ldh [12] ; Load EtherType
|
|
390
|
+
(001) jeq #0x800 jt 2 jf 9 ; Is IPv4?
|
|
391
|
+
(002) ldb [23] ; Load IP Protocol byte
|
|
392
|
+
(003) jeq #6 jt 4 jf 9 ; Is TCP (6)?
|
|
393
|
+
(004) ldxb 4*([14]&0xf) ; Load IPv4 Header Length into X
|
|
394
|
+
(005) ldh [x + 14] ; Load TCP Source Port
|
|
395
|
+
(006) jeq #80 jt 8 jf 7 ; Is src port 80? (If yes, accept)
|
|
396
|
+
(007) ldh [x + 16] ; Load TCP Destination Port
|
|
397
|
+
(008) jeq #80 jt 10 jf 9 ; Is dst port 80? (If yes, accept; else reject)
|
|
398
|
+
(009) ret #0 ; REJECT (Return 0)
|
|
399
|
+
(010) ret #262144 ; ACCEPT (Return max buffer)
|
|
400
|
+
```
|
|
401
|
+
|
|
402
|
+
---
|
|
403
|
+
|
|
404
|
+
### Example 2: `tcp[13] & 2 != 0` (TCP SYN)
|
|
405
|
+
|
|
406
|
+
```python
|
|
407
|
+
insns = BpfCompiler.compile_str("tcp[13] & 2 != 0")
|
|
408
|
+
print(disassemble(insns))
|
|
409
|
+
```
|
|
410
|
+
|
|
411
|
+
```
|
|
412
|
+
(000) ldxb 4*([14]&0xf) ; X = IPv4 IHL
|
|
413
|
+
(001) ldb [x + 27] ; Load byte 13 of TCP (14 + 13 = 27)
|
|
414
|
+
(002) jset #2 jt 4 jf 3 ; Test bit 2 (SYN flag)
|
|
415
|
+
(003) ret #0 ; REJECT
|
|
416
|
+
(004) ret #262144 ; ACCEPT
|
|
417
|
+
```
|
|
418
|
+
|
|
419
|
+
---
|
|
420
|
+
|
|
421
|
+
### Example 3: `host 192.168.1.1`
|
|
422
|
+
|
|
423
|
+
```python
|
|
424
|
+
insns = BpfCompiler.compile_str("host 192.168.1.1")
|
|
425
|
+
print(disassemble(insns))
|
|
426
|
+
```
|
|
427
|
+
|
|
428
|
+
```
|
|
429
|
+
(000) ldh [12] ; Load EtherType
|
|
430
|
+
(001) jeq #0x800 jt 2 jf 5 ; Is IPv4?
|
|
431
|
+
(002) ld [26] ; Load Source IP
|
|
432
|
+
(003) jeq #0xc0a80101 jt 6 jf 4 ; Matches 192.168.1.1?
|
|
433
|
+
(004) ld [30] ; Load Destination IP
|
|
434
|
+
(005) jeq #0xc0a80101 jt 6 jf 5 ; Matches 192.168.1.1?
|
|
435
|
+
(006) ret #0 ; REJECT
|
|
436
|
+
(007) ret #262144 ; ACCEPT
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
---
|
|
440
|
+
|
|
441
|
+
### Example 4: `vlan 100`
|
|
442
|
+
|
|
443
|
+
```python
|
|
444
|
+
insns = BpfCompiler.compile_str("vlan 100")
|
|
445
|
+
print(disassemble(insns))
|
|
446
|
+
```
|
|
447
|
+
|
|
448
|
+
```
|
|
449
|
+
(000) ldh [12] ; Load EtherType
|
|
450
|
+
(001) jeq #0x8100 jt 2 jf 5 ; Is 802.1Q VLAN?
|
|
451
|
+
(002) ldh [14] ; Load TCI (Tag Control Information)
|
|
452
|
+
(003) and #0xfff ; Mask out Priority & DEI to get 12-bit VID
|
|
453
|
+
(004) jeq #100 jt 6 jf 5 ; Is VID == 100?
|
|
454
|
+
(005) ret #0 ; REJECT
|
|
455
|
+
(006) ret #262144 ; ACCEPT
|
|
456
|
+
```
|
|
457
|
+
|
|
458
|
+
---
|
|
459
|
+
|
|
460
|
+
### Example 5: `(ip[0] & 0xf) * 4 > 20`
|
|
461
|
+
|
|
462
|
+
Checks if the IPv4 packet contains IP Options (IHL > 20 bytes):
|
|
463
|
+
|
|
464
|
+
```python
|
|
465
|
+
insns = BpfCompiler.compile_str("(ip[0] & 0xf) * 4 > 20")
|
|
466
|
+
print(disassemble(insns))
|
|
467
|
+
```
|
|
468
|
+
|
|
469
|
+
```
|
|
470
|
+
(000) ldb [14] ; Load byte 0 of IP header
|
|
471
|
+
(001) and #15 ; Mask lower 4 bits (IHL)
|
|
472
|
+
(002) mul #4 ; Multiply by 4 (header length in bytes)
|
|
473
|
+
(003) jgt #20 jt 5 jf 4 ; Is header length > 20?
|
|
474
|
+
(004) ret #0 ; REJECT
|
|
475
|
+
(005) ret #262144 ; ACCEPT
|
|
476
|
+
```
|
|
477
|
+
|
|
478
|
+
---
|
|
479
|
+
|
|
480
|
+
## Verification & Testing
|
|
481
|
+
|
|
482
|
+
The engine is validated through two test suites covering unit tests, emulation tests, and native C libpcap cross-verification:
|
|
483
|
+
|
|
484
|
+
```bash
|
|
485
|
+
pytest tests/test_bpf.py tests/test_lightbpf.py -v
|
|
486
|
+
```
|
|
487
|
+
|
|
488
|
+
### What is tested:
|
|
489
|
+
1. **Lexer & Parser**: Operator precedence, token classification, strict mode, syntax errors.
|
|
490
|
+
2. **Bytecode Compilation**: Opcode correctness, backpatching, jump offsets, and disassembly.
|
|
491
|
+
3. **Emulator Verification (`BpfEmulator`)**: Real packet matching and rejection across IPv4, IPv6, ARP, VLAN, and MPLS packets.
|
|
492
|
+
4. **Native C Libpcap Cross-Verification**: If `libpcap.so` is present on the host, tests execute each compiled filter through native C `bpf_filter()` to guarantee 100% bytecode compatibility with libpcap/tcpdump.
|
|
493
|
+
|
|
494
|
+
---
|
|
495
|
+
|
|
496
|
+
## Opcode & Constant Reference
|
|
497
|
+
|
|
498
|
+
### Instruction Classes (`code & 0x07`)
|
|
499
|
+
| Constant | Hex | Description |
|
|
500
|
+
|---|---|---|
|
|
501
|
+
| `BPF_LD` | `0x00` | Load into Register A |
|
|
502
|
+
| `BPF_LDX` | `0x01` | Load into Register X |
|
|
503
|
+
| `BPF_ST` | `0x02` | Store Register A into `M[k]` |
|
|
504
|
+
| `BPF_STX` | `0x03` | Store Register X into `M[k]` |
|
|
505
|
+
| `BPF_ALU` | `0x04` | Arithmetic or Bitwise operation on A |
|
|
506
|
+
| `BPF_JMP` | `0x05` | Jump operation |
|
|
507
|
+
| `BPF_RET` | `0x06` | Return from filter |
|
|
508
|
+
| `BPF_MISC` | `0x07` | Register transfer (`tax`, `txa`) |
|
|
509
|
+
|
|
510
|
+
### Addressing Modes (`code & 0xe0`)
|
|
511
|
+
| Constant | Hex | Description |
|
|
512
|
+
|---|---|---|
|
|
513
|
+
| `BPF_IMM` | `0x00` | Immediate constant `k` |
|
|
514
|
+
| `BPF_ABS` | `0x20` | Absolute packet byte offset `packet[k]` |
|
|
515
|
+
| `BPF_IND` | `0x40` | Indexed packet offset `packet[X + k]` |
|
|
516
|
+
| `BPF_MEM` | `0x60` | Scratch memory word `M[k]` |
|
|
517
|
+
| `BPF_LEN` | `0x80` | Packet length |
|
|
518
|
+
| `BPF_MSH` | `0xa0` | Load $4 \times (\text{packet}[k] \ \& \ \text{0x0F})$ into X |
|
|
519
|
+
|
|
520
|
+
### ALU Operations (`code & 0xf0`)
|
|
521
|
+
| Constant | Hex | Operation |
|
|
522
|
+
|---|---|---|
|
|
523
|
+
| `BPF_ADD` | `0x00` | `A = A + operand` |
|
|
524
|
+
| `BPF_SUB` | `0x10` | `A = A - operand` |
|
|
525
|
+
| `BPF_MUL` | `0x20` | `A = A * operand` |
|
|
526
|
+
| `BPF_DIV` | `0x30` | `A = A / operand` |
|
|
527
|
+
| `BPF_OR` | `0x40` | `A = A \| operand` |
|
|
528
|
+
| `BPF_AND` | `0x50` | `A = A & operand` |
|
|
529
|
+
| `BPF_LSH` | `0x60` | `A = A << operand` |
|
|
530
|
+
| `BPF_RSH` | `0x70` | `A = A >> operand` |
|
|
531
|
+
| `BPF_NEG` | `0x80` | `A = -A` |
|
|
532
|
+
| `BPF_MOD` | `0x90` | `A = A % operand` |
|
|
533
|
+
| `BPF_XOR` | `0xa0` | `A = A ^ operand` |
|
|
534
|
+
|
|
535
|
+
### Jump Operations (`code & 0xf0`)
|
|
536
|
+
| Constant | Hex | Jump Condition |
|
|
537
|
+
|---|---|---|
|
|
538
|
+
| `BPF_JA` | `0x00` | Unconditional jump |
|
|
539
|
+
| `BPF_JEQ` | `0x10` | Jump if `A == operand` |
|
|
540
|
+
| `BPF_JGT` | `0x20` | Jump if `A > operand` |
|
|
541
|
+
| `BPF_JGE` | `0x30` | Jump if `A >= operand` |
|
|
542
|
+
| `BPF_JSET` | `0x40` | Jump if `(A & operand) != 0` |
|