necksight 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- necksight-1.0.0/LICENSE +21 -0
- necksight-1.0.0/MANIFEST.in +13 -0
- necksight-1.0.0/PKG-INFO +246 -0
- necksight-1.0.0/README.md +210 -0
- necksight-1.0.0/csrc/Makefile +21 -0
- necksight-1.0.0/csrc/necksight.c +733 -0
- necksight-1.0.0/csrc/necksight.h +133 -0
- necksight-1.0.0/csrc/tools/necksight_selftest.c +328 -0
- necksight-1.0.0/necksight/__init__.py +24 -0
- necksight-1.0.0/necksight/__main__.py +6 -0
- necksight-1.0.0/necksight/cli.py +348 -0
- necksight-1.0.0/necksight/emd.py +283 -0
- necksight-1.0.0/necksight/looming.py +513 -0
- necksight-1.0.0/necksight/retina.py +225 -0
- necksight-1.0.0/necksight.egg-info/PKG-INFO +246 -0
- necksight-1.0.0/necksight.egg-info/SOURCES.txt +20 -0
- necksight-1.0.0/necksight.egg-info/dependency_links.txt +1 -0
- necksight-1.0.0/necksight.egg-info/entry_points.txt +2 -0
- necksight-1.0.0/necksight.egg-info/requires.txt +8 -0
- necksight-1.0.0/necksight.egg-info/top_level.txt +1 -0
- necksight-1.0.0/pyproject.toml +67 -0
- necksight-1.0.0/setup.cfg +4 -0
necksight-1.0.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Amandeep Singh Bains
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
include LICENSE
|
|
2
|
+
include README.md
|
|
3
|
+
include pyproject.toml
|
|
4
|
+
recursive-include necksight *.py
|
|
5
|
+
include csrc/Makefile
|
|
6
|
+
include csrc/necksight.h
|
|
7
|
+
include csrc/necksight.c
|
|
8
|
+
include csrc/tools/necksight_selftest.c
|
|
9
|
+
prune assets
|
|
10
|
+
prune csrc/build
|
|
11
|
+
prune results
|
|
12
|
+
prune scripts
|
|
13
|
+
prune tests
|
necksight-1.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,246 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: necksight
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Drosophila optic-lobe T4/T5 motion and LPLC2 4-quadrant looming collision reflex engine in Python and zero-heap ANSI C99
|
|
5
|
+
Author: Amandeep Singh Bains
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/indian-aman/necksight
|
|
8
|
+
Project-URL: Repository, https://github.com/indian-aman/necksight
|
|
9
|
+
Project-URL: Releases, https://github.com/indian-aman/necksight/releases
|
|
10
|
+
Project-URL: Issues, https://github.com/indian-aman/necksight/issues
|
|
11
|
+
Project-URL: Documentation, https://github.com/indian-aman/necksight#readme
|
|
12
|
+
Keywords: necksight,necklink,drosophila,optic-lobe,looming,collision-avoidance,optical-flow,hassenstein-reichardt,barlow-levick,lplc2,t4-t5,robotics,drones,usv,esp32
|
|
13
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: Intended Audience :: Science/Research
|
|
16
|
+
Classifier: Operating System :: OS Independent
|
|
17
|
+
Classifier: Programming Language :: C
|
|
18
|
+
Classifier: Programming Language :: Python :: 3
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
23
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
24
|
+
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
|
|
25
|
+
Classifier: Topic :: Scientific/Engineering :: Image Processing
|
|
26
|
+
Requires-Python: >=3.9
|
|
27
|
+
Description-Content-Type: text/markdown
|
|
28
|
+
License-File: LICENSE
|
|
29
|
+
Requires-Dist: numpy>=1.24
|
|
30
|
+
Provides-Extra: nltp
|
|
31
|
+
Requires-Dist: pynltp>=1.2.0; extra == "nltp"
|
|
32
|
+
Provides-Extra: dev
|
|
33
|
+
Requires-Dist: pytest>=7.0; extra == "dev"
|
|
34
|
+
Requires-Dist: matplotlib>=3.7; extra == "dev"
|
|
35
|
+
Dynamic: license-file
|
|
36
|
+
|
|
37
|
+
<div align="center">
|
|
38
|
+
|
|
39
|
+
# NeckSight (`necksight` v1.0.0)
|
|
40
|
+
|
|
41
|
+
***Drosophila* Optic-Lobe `T4`/`T5` Motion & `LPLC2` 4-Quadrant Looming Collision-Reflex Engine — in Python (`NumPy`) and Zero-Heap, Zero-Float ANSI C99 (`< 15 KB` RAM, `> 12,000 FPS`).**
|
|
42
|
+
|
|
43
|
+
[](https://pypi.org/project/necksight/)
|
|
44
|
+
[](https://github.com/indian-aman/necksight/releases/tag/v1.0.0)
|
|
45
|
+
[](https://pypi.org/project/necksight/)
|
|
46
|
+
[](csrc/)
|
|
47
|
+
[](csrc/necksight.h)
|
|
48
|
+
[-10B981?style=flat-square)](results/benchmark_summary.json)
|
|
49
|
+
[](https://github.com/indian-aman/necklink)
|
|
50
|
+
[](LICENSE)
|
|
51
|
+
|
|
52
|
+
[**30-Second Quick Start**](#-30-second-quick-start) ·
|
|
53
|
+
[**Biophysical Circuit**](#-how-the-optic-lobe-circuit-works) ·
|
|
54
|
+
[**ANSI C99 / ESP32-S3**](#-zero-heap-zero-float-ansi-c99-core-esp32-s3-ready) ·
|
|
55
|
+
[**Benchmarks**](#-empirical-benchmarks-n--60-cluttered-sequences) ·
|
|
56
|
+
[**When to Use (and When Not To)**](#-when-to-use-necksight--and-when-not-to)
|
|
57
|
+
|
|
58
|
+
</div>
|
|
59
|
+
|
|
60
|
+
<p align="center">
|
|
61
|
+
<img src="https://raw.githubusercontent.com/indian-aman/necksight/main/assets/necksight_benchmarks.png" alt="NeckSight v1.0.0 Drosophila Optic Lobe Looming Reflex Benchmarks" width="100%">
|
|
62
|
+
</p>
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## Why NeckSight?
|
|
67
|
+
|
|
68
|
+
Conventional lightweight collision triggers fail outdoors on moving robots, autonomous surface vessels (USVs), and micro-drones:
|
|
69
|
+
- **Frame-differencing** explodes on sudden cloud-to-sunlight glare steps (`10x`–`100x` lux jumps) and violent wave/wind camera turns (`100%` false-alarm rate on yaw/glare).
|
|
70
|
+
- **Spatial-gradient optical flow divergence (`Lucas-Kanade`)** breaks brightness constancy under rapid illumination flashes and suffers aperture/aliasing spikes during fast `3–4 px/frame` snap-pans across cluttered horizons (`55.6%` false-alarm rate at `95%` TPR).
|
|
71
|
+
- **Linear un-gated motion divergence** fires whenever a high-contrast foreground edge sweeps into view against open sky or water (`55.6%` false-alarm rate).
|
|
72
|
+
|
|
73
|
+
**NeckSight** (`Product #4` in the **NeckLink** fruit-fly brain stack) solves pre-collision reflex gating the way *Drosophila melanogaster* flies through cluttered canopies at high angular turn rates without flinching at its own turns:
|
|
74
|
+
|
|
75
|
+
1. **`R1–R6` Log/Naka-Rushton Photoreceptor Adaptation + Wide-Field Glare Cancellation (`RetinaLaminaPreproc`)**: Compresses arbitrary `H×W` camera frames onto `32×32` or `64×64` grids, cancels `100x` global sunlight/auto-exposure jumps in a single frame without blinding recovery lag, and splits transients into parallel **`L1` (`ON` brightening)** and **`L2` (`OFF` darkening)** channels.
|
|
76
|
+
2. **4-Cardinal 3-Arm `T4` (`ON`) & `T5` (`OFF`) Elementary Motion Detectors (`T4T5MotionEngine`)**: Combines preferred-arm Hassenstein-Reichardt correlation (`delayed_back * center` with `Mi4`-like center adaptation) and null-arm Barlow-Levick divisive shunting inhibition (`center / (1 + g * delayed_front)`), completely rejecting stationary contrast flicker while feeding wide-field **`LPTC`** tangential ego-motion cells.
|
|
77
|
+
3. **`LPLC2` 4-Quadrant Radial Symmetry Gate -> `Giant Fiber` Reflex (`LPLC2LoomingReflex`)**: Evaluates outward centrifugal motion minus local `LPi` inward opponency across all four cruciform arms (`Q_L, Q_R, Q_U, Q_D`) via `O(1)` Summed-Area Tables and applies a **4-quadrant geometric-mean gate** `(Q_L * Q_R * Q_U * Q_D)^0.25` suppressed by wide-field `LPTC` pan flow. During fast camera yaw/pitch turns, receding motion, or passing cross-traffic, at least one opposing arm sees inward motion (`Q_opposing == 0`), collapsing false alarms to **`0.0%` at `95%` TPR** (`AUROC = 0.9988`).
|
|
78
|
+
|
|
79
|
+
```mermaid
|
|
80
|
+
flowchart LR
|
|
81
|
+
A["Camera Frame (H x W)"] --> B["R1-R6 Log + Glare Adapt\n& Lamina L1 (ON) / L2 (OFF)"]
|
|
82
|
+
B --> C["3-Arm T4 (ON) & T5 (OFF)\nHR Correlation + BL Veto"]
|
|
83
|
+
C --> D["LPTC Wide-Field Flow\n(vx, vy, yaw_pan, div)"]
|
|
84
|
+
C --> E["LPLC2 4-Arm Radial Gate\n(Q_L * Q_R * Q_U * Q_D)^0.25"]
|
|
85
|
+
D -->|"Tangential Pan Inhibition"| E
|
|
86
|
+
E --> F["GiantFiberTrigger\n(triggered, score, est_ttc_s, focus_xy)"]
|
|
87
|
+
F -.->|"Optional 0-HoL Bridge"| G["pynltp send_reflex('LOOM_ESTOP')"]
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## ⚡ 30-Second Quick Start
|
|
93
|
+
|
|
94
|
+
### 1. Install
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
pip install necksight
|
|
98
|
+
# Optional: include pynltp for encrypted UDP Giant-Fiber reflex dispatch
|
|
99
|
+
pip install "necksight[nltp]"
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
### 2. End-to-End Python Looming Reflex (`NeckSightPipeline`)
|
|
103
|
+
|
|
104
|
+
```python
|
|
105
|
+
import numpy as np
|
|
106
|
+
from necksight import NeckSightPipeline
|
|
107
|
+
|
|
108
|
+
pipe = NeckSightPipeline(grid_size=64, threshold=0.045)
|
|
109
|
+
|
|
110
|
+
# Feed arbitrary HxW grayscale (or HxWx3 RGB) frames at 60 Hz
|
|
111
|
+
for frame in camera_frames:
|
|
112
|
+
trig = pipe.step(frame, dt=1.0 / 60.0)
|
|
113
|
+
if trig.triggered:
|
|
114
|
+
print(
|
|
115
|
+
f"COLLISION WARNING! score={trig.looming_score:.3f}, "
|
|
116
|
+
f"TTC={trig.est_ttc_s:.3f}s, focus={trig.focus_xy}"
|
|
117
|
+
)
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
### 3. Stage-by-Stage Optic-Lobe Inspection + `pynltp` Out-of-Line Reflex Bridge
|
|
121
|
+
|
|
122
|
+
When paired with [`pynltp`](https://pypi.org/project/pynltp/) (`NeckLink` v1.2+), `attach_nltp_reflex` dispatches an urgent out-of-line `send_reflex("LOOM_ESTOP")` datagram with zero head-of-line blocking even while bulk telemetry/files are mid-transfer:
|
|
123
|
+
|
|
124
|
+
```python
|
|
125
|
+
from necksight import (
|
|
126
|
+
RetinaLaminaPreproc,
|
|
127
|
+
T4T5MotionEngine,
|
|
128
|
+
LPLC2LoomingReflex,
|
|
129
|
+
attach_nltp_reflex,
|
|
130
|
+
)
|
|
131
|
+
|
|
132
|
+
retina = RetinaLaminaPreproc(grid_size=64)
|
|
133
|
+
emd = T4T5MotionEngine(grid_size=64)
|
|
134
|
+
lplc2 = LPLC2LoomingReflex(grid_size=64)
|
|
135
|
+
|
|
136
|
+
# Bind any pynltp NLTPClient (or compatible object exposing .send_reflex(msg))
|
|
137
|
+
attach_nltp_reflex(nltp_client, msg="LOOM_ESTOP", detector=lplc2)
|
|
138
|
+
|
|
139
|
+
lamina_out = retina.step(frame, dt=1 / 60)
|
|
140
|
+
motion_out = emd.step(lamina_out)
|
|
141
|
+
vx, vy, yaw_pan, radial_div = motion_out.lptc
|
|
142
|
+
trigger = lplc2.step(motion_out, dt=1 / 60)
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
### 4. Command-Line Interface (`necksight`)
|
|
146
|
+
|
|
147
|
+
```bash
|
|
148
|
+
# Run 4 synthetic optic-lobe scenarios (grating, fast yaw pan, 100x glare, looming)
|
|
149
|
+
necksight demo
|
|
150
|
+
|
|
151
|
+
# Run a quick multi-sequence accuracy, lead-time, and FPS benchmark
|
|
152
|
+
necksight benchmark-quick --json
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## 🔧 Zero-Heap, Zero-Float ANSI C99 Core (`ESP32-S3` Ready)
|
|
158
|
+
|
|
159
|
+
[`csrc/necksight.h`](csrc/necksight.h) and [`csrc/necksight.c`](csrc/necksight.c) implement the complete `32×32` pipeline using **pure fixed-point integers (`int32_t` / `uint16_t` / `uint8_t`)**:
|
|
160
|
+
|
|
161
|
+
- **`0` heap allocations**: Zero `malloc` / `calloc` / `free` anywhere in the codebase.
|
|
162
|
+
- **`0` floating-point instructions**: Integer `Q8` `log2` lookup, 64-point lattice median glare filter, integer `33×33` Summed-Area Table, and bit-shift integer `isqrt` (`(Q_L * Q_R * Q_U * Q_D)^0.25 = isqrt(isqrt(Q_L * Q_R) * isqrt(Q_U * Q_D))`).
|
|
163
|
+
- **`14,616` bytes (`14.27 KB < 16 KB`) total static state**: Enforced at compile time (`necksight_state_under_16kb_check`), keeping FreeRTOS task stack usage under `1 KB`.
|
|
164
|
+
- **Direct `ESP32-S3` `OV2640` / `OV5640` camera support**: `necksight_step_frame_u8()` box-downsamples arbitrary `QVGA` (`240×320`) or `VGA` (`480×640`) `uint8_t` buffers into `Q4` `32×32` precision in a single call.
|
|
165
|
+
|
|
166
|
+
```c
|
|
167
|
+
#include "necksight.h"
|
|
168
|
+
|
|
169
|
+
static necksight_state_t optic_lobe;
|
|
170
|
+
|
|
171
|
+
void setup_vision(void) {
|
|
172
|
+
necksight_init(&optic_lobe);
|
|
173
|
+
}
|
|
174
|
+
|
|
175
|
+
void on_camera_frame(const uint8_t *fb_buf, uint16_t h, uint16_t w) {
|
|
176
|
+
necksight_result_t res;
|
|
177
|
+
necksight_step_frame_u8(&optic_lobe, fb_buf, h, w, w, 16U, &res);
|
|
178
|
+
if (res.triggered) {
|
|
179
|
+
/* Trigger hardware estop or dispatch NLTP reflex packet */
|
|
180
|
+
trigger_evasive_maneuver(res.focus_x, res.focus_y, res.est_ttc_ms);
|
|
181
|
+
}
|
|
182
|
+
}
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
Build and run the strict ANSI C99 self-test (`-O2 -std=c99 -Wall -Wextra -pedantic -Werror`):
|
|
186
|
+
|
|
187
|
+
```bash
|
|
188
|
+
cd csrc && make clean all test
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
---
|
|
192
|
+
|
|
193
|
+
## 📊 Empirical Benchmarks (`N = 60` Cluttered Sequences)
|
|
194
|
+
|
|
195
|
+
Reproduce the full benchmark JSON ([`results/benchmark_summary.json`](results/benchmark_summary.json)) and 3-panel chart ([`assets/necksight_benchmarks.png`](assets/necksight_benchmarks.png)) via [`scripts/benchmark_necksight.py`](scripts/benchmark_necksight.py):
|
|
196
|
+
|
|
197
|
+
```bash
|
|
198
|
+
python scripts/benchmark_necksight.py
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
### 1. Looming Collision Selectivity vs. Hard Negatives (`64×64`, `60 Hz`)
|
|
202
|
+
|
|
203
|
+
Evaluated across `24` approaching obstacles (dark, bright, textured, and off-center on drifting cluttered backgrounds) vs. `36` hard negatives (`12` fast `1.8–4.2 px/frame` camera yaw/pitch snap-pans across clutter & masts, `12` sudden `15x–100x` sunlight glare flashes & water glints, and `12` receding / lateral cross-traffic objects):
|
|
204
|
+
|
|
205
|
+
| Detector | Overall AUROC (`↑`) | Looming vs Yaw/Glare AUROC (`↑`) | False-Alarm Rate @ 95% TPR (`↓`) | Yaw/Glare FAR @ 95% TPR (`↓`) | Mean Warning Lead Time (`↑`) | CPU Speed (`FPS`) | State RAM (`KB`) |
|
|
206
|
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
|
207
|
+
| **NeckSight (`T4/T5 + LPLC2`, C99 `32×32`)** | — | — | **`0.0%`** (selftest) | **`0.0%`** | Pre-impact | **`12,463` FPS** | **`14.27 KB`** |
|
|
208
|
+
| **NeckSight (`T4/T5 + LPLC2`, Python `64×64`)** | **`0.9988`** | **`0.9983`** | **`0.0%`** | **`0.0%`** | **`402.9 ms`** | `512`–`750` FPS | `48.0 KB` |
|
|
209
|
+
| **Naive Un-gated EMD** *(no `LPLC2` gate)* | `0.6100` | `0.5955` | `55.6%` | `50.0%` | `342.8 ms` | `861` FPS | `48.0 KB` |
|
|
210
|
+
| **Lucas-Kanade Divergence** *(`5×5` tensor)* | `0.5220` | `0.2830` | `55.6%` | `83.3%` | `186.2 ms` | `1,154` FPS | `96.0 KB` |
|
|
211
|
+
| **Frame-Difference Motion Energy** | `0.3530` | `0.0295` | `66.7%` | `100.0%` | `215.2 ms` | `62,769` FPS | `16.0 KB` |
|
|
212
|
+
|
|
213
|
+
### 2. Honest Sub-Pixel Odometry Check (`0.05 – 0.35 px/frame` Slow Drift)
|
|
214
|
+
|
|
215
|
+
| Task (`64×64`, Varying Scene Contrast `0.45x – 1.0x`) | Spatial-Gradient Lucas-Kanade | NeckSight `T4/T5` Correlational `LPTC` | Winner |
|
|
216
|
+
| :--- | :---: | :---: | :---: |
|
|
217
|
+
| **Fine Sub-Pixel Translational Velocity MAE (`↓`)** | **`0.0549 px/frame`** | `0.1325 px/frame` | **Lucas-Kanade (`2.4x` lower error)** |
|
|
218
|
+
|
|
219
|
+
---
|
|
220
|
+
|
|
221
|
+
## 🧭 When to Use NeckSight — and When Not To
|
|
222
|
+
|
|
223
|
+
### Use NeckSight When:
|
|
224
|
+
- **You need an ultra-reliable, zero-GPU collision reflex on moving platforms**: Micro-drones, USVs, rovers, and mast cameras that pitch, roll, and snap-yaw violently across high-contrast scenery where standard motion or optical-flow divergence triggers constant false e-stops.
|
|
225
|
+
- **Outdoor lighting is harsh and unpredictable**: Sudden `20x`–`100x` sunlight glare steps, cloud breaks, specular water glints, or tunnel exits.
|
|
226
|
+
- **RAM, power, and latency budgets are tiny**: Runs in `< 15 KB` static SRAM with zero `malloc` and zero floats on `ESP32-S3`, `STM32`, `RP2040/RP2350`, or alongside heavy workloads on Raspberry Pi / Jetson CPU cores (`< 2 ms` per frame in Python, `< 0.1 ms` in C99).
|
|
227
|
+
|
|
228
|
+
### Do NOT Use NeckSight When:
|
|
229
|
+
- **You need metric sub-pixel translational visual odometry**: Biophysical `T4/T5` Hassenstein-Reichardt correlators couple velocity with pattern contrast and spatial frequency (`MAE = 0.1325 px/frame` across varying contrast vs. **`0.0549 px/frame` for windowed Lucas-Kanade**). Use `Lucas-Kanade` or feature-based VIO for metric speed estimation.
|
|
230
|
+
- **You need semantic object recognition or static obstacle mapping**: Stationary obstacles on a stationary camera produce zero retinal temporal transients (`L1 = L2 = 0`). Pair `NeckSight` as a fast `>60 Hz` reflex safety layer alongside stereo depth, LiDAR, or a CNN/ViT semantic detector.
|
|
231
|
+
- **Obstacles approach purely along a 1D thin wire `< 1` ommatidium wide**: Because `LPLC2` requires outward expansion across all 4 cruciform quadrants, a thin horizontal wire expanding only vertically (`Q_L = Q_R = 0`) will not fire the radial 4-quadrant gate.
|
|
232
|
+
|
|
233
|
+
---
|
|
234
|
+
|
|
235
|
+
## 📚 Neuroscience & Algorithmic References
|
|
236
|
+
|
|
237
|
+
1. **Maisak, M. S. et al.** (2013). *A directional tuning map of Drosophila elementary motion detectors.* **Nature**, 500, 212–216.
|
|
238
|
+
2. **Haag, J., Mishra, A., & Borst, A.** (2017). *A common directional tuning mechanism of Drosophila motion-sensing neurons in the ON and in the OFF pathway.* **eLife**, 6, e29044.
|
|
239
|
+
3. **Klapoetke, N. C. et al.** (2017). *Ultra-selective looming detection from radial motion opponency.* **Nature**, 551, 237–241.
|
|
240
|
+
4. **Ache, J. M. et al.** (2019). *Neural basis for looming size and velocity encoding in the Drosophila giant fiber escape pathway.* **Current Biology**, 29(6), 1073–1081.
|
|
241
|
+
|
|
242
|
+
---
|
|
243
|
+
|
|
244
|
+
## License
|
|
245
|
+
|
|
246
|
+
Released under the [MIT License](LICENSE). Copyright (c) 2026 Amandeep Singh Bains.
|
|
@@ -0,0 +1,210 @@
|
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
# NeckSight (`necksight` v1.0.0)
|
|
4
|
+
|
|
5
|
+
***Drosophila* Optic-Lobe `T4`/`T5` Motion & `LPLC2` 4-Quadrant Looming Collision-Reflex Engine — in Python (`NumPy`) and Zero-Heap, Zero-Float ANSI C99 (`< 15 KB` RAM, `> 12,000 FPS`).**
|
|
6
|
+
|
|
7
|
+
[](https://pypi.org/project/necksight/)
|
|
8
|
+
[](https://github.com/indian-aman/necksight/releases/tag/v1.0.0)
|
|
9
|
+
[](https://pypi.org/project/necksight/)
|
|
10
|
+
[](csrc/)
|
|
11
|
+
[](csrc/necksight.h)
|
|
12
|
+
[-10B981?style=flat-square)](results/benchmark_summary.json)
|
|
13
|
+
[](https://github.com/indian-aman/necklink)
|
|
14
|
+
[](LICENSE)
|
|
15
|
+
|
|
16
|
+
[**30-Second Quick Start**](#-30-second-quick-start) ·
|
|
17
|
+
[**Biophysical Circuit**](#-how-the-optic-lobe-circuit-works) ·
|
|
18
|
+
[**ANSI C99 / ESP32-S3**](#-zero-heap-zero-float-ansi-c99-core-esp32-s3-ready) ·
|
|
19
|
+
[**Benchmarks**](#-empirical-benchmarks-n--60-cluttered-sequences) ·
|
|
20
|
+
[**When to Use (and When Not To)**](#-when-to-use-necksight--and-when-not-to)
|
|
21
|
+
|
|
22
|
+
</div>
|
|
23
|
+
|
|
24
|
+
<p align="center">
|
|
25
|
+
<img src="https://raw.githubusercontent.com/indian-aman/necksight/main/assets/necksight_benchmarks.png" alt="NeckSight v1.0.0 Drosophila Optic Lobe Looming Reflex Benchmarks" width="100%">
|
|
26
|
+
</p>
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Why NeckSight?
|
|
31
|
+
|
|
32
|
+
Conventional lightweight collision triggers fail outdoors on moving robots, autonomous surface vessels (USVs), and micro-drones:
|
|
33
|
+
- **Frame-differencing** explodes on sudden cloud-to-sunlight glare steps (`10x`–`100x` lux jumps) and violent wave/wind camera turns (`100%` false-alarm rate on yaw/glare).
|
|
34
|
+
- **Spatial-gradient optical flow divergence (`Lucas-Kanade`)** breaks brightness constancy under rapid illumination flashes and suffers aperture/aliasing spikes during fast `3–4 px/frame` snap-pans across cluttered horizons (`55.6%` false-alarm rate at `95%` TPR).
|
|
35
|
+
- **Linear un-gated motion divergence** fires whenever a high-contrast foreground edge sweeps into view against open sky or water (`55.6%` false-alarm rate).
|
|
36
|
+
|
|
37
|
+
**NeckSight** (`Product #4` in the **NeckLink** fruit-fly brain stack) solves pre-collision reflex gating the way *Drosophila melanogaster* flies through cluttered canopies at high angular turn rates without flinching at its own turns:
|
|
38
|
+
|
|
39
|
+
1. **`R1–R6` Log/Naka-Rushton Photoreceptor Adaptation + Wide-Field Glare Cancellation (`RetinaLaminaPreproc`)**: Compresses arbitrary `H×W` camera frames onto `32×32` or `64×64` grids, cancels `100x` global sunlight/auto-exposure jumps in a single frame without blinding recovery lag, and splits transients into parallel **`L1` (`ON` brightening)** and **`L2` (`OFF` darkening)** channels.
|
|
40
|
+
2. **4-Cardinal 3-Arm `T4` (`ON`) & `T5` (`OFF`) Elementary Motion Detectors (`T4T5MotionEngine`)**: Combines preferred-arm Hassenstein-Reichardt correlation (`delayed_back * center` with `Mi4`-like center adaptation) and null-arm Barlow-Levick divisive shunting inhibition (`center / (1 + g * delayed_front)`), completely rejecting stationary contrast flicker while feeding wide-field **`LPTC`** tangential ego-motion cells.
|
|
41
|
+
3. **`LPLC2` 4-Quadrant Radial Symmetry Gate -> `Giant Fiber` Reflex (`LPLC2LoomingReflex`)**: Evaluates outward centrifugal motion minus local `LPi` inward opponency across all four cruciform arms (`Q_L, Q_R, Q_U, Q_D`) via `O(1)` Summed-Area Tables and applies a **4-quadrant geometric-mean gate** `(Q_L * Q_R * Q_U * Q_D)^0.25` suppressed by wide-field `LPTC` pan flow. During fast camera yaw/pitch turns, receding motion, or passing cross-traffic, at least one opposing arm sees inward motion (`Q_opposing == 0`), collapsing false alarms to **`0.0%` at `95%` TPR** (`AUROC = 0.9988`).
|
|
42
|
+
|
|
43
|
+
```mermaid
|
|
44
|
+
flowchart LR
|
|
45
|
+
A["Camera Frame (H x W)"] --> B["R1-R6 Log + Glare Adapt\n& Lamina L1 (ON) / L2 (OFF)"]
|
|
46
|
+
B --> C["3-Arm T4 (ON) & T5 (OFF)\nHR Correlation + BL Veto"]
|
|
47
|
+
C --> D["LPTC Wide-Field Flow\n(vx, vy, yaw_pan, div)"]
|
|
48
|
+
C --> E["LPLC2 4-Arm Radial Gate\n(Q_L * Q_R * Q_U * Q_D)^0.25"]
|
|
49
|
+
D -->|"Tangential Pan Inhibition"| E
|
|
50
|
+
E --> F["GiantFiberTrigger\n(triggered, score, est_ttc_s, focus_xy)"]
|
|
51
|
+
F -.->|"Optional 0-HoL Bridge"| G["pynltp send_reflex('LOOM_ESTOP')"]
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## ⚡ 30-Second Quick Start
|
|
57
|
+
|
|
58
|
+
### 1. Install
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
pip install necksight
|
|
62
|
+
# Optional: include pynltp for encrypted UDP Giant-Fiber reflex dispatch
|
|
63
|
+
pip install "necksight[nltp]"
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
### 2. End-to-End Python Looming Reflex (`NeckSightPipeline`)
|
|
67
|
+
|
|
68
|
+
```python
|
|
69
|
+
import numpy as np
|
|
70
|
+
from necksight import NeckSightPipeline
|
|
71
|
+
|
|
72
|
+
pipe = NeckSightPipeline(grid_size=64, threshold=0.045)
|
|
73
|
+
|
|
74
|
+
# Feed arbitrary HxW grayscale (or HxWx3 RGB) frames at 60 Hz
|
|
75
|
+
for frame in camera_frames:
|
|
76
|
+
trig = pipe.step(frame, dt=1.0 / 60.0)
|
|
77
|
+
if trig.triggered:
|
|
78
|
+
print(
|
|
79
|
+
f"COLLISION WARNING! score={trig.looming_score:.3f}, "
|
|
80
|
+
f"TTC={trig.est_ttc_s:.3f}s, focus={trig.focus_xy}"
|
|
81
|
+
)
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
### 3. Stage-by-Stage Optic-Lobe Inspection + `pynltp` Out-of-Line Reflex Bridge
|
|
85
|
+
|
|
86
|
+
When paired with [`pynltp`](https://pypi.org/project/pynltp/) (`NeckLink` v1.2+), `attach_nltp_reflex` dispatches an urgent out-of-line `send_reflex("LOOM_ESTOP")` datagram with zero head-of-line blocking even while bulk telemetry/files are mid-transfer:
|
|
87
|
+
|
|
88
|
+
```python
|
|
89
|
+
from necksight import (
|
|
90
|
+
RetinaLaminaPreproc,
|
|
91
|
+
T4T5MotionEngine,
|
|
92
|
+
LPLC2LoomingReflex,
|
|
93
|
+
attach_nltp_reflex,
|
|
94
|
+
)
|
|
95
|
+
|
|
96
|
+
retina = RetinaLaminaPreproc(grid_size=64)
|
|
97
|
+
emd = T4T5MotionEngine(grid_size=64)
|
|
98
|
+
lplc2 = LPLC2LoomingReflex(grid_size=64)
|
|
99
|
+
|
|
100
|
+
# Bind any pynltp NLTPClient (or compatible object exposing .send_reflex(msg))
|
|
101
|
+
attach_nltp_reflex(nltp_client, msg="LOOM_ESTOP", detector=lplc2)
|
|
102
|
+
|
|
103
|
+
lamina_out = retina.step(frame, dt=1 / 60)
|
|
104
|
+
motion_out = emd.step(lamina_out)
|
|
105
|
+
vx, vy, yaw_pan, radial_div = motion_out.lptc
|
|
106
|
+
trigger = lplc2.step(motion_out, dt=1 / 60)
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
### 4. Command-Line Interface (`necksight`)
|
|
110
|
+
|
|
111
|
+
```bash
|
|
112
|
+
# Run 4 synthetic optic-lobe scenarios (grating, fast yaw pan, 100x glare, looming)
|
|
113
|
+
necksight demo
|
|
114
|
+
|
|
115
|
+
# Run a quick multi-sequence accuracy, lead-time, and FPS benchmark
|
|
116
|
+
necksight benchmark-quick --json
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## 🔧 Zero-Heap, Zero-Float ANSI C99 Core (`ESP32-S3` Ready)
|
|
122
|
+
|
|
123
|
+
[`csrc/necksight.h`](csrc/necksight.h) and [`csrc/necksight.c`](csrc/necksight.c) implement the complete `32×32` pipeline using **pure fixed-point integers (`int32_t` / `uint16_t` / `uint8_t`)**:
|
|
124
|
+
|
|
125
|
+
- **`0` heap allocations**: Zero `malloc` / `calloc` / `free` anywhere in the codebase.
|
|
126
|
+
- **`0` floating-point instructions**: Integer `Q8` `log2` lookup, 64-point lattice median glare filter, integer `33×33` Summed-Area Table, and bit-shift integer `isqrt` (`(Q_L * Q_R * Q_U * Q_D)^0.25 = isqrt(isqrt(Q_L * Q_R) * isqrt(Q_U * Q_D))`).
|
|
127
|
+
- **`14,616` bytes (`14.27 KB < 16 KB`) total static state**: Enforced at compile time (`necksight_state_under_16kb_check`), keeping FreeRTOS task stack usage under `1 KB`.
|
|
128
|
+
- **Direct `ESP32-S3` `OV2640` / `OV5640` camera support**: `necksight_step_frame_u8()` box-downsamples arbitrary `QVGA` (`240×320`) or `VGA` (`480×640`) `uint8_t` buffers into `Q4` `32×32` precision in a single call.
|
|
129
|
+
|
|
130
|
+
```c
|
|
131
|
+
#include "necksight.h"
|
|
132
|
+
|
|
133
|
+
static necksight_state_t optic_lobe;
|
|
134
|
+
|
|
135
|
+
void setup_vision(void) {
|
|
136
|
+
necksight_init(&optic_lobe);
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
void on_camera_frame(const uint8_t *fb_buf, uint16_t h, uint16_t w) {
|
|
140
|
+
necksight_result_t res;
|
|
141
|
+
necksight_step_frame_u8(&optic_lobe, fb_buf, h, w, w, 16U, &res);
|
|
142
|
+
if (res.triggered) {
|
|
143
|
+
/* Trigger hardware estop or dispatch NLTP reflex packet */
|
|
144
|
+
trigger_evasive_maneuver(res.focus_x, res.focus_y, res.est_ttc_ms);
|
|
145
|
+
}
|
|
146
|
+
}
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
Build and run the strict ANSI C99 self-test (`-O2 -std=c99 -Wall -Wextra -pedantic -Werror`):
|
|
150
|
+
|
|
151
|
+
```bash
|
|
152
|
+
cd csrc && make clean all test
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## 📊 Empirical Benchmarks (`N = 60` Cluttered Sequences)
|
|
158
|
+
|
|
159
|
+
Reproduce the full benchmark JSON ([`results/benchmark_summary.json`](results/benchmark_summary.json)) and 3-panel chart ([`assets/necksight_benchmarks.png`](assets/necksight_benchmarks.png)) via [`scripts/benchmark_necksight.py`](scripts/benchmark_necksight.py):
|
|
160
|
+
|
|
161
|
+
```bash
|
|
162
|
+
python scripts/benchmark_necksight.py
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
### 1. Looming Collision Selectivity vs. Hard Negatives (`64×64`, `60 Hz`)
|
|
166
|
+
|
|
167
|
+
Evaluated across `24` approaching obstacles (dark, bright, textured, and off-center on drifting cluttered backgrounds) vs. `36` hard negatives (`12` fast `1.8–4.2 px/frame` camera yaw/pitch snap-pans across clutter & masts, `12` sudden `15x–100x` sunlight glare flashes & water glints, and `12` receding / lateral cross-traffic objects):
|
|
168
|
+
|
|
169
|
+
| Detector | Overall AUROC (`↑`) | Looming vs Yaw/Glare AUROC (`↑`) | False-Alarm Rate @ 95% TPR (`↓`) | Yaw/Glare FAR @ 95% TPR (`↓`) | Mean Warning Lead Time (`↑`) | CPU Speed (`FPS`) | State RAM (`KB`) |
|
|
170
|
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
|
171
|
+
| **NeckSight (`T4/T5 + LPLC2`, C99 `32×32`)** | — | — | **`0.0%`** (selftest) | **`0.0%`** | Pre-impact | **`12,463` FPS** | **`14.27 KB`** |
|
|
172
|
+
| **NeckSight (`T4/T5 + LPLC2`, Python `64×64`)** | **`0.9988`** | **`0.9983`** | **`0.0%`** | **`0.0%`** | **`402.9 ms`** | `512`–`750` FPS | `48.0 KB` |
|
|
173
|
+
| **Naive Un-gated EMD** *(no `LPLC2` gate)* | `0.6100` | `0.5955` | `55.6%` | `50.0%` | `342.8 ms` | `861` FPS | `48.0 KB` |
|
|
174
|
+
| **Lucas-Kanade Divergence** *(`5×5` tensor)* | `0.5220` | `0.2830` | `55.6%` | `83.3%` | `186.2 ms` | `1,154` FPS | `96.0 KB` |
|
|
175
|
+
| **Frame-Difference Motion Energy** | `0.3530` | `0.0295` | `66.7%` | `100.0%` | `215.2 ms` | `62,769` FPS | `16.0 KB` |
|
|
176
|
+
|
|
177
|
+
### 2. Honest Sub-Pixel Odometry Check (`0.05 – 0.35 px/frame` Slow Drift)
|
|
178
|
+
|
|
179
|
+
| Task (`64×64`, Varying Scene Contrast `0.45x – 1.0x`) | Spatial-Gradient Lucas-Kanade | NeckSight `T4/T5` Correlational `LPTC` | Winner |
|
|
180
|
+
| :--- | :---: | :---: | :---: |
|
|
181
|
+
| **Fine Sub-Pixel Translational Velocity MAE (`↓`)** | **`0.0549 px/frame`** | `0.1325 px/frame` | **Lucas-Kanade (`2.4x` lower error)** |
|
|
182
|
+
|
|
183
|
+
---
|
|
184
|
+
|
|
185
|
+
## 🧭 When to Use NeckSight — and When Not To
|
|
186
|
+
|
|
187
|
+
### Use NeckSight When:
|
|
188
|
+
- **You need an ultra-reliable, zero-GPU collision reflex on moving platforms**: Micro-drones, USVs, rovers, and mast cameras that pitch, roll, and snap-yaw violently across high-contrast scenery where standard motion or optical-flow divergence triggers constant false e-stops.
|
|
189
|
+
- **Outdoor lighting is harsh and unpredictable**: Sudden `20x`–`100x` sunlight glare steps, cloud breaks, specular water glints, or tunnel exits.
|
|
190
|
+
- **RAM, power, and latency budgets are tiny**: Runs in `< 15 KB` static SRAM with zero `malloc` and zero floats on `ESP32-S3`, `STM32`, `RP2040/RP2350`, or alongside heavy workloads on Raspberry Pi / Jetson CPU cores (`< 2 ms` per frame in Python, `< 0.1 ms` in C99).
|
|
191
|
+
|
|
192
|
+
### Do NOT Use NeckSight When:
|
|
193
|
+
- **You need metric sub-pixel translational visual odometry**: Biophysical `T4/T5` Hassenstein-Reichardt correlators couple velocity with pattern contrast and spatial frequency (`MAE = 0.1325 px/frame` across varying contrast vs. **`0.0549 px/frame` for windowed Lucas-Kanade**). Use `Lucas-Kanade` or feature-based VIO for metric speed estimation.
|
|
194
|
+
- **You need semantic object recognition or static obstacle mapping**: Stationary obstacles on a stationary camera produce zero retinal temporal transients (`L1 = L2 = 0`). Pair `NeckSight` as a fast `>60 Hz` reflex safety layer alongside stereo depth, LiDAR, or a CNN/ViT semantic detector.
|
|
195
|
+
- **Obstacles approach purely along a 1D thin wire `< 1` ommatidium wide**: Because `LPLC2` requires outward expansion across all 4 cruciform quadrants, a thin horizontal wire expanding only vertically (`Q_L = Q_R = 0`) will not fire the radial 4-quadrant gate.
|
|
196
|
+
|
|
197
|
+
---
|
|
198
|
+
|
|
199
|
+
## 📚 Neuroscience & Algorithmic References
|
|
200
|
+
|
|
201
|
+
1. **Maisak, M. S. et al.** (2013). *A directional tuning map of Drosophila elementary motion detectors.* **Nature**, 500, 212–216.
|
|
202
|
+
2. **Haag, J., Mishra, A., & Borst, A.** (2017). *A common directional tuning mechanism of Drosophila motion-sensing neurons in the ON and in the OFF pathway.* **eLife**, 6, e29044.
|
|
203
|
+
3. **Klapoetke, N. C. et al.** (2017). *Ultra-selective looming detection from radial motion opponency.* **Nature**, 551, 237–241.
|
|
204
|
+
4. **Ache, J. M. et al.** (2019). *Neural basis for looming size and velocity encoding in the Drosophila giant fiber escape pathway.* **Current Biology**, 29(6), 1073–1081.
|
|
205
|
+
|
|
206
|
+
---
|
|
207
|
+
|
|
208
|
+
## License
|
|
209
|
+
|
|
210
|
+
Released under the [MIT License](LICENSE). Copyright (c) 2026 Amandeep Singh Bains.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
CC ?= gcc
|
|
2
|
+
CFLAGS ?= -O2 -std=c99 -Wall -Wextra -pedantic -Werror -I.
|
|
3
|
+
BUILD ?= build
|
|
4
|
+
|
|
5
|
+
CORE_SRCS := necksight.c
|
|
6
|
+
|
|
7
|
+
.PHONY: all test clean
|
|
8
|
+
|
|
9
|
+
all: $(BUILD)/necksight_selftest
|
|
10
|
+
|
|
11
|
+
$(BUILD):
|
|
12
|
+
mkdir -p $(BUILD)
|
|
13
|
+
|
|
14
|
+
$(BUILD)/necksight_selftest: tools/necksight_selftest.c $(CORE_SRCS) necksight.h | $(BUILD)
|
|
15
|
+
$(CC) $(CFLAGS) -o $@ tools/necksight_selftest.c $(CORE_SRCS)
|
|
16
|
+
|
|
17
|
+
test: $(BUILD)/necksight_selftest
|
|
18
|
+
./$(BUILD)/necksight_selftest
|
|
19
|
+
|
|
20
|
+
clean:
|
|
21
|
+
rm -rf $(BUILD)
|