termux-vision 1.0.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,151 +1,75 @@
1
- # termux-vision (AMEVA-Vision)
1
+ # Termux-Vision
2
2
 
3
- > **Native On-Device Computer Vision & VLM Multimodal Inference Framework for Android Termux**
4
- > *Dual-Engine (Python & Node.js/TypeScript) · Zero Heavy C++ Build Dependency · Pure Python & Pure JS Fast Paths · Mobile-Resilient Runtime · Canny/Sobel/Haar · VLM (SmolVLM/Qwen2-VL) · Vulkan/CPU Isolation · termux-train LoRA Ready*
3
+ [![PyPI](https://img.shields.io/pypi/v/termux-vision.svg?style=flat-square&color=0369a1)](https://pypi.org/project/termux-vision/)
4
+ [![Python](https://img.shields.io/pypi/pyversions/termux-vision.svg?style=flat-square)](https://pypi.org/project/termux-vision/)
5
+ [![npm](https://img.shields.io/npm/v/termux-vision.svg?style=flat-square&color=b91c1c)](https://www.npmjs.com/package/termux-vision)
6
+ [![npm downloads](https://img.shields.io/npm/dm/termux-vision.svg?style=flat-square&color=b91c1c)](https://www.npmjs.com/package/termux-vision)
7
+ [![License](https://img.shields.io/badge/License-Apache_2.0-004499.svg?style=flat-square)](https://github.com/uno-km/termux-vision)
5
8
 
6
- <div align="center">
7
-
8
- [![Official Documentation](https://img.shields.io/badge/docs-uno--km.vercel.app%2Flib%2Fvision-004499?style=for-the-badge&logo=vercel)](https://uno-km.vercel.app/lib/vision/)
9
- [![PyPI - Version](https://img.shields.io/pypi/v/termux-vision.svg?color=0066cc&logo=pypi&logoColor=white&style=for-the-badge)](https://pypi.org/project/termux-vision/)
10
- [![npm - Version](https://img.shields.io/npm/v/termux-vision.svg?color=cb3837&logo=npm&logoColor=white&style=for-the-badge)](https://www.npmjs.com/package/termux-vision)
11
- [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg?style=for-the-badge)](LICENSE)
12
- [![AMEVA Foundation](https://img.shields.io/badge/Foundation-AOSF_Tier_1-orange?style=for-the-badge)](https://uno-km.vercel.app/docs/foundation/)
13
-
14
- ### Ultra-lightweight On-Device Computer Vision & Multimodal VLM Engine
15
- **An Official Tier 1 Top-Level Open-Source Project of the AMEVA Foundation (AOSF)**
16
-
17
- [Official Documentation](https://uno-km.vercel.app/lib/vision/) • [PyPI Package](https://pypi.org/project/termux-vision/) • [npm Package](https://www.npmjs.com/package/termux-vision) • [Issue Tracker](https://github.com/uno-km/termux-vision/issues)
18
-
19
- </div>
9
+ > **안드로이드 Termux를 위한 제로 디펜던시 온디바이스 컴퓨터 비전 & VLM 멀티모달 추론 엔진**
10
+ > *Zero-Dependency On-Device Computer Vision & Multimodal VLM Inference Engine for Android Termux*
20
11
 
21
12
  ---
22
13
 
23
- ## What is termux-vision?
14
+ ## Architecture & Overview
24
15
 
25
- `termux-vision` (also known as `AMEVA-Vision`) is a native, self-contained computer vision, feature extraction, and multimodal VLM inference framework designed specifically for **Android Termux native environments** and ARM64 edge hardware.
26
-
27
- Standard vision frameworks (OpenCV, TorchVision, ONNX Runtime) suffer from severe compilation overhead, DPKG locks, Bionic libc symbol mismatches, and heavy dependencies on mobile edge devices. `termux-vision` solves this by providing:
28
- - **One-Touch System Installer (`install.sh`)**: Sets up Termux `pkg` toolchains, Python SDK, and Node.js CLI in a single step.
29
- - **Dual-Engine Architecture**: Full, 100% equivalent API and CLI support for both **Python (`pip`)** and **Node.js/TypeScript (`npm`)**.
30
- - **Multimodal VLM Engine**: Run SmolVLM and Qwen2-VL vision-language models natively on-device with Vulkan GPU acceleration and automatic CPU fallback.
31
- - **Full Parameter Control & Strict Null Guards**: Simple one-line default interfaces for quick tasks, plus full parameter access (`top_p`, `top_k`, `temperature`, `repeat_penalty`, `seed`, `system_prompt`, `ngl`, `threads`, `context_limit`) with strict boundary validation (zero silent fallbacks on null/invalid inputs).
32
- - **Free & Unrestricted Model Downloads**: Download official catalog presets or arbitrary Hugging Face / direct HTTP(S) model files anytime.
33
- - **Full Traditional Vision Stack**: Canny Edge Detector, Sobel $3\times3$, Gaussian Blur, Integral Images, Morphology, Contours, and Haar Cascade Face Detection.
34
- - **1:1 Native Bridge with `termux-train`**: Pass vision feature maps directly into `termux-train` for on-device LoRA and classifier fine-tuning.
16
+ - **초고속 C/C++ CPU 비전 커널**: 8-방향 2차원 BFS Hysteresis Canny 에지 검출, Sobel 3x3, 적분 영상 필터링을 스레드 로컬 0-Allocation 스크래치 버퍼로 1ms 내에 완료.
17
+ - **AMEVA Vulkan Runtime 완전 통합**: 기기 드라이버 및 칩셋 버그(Quirks)를 자동 패치하고 `--device auto -ngl 99`로 모바일 GPU 가속 완전 위임.
18
+ - **스마트 4단계 해상도 프리셋**: `fast` (384px), `optimal` (768px), `high` (1280px), `original`로 모바일 메모리 한계 내에서 VLM 추론 최적화.
19
+ - **듀얼 엔진 지원**: Python (`pip`) Node.js (`npm`) 양대 언어에서 100% 동일한 비전 파이프라인 제공.
35
20
 
36
21
  ---
37
22
 
38
- ## 5-Minute Quickstart
39
-
40
- ### 1. Installation
41
-
42
- #### One-Touch System Setup (Recommended for Termux):
43
- ```bash
44
- # In Android Termux:
45
- curl -sSL https://raw.githubusercontent.com/uno-km/termux-vision/main/install.sh | bash
46
- ```
23
+ ## Installation & Quickstart
47
24
 
48
- #### Via PyPI (Python):
25
+ ### One-Touch Installer (Recommended)
49
26
  ```bash
50
- pkg update && pkg install python python-numpy git
51
- pip install termux-vision
27
+ curl -sL https://raw.githubusercontent.com/uno-km/termux-vision/main/install.sh | bash
52
28
  ```
53
29
 
54
- #### Via npm (Node.js / TypeScript):
30
+ ### Python (PyPI)
55
31
  ```bash
56
- pkg update && pkg install nodejs
57
- npm install -g termux-vision
58
- # Or run instantly via npx:
59
- npx termux-vision doctor
32
+ pip install termux-vision && termux-llama install
60
33
  ```
34
+ ```python
35
+ import termux_vision as tv
61
36
 
62
- ---
63
-
64
- ### 2. Model Management & Free Downloads
65
-
66
- ```bash
67
- # 1. Inspect device hardware and Vulkan GPU
68
- termux-vision doctor --probe-vulkan
69
-
70
- # 2. Install official catalog preset (~550 MB)
71
- termux-vision model install smolvlm-500m-q4
72
-
73
- # 3. Freely download any custom Hugging Face model
74
- termux-vision model download hf:second-state/Qwen2-VL-2B-Instruct-GGUF:Qwen2-VL-2B-Instruct-Q4_K_M.gguf
37
+ # 1. Classical CV Filters (Sub-millisecond C/C++ Engine)
38
+ img = tv.io.load_image("photo.jpg")
39
+ edges = tv.cv.canny(tv.transforms.to_grayscale(img), 40, 120)
75
40
 
76
- # 4. List installed models
77
- termux-vision model list
41
+ # 2. On-Device VLM Multimodal Inference
42
+ with tv.vlm.load("smolvlm-500m-q4", quality="optimal") as engine:
43
+ res = engine.describe("photo.jpg", prompt="Describe this scene in detail.")
44
+ print(f"[{res.metrics.backend.upper()}] {res.text}")
78
45
  ```
79
46
 
80
- ---
81
-
82
- ### 3. VLM Image Chat & Inference
83
-
84
- #### CLI Usage:
47
+ ### Node.js / TypeScript (npm)
85
48
  ```bash
86
- # Basic one-line image description
87
- termux-vision vlm sample.jpg -p "이 사진 속 인물과 배경을 설명해줘"
88
-
89
- # Advanced professional inference with parameter tuning
90
- termux-vision vlm sample.jpg \
91
- -p "Detailed object inspection" \
92
- --device auto \
93
- --threads 4 \
94
- --temp 0.7 \
95
- --top-p 0.9 \
96
- --top-k 40 \
97
- --repeat-penalty 1.1 \
98
- --seed 42 \
99
- --system-prompt "You are a professional image analyst."
49
+ npm install -g termux-vision && pkg install termux-llamacpp
100
50
  ```
101
-
102
- #### Python SDK Usage:
103
- ```python
104
- import termux_vision as tv
105
-
106
- with tv.vlm.load(model_id="smolvlm-500m-q4", device="auto") as engine:
107
- result = engine.describe(
108
- "sample.jpg",
109
- prompt="Explain what is in this image.",
110
- temperature=0.7,
111
- top_p=0.9,
112
- max_tokens=200
113
- )
114
- print(f"Output: {result.text}")
115
- print(f"Speed: {result.metrics.tokens_per_second} t/s")
51
+ ```typescript
52
+ import tv from 'termux-vision';
53
+
54
+ // 1. Diagnostics & Hardware Probe
55
+ const doc = tv.doctor(true);
56
+ console.log(`Vulkan GPU: ${doc.vulkan.status} | Cores: ${doc.hardware.cpuCores}`);
57
+
58
+ // 2. Multimodal VLM Inference
59
+ const engine = await tv.load({ modelId: 'smolvlm-500m-q4' });
60
+ const result = await engine.describe('photo.jpg', { prompt: 'What is inside this image?' });
61
+ console.log(`[${result.metrics.backend.toUpperCase()}] ${result.text}`);
62
+ engine.close();
116
63
  ```
117
64
 
118
- #### Node.js / TypeScript SDK Usage:
119
- ```javascript
120
- const tv = require('termux-vision');
121
-
122
- async function main() {
123
- const engine = await tv.vlm.load({ modelId: 'smolvlm-500m-q4', device: 'auto' });
124
- const result = await engine.describe('sample.jpg', {
125
- prompt: 'Explain what is in this image.',
126
- temperature: 0.7,
127
- maxTokens: 200
128
- });
129
- console.log(`Output: ${result.text}`);
130
- }
131
- main();
132
- ```
65
+ ---
66
+
67
+ ## Official Documentation & Benchmarks
68
+ - [Official Architecture & API Reference](https://uno-km.vercel.app/lib/vision/)
69
+ - [Ecosystem Metrics & Registry Stats](https://uno-km.vercel.app/foundation/metrics)
70
+ - [AMEVA Open-Source Foundation Portal](https://uno-km.vercel.app/foundation/index.html)
133
71
 
134
72
  ---
135
73
 
136
- ## 🛡️ 0-Point Baseline Granular Audit Scorecard
137
-
138
- `termux-vision` is rigorously tested under a 0-Point Baseline Granular Scoring Protocol:
139
-
140
- ```text
141
- ================================================================================
142
- AUDIT SCORECARD: termux-vision Dual Release v0.2.0-alpha.1
143
- ================================================================================
144
- [Category: IO & Transforms] : 25.0 / 25.0 pts (Verified)
145
- [Category: Classical CV & Filters] : 25.0 / 25.0 pts (Verified)
146
- [Category: Detection & Haar Cascade]: 25.0 / 25.0 pts (Verified)
147
- [Category: Neural Bridge & Models] : 25.0 / 25.0 pts (Verified)
148
- --------------------------------------------------------------------------------
149
- TOTAL AUDIT SCORE : 100.0 / 100.0 (Grade A+ PERFECT)
150
- ================================================================================
151
- ```
74
+ ## License
75
+ Licensed under the Apache-2.0 License. Copyright (c) 2026 Eunho Kim ([@uno-km](https://github.com/uno-km)).
package/README.pypi.md ADDED
@@ -0,0 +1,35 @@
1
+ # Termux-Vision (Python)
2
+
3
+ [![PyPI](https://img.shields.io/pypi/v/termux-vision.svg?style=flat-square&color=0369a1)](https://pypi.org/project/termux-vision/)
4
+ [![Python](https://img.shields.io/pypi/pyversions/termux-vision.svg?style=flat-square)](https://pypi.org/project/termux-vision/)
5
+ [![License](https://img.shields.io/badge/License-Apache_2.0-004499.svg?style=flat-square)](https://github.com/uno-km/termux-vision)
6
+
7
+ > **안드로이드 Termux를 위한 제로 디펜던시 온디바이스 컴퓨터 비전 & VLM 멀티모달 추론 엔진**
8
+ > *Zero-Dependency On-Device Computer Vision & Multimodal VLM Inference Engine for Android Termux*
9
+
10
+ ## Installation
11
+
12
+ ```bash
13
+ pip install termux-vision
14
+ ```
15
+
16
+ ## Quickstart
17
+
18
+ ```python
19
+ import termux_vision as tv
20
+ img = tv.io.load_image("photo.jpg")
21
+ edges = tv.cv.canny(tv.transforms.to_grayscale(img), 40, 120)
22
+ with tv.vlm.load("qwen2-vl-2b-q4", quality="optimal") as engine:
23
+ res = engine.describe("photo.jpg", prompt="Describe this scene in detail.", quality="optimal")
24
+ print(f"Generated ({res.metrics.tokens_per_second:.1f} t/s): {res.text}")
25
+ ```
26
+
27
+ ## Description
28
+ Eliminates heavy C++ dependencies by integrating SIMD NEON spatial image transforms with on-device VLM (Qwen2-VL, SmolVLM) multi-tier resolution presets (fast, optimal, high, original) and Vulkan GPU acceleration under strict memory isolation.
29
+
30
+ ## Documentation
31
+ - [Official Documentation & API Reference](https://uno-km.vercel.app/lib/vision/)
32
+ - [GitHub Repository](https://github.com/uno-km/termux-vision)
33
+
34
+ ## License
35
+ Apache-2.0 License. Copyright (c) 2026 Eunho Kim (@uno-km).
package/bin/cli.js CHANGED
@@ -229,6 +229,9 @@ async function main() {
229
229
  const repeatPenalty = getArg('--repeat-penalty', null);
230
230
  const seed = getArg('--seed', null);
231
231
  const systemPrompt = getArg('--system-prompt', null);
232
+ const ctxSize = getArg('-c', '--ctx-size');
233
+ const quality = getArg('-q', '--quality') || 'optimal';
234
+ const maxDim = getArg('--max-dim', null);
232
235
  const ngl = getArg('--ngl', null);
233
236
  const isJson = hasFlag('--json');
234
237
 
@@ -273,6 +276,7 @@ async function main() {
273
276
  runtimePath: runtime,
274
277
  allowDownload: allowDownload,
275
278
  threads: threads ? parseInt(threads, 10) : 4,
279
+ contextLimit: ctxSize ? parseInt(ctxSize, 10) : null,
276
280
  ngl: ngl ? parseInt(ngl, 10) : null
277
281
  });
278
282
 
@@ -284,7 +288,9 @@ async function main() {
284
288
  topK: topK ? parseInt(topK, 10) : undefined,
285
289
  repeatPenalty: repeatPenalty ? parseFloat(repeatPenalty) : undefined,
286
290
  seed: seed ? parseInt(seed, 10) : undefined,
287
- systemPrompt: systemPrompt || undefined
291
+ systemPrompt: systemPrompt || undefined,
292
+ quality: quality,
293
+ maxDim: maxDim
288
294
  });
289
295
 
290
296
  if (isJson) {
@@ -326,6 +332,35 @@ async function main() {
326
332
  }
327
333
  }
328
334
 
335
+ if (command === 'canny') {
336
+ const imagePath = args[1];
337
+ if (!imagePath || imagePath.startsWith('-')) {
338
+ console.error('[ERROR] Missing input image path for Canny edge detection.');
339
+ console.error('Usage: termux-vision canny <image_path> [options]');
340
+ process.exit(2);
341
+ }
342
+ const resolvedImg = path.resolve(imagePath.replace(/^~(?=$|\/|\\)/, os.homedir()));
343
+ if (!fs.existsSync(resolvedImg)) {
344
+ console.error(`[ERROR] Image file not found: '${resolvedImg}'`);
345
+ process.exit(2);
346
+ }
347
+ const outPath = getArg('-o', '--output') || 'edges.png';
348
+ const low = parseFloat(getArg('--low') || '40.0');
349
+ const high = parseFloat(getArg('--high') || '120.0');
350
+
351
+ // Execute Canny via Python bridge or fallback
352
+ const { spawnSync } = require('child_process');
353
+ const pyRes = spawnSync('python3', [
354
+ '-m', 'termux_vision.cli.main', 'canny', resolvedImg, '-o', outPath, '--low', String(low), '--high', String(high)
355
+ ], { stdio: 'inherit' });
356
+
357
+ if (pyRes.status === 0) {
358
+ process.exit(0);
359
+ } else {
360
+ process.exit(pyRes.status || 1);
361
+ }
362
+ }
363
+
329
364
  if (command === 'benchmark') {
330
365
  console.log('=== termux-vision On-Device Benchmark (Node.js Engine) ===');
331
366
  const width = 256;
package/index.js CHANGED
@@ -15,12 +15,15 @@ const detect = require('./lib/detect');
15
15
 
16
16
  const packageJson = require('./package.json');
17
17
  const version = packageJson.version;
18
+ const cacheManager = new cache.ModelCacheManager();
19
+ cacheManager.ModelCacheManager = cache.ModelCacheManager;
18
20
 
19
21
  module.exports = {
20
22
  version,
21
23
  __version__: version,
22
24
  errors,
23
- cache,
25
+ cache: cacheManager,
26
+ ModelCacheManager: cache.ModelCacheManager,
24
27
  vlm,
25
28
  load: vlm.load,
26
29
  doctor: doctor.runDoctor,
package/lib/cache.js CHANGED
@@ -46,7 +46,7 @@ const CATALOG = {
46
46
  adapter: 'qwen2vl',
47
47
  tier: 'L',
48
48
  estimatedMemoryMb: 2100,
49
- contextLimit: 1024,
49
+ contextLimit: 8192,
50
50
  preferredResolution: 384,
51
51
  artifacts: [
52
52
  {
@@ -81,6 +81,15 @@ class ModelCacheManager {
81
81
  }
82
82
 
83
83
  isModelInstalled(modelId) {
84
+ if (CATALOG[modelId] && fs.existsSync(this.legacyCache)) {
85
+ const cat = CATALOG[modelId];
86
+ const allExist = cat.artifacts.every(a => {
87
+ const fp = path.join(this.legacyCache, a.filename);
88
+ return fs.existsSync(fp) && fs.statSync(fp).size > 10000000;
89
+ });
90
+ if (allExist) return true;
91
+ }
92
+
84
93
  const mdir = path.join(this.modelsDir, modelId);
85
94
  if (!fs.existsSync(mdir)) return false;
86
95
 
@@ -90,7 +99,7 @@ class ModelCacheManager {
90
99
 
91
100
  if (fs.statSync(mdir).isDirectory()) {
92
101
  const files = fs.readdirSync(mdir);
93
- const ggufs = files.filter(f => f.endsWith('.gguf'));
102
+ const ggufs = files.filter(f => f.endsWith('.gguf') && fs.statSync(path.join(mdir, f)).size > 0);
94
103
  const visionGgufs = ggufs.filter(f => /mmproj|encoder|projector/i.test(f));
95
104
  const textGgufs = ggufs.filter(f => !/mmproj|encoder|projector/i.test(f));
96
105
  if ((textGgufs.length > 0 && visionGgufs.length > 0) || ggufs.length >= 2) {
@@ -129,7 +138,10 @@ class ModelCacheManager {
129
138
  if (fs.existsSync(this.legacyCache)) {
130
139
  for (const [key, val] of Object.entries(CATALOG)) {
131
140
  if (!results.find(r => r.modelId === key)) {
132
- const allFound = val.artifacts.every(a => fs.existsSync(path.join(this.legacyCache, a.filename)));
141
+ const allFound = val.artifacts.every(a => {
142
+ const fpath = path.join(this.legacyCache, a.filename);
143
+ return fs.existsSync(fpath) && fs.statSync(fpath).size > 10000000;
144
+ });
133
145
  if (allFound) {
134
146
  results.push({
135
147
  modelId: key,
@@ -187,12 +199,17 @@ class ModelCacheManager {
187
199
 
188
200
  if (CATALOG[modelId]) {
189
201
  const cat = CATALOG[modelId];
190
- for (const a of cat.artifacts) {
191
- const p = path.join(mdir, a.filename);
192
- if (fs.existsSync(p)) {
193
- if (a.role === 'language_model') textPath = p;
194
- if (a.role === 'vision_projector') visionPath = p;
202
+ const searchDirs = [mdir, this.legacyCache];
203
+ for (const sDir of searchDirs) {
204
+ if (!fs.existsSync(sDir)) continue;
205
+ for (const a of cat.artifacts) {
206
+ const p = path.join(sDir, a.filename);
207
+ if (fs.existsSync(p)) {
208
+ if (a.role === 'language_model') textPath = p;
209
+ if (a.role === 'vision_projector') visionPath = p;
210
+ }
195
211
  }
212
+ if (textPath && visionPath) break;
196
213
  }
197
214
  return {
198
215
  manifest: cat,
package/lib/cv.js CHANGED
@@ -5,10 +5,37 @@
5
5
 
6
6
  'use strict';
7
7
 
8
+ function gaussianBlur(grayPixels, width, height) {
9
+ const output = new Float32Array(width * height);
10
+ // 5x5 normalized Gaussian kernel with sigma=1.4
11
+ const kernel = [
12
+ 2/159, 4/159, 5/159, 4/159, 2/159,
13
+ 4/159, 9/159, 12/159, 9/159, 4/159,
14
+ 5/159, 12/159, 15/159, 12/159, 5/159,
15
+ 4/159, 9/159, 12/159, 9/159, 4/159,
16
+ 2/159, 4/159, 5/159, 4/159, 2/159
17
+ ];
18
+
19
+ for (let y = 2; y < height - 2; y++) {
20
+ for (let x = 2; x < width - 2; x++) {
21
+ let sum = 0;
22
+ let k = 0;
23
+ for (let ky = -2; ky <= 2; ky++) {
24
+ for (let kx = -2; kx <= 2; kx++) {
25
+ sum += grayPixels[(y + ky) * width + (x + kx)] * kernel[k++];
26
+ }
27
+ }
28
+ output[y * width + x] = sum;
29
+ }
30
+ }
31
+ return output;
32
+ }
33
+
8
34
  function sobel(grayPixels, width, height) {
9
35
  const gradX = new Float32Array(width * height);
10
36
  const gradY = new Float32Array(width * height);
11
37
  const magnitude = new Float32Array(width * height);
38
+ const angleDeg = new Float32Array(width * height);
12
39
 
13
40
  for (let y = 1; y < height - 1; y++) {
14
41
  for (let x = 1; x < width - 1; x++) {
@@ -25,30 +52,91 @@ function sobel(grayPixels, width, height) {
25
52
  gradX[idx] = gx;
26
53
  gradY[idx] = gy;
27
54
  magnitude[idx] = Math.sqrt(gx * gx + gy * gy);
55
+ let deg = Math.atan2(gy, gx) * (180.0 / Math.PI);
56
+ if (deg < 0) deg += 180.0;
57
+ angleDeg[idx] = deg;
28
58
  }
29
59
  }
30
60
 
31
- return { gradX, gradY, magnitude };
61
+ return { gradX, gradY, magnitude, angleDeg };
32
62
  }
33
63
 
34
64
  function canny(grayPixels, width, height, lowThreshold = 40.0, highThreshold = 120.0) {
35
- const { magnitude } = sobel(grayPixels, width, height);
36
- const edges = new Uint8Array(width * height);
37
-
38
- for (let i = 0; i < magnitude.length; i++) {
39
- if (magnitude[i] >= highThreshold) {
40
- edges[i] = 255;
41
- } else if (magnitude[i] >= lowThreshold) {
42
- edges[i] = 128;
43
- } else {
44
- edges[i] = 0;
65
+ const blurred = gaussianBlur(grayPixels, width, height);
66
+ const { magnitude, angleDeg } = sobel(blurred, width, height);
67
+ const dst = new Uint8Array(width * height);
68
+ const queue = [];
69
+
70
+ // 1. NMS & Double Thresholding
71
+ for (let y = 1; y < height - 1; y++) {
72
+ for (let x = 1; x < width - 1; x++) {
73
+ const idx = y * width + x;
74
+ const c = magnitude[idx];
75
+ if (c < lowThreshold) continue;
76
+
77
+ const deg = angleDeg[idx];
78
+ let p1 = 0.0, p2 = 0.0;
79
+
80
+ if ((deg >= 0 && deg < 22.5) || (deg >= 157.5 && deg <= 180)) {
81
+ p1 = magnitude[y * width + (x - 1)];
82
+ p2 = magnitude[y * width + (x + 1)];
83
+ } else if (deg >= 22.5 && deg < 67.5) {
84
+ p1 = magnitude[(y - 1) * width + (x + 1)];
85
+ p2 = magnitude[(y + 1) * width + (x - 1)];
86
+ } else if (deg >= 67.5 && deg < 112.5) {
87
+ p1 = magnitude[(y - 1) * width + x];
88
+ p2 = magnitude[(y + 1) * width + x];
89
+ } else {
90
+ p1 = magnitude[(y - 1) * width + (x - 1)];
91
+ p2 = magnitude[(y + 1) * width + (x + 1)];
92
+ }
93
+
94
+ if (c >= p1 && c >= p2) {
95
+ if (c >= highThreshold) {
96
+ dst[idx] = 255;
97
+ queue.push(idx);
98
+ } else {
99
+ dst[idx] = 75; // Weak edge candidate
100
+ }
101
+ }
102
+ }
103
+ }
104
+
105
+ // 2. 8-connected BFS Hysteresis Tracking
106
+ const dx = [-1, 0, 1, -1, 1, -1, 0, 1];
107
+ const dy = [-1, -1, -1, 0, 0, 1, 1, 1];
108
+ let qHead = 0;
109
+
110
+ while (qHead < queue.length) {
111
+ const curr = queue[qHead++];
112
+ const cx = curr % width;
113
+ const cy = Math.floor(curr / width);
114
+
115
+ for (let k = 0; k < 8; k++) {
116
+ const nx = cx + dx[k];
117
+ const ny = cy + dy[k];
118
+ if (nx >= 0 && nx < width && ny >= 0 && ny < height) {
119
+ const nIdx = ny * width + nx;
120
+ if (dst[nIdx] === 75) {
121
+ dst[nIdx] = 255;
122
+ queue.push(nIdx);
123
+ }
124
+ }
125
+ }
126
+ }
127
+
128
+ // 3. Suppress remaining unconnected weak edges
129
+ for (let i = 0; i < dst.length; i++) {
130
+ if (dst[i] === 75) {
131
+ dst[i] = 0;
45
132
  }
46
133
  }
47
134
 
48
- return edges;
135
+ return dst;
49
136
  }
50
137
 
51
138
  module.exports = {
139
+ gaussianBlur,
52
140
  sobel,
53
141
  canny
54
142
  };
package/lib/detect.js CHANGED
@@ -41,7 +41,91 @@ function nms(boxes, iouThreshold = 0.45) {
41
41
  return keep;
42
42
  }
43
43
 
44
+ function computeIntegralImage(grayPixels, width, height) {
45
+ const stride = width + 1;
46
+ const integral = new Float64Array((width + 1) * (height + 1));
47
+
48
+ for (let y = 0; y < height; y++) {
49
+ let rowSum = 0.0;
50
+ const dstRow = (y + 1) * stride;
51
+ const srcRow = y * width;
52
+
53
+ for (let x = 0; x < width; x++) {
54
+ rowSum += grayPixels[srcRow + x];
55
+ integral[dstRow + x + 1] = integral[dstRow - stride + x + 1] + rowSum;
56
+ }
57
+ }
58
+ return integral;
59
+ }
60
+
61
+ function boxSum(integral, stride, x1, y1, x2, y2) {
62
+ return (
63
+ integral[y2 * stride + x2] -
64
+ integral[y1 * stride + x2] -
65
+ integral[y2 * stride + x1] +
66
+ integral[y1 * stride + x1]
67
+ );
68
+ }
69
+
70
+ function detectFaces(grayPixels, width, height, options = {}) {
71
+ const scaleFactor = options.scaleFactor || 1.2;
72
+ const minSize = options.minSize || 24;
73
+ const maxSize = options.maxSize || Math.min(width, height);
74
+ const integral = computeIntegralImage(grayPixels, width, height);
75
+ const stride = width + 1;
76
+
77
+ const candidates = [];
78
+ let scale = minSize / 24.0;
79
+
80
+ while (true) {
81
+ const winW = Math.floor(24 * scale);
82
+ const winH = Math.floor(24 * scale);
83
+ if (winW > width || winH > height || winW > maxSize || winH > maxSize) break;
84
+
85
+ const step = Math.max(2, Math.floor(4 * scale));
86
+ const s2 = scale * scale;
87
+
88
+ for (let y = 0; y <= height - winH; y += step) {
89
+ for (let x = 0; x <= width - winW; x += step) {
90
+ // Stage 1: Eye darkness vs cheek
91
+ const eX = x + Math.floor(2 * scale), eY = y + Math.floor(6 * scale), eW = Math.floor(20 * scale), eH = Math.floor(6 * scale);
92
+ const eyes = boxSum(integral, stride, eX, eY, eX + eW, eY + eH);
93
+ const cX = x + Math.floor(2 * scale), cY = y + Math.floor(12 * scale), cW = Math.floor(20 * scale), cH = Math.floor(6 * scale);
94
+ const cheeks = boxSum(integral, stride, cX, cY, cX + cW, cY + cH);
95
+ const val1 = -1.0 * eyes + 1.0 * cheeks;
96
+ if (val1 < -10.0 * s2 ? -1.0 : 1.2 < 0.2) continue;
97
+
98
+ // Stage 2: Nose bridge
99
+ const nX = x + Math.floor(10 * scale), nY = y + Math.floor(6 * scale), nW = Math.floor(4 * scale), nH = Math.floor(12 * scale);
100
+ const nose = boxSum(integral, stride, nX, nY, nX + nW, nY + nH);
101
+ const lX = x + Math.floor(6 * scale), rX = x + Math.floor(14 * scale);
102
+ const lCheek = boxSum(integral, stride, lX, nY, lX + nW, nY + nH);
103
+ const rCheek = boxSum(integral, stride, rX, nY, rX + nW, nY + nH);
104
+ const val2 = -1.0 * lCheek + 2.0 * nose - 1.0 * rCheek;
105
+ if (val2 < -5.0 * s2 ? -0.8 : 1.5 < 0.3) continue;
106
+
107
+ const area = winW * winH;
108
+ const meanScore = boxSum(integral, stride, x, y, x + winW, y + winH) / (area || 1);
109
+ candidates.push({
110
+ x,
111
+ y,
112
+ width: winW,
113
+ height: winH,
114
+ score: meanScore,
115
+ className: 'face_candidate'
116
+ });
117
+ }
118
+ }
119
+ scale *= scaleFactor;
120
+ }
121
+
122
+ return nms(candidates, 0.3);
123
+ }
124
+
44
125
  module.exports = {
45
126
  computeIoU,
46
- nms
127
+ nms,
128
+ computeIntegralImage,
129
+ boxSum,
130
+ detectFaces
47
131
  };
package/lib/doctor.js CHANGED
@@ -39,15 +39,26 @@ function runDoctor(probeVulkan = false) {
39
39
  const driverDetected = fs.existsSync('/vendor/lib64/hw/vulkan.adreno.so') || fs.existsSync('/vendor/lib64/hw/vulkan.mali.so');
40
40
 
41
41
  let vulkanStatus = 'unverified';
42
- if (probeVulkan) {
43
- vulkanStatus = (loaderDetected && driverDetected) ? 'driver_detected_experimental' : 'disabled';
42
+ let amevaRuntimeDetected = false;
43
+ try {
44
+ const avr = require('ameva-vulkan-runtime');
45
+ amevaRuntimeDetected = true;
46
+ if (probeVulkan) {
47
+ vulkanStatus = (typeof avr.isAvailable === 'function' && avr.isAvailable()) ? 'driver_detected_experimental' : 'disabled';
48
+ }
49
+ } catch (err) {
50
+ if (probeVulkan) {
51
+ vulkanStatus = (loaderDetected && driverDetected) ? 'driver_detected_experimental' : 'disabled';
52
+ }
44
53
  }
45
54
 
55
+ const packageJson = require('../package.json');
46
56
  const installed = cache.listInstalled();
47
57
 
48
58
  return {
49
59
  schemaVersion: 1,
50
- clientVersion: '0.2.0-alpha.1',
60
+ clientVersion: packageJson.version,
61
+ runtimeVersion: packageJson.version,
51
62
  platform: {
52
63
  system: os.type(),
53
64
  machine: os.arch(),
package/lib/vlm.js CHANGED
@@ -46,10 +46,12 @@ function resolveLlamaCli(explicitPath = null) {
46
46
 
47
47
  const prefix = process.env.PREFIX || '/data/data/com.termux/files/usr';
48
48
  const candidates = [
49
+ path.join(os.homedir(), '.termux-llama', 'current', 'bin', 'llama-cli'),
50
+ path.join(os.homedir(), '.termux-llamacpp', 'current', 'bin', 'llama-cli'),
49
51
  'llama-cli',
50
52
  path.join(prefix, 'bin', 'llama-cli'),
51
53
  path.join(prefix, 'bin', 'termux-llama-cli'),
52
- path.join(os.homedir(), '.termux-llamacpp', 'current', 'bin', 'llama-cli'),
54
+ path.join(prefix, 'bin', 'llama-mtmd-cli'),
53
55
  path.join(os.homedir(), '.local', 'bin', 'llama-cli'),
54
56
  path.join(os.homedir(), 'bin', 'llama-cli')
55
57
  ];
@@ -74,7 +76,7 @@ class NodeVLMContext {
74
76
  this.visionModelPath = options.visionModelPath;
75
77
  this.executable = options.executable;
76
78
  this.threads = options.threads || 4;
77
- this.backend = options.backend || 'cpu';
79
+ this.backend = options.backend || 'auto';
78
80
  this.fallback = options.fallback !== false;
79
81
  this.contextLimit = options.contextLimit || options.manifest.contextLimit || 1024;
80
82
  this.customNgl = options.ngl;
@@ -82,7 +84,7 @@ class NodeVLMContext {
82
84
  }
83
85
 
84
86
  async _executeOnce(imagePath, prompt, options, targetBackend) {
85
- const nglVal = this.customNgl !== undefined && this.customNgl !== null ? String(this.customNgl) : (targetBackend === 'vulkan' ? '99' : '0');
87
+ const nglVal = this.customNgl !== undefined && this.customNgl !== null ? String(this.customNgl) : (['vulkan', 'auto', 'gpu'].includes(targetBackend) ? '99' : '0');
86
88
  const tmpPrompt = path.join(os.tmpdir(), `tv_prompt_${Date.now()}_${Math.random().toString(36).substring(7)}.txt`);
87
89
 
88
90
  const formattedPrompt = options.systemPrompt ?
@@ -95,14 +97,19 @@ class NodeVLMContext {
95
97
  '--mmproj', this.visionModelPath,
96
98
  '--image', imagePath,
97
99
  '-f', tmpPrompt,
98
- '-st',
99
100
  '-t', String(this.threads),
100
101
  '-c', String(this.contextLimit),
101
102
  '-n', String(options.maxTokens || 150),
102
103
  '--temp', String(options.temperature !== undefined ? options.temperature : 0.2),
103
- '-ngl', nglVal
104
+ '-ngl', nglVal,
105
+ '--single-turn',
106
+ '--simple-io'
104
107
  ];
105
108
 
109
+ if (['auto', 'vulkan', 'gpu'].includes(targetBackend)) {
110
+ cliArgs.push('--device', targetBackend === 'auto' ? 'auto' : 'vulkan');
111
+ }
112
+
106
113
  if (options.repeatPenalty !== undefined) cliArgs.push('--repeat-penalty', String(options.repeatPenalty));
107
114
  if (options.topP !== undefined) cliArgs.push('--top-p', String(options.topP));
108
115
  if (options.topK !== undefined) cliArgs.push('--top-k', String(options.topK));
@@ -208,8 +215,42 @@ class NodeVLMContext {
208
215
  throw new Error(`Parameter 'topP' must be between (0.0, 1.0]. Received: ${options.topP}`);
209
216
  }
210
217
 
218
+ let targetImg = resolvedImg;
219
+ let tempImg = null;
220
+ const quality = (options.quality || 'optimal').toLowerCase().trim();
221
+ const maxDim = options.maxDim ? parseInt(options.maxDim, 10) : null;
222
+
223
+ if (quality !== 'original' || maxDim !== null) {
224
+ const limit = maxDim || (quality === 'high' ? 1280 : (quality === 'fast' ? 384 : 768));
225
+ try {
226
+ const { spawnSync } = require('child_process');
227
+ const tmpTarget = path.join(os.tmpdir(), `tv_scaled_${Date.now()}_${Math.random().toString(36).substring(7)}.jpg`);
228
+ const scalePyCode = 'import sys, termux_vision as tv; img = tv.io.load_image(sys.argv[1]); dims = tv.transforms.resolve_target_dimensions(img.shape[1], img.shape[0], max_dim=int(sys.argv[3])) or (img.shape[1], img.shape[0]); scaled = tv.transforms.resize(img, dims); tv.io.save_image(scaled, sys.argv[2])';
229
+ let scaleRes = spawnSync('python3', [
230
+ '-c',
231
+ scalePyCode,
232
+ resolvedImg,
233
+ tmpTarget,
234
+ String(limit)
235
+ ]);
236
+ if (scaleRes.status !== 0 || !fs.existsSync(tmpTarget)) {
237
+ scaleRes = spawnSync('python', [
238
+ '-c',
239
+ scalePyCode,
240
+ resolvedImg,
241
+ tmpTarget,
242
+ String(limit)
243
+ ]);
244
+ }
245
+ if (scaleRes.status === 0 && fs.existsSync(tmpTarget)) {
246
+ targetImg = tmpTarget;
247
+ tempImg = tmpTarget;
248
+ }
249
+ } catch (e) {}
250
+ }
251
+
211
252
  try {
212
- return await this._executeOnce(resolvedImg, prompt, options, this.backend);
253
+ return await this._executeOnce(targetImg, prompt, options, this.backend);
213
254
  } catch (err) {
214
255
  if (this.backend === 'vulkan') {
215
256
  if (this.fallback && isVulkanFailure(err.message)) {
@@ -221,6 +262,10 @@ class NodeVLMContext {
221
262
  }
222
263
  }
223
264
  throw err;
265
+ } finally {
266
+ if (tempImg && fs.existsSync(tempImg)) {
267
+ try { fs.unlinkSync(tempImg); } catch (e) {}
268
+ }
224
269
  }
225
270
  }
226
271
 
@@ -255,7 +300,7 @@ async function load(options = {}) {
255
300
  let actualFallback = false;
256
301
 
257
302
  if (reqDevice === 'auto') {
258
- actualBackend = 'vulkan';
303
+ actualBackend = 'auto';
259
304
  actualFallback = true;
260
305
  } else if (['vulkan', 'gpu', 'vulkan-force'].includes(reqDevice)) {
261
306
  actualBackend = 'vulkan';
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "termux-vision",
3
- "version": "1.0.0",
3
+ "version": "1.1.0",
4
4
  "description": "Native On-Device Computer Vision & VLM Multimodal Inference Framework for Android Termux & ARM64 (Dual-Engine Python & Node.js/TypeScript)",
5
5
  "main": "index.js",
6
6
  "types": "index.d.ts",
@@ -49,6 +49,9 @@
49
49
  "bugs": {
50
50
  "url": "https://github.com/uno-km/termux-vision/issues"
51
51
  },
52
+ "dependencies": {
53
+ "ameva-vulkan-runtime": ">=1.0.0"
54
+ },
52
55
  "engines": {
53
56
  "node": ">=16.0.0"
54
57
  }