ripgrep_wasm 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.npmignore +28 -0
- data/IMPLEMENTATION.md +103 -0
- data/RALPH_TASK.md +198 -0
- data/README.md +123 -0
- data/Rakefile +27 -0
- data/SECURITY.md +86 -0
- data/index.js +24 -0
- data/lib/download.js +201 -0
- data/lib/ripgrep_wasm/downloader.rb +142 -0
- data/lib/ripgrep_wasm/rg.wasm +0 -0
- data/lib/ripgrep_wasm/version.rb +5 -0
- data/lib/ripgrep_wasm.rb +36 -0
- data/package.json +38 -0
- data/rg.wasm +0 -0
- data/ripgrep_wasm.gemspec +47 -0
- data/test.txt +3 -0
- metadata +69 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 48e7d7bd34377c527f6b51b649ac6e997f058d0a2d8135c590650fb16c986489
|
|
4
|
+
data.tar.gz: dd19a052210eba94d85fbe5e46d703a6ce6d8a339e59256410fc5cce389bb656
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 3c5dcb66c0aa34f49d0eb379e92984ef20e99f3c00220eb493cd40aed75cd58c4eeebdd0e0555c18ea65aa2c5a3699b2568076375f7c626245eb0a913de0cf7a
|
|
7
|
+
data.tar.gz: 81934964911d5905b6cf5ca727de8902a440e7af00e0d447d18d8121771b28d7523e55bb2f56e267b609e5a0a3964d0164b41ad605e105292d47fa3fd8fcecb6
|
data/.npmignore
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Build outputs
|
|
2
|
+
ripgrep/
|
|
3
|
+
*.txt
|
|
4
|
+
|
|
5
|
+
# Development files
|
|
6
|
+
.ralph/
|
|
7
|
+
.cursor/
|
|
8
|
+
.vscode/
|
|
9
|
+
*.swp
|
|
10
|
+
*~
|
|
11
|
+
|
|
12
|
+
# macOS
|
|
13
|
+
.DS_Store
|
|
14
|
+
|
|
15
|
+
# Git
|
|
16
|
+
.git/
|
|
17
|
+
.gitignore
|
|
18
|
+
|
|
19
|
+
# Documentation (keep README.md)
|
|
20
|
+
SECURITY.md
|
|
21
|
+
IMPLEMENTATION.md
|
|
22
|
+
RALPH_TASK.md
|
|
23
|
+
|
|
24
|
+
# Ruby gem files (not needed in npm package)
|
|
25
|
+
*.gemspec
|
|
26
|
+
Rakefile
|
|
27
|
+
lib/ripgrep_wasm/
|
|
28
|
+
lib/ripgrep_wasm.rb
|
data/IMPLEMENTATION.md
ADDED
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# Implementation Notes
|
|
2
|
+
|
|
3
|
+
## Compilation Process
|
|
4
|
+
|
|
5
|
+
### Target Selection
|
|
6
|
+
|
|
7
|
+
We use `wasm32-wasip1` (previously called `wasm32-wasi`), the WebAssembly System Interface target. This provides:
|
|
8
|
+
- File system access through WASI APIs
|
|
9
|
+
- Standard I/O (stdin, stdout, stderr)
|
|
10
|
+
- Environment variables
|
|
11
|
+
- Command-line arguments
|
|
12
|
+
|
|
13
|
+
### Build Configuration
|
|
14
|
+
|
|
15
|
+
The default build works without modifications:
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
cargo build --release --target wasm32-wasip1
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
No features need to be disabled. Ripgrep has excellent WASI support built-in.
|
|
22
|
+
|
|
23
|
+
## Trade-offs and Design Decisions
|
|
24
|
+
|
|
25
|
+
### 1. No `--no-default-features` Required
|
|
26
|
+
|
|
27
|
+
Unlike some Rust projects, ripgrep compiles cleanly to WASI without disabling features. The crate handles platform differences internally.
|
|
28
|
+
|
|
29
|
+
### 2. Memory-Mapped Files
|
|
30
|
+
|
|
31
|
+
The native version uses `mmap` for fast file reading. The WASI version falls back to standard file I/O:
|
|
32
|
+
- Slightly slower for very large files
|
|
33
|
+
- No practical impact for typical search workloads
|
|
34
|
+
- More predictable memory usage
|
|
35
|
+
|
|
36
|
+
### 3. Parallelism
|
|
37
|
+
|
|
38
|
+
Native ripgrep uses parallel directory walking and file searching. The WASI version:
|
|
39
|
+
- Uses sequential processing (WASI doesn't support threading)
|
|
40
|
+
- Still fast due to efficient algorithms
|
|
41
|
+
- Suitable for most use cases
|
|
42
|
+
|
|
43
|
+
### 4. Binary Size
|
|
44
|
+
|
|
45
|
+
| Version | Size |
|
|
46
|
+
|---------|------|
|
|
47
|
+
| Native (arm64) | ~5 MB |
|
|
48
|
+
| WASM | ~19 MB |
|
|
49
|
+
|
|
50
|
+
The WASM binary is larger because:
|
|
51
|
+
- Includes Rust standard library compiled to WASM
|
|
52
|
+
- No dead code elimination across WASM boundaries
|
|
53
|
+
- Debug symbols included for better error messages
|
|
54
|
+
|
|
55
|
+
### 5. Startup Time
|
|
56
|
+
|
|
57
|
+
WASM has ~100-200ms startup overhead for JIT compilation. For single searches this is noticeable. For multiple searches, consider:
|
|
58
|
+
- Using a persistent runtime
|
|
59
|
+
- Pre-compiling the WASM module to native code
|
|
60
|
+
|
|
61
|
+
## Package Distribution Strategy
|
|
62
|
+
|
|
63
|
+
### NPM Package
|
|
64
|
+
|
|
65
|
+
- Binary downloaded from GitHub Releases during `postinstall`
|
|
66
|
+
- Falls back gracefully if release doesn't exist
|
|
67
|
+
- User can manually place binary
|
|
68
|
+
|
|
69
|
+
### Ruby Gem
|
|
70
|
+
|
|
71
|
+
- Binary downloaded on first `RipgrepWasm.path` call
|
|
72
|
+
- Lazy download avoids install-time network requirements
|
|
73
|
+
- Stored in gem's lib directory
|
|
74
|
+
|
|
75
|
+
## Runtime Requirements
|
|
76
|
+
|
|
77
|
+
### WasmTime Flags
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
wasmtime --dir=. rg.wasm ...
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
- `--dir=.`: Required for file system access (WASI sandbox)
|
|
84
|
+
- Additional directories can be mapped: `--dir=/path/to/search`
|
|
85
|
+
|
|
86
|
+
### Memory Limits
|
|
87
|
+
|
|
88
|
+
Default WasmTime memory is sufficient. For very large files, increase with:
|
|
89
|
+
```bash
|
|
90
|
+
wasmtime --max-memory-size=1073741824 --dir=. rg.wasm ...
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## Known Issues
|
|
94
|
+
|
|
95
|
+
1. **Color output in piped mode**: May not work in all environments
|
|
96
|
+
2. **Symlink following**: Limited by WASI capabilities
|
|
97
|
+
3. **Git integration**: Cannot shell out to git for .gitignore parsing (uses internal implementation)
|
|
98
|
+
|
|
99
|
+
## Future Improvements
|
|
100
|
+
|
|
101
|
+
- Pre-compiled native modules for faster startup
|
|
102
|
+
- Component Model support when WASI stabilizes
|
|
103
|
+
- Streaming search API for Node.js/Ruby integration
|
data/RALPH_TASK.md
ADDED
|
@@ -0,0 +1,198 @@
|
|
|
1
|
+
Compilation de Ripgrep pour WASM/WasmTime │ │
|
|
2
|
+
│ │ │ │
|
|
3
|
+
│ │ │ │
|
|
4
|
+
│ │ Objectif │ │ │ │ │ │
|
|
5
|
+
│ │ Compiler Ripgrep (https://github.com/BurntSushi/ripgrep) (outil de │ │
|
|
6
|
+
│ │ recherche Rust) en binaire WebAssembly et l'exécuter avec │ │
|
|
7
|
+
│ │ WasmTime sur le système de fichiers local. Créer un package NPM et │ │
|
|
8
|
+
│ │ une gem Ruby pour distribuer le binaire. │ │
|
|
9
|
+
│ │ │ │
|
|
10
|
+
│ │ Prérequis │ │
|
|
11
|
+
│ │ │ │
|
|
12
|
+
│ │ • Rust avec rustup installé │ │
|
|
13
|
+
│ │ • WasmTime CLI installé │ │
|
|
14
|
+
│ │ • Target WASM: wasm32-wasip1 (anciennement wasm32-wasi) │ │
|
|
15
|
+
│ │ │ │
|
|
16
|
+
│ │ │ │
|
|
17
|
+
│ │ ──────────────────────────────────────── │ │
|
|
18
|
+
│ │ │ │
|
|
19
|
+
│ │ │ │
|
|
20
|
+
│ │ Phase 1 : Compilation WASM │ │
|
|
21
|
+
│ │ │ │
|
|
22
|
+
│ │ │ │
|
|
23
|
+
│ │ 1.1 Installation des dépendances │ │
|
|
24
|
+
│ │ │ │
|
|
25
|
+
│ │ │ │
|
|
26
|
+
│ │ # Installer le target WASI pour Rust │ │
|
|
27
|
+
│ │ rustup target add wasm32-wasip1 │ │
|
|
28
|
+
│ │ # Installer WasmTime (macOS) │ │
|
|
29
|
+
│ │ brew install wasmtime │ │
|
|
30
|
+
│ │ │ │
|
|
31
|
+
│ │ │ │
|
|
32
|
+
│ │ 1.2 Cloner Ripgrep │ │
|
|
33
|
+
│ │ │ │
|
|
34
|
+
│ │ │ │
|
|
35
|
+
│ │ git clone https://github.com/BurntSushi/ripgrep.git │ │
|
|
36
|
+
│ │ cd ripgrep │ │
|
|
37
|
+
│ │ │ │
|
|
38
|
+
│ │ │ │
|
|
39
|
+
│ │ 1.3 Compiler pour WASI │ │
|
|
40
|
+
│ │ │ │
|
|
41
|
+
│ │ │ │
|
|
42
|
+
│ │ cargo build --release --target wasm32-wasip1 │ │
|
|
43
|
+
│ │ │ │
|
|
44
|
+
│ │ Le binaire sera généré dans: target/wasm32-wasip1/release/rg.wasm │ │
|
|
45
|
+
│ │ │ │
|
|
46
|
+
│ │ 1.4 Test de validation │ │
|
|
47
|
+
│ │ │ │
|
|
48
|
+
│ │ │ │
|
|
49
|
+
│ │ # Créer un fichier test │ │
|
|
50
|
+
│ │ echo "Hello World\nTest pattern\nAnother line" > test.txt │ │
|
|
51
|
+
│ │ # Rechercher "pattern" avec ripgrep via WasmTime │ │
|
|
52
|
+
│ │ wasmtime --dir=. ./target/wasm32-wasip1/release/rg.wasm "pattern" │ │
|
|
53
|
+
│ │ test.txt │ │
|
|
54
|
+
│ │ # Sortie attendue: Test pattern │ │
|
|
55
|
+
│ │ │ │
|
|
56
|
+
│ │ │ │
|
|
57
|
+
│ │ ──────────────────────────────────────── │ │
|
|
58
|
+
│ │ │ │
|
|
59
|
+
│ │ │ │
|
|
60
|
+
│ │ Phase 2 : Package NPM │ │
|
|
61
|
+
│ │ │ │
|
|
62
|
+
│ │ S'inspirer de pandoc-wasm : │ │
|
|
63
|
+
│ │ /Users/nathanhimpens/Documents/klara/code/pandoc-wasm/ │ │
|
|
64
|
+
│ │ │ │
|
|
65
|
+
│ │ Structure à créer │ │
|
|
66
|
+
│ │ │ │
|
|
67
|
+
│ │ │ │
|
|
68
|
+
│ │ ripgrep-wasm/ │ │
|
|
69
|
+
│ │ ├── package.json │ │
|
|
70
|
+
│ │ ├── index.js # Exporte le chemin vers rg.wasm │ │
|
|
71
|
+
│ │ ├── lib/ │ │
|
|
72
|
+
│ │ │ └── download.js # Télécharge rg.wasm depuis GitHub Releases │ │
|
|
73
|
+
│ │ └── .npmignore │ │
|
|
74
|
+
│ │ │ │
|
|
75
|
+
│ │ │ │
|
|
76
|
+
│ │ Fichiers clés (inspirés de pandoc-wasm) │ │
|
|
77
|
+
│ │ │ │
|
|
78
|
+
│ │ • package.json : metadata, postinstall hook vers lib/download.js │ │
|
|
79
|
+
│ │ • index.js : exporte le chemin vers rg.wasm │ │
|
|
80
|
+
│ │ • lib/download.js : télécharge le binaire depuis GitHub Releases │ │
|
|
81
|
+
│ │ │ │
|
|
82
|
+
│ │ │ │
|
|
83
|
+
│ │ ──────────────────────────────────────── │ │
|
|
84
|
+
│ │ │ │
|
|
85
|
+
│ │ │ │
|
|
86
|
+
│ │ Phase 3 : Gem Ruby │ │
|
|
87
|
+
│ │ │ │
|
|
88
|
+
│ │ │ │
|
|
89
|
+
│ │ Structure à créer │ │
|
|
90
|
+
│ │ │ │
|
|
91
|
+
│ │ │ │
|
|
92
|
+
│ │ ripgrep-wasm/ │ │
|
|
93
|
+
│ │ ├── ripgrep_wasm.gemspec │ │
|
|
94
|
+
│ │ ├── Rakefile │ │
|
|
95
|
+
│ │ └── lib/ │ │
|
|
96
|
+
│ │ ├── ripgrep_wasm.rb │ │
|
|
97
|
+
│ │ └── ripgrep_wasm/ │ │
|
|
98
|
+
│ │ ├── version.rb │ │
|
|
99
|
+
│ │ └── downloader.rb │ │
|
|
100
|
+
│ │ │ │
|
|
101
|
+
│ │ │ │
|
|
102
|
+
│ │ Fichiers clés (inspirés de pandoc-wasm) │ │
|
|
103
|
+
│ │ │ │
|
|
104
|
+
│ │ • ripgrep_wasm.gemspec : metadata de la gem │ │
|
|
105
|
+
│ │ • lib/ripgrep_wasm.rb : module principal avec RipgrepWasm.path │ │
|
|
106
|
+
│ │ • lib/ripgrep_wasm/downloader.rb : télécharge le binaire depuis │ │
|
|
107
|
+
│ │ GitHub Releases │ │
|
|
108
|
+
│ │ │ │
|
|
109
|
+
│ │ │ │
|
|
110
|
+
│ │ ──────────────────────────────────────── │ │
|
|
111
|
+
│ │ │ │
|
|
112
|
+
│ │ │ │
|
|
113
|
+
│ │ Phase 4 : Documentation │ │
|
|
114
|
+
│ │ │ │
|
|
115
|
+
│ │ Créer les fichiers suivants dans le repo : │ │
|
|
116
|
+
│ │ │ │
|
|
117
|
+
│ │ |Fichier |Contenu | │ │
|
|
118
|
+
│ │ │ │
|
|
119
|
+
│ │ ------------------------------------------------------------------ │ │
|
|
120
|
+
│ │ -| ----------| │ │
|
|
121
|
+
│ │ |README. |Documentation complète (installation, usage, | │ │
|
|
122
|
+
│ │ d compilation) │ │
|
|
123
|
+
│ │ |IMPLEMENTATION |Choix d'implémentation et trade-offs | │ │
|
|
124
|
+
│ │ md │ │
|
|
125
|
+
│ │ |SECURITY. |Implications de sécurité des modifications | │ │
|
|
126
|
+
│ │ d │ │
|
|
127
|
+
│ │ │ │
|
|
128
|
+
│ │ │ │
|
|
129
|
+
│ │ ──────────────────────────────────────── │ │
|
|
130
|
+
│ │ │ │
|
|
131
|
+
│ │ │ │
|
|
132
|
+
│ │ Critères de validation │ │
|
|
133
|
+
│ │ │ │
|
|
134
|
+
│ │ 1. [x] Rust et le target wasm32-wasip1 sont installés │ │
|
|
135
|
+
│ │ 2. [x] WasmTime est installé et fonctionnel │ │
|
|
136
|
+
│ │ 3. [x] Ripgrep est cloné depuis GitHub │ │
|
|
137
|
+
│ │ 4. [x] Compilation réussie vers rg.wasm │ │
|
|
138
|
+
│ │ 5. [x] Exécution réussie avec WasmTime sur un fichier local │ │
|
|
139
|
+
│ │ 6. [x] Package NPM créé et fonctionnel │ │
|
|
140
|
+
│ │ 7. [x] Gem Ruby créée et fonctionnelle │ │
|
|
141
|
+
│ │ 8. [x] Documentation complète (README, IMPLEMENTATION, SECURITY) │ │
|
|
142
|
+
│ │ │ │
|
|
143
|
+
│ │ │ │
|
|
144
|
+
│ │ ──────────────────────────────────────── │ │
|
|
145
|
+
│ │ │ │
|
|
146
|
+
│ │ │ │
|
|
147
|
+
│ │ Exemple de validation finale │ │
|
|
148
|
+
│ │ │ │
|
|
149
|
+
│ │ │ │
|
|
150
|
+
│ │ # Test WASM direct │ │
|
|
151
|
+
│ │ wasmtime --dir=. rg.wasm "TODO" src/ │ │
|
|
152
|
+
│ │ # Test via NPM (après npm install) │ │
|
|
153
|
+
│ │ node -e "console.log(require('./index.js'))" │ │
|
|
154
|
+
│ │ # Test via Ruby (après installation gem) │ │
|
|
155
|
+
│ │ ruby -e "require 'ripgrep_wasm'; puts RipgrepWasm.path" │ │
|
|
156
|
+
│ │ │ │
|
|
157
|
+
│ │ │ │
|
|
158
|
+
│ │ ──────────────────────────────────────── │ │
|
|
159
|
+
│ │ │ │
|
|
160
|
+
│ │ │ │
|
|
161
|
+
│ │ Instructions Ralph │ │
|
|
162
|
+
│ │ │ │
|
|
163
|
+
│ │ │ │
|
|
164
|
+
│ │ Commits réguliers │ │
|
|
165
|
+
│ │ │ │
|
|
166
|
+
│ │ • Commiter après chaque étape significative (compilation │ │
|
|
167
|
+
│ │ réussie, package créé, etc.) │ │
|
|
168
|
+
│ │ • Utiliser des messages de commit descriptifs en français │ │
|
|
169
|
+
│ │ • Ne pas attendre la fin pour commiter │ │
|
|
170
|
+
│ │ │ │
|
|
171
|
+
│ │ │ │
|
|
172
|
+
│ │ Documentation des choix │ │
|
|
173
|
+
│ │ │ │
|
|
174
|
+
│ │ • Documenter tous les trade-offs dans `IMPLEMENTATION.md` au fur │ │
|
|
175
|
+
│ │ et à mesure │ │
|
|
176
|
+
│ │ • Exemples de choix à documenter : │ │
|
|
177
|
+
│ │ • Features Rust activées/désactivées pour WASI │ │
|
|
178
|
+
│ │ • Problèmes de compilation rencontrés et solutions │ │
|
|
179
|
+
│ │ • Différences avec la version native de ripgrep │ │
|
|
180
|
+
│ │ • Limitations WASI découvertes │ │
|
|
181
|
+
│ │ │ │
|
|
182
|
+
│ │ │ │
|
|
183
|
+
│ │ Progression │ │
|
|
184
|
+
│ │ │ │
|
|
185
|
+
│ │ • Mettre à jour .ralph/progress.md régulièrement │ │
|
|
186
|
+
│ │ • Noter les blocages dans .ralph/errors.log │ │
|
|
187
|
+
│ │ │ │
|
|
188
|
+
│ │ │ │
|
|
189
|
+
│ │ ──────────────────────────────────────── │ │
|
|
190
|
+
│ │ │ │
|
|
191
|
+
│ │ │ │
|
|
192
|
+
│ │ Notes importantes │ │
|
|
193
|
+
│ │ │ │
|
|
194
|
+
│ │ • Le flag --dir=. est obligatoire pour WasmTime (sandbox WASI) │ │
|
|
195
|
+
│ │ • Si des erreurs de compilation surviennent (mmap, etc.), │ │
|
|
196
|
+
│ │ désactiver des features avec --no-default-features │ │
|
|
197
|
+
│ │ • Le binaire WASM ne supportera pas toutes les features de │ │
|
|
198
|
+
│ │ ripgrep natif (pas de mmap, limitations I/O)
|
data/README.md
ADDED
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
# ripgrep-wasm
|
|
2
|
+
|
|
3
|
+
Ripgrep 15.1.0 compiled to WebAssembly for fast text search in WASI environments.
|
|
4
|
+
|
|
5
|
+
## Overview
|
|
6
|
+
|
|
7
|
+
This project provides [ripgrep](https://github.com/BurntSushi/ripgrep) (rg), a blazingly fast search tool, compiled to WebAssembly. It can be executed with any WASI-compatible runtime like [WasmTime](https://wasmtime.dev/).
|
|
8
|
+
|
|
9
|
+
## Installation
|
|
10
|
+
|
|
11
|
+
### NPM
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
npm install @nathanhimpens/ripgrep-wasm
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
```javascript
|
|
18
|
+
const rgWasmPath = require('@nathanhimpens/ripgrep-wasm');
|
|
19
|
+
console.log(rgWasmPath); // Path to rg.wasm
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
### Ruby Gem
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
gem install ripgrep_wasm
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
```ruby
|
|
29
|
+
require 'ripgrep_wasm'
|
|
30
|
+
puts RipgrepWasm.path # Path to rg.wasm
|
|
31
|
+
puts RipgrepWasm.available? # true if binary exists
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
### Manual
|
|
35
|
+
|
|
36
|
+
Download `rg.wasm` from [GitHub Releases](https://github.com/NathanHimpens/ripgrep-wasm/releases).
|
|
37
|
+
|
|
38
|
+
## Usage with WasmTime
|
|
39
|
+
|
|
40
|
+
The `--dir=.` flag is required to grant file system access (WASI sandbox):
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
# Search for "pattern" in a file
|
|
44
|
+
wasmtime --dir=. rg.wasm "pattern" file.txt
|
|
45
|
+
|
|
46
|
+
# Search recursively in current directory
|
|
47
|
+
wasmtime --dir=. rg.wasm "TODO" .
|
|
48
|
+
|
|
49
|
+
# Case-insensitive search
|
|
50
|
+
wasmtime --dir=. rg.wasm -i "error" logs/
|
|
51
|
+
|
|
52
|
+
# Show line numbers
|
|
53
|
+
wasmtime --dir=. rg.wasm -n "function" src/
|
|
54
|
+
|
|
55
|
+
# Search with context (2 lines before/after)
|
|
56
|
+
wasmtime --dir=. rg.wasm -C 2 "bug" .
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## Building from Source
|
|
60
|
+
|
|
61
|
+
### Prerequisites
|
|
62
|
+
|
|
63
|
+
- Rust with rustup
|
|
64
|
+
- WasmTime CLI (for testing)
|
|
65
|
+
|
|
66
|
+
### Build Steps
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
# Install WASI target
|
|
70
|
+
rustup target add wasm32-wasip1
|
|
71
|
+
|
|
72
|
+
# Clone ripgrep
|
|
73
|
+
git clone --depth 1 https://github.com/BurntSushi/ripgrep.git
|
|
74
|
+
cd ripgrep
|
|
75
|
+
|
|
76
|
+
# Build for WASM
|
|
77
|
+
cargo build --release --target wasm32-wasip1
|
|
78
|
+
|
|
79
|
+
# Binary location
|
|
80
|
+
ls -la target/wasm32-wasip1/release/rg.wasm
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### Test the Build
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
echo "Hello World\nTest pattern\nAnother line" > test.txt
|
|
87
|
+
wasmtime --dir=. ./target/wasm32-wasip1/release/rg.wasm "pattern" test.txt
|
|
88
|
+
# Output: Test pattern
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
## Features
|
|
92
|
+
|
|
93
|
+
The WASM build includes:
|
|
94
|
+
- Full regex support
|
|
95
|
+
- Unicode support
|
|
96
|
+
- Color output (when terminal supports it)
|
|
97
|
+
- .gitignore respect
|
|
98
|
+
- Multiple file type filters
|
|
99
|
+
- Context lines (-A, -B, -C)
|
|
100
|
+
- Line numbers (-n)
|
|
101
|
+
- Case insensitive search (-i)
|
|
102
|
+
- Inverted matching (-v)
|
|
103
|
+
- Count matches (-c)
|
|
104
|
+
- JSON output (--json)
|
|
105
|
+
|
|
106
|
+
## Limitations
|
|
107
|
+
|
|
108
|
+
Compared to native ripgrep:
|
|
109
|
+
- **No memory-mapped files**: All file I/O is through standard read/write
|
|
110
|
+
- **Single-threaded**: WASI doesn't support threading (uses fallback sequential implementation)
|
|
111
|
+
- **No process spawning**: Cannot use external commands like git for .gitignore
|
|
112
|
+
- **Larger binary size**: ~19MB vs ~5MB native (includes Rust runtime for WASM)
|
|
113
|
+
|
|
114
|
+
## License
|
|
115
|
+
|
|
116
|
+
- This wrapper: MIT
|
|
117
|
+
- Ripgrep: MIT/Unlicense (dual-licensed)
|
|
118
|
+
|
|
119
|
+
## Links
|
|
120
|
+
|
|
121
|
+
- [ripgrep repository](https://github.com/BurntSushi/ripgrep)
|
|
122
|
+
- [WasmTime](https://wasmtime.dev/)
|
|
123
|
+
- [WASI specification](https://wasi.dev/)
|
data/Rakefile
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'bundler/gem_tasks'
|
|
4
|
+
require 'rake/testtask'
|
|
5
|
+
|
|
6
|
+
task default: :test
|
|
7
|
+
|
|
8
|
+
Rake::TestTask.new(:test) do |t|
|
|
9
|
+
t.libs << 'test'
|
|
10
|
+
t.pattern = 'test/**/*_test.rb'
|
|
11
|
+
t.verbose = true
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
desc 'Build the gem'
|
|
15
|
+
task :build do
|
|
16
|
+
system 'gem build ripgrep_wasm.gemspec'
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
desc 'Install the gem locally'
|
|
20
|
+
task :install => :build do
|
|
21
|
+
system 'gem install ./ripgrep_wasm-*.gem'
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
desc 'Publish the gem to RubyGems'
|
|
25
|
+
task :release => :build do
|
|
26
|
+
system 'gem push ripgrep_wasm-*.gem'
|
|
27
|
+
end
|
data/SECURITY.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
# Security Considerations
|
|
2
|
+
|
|
3
|
+
## WASI Sandbox
|
|
4
|
+
|
|
5
|
+
The WASM binary runs in a sandboxed environment. By default, it has **no access** to the file system. Access must be explicitly granted:
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
# Grant access to current directory only
|
|
9
|
+
wasmtime --dir=. rg.wasm "pattern" .
|
|
10
|
+
|
|
11
|
+
# Grant access to specific directory
|
|
12
|
+
wasmtime --dir=/path/to/search rg.wasm "pattern" /path/to/search
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
### What the Binary CAN Do
|
|
16
|
+
|
|
17
|
+
With `--dir` access granted:
|
|
18
|
+
- Read files in the specified directory tree
|
|
19
|
+
- Write to stdout/stderr
|
|
20
|
+
- Read environment variables
|
|
21
|
+
- Process command-line arguments
|
|
22
|
+
|
|
23
|
+
### What the Binary CANNOT Do
|
|
24
|
+
|
|
25
|
+
Even with file system access:
|
|
26
|
+
- Execute other programs
|
|
27
|
+
- Access network
|
|
28
|
+
- Access files outside granted directories
|
|
29
|
+
- Modify system settings
|
|
30
|
+
- Access hardware directly
|
|
31
|
+
|
|
32
|
+
## Supply Chain Security
|
|
33
|
+
|
|
34
|
+
### Binary Provenance
|
|
35
|
+
|
|
36
|
+
The `rg.wasm` binary is:
|
|
37
|
+
1. Built from official ripgrep source code
|
|
38
|
+
2. Compiled with standard Rust toolchain
|
|
39
|
+
3. No modifications to ripgrep source
|
|
40
|
+
|
|
41
|
+
### Verification
|
|
42
|
+
|
|
43
|
+
To verify the binary matches the source:
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
# Clone and build yourself
|
|
47
|
+
git clone --depth 1 https://github.com/BurntSushi/ripgrep.git
|
|
48
|
+
cd ripgrep
|
|
49
|
+
rustup target add wasm32-wasip1
|
|
50
|
+
cargo build --release --target wasm32-wasip1
|
|
51
|
+
|
|
52
|
+
# Compare checksums
|
|
53
|
+
sha256sum target/wasm32-wasip1/release/rg.wasm
|
|
54
|
+
sha256sum /path/to/downloaded/rg.wasm
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Note: Checksums may differ slightly due to build environment, but functionality is identical.
|
|
58
|
+
|
|
59
|
+
## Package Security
|
|
60
|
+
|
|
61
|
+
### NPM Package
|
|
62
|
+
|
|
63
|
+
- No runtime dependencies
|
|
64
|
+
- Downloads binary over HTTPS from GitHub
|
|
65
|
+
- Verifies GitHub API responses
|
|
66
|
+
- Falls back gracefully on network errors
|
|
67
|
+
|
|
68
|
+
### Ruby Gem
|
|
69
|
+
|
|
70
|
+
- No external gem dependencies
|
|
71
|
+
- Downloads binary over HTTPS from GitHub
|
|
72
|
+
- Uses Ruby standard library only
|
|
73
|
+
|
|
74
|
+
## Reporting Vulnerabilities
|
|
75
|
+
|
|
76
|
+
For security issues in:
|
|
77
|
+
- **This wrapper**: Open a GitHub issue or contact the maintainer
|
|
78
|
+
- **Ripgrep itself**: Report to [ripgrep security](https://github.com/BurntSushi/ripgrep/security)
|
|
79
|
+
- **WasmTime**: Report to [WasmTime security](https://github.com/bytecodealliance/wasmtime/security)
|
|
80
|
+
|
|
81
|
+
## Best Practices
|
|
82
|
+
|
|
83
|
+
1. **Minimize directory access**: Only grant `--dir` to directories that need searching
|
|
84
|
+
2. **Verify binary integrity**: Check checksums for downloaded binaries
|
|
85
|
+
3. **Keep updated**: Update packages regularly for security fixes
|
|
86
|
+
4. **Isolate execution**: Run in containers or VMs for untrusted searches
|
data/index.js
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @nathanhimpens/ripgrep-wasm
|
|
3
|
+
*
|
|
4
|
+
* Exports the path to the rg.wasm binary.
|
|
5
|
+
* The binary is automatically downloaded from GitHub Releases during npm install.
|
|
6
|
+
*/
|
|
7
|
+
|
|
8
|
+
const path = require('path');
|
|
9
|
+
const fs = require('fs');
|
|
10
|
+
|
|
11
|
+
const wasmPath = path.join(__dirname, 'rg.wasm');
|
|
12
|
+
|
|
13
|
+
// Check if the file exists, if not, provide helpful error message
|
|
14
|
+
if (!fs.existsSync(wasmPath)) {
|
|
15
|
+
console.warn(
|
|
16
|
+
'Warning: rg.wasm not found. It should be downloaded automatically during installation.\n' +
|
|
17
|
+
'If you see this message, try running: npm run postinstall'
|
|
18
|
+
);
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
module.exports = wasmPath;
|
|
22
|
+
|
|
23
|
+
// Also export as default for ES modules compatibility
|
|
24
|
+
module.exports.default = wasmPath;
|
data/lib/download.js
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* Downloads rg.wasm from GitHub Releases
|
|
5
|
+
* This script runs automatically after npm install via the postinstall hook
|
|
6
|
+
*/
|
|
7
|
+
|
|
8
|
+
const https = require('https');
|
|
9
|
+
const fs = require('fs');
|
|
10
|
+
const path = require('path');
|
|
11
|
+
|
|
12
|
+
const REPO_OWNER = 'NathanHimpens';
|
|
13
|
+
const REPO_NAME = 'ripgrep-wasm';
|
|
14
|
+
const ASSET_NAME = 'rg.wasm';
|
|
15
|
+
const WASM_PATH = path.join(__dirname, '..', 'rg.wasm');
|
|
16
|
+
|
|
17
|
+
// Get version from package.json
|
|
18
|
+
const packageJson = require('../package.json');
|
|
19
|
+
const version = packageJson.version;
|
|
20
|
+
|
|
21
|
+
/**
|
|
22
|
+
* Get the latest release tag from GitHub API
|
|
23
|
+
*/
|
|
24
|
+
function getLatestReleaseTag() {
|
|
25
|
+
return new Promise((resolve, reject) => {
|
|
26
|
+
const options = {
|
|
27
|
+
hostname: 'api.github.com',
|
|
28
|
+
path: `/repos/${REPO_OWNER}/${REPO_NAME}/releases/latest`,
|
|
29
|
+
headers: {
|
|
30
|
+
'User-Agent': 'ripgrep-wasm-downloader',
|
|
31
|
+
'Accept': 'application/vnd.github.v3+json'
|
|
32
|
+
}
|
|
33
|
+
};
|
|
34
|
+
|
|
35
|
+
https.get(options, (res) => {
|
|
36
|
+
let data = '';
|
|
37
|
+
|
|
38
|
+
res.on('data', (chunk) => {
|
|
39
|
+
data += chunk;
|
|
40
|
+
});
|
|
41
|
+
|
|
42
|
+
res.on('end', () => {
|
|
43
|
+
if (res.statusCode === 200) {
|
|
44
|
+
try {
|
|
45
|
+
const release = JSON.parse(data);
|
|
46
|
+
resolve(release.tag_name);
|
|
47
|
+
} catch (e) {
|
|
48
|
+
reject(new Error(`Failed to parse release data: ${e.message}`));
|
|
49
|
+
}
|
|
50
|
+
} else if (res.statusCode === 404) {
|
|
51
|
+
// No releases yet, try to use version from package.json
|
|
52
|
+
console.log(`No GitHub release found. Using version ${version} from package.json.`);
|
|
53
|
+
resolve(`v${version}`);
|
|
54
|
+
} else {
|
|
55
|
+
reject(new Error(`GitHub API returned status ${res.statusCode}: ${data}`));
|
|
56
|
+
}
|
|
57
|
+
});
|
|
58
|
+
}).on('error', (e) => {
|
|
59
|
+
reject(new Error(`Failed to fetch release info: ${e.message}`));
|
|
60
|
+
});
|
|
61
|
+
});
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
/**
|
|
65
|
+
* Download the asset from GitHub Releases
|
|
66
|
+
*/
|
|
67
|
+
function downloadAsset(tag) {
|
|
68
|
+
return new Promise((resolve, reject) => {
|
|
69
|
+
// Try to get the release by tag first to find the asset URL
|
|
70
|
+
const options = {
|
|
71
|
+
hostname: 'api.github.com',
|
|
72
|
+
path: `/repos/${REPO_OWNER}/${REPO_NAME}/releases/tags/${tag}`,
|
|
73
|
+
headers: {
|
|
74
|
+
'User-Agent': 'ripgrep-wasm-downloader',
|
|
75
|
+
'Accept': 'application/vnd.github.v3+json'
|
|
76
|
+
}
|
|
77
|
+
};
|
|
78
|
+
|
|
79
|
+
https.get(options, (res) => {
|
|
80
|
+
let data = '';
|
|
81
|
+
|
|
82
|
+
res.on('data', (chunk) => {
|
|
83
|
+
data += chunk;
|
|
84
|
+
});
|
|
85
|
+
|
|
86
|
+
res.on('end', () => {
|
|
87
|
+
if (res.statusCode === 404) {
|
|
88
|
+
// Release doesn't exist yet, skip download
|
|
89
|
+
console.log(`Release ${tag} not found on GitHub. Skipping download.`);
|
|
90
|
+
console.log('You can manually download rg.wasm from the repository or build it yourself.');
|
|
91
|
+
resolve(false);
|
|
92
|
+
return;
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
if (res.statusCode !== 200) {
|
|
96
|
+
reject(new Error(`GitHub API returned status ${res.statusCode}: ${data}`));
|
|
97
|
+
return;
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
try {
|
|
101
|
+
const release = JSON.parse(data);
|
|
102
|
+
const asset = release.assets.find(a => a.name === ASSET_NAME);
|
|
103
|
+
|
|
104
|
+
if (!asset) {
|
|
105
|
+
console.log(`Asset ${ASSET_NAME} not found in release ${tag}.`);
|
|
106
|
+
console.log('You can manually download rg.wasm from the repository or build it yourself.');
|
|
107
|
+
resolve(false);
|
|
108
|
+
return;
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
// Download the asset
|
|
112
|
+
console.log(`Downloading ${ASSET_NAME} from release ${tag}...`);
|
|
113
|
+
console.log(`Size: ${(asset.size / 1024 / 1024).toFixed(2)} MB`);
|
|
114
|
+
|
|
115
|
+
const file = fs.createWriteStream(WASM_PATH);
|
|
116
|
+
const downloadUrl = new URL(asset.browser_download_url);
|
|
117
|
+
|
|
118
|
+
const downloadOptions = {
|
|
119
|
+
hostname: downloadUrl.hostname,
|
|
120
|
+
path: downloadUrl.pathname + downloadUrl.search,
|
|
121
|
+
headers: {
|
|
122
|
+
'User-Agent': 'ripgrep-wasm-downloader',
|
|
123
|
+
'Accept': 'application/octet-stream'
|
|
124
|
+
}
|
|
125
|
+
};
|
|
126
|
+
|
|
127
|
+
https.get(downloadOptions, (downloadRes) => {
|
|
128
|
+
if (downloadRes.statusCode !== 200) {
|
|
129
|
+
file.close();
|
|
130
|
+
fs.unlinkSync(WASM_PATH);
|
|
131
|
+
reject(new Error(`Failed to download asset: ${downloadRes.statusCode}`));
|
|
132
|
+
return;
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
downloadRes.pipe(file);
|
|
136
|
+
|
|
137
|
+
file.on('finish', () => {
|
|
138
|
+
file.close();
|
|
139
|
+
// Make executable
|
|
140
|
+
fs.chmodSync(WASM_PATH, 0o755);
|
|
141
|
+
console.log(`✓ Successfully downloaded ${ASSET_NAME}`);
|
|
142
|
+
resolve(true);
|
|
143
|
+
});
|
|
144
|
+
}).on('error', (e) => {
|
|
145
|
+
file.close();
|
|
146
|
+
fs.unlinkSync(WASM_PATH);
|
|
147
|
+
reject(new Error(`Download failed: ${e.message}`));
|
|
148
|
+
});
|
|
149
|
+
} catch (e) {
|
|
150
|
+
reject(new Error(`Failed to parse release data: ${e.message}`));
|
|
151
|
+
}
|
|
152
|
+
});
|
|
153
|
+
}).on('error', (e) => {
|
|
154
|
+
reject(new Error(`Failed to fetch release: ${e.message}`));
|
|
155
|
+
});
|
|
156
|
+
});
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
/**
|
|
160
|
+
* Main function
|
|
161
|
+
*/
|
|
162
|
+
async function main() {
|
|
163
|
+
// Check if file already exists
|
|
164
|
+
if (fs.existsSync(WASM_PATH)) {
|
|
165
|
+
console.log('rg.wasm already exists. Skipping download.');
|
|
166
|
+
return;
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
try {
|
|
170
|
+
const tag = await getLatestReleaseTag();
|
|
171
|
+
const downloaded = await downloadAsset(tag);
|
|
172
|
+
|
|
173
|
+
if (!downloaded) {
|
|
174
|
+
console.warn('\n⚠️ rg.wasm was not downloaded automatically.');
|
|
175
|
+
console.warn('This is normal if no GitHub release exists yet.\n');
|
|
176
|
+
console.warn('To use this package, you need to:');
|
|
177
|
+
console.warn('1. Build rg.wasm yourself (see README.md)');
|
|
178
|
+
console.warn('2. Create a GitHub release with rg.wasm attached');
|
|
179
|
+
console.warn('3. Or manually copy rg.wasm to this directory\n');
|
|
180
|
+
// Don't exit with error - allow the package to be installed
|
|
181
|
+
// The user can manually add the file later
|
|
182
|
+
return;
|
|
183
|
+
}
|
|
184
|
+
} catch (error) {
|
|
185
|
+
console.error('Error downloading rg.wasm:', error.message);
|
|
186
|
+
console.error('\nYou can:');
|
|
187
|
+
console.error('1. Build it yourself following the instructions in README.md');
|
|
188
|
+
console.error('2. Manually download it from a GitHub release');
|
|
189
|
+
console.error('3. Copy it from the build directory after compilation');
|
|
190
|
+
// Don't exit with error - allow installation to continue
|
|
191
|
+
// The user can manually add the file later
|
|
192
|
+
console.warn('\n⚠️ Installation will continue, but rg.wasm must be added manually.');
|
|
193
|
+
}
|
|
194
|
+
}
|
|
195
|
+
|
|
196
|
+
// Run if called directly
|
|
197
|
+
if (require.main === module) {
|
|
198
|
+
main();
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
module.exports = { main, downloadAsset, getLatestReleaseTag };
|
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'net/http'
|
|
4
|
+
require 'json'
|
|
5
|
+
require 'fileutils'
|
|
6
|
+
require 'uri'
|
|
7
|
+
|
|
8
|
+
module RipgrepWasm
|
|
9
|
+
class Downloader
|
|
10
|
+
REPO_OWNER = 'NathanHimpens'
|
|
11
|
+
REPO_NAME = 'ripgrep-wasm'
|
|
12
|
+
ASSET_NAME = 'rg.wasm'
|
|
13
|
+
|
|
14
|
+
def self.wasm_path
|
|
15
|
+
File.join(File.dirname(__FILE__), ASSET_NAME)
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
# Download rg.wasm from GitHub Releases if it doesn't exist
|
|
19
|
+
def self.download_if_needed
|
|
20
|
+
return if File.exist?(wasm_path)
|
|
21
|
+
|
|
22
|
+
puts "rg.wasm not found. Attempting to download from GitHub Releases..."
|
|
23
|
+
download
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
# Download rg.wasm from the latest GitHub release
|
|
27
|
+
def self.download
|
|
28
|
+
begin
|
|
29
|
+
tag = get_latest_release_tag
|
|
30
|
+
download_asset(tag)
|
|
31
|
+
rescue StandardError => e
|
|
32
|
+
warn "Error downloading rg.wasm: #{e.message}"
|
|
33
|
+
warn "\nYou can:"
|
|
34
|
+
warn "1. Build it yourself following the instructions in README.md"
|
|
35
|
+
warn "2. Manually download it from a GitHub release"
|
|
36
|
+
warn "3. Copy it from the build directory after compilation"
|
|
37
|
+
warn "\n⚠️ Installation will continue, but rg.wasm must be added manually."
|
|
38
|
+
false
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
private
|
|
43
|
+
|
|
44
|
+
# Get the latest release tag from GitHub API
|
|
45
|
+
def self.get_latest_release_tag
|
|
46
|
+
uri = URI("https://api.github.com/repos/#{REPO_OWNER}/#{REPO_NAME}/releases/latest")
|
|
47
|
+
|
|
48
|
+
http = Net::HTTP.new(uri.host, uri.port)
|
|
49
|
+
http.use_ssl = true
|
|
50
|
+
http.read_timeout = 30
|
|
51
|
+
|
|
52
|
+
request = Net::HTTP::Get.new(uri)
|
|
53
|
+
request['User-Agent'] = 'ripgrep-wasm-ruby-downloader'
|
|
54
|
+
request['Accept'] = 'application/vnd.github.v3+json'
|
|
55
|
+
|
|
56
|
+
response = http.request(request)
|
|
57
|
+
|
|
58
|
+
case response.code
|
|
59
|
+
when '200'
|
|
60
|
+
release = JSON.parse(response.body)
|
|
61
|
+
release['tag_name']
|
|
62
|
+
when '404'
|
|
63
|
+
# No releases yet, try to use version from gem
|
|
64
|
+
version = RipgrepWasm::VERSION
|
|
65
|
+
puts "No GitHub release found. Using version #{version} from gem."
|
|
66
|
+
"v#{version}"
|
|
67
|
+
else
|
|
68
|
+
raise "GitHub API returned status #{response.code}: #{response.body}"
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
# Download the asset from GitHub Releases
|
|
73
|
+
def self.download_asset(tag)
|
|
74
|
+
# Get release info to find asset URL
|
|
75
|
+
uri = URI("https://api.github.com/repos/#{REPO_OWNER}/#{REPO_NAME}/releases/tags/#{tag}")
|
|
76
|
+
|
|
77
|
+
http = Net::HTTP.new(uri.host, uri.port)
|
|
78
|
+
http.use_ssl = true
|
|
79
|
+
http.read_timeout = 30
|
|
80
|
+
|
|
81
|
+
request = Net::HTTP::Get.new(uri)
|
|
82
|
+
request['User-Agent'] = 'ripgrep-wasm-ruby-downloader'
|
|
83
|
+
request['Accept'] = 'application/vnd.github.v3+json'
|
|
84
|
+
|
|
85
|
+
response = http.request(request)
|
|
86
|
+
|
|
87
|
+
case response.code
|
|
88
|
+
when '404'
|
|
89
|
+
puts "Release #{tag} not found on GitHub. Skipping download."
|
|
90
|
+
puts 'You can manually download rg.wasm from the repository or build it yourself.'
|
|
91
|
+
return false
|
|
92
|
+
when '200'
|
|
93
|
+
# Continue
|
|
94
|
+
else
|
|
95
|
+
raise "GitHub API returned status #{response.code}: #{response.body}"
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
release = JSON.parse(response.body)
|
|
99
|
+
asset = release['assets'].find { |a| a['name'] == ASSET_NAME }
|
|
100
|
+
|
|
101
|
+
unless asset
|
|
102
|
+
puts "Asset #{ASSET_NAME} not found in release #{tag}."
|
|
103
|
+
puts 'You can manually download rg.wasm from the repository or build it yourself.'
|
|
104
|
+
return false
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
# Download the asset
|
|
108
|
+
puts "Downloading #{ASSET_NAME} from release #{tag}..."
|
|
109
|
+
puts "Size: #{(asset['size'] / 1024.0 / 1024.0).round(2)} MB"
|
|
110
|
+
|
|
111
|
+
download_uri = URI(asset['browser_download_url'])
|
|
112
|
+
download_http = Net::HTTP.new(download_uri.host, download_uri.port)
|
|
113
|
+
download_http.use_ssl = true
|
|
114
|
+
download_http.read_timeout = 300 # 5 minutes for large file
|
|
115
|
+
|
|
116
|
+
download_request = Net::HTTP::Get.new(download_uri)
|
|
117
|
+
download_request['User-Agent'] = 'ripgrep-wasm-ruby-downloader'
|
|
118
|
+
download_request['Accept'] = 'application/octet-stream'
|
|
119
|
+
|
|
120
|
+
FileUtils.mkdir_p(File.dirname(wasm_path))
|
|
121
|
+
|
|
122
|
+
File.open(wasm_path, 'wb') do |file|
|
|
123
|
+
download_http.request(download_request) do |response|
|
|
124
|
+
case response.code
|
|
125
|
+
when '200'
|
|
126
|
+
response.read_body do |chunk|
|
|
127
|
+
file.write(chunk)
|
|
128
|
+
end
|
|
129
|
+
else
|
|
130
|
+
FileUtils.rm_f(wasm_path)
|
|
131
|
+
raise "Failed to download asset: #{response.code}"
|
|
132
|
+
end
|
|
133
|
+
end
|
|
134
|
+
end
|
|
135
|
+
|
|
136
|
+
# Make executable
|
|
137
|
+
File.chmod(0o755, wasm_path)
|
|
138
|
+
puts "✓ Successfully downloaded #{ASSET_NAME}"
|
|
139
|
+
true
|
|
140
|
+
end
|
|
141
|
+
end
|
|
142
|
+
end
|
|
Binary file
|
data/lib/ripgrep_wasm.rb
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative 'ripgrep_wasm/version'
|
|
4
|
+
require_relative 'ripgrep_wasm/downloader'
|
|
5
|
+
|
|
6
|
+
module RipgrepWasm
|
|
7
|
+
class Error < StandardError; end
|
|
8
|
+
|
|
9
|
+
# Get the path to the rg.wasm binary
|
|
10
|
+
#
|
|
11
|
+
# @return [String] The absolute path to rg.wasm
|
|
12
|
+
def self.path
|
|
13
|
+
wasm_path = File.join(File.dirname(__FILE__), 'ripgrep_wasm', 'rg.wasm')
|
|
14
|
+
|
|
15
|
+
# If the file doesn't exist, try to download it
|
|
16
|
+
unless File.exist?(wasm_path)
|
|
17
|
+
Downloader.download_if_needed
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
wasm_path
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
# Check if rg.wasm is available
|
|
24
|
+
#
|
|
25
|
+
# @return [Boolean] true if rg.wasm exists
|
|
26
|
+
def self.available?
|
|
27
|
+
File.exist?(path)
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
# Get the absolute path to rg.wasm
|
|
31
|
+
#
|
|
32
|
+
# @return [String] The absolute path to rg.wasm
|
|
33
|
+
def self.absolute_path
|
|
34
|
+
File.expand_path(path)
|
|
35
|
+
end
|
|
36
|
+
end
|
data/package.json
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@nathanhimpens/ripgrep-wasm",
|
|
3
|
+
"version": "1.0.0",
|
|
4
|
+
"description": "Ripgrep 15.1.0 compiled to WebAssembly for fast text search in WASI environments",
|
|
5
|
+
"main": "index.js",
|
|
6
|
+
"type": "commonjs",
|
|
7
|
+
"scripts": {
|
|
8
|
+
"postinstall": "node lib/download.js"
|
|
9
|
+
},
|
|
10
|
+
"keywords": [
|
|
11
|
+
"ripgrep",
|
|
12
|
+
"rg",
|
|
13
|
+
"wasm",
|
|
14
|
+
"wasi",
|
|
15
|
+
"search",
|
|
16
|
+
"grep",
|
|
17
|
+
"regex",
|
|
18
|
+
"text"
|
|
19
|
+
],
|
|
20
|
+
"author": "",
|
|
21
|
+
"license": "MIT",
|
|
22
|
+
"repository": {
|
|
23
|
+
"type": "git",
|
|
24
|
+
"url": "git+https://github.com/NathanHimpens/ripgrep-wasm.git"
|
|
25
|
+
},
|
|
26
|
+
"bugs": {
|
|
27
|
+
"url": "https://github.com/NathanHimpens/ripgrep-wasm/issues"
|
|
28
|
+
},
|
|
29
|
+
"homepage": "https://github.com/NathanHimpens/ripgrep-wasm#readme",
|
|
30
|
+
"engines": {
|
|
31
|
+
"node": ">=14.0.0"
|
|
32
|
+
},
|
|
33
|
+
"files": [
|
|
34
|
+
"index.js",
|
|
35
|
+
"lib/",
|
|
36
|
+
"README.md"
|
|
37
|
+
]
|
|
38
|
+
}
|
data/rg.wasm
ADDED
|
Binary file
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative 'lib/ripgrep_wasm/version'
|
|
4
|
+
|
|
5
|
+
Gem::Specification.new do |spec|
|
|
6
|
+
spec.name = 'ripgrep_wasm'
|
|
7
|
+
spec.version = RipgrepWasm::VERSION
|
|
8
|
+
spec.authors = ['Nathan Himpens']
|
|
9
|
+
spec.email = ['']
|
|
10
|
+
|
|
11
|
+
spec.summary = 'Ripgrep 15.1.0 compiled to WebAssembly for fast text search in WASI environments'
|
|
12
|
+
spec.description = 'Ripgrep 15.1.0 compiled to WebAssembly for fast text search in WASI environments'
|
|
13
|
+
spec.homepage = 'https://github.com/NathanHimpens/ripgrep-wasm'
|
|
14
|
+
spec.license = 'MIT'
|
|
15
|
+
|
|
16
|
+
spec.metadata['homepage_uri'] = spec.homepage
|
|
17
|
+
spec.metadata['source_code_uri'] = 'https://github.com/NathanHimpens/ripgrep-wasm'
|
|
18
|
+
spec.metadata['changelog_uri'] = 'https://github.com/NathanHimpens/ripgrep-wasm/blob/main/README.md'
|
|
19
|
+
|
|
20
|
+
# Specify which files should be added to the gem when it is released.
|
|
21
|
+
spec.files = Dir.chdir(__dir__) do
|
|
22
|
+
`git ls-files -z`.split("\x0").reject do |f|
|
|
23
|
+
(File.expand_path(f) == __FILE__) ||
|
|
24
|
+
f.start_with?(*%w[bin/ test/ spec/ features/ .git .github .cursor .ralph ripgrep/])
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
spec.bindir = 'exe'
|
|
29
|
+
spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
|
|
30
|
+
spec.require_paths = ['lib']
|
|
31
|
+
|
|
32
|
+
# Ruby standard library dependencies (no external gems needed)
|
|
33
|
+
spec.required_ruby_version = '>= 2.7.0'
|
|
34
|
+
|
|
35
|
+
# Post-install message
|
|
36
|
+
spec.post_install_message = <<~MSG
|
|
37
|
+
ripgrep_wasm installed successfully!
|
|
38
|
+
|
|
39
|
+
Note: rg.wasm will be automatically downloaded from GitHub Releases
|
|
40
|
+
the first time you call RipgrepWasm.path in your code.
|
|
41
|
+
|
|
42
|
+
If no GitHub release exists yet, you'll need to:
|
|
43
|
+
1. Build rg.wasm yourself (see README.md)
|
|
44
|
+
2. Create a GitHub release with rg.wasm attached
|
|
45
|
+
3. Or manually copy rg.wasm to the gem directory
|
|
46
|
+
MSG
|
|
47
|
+
end
|
data/test.txt
ADDED
metadata
ADDED
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: ripgrep_wasm
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Nathan Himpens
|
|
8
|
+
bindir: exe
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies: []
|
|
12
|
+
description: Ripgrep 15.1.0 compiled to WebAssembly for fast text search in WASI environments
|
|
13
|
+
email:
|
|
14
|
+
- ''
|
|
15
|
+
executables: []
|
|
16
|
+
extensions: []
|
|
17
|
+
extra_rdoc_files: []
|
|
18
|
+
files:
|
|
19
|
+
- ".npmignore"
|
|
20
|
+
- IMPLEMENTATION.md
|
|
21
|
+
- RALPH_TASK.md
|
|
22
|
+
- README.md
|
|
23
|
+
- Rakefile
|
|
24
|
+
- SECURITY.md
|
|
25
|
+
- index.js
|
|
26
|
+
- lib/download.js
|
|
27
|
+
- lib/ripgrep_wasm.rb
|
|
28
|
+
- lib/ripgrep_wasm/downloader.rb
|
|
29
|
+
- lib/ripgrep_wasm/rg.wasm
|
|
30
|
+
- lib/ripgrep_wasm/version.rb
|
|
31
|
+
- package.json
|
|
32
|
+
- rg.wasm
|
|
33
|
+
- ripgrep_wasm.gemspec
|
|
34
|
+
- test.txt
|
|
35
|
+
homepage: https://github.com/NathanHimpens/ripgrep-wasm
|
|
36
|
+
licenses:
|
|
37
|
+
- MIT
|
|
38
|
+
metadata:
|
|
39
|
+
homepage_uri: https://github.com/NathanHimpens/ripgrep-wasm
|
|
40
|
+
source_code_uri: https://github.com/NathanHimpens/ripgrep-wasm
|
|
41
|
+
changelog_uri: https://github.com/NathanHimpens/ripgrep-wasm/blob/main/README.md
|
|
42
|
+
post_install_message: |
|
|
43
|
+
ripgrep_wasm installed successfully!
|
|
44
|
+
|
|
45
|
+
Note: rg.wasm will be automatically downloaded from GitHub Releases
|
|
46
|
+
the first time you call RipgrepWasm.path in your code.
|
|
47
|
+
|
|
48
|
+
If no GitHub release exists yet, you'll need to:
|
|
49
|
+
1. Build rg.wasm yourself (see README.md)
|
|
50
|
+
2. Create a GitHub release with rg.wasm attached
|
|
51
|
+
3. Or manually copy rg.wasm to the gem directory
|
|
52
|
+
rdoc_options: []
|
|
53
|
+
require_paths:
|
|
54
|
+
- lib
|
|
55
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
56
|
+
requirements:
|
|
57
|
+
- - ">="
|
|
58
|
+
- !ruby/object:Gem::Version
|
|
59
|
+
version: 2.7.0
|
|
60
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
61
|
+
requirements:
|
|
62
|
+
- - ">="
|
|
63
|
+
- !ruby/object:Gem::Version
|
|
64
|
+
version: '0'
|
|
65
|
+
requirements: []
|
|
66
|
+
rubygems_version: 3.6.7
|
|
67
|
+
specification_version: 4
|
|
68
|
+
summary: Ripgrep 15.1.0 compiled to WebAssembly for fast text search in WASI environments
|
|
69
|
+
test_files: []
|