@effectstream/npm-midnight-indexer 0.100.15 → 0.100.17

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +48 -207
  2. package/package.json +3 -3
package/README.md CHANGED
@@ -1,26 +1,13 @@
1
1
  # @effectstream/npm-midnight-indexer
2
2
 
3
- A Node.js package that downloads and runs the
4
- [Midnight](https://midnight.network) Indexer. The indexer requires a running
5
- Midnight Node (see `@effectstream/npm-midnight-node`) to function properly.
3
+ NPM wrapper that runs the [Midnight](https://midnight.network) Indexer either as a Docker container or as a native binary. Boots alongside `@effectstream/npm-midnight-node` and `@effectstream/npm-midnight-proof-server` to give Effectstream a local indexer to consume.
6
4
 
7
- ## Overview
5
+ - Docker or binary mode, with platform-aware defaults (macOS arm64 and Linux can use either; Windows is Docker-only).
6
+ - One env var to set: `APP__INFRA__SECRET`. Everything else has a default that works against the local Midnight stack.
7
+ - Used by the orchestrator's Midnight step; sits in front of `MidnightFetcher` on the sync side.
8
+ - Maps port 8088 (Docker) or runs on localhost (binary) so the rest of the local stack reaches it the same way.
8
9
 
9
- This package provides flexible execution options for the Midnight Indexer:
10
-
11
- - **Docker mode**: Runs the indexer in a Docker container (recommended)
12
- - **Binary mode**: Downloads and runs the native binary locally (Linux, Windows,
13
- macOS arm64)
14
-
15
- ### Platform Support
16
-
17
- | Platform | Docker | Binary |
18
- | -------- | ------ | ------ |
19
- | macOS | ✅ Yes | ✅ Yes |
20
- | Linux | ✅ Yes | ✅ Yes |
21
- | Windows | ❌ No | ✅ Yes |
22
-
23
- ## Installation
10
+ ## Install
24
11
 
25
12
  ```bash
26
13
  bun add @effectstream/npm-midnight-indexer
@@ -28,220 +15,74 @@ bun add @effectstream/npm-midnight-indexer
28
15
  npm install @effectstream/npm-midnight-indexer
29
16
  ```
30
17
 
31
- ## Prerequisites
32
-
33
- ### For Docker Mode
34
-
35
- - Docker installed and running
36
- - `APP__INFRA__SECRET` environment variable (required)
37
-
38
- ### For Binary Mode
39
-
40
- - Supported platform (Linux/macOS ARM64)
41
- - Local Midnight Node and Proof Server running on standard ports. But they can
42
- be set using env variables
43
- - `APP__INFRA__SECRET` environment variable (required)
44
-
45
- ## Usage
46
-
47
- ### Command Line Options
48
-
49
- ```bash
50
- node index.js [options] [args...]
51
- ```
52
-
53
- **Options:**
18
+ Requires a running Midnight node (`@effectstream/npm-midnight-node`) and proof server (`@effectstream/npm-midnight-proof-server`) on their standard ports, or set the override env vars below.
54
19
 
55
- - `--docker` - Force Docker execution
56
- - `--binary` - Force binary execution (available on Linux, Windows, macOS arm64)
57
- - `--help` - Show help information
20
+ ## Standalone usage
58
21
 
59
- **Examples:**
22
+ Pick a mode and pass `APP__INFRA__SECRET`. Without a flag, the wrapper prompts interactively.
60
23
 
61
24
  ```bash
62
- # Force Docker usage
63
- APP__INFRA__SECRET=mysecret node index.js --docker
64
-
65
- # Force binary usage (Linux/Windows/macOS arm64)
66
- APP__INFRA__SECRET=mysecret node index.js --binary
67
-
68
- # Interactive mode - prompts for Docker vs binary choice
69
- APP__INFRA__SECRET=mysecret node index.js
70
-
71
- # Show help
72
- node index.js --help
73
- ```
74
-
75
- ## Environment Variables
76
-
77
- The indexer supports several environment variables with different defaults based
78
- on execution mode:
79
-
80
- | Variable | Required | Docker Default | Binary Default | Description |
81
- | ------------------------------- | -------- | -------------------------- | ----------------------- | ------------------------------ |
82
- | `APP__INFRA__SECRET` | Yes | - | - | Secret key for the application |
83
- | `LEDGER_NETWORK_ID` | No | `Undeployed` | `Undeployed` | Ledger network identifier |
84
- | `SUBSTRATE_NODE_WS_URL` | No | `ws://node:9944` | `ws://localhost:9944` | Substrate node WebSocket URL |
85
- | `FEATURES_WALLET_ENABLED` | No | `true` | `true` | Enable wallet features |
86
- | `APP__INFRA__PROOF_SERVER__URL` | No | `http://proof-server:6300` | `http://localhost:6300` | Proof server URL |
87
- | `APP__INFRA__NODE__URL` | No | `ws://node:9944` | `ws://localhost:9944` | Node WebSocket URL |
88
-
89
- ### Environment Variable Examples
90
-
91
- ```bash
92
- # Minimal Docker setup
93
- APP__INFRA__SECRET=A5F07FCAC914A181EC0998CCCA68312DA5F07FCAC914A181EC0998CCCA68312D \
94
- node index.js --docker
95
-
96
- # Custom configuration
97
- APP__INFRA__SECRET=mysecret \
98
- LEDGER_NETWORK_ID=CustomNetwork \
99
- SUBSTRATE_NODE_WS_URL=ws://localhost:9944 \
100
- APP__INFRA__PROOF_SERVER__URL=http://localhost:6300 \
101
- node index.js --docker
102
-
103
- # Binary mode (uses localhost defaults automatically)
104
- APP__INFRA__SECRET=mysecret node index.js --binary
105
- ```
106
-
107
- ## Docker Mode Details
25
+ # Docker (recommended where available)
26
+ APP__INFRA__SECRET=<secret> bunx npm-midnight-indexer --docker
108
27
 
109
- When using Docker mode, the indexer:
28
+ # Native binary (Linux, macOS arm64)
29
+ APP__INFRA__SECRET=<secret> bunx npm-midnight-indexer --binary
110
30
 
111
- 1. **Pulls the latest image**: `midnightntwrk/indexer-standalone`
112
- 2. **Maps port 8088**: Container port 8088 → Host port 8088
113
- 3. **Uses container networking**: Defaults work with Docker Compose setups
114
- 4. **Auto-cleanup**: Removes existing containers before starting new ones
31
+ # Interactive: prompts for Docker vs binary
32
+ APP__INFRA__SECRET=<secret> bunx npm-midnight-indexer
115
33
 
116
- ### Docker Command Generated
117
-
118
- The package generates Docker commands similar to:
119
-
120
- ```bash
121
- docker run --rm --name midnight-local-indexer \
122
- -p 8088:8088 \
123
- -e LEDGER_NETWORK_ID=Undeployed \
124
- -e SUBSTRATE_NODE_WS_URL=ws://node:9944 \
125
- -e APP__INFRA__SECRET=A5F07FCAC914A181EC0998CCCA68312DA5F07FCAC914A181EC0998CCCA68312D \
126
- -e FEATURES_WALLET_ENABLED=true \
127
- -e APP__INFRA__PROOF_SERVER__URL=http://proof-server:6300 \
128
- -e APP__INFRA__NODE__URL=ws://node:9944 \
129
- midnightntwrk/indexer-standalone
34
+ # Help
35
+ bunx npm-midnight-indexer --help
130
36
  ```
131
37
 
132
- ## Binary Mode Details
133
-
134
- When using binary mode, the indexer:
38
+ The Docker path pulls `midnightntwrk/indexer-standalone` and maps container port 8088 to host 8088. The binary path downloads a platform-specific binary on first run and points at localhost services.
135
39
 
136
- 1. **Downloads platform-specific binary** if not already present
137
- 2. **Uses localhost URLs** for all service connections
138
- 3. **Runs natively** on the host system
40
+ ### Environment variables
139
41
 
140
- ## Path Resolution
42
+ | Variable | Required | Docker default | Binary default | Purpose |
43
+ | --- | --- | --- | --- | --- |
44
+ | `APP__INFRA__SECRET` | yes | - | - | Indexer secret. |
45
+ | `LEDGER_NETWORK_ID` | no | `Undeployed` | `Undeployed` | Ledger network selector. |
46
+ | `SUBSTRATE_NODE_WS_URL` | no | `ws://node:9944` | `ws://localhost:9944` | Substrate node WS. |
47
+ | `FEATURES_WALLET_ENABLED` | no | `true` | `true` | Wallet features. |
48
+ | `APP__INFRA__PROOF_SERVER__URL` | no | `http://proof-server:6300` | `http://localhost:6300` | Proof server. |
49
+ | `APP__INFRA__NODE__URL` | no | `ws://node:9944` | `ws://localhost:9944` | Node URL. |
141
50
 
142
- The indexer relies on two user-supplied file paths and **both are interpreted
143
- relative to the process’s current working directory (CWD)** when they are not
144
- absolute:
51
+ ### Path resolution
145
52
 
146
- | Configuration location | Purpose | If relative, resolved against |
147
- | ---------------------------------- | -------------------------------------------- | ------------------------------------------------------------------------- |
148
- | `CONFIG_FILE` environment variable | Location of the YAML config file | The directory you launch `node index.js …` from (or the Docker `WORKDIR`) |
149
- | `infra.storage.cnn_url` in YAML | SQLite database used by the standalone build | The CWD at runtime of the indexer process |
53
+ `CONFIG_FILE` and `infra.storage.cnn_url` are both resolved relative to the process's current working directory when they are not absolute. Prefer absolute paths if your launch script's CWD is non-obvious. In binary mode this package sets the CWD to the bundled `indexer-standalone` folder, so a default `cnn_url: "./indexer.sqlite"` lands next to the binary. In Docker mode the image's `WORKDIR` is `/opt/indexer-standalone`; bind-mount accordingly.
150
54
 
151
- ### Practical tips
55
+ ### Supported binary platforms
152
56
 
153
- 1. Prefer absolute paths when you need to be explicit.
154
- 2. In **binary mode** this package sets the CWD to the bundled
155
- `indexer-standalone` folder, so a default `cnn_url: "./indexer.sqlite"` ends
156
- up next to the binary.
157
- 3. In **Docker mode** the image’s `WORKDIR` is `/opt/indexer-standalone`.
158
- Bind-mount volumes accordingly if you want the database file somewhere else.
57
+ Linux arm64, Linux amd64, macOS arm64.
159
58
 
160
- ---
59
+ ## Inside Effectstream
161
60
 
162
- ### Supported Binary Platforms
163
-
164
- - Linux ARM64 (`linux-arm64`)
165
- - Linux AMD64 (`linux-amd64`)
166
- - macOS ARM64 (`macos-arm64`)
61
+ The orchestrator's Midnight step starts this indexer behind `@effectstream/npm-midnight-node` + proof server. The runtime's `@effectstream/sync` `MidnightFetcher` queries it. You usually don't invoke this package by hand; you add it to your orchestrator config and the rest happens automatically.
167
62
 
168
63
  ## Troubleshooting
169
64
 
170
- ### Common Issues
171
-
172
- **"Docker is not installed or not available"**
173
-
174
- - Install Docker Desktop or Docker Engine
175
- - Ensure Docker is running
176
- - Check Docker is accessible from command line: `docker --version`
177
-
178
- **"APP__INFRA__SECRET environment variable is required"**
179
-
180
- - Set the secret: `export APP__INFRA__SECRET=your_secret_here`
181
- - Or pass inline: `APP__INFRA__SECRET=your_secret node index.js --docker`
182
- - Required for both Docker and binary execution modes
183
-
184
- **"Binary execution is only supported on macOS ARM64"**
185
-
186
- - Use `--docker` flag or run without flags to use Docker automatically
187
- - Install Docker if not already installed
188
-
189
- **"Failed to start midnight-indexer"**
190
-
191
- - Check if ports 8088, 6300, 9944 are available
192
- - Verify Midnight Node is running and accessible
193
- - Check environment variables are correctly set
65
+ A few common failures and where to look:
194
66
 
195
- ### Logs and Debugging
67
+ - `Docker is not installed or not available` - install Docker Desktop / Engine and confirm `docker --version` from the same shell.
68
+ - `APP__INFRA__SECRET environment variable is required` - required for both modes; export it or pass inline.
69
+ - `Failed to start midnight-indexer` - check that ports 8088, 6300, and 9944 are free and that the Midnight node is reachable.
196
70
 
197
- The indexer outputs detailed logs including:
71
+ ## Examples
198
72
 
199
- - Download progress for binaries
200
- - Docker pull progress
201
- - Process IDs and status
202
- - Error messages with suggestions
73
+ End-to-end Midnight startup is exercised by the templates that target Midnight:
203
74
 
204
- ## Development
75
+ - [`templates/evm-midnight-v2/`](https://github.com/effectstream/effectstream/tree/main/templates/evm-midnight-v2)
76
+ - [`templates/night-bitcoin/`](https://github.com/effectstream/effectstream/tree/main/templates/night-bitcoin)
77
+ - [`templates/zswap-da/`](https://github.com/effectstream/effectstream/tree/main/templates/zswap-da)
205
78
 
206
- ### Package Structure
79
+ ## Links
207
80
 
208
- ```
209
- npm-midnight-indexer/
210
- ├── index.js # Main entry point
211
- ├── binary.js # Binary download logic
212
- ├── docker.js # Docker execution logic
213
- ├── run_midnight_indexer.js # Binary execution logic
214
- ├── package.json # Package configuration
215
- └── README.md # This file
216
- ```
217
-
218
- ### Testing
219
-
220
- ```bash
221
- # Test Docker detection
222
- node -e "const { checkIfDockerExists } = require('./docker.js'); checkIfDockerExists().then(console.log)"
223
-
224
- # Test help
225
- node index.js --help
226
-
227
- # Test platform detection
228
- node -e "console.log('Platform:', require('os').platform())"
229
- ```
81
+ - Docs: https://effectstream.github.io/docs/packages/binaries/midnight-indexer
82
+ - Source: https://github.com/effectstream/effectstream/tree/main/packages/binaries/midnight-indexer
83
+ - Midnight Network: https://midnight.network
84
+ - Indexer image: https://hub.docker.com/r/midnightntwrk/indexer-standalone
230
85
 
231
86
  ## License
232
87
 
233
- ISC
234
-
235
- ## Support
236
-
237
- For issues related to:
238
-
239
- - **This package**: Open an issue in the repository
240
- - **Midnight Protocol**: Visit [midnight.network](https://midnight.network)
241
- - **Docker**: Check [Docker documentation](https://docs.docker.com/)
242
-
243
- ## Related Links
244
-
245
- - [Midnight Network](https://midnight.network)
246
- - [Docker Hub - Midnight Indexer](https://hub.docker.com/r/midnightntwrk/indexer-standalone)
247
- - [Node.js](https://nodejs.org/)
88
+ ISC.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@effectstream/npm-midnight-indexer",
3
- "version": "0.100.15",
3
+ "version": "0.100.17",
4
4
  "description": "Downloads and runs the Midnight Indexer. It needs a running Midnight Node",
5
5
  "main": "index.js",
6
6
  "scripts": {
@@ -16,11 +16,11 @@
16
16
  "homepage": "https://effectstream.github.io/docs/",
17
17
  "repository": {
18
18
  "type": "git",
19
- "url": "https://github.com/PaimaStudios/paima-engine",
19
+ "url": "https://github.com/effectstream/effectstream",
20
20
  "directory": "packages/binaries/midnight-indexer"
21
21
  },
22
22
  "bugs": {
23
- "url": "https://github.com/PaimaStudios/paima-engine/issues"
23
+ "url": "https://github.com/effectstream/effectstream/issues"
24
24
  },
25
25
  "supportedPlatforms": [
26
26
  "linux-arm64",