@effectstream/npm-midnight-indexer 0.100.14 → 0.100.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +52 -209
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -1,245 +1,88 @@
1
- # Midnight Indexer
1
+ # @effectstream/npm-midnight-indexer
2
2
 
3
- A Node.js package that downloads and runs the
4
- [Midnight](https://midnight.network) Indexer. The indexer requires a running
5
- Midnight Node to function properly.
3
+ NPM wrapper that runs the [Midnight](https://midnight.network) Indexer either as a Docker container or as a native binary. Boots alongside `@effectstream/npm-midnight-node` and `@effectstream/npm-midnight-proof-server` to give Effectstream a local indexer to consume.
6
4
 
7
- ## Overview
5
+ - Docker or binary mode, with platform-aware defaults (macOS arm64 and Linux can use either; Windows is Docker-only).
6
+ - One env var to set: `APP__INFRA__SECRET`. Everything else has a default that works against the local Midnight stack.
7
+ - Used by the orchestrator's Midnight step; sits in front of `MidnightFetcher` on the sync side.
8
+ - Maps port 8088 (Docker) or runs on localhost (binary) so the rest of the local stack reaches it the same way.
8
9
 
9
- This package provides flexible execution options for the Midnight Indexer:
10
-
11
- - **Docker mode**: Runs the indexer in a Docker container (recommended)
12
- - **Binary mode**: Downloads and runs the native binary locally (Linux, Windows,
13
- macOS arm64)
14
-
15
- ### Platform Support
16
-
17
- | Platform | Docker | Binary |
18
- | -------- | ------ | ------ |
19
- | macOS | ✅ Yes | ✅ Yes |
20
- | Linux | ✅ Yes | ✅ Yes |
21
- | Windows | ❌ No | ✅ Yes |
22
-
23
- ## Installation
24
-
25
- ```bash
26
- npm install npm-midnight-indexer
27
- ```
28
-
29
- ## Prerequisites
30
-
31
- ### For Docker Mode
32
-
33
- - Docker installed and running
34
- - `APP__INFRA__SECRET` environment variable (required)
35
-
36
- ### For Binary Mode
37
-
38
- - Supported platform (Linux/macOS ARM64)
39
- - Local Midnight Node and Proof Server running on standard ports. But they can
40
- be set using env variables
41
- - `APP__INFRA__SECRET` environment variable (required)
42
-
43
- ## Usage
44
-
45
- ### Command Line Options
10
+ ## Install
46
11
 
47
12
  ```bash
48
- node index.js [options] [args...]
13
+ bun add @effectstream/npm-midnight-indexer
14
+ # or
15
+ npm install @effectstream/npm-midnight-indexer
49
16
  ```
50
17
 
51
- **Options:**
18
+ Requires a running Midnight node (`@effectstream/npm-midnight-node`) and proof server (`@effectstream/npm-midnight-proof-server`) on their standard ports, or set the override env vars below.
52
19
 
53
- - `--docker` - Force Docker execution
54
- - `--binary` - Force binary execution (available on Linux, Windows, macOS arm64)
55
- - `--help` - Show help information
20
+ ## Standalone usage
56
21
 
57
- **Examples:**
22
+ Pick a mode and pass `APP__INFRA__SECRET`. Without a flag, the wrapper prompts interactively.
58
23
 
59
24
  ```bash
60
- # Force Docker usage
61
- APP__INFRA__SECRET=mysecret node index.js --docker
62
-
63
- # Force binary usage (Linux/Windows/macOS arm64)
64
- APP__INFRA__SECRET=mysecret node index.js --binary
65
-
66
- # Interactive mode - prompts for Docker vs binary choice
67
- APP__INFRA__SECRET=mysecret node index.js
68
-
69
- # Show help
70
- node index.js --help
71
- ```
72
-
73
- ## Environment Variables
74
-
75
- The indexer supports several environment variables with different defaults based
76
- on execution mode:
77
-
78
- | Variable | Required | Docker Default | Binary Default | Description |
79
- | ------------------------------- | -------- | -------------------------- | ----------------------- | ------------------------------ |
80
- | `APP__INFRA__SECRET` | Yes | - | - | Secret key for the application |
81
- | `LEDGER_NETWORK_ID` | No | `Undeployed` | `Undeployed` | Ledger network identifier |
82
- | `SUBSTRATE_NODE_WS_URL` | No | `ws://node:9944` | `ws://localhost:9944` | Substrate node WebSocket URL |
83
- | `FEATURES_WALLET_ENABLED` | No | `true` | `true` | Enable wallet features |
84
- | `APP__INFRA__PROOF_SERVER__URL` | No | `http://proof-server:6300` | `http://localhost:6300` | Proof server URL |
85
- | `APP__INFRA__NODE__URL` | No | `ws://node:9944` | `ws://localhost:9944` | Node WebSocket URL |
86
-
87
- ### Environment Variable Examples
88
-
89
- ```bash
90
- # Minimal Docker setup
91
- APP__INFRA__SECRET=A5F07FCAC914A181EC0998CCCA68312DA5F07FCAC914A181EC0998CCCA68312D \
92
- node index.js --docker
93
-
94
- # Custom configuration
95
- APP__INFRA__SECRET=mysecret \
96
- LEDGER_NETWORK_ID=CustomNetwork \
97
- SUBSTRATE_NODE_WS_URL=ws://localhost:9944 \
98
- APP__INFRA__PROOF_SERVER__URL=http://localhost:6300 \
99
- node index.js --docker
100
-
101
- # Binary mode (uses localhost defaults automatically)
102
- APP__INFRA__SECRET=mysecret node index.js --binary
103
- ```
104
-
105
- ## Docker Mode Details
25
+ # Docker (recommended where available)
26
+ APP__INFRA__SECRET=<secret> bunx npm-midnight-indexer --docker
106
27
 
107
- When using Docker mode, the indexer:
28
+ # Native binary (Linux, macOS arm64)
29
+ APP__INFRA__SECRET=<secret> bunx npm-midnight-indexer --binary
108
30
 
109
- 1. **Pulls the latest image**: `midnightntwrk/indexer-standalone`
110
- 2. **Maps port 8088**: Container port 8088 → Host port 8088
111
- 3. **Uses container networking**: Defaults work with Docker Compose setups
112
- 4. **Auto-cleanup**: Removes existing containers before starting new ones
31
+ # Interactive: prompts for Docker vs binary
32
+ APP__INFRA__SECRET=<secret> bunx npm-midnight-indexer
113
33
 
114
- ### Docker Command Generated
115
-
116
- The package generates Docker commands similar to:
117
-
118
- ```bash
119
- docker run --rm --name midnight-local-indexer \
120
- -p 8088:8088 \
121
- -e LEDGER_NETWORK_ID=Undeployed \
122
- -e SUBSTRATE_NODE_WS_URL=ws://node:9944 \
123
- -e APP__INFRA__SECRET=A5F07FCAC914A181EC0998CCCA68312DA5F07FCAC914A181EC0998CCCA68312D \
124
- -e FEATURES_WALLET_ENABLED=true \
125
- -e APP__INFRA__PROOF_SERVER__URL=http://proof-server:6300 \
126
- -e APP__INFRA__NODE__URL=ws://node:9944 \
127
- midnightntwrk/indexer-standalone
34
+ # Help
35
+ bunx npm-midnight-indexer --help
128
36
  ```
129
37
 
130
- ## Binary Mode Details
131
-
132
- When using binary mode, the indexer:
38
+ The Docker path pulls `midnightntwrk/indexer-standalone` and maps container port 8088 to host 8088. The binary path downloads a platform-specific binary on first run and points at localhost services.
133
39
 
134
- 1. **Downloads platform-specific binary** if not already present
135
- 2. **Uses localhost URLs** for all service connections
136
- 3. **Runs natively** on the host system
40
+ ### Environment variables
137
41
 
138
- ## Path Resolution
42
+ | Variable | Required | Docker default | Binary default | Purpose |
43
+ | --- | --- | --- | --- | --- |
44
+ | `APP__INFRA__SECRET` | yes | — | — | Indexer secret. |
45
+ | `LEDGER_NETWORK_ID` | no | `Undeployed` | `Undeployed` | Ledger network selector. |
46
+ | `SUBSTRATE_NODE_WS_URL` | no | `ws://node:9944` | `ws://localhost:9944` | Substrate node WS. |
47
+ | `FEATURES_WALLET_ENABLED` | no | `true` | `true` | Wallet features. |
48
+ | `APP__INFRA__PROOF_SERVER__URL` | no | `http://proof-server:6300` | `http://localhost:6300` | Proof server. |
49
+ | `APP__INFRA__NODE__URL` | no | `ws://node:9944` | `ws://localhost:9944` | Node URL. |
139
50
 
140
- The indexer relies on two user-supplied file paths and **both are interpreted
141
- relative to the process’s current working directory (CWD)** when they are not
142
- absolute:
51
+ ### Path resolution
143
52
 
144
- | Configuration location | Purpose | If relative, resolved against |
145
- | ---------------------------------- | -------------------------------------------- | ------------------------------------------------------------------------- |
146
- | `CONFIG_FILE` environment variable | Location of the YAML config file | The directory you launch `node index.js …` from (or the Docker `WORKDIR`) |
147
- | `infra.storage.cnn_url` in YAML | SQLite database used by the standalone build | The CWD at runtime of the indexer process |
53
+ `CONFIG_FILE` and `infra.storage.cnn_url` are both resolved relative to the process's current working directory when they are not absolute. Prefer absolute paths if your launch script's CWD is non-obvious. In binary mode this package sets the CWD to the bundled `indexer-standalone` folder, so a default `cnn_url: "./indexer.sqlite"` lands next to the binary. In Docker mode the image's `WORKDIR` is `/opt/indexer-standalone`; bind-mount accordingly.
148
54
 
149
- ### Practical tips
55
+ ### Supported binary platforms
150
56
 
151
- 1. Prefer absolute paths when you need to be explicit.
152
- 2. In **binary mode** this package sets the CWD to the bundled
153
- `indexer-standalone` folder, so a default `cnn_url: "./indexer.sqlite"` ends
154
- up next to the binary.
155
- 3. In **Docker mode** the image’s `WORKDIR` is `/opt/indexer-standalone`.
156
- Bind-mount volumes accordingly if you want the database file somewhere else.
57
+ Linux arm64, Linux amd64, macOS arm64.
157
58
 
158
- ---
59
+ ## Inside Effectstream
159
60
 
160
- ### Supported Binary Platforms
161
-
162
- - Linux ARM64 (`linux-arm64`)
163
- - Linux AMD64 (`linux-amd64`)
164
- - macOS ARM64 (`macos-arm64`)
61
+ The orchestrator's Midnight step starts this indexer behind `@effectstream/npm-midnight-node` + proof server. The runtime's `@effectstream/sync` `MidnightFetcher` queries it. You usually don't invoke this package by hand; you add it to your orchestrator config and the rest happens automatically.
165
62
 
166
63
  ## Troubleshooting
167
64
 
168
- ### Common Issues
169
-
170
- **"Docker is not installed or not available"**
171
-
172
- - Install Docker Desktop or Docker Engine
173
- - Ensure Docker is running
174
- - Check Docker is accessible from command line: `docker --version`
175
-
176
- **"APP__INFRA__SECRET environment variable is required"**
177
-
178
- - Set the secret: `export APP__INFRA__SECRET=your_secret_here`
179
- - Or pass inline: `APP__INFRA__SECRET=your_secret node index.js --docker`
180
- - Required for both Docker and binary execution modes
181
-
182
- **"Binary execution is only supported on macOS ARM64"**
183
-
184
- - Use `--docker` flag or run without flags to use Docker automatically
185
- - Install Docker if not already installed
186
-
187
- **"Failed to start midnight-indexer"**
188
-
189
- - Check if ports 8088, 6300, 9944 are available
190
- - Verify Midnight Node is running and accessible
191
- - Check environment variables are correctly set
65
+ A few common failures and where to look:
192
66
 
193
- ### Logs and Debugging
67
+ - `Docker is not installed or not available` — install Docker Desktop / Engine and confirm `docker --version` from the same shell.
68
+ - `APP__INFRA__SECRET environment variable is required` — required for both modes; export it or pass inline.
69
+ - `Failed to start midnight-indexer` — check that ports 8088, 6300, and 9944 are free and that the Midnight node is reachable.
194
70
 
195
- The indexer outputs detailed logs including:
71
+ ## Examples
196
72
 
197
- - Download progress for binaries
198
- - Docker pull progress
199
- - Process IDs and status
200
- - Error messages with suggestions
73
+ End-to-end Midnight startup is exercised by the templates that target Midnight:
201
74
 
202
- ## Development
75
+ - [`templates/evm-midnight-v2/`](https://github.com/effectstream/effectstream/tree/main/templates/evm-midnight-v2)
76
+ - [`templates/night-bitcoin/`](https://github.com/effectstream/effectstream/tree/main/templates/night-bitcoin)
77
+ - [`templates/zswap-da/`](https://github.com/effectstream/effectstream/tree/main/templates/zswap-da)
203
78
 
204
- ### Package Structure
79
+ ## Links
205
80
 
206
- ```
207
- npm-midnight-indexer/
208
- ├── index.js # Main entry point
209
- ├── binary.js # Binary download logic
210
- ├── docker.js # Docker execution logic
211
- ├── run_midnight_indexer.js # Binary execution logic
212
- ├── package.json # Package configuration
213
- └── README.md # This file
214
- ```
215
-
216
- ### Testing
217
-
218
- ```bash
219
- # Test Docker detection
220
- node -e "const { checkIfDockerExists } = require('./docker.js'); checkIfDockerExists().then(console.log)"
221
-
222
- # Test help
223
- node index.js --help
224
-
225
- # Test platform detection
226
- node -e "console.log('Platform:', require('os').platform())"
227
- ```
81
+ - Docs: https://effectstream.github.io/docs/packages/binaries/midnight-indexer
82
+ - Source: https://github.com/effectstream/effectstream/tree/main/packages/binaries/midnight-indexer
83
+ - Midnight Network: https://midnight.network
84
+ - Indexer image: https://hub.docker.com/r/midnightntwrk/indexer-standalone
228
85
 
229
86
  ## License
230
87
 
231
- ISC
232
-
233
- ## Support
234
-
235
- For issues related to:
236
-
237
- - **This package**: Open an issue in the repository
238
- - **Midnight Protocol**: Visit [midnight.network](https://midnight.network)
239
- - **Docker**: Check [Docker documentation](https://docs.docker.com/)
240
-
241
- ## Related Links
242
-
243
- - [Midnight Network](https://midnight.network)
244
- - [Docker Hub - Midnight Indexer](https://hub.docker.com/r/midnightntwrk/indexer-standalone)
245
- - [Node.js](https://nodejs.org/)
88
+ ISC.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@effectstream/npm-midnight-indexer",
3
- "version": "0.100.14",
3
+ "version": "0.100.16",
4
4
  "description": "Downloads and runs the Midnight Indexer. It needs a running Midnight Node",
5
5
  "main": "index.js",
6
6
  "scripts": {