n8n-nodes-dremio 2.0.16 → 2.0.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +126 -83
- package/package.json +2 -2
- /package/dist/{Dremio → nodes/Dremio}/dremio.svg +0 -0
package/README.md
CHANGED
|
@@ -1,111 +1,154 @@
|
|
|
1
1
|
# Dremio n8n Connector
|
|
2
2
|
|
|
3
|
-
A
|
|
3
|
+
A community n8n node for executing SQL queries against **Dremio Cloud** and **Dremio Software** directly from your n8n workflows.
|
|
4
|
+
|
|
5
|
+
[](https://www.npmjs.com/package/n8n-nodes-dremio)
|
|
6
|
+
[](LICENSE)
|
|
4
7
|
|
|
5
8
|
## Features
|
|
6
9
|
|
|
7
|
-
-
|
|
8
|
-
-
|
|
9
|
-
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
-
|
|
10
|
+
- **Execute SQL** — Run any SQL query supported by Dremio
|
|
11
|
+
- **Dual Compatibility** — First-class support for both Dremio Cloud and Dremio Software (self-hosted)
|
|
12
|
+
- **Credential-Based Auth** — Authentication via Personal Access Token (PAT) using n8n's built-in `httpRequestWithAuthentication`
|
|
13
|
+
- **Automatic Credential Test** — Validates your credentials when you save them in n8n
|
|
14
|
+
- **SSL Flexibility** — Option to skip SSL certificate validation for self-hosted instances with self-signed certificates
|
|
15
|
+
- **Data Lineage** — Full `pairedItem` tracking for n8n's data lineage feature
|
|
16
|
+
- **Proper Error Handling** — Uses `NodeOperationError` for clear, actionable error messages
|
|
13
17
|
|
|
14
18
|
## Installation
|
|
15
19
|
|
|
16
|
-
###
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
npm run build
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
2. **Link**:
|
|
46
|
-
Navigate to your n8n custom extension directory (usually `~/.n8n/custom`):
|
|
47
|
-
```bash
|
|
48
|
-
mkdir -p ~/.n8n/custom
|
|
49
|
-
cd ~/.n8n/custom
|
|
50
|
-
npm link /path/to/dremio-n8n
|
|
51
|
-
```
|
|
52
|
-
|
|
53
|
-
3. **Restart**: Restart your n8n instance.
|
|
20
|
+
### n8n Community Nodes (Recommended)
|
|
21
|
+
|
|
22
|
+
1. Go to your n8n dashboard
|
|
23
|
+
2. Navigate to **Settings** > **Community Nodes**
|
|
24
|
+
3. Click **Install a community node**
|
|
25
|
+
4. Enter: `n8n-nodes-dremio`
|
|
26
|
+
5. Click **Install**
|
|
27
|
+
|
|
28
|
+
The **Dremio** node will now be available in your workflow editor.
|
|
29
|
+
|
|
30
|
+
### Self-Hosted n8n (Manual)
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
# Clone and build
|
|
34
|
+
git clone https://github.com/developer-advocacy-dremio/dremio-n8n-connector.git
|
|
35
|
+
cd dremio-n8n-connector
|
|
36
|
+
npm install
|
|
37
|
+
npm run build
|
|
38
|
+
|
|
39
|
+
# Link to n8n
|
|
40
|
+
mkdir -p ~/.n8n/custom
|
|
41
|
+
cd ~/.n8n/custom
|
|
42
|
+
npm link /path/to/dremio-n8n-connector
|
|
43
|
+
|
|
44
|
+
# Restart n8n
|
|
45
|
+
```
|
|
54
46
|
|
|
55
47
|
## Usage Guide
|
|
56
48
|
|
|
57
49
|
### 1. Add the Node
|
|
58
|
-
Open your n8n workflow, click
|
|
50
|
+
Open your n8n workflow, click **+**, and search for **Dremio**.
|
|
59
51
|
|
|
60
52
|
### 2. Configure Credentials
|
|
61
|
-
You can configure a single credential to use across multiple nodes.
|
|
62
53
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
54
|
+
Click the credential dropdown and create a new **Dremio API** credential.
|
|
55
|
+
|
|
56
|
+
#### Dremio Cloud
|
|
57
|
+
| Field | Value |
|
|
58
|
+
|:---|:---|
|
|
59
|
+
| **Type** | `Cloud` |
|
|
60
|
+
| **Base URL** | `https://api.dremio.cloud` (default) or `https://api.eu.dremio.cloud` for EU |
|
|
61
|
+
| **Project ID** | Found in Dremio Cloud → Project Settings |
|
|
62
|
+
| **Access Token** | Your Personal Access Token (PAT) |
|
|
63
|
+
| **Ignore SSL Issues** | `False` |
|
|
64
|
+
|
|
65
|
+
#### Dremio Software
|
|
66
|
+
| Field | Value |
|
|
67
|
+
|:---|:---|
|
|
68
|
+
| **Type** | `Software` |
|
|
69
|
+
| **Base URL** | `https://<HOST>:9047/api/v3` |
|
|
70
|
+
| **Access Token** | Your Personal Access Token (PAT) |
|
|
71
|
+
| **Ignore SSL Issues** | `True` if using a self-signed certificate |
|
|
69
72
|
|
|
70
|
-
|
|
71
|
-
- **Type**: Select `Software`
|
|
72
|
-
- **Base URL**: Your Dremio API base URL, e.g., `http://dremio.example.com:9047/api/v3`.
|
|
73
|
-
- **Access Token**: Your Personal Access Token (PAT).
|
|
74
|
-
- **Ignore SSL Issues**: set to `True` if using a self-signed certificate.
|
|
73
|
+
> **Tip:** Click **Test Credential** to verify connectivity before saving.
|
|
75
74
|
|
|
76
75
|
### 3. Execute SQL
|
|
77
|
-
-
|
|
78
|
-
-
|
|
79
|
-
-
|
|
80
|
-
|
|
81
|
-
|
|
76
|
+
- **Resource**: `Query`
|
|
77
|
+
- **Operation**: `Execute`
|
|
78
|
+
- **SQL Query**: Your SQL statement
|
|
79
|
+
|
|
80
|
+
**Examples:**
|
|
81
|
+
```sql
|
|
82
|
+
-- Query sample data
|
|
83
|
+
SELECT * FROM "Samples"."samples.dremio.com"."NYC-taxi-trips" LIMIT 10
|
|
84
|
+
|
|
85
|
+
-- Use with expressions from previous nodes
|
|
86
|
+
SELECT * FROM my_table WHERE id = '{{ $json.id }}'
|
|
87
|
+
```
|
|
82
88
|
|
|
83
|
-
## How
|
|
84
|
-
The connector uses the Dremio REST API to submit and monitor jobs:
|
|
85
|
-
1. **Submission**: It posts the SQL query to the `/sql` endpoint.
|
|
86
|
-
2. **Polling**: It receives a Job ID and polls the Job Status endpoint until the state is `COMPLETED`.
|
|
87
|
-
3. **Retrieval**: Once completed, it fetches the results from the `/results` endpoint and returns them as JSON items.
|
|
89
|
+
## How It Works
|
|
88
90
|
|
|
89
|
-
|
|
91
|
+
The connector uses the Dremio REST API's asynchronous job execution model:
|
|
90
92
|
|
|
91
|
-
|
|
93
|
+
1. **Submit** — `POST` the SQL query to the `/sql` endpoint → receives a Job ID
|
|
94
|
+
2. **Poll** — Checks job status every second until `COMPLETED`, `FAILED`, or `CANCELED`
|
|
95
|
+
3. **Fetch** — Retrieves results from `/job/{id}/results` and outputs rows as n8n items
|
|
96
|
+
|
|
97
|
+
Each output item includes `pairedItem` metadata for n8n's data lineage tracking.
|
|
98
|
+
|
|
99
|
+
## Project Structure
|
|
92
100
|
|
|
93
101
|
| Path | Description |
|
|
94
|
-
|
|
95
|
-
|
|
|
96
|
-
|
|
|
97
|
-
| `nodes/Dremio/
|
|
98
|
-
|
|
|
99
|
-
|
|
|
100
|
-
|
|
|
101
|
-
|
|
102
|
+
|:---|:---|
|
|
103
|
+
| `credentials/DremioApi.credentials.ts` | Credential definition — auth, SSL, and credential test |
|
|
104
|
+
| `nodes/Dremio/Dremio.node.ts` | Node logic — SQL execution via submit/poll/fetch |
|
|
105
|
+
| `nodes/Dremio/dremio.svg` | Node icon |
|
|
106
|
+
| `scripts/verify_api.js` | Standalone API test script (reads from `.env`) |
|
|
107
|
+
| `.github/workflows/publish.yml` | GitHub Actions — auto-publish to npm with provenance on release |
|
|
108
|
+
| `.env.template` | Template for local API testing credentials |
|
|
109
|
+
|
|
110
|
+
## Development
|
|
111
|
+
|
|
112
|
+
See the **[Developer Guide](DEVELOPMENT.md)** for implementation details.
|
|
113
|
+
|
|
114
|
+
### Quick Start
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
npm install # Install dependencies
|
|
118
|
+
npm run build # Compile TypeScript + copy SVG assets
|
|
119
|
+
npm run lint # Run ESLint
|
|
120
|
+
npm run dev # Watch mode for development
|
|
121
|
+
```
|
|
102
122
|
|
|
103
|
-
|
|
123
|
+
### Testing API Connectivity
|
|
104
124
|
|
|
105
|
-
|
|
125
|
+
```bash
|
|
126
|
+
cp .env.template .env # Fill in your Dremio credentials
|
|
127
|
+
node scripts/verify_api.js
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
### Publishing
|
|
131
|
+
|
|
132
|
+
Releases are automated via GitHub Actions. To publish a new version:
|
|
133
|
+
|
|
134
|
+
1. Bump the version in `package.json`
|
|
135
|
+
2. Commit, tag, and push:
|
|
136
|
+
```bash
|
|
137
|
+
git tag v2.x.x
|
|
138
|
+
git push origin main --tags
|
|
139
|
+
```
|
|
140
|
+
3. Create a **GitHub Release** from the tag — this triggers the publish workflow
|
|
106
141
|
|
|
107
142
|
## Troubleshooting
|
|
108
143
|
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
144
|
+
| Problem | Solution |
|
|
145
|
+
|:---|:---|
|
|
146
|
+
| **Job Failed** | Check the Dremio UI Jobs page for detailed SQL error messages |
|
|
147
|
+
| **SSL Error** | Toggle **Ignore SSL Issues** to `True` in credentials |
|
|
148
|
+
| **Timeout** | Use `LIMIT` clauses or paginate large result sets |
|
|
149
|
+
| **Credential Test Fails** | Verify your PAT is valid and Project ID is correct (Cloud) |
|
|
150
|
+
| **401 Unauthorized** | Regenerate your Personal Access Token in Dremio |
|
|
151
|
+
|
|
152
|
+
## License
|
|
153
|
+
|
|
154
|
+
[MIT](LICENSE)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "n8n-nodes-dremio",
|
|
3
|
-
"version": "2.0.
|
|
3
|
+
"version": "2.0.17",
|
|
4
4
|
"description": "n8n node for Dremio",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"n8n-community-node-package"
|
|
@@ -17,7 +17,7 @@
|
|
|
17
17
|
},
|
|
18
18
|
"main": "index.js",
|
|
19
19
|
"scripts": {
|
|
20
|
-
"build": "tsc && copyfiles -u
|
|
20
|
+
"build": "tsc && copyfiles -u 0 \"nodes/**/*.svg\" dist/",
|
|
21
21
|
"dev": "tsc --watch",
|
|
22
22
|
"test": "jest",
|
|
23
23
|
"lint": "eslint nodes/ credentials/ --ext .ts",
|
|
File without changes
|