@bkmj/node-red-contrib-odbcmj 1.6.6 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +63 -107
- package/odbc.html +224 -222
- package/odbc.js +380 -228
- package/package.json +26 -7
package/README.md
CHANGED
|
@@ -1,147 +1,103 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Node-RED Contrib ODBC MJ
|
|
2
2
|
|
|
3
|
-
A Node-RED
|
|
3
|
+
A powerful and robust Node-RED node to connect to any ODBC data source. It features connection pooling, advanced retry logic, secure credential management, and result set streaming.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
## Acknowledgment
|
|
7
|
-
|
|
8
|
-
This node is an unofficial fork of node-red-contrib-odbc by Mark Irish (https://github.com/markdirish/node-red-contrib-odbc) and is vastly inspired by it. It also takes ideas from node-red-contrib-odbc2 by AIS Automation (https://github.com/AISAutomation/node-red-contrib-odbc2).
|
|
9
|
-
|
|
10
|
-
**Overall changes:**
|
|
11
|
-
|
|
12
|
-
* Can use Mustache as well as a parameter array.
|
|
13
|
-
* Warnings when Mustache will render an undefined variable.
|
|
14
|
-
* Fixes the output field option so that nested objects can be used.
|
|
15
|
-
* Fixes the checkbox for the pool shrink option.
|
|
16
|
-
* Uses ace/mode/sql for the SQL input field.
|
|
17
|
-
* Connection nodes can have individually defined names.
|
|
18
|
-
* Selectable SQL syntax checker.
|
|
19
|
-
* Allows parameters to be passed as an object, mapping values to named parameters in the query.
|
|
20
|
-
* Automatically handles prepared statements based on the query and parameters.
|
|
21
|
-
|
|
22
|
-
## Installation
|
|
23
|
-
|
|
24
|
-
This package is not available from within the Node-RED palette tool. Instead, in your Node-RED user directory (usually `~/.node-red/`), download through the `npm` utility:
|
|
25
|
-
```
|
|
26
|
-
npm install node-red-contrib-odbcmj
|
|
27
|
-
```
|
|
5
|
+
This node is a fork with significant enhancements to provide stability and advanced features for enterprise use cases.
|
|
28
6
|
|
|
29
|
-
|
|
7
|
+
## Features
|
|
30
8
|
|
|
31
|
-
|
|
9
|
+
- **Connection Pooling**: Efficiently manages database connections for high performance.
|
|
10
|
+
- **Hybrid Configuration**: Configure connections using simple structured fields or a full connection string for maximum flexibility.
|
|
11
|
+
- **Secure Credential Storage**: Passwords are saved using Node-RED's built-in credential system.
|
|
12
|
+
- **Connection Tester**: Instantly validate your connection settings from the configuration panel.
|
|
13
|
+
- **Advanced Retry Logic**: Automatically handles connection errors with configurable delays and retries to ensure flow resilience.
|
|
14
|
+
- **Result Streaming**: Process queries with millions of rows without exhausting memory by streaming results as chunks.
|
|
15
|
+
- **Syntax Checker**: Optionally parse the SQL query to validate its structure.
|
|
32
16
|
|
|
33
|
-
|
|
17
|
+
---
|
|
34
18
|
|
|
35
|
-
|
|
36
|
-
parameters.
|
|
37
|
-
* **`odbc`**: A node for running queries with or without parameters.
|
|
19
|
+
## Nodes
|
|
38
20
|
|
|
39
21
|
### `odbc config`
|
|
40
22
|
|
|
41
|
-
A configuration node that manages
|
|
42
|
-
an `odbc` node receives an input message.
|
|
43
|
-
|
|
44
|
-
#### Properties
|
|
45
|
-
|
|
46
|
-
* (**required**) **`connectionString`**: <`string`>
|
|
47
|
-
|
|
48
|
-
An ODBC connection string that defines your DSN and/or connection string options.
|
|
49
|
-
Check your ODBC driver documentation for more information about valid connection strings.
|
|
50
|
-
|
|
51
|
-
Example:
|
|
52
|
-
```
|
|
53
|
-
DSN=MyDSN;DFT=2;
|
|
54
|
-
```
|
|
23
|
+
A configuration node that manages the connection to your database.
|
|
55
24
|
|
|
56
|
-
|
|
25
|
+
#### Connection Modes
|
|
57
26
|
|
|
58
|
-
|
|
27
|
+
Version 2.0 introduces two ways to configure your connection:
|
|
59
28
|
|
|
60
|
-
|
|
29
|
+
##### 1. Structured Fields Mode (Recommended)
|
|
61
30
|
|
|
62
|
-
|
|
31
|
+
This is the easiest and most secure way to set up a connection for common databases.
|
|
63
32
|
|
|
64
|
-
|
|
33
|
+
- **Database Type**: Select your database (e.g., SQL Server, PostgreSQL, MySQL). The node will use the appropriate driver name and connection string syntax. For unlisted databases, choose "Other" and provide the driver name manually.
|
|
34
|
+
- **Server**: The hostname or IP address of the database server, optionally followed by a comma and the port number (e.g., `mydb.server.com,1433`).
|
|
35
|
+
- **Database**: The name of the database to connect to (optional).
|
|
36
|
+
- **User**: The username for authentication.
|
|
37
|
+
- **Password**: The password for authentication. This is stored securely using Node-RED's credential system.
|
|
65
38
|
|
|
66
|
-
|
|
39
|
+
##### 2. Connection String Mode (Advanced)
|
|
67
40
|
|
|
68
|
-
|
|
41
|
+
This mode gives you full control for complex or non-standard connection strings.
|
|
69
42
|
|
|
70
|
-
|
|
43
|
+
- **Connection String**: Enter the complete ODBC connection string.
|
|
44
|
+
- **Password Handling**: For security, **do not** write your password directly in the string. Instead, use the `{{{password}}}` placeholder. The node will automatically replace it with the password entered in the secure `Password` field below.
|
|
45
|
+
- Example: `DRIVER={...};SERVER=...;UID=myuser;PWD={{{password}}};`
|
|
71
46
|
|
|
72
|
-
|
|
47
|
+
#### Test Connection
|
|
73
48
|
|
|
74
|
-
|
|
49
|
+
A **Test Connection** button in the configuration panel allows you to instantly verify your settings without deploying the flow.
|
|
75
50
|
|
|
76
|
-
|
|
51
|
+
#### Pool Options
|
|
77
52
|
|
|
78
|
-
|
|
79
|
-
Default:
|
|
53
|
+
- **`initialSize`** `<number>` (optional): The number of connections to create when the pool is initialized. Default: 5.
|
|
54
|
+
- **`incrementSize`** `<number>` (optional): The number of connections to create when the pool is exhausted. Default: 5.
|
|
55
|
+
- **`maxSize`** `<number>` (optional): The maximum number of connections allowed in the pool. Default: 15.
|
|
56
|
+
- **`shrinkPool`** `<boolean>` (optional): Whether to reduce the number of connections to `initialSize` when they are returned to the pool. Default: true.
|
|
57
|
+
- **`connectionTimeout`** `<number>` (optional): The number of seconds for a connection to remain idle before closing. Default: 3.
|
|
58
|
+
- **`loginTimeout`** `<number>` (optional): The number of seconds for an attempt to create a connection to succeed. Default: 3.
|
|
80
59
|
|
|
81
|
-
|
|
60
|
+
#### Error Handling & Retry
|
|
82
61
|
|
|
83
|
-
|
|
62
|
+
- **`retryFreshConnection`** `<boolean>` (optional): If a query fails, the node will retry once with a brand new connection. If this succeeds, the entire connection pool is reset to clear any stale connections. Default: false.
|
|
63
|
+
- **`retryDelay`** `<number>` (optional): If both the pooled and the fresh connection attempts fail, this sets a delay in seconds before another retry is attempted. This prevents infinite loops. A value of **0** disables further automatic retries. Default: 5.
|
|
64
|
+
- **`retryOnMsg`** `<boolean>` (optional): If the node is waiting for a timed retry, a new incoming message can override the timer and trigger an immediate retry. Default: true.
|
|
84
65
|
|
|
85
|
-
|
|
66
|
+
#### Advanced
|
|
86
67
|
|
|
87
|
-
|
|
68
|
+
- **`syntaxChecker`** `<boolean>` (optional): If activated, the query string will be [parsed](https://www.npmjs.com/package/node-sql-parser#create-ast-for-sql-statement) and appended as an object to the output message at `msg.parsedQuery`. Default: false.
|
|
69
|
+
- **`syntax`** `<string>` (optional): The SQL flavor to use for the syntax checker. Default: mysql.
|
|
88
70
|
|
|
71
|
+
---
|
|
89
72
|
|
|
90
73
|
### `odbc`
|
|
91
74
|
|
|
92
|
-
|
|
75
|
+
This node executes a query against the configured database when it receives a message.
|
|
93
76
|
|
|
94
77
|
#### Properties
|
|
95
78
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
* (optional) **`query`**: <`string`>
|
|
101
|
-
|
|
102
|
-
A valid SQL query string.
|
|
103
|
-
|
|
104
|
-
* Can contain parameters inserted using Mustache syntax (e.g., `{{{payload}}}`).
|
|
105
|
-
* Can use placeholders (`?`) for parameters.
|
|
106
|
-
* Can embed parameters directly in the query string.
|
|
107
|
-
|
|
108
|
-
* (**required**) **`result to`**: <`dot-notation string`>
|
|
109
|
-
|
|
110
|
-
The JSON nested element structure that will contain the result output. The string must be a valid JSON object structure using dot-notation, minus the `msg.` (e.g., `payload.results`) and must not start or end with a period. Square bracket notation is not allowed. The node input object is carried out to the output, as long as the output object name does not conflict with it. If the targeted output JSON object was already present in the input, the result from the query will be appended to it
|
|
111
|
-
if it was itself an object (but not an array); otherwise, the original key/value pair will be overwritten.
|
|
112
|
-
|
|
113
|
-
Example:
|
|
114
|
-
|
|
115
|
-
* `input msg: {"payload": {"result": {"othervalue": 10} } };`
|
|
116
|
-
* `result to: payload.results.values`
|
|
117
|
-
|
|
118
|
-
In this case, `values` will be appended to `result` without overwriting `othervalue`. If `result` had been a string, then it would have been replaced by `values`.
|
|
79
|
+
- **`connection`** `<odbc config>` (**required**): The configuration node that defines the connection settings.
|
|
80
|
+
- **`query`** `<string>` (optional): The SQL query to execute. Can contain Mustache syntax (e.g., `{{{payload.id}}}`) which will be rendered using the incoming message object.
|
|
81
|
+
- **`result to`** `<string>` (**required**): The property of the output message where the results will be stored (e.g., `payload`). Default: `payload`.
|
|
119
82
|
|
|
120
83
|
#### Inputs
|
|
121
84
|
|
|
122
|
-
The
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
* **`payload`**:
|
|
126
|
-
* A JSON string containing a `query` property with the SQL string.
|
|
127
|
-
* An object with a `query` property containing the SQL string.
|
|
128
|
-
* **`parameters`**: <`array` or `object`>
|
|
129
|
-
* Can be an array of values or an object mapping parameter names to values.
|
|
130
|
-
* If the query contains placeholders (`?`) and `msg.parameters` is an object, the values will be automatically mapped to the placeholders based on the order of the parameters in the query.
|
|
131
|
-
|
|
132
|
-
#### Outputs
|
|
133
|
-
|
|
134
|
-
Returns a message containing:
|
|
85
|
+
The node can be configured dynamically using the incoming `msg` object:
|
|
86
|
+
- **`msg.query`**: A query string that will override the one configured in the node.
|
|
87
|
+
- **`msg.parameters`**: An array of values for prepared statements (when the query contains `?` placeholders).
|
|
135
88
|
|
|
136
|
-
|
|
137
|
-
* **`odbc`**: <`object`> Contains additional information returned by the `odbc` module.
|
|
138
|
-
* **`parsedQuery`**: <`object`> (Optional) The parsed SQL query if the syntax checker is enabled.
|
|
89
|
+
#### Streaming Results
|
|
139
90
|
|
|
140
|
-
|
|
91
|
+
For queries that return a large number of rows, streaming prevents high memory usage.
|
|
141
92
|
|
|
142
|
-
|
|
93
|
+
- **`Stream Results`** `<boolean>`: Enables or disables streaming mode. When enabled, the node will output multiple messages, one for each chunk of rows. Default: false.
|
|
94
|
+
- **`Chunk Size`** `<number>`: The number of rows to include in each output message. A value of `1` means one message will be sent for every single row. Default: 1.
|
|
143
95
|
|
|
144
|
-
|
|
145
|
-
* **`msg.parameters` Object:** If the `msg.parameters` object is provided (either as an array or an object), the node will use a prepared statement.
|
|
96
|
+
##### Streaming Output Format
|
|
146
97
|
|
|
147
|
-
|
|
98
|
+
When streaming is active, each output message will contain:
|
|
99
|
+
- A payload (or the configured output property) containing an array of rows for the current chunk.
|
|
100
|
+
- A `msg.odbc_stream` object with metadata for tracking progress:
|
|
101
|
+
- `index`: The starting index of the current chunk (e.g., 0, 100, 200...).
|
|
102
|
+
- `count`: The number of rows in the current chunk.
|
|
103
|
+
- `complete`: A boolean that is `true` only on the very last message of the stream (or if the result set was empty), and `false` otherwise. This is useful for triggering a downstream action once all rows have been processed.
|