@bkmj/node-red-contrib-odbcmj 2.0.2 → 2.1.0

package/README.md

@@ -1,6 +1,6 @@
 # Node-RED Contrib ODBC MJ
 
-A powerful and robust Node-RED node to connect to any ODBC data source. It features connection pooling, advanced retry logic, secure credential management, and result set streaming.
+A powerful and robust Node-RED node to connect to any ODBC data source. It features connection pooling, advanced retry logic, secure credential management, dynamic query sources, and result set streaming.
 
 This node is a fork with significant enhancements to provide stability and advanced features for enterprise use cases.
 
@@ -10,6 +10,7 @@ This node is a fork with significant enhancements to provide stability and advan
 -   **Hybrid Configuration**: Configure connections using simple structured fields or a full connection string for maximum flexibility.
 -   **Secure Credential Storage**: Passwords are saved using Node-RED's built-in credential system.
 -   **Connection Tester**: Instantly validate your connection settings from the configuration panel.
+-   **Dynamic Inputs**: Source your SQL query and parameters from message properties, flow/global context, or environment variables.
 -   **Advanced Retry Logic**: Automatically handles connection errors with configurable delays and retries to ensure flow resilience.
 -   **Result Streaming**: Process queries with millions of rows without exhausting memory by streaming results as chunks.
 -   **Syntax Checker**: Optionally parse the SQL query to validate its structure.
@@ -38,11 +39,9 @@ This is the easiest and most secure way to set up a connection for common databa
 
 ##### 2. Connection String Mode (Advanced)
 
-This mode gives you full control for complex or non-standard connection strings.
+This mode gives you full control for complex or non-standard connection strings. In this mode, you are responsible for the entire content of the string.
 
 -   **Connection String**: Enter the complete ODBC connection string.
--   **Password Handling**: For security, **do not** write your password directly in the string. Instead, use the `{{{password}}}` placeholder. The node will automatically replace it with the password entered in the secure `Password` field below.
-    -   Example: `DRIVER={...};SERVER=...;UID=myuser;PWD={{{password}}};`
 
 #### Test Connection
 
@@ -60,7 +59,7 @@ A **Test Connection** button in the configuration panel allows you to instantly
 #### Error Handling & Retry
 
 -   **`retryFreshConnection`** `<boolean>` (optional): If a query fails, the node will retry once with a brand new connection. If this succeeds, the entire connection pool is reset to clear any stale connections. Default: false.
--   **`retryDelay`** `<number>` (optional): If both the pooled and the fresh connection attempts fail, this sets a delay in seconds before another retry is attempted. This prevents infinite loops. A value of **0** disables further automatic retries. Default: 5.
+-   **`retryDelay`** `<number>` (optional): If both the pooled and the fresh connection attempts fail, this sets a delay in seconds before another retry is attempted. A value of **0** disables further automatic retries. Default: 5.
 -   **`retryOnMsg`** `<boolean>` (optional): If the node is waiting for a timed retry, a new incoming message can override the timer and trigger an immediate retry. Default: true.
 
 #### Advanced
@@ -77,14 +76,20 @@ This node executes a query against the configured database when it receives a me
 #### Properties
 
 -   **`connection`** `<odbc config>` (**required**): The configuration node that defines the connection settings.
--   **`query`** `<string>` (optional): The SQL query to execute. Can contain Mustache syntax (e.g., `{{{payload.id}}}`) which will be rendered using the incoming message object.
--   **`result to`** `<string>` (**required**): The property of the output message where the results will be stored (e.g., `payload`). Default: `payload`.
+-   **`Query`** `<string>` (optional): A default SQL query to execute if no query is provided dynamically from an input source. Can contain Mustache syntax (e.g., `{{{payload.id}}}`).
+-   **`Result to`** `<string>` (**required**): The property of the output message where the results will be stored (e.g., `payload`). Default: `payload`.
 
-#### Inputs
+#### Dynamic Inputs
 
-The node can be configured dynamically using the incoming `msg` object:
--   **`msg.query`**: A query string that will override the one configured in the node.
--   **`msg.parameters`**: An array of values for prepared statements (when the query contains `?` placeholders).
+To make the node highly flexible, the SQL query and its parameters can be sourced dynamically using **Typed Inputs**.
+
+-   **`Query Source`**: A Typed Input that specifies where to find the query string at runtime. This value **overrides** the static query defined in the editor.
+    -   *Default*: `msg.query` (for backward compatibility).
+    -   *Example*: Set to `flow.mySqlQuery` to read the query from a flow context variable.
+
+-   **`Parameters Source`**: A Typed Input that specifies where to find the array or object of parameters for prepared statements.
+    -   *Default*: `msg.parameters`.
+    -   *Example*: Set to `msg.payload.bindings` to use the array found in that property.
 
 #### Streaming Results
 

package/odbc.html

@@ -31,13 +31,11 @@
         oneditprepare: function () {
             var node = this;
 
-            // Logique pour les sections déroulantes
             $('.form-section-header').on('click', function() {
                 $(this).toggleClass('expanded');
                 $(this).next('.form-section-content').slideToggle();
             });
 
-            // Logique pour le mode de connexion hybride
             function toggleConnectionMode(mode) {
                 if (mode === 'structured') {
                     $(".config-mode-structured").show();
@@ -63,7 +61,6 @@
                 toggleDriverField($(this).val());
             }).trigger("change");
 
-            // Logiques pour les cases à cocher
             $("#node-config-input-syntaxtick").on("change", function () {
                 $(".input-syntax").toggle(this.checked);
             }).trigger("change");
@@ -72,7 +69,6 @@
                 $(".retry-options").toggle(this.checked);
             }).trigger("change");
 
-            // Logique pour le bouton de test
             $('#node-config-test-connection').on('click', function() {
                 var button = $(this);
                 var originalText = "Test Connection";
@@ -81,7 +77,6 @@
                 button.text(' Testing...').prop('disabled', true);
                 
                 var connectionMode = $("#node-config-input-connectionMode").val();
-                
                 var configData = {
                     connectionMode: connectionMode,
                     dbType: $("#node-config-input-dbType").val(),
@@ -90,7 +85,6 @@
                     database: $("#node-config-input-database").val(),
                     user: $("#node-config-input-user").val(),
                     connectionString: $("#node-config-input-connectionString").val(),
-                    // Le mot de passe n'est envoyé que si on est en mode structuré
                     password: (connectionMode === 'structured') ? $("#node-config-input-password").val() : null
                 };
 
@@ -277,7 +271,12 @@
             query: { value: "" },
             outputObj: { value: "payload" },
             streaming: { value: false },
-            streamChunkSize: { value: 1, validate: RED.validators.number() }
+            streamChunkSize: { value: 1, validate: RED.validators.number() },
+            // Nouveaux `defaults` pour les Typed Inputs
+            querySource: { value: "query", required: false },
+            querySourceType: { value: "msg", required: false },
+            paramsSource: { value: "parameters", required: false },
+            paramsSourceType: { value: "msg", required: false }
         },
         inputs: 1,
         outputs: 1,
@@ -295,6 +294,19 @@
             $("#node-input-streaming").on("change", function() {
                 $(".stream-options").toggle(this.checked);
             }).trigger("change");
+
+            // Initialisation des Typed Inputs
+            $("#node-input-querySource").typedInput({
+                default: 'msg',
+                typeField: "#node-input-querySourceType",
+                types: ['msg', 'flow', 'global', 'env', 'str', 'jsonata']
+            });
+            
+            $("#node-input-paramsSource").typedInput({
+                default: 'msg',
+                typeField: "#node-input-paramsSourceType",
+                types: ['msg', 'flow', 'global', 'env', 'jsonata']
+            });
         },
         oneditsave: function () {
             this.query = this.editor.getValue();
@@ -318,14 +330,29 @@
         <input type="text" id="node-input-connection" />
     </div>
     <div class="form-row node-text-editor-row">
-        <label for="node-input-query" style="width: 100% !important;"><i class="fa fa-search"></i> Query</label>
+        <label for="node-input-query" style="width: 100% !important;"><i class="fa fa-file-code-o"></i> Query (fallback)</label>
         <div style="height: 250px;" class="node-text-editor" id="node-input-query-editor"></div>
     </div>
     <div class="form-row">
-        <label for="node-input-outputObj"><i class="fa fa-edit"></i> Result to</label>
+        <label for="node-input-outputObj"><i class="fa fa-sign-out"></i> Result to</label>
         <span>msg.</span><input type="text" id="node-input-outputObj" placeholder="payload" style="width: 64%;"/>
     </div>
+    
+    <hr/>
+    
+    <div class="form-row">
+        <label for="node-input-querySource"><i class="fa fa-crosshairs"></i> Query Source</label>
+        <input type="text" id="node-input-querySource" style="width: 70%;">
+        <input type="hidden" id="node-input-querySourceType">
+    </div>
+    <div class="form-row">
+        <label for="node-input-paramsSource"><i class="fa fa-list-ol"></i> Parameters Source</label>
+        <input type="text" id="node-input-paramsSource" style="width: 70%;">
+        <input type="hidden" id="node-input-paramsSourceType">
+    </div>
+
     <hr/>
+
     <div class="form-row">
         <label for="node-input-streaming" style="width: auto;"><i class="fa fa-arrows-v"></i> Stream Results</label>
         <input type="checkbox" id="node-input-streaming" style="display: inline-block; width: auto; vertical-align: top;">
@@ -335,4 +362,73 @@
         <input type="number" id="node-input-streamChunkSize" placeholder="1" style="width: 100px;">
         <span class="form-tips">Number of rows per output message.</span>
     </div>
+</script>
+
+
+<script type="text/markdown" data-help-name="odbc config">
+A configuration node that manages the connection to your database.
+
+### Connection Modes
+Version 2.0 introduces two ways to configure your connection:
+
+#### 1. Structured Fields Mode (Recommended)
+This is the easiest and most secure way to set up a connection for common databases.
+- **Database Type**: Select your database (e.g., SQL Server, PostgreSQL, MySQL). The node will use the appropriate driver name and connection string syntax. For unlisted databases, choose "Other" and provide the driver name manually.
+- **Server**: The hostname or IP address of the database server, optionally followed by a comma and the port number (e.g., `mydb.server.com,1433`).
+- **Database**: The name of the database to connect to (optional).
+- **User**: The username for authentication.
+- **Password**: The password for authentication. This is stored securely using Node-RED's credential system.
+
+#### 2. Connection String Mode (Advanced)
+This mode gives you full control for complex or non-standard connection strings.
+- **Connection String**: Enter the complete ODBC connection string. It is your responsibility to provide a valid string for your driver.
+
+### Test Connection
+A **Test Connection** button in the configuration panel allows you to instantly verify your settings without deploying the flow.
+
+### Pool Options
+- **`initialSize`**: The number of connections to create when the pool is initialized. Default: 5.
+- **`maxSize`**: The maximum number of connections allowed in the pool. Default: 15.
+- (See `odbc` package documentation for more details on pool options).
+
+### Error Handling & Retry
+- **`retryFreshConnection`**: If a query fails, the node will retry once with a brand new connection. If this succeeds, the entire connection pool is reset to clear any stale connections.
+- **`retryDelay`**: If both attempts fail, this sets a delay in seconds before another retry is attempted. A value of **0** disables further automatic retries.
+- **`retryOnMsg`**: If the node is waiting for a timed retry, a new incoming message can override the timer and trigger an immediate retry.
+
+### Advanced
+- **`syntaxChecker`**: If activated, the query string will be parsed and appended to the output message at `msg.parsedQuery`.
+</script>
+
+<script type="text/markdown" data-help-name="odbc">
+Executes a query against a configured ODBC data source.
+
+### Inputs
+
+- **Query Source** (optional): Specify where to get the query string from. You can use a message property (`msg.`), flow context (`flow.`), global context (`global.`), an environment variable (`env.`), or a JSONata expression. If the source is empty or not found, the node will use the query from the "Query (fallback)" editor below.
+  - *Default behavior (for backward compatibility): `msg.query`*
+
+- **Parameters Source** (optional): Specify where to get the parameters for a prepared statement. This should resolve to an array of values.
+  - *Default behavior (for backward compatibility): `msg.parameters`*
+
+### Properties
+
+- **Connection**: The `odbc config` node to use.
+- **Query (fallback)**: A static SQL query to run if the "Query Source" does not provide one. Can contain Mustache syntax (e.g., `{{{payload.id}}}`).
+- **Result to**: The `msg` property where the query result will be stored. Default: `payload`.
+
+### Streaming Results
+
+For queries that return a large number of rows, streaming prevents high memory usage by sending multiple messages instead of one large one.
+
+- **`Stream Results`**: Enables or disables streaming mode.
+- **`Chunk Size`**: The number of rows to include in each output message. A value of `1` means one message will be sent for every single row.
+
+#### Streaming Output Format
+When streaming is active, each output message will contain:
+- A payload (or the configured output property) containing an array of rows for the current chunk.
+- A `msg.odbc_stream` object with metadata for tracking progress:
+    - `index`: The starting index of the current chunk (e.g., 0, 100, 200...).
+    - `count`: The number of rows in the current chunk.
+    - `complete`: A boolean that is `true` only on the very last message of the stream, useful for triggering a final action.
 </script>

package/odbc.js

@@ -1,7 +1,7 @@
 module.exports = function (RED) {
     const odbcModule = require("odbc");
-    const mustache = require("mustache"); // Utilisé dans runQuery
-    const objPath = require("object-path"); // Utilisé pour mustache et le positionnement du résultat
+    const mustache = require("mustache");
+    const objPath = require("object-path");
 
     // --- ODBC Configuration Node ---
     function poolConfig(config) {
@@ -33,9 +33,6 @@ module.exports = function (RED) {
                 return parts.join(';');
             } else {
                 let connStr = this.config.connectionString || "";
-                if (this.credentials && this.credentials.password && connStr.includes('{{{password}}}')) {
-                    connStr = connStr.replace('{{{password}}}', this.credentials.password);
-                }
                 return connStr;
             }
         };
@@ -122,12 +119,12 @@ module.exports = function (RED) {
 
     RED.httpAdmin.post("/odbc_config/:id/test", RED.auth.needsPermission("odbc.write"), async function(req, res) {
         const tempConfig = req.body;
-        const tempCredentials = { password: tempConfig.password };
-        delete tempConfig.password;
 
         const buildTestConnectionString = () => {
-             if (tempConfig.connectionMode === 'structured') {
-                if (!tempConfig.dbType || !tempConfig.server) return res.status(400).send("Mode structuré : le type de BD et le serveur sont requis.");
+            if (tempConfig.connectionMode === 'structured') {
+                if (!tempConfig.dbType || !tempConfig.server) {
+                    throw new Error("En mode structuré, le type de base de données et le serveur sont requis.");
+                }
                 let driver;
                 let parts = [];
                 switch (tempConfig.dbType) {
@@ -140,12 +137,12 @@ module.exports = function (RED) {
                 parts.push(`SERVER=${tempConfig.server}`);
                 if (tempConfig.database) parts.push(`DATABASE=${tempConfig.database}`);
                 if (tempConfig.user) parts.push(`UID=${tempConfig.user}`);
-                if (tempCredentials.password) parts.push(`PWD=${tempCredentials.password}`);
+                if (tempConfig.password) parts.push(`PWD=${tempConfig.password}`);
                 return parts.join(';');
             } else {
                 let connStr = tempConfig.connectionString || "";
-                if (tempCredentials.password && connStr.includes('{{{password}}}')) {
-                    connStr = connStr.replace('{{{password}}}', tempCredentials.password);
+                if (!connStr) {
+                    throw new Error("La chaîne de connexion ne peut pas être vide.");
                 }
                 return connStr;
             }
@@ -154,13 +151,14 @@ module.exports = function (RED) {
         let connection;
         try {
             const testConnectionString = buildTestConnectionString();
-            if (!testConnectionString) return res.status(400).send("La chaîne de connexion est vide.");
             connection = await odbcModule.connect(testConnectionString);
             res.sendStatus(200);
         } catch (err) {
             res.status(500).send(err.message || "Erreur inconnue durant le test.");
         } finally {
-            if (connection) await connection.close();
+            if (connection) {
+                await connection.close();
+            }
         }
     });
 
@@ -304,8 +302,6 @@ module.exports = function (RED) {
         };
 
         this.runQuery = async (msg, send, done) => {
-             let currentQueryString = this.config.query || "";
-             let currentQueryParams = msg.parameters;
              let isPreparedStatement = false;
              let connectionFromPool = null;
 
@@ -313,6 +309,27 @@ module.exports = function (RED) {
                 this.status({ fill: "blue", shape: "dot", text: "preparing..." });
                 this.config.outputObj = msg?.output || this.config?.outputObj || "payload";
 
+                const querySourceType = this.config.querySourceType || 'msg';
+                const querySource = this.config.querySource || 'query';
+                const paramsSourceType = this.config.paramsSourceType || 'msg';
+                const paramsSource = this.config.paramsSource || 'parameters';
+
+                let currentQueryParams = await new Promise((resolve, reject) => {
+                    RED.util.evaluateNodeProperty(paramsSource, paramsSourceType, this, msg, (err, value) => {
+                        if (err) { resolve(undefined); }
+                        else { resolve(value); }
+                    });
+                });
+
+                let currentQueryString = await new Promise((resolve, reject) => {
+                     RED.util.evaluateNodeProperty(querySource, querySourceType, this, msg, (err, value) => {
+                        if (err) { resolve(undefined); }
+                        else { resolve(value || this.config.query || ""); }
+                    });
+                });
+                
+                if (!currentQueryString) { throw new Error("No query to execute"); }
+                
                 isPreparedStatement = currentQueryParams || (currentQueryString && currentQueryString.includes("?"));
                 if (!isPreparedStatement && currentQueryString) {
                     for (const parsed of mustache.parse(currentQueryString)) {
@@ -322,8 +339,6 @@ module.exports = function (RED) {
                     }
                     currentQueryString = mustache.render(currentQueryString, msg);
                 }
-                if (msg?.query) { currentQueryString = msg.query; }
-                if (!currentQueryString) { throw new Error("No query to execute"); }
 
                 const execute = async (conn) => {
                     if (this.config.streaming) {
@@ -335,7 +350,7 @@ module.exports = function (RED) {
                         if(done) done();
                     }
                 };
-
+                
                 let firstAttemptError = null;
                 try {
                     connectionFromPool = await this.poolNode.connect();
@@ -418,7 +433,7 @@ module.exports = function (RED) {
                 if (this.poolNode && this.poolNode.config.retryOnMsg) {
                     this.log("New message received, overriding retry timer and attempting query now.");
                     clearTimeout(this.retryTimer);
-this.retryTimer = null;
+                    this.retryTimer = null;
                     this.isAwaitingRetry = false;
                 } else {
                     this.warn("Node is in a retry-wait state. New message ignored as per configuration.");

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bkmj/node-red-contrib-odbcmj",
-  "version": "2.0.2",
+  "version": "2.1.0",
   "description": "A powerful Node-RED node to connect to any ODBC data source, with connection pooling, advanced retry logic, and result streaming.",
   "keywords": [
     "node-red",