orbit-code-ai 0.1.22 → 0.1.24

package/_agent_reference/components/audit-logging.md

@@ -21,12 +21,18 @@ The framework provides configuration-driven audit logging with field masking. Ev
 **Purpose**: Reads config from `Environment.Properties`, builds the audit XML structure, and routes to the audit queue.
 
 **Key behavior**:
-- Reads `AuditEnabled` and `LogEnabled` flags — does nothing if both are FALSE
-- Extracts field values using XPath expressions defined in `value1`–`value10` properties
-- Masks fields listed in `maskingIn` / `maskingOut` using EVAL (dynamic path resolution)
-- Adds flow metadata: broker name, execution group, flow name, timestamps
-
-**Masking**: Fields listed in `maskingIn`/`maskingOut` have their values replaced with `****` before logging. Example masked field: `*.Header.ChannelUserInfo.UserPassword`
+- Reads `AuditEnabled` / `LogEnabled` (the gate is the `Is*AuditEnabled` filter; AuditPoint builds the record)
+- Tags `auditHeader.flowStart` from the node EXTERNAL `FLOW_START BOOLEAN` — `LogAudit` sets it TRUE
+  (incoming), `LogOutgoingAudit` sets it FALSE (outgoing); incoming vs outgoing masking is chosen from this
+- Sets `auditHeader.retryCount` from `InputRoot.MQRFH2.usr.retry.count` (0 when absent)
+- Extracts field values using the paths in `value1`–`value10` properties (labelled by `key1`–`key10`)
+- Masks the paths listed in `maskingIn` (incoming) / `maskingOut` (outgoing)
+- Adds flow metadata: `applicationName`, `messageFlowName`, `brokerName`, `executionGroupName`, timestamps
+
+**AuditPoint node external**: `DECLARE FLOW_START EXTERNAL BOOLEAN FALSE;`
+
+**Masking**: fields listed in `maskingIn`/`maskingOut` are masked before logging. Example masked field:
+`*.Header.ChannelUserInfo.UserPassword`.
 
 ---
 

package/_agent_reference/components/common-utils.md

@@ -1,146 +1,315 @@
 # Common Utilities (CommonUtils.esql)
 
-`CommonUtils.esql` is a 47KB shared ESQL library containing constants, queue routing, correlation ID generation, database access, and utility functions. Include it in every flow that needs routing or utility functions.
+`CommonUtils.esql` is a ~48 KB (~1,200 line) shared ESQL library holding ESB constants, queue routing, channel lookups, correlation-ID generation, error-code/description mapping, response-header builders, exception parsing, and string/date helpers.
 
 **Location**: `Qiwa/Framework/Lib/CommonUtils.esql`
+**Schema**: `BROKER SCHEMA Qiwa.Framework.Lib` (declared with `PATH Qiwa.Framework.Lib;`)
 
----
-
-## Error Code Constants
+To use any routine below from an application module, the app must reference the shared library **and** declare the path:
 
 ```esql
--- Always use these constants, never hardcode error codes
-DECLARE ESB_SUCCESS_RESPONSE_CODE               CONSTANT CHAR '00000000';
-DECLARE ESB_RUNTIME_TECHNICAL_ERROR_CODE        CONSTANT CHAR '99999999';
-DECLARE ESB_MIDDLEWARE_DATABASE_ERROR_RESPONSE_CODE CONSTANT CHAR 'E0199998';
+BROKER SCHEMA com.qiwa.esb.<package>
+PATH Qiwa.Framework.Lib;     -- required to resolve these routines
 ```
 
-Backend-specific error code ranges are also defined — read `ErrorHandler.esql` for the full list per backend system.
+> **Calling convention matters.** Most routines here are `CREATE FUNCTION` and return a value — call them **inline** (`SET x = Fn(...)`), not with `CALL`. Only the `CREATE PROCEDURE` routines (marked below) are invoked with `CALL`. The signatures below are the real ones from source — do not guess parameters.
 
 ---
 
-## Key Functions
+## Constants
+
+All ESB codes are declared `EXTERNAL CHARACTER` (user-defined properties — overridable per-deploy via the BAR/`.properties`, **not** `CONSTANT`). Read them by name; never hardcode the literals.
+
+| Constant | Value | Meaning |
+|---|---|---|
+| `ESB_SUCCESS_RESPONSE_CODE` | `'00000000'` | Success |
+| `ESB_RUNTIME_TECHNICAL_ERROR_CODE` | `'99999999'` | Generic runtime/technical error |
+| `ESB_MIDDLEWARE_RUNTIME_ERROR_RESPONSE_CODE` | `'99999999'` | Middleware runtime error |
+| `ESB_MIDDLEWARE_DATABASE_ERROR_RESPONSE_CODE` | `'E0199998'` | Middleware DB error |
+| `ESB_MIDDLEWARE_IN_ACTIVE_CHANNEL_RESPONSE_CODE` | `'E9009999'` | Channel inactive |
+| `ESB_MIDDLEWARE_NOT_EXIST_CHANNEL_RESPONSE_CODE` | `'55555555'` | Channel not found |
+| `ESB_MIDDLEWARE_UN_AUTHORIZED_OPERATION_RESPONSE_CODE` | `'44444444'` | Channel not authorized for the service |
+| `ESB_EMPTY_HEADER_RESPONSE_CODE` | `'E5500003'` | Empty/missing header |
+| `ESB_WS_ADAPTER_GATEWAY_RS_FLOW_QUEUE` | `'ESB.WS.OUT'` | WS adapter response-flow queue |
+| `Billing_TIMEOUT` | `'1000'` | Billing timeout (ms) |
+| `Billing_NOREPLYCODE` | `'NOREPLY'` | Billing no-reply marker |
+| `DATABASE_SCHEMA_NAME` | `'dbo'` | DB schema prefix used in SQL |
+| `MCR_DSN` | `'MWMCR_DS'` | ODBC DSN for all `PASSTHRU` calls |
+| `AUDIT_SCHEMA_NAME` | `'MCR_AUDIT'` | Audit schema |
+| `CR` | `X'0A'` as char | Newline char |
+
+Plain (non-EXTERNAL) DB error constants:
+
+| Constant | Value | Meaning |
+|---|---|---|
+| `MCRDBTIMEOUTORDEADLOCKERROR` | `'E0500000'` | SQLCODE −913/−911 (timeout/deadlock) |
+| `MCRDBGENERALERROR` | `'E0599999'` | Any other SQL error |
 
-### Queue Routing
+Prefer the getter functions over reading constants directly: `GetESBSuccessCode()`, `GetESBUnAuthorizedOperationCode()`, `GetESBInactiveChannelCode()`, `GetESBNotExistChannelCode()`, `GetWSAdapterRsFlowQueue()`, `GetESBCR_NEW_LINE()` — each `RETURNS CHAR` of the corresponding constant.
+
+---
+
+## Queue Routing & Channel Lookups
 
 ```esql
--- Set request and response queues based on ServiceCode from message header
--- Reads from MCR_CHANNESBFUNC_TBL database table
-CALL SetQueues(IN serviceCode CHARACTER, OUT requestQueue CHARACTER, OUT responseQueue CHARACTER);
+-- Resolve request/response queues for the inbound message and wire them onto the
+-- output environment + MQMD. Reads cache CacheQueueTable (loaded from
+-- dbo.MCR_CHANNAUTHQUEUES_VIEW). Returns '' on success, or an ESB error code
+-- (NOT_EXIST_CHANNEL / UN_AUTHORIZED_OPERATION) on failure.
+CREATE FUNCTION SetQueues(IN rEsbXMLHeader REFERENCE,
+                         INOUT rOutLocalEnv REFERENCE,
+                         INOUT rOutRoot REFERENCE) RETURNS CHAR;
+-- On success it sets:
+--   rOutLocalEnv.Destination.MQ.DestinationData[1].queueName  (request queue)
+--   rOutRoot.MQMD.ReplyToQ                                     (response queue)
+--   rOutLocalEnv.MQ.GET.QueueName                              (response queue)
+
+-- Output queue for a channel (from dbo.MCR_CHANN_TBL.OUTQUEUE), cached.
+-- Defaults to 'ESBOUT' when not found.
+CREATE FUNCTION getChannelOutQueue(IN ChannelId CHARACTER) RETURNS CHARACTER;
+
+-- Numeric CHANNID for a channel name (from dbo.MCR_CHANN_TBL), cached.
+CREATE FUNCTION getEsbChannelId(IN channelName CHARACTER) RETURNS INTEGER;
+
+-- Same as above but reports DB failures: sets errorcode and returns 0 on error.
+CREATE FUNCTION getEsbChannelIdWithError(IN channelName CHARACTER,
+                                        INOUT errorcode CHARACTER) RETURNS INTEGER;
+
+-- Channel STATUS column (active/inactive flag) from dbo.MCR_CHANN_TBL.
+CREATE FUNCTION getEsbChannelStatus(IN channelName CHARACTER) RETURNS INTEGER;
+
+-- Non-cached CHANNID lookups (INTEGER and CHAR variants).
+CREATE FUNCTION getChannelIDByName(IN channelName CHARACTER) RETURNS INTEGER;
+CREATE FUNCTION getChannelIDFromChannelCode(IN channel CHARACTER) RETURNS CHARACTER;
+
+-- Invalidate all SHARED routing/error caches (force reload on next call).
+CREATE FUNCTION refreshESBRoutingCache();
+```
+
+---
+
+## Correlation ID
 
--- Get output queue for a specific channel
-CALL getChannelOutQueue(IN channelId CHARACTER, IN serviceCode CHARACTER, OUT outQueue CHARACTER);
+```esql
+-- Build a 48-hex-digit correlation-id BLOB (channel prefix + transaction id +
+-- optional offset). offset is the 3-char suffix used by the internal service
+-- caller when one session calls several services; pass '' otherwise.
+CREATE FUNCTION getChannelCorrelationID(IN ChannelId CHAR,
+                                       IN TransactionId CHAR,
+                                       IN offset CHAR) RETURNS BLOB;
 ```
 
-### Correlation ID
+---
+
+## Database Access
 
 ```esql
--- Generate a unique correlation ID for tracking this message across systems
-CALL getChannelCorrelationID(IN channelId CHARACTER, IN sessionId CHARACTER, OUT correlationId CHARACTER);
+-- PASSTHRU a SELECT to Database.{MCR_DSN}; sets retRow to the result row.
+-- Returns GetESBSuccessCode() on success, MCRDBTIMEOUTORDEADLOCKERROR on
+-- SQLCODE -913/-911, MCRDBGENERALERROR otherwise. Also sets SHARED SQLErrorDesc.
+CREATE FUNCTION MCRDB_Selection(IN sqlString CHAR, INOUT retRow REFERENCE) RETURNS CHAR;
 
--- Get correlation ID from the ESB channel
-CALL getEsbChannelId(IN channelId CHARACTER, OUT esbChannelId CHARACTER);
+-- Array variant: result rows land under retRow.val[].
+CREATE FUNCTION MCRDB_Selection_Array(IN sqlString CHAR, INOUT retRow REFERENCE) RETURNS CHAR;
 ```
 
-### Timestamp
+> All DB access in this library uses `PASSTHRU(... TO Database.{MCR_DSN})` against schema `dbo`. The DSN `MWMCR_DS` must be configured on the integration server.
+
+---
+
+## Error-Code Descriptions
+
+All three read the `dbo.ESB_ERROR_CODES` table (cached daily into SHARED rows):
 
 ```esql
--- Get formatted timestamp string (ISO 8601)
-CALL GetTimestampFormat(OUT timestamp CHARACTER);
--- Returns: '2024-01-15T14:30:00.000Z'
+-- Look up English/Arabic description + status by ESB error code.
+CREATE FUNCTION getEsbErrorCodeDescriptionFrom(IN ErrorCode CHARACTER,
+    OUT EnglishDescription CHAR, OUT ArabicDescription CHAR, OUT STATUS CHAR);
+
+-- Channel-aware variant (takes ChannelID + ServiceCode).
+CREATE FUNCTION getEsbChannlsErrorCodeDescriptionFrom(IN ErrorCode CHARACTER,
+    IN ChannelID CHAR, IN ServiceCode CHARACTER,
+    OUT EnglishDescription CHAR, OUT ArabicDescription CHAR, OUT STATUS CHAR);
+
+-- Map a backend (BE_NAME/BE_CODE) error to the ESB code + descriptions.
+CREATE FUNCTION getBackendEsb_MapErrorCode(IN BackendName CHARACTER,
+    IN BackendErrorCode CHARACTER, IN ChannelId CHARACTER,
+    OUT EnglishDescription CHAR, OUT ArabicDescription CHAR,
+    OUT STATUS CHAR, OUT ESB_CODE CHAR);
 ```
 
-### Database Access
+---
+
+## Response Header Builders (PROCEDUREs — use `CALL`)
 
 ```esql
--- Execute a SQL query with error handling
--- DSN is configured at the integration node level
-CALL MCRDB_Selection(IN sqlQuery CHARACTER, OUT resultSet ROW);
+-- Copy request-header fields to the response header and fill ResponseStatus
+-- (Status/Code/ArabicMsg/EnglishMsg) from the ESB error code.
+CALL Create_esbXML_Response_Header(esbResponseCode, RequestHeaderRef, ResponseHeaderRef);
+
+-- Overload that derives ResponseStatus from a backend error code.
+CALL Create_esbXML_Response_Header_Using_Backend_Code(BackEndName, BackEndCode,
+                                                      RequestHeaderRef, ResponseHeaderRef);
+
+-- Build the whole response skeleton: creates XMLNSC.<serviceResponseName>.Header
+-- under outputRootRef from inputRootRef and fills it.
+CALL CreateEsbXMLResponseHeaderFromRoot(esbResponseCode, inputRootRef, outputRootRef,
+                                        serviceResponseName);
+
+-- Backend-code overload of the above.
+CALL CreateEsbXMLResponseHeaderFromRootUsingBackendCode(BackEndName, BackEndCode,
+                                        inputRootRef, outputRootRef, serviceResponseName);
 ```
 
-### Flow Name Utility
+---
+
+## Timestamp / Format Helpers
+
+These return **format pattern strings** to use with `CAST(... AS CHAR FORMAT ...)` — they do **not** return a formatted value.
 
 ```esql
--- Strip schema prefix from flow name for use in property lookups
-CALL getFlowNameWithouSchema(IN fullFlowName CHARACTER, OUT flowName CHARACTER);
--- Input:  'Qiwa.BS_Account.GetAccountDetails'
--- Output: 'GetAccountDetails'
+CREATE FUNCTION GetTimestampFormat()     RETURNS CHAR;     -- 'yyyy-MM-dd HH:mm:ss.SSS'
+CREATE FUNCTION GetOMNITimestampFormat() RETURNS CHAR;     -- 'yyyy-MM-dd HH:mm:ss.SSSSS'
+CREATE FUNCTION GetPCKGTimestampFormat() RETURNS CHAR;     -- 'yyyyMMddHHmmss'
+CREATE FUNCTION GetDateFormat()          RETURNS CHAR;     -- 'yyyy-MM-dd'
+CREATE FUNCTION GetDefaultCharacterSet() RETURNS INTEGER;  -- 1208 (UTF-8 CCSID)
+
+-- e.g. format "now" as an ESB timestamp:
+SET ResponseHeader.MWResponseTime = CAST(CURRENT_TIMESTAMP AS CHAR FORMAT GetTimestampFormat());
+
+-- Reformat a date string from one pattern to another (NULL-safe).
+CREATE FUNCTION FormatDate(IN inputDate CHARACTER, IN inFormat CHARACTER,
+                           IN outFormat CHARACTER) RETURNS CHARACTER;
 ```
 
-### Response Codes
+---
+
+## String / Numeric Helpers
 
 ```esql
--- Get the standard success code
-CALL GetESBSuccessCode(OUT successCode CHARACTER);
--- Returns: '00000000'
+-- Insert a decimal point `point` digits from the right and cast to DECIMAL.
+CREATE FUNCTION formatDecimal(IN amount CHARACTER, IN point CHARACTER) RETURNS CHARACTER;
+
+-- Left-pad `id` with '0' to width `len`.
+CREATE FUNCTION padding(IN id CHARACTER, IN len INTEGER) RETURNS CHARACTER;
+
+-- Repeat a single char `len` times.
+CREATE FUNCTION charseq(IN aChar CHAR, IN len INTEGER) RETURNS CHARACTER;
+
+-- TRUE if the value casts to an integer.
+CREATE FUNCTION IsNumber(IN AmtVal CHARACTER) RETURNS BOOLEAN;
+
+-- nth delimited subfield (1-based), e.g. SelectSubField('a b c',' ',2) -> 'b'.
+CREATE FUNCTION SelectSubField(SourceString CHAR, Delimiter CHAR,
+                               TargetStringPosition INT) RETURNS CHAR;
+
+-- Request name -> response name: 'XxRq' -> 'XxRs', else 'Xx' -> 'XxResponse'.
+CREATE FUNCTION getServiceResponseName(IN requestName CHAR) RETURNS CHAR;
+
+-- Split S on Delim into Env.Split.Array[1..n] (PROCEDURE — use CALL).
+CALL Split(S, EnvRef, Delim);
 
--- Get default character set
-CALL GetDefaultCharacterSet(OUT charset CHARACTER);
--- Returns: 'UTF-8'
+-- Last segment of the CURRENT flow's broker MessageFlowLabel, split on '.'.
+-- Note: it reads MessageFlowLabel internally and writes Split results under
+-- EnvRef; it does NOT take a flow-name string. Pass an Environment reference.
+CREATE FUNCTION getFlowNameWithouSchema(IN EnvRef REFERENCE) RETURNS CHAR;
 ```
 
 ---
 
-## Shared Row (In-Memory Cache)
+## Exception Parsing
 
 ```esql
-SHARED ROW CacheQueueTable;
+-- Move CurrentError to the most relevant (deepest) exception in the list
+-- (PROCEDURE — use CALL).
+CALL GetLastException(CurrentErrorRef);
+
+-- Walk ExceptionList and populate ErrorInfo with FLOW_NAME, NODE_NAME,
+-- ERROR_TYPE, ERROR_TIME, ERROR_SEVERITY, ERROR_NUMBER, ERROR_DETAILS,
+-- MSG_ID, MSG_DATA, CompleteErrorMsg (PROCEDURE — use CALL).
+CALL PARSEExceptionList(InputExceptionList, ErrorInfoRef);
 ```
 
-`CacheQueueTable` is used by `SetQueues()` to cache queue routing lookups. It persists across message invocations within the same execution group — do not clear it manually.
+`Generic_Prepare_Fault` is a `CREATE COMPUTE MODULE` that builds a SOAP fault response from the exception list (uses `PARSEExceptionList`). Reference it as a node's compute module, not as a function.
 
 ---
 
-## Database Tables Referenced
+## SHARED State (In-Memory Caches)
 
-| Table | Columns | Purpose |
+Declared with `DECLARE <name> SHARED ROW;` (note the order — `DECLARE` first). These persist across invocations within an execution group; do not clear them manually (use `refreshESBRoutingCache()`):
+
+`CacheQueueTable`, `CacheChannelOutQueues`/`...Time`, `CacheChannelIds`/`...Time`,
+`CacheBackendMappedCodes`/`...Time` + `ErrorBERecord`,
+`CacheESBCodesDescriptions`/`...Time` + `CashErrorRecord`,
+`CacheESBChannelsCodesDescriptions`/`...Time` + `ErrorRecord`,
+plus `SQLErrorDesc` (last SQL error text) and `DefaultCharacterSet` (1208).
+
+---
+
+## Database Objects Referenced
+
+| Object | Columns used | Used by |
 |---|---|---|
-| `MCR_CHANNESBFUNC_TBL` | `ESBFUNCID`, `ESBFUNCNAME`, `REQUEST_QUEUE_NAME`, `RESPONSE_QUEUE_NAME` | Queue routing by service code |
-| `MCR_CHANNAUTH_TBL` | `CHANNID`, `ESBFUNCID` | Channel authorization check |
+| `dbo.MCR_CHANNAUTHQUEUES_VIEW` | `ESBFUNCID`, `CHANNID`, `REQUEST_QUEUE_NAME`, `RESPONSE_QUEUE_NAME` | `SetQueues` |
+| `dbo.MCR_CHANN_TBL` | `CHANNID`, `CHANNNAME`, `OUTQUEUE`, `STATUS` | `getChannelOutQueue`, `getEsbChannelId`, `getEsbChannelStatus`, `getChannelIDByName`, `getChannelIDFromChannelCode` |
+| `dbo.ESB_ERROR_CODES` | `STATUS`, `CODE`, `CHANNID`, `AR_DESCRIPTION`, `EN_DESCRIPTION`, `BE_NAME`, `BE_CODE` | the three error-description functions |
+
+All reached through `Database.{MCR_DSN}` (`MWMCR_DS`).
 
 ---
 
-## Usage Pattern in ESQL
+## ESB Header Field Paths
 
-```esql
--- At top of your ESQL module, no explicit import needed —
--- CommonUtils is part of the shared library automatically
-
--- Example: get queues for this service
-DECLARE reqQueue CHARACTER;
-DECLARE resQueue CHARACTER;
-DECLARE serviceCode CHARACTER;
-SET serviceCode = InputRoot.XMLNSC.Root.Header.ServiceCode;
-CALL SetQueues(serviceCode, reqQueue, resQueue);
-
--- Set output destination
-SET Environment.Destination.MQ.DestinationData.queueName = reqQueue;
-
--- Example: generate correlation ID
-DECLARE corrId CHARACTER;
-CALL getChannelCorrelationID(
-    InputRoot.XMLNSC.Root.Header.ChannelId,
-    InputRoot.XMLNSC.Root.Header.SessionId,
-    corrId
-);
-SET OutputRoot.MQMD.CorrelId = CAST(corrId AS BLOB CCSID 1208);
+The ESB message wraps a service element whose name varies per service; the header lives **under that service element**, not a literal `Root`. Reading code uses `InputRoot.XMLNSC.*[<].Header.*` (first child = the service element); the response builders create `OutputRoot.XMLNSC.<serviceResponseName>.Header.*`.
+
+Header fields populated/copied by the builders:
+
+```
+.Header.TransactionId
+.Header.ChannelId
+.Header.SessionId
+.Header.ServiceCode
+.Header.RequestTime
+.Header.MWRequestTime
+.Header.MWResponseTime
+.Header.DebugFlag
+.Header.ChannelUserInfo.UserName
+.Header.UDF
+.Header.ResponseStatus.Status
+.Header.ResponseStatus.Code
+.Header.ResponseStatus.EnglishMsg
+.Header.ResponseStatus.ArabicMsg
 ```
 
 ---
 
-## Standard Header Field Paths
+## Usage Example
 
 ```esql
--- Standard paths for common header fields
-InputRoot.XMLNSC.Root.Header.TransactionId
-InputRoot.XMLNSC.Root.Header.ChannelId
-InputRoot.XMLNSC.Root.Header.ServiceCode
-InputRoot.XMLNSC.Root.Header.SessionId
-InputRoot.XMLNSC.Root.Header.RequestTime
-InputRoot.XMLNSC.Root.Header.ChannelUserInfo.UserName
-InputRoot.XMLNSC.Root.Header.ChannelUserInfo.UserPassword  -- ALWAYS MASK
-InputRoot.XMLNSC.Root.Header.ResponseStatus.Code
-InputRoot.XMLNSC.Root.Header.ResponseStatus.Status
-InputRoot.XMLNSC.Root.Header.ResponseStatus.EnglishMsg
-InputRoot.XMLNSC.Root.Body                                 -- service-specific data
+BROKER SCHEMA com.qiwa.esb.myservice
+PATH Qiwa.Framework.Lib;
+
+CREATE COMPUTE MODULE MyFlow_Compute
+  CREATE FUNCTION Main() RETURNS BOOLEAN
+  BEGIN
+    SET OutputRoot = InputRoot;
+
+    -- Header reference (service element is the first child under XMLNSC)
+    DECLARE inHdr REFERENCE TO InputRoot.XMLNSC.*[<].Header;
+
+    -- Route to the correct request/response queues
+    DECLARE rc CHAR;
+    SET rc = SetQueues(inHdr, OutputLocalEnvironment, OutputRoot);
+    IF rc <> GetESBSuccessCode() AND rc <> '' THEN
+      -- rc holds an ESB error code (e.g. not-exist / unauthorized channel)
+      RETURN TRUE;
+    END IF;
+
+    -- Generate a correlation id (no offset)
+    SET OutputRoot.MQMD.CorrelId =
+      getChannelCorrelationID(FIELDVALUE(inHdr.ChannelId), FIELDVALUE(inHdr.TransactionId), '');
+
+    RETURN TRUE;
+  END;
+END MODULE;
 ```