@sfdxy/mule-lint 1.21.0 → 1.23.0

package/docs/best-practices/performance.md

@@ -0,0 +1,273 @@
+# Performance Best Practices
+
+> **Applies to:** All  
+> **Related Rules:** `MULE-501` · `MULE-502` · `MULE-503` · `PERF-002` · `RES-001` · `RES-002` · `MULE-801`  
+> **Last Updated:** April 2026
+
+## When to Read This
+
+Read this when optimizing flow performance, configuring connection pooling, implementing async patterns, or reviewing a project for production readiness.
+
+---
+
+## Key Principles
+
+1. **Set explicit timeouts** — avoid hanging connections and resource leaks
+2. **Handle async errors** — async scopes silently swallow errors
+3. **Limit choice complexity** — refactor large choice blocks to DWL lookups
+4. **Configure connection pools** — essential for production throughput
+5. **Use streaming for large payloads** — avoid loading everything into memory
+
+---
+
+## Patterns
+
+### Pattern 1: Async Scope Error Handling
+
+Async scopes don't propagate errors to the parent flow — they're silently lost:
+
+```xml
+<!-- ❌ Bad — errors silently swallowed -->
+<async>
+    <http:request config-ref="HTTP_Config" path="/webhook"/>
+</async>
+
+<!-- ✅ Good — explicit error handling inside async -->
+<async>
+    <try>
+        <http:request config-ref="HTTP_Config" path="/webhook"/>
+        <error-handler>
+            <on-error-continue>
+                <logger category="com.myorg.async" level="ERROR"
+                        message="#['Async webhook failed: ' ++ error.description]"/>
+            </on-error-continue>
+        </error-handler>
+    </try>
+</async>
+```
+
+### Pattern 2: Refactor Large Choice Blocks
+
+```xml
+<!-- ❌ Bad — too many when clauses (> 5) -->
+<choice>
+    <when expression="#[vars.status == 'NEW']">...</when>
+    <when expression="#[vars.status == 'PENDING']">...</when>
+    <when expression="#[vars.status == 'APPROVED']">...</when>
+    <!-- 10+ more conditions -->
+    <otherwise>...</otherwise>
+</choice>
+
+<!-- ✅ Good — DataWeave lookup + dynamic flow-ref -->
+<ee:transform>
+    <ee:set-variable variableName="handler"><![CDATA[%dw 2.0
+var handlers = {
+    "NEW": "new-handler-subflow",
+    "PENDING": "pending-handler-subflow",
+    "APPROVED": "approved-handler-subflow"
+}
+---
+handlers[vars.status] default "default-handler-subflow"
+]]></ee:set-variable>
+</ee:transform>
+<flow-ref name="#[vars.handler]"/>
+```
+
+### Pattern 3: HTTP Timeout Configuration
+
+```xml
+<!-- ✅ Always set explicit timeouts -->
+<http:request-config name="API_Config"
+    responseTimeout="${https.request.responseTimeout}">
+    <http:request-connection host="${api.host}" port="${api.port}">
+        <http:client-socket-properties>
+            <http:tcp-client-socket-properties connectionTimeout="30000"/>
+        </http:client-socket-properties>
+    </http:request-connection>
+</http:request-config>
+```
+
+> ⚠️ **Grizzly rule:** `connectionIdleTimeout` must be ≥ `responseTimeout`. If idle timeout fires first, it kills the connection before the response arrives.
+
+### Pattern 4: Connection Pooling
+
+```xml
+<!-- ✅ Good — HTTP with pooling configured -->
+<http:request-config name="API_Config">
+    <http:request-connection host="${api.host}" port="${api.port}"
+        maxConnections="20"
+        connectionIdleTimeout="300000"/>
+</http:request-config>
+
+<!-- ✅ Good — Database with pooling -->
+<db:config name="Database_Config">
+    <db:my-sql-connection host="${db.host}" port="${db.port}"
+        database="${db.name}" user="${db.user}"
+        password="${secure::db.password}">
+        <db:pooling-profile maxPoolSize="10"
+            minPoolSize="2" maxWait="30000"/>
+    </db:my-sql-connection>
+</db:config>
+```
+
+### Pattern 5: Scatter-Gather Limits
+
+Keep parallel routes manageable (< 5–7 routes):
+
+```xml
+<scatter-gather>
+    <route><flow-ref name="service1-subflow"/></route>
+    <route><flow-ref name="service2-subflow"/></route>
+    <route><flow-ref name="service3-subflow"/></route>
+</scatter-gather>
+```
+
+### Pattern 6: Streaming for Large Payloads
+
+Mule 4 uses **file-stored repeatable streams** by default (512 KB in-memory buffer). For large payloads:
+
+- **Don't** call `payload` multiple times — it forces stream re-read
+- **Do** store needed values in variables before consuming the stream
+- **Do** use streaming-compatible connectors and DataWeave
+
+```xml
+<!-- ✅ Store needed values before processing -->
+<set-variable variableName="recordCount" value="#[sizeOf(payload)]"/>
+<ee:transform>
+    <ee:set-payload resource="dwl/transforms/process-batch.dwl"/>
+</ee:transform>
+```
+
+### Pattern 7: Pre-batch Bulk Lookup (N+1 Prevention)
+
+**Use when:** iterating over 10+ records with `foreach` where each record requires 1+ related record lookups from an external system. Without this pattern, 100 records × 3 lookups = 300 API calls. With it: 3 API calls.
+
+**Do NOT use when:** iterating < 5 records (overhead not worth it), lookups are already cached in ObjectStore, or the downstream system doesn't support bulk/IN queries.
+
+| Scenario                                   | Bulk Lookup? | Why                      |
+| ------------------------------------------ | ------------ | ------------------------ |
+| 100 records, each needs 3 related lookups  | ✅           | 300 calls → 3 calls      |
+| 3 records, each needs 1 lookup             | ❌           | Overhead exceeds savings |
+| Lookups already cached in ObjectStore      | ❌           | Already optimized        |
+| Downstream doesn't support IN/bulk queries | ❌           | Can't batch anyway       |
+
+**Architecture:**
+
+```
+1. Save batch to variable
+2. Extract unique lookup IDs (distinctBy, filter !isEmpty)
+3. Scatter-gather: bulk-fetch all related records in parallel
+4. Build groupBy lookup maps → O(1) resolution per record
+5. Restore original batch as payload
+6. Foreach: resolve from maps (zero API calls)
+```
+
+**Implementation:**
+
+```xml
+<sub-flow name="bulkResolveLookupsSubFlow">
+    <!-- 1. Save the original batch -->
+    <set-variable value="#[payload]" variableName="batch"/>
+
+    <!-- 2. Extract unique lookup IDs -->
+    <ee:transform>
+        <ee:variables>
+            <ee:set-variable variableName="lookupIds"><![CDATA[%dw 2.0
+output application/java
+---
+{
+    projectIds: (vars.batch.projectId default [])
+                    filter (!isEmpty($)) distinctBy $,
+    contactIds: (vars.batch.contactId default [])
+                    filter (!isEmpty($)) distinctBy $
+}]]></ee:set-variable>
+        </ee:variables>
+    </ee:transform>
+
+    <!-- 3. Bulk fetch in parallel -->
+    <scatter-gather>
+        <route>
+            <http:request method="GET" path="/Project__c">
+                <http:query-params>#[output application/java --- {
+                    "query": "SELECT Id, Project_ID__c FROM Project__c
+                              WHERE Project_ID__c IN ('"
+                        ++ (vars.lookupIds.projectIds joinBy "','") ++ "')"
+                }]</http:query-params>
+            </http:request>
+        </route>
+        <route>
+            <http:request method="GET" path="/Contact">
+                <http:query-params>#[output application/java --- {
+                    "query": "SELECT Id, AccountId FROM Contact
+                              WHERE Id IN ('"
+                        ++ (vars.lookupIds.contactIds joinBy "','") ++ "')"
+                }]</http:query-params>
+            </http:request>
+        </route>
+    </scatter-gather>
+
+    <!-- 4. Build lookup maps with groupBy -->
+    <ee:transform>
+        <ee:variables>
+            <ee:set-variable variableName="lookupMaps"><![CDATA[%dw 2.0
+output application/java
+---
+{
+    projects: (payload.'0'.payload default []) groupBy $.Project_ID__c,
+    contacts: (payload.'1'.payload default []) groupBy $.Id
+}]]></ee:set-variable>
+        </ee:variables>
+    </ee:transform>
+
+    <!-- 5. Restore original batch (scatter-gather replaced payload) -->
+    <set-payload value="#[vars.batch]"/>
+</sub-flow>
+```
+
+**Then in the foreach — zero API calls per iteration:**
+
+```dataweave
+var resolvedProject = (vars.lookupMaps.projects[payload.projectId] default [])[0]
+---
+{
+    message: payload ++ { resolvedProject: resolvedProject },
+    recordType: "scheduled-entity"
+}
+```
+
+**Key rules:**
+
+- **Always save-and-restore payload** — `scatter-gather` replaces `payload` with its own result object
+- **`distinctBy $`** — deduplicate IDs (same project may appear on 50 records)
+- **`filter !isEmpty($)`** — exclude null/empty IDs before building the IN clause
+- **`groupBy` for O(1) resolution** — returns an array; take `[0]` for the first match
+- **Guard the scatter-gather** — if no IDs exist, skip the bulk fetch entirely (set empty maps)
+
+---
+
+## Flow Complexity Guidelines
+
+| Metric                | Threshold  | Action                  |
+| --------------------- | ---------- | ----------------------- |
+| Processors per flow   | ≤ 15       | Split into sub-flows    |
+| Choice branches       | ≤ 5        | Use DWL lookup pattern  |
+| Scatter-gather routes | ≤ 7        | Consolidate or sequence |
+| Flow-ref depth        | ≤ 5 levels | Flatten chain           |
+| Loggers per flow      | ≤ 5        | Move to DEBUG level     |
+
+---
+
+## Checklist
+
+- [ ] `responseTimeout` set on all HTTP request configs
+- [ ] `connectionIdleTimeout >= responseTimeout` (Grizzly rule)
+- [ ] Connection pooling configured for HTTP and Database connectors
+- [ ] Async scopes have internal error handling
+- [ ] Choice blocks have ≤ 5 branches (refactor large ones)
+- [ ] Scatter-gather has ≤ 7 routes
+- [ ] Reconnection strategies on all outbound connectors
+- [ ] Pre-batch bulk lookup used for foreach with 10+ records needing related lookups (Pattern 7)
+
+---
+
+**See also:** [Connector Patterns](connector-patterns.md) · [Error Handling](error-handling.md)

package/docs/best-practices/security.md

@@ -0,0 +1,181 @@
+# Security Best Practices
+
+> **Applies to:** All  
+> **Related Rules:** `MULE-004` · `MULE-201` · `MULE-202` · `SEC-002` – `SEC-010` · `YAML-004` · `LOG-004`  
+> **Last Updated:** April 2026
+
+## When to Read This
+
+Read this when handling credentials, configuring TLS, securing API endpoints, or reviewing a project for security compliance.
+
+---
+
+## Key Principles
+
+1. **Never hardcode secrets** — use `${secure::...}` property placeholders
+2. **Never hardcode URLs** — all hosts, ports, and paths go in YAML config
+3. **Encrypt sensitive properties** — AES/CBC with externalized key
+4. **Use TLS 1.2+ only** — TLS 1.0/1.1 and SSLv3 are deprecated
+5. **Shift-left security** — validate security in design phase, not after deployment
+
+---
+
+## Patterns
+
+### Pattern 1: Secure Properties Configuration
+
+```xml
+<!-- Load secure properties with externalized encryption key -->
+<secure-properties:config name="Secure_Properties_Config"
+    file="config/secure-${mule.env}.yaml"
+    key="${secure.key}">
+    <secure-properties:encrypt algorithm="AES" mode="CBC"/>
+</secure-properties:config>
+```
+
+```yaml
+# config/secure-dev.yaml (encrypted values)
+salesforce:
+  jwt:
+    consumerKey: '![encrypted-value]'
+    storePassword: '![encrypted-value]'
+```
+
+**Rules:**
+
+- `key` attribute MUST be a property placeholder (`${secure.key}`) — never hardcoded
+- Use `AES` or `Blowfish` — `DES` is considered weak
+- Encrypt using Anypoint Secure Properties Tool with runtime property `secure.key`
+
+### Pattern 2: Connector Credential Security
+
+```xml
+<!-- ❌ Bad — plaintext credentials -->
+<http:request-config username="admin" password="secret123"/>
+
+<!-- ❌ Bad — non-secure property -->
+<http:request-config username="${api.username}" password="${api.password}"/>
+
+<!-- ✅ Good — secure property reference -->
+<http:request-config
+    username="${api.username}"
+    password="${secure::api.password}"/>
+```
+
+Credential attributes that must use `${secure::...}`:
+
+- `password`, `storePassword`
+- `clientId`, `clientSecret`, `consumerKey`, `consumerSecret`
+- `token`, `tokenSecret`, `tokenId`
+
+### Pattern 3: TLS Configuration
+
+```xml
+<!-- ❌ Bad — insecure TLS -->
+<tls:context name="Insecure_TLS">
+    <tls:trust-store insecure="true"/>
+</tls:context>
+
+<!-- ❌ Bad — deprecated protocol -->
+<tls:context enabledProtocols="TLSv1.1,TLSv1.2">
+
+<!-- ✅ Good — proper certificate validation -->
+<tls:context name="Secure_TLS" enabledProtocols="TLSv1.2,TLSv1.3">
+    <tls:trust-store path="${tls.truststore.path}"
+                     password="${secure::tls.truststore.password}"/>
+</tls:context>
+```
+
+### Pattern 4: Never Log Sensitive Data
+
+```xml
+<!-- ❌ Bad — logs sensitive values -->
+<logger message="#['Token: ' ++ vars.accessToken]"/>
+<logger message="#[payload]"/>  <!-- May contain PII -->
+
+<!-- ✅ Good — logs only identifiers -->
+<logger message="#['Auth successful for user: ' ++ vars.userId]"/>
+<logger message="#['Processing order: ' ++ payload.orderId]"/>
+```
+
+**Detected patterns** (flagged by `SEC-006` and `LOG-004`):
+`encrypt.*key`, `password`, `credentials`, `api_key`, `secret.*key`, `accessToken`, `SSN`, `creditCard`
+
+### Pattern 5: API Rate Limiting
+
+For CloudHub-deployed APIs, rate limiting is applied via **API Manager policies** — not inline XML:
+
+```yaml
+# dev.yaml — populate for API Manager autodiscovery
+api:
+  id: '12345678' # API Manager API instance ID
+```
+
+For self-managed deployments, configure inline:
+
+```xml
+<throttling:config name="Rate_Limit_Config" maxRequestsPerPeriod="100" timePeriodInMilliseconds="60000"/>
+```
+
+### Pattern 6: Input Validation
+
+```xml
+<!-- ✅ Good — validate incoming payload against schema -->
+<flow name="post:\orders:api-config">
+    <json:validate-schema schema="schemas/order-request.json"/>
+    <!-- Process only after validation passes -->
+</flow>
+```
+
+For APIs using APIKit with RAML, schema validation is handled by the APIKit router based on the RAML type definitions.
+
+---
+
+## Zero-Trust API Architecture (2026+)
+
+Modern MuleSoft deployments follow Zero-Trust principles:
+
+| Layer             | Control                              | Implementation                           |
+| ----------------- | ------------------------------------ | ---------------------------------------- |
+| **Network**       | mTLS between services                | TLS context with client certificates     |
+| **Identity**      | OAuth 2.0 / JWT verified per request | API Manager policies + Flex Gateway      |
+| **Authorization** | Scoped permissions per API consumer  | Client ID enforcement + scope validation |
+| **Data**          | Encrypt at rest and in transit       | Secure properties + TLS 1.3              |
+| **Observability** | Audit all access                     | Correlation IDs + structured logging     |
+
+---
+
+## Environment Property Separation
+
+```
+config/
+├── global.yaml              # Shared defaults (timeouts, paths)
+├── dev.yaml                 # Dev-specific non-sensitive config
+├── prod.yaml                # Prod-specific non-sensitive config
+├── secure-dev.yaml          # Dev encrypted credentials
+└── secure-prod.yaml         # Prod encrypted credentials
+```
+
+**Never store:**
+
+- Plaintext passwords in any YAML file
+- API keys or tokens in non-secure YAML files
+- Encryption keys inline in XML
+
+---
+
+## Checklist
+
+- [ ] All credentials use `${secure::...}` property placeholders
+- [ ] Secure properties encryption key is externalized (`${secure.key}`)
+- [ ] Encryption algorithm is AES or Blowfish (not DES)
+- [ ] TLS contexts use TLSv1.2+ only — no SSLv3, TLSv1.0, TLSv1.1
+- [ ] No `insecure="true"` on trust-stores
+- [ ] No hardcoded URLs, hosts, or ports in XML
+- [ ] No sensitive data in logger messages
+- [ ] Input validation configured for all POST/PUT/PATCH endpoints
+- [ ] API Manager `api.id` populated in environment YAML
+
+---
+
+**See also:** [Connector Patterns](connector-patterns.md) · [Logging](logging.md) · [Configuration Management](variable-contracts.md)

package/docs/best-practices/testing.md

@@ -0,0 +1,190 @@
+# Testing with MUnit
+
+> **Applies to:** All  
+> **Related Rules:** `EXP-003`  
+> **Last Updated:** April 2026
+
+## When to Read This
+
+Read this when writing MUnit tests, setting up test infrastructure, or testing event-driven flows.
+
+---
+
+## Test Coverage Goals
+
+| Test Type            | Coverage Target    | Purpose                        |
+| -------------------- | ------------------ | ------------------------------ |
+| Unit Tests           | 80%+ flow coverage | Validate individual flow logic |
+| Integration Tests    | All critical paths | Validate end-to-end scenarios  |
+| Error Scenario Tests | All error handlers | Validate error responses       |
+
+---
+
+## Patterns
+
+### Pattern 1: Standard MUnit Test Structure
+
+```xml
+<munit:test name="create-order-success-test"
+            description="Validates successful order creation">
+
+    <!-- Mock external dependencies -->
+    <munit:behavior>
+        <munit-tools:mock-when processor="http:request">
+            <munit-tools:with-attributes>
+                <munit-tools:with-attribute attributeName="config-ref"
+                                            whereValue="API_HTTP_Config"/>
+            </munit-tools:with-attributes>
+            <munit-tools:then-return>
+                <munit-tools:payload value='{"orderId": "12345"}'/>
+            </munit-tools:then-return>
+        </munit-tools:mock-when>
+    </munit:behavior>
+
+    <!-- Execute the flow -->
+    <munit:execution>
+        <flow-ref name="create-order-flow"/>
+    </munit:execution>
+
+    <!-- Assert results -->
+    <munit:validation>
+        <munit-tools:assert-that expression="#[payload.orderId]"
+                                  is="#[MunitTools::notNullValue()]"/>
+    </munit:validation>
+</munit:test>
+```
+
+### Pattern 2: Testing Error Handlers
+
+```xml
+<munit:test name="error-handler-400-test"
+            description="Validates 400 Bad Request error handling"
+            expectedErrorType="APIKIT:BAD_REQUEST">
+
+    <munit:behavior>
+        <munit-tools:mock-when processor="apikit:router">
+            <munit-tools:then-call exception="APIKIT:BAD_REQUEST"/>
+        </munit-tools:mock-when>
+    </munit:behavior>
+
+    <munit:execution>
+        <flow-ref name="api-main"/>
+    </munit:execution>
+
+    <munit:validation>
+        <munit-tools:assert-that expression="#[vars.httpStatus]"
+                                  is="#[MunitTools::equalTo(400)]"/>
+        <munit-tools:assert-that expression="#[payload.error]"
+                                  is="#[MunitTools::equalTo('InvalidInput')]"/>
+    </munit:validation>
+</munit:test>
+```
+
+### Pattern 3: Testing Event-Driven Flows
+
+For PAPI-style flows driven by Platform Events, mock the event payload and downstream SAPI calls:
+
+```xml
+<munit:test name="account-event-processing-test"
+            description="Validates Account event → NS Customer sync">
+
+    <munit:behavior>
+        <!-- Mock the SAPI HTTP call -->
+        <munit-tools:mock-when processor="http:request">
+            <munit-tools:with-attributes>
+                <munit-tools:with-attribute attributeName="config-ref"
+                    whereValue="SAPI_HTTP_Config"/>
+            </munit-tools:with-attributes>
+            <munit-tools:then-return>
+                <munit-tools:payload value='{"status":"success","internalId":"123"}'/>
+            </munit-tools:then-return>
+        </munit-tools:mock-when>
+
+        <!-- Mock the writeback SAPI call -->
+        <munit-tools:mock-when processor="http:request">
+            <munit-tools:with-attributes>
+                <munit-tools:with-attribute attributeName="doc:name"
+                    whereValue="Writeback Request"/>
+            </munit-tools:with-attributes>
+            <munit-tools:then-return>
+                <munit-tools:payload value='{"success":true}'/>
+            </munit-tools:then-return>
+        </munit-tools:mock-when>
+    </munit:behavior>
+
+    <munit:execution>
+        <!-- Set variables as the event listener would -->
+        <set-variable variableName="correlationId" value="test-uuid-123"/>
+        <set-variable variableName="logCategory" value="com.myorg.papi"/>
+        <set-variable variableName="salesforceId" value="001XXXXX"/>
+        <set-payload value='#[readUrl("classpath://test-data/account-event.json", "application/json")]'/>
+        <flow-ref name="account-process-subflow"/>
+    </munit:execution>
+
+    <munit:validation>
+        <munit-tools:assert-that expression="#[payload.status]"
+                                  is="#[MunitTools::equalTo('success')]"/>
+    </munit:validation>
+</munit:test>
+```
+
+### Pattern 4: Test Organization
+
+```
+src/test/
+├── munit/
+│   ├── salesforce-upsert-test.xml       # Operation-specific tests
+│   ├── salesforce-create-test.xml
+│   ├── salesforce-query-test.xml
+│   ├── salesforce-delete-test.xml
+│   ├── salesforce-process-subflow-test.xml  # Routing tests
+│   ├── error-handling-test.xml          # Error scenario tests
+│   └── common-test-resources.xml        # Shared mocks
+└── resources/
+    ├── log4j2-test.xml                  # Console-only, noise-suppressed
+    └── test-data/                       # Test payloads
+        ├── account-event.json
+        └── order-request.json
+```
+
+---
+
+## Key Principles
+
+1. **Mock all external dependencies** — never call real systems in unit tests
+2. **Test all error handler branches** — verify each HTTP status code / error type
+3. **Use descriptive test names** — names should describe the scenario being tested
+4. **Isolate tests** — each test should be independent (no shared state)
+5. **Separate test log config** — use `log4j2-test.xml` with console appender and suppressed noise
+
+---
+
+## Running Tests
+
+```bash
+# Full test suite
+mvn clean test -Dmule.env=dev -Dsecure.key=test
+
+# Specific test suite
+mvn test -Dmule.env=dev -Dsecure.key=test -Dtest=salesforce-upsert-test
+
+# Package (skip tests)
+mvn clean package -DskipTests
+```
+
+> **Note:** MUnit requires MuleSoft Enterprise Edition license. If the `licm` check fails in your dev environment, verify with `mvn process-classes` (schema validation passes without EE license).
+
+---
+
+## Checklist
+
+- [ ] 80%+ flow coverage with MUnit
+- [ ] All error handler branches tested
+- [ ] External dependencies mocked (HTTP, connectors, databases)
+- [ ] Test names clearly describe the scenario
+- [ ] `log4j2-test.xml` configured for test environment
+- [ ] Test data externalized to `test-data/` directory
+
+---
+
+**See also:** [Error Handling](error-handling.md) · [CI/CD Integration](ci-cd.md)