nui-python-shared-utils 2.3__tar.gz → 2.4__tar.gz

nui_python_shared_utils-2.4/CHANGELOG.md

@@ -0,0 +1,251 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and breaking changes are called out explicitly below, including historical minor releases
+that contained them.
+
+## [Unreleased]
+
+### Added
+
+- Add an opt-in `reraise_exceptions` parameter to `powertools_handler`. When enabled,
+  handler failures are logged and alerted before the original exception is re-raised so
+  stream and asynchronous event sources can apply their configured failure handling.
+  The default remains the existing 500 response, while SQS events continue to re-raise
+  unconditionally.
+
+## [2.3] - 2026-10-12
+
+### Fixed
+
+- `powertools_handler` re-raises exceptions for SQS event-source mappings after logging and
+  alerting, so failed batches are redelivered and can reach a configured DLQ instead of
+  being acknowledged by the wrapper's 500 response.
+
+## [2.2] - 2026-09-14
+
+### Added
+
+- Add response-returning Slack write methods (`post_message`, `post_thread_reply`,
+  `post_update`, and `post_file`) for callers that need message timestamps, channel IDs,
+  or the raw upload response. The existing bool-returning `send_*` methods retain their
+  error-swallowing behavior.
+- Add `SentMessage` as the result type for posted messages and thread replies.
+
+### Changed
+
+- `send_file` and `post_file` accept bytes and can upload into a thread with `thread_ts`.
+
+## [2.1] - 2026-08-03
+
+### Changed
+
+- **BREAKING**: replace `pytz` with the standard-library `zoneinfo` API and remove `pytz`
+  from the base dependencies. Code importing timezone constants must replace `.localize()`
+  with `tzinfo=...`, passing `fold=1` for repeated wall times where the old `is_dst=False`
+  default mattered, and replace `.zone` with `.key`. Declare `pytz` directly if application
+  code still imports it. The `[timezone]` extra supplies `tzdata` for minimal images without
+  a system IANA database.
+- Allow Elasticsearch client majors 7.17, 8.x, and 9.x through the `[elasticsearch]` and
+  `all` extras. See
+  [ADR-007](docs/decisions/ADR-007-elasticsearch-client-version-range.md).
+
+### Fixed
+
+- Send Elasticsearch search `size` inside a copied request body, avoiding caller-dict
+  mutation and duplicate-value errors on client majors 8.x and 9.x.
+- Return a plain `list` from `ElasticsearchClient.get_indices_info()` on every supported
+  Elasticsearch client major.
+- Make no-argument `slack_formatter.format_nz_time()` independent of the host timezone.
+
+### Added
+
+- Add an Elasticsearch compatibility test matrix for client majors 7.17, 8.x, and 9.x.
+- Add ADRs covering Elasticsearch version support and the move to `zoneinfo`.
+
+## [2.0] - 2026-07-13
+
+### Removed
+
+- **BREAKING**: the `nui_lambda_shared_utils` import shim. Import from `nui_shared_utils` instead (`from nui_shared_utils import ...`). The shim forwarded to `nui_shared_utils` with a `DeprecationWarning` since the 1.3.1 import-root rename and was always slated for removal in 2.0.0.
+  - The old distribution name still installs: `pip install nui-lambda-shared-utils` continues to resolve via the redirect package (which depends on `nui-python-shared-utils`). Only the old import root is gone.
+
+### Added
+
+- Add ADRs for the package's generic scope, lazy integrations, credential precedence,
+  explicit Secrets Manager region resolution, and Snowflake SQL API choice.
+
+## [1.6.0] - 2026-07-13
+
+### Added
+
+- Add a backend-neutral structured logging field registry, request-scoped Powertools and
+  stdlib binders, binding helpers, field validation, and recursive payload redaction.
+- Add the structured logging contract guide and ADR-001.
+
+## [1.5.0] - 2026-07-02
+
+### Added
+
+- Add optional `unfurl_links` and `unfurl_media` controls to `SlackClient.send_message`.
+  Unset values preserve Slack's existing defaults.
+
+## [1.4.1] - 2026-06-29
+
+### Fixed
+
+- Replace the Snowflake extra's direct Git dependency with the published
+  `snowflake-sql-api>=0.1.1,<0.2.0` range so PyPI accepts the package metadata.
+
+## [1.4.0] - 2026-06-29
+
+### Added
+
+- Add the pure-Python Snowflake SQL API client behind the optional `[snowflake]` extra,
+  including sync and async factories, key-pair credential resolution, query logging, and
+  offline tests.
+- Add a generic Anthropic client helper behind the optional `[llm]` extra, with API-key and
+  Bedrock authentication, forced tool-use calls, and text calls.
+- Add Snowflake and LLM integration guides.
+
+### Changed
+
+- **BREAKING**: resolve the Secrets Manager region from the boto3 session, `AWS_REGION`, or
+  `AWS_DEFAULT_REGION`, and fail explicitly instead of silently defaulting to
+  `ap-southeast-2`.
+
+## [1.3.3] - 2026-05-06
+
+### Performance
+
+- Cold-start init for Lambda consumers reduced by ~400ms.
+  - `boto3` and `botocore` are no longer imported at module load. They are deferred to first use inside `secrets_helper.get_secret`, `utils.create_aws_client`/`resolve_aws_region`, and `cloudwatch_metrics.MetricsPublisher` (now lazy-constructs its boto3 client on first metric publish).
+  - The package `__init__` uses PEP 562 `__getattr__` so submodules (`slack_client`, `es_client`, `db_client`, `jwt_auth`, `powertools_helpers`, etc.) only load when their attributes are first accessed. `import nui_shared_utils` is ~10ms, down from ~700ms+.
+- Public API is unchanged. Optional integrations still resolve to `None` when their backing dependency is not installed (matching the previous behaviour).
+
+### Added
+
+- `scripts/bench_imports.py` for measuring `python -X importtime` cumulative cost across common entry points.
+- `tests/test_lazy_imports.py` regression tests (boto3 must not appear in `sys.modules` after a bare package import or after importing `jwt_auth`/`powertools_helpers`).
+
+## [1.3.2] - 2026-04-10
+
+### Fixed
+
+- `get_elasticsearch_credentials()` now checks env vars (`ES_PASSWORD`/`ES_PASS`) before Secrets Manager, aligning with `BaseServiceClient` credential precedence
+- `ESClient._resolve_credentials_from_env()` accepts both `ES_PASSWORD`/`ES_USERNAME` and `ES_PASS`/`ES_USER` conventions
+
+## [1.3.1] - 2026-04-05
+
+### Changed
+
+- Renamed internal import from `nui_lambda_shared_utils` to `nui_shared_utils`
+
+## [1.3.0] - 2026-04-05
+
+### Changed
+
+- Renamed PyPI package from `nui-lambda-shared-utils` to `nui-python-shared-utils`
+- Added explicit license field for twine 6.x compatibility
+- Updated CI publish workflow for new package name and tag pattern
+
+## [1.0.0 - 1.2.3] - 2025-11 to 2026-03
+
+### Added
+
+- **Log Processors Module** - `log_processors` for Kinesis/CloudWatch log extraction
+- **AWS Powertools Integration** - `powertools_helpers` module with `get_powertools_logger()` and `powertools_handler()`
+- **JWT Authentication** - RS256 token validation for API Gateway Lambdas
+- **Slack Block Kit improvements** - Enhanced block builder patterns
+- **`slack-channel-setup` CLI tool** for Slack workspace automation
+- `service_name` parameter to `SlackClient` for custom display names
+- `[powertools]` optional dependency group
+- Streaming bulk operations for Elasticsearch
+- Environment-based credential resolution in `BaseServiceClient`
+- Pass-through account names support
+
+### Changed
+
+- Rebranded as generic production-ready package (not NUI-specific)
+- Improved Slack Lambda header formatting (cleaner 2-line layout)
+- Documentation reorganization and expansion
+
+### Breaking Changes
+
+- Removed `SERVICE_EMOJI` from public API exports
+- Removed `get_entity_stats()` and `get_record_stats()` from `DatabaseClient`
+- `format_currency()` now requires explicit currency parameter
+
+## [0.0.6] - 2025-11-16
+
+### Added
+
+- Configuration YAML support for account info
+- RabbitMQ integration utilities
+
+### Changed
+
+- Enhanced configuration system
+
+## [0.0.5] - 2025-09-29
+
+### Added
+
+- Comprehensive test improvements
+- Better test coverage across modules
+
+### Fixed
+
+- Various test fixes and improvements
+
+## [0.0.4] - 2025-09-05
+
+### Changed
+
+- Minor improvements and fixes
+
+## [0.0.3] - 2025-09-05
+
+### Changed
+
+- Initial improvements
+
+## [0.0.2] - 2025-09-05
+
+### Added
+
+- Initial package structure
+- Core client implementations (Slack, Elasticsearch, Database)
+- Metrics publishing utilities
+- Error handling decorators
+- Timezone utilities for NZ timezone handling
+
+## [0.0.1] - 2025-09-05
+
+### Added
+
+- Initial release
+- Basic project structure
+- Python package configuration
+
+[Unreleased]: https://github.com/nuimarkets/nui-python-shared-utils/compare/2.3...HEAD
+[2.3]: https://github.com/nuimarkets/nui-python-shared-utils/compare/2.2...2.3
+[2.2]: https://github.com/nuimarkets/nui-python-shared-utils/compare/2.1...2.2
+[2.1]: https://github.com/nuimarkets/nui-python-shared-utils/compare/2.0...2.1
+[2.0]: https://github.com/nuimarkets/nui-python-shared-utils/compare/1.6.0...2.0
+[1.6.0]: https://github.com/nuimarkets/nui-python-shared-utils/compare/1.5.0...1.6.0
+[1.5.0]: https://github.com/nuimarkets/nui-python-shared-utils/compare/1.4.1...1.5.0
+[1.4.1]: https://github.com/nuimarkets/nui-python-shared-utils/compare/1.4.0...1.4.1
+[1.4.0]: https://github.com/nuimarkets/nui-python-shared-utils/compare/1.3.3...1.4.0
+[1.3.3]: https://github.com/nuimarkets/nui-python-shared-utils/compare/1.3.2...1.3.3
+[1.3.2]: https://github.com/nuimarkets/nui-python-shared-utils/compare/1.3.1...1.3.2
+[1.3.1]: https://github.com/nuimarkets/nui-python-shared-utils/compare/1.3.0...1.3.1
+[1.3.0]: https://github.com/nuimarkets/nui-python-shared-utils/compare/0.0.6...1.3.0
+[0.0.6]: https://github.com/nuimarkets/nui-python-shared-utils/compare/0.0.5...0.0.6
+[0.0.5]: https://github.com/nuimarkets/nui-python-shared-utils/compare/0.0.4...0.0.5
+[0.0.4]: https://github.com/nuimarkets/nui-python-shared-utils/compare/0.0.3...0.0.4
+[0.0.3]: https://github.com/nuimarkets/nui-python-shared-utils/compare/0.0.2...0.0.3
+[0.0.2]: https://github.com/nuimarkets/nui-python-shared-utils/compare/0.0.1...0.0.2
+[0.0.1]: https://github.com/nuimarkets/nui-python-shared-utils/releases/tag/0.0.1

{nui_python_shared_utils-2.3 → nui_python_shared_utils-2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nui-python-shared-utils
-Version: 2.3
+Version: 2.4
 Summary: Shared Python utilities for AWS Lambda, CLI tools, and agents with Slack, Elasticsearch, and monitoring integrations
 Home-page: https://github.com/nuimarkets/nui-python-shared-utils
 Author: NUI Markets

{nui_python_shared_utils-2.3 → nui_python_shared_utils-2.4}/docs/guides/powertools-integration.md

@@ -168,6 +168,9 @@ def powertools_handler(
     service_name: str,
     metrics_namespace: Optional[str] = None,
     slack_alert_channel: Optional[str] = None,
+    slack_account_names: Optional[Dict[str, str]] = None,
+    slack_account_names_config: Optional[str] = None,
+    reraise_exceptions: bool = False,
 ):
     """
     Decorator for Lambda handlers with logging, metrics, and error handling.
@@ -176,6 +179,9 @@ def powertools_handler(
         service_name: Service identifier for logging and metrics dimensions
         metrics_namespace: CloudWatch namespace (e.g., "NUI/TenderAnalyser")
         slack_alert_channel: Slack channel for error alerts (e.g., "#alerts")
+        slack_account_names: AWS account ID to display-name mapping for alerts
+        slack_account_names_config: YAML file containing account-name defaults
+        reraise_exceptions: Re-raise after logging and alerting. Defaults to False.
     """
 ```
 
@@ -193,7 +199,46 @@ The decorator logs the exception and sends the Slack alert (if configured), then
   group waits until the batch succeeds or redrives, and the Slack alert fires on every
   receive. For per-record retries, enable `ReportBatchItemFailures` on the mapping and
   return `{"batchItemFailures": [...]}` from the handler; the decorator passes it through.
-- **Every other event**: returns `{"statusCode": 500, "body": "Internal Server Error"}`.
+- **Every other event, by default**: returns
+  `{"statusCode": 500, "body": "Internal Server Error"}` to preserve existing behavior.
+- **Every other event with `reraise_exceptions=True`**: re-raises after logging the
+  exception and sending the configured Slack alert. Use this for Kinesis and DynamoDB
+  Streams, where a raised exception prevents the checkpoint from advancing, and for
+  asynchronous sources such as SNS, S3, and EventBridge, where Lambda failure handling
+  depends on the invocation failing.
+
+The option is handler-wide rather than an event-source list because the handler's trigger
+configuration determines whether Lambda failure semantics are appropriate. SQS does not
+require the option and continues to re-raise unconditionally.
+
+For synchronous triggers such as API Gateway, enabling this option changes failures from
+the decorator's 500 response into an unhandled Lambda invocation error. Enable it only when
+that caller-visible behavior is intended.
+
+### Re-raise Failures for Stream and Async Sources
+
+```python
+from nui_shared_utils import powertools_handler
+
+
+@powertools_handler(
+    service_name="event-processor",
+    slack_alert_channel="#production-alerts",
+    reraise_exceptions=True,
+)
+def handler(event, context):
+    records = event.get("Records", [])
+    for record in records:
+        print(record)
+    return {"processed": len(records)}
+```
+
+Logging and Slack alerting run before the original exception is re-raised. Configure the
+trigger's retry, bisect, failure destination, or dead-letter behavior separately in AWS.
+Every retry invokes the decorator again, so a configured Slack alert can be sent once per
+attempt. For Kinesis and DynamoDB Streams, maximum retry attempts and maximum record age
+bound retries. Bisect-on-error isolates a failing record by adding invocations, which do not
+consume the retry quota and can produce additional alerts.
 
 ### Basic Usage (Logging Only)
 

{nui_python_shared_utils-2.3 → nui_python_shared_utils-2.4}/nui_python_shared_utils.egg-info/SOURCES.txt

@@ -1,6 +1,7 @@
 .editorconfig
 .markdownlint-cli2.yaml
 AGENTS.md
+CHANGELOG.md
 CLAUDE.md
 CONTRIBUTING.md
 LICENSE

{nui_python_shared_utils-2.3 → nui_python_shared_utils-2.4}/nui_shared_utils/powertools_helpers.py

@@ -162,6 +162,7 @@ def powertools_handler(
     slack_alert_channel: Optional[str] = None,
     slack_account_names: Optional[Dict[str, str]] = None,
     slack_account_names_config: Optional[str] = None,
+    reraise_exceptions: bool = False,
 ):
     """
     Decorator for Lambda handlers with logging, metrics, and error handling.
@@ -175,6 +176,7 @@ def powertools_handler(
     - Optional metrics.log_metrics integration (if metrics_namespace provided)
     - Structured exception logging with traceback
     - Optional Slack alerts on failures (if slack_alert_channel provided)
+    - Optional exception re-raising for event sources that use Lambda failure semantics
     - Graceful degradation if Slack client unavailable
     - Proper Lambda error response formatting
 
@@ -190,6 +192,11 @@ def powertools_handler(
             Values loaded from this file are used as defaults, but any keys
             provided in ``slack_account_names`` take precedence and override
             the YAML-loaded values.
+        reraise_exceptions: Re-raise handler exceptions after logging and Slack
+            alerting. Enable this for event sources such as Kinesis, DynamoDB
+            Streams, SNS, S3, and EventBridge that rely on a raised exception to
+            record a failed invocation and apply their retry or failure handling.
+            Defaults to False to preserve the existing 500 response behavior.
 
     Returns:
         Decorator function for Lambda handlers
@@ -214,13 +221,14 @@ def powertools_handler(
         for API Gateway, ``{"batchItemFailures": [...]}`` for an SQS mapping with
         ``ReportBatchItemFailures``.
 
-        On exception, returns ``{"statusCode": 500, "body": "Internal Server Error"}``,
-        except for SQS events, where the exception is re-raised after logging and
-        alerting. A returned value tells the SQS mapping the batch succeeded (with
-        ``ReportBatchItemFailures``, a response without ``batchItemFailures`` means
-        every record did), so the 500 dict would delete the batch. Re-raising leaves
-        it on the queue for redelivery, and in the DLQ after ``maxReceiveCount``
-        when the queue has a redrive policy.
+        On exception, returns ``{"statusCode": 500, "body": "Internal Server Error"}``
+        by default. Set ``reraise_exceptions=True`` to re-raise after logging and
+        alerting when the event source relies on Lambda failure semantics. SQS events
+        are always re-raised, regardless of this option: a returned value tells the
+        SQS mapping the batch succeeded (with ``ReportBatchItemFailures``, a response
+        without ``batchItemFailures`` means every record did), so the 500 dict would
+        delete the batch. Re-raising leaves it on the queue for redelivery, and in the
+        DLQ after ``maxReceiveCount`` when the queue has a redrive policy.
     """
 
     def decorator(func: Callable) -> Callable:
@@ -297,8 +305,9 @@ def powertools_handler(
                     except Exception as slack_error:
                         logger.warning("Failed to send Slack alert: %s", slack_error)
 
-                # Re-raise for SQS so the batch stays on the queue; see the Note above.
-                if is_sqs:
+                # Re-raise for SQS so the batch stays on the queue, or when the caller
+                # opts into Lambda failure semantics; see the Note above.
+                if is_sqs or reraise_exceptions:
                     raise
 
                 # Return proper Lambda error response

{nui_python_shared_utils-2.3 → nui_python_shared_utils-2.4}/tests/test_powertools_helpers.py

@@ -7,7 +7,6 @@ from unittest.mock import MagicMock, patch
 
 import pytest
 
-
 # Test markers
 pytestmark = pytest.mark.unit
 
@@ -338,6 +337,18 @@ def _sqs_event(*message_ids):
     return {"Records": [{"messageId": mid, "eventSource": "aws:sqs", "body": "{}"} for mid in message_ids]}
 
 
+def _kinesis_event():
+    return {
+        "Records": [
+            {
+                "eventID": "shardId-000000000000:1",
+                "eventSource": "aws:kinesis",
+                "kinesis": {"sequenceNumber": "1", "data": "e30="},
+            }
+        ]
+    }
+
+
 class TestPowertoolsHandlerSqs:
     """An SQS batch must stay on the queue when the handler raises.
 
@@ -481,6 +492,69 @@ class TestPowertoolsHandlerSqs:
         assert result == {"statusCode": 500, "body": "Internal Server Error"}
 
 
+class TestPowertoolsHandlerReraise:
+    def test_kinesis_exception_is_reraised_when_enabled(self, monkeypatch):
+        monkeypatch.delenv("AWS_LAMBDA_RUNTIME_API", raising=False)
+
+        from nui_shared_utils.powertools_helpers import powertools_handler
+
+        error = RuntimeError("Kinesis record failed")
+
+        @powertools_handler(service_name="test-service", reraise_exceptions=True)
+        def handler(event, context):
+            raise error
+
+        with pytest.raises(RuntimeError) as excinfo:
+            handler(_kinesis_event(), MagicMock())
+
+        assert excinfo.value is error
+
+    def test_kinesis_exception_keeps_500_by_default(self, monkeypatch):
+        monkeypatch.delenv("AWS_LAMBDA_RUNTIME_API", raising=False)
+
+        from nui_shared_utils.powertools_helpers import powertools_handler
+
+        @powertools_handler(service_name="test-service", reraise_exceptions=False)
+        def handler(event, context):
+            raise RuntimeError("Kinesis record failed")
+
+        result = handler(_kinesis_event(), MagicMock())
+
+        assert result == {"statusCode": 500, "body": "Internal Server Error"}
+
+    def test_eventbridge_exception_alerts_before_reraising(self, monkeypatch):
+        monkeypatch.delenv("AWS_LAMBDA_RUNTIME_API", raising=False)
+
+        from nui_shared_utils.powertools_helpers import powertools_handler
+
+        calls = []
+        mock_logger = MagicMock()
+        mock_logger.inject_lambda_context.side_effect = lambda func: func
+        mock_logger.exception.side_effect = lambda *args, **kwargs: calls.append("log")
+        mock_slack_client = MagicMock()
+        mock_slack_client.send_message.side_effect = lambda **kwargs: calls.append("slack")
+
+        with patch("nui_shared_utils.powertools_helpers.get_powertools_logger", return_value=mock_logger):
+            with patch("nui_shared_utils.powertools_helpers.SLACK_CLIENT_AVAILABLE", True):
+                with patch("nui_shared_utils.powertools_helpers.SlackClient", return_value=mock_slack_client):
+
+                    @powertools_handler(
+                        service_name="test-service",
+                        slack_alert_channel="#errors",
+                        reraise_exceptions=True,
+                    )
+                    def handler(event, context):
+                        raise ValueError("EventBridge event failed")
+
+                    with pytest.raises(ValueError, match="EventBridge event failed"):
+                        handler(
+                            {"source": "example.orders", "detail-type": "Order Created", "detail": {}},
+                            MagicMock(),
+                        )
+
+        assert calls == ["log", "slack"]
+
+
 class TestOptionalImports:
     """Tests for graceful degradation of optional dependencies"""