nui-python-shared-utils 1.6.0__tar.gz → 2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/.github/workflows/ci.yml +1 -2
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/.github/workflows/publish.yml +1 -29
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/MANIFEST.in +0 -2
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/PKG-INFO +1 -2
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/README.md +0 -1
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/README.md +1 -1
- nui_python_shared_utils-2.0/docs/decisions/ADR-002-keep-shared-utilities-generic.md +94 -0
- nui_python_shared_utils-2.0/docs/decisions/ADR-003-lazy-load-integrations.md +110 -0
- nui_python_shared_utils-2.0/docs/decisions/ADR-004-credential-resolution-precedence.md +98 -0
- nui_python_shared_utils-2.0/docs/decisions/ADR-005-explicit-secrets-manager-region.md +88 -0
- nui_python_shared_utils-2.0/docs/decisions/ADR-006-snowflake-sql-api-client.md +105 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/decisions/README.md +5 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/getting-started/installation.md +1 -1
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_python_shared_utils.egg-info/SOURCES.txt +5 -23
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/pyproject.toml +1 -1
- nui_python_shared_utils-2.0/redirect/README.md +22 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_lazy_imports.py +0 -24
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_snowflake_client.py +1 -12
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/__init__.py +0 -44
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/base_client.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/cli.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/cloudwatch_metrics.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/config.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/db_client.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/error_handler.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/es_client.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/es_query_builder.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/jwt_auth.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/lambda_helpers.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/log_processors.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/powertools_helpers.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/secrets_helper.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/slack_client.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/slack_formatter.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/slack_setup/__init__.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/slack_setup/channel_creator.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/slack_setup/channel_definitions.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/slack_setup/setup_helpers.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/snowflake_client.py +0 -3
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/timezone.py +0 -2
- nui_python_shared_utils-1.6.0/nui_lambda_shared_utils/utils.py +0 -2
- nui_python_shared_utils-1.6.0/redirect/README.md +0 -27
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/.editorconfig +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/.github/workflows/test.yml +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/.markdownlint-cli2.yaml +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/AGENTS.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/CLAUDE.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/CONTRIBUTING.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/LICENSE +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/decisions/ADR-001-structured-logging-field-vocabulary.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/development/testing.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/getting-started/configuration.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/getting-started/quickstart.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/cli-tools.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/elasticsearch-integration.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/jwt-authentication.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/lambda-utilities.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/llm-integration.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/log-processing.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/powertools-integration.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/shared-types.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/slack-integration.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/snowflake-integration.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/guides/structured-logging-contract.md +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/mypy.ini +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/__init__.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/base_client.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/cli.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/cloudwatch_metrics.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/config.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/db_client.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/error_handler.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/es_client.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/es_query_builder.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/jwt_auth.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/lambda_helpers.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/llm.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/log_processors.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/logging/__init__.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/logging/binding.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/logging/fields.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/logging/redaction.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/powertools_helpers.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/secrets_helper.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/slack_client.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/slack_formatter.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/slack_setup/__init__.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/slack_setup/channel_creator.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/slack_setup/channel_definitions.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/slack_setup/setup_helpers.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/snowflake_client.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/timezone.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/nui_shared_utils/utils.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/pytest.ini +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/redirect/pyproject.toml +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/redirect/setup.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/requirements-test.txt +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/scripts/bench_imports.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/setup.cfg +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/setup.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/__init__.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_aws_utils.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_base_client.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_cloudwatch_metrics.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_config.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_db_client.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_error_handler.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_es_client.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_es_query_builder.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_jwt_auth.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_lambda_helpers.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_llm.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_log_processors.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_logging_binding.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_logging_fields.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_powertools_helpers.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_redaction.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_secrets_helper.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_slack_client.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_slack_formatter.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_timezone.py +0 -0
- {nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/tests/test_utils.py +0 -0
|
@@ -109,5 +109,4 @@ jobs:
|
|
|
109
109
|
run: |
|
|
110
110
|
pip install dist/*.whl
|
|
111
111
|
python -c "import nui_shared_utils; print('✅ Package imports successfully')"
|
|
112
|
-
python -c "from nui_shared_utils import Config, get_secret; print('✅ Core imports work')"
|
|
113
|
-
python -W ignore::DeprecationWarning -c "import nui_lambda_shared_utils; print('✅ Backwards-compat shim works')"
|
|
112
|
+
python -c "from nui_shared_utils import Config, get_secret; print('✅ Core imports work')"
|
|
@@ -107,32 +107,4 @@ jobs:
|
|
|
107
107
|
with:
|
|
108
108
|
verbose: true
|
|
109
109
|
print-hash: true
|
|
110
|
-
skip-existing: true # No-op instead of failing if the version is already on PyPI (safe re-runs)
|
|
111
|
-
|
|
112
|
-
publish-test:
|
|
113
|
-
name: Publish to TestPyPI
|
|
114
|
-
runs-on: ubuntu-latest
|
|
115
|
-
needs: [test, build]
|
|
116
|
-
# Publish to TestPyPI on manual (non-tag) dispatch only; tag pushes go to PyPI above
|
|
117
|
-
if: github.event_name == 'workflow_dispatch' && !startsWith(github.ref, 'refs/tags/')
|
|
118
|
-
environment:
|
|
119
|
-
name: testpypi
|
|
120
|
-
url: https://test.pypi.org/project/nui-python-shared-utils/
|
|
121
|
-
|
|
122
|
-
permissions:
|
|
123
|
-
id-token: write
|
|
124
|
-
contents: read
|
|
125
|
-
|
|
126
|
-
steps:
|
|
127
|
-
- name: Download artifacts
|
|
128
|
-
uses: actions/download-artifact@v7
|
|
129
|
-
with:
|
|
130
|
-
name: dist
|
|
131
|
-
path: dist/
|
|
132
|
-
|
|
133
|
-
- name: Publish to TestPyPI
|
|
134
|
-
uses: pypa/gh-action-pypi-publish@release/v1
|
|
135
|
-
with:
|
|
136
|
-
repository-url: https://test.pypi.org/legacy/
|
|
137
|
-
verbose: true
|
|
138
|
-
print-hash: true
|
|
110
|
+
skip-existing: true # No-op instead of failing if the version is already on PyPI (safe re-runs)
|
|
@@ -1,12 +1,10 @@
|
|
|
1
1
|
# Include the license and readme
|
|
2
2
|
include LICENSE
|
|
3
3
|
include README.md
|
|
4
|
-
include CHANGELOG.md
|
|
5
4
|
|
|
6
5
|
# Include package data
|
|
7
6
|
recursive-include nui_shared_utils *.py
|
|
8
7
|
recursive-include nui_shared_utils/slack_setup *.yaml
|
|
9
|
-
recursive-include nui_lambda_shared_utils *.py
|
|
10
8
|
|
|
11
9
|
# Exclude development and build files
|
|
12
10
|
exclude .gitignore
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: nui-python-shared-utils
|
|
3
|
-
Version:
|
|
3
|
+
Version: 2.0
|
|
4
4
|
Summary: Shared Python utilities for AWS Lambda, CLI tools, and agents with Slack, Elasticsearch, and monitoring integrations
|
|
5
5
|
Home-page: https://github.com/nuimarkets/nui-python-shared-utils
|
|
6
6
|
Author: NUI Markets
|
|
@@ -526,7 +526,6 @@ slack-channel-setup --config channels.yaml
|
|
|
526
526
|
- **GitHub Repository**: https://github.com/nuimarkets/nui-python-shared-utils
|
|
527
527
|
- **Issue Tracker**: https://github.com/nuimarkets/nui-python-shared-utils/issues
|
|
528
528
|
- **PyPI Package**: https://pypi.org/project/nui-python-shared-utils/
|
|
529
|
-
- **Changelog**: [CHANGELOG.md](CHANGELOG.md)
|
|
530
529
|
|
|
531
530
|
### 💬 Support
|
|
532
531
|
|
|
@@ -427,7 +427,6 @@ slack-channel-setup --config channels.yaml
|
|
|
427
427
|
- **GitHub Repository**: https://github.com/nuimarkets/nui-python-shared-utils
|
|
428
428
|
- **Issue Tracker**: https://github.com/nuimarkets/nui-python-shared-utils/issues
|
|
429
429
|
- **PyPI Package**: https://pypi.org/project/nui-python-shared-utils/
|
|
430
|
-
- **Changelog**: [CHANGELOG.md](CHANGELOG.md)
|
|
431
430
|
|
|
432
431
|
### 💬 Support
|
|
433
432
|
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# ADR-002: Keep shared utilities generic
|
|
2
|
+
|
|
3
|
+
**Status**: Proposed
|
|
4
|
+
**Date**: 2026-07-13
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
This package ships to PyPI as `nui-python-shared-utils` and is imported by many independent Lambda
|
|
9
|
+
functions and CLI tools. `README.md` targets two audiences on purpose: the origin team, whose
|
|
10
|
+
defaults match, and external teams, who share none of its schema, currency, or service taxonomy. A
|
|
11
|
+
shared library that bakes one consumer's domain into its public surface forces every other consumer
|
|
12
|
+
to either carry dead weight or fork.
|
|
13
|
+
|
|
14
|
+
The pressure to bake it in is recurring and structural, not a one-time slip. It is always faster to
|
|
15
|
+
hardcode a caller's known table names, service list, or home currency into a "convenience" method
|
|
16
|
+
than to thread those through as parameters, and each such shortcut looks locally harmless. So the
|
|
17
|
+
constraint has to be a standing rule that every new method is measured against, not a cleanup done
|
|
18
|
+
once.
|
|
19
|
+
|
|
20
|
+
The line that survives that pressure is **infrastructure vs. business**. The domain-neutral half
|
|
21
|
+
(Slack messaging, Elasticsearch query building, database connection and query execution, CloudWatch
|
|
22
|
+
metrics, JWT validation, secrets, timezone) is what belongs in the shared core. The business half
|
|
23
|
+
(what a row means, which services exist, what currency a number is in, which status values are
|
|
24
|
+
terminal) is the consumer's, and it changes per deployment.
|
|
25
|
+
|
|
26
|
+
## Decision
|
|
27
|
+
|
|
28
|
+
Public APIs expose configurable infrastructure utilities and leave business-specific queries, names,
|
|
29
|
+
currencies, and mappings to the consumer.
|
|
30
|
+
|
|
31
|
+
- **Database access is generic.** `query()` / `execute()` take SQL and parameters. No method assumes
|
|
32
|
+
a table schema, column names, or a soft-delete/status convention.
|
|
33
|
+
- **Values that vary by geography or deployment are required parameters or explicit configuration**,
|
|
34
|
+
never defaulted to one org's convention. Currency, service name, timezone, and environment label
|
|
35
|
+
are supplied by the caller.
|
|
36
|
+
- **No org-specific mapping constants** (service-to-emoji tables, status vocabularies) live in the
|
|
37
|
+
shipped surface. A caller that wants one passes it in.
|
|
38
|
+
|
|
39
|
+
The rule is enforced continuously by `AGENTS.md`: the "Keeping the Package Generic" DO/DON'T list,
|
|
40
|
+
the three-question "When Adding New Features" gate ("Is this specific to NUI, or useful for any
|
|
41
|
+
Lambda project? / Could this be made configurable rather than hardcoded? / Can someone use this
|
|
42
|
+
without NUI-specific knowledge?"), the Pull Request Checklist item **"No hardcoded
|
|
43
|
+
organization-specific values (service names, currencies, business logic)"**, and the "Code Review
|
|
44
|
+
Focus Areas" ("Generic utility patterns vs vendor-specific code"). Commit `ce48031` applied the rule
|
|
45
|
+
(removed `get_entity_stats` / `get_record_stats`, the `SERVICE_EMOJI` map, the `currency="NZD"`
|
|
46
|
+
default, and value-key currency inference); that is evidence the rule bites, not the decision itself.
|
|
47
|
+
|
|
48
|
+
## Alternatives considered
|
|
49
|
+
|
|
50
|
+
- **Ship domain convenience helpers in the core** (org-specific stats queries, a service-emoji map,
|
|
51
|
+
an assumed row schema). Ergonomic for the one consumer whose schema matches. Rejected: every other
|
|
52
|
+
consumer imports code that queries tables it does not have, and a schema change in one consumer
|
|
53
|
+
becomes a shared-library release. `get_entity_stats`/`get_record_stats` embedded `entities`/`users`
|
|
54
|
+
tables, an `entity_id` foreign key, a `deleted_at` soft-delete column, and hardcoded
|
|
55
|
+
`'confirmed'`/`'cancelled'` status literals: none of that is a shared library's business.
|
|
56
|
+
- **Keep configurable defaults but seed them with the origin org's values** (e.g. `currency="NZD"`).
|
|
57
|
+
Rejected: a default silently encodes one geography. A caller in another currency gets numerically
|
|
58
|
+
wrong output with no error to catch it. A required parameter forces the choice at the call site.
|
|
59
|
+
- **A kwargs escape hatch** (`get_record_stats(status_col=..., value_col=..., created_col=...)`).
|
|
60
|
+
Rejected: the mapping kwargs prove the method is not generic, it is one schema with holes punched in
|
|
61
|
+
it. A raw `query()` taking the caller's own SQL is both simpler and actually neutral.
|
|
62
|
+
- **Split the business helpers into a separate consumer-specific package.** Viable, and the correct
|
|
63
|
+
home if such a helper is ever genuinely shared across consumers. Not built now: the removed helpers
|
|
64
|
+
were thin logic wrapping a SQL string, so it is cheaper for each consumer to own its query than to
|
|
65
|
+
version a shared domain layer for it.
|
|
66
|
+
|
|
67
|
+
## Consequences
|
|
68
|
+
|
|
69
|
+
- Consumers write their own domain queries and pass their own currency / service / schema values.
|
|
70
|
+
More boilerplate per consumer, in exchange for zero cross-consumer coupling.
|
|
71
|
+
- The package is adoptable by a team with no knowledge of the origin platform, which is the third
|
|
72
|
+
gate question made real.
|
|
73
|
+
- Reviewers get a bright-line test (does this name a table, currency, service, or status a specific
|
|
74
|
+
org uses?) instead of a per-PR judgment call.
|
|
75
|
+
- Convenience is pushed to the consuming edges; the core stays small, which also serves the
|
|
76
|
+
optional-extras bundle-size goal.
|
|
77
|
+
- One deliberate exception is on record: ADR-006 ships overridable `Pacific/Auckland` / `NUI_LAMBDA`
|
|
78
|
+
Snowflake session defaults, governed as a bounded exception in that ADR. The bright-line test still
|
|
79
|
+
holds everywhere else; a new baked geography/org default needs the same explicit, ADR-level
|
|
80
|
+
justification, or it is out.
|
|
81
|
+
|
|
82
|
+
## Load-bearing premises (supersede triggers)
|
|
83
|
+
|
|
84
|
+
- **This package stays a general-purpose shared library with multiple independent consumers**
|
|
85
|
+
(including external PyPI users). If it collapsed to a single internal consumer, folded into one
|
|
86
|
+
application as an internal module, the generic constraint reopens: a library with one caller has no
|
|
87
|
+
one to stay neutral for, and inlining that caller's schema becomes reasonable.
|
|
88
|
+
- **The infrastructure/business split stays clean.** Connection, query execution, formatting, and
|
|
89
|
+
metrics stay domain-neutral; row meaning, service list, currency, and status vocabulary stay the
|
|
90
|
+
consumer's. If a domain concept emerged that every consumer implemented identically, a shared
|
|
91
|
+
domain layer (likely a separate package, not a bulge in the generic core) would be reconsidered.
|
|
92
|
+
- **Requiring the caller to supply currency / service / schema stays cheap enough that no default is
|
|
93
|
+
worth the geography lock-in.** If a default ever became strictly necessary, it would come from
|
|
94
|
+
explicit configuration, not from a baked geography or currency constant.
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
# ADR-003: Lazy-load integrations to keep package import cheap
|
|
2
|
+
|
|
3
|
+
**Status**: Proposed
|
|
4
|
+
**Date**: 2026-07-13
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
This package is a shared utility layer for a fleet of AWS Lambda functions and CLI tools. Both
|
|
9
|
+
workloads are startup-latency sensitive: a Lambda pays the package's import cost on every cold start,
|
|
10
|
+
and a CLI pays it on every invocation. The package spans many integrations (Slack, Elasticsearch,
|
|
11
|
+
database drivers, CloudWatch/boto3, RS256 JWT via `rsa`, AWS Lambda Powertools, Snowflake, and an
|
|
12
|
+
Anthropic LLM helper), and most of those pull heavy third-party dependencies.
|
|
13
|
+
|
|
14
|
+
The public surface is a flat namespace: consumers write `import nui_shared_utils as nui` and reach
|
|
15
|
+
`nui.SlackClient`, `nui.get_secret`, `nui.bind_request`, etc. A naive way to offer that flat surface
|
|
16
|
+
is to import every submodule at the top of the package `__init__`, which means `import
|
|
17
|
+
nui_shared_utils` transitively imports boto3, the Slack SDK, elasticsearch, and everything else,
|
|
18
|
+
whether or not the caller touches them.
|
|
19
|
+
|
|
20
|
+
Two properties of the stack make that cost avoidable:
|
|
21
|
+
|
|
22
|
+
- **The integrations are independent.** A Lambda that only validates a JWT never needs boto3; one
|
|
23
|
+
that only publishes metrics never needs the Slack SDK. Import cost is dominated by dependencies the
|
|
24
|
+
caller does not use.
|
|
25
|
+
- **The dependencies are already optional at install time.** The package exposes per-integration
|
|
26
|
+
extras (`elasticsearch`, `database`, `slack`, `jwt`, `snowflake`, `llm`, plus `all`) so a consumer
|
|
27
|
+
installs only the heavy deps it needs. Eager top-level imports would either break (an uninstalled
|
|
28
|
+
extra raises `ImportError` at package import) or force every consumer onto the full dependency set.
|
|
29
|
+
|
|
30
|
+
## Decision
|
|
31
|
+
|
|
32
|
+
The package's public API resolves lazily through a PEP 562 module-level `__getattr__`
|
|
33
|
+
(`nui_shared_utils/__init__.py`). `import nui_shared_utils` executes almost no integration code; a
|
|
34
|
+
heavy dependency loads only when the export that needs it is first accessed.
|
|
35
|
+
|
|
36
|
+
- **An export map, not eager imports.** `_LAZY_EXPORTS` maps each public name to its
|
|
37
|
+
`(submodule, attr)` pair. On first access to a missing attribute, `__getattr__` imports that one
|
|
38
|
+
submodule, pulls the attribute, and caches it into the module `globals()` so subsequent access
|
|
39
|
+
skips the loader entirely (PEP 562 `__getattr__` fires only for names not already in the module
|
|
40
|
+
dict).
|
|
41
|
+
- **Optional integrations degrade to `None`.** Submodules whose heavy dep is an optional extra are
|
|
42
|
+
listed in `_OPTIONAL_SUBMODULES`; an `ImportError` during lazy resolution yields `None` instead of
|
|
43
|
+
propagating, so a capability check like `if nui.SlackClient is not None` still works when the extra
|
|
44
|
+
is not installed. Non-optional submodules still raise.
|
|
45
|
+
- **The flat surface is preserved.** `__all__` and `__dir__` enumerate the full export set, so star
|
|
46
|
+
imports and interactive/tab completion behave as if everything were eagerly imported. A
|
|
47
|
+
`TYPE_CHECKING`-guarded block holds the real `from .submodule import ...` statements for static type
|
|
48
|
+
checkers and IDEs; that block is never executed at runtime.
|
|
49
|
+
- **Core boto3 use defers too.** Modules that back always-present exports (`secrets_helper`, `utils`,
|
|
50
|
+
`cloudwatch_metrics`) construct their boto3 clients on first call rather than at import, so even the
|
|
51
|
+
non-extra AWS path does not tax a cold start that never reaches it.
|
|
52
|
+
|
|
53
|
+
Commit `c301214` (PR #24) records the effect: `import nui_shared_utils` drops from ~700ms to ~10ms
|
|
54
|
+
locally. `tests/test_lazy_imports.py` locks it in, asserting in fresh subprocesses that a bare
|
|
55
|
+
package import loads neither boto3/botocore, nor `anthropic`, nor the `logging` subpackage, and that
|
|
56
|
+
representative integrations stay lazy until first touched (`jwt_auth` and `powertools_helpers` pull no
|
|
57
|
+
heavy dep on access, the legacy shim reaches an attribute without loading `db_client`/`es_client`, and
|
|
58
|
+
`secrets_helper` builds its boto3 client only on first call). The `__getattr__` mechanism makes every
|
|
59
|
+
export lazy; the suite pins this representative subset rather than asserting it for each integration.
|
|
60
|
+
|
|
61
|
+
## Alternatives considered
|
|
62
|
+
|
|
63
|
+
- **Eager import of every integration submodule (and its heavy deps) at package import.** The
|
|
64
|
+
simplest flat-API implementation, and the standing rejected option-class. Rejected because it makes
|
|
65
|
+
every consumer pay every dependency's import cost regardless of use (the ~700ms cold-start tax), and
|
|
66
|
+
because it forces the full dependency set to be installed or `import nui_shared_utils` breaks,
|
|
67
|
+
defeating the optional-extras model.
|
|
68
|
+
- **Drop the flat re-exports; require explicit submodule imports** (`from nui_shared_utils.slack_client
|
|
69
|
+
import SlackClient`). Achieves laziness but breaks the established `nui.X` API and star imports for
|
|
70
|
+
every caller. PEP 562 delivers the same laziness while keeping the flat surface.
|
|
71
|
+
- **`try/except ImportError` guards at module top.** A common optional-dependency idiom, but it still
|
|
72
|
+
imports every *installed* dependency eagerly, so an environment with the `all` extra pays the full
|
|
73
|
+
import cost on startup. It solves availability, not cold start.
|
|
74
|
+
- **Split each integration into its own distribution.** Per-integration install granularity already
|
|
75
|
+
comes from extras; N packages multiply versioning and release overhead without improving import
|
|
76
|
+
cost beyond what lazy loading already gives.
|
|
77
|
+
|
|
78
|
+
## Consequences
|
|
79
|
+
|
|
80
|
+
- `import nui_shared_utils` is cheap (~10ms); heavy deps load on first use of the export that needs
|
|
81
|
+
them. Cold-start and CLI-startup cost scale with what a caller uses, not with what the package
|
|
82
|
+
ships.
|
|
83
|
+
- The flat public API and star imports are unchanged, so no consumer edit is required to gain the
|
|
84
|
+
saving. The legacy import shim (`nui_lambda_shared_utils`) forwards through the same PEP 562 path,
|
|
85
|
+
so callers on the old name benefit too.
|
|
86
|
+
- New integrations must follow the pattern: add the export to `_LAZY_EXPORTS`, add the submodule to
|
|
87
|
+
`_OPTIONAL_SUBMODULES` if its dep is an optional extra, mirror it in the `TYPE_CHECKING` block for
|
|
88
|
+
IDE/type-checker visibility, and add a laziness assertion to `tests/test_lazy_imports.py`. A missing
|
|
89
|
+
`_LAZY_EXPORTS` entry breaks runtime access; a missing `TYPE_CHECKING` entry only costs static
|
|
90
|
+
completion.
|
|
91
|
+
- Two lists (`_LAZY_EXPORTS` and the `TYPE_CHECKING` imports) describe the same surface and must stay
|
|
92
|
+
in sync by hand. `__all__` and `__dir__` derive from `_LAZY_EXPORTS`, so the runtime export set stays
|
|
93
|
+
self-consistent (`test_dir_includes_lazy_exports` guards a sample of it); nothing cross-checks the
|
|
94
|
+
`TYPE_CHECKING` block against `_LAZY_EXPORTS`, so a gap there costs only static completion, as above.
|
|
95
|
+
- Import errors surface on first use rather than at package import. For optional integrations that is
|
|
96
|
+
intentional (`None` sentinel); for a genuinely missing required dependency the failure moves from
|
|
97
|
+
import time to first access, which the lazy tests and normal exercise cover.
|
|
98
|
+
|
|
99
|
+
## Load-bearing premises (supersede triggers)
|
|
100
|
+
|
|
101
|
+
- Package-import latency and Lambda bundle/cold-start cost remain a real, recurring cost paid per new
|
|
102
|
+
integration. If all consumers moved to always-on, long-running services where cold start is
|
|
103
|
+
irrelevant, the eager-import simplicity could be reconsidered.
|
|
104
|
+
- The optional-extras model stays: a given integration's heavy dependency is installed only when its
|
|
105
|
+
extra is requested. If every dependency became a hard requirement of the package, eager import would
|
|
106
|
+
no longer risk breaking on a missing dep (though the startup-cost argument would still stand on its
|
|
107
|
+
own).
|
|
108
|
+
- PEP 562 module-level `__getattr__` stays available on the supported interpreters. If the package
|
|
109
|
+
had to run where it is unavailable, the mechanism would need replacing (explicit submodule imports)
|
|
110
|
+
and this decision would reopen.
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# ADR-004: Credential resolution precedence
|
|
2
|
+
|
|
3
|
+
**Status**: Proposed
|
|
4
|
+
**Date**: 2026-07-13
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
This package ships service clients (Slack, Elasticsearch, database, Snowflake) that run in two
|
|
9
|
+
very different places. The primary runtime is short-lived functions with an IAM role and no
|
|
10
|
+
secrets in the environment, where a named secret in a managed secret store (AWS Secrets Manager)
|
|
11
|
+
is the idiomatic credential source: rotatable, IAM-scoped, absent from the process listing. The
|
|
12
|
+
secondary runtime is local development, CLI tools, and the unit-test suite, where AWS may be
|
|
13
|
+
unreachable, no role is attached, and the caller already holds the credentials or has them in the
|
|
14
|
+
environment.
|
|
15
|
+
|
|
16
|
+
A client that reaches for Secrets Manager unconditionally imposes that store on both runtimes. It
|
|
17
|
+
forces every construction to have AWS reachable plus IAM permission plus a network round-trip,
|
|
18
|
+
even when the caller passed the credentials in directly, and it couples what should be a pure
|
|
19
|
+
object construction to AWS mocking in tests. Callers legitimately supply credentials three ways:
|
|
20
|
+
an explicit dict at construction, per-client environment variables, or (the default) a named
|
|
21
|
+
secret. The store cannot be the only path without breaking local dev, CLI, and test isolation.
|
|
22
|
+
|
|
23
|
+
The function-based helper historically undercut this. `get_elasticsearch_credentials`
|
|
24
|
+
(`secrets_helper.py`) called Secrets Manager first even when `ES_PASSWORD` was set, so CLI and
|
|
25
|
+
local-dev paths paid an avoidable AWS call. Commit `0d3f8e8` (PR #23) added the env-var check
|
|
26
|
+
ahead of the store lookup, aligning the helper with the precedence the clients already enforced.
|
|
27
|
+
|
|
28
|
+
## Decision
|
|
29
|
+
|
|
30
|
+
Credentials resolve in a fixed order, first match wins, owned by `BaseClient._resolve_credentials`
|
|
31
|
+
in `base_client.py`:
|
|
32
|
+
|
|
33
|
+
1. **Explicit `credentials` dict** passed to the constructor. Returned as-is; nothing else is
|
|
34
|
+
consulted. The caller owns these values and the package does not merge or second-guess them.
|
|
35
|
+
2. **Per-client environment variables**, via each client's `_resolve_credentials_from_env` override
|
|
36
|
+
(Elasticsearch reads `ES_PASSWORD`/`ES_PASS` in `es_client.py`; database requires
|
|
37
|
+
`DB_HOST` + `DB_PASSWORD` in `db_client.py`; Slack requires `SLACK_BOT_TOKEN` in
|
|
38
|
+
`slack_client.py`). When the required vars are present, the resolved dict is returned without
|
|
39
|
+
touching AWS.
|
|
40
|
+
3. **AWS Secrets Manager** as the default fallback, via `_fetch_credentials_from_sm` and
|
|
41
|
+
`get_secret`, reading the named secret resolved from constructor arg, env var, or config.
|
|
42
|
+
|
|
43
|
+
A later tier fires only when every earlier tier is absent. An AWS call happens only when neither
|
|
44
|
+
an explicit dict nor env vars resolve, so local-dev, CLI, and test paths that supply explicit or
|
|
45
|
+
environment credentials make **zero** Secrets Manager calls. The direct function helper mirrors
|
|
46
|
+
the tail of this ladder for callers that bypass the client classes:
|
|
47
|
+
`get_elasticsearch_credentials` checks env vars, then the store.
|
|
48
|
+
|
|
49
|
+
## Alternatives considered
|
|
50
|
+
|
|
51
|
+
- **Always call Secrets Manager (first, or unconditionally).** Simplest single code path, and the
|
|
52
|
+
right default for the Lambda runtime taken alone. Rejected: it forces AWS reachability, IAM
|
|
53
|
+
permission, and a network round-trip onto every construction in every runtime, including local
|
|
54
|
+
dev, CLI, and unit tests that have no reason to touch AWS. Explicit and env credentials degrade
|
|
55
|
+
to dead options or a bolted-on override, and the test suite has to mock AWS to build an object.
|
|
56
|
+
This is the live rejected option: precedence exists specifically so the store is a fallback, not
|
|
57
|
+
a precondition.
|
|
58
|
+
- **Environment variables only (strict 12-factor).** Clean for containers, but the primary runtime's
|
|
59
|
+
idiomatic store is Secrets Manager (rotation, IAM scoping, secrets kept out of the env). Dropping
|
|
60
|
+
the store would push plaintext secrets into every function's environment.
|
|
61
|
+
- **Explicit injection everywhere (no env, no store).** Maximally testable, but forces every
|
|
62
|
+
entrypoint to hand-write secret retrieval, defeating the point of a shared client that just works
|
|
63
|
+
under an attached role.
|
|
64
|
+
- **Per-client ad-hoc ordering.** Each client invents its own precedence. Rejected: callers cannot
|
|
65
|
+
reason uniformly about where credentials come from, and the orders drift. One ladder lives in the
|
|
66
|
+
base class; clients override only the env-detection and default-secret hooks.
|
|
67
|
+
|
|
68
|
+
## Consequences
|
|
69
|
+
|
|
70
|
+
- One predictable resolution order across every client. A new client implements
|
|
71
|
+
`_resolve_credentials_from_env` and `_get_default_secret_name` and inherits the full ladder.
|
|
72
|
+
- Unit tests construct clients from an explicit dict with no AWS mock needed to do so; the explicit-
|
|
73
|
+
and env-credential tests assert the store is not called (`get_secret_value.assert_not_called()` in
|
|
74
|
+
`tests/test_secrets_helper.py`). Fallback-path tests still patch `get_secret` where they exercise the
|
|
75
|
+
store tier.
|
|
76
|
+
- Local dev and CLI run against real services using environment variables with no AWS credentials
|
|
77
|
+
present.
|
|
78
|
+
- Secrets Manager stays the zero-config default in the Lambda runtime: moving from local env creds
|
|
79
|
+
to the store needs no code change, only unsetting the env vars.
|
|
80
|
+
- Explicit-over-remote is a deliberate trust choice. A caller passing a `credentials` dict is
|
|
81
|
+
trusted to own those values in-process; the package neither validates them against the store nor
|
|
82
|
+
falls back when they are wrong.
|
|
83
|
+
|
|
84
|
+
## Load-bearing premises (supersede triggers)
|
|
85
|
+
|
|
86
|
+
- Explicit dict and environment variables stay first-class, supported paths for local dev, CLI, and
|
|
87
|
+
tests. If every runtime became Secrets-Manager-only (no consumer ever supplies explicit or env
|
|
88
|
+
credentials), tiers 1 and 2 are dead weight and the precedence collapses to "always store".
|
|
89
|
+
- The caller owns any credentials it passes explicitly or sets in the environment. The
|
|
90
|
+
explicit/env-beats-remote posture holds only while supplying credentials is an intentional act by
|
|
91
|
+
a trusted in-process caller, not attacker-controlled input. If untrusted input could populate the
|
|
92
|
+
`credentials` dict or the environment, the ordering needs re-evaluation.
|
|
93
|
+
- Secrets Manager remains the right default store for the primary runtime. A move to a different
|
|
94
|
+
managed store changes tier 3's implementation but not the ladder's shape.
|
|
95
|
+
- Env-var **presence** is the trigger, not correctness. A required var being set means "use env"
|
|
96
|
+
(`ES_PASSWORD`; `DB_HOST` + `DB_PASSWORD`; `SLACK_BOT_TOKEN`); a set-but-wrong value fails loudly
|
|
97
|
+
rather than silently falling through to the store. If callers expected fallback-on-bad-env, the
|
|
98
|
+
trigger semantics reopen.
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# ADR-005: Explicit Secrets Manager region resolution
|
|
2
|
+
|
|
3
|
+
**Status**: Proposed
|
|
4
|
+
**Date**: 2026-07-13
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
`get_secret` in `secrets_helper.py` is the single door through which every consumer of this package
|
|
9
|
+
reaches AWS Secrets Manager. It is a shared credential entry point for a fleet of Lambda functions
|
|
10
|
+
and CLI tools that run across multiple AWS accounts and multiple regions.
|
|
11
|
+
|
|
12
|
+
A boto3 Secrets Manager client cannot be built without a region, and where that region comes from
|
|
13
|
+
decides whether a lookup hits the right account's secret store. The region is available in different
|
|
14
|
+
places depending on the runtime: a Lambda execution environment exports `AWS_REGION`; a CLI or a
|
|
15
|
+
differently-wired process gets it from the boto3 session (a configured profile / config file) or
|
|
16
|
+
`AWS_DEFAULT_REGION`; and in some runtimes it is simply absent.
|
|
17
|
+
|
|
18
|
+
Getting the region wrong is **not** a loud failure by default. A client built against the wrong
|
|
19
|
+
region reads a different account/region's secret namespace and typically returns
|
|
20
|
+
`ResourceNotFoundException`, which reads to the caller as "this secret does not exist" rather than
|
|
21
|
+
"you are pointed at the wrong region." A region *default* baked into the helper is the worst version
|
|
22
|
+
of this: it masks a missing config as one specific (usually wrong) region, and does so silently for
|
|
23
|
+
every consumer that never set a region. Commit `342b24b` (PR #26) is the point of record for
|
|
24
|
+
removing such a default from this helper.
|
|
25
|
+
|
|
26
|
+
## Decision
|
|
27
|
+
|
|
28
|
+
`get_secret` derives its region from an **explicit resolution chain, first match wins**, and never
|
|
29
|
+
supplies a region of its own:
|
|
30
|
+
|
|
31
|
+
1. **boto3 session region**, from `boto3.session.Session().region_name` (a configured profile or
|
|
32
|
+
`AWS_DEFAULT_REGION`, which the Lambda runtime also sets). boto3 does not natively resolve
|
|
33
|
+
`AWS_REGION`, so step 2 adds it explicitly for runtimes that set only that variable.
|
|
34
|
+
2. **`AWS_REGION`** environment variable.
|
|
35
|
+
3. **`AWS_DEFAULT_REGION`** environment variable.
|
|
36
|
+
|
|
37
|
+
When none of these resolve, the call **fails loudly before any client is constructed**, raising with
|
|
38
|
+
an explicit message that names the fix
|
|
39
|
+
(`"AWS region not configured for Secrets Manager lookup; set AWS_REGION or AWS_DEFAULT_REGION"`). No
|
|
40
|
+
client is ever built against a guessed region.
|
|
41
|
+
|
|
42
|
+
The helper carries **no hardcoded region**. A missing region is a configuration error surfaced to
|
|
43
|
+
the caller, not a value the package invents. This behaviour is pinned by two tests in
|
|
44
|
+
`tests/test_secrets_helper.py`: `test_get_secret_uses_aws_region_when_session_region_missing`
|
|
45
|
+
(the env-var fallback path builds the client with the configured region) and
|
|
46
|
+
`test_get_secret_requires_region_when_session_and_env_missing` (no session region and empty env
|
|
47
|
+
raises the clear error and asserts no client is created).
|
|
48
|
+
|
|
49
|
+
## Alternatives considered
|
|
50
|
+
|
|
51
|
+
- **A hidden hardcoded region default** (e.g. silently defaulting to `ap-southeast-2` inside the
|
|
52
|
+
helper). This is the cheapest thing to reach for whenever a new AWS call needs a region, so it
|
|
53
|
+
stays a live temptation. Rejected: it binds a store-neutral, geography-neutral package to one
|
|
54
|
+
region, and any consumer running elsewhere silently reads the wrong region's secret namespace and
|
|
55
|
+
gets a `ResourceNotFoundException` that reads as "secret missing," not "wrong region." Removed in
|
|
56
|
+
commit `342b24b` (PR #26).
|
|
57
|
+
- **Best-effort silent fallthrough**: skip the region entirely and let boto3 pick whatever it can,
|
|
58
|
+
or return empty on a miss. Rejected: fail-loud on missing config beats silent best-effort. A
|
|
59
|
+
lookup that cannot name its own region should stop with a clear error, not guess and read from an
|
|
60
|
+
unknown place.
|
|
61
|
+
- **Require region as an explicit function argument.** Rejected: consumers already carry region in
|
|
62
|
+
the standard boto3/session and environment locations, and Lambda sets `AWS_REGION` for free.
|
|
63
|
+
Threading a region parameter through every call site duplicates what the session/env already
|
|
64
|
+
answer.
|
|
65
|
+
|
|
66
|
+
## Consequences
|
|
67
|
+
|
|
68
|
+
- A consumer in **any** AWS account/region works as long as region is configured the standard way
|
|
69
|
+
(Lambda ambient, boto3 profile, or `AWS_REGION` / `AWS_DEFAULT_REGION`). The package assumes no
|
|
70
|
+
home region.
|
|
71
|
+
- Misconfiguration **fails fast with a message that names the fix**, instead of a downstream
|
|
72
|
+
`ResourceNotFoundException` that misattributes the cause to a missing secret and costs an operator
|
|
73
|
+
a debugging session.
|
|
74
|
+
- Every future AWS client added to this package inherits the same rule: **no hidden region default;
|
|
75
|
+
resolve explicitly or fail loud.** This is ADR-002's keep-generic principle applied to region: a
|
|
76
|
+
hidden geo default is exactly the vendor/geography assumption the package forbids (see ADR-002, and
|
|
77
|
+
the "Keeping the Package Generic" design principle in `AGENTS.md`).
|
|
78
|
+
|
|
79
|
+
## Load-bearing premises (supersede triggers)
|
|
80
|
+
|
|
81
|
+
- The package must run in **arbitrary AWS accounts/regions with no assumed home region**. If it were
|
|
82
|
+
ever deliberately scoped to a single fixed region (a single-account, single-region product), a
|
|
83
|
+
region default could be reconsidered.
|
|
84
|
+
- **Fail-loud on missing config is preferred over silent best-effort.** If that preference inverted
|
|
85
|
+
under some "degrade quietly" runtime contract, the explicit raise would reopen.
|
|
86
|
+
- **Region is reachable from the boto3 session or the environment at call time.** If a supported
|
|
87
|
+
runtime cannot expose region through either path, an explicit-argument region parameter would
|
|
88
|
+
reopen.
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# ADR-006: Snowflake access via the pure-Python SQL API client
|
|
2
|
+
|
|
3
|
+
**Status**: Proposed
|
|
4
|
+
**Date**: 2026-07-13
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
This package is a shared utility layer for AWS Lambda functions, CLIs, and async API
|
|
9
|
+
services. Bundle size and pure-Python portability are first-order constraints: every
|
|
10
|
+
integration ships as an optional extra so a consumer pulls only the dependency it uses,
|
|
11
|
+
and the package avoids native/C-extension builds so one wheel runs unchanged across
|
|
12
|
+
Lambda runtimes, layers, and local tooling with no compile step.
|
|
13
|
+
|
|
14
|
+
Snowflake access comes in two dependency shapes. The official
|
|
15
|
+
`snowflake-connector-python` is a large connector with native (C-extension) components:
|
|
16
|
+
Arrow result decoding, pyarrow/cython wheels, a binary footprint that dominates a Lambda
|
|
17
|
+
bundle and pins the build to a specific platform/Python ABI. The Snowflake **SQL REST
|
|
18
|
+
API** needs none of that. It is authenticated HTTP (keypair-minted JWT bearer tokens,
|
|
19
|
+
positional `?` bindings, JSON result sets), fully expressible in pure Python.
|
|
20
|
+
|
|
21
|
+
The consumers here (analytics reads and simple SQL execution from Lambdas and a FastAPI
|
|
22
|
+
service) need the query surface the SQL API already covers, not the connector's native
|
|
23
|
+
result-streaming or stateful-session features.
|
|
24
|
+
|
|
25
|
+
## Decision
|
|
26
|
+
|
|
27
|
+
Snowflake access is provided by wrapping the pure-Python `snowflake-sql-api` package
|
|
28
|
+
(the Snowflake SQL REST API) with a thin NUI adapter, exposed through the optional
|
|
29
|
+
`snowflake` extra (`snowflake-sql-api>=0.1.1,<0.2.0`). The generic package stays
|
|
30
|
+
vendor-neutral; all NUI opinion lives in the adapter
|
|
31
|
+
(`nui_shared_utils/snowflake_client.py`):
|
|
32
|
+
|
|
33
|
+
- **Sync and async client factories.** `create_snowflake_client` (the default) and
|
|
34
|
+
`create_async_snowflake_client` (for a Lambda or FastAPI app already on an event loop)
|
|
35
|
+
return ready `SnowflakeClient` / `AsyncSnowflakeClient` instances. Both run through one
|
|
36
|
+
shared keyword-assembly path, so sync and async stay at feature parity.
|
|
37
|
+
- **Keypair authentication.** Credentials (account, user, PEM private key, optional
|
|
38
|
+
passphrase) resolve per field with strict `explicit-arg > SNOWFLAKE_* env > AWS Secrets
|
|
39
|
+
Manager` precedence; the underlying SQL API client mints short-lived JWT bearer tokens
|
|
40
|
+
from the keypair. Secrets Manager is contacted only for fields still unresolved after
|
|
41
|
+
the explicit/env tiers. This per-field resolution is a deliberate divergence from
|
|
42
|
+
ADR-004's whole-dict ladder (where an explicit `credentials` dict is used as-is and nothing
|
|
43
|
+
else is consulted): a keypair's fields legitimately come from different sources (account and
|
|
44
|
+
user from env, private key from Secrets Manager), so each resolves independently. A field left
|
|
45
|
+
unset at every tier surfaces as a missing-credential error at client construction, not a silent
|
|
46
|
+
partial.
|
|
47
|
+
- **Redacted query logging.** A default `redacting_query_logger` hook records the
|
|
48
|
+
(truncated) statement text and the *count* of bind parameters, never the bind values,
|
|
49
|
+
JWTs, key material, or secret names. Callers can pass a custom `on_query` hook (for
|
|
50
|
+
metrics, for example) or disable logging.
|
|
51
|
+
- **Overridable session defaults.** `TIMEZONE='Pacific/Auckland'` and role `NUI_LAMBDA`
|
|
52
|
+
apply unless overridden. These are baked geography/org defaults, the class ADR-002 (and the
|
|
53
|
+
`AGENTS.md` DON'T list) caution against, and are kept here as a deliberate, bounded exception:
|
|
54
|
+
both are documented and overridable per call, so a consumer in another region or account
|
|
55
|
+
supplies its own at the factory call rather than inheriting a value that silently encodes
|
|
56
|
+
one geography.
|
|
57
|
+
|
|
58
|
+
Import is guarded: using a factory without the extra raises a clear `ImportError` rather
|
|
59
|
+
than failing at package import. Introduced in commit `d0af0b6` (PR #25).
|
|
60
|
+
|
|
61
|
+
## Alternatives considered
|
|
62
|
+
|
|
63
|
+
- **Bundle `snowflake-connector-python`.** The official, fullest-featured client: native
|
|
64
|
+
Arrow result streaming, richer session/cursor semantics, `%s` and named bindings.
|
|
65
|
+
Rejected as the dependency here because its native wheels and binary footprint fight
|
|
66
|
+
both package constraints at once, bundle size (it dominates a Lambda artifact) and
|
|
67
|
+
pure-Python portability (the C-extension build pins platform/ABI and needs a compile
|
|
68
|
+
step). Its extra surface is not what these consumers need.
|
|
69
|
+
- **Vendor the SQL API calls directly** (hand-rolled HTTP client plus JWT minting inside
|
|
70
|
+
this package). Rejected: the generic `snowflake-sql-api` package already owns that
|
|
71
|
+
surface plus an offline test transport (`FakeSnowflake`); duplicating it here couples
|
|
72
|
+
the shared utils to Snowflake's REST contract for no gain.
|
|
73
|
+
- **Ship no Snowflake helper and let each consumer wire its own.** Rejected: credential
|
|
74
|
+
resolution, redaction, and session defaults are exactly the cross-consumer patterns a
|
|
75
|
+
shared layer exists to standardize; re-implementing them per service re-introduces
|
|
76
|
+
drift and redaction gaps.
|
|
77
|
+
|
|
78
|
+
## Consequences
|
|
79
|
+
|
|
80
|
+
- The Snowflake path stays pure-Python and small: the `snowflake` extra adds an HTTP
|
|
81
|
+
client, not a native connector, so it fits Lambda bundles and runs unchanged across
|
|
82
|
+
runtimes.
|
|
83
|
+
- The adapter's reach is bounded by the SQL REST API surface. A concrete limit already
|
|
84
|
+
lives with this: the SQL API accepts only an allow-list of session parameters per
|
|
85
|
+
request and rejects `WEEK_START` with HTTP 400, and being stateless per statement it
|
|
86
|
+
offers no `ALTER SESSION` escape hatch, so Monday-first weeks must be set on the
|
|
87
|
+
Snowflake user/role default instead. Capabilities outside the REST surface (native
|
|
88
|
+
result streaming, stateful session mutation) are unavailable by construction.
|
|
89
|
+
- One credential/redaction/session-default contract across sync and async consumers;
|
|
90
|
+
bind values and key material stay out of logs by default.
|
|
91
|
+
|
|
92
|
+
## Load-bearing premises (supersede triggers)
|
|
93
|
+
|
|
94
|
+
- **The SQL REST API covers the query surface these consumers need.** If a required
|
|
95
|
+
capability were SQL-API-unavailable (native result streaming, a stateful session
|
|
96
|
+
feature the REST contract cannot express), the native connector reopens as the
|
|
97
|
+
dependency for that need.
|
|
98
|
+
- **Pure-Python / small-bundle portability outweighs the native connector's extra
|
|
99
|
+
features.** If this package stopped targeting size-constrained, compile-free runtimes,
|
|
100
|
+
the bundle-size/portability tradeoff that rejects `snowflake-connector-python` no
|
|
101
|
+
longer holds and that choice returns to the table.
|
|
102
|
+
- **The generic `snowflake-sql-api` package remains the SQL API surface.** The adapter is
|
|
103
|
+
a thin layer over it; if that package stopped tracking the REST contract or dropped its
|
|
104
|
+
offline test transport, vendoring the calls (or adopting another SQL-API client)
|
|
105
|
+
returns as an option.
|
|
@@ -24,3 +24,8 @@ Keep each ADR under two pages. Longer than that means it is a plan, not an ADR.
|
|
|
24
24
|
| # | Decision | Status | Date |
|
|
25
25
|
|---|----------|--------|------|
|
|
26
26
|
| [001](ADR-001-structured-logging-field-vocabulary.md) | Structured logging field vocabulary (dotted registry + helpers + consistency test) | Proposed | 2026-07-11 |
|
|
27
|
+
| [002](ADR-002-keep-shared-utilities-generic.md) | Keep shared utilities generic: configurable infrastructure, business queries/names/currencies to the consumer | Proposed | 2026-07-13 |
|
|
28
|
+
| [003](ADR-003-lazy-load-integrations.md) | Lazy-load integrations via PEP 562 so package import stays cheap; heavy deps load on first use | Proposed | 2026-07-13 |
|
|
29
|
+
| [004](ADR-004-credential-resolution-precedence.md) | Credential resolution precedence: explicit dict, then env vars, then Secrets Manager | Proposed | 2026-07-13 |
|
|
30
|
+
| [005](ADR-005-explicit-secrets-manager-region.md) | Resolve the Secrets Manager region explicitly (session, then env); no hidden default, fail loud | Proposed | 2026-07-13 |
|
|
31
|
+
| [006](ADR-006-snowflake-sql-api-client.md) | Snowflake access via the pure-Python SQL API client, not the native connector | Proposed | 2026-07-13 |
|
{nui_python_shared_utils-1.6.0 → nui_python_shared_utils-2.0}/docs/getting-started/installation.md
RENAMED
|
@@ -183,7 +183,7 @@ zip -r nui-utils-layer.zip python/
|
|
|
183
183
|
|
|
184
184
|
# Upload to AWS Lambda Layers (via AWS CLI)
|
|
185
185
|
aws lambda publish-layer-version \
|
|
186
|
-
--layer-name nui-
|
|
186
|
+
--layer-name nui-python-shared-utils \
|
|
187
187
|
--description "NUI Lambda Shared Utilities" \
|
|
188
188
|
--zip-file fileb://nui-utils-layer.zip \
|
|
189
189
|
--compatible-runtimes python3.9 python3.10 python3.11
|