datus-athena 0.1.0__tar.gz

datus_athena-0.1.0/.gitignore

@@ -0,0 +1 @@
+.aws-test/

datus_athena-0.1.0/AWS_TESTING.md

@@ -0,0 +1,290 @@
+# Minimal AWS test environment
+
+Use the local AWS CLI in `us-east-1`. The CloudFormation template creates only
+one S3 bucket, one Glue database and one Athena SQL engine v3 Workgroup. SQL uses
+on-demand scanned-data billing. The scan cutoff is 10 MiB per query. There is no
+capacity reservation, Spark session, Glue ETL job, crawler, customer KMS key or
+database server. S3 requests/storage and Glue catalog usage are billed separately.
+See [Athena pricing](https://aws.amazon.com/athena/pricing/) and
+[scan controls](https://docs.aws.amazon.com/athena/latest/ug/workgroups-setting-control-limits-cloudwatch.html).
+
+The private bucket uses SSE-S3 and blocks public access. Query results expire
+after one day; fixture data expires after seven days. Lifecycle expiration does
+not delete the stack or Glue definitions. Recreate fixtures under a fresh empty
+prefix after expiration. There is no automatic account-wide spending cutoff.
+
+## Create
+
+Run from `datus-athena/` after installing this checkout's packages. The existing
+repository-root virtual environment is referenced as `../.venv`; substitute the
+package-local `.venv` if following the README's installation commands.
+
+```bash
+export AWS_PROFILE=default
+export AWS_REGION=us-east-1
+export AWS_DEFAULT_REGION=us-east-1
+aws sts get-caller-identity
+
+export ATHENA_TEST_RUN_ID="$(date -u +%Y%m%d-%H%M%S)-$(../.venv/bin/python -c 'import secrets; print(secrets.token_hex(2))')"
+export ATHENA_TEST_STACK="datus-athena-test-$ATHENA_TEST_RUN_ID"
+export ATHENA_DATABASE="datus_athena_${ATHENA_TEST_RUN_ID//-/_}"
+
+aws cloudformation create-stack \
+  --stack-name "$ATHENA_TEST_STACK" \
+  --template-body file://scripts/aws-test-environment.yaml \
+  --parameters "ParameterKey=RunId,ParameterValue=$ATHENA_TEST_RUN_ID" \
+    "ParameterKey=DatabaseName,ParameterValue=$ATHENA_DATABASE" \
+  --tags Key=Purpose,Value=datus-athena-acceptance "Key=RunId,Value=$ATHENA_TEST_RUN_ID" \
+  --timeout-in-minutes 10 --on-failure DELETE
+aws cloudformation wait stack-create-complete --stack-name "$ATHENA_TEST_STACK"
+
+export ATHENA_TEST_BUCKET="$(aws cloudformation describe-stacks \
+  --stack-name "$ATHENA_TEST_STACK" \
+  --query "Stacks[0].Outputs[?OutputKey=='BucketName'].OutputValue | [0]" --output text)"
+export ATHENA_WORKGROUP="datus-athena-test-$ATHENA_TEST_RUN_ID"
+export ATHENA_CATALOG=AwsDataCatalog
+export ATHENA_DATA_LOCATION="s3://$ATHENA_TEST_BUCKET/fixtures/"
+export ATHENA_EXPECT_MANAGED_RESULTS=false
+unset ATHENA_OUTPUT_LOCATION
+
+../.venv/bin/python scripts/init_tpch_data.py
+../.venv/bin/python -m pytest tests/integration -m integration -q
+```
+
+The Workgroup initially allows CTAS `external_location`; the fixture initializer
+needs that setting and an empty S3 prefix. Do not initialize under an enforced
+result location ([AWS CTAS restrictions](https://docs.aws.amazon.com/athena/latest/ug/create-table-as.html)).
+The initializer creates the five tiny shared TPC-H tables and one view, without
+copying production data. No credentials are written to repository files.
+
+## Reuse the same Workgroup for result-mode acceptance
+
+Run these steps sequentially, with no other test run using the Workgroup.
+Enforce its S3 result location after fixture initialization:
+
+```bash
+aws athena update-work-group --work-group "$ATHENA_WORKGROUP" \
+  --configuration-updates '{"EnforceWorkGroupConfiguration":true}'
+ATHENA_EXPECT_MANAGED_RESULTS=false \
+  ../.venv/bin/python -m pytest tests/integration -m integration -q
+```
+
+Switch to managed results, using the default AWS owned key:
+
+```bash
+aws athena update-work-group --work-group "$ATHENA_WORKGROUP" \
+  --configuration-updates '{"EnforceWorkGroupConfiguration":true,"ResultConfigurationUpdates":{"RemoveOutputLocation":true,"RemoveEncryptionConfiguration":true},"ManagedQueryResultsConfigurationUpdates":{"Enabled":true}}'
+ATHENA_EXPECT_MANAGED_RESULTS=true \
+  ../.venv/bin/python -m pytest tests/integration -m integration -q
+```
+
+Restore the template's S3 configuration after acceptance. This keeps the same
+minimal resources and permits future fixture initialization:
+
+```bash
+aws athena update-work-group --work-group "$ATHENA_WORKGROUP" \
+  --configuration-updates "{\"EnforceWorkGroupConfiguration\":false,\"ManagedQueryResultsConfigurationUpdates\":{\"Enabled\":false},\"ResultConfigurationUpdates\":{\"OutputLocation\":\"s3://$ATHENA_TEST_BUCKET/query-results/\",\"EncryptionConfiguration\":{\"EncryptionOption\":\"SSE_S3\"}}}"
+export ATHENA_EXPECT_MANAGED_RESULTS=false
+```
+
+Managed query results are available for 24 hours. Source fixture data still lives
+in the test bucket. See [managed results](https://docs.aws.amazon.com/athena/latest/ug/managed-results.html).
+Optional denied-identity and KMS/Lake Formation cases require their separately
+documented environment variables; this minimal stack does not create IAM roles,
+KMS keys or Lake Formation grants. Their skips do not mean those behaviors passed.
+
+To include creation, insertion, metadata, typed retrieval and cleanup through
+the shared `TableContractCase`, set
+`ATHENA_WRITABLE_DATA_LOCATION="s3://$ATHENA_TEST_BUCKET/contracts/"`.
+Grant scoped Glue table creation/deletion and S3 list/read/write/delete access.
+The test creates one UUID-named external Parquet table and removes both its
+definition and its private child prefix, including files written before a
+failed assertion. Missing this optional variable skips only the writable case;
+service or cleanup errors after configuration fail the test. It uses explicit
+CREATE EXTERNAL TABLE and INSERT rather than CTAS, so it does not require
+disabling Workgroup-enforced result settings.
+
+For extended acceptance, provide isolated test resources and SDK profiles. Do not
+change account-wide Lake Formation defaults or apply policies to existing users:
+
+- `ATHENA_DENIED_PROFILE`: a valid AssumeRole profile with discovery permissions
+  and an explicit `athena:StartQueryExecution` deny on the test Workgroup.
+- `ATHENA_ROLE_PROFILE`: an allowed reader profile with `duration_seconds=900`.
+  Set `ATHENA_RUN_ROLE_EXPIRY_TEST=1` for the 15-minute real expiry case. The test
+  keeps its client/Session, proves the old token is rejected by STS, and verifies
+  that the provider renews credentials for a successful subsequent query.
+- `ATHENA_RESTRICTED_PROFILE`: the same query/data scope without the protected
+  fixtures' KMS decrypt and Lake Formation SELECT grants.
+- `ATHENA_KMS_SQL` and `ATHENA_LAKEFORMATION_SQL`: each must select `r_regionkey`
+  from a dedicated copy of `tpch_region`, ordered and limited to row 0. The allowed
+  role must return `[{"r_regionkey": 0}]`; the restricted role must be rejected.
+  Use a customer KMS key only for the KMS fixture. Register only the LF fixture
+  prefix; revoke IAMAllowedPrincipals on that table, grant SELECT to the allowed
+  role and only DESCRIBE to the restricted role. LF can hide inaccessible columns
+  and report a column authorization error rather than naming Lake Formation.
+- `ATHENA_EXPECT_FAILURE_SQL`: an additional denied-data query. Set
+  `ATHENA_FAILURE_PROFILE` when it needs a different profile from the main suite.
+- `ATHENA_RUN_NETWORK_FAULT_TESTS=1`: requires OpenSSL. A loopback proxy generates
+  a temporary trusted certificate for the Athena hostname, forwards signed HTTPS
+  requests to real AWS, and injects one HTTP throttle, a lost submission response,
+  or socket read timeouts. Upstream TLS verification stays enabled. These are
+  controlled transport faults; they do not induce a real AWS service quota breach.
+
+Run short cases separately from the real-expiry case, using a private
+`AWS_CONFIG_FILE` rather than editing the workstation's shared config:
+
+```bash
+ATHENA_RUN_NETWORK_FAULT_TESTS=1 ../.venv/bin/python -m pytest tests/integration \
+  -m integration -k 'not real_sts_expiry' -q
+ATHENA_RUN_ROLE_EXPIRY_TEST=1 ../.venv/bin/python -m pytest \
+  tests/integration/test_aws_authentication.py::test_cached_client_refreshes_after_real_sts_expiry -q
+```
+
+After extended testing, revoke the fixture's LF grants, deregister its prefix,
+delete the two copied tables/objects and the disposable roles, and schedule the
+customer KMS key for deletion with the minimum seven-day waiting period. Run the
+full cleanup below after acceptance is complete. A pending-deletion key cannot be used;
+see [KMS key deletion](https://docs.aws.amazon.com/kms/latest/developerguide/deleting-keys.html).
+
+## Existing local environment
+
+For an environment prepared in this checkout, source `.aws-test/env.sh` instead
+of creating another stack. The ignored `.aws-test/` directory records resource
+names, CLI requests, query IDs/statistics and test reports, without AWS credentials.
+Check stack status and fixture age before reusing it.
+The environment recorded on 2026-10-16 has since been fully decommissioned;
+its local `env.sh` is disabled. Recreate the stack before future cloud tests.
+
+```bash
+source .aws-test/env.sh
+aws cloudformation describe-stacks --stack-name "$ATHENA_TEST_STACK" \
+  --query 'Stacks[0].StackStatus' --output text
+../.venv/bin/python -m pytest tests/integration -m integration -q
+```
+
+## Cleanup
+
+Run only for the dedicated environment above. First remove fixture definitions,
+then verify the bucket belongs to this stack before emptying it. The template
+has no bucket versioning. Deleting a nonempty bucket through CloudFormation fails.
+
+```bash
+../.venv/bin/python scripts/init_tpch_data.py --drop
+owned_bucket="$(aws cloudformation describe-stack-resource \
+  --stack-name "$ATHENA_TEST_STACK" --logical-resource-id TestBucket \
+  --query 'StackResourceDetail.PhysicalResourceId' --output text)"
+test -n "$ATHENA_TEST_BUCKET" && test "$owned_bucket" = "$ATHENA_TEST_BUCKET" && \
+  aws s3 rm "s3://$ATHENA_TEST_BUCKET/" --recursive --only-show-errors && \
+  aws cloudformation delete-stack --stack-name "$ATHENA_TEST_STACK" && \
+  aws cloudformation wait stack-delete-complete --stack-name "$ATHENA_TEST_STACK"
+```
+
+## Validation record: 2026-10-16
+
+This checkout used the local CLI `default` identity in `us-east-1`, Python
+3.12.10, boto3/botocore 1.43.114, and Athena SQL engine v3. The stack reached
+`CREATE_COMPLETE` with exactly the three declared resources. Fixture initialization
+created five Parquet objects totaling 5,373 bytes and one Glue view. The Workgroup
+was restored to non-enforced S3 output after testing; the 10 MiB cutoff remains.
+
+| Check | Result |
+| --- | --- |
+| Unit tests | 111 passed; 98% source coverage |
+| Enforced S3 results, extended suite | 28 passed, zero skipped, 195.17 seconds; real-expiry case deselected and run separately |
+| Managed results, same Workgroup | 28 passed, zero skipped, 203.32 seconds; real-expiry case run separately |
+| Actual STS expiry and renewal | 1 passed, 929.80 seconds; AWS rejected the expired static token, and the same client/Session renewed credentials for a successful query |
+| IAM denial | Valid denied role could discover the catalog; StartQueryExecution was denied before a query ID was created |
+| Source KMS and Lake Formation | Allowed/restricted roles queried the same copied fixtures; successful controls proved table/column existence before denial assertions |
+| Wait/result deadlines and budgets | Actual AWS wait cancellation, result read deadline, row/byte truncation metadata passed |
+| Transport faults | Controlled HTTP 429, a lost submission response, and socket read timeouts; retry/recovery passed against real AWS |
+| Native idempotency | Retried HTTP submissions retained one token and one actual QueryExecutionId |
+| Formatting and lint | Ruff lint/format and whitespace checks passed |
+
+The extended matrix used four temporary scoped IAM roles, one temporary customer
+KMS key, two copied fixture tables/objects and one LF-registered prefix. Account
+Lake Formation defaults and existing IAM user policies were not changed. The LF
+table was verified as registered; its restricted identity had DESCRIBE without
+SELECT, so AWS hid inaccessible columns and returned an authorization diagnostic.
+The KMS object was verified as SSE-KMS using the test key; only the allowed role
+had decrypt permission. No stored access keys or global AWS config changes were
+required. Test profiles and reports stayed in the ignored `.aws-test/` directory.
+
+Cleanup removed all four temporary roles, both protected tables/objects and the
+LF prefix registration. The test KMS key is `PendingDeletion` with a seven-day
+waiting period. A subsequent full teardown on 2026-10-16 reached
+`DELETE_COMPLETE`: the original bucket (328 objects), database and Workgroup
+were deleted. Account LF settings were verified unchanged. The final query audit found no
+active queries and confirmed that the client output override was ignored.
+These zero-skip results used the temporary extended environment. Future cloud
+runs require the environment and any optional permission/role profiles to be
+recreated. The writable table contract added on 2026-10-19 is not included in
+the historical counts above; its separate live validation is recorded below.
+
+The network tests inject faults at a local TLS proxy while forwarding other
+signed requests to real AWS with normal upstream TLS verification. They do not
+generate AWS service quota pressure or reproduce a regional AWS outage. Socket
+timeouts may cause valid HTTP submission retries; assertions check one logical
+token/query ID rather than requiring exactly one physical HTTP request.
+
+Reported scanned bytes are not the billing quantity: billing can apply per-query
+minimums, and S3 requests/storage and catalog usage are separate. Source-data
+SSE-KMS allow/deny and AWS-owned managed results were tested; customer-KMS managed
+result encryption, cross-account trust, SSO and web identity are outside this run.
+The cloud CI target remains disabled until CI credentials and its workflow are
+assigned; the local default identity is not copied to CI.
+
+Real connection setup exposed an assumption in `test_connection()`: the service
+listed `AwsDataCatalog` as `GLUE`, while `GetDataCatalog` returned not found.
+Connection validation now reuses paginated catalog discovery for type/access
+checks, then verifies database metadata and an actual SELECT. It does not add
+catalog-name exceptions or suppress metadata errors. The existing connection-test
+category caught this during live acceptance; offline regression covers discovery,
+missing/unsupported catalogs and the actual-query requirement.
+
+## Additional EXPLAIN validation: 2026-10-19
+
+The standalone `test_explain_returns_query_plan` case ran twice against actual
+Athena SQL engine v3 in `us-east-1` with managed results: one passed case per run
+(3.84 and 3.97 seconds), zero skips, and `DataScannedInBytes=0` for each query.
+The test uses `EXPLAIN SELECT 1 AS datus_explain_value` and verifies a successful,
+nonempty plan containing the output alias. It requires only region/Workgroup
+configuration, without a source database or prepared data.
+
+The captured service response reported `StatementType=DML`, with `Query Plan`
+as the first API row and `Fragment 0 [SINGLE]` as the first plan row. The connector
+correctly removes the DML header and retains the plan. The unit regression now
+uses this observed protocol rather than fabricating a headerless UTILITY EXPLAIN.
+
+Each run created one uniquely tagged temporary Workgroup with managed results
+and a 10 MiB scan cutoff, then disabled/deleted it and verified it was absent.
+No S3 buckets, Glue databases, capacity reservations or servers were created.
+These extra EXPLAIN checks are separate from the historical 28-case result-mode
+counts.
+
+See [AWS EXPLAIN behavior and scan considerations](https://docs.aws.amazon.com/athena/latest/ug/athena-explain-statement.html).
+
+## Additional writable validation: 2026-10-19
+
+A fresh stack in `us-east-1` created exactly one private SSE-S3 bucket, one Glue
+database and one Athena SQL engine v3 Workgroup with a 10 MiB scan cutoff. No
+TPC-H initialization or additional IAM/KMS resources were needed. The optional
+writable table contract and EXPLAIN ran against this environment in both result
+modes with Workgroup enforcement enabled:
+
+| Result mode | Result |
+| --- | --- |
+| S3 results | 2 passed, zero skipped, 25.41 seconds |
+| Managed results | 2 passed, zero skipped, 24.38 seconds |
+
+The shared table contract created physical `Mixed Case` and `special-name`
+columns, inserted both typed rows, verified Glue schema plus qualified SELECT
+and LIMIT results, and dropped its table. After each run, Glue listed no tables
+and the `contracts/` prefix contained no objects. This validates the actual Hive
+DDL and Trino INSERT/SELECT identifiers rather than result aliases alone.
+
+All 16 queries succeeded and reported 1,006 scanned bytes in total; per-query
+billing minimums and S3/catalog requests apply separately. Teardown removed the
+13 remaining query-result objects and the stack reached `DELETE_COMPLETE`.
+Subsequent AWS API checks verified the bucket, Glue database and Workgroup were
+absent. These four passed cases are separate from the historical 28-case suites
+and the earlier standalone EXPLAIN checks.

datus_athena-0.1.0/PKG-INFO

@@ -0,0 +1,266 @@
+Metadata-Version: 2.5
+Name: datus-athena
+Version: 0.1.0
+Summary: AWS Athena database adapter for Datus using the official boto3 SDK
+Project-URL: Repository, https://github.com/Datus-ai/datus-db-adapters
+Project-URL: Issues, https://github.com/Datus-ai/datus-db-adapters/issues
+Author-email: DatusAI <support@datus.ai>
+License: Apache-2.0
+Requires-Python: >=3.12
+Requires-Dist: boto3<2,>=1.43.91
+Requires-Dist: datus-db-core>=0.1.10
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: pyarrow>=18.0.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: sqlglot>=27.27.0
+Provides-Extra: test
+Requires-Dist: pytest-cov>=4; extra == 'test'
+Requires-Dist: pytest>=8; extra == 'test'
+Description-Content-Type: text/markdown
+
+# datus-athena
+
+Amazon Athena adapter using the official boto3 SDK. Python 3.12 or later.
+
+```bash
+pip install datus-athena
+```
+
+## Configuration
+
+```python
+from datus_athena import AthenaConfig, AthenaConnector
+
+with AthenaConnector(AthenaConfig(
+    region="us-east-1",
+    workgroup="datus-test",
+    catalog="AwsDataCatalog",
+    database="analytics",
+)) as connector:
+    result = connector.execute_query("SELECT '数据' AS message", "list")
+    assert result.success, result.error
+    print(result.sql_return, result.query_execution_id, result.execution_info)
+```
+
+Datus Agent discovers the `datus.adapters` entry point. Adapter fields pass
+through `DbConfig.extra`; no Agent database-name branch is needed:
+
+```yaml
+services:
+  datasources:
+    lake:
+      type: athena
+      aws:
+        region: us-east-1
+        # profile: datus-role   # optional standard AWS role profile
+      workgroup: datus-test
+      catalog: AwsDataCatalog
+      database: analytics
+      query_timeout_seconds: 600
+      max_rows: 10000
+```
+
+The namespace is `catalog.database.table`. There is no additional Datus schema
+level. Per-call `catalog_name` and `database_name` override defaults without
+changing another thread/task's context. SELECT identifiers use double quotes;
+Hive table DDL uses backticks. The registered SQLGlot dialect is `athena`.
+
+| Setting | Default | Meaning |
+| --- | --- | --- |
+| `region` | SDK configuration | AWS region |
+| `profile_name` | SDK default chain | Named profile, including AssumeRole profiles |
+| `ca_bundle` | SDK trust configuration | Optional trusted CA bundle; verification stays enabled |
+| `workgroup` | `primary` | Athena SQL Workgroup |
+| `catalog` | `AwsDataCatalog` | AWS Glue data catalog |
+| `database` | unset | Default database |
+| `output_location` | unset | Optional `s3://bucket/prefix/` for query results |
+| `timeout_seconds` | 30 | Each SDK connection/socket read timeout |
+| `query_timeout_seconds` | 600 | Foreground query wait deadline |
+| `result_timeout_seconds` | 60 | Result pagination deadline |
+| `cancel_timeout_seconds` | 10 | Wait after native cancellation; zero requests only |
+| `poll_interval_seconds` | 1 | State polling interval |
+| `max_rows` | 10000 | Maximum returned rows |
+| `max_result_bytes` | 16777216 | Maximum retained UTF-8 cell bytes before type conversion |
+| `page_size` | 1000 | Result API rows per response, including any header |
+
+## Agent SQL skill
+
+The adapter packages [`db-athena-sql`](datus_athena/skills/db-athena-sql/SKILL.md)
+through the `datus.skills` entry point. Datus Agent 0.4.4 loads it for a bound
+Athena connector. It explains Athena's Trino/Hive syntax split, names and types,
+numeric and time semantics, nested data, partition pruning and Workgroup limits.
+[Table design and writes](datus_athena/skills/db-athena-sql/references/table-design-and-writes.md)
+is a task-specific reference for S3 DDL, CTAS, INSERT and Iceberg operations.
+The SQL-generation notes hook reads the same skill body so the two paths share
+one set of maintained rules.
+
+## AWS identity and permissions
+
+Credentials use the SDK default provider chain (environment, shared profiles,
+SSO, web identity, container or instance role). No access keys or short-lived
+tokens are stored in this adapter's configuration. Explicit temporary credentials
+in environment variables expire when their upstream issuer says they expire;
+use a refreshable provider for long-lived processes.
+
+For cross-account access, configure a standard AWS profile and set
+`profile_name="datus-role"`:
+
+```ini
+# ~/.aws/config; the source profile's credentials remain outside agent.yml
+[profile datus-role]
+role_arn = arn:aws:iam::123456789012:role/DatusAthenaReader
+source_profile = development
+region = us-east-1
+```
+
+Boto3 owns AssumeRole caching and refresh. This package uses the shared
+`datus-db-core>=0.1.8` `AWSIdentity` for Session/client ownership, timeouts, retries
+and cleanup. The AWS connection fields are `region`, `profile_name`, and
+`ca_bundle`. The shared CORE-01 shape
+`aws: {region: ..., profile: ..., ca_bundle: ...}` is also accepted and normalized
+to these fields; conflicting nested/flat values fail configuration validation.
+An existing `boto3.Session` can also be passed as `session=`.
+
+Grant the test identity the scoped Athena actions used here:
+`GetWorkGroup`, `ListDataCatalogs`, `ListDatabases`,
+`GetDatabase`, `ListTableMetadata`, `GetTableMetadata`, `StartQueryExecution`,
+`GetQueryExecution`, `GetQueryResults`, and `StopQueryExecution`.
+Also grant the required Glue database/table access and source S3 reads. S3
+results require access to the result prefix, including `s3:GetObject` and query
+result writes. Encrypted data/results require the corresponding KMS permissions;
+Lake Formation protected data requires its grants as well as IAM access.
+Managed results use Workgroup-scoped Athena result permissions and the managed
+results KMS key policy when configured. Metadata access alone does not prove
+query/data access: `test_connection()` performs both checks and a small SELECT.
+
+## Query behavior and support boundary
+
+Each submission reads the Workgroup configuration. Managed results and enforced
+Workgroup settings take precedence: the adapter omits client output overrides.
+Otherwise `output_location` is passed only when explicitly supplied. A Workgroup
+with its own S3 result location needs no client output location. Both result
+modes use `GetQueryResults`; the adapter does not download complete S3 files.
+
+Queries return an `AthenaExecuteSQLResult`, a subclass of `ExecuteSQLResult`.
+`query_execution_id` identifies the native query; `execution_info` includes state,
+Athena error details, engine version, scan/timing statistics and truncation.
+List results preserve NULL, Decimal, dates, timestamps and Chinese text. Complex
+types, timezone timestamps, and high-precision timestamp signatures stay as text;
+no lossy complex-type parsing is attempted. A plain `timestamp` value with more
+than six fractional digits fails with a `CAST(... AS VARCHAR)` directive rather
+than silently losing precision. Duplicate column names require aliases
+for list output; positional Arrow/pandas/CSV results retain them.
+
+Result row/byte budgets stop pagination and set `truncated` and
+`truncation_reason`. The byte budget counts retained cell text, not HTTP response
+bytes or Python object overhead; one SDK response is still received in full.
+Query and pagination deadlines are checked between SDK calls; socket timeouts
+and up to three SDK attempts can extend wall-clock completion beyond a deadline.
+Neither a result cap nor SQL LIMIT is a scan-cost guarantee. Set Workgroup
+per-query scanned-byte controls and service-side limits before running queries.
+
+On wait timeout or polling failure, the adapter requests `StopQueryExecution`,
+then polls for a bounded period. `cancellation_requested` and
+`cancellation_confirmed` are separate; a running query after cancellation wait
+is reported with its last state and ID. Failed queries are not resubmitted even
+when Athena marks them retryable. One UUID token is generated per logical
+execution and reused by SDK request retries. `execute_query(...,
+client_request_token=...)` permits explicit retry identity; the caller must reuse
+the same SQL/context with a token. Cancellation is also available through
+`cancel_query(query_execution_id)`.
+
+Glue Catalog queries, tables, views, partition columns, SHOW CREATE metadata and
+bounded sampling are implemented. DDL and DML are sent unchanged to Athena;
+availability depends on table format and catalog permissions (UPDATE/DELETE/MERGE
+require supported Iceberg tables). Generic dataframe migration is explicitly
+unsupported. Transactions, session SET/USE, Spark Workgroups, federation-specific
+features and durable cross-process task recovery are outside this release.
+
+## Testing
+
+Install from this checkout in a dedicated environment:
+
+```bash
+uv venv --python 3.12 .venv
+uv pip install --python .venv/bin/python -e ../datus-db-core -e '.[test]'
+.venv/bin/python -m pytest tests/unit -m 'not integration'
+.venv/bin/python -m pytest tests/integration -m integration
+```
+
+Unit tests use real botocore `Stubber` responses, never AWS network access.
+The cloud target remains disabled until CI-specific credentials and its workflow
+are configured. No paid LocalStack dependency is required.
+
+The EXPLAIN smoke case only needs `AWS_REGION` (or `AWS_DEFAULT_REGION`) and
+`ATHENA_WORKGROUP`. It can use managed results without a source database or S3
+fixtures:
+
+```bash
+env -u ATHENA_DATABASE .venv/bin/python -m pytest \
+  tests/integration/test_sql_execution.py::test_explain_returns_query_plan -q
+```
+
+The prepared-fixture suite supports read-only source permissions. Set
+`ATHENA_WRITABLE_DATA_LOCATION=s3://test-bucket/contracts/` to additionally run
+the shared writable table contract. This requires scoped Glue table creation,
+insertion and deletion, plus S3 list/read/write/delete access. The test creates
+one UUID-named Parquet table under a new child prefix and cleans up its metadata
+and data. The supplied prefix must be dedicated to disposable test data.
+The writable contract creates physical `Mixed Case` and `special-name` columns,
+using backticks in Hive DDL and double quotes in INSERT/SELECT. Metadata must
+report those names; the read contract also checks quoted result aliases.
+
+For real AWS validation, provision a dedicated SQL Workgroup with a per-query
+scan cutoff and either an S3 result prefix or managed results, a disposable Glue
+database, and a dedicated S3 data prefix. Set `AWS_REGION`, `ATHENA_WORKGROUP`,
+`ATHENA_DATABASE`, `ATHENA_DATA_LOCATION`, and optionally `AWS_PROFILE` and
+`ATHENA_OUTPUT_LOCATION`. `ATHENA_DATA_LOCATION` must be a dedicated empty
+`s3://bucket/prefix/`; do not reuse application data locations.
+
+For the smallest local-CLI environment in `us-east-1`, use the three-resource
+[CloudFormation setup and cleanup guide](AWS_TESTING.md). It reuses one Workgroup
+for S3 and managed-results acceptance, with a 10 MiB per-query scan cutoff.
+
+```bash
+# Creates five tiny Parquet TPC-H tables and a view from shared fixtures. Run once.
+.venv/bin/python scripts/init_tpch_data.py
+# Cloud tests query the prepared fixtures; no fixture initialization is skipped on error.
+.venv/bin/python -m pytest tests/integration -m integration
+# Removes only fixture table definitions. S3 objects remain for operator-controlled cleanup.
+.venv/bin/python scripts/init_tpch_data.py --drop
+```
+
+Run the suite separately against S3-results and managed-results Workgroups.
+Set `ATHENA_EXPECT_MANAGED_RESULTS=true` or `false` to assert the storage mode.
+CTAS fixture initialization uses explicit data locations: prepare fixtures with
+a Workgroup that does not enforce its result location, then run read-only
+acceptance with enforced Workgroups against the same database. Athena rejects
+CTAS `external_location` under an enforced result location
+([AWS CTAS restrictions](https://docs.aws.amazon.com/athena/latest/ug/create-table-as.html)).
+Without required environment variables, cloud fixtures skip with their names;
+with those variables present, failed authentication/readiness/queries fail tests.
+The environment owner must remove the dedicated S3 fixture/output prefix and
+database after acceptance; the script does not delete buckets or arbitrary data.
+
+Local AWS acceptance in `us-east-1` on 2026-10-16 passed with SQL engine v3:
+28 short integration tests passed in enforced S3-results mode and 28 in
+managed-results mode, with zero skipped. The actual STS-expiry case separately
+passed after 15 minutes 29 seconds. These include scoped IAM denial, source KMS
+and Lake Formation allow/deny controls, native cancellation, result/wait deadlines,
+result budgets and controlled loopback TLS transport faults with real AWS queries.
+See [environment setup, limits and validation](AWS_TESTING.md). CI credentials
+and its workflow remain unassigned; the local identity is not copied to CI.
+On 2026-10-19 the EXPLAIN smoke case also passed against SQL engine v3 with
+managed results, reporting zero scanned bytes. Its temporary Workgroups were
+deleted immediately after validation. The observed DML/header result shape is
+recorded in [AWS_TESTING.md](AWS_TESTING.md#additional-explain-validation-2026-10-19).
+On the same date, a fresh minimal environment passed the writable table contract
+and EXPLAIN in both enforced S3 and managed-results modes (two passed cases per
+mode, zero skips). Physical quoted columns, typed inserts, schema and cleanup
+were verified against AWS, then the bucket, database and Workgroup were deleted.
+These checks are separate from the historical counts above; see the
+[additional writable validation record](AWS_TESTING.md#additional-writable-validation-2026-10-19).
+
+References: [Athena API](https://docs.aws.amazon.com/boto3/latest/reference/services/athena.html),
+[SDK credential providers](https://docs.aws.amazon.com/boto3/latest/guide/credentials.html),
+[managed results](https://docs.aws.amazon.com/athena/latest/ug/managed-results.html).