local-serverless-stack 0.8.0 → 0.10.0

package/CHANGELOG.md

@@ -5,6 +5,61 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.10.0] - 2026-07-13
+
+DynamoDB transactions land on the self engine (`TransactWriteItems`/`TransactGetItems` with faithful `CancellationReasons`), and per-invocation Lambda log capture now works for structured loggers (pino/winston/bunyan) that write straight to the stream or file descriptor instead of `console.*`. Also: dashboard header split into a brand row + navigation row with the configured region pre-selected; the registered-services cache is now scoped per project — fixing orchestrators of one example silently binding (and stealing) another example's gateway ports; gateways recover automatically from port conflicts; each example runs in its own AWS region; `examples/localstack-ultimate` now exercises EventBridge, schedules and a DLQ, with its own VS Code task palette.
+
+### Added
+- **DynamoDB transactions on the self engine**: `TransactWriteItems` supports Put/Update/Delete/ConditionCheck with all-or-nothing semantics — every condition is evaluated against current state first, and any failure cancels the request with a `TransactionCanceledException` carrying per-item `CancellationReasons` (`Code`/`Message`, plus the old `Item` when `ReturnValuesOnConditionCheckFailure: ALL_OLD`), the shape clients inspect to map "condition failed on entry N" to domain errors (e.g. duplicate e-mail → 409 instead of an opaque 500). Committed writes emit stream records (so `stream`-triggered functions and the events relay see transactional writes); cancelled transactions emit none. Update computations are dry-run in the same pre-commit pass — a failing Update (type mismatch, invalid document path, post-update item over 400 KB) cancels the transaction with a `ValidationError` cancellation reason instead of half-applying it, while statically invalid updates (syntax errors, expressions touching a key attribute) reject the request up front. Duplicate targets ("multiple operations on one item"), the 100-item cap and per-member required fields are validated with AWS-worded errors, and `ClientRequestToken` gets real idempotency, namespaced per region: successful transactions replay from a 10-minute cache, token reuse with different parameters answers `IdempotentParameterMismatchException`, an overlapping in-flight attempt answers `TransactionInProgressException`, and cancelled transactions are not cached (a retry re-executes). `TransactGetItems` (ordered `Responses` read as one consistent snapshot, `{}` placeholders for misses, `ProjectionExpression` support, duplicate-item rejection, `ConsumedCapacity`) comes along. Previously any handler using `TransactWriteCommand` — signup flows enforcing e-mail uniqueness, refresh-token rotation — hit `NotImplemented` on the self engine, which generic HTTP error middlewares surface as a 500 with no logs to explain it.
+- **Lambda log capture at the stream/fd level**: the runtime worker used to patch `console.*` only, so structured loggers that write JSON straight to the output stream or to file descriptor 1/2 — pino via sonic-boom (`fs.write(1, …)`), winston/bunyan via `process.stdout.write` — produced invocations with `"logs": []` in the dashboard and no output anywhere. Capture now intercepts `process.stdout/stderr.write` and `fs.write/writeSync/writev/writevSync` on fds 1/2 (new `src/server/runtime/log-capture.ts`), synchronously at the call site so `AsyncLocalStorage` still attributes each line to the invocation that emitted it; `console.*` keeps its `INFO`/`WARN`/`ERROR` level tags, partial lines are reassembled across chunks and flushed with the result, and cold-start module output attributes to the only in-flight invocation when unambiguous. Covered end to end by a test that forks the real worker and asserts pino-style fd writes land in the invocation record.
+- **Worker stdout/stderr surfaces as service-level output**: unattributable worker output (logger transports on worker threads, child processes, writes after an invocation completed) used to be silently discarded; it now reaches the orchestrator log prefixed `[<service>:worker]`.
+- **HTTP status on invocation records**: an invocation whose handler resolved an API-Gateway-shaped payload records its `statusCode`, and the dashboard's Recent invocations list badges 4xx (warning) and 5xx (danger) — a handler answering `{"error":"internal_error"}` with a 500 no longer reads as a healthy green invocation.
+- **`examples/localstack-ultimate` expanded** to showcase the full LSS resource menu on the Pro path: an `AWS::Events::EventBus` (`ultimate-order-events`) receiving `order.processed` events from the SQS consumer, a new `auditEvents` function (eventBridge pattern trigger) persisting every matched event into a new `AuditTrail` table readable at `GET /audit`, a `purgeSessions` function on `schedule: rate(2 minutes)` that deletes expired sessions and emits `sessions.purged`, and an `OrderProcessingDLQ` with a `RedrivePolicy` (`maxReceiveCount: 3`) on the order queue — `processOrderQueue` now reports poison orders (missing `userId`, `total <= 0`) via `ReportBatchItemFailures` so they land in the DLQ and show up in the dashboard's Queues tab. Since LSS's provisioner does not yet apply `RedrivePolicy` from CloudFormation, `scripts/wire-dlq.js` (hooked into `npm run offline`) sets it on the live queue; native support is tracked as a follow-up. New `seeds/localstack-ultimate-Sessions.json` feeds the purge demo; README and the request-driver console gained flows for audit trail, poison order → DLQ and schedule observation.
+- **VS Code tasks for `examples/localstack-ultimate`** (`.vscode/tasks.json`): setup, start (Docker + `LOCALSTACK_AUTH_TOKEN`), offline, seed, drive demo, drive catalog (OpenSearch) and stop — same style as the existing example task sections, which previously covered only `self-hosted` and `localstack-free`.
+- **OpenSearch Serverless on LocalStack engines — the aoss sidecar**: no LocalStack edition emulates `aoss` (localstack/localstack#8707 is still backlog), so the orchestrator now serves the self engine's OpenSearch emulator **in-process** on a dedicated port (default `14567`) whenever the active engine is LocalStack. `AWS::OpenSearchServerless::Collection` resources provision into it transparently (`SecurityPolicy`/`AccessPolicy` stay accepted-and-skipped), the dashboard's resources/collections views work unchanged, and `GET /api/health` gains an `aossSidecar {enabled, running, port}` block. New optional config block `aossSidecar: { enabled?, port? }` (`enabled: false` opts out; never runs on the self engine, which serves aoss natively); data persists under `<stateDir>/aoss` (else `~/.lss/aoss`). Implementation reuses the engine store, OpenSearch emulator and AWS-wire router as-is (`src/server/engine/aoss-sidecar.ts`); documented in `docs/CONFIGURATION.md`. Verified end to end on the Pro image: collection provisioned on registration, full-text search/filters/aggregations served from the sidecar, `enabled/running` in health.
+- **OpenSearch tab in the dashboard**: the nav gains an OpenSearch section alongside S3/DynamoDB — a collections list (owning service, data-plane endpoint) with a per-collection browser: indices with doc counts/health, plus a search panel (free-text `q`, index scope, page size) rendering hits with their `_source`. Backed by new `GET /api/opensearch/collections`, `GET /api/opensearch/collections/:name/indices` and `POST /api/opensearch/collections/:name/search` routes and an `OpenSearchExplorer` service that targets the aoss sidecar or the self engine via the same endpoint-selection rule as the provisioner; the region selector is honored on the data plane by pinning the region through the SigV4 credential-scope query parameter (the local engines parse it without verifying signatures).
+- **`examples/localstack-ultimate` gains an OpenSearch Serverless catalog** (`ultimate-products` collection): `POST/GET/DELETE /products`, `GET /search?q=&category=&maxPrice=` and `GET /stats` routes served through serverless-offline, with fetch-based handlers (no SDK dependency), a console flow, a `drive catalog (OpenSearch)` VS Code task, README/ports documentation, and `stateDir: ".lss"` so orchestrator state (pid/log/aoss data) stays project-local like the other examples.
+
+### Changed
+- **Dashboard header split into two rows**: a brand bar (logo + title + subtitle on the left; engine status, region selector and theme menu on the right) over a dedicated navigation bar with the section tabs — the previous single row collided at moderate widths. The whole header stays sticky; the tab row scrolls horizontally on narrow screens.
+- **The region selector now pre-selects the configured region**: on load the dashboard fetches `/api/config` and adopts the `region` from `lss.config.json` (env `AWS_REGION` override included). `localStorage` keeps acting as a first-paint cache and in-session changes still stick, but the configured region wins on every full load; a configured region outside the built-in list is added to the dropdown instead of being ignored.
+- **Each example now runs in its own region**: `localstack-free` → `sa-east-1`, `localstack-ultimate` → `eu-west-1`, `self-hosted` → `us-west-2` — config, `serverless.yml`s, SDK fallbacks, embedded ARNs and docs updated coherently (`examples/README.md` gained a Region column).
+- `examples/localstack-ultimate` now lists `events` in `lss.config.json` `services` (EventBridge was missing from the LocalStack container's `SERVICES`).
+- **`serverless-lss` peer dependency restored to `osls 4.0.0`** (the open-source Serverless Framework v3 fork — no auth token, unlike `serverless` v4, and not EOL like v3): an unreleased change had moved it to an optional `serverless >=3.40.0`. Example lockfiles re-synced accordingly, and the plugin's vestigial nested `package-lock.json` (npm workspaces ignore it, so it only drifted) was removed. `examples/localstack-ultimate` keeps its `overrides: { "serverless": "npm:osls@4.0.0" }` — that alias is what stops `serverless-offline`'s `serverless` peer from pulling the token-gated framework; the self-hosted example references no framework package besides `osls` itself.
+- The root package version is now `0.10.0`; `serverless-lss` shipped as `0.2.2` (the version bump let the publish workflow ship the peer-dependency restore — the `0.2.0` on npm still carries the `serverless 3.40.0` peer).
+
+### Fixed
+- **`NotImplemented` errors no longer point at a dead end**: operation-level gaps in services the self engine serves (e.g. DynamoDB PartiQL) suggested `selfEngine.fallbackEndpoint`, but fallback only forwards *whole services* the engine does not serve — and the service's data lives in the self engine, so forwarding a single operation would read the wrong state (a `ResourceNotFoundException` from LocalStack for a table that exists in the engine). The message now says exactly that; the service-level variant (whole service unknown, where fallback genuinely applies) keeps the suggestion. Same fix for the OpenSearch data-plane `unsupported` message.
+- **Cross-project service-cache contamination**: the registered-services cache lived flat in `~/.lss/orchestrator/cache`, keyed by service name only, and `rehydrateAll()` reactivated *every* cached entry at boot — so starting one example's orchestrator also bound the gateway ports of services registered by *other* projects (observed as `self-hosted`'s catalog-service `:3634` being held by the `localstack-free` orchestrator), and same-named services in two projects overwrote each other. The cache is now scoped per project under `cache/projects/<slug>-<hash-of-root>` — the project root is the config file's directory (symlink-resolved), falling back to the launch cwd when no project-local config exists, so no-config setups should keep starting the orchestrator from the same directory. Rehydration logs a note when a cached service's root lies outside the project root (a legitimate layout — config under `infra/`, services elsewhere) but still reactivates it. Existing flat-layout entries are ignored — re-register services once after upgrading.
+- **Gateway port conflicts now self-heal**: on `EADDRINUSE` the listener still reports `port-conflict`, but LSS retries the bind every 5 s and flips the service back to online as soon as the port frees (previously the listener was disabled until re-registration, while registration still reported success).
+- `examples/self-hosted/catalog-service/package.json` description said port `3641` (invoke `13641`); the real ports are `3634`/`13634`.
+
+## [0.9.0] - 2026-07-12
+
+OpenSearch Serverless support: collections declared as `AWS::OpenSearchServerless::Collection` are provisioned on registration and served **in-process** by the self engine — control plane (`aoss`) plus the OpenSearch REST data plane (document CRUD, `_bulk`, `_search` with the everyday query DSL, aggregations) at `http://localhost:14566/_aoss/<collection>`. No Docker, no OpenSearch cluster, no auth token. Runnable demo: `examples/opensearch-sample`.
+
+### Added
+- **OpenSearch Serverless emulator** (`src/server/engine/emulators/opensearch/`), served by the self engine under the `aoss` SigV4 scope:
+  - *Control plane* (AWS JSON 1.0, `X-Amz-Target: OpenSearchServerless.*`): `CreateCollection` (AOSS naming rule enforced, deterministic ids, `ACTIVE` immediately), `BatchGetCollection` (by names or ids, hands out the local `collectionEndpoint`/`dashboardEndpoint`), `ListCollections`, `DeleteCollection` (by id, or name for convenience; drops the collection's indices and documents). Errors carry the AWS names the provisioner branches on (`ConflictException`, `ResourceNotFoundException`).
+  - *Data plane* (OpenSearch REST under `<engine>/_aoss/<collection>`): index lifecycle (create with mappings/settings, get, delete, HEAD, `_mapping` get/merge, `_cat/indices`, `_refresh`), document CRUD (`_doc`, `_create`, `_update` with deep merge + `doc_as_upsert`/`upsert`, versioning, auto-create index on first write), `_bulk` NDJSON with per-item results, `_search`/`_count` with `match`, `match_phrase`, `multi_match`, `term`, `terms`, `range`, `prefix`, `wildcard`, `exists`, `ids` and `bool` (+`minimum_should_match`), `sort`, `from`/`size`, `_source` filtering, the `?q=` shorthand, and `terms`/`avg`/`sum`/`min`/`max`/`value_count` aggregations. Errors are OpenSearch-shaped (`{"error": {...}, "status": N}`); unsupported operations/operators fail loudly naming the feature. Collections/index metadata persist as JSON catalogs, documents as one JSONL table per index (lazy hydration/idle unload like DynamoDB items).
+- **`AWS::OpenSearchServerless::Collection` provisioning**: parsed from `resources:` (name defaults to the lowercased logical id), created via `@aws-sdk/client-opensearchserverless` on registration (idempotent on `ConflictException`), deleted on cleanup (id resolved via `BatchGetCollection`). On an engine without an aoss provider (community LocalStack) the failure message points at the self engine. `SecurityPolicy`/`AccessPolicy`/`VpcEndpoint` resources are accepted and skipped with a registration warning — nothing enforces them locally, and the template stays deployable to real AWS.
+- **Dashboard visibility**: OpenSearch collections show up in the service detail page (own section) and as a `resourceBreakdown.collections` counter in the services list; `GET /api/resources` gains a `collections` array (plus `/owners` mapping) and the Overview page a collections stat tile.
+- **Example `examples/opensearch-sample`**: a product-catalog service (`POST /products`, `GET/DELETE /products/{id}`, `GET /search?q=&category=&maxPrice=`, `GET /stats`) exercising full-text search, `bool` filters, sorting and aggregations against the collection declared in its `serverless.yml` — handlers speak the OpenSearch REST API with Node's global `fetch`, zero runtime dependencies. Verified end to end on the self engine, including document persistence across an orchestrator restart.
+- **Dashboard branding via `lss.config.json`**: new optional `branding` key — `title`, `subtitle`, `logo`, `favicon` (http(s)/data URL, or a file path resolved next to the config file and served at `GET /api/config/branding/logo|favicon`), `defaultTheme` (`dark`/`light`), and TreeUI color-token overrides via `colors` (both themes) / `themeColors.dark|light` (per theme; keys accept the token suffix like `brand-primary` or a full `--tree-*` custom property). Exposed at `GET /api/config/branding` and inside `GET /api/config`; the UI applies title/logo/favicon/theme/colors on boot. The user's theme choice in the UI menu is now persisted per browser (`localStorage`) and wins over `defaultTheme`.
+
+### Changed
+- New runtime dependency: `@aws-sdk/client-opensearchserverless`.
+- The self engine's `/_localstack/health` alias and `healthDetail()` now report `aoss` among the available services; unsigned requests to `/_aoss/...` paths route to the OpenSearch emulator (ahead of the S3 path fallback).
+- The root package version is now `0.9.0`. The `serverless-lss` plugin package was not changed.
+- **Dashboard component library updated**: `@treeui/vue` `^0.10.0` → `^0.11.0`. No component API changes required.
+- Docs overhaul, audited against the v0.8.0 implementation: README rewritten (correct npm scripts, plugin keys `custom.orchestrator.orchestratorUrl` + `custom.lss.{apiPort,invokePort}`, full CLI surface incl. `seed`/`seed:clear`/`--config`, accurate registration/event-flow mermaid diagrams, stale "Project Structure"/"Project Status"/publishing sections removed); `docs/README.md` is now a real documentation index (the original design proposal moved to `docs/archive/DESIGN_PROPOSAL.md` with a historical banner); `docs/RELEASE.md` matches the actual publish workflow (no tests, no auto-tags); `docs/CONFIGURATION.md` gains `--config`/`LSS_CONFIG_PATH`, `seedsDir`, `stateDir`, `branding`, the missing env vars and a config-precedence diagram; `docs/FEATURES.md`/`docs/SELF_ENGINE.md`/`CONTRIBUTING.md`/example READMEs corrected against the code (incl. two real self-engine divergences now tracked in TODO.md: SQS `RedrivePolicy` not enforced, S3 ACL/policy sub-resources not recognized).
+- `serverless.yml.example` moved to `docs/serverless.yml.example` and updated (runtime `nodejs20.x`, `custom.lss` ports as the primary mechanism).
+- `lss.config.json.example` now showcases `engine`/`selfEngine` and `branding`; the repo's dev `lss.config.json` gains `"events"` in `services`.
+- `TODO.md` re-audited: delivered items removed (npm publication, CLI unit tests, orchestrator integration tests, auto-rebuild on serverless.yml changes, architecture diagrams), half-done items narrowed, self-engine hardening backlog added.
+
+### Removed
+- Stale root files: `local-serverless-stack-0.0.3.tgz` (old package artifact, now gitignored via `*.tgz`), `run-tests.sh` (referenced npm scripts that no longer exist; CI uses npm scripts directly), `.npmignore` (redundant — the `files` allowlist in package.json controls the published content; verified identical `npm pack` output).
+
 ## [0.8.0] - 2026-07-10
 
 The self engine: an in-process AWS emulator that replaces LocalStack for the typical serverless dev loop — DynamoDB, SQS, S3, EventBridge, SNS (minimal), Lambda control plane and STS served by the orchestrator itself on one port. No Docker, no container, no auth token; boots in milliseconds; data persisted in local files. Opt-in (`engine: "self"` / `lss start --self-engine`) — LocalStack mode remains the default and fully supported. Design: `docs/PRD_SELF_ENGINE.md`; coverage matrix: `docs/SELF_ENGINE.md`.