@quiltdata/benchling-webhook 0.4.13 → 0.5.0

package/doc/PARAMETERS.md

@@ -1,203 +0,0 @@
-# CloudFormation Parameters
-
-This stack supports runtime-configurable parameters that can be updated without redeploying the entire stack.
-
-## Available Parameters
-
-### Security and Configuration
-
-#### WebhookAllowList
-
-- **Type**: String (comma-separated IP addresses)
-- **Description**: List of IP addresses allowed to send webhooks
-- **Default**: Value from `WEBHOOK_ALLOW_LIST` environment variable (or empty to allow all IPs)
-- **Example**: `34.216.192.90,34.217.183.162`
-
-#### QuiltCatalog
-
-- **Type**: String
-- **Description**: Quilt catalog URL for package links
-- **Default**: `open.quiltdata.com`
-- **Example**: `catalog.example.com`
-
-### Infrastructure Parameters
-
-#### BucketName
-
-- **Type**: String
-- **Description**: S3 bucket name for storing packages (your data bucket)
-- **Default**: Value from `QUILT_USER_BUCKET` environment variable
-- **Example**: `my-packages-bucket`
-
-#### PackagePrefix
-
-- **Type**: String
-- **Description**: Prefix for package names (no slashes allowed)
-- **Default**: Value from `PREFIX` environment variable
-- **Example**: `benchling` or `my-org`
-
-#### QueueName
-
-- **Type**: String
-- **Description**: SQS queue name for package notifications
-- **Default**: Value from `QUEUE_NAME` environment variable
-- **Example**: `my-packager-queue`
-
-**Note**: Benchling credentials (CLIENT_ID, CLIENT_SECRET, TENANT) are **not** parameters because they're used to create AWS resources at deployment time and contain sensitive information.
-
-## Updating Parameters
-
-### Using AWS CLI
-
-Update parameters without redeploying the stack:
-
-```bash
-# Update webhook allow list only
-aws cloudformation update-stack \
-  --stack-name BenchlingWebhookStack \
-  --use-previous-template \
-  --parameters \
-    ParameterKey=WebhookAllowList,ParameterValue="1.2.3.4,5.6.7.8" \
-    ParameterKey=QuiltCatalog,UsePreviousValue=true \
-    ParameterKey=BucketName,UsePreviousValue=true \
-    ParameterKey=PackagePrefix,UsePreviousValue=true \
-    ParameterKey=QueueName,UsePreviousValue=true
-
-# Update multiple parameters
-aws cloudformation update-stack \
-  --stack-name BenchlingWebhookStack \
-  --use-previous-template \
-  --parameters \
-    ParameterKey=WebhookAllowList,ParameterValue="34.216.192.90,34.217.183.162" \
-    ParameterKey=QuiltCatalog,ParameterValue="my-catalog.quiltdata.com" \
-    ParameterKey=BucketName,ParameterValue="new-bucket-name" \
-    ParameterKey=PackagePrefix,UsePreviousValue=true \
-    ParameterKey=QueueName,UsePreviousValue=true
-```
-
-### Using AWS Console
-
-1. Go to CloudFormation in AWS Console
-2. Select the `BenchlingWebhookStack` stack
-3. Click **Update**
-4. Select **Use current template**
-5. Click **Next**
-6. Update the parameter values as needed:
-   - **WebhookAllowList**: Comma-separated IP addresses
-   - **QuiltCatalog**: Catalog URL
-   - **BucketName**: S3 bucket name
-   - **PackagePrefix**: Package prefix
-   - **QueueName**: SQS queue name
-7. Click **Next** through the remaining screens
-8. Click **Update stack**
-
-### Using CDK
-
-Parameters are automatically created during CDK deployment. The initial values come from environment variables, but subsequent updates can be done via CloudFormation (as shown above) without redeploying via CDK.
-
-## Benefits
-
-- **No Code Deployment**: Update security settings (IP allowlist) without redeploying code
-- **Fast Updates**: CloudFormation parameter updates are much faster than full stack updates
-- **Zero Downtime**: Parameter updates don't require Lambda redeployment
-- **Audit Trail**: All parameter changes are tracked in CloudFormation change sets
-
-## Implementation Details
-
-### Architecture
-
-1. **CloudFormation Parameters**: Defined at stack level
-2. **Lambda Environment Variables**: Automatically updated from parameters
-3. **API Gateway Resource Policy**: Automatically updated from parameters
-
-### How It Works
-
-The stack uses a hybrid approach:
-
-- **Initial Deployment**: Uses environment variable values from `bin/benchling-webhook.ts`
-- **Subsequent Updates**: Uses CloudFormation parameter values
-- **Lambda Environment**: Automatically reflects parameter changes via CloudFormation
-
-When you update a parameter via CloudFormation:
-
-1. CloudFormation updates the Lambda function's environment variables
-2. CloudFormation updates the API Gateway Resource Policy
-3. Lambda automatically picks up the new values on next invocation (no cold start required)
-4. API Gateway immediately enforces the new IP allowlist
-
-## Examples
-
-### Add a new IP to the allowlist
-
-```bash
-# Current IPs: 34.216.192.90,34.217.183.162
-# Adding new IP: 203.0.113.10
-
-aws cloudformation update-stack \
-  --stack-name BenchlingWebhookStack \
-  --use-previous-template \
-  --parameters \
-    ParameterKey=WebhookAllowList,ParameterValue="34.216.192.90,34.217.183.162,203.0.113.10" \
-    ParameterKey=QuiltCatalog,UsePreviousValue=true \
-    ParameterKey=BucketName,UsePreviousValue=true \
-    ParameterKey=PackagePrefix,UsePreviousValue=true \
-    ParameterKey=QueueName,UsePreviousValue=true
-```
-
-### Switch to a different S3 bucket
-
-```bash
-aws cloudformation update-stack \
-  --stack-name BenchlingWebhookStack \
-  --use-previous-template \
-  --parameters \
-    ParameterKey=WebhookAllowList,UsePreviousValue=true \
-    ParameterKey=QuiltCatalog,UsePreviousValue=true \
-    ParameterKey=BucketName,ParameterValue="new-packages-bucket" \
-    ParameterKey=PackagePrefix,UsePreviousValue=true \
-    ParameterKey=QueueName,UsePreviousValue=true
-```
-
-### Change package prefix
-
-```bash
-aws cloudformation update-stack \
-  --stack-name BenchlingWebhookStack \
-  --use-previous-template \
-  --parameters \
-    ParameterKey=WebhookAllowList,UsePreviousValue=true \
-    ParameterKey=QuiltCatalog,UsePreviousValue=true \
-    ParameterKey=BucketName,UsePreviousValue=true \
-    ParameterKey=PackagePrefix,ParameterValue="my-new-prefix" \
-    ParameterKey=QueueName,UsePreviousValue=true
-```
-
-### Allow all IPs (remove IP filtering)
-
-```bash
-aws cloudformation update-stack \
-  --stack-name BenchlingWebhookStack \
-  --use-previous-template \
-  --parameters \
-    ParameterKey=WebhookAllowList,ParameterValue="" \
-    ParameterKey=QuiltCatalog,UsePreviousValue=true \
-    ParameterKey=BucketName,UsePreviousValue=true \
-    ParameterKey=PackagePrefix,UsePreviousValue=true \
-    ParameterKey=QueueName,UsePreviousValue=true
-```
-
-## Troubleshooting
-
-### Parameter update fails
-
-If the CloudFormation update fails, check:
-
-- IP addresses are valid (no spaces, proper CIDR notation if needed)
-- Catalog URL is a valid hostname
-- You're using `--use-previous-template` flag
-
-### Changes not taking effect
-
-- API Gateway Resource Policy updates are immediate
-- Lambda environment variable updates require the Lambda to be invoked again
-- Check CloudFormation Events tab for update status

package/doc/RELEASE.md

@@ -1,297 +0,0 @@
-# Release Process
-
-This document describes how to create releases for the benchling-webhook project.
-
-## Overview
-
-The project uses an automated CI/CD pipeline that:
-
-1. Runs tests on all pushes and pull requests
-2. Builds and pushes Docker images when tags are created
-3. Creates GitHub releases with auto-generated release notes
-4. Publishes to NPM and GitHub Packages
-
-## Quick Start
-
-### Create a Development Release (for testing)
-
-```bash
-npm run version:dev
-# This creates version 0.4.8-dev.0 (or bumps existing dev version)
-# Then push the tag:
-git push origin v0.4.8-dev.0
-```
-
-### Create a Production Release
-
-```bash
-# For patch releases (0.4.7 -> 0.4.8)
-npm run version:patch
-
-# For minor releases (0.4.7 -> 0.5.0)
-npm run version:minor
-
-# For major releases (0.4.7 -> 1.0.0)
-npm run version:major
-
-# Then push the tag:
-git push origin v0.4.8
-```
-
-## Detailed Workflow
-
-### 1. Prepare Your Release
-
-Before creating a release:
-
-1. **Ensure all changes are committed and pushed**
-
-   ```bash
-   git status  # Should show "nothing to commit, working tree clean"
-   ```
-
-2. **Update CHANGELOG.md** (optional but recommended)
-
-   ```markdown
-   ## [0.4.8] - 2025-10-27
-
-   ### Added
-   - New feature description
-
-   ### Changed
-   - Updated component behavior
-
-   ### Fixed
-   - Bug fix description
-   ```
-
-3. **Ensure you're on the main branch**
-
-   ```bash
-   git checkout main
-   git pull origin main
-   ```
-
-### 2. Create a Release
-
-#### Development Releases (for testing)
-
-Development releases are marked as "pre-release" in GitHub and are NOT published to NPM.
-
-```bash
-# Create a new dev release
-npm run version:dev
-
-# Or bump an existing dev release
-npm run version:dev-bump
-```
-
-This will:
-
-- Update `package.json` with the new version
-- Commit the version change
-- Create a git tag (e.g., `v0.4.8-dev.0`)
-- Show instructions for pushing the tag
-
-#### Production Releases
-
-```bash
-# Choose the appropriate bump type:
-npm run version:patch  # For bug fixes (0.4.7 -> 0.4.8)
-npm run version:minor  # For new features (0.4.7 -> 0.5.0)
-npm run version:major  # For breaking changes (0.4.7 -> 1.0.0)
-```
-
-### 3. Push the Tag
-
-After creating the tag, push it to trigger the CI/CD pipeline:
-
-```bash
-git push origin v0.4.8  # Replace with your actual tag
-```
-
-### 4. Monitor the Release
-
-1. **Watch the GitHub Actions workflow**: <https://github.com/quiltdata/benchling-webhook/actions>
-2. **Check the workflow progress**:
-   - Test job runs first (Python + Node.js tests)
-   - Docker job runs after tests pass (builds and pushes image)
-   - Release job creates GitHub release and publishes packages
-
-3. **Verify the release**:
-   - GitHub Release: <https://github.com/quiltdata/benchling-webhook/releases>
-   - Docker Image in ECR: `{account-id}.dkr.ecr.us-east-1.amazonaws.com/quiltdata/benchling:{version}`
-   - NPM Package: <https://www.npmjs.com/package/quilt-benchling-webhook>
-   - GitHub Package: <https://github.com/quiltdata/benchling-webhook/pkgs/npm/quilt-benchling-webhook>
-
-## CI/CD Pipeline Details
-
-### Workflow Triggers
-
-The CI workflow (`.github/workflows/ci.yaml`) is triggered by:
-
-- **Push to main**: Runs tests only
-- **Pull requests**: Runs tests only
-- **Tags matching `v*.*.*`**: Runs tests, builds Docker, creates release
-- **Tags matching `v*.*.*-dev.*`**: Same as above, but marks as pre-release
-
-### Jobs
-
-1. **test** (always runs)
-   - Sets up Node.js 22 and Python 3.12
-   - Installs dependencies
-   - Runs Python tests (pytest)
-   - Runs Node.js tests (jest)
-   - Builds the package
-
-2. **docker** (only on tags, after test passes)
-   - Builds Docker image using `make push-ci`
-   - Pushes to ECR with version tag and `latest` tag
-   - Outputs image URI for release notes
-
-3. **release** (only on tags, after docker completes)
-   - Generates release notes from CHANGELOG.md
-   - Creates GitHub release (pre-release for dev tags)
-   - Publishes to NPM (production releases only)
-   - Publishes to GitHub Packages
-
-## Version Management Script
-
-The `bin/version.js` script handles version bumping and tagging:
-
-```bash
-# View current version and help
-npm run release
-
-# Bump versions
-npm run version:major      # 0.4.7 -> 1.0.0
-npm run version:minor      # 0.4.7 -> 0.5.0
-npm run version:patch      # 0.4.7 -> 0.4.8
-npm run version:dev        # 0.4.7 -> 0.4.8-dev.0 or 0.4.8-dev.0 -> 0.4.8-dev.1
-npm run version:dev-bump   # 0.4.8-dev.0 -> 0.4.8-dev.1
-
-# Update version without creating tag
-node bin/version.js patch --no-tag
-```
-
-## Docker Images
-
-Docker images are automatically built and pushed to ECR with:
-
-- Version tag: `quiltdata/benchling:0.4.8`
-- Latest tag: `quiltdata/benchling:latest`
-
-To pull and run:
-
-```bash
-docker pull {account-id}.dkr.ecr.us-east-1.amazonaws.com/quiltdata/benchling:0.4.8
-docker run -p 5000:5000 --env-file .env {account-id}.dkr.ecr.us-east-1.amazonaws.com/quiltdata/benchling:0.4.8
-```
-
-## Troubleshooting
-
-### Tag already exists
-
-```bash
-# Delete local tag
-git tag -d v0.4.8
-
-# Delete remote tag (if pushed)
-git push origin :refs/tags/v0.4.8
-```
-
-### Workflow failed
-
-1. Check the GitHub Actions logs
-2. Common issues:
-   - Tests failing: Fix tests and create a new tag
-   - Docker build failing: Check `docker/` directory and Makefile
-   - AWS credentials: Verify secrets are configured
-   - ECR repository doesn't exist: Run `make -C docker docker-ecr-create`
-
-### Need to re-release
-
-1. Delete the tag (locally and remotely)
-2. Delete the GitHub release
-3. Create a new tag with the same or different version
-
-## Manual NPM Publishing
-
-In cases where you need to manually publish to NPM (e.g., CI/CD is unavailable, emergency hotfix, or testing), you can use the manual publish script:
-
-### Prerequisites
-
-1. **NPM Access Token**: You need an NPM access token with publish permissions
-   - Visit: <https://www.npmjs.com/settings/[your-username]/tokens>
-   - Click "Generate New Token"
-   - Choose "Automation" (for CI/CD) or "Publish" (for manual use)
-   - Copy the token (it starts with `npm_`)
-
-### Usage
-
-```bash
-# Test the publish process (dry-run)
-NPM_TOKEN=npm_xxxxx npm run publish:manual -- --dry-run
-
-# Publish to NPM
-NPM_TOKEN=npm_xxxxx npm run publish:manual
-
-# Publish with a specific dist-tag
-NPM_TOKEN=npm_xxxxx npm run publish:manual -- --tag beta
-
-# View help
-npm run publish:manual -- --help
-```
-
-### How It Works
-
-The script:
-1. Validates your NPM token
-2. Checks for uncommitted changes (with confirmation prompt)
-3. Creates a temporary `.npmrc` file with your token
-4. Runs `npm publish --access public`
-5. Cleans up the temporary `.npmrc` file
-6. Restores any existing `.npmrc` backup
-
-### Security Notes
-
-- The script creates `.npmrc` with restricted permissions (0600)
-- Your token is never committed to git (`.npmrc` is in `.gitignore`)
-- The temporary `.npmrc` is automatically cleaned up, even on errors
-- Always use `--dry-run` first to test before publishing
-
-## NPM Scripts Reference
-
-| Script | Description |
-|--------|-------------|
-| `npm run release` | Show current version and help |
-| `npm run version:major` | Bump major version (breaking changes) |
-| `npm run version:minor` | Bump minor version (new features) |
-| `npm run version:patch` | Bump patch version (bug fixes) |
-| `npm run version:dev` | Create or bump dev version |
-| `npm run version:dev-bump` | Bump dev counter only |
-| `npm run docker-push` | Build and push Docker image locally |
-| `npm run docker-check` | Validate Docker images in registry |
-| `npm run publish:manual` | Manually publish to NPM using access token |
-
-## Best Practices
-
-1. **Always test before releasing**: Run `npm test` and manual testing
-2. **Use dev releases for testing**: Test the CI/CD pipeline with dev tags first
-3. **Update CHANGELOG.md**: Document changes before releasing
-4. **Follow semantic versioning**:
-   - Major: Breaking changes
-   - Minor: New features (backward compatible)
-   - Patch: Bug fixes (backward compatible)
-5. **Review the release**: Check GitHub release notes and test the published packages
-6. **Coordinate with team**: Communicate releases in team channels
-
-## Migration Notes
-
-The previous release process using `bin/create-release.sh` and separate `release.yaml` workflow has been replaced with this integrated CI/CD pipeline. Key changes:
-
-- **Single workflow**: All release steps are now in `.github/workflows/ci.yaml`
-- **Automated**: No manual Docker builds or release note creation needed
-- **Tag-based**: Simply push a tag to trigger the release
-- **Pre-release support**: Dev tags are automatically marked as pre-releases
-- **Integrated testing**: Docker images are only built after tests pass

package/doc/RELEASE_NOTES.md

@@ -1,64 +0,0 @@
-# Benchling Webhook Integration for Quilt
-
-Connects Benchling lab notebook entries to Quilt data packages via webhooks.
-
-## Quick Install
-
-**Prerequisites:** AWS account, Node.js 18+, Docker, existing Quilt deployment
-
-```bash
-# 1. Clone and install
-git clone https://github.com/quiltdata/benchling-webhook.git
-cd benchling-webhook
-npm install
-
-# 2. Configure (auto-infer from Quilt catalog)
-npm run get-env -- https://quilt-catalog.yourcompany.com --write
-cp env.inferred .env
-# Edit .env to add Benchling credentials
-
-# 3. Deploy
-source .env
-npx cdk bootstrap aws://$CDK_DEFAULT_ACCOUNT/$CDK_DEFAULT_REGION  # first time only
-npm run deploy
-
-# 4. Configure Benchling app
-# - Create app from app-manifest.yaml
-# - Set webhook URL from .env.deploy
-# - Install and activate
-
-# 5. Verify
-source .env.deploy
-curl $WEBHOOK_ENDPOINT/health
-```
-
-## Usage
-
-1. Create entry in Benchling
-2. Insert Canvas → "Quilt Integration"
-3. Click "Create" to make package
-4. Add files and click "Update package"
-
-## Documentation
-
-- [AGENTS.md](AGENTS.md) - Complete deployment guide, architecture, configuration
-- [docker/README.md](docker/README.md) - Development workflows
-- [doc/RELEASE.md](doc/RELEASE.md) - Release process
-
-## License
-
-Apache-2.0
-
----
-
-## Docker Images (for custom deployments)
-
-**Latest Release (v0.4.12):**
-- Production: `public.ecr.aws/quiltdata/benchling:0.4.12`
-- Latest: `public.ecr.aws/quiltdata/benchling:latest`
-
-Pull and run:
-```bash
-docker pull public.ecr.aws/quiltdata/benchling:0.4.12
-docker run -p 5000:5000 --env-file .env public.ecr.aws/quiltdata/benchling:0.4.12
-```

package/jest.config.js

@@ -1,14 +0,0 @@
-module.exports = {
-  testEnvironment: 'node',
-  roots: ['<rootDir>/test'],
-  testMatch: ['**/*.test.ts'],
-  transform: {
-    '^.+\\.tsx?$': ['ts-jest', {
-      tsconfig: {
-        module: 'node20',
-        resolveJsonModule: true
-      }
-    }]
-  },
-  cacheDirectory: '<rootDir>/.jest-cache'
-};

package/lib/README.md

@@ -1,50 +0,0 @@
-# Benchling Webhook Stack Architecture
-
-This directory contains the core CDK infrastructure code split across multiple files for better organization:
-
-## Core Components
-
-### benchling-webhook-stack.ts
-
-The main stack file that orchestrates the entire infrastructure. It:
-
-- Creates the S3 bucket reference
-- Sets up the Benchling OAuth connection
-- Instantiates the state machine and API gateway
-- Manages stack outputs
-
-### state-machine.ts
-
-Contains the Step Functions state machine definition that:
-
-- Processes incoming webhook events
-- Stores event data in S3
-- Fetches entry details from Benchling API
-- Sends notifications to SQS
-
-### api-gateway.ts
-
-Handles the API Gateway configuration including:
-
-- REST API setup with logging
-- IAM roles and permissions
-- Webhook endpoints for different event types
-- Integration with Step Functions
-
-## Flow
-
-1. API Gateway receives webhook POST requests
-2. Requests trigger Step Functions execution
-3. State machine:
-   - Stores raw event in S3
-   - Fetches entry details from Benchling
-   - Stores entry data in S3
-   - Sends notification to SQS
-
-## Key Features
-
-- Modular architecture for maintainability
-- Separation of concerns between components
-- Reusable constructs
-- Proper error handling
-- Comprehensive logging

package/lib/oauth-tester.json

@@ -1,35 +0,0 @@
-{
-  "Comment": "OAuth 2.0 with EventBridge Connection in Step Functions",
-  "StartAt": "RequestAccessToken",
-  "States": {
-    "RequestAccessToken": {
-      "Type": "Task",
-      "Resource": "arn:aws:states:::http:invoke",
-      "Parameters": {
-        "Method": "POST",
-        "ApiEndpoint": "https://quilt-dtt.benchling.com/api/v2/token",
-        "Authentication": {
-          "ConnectionArn": "arn:aws:events:us-east-1:123456789012:connection/BenchlingOAuthConnection"
-        }
-      },
-      "ResultSelector": {
-        "access_token": "$.Body.access_token"
-      },
-      "ResultPath": "$.auth",
-      "Next": "CallListEntries"
-    },
-    "CallListEntries": {
-      "Type": "Task",
-      "Resource": "arn:aws:states:::http:invoke",
-      "Parameters": {
-        "Method": "GET",
-        "ApiEndpoint": "https://quilt-dtt.benchling.com/api/v2/entries",
-        "Headers": {
-          "Authorization.$": "States.Format('Bearer {}', $.auth.access_token)",
-          "Content-Type": "application/json"
-        }
-      },
-      "End": true
-    }
-  }
-}

package/tsconfig.json

@@ -1,34 +0,0 @@
-{
-  "compilerOptions": {
-    "target": "ES2020",
-    "module": "node20",
-    "lib": [
-      "es2020",
-      "dom"
-    ],
-    "declaration": true,
-    "strict": true,
-    "noImplicitAny": true,
-    "strictNullChecks": true,
-    "noImplicitThis": true,
-    "alwaysStrict": true,
-    "noUnusedLocals": false,
-    "noUnusedParameters": false,
-    "noImplicitReturns": true,
-    "noFallthroughCasesInSwitch": false,
-    "inlineSourceMap": true,
-    "inlineSources": true,
-    "experimentalDecorators": true,
-    "strictPropertyInitialization": false,
-    "esModuleInterop": true,
-    "resolveJsonModule": true,
-    "noEmit": true,
-    "typeRoots": [
-      "./node_modules/@types"
-    ]
-  },
-  "exclude": [
-    "node_modules",
-    "cdk.out"
-  ]
-}