insforge 0.3.3 → 1.3.0

package/.github/copilot-instructions.md

@@ -1,147 +1,147 @@
-# Project Overview
-
-InsForge is an open-source Backend-as-a-Service (BaaS) platform designed specifically for AI agents. It provides a comprehensive solution for managing backend services including authentication, database operations, storage, and serverless functions, all accessible through REST APIs with PostgreSQL as the primary database.
-
-## Architecture
-
-This is a monorepo containing:
-- **Backend**: Node.js with Express.js, providing RESTful APIs
-- **Frontend**: React with Vite, offering an admin dashboard
-- **Functions**: Serverless function runtime using Deno
-- **Shared-schemas**: Common TypeScript schemas shared across modules
-
-## Folder Structure
-
-- `/backend`: Node.js backend server
-  - `/src/api`: API routes and middleware
-  - `/src/controllers`: Business logic controllers
-  - `/src/core`: Core services (auth, database, storage, metadata)
-  - `/src/types`: TypeScript type definitions
-  - `/src/utils`: Utility functions and helpers
-- `/frontend`: React dashboard application
-  - `/src/components`: Reusable UI components
-  - `/src/features`: Feature-specific modules (auth, database, storage, logs)
-  - `/src/lib`: Shared libraries, hooks, and utilities
-- `/functions`: Serverless function runtime
-- `/shared-schemas`: Shared TypeScript schemas
-- `/docker-init`: Docker initialization scripts
-- `/openapi`: API documentation in OpenAPI format
-- `/tests`: Comprehensive test suites
-
-## Libraries and Frameworks
-
-### Backend
-- Express.js for REST API framework
-- PostgreSQL with pg driver for database
-- Better-auth for authentication
-- AWS SDK for S3-compatible storage
-- Zod for schema validation
-- JWT for token management
-- TypeScript for type safety
-
-### Frontend
-- React 19 with functional components
-- Vite for build tooling
-- TanStack Query for data fetching
-- React Hook Form with Zod validation
-- Tailwind CSS for styling
-- Radix UI for accessible component primitives
-- React Router for navigation
-- TypeScript for type safety
-
-## Coding Standards
-
-### General
-- Use TypeScript for all code files
-- Prefer descriptive, unabbreviated variable and function names
-- Follow consistent file naming: kebab-case for files, PascalCase for components
-- Maintain clear folder structure with feature-based organization
-- Use ES modules (import/export syntax)
-- Implement proper error handling with try-catch blocks
-- Add appropriate logging for debugging
-
-### Backend Specific
-- Use async/await for asynchronous operations
-- Implement proper middleware for authentication and error handling
-- Follow RESTful conventions for API endpoints
-- Use Zod schemas for request/response validation
-- Return consistent API responses using utility functions
-- Implement proper database transaction handling
-
-### Frontend Specific
-- Use functional components with hooks exclusively
-- Implement proper TypeScript interfaces for all props
-- Use React Hook Form for form handling
-- Follow component composition patterns
-- Implement proper loading and error states
-- Use TanStack Query for server state management
-- Keep components focused and single-purpose
-
-#### Naming Conventions
-- **Variables, parameters, properties, and functions**: Use camelCase
-  - Examples: `userName`, `getUserData()`, `isLoading`
-- **React components and TypeScript types/interfaces**: Use PascalCase
-  - Examples: `UserProfileDialog`, `interface User`, `type TableSchema`
-- **Constants and Enum members**: Use UPPER_CASE
-  - Examples: `const API_BASE_URL`, `enum Status { ACTIVE, INACTIVE }`
-- **File naming**:
-  - Components: `PascalCase.tsx` (e.g., `DataGrid.tsx`)
-  - Hooks: `use` + `PascalCase.ts` (e.g., `useAuth.ts`)
-  - Services: `camelCase.service.ts` (e.g., `auth.service.ts`)
-  - Utils: `kebab-case.ts` or `camelCase.ts` (e.g., `database-utils.ts`)
-- **No snake_case**: Avoid snake_case except in SQL queries and external APIs
-
-### Code Style
-- Use single quotes for strings
-- Include semicolons at the end of statements
-- Use arrow functions for callbacks and inline functions
-- Prefer const over let, avoid var
-- Use template literals for string interpolation
-- Maintain consistent indentation (2 spaces)
-- Format code with Prettier configuration
-
-## UI Guidelines
-
-- Follow a clean, modern design with consistent spacing
-- Use Radix UI primitives for accessibility
-- Maintain consistent color scheme using CSS variables
-- Use appropriate loading skeletons for data fetching
-- Display clear error messages with actionable feedback
-- Implement proper form validation with inline errors
-- Use consistent icon set from lucide-react
-
-## Testing Standards
-
-- Implement integration tests for API endpoints
-- Maintain test coverage above 70%
-- Mock external dependencies appropriately
-
-## Security Considerations
-
-- Never commit secrets or API keys
-- Use environment variables for configuration
-- Implement proper authentication and authorization
-- Validate all user inputs
-- Sanitize data before database operations
-- Use prepared statements for SQL queries
-- Implement rate limiting for API endpoints
-- Follow OWASP security best practices
-
-## Performance Guidelines
-
-- Implement proper database indexing
-- Use pagination for large data sets
-- Optimize React component re-renders
-- Implement proper caching strategies
-- Use lazy loading for code splitting
-- Optimize bundle size with proper imports
-- Monitor and log performance metrics
-
-## Documentation
-
-- Document all API endpoints with clear descriptions
-- Include JSDoc comments for complex functions
-- Maintain up-to-date README files
-- Document environment variables and configuration
-- Provide clear setup instructions
+# Project Overview
+
+InsForge is an open-source Backend-as-a-Service (BaaS) platform designed specifically for AI agents. It provides a comprehensive solution for managing backend services including authentication, database operations, storage, and serverless functions, all accessible through REST APIs with PostgreSQL as the primary database.
+
+## Architecture
+
+This is a monorepo containing:
+- **Backend**: Node.js with Express.js, providing RESTful APIs
+- **Frontend**: React with Vite, offering an admin dashboard
+- **Functions**: Serverless function runtime using Deno
+- **Shared-schemas**: Common TypeScript schemas shared across modules
+
+## Folder Structure
+
+- `/backend`: Node.js backend server
+  - `/src/api`: API routes and middleware
+  - `/src/controllers`: Business logic controllers
+  - `/src/core`: Core services (auth, database, storage, metadata)
+  - `/src/types`: TypeScript type definitions
+  - `/src/utils`: Utility functions and helpers
+- `/frontend`: React dashboard application
+  - `/src/components`: Reusable UI components
+  - `/src/features`: Feature-specific modules (auth, database, storage, logs)
+  - `/src/lib`: Shared libraries, hooks, and utilities
+- `/functions`: Serverless function runtime
+- `/shared-schemas`: Shared TypeScript schemas
+- `/docker-init`: Docker initialization scripts
+- `/openapi`: API documentation in OpenAPI format
+- `/tests`: Comprehensive test suites
+
+## Libraries and Frameworks
+
+### Backend
+- Express.js for REST API framework
+- PostgreSQL with pg driver for database
+- Better-auth for authentication
+- AWS SDK for S3-compatible storage
+- Zod for schema validation
+- JWT for token management
+- TypeScript for type safety
+
+### Frontend
+- React 19 with functional components
+- Vite for build tooling
+- TanStack Query for data fetching
+- React Hook Form with Zod validation
+- Tailwind CSS for styling
+- Radix UI for accessible component primitives
+- React Router for navigation
+- TypeScript for type safety
+
+## Coding Standards
+
+### General
+- Use TypeScript for all code files
+- Prefer descriptive, unabbreviated variable and function names
+- Follow consistent file naming: kebab-case for files, PascalCase for components
+- Maintain clear folder structure with feature-based organization
+- Use ES modules (import/export syntax)
+- Implement proper error handling with try-catch blocks
+- Add appropriate logging for debugging
+
+### Backend Specific
+- Use async/await for asynchronous operations
+- Implement proper middleware for authentication and error handling
+- Follow RESTful conventions for API endpoints
+- Use Zod schemas for request/response validation
+- Return consistent API responses using utility functions
+- Implement proper database transaction handling
+
+### Frontend Specific
+- Use functional components with hooks exclusively
+- Implement proper TypeScript interfaces for all props
+- Use React Hook Form for form handling
+- Follow component composition patterns
+- Implement proper loading and error states
+- Use TanStack Query for server state management
+- Keep components focused and single-purpose
+
+#### Naming Conventions
+- **Variables, parameters, properties, and functions**: Use camelCase
+  - Examples: `userName`, `getUserData()`, `isLoading`
+- **React components and TypeScript types/interfaces**: Use PascalCase
+  - Examples: `UserProfileDialog`, `interface User`, `type TableSchema`
+- **Constants and Enum members**: Use UPPER_CASE
+  - Examples: `const API_BASE_URL`, `enum Status { ACTIVE, INACTIVE }`
+- **File naming**:
+  - Components: `PascalCase.tsx` (e.g., `DataGrid.tsx`)
+  - Hooks: `use` + `PascalCase.ts` (e.g., `useAuth.ts`)
+  - Services: `camelCase.service.ts` (e.g., `auth.service.ts`)
+  - Utils: `kebab-case.ts` or `camelCase.ts` (e.g., `database-utils.ts`)
+- **No snake_case**: Avoid snake_case except in SQL queries and external APIs
+
+### Code Style
+- Use single quotes for strings
+- Include semicolons at the end of statements
+- Use arrow functions for callbacks and inline functions
+- Prefer const over let, avoid var
+- Use template literals for string interpolation
+- Maintain consistent indentation (2 spaces)
+- Format code with Prettier configuration
+
+## UI Guidelines
+
+- Follow a clean, modern design with consistent spacing
+- Use Radix UI primitives for accessibility
+- Maintain consistent color scheme using CSS variables
+- Use appropriate loading skeletons for data fetching
+- Display clear error messages with actionable feedback
+- Implement proper form validation with inline errors
+- Use consistent icon set from lucide-react
+
+## Testing Standards
+
+- Implement integration tests for API endpoints
+- Maintain test coverage above 70%
+- Mock external dependencies appropriately
+
+## Security Considerations
+
+- Never commit secrets or API keys
+- Use environment variables for configuration
+- Implement proper authentication and authorization
+- Validate all user inputs
+- Sanitize data before database operations
+- Use prepared statements for SQL queries
+- Implement rate limiting for API endpoints
+- Follow OWASP security best practices
+
+## Performance Guidelines
+
+- Implement proper database indexing
+- Use pagination for large data sets
+- Optimize React component re-renders
+- Implement proper caching strategies
+- Use lazy loading for code splitting
+- Optimize bundle size with proper imports
+- Monitor and log performance metrics
+
+## Documentation
+
+- Document all API endpoints with clear descriptions
+- Include JSDoc comments for complex functions
+- Maintain up-to-date README files
+- Document environment variables and configuration
+- Provide clear setup instructions
 - Include examples for common use cases

package/.github/workflows/build-image.yml

@@ -1,65 +1,66 @@
-name: Build and Push Docker Image on github container registry
-
-on:
-  push:
-    tags:
-      - 'v*'
-  workflow_dispatch:
-    inputs:
-      test_tag:
-        description: 'Test tag name (e.g., v0.1.1-test)'
-        required: false
-        default: 'v0.1.1-test'
-
-env:
-  REGISTRY: ghcr.io
-  IMAGE_NAME: ${{ github.repository_owner }}/insforge-oss
-
-jobs:
-  build-and-push:
-    runs-on: ubuntu-latest
-    permissions:
-      contents: read
-      packages: write
-      id-token: write
-      
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v4
-
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v3
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
-
-      - name: Log in to GitHub Container Registry
-        uses: docker/login-action@v3
-        with:
-          registry: ${{ env.REGISTRY }}
-          username: ${{ github.actor }}
-          password: ${{ secrets.GHCR_PAT }}
-
-      - name: Extract metadata
-        id: meta
-        uses: docker/metadata-action@v5
-        with:
-          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
-          tags: |
-            type=ref,event=tag
-            type=raw,value=${{ github.event.inputs.test_tag }},enable=${{ github.event_name == 'workflow_dispatch' }}
-
-      - name: Build and push Docker image
-        uses: docker/build-push-action@v5
-        with:
-          context: .
-          platforms: linux/amd64,linux/arm64
-          push: true
-          tags: ${{ steps.meta.outputs.tags }}
-          labels: ${{ steps.meta.outputs.labels }}
-          cache-from: type=gha
-          cache-to: type=gha,mode=max
-          build-args: |
-            VERSION=${{ github.ref_name }}
-            BUILD_DATE=${{ github.event.head_commit.timestamp }}
-            COMMIT_SHA=${{ github.sha }}
+name: Build and Push Docker Image on github container registry
+
+on:
+  push:
+    tags:
+      - 'v*'
+  workflow_dispatch:
+    inputs:
+      test_tag:
+        description: 'Test tag name (e.g., v0.1.1-test)'
+        required: false
+        default: 'v0.1.1-test'
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository_owner }}/insforge-oss
+
+jobs:
+  build-and-push:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+      id-token: write
+      
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Log in to GitHub Container Registry
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GHCR_PAT }}
+
+      - name: Extract metadata
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+          tags: |
+            type=ref,event=tag
+            type=raw,value=${{ github.event.inputs.test_tag }},enable=${{ github.event_name == 'workflow_dispatch' }}
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v5
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+          build-args: |
+            VERSION=${{ github.ref_name }}
+            BUILD_DATE=${{ github.event.head_commit.timestamp }}
+            COMMIT_SHA=${{ github.sha }}
+            VITE_PUBLIC_POSTHOG_KEY=${{ secrets.POSTHOG_KEY }}

package/.github/workflows/ci-premerge-check.yml

@@ -1,24 +1,24 @@
-name: CI Pre-merge Check
-
-on:
-  pull_request:
-    branches: [ main, master, develop ]
-  push:
-    branches: [ main, master, develop ]
-  workflow_dispatch:  # Allows manual triggering for testing
-
-jobs:
-  docker-build:
-    name: CI Pre-merge Check
-    runs-on: ubuntu-latest
-    
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v4
-    
-    - name: Build Docker image
-      uses: docker/build-push-action@v5
-      with:
-        context: .
-        push: false
+name: CI Pre-merge Check
+
+on:
+  pull_request:
+    branches: [ main, master, develop ]
+  push:
+    branches: [ main, master, develop ]
+  workflow_dispatch:  # Allows manual triggering for testing
+
+jobs:
+  docker-build:
+    name: CI Pre-merge Check
+    runs-on: ubuntu-latest
+    
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v4
+    
+    - name: Build Docker image
+      uses: docker/build-push-action@v5
+      with:
+        context: .
+        push: false
         tags: insforge:test

package/.github/workflows/e2e.yml

@@ -0,0 +1,63 @@
+name: E2E Tests
+
+on:
+  pull_request:
+    branches: [main, master, develop]
+  workflow_dispatch:
+
+jobs:
+  e2e:
+    name: E2E Tests
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Create .env file
+        run: |
+          cat > .env << EOF
+          POSTGRES_USER=postgres
+          POSTGRES_PASSWORD=postgres
+          POSTGRES_DB=insforge
+          JWT_SECRET=test-secret-key-for-ci-cd-pipeline
+          ADMIN_EMAIL=admin@example.com
+          ADMIN_PASSWORD=test-admin-password-for-ci
+          TEST_API_BASE=http://localhost:7130/api
+          EOF
+
+      - name: Start Docker Compose services
+        run: docker compose up -d
+
+      - name: Wait for PostgreSQL to be ready
+        run: |
+          echo "Waiting for PostgreSQL..."
+          timeout 60 bash -c 'until docker compose exec -T postgres pg_isready -U postgres; do sleep 2; done'
+
+      - name: Wait for InsForge backend to be ready
+        run: |
+          echo "Waiting for InsForge backend..."
+          timeout 120 bash -c 'until curl -f http://localhost:7130/api/health 2>/dev/null; do echo "Waiting..."; sleep 5; done'
+          echo "Backend is ready!"
+
+      - name: Install test dependencies in container
+        run: docker compose exec -T insforge apk add --no-cache bash curl jq
+
+      - name: Run E2E tests
+        run: docker compose exec -T insforge npm run test:e2e
+
+      - name: Show container logs on failure
+        if: failure()
+        run: |
+          echo "=== PostgreSQL logs ==="
+          docker compose logs postgres
+          echo ""
+          echo "=== InsForge backend logs ==="
+          docker compose logs insforge
+          echo ""
+          echo "=== PostgREST logs ==="
+          docker compose logs postgrest
+
+      - name: Cleanup
+        if: always()
+        run: docker compose down -v

package/.github/workflows/lint-and-format.yml

@@ -1,33 +1,33 @@
-name: Lint and Format Check
-
-on:
-  push:
-    branches: [ main ]
-  pull_request:
-    branches: [ '*' ]
-
-jobs:
-  lint-and-format:
-    runs-on: ubuntu-latest
-
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v4
-
-      - name: Setup Node.js
-        uses: actions/setup-node@v4
-        with:
-          node-version: '20'
-          cache: 'npm'
-
-      - name: Install dependencies
-        run: npm ci
-
-      - name: Run ESLint
-        run: npm run lint
-
-      - name: Check Prettier formatting
-        run: npx prettier --check .
-
-      - name: Run TypeScript type checking
+name: Lint and Format Check
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ '*' ]
+
+jobs:
+  lint-and-format:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: '20'
+          cache: 'npm'
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Run ESLint
+        run: npm run lint
+
+      - name: Check Prettier formatting
+        run: npx prettier --check .
+
+      - name: Run TypeScript type checking
         run: npm run typecheck