metal-orm 1.0.0

package/README.md

@@ -0,0 +1,30 @@
+<div align="center">
+<img width="1200" height="475" alt="GHBanner" src="https://github.com/user-attachments/assets/0aa67016-6eaf-458a-adb2-6e31a0763ed6" />
+</div>
+
+# Run and deploy your AI Studio app
+
+This contains everything you need to run your app locally.
+
+View your app in AI Studio: https://ai.studio/apps/drive/1vXchpJ36cRXpwzrizwz6I-M1Ks1GAkz2
+
+## Run Locally
+
+**Prerequisites:**  Node.js
+
+
+1. Install dependencies:
+   `npm install`
+2. Set the `GEMINI_API_KEY` in [.env.local](.env.local) to your Gemini API key
+3. Run the app:
+   `npm run dev`
+
+## Project layout
+
+- `src/` hosts the React-based playground UI that backs the demo.
+- `src/metal-orm/src/` contains the real MetalORM implementation (AST, builder, dialects, runtime) consumed by the playground.
+- Legacy `orm/` and `services/orm/` directories were removed because they were unused duplicates, so future work belongs in `src/metal-orm/src`.
+
+## Parameterized Queries
+
+Literal values in expressions are now emitted as parameter placeholders and collected in a binding list. Use `SelectQueryBuilder.compile(dialect)` to get `{ sql, params }` and pass both to your database driver (`client.executeSql(sql, params)`); `SelectQueryBuilder.toSql(dialect)` still returns just the SQL string for quick debugging.

package/ROADMAP.md

@@ -0,0 +1,125 @@
+# Metal ORM Roadmap
+
+## Current ORM Capabilities
+
+The current implementation supports:
+
+- Basic SELECT with projections and aliases
+- INNER/LEFT JOINs with manual conditions
+- Smart relationship joins via `joinRelation()`
+- Eager loading with `include()` for 1:1 and 1:N relationships
+- Basic WHERE clauses with operators (eq, gt, like, in, null checks)
+- GROUP BY, ORDER BY, LIMIT, OFFSET
+- Aggregation functions (COUNT, SUM, AVG)
+- JSON path extraction (dialect-specific)
+- Hydration of nested objects from flat SQL results
+- EXISTS and NOT EXISTS subqueries
+- Scalar correlated subqueries in SELECT and WHERE
+- CASE expressions (simple and searched)
+- HAVING clause for post-aggregation filtering
+
+## Identified Absences
+
+### 1. CTE (Common Table Expressions) ✅
+
+**Completed:** Full CTE support has been implemented
+
+- Features: Simple CTEs, Recursive CTEs, Multiple CTEs, CTE with column aliases
+- **Implementation:** CTE AST node and dialect compilation for SQLite, MySQL, and MSSQL
+- Recursive CTEs properly handle the `WITH RECURSIVE` keyword (SQLite/MySQL only, MSSQL uses plain `WITH`)
+- Mixed recursive and non-recursive CTEs are supported
+
+### 2. Window Functions ✅
+
+**Completed:** Comprehensive window function support has been implemented
+
+- Features: `ROW_NUMBER()`, `RANK()`, `DENSE_RANK()`, `LAG()`, `LEAD()`, `NTILE()`, `FIRST_VALUE()`, `LAST_VALUE()`
+- **Implementation:** Window function AST nodes with `PARTITION BY` and `ORDER BY` support
+- All three dialects (SQLite, MySQL, MSSQL) support window functions
+
+### 3. RIGHT JOIN Support
+
+**Missing:** Only INNER, LEFT, and CROSS joins are defined
+
+- **Required Addition:** RIGHT JOIN support in join.ts and dialect compilation
+
+### 4. Complex Aggregation Functions
+
+**Missing:** Limited aggregation functions
+
+- Queries: `5-top-platform-contributor-per-project`, `5-mega-user-engagement-analytics`
+- **Required Addition:** MIN, MAX, AVG functions, and GROUP_CONCAT
+
+### 5. Parameterized Queries
+
+**Missing:** No parameter binding support
+
+- Queries: `1-1-parameterized-user-by-twitter`
+- **Required Addition:** Parameter placeholder support and binding mechanism
+
+### 6. Advanced JSON Operations
+
+**Missing:** Limited JSON functionality
+
+- Queries: `1-1-profile-json-as-subquery-column`, `5-mega-user-engagement-analytics`
+- **Required Addition:** JSON_OBJECT, JSON_ARRAYAGG functions
+
+### 7. Recursive CTEs ✅
+
+**Completed:** See Section 1 (CTE) above
+
+- Recursive CTEs are fully supported as part of the CTE implementation
+
+### 8. Complex Ordering
+
+**Missing:** Limited ORDER BY expressions
+
+- Queries: `1-1-order-by-subquery-profile-field`, `5-mega-user-engagement-analytics`
+- **Required Addition:** ORDER BY with expressions and NULLS FIRST/LAST
+
+### 9. DISTINCT ON (PostgreSQL-style)
+
+**Missing:** No DISTINCT ON support
+
+- **Required Addition:** Dialect-specific DISTINCT ON compilation
+
+### 10. Subquery Aliasing
+
+**Missing:** No support for subqueries as derived tables
+
+- **Required Addition:** Derived table AST node
+
+### 11. Advanced EXISTS Patterns
+
+**Missing:** EXISTS with complex correlated subqueries
+
+- Queries: `1-1-boolean-flag-from-subquery-on-profile`
+- **Required Addition:** Complex correlation support
+
+## Priority Implementation Order
+
+### Completed ✅
+
+- ~~CTE support~~ (Completed)
+- ~~Window functions~~ (Completed)
+- ~~Recursive CTEs~~ (Completed)
+
+### High Priority:
+
+- Parameterized queries
+- RIGHT JOIN
+- Complex aggregation functions (MIN, MAX, GROUP_CONCAT)
+
+### Medium Priority:
+
+- Advanced JSON operations
+- Complex ordering (expressions, NULLS FIRST/LAST)
+
+### Lower Priority:
+
+- Subquery Aliasing (Derived tables)
+- Advanced EXISTS Patterns
+
+## Implementation Notes
+
+The current ORM is well-architected with a clear AST structure and dialect abstraction, making it relatively straightforward to add these missing features by extending the existing patterns.

package/metadata.json

@@ -0,0 +1,5 @@
+{
+  "name": "metal-orm",
+  "description": "A high-performance, close-to-metal TypeScript ORM playground and documentation site, demonstrating AST-based query building and dialect compilation.",
+  "requestFramePermissions": []
+}

package/package.json

@@ -0,0 +1,45 @@
+{
+  "name": "metal-orm",
+  "version": "1.0.0",
+  "type": "module",
+  "exports": {
+    ".": {
+      "import": "./dist/index.mjs",
+      "require": "./dist/index.js"
+    }
+  },
+  "scripts": {
+    "build": "tsup",
+    "check": "tsc --noEmit",
+    "playground": "vite",
+    "test": "vitest",
+    "test:ui": "vitest --ui"
+  },
+  "dependencies": {
+    "@mantine/core": "^8.3.9",
+    "@mantine/hooks": "^8.3.9",
+    "@tabler/icons-react": "^3.35.0",
+    "@types/react-syntax-highlighter": "^15.5.13",
+    "mysql2": "^3.9.0",
+    "postcss": "^8.5.6",
+    "postcss-preset-mantine": "^1.18.0",
+    "react": "^18.0.0",
+    "react-dom": "^18.0.0",
+    "react-syntax-highlighter": "^16.1.0",
+    "sqlite3": "^5.1.6",
+    "tarn": "^3.0.0",
+    "tedious": "^16.0.0"
+  },
+  "devDependencies": {
+    "@types/react": "^18.0.0",
+    "@types/react-dom": "^18.0.0",
+    "@types/sqlite3": "^3.1.11",
+    "@vitejs/plugin-react": "^4.0.0",
+    "@vitest/ui": "^4.0.14",
+    "tsconfig-paths": "^4.2.0",
+    "tsup": "^8.0.0",
+    "typescript": "^5.0.0",
+    "vite": "^5.0.0",
+    "vitest": "^4.0.14"
+  }
+}

package/playground/api/playground-api.ts

@@ -0,0 +1,94 @@
+import { IncomingMessage, ServerResponse } from 'node:http';
+import { SqliteClient } from '../../src/playground/features/playground/clients/SqliteClient';
+import { SCENARIOS } from '../../src/playground/features/playground/data/scenarios';
+import { QueryExecutionService } from '../../src/playground/features/playground/services/QueryExecutionService';
+import type { ApiStatusResponse } from '../../src/playground/features/playground/api/types';
+
+export const PLAYGROUND_API_PREFIX = '/api/playground';
+
+const sqliteClient = new SqliteClient();
+const queryExecutionService = new QueryExecutionService(sqliteClient);
+
+const sendJson = (res: ServerResponse, payload: unknown, status = 200) => {
+  res.statusCode = status;
+  res.setHeader('Content-Type', 'application/json');
+  res.end(JSON.stringify(payload));
+};
+
+const parseJsonBody = (req: IncomingMessage) =>
+  new Promise<Record<string, unknown>>((resolve, reject) => {
+    const chunks: Buffer[] = [];
+    req.on('data', (chunk) => chunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk)));
+    req.on('end', () => {
+      const body = Buffer.concat(chunks).toString('utf8');
+      if (!body) {
+        resolve({});
+        return;
+      }
+
+      try {
+        resolve(JSON.parse(body));
+      } catch (error) {
+        reject(error);
+      }
+    });
+    req.on('error', reject);
+  });
+
+const handleStatus = (_req: IncomingMessage, res: ServerResponse) => {
+  const body: ApiStatusResponse = {
+    ready: sqliteClient.isReady,
+    error: sqliteClient.error
+  };
+  sendJson(res, body);
+};
+
+const handleExecute = async (req: IncomingMessage, res: ServerResponse) => {
+  try {
+    const { scenarioId } = await parseJsonBody(req);
+    if (!scenarioId || typeof scenarioId !== 'string') {
+      sendJson(res, { error: 'Missing scenarioId' }, 400);
+      return;
+    }
+
+    const scenario = SCENARIOS.find(s => s.id === scenarioId);
+    if (!scenario) {
+      sendJson(res, { error: `Scenario '${scenarioId}' not found` }, 404);
+      return;
+    }
+
+    const result = await queryExecutionService.executeScenario(scenario);
+    sendJson(res, result);
+  } catch (error) {
+    const message = error instanceof Error ? error.message : 'Unknown error';
+    sendJson(res, { error: message }, 500);
+  }
+};
+
+export const createPlaygroundApiMiddleware = () => {
+  return async (req: IncomingMessage, res: ServerResponse, next: () => void) => {
+    const pathname = req.url ? req.url.split('?')[0] : '';
+
+    if (req.method === 'GET' && pathname === '/status') {
+      handleStatus(req, res);
+      return;
+    }
+
+    if (req.method === 'POST' && pathname === '/execute') {
+      await handleExecute(req, res);
+      return;
+    }
+
+    next();
+  };
+};
+
+export const playgroundApiPlugin = () => ({
+  name: 'metal-orm-playground-api',
+  configureServer(server) {
+    server.middlewares.use(PLAYGROUND_API_PREFIX, createPlaygroundApiMiddleware());
+  },
+  configurePreviewServer(server) {
+    server.middlewares.use(PLAYGROUND_API_PREFIX, createPlaygroundApiMiddleware());
+  },
+});

package/playground/index.html

@@ -0,0 +1,15 @@
+<!DOCTYPE html>
+<html lang="en">
+
+<head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Metal ORM Playground</title>
+</head>
+
+<body>
+    <div id="root"></div>
+    <script type="module" src="/src/main.tsx"></script>
+</body>
+
+</html>

package/playground/src/App.css

@@ -0,0 +1 @@
+/* App.css is no longer needed as we are using Mantine UI */

package/playground/src/App.tsx

@@ -0,0 +1,114 @@
+import { useState, useEffect, useMemo } from 'react';
+import { MantineProvider, AppShell, Burger, Group, Title, Text, ActionIcon, useMantineTheme } from '@mantine/core';
+import { useDisclosure } from '@mantine/hooks';
+import { SCENARIOS, type Scenario } from './data/scenarios';
+import { PlaygroundApiService } from './services/PlaygroundApiService';
+import { ScenarioList } from './components/ScenarioList';
+import { QueryExecutor } from './components/QueryExecutor';
+import '@mantine/core/styles.css';
+import './App.css'; // Keeping for custom overrides if needed, but will likely remove
+
+function App() {
+    const [selectedScenario, setSelectedScenario] = useState<Scenario | null>(null);
+    const [apiReady, setApiReady] = useState(false);
+    const [statusMessage, setStatusMessage] = useState<string | null>(null);
+    const [opened, { toggle }] = useDisclosure();
+
+    const queryService = useMemo(() => new PlaygroundApiService(), []);
+
+    useEffect(() => {
+        let isMounted = true;
+        let timerId: ReturnType<typeof setTimeout> | null = null;
+
+        const pollStatus = async () => {
+            const status = await queryService.getStatus();
+            if (!isMounted) return;
+
+            if (status.error) {
+                setStatusMessage(status.error);
+            } else {
+                setStatusMessage(null);
+            }
+
+            if (status.ready) {
+                setApiReady(true);
+                timerId && clearTimeout(timerId);
+                return;
+            }
+
+            timerId = setTimeout(pollStatus, 250);
+        };
+
+        pollStatus();
+
+        return () => {
+            isMounted = false;
+            if (timerId) {
+                clearTimeout(timerId);
+            }
+        };
+    }, [queryService]);
+
+    // Auto-select first scenario when API is ready
+    useEffect(() => {
+        if (apiReady && !selectedScenario && SCENARIOS.length > 0) {
+            setSelectedScenario(SCENARIOS[0]);
+        }
+    }, [apiReady, selectedScenario]);
+
+    const handleScenarioSelect = (scenario: Scenario) => {
+        setSelectedScenario(scenario);
+        if (window.innerWidth < 768) {
+            toggle(); // Close sidebar on mobile selection
+        }
+    };
+
+    if (!apiReady) {
+        return (
+            <MantineProvider defaultColorScheme="dark">
+                <div style={{ display: 'flex', alignItems: 'center', justifyContent: 'center', height: '100vh', flexDirection: 'column' }}>
+                    <Title order={2}>Initializing Metal ORM Playground</Title>
+                    <Text c="dimmed">Waiting for the playground API to become ready...</Text>
+                    {statusMessage && <Text c="red">Error: {statusMessage}</Text>}
+                </div>
+            </MantineProvider>
+        );
+    }
+
+    return (
+        <MantineProvider defaultColorScheme="dark">
+            <AppShell
+                header={{ height: 60 }}
+                navbar={{ width: 300, breakpoint: 'sm', collapsed: { mobile: !opened } }}
+                padding="md"
+            >
+                <AppShell.Header>
+                    <Group h="100%" px="md">
+                        <Burger opened={opened} onClick={toggle} hiddenFrom="sm" size="sm" />
+                        <Group justify="space-between" style={{ flex: 1 }}>
+                            <Title order={3}>⚡ Metal ORM Playground</Title>
+                            <Text size="sm" c="dimmed" visibleFrom="sm">Explore and test ORM query scenarios</Text>
+                        </Group>
+                    </Group>
+                </AppShell.Header>
+
+                <AppShell.Navbar p="md">
+                    <ScenarioList
+                        scenarios={SCENARIOS}
+                        selectedId={selectedScenario?.id || null}
+                        onSelect={handleScenarioSelect}
+                    />
+                </AppShell.Navbar>
+
+                <AppShell.Main>
+                    <QueryExecutor
+                        scenario={selectedScenario}
+                        queryService={queryService}
+                    />
+                </AppShell.Main>
+            </AppShell>
+        </MantineProvider>
+    );
+}
+
+export default App;

package/playground/src/components/CodeDisplay.tsx

@@ -0,0 +1,43 @@
+import React from 'react';
+import { Prism as SyntaxHighlighter } from 'react-syntax-highlighter';
+import { oneDark } from 'react-syntax-highlighter/dist/esm/styles/prism';
+import { ScrollArea } from '@mantine/core';
+
+interface CodeDisplayProps {
+    code: string;
+    language?: 'sql' | 'typescript';
+    title?: string;
+}
+
+/**
+ * Component responsible for displaying syntax-highlighted code
+ * Follows SRP by handling only code display formatting
+ */
+export const CodeDisplay: React.FC<CodeDisplayProps> = ({
+    code,
+    language = 'sql',
+    title
+}) => {
+    return (
+        <ScrollArea bg="#282c34">
+            <SyntaxHighlighter
+                language={language}
+                style={oneDark}
+                customStyle={{
+                    margin: 0,
+                    padding: '1.25rem',
+                    background: 'transparent',
+                    fontSize: '0.9rem',
+                    lineHeight: '1.6'
+                }}
+                codeTagProps={{
+                    style: {
+                        fontFamily: 'JetBrains Mono, Fira Code, monospace'
+                    }
+                }}
+            >
+                {code}
+            </SyntaxHighlighter>
+        </ScrollArea>
+    );
+};

package/playground/src/components/QueryExecutor.tsx

@@ -0,0 +1,189 @@
+import React, { useState, useEffect } from 'react';
+import { Card, Button, Title, Text, Tabs, Loader, Group, Stack, Badge, ActionIcon, CopyButton, Tooltip } from '@mantine/core';
+import { IconPlayerPlay, IconCode, IconDatabase, IconCheck, IconCopy } from '@tabler/icons-react'; // Assuming tabler icons are available or we use text
+import type { Scenario } from '../data/scenarios';
+import type { QueryExecutionResult } from '@orm/playground/features/playground/api/types';
+import { CodeDisplay } from './CodeDisplay';
+import { ResultsTabs } from './ResultsTabs';
+import { PlaygroundApiService } from '../services/PlaygroundApiService';
+
+const describeBinding = (value: unknown): { display: string; type: string } => {
+    if (value === null) {
+        return { display: 'null', type: 'null' };
+    }
+    if (typeof value === 'undefined') {
+        return { display: 'undefined', type: 'undefined' };
+    }
+    if (typeof value === 'string') {
+        return { display: `"${value}"`, type: 'string' };
+    }
+    if (typeof value === 'number') {
+        return { display: value.toString(), type: Number.isInteger(value) ? 'integer' : 'number' };
+    }
+    if (typeof value === 'boolean') {
+        return { display: value ? 'true' : 'false', type: 'boolean' };
+    }
+    if (Array.isArray(value)) {
+        return { display: JSON.stringify(value), type: 'array' };
+    }
+    return { display: JSON.stringify(value), type: typeof value };
+};
+
+const BindingsDisplay: React.FC<{ params: unknown[] }> = ({ params }) => {
+    const hasParams = Array.isArray(params) && params.length > 0;
+
+    return (
+        <div
+            style={{
+                border: '1px solid var(--mantine-color-dark-5)',
+                borderRadius: 'var(--mantine-radius-md)',
+                padding: 'var(--mantine-spacing-md)',
+                background: 'var(--mantine-color-dark-8)'
+            }}
+        >
+            <Group justify="space-between" mb="sm">
+                <Text fw={500}>Bindings</Text>
+                {hasParams && (
+                    <CopyButton value={JSON.stringify(params, null, 2)}>
+                        {({ copied, copy }) => (
+                            <Tooltip label={copied ? 'Copied' : 'Copy JSON'} withArrow>
+                                <ActionIcon variant="subtle" color={copied ? 'teal' : 'blue'} onClick={copy}>
+                                    {copied ? <IconCheck size={14} /> : <IconCopy size={14} />}
+                                </ActionIcon>
+                            </Tooltip>
+                        )}
+                    </CopyButton>
+                )}
+            </Group>
+            {!hasParams && <Text c="dimmed" fz="sm">Query has no parameter bindings.</Text>}
+            {hasParams && (
+                <Stack gap="xs">
+                    {params.map((value, index) => {
+                        const { display, type } = describeBinding(value);
+                        return (
+                        <Group
+                            key={`${index}-${String(value)}`}
+                            justify="space-between"
+                            style={{
+                                border: '1px solid var(--mantine-color-dark-5)',
+                                borderRadius: 'var(--mantine-radius-sm)',
+                                padding: 'var(--mantine-spacing-xs)'
+                            }}
+                        >
+                            <Badge variant="light" color="grape">#{index + 1}</Badge>
+                            <div style={{ textAlign: 'right' }}>
+                                <Text style={{ fontFamily: 'var(--mantine-font-family-monospace)' }}>
+                                    {display}
+                                </Text>
+                                <Text fz="xs" c="dimmed">
+                                    {type.toUpperCase()}
+                                </Text>
+                            </div>
+                        </Group>
+                        );
+                    })}
+                </Stack>
+            )}
+        </div>
+    );
+};
+
+interface QueryExecutorProps {
+    scenario: Scenario | null;
+    queryService: PlaygroundApiService;
+}
+
+/**
+ * Component responsible for executing queries and displaying results
+ * Follows SRP by coordinating query execution and result display
+ */
+export const QueryExecutor: React.FC<QueryExecutorProps> = ({
+    scenario,
+    queryService
+}) => {
+    const [result, setResult] = useState<QueryExecutionResult | null>(null);
+    const [isLoading, setIsLoading] = useState(false);
+
+    useEffect(() => {
+        if (scenario) {
+            executeQuery();
+        }
+    }, [scenario]);
+
+    const executeQuery = async () => {
+        if (!scenario) return;
+
+        setIsLoading(true);
+        const executionResult = await queryService.executeScenario(scenario.id);
+        setResult(executionResult);
+        setIsLoading(false);
+    };
+
+    if (!scenario) {
+        return (
+            <div style={{ display: 'flex', alignItems: 'center', justifyContent: 'center', height: '100%', color: 'var(--mantine-color-dimmed)' }}>
+                <Text>Select a scenario from the list to execute</Text>
+            </div>
+        );
+    }
+
+    return (
+        <Stack gap="lg">
+            <div>
+                <Group justify="space-between" align="start" mb="md">
+                    <div>
+                        <Title order={2}>{scenario.title}</Title>
+                        <Text c="dimmed" mt="xs">{scenario.description}</Text>
+                    </div>
+                    <Button
+                        onClick={executeQuery}
+                        loading={isLoading}
+                        leftSection={<span>▶</span>}
+                        variant="gradient"
+                        gradient={{ from: 'indigo', to: 'cyan' }}
+                    >
+                        Re-execute Query
+                    </Button>
+                </Group>
+            </div>
+
+            {result && (
+                <>
+                    <Card withBorder shadow="sm" radius="md" p={0}>
+                        <Tabs defaultValue="sql">
+                            <Tabs.List>
+                                <Tabs.Tab value="sql" leftSection={<span>SQL</span>}>Generated SQL</Tabs.Tab>
+                                <Tabs.Tab value="typescript" leftSection={<span>TS</span>}>TypeScript</Tabs.Tab>
+                            </Tabs.List>
+
+                            <Tabs.Panel value="sql">
+                                <Stack gap="sm" p="md">
+                                    <CodeDisplay code={result.sql} language="sql" />
+                                    <BindingsDisplay params={result.params} />
+                                </Stack>
+                            </Tabs.Panel>
+
+                            <Tabs.Panel value="typescript">
+                                <CodeDisplay code={result.typescriptCode} language="typescript" />
+                            </Tabs.Panel>
+                        </Tabs>
+                    </Card>
+
+                    <ResultsTabs
+                        results={result.results}
+                        hydratedResults={result.hydratedResults}
+                        executionTime={result.executionTime}
+                        error={result.error}
+                    />
+                </>
+            )}
+
+            {isLoading && (
+                <Group justify="center" p="xl">
+                    <Loader type="dots" />
+                    <Text>Executing query...</Text>
+                </Group>
+            )}
+        </Stack>
+    );
+};