@posthog/hogql-parser 0.0.1 → 0.0.3

package/README.md

@@ -1,45 +1,98 @@
-# @posthog/hogql-parser
+# HogQL Parser
 
-## ⚠️ IMPORTANT NOTICE ⚠️
+ANTLR4-based parser for [HogQL](https://posthog.com/docs/hogql) and [Hog](https://posthog.com/docs/hog),
+available as both a Python C++ extension and a WebAssembly module for JavaScript/TypeScript.
 
-**This package is created solely for the purpose of setting up OIDC (OpenID Connect) trusted publishing with npm.**
+## Packages
 
-This is **NOT** a functional package and contains **NO** code or functionality beyond the OIDC setup configuration.
+| Package                 | Runtime                          | Registry                                                   |
+| ----------------------- | -------------------------------- | ---------------------------------------------------------- |
+| `hogql_parser`          | CPython (native C++ extension)   | [PyPI](https://pypi.org/project/hogql-parser/)             |
+| `@posthog/hogql-parser` | Any JS environment (WebAssembly) | [npm](https://www.npmjs.com/package/@posthog/hogql-parser) |
 
-## Purpose
+Both packages share the same C++ parser core and ANTLR4 grammar,
+so they produce identical ASTs.
 
-This package exists to:
-1. Configure OIDC trusted publishing for the package name `@posthog/hogql-parser`
-2. Enable secure, token-less publishing from CI/CD workflows
-3. Establish provenance for packages published under this name
+## npm package (`@posthog/hogql-parser`)
 
-## What is OIDC Trusted Publishing?
+### Installation
 
-OIDC trusted publishing allows package maintainers to publish packages directly from their CI/CD workflows without needing to manage npm access tokens. Instead, it uses OpenID Connect to establish trust between the CI/CD provider (like GitHub Actions) and npm.
+```bash
+npm install @posthog/hogql-parser
+```
 
-## Setup Instructions
+### Usage
 
-To properly configure OIDC trusted publishing for this package:
+```typescript
+import createHogQLParser from '@posthog/hogql-parser'
 
-1. Go to [npmjs.com](https://www.npmjs.com/) and navigate to your package settings
-2. Configure the trusted publisher (e.g., GitHub Actions)
-3. Specify the repository and workflow that should be allowed to publish
-4. Use the configured workflow to publish your actual package
+const parser = await createHogQLParser()
 
-## DO NOT USE THIS PACKAGE
+// Parse a HogQL expression
+const exprAST = JSON.parse(parser.parseExpr('1 + 2'))
 
-This package is a placeholder for OIDC configuration only. It:
-- Contains no executable code
-- Provides no functionality
-- Should not be installed as a dependency
-- Exists only for administrative purposes
+// Parse a SELECT statement
+const selectAST = JSON.parse(parser.parseSelect('SELECT event FROM events WHERE timestamp > now()'))
 
-## More Information
+// Parse a Hog program
+const programAST = JSON.parse(parser.parseProgram('let x := 42; return x;'))
+```
 
-For more details about npm's trusted publishing feature, see:
-- [npm Trusted Publishing Documentation](https://docs.npmjs.com/generating-provenance-statements)
-- [GitHub Actions OIDC Documentation](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect)
+All parse functions return JSON strings.
+On failure they return a JSON error object instead of throwing:
 
----
+```typescript
+const result = JSON.parse(parser.parseExpr('!!!'))
+if ('error' in result) {
+  console.error(result.type, result.message) // e.g. "SyntaxError ..."
+}
+```
 
-**Maintained for OIDC setup purposes only**
+### API
+
+| Method                                        | Description                        |
+| --------------------------------------------- | ---------------------------------- |
+| `parseExpr(input, isInternal?)`               | Parse a HogQL expression           |
+| `parseSelect(input, isInternal?)`             | Parse a SELECT statement           |
+| `parseOrderExpr(input, isInternal?)`          | Parse an ORDER BY expression       |
+| `parseProgram(input, isInternal?)`            | Parse a Hog program                |
+| `parseFullTemplateString(input, isInternal?)` | Parse a template string (`f'...'`) |
+| `parseStringLiteralText(input)`               | Unquote a string literal           |
+
+Setting `isInternal` to `true` omits position information from the AST.
+
+## Python package (`hogql_parser`)
+
+### Installation
+
+```bash
+pip install hogql_parser
+```
+
+The Python package is a native C++ extension and requires a platform with prebuilt wheels
+(macOS and Linux, x86_64 and arm64).
+
+### Local development
+
+```bash
+pip install ./common/hogql_parser
+```
+
+## Building from source
+
+### Python
+
+```bash
+pip install ./common/hogql_parser
+```
+
+### WebAssembly
+
+Requires the [Emscripten](https://emscripten.org/) toolchain and [Ninja](https://ninja-build.org/).
+
+```bash
+cd common/hogql_parser
+npm run build
+```
+
+This compiles the parser to WASM and places the output in `dist/`.

package/dist/index.cjs

@@ -0,0 +1,5 @@
+// CommonJS wrapper for the ES module WASM build
+// This allows Jest and other CommonJS environments to import the module
+
+module.exports = () => import('./hogql_parser_wasm.js').then(m => m.default);
+module.exports.default = module.exports;

package/dist/index.d.ts

@@ -0,0 +1,157 @@
+/**
+ * HogQL Parser WebAssembly Module
+ *
+ * High-performance HogQL parser compiled to WebAssembly.
+ * All parse functions return JSON strings that should be parsed with JSON.parse().
+ */
+
+/**
+ * AST Position information
+ */
+export interface Position {
+    line: number
+    column: number
+    offset: number
+}
+
+/**
+ * Base AST node with position information
+ */
+export interface ASTNode {
+    node: string
+    start: Position
+    end: Position
+    [key: string]: any
+}
+
+/**
+ * Parse error object
+ */
+export interface ParseError {
+    error: true
+    type: 'SyntaxError' | 'ParsingError' | 'NotImplementedError'
+    message: string
+    start: Position
+    end: Position
+}
+
+/**
+ * Parse result - either an AST node or an error
+ */
+export type ParseResult = ASTNode | ParseError
+
+/**
+ * HogQL Parser instance
+ */
+export interface HogQLParser {
+    /**
+     * Parse a HogQL expression
+     *
+     * @param input - The HogQL expression string to parse
+     * @param isInternal - If true, omits position information from the AST (default: false)
+     * @returns JSON string representing the AST or error
+     *
+     * @example
+     * ```typescript
+     * const result = parser.parseExpr('user_id + 100');
+     * const ast = JSON.parse(result);
+     * ```
+     */
+    parseExpr(input: string, isInternal?: boolean): string
+
+    /**
+     * Parse a complete SELECT statement
+     *
+     * @param input - The SELECT statement string to parse
+     * @param isInternal - If true, omits position information from the AST (default: false)
+     * @returns JSON string representing the AST or error
+     *
+     * @example
+     * ```typescript
+     * const result = parser.parseSelect('SELECT * FROM events WHERE timestamp > now()');
+     * const ast = JSON.parse(result);
+     * ```
+     */
+    parseSelect(input: string, isInternal?: boolean): string
+
+    /**
+     * Parse an ORDER BY expression
+     *
+     * @param input - The ORDER BY expression string to parse
+     * @param isInternal - If true, omits position information from the AST (default: false)
+     * @returns JSON string representing the AST or error
+     *
+     * @example
+     * ```typescript
+     * const result = parser.parseOrderExpr('timestamp DESC, user_id ASC');
+     * const ast = JSON.parse(result);
+     * ```
+     */
+    parseOrderExpr(input: string, isInternal?: boolean): string
+
+    /**
+     * Parse a complete Hog program
+     *
+     * @param input - The Hog program string to parse
+     * @param isInternal - If true, omits position information from the AST (default: false)
+     * @returns JSON string representing the AST or error
+     *
+     * @example
+     * ```typescript
+     * const result = parser.parseProgram('let x := 42; return x;');
+     * const ast = JSON.parse(result);
+     * ```
+     */
+    parseProgram(input: string, isInternal?: boolean): string
+
+    /**
+     * Parse a Hog template string (f'...' syntax)
+     *
+     * @param input - The template string to parse
+     * @param isInternal - If true, omits position information from the AST (default: false)
+     * @returns JSON string representing the AST or error
+     *
+     * @example
+     * ```typescript
+     * const result = parser.parseFullTemplateString("f'Hello {name}'");
+     * const ast = JSON.parse(result);
+     * ```
+     */
+    parseFullTemplateString(input: string, isInternal?: boolean): string
+
+    /**
+     * Unquote a string literal
+     *
+     * @param input - The quoted string literal
+     * @returns The unquoted string content
+     *
+     * @example
+     * ```typescript
+     * const text = parser.parseStringLiteralText("'hello world'");
+     * // Returns: "hello world"
+     * ```
+     */
+    parseStringLiteralText(input: string): string
+}
+
+/**
+ * Factory function to create a HogQL Parser instance
+ *
+ * @returns Promise that resolves to a HogQLParser instance
+ *
+ * @example
+ * ```typescript
+ * import createHogQLParser from '@posthog/hogql-parser';
+ *
+ * const parser = await createHogQLParser();
+ * const result = parser.parseExpr('1 + 2');
+ * const ast: ParseResult = JSON.parse(result);
+ *
+ * if ('error' in ast) {
+ *   console.error('Parse error:', ast.message);
+ * } else {
+ *   console.log('Parsed successfully:', ast);
+ * }
+ * ```
+ */
+export default function createHogQLParser(): Promise<HogQLParser>

package/package.json

@@ -1,10 +1,44 @@
 {
-  "name": "@posthog/hogql-parser",
-  "version": "0.0.1",
-  "description": "OIDC trusted publishing setup package for @posthog/hogql-parser",
-  "keywords": [
-    "oidc",
-    "trusted-publishing",
-    "setup"
-  ]
+    "name": "@posthog/hogql-parser",
+    "version": "0.0.3",
+    "description": "WebAssembly HogQL Parser - Parse HogQL queries in JavaScript/TypeScript",
+    "type": "module",
+    "main": "dist/index.cjs",
+    "module": "dist/hogql_parser_wasm.js",
+    "types": "dist/index.d.ts",
+    "exports": {
+        ".": {
+            "types": "./dist/index.d.ts",
+            "import": "./dist/hogql_parser_wasm.js",
+            "require": "./dist/index.cjs"
+        }
+    },
+    "files": [
+        "dist/hogql_parser_wasm.js",
+        "dist/index.cjs",
+        "dist/index.d.ts"
+    ],
+    "scripts": {
+        "build:compile": "emmake cmake --build build_wasm --config Release && cmake --install build_wasm",
+        "build:make": "emcmake cmake -S . -B build_wasm -DCMAKE_BUILD_TYPE=Release && npm run build:compile || echo 'Warning: hogql-parser build failed, skipping'",
+        "build": "emcmake cmake -S . -B build_wasm -G Ninja -DCMAKE_BUILD_TYPE=Release && npm run build:compile || echo 'Warning: hogql-parser build failed, skipping'"
+    },
+    "keywords": [
+        "hogql",
+        "parser",
+        "wasm",
+        "webassembly",
+        "antlr4"
+    ],
+    "author": "PostHog Inc.",
+    "license": "MIT",
+    "repository": {
+        "type": "git",
+        "url": "https://github.com/PostHog/posthog.git",
+        "directory": "common/hogql_parser"
+    },
+    "bugs": {
+        "url": "https://github.com/PostHog/posthog/issues"
+    },
+    "homepage": "https://github.com/PostHog/posthog/tree/master/common/hogql_parser"
 }