@graphenedata/cli 0.0.14 → 0.0.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (121) hide show
  1. package/LICENSE.md +3 -3
  2. package/README.md +138 -0
  3. package/THIRD_PARTY_NOTICES.md +1 -0
  4. package/bin.js +2 -2
  5. package/dist/cli/bigQuery-I3F46SC6.js +75 -0
  6. package/dist/cli/bigQuery-I3F46SC6.js.map +7 -0
  7. package/dist/cli/chunk-OVWODUTJ.js +12849 -0
  8. package/dist/cli/chunk-OVWODUTJ.js.map +7 -0
  9. package/dist/cli/chunk-QAXEOZ43.js +53 -0
  10. package/dist/cli/chunk-QAXEOZ43.js.map +7 -0
  11. package/dist/cli/cli.js +245 -10290
  12. package/dist/cli/clickhouse-ZN5AN2UL.js +64 -0
  13. package/dist/cli/clickhouse-ZN5AN2UL.js.map +7 -0
  14. package/dist/cli/duckdb-IYBIO5KJ.js +87 -0
  15. package/dist/cli/duckdb-IYBIO5KJ.js.map +7 -0
  16. package/dist/cli/serve2-TNN5EROW.js +447 -0
  17. package/dist/cli/serve2-TNN5EROW.js.map +7 -0
  18. package/dist/cli/snowflake-MOQB5GA4.js +128 -0
  19. package/dist/cli/snowflake-MOQB5GA4.js.map +7 -0
  20. package/dist/index.d.ts +63 -0
  21. package/dist/lang/index.d.ts +63 -0
  22. package/dist/skills/graphene/SKILL.md +235 -0
  23. package/dist/skills/graphene/references/big-value.md +20 -0
  24. package/dist/skills/graphene/references/date-range.md +64 -0
  25. package/dist/skills/graphene/references/dropdown.md +62 -0
  26. package/dist/skills/graphene/references/echarts.md +162 -0
  27. package/dist/skills/graphene/references/gsql.md +393 -0
  28. package/dist/skills/graphene/references/model-gsql.md +72 -0
  29. package/dist/skills/graphene/references/table.md +143 -0
  30. package/dist/skills/graphene/references/text-input.md +29 -0
  31. package/dist/ui/app.css +263 -299
  32. package/dist/ui/component-utilities/dataShaping.ts +484 -0
  33. package/dist/ui/component-utilities/dataSummary.ts +57 -0
  34. package/dist/ui/component-utilities/enrich.ts +763 -0
  35. package/dist/ui/component-utilities/format.ts +177 -0
  36. package/dist/ui/component-utilities/inputUtils.ts +48 -9
  37. package/dist/ui/component-utilities/theme.ts +200 -0
  38. package/dist/ui/component-utilities/themeStores.ts +26 -21
  39. package/dist/ui/component-utilities/types.ts +70 -0
  40. package/dist/ui/components/AreaChart.svelte +57 -105
  41. package/dist/ui/components/BarChart.svelte +71 -129
  42. package/dist/ui/components/BigValue.svelte +24 -40
  43. package/dist/ui/components/Column.svelte +11 -19
  44. package/dist/ui/components/DateRange.svelte +71 -34
  45. package/dist/ui/components/Dropdown.svelte +82 -49
  46. package/dist/ui/components/DropdownOption.svelte +1 -2
  47. package/dist/ui/components/ECharts.svelte +179 -60
  48. package/dist/ui/components/InlineDelta.svelte +51 -32
  49. package/dist/ui/components/LineChart.svelte +54 -125
  50. package/dist/ui/components/PieChart.svelte +27 -37
  51. package/dist/ui/components/QueryLoad.svelte +78 -44
  52. package/dist/ui/components/Row.svelte +2 -1
  53. package/dist/ui/components/ScatterPlot.svelte +52 -0
  54. package/dist/ui/components/Skeleton.svelte +32 -0
  55. package/dist/ui/components/Table.svelte +3 -2
  56. package/dist/ui/components/TableGroupRow.svelte +28 -36
  57. package/dist/ui/components/TableHarness.svelte +32 -0
  58. package/dist/ui/components/TableHeader.svelte +34 -59
  59. package/dist/ui/components/TableRow.svelte +15 -39
  60. package/dist/ui/components/TableSubtotalRow.svelte +26 -21
  61. package/dist/ui/components/TableTotalRow.svelte +27 -37
  62. package/dist/ui/components/TextInput.svelte +17 -14
  63. package/dist/ui/components/Value.svelte +25 -0
  64. package/dist/ui/components/_Table.svelte +80 -76
  65. package/dist/ui/internal/ChartGallery.svelte +527 -0
  66. package/dist/ui/internal/ErrorDisplay.svelte +60 -0
  67. package/dist/ui/internal/LocalApp.svelte +87 -19
  68. package/dist/ui/internal/PageNavGroup.svelte +269 -0
  69. package/dist/ui/internal/Sidebar.svelte +178 -0
  70. package/dist/ui/internal/SidebarToggle.svelte +47 -0
  71. package/dist/ui/internal/StyleGallery.svelte +244 -0
  72. package/dist/ui/internal/clientCache.ts +15 -13
  73. package/dist/ui/internal/pageInputs.svelte.js +292 -0
  74. package/dist/ui/internal/queryEngine.ts +124 -132
  75. package/dist/ui/internal/runSocket.ts +59 -0
  76. package/dist/ui/internal/sidebar.svelte.js +18 -0
  77. package/dist/ui/internal/telemetry.ts +52 -17
  78. package/dist/ui/internal/types.d.ts +7 -0
  79. package/dist/ui/web.js +55 -13
  80. package/package.json +40 -41
  81. package/dist/docs/agent-instructions.md +0 -18
  82. package/dist/docs/base.md +0 -98
  83. package/dist/docs/cli.md +0 -22
  84. package/dist/docs/graphene.md +0 -1462
  85. package/dist/ui/component-utilities/autoFormatting.js +0 -301
  86. package/dist/ui/component-utilities/builtInFormats.js +0 -482
  87. package/dist/ui/component-utilities/chartContext.js +0 -12
  88. package/dist/ui/component-utilities/chartWindowDebug.js +0 -21
  89. package/dist/ui/component-utilities/checkInputs.js +0 -95
  90. package/dist/ui/component-utilities/convert.js +0 -15
  91. package/dist/ui/component-utilities/dateParsing.js +0 -57
  92. package/dist/ui/component-utilities/dropdownContext.ts +0 -1
  93. package/dist/ui/component-utilities/echarts.js +0 -272
  94. package/dist/ui/component-utilities/echartsThemes.js +0 -453
  95. package/dist/ui/component-utilities/formatTitle.js +0 -24
  96. package/dist/ui/component-utilities/formatting.js +0 -250
  97. package/dist/ui/component-utilities/getColumnExtents.js +0 -79
  98. package/dist/ui/component-utilities/getColumnSummary.js +0 -67
  99. package/dist/ui/component-utilities/getCompletedData.js +0 -114
  100. package/dist/ui/component-utilities/getDistinctCount.js +0 -7
  101. package/dist/ui/component-utilities/getDistinctValues.js +0 -15
  102. package/dist/ui/component-utilities/getSeriesConfig.js +0 -237
  103. package/dist/ui/component-utilities/getSortedData.js +0 -7
  104. package/dist/ui/component-utilities/getStackPercentages.js +0 -43
  105. package/dist/ui/component-utilities/getStackedData.js +0 -17
  106. package/dist/ui/component-utilities/getYAxisIndex.js +0 -15
  107. package/dist/ui/component-utilities/globalContexts.js +0 -1
  108. package/dist/ui/component-utilities/helpers/getCompletedData.helpers.js +0 -119
  109. package/dist/ui/component-utilities/replaceNulls.js +0 -14
  110. package/dist/ui/component-utilities/tableUtils.ts +0 -120
  111. package/dist/ui/components/Area.svelte +0 -214
  112. package/dist/ui/components/Bar.svelte +0 -350
  113. package/dist/ui/components/Chart.svelte +0 -989
  114. package/dist/ui/components/ErrorChart.svelte +0 -118
  115. package/dist/ui/components/Line.svelte +0 -227
  116. package/dist/ui/internal/NavSidebar.svelte +0 -396
  117. package/dist/ui/internal/PageError.svelte +0 -23
  118. package/dist/ui/internal/checkSocket.ts +0 -48
  119. package/dist/ui/internal/theme.ts +0 -88
  120. package/dist/ui/public/inter-latin-ext.woff2 +0 -0
  121. package/dist/ui/public/inter-latin.woff2 +0 -0
@@ -0,0 +1,7 @@
1
+ {
2
+ "version": 3,
3
+ "sources": ["../../serve2.ts", "../../mdCompile.ts"],
4
+ "sourcesContent": ["import {svelte, vitePreprocess} from '@sveltejs/vite-plugin-svelte'\nimport crypto from 'crypto'\nimport fs from 'fs-extra'\n// import sveltePreprocess from 'svelte-preprocess' // this would be nice, but it breaks sourcemaps by default\nimport {type IncomingMessage, type ServerResponse} from 'http'\nimport {mdsvex} from 'mdsvex'\nimport {createRequire} from 'module'\nimport path from 'path'\nimport {fileURLToPath} from 'url'\nimport {createServer, type InlineConfig, optimizeDeps, resolveConfig, type ViteDevServer} from 'vite'\n\nimport type {AnalysisResult, WorkspaceFileInput} from '../lang/types.ts'\n\nimport {config} from '../lang/config.ts'\nimport {analyzeWorkspace, loadWorkspace, toSql} from '../lang/core.ts'\nimport {runQuery} from './connections/index.ts'\nimport {extractFrontmatter, injectComponentImports, remarkPlugins, rehypePlugins} from './mdCompile.ts'\nimport {mockFileMap} from './mockFiles.ts'\nimport {runVitePlugin} from './run.ts'\nimport {getWorkspaceScanCounts, type CliTelemetry} from './telemetry/index.ts'\n\n// Collect Svelte compiler warnings for test assertions\nexport type SvelteWarning = {code: string; message: string; filename?: string}\nexport const svelteWarnings: SvelteWarning[] = []\nexport function clearSvelteWarnings() {\n svelteWarnings.length = 0\n}\n\n// Bump this whenever the query response shape changes so client caches invalidate.\nconst QUERY_VERSION = 1\n\nlet uiRoot: string\nlet nodeRequire = createRequire(import.meta.url)\n\nexport async function serve2(telemetry?: CliTelemetry): Promise<ViteDevServer> {\n let server = await createServer(await createConfig(telemetry))\n // I originally added this to avoid the page refreshing immediately on load.\n // We def don't want to run it in tests, because its not safe to do in parallel.\n // I'm not sure it's still needed, now that we explicitly list out `optimizeDeps.includes`, refreshes should be rare\n // await optimizeDeps(server.config, true)\n await server.listen()\n console.log(`Server running at http://localhost:${server.config.server.port}`)\n\n return server\n}\n\nasync function createConfig(telemetry?: CliTelemetry): Promise<InlineConfig> {\n uiRoot = path.join(fileURLToPath(import.meta.url), '../../ui')\n let port = Number(process.env.GRAPHENE_PORT) || 4000\n let svelteRoot = path.dirname(nodeRequire.resolve('svelte/package.json'))\n let sveltePackage = nodeRequire('svelte/package.json')\n let svelteDependencyRoot = path.dirname(svelteRoot)\n let svelteExport = (name: string) => path.join(svelteRoot, sveltePackage.exports[name].browser || sveltePackage.exports[name].default)\n let packaged = path.basename(path.dirname(uiRoot)) == 'dist'\n await fs.ensureDir(path.resolve(config.root, 'node_modules/.graphene'))\n\n // Bind to 0.0.0.0 when running in a container so port forwarding works from the host\n let inContainer = fs.existsSync('/.dockerenv')\n let host = inContainer ? '0.0.0.0' : '127.0.0.1'\n\n return {\n root: config.root,\n logLevel: process.env.NODE_ENV == 'test' ? 'silent' : 'info',\n plugins: [\n svelte({\n configFile: false,\n extensions: ['.svelte', '.md'],\n preprocess: [\n vitePreprocess(),\n mdsvex({\n extensions: ['.md'],\n remarkPlugins,\n rehypePlugins,\n }) as any,\n injectComponentImports(),\n ],\n onwarn(warning, defaultHandler) {\n if (process.env.NODE_ENV === 'test') {\n svelteWarnings.push({code: warning.code, message: warning.message, filename: warning.filename})\n }\n defaultHandler?.(warning) // Still call the default handler to print warnings\n },\n }),\n fixSvelteDepsInTests(),\n fixHmrForFailedModules(),\n runVitePlugin(),\n handleRequestPlugin,\n updateWorkspacePlugin(telemetry),\n mockFilesForTests(),\n ],\n publicDir: path.resolve(uiRoot, 'public'),\n // on the fence about this one. This would make it less likely we need to optimize when alternating between dev and tests.\n // cacheDir: process.env.NODE_ENV == 'test' ? 'node_modules/.vite-tests' : 'node_modules/.vite',\n server: {\n port,\n host,\n fs: {strict: false},\n strictPort: true,\n hmr: {overlay: false}, // we handle compilation errors ourselves (see LocalApp.svelte)\n },\n resolve: {\n alias: [\n {find: /^graphene$/, replacement: path.resolve(uiRoot, 'web.js')},\n // Vite runs in a user project, but svelte is a direct dependency of the cli, and thus transitive to the user project.\n // So when Vite tries to resolve `svelte` from a compiled md page, it can't find it without these aliases.\n {find: /^svelte$/, replacement: svelteExport('.')},\n {find: /^svelte\\/animate$/, replacement: svelteExport('./animate')},\n {find: /^svelte\\/attachments$/, replacement: svelteExport('./attachments')},\n {find: /^svelte\\/easing$/, replacement: svelteExport('./easing')},\n {find: /^svelte\\/events$/, replacement: svelteExport('./events')},\n {find: /^svelte\\/internal$/, replacement: svelteExport('./internal')},\n {find: /^svelte\\/internal\\/client$/, replacement: svelteExport('./internal/client')},\n {find: /^svelte\\/internal\\/disclose-version$/, replacement: svelteExport('./internal/disclose-version')},\n {find: /^svelte\\/internal\\/flags\\/async$/, replacement: svelteExport('./internal/flags/async')},\n {find: /^svelte\\/internal\\/flags\\/legacy$/, replacement: svelteExport('./internal/flags/legacy')},\n {find: /^svelte\\/internal\\/flags\\/tracing$/, replacement: svelteExport('./internal/flags/tracing')},\n {find: /^svelte\\/legacy$/, replacement: svelteExport('./legacy')},\n {find: /^svelte\\/motion$/, replacement: svelteExport('./motion')},\n {find: /^svelte\\/reactivity$/, replacement: svelteExport('./reactivity')},\n {find: /^svelte\\/reactivity\\/window$/, replacement: svelteExport('./reactivity/window')},\n {find: /^svelte\\/store$/, replacement: svelteExport('./store')},\n {find: /^svelte\\/transition$/, replacement: svelteExport('./transition')},\n {find: /^clsx$/, replacement: path.join(svelteDependencyRoot, 'clsx/dist/clsx.mjs')},\n ],\n },\n\n optimizeDeps: {\n noDiscovery: process.env.NODE_ENV == 'test', // tests manually optimize before starting test workers\n exclude: ['virtual:nav'], // provided by a plugin, so don't try and optimize it\n // Vite running in a user project will not naturally discover and optimize these transitive deps.\n // When you launch the server, your first page load will automatically refresh after a second or two as Vite now sees and optimizes these.\n // This line makes it do that up-front, avoiding that reload jank. The packaged CLI also pre-bundles the `graphene` alias itself;\n // doing that from source causes trouble in examples/tests because the alias points outside node_modules.\n // `graphene` here is a special case: when packaged up it is considered a dependency, but in examples/tests, including it would cause errors.\n // oxfmt-ignore\n include: [\n ...(packaged ? ['graphene'] : []),\n '@graphenedata/cli > svelte',\n '@graphenedata/cli > chroma-js',\n '@graphenedata/cli > echarts',\n '@graphenedata/cli > @graphenedata/html2canvas',\n '@graphenedata/cli > @graphenedata/ui > svelte',\n '@graphenedata/cli > @graphenedata/ui > chroma-js',\n '@graphenedata/cli > @graphenedata/ui > echarts/dist/echarts.esm.js',\n '@graphenedata/cli > @graphenedata/ui > @graphenedata/html2canvas',\n ],\n },\n }\n}\n\nasync function handleQuery(req: IncomingMessage, res: ServerResponse<IncomingMessage>) {\n let chunks = [] as any[]\n for await (let chunk of req) chunks.push(chunk)\n let {gsql, params, hashes} = JSON.parse(Buffer.concat(chunks).toString())\n res.setHeader('Content-Type', 'application/json')\n\n await workspaceLoadPromise\n\n // queries should not analyze md files\n let gsqlFiles = workspaceFiles.filter(file => !file.path.endsWith('.md'))\n let result = analyzeWorkspace({config, files: [...gsqlFiles, {path: 'input', contents: gsql}]})\n updateParsedFiles(result)\n\n let diagnostics = result.diagnostics\n if (diagnostics.length) {\n res.statusCode = 400\n res.end(JSON.stringify(diagnostics[0]))\n return\n }\n\n let queries = result.files.find(file => file.path == 'input')?.queries || []\n if (queries.length > 1) throw new Error('Found multiple queries, which could be a parsing error')\n let sql = toSql(queries[0], params)\n\n // If the client already has this data, dont run the query\n let hash = crypto.createHash('SHA1').update(`query-v${QUERY_VERSION}|${sql}`).digest('hex')\n res.setHeader('ETag', hash)\n if (hashes.includes(hash) && req.headers['cache-control'] != 'no-cache') {\n res.statusCode = 304\n return res.end()\n }\n\n let queryResults = await runQuery(sql)\n let totalRows = queryResults.totalRows ?? queryResults.rows.length\n if (totalRows > queryResults.rows.length) throw new Error('Query returns too many rows')\n let fields = queries[0].fields.map(field => ({name: field.name, type: field.type, metadata: field.metadata || {}}))\n res.end(JSON.stringify({rows: queryResults.rows, hash, fields, sql}))\n}\n\nasync function handlePage(server: ViteDevServer, res: ServerResponse<IncomingMessage>) {\n res.setHeader('Content-Type', 'text/html')\n\n // Use a .html URL for transformIndexHtml so Vite doesn't run the svelte plugin on our HTML template.\n let html = await server.transformIndexHtml(\n '/index.html',\n `<!doctype html>\n <html lang=\"en\">\n <head>\n <meta charset=\"UTF-8\" />\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\" />\n <title>Graphene</title>\n <link rel=\"icon\" href=\"/favicon.ico\" />\n </head>\n <body>\n <script type=\"module\">\n import 'graphene'\n </script>\n </body>\n </html>`,\n )\n return res.end(html)\n}\n\n// Runs vite's pre-bundling of dependencies. Used by tests to do this once, instead of for each worker.\nexport async function prepareDeps() {\n let cfg = await resolveConfig(await createConfig(), 'serve')\n await optimizeDeps(cfg, true)\n}\n\n// Svelte forces optimizeDeps whenever its own metadata has changed.\n// For tests, we already optimizeDeps before any tests start up, so we don't need this, and it causes problems\n// if multiple workers are all trying to optimizeDeps at the same time (vite isn't exactly concurrency-safe).\nfunction fixSvelteDepsInTests() {\n let viteConfig: any\n\n function configResolved(cfg: any) {\n viteConfig = cfg\n }\n\n // This must run AFTER Svelte's buildStart which sets force=true based on metadata changes.\n // Using enforce:'post' and sequential:true ensures we run last and can override.\n function buildStart() {\n if (process.env.NODE_ENV != 'test') return\n viteConfig.optimizeDeps.force = false\n }\n buildStart.sequential = true // force running after other sequential hooks (like svelte's)\n return {name: 'fix-svelte-deps', enforce: 'post' as const, configResolved, buildStart}\n}\n\n// When a module's transform fails (e.g. Svelte compilation error in an md file), Vite's import analysis\n// never runs on it, leaving `isSelfAccepting` as undefined. Vite's `propagateUpdate` silently skips\n// unanalyzed modules, so fixing the file produces no HMR update and the page stays broken.\n// We detect this and send a full-reload instead, since the module was never successfully loaded\n// and can't be hot-swapped.\nfunction fixHmrForFailedModules() {\n return {\n name: 'fix-hmr-for-failed-modules',\n hotUpdate(this: any, {modules}: {modules: any[]}) {\n // When a module's last transform failed, its transformResult is null. Vite's normal HMR can't\n // hot-swap a module that has no valid transform \u2014 either because it was never analyzed\n // (isSelfAccepting === undefined) or because it was previously working but is now broken.\n // In both cases, force a full page reload so the browser re-requests everything fresh.\n let hasFailed = modules.some(m => !m.transformResult)\n if (hasFailed) {\n this.environment.hot.send({type: 'full-reload', path: '*'})\n return []\n }\n },\n }\n}\n\n// Watch for changes to gsql files and reload the workspace.\n// This reload blocks all requests, so we shouldn't ever analyze without a workspace.\n// Also tracks all the md files in the workspace to populate the nav sidebar\nlet workspaceLoadPromise: Promise<void> | undefined\nlet workspaceFiles: WorkspaceFileInput[] = []\nlet mdFiles: {path: string; title?: string}[] = []\nfunction updateWorkspacePlugin(telemetry?: CliTelemetry) {\n return {\n name: 'updateWorkspace',\n resolveId(id: string) {\n if (id == 'virtual:nav') return '\\0virtual:nav'\n },\n load(id: string) {\n if (id != '\\0virtual:nav') return\n\n // in tests, inject mock files into the nav.\n // we do this on `load` as each test doesn't always refresh the workspace\n // TODO, we should prob inject these into `loadWorkspace`, then we wouldn't need this block at all\n let res = [...mdFiles]\n if (process.env.NODE_ENV == 'test') {\n for (let [path, contents] of Object.entries(mockFileMap)) {\n let mockFile = {path, title: extractFrontmatter(contents).title}\n let idx = res.findIndex(file => file.path == path)\n if (idx >= 0) res.splice(idx, 1, mockFile)\n else res.push(mockFile)\n }\n }\n\n return `export default ${JSON.stringify(res)}`\n },\n configureServer: (s: ViteDevServer) => {\n let refresh = async () => {\n workspaceLoadPromise = (async () => {\n let loaded = await loadWorkspace(config.root, true, config.ignoredFiles)\n telemetry?.event('workspace_scanned', {command: 'serve', ...getWorkspaceScanCounts(loaded)})\n workspaceFiles = loaded.map(file => {\n let existing = workspaceFiles.find(existing => existing.path == file.path && existing.contents == file.contents)\n return existing?.parsed ? {...file, parsed: existing.parsed} : file\n })\n })()\n await workspaceLoadPromise\n\n // store md file path/title so we can serve it as virtual:nav for the sidebar\n mdFiles = workspaceFiles.filter(file => file.path.endsWith('.md')).map(f => ({path: f.path, title: extractFrontmatter(f.contents).title}))\n\n let mod = s.moduleGraph.getModuleById('\\0virtual:nav')\n if (!mod) return\n s.reloadModule(mod) // triggers HMR of any `virtual:nav` imports\n }\n\n s.watcher.add(['**/*.gsql', '**/*.md'])\n s.watcher.on('all', refresh)\n refresh()\n },\n }\n}\n\nfunction updateParsedFiles(analysis: AnalysisResult) {\n workspaceFiles = workspaceFiles.map(file => {\n let analyzed = analysis.files.find(next => next.path == file.path)\n if (!analyzed) return file\n return {\n ...file,\n parsed: {\n tree: analyzed.tree!,\n virtualContents: analyzed.virtualContents,\n virtualToMarkdownOffset: analyzed.virtualToMarkdownOffset,\n },\n }\n })\n}\n\nconst handleRequestPlugin = {\n name: 'handleRequest',\n configureServer: (s: ViteDevServer) => {\n s.middlewares.use(async function handleRequest(req, res, next) {\n try {\n let [pathName] = (req.url || '').split('?')\n if (pathName == '/_api/query') return await handleQuery(req, res)\n if (pathName) if (pathName == '/__ct' || pathName == '/_charts' || pathName == '/_styles') return await handlePage(s, res)\n\n if (!pathName || pathName == '/') pathName = 'index'\n let relativeMdPath = pathName.replace(/^\\//, '') + '.md'\n let mdPath = path.join(config.root, relativeMdPath)\n\n if (mockFileMap[relativeMdPath] || (await fs.exists(mdPath))) {\n await handlePage(s, res)\n } else {\n next()\n }\n } catch (err: any) {\n if (process.env.NODE_ENV != 'test') console.error(err) // ignore in tests because they're noisy, and any unexpected errors should be captured by browserConsole.\n res.statusCode = 500\n res.end(JSON.stringify({message: err.message, stack: err.stack}))\n }\n })\n },\n}\n\nfunction mockFilesForTests() {\n if (process.env.NODE_ENV !== 'test') return null\n\n function toMockKey(id: string) {\n // Handle both absolute paths (/wt/.../index.md) and root-relative paths (/index.md)\n return id.replace(config.root + '/', '').replace(/^\\//, '')\n }\n\n return {\n name: 'mock-files-for-tests',\n enforce: 'pre' as const,\n resolveId(id: any) {\n if (!mockFileMap[toMockKey(id)]) return\n // Always resolve to the absolute path so the module graph key matches\n // what updateMockFile emits via server.watcher (needed for HMR to work).\n return path.join(config.root, toMockKey(id)) + '?mock'\n },\n load(id: any) {\n if (!id.endsWith('?mock')) return null\n return mockFileMap[toMockKey(id.replace(/\\?mock$/, ''))]\n },\n }\n}\n", "import type {Plugin} from 'unified'\n\nimport {decodeHTML} from 'entities'\nimport fs from 'fs'\nimport yaml from 'js-yaml'\nimport JSON5 from 'json5'\nimport path from 'path'\nimport sanitizeHtml from 'sanitize-html'\nimport {visit} from 'unist-util-visit'\n\nfunction escapeHtml(str: string) {\n return str.replace(/&/g, '&amp;').replace(/\"/g, '&quot;').replace(/</g, '&lt;').replace(/>/g, '&gt;')\n}\n\n// Takes the contents of a <ECharts> tag, and json5 parses it\nexport function liftInlineEChartsConfig(content: string) {\n return content.replace(/<ECharts\\b([^>]*)>([\\s\\S]*?)<\\/ECharts>/g, (match: string, attrs = '', body = '') => {\n let inline = body.trim()\n if (!inline) return match\n if (/\\sconfig\\s*=/.test(attrs)) return match\n let source = inline.startsWith('{') ? inline : `{${inline}}`\n let config = JSON.stringify(JSON5.parse(source), (_key, value) => (typeof value == 'string' ? decodeHTML(value) : value))\n return `<ECharts${attrs} config={${config}}></ECharts>`\n })\n}\n\n// Turn code fences into <GrapheneQuery> tags, which register those queries\nexport function extractQueries() {\n return function transformer(tree: any) {\n visit(tree, 'code', (node, index, parent) => {\n if (index === null) return\n let name = typeof node.meta === 'string' ? node.meta : ''\n let code = typeof node.value === 'string' ? node.value.trim() : ''\n parent.children[index] = {type: 'html', value: `<GrapheneQuery name=\"${escapeHtml(name)}\" code=\"${escapeHtml(code)}\" />`}\n })\n }\n}\n\n// remark will leave less-than and greater-than unescaped, which breaks svelte and prevents the page from loading.\nexport function escapeAngles() {\n return function transformer(tree: any) {\n visit(tree, 'text', (node: any) => {\n if (!node.value || typeof node.value !== 'string') return\n if (!node.value.includes('<')) return\n node.value = node.value.replace(/</g, '&lt;')\n })\n }\n}\n\n// remark can split one html block into adjacent html nodes when self-closing tags are involved.\n// Merge those sibling html nodes so downstream rehype/sanitize work on the full block.\nexport function mergeAdjacentHtml() {\n return function transformer(tree: any) {\n visit(tree, (parent: any) => {\n if (!Array.isArray(parent?.children)) return\n\n for (let i = 0; i < parent.children.length; i++) {\n if (parent.children[i]?.type !== 'html') continue\n\n let j = i\n while (j + 1 < parent.children.length && parent.children[j + 1]?.type === 'html') j++\n if (j == i) continue\n\n let value = parent.children\n .slice(i, j + 1)\n .map((node: any) => node.value || '')\n .join('\\n')\n parent.children.splice(i, j - i + 1, {type: 'html', value})\n }\n })\n }\n}\n\n// Restrict allowed components in markdown files to avoid xss issues.\n// This uses sanitize-html rather than rehype-sanitize because the latter had lots of issues with preserving tag casing,\n// as well as allowing all attributes on our allowlisted components.\nexport function sanitizeMarkdown() {\n return function transformer(tree: any) {\n visit(tree, 'raw', (node: any) => {\n if (typeof node.value !== 'string') return\n\n // sanitize-html doesn't like non-standard self-closing tags, so we need to rewrite them into open+close tags\n let expanded = node.value.replace(/<(\\w+)((?:\\s[^<>]*?)?)\\s*\\/>/gi, (_: string, name: string, attrs = '') => {\n let spacing = attrs\n return `<${name}${spacing}></${name}>`\n })\n\n let sanitized = sanitizeHtml(expanded, {\n ...sanitizeHtml.defaults,\n allowedTags: [...sanitizeHtml.defaults.allowedTags, ...componentNames()],\n allowedAttributes: {\n ...sanitizeHtml.defaults.allowedAttributes,\n ...Object.fromEntries(componentNames().map(n => [n, ['*']])),\n },\n parser: {\n ...((sanitizeHtml.defaults as any).parser || {}),\n lowerCaseAttributeNames: false,\n lowerCaseTags: false,\n },\n })\n node.value = sanitized\n })\n }\n}\n\n// We don't want users to have to manually import components in their md files, so we auto-import them.\nexport function injectComponentImports() {\n let imp = `const {${componentNames().join(', ')}} = window.$GRAPHENE.components`\n\n return {\n markup: ({content, filename}: {content: string; filename: string}) => {\n if (!filename.endsWith('.md')) return // only auto-import components for md files\n content = liftInlineEChartsConfig(content)\n if (content.includes('<script>')) {\n content = content.replace('<script>', `<script>\\n${imp}`)\n } else {\n content = `<script>\\n${imp}\\n</script>\\n${content}`\n }\n return {code: content}\n },\n style: () => {},\n script: () => {},\n }\n}\n\n// List out the component names from ui/components\nlet cachedComponentNames: string[] | null = null\nexport function componentNames() {\n if (cachedComponentNames) return cachedComponentNames\n\n let files = fs.readdirSync(path.join(import.meta.dirname, '../ui/components'))\n cachedComponentNames = files.map(f => path.basename(f, '.svelte')).filter(f => !f.startsWith('_'))\n return cachedComponentNames || []\n}\n\nexport type PageFrontmatter = {title?: string}\n\n// Parse YAML frontmatter from the --- delimited block at the top of a markdown file.\nconst frontmatterRe = /^---\\s*\\n([\\s\\S]*?)\\n---(?:\\n|$)/\nexport function extractFrontmatter(contents: string): PageFrontmatter {\n let match = contents.trimStart().match(frontmatterRe)\n if (!match) return {}\n let raw = yaml.safeLoad(match[1]) as Record<string, any> | undefined\n return {title: raw?.title ? String(raw.title) : undefined}\n}\n\nexport const remarkPlugins: Array<Plugin> = [extractQueries, escapeAngles, mergeAdjacentHtml]\nexport const rehypePlugins: Array<Plugin> = [sanitizeMarkdown]\n"],
5
+ "mappings": ";;;;;;;;;;;;;;AAAA,SAAQ,QAAQ,sBAAqB;AACrC,OAAO,YAAY;AACnB,OAAOA,SAAQ;AAGf,SAAQ,cAAa;AACrB,SAAQ,qBAAoB;AAC5B,OAAOC,WAAU;AACjB,SAAQ,qBAAoB;AAC5B,SAAQ,cAAiC,cAAc,qBAAwC;;;ACP/F,SAAQ,kBAAiB;AACzB,OAAO,QAAQ;AACf,OAAO,UAAU;AACjB,OAAO,WAAW;AAClB,OAAO,UAAU;AACjB,OAAO,kBAAkB;AACzB,SAAQ,aAAY;AAEpB,SAAS,WAAW,KAAa;AAC/B,SAAO,IAAI,QAAQ,MAAM,OAAO,EAAE,QAAQ,MAAM,QAAQ,EAAE,QAAQ,MAAM,MAAM,EAAE,QAAQ,MAAM,MAAM;AACtG;AAGO,SAAS,wBAAwB,SAAiB;AACvD,SAAO,QAAQ,QAAQ,4CAA4C,CAAC,OAAe,QAAQ,IAAI,OAAO,OAAO;AAC3G,QAAI,SAAS,KAAK,KAAK;AACvB,QAAI,CAAC,OAAQ,QAAO;AACpB,QAAI,eAAe,KAAK,KAAK,EAAG,QAAO;AACvC,QAAI,SAAS,OAAO,WAAW,GAAG,IAAI,SAAS,IAAI,MAAM;AACzD,QAAIC,UAAS,KAAK,UAAU,MAAM,MAAM,MAAM,GAAG,CAAC,MAAM,UAAW,OAAO,SAAS,WAAW,WAAW,KAAK,IAAI,KAAM;AACxH,WAAO,WAAW,KAAK,YAAYA,OAAM;AAAA,EAC3C,CAAC;AACH;AAGO,SAAS,iBAAiB;AAC/B,SAAO,SAAS,YAAY,MAAW;AACrC,UAAM,MAAM,QAAQ,CAAC,MAAM,OAAO,WAAW;AAC3C,UAAI,UAAU,KAAM;AACpB,UAAI,OAAO,OAAO,KAAK,SAAS,WAAW,KAAK,OAAO;AACvD,UAAI,OAAO,OAAO,KAAK,UAAU,WAAW,KAAK,MAAM,KAAK,IAAI;AAChE,aAAO,SAAS,KAAK,IAAI,EAAC,MAAM,QAAQ,OAAO,wBAAwB,WAAW,IAAI,CAAC,WAAW,WAAW,IAAI,CAAC,OAAM;AAAA,IAC1H,CAAC;AAAA,EACH;AACF;AAGO,SAAS,eAAe;AAC7B,SAAO,SAAS,YAAY,MAAW;AACrC,UAAM,MAAM,QAAQ,CAAC,SAAc;AACjC,UAAI,CAAC,KAAK,SAAS,OAAO,KAAK,UAAU,SAAU;AACnD,UAAI,CAAC,KAAK,MAAM,SAAS,GAAG,EAAG;AAC/B,WAAK,QAAQ,KAAK,MAAM,QAAQ,MAAM,MAAM;AAAA,IAC9C,CAAC;AAAA,EACH;AACF;AAIO,SAAS,oBAAoB;AAClC,SAAO,SAAS,YAAY,MAAW;AACrC,UAAM,MAAM,CAAC,WAAgB;AAC3B,UAAI,CAAC,MAAM,QAAQ,QAAQ,QAAQ,EAAG;AAEtC,eAAS,IAAI,GAAG,IAAI,OAAO,SAAS,QAAQ,KAAK;AAC/C,YAAI,OAAO,SAAS,CAAC,GAAG,SAAS,OAAQ;AAEzC,YAAI,IAAI;AACR,eAAO,IAAI,IAAI,OAAO,SAAS,UAAU,OAAO,SAAS,IAAI,CAAC,GAAG,SAAS,OAAQ;AAClF,YAAI,KAAK,EAAG;AAEZ,YAAI,QAAQ,OAAO,SAChB,MAAM,GAAG,IAAI,CAAC,EACd,IAAI,CAAC,SAAc,KAAK,SAAS,EAAE,EACnC,KAAK,IAAI;AACZ,eAAO,SAAS,OAAO,GAAG,IAAI,IAAI,GAAG,EAAC,MAAM,QAAQ,MAAK,CAAC;AAAA,MAC5D;AAAA,IACF,CAAC;AAAA,EACH;AACF;AAKO,SAAS,mBAAmB;AACjC,SAAO,SAAS,YAAY,MAAW;AACrC,UAAM,MAAM,OAAO,CAAC,SAAc;AAChC,UAAI,OAAO,KAAK,UAAU,SAAU;AAGpC,UAAI,WAAW,KAAK,MAAM,QAAQ,kCAAkC,CAAC,GAAW,MAAc,QAAQ,OAAO;AAC3G,YAAI,UAAU;AACd,eAAO,IAAI,IAAI,GAAG,OAAO,MAAM,IAAI;AAAA,MACrC,CAAC;AAED,UAAI,YAAY,aAAa,UAAU;AAAA,QACrC,GAAG,aAAa;AAAA,QAChB,aAAa,CAAC,GAAG,aAAa,SAAS,aAAa,GAAG,eAAe,CAAC;AAAA,QACvE,mBAAmB;AAAA,UACjB,GAAG,aAAa,SAAS;AAAA,UACzB,GAAG,OAAO,YAAY,eAAe,EAAE,IAAI,OAAK,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC;AAAA,QAC7D;AAAA,QACA,QAAQ;AAAA,UACN,GAAK,aAAa,SAAiB,UAAU,CAAC;AAAA,UAC9C,yBAAyB;AAAA,UACzB,eAAe;AAAA,QACjB;AAAA,MACF,CAAC;AACD,WAAK,QAAQ;AAAA,IACf,CAAC;AAAA,EACH;AACF;AAGO,SAAS,yBAAyB;AACvC,MAAI,MAAM,UAAU,eAAe,EAAE,KAAK,IAAI,CAAC;AAE/C,SAAO;AAAA,IACL,QAAQ,CAAC,EAAC,SAAS,SAAQ,MAA2C;AACpE,UAAI,CAAC,SAAS,SAAS,KAAK,EAAG;AAC/B,gBAAU,wBAAwB,OAAO;AACzC,UAAI,QAAQ,SAAS,UAAU,GAAG;AAChC,kBAAU,QAAQ,QAAQ,YAAY;AAAA,EAAa,GAAG,EAAE;AAAA,MAC1D,OAAO;AACL,kBAAU;AAAA,EAAa,GAAG;AAAA;AAAA,EAAgB,OAAO;AAAA,MACnD;AACA,aAAO,EAAC,MAAM,QAAO;AAAA,IACvB;AAAA,IACA,OAAO,MAAM;AAAA,IAAC;AAAA,IACd,QAAQ,MAAM;AAAA,IAAC;AAAA,EACjB;AACF;AAGA,IAAI,uBAAwC;AACrC,SAAS,iBAAiB;AAC/B,MAAI,qBAAsB,QAAO;AAEjC,MAAI,QAAQ,GAAG,YAAY,KAAK,KAAK,YAAY,SAAS,kBAAkB,CAAC;AAC7E,yBAAuB,MAAM,IAAI,OAAK,KAAK,SAAS,GAAG,SAAS,CAAC,EAAE,OAAO,OAAK,CAAC,EAAE,WAAW,GAAG,CAAC;AACjG,SAAO,wBAAwB,CAAC;AAClC;AAKA,IAAM,gBAAgB;AACf,SAAS,mBAAmB,UAAmC;AACpE,MAAI,QAAQ,SAAS,UAAU,EAAE,MAAM,aAAa;AACpD,MAAI,CAAC,MAAO,QAAO,CAAC;AACpB,MAAI,MAAM,KAAK,SAAS,MAAM,CAAC,CAAC;AAChC,SAAO,EAAC,OAAO,KAAK,QAAQ,OAAO,IAAI,KAAK,IAAI,OAAS;AAC3D;AAEO,IAAM,gBAA+B,CAAC,gBAAgB,cAAc,iBAAiB;AACrF,IAAM,gBAA+B,CAAC,gBAAgB;;;AD5HtD,IAAM,iBAAkC,CAAC;AACzC,SAAS,sBAAsB;AACpC,iBAAe,SAAS;AAC1B;AAGA,IAAM,gBAAgB;AAEtB,IAAI;AACJ,IAAI,cAAc,cAAc,YAAY,GAAG;AAE/C,eAAsB,OAAO,WAAkD;AAC7E,MAAI,SAAS,MAAM,aAAa,MAAM,aAAa,SAAS,CAAC;AAK7D,QAAM,OAAO,OAAO;AACpB,UAAQ,IAAI,sCAAsC,OAAO,OAAO,OAAO,IAAI,EAAE;AAE7E,SAAO;AACT;AAEA,eAAe,aAAa,WAAiD;AAC3E,WAASC,MAAK,KAAK,cAAc,YAAY,GAAG,GAAG,UAAU;AAC7D,MAAI,OAAO,OAAO,QAAQ,IAAI,aAAa,KAAK;AAChD,MAAI,aAAaA,MAAK,QAAQ,YAAY,QAAQ,qBAAqB,CAAC;AACxE,MAAI,gBAAgB,YAAY,qBAAqB;AACrD,MAAI,uBAAuBA,MAAK,QAAQ,UAAU;AAClD,MAAI,eAAe,CAAC,SAAiBA,MAAK,KAAK,YAAY,cAAc,QAAQ,IAAI,EAAE,WAAW,cAAc,QAAQ,IAAI,EAAE,OAAO;AACrI,MAAI,WAAWA,MAAK,SAASA,MAAK,QAAQ,MAAM,CAAC,KAAK;AACtD,QAAMC,IAAG,UAAUD,MAAK,QAAQ,OAAO,MAAM,wBAAwB,CAAC;AAGtE,MAAI,cAAcC,IAAG,WAAW,aAAa;AAC7C,MAAI,OAAO,cAAc,YAAY;AAErC,SAAO;AAAA,IACL,MAAM,OAAO;AAAA,IACb,UAAU,QAAQ,IAAI,YAAY,SAAS,WAAW;AAAA,IACtD,SAAS;AAAA,MACP,OAAO;AAAA,QACL,YAAY;AAAA,QACZ,YAAY,CAAC,WAAW,KAAK;AAAA,QAC7B,YAAY;AAAA,UACV,eAAe;AAAA,UACf,OAAO;AAAA,YACL,YAAY,CAAC,KAAK;AAAA,YAClB;AAAA,YACA;AAAA,UACF,CAAC;AAAA,UACD,uBAAuB;AAAA,QACzB;AAAA,QACA,OAAO,SAAS,gBAAgB;AAC9B,cAAI,QAAQ,IAAI,aAAa,QAAQ;AACnC,2BAAe,KAAK,EAAC,MAAM,QAAQ,MAAM,SAAS,QAAQ,SAAS,UAAU,QAAQ,SAAQ,CAAC;AAAA,UAChG;AACA,2BAAiB,OAAO;AAAA,QAC1B;AAAA,MACF,CAAC;AAAA,MACD,qBAAqB;AAAA,MACrB,uBAAuB;AAAA,MACvB,cAAc;AAAA,MACd;AAAA,MACA,sBAAsB,SAAS;AAAA,MAC/B,kBAAkB;AAAA,IACpB;AAAA,IACA,WAAWD,MAAK,QAAQ,QAAQ,QAAQ;AAAA;AAAA;AAAA,IAGxC,QAAQ;AAAA,MACN;AAAA,MACA;AAAA,MACA,IAAI,EAAC,QAAQ,MAAK;AAAA,MAClB,YAAY;AAAA,MACZ,KAAK,EAAC,SAAS,MAAK;AAAA;AAAA,IACtB;AAAA,IACA,SAAS;AAAA,MACP,OAAO;AAAA,QACL,EAAC,MAAM,cAAc,aAAaA,MAAK,QAAQ,QAAQ,QAAQ,EAAC;AAAA;AAAA;AAAA,QAGhE,EAAC,MAAM,YAAY,aAAa,aAAa,GAAG,EAAC;AAAA,QACjD,EAAC,MAAM,qBAAqB,aAAa,aAAa,WAAW,EAAC;AAAA,QAClE,EAAC,MAAM,yBAAyB,aAAa,aAAa,eAAe,EAAC;AAAA,QAC1E,EAAC,MAAM,oBAAoB,aAAa,aAAa,UAAU,EAAC;AAAA,QAChE,EAAC,MAAM,oBAAoB,aAAa,aAAa,UAAU,EAAC;AAAA,QAChE,EAAC,MAAM,sBAAsB,aAAa,aAAa,YAAY,EAAC;AAAA,QACpE,EAAC,MAAM,8BAA8B,aAAa,aAAa,mBAAmB,EAAC;AAAA,QACnF,EAAC,MAAM,wCAAwC,aAAa,aAAa,6BAA6B,EAAC;AAAA,QACvG,EAAC,MAAM,oCAAoC,aAAa,aAAa,wBAAwB,EAAC;AAAA,QAC9F,EAAC,MAAM,qCAAqC,aAAa,aAAa,yBAAyB,EAAC;AAAA,QAChG,EAAC,MAAM,sCAAsC,aAAa,aAAa,0BAA0B,EAAC;AAAA,QAClG,EAAC,MAAM,oBAAoB,aAAa,aAAa,UAAU,EAAC;AAAA,QAChE,EAAC,MAAM,oBAAoB,aAAa,aAAa,UAAU,EAAC;AAAA,QAChE,EAAC,MAAM,wBAAwB,aAAa,aAAa,cAAc,EAAC;AAAA,QACxE,EAAC,MAAM,gCAAgC,aAAa,aAAa,qBAAqB,EAAC;AAAA,QACvF,EAAC,MAAM,mBAAmB,aAAa,aAAa,SAAS,EAAC;AAAA,QAC9D,EAAC,MAAM,wBAAwB,aAAa,aAAa,cAAc,EAAC;AAAA,QACxE,EAAC,MAAM,UAAU,aAAaA,MAAK,KAAK,sBAAsB,oBAAoB,EAAC;AAAA,MACrF;AAAA,IACF;AAAA,IAEA,cAAc;AAAA,MACZ,aAAa,QAAQ,IAAI,YAAY;AAAA;AAAA,MACrC,SAAS,CAAC,aAAa;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,MAOvB,SAAS;AAAA,QACP,GAAI,WAAW,CAAC,UAAU,IAAI,CAAC;AAAA,QAC/B;AAAA,QACA;AAAA,QACA;AAAA,QACA;AAAA,QACA;AAAA,QACA;AAAA,QACA;AAAA,QACA;AAAA,MACF;AAAA,IACF;AAAA,EACF;AACF;AAEA,eAAe,YAAY,KAAsB,KAAsC;AACrF,MAAI,SAAS,CAAC;AACd,iBAAe,SAAS,IAAK,QAAO,KAAK,KAAK;AAC9C,MAAI,EAAC,MAAM,QAAQ,OAAM,IAAI,KAAK,MAAM,OAAO,OAAO,MAAM,EAAE,SAAS,CAAC;AACxE,MAAI,UAAU,gBAAgB,kBAAkB;AAEhD,QAAM;AAGN,MAAI,YAAY,eAAe,OAAO,UAAQ,CAAC,KAAK,KAAK,SAAS,KAAK,CAAC;AACxE,MAAI,SAAS,iBAAiB,EAAC,QAAQ,OAAO,CAAC,GAAG,WAAW,EAAC,MAAM,SAAS,UAAU,KAAI,CAAC,EAAC,CAAC;AAC9F,oBAAkB,MAAM;AAExB,MAAI,cAAc,OAAO;AACzB,MAAI,YAAY,QAAQ;AACtB,QAAI,aAAa;AACjB,QAAI,IAAI,KAAK,UAAU,YAAY,CAAC,CAAC,CAAC;AACtC;AAAA,EACF;AAEA,MAAI,UAAU,OAAO,MAAM,KAAK,UAAQ,KAAK,QAAQ,OAAO,GAAG,WAAW,CAAC;AAC3E,MAAI,QAAQ,SAAS,EAAG,OAAM,IAAI,MAAM,wDAAwD;AAChG,MAAI,MAAM,MAAM,QAAQ,CAAC,GAAG,MAAM;AAGlC,MAAI,OAAO,OAAO,WAAW,MAAM,EAAE,OAAO,UAAU,aAAa,IAAI,GAAG,EAAE,EAAE,OAAO,KAAK;AAC1F,MAAI,UAAU,QAAQ,IAAI;AAC1B,MAAI,OAAO,SAAS,IAAI,KAAK,IAAI,QAAQ,eAAe,KAAK,YAAY;AACvE,QAAI,aAAa;AACjB,WAAO,IAAI,IAAI;AAAA,EACjB;AAEA,MAAI,eAAe,MAAM,SAAS,GAAG;AACrC,MAAI,YAAY,aAAa,aAAa,aAAa,KAAK;AAC5D,MAAI,YAAY,aAAa,KAAK,OAAQ,OAAM,IAAI,MAAM,6BAA6B;AACvF,MAAI,SAAS,QAAQ,CAAC,EAAE,OAAO,IAAI,YAAU,EAAC,MAAM,MAAM,MAAM,MAAM,MAAM,MAAM,UAAU,MAAM,YAAY,CAAC,EAAC,EAAE;AAClH,MAAI,IAAI,KAAK,UAAU,EAAC,MAAM,aAAa,MAAM,MAAM,QAAQ,IAAG,CAAC,CAAC;AACtE;AAEA,eAAe,WAAW,QAAuB,KAAsC;AACrF,MAAI,UAAU,gBAAgB,WAAW;AAGzC,MAAI,OAAO,MAAM,OAAO;AAAA,IACtB;AAAA,IACA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAcF;AACA,SAAO,IAAI,IAAI,IAAI;AACrB;AAGA,eAAsB,cAAc;AAClC,MAAI,MAAM,MAAM,cAAc,MAAM,aAAa,GAAG,OAAO;AAC3D,QAAM,aAAa,KAAK,IAAI;AAC9B;AAKA,SAAS,uBAAuB;AAC9B,MAAI;AAEJ,WAAS,eAAe,KAAU;AAChC,iBAAa;AAAA,EACf;AAIA,WAAS,aAAa;AACpB,QAAI,QAAQ,IAAI,YAAY,OAAQ;AACpC,eAAW,aAAa,QAAQ;AAAA,EAClC;AACA,aAAW,aAAa;AACxB,SAAO,EAAC,MAAM,mBAAmB,SAAS,QAAiB,gBAAgB,WAAU;AACvF;AAOA,SAAS,yBAAyB;AAChC,SAAO;AAAA,IACL,MAAM;AAAA,IACN,UAAqB,EAAC,QAAO,GAAqB;AAKhD,UAAI,YAAY,QAAQ,KAAK,OAAK,CAAC,EAAE,eAAe;AACpD,UAAI,WAAW;AACb,aAAK,YAAY,IAAI,KAAK,EAAC,MAAM,eAAe,MAAM,IAAG,CAAC;AAC1D,eAAO,CAAC;AAAA,MACV;AAAA,IACF;AAAA,EACF;AACF;AAKA,IAAI;AACJ,IAAI,iBAAuC,CAAC;AAC5C,IAAI,UAA4C,CAAC;AACjD,SAAS,sBAAsB,WAA0B;AACvD,SAAO;AAAA,IACL,MAAM;AAAA,IACN,UAAU,IAAY;AACpB,UAAI,MAAM,cAAe,QAAO;AAAA,IAClC;AAAA,IACA,KAAK,IAAY;AACf,UAAI,MAAM,gBAAiB;AAK3B,UAAI,MAAM,CAAC,GAAG,OAAO;AACrB,UAAI,QAAQ,IAAI,YAAY,QAAQ;AAClC,iBAAS,CAACA,OAAM,QAAQ,KAAK,OAAO,QAAQ,WAAW,GAAG;AACxD,cAAI,WAAW,EAAC,MAAAA,OAAM,OAAO,mBAAmB,QAAQ,EAAE,MAAK;AAC/D,cAAI,MAAM,IAAI,UAAU,UAAQ,KAAK,QAAQA,KAAI;AACjD,cAAI,OAAO,EAAG,KAAI,OAAO,KAAK,GAAG,QAAQ;AAAA,cACpC,KAAI,KAAK,QAAQ;AAAA,QACxB;AAAA,MACF;AAEA,aAAO,kBAAkB,KAAK,UAAU,GAAG,CAAC;AAAA,IAC9C;AAAA,IACA,iBAAiB,CAAC,MAAqB;AACrC,UAAI,UAAU,YAAY;AACxB,gCAAwB,YAAY;AAClC,cAAI,SAAS,MAAM,cAAc,OAAO,MAAM,MAAM,OAAO,YAAY;AACvE,qBAAW,MAAM,qBAAqB,EAAC,SAAS,SAAS,GAAG,uBAAuB,MAAM,EAAC,CAAC;AAC3F,2BAAiB,OAAO,IAAI,UAAQ;AAClC,gBAAI,WAAW,eAAe,KAAK,CAAAE,cAAYA,UAAS,QAAQ,KAAK,QAAQA,UAAS,YAAY,KAAK,QAAQ;AAC/G,mBAAO,UAAU,SAAS,EAAC,GAAG,MAAM,QAAQ,SAAS,OAAM,IAAI;AAAA,UACjE,CAAC;AAAA,QACH,GAAG;AACH,cAAM;AAGN,kBAAU,eAAe,OAAO,UAAQ,KAAK,KAAK,SAAS,KAAK,CAAC,EAAE,IAAI,QAAM,EAAC,MAAM,EAAE,MAAM,OAAO,mBAAmB,EAAE,QAAQ,EAAE,MAAK,EAAE;AAEzI,YAAI,MAAM,EAAE,YAAY,cAAc,eAAe;AACrD,YAAI,CAAC,IAAK;AACV,UAAE,aAAa,GAAG;AAAA,MACpB;AAEA,QAAE,QAAQ,IAAI,CAAC,aAAa,SAAS,CAAC;AACtC,QAAE,QAAQ,GAAG,OAAO,OAAO;AAC3B,cAAQ;AAAA,IACV;AAAA,EACF;AACF;AAEA,SAAS,kBAAkB,UAA0B;AACnD,mBAAiB,eAAe,IAAI,UAAQ;AAC1C,QAAI,WAAW,SAAS,MAAM,KAAK,UAAQ,KAAK,QAAQ,KAAK,IAAI;AACjE,QAAI,CAAC,SAAU,QAAO;AACtB,WAAO;AAAA,MACL,GAAG;AAAA,MACH,QAAQ;AAAA,QACN,MAAM,SAAS;AAAA,QACf,iBAAiB,SAAS;AAAA,QAC1B,yBAAyB,SAAS;AAAA,MACpC;AAAA,IACF;AAAA,EACF,CAAC;AACH;AAEA,IAAM,sBAAsB;AAAA,EAC1B,MAAM;AAAA,EACN,iBAAiB,CAAC,MAAqB;AACrC,MAAE,YAAY,IAAI,eAAe,cAAc,KAAK,KAAK,MAAM;AAC7D,UAAI;AACF,YAAI,CAAC,QAAQ,KAAK,IAAI,OAAO,IAAI,MAAM,GAAG;AAC1C,YAAI,YAAY,cAAe,QAAO,MAAM,YAAY,KAAK,GAAG;AAChE,YAAI;AAAU,cAAI,YAAY,WAAW,YAAY,cAAc,YAAY,WAAY,QAAO,MAAM,WAAW,GAAG,GAAG;AAAA;AAEzH,YAAI,CAAC,YAAY,YAAY,IAAK,YAAW;AAC7C,YAAI,iBAAiB,SAAS,QAAQ,OAAO,EAAE,IAAI;AACnD,YAAI,SAASF,MAAK,KAAK,OAAO,MAAM,cAAc;AAElD,YAAI,YAAY,cAAc,KAAM,MAAMC,IAAG,OAAO,MAAM,GAAI;AAC5D,gBAAM,WAAW,GAAG,GAAG;AAAA,QACzB,OAAO;AACL,eAAK;AAAA,QACP;AAAA,MACF,SAAS,KAAU;AACjB,YAAI,QAAQ,IAAI,YAAY,OAAQ,SAAQ,MAAM,GAAG;AACrD,YAAI,aAAa;AACjB,YAAI,IAAI,KAAK,UAAU,EAAC,SAAS,IAAI,SAAS,OAAO,IAAI,MAAK,CAAC,CAAC;AAAA,MAClE;AAAA,IACF,CAAC;AAAA,EACH;AACF;AAEA,SAAS,oBAAoB;AAC3B,MAAI,QAAQ,IAAI,aAAa,OAAQ,QAAO;AAE5C,WAAS,UAAU,IAAY;AAE7B,WAAO,GAAG,QAAQ,OAAO,OAAO,KAAK,EAAE,EAAE,QAAQ,OAAO,EAAE;AAAA,EAC5D;AAEA,SAAO;AAAA,IACL,MAAM;AAAA,IACN,SAAS;AAAA,IACT,UAAU,IAAS;AACjB,UAAI,CAAC,YAAY,UAAU,EAAE,CAAC,EAAG;AAGjC,aAAOD,MAAK,KAAK,OAAO,MAAM,UAAU,EAAE,CAAC,IAAI;AAAA,IACjD;AAAA,IACA,KAAK,IAAS;AACZ,UAAI,CAAC,GAAG,SAAS,OAAO,EAAG,QAAO;AAClC,aAAO,YAAY,UAAU,GAAG,QAAQ,WAAW,EAAE,CAAC,CAAC;AAAA,IACzD;AAAA,EACF;AACF;",
6
+ "names": ["fs", "path", "config", "path", "fs", "existing"]
7
+ }
@@ -0,0 +1,128 @@
1
+ import {
2
+ config
3
+ } from "./chunk-QAXEOZ43.js";
4
+
5
+ // connections/snowflake.ts
6
+ import { createPrivateKey } from "node:crypto";
7
+ import snowflake from "snowflake-sdk";
8
+ var SnowflakeConnection = class {
9
+ ready;
10
+ connection;
11
+ constructor(opts) {
12
+ this.ready = this.initialize(opts || {});
13
+ }
14
+ async initialize(opts) {
15
+ let privateKeyPath = opts.privateKeyPath || config.snowflake?.privateKeyPath;
16
+ let authOptions = {};
17
+ if (privateKeyPath) {
18
+ authOptions = { privateKeyPath, privateKeyPass: opts.privateKeyPass };
19
+ } else if (opts.privateKey) {
20
+ let privateKey = createPrivateKey({ key: opts.privateKey, format: "pem", passphrase: opts.privateKeyPass });
21
+ authOptions = { privateKey: privateKey.export({ format: "pem", type: "pkcs8" }) };
22
+ }
23
+ snowflake.configure({ logLevel: opts.logLevel || "WARN", logFilePath: "/dev/null" });
24
+ this.connection = snowflake.createConnection({
25
+ ...opts,
26
+ ...config.snowflake || {},
27
+ ...authOptions,
28
+ authenticator: "SNOWFLAKE_JWT",
29
+ application: "Graphene"
30
+ });
31
+ await new Promise((resolve, reject) => {
32
+ this.connection.connect((err, conn) => err ? reject(err) : resolve(conn));
33
+ });
34
+ }
35
+ async runQuery(sql, params) {
36
+ await this.ready;
37
+ return await new Promise((resolve, reject) => {
38
+ let rows = [];
39
+ this.connection.execute({
40
+ sqlText: sql,
41
+ binds: params,
42
+ streamResult: true,
43
+ complete: (error, statement) => {
44
+ if (error) {
45
+ reject(new Error(`Snowflake query failed: ${error.message || error}`));
46
+ return;
47
+ }
48
+ let stream = statement.streamRows();
49
+ stream.on("error", (err) => reject(err));
50
+ stream.on("readable", function(row) {
51
+ while ((row = this.read()) !== null) {
52
+ rows.push(row);
53
+ }
54
+ });
55
+ stream.on("end", () => {
56
+ let totalRows = Number(statement.getNumRows());
57
+ resolve({ rows, totalRows });
58
+ });
59
+ }
60
+ });
61
+ });
62
+ }
63
+ async listDatasets() {
64
+ let res = await this.runQuery("show databases");
65
+ return res.rows.map((row) => String(row["name"] || ""));
66
+ }
67
+ async listSchemas(database) {
68
+ let resolvedDatabase = await this.resolveDatabaseName(database);
69
+ let res = await this.runQuery(`
70
+ select schema_name as "schema_name"
71
+ from ${snowflakeIdent(resolvedDatabase)}.INFORMATION_SCHEMA.SCHEMATA
72
+ where schema_name != 'INFORMATION_SCHEMA'
73
+ order by schema_name
74
+ `);
75
+ return res.rows.map((row) => String(row["schema_name"]).toLowerCase());
76
+ }
77
+ async listTables(dataset) {
78
+ let parts = dataset.split(".");
79
+ let database = await this.resolveDatabaseName(parts.shift() || "");
80
+ let schema = parts.join(".");
81
+ let res = await this.runQuery(
82
+ `
83
+ select table_schema as "table_schema", table_name as "table_name"
84
+ from ${snowflakeIdent(database)}.INFORMATION_SCHEMA.TABLES
85
+ where table_type in ('BASE TABLE', 'VIEW') and upper(table_schema) = upper(?)
86
+ order by table_name
87
+ `,
88
+ [schema]
89
+ );
90
+ return res.rows.map((row) => `${String(row["table_schema"]).toLowerCase()}.${String(row["table_name"]).toLowerCase()}`);
91
+ }
92
+ async describeTable(target) {
93
+ let parts = target.split(".");
94
+ let database = await this.resolveDatabaseName(parts.shift() || "");
95
+ let table = parts.pop() || "";
96
+ let schema = parts.join(".");
97
+ let res = await this.runQuery(
98
+ `
99
+ select column_name as "column_name", data_type as "data_type", ordinal_position as ordinal_position
100
+ from ${snowflakeIdent(database)}.INFORMATION_SCHEMA.COLUMNS
101
+ where upper(table_schema) = upper(?) and upper(table_name) = upper(?)
102
+ order by ordinal_position
103
+ `,
104
+ [schema, table]
105
+ );
106
+ return res.rows.map((row) => {
107
+ return { name: String(row["column_name"]).toLowerCase(), dataType: String(row["data_type"]) };
108
+ });
109
+ }
110
+ async resolveDatabaseName(name) {
111
+ let databases = await this.listDatasets();
112
+ return databases.find((db) => db.toLowerCase() == name.toLowerCase()) || name;
113
+ }
114
+ async close() {
115
+ await this.ready;
116
+ await new Promise((resolve, reject) => {
117
+ this.connection.destroy((err) => err ? reject(err) : resolve());
118
+ });
119
+ }
120
+ };
121
+ function snowflakeIdent(value) {
122
+ if (!value) throw new Error("Snowflake identifiers cannot be empty");
123
+ return `"${value.replace(/"/g, '""')}"`;
124
+ }
125
+ export {
126
+ SnowflakeConnection
127
+ };
128
+ //# sourceMappingURL=snowflake-MOQB5GA4.js.map
@@ -0,0 +1,7 @@
1
+ {
2
+ "version": 3,
3
+ "sources": ["../../connections/snowflake.ts"],
4
+ "sourcesContent": ["import {createPrivateKey} from 'node:crypto'\nimport snowflake from 'snowflake-sdk'\n\nimport {config} from '../../lang/config.ts'\nimport {type QueryConnection, type QueryResult, type SchemaColumn, type QueryParams} from './types.ts'\n\ninterface SnowflakeOptions {\n username?: string\n account?: string\n privateKey?: string\n privateKeyPath?: string\n privateKeyPass?: string\n logLevel?: string\n}\n\n// Raw notes on setting up a new user:\n// * create a `demouser` with a new `demorole`. It should have\n// That role needs `operate` and `usage` on a warehouse, and `usage` on the relevant db or schema\n// `ALTER USER DEMOUSER SET DEFAULT_WAREHOUSE = COMPUTE_WH;`\n// `ALTER USER DEMOUSER SET DEFAULT_ROLE = DEMOREAD;`\n// You can get the `account` by looking at the bit before \"snowflakecomputing\" in the account url (which is found in the snowflake ui)\n// Instructions for generating private/public keys: https://docs.snowflake.com/en/user-guide/key-pair-auth#generate-the-private-keys\n\nexport class SnowflakeConnection implements QueryConnection {\n private ready: Promise<void>\n private connection!: snowflake.Connection\n\n constructor(opts: SnowflakeOptions) {\n this.ready = this.initialize(opts || {})\n }\n\n async initialize(opts: SnowflakeOptions) {\n let privateKeyPath = opts.privateKeyPath || config.snowflake?.privateKeyPath\n\n let authOptions: any = {}\n if (privateKeyPath) {\n authOptions = {privateKeyPath, privateKeyPass: opts.privateKeyPass}\n } else if (opts.privateKey) {\n let privateKey = createPrivateKey({key: opts.privateKey, format: 'pem', passphrase: opts.privateKeyPass})\n authOptions = {privateKey: privateKey.export({format: 'pem', type: 'pkcs8'})}\n }\n\n // default is info, which is kinda chatty on success. TRACE is super useful for debugging though\n snowflake.configure({logLevel: (opts.logLevel as any) || 'WARN', logFilePath: '/dev/null'})\n\n this.connection = snowflake.createConnection({\n ...opts,\n ...(config.snowflake || {}),\n ...authOptions,\n authenticator: 'SNOWFLAKE_JWT',\n application: 'Graphene',\n })\n\n await new Promise((resolve, reject) => {\n this.connection.connect((err, conn) => (err ? reject(err) : resolve(conn)))\n })\n }\n\n async runQuery(sql: string, params?: QueryParams): Promise<QueryResult> {\n await this.ready\n return await new Promise<QueryResult>((resolve, reject) => {\n let rows: any[] = []\n this.connection.execute({\n sqlText: sql,\n binds: params as any,\n streamResult: true,\n complete: (error, statement) => {\n if (error) {\n reject(new Error(`Snowflake query failed: ${error.message || error}`))\n return\n }\n\n let stream = statement.streamRows()\n stream.on('error', err => reject(err))\n stream.on('readable', function (this: any, row) {\n while ((row = this.read()) !== null) {\n rows.push(row)\n }\n })\n stream.on('end', () => {\n let totalRows = Number(statement.getNumRows())\n resolve({rows, totalRows})\n })\n },\n })\n })\n }\n\n async listDatasets(): Promise<string[]> {\n let res = await this.runQuery('show databases')\n return res.rows.map(row => String(row['name'] || ''))\n }\n\n async listSchemas(database: string): Promise<string[]> {\n let resolvedDatabase = await this.resolveDatabaseName(database)\n let res = await this.runQuery(`\n select schema_name as \"schema_name\"\n from ${snowflakeIdent(resolvedDatabase)}.INFORMATION_SCHEMA.SCHEMATA\n where schema_name != 'INFORMATION_SCHEMA'\n order by schema_name\n `)\n return res.rows.map(row => String(row['schema_name']).toLowerCase())\n }\n\n async listTables(dataset: string): Promise<string[]> {\n let parts = dataset.split('.')\n let database = await this.resolveDatabaseName(parts.shift() || '')\n let schema = parts.join('.')\n\n let res = await this.runQuery(\n `\n select table_schema as \"table_schema\", table_name as \"table_name\"\n from ${snowflakeIdent(database)}.INFORMATION_SCHEMA.TABLES\n where table_type in ('BASE TABLE', 'VIEW') and upper(table_schema) = upper(?)\n order by table_name\n `,\n [schema],\n )\n return res.rows.map(row => `${String(row['table_schema']).toLowerCase()}.${String(row['table_name']).toLowerCase()}`)\n }\n\n async describeTable(target: string): Promise<SchemaColumn[]> {\n let parts = target.split('.')\n let database = await this.resolveDatabaseName(parts.shift() || '')\n let table = parts.pop() || ''\n let schema = parts.join('.')\n\n let res = await this.runQuery(\n `\n select column_name as \"column_name\", data_type as \"data_type\", ordinal_position as ordinal_position\n from ${snowflakeIdent(database)}.INFORMATION_SCHEMA.COLUMNS\n where upper(table_schema) = upper(?) and upper(table_name) = upper(?)\n order by ordinal_position\n `,\n [schema, table],\n )\n return res.rows.map(row => {\n return {name: String(row['column_name']).toLowerCase(), dataType: String(row['data_type'])}\n })\n }\n\n async resolveDatabaseName(name: string): Promise<string> {\n let databases = await this.listDatasets()\n return databases.find(db => db.toLowerCase() == name.toLowerCase()) || name\n }\n\n async close(): Promise<void> {\n await this.ready\n await new Promise<void>((resolve, reject) => {\n this.connection.destroy(err => (err ? reject(err) : resolve()))\n })\n }\n}\n\nfunction snowflakeIdent(value: string) {\n if (!value) throw new Error('Snowflake identifiers cannot be empty')\n return `\"${value.replace(/\"/g, '\"\"')}\"`\n}\n"],
5
+ "mappings": ";;;;;AAAA,SAAQ,wBAAuB;AAC/B,OAAO,eAAe;AAsBf,IAAM,sBAAN,MAAqD;AAAA,EAClD;AAAA,EACA;AAAA,EAER,YAAY,MAAwB;AAClC,SAAK,QAAQ,KAAK,WAAW,QAAQ,CAAC,CAAC;AAAA,EACzC;AAAA,EAEA,MAAM,WAAW,MAAwB;AACvC,QAAI,iBAAiB,KAAK,kBAAkB,OAAO,WAAW;AAE9D,QAAI,cAAmB,CAAC;AACxB,QAAI,gBAAgB;AAClB,oBAAc,EAAC,gBAAgB,gBAAgB,KAAK,eAAc;AAAA,IACpE,WAAW,KAAK,YAAY;AAC1B,UAAI,aAAa,iBAAiB,EAAC,KAAK,KAAK,YAAY,QAAQ,OAAO,YAAY,KAAK,eAAc,CAAC;AACxG,oBAAc,EAAC,YAAY,WAAW,OAAO,EAAC,QAAQ,OAAO,MAAM,QAAO,CAAC,EAAC;AAAA,IAC9E;AAGA,cAAU,UAAU,EAAC,UAAW,KAAK,YAAoB,QAAQ,aAAa,YAAW,CAAC;AAE1F,SAAK,aAAa,UAAU,iBAAiB;AAAA,MAC3C,GAAG;AAAA,MACH,GAAI,OAAO,aAAa,CAAC;AAAA,MACzB,GAAG;AAAA,MACH,eAAe;AAAA,MACf,aAAa;AAAA,IACf,CAAC;AAED,UAAM,IAAI,QAAQ,CAAC,SAAS,WAAW;AACrC,WAAK,WAAW,QAAQ,CAAC,KAAK,SAAU,MAAM,OAAO,GAAG,IAAI,QAAQ,IAAI,CAAE;AAAA,IAC5E,CAAC;AAAA,EACH;AAAA,EAEA,MAAM,SAAS,KAAa,QAA4C;AACtE,UAAM,KAAK;AACX,WAAO,MAAM,IAAI,QAAqB,CAAC,SAAS,WAAW;AACzD,UAAI,OAAc,CAAC;AACnB,WAAK,WAAW,QAAQ;AAAA,QACtB,SAAS;AAAA,QACT,OAAO;AAAA,QACP,cAAc;AAAA,QACd,UAAU,CAAC,OAAO,cAAc;AAC9B,cAAI,OAAO;AACT,mBAAO,IAAI,MAAM,2BAA2B,MAAM,WAAW,KAAK,EAAE,CAAC;AACrE;AAAA,UACF;AAEA,cAAI,SAAS,UAAU,WAAW;AAClC,iBAAO,GAAG,SAAS,SAAO,OAAO,GAAG,CAAC;AACrC,iBAAO,GAAG,YAAY,SAAqB,KAAK;AAC9C,oBAAQ,MAAM,KAAK,KAAK,OAAO,MAAM;AACnC,mBAAK,KAAK,GAAG;AAAA,YACf;AAAA,UACF,CAAC;AACD,iBAAO,GAAG,OAAO,MAAM;AACrB,gBAAI,YAAY,OAAO,UAAU,WAAW,CAAC;AAC7C,oBAAQ,EAAC,MAAM,UAAS,CAAC;AAAA,UAC3B,CAAC;AAAA,QACH;AAAA,MACF,CAAC;AAAA,IACH,CAAC;AAAA,EACH;AAAA,EAEA,MAAM,eAAkC;AACtC,QAAI,MAAM,MAAM,KAAK,SAAS,gBAAgB;AAC9C,WAAO,IAAI,KAAK,IAAI,SAAO,OAAO,IAAI,MAAM,KAAK,EAAE,CAAC;AAAA,EACtD;AAAA,EAEA,MAAM,YAAY,UAAqC;AACrD,QAAI,mBAAmB,MAAM,KAAK,oBAAoB,QAAQ;AAC9D,QAAI,MAAM,MAAM,KAAK,SAAS;AAAA;AAAA,aAErB,eAAe,gBAAgB,CAAC;AAAA;AAAA;AAAA,KAGxC;AACD,WAAO,IAAI,KAAK,IAAI,SAAO,OAAO,IAAI,aAAa,CAAC,EAAE,YAAY,CAAC;AAAA,EACrE;AAAA,EAEA,MAAM,WAAW,SAAoC;AACnD,QAAI,QAAQ,QAAQ,MAAM,GAAG;AAC7B,QAAI,WAAW,MAAM,KAAK,oBAAoB,MAAM,MAAM,KAAK,EAAE;AACjE,QAAI,SAAS,MAAM,KAAK,GAAG;AAE3B,QAAI,MAAM,MAAM,KAAK;AAAA,MACnB;AAAA;AAAA,aAEO,eAAe,QAAQ,CAAC;AAAA;AAAA;AAAA;AAAA,MAI/B,CAAC,MAAM;AAAA,IACT;AACA,WAAO,IAAI,KAAK,IAAI,SAAO,GAAG,OAAO,IAAI,cAAc,CAAC,EAAE,YAAY,CAAC,IAAI,OAAO,IAAI,YAAY,CAAC,EAAE,YAAY,CAAC,EAAE;AAAA,EACtH;AAAA,EAEA,MAAM,cAAc,QAAyC;AAC3D,QAAI,QAAQ,OAAO,MAAM,GAAG;AAC5B,QAAI,WAAW,MAAM,KAAK,oBAAoB,MAAM,MAAM,KAAK,EAAE;AACjE,QAAI,QAAQ,MAAM,IAAI,KAAK;AAC3B,QAAI,SAAS,MAAM,KAAK,GAAG;AAE3B,QAAI,MAAM,MAAM,KAAK;AAAA,MACnB;AAAA;AAAA,aAEO,eAAe,QAAQ,CAAC;AAAA;AAAA;AAAA;AAAA,MAI/B,CAAC,QAAQ,KAAK;AAAA,IAChB;AACA,WAAO,IAAI,KAAK,IAAI,SAAO;AACzB,aAAO,EAAC,MAAM,OAAO,IAAI,aAAa,CAAC,EAAE,YAAY,GAAG,UAAU,OAAO,IAAI,WAAW,CAAC,EAAC;AAAA,IAC5F,CAAC;AAAA,EACH;AAAA,EAEA,MAAM,oBAAoB,MAA+B;AACvD,QAAI,YAAY,MAAM,KAAK,aAAa;AACxC,WAAO,UAAU,KAAK,QAAM,GAAG,YAAY,KAAK,KAAK,YAAY,CAAC,KAAK;AAAA,EACzE;AAAA,EAEA,MAAM,QAAuB;AAC3B,UAAM,KAAK;AACX,UAAM,IAAI,QAAc,CAAC,SAAS,WAAW;AAC3C,WAAK,WAAW,QAAQ,SAAQ,MAAM,OAAO,GAAG,IAAI,QAAQ,CAAE;AAAA,IAChE,CAAC;AAAA,EACH;AACF;AAEA,SAAS,eAAe,OAAe;AACrC,MAAI,CAAC,MAAO,OAAM,IAAI,MAAM,uCAAuC;AACnE,SAAO,IAAI,MAAM,QAAQ,MAAM,IAAI,CAAC;AACtC;",
6
+ "names": []
7
+ }
@@ -0,0 +1,63 @@
1
+ // Standard Graphene error shape used by query responses and diagnostics.
2
+ // Result payload returned by Graphene query execution endpoints.
3
+ export interface QueryResult {
4
+ rows: any[]
5
+ fields: Field[]
6
+ error?: GrapheneError
7
+ hash?: string // hash of the compiled sql for caching
8
+ sql?: string
9
+ }
10
+
11
+ // A single output column in a query result.
12
+ export type Field = {
13
+ name: string
14
+ type: FieldType
15
+ metadata?: FieldMeta
16
+ }
17
+
18
+ // Metadata attached to fields.
19
+ // There are a few built-in ones that Graphene already uses, but you can always attach your own metadata:
20
+ // `price: cogs * 1.15 #ratio #format="US Dollar"` -> {ratio: true, format: 'US Dollar'}
21
+ export type FieldMeta = {
22
+ ratio?: true // 0 to 1 value
23
+ pct?: true // 0 to 100 value
24
+ units?: string
25
+ timeGrain?: TimeGrain // resolution when the field is a date or timestamp
26
+ timePart?: string // extracted temporal part, normalized across backend spellings
27
+ timeOrdinal?: TimeOrdinal // if the value represents something special like day_of_week, week_of_year, etc
28
+ defaultName?: string // preferred output column name when an expression is selected without an alias
29
+ [key: string]: string | true | undefined
30
+ }
31
+
32
+ export type FieldType = ScalarField | ArrayField
33
+ export type ScalarField = 'string' | 'number' | 'boolean' | 'date' | 'timestamp' | 'json' | 'sql native' | 'error' | 'null' | 'interval' | 'record'
34
+ export type ArrayField = {type: 'array'; elementType: FieldType}
35
+
36
+ export type TimeGrain = 'year' | 'quarter' | 'month' | 'week' | 'day' | 'hour' | 'minute' | 'second'
37
+ export type TimeOrdinal = 'hour_of_day' | DayOfWeekOrdinal | 'day_of_month' | 'day_of_year' | 'week_of_year' | 'month_of_year' | 'quarter_of_year'
38
+
39
+ export type DayOfWeekOrdinal =
40
+ | 'dow_1s' // 1-7, starting sunday
41
+ | 'dow_0s' // 0-6, starting sunday
42
+ | 'dow_1m' // 1-7, starting monday
43
+
44
+ export interface GrapheneError {
45
+ message: string
46
+ name?: string
47
+ stack?: string
48
+ cause?: unknown
49
+ severity?: 'error' | 'warn'
50
+ componentId?: string
51
+ file?: string
52
+ from?: Position
53
+ to?: Position
54
+ frame?: string
55
+ }
56
+
57
+ export interface Position {
58
+ offset: number
59
+ line: number
60
+ col: number
61
+ lineStart?: number
62
+ lineText?: string
63
+ }
@@ -0,0 +1,63 @@
1
+ // Standard Graphene error shape used by query responses and diagnostics.
2
+ // Result payload returned by Graphene query execution endpoints.
3
+ export interface QueryResult {
4
+ rows: any[]
5
+ fields: Field[]
6
+ error?: GrapheneError
7
+ hash?: string // hash of the compiled sql for caching
8
+ sql?: string
9
+ }
10
+
11
+ // A single output column in a query result.
12
+ export type Field = {
13
+ name: string
14
+ type: FieldType
15
+ metadata?: FieldMeta
16
+ }
17
+
18
+ // Metadata attached to fields.
19
+ // There are a few built-in ones that Graphene already uses, but you can always attach your own metadata:
20
+ // `price: cogs * 1.15 #ratio #format="US Dollar"` -> {ratio: true, format: 'US Dollar'}
21
+ export type FieldMeta = {
22
+ ratio?: true // 0 to 1 value
23
+ pct?: true // 0 to 100 value
24
+ units?: string
25
+ timeGrain?: TimeGrain // resolution when the field is a date or timestamp
26
+ timePart?: string // extracted temporal part, normalized across backend spellings
27
+ timeOrdinal?: TimeOrdinal // if the value represents something special like day_of_week, week_of_year, etc
28
+ defaultName?: string // preferred output column name when an expression is selected without an alias
29
+ [key: string]: string | true | undefined
30
+ }
31
+
32
+ export type FieldType = ScalarField | ArrayField
33
+ export type ScalarField = 'string' | 'number' | 'boolean' | 'date' | 'timestamp' | 'json' | 'sql native' | 'error' | 'null' | 'interval' | 'record'
34
+ export type ArrayField = {type: 'array'; elementType: FieldType}
35
+
36
+ export type TimeGrain = 'year' | 'quarter' | 'month' | 'week' | 'day' | 'hour' | 'minute' | 'second'
37
+ export type TimeOrdinal = 'hour_of_day' | DayOfWeekOrdinal | 'day_of_month' | 'day_of_year' | 'week_of_year' | 'month_of_year' | 'quarter_of_year'
38
+
39
+ export type DayOfWeekOrdinal =
40
+ | 'dow_1s' // 1-7, starting sunday
41
+ | 'dow_0s' // 0-6, starting sunday
42
+ | 'dow_1m' // 1-7, starting monday
43
+
44
+ export interface GrapheneError {
45
+ message: string
46
+ name?: string
47
+ stack?: string
48
+ cause?: unknown
49
+ severity?: 'error' | 'warn'
50
+ componentId?: string
51
+ file?: string
52
+ from?: Position
53
+ to?: Position
54
+ frame?: string
55
+ }
56
+
57
+ export interface Position {
58
+ offset: number
59
+ line: number
60
+ col: number
61
+ lineStart?: number
62
+ lineText?: string
63
+ }
@@ -0,0 +1,235 @@
1
+ ---
2
+ name: graphene
3
+ description: How to use Graphene, our framework for data modeling, analysis, and visualization.
4
+ ---
5
+
6
+ Graphene is a framework for doing data analysis and BI as code. Schema definitions and semantic models are in `.gsql` files, dashboards/notebooks (called pages) in `.md`.
7
+
8
+ # GSQL
9
+ GSQL extends ANSI SQL with dimensions, measures, and join relationships. Declare them in `table` statements:
10
+
11
+ ```sql
12
+ table orders (
13
+ id bigint
14
+ created_at datetime
15
+ user_id bigint
16
+ amount float #units=usd
17
+ status string
18
+ join one users on user_id = users.id -- many orders per user
19
+ is_complete: status = 'Complete' -- dimension (scalar expression)
20
+ revenue: sum(amount) -- measure (agg expression) #units=usd
21
+ avg_order: revenue / count(*) -- measures can compose #units=usd
22
+ )
23
+ table users (
24
+ id bigint
25
+ name varchar
26
+ join many orders on id = orders.user_id
27
+ )
28
+ ```
29
+
30
+ Other statements:
31
+ - `table X as (select ...)` defines a table as the result of a query
32
+ - `extend` adds dimensions, measures, and joins to an existing table, usually used with `table X as (select ...)`
33
+
34
+ ## Using semantics in queries
35
+ ### Implicit joins
36
+ - `from orders select status, user.name` will automatically join users on to orders per the model-defined join
37
+ - Use multiple dot operators to traverse multi-hop joins e.g. `from orders select status, order_items.inventory_items.avg_days_in_inventory -- orders -> order_items -> inventory_items
38
+ - Normal ANSI joins (`inner join`, `left join`, etc.) supported in `select` as well, if the join you need is not already modeled
39
+
40
+ ### Dimension and measure expansion
41
+ Dimensions and measures are like macros that expand inline when GSQL compiles to database SQL. For example, `from users select id, orders.revenue` automatically expands to `select users.id, sum(orders.amount) ...`
42
+ - NEVER(!): `sum(revenue)` or `group by revenue` because `revenue` is already an agg expression
43
+ - OK: `floor(revenue)`, `revenue / cost`
44
+ - OK: `sum(case when is_complete then 1 else 0 end)` or `group by is_complete` (because `is_complete` is a dimension, not a measure)
45
+
46
+ ### Arrays
47
+ - Array columns and casts use `array<T>` syntax in GSQL, for example `tags array<string>` or `cast(tags as array<string>)`
48
+ - Arrays can be expanded in queries with `cross join unnest(tags) as tag`
49
+
50
+ ### Special features
51
+ - `group by all` is implied when aggregates exist, and does not need to be put in GSQL
52
+ - Agg function `pXX(column)` computes the XXth percentile (e.g., p50, p975, p9999)
53
+ - `select`, `from`, `order by`, etc. in any order
54
+
55
+ ### Supported
56
+ - All scalar, agg, and window functions of the connected database
57
+ - ANSI joins, CTEs, subqueries, set operations
58
+
59
+ ### Not supported
60
+ - Table functions
61
+ - UDFs
62
+ - `pivot`, `lateral`
63
+ - `variant`, `object`, `record` types
64
+
65
+ # Pages
66
+ Graphene pages extend Markdown with the following:
67
+ - GSQL queries in code fences
68
+ - Visualization and input components
69
+
70
+ ````md
71
+ ---
72
+ title: My First Dashboard
73
+ layout: dashboard
74
+ ---
75
+
76
+ ```sql sales_by_status
77
+ select extract(year from created_at) AS year, status, revenue
78
+ from orders
79
+ where status <> 'cancelled'
80
+ ```
81
+
82
+ <BigValue data="orders" value="revenue" />
83
+ <BarChart data="sales_by_status" x="year" y="revenue" splitBy="status"/>
84
+ ````
85
+
86
+ Queries can be referenced by other queries in the `from` or `join` to form DAGs of data logic within the page.
87
+
88
+ ## Page frontmatter
89
+ You can add YAML frontmatter at the top of a page. The following attributes are supported:
90
+ - `title`: title displayed at the top of the page
91
+ - `layout`: `notebook` is the default, good for prose interspersed with charts. `dashboard` has a wider max-width, for chart-heavy pages with lots of `<Row>`s.
92
+
93
+ ## Viz and display components
94
+ - LineChart: title, data, x, y, y2, splitBy, sort, height, width
95
+ - AreaChart: title, data, x, y, y2, splitBy, arrange (`stack` (default) or `stack100`), sort, height, width
96
+ - BarChart: title, data, x, y, y2, splitBy, arrange (`stack` (default), `group`, or `stack100`), label (true or false (default); shows labels above bars), sort, height, width
97
+ - ScatterPlot: title, data, x, y, splitBy, height, width
98
+ - PieChart: title, data, category, value, height, width
99
+ - ECharts: data, height, width, renderer
100
+ - BigValue: title, data, value
101
+ - Table: title, data, rows, sortable, sort, groupBy, groupType, subtotals, totalRow, link, showLinkCol, rowShading, rowLines, rowNumbers, compact, headerColor, headerFontColor, totalRowColor, totalFontColor, backgroundColor, emptyMessage
102
+ - Column (sub-component of Table): id (column name), title, description, align, wrap, wrapTitle, colGroup, contentType, totalAgg, redNegatives
103
+ - Value: data, column, row
104
+ - Row (layout container, distributes children horizontally): No attributes
105
+
106
+ Notes on common attributes:
107
+ - `data` can also point at a modeled GSQL table.
108
+ - Any attribute that accepts a column can also accept an arbitrary GSQL expression. These attributes are x, y, y2, splitBy, category, value, link, groupBy, scaleColumn
109
+ - `splitBy` creates a series for each distinct value in the column (long format data).
110
+ - `y` can take a comma-separated list of columns/expressions, to map multiple fields to the same y-axis as separate series (wide format data).
111
+ - `sort` takes a column name followed by `asc` or `desc`, eg. `my_col desc`. Useful when you want something sorted differently than its inherent alphanumeric ordering.
112
+ - `height` and `width` accept any CSS size units eg. `240px` or `50%`.
113
+
114
+ ### `<ECharts>`
115
+ To create visualizations or customizations beyond Graphene's out-of-the-box components, specify an ECharts config via `<ECharts>`.
116
+
117
+ This example creates a stacked bar chart with a purple x-axis.
118
+
119
+ ```md
120
+ <ECharts data="sales_by_status">
121
+ title: {text: "Annual Revenue by Status"},
122
+ xAxis: {axisLine: {lineStyle: {color: 'purple'}}},
123
+ series: [{type: "bar", stack: "bar-stack", encode: {x: "year", y: "revenue", splitBy: "status"}],
124
+ </ECharts>
125
+ ```
126
+
127
+ Use `encode` to map objects to the columns of the data source. In Graphene, `encode` also accepts `splitBy` which automatically expands one template into multiple series. For bar charts, `splitBy` can be a two-item list (`[groupBy, stackBy]`) for grouped+stacked bars.
128
+
129
+ Graphene will handle axes, layout, and styles for you, so you can omit those configurations unless you explicitly want to override them.
130
+
131
+ Unsupported:
132
+ - `{@colName}` formatter templates (but `{a}`, `{b}`, `{c}` work)
133
+ - Javascript of any kind
134
+
135
+ ### `<Value>`
136
+ `<Value>` is used for inlining SQL-derived values within markdown text. You can place them anywhere in markdown, including headers, and they can be styled with `**` or `_`.
137
+
138
+ ```md
139
+ ### Top 3 Most Active Airplane Models
140
+ 1. **<Value data=top_airplane_models column=manufacturer_model row=0 />** ...
141
+ ```
142
+
143
+ ## Input components
144
+ - Dropdown: title, name, data, value (column to populate list with), label, defaultValue, multiple
145
+ - TextInput: title, name, placeholder
146
+ - DateRange: title, name, data, dates, start, end, defaultValue, presetRanges, description
147
+
148
+ Inject input values into queries by referring to their `name` attribute as `$name` in GSQL.
149
+
150
+ ````md
151
+ <Dropdown name=status .../>
152
+
153
+ ```sql my_query
154
+ select ...
155
+ where status = $status
156
+ ```
157
+ ````
158
+
159
+ DateRange components emit two referenceable values via `${name}_start` and `${name}_end`.
160
+
161
+ Input values also sync into the page URL query string (eg. `localhost:4000/my_dashboard?status=cancelled`), so reloads and shared links preserve the same dashboard state.
162
+
163
+ # CLI
164
+
165
+ Invoke the CLI via your project's package manager (e.g. `pnpm graphene check`, `npm exec graphene run`).
166
+
167
+ ```bash
168
+ graphene check # Check diagnostics across all .gsql files in the project
169
+ graphene check path/to/file.gsql # Check diagnostics for one specific gsql file
170
+ graphene check path/to/page.md # Check diagnostics for one specific markdown file
171
+
172
+ graphene run "from flights select count() as total" # Run inline GSQL and print results
173
+ graphene run - # Read GSQL from stdin and print results
174
+
175
+ graphene run path/to/page.md # Run the page and save a full-page screenshot
176
+
177
+ # `-c/--chart` can target either a chart title or the chart's `queryId`. For charts without titles use `graphene list` to see the exact IDs for charts on a page.
178
+ graphene run path/to/page.md -c "Chart Title" # Run the page and screenshot one chart by title
179
+ graphene run path/to/page.md -c 'Query (data="query_name" x="category" y="total")' # Run the page and screenshot one chart by queryId
180
+
181
+ # `-q/--query` can target anything usable in a chart `data` prop (for example, a gsql table or a named code-fenced query in the markdown file).
182
+ graphene run path/to/page.md -q query_name # Run a named query/table from the markdown context and print results
183
+
184
+ graphene compile "[GSQL]" # Show the compiled, dialect-specific SQL
185
+
186
+ # `schema` is for implementation/migration purposes and is NOT for exploring GSQL models
187
+ graphene schema # List datasets/schemas in the connected database
188
+ graphene schema my_dataset # List schemas (or tables) in a dataset
189
+ graphene schema my_dataset.table # Print the GSQL table statement for a database table
190
+
191
+ graphene serve # Start the local dev server (foreground)
192
+ graphene serve --bg # Start the local dev server in the background
193
+ graphene stop # Stop the background dev server
194
+ ```
195
+
196
+ # Best practices
197
+ - **Leverage models** - Use modeled joins, dimensions, and measures whenever possible
198
+ - Use `check` after editing .gsql files to catch syntax errors
199
+ - Use `run path/to/page.md` after editing a page to catch runtime errors and to review the screenshot
200
+ - Use `run path/to/page.md -c "[Chart Title | Query ID]"` after editing an ECharts component to view the screenshot
201
+ - When iterating on code-fenced GSQL queries, use `run path/to/page.md -q query_name` instead of `run "[GSQL]"`
202
+ - Rely on Graphene's defaults for value formatting and chart styles before trying to override in SQL or ECharts
203
+ - Keep numbers grounded - Use the `<Value/>` component in prose instead of hard-coding numbers
204
+ - When adding viz, think like Edward Tufte. What is _the_ most effective way to illustrate the data?
205
+
206
+ If the user asks:
207
+ - An open-ended question => notebook
208
+ - To create a page for monitoring purposes => dashboard
209
+ - A more straightforward, tactical question => no page, answer in chat
210
+
211
+ ## Notebook guide
212
+ - Use `layout: notebook` in frontmatter
213
+ - Queries should be point-in-time via absolute time filters
214
+ - Page should read like a narrative with prose interspersed with tables or visuals
215
+ - Try to use the most scientifically rigorous means possible to derive insights: think like a statistician or data scientist
216
+
217
+ ## Dashboard guide
218
+ - Use `layout: dashboard` in frontmatter
219
+ - Queries generally use relative time filters eg. last 3 months. If user doesn't specify, ask.
220
+ - Avoid narratives - Don't use headers, prose, other markdown content much if at all
221
+ - Use `<Row>` to create grids to fit the maximum amount of information in the viewport
222
+ - Ask the user if they would like any inputs to dynamically filter things
223
+
224
+ # Reference documentation
225
+ Consult the reference documentation for more detailed information on using Graphene.
226
+ For semantic modeling with GSQL references, read `references/model-gsql.md`.
227
+
228
+ - references/big-value.md
229
+ - references/date-range.md
230
+ - references/dropdown.md
231
+ - references/echarts.md
232
+ - references/gsql.md
233
+ - references/model-gsql.md
234
+ - references/table.md
235
+ - references/text-input.md
@@ -0,0 +1,20 @@
1
+ Use a BigValue to display a single large value standalone.
2
+
3
+ Here's an example:
4
+
5
+ ```markdown
6
+ <BigValue
7
+ data=orders
8
+ value=num_orders
9
+ title="Total Orders"
10
+ />
11
+ ```
12
+
13
+ # Attributes
14
+
15
+ | Attribute | Description | Required | Options | Default |
16
+ |------|-------------|----------|---------|---------|
17
+ | data | GSQL query or table name | true | query name | - |
18
+ | value | Column or expression to pull the main value from | true | column name, stored expression name, GSQL expression | - |
19
+ | title | Title displayed above the value | false | string | - |
20
+ | subtitle | Subtitle displayed below the title | false | string | - |