@graphenedata/cli 0.0.14 → 0.0.16

package/LICENSE.md

@@ -5,7 +5,7 @@ default throughout the repository is a license under the Elastic License 2.0,
 unless a file header or a license file in a subdirectory specifies another
 license.
 
---------------------------------------------------------------------------------
+---
 
 Elastic License 2.0
 
@@ -72,10 +72,10 @@ to terminate automatically and permanently.
 
 ## No Liability
 
-*As far as the law allows, the software comes as is, without any warranty or
+_As far as the law allows, the software comes as is, without any warranty or
 condition, and the licensor will not be liable to you for any damages arising
 out of these terms or the use or nature of the software, under any kind of
-legal claim.*
+legal claim._
 
 ## Definitions
 

package/README.md

@@ -0,0 +1,138 @@
+<div align="center">
+  <br/>
+  <a href="https://graphenedata.com">
+      <img height="125" alt="Graphene Logo" src="./assets/logo.png" />
+  </a>
+  <br/>
+  <br/>
+</div>
+
+<p align="center">
+  <b>Graphene</b> is a data analytics framework built for agents.
+  <br/>
+  Ask questions and build visualizations 10x faster when agents do the work.
+</p>
+
+<div align="center">
+  <a href="https://graphenedata.com">Website</a>
+  &nbsp;•&nbsp;
+  <a href="https://github.com/graphene-data/example-flights">Demo Project</a>
+  &nbsp;•&nbsp;
+  <a href="/docs/setup.md">Setup</a>
+</div>
+
+<br/>
+
+## Why Graphene?
+
+In the future, we believe **most low-level data analytics work will be done by agents**, allowing humans to focus on insights and decision-making. However, today's tools weren't built with agents in mind:
+
+- They are GUI centric. Lots of actions can only be done via GUI and aren't accessible to external agents like Claude.
+- They focus on raising the floor at the expense of lowering the ceiling (limited viz types, simplified querying APIs).
+- They assume the human user has the tribal knowledge and business context necessary for analysis.
+
+If we really want agents to be more productive with data, an entirely new toolset is needed.
+
+Graphene is:
+
+- [x] **Built for agents**. Everything is code, written only in languages that are prevalent in training data (SQL, Markdown, HTML). All actions are CLIs; nothing is trapped in a button.
+- [x] **High-ceiling**. Agents can create any visualization that's supported by ECharts, one of the most feature-complete visualization libraries. And Graphene's query language is as powerful as ANSI SQL, which supports 170+ functions, CTEs, subqueries, set operations, window functions, arrays, and more.
+- [x] **Optimized for agent context**. Graphene's SQL language contains a semantic layer which allows metrics and join relationships to be invoked in queries. When combined with [agent skills](https://agentskills.io/home) for general business context and best practices, agents perform at human levels of competency.
+
+### Open, forever
+
+Importantly, Graphene is **open**. You can use this project for internal purposes for free, forever, and aren't locked in to a contract with us. More details [below](#faq).
+
+### Rich visualizations
+
+Graphene pages support visualizations, input components for filtering and dynamic behaviors, and layout modes for monitoring-oriented dashboards vs. narrative-oriented notebooks.
+
+<img alt="Graphene Screenshots" src="./assets/page_examples.png"/>
+
+### Powerful workflows
+
+When you deconstruct data analytics into code, CLIs, and coding agents, things that used to be hard become easy:
+
+- Promote metrics from pages into the model, or demote metrics out of the model back into pages
+- Bulk refactors in a single atomic commit/PR
+- Ability to use extensive skill/MCP ecosystem to augment agent behavior
+- Iterate on a dashboard (edit, run, view) without needing to push up to some API or open a SaaS portal
+- Validate SQL and page syntax instantaneously as you type
+- Set up a recurring agent that de-bloats, consolidates your model over time
+
+## Get started
+
+- [Try the demo project](https://github.com/graphene-data/example-flights)
+- [Create a new Graphene project](/docs/setup.md)
+
+Graphene currently supports Snowflake, BigQuery, ClickHouse, and local data (via DuckDB) as data sources. It is easy for us to add more - just ask.
+
+Once your project is set up, simply start the dev server via `npm exec graphene serve` (or `pnpm graphene serve`, etc. based on your package manager) and then prompt your coding agent to do analytics work: answer a data question, build a dashboard, edit the model, etc.
+
+## How it works
+
+A Graphene project can either be a standalone repo or a directory within a larger codebase (such as dbt). It is comprised of:
+
+- **Semantic models**, via .gsql files. GSQL is both a modeling language and a query language, in the same way that SQL has both DDL and DML.
+- **Pages**, via .md files. Pages are typically used for dashboards, but can also contain notebook-style narratives, documentation, and other visual content.
+
+Graphene itself is a CLI which can be installed via npm (or pnpm, yarn, etc.). The CLI can run and compile GSQL queries, render pages in the browser, check syntax, print screenshots, and more.
+
+### GSQL and Graphene markdown
+
+Semantic models are defined like so:
+
+```sql
+table orders (
+  id BIGINT
+  user_id BIGINT
+  amount FLOAT
+  status STRING
+
+  join one users on user_id = users.id  -- many orders per user
+
+  is_complete: status = 'Complete'      -- dimension (scalar expression)
+  revenue: sum(amount)                  -- measure (agg expression)
+  aov: revenue / count(*)               -- measures can compose
+)
+
+table users (
+  id BIGINT
+  name VARCHAR
+
+  join many orders on id = orders.user_id
+)
+```
+
+Models are then queried via `select`, either directly via CLI or inside a Graphene markdown page like this.
+
+````md
+```sql top_customers
+select
+  users.name as name,   -- Use the dot operator to traverse the modeled join relationship
+  revenue               -- Invokes the measure
+from orders             -- A join statement here is not needed
+group by 1
+order by 2 desc
+limit 10
+```
+
+<BigValue data="orders" value="revenue" />
+<BarChart data="top_customers" x="name" y="revenue" />
+````
+
+## Documentation
+
+Graphene's entire documentation ships as an agent skill in the Graphene npm package. The source files are available [here](/docs).
+
+## FAQ
+
+<details>
+<summary><b>So does everyone have to use git and a coding agent to use Graphene for BI?</b></summary>
+If you just want to use this project and nothing more, yes. Our managed service, Graphene Cloud, offers a Slack agent, MCP server, and browser-based SaaS experience.
+</details>
+
+<details>
+<summary><b>What software license does this use?</b></summary>
+Graphene is licensed under the Elastic License 2.0 which allows you to use it for internal use cases for free, forever. If you would like to use the Graphene Cloud services above, or if you would like a commercial license, please contact us <a href="https://graphenedata.com/contact-us">here</a>.
+</details>

package/THIRD_PARTY_NOTICES.md

@@ -1,6 +1,7 @@
 This project includes code from the [Evidence](https://github.com/evidence-dev/evidence) open source project.
 
 ---
+
 MIT License
 Copyright (c) 2023 Evidence
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

package/bin.js

@@ -1,8 +1,8 @@
 #!/usr/bin/env node
 
-import {fileURLToPath} from 'url'
-import {dirname, join} from 'path'
 import {existsSync} from 'fs'
+import {dirname, join} from 'path'
+import {fileURLToPath} from 'url'
 
 const __dirname = dirname(fileURLToPath(import.meta.url))
 const devCli = join(__dirname, 'cli.ts')

package/dist/cli/bigQuery-I3F46SC6.js

@@ -0,0 +1,75 @@
+import {
+  config
+} from "./chunk-QAXEOZ43.js";
+
+// connections/bigQuery.ts
+import { BigQuery, BigQueryDate, BigQueryTimestamp } from "@google-cloud/bigquery";
+function validateBigQueryIdent(ident) {
+  if (!/^[\w.-]+$/.test(ident)) throw new Error(`Invalid BigQuery identifier: ${ident}`);
+}
+var BigQueryConnection = class {
+  client;
+  projectId;
+  defaultNamespace;
+  constructor(options = {}) {
+    options.projectId ||= config.bigquery?.projectId;
+    if (!options.projectId) throw new Error("projectId must be set in config or provided in service account credentials");
+    this.projectId = options.projectId;
+    this.client = new BigQuery({ ...options, userAgent: "Graphene" });
+    this.defaultNamespace = config.defaultNamespace;
+  }
+  async runQuery(sql, params) {
+    let [job] = await this.client.createQueryJob({ query: sql, useLegacySql: false, params });
+    let [rows] = await job.getQueryResults({ maxResults: 1e4 });
+    let metadata = job.metadata || (await job.getMetadata())[0];
+    let totalRows = Number(metadata?.statistics?.query?.totalRows ?? rows.length);
+    rows.forEach((r) => {
+      Object.entries(r).forEach(([k, v]) => {
+        if (v instanceof BigQueryTimestamp) r[k] = v.value;
+        if (v instanceof BigQueryDate) r[k] = v.value;
+      });
+    });
+    return { rows, totalRows };
+  }
+  async listDatasets() {
+    let [datasets] = await this.client.getDatasets();
+    return datasets.map((d) => String(d.id || d.metadata.datasetReference?.datasetId || "").toLowerCase());
+  }
+  async listTables(dataset) {
+    if (!dataset) throw new Error("BigQuery requires a dataset");
+    validateBigQueryIdent(dataset);
+    let resolvedDataset = await this.resolveDatasetName(dataset);
+    let res = await this.runQuery(`select table_name as table_name
+      from \`${resolvedDataset}.INFORMATION_SCHEMA.TABLES\`
+      where table_type in ('BASE TABLE', 'VIEW') order by table_name`);
+    return res.rows.map((r) => `${resolvedDataset.toLowerCase()}.${String(r["table_name"]).toLowerCase()}`);
+  }
+  async describeTable(target) {
+    let parts = target.split(".");
+    let table = parts.pop() || "";
+    let dataset = parts.join(".") || this.defaultNamespace;
+    if (!dataset) throw new Error("No dataset specified and no default namespace configured");
+    validateBigQueryIdent(dataset);
+    let resolvedDataset = await this.resolveDatasetName(dataset);
+    let sql = `
+      select column_name as column_name, data_type as data_type, ordinal_position as ordinal_position
+      from \`${resolvedDataset}.INFORMATION_SCHEMA.COLUMNS\`
+      where lower(table_name) = lower(@table)
+      order by ordinal_position
+    `.trim();
+    let res = await this.runQuery(sql, { table });
+    return res.rows.map((row) => {
+      return { name: String(row["column_name"]).toLowerCase(), dataType: String(row["data_type"]) };
+    });
+  }
+  async resolveDatasetName(name) {
+    let datasets = await this.listDatasets();
+    return datasets.find((ds) => ds.toLowerCase() == name.toLowerCase()) || name;
+  }
+  async close() {
+  }
+};
+export {
+  BigQueryConnection
+};
+//# sourceMappingURL=bigQuery-I3F46SC6.js.map

package/dist/cli/bigQuery-I3F46SC6.js.map

@@ -0,0 +1,7 @@
+{
+  "version": 3,
+  "sources": ["../../connections/bigQuery.ts"],
+  "sourcesContent": ["import {BigQuery, BigQueryDate, BigQueryTimestamp, type BigQueryOptions} from '@google-cloud/bigquery'\n\nimport {config} from '../../lang/config.ts'\nimport {type QueryConnection, type QueryResult, type SchemaColumn, type QueryParams} from './types.ts'\n\n// BigQuery identifiers can contain letters, numbers, underscores, and hyphens\nfunction validateBigQueryIdent(ident: string) {\n  if (!/^[\\w.-]+$/.test(ident)) throw new Error(`Invalid BigQuery identifier: ${ident}`)\n}\n\nexport class BigQueryConnection implements QueryConnection {\n  private readonly client: BigQuery\n  private readonly projectId: string\n  private readonly defaultNamespace?: string\n\n  constructor(options: BigQueryOptions = {}) {\n    options.projectId ||= config.bigquery?.projectId\n    if (!options.projectId) throw new Error('projectId must be set in config or provided in service account credentials')\n    this.projectId = options.projectId\n    this.client = new BigQuery({...options, userAgent: 'Graphene'})\n    this.defaultNamespace = config.defaultNamespace\n  }\n\n  async runQuery(sql: string, params?: QueryParams): Promise<QueryResult> {\n    let [job] = await this.client.createQueryJob({query: sql, useLegacySql: false, params})\n    let [rows] = await job.getQueryResults({maxResults: 10000})\n    let metadata = job.metadata || (await job.getMetadata())[0]\n    let totalRows = Number(metadata?.statistics?.query?.totalRows ?? rows.length)\n\n    rows.forEach(r => {\n      Object.entries(r).forEach(([k, v]) => {\n        if (v instanceof BigQueryTimestamp) r[k] = v.value\n        if (v instanceof BigQueryDate) r[k] = v.value\n      })\n    })\n\n    return {rows, totalRows}\n  }\n\n  async listDatasets(): Promise<string[]> {\n    let [datasets] = await this.client.getDatasets()\n    return datasets.map(d => String(d.id || d.metadata.datasetReference?.datasetId || '').toLowerCase())\n  }\n\n  async listTables(dataset?: string): Promise<string[]> {\n    if (!dataset) throw new Error('BigQuery requires a dataset')\n    validateBigQueryIdent(dataset)\n\n    let resolvedDataset = await this.resolveDatasetName(dataset)\n    let res = await this.runQuery(`select table_name as table_name\n      from \\`${resolvedDataset}.INFORMATION_SCHEMA.TABLES\\`\n      where table_type in ('BASE TABLE', 'VIEW') order by table_name`)\n\n    return res.rows.map(r => `${resolvedDataset.toLowerCase()}.${String(r['table_name']).toLowerCase()}`)\n  }\n\n  async describeTable(target: string): Promise<SchemaColumn[]> {\n    let parts = target.split('.')\n    let table = parts.pop() || ''\n    let dataset = parts.join('.') || this.defaultNamespace\n    if (!dataset) throw new Error('No dataset specified and no default namespace configured')\n    validateBigQueryIdent(dataset)\n    let resolvedDataset = await this.resolveDatasetName(dataset)\n    let sql = `\n      select column_name as column_name, data_type as data_type, ordinal_position as ordinal_position\n      from \\`${resolvedDataset}.INFORMATION_SCHEMA.COLUMNS\\`\n      where lower(table_name) = lower(@table)\n      order by ordinal_position\n    `.trim()\n    let res = await this.runQuery(sql, {table})\n    return res.rows.map(row => {\n      return {name: String(row['column_name']).toLowerCase(), dataType: String(row['data_type'])}\n    })\n  }\n\n  async resolveDatasetName(name: string): Promise<string> {\n    let datasets = await this.listDatasets()\n    return datasets.find(ds => ds.toLowerCase() == name.toLowerCase()) || name\n  }\n\n  async close(): Promise<void> {}\n}\n"],
+  "mappings": ";;;;;AAAA,SAAQ,UAAU,cAAc,yBAA8C;AAM9E,SAAS,sBAAsB,OAAe;AAC5C,MAAI,CAAC,YAAY,KAAK,KAAK,EAAG,OAAM,IAAI,MAAM,gCAAgC,KAAK,EAAE;AACvF;AAEO,IAAM,qBAAN,MAAoD;AAAA,EACxC;AAAA,EACA;AAAA,EACA;AAAA,EAEjB,YAAY,UAA2B,CAAC,GAAG;AACzC,YAAQ,cAAc,OAAO,UAAU;AACvC,QAAI,CAAC,QAAQ,UAAW,OAAM,IAAI,MAAM,4EAA4E;AACpH,SAAK,YAAY,QAAQ;AACzB,SAAK,SAAS,IAAI,SAAS,EAAC,GAAG,SAAS,WAAW,WAAU,CAAC;AAC9D,SAAK,mBAAmB,OAAO;AAAA,EACjC;AAAA,EAEA,MAAM,SAAS,KAAa,QAA4C;AACtE,QAAI,CAAC,GAAG,IAAI,MAAM,KAAK,OAAO,eAAe,EAAC,OAAO,KAAK,cAAc,OAAO,OAAM,CAAC;AACtF,QAAI,CAAC,IAAI,IAAI,MAAM,IAAI,gBAAgB,EAAC,YAAY,IAAK,CAAC;AAC1D,QAAI,WAAW,IAAI,aAAa,MAAM,IAAI,YAAY,GAAG,CAAC;AAC1D,QAAI,YAAY,OAAO,UAAU,YAAY,OAAO,aAAa,KAAK,MAAM;AAE5E,SAAK,QAAQ,OAAK;AAChB,aAAO,QAAQ,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,MAAM;AACpC,YAAI,aAAa,kBAAmB,GAAE,CAAC,IAAI,EAAE;AAC7C,YAAI,aAAa,aAAc,GAAE,CAAC,IAAI,EAAE;AAAA,MAC1C,CAAC;AAAA,IACH,CAAC;AAED,WAAO,EAAC,MAAM,UAAS;AAAA,EACzB;AAAA,EAEA,MAAM,eAAkC;AACtC,QAAI,CAAC,QAAQ,IAAI,MAAM,KAAK,OAAO,YAAY;AAC/C,WAAO,SAAS,IAAI,OAAK,OAAO,EAAE,MAAM,EAAE,SAAS,kBAAkB,aAAa,EAAE,EAAE,YAAY,CAAC;AAAA,EACrG;AAAA,EAEA,MAAM,WAAW,SAAqC;AACpD,QAAI,CAAC,QAAS,OAAM,IAAI,MAAM,6BAA6B;AAC3D,0BAAsB,OAAO;AAE7B,QAAI,kBAAkB,MAAM,KAAK,mBAAmB,OAAO;AAC3D,QAAI,MAAM,MAAM,KAAK,SAAS;AAAA,eACnB,eAAe;AAAA,qEACuC;AAEjE,WAAO,IAAI,KAAK,IAAI,OAAK,GAAG,gBAAgB,YAAY,CAAC,IAAI,OAAO,EAAE,YAAY,CAAC,EAAE,YAAY,CAAC,EAAE;AAAA,EACtG;AAAA,EAEA,MAAM,cAAc,QAAyC;AAC3D,QAAI,QAAQ,OAAO,MAAM,GAAG;AAC5B,QAAI,QAAQ,MAAM,IAAI,KAAK;AAC3B,QAAI,UAAU,MAAM,KAAK,GAAG,KAAK,KAAK;AACtC,QAAI,CAAC,QAAS,OAAM,IAAI,MAAM,0DAA0D;AACxF,0BAAsB,OAAO;AAC7B,QAAI,kBAAkB,MAAM,KAAK,mBAAmB,OAAO;AAC3D,QAAI,MAAM;AAAA;AAAA,eAEC,eAAe;AAAA;AAAA;AAAA,MAGxB,KAAK;AACP,QAAI,MAAM,MAAM,KAAK,SAAS,KAAK,EAAC,MAAK,CAAC;AAC1C,WAAO,IAAI,KAAK,IAAI,SAAO;AACzB,aAAO,EAAC,MAAM,OAAO,IAAI,aAAa,CAAC,EAAE,YAAY,GAAG,UAAU,OAAO,IAAI,WAAW,CAAC,EAAC;AAAA,IAC5F,CAAC;AAAA,EACH;AAAA,EAEA,MAAM,mBAAmB,MAA+B;AACtD,QAAI,WAAW,MAAM,KAAK,aAAa;AACvC,WAAO,SAAS,KAAK,QAAM,GAAG,YAAY,KAAK,KAAK,YAAY,CAAC,KAAK;AAAA,EACxE;AAAA,EAEA,MAAM,QAAuB;AAAA,EAAC;AAChC;",
+  "names": []
+}