kpi_assembler 0.5.0 → 0.5.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 73810fbb8d6411ed8f52fd84ac84bed1c50372b2fa9cac5bb5bbce43de714130
-  data.tar.gz: efd60220f7d69ab53a23162048a65226a5ac5f4ab852e6685cde31c0298d4bab
+  metadata.gz: f8327e4d084d6aef240991acae7bfed8ed06af1059fb28154e7ed911f4359976
+  data.tar.gz: f4d84ae33c76e3b8174358d60509a1e5091703b081f9d4ee73d5317f7940115b
 SHA512:
-  metadata.gz: 10baca12a1597969b0bf610da6cbf7b505655b74b55b07d1532a661f7649e13ebc704f8e2a119855871877afa0d7c7f9fad349bf7460189d8ee98ae59f72e3e1
-  data.tar.gz: bdab7aef95e09eea0ae8d62a622859d0bd096e2c96072e642fd4e164a6835d1535a671b848a346d3e3d32cfa7285c389f304171bc8278e8b4803e50f50357738
+  metadata.gz: 992360ae2eb9c585c7e5ea48841feb6ff177d92b827d734038b4abc152e08bb743dd52dd5b51b2f02f75d45e0bb1355b609f206d6f1fa054e8ad75fc661f3bb1
+  data.tar.gz: a57880ff32be45512aa1bf409991090549e07192c685647119298f4f5afc4ebb1880afbd168d5ce8f96ef5773daece8ebabab597eecc6e92de2c75057dd166d3

data/README.md

@@ -1,82 +1,56 @@
 # KPIAssembler
 
-KPIAssembler discovers application schemas, proposes useful business metrics,
-and deterministically certifies generated SQL before publishing a KPI pack.
+KPIAssembler discovers application schemas, proposes business metrics, and
+certifies generated SQL before a number is trusted.
 
-The LLM proposes; deterministic code certifies. Candidate queries are checked
-against the real schema, tenant boundary, SQL safety rules, query planner, and
-sample execution before they can be marked certified.
+The LLM proposes. Deterministic code certifies.
 
-## Features
-
-- Mountable Rails Engine with an interactive workspace and JSON API
-- Standalone Rack application and command-line interface
-- SQLite and PostgreSQL schema introspection
-- Gemini and Ollama KPI proposal providers
-- Schema-driven heuristic fallback when no model is available
-- Deterministic SQL, join, division, tenant, and execution checks
-- JSON and HTML KPI packs
-
-## Install
-
-Add the gem to your application:
+## Install (Rails)
 
 ```ruby
-gem "kpi_assembler"
+# Gemfile
+gem "kpi_assembler", "~> 0.5"
 ```
 
-Then run:
-
 ```bash
 bundle install
 bin/rails generate kpi_assembler:install
 ```
 
-Mount the engine:
-
 ```ruby
+# config/routes.rb
 mount KPIAssembler::Engine => "/kpi-assembler"
 ```
 
-See [Rails Engine integration](docs/rails-engine.md) for configuration and
-[standalone integration](docs/integration.md) for Rack, CLI, and API usage.
+Then:
 
-## LLM configuration
+1. Edit `config/initializers/kpi_assembler.rb` (database pool, tenant, auth).
+2. Set `GEMINI_API_KEY` and `KPI_LLM_PROVIDER=gemini` in the host `.env`,
+   or use Ollama / `KPI_USE_LLM=false`.
+3. Restart the app and open `/kpi-assembler`.
 
-Gemini:
+Full walkthrough, env vars, and troubleshooting:
+**[Setup guide](docs/setup.md)**.
 
-```bash
-KPI_LLM_PROVIDER=gemini
-GEMINI_API_KEY=your-key
-KPI_GEMINI_MODEL=gemini-2.0-flash
-```
+Rails engine details: [docs/rails-engine.md](docs/rails-engine.md).
+Standalone Rack/CLI/API: [docs/integration.md](docs/integration.md).
 
-Ollama:
+## What it does
 
-```bash
-KPI_LLM_PROVIDER=ollama
-KPI_OLLAMA_MODEL=llama3.2:3b
-KPI_OLLAMA_URL=http://localhost:11434/api/generate
-```
-
-Set `KPI_USE_LLM=false` to use schema-driven heuristics only.
-
-## Development
-
-```bash
-bundle install
-bundle exec rspec
-bundle exec ruby bin/kpi_assembler --sample-db
-bundle exec ruby bin/kpi_assembler_web
-```
+1. **Discover** — introspect tables, columns, and foreign keys
+2. **Propose** — Gemini, Ollama, or schema heuristics draft KPI SQL
+3. **Accept** — you choose which candidates to certify
+4. **Certify** — read-only SELECT, join safety, tenant scope, planner, sample run
+5. **Publish** — certified pack as JSON (and HTML from the CLI)
 
-The standalone workspace starts at `http://127.0.0.1:9292`.
+Rejected candidates never reach certification. **Draft** means a selected KPI
+failed a certification check.
 
-## Security
+## Requirements
 
-Use a read-only database user or replica. Generated candidate SQL is executed
-during deterministic certification. Configure tenant scoping and authorization
-before exposing the engine or API.
+- Ruby >= 3.0
+- Rails 7 for the engine
+- Read-only database access (replica or reporting user)
 
 ## License
 

data/docs/integration.md

@@ -1,7 +1,9 @@
 # Standalone integration
 
-KPIAssembler can run as a Rack service and expose its workspace and JSON API to
-applications written in any language.
+For the Rails engine, start with the [setup guide](setup.md).
+
+KPIAssembler can also run as a Rack service and expose its workspace and JSON
+API to applications written in any language.
 
 ## Start the service
 

data/docs/rails-engine.md

@@ -1,92 +1,47 @@
-# Rails Engine integration
+# Rails Engine notes
 
-KPIAssembler is a mountable Rails Engine. It uses the host application's
-session, authorization, and Active Record connection pool.
+Use this after the [setup guide](setup.md). KPIAssembler mounts inside a Rails 7
+app, reuses the host session, and reads through Active Record.
 
-## Install
-
-```ruby
-gem "kpi_assembler"
-```
+## Generator
 
 ```bash
 bundle install
 bin/rails generate kpi_assembler:install
 ```
 
-The generator creates `config/initializers/kpi_assembler.rb`.
-
-## Configure
-
-```ruby
-KPIAssembler.configure do |config|
-  config.connection_provider = lambda do |_controller|
-    ApplicationRecord.connected_to(role: :reading) do
-      ApplicationRecord.connection_pool
-    end
-  end
-
-  config.parent_controller = "ApplicationController"
-  config.tenant_column = "account_id"
-  config.tenant_id_resolver = lambda do |controller|
-    controller.send(:current_account).id
-  end
-
-  config.include_tables = []
-  config.max_tables = 30
-  config.schema_name = "public"
-
-  config.authorize_with = lambda do |controller|
-    controller.send(:authenticate_user!)
-    controller.send(:current_account).present?
-  end
-
-  config.llm_provider = ENV.fetch("KPI_LLM_PROVIDER", "gemini").to_sym
-  config.use_llm = ENV["KPI_USE_LLM"] != "false"
-
-  config.gemini_api_key = ENV["GEMINI_API_KEY"]
-  config.gemini_model = ENV.fetch("KPI_GEMINI_MODEL", "gemini-2.0-flash")
-
-  config.ollama_model = ENV.fetch("KPI_OLLAMA_MODEL", "llama3.2:3b")
-  config.ollama_url = ENV.fetch(
-    "KPI_OLLAMA_URL",
-    "http://localhost:11434/api/generate"
-  )
-end
-```
-
-Store secrets in the host application's environment rather than the initializer:
-
-```bash
-KPI_LLM_PROVIDER=gemini
-GEMINI_API_KEY=your-key
-KPI_GEMINI_MODEL=gemini-2.0-flash
-```
-
-Restart the Rails server after changing environment variables.
+Creates `config/initializers/kpi_assembler.rb`. The template uses
+`current_company` / `company_id` as placeholders — change them to match your
+app (`current_account`, `tenant_id`, and so on).
 
-## Mount
-
-Add the engine to `config/routes.rb`:
+## Routes
 
 ```ruby
 mount KPIAssembler::Engine => "/kpi-assembler"
 ```
 
-The mounted endpoints are:
-
 ```text
 GET/POST /kpi-assembler/api/v1/discover
 POST     /kpi-assembler/api/v1/certify
 GET      /kpi-assembler/api/v1/pack
 ```
 
-## Production guidance
+Static UI files are served at `/kpi-assembler/assets/*` outside the host
+controller stack so JavaScript is not blocked as a cross-origin response.
+
+## Connection
+
+`connection_provider` must return a pool or connection `KPIAssembler::Connection`
+can wrap: `ActiveRecord::Base.connection_pool`, another pool, or a wrapped
+adapter. Prefer `connected_to(role: :reading)` when the host has a replica.
+
+## Tenancy
+
+If `tenant_column` is set and that column exists on a fact table, every
+certified query on that table must include `column = tenant_id`. Gemini often
+omits this; certification then marks the KPI draft.
 
-Point `connection_provider` at a read-only pool. Replace the example
-authorization callback with the host application's permission policy. Tenant
-scoping must correspond to real columns in the inspected tables.
+## Packs
 
-The engine currently keeps the latest pack in process memory per tenant.
-Production applications should persist versioned packs and define an
-invalidation policy.
+The engine stores the latest pack in memory per tenant. Restarting the process
+clears it. Persist packs in the host app before production use.

data/docs/setup.md

@@ -0,0 +1,236 @@
+# Setup guide
+
+KPIAssembler inspects your application schema, proposes KPI definitions, and
+certifies their SQL before anything is published. The model only proposes.
+Deterministic Ruby certifies.
+
+There are two install paths:
+
+1. **Rails engine** — mount the workspace inside an existing Rails 7 app
+2. **Standalone service** — run the Rack UI/API next to any application
+
+Most teams should use the Rails engine.
+
+## Requirements
+
+- Ruby 3.0 or newer
+- Rails 7.x for the engine (`railties >= 7.0, < 8`)
+- A **read-only** database user, replica, or reporting pool
+- Optional: a [Gemini API key](https://aistudio.google.com/apikey) or a local
+  [Ollama](https://ollama.com) daemon
+
+Do not point certification at a production write primary.
+
+## Rails engine
+
+### 1. Add the gem
+
+```ruby
+# Gemfile
+gem "kpi_assembler", "~> 0.5"
+```
+
+```bash
+bundle install
+bin/rails generate kpi_assembler:install
+```
+
+The generator writes `config/initializers/kpi_assembler.rb`.
+
+### 2. Mount the workspace
+
+In `config/routes.rb`, mount inside whatever scope already requires a signed-in
+user:
+
+```ruby
+mount KPIAssembler::Engine => "/kpi-assembler"
+```
+
+Open `/kpi-assembler` after signing in.
+
+JSON endpoints:
+
+```text
+GET/POST /kpi-assembler/api/v1/discover
+POST     /kpi-assembler/api/v1/certify
+GET      /kpi-assembler/api/v1/pack
+```
+
+### 3. Edit the initializer
+
+Replace the sample hooks with your app's connection, tenant, and auth.
+
+```ruby
+KPIAssembler.configure do |config|
+  # Prefer a read/report pool, not the write primary.
+  config.connection_provider = lambda do |_controller|
+    ApplicationRecord.connected_to(role: :reading) do
+      ApplicationRecord.connection_pool
+    end
+  end
+
+  config.parent_controller = "ApplicationController"
+
+  # Column that exists on fact tables. Omit tenant_column if you have none.
+  config.tenant_column = "account_id"
+  config.tenant_id_resolver = lambda do |controller|
+    controller.send(:current_account).id
+  end
+
+  # Empty allowlist = inspect the live schema (up to max_tables).
+  # Names must match the database (people, not Person).
+  config.include_tables = []
+  config.max_tables = 30
+  config.schema_name = "public"
+
+  config.authorize_with = lambda do |controller|
+    controller.send(:authenticate_user!)
+    controller.send(:current_account).present?
+  end
+
+  config.llm_provider = ENV.fetch("KPI_LLM_PROVIDER", "gemini").to_sym
+  config.use_llm = ENV["KPI_USE_LLM"] != "false"
+  config.gemini_api_key = ENV["GEMINI_API_KEY"]
+  config.gemini_model = ENV.fetch("KPI_GEMINI_MODEL", "gemini-2.0-flash")
+  config.ollama_model = ENV.fetch("KPI_OLLAMA_MODEL", "llama3.2:3b")
+  config.ollama_url = ENV.fetch("KPI_OLLAMA_URL", "http://localhost:11434/api/generate")
+end
+```
+
+If you have no `reading` role, return `ActiveRecord::Base.connection_pool`
+(still with a read-only user).
+
+### 4. Set environment variables
+
+Put secrets in the host app `.env` (or your secret manager). Do not commit keys.
+
+**Gemini (cloud):**
+
+```bash
+KPI_LLM_PROVIDER=gemini
+GEMINI_API_KEY=your-key
+KPI_GEMINI_MODEL=gemini-2.0-flash
+```
+
+You do not set a Gemini URL. The default endpoint is built from the model name.
+
+**Ollama (local):**
+
+```bash
+KPI_LLM_PROVIDER=ollama
+KPI_OLLAMA_MODEL=llama3.2:3b
+KPI_OLLAMA_URL=http://localhost:11434/api/generate
+```
+
+Start the daemon first: `ollama serve`, then `ollama pull llama3.2:3b`.
+
+**Heuristics only:**
+
+```bash
+KPI_USE_LLM=false
+```
+
+Restart `bin/rails server` after changing env vars or the initializer.
+
+### 5. First run
+
+1. Sign in and open `/kpi-assembler`.
+2. Click **Discover metrics**. Schema cards and candidate KPIs should appear.
+3. Select the definitions you want. Rejected items never reach certification.
+4. Click **Certify**. Certified KPIs become the pack. Failures land in **draft**
+   with reasons (unsafe SQL, missing tenant scope, planner error, NULL sample
+   replay). Draft is not the same as rejecting a KPI in the UI.
+
+The workspace shows whether the proposer was `llm` or `heuristics`, the model
+name, and any tables the model ranked.
+
+## Configuration reference
+
+| Setting | Purpose |
+| --- | --- |
+| `connection_provider` | Returns an AR pool or connection used for inspect + certify |
+| `tenant_column` / `tenant_id_resolver` | Injects tenant filters into generated SQL |
+| `include_tables` | Optional allowlist. Empty means inspect the live schema |
+| `max_tables` | Cap after ranking; omitted tables are reported in the UI |
+| `schema_name` | PostgreSQL schema (default `public`) |
+| `parent_controller` | Host controller the engine inherits (auth callbacks) |
+| `authorize_with` | Extra gate; must not return `false` |
+| `llm_provider` | `:gemini` or `:ollama` |
+| `use_llm` | `false` skips the model and uses schema heuristics |
+| `gemini_api_key` / `gemini_model` | Gemini credentials |
+| `ollama_model` / `ollama_url` | Local Ollama |
+
+Environment variables the gem reads: `KPI_LLM_PROVIDER`, `KPI_USE_LLM`,
+`GEMINI_API_KEY`, `KPI_GEMINI_MODEL`, `KPI_GEMINI_URL`, `KPI_OLLAMA_MODEL`,
+`KPI_OLLAMA_URL`, `KPI_USE_OLLAMA` (Ollama-only off switch).
+
+## Standalone service
+
+Use this when the host app is not Rails, or you want a separate process.
+
+```bash
+gem install kpi_assembler
+# or clone the repo and: bundle install
+```
+
+From the gem source checkout:
+
+```bash
+cp .env.example .env
+# set KPI_DATABASE_URL or KPI_DATABASE_PATH, plus LLM vars
+bundle exec ruby bin/kpi_assembler_web
+```
+
+Opens `http://127.0.0.1:9292`.
+
+Useful env vars:
+
+```bash
+HOST=0.0.0.0
+PORT=9292
+KPI_DATABASE_URL=postgres://USER:PASSWORD@HOST:5432/application
+KPI_DATABASE_PATH=/absolute/path/to/application.sqlite3
+KPI_INCLUDE_TABLES=orders,customers,accounts
+KPI_TENANT_COLUMN=account_id
+KPI_TENANT_ID=123
+KPI_ALLOWED_ORIGIN=https://your-app.example
+KPI_LLM_PROVIDER=gemini
+GEMINI_API_KEY=your-key
+```
+
+Embed:
+
+```html
+<script src="http://localhost:9292/kpi-assembler.js"></script>
+<kpi-assembler service-url="http://localhost:9292" height="820px"></kpi-assembler>
+```
+
+CLI against the sample database:
+
+```bash
+bundle exec ruby bin/kpi_assembler --sample-db
+```
+
+See [standalone integration](integration.md) for the JSON API.
+
+## Troubleshooting
+
+| Symptom | What to check |
+| --- | --- |
+| Discover does nothing / non-JSON error | Host redirected to login. Send the request while signed in. Confirm `/kpi-assembler/assets/app.js` returns 200. |
+| `heuristics` proposer, Gemini fallback | `GEMINI_API_KEY` missing or HTTP error. The UI shows `proposer_fallback`. |
+| `heuristics` proposer, Ollama fallback | `ollama serve` not running, or model not pulled. Fallback text names installed models. |
+| UI does not show tables you added | `include_tables` names must match the DB. Check `missing_tables` / `omitted_tables` (`max_tables`). |
+| Many KPIs become **draft** after certify | Open `reasons`. Common: missing `tenant_column = id`, invented tables, `NULLIF` producing NULL on empty windows. |
+| `json` / `quirks_mode` on Rails 7 | Pin `gem "json", "< 3"` in the **host** Gemfile. |
+| Slow or truncated LLM output | Lower `max_tables`, or use `gemini-2.0-flash` / a small Ollama model. |
+
+## Security
+
+- Authorize the mount the same way as the rest of the app. The extra
+  `authorize_with` gate is not a substitute for a real permission check.
+- Certification **executes** candidate `SELECT`s. Use a least-privilege
+  read-only role.
+- Tenant SQL is only injected when `tenant_column` exists on the fact table.
+- The latest pack is kept in process memory per tenant. Persist packs yourself
+  before relying on this in production.

data/lib/generators/kpi_assembler/install_generator.rb

@@ -19,6 +19,8 @@ module KPIAssembler
       def print_mount_instruction
         say "\nMount KPIAssembler in config/routes.rb:", :green
         say '  mount KPIAssembler::Engine => "/kpi-assembler"'
+        say "\nThen set GEMINI_API_KEY (or KPI_LLM_PROVIDER=ollama) and restart."
+        say "Setup guide: https://github.com/Akshatsrivastava700/kpi_assembler/blob/main/docs/setup.md"
       end
     end
   end

data/lib/generators/kpi_assembler/templates/kpi_assembler.rb

@@ -42,6 +42,6 @@ KPIAssembler.configure do |config|
   config.gemini_model = ENV.fetch("KPI_GEMINI_MODEL", "gemini-2.0-flash")
 
   # Ollama configuration
-  config.ollama_model = ENV.fetch("KPI_OLLAMA_MODEL", "llama3")
+  config.ollama_model = ENV.fetch("KPI_OLLAMA_MODEL", "llama3.2:3b")
   config.ollama_url = ENV.fetch("KPI_OLLAMA_URL", "http://localhost:11434/api/generate")
 end

data/lib/kpi_assembler/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module KPIAssembler
-  VERSION = "0.5.0"
+  VERSION = "0.5.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: kpi_assembler
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.5.1
 platform: ruby
 authors:
 - KPIAssembler contributors
@@ -47,6 +47,7 @@ files:
 - config/routes.rb
 - docs/integration.md
 - docs/rails-engine.md
+- docs/setup.md
 - lib/generators/kpi_assembler/install_generator.rb
 - lib/generators/kpi_assembler/templates/kpi_assembler.rb
 - lib/kpi_assembler.rb
@@ -79,6 +80,7 @@ metadata:
   homepage_uri: https://github.com/Akshatsrivastava700/kpi_assembler
   source_code_uri: https://github.com/Akshatsrivastava700/kpi_assembler
   changelog_uri: https://github.com/Akshatsrivastava700/kpi_assembler/releases
+  documentation_uri: https://github.com/Akshatsrivastava700/kpi_assembler/blob/main/docs/setup.md
 post_install_message: 
 rdoc_options: []
 require_paths: