blazer 2.0.0 → 2.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 71916198bf2c9162e234001aa3a76c5620c1c445d30377276dab114cbee6a4e7
-  data.tar.gz: abc445f95958ddec14cc49e6f39ab41993d0ebe10536b0b4349bf60cfcdf63c1
+  metadata.gz: 9f1910484a7cc50172c7b6dface23cc593514fcf9382abc726da8b08169af1e9
+  data.tar.gz: 98db8372f805a616a944d7c4bf595c3a6cf4d9ee138e150eb54e7a27fed5dd48
 SHA512:
-  metadata.gz: 45b008e981848ae5ab0b6cffffeb1954359c3c7815511d5025c8ca63def983341dbc2a0f1a97f71d9ae883ba058934656559db69feb96fbc39b7d38b13173698
-  data.tar.gz: 73c09348ab9ac9556283585dbc0b74bef4bb58f415798023e710c5de489c9b548af4d7830acdf4250f84a0f70c3ed6b44f2636b11fe6758bee0d3058af1bda95
+  metadata.gz: 373a56a0559c54617a03d18df7849d68840fb6275f5b3150bc35d800ee5f3efaca15bd5d3aaec5e4df25c3a3c450e2c98673ec9a5242979ed02565fc9c6c66e3
+  data.tar.gz: b8118434afa964ff808252e4cb2bf2855d3088b718e18675c9efb32a7be3934082558b703354f1f160687a8416fd068c60bf88af905dd5ebe8326c9919788b7b

data/CHANGELOG.md

@@ -1,3 +1,14 @@
+## 2.0.1
+
+- Added favicon
+- Added search for checks and schema
+- Added pie charts
+- Added Trend anomaly detection
+- Added forecasting
+- Improved tooltips
+- Improved docs for new installs
+- Fixed error with canceling queries
+
 ## 2.0.0
 
 - Added support for Slack

data/README.md

@@ -6,6 +6,8 @@ Explore your data with SQL. Easily create charts and dashboards, and share them
 
 [![Screenshot](https://blazer.dokkuapp.com/assets/screenshot-6ca3115a518b488026e48be83ba0d4c9.png)](https://blazer.dokkuapp.com)
 
+Blazer 2.0 was recently released! See [instructions for upgrading](#20).
+
 :tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
 
 ## Features
@@ -92,45 +94,6 @@ BLAZER_SLACK_WEBHOOK_URL=https://hooks.slack.com/...
 
 Name the webhook “Blazer” and add a cool icon.
 
-## Permissions
-
-### PostgreSQL
-
-Create a user with read only permissions:
-
-```sql
-BEGIN;
-CREATE ROLE blazer LOGIN PASSWORD 'secret123';
-GRANT CONNECT ON DATABASE database_name TO blazer;
-GRANT USAGE ON SCHEMA public TO blazer;
-GRANT SELECT ON ALL TABLES IN SCHEMA public TO blazer;
-ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO blazer;
-COMMIT;
-```
-
-### MySQL
-
-Create a user with read only permissions:
-
-```sql
-GRANT SELECT, SHOW VIEW ON database_name.* TO blazer@’127.0.0.1′ IDENTIFIED BY ‘secret123‘;
-FLUSH PRIVILEGES;
-```
-
-### MongoDB
-
-Create a user with read only permissions:
-
-```
-db.createUser({user: "blazer", pwd: "password", roles: ["read"]})
-```
-
-Also, make sure authorization is enabled when you start the server.
-
-### Sensitive Data
-
-Check out [Hypershield](https://github.com/ankane/hypershield) to shield sensitive data.
-
 ## Authentication
 
 Don’t forget to protect the dashboard in production.
@@ -171,6 +134,47 @@ end
 
 Be sure to render or redirect for unauthorized users.
 
+## Permissions
+
+Blazer runs each query in a transaction and rolls it back to prevent queries from modifying data. As an additional line of defense, we recommend using a read only user.
+
+### PostgreSQL
+
+Create a user with read only permissions:
+
+```sql
+BEGIN;
+CREATE ROLE blazer LOGIN PASSWORD 'secret123';
+GRANT CONNECT ON DATABASE database_name TO blazer;
+GRANT USAGE ON SCHEMA public TO blazer;
+GRANT SELECT ON ALL TABLES IN SCHEMA public TO blazer;
+ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO blazer;
+COMMIT;
+```
+
+### MySQL
+
+Create a user with read only permissions:
+
+```sql
+GRANT SELECT, SHOW VIEW ON database_name.* TO blazer@’127.0.0.1′ IDENTIFIED BY ‘secret123‘;
+FLUSH PRIVILEGES;
+```
+
+### MongoDB
+
+Create a user with read only permissions:
+
+```
+db.createUser({user: "blazer", pwd: "password", roles: ["read"]})
+```
+
+Also, make sure authorization is enabled when you start the server.
+
+## Sensitive Data
+
+If your database contains sensitive or personal data, check out [Hypershield](https://github.com/ankane/hypershield) to shield it.
+
 ## Queries
 
 ### Variables
@@ -314,12 +318,20 @@ SELECT gender, zip_code, COUNT(*) FROM users GROUP BY 1, 2
 
 ### Scatter Chart
 
-2 columns - both numeric
+2 columns - both numeric - [Example](https://blazer.dokkuapp.com/queries/16-scatter-chart)
 
 ```sql
 SELECT x, y FROM table
 ```
 
+### Pie Chart
+
+2 columns - string, numeric - and last column named `pie` - [Example](https://blazer.dokkuapp.com/queries/17-pie-chart)
+
+```sql
+SELECT gender, COUNT(*) AS pie FROM users GROUP BY 1
+```
+
 ### Maps
 
 Columns named `latitude` and `longitude` or `lat` and `lon` or `lat` and `lng` - [Example](https://blazer.dokkuapp.com/queries/15-map)
@@ -360,7 +372,27 @@ Then create check with optional emails if you want to be notified. Emails are se
 
 ## Anomaly Detection
 
-Anomaly detection is supported thanks to Twitter’s [AnomalyDetection](https://github.com/twitter/AnomalyDetection) library.
+Blazer supports two different approaches to anomaly detection.
+
+### Trend
+
+[Trend](https://trendapi.org/) is easiest to set up but uses an external service.
+
+Add [trend](https://github.com/ankane/trend) to your Gemfile:
+
+```ruby
+gem 'trend'
+```
+
+And add to `config/blazer.yml`:
+
+```yml
+anomaly_checks: trend
+```
+
+### R
+
+R is harder to set up but doesn’t use an external service. It uses Twitter’s [AnomalyDetection](https://github.com/twitter/AnomalyDetection) library.
 
 First, [install R](https://cloud.r-project.org/). Then, run:
 
@@ -372,13 +404,33 @@ devtools::install_github("twitter/AnomalyDetection")
 And add to `config/blazer.yml`:
 
 ```yml
-anomaly_checks: true
+anomaly_checks: r
 ```
 
 If upgrading from version 1.4 or below, also follow the [upgrade instructions](#15).
 
 If you’re on Heroku, follow [these additional instructions](#anomaly-detection-on-heroku).
 
+## Forecasting
+
+Blazer has experimental support for forecasting through [Trend](https://trendapi.org/).
+
+[Example](https://blazer.dokkuapp.com/queries/18-forecast?forecast=t)
+
+Add [trend](https://github.com/ankane/trend) to your Gemfile:
+
+```ruby
+gem 'trend'
+```
+
+And add to `config/blazer.yml`:
+
+```yml
+forecasting: trend
+```
+
+A forecast link will appear for queries that return 2 columns with types timestamp and numeric.
+
 ## Data Sources
 
 Blazer supports multiple data sources :tada:

data/app/assets/javascripts/blazer/routes.js

@@ -6,6 +6,9 @@ var Routes = {
     return rootPath + "queries/cancel"
   },
   schema_queries_path: function(params) {
+    return rootPath + "queries/schema?" + $.param(params)
+  },
+  docs_queries_path: function(params) {
     return rootPath + "queries/docs?" + $.param(params)
   },
   tables_queries_path: function(params) {

data/app/assets/stylesheets/blazer/application.css

@@ -12,8 +12,8 @@ pre {
 }
 
 body {
-  padding-top: 20px;
-  padding-bottom: 20px;
+  padding-top: 15px;
+  padding-bottom: 15px;
 }
 
 .results-table th {
@@ -146,9 +146,10 @@ input.search:focus {
   top: 0;
   left: 0;
   right: 0;
-  background-color: whitesmoke;
   height: 60px;
   z-index: 1001;
+  border-bottom: solid 1px whitesmoke;
+  background-color: #fff;
 }
 
 .glyphicon-remove {
@@ -185,7 +186,6 @@ input.search:focus {
 }
 
 .chart-container {
-  padding-top: 10px;
   clear: both;
 }
 
@@ -201,10 +201,34 @@ input.search:focus {
   color: red;
 }
 
+.small-form {
+  margin-right: auto;
+  margin-left: auto;
+  max-width: 400px;
+}
+
+.alert {
+  padding-top: 8px;
+  padding-bottom: 8px;
+}
+
+h1, h2, h3, h4, p, hr, .table, .navbar, #header, .alert, .form-group {
+  margin-top: 0;
+  margin-bottom: 15px;
+}
+
+.double-margin, .chart-container {
+  margin-bottom: 30px;
+}
+
 h1 {
-  font-size: 32px;
+  font-size: 24px;
 }
 
 h2 {
-  font-size: 24px;
+  font-size: 20px;
+}
+
+.schema-table {
+  max-width: 500px;
 }

data/app/controllers/blazer/dashboards_controller.rb

@@ -2,10 +2,6 @@ module Blazer
   class DashboardsController < BaseController
     before_action :set_dashboard, only: [:show, :edit, :update, :destroy, :refresh]
 
-    def index
-      redirect_to root_path(filter: "dashboards")
-    end
-
     def new
       @dashboard = Blazer::Dashboard.new
     end

data/app/controllers/blazer/queries_controller.rb

@@ -1,16 +1,12 @@
 module Blazer
   class QueriesController < BaseController
     before_action :set_query, only: [:show, :edit, :update, :destroy, :refresh]
-    before_action :set_data_source, only: [:tables, :docs, :cancel]
+    before_action :set_data_source, only: [:tables, :docs, :schema, :cancel]
 
     def home
-      if params[:filter] == "dashboards"
-        @queries = []
-      else
-        set_queries(1000)
-      end
+      set_queries(1000)
 
-      if params[:filter] && params[:filter] != "dashboards"
+      if params[:filter]
         @dashboards = [] # TODO show my dashboards
       else
         @dashboards = Blazer::Dashboard.order(:name)
@@ -137,6 +133,13 @@ module Blazer
           @cached_at = @result.cached_at
           @just_cached = @result.just_cached
 
+          @forecast = @query && @result.forecastable? && params[:forecast]
+          if @forecast
+            @result.forecast
+            @forecast_error = @result.forecast_error
+            @forecast = @forecast_error.nil?
+          end
+
           render_run
         else
           @timestamp = Time.now.to_i
@@ -181,6 +184,12 @@ module Blazer
     end
 
     def docs
+      @smart_variables = @data_source.smart_variables
+      @linked_columns = @data_source.linked_columns
+      @smart_columns = @data_source.smart_columns
+    end
+
+    def schema
       @schema = @data_source.schema
     end
 

data/app/helpers/blazer/base_helper.rb

@@ -12,7 +12,7 @@ module Blazer
     BLAZER_IMAGE_EXT = %w[png jpg jpeg gif]
 
     def blazer_format_value(key, value)
-      if value.is_a?(Integer) && !key.to_s.end_with?("id") && !key.to_s.start_with?("id")
+      if value.is_a?(Numeric) && !key.to_s.end_with?("id") && !key.to_s.start_with?("id")
         number_with_delimiter(value)
       elsif value =~ BLAZER_URL_REGEX
         # see if image or link

data/app/views/blazer/_nav.html.erb

@@ -5,7 +5,6 @@
     <span class="sr-only">Toggle Dropdown</span>
   </button>
   <ul class="dropdown-menu">
-    <li><%= link_to "Dashboards", dashboards_path %></li>
     <li><%= link_to "Checks", checks_path %></li>
     <li role="separator" class="divider"></li>
     <li><%= link_to "New Query", new_query_path %></li>

data/app/views/blazer/_variables.html.erb

@@ -8,7 +8,7 @@
       return moment.tz(time.format(format), timeZone)
     }
   </script>
-  <form id="bind" method="get" action="<%= action %>" class="form-inline" style="margin-bottom: 10px;">
+  <form id="bind" method="get" action="<%= action %>" class="form-inline" style="margin-bottom: 15px;">
     <% date_vars = ["start_time", "end_time"] %>
     <% if (date_vars - @bind_vars).empty? %>
       <% @bind_vars = @bind_vars - date_vars %>
@@ -49,9 +49,11 @@
               datePicker.find("span").html(toDate(picker.startDate).format("MMMM D, YYYY"))
               input.val(toDate(picker.startDate).utc().format())
               submitIfCompleted($("#<%= var %>").closest("form"))
-            });
-            var picker = datePicker.data("daterangepicker")
-            datePicker.find("span").html(toDate(picker.startDate).format("MMMM D, YYYY"))
+            })
+            if (input.val().length > 0) {
+              var picker = datePicker.data("daterangepicker")
+              datePicker.find("span").html(toDate(picker.startDate).format("MMMM D, YYYY"))
+            }
           })()
         </script>
       <% else %>

data/app/views/blazer/checks/_form.html.erb

@@ -1,12 +1,12 @@
-<% unless @check.respond_to?(:invert) %>
-  <p class="text-muted">Checks are designed to identify bad data. A check fails if there are any results.</p>
-<% end %>
+<%= form_for @check, html: {class: "small-form"} do |f| %>
+  <% unless @check.respond_to?(:check_type) || @check.respond_to?(:invert) %>
+    <p class="text-muted">Checks are designed to identify bad data. A check fails if there are any results.</p>
+  <% end %>
 
-<% if @check.errors.any? %>
-  <div class="alert alert-danger"><%= @check.errors.full_messages.first %></div>
-<% end %>
+  <% if @check.errors.any? %>
+    <div class="alert alert-danger"><%= @check.errors.full_messages.first %></div>
+  <% end %>
 
-<%= form_for @check do |f| %>
   <div class="form-group">
     <%= f.label :query_id, "Query" %>
     <div class="hide">

data/app/views/blazer/checks/edit.html.erb

@@ -1 +1,3 @@
+<% blazer_title "Edit Check" %>
+
 <%= render partial: "form" %>

data/app/views/blazer/checks/index.html.erb

@@ -1,9 +1,26 @@
 <% blazer_title "Checks" %>
 
-<p style="float: right;"><%= link_to "New Check", new_check_path, class: "btn btn-info" %></p>
-<%= render partial: "blazer/nav" %>
+<div id="header">
+  <div class="pull-right" style="line-height: 34px;">
+    <div class="btn-group">
+      <%= link_to "New Check", new_check_path, class: "btn btn-info" %>
+      <button type="button" class="btn btn-info dropdown-toggle" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
+        <span class="caret"></span>
+        <span class="sr-only">Toggle Dropdown</span>
+      </button>
+      <ul class="dropdown-menu">
+        <li><%= link_to "Home", root_path %></li>
+        <li role="separator" class="divider"></li>
+        <li><%= link_to "New Query", new_query_path %></li>
+        <li><%= link_to "New Dashboard", new_dashboard_path %></li>
+      </ul>
+    </div>
+  </div>
 
-<table class="table">
+  <input id="search" type="text" placeholder="Start typing a query or state" style="width: 300px; display: inline-block;" class="search form-control" />
+</div>
+
+<table id="checks" class="table">
   <thead>
     <tr>
       <th>Query</th>
@@ -41,3 +58,12 @@
     <% end %>
   </tbody>
 </table>
+
+<script>
+  $("#search").on("keyup", function() {
+    var value = $(this).val().toLowerCase()
+    $("#checks tbody tr").filter( function() {
+      $(this).toggle($(this).text().toLowerCase().indexOf(value) > -1)
+    })
+  }).focus()
+</script>

data/app/views/blazer/checks/new.html.erb

@@ -1 +1,3 @@
+<% blazer_title "New Check" %>
+
 <%= render partial: "form" %>

data/app/views/blazer/dashboards/_form.html.erb

@@ -1,8 +1,8 @@
-<% if @dashboard.errors.any? %>
-  <div class="alert alert-danger"><%= @dashboard.errors.full_messages.first %></div>
-<% end %>
+<%= form_for @dashboard, url: (@dashboard.persisted? ? dashboard_path(@dashboard, variable_params) : dashboards_path(variable_params)), html: {id: "app", class: "small-form"} do |f| %>
+  <% if @dashboard.errors.any? %>
+    <div class="alert alert-danger"><%= @dashboard.errors.full_messages.first %></div>
+  <% end %>
 
-<%= form_for @dashboard, url: (@dashboard.persisted? ? dashboard_path(@dashboard, variable_params) : dashboards_path(variable_params)), html: {id: "app"} do |f| %>
   <div class="form-group">
     <%= f.label :name %>
     <%= f.text_field :name, class: "form-control" %>

data/app/views/blazer/dashboards/edit.html.erb

@@ -1 +1,3 @@
+<% blazer_title "Edit Dashboard" %>
+
 <%= render partial: "form" %>

data/app/views/blazer/dashboards/new.html.erb

@@ -1 +1,3 @@
+<% blazer_title "New Dashboard" %>
+
 <%= render partial: "form" %>

data/app/views/blazer/dashboards/show.html.erb

@@ -5,7 +5,7 @@
     <div class="row" style="padding-top: 13px;">
       <div class="col-sm-9">
         <%= render partial: "blazer/nav" %>
-        <h3 style="margin: 0; line-height: 34px; display: inline;">
+        <h3 style="line-height: 34px; display: inline; margin-left: 5px;">
           <%= @dashboard.name %>
         </h3>
       </div>
@@ -25,7 +25,11 @@
   </p>
 <% end %>
 
-<%= render partial: "blazer/variables", locals: {action: dashboard_path(@dashboard)} %>
+<% if @bind_vars.any? %>
+  <%= render partial: "blazer/variables", locals: {action: dashboard_path(@dashboard)} %>
+<% else %>
+  <div style="padding-bottom: 15px;"></div>
+<% end %>
 
 <% @queries.each_with_index do |query, i| %>
   <div class="chart-container">
@@ -35,7 +39,7 @@
     </div>
   </div>
   <script>
-    <%= blazer_js_var "data", {statement: query.statement, query_id: query.id, only_chart: true} %>
+    <%= blazer_js_var "data", {statement: query.statement, query_id: query.id, data_source: query.data_source, only_chart: true} %>
 
     runQuery(data, function (data) {
       $("#chart-<%= i %>").html(data)

data/app/views/blazer/queries/_form.html.erb

@@ -12,11 +12,13 @@
             <div id="editor" :style="{ height: editorHeight }"><%= @query.statement %></div>
           </div>
         </div>
-        <div class="form-group text-right">
-          <div class="pull-left" style="margin-top: 9px;">
+        <div class="form-group text-right" style="margin-bottom: 8px;">
+          <div class="pull-left" style="margin-top: 8px;">
             <%= link_to "Back", :back %>
+            <a :href="docsPath" target="_blank" style="margin-left: 40px;">Docs</a>
+            <a :href="schemaPath" target="_blank" style="margin-left: 40px;">Schema</a>
           </div>
-          <a :href="dataSourcePath" target="_blank" style="margin-right: 10px;">Docs</a>
+
           <%= f.select :data_source, Blazer.data_sources.values.select { |ds| q = @query.dup; q.data_source = ds.id; q.editable?(blazer_user) }.map { |ds| [ds.name, ds.id] }, {}, class: ("hide" if Blazer.data_sources.size <= 1), style: "width: 140px;" %>
           <div id="tables" style="display: inline-block; width: 250px; margin-right: 10px;">
             <select id="table_names" style="width: 240px;" placeholder="Preview table"></select>
@@ -34,7 +36,7 @@
           <%= f.label :description %>
           <%= f.text_area :description, placeholder: "Optional", style: "height: 80px;", class: "form-control" %>
         </div>
-        <div class="text-right">
+        <div class="form-group text-right">
           <%= f.submit "For Enter Press", class: "hide" %>
           <% if @query.persisted? %>
             <%= link_to "Delete", query_path(@query), method: :delete, "data-confirm" => "Are you sure?", class: "btn btn-danger" %>
@@ -49,7 +51,7 @@
           <% words << pluralize(dashboards_count, "dashboard") if dashboards_count > 0 %>
           <% words << pluralize(checks_count, "check") if checks_count > 0 %>
           <% if words.any? %>
-            <div class="alert alert-info" style="margin-top: 10px; padding: 8px 12px;">
+            <div class="alert alert-info" style="margin-bottom: 0;">
               Part of <%= words.to_sentence %>. Be careful when editing.
             </div>
           <% end %>
@@ -79,8 +81,11 @@
       editorHeight: "180px"
     },
     computed: {
-      dataSourcePath: function() {
+      schemaPath: function() {
         return Routes.schema_queries_path({data_source: this.dataSource})
+      },
+      docsPath: function() {
+        return Routes.docs_queries_path({data_source: this.dataSource})
       }
     },
     methods: {

data/app/views/blazer/queries/docs.html.erb

@@ -1,69 +1,83 @@
-<% blazer_title @data_source.name %>
+<% blazer_title "Docs: #{@data_source.name}" %>
 
-<h1><%= @data_source.name %></h1>
+<h1>Docs: <%= @data_source.name %></h1>
+
+<hr />
 
 <h2>Smart Variables</h2>
 
-<p>Use these variable names to get a dropdown.</p>
+<% if @smart_variables.any? %>
+  <p>Use these variable names to get a dropdown of values.</p>
 
-<table class="table" style="max-width: 500px;">
-  <thead>
-    <tr>
-      <th>Variable</th>
-    </tr>
-  </thead>
-  <tbody>
-    <% @data_source.smart_variables.each do |k, _| %>
+  <table class="table" style="max-width: 500px;">
+    <thead>
       <tr>
-        <td><code>{<%= k %>}</code></td>
+        <th>Variable</th>
       </tr>
-    <% end %>
-  </tbody>
-</table>
+    </thead>
+    <tbody>
+      <% @smart_variables.each do |k, _| %>
+        <tr>
+          <td><code>{<%= k %>}</code></td>
+        </tr>
+      <% end %>
+    </tbody>
+  </table>
 
-<p>Use <code>{start_time}</code> and <code>{end_time}</code> for a date range selector. End a variable name with <code>_at</code> for a date selector.</p>
+  <p>Use <code>{start_time}</code> and <code>{end_time}</code> for a date range selector. End a variable name with <code>_at</code> for a date selector.</p>
+<% else %>
+  <p>None set - add them in <code>config/blazer.yml</code>.</p>
+<% end %>
 
 <h2>Linked Columns</h2>
 
-<p>Use these column names to link results to other pages.</p>
+<% if @linked_columns.any? %>
+  <p>Use these column names to link results to other pages.</p>
 
-<table class="table" style="max-width: 500px;">
-  <thead>
-    <tr>
-      <th style="width: 20%;">Name</th>
-      <th>URL</th>
-    </tr>
-  </thead>
-  <tbody>
-    <% @data_source.linked_columns.each do |k, v| %>
+  <table class="table" style="max-width: 500px;">
+    <thead>
       <tr>
-        <td><%= k %></td>
-        <td><%= v %></td>
+        <th style="width: 20%;">Name</th>
+        <th>URL</th>
       </tr>
-    <% end %>
-  </tbody>
-</table>
+    </thead>
+    <tbody>
+      <% @linked_columns.each do |k, v| %>
+        <tr>
+          <td><%= k %></td>
+          <td><%= v %></td>
+        </tr>
+      <% end %>
+    </tbody>
+  </table>
 
-<p>Values that match the format of a URL will be linked automatically.</p>
+  <p>Values that match the format of a URL will be linked automatically.</p>
+<% else %>
+  <p>None set - add them in <code>config/blazer.yml</code>.</p>
+<% end %>
 
 <h2>Smart Columns</h2>
 
-<p>Use these column names to show additional data.</p>
+<% if @smart_columns.any? %>
+  <p>Use these column names to show additional data.</p>
 
-<table class="table" style="max-width: 500px;">
-  <thead>
-    <tr>
-      <th>Name</th>
-    </tr>
-  </thead>
-  <tbody>
-    <% @data_source.smart_columns.each do |k, _| %>
+  <table class="table" style="max-width: 500px;">
+    <thead>
       <tr>
-        <td><%= k %></td>
+        <th>Name</th>
       </tr>
-    <% end %>
-  </tbody>
-</table>
+    </thead>
+    <tbody>
+      <% @smart_columns.each do |k, _| %>
+        <tr>
+          <td><%= k %></td>
+        </tr>
+      <% end %>
+    </tbody>
+  </table>
+<% else %>
+  <p>None set - add them in <code>config/blazer.yml</code>.</p>
+<% end %>
 
 <h2>Charts</h2>
 
@@ -97,6 +111,10 @@
       <td>Scatter</td>
       <td>2 columns - both numeric</td>
     </tr>
+    <tr>
+      <td>Pie</td>
+      <td>2 columns - string, numeric - and last column named <code>pie</code></td>
+    </tr>
     <tr>
       <td>Map</td>
       <td>
@@ -111,28 +129,3 @@
 </table>
 
 <p>Use the column name <code>target</code> to draw a line for goals.</p>
-
-<h2>Schema</h2>
-
-<% @schema.each do |table| %>
-  <table class="table" style="max-width: 500px;">
-    <thead>
-      <tr>
-        <th colspan="2">
-          <%= table[:table] %>
-          <% if table[:schema] != "public" %>
-            <span class="text-muted" style="font-weight: normal;"><%= table[:schema] %></span>
-          <% end %>
-        </th>
-      </tr>
-    </thead>
-    <tbody>
-      <% table[:columns].each do |column| %>
-        <tr>
-          <td style="width: 60%;"><%= column[:name] %></td>
-          <td class="text-muted"><%= column[:data_type] %></td>
-        </tr>
-      <% end %>
-    </tbody>
-  </table>
-<% end %>

data/app/views/blazer/queries/home.html.erb

@@ -1,6 +1,6 @@
 <div id="queries">
-  <div id="header" style="margin-bottom: 20px;">
-    <div class="pull-right">
+  <div id="header">
+    <div class="pull-right" style="line-height: 34px;">
       <% if blazer_user %>
         <%= link_to "All", root_path, class: !params[:filter] ? "active" : nil, style: "margin-right: 40px;" %>
 
@@ -10,6 +10,7 @@
 
         <%= link_to "Mine", root_path(filter: "mine"), class: params[:filter] == "mine" ? "active" : nil, style: "margin-right: 40px;" %>
       <% end %>
+
       <div class="btn-group">
         <%= link_to "New Query", new_query_path, class: "btn btn-info" %>
         <button type="button" class="btn btn-info dropdown-toggle" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
@@ -17,7 +18,6 @@
           <span class="sr-only">Toggle Dropdown</span>
         </button>
         <ul class="dropdown-menu">
-          <li><%= link_to "Dashboards", dashboards_path %></li>
           <li><%= link_to "Checks", checks_path %></li>
           <li role="separator" class="divider"></li>
           <li><%= link_to "New Dashboard", new_dashboard_path %></li>
@@ -25,7 +25,7 @@
         </ul>
       </div>
     </div>
-    <input type="text" v-model="searchTerm" placeholder="Start typing a query or person" style="width: 300px; display: inline-block;" autofocus=true class="search form-control" />
+    <input type="text" v-model="searchTerm" placeholder="Start typing a query, dashboard, or person" style="width: 300px; display: inline-block;" v-focus class="search form-control" />
   </div>
 
   <table class="table">
@@ -147,6 +147,13 @@
           return Routes.query_path(item.to_param)
         }
       }
+    },
+    directives: {
+      focus: {
+        inserted: function (el) {
+          el.focus()
+        }
+      }
     }
   })
 </script>

data/app/views/blazer/queries/run.html.erb

@@ -24,7 +24,7 @@
         <% end %>
       </p>
     <% end %>
-    <p class="text-muted">
+    <p class="text-muted" style="margin-bottom: 10px;">
       <%= pluralize(@rows.size, "row") %>
 
       <% @checks.select(&:state).each do |check| %>
@@ -33,19 +33,46 @@
           &middot; <%= check.message %>
         <% end %>
       <% end %>
+
+      <% if @query && @result.forecastable? && !params[:forecast] %>
+        &middot;
+        <%= link_to "Forecast", query_path(@query, {forecast: "t"}.merge(variable_params)) %>
+      <% end %>
     </p>
   <% end %>
+  <% if @forecast_error %>
+    <div class="alert alert-danger"><%= @forecast_error %></div>
+  <% end %>
   <% if @rows.any? %>
     <% values = @rows.first %>
     <% chart_id = SecureRandom.hex %>
     <% column_types = @result.column_types %>
     <% chart_type = @result.chart_type %>
-    <% chart_options = {id: chart_id, min: nil} %>
+    <% chart_options = {id: chart_id} %>
+    <% if ["line", "line2"].include?(chart_type) %>
+      <% chart_options.merge!(min: nil) %>
+    <% end %>
+    <% if chart_type == "scatter" %>
+      <% chart_options.merge!(library: {tooltips: {intersect: false}}) %>
+    <% elsif ["bar", "bar2"].include?(chart_type) %>
+      <% chart_options.merge!(library: {tooltips: {intersect: false, axis: 'x'}}) %>
+    <% elsif chart_type != "pie" %>
+      <% if column_types.size == 2 || @forecast %>
+        <% chart_options.merge!(library: {tooltips: {intersect: false, axis: 'x'}}) %>
+      <% else %>
+        <%# chartjs axis: 'x' has poor behavior with multiple series %>
+        <% chart_options.merge!(library: {tooltips: {intersect: false}}) %>
+      <% end %>
+    <% end %>
     <% series_library = {} %>
     <% target_index = @columns.index { |k| k.downcase == "target" } %>
     <% if target_index %>
       <% series_library[target_index - 1] = {pointStyle: "line", hitRadius: 5, borderColor: "#109618", pointBackgroundColor: "#109618", backgroundColor: "#109618"} %>
     <% end %>
+    <% if @forecast %>
+      <% color = "#54a3ee" %>
+      <% series_library[1] = {borderDash: [8], borderColor: color, pointBackgroundColor: color, backgroundColor: color, pointHoverBackgroundColor: color} %>
+    <% end %>
     <% if blazer_maps? && @markers.any? %>
       <div id="map" style="height: <%= @only_chart ? 300 : 500 %>px;"></div>
       <script>
@@ -76,11 +103,14 @@
         map.fitBounds(featureLayer.getBounds());
       </script>
     <% elsif chart_type == "line" %>
-      <%= line_chart @columns[1..-1].each_with_index.map{ |k, i| {name: blazer_series_name(k), data: @rows.map{ |r| [r[0], r[i + 1]] }, library: series_library[i]} }, chart_options %>
+      <% chart_data = @columns[1..-1].each_with_index.map{ |k, i| {name: blazer_series_name(k), data: @rows.map{ |r| [r[0], r[i + 1]] }, library: series_library[i]} } %>
+      <%= line_chart chart_data, chart_options %>
     <% elsif chart_type == "line2" %>
       <%= line_chart @rows.group_by { |r| v = r[1]; (@boom[@columns[1]] || {})[v.to_s] || v }.each_with_index.map { |(name, v), i| {name: blazer_series_name(name), data: v.map { |v2| [v2[0], v2[2]] }, library: series_library[i]} }, chart_options %>
+    <% elsif chart_type == "pie" %>
+      <%= pie_chart @rows.map { |r| [(@boom[@columns[0]] || {})[r[0].to_s] || r[0], r[1]] }, chart_options %>
     <% elsif chart_type == "bar" %>
-      <%= column_chart (values.size - 1).times.map { |i| name = @columns[i + 1]; {name: blazer_series_name(name), data: @rows.first(20).map { |r| [(@boom[@columns[0]] || {})[r[0].to_s] || r[0], r[i + 1]] } } }, id: chart_id %>
+      <%= column_chart (values.size - 1).times.map { |i| name = @columns[i + 1]; {name: blazer_series_name(name), data: @rows.first(20).map { |r| [(@boom[@columns[0]] || {})[r[0].to_s] || r[0], r[i + 1]] } } }, chart_options %>
     <% elsif chart_type == "bar2" %>
       <% first_20 = @rows.group_by { |r| r[0] }.values.first(20).flatten(1) %>
       <% labels = first_20.map { |r| r[0] }.uniq %>
@@ -90,9 +120,9 @@
           <% first_20 << [l, s, 0] unless first_20.find { |r| r[0] == l && r[1] == s } %>
         <% end %>
       <% end %>
-      <%= column_chart first_20.group_by { |r| v = r[1]; (@boom[@columns[1]] || {})[v.to_s] || v }.each_with_index.map { |(name, v), i| {name: blazer_series_name(name), data: v.sort_by { |r2| labels.index(r2[0]) }.map { |v2| v3 = v2[0]; [(@boom[@columns[0]] || {})[v3.to_s] || v3, v2[2]] }} }, id: chart_id %>
+      <%= column_chart first_20.group_by { |r| v = r[1]; (@boom[@columns[1]] || {})[v.to_s] || v }.each_with_index.map { |(name, v), i| {name: blazer_series_name(name), data: v.sort_by { |r2| labels.index(r2[0]) }.map { |v2| v3 = v2[0]; [(@boom[@columns[0]] || {})[v3.to_s] || v3, v2[2]] }} }, chart_options %>
     <% elsif chart_type == "scatter" %>
-      <%= scatter_chart @rows, xtitle: @columns[0], ytitle: @columns[1], id: chart_id %>
+      <%= scatter_chart @rows, xtitle: @columns[0], ytitle: @columns[1], **chart_options %>
     <% elsif @only_chart %>
       <% if @rows.size == 1 && @rows.first.size == 1 %>
         <% v = @rows.first.first %>

data/app/views/blazer/queries/schema.html.erb

@@ -0,0 +1,58 @@
+<% blazer_title "Schema: #{@data_source.name}" %>
+
+<h1>Schema: <%= @data_source.name %></h1>
+
+<hr />
+
+<div id="header">
+  <input id="search" type="text" placeholder="Start typing a table or column" style="width: 300px; display: inline-block;" class="search form-control" />
+</div>
+
+<% @schema.each do |table| %>
+  <table class="table schema-table">
+    <thead>
+      <tr>
+        <th colspan="2">
+          <%= table[:table] %>
+          <% if table[:schema] != "public" %>
+            <span class="text-muted" style="font-weight: normal;"><%= table[:schema] %></span>
+          <% end %>
+        </th>
+      </tr>
+    </thead>
+    <tbody>
+      <% table[:columns].each do |column| %>
+        <tr>
+          <td style="width: 60%;"><%= column[:name] %></td>
+          <td class="text-muted"><%= column[:data_type] %></td>
+        </tr>
+      <% end %>
+    </tbody>
+  </table>
+<% end %>
+
+<script>
+  $("#search").on("keyup", function() {
+    var value = $(this).val().toLowerCase()
+    $(".schema-table").filter(function() {
+      // if found in table name, show entire table
+      // if just found in rows, show row
+
+      var found = $(this).find("thead").text().toLowerCase().indexOf(value) > -1
+
+      if (found) {
+        $(this).find("tbody tr").toggle(true)
+      } else {
+        $(this).find("tbody tr").filter(function() {
+          var found2 = $(this).text().toLowerCase().indexOf(value) > -1
+          $(this).toggle(found2)
+          if (found2) {
+            found = true
+          }
+        })
+      }
+
+      $(this).toggle(found)
+    })
+  }).focus()
+</script>

data/app/views/blazer/queries/show.html.erb

@@ -5,7 +5,7 @@
     <div class="row" style="padding-top: 13px;">
       <div class="col-sm-9">
         <%= render partial: "blazer/nav" %>
-        <h3 style="margin: 0; line-height: 34px; display: inline;">
+        <h3 style="line-height: 34px; display: inline; margin-left: 5px;">
           <%= @query.name %>
         </h3>
       </div>
@@ -14,7 +14,7 @@
         <%= link_to "Fork", new_query_path(variable_params.merge(fork_query_id: @query.id, data_source: @query.data_source, name: @query.name)), class: "btn btn-info" %>
 
         <% if !@error && @success %>
-          <%= button_to "Download", run_queries_path(query_id: @query.id, format: "csv"), params: {statement: @statement}, class: "btn btn-primary" %>
+          <%= button_to "Download", run_queries_path(query_id: @query.id, format: "csv", forecast: params[:forecast]), params: {statement: @statement}, class: "btn btn-primary" %>
         <% end %>
       </div>
     </div>
@@ -56,7 +56,7 @@
       $("#results").addClass("query-error").html(message)
     }
 
-    <%= blazer_js_var "data", variable_params.merge(statement: @statement, query_id: @query.id) %>
+    <%= blazer_js_var "data", variable_params.merge(statement: @statement, query_id: @query.id, data_source: @query.data_source) %>
 
     runQuery(data, showRun, showError)
   </script>

data/app/views/layouts/blazer/application.html.erb

@@ -4,7 +4,7 @@
     <title><%= blazer_title ? blazer_title : "Blazer" %></title>
 
     <meta charset="utf-8" />
-
+    <%= favicon_link_tag "blazer/favicon.png" %>
     <%= stylesheet_link_tag "blazer/application" %>
     <%= javascript_include_tag "blazer/application" %>
     <script>

data/config/routes.rb

@@ -4,13 +4,17 @@ Blazer::Engine.routes.draw do
     post :cancel, on: :collection
     post :refresh, on: :member
     get :tables, on: :collection
+    get :schema, on: :collection
     get :docs, on: :collection
   end
+
   resources :checks, except: [:show] do
     get :run, on: :member
   end
-  resources :dashboards do
+
+  resources :dashboards, except: [:index] do
     post :refresh, on: :member
   end
+
   root to: "queries#home"
 end

data/lib/blazer.rb

@@ -1,11 +1,16 @@
+# dependencies
 require "csv"
 require "yaml"
 require "chartkick"
 require "safely/core"
+
+# modules
 require "blazer/version"
 require "blazer/data_source"
 require "blazer/result"
 require "blazer/run_statement"
+
+# adapters
 require "blazer/adapters/base_adapter"
 require "blazer/adapters/athena_adapter"
 require "blazer/adapters/bigquery_adapter"
@@ -17,6 +22,8 @@ require "blazer/adapters/mongodb_adapter"
 require "blazer/adapters/presto_adapter"
 require "blazer/adapters/sql_adapter"
 require "blazer/adapters/snowflake_adapter"
+
+# engine
 require "blazer/engine"
 
 module Blazer
@@ -35,6 +42,7 @@ module Blazer
     attr_accessor :transform_statement
     attr_accessor :check_schedules
     attr_accessor :anomaly_checks
+    attr_accessor :forecasting
     attr_accessor :async
     attr_accessor :images
     attr_accessor :query_viewable
@@ -46,6 +54,7 @@ module Blazer
   self.user_name = :name
   self.check_schedules = ["5 minutes", "1 hour", "1 day"]
   self.anomaly_checks = false
+  self.forecasting = false
   self.async = false
   self.images = false
   self.override_csp = false

data/lib/blazer/engine.rb

@@ -6,6 +6,7 @@ module Blazer
       # use a proc instead of a string
       app.config.assets.precompile << proc { |path| path =~ /\Ablazer\/application\.(js|css)\z/ }
       app.config.assets.precompile << proc { |path| path =~ /\Ablazer\/.+\.(eot|svg|ttf|woff)\z/ }
+      app.config.assets.precompile << proc { |path| path == "blazer/favicon.png" }
 
       Blazer.time_zone ||= Blazer.settings["time_zone"] || Time.zone
       Blazer.audit = Blazer.settings.key?("audit") ? Blazer.settings["audit"] : true
@@ -16,6 +17,7 @@ module Blazer
       Blazer.cache ||= Rails.cache
 
       Blazer.anomaly_checks = Blazer.settings["anomaly_checks"] || false
+      Blazer.forecasting = Blazer.settings["forecasting"] || false
       Blazer.async = Blazer.settings["async"] || false
       if Blazer.async
         require "blazer/run_statement_job"

data/lib/blazer/result.rb

@@ -1,6 +1,6 @@
 module Blazer
   class Result
-    attr_reader :data_source, :columns, :rows, :error, :cached_at, :just_cached
+    attr_reader :data_source, :columns, :rows, :error, :cached_at, :just_cached, :forecast_error
 
     def initialize(data_source, columns, rows, error, cached_at, just_cached)
       @data_source = data_source
@@ -69,6 +69,8 @@ module Blazer
           "line"
         elsif column_types == ["time", "string", "numeric"]
           "line2"
+        elsif column_types == ["string", "numeric"] && @columns.last == "pie"
+          "pie"
         elsif column_types.compact.size >= 2 && column_types == ["string"] + (column_types.compact.size - 1).times.map { "numeric" }
           "bar"
         elsif column_types == ["string", "string", "numeric"]
@@ -79,6 +81,32 @@ module Blazer
       end
     end
 
+    def forecastable?
+      @forecastable ||= Blazer.forecasting && column_types == ["time", "numeric"] && @rows.size >= 10
+    end
+
+    def forecast
+      # TODO cache it?
+      # don't want to put result data (even hashed version)
+      # into cache without developer opt-in
+      forecast = Trend.forecast(Hash[@rows], count: 30)
+      @rows.each do |row|
+        row[2] = nil
+      end
+      @rows.unshift(*forecast.map { |k, v| [k, nil, v] })
+      @columns << "forecast"
+
+      # reset cache
+      @column_types = nil
+      @chart_type = nil
+
+      forecast
+    rescue => e
+      @forecast_error = String.new("Error generating forecast")
+      @forecast_error << ": #{e.message.sub("Invalid parameter: ", "")}"
+      nil
+    end
+
     def detect_anomaly
       anomaly = nil
       message = nil
@@ -131,39 +159,44 @@ module Blazer
     def anomaly?(series)
       series = series.reject { |v| v[0].nil? }.sort_by { |v| v[0] }
 
-      csv_str =
-        CSV.generate do |csv|
-          csv << ["timestamp", "count"]
-          series.each do |row|
-            csv << row
+      if Blazer.anomaly_checks == "trend"
+        anomalies = Trend.anomalies(Hash[series])
+        anomalies.include?(series.last[0])
+      else
+        csv_str =
+          CSV.generate do |csv|
+            csv << ["timestamp", "count"]
+            series.each do |row|
+              csv << row
+            end
           end
-        end
 
-      r_script = %x[which Rscript].chomp
-      type = series.any? && series.last.first.to_time - series.first.first.to_time >= 2.weeks ? "ts" : "vec"
-      args = [type, csv_str]
-      raise "R not found" if r_script.empty?
-      command = "#{r_script} --vanilla #{File.expand_path("../detect_anomalies.R", __FILE__)} #{args.map { |a| Shellwords.escape(a) }.join(" ")}"
-      output = %x[#{command}]
-      if output.empty?
-        raise "Unknown R error"
-      end
+        r_script = %x[which Rscript].chomp
+        type = series.any? && series.last.first.to_time - series.first.first.to_time >= 2.weeks ? "ts" : "vec"
+        args = [type, csv_str]
+        raise "R not found" if r_script.empty?
+        command = "#{r_script} --vanilla #{File.expand_path("../detect_anomalies.R", __FILE__)} #{args.map { |a| Shellwords.escape(a) }.join(" ")}"
+        output = %x[#{command}]
+        if output.empty?
+          raise "Unknown R error"
+        end
 
-      rows = CSV.parse(output, headers: true)
-      error = rows.first && rows.first["x"]
-      raise error if error
+        rows = CSV.parse(output, headers: true)
+        error = rows.first && rows.first["x"]
+        raise error if error
 
-      timestamps = []
-      if type == "ts"
-        rows.each do |row|
-          timestamps << Time.parse(row["timestamp"])
-        end
-        timestamps.include?(series.last[0].to_time)
-      else
-        rows.each do |row|
-          timestamps << row["index"].to_i
+        timestamps = []
+        if type == "ts"
+          rows.each do |row|
+            timestamps << Time.parse(row["timestamp"])
+          end
+          timestamps.include?(series.last[0].to_time)
+        else
+          rows.each do |row|
+            timestamps << row["index"].to_i
+          end
+          timestamps.include?(series.length)
         end
-        timestamps.include?(series.length)
       end
     end
   end

data/lib/blazer/version.rb

@@ -1,3 +1,3 @@
 module Blazer
-  VERSION = "2.0.0"
+  VERSION = "2.0.1"
 end

data/lib/generators/blazer/templates/config.yml.tt

@@ -60,3 +60,11 @@ check_schedules:
   - "1 day"
   - "1 hour"
   - "5 minutes"
+
+# enable anomaly detection
+# note: with trend, time series are sent to https://trendapi.org
+# anomaly_checks: trend / r
+
+# enable forecasting
+# note: with trend, time series are sent to https://trendapi.org
+# forecasting: trend

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: blazer
 version: !ruby/object:Gem::Version
-  version: 2.0.0
+  version: 2.0.1
 platform: ruby
 authors:
 - Andrew Kane
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-01-03 00:00:00.000000000 Z
+date: 2019-01-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: railties
@@ -109,6 +109,7 @@ files:
 - app/assets/fonts/blazer/glyphicons-halflings-regular.ttf
 - app/assets/fonts/blazer/glyphicons-halflings-regular.woff
 - app/assets/fonts/blazer/glyphicons-halflings-regular.woff2
+- app/assets/images/blazer/favicon.png
 - app/assets/javascripts/blazer/Chart.js
 - app/assets/javascripts/blazer/Sortable.js
 - app/assets/javascripts/blazer/ace.js
@@ -171,6 +172,7 @@ files:
 - app/views/blazer/queries/home.html.erb
 - app/views/blazer/queries/new.html.erb
 - app/views/blazer/queries/run.html.erb
+- app/views/blazer/queries/schema.html.erb
 - app/views/blazer/queries/show.html.erb
 - app/views/layouts/blazer/application.html.erb
 - config/routes.rb