blazer 2.4.2 → 2.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8c8356dddcf7221e7bbf93eb4cc4d1e1b7804f08c29dd57006517d0e157f1ff4
-  data.tar.gz: 7ac1354b00daa9905901a6c11c6e8a948962e2c6830feeaa19d065bfa1bde9a6
+  metadata.gz: 8bdfc1e428f7e01bf9a06461a5f0143a6e940cd1e3d708ba2bd504132bbaa1db
+  data.tar.gz: 729e9a408e7f4fa5203ab4c133800e49087fccf634efcf7c9dd6ca80a610a861
 SHA512:
-  metadata.gz: 5959df81b7c5dbe0b608bfe565f1d98d2a7119a6309bbf934d86d1154227cd6315051ec131c9e8a8d9641022468c42724152620b4a226ee8300791d1095df858
-  data.tar.gz: 694140d5e5fe1904f16ccbcd95f9e18b8fcd54e7630d0df05637ddc977ab223220cb749798c8d365d77f962b131a91135bcfa9c12dc4d849f3d4caea1a642f79
+  metadata.gz: e6c7a7be80246c1030170a5df95656947b2431cf908bb6d37d668cbe3f57006ef096d63186976c63c683f0e2511873cabb5d2be56e389de331487666ba297dc3
+  data.tar.gz: 9feb70216244f77d37357376cd68b9ec85a6d3eeb3c89f3e58196688e81618ca2ae38372e5491e0811bdd7d9e0082317a711040f5782cc54f5d9f99ec57d624a

data/CHANGELOG.md

@@ -1,3 +1,49 @@
+## 2.6.0 (2022-04-20)
+
+- Fixed quoting issue with variables
+- Custom adapters now need to specify how to quote variables in queries
+- Added experimental support for Propshaft
+- Fixed error with empty results with InfluxDB
+
+## 2.5.0 (2022-01-04)
+
+- Added support for Slack OAuth tokens
+- Added experimental support for AnomalyDetection.rb
+- Improved table preview for MySQL
+- Fixed cohort analysis for MySQL
+
+## 2.4.8 (2021-12-07)
+
+- Added support for OpenSearch
+- Removed `elasticsearch-xpack` dependency for Elasticsearch
+
+## 2.4.7 (2021-09-25)
+
+- Made Action Mailer optional
+- Fixed error with multiple maps on dashboard
+
+## 2.4.6 (2021-09-20)
+
+- Added support for workgroup with Amazon Athena
+- Added casting for timestamp with time zone columns with Amazon Athena
+- Added support for setting credentials in config file with Amazon Athena
+- Made output location optional with Amazon Athena
+- Fixed casting error for `NULL` values with Amazon Athena
+- Fixed issue with Google BigQuery only showing first page of results
+
+## 2.4.5 (2021-09-15)
+
+- Improved fix for some forked queries not appearing on home page
+
+## 2.4.4 (2021-09-15)
+
+- Fixed issue with some forked queries not appearing on home page
+
+## 2.4.3 (2021-07-27)
+
+- Added Prophet anomaly detection
+- Fixed style for new select items
+
 ## 2.4.2 (2021-02-08)
 
 - Added support for Apache Ignite

data/README.md

@@ -10,6 +10,8 @@ Blazer is also available as a [Docker image](https://github.com/ankane/blazer-do
 
 :tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
 
+[![Build Status](https://github.com/ankane/blazer/workflows/build/badge.svg?branch=master)](https://github.com/ankane/blazer/actions)
+
 ## Features
 
 - **Multiple data sources** - PostgreSQL, MySQL, Redshift, and [many more](#full-list)
@@ -37,7 +39,7 @@ Blazer is also available as a [Docker image](https://github.com/ankane/blazer-do
 Add this line to your application’s Gemfile:
 
 ```ruby
-gem 'blazer'
+gem "blazer"
 ```
 
 Run:
@@ -59,7 +61,7 @@ For production, specify your database:
 ENV["BLAZER_DATABASE_URL"] = "postgres://user:password@hostname:5432/database"
 ```
 
-Blazer tries to protect against queries which modify data (by running each query in a transaction and rolling it back), but a safer approach is to use a read-only user. [See how to create one](#permissions).
+When possible, Blazer tries to protect against queries which modify data by running each query in a transaction and rolling it back, but a safer approach is to use a read-only user. [See how to create one](#permissions).
 
 #### Checks (optional)
 
@@ -142,11 +144,9 @@ Be sure to render or redirect for unauthorized users.
 
 ## Permissions
 
-Blazer runs each query in a transaction and rolls it back to prevent queries from modifying data. As an additional line of defense, we recommend using a read only user.
-
 ### PostgreSQL
 
-Create a user with read only permissions:
+Create a user with read-only permissions:
 
 ```sql
 BEGIN;
@@ -160,7 +160,7 @@ COMMIT;
 
 ### MySQL
 
-Create a user with read only permissions:
+Create a user with read-only permissions:
 
 ```sql
 GRANT SELECT, SHOW VIEW ON database_name.* TO blazer@’127.0.0.1′ IDENTIFIED BY ‘secret123‘;
@@ -169,7 +169,7 @@ FLUSH PRIVILEGES;
 
 ### MongoDB
 
-Create a user with read only permissions:
+Create a user with read-only permissions:
 
 ```
 db.createUser({user: "blazer", pwd: "password", roles: ["read"]})
@@ -412,74 +412,62 @@ SELECT users.id AS user_id, orders.created_at AS conversion_time, users.created_
 FROM users LEFT JOIN orders ON orders.user_id = users.id
 ```
 
-This feature requires PostgreSQL or MySQL.
+This feature requires PostgreSQL or MySQL 8.
 
 ## Anomaly Detection
 
-Blazer supports two different approaches to anomaly detection.
+Blazer supports three different approaches to anomaly detection.
 
-### Trend
+### Prophet
 
-[Trend](https://trendapi.org/) is easiest to set up. By default, it uses an external service, but you can run it on your own infrastructure as well.
-
-Add [trend](https://github.com/ankane/trend) to your Gemfile:
+Add [prophet-rb](https://github.com/ankane/prophet) to your Gemfile:
 
 ```ruby
-gem 'trend'
+gem "prophet-rb"
 ```
 
 And add to `config/blazer.yml`:
 
 ```yml
-anomaly_checks: trend
+anomaly_checks: prophet
 ```
 
-For the [self-hosted API](https://github.com/ankane/trend-api), create an initializer with:
-
-```ruby
-Trend.url = "http://localhost:8000"
-```
-
-### R
+### Trend
 
-R is harder to set up but doesn’t use an external service. It uses Twitter’s [AnomalyDetection](https://github.com/twitter/AnomalyDetection) library.
+[Trend](https://trendapi.org/) uses an external service by default, but you can run it on your own infrastructure as well.
 
-First, [install R](https://cloud.r-project.org/). Then, run:
+Add [trend](https://github.com/ankane/trend) to your Gemfile:
 
-```R
-install.packages("remotes")
-remotes::install_github("twitter/AnomalyDetection")
+```ruby
+gem "trend"
 ```
 
 And add to `config/blazer.yml`:
 
 ```yml
-anomaly_checks: r
+anomaly_checks: trend
 ```
 
-If upgrading from version 1.4 or below, also follow the [upgrade instructions](#15).
+For the [self-hosted API](https://github.com/ankane/trend-api), create an initializer with:
 
-If you’re on Heroku, follow the additional instructions below.
+```ruby
+Trend.url = "http://localhost:8000"
+```
 
-### R on Heroku
+### AnomalyDetection.rb (experimental)
 
-Add the [R buildpack](https://github.com/virtualstaticvoid/heroku-buildpack-r) to your app.
+Add [anomaly_detection](https://github.com/ankane/AnomalyDetection.rb) to your Gemfile:
 
-```sh
-heroku buildpacks:add --index 1 https://github.com/virtualstaticvoid/heroku-buildpack-r.git
+```ruby
+gem "anomaly_detection"
 ```
 
-And create an `init.R` with:
+And add to `config/blazer.yml`:
 
-```r
-if (!"AnomalyDetection" %in% installed.packages()) {
-  install.packages("remotes")
-  remotes::install_github("twitter/AnomalyDetection")
-}
+```yml
+anomaly_checks: anomaly_detection
 ```
 
-Commit and deploy away. The first deploy may take a few minutes.
-
 ## Forecasting
 
 Blazer supports for two different forecasting methods. [Example](https://blazer.dokkuapp.com/queries/18-forecast?forecast=t)
@@ -488,10 +476,10 @@ A forecast link will appear for queries that return 2 columns with types timesta
 
 ### Prophet
 
-Add [prophet](https://github.com/ankane/prophet) to your Gemfile:
+Add [prophet-rb](https://github.com/ankane/prophet) to your Gemfile:
 
 ```ruby
-gem 'prophet-rb', '>= 0.2.1'
+gem "prophet-rb", ">= 0.2.1"
 ```
 
 And add to `config/blazer.yml`:
@@ -502,12 +490,12 @@ forecasting: prophet
 
 ### Trend
 
-[Trend](https://trendapi.org/) uses an external service.
+[Trend](https://trendapi.org/) uses an external service by default, but you can run it on your own infrastructure as well.
 
 Add [trend](https://github.com/ankane/trend) to your Gemfile:
 
 ```ruby
-gem 'trend'
+gem "trend"
 ```
 
 And add to `config/blazer.yml`:
@@ -516,6 +504,12 @@ And add to `config/blazer.yml`:
 forecasting: trend
 ```
 
+For the [self-hosted API](https://github.com/ankane/trend-api), create an initializer with:
+
+```ruby
+Trend.url = "http://localhost:8000"
+```
+
 ## Uploads
 
 Creating database tables from CSV files. [Example](https://blazer.dokkuapp.com/uploads)
@@ -567,7 +561,7 @@ data_sources:
 - [Amazon Redshift](#amazon-redshift)
 - [Apache Drill](#apache-drill)
 - [Apache Hive](#apache-hive)
-- [Apache Ignite](#apache-ignite) [master]
+- [Apache Ignite](#apache-ignite)
 - [Apache Spark](#apache-spark)
 - [Cassandra](#cassandra)
 - [Druid](#druid)
@@ -578,6 +572,7 @@ data_sources:
 - [MongoDB](#mongodb-1)
 - [MySQL](#mysql-1)
 - [Neo4j](#neo4j)
+- [OpenSearch](#opensearch)
 - [Oracle](#oracle)
 - [PostgreSQL](#postgresql-1)
 - [Presto](#presto)
@@ -606,9 +601,49 @@ data_sources:
   my_source:
     adapter: athena
     database: database
+
+    # optional settings
     output_location: s3://some-bucket/
+    workgroup: primary
+    access_key_id: ...
+    secret_access_key: ...
+```
+
+Here’s an example IAM policy:
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Action": [
+                "athena:GetQueryExecution",
+                "athena:GetQueryResults",
+                "athena:StartQueryExecution"
+            ],
+            "Resource": [
+                "arn:aws:athena:region:account-id:workgroup/primary"
+            ]
+        },
+        {
+            "Effect": "Allow",
+            "Action": [
+                "glue:GetTable",
+                "glue:GetTables"
+            ],
+            "Resource": [
+                "arn:aws:glue:region:account-id:catalog",
+                "arn:aws:glue:region:account-id:database/default",
+                "arn:aws:glue:region:account-id:table/default/*"
+            ]
+        }
+    ]
+}
 ```
 
+You also need to configure [S3 permissions](https://aws.amazon.com/premiumsupport/knowledge-center/access-denied-athena/).
+
 ### Amazon Redshift
 
 Add [activerecord6-redshift-adapter](https://github.com/kwent/activerecord6-redshift-adapter) or [activerecord5-redshift-adapter](https://github.com/ConsultingMD/activerecord5-redshift-adapter) to your Gemfile and set:
@@ -619,6 +654,8 @@ data_sources:
     url: redshift://user:password@hostname:5439/database
 ```
 
+Use a [read-only user](https://docs.aws.amazon.com/redshift/latest/dg/r_GRANT.html).
+
 ### Apache Drill
 
 Add [drill-sergeant](https://github.com/ankane/drill-sergeant) to your Gemfile and set:
@@ -630,6 +667,8 @@ data_sources:
     url: http://hostname:8047
 ```
 
+Use a [read-only user](https://drill.apache.org/docs/roles-and-privileges/).
+
 ### Apache Hive
 
 Add [hexspace](https://github.com/ankane/hexspace) to your Gemfile and set:
@@ -653,6 +692,8 @@ data_sources:
     url: ignite://user:password@hostname:10800
 ```
 
+Use a [read-only user](https://www.gridgain.com/docs/latest/administrators-guide/security/authorization-permissions) (requires a third-party plugin).
+
 ### Apache Spark
 
 Add [hexspace](https://github.com/ankane/hexspace) to your Gemfile and set:
@@ -676,6 +717,8 @@ data_sources:
     url: cassandra://user:password@hostname:9042/keyspace
 ```
 
+Use a [read-only role](https://docs.datastax.com/en/cql-oss/3.3/cql/cql_using/useSecurePermission.html).
+
 ### Druid
 
 Enable [SQL support](http://druid.io/docs/latest/querying/sql.html#configuration) on the broker and set:
@@ -687,9 +730,11 @@ data_sources:
     url: http://hostname:8082
 ```
 
+Use a [read-only role](https://druid.apache.org/docs/latest/development/extensions-core/druid-basic-security.html).
+
 ### Elasticsearch
 
-Add [elasticsearch](https://github.com/elastic/elasticsearch-ruby) and [elasticsearch-xpack](https://github.com/elastic/elasticsearch-ruby/tree/master/elasticsearch-xpack) to your Gemfile and set:
+Add [elasticsearch](https://github.com/elastic/elasticsearch-ruby) to your Gemfile and set:
 
 ```yml
 data_sources:
@@ -698,6 +743,8 @@ data_sources:
     url: http://user:password@hostname:9200
 ```
 
+Use a [read-only role](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-privileges.html).
+
 ### Google BigQuery
 
 Add [google-cloud-bigquery](https://github.com/GoogleCloudPlatform/google-cloud-ruby/tree/master/google-cloud-bigquery) to your Gemfile and set:
@@ -720,6 +767,8 @@ data_sources:
     url: ibm-db://user:password@hostname:50000/database
 ```
 
+Use a [read-only user](https://www.ibm.com/support/pages/creating-read-only-database-permissions-user).
+
 ### InfluxDB
 
 Add [influxdb](https://github.com/influxdata/influxdb-ruby) to your Gemfile and set:
@@ -731,10 +780,12 @@ data_sources:
     url: http://user:password@hostname:8086/database
 ```
 
-Supports [InfluxQL](https://docs.influxdata.com/influxdb/v1.8/query_language/explore-data/)
+Use a [read-only user](https://docs.influxdata.com/influxdb/v1.8/administration/authentication_and_authorization/). Supports [InfluxQL](https://docs.influxdata.com/influxdb/v1.8/query_language/explore-data/).
 
 ### MongoDB
 
+*Requires MongoDB < 4.2 at the moment*
+
 Add [mongo](https://github.com/mongodb/mongo-ruby-driver) to your Gemfile and set:
 
 ```yml
@@ -743,6 +794,8 @@ data_sources:
     url: mongodb://user:password@hostname:27017/database
 ```
 
+Use a [read-only user](#mongodb).
+
 ### MySQL
 
 Add [mysql2](https://github.com/brianmario/mysql2) to your Gemfile (if it’s not there) and set:
@@ -753,6 +806,8 @@ data_sources:
     url: mysql2://user:password@hostname:3306/database
 ```
 
+Use a [read-only user](#mysql).
+
 ### Neo4j
 
 Add [neo4j-core](https://github.com/neo4jrb/neo4j-core) to your Gemfile and set:
@@ -764,6 +819,21 @@ data_sources:
     url: http://user:password@hostname:7474
 ```
 
+Use a [read-only user](https://neo4j.com/docs/cypher-manual/current/access-control/manage-privileges/).
+
+### OpenSearch
+
+Add [opensearch-ruby](https://github.com/opensearch-project/opensearch-ruby) to your Gemfile and set:
+
+```yml
+data_sources:
+  my_source:
+    adapter: opensearch
+    url: http://user:password@hostname:9200
+```
+
+Use a [read-only user](https://opensearch.org/docs/latest/security-plugin/access-control/permissions/).
+
 ### Oracle
 
 Add [activerecord-oracle_enhanced-adapter](https://github.com/rsim/oracle-enhanced) and [ruby-oci8](https://github.com/kubo/ruby-oci8) to your Gemfile and set:
@@ -774,6 +844,8 @@ data_sources:
     url: oracle-enhanced://user:password@hostname:1521/database
 ```
 
+Use a [read-only user](https://docs.oracle.com/cd/B19306_01/network.102/b14266/authoriz.htm).
+
 ### PostgreSQL
 
 Add [pg](https://github.com/ged/ruby-pg) to your Gemfile (if it’s not there) and set:
@@ -784,6 +856,8 @@ data_sources:
     url: postgres://user:password@hostname:5432/database
 ```
 
+Use a [read-only user](#postgresql).
+
 ### Presto
 
 Add [presto-client](https://github.com/treasure-data/presto-client-ruby) to your Gemfile and set:
@@ -794,6 +868,8 @@ data_sources:
     url: presto://user@hostname:8080/catalog
 ```
 
+Use a [read-only user](https://prestodb.io/docs/current/security/built-in-system-access-control.html).
+
 ### Salesforce
 
 Add [restforce](https://github.com/restforce/restforce) to your Gemfile and set:
@@ -815,7 +891,7 @@ SALESFORCE_CLIENT_SECRET="client secret"
 SALESFORCE_API_VERSION="41.0"
 ```
 
-Supports [SOQL](https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql.htm)
+Use a read-only user. Supports [SOQL](https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql.htm).
 
 ### Socrata Open Data API (SODA)
 
@@ -829,7 +905,7 @@ data_sources:
     app_token: ...
 ```
 
-Supports [SoQL](https://dev.socrata.com/docs/functions/)
+Supports [SoQL](https://dev.socrata.com/docs/functions/).
 
 ### Snowflake
 
@@ -863,6 +939,8 @@ data_sources:
     conn_str: Driver=/path/to/libSnowflake.so;uid=user;pwd=password;server=host.snowflakecomputing.com
 ```
 
+Use a [read-only role](https://docs.snowflake.com/en/user-guide/security-access-control-configure.html).
+
 ### SQLite
 
 Add [sqlite3](https://github.com/sparklemotion/sqlite3-ruby) to your Gemfile and set:
@@ -883,6 +961,8 @@ data_sources:
     url: sqlserver://user:password@hostname:1433/database
 ```
 
+Use a [read-only user](https://docs.microsoft.com/en-us/sql/relational-databases/security/authentication-access/getting-started-with-database-engine-permissions?view=sql-server-ver15).
+
 ## Creating an Adapter
 
 Create an adapter for any data store with:
@@ -959,6 +1039,22 @@ override_csp: true
 
 ## Upgrading
 
+### 2.6
+
+Custom adapters now need to specify how to quote variables in queries (there is no longer a default)
+
+```ruby
+class FooAdapter < Blazer::Adapters::BaseAdapter
+  def quoting
+    :backslash_escape # single quote strings and convert ' to \' and \ to \\
+    # or
+    :single_quote_escape # single quote strings and convert ' to ''
+    # or
+    ->(value) { ... } # custom method
+  end
+end
+```
+
 ### 2.3
 
 To archive queries, create a migration