active_enquo 0.2.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fc6de3e861f5cd9e32b59bcc95042fd67fe0302433ca390aca51dcc7d2073b53
-  data.tar.gz: 79735074ab96ccaa1dc31384d313b075ced36630b8253276f1a393ce433ae090
+  metadata.gz: e643c69efa8243d95e44c59a6d49f20c40a23dc3a914da1d79342fa5d3969ca9
+  data.tar.gz: fcb70caad8b2d949fca672198e269e574924cd63748dfa221164ec6a1eb7842d
 SHA512:
-  metadata.gz: ec344ac14ee119cb38f1718f22b4f45d34fbccf20917196c47c3cdcd90ead5c050cab422899bdceca5a4fb6c8cd270e058815374b9504698bc1dbb67383ec101
-  data.tar.gz: 6e127eb2ba8a7a027191fe83827a0af663f82f760e3364ffc025b650a3d58ab0ba1f1897f9bd667373f7ad9e103a835941437473e5c01c51ac9f387707fa7676
+  metadata.gz: 7ce8dd63cae62013d026ddf7200878975471bf30c157e059e50b269124e886a219ebb526581038bed3fed1de00500416bfdee7740497c1af2b775c90c46dfd53
+  data.tar.gz: 02a1944917d45d1517cd5143dbe56c05c28f8c8a2fffe5840f317ce865a944aa563a04560f3e39773eadb7f2ca5a0f54835bdab2f4765e5aacc456c9a88f88b8

data/README.md

@@ -3,12 +3,12 @@ This allows you to keep the data you store safe, by encrypting it, without compr
 
 Sounds like magic?
 Well, maybe a little bit.
-Read our [how it works](https://enquo.org/how-it-works) if you're interested in the details, or read on for how to use it.
+Read our [how it works](https://enquo.org/how-it-works) if you're interested in the gory cryptographic details, or read on for how to use it.
 
 
 # Pre-requisites
 
-In order to make use of this extension, you must be running Postgres 10 or higher, with the [`pg_enquo`](https://github.com/enquo/pg_enquo) extension enabled in the database you're working in.
+In order to make use of ActiveRecord extension, you must be running Postgres 11 or higher, with the [`pg_enquo`](https://github.com/enquo/pg_enquo) extension enabled in the database you're working in.
 See [the `pg_enquo` installation guide](https://github.com/enquo/pg_enquo/tree/main/doc/installation.md) for instructions on how to install `pg_enquo`.
 
 Also, if you're installing this gem from source, you'll need a reasonably recent [Rust](https://rust-lang.org) toolchain installed.
@@ -16,33 +16,41 @@ Also, if you're installing this gem from source, you'll need a reasonably recent
 
 # Installation
 
-It's a gem:
+It's a gem, so the usual methods should work Just Fine:
 
-    gem install active_enquo
-
-There's also the wonders of [the Gemfile](http://bundler.io):
+```sh
+gem install active_enquo
+# OR
+echo "gem 'active_enquo'" >> Gemfile
+```
 
-    gem 'active_enquo'
+On macOS, and Linux `x86-64`/`aarch64`, you'll get a pre-built binary gem that contains everything you need.
+For other platforms, you'll need to have Rust 1.59.0 or later installed in order to build the native code portion of the gem.
 
-If you're the sturdy type that likes to run from git:
 
-    rake install
+# Configuration
 
-Or, if you've eschewed the convenience of Rubygems entirely, then you
-presumably know what to do already.
+The only setting that ActiveEnquo needs is to be given a "root" key, which is used to derive the keys which are used to actually encrypt data.
 
 
-# Configuration
+## Step 1: Generate a Root Key
 
-The only setting that ActiveEnquo needs is to be given a "root" key, which is used to derive the keys which are used to actually encrypt data.
-This key ***MUST*** be generated by a cryptographically-secure random number generator, and must also be 64 hex digits in length.
+The ActiveEnquo root key ***MUST*** be generated by a cryptographically-secure random number generator, and must also be 64 hex digits in length.
 A good way to generate this key is with the `SecureRandom` module:
 
 ```sh
 ruby -r securerandom -e 'puts SecureRandom.hex(32)'
 ```
 
-With this key in hand, you can store it in the Rails credential store, like this:
+
+## Step 2: Configure Your Application
+
+With this key in hand, you need to store it somewhere.
+
+
+### Using Rails Credential Store (Recommended)
+
+The recommended way to store your root key, at present, is in the [Rails credentials store](https://guides.rubyonrails.org/security.html#custom-credentials).
 
 1. Open up the Rails credentials editor:
 
@@ -59,7 +67,11 @@ With this key in hand, you can store it in the Rails credential store, like this
 
 3. Save and exit the editor.  Commit the changes to your revision control system.
 
-This only works if you are using Rails, of course; if you're using ActiveRecord by itself, you must set the root key yourself during application initialization.
+
+### Direct Assignment (Only If You Must)
+
+Using the Rails credential store only works if you are using Rails, of course.
+If you're using ActiveRecord by itself, you must set the root key yourself during application initialization.
 You do this by assigning a `RootKey` to `ActiveEnquo.root_key`, like this:
 
 ```ruby
@@ -67,9 +79,12 @@ You do this by assigning a `RootKey` to `ActiveEnquo.root_key`, like this:
 ActiveEnquo.root_key = ActiveEnquo::RootKey::Static.new("0000000000000000000000000000000000000000000000000000000000000000")
 ```
 
-However, this leaves the key exposed to anyone who comes along and takes a glance at your code.
-Losing control of this root key is catastrophic for the security of your system.
-Instead, you should use a secrets vault of some sort to store the key, and pass it into your initialization code somehow.
+Preferably, you would pass the key into your application via, say, an environment variable, and then immediately clear the environment variable:
+
+```ruby
+ActiveEnquo.root_key = ActiveEnquo::RootKey::Static.new(ENV.fetch("ENQUO_ROOT_KEY"))
+ENV.delete("ENQUO_ROOT_KEY")
+```
 
 Support for cloud keystores, such as AWS KMS, GCP KMS, Azure KeyVault, and HashiCorp Vault, will be implemented sooner or later.
 If you have a burning desire to see that more on the "sooner" end than "later", PRs are welcome.
@@ -77,67 +92,72 @@ If you have a burning desire to see that more on the "sooner" end than "later",
 
 # Usage
 
-Start by creating a column in your database that uses one of the [available enquo types](https://github.com/enquo/pg_enquo/doc/data_types), with a Rails migration:
+We try to make using ActiveEnquo as simple as possible.
+
+
+## Create Your Encrypted Column
+
+Start by creating a column in your database that uses one of the [available `enquo_*` types](https://github.com/enquo/pg_enquo/tree/main/doc/data_types), with a Rails migration:
 
 ```ruby
 class AddEncryptedBigintColumn < ActiveRecord::Migration[6.0]
   def change
-    add_column :users, :age, :enquo_bigint
+    add_column :users, :date_of_birth, :enquo_date
   end
 end
 ```
 
+Apply this migration in the usual fashion (`rails db:migrate`).
+
 
 ## Reading and Writing
 
-You can now use that attribute in your models as you would normally.
-For example, insert a new record:
+You can now, without any further ado, use that attribute in your models as you would normally.
+For example, you can insert a new record:
 
 ```ruby
-User.create!([{name: "Clara Bloggs", username: "cbloggs", age: 42}])
+User.create!([{name: "Clara Bloggs", username: "cbloggs", date_of_birth: Date(1970, 1, 1)}])
 ```
 
 When you retrieve a record, the value is there for you to read:
 
 ```ruby
-User.where(username: "cbloggs").first.age  # => 42
+User.where(username: "cbloggs").first.date_of_birth.to_s  # => "1970-01-01"
 ```
 
-So far, nothing more spectacular than what AR Encryption will get you.
-The fun begins now...
-
 
 ## Querying
 
-You can query for records with an exact age:
+This is where things get *neat*.
+
+Performing a query on Enquo-encrypted data is done the same way as on unencrypted data, with one exception: you need to wrap the values of the query in `<Model>.enquo` calls, as in the examples below.
+
+You can query for records that have the exact value you're looking for:
 
 ```ruby
-User.where(age: User.enquo(:age, 42))
+User.where(age: User.enquo(:date_of_birth, Date(1970, 1, 1)))
 ```
 
-Or you can query for records with an age of less than 50:
+Or you can query for users born less than 50 years ago:
 
 ```ruby
-# This is AR's idiomatic way of saying "less-than"
-User.where(age: User.enquo(:age, ...50))
+User.where(age: User.enquo(:date, Date.today - 50.years)..)
 ```
 
-*However*, if you examine the record in the database, it's encrypted:
+This doesn't seem so magical, until you take a peek in the database, and realise that *all the data is still encrypted*:
 
 ```sh
-psql> SELECT age FROM users WHERE username='cbloggs';
+psql> SELECT date_of_birth FROM users WHERE username='cbloggs';
   age
 -------
- {"ct":[<lots of numbers>],"ore":[<lots and LOTS of numbers>]}
+ {"v1":{"a":[<lots of numbers>],"y":[<lots and LOTS of numbers>],<etc etc>}}
 ```
 
-And that, as they say, is that.
-
 
 ## Indexing and Ordering
 
-To maintain [security](https://enquo.org/about/threat-models#snapshot-security), ActiveEnquo isn't able to `ORDER BY` or index columns.
-This is fine for many situations -- many columns don't need indexes.
+To maintain [security by default](https://enquo.org/about/threat-models#snapshot-security), ActiveEnquo doesn't provide the ability to `ORDER BY` or index columns by default.
+This is fine for many situations -- many columns don't need indexes or to be ordered in a query.
 
 For those columns that *do* need indexes or `ORDER BY` support, you can enable support for them by setting the `enable_reduced_security_operations` flag on the attribute, like this:
 
@@ -148,10 +168,11 @@ class User < ApplicationRecord
 end
 ```
 
-### SECURITY ALERT
+### Security Considerations
 
 As the name implies, "reduced security operations" require that the security of the data in the column be lower than [Enquo's default security properties](https://enquo.org/about/threat-models#snapshot-security).
-In particular, extra data is stored in the value which can be used by an attacker to:
+Specifically, extra data needs to be stored in the value to enable indexing and ordering.
+This extra data can be used by an attacker to:
 
 * Identify all rows which have the same value for the column (although not what that value actually *is*); and
 
@@ -174,22 +195,14 @@ class User < ApplicationRecord
 end
 ```
 
+More accurate indications of the disk space requirements for the supported data types can be found in [the description of each data type](https://github.com/enquo/pg_enquo/tree/main/doc/data_types).
 
-# Future Developments
 
-These are some of the things that are definitely planned to be added to ActiveEnquo in the nearish future.
-
-* **Support for key rotation**: this isn't tricky, just a bit fiddly.
-  Encrypted values have a "key ID" associated with them, so we can find which values are out-of-date, but multiple keys need to useable for decryption.
-  Closely related to this is **support for renaming columns**, which is problematic because keys are derived based on the column name.
-  Again, key IDs (to find values encrypted with the previous column name) and the ability to attempt decryption with a key based on the previous column name sorts this out.
-
-* **Strings**: a great deal of the sensitive data that needs protecting is in the form of strings.
-  Querying strings is somewhat more involved than numeric data, and so the means of encrypting strings such that they're queryable *and* secure are more complex.
-  Enquo cannot be a really useful library for supporting encrypted querying until at least some common string query operations are supported, though, so it is important that this be implemented.
+# Future Developments
 
-* **Other data types**: while you can go a long way with strings and bigints, part of the power of SQL is the sheer variety of interesting data types it has, and the variety of things you can do with them.
-  So more integer types, floats, decimals, dates, times, timestamps, and more, are all on the cards for future development.
+ActiveEnquo is far from finished.
+Many more features are coming in the future.
+See [the Enquo project roadmap](https://enquo.org/roadmap) for details of what we're still intending to implement.
 
 
 # Contributing

data/active_enquo.gemspec

@@ -27,7 +27,7 @@ Gem::Specification.new do |s|
 
 	s.required_ruby_version = ">= 2.7.0"
 
-	s.add_runtime_dependency "enquo-core", "~> 0.4"
+	s.add_runtime_dependency "enquo-core", "~> 0.6"
 	s.add_runtime_dependency "activerecord", ">= 6"
 
 	s.add_development_dependency "bundler"

data/lib/active_enquo.rb

@@ -29,6 +29,7 @@ module ActiveEnquo
 				if t.is_a?(::ActiveEnquo::Type)
 					relation = self.class.arel_table.name
 					value = @attributes.fetch_value(attr_name, &block)
+					return nil if value.nil?
 					field = ::ActiveEnquo.root.field(relation, attr_name)
 					begin
 						t.decrypt(value, @attributes.fetch_value(@primary_key).to_s, field)
@@ -81,6 +82,20 @@ module ActiveEnquo
 				end
 			end
 		end
+
+		module TableDefinitionExtension
+			def enquo_bigint(name, **options)
+				column(name, :enquo_bigint, **options)
+			end
+
+			def enquo_date(name, **options)
+				column(name, :enquo_date, **options)
+			end
+
+			def enquo_text(name, **options)
+				column(name, :enquo_text, **options)
+			end
+		end
 	end
 
 	module Postgres
@@ -88,6 +103,7 @@ module ActiveEnquo
 			def initialize_type_map(m = type_map)
 				m.register_type "enquo_bigint", ActiveEnquo::Type::Bigint.new
 				m.register_type "enquo_date", ActiveEnquo::Type::Date.new
+				m.register_type "enquo_text", ActiveEnquo::Type::Text.new
 
 				super
 			end
@@ -135,16 +151,46 @@ module ActiveEnquo
 				end
 			end
 		end
+
+		class Text < Type
+			def type
+				:enquo_text
+			end
+
+			def encrypt(value, context, field, safety: true, no_query: false)
+				field.encrypt_text(value, context, safety: safety, no_query: no_query)
+			end
+
+			def decrypt(value, context, field)
+				field.decrypt_text(value, context)
+			end
+		end
+	end
+
+	if defined?(Rails::Railtie)
+		class Initializer < Rails::Railtie
+			initializer "active_enquo.root_key" do |app|
+				if app
+					if root_key = app.credentials.active_enquo.root_key
+						ActiveEnquo.root_key = Enquo::RootKey::Static.new(root_key)
+					else
+						Rails.warn "Could not initialize ActiveEnquo, as no active_enquo.root_key credential was found for this environment"
+					end
+				end
+			end
+		end
 	end
 end
 
 ActiveSupport.on_load(:active_record) do
 	::ActiveRecord::Base.send :include, ActiveEnquo::ActiveRecord::ModelExtension
 
+	::ActiveRecord::ConnectionAdapters::Table.include ActiveEnquo::ActiveRecord::TableDefinitionExtension
+	::ActiveRecord::ConnectionAdapters::TableDefinition.include ActiveEnquo::ActiveRecord::TableDefinitionExtension
+
 	::ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.prepend ActiveEnquo::Postgres::ConnectionAdapter
-#	::ActiveRecord::Type.register(:enquo_bigint, ActiveEnquo::Type::Bigint, adapter: :postgresql)
 
-	unless ActiveRecord::VERSION::MAJOR == 7
-		::ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::NATIVE_DATABASE_TYPES[:enquo_bigint] = {}
-	end
+	::ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::NATIVE_DATABASE_TYPES[:enquo_bigint] = { name: "enquo_bigint" }
+	::ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::NATIVE_DATABASE_TYPES[:enquo_date]   = { name: "enquo_date" }
+	::ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::NATIVE_DATABASE_TYPES[:enquo_text]   = { name: "enquo_text" }
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: active_enquo
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.4.0
 platform: ruby
 authors:
 - Matt Palmer
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2022-10-04 00:00:00.000000000 Z
+date: 2022-11-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: enquo-core
@@ -16,14 +16,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.4'
+        version: '0.6'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.4'
+        version: '0.6'
 - !ruby/object:Gem::Dependency
   name: activerecord
   requirement: !ruby/object:Gem::Requirement