asherah 0.2.0-x86_64-linux → 0.4.2-x86_64-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1a3ea6ba3aca16701b48b1435b5dde4a014d0e3769cc66d6609af670b3d4b6cf
-  data.tar.gz: d2cf924ab75bc682deeea77addea8fad85585460748e702e66005dd0d2af18e7
+  metadata.gz: 62bfb024529e4e36f27690f83dc7ddcd0a1cde72cc719a883ca3c5490cae1041
+  data.tar.gz: 998521233edd5cbcab1c585702948d52f7697bcdecbbe986e670e52735bd008c
 SHA512:
-  metadata.gz: 23a1ebfb0229e3245111a9b9abd7298e194a61b5e5d5172afcede238e4f5bc8456f54b81f585bf665354fce05420220d45e0593678c984fbd12471c95b77b01e
-  data.tar.gz: d094f921ae9a0c2150ccff3285475449ead10a0609829449bc5336a9e968addb7ae9263e43a4f6899dbbd785a9a298ce8360165525759436c6b3dc2b16cf542b
+  metadata.gz: b0e72a6bdb8550d1bdebac4e9f369cb11a0c8974c39236e87eaf9b12158d42eb8c6e8bbc716e59331e603449ad5d43c3c6f57038df2846bfcfb1f79f4738bca9
+  data.tar.gz: 46cf04e73527f941ded98404bbc9a8a2934cb34ff8bb61623d3c1491347d33697f9988192a0e3ad1745932016cccf0ff9040604c0ec141275932f962591698b3

data/.rubocop.yml

@@ -32,3 +32,6 @@ Metrics/AbcSize:
 
 Style/GuardClause:
   Enabled: false
+
+Naming/AccessorMethodName:
+  Enabled: false

data/CHANGELOG.md

@@ -1,5 +1,25 @@
 ## [Unreleased]
 
+## [0.4.2] - 2022-07-25
+
+- Upgrade to use asherah-cobhan v0.4.15
+- Add `set_env` method to set environment variables for Asherah
+
+## [0.4.1] - 2022-03-25
+
+- Build and release platform gems
+
+## [0.4.0] - 2022-03-25
+
+- Download native file during gem install and verify checksum
+- Upgrade to use asherah-cobhan v0.4.11
+
+## [0.3.0] - 2022-03-22
+
+- Free up cobhan buffers after encrypt/decrypt to prevent growing heap memory
+- Use local `estimate_buffer` calculation instead of FFI call
+- Upgrade to use asherah-cobhan v0.4.3
+
 ## [0.2.0] - 2022-03-21
 
 - Implement versioning for asherah-cobhan binaries

data/README.md

@@ -1,8 +1,8 @@
 # Asherah
 
-Asherah is a Ruby wrapper around [Asherah Go](https://github.com/godaddy/asherah) application-layer encryption SDK that provides advanced encryption features and defense in depth against compromise. It uses a technique known as "envelope encryption" and supports cloud-agnostic data storage and key management.
+Asherah is a Ruby FFI wrapper around Go version of [Asherah](https://github.com/godaddy/asherah) application-layer encryption SDK. Asherah provides advanced encryption features and defense in depth against compromise. It uses a technique known as "envelope encryption" and supports cloud-agnostic data storage and key management.
 
-Check out the following documentation to get more familiar with its concepts:
+Check out the following documentation to get more familiar with the concepts and configuration options:
 
 - [Design and Architecture](https://github.com/godaddy/asherah/blob/master/docs/DesignAndArchitecture.md)
 - [Key Caching](https://github.com/godaddy/asherah/blob/master/docs/KeyCaching.md)
@@ -10,6 +10,10 @@ Check out the following documentation to get more familiar with its concepts:
 - [Metastore](https://github.com/godaddy/asherah/blob/master/docs/Metastore.md)
 - [System Requirements](https://github.com/godaddy/asherah/blob/master/docs/SystemRequirements.md)
 
+## Supported Platforms
+
+Currently supported platforms are Linux and Darwin operating systems for x64 and arm64 CPU architectures.
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -45,7 +49,7 @@ Encrypt some data for a `partition_id`
 
 ```ruby
 partition_id = 'user_1'
-data = 'Some PII data'
+data = 'PII data'
 data_row_record_json = Asherah.encrypt(partition_id, data)
 puts data_row_record_json
 ```
@@ -61,23 +65,22 @@ puts decrypted_data
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+To install this gem onto your local machine, run `rake install`.
 
-## Contributing
-
-Bug reports and pull requests are welcome on GitHub at https://github.com/godaddy/asherah-ruby.
-
-## Releasing new gem version
+To release a new version, update the version number in `version.rb`, create and push a version tag:
 
 ```
-# Create and push a version tag
 git tag -a v$(rake version) -m "Version $(rake version)"
 git push origin v$(rake version)
-
-# Create a release in Github to trigger .github/workflows/publish.yml workflow
-echo "Version $(rake version)"
 ```
 
+And then create a release in Github with title `echo "Version $(rake version)"` that will trigger `.github/workflows/publish.yml` workflow and push the `.gem` file to [rubygems.org](https://rubygems.org):
+
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/godaddy/asherah-ruby.
+
 ## License
 
 The gem is available as open source under the terms of the [MIT License](LICENSE.txt).

data/Rakefile

@@ -2,7 +2,6 @@
 
 require 'bundler/gem_tasks'
 require 'rspec/core/rake_task'
-require 'rubygems/package'
 
 RSpec::Core::RakeTask.new(:spec)
 
@@ -10,112 +9,21 @@ require 'rubocop/rake_task'
 
 RuboCop::RakeTask.new
 
-task default: %i[spec rubocop]
-
-ASHERAH_BIN = 'bin/download-asherah.sh'
-DISTRIBUTIONS = {
-  'x86_64-linux' => ['libasherah-x64.so'],
-  'x86_64-darwin' => ['libasherah-x64.dylib'],
-  'aarch64-linux' => ['libasherah-arm64.so'],
-  'arm64-darwin' => ['libasherah-arm64.dylib']
-}.freeze
-
-def current_filename
-  @current_filename ||=
-    begin
-      require 'cobhan'
-      Class.new.extend(Cobhan).library_file_name('libasherah')
-    end
-end
-
-def current_platform
-  @distribution ||= DISTRIBUTIONS.detect { |_k, v| v.include?(current_filename) }
-  @distribution.first
-end
-
-def native_build(platform, native_files)
-  puts "Building gem for #{platform}"
-
-  pkg_dir = File.join(__dir__, 'pkg')
-  FileUtils.mkdir_p(pkg_dir)
-
-  tmp_gem_dir = File.join(__dir__, 'tmp', platform)
-  FileUtils.rm_rf(tmp_gem_dir, verbose: true)
-  FileUtils.mkdir_p(tmp_gem_dir, verbose: true)
-
-  # Copy files to tmp gem dir
-  gemspec = Bundler.load_gemspec('asherah.gemspec')
-  (gemspec.files + [ASHERAH_BIN]).each do |file|
-    dir = File.dirname(file)
-    filename = File.basename(file)
-    FileUtils.mkdir_p(File.join(tmp_gem_dir, dir))
-    FileUtils.copy_file(file, File.join(tmp_gem_dir, dir, filename))
-  end
-
-  # Set platform for native gem build
-  gemspec.platform = Gem::Platform.new(platform)
-
-  native_dir = 'lib/asherah/native'
-  FileUtils.cd(tmp_gem_dir, verbose: true) do
-    FileUtils.mkdir_p(native_dir)
-    native_files.each do |native_file|
-      native_file_path = File.join(native_dir, native_file)
-
-      # Download native file
-      download_asherah_path = File.join(tmp_gem_dir, ASHERAH_BIN)
-      system("#{download_asherah_path} #{native_file}")
-
-      # Add native file in gemspec
-      gemspec.files << native_file_path
-    end
-
-    package = Gem::Package.build(gemspec)
-    FileUtils.mv package, File.join(pkg_dir, package)
+desc 'Download the binary for the current platform'
+task :download do
+  tmp_dir = 'tmp'
+  FileUtils.mkdir_p(tmp_dir)
+  FileUtils.cd(tmp_dir, verbose: true) do
+    system('ruby ../ext/asherah/extconf.rb')
   end
 end
 
-namespace :native do
-  desc 'Build all native gems'
-  task :build do
-    DISTRIBUTIONS.each do |platform, native_files|
-      native_build(platform, native_files)
-    end
-  end
-
-  namespace :build do
-    DISTRIBUTIONS.each do |platform, native_files|
-      desc "Build native gem for #{platform}"
-      task :"#{platform}" do
-        native_build(platform, native_files)
-      end
-    end
-  end
-
-  namespace :current do
-    desc 'Download asherah binary for current platform'
-    task :download do
-      download_asherah_path = File.join(__dir__, ASHERAH_BIN)
-      system("#{download_asherah_path} #{current_filename}")
-    end
-
-    desc 'Build native gem for current platform'
-    task :build do
-      native_build(current_platform, DISTRIBUTIONS[current_platform])
-    end
-
-    desc 'Smoke test native gem for current platform'
-    task smoke: :build do
-      platform = current_platform
-      gemspec = Bundler.load_gemspec('asherah.gemspec')
-      gemspec.platform = Gem::Platform.new(platform)
-
-      sh('gem uninstall asherah')
-      sh("gem install pkg/#{gemspec.file_name}")
-      sh('ruby spec/smoke_test.rb')
-    end
-  end
-end
+task default: %i[spec rubocop]
+task spec: :download
 
+desc 'Print current version'
 task :version do
   puts Asherah::VERSION
 end
+
+Rake.add_rakelib 'tasks'

data/asherah.gemspec

@@ -27,12 +27,13 @@ Gem::Specification.new do |spec|
   # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
   spec.files = Dir.chdir(File.expand_path(__dir__)) do
     `git ls-files -z`.split("\x0").reject do |f|
-      (f == __FILE__) || f.match(%r{\A(?:(?:bin|test|spec|features)/|\.(?:git|travis|circleci)|appveyor)})
+      (f == __FILE__) || f.match(%r{\A(?:(?:bin|test|spec|features|tasks)/|\.(?:git|travis|circleci)|appveyor)})
     end
   end
   spec.bindir = 'exe'
   spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
   spec.require_paths = ['lib']
+  spec.extensions = ['ext/asherah/extconf.rb']
 
   spec.add_dependency 'cobhan', '~> 0.2.0'
   spec.add_development_dependency 'dotenv', '~> 2.7.6'

data/ext/asherah/checksums.yml

@@ -0,0 +1,5 @@
+version:                v0.4.15
+libasherah-arm64.so:    43122390d0f851ac67bb197d688dd040832292e79675e4a9c9268d4ef5d3aef7
+libasherah-x64.so:      3d29f32f6560858c54dc3cc87fa59347f2981a0e206849f0f3ab9a905de02242
+libasherah-arm64.dylib: e7d64c2857c120b065c1761445d797f6fd5f6696c26676857ce327a772d6c025
+libasherah-x64.dylib:   331ea09b160de80a3e40aeeb737779da3c043e2476f7cbfe4b83cf55fccdf0c7

data/ext/asherah/extconf.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+require 'mkmf'
+create_makefile('asherah/asherah')
+
+require_relative 'native_file'
+NativeFile.download

data/ext/asherah/native_file.rb

@@ -0,0 +1,64 @@
+# frozen_string_literal: true
+
+require 'open-uri'
+require 'fileutils'
+require 'digest'
+require 'yaml'
+require 'cobhan'
+
+# Downloads native file and verifies checksum
+class NativeFile
+  LIB_NAME = 'libasherah'
+  ROOT_DIR = File.expand_path('../../', __dir__)
+  CHECKSUMS_FILE = File.expand_path('checksums.yml', __dir__)
+  CHECKSUMS = YAML.load_file(CHECKSUMS_FILE)
+  VERSION = CHECKSUMS.fetch('version')
+  RETRIES = 3
+  RETRY_DELAY = 1
+
+  class << self
+    def download(
+      file_name: Class.new.extend(Cobhan).library_file_name(LIB_NAME),
+      dir: File.join(ROOT_DIR, 'lib/asherah/native')
+    )
+      file_path = File.join(dir, file_name)
+      if File.exist?(file_path)
+        puts "#{file_path} already exists ... skipping download"
+        return
+      end
+
+      checksum = CHECKSUMS.fetch(file_name) do
+        abort "Unsupported platform #{RUBY_PLATFORM}"
+      end
+
+      content = download_content(file_name)
+
+      sha256 = Digest::SHA256.hexdigest(content)
+      abort "Could not verify checksum of #{file_name}" if sha256 != checksum
+
+      FileUtils.mkdir_p(dir)
+      File.binwrite(file_path, content)
+    end
+
+    private
+
+    def download_content(file_name)
+      tries = 0
+
+      begin
+        tries += 1
+        url = "https://github.com/godaddy/asherah-cobhan/releases/download/#{VERSION}/#{file_name}"
+        puts "Downloading #{url}"
+        URI.parse(url).open.read
+      rescue Net::OpenTimeout, Net::ReadTimeout => e
+        if tries <= RETRIES
+          puts "Got #{e.class}... retrying in #{RETRY_DELAY} seconds"
+          sleep RETRY_DELAY
+          retry
+        else
+          raise e
+        end
+      end
+    end
+  end
+end

data/lib/asherah/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Asherah
-  VERSION = '0.2.0'
+  VERSION = '0.4.2'
 end

data/lib/asherah.rb

@@ -11,14 +11,34 @@ module Asherah
 
   LIB_ROOT_PATH = File.expand_path('asherah/native', __dir__)
   load_library(LIB_ROOT_PATH, 'libasherah', [
+    [:SetEnv, [:pointer], :int32],
     [:SetupJson, [:pointer], :int32],
     [:EncryptToJson, [:pointer, :pointer, :pointer], :int32],
     [:DecryptFromJson, [:pointer, :pointer, :pointer], :int32],
-    [:EstimateBuffer, [:int32, :int32], :int32],
     [:Shutdown, [], :void]
   ].freeze)
 
+  ESTIMATED_ENCRYPTION_OVERHEAD = 48
+  ESTIMATED_ENVELOPE_OVERHEAD = 185
+  BASE64_OVERHEAD = 1.34
+
   class << self
+    # Set environment variables needed by Asherah dependencies for when
+    # Go os.Getenv() doesn't see variables set by C.setenv().
+    # References:
+    #   https://github.com/golang/go/wiki/cgo#environmental-variables
+    #   https://github.com/golang/go/issues/44108
+    #
+    # @yield [Config]
+    # @param env [Hash], Key-value pairs to set Asherah ENV
+    # @return [void]
+    def set_env(env = {})
+      env_buffer = string_to_cbuffer(env.to_json)
+
+      result = SetEnv(env_buffer)
+      Error.check_result!(result, 'SetEnv failed')
+    end
+
     # Configures Asherah
     #
     # @yield [Config]
@@ -27,6 +47,7 @@ module Asherah
       config = Config.new
       yield config
       config.validate!
+      @intermediated_key_overhead_bytesize = config.product_id.bytesize + config.service_name.bytesize
 
       config_buffer = string_to_cbuffer(config.to_json)
 
@@ -52,13 +73,15 @@ module Asherah
     def encrypt(partition_id, data)
       partition_id_buffer = string_to_cbuffer(partition_id)
       data_buffer = string_to_cbuffer(data)
-      estimated_length = EstimateBuffer(data.bytesize, partition_id.bytesize)
-      output_buffer = allocate_cbuffer(estimated_length)
+      estimated_buffer_bytesize = estimate_buffer(data.bytesize, partition_id.bytesize)
+      output_buffer = allocate_cbuffer(estimated_buffer_bytesize)
 
       result = EncryptToJson(partition_id_buffer, data_buffer, output_buffer)
       Error.check_result!(result, 'EncryptToJson failed')
 
       cbuffer_to_string(output_buffer)
+    ensure
+      [partition_id_buffer, data_buffer, output_buffer].map(&:free)
     end
 
     # Decrypts a DataRowRecord in JSON format for a partition_id and returns decrypted data.
@@ -75,11 +98,22 @@ module Asherah
       Error.check_result!(result, 'DecryptFromJson failed')
 
       cbuffer_to_string(output_buffer)
+    ensure
+      [partition_id_buffer, data_buffer, output_buffer].map(&:free)
     end
 
     # Stop the Asherah instance
     def shutdown
       Shutdown()
     end
+
+    private
+
+    def estimate_buffer(data_bytesize, partition_bytesize)
+      ESTIMATED_ENVELOPE_OVERHEAD +
+        @intermediated_key_overhead_bytesize +
+        partition_bytesize +
+        ((data_bytesize + ESTIMATED_ENCRYPTION_OVERHEAD) * BASE64_OVERHEAD)
+    end
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: asherah
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.4.2
 platform: x86_64-linux
 authors:
 - GoDaddy
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2022-03-21 00:00:00.000000000 Z
+date: 2022-07-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: cobhan
@@ -100,7 +100,8 @@ description: |
 email:
 - oss@godaddy.com
 executables: []
-extensions: []
+extensions:
+- ext/asherah/extconf.rb
 extra_rdoc_files: []
 files:
 - ".rspec"
@@ -115,6 +116,9 @@ files:
 - Rakefile
 - SECURITY.md
 - asherah.gemspec
+- ext/asherah/checksums.yml
+- ext/asherah/extconf.rb
+- ext/asherah/native_file.rb
 - lib/asherah.rb
 - lib/asherah/config.rb
 - lib/asherah/error.rb