appmap 0.28.0 → 0.34.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d94a20303fe329af9dfa6f4a0d4b14666a7eb25cba58d0ea6ede64b5991e85f9
-  data.tar.gz: c441d6ee26453338fa449fed7762a0768155eaa934168c2245c47a5deb78cb38
+  metadata.gz: 47030df4910179ac636b8a2c4c3c29c25225deb1b9cbce851264d1e2ae8dacf1
+  data.tar.gz: 84bc3de36a1f7700fe32128c40f8d2bf6d698208689e856fd2bc696c38be6500
 SHA512:
-  metadata.gz: a66524169eeb5e8d78974a8ac37122f52d5e2285cb4ff6f88f58947e8016d2b4ac458430df07a5b4e9c5ab37fdc7122bb887c694e375a7013bc8f34bf1b6b6b2
-  data.tar.gz: 665354eca628d2d0f68706cf297667b9b3392a43e6dc364923f1f45f5316a6d3db54dea2d1e75ae83cd53c3ed9cb40ecc3794c10c9d6ee1c310aeed6710bb931
+  metadata.gz: 98d8933599f9cdb39d2ab9d69d1c11c7f06972603c30b5d79b3c57dc2ec246aec2c432da4835bcab805d49ec17e757fc0377d583908c6894380e951071b2d797
+  data.tar.gz: eefe2496acc6be7ebfe20eee2ab9ed0e708658a039c196757dd00ca18351e0425d3a6d1f0599cca37bd4a2c786469d61fdc7ad0e50a56d9223360204e2bc4206

data/CHANGELOG.md

@@ -1,3 +1,35 @@
+# v0.34.0
+
+* Records builtin security and I/O methods from `OpenSSL`, `Net`, and `IO`.
+
+# v0.33.0
+
+* Added command `AppMap.open` to open an AppMap in the browser.
+
+# v0.32.0
+
+* Removes un-necessary fields from `return` events.
+
+# v0.31.0
+
+* Add the ability to hook methods by default, and optionally add labels to them in the
+  classmap. Use it to hook `ActiveSupport::SecurityUtils.secure_compare`.
+  
+# v0.30.0
+
+* Add support for Minitest.
+
+# v0.29.0
+
+* Add `lib/appmap/record.rb`, which can be `require`d to record the rest of the process.
+
+# v0.28.1
+
+* Fix the `defined_class` recorded in an appmap for an instance method included in a class
+  at runtime.
+* Only include the `static` attribute on `call` events in an appmap. Determine its value
+  based on the receiver of the method call.
+
 # v0.28.0
 
 * Change behavior of **AppMap.record** to return a complete AppMap as a Hash.

data/README.md

@@ -3,6 +3,7 @@
 - [Configuration](#configuration)
 - [Running](#running)
   - [RSpec](#rspec)
+  - [Minitest](#minitest)
   - [Cucumber](#cucumber)
   - [Remote recording](#remote-recording)
   - [Ruby on Rails](#ruby-on-rails)
@@ -32,7 +33,7 @@ There are several ways to record AppMaps of your Ruby program using the `appmap`
 
 Once you have recorded some AppMaps (for example, by running RSpec tests), you use the `appland upload` command
 to upload them to the AppLand server. This command, and some others, is provided
-by the [AppLand CLI](https://github.com/applandinc/appland-cli/releases), to
+by the [AppLand CLI](https://github.com/applandinc/appland-cli/releases).
 Then, on the [AppLand website](https://app.land), you can
 visualize the design of your code and share links with collaborators.
 
@@ -86,12 +87,25 @@ Each entry in the `packages` list is a YAML object which has the following keys:
 
 To record RSpec tests, follow these additional steps:
 
-1) Require `appmap/rspec` in your `spec_helper.rb`.
+1) Require `appmap/rspec` in your `spec_helper.rb` before any other classes are loaded.
 
 ```ruby
 require 'appmap/rspec'
 ```
 
+Note that `spec_helper.rb` in a Rails project typically loads the application's classes this way:
+
+```ruby
+require File.expand_path("../../config/environment", __FILE__)
+```
+
+and `appmap/rspec` must be required before this:
+
+```ruby
+require 'appmap/rspec'
+require File.expand_path("../../config/environment", __FILE__)
+```
+
 2) *Optional* Add `feature: '<feature name>'` and `feature_group: '<feature group name>'` annotations to your 
    examples. 
 
@@ -125,6 +139,42 @@ If you include the `feature` and `feature_group` metadata, these attributes will
 
 If you don't explicitly declare `feature` and `feature_group`, then they will be inferred from the spec name and example descriptions.
 
+## Minitest
+
+To record Minitest tests, follow these additional steps:
+
+1) Require `appmap/minitest` in `test_helper.rb`
+
+```ruby
+require 'appmap/minitest'
+```
+
+Note that `test_helper.rb` in a Rails project typically loads the application's classes this way:
+
+```ruby
+require_relative '../config/environment'
+```
+
+and `appmap/rspec` must be required before this:
+
+```ruby
+require 'appmap/rspec'
+require_relative '../config/environment'
+```
+
+2) Run the tests with the environment variable `APPMAP=true`:
+
+```sh-session
+$ APPMAP=true bundle exec -Ilib -Itest test/*
+```
+
+Each Minitest test will output an AppMap file into the directory `tmp/appmap/minitest`. For example:
+
+```
+$ find tmp/appmap/minitest
+Hello_says_hello_when_prompted.appmap.json
+```
+
 ## Cucumber
 
 To record Cucumber tests, follow these additional steps:
@@ -135,6 +185,8 @@ To record Cucumber tests, follow these additional steps:
 require 'appmap/cucumber'
 ```
 
+Be sure to require it before `config/environment` is required.
+
 2) Create an `Around` hook in `support/hooks.rb` to record the scenario:
 
 

data/Rakefile

@@ -122,7 +122,7 @@ end
 Rake::TestTask.new(:minitest) do |t|
   t.libs << 'test'
   t.libs << 'lib'
-  t.test_files = FileList['test/**/*_test.rb']
+  t.test_files = FileList['test/*_test.rb']
 end
 
 task spec: "spec:all"

data/appmap.gemspec

@@ -26,6 +26,7 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'faraday'
   spec.add_dependency 'gli'
   spec.add_dependency 'parser'
+  spec.add_dependency 'rack'
 
   spec.add_development_dependency 'bundler', '~> 1.16'
   spec.add_development_dependency 'minitest', '~> 5.0'

data/lib/appmap.rb

@@ -8,19 +8,26 @@ rescue NameError
 end
 
 require 'appmap/version'
+require 'appmap/hook'
+require 'appmap/config'
+require 'appmap/trace'
+require 'appmap/class_map'
+require 'appmap/metadata'
+require 'appmap/util'
+require 'appmap/open'
 
 module AppMap
   class << self
     @configuration = nil
     @configuration_file_path = nil
 
-    # configuration gets the configuration. If there is no configuration, the default
+    # Gets the configuration. If there is no configuration, the default
     # configuration is initialized.
     def configuration
-      @configuration ||= configure
+      @configuration ||= initialize
     end
 
-    # configuration= sets the configuration. This is only expected to happen once per
+    # Sets the configuration. This is only expected to happen once per
     # Ruby process.
     def configuration=(config)
       warn 'AppMap is already configured' if @configuration && config
@@ -28,24 +35,24 @@ module AppMap
       @configuration = config
     end
 
-    # initialize configures AppMap for recording. Default behavior is to configure from "appmap.yml".
+    # Configures AppMap for recording. Default behavior is to configure from "appmap.yml".
     # This method also activates the code hooks which record function calls as trace events.
     # Call this function before the program code is loaded by the Ruby VM, otherwise
     # the load events won't be seen and the hooks won't activate.
     def initialize(config_file_path = 'appmap.yml')
       warn "Configuring AppMap from path #{config_file_path}"
-      require 'appmap/hook'
-      self.configuration = Hook::Config.load_from_file(config_file_path)
-      Hook.hook(configuration)
+      Config.load_from_file(config_file_path).tap do |configuration|
+        self.configuration = configuration
+        Hook.new(configuration).enable
+      end
     end
 
-    # tracing can be used to start tracing, stop tracing, and record events.
+    # Used to start tracing, stop tracing, and record events.
     def tracing
-      require 'appmap/trace'
       @tracing ||= Trace::Tracing.new
     end
 
-    # record records the events which occur while processing a block,
+    # Records the events which occur while processing a block,
     # and returns an AppMap as a Hash.
     def record
       tracer = tracing.trace
@@ -66,16 +73,20 @@ module AppMap
       }
     end
 
-    # class_map builds a class map from a config and a list of Ruby methods.
+    # Uploads an AppMap to the AppLand website and displays it.
+    def open(appmap = nil, &block)
+      appmap ||= AppMap.record(&block)
+      AppMap::Open.new(appmap).perform
+    end
+
+    # Builds a class map from a config and a list of Ruby methods.
     def class_map(methods)
-      require 'appmap/class_map'
       ClassMap.build_from_methods(configuration, methods)
     end
 
-    # detect_metadata returns default metadata detected from the Ruby system and from the
+    # Returns default metadata detected from the Ruby system and from the
     # filesystem.
     def detect_metadata
-      require 'appmap/metadata'
       @metadata ||= Metadata.detect.freeze
       @metadata.deep_dup
     end

data/lib/appmap/algorithm/stats.rb

@@ -74,10 +74,11 @@ module AppMap
 
         class_name_func = ->(event) { event['defined_class'] }
         full_name_func = lambda do |event|
+          call = event['event'] == 'call'
           class_name = event['defined_class']
           static = event['static']
           function_name = event['method_id']
-          [ class_name, static ? '.' : '#', function_name ].join if class_name && !static.nil? && function_name
+          [ class_name, static ? '.' : '#', function_name ].join if call && class_name && !static.nil? && function_name
         end
 
         class_frequency = frequency_calc.call(class_name_func)

data/lib/appmap/class_map.rb

@@ -1,7 +1,5 @@
 # frozen_string_literal: true
 
-require 'active_support/core_ext'
-
 module AppMap
   class ClassMap
     module HasChildren
@@ -50,7 +48,7 @@ module AppMap
         end
       end
       Function = Struct.new(:name) do
-        attr_accessor :static, :location
+        attr_accessor :static, :location, :labels
 
         def type
           'function'
@@ -61,8 +59,9 @@ module AppMap
             name: name,
             type: type,
             location: location,
-            static: static
-          }
+            static: static,
+            labels: labels
+          }.delete_if { |k,v| v.nil? || v == [] }
         end
       end
     end
@@ -71,35 +70,21 @@ module AppMap
       def build_from_methods(config, methods)
         root = Types::Root.new
         methods.each do |method|
-          package = package_for_method(config.packages, method)
-          add_function root, package.path, method
+          package = config.package_for_method(method) \
+            or raise "No package found for method #{method}"
+          add_function root, package, method
         end
         root.children.map(&:to_h)
       end
 
       protected
 
-      def package_for_method(packages, method)
-        location = method.method.source_location
-        location_file, = location
-        location_file = location_file[Dir.pwd.length + 1..-1] if location_file.index(Dir.pwd) == 0
-
-        packages.find do |pkg|
-          (location_file.index(pkg.path) == 0) &&
-            !pkg.exclude.find { |p| location_file.index(p) }
-        end or raise "No package found for method #{method}"
-      end
-
-      def add_function(root, package_name, method)
-        location = method.method.source_location
-        location_file, lineno = location
-        location_file = location_file[Dir.pwd.length + 1..-1] if location_file.index(Dir.pwd) == 0
-
-        static = method.method.owner.singleton_class?
+      def add_function(root, package, method)
+        static = method.static
 
         object_infos = [
           {
-            name: package_name,
+            name: package.path,
             type: 'package'
           }
         ]
@@ -109,12 +94,25 @@ module AppMap
             type: 'class'
           }
         end
-        object_infos << {
-          name: method.method.name,
+        function_info = {
+          name: method.name,
           type: 'function',
-          location: [ location_file, lineno ].join(':'),
           static: static
         }
+        location = method.source_location
+
+        function_info[:location] = \
+          if location
+            location_file, lineno = location
+            location_file = location_file[Dir.pwd.length + 1..-1] if location_file.index(Dir.pwd) == 0
+            [ location_file, lineno ].join(':')
+          else
+            [ method.defined_class, static ? '.' : '#', method.name ].join
+          end
+
+        function_info[:labels] = package.labels if package.labels
+        object_infos << function_info
+
         parent = root
         object_infos.each do |info|
           parent = find_or_create parent.children, info do

data/lib/appmap/config.rb

@@ -0,0 +1,115 @@
+# frozen_string_literal: true
+
+module AppMap
+  class Config
+    Package = Struct.new(:path, :package_name, :exclude, :labels) do
+      def initialize(path, package_name: nil, exclude: [], labels: [])
+        super path, package_name, exclude, labels
+      end
+  
+      def to_h
+        {
+          path: path,
+          package_name: package_name,
+          exclude: exclude.blank? ? nil : exclude,
+          labels: labels.blank? ? nil : labels
+        }.compact
+      end
+    end
+
+    Hook = Struct.new(:method_names, :package) do
+    end
+
+    OPENSSL_PACKAGE = Package.new('openssl', package_name: 'openssl', labels: %w[security crypto])
+
+    # Methods that should always be hooked, with their containing
+    # package and labels that should be applied to them.
+    HOOKED_METHODS = {
+      'ActiveSupport::SecurityUtils' => Hook.new(:secure_compare, Package.new('active_support', package_name: 'active_support', labels: %w[security crypto]))
+    }.freeze
+
+    BUILTIN_METHODS = {
+      'OpenSSL::PKey::PKey' => Hook.new(:sign, OPENSSL_PACKAGE),
+      'Digest::Instance' => Hook.new(:digest, OPENSSL_PACKAGE),
+      'OpenSSL::X509::Request' => Hook.new(%i[sign verify], OPENSSL_PACKAGE),
+      'OpenSSL::PKCS5' => Hook.new(%i[pbkdf2_hmac_sha1 pbkdf2_hmac], OPENSSL_PACKAGE),
+      'OpenSSL::Cipher' => Hook.new(%i[encrypt decrypt final], OPENSSL_PACKAGE),
+      'OpenSSL::X509::Certificate' => Hook.new(:sign, OPENSSL_PACKAGE),
+      'Logger' => Hook.new(:add, Package.new('logger', labels: %w[log io])),
+      'Net::HTTP' => Hook.new(:request, Package.new('net/http', package_name: 'net/http', labels: %w[http io])),
+      'Net::SMTP' => Hook.new(:send, Package.new('net/smtp', package_name: 'net/smtp', labels: %w[smtp email io])),
+      'Net::POP3' => Hook.new(:mails, Package.new('net/pop3', package_name: 'net/pop', labels: %w[pop pop3 email io])),
+      'Net::IMAP' => Hook.new(:send_command, Package.new('net/imap', package_name: 'net/imap', labels: %w[imap email io])),
+      'IO' => Hook.new(%i[read write open close], Package.new('io', labels: %w[io])),
+      'Marshal' => Hook.new(%i[dump load], Package.new('marshal', labels: %w[serialization marshal])),
+      'Psych' => Hook.new(%i[dump dump_stream load load_stream parse parse_stream], Package.new('yaml', package_name: 'psych', labels: %w[serialization yaml])),
+      'JSON::Ext::Parser' => Hook.new(:parse, Package.new('json', package_name: 'json', labels: %w[serialization json])),
+      'JSON::Ext::Generator::State' => Hook.new(:generate, Package.new('json', package_name: 'json', labels: %w[serialization json]))
+    }.freeze
+
+    attr_reader :name, :packages
+
+    def initialize(name, packages = [])
+      @name = name
+      @packages = packages
+    end
+
+    class << self
+      # Loads configuration data from a file, specified by the file name.
+      def load_from_file(config_file_name)
+        require 'yaml'
+        load YAML.safe_load(::File.read(config_file_name))
+      end
+
+      # Loads configuration from a Hash.
+      def load(config_data)
+        packages = (config_data['packages'] || []).map do |package|
+          Package.new(package['path'], exclude: package['exclude'] || [])
+        end
+        Config.new config_data['name'], packages
+      end
+    end
+
+    def to_h
+      {
+        name: name,
+        packages: packages.map(&:to_h)
+      }
+    end
+
+    def package_for_method(method)
+      defined_class, _, method_name = ::AppMap::Hook.qualify_method_name(method)
+      package = find_package(defined_class, method_name)
+      return package if package
+
+      location = method.source_location
+      location_file, = location
+      return unless location_file
+
+      location_file = location_file[Dir.pwd.length + 1..-1] if location_file.index(Dir.pwd) == 0
+      packages.find do |pkg|
+        (location_file.index(pkg.path) == 0) &&
+          !pkg.exclude.find { |p| location_file.index(p) }
+      end
+    end
+
+    def included_by_location?(method)
+      !!package_for_method(method)
+    end
+
+    def always_hook?(defined_class, method_name)
+      !!find_package(defined_class, method_name)
+    end
+
+    def find_package(defined_class, method_name)
+      hook = find_hook(defined_class)
+      return nil unless hook
+
+      Array(hook.method_names).include?(method_name) ? hook.package : nil      
+    end
+
+    def find_hook(defined_class)
+      HOOKED_METHODS[defined_class] || BUILTIN_METHODS[defined_class]
+    end
+  end
+end