appmap 0.40.0 → 0.42.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1b0943dadde84e67780d724918fa26377b869791eaab6edfb714e5081ecb5c80
-  data.tar.gz: c492323d80541a913b800924d04a7c24a6c8682263ba3ec4deabfa8d7ade3908
+  metadata.gz: c70b96dcf283fa03ea3acc787a78ecf3a732a78808834ba0acf09d4e398172c6
+  data.tar.gz: ec83f4e0eb8908e971bc67e27696e5b37252289a50b848b85bd54e56520d3a75
 SHA512:
-  metadata.gz: 2a27e5b9bc3dafbba8aafd7035732b6ffcb813d57bb216d0e544af30260c0e26f23a920d1b2d6d35882046e74a57f8b492b12b9869de5d820ff6a4e09cafd716
-  data.tar.gz: f1bdc22fda7a6fdf0a37fc1d5bd419947e0fffa16511873dbe8a1549f30bf862566fe74fa4c9952c4338f5263414c99f3d598daec0a599b467e52ebac374b4f7
+  metadata.gz: 1e19e607c94eee4cfebf6646f1802e27446330807a1dd252bbd94d823def31ed9abf59668f3d0672a6bfac9261e076338fe13d27a0dbea81e83da80ff726cf7c
+  data.tar.gz: 3221cdfb7c164efecbe022f89829c4e80568b3fa5a7343c2850c8e1d4387f807e5ebcba0d04345bd5641ab3ee9a32d92faba3337dfe5f4bbe5f254fd9320d4a4

data/CHANGELOG.md

@@ -1,3 +1,29 @@
+# v0.42.1
+
+* Add missing require `set`.
+* Check `cls.respond_to?(:singleton_class)`, since it oddly, may not.
+
+# v0.42.0
+
+* Remove `feature_group` and `feature` metadata from minitest and RSpec AppMaps.
+* Add `metadata.source_location`.
+
+# v0.41.2
+
+* Don't rely on `gemspec.source_paths` to list all the source locations in a gem. Hook any code that's loaded
+  from inside the `gem_dir`.
+
+# v0.41.1
+
+* Make best effort to ensure that class name is not `null` in the appmap.json.
+* Don't try and instrument gems which are a dependency of the this gem.
+* Fix a nil exception when applying the exclude list to builtins.
+
+# v0.41.0
+
+* Adjust some label names to match `provider.*`, `format.*`.
+* Add global `exclude` list to *appmap.yml* which can be used to definitively exclude specific classes and methods.
+
 # v0.40.0
 
 * Parse source code comments into function labels.

data/README.md

@@ -9,6 +9,7 @@
   - [Minitest](#minitest)
   - [Cucumber](#cucumber)
   - [Remote recording](#remote-recording)
+  - [Server process recording](#server-process-recording)
 - [AppMap for VSCode](#appmap-for-vscode)
 - [Uploading AppMaps](#uploading-appmaps)
 - [Development](#development)
@@ -53,15 +54,23 @@ Support for new versions is added frequently, please check back regularly for up
 <a href="https://www.loom.com/share/78ab32a312ff4b85aa8827a37f1cb655"> <p>Quick and easy setup of the AppMap gem for Rails - Watch Video</p> <img style="max-width:300px;" src="https://cdn.loom.com/sessions/thumbnails/78ab32a312ff4b85aa8827a37f1cb655-with-play.gif"> </a>
 
 
-Add `gem 'appmap'` to your Gemfile just as you would any other dependency. We recommend that the Gem be added to the `:development, :test` section.
+Add `gem 'appmap'` to **beginning** of your Gemfile. We recommend that you add the `appmap` gem to the `:development, :test` group. Your Gemfile should look something like this:
 
 ```
+source 'https://rubygems.org'
+git_source(:github) { |repo| "https://github.com/#{repo}.git" }
+
+# Optional rubRuby version
+# ruby '2.7.2'
+
 group :development, :test do
   gem 'appmap'
 end
 ```
 
-Then install with `bundle`.
+Install with `bundle install`, as usual.
+
+It's important to add the `appmap` gem before any other gems that you may want to instrument. There is more about this in the section on adding gems to the *appmap.yml*.
 
 **Railtie**
 
@@ -73,13 +82,30 @@ If you are using Ruby on Rails, require the railtie after Rails is loaded.
 require 'appmap/railtie' if defined?(AppMap).
 ```
 
+**application.rb**
+
+Add this line to *application.rb*, to enable server recording with `APPMAP_RECORD=true`:
+
+```ruby
+module MyApp
+  class Application < Rails::Application
+    ...
+  
+    config.appmap.enabled = true if ENV['APPMAP_RECORD']
+    
+    ...
+  end
+end
+```
+
 # Configuration
 
 When you run your program, the `appmap` gem reads configuration settings from `appmap.yml`. Here's a sample configuration
 file for a typical Rails project:
 
 ```yaml
-name: MyProject
+# 'name' should generally be the same as the code repo name.
+name: my_project
 packages:
 - path: app/controllers
 - path: app/models
@@ -91,10 +117,15 @@ packages:
 - gem: devise
 - gem: aws-sdk
 - gem: will_paginate
+exclude:
+- MyClass
+- MyClass#my_instance_method
+- MyClass.my_class_method
 ```
 
 * **name** Provides the project name (required)
-* **packages** A list of source code directories which should be instrumented.
+* **packages** A list of source code directories which should be recorded.
+* **exclude** A list of classes and/or methods to definitively exclude from recording.
 
 **packages**
 
@@ -102,13 +133,18 @@ Each entry in the `packages` list is a YAML object which has the following keys:
 
 * **path** The path to the source code directory. The path may be relative to the current working directory, or it may
   be an absolute path.
-* **gem** As an alternative to specifying the path, specify the name of a dependency gem. When using `gem`, don't specify `path`.
+* **gem** As an alternative to specifying the path, specify the name of a dependency gem. When using `gem`, don't specify `path`. In your `Gemfile`, the `appmap` gem **must** be listed **before** any gem that you specify in your *appmap.yml*.
 * **exclude** A list of files and directories which will be ignored. By default, all modules, classes and public
-  functions are inspected.
+  functions are inspected. See also: global `exclude` list.
 * **shallow** When set to `true`, only the first function call entry into a package will be recorded. Subsequent function calls within 
   the same package are not recorded unless code execution leaves the package and re-enters it. Default: `true` when using `gem`,
   `false` when using `path`.
 
+**exclude**
+
+Optional list of fully qualified class and method names. Separate class and method names with period (`.`) for class methods and hash (`#`) for instance methods.
+
+
 # Labels
 
 The [AppMap data format](https://github.com/applandinc/appmap) provides for class and function `labels`, which can be used to enhance the AppMap visualizations, and to programatically analyze the data.
@@ -287,6 +323,12 @@ $ bundle exec rails server
 
 6. Open the AppLand browser extension and push `Stop`. The recording will be transferred to the AppLand website and opened in your browser.
 
+## Server process recording
+
+Run your Rails server with `APPMAP_RECORD=true`. When the server exits, an *appmap.json* file will be written to the project directory. This is a great way to start the server, interact with your app as a user (or through it's API), and then view an AppMap of everything that happened.
+
+Be sure and set `WEB_CONCURRENCY=1`, if you are using a webserver that can run multiple processes. You only want there to be one process while you are recording, otherwise they will both try and write *appmap.json* and one of them will clobber the other.
+
 # AppMap for VSCode
 
 The [AppMap extension for VSCode](https://marketplace.visualstudio.com/items?itemName=appland.appmap) is a great way to onboard developers to new code, and troubleshoot hard-to-understand bugs with visuals.
@@ -298,7 +340,7 @@ The [AppMap extension for VSCode](https://marketplace.visualstudio.com/items?ite
 For instructions on uploading, see the documentation of the [AppLand CLI](https://github.com/applandinc/appland-cli).
 
 # Development
-[![Build Status](https://travis-ci.org/applandinc/appmap-ruby.svg?branch=master)](https://travis-ci.org/applandinc/appmap-ruby)
+[![Build Status](https://travis-ci.com/applandinc/appmap-ruby.svg?branch=master)](https://travis-ci.com/applandinc/appmap-ruby)
 
 ## Running tests
 

data/lib/appmap.rb

@@ -50,6 +50,11 @@ module AppMap
       end
     end
 
+    # Whether to include source and comments in all class maps.
+    def include_source?
+      ENV['APPMAP_SOURCE'] == 'true'
+    end
+
     # Used to start tracing, stop tracing, and record events.
     def tracing
       @tracing ||= Trace::Tracing.new

data/lib/appmap/command/record.rb

@@ -27,7 +27,7 @@ module AppMap
           event_thread.join
           yield AppMap::APPMAP_FORMAT_VERSION,
                 AppMap.detect_metadata,
-                AppMap.class_map(tracer.event_methods),
+                AppMap.class_map(tracer.event_methods, include_source: AppMap.include_source?),
                 events
         end
 

data/lib/appmap/config.rb

@@ -16,20 +16,20 @@ module AppMap
         end
 
         def build_from_gem(gem, shallow: true, package_name: nil, exclude: [], labels: [])
-          gem_paths(gem).map do |gem_path|
-            Package.new(gem_path, gem, package_name, exclude, labels, shallow)
+          if %w[method_source activesupport].member?(gem)
+            warn "WARNING: #{gem} cannot be AppMapped because it is a dependency of the appmap gem"
+            return
           end
+          Package.new(gem_path(gem), gem, package_name, exclude, labels, shallow)
         end
 
         private_class_method :new
 
         protected
 
-        def gem_paths(gem)
+        def gem_path(gem)
           gemspec = Gem.loaded_specs[gem] or raise "Gem #{gem.inspect} not found"
-          gemspec.source_paths.map do |path|
-            File.join(gemspec.gem_dir, path)
-          end
+          gemspec.gem_dir
         end
       end
 
@@ -57,32 +57,36 @@ module AppMap
     # Methods that should always be hooked, with their containing
     # package and labels that should be applied to them.
     HOOKED_METHODS = {
-      'ActiveSupport::SecurityUtils' => Hook.new(:secure_compare, Package.build_from_path('active_support', package_name: 'active_support', labels: %w[security crypto])),
-      'ActionView::Renderer' => Hook.new(:render, Package.build_from_path('action_view', package_name: 'action_view', labels: %w[view]))
+      'ActiveSupport::SecurityUtils' => Hook.new(:secure_compare, Package.build_from_path('active_support', labels: %w[provider.secure_compare])),
+      'ActionView::Renderer' => Hook.new(:render, Package.build_from_path('action_view', labels: %w[mvc.view])),
+      'ActionDispatch::Cookies::CookieJar' => Hook.new(%i[[]= clear update delete recycle], Package.build_from_path('action_pack', labels: %w[provider.http.cookie])),
+      'ActionDispatch::Cookies::EncryptedCookieJar' => Hook.new(%i[[]=], Package.build_from_path('action_pack', labels: %w[provider.http.cookie crypto])),
+      'CanCan::ControllerAdditions' => Hook.new(%i[authorize! can? cannot?], Package.build_from_path('cancancan', labels: %w[provider.authorization])),
+      'CanCan::Ability' => Hook.new(%i[authorize!], Package.build_from_path('cancancan', labels: %w[provider.authorization])),
     }.freeze
 
     BUILTIN_METHODS = {
       'OpenSSL::PKey::PKey' => Hook.new(:sign, OPENSSL_PACKAGES),
-      'Digest::Instance' => Hook.new(:digest, OPENSSL_PACKAGES),
       'OpenSSL::X509::Request' => Hook.new(%i[sign verify], OPENSSL_PACKAGES),
       'OpenSSL::PKCS5' => Hook.new(%i[pbkdf2_hmac_sha1 pbkdf2_hmac], OPENSSL_PACKAGES),
       'OpenSSL::Cipher' => Hook.new(%i[encrypt decrypt final], OPENSSL_PACKAGES),
       'OpenSSL::X509::Certificate' => Hook.new(:sign, OPENSSL_PACKAGES),
-      'Net::HTTP' => Hook.new(:request, Package.build_from_path('net/http', package_name: 'net/http', labels: %w[http io])),
-      'Net::SMTP' => Hook.new(:send, Package.build_from_path('net/smtp', package_name: 'net/smtp', labels: %w[smtp email io])),
-      'Net::POP3' => Hook.new(:mails, Package.build_from_path('net/pop3', package_name: 'net/pop', labels: %w[pop pop3 email io])),
-      'Net::IMAP' => Hook.new(:send_command, Package.build_from_path('net/imap', package_name: 'net/imap', labels: %w[imap email io])),
-      'Marshal' => Hook.new(%i[dump load], Package.build_from_path('marshal', labels: %w[serialization marshal])),
-      'Psych' => Hook.new(%i[dump dump_stream load load_stream parse parse_stream], Package.build_from_path('yaml', package_name: 'psych', labels: %w[serialization yaml])),
-      'JSON::Ext::Parser' => Hook.new(:parse, Package.build_from_path('json', package_name: 'json', labels: %w[serialization json])),
-      'JSON::Ext::Generator::State' => Hook.new(:generate, Package.build_from_path('json', package_name: 'json', labels: %w[serialization json])),
+      'Net::HTTP' => Hook.new(:request, Package.build_from_path('net/http', package_name: 'net/http', labels: %w[protocol.http io])),
+      'Net::SMTP' => Hook.new(:send, Package.build_from_path('net/smtp', package_name: 'net/smtp', labels: %w[protocol.smtp protocol.email io])),
+      'Net::POP3' => Hook.new(:mails, Package.build_from_path('net/pop3', package_name: 'net/pop', labels: %w[protocol.pop protocol.email io])),
+      'Net::IMAP' => Hook.new(:send_command, Package.build_from_path('net/imap', package_name: 'net/imap', labels: %w[protocol.imap protocol.email io])),
+      'Marshal' => Hook.new(%i[dump load], Package.build_from_path('marshal', labels: %w[format.marshal provider.serialization])),
+      'Psych' => Hook.new(%i[dump dump_stream load load_stream parse parse_stream], Package.build_from_path('yaml', package_name: 'psych', labels: %w[format.yaml provider.serialization])),
+      'JSON::Ext::Parser' => Hook.new(:parse, Package.build_from_path('json', package_name: 'json', labels: %w[format.json provider.serialization])),
+      'JSON::Ext::Generator::State' => Hook.new(:generate, Package.build_from_path('json', package_name: 'json', labels: %w[format.json provider.serialization])),
     }.freeze
 
-    attr_reader :name, :packages
+    attr_reader :name, :packages, :exclude
 
-    def initialize(name, packages = [])
+    def initialize(name, packages = [], exclude = [])
       @name = name
       @packages = packages
+      @exclude = exclude
     end
 
     class << self
@@ -105,44 +109,60 @@ module AppMap
             shallow = true if shallow.nil?
             Package.build_from_gem(gem, exclude: package['exclude'] || [], shallow: shallow)
           else
-            [ Package.build_from_path(path, exclude: package['exclude'] || [], shallow: package['shallow']) ]
+            Package.build_from_path(path, exclude: package['exclude'] || [], shallow: package['shallow'])
           end
-        end.flatten
-        Config.new config_data['name'], packages
+        end.compact
+        Config.new config_data['name'], packages, config_data['exclude'] || []
       end
     end
 
     def to_h
       {
         name: name,
-        packages: packages.map(&:to_h)
+        packages: packages.map(&:to_h),
+        exclude: exclude
       }
     end
 
+    # package_for_method finds the Package, if any, which configures the hook
+    # for a method.
     def package_for_method(method)
+      package_hooked_by_class(method) || package_hooked_by_source_location(method)
+    end
+
+    def package_hooked_by_class(method)
       defined_class, _, method_name = ::AppMap::Hook.qualify_method_name(method)
-      package = find_package(defined_class, method_name)
-      return package if package
+      return find_package(defined_class, method_name)
+    end
 
+    def package_hooked_by_source_location(method)
       location = method.source_location
       location_file, = location
       return unless location_file
 
       location_file = location_file[Dir.pwd.length + 1..-1] if location_file.index(Dir.pwd) == 0
-      packages.find do |pkg|
+      packages.select { |pkg| pkg.path }.find do |pkg|
         (location_file.index(pkg.path) == 0) &&
           !pkg.exclude.find { |p| location_file.index(p) }
       end
     end
 
-    def included_by_location?(method)
-      !!package_for_method(method)
+    def never_hook?(method)
+      defined_class, separator, method_name = ::AppMap::Hook.qualify_method_name(method)
+      return true if exclude.member?(defined_class) || exclude.member?([ defined_class, separator, method_name ].join)
     end
 
+    # always_hook? indicates a method that should always be hooked.
     def always_hook?(defined_class, method_name)
       !!find_package(defined_class, method_name)
     end
 
+    # included_by_location? indicates a method whose source location matches a method definition that has been
+    # configured for inclusion.
+    def included_by_location?(method)
+      !!package_for_method(method)
+    end
+
     def find_package(defined_class, method_name)
       hook = find_hook(defined_class)
       return nil unless hook

data/lib/appmap/event.rb

@@ -38,6 +38,15 @@ module AppMap
 
         protected
 
+        # Heuristic for dynamically defined class whose name can be nil
+        def best_class_name(value)
+          value_cls = value.class
+          while value_cls.name.nil?
+            value_cls = value_cls.superclass
+          end
+          value_cls.name
+        end
+
         def custom_display_string(value)
           case value
           when File
@@ -77,6 +86,7 @@ module AppMap
 
       class << self
         def build_from_invocation(mc = MethodCall.new, defined_class, method, receiver, arguments)
+          defined_class ||= 'Class'
           mc.tap do
             static = receiver.is_a?(Module)
             mc.defined_class = defined_class
@@ -110,14 +120,14 @@ module AppMap
                 end
               {
                 name: param_name,
-                class: value.class.name,
+                class: best_class_name(value),
                 object_id: value.__id__,
                 value: display_string(value),
                 kind: param_type
               }
             end
             mc.receiver = {
-              class: receiver.class.name,
+              class: best_class_name(receiver),
               object_id: receiver.__id__,
               value: display_string(receiver)
             }
@@ -172,7 +182,7 @@ module AppMap
           mr.tap do |_|
             if return_value
               mr.return_value = {
-                class: return_value.class.name,
+                class: best_class_name(return_value),
                 value: display_string(return_value),
                 object_id: return_value.__id__
               }
@@ -183,7 +193,7 @@ module AppMap
               while next_exception
                 exception_backtrace = next_exception.backtrace_locations.try(:[], 0)
                 exceptions << {
-                  class: next_exception.class.name,
+                  class: best_class_name(next_exception),
                   message: next_exception.message,
                   object_id: next_exception.__id__,
                   path: exception_backtrace&.path,

data/lib/appmap/hook.rb

@@ -46,7 +46,16 @@ module AppMap
         cls = trace_point.self
 
         instance_methods = cls.public_instance_methods(false) - OBJECT_INSTANCE_METHODS
-        class_methods = cls.singleton_class.public_instance_methods(false) - instance_methods - OBJECT_STATIC_METHODS
+        # NoMethodError: private method `singleton_class' called for Rack::MiniProfiler:Class
+        class_methods = begin
+          if cls.respond_to?(:singleton_class)
+            cls.singleton_class.public_instance_methods(false) - instance_methods - OBJECT_STATIC_METHODS
+          else
+            []
+          end
+        rescue NameError
+          []
+        end
 
         hook = lambda do |hook_cls|
           lambda do |method_id|
@@ -63,6 +72,8 @@ module AppMap
             # Skip methods that have no instruction sequence, as they are obviously trivial.
             next unless disasm
 
+            next if config.never_hook?(method)
+
             next unless \
               config.always_hook?(hook_cls, method.name) ||
               config.included_by_location?(method)
@@ -78,12 +89,20 @@ module AppMap
         end
 
         instance_methods.each(&hook.(cls))
-        class_methods.each(&hook.(cls.singleton_class))
+        # NoMethodError: private method `singleton_class' called for Rack::MiniProfiler:Class
+        begin
+          class_methods.each(&hook.(cls.singleton_class)) if cls.respond_to?(:singleton_class)
+        rescue NameError
+          # NameError:
+          #   uninitialized constant Faraday::Connection
+        end
       end
 
       tp.enable(&block)
     end
 
+    # hook_builtins builds hooks for code that is built in to the Ruby standard library.
+    # No TracePoint events are emitted for builtins, so a separate hooking mechanism is needed. 
     def hook_builtins
       return unless self.class.lock_builtins
 
@@ -97,6 +116,7 @@ module AppMap
         require hook.package.package_name if hook.package.package_name
         Array(hook.method_names).each do |method_name|
           method_name = method_name.to_sym
+
           cls = class_from_string.(class_name)
           method = \
             begin
@@ -105,6 +125,8 @@ module AppMap
               cls.method(method_name) rescue nil
             end
 
+          next if config.never_hook?(method)
+
           if method
             Hook::Method.new(hook.package, cls, method).activate
           else

data/lib/appmap/hook/method.rb

@@ -35,7 +35,7 @@ module AppMap
                 else
                   "#{hook_method.name} (class resolution deferred)"
                 end
-          warn "AppMap: Hooking " + msg
+          warn "AppMap: Hooking #{msg} at line #{(hook_method.source_location || []).join(':')}"
         end
 
         defined_class = @defined_class

data/lib/appmap/middleware/remote_recording.rb

@@ -67,7 +67,7 @@ module AppMap
 
         response = JSON.generate \
           version: AppMap::APPMAP_FORMAT_VERSION,
-          classMap: AppMap.class_map(tracer.event_methods),
+          classMap: AppMap.class_map(tracer.event_methods, include_source: AppMap.include_source?),
           metadata: metadata,
           events: @events
 

data/lib/appmap/minitest.rb

@@ -7,20 +7,25 @@ module AppMap
   # be activated around each test.
   module Minitest
     APPMAP_OUTPUT_DIR = 'tmp/appmap/minitest'
-    LOG = false
+    LOG = ( ENV['APPMAP_DEBUG'] == 'true' || ENV['DEBUG'] == 'true' )
 
     def self.metadata
       AppMap.detect_metadata
     end
 
-    Recording = Struct.new(:test) do
-      def initialize(test)
+    Recording = Struct.new(:test, :test_name) do
+      def initialize(test, test_name)
         super
 
-        warn "Starting recording of test #{test.class}.#{test.name}" if AppMap::Minitest::LOG
+        warn "Starting recording of test #{test.class}.#{test.name}@#{source_location}" if AppMap::Minitest::LOG
         @trace = AppMap.tracing.trace
       end
 
+      def source_location
+        test.method(test_name).source_location.join(':')
+      end
+
+
       def finish
         warn "Finishing recording of test #{test.class}.#{test.name}" if AppMap::Minitest::LOG
 
@@ -31,7 +36,7 @@ module AppMap
 
         AppMap::Minitest.add_event_methods @trace.event_methods
 
-        class_map = AppMap.class_map(@trace.event_methods)
+        class_map = AppMap.class_map(@trace.event_methods, include_source: AppMap.include_source?)
 
         feature_group = test.class.name.underscore.split('_')[0...-1].join('_').capitalize
         feature_name = test.name.split('_')[1..-1].join(' ')
@@ -39,9 +44,8 @@ module AppMap
 
         AppMap::Minitest.save scenario_name,
                               class_map,
-                              events: events,
-                              feature_name: feature_name,
-                              feature_group_name: feature_group
+                              source_location,
+                              events: events
       end
     end
 
@@ -55,8 +59,8 @@ module AppMap
         FileUtils.mkdir_p APPMAP_OUTPUT_DIR
       end
 
-      def begin_test(test)
-        @recordings_by_test[test.object_id] = Recording.new(test)
+      def begin_test(test, name)
+        @recordings_by_test[test.object_id] = Recording.new(test, name)
       end
 
       def end_test(test)
@@ -74,12 +78,11 @@ module AppMap
         @event_methods += event_methods
       end
 
-      def save(example_name, class_map, events: nil, feature_name: nil, feature_group_name: nil, labels: nil)
+      def save(example_name, class_map, source_location, events: nil, labels: nil)
         metadata = AppMap::Minitest.metadata.tap do |m|
           m[:name] = example_name
+          m[:source_location] = source_location
           m[:app] = AppMap.configuration.name
-          m[:feature] = feature_name if feature_name
-          m[:feature_group] = feature_group_name if feature_group_name
           m[:frameworks] ||= []
           m[:frameworks] << {
             name: 'minitest',
@@ -128,7 +131,7 @@ if AppMap::Minitest.enabled?
     alias run_without_hook run
 
     def run
-      AppMap::Minitest.begin_test self
+      AppMap::Minitest.begin_test self, name
       begin
         run_without_hook
       ensure

data/lib/appmap/rspec.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require 'appmap/util'
+require 'set'
 
 module AppMap
   # Integration of AppMap with RSpec. When enabled with APPMAP=true, the AppMap tracer will
@@ -13,58 +14,9 @@ module AppMap
       AppMap.detect_metadata
     end
 
-    module FeatureAnnotations
-      def feature
-        return nil unless annotations
-
-        annotations[:feature]
-      end
-
-      def labels
-        labels = metadata[:appmap]
-        if labels.is_a?(Array)
-          labels
-        elsif labels.is_a?(String) || labels.is_a?(Symbol)
-          [ labels ]
-        else
-          []
-        end
-      end
-
-      def feature_group
-        return nil unless annotations
-
-        annotations[:feature_group]
-      end
-
-      def annotations
-        metadata.tap do |md|
-          description_args_hashes.each do |h|
-            md.merge! h
-          end
-        end
-      end
-
-      protected
-
-      def metadata
-        return {} unless example_obj.respond_to?(:metadata)
-
-        example_obj.metadata
-      end
-
-      def description_args_hashes
-        return [] unless example_obj.respond_to?(:metadata)
-
-        (example_obj.metadata[:description_args] || []).select { |arg| arg.is_a?(Hash) }
-      end
-    end
-
     # ScopeExample and ScopeExampleGroup is a way to handle the weird way that RSpec
     # stores the nested example names.
     ScopeExample = Struct.new(:example) do
-      include FeatureAnnotations
-
       alias_method :example_obj, :example
 
       def description?
@@ -83,8 +35,6 @@ module AppMap
     # As you can see here, the way that RSpec stores the example description and
     # represents the example group hierarchy is pretty weird.
     ScopeExampleGroup = Struct.new(:example_group) do
-      include FeatureAnnotations
-
       alias_method :example_obj, :example_group
 
       def description_args
@@ -133,11 +83,17 @@ module AppMap
           page.driver.options[:http_client].instance_variable_get('@server_url').port
         end
 
-        warn "Starting recording of example #{example}" if AppMap::RSpec::LOG
+        warn "Starting recording of example #{example}@#{source_location}" if AppMap::RSpec::LOG
         @trace = AppMap.tracing.trace
         @webdriver_port = webdriver_port.()
       end
 
+      def source_location
+        result = example.location_rerun_argument.split(':')[0]
+        result = result[2..-1] if result.index('./') == 0
+        result
+      end
+
       def finish
         warn "Finishing recording of example #{example}" if AppMap::RSpec::LOG
 
@@ -148,22 +104,16 @@ module AppMap
 
         AppMap::RSpec.add_event_methods @trace.event_methods
 
-        class_map = AppMap.class_map(@trace.event_methods, include_source: false)
+        class_map = AppMap.class_map(@trace.event_methods, include_source: AppMap.include_source?)
 
         description = []
         scope = ScopeExample.new(example)
-        feature_group = feature = nil
 
-        labels = []
         while scope
-          labels += scope.labels
           description << scope.description
-          feature ||= scope.feature
-          feature_group ||= scope.feature_group
           scope = scope.parent
         end
 
-        labels = labels.map(&:to_s).map(&:strip).reject(&:blank?).map(&:downcase).uniq
         description.reject!(&:nil?).reject!(&:blank?)
         default_description = description.last
         description.reverse!
@@ -177,24 +127,10 @@ module AppMap
 
         full_description = normalize.call(description.join(' '))
 
-        compute_feature_name = lambda do
-          return 'unknown' if description.empty?
-
-          feature_description = description.dup
-          num_tokens = [2, feature_description.length - 1].min
-          feature_description[0...num_tokens].map(&:strip).join(' ')
-        end
-
-        feature_group ||= normalize.call(default_description).underscore.gsub('/', '_').humanize
-        feature_name = feature || compute_feature_name.call if feature_group
-        feature_name = normalize.call(feature_name) if feature_name
-
         AppMap::RSpec.save full_description,
                            class_map,
-                           events: events,
-                           feature_name: feature_name,
-                           feature_group_name: feature_group,
-                           labels: labels.blank? ? nil : labels
+                           source_location,
+                           events: events
       end
     end
 
@@ -227,12 +163,11 @@ module AppMap
         @event_methods += event_methods
       end
 
-      def save(example_name, class_map, events: nil, feature_name: nil, feature_group_name: nil, labels: nil)
+      def save(example_name, class_map, source_location, events: nil, labels: nil)
         metadata = AppMap::RSpec.metadata.tap do |m|
           m[:name] = example_name
+          m[:source_location] = source_location
           m[:app] = AppMap.configuration.name
-          m[:feature] = feature_name if feature_name
-          m[:feature_group] = feature_group_name if feature_group_name
           m[:labels] = labels if labels
           m[:frameworks] ||= []
           m[:frameworks] << {

data/lib/appmap/version.rb

@@ -3,7 +3,7 @@
 module AppMap
   URL = 'https://github.com/applandinc/appmap-ruby'
 
-  VERSION = '0.40.0'
+  VERSION = '0.42.1'
 
   APPMAP_FORMAT_VERSION = '1.4'
 end

data/spec/abstract_controller_base_spec.rb

@@ -8,7 +8,7 @@ describe 'AbstractControllerBase' do
         FileUtils.rm_rf tmpdir
         FileUtils.mkdir_p tmpdir
         cmd = <<~CMD.gsub "\n", ' '
-          docker-compose run --rm -e APPMAP=true
+          docker-compose run --rm -e RAILS_ENV=test -e APPMAP=true
           -v #{File.absolute_path tmpdir}:/app/tmp app ./bin/rspec #{spec_name}
         CMD
         run_cmd cmd, chdir: fixture_dir
@@ -137,7 +137,7 @@ describe 'AbstractControllerBase' do
                   'name' => 'Renderer',
                   'children' => include(hash_including(
                     'name' => 'render',
-                    'labels' => ['view']
+                    'labels' => ['mvc.view']
                   ))
                 ))
               ))

data/spec/config_spec.rb

@@ -7,6 +7,7 @@ require 'appmap/config'
 describe AppMap::Config, docker: false do
   it 'loads from a Hash' do
     config_data = {
+      exclude: [],
       name: 'test',
       packages: [
         {

data/spec/fixtures/hook/exclude.rb

@@ -0,0 +1,15 @@
+class ExcludeTest
+  def instance_method
+    'instance_method'
+  end
+
+  class << self
+    def singleton_method
+      'singleton_method'
+    end
+  end
+
+  def self.cls_method
+    'class_method'
+  end
+end

data/spec/fixtures/rails5_users_app/appmap.yml

@@ -1,4 +1,7 @@
 name: rails5_users_app
 packages:
-  - path: app
+  - path: app/models
+    labels: [ mvc.model ]
+  - path: app/controllers
+    labels: [ mvc.controller ]
   - gem: sequel

data/spec/fixtures/rails5_users_app/docker-compose.yml

@@ -19,6 +19,9 @@ services:
     environment:
       RAILS_ENV:
       ORM_MODULE:
+      PGHOST: pg
+      PGPORT: '5432'
+      DATABASE_URL: postgres://postgres@pg
       APPMAP:
     volumes:
     - .:/src/app

data/spec/fixtures/rails5_users_app/spec/controllers/users_controller_api_spec.rb

@@ -1,8 +1,8 @@
 require 'rails_helper'
 require 'rack/test'
 
-RSpec.describe Api::UsersController, feature_group: 'Users', type: :controller, appmap: true do
-  describe 'POST /api/users', feature: 'Create a user' do
+RSpec.describe Api::UsersController, type: :controller do
+  describe 'POST /api/users' do
     describe 'with required parameters' do
       it 'creates a user' do
         post :create, params: { login: 'alice', password: 'foobar' }
@@ -16,7 +16,7 @@ RSpec.describe Api::UsersController, feature_group: 'Users', type: :controller,
       end
     end
   end
-  describe 'GET /api/users', feature: 'List users' do
+  describe 'GET /api/users' do
     before do
       post :create, params: { login: 'alice' }
     end

data/spec/fixtures/rails5_users_app/spec/models/user_spec.rb

@@ -1,6 +1,6 @@
 require 'rails_helper'
 
-describe User, feature_group: 'User', appmap: true do
+describe User do
   # TODO: appmap/rspec doesn't handle shared_examples_for 100% correctly yet.
   # In my tests, only one of these two tests will be emitted as an appmap.
   shared_examples_for 'creates the user' do |username|
@@ -11,17 +11,7 @@ describe User, feature_group: 'User', appmap: true do
     end
   end
 
-  describe 'creation', feature: 'Create a user' do
-    context 'using shared_examples_for' do
-      # AppMap.
-#      context "with username 'alice'" do
-#        it_should_behave_like 'creates the user', 'alice'
-#      end
-#      context "with username 'bob'" do
-#        it_should_behave_like 'creates the user', 'bob'
-#      end
-    end
-
+  describe 'creation' do
     # So, instead of shared_examples_for, let's go with a simple method
     # containing the assertions. The method can be called from within an example.
     def save_and_verify

data/spec/fixtures/rails6_users_app/appmap.yml

@@ -1,5 +1,8 @@
 name: rails6_users_app
 packages:
-  - path: app
+  - path: app/models
+    labels: [ mvc.model ]
+  - path: app/controllers
+    labels: [ mvc.controller ]
   - gem: sequel
 

data/spec/fixtures/rails6_users_app/docker-compose.yml

@@ -19,6 +19,9 @@ services:
     environment:
       RAILS_ENV:
       ORM_MODULE:
+      PGHOST: pg
+      PGPORT: '5432'
+      DATABASE_URL: postgres://postgres@pg
       APPMAP:
     volumes:
     - .:/src/app

data/spec/fixtures/rails6_users_app/spec/controllers/users_controller_api_spec.rb

@@ -1,8 +1,8 @@
 require 'rails_helper'
 require 'rack/test'
 
-RSpec.describe Api::UsersController, feature_group: 'Users', type: :controller, appmap: true do
-  describe 'POST /api/users', feature: 'Create a user' do
+RSpec.describe Api::UsersController, type: :controller do
+  describe 'POST /api/users' do
     describe 'with required parameters' do
       it 'creates a user' do
         post :create, params: { login: 'alice', password: 'foobar' }
@@ -16,7 +16,7 @@ RSpec.describe Api::UsersController, feature_group: 'Users', type: :controller,
       end
     end
   end
-  describe 'GET /api/users', feature: 'List users' do
+  describe 'GET /api/users' do
     before do
       post :create, params: { login: 'alice' }
     end

data/spec/fixtures/rails6_users_app/spec/models/user_spec.rb

@@ -1,6 +1,6 @@
 require 'rails_helper'
 
-describe User, feature_group: 'User', appmap: true do
+describe User  do
   # TODO: appmap/rspec doesn't handle shared_examples_for 100% correctly yet.
   # In my tests, only one of these two tests will be emitted as an appmap.
   shared_examples_for 'creates the user' do |username|
@@ -11,17 +11,7 @@ describe User, feature_group: 'User', appmap: true do
     end
   end
 
-  describe 'creation', feature: 'Create a user' do
-    context 'using shared_examples_for' do
-      # AppMap.
-#      context "with username 'alice'" do
-#        it_should_behave_like 'creates the user', 'alice'
-#      end
-#      context "with username 'bob'" do
-#        it_should_behave_like 'creates the user', 'bob'
-#      end
-    end
-
+  describe 'creation' do
     # So, instead of shared_examples_for, let's go with a simple method
     # containing the assertions. The method can be called from within an example.
     def save_and_verify

data/spec/hook_spec.rb

@@ -61,6 +61,16 @@ describe 'AppMap class Hooking', docker: false do
     AppMap.configuration = nil
   end
 
+  it 'excludes named classes and methods' do
+    load 'spec/fixtures/hook/exclude.rb'
+    package = AppMap::Config::Package.build_from_path('spec/fixtures/hook/exclude.rb')
+    config = AppMap::Config.new('hook_spec', [ package ], %w[ExcludeTest])
+    AppMap.configuration = config
+
+    expect(config.never_hook?(ExcludeTest.new.method(:instance_method))).to be_truthy
+    expect(config.never_hook?(ExcludeTest.method(:cls_method))).to be_truthy
+  end
+
   it 'parses labels from comments' do
     _, tracer = invoke_test_file 'spec/fixtures/hook/labels.rb' do
       ClassWithLabel.new.fn_with_label
@@ -827,7 +837,7 @@ describe 'AppMap class Hooking', docker: false do
       entry = cm[1][:children][0][:children][0][:children][0]
       # Sanity check, make sure we got the right one
       expect(entry[:name]).to eq('secure_compare')
-      expect(entry[:labels]).to eq(%w[security crypto])
+      expect(entry[:labels]).to eq(%w[provider.secure_compare])
     end
   end
 

data/spec/record_sql_rails_pg_spec.rb

@@ -5,7 +5,7 @@ describe 'SQL events' do
     around(:each) do |example|
       FileUtils.rm_rf tmpdir
       FileUtils.mkdir_p tmpdir
-      cmd = "docker-compose run --rm -e ORM_MODULE=#{orm_module} -e APPMAP=true -v #{File.absolute_path tmpdir}:/app/tmp app ./bin/rspec spec/controllers/users_controller_api_spec.rb:#{test_line_number}"
+      cmd = "docker-compose run --rm -e ORM_MODULE=#{orm_module} -e RAILS_ENV=test -e APPMAP=true -v #{File.absolute_path tmpdir}:/app/tmp app ./bin/rspec spec/controllers/users_controller_api_spec.rb:#{test_line_number}"
       run_cmd cmd, chdir: fixture_dir
 
       example.run

data/test/fixtures/gem_test/appmap.yml

@@ -1,3 +1,3 @@
 name: gem_test
 packages:
-  - gem: activesupport
+  - gem: parser

data/test/fixtures/gem_test/test/parser_test.rb

@@ -0,0 +1,12 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'appmap/minitest'
+require 'minitest/autorun'
+require 'parser/current'
+
+class ParserTest < ::Minitest::Test
+  def test_parser
+    Parser::CurrentRuby.parse(File.read(__FILE__))
+  end
+end

data/test/fixtures/rspec_recorder/spec/decorated_hello_spec.rb

@@ -2,7 +2,7 @@ require 'rspec'
 require 'appmap/rspec'
 require 'hello'
 
-describe Hello, feature_group: 'Saying hello' do
+describe Hello do
   before do
     # Trick appmap-ruby into thinking we're a Rails app.
     stub_const('Rails', double('rails', version: 'fake.0'))
@@ -11,11 +11,11 @@ describe Hello, feature_group: 'Saying hello' do
   # The order of these examples is important. The tests check the
   # appmap for 'says hello', and we want another example to get run
   # before it.
-  it 'does not say goodbye', feature: 'Speak hello', appmap: true do
+  it 'does not say goodbye' do
     expect(Hello.new.say_hello).not_to eq('Goodbye!')
   end
 
-  it 'says hello', feature: 'Speak hello', appmap: true do
+  it 'says hello' do
     expect(Hello.new.say_hello).to eq('Hello!')
   end
 end

data/test/fixtures/rspec_recorder/spec/plain_hello_spec.rb

@@ -2,7 +2,7 @@ require 'rspec'
 require 'appmap/rspec'
 require 'hello'
 
-describe Hello, appmap: true do
+describe Hello do
   it 'says hello' do
     expect(Hello.new.say_hello).to eq('Hello!')
   end

data/test/gem_test.rb

@@ -19,14 +19,14 @@ class MinitestTest < Minitest::Test
   end
 
   def test_record_gem
-    perform_gem_test 'to_param' do
-      appmap_file = 'tmp/appmap/minitest/To_param_to_param.appmap.json'
+    perform_gem_test 'parser' do
+      appmap_file = 'tmp/appmap/minitest/Parser_parser.appmap.json'
       appmap = JSON.parse(File.read(appmap_file))
       events = appmap['events']
       assert_equal 2, events.size
       assert_equal 'call', events.first['event']
-      assert_equal 'to_param', events.first['method_id']
-      assert_equal "#{Gem.loaded_specs['activesupport'].gem_dir}/lib/active_support/core_ext/object/to_query.rb", events.first['path']
+      assert_equal 'default_parser', events.first['method_id']
+      assert_equal "#{Gem.loaded_specs['parser'].gem_dir}/lib/parser/base.rb", events.first['path']
       assert_equal 'return', events.second['event']
       assert_equal 1, events.second['parent_id']
     end

data/test/minitest_test.rb

@@ -30,9 +30,8 @@ class MinitestTest < Minitest::Test
       assert_equal 'minitest_recorder', metadata['app']
       assert_equal 'minitest', metadata['recorder']['name']
       assert_equal 'ruby', metadata['language']['name']
-      assert_equal 'Hello', metadata['feature_group']
-      assert_equal 'hello', metadata['feature']
       assert_equal 'Hello hello', metadata['name']
+      assert_equal 'test/hello_test.rb:9', metadata['source_location']
     end
   end
 end

data/test/rspec_test.rb

@@ -28,8 +28,6 @@ class RSpecTest < Minitest::Test
       assert_includes appmap.keys, 'metadata'
       metadata = appmap['metadata']
       assert_equal 'Inventory', metadata['name']
-      assert_includes metadata.keys, 'labels'
-      assert_equal metadata['labels'], %w[inventory]
     end
   end
 
@@ -42,9 +40,8 @@ class RSpecTest < Minitest::Test
       assert_equal AppMap::APPMAP_FORMAT_VERSION, appmap['version']
       assert_includes appmap.keys, 'metadata'
       metadata = appmap['metadata']
-      assert_equal 'Saying hello', metadata['feature_group']
-      assert_equal 'Speak hello', metadata['feature']
       assert_equal 'Hello says hello', metadata['name']
+      assert_equal 'spec/decorated_hello_spec.rb', metadata['source_location']
       assert_includes metadata.keys, 'client'
       assert_equal({ name: 'appmap', url: AppMap::URL, version: AppMap::VERSION }.stringify_keys, metadata['client'])
       assert_includes metadata.keys, 'recorder'
@@ -63,8 +60,6 @@ class RSpecTest < Minitest::Test
       appmap = JSON.parse(File.read(appmap_file))
       assert_includes appmap.keys, 'metadata'
       metadata = appmap['metadata']
-      assert_equal 'Hello', metadata['feature_group']
-      assert_equal 'Hello', metadata['feature']
       assert_equal 'Hello says hello', metadata['name']
     end
   end
@@ -76,7 +71,6 @@ class RSpecTest < Minitest::Test
       appmap = JSON.parse(File.read(appmap_file))
       assert_includes appmap.keys, 'metadata'
       metadata = appmap['metadata']
-      assert_equal %w[hello speak], metadata['labels'].sort
     end
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: appmap
 version: !ruby/object:Gem::Version
-  version: 0.40.0
+  version: 0.42.1
 platform: ruby
 authors:
 - Kevin Gilpin
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-01-29 00:00:00.000000000 Z
+date: 2021-03-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -386,6 +386,7 @@ files:
 - spec/fixtures/hook/compare.rb
 - spec/fixtures/hook/constructor.rb
 - spec/fixtures/hook/exception_method.rb
+- spec/fixtures/hook/exclude.rb
 - spec/fixtures/hook/instance_method.rb
 - spec/fixtures/hook/labels.rb
 - spec/fixtures/hook/singleton_method.rb
@@ -547,7 +548,6 @@ files:
 - spec/railtie_spec.rb
 - spec/record_sql_rails_pg_spec.rb
 - spec/remote_recording_spec.rb
-- spec/rspec_feature_metadata_spec.rb
 - spec/spec_helper.rb
 - spec/util_spec.rb
 - test/cli_test.rb
@@ -572,7 +572,7 @@ files:
 - test/fixtures/cucumber_recorder/lib/hello.rb
 - test/fixtures/gem_test/Gemfile
 - test/fixtures/gem_test/appmap.yml
-- test/fixtures/gem_test/test/to_param_test.rb
+- test/fixtures/gem_test/test/parser_test.rb
 - test/fixtures/minitest_recorder/Gemfile
 - test/fixtures/minitest_recorder/appmap.yml
 - test/fixtures/minitest_recorder/lib/hello.rb

data/spec/rspec_feature_metadata_spec.rb

@@ -1,31 +0,0 @@
-# frozen_string_literal: true
-
-require 'rails_spec_helper'
-
-describe 'RSpec feature and feature group metadata' do
-  include_examples 'Rails app pg database', 'spec/fixtures/rails5_users_app' do
-    around(:each) do |example|
-      FileUtils.rm_rf tmpdir
-      FileUtils.mkdir_p tmpdir
-      cmd = "docker-compose run --rm -e APPMAP=true -v #{File.absolute_path(tmpdir).shellescape}:/app/tmp app ./bin/rspec spec/models/user_spec.rb"
-      run_cmd cmd, chdir: fixture_dir
-
-      example.run
-    end
-
-    let(:tmpdir) { 'tmp/spec/RSpec feature and feature group metadata' }
-    let(:appmap_json) { File.join(tmpdir, %(appmap/rspec/User_creation_creates_charles.appmap.json)) }
-
-    describe do
-      it 'are recorded in the appmap' do
-        expect(File).to exist(appmap_json)
-        appmap = JSON.parse(File.read(appmap_json)).to_yaml
-
-        expect(appmap).to include(<<-METADATA.strip)
-  feature: Create a user
-  feature_group: User
-        METADATA
-      end
-    end
-  end
-end

data/test/fixtures/gem_test/test/to_param_test.rb

@@ -1,14 +0,0 @@
-#!/usr/bin/env ruby
-# frozen_string_literal: true
-
-require 'appmap/minitest'
-require 'minitest/autorun'
-require 'active_support'
-require 'active_support/core_ext'
-
-class ToParamTest < ::Minitest::Test
-  def test_to_param
-    # record use of a core extension
-    assert_equal 'my+id', 'my+id'.to_param
-  end
-end