appmap 0.47.0 → 0.49.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f1f2b43daf0437ceb6073a64272148ca9168c4ac781c01e2e73c36259a3bcc7b
-  data.tar.gz: 2a7275d2be15d2bfcf0c2dfc898807212840506c3692232b2d049bc771a4a3b9
+  metadata.gz: '02814a0f9d0927d19c2d341a22d8f8ec66b83bc022833a09a7777ce81ec4edfc'
+  data.tar.gz: 11b2786d88e360fc78e046675ae799e7106625dd02b4a3070c3835bfe8feea05
 SHA512:
-  metadata.gz: 5000bdc938facaa5e181de6e3d6023c6a1b1a1e48a260545ae3245236bf271e22ab6462c16d1c73fb44b4e9bc4e6533f8be12428e2d7ff9e1c06f65ec1992bf7
-  data.tar.gz: 10db3cd34b1f9eb5f81070607d12b6a714b12735ac25828df4c6a3e50d0efa354ad9ff9206bc42cd2d7ccb61ffef83f663d41b3896928f0fd9bdd1e53ed4e543
+  metadata.gz: c852ae464d52f9f29ef9222fa1627f99b4f5baf1907b1cd54cbfb969ff2323cb19e0d5b53bad1356518ec21a07243ea71237ad831a62ffe825a2601cca1a5cf7
+  data.tar.gz: 49c5ba0d0c20d8bee6e4c768544ab6abe0f0f0d71a1a59bc04fcf867e93bc743eda4cc94f7505c0a1194c07e95e2b8e7f875a42b16aa3da2a21068a61ba42a10

data/.dockerignore

@@ -2,4 +2,3 @@ vendor
 node_modules
 spec/fixtures/rails*_users_app
 spec/fixtures/rack_users_app
-

data/CHANGELOG.md

@@ -1,3 +1,40 @@
+# [0.49.0](https://github.com/applandinc/appmap-ruby/compare/v0.48.2...v0.49.0) (2021-06-16)
+
+
+### Features
+
+* Add refinement to the labels ([6a93396](https://github.com/applandinc/appmap-ruby/commit/6a93396ba73f1b3ed21b4e9e15a2c271af04d866))
+
+## [0.48.2](https://github.com/applandinc/appmap-ruby/compare/v0.48.1...v0.48.2) (2021-05-26)
+
+
+### Bug Fixes
+
+* Correct the method-hooking logic to capture some missing model methods ([be529bd](https://github.com/applandinc/appmap-ruby/commit/be529bdce7d4fdf9f1a2fdd32259d792f29f4f13))
+
+## [0.48.1](https://github.com/applandinc/appmap-ruby/compare/v0.48.0...v0.48.1) (2021-05-25)
+
+
+### Bug Fixes
+
+* Account for bundle path when normalizing source path ([095c278](https://github.com/applandinc/appmap-ruby/commit/095c27818fc8ae8dfa39b30516d37c6dfd642d9c))
+* Scan exception messages for non-UTF8 characters ([3dcaeae](https://github.com/applandinc/appmap-ruby/commit/3dcaeae44da5e40e432eda41caf5b9ebff5bea12))
+
+# [0.48.0](https://github.com/applandinc/appmap-ruby/compare/v0.47.1...v0.48.0) (2021-05-19)
+
+
+### Features
+
+* Hook the code only when APPMAP=true ([dd9e383](https://github.com/applandinc/appmap-ruby/commit/dd9e383024d1d9205a617d46bd64b90820035533))
+* Remove server process recording from doc and tests ([383ba0a](https://github.com/applandinc/appmap-ruby/commit/383ba0ad444922a0a85409477d11bc7ed06a9160))
+
+## [0.47.1](https://github.com/applandinc/appmap-ruby/compare/v0.47.0...v0.47.1) (2021-05-13)
+
+
+### Bug Fixes
+
+* Add the proper template function hooks for Rails 6.0.7 ([175f489](https://github.com/applandinc/appmap-ruby/commit/175f489acbaed77ad52a18d805e4b6eeae1abfdb))
+
 # [0.47.0](https://github.com/applandinc/appmap-ruby/compare/v0.46.0...v0.47.0) (2021-05-13)
 
 

data/README.md

@@ -1,17 +1,6 @@
 
 - [About](#about)
-    - [Supported versions](#supported-versions)
-- [Installation](#installation)
-- [Configuration](#configuration)
-- [Labels](#labels)
-- [Running](#running)
-  - [RSpec](#rspec)
-  - [Minitest](#minitest)
-  - [Cucumber](#cucumber)
-  - [Remote recording](#remote-recording)
-- [AppMap for VSCode](#appmap-for-vscode)
-- [AppMap Swagger](#appmap-swagger)
-- [Uploading AppMaps](#uploading-appmaps)
+- [Usage](#usage)
 - [Development](#development)
   - [Internal architecture](#internal-architecture)
   - [Running tests](#running-tests)
@@ -19,7 +8,6 @@
     - [`test/fixtures`](#testfixtures)
     - [`spec/fixtures`](#specfixtures)
 
-
 # About
 
 `appmap-ruby` is a Ruby Gem for recording
@@ -29,342 +17,9 @@
 SHA, labels, etc). It's more granular than a performance profile, but it's less
 granular than a full debug trace. It's designed to be optimal for understanding the design intent and structure of code and key data flows.
 
-There are several ways to record AppMaps of your Ruby program using the `appmap` gem:
-
-* Run your tests (RSpec, Minitest, Cucumber) with the environment variable `APPMAP=true`. An AppMap will be generated for each spec.
-* Run your application server with AppMap remote recording enabled, and use the [AppLand
-  browser extension](https://github.com/applandinc/appland-browser-extension) to start,
-  stop, and upload recordings.
-* Wrap some code in an `AppMap.record` block, which returns JSON containing the code execution trace.
-
-Once you have made a recording, there are two ways to view automatically generated diagrams of the AppMaps.
-
-The first option is to load the diagrams directly in your IDE, using the [AppMap extension for VSCode](https://marketplace.visualstudio.com/items?itemName=appland.appmap).
-
-The second option is to upload them to the [AppLand server](https://app.land) using the [AppLand CLI](https://github.com/applandinc/appland-cli/releases).
-
-### Supported versions
-
-* Ruby 2.5, 2.6, 2.7
-* Rails 5, 6
-
-Support for new versions is added frequently, please check back regularly for updates.
-
-# Installation
-
-<a href="https://www.loom.com/share/78ab32a312ff4b85aa8827a37f1cb655"> <p>Quick and easy setup of the AppMap gem for Rails - Watch Video</p> <img style="max-width:300px;" src="https://cdn.loom.com/sessions/thumbnails/78ab32a312ff4b85aa8827a37f1cb655-with-play.gif"> </a>
-
-
-Add `gem 'appmap'` to **beginning** of your Gemfile. We recommend that you add the `appmap` gem to the `:development, :test` group. Your Gemfile should look something like this:
-
-```
-source 'https://rubygems.org'
-git_source(:github) { |repo| "https://github.com/#{repo}.git" }
-
-# Optional rubRuby version
-# ruby '2.7.2'
-
-group :development, :test do
-  gem 'appmap'
-end
-```
-
-Install with `bundle install`, as usual.
-
-It's important to add the `appmap` gem before any other gems that you may want to instrument. There is more about this in the section on adding gems to the *appmap.yml*.
-
-**Railtie**
-
-If you are using Ruby on Rails, require the railtie after Rails is loaded. 
-
-```
-# application.rb is a good place to do this, along with all the other railties.
-# Don't require the railtie in environments that don't bundle the appmap gem.
-require 'appmap/railtie' if defined?(AppMap).
-```
-
-**application.rb**
-
-Add this line to *application.rb*, to enable server recording with `APPMAP_RECORD=true`:
-
-```ruby
-module MyApp
-  class Application < Rails::Application
-    ...
-  
-    config.appmap.enabled = true if ENV['APPMAP_RECORD']
-    
-    ...
-  end
-end
-```
-
-# Configuration
-
-When you run your program, the `appmap` gem reads configuration settings from `appmap.yml`. Here's a sample configuration
-file for a typical Rails project:
-
-```yaml
-# 'name' should generally be the same as the code repo name.
-name: my_project
-packages:
-- path: app/controllers
-- path: app/models
-  # Exclude sub-paths within the package path
-  exclude:
-  - concerns/accessor
-- path: app/jobs
-- path: app/helpers
-# Include the gems that you want to see in the dependency maps.
-# These are just examples.
-- gem: activerecord
-- gem: devise
-- gem: aws-sdk
-- gem: will_paginate
-# Global exclusion of a class name
-exclude:
-- MyClass
-- MyClass#my_instance_method
-- MyClass.my_class_method
-functions:
-- packages: myapp
-  class: ControllerHelper
-  function: logged_in_user
-  labels: [ authentication ]
-```
-
-* **name** Provides the project name (required)
-* **packages** A list of source code directories which should be recorded.
-* **exclude** A list of classes and/or methods to definitively exclude from recording.
-* **functions** A list of specific functions, scoped by package and class, to record.
-
-**packages**
-
-Each entry in the `packages` list is a YAML object which has the following keys:
-
-* **path** The path to the source code directory. The path may be relative to the current working directory, or it may
-  be an absolute path.
-* **gem** As an alternative to specifying the path, specify the name of a dependency gem. When using `gem`, don't specify `path`. In your `Gemfile`, the `appmap` gem **must** be listed **before** any gem that you specify in your *appmap.yml*.
-* **exclude** A list of files and directories which will be ignored. By default, all modules, classes and public
-  functions are inspected. See also: global `exclude` list.
-* **shallow** When set to `true`, only the first function call entry into a package will be recorded. Subsequent function calls within 
-  the same package are not recorded unless code execution leaves the package and re-enters it. Default: `true` when using `gem`,
-  `false` when using `path`.
-
-**exclude**
-
-Optional list of fully qualified class and method names. Separate class and method names with period (`.`) for class methods and hash (`#`) for instance methods.
-
-**functions**
-
-Optional list of `class, function` pairs. The `package` name is used to place the function within the class map, and does not have to match
-the folder or gem name. The primary use of `functions` is to apply specific labels to functions whose source code is not accessible (e.g., it's in a Gem).
-For functions which are part of the application code, use `@label` or `@labels` in code comments to apply labels.
-
-# Labels
-
-The [AppMap data format](https://github.com/applandinc/appmap) provides for class and function `labels`, which can be used to enhance the AppMap visualizations, and to programatically analyze the data.
-
-You can apply function labels using source code comments in your Ruby code. To apply a labels to a function, add a `@label` or `@labels` line to the comment which immediately precedes a function.
-
-For example, if you add this comment to your source code:
-
-```ruby
-class ApiKey
-  # @labels provider.authentication security
-  def authenticate(key)
-    # logic to verify the key here...
-  end
-end
-```
-
-Then the AppMap metadata section for this function will include:
-
-```json
-  {
-    "name": "authenticate",
-    "type": "function",
-    "labels": [ "provider.authentication", "security" ]
-  }
-```
-
-
-# Running
-
-## RSpec
-
-To record RSpec tests, follow these additional steps:
-
-1) Require `appmap/rspec` in your `spec_helper.rb` before any other classes are loaded.
-
-```ruby
-require 'appmap/rspec'
-```
-
-Note that `spec_helper.rb` in a Rails project typically loads the application's classes this way:
-
-```ruby
-require File.expand_path("../../config/environment", __FILE__)
-```
-
-and `appmap/rspec` must be required before this:
-
-```ruby
-require 'appmap/rspec'
-require File.expand_path("../../config/environment", __FILE__)
-```
-
-2) Run the tests with the environment variable `APPMAP=true`:
-
-```sh-session
-$ APPMAP=true bundle exec rspec
-```
-
-Each RSpec test will output an AppMap file into the directory `tmp/appmap/rspec`. For example:
-
-```
-$ find tmp/appmap/rspec
-Hello_says_hello_when_prompted.appmap.json
-```
-
-## Minitest
-
-To record Minitest tests, follow these additional steps:
-
-1) Require `appmap/minitest` in `test_helper.rb`
-
-```ruby
-require 'appmap/minitest'
-```
-
-Note that `test_helper.rb` in a Rails project typically loads the application's classes this way:
-
-```ruby
-require_relative '../config/environment'
-```
-
-and `appmap/minitest` must be required before this:
-
-```ruby
-require 'appmap/minitest'
-require_relative '../config/environment'
-```
-
-2) Run your tests as you normally would with the environment variable `APPMAP=true`. For example:  
-
-```
-$ APPMAP=true bundle exec rake test
-```
-
-or
-
-```
-$ APPMAP=true bundle exec ruby -Ilib -Itest test/*_test.rb
-```
-
-Each Minitest test will output an AppMap file into the directory `tmp/appmap/minitest`. For example:
-
-```
-$ find tmp/appmap/minitest
-Hello_says_hello_when_prompted.appmap.json
-```
-
-## Cucumber
-
-To record Cucumber tests, follow these additional steps:
-
-1) Require `appmap/cucumber` in `support/env.rb`:
-
-```ruby
-require 'appmap/cucumber'
-```
-
-Be sure to require it before `config/environment` is required.
-
-2) Create an `Around` hook in `support/hooks.rb` to record the scenario:
-
-
-```ruby
-if AppMap::Cucumber.enabled?
-  Around('not @appmap-disable') do |scenario, block|
-    appmap = AppMap.record do
-      block.call
-    end
-
-    AppMap::Cucumber.write_scenario(scenario, appmap)
-  end
-end
-```
-
-3) Run the tests with the environment variable `APPMAP=true`:
-
-```sh-session
-$ APPMAP=true bundle exec cucumber
-```
-
-Each Cucumber test will output an AppMap file into the directory `tmp/appmap/cucumber`. For example:
-
-```
-$ find tmp/appmap/cucumber
-Hello_Says_hello_when_prompted.appmap.json
-```
-
-## Remote recording
-
-To manually record ad-hoc AppMaps of your Ruby app, use AppMap remote recording.
-
-1. Add the AppMap remote recording middleware. For example, in `config/initializers/appmap_remote_recording.rb`:
-
-```ruby
-if defined?(AppMap)
-  require 'appmap/middleware/remote_recording'
-
-  Rails.application.config.middleware.insert_after \
-    Rails::Rack::Logger,
-    AppMap::Middleware::RemoteRecording
-end
-```
-
-2. (optional) Download and unpack the [AppLand browser extension](https://github.com/applandinc/appland-browser-extension). Install into Chrome using `chrome://extensions/`. Turn on "Developer Mode" and then load the extension using the "Load unpacked" button.
-
-3. Start your Rails application server, with `APPMAP_RECORD=true`. For example:
-
-```sh-session
-$ APPMAP_RECORD=true bundle exec rails server
-```
-
-4. Start the recording
-
-Option 1: Open the AppLand browser extension and push `Start`.
-Option 2: `curl -XPOST localhost:3000/_appmap/record` (be sure and get the port number right)
-
-5. Use your app. For example, perform a login flow, or run through a manual UI test.
-
-6. Finish the recording.
-
-Option 1: Open the AppLand browser extension and push `Stop`. The recording will be transferred to the AppLand website and opened in your browser.
-Option 2: `curl -XDELETE localhost:3000/_appmap/record > recording.appmap.json` - Saves the recording as a local file.
-
-
-# AppMap for VSCode
-
-The [AppMap extension for VSCode](https://marketplace.visualstudio.com/items?itemName=appland.appmap) helps you navigate your code more efficiently with interactive, accurate software architecture diagrams right in your IDE. In less than two minutes you can go from installing the AppMap extension to exploring maps of your code's architecture. AppMap helps you:
-
-* Onboard to code architecture, with no extra work for the team
-* Conduct code and design reviews using live and accurate data
-* Troubleshoot hard-to-understand bugs using a "top-down" approach.
-
-Each interactive diagram links directly to the source code, and the information is easy to share.
-
-# AppMap Swagger
-
-[appmap_swagger](https://github.com/applandinc/appmap_swagger-ruby) is a tool to generate Swagger files from AppMap data. With `appmap_swagger`, you can add Swagger to your Ruby or Ruby on Rails project, with no need to write or modify code. Use the Swagger UI to interact with your web services API as you build it, and use diffs of Swagger to perform code review of web service changes.
-
-# Uploading AppMaps
-
-[https://app.land](https://app.land) can be used to store, analyze, and share AppMaps.
+# Usage
 
-For instructions on uploading, see the documentation of the [AppLand CLI](https://github.com/applandinc/appland-cli).
+Visit the [AppMap for Ruby](https://appland.com/docs/reference/appmap-ruby.html) reference page on AppLand.com for a complete reference guide.
 
 # Development
 [![Build Status](https://travis-ci.com/applandinc/appmap-ruby.svg?branch=master)](https://travis-ci.com/applandinc/appmap-ruby)

data/lib/appmap.rb

@@ -98,4 +98,4 @@ module AppMap
 end
 
 require 'appmap/railtie' if defined?(::Rails::Railtie)
-AppMap.initialize unless ENV['APPMAP_INITIALIZE'] == 'false'
+AppMap.initialize if ENV['APPMAP'] == 'true'

data/lib/appmap/config.rb

@@ -85,18 +85,7 @@ module AppMap
       end
     end
 
-    Function = Struct.new(:package, :cls, :labels, :function_names) do # :nodoc:
-      def to_h
-        {
-          package: package,
-          class: cls,
-          labels: labels,
-          functions: function_names.map(&:to_sym)
-        }.compact
-      end
-    end
-    private_constant :Function
-
+    # Identifies specific methods within a package which should be hooked.
     class TargetMethods # :nodoc:
       attr_reader :method_names, :package
 
@@ -118,28 +107,93 @@ module AppMap
     end
     private_constant :TargetMethods
 
-    OPENSSL_PACKAGES = ->(labels) { Package.build_from_path('openssl', package_name: 'openssl', labels: labels) }
+    # Function represents a specific function configured for hooking by the +functions+
+    # entry in appmap.yml. When the Config is initialized, each Function is converted into
+    # a Package and TargetMethods. It's called a Function rather than a Method, because Function
+    # is the AppMap terminology.
+    Function = Struct.new(:package, :cls, :labels, :function_names) do # :nodoc:
+      def to_h
+        {
+          package: package,
+          class: cls,
+          labels: labels,
+          functions: function_names.map(&:to_sym)
+        }.compact
+      end
+    end
+    private_constant :Function
 
-    # Methods that should always be hooked, with their containing
-    # package and labels that should be applied to them.
-    HOOKED_METHODS = {
-      'ActionView::Renderer' => TargetMethods.new(:render, Package.build_from_gem('actionview', shallow: false, package_name: 'action_view', labels: %w[mvc.view], optional: true).tap do |package|
-        package.handler_class = AppMap::Handler::Rails::Template::RenderHandler if package
-      end),
-      'ActionView::Resolver' => TargetMethods.new(%i[find_all find_all_anywhere], Package.build_from_gem('actionview', shallow: false, package_name: 'action_view', labels: %w[mvc.template.resolver], optional: true).tap do |package|
-        package.handler_class = AppMap::Handler::Rails::Template::ResolverHandler if package
-      end),
-      'ActionDispatch::Request::Session' => TargetMethods.new(%i[destroy [] dig values []= clear update delete fetch merge], Package.build_from_gem('actionpack', shallow: false, package_name: 'action_dispatch', labels: %w[http.session], optional: true)),
-      'ActionDispatch::Cookies::CookieJar' => TargetMethods.new(%i[[]= clear update delete recycle], Package.build_from_gem('actionpack', shallow: false, package_name: 'action_dispatch', labels: %w[http.cookie], optional: true)),
-      'ActionDispatch::Cookies::EncryptedCookieJar' => TargetMethods.new(%i[[]=], Package.build_from_gem('actionpack', shallow: false, package_name: 'action_dispatch', labels: %w[http.cookie crypto.encrypt], optional: true)),
-      'CanCan::ControllerAdditions' => TargetMethods.new(%i[authorize! can? cannot?], Package.build_from_gem('cancancan', shallow: false, labels: %w[security.authorization], optional: true)),
-      'CanCan::Ability' => TargetMethods.new(%i[authorize!], Package.build_from_gem('cancancan', shallow: false, labels: %w[security.authorization], optional: true)),
-      'ActionController::Instrumentation' => [
-        TargetMethods.new(%i[process_action send_file send_data redirect_to], Package.build_from_gem('actionpack', shallow: false, package_name: 'action_controller', labels: %w[mvc.controller], optional: true))
-      ]
-    }.freeze
+    ClassTargetMethods = Struct.new(:cls, :target_methods) # :nodoc:
+    private_constant :ClassTargetMethods
+
+    MethodHook = Struct.new(:cls, :method_names, :labels) # :nodoc:
+    private_constant :MethodHook
+    
+    class << self
+      def package_hooks(gem_name, methods, handler_class: nil, package_name: nil)
+        Array(methods).map do |method|
+          package = Package.build_from_gem(gem_name, package_name: package_name, labels: method.labels, shallow: false, optional: true)
+          next unless package
+
+          package.handler_class = handler_class if handler_class
+          ClassTargetMethods.new(method.cls, TargetMethods.new(Array(method.method_names), package))
+        end.compact
+      end
 
-    BUILTIN_METHODS = {
+      def method_hook(cls, method_names, labels)
+        MethodHook.new(cls, method_names, labels)
+      end
+    end
+
+    # Hook well-known functions. When a function configured here is available in the bundle, it will be hooked with the
+    # predefined labels specified here. If any of these hooks are not desired, they can be disabled in the +exclude+ section
+    # of appmap.yml.
+    METHOD_HOOKS = [
+      package_hooks('actionview',
+        [
+          method_hook('ActionView::Renderer', :render, %w[mvc.view]),
+          method_hook('ActionView::TemplateRenderer', :render, %w[mvc.view]),
+          method_hook('ActionView::PartialRenderer', :render, %w[mvc.view])
+        ],
+        handler_class: AppMap::Handler::Rails::Template::RenderHandler,
+        package_name: 'action_view'
+      ),
+      package_hooks('actionview',
+        [
+          method_hook('ActionView::Resolver', %i[find_all find_all_anywhere], %w[mvc.template.resolver])
+        ],
+        handler_class: AppMap::Handler::Rails::Template::ResolverHandler,
+        package_name: 'action_view'
+      ),
+      package_hooks('actionpack',
+        [
+          method_hook('ActionDispatch::Request::Session', %i[[] dig values fetch], %w[http.session.read]),
+          method_hook('ActionDispatch::Request::Session', %i[destroy[]= clear update delete merge], %w[http.session.write]),
+          method_hook('ActionDispatch::Cookies::CookieJar', %i[[]= clear update delete recycle], %w[http.session.read]),
+          method_hook('ActionDispatch::Cookies::CookieJar', %i[[]= clear update delete recycle], %w[http.session.write]),
+          method_hook('ActionDispatch::Cookies::EncryptedCookieJar', %i[[]= clear update delete recycle], %w[http.cookie crypto.encrypt])
+        ],
+        package_name: 'action_dispatch'
+      ),
+      package_hooks('cancancan',
+        [
+          method_hook('CanCan::ControllerAdditions', %i[authorize! can? cannot?], %w[security.authorization]),
+          method_hook('CanCan::Ability', %i[authorize?], %w[security.authorization])
+        ]
+      ),
+      package_hooks('actionpack',
+        [
+          method_hook('ActionController::Instrumentation', %i[process_action send_file send_data redirect_to], %w[mvc.controller])
+        ],
+        package_name: 'action_controller'
+      )
+    ].flatten.freeze
+
+    OPENSSL_PACKAGES = ->(labels) { Package.build_from_path('openssl', package_name: 'openssl', labels: labels) }
+
+    # Hook functions which are builtin to Ruby. Because they are builtins, they may be loaded before appmap.
+    # Therefore, we can't rely on TracePoint to report the loading of this code.
+    BUILTIN_HOOKS = {
       'OpenSSL::PKey::PKey' => TargetMethods.new(:sign, OPENSSL_PACKAGES.(%w[crypto.pkey])),
       'OpenSSL::X509::Request' => TargetMethods.new(%i[sign verify], OPENSSL_PACKAGES.(%w[crypto.x509])),
       'OpenSSL::PKCS5' => TargetMethods.new(%i[pbkdf2_hmac_sha1 pbkdf2_hmac], OPENSSL_PACKAGES.(%w[crypto.pkcs5])),
@@ -161,31 +215,37 @@ module AppMap
       # This is happening: Method send_command not found on Net::IMAP
       # 'Net::IMAP' => TargetMethods.new(:send_command, Package.build_from_path('net/imap', package_name: 'net/imap', labels: %w[protocol.email.imap])),
       # 'Marshal' => TargetMethods.new(%i[dump load], Package.build_from_path('marshal', labels: %w[format.marshal])),
-      'Psych' => TargetMethods.new(%i[dump dump_stream load load_stream parse parse_stream], Package.build_from_path('yaml', package_name: 'psych', labels: %w[format.yaml])),
-      'JSON::Ext::Parser' => TargetMethods.new(:parse, Package.build_from_path('json', package_name: 'json', labels: %w[format.json])),
-      'JSON::Ext::Generator::State' => TargetMethods.new(:generate, Package.build_from_path('json', package_name: 'json', labels: %w[format.json])),
+      'Psych' => [
+        TargetMethods.new(%i[load load_stream parse parse_stream], Package.build_from_path('yaml', package_name: 'psych', labels: %w[format.yaml.parse])),
+        TargetMethods.new(%i[dump dump_stream], Package.build_from_path('yaml', package_name: 'psych', labels: %w[format.yaml.generate])),
+      ],
+      'JSON::Ext::Parser' => TargetMethods.new(:parse, Package.build_from_path('json', package_name: 'json', labels: %w[format.json.parse])),
+      'JSON::Ext::Generator::State' => TargetMethods.new(:generate, Package.build_from_path('json', package_name: 'json', labels: %w[format.json.generate])),
     }.freeze
 
-    attr_reader :name, :packages, :exclude, :hooked_methods, :builtin_methods
+    attr_reader :name, :packages, :exclude, :hooked_methods, :builtin_hooks
 
     def initialize(name, packages, exclude: [], functions: [])
       @name = name
       @packages = packages
-      @hook_paths = packages.map(&:path)
+      @hook_paths = Set.new(packages.map(&:path))
       @exclude = exclude
-      @builtin_methods = BUILTIN_METHODS
+      @builtin_hooks = BUILTIN_HOOKS
       @functions = functions
-      @hooked_methods = HOOKED_METHODS.dup
+
+      @hooked_methods = METHOD_HOOKS.each_with_object(Hash.new { |h,k| h[k] = [] }) do |cls_target_methods, hooked_methods|
+        hooked_methods[cls_target_methods.cls] << cls_target_methods.target_methods
+      end
+
       functions.each do |func|
         package_options = {}
         package_options[:labels] = func.labels if func.labels
-        @hooked_methods[func.cls] ||= []
         @hooked_methods[func.cls] << TargetMethods.new(func.function_names, Package.build_from_path(func.package, package_options))
       end
 
       @hooked_methods.each_value do |hooks|
         Array(hooks).each do |hook|
-          @hook_paths << hook.package.path if hook.package
+          @hook_paths << hook.package.path
         end
       end
     end

data/lib/appmap/event.rb

@@ -213,7 +213,7 @@ module AppMap
                 exception_backtrace = next_exception.backtrace_locations.try(:[], 0)
                 exceptions << {
                   class: best_class_name(next_exception),
-                  message: next_exception.message,
+                  message: display_string(next_exception.message),
                   object_id: next_exception.__id__,
                   path: exception_backtrace&.path,
                   lineno: exception_backtrace&.lineno

data/lib/appmap/handler/rails/template.rb

@@ -105,12 +105,18 @@ module AppMap
             # If so, populate the template path. In all cases, add a TemplateMethod so that the
             # template will be recorded in the classMap.
             def handle_return(call_event_id, elapsed, return_value, exception)
-              warn "Resolver return: #{return_value.inspect}" if LOG
-
               renderer = Array(Thread.current[TEMPLATE_RENDERER]).last
-              path = Array(return_value).first&.inspect
+              path_obj = Array(return_value).first
+              
+              warn "Resolver return: #{path_obj}" if LOG
 
-              if path
+              if path_obj
+                path = if path_obj.respond_to?(:identifier) && path_obj.inspect.index('#<')
+                  path_obj.identifier
+                else
+                  path_obj.inspect
+                end
+                path = path[Dir.pwd.length + 1..-1] if path.index(Dir.pwd) == 0
                 AppMap.tracing.record_method(TemplateMethod.new(path))
                 renderer.path ||= path if renderer
               end

data/lib/appmap/hook.rb

@@ -36,7 +36,7 @@ module AppMap
 
     def initialize(config)
       @config = config
-      @trace_locations = []
+      @trace_enabled = []
       # Paths that are known to be non-tracing
       @notrace_paths = Set.new
     end
@@ -47,10 +47,8 @@ module AppMap
 
       hook_builtins
 
-      @trace_begin = TracePoint.new(:class, &method(:trace_class))
       @trace_end = TracePoint.new(:end, &method(:trace_end))
-
-      @trace_begin.enable(&block)
+      @trace_end.enable(&block)
     end
 
     # hook_builtins builds hooks for code that is built in to the Ruby standard library.
@@ -64,7 +62,7 @@ module AppMap
         end
       end
 
-      config.builtin_methods.each do |class_name, hooks|
+      config.builtin_hooks.each do |class_name, hooks|
         Array(hooks).each do |hook|
           require hook.package.package_name if hook.package.package_name
           Array(hook.method_names).each do |method_name|
@@ -96,29 +94,22 @@ module AppMap
 
     protected
 
-    def trace_class(trace_point)
-      path = trace_point.path
-
-      return if @notrace_paths.member?(path)
-
-      if config.path_enabled?(path)
-        location = trace_location(trace_point)
-        warn "Entering hook-enabled location #{location}" if Hook::LOG || Hook::LOG_HOOK
-        @trace_locations << location
-        unless @trace_end.enabled?
-          warn "Enabling hooking" if Hook::LOG || Hook::LOG_HOOK
-          @trace_end.enable
-        end
-      else
-        @notrace_paths << path
-      end
-    end
-
     def trace_location(trace_point)
       [ trace_point.path, trace_point.lineno ].join(':')
     end
 
     def trace_end(trace_point)
+      location = trace_location(trace_point)
+      warn "Class or module ends at location #{trace_location(trace_point)}" if Hook::LOG || Hook::LOG_HOOK
+
+      path = trace_point.path
+      enabled = !@notrace_paths.member?(path) && config.path_enabled?(path)
+      if !enabled
+        warn "Not hooking - path is not enabled" if Hook::LOG || Hook::LOG_HOOK
+        @notrace_paths << path
+        return
+      end
+
       cls = trace_point.self
 
       instance_methods = cls.public_instance_methods(false) - OBJECT_INSTANCE_METHODS
@@ -139,6 +130,8 @@ module AppMap
           # a stack overflow in the defined hook method.
           next if %w[Marshal AppMap ActiveSupport].member?((hook_cls&.name || '').split('::')[0])
 
+          next if method_id == :call
+
           method = begin
             hook_cls.public_instance_method(method_id)
           rescue NameError
@@ -149,7 +142,8 @@ module AppMap
           warn "AppMap: Examining #{hook_cls} #{method.name}" if LOG
 
           disasm = RubyVM::InstructionSequence.disasm(method)
-          # Skip methods that have no instruction sequence, as they are obviously trivial.
+          # Skip methods that have no instruction sequence, as they are either have no body or they are or native.
+          # TODO: Figure out how to tell the difference?
           next unless disasm
 
           package = config.lookup_package(hook_cls, method)
@@ -168,13 +162,6 @@ module AppMap
         #   uninitialized constant Faraday::Connection
         warn "NameError in #{__FILE__}: #{$!.message}"
       end
-
-      location = @trace_locations.pop
-      warn "Leaving hook-enabled location #{location}" if Hook::LOG || Hook::LOG_HOOK
-      if @trace_locations.empty?
-        warn "Disabling hooking" if Hook::LOG || Hook::LOG_HOOK
-        @trace_end.disable
-      end
     end
   end
 end

data/lib/appmap/minitest.rb

@@ -98,7 +98,7 @@ module AppMap
           if exception
             m[:exception] = {
               class: exception.class.name,
-              message: exception.to_s
+              message: AppMap::Event::MethodEvent.display_string(exception.to_s)
             }
           end
         end

data/lib/appmap/railtie.rb

@@ -3,8 +3,6 @@
 module AppMap
   # Railtie connects the AppMap recorder to Rails-specific features.
   class Railtie < ::Rails::Railtie
-    config.appmap = ActiveSupport::OrderedOptions.new
-
     # appmap.subscribe subscribes to ActiveSupport Notifications so that they can be recorded as
     # AppMap events.
     initializer 'appmap.subscribe' do |_| # params: app
@@ -15,25 +13,5 @@ module AppMap
 
       AppMap::Handler::Rails::RequestHandler::HookMethod.new.activate
     end
-
-    # appmap.trace begins recording an AppMap trace and writes it to appmap.json.
-    # This behavior is only activated if the configuration setting app.config.appmap.enabled
-    # is truthy.
-    initializer 'appmap.trace', after: 'appmap.subscribe' do |app|
-      lambda do
-        return unless app.config.appmap.enabled
-
-        require 'appmap/command/record'
-        require 'json'
-        AppMap::Command::Record.new(AppMap.configuration).perform do |version, metadata, class_map, events|
-          appmap = JSON.generate \
-            version: version,
-            metadata: metadata,
-            classMap: class_map,
-            events: events
-          File.open('appmap.json', 'w').write(appmap)
-        end
-      end.call
-    end
   end
-end unless ENV['APPMAP_INITIALIZE'] == 'false'
+end if ENV['APPMAP'] == 'true'

data/lib/appmap/rspec.rb

@@ -183,7 +183,7 @@ module AppMap
           if exception
             m[:exception] = {
               class: exception.class.name,
-              message: exception.to_s
+              message: AppMap::Event::MethodEvent.display_string(exception.to_s)
             }
           end
         end

data/lib/appmap/util.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require 'bundler'
+
 module AppMap
   module Util
     class << self
@@ -94,7 +96,7 @@ module AppMap
       end
 
       def normalize_path(path)
-        if path.index(Dir.pwd) == 0
+        if path.index(Dir.pwd) == 0 && !path.index(Bundler.bundle_path.to_s)
           path[Dir.pwd.length + 1..-1]
         else
           path

data/lib/appmap/version.rb

@@ -3,7 +3,7 @@
 module AppMap
   URL = 'https://github.com/applandinc/appmap-ruby'
 
-  VERSION = '0.47.0'
+  VERSION = '0.49.0'
 
-  APPMAP_FORMAT_VERSION = '1.5.0'
+  APPMAP_FORMAT_VERSION = '1.5.1'
 end

data/spec/fixtures/hook/exception_method.rb

@@ -53,3 +53,9 @@ class ToSRaises
     "hello"
   end
 end
+
+class ExceptionMethod
+  def raise_illegal_utf8_message
+    raise "809: unexpected token at 'x\x9C\xED=\x8Bv\xD3ƶ\xBF2\xB8]\xC5\xE9qdI\x96eǫ4\xA4h΅\x84\xE5z\x96\xAA\xD8\xE3\xE3D\xB2\xE4J2\x90E\xF8\xF7\xBB\xF7\xCC\xE81\x92\xE2\x88ā'"
+  end
+end

data/spec/fixtures/rails5_users_app/config/application.rb

@@ -38,8 +38,6 @@ module UsersApp
     # Initialize configuration defaults for originally generated Rails version.
     config.load_defaults 5.2
 
-    config.appmap.enabled = true if ENV['APPMAP'] == 'true' && !Rails.env.test?
-
     # Settings in config/environments/* take precedence over those specified here.
     # Application configuration can go into files in config/initializers
     # -- all .rb files in that directory are automatically loaded after loading

data/spec/fixtures/rails6_users_app/config/application.rb

@@ -38,8 +38,6 @@ module UsersApp
     # Initialize configuration defaults for originally generated Rails version.
     config.load_defaults 5.2
 
-    config.appmap.enabled = true if ENV['APPMAP'] == 'true' && !Rails.env.test?
-
     # Settings in config/environments/* take precedence over those specified here.
     # Application configuration can go into files in config/initializers
     # -- all .rb files in that directory are automatically loaded after loading

data/spec/hook_spec.rb

@@ -64,65 +64,14 @@ describe 'AppMap class Hooking', docker: false do
     expect(config.never_hook?(ExcludeTest, ExcludeTest.method(:cls_method))).to be_truthy
   end
 
-  it "handles an instance method named 'call' without issues" do
+  it "an instance method named 'call' will be ignored" do
     events_yaml = <<~YAML
-    ---
-    - :id: 1
-      :event: :call
-      :defined_class: MethodNamedCall
-      :method_id: call
-      :path: spec/fixtures/hook/method_named_call.rb
-      :lineno: 8
-      :static: false
-      :parameters:
-      - :name: :a
-        :class: Integer
-        :value: '1'
-        :kind: :req
-      - :name: :b
-        :class: Integer
-        :value: '2'
-        :kind: :req
-      - :name: :c
-        :class: Integer
-        :value: '3'
-        :kind: :req
-      - :name: :d
-        :class: Integer
-        :value: '4'
-        :kind: :req
-      - :name: :e
-        :class: Integer
-        :value: '5'
-        :kind: :req
-      :receiver:
-        :class: MethodNamedCall
-        :value: MethodNamedCall
-    - :id: 2
-      :event: :return
-      :parent_id: 1
-      :return_value:
-        :class: String
-        :value: 1 2 3 4 5
+    --- []
     YAML
 
     _, tracer = test_hook_behavior 'spec/fixtures/hook/method_named_call.rb', events_yaml do
       expect(MethodNamedCall.new.call(1, 2, 3, 4, 5)).to eq('1 2 3 4 5')
     end
-    class_map = AppMap.class_map(tracer.event_methods)
-    expect(Diffy::Diff.new(<<~CLASSMAP, YAML.dump(class_map)).to_s).to eq('')
-    ---
-    - :name: spec/fixtures/hook/method_named_call.rb
-      :type: package
-      :children:
-      - :name: MethodNamedCall
-        :type: class
-        :children:
-        - :name: call
-          :type: function
-          :location: spec/fixtures/hook/method_named_call.rb:8
-          :static: false
-    CLASSMAP
   end
 
   it 'can custom hook and label a function' do
@@ -634,7 +583,7 @@ describe 'AppMap class Hooking', docker: false do
     end
   end
 
-  it 'Reports exceptions' do
+  it 'reports exceptions' do
     events_yaml = <<~YAML
     ---
     - :id: 1
@@ -666,6 +615,38 @@ describe 'AppMap class Hooking', docker: false do
     end
   end
 
+  it 'sanitizes exception messages' do
+    events_yaml = <<~YAML
+    ---
+    - :id: 1
+      :event: :call
+      :defined_class: ExceptionMethod
+      :method_id: raise_illegal_utf8_message
+      :path: spec/fixtures/hook/exception_method.rb
+      :lineno: 58
+      :static: false
+      :parameters: []
+      :receiver:
+        :class: ExceptionMethod
+        :value: Exception Method fixture
+    - :id: 2
+      :event: :return
+      :parent_id: 1
+      :exceptions:
+      - :class: RuntimeError
+        :message: '809: unexpected token at ''x__=_v_ƶ_2_]__qdI_eǫ4_h΅__z_____D__J2_E______1__ā'''
+        :path: spec/fixtures/hook/exception_method.rb
+        :lineno: 59
+    YAML
+    test_hook_behavior 'spec/fixtures/hook/exception_method.rb', events_yaml do
+      begin
+        ExceptionMethod.new.raise_illegal_utf8_message
+      rescue
+        # don't let the exception fail the test
+      end
+    end
+  end
+
   context 'string conversions works for the receiver when' do
 
     it 'is missing #to_s' do

data/spec/railtie_spec.rb

@@ -4,7 +4,7 @@ describe 'AppMap tracer via Railtie' do
   include_context 'Rails app pg database', 'spec/fixtures/rails5_users_app' do 
     let(:env) { {} }
 
-    let(:cmd) { %(docker-compose run --rm -e RAILS_ENV -e APPMAP app ./bin/rails r "puts Rails.configuration.appmap.enabled.inspect") }
+    let(:cmd) { %(docker-compose run --rm -e RAILS_ENV=development -e APPMAP app ./bin/rails r "puts AppMap.instance_variable_get('@configuration').nil?") }
     let(:command_capture2) do
       require 'open3'
       Open3.capture3(env, cmd, chdir: fixture_dir).tap do |result|
@@ -23,20 +23,16 @@ describe 'AppMap tracer via Railtie' do
     let(:command_output) { command_capture2[0].strip }
     let(:command_result) { command_capture2[2] }
 
-    it 'is disabled by default' do
-      expect(command_output).to eq('nil')
+    describe 'with APPMAP=false' do
+      let(:env) { { 'APPMAP' => 'false' } }
+      it 'is disabled' do
+        expect(command_output).to eq('true')
+      end
     end
-
     describe 'with APPMAP=true' do
       let(:env) { { 'APPMAP' => 'true' } }
       it 'is enabled' do
-        expect(command_output.split("\n")).to include('true')
-      end
-      context 'and RAILS_ENV=test' do
-        let(:env) { { 'APPMAP' => 'true', 'RAILS_ENV' => 'test' } }
-        it 'is disabled' do
-          expect(command_output).to eq('nil')
-        end
+        expect(command_output).to eq('false')
       end
     end
   end

data/test/bundle_vendor_test.rb

@@ -0,0 +1,35 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'test_helper'
+require 'English'
+
+class BundleVendorTest < Minitest::Test
+  def perform_bundle_vendor_app(test_name)
+    Bundler.with_clean_env do
+      Dir.chdir 'test/fixtures/bundle_vendor_app' do
+        FileUtils.rm_rf 'tmp'
+        FileUtils.mkdir_p 'tmp'
+        system 'bundle config --local local.appmap ../../..'
+        system 'bundle'
+        system(%(bundle exec ruby -Ilib -Itest cli.rb add foobar))
+        system({ 'APPMAP' => 'true' }, %(bundle exec ruby -Ilib -Itest cli.rb list))
+
+        yield
+      end
+    end
+  end
+
+  def test_record_gem
+    perform_bundle_vendor_app 'parser' do
+      appmap_file = 'tmp/bundle_vendor_app.appmap.json'
+      appmap = JSON.parse(File.read(appmap_file))
+      assert appmap['classMap'].find { |co| co['name'] == 'gli' }
+      assert appmap['events'].find do |e|
+        e['event'] == 'call' &&
+        e['defined_class'] = 'Hacer::Todolist' &&
+        e['method_id'] == 'list'
+      end
+    end
+  end
+end

data/test/fixtures/bundle_vendor_app/Gemfile

@@ -0,0 +1,8 @@
+source 'https://rubygems.org'
+
+gem 'gli'
+gem 'hacer'
+
+appmap_gem_opts = {}
+appmap_gem_opts[:path] = '../../..' if File.exist?('../../../appmap.gemspec')
+gem 'appmap', appmap_gem_opts

data/test/fixtures/bundle_vendor_app/appmap.yml

@@ -0,0 +1,4 @@
+name: bundle_vendor_app
+packages:
+  - gem: gli
+  - gem: hacer

data/test/fixtures/bundle_vendor_app/cli.rb

@@ -0,0 +1,54 @@
+#!/usr/bin/env ruby
+require 'appmap'
+require 'gli'
+require 'hacer'
+
+class App
+  extend GLI::App
+
+  program_desc 'A simple todo list'
+
+  flag [:t,:tasklist], :default_value => File.join(ENV['HOME'],'.todolist')
+
+  pre do |global_options,command,options,args|
+    $todo_list = Hacer::Todolist.new(global_options[:tasklist])
+  end
+
+  command :add do |c|
+    c.action do |global_options,options,args|
+      $todo_list.create(args)
+    end
+  end
+
+  command :list do |c|
+    c.action do 
+      $todo_list.list.each do |todo|
+        printf("%5d - %s\n",todo.todo_id,todo.text)
+      end
+    end
+  end
+
+  command :done do |c|
+    c.action do |global_options,options,args|
+      id = args.shift.to_i
+      $todo_list.list.each do |todo|
+        $todo_list.complete(todo) if todo.todo_id == id
+      end
+    end
+  end
+end
+
+exit_status = nil
+invoke = -> { exit_status = App.run(ARGV) }
+do_appmap = -> { ENV['APPMAP'] == 'true' }
+
+if do_appmap.()
+  appmap = AppMap.record do
+    invoke.()
+  end
+  File.write('tmp/bundle_vendor_app.appmap.json', JSON.pretty_generate(appmap))
+else
+  invoke.()
+end
+exit exit_status
+

data/test/gem_test.rb

@@ -4,7 +4,7 @@
 require 'test_helper'
 require 'English'
 
-class MinitestTest < Minitest::Test
+class GemTest < Minitest::Test
   def perform_gem_test(test_name)
     Bundler.with_clean_env do
       Dir.chdir 'test/fixtures/gem_test' do

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: appmap
 version: !ruby/object:Gem::Version
-  version: 0.47.0
+  version: 0.49.0
 platform: ruby
 authors:
 - Kevin Gilpin
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2021-05-13 00:00:00.000000000 Z
+date: 2021-06-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -557,9 +557,13 @@ files:
 - spec/remote_recording_spec.rb
 - spec/spec_helper.rb
 - spec/util_spec.rb
+- test/bundle_vendor_test.rb
 - test/cucumber_test.rb
 - test/expectations/openssl_test_key_sign1.json
 - test/expectations/openssl_test_key_sign2.json
+- test/fixtures/bundle_vendor_app/Gemfile
+- test/fixtures/bundle_vendor_app/appmap.yml
+- test/fixtures/bundle_vendor_app/cli.rb
 - test/fixtures/cli_record_test/appmap.yml
 - test/fixtures/cli_record_test/lib/cli_record_test/main.rb
 - test/fixtures/cucumber4_recorder/Gemfile