rack-mini-profiler 2.0.4 → 2.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2e54608bc4885796e11d6e4fac9cc56fe478f60c5cd43d53637451ef3524f086
-  data.tar.gz: 6a87a58c7821a279ce08b66432de9a199a6f775f897610e03e12d497eb7ccc46
+  metadata.gz: 42690a86bcd4e4cfd8bf17289dc7b2f583bdd4ec286b4bbcdd50454f0f73551b
+  data.tar.gz: 723291f0ae196b09f585a331fc4e33b54bfd4be7fba527b623d851dc77524cce
 SHA512:
-  metadata.gz: 3730c9e4610fd90cc0fac86c1cddcf6181e15cef5932ba86f8f8cbca1df77c02ba7847c2e1868b48e51299b6c51224fe533818c3cc245dd02a2da11b2298569c
-  data.tar.gz: 98f54eed006f7f13840481b777dbe4add5f9983c5ed367f8046a942a584a5ad3eb4ea5ae6e3aef7e4aa0760b37c78ac5dd6ee5bcd45e42091a943dfabdb343b1
+  metadata.gz: ca4bd7e3f35d42171bf2e39775a4a8fb565b907ba1eab40002886f7933bb4e9d30bfb8d9a87a7cb408e68b3f7e1a5dbbd360fa633f8e5c94793218abb40b32d5
+  data.tar.gz: 877164b6caf03efc6fa4183bed8f6dfce09c691e98c9ac3588fae8307d6b1632be8e666140c4a581280b6f0bfb69c2f5d4cb100ecca0f4afe1d3a9cd770b8e7f

data/CHANGELOG.md

@@ -1,5 +1,18 @@
 # CHANGELOG
 
+## 2.2.0 - 2020-10-19
+
+- [UX] Enhancements to snapshots UI
+- [FEATURE] Mini Profiler cookie is now sameSite=lax
+- [FEATURE] Snapshots transporter
+- [FEATURE] Redact SQL queries in snapshots by default
+
+## 2.1.0 - 2020-09-17
+
+- [FEATURE] Allow assets to be precompiled with Sprockets
+- [FEATURE] Snapshots sampling (see README in repo)
+- [FEATURE] Allow `skip_paths` config to contain regular expressions
+
 ## 2.0.4 - 2020-08-04
 
 - [FIX] webpacker may exist with no config, allow for that

data/README.md

@@ -186,6 +186,28 @@ There are two additional `pp` options that can be used to analyze memory which d
 * Use `?pp=profile-gc` to report on Garbage Collection statistics
 * Use `?pp=analyze-memory` to report on ObjectSpace statistics
 
+### Snapshots Sampling
+
+In a complex web application, it's possible for a request to trigger rare conditions that result in poor performance. Mini Profiler ships with a feature to help detect those rare conditions and fix them. It works by enabling invisible profiling on one request every N requests, and saving the performance metrics that are collected during the request (a.k.a snapshot of the request) so that they can be viewed later. To turn this feature on, set the `snapshot_every_n_requests` config to a value larger than 0. The larger the value is, the less frequently requests are profiled.
+
+Mini Profiler will exclude requests that are made to skippd paths (see `skip_paths` config below) from being sampled. Additionally, if profiling is enabled for a request that later finishes with a non-2xx status code, Mini Profiler will discard the snapshot and not save it (this behavior may change in the future).
+
+After enabling snapshots sampling, you can see the snapshots that have been collected at `/mini-profiler-resources/snapshots` (or if you changed the `base_url_path` config, substitute `mini-profiler-resources` with your value of the config). You'll see on that page a table where each row represents a group of snapshots with the duration of the worst snapshot in that group. The worst snapshot in a group is defined as the snapshot whose request took longer than all of the snapshots in the same group. Snapshots grouped by HTTP method and path of the request, and if your application is a Rails app, Mini Profiler will try to convert the path to `controller#action` and group by that instead of request path. Clicking on a group will display the snapshots of that group sorted from worst to best. From there, you can click on a snapshot's ID to see the snapshot with all the performance metrics that were collected.
+
+Access to the snapshots page is restricted to only those who can see the speed badge on their own requests, see the section below this one about access control.
+
+Mini Profiler will keep a maximum of 1000 snapshots by default, and you can change that via the `snapshots_limit` config. When snapshots reach the configured limit, Mini Profiler will save a new snapshot only if it's worse than at least one of the existing snapshots and delete the best one (i.e. the snapshot whose request took the least time compared to other snapshots).
+
+#### Snapshots Transporter
+
+Mini Profiler can be configured so that it sends snapshots over HTTP using the snapshots transporter. The main use-case of the transporter is to allow the aggregation of snapshots from multiple applications/sources in a single place. To enable the snapshots transporter, you need to provide a destination URL to the `snapshots_transport_destination_url` config, and a secure key to the `snapshots_transport_auth_key` config (will be used for authorization). Both of these configs are required for the transporter to be enabled.
+
+The transporter uses a buffer to temporarily hold snapshots in memory with a limit of 100 snapshots. Every 10 seconds, *if* the buffer is not empty, the transporter will make a `POST` request with the buffer content to the destination URL. Requests made by the transporter will have a `Mini-Profiler-Transport-Auth` header with the value of the `snapshots_transport_auth_key` config. The destination should only accept requests that include this header AND the header's value matches the key you set to the `snapshots_transport_auth_key` config.
+
+The body of the requests made by the transporter is a JSON string with a single top-level key called `snapshots` and it has an array of snapshots. The structure of a snapshot is too complex to be explained here, but it has the same structure that Mini Profiler client expects. So if your use-case is to simply be able to view snapshots from multiple sources in one place, you should simply store the snapshots as-is, and then serve them to Mini Profiler client to consume. If the destination application also has Mini Profiler, you can simply use the API of the storage backends to store the incoming snapshots and Mini Profiler will treat them the same as local snapshots (e.g. they'll be grouped and displayed in the same manner described in the previous section).
+
+Mini Profiler offers an API to add extra fields (a.k.a custom fields) to snapshots. For example, you may want to add whether the request was made by a logged-in or anonymous user, the version of your application or any other things that are specific to your application. To add custom fields to a snapshot, call the `Rack::MiniProfiler.add_snapshot_custom_field(<key>, <value>)` method anywhere during the lifetime of a request, and the snapshot of that request will include the fields you added. If you have a Rails app, you can call that method in an `after_action` callback. Custom fields are cleared between requests.
+
 ## Access control in non-development environments
 
 rack-mini-profiler is designed with production profiling in mind. To enable that run `Rack::MiniProfiler.authorize_request` once you know a request is allowed to profile.
@@ -329,6 +351,15 @@ _Note:_ The GUID (`data-version` and the `?v=` parameter on the `src`) will chan
 #### Using MiniProfiler's built in route for apps without HTML responses
 MiniProfiler also ships with a `/rack-mini-profiler/requests` route that displays the speed badge on a blank HTML page. This can be useful when profiling an application that does not render HTML.
 
+#### Register MiniProfiler's assets in the Rails assets pipeline
+MiniProfiler can be configured so it registers its assets in the assets pipeline. To do that, you'll need to provide a lambda (or proc) to the `assets_url` config (see the below section). The callback will receive 3 arguments which are: `name` represents asset name (currently it's either `rack-mini-profiling.js` or `rack-mini-profiling.css`), `assets_version` is a 32 characters long hash of MiniProfiler's assets, and `env` which is the `env` object of the request. MiniProfiler expects the `assets_url` callback to return a URL from which the asset can be loaded (the return value will be used as a `href`/`src` attribute in the DOM). If the `assets_url` callback is not set (the default) or it returns a non-truthy value, MiniProfiler will fallback to loading assets from its own middleware (`/mini-profiler-resources/*`). The following callback should work for most applications:
+
+```ruby
+Rack::MiniProfiler.config.assets_url = ->(name, version, env) {
+  ActionController::Base.helpers.asset_path(name)
+}
+```
+
 ### Configuration Options
 
 You can set configuration options using the configuration accessor on `Rack::MiniProfiler`.
@@ -344,7 +375,7 @@ Option|Default|Description
 -------|---|--------
 pre_authorize_cb|Rails: dev only<br>Rack: always on|A lambda callback that returns true to make mini_profiler visible on a given request.
 position|`'top-left'`|Display mini_profiler on `'top-right'`, `'top-left'`, `'bottom-right'` or `'bottom-left'`.
-skip_paths|`[]`|Paths that skip profiling.
+skip_paths|`[]`|An array of paths that skip profiling. Both `String` and `Regexp` are acceptable in the array.
 skip_schema_queries|Rails dev: `true`<br>Othwerwise: `false`|`true` to skip schema queries.
 auto_inject|`true`|`true` to inject the miniprofiler script in the page.
 backtrace_ignores|`[]`|Regexes of lines to be removed from backtraces.
@@ -360,6 +391,13 @@ max_traces_to_show|20|Maximum number of mini profiler timing blocks to show on o
 html_container|`body`|The HTML container (as a jQuery selector) to inject the mini_profiler UI into
 show_total_sql_count|`false`|Displays the total number of SQL executions.
 enable_advanced_debugging_tools|`false`|Enables sensitive debugging tools that can be used via the UI. In production we recommend keeping this disabled as memory and environment debugging tools can expose contents of memory that may contain passwords.
+assets_url|`nil`|See the "Register MiniProfiler's assets in the Rails assets pipeline" section above.
+snapshot_every_n_requests|`-1`|Determines how frequently snapshots are taken. See the "Snapshots Sampling" above for more details.
+snapshots_limit|`1000`|Determines how many snapshots Mini Profiler is allowed to keep.
+snapshot_hidden_custom_fields|`[]`|Each snapshot custom field will have a dedicated column in the UI by default. Use this config to exclude certain custom fields from having their own columns.
+snapshots_transport_destination_url|`nil`|Set this config to a valid URL to enable snapshots transporter which will `POST` snapshots to the given URL. The transporter requires `snapshots_transport_auth_key` config to be set as well.
+snapshots_transport_auth_key|`nil`|`POST` requests made by the snapshots transporter to the destination URL will have a `Mini-Profiler-Transport-Auth` header with the value of this config. Make sure you use a secure and random key for this config.
+snapshots_redact_sql_queries|`true`|When this is true, SQL queries will be redacted from sampling snapshots, but the backtrace and duration of each SQL query will be saved with the snapshot to keep debugging performance issues possible.
 
 ### Using MiniProfiler with `Rack::Deflate` middleware
 

data/lib/html/includes.css

@@ -1,9 +1,20 @@
 @charset "UTF-8";
+.mp-snapshots,
 .profiler-result,
 .profiler-queries {
   color: #555;
   line-height: 1;
   font-size: 12px; }
+  .mp-snapshots pre,
+  .mp-snapshots code,
+  .mp-snapshots label,
+  .mp-snapshots table,
+  .mp-snapshots tbody,
+  .mp-snapshots thead,
+  .mp-snapshots tfoot,
+  .mp-snapshots tr,
+  .mp-snapshots th,
+  .mp-snapshots td,
   .profiler-result pre,
   .profiler-result code,
   .profiler-result label,
@@ -33,27 +44,40 @@
     background-color: transparent;
     overflow: visible;
     max-height: none; }
+  .mp-snapshots table,
   .profiler-result table,
   .profiler-queries table {
     border-collapse: collapse;
     border-spacing: 0; }
+  .mp-snapshots a,
+  .mp-snapshots a:hover,
   .profiler-result a,
   .profiler-result a:hover,
   .profiler-queries a,
   .profiler-queries a:hover {
     cursor: pointer;
     color: #0077cc; }
+  .mp-snapshots a,
   .profiler-result a,
   .profiler-queries a {
     text-decoration: none; }
+    .mp-snapshots a:hover,
     .profiler-result a:hover,
     .profiler-queries a:hover {
       text-decoration: underline; }
+  .mp-snapshots .custom-fields-title,
+  .profiler-result .custom-fields-title,
+  .profiler-queries .custom-fields-title {
+    color: #555;
+    font: Helvetica, Arial, sans-serif;
+    font-size: 14px; }
 
 .profiler-result {
   font-family: Helvetica, Arial, sans-serif; }
   .profiler-result .profiler-toggle-duration-with-children {
     float: right; }
+  .profiler-result .profiler-snapshots-page-link {
+    float: left; }
   .profiler-result table.profiler-client-timings {
     margin-top: 10px; }
   .profiler-result .profiler-label {
@@ -402,3 +426,17 @@
     background: #ffffbb; }
   100% {
     background: #fff; } }
+
+.mp-snapshots {
+  font-family: Helvetica, Arial, sans-serif;
+  font-size: 16px; }
+  .mp-snapshots .snapshots-table thead {
+    background: #6a737c;
+    color: #ffffff; }
+  .mp-snapshots .snapshots-table th, .mp-snapshots .snapshots-table td {
+    padding: 5px 10px;
+    box-sizing: border-box; }
+    .mp-snapshots .snapshots-table th:not(.request-group), .mp-snapshots .snapshots-table td:not(.request-group) {
+      text-align: center; }
+  .mp-snapshots .snapshots-table th {
+    border-right: 1px solid #ffffff; }

data/lib/html/includes.js

@@ -673,6 +673,32 @@ var MiniProfiler = (function() {
     });
   };
 
+  var initSnapshots = function initSnapshots(dataElement) {
+    var data = JSON.parse(dataElement.textContent);
+    var temp = document.createElement("DIV");
+    if (data.page === "overview") {
+      temp.innerHTML = MiniProfiler.templates.snapshotsGroupsList(data);
+    } else if (data.group_name) {
+      var allCustomFieldsNames = [];
+      data.list.forEach(function (snapshot) {
+        Object.keys(snapshot.custom_fields).forEach(function (k) {
+          if (allCustomFieldsNames.indexOf(k) === -1 &&
+            options.hiddenCustomFields.indexOf(k.toLowerCase()) === -1) {
+            allCustomFieldsNames.push(k);
+          }
+        });
+      });
+      allCustomFieldsNames.sort();
+      temp.innerHTML = MiniProfiler.templates.snapshotsList({
+        data: data,
+        allCustomFieldsNames: allCustomFieldsNames
+      });
+    }
+    Array.from(temp.children).forEach(function (child) {
+      document.body.appendChild(child);
+    });
+  };
+
   var initControls = function initControls(container) {
     if (options.showControls) {
       var _controls = document.createElement("div");
@@ -952,7 +978,7 @@ var MiniProfiler = (function() {
       var script = document.getElementById("mini-profiler");
       if (!script || !script.getAttribute) return;
 
-      options = (function() {
+      this.options = options = (function() {
         var version = script.getAttribute("data-version");
         var path = script.getAttribute("data-path");
         var currentId = script.getAttribute("data-current-id");
@@ -980,6 +1006,8 @@ var MiniProfiler = (function() {
           script.getAttribute("data-start-hidden") === "true" ||
           sessionStorage["rack-mini-profiler-start-hidden"] === "true";
         var htmlContainer = script.getAttribute("data-html-container");
+        var cssUrl = script.getAttribute("data-css-url");
+        var hiddenCustomFields = script.getAttribute("data-hidden-custom-fields").toLowerCase().split(",");
         return {
           ids: ids,
           path: path,
@@ -996,11 +1024,19 @@ var MiniProfiler = (function() {
           toggleShortcut: toggleShortcut,
           startHidden: startHidden,
           collapseResults: collapseResults,
-          htmlContainer: htmlContainer
+          htmlContainer: htmlContainer,
+          cssUrl: cssUrl,
+          hiddenCustomFields: hiddenCustomFields
         };
       })();
 
       var doInit = function doInit() {
+        var snapshotsElement = document.getElementById("snapshots-data");
+        if (snapshotsElement != null) {
+          initSnapshots(snapshotsElement);
+          return;
+        }
+
         // when rendering a shared, full page, this div will exist
         container = document.querySelectorAll(".profiler-result-full");
 
@@ -1056,7 +1092,7 @@ var MiniProfiler = (function() {
 
       var init = function init() {
         if (options.authorized) {
-          var url = options.path + "includes.css?v=" + options.version;
+          var url = options.cssUrl;
 
           if (document.createStyleSheet) {
             document.createStyleSheet(url);
@@ -1394,8 +1430,34 @@ var MiniProfiler = (function() {
     },
     showTotalSqlCount: function showTotalSqlCount() {
       return options.showTotalSqlCount;
+    },
+    timestampToRelative: function timestampToRelative(timestamp) {
+      var now = Math.round((new Date()).getTime() / 1000);
+      timestamp = Math.round(timestamp / 1000);
+      var diff = now - timestamp;
+      if (diff < 60) {
+        return "< 1 minute";
+      }
+      var buildDisplayTime = function buildDisplayTime(num, unit) {
+        var res = num + " " + unit;
+        if (num !== 1) {
+          res += "s";
+        }
+        return res;
+      }
+      diff = Math.round(diff / 60);
+      if (diff <= 60) {
+        return buildDisplayTime(diff, "minute");
+      }
+      diff = Math.round(diff / 60);
+      if (diff <= 24) {
+        return buildDisplayTime(diff, "hour");
+      }
+      diff = Math.round(diff / 24);
+      return buildDisplayTime(diff, "day");
     }
   };
 })();
 
+window.MiniProfiler = MiniProfiler;
 MiniProfiler.init();

data/lib/html/includes.scss

@@ -14,9 +14,10 @@ $codeFonts: Consolas, monospace, serif;
 $zindex: 2147483640; // near 32bit max 2147483647
 
 // do some resets
+.mp-snapshots,
 .profiler-result,
 .profiler-queries {
-  color: #555;
+  color: $textColor;
   line-height: 1;
   font-size: 12px;
 
@@ -55,6 +56,11 @@ $zindex: 2147483640; // near 32bit max 2147483647
       text-decoration: underline;
     }
   }
+  .custom-fields-title {
+    color: $textColor;
+    font: $normalFonts;
+    font-size: 14px;
+  }
 }
 
 // styles shared between popup view and full view
@@ -62,6 +68,9 @@ $zindex: 2147483640; // near 32bit max 2147483647
   .profiler-toggle-duration-with-children {
     float: right;
   }
+  .profiler-snapshots-page-link {
+    float: left;
+  }
   table.profiler-client-timings {
     margin-top: 10px;
   }
@@ -199,7 +208,7 @@ $zindex: 2147483640; // near 32bit max 2147483647
 
     th {
       background-color: #fff;
-      border-bottom: 1px solid #555;
+      border-bottom: 1px solid $textColor;
       font-weight: bold;
       padding: 15px;
       white-space: nowrap;
@@ -452,7 +461,7 @@ $zindex: 2147483640; // near 32bit max 2147483647
     line-height: 18px;
     overflow: auto;
 
-    @include box-shadow(0px, 1px, 15px, #555);
+    @include box-shadow(0px, 1px, 15px, $textColor);
 
     .profiler-info {
       margin-bottom: 3px;
@@ -592,7 +601,7 @@ $zindex: 2147483640; // near 32bit max 2147483647
       }
       th {
         font-size: 16px;
-        color: #555;
+        color: $textColor;
         line-height: 20px;
       }
 
@@ -617,3 +626,25 @@ $zindex: 2147483640; // near 32bit max 2147483647
     background: #fff;
   }
 }
+
+.mp-snapshots {
+  font-family: $normalFonts;
+  font-size: 16px;
+
+  .snapshots-table {
+    thead {
+      background: #6a737c;
+      color: #ffffff;
+    }
+    th, td {
+      padding: 5px 10px;
+      box-sizing: border-box;
+      &:not(.request-group) {
+        text-align: center;
+      }
+    }
+    th {
+      border-right: 1px solid #ffffff;
+    }
+  }
+}

data/lib/html/includes.tmpl

@@ -17,7 +17,7 @@
         <span class="profiler-name">
           {{= it.name}} <span class="profiler-overall-duration">({{= MiniProfiler.formatDuration(it.duration_milliseconds)}} ms)</span>
         </span>
-        <span class="profiler-server-time">{{= it.machine_name}} on {{= MiniProfiler.renderDate(it.started)}}</span>
+        <span class="profiler-server-time">{{= it.machine_name}} on {{= MiniProfiler.renderDate(it.started_formatted)}}</span>
       </div>
       <div class="profiler-output">
         <table class="profiler-timings">
@@ -45,6 +45,10 @@
                    {{= MiniProfiler.templates.linksTemplate({timing: it.root, page: it}) }}
                 {{?}}
                 <a class="profiler-toggle-duration-with-children" title="toggles column with aggregate child durations">show time with children</a>
+                <a
+                  class="profiler-snapshots-page-link"
+                  title="Go to snapshots page"
+                  href="{{= MiniProfiler.options.path }}snapshots">snapshots</a>
               </td>
               {{? it.has_sql_timings}}
                 <td colspan="2" class="profiler-number profiler-percent-in-sql" title="{{= MiniProfiler.getSqlTimingsCount(it.root) }} queries spent {{= MiniProfiler.formatDuration(it.duration_milliseconds_in_sql) }} ms of total request time">
@@ -92,6 +96,19 @@
             </tfoot>
           </table>
         {{?}}
+        {{? it.custom_fields && Object.keys(it.custom_fields).length > 0 }}
+          <p class="custom-fields-title">Snapshot custom fields</p>
+          <table class="profiler-timings">
+            <tbody>
+              {{~ Object.keys(it.custom_fields) :key }}
+                <tr>
+                  <td class="profiler-label">{{= key }}</td>
+                  <td class="profiler-label">{{= it.custom_fields[key] }}</td>
+                </tr>
+              {{~}}
+            </tbody>
+          </table>
+        {{?}}
       </div>
     </div>
 
@@ -200,7 +217,11 @@
     <td>
       <div class="query">
         <pre class="profiler-stack-trace">{{= it.s.stack_trace_snippet }}</pre>
-        <pre class="prettyprint lang-sql"><code>{{= it.s.formatted_command_string }}; {{= MiniProfiler.formatParameters(it.s.parameters) }}</code></pre>
+        {{? it.s.formatted_command_string}}
+          <pre class="prettyprint lang-sql"><code>{{= it.s.formatted_command_string }}; {{= MiniProfiler.formatParameters(it.s.parameters) }}</code></pre>
+        {{??}}
+          <i>Query redacted</i>
+        {{?}}
       </div>
     </td>
   </tr>
@@ -216,3 +237,72 @@
     </td>
   </tr>
 </script>
+
+<script id="snapshotsGroupsList" type="text/x-dot-tmpl">
+  {{? it.list && it.list.length }}
+    <table class="snapshots-table">
+      <thead>
+        <tr>
+          <th>Requests Group</th>
+          <th>Worst Time (ms)</th>
+          <th>Best Time (ms)</th>
+          <th>No. of Snapshots</th>
+        </tr>
+      </thead>
+      <tbody>
+        {{~ it.list :row}}
+          <tr>
+            <td class="request-group"><a href="{{= row.url }}">{{= row.name }}</a></td>
+            <td>{{= MiniProfiler.formatDuration(row.worst_score) }}</td>
+            <td>{{= MiniProfiler.formatDuration(row.best_score) }}</td>
+            <td>{{= row.snapshots_count }}</td>
+          </tr>
+        {{~}}
+      </tbody>
+    </table>
+  {{??}}
+    <h2>No snapshots exist</h2>
+  {{?}}
+</script>
+
+<script id="snapshotsList" type="text/x-dot-tmpl">
+  {{ var data = it.data; }}
+  {{ var customFieldsNames = it.allCustomFieldsNames; }}
+  {{? data.list && data.list.length }}
+    <h2>Snapshots for {{= data.group_name }}</h2>
+    <table class="snapshots-table">
+      <thead>
+        <tr>
+          <th>ID</th>
+          <th>Duration (ms)</th>
+          <th>SQL Count</th>
+          {{~ customFieldsNames :name }}
+            <th>{{= name }}</th>
+          {{~}}
+          <th>Age</th>
+        </tr>
+      </thead>
+      <tbody>
+        {{~ data.list :row}}
+          <tr>
+            <td><a href="{{= row.url }}">
+              {{= row.id }}
+            </a></td>
+            <td>{{= MiniProfiler.formatDuration(row.duration) }}</td>
+            <td>{{= row.sql_count }}</td>
+            {{~ customFieldsNames :name }}
+              <td>{{= row.custom_fields[name] || "" }}</td>
+            {{~}}
+            <td>
+              {{? row.timestamp }}
+                {{= MiniProfiler.timestampToRelative(row.timestamp) }}
+              {{?}}
+            </td>
+          </tr>
+        {{~}}
+      </tbody>
+    </table>
+  {{??}}
+    <h2>No snapshots for {{= data.group_name }}</h2>
+  {{?}}
+</script>