startup-time 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: a9ba80fb5d1e63389c3ca3b8b3dc3b0a8cc338504f6a23c810d14335e0e044f5
+  data.tar.gz: 8e199bbea99112e8dc993d6c3500bad99048cb72fd8f6bd96c498da40ee9b20d
+SHA512:
+  metadata.gz: 9de0b24fbd653e0d52e95b39838d2af9258de9fbd0e2e16f75827413c37d4ed86522da5f74b531fd71404faf9cc1f73d3bfe5afd19ded0be2eb0960b220d297c
+  data.tar.gz: a2432428c758ccc48fa0e7616c88fa61e0acd73abab6ec4751d4f3cb2cc93f4fe7efed31c58f611c4e6173d2aa993ff13e6053b1559945a34e036d8e92d12de1

data/CHANGELOG.md

@@ -0,0 +1,6 @@
+## 1.0.0 - 2019-02-21
+
+- gemify
+- add tests for Deno, GraalVM JavaScript, Nim, and TruffleRuby
+- add --json option to render the results as a JSON array
+- add changelog

data/LICENSE.md

@@ -0,0 +1,192 @@
+The Artistic License 2.0
+========================
+
+_Copyright © 2000-2006, The Perl Foundation._
+
+Everyone is permitted to copy and distribute verbatim copies
+of this license document, but changing it is not allowed.
+
+### Preamble
+
+This license establishes the terms under which a given free software
+Package may be copied, modified, distributed, and/or redistributed.
+The intent is that the Copyright Holder maintains some artistic
+control over the development of that Package while still keeping the
+Package available as open source and free software.
+
+You are always permitted to make arrangements wholly outside of this
+license directly with the Copyright Holder of a given Package.  If the
+terms of this license do not permit the full use that you propose to
+make of the Package, you should contact the Copyright Holder and seek
+a different licensing arrangement.
+
+### Definitions
+
+“Copyright Holder” means the individual(s) or organization(s)
+named in the copyright notice for the entire Package.
+
+“Contributor” means any party that has contributed code or other
+material to the Package, in accordance with the Copyright Holder's
+procedures.
+
+“You” and “your” means any person who would like to copy,
+distribute, or modify the Package.
+
+“Package” means the collection of files distributed by the
+Copyright Holder, and derivatives of that collection and/or of
+those files. A given Package may consist of either the Standard
+Version, or a Modified Version.
+
+“Distribute” means providing a copy of the Package or making it
+accessible to anyone else, or in the case of a company or
+organization, to others outside of your company or organization.
+
+“Distributor Fee” means any fee that you charge for Distributing
+this Package or providing support for this Package to another
+party.  It does not mean licensing fees.
+
+“Standard Version” refers to the Package if it has not been
+modified, or has been modified only in ways explicitly requested
+by the Copyright Holder.
+
+“Modified Version” means the Package, if it has been changed, and
+such changes were not explicitly requested by the Copyright
+Holder.
+
+“Original License” means this Artistic License as Distributed with
+the Standard Version of the Package, in its current version or as
+it may be modified by The Perl Foundation in the future.
+
+“Source” form means the source code, documentation source, and
+configuration files for the Package.
+
+“Compiled” form means the compiled bytecode, object code, binary,
+or any other form resulting from mechanical transformation or
+translation of the Source form.
+
+### Permission for Use and Modification Without Distribution
+
+**(1)**  You are permitted to use the Standard Version and create and use
+Modified Versions for any purpose without restriction, provided that
+you do not Distribute the Modified Version.
+
+### Permissions for Redistribution of the Standard Version
+
+**(2)**  You may Distribute verbatim copies of the Source form of the
+Standard Version of this Package in any medium without restriction,
+either gratis or for a Distributor Fee, provided that you duplicate
+all of the original copyright notices and associated disclaimers.  At
+your discretion, such verbatim copies may or may not include a
+Compiled form of the Package.
+
+**(3)**  You may apply any bug fixes, portability changes, and other
+modifications made available from the Copyright Holder.  The resulting
+Package will still be considered the Standard Version, and as such
+will be subject to the Original License.
+
+### Distribution of Modified Versions of the Package as Source
+
+**(4)**  You may Distribute your Modified Version as Source (either gratis
+or for a Distributor Fee, and with or without a Compiled form of the
+Modified Version) provided that you clearly document how it differs
+from the Standard Version, including, but not limited to, documenting
+any non-standard features, executables, or modules, and provided that
+you do at least ONE of the following:
+
+* **(a)**  make the Modified Version available to the Copyright Holder
+of the Standard Version, under the Original License, so that the
+Copyright Holder may include your modifications in the Standard
+Version.
+* **(b)**  ensure that installation of your Modified Version does not
+prevent the user installing or running the Standard Version. In
+addition, the Modified Version must bear a name that is different
+from the name of the Standard Version.
+* **(c)**  allow anyone who receives a copy of the Modified Version to
+make the Source form of the Modified Version available to others
+under
+        * **(i)**  the Original License or
+        * **(ii)**  a license that permits the licensee to freely copy,
+        modify and redistribute the Modified Version using the same
+        licensing terms that apply to the copy that the licensee
+        received, and requires that the Source form of the Modified
+        Version, and of any works derived from it, be made freely
+        available in that license fees are prohibited but Distributor
+        Fees are allowed.
+
+### Distribution of Compiled Forms of the Standard Version
+###  or Modified Versions without the Source
+
+**(5)**  You may Distribute Compiled forms of the Standard Version without
+the Source, provided that you include complete instructions on how to
+get the Source of the Standard Version.  Such instructions must be
+valid at the time of your distribution.  If these instructions, at any
+time while you are carrying out such distribution, become invalid, you
+must provide new instructions on demand or cease further distribution.
+If you provide valid instructions or cease distribution within thirty
+days after you become aware that the instructions are invalid, then
+you do not forfeit any of your rights under this license.
+
+**(6)**  You may Distribute a Modified Version in Compiled form without
+the Source, provided that you comply with Section 4 with respect to
+the Source of the Modified Version.
+
+### Aggregating or Linking the Package
+
+**(7)**  You may aggregate the Package (either the Standard Version or
+Modified Version) with other packages and Distribute the resulting
+aggregation provided that you do not charge a licensing fee for the
+Package.  Distributor Fees are permitted, and licensing fees for other
+components in the aggregation are permitted. The terms of this license
+apply to the use and Distribution of the Standard or Modified Versions
+as included in the aggregation.
+
+**(8)** You are permitted to link Modified and Standard Versions with
+other works, to embed the Package in a larger work of your own, or to
+build stand-alone binary or bytecode versions of applications that
+include the Package, and Distribute the result without restriction,
+provided the result does not expose a direct interface to the Package.
+
+### Items That are Not Considered Part of a Modified Version
+
+**(9)** Works (including, but not limited to, modules and scripts) that
+merely extend or make use of the Package, do not, by themselves, cause
+the Package to be a Modified Version.  In addition, such works are not
+considered parts of the Package itself, and are not subject to the
+terms of this license.
+
+### General Provisions
+
+**(10)**  Any use, modification, and distribution of the Standard or
+Modified Versions is governed by this Artistic License. By using,
+modifying or distributing the Package, you accept this license. Do not
+use, modify, or distribute the Package, if you do not accept this
+license.
+
+**(11)**  If your Modified Version has been derived from a Modified
+Version made by someone other than you, you are nevertheless required
+to ensure that your Modified Version complies with the requirements of
+this license.
+
+**(12)**  This license does not grant you the right to use any trademark,
+service mark, tradename, or logo of the Copyright Holder.
+
+**(13)**  This license includes the non-exclusive, worldwide,
+free-of-charge patent license to make, have made, use, offer to sell,
+sell, import and otherwise transfer the Package with respect to any
+patent claims licensable by the Copyright Holder that are necessarily
+infringed by the Package. If you institute patent litigation
+(including a cross-claim or counterclaim) against any party alleging
+that the Package constitutes direct or contributory patent
+infringement, then this Artistic License to you shall terminate on the
+date that such litigation is filed.
+
+**(14)**  **Disclaimer of Warranty:**
+
+THE PACKAGE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS "AS
+IS' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES. THE IMPLIED
+WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR
+NON-INFRINGEMENT ARE DISCLAIMED TO THE EXTENT PERMITTED BY YOUR LOCAL
+LAW. UNLESS REQUIRED BY LAW, NO COPYRIGHT HOLDER OR CONTRIBUTOR WILL
+BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
+DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THE PACKAGE, EVEN IF
+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.md

@@ -0,0 +1,121 @@
+# startup-time
+
+[![Build Status](https://travis-ci.org/chocolateboy/startup-time.svg)](https://travis-ci.org/chocolateboy/startup-time)
+[![Gem Version](https://img.shields.io/gem/v/startup-time.svg)](https://rubygems.org/gems/startup-time)
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+
+- [NAME](#name)
+- [USAGE](#usage)
+- [INSTALLATION](#installation)
+- [DESCRIPTION](#description)
+  - [Why?](#why)
+  - [Example Output](#example-output)
+- [PREREQUISITES](#prerequisites)
+- [Further Reading](#further-reading)
+- [See Also](#see-also)
+- [Author](#author)
+- [Version](#version)
+- [Copyright and License](#copyright-and-license)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## NAME
+
+startup-time - a command-line benchmarking tool which measures the startup times of programs in various languages
+
+## USAGE
+
+```sh
+# run all available tests
+$ startup-time
+
+# only run the "fast" tests
+$ startup-time --only fast
+
+# run all but the "slow" tests
+$ startup-time --omit slow
+
+# only run the JVM tests (e.g. Java, Scala etc.)
+$ startup-time --only jvm
+
+# only run tests which finish quickly
+$ startup-time --only fast --omit slow-compile
+
+# increase the number of times each test is run (default: 10)
+$ startup-time --rounds 100
+```
+
+## INSTALLATION
+
+```sh
+$ gem install startup-time
+```
+
+## DESCRIPTION
+
+A command-line tool which measures how long it takes to execute ["Hello, world!"](https://en.wikipedia.org/wiki/%22Hello,_World!%22_program) programs written in various languages. It records the fastest time for each program and prints a sorted table of the times after each run. Apart from the [prerequisites](#prerequisites) listed below, the tool doesn't require any of the tested languages to be installed: if a compiler/interpreter is not available, the test is skipped.
+
+### Why?
+
+To determine which languages are practical (or impractical) to use for command-line interface (CLI) programs. Anything under
+[100 milliseconds](https://www.nngroup.com/articles/response-times-3-important-limits/) is perceived as instantaneous. Anything over that is perceived as delayed, which can tangibly impair productivity and flow, and even risk breaking the user's train of thought.
+
+### Example Output
+
+    Test                  Time (ms)
+    C (gcc)                    0.33
+    Nim                        0.45
+    LuaJIT                     0.62
+    Rust                       0.63
+    Kotlin Native (konan)      0.66
+    Go                         0.75
+    Lua                        1.00
+    Haskell (GHC)              1.17
+    C++ (g++)                  1.24
+    Perl                       1.75
+    Java Native (GraalVM)      1.95
+    Crystal                    2.20
+    Bash                       3.37
+    JavaScript (Deno)         11.18
+    Python 3                  26.41
+    JavaScript (GraalVM)      35.58
+    Java                      56.40
+    Python 2                  57.00
+    JavaScript (Node.js)      71.61
+    Ruby                      77.30
+    Ruby (TruffleRuby)        91.28
+    Kotlin                   103.02
+    Scala                    801.21
+
+## PREREQUISITES
+
+- Ruby >= 2.4
+
+## Further Reading
+
+- [Response Times: The 3 Important Limits](https://www.nngroup.com/articles/response-times-3-important-limits/)
+- [100 milliseconds](http://cogsci.stackexchange.com/questions/1664/what-is-the-threshold-where-actions-are-perceived-as-instant)
+- [The Great Startup Problem](http://mail.openjdk.java.net/pipermail/mlvm-dev/2014-August/005866.html)
+
+## See Also
+
+- [gnustavo/startup-times](https://github.com/gnustavo/startup-times) - a script to investigate the startup times of several programming languages
+- [jwiegley/helloworld](https://github.com/jwiegley/helloworld) - a comparison of "Hello, world" startup times in various languages
+- [sharkdp/hyperfine](https://github.com/sharkdp/hyperfine) - a command-line benchmarking tool
+
+## Author
+
+[chocolateboy](mailto:chocolate@cpan.org)
+
+## Version
+
+1.0.0
+
+## Copyright and License
+
+Copyright © 2015-2019 by chocolateboy
+
+This benchmark game is free software; you can redistribute it and/or modify it under the
+terms of the [Artistic License 2.0](http://www.opensource.org/licenses/artistic-license-2.0.php).

data/bin/startup-time

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+
+# frozen_string_literal: true
+
+require 'cli-pasta'
+require_relative '../lib/startup_time'
+
+StartupTime::App.new.run

data/lib/startup_time.rb

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+require_relative 'startup_time/version'
+require_relative 'startup_time/services'
+require_relative 'startup_time/util'
+require_relative 'startup_time/app'
+require_relative 'startup_time/builder'
+require_relative 'startup_time/options'
+require_relative 'startup_time/registry'

data/lib/startup_time/app.rb

@@ -0,0 +1,142 @@
+# frozen_string_literal: true
+
+require 'benchmark'
+require 'komenda'
+require 'shellwords' # for Array#shelljoin
+require 'tty/table'
+
+module StartupTime
+  # StartupTime::App - the entry point for the app.
+  # selects an action based on the command-line options and runs it.
+  class App
+    EXPECTED_OUTPUT = /\AHello, world!\r?\n\z/
+
+    include Util # for `which`
+    include Services.mixin %i[builder ids_to_groups options selected_tests]
+
+    def initialize(args = ARGV)
+      @options = Options.new(args)
+      @verbosity = @options.verbosity
+      @times = []
+
+      # provide/publish the Options instance we've just created so it's
+      # available to other components
+      Services.once(:options) { @options }
+    end
+
+    # run the command corresponding to the command-line options:
+    # either an auxiliary command (e.g. clean the build dir
+    # or print a help message) or the default command, which runs
+    # the selected benchmark-tests
+    def run
+      if @verbosity == :verbose
+        # used by StartupTime::App#time to dump the command line
+        require 'shellwords'
+      end
+
+      case @options.action
+      when :clean
+        builder.clean!
+      when :help
+        puts @options.usage
+      when :show_ids
+        puts render_ids_to_groups
+      when :version
+        puts VERSION
+      else
+        benchmark
+      end
+    end
+
+    private
+
+    # run the selected benchmark tests:
+    #
+    # 1) ensure everything in the build directory is up to date
+    # 2) run the tests in random order and time each one
+    # 3) sort the results from the fastest to the slowest
+    # 4) display the results in the specified format (default: ASCII table)
+    def benchmark
+      builder.build!
+
+      selected_tests.entries.shuffle.each do |id, test|
+        time(id, test)
+      end
+
+      sorted = @times.sort_by { |result| result[:time] }
+
+      if options.format == :json
+        require 'json'
+        puts sorted.to_json
+      elsif !sorted.empty?
+        pairs = sorted.map { |result| [result[:name], '%.02f' % result[:time]] }
+        table = TTY::Table.new(['Test', 'Time (ms)'], pairs)
+        puts unless @verbosity == :quiet
+        puts table.render(:basic, alignments: %i[left right])
+      end
+    end
+
+    # an ASCII table representation of the mapping from test IDs (e.g. "scala")
+    # to group IDs (e.g. "compiled, jvm, slow")
+    def render_ids_to_groups
+      table = TTY::Table.new(%w[Test Groups], ids_to_groups)
+      table.render
+    end
+
+    # takes a test ID and a test spec and measures how long it takes to execute
+    # the test if either:
+    #
+    #   - its interpreter exists
+    #   - it's a compiled executable (i.e. its compiler exists)
+    #
+    # otherwise, skip the test
+    def time(id, test)
+      args = Array(test[:command])
+
+      if args.size == 1 # native executable
+        cmd = File.absolute_path(args.first)
+        return unless File.exist? cmd
+      else # interpreter + source
+        cmd = which(args.first)
+        return unless cmd
+      end
+
+      argv0 = args.shift
+      command = [cmd, *args]
+
+      # make sure the command produces the expected output
+      result = Komenda.run(command)
+      output = result.output
+
+      if result.error?
+        abort "error running #{id} (#{result.status}): #{output}"
+      end
+
+      unless output.match?(EXPECTED_OUTPUT)
+        abort "invalid output for #{id}: #{output.inspect}"
+      end
+
+      times = []
+
+      unless @verbosity == :quiet
+        if @verbosity == :verbose
+          puts "#{id}: #{command.shelljoin}"
+        else
+          print '.'
+        end
+      end
+
+      @options.rounds.times do
+        times << Benchmark.realtime do
+          system([cmd, argv0], *args, out: File::NULL)
+        end
+      end
+
+      @times << {
+        id: id,
+        name: test[:name],
+        time: (times.min * 1000).truncate(2)
+      }
+    end
+  end
+end