rundoc 2.0.1 → 3.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4ffd4e661f146495e1f5a1c450ab4b053a0c625a980ef877c9d1b77b1061a9a0
-  data.tar.gz: e2fa4c55862c6bb84a5aa5c9fd393942a47991ec8b7af6ecfe74a6bcfae91d3f
+  metadata.gz: 89f953193be8aea0831e65e251a1d0cc8a3796573fe5818a6eb4ae2ed0e489d4
+  data.tar.gz: 9ea2f3a42906fa0c79cbb39862ec5300b4688dcce15df8991cf0a99e2d7c60da
 SHA512:
-  metadata.gz: 5ebdd506647124c812344520d0ed85a3c28cfc0443fba6d1436a3605c7b2156f99a9620f58bc56125303aeca3a33eef5bc84a17cc7031d9d86793c498203b492
-  data.tar.gz: 1dec5d073813c7b3853ee29b9e6425a74e201eb14adb47c9eb662f17ccc5e1e35644d463e93e80d5bf315e235ff50d9533e9c67ebd43f1041770d6fd154d3f7c
+  metadata.gz: 4aba6794da79ce27d6a4072e0cda4c7e2525030f0bd7c7925aaca8708a3adbd2632a0bb32c2ea451bc5f58ebc4a674ab4012ec6ddaeae7114ffbcefbafb53b35
+  data.tar.gz: fef87625649965fdd3dd5817e55d73299e1ebf67cfb3f9895c45d2c26b410af566f43540490a7c4ec7198b31f86d9c14079701b25981f7689b99644775f8e55f

data/.gitignore

@@ -1,14 +1,17 @@
-test/fixtures/*/project/
-test/fixtures/*/tmp/
-test/fixtures/*/myapp/
-
-test/fixtures/*/*/project/
-test/fixtures/*/*/tmp/
-test/fixtures/*/*/myapp/
+test/fixtures/**/*/rundoc_output/
+test/fixtures/**/*/rundoc_failure/
+test/fixtures/**/*/project/
+test/fixtures/**/*/tmp/
+test/fixtures/**/*/myapp/
 
+coverage/
 
 *.gem
 
 .DS_Store
 Gemfile.lock
 .env
+
+# IDEs
+.idea/
+.vscode/

data/CHANGELOG.md

@@ -1,5 +1,20 @@
 ## HEAD
 
+## 3.0.1
+
+- Fix: Save in-progress work in the "failure" directory when the rundoc command is interrupted via a signal such as `SIGTERM` (https://github.com/zombocom/rundoc/pull/67)
+
+## 3.0.0
+
+- Change: Default directories are now `rundoc_output` (instead of `project`) and `rundoc_failure` (instead of `tmp`).
+- Change: Non-empty directories for success or failure paths will now halt execution unless `--force` is used
+- Change: The CLI command `rundoc build --path <path>` is now `rundoc <path>`
+- Change: `rundoc.depend_on` is removed in favor of `:::-- rundoc.require` (https://github.com/zombocom/rundoc/pull/58)
+- Change: `Rundoc.project_root=` is removed, please use `Rundoc.after_build` instead (https://github.com/zombocom/rundoc/pull/58)
+- Change: Location of screenshots is now consistent (https://github.com/zombocom/rundoc/pull/57)
+- Fix: Bash commands now stream their outputs while they're running (https://github.com/zombocom/rundoc/pull/57)
+- Change: Minimum selenium-webdriver is now 4.x. (https://github.com/zombocom/rundoc/pull/41)
+
 ## 2.0.1
 
 - Bugfix: Code blocks without a following newline are now being recognized and executed (https://github.com/zombocom/rundoc/pull/51)

data/CONTRIBUTING.md

@@ -0,0 +1,25 @@
+# Contributing
+
+## Run the tests
+
+```
+$ bundle exec rake test
+```
+
+## Run one test
+
+```
+$ bundle exec m <path/to/file.rb>
+```
+
+## Run the linter
+
+```
+$ bundle exec standardrb
+```
+
+## Run the local cli
+
+```
+$ bin/rundoc <path/to/rundoc.md>
+```

data/README.md

@@ -4,29 +4,37 @@
 
 ## What
 
-This library allows you to "run" your docs and embed the code as well as results back into the documentation. Here's a quick example:
+Turn your tutorials into tests and never let your docs be out of date again.
 
-Write documentation:
+Start off by writing your tutorial in modified-markdown, then execute it with `rundoc`. If there's a problem with following the directions, then your tutorial will fail to build. When it succeeds, the  real world output is embedded in the output markdown file. That means your tutorials will have the EXACT output that your readers will see.
 
-    Install by running:
+## Quickstart
 
-    ```
-    :::>> $ gem install rails --no-document
-    ```
+Install the Ruby library:
 
-Now if you "run" this document you'll get this output:
+    $ gem install rundoc
 
-    Install by running:
+Make a rundoc file:
 
+    $ mkdir /tmp/rundoc-demo
+    $ cd /tmp/rundoc-demo
+    $ cat <<'EOF' > ./RUNDOC.md
     ```
-    $ gem install rails --no-document
-    Successfully installed rails-5.2.2
-    1 gem installed
+    :::>> $ echo Hello World
     ```
+    EOF
 
-The idea is that as your documentation "runs" it builds a working tutorial. It also acts as tests since if your docs become incorrect due to a typo or bit-rot then when you try to generate them, the process will fail.
+Run it:
 
-Think of RunDOC as your ever-vigilant tech editor and writing partner.
+    $ rundoc --on-success-dir=rundoc_output ./RUNDOC.md
+
+View the output
+
+    $ cat rundoc_output/README.md
+    ```
+    $ echo Hello World
+    Hello World
+    ```
 
 ## Install
 
@@ -44,15 +52,15 @@ gem 'rundoc'
 
 ## Use It
 
-Run the `rundoc build` command on any markdown file
+Run the `rundoc` command on any rundoc-flavored markdown file:
 
 ```sh
-$ bin/rundoc build --path <test/fixtures/rails_7/rundoc.md>
+$ rundoc <test/fixtures/rails_7/rundoc.md>
 ```
 
 > Note: This command will create and manipulate directories in the working directory of your source markdown file. Best practice is to have your source markdown file in its own empty directory.
 
-This will generate a project folder with your project in it, and a markdown README.md with the parsed output of the markdown docs, and a copy of the source.
+This will generate a project folder with your project in it, and a markdown `README.md` with the parsed output of the markdown docs. See `rundoc --help` for more configuration options.
 
 ## Quick docs
 
@@ -87,7 +95,6 @@ This will generate a project folder with your project in it, and a markdown READ
 - Configure RunDOC
   - [rundoc.configure](#configure)
 - Import and compose documents
-  - [rundoc.depend_on](#compose-multiple-rundoc-documents)
   - [rundoc.require](#compose-multiple-rundoc-documents)
 
 ## RunDOC Syntax
@@ -164,7 +171,6 @@ Different commands will do different things with this input. For example the `ru
     end
     ```
 
-
 And the `website.visit` command allows you to navigate and manipulate a webpage via a Capybara API:
 
     ```
@@ -220,14 +226,12 @@ Current Commands:
 
 Anything you pass to `$` will be run in a shell. If a shell command returns a non-zero exit status an error will be raised. If you expect a non-zero exit status use `fail.$` instead:
 
-
     ```
     :::>> fail.$ cat /dev/null/foo
     ```
 
 Even though this command returns a non zero exit status, the contents of the command will be written since we're stating that we don't care if the command fails. This would be the output:
 
-
     ```
     $ cat /dev/null/foo
     cat: /dev/null/foo: Not a directory
@@ -235,7 +239,6 @@ Even though this command returns a non zero exit status, the contents of the com
 
 Some commands may be custom, for example when running `cd` you likely want to change the working directory that your script is running in. To do this we need to run `Dir.chdir` instead of shelling out. So this works as you would expect:
 
-
     ```
     :::>> $ cd myapp/config
     :::>> $ cat database.yml
@@ -455,7 +458,6 @@ The result of the screenshot command will be to replace the code section with a
 
 Once you've visited a website you can further navigate using `website.nav` or `website.navigate`:
 
-
 ```
 :::>> website.visit(name: "localhost", url: "http://localhost:3000")
 :::>> website.navigate(name: "localhost")
@@ -487,26 +489,20 @@ The bucketeer addon on Heroku is supported out of the box. To specify project sp
 
 ## Compose multiple RunDOC documents
 
-If you're writing multiple tutorials that all are used together to build one larger project then you can declare dependencies inside of your RunDOC document.
-
-For example on day two (`day_two/rundoc.md`) of the tutorials you could:
+You can also break up your document into smaller components using `rundoc.require`:
 
 ```
-:::-- rundoc.depend_on "../day_one/rundoc.md"
+:::>> rundoc.require "../day_one/rundoc.md"
 ```
 
-Now when you build `day_two/rundoc.md` it will also run the steps in `day_one/rundoc.md` first. This way you don't have to copy and paste previous commands.
-
-You can also break up your document into smaller components:
+This will prepend the code section with the generated contents of `rundoc.require`.
 
+If you want to execute another tutorial as a pre-requisite but not embed the results you can use `:::--`:
 
 ```
-:::>> rundoc.require "../shared/rails_new.md"
+:::-- rundoc.require "../day_one/rundoc.md"
 ```
 
-This will replace the code section with the generated contents of `rundoc.require`.
-
-
 ## Dotenv support
 
 If you need to specify project specific environment variables create a file called `.env` at the same directory as your `rundoc.md` and it will be imported. Add this file to your `.gitignore` so you don't accidentally share with the world
@@ -523,13 +519,12 @@ Note: Make sure you run this as a hidden command (with `-`).
 
 **After Build**
 
-This will eval any code you put under that line (in Ruby). If you want to run some code after you're done building your docs you could use `Rundoc.configure` block and call the `after_build` method like this:
-
+This will eval any code you put under that line (in Ruby) when the build was successful but before the contents are finalized on disk. If you want to run some code after you're done building your docs you could use `Rundoc.configure` block and call the `after_build` method like this:
 
     ```
     :::-- rundoc.configure
     Rundoc.configure do |config|
-      config.after_build do
+      config.after_build do |context|
         puts "you could push to GitHub here"
         puts "You could do anything here"
         puts "This code will run after the docs are done building"
@@ -537,19 +532,11 @@ This will eval any code you put under that line (in Ruby). If you want to run so
     end
     ```
 
+The `context` object will have details about the structure of the output directory structure. The stable API is:
 
-**Project Root**
-
-By default your app builds in a `tmp` directory. If any failures occur the results will remain in `tmp`. On a successful build the contents are copied over to `project`. If you are generating a new rails project in your code `$ rails new myapp`. Then the finished directory would be in `project/myapp`. If you don't like the `./project` prefix you could tell RunDOC to output contents in `./myapp` instead.
-
-    ```
-    :::-- rundoc.configure
-    Rundoc.configure do |config|
-      config.project_root = "myapp"
-    end
-    ```
-
-This will also be the root directory that the `after_build` is executed in.
+- `context.output_dir`: A [Pathname](https://rubyapi.org/3.3/o/pathname) containing the absolute path to the top level directory where all commands are were executed. If your script runs `rails new myapp` then this directory would contain another directory named `myapp`. Only modifications to this directory will be persisted to the final `--output-dir`.
+- `context.screenshots_dir`: A [Pathname](https://rubyapi.org/3.3/o/pathname) containing the absolute path to the directory where screenshots were saved. It is guaranteed to be somewhere within the `context.output_dir`
+- `context.output_markdown_path`: A [Pathname](https://rubyapi.org/3.3/o/pathname) containing the absolute path to the final markdown file. This is guaranteed to be in the `context.output_dir`
 
 **Filter Sensitive Info**
 

data/Rakefile

@@ -10,6 +10,10 @@ task default: [:test]
 Rake::TestTask.new(:test) do |t|
   t.libs << "lib"
   t.libs << "test"
-  t.pattern = "test/rundoc/**/*_test.rb"
+  t.pattern = [
+    "test/rundoc/**/*_test.rb",
+    "test/system/**/*_test.rb",
+    "test/integration/**/*_test.rb"
+  ]
   t.verbose = false
 end

data/bin/rundoc

@@ -1,6 +1,7 @@
 #!/usr/bin/env ruby
 
 $stdout.sync = true
+$stderr.sync = true
 
 unless File.respond_to? :realpath
   class File #:nodoc:
@@ -13,22 +14,8 @@ end
 $: << File.expand_path(File.dirname(File.realpath(__FILE__)) + '/../lib')
 
 require 'rundoc'
-require 'thor'
 
-class RundocThorCLI < Thor
+options = Rundoc::CLIArgumentParser.new(argv: ARGV).call.options
 
-  def initialize(*args)
-    super
-    @path = options[:path]
-  end
-
-  default_task :help
-
-  desc "build", "turns rundoc file into docs and a project"
-  class_option  :path,   banner: "path/to/file.md",          optional: true, default: 'rundoc.md'
-  def build
-    Rundoc::CLI.new.build(path: @path)
-  end
-end
-
-RundocThorCLI.start(ARGV)
+cli = Rundoc::CLI.new(**options)
+cli.call

data/lib/rundoc/cli.rb

@@ -1,82 +1,241 @@
 module Rundoc
   class CLI
-    def build(path:)
-      @path = Pathname.new(path).expand_path
-      raise "#{@path} does not exist" unless File.exist?(@path)
-      raise "Expecting #{@path} to be a rundoc markdown file" unless File.file?(@path)
-      @working_dir = Pathname.new(File.expand_path("../", @path))
-
-      dot_env_path = File.expand_path("../.env", @path)
-      if File.exist?(dot_env_path)
+    module DEFAULTS
+      ON_FAILURE_DIR = # <path/to/rundoc.md/..> +
+        "rundoc_failure"
+      ON_SUCCESS_DIR = # <path/to/rundoc.md/..> +
+        "rundoc_output"
+      DOTENV_PATH = # <path/to/rundoc.md/..> +
+        ".env"
+      OUTPUT_FILENAME = "README.md"
+      SCREENSHOTS_DIR = "screenshots"
+    end
+
+    attr_reader :io, :on_success_dir, :on_failure_dir, :dotenv_path, :cli_cmd, :cli_args, :force
+    attr_reader :execution_context, :after_build_context
+
+    def initialize(
+      source_path:,
+      io: $stderr,
+      cli_cmd: $0,
+      cli_args: $*,
+      force: false,
+      dotenv_path: nil,
+      on_success_dir: nil,
+      on_failure_dir: nil,
+      output_filename: nil,
+      screenshots_dirname: nil
+    )
+      @io = io
+      @force = force
+      @cli_cmd = cli_cmd
+      @cli_args = cli_args
+
+      screenshots_dirname = check_relative_path(screenshots_dirname || DEFAULTS::SCREENSHOTS_DIR)
+      output_filename = check_relative_path(output_filename || DEFAULTS::OUTPUT_FILENAME)
+
+      @execution_context = Rundoc::Context::Execution.new(
+        output_dir: Dir.mktmpdir,
+        source_path: source_path,
+        screenshots_dirname: screenshots_dirname
+      )
+
+      @after_build_context = Context::AfterBuild.new(
+        output_dir: execution_context.output_dir,
+        screenshots_dir: execution_context.screenshots_dir,
+        output_markdown_path: @execution_context.output_dir.join(output_filename)
+      )
+
+      @dotenv_path = if dotenv_path
+        Pathname(dotenv_path)
+      else
+        @execution_context.source_dir.join(DEFAULTS::DOTENV_PATH)
+      end
+
+      @on_success_dir = if on_success_dir
+        Pathname(on_success_dir)
+      else
+        @execution_context.source_dir.join(DEFAULTS::ON_SUCCESS_DIR)
+      end.expand_path
+
+      @on_failure_dir = if on_failure_dir
+        Pathname(on_failure_dir)
+      else
+        @execution_context.source_dir.join(DEFAULTS::ON_FAILURE_DIR)
+      end.expand_path
+    end
+
+    def force?
+      force
+    end
+
+    # Ensures that the value passed in cannot escape the current directory
+    #
+    # Examples:
+    #
+    # - `/tmp/whatever` is invalid
+    # - `whatever/../..` is invalid
+    #
+    private def check_relative_path(path)
+      path = Pathname(path)
+      if path.absolute?
+        raise "Path must be relative but it is not: #{path}"
+      end
+
+      if path.each_filename.any? { |part| part == ".." }
+        raise "path cannot contain `..` but it does: #{path}"
+      end
+      path
+    end
+
+    def load_dotenv
+      if File.exist?(dotenv_path)
+        io.puts("Found .env file #{dotenv_path}, loading")
         require "dotenv"
-        Dotenv.load(dot_env_path)
-        ENV["AWS_ACCESS_KEY_ID"] ||= ENV["BUCKETEER_AWS_ACCESS_KEY_ID"]
+        Dotenv.load(dotenv_path)
         ENV["AWS_REGION"] ||= ENV["BUCKETEER_AWS_REGION"]
-        ENV["AWS_SECRET_ACCESS_KEY"] ||= ENV["BUCKETEER_AWS_SECRET_ACCESS_KEY"]
         ENV["AWS_BUCKET_NAME"] ||= ENV["BUCKETEER_BUCKET_NAME"]
+        ENV["AWS_ACCESS_KEY_ID"] ||= ENV["BUCKETEER_AWS_ACCESS_KEY_ID"]
+        ENV["AWS_SECRET_ACCESS_KEY"] ||= ENV["BUCKETEER_AWS_SECRET_ACCESS_KEY"]
+      else
+        io.puts("## No .env file found #{dotenv_path}, skipping dotenv loading")
       end
+    end
 
-      source_contents = File.read(@path)
-      tmp_dir = @working_dir.join("tmp")
-
-      FileUtils.remove_entry_secure(tmp_dir) if tmp_dir.exist?
-      tmp_dir.mkdir
-      banner = <<~HEREDOC
+    def prepend_cli_banner(contents)
+      <<~HEREDOC
         <!-- STOP
           This file was generated by a rundoc script, do not modify it.
 
           Instead modify the rundoc script and re-run it.
 
-          Command: #{$0} #{$*.join(" ")}
+          Command: #{cli_cmd} #{cli_args.join(" ")}
         STOP -->
+        #{contents}
       HEREDOC
+    end
+
+    def check_directories_empty!
+      [on_success_dir, on_failure_dir].each do |dir|
+        dir.mkpath
+
+        next if Dir.empty?(dir)
 
-      puts "== Running your docs"
-      Dir.chdir(tmp_dir) do
-        @output = Rundoc::Parser.new(source_contents, document_path: @path).to_md
-        Rundoc.sanitize(@output)
-        @output = "#{banner}\n#{@output}"
+        if force?
+          io.puts "## WARNING: #{dir} is not empty, it may be cleared due to running with the `--force` flag"
+        else
+          raise "## ABORTING: #{dir} is not empty, clear it or re-run with `--force` flag"
+        end
       end
+    end
 
-      puts "== Done, run was successful"
-      project_name = if Rundoc.project_root
-        Rundoc.project_root.split("/").last
-      else
-        "project"
+    def call
+      io.puts "## Running your docs"
+      load_dotenv
+      check_directories_empty!
+
+      source_contents = execution_context.source_path.read
+      if on_failure_dir.exist? && !Dir.empty?(on_failure_dir)
+        io.puts "## earing on failure directory #{on_failure_dir}"
+        clean_dir(
+          dir: on_failure_dir,
+          description: "on failure directory"
+        )
       end
 
-      project_dir = @working_dir.join(project_name)
+      io.puts "## Working dir is #{execution_context.output_dir}"
+      Dir.chdir(execution_context.output_dir) do
+        parser = Rundoc::Parser.new(
+          source_contents,
+          context: execution_context
+        )
+        output = begin
+          parser.to_md
+        rescue StandardError, SignalException => e
+          warn "Received exception: #{e.inspect}, cleaning up before re-raise"
+          on_fail
+          raise e
+        end
+
+        on_success(output)
+      end
+    ensure
+      # Stop any hanging background tasks
+      Rundoc::CodeCommand::Background::ProcessSpawn.tasks.each do |name, task|
+        next unless task.alive?
 
-      FileUtils.remove_entry_secure(project_dir) if project_dir.exist?
+        io.puts "Warning background task is still running, cleaning up: name: #{name}"
+        task.stop
+      end
 
-      cp_root = if Rundoc.project_root
-        tmp_dir.join(Rundoc.project_root, ".")
-      else
-        tmp_dir.join(".")
+      if execution_context.output_dir.exist?
+        clean_dir(
+          dir: execution_context.output_dir,
+          description: "tmp working directory"
+        )
       end
+    end
+
+    private def clean_dir(dir:, description:)
+      io.puts "## Cleaning #{description}, #{dir}"
+
+      FileUtils.remove_entry_secure(dir)
+    end
 
-      FileUtils.cp_r(cp_root, project_dir)
+    private def on_fail
+      io.puts "## Failed, debug contents are in #{on_failure_dir}"
+      on_failure_dir.mkpath
 
-      FileUtils.remove_entry_secure(tmp_dir) if tmp_dir.exist?
+      move_dir_contents(
+        from: execution_context.output_dir,
+        to: on_failure_dir
+      )
+    end
 
-      source_path = project_dir.join("README.md")
-      puts "== Done, writing original source to #{source_path}"
-      File.write(source_path, @output)
+    private def on_success(output)
+      io.puts "## Success! sanitizing output"
+      Rundoc.sanitize!(output)
+      output = prepend_cli_banner(output)
 
-      puts "== Copying source"
-      source_path = project_dir.join("copied-#{@path.to_s.split("/").last}")
-      File.write(source_path, source_contents)
+      io.puts "## Writing RUNdoc output to #{after_build_context.output_markdown_path}"
+      after_build_context.output_markdown_path.write(output)
 
-      Dir.chdir(project_dir) do
-        Rundoc.run_after_build
+      begin
+        io.puts "## Running after build scripts "
+        Rundoc.run_after_build(after_build_context)
+      rescue => e
+        on_fail
+        raise e
       end
-    ensure
-      Rundoc::CodeCommand::Background::ProcessSpawn.tasks.each do |name, task|
-        next unless task.alive?
 
-        puts "Warning background task is still running, cleaning up: name: #{name}"
-        task.stop
+      io.puts "## Saving to #{on_success_dir}"
+      if on_success_dir.exist?
+        clean_dir(
+          dir: on_success_dir,
+          description: "on success directory"
+        )
+      end
+      on_success_dir.mkpath
+
+      move_dir_contents(
+        from: execution_context.output_dir,
+        to: on_success_dir
+      )
+
+      io.puts "## Finished"
+    end
+
+    def move_dir_contents(from:, to:)
+      io.puts("## Moving contents from #{from} to #{to}")
+
+      Dir.glob(File.join(from, "{*,.*}")).each do |item|
+        next if item.end_with?(".", "..")
+
+        FileUtils.mv(item, to)
       end
     end
   end
 end
+
+require_relative "context/execution"
+require_relative "context/after_build"