rundoc 0.0.2 → 1.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 7935527c5d717e2c4bdc0883a285dd3c7f834f65
-  data.tar.gz: ffc9bf341d82df737a5dc3ba4bb557160ae049b9
+SHA256:
+  metadata.gz: 64447cf981514498089c30f81d5a9885059db08ac2120b8bb700eed4089d0105
+  data.tar.gz: b46c802f280388c043adddcdfbe0a385fcf2bee8aa4c1fd76563452c7d24698c
 SHA512:
-  metadata.gz: e4839a376fc7911464bfd37ec2a64cc824963a52bef72d1e9e4876c07c8e1f5d6d1d99bab48243ef1c769cb98dabf186624dfe4ecf5a411480a6a748fd0293ab
-  data.tar.gz: 6cd4c38c55413efe85e174b561f39e99bc5788d21e2f63e93cef0c709563609a3864f704cb05c8c04b139d999ef789addeda7105b0b71c8f524890cc4aad2a4d
+  metadata.gz: cf89810f545312fd23e6759015473f5d64e9991f7348344649a4b672a3e97e774702ec04d5d3b91f64b8ebb14caecf78ac0f2d583d423b1c3f25c1915eaa1e5a
+  data.tar.gz: 9b7f288a0b23806729f07690faf8ec1b295c45652e7db07253aba38019c50d0e30a1cfb4b9973cc97ef36ac78bb8a1139228ed9b9fc5cbf15658770817a13e5e

data/.github/workflows/check_changelog.yml

@@ -0,0 +1,13 @@
+name: Check Changelog
+
+on:
+ pull_request:
+  types: [opened, reopened, edited, synchronize]
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v1
+    - name: Check that CHANGELOG is touched
+      run: |
+        cat $GITHUB_EVENT_PATH | jq .pull_request.title |  grep -i '\[\(\(changelog skip\)\|\(ci skip\)\)\]' ||  git diff remotes/origin/${{ github.base_ref }} --name-only | grep CHANGELOG.md

data/.gitignore

@@ -1,7 +1,14 @@
 test/fixtures/*/project/
 test/fixtures/*/tmp/
 test/fixtures/*/myapp/
+
+test/fixtures/*/*/project/
+test/fixtures/*/*/tmp/
+test/fixtures/*/*/myapp/
+
+
 *.gem
 
 .DS_Store
 Gemfile.lock
+.env

data/.travis.yml

@@ -0,0 +1,8 @@
+language: ruby
+rvm:
+  - 2.4
+
+sudo: false
+
+script: bundle exec rake test
+

data/CHANGELOG.md

@@ -0,0 +1,26 @@
+## HEAD
+
+## 1.1.2
+
+- Fix pipe support (https://github.com/schneems/rundoc/pull/28)
+
+## 1.1.1
+
+- Fix log read race condition (https://github.com/schneems/rundoc/pull/25)
+
+## 1.1.0
+
+- Pipe logic is now implemented through a parser (https://github.com/schneems/rundoc/pull/22)
+- Bugfix, background processes error when the log file is not touched before read, not sure why this is possible but here's a fix for it anyway (https://github.com/schneems/rundoc/commit/620ae55d8a5d3d443cf5e8cb77950a841f92900c)
+
+## 1.0.1
+
+- Allow non-headless browser sessions and navigation [#23](https://github.com/schneems/rundoc/pull/23)
+- Fix issue where a background task's log file was not present before it was attempted to be opened.
+- Allow composability of documents `rundoc.depend_on` and `rundoc.require` [#19](https://github.com/schneems/rundoc/pull/19)
+- The `rundoc` command is now `rundoc.configure`.
+- Ignore chdir warning since that affect is intentional [#20](https://github.com/schneems/rundoc/pull/20)
+
+## 1.0.0
+
+- Now using a propper PEG parser (parslet)

data/Dockerfile

@@ -0,0 +1,24 @@
+FROM ruby:2.6.3-stretch
+
+RUN useradd rundoc
+RUN curl https://cli-assets.heroku.com/install-ubuntu.sh | sh
+
+RUN apt-get clean && apt-get update && apt-get install -y locales nodejs
+
+RUN locale-gen en_US.UTF-8
+ENV LC_ALL=C.UTF-8
+ENV LANG=en_US.UTF-8
+ENV LANGUAGE=en_US.UTF-8
+ENV DISABLE_SPRING=1
+
+WORKDIR /home/rundoc
+RUN mkdir -p /home/rundoc/ && chown -R rundoc:rundoc /home/rundoc
+USER rundoc
+RUN mkdir -p /home/rundoc/workdir
+
+RUN git config --global user.email "developer@example.com"
+RUN git config --global user.name "Dev Eloper"
+
+ADD Gemfile Gemfile.lock
+
+RUN bundle install

data/Gemfile

@@ -1,3 +1,4 @@
 source "http://rubygems.org"
+gem 'm'
 
 gemspec

data/README.md

@@ -1,37 +1,32 @@
 # RunDOC
 
-![](https://www.dropbox.com/s/u354td51brynr4h/Screenshot%202017-05-09%2009.36.33.png?dl=1)
+![](https://www.dropbox.com/s/u354td51brynr4h/Screenshot%202017-05-09%2009.36.33.png?raw=1)
 
 ## What
 
-This library allows you to "run" your docs and embed the code as well as results back into the documentation.
+This library allows you to "run" your docs and embed the code as well as results back into the documentation. Here's a quick example:
 
-Write in a runDOC compatible markdown format, then run your docs to generate matching projects. Instead of writing a tutorial and then building an example separately, your documentation can build the example app for you. Not only does this keep your doc writing DRY, it also enforces consistency and accuracy. If you make a typo in your docs your project won't build...you'll get early warning and be able to fix it before it has the opportunity to confuse your reader.
+Write documentation:
 
-Think of runDOC as your ever-vigilant tech editor and writing partner.
+    Install by running:
 
-Once docs are run, they output a project and fully valid markdown doc (without any of the special runDOC tags). You could configure your project to be automatically pushed to github or anything else you want afterwards, check out the config section.
-
-Write more technical content, faster and with a better consistency by using runDOC.
-
-## Why
-
-I wrote a [Rails course for the University of Texas](http://schneems.com), that required I build an app and write
-documentation at the same time. I enjoyed the experience but having to do both essentially doubled my work load, worse than the time wasted copying snippets between the two was my docs were prone to paste errors, keyboard slips, or me just forgetting to add sections that I had implemented in the app. The only way for me to find these errors was to give the docs to someone to actually follow them and build the project. This method of manually checking is extremely time consuming, prone to errors (the developer may work around problems instead of reporting them to you), and makes making minor edits a major pain. Instead of writing your docs once and iterating, I found adding sections required me to start from scratch.
-
-
-While I was writing the course I dreamed of a system where the docs and the
-code could automatically stay in sync. One where if I had a typo in my tutorials, an automatic tech-editor would know and tell me. One where I couldn't accidentally skip over critical sections leaving true novices confused.
+    ```
+    :::>> $ gem install rails --no-document
+    ```
 
-Dream no more, because runDOC does just that:
+Now if you "run" this document you'll get this output:
 
-Write docs, build software.
+    Install by running:
 
-## Isn't this Overkill?
+    ```
+    $ gem install rails --no-document
+    Successfully installed rails-5.2.2
+    1 gem installed
+    ```
 
-No. Many new doc writers skip steps accidentally, or omit lines of code with `...` and assume their readers can follow along. Even if this is true for 80% of your users, 20% of people will become frustrated and may give up as a result. I found by including [check steps](http://schneems.com/post/60359275700/prepare-do-test-make-your-technical-writing-shine) such as running `ls` to ensure directory contents were the difference between good docs and great ones. The only problem: the output of `ls` on a Rails 4.0.0 and 4.0.1 project may be different. So the only way to ensure output is to actually run the command and copy it into your docs. With runDOC you don't need to do that. Rundoc runs the command then it can insert the output for you.
+The idea is that as your documentation "runs" it builds a working tutorial. It also acts as tests since if your docs become incorrect due to a typo or bit-rot then when you try to generate them, the process will fail.
 
-If you don't intend on updating or revising your content, then this project is overkill. On the other hand if you're writing docs without the intent of revising them, you probably shouldn't be writing technical docs.
+Think of RunDOC as your ever-vigilant tech editor and writing partner.
 
 ## Install
 
@@ -44,7 +39,7 @@ $ gem install rundoc
 or add it to your Gemfile:
 
 ```
-gem 'rundoc`
+gem 'rundoc'
 ```
 
 ## Use It
@@ -52,24 +47,57 @@ gem 'rundoc`
 Run the `rundoc build` command on any markdown file
 
 ```sh
-$ rundoc build --path runDOC.md
+$ rundoc build --path my/path/to/run_doc.md
 ```
 
-Note: This command will create and manipulate directories in the working directory of your source markdown file. Best practice is to have your source markdown file in it's own empty directory.
+> Note: This command will create and manipulate directories in the working directory of your source markdown file. Best practice is to have your source markdown file in its own empty directory.
 
 This will generate a project folder with your project in it, and a markdown README.md with the parsed output of the markdown docs, and a copy of the source.
 
-## Write it:
-
-Rundoc uses github flavored markdown. This means you write like normal but in your code sections
-you can add special annotations that when run through runDOC can
+## Quick docs
+
+- [Understanding the Syntax](#rundoc-syntax)
+- [Dotenv support](#dotenv-support)
+- [Rendering cheat sheet](#rendering-cheat-sheet)
+
+### Commands
+
+- Execute Bash Commands
+  - [$](#shell-commands)
+  - [fail.$](#shell-commands)
+- Chain commands
+  - [pipe](#pipe)
+  - [|](#pipe)
+- Manipulate Files
+  - [file.write](#file-commands)
+  - [file.append](#file-commands)
+  - [file.remove](#file-commands)
+- Boot background processes such as a local server
+  - [background.start](#background)
+  - [background.stop](#background)
+  - [background.log.read](#background)
+  - [background.log.clear](#background)
+- Take screenshots
+  - [website.visit](#screenshots)
+  - [website.nav](#screenshots)
+  - [website.screenshot](#screenshots)
+- Configure RunDOC
+  - [rundoc.configure](#configure)
+- Import and compose documents
+  - [rundoc.depend_on](#compose-multiple-rundoc-documents)
+  - [rundoc.require](#compose-multiple-rundoc-documents)
+
+## RunDOC Syntax
+
+RunDOC uses GitHub flavored markdown. This means you write like normal but in your code sections
+you can add special annotations that when run through RunDOC can
 generate a project.
 
-All runDOC commands are prefixed with three colons `:::` and are inclosed in a code block a
+All RunDOC commands are prefixed with three colons `:::` and are inclosed in a code block a
 command such as `$` which is an alias for `bash` commands like this:
 
     ```
-    :::> $ git init .
+    :::>- $ git init .
     ```
 
 Nothing before the three colons matters. The space between the colons
@@ -80,7 +108,7 @@ can add a minus symbol `-` to the end to prevent it from being
 rendered.
 
     ```
-    :::- $ git init .
+    :::-- $ git init .
     ```
 
 > Note: If all commands inside of a code block are hidden, the entire codeblock will not be rendered.
@@ -100,26 +128,7 @@ This code block might generate an output something like this to your markdown do
         Gemfile.lock  Rakefile  config    db    lib   public    test    vendor
     ```
 
-That's the syntax, let's look at different runDOC commands
-
-## Rendering Cheat Sheet
-
-An arrow `>` is shorthand for "render this" and a dash `-` is shorthand for skip this section. The posions two positions are command first and result second. You can skip a trailing `-`.
-
-
-- `:::>`  (yes command, not result)
-- `:::>>` (yes command, yes result)
-- `:::-`  (not command, not result)
-- `:::->` (not command, yes result)
-
-## Shell Commands
-
-Current Commands:
-
-- `$`
-- `fail.$`
-
-Anything you pass to `$` will be run in a shell. Any items below the command will be passed into the stdin of the bash command so:
+Any items below the command will be passed into the stdin of the command. For example using a `$` command you can effectively pipe contents to stdin:
 
     ```
     :::>> $ tail -n 2
@@ -139,7 +148,76 @@ Would output:
 
 This STDIN feature could be useful if you are running an interactive command such as `play new` which requires user input. For more fine grained input you'll need to use a custom repl object (will be covered later).
 
-If a shell command returns a non-zero exit status an error will be raised, if you expect a command to fail you can run it with `fail.$` keyword
+Different commands will do different things with this input. For example the `rundoc` command executes Ruby configuration code:
+
+    ```
+    :::-- rundoc
+    Rundoc.configure do |config|
+      config.after_build do
+        puts "you could push to GitHub here"
+        puts "You could do anything here"
+        puts "This code will run after the docs are done building"
+      end
+    end
+    ```
+
+
+And the `website.visit` command allows you to navigate and manipulate a webpage via a Capybara API:
+
+    ```
+    :::>> website.visit(name: "localhost", url: "http://localhost:3000", scroll: 100)
+    session.execute_script "window.scrollBy(0,100)"
+    session.click("sign up")
+    ```
+
+### Exact output
+
+RunDOC only cares about things that come after a `:::` section. If you have a "regular" code section, it will be rendered as as normal:
+
+    ```
+    $ echo "I won't run since i'm missing the :::>> at the front"
+    ```
+
+You can mix non-command code and commands, as long as the things that aren't rendering come first. This can be used to "fake" a command, for example:
+
+```
+$ rails new myapp # Not a command since it's missing the ":::>>""
+:::-> $ rails new myapp --skip-test --skip-yarn --skip-sprockets
+:::>> | $ head -n 5
+```
+
+This will render as:
+
+```
+$ rails new myapp # Not a command since it's missing the ":::>>""
+      create
+      create  README.md
+      create  Rakefile
+      create  .ruby-version
+      create  config.ru
+```
+
+It looks like the command was run without any flags, but in reality `rails new myapp --skip-test --skip-yarn --skip-sprockets | head -n 5` was executed.
+
+## Rendering Cheat Sheet
+
+An arrow `>` is shorthand for "render this" and a dash `-` is shorthand for skip this section. The two positions are **command** first and **result** second.
+
+
+- `:::>-` (YES command output, not result output)
+- `:::>>` (YES command output, YES result output)
+- `:::--` (not command output, not result output)
+- `:::->` (not command output, YES result output)
+
+## Shell Commands
+
+Current Commands:
+
+- `$`
+- `fail.$`
+
+Anything you pass to `$` will be run in a shell. If a shell command returns a non-zero exit status an error will be raised. If you expect a non-zero exit status use `fail.$` instead:
+
 
     ```
     :::>> fail.$ cat /dev/null/foo
@@ -164,7 +242,8 @@ Some commands may be custom, for example when running `cd` you likely want to ch
 However this command would fall on its face:
 
     ```
-    :::>> $ cd myapp/config && cat database.yml
+    :::>> $ cd myapp && cat config/database.yml
+    :::>> $ rails g scaffold users # <=== This command would be in the wrong directory, not `myapp`
     ```
 
 These custom commands are kept to a minimum, and for the most part behave as you would expect them to. Write your docs as you normally would and check the output frequently.
@@ -179,10 +258,10 @@ Current Commands:
 - `file.append`
 - `file.remove`
 
-Use the `file.write` keyword followed by a filename, on the next line(s) put the contents of the file
+Use the `file.write` keyword followed by a filename, on the next line(s) put the contents of the file:
 
     ```
-    :::> file.write config/routes.rb
+    :::>- file.write config/routes.rb
 
     Example::Application.routes.draw do
       root        :to => "pages#index"
@@ -193,10 +272,12 @@ Use the `file.write` keyword followed by a filename, on the next line(s) put the
     end
     ```
 
+> If the exact filename is not known you can use a [file glob (\*)](https://GitHub.com/schneems/rundoc/pull/6).
+
 If you wanted to change `users` to `products` you could write to the same file again.
 
     ```
-    :::> file.write config/routes.rb
+    :::>- file.write config/routes.rb
     Example::Application.routes.draw do
       root        :to => "pages#index"
 
@@ -256,27 +337,140 @@ Anything after the pipe `|` will generate a new command with the output of the p
 
 This command is currently hacked together, and needs a refactor. Use it, but if something does not behave as you would expected open an issue and explain it.
 
+## Background
+
+Sometimes you want to start a long lived process like a server in the background. In that case, the `background` namespace has your, well, back.
+
+To start a process, pass in the command as the first arg, and give it a name (so it can be referenced later):
+
+```
+:::>> background.start("rails server", name: "server")
+```
+
+You can make the background process wait until it receives a certain string in the logs. For instance to make sure that the server is fully booted:
+
+```
+:::>> background.start("rails server", name: "server", wait: "Listening on")
+```
+
+You can stop the process by referencing the name:
+
+```
+:::-- background.stop(name: "server")
+```
+
+You can also get the log contents:
+
+```
+:::>> background.log.read(name: "server")
+```
+
+You can also truncate the logs:
+
+```
+:::>> background.log.clear(name: "server")
+```
+
+## Screenshots
+
+You'll need selenium and `chromedriver` installed on your system to make screenshots work. On a mac you can run:
+
+```
+$ brew cask install chromedriver
+```
+
+To take a screenshot first "visit" a website. The values you pass in to stdin can be used to further navigate. For more information see the [Capybara DSL](https://www.rubydoc.info/GitHub/teamcapybara/capybara/master#the-dsl). Use the keyword `session`
+
+Once you're on the page you want to capture you can execute `website.screenshot`:
+
+```
+:::>> website.visit(name: "localhost", url: "http://localhost:3000", scroll: 100)
+session.execute_script "window.scrollBy(0,100)"
+session.first(:link, "sign up").click
+
+:::>> website.screenshot(name: "localhost")
+```
+
+The result of the screenshot command will be to replace the code section with a markdown link to a relative path of the screenshot.
+
+Once you've visited a website you can further navigate using `website.nav` or `website.navigate`:
+
+
+```
+:::>> website.visit(name: "localhost", url: "http://localhost:3000")
+:::>> website.navigate(name: "localhost")
+session.execute_script "window.scrollBy(0,100)"
+session.first(:link, "sign up").click
+
+:::>> website.screenshot(name: "localhost")
+```
+
+## Upload Screenshots
+
+You can specify that you want to upload files to S3 instead of hosting them locally by passing in `upload: "s3"` to the screenshot command:
+
+```
+:::>> website.visit(name: "localhost", url: "http://localhost:3000", scroll: 100)
+:::>> website.screenshot(name: "localhost", upload: "s3")
+```
+
+To authorize, you'll need to set these environment variables:
+
+```
+AWS_ACCESS_KEY_ID
+AWS_REGION
+AWS_SECRET_ACCESS_KEY
+AWS_BUCKET_NAME
+```
+
+The bucketeer addon on Heroku is supported out of the box. To specify project specific environment variables see the "dotenv" section below.
+
+## Compose multiple RunDOC documents
+
+If you're writing multiple tutorials that all are used together to build one larger project then you can declare dependencies inside of your RunDOC document.
+
+For example on day two (`day_two/rundoc.md`) of the tutorials you could:
+
+```
+:::-- rundoc.depend_on "../day_one/rundoc.md"
+```
+
+Now when you build `day_two/rundoc.md` it will also run the steps in `day_one/rundoc.md` first. This way you don't have to copy and paste previous commands.
+
+You can also break up your document into smaller components:
+
+
+```
+:::>> rundoc.require "../shared/rails_new.md"
+```
+
+This will replace the code section with the generated contents of `rundoc.require`.
+
+
+## Dotenv support
+
+If you need to specify project specific environment variables create a file called `.env` at the same directory as your `rundoc.md` and it will be imported. Add this file to your `.gitignore` so you don't accidentally share with the world
 
 ## Configure
 
-You can configure your docs in your docs use the `runDOC` command
+You can configure your docs in your docs use the `RunDOC` command
 
     ```
-    :::- rundoc
+    :::-- rundoc.configure
     ```
 
 Note: Make sure you run this as a hidden command (with `-`).
 
-** After Build**
+**After Build**
 
 This will eval any code you put under that line (in Ruby). If you want to run some code after you're done building your docs you could use `Rundoc.configure` block and call the `after_build` method like this:
 
 
     ```
-    :::- rundoc
+    :::-- rundoc.configure
     Rundoc.configure do |config|
       config.after_build do
-        puts "you could push to github here"
+        puts "you could push to GitHub here"
         puts "You could do anything here"
         puts "This code will run after the docs are done building"
       end
@@ -286,10 +480,10 @@ This will eval any code you put under that line (in Ruby). If you want to run so
 
 **Project Root**
 
-By default your app builds in a `tmp` directory. If any failures occur the results will remain in `tmp`. On a successful build the contents are copied over to `project`. If you are generating a new rails project in your code `$ rails new myapp`. Then the finished directory would be in `project/myapp`. If you don't like the `./project` prefix you could tell runDOC to output contents in `./myapp` instead.
+By default your app builds in a `tmp` directory. If any failures occur the results will remain in `tmp`. On a successful build the contents are copied over to `project`. If you are generating a new rails project in your code `$ rails new myapp`. Then the finished directory would be in `project/myapp`. If you don't like the `./project` prefix you could tell RunDOC to output contents in `./myapp` instead.
 
     ```
-    :::- rundoc
+    :::-- rundoc.configure
     Rundoc.configure do |config|
       config.project_root = "myapp"
     end
@@ -302,39 +496,14 @@ This will also be the root directory that the `after_build` is executed in.
 Sometimes sensitive info like usernames, email addresses, or passwords may be introduced to the output readme. Let's say that your email address was `schneems@example.com` you could filter this out of your final document and replace it with `developer@example.com` instead like this:
 
     ```
-    :::- rundoc
+    :::-- rundoc.configure
     Rundoc.configure do |config|
-      config.filter_sensitive("schneems@exmaple.com" => "developer@example.com")
+      config.filter_sensitive("schneems@example.com" => "developer@example.com")
     end
     ```
 
 This command `filter_sensitive` can be called multiple times with different values. Since the config is in Ruby you could iterate over an array of sensitive data
 
-## TODO
-
-This is a section for brainstorming. If it's here it's not guaranteed to get worked on, but it will be considered.
-
-- Seperate parsing from running. This will help for easier linting of syntax etc.
-- Cache SHAs and output of each code block. If one sha changes, re-generate all code blocks, otherwise allow a re-render without code execution. Configure a check sum for busting the cache for instance a new version of Rails is released.
-
-
-- A way to run background processes indefinitely such as `rails server`
-  - Maybe a way to truncate them after only a period of time such as grab a few lines of `heroku local`.
-
-
-    ```
-    :::> background.start(command: "rails server")
-    ```
-
-    ```
-    :::>> background.read("rails server")
-    :::> | $ head -n 23
-    :::> background.clear
-    ```
-
+## Copyright
 
-  - Breakpoints?
-- Better line matching for backtrace
-- `-=` command (runs command, only shows output, does not show command) ?
-- An easy test syntax?
-- Screenshot tool(s) ?!?!?!?!?!?! :)
+All content Copyright Richard Schneeman © 2020