jets 0.0.1 → 0.2.0

data/lib/jets/util.rb

@@ -0,0 +1,13 @@
+module Jets::Util
+  # Ensures trailing slash
+  # Useful for appending a './' in front of a path or leaving it alone.
+  # Returns: '/path/with/trailing/slash/' or './'
+  @@root = nil
+  def root
+    return @@root if @@root
+    @@root = ENV['PROJECT_ROOT'].to_s
+    @@root = '.' if @@root == ''
+    @@root = "#{@@root}/" unless @@root.ends_with?('/')
+    @@root
+  end
+end

data/lib/jets/version.rb

@@ -1,3 +1,3 @@
 module Jets
-  VERSION = "0.0.1"
+  VERSION = "0.2.0"
 end

data/notes/design.md

@@ -0,0 +1,107 @@
+## What is Jets?
+
+Jet's is a Lambda Framework that allows you to create AWS Lambda serverless applications with ruby.  It includes everything required to build a web application and deploy it to AWS Lambda.  It also can be used as a cost-effective worker system. For example, it can be used to write "cron" jobs.
+
+It is key to conceptually understand AWS Lambda and API Gateway to understand Jets.  Jets maps your code to Lambda functions and API Gatway.
+
+* **AWS Lambda** is Functions as a Service. It allows you to upload and run functions without worrying about the underlying infrastructure.
+* **API Gateway** is a routing layer for Lambda. You can use it to route url endpoints to your Lambda functions.
+
+## How It Works
+
+With Jets, you focus on your business logic and Jet's does the mechanical work. You write controllers, workers and functions and Jets automatically wire these up to Lambda and API Gateway for you.
+
+### What is a Jets controller?
+
+A Jets controller handles a web request and rendering a response back to the user.  Here's an example
+
+`app/controllers/posts_controller.rb`:
+
+```ruby
+class PostsController < Jets::BaseController
+  def create
+    # event and context is automatically available as a Hash
+    # render returns Lambda Proxy struture for web requests
+    render json: event, status: 200
+  end
+
+  def update
+    render json: event.merge(a: "update"), status: 200
+  end
+end
+```
+
+Jets creates up lambda functions for each of the controller public methods above when you run the `jets deploy` command.
+
+### How do I connect API Gateway to the Lambda functions?
+
+You can hook the Lambda functions to url endpoints via API Gateway.  To route an API Gateway endpoint to the controller action, add the following to your routes file:
+
+`config/routes.rb`:
+
+```ruby
+# API Gateway resources are only created if the controller action exists.
+get  "posts", to: "posts#index"
+get  "posts/:id", to: "posts#show"
+post "posts", to: "posts#create"
+get  "posts/:id/edit", to: "posts#edit"
+put  "posts", to: "posts#update"
+delete  "posts", to: "posts#destroy"
+
+resources :posts # macro that will creaet all the routes above
+
+any "posts/hot", to: "posts#hot" # GET, POST, PUT, etc request all work
+```
+
+Run the `jets deploy` command to add the routes to API Gateway.
+
+### Project Structure
+
+Here's an overview of the project structure.
+
+File / Directory  | Description
+------------- | -------------
+app/controllers  | controllers code
+app/workers  | workers code
+app/functions  | functions code
+config/project.yml  | project configs
+config/events.yml  | events configs
+config/routes.rb  | routes
+
+
+## Usage
+
+### Quick Start
+
+```sh
+jets deploy
+
+jets deploy
+jets deploy function xxx
+jets deploy controller xxx
+jets deploy worker xxx
+```
+
+### Scaffolding
+
+
+
+## Under the hood
+
+Lambda does not yet support ruby. So Jets uses a node shim and a bundled version of ruby to add support.
+
+## Test
+
+Testing controller processing without node shim.
+
+```
+jets process controller '{ "we" : "love", "using" : "Lambda" }' '{"test": "1"}' "handlers/controllers/posts.create"
+```
+
+Testing the generated node shim handler and the controller processing.
+
+```
+cd spec/fixtures/project
+jets build # generates the handlers
+node handlers/controllers/posts.js
+```

data/notes/faq.md

@@ -0,0 +1,3 @@
+Q: Is it slower using a node shim?
+Yes, the node shim adds a little bit of overhead. The overhead is of the node shim is about 500ms with a function configured to use 1.5GB of RAM. If your application requires faster response times than than then you will have to wait for official ruby support from AWS.
+

data/notes/lambda_ruby_info.md

@@ -0,0 +1,34 @@
+```sh
+RUBY_VERSION 2.2.2
+/var/task/bundled/gems/ruby/2.2.0/gems/bin/print_ruby_info
+LOAD_PATH:
+/var/task/bundled/gems/ruby/2.2.0/gems/print_ruby_info-0.0.3/lib
+/var/task/bundled/gems/ruby/2.2.0/gems/jets-0.1.1/lib
+/var/task/bundled/gems/ruby/2.2.0/gems/thor-0.20.0/lib
+/var/task/bundled/gems/ruby/2.2.0/gems/hashie-3.5.6/lib
+/var/task/bundled/gems/ruby/2.2.0/gems/colorize-0.8.1/lib
+/var/task/bundled/ruby/lib/ruby/gems/2.2.0/gems/bundler-1.9.9/lib/gems/bundler-1.9.9/lib
+/var/task/bundled/gems/ruby/2.2.0/gems/activesupport-5.1.4/lib
+/var/task/bundled/gems/ruby/2.2.0/gems/tzinfo-1.2.3/lib
+/var/task/bundled/gems/ruby/2.2.0/gems/thread_safe-0.3.6/lib
+/var/task/bundled/gems/ruby/2.2.0/gems/minitest-5.10.3/lib
+/var/task/bundled/gems/ruby/2.2.0/gems/i18n-0.9.0/lib
+/var/task/bundled/gems/ruby/2.2.0/gems/concurrent-ruby-1.0.5/lib
+/var/task/bundled/ruby/lib/ruby/gems/2.2.0/gems/bundler-1.9.9/lib
+/var/task/bundled/ruby/lib/ruby/site_ruby/2.2.0
+/var/task/bundled/ruby/lib/ruby/site_ruby/2.2.0/x86_64-linux
+/var/task/bundled/ruby/lib/ruby/site_ruby
+/var/task/bundled/ruby/lib/ruby/vendor_ruby/2.2.0
+/var/task/bundled/ruby/lib/ruby/vendor_ruby/2.2.0/x86_64-linux
+/var/task/bundled/ruby/lib/ruby/vendor_ruby
+/var/task/bundled/ruby/lib/ruby/2.2.0
+/var/task/bundled/ruby/lib/ruby/2.2.0/x86_64-linux
+/tmp/ruby/lib/ruby/site_ruby/2.2.0
+/tmp/ruby/lib/ruby/site_ruby/2.2.0/x86_64-linux
+/tmp/ruby/lib/ruby/site_ruby
+/tmp/ruby/lib/ruby/vendor_ruby/2.2.0
+/tmp/ruby/lib/ruby/vendor_ruby/2.2.0/x86_64-linux
+/tmp/ruby/lib/ruby/vendor_ruby
+/tmp/ruby/lib/ruby/2.2.0
+/tmp/ruby/lib/ruby/2.2.0/x86_64-linux
+```

data/notes/traveling-ruby-packaging-jets.md

@@ -0,0 +1,26 @@
+# start off at project root
+
+mkdir -p /tmp/jets_build
+cp Gemfile* /tmp/jets_build/
+cd /tmp/jets_build # cd into there to build TravelingRuby
+
+wget http://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20150715-2.2.2-linux-x86_64.tar.gz .
+mkdir -p bundled/ruby # tmp/jets_build/bundled/ruby
+tar -xvf traveling-ruby-20150715-2.2.2-linux-x86_64.tar.gz -C bundled/ruby
+# ls ruby => bin  bin.real  info  lib
+# bundled/ruby/bin/ruby -v # works now :)
+
+# had to modify Gemfile and update the local path for linux
+bundle install --path bundled/gems
+
+# DONT THINK THAT I NEED TO COPY THE Gemfile into bundled/gems...
+# mv Gemfile* bundled/gems/ # copy Gemfile from the project to bundled/gems
+# IMPORTANT: the Gemfile must be in the same bundled/gems folder
+
+# now we have both bundled/gems bundled/ruby :)
+bundled/gems/ruby/2.2.0/bin/print_ruby_info # should work
+
+
+# Let's move back to the project and test the wrapper, it should work also
+mv bundled ~/jets-test/jets/spec/fixtures/project/
+cd ~/jets-test/jets/spec/fixtures/project # back to project root

data/notes/traveling-ruby-packaging.md

@@ -0,0 +1,103 @@
+### Packaging Gems Info
+
+### Structure
+
+This is the structure that TravelingRuby uses in it's tutorial.
+
+```sh
+PROJECT/YOUR_BINARY_WRAPPER (hello)
+PROJECT/lib/ruby/bin/ruby -> PROJECT/lib/ruby/bin.real/ruby
+PROJECT/lib/vendor/ruby/2.2.0/bin/print_ruby_info # the gem binaries are here
+```
+
+* Instead calling the ruby binary `jets process` command directly.
+* Jets will require 'bundler/setup' so the user's gems will be required properly
+* Skip the overhead of having another wrapper
+
+```sh
+PROJECT/vendor/ruby/2.2.0/bin/jets # the gem binaries are here
+```
+
+### Packaging Gems Commands
+
+```
+mkdir packaging/tmp
+cp Gemfile Gemfile.lock packaging/tmp/ # this are from the user's project
+cd packaging/tmp
+
+# here's where the gems are instaleld into packaging/vendor/
+BUNDLE_IGNORE_CONFIG=1 bundle install --path ../vendor --without development
+# IMPORTANT: Think I'll make the user switch to 2.2.0 and error the build proccess.
+
+##############
+I can call ruby bin files directly.
+I was WRONG. Cannot call gem bin files directly because I need to make sure that
+bundler/setup gets loaded before calling the gem bin.
+Tried moving bundler/setup into the jets library itself but get all sorts of warnings.
+
+hello-1.0.0-linux-x86_64/lib/vendor/ruby/2.2.0/bin/jets help
+BUT the shabang line has: #!/usr/bin/env ruby2.0 .. but only on linux..
+Simply cannot rename the darn ruby version folder.
+#############
+
+
+cd ../..
+rm -rf packaging/tmp # remove working space
+
+# reduce the zip package size!
+rm -f packaging/vendor/*/*/cache/*
+```
+
+Now we can copy over the generated vendored files
+
+##################################
+# clean
+rm -rf packaging/vendor/
+
+bundle update
+rake package:linux:x86_64 DIR_ONLY=1
+
+mkdir packaging/tmp
+cp Gemfile Gemfile.lock packaging/tmp/
+cd packaging/tmp
+BUNDLE_IGNORE_CONFIG=1 bundle install --path ../vendor --without development
+cd ../..
+
+cp -pR packaging/vendor hello-1.0.0-linux-x86_64/lib/
+cp Gemfile Gemfile.lock hello-1.0.0-linux-x86_64/lib/vendor/
+mkdir hello-1.0.0-linux-x86_64/lib/vendor/.bundle
+cp packaging/bundler-config hello-1.0.0-linux-x86_64/lib/vendor/.bundle/config
+
+find . -name print_ruby_info
+
+
+# Wrapper script `jets`
+
+```bash
+#!/bin/bash
+set -e
+
+# Figure out where this script is located.
+SELFDIR="`dirname \"$0\"`"
+SELFDIR="`cd \"$SELFDIR\" && pwd`"
+
+# Tell Bundler where the Gemfile and gems are.
+export BUNDLE_GEMFILE="$SELFDIR/lib/vendor/Gemfile"
+unset BUNDLE_IGNORE_CONFIG
+
+# Run the actual app using the bundled Ruby interpreter, with Bundler activated.
+exec "$SELFDIR/lib/ruby/bin/ruby" -rbundler/setup "$SELFDIR/lib/app/hello.rb"
+```
+
+
+### Building Ruby
+http://cache.ruby-lang.org/pub/ruby/2.2/
+
+```sh
+wget http://cache.ruby-lang.org/pub/ruby/2.2/ruby-2.2.2.tar.gz
+tar xvfvz ruby-2.2.2.tar.gz
+cd ruby-2.2.2
+./configure
+make
+sudo make install
+```

data/notes/traveling-ruby-structure.md

@@ -0,0 +1,6 @@
+## Traveling Ruby Structure
+
+* PROJECT/lib/ruby/bin/ruby -> PROJECT/lib/ruby/bin.real/ruby
+* PROJECT/lib/vendor/ruby/2.2.0/bin/jets
+* PROJECT/YOUR_BINARY_WRAPPER
+

data/notes/traveling-ruby.md

@@ -0,0 +1,82 @@
+## Overview
+
+There are 2 things to point out:
+
+1. What the user's environment contains - cannot control this, can only suggest
+2. What the lambda environment contains - can fully control this
+
+* User can develop with ruby 2.4 and bundle gems and test and be happy. Hopefully they eventually learn to test with ruby 2.2.0 (when they run into bugs).
+* When `jets build` runs it will bundle it in ruby 2.2.0 though.
+* When lambda runs it will call `jets` with ruby 2.2.0. The gems will be installed in 2.2.0
+	* Hope all works
+
+## Download ruby commands
+
+```sh
+# linux 64bit
+wget http://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20150715-2.2.2-linux-x86_64.tar.gz .
+mkdir ruby-linux
+tar -xvf traveling-ruby-20150715-2.2.2-linux-x86_64.tar.gz -C ruby-linux
+# mac
+wget http://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20150715-2.2.2-osx.tar.gz
+mkdir ruby-mac
+tar -xvf traveling-ruby-20150715-2.2.2-osx.tar.gz -C ruby-mac
+
+# another version
+mkdir hello-1.0.0-linux-x86_64/lib/ruby && tar -xzf packaging/traveling-ruby-20150715-2.2.2-linux-x86_64.tar.gz -C hello-1.0.0-linux-x86_64/lib/ruby
+```
+
+## Launch Instance for Testing
+
+```sh
+aws ec2 run-instances --image-id ami-8c1be5f6 --count 1 --instance-type t2.micro --key-name default --security-groups demo
+```
+
+## Tree Structure
+
+```sh
+lib/
+├── app
+│   └── hello.rb
+├── ruby
+│   ├── bin
+│   │   ├── gem
+│   │   ├── irb
+│   │   ├── rake
+│   │   ├── ruby
+│   │   └── ruby_environment
+│   ├── bin.real
+│   │   ├── bundle
+│   │   ├── bundler
+│   │   ├── gem
+│   │   ├── irb
+│   │   ├── rake
+│   │   └── ruby
+│   └── lib
+│       └── ruby
+│           ├── 2.2.0
+...
+│           │   └── yaml.rb
+│           ├── gems
+│           │   └── 2.2.0
+│           │       ├── gems
+│           │       │   ├── attr_extras-5.2.0
+│           │       │   │   ├── attr_extras.gemspec
+...
+│           │       │   ├── bundler-1.9.9
+...
+│           │           └── rake-12.2.1.gemspec
+...
+└── vendor
+    ├── Gemfile
+    ├── Gemfile.lock
+    └── ruby
+        └── 2.2.0 <= IMPORTANT: bundler copies gems to folder with running ruby version
+            ├── gems
+            │   ├── concurrent-ruby-1.0.5
+            │   ├── faker-1.7.3
+            │   └── i18n-0.9.0
+...
+```
+
+* [Full tree](https://gist.github.com/tongueroo/c42f9d35b15b06eb810802243f4e2f6d)

data/spec/fixtures/classes.rb

@@ -0,0 +1,5 @@
+class CommentsController < Jets::BaseController
+  def create; end
+  def update; end
+end
+

data/spec/fixtures/project/.gitignore

@@ -0,0 +1,3 @@
+vendor
+bundled
+jets.zip

data/spec/fixtures/project/.ruby-version

@@ -0,0 +1 @@
+2.2.2

data/spec/fixtures/project/Gemfile

@@ -0,0 +1,4 @@
+source "https://rubygems.org"
+
+gem "jets", git: "git@github.com:tongueroo/jets.git"
+gem "print_ruby_info" # test

data/spec/fixtures/project/Gemfile.lock

@@ -0,0 +1,23 @@
+GIT
+  remote: git@github.com:tongueroo/jets.git
+  revision: 93c085671d2fb6de73ef0246ab89a0c7fecc92ea
+  specs:
+    jets (0.0.1)
+      colorize
+      hashie
+      thor
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    colorize (0.8.1)
+    hashie (3.5.6)
+    print_ruby_info (0.0.3)
+    thor (0.20.0)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  jets!
+  print_ruby_info

data/spec/fixtures/project/app/controllers/application_controller.rb

@@ -0,0 +1,2 @@
+class ApplicationController < Jets::BaseController
+end

data/spec/fixtures/project/app/controllers/posts_controller.rb

@@ -0,0 +1,12 @@
+class PostsController < Jets::BaseController
+  def create
+    # render text: "test2" # more consistent for web controllers
+
+    # render returns Lambda Proxy struture for web requests
+    render json: event.merge(a: "create"), status: 200
+  end
+
+  def update
+    render json: event.merge(a: "update"), status: 200
+  end
+end