flok 0.0.12 → 0.0.14

data/bin/flok

@@ -8,34 +8,78 @@ require 'closure-compiler'
 class FlokCLI < Thor
   desc "new <path>", "Create a new flok project and/or module, you may supply an absolute path or relative, the last entry in the path will be the module name and folder name of the project"
   def new path
+    #Name of this project
     name = File.basename(path)
+
+    #Get the directory of the given path, if path is only 'foo', then this will go into '.' (nop)
     Dir.chdir File.dirname(path) do
-      `bundle gem #{name}`
+      Flok::Project.create name
+    end
+  end
+
+  desc "build <platform>", "Build the products for a platform"
+  def build platform
+    #Create a products folder if it dosen't already exist
+    Dir.mkdir("./products") unless File.exists?("./products")
+
+    ENV["PLATFORM"] = platform
+
+    #Go into the flok gem project
+    local_products_path = File.join(Dir.pwd, "products")
+    Dir.chdir(File.join(File.dirname(__FILE__), "../")) do
+      #1. Use the rake task
+      system('rake build:world')
+
+      #2. Copy everything in the gems ./flok/products/$PLATFORM -> $PROJECT/products/$PLATFORM
+      FileUtils.cp_r "./products/#{platform}", local_products_path
+    end
+
+    #3. Build the client's ./app/controllers/*.rb into './products/$PLATFORM/user_compiler.js'
+    controller_glob_path = "#{local_products_path}/#{platform}/glob/controllers.rb" 
+    Flok.src_glob("rb", './app/controllers', controller_glob_path)
+    user_compiler_js = Flok::UserCompiler.compile File.read(controller_glob_path)
 
-      Dir.chdir name do
-        Dir.mkdir "app"
-        Dir.mkdir "config"
+    Dir.chdir File.join(local_products_path, platform) do
+      #3. Put user_compiler_js in glob folder
+      File.write "glob/user_compiler.js", user_compiler_js
+
+      #4. Move application.js to the glob folder
+      FileUtils.cp "application.js", "glob/application.js"
+      FileUtils.rm "application.js"
+
+      #5. Combine application.js and user_compiler.js.erb
+      File.open "glob/application_user.js.erb", "w" do |f|
+        f.puts File.read("glob/application.js")
+        f.puts File.read("glob/user_compiler.js")
       end
+
+      #6. The `erb` files is then sent to `./products/$PLATFORM/glob/application_user.js` with the below `ERB` variables allowable.
+      erb_src = File.read "glob/application_user.js.erb"
+      renderr = ERB.new(erb_src)
+      context = ERBUserApplicationContext.new()
+      new_src = renderr.result(context.get_binding)
+      File.write "application_user.js", new_src
     end
   end
 
-  option :compress, type: :boolean
-  desc "build", "Build a flok project, you must be in the root of the project"
-  def build
-    Dir.mkdir("./public") unless File.exists?("./public")
-    FileUtils.touch "./products/application.js"
-    FileUtils.touch "../FlokTest/app/assets/javascripts/flok.js"
-
-    src = Flok::MergeSource.merge_all
+  desc "server", "Monitors for changes within your flok application and triggers an automatic rebuild of ./products/* for a PLATFORM"
+  def server platform
+    system('bundle exec guard')
+  end
 
-    #Compress by writing to a temporary dir, run closure, and then put back
-    if options["compress"] == true
-      src = Closure::Compiler.new.compile(src)
+  class ERBUserApplicationContext
+    def get_binding
+      return binding
     end
 
-    File.write("./products/application.js", src)
-    File.write("../FlokTest/app/assets/javascripts/flok.js", src)
+    def initialize
+      #Debug / Release
+      @debug = (ENV['FLOK_ENV'] == "DEBUG")
+      @release = (ENV['FLOK_ENV'] == "RELEASE")
+    end
   end
 end
 
 FlokCLI.start(ARGV)
+
+

data/docs/architecture.md

@@ -1,18 +1,6 @@
 # Architecture
-Flok's architecture is a non-pre-emptive (realtime) event-driven tickless monolithic meta-kernel divided into several parts.
-```
-#(a) <Platform specific drivers for handling generic/custom interrupts and generic/custom IO>
-#---------^|------------------------------------
-#---------||------------------------------------
-#=========|v====================================          <--------------------- Abstraction barrier
-#(b) <Standard driver interface> <Custom driver interfaces>                         
-#---------^|------------------------------------
-#---------||------------------------------------
-#---------|v------------------------------------
-#(c) <Generic kernel systems (ui, pipes, etc)> <Your Kernel 'tasks'>
-```
+Flok uses a server/client architecture and has one two-endpoint communication channel that uses an extremely efficient pipelined messaging protocol.  See [Messaging](./messaging.md) for details.
 
-* (a) - Drivers are written in any languages and must implement (b) the standard driver interface.
-* (b) - All driver communication must pass directly through this layer
-* (c) - This layer handles all generic activity like setting up pipes between tasks, etc.
+The communications interface is divided up into two parts; interfaces (`if_*`) and interrupts (`int_*`).  Interfaces are going from the flok server to the client.  Interrupts are from the client to the flok server. There is only one real interface and only one real interrupt, these are `int_dispatch` and `if_dispatch`.  All the other interrupts and interfaces are `virtual` as in they are decoded via the flok server `int_dispatch` or for each platform, `if_dispatch` decodes their messages.
 
+![](./images/flok_arch.png)

data/docs/client_api.md

@@ -0,0 +1,21 @@
+#Client API
+Client API covers controller action event handlers.
+
+### Controller MACROS
+  * Embed(view_controller_name, spot_name, context) - Embed a view controller with the name `view_controller_name` inside the current view controller at the spot with a context
+  * Goto(action_name) - Change actions
+  * Request(service_name, info, event_cb_name) - Initiate a service.  See [Services](./services.md) for more info.
+  * Send(event_name, info) - Send a custom event on the main queue.
+  * Raise(event_name, info) - Will send an event to the parent view controller (and it will bubble up, following `event_gw` which is set in `Embed` as the parent controller
+  * Lower(spot_name, event_name, info) - Send an event to a particular spot
+
+### Controller Event Handlers
+  * Variables
+    * `context` - The information for the controllers context
+    * `params` - What was passed in the event
+    * `__base__` - The address of the controller
+    * `__info__` - Holds the `context`, current action, etc. See [Datatypes](./datatypes.md)
+### Controller on_entry
+    * `context` - The information for the controllers context
+    * `__base__` - The address of the controller
+    * `__info__` - Holds the `context`, current action, etc. See [Datatypes](./datatypes.md)

data/docs/compilation.md

@@ -1,8 +1,8 @@
 # Compliation
 
-Compilation is handled by the `rake build_world PLATFORM=YOUR_PLATFORM` and then end result of that compliation is put in ./products/
+Compilation is handled by the `rake build:world PLATFORM=YOUR_PLATFORM FLOK_ENV=ENVIRONMENT` and then end result of that compliation is put in ./products/
 Compilation *always* results in a `./products/$PLATFORM/application.js` file along with other files in `./products/$PLATFORM/drivers/` that
-were deemed necessary by the platform driver `build` scripts.
+were deemed necessary by the platform driver `build` scripts. The environment can either be `DEBUG` or `RELEASE`.
 
 ### Build Order
 *Unless otherwise stated, all files execute in alpha-numerical order. (`0foo.js` would execute before `1foo.js`).  Please use this convention only
@@ -12,6 +12,32 @@ as necessary.*
 
  1. `rake build` is run inside `./app/drivers/$PLATFORM` with the environmental variables set to BUILD_PATH=`./produts/$PLATFORM/driver` (and folder
  2. All js files in `./app/kern/config/*.js` are globbed togeather and sent to `./products/$PLATFORM/glob/1kern_config.js`
- 3. All js files in `./app/kern/*.js` are globbed togeather and sent to `./products/$PLATFORM/glob/2kern.js`
- 4. All js files are globbed from `./products/$PLATFORM/glob` and combined into `./products/$PLATFORM/application.js`
- 5. Auto-generated code is placed at the end (like uname_p)
+ 3. All js files in `./app/kern/*.js` are globbed togeather and sent to `./products/$PLATFORM/glob/2kern.pre_macro.js`
+ 4. All rb files inside `./app/kern/services/` are globbed into `./products/$PLATFORM/glob/kern_services.rb`
+ 5. `./products/$PLATFORM/glob/kern_serivces.rb` is processed via `Flok::Services` and then exported as `./products/$PLATFORM/glob/kern_services.pre_macro.js`
+ 6. All js files in `./products/$PLATFORM/glob/2kern.pre_macro.js` are run through `./app/kern/macro.rb's macro_process` and then sent to ./products/$PLATFORM/glob/2kern.js
+ 7. All js files in `./products/$PLATFORM/glob/kern_services.pre_macro.js` are run through `./app/kern/macro.rb's macro_process` and then sent to ./products/$PLATFORM/glob/kern_services.js
+ 8. All js files are globbed from `./products/$PLATFORM/glob` and combined into `./products/$PLATFORM/glob/application.js.erb`
+ 9. Auto-generated code is placed at the end (like PLATFORM global)
+ 10. The module specific code in `./kern/mod/.*js` are added when the name of the file (without the js part) is mentioned in the `./app/drivers/$PLATFORM/config.yml` `mods` section and appended to `glob/application.js.erb`
+ 11. The compiled `glob/application.js.erb` file is run through the ERB compiler and formed into `application.js`
+
+##Erb variables
+All kernel source files support embedded ERB code like `<% if DEBUG %>Code<% end %>`. These files include:
+  * `./app/kern/*.js`
+  * `./app/kern/mod/*.js`
+  * `./app/kern/services/*.js` - Services only support ERB in the javascript code sections
+
+####Supported variables
+  * `@debug` - Set to `true` when FLOK_ENV=DEBUG
+  * `@release` - Set to `true` FLOK_ENV=RELEASE
+  * `@mods` - The set of modules supported by this platform and build configuration
+
+```js
+  //Example JS code for debug / release mode
+  <% if @debug %>
+    //JS Code for debug
+  <% else %>
+    //JS code for not debug mode
+  <% end %>
+```

data/docs/controllers.md

@@ -0,0 +1,97 @@
+#Controllers
+Controllers are a lot like the controllers from `MVC` triads. They are the heart of your user-defined behavior in the same way rails controllers define your web-application.
+
+See [Client API](./client_api.md) for details on the functions available to controllers.
+
+##Kernel & Client Controllers
+There are two types of controllers.  One type is a universal `flok user controller` (**fuc**) that manages state information, the view hierarchy, and is written in javascript; the other type is present on your device, the `device view controller` (**dvc**). The `device view controller` expresses the appereance of the state and view names from `flok user controller`.  The `device view controller` also sends events to the `flok user controller` when an action is commited, like a `button_clicked` event.
+
+##Events
+Events can be sent bi-directionally from the `fuc` and `dvc` via the [Event Module](./mod/event.md).  Controller communication is done entirely over the event system with the exception of `if_init_controller` to bootstrap the controller, and the views which are managed by the [UI Module](./mod/ui.md) by the `fuc`.
+
+####`fuc => dvc`
+When an event is sent from the `fuc`, the event pointer should be interpreted as the opaque device defined address of the `dvc`. The `dvc` would have been given in `if_init_controller` and the device is required to record that information and dispatch `if_event` messages to the matching `dvc`.
+
+####`dvc` => `fuc`
+When an event is sent from `dvc`, then `fuc` calls an appropriate `on` handler to manage the event. If there are no `on` handlers that can take the
+event, the event is passed to the `event_gw` (event gateway) of the `controller_info` structure [See datatypes](./datatypes.md) for more information
+on the structure. If `event_gw` is `null`, then the event is ignored.
+
+###Writing a `flok user controller (fuc)`
+Let's write a `fuc` controller for a view that has 2 tabs and a content area.
+
+```ruby
+controller "tab_controller" do
+  view "tab_container"
+  spaces "content"
+
+  #You can also define macros for shared action traits
+  macro "my_macro" do
+    on "shared_clicked" do
+      Goto("home")
+    end
+  end
+
+  #The home tab
+  action "home" do
+    #Macros can be called via their name
+    my_macro
+
+    on_entry %{
+      Embed("home_controller", "content", {});
+    }
+    
+    on "about_clicked", %{
+      Goto("about");
+    }
+  end
+  
+  #The about tab
+  action "about" do
+    on_entry %{
+      Embed("about_controller", "content", {});
+    }
+    
+    on "home_clicked", %{
+      Goto("home");
+    }
+  end
+end
+```
+
+###`fuc` API
+  See [Client API](./client_api.md)
+
+###`fuc` registrations
+When a `fuc` is initialized, there are various things that are configured that must be cleaned up when the `fuc` is torn down.
+
+###`fuc` internals
+####View Init Procedure
+  1. `if_init_view`
+  2. `if_controller_init`
+  3. `if_attach_view`
+
+####Registrations
+  * Entire `fuc` lifetime
+    * The `tp` of the `fuc` contains the `controller_info`
+    * The `root_view` of the `fuc`
+    * The entry for the `fuc tp` in `event vector table`
+  * Per action
+    * A set of embeds for more `fuc`
+
+###Internals
+This controller is put inside the `ctable` for all the things that don't change. see [datatypes](./datatypes.md) for information on the layout of the ctable.
+for each controller.
+
+In order for changes to be represented in a controller, a controller must be *initialized*. Unlike rails, Flok allows you to use many controllers within one-another,
+and many instances of the same controller if you wish. 
+
+Controller initialization is done via `_embed` or the `embed` macro if you are inside a controller. Embedding
+  1. Requests a set of sequential pointers via `tels`, `n(spots)`.  `main` is always a spot, so there is always at least two pointer. The first pointer is the view controller itself `vc`, and the second is the `main` (root) view.
+    * Spots looks like ['vc', 'main', ...]
+  2. Initializes the root view of the controller with `bp+1` and retrieve the spots array from the controller `main` + whatever you declared in `spots`
+  3. Attaches that view to the `bp+2+spots.indexOf(spot)` (which is a tele-pointer) given in the embed call.
+  4. Sets up the view controller's info structure.
+  5. Explicitly registers the view controller's info via `tel_reg_ptr(info, base)`
+  6. Configures the `evt` to receive events at the base pointer
+  6. Invokes the view controllers `on_entry` function with the info structure.

data/docs/datatypes.md

@@ -0,0 +1,42 @@
+#Data types
+Instances labeled `static` mean that they are not dynamically created and are meant to be used by many things. Somewhat like a singleton or a const in a functional language.
+
+###ctable_entry (static)
+The controller info contains all things that are necessary to initialize and use a controller. It is never duplicated and stands
+as a static structure. Each controller type has one static matching object that is part of the controller table (`ctable`) and can
+be looked up as `ctable[controller_name]`.
+
+```javascript
+ctable_entry {
+  root_view,     //A constant string of the name of the view this controller sets as it's root view.
+  actions,       //A dictionary [String:action_info] that corresponds to a dictionary of action_info object's based on the action's name.
+  spots,         //An array fo spot names for this controller
+  name           //The name of the controller, useful for certain lookup operations, this is also the ctable key
+}
+```
+
+###action_info (static)
+The bulk of the controller logic is handled here. Each controller contains an array of pointers to action_info; these action_info objects
+dictate what happends when events come in, etc.
+```javascript
+action_info {
+  on_entry      //A function that is called when this action is initialized.
+  handlers      //A dictionary [String:f(context, info)] of event handlers for events that occur
+}
+```
+
+###controller_info
+Created when a new controller is established via `_embed`. The tele-pointer system maintains the pointers to controller instances
+via `tel_reg`.  The `main` view pointer acts as the base-pointer for the controller. This means that on *our* side, the `main` view
+pointer refers to the controller and on the `driver` side, the base pointer refers to our root-view. This would be apparent if we received
+an event from the root-view as it would the same pointer to the controller. If we sent an event to the `driver`, then that `driver` would
+receive the event in the `root` view of the controller.
+
+```javascript
+controller_info {
+  context: {}, //A hash that contains context information (user supplied)
+  action: //The name of the current action that is active
+  cte:    //The ctable entry pertaining to this controller
+  embeds: //An array of arrays, where position 0 is the spot after `main`, each element in the array is a view controller base pointer.
+}
+```

data/docs/debug_server.md

@@ -0,0 +1,17 @@
+#Debug Server
+The debug server should be started whenever a driver is built in `debug` mode. The server supported should be defined in `debug_attach`.
+the behavior of one of the protocols listed below. You should declare which server type you use in the driver because the spec testers use this
+information. The key `debug_attach` holds a protocol key like `socket_io`. This key should only be in the `DEBUG` section of your `config.yml`
+The `debug` module implements the `debug server`. Although, the server is a seperate
+piece of code that dosen't fit into the same `if` and `int` paradigm. The specs in the `debug server` also use various `debug` module helpers.
+
+## `socket_io` Protocol
+This protocol states that the driver must repeadeately attempt to connect to the socket.io port located at `localhost:9999`.
+
+### Attach event
+After `attach` is received from socket.io, the driver should forward all `if_dispatch` events to `if_dispatch`
+of the `socket.io` and forward all `int_dispatch` events going to the flok server to `int_dispatch` of socket.io.  The driver should then
+forward all `int_disptach` events it receives to `int_dispatch` of the flok server and all `if_dispatch` events socket.io it receives to `if_dispatch` 
+event on the driver. The information passed along for these messages should be the same as the originals.
+
+Attach may be called multiple times. If it is, the driver is to ignore this.

data/docs/drivers.md

@@ -0,0 +1,52 @@
+#Drivers
+A driver in flok is for one specific platform. The driver lives in the `./app/drivers/$PLATFORM` folder.
+This folder is required to have a `config.yml` and a `Rakefile`.  Everything else that goes in this folder is up to the implementor.
+
+## config.yml
+Driver configuration file.  Contains the following keys:
+  * `$ENVIRONMENT` - Can either be `RELEASE` or `DEBUG`
+    * `mods` - A list of module names this driver supports. See [Modules](./modules.md) for details.
+    * `debug_attach` - See [Debug server](./debug_server.md) for information
+
+```yml
+DEBUG:
+  debug_attach: socket_io
+  mods:
+    ui
+    debug
+RELEASE:
+  mods:
+    ui
+```
+
+## Rakefile
+Contains the following tasks:
+  * `build` - A request for this driver to build it's files into the folder `$BUILD_PATH`
+  * `spec`  - Run any unit tests that the implementor deems necessary that do not need the pipe. Given `$BUILD_PATH`. Should return 0 on success and
+      non-0 on failure
+  * `pipe`  - Establish a 2-way pipe on standard io where input goes to `if_dispatch` and `int_dispatch` goes to output.  
+
+## Minimum interface
+Every driver *must* export `if_dispatch` into the context managing `application.js` (See [Messaging](./messaging.md)). The driver should then handle `if_dispatch` and it's implementation should handle messages appropriately so that the messages it receives can be fulfilled as stated in the modules
+contract. The driver should also offer some way to call `int_dispatch` *inside* the `application.js` context from the outside world.
+
+## Spec (Testing)
+Driver tests are chosen by the platform implementor.  The only requriements for a driver test are that it be run when `rake spec` is executed within the `./app/driver/$PLATFORM/` folder. These tests should **never** do anything that can be tested by only using the driver's pipe interface. Tests that can be done through the pipe interface should be done through the **Driver Interface** tests (See the section on *Interface* in [Testing](./testing.md).
+
+  * Running - `rake spec`
+  * Location - `./app/drivers/$PLATFORM/`
+  * Environment
+    - `$BUILD_PATH` - The same folder the driver was given during `build`
+    - `$PWD` - `./app/drivers/$PLATFORM/`
+
+## Debug View & Controller
+When a driver receives a `if_init_view` it cannot handle because the view is not something it has been programmed to understand, (typically through a template system), that driver is suggested to show a `debug` view if 
+the driver is built with the environmental variable `$FLOK_ENV` set to `DEBUG` and the `debug` module is present. The *debug* view should have one associated *debug* controller for all debug views that does the same thing.
+
+The *debug* view and controller should show something along the lines of:
+  * The controller name - This is given in `if_init_controller`
+  * The current action name - This is given in the `action` event `to` field.
+  * The last event received - You can capture this information in your custom event handler.
+  * The context information - This is passed in `if_init_controller` and should be accesssible in your view controller via `context`
+  * N Spaces for each attempted attachment of a view controller - When `if_init_view` is called, it has a list of spaces. Use these to dynamically create them.
+  * A list of buttons where each button triggers an event associated with the current action - This is set with `if_debug_assoc(controller_name, "action_events", [..])`

data/docs/interactive.md

@@ -0,0 +1,35 @@
+#Interactive
+
+##Pipes
+###Starting pipes
+  * Client (Driver/Platform)
+    * `cd ./app/drivers/$PLATFORM/; rake pipe` *or* `cd ./; rake pipe:driver PLATFORM=$PLATFORM`
+  * Server
+    * `cd ./; rake pipe:server PLATFORM=$PLATFORM`
+
+###What $stdout and $stdin does for pipes
+
+  * Server
+  	- `$stdin` => `int_dispatch`
+  	- `if_dispatch` => `$stdout`
+  * Client
+  	- `$stdin` => `if_dispatch`
+  	- `int_dispatch` => `$stdout`
+  
+All communication *coming* from `$stdin` and *going* to `$stdout` is in un-escaped JSON formatting that follows the conventions mentioned in [Messaging](./messaging.md).
+
+The test suites assume particular behavior of the pipes. Please review [./spec/env/iface.rb](../spec/env/iface.rb) for the method named `pipe_suite` for the proper behavior.
+
+####Examples
+
+#####Server ➜ Client
+ 1. **Server** calls `if_disptatch([[0, 0, 'hello']])` (**The additional 0 at the beginning is the queue, 0 in this case is the main queue**)
+ 2. **Server** `$stdout` *writes* `[0, 0, 'hello']\n`
+ 3. **Client** `$stdin` *gets* `[0, 0, 'hello']\n`
+ 4. **Client** calls `if_dispatch([0, 0, 'hello'])`
+
+#####Client ➜ Server
+ 1. **Client** calls `int_dispatch([0, 'hello'])`
+ 2. **Client** `$stdout` *writes* `[0, 'hello']\n`
+ 3. **Server** `$stdin` *gets* `[0, 'hello']\n`
+ 4. **Server** calls `int_dispatch([0, 'hello'])`

data/docs/kernel_api.md

@@ -0,0 +1,37 @@
+#Kernel API
+
+##Telepointer (Mapped I/O)
+  * `tels(n)` - Returns one number that represents the base index of the telepointer.  See [Messaging](Messaging.md) for details on telepointers.
+  * `tel_reg(f)` - Returns a number that represents the base pointer of a function callback.
+  * `tel_del(n)` - Delete a telepointer at some index. Does nothing if not registered with something.
+  * `tel_reg_ptr(f, tp)` - Register an explicit telepointer address at an index. Usually for spec assistance. Do not do as a normal user, use `tel_del`
+  * `tel_deref(i)` - Convert a tel pointer into an object. Returns the object.
+  * `tel_exists(tp)` - Returns true or false depending on whether there is a telepointer that matches
+instead.
+
+##Events
+  * `reg_evt(ep, f)` - Register a function to be called when an event is processed by `int_event`. The function will receive `(ep, event_name, info)`.
+
+  * `dereg_evt(ep)` - Do not do anything if ep is received as an `int_event`
+
+##Networking
+  * `get_req(owner_tp, url, params, callback)` - Request some RESTFUL get request.  The callback receives `(info)` with a data payload. Will retry until successful, will never fail. The request will be abandoned if the owner object no longer exists.
+
+##Controllers
+  * `_embed(vc_name, sp, context, event_gw)` - Embed a view controller in a surface-pointer. Following the rules of the ui device, embedded to a sp of
+      0 is the master root view. Returns base pointer. `event_gw` is a pointer to a `vc`. If it is null, then any events coming in will not be sent to
+      somewhere else if they do not match any 'on' for the current action.
+
+##Things that are compiled into the kernel from user given data
+`MODS` - A list of modules that was specified in `./app/drivers/$PLATFORM/config.yml`
+`PLATFORM` - The platform that this kernel was compiled with
+
+#Messaging
+`SEND(queue_index, message_name, *params)` - Queue a message to be sent out. This is a macro, you must not put any characters beyond quotes and
+variables in here.  If you need to pass a hash literal, array literal, etc, please assign the variable *before* you put it in here like
+```js
+var payload = {from: null, to: action};
+SEND("main", "if_event", base, "action", payload);
+```
+
+