rhoconnect 3.4.5 → 4.0.0.beta.10

data/doc/command-line.txt

@@ -39,10 +39,10 @@ For example:
 			 [ADDED]  storeserver/Rakefile
 			 [ADDED]  storeserver/Gemfile
 			 [ADDED]  storeserver/spec/spec_helper.rb
-	
+
 Here's a basic rundown of each of the files that were just generated:
-	
-	
+
+
 * **config.ru** - This is a rackup file which will load your application.  RhoConnect uses the excellent [sinatra](http://www.sinatrarb.com/) web framework to run the application.
 * **settings/settings.yml** - This contains settings such as your redis connection, which source adapters exist and their parameters.  You can also put custom settings here.
 * **settings/license.key** - The RhoConnect [license](/rhoconnect/licensing) associated with your application.  The default license allows up to 10 devices to synchronize with the application.
@@ -67,35 +67,42 @@ rhoconnect, type `rhoconnect` or `rhoconnect help`.
 	:::term
 	$ rhoconnect help
 	Tasks:
-	  rhoconnect app NAME               # Generates a new rhoconnect application.
+	  rhoconnect app NAME               # Generate a new rhoconnect application
 	  rhoconnect attach                 # Attach to rhoconnect console
-	  rhoconnect clean-start            # Clean rhoconnect, get token, and create new user
-	  rhoconnect create-user            # Creates and subscribes user for application in rhoconnect
-	  rhoconnect delete-device          # Deletes a device from rhoconnect
-	  rhoconnect delete-user            # Deletes a user from rhoconnect
+	  rhoconnect console [environment]  # Run rhoconnect console
 	  rhoconnect dtach-about            # About dtach
-	  rhoconnect dtach-install          # Install dtach 0.8 from source
+	  rhoconnect dtach-install          # Install dtach program from sources
 	  rhoconnect flushdb                # Flush data store - WARNING: THIS REMOVES ALL DATA IN RHOCONNECT
-	  rhoconnect get-token              # Fetches current api token from rhoconnect
+	  rhoconnect get-token              # Fetch current api token from rhoconnect
 	  rhoconnect help [TASK]            # Describe available tasks or one specific task
 	  rhoconnect redis-about            # About redis
 	  rhoconnect redis-attach           # Attach to redis dtach socket
-	  rhoconnect redis-download         # Download redis package
-	  rhoconnect redis-install          # Install the latest version of Redis from Github (requires git, duh)
-	  rhoconnect redis-restart          # Restart redis
-	  rhoconnect redis-start            # Start redis
-	  rhoconnect redis-stop             # Stop redis
-	  rhoconnect reset                  # Reset the rhoconnect database (you will need to run rhoconnect:get_token afterwards)
-	  rhoconnect reset-refresh          # Reset source refresh time
-	  rhoconnect restart                # Alias for `rhoconnect stop; rhoconnect start`
+	  rhoconnect redis-download         # Download redis release 2.6.11
+	  rhoconnect redis-install          # Install the latest verison of Redis from Github (requires git, duh)
+	  rhoconnect redis-restart          # Restart redis on localhost
+	  rhoconnect redis-start            # Start redis on localhost
+	  rhoconnect redis-status           # Show status of redis servers
+	  rhoconnect redis-stop             # Stop redis running on localhost
+	  rhoconnect restart [options]      # Alias for `rhoconnect stop; rhoconnect start`
 	  rhoconnect secret                 # Generate a cryptographically secure secret session key
-	  rhoconnect set-admin-password     # Sets the admin password
-	  rhoconnect source NAME            # Generates a new source adapter.
-	  rhoconnect spec                   # Run source adapter specs
-	  rhoconnect start                  # Start rhoconnect server
+	  rhoconnect set-admin-password     # Set the admin password
+	  rhoconnect source NAME            # Generate a new source adapter
+	  rhoconnect start [options]        # Start rhoconnect server
 	  rhoconnect stop                   # Stop rhoconnect server
-	  rhoconnect war                    # Build executable WAR file to be used in Java App Servers
-	  rhoconnect web                    # Launch the web console in a browser - uses :syncserver: in settings.yml
+	  rhoconnect update                 # Update an existing application to the latest rhoconnect release
+	  rhoconnect version                # Display rhoconnect version
+	  rhoconnect web                    # Launch the web console in a browser
+
+**NOTE: The `rhoconnect war` command is available only if you switch to jruby, such as using the command `rvm use jruby`.**
+
+`rhoconnect start` and `rhoconnect restart` have the following options.
+
+	-p, [--port=N]                   # Use port (default: 9292)
+	-r, [--redis=REDIS]              # Redis server settings: (default: localhost:6379)
+	-P, [--push-server=PUSH_SERVER]  # Push server settings (default: http://someappname@localhost:8675/)
+	-t, [--api-token=API_TOKEN]      # API token  (default: my-rhoconnect-token)
+	-f, [--config=CONFIG]            # /path/to/rhoconnect/settings.yml file
+	-l, [--licenseflie=LICENSEFLIE]  # /path/to/license.key file
 
 ### Installing DTach (Mac OS / Linux only)
 
@@ -115,6 +122,26 @@ Next, you will need to setup and start a [redis](http://redis.io/) server:
 
 The server will show that it is running on the console.
 
+### Redis Horizontal Scaling
+
+Starting from v4.0, RhoConnect supports multiple Redis instances per app.
+To enable this feature, edit your app `settings/settings.yml` file and replace the `:redis:` configuration
+with an array of redis servers you are going to use:
+
+	:::yaml
+	# ...
+	:production:
+	  :licensefile: settings/license.key
+	  :syncserver: http://localhost:9292/api/application/
+	  :push_server: http://appname:secret@localhost:8675/
+	  # One or more redis servers
+	  :redis:
+	    - localhost:6379
+	    - localhost:6380
+	    - localhost:6381
+	    - localhost:6382
+	    # ...
+
 ### Start the Application
 
 Open a new console window (since the redis server is running in the previous console window). Then change directories to your rhoconnect application and start the application.
@@ -123,25 +150,56 @@ Open a new console window (since the redis server is running in the previous con
 	$ cd storeserver
 	$ rhoconnect start
 
-If everything went well, you should see the following in your console:
+#### Starting RhoConnect without a Source Adapter Application
+
+You can also start RhoConnect in a directory that does not contain a RhoConnect source adapter application. This is useful for RhoConnect plugin developers, who do not write a RhoConnect source adapter application: they can start a RhoConnect server from an empty directory.
+
+	:::term
+	$ cd (an empty directory)
+	$ rhoconnect start
+
+When you start RhoConnect in a directory that does not contain a rhoconnect source adapter application, a hidden settings file, .rhoconnect.yml, is created in your user home directory. Here is a partial sample listing.
+
+	:development: &70196107903780
+	  :syncserver: http://localhost:9393
+	  :redis: localhost:6379
+	  :push_server: http://someappname@localhost:8675/
+	  :api_token: my-rhoconnect-token
+	  :secret: !binary |-
+	    ...
+	:test: *70196107903780
+	:production: *70196107903780
+
+Once .rhoconnect.yml has been created in your user home directory, you can manually modify it for the next time you start RhoConnect. For example, you could change the port upon which the syncserver runs.
+
+Running RhoConnect this way uses the default license key in the RhoConnect gem. You can use other license keys by using the `-l` option in the `rhoconnect start` command.
+
+#### Opening the Web Console
+
+If everything went well when you started RhoConnect, you should see the following in your console:
+
+	[03:52:15 PM 2013-03-12] Rhoconnect Server v4.0.0 started...
 
-	[05:47:10 PM 2011-04-19] Rhoconnect Server v3.0.0 started...
-	
 To open the web console, navigate to [http://localhost:9292/](http://localhost:9292) in your browser or run:
 
 	:::term
 	$ rhoconnect web
-	
+
 ### Creating executable WAR containers (JRuby environment only)
 
 It is possible to create Java WAR containers for your Rhoconnect application that can be deployed into the
-Java App Servers (for example, JBoss). To create the executable WAR container, execute the following command in your app's root directory:
-	
+Java App Servers (for example, Tomcat, JBoss). To create the executable WAR container, execute the following command in your app's root directory:
+
 	:::term
 	$ cd storeserver
 	$ rhoconnect war
-	
-This will build the WAR file named <your_app>.war that can be deployed into the Java App Server.
+
+This will build the WAR file named `storeserver.war` that can be deployed into the Java App Server. Also warbler package includes 
+ an application launcher script,  so you can run it standalone on port 8080 in java environment:
+	:::term
+	$ java -jar storeserver.war
+
+To open the web console, navigate to [http://localhost:8080/](http://localhost:8080) in your browser.
 
 ### Common Tasks
 
@@ -149,195 +207,42 @@ RhoConnect applications come with many useful rhoconnect tasks for administrativ
 
 * 	`rhoconnect dtach-install` - (Mac OS / Linux only) This will install [dtach](http://dtach.sourceforge.net/) to /usr/local/bin.
 
-* 	`rhoconnect redis-install` - This will install to the system.  On Mac OS / Linux, it will attempt to install in '/usr/local/bin' (on windows, C:\redis-<version>), unless you have specified a REDIS_HOME to use instead. 
+* 	`rhoconnect redis-install` - This will install to the system.  On Mac OS / Linux, it will attempt to install in '/usr/local/bin' (on windows, C:\redis-<version>), unless you have specified a REDIS_HOME to use instead.
 If you cannot install redis on Linux due to insufficient user permissions, then follows these [instructions](/rhoconnect/install).
-	
+
 * 	`rhoconnect set-admin-password` - Set the RhoConnect administrator password.  The default password is blank ''.
 
 *	`rhoconnect get-token` - Get a RhoConnect API token to use for calling the RhoConnect REST API.
-		
+
 **NOTE: On Mac OS / Linux, you need to install dtach (i.e. by running `rhoconnect dtach-install`) to run the following tasks: **
 
-* 	`rhoconnect redis-start` - This will launch a redis server in a new console.
-	
-* 	`rhoconnect redis-stop` - Stop the currently-running redis server.
+* 	`rhoconnect redis-start` - Start redis on localhost.
+
+* 	`rhoconnect redis-stop` - Stop redis running on localhost.
 
-*	`rhoconnect redis:attach` - (Mac OS / Linux only) Attach to a currently-running redis server.
+* 	`rhoconnect redis-status` - Show status of redis servers.
+
+*	`rhoconnect redis-attach` - (Mac OS / Linux only) Attach to a currently-running redis server.
 
 * 	`rhoconnect start` - Start the RhoConnect application in a new console.  This requires a running redis server.
 
 * 	`rhoconnect stop` - Stop the currently-running RhoConnect application.
-	
-*	`rhoconnect attach` - (Mac OS / Linux only) Attach to a currently-running RhoConnect application. 
 
-* 	`rhoconnect clean-start` - Useful for development, resets and bootstraps the RhoConnect application with a test user.
+*	`rhoconnect attach` - (Mac OS / Linux only) Attach to a currently-running RhoConnect application.
 
 Some other useful development tasks:
 
-*	`rhoconnect reset` - Reset the RhoConnect application to the initial state.  This WILL DELETE ALL DATA IN REDIS! So be sure you don't run this in production!
-
 *	`rhoconnect spec` - Run all source adapter specs in spec/sources/.
-	
+
 *	`rhoconnect web` - Open the RhoConnect web console in a browser.
 
 ## Generate Source
 
 Connecting to a backend service with RhoConnect requires that you write a small amount of code for the query, create, update and delete operations of your particular enterprise backend. The collection of the code for these operations is called a source. You can create source for your RhoConnect application two ways:
- 
- * [RhoConnect source adapter](/rhoconnect/source-adapters): generate a Ruby code source adapter from RhoStudio or from the command line, similar to how you generate a RhoConnect app. This source adapter contains code for the query, create, update and delete operations.
- 
- * [RhoConnect plugin](/rhoconnect/plugin-intro): write the source code (the query, create, update and delete operations) into your backend application, and plug a RhoConnect plugin in the application. The plugin is written in the language that matchs your backend application, such as Java or .NET.
-
-## Rake Tasks (Rhoconnect v.3.1 and lower)
-<span class="label label-important">As of version 4.0 of rhoconnect, rake tasks will be deprecated!!</span> 
-
-Each RhoConnect (v.3.1 and lower) application uses [rake](https://github.com/jimweirich/rake) to manage development tasks such as starting/stopping the app, starting resque workers, and running specs.
 
-To run a RhoConnect rake task, simply type `rake the-task-to-run` in your RhoConnect application's root directory.
+ * [RhoConnect source adapter](/rhoconnect/source-adapters): generate a Ruby code source adapter from RhoStudio or from the command line, similar to how you generate a RhoConnect app. This source adapter contains code for the query, create, update and delete operations.
 
-Here is a complete list of the rake tasks available in a RhoConnect v.3.1 and lower application:
+ * [RhoConnect plugin](/rhoconnect/plugin-intro): write the source code (the query, create, update and delete operations) into your backend application, and plug a RhoConnect plugin in the application. The plugin is written in the language that matchs your backend application, such as Java or .NET.
 
-	:::term
-	$ cd storeserver
-	$ rake -T
-	(in ~/storeserver)
-	rake dtach:about                    # About dtach
-	rake dtach:install                  # Install dtach 0.8 from source
-	rake redis:about                    # About redis
-	rake redis:attach                   # Attach to redis dtach socket
-	rake redis:download                 # Download package
-	rake redis:install                  # Install the latest version of Redis from Github (requires git, duh)
-	rake redis:restart                  # Restart redis
-	rake redis:start                    # Start redis
-	rake redis:stop                     # Stop redis
-	rake resque:work                    # Start a Resque worker
-	rake resque:workers                 # Start multiple Resque workers.
-	rake rhoconnect:attach              # Attach to rhoconnect console
-	rake rhoconnect:clean_start         # Clean rhoconnect, get token, and create new user
-	rake rhoconnect:create_user         # Creates and subscribes user for application in rhoconnect
-	rake rhoconnect:delete_device       # Deletes a device from rhoconnect
-	rake rhoconnect:delete_user         # Deletes a user from rhoconnect
-	rake rhoconnect:flushdb             # Flush data store - WARNING: THIS REMOVES ALL DATA IN RHOCONNECT
-	rake rhoconnect:get_token           # Fetches current api token from rhoconnect
-	rake rhoconnect:reset               # Reset the rhoconnect database (you will need to run rhoconnect:get_token afterwards)
-	rake rhoconnect:reset_refresh       # Reset source refresh time
-	rake rhoconnect:restart             # Alias for `rake rhoconnect:stop; rake rhoconnect:start`
-	rake rhoconnect:set_admin_password  # Sets the admin password
-	rake rhoconnect:spec                # Run source adapter specs
-	rake rhoconnect:start               # Start rhoconnect server
-	rake rhoconnect:stop                # Stop rhoconnect server
-	rake rhoconnect:war                 # Build executable WAR file to be used in Java App Servers
-	rake rhoconnect:web                 # Launch the web console in a browser - uses :syncserver: in settings.yml
-
-Below is provided a list of rake tasks and their corresponding RhoConnect v.3.2+ command.
-
-<table border="1", cellpadding="5", width="100%">
-	<tr>
-		<th>rake Command</th>
-		<th>rhoconnect command</th>
-	</tr>
-	<tr>
-		<td>rake -T</td>
-		<td>rhoconnect help</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:attach</td>
-		<td>rhoconnect attach</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:clean_start</td>
-		<td>rhoconnect clean-start</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:reate_user</td>
-		<td>rhoconnect create-user</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:delete_device</td>
-		<td>rhoconnect delete-device</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:delete_user</td>
-		<td>rhoconnect delete-user</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:flushdb</td>
-		<td>rhoconnect flushdb</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:get_token</td>
-		<td>rhoconnect get-token</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:reset</td>
-		<td>rhoconnect reset</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:reset_refresh</td>
-		<td>rhoconnect reset-refresh</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:restart</td>
-		<td>rhoconnect restart</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:set_admin_password</td>
-		<td>rhoconnect set-admin-password</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:spec</td>
-		<td>rhoconnect spec</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:start</td>
-		<td>rhoconnect start</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:stop</td>
-		<td>rhoconnect stop</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:war</td>
-		<td>rhoconnect war</td>
-	</tr>
-	<tr>
-		<td>rake rhoconnect:web</td>
-		<td>rhoconnect web</td>
-	</tr>
-	<tr>
-		<td>rake dtach:about</td>
-		<td>rhoconnect dtach-about</td>
-	</tr>
-	<tr>
-		<td>rake dtach:install</td>
-		<td>rhoconnect dtach-install</td>
-	</tr>
-	<tr>
-		<td>rake redis:about</td>
-		<td>rhoconnect redis-about</td>
-	</tr>
-	<tr>
-		<td>rake redis:attach</td>
-		<td>rhoconnect redis-attach</td>
-	</tr>
-	<tr>
-		<td>rake redis:download</td>
-		<td>rhoconnect redis-download</td>
-	</tr>
-	<tr>
-		<td>rake redis:install</td>
-		<td>rhoconnect redis-install</td>
-	</tr>
-	<tr>
-		<td>rake redis:restart</td>
-		<td>rhoconnect redis-restart</td>
-	</tr>
-	<tr>
-		<td>rake redis:start</td>
-		<td>rhoconnect redis-start</td>
-	</tr>
-	<tr>
-		<td>rake redis:stop</td>
-		<td>rhoconnect redis-stop</td>
-	</tr>
-</table>
+## Rake Tasks (Removed)
+<span class="label label-important">As of version 4.0, the rhoconnect rake tasks have been removed.  You can use `rhoconnect --help` to see the list of available rhoconnect commands.</span>

data/doc/data-partitioning.txt

@@ -0,0 +1,40 @@
+Data Partitioning & Pass-Through
+===
+
+## Data Partitioning
+
+Data is stored in RhoConnect using [redis sets](http://redis.io/commands#set).  The `@result` hash from the `query` method is stored in redis and referred to as the Master Document or MD.
+
+The MD is referenced in RhoConnect by a corresponding partition.  Source adapters can partition data in two ways: user and app.  As you might have guessed, user partitioning stores a copy of the source adapter MD for each user (one copy shared across all devices for a given user).
+
+Likewise, app partitioning stores one copy of the source adapter MD for the entire application (all users and devices share the same data).  App partitioning can be particularly useful if you have source adapter models which retrieve large amounts of data that is fixed from user to user, for example a global product catalog.  Using app partitioning wherever possible ***greatly reduces*** the amount of data in redis.
+
+### User Partition
+User partitioning is the default mode for source adapters, however you can explicitly define it in `settings/settings.yml` with:
+
+    :::yaml
+    :sources:
+      Product:
+      :poll_interval: 300
+      :partition_type: user
+
+### App Partition
+Enable app partitioning the same way:
+
+    :::yaml
+    :sources:
+      Product:
+      :poll_interval: 300
+      :partition_type: app
+
+Now you have a single copy of the `Product` source adapter dataset for all users.
+
+## Pass Through
+RhoConnect provides a simple way to keep data out of redis.  If you have sensitive data that you do not want saved in redis, add the `pass_through` option in settings/settings.yml for each source:
+
+    :::yaml
+    :sources:
+      Product:
+      :pass_through: true
+
+**NOTE: When running query or search the entire data set will be returned from your backend service. **