waz-storage 1.1.1 → 1.1.2

data/.gitignore

@@ -0,0 +1,7 @@
+tests/waz/func/*.rb
+coverage
+pkg/*
+rdoc/*
+sdk/*
+sdk
+.rvmrc

data/CHANGELOG.rdoc

@@ -0,0 +1,72 @@
+= Revision History for Windows Azure Storage for Ruby
+=== v1.1 - (smarx)
+- Upload from stream, plus XML/URI escaping in various places, and a few other minor fixes 
+- Add upload method to container
+- Add put_block_list to support upload
+- Fix message canonicalization of ?comp= parameters for versions prior to 2009-09-19 (seems put_block was broken)
+- Unescape query string parameters when constructing signatures (needed for put_block, since some base64-encoded names include non-URI-friendly characters)
+- XML-escape property values for table entities (needed to insert things containing &, <, etc.)
+- Remove Unicode characters from rakefile (was breaking something on Windows)
+- Add tests for put_block_list and upload
+- Fix tests for content type (typo, = instead of .should ==)
+- Fix tests using RestClient.beautify_headers (expects an array, not a scalar)
+
+=== v1.0.6 - (hermes.logicalbricks)
+- Update tests to rspec 2.5.0
+- Allow authorization using only SharedAccessSignature 
+
+=== v1.0.5 - (tomconte)
+- Fixed for list_blobs as it wasn't fully merged to 2009-09-19. Now it works without 403 exceptions
+
+=== v1.0.3 - (Johnny Halife)
+- Merged with sriramk fix for loading path issues on some environments
+
+=== v1.0.2 - (Johnny Halife)
+- Completed Blobs API migration to 2009-09-19, _fully supporting_ what third-party tools does (e.g. Cerebrata) [thanks percent20] 
+
+=== v1.0.1 - (Johnny halife)
+- Added syntax's sugar for ensuring a queue (get or create)
+
+=== v1.0 - (Juan Pablo Garcia Dalolla)
+- Added support for table service to query, get_one, insert, update, merge and delete entities.
+- Added support for running against the Storage Developement Fabriq shipped with Microsoft SDK.
+- Added signature support for Tables service according to msdn.microsoft.com/en-us/library/dd179428.aspx
+- Added support to enumerate, create, and delete tables on give storage account.
+- Improved support for stacked connection management.
+
+=== v0.5.81 - (Johnny Halife)
+- When simulating a container using a forward slash starting the blob name it crashed with 404, now it's fixed.
+
+=== v0.5.8 - (Johnny Halife)
+- Fixed an issue on the Queues Documentation (thanks @ezequielm)
+- Added _naming validation for Containers_ according to http://msdn.microsoft.com/en-us/library/dd135715.aspx (thanks @panosjee)
+- Added _naming validation for Queues_ according to http://msdn.microsoft.com/en-us/library/dd179349.aspx (thanks @panosjee)
+- Added _support for Blob snapshots_ 
+- Added support for rest-client 1.3.0. (thanks @jpgd)
+
+=== v0.5.7 - (Johnny Halife)
+- Fixes a critical bug on URL management that some time prevents message from being deleted, caused by a missing encoding on the URL 	parameters.
+
+=== v0.5.6 -  (Johnny Halife)
+- Added signature support for 2009-09-19 Version of the Storage API
+- Queues API has been migrated to the 2009-09-19 Version of the Storage API
+- Added a new parameter for listing queues with metadata
+- Added support for DequeueCount on messages being retrieved from the Queue
+- <strong>Known Issue</strong>: Creating a queue multiple times with same metadata throws 409.
+
+=== v0.5.5 -  (Johnny Halife)
+- Added a Change Log
+- Added support for copy-blob [WAZ::Blobs::BlobObject.copy("container/blob_name")]
+- Refactored out the whole request logic to an _execute_ method on the SharedKeyCoreService
+- Added transparent support for block management (users don't know if their blob is being uploaded chunked or not)
+- Added support for a blob to list its blocks
+
+=== v0.5.4 - Sat Oct 17 19:35:48 2009 -0300 (Johnny Halife)
+- Fixed method name for String.start_with? (previously starts_with?)
+
+=== v0.5.3 - Sat Oct 17 18:20:51 2009 -0300 (Johnny Halife)
+- Added the stack-based contextual connection handling (WAZ::Storage::Base.establish_connection {|block| } 
+- Added string extensions for non rails environments (and work to work on heroku) String.starts_with?
+
+=== v0.5.2 - Thu Oct 15 22:08:01 2009 -0300 (Johnny Halife)
+- Initial Version

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in waz-storage.gemspec
+gemspec

data/Gemfile.lock

@@ -0,0 +1,36 @@
+PATH
+  remote: .
+  specs:
+    waz-storage (1.1.1)
+      rest-client
+      ruby-hmac
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    diff-lcs (1.1.3)
+    metaclass (0.0.1)
+    mime-types (1.18)
+    mocha (0.10.5)
+      metaclass (~> 0.0.1)
+    rcov (1.0.0)
+    rest-client (1.6.7)
+      mime-types (>= 1.16)
+    rspec (2.9.0)
+      rspec-core (~> 2.9.0)
+      rspec-expectations (~> 2.9.0)
+      rspec-mocks (~> 2.9.0)
+    rspec-core (2.9.0)
+    rspec-expectations (2.9.1)
+      diff-lcs (~> 1.1.3)
+    rspec-mocks (2.9.0)
+    ruby-hmac (0.4.0)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  mocha
+  rcov
+  rspec
+  waz-storage!

data/LICENSE

@@ -0,0 +1,19 @@
+Copyright (c) 2009 Johnny G. Halife
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,307 @@
+= Why this fork ?
+
+The main difference with the original library is the adding of the method "railsetag".
+This method add to any blob object an "x_ms_meta_railsetag" header, its value is calculated as the rails etag way (Digest::MD5.hexdigest(file))
+
+By doing so we can easily know if a locale copy of a file is up-to-date on Azure CDN.
+This is very much used in the "waz-sync" gem which purpose is to sync file between local storage and Azure. 
+
+= Windows Azure Storage library — simple gem for accessing WAZ‘s Storage REST API 
+A simple implementation of Windows Azure Storage API for Ruby, inspired by the S3 gems and self experience of dealing with queues. The major
+goal of the whole gem is to enable ruby developers [like me =)] to leverage Windows Azure Storage features and have another option 
+for cloud storage.
+
+The whole gem is implemented based on Microsoft's specs from the communication and underlying service description and protocol (REST). The API
+is for ruby developers built by a ruby developer. I'm trying to follow idioms, patterns and fluent type of doing APIs on Ruby.
+
+This work isn't related at all with StorageClient Sample shipped with Microsoft SDK and written in .NET, the whole API is based on my own understanding,
+experience and values of elegance and ruby development.
+
+Full documentation for the gem is available at http://waz-storage.heroku.com
+
+=== How does this differ from <em>waz-queues</em> and <em>waz-blobs</em> work?
+Well, this is a sum up of the whole experience of writing those gems and getting them to work together to simplify 
+end user experience. Although there're some breaking changes, it's pretty backward compatible with existing gems.
+
+=== What's new on the v1.1 version? [thanks to smarx]
+- Upload from stream, plus XML/URI escaping in various places, and a few other minor fixes 
+- Add upload method to container
+- Add put_block_list to support upload
+- Fix message canonicalization of ?comp= parameters for versions prior to 2009-09-19 (seems put_block was broken)
+- Unescape query string parameters when constructing signatures (needed for put_block, since some base64-encoded names include non-URI-friendly characters)
+- XML-escape property values for table entities (needed to insert things containing &, <, etc.)
+- Remove Unicode characters from rakefile (was breaking something on Windows)
+- Add tests for put_block_list and upload
+- Fix tests for content type (typo, = instead of .should ==)
+- Fix tests using RestClient.beautify_headers (expects an array, not a scalar)
+
+=== What's new on the v1.0.6 version? [thanks to hermes.logicalbricks]
+- Update tests to rspec 2.5.0
+- Allow authorization using only SharedAccessSignature
+
+=== What's new on the version version?
+- Fixed for list_blobs as it wasn't fully merged to 2009-09-19. Now it works without 403 exceptions [Thanks tomconte]
+
+=== What's new on the v1.0.3 version?
+- Merged with sriramk fix for loading path issues on some environments
+
+=== What's new on the 1.0.2 version?
+- Completed Blobs API migration to 2009-09-19, _fully supporting_ what third-party tools does (e.g. Cerebrata) [thanks percent20]
+
+=== What's new on the 1.0.1 version?
+- Added Syntax's sugar for ensuring a Queue (get or create)
+
+=== What's new on the 1.0 version?
+- Added support for table service to query, get_one, insert, update, merge and delete entities.
+- Added support for running against the Storage Developement Fabriq shipped with Microsoft SDK.
+- Added signature support for Tables service according to msdn.microsoft.com/en-us/library/dd179428.aspx
+- Added support to enumerate, create, and delete tables on give storage account.
+- Improved support for stacked connection management.
+
+=== What's new on the 0.5.81 version?
+- When simulating a container using a forward slash starting the blob name it crashed with 404, now it's fixed.
+
+=== What's new on the 0.5.8 version?
+- Fixed an issue on the Queues Documentation (thanks @ezequielm)
+- Added <b>naming validation for Containers</b> according to http://msdn.microsoft.com/en-us/library/dd135715.aspx (thanks @panosjee)
+- Added <b>naming validation for Queues</b> according to http://msdn.microsoft.com/en-us/library/dd179349.aspx (thanks @panosjee)
+- Added <b>support for Blob snapshots</b> 
+- Added support for rest-client 1.3.0. (thanks @jpgd)
+
+=== What's new on the 0.5.7 version?
+- Fixes a critical bug on URL management that some time prevents messages from being deleted, caused by a missing encoding on the URL	parameters.
+
+=== What's new on the 0.5.6 version?
+- Added new <b>shared key authentication support for 2009-09-19 Version of the Storage API</b>
+- Queues API has been migrated to the <b>2009-09-19 Version of the Storage API</b>
+- Added a new parameter for <b>listing queues with metadata</b>
+- Added support for <b>DequeueCount</b> on messages being retrieved from the Queue
+- <b>Known Issue</b>: Creating a queue multiple times with same metadata throws 409.
+
+== Getting started
+	sudo gem install waz-storage --source http://gemcutter.org
+	
+== Basic Configuration
+One of the major changes from the waz-queues and waz-blobs APIs was the ability to set a single storage account (99% of the cases) to be
+used on your code just once. Since Windows Azure Storage Services are all related to a single account that includes Tables, Queues and Blobs,
+I've changed the API in order to make ease that administration by requiring you to set the configuration just once:
+
+	require 'waz-queues' #=> if you want to use queues 
+	require 'waz-blobs' #=> if you want to use blobs (or you can include both for using both) 
+
+	WAZ::Storage::Base.establish_connection!(:account_name => account_name,
+	                                         :access_key => access_key)
+	
+As you can see you, the way to get started it pretty simple, just include the establish_connection! call on your application bootstrapper, for example 
+if you are doing a sinatra-rb application it can be
+
+	configure :production do
+		# here is were you hook up with WAZ Storage Stuff.
+	end
+
+That's all you need to get started and running your Windows Azure Code on a sinatra application.
+
+== Usage: Queues
+Windows Azure Queues are extremely powerful for asyc programming, they are very useful when you don’t need "always consistent" 
+data operations allowing you to enqueue and process on background. It’s important to mention that being WAZ-Queues a buffered 
+message system, you should design for idempotent operations given the fact that there’s no guarantee that a message can be repeated.
+
+The implementation of the Windows Azure Queues REST API available online at http://msdn.microsoft.com/en-us/library/dd179363.aspx is fully 
+covered here. 
+
+Here's a quick getting started of the usage of the API:
+
+	WAZ::Storage::Base.establish_connection!(:account_name => account_name,
+	                                         :access_key => access_key)
+
+	# NEW! Now you can do ensure and it will return the queue, first trying to retrieve it 
+	# and then if it doesn't exist, will created it
+	queue = WAZ::Queues::Queue.ensure('my-queue')
+	
+	# excepts that the metadata for the queue changes this method behaves as PUT (create/replace)
+	# remarks: it performs a validation whether metadata changed or not (if changed HTTP 409 conflict)
+	queue = WAZ::Queues::Queue.create('my-queue')
+
+	10.times do |m|
+	  # enqueue a receives string. Message content can be anything up to 8KB
+	  # you can serialize and send anything that serializes to UTF-8 string (JSON, XML, etc)
+	  queue.enqueue!("message##{m}")
+	end
+
+	while(queue.size > 0) do
+	  # Since WAZ implements the peek lock pattern we are locking messages (not dequeuing)
+	  # it has two parameters how many messages and for how long they are locked
+	  messages = queue.lock(10)
+
+	  puts "dequeued message: #{messages.size}"
+
+	  # deletes the message from the queue so other clients do not pick it after
+	  # visibility time out expires
+	  messages.each {|m| m.destroy!}
+	end
+		
+It's pretty intuitive, but full documentation (RDoc) is available for the API for further reference.
+
+== Usage: Blobs
+The blobs implementation inside this gem is fully compliant with the spec available at http://msdn.microsoft.com/en-us/library/dd135733.aspx. The Windows 
+Azure Blobs REST API isn't fully covered here (see TODO's for more information). It's pretty usable and stable right now, 
+I've been doing lot of testing around and it works seamlessly with the current Windows Azure implementation.
+
+	require 'waz-blobs'
+
+	WAZ::Storage::Base.establish_connection!(:account_name => account_name,
+	                                         :access_key => access_key)
+
+	# creates a container
+	container = WAZ::Blobs::Container.create('my-container')
+
+	# stores a blob with custom properties (metadata)
+	blob = container.store('my_blob.txt',  
+				 'this is the content of my blob', 
+				 'plain/text', 
+				 {:x_ms_meta_Custom_Property => "custom_value" })
+
+	# return a specific blob from a container
+	blob = container['my_blob.txt']
+
+	# retrieves a blob value
+	blob.value
+
+It's pretty intuitive, but full documentation (RDoc) is available for the API for further reference.	
+
+=== Using Blob with Shared Acess Signature
+This feature allow to access a blob only with the information of the blob's name, the container, and the Shared Access Signature. If you don't have permission to perform an action (for example: list the blobs, create a new blob or something like that) you will receive an exception. If you want more details about this: http://msdn.microsoft.com/en-us/library/ee395415.aspx  
+
+	require 'waz-blobs'
+	
+	WAZ::Storage::Base.establish_connection!(:account_name=> 'mysharedblob', 
+						:use_sas_auth_only=> true,
+						 :sharedaccesssignature =>"?se=XXXXXX&sr=c&si=XXXXX&sig=XXXXXXXXXXXXXXXXXXXXX")
+
+	container = WAZ::Blobs::Container.new(:name=>'container')
+
+	blob = container.store('myfile.txt',File.read('fileofthecontent.txt'),'plain/text')
+
+The feature was only tested storing a blob.
+
+== Usage: Tables
+The tables implementation inside this gem is fully compliant with the spec available at http://msdn.microsoft.com/en-us/library/dd179423.aspx. The Windows 
+Azure Tables REST API is fully covered here.
+
+	require 'waz-storage'
+	require 'waz-tables'
+
+	WAZ::Storage::Base.establish_connection!(:account_name => account_name,
+	                                     :access_key => access_key)
+
+	# creates a new table
+	table = WAZ::Tables::Table.create('my-table')
+
+	# list available tables 
+	# (returns a maximum of 1,000 items at one time (more details here: http://msdn.microsoft.com/en-us/library/dd135718.aspx)
+	tables = WAZ::Tables::Table.list
+
+	# more tables
+	WAZ::Tables::Table.list(tables.continuation_token)
+
+	# get a specific table, returning nil when the specified table is not found
+	my_table = WAZ::Tables::Table.find('my-table')
+
+	# table properties
+	my_table.name
+	my_table.url
+
+	# delete a table
+	my_table.destroy!
+
+	# Entity operations using the Table service
+
+	# get the existing service instance
+	service = WAZ::Tables::Table.service_instance
+
+	# define a new entity
+    entity = { :address => 'Mountain View',
+               :age => 23,
+               :amount_due => 200.23,
+               :binary_data => File.open(__FILE__),
+               :customer_code => 'aaaaaaaa-bbbb-cccc-dddd-aaaabbbbcccc',
+               :customer_since => Time.now.utc,
+               :is_active => true,
+               :num_of_orders => 255,
+               :partition_key => 'customer',
+               :row_key => "myRowKey#{rand(2000000).to_s}",
+               :Timestamp => Time.now.utc }
+
+	# inserts a new entity
+	service.insert_entity('customer_table', entity)
+
+	# retrieves all entities from a table 
+	# (returns a maximum of 1,000 items at one time (more details here: http://msdn.microsoft.com/en-us/library/dd135718.aspx)
+	# Remarks on development storage it retrieves all items instead the first 1,000
+	entities = service.query('customer_table')
+
+	# retrieves more entities providing the obtained continuation_token
+	service.query('customer_table', {:continuation_token => entities.continuation_token} )
+
+	# retrieves all records that match with the specified query but only the first fifteen rows
+	service.query('customer_table', {:expression => "(PartitionKey eq 'customer') and (Age eq 23)", :top => 15} )
+
+	# get an existing entity by its partion_key and row_key
+	entity = service.get_entity('customer_table', 'customer', 'rowKey1')
+
+	# updates an entity
+	entity[:age] = 90
+	service.update_entity('customer_table', new_entity)
+
+	# merges an entity (more details here: http://msdn.microsoft.com/en-us/library/dd179392.aspx)
+	entity[:age] = 20
+	service.merge_entity('customer_table', new_entity)
+
+	# deletes an entity
+	service.delete_entity('customer_table', 'customer', 'rowKey1')
+
+It's pretty intuitive, but full documentation (RDoc) is available for the API for further reference.
+
+== Usage: Contextual Connection Handling
+Sometimes while you are building a web application you may require handling different storage account but contextualized. The sample
+that comes to my mind is something like a Storage Explorer or Account Monitor for WAZ. 
+
+That is why? I've added a new way of handling a stack-based contextual connection handling. The usage is pretty simple:
+	
+	WAZ::Storage::Base.establish_connection(options) do
+		container = WAZ::Blobs::Container.find('container-name')
+		blob = container['blob-name']
+		blob.destroy!
+	end
+
+As it is described on the example above, there's a new way of establishing a connection and use it on a given block. The whole implementation
+is stack based, and will let you scope your context for some rare cases where you have another account.
+
+*Disclaimer*: Moving objects across context isn't contemplated yet, and if you try to do changes among scopes you will get 
+to some wired Windows Azure Errors regarding objects that may not exist.
+
+=== Remarks
+Windows Azure Storage API works flawlessly from Heroku and custom ruby hosting deployments on EC2, as far as I tested it. You can leverage the storage
+services without the need of having to write the application on .NET or hosting your application on Windows Azure.
+
+The documentation and implementation exposed here is for the pre-release version and is subject to change on the future. 
+
+=== TODO's
+As far as users start using it, I'll be building a backlog and probably handling a wish-list of features, but right now I've the following 
+TODO's already enqueued for further releases of the waz-storage API.
+
+-Generate a sample application to better show the usage.
+
+The things listed above do not represent any sort of priority, or the order they are going to be tackled. It's just a list. 
+
+== Meta
+
+Written by Johnny G. Halife (johnny.halife at me dot com)
+
+contributed by: Ezequiel Morito (http://twitter.com/ezequielm), Juan Pablo Garcia (http://twitter.com/jpgd), Steve Marx (http://twitter.com/smarx)
+
+Released under the MIT License: http://www.opensource.org/licenses/mit-license.php
+
+http://waz-storage.heroku.com
+
+http://github.com/johnnyhalife/waz-storage