megar 0.0.1 → 0.0.2

data/CHANGELOG

@@ -1,5 +1,10 @@
-0.0.1 Preliminary Release  26-Feb-2013
-======================================
+0.0.2 little more hearty release                 2-Mar-2013
+===========================================================
+* basic file download support added
+* refactor and extend the folder/file navigation API
+
+0.0.1 Preliminary Release                       26-Feb-2013
+===========================================================
 * An initial release with minimal functionality
 * API support: connect and get file/folder listings
 * basic CLI to connect and get file listing

data/README.rdoc

@@ -2,14 +2,16 @@
 
 Megar ("megaargh!" in pirate-speak) is a Ruby wrapper and command-line (CLI) client for the {Mega API}[https://mega.co.nz/#developers].
 
-It needs a few more API functions implemented to make it halfway useful, but it's a start.
-You can connect and get file/folder listings at the moment.
+So far this is "experimental". Megar has coverage of the basic file/folder operations: connect, get file/folder listings and details, and
+download files. There's more in the API that would be worth supporting .. a job for tomorrow!
+
 
 == Requirements and Known Limitations
 
 Consider this totally alpha at this point, especially since the Mega API has yet to be formally and fully documented.
 
 * Currently tested with MRI 1.9.3
+* Requires OpenSSL 1.0.1+ (and properly linked with ruby)
 * MRI 1.9.2 and 2.x not yet tested
 * Rubinius and JRuby 1.9 modes not yet tested
 * No plans to support 1.8 Ruby branches
@@ -20,6 +22,7 @@ Supported API functions:
 * User/session management: Complete accounts
 * Login session challenge/response
 * Retrieve file and folder nodes
+* Request download URL (and download file contents)
 
 Currently unsupported API functions:
 * User/session management: Ephemeral accounts
@@ -32,7 +35,6 @@ Currently unsupported API functions:
 * Create/delete public handle
 * Create/modify/delete outgoing share
 * Key handling - Set or request share/node keys
-* Request download URL
 * Request upload URL
 * Add/update user
 * Get user
@@ -63,6 +65,9 @@ Or install it yourself as:
 
     $ gem install megar
 
+If you want to do a little hacking around, you can also use {megar_rt}[https://github.com/tardate/megar_rt]
+to bootstrap an isolated environment and be up and running super fast.
+
 
 === How to start an authenticated session
 
@@ -79,18 +84,33 @@ It's possible to delay the authentication
     # then explicitly login:
     session.connect!
 
+
 === How to get a listing of all folders
 
 Folders are accessed through an authenticated <tt>Megar::Session</tt> object.
 
     all_folders = session.folders
 
-<tt>folders</tt> is an enumerable collection, so you can iterate as you would expect:
+<tt>folders</tt> is an {Enumerable}[http://ruby-doc.org/core-2.0/Enumerable.html] collection,
+so you can iterate as you would expect:
 
+    puts session.folders.count
     session.folders.each do |folder|
       puts folder.name
+      if parent_folder = folder.parent_folder
+          puts parent_folder.name
+      end
     end
 
+The <tt>find_all_by_type</tt> can be used to find specific node types (1 == normal folder),
+and <tt>find_by_id</tt> method will find the first id match:
+
+    session.folders.find_all_by_type(1).first.id
+     => "jtwkAQaK"
+    session.folders.find_by_id("jtwkAQaK").id
+     => "74ZTXbyR"
+
+
 === How to get a handle to the special Mega folders
 
 In addition to any folders you create yourself, Mega provides three standard folders:
@@ -102,6 +122,7 @@ grab a handle on these directly:
     my_folders.inbox
     my_folders.trash
 
+
 === What are all the attributes of a folder?
 
 Once you have a folder object (e.g. <tt>session.folders.first</tt>), the following are the
@@ -109,6 +130,20 @@ primary attributes it exposes:
 
 * id: the unique ID, corresponds to the Mega node handle
 * name: the name of the folder
+* parent_folder: handle to the parent folder (if one is assigned)
+* folders: collection of any child folders available
+* files: collection of any child files available
+
+
+=== How to get a listing of all sub-folders of a folder
+
+Folders are hierarchical, and a given folder may have a collection of sub-folders
+
+    a_folder = session.folders.first
+    a_folder.folders.each do |folder|
+      puts folder.name
+      folder.parent_folder == a_folder # true!
+    end
 
 
 === How to get a listing of all files in a folder
@@ -126,14 +161,24 @@ All files can be accessed directly without needing to navigate via a folder obje
 
     all_files = session.files
 
-<tt>files</tt> is an enumerable collection, so you can iterate as you would expect:
+<tt>files</tt> is an {Enumerable}[http://ruby-doc.org/core-2.0/Enumerable.html] collection,
+so you can iterate as you would expect:
 
+    puts session.files.count
     session.files.each do |file|
       puts file.id
       puts file.name
       puts file.size
+      puts file.parent_folder.name
     end
 
+The <tt>find_by_id</tt> method will find the first id match:
+
+    session.files.first.id
+     => "74ZTXbyR"
+    session.files.find_by_id("74ZTXbyR").id
+     => "74ZTXbyR"
+
 
 === What are all the attributes of a file?
 
@@ -143,6 +188,15 @@ primary attributes it exposes:
 * id: the unique ID, corresponds to the Mega node handle
 * name: the name of the file
 * size: the size of the file content (in bytes)
+* parent_folder: handle to the parent folder
+* body: get (download) the body content of the file
+
+=== How to download a file
+
+Once you have a file object (e.g. <tt>session.files.first</tt>):
+
+  file = session.files.first
+  file.body # returns the full decrypted body content
 
 
 === CLI: How to use the command-line client
@@ -168,6 +222,21 @@ Use the <tt>ls</tt> command:
               137080 bytes  L0xHwayA    mega.png
 
 
+=== CLI: How to download a file
+
+Use the <tt>get</tt> command:
+
+    $ megar --email=my@email.com --password=mypassword get L0xHwayA
+    Connecting to mega as my@email.com..
+    Downloading L0xHwayA to mega.png...
+
+Note: this is very simple interface at the moment:
+* you can only identify the file by the ID (not name)
+* it downloads and saves using the same name as on the server
+* and it doesn't check if you are overwriting a local file
+* no file integrity testing yet
+
+
 === How to run tests
 
 Test are implemented using rspec. Run tests with <tt>rake</tt> or <tt>rake spec</tt>.
@@ -196,6 +265,42 @@ commit and distribute credentials. Change the password on the account first.
 The tests will still run as intended with the snapshot of crypto details stored in the expectations file.
 
 
+=== Getting my OpenSSL installation up to snuff (MacOSX)
+
+These notes are specific to MacOSX. Mileage on other operating systems may vary, but this should give you
+a guide:
+
+Check the openssl version installed
+
+  $ irb -r openssl
+  1.9.3p327 :001 > OpenSSL::VERSION
+   => "1.1.0"
+  1.9.3p327 :002 > OpenSSL::OPENSSL_VERSION
+   => "OpenSSL 0.9.8r 8 Feb 2011"
+
+OK, that's too old. The OS-installed version is probably not going to change any time soon, and we don't
+want to replace it (too many unintended consequences) but we need a later version to use with ruby.
+
+You can install form source, use brew or macports, or use rvm.
+
+==== Installing OpenSSL Using RVM:
+
+See the {rvm openssl}[https://rvm.io//packages/openssl/] page for details..
+
+  $  rvm pkg install openssl
+  Fetching openssl-1.0.1c.tar.gz to ...
+
+  $ rvm reinstall 1.9.3 --with-openssl-dir=$rvm_path/usr
+
+  $ irb -r openssl
+  irb(main):001:0> OpenSSL::VERSION
+  => "1.1.0"
+  irb(main):002:0> OpenSSL::OPENSSL_VERSION
+  => "OpenSSL 1.0.1c 10 May 2012"
+
+Sweet.
+
+
 == References
 
 * {Mega API is documentation}[https://mega.co.nz/#developers]

data/lib/megar.rb

@@ -8,4 +8,5 @@ require 'megar/shell'
 require 'megar/crypto'
 require 'megar/connection'
 require 'megar/session'
+require 'megar/adapters'
 require 'megar/catalog'

data/lib/megar/adapters.rb

@@ -0,0 +1 @@
+require 'megar/adapters/file_downloader'

data/lib/megar/adapters/file_downloader.rb

@@ -0,0 +1,132 @@
+require 'open-uri'
+
+# Encapsulates a file download task. This is intended as a one-shot helper.
+#
+# Javascript reference implementation: function startdownload2(res,ctx)
+#
+class Megar::FileDownloader
+  include Megar::CryptoSupport
+
+  attr_reader :session
+  attr_reader :file
+
+  def initialize(options={})
+    @file = options[:file]
+    @session = @file && @file.session
+  end
+
+  # Returns an io stream to the file content
+  def stream
+    return unless live_session?
+    @stream ||= if url = download_url
+      open(url)
+    end
+  end
+
+  # Returns the complete decrypted content.
+  # If anything goes wrong here, it's going to bubble up an unhandled error.
+  #
+  def content
+    return unless live_session?
+    decoded_content = ''
+
+    # TODO here: init mac calculation (perhaps should be an option to use of not)
+    decryptor = session.get_file_decrypter(file.decomposed_key,iv)
+    get_chunks(download_size).each do |chunk_start, chunk_size|
+      chunk = stream.readpartial(chunk_size)
+      decoded_chunk = decryptor.update(chunk)
+      decoded_content << decoded_chunk
+      # TODO here: calculate chunk mac
+    end
+    # TODO here: perform integrity check against expected file mac
+
+    decoded_content
+  end
+
+  # Returns the complete encrypted content (mainly for testing purposes)
+  def raw_content
+    return unless live_session?
+    stream.read
+  end
+
+  # Returns a download url for the file content
+  def download_url
+    download_url_response['g']
+  end
+
+  # Returns a download size for the file content
+  def download_size
+    download_url_response['s']
+  end
+
+  # Returns the decrypted download attributes
+  def download_attributes
+    if attributes = download_url_response['at']
+      decrypt_file_attributes(attributes,file.decomposed_key)
+    end
+  end
+
+  # Returns the initialisation vector
+  def iv
+    @iv ||= file.key[4,2] + [0, 0]
+  end
+
+  # Returns the initial value for AES counter
+  def initial_counter_value
+    ((iv[0] << 32) + iv[1]) << 64
+  end
+
+  # Returns and caches a file download response
+  def download_url_response
+    @download_url_response ||= if live_session?
+      session.get_file_download_url_response(file.id)
+    else
+      {}
+    end
+  end
+
+  protected
+
+  # Returns true if live session/file properly set
+  def live_session?
+    !!(session && file)
+  end
+
+  # Returns an array of chunk sizes given total file +size+
+  #
+  # Chunk boundaries are located at the following positions:
+  # 0 / 128K / 384K / 768K / 1280K / 1920K / 2688K / 3584K / 4608K / ... (every 1024 KB) / EOF
+  def get_chunks(size)
+    chunks = []
+    p = pp = 0
+    i = 1
+
+    while i <= 8 and p < size - i * 0x20000 do
+      chunk_size =  i * 0x20000
+      chunks << [p, chunk_size]
+      pp = p
+      p += chunk_size
+      i += 1
+    end
+
+    while p < size - 0x100000 do
+      chunk_size =  0x100000
+      chunks << [p, chunk_size]
+      pp = p
+      p += chunk_size
+    end
+
+    chunks << [p, size - p] if p < size
+
+    chunks
+  end
+
+  def calculate_chunk_mac(iv,chunk)
+    chunk_mac = [iv[0], iv[1], iv[0], iv[1]]
+    (1..chunk.length).take(16).each do |bit|
+      # NYI
+    end
+    chunk_mac
+  end
+
+end

data/lib/megar/catalog/catalog_item.rb

@@ -13,6 +13,9 @@ module Megar::CatalogItem
     self.attributes = attributes
   end
 
+  # A soft-reference to the owning session
+  attr_accessor :session
+
   # The ID (Mega handle)
   attr_accessor :id
 
@@ -23,6 +26,29 @@ module Megar::CatalogItem
   # The literal mega node descriptor (as received from API)
   attr_accessor :payload
 
+  # The parent folder id
+  attr_accessor :parent_folder_id
+  alias_method :p=, :parent_folder_id=
+
+
+  # The decrypted node key
+  attr_accessor :key
+
+  # The folder type id
+  #   0: File
+  #   1: Directory
+  #   2: Special node: Root (“Cloud Drive”)
+  #   3: Special node: Inbox
+  #   4: Special node: Trash Bin
+  attr_accessor :type
+
+  # Returns a handle to the enclosing folder (if any)
+  def parent_folder
+    if session && parent_folder_id
+      session.folders.find_by_id(parent_folder_id)
+    end
+  end
+
   # Assigns the payload attribute, also splitting out separate attribute assignments from +value+ if a hash
   def payload=(value)
     self.attributes = value
@@ -39,26 +65,22 @@ module Megar::CatalogItem
     end
   end
 
-  # The decrypted node key
-  attr_accessor :key
-
-  # The folder type id
-  #   0: File
-  #   1: Directory
-  #   2: Special node: Root (“Cloud Drive”)
-  #   3: Special node: Inbox
-  #   4: Special node: Trash Bin
-  attr_accessor :type
-
-
   # Generic interface to return the currently applicable collection
   def collection
     @collection ||= []
   end
 
+  # Adds an item to the local cached collection given +attributes+ hash.
+  def add(attributes)
+    return false unless resource_class
+    collection << resource_class.new(attributes.merge(session: self.session))
+  end
+
   # Returns the expected class of items in the collection
   def resource_class
     "#{self.class.name}".chomp('s').constantize
+  rescue
+    nil
   end
 
   # Command: clears/re-initialises the collection
@@ -82,9 +104,29 @@ module Megar::CatalogItem
   end
   alias_method :eql?, :==
 
+  # Returns the first record matching +id+
+  def find_by_id(id)
+    find { |r| r.id == id }
+  end
+
   # Returns the first record matching +type+
   def find_by_type(type)
     find { |r| r.type == type }
   end
 
+  # Returns all records matching +type+
+  def find_all_by_type(type)
+    find_all { |r| r.type == type }
+  end
+
+  # Returns the first record matching +name+
+  def find_by_name(name)
+    find { |r| r.name == name }
+  end
+
+  # Returns all records matching +parent_folder_id+
+  def find_all_by_parent_folder_id(parent_folder_id)
+    find_all { |r| r.parent_folder_id == parent_folder_id }
+  end
+
 end