megar 0.0.1 → 0.0.2
Sign up to get free protection for your applications and to get access to all the features.
- data/CHANGELOG +7 -2
- data/README.rdoc +110 -5
- data/lib/megar.rb +1 -0
- data/lib/megar/adapters.rb +1 -0
- data/lib/megar/adapters/file_downloader.rb +132 -0
- data/lib/megar/catalog/catalog_item.rb +54 -12
- data/lib/megar/catalog/file.rb +13 -0
- data/lib/megar/catalog/files.rb +0 -8
- data/lib/megar/catalog/folder.rb +12 -0
- data/lib/megar/catalog/folders.rb +0 -8
- data/lib/megar/crypto.rb +1 -399
- data/lib/megar/crypto/support.rb +422 -0
- data/lib/megar/exception.rb +3 -0
- data/lib/megar/session.rb +35 -19
- data/lib/megar/shell.rb +21 -3
- data/lib/megar/version.rb +1 -1
- data/spec/fixtures/crypto_expectations/megar_test_sample_1.txt.enc +1 -0
- data/spec/fixtures/crypto_expectations/megar_test_sample_2.png.enc +0 -0
- data/spec/fixtures/crypto_expectations/sample_user.json +90 -6
- data/spec/fixtures/sample_files/megar_test_sample_1.txt +1 -0
- data/spec/fixtures/sample_files/megar_test_sample_2.png +0 -0
- data/spec/support/crypto_expectations_helper.rb +81 -8
- data/spec/support/mocks_helper.rb +26 -4
- data/spec/unit/adapters/file_downloader_spec.rb +100 -0
- data/spec/unit/catalog/file_spec.rb +22 -5
- data/spec/unit/catalog/folder_spec.rb +54 -3
- data/spec/unit/{crypto_spec.rb → crypto/support_spec.rb} +33 -6
- data/spec/unit/exception_spec.rb +8 -0
- data/spec/unit/session_spec.rb +41 -0
- data/spec/unit/shell_spec.rb +28 -0
- metadata +18 -5
data/CHANGELOG
CHANGED
@@ -1,5 +1,10 @@
|
|
1
|
-
0.0.
|
2
|
-
|
1
|
+
0.0.2 little more hearty release 2-Mar-2013
|
2
|
+
===========================================================
|
3
|
+
* basic file download support added
|
4
|
+
* refactor and extend the folder/file navigation API
|
5
|
+
|
6
|
+
0.0.1 Preliminary Release 26-Feb-2013
|
7
|
+
===========================================================
|
3
8
|
* An initial release with minimal functionality
|
4
9
|
* API support: connect and get file/folder listings
|
5
10
|
* basic CLI to connect and get file listing
|
data/README.rdoc
CHANGED
@@ -2,14 +2,16 @@
|
|
2
2
|
|
3
3
|
Megar ("megaargh!" in pirate-speak) is a Ruby wrapper and command-line (CLI) client for the {Mega API}[https://mega.co.nz/#developers].
|
4
4
|
|
5
|
-
|
6
|
-
|
5
|
+
So far this is "experimental". Megar has coverage of the basic file/folder operations: connect, get file/folder listings and details, and
|
6
|
+
download files. There's more in the API that would be worth supporting .. a job for tomorrow!
|
7
|
+
|
7
8
|
|
8
9
|
== Requirements and Known Limitations
|
9
10
|
|
10
11
|
Consider this totally alpha at this point, especially since the Mega API has yet to be formally and fully documented.
|
11
12
|
|
12
13
|
* Currently tested with MRI 1.9.3
|
14
|
+
* Requires OpenSSL 1.0.1+ (and properly linked with ruby)
|
13
15
|
* MRI 1.9.2 and 2.x not yet tested
|
14
16
|
* Rubinius and JRuby 1.9 modes not yet tested
|
15
17
|
* No plans to support 1.8 Ruby branches
|
@@ -20,6 +22,7 @@ Supported API functions:
|
|
20
22
|
* User/session management: Complete accounts
|
21
23
|
* Login session challenge/response
|
22
24
|
* Retrieve file and folder nodes
|
25
|
+
* Request download URL (and download file contents)
|
23
26
|
|
24
27
|
Currently unsupported API functions:
|
25
28
|
* User/session management: Ephemeral accounts
|
@@ -32,7 +35,6 @@ Currently unsupported API functions:
|
|
32
35
|
* Create/delete public handle
|
33
36
|
* Create/modify/delete outgoing share
|
34
37
|
* Key handling - Set or request share/node keys
|
35
|
-
* Request download URL
|
36
38
|
* Request upload URL
|
37
39
|
* Add/update user
|
38
40
|
* Get user
|
@@ -63,6 +65,9 @@ Or install it yourself as:
|
|
63
65
|
|
64
66
|
$ gem install megar
|
65
67
|
|
68
|
+
If you want to do a little hacking around, you can also use {megar_rt}[https://github.com/tardate/megar_rt]
|
69
|
+
to bootstrap an isolated environment and be up and running super fast.
|
70
|
+
|
66
71
|
|
67
72
|
=== How to start an authenticated session
|
68
73
|
|
@@ -79,18 +84,33 @@ It's possible to delay the authentication
|
|
79
84
|
# then explicitly login:
|
80
85
|
session.connect!
|
81
86
|
|
87
|
+
|
82
88
|
=== How to get a listing of all folders
|
83
89
|
|
84
90
|
Folders are accessed through an authenticated <tt>Megar::Session</tt> object.
|
85
91
|
|
86
92
|
all_folders = session.folders
|
87
93
|
|
88
|
-
<tt>folders</tt> is an
|
94
|
+
<tt>folders</tt> is an {Enumerable}[http://ruby-doc.org/core-2.0/Enumerable.html] collection,
|
95
|
+
so you can iterate as you would expect:
|
89
96
|
|
97
|
+
puts session.folders.count
|
90
98
|
session.folders.each do |folder|
|
91
99
|
puts folder.name
|
100
|
+
if parent_folder = folder.parent_folder
|
101
|
+
puts parent_folder.name
|
102
|
+
end
|
92
103
|
end
|
93
104
|
|
105
|
+
The <tt>find_all_by_type</tt> can be used to find specific node types (1 == normal folder),
|
106
|
+
and <tt>find_by_id</tt> method will find the first id match:
|
107
|
+
|
108
|
+
session.folders.find_all_by_type(1).first.id
|
109
|
+
=> "jtwkAQaK"
|
110
|
+
session.folders.find_by_id("jtwkAQaK").id
|
111
|
+
=> "74ZTXbyR"
|
112
|
+
|
113
|
+
|
94
114
|
=== How to get a handle to the special Mega folders
|
95
115
|
|
96
116
|
In addition to any folders you create yourself, Mega provides three standard folders:
|
@@ -102,6 +122,7 @@ grab a handle on these directly:
|
|
102
122
|
my_folders.inbox
|
103
123
|
my_folders.trash
|
104
124
|
|
125
|
+
|
105
126
|
=== What are all the attributes of a folder?
|
106
127
|
|
107
128
|
Once you have a folder object (e.g. <tt>session.folders.first</tt>), the following are the
|
@@ -109,6 +130,20 @@ primary attributes it exposes:
|
|
109
130
|
|
110
131
|
* id: the unique ID, corresponds to the Mega node handle
|
111
132
|
* name: the name of the folder
|
133
|
+
* parent_folder: handle to the parent folder (if one is assigned)
|
134
|
+
* folders: collection of any child folders available
|
135
|
+
* files: collection of any child files available
|
136
|
+
|
137
|
+
|
138
|
+
=== How to get a listing of all sub-folders of a folder
|
139
|
+
|
140
|
+
Folders are hierarchical, and a given folder may have a collection of sub-folders
|
141
|
+
|
142
|
+
a_folder = session.folders.first
|
143
|
+
a_folder.folders.each do |folder|
|
144
|
+
puts folder.name
|
145
|
+
folder.parent_folder == a_folder # true!
|
146
|
+
end
|
112
147
|
|
113
148
|
|
114
149
|
=== How to get a listing of all files in a folder
|
@@ -126,14 +161,24 @@ All files can be accessed directly without needing to navigate via a folder obje
|
|
126
161
|
|
127
162
|
all_files = session.files
|
128
163
|
|
129
|
-
<tt>files</tt> is an
|
164
|
+
<tt>files</tt> is an {Enumerable}[http://ruby-doc.org/core-2.0/Enumerable.html] collection,
|
165
|
+
so you can iterate as you would expect:
|
130
166
|
|
167
|
+
puts session.files.count
|
131
168
|
session.files.each do |file|
|
132
169
|
puts file.id
|
133
170
|
puts file.name
|
134
171
|
puts file.size
|
172
|
+
puts file.parent_folder.name
|
135
173
|
end
|
136
174
|
|
175
|
+
The <tt>find_by_id</tt> method will find the first id match:
|
176
|
+
|
177
|
+
session.files.first.id
|
178
|
+
=> "74ZTXbyR"
|
179
|
+
session.files.find_by_id("74ZTXbyR").id
|
180
|
+
=> "74ZTXbyR"
|
181
|
+
|
137
182
|
|
138
183
|
=== What are all the attributes of a file?
|
139
184
|
|
@@ -143,6 +188,15 @@ primary attributes it exposes:
|
|
143
188
|
* id: the unique ID, corresponds to the Mega node handle
|
144
189
|
* name: the name of the file
|
145
190
|
* size: the size of the file content (in bytes)
|
191
|
+
* parent_folder: handle to the parent folder
|
192
|
+
* body: get (download) the body content of the file
|
193
|
+
|
194
|
+
=== How to download a file
|
195
|
+
|
196
|
+
Once you have a file object (e.g. <tt>session.files.first</tt>):
|
197
|
+
|
198
|
+
file = session.files.first
|
199
|
+
file.body # returns the full decrypted body content
|
146
200
|
|
147
201
|
|
148
202
|
=== CLI: How to use the command-line client
|
@@ -168,6 +222,21 @@ Use the <tt>ls</tt> command:
|
|
168
222
|
137080 bytes L0xHwayA mega.png
|
169
223
|
|
170
224
|
|
225
|
+
=== CLI: How to download a file
|
226
|
+
|
227
|
+
Use the <tt>get</tt> command:
|
228
|
+
|
229
|
+
$ megar --email=my@email.com --password=mypassword get L0xHwayA
|
230
|
+
Connecting to mega as my@email.com..
|
231
|
+
Downloading L0xHwayA to mega.png...
|
232
|
+
|
233
|
+
Note: this is very simple interface at the moment:
|
234
|
+
* you can only identify the file by the ID (not name)
|
235
|
+
* it downloads and saves using the same name as on the server
|
236
|
+
* and it doesn't check if you are overwriting a local file
|
237
|
+
* no file integrity testing yet
|
238
|
+
|
239
|
+
|
171
240
|
=== How to run tests
|
172
241
|
|
173
242
|
Test are implemented using rspec. Run tests with <tt>rake</tt> or <tt>rake spec</tt>.
|
@@ -196,6 +265,42 @@ commit and distribute credentials. Change the password on the account first.
|
|
196
265
|
The tests will still run as intended with the snapshot of crypto details stored in the expectations file.
|
197
266
|
|
198
267
|
|
268
|
+
=== Getting my OpenSSL installation up to snuff (MacOSX)
|
269
|
+
|
270
|
+
These notes are specific to MacOSX. Mileage on other operating systems may vary, but this should give you
|
271
|
+
a guide:
|
272
|
+
|
273
|
+
Check the openssl version installed
|
274
|
+
|
275
|
+
$ irb -r openssl
|
276
|
+
1.9.3p327 :001 > OpenSSL::VERSION
|
277
|
+
=> "1.1.0"
|
278
|
+
1.9.3p327 :002 > OpenSSL::OPENSSL_VERSION
|
279
|
+
=> "OpenSSL 0.9.8r 8 Feb 2011"
|
280
|
+
|
281
|
+
OK, that's too old. The OS-installed version is probably not going to change any time soon, and we don't
|
282
|
+
want to replace it (too many unintended consequences) but we need a later version to use with ruby.
|
283
|
+
|
284
|
+
You can install form source, use brew or macports, or use rvm.
|
285
|
+
|
286
|
+
==== Installing OpenSSL Using RVM:
|
287
|
+
|
288
|
+
See the {rvm openssl}[https://rvm.io//packages/openssl/] page for details..
|
289
|
+
|
290
|
+
$ rvm pkg install openssl
|
291
|
+
Fetching openssl-1.0.1c.tar.gz to ...
|
292
|
+
|
293
|
+
$ rvm reinstall 1.9.3 --with-openssl-dir=$rvm_path/usr
|
294
|
+
|
295
|
+
$ irb -r openssl
|
296
|
+
irb(main):001:0> OpenSSL::VERSION
|
297
|
+
=> "1.1.0"
|
298
|
+
irb(main):002:0> OpenSSL::OPENSSL_VERSION
|
299
|
+
=> "OpenSSL 1.0.1c 10 May 2012"
|
300
|
+
|
301
|
+
Sweet.
|
302
|
+
|
303
|
+
|
199
304
|
== References
|
200
305
|
|
201
306
|
* {Mega API is documentation}[https://mega.co.nz/#developers]
|
data/lib/megar.rb
CHANGED
@@ -0,0 +1 @@
|
|
1
|
+
require 'megar/adapters/file_downloader'
|
@@ -0,0 +1,132 @@
|
|
1
|
+
require 'open-uri'
|
2
|
+
|
3
|
+
# Encapsulates a file download task. This is intended as a one-shot helper.
|
4
|
+
#
|
5
|
+
# Javascript reference implementation: function startdownload2(res,ctx)
|
6
|
+
#
|
7
|
+
class Megar::FileDownloader
|
8
|
+
include Megar::CryptoSupport
|
9
|
+
|
10
|
+
attr_reader :session
|
11
|
+
attr_reader :file
|
12
|
+
|
13
|
+
def initialize(options={})
|
14
|
+
@file = options[:file]
|
15
|
+
@session = @file && @file.session
|
16
|
+
end
|
17
|
+
|
18
|
+
# Returns an io stream to the file content
|
19
|
+
def stream
|
20
|
+
return unless live_session?
|
21
|
+
@stream ||= if url = download_url
|
22
|
+
open(url)
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
# Returns the complete decrypted content.
|
27
|
+
# If anything goes wrong here, it's going to bubble up an unhandled error.
|
28
|
+
#
|
29
|
+
def content
|
30
|
+
return unless live_session?
|
31
|
+
decoded_content = ''
|
32
|
+
|
33
|
+
# TODO here: init mac calculation (perhaps should be an option to use of not)
|
34
|
+
decryptor = session.get_file_decrypter(file.decomposed_key,iv)
|
35
|
+
get_chunks(download_size).each do |chunk_start, chunk_size|
|
36
|
+
chunk = stream.readpartial(chunk_size)
|
37
|
+
decoded_chunk = decryptor.update(chunk)
|
38
|
+
decoded_content << decoded_chunk
|
39
|
+
# TODO here: calculate chunk mac
|
40
|
+
end
|
41
|
+
# TODO here: perform integrity check against expected file mac
|
42
|
+
|
43
|
+
decoded_content
|
44
|
+
end
|
45
|
+
|
46
|
+
# Returns the complete encrypted content (mainly for testing purposes)
|
47
|
+
def raw_content
|
48
|
+
return unless live_session?
|
49
|
+
stream.read
|
50
|
+
end
|
51
|
+
|
52
|
+
# Returns a download url for the file content
|
53
|
+
def download_url
|
54
|
+
download_url_response['g']
|
55
|
+
end
|
56
|
+
|
57
|
+
# Returns a download size for the file content
|
58
|
+
def download_size
|
59
|
+
download_url_response['s']
|
60
|
+
end
|
61
|
+
|
62
|
+
# Returns the decrypted download attributes
|
63
|
+
def download_attributes
|
64
|
+
if attributes = download_url_response['at']
|
65
|
+
decrypt_file_attributes(attributes,file.decomposed_key)
|
66
|
+
end
|
67
|
+
end
|
68
|
+
|
69
|
+
# Returns the initialisation vector
|
70
|
+
def iv
|
71
|
+
@iv ||= file.key[4,2] + [0, 0]
|
72
|
+
end
|
73
|
+
|
74
|
+
# Returns the initial value for AES counter
|
75
|
+
def initial_counter_value
|
76
|
+
((iv[0] << 32) + iv[1]) << 64
|
77
|
+
end
|
78
|
+
|
79
|
+
# Returns and caches a file download response
|
80
|
+
def download_url_response
|
81
|
+
@download_url_response ||= if live_session?
|
82
|
+
session.get_file_download_url_response(file.id)
|
83
|
+
else
|
84
|
+
{}
|
85
|
+
end
|
86
|
+
end
|
87
|
+
|
88
|
+
protected
|
89
|
+
|
90
|
+
# Returns true if live session/file properly set
|
91
|
+
def live_session?
|
92
|
+
!!(session && file)
|
93
|
+
end
|
94
|
+
|
95
|
+
# Returns an array of chunk sizes given total file +size+
|
96
|
+
#
|
97
|
+
# Chunk boundaries are located at the following positions:
|
98
|
+
# 0 / 128K / 384K / 768K / 1280K / 1920K / 2688K / 3584K / 4608K / ... (every 1024 KB) / EOF
|
99
|
+
def get_chunks(size)
|
100
|
+
chunks = []
|
101
|
+
p = pp = 0
|
102
|
+
i = 1
|
103
|
+
|
104
|
+
while i <= 8 and p < size - i * 0x20000 do
|
105
|
+
chunk_size = i * 0x20000
|
106
|
+
chunks << [p, chunk_size]
|
107
|
+
pp = p
|
108
|
+
p += chunk_size
|
109
|
+
i += 1
|
110
|
+
end
|
111
|
+
|
112
|
+
while p < size - 0x100000 do
|
113
|
+
chunk_size = 0x100000
|
114
|
+
chunks << [p, chunk_size]
|
115
|
+
pp = p
|
116
|
+
p += chunk_size
|
117
|
+
end
|
118
|
+
|
119
|
+
chunks << [p, size - p] if p < size
|
120
|
+
|
121
|
+
chunks
|
122
|
+
end
|
123
|
+
|
124
|
+
def calculate_chunk_mac(iv,chunk)
|
125
|
+
chunk_mac = [iv[0], iv[1], iv[0], iv[1]]
|
126
|
+
(1..chunk.length).take(16).each do |bit|
|
127
|
+
# NYI
|
128
|
+
end
|
129
|
+
chunk_mac
|
130
|
+
end
|
131
|
+
|
132
|
+
end
|
@@ -13,6 +13,9 @@ module Megar::CatalogItem
|
|
13
13
|
self.attributes = attributes
|
14
14
|
end
|
15
15
|
|
16
|
+
# A soft-reference to the owning session
|
17
|
+
attr_accessor :session
|
18
|
+
|
16
19
|
# The ID (Mega handle)
|
17
20
|
attr_accessor :id
|
18
21
|
|
@@ -23,6 +26,29 @@ module Megar::CatalogItem
|
|
23
26
|
# The literal mega node descriptor (as received from API)
|
24
27
|
attr_accessor :payload
|
25
28
|
|
29
|
+
# The parent folder id
|
30
|
+
attr_accessor :parent_folder_id
|
31
|
+
alias_method :p=, :parent_folder_id=
|
32
|
+
|
33
|
+
|
34
|
+
# The decrypted node key
|
35
|
+
attr_accessor :key
|
36
|
+
|
37
|
+
# The folder type id
|
38
|
+
# 0: File
|
39
|
+
# 1: Directory
|
40
|
+
# 2: Special node: Root (“Cloud Drive”)
|
41
|
+
# 3: Special node: Inbox
|
42
|
+
# 4: Special node: Trash Bin
|
43
|
+
attr_accessor :type
|
44
|
+
|
45
|
+
# Returns a handle to the enclosing folder (if any)
|
46
|
+
def parent_folder
|
47
|
+
if session && parent_folder_id
|
48
|
+
session.folders.find_by_id(parent_folder_id)
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
26
52
|
# Assigns the payload attribute, also splitting out separate attribute assignments from +value+ if a hash
|
27
53
|
def payload=(value)
|
28
54
|
self.attributes = value
|
@@ -39,26 +65,22 @@ module Megar::CatalogItem
|
|
39
65
|
end
|
40
66
|
end
|
41
67
|
|
42
|
-
# The decrypted node key
|
43
|
-
attr_accessor :key
|
44
|
-
|
45
|
-
# The folder type id
|
46
|
-
# 0: File
|
47
|
-
# 1: Directory
|
48
|
-
# 2: Special node: Root (“Cloud Drive”)
|
49
|
-
# 3: Special node: Inbox
|
50
|
-
# 4: Special node: Trash Bin
|
51
|
-
attr_accessor :type
|
52
|
-
|
53
|
-
|
54
68
|
# Generic interface to return the currently applicable collection
|
55
69
|
def collection
|
56
70
|
@collection ||= []
|
57
71
|
end
|
58
72
|
|
73
|
+
# Adds an item to the local cached collection given +attributes+ hash.
|
74
|
+
def add(attributes)
|
75
|
+
return false unless resource_class
|
76
|
+
collection << resource_class.new(attributes.merge(session: self.session))
|
77
|
+
end
|
78
|
+
|
59
79
|
# Returns the expected class of items in the collection
|
60
80
|
def resource_class
|
61
81
|
"#{self.class.name}".chomp('s').constantize
|
82
|
+
rescue
|
83
|
+
nil
|
62
84
|
end
|
63
85
|
|
64
86
|
# Command: clears/re-initialises the collection
|
@@ -82,9 +104,29 @@ module Megar::CatalogItem
|
|
82
104
|
end
|
83
105
|
alias_method :eql?, :==
|
84
106
|
|
107
|
+
# Returns the first record matching +id+
|
108
|
+
def find_by_id(id)
|
109
|
+
find { |r| r.id == id }
|
110
|
+
end
|
111
|
+
|
85
112
|
# Returns the first record matching +type+
|
86
113
|
def find_by_type(type)
|
87
114
|
find { |r| r.type == type }
|
88
115
|
end
|
89
116
|
|
117
|
+
# Returns all records matching +type+
|
118
|
+
def find_all_by_type(type)
|
119
|
+
find_all { |r| r.type == type }
|
120
|
+
end
|
121
|
+
|
122
|
+
# Returns the first record matching +name+
|
123
|
+
def find_by_name(name)
|
124
|
+
find { |r| r.name == name }
|
125
|
+
end
|
126
|
+
|
127
|
+
# Returns all records matching +parent_folder_id+
|
128
|
+
def find_all_by_parent_folder_id(parent_folder_id)
|
129
|
+
find_all { |r| r.parent_folder_id == parent_folder_id }
|
130
|
+
end
|
131
|
+
|
90
132
|
end
|