arpie 0.0.4 → 0.0.5
Sign up to get free protection for your applications and to get access to all the features.
- data/BINARY_SPEC +218 -0
- data/README +46 -0
- data/Rakefile +2 -2
- data/lib/arpie/binary.rb +754 -0
- data/lib/arpie/error.rb +39 -0
- data/lib/arpie/protocol.rb +103 -106
- data/lib/arpie/xmlrpc.rb +4 -4
- data/lib/arpie.rb +2 -0
- data/spec/protocol_merge_and_split_spec.rb +28 -8
- data/spec/protocol_spec.rb +4 -0
- metadata +12 -8
data/BINARY_SPEC
ADDED
@@ -0,0 +1,218 @@
|
|
1
|
+
== What is a Binary?
|
2
|
+
|
3
|
+
A Binary is a class definition which can be used to pack/unpack part
|
4
|
+
of a data stream into/from a logical structure.
|
5
|
+
|
6
|
+
What a mouthful.
|
7
|
+
|
8
|
+
Let's try with an example:
|
9
|
+
|
10
|
+
class MyBinary < Arpie::Binary
|
11
|
+
field :status, :uint8
|
12
|
+
field :name, :string, :sizeof => :uint8
|
13
|
+
end
|
14
|
+
|
15
|
+
Looks simple enough, doesn't it?
|
16
|
+
|
17
|
+
Use it simply by invoking the .from class method of your newly-defined class:
|
18
|
+
|
19
|
+
irb(main):005:0> a = MyBinary.from("\x01\x05Arpie")
|
20
|
+
=> [#<MyBinary {:status=>1, :name=>"Arpie"}>, 7]
|
21
|
+
|
22
|
+
.from returns the the unpacked data structure (an instance of MyBinary), and
|
23
|
+
the number of bytes the data structure "ate".
|
24
|
+
|
25
|
+
This works both ways, of course:
|
26
|
+
|
27
|
+
irb(main):006:0> a[0].to
|
28
|
+
=> "\001\005Arpie"
|
29
|
+
|
30
|
+
|
31
|
+
== Usage within Arpie::Protocol
|
32
|
+
|
33
|
+
You can use Binary within a Arpie::ProtocolChain, but are by no means required to do so.
|
34
|
+
|
35
|
+
Binary raises EIncomplete when not enough data is available to construct a
|
36
|
+
Binary instance; so you can simply call it within a Protocol to parse a message,
|
37
|
+
and it will ask for more data transparently.
|
38
|
+
|
39
|
+
class MyProtocol < Arpie::Protocol
|
40
|
+
def from binary
|
41
|
+
bin, consumed = MyBinary.from(binary)
|
42
|
+
yield bin
|
43
|
+
return consumed
|
44
|
+
end
|
45
|
+
end
|
46
|
+
|
47
|
+
|
48
|
+
== Available data types
|
49
|
+
|
50
|
+
Now, this is a rather big list and subject to change. Luckily, Arpie includes a little
|
51
|
+
helper to show all registered data types. Just run this from a shell:
|
52
|
+
|
53
|
+
ruby -rrubygems -e 'require "arpie"; puts Arpie::Binary.describe_all_types'
|
54
|
+
|
55
|
+
and it will print a human-readable list of all data types defined.
|
56
|
+
|
57
|
+
See below for a partial list and some generic information on data types.
|
58
|
+
|
59
|
+
|
60
|
+
== opts, or parameters to types
|
61
|
+
|
62
|
+
Fields can and will have +opts+ - parameters to a field definition, which will define
|
63
|
+
how this particular field behaves. These options are mostly specific to a field type, except
|
64
|
+
where otherwise noted.
|
65
|
+
|
66
|
+
=== :optional => true
|
67
|
+
Mark this field as optional. This means that a binary string can be parsed
|
68
|
+
even if the given field is absent. If :default is given, that value will be
|
69
|
+
inserted instead of nil.
|
70
|
+
|
71
|
+
=== :default => value
|
72
|
+
Set a default value on +SomeBinary.new+, and if the field was flagged as :optional and
|
73
|
+
no data was available to populate it.
|
74
|
+
Note that the default value is expected to be in UNPACKED format, not packed.
|
75
|
+
|
76
|
+
=== :sizeof and :length
|
77
|
+
Most field types take :sizeof OR :length as an argument.
|
78
|
+
|
79
|
+
==== :length
|
80
|
+
Tell Binary to expect exactly :length items of the given type. Think of it as a fixed-size array.
|
81
|
+
|
82
|
+
=== :sizeof
|
83
|
+
This includes a prefixed "non-visible" field, which will be used to determine the
|
84
|
+
actual expected length of the data. Example:
|
85
|
+
|
86
|
+
field :blurbel, :bytes, :sizeof => :lint16
|
87
|
+
|
88
|
+
Will expect a network-order short (16 bits), followed by the amout of bytes the short resolves to.
|
89
|
+
|
90
|
+
If the field type given in :sizeof requires additional parameters, you can pass them with
|
91
|
+
:sizeof_opts (just like with :list - :of).
|
92
|
+
|
93
|
+
== :list
|
94
|
+
|
95
|
+
A :list is an array of arbitary, same-type elements. The element type is given in the :list-specific
|
96
|
+
:of parameter:
|
97
|
+
|
98
|
+
field :my_list, :list, :of => :lint16
|
99
|
+
|
100
|
+
This will complain of not being able to determine the size of the list - pass either a :sizeof,
|
101
|
+
or a :length parameter, described as above.
|
102
|
+
|
103
|
+
If your :of requires additional argument (a list of lists, for example), you can pass theses with :of_opts:
|
104
|
+
|
105
|
+
field :my_list_2, :list, :sizeof => :uint8, :of => :string,
|
106
|
+
:of_opts => { :sizeof, :nint16 }
|
107
|
+
|
108
|
+
== :bitfield
|
109
|
+
|
110
|
+
The bitfield type unpacks one or more bytes into their bit values, for individual addressing:
|
111
|
+
|
112
|
+
class TestClass < Arpie::Binary
|
113
|
+
field :flags, :msb_bitfield, :length => 8 do
|
114
|
+
field :bool_1, :bit
|
115
|
+
field :compound, :bit, :length => 7
|
116
|
+
# Take care not to leave any bits unmanaged - weird things happen otherwise.
|
117
|
+
end
|
118
|
+
end
|
119
|
+
|
120
|
+
irb(main):008:0> a, b = TestClass.from("\xff")
|
121
|
+
=> [#<TestClass {:flags=>#<Anon[:flags, :msb_bitfield, {:length=>8}] {:bool_1=>true, :compound=>[true, true, true, true, true, true, true]}>}>, 1]
|
122
|
+
irb(main):009:0> a.to
|
123
|
+
=> "\377"
|
124
|
+
irb(main):010:0> a.flags.bool_1 = false
|
125
|
+
=> false
|
126
|
+
irb(main):011:0> a.to
|
127
|
+
=> "\177"
|
128
|
+
|
129
|
+
This is pretty much all that you can do with it, for now.
|
130
|
+
|
131
|
+
== :fixed
|
132
|
+
|
133
|
+
The fixed type allows defining fixed strings that are always the same, both acting as a filler
|
134
|
+
and a safeguard (it will complain if it does not match):
|
135
|
+
|
136
|
+
field :blah, :fixed, :value => "FIXED"
|
137
|
+
|
138
|
+
== Nested Classes
|
139
|
+
|
140
|
+
Instead of pre-registered primitive data fiels you can pass in class names:
|
141
|
+
|
142
|
+
class Outer < Arpie::Binary
|
143
|
+
class Nested < Arpie::Binary
|
144
|
+
field :a, :uint8
|
145
|
+
field :b, :uint8
|
146
|
+
end
|
147
|
+
|
148
|
+
field :hah, :list, :of => Nested, :sizeof => :uint8
|
149
|
+
end
|
150
|
+
|
151
|
+
== Inline Anonymous Classes
|
152
|
+
|
153
|
+
Also, you can specify anonymous nested classes, which can be used to split data of the same type more fine-grainedly:
|
154
|
+
|
155
|
+
class TestClass < Arpie::Binary
|
156
|
+
field :outer, :bytes, :length => 16 do
|
157
|
+
field :key1, :bytes, :length => 8
|
158
|
+
field :key2, :bytes, :length => 8
|
159
|
+
end
|
160
|
+
end
|
161
|
+
|
162
|
+
This will create a anonymous class instance of Binary. :outer will be, just like in the Nested Classes example, passed
|
163
|
+
to the inner class for further parsing, and then be accessible in the resulting class instance:
|
164
|
+
|
165
|
+
irb(main):013:0> a, b = TestClass.from("12345678abcdefgh")
|
166
|
+
=> [#<TestClass {:outer=>#<Anon[:outer, :bytes, {:length=>16}] {:key2=>"abcdefgh", :key1=>"12345678"}>}>, 16]
|
167
|
+
irb(main):014:0> a.outer.key1
|
168
|
+
=> "12345678"
|
169
|
+
irb(main):015:0> a.outer.key2
|
170
|
+
=> "abcdefgh"
|
171
|
+
|
172
|
+
irb(main):016:0> a.outer.key2 = "test"
|
173
|
+
=> "test"
|
174
|
+
irb(main):017:0> a.to
|
175
|
+
=> "12345678test\000\000\000\000"
|
176
|
+
|
177
|
+
== virtuals
|
178
|
+
|
179
|
+
A virtual is a field definition that is not actually part of the binary data.
|
180
|
+
|
181
|
+
As you get to parse complex data structures, you might encounter the following case:
|
182
|
+
|
183
|
+
class TestClass < Arpie::Binary
|
184
|
+
field :len_a, :uint8
|
185
|
+
field :len_b, :uint8
|
186
|
+
|
187
|
+
field :middle, :something
|
188
|
+
|
189
|
+
field :matrix, :list, :of => :uint8, :sizeof => (value of :len_a * :len_b)
|
190
|
+
end
|
191
|
+
|
192
|
+
In this case, you will need to use a virtual attribute:
|
193
|
+
|
194
|
+
class TestClass < Arpie::Binary
|
195
|
+
field :len_a, :uint8
|
196
|
+
field :len_b, :uint8
|
197
|
+
|
198
|
+
field :middle, :something
|
199
|
+
|
200
|
+
virtual :v_len do |o| o.len_a * o.len_b end
|
201
|
+
field :hah, :list, :of => Nested, :sizeof => :v_len
|
202
|
+
|
203
|
+
pre_to do |o|
|
204
|
+
o.len_a = 4
|
205
|
+
o.len_b = 2
|
206
|
+
o
|
207
|
+
end
|
208
|
+
end
|
209
|
+
|
210
|
+
virtual attributes are one-way - obviously they cannot be used to write out data; there is no "#to".
|
211
|
+
|
212
|
+
That is what the pre_to is for - it recalculates len_a and len_b to your specifications.
|
213
|
+
|
214
|
+
== hooks
|
215
|
+
|
216
|
+
Binary provides several hooks that can be used to mangle data in the transformation process.
|
217
|
+
|
218
|
+
See Arpie::Binary, and look for pre_to, post_to, pre_from and post_from. An usage example is given above.
|
data/README
CHANGED
@@ -79,6 +79,52 @@ to get the newest version.
|
|
79
79
|
puts p.reverse "hi"
|
80
80
|
# => "ih"
|
81
81
|
|
82
|
+
== Writing custom Protocols
|
83
|
+
|
84
|
+
You can use arpies Protocol layer to write your custom protocol parser/emitters.
|
85
|
+
Consider the following, again very contrived, example. You have a linebased wire format,
|
86
|
+
which sends regular object updates in multiple lines, each holding a property to be updated.
|
87
|
+
What objects get updated is not relevant to this example.
|
88
|
+
|
89
|
+
For this example, we'll be using the SeparatorProtocol already contained in protocols.rb as
|
90
|
+
a base.
|
91
|
+
|
92
|
+
class AssembleExample < Arpie::Protocol
|
93
|
+
|
94
|
+
def from binary
|
95
|
+
# The wire format is simply a collection of lines
|
96
|
+
# where the first one is a number containing the
|
97
|
+
# # of lines to expect.
|
98
|
+
assemble! binary do |binaries, meta|
|
99
|
+
binaries.size >= 1 or incomplete!
|
100
|
+
binaries.size - 1 >= binaries[0].to_i or incomplete!
|
101
|
+
|
102
|
+
# Here, you can wrap all collected updates in
|
103
|
+
# whatever format you want it to be. We're just
|
104
|
+
# "joining" them to be a single array.
|
105
|
+
binaries.shift
|
106
|
+
binaries
|
107
|
+
end
|
108
|
+
end
|
109
|
+
|
110
|
+
def to object
|
111
|
+
yield object.size
|
112
|
+
object.each {|oo|
|
113
|
+
yield oo
|
114
|
+
}
|
115
|
+
end
|
116
|
+
end
|
117
|
+
|
118
|
+
p = Arpie::ProtocolChain.new(
|
119
|
+
AssembleExample.new,
|
120
|
+
Arpie::SeparatorProtocol.new
|
121
|
+
)
|
122
|
+
r, w = IO.pipe
|
123
|
+
|
124
|
+
p.write_message(w, %w{we want to be assembled})
|
125
|
+
|
126
|
+
p p.read_message(r)
|
127
|
+
# => ["we", "want", "to", "be", "assembled"]
|
82
128
|
|
83
129
|
== Replay protection
|
84
130
|
|
data/Rakefile
CHANGED
@@ -9,13 +9,13 @@ include FileUtils
|
|
9
9
|
# Configuration
|
10
10
|
##############################################################################
|
11
11
|
NAME = "arpie"
|
12
|
-
VERS = "0.0.
|
12
|
+
VERS = "0.0.5"
|
13
13
|
CLEAN.include ["**/.*.sw?", "pkg", ".config", "rdoc", "coverage"]
|
14
14
|
RDOC_OPTS = ["--quiet", "--line-numbers", "--inline-source", '--title', \
|
15
15
|
"#{NAME}: A high-performing layered networking protocol framework. Simple to use, simple to extend.", \
|
16
16
|
'--main', 'README']
|
17
17
|
|
18
|
-
DOCS = ["README", "COPYING"]
|
18
|
+
DOCS = ["README", "COPYING", "BINARY_SPEC"]
|
19
19
|
|
20
20
|
Rake::RDocTask.new do |rdoc|
|
21
21
|
rdoc.rdoc_dir = "rdoc"
|