bindata 0.11.0 → 0.11.1

data/ChangeLog

@@ -1,5 +1,11 @@
 = BinData Changelog
 
+== Version 0.11.1 (2009-08-29)
+
+* Allow wrapped types to work with struct's :onlyif parameter
+* Use Array#index instead of #find_index for compatibility with ruby 1.8.6
+  (patch courtesy of Joe Rozner).
+
 == Version 0.11.0 (2009-06-28)
 
 * Sanitizing code was refactored for speed.

data/TODO

@@ -1,5 +1,22 @@
+== Documentation
+
 * Write a detailed tutorial (probably as a web page).
 
 * Need more examples.
 
+== Pending refactorings
+
+* Refactor registry into registry and numeric registry
+    update specs accordingly
+
 * Perhaps refactor snapshot -> _snapshot et al
+
+== Bugs
+
+class A < BinData::Record
+  array :a, :type => :unknown
+end
+
+should give correct error message
+
+

data/TUTORIAL

@@ -0,0 +1,949 @@
+= BinData
+
+A declarative way to read and write structured binary data.
+
+== What is it for?
+
+Do you ever find yourself writing code like this?
+
+  io = File.open(...)
+  len = io.read(2).unpack("v")[0]
+  name = io.read(len)
+  width, height = io.read(8).unpack("VV")
+  puts "Rectangle #{name} is #{width} x #{height}"
+
+It's ugly, violates DRY and feels like you're writing Perl, not Ruby.
+
+There is a better way.
+
+  class Rectangle < BinData::Record
+    endian :little
+    uint16 :len
+    string :name, :read_length => :len
+    uint32 :width
+    uint32 :height
+  end
+
+  io = File.open(...)
+  r = Rectangle.read(io)
+  puts "Rectangle #{r.name} is #{r.width} x #{r.height}"
+
+BinData makes it easy to specify the structure of the data you are
+manipulating.
+
+Read on for the tutorial, or go straight to the
+download[http://rubyforge.org/frs/?group_id=3252] page.
+
+== License
+
+BinData is released under the same license as Ruby.
+
+Copyright (c) 2007 - 2009 Dion Mendel
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Overview
+
+BinData declarations are easy to read.  Here's an example.
+
+  class MyFancyFormat < BinData::Record
+    stringz :comment
+    uint8   :num_ints, :check_value => lambda { value.even? }
+    array   :some_ints, :type => :int32be, :initial_length => :num_ints
+  end
+
+This fancy format describes the following collection of data:
+
+1. A zero terminated string
+2. An unsigned 8bit integer which must by even
+3. A sequence of unsigned 32bit integers in big endian form, the total
+   number of which is determined by the value of the 8bit integer.
+
+The BinData declaration matches the english description closely.  Compare
+the above declaration with the equivalent #unpack code to read such a data
+record.
+
+  def read_fancy_format(io)
+    comment, num_ints, rest = io.read.unpack("Z*Ca*")
+    raise ArgumentError, "ints must be even" unless num_ints.even?
+    some_ints = rest.unpack("N#{num_ints}")
+    {:comment => comment, :num_ints => num_ints, :some_ints => *some_ints}
+  end
+
+The BinData declaration clearly shows the structure of the record.  The
+#unpack code makes this structure opaque.
+
+The general usage of BinData is to declare a structured collection of data
+as a user defined record.  This record can be instantiated, read, written
+and manipulated without the user having to be concerned with the underlying
+binary representation of the data.
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Primitive Types
+
+BinData provides support for the most commonly used primitive types that
+are used when working with binary data.  Namely:
+
+* fixed size strings
+* zero terminated strings
+* byte based integers - signed or unsigned, big or little endian and of
+  any size
+* bit based integers - unsigned big or little endian integers of any size
+* floating point numbers - single or double precision floats in either
+  big or little endian
+
+Primitives may be manipulated individually, but is more common to work
+with them as part of a record.
+
+Examples of individual usage:
+
+  int16 = BinData::Int16be.new
+  int16.value = 941
+  int16.to_binary_s #=> "\003\255"
+
+  fl = BinData::FloatBe.read("\100\055\370\124") #=> 2.71828174591064
+  fl.num_bytes #=> 4
+
+  fl * int16 #=> 2557.90320057996
+
+There are several parameters that are specific to primitives.
+
+:initial_value
+
+This contains the initial value that the primitive will contain after
+initialization.  This is useful for setting default values.
+
+  obj = BinData::String.new(:initial_value => "hello ")
+  obj + "world" #=> "hello world"
+
+  obj.assign("good-bye " )
+  obj + "world" #=> "good-bye world"
+
+:value
+
+The primitive will always contain this value.  Reading or assigning will
+not change the value.  This parameter is used to define constants or
+dependent fields.
+
+  pi = BinData::FloatLe.new(:value => Math::PI)
+  pi.assign(3)
+  puts pi #=> 3.14159265358979
+
+:check_value
+
+When reading, will raise a ValidityError if the value read does not match
+the value of this parameter.
+
+  obj = BinData::String.new(:check_value => lambda { /aaa/ =~ value })
+  obj.read("baaa!") #=> "baaa!"
+  obj.read("bbb") #=> raises ValidityError
+
+  obj = BinData::String.new(:check_value => "foo")
+  obj.read("foo") #=> "foo"
+  obj.read("bar") #=> raises ValidityError
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Common Operations
+
+There are operations common to all BinData types, including user defined
+ones.  There are summarised here.
+
+Reading and writing
+
+  ::read(io)
+
+  Creates a BinData object and reads its value from the given string or IO.
+  The newly created object is returned.
+
+    str = BinData::Stringz::read("string1\0string2")
+    str.snapshot #=> "string1"
+
+  #read(io)
+
+  Reads and assigns binary data read from io.
+
+    obj = BinData::Uint16be.new
+    obj.read("\022\064")
+    obj.value #=> 4660
+
+  #write(io)
+
+  Writes the binary representation of the object to io.
+
+    File.open("...", "wb") do |io|
+      obj = BinData::Uint64be.new
+      obj.value = 568290145640170
+      obj.write(io)
+    end
+
+  #to_binary_s
+
+  Returns the binary representation of this object as a string.
+
+    obj = BinData::Uint16be.new
+    obj.assign(4660)
+    obj.to_binary_s #=> "\022\064"
+
+Manipulating
+
+  #assign(value)
+
+  Assigns the given value to this object.  value can be of the same format
+  as produced by #snapshot, or it can be a compatible data object.
+  
+    arr = BinData::Array.new(:type => :uint8)
+    arr.assign([1, 2, 3, 4])
+    arr.snapshot #=> [1, 2, 3, 4]
+
+  #clear
+
+  Resets this object to its initial state.
+
+    obj = BinData::Int32be.new(:initial_value => 42)
+    obj.assign(50)
+    obj.clear
+    obj.value #=> 42
+
+  #clear?
+
+  Returns whether this object is in its initial state.
+
+    arr = BinData::Array.new(:type => :uint16be, :initial_length => 5)
+    arr[3] = 42
+    arr.clear? #=> false
+
+    arr[3].clear
+    arr.clear? #=> true
+
+Inspecting
+
+  #num_bytes
+
+  Returns the number of bytes required for the binary representation of
+  this object.
+
+    arr = BinData::Array.new(:type => :uint16be, :initial_length => 5)
+    arr[0].num_bytes #=> 2
+    arr.num_bytes #=> 10
+
+  #snapshot
+
+  Returns the value of this object as primitive Ruby objects (numerics,
+  strings, arrays and hashs).  This may be useful for serialization or
+  reducing memory usage.
+
+    obj = BinData::Uint8.new
+    obj.assign(3)
+    obj + 3 #=> 6
+
+    obj.snapshot #=> 3
+    obj.snapshot.class #=> Fixnum
+
+  #offset
+
+  Returns the offset of this object with respect to the parent structure
+  it is contained within.  This is most likely to be used with arrays and
+  records.
+
+    arr = BinData::Array.new(:type => :uint16le, :initial_length => 5)
+    arr[2].offset #=> 4
+
+  #inspect
+
+  Returns a human readable representation of this object.  This is a
+  shortcut to #snapshot.inspect.
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Records
+
+The general format of a BinData record declaration is a class containing
+one or more fields.
+
+  class MyName < BinData::Record
+    type field_name, :param1 => "foo", :param2 => bar, ...
+    ...
+  end
+
+*type* is the name of a supplied type (e.g. +uint32be+, +string+, +array+)
+or a user defined type.  For user defined types, the class name is
+converted from CamelCase to lowercase underscore_style.
+
+*field_name* is the name by which you can access the data.  Use either a
+String or a Symbol.
+
+Each field may have optional *parameters* for how to process the data.  The
+parameters are passed as a Hash with Symbols for keys.  Parameters are
+designed to be lazily evaluated, possibly multiple times.  This means that any
+parameter value must not have side effects.
+
+Here are some examples of legal values for parameters.
+
+  * :param => 5
+  * :param => lambda { 5 + 2 }
+  * :param => lambda { foo + 2 }
+  * :param => :foo
+
+The simplest case is when the value is a literal value, such as 5.
+
+If the value is not a literal, it is expected to be a lambda.  The lambda
+will be evaluated in the context of the parent, in this case the parent is
+an instance of +MyName+.
+
+If the value is a symbol, it is taken as syntactic sugar for a lambda
+containing the value of the symbol.
+e.g <tt>:param => :foo</tt> is <tt>:param => lambda { foo }</tt>
+
+=== Specifying default endian
+
+The endianess of numeric types must be explicitly defined so that the code
+produced is independent of architecture.  However, explicitly specifying
+the endian for each numeric field can result in a bloated declaration that
+can be difficult to read.
+
+  class A < BinData::Record
+    int16be  :a
+    int32be  :b
+    int16le  :c  # <-- Note little endian!
+    int32be  :d
+    float_be :e
+    array    :f, :type => :uint32be
+  end
+
+The endian keyword can be used to set the default endian.  This makes the
+declaration easier to read.  Any numeric field that doesn't use the default
+endian can explicitly override it.
+
+  class A < BinData::Record
+    endian :big
+
+    int16   :a
+    int32   :b
+    int16le :c   # <-- Note how this little endian now stands out
+    int32   :d
+    float   :e
+    array   :f, :type => :uint32
+  end
+
+The increase in clarity can be seen with the above example.  The endian
+keyword will cascade to nested types, as illustrated with the array in the
+above example.
+
+=== Optional fields
+
+A record may contain optional fields.  The optional state of a field is decided
+by the :onlyif parameter.  If the value of this parameter is false, then the
+field will be as if it didn't exist in the record.
+
+  class RecordWithOptionalField < BinData::Record
+    ...
+    uint8  :comment_flag
+    string :comment, :length => 20, :onlyif => :has_comment?
+
+    def has_comment?
+      comment_flag.nonzero?
+    end
+  end
+
+In the above example, the comment field is only included in the record if the
+value of the comment_flag field is non zero.
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Handling dependencies between fields
+
+A common occurance in binary file formats is one field depending upon the
+value of another.  e.g. A string preceded by it's length.
+
+As an example, let's assume a Pascal style string where the byte preceding
+the string contains the string's length.
+
+  # reading
+  io = File.open(...)
+  len = io.getc
+  str = io.read(len)
+  puts "string is " + str
+
+  # writing
+  io = File.open(...)
+  str = "this is a string"
+  io.putc(str.length)
+  io.write(str)
+
+Here's how we'd implement the same example with BinData.
+
+  class PascalString < BinData::Record
+    uint8  :len,  :value => lambda { data.length }
+    string :data, :read_length => :len
+  end
+
+  # reading
+  io = File.open(...)
+  ps = PascalString.new
+  ps.read(io)
+  puts "string is " + ps.data
+
+  # writing
+  io = File.open(...)
+  ps = PascalString.new
+  ps.data = "this is a string"
+  ps.write(io)
+
+This syntax needs explaining.  Let's simplify by examining reading and
+writing separately.
+
+  class PascalStringReader < BinData::Record
+    uint8  :len
+    string :data, :read_length => :len
+  end
+
+This states that when reading the string, the initial length of the string
+(and hence the number of bytes to read) is determined by the value of the
++len+ field.
+
+Note that <tt>:read_length => :len</tt> is syntactic sugar for
+<tt>:read_length => lambda { len }</tt>, as described in the Record section.
+
+  class PascalStringWriter < BinData::Record
+    uint8  :len, :value => lambda { data.length }
+    string :data
+  end
+
+This states that the value of +len+ is always equal to the length of +data+.
++len+ may not be manually modified.
+
+Combining these two definitions gives the definition for +PascalString+ as
+previously defined.
+
+It is important to note with dependencies, that a field can only depend on one
+before it.  You can't have a string which has the characters first and the
+length afterwards.
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Numerics
+
+There are three kinds of numeric types that are supported by BinData.
+
+=== Byte based integers
+
+These are the common integers that are used in most low level programming
+languages (C, C++, Java etc).  These integers can be signed or unsigned.
+The endian must be specified so that the conversion is independent of
+architecture.  The bit size of these integers must be a multiple of 8.
+Examples of byte based integers are:
+
+* uint16be  - unsigned 16 bit big endian integer
+* int8      - signed 8 bit integer
+* int32le   - signed 32 bit little endian integer
+* uint40be  - unsigned 40 bit big endian integer
+
+The be | le suffix may be omitted if the endian keyword is in use.
+
+=== Bit based integers
+
+These unsigned integers are used to define bitfields in records.  Bitfields
+are big endian by default but little endian may be specified explicitly.
+Little endian bitfields are rare, but do occur in older file formats
+(e.g.  The file allocation table for FAT12 filesystems is stored as an
+array of 12bit little endian integers).
+
+An array of bit based integers will be packed according to their endian.
+
+In a record, adjacent bitfields will be packed according to their endian.
+All other fields are byte aligned.
+
+Examples of bit based integers are:
+
+* bit1    - 1 bit big endian integer (may be used as boolean)
+* bit4_le - 4 bit little endian integer
+* bit32   - 32 bit big endian integer
+
+The difference between byte and bit base integers of the same number of
+bits (e.g. uint8 vs bit8) is one of alignment.
+
+This example is packed as 3 bytes
+
+  class A < BinData::Record
+    bit4  :a
+    uint8 :b
+    bit4  :c
+  end
+
+  Data is stored as: AAAA0000 BBBBBBBB CCCC0000
+
+Whereas this example is packed into only 2 bytes
+
+  class B < BinData::Record
+    bit4 :a
+    bit8 :b
+    bit4 :c
+  end
+
+  Data is stored as: AAAABBBB BBBBCCCC
+
+=== Floating point numbers
+
+BinData supports 32 and 64 bit floating point numbers, in both big and
+little endian format.  These types are:
+
+* float_le  - single precision 32 bit little endian float
+* float_be  - single precision 32 bit big endian float
+* double_le - double precision 64 bit little endian float
+* double_be - double precision 64 bit big endian float
+
+The _be | _le suffix may be omitted if the endian keyword is in use.
+
+== Example
+
+Here is an example declaration for an Internet Protocol network packet.
+Three of the fields have parameters.
+
+* The version field always has the value 4, as per the standard.
+* The options field is read as a raw string, but not processed.
+* The data field contains the payload of the packet.  Its length is
+  calculated as the total length of the packet minus the length of the
+  header.
+
+  class IP_PDU < BinData::Record
+    endian :big
+
+    bit4   :version, :value => 4
+    bit4   :header_length
+    uint8  :tos
+    uint16 :total_length
+    uint16 :ident
+    bit3   :flags
+    bit13  :frag_offset
+    uint8  :ttl
+    uint8  :protocol
+    uint16 :checksum
+    uint32 :src_addr
+    uint32 :dest_addr
+    string :options, :read_length => :options_length_in_bytes
+    string :data, :read_length => lambda { total_length - header_length_in_bytes }
+
+    def header_length_in_bytes
+      header_length * 4
+    end
+
+    def options_length_in_bytes
+      header_length_in_bytes - 20
+    end
+  end
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Strings
+
+BinData supports two types of strings - fixed size and zero terminated.
+Strings are treated as a sequence of 8bit bytes.  This is the same as
+strings in Ruby 1.8.  The issue of character encoding is ignored by
+BinData.
+
+=== Fixed Sized Strings
+
+Fixed sized strings may have a set length.  If an assigned value is shorter
+than this length, it will be padded to this length.  If no length is set,
+the length is taken to be the length of the assigned value.
+
+There are several parameters that are specific to fixed sized strings.
+
+:read_length
+
+  The length to use when reading a value.
+
+    obj = BinData::String.new(:read_length => 5)
+    obj.read("abcdefghij")
+    obj.value #=> "abcde"
+
+:length
+
+  The fixed length of the string.  If a shorter string is set, it will be
+  padded to this length.  Longer strings will be truncated.
+
+    obj = BinData::String.new(:length => 6)
+    obj.read("abcdefghij")
+    obj.value #=> "abcdef"
+
+    obj = BinData::String.new(:length => 6)
+    obj.value = "abcd"
+    obj.value #=> "abcd\000\000"
+
+    obj = BinData::String.new(:length => 6)
+    obj.value = "abcdefghij"
+    obj.value #=> "abcdef"
+
+:pad_char
+
+  The character to use when padding a string to a set length.  Valid values
+  are Integers and Strings of length 1.  "\0" is the default.
+
+    obj = BinData::String.new(:length => 6, :pad_char => 'A')
+    obj.value = "abcd"
+    obj.value #=> "abcdAA"
+    obj.to_binary_s #=> "abcdAA"
+
+:trim_padding
+
+  Boolean, default false.  If set, the value of this string will have all
+  pad_chars trimmed from the end of the string.  The value will not be
+  trimmed when writing.
+
+    obj = BinData::String.new(:length => 6, :trim_value => true)
+    obj.value = "abcd"
+    obj.value #=> "abcd"
+    obj.to_binary_s #=> "abcd\000\000"
+
+=== Zero Terminated Strings
+
+These strings are modelled on the C style of string - a sequence of
+bytes terminated by a null ("\0") character.
+
+  obj = BinData::Stringz.new
+  obj.read("abcd\000efgh")
+  obj.value #=> "abcd"
+  obj.num_bytes #=> 5
+  obj.to_binary_s #=> "abcd\000"
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Arrays
+
+A BinData array is a list of data objects of the same type.  It behaves
+much the same as the standard Ruby array, supporting most of the common
+methods.
+
+When instantiating an array, the type of object it contains must be
+specified.
+
+  arr = BinData::Array.new(:type => :uint8)
+  arr[3] = 5
+  arr.snapshot #=> [0, 0, 0, 5]
+
+Parameters can be passed to this object with a slightly clumsy syntax.
+
+  arr = BinData::Array.new(:type => [:uint8, {:initial_value => :index}])
+  arr[3] = 5
+  arr.snapshot #=> [0, 1, 2, 5]
+
+There are two different parameters that specify the length of the array.
+
+:initial_length
+
+  obj = BinData::Array.new(:type => :int8, :initial_length => 4)
+  obj.read("\002\003\004\005\006\007")
+  obj.snapshot #=> [2, 3, 4, 5]
+
+:read_until
+
+  While reading, elements are read until this condition is true.  This is
+  typically used to read an array until a sentinel value is found.  The
+  variables +index+, +element+ and +array+ are made available to any lambda
+  assigned to this parameter.  If the value of this parameter is the symbol
+  :eof, then the array will read as much data from the stream as possible.
+  
+    obj = BinData::Array.new(:type => :int8,
+                             :read_until => lambda { index == 1 })
+    obj.read("\002\003\004\005\006\007")
+    obj.snapshot #=> [2, 3]
+
+    obj = BinData::Array.new(:type => :int8,
+                             :read_until => lambda { element >= 3.5 })
+    obj.read("\002\003\004\005\006\007")
+    obj.snapshot #=> [2, 3, 4]
+
+    obj = BinData::Array.new(:type => :int8,
+            :read_until => lambda { array[index] + array[index - 1] == 9 })
+    obj.read("\002\003\004\005\006\007")
+    obj.snapshot #=> [2, 3, 4, 5]
+
+    obj = BinData::Array.new(:type => :int8, :read_until => :eof)
+    obj.read("\002\003\004\005\006\007")
+    obj.snapshot #=> [2, 3, 4, 5, 6, 7]
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Offset checking / adjustment
+
+TODO
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Choices
+
+A Choice is a collection of data objects of which only one is active at any
+particular time.  Method calls will be delegated to the active choice.
+The possible types of objects that a choice contains is controlled by the
+:choices parameter, while the :selection parameter specifies the active
+choice.
+
+:choices
+
+  Either an array or a hash specifying the possible data objects.  The
+  format of the array/hash.values is a list of symbols representing the
+  data object type.  If a choice is to have params passed to it, then it
+  should be provided as [type_symbol, hash_params].  An implementation
+  constraint is that the hash may not contain symbols as keys.
+
+:selection
+
+  An index/key into the :choices array/hash which specifies the currently
+  active choice.
+
+:copy_on_change
+
+  If set to true, copy the value of the previous selection to the current
+  selection whenever the selection changes.  Default is false.
+
+Examples
+
+  type1 = [:string, {:value => "Type1"}]
+  type2 = [:string, {:value => "Type2"}]
+  
+  choices = {5 => type1, 17 => type2}
+  obj = BinData::Choice.new(:choices => choices, :selection => 5)
+  obj.value # => "Type1"
+
+  choices = [ type1, type2 ]
+  obj = BinData::Choice.new(:choices => choices, :selection => 1)
+  obj.value # => "Type2"
+
+  choices = [ nil, nil, nil, type1, nil, type2 ]
+  obj = BinData::Choice.new(:choices => choices, :selection => 3)
+  obj.value # => "Type1"
+
+  class MyNumber < BinData::Record
+    int8 :is_big_endian
+    choice :data, :choices => { true => :int32be, false => :int32le },
+                  :selection => lambda { is_big_endian != 0 },
+                  :copy_on_change => true
+  end
+
+  obj = MyNumber.new
+  obj.is_big_endian = 1
+  obj.data = 5
+  obj.to_binary_s #=> "\001\000\000\000\005"
+
+  obj.is_big_endian = 0
+  obj.to_binary_s #=> "\000\005\000\000\000"
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Wrappers
+
+Sometimes you wish to create a new type that is simply an existing type
+with some predefined parameters.  Examples could be an array with a
+specified type, or an integer with an initial value.
+
+This can be achieved with a wrapper.  A wrapper creates a new type based on
+an existing type which has predefined parameters.  These parameters can of
+course be overridden at initialisation time.
+Here we define an array that contains big endian 16 bit integers.  The
+array has a preferred initial length.
+
+  class IntArray < BinData::Wrapper
+    endian :big
+    array :type => :uint16, :initial_length => 5
+  end
+
+  arr = IntArray.new
+  arr.size #=> 5
+
+The initial length can be overridden at initialisation time.
+
+  arr = IntArray.new(:initial_length => 8)
+  arr.size #=> 8
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Parameterizing User Defined Types
+
+All BinData types have parameters that allow the behaviour of an object to
+be specified at initialization time.  User defined types may also specify
+parameters.  There are two types of parameters - mandatory and default.
+
+=== Mandatory Parameters
+
+Mandatory parameters must be specified when creating an instance of the
+type.  The :type parameter of Array is an example of a mandatory type.
+
+  class IntArray < BinData::Wrapper
+    mandatory_parameter :half_count
+
+    array :type => :uint8, :initial_length => lambda { half_count * 2 }
+  end
+
+  arr = IntArray.new #=> raises ArgumentError: parameter 'half_count' must
+be specified in IntArray
+
+  arr = IntArray.new(:half_count => lambda { 1 + 2 })
+  arr.snapshot #=> [0, 0, 0, 0, 0, 0]
+
+=== Default Parameters
+
+Default parameters are optional.  These parameters have a default value
+that may be overridden when an instance of the type is created.
+
+  class Phrase < BinData::Primitive
+    default_parameter :number => "three"
+    default_parameter :adjective => "blind"
+    default_parameter :noun => "mice"
+
+    stringz :a, :initial_value => :number
+    stringz :b, :initial_value => :adjective
+    stringz :c, :initial_value => :noun
+
+    def get; "#{a} #{b} #{c}"; end
+    def set(v)
+      if /(.*) (.*) (.*)/ =~ v
+        self.a, self.b, self.c = $1, $2, $3
+      end
+    end
+  end
+
+  obj = Phrase.new(:number => "two", :adjective => "deaf")
+  obj.to_s #=> "two deaf mice"
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== User Defined Primitive Types
+
+Most user defined types will be Records, but occasionally we'd like to
+create a custom type of primitive.
+
+Let us revisit the Pascal String example.
+
+  class PascalString < BinData::Record
+    uint8  :len,  :value => lambda { data.length }
+    string :data, :read_length => :len
+  end
+
+We'd like to make PascalString a user defined type that behaves like a
+BinData::BasePrimitive object so we can use :initial_value etc.  Here's an
+example usage of what we'd like:
+
+  class Favourites < BinData::Record
+    pascal_string :language, :initial_value => "ruby"
+    pascal_string :os,       :initial_value => "unix"
+  end
+
+  f = Favourites.new
+  f.os = "freebsd"
+  f.to_binary_s #=> "\004ruby\007freebsd"
+
+We create this type of custom string by inheriting from BinData::Primitive
+(instead of BinData::Record) and implementing the #get and #set methods.
+
+  class PascalString < BinData::Primitive
+    uint8  :len,  :value => lambda { data.length }
+    string :data, :read_length => :len
+
+    def get;   self.data; end
+    def set(v) self.data = v; end
+  end
+
+=== Advanced User Defined Primitive Types
+
+Sometimes a user defined primitive type can not easily be declaratively
+defined.  In this case you should inherit from BinData::BasePrimitive and
+implement the following three methods:
+
+* value_to_binary_string(value)
+* read_and_return_value(io)
+* sensible_default()
+
+  # A custom big integer format.  Binary format is:
+  #   1 byte  : 0 for positive, non zero for negative
+  #   x bytes : Little endian stream of 7 bit bytes representing the
+  #             positive form of the integer.  The upper bit of each byte
+  #             is set when there are more bytes in the stream.
+  class BigInteger < BinData::BasePrimitive
+    def value_to_binary_string(value)
+      negative = (value < 0) ? 1 : 0
+      value = value.abs
+      bytes = [negative]
+      loop do
+        seven_bit_byte = value & 0x7f
+        value >>= 7
+        has_more = value.nonzero? ? 0x80 : 0
+        byte = has_more | seven_bit_byte
+        bytes.push(byte)
+
+        break if has_more.zero?
+      end
+
+      bytes.inject("") { |str, b| str << b.chr }
+    end
+
+    def read_and_return_value(io)
+      negative = read_uint8(io).nonzero?
+      value = 0
+      bit_shift = 0
+      loop do
+        byte = read_uint8(io)
+        has_more = byte & 0x80
+        seven_bit_byte = byte & 0x7f
+        value |= seven_bit_byte << bit_shift
+        bit_shift += 7
+
+        break if has_more.zero?
+      end
+
+      negative ? -value : value
+    end
+
+    def sensible_default
+      0
+    end
+
+    def read_uint8(io)
+      io.readbytes(1).unpack("C").at(0)
+    end
+  end
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Debugging
+
+TODO
+=== Tracing
+
+  class A < BinData::Record
+    int8  :a
+    bit4  :b
+    bit2  :c
+    array :d, :initial_length => 6, :type => :bit1
+  end
+
+  BinData::trace_reading do
+    A.read("\373\225\220")
+  end
+
+obj.a => -5
+obj.b => 9
+obj.c => 1
+obj.d[0] => 0
+obj.d[1] => 1
+obj.d[2] => 1
+obj.d[3] => 0
+obj.d[4] => 0
+obj.d[5] => 1
+
+=== Rest
+
+=== Hidden fields
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
+
+== Comparison
+
+TODO
+http://github.com/marcandre/packable/tree/master
+http://metafuzz.rubyforge.org/binstruct/
+http://rubyforge.org/projects/bitpack/
+http://binaryparse.rubyforge.org/
+http://redshift.sourceforge.net/bit-struct/
+
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-