bit-struct 0.13.1

data/.gitignore

@@ -0,0 +1,2 @@
+*.bck
+pkg/*

data/History.txt

@@ -0,0 +1,102 @@
+bit-struct 0.12
+
+- Added vectors.
+
+- BitStruct#initialize can take an IO argument.
+
+bit-struct 0.11
+
+- Allow unaligned fields to cross up to two byte boundaries.
+  See examples/byte-bdy.rb.
+
+bit-struct 0.10
+
+- Fixed a bug when calling #to_yaml on a BitStruct with a pad field. Thanks
+  to Jay Reitz for reporting it and providing the solution.
+
+- Added BitStruct.default_options. Particularly useful for default endian 
+  settings--see examples/native.rb.
+
+- Fixed a bug that prevented warning about field name conflicts with
+  existing methods when the name is given as a symbol.
+
+- Fixed point fields may now have fractional divisors (like :fixed => 0.001).
+
+bit-struct 0.9
+
+- Added examples/field-ripper.rb.
+
+- Added BitStruct#field_by_name.
+
+- Added more warnings about nested field accessors returning a *copy* of the
+  nested data.
+
+- Added "pad" fields. See documentation in pad-field.rb and examples/pad.rb.
+
+- The #initial_value method now yields the value to a block, if given.
+
+- A BitStruct class is only closed when an instance is created. (After this
+  point no fields can be added.) Formerly, a class was closed when any method
+  requested the total length of the structure. This change makes it easier
+  to add groups of fields modularly, each with their own initial_value block.
+
+- Added examples/modular-def.rb to explain how to factor a BitStruct
+  definition into modules.
+
+bit-struct 0.8
+
+- Signed fields can now (like unsigned fields) be any multiple of 8 bits, and are accessed as fixnums or bignums, as needed.
+
+- It's easier to subclass BitStruct::OctetField. BitStruct::HexOctetField is now implemented in this way, just by defining three constants and two trivial class methods.
+
+bit-struct 0.7
+
+- BitStruct.describe now takes an option hash, and the :expand=>true option causes nested fields to be expanded in the description.
+
+- Unsigned integer fields can now be any multiple of 8 bytes: 8, 16, 24, 32, 40, ... (Fields of 1..15 bits in length are of course still supported.)
+
+- Added the :endian => :little option to signed integer, unsigned integer, and float fields. The full set of endian options is now [:little, :big, :network, :native]. The default is always :network.
+
+- Option names may be strings or symbols. Values that can be symbols can also be strings.
+
+- Added examples/bignum.rb.
+
+- Added support for the YAML in ruby 1.8.2 (the YAML in 1.8.4 was already supported).
+
+bit-struct 0.6
+
+- Added the :endian => :native option for numerical fields (signed, unsigned, float).
+
+- Fixed error message with 9..15 bit fields aligned on byte boundary.
+
+- The #initial_value is now inherited (before applying defaults).
+
+- New examples: raw.rb, native.rb.
+
+bit-struct 0.5
+
+- Integer fields may now cross byte boundaries, as long as the field fits within two whole bytes.
+
+bit-struct 0.4
+
+- Fixed a bug in reading text and char fields in YAML: if the value was interpreted by YAML as something other than a string, an error would result.
+
+- When BitStructs are loaded from yaml, the key is treated as a setter method, rather than a field. This is useful in case there is a field that needs special setters to accept humanly readable input. (For example, a char field with length-prefixed subfields.)
+
+bit-struct 0.3
+
+- BitStruct classes are now YAML friendly.
+
+- The default behavior of BitStruct#inspect and BitStruct#inspect_detailed is changed to print out the "rest" field, if any. This can be disabled by passing inspect an options hash with :include_rest => false. See BitStruct::DEFAULT_INSPECT_OPTS and BitStruct::DETAILED_INSPECT_OPTS. See examples/ip.rb and examples/rest.rb.
+
+- The default behavior of BitStruct#to_h is changed to include the "rest" field. As above, this can be disabled by passing to_h an options hash with :include_rest => false.
+
+- The default behavior of BitStruct#to_a is changed to include the "rest" field. As above, this can be disabled by passing false as an argument to to_a.
+
+bit-struct 0.2
+
+- first public release
+
+bit-struct 0.1
+
+- first release

data/README.txt

@@ -0,0 +1,189 @@
+= BitStruct
+
+Class for packed binary data stored in ruby Strings. BitStruct accessors, generated from user declared fields, use pack/unpack to treat substrings as fields with a specified portable format.
+
+Field types include:
+
+* signed and unsigned integer (1..16 bits, or 24, 32, 40, 48... bits)
+
+* numeric fields (signed, unsigned, float) can be designated as any of the following endians: little, big, native, network (default)
+
+* fixed point, with arbitrary scale factor
+
+* fixed length character array
+
+* null-terminated character array for printable text
+
+* octets (hex and decimal representation options; useful for IP and MAC addrs)
+
+* float
+
+* nested BitStruct
+
+* vectors of embedded BitStructs
+
+* free-form "rest" field (e.g., for the variable-size payload of a packet)
+
+Field options (specifiable as :foo => val or "foo" => val) include:
+
+* *display_name*: used in BitStruct#inspect_detailed and BitStruct#describe outputs.
+
+* *default*: default field value
+
+* *format*: alternate format string for inspect
+
+* *endian*: for byte ordering of numeric fields (unsigned, signed, float): little, big, native, network (default)
+
+* *fixed*: float stored as fixed-point integer, with specified scale factor
+
+
+== Installation
+
+For .gem:
+
+  gem install bit-struct
+
+For .tgz, unpack and then:
+
+  ruby install.rb config
+  ruby install.rb setup
+  ruby install.rb install
+
+== Uses
+
+BitStruct is useful for defining packets used in network protocols. This is especially useful for raw IP--see examples/ping-recv.rb. All multibyte numeric fields are stored by default in network order.
+
+BitStruct is most efficient when your data is primarily treated as a binary string, and only secondarily treated as a data structure. (For instance, you are routing packets from one socket to another, possibly looking at one or two fields as it passes through or munging some headers.) If accessor operations are a bottleneck, a better approach is to define a class that wraps an array and uses pack/unpack when the object needs to behave like a binary string.
+
+== Features
+
+* Extensible with user-defined field classes.
+
+* Fields are fully introspectable and can be defined programmatically.
+
+* BitStruct.describe prints out documentation of all the fields of a BitStruct subclass, based on declarations. This is useful for communicating with developers who are not using ruby, but need to talk the same protocols. See Example, below.
+
+* Fields are inherited by subclasses. (The free-form "rest" field does not inherit, because it usually represents a payload whose structure is defined in subclasses using the fixed-size fields.)
+
+* BitStruct#inspect and BitStruct#inspect_detailed can be used for prettified display of contents. (More generally, BitStruct#inspect takes some options that control formatting and detail level.) See Example, below.
+
+* BitStruct inherits from String, so all the usual methods are available, and string-sharing (copy-on-write) is in effect.
+
+* Easy access to a "prototype" instance of each BitStruct subclass, from which all instances of that subclass are initialized as a copy (in the absence of other initialization parameters, such as a hash, a string, or a block). See BitStruct.initial_value, and BitStruct#initialize. See Example, below.
+
+* Easy conversion to and from hashes, using BitStruct#to_h and BitStruct.new.
+
+* BitStructs can persist using Marshal (a BitStruct is after all just a string) or using YAML (with human readable representation of the fields).
+
+* Includes tests, examples, and rdoc API documentation.
+
+== Limitations
+
+* Fields that are not aligned on byte boundaries may cross no more than two bytes boundaries. (See examples/byte-bdy.rb.)
+
+* No variable length fields (except the #rest field).
+ 
+== Future plans
+
+* Currently, the library is written in pure ruby. The implementation uses Array#pack and String#unpack calls, as well as shifting and masking in pure ruby. Future versions will optionally generate a customized C extension for better efficiency.
+
+* A debug mode in which a class identifier is prepended to every BitStruct, so that protocol errors can be detected. (This feature has been implemented in an app that uses BitStruct, but needs to be refactored into the BitStruct library itself.)
+
+* Remove field size and alignment limitations.
+
+== Example
+
+An IP packet can be defined and used like this:
+
+  require 'bit-struct'
+
+  class IP < BitStruct
+    unsigned    :ip_v,     4,     "Version"
+    unsigned    :ip_hl,    4,     "Header length"
+    unsigned    :ip_tos,   8,     "TOS"
+    unsigned    :ip_len,  16,     "Length"
+    unsigned    :ip_id,   16,     "ID"
+    unsigned    :ip_off,  16,     "Frag offset"
+    unsigned    :ip_ttl,   8,     "TTL"
+    unsigned    :ip_p,     8,     "Protocol"
+    unsigned    :ip_sum,  16,     "Checksum"
+    octets      :ip_src,  32,     "Source addr"
+    octets      :ip_dst,  32,     "Dest addr"
+    rest        :body,            "Body of message"
+
+    note "     rest is application defined message body"
+
+    initial_value.ip_v    = 4
+    initial_value.ip_hl   = 5
+  end
+
+  ip = IP.new
+  ip.ip_tos = 0
+  ip.ip_len = 0
+  ip.ip_id  = 0
+  ip.ip_off = 0
+  ip.ip_ttl = 255
+  ip.ip_p   = 255
+  ip.ip_sum = 0
+  ip.ip_src = "192.168.1.4"
+  ip.ip_dst = "192.168.1.255"
+  ip.body   = "This is the payload text."
+  ip.ip_len = ip.length
+
+  puts ip.inspect
+  puts "-"*50
+  puts ip.inspect_detailed
+  puts "-"*50
+  puts IP.describe
+
+(Note that you can also construct an IP packet by passing a string to new, or by passing a hash of <tt>field,value</tt> pairs, or by providing a block that is yielded the new BitStruct.)
+
+The output of this fragment is:
+
+  #<IP ip_v=4, ip_hl=5, ip_tos=0, ip_len=45, ip_id=0, ip_off=0, ip_ttl=255, ip_p=255, ip_sum=0, ip_src="192.168.1.4", ip_dst="192.168.1.255", body="This is the payload text.">
+  --------------------------------------------------
+  IP:
+                         Version = 4
+                   Header length = 5
+                             TOS = 0
+                          Length = 45
+                              ID = 0
+                     Frag offset = 0
+                             TTL = 255
+                        Protocol = 255
+                        Checksum = 0
+                     Source addr = "192.168.1.4"
+                       Dest addr = "192.168.1.255"
+                 Body of message = "This is the payload text."
+  --------------------------------------------------
+
+  Description of IP Packet:
+      byte: type         name          [size] description
+  ----------------------------------------------------------------------
+        @0: unsigned     ip_v          [  4b] Version
+        @0: unsigned     ip_hl         [  4b] Header length
+        @1: unsigned     ip_tos        [  8b] TOS
+        @2: unsigned     ip_len        [ 16b] Length
+        @4: unsigned     ip_id         [ 16b] ID
+        @6: unsigned     ip_off        [ 16b] Frag offset
+        @8: unsigned     ip_ttl        [  8b] TTL
+        @9: unsigned     ip_p          [  8b] Protocol
+       @10: unsigned     ip_sum        [ 16b] Checksum
+       @12: octets       ip_src        [ 32b] Source addr
+       @16: octets       ip_dst        [ 32b] Dest addr
+       rest is application defined message body
+
+== Version
+
+bit-struct 0.12
+
+The current version of this software can be found at http://redshift.sourceforge.net/bit-struct.
+
+== License
+
+This software is distributed under the Ruby license. See http://www.ruby-lang.org.
+
+== Author
+
+Joel VanderWerf, mailto:vjoel@users.sourceforge.net
+Copyright (c) 2005-2009, Joel VanderWerf.

data/Rakefile

@@ -0,0 +1,34 @@
+# Look in the tasks/setup.rb file for the various options that can be
+# configured in this Rakefile. The .rake files in the tasks directory
+# are where the options are used.
+
+begin
+  require 'bones'
+  Bones.setup
+rescue LoadError
+  begin
+    load 'tasks/setup.rb'
+  rescue LoadError
+    raise RuntimeError, '### please install the "bones" gem ###'
+  end
+end
+
+ensure_in_path 'lib'
+require 'bit-struct/bit-struct'
+
+task :default => 'spec:run'
+
+PROJ.name = 'bit-struct'
+PROJ.authors = 'Joel VanderWerf'
+PROJ.email = 'vjoel@users.sourceforge.net'
+PROJ.url = 'http://rubyforge.org/projects/bit-struct/'
+PROJ.version = BitStruct::VERSION
+PROJ.rubyforge.name = 'bit-struct'
+PROJ.summary = "Library for packed binary data stored in ruby Strings"
+PROJ.description = <<END
+Library for packed binary data stored in ruby Strings. Useful for accessing fields in network packets and binary files.
+END
+
+PROJ.spec.opts << '--color'
+
+# EOF

data/TODO

@@ -0,0 +1,23 @@
+website:
+
+  scp doc/ vjoel@rubyforge.org:/var/www/gforge-projects/bit-struct/
+
+
+easy way to define wrappers
+
+
+to do:
+
+generate C code from bit-struct spec?
+
+variable-length embedded fields, with referenced length field
+
+terminated arrays?
+
+use with mmap; in general: bit struct that references a position in another string or string-like object (e.g. a Mmap).
+
+use require 'ipaddr'?
+
+See ideas in
+
+  http://www.notwork.org/~gotoken/ruby/p/as-is/pack.rb

data/TODO-ALSO

@@ -0,0 +1,71 @@
+rest :name, :terminator => ...
+
+> * Allow block also for "nest" ?
+> 
+> It may sound redundant in the first place, but when it's just for
+> providing more structure without requiring an extra class definition, it
+> could be useful:
+> 
+>   class Foo < BitStruct
+>     nest :coord do
+>       signed :x, 8
+>       signed :y, 8
+>     end
+>   end
+> 
+>   Foo.new.coord.x
+
+Agree completely. 
+
+> * Is explicit :length for vector necessary?
+> 
+>   class Tag < BitStruct
+>     signed  :flags, 32
+>     vector  :axis, Vec, :length => 3
+>   end
+> 
+> compared to
+> 
+>   class Tag < BitStruct
+>     signed  :flags, 32
+>     vector  :axis, Vec, 3
+>   end
+> 
+> Or am I missing any ambiguities because of the simple cases?
+
+Nothing ambiguous, I just get giddy when there are too many positional args. But since length is required, it does make sense to pass it as a positional arg. I'll change that...
+
+> * Allow length not only in bits, but also bytes?
+> 
+> In my cases I've never encountered a situation where I needed a member
+> of a structure to represent anything which is not on a byte boundary.
+
+Sigh, if only life we always like that. There's a reason I call the library _bit_-struct :)
+
+> All my signed and unsigned int have to read "32" (32bit, 4bytes) all
+> over the place. For strings I'm writing "64*8" for string with 64
+> characters.
+> 
+> How about a new default_option :granularity which is "1" by default.
+> When set to "8", every specified length (except for vector ...) is
+> multiplied by that factor before used?
+
+Hm, will think about that. I wish I could define Fixnum#bytes, but only in the context of field declarations in a BitStruct subclass.
+
+I don't like granularity because it's really only meaningful if the value is 1 or 8. Perhaps :granularity => :bit, or :granularity => :byte. But I still don't like an option that you have to keep in mind to understand whether
+
+  char :x, 80
+  
+is 8 or 80 chars. It's too much context sensitivity.
+
+What about
+
+  signed :x, bytes(4)
+
+Is that too clunky? It would be trivial to implement...
+
+> * Why "vector" and not just "array"?
+
+For consistency with things like NArray, which has Vector, Matrix, and Array classes, where vectors are essentially one-dim arrays.
+
+Also, "array" might lead to confusion with ruby arrays.

data/examples/ara-player-data.rb

@@ -0,0 +1,82 @@
+class PlayerData
+  class << self
+    def create(*a)
+      new(a.pack(FORMAT))
+    end
+  end
+
+  SPEC = [
+    %w( pid        i ),
+    %w( x_position f ),
+    %w( y_position f ),
+    %w( z_position f ),
+    %w( foobar     i ),
+  ]
+
+  ATTRIBUTES = SPEC.map{|s| s.first}
+
+  FORMAT = SPEC.map{|s| s.last}.join
+
+  SPEC.each_with_index do |spec, ix|
+    at, format = spec
+    eval <<-src
+      def #{ at }
+        @#{ at } ||= @data[#{ ix }]
+      end
+      def #{ at }= value
+        raise TypeError unless self.#{ at }.class == value.class
+        uncache
+        @#{ at } = @data[#{ ix }] = value
+      end
+    src
+  end
+
+  def update buffer
+    uncache
+    @data = buffer.unpack FORMAT
+  end
+  alias initialize update
+  def uncache
+    @to_s = @to_bin = nil
+  end
+  def to_s
+    @to_s ||= ATTRIBUTES.inject(''){|s,a| s << "#{ a } : #{ send a }, " }.chop.chop
+  end
+  def to_bin
+    @to_bin ||= @data.pack(FORMAT)
+  end
+end
+
+id, x_position, y_position, z_position, foobar =
+  400, 1.0, 2.0, 3.0, 0b101010
+
+pd = PlayerData::create id, x_position, y_position, z_position, foobar
+
+p pd
+p pd.pid
+p pd.x_position
+p pd.foobar
+puts pd
+
+begin
+  require 'timeout'
+  n = 0
+  sec = 10
+  Timeout::timeout(sec) do
+    loop do
+      PlayerData::create id, x_position, y_position, z_position, foobar
+      n += 1
+    end
+  end
+rescue Timeout::Error
+  puts "creations per second : #{ n / sec }"
+end
+
+__END__
+
+#<PlayerData:0xb7e29b7c @to_s=nil, @data=[400, 1.0, 2.0, 3.0, 42], @to_bin=nil>
+400
+1.0
+42
+pid : 400, x_position : 1.0, y_position : 2.0, z_position : 3.0, foobar : 42
+creations per second : 131746