amq-protocol 0.0.1.pre → 0.5.0

data/.gitignore

@@ -1,4 +1,6 @@
 /*.gem
-.rvmrc
+/.rvmrc
 tmp
 *.pyc
+/vendor/bundle
+/coverage

data/.travis.yml

@@ -0,0 +1,7 @@
+script: "bundle exec rspec spec"
+rvm:
+  - 1.8.7
+  - 1.9.2
+  - ree
+  - rbx
+  - jruby

data/CONTRIBUTORS

@@ -0,0 +1,2 @@
+Jakub Šťastný aka Botanicus: 93 (5458 LOC)
+Michael S. Klishin: 26 (792 LOC)

data/Gemfile

@@ -0,0 +1,16 @@
+# encoding: utf-8
+
+source "http://gemcutter.org"
+
+group(:development) do
+  gem "nake",         :platform => :ruby_19
+  gem "contributors", :platform => :ruby_19
+
+  # excludes Windows, Rubinius and JRuby
+  gem "perftools.rb", :platform => :mri
+end
+
+group(:test) do
+  gem "rspec", ">=2.0.0"
+  gem 'simplecov', :platform => :ruby_19
+end

data/Gemfile.lock

@@ -0,0 +1,31 @@
+GEM
+  remote: http://gemcutter.org/
+  specs:
+    contributors (0.1.2)
+    diff-lcs (1.1.2)
+    nake (0.0.8)
+      term-ansicolor
+    perftools.rb (0.5.6)
+    rspec (2.5.0)
+      rspec-core (~> 2.5.0)
+      rspec-expectations (~> 2.5.0)
+      rspec-mocks (~> 2.5.0)
+    rspec-core (2.5.1)
+    rspec-expectations (2.5.0)
+      diff-lcs (~> 1.1.2)
+    rspec-mocks (2.5.0)
+    simplecov (0.4.2)
+      simplecov-html (~> 0.4.4)
+    simplecov-html (0.4.4)
+    term-ansicolor (1.0.5)
+
+PLATFORMS
+  java
+  ruby
+
+DEPENDENCIES
+  contributors
+  nake
+  perftools.rb
+  rspec (>= 2.0.0)
+  simplecov

data/LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2010 Jakub Šťastný aka Botanicus
+Copyright (c) 2010 – 2011 Jakub Šťastný aka Botanicus
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

data/PROFILING.md

@@ -0,0 +1,81 @@
+# Profiling
+
+To profile the encoding and decoding code use Aman Gupta's perftools.rb (https://github.com/tmm1/perftools.rb). It's really easy to use once you've set it up, and it generates graphs that make it pretty obvious where the  hotspots are.
+
+## Setup
+
+Start by installing perftools.rb:
+
+    gem install perftools.rb
+
+I have it installed in my global gemset, which is handy since you can profile projects that uses any other gemset without having to add perftools.rb to its Gemfile:
+
+    rvm gemset use global
+    gem install perftools.rb
+
+If you look at the README for perftools.rb the way to use it seems really icky, having to set environment variables and all that. I have the following script saved as `~/bin/profile` to encapsulate all that:
+
+    #!/bin/bash
+
+    profile_id=$(date +'%Y%m%d%H%M%S')
+    CPUPROFILE=/tmp/$profile_id RUBYOPT="-r`gem which perftools | tail -1`" $@
+    pprof.rb --pdf /tmp/$profile_id > /tmp/$profile_id.pdf
+    open /tmp/$profile_id.pdf
+
+With this script you can prefix any command with "profile" and perftools.rb will be enabed, and a nice graph will be opened in Preview as soon as the program exits. This only works on OS X, obviously, but I'm sure it can be made to work on Linux too.
+
+## How to profile the code
+
+Most of the time you want to profile a specific method or piece of code. The easiest thing to do is write a short script that runs that code over and over again. This will make sure that the profile graph isn't swamped by noise from other parts of the code, and running lots of iterations means that the numbers will be more stable. Perhaps most importantly it will also show the impact of the GC.
+
+    100000.times do
+      strs.each do |str|
+        AMQ::Client::Framing::String::Frame.decode(str)
+      end
+    end
+
+In the code above the `strs` variable holds a list of strings I've stolen from the amqp-benchmarks repository. I call the `decode` method of the `AMQ::Client::Framing::String::Frame` class over and over again and after 100K iterations (this takes around ten seconds to run) I get a graph that shows me which parts of the code take most time.
+
+This is how you run the script above (assuming you fill in the `strs` list and add the necessary require's -- and that you've followed the instructions at the top of this document -- and that you've named it "benchmark_script.rb"):
+
+    profile ruby benchmark_script.rb
+
+It will run for a few seconds and then open a graph. Next I'll explain how to interpret that graph.
+
+## Interpreting the profile graph
+
+The graph generated by perftools.rb shows all the methods that have run, which methods they call, how much time was spent in each method, and a lot of other very interesting things. 
+
+At the top left of the picture it says mow many samples were taken. perftools.rb samples 100 times per second, so 1257 samples means that it ran for 12.57 seconds. The number of samples per second can be tweaked, but I've never had any reason to. The next three numbers just tell you that there are methods that were called not very often, and that these are not shown in the graph, to make it easier to read.
+
+There are usually two, and sometimes three, nodes which don't have any arrows pointing into them. The first is the entry point into your application, the second the GC, and sometimes there is a third which usually is RubyGems or Bundler. There is no reason there can't be more top level nodes, but those are the ones I've seen.
+
+Each node represents a method. The label is the class and method name, in classic Ruby style: ClassName#instance_method_name. Unfortunately it doesn't handle class methods very well, so all end up ass Class#method_name. This is bad for us, since most methods in amq-protocol are class methods. It also means that all class methods of the same name, regardless of which class then belong are represented as one node. That's really bad for us, since a lot of methods that we want to profile are called "decode" and "encode".
+
+The numbers below the label are the self time and cumulative time measurements, first as samples and then, in parentheses as percentages of the whole. Self time means the time spent in the method, excluding other methods, and cumulative time means the total time spent in the method, and all methods below it in the call stack. In other words, if method a calls b and c, the cumulative time for a is its self time, plus the cumulative time of b and c (which is the self time of each of those, plus the cumulative time of the methods they call, and so on). 
+
+The number by the arrow from one node to another is the number of samples that saw the source node call the destination node. Looking at the numbers by the arrows into a node you can see roughly which method calls the method represented by that node the most.
+
+That's more or less it. I find the percentages most useful to work with, as the number of samples can vary a bit from time to time, its basically the same problem as using `time` to benchmark. Percentages are often much more stable.
+
+In the next section I'll explain how to act on the information that the graph gives.
+
+## Optimizing
+
+The first thing to look at is the GC node. It will always be high (this is Ruby after all), but if it's above 20% this is what you should start with. The reason the GC takes so much time is because the code creates too many objects. It's not always obvious what is creating so many objects, but read the next section for how to remove the worst offenders. When you've picked the low hanging fruit and still have a high GC percentage, go over your code and try to see if there are places where you can memoize, cache, etc. If your objects are immutable you should never have to create two identical objects (on the other hand, it's easy to leak memory if you cache too much).
+
+The next thing to look at is the node with the highest self time percentage. This is your first hotspot. Apply all the tricks in the next section and see if it makes any difference. Sometimes hotspots are hotspots because of design rather than slow code. If a method is called many, many times you may not be able to optimize it further (but try, each tiny thing will make a huge difference).
+
+The third thing to look for are methods will high cumulative time. If the methods they call don't look too bad and can be optimized first, you may be able to rewrite the method to call them fewer times by caching values, for example.
+
+While optimizing run the profiler for each change to make sure you're going in the right direction. It's not uncommon for something that you think should be faster to be slower because of something you couldn't foresee or some unexpected side effect. When the method you're working on is no longer the worst offender, move on to the next, and iterate.
+
+## Tips
+
+These are some general tips I've gathered while optimizing Ruby code:
+
+* Avoid string literals. In contrast to Java, strings in Ruby are mutable, so each literal string is a separate object (otherwise 'a'.upcase! would have very strange side effects). Move string literals to constants.
+* Avoid range literals. Just as with strings, ranges are objects and they are created every time they are encountered. Ranges are most often used to extract parts of a string, like `str[1..5]`. This can always be rewritten as a slice instead: `str[1, 4]` (the second argument is a length, not an index, so make sure you get the right value). Code like `data[offset..(offset + 4)]` is especially wasteful since it would be much clearer with the slice syntax: `data[offset, 5]`.
+* Avoid regular expressions. They are convenient, but for code that will be called thousands of times you need to rewrite them using String#index and String#slice/[]. Make sure that your rewrite doesn't create a lot of temporary string objects.
+* When reading files don't read line by line, and absolutely never byte by byte. It's much faster to read a big chunk (as in megabytes) and use String#split to split into lines. The same goes for writing. Buffer the lines and write them in one big chunk. Avoid IO#tell and #seek unless you're trying to read a few bytes out of a gigabyte file.
+* Never ever, ever, use Date or DateTime, or require 'date' (to get Time.parse, for example). Date is so slow that it's not even funny. If you use Date.parse you can be sure that whatever else you do that method will account for at least half of the running time. If you really, really need date parsing and date perform date calculations, use the home_run gem, but for almost everything you can use the basic methods on Time (but those aren't cheap either).

data/README.textile

@@ -1,32 +1,49 @@
-h1. About
+h1. About amq-protocol.
 
-This is an AMQP parser for AMQP 0.9.1. It isn't an AMQP client, just the parser, so if you want to write your own AMQP client without digging into the protocol, this gem can help you with that.
+amq-protocol is an AMQP 0.9.1 serialization library for Ruby. It is not an
+AMQP client: amq-protocol only handles serialization and deserialization.
+If you want to write your own AMQP client, this gem can help you with that.
 
-h1. Usage
+h2. How does amq-protocol relate to amqp gem, amq-client and libraries like bunny?
 
-<pre>
-gem install amqp-parser
-</pre>
+See "this page about AMQP gems family":https://github.com/ruby-amqp/amq-client/blob/master/README.textile
 
-Check the @examples@ directory for some examples.
 
-h1. Development
+h2. Installation
+
+If you use Bundler and want to use the very latest version, add this to your Gemfile:
 
 <pre>
-git clone git://github.com/botanicus/amq-protocol.git --recursive
+gem 'amq-protocol', :git => 'https://github.com/ruby-amqp/amq-protocol.git'
 </pre>
 
-Or, if you are on an older version of Git:
+
+h2. Development
+
+h3. Nightly Builds
+
+You can always find nightly builds at "gems.101ideas.cz":http://gems.101ideas.cz.
+
+You can install them thusly:
 
 <pre>
-git clone git://github.com/botanicus/amq-protocol.git
-git submodule update --init
+wget http://gems.101ideas.cz/amq-protocol-nightly.gem
+gem install amq-protocol-nightly.gem
 </pre>
 
-If you want to change some code, don't do so in @lib/amq/protocol.rb@, but in @protocol.rb.pytemplate@ which is a template file for @codegen.py@. You can re-generate the code by @./tasks.rb generate@.
+h3. Contributing
+
+If you want to change some code, don't edit @lib/amq/protocol/client.rb@ —
+that file is generated from @protocol.rb.pytemplate@
+which is a template file used by @codegen.py@. You can re-generate by running
+<pre>nake ./tasks.rb generate</pre>
+Please note that nake gem only runs on Ruby 1.9.2.
 
-h1. Links
+h2. Links
 
-* "RDoc.info API Docs":http://rdoc.info/github/botanicus/amq-protocol/master/frames
-* "Examples":https://github.com/botanicus/amq-protocol/tree/master/examples/
-* "Bug reporting":http://github.com/botanicus/amq-protocol/issues
+* "API Documentation":http://rdoc.info/github/ruby-amqp/amq-protocol/master/frames
+* "Examples":https://github.com/ruby-amqp/amq-protocol/tree/master/examples/
+* "Jabber room for contributors":xmpp://amqp-dev@conf.netlab.cz
+* "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp
+* "Issue tracker":http://github.com/ruby-amqp/amq-protocol/issues
+* "Continous integration server":http://travis-ci.org/#!/ruby-amqp/amq-protocol

data/TODO.todo

@@ -1,3 +1,27 @@
 - YARD documentation
 - benchmarks
 - optimisations (see puka)
+- Don't let users set the deprecated arguments.
+
+OpenOk
+if $DEBUG
+  if data[offset..(offset + 1)] != "\x00"
+    raise BadResponseError.new("known_hosts", "\x00", data)
+  end
+end
+
+
+DO NOT GENERATE THE FOLLOWING:
+attr_reader :known_hosts
+def initialize(known_hosts)
+  @known_hosts = known_hosts
+end
+
+arguments = Basic::Publish.method(:encode).parameters.map { |array| array.last }
+arguments.should include(:frame_size)
+
+arguments = Connection::Open.method(:encode).map { |array| array.last }
+arguments.should_not include(:capabilities)
+arguments.should_not include(:insist)
+
+generate the spec file(s)?

data/amq-protocol.gemspec

@@ -3,30 +3,35 @@
 
 require "base64"
 
+require File.expand_path("../lib/amq/protocol/version", __FILE__)
+
 Gem::Specification.new do |s|
   s.name = "amq-protocol"
-  s.version = "0.0.1"
-  s.authors = ["Jakub Stastny"]
-  s.homepage = "http://github.com/botanicus/amq-protocol"
+  s.version = AMQ::Protocol::VERSION
+  s.authors = ["Jakub Stastny", "Michael S. Klishin", "Theo Hultberg", "Mark Abramov"]
+  s.homepage = "http://github.com/ruby-amqp/amq-protocol"
   s.summary = "AMQP 0.9.1 encoder & decoder."
-  s.description = "This is an AMQP encoder & decoder for AMQP 0.9.1. It isn't an AMQP client, just the parser, so if you want to write your own AMQP client without digging into the protocol, this gem can help you with that."
+  s.description = <<-DESC
+  amq-protocol is an AMQP 0.9.1 serialization library for Ruby. It is not an
+  AMQP client: amq-protocol only handles serialization and deserialization.
+  If you want to write your own AMQP client, this gem can help you with that.
+  DESC
   s.cert_chain = nil
-  s.email = Base64.decode64("c3Rhc3RueUAxMDFpZGVhcy5jeg==\n")
+  s.email = ["bWljaGFlbEBub3ZlbWJlcmFpbi5jb20=\n", "c3Rhc3RueUAxMDFpZGVhcy5jeg==\n"].map { |i| Base64.decode64(i) }
   s.has_rdoc = true
 
   # files
-  s.files = `git ls-files`.split("\n")
+  s.files = `git ls-files`.split("\n").reject { |file| file =~ /^vendor\// }
   s.require_paths = ["lib"]
 
-  # Ruby version
-  s.required_ruby_version = ::Gem::Requirement.new("~> 1.9")
+  s.extra_rdoc_files = ["README.textile"] + Dir.glob("doc/*")
+
 
   begin
     require "changelog"
-  rescue LoadError
-    warn "You have to have changelog gem installed for post install message"
-  else
     s.post_install_message = CHANGELOG.new.version_changes
+  rescue LoadError
+    # warn "You have to have changelog gem installed for post install message"
   end
 
   # RubyForge

data/amqp_0.9.1_changes.json

@@ -1 +1 @@
-{"tx": {"select-ok": ["client"], "rollback": ["server"], "commit": ["server"], "rollback-ok": ["client"], "select": ["server"], "commit-ok": ["client"]}, "exchange": {"delete-ok": ["client"], "declare-ok": ["client"], "declare": ["server"], "delete": ["server"], "bind": ["server"], "bind-ok": ["client"], "unbind": ["server"], "unbind-ok": ["client"]}, "queue": {"unbind": ["server"], "unbind-ok": ["client"], "purge-ok": ["client"], "bind": ["server"], "purge": ["server"], "declare-ok": ["client"], "delete-ok": ["client"], "delete": ["server"], "declare": ["server"], "bind-ok": ["client"]}, "connection": {"secure": ["client"], "secure-ok": ["server"], "open-ok": ["client"], "close-ok": ["client", "server"], "start": ["client"], "tune": ["client"], "start-ok": ["server"], "close": ["client", "server"], "open": ["server"], "tune-ok": ["server"]}, "basic": {"qos": ["server"], "consume": ["server"], "reject": ["server"], "get": ["server"], "ack": ["server"], "get-ok": ["client"], "consume-ok": ["client"], "deliver": ["client"], "recover-ok": ["client"], "publish": ["server"], "cancel": ["server"], "recover-async": ["server"], "get-empty": ["client"], "qos-ok": ["client"], "return": ["client"], "recover": ["server"], "cancel-ok": ["client"]}, "channel": {"flow-ok": ["server", "client"], "flow": ["server", "client"], "open-ok": ["client"], "close-ok": ["client", "server"], "close": ["client", "server"], "open": ["server"]}}
+{"tx": {"select-ok": ["client"], "rollback": ["server"], "commit": ["server"], "rollback-ok": ["client"], "select": ["server"], "commit-ok": ["client"]}, "exchange": {"delete-ok": ["client"], "declare-ok": ["client"], "declare": ["server"], "delete": ["server"], "bind": ["server"], "bind-ok": ["client"], "unbind": ["server"], "unbind-ok": ["client"]}, "queue": {"unbind": ["server"], "unbind-ok": ["client"], "purge-ok": ["client"], "bind": ["server"], "purge": ["server"], "declare-ok": ["client"], "delete-ok": ["client"], "delete": ["server"], "declare": ["server"], "bind-ok": ["client"]}, "connection": {"secure": ["client"], "secure-ok": ["server"], "open-ok": ["client"], "close-ok": ["client", "server"], "start": ["client"], "tune": ["client"], "start-ok": ["server"], "close": ["client", "server"], "open": ["server"], "tune-ok": ["server"]}, "basic": {"qos": ["server"], "consume": ["server"], "reject": ["server"], "get": ["server"], "ack": ["client", "server"], "get-ok": ["client"], "consume-ok": ["client"], "deliver": ["client"], "recover-ok": ["client"], "publish": ["server"], "cancel": ["server"], "recover-async": ["server"], "get-empty": ["client"], "qos-ok": ["client"], "return": ["client"], "recover": ["server"], "cancel-ok": ["client"]}, "channel": {"flow-ok": ["server", "client"], "flow": ["server", "client"], "open-ok": ["client"], "close-ok": ["client", "server"], "close": ["client", "server"], "open": ["server"]}}

data/bin/jenkins.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+echo -e "\n\n==== Setup ===="
+source /etc/profile
+git fetch && git reset origin/master --hard
+
+echo -e "\n\n==== Ruby 1.9.2 Head ===="
+rvm use 1.9.2-head@ruby-amqp
+gem install bundler --no-ri --no-rdoc
+bundle install --path vendor/bundle/1.9.2 --without development; echo
+# bundle update --without development; echo
+bundle exec rspec spec
+return_status=$?
+
+echo -e "\n\n==== Ruby 1.8.7 ===="
+rvm use 1.8.7@ruby-amqp
+gem install bundler --no-ri --no-rdoc
+bundle install --path vendor/bundle/1.8.7 --without development; echo
+bundle update; echo
+bundle exec rspec spec
+return_status=$(expr $return_status + $?)
+
+test $return_status -eq 0

data/codegen.py

@@ -1,6 +1,9 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
+# Documentation for Mako templates:
+# http://www.makotemplates.org/docs/syntax.html
+
 import os, sys, re
 
 sys.path.append(os.path.join("vendor", "rabbitmq-codegen"))
@@ -14,10 +17,9 @@ except ImportError:
 
 # main class
 class AmqpSpecObject(AmqpSpec):
-    IGNORED_CLASSES = ["access", "tx"]
+    IGNORED_CLASSES = ["access"]
     IGNORED_FIELDS = {
         'ticket': 0,
-        'nowait': 0,
         'capabilities': '',
         'insist' : 0,
         'out_of_band': '',
@@ -30,10 +32,10 @@ class AmqpSpecObject(AmqpSpec):
         def extend_field(field):
             field.ruby_name = re.sub("[- ]", "_", field.name)
             field.type = self.resolveDomain(field.domain)
-            field.banned = bool(field.name in self.__class__.IGNORED_FIELDS)
+            field.ignored = bool(field.name in self.__class__.IGNORED_FIELDS) # I. e. deprecated
 
         for klass in self.classes:
-            klass.banned = bool(klass.name in self.__class__.IGNORED_CLASSES)
+            klass.ignored = bool(klass.name in self.__class__.IGNORED_CLASSES)
 
             for field in klass.fields:
                 extend_field(field)
@@ -42,7 +44,7 @@ class AmqpSpecObject(AmqpSpec):
                 for field in method.arguments:
                     extend_field(field)
 
-        self.classes = filter(lambda klass: not klass.banned, self.classes)
+        self.classes = filter(lambda klass: not klass.ignored, self.classes)
 
 # I know, I'm a bad, bad boy, but come on guys,
 # monkey-patching is just handy for this case.
@@ -73,16 +75,44 @@ def accepted_by(self, *receivers):
 
 AmqpMethod.accepted_by = accepted_by
 
+def convert_value_to_ruby(value):
+    values = {None: "nil", False: "false", True: "true", "": "EMPTY_STRING"}
+
+    try:
+        return values[value]
+    except:
+        return value.__repr__()
+
 def convert_to_ruby(field):
     name = re.sub("-", "_", field.name) # TODO: use ruby_name
-    if field.defaultvalue == None:
-        return "%s = nil" % (name,)
-    elif field.defaultvalue == False:
-        return "%s = false" % (name,)
-    elif field.defaultvalue == True:
-        return "%s = true" % (name,)
+    if name == "ticket":
+        return "%s = %s" % (name, field.defaultvalue) # we want to keep it as an int, not as a boolean
     else:
-        return "%s = %r" % (name, field.defaultvalue)
+        return "%s = %s" % (name, convert_value_to_ruby(field.defaultvalue))
+
+def not_ignored_args(self):
+    if self.hasContent:
+        return ["payload", "user_headers"] + map(lambda argument: argument.ruby_name, filter(lambda argument: not argument.ignored, self.arguments)) + ["frame_size"]
+    else:
+        return map(lambda argument: argument.ruby_name, filter(lambda argument: not argument.ignored, self.arguments))
+
+AmqpMethod.not_ignored_args = not_ignored_args
+
+def ignored_args(self):
+    return filter(lambda argument: argument.ignored, self.arguments)
+
+AmqpMethod.ignored_args = ignored_args
+
+# helpers
+def to_ruby_name(name):
+    return re.sub("[- ]", "_", name)
+
+def to_ruby_class_name(name):
+    parts = re.split("[- ]", name)
+    ruby_class_name = ""
+    for part in parts:
+        ruby_class_name = ruby_class_name + part[0].upper() + part[1:].lower()
+    return ruby_class_name
 
 def params(self):
     buffer = []
@@ -113,9 +143,13 @@ def render(path, **context):
     template = Template(file.read())
     return template.render(**context)
 
-def main(json_spec_path):
-    spec = AmqpSpecObject(json_spec_path)
-    print render("protocol.rb.pytemplate", spec = spec)
+def generateMain(type):
+    def main(json_spec_path):
+        spec = AmqpSpecObject(json_spec_path)
+        spec.type = type
+        print render("protocol.rb.pytemplate", spec = spec)
+
+    return main
 
 if __name__ == "__main__":
-    do_main_dict({"spec": main})
+    do_main_dict({"client": generateMain("client"), "server": generateMain("server")})

data/codegen_helpers.py

@@ -4,72 +4,124 @@ def genSingleEncode(spec, cValue, unresolved_domain):
     buffer = []
     type = spec.resolveDomain(unresolved_domain)
     if type == 'shortstr':
-        buffer.append("pieces << %s.bytesize.chr" % (cValue,))
-        buffer.append("pieces << %s" % (cValue,))
+        buffer.append("buffer << %s.bytesize.chr" % (cValue,))
+        buffer.append("buffer << %s" % (cValue,))
     elif type == 'longstr':
-        buffer.append("pieces << [%s.bytesize].pack('N')" % (cValue,))
-        buffer.append("pieces << %s" % (cValue,))
+        buffer.append("buffer << [%s.bytesize].pack(PACK_UINT32)" % (cValue,))
+        buffer.append("buffer << %s" % (cValue,))
     elif type == 'octet':
-        buffer.append("pieces << [%s].pack('B')" % (cValue,))
+        buffer.append("buffer << [%s].pack(PACK_CHAR)" % (cValue,))
     elif type == 'short':
-        buffer.append("pieces << [%s].pack('n')" % (cValue,))
+        buffer.append("buffer << [%s].pack(PACK_UINT16)" % (cValue,))
     elif type == 'long':
-        buffer.append("pieces << [%s].pack('N')" % (cValue,))
+        buffer.append("buffer << [%s].pack(PACK_UINT32)" % (cValue,))
     elif type == 'longlong':
-        buffer.append("pieces << [%s].pack('>Q')" % (cValue,))
+        buffer.append("buffer << AMQ::Hacks.pack_64_big_endian(%s)" % (cValue,))
     elif type == 'timestamp':
-        buffer.append("pieces << [%s].pack('>Q')" % (cValue,))
+        buffer.append("buffer << AMQ::Hacks.pack_64_big_endian(%s)" % (cValue,))
     elif type == 'bit':
         raise "Can't encode bit in genSingleEncode"
     elif type == 'table':
-        buffer.append("pieces << AMQ::Protocol::Table.encode(%s)" % (cValue,))
+        buffer.append("buffer << AMQ::Protocol::Table.encode(%s)" % (cValue,))
     else:
         raise "Illegal domain in genSingleEncode", type
 
     return buffer
 
-def genSingleDecode(spec, cLvalue, unresolved_domain):
+def genSingleDecode(spec, field):
+    cLvalue = field.ruby_name
+    unresolved_domain = field.domain
+
+    if cLvalue == "known_hosts":
+        import sys
+        print >> sys.stderr, field, field.ignored
+
     type = spec.resolveDomain(unresolved_domain)
     buffer = []
     if type == 'shortstr':
-        buffer.append("length = data[offset..(offset + 1)].unpack('N')[0]")
+        buffer.append("length = data[offset, 1].unpack(PACK_CHAR).first")
         buffer.append("offset += 1")
-        buffer.append("%s = data[offset..(offset + length)]" % (cLvalue,))
+        buffer.append("%s = data[offset, length]" % (cLvalue,))
         buffer.append("offset += length")
     elif type == 'longstr':
-        buffer.append("length = data[offset..(offset + 4)].unpack('N').first")
+        buffer.append("length = data[offset, 4].unpack(PACK_UINT32).first")
         buffer.append("offset += 4")
-        buffer.append("%s = data[offset..(offset + length)]" % (cLvalue,))
+        buffer.append("%s = data[offset, length]" % (cLvalue,))
         buffer.append("offset += length")
     elif type == 'octet':
-        buffer.append("%s = data[offset...(offset + 1)].unpack('c').first" % (cLvalue,))
+        buffer.append("%s = data[offset, 1].unpack(PACK_CHAR).first" % (cLvalue,))
         buffer.append("offset += 1")
     elif type == 'short':
-        buffer.append("%s = data[offset..(offset + 2)].unpack('n').first" % (cLvalue,))
+        buffer.append("%s = data[offset, 2].unpack(PACK_UINT16).first" % (cLvalue,))
         buffer.append("offset += 2")
     elif type == 'long':
-        buffer.append("%s = data[offset..(offset + 4)].unpack('N').first" % (cLvalue,))
+        buffer.append("%s = data[offset, 4].unpack(PACK_UINT32).first" % (cLvalue,))
         buffer.append("offset += 4")
     elif type == 'longlong':
-        buffer.append("%s = data[offset..(offset + 8)].unpack('N2').first" % (cLvalue,))
+        buffer.append("%s = AMQ::Hacks.unpack_64_big_endian(data[offset, 8]).first" % (cLvalue,))
         buffer.append("offset += 8")
     elif type == 'timestamp':
-        buffer.append("%s = data[offset..(offset + 8)].unpack('N2').first" % (cLvalue,))
+        buffer.append("%s = data[offset, 8].unpack(PACK_UINT32_X2).first" % (cLvalue,))
         buffer.append("offset += 8")
     elif type == 'bit':
         raise "Can't decode bit in genSingleDecode"
     elif type == 'table':
-        buffer.append("table_length = Table.length(data[offset..(offset + 4)])")
-        buffer.append("%s = Table.decode(data[offset..table_length])" % (cLvalue,))
+        buffer.append("table_length = Table.length(data[offset, 4])")
+        buffer.append("%s = Table.decode(data[offset, table_length + 4])" % (cLvalue,))
+        buffer.append("offset += table_length + 4")
+    else:
+        raise StandardError("Illegal domain '" + type + "' in genSingleDecode")
+
+    return buffer
+
+
+
+def genSingleSimpleDecode(spec, field):
+    cLvalue = field.ruby_name
+    unresolved_domain = field.domain
+
+    if cLvalue == "known_hosts":
+        import sys
+        print >> sys.stderr, field, field.ignored
+
+    type = spec.resolveDomain(unresolved_domain)
+    buffer = []
+    if type == 'shortstr':
+        # buffer.append("length = data.unpack(PACK_CHAR)[0]")
+        # buffer.append("result = data[offset..-1]")
+        # buffer.append("raise 'Bad size: #{length} expected, got #{result.bytesize}' if result.bytesize != length")
+        # buffer.append("result")
+        buffer.append("data")
+    elif type == 'longstr':
+        # buffer.append("length = data.unpack(PACK_CHAR)[0]")
+        # buffer.append("result = data[offset..-1]")
+        # buffer.append("raise 'Bad size: #{length} expected, got #{result.bytesize}' if result.bytesize != length")
+        # buffer.append("result")
+        buffer.append("data")
+    elif type == 'octet':
+        buffer.append("data.unpack(PACK_CHAR).first")
+    elif type == 'short':
+        buffer.append("data.unpack(PACK_UINT16).first")
+    elif type == 'long':
+        buffer.append("data.unpack(PACK_UINT32).first")
+    elif type == 'longlong':
+        buffer.append("AMQ::Hacks.unpack_64_big_endian(data).first")
+    elif type == 'timestamp':
+        buffer.append("Time.at(data.unpack(PACK_UINT32_X2).last)")
+    elif type == 'bit':
+        raise "Can't decode bit in genSingleDecode"
+    elif type == 'table':
+        buffer.append("Table.decode(data)")
     else:
-        raise "Illegal domain in genSingleDecode", type
+        raise StandardError("Illegal domain '" + type + "' in genSingleSimpleDecode")
 
     return buffer
 
+
 def genEncodeMethodDefinition(spec, m):
     def finishBits():
         if bit_index is not None:
-            return "pieces << [bit_buffer].pack('B')"
+            buffer.append("buffer << [bit_buffer].pack(PACK_CHAR)")
 
     bit_index = None
     buffer = []
@@ -103,11 +155,14 @@ def genDecodeMethodDefinition(spec, m):
             if bitindex >= 8:
                 bitindex = 0
             if bitindex == 0:
-                buffer.append("bit_buffer = data[offset..(offset + 1)].unpack('c').first")
+                buffer.append("bit_buffer = data[offset, 1].unpack(PACK_CHAR).first")
                 buffer.append("offset += 1")
                 buffer.append("%s = (bit_buffer & (1 << %d)) != 0" % (f.ruby_name, bitindex))
+                #### TODO: ADD bitindex TO THE buffer
+            else:
+                buffer.append("%s = (bit_buffer & (1 << %d)) != 0" % (f.ruby_name, bitindex))
             bitindex = bitindex + 1
         else:
             bitindex = None
-            buffer += genSingleDecode(spec, f.ruby_name, f.domain)
+            buffer += genSingleDecode(spec, f)
     return buffer