nwn-lib 0.4.8 → 0.4.9

data/CHANGELOG

@@ -251,3 +251,21 @@ Bernhard Stoeckner <elven@swordcoast.net> (6):
 
       Another bugfix and compatibility release. This adds real NWN2 support
       to gff reading, adds a few new SETTINGS, and adds proper IO reading.
+
+=== 0.4.9
+Bernhard Stoeckner <elven@swordcoast.net> (10):
+      Gff::Reader/YAML: do not taint read objects
+      Scripting: satisfy win32 compatibility
+      nwn-irb: work on IO object instead of string
+      nwn-erf: fix typo in -h: -1 saying ERF 1.0 instead of ERF 1.1
+      Scripting: satisfy: close opened fds after loading GFF data
+      api.rb: merge into field.rb and struct.rb
+      Gff::Struct#by_path: print list index in current_path on error
+      Gff::Struct#by_path: / alias, $ and % mods, substruct & locname paths
+      TwoDA::Table: fix always printing errors for invalid IDs on table reading
+      Update documentation, add HOWTO
+      0.4.9-rel
+
+      Some medium rare bugfixes, and a overloaded operator for GFF structs,
+      which can be used to comfortable walk on trees.
+

data/COPYING

@@ -1,4 +1,4 @@
-Copyright (C) 2008 Bernhard Stoeckner <elven@swordcoast.net> and contributors
+Copyright (C) 2008-current Bernhard Stoeckner <elven@swordcoast.net> and contributors
 
 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by

data/HOWTO

@@ -0,0 +1,170 @@
+=== Writing scripts to do the work for you
+
+First, read SCRIPTING. Since an example is more worth than a thousand words,
+here is a quick introduction in the form of a script that handles input
+files from ARGV, does nothing with them, and then saves them back to the same
+file from which they came from.
+
+ #!/usr/bin/env nwn-dsl
+
+ ARGV.each_with_index {|file, index|
+   # The script will abort hard if file is not an ARE.
+   # This looks up the data_type, which are usually the
+   # first four bytes of a file.
+   gff = need file, :are
+
+   # This will save gff back to 'file'.
+   save gff
+
+   # This will prefix all lines printed with log with
+   # a percentage. It's usage is optional, and the sole
+   # reason for using each_with_index instead of each above.
+   progress index
+ }
+
+To run this example, type the following in a directory containing some .are
+files, with script.rb being the name you saved the above script in.
+
+ nwn-dsl path/to/script.rb *.are
+
+The shebang (#!/..) is for unixoid systems and Cygwin, and as such optional.
+
+For all available commands to nwn-dsl scripts, see NWN::Gff::Scripting.
+
+<b>All code snippets shown here assume that you put <tt>include NWN</tt>
+in your application. It is not needed for nwn-dsl, which imports that
+namespace for you.</b>
+
+=== Accessing GFF paths
+
+You can access GFF paths within structs by using the overloaded division
+operator.
+
+ #!/usr/bin/env nwn-dsl
+
+ ARGV.each_with_index {|file, index|
+   gff = need file, :are
+
+   log (gff / 'Name/0')
+
+   progress index
+ }
+
+For a full reference of the path syntax, see NWN::Gff::Struct#by_path.
+
+You cannot use paths to assign new values to structs - you can only
+modify existing values:
+
+ (gff / 'Name').v[0] = "New Localized Name with Language ID 0"
+
+Wrong, will raise error:
+
+ (gff / 'Name/0') = "New .."
+
+Please note that using the [] method will NOT evaluate paths, just
+access labels in the CURRENT struct (which is actually a hash).
+
+=== Creating new GFF Structs and Elements
+
+You can add new fields to existing structs via NWN::Gff::Struct#add_field:
+
+ gff.add_field 'LocalVersion', :int, 1
+
+ # This will print "1"
+ log (gff / 'LocalVersion$')
+
+You can also the dynamic methods to save some typing:
+
+ gff.add_int 'LocalVersion', 1
+
+You can just as well create whole GFF structures on the fly:
+
+ Gff::Struct.new do |s|
+   s.add_byte 'ImaByte', :int, 1
+   list = s.add_list 'ImaList', [] do |l|
+     l.add_struct(1) do |ss|
+       ss.add_byte 'ImaByteToo', 2
+     end
+   end
+ end
+
+Further reading: NWN::Gff::Struct#add_field, NWN::Gff::List#add_struct.
+
+=== Working with .2da files
+
+Working with TwoDA files is easy and painless:
+
+ #!/usr/bin/env nwn-dsl
+
+ data = IO.read('path/to/baseitems.2da')
+ table = TwoDA::Table.parse(data)
+
+ # This will print "twobladedsword" with NWN1 1.69
+ log table[12].Label
+
+ # You can re-format (and save) any valid table:
+ File.open("/tmp/out", "wb") {|f|
+   f.write(table.to_2da)
+   # OR
+   table.write_to(f)
+ }
+
+For more documentation, see NWN::TwoDA::Table and
+NWN::TwoDA::Row.
+
+You can also set up a default location for 2da files,
+after which you can simply use the following to read
+2da files:
+
+ table = TwoDA.get('baseitems')
+
+You can set up the TwoDA::Cache by either setting the
+environment variable (recommended, see SETTINGS), or
+like this (see NWN::TwoDA::Cache.setup):
+
+ TwoDA::Cache.setup("path_a:path_b/blah:path_c")
+
+=== Accessing .tlk data
+
+You can access individual .tlk files, or use a TlkSet,
+which will emulate the way NWN1/2 reads it's .tlk files.
+
+Read a simple .tlk file:
+
+ io = File.open("/path/to/dialog.tlk", "rb")
+ tlk = NWN::Tlk::Tlk.new(io)
+
+Note that Tlk::Tlk seeks and reads from +io+ as needed,
+so if you close the file handle, any further accesses
+will fail.
+
+ # Retrieve strref 12
+ tlk[12][:text]
+
+ # Retrieve the attached sound resref, if any:
+ tlk[12][:sound]
+
+ # prints the highest strref used
+ log tlk.highest_id
+
+ # Add a new strrref.
+ new_strref = tlk.add 'New text'
+
+ # And save the new TLK somewhere else.
+ File.open("/tmp/new.tlk", "wb") {|another_io|
+   tlk.write_to(another_io)
+ }
+
+Now read-only access with a TlkSet:
+
+ # The arguments are dialog.tlk, dialogf.tlk,
+ # custom.tlk and a customf.tlk each wrapped in
+ # a Tlk::Tlk as shown above.
+ set = Tlk::TlkSet.new(tlk, tlkf, custom, customf)
+
+ # Retrieve str_ref 12 as the female variant (dialogf.tlk)
+ # if present, :male otherwise.
+ log set[12, :female]
+
+You cannot use TlkSet to write out .tlk files or modify
+existing entries - it is merely a wrapper.

data/README

@@ -50,7 +50,7 @@ And do the following in a script of your own devising:
   require 'rubygems'
   require 'nwn/all'
 
-Also, Read BINARIES.
+Also, read BINARIES and HOWTO.
 
 For nwn-lib scripts, see SCRIPTING.
 

data/Rakefile

@@ -9,13 +9,13 @@ include FileUtils
 # Configuration
 ##############################################################################
 NAME = "nwn-lib"
-VERS = "0.4.8"
+VERS = "0.4.9"
 CLEAN.include ["**/.*.sw?", "pkg", ".config", "rdoc", "coverage"]
 RDOC_OPTS = ["--quiet", "--line-numbers", "--inline-source", '--title', \
   'nwn-lib: a ruby library for accessing NWN resource files', \
   '--main', 'README']
 
-DOCS = ["README", "BINARIES", "SCRIPTING", "SETTINGS", "CHEATSHEET", "CHANGELOG", "COPYING"]
+DOCS = ["README", "BINARIES", "HOWTO", "SCRIPTING", "SETTINGS", "CHEATSHEET", "CHANGELOG", "COPYING"]
 
 Rake::RDocTask.new do |rdoc|
   rdoc.rdoc_dir = "rdoc"

data/bin/nwn-erf

@@ -66,7 +66,7 @@ begin OptionParser.new do |o|
   o.on "-0", "Create (only -c) V1.0 ERF, 16 byte resrefs. (NWN1, default)" do
     $version = "V1.0"
   end
-  o.on "-1", "Create (only -c) V1.0 ERF, 32 byte resrefs. (NWN2)." do
+  o.on "-1", "Create (only -c) V1.1 ERF, 32 byte resrefs. (NWN2)." do
     $version = "V1.1"
   end
 

data/bin/nwn-irb

@@ -29,7 +29,7 @@ def read file
   file = File.expand_path(file || $file)
   $stderr.puts "Reading `#{file}' .."
   fmt = NWN::Gff.guess_file_format(file)
-  $gff = NWN::Gff.read(IO.read(file), fmt)
+  $gff = NWN::Gff.read(File.open(file, "rb"), fmt)
 
   $stderr.puts "Your GFF file is in `$gff' (data_type: #{$gff.data_type.inspect})."
   $stderr.puts "Type `save' to save to the filename it came from (make a backup!), `exit' (or Ctrl+D) to exit (without saving)."

data/lib/nwn/gff.rb

@@ -116,4 +116,3 @@ require 'nwn/gff/list'
 require 'nwn/gff/cexolocstr'
 require 'nwn/gff/reader'
 require 'nwn/gff/writer'
-require 'nwn/gff/api'

data/lib/nwn/gff/field.rb

@@ -158,4 +158,36 @@ module NWN::Gff::Field
         false
     end
   end
+
+#:stopdoc:
+  # Used by NWN::Gff::Struct#by_flat_path
+  def each_by_flat_path &block
+    case field_type
+      when :cexolocstr
+        yield("", self)
+        field_value.sort.each {|lid, str|
+          yield("/" + lid.to_s, str)
+        }
+
+      when :struct
+        yield("", self)
+        field_value.each_by_flat_path {|v, x|
+          yield(v, x)
+        }
+
+      when :list
+        yield("", self)
+        field_value.each_with_index {|item, index|
+          yield("[" + index.to_s + "]", item)
+          item.each_by_flat_path("/") {|v, x|
+            yield("[" + index.to_s + "]" + v, x)
+          }
+        }
+
+      else
+        yield("", self)
+    end
+  end
+#:startdoc:
+
 end

data/lib/nwn/gff/reader.rb

@@ -67,7 +67,7 @@ class NWN::Gff::Reader
 
   # This iterates through a struct and reads all fields into a hash, which it returns.
   def read_struct index, file_type = nil, file_version = nil
-    struct = {}.taint
+    struct = {}
     struct.extend(NWN::Gff::Struct)
 
     type = @structs[index * 3]
@@ -101,9 +101,7 @@ class NWN::Gff::Reader
 
   # Reads the field at +index+ and returns [label_name, Gff::Field]
   def read_field index, parent_of
-    gff = {}
-
-    field = {}.taint
+    field = {}
     field.extend(NWN::Gff::Field)
 
     index *= 3
@@ -184,7 +182,7 @@ class NWN::Gff::Reader
         len = total_size + 4
         # Filter out empty strings.
         exostr.reject! {|k,v| v.nil? || v.empty?}
-        exostr.taint
+        exostr
 
       when :void
         len = @field_data[data_or_offset, 4].unpack("V")[0]
@@ -215,7 +213,7 @@ class NWN::Gff::Reader
           list << read_struct(@list_indices[i], field.path, field.parent.data_version)
         end
 
-        list.taint
+        list
 
     end
 
@@ -226,7 +224,7 @@ class NWN::Gff::Reader
     [value].compact.flatten.each {|iv|
       iv.element = field if iv.respond_to?('element=')
     }
-    field['value'] = value.taint
+    field['value'] = value
 
     # We extend all fields and field_values with matching classes.
     field.extend_meta_classes

data/lib/nwn/gff/struct.rb

@@ -104,4 +104,100 @@ module NWN::Gff::Struct
   def to_s
     "<NWN::Gff::Struct #{self.data_type}/#{self.data_version}, #{self.keys.size} fields>"
   end
+
+
+  # Iterates this struct, yielding flat, absolute
+  # paths and the Gff::Field for each element found.
+
+  # Example:
+  # "/AddCost" => {"type"=>:dword, ..}
+  def each_by_flat_path prefix = "/", &block
+    sort.each {|label, field|
+      field.each_by_flat_path do |ll, lv|
+        yield(prefix + label + ll, lv)
+      end
+    }
+  end
+
+  # Retrieve an object from within the given tree.
+  # Path is a slash-separated destination, given as
+  # a string
+  #
+  # Prefixed/postfixed slashes are optional.
+  #
+  # You can retrieve CExoLocString values by giving the
+  # language ID as the last label:
+  #  /FirstName/0
+  #
+  # You can retrieve list values by specifying the index
+  # in square brackets:
+  #  /SkillList[0]
+  #  /SkillList[0]/Rank   => {"Rank"=>{"label"=>"Rank", "value"=>0, "type"=>:byte}}
+  #
+  # You can directly retrieve field values and types
+  # instead of the field itself:
+  #  /SkillList[0]/Rank$  => 0
+  #  /SkillList[0]/Rank?   => :byte
+  #
+  # This will raise an error for non-field paths, naturally:
+  #  SkillList[0]$        => undefined method `field_value' for {"Rank"=>{"label"=>"Rank", "value"=>0, "type"=>:byte}}:Hash
+  #  SkillList[0]?        => undefined method `field_type' for {"Rank"=>{"label"=>"Rank", "value"=>0, "type"=>:byte}}:Hash
+  #
+  # For CExoLocStrings, you can retrieve the str_ref:
+  #  FirstName%           => 4294967295
+  # This will return DEFAULT_STR_REF (0xffffffff) if the given path does not have
+  # a str_ref.
+  def by_path path
+    struct = self
+    current_path = ""
+    path = path.split('/').map {|v| v.strip }.reject {|v| v.empty?}.join('/')
+
+    path, mod = $1, $2 if path =~ /^(.+?)([\$\?%])?$/
+
+    path.split('/').each_with_index {|v, path_index|
+      if struct.is_a?(NWN::Gff::Field) && struct.field_type == :cexolocstr &&
+          v =~ /^\d+$/ && path_index == path.split('/').size - 1
+        struct = struct.field_value[v.to_i]
+        break
+      end
+
+      v, index = $1, $2 if v =~ /^(.+?)\[(\d+)\]$/
+
+      struct = struct.v if struct.is_a?(NWN::Gff::Field) &&
+        struct.field_type == :struct
+
+      struct = struct[v]
+      if index
+        struct.field_type == :list or raise NWN::Gff::GffPathInvalidError,
+          "Specified a list offset for a non-list item: #{v}[#{index}]."
+
+        struct = struct.field_value[index.to_i]
+      end
+
+
+      raise NWN::Gff::GffPathInvalidError,
+        "Cannot find a path to /#{path} (at: #{current_path})." unless struct
+
+      current_path += "/" + v
+      current_path += "[#{index}]" if index
+    }
+
+    case mod
+      when "$"
+        struct.field_value
+      when "?"
+        struct.field_type
+      when "%"
+        struct.has_str_ref? ? struct.str_ref :
+          NWN::Gff::Cexolocstr::DEFAULT_STR_REF
+      else
+        struct
+    end
+  end
+
+  # An alias for +by_path+.
+  def / path
+    by_path(path)
+  end
+
 end

data/lib/nwn/scripting.rb

@@ -3,7 +3,7 @@
 # Include this if you want to eval nwn-gff-dsl scripts.
 module NWN::Gff::Scripting
 
-  class Sandbox
+  class Sandbox #:nodoc:
     include NWN
     include NWN::Gff::Scripting
   end
@@ -99,11 +99,13 @@ module NWN::Gff::Scripting
   # If only a filename/string is given and no further arguments,
   # the read object will be returned as-is.
   def satisfy *what
+    close_me = false
     if $standalone
       fn = what.shift
       io = case fn
         when String
-          File.new(fn, "r")
+          close_me = true
+          File.new(fn, "rb")
         when IO
           fn
         else
@@ -112,7 +114,12 @@ module NWN::Gff::Scripting
           #  "`need', `want' and `satisfy' need a filename or a IO " +
           #  "object to read from (usually the first script argument)."
       end
-      obj = NWN::Gff.read(io, NWN::Gff.guess_file_format(fn))
+
+      obj = begin
+        NWN::Gff.read(io, NWN::Gff.guess_file_format(fn))
+      ensure
+        io.close if close_me
+      end
       log "satisfied #{fn} -> #{obj.to_s}"
       $satisfy_loaded[obj.object_id] = [fn, obj.hash]
 

data/lib/nwn/twoda.rb

@@ -132,7 +132,7 @@ module NWN
         data.each_with_index {|row, idx|
           id = row.shift
 
-          NWN.log_debug "Warning: invalid ID in line #{idx}: #{id.inspect}" if $id !~ /^\d+$/
+          NWN.log_debug "Warning: invalid ID in line #{idx}: #{id.inspect}" if id !~ /^\d+$/
 
           id = id.to_i + id_offset
 
@@ -310,7 +310,7 @@ module NWN
 
       # Set the file system path spec where all 2da files reside.
       # Call this on application startup.
-      # path spec is a colon-separated list of pathes, just like $PATH.
+      # path spec is a colon-separated list of paths, just like $PATH.
       def self.setup root_directories
         @_roots = root_directories.split(':').compact.reject {|x| "" == x.strip }
       end

data/lib/nwn/yaml.rb

@@ -74,7 +74,7 @@ end
 
 # This parses the struct and extends all fields with their proper type.
 YAML.add_domain_type(NWN::YAML_DOMAIN,'struct') {|t,hash|
-  struct = {}.taint
+  struct = {}
   struct.extend(NWN::Gff::Struct)
 
   # The metadata
@@ -96,7 +96,7 @@ YAML.add_domain_type(NWN::YAML_DOMAIN,'struct') {|t,hash|
     element.extend_meta_classes
     element.validate
 
-    struct[label] = element.taint
+    struct[label] = element
   }
 
   struct

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: nwn-lib
 version: !ruby/object:Gem::Version 
-  version: 0.4.8
+  version: 0.4.9
 platform: ruby
 authors: 
 - Bernhard Stoeckner
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-06-30 00:00:00 +02:00
+date: 2009-07-19 00:00:00 +02:00
 default_executable: 
 dependencies: []
 
@@ -25,6 +25,7 @@ extensions: []
 extra_rdoc_files: 
 - README
 - BINARIES
+- HOWTO
 - SCRIPTING
 - SETTINGS
 - CHEATSHEET
@@ -35,51 +36,51 @@ files:
 - CHANGELOG
 - README
 - Rakefile
-- bin/nwn-gff
 - bin/nwn-dsl
-- bin/nwn-erf
+- bin/nwn-gff
 - bin/nwn-irb
-- spec/bin_dsl_spec.rb
+- bin/nwn-erf
+- spec/wellformed_gff.bic
 - spec/gff_spec.rb
-- spec/bin_gff_spec.rb
-- spec/bin_erf_spec.rb
+- spec/res_spec.rb
+- spec/spec.opts
+- spec/erf_spec.rb
 - spec/field_spec.rb
 - spec/cexolocstr_spec.rb
-- spec/erf_spec.rb
 - spec/struct_spec.rb
-- spec/spec.opts
-- spec/res_spec.rb
+- spec/spec_helper.rb
 - spec/tlk_spec.rb
+- spec/bin_dsl_spec.rb
+- spec/bin_gff_spec.rb
 - spec/rcov.opts
+- spec/bin_erf_spec.rb
 - spec/twoda_spec.rb
-- spec/spec_helper.rb
-- spec/wellformed_gff.bic
-- lib/nwn/erf.rb
-- lib/nwn/io.rb
-- lib/nwn/settings.rb
+- lib/nwn/scripting.rb
 - lib/nwn/yaml.rb
-- lib/nwn/all.rb
-- lib/nwn/res.rb
 - lib/nwn/twoda.rb
-- lib/nwn/tlk.rb
+- lib/nwn/erf.rb
+- lib/nwn/settings.rb
 - lib/nwn/gff.rb
+- lib/nwn/tlk.rb
+- lib/nwn/io.rb
+- lib/nwn/res.rb
 - lib/nwn/kivinen.rb
-- lib/nwn/scripting.rb
-- lib/nwn/gff/api.rb
-- lib/nwn/gff/writer.rb
-- lib/nwn/gff/struct.rb
-- lib/nwn/gff/list.rb
+- lib/nwn/all.rb
+- lib/nwn/gff/field.rb
 - lib/nwn/gff/reader.rb
 - lib/nwn/gff/cexolocstr.rb
-- lib/nwn/gff/field.rb
+- lib/nwn/gff/struct.rb
+- lib/nwn/gff/list.rb
+- lib/nwn/gff/writer.rb
 - tools/verify.sh
 - tools/migrate_03x_to_04x.sh
 - scripts/truncate_floats.rb
+- scripts/reformat_2da
 - scripts/clean_locstrs.rb
-- scripts/debug_check_objid.rb
 - scripts/extract_all_items.rb
-- scripts/reformat_2da
+- scripts/debug_check_objid.rb
 - BINARIES
+- HOWTO
 - SCRIPTING
 - SETTINGS
 - CHEATSHEET

data/lib/nwn/gff/api.rb

@@ -1,88 +0,0 @@
-
-module NWN::Gff::Field
-#:stopdoc:
-  # Used by NWN::Gff::Struct#by_flat_path
-  def each_by_flat_path &block
-    case field_type
-      when :cexolocstr
-        yield("", self)
-        field_value.sort.each {|lid, str|
-          yield("/" + lid.to_s, str)
-        }
-
-      when :struct
-        yield("", self)
-        field_value.each_by_flat_path {|v, x|
-          yield(v, x)
-        }
-
-      when :list
-        yield("", self)
-        field_value.each_with_index {|item, index|
-          yield("[" + index.to_s + "]", item)
-          item.each_by_flat_path("/") {|v, x|
-            yield("[" + index.to_s + "]" + v, x)
-          }
-        }
-
-      else
-        yield("", self)
-    end
-  end
-#:startdoc:
-end
-
-module NWN::Gff::Struct
-
-  # Iterates this struct, yielding flat, absolute
-  # pathes and the Gff::Field for each element found.
-
-  # Example:
-  # "/AddCost" => {"type"=>:dword, ..}
-  def each_by_flat_path prefix = "/", &block
-    sort.each {|label, field|
-      field.each_by_flat_path do |ll, lv|
-        yield(prefix + label + ll, lv)
-      end
-    }
-  end
-
-  # Retrieve an object from within the given tree.
-  # Path is a slash-separated destination, given as
-  # a string
-  #
-  # Prefixed/postfixed slashes are optional.
-  #
-  # Examples:
-  # /
-  # /AddCost
-  # /PropertiesList/
-  # /PropertiesList[0]/CostValue
-  def by_path path
-    struct = self
-    current_path = ""
-    path = path.split('/').map {|v| v.strip }.reject {|v| v.empty?}.join('/')
-
-    path.split('/').each {|v|
-      if v =~ /^(.+?)\[(\d+)\]$/
-        v, index = $1, $2
-      end
-
-      struct = struct[v]
-      if index
-        struct.field_type == :list or raise NWN::Gff::GffPathInvalidError,
-          "Specified a list offset for a non-list item: #{v}[#{index}]."
-
-        struct = struct.field_value[index.to_i]
-      end
-
-      raise NWN::Gff::GffPathInvalidError,
-        "Cannot find a path to /#{path} (at: /#{current_path})." unless struct
-
-      current_path += v
-    }
-
-    struct
-  end
-
-end