dgd-tools 0.1.11 → 0.1.12

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 838abb7957cb56eb3f096e8e345b3afda6d12db91a6110b7fa07029d589124d5
-  data.tar.gz: aa0a2551b3763eb418f99352cb0d40d58bcae92ff50bfce31bff22c644ae7019
+  metadata.gz: ed7c35c57fad646f3083312c2c4a0d3b2ed1dcc9fa0d7f95f4b2c89d49ddadba
+  data.tar.gz: 8124dbf072f51dfd84ba1bde82fe60e3521d20b7eb53ef8dc413b8d31f78a57b
 SHA512:
-  metadata.gz: a661db27d13a97f9086c7802769e1a08ac3f21f7c120f8bea6a6efe6e332a3d2ecb7e2571e0020dd36c8188019af907023662e248c1c1ffadfd9ee3c1b9c5b2a
-  data.tar.gz: 15e3742398cb16222e176dc5bc9385445219e23fce1a0cdf9205ced86ad47570c4c422d637734386f6f79e154282c3f57cf02784f9e4238f1ec8208dc4944878
+  metadata.gz: 0db04c0632d1ee81659be61f9d5341602f1112a148577b32f89fe9d327ec607910d979e2843cd3b6466269fdb7bb11ddd1c5eda29a111f019e6e753d5619c7bb
+  data.tar.gz: 0b888a3f260a47cf77d3024555cdb38322b646ce016511a5edc7331ae6b0e43852c6f698e2fdf0a022071a763e31ebdfbb22e30d9cdc7a673036b8d549d6ea28

data/Gemfile.lock

@@ -1,9 +1,10 @@
 PATH
   remote: .
   specs:
-    dgd-tools (0.1.10)
+    dgd-tools (0.1.11)
       nokogiri (~> 1.10.5)
       optimist (~> 3.0.1)
+      treetop
 
 GEM
   remote: https://rubygems.org/
@@ -13,7 +14,10 @@ GEM
     nokogiri (1.10.10)
       mini_portile2 (~> 2.4.0)
     optimist (3.0.1)
+    polyglot (0.3.5)
     rake (12.3.3)
+    treetop (1.6.11)
+      polyglot (~> 0.3)
 
 PLATFORMS
   ruby

data/README.md

@@ -20,7 +20,7 @@ You can find example "goods" (library) files under the "goods" subdirectory of t
 
 ## WOE Objects and skotos-xml-diff
 
-SkotOS-based games use an XML format for in-game objects called WOE, which is [documented in SkotOS-Doc](https://ChatTheatre.github.io/SkotOS-Doc). The skotos-xml-diff utility will diff between WOE objects or directories of WOE objects.
+SkotOS-based games use an XML format for in-game objects ("/base/obj/thing"), which is [documented in SkotOS-Doc](https://ChatTheatre.github.io/SkotOS-Doc). The skotos-xml-diff utility will diff between Things or directories of Things.
 
 See SkotOS-Doc for more detail about how this can be used with a SkotOS game.
 
@@ -30,15 +30,21 @@ Run "skotos-xml-diff --help" for a list of options. You can tell it to ignore wh
 
 You would normally install DGDTools directly: `gem install dgd-tools`.
 
-It's possible to add it to a Ruby's application's Gemfile. Ordinarily you wouldn't.
+If you have a Ruby Gemfile or gems.rb that uses it, you can "bundle install" as normal.
 
 ## Usage
 
-If you have a DGD application that uses DGD Manifest, run `dgd-manifest install` to download its dependencies and create a fully-assembled DGD directory for it. You can also `dgd-manifest test` to make sure its dependencies are downloaded and satisfied without building an application directory.
+If you have a DGD application that uses DGD Manifest, run `dgd-manifest install` to download its dependencies and create a clean, fully-assembled DGD directory for it. You can also `dgd-manifest test` to make sure its dependencies are downloaded and satisfied without building an application directory. After the initial install, "dgd-manifest update" will avoid deleting any files you may have added.
+
+NOTE: "dgd-manifest install" will delete any extra files you may have created in the DGD root. "dgd-manifest update" will not. Neither of these is always the right answer.
 
 That fully-assembled DGD directory is named ".root". To run your dgd server, type "dgd-manifest server".
 
-If you update files in your root and want to update files under the generated root directory, use "dgd-manifest update".
+## Creating a New DGD Manifest App
+
+You can type "dgd-manifest new my_app_name" to create a new application using the appropriate DGD manifest structure. This is an easy way to set up an appropriate .gitignore file and similar.
+
+You can, of course, do the same without the command if you like.
 
 ## Using DGD Manifest with your DGD Application
 
@@ -94,6 +100,12 @@ Here's what those look like:
 }
 ```
 
+Fields in the Goods file are the same as fields in unbundled_goods.
+
+## Implementation Limits
+
+Right now it's not possible to use the same repo multiple times with different branches. So for instance, you can't use one branch of ChatTheatre/SkotOS for your main library while using a different branch of it for some other library. For this reason, it's recommended that you extract smaller libraries into their own repositories rather than keeping multiple libraries in the same repo.
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests.

data/dgd-tools.gemspec

@@ -26,4 +26,5 @@ Gem::Specification.new do |spec|
 
   spec.add_runtime_dependency "nokogiri", "~>1.10.5"
   spec.add_runtime_dependency "optimist", "~>3.0.1"
+  spec.add_runtime_dependency "treetop"
 end

data/lib/dgd-tools/dgd-doc.rb

@@ -0,0 +1,56 @@
+require "dgd-tools/version"
+
+require "treetop"
+
+require_relative "dgd_grammar_support"
+Treetop.load "#{__dir__}/dgd_grammar.tt"
+
+# Preprocessor incantation: cpp -C -I. -D__DGD__ name_of_file.c
+
+module DGD; end
+
+module DGD::Doc
+
+  class SourceFile
+    attr_reader :parser
+
+    def initialize(path, dgd_root:)
+      unless File.exist?(path)
+        raise "No such source file for DGD::Doc::Sourcefile: #{path.inspect}"
+      end
+
+      @parser = DGDGrammarParser.new
+      @path = path
+      @dgd_root = dgd_root
+
+      parse_contents
+    end
+
+    private
+
+    def parse_contents
+      preproc_output = `cpp -C -I#{@dgd_root}/include -D__DGD__ #{@path}`
+      preproc_output.gsub!(/\n+/, "\n")  # The preprocessor often winds up leaving a lot of newlines from various sources
+      preproc_output.gsub!(/^# \d+ ".*"\w*\d*$/, "")
+      data = @parser.parse(preproc_output)
+      if data
+        clean_tree(data)
+        #puts data.inspect
+      else
+        puts @parser.failure_reason
+        puts @parser.failure_line
+        puts @parser.failure_column
+
+        #puts "Preproc output was:#{preproc_output}\n\n\n"
+        raise "Parse error!"
+      end
+    end
+
+    def clean_tree(root_node)
+      return if(root_node.elements.nil?)
+      root_node.elements.each {|node| self.clean_tree(node) }
+      root_node.elements.delete_if {|node| node.class.name == "Treetop::Runtime::SyntaxNode" && node.text_value.size < 20 }
+    end
+
+  end
+end

data/lib/dgd-tools/dgd_grammar.tt

@@ -0,0 +1,373 @@
+grammar DGDGrammar
+  rule source_file
+    ws (inherit ws)* (top_level_decl ws)*
+  end
+
+  rule inherit
+    ("private" WS)? "inherit" WS (ident WS)? ('object' WS)? string_exp ws ';' <Inherit>
+  end
+
+  rule top_level_decl
+    data_decl / func_decl
+  end
+
+  rule ident
+    [a-zA-Z_] [a-zA-Z_0-9]* <Identifier>
+  end
+
+  rule c_comment
+    '/*'
+    (
+      !'*/'
+      (. / "\n")
+    )*
+    '*/'
+  end
+
+  rule preproc_hash_line
+    "\n"? "# " decimal " "+ simple_string (" "+ decimal)? "\n"
+  end
+
+  rule simple_string
+    '"' (
+      !["\n\\]
+      .
+      )*
+    '"' <StringLiteral>
+  end
+
+  rule complex_string
+    '"' ( ((!["\n\\] ) .) / . )+ '"' <StringLiteral>
+  end
+
+  rule simple_char
+    "'" ((!['\n\\]) .)+ "'" <CharLiteral>
+  end
+
+  rule complex_char
+    "'" (((!['\n\\]) .) / ("\\" .))+ "'" <CharLiteral>
+  end
+
+  rule decimal
+    [1-9] [0-9]* <DecimalLiteral>
+  end
+
+  rule octal
+    '0' [0-7]* <OctalLiteral>
+  end
+
+  rule hexadecimal
+    '0' [xX] [a-fA-F0-9]+ <HexadecimalLiteral>
+  end
+
+  rule float
+    ([0-9]+ "." [0-9]+ ([eE] [-+]? [0-9]+)?) <FloatLiteral> /
+    ("." [0-9]+ ([eE] [-+]? [0-9]+)?) <FloatLiteral> /
+    ([0-9]+ [eE] [-+]? [0-9]+) <FloatLiteral>
+  end
+
+  rule single_WS
+    [ \t\v\f\r\n] / c_comment / preproc_hash_line
+  end
+
+  rule ws
+    single_WS*
+  end
+
+  rule WS
+    single_WS+
+  end
+
+  rule string
+    simple_string /
+    complex_string
+  end
+
+  rule string_exp
+    string /
+    '(' ws composite_string ws ')'
+  end
+
+  rule composite_string
+    string_exp / string_exp '+' composite_string
+  end
+
+  rule klass
+    (class_spec WS)* class_spec
+  end
+
+  rule class_spec
+    'private' / 'static' / 'atomic' / 'nomask' / 'varargs'
+  end
+
+  rule type_spec
+    'int' / 'float' / 'string' / 'object' / 'mapping' / 'mixed' / 'void'
+  end
+
+  rule stars
+    ('*' ws)*
+  end
+
+  rule formals
+    # If you put these in a different order with '' and 'void' first, parses start failing. That seems bad.
+    formal_list (ws '...')? / '' / 'void'
+  end
+
+  rule formal_list
+    formal (ws ',' ws formal)*
+  end
+
+  rule formal
+    class_type data_declarator /
+    ident
+  end
+
+  rule data_decl
+    class_type WS declarators ws ';' <DataDecl>
+  end
+
+  rule func_decl
+    class_type WS function_declarator ws compound_statement <FuncDecl> /
+    klass WS function_name ws '(' ws formals ws ')' ws compound_statement <FuncDecl>
+  end
+
+  rule class_type
+    (class_spec WS)* type_spec /
+    (class_spec WS)* 'object' list_exp
+  end
+
+  rule data_declarator
+    stars ws ident
+  end
+
+  rule function_name
+    ident /
+    'operator' ws '+' /
+    'operator' ws '-' /
+    'operator' ws '*' /
+    'operator' ws '/' /
+    'operator' ws '%' /
+    'operator' ws '&' /
+    'operator' ws '^' /
+    'operator' ws '|' /
+    'operator' ws '<' /
+    'operator' ws '>' /
+    'operator' ws '>=' /
+    'operator' ws '<=' /
+    'operator' ws '<<' /
+    'operator' ws '>>' /
+    'operator' ws '~' /
+    'operator' ws '++' /
+    'operator' ws '--' /
+    'operator' ws '[' ws ']' /
+    'operator' ws '[' ws ']' ws '=' /
+    'operator' ws '[' ws '..' ws ']'
+  end
+
+  rule function_declarator
+    stars ws function_name ws '(' ws formals ws ')'
+  end
+
+  rule declarator
+    function_declarator / data_declarator
+  end
+
+  rule declarators
+    declarator (ws ',' ws declarator)*
+  end
+
+  rule locals
+    (ws data_decl ws)*
+  end
+
+  rule statements
+    (ws statement ws)*
+  end
+
+  rule statement
+    'if' ws '(' ws list_exp ws ')' ws statement (WS 'else' statement)? /
+    'do' WS statement WS 'while' ws '(' ws list_exp ws ')' ws ';' /
+    'while' ws '(' ws list_exp ws ')' ws statement /
+    'for' ws '(' ws list_exp? ws ';' ws list_exp? ws ';' ws list_exp? ws ')' ws statement /
+    'rlimits' ws '(' ws list_exp ws ';' ws list_exp ws ')' ws compound_statement /
+    'catch' ws compound_statement ws ':' statement /
+    'catch' ws compound_statement /
+    'switch' ws '(' ws list_exp ws ')' ws compound_statement /
+    'case' WS exp ws ':' ws statement /
+    'case' WS exp ws '..' ws exp ws ':' statement /
+    'default' ws ':' ws statement /
+    'goto' WS ident ws ';' /
+    'break' ws ';' /
+    'continue' ws ';' /
+    'return' WS list_exp WS ';' /
+    'return' ws ';' /
+    ident ws ':' ws statement /  # label statement
+    list_exp /
+    compound_statement /
+    ';'
+  end
+
+  rule compound_statement
+    '{' ws locals ws statements ws '}'
+  end
+
+  rule function_call
+    ('::' ws function_name) /
+    (ident ws '::' ws function_name) /
+    function_name
+  end
+
+  rule list_exp
+    exp ws ',' ws list_exp /
+    exp
+  end
+
+  rule exp
+    cond_exp ws '=' ws exp /
+    cond_exp ws '+=' ws exp /
+    cond_exp ws '-=' ws exp /
+    cond_exp ws '*=' ws exp /
+    cond_exp ws '/=' ws exp /
+    cond_exp ws '%=' ws exp /
+    cond_exp ws '<<=' ws exp /
+    cond_exp ws '>>=' ws exp /
+    cond_exp ws '&=' ws exp /
+    cond_exp ws '^=' ws exp /
+    cond_exp ws '|=' ws exp /
+    cond_exp
+  end
+
+  rule cond_exp
+    or_exp ws '?' ws list_exp ws ':' ws cond_exp / or_exp
+  end
+
+  rule or_exp
+    and_exp ws '||' ws or_exp / and_exp
+  end
+
+  rule and_exp
+    bitor_exp ws '&&' ws and_exp / bitor_exp
+  end
+
+  rule bitor_exp
+    bitxor_exp ws '|' ws bitor_exp / bitxor_exp
+  end
+
+  rule bitxor_exp
+    bitand_exp ws '^' ws bitxor_exp / bitand_exp
+  end
+
+  rule bitand_exp
+    equ_exp ws '&' bitand_exp / equ_exp
+  end
+
+  rule equ_exp
+    rel_exp ws '==' ws equ_exp /
+    rel_exp ws '!=' ws equ_exp /
+    rel_exp
+  end
+
+  rule rel_exp
+    shift_exp ws '<' ws rel_exp /
+    shift_exp ws '>' ws rel_exp /
+    shift_exp ws '<=' ws rel_exp /
+    shift_exp ws '>=' ws rel_exp /
+    shift_exp
+  end
+
+  rule shift_exp
+    add_exp ws '<<' ws shift_exp /
+    add_exp ws '>>' ws shift_exp /
+    add_exp
+  end
+
+  rule add_exp
+    mult_exp ws '+' ws add_exp /
+    mult_exp ws '-' ws add_exp /
+    mult_exp
+  end
+
+  rule mult_exp
+    cast_exp ws '*' ws mult_exp /
+    cast_exp ws '/' ws mult_exp /
+    cast_exp ws '%' ws mult_exp /
+    cast_exp
+  end
+
+  rule cast_exp
+    '(' ws class_type ws stars ws ')' ws cast_exp /
+    prefix_exp
+  end
+
+  rule prefix_exp
+    '++' ws cast_exp /
+    '--' ws cast_exp /
+    '+' ws cast_exp /
+    '-' ws cast_exp /
+    '!' ws cast_exp /
+    '~' ws cast_exp /
+    postfix_exp
+  end
+
+  rule postfix_exp
+    exp2 (ws ('++' / '--'))?
+  end
+
+  rule exp2
+    exp1 ws '->' ws ident ws '(' ws opt_arg_list ws ')' /
+    exp1 ws '<-' ws string_exp /
+    exp1 ((ws '[' ws list_exp ws ']') / (ws '[' ws (list_exp ws)? '..' ws (list_exp ws)? ']'))*
+  end
+
+  rule exp1
+    # This works (for now) with function_call up front, but not when it was lower down...
+    # There's some kind of bad stuff going on with precedence that's causing parse failures.
+    # That's weird, since I assume it should try all combinations/precedences before giving up...
+    function_call ws '(' ws opt_arg_list ws ')' /
+    'catch' ws '(' list_exp ws ')' /
+    'new' (WS 'object')? ws string_exp (ws '(' ws opt_arg_list ws ')')? /
+    'nil' /
+    '(' ws '{' ws opt_arg_list_comma ws '}' ws ')' /
+    '(' ws '[' ws opt_assoc_list_comma ws ']' ws ')' /
+    '::' ws ident /
+    decimal /
+    octal /
+    hexadecimal /
+    simple_char /
+    complex_char /
+    float /
+    string /
+    ident /
+    '(' ws list_exp ws ')'
+  end
+
+  rule arg_list
+    exp ws ',' ws arg_list /
+    exp
+  end
+
+  rule opt_arg_list
+    arg_list ws '...' /
+    arg_list?
+  end
+
+  rule opt_arg_list_comma
+    arg_list ws ',' /
+    arg_list?
+  end
+
+  rule assoc_list
+    assoc_pair ws ',' ws assoc_list /
+    assoc_pair
+  end
+
+  rule assoc_pair
+    exp ws ':' ws exp
+  end
+
+  rule opt_assoc_list_comma
+    assoc_list ws ',' /
+    assoc_list?
+  end
+
+end

data/lib/dgd-tools/dgd_grammar_support.rb

@@ -0,0 +1,36 @@
+#class StringLiteral < Treetop::Runtime::SyntaxNode
+#  def some_name
+#    # This is a pretty crude way of handling character escaping, etc.
+#    return eval self.text_value
+#  end
+#end
+
+class DataDecl < Treetop::Runtime::SyntaxNode
+end
+
+class FuncDecl < Treetop::Runtime::SyntaxNode
+end
+
+class Inherit < Treetop::Runtime::SyntaxNode
+end
+
+class Identifier < Treetop::Runtime::SyntaxNode
+end
+
+class DecimalLiteral < Treetop::Runtime::SyntaxNode
+end
+
+class OctalLiteral < Treetop::Runtime::SyntaxNode
+end
+
+class HexadecimalLiteral < Treetop::Runtime::SyntaxNode
+end
+
+class FloatLiteral < Treetop::Runtime::SyntaxNode
+end
+
+class StringLiteral < Treetop::Runtime::SyntaxNode
+end
+
+class CharLiteral < Treetop::Runtime::SyntaxNode
+end

data/lib/dgd-tools/manifest.rb

@@ -128,6 +128,8 @@ module DGD::Manifest
 
         def assemble_app(location, verbose:)
             Dir[File.join(dgd_root(location), "*")].each { |dir| FileUtils.rm_rf dir }
+            Dir[File.join(dgd_root(location), "state", "*")].each { |dir| FileUtils.rm_rf dir }
+            Dir[File.join(dgd_root(location), ".repos", "*")].each { |dir| FileUtils.rm_f dir }
 
             write_app_files(location, verbose: verbose)
         end
@@ -142,6 +144,12 @@ module DGD::Manifest
             Dir.chdir(location) do
                 write_config_file("#{location}/dgd.config")
                 FileUtils.mkdir_p("#{location}/state") # Statedir for statedumps, editor files, etc.
+                repos_dir = "#{location}/.repos"
+                FileUtils.mkdir_p(repos_dir) # Links to repos in manifest
+
+                @manifest_file.ordered_specs.each do |spec|
+                    FileUtils.ln_s(spec.source.local_dir, File.join(".repos", spec.name))
+                end
 
                 assembly_operations(location, verbose: verbose).each do |sd_hash|
                     to_path = "#{dgd_root(location)}/#{sd_hash[:to]}"
@@ -429,6 +437,13 @@ module DGD::Manifest
                 .root
                 dgd.config
                 state/*
+                wafer
+                websocket-to-tcp-tunnel
+                dgd
+                log/*
+                skotos.database
+                skotos.database.old
+                .repos/**
             FILE_CONTENTS
         end
 
@@ -570,7 +585,6 @@ FILE_CONTENTS
                 end
 
                 ports = data.map { |p| [ "*", Integer(p) ] }
-                STDERR.puts "Arrayified: #{ports.inspect}"
                 return ports
             elsif data.is_a?(Integer)
                 return [ [ "*", data ] ]

data/lib/dgd-tools/version.rb

@@ -1,3 +1,3 @@
 module DGD
-  VERSION = "0.1.11"
+  VERSION = "0.1.12"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: dgd-tools
 version: !ruby/object:Gem::Version
-  version: 0.1.11
+  version: 0.1.12
 platform: ruby
 authors:
 - Noah Gibbs
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2021-03-18 00:00:00.000000000 Z
+date: 2021-05-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: nokogiri
@@ -38,6 +38,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 3.0.1
+- !ruby/object:Gem::Dependency
+  name: treetop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: dgd-tools supplies DGD Manifest and eventually perhaps other tools. DGD
   Manifest is an experimental DGD library and packaging system.
 email:
@@ -69,6 +83,9 @@ files:
 - exe/skotos-xml-diff
 - goods/chattheatre_kernellib.goods
 - goods/skotos_httpd.goods
+- lib/dgd-tools/dgd-doc.rb
+- lib/dgd-tools/dgd_grammar.tt
+- lib/dgd-tools/dgd_grammar_support.rb
 - lib/dgd-tools/manifest.rb
 - lib/dgd-tools/skotos_xml_obj.rb
 - lib/dgd-tools/version.rb