dataMetaPii 1.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 6f161338ef8796386b9b83b7fd82251d1554b241
+  data.tar.gz: 20721eb45e7d5090dd4e8922dfd7994e9ac8cd3e
+SHA512:
+  metadata.gz: 68ba84d15ab2eaef7267a64a1da4f08376a4ff5acbdde3b4af01b17110b8aef6d9f8ea72f7ce1518b06f5218afc03b24d0487ccad1cae4a4dd263e3ac8aedbfd
+  data.tar.gz: 838781277828e11934547bbbb1d2261167af45fa1da34c4b447513d6cf3efe1025d23c4f800b8efc062c5465d8ed003bf83264068f3c95687f12ec8253ff45c2

data/.yardopts

@@ -0,0 +1 @@
+--title "DataMeta/PII" -r README.md --charset UTF-8 lib/**/* - README.md

data/History.md

@@ -0,0 +1,10 @@
+# `dataMetaPii` Release history:
+
+## `1.0.1` released `2017-02-16 Thu` by [`mub`](https://github.com/mub)
+* 1 update:
+    * bug fix: the code generator was wiping out the destination directory which made generating multiple versions
+        impossible.
+
+## `1.0.0` released `2017-01-15`
+* 1 major enhancement:
+    * Initial release

data/PostInstall.txt

@@ -0,0 +1,2 @@
+No special steps
+

data/README.md

@@ -0,0 +1,77 @@
+## dataMetaPii
+
+The PII (Personally Identifiable Information) registry support: master data field definition and link to using those to
+applications/services code.
+
+References to this gem's:
+
+* [Source](https://github.com/eBayDataMeta/DataMeta-gems)
+
+## DESCRIPTION:
+
+This gem provides:
+
+* The [DSL](https://en.wikipedia.org/wiki/Domain-specific_language) to describe PII fields definition with properties
+    such as impact level and any other properties you may want to add.
+* And another [DSL](https://en.wikipedia.org/wiki/Domain-specific_language) to describe how the PII fields are related
+    to the application/services code.
+    such as impact level and any other properties you may want to add.
+* Parsers for the [DSLs](https://en.wikipedia.org/wiki/Domain-specific_language)↑
+* Exporters of the PII definition fo Java, Scala, Python and JSON.
+
+## FEATURES/PROBLEMS:
+
+The gem features the API and the executable for the code export.
+
+## SYNOPSIS:
+
+### Code Export
+
+The Gem has the executable named `dmPiiGenCode.rb` which is used to export
+DataMetaPII definitions into different formats.
+
+The executable's help text you get by running `dmPiiGenCode.rb` without arguments
+looks like this:
+
+```
+Usage: dmPiiGenCode.rb <Scope> <ExportFormat> <OutputRoot> [ Namespace ]
+
+Exports the PII definition into sources
+
+Parameters:
+   <Scope> - one of: abstract, app
+
+   <ExportFormat>  - one of: java, scala, python, json
+
+   <OutputRoot> - must be a valid directory in the local file system.
+
+   [ Namespace ] - for Java and Scala - package name, for Python - module name, for JSON - does not matter.
+
+   DataMetaPII sources should be piped in.
+
+```
+
+Therefore, to export the version `1.0.0` of the Abstract Fields master to 
+JSON into current directory, you would run:
+
+```
+cat abstract-1.0.0.dmPii | piiGenCode.rb abstract json .
+```
+
+Which would write a file named `PiiAbstractDef_1_0_0.json` in current directory.
+
+### AppLink API
+
+The gem provides the AppLink DSL parsing API to obtain the data structure describing the AppLink 
+and use it to generate code.
+
+Run `DataMetaPii.parseAppLink(source)` to get the AppLink data structures in memory
+
+## INSTALL:
+
+    gem install dataMetaPii
+
+## LICENSE:
+
+[Apache v 2.0](https://github.com/eBayDataMeta/DataMeta/blob/master/LICENSE.md)
+

data/Rakefile

@@ -0,0 +1,13 @@
+%w(yard rake/testtask ./lib/dataMetaPii).each{ |r| require r}
+
+Rake::TestTask.new do |t|
+  t.libs << 'test'
+end
+
+desc 'Regen Yard docs'
+task :default => :docs
+
+YARD::Rake::YardocTask.new('docs') {|r|
+    r.stats_options = ['--list-undoc']
+}
+

data/bin/dmPiiGenCode.rb

@@ -0,0 +1,65 @@
+#!/usr/bin/env ruby
+
+require 'dataMetaDom'
+require 'dataMetaDom/record'
+require 'dataMetaDom/field'
+require 'dataMetaDom/ver'
+require 'dataMetaDom/dataType'
+require 'dataMetaPii'
+
+L = Logger.new("#{File.basename(__FILE__)[0..-4]}.log", 0, 10_000_000)
+
+L.datetime_format = '%Y-%m-%d %H:%M:%S'
+
+=begin
+For testing, run, for example:
+
+    cat source.dmPii | piiGenCode.rb abstract java .tmp
+=end
+def help(errorText = nil)
+
+    puts %Q|
+Usage: #{File.basename(__FILE__)} <Scope> <ExportFormat> <OutputRoot> [ Namespace ]
+
+Exports the PII definition into sources
+
+Parameters:
+   <Scope> - one of: #{DataMetaPii::Scope.constants.map{|c| DataMetaPii::Scope.const_get(c).to_s}.join(', ')}
+
+   <ExportFormat>  - one of: #{DataMetaPii::ExportFmts.constants.map{|c| DataMetaPii::ExportFmts.const_get(c).to_s}.join(', ')}
+
+   <OutputRoot> - must be a valid directory in the local file system.
+
+   [ Namespace ] - for Java and Scala - package name, for Python - module name, for JSON - does not matter.
+
+   DataMeta PII sources should be piped in.
+
+|
+    $stderr.puts(%Q<\nERROR: #{errorText}>) if errorText
+
+    exit errorText ? 2 : 1
+end
+
+@scopeDef, @outFmtDef, @outDirName, @namespace = $*
+help unless @scopeDef && @outFmtDef && @outDirName
+
+@scope, @outFmt = [@scopeDef, @outFmtDef].map(&:to_sym)
+
+help %q<Source is not piped in> if $stdin.tty?
+
+puts %Q|
+Exporting #{@scope} to #{@outFmt}:
+
+Generating results to #{@outDirName}...
+
+|
+
+begin
+    DataMetaPii.genCode(@scope, @outFmt, @outDirName, $stdin.read, @namespace)
+    puts 'Done.'
+rescue Exception => x
+    L.error(%<#{x}\n#{DataMetaPii::INDENT}#{x.backtrace.join("\n#{DataMetaPii::INDENT}")}>)
+    help x.to_s
+end
+
+

data/grammar/appLink.treetop

@@ -0,0 +1,214 @@
+# PII Master Registry to Application Link DSL grammar
+# to debug, uncomment next line and add this anywhere: &{|s| debugger; true }
+#require 'ruby-debug'
+
+grammar PiiAppLink
+
+# include the PII commons definition, rules shared between PII grammars; it's likely in the same directory
+# as this file
+    include PiiCommons
+
+# Top level rule - encompasses the whole definition file
+    rule appLink
+        piiWss? verDef:version piiWss? ad:attrbDiv? piiWss? al:appLinkDiv piiWss?
+    end
+
+# Rule for the Attributes Division that defines reusable attributes
+    rule attrbDiv
+        'attributes'  piiWss? '{' piiWss? a:attrbSectionSet piiWss? '}' {
+           def type; 'attrbDiv' end
+        }
+    end
+
+# Helper rule: a set of attribute sections
+    rule attrbSectionSet
+        attrbSection+
+    end
+
+# Attribute Section with the attribute set's ID and the attributes under it.
+    rule attrbSection
+        piiWss? k:symbol piiWss? '{' piiWss? a:attrbList piiWss? '}' piiWss? {
+            def type; 'attrbSect' end
+            def pk; k.text_value end
+        }
+    end
+
+# Application Link Division:
+    rule appLinkDiv
+        'apps' piiWss? '{' piiWss? a:appLinkSectionSet+ '}' piiWss? {
+            def type; 'appLinkDiv' end
+        }
+    end
+
+# Set of Application Link definitions for one application/service: App/Service ID and matching attributes under it,
+# including reusables.
+    rule appLinkSectionSet
+      piiWs? k:symbol piiWss? '{' piiWss? a:appLinkList piiWss? '}' piiWss? {
+            def type
+               'appLinkAapps'
+            end
+            def ak; k.text_value end
+        }
+    end
+
+# An aggregate of Application Link definitions for one Abstract PII ID
+    rule appLinkList
+        appLinkAttrbs+ {
+            def type; 'appLinkList' end
+        }
+    end
+
+# The PII key with attributes (one or more), i.e. key=value, optionally ornamented/separated by comments, whitespaces including newlines
+    rule appLinkAttrbs
+      piiWss? k:piiKey piiWss? '{' piiWss? a:attrbList piiWss? '}' piiWss? {
+            def type
+               'appLinkAttrbs'
+            end
+            def pk; k.text_value end
+        }
+    end
+
+# The aggregate of attributes belonging to a PII ID under Application/Service ID
+    rule attrbList
+        h:attrbDef t:(c:',' w:piiWss? v:attrbDef)* {
+            def type; DataMetaPii::ATTRB_LIST_NODE_TYPE end
+            # seemingly, an alternative to deep digging
+            # def getAll; result = [h]; if t; result << t.v.elements.map{|e| e} end; result.flatten end
+            # but it does not work: the parser does not collect all the v elements under t
+        }
+    end
+
+# One single attribute definition: it can be either constant, or a reference to a reusable or a special
+# case of the VO (Value Object) class such as Scala Case Class or a Java Storage Class/Java Bean.
+    rule attrbDef
+        ch:(c:constDef / r:refVal / v:voClass) piiWss? {
+            def type; DataMetaPii::ATTRB_DEF_NODE_TYPE end
+            def val
+                case nodeType
+                    when DataMetaPii::REF_KEY
+                        ch.r.sym
+                    when  DataMetaPii::CONST_KEY
+                        ch.c.val
+                    when DataMetaPii::VO_CLASS_KEY
+                        ch.v.klass
+                else # neither -- raise an error
+                    raise ArgumentError, %<Unsupported attribute definition type in "#{self.text_value}">
+                end
+            end
+            def node
+                case nodeType
+                    when DataMetaPii::REF_KEY
+                        ch.r
+                    when  DataMetaPii::CONST_KEY
+                        ch.c
+                    when DataMetaPii::VO_CLASS_KEY
+                        ch.v
+                else # neither -- raise an error
+                    raise ArgumentError, %<Unsupported attribute definition type in "#{self.text_value}">
+                end
+            end
+            def nodeType # figure out the type of this definition
+                if ch.respond_to?(:r)
+                    DataMetaPii::REF_KEY
+                elsif ch.respond_to?(:c)
+                    DataMetaPii::CONST_KEY
+                elsif ch.respond_to?(:v)
+                    DataMetaPii::VO_CLASS_KEY
+                else # neither -- raise an error
+                    raise ArgumentError, %<Unsupported attribute definition type in "#{self.text_value}">
+                end
+            end
+        }
+    end
+
+# Value Object (such as Java Bean or Scala Case class)
+    rule voClass
+        w 'voClass' w ':' w c:properCaseSym piiWss? {
+            def type; DataMetaPii::VO_CLASS_KEY end
+            def klass; c.text_value end
+        }
+    end
+
+# Reference to a reusable definition
+    rule refVal
+        '@' symElem:symbol piiWss? {
+            def type; DataMetaPii::REF_KEY end
+            def sym; symElem.text_value end
+        }
+    end
+
+# Constant definition
+    rule constDef
+        k:symbol w ':' w v:constValue {
+            def type; DataMetaPii::CONST_KEY end
+            def key; k.text_value end
+            def val; v.text_value end
+            def valNode; v end
+            def keyNode; k end
+            # has to be defined here because the definition on constValue does not work
+            def dataType # report the type of this constant
+                if v.respond_to?(:s)
+                    DataMetaPii::STR_CONST_DT # it's a string
+                elsif v.respond_to?(:d)
+                    DataMetaPii::DECIMAL_CONST_DT # it's a decimal number with a fraction
+                elsif v.respond_to?(:i)
+                    DataMetaPii::INT_CONST_DT # it's an integer number without a fraction
+                else
+                    raise ArgumentError, %<Unsupported constant type in "#{self.text_value}">
+                end
+            end
+            # has to be defined here because the definition on constValue does not work
+            def nodeVal
+                case dataType
+                    when DataMetaPii::STR_CONST_DT # it's a string
+                        eval(val)
+                    when DataMetaPii::DECIMAL_CONST_DT # it's a decimal number with a fraction
+                        BigDecimal.new(val)
+                    when DataMetaPii::INT_CONST_DT # it's an integer number without a fraction
+                        val.to_i
+                else
+                    raise ArgumentError, %<Unsupported constant type in "#{self.text_value}">
+                end
+            end
+        }
+    end
+
+# constant value: either an optionally signed integer or optionally signed decimal with fraction
+    rule constValue
+        d:fixedDecimal / i:decIntSignable / s:string {
+            def type
+                'constValue'
+            end
+            # This does not work, probably because of the "OR" condition. The one on constDef does.
+            # Keeping it with the comment to avoid the temptation to move it back here in future, unless
+            # the treetop team fixes the bug
+            def nodeVal
+                case dataType
+                    when DataMetaPii::STR_CONST_DT # it's a string
+                        eval(eval(text_value))
+                    when DataMetaPii::DECIMAL_CONST_DT # it's a decimal number with a fraction
+                        BigDecimal.new(text_value)
+                    when DataMetaPii::INT_CONST_DT # it's an integer number without a fraction
+                        text_value.to_i
+                else
+                    raise ArgumentError, %<Unsupported constant type in "#{self.text_value}">
+                end
+            end
+            # This does not work, probably because of the "OR" condition. The one on constDef does.
+            # Keeping it with the comment to avoid the temptation to move it back here in future, unless
+            # the treetop team fixes the bug
+            def dataType # report the type of this constant
+                if self.respond_to?(:s)
+                    DataMetaPii::STR_CONST_DT # it's a string
+                elsif self.respond_to?(:d)
+                    DataMetaPii::DECIMAL_CONST_DT # it's a decimal number with a fraction
+                elsif self.respond_to?(:i)
+                    DataMetaPii::INT_CONST_DT # it's an integer number without a fraction
+                else
+                    raise ArgumentError, %<Unsupported constant type in "#{self.text_value}">
+                end
+            end
+        }
+    end
+
+end

data/grammar/piiCommons.treetop

@@ -0,0 +1,46 @@
+# PII rules common between the concrete grammers, to be reused.
+grammar PiiCommons
+
+    # Load the commons from the DataMetaParse which lives in:
+    #    binary: http://FIXME
+    #    documentation: http://FIXME
+    #    source: https://FIXME
+    include DataMetaCommonsRoot
+
+# PII field key - a sequence of symChar, one or more
+    rule piiKey
+        symChar+
+    end
+
+# One or more PII Whitespaces
+    rule piiWss
+       piiWs+
+    end
+
+# PII Whitespace: a comment with endline or a plain whitespace as defined in DataMeta Parsing Commons
+    rule piiWs
+       sEol / W
+    end
+
+# Symbol with the first letter lowercase - suitable for a variable name.
+    rule symbol
+       [a-z] symChar*
+    end
+
+# Symbol with the first letter uppercase - suitable for a class name.
+    rule properCaseSym
+       [A-Z] symChar*
+    end
+
+# Symbol character: letter, number, underscore
+    rule symChar
+        [a-zA-Z0-9_]
+    end
+
+    rule version
+        'ver' W v:string {
+            def type; 'version' end
+            def ver; eval(v.text_value) end
+        }
+    end
+end