calyx 0.15.1 → 0.16.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 5b983f729355f2a8ee4266d2d2564c041f4d3fd5
-  data.tar.gz: 2815d2887cefe608c285537b8ef1cd9d34204497
+  metadata.gz: 2dd4b07658e5e6d37f0b1c7a1f75afb08d61ae5f
+  data.tar.gz: 6cf84a9ef492e359ff297a84129abca738c4ca91
 SHA512:
-  metadata.gz: f0a44222fd6aed60b2ae457e4e0b3cf3b9cae69c7433853f198420a4b9fd2091199a1d379c9c3e64eff281cc0dc39bb61ada8d366304f25319ddb12a5a2319f0
-  data.tar.gz: 565bfb3335ef9c3c87c2769da6c7f1276432998bcd2b6cf273eda62536f8dcd62123005f4a1aaf5d1cc55882a0119c8fb682df1d0717fad07a8344662e0c08a0
+  metadata.gz: c8bcd0a0eddc36040f008dc22de6d9df8b8f8e204e036c77f0efaceb226ab835888cfe997b4d9807f01e002b067de1f5b7750c1698711d3e0b05e92d3aac795f
+  data.tar.gz: fe055edd5d1062a6e55ab474ed9d923bc228ed276cb22dca3ce3c5c0cb218361208635bfeeff24112ca159f44d9914ac1ed535a1301f23c68bd73411f0b2fb5d

data/README.md

@@ -282,11 +282,19 @@ end
 Basic rule substitution uses single curly brackets as delimiters for template expressions:
 
 ```ruby
-class Fruit < Calyx::Grammar
+fruit = Calyx::Grammar.new do
   start '{colour} {fruit}'
   colour 'red', 'green', 'yellow'
   fruit 'apple', 'pear', 'tomato'
 end
+
+6.times { fruit.generate }
+# => "yellow pear"
+# => "red apple"
+# => "green tomato"
+# => "red pear"
+# => "yellow tomato"
+# => "green apple"
 ```
 
 ## String Modifiers
@@ -294,11 +302,12 @@ end
 Dot-notation is supported in template expressions, allowing you to call any available method on the `String` object returned from a rule. Formatting methods can be chained arbitrarily and will execute in the same way as they would in native Ruby code.
 
 ```ruby
-class Greeting < Calyx::Grammar
+greeting = Calyx::Grammar.new do
   start '{hello.capitalize} there.', 'Why, {hello} there.'
   hello 'hello', 'hi'
 end
 
+4.times { greeting.generate }
 # => "Hello there."
 # => "Hi there."
 # => "Why, hello there."
@@ -312,15 +321,16 @@ You can also extend the grammar with custom modifiers that provide useful format
 Filters accept an input string and return the transformed output:
 
 ```ruby
-class Greeting < Calyx::Grammar
+greeting = Calyx::Grammar.new do
   filter :shoutycaps do |input|
     input.upcase
   end
 
-  start '{hello.shoutycaps} there.', 'Why, {hello} there.'
+  start '{hello.shoutycaps} there.', 'Why, {hello.shoutycaps} there.'
   hello 'hello', 'hi'
 end
 
+4.times { greeting.generate }
 # => "HELLO there."
 # => "HI there."
 # => "Why, HELLO there."
@@ -332,12 +342,13 @@ end
 The mapping shortcut allows you to specify a map of regex patterns pointing to their resulting substitution strings:
 
 ```ruby
-class GreenBottle < Calyx::Grammar
+green_bottle = Calyx::Grammar.new do
   mapping :pluralize, /(.+)/ => '\\1s'
   start 'One green {bottle}.', 'Two green {bottle.pluralize}.'
   bottle 'bottle'
 end
 
+2.times { green_bottle.generate }
 # => "One green bottle."
 # => "Two green bottles."
 ```
@@ -355,12 +366,13 @@ module FullStop
   end
 end
 
-class Hello < Calyx::Grammar
+hello = Calyx::Grammar.new do
   modifier FullStop
   start '{hello.capitalize.full_stop}'
   hello 'hello'
 end
 
+hello.generate
 # => "Hello."
 ```
 
@@ -395,11 +407,12 @@ class String
   include FullStop
 end
 
-class NounsWithArticles < Calyx::Grammar
+noun_articles = Calyx::Grammar.new do
   start '{fruit.with_indefinite_article.capitalize.full_stop}'
   fruit 'apple', 'orange', 'banana', 'pear'
 end
 
+4.times { noun_articles.generate }
 # => "An apple."
 # => "An orange."
 # => "A banana."
@@ -507,6 +520,7 @@ Rough plan for stabilising the API and features for a `1.0` release.
 | `0.14`   | ~~Support for Ruby 2.4~~ |
 | `0.15`   | Options config and ‘strict mode’ error handling |
 | `0.16`   | Improve representation of weighted probability selection |
+| `0.17`   | Introduce wildcard syntax for meta rules (rules returning rules) |
 
 ## Credits
 

data/SYNTAX.md

@@ -0,0 +1,200 @@
+# Calyx Syntax Specification
+
+> An ad-hoc, informally specified, bug-ridden, etc... etc...
+
+## Background
+
+Since `v0.11`, Calyx has supported loading grammars from external JSON files—a very similar format to Tracery<sup>[1][1]</sup>—but the precise syntax and structure used by these files was never properly documented or defined in a schema<sup>[2][2]</sup>.
+
+This is worth documenting for several reasons:
+
+1) It’s rather obvious that having good documentation will make it easier for new users to get started and for advanced users to learn about the limits of what they can do with the tool.
+2) A well-defined schema reduces ambiguity and helps focus on authoring concerns, rather than drifting towards implementation concerns. This is currently a particular risk in Calyx because of the impedance mismatch between the Ruby DSL and JSON data.
+3) A well-defined schema opens up potential for collaboration with authors of other similar tools and could help provide a future foundation for a standard data format that enables sharing grammars across languages and tools. This would be of particular benefit to authors, making it easier to build up reusable content libraries. It could also provide a foundation for new innovations in authoring UIs that aren’t tied to a specific language or tool.
+
+## Format
+
+### Files
+
+External grammars are defined in JSON files. They must be encoded as `utf-8`, have a `.json` extension and conform to standard JSON syntax rules.
+
+### Structure
+
+#### Top Level
+
+The top-level structure of the grammar must be a map/object-literal with each key representing a single left-hand rule symbol and the value representing the grammar productions for that rule:
+
+```json
+{
+  "start": "Colorless green ideas sleep furiously."
+}
+```
+
+Empty grammars should be represented by an empty object:
+
+```json
+{}
+```
+
+#### Production Rules
+
+Left hand side rules must be string symbols conforming to the following pattern:
+
+```ruby
+/^[A-Za-z0-9_\-]+$/
+```
+
+Grammars are not context-sensitive<sup>[3][3]</sup>. The left-hand side rules must be a direct symbol reference, not a production that can be expanded.
+
+Right-hand side productions can be either single strings, arrays of strings or weighted probability objects.
+
+Strings represent the template for a single choice that the production will always resolve to:
+
+```json
+{
+  "start": "Colorless green ideas sleep furiously."
+}
+```
+
+Arrays of strings represent multiple choices that can produce any one of the possible output strings. Each string should have a (roughly) equal chance of being selected to expand to a result.
+
+```json
+{
+  "start": ["red", "green", "blue"]
+}
+```
+
+Weighted probability objects represent a mapping of possible output strings to their probability of expanding to a result. The keys represent the possible output strings, and the values represent their probability of the string being selected.
+
+Supported intervals are:
+
+- 0..1 (`Number`)
+
+The following example shows `red` with a 50% chance of being selected; `green` and `blue` with 25% chances:
+
+```json
+{
+  "start": {
+    "red": 0.5,
+    "green": 0.25,
+    "blue": 0.25
+  }
+}
+```
+
+#### Template Expansions
+
+Productions can be recursively expanded by embedding rules using the template expression syntax, with the expressions delimited by `{` and `}` characters. Everything outside of the delimiters is treated as literal text.
+
+Basic syntax:
+
+```json
+"{weather}"
+```
+
+Expanding a simple rule:
+
+```json
+{
+  "start": "The sky was {weather}.",
+  "weather": ["cloudy", "dark", "clear", "bright"]
+}
+```
+
+A chain of nested expansions:
+
+```json
+{
+  "start": "{best} {worst}",
+  "best": "{twas} the {best_adj} of times.",
+  "worst": "{twas} the {worst_adj} of times.",
+  "twas": ["It was", "'Twas"],
+  "best_adj": ["best", "greatest"],
+  "worst_adj": ["worst", "most insufferable"]
+}
+```
+
+#### Expression Modifiers
+
+There are two different forms of expression modifiers—**Selection Modifiers** and **Output Modifiers**.
+
+Selection modifiers apply to the grammar production itself, influencing how the rule is expanded. They are defined by prefixing a rule expression with a sigil that defines the behaviour of the selection.
+
+```json
+"{$unique_rule}"
+"{@memoized_rule}"
+```
+
+Output modifiers format the string that is generated by the grammar production. They are defined by a chain of `.` separated references following the rule.
+
+```json
+"{formatted_rule.upcase}"
+"{formatted_rule.downcase.capitalize}"
+```
+
+#### Unique Choices
+
+Unique choices are prefixed with the `$` sigil in an expression.
+
+This ensures that multiple references to the same production will always result in a unique value being chosen (until the choices in the production are exhausted).
+
+```json
+{
+  "start": "{$medal}. {$medal}. {$medal}.",
+  "medal": ["Gold", "Silver", "Bronze"]
+}
+```
+
+```json
+{
+  "start": "It was the {$adj} of times; it was the {$adj} of times.",
+  "adj": ["best", "worst"]
+}
+```
+
+#### Memoized Choices
+
+Memoized choices are prefixed with the `@` sigil in an expression.
+
+This ensures that multiple references to the same production will always result in the first selected value being repeated.
+
+```json
+{
+  "start": "The {@pet} ran to join the other {@pet}s.",
+  "pet": ["cat", "dog"]
+}
+```
+
+#### Output Modifiers
+
+Due to their dependency on Ruby string methods and Calyx internals, output modifiers are currently a bit of a nightmare for interoperability.
+
+All basic Ruby string formatting methods with arity 0 are supported by default<sup>[4][4]</sup>.
+
+```json
+"{my_rule.downcase}"
+"{my_rule.upcase}"
+"{my_rule.capitalize}"
+"{my_rule.reverse}"
+"{my_rule.swapcase}"
+"{my_rule.strip}"
+"{my_rule.lstrip}"
+"{my_rule.rstrip}"
+"{my_rule.succ}"
+"{my_rule.chop}"
+"{my_rule.chomp}"
+```
+
+The Ruby DSL provides a variety of methods for extending the supported range of modifiers. This behaviour currently won’t work at all when grammars are defined in JSON.
+
+## References
+
+[1]: http://tracery.io/
+[2]: http://json-schema.org/
+[3]: https://en.wikipedia.org/wiki/Context-sensitive_grammar
+[4]: https://ruby-doc.org/core-2.4.0/String.html
+
+1) http://tracery.io/
+2) http://json-schema.org/
+3) https://en.wikipedia.org/wiki/Context-sensitive_grammar
+4) https://ruby-doc.org/core-2.4.0/String.html

data/lib/calyx/production/weighted_choices.rb

@@ -10,13 +10,32 @@ module Calyx
       # @param [Array<Array>, Hash<#to_s, Float>] productions
       # @param [Calyx::Registry] registry
       def self.parse(productions, registry)
-        weights_sum = productions.reduce(0) do |memo, choice|
-          memo += choice.last
-        end
+        if productions.first.last.is_a?(Range)
+          range_max = productions.max { |a,b| a.last.max <=> b.last.max }.last.max
+
+          weights_sum = productions.reduce(0) do |memo, choice|
+            memo += choice.last.size
+          end
+
+          if range_max != weights_sum
+            raise Errors::InvalidDefinition, "Weights must sum to total: #{range_max}"
+          end
 
-        raise Errors::InvalidDefinition, 'Weights must sum to 1' if weights_sum != 1.0
+          normalized_productions = productions.map do |choice|
+            weight = choice.last.size / range_max.to_f
+            [choice.first, weight]
+          end
+        else
+          weights_sum = productions.reduce(0) do |memo, choice|
+            memo += choice.last
+          end
+
+          raise Errors::InvalidDefinition, 'Weights must sum to 1' if weights_sum != 1.0
+
+          normalized_productions = productions
+        end
 
-        choices = productions.map do |choice, weight|
+        choices = normalized_productions.map do |choice, weight|
           if choice.is_a?(String)
             [Concat.parse(choice, registry), weight]
           elsif choice.is_a?(Symbol)

data/lib/calyx/version.rb

@@ -1,3 +1,3 @@
 module Calyx
-  VERSION = '0.15.1'.freeze
+  VERSION = '0.16.0'.freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: calyx
 version: !ruby/object:Gem::Version
-  version: 0.15.1
+  version: 0.16.0
 platform: ruby
 authors:
 - Mark Rickerby
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-08-28 00:00:00.000000000 Z
+date: 2017-08-31 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -66,6 +66,7 @@ files:
 - Gemfile
 - LICENSE
 - README.md
+- SYNTAX.md
 - calyx.gemspec
 - examples/any_gradient.rb
 - examples/faker.rb
@@ -108,7 +109,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.6.11
+rubygems_version: 2.6.13
 signing_key: 
 specification_version: 4
 summary: Generate text with declarative recursive grammars