attentive 0.1.1 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +170 -15
  3. data/lib/attentive.rb +19 -0
  4. data/lib/attentive/abbreviations.rb +1 -1
  5. data/lib/attentive/composite_entity.rb +4 -5
  6. data/lib/attentive/config.rb +2 -0
  7. data/lib/attentive/cursor.rb +35 -3
  8. data/lib/attentive/entities/core.rb +3 -0
  9. data/lib/attentive/entities/core/date.rb +6 -0
  10. data/lib/attentive/entities/core/date/month.rb +7 -0
  11. data/lib/attentive/entities/core/date/relative.rb +6 -0
  12. data/lib/attentive/entities/core/date/relative/future.rb +27 -0
  13. data/lib/attentive/entities/core/date/relative/past.rb +24 -0
  14. data/lib/attentive/entities/core/date/wday.rb +7 -0
  15. data/lib/attentive/entities/core/email.rb +8 -0
  16. data/lib/attentive/entities/core/number.rb +8 -0
  17. data/lib/attentive/entities/core/number/float.rb +6 -0
  18. data/lib/attentive/entities/core/number/float/negative.rb +6 -0
  19. data/lib/attentive/entities/core/number/float/positive.rb +6 -0
  20. data/lib/attentive/entities/core/number/integer.rb +6 -0
  21. data/lib/attentive/entities/core/number/integer/negative.rb +5 -0
  22. data/lib/attentive/entities/core/number/integer/positive.rb +5 -0
  23. data/lib/attentive/entities/core/number/negative.rb +6 -0
  24. data/lib/attentive/entities/core/number/positive.rb +6 -0
  25. data/lib/attentive/entity.rb +37 -10
  26. data/lib/attentive/listener.rb +2 -2
  27. data/lib/attentive/matcher.rb +24 -35
  28. data/lib/attentive/token.rb +12 -0
  29. data/lib/attentive/tokenizer.rb +153 -108
  30. data/lib/attentive/tokens.rb +2 -2
  31. data/lib/attentive/tokens/any_of.rb +3 -3
  32. data/lib/attentive/tokens/regexp.rb +19 -2
  33. data/lib/attentive/version.rb +1 -1
  34. metadata +19 -4
  35. data/lib/attentive/entities/integer.rb +0 -5
  36. data/lib/attentive/entities/relative_date.rb +0 -44
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 96872811cbdc5d9db062e11cca2471dc4079a26d
4
- data.tar.gz: 0df0336e5edf492c51048473605c69a7e9fe1886
3
+ metadata.gz: b5952398cd8d68e82b27a4e9be335b748b315845
4
+ data.tar.gz: 77f1e15ef3e6952861d110f3c5d7605664e2979f
5
5
  SHA512:
6
- metadata.gz: 61869eae6aacb6c2a66aca38e55ef1ca927236d8239ad402c761744e88a8a9dad99006b5fec14648fe740d43df83a2678c45fad1403ce0f3444fa444261971c2
7
- data.tar.gz: 4305e44d30e464ff734a72aa3326b9525776ba35f45330cbef4018d8541f754300bd41e4395531831cc628b6ba58ed7f1f7f26511f29d2bdfcd5f87680e3dec5
6
+ metadata.gz: 49de7e3eff7a8964082ffd35ac964497c5a7c706194f160fcb29bad5710c6977fc1d58ff86189d788cbe8290145575b07147cb35cafcbe16167b10607cd29063
7
+ data.tar.gz: b7692af23cbaa6b44467ab64c989883477ade8c1a9e26133c6c27e69168dbd132ba4e6dc7b4abbd8a8b5efeda09714a3138181e3e49290d16f449320f7e4d7d5
data/README.md CHANGED
@@ -1,43 +1,196 @@
1
1
  # Attentive
2
2
 
3
- Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/attentive`. To experiment with that code, run `bin/console` for an interactive prompt.
3
+ Attentive is a library for matching messages to natural-language listeners.
4
4
 
5
- TODO: Delete this and the text above, and describe your gem
6
5
 
7
6
 
7
+ <br/>
8
+ ## Usage
8
9
 
9
- ## Installation
10
+ Its basic usage is like this:
11
+
12
+ ```ruby
13
+ include Attentive
14
+
15
+ listen_for "hi", context: { in: :any } do
16
+ puts "nice to meet you!"
17
+ end
18
+
19
+ hear! "hi!" # => "nice to meet you!"
20
+ ```
21
+
22
+ In the snippet above,
23
+
24
+ 1. We defined a [listener](#listeners) that is active in any [context](#contexts).
25
+ 2. We received a message.
26
+ 3. Attentive matched the message to our listener and invoked the block.
10
27
 
11
- Add this line to your application's Gemfile:
28
+
29
+ <br/>
30
+ #### Optional Characters
31
+
32
+ You'll notice that we listened for `"hi"` but _heard_ `"hi!"`. Attentive treats punctuation and emojis as optional; but we can make them required by putting them in the listener:
12
33
 
13
34
  ```ruby
14
- gem 'attentive'
35
+ listen_for "hi!", context: { in: :any } do
36
+ puts "nice to meet you!"
37
+ end
38
+
39
+ hear! "hi" # => nothing happened, the listener is expecting the exclamation mark
15
40
  ```
16
41
 
17
- And then execute:
42
+ > It's best to leave all but the most necessary punctuation out of listeners.
18
43
 
19
- $ bundle
20
44
 
21
- Or install it yourself as:
45
+ <br/>
46
+ #### Contractions and Abbreviations
22
47
 
23
- $ gem install attentive
48
+ Attentive understands contractions and abbreviations and can match those:
24
49
 
50
+ ```ruby
51
+ listen_for "hi", context: { in: :any } do
52
+ puts "nice to meet you!"
53
+ end
25
54
 
55
+ hear! "hello!" # => "nice to meet you!"
56
+
57
+ listen_for "what is for lunch", context: { in: :any } do
58
+ puts "HAMBURGERS!"
59
+ end
60
+
61
+ hear! "what's for lunch?" # => "HAMBURGERS!"
62
+ ```
63
+
64
+ > Although you _can_ use contractions and abbreviations in listeners, it's a good habit not to. Attentive will not let you define listeners that use ambiguous contractions like `"where's"` (`"where's"` might be a contraction for `"where is"`, `"where does"`, or `"where has"`, or `"where was"`).
26
65
 
27
- ## Usage
66
+
67
+ <br/>
68
+ #### Listeners
69
+
70
+ Listeners are defined with three things:
71
+
72
+ 1. One or more phrases
73
+ 2. A set of [contexts](#contexts) where they're active
74
+ 3. A block to be invoked when the listener is matched
75
+
76
+ Here's an example of a listener that matches more than one phrase:
28
77
 
29
78
  ```ruby
30
- include Attentive
79
+ listen_for "what is for lunch",
80
+ "what is for lunch {{date:core.date.relative.future}}",
81
+ "what is for lunch on {{date:core.date}}",
82
+ "show me the menu for {{date:core.date.relative.future}}",
83
+ "show me the menu for {{date:core.date}}" do
84
+ # ...
85
+ end
86
+ ```
87
+
88
+ (In the example above, the phrases `{{date:core.date.relative.future}}` and `{{date:core.date}}` are [entities](#entities): which we'll cover in a minute.)
89
+
90
+
91
+ <br/>
92
+ #### Contexts
93
+
94
+ A listener can require that messages be heard in a certain context in order to be matched or it can ignore messages if they are heard in certain contexts.
31
95
 
32
- listen_for "hi there!" do
33
- puts "heard greeting"
96
+ The following is a listener that will only match messages heard in the "#general" channel and only then if the conversation is not "serious".
97
+
98
+ ```ruby
99
+ listen_for "ouch", context: { in: %i{general}, not_in: %i{serious} } do
100
+ puts "On a scale of 1 to 10, how would you rate your pain?"
34
101
  end
35
102
 
36
- hear "hi, there :wink:"
103
+ hear! "ouch" # => message has no context, listener isn't triggered
104
+ hear! "ouch", contexts: %i{general} # => "On a scale of 1 to 10..."
105
+ hear! "ouch", contexts: %i{general serious} # => listener ignores "serious" messages
37
106
  ```
38
107
 
108
+ If you don't specify context requirements for listeners, Attentive requires `conversation` and prohibits `quotation` by default:
109
+
110
+ ```ruby
111
+ # These two are the same:
112
+ listen_for "ouch"
113
+ listen_for "ouch", context: { in: %i{conversation}, not_in: %i{quotation} }
114
+ ```
115
+
116
+
117
+
118
+ <br/>
119
+ #### Entities
120
+
121
+ Entities allow Attentive to match **_concepts_** rather than specific words.
122
+
123
+ There are built-in entities like `core.date`, `core.number`, and `core.email` for recognizing dates, numbers, and email addresses (see [Core Entities](https://github.com/houston/attentive/wiki/Core-Entities) for a complete list); but you can also define entities for domain-specific concepts. For example:
124
+
125
+ ```ruby
126
+ Attentive::Entity.define "deweys.menu.beers",
127
+ "Bell's Oberon",
128
+ "Rogue Dead Guy Ale",
129
+ "Schalfly Dry Hopped IPA",
130
+ "4 Hands Contact High",
131
+ "Scrimshaw Pilsner"
132
+ ```
133
+
134
+ Now we can take drink orders:
135
+
136
+ ```ruby
137
+ listen_for "I will have a pint of the {{deweys.menu.beers}}" do
138
+ puts "Good choice"
139
+ end
140
+ ```
141
+
142
+ > It is a good idea to namespace entities (i.e. `deweys.menu.beers`). Attentive's convention is to treat namespaces as a taxonomy for concepts.
143
+
144
+
145
+ <br/>
146
+ #### Regular Expressions
147
+
148
+ As useful as enumerations are, entities can also be defined with regular expressions and with a block that converts the matched part of the message to a more useful value:
149
+
150
+ ```ruby
151
+ # Usernames can be up to 21 characters long.
152
+ # They can contain lowercase letters a to z
153
+ # (without accents), and numbers 0 to 9.
154
+
155
+ Attentive::Entity.define "slack.user", %q{(?<username>[a-z0-9]{1,21})} do |match|
156
+ Slack::User.find match["username"]
157
+ end
158
+ ```
159
+
160
+ > Whenever possible, though, prefer composing entities to using regular expressions.
161
+ > For example:
162
+ > ```ruby
163
+ Attentive::Entity.define "core.date.relative.future",
164
+ "next {{core.date.wday}}"
165
+ ```
166
+ > is better than:
167
+ > ```ruby
168
+ Attentive::Entity.define "core.date.relative.future",
169
+ "next (?<weekday>(:sun|mon|tues|wednes|thurs|fri|satur)day)"
170
+ ```
171
+
172
+
173
+
174
+ <br/>
175
+ ## Installation
176
+
177
+ Add this line to your application's Gemfile:
178
+
179
+ ```ruby
180
+ gem 'attentive'
181
+ ```
182
+
183
+ And then execute:
184
+
185
+ $ bundle
186
+
187
+ Or install it yourself as:
188
+
189
+ $ gem install attentive
190
+
39
191
 
40
192
 
193
+ <br/>
41
194
  ## Development
42
195
 
43
196
  After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
@@ -46,12 +199,14 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
46
199
 
47
200
 
48
201
 
202
+ <br/>
49
203
  ## Contributing
50
204
 
51
- Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/attentive.
205
+ Bug reports and pull requests are welcome on GitHub at https://github.com/houston/attentive.
52
206
 
53
207
 
54
208
 
209
+ <br/>
55
210
  ## License
56
211
 
57
212
  The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
@@ -7,6 +7,14 @@ module Attentive
7
7
  # Default configuration
8
8
  self.invocations = ["@me".freeze]
9
9
 
10
+ # Default contexts that listeners will require
11
+ # a message to be heard in.
12
+ self.default_required_contexts = %i{conversation}
13
+
14
+ # Default contexts in which listeners will ignore messages.
15
+ self.default_prohibited_contexts = %i{quotation}
16
+
17
+
10
18
 
11
19
  # Attentive DSL
12
20
 
@@ -18,11 +26,22 @@ module Attentive
18
26
  listeners.listen_for(*args, &block)
19
27
  end
20
28
 
29
+ # Matches a message against all listeners
30
+ # and returns an array of matches
21
31
  def hear(message, params={})
22
32
  message = Attentive::Message.new(message, params) unless message.is_a?(Attentive::Message)
23
33
  listeners.hear message
24
34
  end
25
35
 
36
+ # Matches a message against all listeners
37
+ # and invokes the first listener that mathes
38
+ def hear!(message, params={})
39
+ hear(message, params).each do |match|
40
+ match.listener.call(match)
41
+ return
42
+ end
43
+ end
44
+
26
45
  end
27
46
 
28
47
  require "attentive/listener_collection"
@@ -1,3 +1,3 @@
1
1
  module Attentive
2
- ABBREVIATIONS = {"bye"=>"goodbye", "gonna"=>"going to", "hi"=>"hello", "ol'"=>"old", "'sup"=>"what is up", "thanks"=>"thank you", "wanna"=>"want to", "mon"=>"monday", "tue"=>"tuesday", "tues"=>"tuesday", "wed"=>"wednesday", "thu"=>"thursday", "thur"=>"thursday", "thurs"=>"thursday", "fri"=>"friday", "sat"=>"saturday", "sun"=>"sunday"}.freeze
2
+ ABBREVIATIONS = {"bye"=>"goodbye", "gonna"=>"going to", "hi"=>"hello", "ol'"=>"old", "'sup"=>"what is up", "thanks"=>"thank you", "wanna"=>"want to", "mon"=>"monday", "tue"=>"tuesday", "tues"=>"tuesday", "wed"=>"wednesday", "thu"=>"thursday", "thur"=>"thursday", "thurs"=>"thursday", "fri"=>"friday", "sat"=>"saturday", "sun"=>"sunday", "jan"=>"january", "feb"=>"february", "mar"=>"march", "apr"=>"april", "jun"=>"june", "jul"=>"july", "aug"=>"august", "sep"=>"september", "sept"=>"september", "oct"=>"october", "nov"=>"november", "dec"=>"december"}.freeze
3
3
  end
@@ -9,10 +9,9 @@ module Attentive
9
9
  attr_accessor :entities
10
10
 
11
11
  def define(entity_name, *entities)
12
- entity_klass = Class.new(Attentive::CompositeEntity)
13
- entity_klass.token_name = entity_name
14
- entity_klass.entities = entities.map { |entity| Entity[entity] }
15
- Entity.register! entity_name, entity_klass
12
+ create! entity_name do |entity_klass|
13
+ entity_klass.entities = entities.map { |entity| Entity[entity] }
14
+ end
16
15
  end
17
16
  end
18
17
 
@@ -23,7 +22,7 @@ module Attentive
23
22
 
24
23
  def matches?(cursor)
25
24
  entities.each do |entity|
26
- match = entity.matches?(cursor.dup)
25
+ match = entity.matches?(cursor)
27
26
  return match if match
28
27
  end
29
28
  false
@@ -2,6 +2,8 @@ module Attentive
2
2
  module Config
3
3
 
4
4
  attr_reader :invocations
5
+ attr_accessor :default_required_contexts
6
+ attr_accessor :default_prohibited_contexts
5
7
 
6
8
  def invocations=(*values)
7
9
  @invocations = values.flatten
@@ -8,21 +8,53 @@ module Attentive
8
8
  end
9
9
 
10
10
  def peek
11
- tokens[pos]
11
+ tokens[pos] || EOF
12
12
  end
13
13
 
14
14
  def pop
15
- @pos += 1
16
- tokens[pos - 1]
15
+ peek.tap do
16
+ advance
17
+ end
18
+ end
19
+
20
+ def new_from_here
21
+ self.class.new(tokens[pos..-1])
17
22
  end
18
23
 
19
24
  def to_s
20
25
  tokens[pos..-1].join
21
26
  end
22
27
 
28
+ def inspect
29
+ "|#{(tokens[0...pos] || []).join}\e[7m#{tokens[pos]}\e[0m#{(tokens[(pos + 1)..-1] || []).join}|"
30
+ end
31
+
23
32
  def offset
24
33
  peek.pos
25
34
  end
26
35
 
36
+ def advance(n=1)
37
+ @pos += n
38
+ end
39
+
40
+ def eof?
41
+ @pos == @tokens.length
42
+ end
43
+
44
+
45
+
46
+ private
47
+
48
+ class Eof
49
+ def whitespace?
50
+ false
51
+ end
52
+
53
+ def eof?
54
+ true
55
+ end
56
+ end
57
+ EOF = Eof.new.freeze
58
+
27
59
  end
28
60
  end
@@ -0,0 +1,3 @@
1
+ require "attentive/entities/core/number"
2
+ require "attentive/entities/core/date"
3
+ require "attentive/entities/core/email"
@@ -0,0 +1,6 @@
1
+ require "attentive/entities/core/date/month"
2
+ require "attentive/entities/core/date/wday"
3
+ require "attentive/entities/core/date/relative"
4
+
5
+ Attentive::CompositeEntity.define "core.date",
6
+ "core.date.relative"
@@ -0,0 +1,7 @@
1
+ require "attentive/entity"
2
+ require "date"
3
+
4
+ month_names = Date::MONTHNAMES.compact.map(&:downcase)
5
+ Attentive::Entity.define "core.date.month", *month_names do |match|
6
+ month_names.index(match.phrase) + 1
7
+ end
@@ -0,0 +1,6 @@
1
+ require "attentive/entities/core/date/relative/past"
2
+ require "attentive/entities/core/date/relative/future"
3
+
4
+ Attentive::CompositeEntity.define "core.date.relative",
5
+ "core.date.relative.future",
6
+ "core.date.relative.past"
@@ -0,0 +1,27 @@
1
+ require "attentive/entity"
2
+ require "date"
3
+
4
+ Attentive::Entity.define "core.date.relative.future",
5
+ "today",
6
+ "tomorrow",
7
+ "{{core.date.wday}}",
8
+ "next {{core.date.wday}}" do |match|
9
+
10
+ today = Date.today
11
+
12
+ if match.matched?("core.date.wday")
13
+ wday = match["core.date.wday"]
14
+ days_until_wday = wday - today.wday
15
+ days_until_wday += 7 if days_until_wday < 0
16
+ date = today + days_until_wday
17
+
18
+ date += 7 if match.to_s.start_with?("next")
19
+ date
20
+ else
21
+ case match.to_s
22
+ when "today" then today
23
+ when "tomorrow" then today + 1
24
+ else raise NotImplementedError, "Unrecognized match: #{match.to_s}"
25
+ end
26
+ end
27
+ end
@@ -0,0 +1,24 @@
1
+ require "attentive/entity"
2
+ require "date"
3
+
4
+ Attentive::Entity.define "core.date.relative.past",
5
+ "today",
6
+ "yesterday",
7
+ "{{core.date.wday}}",
8
+ "last {{core.date.wday}}" do |match|
9
+
10
+ today = Date.today
11
+
12
+ if match.matched?("core.date.wday")
13
+ wday = match["core.date.wday"]
14
+ days_since_wday = today.wday - wday
15
+ days_since_wday += 7 if days_since_wday < 0
16
+ today - days_since_wday
17
+ else
18
+ case match.to_s
19
+ when "today" then today
20
+ when "yesterday" then today - 1
21
+ else raise NotImplementedError, "Unrecognized match: #{match.to_s}"
22
+ end
23
+ end
24
+ end
@@ -0,0 +1,7 @@
1
+ require "attentive/entity"
2
+ require "date"
3
+
4
+ day_names = Date::DAYNAMES.map(&:downcase)
5
+ Attentive::Entity.define "core.date.wday", *day_names do |match|
6
+ day_names.index(match.phrase)
7
+ end
@@ -0,0 +1,8 @@
1
+ require "attentive/entity"
2
+
3
+ # Email regex asserts that there are no @ symbols or whitespaces in either the
4
+ # localpart or the domain, and that there is a single @ symbol separating the
5
+ # localpart and the domain.
6
+ Attentive::Entity.define "core.email", %q{(?<email>[^@\s]+@[^@\s]+)} do |match|
7
+ match["email"]
8
+ end
@@ -0,0 +1,8 @@
1
+ require "attentive/entities/core/number/integer"
2
+ require "attentive/entities/core/number/float"
3
+ require "attentive/entities/core/number/positive"
4
+ require "attentive/entities/core/number/negative"
5
+
6
+ Attentive::CompositeEntity.define "core.number",
7
+ "core.number.float",
8
+ "core.number.integer"
@@ -0,0 +1,6 @@
1
+ require "attentive/entities/core/number/float/positive"
2
+ require "attentive/entities/core/number/float/negative"
3
+
4
+ Attentive::CompositeEntity.define "core.number.float",
5
+ "core.number.float.positive",
6
+ "core.number.float.negative"
@@ -0,0 +1,6 @@
1
+ require "attentive/entity"
2
+ require "bigdecimal"
3
+
4
+ Attentive::Entity.define "core.number.float.negative", %q{(?<float>\-[\d,]+\.\d+)} do |match|
5
+ BigDecimal.new(match["float"].gsub(",", ""))
6
+ end
@@ -0,0 +1,6 @@
1
+ require "attentive/entity"
2
+ require "bigdecimal"
3
+
4
+ Attentive::Entity.define "core.number.float.positive", %q{(?<float>[\d,]+\.\d+)} do |match|
5
+ BigDecimal.new(match["float"].gsub(",", ""))
6
+ end
@@ -0,0 +1,6 @@
1
+ require "attentive/entities/core/number/integer/positive"
2
+ require "attentive/entities/core/number/integer/negative"
3
+
4
+ Attentive::CompositeEntity.define "core.number.integer",
5
+ "core.number.integer.positive",
6
+ "core.number.integer.negative"
@@ -0,0 +1,5 @@
1
+ require "attentive/entity"
2
+
3
+ Attentive::Entity.define "core.number.integer.negative", %q{(?<integer>\-\d+)} do |match|
4
+ match["integer"].gsub(",", "").to_i
5
+ end
@@ -0,0 +1,5 @@
1
+ require "attentive/entity"
2
+
3
+ Attentive::Entity.define "core.number.integer.positive", %q{(?<integer>[\d,]+)} do |match|
4
+ match["integer"].gsub(",", "").to_i
5
+ end
@@ -0,0 +1,6 @@
1
+ require "attentive/entities/core/number/integer/negative"
2
+ require "attentive/entities/core/number/float/negative"
3
+
4
+ Attentive::CompositeEntity.define "core.number.negative",
5
+ "core.number.float.negative",
6
+ "core.number.integer.negative"
@@ -0,0 +1,6 @@
1
+ require "attentive/entities/core/number/integer/positive"
2
+ require "attentive/entities/core/number/float/positive"
3
+
4
+ Attentive::CompositeEntity.define "core.number.positive",
5
+ "core.number.float.positive",
6
+ "core.number.integer.positive"
@@ -12,25 +12,44 @@ module Attentive
12
12
  attr_accessor :token_name
13
13
 
14
14
  def [](entity_name)
15
+ entity_name = entity_name.to_sym
15
16
  @entities.fetch(entity_name)
16
17
  rescue KeyError
17
18
  raise Attentive::UndefinedEntityError.new("Undefined Entity #{entity_name.inspect}")
18
19
  end
19
20
 
20
21
  def define(entity_name, *phrases, &block)
21
- entity_klass = Class.new(Attentive::Entity)
22
- entity_klass.token_name = entity_name
23
- entity_klass.phrases = phrases.map do |phrase|
24
- Attentive::Tokenizer.tokenize(phrase, entities: true, regexps: true, ambiguous: false)
22
+ create! entity_name do |entity_klass|
23
+ entity_klass.phrases = phrases.map do |phrase|
24
+ Attentive::Tokenizer.tokenize(phrase, entities: true, regexps: true, ambiguous: false)
25
+ end
26
+ entity_klass.send :define_method, :_value_from_match, &block if block_given?
25
27
  end
26
- entity_klass.send :define_method, :_value_from_match, &block
27
- register! entity_name, entity_klass
28
+ end
29
+
30
+ def undefine(entity_name)
31
+ entity_symbol = entity_name.to_sym
32
+ unregister! entity_symbol
33
+ end
34
+
35
+ protected
36
+
37
+ def create!(entity_name)
38
+ entity_symbol = entity_name.to_sym
39
+ entity_klass = Class.new(self)
40
+ entity_klass.token_name = entity_symbol
41
+ yield entity_klass
42
+ Entity.register! entity_symbol, entity_klass
28
43
  end
29
44
 
30
45
  def register!(entity_name, entity_klass)
31
- # TODO: raise already registered error
46
+ raise ArgumentError, "Entity #{entity_name.inspect} has already been defined" if @entities.key?(entity_name)
32
47
  @entities[entity_name] = entity_klass
33
48
  end
49
+
50
+ def unregister!(entity_name)
51
+ @entities.delete entity_name
52
+ end
34
53
  end
35
54
 
36
55
 
@@ -46,7 +65,11 @@ module Attentive
46
65
  end
47
66
 
48
67
  def to_s
49
- "{{#{variable_name}:#{self.class.token_name}}}"
68
+ if variable_name.to_s == self.class.token_name.to_s
69
+ "{{#{self.class.token_name}}}"
70
+ else
71
+ "{{#{variable_name}:#{self.class.token_name}}}"
72
+ end
50
73
  end
51
74
 
52
75
  def entity?
@@ -55,15 +78,19 @@ module Attentive
55
78
 
56
79
  def matches?(cursor)
57
80
  self.class.phrases.each do |phrase|
58
- cursor_copy = cursor.dup
81
+ cursor_copy = cursor.new_from_here
59
82
  match = Attentive::Matcher.new(phrase, cursor_copy).match!
60
83
  if match
61
- cursor.instance_variable_set :@pos, cursor_copy.pos
84
+ cursor.advance cursor_copy.pos
62
85
  return { variable_name => _value_from_match(match) }
63
86
  end
64
87
  end
65
88
  false
66
89
  end
67
90
 
91
+ def _value_from_match(match)
92
+ match.to_s
93
+ end
94
+
68
95
  end
69
96
  end
@@ -8,10 +8,10 @@ module Attentive
8
8
 
9
9
  def initialize(listeners, phrases, options, callback)
10
10
  context_options = options.fetch(:context, {})
11
- @required_contexts = context_options.fetch(:in, %i{conversation})
11
+ @required_contexts = context_options.fetch(:in, Attentive.default_required_contexts)
12
12
  @required_contexts = [] if @required_contexts == :any
13
13
  @required_contexts = Set[*@required_contexts]
14
- @prohibited_contexts = context_options.fetch(:not_in, %i{quotation})
14
+ @prohibited_contexts = context_options.fetch(:not_in, Attentive.default_prohibited_contexts)
15
15
  @prohibited_contexts = Set[*@prohibited_contexts]
16
16
 
17
17
  @listeners = listeners
@@ -2,16 +2,21 @@ require "attentive/match"
2
2
 
3
3
  module Attentive
4
4
  class Matcher
5
- attr_reader :phrase, :cursor, :pos
5
+ attr_reader :phrase, :message, :cursor
6
6
 
7
- def initialize(phrase, cursor, params={})
7
+ def initialize(phrase, message, params={})
8
8
  @phrase = phrase
9
- @cursor = cursor
10
- @pos = params.fetch(:pos, 0)
9
+ @cursor = Cursor.new(phrase, params.fetch(:pos, 0))
10
+ @message = message
11
11
  @match_params = params.each_with_object({}) { |(key, value), new_hash| new_hash[key] = value if %i{listener message}.member?(key) }
12
- @pos += 1 while phrase[pos] && phrase[pos].whitespace?
13
12
  @match_data = {}
14
13
  @state = :matching
14
+
15
+ cursor.pop while cursor.peek.whitespace?
16
+ end
17
+
18
+ def pos
19
+ cursor.pos
15
20
  end
16
21
 
17
22
  def matching?
@@ -23,57 +28,41 @@ module Attentive
23
28
  end
24
29
 
25
30
  def match!
26
- while token = cursor.peek
31
+ until (token = message.peek).eof?
27
32
  if token.ambiguous?
28
- unless match_subphrase!(token.possibilities)
33
+ unless match_any!(token.possibilities)
29
34
  @state = :mismatch
30
35
  break
31
36
  end
32
- @pos += 1 while phrase[pos] && phrase[pos].whitespace?
37
+ cursor.pop while cursor.peek.whitespace?
33
38
 
34
- elsif match_data = phrase[pos].matches?(cursor)
35
- if match_data.is_a?(MatchData)
36
- new_character_index = cursor.offset + match_data.to_s.length
37
- @match_data.merge! Hash[match_data.names.zip(match_data.captures)]
38
-
39
- # Advance the cursor to the first token after the regexp match
40
- cursor_pos = cursor.tokens.index { |token| token.pos >= new_character_index }
41
- cursor_pos = cursor.tokens.length unless cursor_pos
42
- cursor.instance_variable_set :@pos, cursor_pos
43
- @pos += 1
44
- else
45
- @match_data.merge!(match_data) unless match_data == true
46
- @pos += 1
47
- end
48
- @pos += 1 while phrase[pos] && phrase[pos].whitespace?
39
+ elsif match_data = cursor.peek.matches?(message)
40
+ @match_data.merge!(match_data) unless match_data == true
41
+ cursor.pop
42
+ cursor.pop while cursor.peek.whitespace?
49
43
  @state = :found
50
44
 
51
-
52
45
  # -> This is the one spot where we instantiate a Match
53
- return Attentive::Match.new(phrase, @match_params.merge(match_data: @match_data)) if pos == phrase.length
46
+ return Attentive::Match.new(phrase, @match_params.merge(match_data: @match_data)) if cursor.eof?
54
47
 
55
48
  elsif !token.skippable?
56
49
  @state = :mismatch
57
50
  break
58
51
  end
59
52
 
60
- cursor.pop
61
- break unless cursor.peek
62
- while cursor.peek.whitespace?
63
- cursor.pop
64
- break unless cursor.peek
65
- end
53
+ message.pop
54
+ message.pop while message.peek.whitespace?
66
55
  end
67
56
 
68
57
  nil
69
58
  end
70
59
 
71
- def match_subphrase!(subphrases)
72
- subphrases.each do |subphrase|
73
- matcher = Matcher.new(phrase, Cursor.new(subphrase), pos: pos)
60
+ def match_any!(messages)
61
+ messages.each do |message|
62
+ matcher = Matcher.new(phrase[pos..-1], Cursor.new(message))
74
63
  matcher.match!
75
64
  unless matcher.mismatch?
76
- @pos = matcher.pos
65
+ cursor.advance matcher.pos
77
66
  return true
78
67
  end
79
68
  end
@@ -26,10 +26,18 @@ module Attentive
26
26
  false
27
27
  end
28
28
 
29
+ def eof?
30
+ false
31
+ end
32
+
29
33
  def matches?(cursor)
30
34
  self == cursor.peek
31
35
  end
32
36
 
37
+ def inspect
38
+ "<#{self.class.name ? self.class.name.split("::").last : "Entity"} #{to_s.inspect}>"
39
+ end
40
+
33
41
  end
34
42
 
35
43
 
@@ -50,6 +58,10 @@ module Attentive
50
58
  string
51
59
  end
52
60
 
61
+ def length
62
+ string.length
63
+ end
64
+
53
65
  def ==(other)
54
66
  self.class == other.class && self.string == other.string
55
67
  end
@@ -8,141 +8,187 @@ require "attentive/errors"
8
8
 
9
9
  module Attentive
10
10
  class Tokenizer
11
- extend Attentive::Tokens
12
-
13
- # Splits apart words and punctuation,
14
- # treats apostrophes and dashes as a word-characters,
15
- # trims each fragment of whitepsace
16
- # SPLITTER = /\s*([\w'-]+)\s*/.freeze
17
- SPLITTER = /(\n|{{|}}|\s+|\.{2,}|[^\s\w'@-])/.freeze
18
- PUNCTUATION = /^\W+$/.freeze
19
- WHITESPACE = /^\s+$/.freeze
20
- ENTITY_START = "{{".freeze
21
- ENTITY_END = "}}".freeze
22
- REGEXP_START = "(".freeze
23
- REGEXP_END = ")".freeze
24
- REGEXP_ESCAPE = "\\".freeze
11
+ include Attentive::Tokens
25
12
 
13
+ attr_reader :message, :chars, :options
26
14
 
27
- def self.split(message)
28
- Attentive::Text.normalize(message).split(SPLITTER).reject(&:empty?)
15
+ def self.tokenize(message, options={})
16
+ self.new(message, options).tokenize
29
17
  end
30
18
 
31
19
 
32
- def self.tokenize(message, options={})
33
- match_entities = options.fetch(:entities, false)
34
- match_regexps = options.fetch(:regexps, false)
35
- fail_if_ambiguous = !options.fetch(:ambiguous, true)
36
- strings = split(message)
37
- tokens = []
20
+
21
+ def initialize(message, options={})
22
+ @message = Attentive::Text.normalize(message)
23
+ @chars = self.message.each_char.to_a
24
+ @options = options
25
+ end
26
+
27
+
28
+
29
+ def tokenize
38
30
  i = 0
39
- pos = 0
40
- while i < strings.length
41
- string = strings[i]
42
- case string
43
- when ""
44
- # do nothing
45
-
46
- when WHITESPACE
47
- tokens << whitespace(string, pos: pos)
48
-
49
- when ":"
50
- if strings[i + 2] == ":"
51
- tokens << emoji(strings[i + 1], pos: pos)
52
- pos += strings[i + 1].length + 1
53
- i += 2
54
- else
55
- tokens << punctuation(":", pos: pos)
56
- end
31
+ tokens = []
32
+ while i < chars.length
33
+ char = chars[i]
57
34
 
58
- when ENTITY_START
59
- if match_entities
60
- j = i + 1
61
- found_entity = false
62
- while j < strings.length
63
- if strings[j] == ENTITY_END
64
- entity = strings[(i + 1)...j] # e.g. ["variable-name", ":" "entity-type"]
65
- tokens << entity(*entity.join.split(":").reverse, pos: pos)
66
- i = j + 1
67
- pos += entity.join.length + 4
68
- found_entity = true
69
- break
70
- end
71
- j += 1
72
- end
73
- next if found_entity
74
- end
75
- tokens << punctuation(ENTITY_START, pos: pos)
76
-
77
- when REGEXP_START
78
- if match_regexps && strings[i + 1] == "?"
79
- j = i + 2
80
- found_regexp = false
81
- parens = 1
82
- inside_square_bracket = false
83
- while j < strings.length
84
- if strings[j] == "[" && strings[j - 1] != REGEXP_ESCAPE
85
- inside_square_bracket = true
86
- elsif strings[j] == "]" && strings[j - 1] != REGEXP_ESCAPE
87
- inside_square_bracket = false
88
- end
89
-
90
- unless inside_square_bracket
91
- if strings[j] == REGEXP_START && strings[j - 1] != REGEXP_ESCAPE
92
- parens += 1
93
- elsif strings[j] == REGEXP_END && strings[j - 1] != REGEXP_ESCAPE
94
- parens -= 1
95
- end
96
-
97
- if parens == 0
98
- tokens << regexp(strings[i..j].join, pos: pos)
99
- pos += strings[i..j].join.length + 2
100
- i = j + 1
101
- found_regexp = true
102
- break
103
- end
104
- end
105
- j += 1
106
- end
107
- next if found_regexp
108
- end
109
- tokens << punctuation(REGEXP_START, pos: pos)
35
+ if EMOJI_START === char && string = match_emoji_at(i)
36
+ tokens << emoji(string, pos: i)
37
+ i += string.length + 2
38
+
39
+ elsif ENTITY_START === char && string = match_entity_at(i)
40
+ tokens << entity(*string.split(":").reverse, pos: i)
41
+ i += string.length + 4
42
+
43
+ elsif REGEXP_START === char && string = match_regexp_at(i)
44
+ tokens << regexp(string, pos: i)
45
+ i += string.length
110
46
 
111
- when PUNCTUATION
112
- tokens << punctuation(string, pos: pos)
47
+ elsif WHITESPACE === char && string = match_whitespace_at(i)
48
+ tokens << whitespace(string, pos: i)
49
+ i += string.length
113
50
 
114
- when *Attentive.invocations
115
- tokens << invocation(string, pos: pos)
51
+ elsif NUMBER_START === char && string = match_number_at(i)
52
+ tokens << word(string, pos: i)
53
+ i += string.length
54
+
55
+ elsif PUNCTUATION === char # =~ /\W/
56
+ tokens << punctuation(char, pos: i)
57
+ i += 1
116
58
 
117
59
  else
118
- if replace_with = Attentive::ABBREVIATIONS[string]
119
- tokens.concat tokenize(replace_with, options)
60
+ string = match_word_at(i)
61
+ if Attentive.invocations.member?(string)
62
+ tokens << invocation(string, pos: i)
63
+
64
+ elsif replace_with = Attentive::ABBREVIATIONS[string]
65
+ tokens.concat self.class.tokenize(replace_with, options)
120
66
 
121
67
  elsif expands_to = Attentive::CONTRACTIONS[string]
122
68
  possibilities = expands_to.map do |possibility|
123
- tokenize(possibility, options)
69
+ self.class.tokenize(possibility, options)
124
70
  end
125
71
 
126
72
  if possibilities.length == 1
127
73
  tokens.concat possibilities[0]
128
74
  else
129
- tokens << any_of(possibilities, pos: pos)
75
+ tokens << any_of(string, possibilities, pos: i)
130
76
  end
77
+
131
78
  else
132
- tokens << word(string, pos: pos)
79
+ tokens << word(string, pos: i)
133
80
  end
81
+ i += string.length
134
82
  end
83
+ end
84
+
85
+ fail_if_ambiguous!(message, tokens) if fail_if_ambiguous?
86
+ Attentive::Phrase.new(tokens)
87
+ end
88
+
89
+
135
90
 
136
- i += 1
137
- pos += string.length
91
+ def match_emoji_at(i)
92
+ emoji = ""
93
+ while (i += 1) < chars.length
94
+ return if_present?(emoji) if EMOJI_END === chars[i]
95
+ return false if WHITESPACE === chars[i]
96
+ emoji << chars[i]
138
97
  end
98
+ false
99
+ end
139
100
 
140
- fail_if_ambiguous!(message, tokens) if fail_if_ambiguous
101
+ def match_entity_at(i)
102
+ return false unless match_entities?
103
+ return false unless chars[i += 1] == "{"
104
+ entity = ""
105
+ while (i += 1) < chars.length
106
+ return if_present?(entity) if ["}", "}"] == chars[i, 2]
107
+ return false unless ENTITY === chars[i]
108
+ entity << chars[i]
109
+ end
110
+ false
111
+ end
141
112
 
142
- Attentive::Phrase.new(tokens)
113
+ def match_regexp_at(i)
114
+ return false unless match_regexps?
115
+ return false unless chars[i += 1] == "?"
116
+ regexp = "(?"
117
+ parens = 1
118
+ inside_square_bracket = false
119
+ while (i += 1) < chars.length
120
+ regexp << chars[i]
121
+ next if chars[i - 1] == "\\"
122
+ inside_square_bracket = true if chars[i] == "["
123
+ inside_square_bracket = false if chars[i] == "]"
124
+ next if inside_square_bracket
125
+ parens += 1 if chars[i] == "("
126
+ parens -= 1 if chars[i] == ")"
127
+ return if_present?(regexp) if parens == 0
128
+ end
129
+ false
130
+ end
131
+
132
+ def match_whitespace_at(i)
133
+ whitespace = chars[i]
134
+ while (i += 1) < chars.length
135
+ break unless WHITESPACE === chars[i]
136
+ whitespace << chars[i]
137
+ end
138
+ whitespace
139
+ end
140
+
141
+ def match_number_at(i)
142
+ return false if CONDITIONAL_NUMBER_START === chars[i] && !(NUMBER === chars[i + 1])
143
+ number = chars[i]
144
+ while (i += 1) < chars.length
145
+ break unless NUMBER === chars[i] || (CONDITIONAL_NUMBER === chars[i] && NUMBER === chars[i + 1])
146
+ number << chars[i]
147
+ end
148
+ number
149
+ end
150
+
151
+ def match_word_at(i)
152
+ word = chars[i]
153
+ while (i += 1) < chars.length
154
+ break unless WORD === chars[i]
155
+ word << chars[i]
156
+ end
157
+ word
158
+ end
159
+
160
+ def if_present?(string)
161
+ string.empty? ? false : string
162
+ end
163
+
164
+
165
+
166
+ def match_entities?
167
+ options.fetch(:entities, false)
143
168
  end
144
169
 
145
- def self.fail_if_ambiguous!(phrase, tokens)
170
+ def match_regexps?
171
+ options.fetch(:regexps, false)
172
+ end
173
+
174
+ def fail_if_ambiguous?
175
+ !options.fetch(:ambiguous, true)
176
+ end
177
+
178
+ WHITESPACE = /\s/.freeze
179
+ PUNCTUATION = /[^\s\w'@-]/.freeze
180
+ EMOJI_START = ":".freeze
181
+ EMOJI_END = ":".freeze
182
+ ENTITY_START = "{".freeze
183
+ ENTITY = /[a-z0-9\.\-:]/.freeze
184
+ REGEXP_START = "(".freeze
185
+ NUMBER_START = /[\d\.\-]/.freeze
186
+ CONDITIONAL_NUMBER_START = /[\.\-]/.freeze
187
+ NUMBER = /\d/.freeze
188
+ CONDITIONAL_NUMBER = /[\.,]/.freeze
189
+ WORD = /[\w'\-@]/.freeze
190
+
191
+ def fail_if_ambiguous!(phrase, tokens)
146
192
  ambiguous_token = tokens.find(&:ambiguous?)
147
193
  return unless ambiguous_token
148
194
 
@@ -159,5 +205,4 @@ end
159
205
  require "attentive/entity"
160
206
  require "attentive/composite_entity"
161
207
 
162
- require "attentive/entities/integer"
163
- require "attentive/entities/relative_date"
208
+ require "attentive/entities/core"
@@ -1,8 +1,8 @@
1
1
  module Attentive
2
2
  module Tokens
3
3
 
4
- def any_of(possibilities, pos: nil)
5
- Attentive::Tokens::AnyOf.new possibilities, pos
4
+ def any_of(string, possibilities, pos: nil)
5
+ Attentive::Tokens::AnyOf.new string, possibilities, pos
6
6
  end
7
7
 
8
8
  def emoji(string, pos: nil)
@@ -2,12 +2,12 @@ require "attentive/token"
2
2
 
3
3
  module Attentive
4
4
  module Tokens
5
- class AnyOf < Token
5
+ class AnyOf < StringToken
6
6
  attr_reader :possibilities
7
7
 
8
- def initialize(possibilities, pos)
8
+ def initialize(string, possibilities, pos)
9
+ super string, pos
9
10
  @possibilities = possibilities
10
- super pos
11
11
  end
12
12
 
13
13
  def ==(other)
@@ -15,11 +15,28 @@ module Attentive
15
15
  end
16
16
 
17
17
  def matches?(cursor)
18
- regexp.match(cursor.to_s)
18
+ # Compare the original, untokenized, message to the regular expression
19
+ match_data = regexp.match(cursor.to_s)
20
+ return false unless match_data
21
+
22
+ # Find the first token following the match
23
+ new_character_index = cursor.offset + match_data.to_s.length
24
+ cursor_pos = cursor.tokens.index { |token| token.pos >= new_character_index }
25
+ cursor_pos = cursor.tokens.length unless cursor_pos
26
+
27
+ # If the match ends in the middle of a token, treat it as a mismatch
28
+ match_end_token = cursor.tokens[cursor_pos - 1]
29
+ return false if match_end_token.pos + match_end_token.length > new_character_index
30
+
31
+ # Advance the cursor to the first token after the regexp match
32
+ cursor.advance cursor_pos - cursor.pos
33
+
34
+ # Return the MatchData as a hash
35
+ Hash[match_data.names.zip(match_data.captures)]
19
36
  end
20
37
 
21
38
  def to_s
22
- regexp.inspect[1...-1]
39
+ regexp.inspect
23
40
  end
24
41
 
25
42
  end
@@ -1,3 +1,3 @@
1
1
  module Attentive
2
- VERSION = "0.1.1"
2
+ VERSION = "0.2.0"
3
3
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: attentive
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Bob Lail
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2016-04-27 00:00:00.000000000 Z
11
+ date: 2016-05-15 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: thread_safe
@@ -158,8 +158,23 @@ files:
158
158
  - lib/attentive/config.rb
159
159
  - lib/attentive/contractions.rb
160
160
  - lib/attentive/cursor.rb
161
- - lib/attentive/entities/integer.rb
162
- - lib/attentive/entities/relative_date.rb
161
+ - lib/attentive/entities/core.rb
162
+ - lib/attentive/entities/core/date.rb
163
+ - lib/attentive/entities/core/date/month.rb
164
+ - lib/attentive/entities/core/date/relative.rb
165
+ - lib/attentive/entities/core/date/relative/future.rb
166
+ - lib/attentive/entities/core/date/relative/past.rb
167
+ - lib/attentive/entities/core/date/wday.rb
168
+ - lib/attentive/entities/core/email.rb
169
+ - lib/attentive/entities/core/number.rb
170
+ - lib/attentive/entities/core/number/float.rb
171
+ - lib/attentive/entities/core/number/float/negative.rb
172
+ - lib/attentive/entities/core/number/float/positive.rb
173
+ - lib/attentive/entities/core/number/integer.rb
174
+ - lib/attentive/entities/core/number/integer/negative.rb
175
+ - lib/attentive/entities/core/number/integer/positive.rb
176
+ - lib/attentive/entities/core/number/negative.rb
177
+ - lib/attentive/entities/core/number/positive.rb
163
178
  - lib/attentive/entity.rb
164
179
  - lib/attentive/errors.rb
165
180
  - lib/attentive/listener.rb
@@ -1,5 +0,0 @@
1
- require "attentive/entity"
2
-
3
- Attentive::Entity.define :integer, %q{(?<integer>\d+)} do |match|
4
- match["integer"].to_i
5
- end
@@ -1,44 +0,0 @@
1
- require "attentive/entity"
2
- require "date"
3
-
4
- weekday_regexp = "(?<weekday>sunday|monday|tuesday|wednesday|thursday|friday|saturday)"
5
- Attentive::Entity.define :"relative-date",
6
- "today",
7
- "tomorrow",
8
- "yesterday",
9
- weekday_regexp,
10
- "next #{weekday_regexp}",
11
- "last #{weekday_regexp}" do |match|
12
-
13
- today = Date.today
14
-
15
- next_wday = lambda do |wday|
16
- days_until_wday = wday - today.wday
17
- days_until_wday += 7 if days_until_wday < 0
18
- today + days_until_wday
19
- end
20
-
21
- if match.matched?("weekday")
22
- date = case weekday = match["weekday"]
23
- when /^sun/ then next_wday[0]
24
- when /^mon/ then next_wday[1]
25
- when /^tue/ then next_wday[2]
26
- when /^wed/ then next_wday[3]
27
- when /^thu/ then next_wday[4]
28
- when /^fri/ then next_wday[5]
29
- when /^sat/ then next_wday[6]
30
- else raise NotImplementedError, "Unrecognized weekday: #{weekday.inspect}"
31
- end
32
-
33
- date += 7 if match.to_s.start_with?("next")
34
- date -= 7 if match.to_s.start_with?("last")
35
- date
36
- else
37
- case match.to_s
38
- when "today" then today
39
- when "tomorrow" then today + 1
40
- when "yesterday" then today - 1
41
- else raise NotImplementedError, "Unrecognized match: #{match.to_s}"
42
- end
43
- end
44
- end