stealth 2.0.0.beta5 → 2.0.0.beta7

data/docs/dev-environment/tunnels.md

@@ -0,0 +1,18 @@
+# Tunnels
+
+When developing locally, message platforms require access to the Stealth server running on your machine in order to transmit user messages. 
+
+Here are some options you can use:
+
+### ngrok
+
+1. Download [ngrok](https://ngrok.com/download)
+2. Start your Stealth server as detailed in [Booting Up](booting-up.md#boot-your-bot).
+3. Open up an ngrok tunnel to your Stealth server and port (default 5000) like this: `ngrok http 5000`. ngrok will output a unique ngrok local tunnel URL to your machine.
+
+When you provide your local ngrok URL to a messaging service, you will have to add `/incoming/<service>`. For example:
+
+* `https://abc1234.ngrok.io/incoming/facebook`
+* `https://abc1234.ngrok.io/incoming/twilio`
+
+More details on service specific settings can be found on the GitHub page for each service gem.

data/docs/flows/flowmap.md

@@ -0,0 +1,40 @@
+# FlowMap
+
+The `FlowMap` is the file that contains your flow and state declarations. It's stored in `config/flow_map.rb` and will be generated by Stealth when you instantiate your bot.
+
+Here is a `FlowMap` similar to the one that is generated for a new bot:
+
+```ruby
+class FlowMap
+
+  include Stealth::Flow
+
+  flow :hello do
+    state :say_hello
+    state :get_hello_response, fails_to: :say_hello
+  end
+
+  flow :goodbye do
+    state :say_goodbye
+  end
+
+  flow :interrupt do
+    state :say_interrupted
+  end
+
+  flow :unrecognized_message do
+    state :handle_unrecognized_message
+  end
+
+  flow :catch_all do
+    state :level1
+  end
+
+end
+```
+
+In this example, we've declared five flows: `hello`, `goodbye`, `interrupt`, `unrecognized_message`, and `catch_all`. These are the default flows that are generated for you when you create a new bot.
+
+Each flow consists of an arbitrary number of states. All of the above flows only have a single state, but the `hello` flow has two. As you build out your bot and add functionality, you'll need to keep the `FlowMap` updated. If you attempt to transition to a flow or state that hasn't yet been declared in the `FlowMap`, you'll encounter a Stealth`::Errors::InvalidStateTransition` exception.
+
+States also support  additional options like the `fails_to` option for the `get_hello_response` state. We'll cover these state options in the [State Option docs](state-options.md) section.

data/docs/flows/overview.md

@@ -0,0 +1,43 @@
+# Flows & States
+
+## Overview
+
+Flows and states are the primary building blocks for Stealth bots. Your bot's users can only be in a single flow and state at any given moment. The relationship between flows and states is one of parent and child, respectively. So a flow can _have many_ states and a state always _belongs to_ a single flow.
+
+The concept is modeled after [finite-state machines](https://en.m.wikipedia.org/wiki/Finite-state\_machine), though you don't need to familiarize yourself with all of that knowledge. The outline we provide in these docs will be sufficient.
+
+Finite-state machines, or more simply state machines, are used throughout engineering to model states within a given machine. Imagine a coin-operated, turnstile you might find in a subway or airport. You insert a coin and the mechanism unlocks to allow you to rotate the arms and pass through.
+
+![Figure 1: A simple, coin-operated turnstile](../../.gitbook/assets/torniqueterevolution.jpg)
+
+The operation of this turnstile can (and probably is) modeled as a state machine. Here is an example of what that model looks like:
+
+![Figure 2: Finite-state machine model for the simple, coin-operated turnstile.](../../.gitbook/assets/2880px-turnstile\_state\_machine\_colored.svg.png)
+
+In Figure 2, the "starting" state is _Locked_ and if someone attempts to _Push_ the turnstile arms while it is in the _Locked_ state it will indefinitely remain in the _Locked_ state. That's what the self-referencing _Push_ action in Figure 2 is showing. Similarly, in Stealth, states can transition a user to a new state or it can keep a user in the same state either indefinitely or until some specific action is taken.
+
+When a user inserts a _Coin_, the state machine in Figure 2 transitions the machine to the _Unlocked_ state. If a user inserts more coins while in this state, the machine just remains in the _Unlocked_ state. When the turnstile arms are _Pushed_, then the machine transitions back to the _Locked_ state.
+
+This turnstile example highlights the mental model of flows and states in Stealth quite well. Specifically, states can transition your users to other states or they can keep your user in the same state. In the section about [Sessions](../controllers/sessions/), we'll cover all the ways these transitions can happen.
+
+## Flows
+
+A **flow** is the term used to describe a complete interaction between a user and the bot. Flows are comprised of `states`, like a finite state machine. In Figure 2 above, the entire finite-state machine is the flow.
+
+For example, if a user is using your bot to receive an insurance quote, the flow might be named `quote`. 
+
+{% hint style="warning" %}
+Stealth requires that flows be named in the singular form, like Ruby on Rails.
+{% endhint %}
+
+A flow consists of the following components:
+
+1. A controller file, named in the plural form. For example, a `quote` flow would have a corresponding `QuotesController`. One controller maps to one flow.
+2. Replies. Each flow will have a directory in the `replies` directory in plural form. Again using the `quote` flow example, the directory would named `quotes`.
+3. An entry in the `FlowMap`. The `FlowMap` file is where each flow and it's respective states are defined. We'll cover the FlowMap file in [FlowMap docs](flowmap.md) section.
+
+## States
+
+A **state** is the logical division of flows. Just like in finite-state machines, users can transition between states. In Stealth, users can even transition between states from different flows. There are no naming conventions enforced by Stealth for states, but in [State Naming section](state-naming.md) we'll cover some best practices.
+
+As mentioned in the above, a user can at most be in a single flow and state at any given moment.

data/docs/flows/state-naming.md

@@ -0,0 +1,53 @@
+# State Naming
+
+While Stealth doesn't enforce any naming requirements for your states, we do recommend following the naming conventions outlined below. It provides continuity across your team and across bots.
+
+Most of your states will fall into the `say`, `get`, and `ask` buckets. On the rare occasion that it does not, feel free to select a name that best describes the state.
+
+## Say, Ask, Get
+
+#### Say
+
+_Say_ actions are for _saying_ something to the user.
+
+For example:
+
+```ruby
+def say_hello
+  send_replies
+end
+```
+
+Typically we'd send the user to a new state, but sometimes it's as simple as just saying something like in the case of the end of a flow or conversation.
+
+#### Ask
+
+_Ask_  actions are for _asking_ something from the user.
+
+For example:
+
+```ruby
+def ask_weather
+  send_replies
+  update_session_to state: 'get_weather_response'
+end
+```
+
+In the above example, we've asked a question via `send_replies` and we've updated the session to a new state. This is the state that will be receiving the response. We'll cover state transitions in detail in the [Sessions Overview](../controllers/sessions/intro.md) section.
+
+#### Get
+
+_Get_  actions are for _getting_ and parsing a message from the user.
+
+For example:
+
+```ruby
+def get_weather_response
+  handle_message(
+    'Sunny'   => proc { step_to state: 'say_wear_sunglasses' },
+    'Raining' => proc { step_to state: 'say_dont_forget_umbrella' }
+  )
+end
+```
+
+In the example above, we're handling two responses by the user. When they say "Sunny" or "Raining". Don't worry too much about the format of `handle_message`. We cover its usage in the [handle\_message docs](../controllers/handle\_message/) section.

data/docs/flows/state-options.md

@@ -0,0 +1,70 @@
+# State Options
+
+In your `FlowMap`, each state may also specify certain options. Some options expose  built-in Stealth functionality, while others are completely custom and can be referenced by your code.
+
+```ruby
+class FlowMap
+
+  include Stealth::Flow
+
+  flow :hello do
+    state :say_hello
+    state :get_hello_response, fails_to: :say_hello
+    state :say_hola, redirects_to: :say_hello
+  end
+
+  flow :goodbye do
+    state :say_goodbye, re_engage: false
+  end
+
+  flow :interrupt do
+    state :say_interrupted
+  end
+
+  flow :unrecognized_message do
+    state :handle_unrecognized_message
+  end
+
+  flow :catch_all do
+    state :level1
+  end
+
+end
+```
+
+We see three states have options defined: `get_hello_response`, `say_hola`, and `say_goodbye`.
+
+#### fails\_to
+
+The `fails_to` option is one of the built-in Stealth state options. By default, it's used in the `CatchAllsController` to specify where a user should be sent in the event of an error. We cover this more in the [CatchAll docs](../controllers/catch-alls.md), but in the `get_hello_response` state above, if Stealth encounters an error the `fails_to` option declares the user to be sent to the `say_hello` state of the same flow.
+
+The `fails_to` value can also be a string if you wish to specify a different flow. So for example:
+
+```ruby
+state :get_hello_response, fails_to: 'goodbye->say_goodbye'
+```
+
+If Stealth encounters an error in this state, it will be sent to the `say_goodbye` state of the `goodbye` flow.
+
+#### redirects\_to
+
+The `redirects_to` option is useful when you're performing a rename of a state and the bot has already been deployed to production. Your production users may have existing sessions attached to the state you are renaming. If you were to perform a state rename without attaching a `redirects_to` to the old state name, the user will receive an error the next time they message your bot.
+
+{% hint style="info" %}
+For the `redirects_to`values, you can use state names as well as the "flow->state\_name" convention like in `fails_to`.
+{% endhint %}
+
+#### Custom Options
+
+In addition to the built-in Stealth state options, you are able to define your own. This is helpful for cases where you want to define metadata for a set of states but don't want to define that logic within the controllers.
+
+In the example `FlowMap` above, we've defined a `re_engage` option on the `say_goodbye` state. If we pretend our bot re-engages leads after a period of time, this option would be useful for allowing us to declare states for which we do not want re-engagements to be sent. In this case, the user has reached the end of the bot and so we don't want to send them any re-engagements.
+
+You can access these custom state options via the `opts` attribute for the state specification.
+
+```ruby
+state_spec = FlowMap.flow_spec[:goodbye].states[:say_goodbye]
+state_spec.opts.present? && state_spec.opts[:re_engage]
+```
+
+Here `state_spec.opts[:re_engage]` contains the value `true`. The hash key will correspond to what you named your option in the `FlowMap`.

data/docs/getting-started.md

@@ -0,0 +1,19 @@
+# Getting Started
+
+Stealth is designed to run on Ruby (MRI) 2.5+
+
+While we don't require any C-based Ruby gems, we haven't yet certified Stealth on other VMs (such as JRuby). However, we do intend to provide official support for JRuby and TruffleRuby soon.
+
+### Installation
+
+You can install Stealth via RubyGems:
+
+```ruby
+gem install stealth
+```
+
+Next, create your new bot:
+
+```
+stealth new <bot_name>
+```

data/docs/glossary.md

@@ -0,0 +1,21 @@
+---
+description: >-
+  Some common terms you may read throughout this doc or as you interact with the
+  Stealth community.
+---
+
+# Glossary
+
+* **message** - An _incoming_ message from a user. A _message_ and a _reply_ are counterparts.
+* **reply** - An _outgoing_ message from your bot. A _message_ and a _reply_ are counterparts.
+* **service message** - This is the long version of _message_. You will likely only see this referenced when developing your own Stealth components.
+* **`current_message`** - This is object that contains the service message. It's available within all controller actions. More info can be found in the [controller docs](controllers/controller-overview.md).
+* **component** - Components are the individual building blocks of Stealth. Stealth itself is the core framework that handles webhooks, replies, etc. Components allow Stealth to connect to messaging platforms, NLP providers, and more. Each component is offered as a separate Ruby gem.
+* **message platform** - Message platforms are the platforms where your bot interacts with its users. E.g., Facebook Messenger, SMS, Whatsapp, Slack, etc.
+* **NLP** - natural language processing. This is the AI subfield that encompasses taking unstructured text (like messages from users) and extracting structured concepts. NLP in Stealth is achieved through components.
+* **NLU** - A subclass of NLP. More aptly describes the type of NLP you'll want to perform with Stealth, but NLP is the more commonly used term.
+* **session** - Sessions allow your Stealth bot to recognize subsequent messages from users. It keeps track of where in the conversation each of your users currently reside.
+* **MVC** - A software design pattern. It's not critical to understand this to get going, but if you're interested you can learn more [here](https://www.google.com/url?sa=t\&rct=j\&q=\&esrc=s\&source=web\&cd=\&cad=rja\&uact=8\&ved=2ahUKEwiGt\_XpzPHtAhXNVc0KHWjiDG8QFjAAegQIBRAC\&url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FModel%25E2%2580%2593view%25E2%2580%2593controller\&usg=AOvVaw1wpuCUJRz1WxG51eRibnYX).
+* **intent** - **** Intents are one of the main things NLP components extract from a message. They are a type of classification. We talk about them in more detail in the `handle_message` [docs](controllers/handle\_message/nlp-matcher.md).
+* **entity** - Entities are individual tokens within a message that an NLP component will extract. So for example, a number or some other trained entity specific to your bot (like car models and makes). More info about these can be found in the `get_match` [docs](controllers/get\_match/entity-match.md).
+* **regex** - A regular expression. These are a programming concept used in string matching. In Stealth, these are most often used in `handle_message` and there are [docs](controllers/handle\_message/regex-matcher.md) for their usage.

data/docs/models/activerecord.md

@@ -0,0 +1,2 @@
+# ActiveRecord
+

data/docs/models/mongoid.md

@@ -0,0 +1,2 @@
+# Mongoid
+

data/docs/models/overview.md

@@ -0,0 +1,2 @@
+# Model Overview
+

data/docs/nlp-nlu/microsoft-luis.md

@@ -0,0 +1,2 @@
+# Microsoft LUIS
+

data/docs/nlp-nlu/openai.md

@@ -0,0 +1,2 @@
+# OpenAI
+

data/docs/nlp-nlu/overview.md

@@ -0,0 +1,2 @@
+# NLP Overview
+

data/docs/platforms/alexa-skills.md

@@ -0,0 +1,2 @@
+# Alexa Skills
+

data/docs/platforms/facebook-messenger.md

@@ -0,0 +1,2 @@
+# Facebook Messenger
+

data/docs/platforms/overview.md

@@ -0,0 +1,2 @@
+# Platform Overview
+

data/docs/platforms/sms-whatsapp.md

@@ -0,0 +1,2 @@
+# SMS/Whatsapp
+

data/docs/platforms/voice.md

@@ -0,0 +1,2 @@
+# Voice
+

data/docs/replies/delays.md

@@ -0,0 +1,2 @@
+# Delays
+

data/docs/replies/erb.md

@@ -0,0 +1,2 @@
+# ERB
+

data/docs/replies/inline-replies.md

@@ -0,0 +1,2 @@
+# Inline Replies
+

data/docs/replies/reply-overview.md

@@ -0,0 +1,2 @@
+# Reply Overview
+

data/docs/replies/variants.md

@@ -0,0 +1,2 @@
+# Variants
+

data/docs/replies/yaml-replies.md

@@ -0,0 +1,2 @@
+# YAML Replies
+

data/docs/testing/integration-testing.md

@@ -0,0 +1,2 @@
+# Integration Testing
+

data/docs/testing/untitled.md

@@ -0,0 +1,2 @@
+# Specs
+

data/lib/stealth/base.rb

@@ -123,7 +123,7 @@ module Stealth
 
     load_bot!
 
-    Sidekiq.options[:reloader] = Stealth.bot_reloader
+    Sidekiq[:reloader] = Stealth.bot_reloader
 
     if defined?(ActiveRecord)
       if ENV['DATABASE_URL'].present?

data/lib/stealth/controller/replies.rb

@@ -96,11 +96,11 @@ module Stealth
               reply: reply
             )
 
-            translated_reply = handler.send(reply.reply_type)
-            client = service_client.new(reply: translated_reply)
+            formatted_reply = handler.send(reply.reply_type)
+            client = service_client.new(reply: formatted_reply)
             client.transmit
 
-            log_reply(reply) if Stealth.config.transcript_logging
+            log_reply(reply, handler) if Stealth.config.transcript_logging
 
             # If this was a 'delay' type of reply, we insert the delay
             if reply.delay?
@@ -264,10 +264,14 @@ module Stealth
             end
           end
 
-          def log_reply(reply)
+          def log_reply(reply, reply_handler)
             message = case reply.reply_type
                       when 'text'
-                        reply['text']
+                        if reply_handler.respond_to?(:translated_reply)
+                          reply_handler.translated_reply
+                        else
+                          reply['text']
+                        end
                       when 'speech'
                         reply['speech']
                       when 'ssml'
@@ -282,6 +286,8 @@ module Stealth
               topic: current_service,
               message: "User #{current_session_id} -> Sending: #{message}"
             )
+
+            message
           end
 
       end # instance methods

data/lib/stealth/server.rb

@@ -35,7 +35,12 @@ module Stealth
       # JSON params need to be parsed and added to the params
       if request.env['CONTENT_TYPE']&.match(/application\/json/i)
         json_params = MultiJson.load(request.body.read)
-        params.merge!(json_params)
+
+        if bandwidth?
+          params.merge!(json_params.first)
+        else
+          params.merge!(json_params)
+        end
       end
 
       dispatcher = Stealth::Dispatcher.new(
@@ -60,5 +65,9 @@ module Stealth
         end
       end
 
+      def bandwidth?
+        params[:service] == "bandwidth"
+      end
+
   end
 end

data/spec/controller/replies_spec.rb

@@ -66,6 +66,10 @@ describe "Stealth::Controller replies" do
       send_replies custom_reply: 'messages/sub1/sub2/say_nested'
     end
 
+    def say_simple_hello
+      send_replies
+    end
+
     def say_inline_reply
       reply = [
         { 'reply_type' => 'text', 'text' => 'Hi, Morty. Welcome to Stealth bot...' },
@@ -609,6 +613,50 @@ describe "Stealth::Controller replies" do
     end
   end
 
+  describe "Logging replies" do
+    let(:stubbed_handler) { double("handler") }
+    let(:stubbed_client) { double("client") }
+
+    before(:each) do
+      allow(Stealth::Services::Facebook::ReplyHandler).to receive(:new).and_return(stubbed_handler)
+      allow(Stealth::Services::Facebook::Client).to receive(:new).and_return(stubbed_client)
+      allow(controller.current_session).to receive(:flow_string).and_return("message")
+      allow(controller.current_session).to receive(:state_string).and_return("say_simple_hello")
+      Stealth.config.auto_insert_delays = false
+      Stealth.config.transcript_logging = true
+    end
+
+    after(:each) do
+      Stealth.config.auto_insert_delays = true
+      Stealth.config.transcript_logging = false
+    end
+
+    it "should log replies if transcript_logging is enabled" do
+      allow(stubbed_client).to receive(:transmit).and_return(true)
+      allow(controller).to receive(:sleep).and_return(true).with(2.0)
+
+      allow(stubbed_handler).to receive(:text).exactly(1).times
+      expect(Stealth::Logger).to receive(:l).with(
+        topic: 'facebook',
+        message: "User #{controller.current_session_id} -> Sending: Hello"
+      )
+      controller.say_simple_hello
+    end
+
+    it "should log translated replies if transcript_logging is enabled and the driver supports it" do
+      allow(stubbed_client).to receive(:transmit).and_return(true)
+      allow(controller).to receive(:sleep).and_return(true).with(2.0)
+
+      allow(stubbed_handler).to receive(:text).exactly(1).times
+      allow(stubbed_handler).to receive(:translated_reply).and_return("Bonjour")
+      expect(Stealth::Logger).to receive(:l).with(
+        topic: 'facebook',
+        message: "User #{controller.current_session_id} -> Sending: Bonjour"
+      )
+      controller.say_simple_hello
+    end
+  end
+
   describe "client errors" do
     let(:stubbed_handler) { double("handler") }
     let(:stubbed_client) { double("client") }

data/spec/replies/messages/say_simple_hello.yml

@@ -0,0 +1,2 @@
+- reply_type: text
+  text: "Hello"

data/stealth.gemspec

@@ -16,12 +16,11 @@ Gem::Specification.new do |s|
   s.add_dependency 'puma', '>= 4.2', '< 6.0'
   s.add_dependency 'thor', '~> 1.0'
   s.add_dependency 'multi_json', '~> 1.12'
-  s.add_dependency 'sidekiq', '~> 6.0'
+  s.add_dependency 'sidekiq', '< 7'
   s.add_dependency 'activesupport', '~> 6.0'
 
   s.add_development_dependency 'rspec', '~> 3.9'
-  s.add_development_dependency 'rspec_junit_formatter', '~> 0.3'
-  s.add_development_dependency 'rack-test', '~> 1.1'
+  s.add_development_dependency 'rack-test', '~> 2.0'
   s.add_development_dependency 'mock_redis', '~> 0.22'
 
   s.files         = `git ls-files`.split("\n")