<- See more posts

Twilio Two Way Messaging

May 18, 2016

Video


Hello everyone, today we’re going to be talking about implementing Twilio into your Ruby on Rails applications to do two-way text messaging.

Before we jump into the code for Twilio let’s talk about our project and goals here:

Introduction


We have a business that wants to be able to message people and allow them to text back. Let’s say we are trying to keep track of people that work for us and periodically we send them questions to get their responses. I’m being vague about this because this could apply to dozens of situations.

Prerequisite Work

We will start our work on a Ruby on Rails app that has Devise and sucker_punch installed along with a few UI related gems. We’ll build on top of the work that has already been done for us by ‘previous’ developers and add the text messaging functionality on top of it.

You can fork the app at this repo and commit to get started from the same place we are:

All of the code can be found at the main repohttps://github.com/king601/twowaymessaging/

Gameplan

The plan is to add all of the related code and send the text messages in a background job. As far as the user is concerned they just want to be able to send a message to a person they are having a Conversation with.

For now our plan is to just let the app send messages and handling the replies to those messages by directing them to the proper conversation but not to add anything fancy like notifications.

Getting Started

To get started you’ll need a Twilio account and need your Account SID and Primary Auth Token. You can find those on the Twilio Console.

In addition we will be leveraging a Messaging Service from Twilio so we will need to create a Messaging Service in the Twilio Console.

Once you create a messaging service you’ll need to grab the messaging service sid. We will be putting these into the Rails secrets.yml file. If you’d prefer you can use environmental variables but for the sake of simplicity we’ll use the secrets.yml file.

One of the last things you’ll need to do is buy and add a phone number to your Messaging Service. Once that’s done you are ready to start sending text messages, just be sure you’ve added funds to your account. The phone number will cost $1 a month and each text is around 1 cent. So it is very reasonable to get started.

You can read more about Messaging Services on Twilio’s documentation, here is a link: https://www.twilio.com/docs/api/rest/sending-messages-copilot

Quick side note: If you’re using the secrets.yml file please make sure you add it to your .gitignore file so it does not commit it to your repositories.

.gitignore
/config/secrets.yml
/config/database.yml

I’ve added config/secrets and config/database to my gitignore to ensure I don’t commit any keys. Be sure if you have to add these you add and commit the changes so it takes effect.

secrets.yml
development:
  secret_key_base: dc93e33ad33f5b83b409fa18f833c687467a730a3b9cd1401b5da7b236d5a393a6c2a8406df7e28d841155e2517452650bb3bc9ad518e1829bea13a127c47930
  twilio_account_sid: AC1fff9b222963243423442343242343
  twilio_token: 2ewffwefwefewfwefwefwfwefwefwefw6
  twilio_messaging_service_sid: MG41rwerwerwerwrwerwerwerwerwer31

Don’t worry those aren’t actually my keys but I wanted to show you what your file should look similar to. Make sure you replace those values with yours.

Once that is done we need to add the twilio-ruby gem to the Gemfile and run bundle install

Gemfile
# Twilio for text messaging
gem 'twilio-ruby', '~> 4.11.1'

bundle install

Be sure to restart your Rails Server after modifying the secrets.yml file and bundling.

Person Scaffold

Great, so before we move further with the Twilio work let’s take a moment to setup our project. Since we’re building a two way messaging system I want a way to keep track of who I’m sending messages to and from. We will use a Person model to represent who we’re sending messages to or receiving them from.

Let’s generate a scaffold to represent a Person
rails g scaffold Person name:string phone:string user:references

Let’s now generate a Conversations model. We’re going to use this to link People with our user. This may not be a completely necessary step but we will also use it for our Messages so we know which thread to display them in,
rails g model Conversation person:references user:references

This will create the associations and indexes on the database for us. Let’s check out the User.rb model and make sure we have has_many :conversations, and check the Person.rb to ensure that it has_one :conversation

I’m using a has_many to has_one relationship here because as the user of the app you’ll have many conversations with many People but a Person when looking up their conversation should have only one conversation.

We also need a way to create a Conversations record for each Person, so to do that I’ll write a callback on the Person model and use the after_create callback to create a Conversation record with the Person ID and User ID of the Person.

app/models/person.rb
after_create :create_conversation!

  def create_conversation!
    Conversation.find_or_create_by(person: self, user: self.user)
  end

We’ll run the migrations as well since we need to do that
rake db:migrate

Let’s go check on the controllers and be sure to add the devise action to authenticate users on the people and conversations controllers.
  before_action :authenticate_user!

Also we need to change the line in the controller for the people in the create action to be:
  @person = current_user.people.build(person_params)

One thing to note is that I am forgoing the ability to do group conversations but that is okay for my use case.

—

When we send messages to Twilio we only really need a phone number it is going to and a body that contains the message to be sent. But receiving messages will come with more information. If you look at the Twilio REST API for the Message you’ll see it sends more params then you may expect, like it will look up the FromCity and FromCountry and pass them to your App with the POST request it will make.

https://www.twilio.com/docs/api/rest/message

We don’t need most of those extra things, you can keep them if you want but for this video I’ll only worry about what I’m considering to be needed. Which will be the to, from, body, message sid, account Sid, messaging service sid, direction and status params.

Routing

For the routing I’ll be using a resources block for conversations and nesting the resources for messages inside, take note that I use a module: :conversations to reflect that the messages belong to a conversation, it’s basically a way to scope or namespace things. It makes sense to scope the messages logic under Conversations because that’s what its for.

We’ll edit our routes file to include

config/routes.rb
resources :conversations, only: [:index] do
  resources :messages, module: :conversations, only: [:index, :create]
end

# Twilio Webhooks
post 'conversations/messages/reply', to: 'conversations/messages#reply'

Because we also need an endpoint for Twilio to POST to when an inbound message is received we will manually specify one and then link it to a reply action on the Conversations::MessagesController. This reply action will handle the inbound message and will create the Message containing the inbound message.

Messages

Let’s go ahead and create the model for the Messages
rails g model Message to from status body message_sid account_sid messaging_service_sid direction user:references conversation:references

Then run the migrations
rake db:migrate

Controllers for Conversations & Messages

Let’s generate our controllers before diving into the views
rails g controller Conversations index

rails g controller Conversations::Messages index

Quickly go back to the routes file and clean up the auto-generated routes since we don’t need them.

We can now find our app/controllers/conversations_controller.rb and app/controllers/conversations/messages_controller.rb files and open them for editing.

Be sure to add the authenticate_user before action to the top of both files before proceeding.
before_action :authenticate_user!

We now need to go edit the conversations controller and views to allow us to list the conversations we have and link to the messages view for each of them.

app/controllers/conversations_controller.rb
class ConversationsController < ApplicationController
  before_action :authenticate_user!

  def index
    @conversations = current_user.conversations.includes(:person).all
  end
end

Views for Conversations

app/views/conversations/index.html.erb
<h1>Conversations</h1>

<%= render @conversations %>

app/views/conversations/_conversation.html.erb
<div class="conversation">
  Person <%= conversation.person.name %> - <%= conversation.person.phone %><br>
  <%= link_to "View Conversation", conversation_messages_path(conversation) %><br>
  <hr>
</div>

Perfect so now we have our list of conversations linking to the page where we will have our messages on. Let’s get the messages being created now. For the moment we’ll just treat it like we normally would and add the actual Twilio sending in a few.

If you remember from the routes file earlier we specified that we only want the index and create routes for the Messages controller. We will also need a reply action so let’s go create the basic ones.

app/controllers/conversations/messages_controller.rb
class Conversations::MessagesController < ApplicationController
  before_action :authenticate_user!
  before_action :set_conversation, except: [:reply]

  def index
    @messages = @conversation.messages.order("created_at ASC")
    @message = @conversation.messages.build
  end

  def create
    @message = @conversation.messages.build(message_params)

    # Set the fields to the proper values since the only thing sent is the body
    @message.update(to: @conversation.person.phone, status: 'pending', direction: 'outbound-api')

    if @message.save
      redirect_to conversation_messages_path(@conversation), notice: 'Message sent!'
    else
      redirect_to conversation_messages_path(@conversation), notice: 'Something went wrong. Please try again'
    end
  end

  def reply
    # handle Twilio POST here
  end

  private
    def set_conversation
      @conversation = current_user.conversations.find(params[:conversation_id])
    end

    def message_params
      params.require(:message).permit(:body)
    end
end

So let’s go through it real quick. The index action we’re setting up instance variables for @messages which will query the database to get all of the messages for the conversation and order them by the created_at date in ascending order.

Then we setup an @message instance variable for the form we use to create a new message. That will pass off to the create action when it is submitted where we create the message and then set the to field to the Person’s phone number and then set the status and direction.

Mostly standard CRUD actions. We’ll get to the reply action and actually sending the Text message with Twilio shortly.

Let’s get the views wrapped up for the messages so we can actually see it start coming together.

app/views/conversations/messages/index.html.erb
<h1>Messages With <%= @conversation.person.name %></h1>
<div id="messages">
  <%= render @messages %>
</div>

<hr>

<%= form_for (@message), url: conversation_messages_path(@conversation) do |f| %>
  <div class="form-group">
    <%= f.text_area :body, class: 'form-control', placeholder: 'Type your message to send' %>
  </div>
    <%= f.submit 'Send Message', class: 'btn btn-primary' %>
<% end %>

app/views/conversations/messages/_message.html.erb
<div class="clearfix <%= message.id %>">
  <div class="alert <%= message.from ? 'alert-info message-inbound' : 'alert-success message-outbound' %>">
    <%= message.body %>
     <div class="text-right">
       <small><%= local_time message.created_at %> </small>
       <% if message.status == "sent" %><small>sent</small><% end %>
     </div>
  </div>
</div>

SMS Outbound Implementation

So let’s start implementing the SMS messaging for outbound texts. We will use a background job to send the text message.

I prefer to create an SMS ruby class to send the message with Twilio and in my background job send the message through that SMS class.

That way if I ever need to switch out Twilio with another service I can handle it there.

So let’s go ahead and create

app/models/sms.rb
class SMS
  def initialize(to_phone_number)
    @to_phone_number = to_phone_number
  end

  def client
    Twilio::REST::Client.new Rails.application.secrets.twilio_account_sid, Rails.application.secrets.twilio_token
  end

  def send(body)
    client.account.messages.create(
      :messaging_service_sid => Rails.application.secrets.twilio_messaging_service_sid,
      :to => @to_phone_number,
      :body => body
    )
  end
end

So basically we’re creating a class that when you call it you pass in the phone number the message is going to then give it a send method with the body you want to pass in. It then makes the REST API call to Twilio to send the message.
rails g job SendSms

Now go and edit the app/jobs/send_sms_job.rb file it just generated to look like
class SendSmsJob < ActiveJob::Base
  queue_as :default

  def perform(message)
    # Send the SMS now, then update the message status
    SMS.new(message.to).send message.body
    message.update_columns(status: "sent")

  end

end

We are passing in the message when we call the job so we will need to call it in our code like so.
SendSmsJob.perform_later(@message)

I added the SendSmsJob to the message controller create action inside of the if block where we make sure @message saves. Now we will fire the job when a message is saved, thus triggering the SMS class to send a text message, then updating the Message record to reflect it has been sent.

Localtunnel / Ngrok

We should now have outgoing messages working. The next step will be to make incoming messages work.

You’ll need to setup some additional software to make this work depending on your development environment, that will allow you to use web hooks locally. I’m going to use localtunnel.me to expose my dev server to Twilio for it to POST to but you can use ngrok or another similar program.

So i’ll open another terminal and run
lt -s mysubdomainhere -p 3000

Here i’m passing the -s argument to request a subdomain and passing -p and 3000 to specify that I want to expose port 3000.

Now I can reach my Rails app at http://mysubdomainhere.localtunnel.me

We’ll update our Twilio webhook in the console on our messaging service to point to our endpoint on our Rails app. Once we update that we’ll be able to reply to the text messages and see the POST request in the logs.

—

Go to your Twilio Console, go to your Messaging Service you created. Ensure the Process Inbound checkbox is checked and in the Request URL enter in the URL for your local tunnel with /conversations/messages/reply added to the end of it.

For example mine looks like https://mysubdomainhere.localtunnel.me/conversations/messages/reply

Then press Save and Twilio will now POST inbound messages to your dev environment as long as it can reach you.

— So let’s go back to our Messages controller and flesh out that reply action.

app/controllers/conversations/messages_controller.rb
  def reply
    # handle Twilio POST here
    from = params[:From].gsub(/^\+\d/, '')
    body = params[:Body]
    status = params[:SmsStatus]
    direction = 'inbound'
    message_sid = params[:MessageSid]

    # Find the conversation it should belong to.
    person = Person.where("phone like ?", "%#{from}%").first
    @conversation = Conversation.where(person: person).first
    @conversation.messages.build(body: body, direction: direction, status: status, from: from, message_sid: message_sid).save
    # the head :ok tells everything is ok
    # This stops the nasty template errors in the console
    head :ok, content_type: "text/html"
  end

So let’s go through this code real quick, We’re turning all the params we are concerned about from Twilio into variables that are easier to work with then on the from variable we are gsubbing out the +1 from phone numbers, this allows us to not require +1 in front of numbers to find the Person.

We then look up the person’s record based on the phone number they replied from. So if you are paying attention it is clear the app will break if the number can’t be found or isn’t recognized but that’s another topic for another day. Then we set the conversation based on the person and build a message with the params.

We then return an OK to Twilo.

Then we need to ensure that devise’s action and the authenticity tokens don’t interfere with Twilio’s POST so we’ll make changes to the before_actions to allow for the request.
  before_action :authenticate_user!, except: [:reply]
  before_action :set_conversation, except: [:reply]
  skip_before_action :verify_authenticity_token, only: [:reply]

That will allow the request to come through and only let the reply action skip authentication since Twilio doesn’t login to our app.

All things considered now, replies should now be able to be handled by the app. Let’s test it out.
About Activity Posts Speaking Courses Email me