Combining incoming SMS message from Twilio

Question

I had a general question about programming in Twilio which I posted to stackoverflow, but since I'm actually using SFDC as my development platform, I'm also going to ask here in case there are Salesforce-specific solutions.

The problem

I'm receiving text messages from Twilio and saving them as custom objects in my Salesforce database. If someone sends me a message that is longer than 160 characters long, it gets chopped up into chunks and sent as multiple messages; in my app, I want to recombine them and save them as a single new record.

When Twilio sends me the incoming messages (as a POST to a Site), there's no way for me to inspect the request and determine whether it's a new message or an addition to a previous one. Instead, I'm doing this in my Apex:

Message__c msg = tryToGetMessageFromThisSenderInLastMinute();
if (msg != null) {
    // Append to previous
    msg.Body__c += newMessage;
} else {
    // Save a new message
    msg = new Message__c(Body__c = newMessage);
}
upsert msg;

The problem is that sometimes the second message arrives before the first one has finished saving. As a result, two records will be created instead of just one. When there are three (or more) message segments, it gets even crazier:

// First message arrives:
record1 == "This is the first message."

// Second message arrives, first isn't saved yet.
record1 == "This is the first message."
record2 == "This is the second message."

// Third message arrives, first has now been saved.
record1 == "This is the first message. This is the third message."
record2 == "This is the second message."

Are there Salesforce-specific solutions?

Some ideas:

Control flow w/ static properties: Unfortunately, each message comes in as a separate request, so static variables don't retain their values.

Go Async or Pause before (re-)checking for an existing record: This would be tricky to do well, but is irrelevant on SFDC since there's no way to 'sleep' within a synchronous flow. Going async (@future) could be even worse b/c I can't guarantee when the records will be processed. It doesn't solve the problem, just delays it.

Other Ideas? Looking for good tricks here - I'm out of my own ideas!

Answer 1

So I built a solution for an NPO using Twilio and SFDC, and this continues to be the biggest issue we face, so I feel your pain. I've talked to Twilio, and there aren't any good options to handle this. In the solution we built, incoming messages are added as comments to a chatter thread on a Case - using similar logic to you, we look for an existing SMS message within 24 hours, and if so, assume it is for that case, and add it to the SMS feed on that case.

So what I ended up doing was building a merge/update publisher action on the feed so users could re-order the messages/merge multi part messages into one message when they arrived in the wrong order. You can see it in action at around 8 mins on this YouTube video. The code to do that is on the github repo I linked to above. I think it is a carrier issue, so I don't see any better solution coming along in the near future...

Answer 2

Thinking out of the box for a minute - could you just store all incoming texts in say a "Text Queue" sObject and process only after having them in the queue for 5 minutes? (based on createDate?)

That should give your connector enough time to receive the multi-part texts and process without having to "guess" or store/forward messages? Of course if the business needs things in more real-time than 5 minute intervals this won't work, but it's an idea!

Answer 3

Could you do something with record locking? I assume this would work, although it could be classified as 'doesn't behave nicely' or consume resources...

Let's assume you're looking a contact whose phone number matches the one just received:

// This while lock will continue to execute until we successfully lock the record
Boolean haveLock = false;
while (!haveLock) {
    try {
        Contact con = [SELECT Id, Name FROM Contact WHERE Phone = :phoneFromTwilio FOR UPDATE];
        haveLock = true;
    } catch (QueryException e) {
        // Another process is already locking this contact... try again
    }
}

// Now you know noone else is using the record, so continue with save as usual.

Answer 4

I was looking at migrating to Twilio from Cardboardfish so we could leverage thier Voice framework, but having looked at Twilio I had exactly the same problem finding how I would merge back these messages.

With Cardboardfish, you get the total number of pieces expected for a message sent in each of the message parts and the number of the piece that you are looking at.

The solution I built here was to to process single part messages immediately but I persisted the multipart messages until I had all parts then merged them back together.

I also found that the messages do not come in in the order they are sent and this is true for all providers I believe so that can't reliably be used to glue them back together.

Sending back a response automatically is also a problem if you can't glue the messages back together as all messages received would cause you to send back a "thanks for the message" text.

Kevin