Helping SMS stink less

We've encountered a recurring problem with SMS message delivery, and wanted to let everyone know about our solution.

Beginning in January 2019, we're going to set a limit on how long an undelivered SMS message can stay active without reaching a recipient before it is discarded. This is known as a validity period, and we're planning on setting it to a maximum of two hours.

What this means is that if your phone is turned off (or in airplane mode), or the provider has a failure, or any other issue arises that prevents a message from actually reaching your device for two hours, the message will expire and be discarded.

If this seems strange or even a little counter-intuitive, well, allow us to explain...

The problem with SMS

As a communication tool, text messages are a little bit of a double-edged sword. On one hand, SMS is immediate, direct, and exceedingly common. 

On the other hand, longer SMS messages are easily malformed (especially when they have to be broken apart by the provider and reassembled on your device), carriers in many parts of the world are notoriously unreliable in terms of delivery, and the service can be very costly to both the sender and the responder.

Another issue with SMS messages that is not shared by other message types such as email or our fantastic mobile apps (available now in the App Store and Google Play) is that it's often impossible to tell when an SMS was sent. 

A diversionary example

Let's say you're just getting on a plane for a cross-country jaunt to another office. Like the seasoned and savvy traveler that you are, you put your phone in Airplane Mode so as to not incur the wrath of the flight attendant. A few hours later, you whip out your phone the instant you land and check for any new messages. 

The first thing that pops up is an SMS message saying that there's a traffic jam between the airport and the office and they expect the highway to be closed for two hours. So you kill some time wandering around the airport admiring the many closed kiosks and terribly overpriced magnets before heading over to the pickup area only to find that the coworker who volunteered to pick you up has been waiting for more than an hour because they sent that SMS message 20 minutes after you boarded the plane and the highway's been open for ages and now they've missed lunch and it's all your fault.

But this is real life

Okay, so maybe that situation could have been solved by sending another SMS or even just picking up the phone, but what if it's more serious? Like, life-or-death serious?

The recent wildfires in California caused a catastrophic disruption across a large part of the state. Thousands of businesses and homes were destroyed, and almost a hundred people lost their lives. 

In the midst of the chaos, xMatters was sending messages of all types to people across the state, warning them of highway closures, fire outbreaks, and other life-threatening situations. During this crisis, one of the major providers had a failure that prevented them from delivering SMS messages. Users in our system were still receiving timely, accurate messages via voice, email, and the mobile apps, but some SMS messages weren't getting through.

Worse yet, when the provider restored service some hours later, some of those SMS messages were delivered to the intended recipient. Without any sort of timestamp or other indication, however, those messages were misinterpreted as meaning that another fire had started or that a highway they thought had been opened was closed again. In these cases, the late messages were adding to the chaos instead of helping people survive it.

Finding a solution

We've been struggling with how to address this issue and still provide a reliable service that SMS users can depend on for their day-to-day needs. It seems that the one common thread in practically all these situations is that a late (and undated) SMS message is often worse than if the message was just never delivered. So after much consideration and consultation with our providers and customers, we decided on the solution outlined above. Here is it again, just in case:

Beginning in January 2019, we're going to set a two-hour validity period on undelivered SMS messages before we mark them as expired and discard them.

Tracking expired messages

Here's what it will look like in xMatters On-Demand when a message exceeds its validity period:

1. When On-Demand delivers the message to the provider and it's scheduled for delivery, the status on the Tracking report will be displayed as "Delivered".
2. If the message can't be delivered to the recipient for two hours, the provider will mark it as expired and send an 811 error code back to xMatters.
3. On-Demand will update the notification's status in the Tracking report:
   • If the notification targeted a user, the status becomes "Failed".
   • If the notification targeted a device directly, the status becomes "Pending" (which in turn changes to "Not Notified" once the event is terminated).

A couple of notes about these updates...

• First, the status of the notification is only updated at the Tracking report/user level. For example, if you drill into the Tracking report after the validity period has elapsed, you might see that the Delivery Status for a user is marked as "Failed", but the status for that user's SMS device is still marked as "Delivered". This is because the device's status is based only on whether the original notification was successfully delivered to the service provider - not necessarily the device itself.
• Second, if you're one of our Communication API customers, you'll only see this change if you request delivery status callbacks. Expired messages will have a status indicating that the validity period elapsed and the message was not delivered.

How to stay informed

We've long recommended other, superior notification methods (insert yet another plug for our amazing mobile apps that let you do so much more than just receive and respond to notifications), but we can't deny that SMS is extremely popular and - when it all works correctly - highly effective. 

So if SMS is your preferred method of notification and it's working for you, this change will likely only mean fewer misleading messages. But consider adding a back-up device for those times when SMS just won't do, and make sure to have an email address handy, just in case.