Yeti recently launched an app on a Friday night, in conjunction with Chelsea Handler’s new Netflix documentary, Chelsea Does: Silicon Valley. The team awoke to a crisis Saturday morning, due to an immediate spike in traffic on the app’s SMS and phone call functionality (i.e., Twilio, Nexmo, Plivo). In this talk Rudy will discuss the cause of the problem, how his team fixed it, and what they could have done differently, sharing some Swift code and the lessons learned along the way.

Introduction and Backstory (0:20)

My name’s Rudy, co-founder and head of tech at Yeti, a local product design and development firm, in San Francisco. I’m also the organizer of the San Francisco Django Meetup group. So I’ve done a lot of Python and Django work, and I’ve had a lot of experience building native iPhone apps, Android apps, and back-end work.

I’ll also go into how when I woke up at seven in the morning and things in our production app were broken, what we tried to do to fix what was wrong, why that didn’t work, and multiple attempts we made to get it back up and running that weekend.

About a year ago, someone reached out to us and asked us to be part of Chelsea Handler’s documentary series where she works with an actual development agency to build an app. We taped it a year ago, and it’s on Netflix now. The process was half faked, half real.

The idea behind the app is that you’re in an uncomfortable situation such as a meeting you don’t want to be in, or a bad date. With the app, you can schedule the app to send you a real phone call or text message at some point in time in the future, you will get the phone call or text messages, and it’ll be an excuse to get out of the situation. It’s similar to when you ask your friend to text you messages to get you out of situations.

The app has a list of excuses, you pick an emoji to represent your excuse. You set up a contact for it to come from, and the messages or calls will look like it’s coming from someone with a real picture for that contact. You schedule how long it should come in the future, and then you can kind of like craft a little bit of a story, so you can add a phone call, you can add text messages

Jan 23: Launch Morning (6:14)

I woke up that morning, and I realized that things were not going well with our app that was launched. All the text messages were being undelivered. The onboarding process involved verification, and only a handful of people had verified accounts.

For our telephony service, we used Plivo. In the onboarding process, the iPhone app sends to the server the number that you put in, the server then pings our telephony service, and it creates a unique code that expires after a certain amount of time. Plivo then sends the code in a text message to that user’s number.

Viper (9:40)

Gotta Go (the name of the app) is built using like the VIPER architecture, it’s an acronym that stands for getting as much code out of the view controller. So the idea is View, Interactor, Presenter, Entity, and Router. The Interactor only handles API requests. The presenter is about passing and sanitizing data to the view controller to show. And then we have a list of our view controllers.

Gotta Go’s a fairly small app, there’s maybe only really three of these modules and that’s kind of how we organize our code. If we make a verification request object, we set what the code is, the user input it in their phone, RestKit will handle sending that to the API for us and turning it into JSON.

Adding Contacts (12:06)

We built Gotta Go before iOS came out with a new contacts framework. The contact code we had to write is a bit rough.

We wanted the app to actually save the phone numbers that we’re sending the text messages and phone calls from. We also want to allow you to edit the contact’s name and the photo all from here. Suppose, we’ve already made the contact. Finding the contact can also be a real problem.

As a solution we just deleted the existing contact and then made a new one. It has the same effect, and the user would not know. Another interesting thing is we also launched the support for the US and Canada, so depending on the user and where they’re from, basically using some combination of their phone number and area code to figure out where they are. Canada has the same format numbers as US, luckily that made it easier to implement. So we have to make sure we’re saving the right numbers, that’s just part of a little bit of a complication.

Store numbers appropriate to user’s country

let country = AppModel.sharedModel.getCountry()
let americanNumbers: ContactNumbers = ContactNumbers(voiceNumber: "(310) 269-5471", textNumber: "792-273")
let canadianNumbers: ContactNumbers = ContactNumbers(voiceNumber: "(604) 425-1155", textNumber: "(604) 425-1155")

Find contact with a matching number

static func findContact(number: String) -> ABRecordRef? {
  if Contact.hasAddressBookAccess() {
    var error: Unmananged<CFError>?
    // create address book instance
    let addressBook: ABAddressBookRef = ABAddressBookCreateWithOptions(nil, &error).takeUnretainedValue()
    // get all contacts
    let contacts = ABAddressBookCopyArrayOfAllPeople(addressBook).takeRetainedValue() as Array
    for record in contacts {
      let currentContact: ABRecordRef = record
      let numbers: ABMultiValueRef = ABRecordCopyValue(currentContact, kABPersonPhoneProperty).takeUnretainedValue()
      // loop through contact's numbers
      for (var j = 0; j < ABMultiValueGetCount(numbers); j++) {
        let phoneNumber = ABMultiValueCopyValueAtIndex(numbers, j).takeUnretainedValue() as! String
        // if the phone number matches the first contact number, we've got a contact match.
        if phoneNumber == number {
          return currentContact
        }
      }
    }
  }
  return nil
}

The idea is we need to check to see if this contact already exists in your address book before we go and create a new one. You get the address book, we look through all the records in there, and we try to find the contact that has one of the phone numbers that we’re saving. This could easily break, if for some reason the user went to their contact book and manually edited the contact we had put in there. That’s kind of just something we had to accept. But basically once you find the record, you loop through all of their phone numbers and try to find one of the phone numbers that we’re trying to save.

Set name, image, and numbers on new contact

func createContact() -> Bool {
  let newContact: ABRecordRef! = ABPersonCreate().takeRetainedValue()
  var error: Unmanaged<CFErrorRef>?

  let firstNameSuccess = ABRecordSetValue(newContact, kABPersonFirstNameProperty, self.firstName, & error)
  let lastNameSuccess = ABRecordSetValue(newContact, kABPersonLastNameProperty, self.lastName, &error)

  if let image = image {
    let pngImage = UIImagePNGRepresentation(image)!
    let cfDataRef = CFDataCreate(nil, UnsafePointer(pngImage.bytes), pngImage.length)
    ABPersonSetImageData(newContact, cfDataRef, &error)
  }

  let multiStringProperty = ABPropertyType(kABMultiStringPropertyType)
  let phoneNumbers: ABMutableMultiValue = ABMultiValueCreateMutable(multiStringProperty).takeUnretainedValue()

  ABMultiValueAddValueAndLabel(phoneNumbers, mainPhoneNumber, kABPersonPhoneMainLabel, nil)
  let mainNumberSuccess = ABRecordSetValue(newContact, kABPersonPhoneProperty, phoneNumbers, &error)

  ABMultiValueAddValueAndLabel(phoneNumbers, mobilePhoneNumber, kABPersonPhoneMobileLabel, nil)
  let mobileNumberSuccess = ABRecordSetValue(newContact, kABPersonPhoneProperty, phoneNumbers, &error)

  return saveContactToPhone(newContact)
}

To create a new contact, you have to create a new person record, input their first name, their last name, the image the user uploaded, and then add these ABMultiValue, add value and label. For each number. So we save like one number as their main number, and we save another number as their cell number. And then save contact to phone, this gets that address book object again and then just saves the record to the address book.

Saving & Creating Excuses (15:27)

We allow users to add one phone call and up to seven text messages. We store all the excuses locally in Core Data.

func getExcuses() -> [Excuse] {
  let fetchRequest = NSFetchRequest(entityName: "Excuse")
  let createdSortDescriptor = NSSortDescriptor(key: "createdAt", ascending: false)
  fetchRequest.sortDescriptors = [createdSortDescriptor]
  return (try! managedObjectContext!.executeFetchRequest(fetchReqeuest)) as! [Excuse]
}

We an NSFetch request to basically ask Core Data, “Do you have any excuses stored in there?” We grab them, and we have that list, and we just show that list on that initial scroll view that shows all the emojis and the different colors.

newExcuse = Excuse.createInManagedObjectContext(managedObjectContext!,
  emoji: " ",
  color: Constants.excuseCellColors[5],
  delayTimeMinutes: 0)

let lockedOutMessages: [String?] = [
  "I'm locked out.",
  "Think I may have triggered the silent alert. Can you get here right away?",
  "Last time I did this the cops showed up. Please Hurry!"
]
addMessagesToExcuse(lockedOutMessages, excuse: newExcuse)

When you first open the app for the first time, if we realize you don’t have any excuses or like this is the first time, you’ve installed and opened the app, we set up three default excuses.

Fixing the Fires Part 1 (21:24)

The reason why the text messages weren’t going through was because they were flagged as spam. My first idea was to buy more numbers, I quickly wrote some Python code that when it sends the text message out, it would just round-robin between the numbers.

The problem with this now is that the text message is coming from a number that we didn’t save in the user’s contact. So it is working, and the text messages are getting sent, it’s okay for the verification codes because it doesn’t matter like what number the verification codes come from.

Fixing the Fires Part 2 (23:24)

We got in touch with someone, and concluded what we needed was to buy a shortcode - one that’s five or six digits. Shortcodes are made for applications that send a high traffic of text messages. Throughout the implementation, no one once suggested that we needed a shortcode for the application to work.

We had implemented a new shortcode, and we started seeing users being verified, which was awesome. But then all of their text, their excuses were now coming from the shortcode and not the number saved in their contacts.

Fixing the Fires Part 3 (24:34)

Our iOS app has a hardcoded list of numbers, which not a great solution. You can submit builds to Apple and you request for them to expedite your review process, which is what we did.

What Did We Do Wrong? (25:10)

The contact numbers probably should of been dynamic. What other people normally do is they do round-robin, with hundreds or thousands of numbers, that gets round-robined.

There’s a threshold if you start sending hundreds of text messages within like a half an hour, you will get banned. And so it’s not that Plivo banned us, it’s that the carriers banned us - Verizon, AT&T, T-Mobile, and they’re the ones that actually marked our text messages as spam.

What Did We Do Right? (26:14)

What did we do right? We had all the monitor tools set up, so that I knew that things were breaking.

We used the following:

• Sentry, an error monitoring tool.
• New Relic, a server monitoring tool.
• Fabric, analytics and crash reporting.

Q&A (27:58)

**Q: Why did you use Core Data? **

We have an internal tool which automatically generates the Core Data models. So the idea is we actually auto generate all of our API code via a Python script we wrote, so it looks at our spec, our API spec, it actually auto generates all the RestKit, and all the API code, and it like auto generates the Core Data models and everything.

**Q: Can you respond to a text message? **

So yeah, users totally do it. We never instruct them to.

**Q: What was the rate limit for these phone numbers? **

I don’t think there’s any hard and fast limit, and different carriers would ban them at different times is what we’ve experienced.