My name is Yusuke Kita, and I work at a Japanese company based in San Francisco named Mercari. Our product is marketplace applications (like eBay), and we’ve been internationalizing our app for a U.S. market.
I’d like to share my experience on protocol buffers, which have many benefits.
Web APIs
Many apps have network communications - such as Twitter and the GitHub client. The calls to APIs usually returns with JSON from the server. Though JSON is a standard way to serialize data, I feel it’s not Swifty enough when I write Swift code.
Consider the following HTTP request when convert JSON object to data a type.
let json = ["id": 1]
let data = try? JSONSerialization.data(
withJSONObject: json,
options: []
)
var request = URLRequest(url: url)
request.httpBody = data
This request is to fetch user information from the server after sending a userId. The key is a string, and the ID itself is an int. Convert this to a data type, and set it to the httpBody.
With the response, convert the data type to a JSON object through URLSession.
{"user": {"name": "Yusuke"}}
URLSession.shared.dataTask(with: request) { (data, _, _) in
let json = (try? JSONSerialization.jsonObject(
with: data!,
options: []
)) as? [String: Any]
let user = json?["user"] as? [String: Any]
let name = user?["name"] as? String
}
Here, if I mistyped the ID (to a string), or had the incorrect case, it will fail. In Swift, everything is a type, and because you must use an Any
to work with JSON, I believe it’s not as Swifty as it can be.
Protocol Buffers
Protocol buffers, a.k.a. Protobuf, was developed by a group in 2008. Protobuf is about serialization formatting.
Example
Reconsider the HTTP request where you convert the JSON object to a data type with Protobuf. Protobuf allows you to define your custom type.
custom type
let userRequest = UserRequest.with {
$0.id = 1
}
let body = try? userRequest.serializedData()
var request = URLRequest(url: url)
request.httpBody = body
You don’t need to write a key name with the string. Protobuf also has serialization methods, serializedData()
. By calling it, it will send to httpBody.
With the HTTP response, using Protobuf, you could have a UserResponse type, which I’ve defined myself. The UserResponse type has user type containing the user names property. All you have to do is to deserialize the data type to UserResponse type so that you could access the property.
UserResponse(user: User(name: "Yusuke"))
URLSession.shared.dataTask(with: request) { (data, _, _) in
// custom type
guard let response = try? UserResponse(serializedData: data!) else {
// error
return
}
let user = response.user
let name = user.name
}
Protobuf is more Swifty, because we have types.
How Does this Work?
You need to install the Protobuf compiler (the latest version is 3.3). You will then also need a plug-in for use with Swift that was officially released by Apple.
Protobuf supports both Objective-C and Swift; you can also use both at the same time!
There are three parts to implementing Protobuf:
- Define message types
- Generate code
- Serialize/Deserialize
Message type defines data structures in .proto files, and it’s key-value pairs based code.
Let’s take the iOSCon profile page as an example in making a .proto file. It has user ID, user name, introduction for URL, user type. Each field has a type (int, string, or enum).
talk.proto syntax = "proto3";
import "user.proto";
message Talk {
int32 id = 1;
string title = 2;
string desc = 3;
User speaker = 4;
repeated string tags = 5; // Array
}
We also have talk.proto file. It has ID, title, description, the user type, and also tags. This is similar, but since I used user type, I import user.proto at the top and you have to specify which version of protocol buffers itself (the latest version is free). That’s why I write proto3 on the top.
Protobuf compiler generates code from .proto
Protobuf generates all the code from .proto
file.
The basic types they support are: int, Bool, String, Dictionary. It supports other languages such as: C++, JavaScript, Ruby, Rust.
These Swift features are from the official program from Apple:
- It’s struct so we have data structures
-
struct
, notclass
. - we could define enum but
RawValue
is going to beint
always.
Each property has default values, which means you cannot have an optional type for the property itself. If you check users name property string, you have to see also if it’s empty or not because if it’s not optional it has the empty strings as the default.
Serialize/deserialize
Serialization is already implemented Protobuf itself, and you can get JSON or text as a result. All you have to do is call seralizedData();
Binary Encoding
message Talk {
int32 id = 1; ← // Field number
string title = 2;
string desc = 3;
User speaker = 4;
repeated string tags = 5;
}
Here, we have group of types. If it’s int, it’s going to be 0; if it’s string it’s going to be 2.
Consider this struct:
message Test1 {
int32 a = 1;
}
test1.a = 300
// encoded message
08 96 01
08 // field number and wire type
96 01 // value which is 300
We said 300 to everybody. A is int, which will result in 8, 96, and 1. The first two digits represent a combination of field number and wire type.
Because of the binary encoding, the size would be smaller than JSON and everything is numeric. And as a result, this gives us high network performance.
You Might Have Concerns About…
Versioning
Protobuf has free backward compatibility. Let’s say if the response has a non-field, it ignores it. Or if something is missing from the response, it’s going to default the value. You can easily add new properties from the server-side, or delete existing, unused property on the client-side.
Coexistence of protobuf & JSON
You can have both JSON and Protobuf because you can look at the content type in the HTTP header. We can set the return as JSON or Protobuf.
Let’s look at the cons
Protobuf not human-readable because it’s binary data. If you want to see row value in the console, you have to deserialize it to be a custom type.
Working with Protobuf is time-consuming at the beginning because you have to know all about Protobuf, in addition to other platforms as it’s not strictly used on iOS project.
Codable
Recently, Swift announced Codable. It allows you to do type safety in terms of serializations. It’s a protocol, and you can do the same thing with Codable as you can with Protobuf. This is officially from the Swift Foundation.
Q & A
Q: Can you use Protobuf with Ruby? Yusuke: Yes, Ruby’s supported.
About the content
This talk was delivered live in June 2017 at AltConf. The video was recorded, produced, and transcribed by Realm, and is published here with the permission of the conference organizers.