The event handling feature in the Realm Object Server allows servers to respond to changes in user data. This allows for implementing business logic in a similar vein to RESTful API calls, but with far less complexity. In this talk from Realm SF, Tim will talk about how event handling in the Realm Object Server works, including an example app that interacts with a Google API.

Introduction (0:00)

I am going to be talking tonight about event handling in our Professional and Enterprise Editions. I’ll talk about what the Realm Platform is really quickly, just in case anyone’s missed what it really does. I’m going to introduce the concept of event handling. This is a feature that we’ve been working on and refining constantly in the Enterprise Edition. We’re really excited to say it’s at a point where we can make it more accessible to other developers through the Professional Edition.

The Realm Platform (2:08)

The point of the Realm Platform is to solve what I would say is a simple problem, but it’s not really a simple problem. In this day and age, thanks to companies like Apple and Google who have created platforms like iCloud and Google’s philosophy of having unlimited data storage for photos on the new Pixel and things like that, users have come to some kind of implicit assumption that mobile devices aren’t really just a thing that holds data. They’re more of a terminal that accesses data from the cloud and what you’ve got on the phone is kind of like just a temporary store for that data.

The greatest example I ever had was I a colleague in Perth who was writing a note storing app. It was kind of like the Notes app on iOS but a bit more feature-rich. But he wrote the app before iCloud became a thing so he never really felt the need to add synchronization or iCloud integration. But one of his users got stuck on the app and didn’t know how to fix it, so they just went, “Oh, that’s easy. I’ll just delete the app and my notes will still be there.”

Sadly, they reinstalled the app and the notes were gone. This user had an implicit assumption that because it was an iPhone app, all the data would be synchronized. But sadly it turns out that that’s a specifically explicit thing that developers have to add. The sad thing is that it ended up with a one star review and no one was happy.

Data Synchronization (3:41)

But this is the thing. A lot of our users assume that this sort of data synchronization is just something that’s always on. The problem is it’s not a thing that’s always on. It’s something that developers have to add and it really isn’t an easy problem.

The main reason why it’s not really an easy problem is because data usually has to be translated from one format to the other to be stored on a server. The most textbook example is normally you have to convert your data from whatever database mechanisms you have on the device, convert it to JSON (a transient transport format), send the data to a server, and the server then has to translate that to its storage format. Then the process has to be reversed if you want to get data back down from the server!

One of my colleagues made this really nice diagram that shows all the stuff that can go wrong in a JSON REST API. In a perfect world you’ll request an endpoint, you’ll get the data, and you can update the UI. But you can see that there’s obviously a lot of things that can go wrong. What you have is a huge amount of complexity to perform such a simple sort of operation.

The Realm way eschews that entire concept of data translation because all we really do is have Realm files on the device and Realm files on the server. What we’re doing is transmitting the changes between the two. There’s no need for a JSON abstraction. There is no need for any kind of data decoding on the server. You’ve got the data on one device and then that same data can then be propagated to other devices.

It’s a lot more efficient, a lot simpler, and it also means it’s a lot easier to add to your apps and it’s a lot easier to work with as opposed to the other mechanisms. The way this thing works is we have Realm on the device, which is the local mobile database, and then we have a thing called the Object Server that’s a process that runs on the server that performs the synchronization. That’s the thing that receives data from the mobile devices and writes that to databases, like copies of the same databases on the server. Then at the same time it will then send that data down to any other devices that are tied to the same account credentials.

Reducing Complexity (6:07)

The crux of this is it makes the whole process a lot easier because there’s no translation involved. It’s the same format going up and down. You can work on your local devices as if you’re saving to a local database and know that whenever you make a write to the local database, that change is being propagated to other devices as well.

Event handling is something we’ve been working on a fair bit. It’s gone through API revisions but it’s at the point where it’s stable enough where we can try it out publicly. The Developer Edition lets you perform data synchronization between the server and devices. The data is on the server and you can see the .realm files on the server, but you can’t open them. If you use our free, open Node.js SDK, that one does not include the synchronization functionality and can’t open synced Realm files.

The Developer Edition lets you do really nice things like ensure that you have consistent state across devices. It’s not too out of the ordinary for a user to have an iPhone and an iPad, and they want to do something with an iPhone and then have that same state replicated on their iPad. They don’t have to worry about synchronization. That’s something the Developer Edition is really useful for.

If you want something more than that, if you’d like to have server-side processing or web services that can respond to changes, this is where the Professional and Enterprise Editions come in. There are a few little differences, but the main differences between the Developer Edition and the Professional and Enterprise Editions is that you get a very special version of the Realm Node.js SDK that has been compiled with the synchronization engine in it so you can open and access these files on the server. The Node.js package runs as a separate process that serves as an extra device, Just like the mobile devices. This one also interacts on a user level with the Realms on the server.

This means you can then write custom business logic that can read data from user Realm files as well as write data to those Realm files, and any changes you write to the Realm files will then be propagated down to their devices. You can also do things like have additional backends, like third-party services, maybe a MySQL database that can then write data to Realm files and that will be automatically sent down to devices as well.

The most powerful thing that you would need is when a user does something that causes the database to change and that change propagates to the server. Your code needs to be notified about that so it can handle the process, do whatever the user said they wanted to do, and you can provide data back to that. This is not polling. It’s a very responsive way to respond to when users do interactions in your app.

// Open a user's Realm server-side in Javascript
let realm = new Realm({
		schema: [{
			name: 'Dog',
			properties: {
				name: {type: 'string'},
				age: {type: 'int'}
			}
			syncUrl: syncUrl,
			syncUserToken: syncToken
		}]
	});

// Add a Dog object to instantly push change to client
realm.write(() => {
	realm.create('Dog', {name: 'Rex', age: 1});
});

The Node.js SDK is very similar to the public one. The only real difference is that you have the syncUrl credentials in the configuration. But apart from that it runs exactly the same as the public SDK.

var SERVER_URL = 'realm://127.0.0.1:9080';
var NOTIFIER_PATH = '.*/myrealm';

var admin_user = Realm.Sync.User.adminUser(REALM_ADMIN_TOKEN);

var change_notification_callback = function(change_event) {
	console.log('A change occurred!');
};

Realm.Sync.addListener(SERVER_URL, admin_user, NOTIFIER_PATH,
'change', change_notification_callback);

To register for user-driven events, it’s a very straightforward API. It’s four arguments and then one method call.

The first argument is the SERVER_URL, which needs to be the base URL of the server. You can’t have any additional bits at the end of the path. It needs to be just the server itself because it checks for the entire set of Realm files in the server.

You then need an admin user to perform read and writes. You get that admin token when you install the Object Server.

The next one is a notifier path, which is a regular expression. You can say, “I only care about when these particular Realm files are changed,” or you can perform granular checks as to which Realm files are being updated.

And then finally we’ve got a function, and then whenever a change is detected by Realm, this function will then be called and you can perform your business logic.

One example myself and my colleagues like to use in sales calls is a very simple concept and something that’s not possible with the Developer Edition. The idea is you have an app that scans a barcode or a coupon code. That coupon code gets written to the Realm file on the device. That coupon code is then sent up to the server. The event handler receives a notification that the Realm file changed, calls extra business logic that performs the coupon code validation. “Is this a valid coupon code?” “Yes, it is!” Then the event handler then writes to the exact same object in the Realm database: “Yes, this is valid.” That gets saved to the server-side Realm and propagated back down to the device-level Realm.

That replicates the experience of a REST API but with far less code and effort. I was going to try and find some more information, but that’s it for event handling. It’s very straightforward. It’s very easy to get started. I’ve been working with it for a long time and it’s very, very quick. All you have to really do is set it up as a separate process.

I’ll do a demo very quickly to show you how. It means you can share data between different services and back ends. A lot of people we talk to say they’d love to be able to have a MySQL integration so this is step one to doing that. You register call backs to be triggered whenever a Realm file changes and when you do make a change on the server, that gets propagated to all user devices straight away.

Demonstrating Event Handling (11:56)

Let me do a really quick demo for this. Originally the demo we had, which is on our Meetup page, was going to be called the TextScanner app. This is a very straightforward app that demonstrates how the event handling works.

The way it was going to work is you take a picture of some text, then that photo is turned into an NSData object that’s saved to Realm. That object is then sent via the Realm Platform up to the server. The event handler converts that data to Base64, which is then sent to Google’s OCR API via JSON. When Google finishes processing, it sends back a REST response, which is another JSON object. The server decodes that and the resulting text is written to the same object, which is then synced back down to the device and the device displays the text that was in the picture.

That was what we were going to show, but something even cooler happened since we announced this event. This little company called IBM got in touch with us and they said, “We’ve been playing with integrating the Realm Platform with Watson, our deep learning machine.” They took our text scanner code and modified it to do something a lot cooler. Watson itself has a Node.js API to take data and send that to Watson for actual image analysis, without even needing to do any kind of REST translations.

So we renamed it the Scanner App because TextScanner was too specific. How it works now is, again you take a picture, you translate it to NSData and put that into a Realm object. It gets automatically synced, but then what happens is it gets encoded to a different image format. I think it gets saved as a JPEG on disk, and that gets sent out to the three separate Watson APIs through their own Node.js SDK from within the event handler. Then as each API asynchronously resolves the information it can get from that image, they will come back at different rates and they all get written back to the same Realm object and then the Realm Platform can then consolidate this information and then send down as needed in either the same events or a consolidated event, what information it managed to get out of there.

Demo (14:14)

Showtime! I’m going to try to do a demo of this and hope it doesn’t fail.

Step one. Let’s set up the Realm Platform. I download the Developer Edition of the Platform from the Realm site. I’ll open that up & create a new admin user. Now we’ll set up the event handler. I’ll go to the Scanner repository. There’s the standard Xcode project, but we have an extra folder down here called the Server.

What we have in here is our Node index.js file, which is where all the logic for the event handler is kept. A lot of this logic is contributed to us from IBM. We also have the package.json file so we know exactly which NPM packages to install. Here is Watson’s Node SDK and the Realm Professional Edition. This .tgz will come when you download the Professional Edition.

Now we’ve got one process, the Realm Platform, listening for synchronization changes between devices, and an extra process, which is now listening for any changes to the Realm file called “scanner”. So if anything happens it will detect that and it will start responding to events.

Now I will build a copy of the scanner on my phone, which is just a single view controller. How it works is you log in through the Realm Platform with this function. When you take a picture it saves the picture as NSData and then how it tracks the state of where the image is in progression is we have this enum at the top that says we’re ready, we’re uploading the picture now. The server’s processing, the server failed or succeeded. How that works is this client can set the status and then back in that script it does Realm queries for objects that are at a specific status.

It tracks the state of objects between the client and the server through that enum. What this means on the client side is you can register for specific events for when Realm properties change automatically. In this case we’re registering a KVO observer for when status changes from the server and then we can respond to that. Hopefully this will work. We’ll find out.

Let’s scan an image of apple pie. Watson picked it up and classified it. There’s an 81% chance that was a picture of some food. Now in the app we have ‘probably food’ and that’s how it works!

I’ll go back to the script and show what the script looks like. We have our dependencies at the top. We have a bunch of credentials like API keys, the admin token, and the same states as we did on the app. The only major thing here is the callback function. That goes through and recompiles the image from the Realm data, saves it to disk, and then uses the Watson SDK to send that information to Watson. That’s all handled in this function.

At the very bottom, you create the admin user, and have a single line of code that sets up the event handler. Whenever it detects a change to the Realm file called “scanner”, then it’ll respond and execute this logic and that’s it. That’s pretty much the Scanner.

Conclusion (22:45)

Realm is hiring. We have a lovely fridge with lots of photos of happy fun times at the company. I definitely recommend joining the company. I’m having a great time.

That’s it for event handling. Thanks for reading!

Next Up: Realm Everywhere: The Realm Platform Reaches v2.0