Introduction
Today I will talk about the implementation detail of the Realm Java. I’m Makoto Yamazaki, Realm Java Team, and I’m living in Japan. Realm has very nice features. This is from my website, and today I’m focusing on the base.
Magic in Simplicity
This code is just creating the object and then assigning the value to the field, and it looks like it’s just assigning the value to the field, but actually it’s passing data into the database file. In addition, just reading the field also reads the data from the database file.
Realm realm = Realm.getDefaultInstance();
realm.executeTransaction(r -> {
final Person person = r.createObject(Person.class);
person.name = "Zaki";
});
Person person = realm.where(PErson.class).findFirst();
Log.d("realm", person.name); // Zaki
realm.close()
The model definition is very simple – this is just defining a field, and it extends a RealmObject. Nothing special, but there’s a lot of magic behind these things.
public class Person extends RealmObject {
public String name;
}
@RealmClass
public class Person implements RealmModel {
public String name;
}
Realm has a lazy loading feature - that means that Realm doesn’t load any data if the user doesn’t need it. For example, this code is creating a query and then adding one condition. Realm does almost nothing, just adding a query condition. findAll() execute the actual query, but it doesn’t load the actual data, it just creates the row index from the table. When the person object is created, still the actual data is not loaded, it’s just creating the proxy object. On the last line the actual value, the name value, is read from the database file. This is lazy loading.
RealmQuery<Person> query = realm.where(Person.class);
.beginsWith("name","Z");
RealmResults<Person> results = query.findAll();
Person person - results.first();
String name = person.name;
Realm Java’s Old Implementation
The old implementation contained some limitations. This was the model definition below – the user needed to follow some rules:
- The field must be private
- The user must define getters and setters
public class Person extends RealmObject {
private String name;
public String getName() { return name; }
public void setName(String name) { this.name = name; }
}
Against that model definition, we create the proxy class by using the annotation processor.
The generated proxy is a subclass of the model class, in this case, person, and it contains a row object. Row is a pointer to the internal database.
public class PersonRealmProxy extends Person {
private Row row =...;
private PersonColumnInfo columnInfo = ...;
@Override
public String getName() {
return row.get(columnInfo.columnNameIndex);
}
@Override
public void setName(String value) {
row.set(columnInfo.columnNameIndex, value);
}
}
This implementation was very simple – it just generates the subclass as a proxy by annotation processor and it overrides getters and setters. Realm returns a managed object, meaning an object that is bound to the database. That is our proxy object, so then proxy just steals access to the properties and reads and writes database instead.
Limitations of the Old Way
That is the basic strategy, and the current implementation is almost the same as that, but we needed to remove some limitations because there are many restrictions in this way:
- Fields must be private. It’s actually okay to public, but many users make mistakes when the field is public, so we prohibit public fields.
- You must have getters and setters, and because we need to override the getters and setters,
- Getters and setters must not have a logic, just read and write the fields.
- No custom methods other than getters and setters, because users can easily make mistakes – for example accessing the fields from custom methods.
- Users must extend a RealmObject class.
Removing Limitations with Bytecode Manipulation
To remove those limitations we use the bytecode manipulation. Bytecode manipulation is modifying the bytecode at compile time or loading time. We are currently modifying them at compile time.
Current Implementation
In our current implementation, users can define the model the same as the old implementation. Currently we’ve removed the private field and getters and setters, but we use the same definition.
Then we generate two proxies, an interface and a class. The interface has the getters and setters for the field, but the name is special, realmGet$name. The setter has the name realmSet$name. The subclass implements those special getters and setters in the proxy class.
public interface PersonRealmProxyInterface {
public String realmGet$name();
public void realmSet$name(String value);
}
public class PersonRealmProxy extends Person implements PersonRealmProxyInterface {
private Row row = ...;
private PersonColumnInfo columnInfo = ...;
public String realmGet$name() {
return row.get(columnInfo.columnNameIndex);
}
public void realmSet$name(String value) {
row.set(columnInfo.columnNameIndex, value);
}
}
We apply bytecode manipulation after compile. First, we add the interface to the model classes to access the property directory from our implementation. By doing this, we can cast the model object to the interface and directly call the getters setters defined by us, and add the getters setters implementation. These getters and setters are just reading or writing the field directory.
public class Person extends RealmObject implements PersonRealmProxyInterface {
private String name;
public String getName() { return realmGet$name(); }
public void setName(Stringname) {realmSet$name(name); }
@Override
public String realmGet$name() { return name; }
@Override
public void realmSet$name(String value) { this.name = name; }
}
The important thing is replacing all accesses to the model’s fields (such as reading or writing the name) with getters or setters calls. By doing this, every access to the fields will use the getters and setters. By doing this, we can allow this definition below.
public class Person extends RealmObject {
public String name;
}
If a user directly accesses the name, we can steal that access by the getters and setters defined in the proxy interface or proxy class.
RealmResults <Person> results = realm.where(Person.class).findAll();
Person person = results.first();
Log.d("realm", person.name);
For example if you have code just accessing a name like the code above, bytecode manipulation replaces the field’s access to the getter call:
Log.d("realm", person.realmGet$name());
How it’s Done
So how do we actually do that? We use two technologies: One is Transform API, provided by Google and Android Gradle plugin, and the other one is Javassist, a library to modify bytecode.
Transform API
When you make app which uses Realm, you can see this line :app:transformClassesWithRealmTransformerForDebug. This means that the transformer is applied to your application.
To apply the transformer, we execute project.android.registerTransform(new RealmTransformer(project)), and the Gradle plugin has an API to register the transformer. When the transformer is registered, the transformer is called at compile time, and then we receive some information for transforming.
Transformer can give some information with context like input classes, referenced libraries, or output directories. In our transformer we add the accessors to the classes and also add the proxy interface to the classes. Each class’s name contains all classes using the application code. Then we modify every field access in every class using methods.
@Override
void transform(...) throws IOException, ... {
// Create and populate the Javassist class pool
ClassPool classPool = createClassPool(inputs, referencedInputs)
inputModelClasses.each {
BytecodeModifier.addRealmAccessors(it)
BytecodeModifier.addRealmProxyInterface(it,classPool)
}
inputClassNames.each {
def ctClass = classPool.getCtClass(it)
BytecodeModifier.useRealmAccessors(ctClass, allManagedFields, allModelClasses)
ctClass.writeFile(getOutputFile(outputProvider).canonicalPath)
}
}
There are many bytecode manipulation libraries - some are low level and some of them are very high level. High level libraries are made for very specific purposes and they’re easy to use. Retrolambda and AspectJ are two common high-level bytecode manipulation libraries. Low level libraries are very powerful, and can do everything the Java virtual machine can do, but they are very hard to use, and require very deep knowledge about commands, bytecodes, and virtual machines.
Realm uses Javassist, which is middle level. There’s no need to know all the details of the bytecode, but we can still use it for very generic purposes. For example, Javassist can add classes and fields. Defining a field or defining class is very easy. When we define a field we just create a Ctfield instance and set the modifier. To define a class we just call makeClass. Adding a field to a class is very easy with addField. You can use Javassist with knowledge of the Java language, but not the bytecode.
Peep the Code
Then this is the actual code in Realm Java:
public static void addRealmAccessors(CtClass clazz) {
def methods = clazz.getDeclaredMethods()*.name
clazz.declaredFields.each { CtField field ->
if (isModelField(field)) {
if (!methods.contains("realmGet\$${field.name}".toString())) {
clazz.addMethod(CtNewMethod.getter("realmGet\$${field.name}", field))
}
if (!methods.contains("realmSet\$${field.name}".toString())) {
clazz.addMethod(CtNewMethod.setter("realmSet\$${field.name}", field))
}
}
}
}
We call the addRealmAccessors from the transformer, and the first line collects all the names of the declared methods. Then we get the declared fields in the class, and if it’s a modelfield, we add the special getters and setters to that class.
This is the code to add the accessors:
public static void useRealmAccessors (CtClass clazz, List<CtField> managedFields) {
clazz.getDeclaredBehaviors().each { behavior ->
if (
(
behavior instanceof CtMethod &&
!behavior.name.startsWith('realmGet$') &&
!behavior.name.startsWith('realmSet$')
) || (
behavior instanceof CtConstructor
)
) {
behavior.instrument(
new FieldAccessToAccessorConverter(managedFields, clazz, behavior))
}
}
}
We’re getting the declared behaviors. If it’s a method but the name of the method doesn’t start with realmGet or realmSet, the constructor will replace their access to the fields.
This is actual converter code:
private static class FieldAccessToAccessorConverter extends ExprEditor {
@Override
void edit(FieldAccess fieldAccess) throws CannotCompileException {
def isRealmFieldAccess = managedFields.find {
fieldAccess.className.equals(it.declaringClass.name) && fieldAccess.fieldName.equals(it.name)
}
if (isRealmFieldAccess != null) {
def fieldName = fieldAccess.fieldName
if (fieldAccess.is.Reader()) {
fieldAccess.replace('$_=$0.realmGet$' + fieldName + '();')
} else if (fieldAccess.isWriter()) {
fieldAccess.replace('$0.realmSet$' + fieldName + '($1);')
}
}
}
}
If it’s access to the model, and if it’s a reader, we replace the read access to the method call realmGet$ + fieldName. If it’s a field write, we replace it with a setter call.
Conclusion
You can find everything here in our repository because Realm is open source. Everything I’ve explained is all open, and you can read the code yourself.
About the content
This content has been published here with the express permission of the author.
