Contacts in the Browser

I'm happy to announce the launch of the Contacts in the Browser project! You can read about it at the Mozilla Labs blog. I've been working on the extension for the last couple months and am glad to get it out in public where people can play with it.

I'll use this space to talk a bit more about the technical underpinnings of the extension.

In essence, Contacts is a local database with some specialized logic to handle duplicate detection, an API for importing records, a browser overlay for form autocompletion, and a security-limited API to query records from web content. I'll take each piece in turn.

The database

The internal database is a Javascript wrapper on the Firefox Storage Service. The Firefox service, in turn, wraps a SQLite embedded database library. We program the database using standard SQL, so we have your standard model of tables, indexes, and queries in there.

Our data model is effectively schema-less. Each person is represented as a GUID and a JSON blob. The people table is declared like this:

CREATE TABLE id INTEGER PRIMARY KEY, guid TEXT UNIQUE NOT NULL, json TEXT NOT NULL

We then have a data-driven scheme to create index tables from the JSON representation, so we have some additional tables and indexes that look like this:

CREATE TABLE displayName (id INTEGER PRIMARY KEY, person_id INTEGER NOT NULL, val TEXT NOT NULL COLLATE NOCASE);
CREATE INDEX displayName_person_id ON displayName (person_id);
CREATE INDEX displayName_val ON displayName (val);

It's the responsibility of the application logic to update the index tables, but we have some helper methods for that. Right now we're indexing on displayName, givenName, familyName, and emails. Callers (who must have chrome-level privileges; i.e. be an extension, not web content) can insert records with the add() method, or update records with update().

Working with sqlite is sometimes great and sometimes a bit of extra work. We learned early on that we had to be careful with our transactions, because committing a transaction on a laptop hard drive can take 10 milliseconds or more. In an early version, we were opening a transaction for each new person record, so importing an address book with 1000 contacts was taking upwards of 20 seconds. We eventually combined the import into a single transaction, which cut the runtime by about 100x.

The most interesting feature of a contact database isn't bulk import, though -- it's de-duplication. It is very common for users to have many repeated copies of contact data scattered all over their computers and the web. We would like Contacts to ultimately help with this problem, which means we need to be able to merge and combine data from multiple sources.

Our current implementation has a pretty trivial union algorithm. It simply compares the email addresses and full displayName of each person against the whole list, and merges the records if it finds a match. This has several problems:

1. It could miss two records for the same person that happen not to have a value in common, either because they are data islands or because of trivial differences in their displayName (e.g. Bob and Robert)
2. It could combine two people that happen to share a data value (e.g. a couple that uses the same e-mail address)
3. By merging the records, it loses the attribution of data back to its source. That makes it harder to implement refresh and to explain to the user what happened.

About the JSON blob

We needed to pick a representation format for the user data that we put into our JSON blob. We settled on Portable Contacts. We have a bunch of thoughts about how to construct a representation system that allows multiple schemas to co-exist, but we needed to pick one to start with, and PoCo (as it is known by it's fans) hits all the right points of openness, adoption, and extensibility.

That said, our current use of PoCo is probably not quite right. We have a notion of multiple documents inside the JSON blob, but we don't have a clean mapping for which service provided what. The next release of Contacts will contain a revision to the schema to handle service attribution better. (Current work on the Mozilla wiki, here)

The Importer system

The second major piece of Contacts is the Importer system. A generic ImporterBackend object is provided as the parent class for implementations of an Importer, which is registered with the PeopleImporterSvc service.

The contract of Importer.import() is pretty simple: it takes a progress function and a completion callback, and does whatever is necessary to get some contacts into the database. Callers are encouraged to call People.add() only once, since it runs much faster that way, and should provide feedback to the user through the callback functions. In our implementation, the messages passed through those functions are rendered into the Contact Manager user interface.

We did a couple importer implementations to get a feel for how they would work.

• The Gmail importer was quite easy, because Gmail provides a simple POST interface to retrieve contact book data. We made a bad decision, in hindsight, to use the VCard export mode instead of the PoCo one. That will probably change in the next release.
• The Twitter importer is interesting because it uses the Twitter API instead of going through the user website. That required us to integrate with the Password Manager to get the user's username and password. We should also provide a form for the user to provide their username and password interactively if they would prefer not to save the credentials in the Password Manager. The twitter API doesn't give us the user's email address, so it's a case where the person-de-duplication logic is important (and usually insufficient, in our current form).
• The last one we did was a native (OS-level) address book. The MacOS address book API is well-documented and has been stable since MacOS 10.2, so we picked that one and ran with it. The Windows API is a bit more complicated, since there were some changes in the Vista timeframe, and we'll get to it later. For this one, we had to write some native C code and compile it into a binary extension library.

We also did a Gravatar importer. Since we wrote this one, I've come to think that this is actually our first instance of a new object, which I'm calling a Discoverer. This is an object that, given a person or a piece of personal data, retrieves some other chunk of personal data. In the case of Gravatar, we examine the list of email addresses to determine if any of them are associated with a Gravatar, and, if so, we return the image URL of the Gravatar to the People service.

Discoverers will perform their discover() method on a person and return one or more records containing new data about that person, which can then be merged into the person record. We can do this automatically on all contacts, or interactively when we view a single contact.

There are a lot of interesting possibilities for discovery. We can do service-specific discovery, like Gravatar and Flickr, or generic discovery protocols like WebFinger and the Google Social Graph API. Because the discovery object can run in the user's web context, it can be used for search into restricted social networks, such as to discover a Facebook page for a contact. Coming soon.

The form autocompletion overlay

The extension includes the PeopleAutoCompleteSearch object, which implements the Firefox Autocomplete interface to provide form autocompletion.

One limitation of that interface is that we only get access to the form name. We don't get the type or the rel attribute, so we can't detect email fields in every case. But, since those fields are frequently named "email", "e-mail", "recipient", or "recipients", we look for those names, and pop up the autocompletion if we find a match.

The Content API

The last piece of the extension is a Javascript API. We dynamically extend the navigator Javascript object by using a XUL injector technique, which is an advanced bit of Firefox hackery. What we do is watch for a state that indicates that the page is loaded, and inject a new function into the page before the rest of the page runs. The function is created dynamically and returned as a closure, so we can restrict the scope of access to internal data.

What this means is that when the page calls navigator.people.find(), we can check with the Permission Manager to see whether the user has granted privileges to the Contacts system, and then check the internal Contacts database to see if field-level permissions have been saved. If the permissions are all there, we run the query and return. If they're not, we can pop up a XUL-based modal dialog that puts the user through a permission flow to grant (or deny) access to the contact database.

What's Next?

I think there are a huge range of exciting applications that are enabled by getting people into the browser.

Address book functionality is only the beginning, though it is an exciting step. I imagine email, phone, and physical address auto-completion and hotlinking everywhere. I want websites to stop asking for my credentials, or even an OAuth ticket, at other sites, simply to get access to my friend list.

There are lots basic address book capabilities that need doing: groups, multiple values for image, disambiguation based on common nicknames, per-service refresh (with timestamps to keep track of how stale data is), bulk edits.

In a future release, I will be adding support for hashed email access. In nearly every social networking setup task, there is no need for me to disclose my peers' real email addresses -- instead, I just need to disclose a stable, unique token for them. If a site could simply retrieve the hash of every email address in my friends group, they could discover all the people that have an account on the website, without actually knowing their names or addresses.

The discovery system has some exciting potential applications: by creating a user interface for web-based people discovery systems, we can help decrease the isolation of personal data into islands scattered across the web.

Much work remains to do! Thanks to everybody that's provided positive feedback on the initial release.