Mozilla Weave: A Bird’s-eye View

For the past week at Labs, many of us have been sprinting on getting Weave to release 0.2, which aims to be able to allow users to sync all their information between multiple browsers. This is something that Google Browser Sync has been doing for a while, but Weave’s adding an interesting twist to it all: because it aims to maximize user privacy, the data stored on the cloud is basically encrypted by a passphrase that only the user knows. As a result, there’s no way for even the owner of the cloud to mine their data. And because Weave also aims to allow users to optionally share parts of their data with other users and third parties, it effectively becomes a tool for all kinds of secure communications.

Because Weave is fully open-source and uses some pretty standard technologies to achieve what I think are some pretty cool things, I thought it’d be useful to write a bit about how it currently works. Many thanks to Dan Mills for explaining most of this to me.

The Weave server at services.mozilla.com is, for the most part, just a WebDAV server; this design was intentional, to allow for straightforward scalability and to let anyone to easily set up their own Weave server if needded. Another part of the reason for such a “thin server” is out of necessity, though: since the server can’t understand what it’s storing—that’s part of the whole “maximizing user privacy” thing—virtually all of the syncing logic has to be done on the client-side.

Snooping Around

After installing Weave and registering on services.mozilla.com for an account and syncing my data for the first time, I snooped into my user directory. This kind of hackery was surprisingly easy—I just went to services.mozilla.com, clicked on the “Sign in to Weave” button, and entered my email address and password.

The contents of my user directory were as follows:

meta/
  version
private/
  privkey
public/
  pubkey
user-data/
  bookmarks/
    deltas.json
    keys.json
    snapshot.json
    status.json
  history/
    deltas.json
    keys.json
    snapshot.json
    status.json

The meta directory is just meant to contain metadata about the user account; right now it just contains the unencrypted file version, which contains a single number, 2. This is the version of the Weave directory schema that the user directory is stored in, and the Weave client checks it to make sure that it’s “on the same page” as the data store, so to speak.

The private/privkey file contains the user’s encrypted private key, and the public/pubkey file contains the user’s unencrypted public key. This is where things get a little complicated.

Asymmetric Cryptography

In order for users to share some of their data with each other and third parties while maintaining maximal privacy and control over all their data, all of a user’s data can’t just be encrypted with their encryption passphrase; otherwise, someone who wanted to share their bookmarks with a friend would have to give them their encryption passphrase, which, to quote Eran Hammer-Lahav, is much like going to dinner and giving your ATM card and PIN code to the waiter when it’s time to pay.

To get around the issue of privilege delegation, Weave uses Asymmetric Cryptography, the same kind of cryptography that powers secure transactions on the Web. In short, others can encrypt a piece of data with your public key, and you can decrypt it with your private key and vice versa. As the names suggest, the public key is something that is publicly available to everyone, and a private key is something that only you can access. And in Weave, you’re literally the only one who can access this private key, despite the fact that it’s stored on the Weave server at private/privkey, because your private key is itself encrypted using your encryption passphrase.

The nice thing about Weave, though, is that it doesn’t expose any of this confusing public-private-key mumbo-jumbo to unsuspecting end-users who just want to share stuff with their friends; it’s purely an implementation detail that only developers and security experts need to be concerned with. I’ll address how public-key cryptography is actually used in delegation momentarily.

Data Types, Snapshots, and Deltas

The user-data directory is just an umbrella directory; each subdirectory contain the data needed for synchronizing and sharing a particular kind of data. The bookmarks folder, as you might guess, contains bookmark data, and history contains browsing history. More folders will be added as new data types become available to Weave, and users or organizations will even be able to make their own data types if they want.

The status.json file contained within each data type directory is an unencrypted JSON-formatted file that contains metadata about the data storage: the number of items stored, the schema version of the data type being used, and so forth.

The encrypted snapshot.json file in each data type directory contains the most recent “snapshot” of the data store for a data type. For instance, it may contain all of my bookmarks from two days ago, when I first started up Weave on one of my computers and synced with it. The encrypted companion file, deltas.json, contains a sequence of actions that can be performed on the snapshot to make it arrive at the current state. Continuing with my bookmark example, the deltas.json file may contain one “add” action for a bookmark that I added yesterday afternoon and another “remove” action for a bookmark that I removed this morning. The “bookmark” data type engine in the Weave client is responsible for examining the current state of the browser and these deltas, resolving any conflicts between them, and updating them as needed (asking the user for help if absolutely necessary).

Choosing Encryption Carefully

The interesting thing is that status.json and deltas.json aren’t actually encrypted using the user’s passphrase, because the user may want to share them with someone else; for the same reason, they’re not encrypted with the user’s public key. And they’re obviously not encrypted with the user’s private key, or else everyone in the world would be able to read them!

Instead, these files are encrypted using a brand-new, completely random shared key that is stored in the one file I haven’t yet discussed: the unencrypted keys.json file, contained within each data type directory. This file maps individual Weave users to encrypted versions of the same shared key—encrypted versions that are intended for their eyes only, because each one is encrypted using the public key of the user it’s mapped to.

Work in Progress

There’s a lot of details here that I haven’t covered; for instance, you might notice that under this schema, while it’s possible to share one’s bookmarks with another without sharing their browsing history, it’s not possible to share some bookmarks but not others. That’s one of the things we’re currently working on, along with a bunch of other things: syncing more data types, moving from OpenSSL to NSS for cryptographic operations, and more. My description is also just a snapshot of what Weave looks like today, and because it’s still in alpha, it could change drastically over the next several months. If you have any questions or are interested in helping out, please feel free to leave a comment on this blog, post to the Weave Forum, or join #labs on irc.mozilla.org.

14 Responses to “Mozilla Weave: A Bird’s-eye View”

  1. Jigar Shah Says:

    Can i setup my own weaves server?

  2. Dante Says:

    “And they’re obviously not encrypted with the user’s private key, or else everyone in the world would be able to read them!”
    It should read “or else nobody in the world”

  3. Atul Says:

    Jigar: I believe you can; you should be able to just set up your own WebDAV server and point Weave to it under the “Advanced” tab in Weave Preferences.

    Dante: Are you sure about that? I believe that anything encrypted by a private key is able to be unencrypted by its corresponding public key, and since the public key is public, anyone should be able to read it. As a side note: you’d think this would be fairly useless, but I believe it’s actually how digital signatures are done, since something I decrypt with your public key is guaranteed to be from you (unless your private key has been compromised).

  4. Dante Says:

    Right, private key encrypting is just signing.
    Mea culpa.

  5. Christopher Blizzard » Blog Archive » some random mozilla-related links for your enjoyment Says:

    [...] Atul Varma talks about how Weave works and some of the wider goals for the project. [...]

  6. Richard Says:

    Hi,

    Anything encrypted with a Private Key, can ONLY be decrypted with the same Private Key. BUT, you don’t need to encrypt with a private key because ANYTHING ENCRYPTED WITH A public key CAN ONLY BE DECRYPTED WITH THE corresponding private key.

    So anyone encrypting something with the Public Key of person X, will only be decryptable by someone knowing the Pass-Phrase of the Private Key of person X. If person X correctly secured is Private Key’s Pass-Phrase, then he should be the only one able to decrypt.

    Now, when signing, we use our private key to create a small signature, so that the other person can verify it by using OUR PUBLIC key to create the same signature and COMPARE IT with the one we attached with the message.

    Of course, signing and encrypting DO NOT USE THE SAME ALGORYTHMS !

    Any question ???

  7. Python-SpiderMonkey Resurrected at Toolness Says:

    [...] I mentioned in my overview of Weave, because the client’s data is decrypted client-side, almost all synchronization logic needs [...]

  8. Robin Says:

    Hi,

    Nice article :) The public / private key arrangements you describe sound a lot like GPG – I’d be interested to know how similar or different the 2 systems are?

    –Robin

  9. dugommier Says:

    Well… I just did all that… read your comments… but can’t understand at all how to USE the whole thing, how to SEE my bookmarks, how to install them on some machine and so on… Were the hell is it possible to get some very simple Help giving some clues to the basic user ?

  10. Remco Says:

    “Can i setup my own weaves server?”

    Yes you can. I wrote a nice article about building your own Weave server at http://remcobressers.nl/2008/07/create-your-own-mozilla-weave-server/
    Don’t hesitate to contact me if you can’t figure it out.

  11. No Fennec for Android! | AndroidGuys Says:

    [...] Mozillaites are currently also involved in a research project christened the Weave, that allows us to remotely access our desktop browser’s browsing history, bookmarks, and [...]

  12. No Fennec for Android! | The Androider Says:

    [...] Mozillaites are currently also involved in a research project christened the Weave, that allows us to remotely access our desktop browser’s browsing history, bookmarks, and [...]

  13. Philip Says:

    How will this enable me sell more stuff or make more stuff from fewer inputs. To me it sounds like computing’s equivalent of the finance sectors credit default swaps.

  14. Weave Intro & UI, Awesome Perf. (2009/9-1) | edilee Says:

    [...] into coding, I got a whole look at the Weave architecture to better understand what pieces od what: A bird’s-eye view [toolness.com] and Client APIs [wiki.mozilla.org]. Also, Weave is written with a pattern called [...]