Couches in Browsers
A little while ago, Vladimir Vukićević wrote an excellent blog post outlining the reasons why he’s not a fan of exposing a specific implementation of SQL to Web Content.
I agree with everything he says in his post; I’ve also been a fan of CouchDB for some time. A CouchDB-like API seems like a nice solution to persistent storage on the Web because so many of its semantics are delegated out to the JavaScript language, which makes it potentially easy to standardize, as well as easy to learn for Web developers. Furthermore, CouchDB’s MapReduce paradigm also naturally takes advantage of multiple processor cores—something that is increasingly common in today’s computing devices.
To explore the possibility, I decided to spend some time prototyping a JavaScript implementation of CouchDB, which I’ve dubbed BrowserCouch. It’s intended to work across all browsers, gracefully upgrading its functionality when support for features like Web Workers and DOM Storage are detected.
Right now this is very much a work-in-progress and there isn’t anything particularly shiny to see; just the test suite and the semi-large data set test. In the future, it’d be great to make CouchDB’s Futon client work entirely using BrowserCouch as its backend instead of a CouchDB server, but that’s a ways away.
If you’d like to see some code samples of what the BrowserCouch API currently looks like, check out the annotated source code for the test suite. You can also read the primary source code documentation for more on BrowserCouch’s implementation. And if you’re interested in hacking on the code, the Mercurial repository is right here.
April 16th, 2009 at 4:02 pm
Cool idea. Dunno if you’ve seen this:
http://labs.mudynamics.com/2009/04/03/interactive-couchdb/ which is now under Apache license on GitHub:
http://github.com/janl/jscouch/tree/master
April 16th, 2009 at 7:50 pm
This is great!
If you could add replication this would give the current Weave stuff a run for it’s money
April 17th, 2009 at 12:28 am
Cool. I agree that the Map/Reduce paradigm is certainly intuitive for javascript development. It breaks my heart to have to deal with SQL for ajax development!
However, there are performance issues of this kind of approach. The good thing about SQL databases is that indexes can be used to give O(log n) access to certain queries. (Ok we are probably talking about tiny databases on the client).
Whilst I can imagine ways of allowing map/reduce style functions to use indexes as well, this gets complex very quickly. Whats your plan?
April 17th, 2009 at 5:37 am
[...] Atul releases BrowserCouch http://www.toolness.com/wp/?p=580 [...]
April 17th, 2009 at 6:20 am
I think the MapReduce angle here is really very very promising. Not only does it scale well across multiple local cores, but extending it to operate on remote resources can be pretty seamless as well.
Thanks for doing this, I think it’s a great contribution to the search for sophisticated, web-appropriate storage capabilities.
April 17th, 2009 at 10:25 am
Please show this to whatwg! Maybe this part of HTML5 can still be changed!
April 17th, 2009 at 12:44 pm
Please do send your findings to the working group. It would be great to discuss something concrete; right now, the only concrete proposal is the SQL stuff, which is why we’re using it at the moment. If people can be convinced by your proposal, maybe we’ll use that instead!
April 17th, 2009 at 3:02 pm
@Atul
Great stuff! As per @kowsik’s comment, if we can merge efforts to get a pure-JS CouchDB implementation (or a close enough approximation to run Futon and other tutorials (like the interactive jscouch on Github), that’d be great.
@Alastair: CouchDB stores intermediate and final MapReduce results in a b+-tree with amortized O(log n) access. Not that Atul is proposing embedding CouchDB, but the concept can be borrowed.
But then, why not do the embedding?
April 19th, 2009 at 6:29 pm
Browser-based databases will further encourage the undesirable Web application monoculture….
Lately, there has been a trend towards using the Web browser as a general application development platform. This is unfortunate in many ways, namely due to the poor nature of the browser for such a purpose. A system inherently designed for displaying d…
April 21st, 2009 at 4:34 am
[...] Varma, a fantastic colleague in Building “S” at Mozilla, has been playing with a JavaScript implementation of CouchDB called BrowserCouch: BrowserCouch is an attempt at an in-browser MapReduce implementation. [...]
April 21st, 2009 at 1:27 pm
[...] recently undertaken an experimental re-implementation of CouchDB in the browser to explore the possibilities of using a simpler standard that delegates many of its [...]
April 22nd, 2009 at 2:13 am
[...] Varma, a fantastic colleague in Building “S” at Mozilla, has been playing with a JavaScript implementation of CouchDB called BrowserCouch: BrowserCouch is an attempt at an in-browser MapReduce implementation. [...]
April 24th, 2009 at 7:38 am
[...] we have seen good discussion on the W3C Web Storage specification. There has been some recent push back on the use [...]
May 9th, 2009 at 3:50 am
[...] Mozilla Labs 的 Atul Varma 发表了一篇博文[看这里]，探讨使用一个浏览器内置的类似 CouchDB 的系统[看这里]来作为浏览器离线存储方案的可能。——于无声处闻惊雷！我得说，这真是一个天才的点子。 [...]
May 20th, 2009 at 8:32 pm
Hey Atul, I always knew you were headed for greatness.
I am a teacher now. Who would have thought. I guess after driving all of my teachers nuts I developed a soft spot for teaching
Josh Mendelson
June 1st, 2009 at 12:02 pm
That’s interesting in how this concerns web storage. Could the principles of CouchDB apply in another area, browser cache? Mozilla ought to improve the browser cache system. CouchDB struck me as an interesting idea. In the end, I don’t think it would work quite right to simply run CouchDB in Erlang inside Firefox. OTOH, I still think investigating a new, document-oriented database approach might produce an innovation and improvement to the cache system. Re-writing CouchDB in Javascript might be feasible. I know this is a bit off-track from what you are discussing here, but I thought I’d mention it. Good luck.
June 8th, 2009 at 5:05 pm
I’ve been thinking about BrowserCouch a lot lately and plan to talk about it at my #NoSQL talk on Thursday.
Hopefully I’ll see you there!
July 12th, 2009 at 10:12 pm
This is really great!
I think this would go very well with PersistJS.
http://pablotron.org/software/persist-js/
It’s a JavaScript library that basically wraps all the client-side storage APIs into one object. It works with localStorage/globalStorage, HTML5 SQL storage (Safari), Google Gears, even IE’s userData behavior, and a Flash-based backend, and cookies as a fallback. So it covers just about every browser.
Interestingly, PersistJS’s storage object almost already implements BrowserCouch’s Storage interface as it is, except that its methods are called get and set instead of get and put.
July 27th, 2009 at 3:04 pm
[...] is currently working on a JavaScript implementation of Couch. Among other things, this would give you the ability to replicate your CouchDB to a local version [...]
August 8th, 2009 at 1:20 pm
What is the license for all this? Didn’t see it anywhere.
I’d like to use this to run a few MapReduce jobs on my blog. But I’d also like to post the result of my toils so that others can do the same. Is this okay?
August 12th, 2009 at 10:11 am
Hey Jonathan, sorry about that! I really need to put the MPL on it–you’re absolutely welcome to use it.
April 6th, 2010 at 1:57 pm
Atul,
This is good work. I was very excited to find it because I need just the thing, and was even prepared to build it myself. But I browsed the code and it seems that replication is not there yet. My main goal is to replicate between BrowserCouch and a real couch instance. I would like to contribute to this project to make that happen. However I couldn’t figure out how to fork your HG project. Could you let me know what’s the best way I can work with you on the code, and what’s the overall status of the project?
April 7th, 2010 at 9:11 am
Hey Toby,
Actually, Mikeal Rogers recently took up the source code–I haven’t had any time to work on it lately, unfortunately–and I believe he’s added replication, too! You can see the latest work here:
http://github.com/mikeal/browsercouch
April 15th, 2010 at 10:20 am
Hi Toby, Atul,
I’ve been working on BrowserCouch, and have indeed got replication working. Here’s my blog about it : http://peterbraden.co.uk/article/browsercouch-intro
And here’s the repo:
http://github.com/peterbraden/browsercouch
Would love your feedback.
Cheers,
Peter