The Challenges of Developing Offline Web Apps

As I mentioned at the end of my last post, there’s a lot of usability problems that make writing an offline web app difficult.

When writing a “native” client-side app using technologies like Microsoft .NET or Apple’s Cocoa framework, it’s assumed that everything your program is doing, and everything it needs, is already installed on the local device. Anything not on the local device needs to be explicitly fetched over the network. Any app is therefore “offline” by default, and it’s very easy to tell, both when reading and writing code, when it needs to access the network.

Understandably, this makes it trivially easy to write an offline native app. One’s very first app works offline, unless there’s something related to the app’s purpose for which network access is obviously required.

On the other hand, offline web apps build on the already confusing, frequently opaque world of resource caching. They also break developer expectations in various ways that I’ll explain shortly. Furthermore, no browsers except Chrome provide tools for introspecting and understanding the behavior of an offline application, which adds to developer frustration.

Problem One: Simple Things Aren’t Simple.

Let’s say you’ve made a simple web-based currency converter, which is fully self-contained in a single HTML file and never requests any resources from the network. A browser won’t let it be accessed while the user is offline unless you do a few extra things:

  1. Create a manifest which enumerates every resource that needs to be available to the app.

  2. Make sure the manifest is served with a mime type of text/cache-manifest, which is done differently depending on the web server software you’re using.

    As I mention in my post On The Webbyness Of An Installable Web App, this single requirement vastly complicates the scope of the problem by requiring developers to understand the HTTP protocol layer and their web server software. Furthermore, Chrome’s Web Inspector also appears to be the only tool that provides helpful error feedback if the manifest is of the wrong type—developers using Safari, Opera, or Firefox will likely be confused as their application silently fails to be cached.

  3. Add a manifest attribute to their HTML file’s <html> element that points to the manifest.

Doing all this will allow you to deploy your simple currency converter app for offline use; it’s clearly much harder than writing an offline native app. But what’s worse is that adding offline support complicates further development of your app in a number of ways.

Problem Two: Your Development Workflow Is Now Broken.

Let’s say you make a simple typo fix in your HTML file. The fix won’t actually propagate unless you also alter your app’s manifest file in some trivial way; even then, because the cache manifest is checked after the cached version is loaded, the page needs to be reloaded an additional time to activate the new version.

Needless to say, this complicates the development workflow for the app, as it impacts the edit-test cycle. Workarounds exist, of course; for All My Etherpads, I’ve added an --ignore-cache-manifests option to the bundled Python development server that allows the server to pretend that the manifest doesn’t exist, so that the app’s offline cache is obsoleted and code changes propagate instantly. Even though such workarounds make it possible to develop as one normally does, developers still must remember to make a trivial change to their manifest file when they re-deploy, or else their existing users won’t see their changes.

Problem Three: Your App Can’t Access The Network When Online.

After creating your first completely offline app, you might think that adding a manifest file would just let you specify what files your app needs for offline use, and that accessing other resources would obey the normal rules of the web: they’d be accessible if the user happened to be online, and they’d fail to load if the user was offline.

But this isn’t the case at all. Simply having a manifest file changes the rules by which a page accesses any network resource, whether the user’s browser is online or offline. Unless a manifest explicitly lists a resource in one of its sections or contains an asterisk in its NETWORK section, attempting to access the resource from an XMLHttpRequest, a src attribute in a HTML tag, or anything else will fail, even if the user is online. No browsers currently provide very good feedback about this, either, leaving developers befuddled.

Problem Four: Cross-Browser Support Is Hard.

Not all browsers support the full application cache standard. Some browsers, for instance, don’t support the FALLBACK manifest category, which will cause apps on those browsers to behave differently.

Problem Five: There’s Not Much Tooling For This.

As I mentioned at the beginning of the post, Chrome is the only major browser that provides any information on the state of the application cache. Developers on other browsers at least need to add some code that monitors the state of the application cache and logs the many kinds of events it can emit, but even that information doesn’t tell you everything you need to know.

This is probably the most important problem, since it exacerbates all the others: if an offline app isn’t behaving as expected, a developer without the right tools can’t tell whether it’s due to many of the problems explained above, or something else entirely.

The Bigger Picture

These issues are symptomatic of a larger problem that most in-browser development tools face, which is that they generally provide little information that tells you exactly why a resource couldn’t be accessed. Aside from the complexities offline apps introduce, cross-origin resource sharing and the single-origin limitations put upon fonts and video make resource access in HTML5 incredibly complex. Yet most of our tools haven’t yet adapted to the development problems that arise when we use the new platform features.

The Good News

One of the best things about offline web apps, which is part of the reason for the additional complexity from the world of native apps, is that automatically updating is simply part of the way they work; you don’t have to put them on an online marketplace to benefit from the functionality, nor do you have to research and invest in any kind of third-party libraries that incessantly nag the user to upgrade. This is awesome.

As for the challenges facing developer ergonomics, help is on the way. Mikeal Rogers and Max Ogden are currently working on a project to make offline web apps easier to build. Folks at Mozilla believe that support for offline apps is an important use-case and want to make the user and developer experience better. Every major browser except Internet Explorer already seems to support the feature, which is also great. I’m already thinking of a few ways to ease these hassles myself.

In the long term, though, as I explained in my previous post, I think that the user perception of offline web apps—and the associated usability problems with even using a browser offline—are at least as important as the developer-facing challenges. Fortunately, the Open Web has some really awesome advocates, and I’m confident we can solve all of these problems once we put our heads together.

5 Responses to “The Challenges of Developing Offline Web Apps”

  1. Shahid Hussain Says:

    Nodding vigorously. Browsers need to be able to cope with every environment a computer will be used in, and offline – users on a train with intermittent wireless, users on a plane with zero wireless – is part of that.

    This is one of those “why haven’t we done this already” things.

  2. Wilfred Nas Says:

    I disagree with this post and wrote about it here. In my opinion it is hard to do this stuff, but we are at the fore front of this. And we just have to struggle along for now (and like it) untill tool makers and browser get their act together.

    It is not productive to complain, just do it and write about the stuff you learn…

  3. Chris Jacob Says:

    Looking at your EtherPad example (https://github.com/toolness/all-my-etherpads#readme) i see it’s hosted on GitHub Pages. Considering GitHub Pages is free hosting, and rather easy to setup (via automated page generator) I think this is a nice deployment environment to introduce people to offline app development… Perhaps you could put up a basic “offline app boilerplate” repo… With just a gh-pages branch…. Easy to fork, and when someone edits and pushes it, it’s automatically hosted. You could put in plenty of code comments and have a github wiki to discuss issues around offline apps.

    I think this repo/project could be a real benefit to getting more people to play with the offline web.

    I already plan to take inspiration from your project… I’ve forked it https://github.com/chrisjacob/all-my-etherpads

    I’ll ping you on twitter incase your interested on work on this I would be keen to be involved.

  4. Chris Jacob Says:

    Oh also WWDC2011 Session 515: “using html5 offline storage” is a great free video to watch… New tools are coming our way… P.s you need a free Apple Developer account to access these videos.

  5. Thomas Says:

    Hi Atul. Any news of the work of Mikeal Rogers and Max Ogden? Is it possible to see progress on GitHub or elsewhere?