June 23, 2011

The Challenges of Developing Offline Web Apps

As I mentioned at the end of my last post, there's a lot of usability problems that make writing an offline web app difficult.

When writing a "native" client-side app using technologies like Microsoft .NET or Apple's Cocoa framework, it's assumed that everything your program is doing, and everything it needs, is already installed on the local device. Anything not on the local device needs to be explicitly fetched over the network. Any app is therefore "offline" by default, and it's very easy to tell, both when reading and writing code, when it needs to access the network.

Understandably, this makes it trivially easy to write an offline native app. One's very first app works offline, unless there's something related to the app's purpose for which network access is obviously required.

On the other hand, offline web apps build on the already confusing, frequently opaque world of resource caching. They also break developer expectations in various ways that I'll explain shortly. Furthermore, no browsers except Chrome provide tools for introspecting and understanding the behavior of an offline application, which adds to developer frustration.

Problem One: Simple Things Aren't Simple.

Let's say you've made a simple web-based currency converter, which is fully self-contained in a single HTML file and never requests any resources from the network. A browser won't let it be accessed while the user is offline unless you do a few extra things:

  1. Create a manifest which enumerates every resource that needs to be available to the app.

  2. Make sure the manifest is served with a mime type of text/cache-manifest, which is done differently depending on the web server software you're using.

    As I mention in my post On The Webbyness Of An Installable Web App, this single requirement vastly complicates the scope of the problem by requiring developers to understand the HTTP protocol layer and their web server software. Furthermore, Chrome's Web Inspector also appears to be the only tool that provides helpful error feedback if the manifest is of the wrong type—developers using Safari, Opera, or Firefox will likely be confused as their application silently fails to be cached.

  3. Add a manifest attribute to their HTML file's <html> element that points to the manifest.

Doing all this will allow you to deploy your simple currency converter app for offline use; it's clearly much harder than writing an offline native app. But what's worse is that adding offline support complicates further development of your app in a number of ways.

Problem Two: Your Development Workflow Is Now Broken.

Let's say you make a simple typo fix in your HTML file. The fix won't actually propagate unless you also alter your app's manifest file in some trivial way; even then, because the cache manifest is checked after the cached version is loaded, the page needs to be reloaded an additional time to activate the new version.

Needless to say, this complicates the development workflow for the app, as it impacts the edit-test cycle. Workarounds exist, of course; for All My Etherpads, I've added an --ignore-cache-manifests option to the bundled Python development server that allows the server to pretend that the manifest doesn't exist, so that the app's offline cache is obsoleted and code changes propagate instantly. Even though such workarounds make it possible to develop as one normally does, developers still must remember to make a trivial change to their manifest file when they re-deploy, or else their existing users won't see their changes.

Problem Three: Your App Can't Access The Network When Online.

After creating your first completely offline app, you might think that adding a manifest file would just let you specify what files your app needs for offline use, and that accessing other resources would obey the normal rules of the web: they'd be accessible if the user happened to be online, and they'd fail to load if the user was offline.

But this isn't the case at all. Simply having a manifest file changes the rules by which a page accesses any network resource, whether the user's browser is online or offline. Unless a manifest explicitly lists a resource in one of its sections or contains an asterisk in its NETWORK section, attempting to access the resource from an XMLHttpRequest, a src attribute in a HTML tag, or anything else will fail, even if the user is online. No browsers currently provide very good feedback about this, either, leaving developers befuddled.

Problem Four: Cross-Browser Support Is Hard.

Not all browsers support the full application cache standard. Some browsers, for instance, don't support the FALLBACK manifest category, which will cause apps on those browsers to behave differently.

Problem Five: There's Not Much Tooling For This.

As I mentioned at the beginning of the post, Chrome is the only major browser that provides any information on the state of the application cache. Developers on other browsers at least need to add some code that monitors the state of the application cache and logs the many kinds of events it can emit, but even that information doesn't tell you everything you need to know.

This is probably the most important problem, since it exacerbates all the others: if an offline app isn't behaving as expected, a developer without the right tools can't tell whether it's due to many of the problems explained above, or something else entirely.

The Bigger Picture

These issues are symptomatic of a larger problem that most in-browser development tools face, which is that they generally provide little information that tells you exactly why a resource couldn't be accessed. Aside from the complexities offline apps introduce, cross-origin resource sharing and the single-origin limitations put upon fonts and video make resource access in HTML5 incredibly complex. Yet most of our tools haven't yet adapted to the development problems that arise when we use the new platform features.

The Good News

One of the best things about offline web apps, which is part of the reason for the additional complexity from the world of native apps, is that automatically updating is simply part of the way they work; you don't have to put them on an online marketplace to benefit from the functionality, nor do you have to research and invest in any kind of third-party libraries that incessantly nag the user to upgrade. This is awesome.

As for the challenges facing developer ergonomics, help is on the way. Mikeal Rogers and Max Ogden are currently working on a project to make offline web apps easier to build. Folks at Mozilla believe that support for offline apps is an important use-case and want to make the user and developer experience better. Every major browser except Internet Explorer already seems to support the feature, which is also great. I'm already thinking of a few ways to ease these hassles myself.

In the long term, though, as I explained in my previous post, I think that the user perception of offline web apps—and the associated usability problems with even using a browser offline—are at least as important as the developer-facing challenges. Fortunately, the Open Web has some really awesome advocates, and I'm confident we can solve all of these problems once we put our heads together.

© Atul Varma 2021