Herdict-Firefox Integration and Better HTML Presentations

I recently wanted to create a short, two-minute and thirty second “pitch” for the Herdict-Firefox integration prototype I’m working on with Jennifer Boriss, Laura Miyakawa, and Jeffrey Licht.

Here is the result. It turned out that the pitch itself was an experiment for me: after fiddling around with Screenflow and iMovie for a bit, I got frustrated with their limitations and decided to just use HTML to put together the presentation.

After writing out the script for the pitch, and recording my narration with Audacity, I saved the file as both Ogg Vorbis and MP3—different browsers support different formats—and set up a directory structure.

As with Mozilla: The Big Picture, I basically stuffed everything into the structure of an HTML page. The first two slides of the presentation, for instance, look something like this:

  <div id="slides">
    <div data-at="0.0">
      <a href="http://www.mozillalabs.com"><img
         id="logo" src="images/labs-logo.png"/></a>
      <h1>Firefox-Herdict Integration Pitch</h1>
    </div>
    <div data-at="4.0">
      <img src="images/server-not-found.png"/>
    </div>

The data-at attribute is an example of the HTML 5 data- attribute and records how many seconds into the audio the slide should be displayed. I marked up subtitles for the presentation in a similar way.

After that, I wrote some JavaScript that just attaches a timeupdate event listener to the presentation’s audio element and synchronizes the current slide and subtitle to its position. The result is something that looks and feels to an end-user like a YouTube video—one can even “scrub” the position slider to quickly rewind and fast-forward. However, I’d argue that this approach is actually superior to standard video in a number of ways:

  1. Slides can have any valid HTML content embedded in them. Text can be copied and pasted, their look and feel can be altered through CSS; images can be hyperlinked to their original sources.
  2. It’s easier to eliminate compression artifacts without sacrificing bandwidth and download size. Text, for instance, is always super-crisp.
  3. Since everything uses HTML, CSS, and JavaScript, anyone can view-source or use a web inspector to investigate how things are put together; as I explain in The Open Web is Magic Ink, they can take it apart, see how it works, and put it back to together in a different way. Doing such things with a pure bitmapped video representation wouldn’t be possible: you’d need the source “project files” for whatever program was used to compose the video, not to mention access to said program.
  4. Subtitles can be toggled on or off, and adding new languages isn’t hard.

This approach has its downsides, too, of course: there wasn’t a really easy way for me to embed the presentation in this blog post, for instance, and it can’t be viewed at all on Internet Explorer, as far as I know.

Still, it was a fun experiment to try, and for this particular use case I actually found it easier to compose everything using Open Web technologies than with the proprietary tools at my disposal.

Please be sure to check out the actual presentation, too, as the stuff we’re doing with Herdict is way cool.

12 Replies to “Herdict-Firefox Integration and Better HTML Presentations”

  1. This is in line with some of the new audio experiments we’re working on now, specifically using data in the audio itself (whether real or synthetic data we put there) to drive js/visualizations/timings/etc. in the page. We should chat about it some time. If you’re ever bored, hop into #processing.js on moznet, or ping me.

  2. Graceful degradation? I’d like to see the slides and have the option of downloading the vorbis file when JS is disabled/the audio tag isn’t implemented.

  3. This is great! Additional benefits include:

    – better accessibilty than regular video (text can be read by a screen reader),
    – easier for search enignes to index,
    – faster load time and playback (no video decoding) – watching this on the n900 is a real pleasure!

    Thanks for putting this together and including a friendly license.

  4. This rocks 🙂 And the “Magic Ink” point is exactly right. If you want a “Download Audio” option in Flash, you have to code it.

    anon: I agree that there should be a fallback link, but if JS is disabled, AFAIK you can right-click the element and select “Save File” or something like that.

    Gerv

  5. A few more thoughts…

    The really cool way to package this up would be a JS library. The first two inputs would be a Vorbis file, and a text script with embedded time markers and slide numbers, like the following. (I think it’s better to have the timing and slide info embedded in the script, rather than in the HTML markup.)

    [0:00:00 1] Firefox’s network error pages are familiar to everyone.
    [0:04:00 2] But they’re not very useful. Most people, if they’re like me, see this wall of text and interpret it to mean “the internet broke and we have no idea why.”
    [0:14:00 3] …

    This would be parsed to automatically generate the subtitles and put up the right slide (the second number is a slide number). Or it could be downloaded for reading by deaf people (or the impatient!).

    The last input would be the slides. I’m not sure if it would be better to have them in one file or multiple files. Say one file; all you’d then have would be:

    This way of doing things also allows you to reuse slides without repeating the markup. You could also have a print view which would print the slide numbers next to the slides, so it could be referenced by someone using the transcript.

    Gerv

  6. Regarding the slide technology, this doesn’t degrade at all well when you can’t or shouldn’t play the audio (I tried to look at this last night on my smartphone, whose browser doesn’t support the audio tag; I might be in a public place without headphones and not want to make noise; etc.) There should be some way of paging through the slides manually.

    Regarding herdict integration, I have a privacy concern. It appears to me from the herdict website that the hostname of the site you were trying to reach is sent to the herdict server in the clear. In some contexts, the mere fact that you wanted to reach a particular site is privacy-sensitive information (consider https://bugzilla.mozilla.org/show_bug.cgi?id=330884 for example) and of interest to eavesdroppers in the middle (China’s filtering system comes to mind). Can we do something about that?

    I also have a scope-creep suggestion. I was just in a cafe with a compromised WiFi router that was tampering with my SSL traffic. This showed up in the Firefox UI as certificate errors on every HTTPS site I tried to visit. The certificate errors were more than usually confusing, but that’s not the point. The point is, if I get a certificate error on a website, it would be very useful if the browser could automatically check whether “everyone” gets the same certificate error, and whether I would have similar problems connecting to other SSL sites.

  7. Atul, this is very cool.

    You’ve got several good ideas going here, and I’m going to explore the possibility of implementing at least a few of them in S5 http://meyerweb.com/eric/tools/s5/ which is currently what I use for all my presentations.

    If done right, someone could even take an original S5 presentation and podcast of a talk, and then annotate the S5 with time indices for the slides according to the usage in the podcast.

    Tantek

  8. This is very nice.

    It would be great to just provide the Vorbis audio and have it played by Java, Flash or Silverlight as a backup.

    How hard would it be to get OpenOffice Impress outputing to this format? Doesn’t it already do flash?

  9. Thanks for the comments, everyone!

    @David: Absolutely, I’ve been keeping up with your blog posts on audio and processing and I’d really like to talk to you about it sometime–will ping you.

    @gerv: Ugh, it did cut out all your example HTML, and I tried editing your comment and it looks like I can’t “restore” it in any way either. 🙁 Stupid WordPress… That said, though, I like the DSL you came up with for the subtitles, since subtitles are usually just text with simple bold/italic markup, but for slides it seems as though having the full expressiveness of HTML at your disposal would be useful…

    @Tantek: Awesome, I’d love to see that happen. It shouldn’t be hard to develop a simple annotation tool/web-app that makes it easy for anyone to contribute a podcast-slideshow synchronization timeline, too.

    @Zack: Yep, the privacy concern re: Herdict is definitely one of the “big problems” we need to solve. Hopefully we’ll get lots of discussion on that once we release the prototype. I also agree with your SSL errors problem, that would be another great problem to help solve, in particular because most users have no idea what to do when encountering an “untrusted certificate” screen.

    I also agree that graceful degradation is a must-have. What really needs to be done is a custom UI for the slider so that we can have it hooked up to a “null implementation” that doesn’t play any audio, or even to a “composite implementation” that makes multiple audio files in sequence seem like a single cohesive audio element (this would make the parts comprising the audio as remixable as the video, and reduce the need for an audio editing program like Audacity).

Comments are closed.