September 2, 2010

Participatory, Scalable, Transparent Competitions

I’ve been involved in the judging pipeline for three competitions now. Today, I judged for an inspiring competition called Node Knockout, held by Joyent and Fortnight Labs.

The first two competitions I participated in didn’t scale. I wasn’t even a judge for the first one—we had a tiny handful of celebrity judges who couldn’t possibly review all of the submissions, so me and some colleagues furiously attempted to cull the list down for them. It wasn’t fun, and there wasn’t any way for the public to participate in the judging process. It also wasn’t really transparent—I was part of the process, yet everything I did was hidden from the public, including any valuable feedback I may have been able to give the entrants.

In the second competition, I was a judge for one of the rounds, but there were so many entrants that I simply didn’t have time to carefully examine each one. It was exhausting, and I didn’t even feel like I was able to give each entry the time it deserved.

In stark contrast, judging for Node Knockout was an amazing experience on three levels.

Knockout judging was participatory. Instead of a tiny handful of judges, Joyent and Fortnight Labs actually enlisted a small army of industry experts, including three of my coworkers. The public could participate, too: ultimately about half of an entrant’s “rating” was determined by them and the other half was determined by the judges. In some sense, the judges were just a pool of “trusted voters” whose votes were weighted more heavily than everyone else’s.

Knockout judging scaled. Since there were around a hundred entries total, the contest runners only required each judge to evaluate 6 or 7 assigned entries, allowing them to carefully examine each one and provide useful feedback. This allowed me to spend lots of time on each one and come out of the evaluation process feeling excited about the competition instead of exhausted.

Knockout judging was transparent. Furthermore, everyone’s comments and ratings were completely public, effectively constituting a body of valuable feedback the entrants could use if they wanted to continue working on their project. Every entrant’s team had a page on the Knockout site that listed all the comments and ratings submitted so far; it read a lot like the comments on a blog post.

In short, Knockout wasn’t just a competition about the Web; it was a competition held in the spirit of the Web, too. Thanks to Joyent and Fortnight Labs for holding such a fun event!

© Atul Varma 2021