« The geography of passion | Main | Do the Green Thing »

28 May 2007

The wisdom of clouds

Leading blog search-engine Technorati has just re-designed its home page. Founder and CEO Dave Sifry explains that the new layout is based on “using the wisdom of crowds as a mirror on ourselves”.


What Sifry means by this rather purple and impenetrable phrase is that the informal online classification system of tagging – where users annotate or tag content with explanatory comments – allows information to be organised democratically, based on what the majority of people are most interested in. 

It’s clear how labelling information ourselves helps us find it again later, but it’s less obvious how these individual descriptions might help other people track down something useful. You’d think one person’s evocative tag could easily be another’s worthless gibberish. One is reminded of the Jorge Luis Borges story about a Chinese encyclopaedia which classified animals in a number of irrelevant, completely incomparable ways: “those that belong to the Emperor”, “those that tremble as if they were mad”, “those drawn with a very fine camel’s hair brush”, “those that have just broken a flower vase” and “those that resemble flies from a distance”.

The type of tag cloud that features prominently on Technorati’s new homepage  – visual snapshots of the most common user categorisations – is a design solution for this very problem. The more popular a tag, the larger and bolder it shows up in a group of hyper-linked words that users can click on to navigate information. Unlike Borges’ encyclopaedia, rational opinions cancel out less rational ones so wisdom rather than nonsense lives to fight another day.

In “The Wisdom of Crowds”, the book title Sifry referenced, author James Surowiecki goes to great lengths explaining how and when this kind of group intelligence can occur. In the case of tag clouds, even if individual descriptions are meaningless, the same law of averages underpinning stock markets and democratic voting systems rules that, under the right conditions, collective meaning will mathematically emerge from a series of discrete, random events.

That said, David Weinberger is fond of pointing out that just because systems self-organise doesn’t always mean the results will be particularly organised. This is possibly the central characteristic of tagging, and indeed the web in general, and a direct reflection of the users who breathe life into it: untidiness. This human fallibility, imperfection and informality is an essential part of what makes the internet work. (Berners-Lee himself has been quoted as saying that “The web will always be a little bit broken.”).

It’s also the single most productive lesson businesses could learn from tagging and other such social systems: organise to better fit how people naturally behave and want to work. Real people are messy. Concepts overlap, ideas like people come with baggage, most stories have at least two sides, and questions (open-ended ones at that) often outnumber answers.

Of course this is exactly why common frameworks like taxonomies which provide structure and order can be so helpful. But it doesn’t have to be mutually exclusive. The infinite granularity of tags work by making a virtue of clutter, but there’s absolutely no reason they can’t sit peaceably alongside broader more hierarchical categorisations.

Sophisticated product designers have been taking account of the variety of different ways people interact with the world around them for a long time. Website information architects, for instance, know that some people like to read a map to find their way around (navigation, site maps), some prefer to ask for directions (search) or suggestions (recommendation engines, user reviews, tags), while others enjoy wandering about and seeing what they come across (browsing).

Organisational systems which recognise and celebrate this difference – in this case, a classification system which allows individuals to express their disordered, ad hoc preferences alongside more orthodox, linear information navigation – surely make a lot of sense. That way we can all get the best of both organised and disorganised worlds.


TrackBack URL for this entry:

Listed below are links to weblogs that reference The wisdom of clouds:


Ozoda Muminova

"...In the case of tag clouds, even if individual descriptions are meaningless, the same law of averages underpinning stock markets and democratic voting systems rules that, under the right conditions, collective meaning will mathematically emerge from a series of discrete, random events..."

This rule would only apply if the number of people tagging is large enough (circa 300), whereas Tecnorati's results depend on miniscule numbers. My biggest problem however is Technorati's 'Popular' - always dull and boring. So I am inclined to believe in "Madness of the Crowds" and stay out of it.

Andy Hobsbawm

Yes, good point. As Surowiecki points out, there are all sorts of conditions needed for group intelligence to emerge, such as intellectual diversity, independence, decentralisation and easy aggregation. A critical mass of opinion should be added to this list. Where did your 'circa 300' number come from btw? As for Technorati's 'popular' - perhaps they should start adding all sorts of Google-type classifiers in there to spice up the rankings - like, say, Flickr's 'interestingness' http://www.flickr.com/explore/interesting/?

Ozoda Muminova

Andy, yes a good idea with "classifiers" and "interestingness" (I did not know flickr had it!) In addition, Technorati could introduce some sort of a social filter, akin to StumbleUpon, where recommendations of your immediate circle are valued more than strangers'. Or much debated "predictive referral", like WhatshouldIreadnext.com's (never gives the answers I want).

Oh, 300 figure comes from sampling: if we want an error of 10% (say, we allow for 10% of people to tag a post about deforestation, as "holidays")and confidence levels of somewhere between 90% and 95%, we need 240-380 people to tag a post = ~ 300

The comments to this entry are closed.