How much wisdom is there in Digg?

I'm in the middle of The Wisdom of Crowds so I found this story about alleged Digg stylesheet thief Steve Mallett, and subsequent follow-ups, quite interesting. Nat Torkington summed up the story well on O'Reilly Radar:

The main claim of stealing CSS was superficially true, but substantially false.

In the meantime, of course, there's a small matter of hundreds of thousands of readers and thousands of active voters voting up the article about how "O'Reilly writer Steve Mallett" is a thief and a spammer. Only if you took the time to read through the hundreds of comments do you get to intrepid readers who tracked the copying back through Pligg (kudos to Digg reader caldroun, who was the first to identify pligg). But it was obvious by the rapidly-increasing Digg count that nobody was doing research (or even reading to see whether the claim had been refuted), they were simply indicating their condemnation of someone who had transgressed against the Digg community.

At the end of the post, Nat concludes:

This is a classic Web 2.0 problem: it's hard to aggregate the wisdom of the crowd without aggregating their madness as well.

Actually, this seems a lot like an information cascade (aka "bandwagon effect"), which happens when people start imitating others' choices rather than making their own. There's nothing particularly Web 2.0 about an information cascade--it happens in the media and markets--except that the web is an efficient way of aggregating choices quickly. (Digg especially, since it only has one "vote" value, and it's positive. Update: Sorry, I pulled the trigger on this too quickly. Digg has multiple ways to vote down a story, but only one way to vote up. Still, the emphasis is on "digging.")

Which makes me wonder: how much "wisdom" is there in Digg? Are other popular links just a result of "me too" voting? And how would you know?

And considering the web is now an exposure economy, how would you design a system where popularity is a better reflection of quality? What if you gave each user a dollar to bet for or against certain stories each day (and then paid the winners)? I don't know exactly, but there has to be a way of injecting crowd wisdom into an editorial system that's better than a simple "Digg it."

Comments

jamesb says...

what would people bet on though? would it be the success of a story i.e. judging the editorial values of readers? or would it be their own editorial values i.e. whether they would like to see that story do well? you'd get very different results depending which route you took. the former could be an adaptation of Yahoo Buzz Game, the latter could be an adaptation of a social bookmarking system [del.icio.us] for news items as it would encourage people to represent themselves in a way they's like to be seen by others... pulling tha most popular news stories datestamped from that day from pre-defined sources as a feed could be really interesting.

Posted on Jan 12, 2006
Gene says...

what would people bet on though?

I thought of pitting two randomly selected stories against each other, maybe with a spread based on the number of diggs (like in sports betting).

Another idea I had was limiting the number of diggs (say 5), and letting people apply some or all of them to a single story. This would mean decoupling a bookmark digg and a vote digg.

But really, I don't know... those are just some ideas that might maintain Digg's openness but improve quality overall.

Posted on Jan 12, 2006
Jay Fienberg says...

One possibly liability of the betting approach might be people using their bets to "pick a winner". Sort of like how the stockmarkets become forms of gambling--(to some significant degree) people don't bet on the best companies, they bet on the companies they think other people will bet on.

(Similar to what you also suggested) I've been thinking about a ranking system where a person has one "vote value unit" that they can divide up and apply as many ways as they want. So, two votes means each vote is worth 1/2 a value unit, three = each a 1/3 unit, etc.

Posted on Jan 13, 2006
Gene says...

Jay - your "one vote value" idea sounds pretty interesting. But wouldn't it make casual users more influential than heavy users? (Though maybe that would be a good thing in some cases.)

I was thinking of a situation where people had a finite number of votes (or diggs, or whatever), and thus might "invest" them wisely.

Posted on Jan 13, 2006
Tim Kanwar says...

[Advance disclaimer: My sincere apologies for a) posting such a long comment and b) posting verbatim my own blog response to this entry. I'm choosing to do this because I really found this post intriguing and, unfortunately, nobody really reads my blog. I figured the only way to enter this discussion was to just post it here. I hope I haven't offended...]

[Sorry, I had to delete that... just too long. But here's a link to Tim's post instead: http://farragonews.blogspot.com/2006/01/sorting-wheat-from-chaff-wisdom-of.html - Gene]

Posted on Jan 13, 2006
Pete Cashmore says...

Interesting point on information cascades. I put up a post with exactly the same premise earlier this week, saying that Digg isn't strictly a wisdom of crowds system because the users don't act independently. Instead, people look at how others have voted and then make a decision. A lot of people chimed in, with some suggesting that the wisdom of crowds isn't at all applicable to deciding whether an article is interesting, since that's a matter of opinion. The post and discussion is here:

Digg and the So-Called "Wisdom of Mobs"

Posted on Jan 15, 2006
Gene says...

Pete - You're right (and thanks for the link). Rashmi Sinha put it in a really concise way:

"If you look at [Digg] through the lens of "Wisdom of Crowds", it does not fulfill the criteria laid out by James Suroweicki. The four conditions are

(1) diversity of opinion

(2) independence of members from one another

(3) decentralization and

(4) a good method for aggregating opinions

Digg fails the "independence of members from one another" criteria.""

The more interesting question, I think, is how we might design an editorial system that meets Surowiecki's four criteria and where popularity isn't self-reinforcing (the way it is in Digg).

Posted on Jan 15, 2006
Jay Fienberg says...

I've been thinking about the "one vote value" idea mostly in terms of the quantity-of-links ranking systems like Technorati. One vote value might shift what's popular from being sites getting lots-of-links to sites getting fewer, but likely more considered, links. (I know that's a way over-simplified description of the shift.)

But, with this, as with Digg, it'd probably be desirable to set an "election period" during which one gets their vote--and then each new period would grant each person another, new, vote.

What's also maybe interesting about this is that votes could be counted simultaneously within different, overlapping voting periods, e.g., "most popular of the day" and "most popular of the week" presenting two views (each with biases towards different levels of users).

Posted on Jan 15, 2006
kaioshin says...

I think the simplicity of the Digg voting system is the key to its success.

I would suggest, however, that the flaw lies within the relative anonymity of the voters.

What if Digg kept track of other people with the same voting patterns as yours and gave their Diggs a progressively higher score than people who have diverging Digg-ing patterns.

In theory, you would then have an affinity with people posting links similar to your interests.

Posted on Jan 29, 2006

Post a comment

Remember me?

Basic HTML is allowed.

 

About this Page

Posted by Gene Smith on Jan 10, 2006. Before this there was Google Video Store underwhelms. Next up is links for 2006-01-11.

About the Author

Gene Smith is a principal with nForm, one of Canada's leading user experience consulting firms. He writes about information architecture, interaction design, community, the web and other such topics. More >

Subscribe

Get the feed Get the RSS feed (full posts, no ads)

My Book

Recent Posts

Archives

Elsewhere

You can also find me on Flickr, Upcoming, LinkedIn, Del.icio.us and Digg.

Work

nForm User Experience

Endorsements

Hosting by Dreamhost.