Can You Digg It? - Market Manipulation vs. Wisdom of Crowds

Let me start off by saying that I am neither a techie nor an experienced Digg user, but the lively debate around the habits of top Digg users and Digg management together with my knowledge of market structure has led to me to an interesting parallel - Digg as a vehicle akin to small-capitalization, thinly-traded stock manipulation versus the "wisdom of crowds," which is what I thought the whole social bookmarking phenomenon was about.

Now I don't want to drag other social bookmarking companies into this dialogue - del.icio.us and reddit, for example - but to focus on Digg and why so many people appear angry about its breach of an implicit social contract. I have observed that this contract seems to encompass the principles of free speech, honesty and oversight for the benefit of the larger user community. Somewhere Digg has fallen down, and I think a brief discussion of the why's could be instructive to understanding how people will preference, share, and analyze information out on the web.

Can you Digg It?

So much has been written about Digg lately that it is beyond tiresome - and I don't want to add to the swarm. However, I am providing a set of links below for those who have either missed it or want a quick review of some of the issues that have been raised over the past several months:

Don't Believe the Hype: Digg, Business Week and Accountability

Digg Paths to Profitability and Social Networking

Could Digg be used for Sun Stock Manipulation?

Digg Corrupted: Editor's Playground, not User-Driven Website

Digg is Corrupted

Is Digg Corrupt?

Suspicious Digging?

Forget Paper Millionaire, Digg's Founder's a Vapormillionaire

All of these posts are from venues that have significant followings in the Web 2.0 community, so the criticism and insights are not to be downplayed. And, more importantly, many of these posts focus on the issue of unethical editorial policies and outright manipulation by Digg management. Bad, bad stuff. Suffice it to say, these are not good days for our $60 million dollar man, Mr. Kevin Rose. This is starting to feel like the Sports Illustrated cover jinx in Business Week terms - get on the cover with a big, smiling, arrogant face with big dollar signs attached - and look out. Now, I have absolutely no axe to grind with Kevin, Digg or anyone else associated with the social bookmarking phenomenon, but I do think all this suspicion (which appears to have basis in fact) makes for an interesting case study.

Out of all the stuff I've read, one post in particular caught my interest:

Top 100 Digg Users Control 56% of Digg's HomePage Content

When folks think of Digg, they're often misled into believing that the content seen on the homepage is representative of what a wide base of Internet users think is news-worthy and important. The numbers tell a different story - that of all stories that make it to the front page of Digg, more than 20% come from a select group of 20 users.

**************************************

These Top 20 have contributed a total of 5257 stories that have reached the frontpage out of a total of 25,260 stories to ever reach that page - 20.81% to be exact. Many of these top users have, as Digg terms it, a popular ratio of 30% and higher, meaning that almost 1 out of every 3 stories they submit will reach the homepage. Several users (specifically DarkHack & OsterMayer) have popular ratios of over 60%.

A logical extension shows that the top 100 Digg Users have contributed 14,249 stories to the homepage, or 56.41%. At Digg, a very select group of users is dominating the popular homepage content. Far from being a mass of opinion, Digg is instead showing, primarily, the content opinions of just a few, select folks.

These are a few excerpts from the post, and the comments that follow are interesting to read as well. But for a markets guy, the first thing that screams out is that this is not an efficient market, and Digg is the owner of the exchange. Not only that, but it is a thinly-traded market that can be both easily and substantially manipulated, and based upon many of the comments I've read this is exactly what appears to be happening. This kind of market structure and "oversight" will not engender confidence among its user base or serve as a template for expanding that user base. And, far from the "wisdom of crowds" moniker afforded the social networking networking phenomenon, Digg's current community appears to provide anything but (i.e., where is the wisdom? where is the crowd?).

What of Verticality?

So, given the inherent weaknesses in Digg, my question is whether this is more a commentary on Digg itself or horizontal (read: non-topic specific) communities. If you aren't going to create a free market and allow users to truly, and purely, drive the community (are you hearing this, Digg?), then is it better to organize along interest-based vertical communities a la, TechCrunch, GizModo and Engadget with editorial oversight? These sites do a good job creating content and insight in subject areas of interest to its communities, while enabling readers to comment and be part of a subject-specific dialogue. Seems like a good model to me, but distinctly different than that of a social bookmarking site.

This seems like more of a compare/contrast between Google and vertical search engines. Assuming that Digg and other social networking sites are run in a true "wisdom of crowds" manner without editorial manipulation, I think they would start to take on the character of Google, i.e., extremely powerful aggregators of information, based on the wisdom of crowds but with some ability to be gamed by the user community. Google certainly isn't perfect but it is directionally powerful and relevant - quite simply, it is what it is and what it is adds a lot of value. Vertical search engines, narrow in scope but more powerful and accurate than horizontal search tools like Google in their respective domains, appear more akin to the TechCrunches GizModos of the world. Certainly not all things to all people, but providing tremendous value their specific user community.

Conclusion

The key, it seems, is that of integrity. You don't need to be all things to all people or be perfect at what you do. However, what you do has to be done with openness, honesty and with an open ear to criticism. Digg seems to have failed on these scores to date. Over time, markets operated in a corrupt manner inevitably fail as people flee to areas where they can find consistency and truth. These are exactly the things that seem to be lacking in Digg. I hope they can find their way back.

Comments

[Robin Harris]

While editorial manipulation is inexcusable, I question the focus on the "top 100 Digg Users have contributed x% thus a select group of users is dominating the popular homepage content" meme.

Don't other people have to Digg the content to get it on the homepage? It doesn't look to me like one Digg is sufficient to get on the homepage. If it isn't, then don't critics have to show that the other needed Diggs come from a similar small group? Then you'd have an argument.

Possible bias: my site, StorageMojo.com has been both /.'d and Digg'd, and Digg generated far more traffic.

[Roger Ehrenberg]

Rob, I totally agree. It is all about disclosure, just like in the markets. People get pissed off when they feel they are being hoodwinked (ergo, the outrage over the stock option pricing scandals), and this appears to be a classic example of that. Nothing wrong with a small number of active and influential Diggers driving traffic and exerting influence over the home page, but to try and hide under a rock and say "gaming the system is hard to do" is just the wrong response. Kevin could take some lessons from J&J management in the wake of the Tylenol crisis - fess up, take responsibility, and make amends ASAP. Hopefully these lessons will be learned and internalized.

[robpas]

I heard Kevin Rose try to address this on a This Week in Tech (TWIT) podcast. Kevin was trying to downplay the "60 million dollar man story" and talk about Digg's model. He has said in the past that it is hard to game the system. However, like most technologies smart people can do smart things to take advatange. Certain users have figured out how to have digg gangs and the Digg folks play catch up in discovering them. With that said I think he should be up front about the limits of the system and recognize the power of the 100. In the markets business we call that disclosure.

As a side note, Jason Calacanis has recognized how powerful these 100 are and intends to pay them if they switch over to the Netscape competitor to DIgg.

[Roger Ehrenberg]

Yaser, I pretty much agree with you. As with Wikipedia and the more vertically-oriented content sites, I think the editorial function can be very valuable. But it is the lack of transparency and obfuscation that really bothers me, creating an uneven playing field that serves no one very well.

[Yaser Anwar]

I have no opinion on the subject at hand, but as a related adecdote, I know someone who owns hundreds upon hundreds of digg accounts, and thousands more on other buzz worthy sites. I know that he owns approximately 5 metafilter accounts, all registered after the $5 registration fee, all well used and difficult to call 'plant' on.

I've no idea if he's using bots or other bad stuff, but he does have the ability to influence what stories are shown on sites like digg.

If Digg is intentionally misleading people it's a problem no question. But if they would just come out already and say that they have legit editors then it's a qausi-newspaper. Is having editors a bad thing?

Kevin Rose say's it's all technology, just scripts running everything; but fails to address the claims about him having editorial control by gaming the system.