The problems of popularity

If you're interested at all in social information architecture, emergent IA, popularity-driven navigation, tagging or related topics, you should read Duncan Watt's recent NY Times article. In it he summarizes some research he did that looks at how social influence (knowledge about what other users are doing) affects the popularity of items in an online music store.

The clever part of the study is that the experimental groups were separated into eight "worlds" so the researchers could observe how popular songs emerged in each world. There were two main findings:

• "In all the social-influence worlds, the most popular songs were much more popular (and the least popular songs were less popular) than in the independent condition."
• "the particular songs that became hits were different in different worlds...Introducing social influence into human decision making, in other words, didn’t just make the hits bigger; it also made them more unpredictable."

Watts explains the research findings pretty concisely:

The reason is that when people tend to like what other people like, differences in popularity are subject to what is called “cumulative advantage,” or the “rich get richer” effect. This means that if one object happens to be slightly more popular than another at just the right point, it will tend to become more popular still. As a result, even tiny, random fluctuations can blow up, generating potentially enormous long-run differences among even indistinguishable competitors — a phenomenon that is similar in some ways to the famous “butterfly effect” from chaos theory. Thus, if history were to be somehow rerun many times, seemingly identical universes with the same set of competitors and the same overall market tastes would quickly generate different winners...

There are interesting implications here for social IA, and in particular the use of popularity-based navigation schemes. The first is that sensitivity to initial conditions plays a big role in what becomes popular. We can't know what will become popular, and even the patterns or indicators we identify in past events may be nothing more than wishful hindsight. As Watts puts it "just because we now know that something happened doesn’t imply that we could have known it was going to happen at the time, even in principle, because at the time, it wasn’t necessarily going to happen at all."

Second, popularity and quality are not tightly coupled. Watts again: "a song in the Top 5 in terms of quality had only a 50 percent chance of finishing in the Top 5 of success."

Finally, popularity is not an inherently democratic notion. Watts: "what the market 'wants' at any point in time can depend very sensitively on its own history: there is no sense in which it simply 'reveals' what people wanted all along."

There's another bit of interesting news in this study. In the Science paper, Watts et al say "as social influence increases... which particular products turn out to be regarded as good or bad becomes increasingly unpredictable, whether unpredictability is measured directly or in terms of quality.... In such a world, there are inherent limits on the predictability of outcomes, irrespective of how much skill or information one has."

Which makes me wonder: why would I care about what's popular if I know that popularity is significantly (or mostly) random? It's also perhaps an argument to move toward implicit creation systems, like Flickr's Interestingness, that limit the impact of social influence.