Goodhart's Law and social web design

A few weeks ago I came across Goodhart's Law, a rule-of-thumb in economics that has some application to the design of social software. In fact, it's particularly relevant to the creation of metrics and algorithms that model things like influence, reputation, and relevance.

According to the Wikipedia entry Goodhart's Law states:

Once a social or economic indicator or other surrogate measure is made a target for the purpose of conducting social or economic policy, then it will lose the information content that would qualify it to play such a role.

In terms of social web design, it means that once you pick a metric for some social property--like using the number of inbound links to a web page to measure its relevance--that metric will begin to decay in value. I think of several factors contribute to this negative feedback loop:

• People want to understand the mechanics of measurement (E.g., How do you count inbound links?)
• People want to know if the metric is a good proxy for the thing it's supposed to be measuring (Is the number of inbound links a good measure of relevance?)
• People want to play with the measurement (Does the metric or algorithm do what it says?)
• And ultimately, people want to position their sites or products well relative to the metric (How can I rank highly?).

And once this starts to happen, your metric becomes less useful. The best and most widely known example of this phenomenon is Digg, which adjusts it's algorithm frequently for these very reasons. (I've written about Digg before, and Joshua Porter has a nice write-up on some of their social design changes.)

Another example, closer to home, comes from Tweeterboard. On the Tweeterboard site I describe how its ranking algorithm works. It's pretty simple, and I think it's a good start for measuring influence on Twitter. It's just one of several metrics Tweeterboard provides and, IMO, less interesting than something like spread. However, conversation about Tweeterboard often focuses on the validity of the algorithm and strategies for higher placement.

Yet another example that's been making the rounds in the past few days is the xkcd Robot9000, an IRC bot that automatically bans people from a channel if they repeat what another user has said. This is a clever moderation tool in concept and in execution. Here's an explanation of how it works along with people's reaction to it:

In zig’s implementation, the moderator bot mutes (-v) chatters for a period after every violation. The mute time starts at two seconds and quadruples with each subsequent violation, so you have five or six tries to get the hang of it. Your mute-time decays by half every six hours (we’re still tweaking the parameters). When looking for matches, the bot ignores punctuation, case, and nicks.

The big problem we ran into, actually, was meta-discussion overwhelming the channel. Every new person wanted to speculate about the rules and their effect, and every violation was followed by a long postmortem. At first, we had a scoreboard showing who was the best at talking without violation, but this quickly turned into a competition, destroying actual chat. When we took down the scoreboard and banished meta-discussion of the channel to #meta-discussion, everything worked out nicely.

Part of the problem in all three cases is that leaderboards and ranking systems encourage people to learn how to work the algorithm rather than engage in the socially productive behaviour that the algorithm is there to promote.

Anyway. If it's true that Goodhart's Law applies to social web design, what can we learn? Here are some initial thoughts:

• One lesson is that our algorithms will always be changing in response to user's curiousity. In fact, if you don't have to change things in response to user exploration, you're probably not successful.
• Another possibility is that social designers will follow the search engines, and recently Digg, and keep the algorithms private. Search has become so much like informational plumbing, an indispensable everyday utility, that I'd like to see more transparency. But with the dollars at stake and aggressive black-hat SEO I can understand the benefits of private algorithms.
• Finally, and this is more of an observation of how things have evolved, algorithms will become more opaque. Metrics will become a combination of variables rather focusing on just one or two. We've already seen changes like this at Digg and Google. Their new metrics probably aren't any better in an objective sense--they don't contain more information about influence, reputation, relevance or whatever is being measured--but they're harder to manipulate.

Your comments, thoughts, speculations, etc., are always welcome. (This was going to be a two paragraph post. Oops.)