The Wisdom of Crowds

At a client conference in my last job, one of the keynote speakers was writer James Surowiecki, author of the book The Wisdom of Crowds. Pulling notes from his book, he made a compelling case that large groups of people are smarter than an elite few, no matter how brilliant; better at fostering innovation, coming to wise decisions, and even predicting the future.

[photopress:crowds.jpg,full,alignright] The most fascinating example (and there were many) of this wisdom is in the investigation of a submarine that had sank and disappeared. The Navy had limited information regarding its location, and all searches came up empty. One smart fellow had the idea to consult a wide range of experts – in oceanography, ballistics, physics, engineering, etc., and ask them to come up with a probable location of the submarine. All the answers were collected and analyzed, and an ‘average’ location was charted based on the data. Chillingly, the sub was found within hundreds of yards of that location.

With large sample sizes, crowds form a network, and the best solutions bubble up from the collective thought. Though it sounds very Borg-like, we witness examples of it every day. The financial markets often sniff out trends and problems before they hit the front page. There is no collusion or any critical mass of explicit cooperation here – these trends are created by the cumulative wisdom of the market’s millions and millions of participants.

Which brings me to the potential wisdom of the Chicago Mercantile Exchange’s housing futures index created by Karl Case and Robert Shiller. Theoretically, this market would allow individual homeowners to hedge their investment by purchasing future contracts based on their metro level housing market. Let’s say I owned an apartment in NYC that is worth $1.5mm today. I could buy contracts that would pay me if the market dropped by 20%. I would effectively lock in a certain value for my property at a future date with these contracts.

If this concept takes off, the swings in the housing market would flatten considerably. If Joe homebuyer sees the contract prices that are based on next year’s housing values reflect a substantial drop, Joe homebuyer will be less likely to overpay for a property. Also, if Joe homebuyer could basically buy insurance against a large correction, the real economic impact of such corrections would be dampened by the payout of this ‘insurance’. However, Surowiecki has doubts that such a ‘wise crowd’ can materialize in the near future. As explained in a New Yorker article on this market, culture and habit matter as much as economic rationale. He writes:

“Even today, it’s clear that otherwise rational people harbor deep-seated beliefs that make housing futures a tough sell. People generally don’t hedge individual investments, because they don’t like to limit their potential gains in advance. That’s especially true when it comes to housing, because of the ingrained assumption that, over time, real estate is guaranteed to be an excellent investment—even though Shiller, in a recent book, shows that, allowing for inflation, American home prices barely budged during the twentieth century. In that sense, the housing-futures market has what is known as a framing problem: selling a contract seems like betting on housing prices to fall, rather than simply insuring yourself in case they do.”

This market debuted only about four months ago, so there is by no means any critical mass to it. It’s thinly traded, and it only offers futures on 10 US markets (Seattle is not one of them). Interestingly, the trading activity indicates a correction in the ten covered markets over the next year (with Denver showing the least downside). I would love to have had Seattle on the list. But, if trading activity increases, I would imagine that more markets would be added – and Seattle’s got to be high on that list.

Given that real estate is extremely localized (e.g. neighborhood by neighborhood), would a market that had critical mass (millions of contracts exchanged per day) be a driving force in the direction of a metropolitan market’s value? Would the average increase or decline be pretty darn close, even if street level values varied significantly block by block? My guess is that they would be extremely influential in how money moved in and out of the housing market. Such an efficient market would provide opportunity for long term homeowners to hedge their investments, speculators to make bets on the direction of the market, and renters to protect themselves from ‘missing out’ on appreciation.

In other words, many of the financial benefits of the American dream of homeownership could be had without ever buying a home. Take things a step further, and perhaps a fully matured and stable housing futures market would advance the dream of disintermediation further than Redfin or Housevalues could ever do. With good market info, long term home buyers wouldn’t have to worry so much about overpaying on a property if the market indicates a strong future value. Therefore, a precise valuation that an agent might be able to give versus that of an automated system may not worth the extra money it would cost in agent fees.

15 thoughts on “The Wisdom of Crowds

  1. Surowiecki’s book is simply brilliant, full of provocative ideas. I’ve been intending to blog about it, but you beat me to it!

    Those who participated in the recent Inkling Markets betting pool on the Greg vs. Ardell blogathon may be interested to note that Inkling Markets is inspired partly by Surowiecki’s book.

    Alas, that market did not have the characteristics needed for a truly efficient prediction market: a relatively large, diverse, and independent group of participants.

  2. In my former career, I tried to utilize similar markets (a solution called Crowd IQ – it was started by someone in our company – to try and determine what the next big online media wave would be (video? peer to peer? social networking?). Our company was nationwide and large, and there was enough diversity (another key component for a ‘crowd’ to come up with the best results) to generate good information, but I couldn’t get it off the ground. I’m sure larger corporations with deeper resources have succeeded.

  3. Galen, good point. Furthermore, Surowieki posits that financial gain isn’t necesssary for a good predictive market assuming it’s large and diverse enough.

    As for Ardell vs. Greg, that looked more like paramutuel betting, and I put my money down on Ardell – not because I was convinced she could out post Greg – but because she was a 7:3 underdog at the time, and I thought she had a better than 30% chance of beating Greg (alas, little did she know she was up against the Terminator of blogging. 🙂

  4. Oh please, the “wisdom of the crowds”? Crowds are easily manipulated, especially in the short term (witness any number of lynchings in this century).

    And wikipedia, while great for a casual bit of information, isn’t necessarily an authoritative source because thousands of people edit it.

    While I’m willing to believe that a large pool of experts is better than relying on a single expert, I think it fails to be seen that large populations of people provide anything more than good data on how easy it is to chart the popular thinking of the time.

  5. I haven’t read the book, but I assume he states that some sort of benefit (either community standing or financial) has to be involved for crowds to make good predictions. If not, markets becomes a game of what people want to happen instead of what they think will happen.

    Here’s Google’s take on markets:
    No actual money trading hands, but you can bet that people who predict well are rewarded.

  6. Jcricket:

    Given you haven’t read the book, and given that your only exposure to this concept is my very minimal post on the subject, I can understand your skepticism.

    Crowds is a loaded word in our culture. A better term (but not as catchy on a book title) might be critical mass intelligence (maybe not :). When trying to grasp this concept as individuals, we load up our biases in how it’s perceived. For example, if you’re like me, you probably ignore and never click on those annoying banner ads online. However, these companies running these ads sell products day in and day out through this advertising channel. There are people out there who click and buy.

    In the same way, with a large enough and diverse enough ‘crowd’, the group intelligence rises to the top.

    I highly recommend you read the book to get a better understanding on the topic than I can give. Also, the reason he spoke at our client summit (I worked for a large, publicly traded digital marketing company) was the relevance to marketing, and how tapping into the ‘crowds’ can lead product development down the right path, thereby saving millions by making the right decisions.

  7. I think more likely users of the Shiller/Case housing futures market would be financial companies with large mortgage portfolios. While a ton of individuals feel comfortable buying and selling stocks, futures and options markets are significantly harder to understand and trade.

    Perhaps insurance companies could start selling “home price insurance” to new home buyers, and hedge their risk using this market.

  8. I have to say, I was also skeptical about the premise of Surowiecki’s book at first. After reading it, I’m much more of a believer in the whole concept. Surowiecki lays out a number of very compelling examples, describes what crowd characteristics lead to the best outcomes, and also acknowledges circumstances in which crowd behavior most emphatically does not lead to good outcomes.

    Two real-life working examples of the concept are the Iowa Electonic Markets, which trades in economic and political futures, and the Hollywood Stock Exchange, which helps predict the success of movies and stars.

    A more recent startup is Casual Observer, which has a fairly active political market going. In the California gubernatorial race, for instance, the Governator is currently ahead of Angelides by a 3:1 margin.

    One of key differences between these sorts of prediction markets and the sorts of polling that we’re more familiar with is that prediction markets ask, “What do you believe is going to happen?” and not “What do you want to happen?”

    If you enjoyed Malcolm Gladwell’s books Blink and The Tipping Point then you’ll definitely enjoy book.

  9. I’m with you Cricket. I wouldn’t want to base anythng on what people who click on banner ads do 🙂 The first, last and ONLY time I clicked on a banner ad, I ended up divorced from a 20 year marriage 🙂

    But then, I’m staying out of this discussion. Crowds to me are mindless…and I don’t deal in mindless behaviors.

  10. I think I’ve read enough on the various sides of the argument to know that I’m a “wisdom of the crowds” non-believer, mainly in the sense that the premise argues true crowds are wise. All the examples of crowds that “work” seem not to be crowds in any meaningful sense. Again, I’m not arguing that single individuals are the best holders of knowledge, just that the overall premise of crowd behavior doesn’t seem widely applicable, and has far too many expections (even by the author’s own admission). I also read this sellsius article for a more in-depth review along the lines of “a confederacy of dunces”:

    Also I question how it’s really valid to think that a housing futures market would bring about wholesale change in the RE industry like you seem to think. There are futures markets for lots of things, but they are as prone to manipulation and swings as any other kind of market. And plenty of smart people could buy puts and calls to protect their equity market positions, but don’t. Where’s your thinking on that?

    Besides, to get a snark in, while you obviously have years of successful experience in Interenet advertising, your blogged-about experience in RE so far doesn’t seem to have been particularly economically fruitful.

  11. Good snark! And you’re right, I’m still new to the game of investing, and with each deal, learn to avoid the same mistakes. I blog warts and all. Some readers will revel in my mistakes, for sure. But my goal is that others might learn from the mistakes I made.

    Also, note my use of the conditional tense in the last paragraph of my post. By no means do I see myself as an expert forecasting the impact of the housing futures market (I’ll leave that to the crowds :), rather, the potential is fascinating, and that is what I wanted to highlight….the ‘what if’ factor.

  12. “Fascinating potential” is what gets a lot of investors into trouble 🙂 Drop that phrase from your vocabulary, and deal more in the realm of “worst case scenario”. It’s a more tried and true method to success.

  13. I’ll never do away with considering fascinating potential, they are just words, after all. When it comes to action, I tend to stick more closely to the conservative side of the ‘what ifs’.

    A line from the second Matrix movie comes to mind, to paraphrase:

    “’s at the same time both man’s greatest strength and his greatest weakness”

Leave a Reply