Posts Tagged ‘Harold Pashler’

Make a guess, any guess

July 14, 2008

There was an interesting article in The Economist a few weeks ago about how to get an accurate estimate of a not-necessarily-well-known quantity (e.g. How many people were on board the RMS Titanic on its premiere voyage?). One way, which was already examined by James Surowiecki is to ask a bunch of people rather than just one. In general, the too-high guesses and too-low guesses start to cancel out, and the average number is a pretty good estimate, or at least more accurate in general than only asking one person.

But what if you don’t have a crowd? Then Edward Vul and Harold Pashler said that you can just answer twice. Indeed, whether you make two guesses one right after the other or two guesses several weeks apart, the average will (on average) be more accurate than the original guess. Better accuracy comes when the guesses are several weeks apart, but several commenters note that this may be because people looked up the answer in the meantime. Edited to add that they, like I myself, should have looked more closely at the original article since (as the eagle-eyed readers below note) the authors point out that this is unlikely because the second guesses were typically worse than the first guesses. They just averaged to something that was a bit better.

Let’s simulate this. I asked four random people to make two guesses as to the number of people who sailed on the Titanic . For comparison, Wikipedia says there were 2223 people aboard that night.

Person #1 First Guess: 900
Person #1 Second Guess: 2300
(Average: 1600, closer than the first guess!)

Person #2 First Guess: 2000
Person #2 Second Guess: 5000
(Average: 3500, a lot worse than the first guess)

Person #3 First Guess: 3200
Person #3 First Guess: 1400
(Average: 2300, nearly exact!)

Person #4 First Guess: 4000
Person #4 First Guess: 5000
(Average: 4500, worse than the first guess)

The average of the first guesses was 2525, which was more accurate than every first guess except the guess of 2000. This illustrates the idea that the average over crowds tends to be more accurate than an individual.

Now for the purposes of illustrating this article, the average guesses should be a little more accurate than the initial guesses. This happened for two people, but not for the other two: the average Average guess was 2975, which is less accurate than 2525. Dang, real data is so uncooperative. Of course, Vul and Pashler asked 438 people instead of 4, so maybe if I had asked more people I’d have gotten a result similar to V&P.

[You can see what seems to be the original article from Psychological Sciences here.]