This mid-term election season I was fascinated by the information on prediction markets such as Polymarket and Kalshi. The wisdom of crowds was often more accurate than the polls, especially on more prominent races, where tens of thousands of people wagered hundreds of millions of dollars. With those, the end result was often within a few tenths of a point different than what was predicted. Smaller races were less reliable, where a market as small as a few dozen participants could easily be manipulated by larger bets.
Prediction markets within organizations
This is the season of annual planning and forecasting, the time of the year we gaze into the crystal ball trying to predict the unpredictable. What is the right balance between real analysis, sand-bagging, and hopium that will let us satisfy the boss and move on to more valuable work?
Everyone involved has some opinion, each grounded in different experiences and knowledge. Some opinions are voiced, others aren't. Some opinions aren't even present as they haven't been brought up in earlier meetings at different levels.
Could the wisdom of crowds concept behind prediction markets be used to surface some of that hidden information to improve the quality of forecasts? You might be surprised to learn it's been tried several times before.
Hewlett-Packard is the earliest example I found. In the late 1990s, groups of about two dozen employees or fewer traded with a small cash stake, and the marketbeat the official forecast in 6 of 8 tries.
Ford tried it. In 2010 it let employees with relevant expertise trade on weekly sales of 6 models over 13 weeks, with several $100 gift certificates as the prize. (Not exactly Kalshi stakes.) The market beat Ford's expert forecast by 25% on mean squared error. I'd want more than 78 data points before I bet a plant on that, since the p-value was 0.104. Still, Ford takes those forecasts seriously, and it's been making them for a long time.
Researchers suspected these markets could pull out information trapped in hierarchies because messengers get shot. An internal market at Siemens saw a software deadline slipping while the official line held. At Google, a senior executive told an all-hands that internal traders had seen a failed hiring goal coming, and joked, "I am shocked and outraged by the lack of brown-nosing at this company." Then the team changed its plan and filled the jobs.
A crowd that lets you disagree
I'd always thought of a crowd as the thing that shuts people up. In Solomon Asch's line experiments of the 1950s, people who made almost no errors alone went along with an obviously wrong group about a third of the time, and 75% caved at least once. Timur Kuran later named the habit preference falsification: saying in public something other than what you believe in private. I've come to think most forecast meetings run on it.
A market, perhaps ironically, turns that crowd inside out. HP worked to keep its trading anonymous, and Google employees bet anonymously, so nobody has to defend a position in front of the boss. A price also moves on conviction rather than headcount, which means one person with bad news can lean against 50 who'd rather not hear it. Asch found something else: give the subject a single ally, and conformity fell from about 32% to about 5%. I suspect a price does the same job, telling the quiet skeptic in accounting that somebody else sees it too. So "wisdom of crowds" may be the wrong name. I think the wisdom belongs to a few people the crowd would normally have silenced, and the market is how they get a vote.
Where it gets messy
Company crowds bring their own biases. Google's insiders leaned optimistic on their own projects, and the bias was worst on deadlines, the outcomes employees control. Prices even rose with the stock. Googlers who sat near each other traded alike, which is a poor recipe for independent judgment.
Ford's feature markets were badly wrong about ideas employees felt lukewarm about, although that's also how an in-car vacuum died. The cheerful part: at Google and in Ford's sales markets the biases faded, because experienced traders bet against them and the worst traders drifted away.
The bigger obstacle sits in the corner office. The Ford paper's authors know of 4 companies where senior management shut down or limited a market after it forecast trouble, one of which became a famous debacle. Dan Schwarz, who later ran Google's second internal market, pitched one at Waymo, and a VP there objected that it would spread safety data his division wanted restricted. Google's supply chain managers agreed a market would improve accuracy and still passed, because they had little to gain from it. Power still creates influence and bias.
Perhaps there's even a lean angle. An andon cord lets anyone on the line stop it, and works because (in effective lean organizations) people know they won't be punished for pulling it. A prediction market looks to me like an andon cord for forecasts, with the added mercy that nobody sees whose hand is on it. When Google's market showed a project running behind, a manager went looking and found glitches, which is what you hope happens after any pull. Shutting down a market that forecast trouble is the same as yelling at the guy who stopped the line.
How I'd try it
Thankfully I'm retired, but if I were back in the chair, I'd start small and boring:
- Pick questions that settle quickly and can't be argued about: next month's shipments, this quarter's ship date.
- Separate the forecast from the target, and never let either touch compensation.
- Invite people adjacent to the work, since they know plenty and have less riding on it. Google's broader group of well-connected engineers offset its optimistic project insiders.
- Keep trades anonymous and skip the cash. Use a leaderboard, and call your lawyer before you get creative. Google also left out the numbers closest to its share price, and you should too.
- Begin with competitors. Forecasting them avoids most of the internal politics. Schwarz's market asked about Google's own AI milestones when executives needed to know about OpenAI's.
- Name an owner. Without a champion, these things quietly die.
The election taught me less than I thought about crowds. What stuck was how good it felt to see a number nobody had massaged. Somewhere in your company there's a figure everyone knows is wrong. Would you let them bet against it? And if the price came back ugly, would you thank whoever pulled the cord, or cut it?