When we consider the social effects of computers in political and social life, we usually think in terms of expanded power and new possibilities. This perspective on computation permeates even our critical visions of technology. But we should also be attentive to the power that computers and the accompanying language of “systems” and “complexity” have to narrow our conception of the politically possible.

But isn’t there a strong financial incentive to try to understand why you’re doing what you’re doing, whether it’s an algorithm or a human executing the trades? Otherwise it seems very easy to lose a lot of money.

Sure. But the market structure of investing dilutes that incentive.

The people who are developing the most sophisticated quantitative techniques work for hedge funds and investment banks. For them, there are two ways to make money. You make money by charging fees on the assets you manage, and you make money on the performance of the fund. That split will give you a sense of why there’s a dilution of the incentive. Because even if your assets don’t perform well, you can still make money on the fees that you’re charging to manage those assets.

The rewards from those fees are so large that if you can sustain a story for why your technique is superior, you can manage assets for a long time and make a ton of money without having to perform well. And, to be fair, sometimes it takes a number of years before you know whether the quantitative technique you tried actually works or not. So even if you aren’t making money in the short term, you could have a reasonable story for why you aren’t.

At the end of the day, for the manager, it’s as important to gather a lot of assets as it is to run a successful strategy. And gathering assets can be largely a marketing game.

Another fallacy in the lead-up to the financial crisis was the assumption that financial markets were so efficient that participants didn’t need to do the underlying work to figure out what the securities were actually worth. Because you could rely on the market to efficiently incorporate all available information about the bond. All you need to think about is the price that someone else is willing to buy it from you at or sell it to you at.

Of course, if all participants believe that, then the price starts to become arbitrary. It starts to become detached from any analysis of what that bond represents. If new forms of quantitative trading rely on assumptions of market efficiency—if they assume that the price of an instrument already reflects all of the information and analysis that you could possibly do—then they are vulnerable to that assumption being false.

Is Uber worth $60 billion? Well, Uber is worth $60 billion because we believe someone is willing to pay $60 billion for it. But maybe Uber is worth zero. Maybe that’s the actual value of the revenues that Uber will make in the future. In the current environment, we rely on liquidity to sustain prices for financial assets. When liquidity dries out and you’re forced to rely on the things that those financial assets actually represent, however, you could see painful shocks if there’s a big disconnect between price and reality—the kind of shocks you saw during the financial crisis.

If people didn’t want to do the analysis before, they’re probably even less inclined to do it now. They figure the machine learning models are taking care of it.

Right. The machines are taking care of it. Or other market participants are taking care of it.

I might think that the share of a particular company is worth 20 dollars. But its price can go up to 100 dollars well before it drops down to 20, in which case I can’t sustain my measure of its actual value. So if all of the computers are pushing the price to 100 dollars, I might as well not do the work of figuring out what the company is actually worth because it’s somewhat irrelevant to the price that it trades at. Paraphrasing Keynes, “Markets can remain irrational longer than you can remain solvent.”