Posted originally on the Santa Fe Institute website (www.santafe.edu)

Investors in stock markets trade by submitting orders to buy or sell. The mainstream view that markets are efficient would naively suggest that the sequence of new buy or sell orders entering the market should be random. But, as discovered in 2004 by (then) SFI postdoc Fabrizio Lillo and SFI Resident Professor J. Doyne Farmer (now a professor at INET Oxford), the sequence of new buy and sell orders is far from random. Instead, a buy order is much more likely to be followed by more buy orders, and a sell order is much more likely to be followed by more sell orders. This presents a challenge for the theory of market efficiency, raising the question of whether the non-randomness in order arrival is also reflected in prices.

This persistence in the arrival of new orders, which has now been observed in stock markets throughout the world, is appropriately termed “long-memory,” and lasts for days, weeks, and even months. To explain why this happens, Farmer, Lillo, and Szabolcs Mike published a paper in 2005 in Physical Review E postulating the cause for long-memory in markets. This past November, their theory was strongly confirmed by a paper in Physical Review Letters.

The model that Lillo, Mike, and Farmer put forward, now known as the LMF Model, postulates that long-memory in stock markets is due to extreme inequality in investor size. Portfolio managers make decisions about what they want to trade, but then buy or sell to realize these decisions over an extended period of time. If there were many investors, all of the same size and acting independently, their decisions would average out under the law of large numbers, and the odds of a new buy or sell order would always be about the same. But LMF assumed there are a few really large investors, like Warren Buffet, for whom buying or selling based on a given decision can take many months; there are more fairly large investors, like SFI Board Member Bill Miller, for whom this can take days or weeks; and there are millions of small investors for whom this takes less than a day.

LMF assumed the distribution of sizes of these different investors follows a “fat-tailed” power law, and showed that this can cause the observed long-memory behavior of the orders entering the market (which follows a power law in time rather than size). According to their theory, periods when there are a preponderance of buy orders entering the market are caused by a few large investors who happen to be buying at the same time, and similarly for sell orders.

The recent paper, published by Yuki Sato and Kiyoshi Kanazawa, used 9 years’ worth of previously unavailable data from the Japanese stock market to test the LMF model. They showed unequivocally that the size of desired institutional trading follows the power law predicted by LMF, and that this explains the long-memory of orders entering the market.

The fact that LMF’s precise prediction was confirmed in every respect demonstrates that financial markets can follow quantitative laws like those in physics. “It is particularly satisfying to see that our model has been proven to be true,” says Farmer, who is now an SFI External Professor based at Oxford University’s Institute for New Economic Thinking. "Successful quantitative predictions of this type are rare in finance and economics.”

The long-memory of order arrival is interesting for its own sake, says Farmer, but notes that its implications for market efficiency are complicated. “Even though buy orders tend to push the price up, and sell orders tend to push it down, the size of the response in the price varies so that on average the resulting deviation from market efficiency in prices is relatively small,” he says. Several papers have since shed light on how this happens, but the precise mechanism, says Farmer, remains to be clarified.

Read the papers:

2004 Farmer, “The Long Memory of the Efficient Market,” Studies in Nonlinear Dynamics & Econometrics. doi: 10.2202/1558-3708.1226

2005 Lillo et al. “Theory for long memory in supply and demand,” Physical Review E. doi: 10.1103/PhysRevE.71.066122

2023, Sato et. al., “Inferring Microscopic Financial Information from the Long Memory in Market-Order Flow: A Quantitative Test of the Lillo-Mike-Farmer Model,” Physical Review Letters. doi: 10.1103/PhysRevLett.131.197401