Why game theory is often useless...

Economic theory relies very heavily on the notion of equilibrium. This is true in any model for competitive equilibrium -- exploring how exchange can in principle lead to an optimal allocation of resources -- or more generally in the context of game theory, which explores stable Nash equilibria in strategic games.

One thing physicists find wholly unsatisfying about equilibrium in either case is economists' near total neglect of the crucial problem of whether the agents in such models might ever plausibly find an equilibrium. You can assume perfectly rational agents and prove the existence of an equilibrium, but this may be an irrelevant mathematical exercise. Realistic agents with finite reasoning powers might never be able to learn their way to such a solution.

More likely, at least in many cases, is that less-than-perfectly rational agents, even if they're quite clever at learning, may never find their way to a neat Nash equilibrium solution, but instead go on changing and adapting and responding to one another in a way that leads to ongoing chaos. Naively, this would seem especially likely in any situation -- think financial markets, or any economy as a whole -- in which the number of possible strategies is enormous and it is simply impossible to "solve the problem" of what to do through perfect rational reflection (no one plays chess by working out the Nash equilibrium).

A brilliant illustration of this insight comes in a new paper by Tobias Galla and Doyne Farmer. This is the first study I've seen (though there may well be others) which addresses this matter of the relevance of equilibrium in complex, high-dimensional games in a  generic way. The conclusion is as important as it is intuitively reasonable:

Here we show that if the players use a standard approach to learning, for complicated games there is a large parameter regime in which one should expect complex dynamics. By this we mean that the players never converge to a fixed strategy. Instead their strategies continually vary as each player responds to past conditions and attempts to do better than the other players. The trajectories in the strategy space display high-dimensional chaos, suggesting that for most intents and purposes the behavior is essentially random, and the future evolution is inherently unpredictable.

In other words, in games of sufficient complexity, the insights coming from equilibrium analyses just don't tell you much. If the agents learn in a plausible way, they never find any equilibrium at all, and the evolution of strategic behaviours simply carries on indefinitely. The system remains out of equilibrium.

A little more detail. Their basic approach is to consider general two player games between, say, Alice and Bob. Let each of the two players have N possible strategies to choose from. The payoff matrices for any such game are NxN matrice (one for each player) giving the payoffs they get for each pair of strategies being played. The cute idea in this analysis is to choose the game randomly by selecting the elements of the payoff matrices for both Alice and Bob from a normal distribution centered on zero. The authors simply choose a game and simulate play as the two players learn through experience -- playing strategies from their repertoire of N possibilities more frequently if those strategies give good results.

With N = 50, the results show clearly that many games do not ever settle into any kind of stable behaviour. Rather, no equilibrium is ever found. The typical dynamics is reflected in the figure below, which shows the difference in payoffs to the two players (Alice's - Bob's) over time. Even though the two agents work hard to learn the optimal strategies, the complexity of the game prevents their success, and the game shows no signs whatsoever of settling down:

As the authors note, this kind of rich, complex, ongoing dynamics looks quite similar to what one sees in real systems such as financial markets (the time series above exhibits clustered volatility, as do market fluctuations). There are periods of relative calm punctured by bouts of extreme volatility. Yet there's nothing intervening here -- no "shocks" to the system -- which would create these changes. It all comes from perfectly natural internal dynamics. And this is in a game with N = 50 strategies. It seems likely things will only grow more chaotic and less likely to settle down if N is larger than 50, as in the real world, or if the number of players grows beyond two.

Hence, I see this as a rather profound demonstration of the likely irrelevance of equilibrium analyses coming from game theory to complex real world settings. Dynamics really matters and cannot be theorized out of existence, however hard economists may try. As the paper concludes:

Our results suggest that under many circumstances it is more useful to abandon the tools of classic game theory in favor of those of dynamical systems. It also suggests that many behaviors that have attracted considerable interest, such as clustered volatility in nancial markets, may simply be specific examples of a highly generic phenomenon, and should be expected to occur in a wide variety of different situations.

The limitations of markets

Economists aren't often as vocal as they should be about the limitations of markets -- especially the extreme assumptions required for them to deliver superior outcomes and some kind of "efficiency." I've documented here before some of the exuberant cheer-leading for the wonders of modern markets that was the norm before the financial crisis of 2008. No self doubt or balanced criticism about the dangers of markets there.

Now, only a few years after the crisis -- and with a global economic crisis just looming up before us -- the old hysteria is again getting underway with calls (especially from US politicians) for more privatization to get the damned inefficient government out of everything. For an intelligent, fact-based perspective, I'm simply going to quote the following extended discussion from economist Mark Thoma. He deserves a medal for saying what most other economists ought to be saying every day to everyone they meet:

To listen to some commentators is to believe that markets are the solution to all of our problems. Health care not working? Bring in the private sector. Need to rebuild a war-torn country? Send in the private contractors. Emergency relief after earthquakes, hurricanes, and tornadoes? Wal-Mart with a contract is the answer.

Whatever the problem, the private sector - markets and their magic - beats government every time. Or so we are told. But this is misplaced faith in markets. There is nothing special about markets per se - they can perform very badly in some circumstances. It is competitive markets that are magic, though even then we have to remember that markets have no concern whatsoever with equity, only efficiency, and sometimes equity can be an overriding concern.

In order to work their magical efficiency, markets need very special conditions to be present. There must be full information available to all participants. Product quality, locations and prices of alternative suppliers, every relevant piece of information must be known. Not quite sure if the wine is good or not? That's an information problem. Not sure if the used car has problems? Don't know where any gas stations are except the ones beside the freeway in a strange town? No way to monitor the quality of the building built in Iraq with U.S. aid? No way to be sure if consultants are worth the amount they are being paid? Information problems are common and they can cause substantial departures from the perfectly competitive, ideal outcome.

There also must be numerous buyers and sellers, enough so that no single buyer or seller's decisions can affect the market price. For example, if a firm can affect the market price by threatening to limit supply, the market does not satisfy this condition. If, as some claim, CEOs are in such short supply that they can individually negotiate their compensation, then the market is not producing an efficient outcome. Whenever there are a small number of participants on either side of the market - suppliers or demanders - this is potentially problematic.

In order for markets to work their magic, the product must be homogeneous. That is, the product or input to production sold by all firms in the market must be perfectly substitutable so that as far as the buyer is concerned, one is as good as the other. If some buyers favor one brand over another, if CEOs are perceived to have different and unique talents, this condition does not hold. In many cases the variety may be worth the inefficiency, not many of us would want just one style and color of shirt to be available in stores, but the inefficiency is there nonetheless.

In order for markets to work their magic there must be free entry and exit. Most people understand free entry, but free exit is sometimes less evident, so let me try to give an example. Starting a blog on Blogger or TypePad is easy. Entry is a snap and you can be up and running in no time at all. It's easy to join the competition and start supplying posts. But suppose that later you decide you want to switch to, say, TypePad from Blogger (or the other way around). That is not so easy. There is no way, at least no simple and convenient way, to export all of your old posts from Blogger and import them into TypePad, a significant barrier to exit if a large number of posts must be moved. Whenever barriers exist in markets that prevent free movement into and out of the marketplace or between firms within a market (on either side - there are sometimes barriers to purchasing as well), markets will underperform.

The list goes on and on. In order for markets to work their magic, there can be no externalities, no public goods, no false market signals, no moral hazard, no principle agent problems, and, importantly, property rights must be well-defined (and I probably missed a few). In general, the incentives that the market provides must be consistent with perfect competition, or nearly so in practical applications. When the incentives present in the marketplace are inconsistent with a competitive outcome, there is no reason to expect the private sector to be efficient.

Markets don't work just because we get out of the way. When government contracts are moved to the private sector without ensuring the proper incentives are in place, there will be problems - waste, inefficiency, higher prices than needed, etc. There is nothing special about markets that guarantees that managers or owners of companies will have an incentive to use public funds in a way that maximizes the public rather than their own personal interests. It is only when market incentives direct choices to coincide with the public interest that the two sets of interests are aligned.

If there is no competition, or insufficient competition in the provision of government services by private sector firms, there is no reason to expect the market to deliver an efficient outcome, an outcome free of waste and inefficiency. Why would we think that giving a private sector firm a monopoly in the provision of a public service would yield an efficient outcome? If the projects are of sufficient scale, or require specialized knowledge so that only one or a few private sector firms are large enough or specialized enough to do the job, why would we expect an ideal outcome just because the private sector is involved? If cronyism limits the participants in the marketplace, why would we expect an outcome that maximizes the public interest?

There is nothing inherent in markets that guarantees a desirable outcome. A market can be a monopoly, a market can be perfectly competitive, a market can be lots of things. Markets with bad incentives produce bad outcomes, markets with good incentives do better.

I believe in markets as much as anyone. But the expression free markets is often misinterpreted to mean that unregulated markets are all that is required for markets to work their wonders and achieve efficient outcomes. But unregulated is not enough, there are many, many other conditions that must be present. Deregulation or privatization may even move the outcome further from the ideal competitive benchmark rather than closer to it, it depends upon the characteristics of the market in question.

For government goods and services, when incentives consistent with a competitive outcome are present, we should get government out of the way and privatize, and there are lots of circumstances where this will be appropriate. There is no reason at all for the government to produce its own pencils and pens, buying them from the private sector is more efficient so long as the bids are competitive.

When competitive conditions are not met but can be regulated, the regulations should be put in place and the private sector left to do its thing (e.g.  mandating that sellers disclose problems with a house to prevent asymmetric information or mandating that government funded projects be subject to competitive bidding and monitoring to ensure contract terms are met). There's no reason for government to do anything except ensure that the incentives to motivate competitive behavior are in place and enforced.

But rampant privatization based upon some misguided notion that markets are always best, privatization that does not proceed by first ensuring that market incentives are consistent with the public interest, doesn't do us any good. There are lots of free market advocates out there and I am with them so long as we understand that free does not mean the absence of government intervention, regulation, or oversight, even libertarians agree that governments must intervene to ensure basics like private property rights. Free means that the conditions for perfect competition are approximated as much as possible and sometimes that means the presence - rather than the absence - of government is required.

Efficiency versus stability

UPDATED BELOW

I had an opinion piece published today in Bloomberg Views looking at the relationship between market efficiency and stability, a topic which hasn't received much attention in the economics literature until recently. The point of the essay was to explore two distinct recent studies which suggest that adding more derivative instruments to markets tends to make them less stable, even if they do push markets toward the ideal of market completeness and efficiency.

I wanted to make available here some further technical information on the two studies I mentioned, but as publication arrived very quickly and I've been pressed with other deadlines I haven't yet managed to write the post as I wanted. However, I can at least offer some information with the idea of updating it very shortly (later today, Thursday 25 August).

I've given some extensive discussion of the first study I mentioned, by economists William Brock, Cars Hommes and Florian Wagener, in an earlier post.

The second study by Matteo Marsili is quite technical and relies for parts of its analysis on ideas and techniques imported from physics. I will tomorrow try to give some simplified discussion of the gist of this argument. What makes this particularly fascinating is that it works fully within the confines of standard general equilibrium models, and examines how market stability should evolve as the market approaches the ideal of market completeness. Agents are assumed to be fully rational, there are no problems with asymmetric information, etc. Even here, however, Marsili finds that the equilibrium becomes more and more unstable as the ideal is approached. Efficient markets are also unstable markets.

UPDATE

Marsili's argument is one he has been developing in a series of papers (with various co-authors) over several years. This paper from last year offers what is perhaps the most concise argument. It looks at a market with informed (fundamentalist) traders and non-informed (noise) traders, and shows, first, that the market becomes efficient as the number of informed traders grows. They are assumed in the model to have different kinds of private information about market outcomes, and the market becomes efficient, roughly speaking, once there are enough traders to cover the space of outcomes so all private information gets aggregated into market prices. The paper then introduces a non-informed trader -- a chartist or trend follower -- and shows that this trader has a maximum impact on the market precisely at the point at which it becomes efficient. The conclusion is very much against standard economic thinking:

[The results suggest} that information efficiency might be a necessary condition for bubble phenomena - induced by the behavior of non-informed traders...

Another paper from two years ago approaches the problem from a slightly different angle. This study looks explicitly at how the proliferation of financial instruments (derivatives) provides more means for diversifying and sharing risks and takes the market to an efficient state. However, it finds that this state is what physicists refer to as a "critical state", which is a state characterized by extreme (essentially infinite) susceptibility to small disturbances. Any small noise stirs up huge fluctuations. Again, efficiency trails instability in its wake. As the paper asserts:

This suggests that the hypothesis of Arbitrage Pricing Theory (the notion that arbitrage works to keep market in an efficient state) may not be compatible with a stable market dynamics.

This paper also makes the important point that market stability really ought to be thought of as a public good because well functioning markets do help everyone. But like most public goods, private individuals acting in their own interests will not likely provide it.

Finally, the paper I discussed in the Bloomberg article is from last year and analyses a model set up specifically so as to include the finance sector. It is very much akin to standard general equilibrium models, and includes essentially two components:

1. There are investors who aim to take their current wealth and preserve it (or make it grow) into the future. They do this by investing in various instruments provided by a sector of financial firms. These investors are assumed to be rational and have full information and they invest their wealth optimally over the set of possible investments.

2. There are financial firms who create the investment instruments and take on risks in supplying them. They also act optimally, and they hedge their risks by trading between themselves. Again, the firms are rational and have full information.

Marsili then studies what happens to this world of investors and financial firms optimally making decisions as the number of different financial instruments grows. The first result confirms expectations -- the financial firms are ever more successful in hedging their risks and they can provide the financial instruments more cheaply. Investors can therefore invest more effectively. The market becomes efficient.

But there are also two unexpected consequences. As Marsili describes them,

As markets approach completeness, however, two "unintended consequences" also arise: equilibrium portfolios develop a marked susceptibility to idiosynchratic shocks and/or parameter uncertainty and hedging engenders divergent trading volumes in the interbank market. Combining these, suggests an inverse relation between financial stability and the size of the financial sector...

In other words, the character of the optimum portfolios for both the investors and the financial firms becomes hugely sensitive to tiny shocks to the economy. As the efficient state is approached, these agents have to work ever harder to adjust their holdings to remain in the optimal condition. The market only remains efficient through an ever faster and more vigorous churning of investment positions. This shows up in the hedging done by the financial firms, where the volume of trading required to remain optimally hedged actually becomes infinite as the market reaches efficiency.

All three of these papers show much the same thing -- efficiency bringing instability along with it. But this latter paper may be the most interesting as it shows directly how the size of the financial sector also naturally explodes as this efficient-unstable regime is approached. The effect sounds suspiciously like what has happened in the past 30 years or so with massive growth in the financial industries in most developed nations.

What I find really remarkable, however, is that all of this comes from the very models that economists have been using for a long time to make arguments about market efficiency. Why did it take a physicist to look at what happens to stability at the same point? This seems bizarre indeed.

Leverage Control -- A Subtle Story

I mentioned recently some work (in progress) by Stefan Thurner and colleagues exploring how leverage influences stability (price volatility) in a competitive, speculative market. Thurner spoke about this at a meeting on Tipping Points in Durham, UK. What I find most appealing about this work is that is explores this question with a model that is rich enough to exhibit many of the basic features we see in speculative markets -- competition between hedge funds and other investment firms to attract investors' funds, the use of leverage to amplify potential gains, the monitoring of leverage by banks who lend to the investment firms, occasional abrupt crashes and bankruptcies, etc.

Is it a perfect model? Of course not, there is no such thing; models are tools for thinking. But it is arguably better than anything else we currently have for running "policy experiments" to test what might happen in such a market if regulators take this or that step -- establishing tight limits to allowed leverage, for example. 

Stefan kindly sent me the slides from his talk, a few of which I'd like to mention here. As I said, this is work in progress, so these are preliminary results. They're interesting because they suggest that avoiding dangerous market instability through leverage limits comes with costs, and that our intuition isn't at all a reliable guide -- we need these kinds of models in which we can discover surprising outcomes (before we discover them in reality).

I won't give a detailed description of the model; it can be found in an early draft of the paper available here. Thurner and colleagues have been working to improve the model over several years, and it now reproduces a number of realistic market behaviors quite naturally. Thurner summarized these as follows:

 

In other words, the hedge funds act to eliminate mis-pricings (taking volatility out of the market), and profit by doing so. Funds have to be aggressive to survive in the face of stuff competition, but suffer if they get too large. Risks shorten the lifetime of a fund. Overall, the models also reproduces the right statistical fluctuations in the market.

As I discussed in my earlier post in this work, competition between hedge funds leads naturally to increasing leverage and drives the market to have a fat-tailed distribution of returns; it becomes subject (like real market) to large price fluctuations as a matter of course driven by its own internal dynamics (no external impacts required). In this condition, the market is highly prone to catastrophic crashes triggered by nothing by small price fluctuations linked to noise traders (unsophisticated investors buying and selling more or less at random). The figure below shows a typical example, plotting the wealth of various funds versus time, with a dramatic crash that affects all funds at once (different colors for different funds):

Now, a natural question is -- could these kinds of events be avoided with proper regulations? One idea would be to restrict the amount of leverage allowed with the aim of keeping the market returns in a mode Gaussian regime, i.e. eliminating fat tails. People could probably argue for decades about whether this would work or not without coming to an answer; this model makes it possible to do an experiment to find out, which is what Thurner and colleagues have done.

Two figures (below) show some of the results, and require some explanation. The different colors correspond to different possible regulatory regimes, and show how behavior changes with maximum allowed hedge fund leverage : BLUE (no other regulations), PALE GREEN (regulations akin to Basel I and II, in which banks loaning to hedge funds are restricted by capital requirements) and RED (a situation in which banks monitor hedge funds and reduce a hedge fund's allowed leverage below the maximum when the volatility in its assets grows; a kind of adaptive leverage control). 

First, consider a figure showing how how the action of hedge funds, and their use of volatility, actually benefits the market -- making it more efficient (in one sense). 

The figure shows the mean square price volatility versus allowed leverage. Increasing leverage lets the hedge funds pounce on opportunities more aggressively and wipe out mis-pricings more effectively. Die hard free market people should love this as it shows that the effect is strongest in the absence of any regulation. The regulated markets require higher leverage to get the same reduction in volatility.

But this isn't the whole story. Now consider another figure for the probability (per unit time) of a failure of one of the hedge funds:

Here the pure free market solution isn't so good, as this probability rises rapidly with increasing leverage. There is a relatively low value of leverage (around 5 in the model's units) where the market benefits of leverage have already been realized, and more leverage only leads to more failures (because it takes the market into the regime of fat-tailed returns; this can happen even if the mean square volatility remains small).

The regulated markets in this case perform marginally better -- the regulations reduce the number of failures, and the cost for this is marginally increased volatility.

A surprising outcome is that these same regulations, in the regime of very high leverage, actually do worse than no regulations at all -- they lead to higher market volatility AND more failures as well, a truly perverse regime.

All in all, then, this model offers a sobering perspective on how regulators might go about trying to avoid crashes linked to fat tails by limiting leverage. Some limitation clearly seems to be good. But too much can be bad, especially when coupled with other market regulations. You can't test out one idea in isolation, because they interact in surprising ways.

I'll probably have some further comments on this in the near future. It's a work in progress, as is my understanding of it -- and of what it means for the bigger picture.