New territory for game theory...

This new paper in PLoS looks fascinating. I haven't had time yet to study it in detail, but it appears to make an important demonstration of how, when thinking about human behavior in strategic games, fixed point or mixed strategy Nash equilibria can be far too restrictive and misleading, ruling out much more complex dynamics, which in reality can occur even for rational people playing simple games: 

Abstract

Recent theories from complexity science argue that complex dynamics are ubiquitous in social and economic systems. These claims emerge from the analysis of individually simple agents whose collective behavior is surprisingly complicated. However, economists have argued that iterated reasoning–what you think I think you think–will suppress complex dynamics by stabilizing or accelerating convergence to Nash equilibrium. We report stable and efficient periodic behavior in human groups playing the Mod Game, a multi-player game similar to Rock-Paper-Scissors. The game rewards subjects for thinking exactly one step ahead of others in their group. Groups that play this game exhibit cycles that are inconsistent with any fixed-point solution concept. These cycles are driven by a “hopping” behavior that is consistent with other accounts of iterated reasoning: agents are constrained to about two steps of iterated reasoning and learn an additional one-half step with each session. If higher-order reasoning can be complicit in complex emergent dynamics, then cyclic and chaotic patterns may be endogenous features of real-world social and economic systems.

...and from the conclusions, ...

Cycles in the belief space of learning agents have been predicted for many years, particularly in games with intransitive dominance relations, like Matching Pennies and Rock-Paper-Scissors, but experimentalists have only recently started looking to these dynamics for experimental predictions. This work should function to caution experimentalists of the dangers of treating dynamics as ephemeral deviations from a static solution concept. Periodic behavior in the Mod Game, which is stable and efficient, challenges the preconception that coordination mechanisms must converge on equilibria or other fixed-point solution concepts to be promising for social applications. This behavior also reveals that iterated reasoning and stable high-dimensional dynamics can coexist, challenging recent models whose implementation of sophisticated reasoning implies convergence to a fixed point [13]. Applied to real complex social systems, this work gives credence to recent predictions of chaos in financial market game dynamics [8]. Applied to game learning, our support for cyclic regimes vindicates the general presence of complex attractors, and should help motivate their adoption into the game theorist’s canon of solution concepts

Rational -- by definition and ideology

I've been doing some background reading on rationality in economics, and came across this fairly unique perspective offered by economist Duncan Foley. It's from 2003. What sets it apart from most other reviews of the role of the rationality assumption in economics, is that Foley tries to trace the history of this approach as it emerged out of the tradition of Hobbes and Locke in political philosophy. As Foley notes, the idea of rationality is in many ways beyond question for most economists, and not at all an empirical matter:

...an orientation toward situating explanations of economic phenomena in relation to rationality has increasingly become the touchstone by which mainstream economists identify themselves and recognize each other. This is not so much a question of adherence to any particular conception of rationality, but of taking rationality of individual behavior as the unquestioned starting point of economic analysis.

 It has become so, he asserts, because this way of thinking has emerged from the "just so" story developed by Hobbes, Locke and others which allegedly explains how property rights and political institutions solve problems arising from the anarchic struggle of man against man in the original state of nature. They place reason at the core of this project, and essentially use this story to explain why things are as they are -- this is the rational world and the only way things can be, if we are to avoid the chaos of anarchy. In essence, the rationality assumption is part of a propaganda campaign. Foley:

A hallmark of these [rationally designed] institutions is that they are in themselves in principle democratic and egalitarian (everyone has an equal right to vote or to hold property) but lead
inexorably to sharp inequalities in economic well-being. It is not hard to see that an economic science whose philosophical starting point was not rational individual action would create an embarrassing discord with this political tradition. The whole point of the Hobbes-Locke “discourse” (to use the jargon of post-modernism) is to rationalize the existing inequalities of power and economic well-being that arise from the institutions of modern society as being unavoidable consequences of the interaction of naturally constituted rational individuals
confronting each other as equals, given the natural and unalterable conditions of human existence. Economic science has a place in this grand project only insofar as it can relate itself to the same philosophical foundations.

I think there's a strong current of truth here. There is in today's economic theory a standing presumption that people should be modelled as rational decision makers (optimizers), and the argument often seems to boil down (in some disguised form) to "we must, because if we do not, we will not be able to prove theorems about equilibrium and its efficiency." This is of course too strong, and research programs in behavioural economics, information asymmetries and so on seem to be working to correct this, but the effort required reflects how much resistance there is to such change and how much intellectual inertia still resides in the idea of thorough-going rationality. Foley suggests that efforts to bring more realistic perspectives such as bounded rationality into core theory have been resisted precisely because they cannot be used to justify the just-so story of the efficient equilibrium:

...in its pragmatic focus on understanding and explaining how people actually behave in modern society, bounded rationality loses contact with the underlying project of rationalizing the institutions of modern society. For example, there really is no logical place in the discourse of bounded rationality for the Fundamental Theorems of Welfare Economics that purport to establish a connection between competitive market equilibrium and an efficient allocation of resources.

 I think he may largely be right. If so, this would go a long way to explaining why economics has persisted with such a narrow set of theoretical concepts for such a long time. Maybe it's not actually trying to explain and understand the world at all, but to rationalize why it is OK that it is as we see it. And that's not encouraging for those of us hoping it will change in a big way:

It will not be easy to create a social science that transcends the antinomies and limitations of rational-actor theory. Certainly we cannot depend on the “usual” processes of scientific self-criticism to accomplish much in this direction. No accumulation of its empirical anomalies, or demonstration of its logical inadequacies will somehow magically dispel the power of rational-actor theory, because its power does not rest in the last instance on the adequacy of its
explanations or the consistency of its logic.

Difficulties with learning...

I just finished reading this wonderful short review of game theory (many thanks to ivansml for pointing this out to me) and its applications and limitations by Martin Shubik. It's a little old -- it appeared in the journal Complexity in 1998 -- but offers a very broad perspective which I think still holds today. Game theory in the pure sense generally views agents as coming to their strategies through rational calculation; this perspective has had huge influence in economics, especially in the context of relatively simple games with few players and not too many possible strategies. This part of game theory is well developed, although Shubik suggests there are probably many surprises left to learn.

Where the article really comes alive, however, is in considering the limitations to this strictly rational approach in games of greater complexity. In physics, the problem of two rigid bodies in gravitational interaction can be solved exactly (ignoring radiation, of course), but you get generic chaos as soon as you have three bodies or more. The same is true, Shubik argues, in game theory. Extend the number of players above three and as the number of possible permutations of strategies proliferates it is no longer plausible to assume that agents act rationally. The decision problems become too complex. One might still try to search for optimal N player solutions as a guide to what might be possible, but the rational agent approach isn't likely to be profitable as a guide to the likely behaviour and dynamics in such complex games. I highly recommend Shubik's short article to anyone interested in game theory, and especially its application to real world problems where people (or other agents) really can't hope to act on the basis of rational calculation, but instead have to use heuristics, follow hunches, and learn adaptively as they go.

Some of the points Shubik raises find perfect illustration in a recent study (I posted on it here) of typical dynamics in two-player games when the number of possible strategies gets large. Choose the structure of the games at random and the most likely outcome is a rich ongoing evolution of strategic behaviour which never settles down into any equilibrium. But these games do seem to show characteristic dynamical behaviour such as "punctuated equilibrium" -- long periods of relative quiescence which get broken apart sporadically by episodes of tumultuous change -- and clustered volatility -- the natural clustering together of periods of high variability. These qualitative aspects appear to be generic features of the non-equilibrium dynamics of complex games. Interesting that they show up generically in markets as well.

When problems are too complex -- which is typically the case -- we try to learn and adapt rather than "solving" the problem in any sense. Our learning itself may also never settle down into any stable form, but continually change as we find something that works well for a time, and then suddenly find it fails and we need to learn again.

Why game theory is often useless...

Economic theory relies very heavily on the notion of equilibrium. This is true in any model for competitive equilibrium -- exploring how exchange can in principle lead to an optimal allocation of resources -- or more generally in the context of game theory, which explores stable Nash equilibria in strategic games.

One thing physicists find wholly unsatisfying about equilibrium in either case is economists' near total neglect of the crucial problem of whether the agents in such models might ever plausibly find an equilibrium. You can assume perfectly rational agents and prove the existence of an equilibrium, but this may be an irrelevant mathematical exercise. Realistic agents with finite reasoning powers might never be able to learn their way to such a solution.

More likely, at least in many cases, is that less-than-perfectly rational agents, even if they're quite clever at learning, may never find their way to a neat Nash equilibrium solution, but instead go on changing and adapting and responding to one another in a way that leads to ongoing chaos. Naively, this would seem especially likely in any situation -- think financial markets, or any economy as a whole -- in which the number of possible strategies is enormous and it is simply impossible to "solve the problem" of what to do through perfect rational reflection (no one plays chess by working out the Nash equilibrium).

A brilliant illustration of this insight comes in a new paper by Tobias Galla and Doyne Farmer. This is the first study I've seen (though there may well be others) which addresses this matter of the relevance of equilibrium in complex, high-dimensional games in a  generic way. The conclusion is as important as it is intuitively reasonable:

Here we show that if the players use a standard approach to learning, for complicated games there is a large parameter regime in which one should expect complex dynamics. By this we mean that the players never converge to a fixed strategy. Instead their strategies continually vary as each player responds to past conditions and attempts to do better than the other players. The trajectories in the strategy space display high-dimensional chaos, suggesting that for most intents and purposes the behavior is essentially random, and the future evolution is inherently unpredictable.

In other words, in games of sufficient complexity, the insights coming from equilibrium analyses just don't tell you much. If the agents learn in a plausible way, they never find any equilibrium at all, and the evolution of strategic behaviours simply carries on indefinitely. The system remains out of equilibrium.

A little more detail. Their basic approach is to consider general two player games between, say, Alice and Bob. Let each of the two players have N possible strategies to choose from. The payoff matrices for any such game are NxN matrice (one for each player) giving the payoffs they get for each pair of strategies being played. The cute idea in this analysis is to choose the game randomly by selecting the elements of the payoff matrices for both Alice and Bob from a normal distribution centered on zero. The authors simply choose a game and simulate play as the two players learn through experience -- playing strategies from their repertoire of N possibilities more frequently if those strategies give good results.

With N = 50, the results show clearly that many games do not ever settle into any kind of stable behaviour. Rather, no equilibrium is ever found. The typical dynamics is reflected in the figure below, which shows the difference in payoffs to the two players (Alice's - Bob's) over time. Even though the two agents work hard to learn the optimal strategies, the complexity of the game prevents their success, and the game shows no signs whatsoever of settling down:

As the authors note, this kind of rich, complex, ongoing dynamics looks quite similar to what one sees in real systems such as financial markets (the time series above exhibits clustered volatility, as do market fluctuations). There are periods of relative calm punctured by bouts of extreme volatility. Yet there's nothing intervening here -- no "shocks" to the system -- which would create these changes. It all comes from perfectly natural internal dynamics. And this is in a game with N = 50 strategies. It seems likely things will only grow more chaotic and less likely to settle down if N is larger than 50, as in the real world, or if the number of players grows beyond two.

Hence, I see this as a rather profound demonstration of the likely irrelevance of equilibrium analyses coming from game theory to complex real world settings. Dynamics really matters and cannot be theorized out of existence, however hard economists may try. As the paper concludes:

Our results suggest that under many circumstances it is more useful to abandon the tools of classic game theory in favor of those of dynamical systems. It also suggests that many behaviors that have attracted considerable interest, such as clustered volatility in nancial markets, may simply be specific examples of a highly generic phenomenon, and should be expected to occur in a wide variety of different situations.