New territory for game theory...

This new paper in PLoS looks fascinating. I haven't had time yet to study it in detail, but it appears to make an important demonstration of how, when thinking about human behavior in strategic games, fixed point or mixed strategy Nash equilibria can be far too restrictive and misleading, ruling out much more complex dynamics, which in reality can occur even for rational people playing simple games: 

Abstract

Recent theories from complexity science argue that complex dynamics are ubiquitous in social and economic systems. These claims emerge from the analysis of individually simple agents whose collective behavior is surprisingly complicated. However, economists have argued that iterated reasoning–what you think I think you think–will suppress complex dynamics by stabilizing or accelerating convergence to Nash equilibrium. We report stable and efficient periodic behavior in human groups playing the Mod Game, a multi-player game similar to Rock-Paper-Scissors. The game rewards subjects for thinking exactly one step ahead of others in their group. Groups that play this game exhibit cycles that are inconsistent with any fixed-point solution concept. These cycles are driven by a “hopping” behavior that is consistent with other accounts of iterated reasoning: agents are constrained to about two steps of iterated reasoning and learn an additional one-half step with each session. If higher-order reasoning can be complicit in complex emergent dynamics, then cyclic and chaotic patterns may be endogenous features of real-world social and economic systems.

...and from the conclusions, ...

Cycles in the belief space of learning agents have been predicted for many years, particularly in games with intransitive dominance relations, like Matching Pennies and Rock-Paper-Scissors, but experimentalists have only recently started looking to these dynamics for experimental predictions. This work should function to caution experimentalists of the dangers of treating dynamics as ephemeral deviations from a static solution concept. Periodic behavior in the Mod Game, which is stable and efficient, challenges the preconception that coordination mechanisms must converge on equilibria or other fixed-point solution concepts to be promising for social applications. This behavior also reveals that iterated reasoning and stable high-dimensional dynamics can coexist, challenging recent models whose implementation of sophisticated reasoning implies convergence to a fixed point [13]. Applied to real complex social systems, this work gives credence to recent predictions of chaos in financial market game dynamics [8]. Applied to game learning, our support for cyclic regimes vindicates the general presence of complex attractors, and should help motivate their adoption into the game theorist’s canon of solution concepts