A course in stochastic game theory

Stochastic games have an element of chance: the state of the next round is determined probabilistically depending upon players' actions and the current state. Successful players need to balance the need for short-term payoffs while ensuring future opportunities remain high. The various techniques needed to analyze these often highly non-trivial games are a showcase of attractive mathematics, including methods from probability, differential equations, algebra, and combinatorics. This book presents a course on the theory of stochastic games going from the basics through to topics of modern research, focusing on conceptual clarity over complete generality. Each of its chapters introduces a new mathematical tool - including contracting mappings, semi-algebraic sets, infinite orbits, and Ramsey's theorem, among others - before discussing the game-theoretic results they can be used to obtain. The author assumes no more than a basic undergraduate curriculum and illustrates the theory with numerous examples and exercises, with solutions available online.

• Introduction
• 1. Markov decision problems
• 2. A Tauberian theorem and uniform -optimality in hidden Markov decision problems
• 3. Strategic-form games - a review
• 4. Stochastic games - the model
• 5. Two-player zero-sum discounted games
• 6. Semi-algebraic sets and the limit of the discounted value
• 7. B-Graphs and the continuity of the limit $\lim_{\lambda \to 0} v_\lambda(s
• q,r)$
• 8. Kakutani's fixed-point theorem and multi-player discounted stochastic games
• 9. Uniform equilibrium
• 10. The vanishing discount factor approach and uniform equilibrium in absorbing games
• 11. Ramsey's theorem and two-player deterministic stopping games
• 12. Infinite orbits and quitting games
• 13. Linear complementarity problems and quitting games
• References
• Index.