Skip to content
MAGICS Lab

Nature Scientific Reports · 2024

Strategic Behavior of Large Language Models and the Role of Game Structure versus Contextual Framing

Do LLMs reason strategically, or just react to framing? Both — and the mix depends on the model.

Nunzio Lorè, Babak Heydari

Three-panel summary: (1) study goal — what shapes LLM strategic behavior more, game structure or contextual framing; (2) experimental setup — four games × five framings × GPT-3.5, GPT-4, LLaMa-2 producing cooperate-or-defect choices; (3) key findings — GPT-3.5 is context-driven, GPT-4 is structure-centric and bimodal, LLaMa-2 balances both.
Three-panel summary: (1) study goal — what shapes LLM strategic behavior more, game structure or contextual framing; (2) experimental setup — four games × five framings × GPT-3.5, GPT-4, LLaMa-2 producing cooperate-or-defect choices; (3) key findings — GPT-3.5 is context-driven, GPT-4 is structure-centric and bimodal, LLaMa-2 balances both.

Do large language models actually reason strategically, or do they just respond to surface framing? Across four canonical two-player games (Prisoner’s Dilemma, Stag Hunt, Snowdrift, Harmony) and five framings (business, diplomacy, environment, teammates, friends), three models behave differently:

  • GPT-3.5 follows context and largely ignores game structure.
  • GPT-4 mostly tracks structure but collapses the four games into a binary cooperate-or-defect choice.
  • LLaMa-2 distinguishes the games more finely while still being moved by framing.

No model fully escapes the influence of how a situation is described — a finding with direct implications for deploying LLMs as strategic agents.