Machine Learning Helps Reveal What Makes Games Complex for People

Machine Learning Reveals Game Complexity for Humans

This article explores how machine learning (ML) and large-scale online experiments are revolutionizing the study of human behavior, particularly in understanding what makes strategic games complex for people. Traditional methods struggle with the inherent variability of human decision-making, but new toolkits combining online experiments with ML models offer powerful insights.

The Challenge of Understanding Human Behavior

Human behavior is notoriously variable, making it difficult to establish "laws" akin to those in physics. Psychologists and economists are now leveraging advanced tools to uncover the principles behind these behaviors.

A Typical Matrix Game Experiment

The research focuses on strategic decisions in economic games, often presented as payoff matrices. In a typical setup, a participant (row player) chooses between options A or B, while an anonymous opponent (column player) chooses between C or D. The outcomes, represented by payoffs (e.g., points convertible to money), are shown in a matrix. For instance, a participant choosing A and their opponent choosing C might result in payoffs of 17 points for the participant and 20 for the opponent.

Figure 1: A typical trial presented to our participants.

The Three-Step ML Pipeline for Behavioral Analysis

The researchers propose a three-step process:

Scale Up Data Collection: Utilize crowdsourcing platforms to gather vast amounts of decision data from participants playing designed games.
Automated Function Discovery with Neural Networks: Employ neural networks to learn the mapping between game features (what the player sees) and player choices (human behavior), capturing behavioral variation.
Opening the Black Box: Use interpretability tools to understand the neural network's learned rules, identifying which game aspects contribute to perceived difficulty and which cues players rely on.

This ML approach allows researchers to explore "what-if" scenarios by treating the trained neural network as a proxy for human decision-making, avoiding the need for new experiments for every variation.

Developing Theories of Strategic Choices with Neural Networks

The study involved creating over 2,400 unique 2x2 matrix games and collecting over 90,000 human decisions. A neural network model was trained on this data, achieving 92% accuracy in predicting human choices on unseen games.

Figure 2: A flexible neural network model based on multilayer perceptron (MLP) that learns how to map each game matrix to the row player's strategic choice.

To gain deeper theoretical insights, the researchers incorporated structures from behavioral game theory, such as level-k thinking (limited reasoning steps) and quantal response (probabilistic choices due to noise). These structured models, where neural networks predict the parameters of behavioral models (like level-2 quantal response), still achieved high accuracy (88%), demonstrating the power of grounding ML in established theories.

Figure 3: A structured neural network model where the final prediction is based on a level-k quantal response (QR) model, with both behavioral noise and belief noise generated by separate neural networks.

Identifying Game Complexity

The success of structured models stems from their ability to adapt noise levels based on game context, a feature traditional models lack. The researchers investigated what drives this noise by using LASSO regression to link game features to noisiness.

Figure 4: Developing an interpretable complexity index for strategic games.

Three key features emerged as predictors of perceived game complexity and resulting behavioral noise:

Payoff Dominance: Clarity of the Nash equilibrium payoff.
Arithmetic Difficulty: Complexity of calculating expected payoff differences.
Thinking Depth: The number of reasoning steps required for a rational decision.

These factors were used to develop a "Game Complexity Index." Higher index scores correlated with more random choices (closer to 50/50) and longer response times, indicating more cognitive effort. The index also proved effective in predicting behavior in new games and participants, showing its generalizability.

Conclusion

Machine learning and AI are not replacing traditional research but are enhancing them. By processing large datasets, these tools provide clear insights into human decision-making, bridging the gap between complex data and actionable theories in economics and psychology.