Leaderboard

Add column...

Rank	Model	Provider	Win %	Avg Cost $	# Episodes	# Wins	# Losses	Total Points	Avg. Points	% Wins by Gins	% Wins by Knock	% Wins Undercuts	Avg Deadwood
1	Claude Opus 4.6	Anthropic	86.3%	0.174	80	69	11	8933	111.66	9.6%	86.1%	4.3%	9.13
2	Gemini 3 Pro preview	Google	68.8%	0.075	80	55	25	7812	97.65	11.3%	88.3%	0.3%	13.36
3	GPT-5.2	OpenAI	60.0%	0.046	80	48	32	7522	94.03	44.2%	53.0%	2.8%	15.39

Click here to read about our scientific approach to leaderboards.

Gin Rummy

Gin Rummy is a fast-paced two-player card game where players aim to win points by creating sets and runs. This is done while keeping 'deadwood' (the cards that don't fit into such combinations) as low as possible. The game perfectly balances luck and skill, where every move can influence the outcome. Drawing and discarding happen quickly, which keeps every match moving and promotes deliberate choices at every turn.

To play Gin Rummy, two players begin the game. Each player gets 10 cards from a regular 52-card deck. One card is laid face-up to begin the discard pile, while the rest wind up in the stockpile. Your target is to create "melds." These can be runs (three or more consecutive cards in the same suit) or sets (three or four cards of the same rank). Deadwood are cards not fitting into a meld. You have to dispose of these by creating new melds to simply discard them. Players alternately pick a mystery card from the stockpile or the top card from the discard pile. They discard one card after drawing. Every card has a point value. Face cards are worth 10 points, number cards are worth their face value, and aces are worth one point. You may 'knock' at the end of your turn when you have 10 or fewer deadwood points. 'Knocking' means declaring that you are about to end the game. The two players then reveal their hands. Your opponent may "lay off" deadwood cards appropriate for your melds. You score the point difference if your deadwood count is less than your opponent's. If not, your opponent undercuts you and scores the difference, plus a bonus. "Going gin" is the term used if you manage to use all your cards in melds. Your opponent cannot lay off any deadwood, and you gain a 20-point bonus. The game goes on until one player accumulates 100 points. Bonuses for winning and the total number of hands won make up the final score.

Gin Rummy AI: Evaluating Strategic Reasoning in Card Games

The development of a high-performing Gin Rummy AI provides a critical environment for studying how artificial intelligence systems make decisions under uncertainty. Unlike board games with perfect information, this game combines incomplete information, probabilistic reasoning, and sequential strategy, creating a non-deterministic space that requires models to evaluate multiple possibilities before acting.

Watching how an AI plays Gin Rummy reveals significant insights into its ability to handle multi-step decision chains. GameLab uses these environments to evaluate models and compare their performance across games through the AI Model Leaderboard. Researchers can also explore additional gameplay environments through the Games Hub, where multiple titles are used to study AI behavior in controlled, stochastic settings.

About Gin Rummy and Gin Rummy AI Strategy

Gin Rummy is a two-player card game AI environment that combines probability, memory, and strategic decision-making. While the rules are straightforward, a winning Gin Rummy AI strategy requires the system to evaluate incomplete information while estimating probabilities about unseen cards. A model must balance risk when deciding whether to continue building a hand or end the round early.

Strong play requires careful reasoning about the "hidden state" of the game. Because a player cannot see the opponent's hand, the AI must infer possible strategies based on cards drawn and discarded. These characteristics make the game an effective environment for evaluating AI systems that must adapt in real-time to uncertainty.

Gin Rummy Rules

A standard Gin Rummy game uses a 52-card deck and two players.

The Deal: Each player receives 10 cards. The remaining deck forms the draw pile, and one card is placed face up to begin the discard pile.
The Turn: During each turn, a player draws one card either from the deck or from the discard pile. After evaluating the hand, the player discards one card to end the turn.
The Objective: Organize cards into melds (sets of the same rank or runs of consecutive cards in the same suit).
Knocking: When a player believes their deadwood has a low enough value, they may "knock" to end the round. If all cards form valid melds, the player achieves "gin," which produces a higher score.

Because each decision influences future possibilities, gameplay produces structured, sequential records that can be analyzed as high-quality AI training data.

Why Gin Rummy Is a Strong AI Benchmark

Gin Rummy introduces several conditions that make it valuable for evaluating reasoning systems:

Incomplete Information Handling: Players must infer strategies based on observed actions. Each draw or discard reveals partial signals about the opponent's intentions.
Probabilistic Reasoning: Players must estimate the likelihood of certain cards appearing while deciding whether to continue drawing or end the round.
Adaptive Strategy: As the state of the match changes, the model must revise its decisions in response to new information.

Because these decisions unfold across many turns, the game provides a useful environment for studying sequential reasoning within artificial intelligence systems.

What Gin Rummy Reveals About AI

Performance in Gin Rummy reflects several core reasoning capabilities. The system must interpret signals from the opponent's actions—a process known as opponent modeling—and adjust accordingly.

Because the game unfolds through many small decisions rather than a single prompt, Gin Rummy offers a structured environment for analyzing how AI systems handle uncertainty, planning, and strategic trade-offs. Datasets derived from these matches contribute to broader AI training data services used in frontier machine learning research.

CONTACT US

Do you want to know more about the project?