Two players share one rack. Solids are 1–7, stripes are 9–15, and the 8-ball is last. Player one breaks from the kitchen. The cue starts behind the head string; you may slide it inside that kitchen area, but you do not have to—the default spot is already legal. After any later foul, ball-in-hand is the whole table.
Groups are not chosen on the break. On a later shot, whoever pockets more solids than stripes (or the reverse) takes that group. Equal counts leave the table open. While the table is open, either group is legal to hit first. You keep shooting after pocketing a ball of your group (or any object ball while open), unless you foul.
Fouls include scratching the cue, hitting the opponent's ball first after groups are assigned, missing all balls after groups are assigned, hitting nothing to a rail when nothing is pocketed, hitting the 8 first while you still have group balls, and an illegal break (fewer than four object balls to a rail and nothing pocketed). The incoming player then has the cue in hand.
Call a pocket only when your seven object balls are gone, then pocket the 8 in that pocket without fouling to win. Pocketing the 8 early, in the wrong pocket, or with a foul is a loss. An 8 pocketed on the break is spotted back and play continues. Object balls do not need to be called; only the 8 does.
8-Ball Billiards as an Evaluation Tool
8-Ball Billiards evaluates spatial reasoning, physical prediction, precision control, and long-horizon strategic planning. With the table state fully visible, success depends on a model’s ability to translate geometric reasoning into effective shots while anticipating how collisions will change the position of multiple balls.
Every shot involves both an immediate objective and a positioning decision. Strong performance requires evaluating angles, shot power, and cue-ball movement to pocket a target ball while creating favorable conditions for subsequent shots. Models that focus solely on the easiest available pot often leave difficult follow-up shots, obstruct their remaining balls, or expose opportunities for the opponent.
The game also requires balancing offensive play against defensive safety shots, managing foul risk, and planning a viable route to the final 8-ball. In a controlled simulation, performance provides a compact test of causal reasoning, action precision, and sequential decision-making, revealing whether a model can connect an intended outcome with the physical actions needed to achieve it.
Harness and Structure
The benchmark uses Arkadium's 8 Ball Billiards environment connected to a universal text-based evaluation harness.
Each prompt contains:
- Current board state including which balls were pocketed and which balls are valid for play, strength to be applied and direction.
The board is represented using a coordinate-based textual format. Models must return a structured placement action specifying both the selected piece and target location.
The harness consists of three major components:
- Prompt Builder: Converts the board and available pieces into a machine-readable representation.
- Response Parser: Extracts placement actions and validates legality.
- Game Engine: Updates the board state, resolves line clears, awards points, and determines termination conditions.
Invalid placements are rejected and counted separately from strategic errors.
Evaluation
Models are evaluated through repeated gameplay sessions.
Performance is measured using:
- Win/loss state
Notes
- Deterministic environment: All information is visible.
- Physics reasoning: Models must evaluate direction, strength and trajectories.
- Resource management: What balls are valid for their play.
- Long-horizon planning: Short-term scoring opportunities can create long-term failures.
- Text-only interaction: The board is represented symbolically rather than visually.
- No external tools: Models rely solely on the information contained within the prompt.
Contact Us
Do you want to know more about the project?