GPQA Diamond

What It Measures

GPQA (Graduate-level PhD Question-and-Answering) is a benchmark of multi-choice questions in physics, chemistry, and biology written by domain PhD students. The 'Diamond' subset is the highest-difficulty tier — questions where the writers consider non-experts would have difficulty even with internet access. The benchmark is designed to test deep specialized knowledge and reasoning rather than recall of widely-available information.

GPQA Diamond has emerged as one of the most credible benchmarks for evaluating frontier model reasoning capability. The questions are hard enough that even strong models score well below 100%, providing room for differentiation; the questions are written by PhD students rather than scraped from public sources, reducing contamination risk; and the subject matter spans enough disciplines that high scores genuinely reflect broad scientific reasoning rather than narrow specialization.

How It Works

Each question is a multi-choice problem with 4 answer choices, written by a domain PhD student in physics, chemistry, or biology. Questions are designed to require deep specialized knowledge — typically the kind of reasoning a graduate student in the relevant subfield would handle but a non-specialist would struggle with even given research time.

Scoring is the percentage of questions answered correctly. The 'Diamond' subset is approximately 200 questions selected as the hardest from the full GPQA pool — these are questions where the question writer estimates non-experts would have difficulty even with internet access during evaluation. Diamond is the standard subset cited in leaderboards and research reports.

Current Leaders