Drug discovery is one of the toughest real-world tests for AI reasoning. MedChemBench, a new benchmark from OpenAI, evaluates how well AI models can interpret molecular structures, reason from experimental evidence, and decide which compounds to pursue in early-stage medicinal chemistry. The benchmark draws on 117 papers and patents, covering 761 distinct molecules, and sets out to measure whether models can handle the tradeoffs that define the field—like balancing potency against absorption and stability.
The headline result: at the highest reasoning setting (xhigh), GPT‑6.2 Astra earned 84.5% of the available credit, with GPT‑6 Astra at 83.3%, GPT‑6.1 Sol at 82.0%, and GPT‑5.6 Sol trailing with 58.2%. The top models excel at reconstructing molecular structures and explaining experimental results, but still struggle with ranking compounds and predicting numerical properties.
What MedChemBench measures
MedChemBench is not a chemistry quiz—it’s a set of 99 problems that replicate real research decisions. For each, models receive context like the biological target, experimental findings, and the molecules studied so far. They’re scored on four fronts:
- Molecular structure: Can the model assemble the correct molecule, including atom connections and stereochemistry?
- Scientific reasoning: Does the explanation match the evidence for why one compound was chosen over another?
- Ranking and selection: Can the model rank compounds in a way that fits the stated requirements?
- Property prediction: How closely do its predictions match measured values for properties like permeability or breakdown rate?
Tradeoffs in molecular properties
One example problem concerns PRMT5 inhibitors, a cancer drug target. The model must predict how three related compounds perform: their activity against PRMT5, their ability to cross cell barriers, and how quickly liver enzymes degrade them. The real experimental data, held back from the model, showed classic tradeoffs: one compound was more potent, another crossed barriers better and was more stable. The benchmark checks if models can match these nuanced property profiles based on molecular structure and evidence.




