Doximity Named Leading Clinical AI Benchmark in Fireworks’ Cross-Industry Index

Joins Sierra and Harvey as category leaders spanning healthcare, customer support, and law

SAN FRANCISCO--(BUSINESS WIRE)--Doximity, Inc. (NYSE: DOCS), the leading digital platform for U.S. medical professionals, today announced that it has been named a founding partner in the launch of a new Specialized Intelligence Index of leading AI benchmarks compiled by Fireworks, the AI training and inference platform.

As part of the launch, Doximity is publishing a new open-source benchmark for measuring clinical-grade AI. Called Bedside Bench, the benchmark has been built to test AI's ability to be a trusted partner to doctors where it matters most: at the bedside.

Bedside Bench is one of several domain-specific evaluations included in Fireworks’ new Index. In addition to clinical medicine, the Index includes benchmarks developed with Harvey for law and Sierra for customer support.

A More Realistic Test for Clinical-Grade AI

Bedside Bench cases emphasize areas that physicians and AI developers have identified as having the biggest impact in practice. Whereas traditional clinical AI benchmarks often use multiple-choice questions to measure recall, Bedside Bench uses clinical scenarios so it more closely measures real-life performance. Test areas include:

• Drug-safety questions where small errors can have major health consequences
• Ability to correctly apply specific evidence from drug guidelines and trials
• Open-ended questions with multiple reasonable answers but with important differences in completeness and safety
• Prompts containing made-up drugs, or nonexistent studies and trials
• Questions designed to uncover health-equity mistakes and bias
• The ability to include important actions from a long list of options while avoiding inappropriate ones

When independently graded by Fireworks, Doximity Ask ranked first versus frontier models. This result follows Doximity Ask outranking all other U.S.-based models in July’s independent NOHARM study from researchers at Stanford and Harvard. This provides yet another signal that AI systems built specifically for medicine exceed leading general-purpose frontier models on clinically relevant tasks.

Doximity’s Head of Medical AI, Dr. Louis-Antoine Mullie said: “Clinicians need to know whether an AI system can be trusted on the kinds of decisions that actually arise in practice. Bedside Bench was built around physician-validated, real-world clinical tasks and the risks that matter most in medicine, from drug safety to missed actions and incorrect use of evidence.”

To learn more about Doximity Ask, visit www.doximity.com.

About Doximity

Founded in 2010, Doximity is the leading digital platform for U.S. medical professionals. The company's network members include more than 85% of U.S. physicians. With AI-powered clinical reference and search capabilities, Doximity helps doctors access trusted, peer-reviewed information and medical literature. Doximity's mission is to help doctors be more productive so they can provide better care for their patients.