U.S. News Best Hospitals Rankings May Be Misleading, According to Comparion

BALTIMORE--()--Comparion Medical Analytics, a company specializing in comparing the cost and quality of medical care, released a new research study today which suggests patients and purchasers may be misled by hospital rankings released in U.S. News & World Report’s 2012-2013 national honor roll of best academic medical centers.

The U.S. News study evaluates major academic medical centers across 16 specialties, 12 of which are assessed on survival rates and patient safety indicators and the other four based on the hospital’s reputation among doctors who practice in that specialty. According to Dr. Thane Forthman, president and CEO of Comparion, the study has a number of potential shortcomings which may limit the usefulness of the rankings. These include:

  • The small number of quality indicators used to assess 75% of the specialties;
  • Use of subjective, reputation-based data to rank the other 25% of specialties;
  • Assumption that comparing academic medical centers to less than 10% of U.S. hospitals nationally provides reliable information regarding their quality of care.

Comparion’s research findings, displayed in Table 1 below, were based on the company’s patent-pending CareChex® composite quality scoring methodology which evaluates hospital care using a comprehensive set of process of care measures, outcome measures, and patient satisfaction scores. “Our study indicates that when a complete set of objective quality measures are used, the U.S. News rankings have no correlation to our CareChex quality rankings. In fact, the correlation is slightly negative.” says Dr. Forthman.

Table 1. Comparison of U.S. News and Comparion Hospital Rankings






Hospital/Medical Center




Hospital/Medical Center

1     Massachusetts General 1 Indiana University
2     Johns Hopkins 2 Mayo Clinic
3     Mayo Clinic 3 Duke University
4     Cleveland Clinic 4 University of Michigan
5     UCLA 5 Northwestern
6     Barnes-Jewish 6 Brigham and Women's
7     New York-Presbyterian 7 UCSF
8     Duke University 8 Massachusetts General
9     Brigham and Women's 9 UCLA
10     University of Pittsburgh 10 University of Pittsburgh
11     NYU Langone 11 New York-Presbyterian
12     Northwestern 12 Barnes-Jewish
13     UCSF 13 NYU Langone
14     Mount Sinai 14 Johns Hopkins
15     University of Pennsylvania 15 University of Pennsylvania
16     Indiana University 16 Cleveland Clinic
17     University of Michigan 17 Mount Sinai
(Spearman’s Correlation Coefficient = -.142; Confidence Interval = 95%; p < .01)

Of the hospitals ranked by U.S. News, only the University of Pittsburgh Medical Center and the Hospital of the University of Pennsylvania remained unchanged in their rankings (10th and 15th respectively). Both Indiana University Health and University of Michigan’s Hospitals and Health Centers experienced a radical improvement in their rankings, moving from the 16th and 17th respectively to 1st and 4th based on their CareChex scores. The largest drop in rankings was experienced by Johns Hopkins Hospital and the Cleveland Clinic which moved from 2nd and 4th respectively to 14th and 16th.

While these findings are significant in themselves, when these hospitals were compared to the national norm of all general, acute, non-federal U.S. hospitals, interesting observations were noted. Specifically, when each hospital’s CareChex composite quality score was evaluated, the distribution of quality performance across academic medical centers varied greatly--with many falling below the national average. Table 2 below identifies how each hospital’s quality performance compared to the nation.

Table 2. Distribution of Comparion’s National Composite Quality Scores by Hospital


Top 5%

  • Indiana University Health (99.6th percentile)
  • Mayo Clinic (97.7th percentile)
  • Duke University Medical Center (96.7th percentile)
  • University of Michigan’s Hospitals and Health Centers (95.5th percentile)

Top 10%

  • Northwestern Memorial Hospital (94.9th percentile)
  • Brigham and Women’s Hospital (94th percentile)

Top 15%

  • UCSF Medical Center (85.5th percentile)

Average (~ 50th percentile)

  • Massachusetts General Hospital
  • UCLA Medical Center
  • University of Pittsburgh Medical Center
  • New York-Presbyterian Hospital

Below Average (25th to 49.9th percentile)

  • Barnes-Jewish Hospital
  • NYU Langone Medical Center
  • The Johns Hopkins Hospital

Well Below Average (< 25th percentile)

  • Hospital of the University of Pennsylvania
  • The Cleveland Clinic
  • Mount Sinai Medical Center

Dr. Forthman noted, “In order to provide reliable comparisons among hospitals, the study removed unusually difficult cases (i.e., outliers) and adjusted for differences in patient risk factors (i.e., clinical and demographic characteristics). Otherwise, differences in quality performance could be attributed to differences in the complexity of cases treated. Additionally, all performance indicators were tested for statistical significance so a hospital’s relative rank and percentile score could be accurately determined.”

The important conclusion of Comparion’s research is patients and purchasers should not assume that academic medical centers ranked by U.S. News necessarily provide the best quality of care. There are many hospitals not affiliated with universities that provide equal or better care, and many U.S. News best hospitals actually perform poorly on comprehensive measures of quality. This conclusion is supported by other research conducted by Comparion which evaluated the quality performance of 118 university hospitals (Fierce Healthcare, November 2010; Becker’s Hospital Review, November 2010).

About Comparion Medical Analytics

Comparion Medical Analytics is one of the nation’s largest privately held healthcare information services companies with offices located in Greenville, SC, Baltimore, MD, and Phoenix, AZ. Over 1,000 healthcare professionals across leading academic medical centers, health systems, Fortune 500 companies, and payers rely on their software solutions to assess the cost and quality of medical services delivered across the continuum of care.

For more information regarding their products and services, visit ComparionAnalytics.com or contact info@comparionanalytics.com.


Comparion Medical Analytics
Dr. Thane Forthman, 864-979-4141

Release Summary

New study suggests US News Best Hospitals may not provide the best quality of care and the rank order of hospitals change dramatically when a more comprehensive set of quality indicators are used.


Comparion Medical Analytics
Dr. Thane Forthman, 864-979-4141