Big Data and Predictive Analytics Can Transform US Healthcare System, According to NYU Stern Study Published in Health Systems

Professors Use Type-2 Diabetes as Case Study

NEW YORK--()--For more than a decade, banks and retailers have been using “big data” to draw actionable conclusions from data they collect through commerce. Now, two NYU Stern researchers say big data can help solve major societal problems, like reducing the skyrocketing cost of healthcare in the US while improving quality.

In a study published in Health Systems, Professor Vasant Dhar, co-director of Stern’s Center for Business Analytics, and his colleague Jon Maguire, use predictive analytics on a large dataset from the healthcare system to identify groups most at risk for diabetic complications and predict treatment costs associated with different treatment patterns. According to the Congressional Budget Office, healthcare spending tripled as a percentage of GDP from 1960-2005 and could more than triple again by 2082 to consume nearly half of GDP unless costs are contained. The researchers argue that big data can show us how to reduce costs and improve outcomes.

“Healthcare providers are understandably focused on individuals and not fully informed by large-scale data-driven patterns of treatments and outcomes,” said Dhar. “The good news is that we now have troves of available data of healthcare system use. What we need is a healthcare system that is willing to let the data speak and show us previously unknown patterns that emerge even though the reasons for such patterns may not always be immediately apparent.”

In their study, they mined pharmacy and medical claims of more than 65,000 newly diagnosed type-2 diabetes patients, age 18 and older, over six years. The NYU Stern study revealed the following actionable patterns that are relevant to patients, healthcare providers and insurers:

  • A large portion of costs arise from very few cases: 68% of utilization among those newly diagnosed with type-2 diabetes are incurred by the “sickest” 10% of the population. Predicting the “soon to be sickest 10%” could significantly cut costs and improve outcomes.
  • People who start with a "lifestyle only" solution after diagnosis (no medicine for up to six months) tend to have higher complication rates than those who go on a simple medication such as Metformin, suggesting that delay may be costly. While some controlled studies suggest that lifestyle factors such as regular vigorous exercise may prevent the onset of diabetes, timely medication may be more effective after its onset than relying on lifestyle alone.
  • Patients with poorer ‘health status’ have higher costs and complication rates. In the study, one proxy for health status was the number of different therapeutic class prescriptions or pre-existing diagnoses before a diabetes diagnosis. This surrogate measure and other related ones should be considered during treatment since they can impact outcomes.

Prof. Dhar is an expert in the study of predictive analytics, data mining, big data and digital marketing.

To arrange an interview with Prof. Dhar, contact Jessica Neville, 416-516-7677,, in Stern’s Office of Public Affairs, or Prof. Dhar at

For access to his Health Systems paper, visit their website.


NYU Stern
Jessica Neville, 416-516-7677

Release Summary

Two NYU Stern researchers say big data can help solve major societal problems, like reducing the skyrocketing cost of healthcare in the US while improving quality.


NYU Stern
Jessica Neville, 416-516-7677