You are here

Building a Model Performance Measure for Examining Clinical Relevance Using Net Benefit Curves

Title: Building a Model Performance Measure for Examining Clinical Relevance Using Net Benefit Curves.
26 views
5 downloads
Name(s): Mukherjee, Anwesha, author
McGee, Daniel, professor directing dissertation
Hurt, Myra M., university representative
Slate, Elizabeth H., committee member
Sinha, Debajyoti, committee member
Florida State University, degree granting institution
College of Arts and Sciences, degree granting college
Department of Statistics, degree granting department
Type of Resource: text
Genre: Text
Doctoral Thesis
Issuance: monographic
Date Issued: 2018
Publisher: Florida State University
Place of Publication: Tallahassee, Florida
Physical Form: computer
online resource
Extent: 1 online resource (105 pages)
Language(s): English
Abstract/Description: ROC curves are often used to evaluate predictive accuracy of statistical prediction models. This thesis studies other measures which not only incorporate the statistical but also the clinical consequences of using a particular prediction model. Depending on the disease and population under study, the mis-classification costs of false positives and false negatives vary. The concept of Decision Curve Analysis (DCA) takes this cost into account, by using the threshold probability (the probability above which a patient opts for treatment). Using the DCA technique, a Net Benefit Curve is built by plotting "Net Benefit", a function of the expected benefit and expected harm of using a model, by the threshold probability. Only the threshold probability range that is relevant to the disease and the population under study is used to plot the net benefit curve to obtain the optimum results using a particular statistical model. This thesis concentrates on the process of construction of a summary measure to find which predictive model yields highest net benefit. The most intuitive approach is to calculate the area under the net benefit curve. We examined whether the use of weights such as, the estimated empirical distribution of the threshold probability to compute the weighted area under the curve, creates a better summary measure. Real data from multiple cardiovascular research studies- The Diverse Population Collaboration (DPC) datasets, is used to compute the summary measures: area under the ROC curve (AUROC), area under the net benefit curve (ANBC) and weighted area under the net benefit curve (WANBC). The results from the analysis are used to compare these measures to examine whether these measures are in agreement with each other and which would be the best to use in specified clinical scenarios. For different models the summary measures and its standard errors (SE) were calculated to study the variability in the measure. The method of meta-analysis is used to summarize these estimated summary measures to reveal if there is significant variability among these studies.
Identifier: 2018_Sp_Mukherjee_fsu_0071E_14350 (IID)
Submitted Note: A Dissertation submitted to the Department of Statistics in partial fulfillment of the requirements for the degree of Doctor of Philosophy.
Degree Awarded: Spring Semester 2018.
Date of Defense: April 11, 2018.
Keywords: Area under ROC Curve, Meta analysis, Net Benefit Curve, Predictive Accuracy, Summary Measure, Threshold Probability
Bibliography Note: Includes bibliographical references.
Advisory Committee: Daniel L. McGee, Professor Directing Dissertation; Myra Hurt, University Representative; Elizabeth Slate, Committee Member; Debajyoti Sinha, Committee Member.
Subject(s): Statistics
Persistent Link to This Record: http://purl.flvc.org/fsu/fd/2018_Sp_Mukherjee_fsu_0071E_14350
Host Institution: FSU

Choose the citation style.
Mukherjee, A. (2018). Building a Model Performance Measure for Examining Clinical Relevance Using Net Benefit Curves. Retrieved from http://purl.flvc.org/fsu/fd/2018_Sp_Mukherjee_fsu_0071E_14350