Assessing the accuracy of predictive models for numerical data: Not r nor r2, why not? Then what?

Assessing the accuracy of predictive models is critical because predictive models have been increasingly used across various disciplines and predictive accuracy determines the quality of resultant predictions. Pearson product-moment correlation coefficient (r) and the coefficient of determination (r2) are among the most widely used measures for assessing predictive models for numerical data, although they are argued to be biased, insufficient and misleading. In this study, geometrical graphs were used to illustrate what were used in the calculation of r and r2 and simulations were used to demonstrate the behaviour of r and r2 and to compare three accuracy measures under various scenarios. Relevant confusions about r and r2, has been clarified. The calculation of r and r2 is not based on the differences between the predicted and observed values. The existing error measures suffer various limitations and are unable to tell the accuracy. Variance explained by predictive models based on cross-validation (VEcv) is free of these limitations and is a reliable accuracy measure. Legates and McCabe’s efficiency (E1) is also an alternative accuracy measure. The r and r2 do not measure the accuracy and are incorrect accuracy measures. The existing error measures suffer limitations. VEcv and E1 are recommended for assessing the accuracy. The applications of these accuracy measures would encourage accuracy-improved predictive models to be developed to generate predictions for evidence-informed decision-making.


<b>Citation:</b> Li J (2017) Assessing the accuracy of predictive models for numerical data: Not r nor r2, why not? Then what? <o>PLoS ONE</i> 12(8): e0183250. https://doi.org/10.1371/journal.pone.0183250

Simple

Identification info

Date (Creation)
2016-08-12
Date (Publication)
2024-12-24T00:51:01
Citation identifier
Geoscience Australia Persistent Identifier/https://pid.geoscience.gov.au/dataset/ga/101500

Cited responsible party
Role Organisation / Individual Name Details
Publisher

Public Library of Science (PLoS) Journals

 External Contact
Author

Li, J.

 External Contact
Name

Public Library of Science (PLoS ONE) Journal

Issue identification

Volume 12(8): e0183250

Page

1-16

Purpose

Manuscript for publication in PLoS ONE journal

Status
Completed
Point of contact
Role Organisation / Individual Name Details
Point of contact

Commonwealth of Australia (Geoscience Australia)

 Voice
Resource provider

Place, Space and Communities Division

 External Contact
Point of contact

Nichol, S.

 Place and Communities Internal Contact
Spatial representation type
Topic category
  • Geoscientific information

Extent

Extent

N
S
E
W


Maintenance and update frequency
As needed

Resource format

Title

Product data repository: Various Formats

Website

Data Store directory containing the digital product files

Data Store directory containing one or more files, possibly in a variety of formats, accessible to Geoscience Australia staff only for internal purposes

theme.ANZRC Fields of Research.rdf

  • EARTH SCIENCES

Discipline

  • predictive models

Data centre

  • predictive accuracy

Keywords

  • Published_External

Resource constraints

Title

Creative Commons Attribution 4.0 International Licence

Alternate title

CC-BY

Edition

4.0

Website

http://creativecommons.org/licenses/

Access constraints
License
Use constraints
License
Other constraints

© Commonwealth of Australia (Geoscience Australia) 2017

Resource constraints

Title

Australian Government Security ClassificationSystem

Edition date
2018-11-01T00:00:00
Website

https://www.protectivesecurity.gov.au/Pages/default.aspx

Classification
Unclassified
Language
English
Character encoding
UTF8

Distribution Information

Distributor contact
Role Organisation / Individual Name Details
Distributor

Commonwealth of Australia (Geoscience Australia)

 Voice facsimile
OnLine resource

Link to Journal

Link to Journal

Distribution format

Resource lineage

Statement

Manuscript for publication in Public Library of Science (PLoS ONE) journal

Hierarchy level
Dataset

Metadata constraints

Title

Australian Government Security ClassificationSystem

Edition date
2018-11-01T00:00:00
Website

https://www.protectivesecurity.gov.au/Pages/default.aspx

Classification
Unclassified

Metadata

Metadata identifier
urn:uuid/d94ba716-6522-4b45-b18f-a62ee5c7d4ae

Title

GeoNetwork UUID

Language
English
Character encoding
UTF8
Contact
Role Organisation / Individual Name Details
Point of contact

Commonwealth of Australia (Geoscience Australia)

 Voice
Point of contact

Nichol, S.

 Place and Communities Internal Contact

Type of resource

Resource scope
Document
Name

Journal Article

Alternative metadata reference

Title

Geoscience Australia - short identifier for metadata record with

uuid

Citation identifier
eCatId/101500

Metadata linkage

https://ecat.ga.gov.au/geonetwork/srv/eng/catalog.search#/metadata/d94ba716-6522-4b45-b18f-a62ee5c7d4ae

Metadata linkage

https://ecat.ga.gov.au/geonetwork/js/eng/catalog.search#/metadata/d94ba716-6522-4b45-b18f-a62ee5c7d4ae

Metadata linkage

https://ecat.ga.gov.au/geonetwork/ofmJ3/eng/catalog.search#/metadata/d94ba716-6522-4b45-b18f-a62ee5c7d4ae

Date info (Creation)
2016-08-12T12:02:30
Date info (Revision)
2016-08-12T12:06:20

Metadata standard

Title

AU/NZS ISO 19115-1:2014

Metadata standard

Title

ISO 19115-1:2014

Metadata standard

Title

ISO 19115-3

Title

Geoscience Australia Community Metadata Profile of ISO 19115-1:2014

Edition

Version 2.0, September 2018

Citation identifier
https://pid.geoscience.gov.au/dataset/ga/122551

 
 

Spatial extent

N
S
E
W


Keywords

predictive accuracy predictive models
theme.ANZRC Fields of Research.rdf
EARTH SCIENCES

Provided by

Access to the portal

Associated resources

Not available