kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Investigating Normalized Conformal Regressors
Jönköping Univ, Dept Comp, Jönköping, Sweden..
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Software and Computer systems, SCS.ORCID iD: 0000-0001-8382-0300
Jönköping Univ, Dept Comp, Jönköping, Sweden..
2021 (English)In: 2021 IEEE Symposium Series on Computational Intelligence, SSCI 2021 - Proceedings, Institute of Electrical and Electronics Engineers (IEEE) , 2021Conference paper, Published paper (Refereed)
Abstract [en]

Conformal prediction can be applied on top of any machine learning predictive regression model, thus turning it into a conformal regressor. Given a significance level epsilon, conformal regressors output valid prediction intervals, i.e., the probability that the interval covers the true value is exactly 1 - epsilon. To obtain validity, a calibration set that is not used for training the model must be set aside. In standard inductive conformal regression, the size of the prediction intervals is then determined by the absolute error made by the predictive model on a specific instance in the calibration set, where different significance levels correspond to different instances. In this setting, all prediction intervals will have the same size, making the resulting models very unspecific. When adding a technique called normalization, however, the difficulty of each instance is estimated, and the interval sizes are adjusted accordingly. An integral part of normalized conformal regressors is a parameter called beta, which determines the relative importance of the difficulty estimation and the error of the model. In this study, the effects of different underlying models, difficulty estimation functions and beta-values are investigated. The results from a large empirical study, using twenty publicly available data sets, show that better difficulty estimation functions will lead to both tighter and more specific prediction intervals. Furthermore, it is found that the beta-values used strongly affect the conformal regressor. While there is no specific beta-value that will always minimize the interval sizes, lower beta-values lead to more variation in the interval sizes, i.e., more specific models. In addition, the analysis also identifies that the normalization procedure introduces a small but unfortunate bias in the models. More specifically, normalization using low beta-values means that smaller intervals are more likely to be erroneous, while the opposite is true for higher beta-values.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE) , 2021.
Keywords [en]
Conformal prediction, Predictive regression, Random forest, Gradient boosting
National Category
Probability Theory and Statistics
Identifiers
URN: urn:nbn:se:kth:diva-316034DOI: 10.1109/SSCI50451.2021.9659853ISI: 000824464300034Scopus ID: 2-s2.0-85125760858OAI: oai:DiVA.org:kth-316034DiVA, id: diva2:1686278
Conference
2021 IEEE Symposium Series on Computational Intelligence, SSCI 2021, Orlando, 5 December 2021 through 7 December 2021
Note

QC 20220809

Part of proceedings: ISBN 978-172819048-8

Available from: 2022-08-09 Created: 2022-08-09 Last updated: 2022-08-09Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Boström, Henrik

Search in DiVA

By author/editor
Boström, Henrik
By organisation
Software and Computer systems, SCS
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 54 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf