Change search
ReferencesLink to record
Permanent link

Direct link
Prediction-driven approach to model selection using feature selection and nonrandom hold-out validation
KTH, School of Architecture and the Built Environment (ABE), Transport Science, System Analysis and Economics.ORCID iD: 0000-0002-6839-8540
KTH, School of Architecture and the Built Environment (ABE), Transport Science, Transport and Location Analysis. KTH, School of Architecture and the Built Environment (ABE), Centres, Centre for Transport Studies, CTS.
KTH, School of Architecture and the Built Environment (ABE), Transport Science, Transport and Location Analysis. KTH, School of Architecture and the Built Environment (ABE), Centres, Centre for Transport Studies, CTS.ORCID iD: 0000-0001-5290-6101
(English)Manuscript (preprint) (Other academic)
Abstract [en]

In this paper, we address the forecast problem and propose a prediction drivenapproach to model selection. Furthermore, we investigate to what extent dierentprediction questions lead to dierent \best" models. Most of the studies in theeld, take an inference-driven approach to select the best model and project theresults to the future population. In contrast, we take a prediction-driven approachfor both selection criteria and validation sample. We use feature (variable) selectionand nonrandom hold-out validation to select the model with the highest predictiveperformance in an out-of-sample prediction manner. The application of interest iscar type choice modeling using the Swedish car eet data. We introduce two dier-ent types of model selection criteria: maximum likelihood which is the conventionalmethod of model selection, and root mean squared error of the prediction quantityof interest. We compare the best models obtained by dierent criterion functions.The results show that the \best" model for the purpose of prediction depends con-siderably on the prediction question of interest. Moreover, when the objective isto predict a sub-section of a population such as the total share of ethanol cars,maximizing log-likelihood is not the most accurate model selection criterion.

Keyword [en]
model selection, nonrandom hold-out validation, feature selection, out of sample prediction, car type choice models
National Category
Social Sciences
Research subject
Transport Science
Identifiers
URN: urn:nbn:se:kth:diva-180341OAI: oai:DiVA.org:kth-180341DiVA: diva2:892919
Note

QS 2016

Available from: 2016-01-11 Created: 2016-01-11 Last updated: 2016-01-15Bibliographically approved
In thesis
1. Prediction-driven approaches to discrete choicemodels with application to forecasting car typedemand
Open this publication in new window or tab >>Prediction-driven approaches to discrete choicemodels with application to forecasting car typedemand
2016 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Models that can predict consumer choices are essential technical support fordecision makers in many contexts. The focus of this thesis is to address predictionproblems in discrete choice models and to develop methods to increase the predictivepower of these models with application to car type choice. In this thesis we challengethe common practice of prediction that is using statistical inference to estimateand select the ‘best’ model and project the results to a future situation. We showthat while the inference approaches are powerful explanatory tools in validating theexisting theories, their restrictive theory-driven assumptions make them not tailormadefor predictions. We further explore how modeling considerations for inferenceand prediction are different.Different papers of this thesis present various aspects of the prediction problemand suggest approaches and solutions to each of them.In paper 1, the problem of aggregation over alternatives, and its effects on bothestimation and prediction, is discussed. The focus of paper 2 is the model selectionfor the purpose of improving the predictive power of discrete choice models. Inpaper 3, the problem of consistency when using disaggregate logit models for anaggregate prediction question is discussed, and a model combination is proposedas tool. In paper 4, an updated version of the Swedish car fleet model is appliedto assess a Bonus-Malus policy package. Finally, in the last paper, we present thereal world applications of the Swedish car fleet model where the sensitivity of logitmodels to the specification of choice set affects prediction accuracy.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2016. xv, 27 p.
Series
TRITA-TSC-PHD, 16:002
National Category
Social Sciences
Research subject
Transport Science
Identifiers
urn:nbn:se:kth:diva-180347 (URN)978-91-87353-82-6 (ISBN)
Public defence
2016-02-03, Kollegiesalen, Brinellvägen 8, KTH, Stockholm, 09:00 (English)
Opponent
Supervisors
Note

QC 20160115

Available from: 2016-01-15 Created: 2016-01-11 Last updated: 2016-01-15Bibliographically approved

Open Access in DiVA

No full text

Search in DiVA

By author/editor
Habibi, ShivaSundberg, MarcusKarlström, Anders
By organisation
System Analysis and EconomicsTransport and Location AnalysisCentre for Transport Studies, CTS
Social Sciences

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 15 hits
ReferencesLink to record
Permanent link

Direct link