Active Learning for Decision-Making from Imbalanced Observational Data
Loading...
Access rights
openAccess
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
10
Series
36th International Conference on Machine Learning, ICML 2019, pp. 10578-10587, Proceedings of Machine Learning Research ; Volume 97
Abstract
Machine learning can help personalized decision support by learning models to predict individual treatment effects (ITE). This work studies the reliability of prediction-based decision-making in a task of deciding which action a to take for a target unit after observing its covariates x~ and predicted outcomes p^(y~∣x~,a). An example case is personalized medicine and the decision of which treatment to give to a patient. A common problem when learning these models from observational data is imbalance, that is, difference in treated/control covariate distributions, which is known to increase the upper bound of the expected ITE estimation error. We propose to assess the decision-making reliability by estimating the ITE model’s Type S error rate, which is the probability of the model inferring the sign of the treatment effect wrong. Furthermore, we use the estimated reliability as a criterion for active learning, in order to collect new (possibly expensive) observations, instead of making a forced choice based on unreliable predictions. We demonstrate the effectiveness of this decision-making aware active learning in two decision-making tasks: in simulated data with binary outcomes and in a medical dataset with synthetic and continuous treatment outcomes.Description
Keywords
Other note
Citation
Sundin, I, Schulam, P, Siivola, E, Vehtari, A, Saria, S & Kaski, S 2019, Active Learning for Decision-Making from Imbalanced Observational Data. in 36th International Conference on Machine Learning, ICML 2019. Proceedings of Machine Learning Research, vol. 97, JMLR, pp. 10578-10587, International Conference on Machine Learning, Long Beach, California, United States, 09/06/2019. < http://proceedings.mlr.press/v97/sundin19a.html >