Distribution level outages generally affect fewer customers than regional or transmission level outages. However, as global temperatures continue to rise, the radial topology and overhead lines typical at this level make it particularly vulnerable to High Impact Low Probability weather events. A Machine Learning model is therefore proposed that uses Multinomial Logistic Regression (MLR) to predict the likelihood of an outage given the weather conditions and the composition of the Distribution System Operator (DSO). The model is tuned by using a traditional binary classification problem as ground truth, but is evaluated based on its probability distributions near outage events. Results show a greater classification confidence for true outages than false outages as well as a probability distribution that is skewed towards actual outage events.