Practitioners now widely use modern modeling techniques such as multivariate adaptive regression splines, support vector machines, and tree-based methods (e.g. boosting, random forests, etc.) to uncover predictive relationships between a set of predictors and a response. Support vector machines and random forests are constructed with characteristics that naturally dampen the effects of samples with unusual response values. However, other techniques like boosting, have been shown to be more affected by samples with unusual response values. This presentation will investigate the impact of unusual samples on the predictive ability of commonly used modern modeling techniques. In addition, we will explore model performance when the response is rank-transformed.