Re-Examining Predictive Model Evaluation: CRISP Approach

Blog Post
April 16, 2020

Some years ago, I was challenged to arrive at a set of rules that would provide both the analyst and the marketer guidance on how to evaluate the results of a predictive modeling exercise. How come, you might ask? Just look at a standard textbook and a whole host of criteria are readily available. These provide value to a more quantitative oriented manager—but to the novice marketer, these evaluation tools can be intimidating. After all, a ROC curve, a Kolmogorov Smirnov test or a Root Mean Squared Error for example, do not give the layman a very comfortable feeling. Yes, they can certainly be explainedbut wouldn’t it be more attractive if we could describe a set of rules that are reasonably intuitive to us all, that are easier to digest and comprehend? Make no mistake, these rules are not new. Indeed, they can all be inferred from existing guidelines. It’s all in how they’re presented.

I, as well as others, have written about these evaluation standards before, but with predictive analytics growing at unprecedented rates, I thought it worthwhile to review some of the more layman-friendly tools.

So lets dive in: