By Ryan Burton and Matt Nissen of Capital Services, and Jill Deckert of FICO
What is net lift modeling? It’s a predictive modeling technique that tells you the incremental impact of a treatment on an individual's behavior. You might have heard of this in a marketing context, used to identify who would be most likely to respond positively to an advertising campaign. It also helps in political campaigns to understand how to market to the right voter using the best messaging and channel. We have decided to apply net lift modeling to collections in order to determine how to contact the right customer using the preferred channel and message at the best time to improve collections success.
Why use net lift modeling? Our path to net lift modeling started when we wanted to learn more about our collections success than what our traditional measurements and reports told us. Reporting gave us the payment rate outcome of a collections call but we wanted to dig in further. We needed to learn if there are optimal treatments that we can apply to different types of customers that would make them more likely to pay. For example, we wanted to know what communication channel or level of aggression to use when communicating with high risk customers and which to use for the medium risk or low risk customers. The first step toward understanding this was using randomized testing to measure causality from a treatment –what resulted from a collections call and what resulted from no call. Then, net lift modeling leverages that randomized test data by targeting the segments with the highest likelihood to be positively influenced by the test. As a result, we can find out the best treatments for customer segments.
Traditional Measurement to Randomized Testing to Net Lift Analysis
You’ve heard it before and I’ll tell you again: correlation does not imply causation. This is important to remember when understanding how net lift analysis helps wean out the deliberately influenced from the coincidentally correlated. For example, traditional collections reporting gives us the payment rate over time but does not tell us if a change in treatment is influencing that payment rate. If the payment rate increases over time, it may be because of collections efforts, or an outside force like a macroeconomic change. Using experimental design to test a treatment allows us to measure the impacts of different treatments. Taking it a step further, we can split our population into segments and see how the treatment impacted individual segments instead of the population in general.
Creating interesting segments can be tedious because you have to guess which segments will add value. Net lift analysis provides a way to systematically segment the population into groups that are most likely to be positively influenced by a treatment. Once we know which segments to assign a certain treatment we can optimally prescribe the treatment and maximize our desired response. Net lift analysis gives us a deeper understanding to identify which collections treatment caused an outcome compared to traditional methods that simply show correlation between a call and an outcome. Let’s play a quick game to drive this concept home.
Question 1: Do you think ice cream prevents the flu? (hint: see graph below)
Looks like the number of flu patients consistently decreased as ice cream production increased. I guess you don’t need that flu shot after all—let’s rejoice in a bowl of mint chip!
Question 2: Do you think this tree influenced Detroit’s population? (hint: see graph below)
Looks like the population of Detroit rose and fell with the branches of the tree. Somebody grab the trimmers!
I hope you enjoyed the game as much as I did; let’s check our work. It turns out that ice cream production does not actually prevent the flu, and that a random picture of a tree does not influence the population of major US cities. While these variables correlate on the graphs, they are pointedly not the cause of the other action. In the first graph, the confounding variable causing the flu patients to decrease and the ice cream production to increase is summer time. The second graph shows that patterns can be a pure coincidence or even the result of an analyst’s creativity, not necessarily indicative of a meaningful relationship. Randomized experiments can stop you from mistaking correlation for causation, and net lift analysis allows us to extract all of the useful information of the experiment by segmenting customers and identifying which segments are likely to be influenced by the variable being tested.
Now that you know the advantages and applications, ask us more about our methods and strategy or how you can apply net lift modeling using your TRIAD or ACS data. Comment here or take your questions to the TRIAD Community.