How to Use Control Groups to Assign Value to Your Data Driven Marketing Campaigns
The key component of the data driven marketing experiment is your control group. You’ve held out a random sample of your target audience from your communication. You’ve executed the campaign and collected the response data. Now it’s time to put that control group to use.
How to calculate your net response rate
In controlled experiments that test medications, the placebo effect is well documented. Some patients who receive a placebo — meaning they don’t receive the actual treatment — still show signs of improvement. This placebo effect needs to be taken into account when the effectiveness of the drug is reported.
You’re in a similar situation with respect to measuring your direct-marketing campaigns. You can’t take sole credit for the responses to your campaign. Advertising campaigns and other marketing efforts have contributed to your customers’ awareness of and interest in your products. But you can take sole credit for some of it. This is what that control group is for.
Gross response rate
The first thing you need to do is calculate the overall response rate, called gross response rate, to your campaign. Suppose you sent out 100,000 communications and 6,000 consumers in your target audience actually responded. They bought something from you. Your gross response rate is the ratio of these two numbers — in this case, 6 percent. The formula is this:
Gross response rate = 100 × (total responders) / (target audience size)
You need to be careful about distinguishing a responder from a response. A particular household might make two different purchases at two different times. That counts as one responder. But when you get around to attaching revenue to this responder, you can count both purchases.
Net response rate
Once you have your gross response rate in hand, you need to look in on your control group. What were these folks doing while your campaign was in market?
To continue the example, let’s suppose that your geek had recommended that your control group contain 5,000 households. Of these 5,000 households, 250 made a purchase that would have qualified as a response to your campaign. That means the control group response rate is 5 percent. Your control group response rate is calculated the same way as your gross response rate:
Control group response rate = 100 × (control group responders) / (control group size)
Now comes the key to your being able to take credit where credit is due. The difference between the gross response rate and the control group response rate is all you. This difference is known as the net response rate — or more commonly, the lift — associated with your campaign:
Lift = (gross response rate) – (control group response rate)
This lift represents the proportion of the responses that can be attributed specifically to your communication. The only difference between the target audience and the control group was whether or not they received your message.
Once you calculate the lift, you need to check with your geek again regarding its significance. Essentially, you need to make sure that this lift is large enough to be meaningful given the size of your control group.
In this example, you had a 6 percent gross response rate and a 5 percent control group response. This gives you a lift of 1 percent, or 1,000 responses that you can take sole credit for. A check with your geek will confirm that this lift is significant — well above the 95% confidence level, given the 5,000 members in your control group.
Net revenue and return on investment
Now that you’ve calculated your lift and hopefully verified that it’s significant, it’s time to calculate your contribution. (If you don’t have significant results, then you can’t justify claiming any contribution.) This calculation depends on two additional numbers. You need your total campaign cost and the total revenue that’s tied to your campaign.
Suppose your campaign cost was $350 per thousand, or 35 cents per piece. This is actually pretty typical of a large postcard campaign. In this case, your total campaign cost would be $35,000:
Total campaign cost = (cost per piece) × (target audience size)
Your total revenue is simply the sum of all the purchases made by your responders. Remember that it’s okay to count multiple purchases made by the same responder as long as those purchases fit your definition of a response.
Let’s say that your 6,000 responders purchased on average $100 worth of merchandise. That means your total revenue came out to $600,000. This is usually referred to as the gross revenue associated with the campaign. Your net revenue is simply gross revenue minus campaign cost.
Now you’re ready to claim credit for your share of that revenue. That share, or the incremental revenue associated with your campaign, is calculated by applying your lift percentage to net revenue:
Incremental revenue = lift × [ (gross revenue) – (campaign cost) ]
The net revenue turns out to be $565,000 ($600K minus $35K). Because your lift is 1%, you can take credit for a $56,500 contribution — a pretty good return on a $35,000 investment. And in fact, campaign results are often presented exactly that way. Your return on investment or ROI is calculated as follows:
ROI = 100 × [(incremental revenue) – (campaign cost)] / (campaign cost)
In the example, this turns out to be a little over 61 percent. This example isn’t particularly outlandish. Database marketing can be an extremely efficient way to spend marketing dollars.