Perhaps the most common application for experiments in data mining, legitimate controlled experiments much like the ones that scientists use, is direct marketing. Direct marketing involves contacting individual people. When you get a text or an email from a retailer, that’s direct marketing.
Traditional mail order catalogs, phone calls from charities, and campaign letters from political candidates are all forms of direct marketing. Successful direct marketers are aggressive experimenters. They may call their experiments A/B tests, split tests, or just plain tests, but these are simply industry terminology for controlled experiments.
Here’s a simple and common example of a direct marketing test:
An online retailer sends emails to follow up with customers who have viewed a specific product but haven’t purchased it after 24 hours. Would changes to the email message improve the response? Perhaps a different subject line would lead more customers to open the message.
This theory can be tested by taking a sample of customers, separating the sample into two groups that are as similar as possible, and sending one group the message that’s already in use, while the other group gets a test message that is identical, except for the subject line.
Analysis of the response to each message reveals whether any difference existed in the performance of the two subject lines, and if so, which worked better and by how much.