clock menu more-arrow no yes mobile

Filed under:

The Stunning Weakness of Predictive Analytics and Business Intelligence

Since this article that mentions the "predictive analytics" used by Target Stores appeared in the NY Times Magazine, people have periodically asked me how well all of this stuff works. They're usually keying off this anecdote:

"Andrew Pole was hired by Target to use the same kinds of insights into consumers’ habits to expand Target’s sales. His assignment was to analyze all the cue-routine-reward loops among shoppers and help the company figure out how to exploit them...Pole’s most important assignment was to identify those unique moments in consumers’ lives when their shopping habits become particularly flexible and the right advertisement or coupon would cause them to begin spending in new ways...

Pole applied his program to every regular female shopper in Target’s national database and soon had a list of tens of thousands of women who were most likely pregnant. If they could entice those women or their husbands to visit Target and buy baby-related products, the company’s cue-routine-reward calculators could kick in and start pushing them to buy groceries, bathing suits, toys and clothing, as well. When Pole shared his list with the marketers, he said, they were ecstatic."

That sounds impressive, but in a society that worries about privacy only on very rare occasions, it unfortunately turned out to be somewhat of a public relations nightmare. So we never got to find out how much extra money they convinced shoppers to spend by tweaking coupons. I think we can use a piece in this weekend's Times magazine to hazard a guess:

"For "couponers," as they call themselves, free product is the holy grail. Freebies are obtained by...receiving, post-purchase, a "catalina" — a coupon from a company called Catalina Marketing that can be redeemed on a future transaction. These little papers, which are spit out by a mini-printer that sits near the register and look like run-of-the-mill receipts, usually meet an unceremonious end in the graveyards of shoppers’ pockets and purses, but couponers regard them as cash."

This should be a slam dunk for predictive analytics - lots of shoppers come into the store and use a loyalty card (or the same credit card) to record every purchase. When they check out, they are presumably part of an experiment that is the core of web analytics: 50% of people get one coupon; 50% get nothing. Then the store tracks their purchasing habits going forward; at some point, you can be confident that the experiment increased your revenues or your profits. Repeat ad infinitum.

I don't think you need to tear your eyes out putting together time series for carrot purchases either. Maybe, as the Target article suggests, if a woman buys cocoa-butter and a bright blue rug, she's a prime target for buying a $1000 swingset at 10% off in a couple of months. But it also suggests that retailers are spending too much time trying to appeal to the 2% of women who are pregnant at any given time in addition to the loyal coupon-lovers profiled in the article who purchased 162 boxes of cereal. It's not clear that well-practiced predictive analytics have resulted in broader coupon use or increased revenues. Indeed, Netflix - which claimed to have an entirely data-driven business model - destroyed its market position with a decision that was completely unrelated to its supposedly fine-tuned analytics.

And I think it's a similar non-analytics decision that makes pockets and purses into "graveyards" for coupons. BI-driven coupons would be much more likely to get used if shoppers got them when they walked in the door, not when they were leaving. All of the predictive analytics in the world can't make up for a poorly-designed system that's not easy to use.