[VALIDATION STUDY] StreetLight Data’s Analytics compared to Men’s Wearhouse Data
"A Note from StreetLight: We are starting a series of blogs on the theme of "data validation." Since StreetLight measures behaviors that are generally hard to measure, it presents a challenge for validation (what's the baseline?). We are collecting cases where good baseline data is available and publishing our findings. If you have suggestions for future editions of the validation blog please get in touch! We're very glad to have a guest blogger, Dave Chipman from Men's Wearhouse, for our initial post. ~Laura Schewel, CEO of StreetLight
by Dave Chipman: Senior Director of Analytics at Men's Wearhouse
This year is an exciting one for analytics at Men’s Wearhouse. Not only are we expanding the Men’s Wearhouse fleet of stores, we’re also integrating our fleet with the Jos. A Bank brand. To keep our momentum going, we are constantly exploring and evaluating new and exciting technologies. We’ve been curious about new data resources from mobile devices for some time now. We wanted to find a way to get some of the really interesting insights about customer mobility in a way that matched our own high standards for protecting our customers’ privacy (as well as the privacy of people who aren’t yet our customers). When we heard about StreetLight Data’s InSight product, we were intrigued.
But, like many data scientists, we were also skeptical. The key question – can StreetLight’s approach accurately describe the patterns of who visits a location? Most importantly for us, can they accurately describe who visits an apparel store?
Our loyalty program, Perfect Fit® Rewards, has one of the highest use rates around – 85% of customers use it. That means that we have a very good validation data set for stores that have been up and running for a while. We decided to do a comparison to see how accurate StreetLight’s metrics really are.
We set up the comparison as follows:
- We looked at all the existing Men’s Wearhouse locations in the greater San Francisco Bay Area (81 locations).
- We divided the Bay Area into “neighborhoods” of 1 square kilometer each.
- For each store and neighborhood pair, we calculated a percentage of shoppers. For example, 3% of shoppers at Store A live in Neighborhood 32. Men’s Wearhouse used the Perfect Fit data history for customer home addresses, StreetLight used their Home Footprint metric (see image). We calculated 17,240 such scores for the region.
- Finally, we calculated the correlation between the Men’s Wearhouse Perfect Fit set, and the StreetLight Data Home Footprint. The graph is below.
As we dug into the individual results, a few other interesting trends emerged. For example, the stores with the worst fit tended to be ones in dense urban environments, like downtown San Francisco. This makes some sense, as these stores tend to have lots of folks on phones in the offices above, or just outside on the sidewalk, that could “confuse” the signal.
In short – we got a great fit.
The graph below shows the fit between Men's Wearhouse Data from the Perfect Fit Loyalty Program for existing stores, and the StreetLight InSight prediction for home footprint. The first is quite strong, leading to the conclusion that, for stores like us, StreetLight InSight can access trade areas with good accuracy.
The map show the "Home Footprint" or the home neighborhoods of people who shop at the Shops at Tanforan, a mall outside of San Francisco. As shown in the pop-up, StreetLight estimates approximately 0.4% of the mall's visitors live in the green square. This is the "StreetLight Shopper Score" for this mall/neighborhood pair.