With the price transparency wrought by the Internet, retailers are finding it harder to compete. But thanks to big data analytics and machine learning, the smartest stores are gaining better margins by making small changes in their operations, increasingly by way of prescriptive analytics.
Since the dawn of the free market, merchants have fretted about setting prices. The invisible hand of the market naturally works to balance prices with supply and demand, but the mechanism works slowly, enabling merchants to hide true market prices from consumers. Now, thanks to the Internet, consumers have a lot more information available in real time, which makes merchants’ jobs a lot tougher.
“How do we price in this world of price transparency?” asks Zach Cross, the president of Atlanta-based strategy consulting firm Revenue Analytics, which serves companies in the retail, travel, automotive, manufacturing, energy, and media industries. “Do we have to match Amazon, or is there still a premium with our brand? You just can’t hide high prices and hope that you’re going to find someone to pay it.”
There’s no simple answer to the pricing conundrum, which is why retailers are increasingly turning to data science and advanced analytic techniques to give them an edge in an industry with razor-thin profit margins. Companies like Revenue Analytics are finding that most retailers already have the data they need to make better pricing and allotment decisions—they just need somebody to bring it all together for them.
Revenue Analytics employs a team of statistical experts who are well-versed in building market response models that use sophisticated machine learning algorithms to simulate the real world. The models combine data about a variety of factors–such as transaction history, inventory mix, individual products, customer trends, geographic location, time of the year, media exposure, and others—to generate specific predictions about what sort of demand the retailer can expect.
“We’re specialists at operating on very large volumes of historic transaction data and customer loyalty data, as opposed to more traditional market science approach that would be to conduct a controlled experiment in the lab or maybe marketplace,” says Jon Higbie, chief science officer for Revenue Analytics. “We find that by baking that into this selection criteria, we can get huge volume of observations that approximate a randomized price experiment.”
Retailers have been hiring statisticians to forecast demand for decades. But thanks to the level of sophistication we’re beginning to see in software and the availability of huge amounts of data, retailers are increasingly looking to build systems that automate the analytical processes, and speed up the cycle of iteration. “It’s something that’s done by many companies on a nightly or continuous basis,” Higbie says.
Companies implementing revenue optimization strategies can benefit substantially. “A lot of our clients are getting a 10x ROI, some as high as 100x, just by investing in these types of predictive analytics capabilities,” Cross says. “There’s a huge financial benefit.”
Thanks to analytics, retailers are getting closer to the goal of achieving the ideal assortment. “Historically a big national retailer might have 20 different zones…but 2,000 locations across the country,” Cross says. “That’s not nearly enough of difference between store to store to provide the ideal assortment. So going into location-based assortment planning strategy–that’s where we’ve seen a lot of the more recent breakthroughs with respect to data and analytics. You can get at a much more granular level on any of these decisions.”
Another firm at the cutting edge of prescriptive analytics in the retail and consumer processed goods (CPG) sector is Profitect. Founded by Guy Yehiav, the company specializes in helping retailers make better decisions around the products that they sell. In that respect, it’s a lot like Demantra, Yehiav’s previous company Demantra, which Oracle bought in 2006 for a reported $41 million (although Oracle has never disclosed what it paid).
But where Profitect departs from the Demantra model is that, in addition to using advanced analytics to create forecasts that can help retailers fine-tune their decision making, Profitect also generates plain-language instructions that ordinary retail managers and workers can understand.
“Demantra was a great ride with a great solution and great customer success,” Yehiav tells Datanami. “But the issue with all the smart predictive analytics systems was, with all that complexity of the box, we relied a lot on the talent of the users.” And when that talented analyst left for greener pastures—as they invariably do—it tended to hurt the optimization system, until new analysts could get the necessary training.
Instead of generating a complicated forecast and expecting an individual or regional store manager to make sense of it, Profitect breaks the forecast down into specific actions that the managers should take, such as raising prices, lowering prices, or expediting shipment for out-of-stock situations.
“Companies are drowning in reports. You have on-shelf availability reports, forecasting reports, financial reports. Tons of reports,” Yehiav says. “Instead of sending a merchant reports, we just tell them what we’d like them to do. Obviously you’ll need the facts. It’s not that there are no reports in this new paradigm, and the facts are attached [in a report]. But with the report comes a pure English-, Spanish-, French-, or other language explanation of the insight.”
Before Profitect recommends what customers should do, it has to identify patterns from the data. The company has invested in developing a sophisticated ETL process that looks for raw data in its clients’ systems. This is a key aspect of the system, Yehiav says.
“It’s critical that it’s not normalized because what you find out is that is through normalizing and data warehouse-building, a lot of the executives are missing anomalies,” he says. “We’re actually banking on those anomalies to explain the behavior.”
Because Profitect is gathering and analyzing data from so many systems—point of sale systems, AS/400s, and ERP systems, to name a few—it’s able to deliver a view of the store’s competitive position that the company may not have otherwise. One of those involves forecasting hidden demand.
“A lot of retailers have 60 to 70 percent, up to 90 percent inventory accuracy, but it’s not at 100 percent, even when they have RFID,” Yehiav explains. “We’re able to calculate the probability of them not having an item on hand, even if system says they do.”
Predictive analytics are well established in the retail realm, and are being used for everything from product recommendations and segmentation to fraud detection and demand forecasting. But to maintain small profit margins, some retailers are starting to make the next step in the journey, which is the move to prescriptive analytics.