How to Improve Demand Forecasting with Machine Learning and Third-Party Data
Here we go again. Consumers are changing their behaviors quickly just as they did in 2020 when the COVID-19 pandemic hit. Two years ago, consumers confounded retailers, economists, and data scientists by panic buying toilet paper, hand cleaners, surface wipes, and a variety of products suited for at-home living such as puzzles and gardening equipment. Many retailers and consumer packaged goods (CPG) were caught unprepared, and they left money on the table.
But now, retailers and CPGs are facing eroding profit margins because consumers are throwing out their cleaning equipment and getting on with their pre-pandemic lives even though the pandemic is still here. As The Wall Street Journal reports,
The resiliency of the American consumer has been a hallmark of modern history. After events such as Hurricane Katrina in New Orleans or the attacks of 9/11, people have shown they will snap back to doing many of their favorite things, given time.
Rarely has it happened so broadly and rapidly as now, two years after a devastating global pandemic began. In the past few months, American consumer tastes have changed rapidly, again, and companies are scrambling to catch up.
The Wall Street Journal noted that some behaviors that were thought to be permanent are turning out to be temporary. For example, in 2020, Americans bought a record amount of household mainstays such as paper towels, toilet paper, flour, and spices. Sales at Clorox soared more than 20%. The company rapidly increased production of cleaning wipes, bleach, and surface cleaners. But the company is scaling back as the pandemic-era buying has subsided. Although demand is still higher than before the pandemic, Clorox is cutting costs and predicting sales will fall 1% to 4% for the fiscal year ending June 30.
So, how might CPG firms and their retailing partners keep pace with the ever-shifting nature of consumer behavior? This is a vexing question especially for demand forecasting, which relies on a precise understanding of how consumer tastes tomorrow. In a recently published Forbes column, I make the case for a solution: a combination of machine learning and third-party data.
As I noted in Forbes, third-party data such as search trends, social media chatter, and weather data can give CPG companies and retailers real-time insight into clues about what consumers will do next. For instance, with real-time weather data, CPGs and retailers can plan in a more agile fashion as changes in weather patterns happen. They can also respond to unseasonal changes in weather such as unexpected cold fronts in warm-weather areas. Consumer search behavior and social media comments provide insight into searches for products with purchase intent.
However, there's a catch: this real-time data is unstructured, meaning it's not the result of a formal survey mechanism or other tool managed by a company. No human being can keep track of all of that search activity, much less look for trends it reveals. With machine learning, CPGs and retailers can shift through it and find patterns and associations that would go undetected by manual means.
Machine learning is especially adept at finding nonlinear connections that are crucial for demand forecasting, such as the less obvious search behavior cited above, where the intent to purchase is not overt. Even an automated platform would have difficulty uncovering those nonlinear associations without machine learning. Machine learning and real-time data together can be a powerful one-two punch.
How to Learn More
And, there’s good news: Centific has been down this path many times. We’ve been helping businesses learn how to combine third-party data with machine learning to get smarter about managing demand. We discuss our lessons learned in a newly published white paper. To download a copy, click here. And contact us to learn how you can improve your business by working with us.