How 2020 Impacted 2021’s Predictive Modeling
Before the pandemic hit, digital disruption was the most important power shaping the route of enterprise fashions and full industries. As 2020 demonstrated, digital disruption appears comparatively tame now in comparison with COVID-19’s impacts on enterprise. The sudden and dramatic shifts in on a regular basis realities negatively impacted predictive mannequin accuracy as a result of they have been so inconsistent with historic knowledge.
“One of the really big things that people grappled with is the fact that they took for granted that the models were built properly,” mentioned Scott Zoldi, chief analytics officer at decisioning platform supplier FICO. “Obviously we were in a time of huge stress so as people were trying to understand how to pivot their business, instead of asking, ‘How can I leverage the asset I have?’ they basically said, ‘Let’s just throw the model out and build a new model’, which comes with a whole set of other issues because we essentially then have models built on nonstationary data.”
Adapting to the New Normal
Companies have historically had years of knowledge that might be used for predictive functions. However, when the whole world modifications so radically in such a short while — provide and demand, provide chain disruption, shuddered companies, keep at dwelling mandates — it is time to get artistic.
“A traditional approach would be to look at sales and understand trends. Now sales is not a good predictor so you have to look for something else,” mentioned Dan Simion, VP of AI and analytics at world consulting agency Capgemini North America.
For instance, one in every of Capgemini’s airline shoppers is utilizing future bookings to foretell enterprise journey as an alternative of gross sales as a result of the demand for enterprise journey evaporated in 2020.
While firms understood they wanted the power to adapt to alter rapidly, in addition they understood they wanted to attenuate dangers by utilizing knowledge to make selections.
“Trying to predict using traditional statistical techniques gets harder and harder because you need a lot of data points and observations,” mentioned Simion. “Before you would have one observation each week or each day and you could go down to every hour.”
The similar factor holds true for different dimensions that may be decomposed into smaller items — zip codes as an alternative of nations or areas, for instance.
“That is increasing the degrees of freedom, the number of observations within the same time frame,” mentioned Simion.
Contingency Planning Is “In”
At a enterprise technique degree, organizational leaders have been warned that they wanted to do contingency planning at a completely completely different degree than that they had earlier than. Instead of getting a plan A and a plan B, world consulting organizations have been advising shoppers to have a number of contingency plans protecting completely different eventualities resembling lock downs and provide chain disruptions. However, the identical kind of considering did not trickle all the way down to the information group in lots of organizations.
“We’re seeing a pickup in demand, especially lately,” mentioned Simion. “The question used to be, ‘What is the contingency plan?’ and now it’s ‘What are my options for shipping route if I can’t ship through traditional routes? Where should I place my containers to account for that?”
Why FICO’s Predictive Models Weathered the Pandemic Better Than Most
FICO had fewer challenges with its predictive fashions in 2020 than most different organizations. Then once more, prospects depend on its fashions to make vital enterprise selections resembling whether or not to situation credit score and at what degree.
“Prior to COVID, we were always criticized. Why do your models take so long to build when this Fintech over here can do it in the cloud [a lot faster]?” mentioned Zoldi. “We would say to the client, ‘You and I both depend on this model and therefore we have to understand it carefully and we have to build it carefully.”
Part of FICO’s secret sauce is a four-prong methodology that features:
- Robust AI, which focuses on mannequin efficiency and stability
- Explainable AI, which is about understanding relationships in a mannequin, together with what the mannequin is studying
- Ethical AI, which includes testing to make sure moral outcomes
- Efficient AI ,which captures info from the earliest phases to:
- Understand the information
- Do state of affairs testing
- Decide whether or not the behaviors that drive the mannequin make sense
- Understand what to watch
Zoldi additionally underscored the significance of a governance mannequin or mannequin improvement governance mannequin.
“If you don’t have a process written down and codified to establish that from this point forward, we’re only going to use these technologies, have these kinds of people review the model, these are the standards for what it means to build a robust and responsible model, and out of that would come things the organization would want to monitor to make sure the model is performing properly,” Zoldi mentioned.
In a forthcoming report sponsored by FICO, 90% of the CIOs, chief knowledge officers and chief AI officers surveyed mentioned they must make basic modifications and funding in how they monitor their fashions.
“I think if 90% of analytics leaders in these different firms say we have a huge amount of work to do in monitoring I think that’s probably one of the big things to look at in 2021,” mentioned Zoldi. “The other thing to focus on in 2021 is that if models are built properly and carefully, you don’t lose their predictiveness but their interpretation changes a little bit meaning you might use a different score threshold than you did before.”
Making Predictive Analytics Work in an Uncertain World
How IT Can Get Predictive Analytics Right
IoT and Predictive Analytics: What We’re Driving Toward
Why Everyone’s Data and Analytics Strategy Just Blew Up
Lisa Morgan is a contract author who covers massive knowledge and BI for InformationWeek. She has contributed articles, reviews, and different kinds of content material to numerous publications and websites starting from SD Times to the Economist Intelligent Unit. Frequent areas of protection embrace … View Full Bio