Predictive analytics should not be part of your initial BI system deployment. When you have finished building the BI system workflows and data warehouse, you have established the foundations for predictive analytics. Predictive analytics uses machine learning and artificial intelligence (ML/AI) applied to historical data to develop predictive models.
Investopedia’s ML/AI definition:
Using statistics and modeling techniques to predict future outcomes and performance.
Modeling the future using inaccurate or incomplete data isn’t helpful. Building the disciplines and systems for a BI system are prerequisites for developing sound predictions.
When I first speak with prospective clients, I ask why they want to deploy a BI system. Their answer includes one or both of the following areas:
- Real-time analytics
- Predictive analytics
My next question is, “What exactly do you mean by ‘real-time’ and ‘predictive’? Can you provide examples of what you mean, exactly?”
The Answers
Typically, the client confuses “timely” with “real-time.” Do you need that report updated every five minutes, every hour or every day?
A typical answer is, “Oh, once every day is plenty. I’m getting only weekly reports without analysis now.”
Okay, that person doesn’t need “real-time.” Real-time costs a lot more to implement than daily. They probably need daily.
Suppose the client is focusing on “predictive” analytics. In that case, I ask them, “Do you know what happened yesterday, last week, last month, last year, in sufficient detail to make informed decisions?” They usually don’t receive these details until long after the fact.
They say, “Our reporting is slow and when it comes, it is incorrect.”
Reality check.
Building predictive analytics with inaccurate historical data can only lead to untrustworthy predictions. At that moment, they get the point. To do predictive analytics well you must have accurate historical data.
Don’t confuse the word “analytics” with “analysis.” In this context, predictive analytics is shorthand for the data, statistics and programming needed to build prediction models. Machine learning and Artificial intelligence (ML/AI) are shorthand for training computers to make predictions. What is essential for you to understand is that you must start with accurate and complete historical data to build reliable predictions.
Build the Historical Descriptive System First
Predictive analytics is the sexy data meme of today. There is no question that predictive analytics is a valuable tool. However, building predictive models on inaccurate historical data is like building a skyscraper on sand without foundation.
Other Ways Predictive Analytics Fail
You must hire knowledgeable data scientists who understand the required computer science, math and statistics. Machine learning and artificial intelligence skills have yet to be widely distributed. Software that lowers the bar for this technical work is coming to market. Still, what is available may not fit your needs.
Attempting predictive analytics without dedicated resources (or contract experts) may not produce much value. To do it reliably, you must invest in people that know how to do the work.
The Data Storage Market Is Converging
Historically, the vendors and consulting companies that do predictive analytics have been different from the companies that sell databases for warehouses and business analytics toolsets. The market dynamics are changing, and cloud data warehouses are now taking on attributes of both historical and predictive workflows. Competition is causing convergence. Several cloud databases can house your historical, descriptive, prescriptive and predictive data.
Best Way to Proceed
Building competency in historical and descriptive analytics is a necessary precursor to reliable (and understood) predictive analytics.
When you move into the predictive domain, you must add data science and statistical modeling skills to your team (or contract that skill). The data science degree has become one of the most sought-after in technology. Demand is high. Salaries have escalated.
Interestingly, companies are now selling pre-packaged predictive models for everyday situations they are applied. This developing market may address your predictive modeling needs. Your team should be able to advance their skills, refining these models to your specific needs.
Lay the foundations by deploying a solid business intelligence system first, then do the more advanced predictive modeling after your BI system has matured for one to two years.
In the final post of this series, I’ll discuss how to create a data culture.
