What Will BI Look Like in 5 Years?


What Will BI Look Like in 5 Years?

I have an ongoing reel playing in my mind about the future of business intelligence that I’d like to share with you all. It’s a dream about the future that I’ve been having since I first started using Tableau in 2007. The dream looks like this:

Current World of BI

The vision is an ecosystem of tools that allow for self-building, easily-accessible and highly visual data. Initially, the components of the tool set might look like this:

  1. A data capture and storage system (Hadoop)
  2. A data visualization system (Tableau)
  3. A data modeling system (Quantrix Modeler)
  4. A fast ETL tool (Syncsort)
  5. A fast columnar analytic database (Vertica)

Excel and Access would still be part of the set. Other tools like Alteryx (more complicated blending) or Metric Insights’ Push Intelligence for Tableau (push dashboards) could be added for those that need those capabilities.

The idea is that even small entities will be generating a lot of data, and public data sources will be more commonly blended with internal (proprietary) data sources than before.

This model works right now. Data can be ingested without building schema in Hadoop. Once a domain of information is desired, schema can be created in Vertica to create a highly-performant structure for analysis.

Forecasting models can be made in a modern spreadsheet. Using very granular data from Vertica, you can model these future scenarios in Quantrix (a spreadsheet that builds formulas based on dimensions and measures, not cell-based formula logic).

If very rapid loading is required, Syncsort can be added to speed the loads to Vertica, Hadoop or both. Hadoop can also be used to archive historical data that is not used frequently. Tableau is then the visualization layer for the entire system. That kind of setup works today and is relatively economical.

The Future Will Be Easier

While this collection of tools works today, it’s still too costly and too technically demanding for smaller entities to deploy. Only small ones with the technical chops, large data volumes and a business imperative will deploy this collection of tools.

Future World of BI

What would happen if Hadoop can self-generate performant schema? This could eliminate the need for the column store or automatically render schema in the column store. The vision is this:

  1. Capture all of your data in Hadoop
  2. Decide what you want to analyze by making selections
  3. Hadoop self-generates performant schema (wherever)
  4. Tableau is used for analysis

Hadoop is still riding the crest of the hype cycle, but the Hadoop ecosystem shows a lot of promise. Hadoop running in conjunction with Hive/Spark/Shark, Impala or any of the other new inventions to make Hadoop a faster source for analysis are becoming viable options. The Hadoop ecosystem will only get better over the next five years.

The world will change when non-technical users can make selections of dimensions and measures that they wish to analyze and the (BI ecosystem) can self-generate performant schema with good data.

The work for technical folks will be to do everything possible to secure proprietary information, manage the store, provide the necessary cloud access or hardware and insure data quality – all very quickly.

It’s an exciting time to be in data. All of this will happen. I may not have all of the pieces nailed, but I’m certain we are going to get there in this decade. What happens when a one-person company and collect, analyze, visualize and understand massive data, probably stored in the cloud?

Business models will be re-written. The world will change.

More About the Author

Dan Murray

Director of Strategic Innovations
Ten Questions for ChatGPT about Tableau and Level of Detail Expressions I had some fun with ChatGPT asking it questions about cohort analysis this week. I’ll spare you the 4,000 words it created on general ...
The BI Cantos: Facilitating Data Culture Environments that encourage learning, sharing and discovery will prosper. Environments that don’t will not achieve high ...

See more from this author →

InterWorks uses cookies to allow us to better understand how the site is used. By continuing to use this site, you consent to this policy. Review Policy OK


Interworks GmbH
Ratinger Straße 9
40213 Düsseldorf
Geschäftsführer: Mel Stephenson

Kontaktaufnahme: markus@interworks.eu
Telefon: +49 (0)211 5408 5301

Amtsgericht Düsseldorf HRB 79752
UstldNr: DE 313 353 072


Love our blog? You should see our emails. Sign up for our newsletter!