Business Information (BI) technology is a collection of hardware, protocols, toolsets and methods that is ever-changing. A common refrain has been “Buy IBM, or Microsoft, or Oracle, or SAP” because these vendors are perceived as safe investments—ways you couldn’t go wrong.
The market has evolved over the last decade. Now, one of the worst decisions you can make as a technology buyer is to invest exclusively in a single-stack vendor. This decision locks you into a vendor’s ecosystem, which is developer-oriented, heavy, expensive and, in most cases, not state of the art. These are designed to keep you in the vendor’s product stack. Most of the best innovation in BI over the past 10 years has come from small vendors. I’ll confine my comments specifically to the BI stack, which encompasses:
- ETL tools
- Databases
- Data visualization
Every major stack vendor has been buying up small innovators to modernize and expand their offerings. This “buy your way to greatness” model has many pitfalls. The most typical pathway after these products are acquired by incumbents? They get ruined inside the stack.
The best ideas today come from new vendors. These “best of breed” solutions usually have a lower total cost of ownership. Better for less equals a win for you.
The API World
It has never been easier to connect disparate sources. APIs have proliferated. Tools like Tableau have many native connectors. In the 1980s and ‘90s, the single-vendor approach was helpful because getting data from different places into your “single version of the truth” data warehouse was time-consuming, difficult work.
Now with the maturing SaaS ecosystem and databases like Snowflake, cloud-based API toolsets that you find with Fivetran and ETL tools like Matillion, the single-vendor solution is no longer optimal; in fact, the single-vendor stack is typically the most expensive way to achieve a suboptimal solution.
Various Kinds of Awful
During the last month, I’ve seen three particularly bad solutions that failed to deliver value for a business. I’m not going to identify the companies or call out any specific consulting firms that are simply billing their clients for solutions that aren’t best for them. In one case, the client was billed over $700,000 and received no tangible benefit. Some of the managers involved didn’t survive the project failure. They lost their jobs.
To avoid getting yourself into this constellation of problems, carefully evaluate the entire ecosystem of products and services. Identify and avoid these traps:
The Single Cloud Vendor Solution
In this example, the vendor touts a “free” solution that is far from free. In fact, they are more often than not selling a retrograde solution that requires purchasing many different products and uses old fashioned “cube” technology to make it work. Data cubes were cool in 1990 when hardware and software weren’t good enough to perform at scale.
My shorthand way of describing data cubes is this: “Achieve high performance by eliminating data, reducing options and spending more time than necessary on adapting to changing business conditions and needs.”
That’s it in a nutshell. Cubes are an old fashioned solution set to the pace and cadence of business and technology 25 years ago. Cubes are inherently retrograde.
Most people have a difficult time figuring out what they need to buy. If the consulting company or stack vendor can’t explain what you need to buy and why in one or two pages, run for the hills. Your likelihood of being unhappy with the outcome is high. Make the vendor or consultant explain it to you in a way that you understand before you buy anything.
The “It’s Free” Solution
You really do get what you pay for. If it’s free, it has (at least) one of three flaws:
- You have to be a coder.
- You have to rely on outsiders.
- It is an incomplete solution.
When you evaluate a solution, ask the vendor or consulting firm what will be required for the solution to work for all of your users (inside and outside of your company), all of the time, for all of your needs. Will the data be secure? How easy is it to learn? What training options are available? How long does it take to develop the necessary skills? How many use cases are “active” in the real world? Can you get an SLA that will provide the necessary guarantees for uptime? What happens if a network outage occurs? How is security and governance achieved, monitored and managed? Ask for references.
Don’t Fall Into This Trap
Failure is part of learning. The key is not to burn down the house with your failures. You want to fail early, often and small. If you realize that you’ve made a mistake with your vendor/consulting solution, just admit it. Document what you learned. Don’t throw good money after bad by trying to make “the pig fly.” Just get on with doing the better solution. Learn from your mistakes. Adapt.
The least costly way out is always to move to the better toolsets as quickly as possible. Better today means picking the best of breed. This is technology. Improvements happen. The world is changing rapidly.
What Does Best of Breed Look Like Today?
The cloud has matured. We have three big vendors competing (Amazon, Microsoft, Google), and all of them provide excellent services. You want toolsets that work with all three platforms, like these below:
- Fivetran: Codeless data pipeline
- Matillion: Cloud ETL
- Snowflake: Scalable cloud data warehouse
- Tableau: Data visualization
Fivetran has connectors for 146 different applications, events, files, logs and data warehouses. They did the mapping for you.
Matillion gives you a cloud-native data transformation pipeline toolset that works wonderfully with Snowflake.
Snowflake provides a modern cloud database that separates storage, compute and management in a way that is affordable, infinitely scalable and easy to test and deploy. It works well on Amazon and Azure today, and by the end of this year, it will be working on Google Cloud.
Tableau gives you the most powerful, flexible and fun-to-use data visualization toolset that has become the standard for how people see and understand data. The combination of Tableau and Snowflake enables the fastest pathway for turning your data into actionable information.
If you have or want to use a web portal for distributing interactive dashboards securely, check out our content management system for Tableau Server.
The Best Thing
The best thing about these cloud-based tools is that any company of any size can use them affordably. Imagine being a small company with 20 employees deploying your first data warehouse and dashboards. It’s affordable and will grow infinitely with your company.
This same toolset also works for very large companies. Cloud solutions were viewed with suspicion five years ago. Nobody wants to be on the news for a data breach. Today, the cloud platforms offer good data governance and security, and breaches still occur. You need to have cloud BI solutions that add extra layers of security and governance, in addition to what the cloud service provides. Security is built into Snowflake at the row level.
There has never been a better time to build and deploy an analytical business information solution that integrates data from many different sources.
If you want to know more about these tools and the market, contact InterWorks. We can help.
