The plans, quotes and project deliverables your selected consulting partner helped you develop should include a proof-of-concept project recommendation based on your combined due diligence.
The proof-of-concept (POC) sprint is a crucial subproject that tests your platform and software selections. The first dashboards your end users can use provide an early litmus test of your discovery sessions and planning. It lets your deployment team gain hands-on experience and knowledge of the tools.
Suppose your initial selections don’t produce the expected outcome. In that case, you can still change your mind before making long-term vendor commitments. The proof-of-concept sprint will validate your planning and provide enough work to hone your project management tactics.
BI system deployment projects are significant endeavors comprised of smaller project sprints. Typical BI deployment project duration is 12 to 18 months, including 6 to 10 project sprints. Each sprint will result in a finished deliverable. Attributes of the ideal POC sprint:
- Validate the software tools and cloud platform you selected.
- Elicit senior leadership agreement with the expected deliverables.
- Include the entire data workflow; Extraction, Loading, Transformation
- Provide at least one working interactive dashboard.
- Obtain end-user feedback and validation of the outputs.
- Limit duration to four-eight weeks.
A successful proof-of-concept sprint is a crucial step forward in the process. It validates your choices and builds excitement within your company. Upon completion of the POC sprint, you should have confidence in the software, your expert consultants and your team’s ability to continue to deliver the entire list of needs your operational leaders identified. If things don’t go as expected, you can correct the problem before it becomes expensive.
Finishing the POC sprint will energize your team. Achieving concrete results, growing confidence using the tools, and increasing certainty that the BI system you are building will deliver the desired outcome. They should also have at least an intermediate-level understanding of the software. Your team will be capable of making sound recommendations about the deliverable workflows for subsequent sprints.
They may feel they can take on more of the technology work your consulting partner delivered during the POC sprint. Or, they may recommend you continue to contract the technical resources to keep the project moving forward briskly. Your consulting partner and team can help you with this decision. You will begin to appreciate the need for maintenance, security and data governance, which we will cover later in this series.
Selecting the Data Workflow for the POC Sprint
An ideal data workflow to use for your POC sprint should include these five attributes:
- Just enough complexity.
- Easily accessible data sources.
- ELT workflow performance verification.
- Database and cloud platform efficacy.
- Interactive dashboards with obvious value.
Most importantly, management will be unimpressed if the POC produces a database. Most people need help understanding SQL or database schema (architecture). Delivering interactive dashboards is the most crucial part of your POC sprint. Dashboards are how your end users can see and understand the data. Dashboards convert data into actionable information. Point-and-click interaction enables inquiry.
Just Enough Complexity
Your POC sprint must be complicated enough to test your cloud platform and software selections.
Things should be as simple as possible, but no simpler.
-Albert Einstein
It should include one of your primary data sources; this raw data typically comes from your Enterprise Resource Planning (ERP) system. The goal is to gain practical knowledge of your selected ELT and database software. Enough complexity means you should have one to two years of history. The data must be the most detailed available for numbers, dates and other dimensions (dates, geographies, products, services, customers, orders, etc.).
Express the field names in the database using language everyone within your company understands. The database design (schema) should comply with design best practices. But allow for simpler designs that match end-user requirements and skill levels.
Store the most detailed data available in your data warehouse. Doing this enables your end users to investigate outliers and answer questions because the database will contain all the details. The dashboards you deliver should provide obvious value. More on that later.
Your team must complete the POC sprint in four to eight weeks. Including enough data and complexity will provide a good stress test of the software you selected and provide confidence that you made the right choices. The more time you take, the higher the cost if the test doesn’t pan out as expected—balance complexity with data condition and complexity to minimize the cost of negative results. This balancing act is what I mean by just enough complexity.
Easily Accessible Data
It is best to avoid data sources that contain many errors or data that is difficult to load and transform without monumental programming effort. Most data sources today provide more than one pathway to extracting data. Many provide pre-built application programming interfaces (APIs) or third-party developers offer them.
ELT Workflow Performance Verification
Your chosen ELT (Extract, Load and Transform) software is one of the most important tools you will test. The POC should test its ability to process your typical workloads for speed, size and variety.
No ELT solution today has “plug-and-play” capability for every possible data source. Still, your chosen tool should at least support the most popular cloud-based services. The software should accommodate custom scripting if you need to program a solution for unique challenges. It should be able to extract and load data fast enough to meet or exceed your needs.
A successful POC sprint will confirm that the cloud platform and software you have selected will process your data in an affordable and timely manner.
Database and Cloud Platform Efficacy
The POC workload should provide enough volume, variety and velocity of data to ensure that your chosen cloud-based database can manage your requirements effectively. It will provide insight into storage costs and the cost to run data queries (compute expenses) associated with your ELT workload. Another significant area to monitor is end-user dashboard compute costs. These arise from the frequency dashboards are updated and their intensity of usage. Achieving fast query speeds (high performance) is easy with cloud databases. The trick is to balance the need for speed with cost.
Interactive Dashboards with Obvious Value
Most of your company will not directly access your data warehouse to get information. They will access the data via interactive dashboards. Dashboard software has become the most popular means for providing non-technical users, analysts, managers and executives access to data contained in databases, spreadsheets and web-based services. The most important deliverable of your POC will be interactive dashboards that enable people to see, understand and interact with the data.
Your POC should provide one or more fully operational dashboards. Staff, management and executive leadership must be able to view, test and ask questions about the new environment at a level of detail appropriate to their needs.
Your consulting partner should build the POC dashboards. The consultant should have expert-level skills. Your deployment team members responsible for building dashboards should monitor this work. This expert will advance your team’s skills much more quickly than they can on their own.
The appropriate dashboard tool should enable reporting, analysis and discovery and not require significant technical skills of information consumers. It should be easy for your primary dashboard developers and analysts to learn to use at an intermediate level. Developing advanced skills requires several months.
A Successful POC Spurs Excitement
Selecting a data workflow for your POC sprint that can meet all these requirements will take time and planning to identify. Still, it will also give you confidence the selected software will provide for your current and future needs.
While this choice may sound daunting, it is easier than you think. Sales data are commonly used in POC sprints because that data interests many company levels. A successful POC sprint tests every aspect of your chosen software stack. It also validates your expert consulting partner selection. Getting this sprint done on time with a great outcome will result in visible success that produces a positive buzz about the project.
In the next post, we will outline the best way to release your first interactive dashboards to staff, managers and executives.
