Back to the Future – Data Edition

Transcript
Welcome, everyone. I'm also very glad you're here and to hear or listen to what I'm going to talk about and let you take part in my thought process. So I am an analytics lead at InterWorks. In that capacity, I am also responsible for asking quite a few more, let's say, philosophical questions. And especially when it comes to AI, there are a lot of them. So today, we are going to talk about exactly that and also the focus tools, as Vicky mentioned already, DBT ad with hotspot. But, of course, let's start with with something that is more well, something that belongs to my daily routine, My own thoughts process when it comes to AI. So quick example that might be representative or not. AI is here. Woo hoo. Cool. Will I still have a job in a few years' time? Now, seriously, will I? And then yeah. Well, probably. I won't take over everything. Right? At least not yet. Or will it? I don't know. There's so much happening around everywhere. There are so many apps for AI out there currently. I need a kind of overview for that. Is there an overview somewhere out there? Seems like a lot of work looking for one. What will we have for lunch today? And then usually the whole thought process starts in you. And this is something I've been confronted with for, well, eighteen months now, quite regularly, but I'm still trying to get ahead of the curve. So with that in mind, we have to go a little bit back. I mean, the whole thing is called back to the future. But before we go into the future, we have to go back a little bit and take a quick history lesson. But it's a, hopefully, very simple one. We start actually with a company, Microsoft, that, well, we see the stock price of Microsoft here over time. But this arrow here on the left is the point in time when this tool, Excel, hits the surface of the Earth. At the time, it looked something like that. It's actually pretty hard to find screenshots of Excel version one from the eighties, or photographs or something like that. This is the only one I could find at all. But then, of course, Excel evolved over time. There was a version number two. Then there was also Lotus one two three in between, and not just in between, also before Excel, of course. I think this, he was in two thousand and no. Not not even two thousand. Nineteen ninety five, probably, with Windows ninety five. The millennium edition time, XP, and Excel, how it looks nowadays. Also, by the way, for those of you with a very keen eye on the upper right, we have this tiny logo, let's make it a little bit bigger. That we will talk about a little later on, of course. But, yeah, this is what Excel looks like, and now you might wonder why the heck is he talking about Excel. The reason for that is a buzzword that is called democratization. Now Excel did something in the nineties that none other tool in the BI world had done before, and this has been democratized the usage of your own data. Suddenly, every person, every single person with a PC at home could analyze their own data, could analyze their own financials, could and finances could analyze anything they had data for, which is actually pretty cool. Now later on, there were more democratization, attempts of other tools, especially when it came to visual analytics. And nowadays, the democratization goes a little step further. So there are a few generations of what we call self service analytics. We have a timeline here. When this timeline exactly starts is not completely clear, but at least we had something called the first generation of self-service analytics, which was self-service for IT. Now you definitely had to go to your IT team, not even an analytics team, but your IT team if you wanted answers from your data. Of course, all the icons that we are seeing here, all those logos, this is not a complete list. There are so many hundreds more of tools around for each and every generation in here. But, yeah, the first generation sub servers for IT, data on premise, always on premise, almost always on a singular server, and you were very dependent on the IT team. The second generation was self-service for analysts, and this is where the first visual analytics tools emerged. The ETL flows became faster. It was the very first attempt for set actual self-service in the analytics world. Still dependent on IT, especially when it came to the data side of everything. But when it came to visual stuff, people got unable to do a lot of things on their own. Then with the third generation, self-service for data power users, this is where tools like Tableau, like Power BI, and even Snowflake emerged and show shine shined shown that light into the world. Now everyone who actually work with data a lot of the time and weren't act necessarily analysts could work with data, which was also pretty cool. But now, of course, we are entering the fourth generation or the current generation for self-service analytics, and this is, of course, self-service for everyone. Now tools like ThoughtSpot and other ones around in the market offer a place where each and every user, especially the business users, can work on the data to find their own answers from the data. So it has been a long way from the eighties since when it comes to data analysis even before that, early sixties, even fifties, actually, a colleague of mine recently sent me a picture of something like a calculator from two thousand before Christ, so a long time ago. For today, this begs, of course, a few questions. For example, do we all need to be machine learning engineers or AI engineers in the future? And after careful consideration, I think not. No. We don't have to be that, especially since the AI is going to take over a lot of that heavy load of work for us in the future. Do we all need to be data literacy experts? Well, on this one, I say yes. Definitely. Everyone in the world should speak the same language when it comes to data, And it's the responsibility of newer tools that we are talking about in our landscape to make that language easier instead of more complicated. Do we need a lot more dashboard developers in the world? On that one, I'm not completely sure. Actually, there's this beautiful curve here that is showing the saturation of dashboards in companies. And what we are seeing here is that this curve is actually not that steep anymore. And that means we kind of hit a peak when it comes to needed dashboards in companies. They are still needed. Don't get me wrong. We still need the dashboard, and we still need a way of giving data to everyone to analyze what is going on. But we still hit a peak there. This is not the future. And, actually, there are quite a few more numbers out there who support or which support that statement. For example, although we don't see numbers, just spaces or rectangles in a few seconds, we have those three bigger groups of people in companies who work with the data. We have the power users, which, by the way, for this example, also includes all the analysts. We have the consumers, data consumers, basically everyone who needs to see charts, who needs to see answers to their questions to make decisions, and who are doing that already. And, of course, we have the bigger chunk of business users. Now the business users are basically everyone else. Everyone else who might have questions about data but hasn't really asked questions about data yet. And, unfortunately, this really, really big group here hasn't been in the focus for decades now when it comes to BI. There's another chart that looks suspiciously similar to this one here. And this is: highly used dashboards, frequently used dashboards, and rarely used or even one off dashboards. One off dashboards meaning there was a requirement to build a dashboard to answer a specific data question. Then this was built, the question was answered, and the dashboard for which someone spent, I don't know, maybe hours, maybe days, sometimes even weeks of time, is going to be discarded and never looked at again. This is a cost. Of course, the cost of this one person building the dashboard, which is just time, but also costs in terms of opportunities. Because in this time with this developer, this analyst has built this dashboard, he could have done so many other things, answered so many other questions for the company. So this year is a challenge, and it's a challenge that a few tools around the world try to solve. But one of them is the one that I'm a really big fan of and which I'm, of course, also going to talk about today called ThoughtSpot. Now what is ThoughtSpot? Thought spot in a nutshell. Okay. My I have cool animations here, which are a little bit wonky currently. So ThoughtSpot in a nutshell. What the heck is ThoughtSpot? If you have never heard of that, it's actually a very simple tool that takes away the effort of asking questions in a super convoluted way. You might remember why Google has been so successful in the nineties and then, of course, still until today. It's because it's so simple. It's just one search bar in the center of your browser, and that's it. ThoughtSpot also has a search bar, and this is the core of the whole tool. It's one search bar. We type in a question into this search bar. For example, how many employees does InterWorkshare have? And within seconds, ThoughtSpot is going to give me an answer to that. And this is basically the gist of the whole tool. We can ask questions, and the tool is giving us the answers depending on the data we provided for ThoughtSpot. Let's say I have another question, like what is the percentage of remote employees? And I will get another answer here. Please note in this case, it's a percentage, so thoughts were adapted to the actual question in here. Maybe I want to ask what was my biggest sale last week, and I get a bar chart. In this case, it's showing me different, I don't know, products, different orders, different whatever I am looking for in this specific moment. And then I wonder, actually, this is an important chart here, something I might have to look into when it comes to the next week, and I want to then look into the current week. So I should store that somewhere. And yes, in the thoughts, but we can thin our answers into what we are calling a life board. It's not a dashboard, although there are similarities, but it's a life board, and the name comes from that this one here is completely live all the time. So when the data adjusts, when the data is updated, the whole live board updates It's as well, it's not a static one. And then I just can just continue to ask questions. For example, let's say, what were the strongest sales hours last week? And I will get a chart for this one as well that I'm also going to pin, by the way. So let's move that to the left, and my animations here are leaving me hanging a little bit Something. Oh, there we are. And I can go on and go on and ask question after question. For example, what were the market shares last week? And then I discover interesting. South here is a little bit of an outlier. Oh, well, not really, but at least it's the highest number in here. And I'm interested in, hey. What is going on there? So I can click on this one and to drill down into this one bar. I can drill down into any dimension, into any data I have in my database. In this case, I drilled into my products, and I can see immediately, performance stuff here. And I pinned that already. Performance stuff seemed to perform pretty well for that time. So this is what ThoughtSpot is doing. We can ask questions, and ThoughtSpot is giving us the answers to those questions. We can store them on a live bot, and this is the core of the whole product. Now I'm very conscious when I say core because this is, of course, although it's the core of everything, only one function of, like, a hundred, two hundred different really cool features that the whole product offers in the end. Okay. But I I'm going to put a stop here because, well, what is this actually bringing us in the end? Well, we talked about the power users, the data consumers, and the business users. ThoughtSpot is focusing on this big chunk of users, the business users, who, at least until now, didn't really have a chance to go into the analysis of the data they specifically needed in that moment. And also for all those rarely used dashboards with ThoughtSpot, those just disappear. The need for them disappears completely because all those one off dashboards can just be answered with a ThoughtSpot answer. You type in a question, you get an answer, and you're done. But however good thought for this, of course, it needs clean data. And this is where the second of our two focus tools today comes in. We have to talk a little bit about how do we get clean data that we can then provide to to tools like ThoughtSpot. I'm going to show you something that I have been confronted with I don't know how many hours in my life, and I have to apologize for the weird animation here. It was supposed to be to go line by line, but you still get the gist here. We are looking at a SQL query, and it's a long SQL query. It's It's a convoluted SQL query, and it may go on forever and forever. And I have to say from my own experience, I have written those queries quite a few times. And what I also did is I repeated subqueries all the time. I basically started always in you. Select star from x, y, zed, and so on and so on. Type in the query already used to know from something else. I rarely just copy and paste it. I mostly even wrote it out. But still, this is, of course, something that is not super efficient in the end. And also, it's really hard to hand something like this over to a colleague later on because this colleague has to go through that and actually decipher what is happening here, what is going on. So there's a tool out there called DBT that there we are. That helps us and especially all those data engineers, analytics engineers, data architects, and everyone has actually to wrangle with the data out. EBT is the acronym for, I hope I'm not wrong here, data building tool or data bill tool. So super creative acronym. I admit that. What does a d b d DBT do? Well, of course, it's part of our ELT or ETL pipeline. ELT, for those of you who have never heard that acronym before, stands for extract, load, and transform. The data has to come from somewhere. It has to be loaded into something, so we can start analyzing it. Let's say you have a survey here on the left, big load of sheets of papers. The data has to go from those sheets of papers into something where we can then work with the data to, in the end, analyze the data. Now DBT is responsible for the t in ELT. It's a transformation tool, meaning we have data points somewhere, and now I'm really curious what my animation is going to do. So we have a few messy data points around there. We can start reshaping them, adjusting the data types, for example, and, of course, in the end, have an ordered dataset. That still looks pretty neat. Now how does DBT do that, or how does it help us with that? Well, let's say you write SQL. You write a query. I am not bothering you with lines of code here. Let's just pretend that this one white rectangle is part of a query, and we append new subqueries to that. And this whole query gets bigger and bigger and a little bit more complicated. I think one or two of you have heard of the big spaghetti monster around in the SQL world or the coding world. With DBT, there's something really cool, and this is we can modularize the code that we have. So, for example, if there are two modules here that we are going to reuse again in other queries later on, we can just store them and, well, just use them again from there later on. This is a very simplified version of what is actually happening in DBT. I hope you forgive me for that. But, actually, we're just here to talk about what the tools can actually do at their heart. So, of course, there are also a few other cool things we can do with the dbt, like there's a version control for the transformation pipeline. We have data quality testing in there, dependency management. Scalability, of course, is a thing with the dbt, not just for the data, also for the people who are working with dbt. It is actually not that complicated to learn. It's very SQL based. So everyone who speaks SQL will also be able to speak dbt. And there's something called the automatic data documentation that a lot of the developers out in the world would love to have speak from experience here. And then there's more. There's also dbt cloud, which, dbt is actually quite proud of. Because with DBT cloud, we have quite a few more really prominent and impactful features around that. The whole orchestration of data projects, I set it up. DBT Cloud does the rest. This is actually a very true statement. We set it up. We configure it configure it once, and then we can leave it alone. The rest is done within dbt cloud. And there are a few more cool features announced or already coming, like the DBT Explorer, which makes it really simple to collaborate on data pipelines, on transformation pipe pipelines as dbt assist, the new semantic layer, which is actually pretty great tool, which is in the news currently. So a lot going on in there. Now I'm going to stop talking about my two favorite tools here for a second and go back to the word I used at the very beginning, democratization. DBT and also thought spot play a heavy role when it comes to democratizing the usage of data. ThoughtSpot, because it makes it available for basically everyone to ask questions of the data. EBT, because it process. It's far more transparent. It's easier to hand over. Process. It's far more transparent. It's easier to hand over, and it actually is a lot of fun to work with that. So let's go back to the philosophical stuff from before. In my opinion, because, well, I have a lot of time to actually ask those kinds of questions, so there are three different buckets when it comes to necessity for analytics and visual analytics in the future. I built this very tiny matrix here for AI and for humans because, hey, we're talking about AI here. Exploration is I need to know what is going on. Education is I need to tell someone what is going on. And decision, of course, is and I don't know what is happening. What am I going to do about it? Now humans, of course, need especially visual analytics now and also in the future. We are very visual creatures, and this is completely fine. AI, on the other hand, for exploration, AI does not need any visuals. It just needs the raw data, the pure data. AI doesn't have eyes. For education, it depends on if the AI is going to educate other AIs or humans when it educates humans, telling humans what is going on. It, of course, also needs a kind of visual expression for the data. And for the decisions, it's quite similar, actually. Those two yeses up here are the ones I am most interested in. And, of course, this begs another question that you might have heard quite a lot of times already in the especially in the last one and a half years. Do I trust in AI? But I don't want to spend too much time on that because there's a far more intriguing question for me, which is, does the AI trust me? Because we are actually praying playing a big role when it comes to giving AI the power to do all that analytics stuff and data stuff in the future. For ThoughtSpot, for example, where we ask questions, there is an AI that transforms the question we are asking into something that the data can understand. And for that, the AI need to needs to trust me that I am capable of asking the correct question there. By the way, what is trust? Also kind of intriguing. Trust is the willingness for the trustor to become vulnerable to the trustee and the presumption that the trustee will act in ways that benefit the truster. And with that, I am going to, well, leave you with a quote and also give you a few more tiny tips for the future. My favorite quote in the world is the best way to predict the future is to design it. I actually stole that from the x files twenty years ago, twenty five years ago. It's still it's still the one code that's sticked with me for decades, and I think this is saying something. The best way to predict the future is to design it. With that in mind, for the future, especially for the next half year, the next year, help yourself. Stay curious. Involve yourself in discussions about AI solutions. Go to conference conventions. Join user groups. There are a lot of them out there. Ask for help. Don't shy away to ask questions, and also ask the stupid questions too, please. Yes. I've heard a lot of times there are no stupid questions in the world. There are just questions I disagree. I have a trainer for data. There are stupid questions, but this doesn't mean they are invalid. Stupid questions are the best because when there is a stupid questions, I am sure half of the audience that is in the same room will have have had the same question as well. Criticize once you understand something. Try stuff out. Block about your experiences. Become a thought leader in your company. A thought leader, meaning be interested in that stuff, be able to talk about that stuff. And, of course, don't be afraid. Don't panic. We are all on the same boat. There are so many solutions out there currently, and there are around forty, fifty, sixty new apps arriving each and every day that are based on AI. It's impossible to currently know everything, and this is okay.

During this webinar, Sebastian Deptalla, Analytics Lead EMEA at InterWorks, explored the evolution of self-service analytics and the influence of AI on the industry. Sebastian traced the journey from Excel’s role in democratizing data to modern tools like ThoughtSpot, which allow non-technical business users to instantly get answers from data using natural language queries. The session also highlighted dbt’s power for modular, transparent data transformation, and discussed why data literacy is now essential for everyone. Key takeaways included how search-driven analytics eliminate unnecessary dashboards, how AI complements human visual exploration, and practical advice for thriving in today’s fast-changing analytics landscape.

InterWorks uses cookies to allow us to better understand how the site is used. By continuing to use this site, you consent to this policy. Review Policy OK

×

Interworks GmbH
Ratinger Straße 9
40213 Düsseldorf
Germany
Geschäftsführer: Mel Stephenson

Kontaktaufnahme: markus@interworks.eu
Telefon: +49 (0)211 5408 5301

Amtsgericht Düsseldorf HRB 79752
UstldNr: DE 313 353 072

×

Love our blog? You should see our emails. Sign up for our newsletter!