Snowflake AI and the Rise of the Semantic Layer

Transcript
Okay, I think it's time to get going. Thanks to everyone for joining us today. Welcome to InterWorks' August webinar, Snowflake AI and the Rise of the Semantic Layer. My name is Tim Currell, I'm a data engineer with InterWorks, and today we will be exploring an exciting, but a little intimidating new set of offerings that are available with your Snowflake instance. Now, before we get started, I'm the one in front of the webcam today, but in the weeks leading up to this moment it's definitely been a team effort with my colleagues here at Innerworks. Have special thanks to go out to Jenny Parnell and Teddy Undercoffler for coordination, promotion, technical assistance, and occasional reassurance. Also to Justin Lemon and Debbie Yu, who more or less got the ball rolling and offered their unwavering support. And finally, a transcontinental shout out to Paul Middlwyck, Solutions Director for the Asia Pacific region with InterWorks, who reviewed the material for accuracy and thoroughness and set me straight before I said something foolish. Earlier this summer, I attended the Snowflake Summit Conference in San Francisco, and between the keynote speeches, the presentations, and the labs I participated in, one thing became very clear to me, and that Snowflake is all in on AI. The company sees AI integration as an integral part of its strategy going forward, and to further that goal, its AI offerings are woven into nearly every corner of its product. The company is no longer going to be simply a cloud data warehouse. It's instead positioning Snowflake as an AI first cloud data platform. Now this move towards AI is a significant shift in the IT landscape. I expect that is why some of you have joined us today. Technology companies like Snowflake move quickly, and in industries that move more slowly like healthcare and financial services, the sudden appearance of AI tools in your cloud data warehouse platform may have been a little jarring. So whether you're a long time Snowflake user or we are you're just getting started with the platform, we hope that you'll come away from today's event with a better understanding of the direction where Snowflake is headed and the benefits available with this change. Okay, so we do have a fair bit to go over today. First, we're going to give you an overview of the Snowflake AI landscape, and that's centered around the two primary user facing tools on the platform. And next, we'll dive into the semantic layer in a general sense, make sure you have a firm grasp of the concepts, because after that, we're going to show you how Snowflake implements its semantic layer using semantic views. And we'll wrap things up by giving you a short demonstration of some of the platform's capabilities, and we'll have a quick preview of some upcoming features that you can look forward to. I will mention here also that we have Zoom's Q and A feature up and running. There's a Q and A icon on your screen, and that will bring up a pop up box, so if you have questions during the presentation, I'll invite you to enter those there. We'll be holding those until the end, and depending on time we'll try to get to at least some of those questions before we close out today. I do want to tell you though that for those questions we don't have time for, we will do our best to reach out to you directly in the coming days. So, when you take a look at the Snow site UI, the left side menu has an option for AI and ML, and quite frankly the list that's featured there can be confusing. What does each one of these things do? How does it help you? And do you have to be a data engineer to use it effectively? We're going to simplify things during our event today, and focus our attention on the two main AI offerings that Snowflake has integrated into its platform. Snowflake Cowork and Cortex Code, which is now being rebranded to simply Coco. These two tools are designed to be user facing, and they're probably the ones that you and your team will be using the most. We're not going get very technical during our discussion today, but I will tell you that under the hood of both Cowork and Cocoa is a large language model, a brain if you will, built by Anthropic, and Anthropic is the same company that offers the Claude set of AI tools, which have been increasingly visible in the AI space of late. Snowflake announced a significant partnership with Anthropic late last year, and since then, the two companies have partnered to bring Cowork and Coco to the Snowflake platform. Snowflake building the scaffolding and Anthropic providing the LLM powering both products. For most people, technical or not, the only parts of Snowflake's AI suite that you'll act with, interact with on a regular basis, will be Cowork and Cocoa. So let's go ahead and take a look at both of these. Cowork is a conversational AI agent built into the SnowSight UI that lets non technical users ask plain language questions and get responses pulled from both structured tables and unstructured documents. No knowledge of SQL is required. In its simplest form, Cowork is a chatbot, but it's only wired into your organization's data. One of the primary use cases for a tool like Cowork is to move the ad hoc data request away from the IT team and put it directly in the hands of the people who need the information. What were the top ten incident ticket categories for the last year? Which stores in the Northwest region accounted for most of the twelve percent increase in net revenue? For many organizations, those questions are answered either by submitting a ticket or emailing somebody on the right team and waiting for a response. With Cowork and the proper configuration, those answers can be obtained directly and in near real time. There are already some more advanced features built into the Cowork product as well, and the one that'll likely hold the most interest for people is called user skills. This allows you to take a routine task that's performed regularly and automate it into a scheduled workflow. So as an example, if the first thing you need when you get to your desk is the numbers from yesterday's outbound call center activity, you can have a coworker perform that query and have the data waiting for you when your machine boots up. Two more advanced capabilities that are already built in are deep research, which coordinates a multi step investigation across a company's data to answer open ended why questions with fully cited sources, and then artifacts, which can turn a chat answer into a persistent and shareable visualization, a chart, a table, a dashboard, and other team members are able to then interact with that. So before we go on, I did want to address something that's come up a couple of times, and people have asked, I'm already paying for Claude, so if Claude and co work have the same brain underneath, why don't I just point Claude at my Snowflake instance and use that? Well, answer is that you definitely can, and I want to be clear that this isn't some backdoor hack either. Snowflake and Anthropic jointly built and market an official connector for exactly this use case. So if you've heard about this from a colleague or read about it online, you were hearing about a real and sanctioned integration. It's actually not difficult to get this all configured as well. Snowflake has documentation describing a four step process that you can complete in about twenty minutes, and Claude can connect to it directly, no separate software required. Easy. And the word easy is exactly what should give you pause here, because easy is not the same thing as safe. Let's say you configured this setup connecting Claude to Snowflake. Once that connection exists, you effectively opened a new door into your data warehouse, and how that door is built makes a difference. The connection runs under whatever permissions your team assigns it, and if that role is broader than it needs to be, the AI can see and touch more than anyone intended. There's also a newer, less intuitive risk called prompt injection, where a document or a support ticket the AI reads can contain hidden instructions that hijack what it does next, and that's a risk that's already produced real, disclosed incidents in this exact product category. So, the last thing to consider here is where the data actually goes once it leaves Snowflake. Claud's LLM doesn't run on your laptop, it runs on Anthropix cloud infrastructure. So how does your data, how is it retained, and for how long? That's an important question for anyone in a regulated industry, and it's one that your compliance team is going to want an answer to. Now, of this means that the connection from Claude to Snowflake is unsafe by definition. What it does mean is that it becomes your responsibility to build the connection, secure it, and monitor it, rather than have it being pre configured, pre tested, and backed up by the organization issued it. So it's pretty clear, the question really isn't whether you can set this up, it's whether you should, and how carefully you do so, and your IT security team will be able to answer that for you. So this actually reveals one of the best selling points for Snowflake's AI tools, and that is that Coco and Cowork are already inside the Snowflake security perimeter, and they're governed by the same role based access control you use for any other data object. Cowork and Coco recognize the user that has logged in, and they make all the inquiries to the data warehouse via that user's default role. So if a user can't see private customer data with a SQL query, they won't see it with Coworker Coco either. The best part is that configuration is already done, your Snowflake administrator doesn't have to lift a finger when you add that new functionality, and as compared to the external connection scenario we just talked about, when you use any of Snowflake's AI offerings, your data stays put. The LLM powering, co work, and Coco resides inside the Snowflake security boundary as well, so your data is never sent across the open internet. I'm not just spouting marketing copy either, Snowflake's AI trust and safety document commits to that in writing. Okay, hopefully you didn't nod off during my security epistle. We're going to get back into the fun stuff now, and take a look at Coco, my new best friend. And this is an AI coding agent that operates natively inside of Snowflake. You can access it directly using the side tab that you'll see, the little blue tab with the white four pointed star, and there's also a standalone desktop application you can connect to, and if you're you can also use a command line interface if that's your preference. They have one of those available, and again, that does sit on your desktop. There are also adapters that allow connections from IDEs like Versus Code, and then there's extensions as well for standalone programs like Microsoft Excel and a whole bunch of others. CocoCAN, it's a coding agent, and so it can write and edit SQL queries, store procedures and functions, create and maintain Python notebooks and DDT packages, and can perform many other tasks on the Snowflake platform. Once you grant it permissions to access selected objects in your data warehouse, it will take those actions and conform to the role based access control you've set down by your security team. So, in my role as a data engineer, the area where I find Coco most useful is debugging. I've used it to tune queries, identify orphan key references, compare two or more entire schema against one another, and perform root cause analysis for result set anomalies that pop up during testing. If you want to know the differences between every field in every row in a table you've populated in development and compare it to its UAT counterpart, turn Coco loose and it'll perform that analysis and give you the results, and dare I say, faster than I could or any other person, I believe. I have to admit, I was skeptical at first. This old dog doesn't usually adapt well to new tricks, but Cocoa's now a valuable tool in my arsenal, and it helps speed up my development and testing process. So Cocoa has two main functions. It's the interface between the developer and the system, and it classifies and assigns tasks, then collates the output between multiple independent workflows. The process works on a plan, approve, execute loop. It interprets what you're asking for, shows you a plan, waits for permission to proceed, then calls the appropriate tool in the background to get that task completed. So, there's a quick overview of the primary user facing AI tools built into the Snowflake Snowflake platform. I'm going to note here for completeness that there are many other AI tools available as well. Those include Cortex Agents, Cortex Analyst, and Cortex Search, but we're going to leave those more advanced topics for another day. Just know that these tools help underpin the two user facing solutions that we just talked about, and they do have valuable functionality that your technical teams will probably want to investigate. So as I said at the outset, Snowflake is all in on AI. It's clear from the toolset that they make available that this is the direction the platform is going. Snowflake, Cowork, and Cocoa can be used straight out of the box, and most people are doing exactly that. But that limits the results that you can get from either tool, and it limits the usefulness of the results that you do get. There is a missing piece to this puzzle. Computers are really good at math and matching. They can compute the sum of two integers, they can assess whether the data in field A is identical to the data in field B. What they don't have is understanding. They know that there's a dollar amount in a field called edit, but they have no ability to grasp what edit is or how it's calculated. Similarly, the AI tools that sit on top of your data don't understand it either. So acting by itself, Snowflake co work can tell you that earnings before interest and taxes is up three point one percent year over year in Q2, but it can't tell you why. It's time to give your Snowflake instance the understanding it needs to answer questions that dive deeper into your data than just raw statistics, and we will do something, do that with something called the semantic layer. Now the concept of the semantic layer has existed in analytics solutions for decades, and it's recently been making inroads into the data warehouse space. It's mostly been positioned between the data layer and the reporting layer, providing organizational context for tools like Tableau and Power BI. It wasn't a necessary component in many instances, but a lot of companies correctly saw the utility in it. So why are we now starting to see the semantic layer being discussed as a must have component for your data platform? With the advent of AI integration, the semantic layer plays a more pivotal role. As we mentioned, the context provided by the semantic layer is critical to allow tools like Cowork and Coco to give the user practical, actionable responses. In a lot of cases, the more robust your semantic layer, the more benefit you'll realize from those AI implementations. Fortunately, the semantic layer will act in both capacities for an organization. The same definitions, formulas, and aggregations will serve equally well for both reporting and AI integration, and this adds another layer of benefit to the define once, use everywhere strategy that's inherent in the semantic layer concept. The information provided by the reporting layer will be as consistent and reliable as responses to queries in a tool like Cowork and vice versa. So we're going to begin exploring the semantic layer with a stock definition that you'll get online. Semantic layer is a translation layer sitting between raw complex data storage and the people who need to use that data. It provides a means for turning commonly used terms specifically to your organization into a means of extracting the data those words represent. It also standardizes definitions and calculations, so similar data requests provide consistent and reliable results. Now this kind of definition has always frustrated me, because it doesn't get to the crux of what the semantic layer actually does. To understand that concept in more concrete terms, I'm going to have you think about a restaurant. Behind the counter, we have cooks, ingredients and tools like ovens and saute pans, in the dining room there's customers trying to order a meal. Now think about how cumbersome that restaurant experience would be if the customer had to know how to make the dish they wanted when they submitted their order, what ingredients, assembled in what order, and cooked for how long at what temperature. If customers could do that effectively, then they might get exactly what they want, but that would mean a lot of research and preparation before they even got seated. Customers wouldn't know what ingredients are available, and what one customer calls chili might be light years away from the way another customer would make the same dish. Then we have the kitchen, where cooks are then looking at orders and trying to interpret them, and they hope they'd be complete, many would be, some of those orders weren't, and they would merely be guessing at what the customer wanted, more often than not getting it wrong. If an order came in that included pepper as an ingredient, that could mean ground black pepper, sweet pepper, jalapeno pepper, and they don't want to get that particular ingredient wrong. What we need is a way to standardize and provide simple language for the order process, so the request from the customer is fulfilled exactly as expected. And of course, in the restaurant analogy, they have a menu, a list of offerings the restaurant can provide at that moment in time. The customer can choose what they want, the kitchen staff knows exactly ingredients, which tools to use, how to make it, meaning they can get every order right every time. Even if that restaurant has locations all over the country, the chili you order at one location will taste the same as the chili you order five hundred miles away. Over time, the menu might change, but those changes are done in an orderly fashion, so the kitchen staff always knows what's on the menu and how to make it. In Snowflake, your semantic layer is your menu. It's a collection of offerings in easily understood language that allows the user to access the underlying data. The user does not need to know formulas, industry jargon, or the name of the specific column for a particular piece of data. All that is baked into the semantic layer programmatically. Even when the data layer changes, the semantic layer is adjusted to ensure consistent results. So when a stakeholder orders quarterly sales by region from the menu, the response is generated consistently each time. So, now that we have a handle on what the semantic layer is, we turn next to its physical manifestation inside your data warehouse, and in Snowflake that concept lives as an object called a semantic view. Now, I want to be very precise about that word object, because a semantic view isn't just a document sitting off to the side somewhere. It's not a separate PDF or wiki page describing your metrics. It lives right inside your database, in the same schema as your tables, sitting right next to the data that it describes. That matters more than it sounds like it should. We'll get to that in just a second. So, what do these things actually look like? At its core, a semantic view is built from a handful of building blocks, and we have this on the following two slides. First, as you see, it's a little small, I'm sorry about that. We have a section at the top called logical tables, which just map a business idea like menu items or customers and orders to the real tables where that data lives. Below that, you have relationships, which tell Snowflake how those tables connect so it will handle those joins quietly in the background for you. Then come the three pieces that do the real translation work. Facts, which are the raw numbers at the finest level of detail, dimensions, which are the things that you want to slice and dice by, and reach, like region, order date, that kind of thing. And then at the bottom, metrics, the actual business numbers people care about, such as total revenue or total orders. All of this gets written once in a single statement, and from that point forward, any person report or AI tool that asks for average customer rating gets the exact same answer calculated the same way every time. There has been a little bit of confusion about the term view, since most everybody who deals with data and databases understands the concept of a regular database view, which is essentially a named query that provides a curated look into an underlying table, or more than one table. It doesn't understand anything though, and it merely hands your data to you when you ask for it. It's unfortunate that the names overlap like this, but this is just something we're going to have to get used to, I'm afraid. It is also worth noting that the semantic view is a replacement for the way Snowflake used to handle semantic layer objects. This distinction might seem esoteric, but it really does streamline the creation and administration of your semantic layer. Before, your semantic layer information resided in a YAML file placed inside managed file storage, but because a file is not considered a database object, those semantic layer YAML files didn't get real permissions, didn't show up in your governance tools, and if you wanted to track changes to them like the rest of your database code, there were a half dozen hoops you had to jump through. The Snowflake semantic view fixes all of that. Because it's a native object, the Semantic View inherits Snowflake's full security model, so you can share or publish it just like any other data asset, and maybe best of all for the engineers joining us today, you can now manage it exactly like software. You can version them in Git, peer review changes before they ship, and even test them automatically, catching a bad join before it ever reaches your users. There's even a DBT package built by Snowflake specifically for semantic views, so you can define your metrics using the DBT framework you may already be using for your transformations. So now that we've seen what a semantic view looks like in the wild, the natural question is how does this actually get built? You do have a couple of choices here, and they range from fully hands on to basically automatic. There are two paths, one of them has cheat code though, we'll point that out in just a second, and most organizations end up using both, depending on who's doing the work and how much time they've got. Path number one is what we already looked at, just writing that semantic view object directly, the way you'd write any other database object, and that's the version with full control. You're deciding every table, every relationship, every metric definition yourself. This is squarely in the domain of the data engineering team. Honestly, it's what you'd expect from any well governed database object. The trade off is exactly what you'd think. It's the most precise, but it's also the slowest. You have to know the syntax and get it right, and it assumes that whoever's doing the work already has a deep understanding of the data that they're trying to interpret. I mentioned that there is a cheat code, and that is to fire up COCO, show it what schema to look at, let it do the analysis and ask you some questions, and because it knows the syntax already, it can do the authoring for you. This is how I created the example semantic view in the slides a couple of slides ago, and, I'm not gonna lie, it was a bit fussy. In my experience, when you have issues in Snowflake, the problem is almost always something to do with roles and permissions, and this was no exception. What ended up working for me was have Coco write the code, give it to you, and then execute it yourself to create the actual semantic view, so you can tuck that one away for later. Then there's the second route, which is what we're going to focus on and show in our demo today. It's a new feature in Snowflake, and it changes the math on all of this. Snowflake now offers what it calls Semantic View Autopilot, and this is as close to an easy button as you can get. You point Autopilot at your data, and it writes the first draft of the semantic view for you. It looks at how your tables are structured and how people have actually been querying that data, and it infers the relationships and the metrics on its own. There's still some work to be done after that, but it gets the process out of the starting blocks. And here's the feature that garners considerable interest. If your organization already has business logic built into visualizations like Tableau Workbook or Power BI, Autopilot can read those files and bring that logic straight into the semantic view. So you're not starting over from a blank page, you're migrating what your team already spent years building. That used what used to be a multi day modeling exercise for a data team becomes something closer to a first draft in minutes, with a human still reviewing it and refining it before it goes live. Now, it's worth pausing for just a second to consider how recent all of this is, and not that long ago, the only way to get any of this set up was a manual configuration file, handwritten and uploaded separately, with none of the structure or oversight we've been talking about. The fact that you can now generate a working draft automatically from data you already have is a genuine it's a genuine new capability, and it's not just an incremental feature update. The last thing worth discussing here, however a semantic view gets built, it isn't static once it's finished. It keeps getting refined based on how people actually use it, which is a nice segue into what happens after day one. So I'm working on a project right now where we're helping a client stand up their first cloud data warehouse, and one of the questions that has come up is, where's the finish line? When is a data warehouse done? And as you might have guessed, the answer is never. Since data is constantly flowing into it, there will be adjustments that need to be made over time. New lines of business, new marketing strategies that yield new revenue streams. The data warehouse needs updating as your organization's needs change. So just like your data warehouse evolves over time, a semantic view isn't something you build once and walk away from. You can think of it more as a living document than a finished blueprint. That's actually the point. As your business changes, and new questions come up, new data gets added, and the definitions need to keep pace with that. So the real question isn't how do you build one, it's how does it stay accurate six months later? Much of that upkeep is automatic, or at least close to it. Snowflake watches how people are actually using a semantic view, what they're asking, where the questions work, and where they come up short, and it surfaces suggestions. Now that could be to add a new metric users keep asking for, or include a related synonym that it sees frequently but doesn't recognize, or implement a filter that would clean up a recurring type of question. Now, nothing gets changed automatically. The background process does not get to make any decisions. These show up as proposals, and now it's up to the user to accept them, tweak them, or ignore them. On top of that, there's an ongoing accuracy check that's running in the background, something like a report card. It retests the semantic view against a known set of good questions every time something changes. So if an update accidentally breaks something, that will show up before your business users ever notice. So once the semantic view is in place and it begins to evolve, how does the user, the person asking the questions, participate in the process? You're actually performing that function just by using the system as it was intended. Bring up co work, ask her questions, then review the answers, and if a response is wrong, say so. That single action actually does two things. First of all, it allows you to work with the cohort tool to get the correct answer, and secondly, that incorrect answer and the remediation path you carve for your correct answer get folded back into the knowledge base. These interactions are accumulated until patterns start to emerge, and that's the point at which co work will propose changes to the semantic view to help refine the object in the direction of a complete, correct, and verified set of parameters. Now, that does raise the question, if co workers out here proposing changes and people are flagging issues, who actually has the authority to make a change stick? Let's say the sales department and the accounting department have different formulas for revenue, Who gets to be right? Well, because the semantic view carries the same access controls as any other piece of governed data in Snowflake, only specific designated roles. Typically, data engineering or data governance can actually modify the shared definition. Everyone else can use it, ask questions of it, and suggest changes, but changing what net revenue actually means and how it's calculated is limited to a discrete list of personnel, and that's by design. Because the semantic view lives in one governed place instead of being scattered across everyone's individual reports, a proposed change to it goes through the same discipline your engineering team already uses for software. It's tracked in version control, it gets reviewed by a peer, and the same approval process applies. Another very handy feature is any change can be automatically tested to make sure it doesn't quietly break something downstream before it goes live. Once all that's done, you can deploy the changes in line with your scheduled cadence or as your team sees fit. So any difference of opinion about metrics or formulas is resolved by a review process the same way that a code change is reviewed before it reaches production. Once it's deployed, the semantic view and the reports and AI tools that rely on it unify behind that single updated definition. This is an area that the company is actively investing in also. Snowflake has previewed a purpose built offering tool that bakes the review and approval workflow directly into how those definitions get built in the first place, rather than bolting it on afterwards. The direction that they're headed towards is clear, it's less tribal knowledge, more formalized, governed, and auditable processes. But I need to back up for just a second, because I skipped a step. Once we create a semantic view, all those dashboards and reports and AI tools don't just magically get connected. They've got to get wired in, so all the tools are able to see it, access it, and utilize it. So this process is going to vary depending on the tools that your organization uses, and I need to be frank with you, this implementation is very new in Snowflake, so as of right now, only a couple of the major players in the industry have plug and play connectivity with Snowflake's semantic views. Let's take a quick peek. Once a semantic view exists, the applications that you'll most likely be attaching to it fall into two broad categories: the AI tools that we've talked about already, like Cowork and Cocoa, and then your dashboards, reports, and whatever BI tool your company already uses. One of those connections takes place automatically, and the other one, not as much. So, the AI side, as you would have guessed, is the easy one. Tools like Snowflake, Cowork, and the AI framework underneath it are all built to plug into a semantic view natively. When you're setting one of these up, there's a step where you simply choose which semantic view it should use, the same way you might pick a data source from a drop down box. There's no rebuilding, no custom code, you're just pointing that AI tool at your governed set of definitions, and fortunately it already knows how to use them. Now the second path is for your existing BI, the dashboards and other visualizations and reports that you have, and as is usually the case with the point where the rubber hits the road, it's a bit of a mixed bag. I expect from where you're sitting, the question most likely is, once we get the semantic view built, will my team have to rewrite all of those dashboards and reports? I understand that concern. I wish I could tell you that there is a magic wand. Unfortunately, that is not the case. We do have some good news though. The act of deploying the semantic view by itself will not break anything that you have right now. Your existing reports will continue to work exactly as they have up until that point, meaning that they're going to still be directly querying those data sources, bypassing the semantic view. But to leverage the benefits of those semantic views and reports and dashboards and the queries that feed data into them, those are going to have to change. And so eventually, yes, rewrites to some or all of your reporting will be necessary once the semantic view is in place. But one feature in particular helps this process go much more smoothly. Snowflake recently made a change that allows a semantic view to be queried using Snowflake's variant of SQL, a language your reporting tool already speaks. So the reporting tool doesn't need some special plugin or a new integration to use a semantic view. In most cases, it's as simple as pointing the query underlying that report at the semantic view instead of the raw tables. Now, I'm going to add in a small word of advice for the BI developers that are joining us today, and that is to read the fine print with respect to syntax. Querying metrics directly requires using Snowflake's AG function in your select statements, so the semantic engine knows to calculate the metric logic on the fly rather than treating it like a static column value. So there's another one to put in your back pocket. If calculations were taking place in the reporting tool before, and you want to utilize the semantic view version of those calculations instead, there will be some modifications needed. However, that query typically gets simpler instead of more complicated, because things like the join logic and the calculation work that the query used to do is now being done inside the semantic view. So you're swapping join five tables and make the calculations myself for just ask semantic view to get me the number. In the vast majority of cases, there won't be any changes needed to the report or dashboard graphics. It'll simply be pulling from the governed single source of truth version of the data, instead of that particular BI tool's private copy of the business logic. Now, as I'm sure you guessed, not every tool has an equally smooth path to integration for this semantic view feature. Tableau, for example, has created an option that allows you to pull in a Snowflake semantic view as its own native data source, and that's pretty much as clean as it gets. Some other major BI tools, and unfortunately that does include Power BI and Excel, they don't yet have that same direct shortcut. Typically need an extra piece of software sitting in between to make the connection work. That's likely going to be a significant question to be answered, as many of you contemplate this change. I would recommend checking with your vendor to find out if the functionality is in development, or if it's on the Launchpad, or if it's possible to activate that right away. Okay, we have covered a lot of ground, but before we get to our demonstration, I realize that there are folks here from the finance side of things. Snowflake is a pay for what you use platform, so with adding on one or more semantic views, plus making tools like Cowork and Cocoa more useful, how is this going to affect your bill? Well, first, let's talk about storage, and the cost there is going to be negligible for two reasons. First, storage is cheap, and secondly, the semantic view files are not very big. You saw the size of the example one that I showed you earlier, it was less than two kilobytes. These will likely grow somewhat as your data in your organization grow, but not to the same proportions like a transaction table, like item sales or policy claims. Next, there's no separate license you buy to turn on AI and Snowflake. You have the ability to enable or disable it, but you're not paying for the privilege to do either one. Cowork and Coco use compute, the same as your notebook or ad hoc queries, and that compute is billed on the same consumption based credit system you're already using for everything else in your account. So you're not paying a flat fee whether you use it or not, you're paying for what actually gets used. However, that does cut both ways, and I'm not going to soft pedal that. Cost scales with adoption, which means if co work takes off with your team, and it does tend to, your bill reflects that success. Snowflake does have you covered here. They give you the controls to set spending limits, and the predictive model that's built into that feature means that you get alerted before the number surprises you when you get the bill. There is also a new feature that allows you to scope both the budget and the notifications to co work use only. So my advice on this topic is if you haven't familiarized yourself with Snowflake's cost management section, it's time to do that. In the left hand menu on the Snowflake UI, find the little wrench down at the bottom, and then select cost management. So, if you are thinking about, if you're convinced that this is worth doing and starting to think about rollout, I'm going to invite you to consider a measured approach. The instinct might be to do a big push to prepare to roll something like this out company wide on day one, and that strategy can easily backfire. There we go. A smarter first move is picking one team in a limited set of real recurring questions that they're tired of asking. From there, you can scale up and out, and practically that looks like this. Use the semantic view autopilot to get a working first draft done quickly, turn co work over to that one team, and let them actually use it for a few weeks before you expand access any further. While that's happening, pay attention to what people are asking, and where the answers are coming up short. Remember that feedback loop we talked about earlier. It's designed to make the system smarter the more it gets used. And before you widen access to a second or third team, go back and double check role configurations, because a pilot with three trusted users can survive a governance mistake that a company wide rollout just simply can't. So, now some of you in the room are thinking, we already turned co work on, but we didn't bring in, we didn't build out our semantic layer yet. There's nothing to worry about, you're not any worse off, co work and Coco just fall back to reading your raw tables directly and making their best guess of what things mean. It's the restaurant without a menu problem that we talked about earlier. The fix here is smaller than you'd think though. You don't need, like we said, a huge company wide data modeling initiative. The rollout can be incremental, and you can target the low hanging fruit. Point semantic view autopilot at the handful of tables your team already gets asked about the most. You'll have a working first draft that you can tweak and prune, and then hand it right back to the co work tool. If you don't get to it right away, Snowflake's already building a safety net for exactly this gap, and we're going to get to that a little bit later on. Start small, get it right, then scale, which is not a flashy answer, but it is one that's been proven to work. And so, speaking of seeing it work, let's stop talking about it. I will stop sharing my screen, and we will get to our demonstration. There we go. So as I am getting my Snowflake instance up and running here, I will begin with a few caveats. The schema, tables, data, and all the related components that I'm about to show you were synthesized for this demonstration. Everything's fake. The advantage that you have is, you're part of a big organization where you're constantly reading and writing data in and out of your cloud data warehouse. I don't have something like that to use for this demonstration, so there will be some differences between what you see here today and how things go when you attempt to do this on your system. The whole point of the semantic layer is to give the computer context for what happens in real life. Well, there is no real life for the sample database that I put together, So you're likely going to be able to leverage your activity in your Snowflake instance that your organization creates as it goes about its business, but you won't be able to see that here. And where am I? Oh, we are. Okay. All right, here we go. Okay, so this is our Snowflake sandbox, and in order to get this up and operational we're going to go to the AI and ML section, we're going to choose analyst under the Cortex menu. Now as I mentioned, most of the things that go with, go wrong with this, type of thing in Snowflake is, roles and permissions, and so you need to make sure that you have the right role set and oh, I was gonna open that in a new window, sorry. We're gonna go back on that, okay. Okay, so there are two buttons here, yeah see, I've already got my permissions wrong, here we go. I need to get this changed, one moment, otherwise this isn't going very far for very long. So we have two buttons here, one's at the bottom create semantic view, one's at the top create with autopilot, they go to exactly the same place, so you can hit either one. I mentioned before, this system will ingest a Tableau data source, it'll ingest files from Power BI, and there are other sources that they're going to be adding as they expand capabilities here. For today, I don't have any BI instances to refer to, so we're going to be using SQL queries. So, when you get to this point, it gives you two options. You can either upload a file, I'll show you that in just a moment or you can select from your query history. This is likely going to be where you end up utilizing the features here, and as I mentioned, I can't do that. I don't have a query history, Anything that we I've run probably, I don't know, maybe fifty to one hundred different queries in here, but only five are showing up for this specific schema that I'm going to demonstrate with today. It does take a little while for Snowflake to ingest everything that's going on, but again, this is a demonstration, it's going to be different for you, so we're going to back that up again and we're going to instead upload a file. I've put together a file of verified queries, and what that actually looks like in practice is this. It's essentially a question, what's the total room revenue by property for twenty twenty four, and then the query that answers that question. And those are all just stacked up inside of a single file. We've loaded it up here, you need to inform the system which one are the questions and which one are the queries, then hit next. We're going give it a name, it is a semantic view that we're creating, this is the schema that we're going to pull it from. We select our tables here, this is a full list of all, everything in the data warehouse, and we don't need that, we only need the selected ones here. And so you've got your list of the fourteen that the system has identified from the queries that you gave it. You can take a look at these and say, wait a minute, ancillary revenue, what's that all about? It'll bring you up a little preview, you can take a peek. Also, if any of these turn out not to be relevant, you can just hit the little X button and it'll take it out of the list and remove it. We are going to keep all fourteen of those though, and then move over here. I'm going to give you the advice to just hit select all at this stage, and the reason for that is that you're not building the semantic view for today, you're building it for tomorrow, and so you're not really sure what kinds of questions are going to come up in the future, and you don't want to sell yourself short by eliminating something that might turn out to be useful down the road. So, go ahead and click select all and then the fun begins, you hit create. Now with this size database that we're using, the schema that I got in testing, it ran about two minutes or so, Not too bad. I'm going to take a little break for a beverage if you'll wait with me for just a moment. Talking for a full hour straight through is not something that I do regularly. It takes a toll on my throat. So what you're going to see here is a graphical representation of what it's creating, and we will take a look at the actual file in just a moment. Here's your logical table section, and it will list these all out and that section's kind of lengthy. It's got relationships here as well, it's already got twenty one relationships that it's nailed down, and as you see over here it's starting to churn. We should be getting to the end of this pretty soon. There are some options here which I haven't explored. Smart filters, not sure exactly what that means. We'll dismiss it for now. Okay, so now that we've got a full picture the creation here, me go back and we'll jump over to our other tab. I want to go and find what we just built and show it to you. I'm going to filter this down, it should. What do you know, there it is. So this is what was just created and it mimics the format of what I showed you earlier in the on the slides. We've got the table section here, we've got the relationship section, below that we will have facts and then dimensions. I will note for the record that there's supposed to be a metric section and we don't have that yet, doesn't exist. So let's jump over here again and I'll show you what's going on with that. So this section on the right, you'll notice at the top it's called suggestions, and it's got twenty seven of them, including ten metrics. And so this is the kind of thing that's going to happen not just upfront when you're building it, but also over time. It's going to say, okay, look, I found this total room revenue, and I think that I can calculate it using this function, and I've got seventy six queries that use this exact thing, can we go ahead and call this a metric total room revenue? Sure. No problem. It's got a few of these, and I'm just gonna go through and click accept on them so that we have an example to show you. The important thing to remember that I found out the hard way yesterday was you need to hit the save button up here when you make some changes. So as you see, this is a pretty significant improvement over having to build it all out yourself by hand. And so when we close that and refresh, we've got a new version of this and bring up the definition, and when we go looking for our metrics, There's the section with those metrics that we just put in place. There it is. Okay, that's all I have for you on that. Now, as I mentioned earlier, Snowflakes laid the foundation with the release of semantic views and added some functional tools like this one that we've just gone over here for development and administration. They are just getting started, and that there are four new technologies that are currently in limited preview that could be making their debut in the coming year. So, I would like to make sure that we're all on the same page. The features I'm about to talk to you about are all coming soon. They are not currently visible in your Snowflake instance, and I'll also note for the record, that these features are so new and their implementation is so well guarded that unfortunately I don't even have screenshots for you, so I'm sorry about that. The first one up here, and I'm going to go through these quickly because we are running out of time, this is called advanced semantics. Once it goes into wide release, you'll find it as an extension of the semantic view that we're building just like we just did. It's best described as an upgrade to how sophisticated a semantic views math can get. It adds the ability to calculate a number at different level of detail than the rest of the report, so if you have a page that's showing everything totaled by month, you can add in a field calculated as, for instance, average order value per customer, and if you work in BI tools a lot, that you know that that's no small feat. There are some fairly gnarly workarounds you have to do in order to get that result with many of the tools that people are using today. Next up is Semantic Studio. This is a dedicated workspace for building and maintaining semantic views. What it does is let someone define a business metric in more approachable terms, without needing to be fluent in the underlying structure. Semantic Studio will have Snowflake's coding agent Coco built directly in, plus the version control peer review process we talked about earlier is integrated with tool itself, rather than being something that your team has to connect separately. Up next is Horizon context, and this is more of a layer working quietly underneath everything else. It's part of Snowflake's broader governance suite. What it does is continuously gather business definitions and context, both from inside Snowflake and from other connected systems, and then it feeds that information back out to the AI tools that need to use it. And finally, and I call this one the most most ambitious of the features, it's called Cortex Sense. I teased this teased you with this a little bit earlier. So if a governed semantic view already exists for whatever is being asked about, then the AI tool is going to use that to craft the response, but if nothing's been formally modeled yet, and there's no semantic view to cover a certain area of the database, then Cortex Sense acts as sort of a judgment layer. Instead of just failing or guessing, it builds its own best understanding of how the data is actually being used elsewhere. It weighs conflicting signals the way that a search engine would rank results, favoring whatever looks most relevant, or most authoritative, or most current. Okay. Before we get to our q and a, I do have a couple of things to bring to your attention. First, if you want to see more Snowflake use cases, check out Innerworks Snowflake use case library, which is full of real, anonymized engagements that we have actually worked. So it's a great way to see what Snowflake does out in the field. And secondly, kicking off, we're kicking off a webinar series in September called Sigma September. Four webinars each covering a different side of Sigma. One that might be especially relevant to this group is the webinar from Snowflake Data to Sigma AI Apps, Building a Sales Pipeline Analyzer. That happens on September thirtieth, and that session is a builders focused, and our team is going to break down what actually separates an AI app from a chatbot, and build one end to end on a sales pipeline. So we have the links for both of these down in the chat. We've got the QR codes that you're welcome to scan up on the screen right now. Now we are down to just a couple of minutes, and I had hoped that we were going to be able to get to questions today, and unfortunately it doesn't look like we're going to. We're almost at time, so I need to wrap things up. Before we sign off, it's worth giving some consideration to how much the role of semantic layers shifted. It started as a convenience, as a way to make reports a little more readable for the people building and maintaining them, but once AI interfaces started answering questions directly, that same layer stopped being about readability and started being about the degree to which those answers can be trusted. That's a fundamental shift in both utility and significance, and it's no longer just about the report that you need this quarter. It's now part of the foundation everything else gets built on, which includes whatever the next wave of AI tools turns out to be. Organizations that already have that foundation in place tend to adopt what comes next without a lot of friction. The ones that don't are unfortunately faced with a far more difficult task to keep pace. So wherever your organization sits today, that's probably worth a closer look, not because the moment demands it, but because the return on getting it right tends to compound quietly over time. I'd encourage you to review your own roadmap, figure out where and how investing in these changes could help you maximize the benefits from your IT infrastructure. And we are right at time. Thanks to everyone for joining us today. This has been a lot of fun. I hope you've gotten some useful information, and we hope to see you again soon. Bye for now.

In this webinar, data engineer Tim Currell walked through the platform’s two user-facing AI tools, Snowflake Cowork and Coco (formerly Cortex Code), both powered by Anthropic models inside Snowflake’s security perimeter. Out of the box, those tools read raw tables and guess at what things mean. The fix is a semantic layer, and in Snowflake that lives as a semantic view: a governed database object holding your tables, relationships, facts, dimensions and metrics. Currell demonstrated Semantic View Autopilot, then covered BI tool connectivity, cost, rollout strategy and four features coming soon.

InterWorks uses cookies to allow us to better understand how the site is used. By continuing to use this site, you consent to this policy. Review Policy OK

×

Interworks GmbH
Ratinger Straße 9
40213 Düsseldorf
Germany
Geschäftsführer: Mel Stephenson

Kontaktaufnahme: markus@interworks.eu
Telefon: +49 (0)211 5408 5301

Amtsgericht Düsseldorf HRB 79752
UstldNr: DE 313 353 072

×

Love our blog? You should see our emails. Sign up for our newsletter!