This blog post is AI-Assisted Content: Written by humans with a helping hand.
For years, enterprise data teams have faced a frustrating choice: a data lake (cheap, flexible, but a governance nightmare) or a data warehouse (fast, secure, but expensive and locked to a single vendor). At a recent Snowflake conference lab, “Power Analytics and AI Wherever Your Data Lives,” a hands-on workshop proved that compromise no longer has to exist.
The lab connected Snowflake to an Apache Iceberg data lake sitting in AWS S3 — and then queried, governed and ran AI models against that data entirely from Snowflake, without moving or copying a single byte. The dataset used was insurance quote records, but the pattern applies to any industry.
The Core Concept: The Open Lakehouse
Iceberg is an open table format that brings warehouse-grade features like ACID transactions, schema evolution and fast queries to raw files sitting in cheap cloud storage like S3.
The big unlock: Your data physically stays in S3, but Snowflake treats it like a native table.
What Was Actually Built
Three objects wired the whole architecture together:
- External Volume: tells Snowflake where the data lives in S3 and how to authenticate.
- Catalog Integration: tells Snowflake how to talk to AWS Glue’s catalog (the index of all tables).
- Catalog-Linked Database: auto-discovers and syncs tables from Glue into Snowflake automatically, with no manual registration.
3 Reasons This Changes Enterprise Data Economics
1. Zero-Copy Governance
Traditionally, masking PII meant duplicating data or building complex transformation pipelines. Here, engineers applied dynamic data masking policies at the query level instead. Analysts see partially hidden PII (emails, phone numbers, dates of birth); data engineers see full values. The raw files in S3 never change — only what’s displayed depends on the viewer’s role, enforced by Snowflake on the fly and completely decoupled from where the data is stored.
2. Natural Language AI Analytics
Raw files in a data lake lack business context, which makes them hard for AI to reason about. Semantic Views solve this by explicitly defining metrics (like “average premium”) and dimensions (like “product type” or “region”) in business terms. On top of that layer, the Cortex Agent lets anyone — technical or not — ask plain-English questions (e.g., “Which postcode has the highest risk premium?”) and get back governed, accurate SQL results straight from the S3 data.
3. Radical Cost Efficiency with Zero Vendor Lock-in
S3 storage is far cheaper than warehouse-native storage, so keeping data in S3 while using Snowflake’s compute delivers the best of both worlds. And because Iceberg is an open format, the data is never trapped — the exact same files can be queried by Spark, Athena or Snowflake simultaneously, with no rewriting or migration required.
Why This Matters
- No vendor lock-in: an open format means your data works across engines, not just one platform.
- Cost efficiency: cheap S3 storage plus Snowflake’s compute, instead of paying warehouse prices for everything.
- Governance without moving data: enterprise-grade masking and role-based access applied in place, with zero ongoing AWS touchpoints after setup.
- AI-ready by design: the semantic layer plus agent pattern is exactly how modern teams are building self-service analytics that non-technical users can trust.
How to Apply This at Work
| Situation | How to Use What You Learned |
|---|---|
| You have raw data in S3 that’s not in Snowflake. | Connect it via a Catalog-Linked Database instead of loading it — faster and cheaper. |
| Teams share data but have different access levels. | Apply dynamic masking policies on a view wrapper — one table, multiple access tiers. |
| A business team needs self-service analytics. | Build a Semantic View + Cortex Agent so they can ask questions in plain English. |
| You’re worried about vendor lock-in. | Store data in Iceberg format in S3 — any compatible engine can read it. |
| Your data lake is growing but governance is weak. | This whole pattern (Glue + Snowflake + masking) is a production-ready governance model. |
The Bottom Line
You no longer have to choose between a governed warehouse and a flexible data lake. This lab showed a production-ready architecture that delivers both. Your data stays wherever it lives, security travels with it automatically and it’s immediately ready for AI-powered, plain-English analytics.
