For years, designing a data lake meant choosing a side. You either built entirely within the AWS ecosystem (Glue, Athena, EMR) or you centralized inside the Snowflake Data Cloud. Choosing one usually meant dealing with painful data duplication, expensive pipeline maintenance, and vendor lock-in to make them talk to each other.

At a recent Snowflake conference, a hands-on lab titled “Build Data Lakes using Apache Iceberg with Snowflake and AWS Glue” set out to erase that boundary. The workshop demonstrated how Apache Iceberg acts as a universal storage layer on AWS S3, allowing both AWS Glue and Snowflake to read, write and govern the exact same files simultaneously, without moving or copying a single byte of data.

Core Concept: What Is Apache Iceberg and Why Use It?

Apache Iceberg is an open table format for huge analytical datasets that enables high performance analytics on open data formats with ACID compliance. The key benefit is interoperability, Because the data lives on S3 in an open format, multiple engines can read and write it without copying or locking data into a proprietary system. Snowflake and AWS both support Iceberg format, enabling customers to drastically improve data interoperability, speed of implementation and performance for integrated data lakes.

The Setup: A Financial Services Scenario

The lab used a financial services use case where insurance data is analyzed. Quotes data is collected from systems and stored as Parquet on S3, while customer and policy data is already available as internal Snowflake tables. The goal was to identify customers who are likely to churn or commit potential fraud based on a high number of recent quote requests. This is representative of a broader pattern: combining data that lives in a cloud object store (S3) with data already managed inside Snowflake, all without moving or duplicating anything.

Walking Through the Lab

The lab walked through an end-to-end open lakehouse architecture across several phases.

1. AWS Infrastructure Setup

The lab began by deploying a CloudFormation template to provision an S3 bucket and IAM role, giving Glue the permissions it needs to operate.

2. In-Place Metadata Conversion: Parquet to Iceberg with AWS Glue

The lab started with raw insurance data stored as standard Parquet files in AWS S3. A Glue ETL job read that raw Parquet quotes data from a public S3 bucket and wrote it as an Iceberg table, updating the Glue Data Catalog table in place, with no data copy or rewrite. This is a key point: Converting to Iceberg is an in-place metadata operation, not an expensive data migration. The underlying data wasn’t duplicated; Glue simply layered the Iceberg framework over existing storage.

3. Security via Lake Formation and Credential Vending

To secure the open tables, the lab configured AWS Lake Formation for fine-grained access control and credential vending on the Iceberg table. This is a meaningful shift. Instead of manually hardcoding static AWS IAM roles or external volume configurations inside Snowflake, credential vending allows Snowflake to dynamically request temporary, short-lived S3 access credentials from AWS Lake Formation at the exact moment a query is executed. No external volume configuration is needed on the Snowflake side.

4. Connecting Snowflake via Catalog Integration and Catalog-Linked Database

A Snowflake Catalog Integration connects to AWS Glue via the Iceberg REST Catalog (IRC) API, using SIGV4 authentication and Lake Formation vended credentials. A Catalog-Linked Database then auto-discovers and syncs all Iceberg tables from the Glue Data Catalog. From there, the external Iceberg data can be queried as if it were a native Snowflake table.

5. Bidirectional Read/Write Integration

The final step joined the external Iceberg quotes table with internal Snowflake CUSTOMER and POLICIES tables, wrote the aggregated results back out as a new Iceberg table on S3 and confirmed that table was immediately visible in the Glue Data Catalog. That means other AWS engines, like Athena or EMR, could query it directly.

The ultimate validation? Engineers joined the external S3 Iceberg data with internal Snowflake customer tables, wrote the results back out as a new Iceberg table on S3, and it was instantly visible and queryable by AWS engines like Athena.

Key Takeaways

• In-place metadata conversion: Converting Parquet to Iceberg with Glue is a metadata operation layered over existing storage, not a data migration.
• Security via credential vending: Lake Formation lets Snowflake request temporary, short-lived S3 credentials at query time instead of relying on static IAM roles or external volume configuration.
• Bidirectional read/write integration: Snowflake’s Catalog Integration, built on the open Iceberg REST Catalog API, automatically discovers and syncs tables from the Glue Data Catalog, and data written back by Snowflake is immediately queryable by AWS engines like Athena.
• No data movement, either direction: Data written by Glue is immediately queryable by Snowflake, and data written by Snowflake is immediately readable by AWS services, all on the same open Iceberg format in S3, governed centrally by Lake Formation.

The Big Picture

The lab demonstrated that you don’t have to choose between Snowflake and the AWS ecosystem. By anchoring a storage strategy around Apache Iceberg, both platforms can read, write and govern the same open tables on S3, on a single, open and secure foundation. You can learn more about this whole process here.