Matillion 101: Starting from Scratch with Matillion

Data

Matillion 101: Starting from Scratch with Matillion

by Kathryn Bridges

Matillion 101 breaks down Matillion to its most fundamental levels and how its interactions with Snowflake and AWS can revolutionize your data practice and ETL processes.

Holt Calder wrote an excellent blog post a couple months ago about why Matillion is so great and why you should be using it. As he writes, “We have been using Matillion for Snowflake with as many of our clients as possible. Here’s why: when you build an extensive data warehouse from the ground up, it can sometimes be tough to hand it off at the end of the gig. Matillion effectively automates most tasks in the ETL process, allowing you to build a data warehouse from the ground up with complex ETL processes, all while leaving very few opportunities for routine errors.”

Getting Started with Matillion

Are you ready to use Matillion but not sure where to start? The cloud environment can be a little intimidating, so let’s walk through how to get set up in Matillion so you can start working some ETL magic.

We’re going to set up and launch a new instance of Matillion (for AWS) on EC2 and then walk through how to log in. If your company already has a Matillion instance set up, you can skip to the bottom and just log in from your EC2 console. While we’re focusing on AWS for this blog, Matillion is also available on Azure and GCP.

Logging into AWS

If you’re doing this on a personal machine or for a personal project, the console URL will be: https://signin.aws.amazon.com/console; if your company has their own AWS infrastructure set up, you’ll want to use their specific URL. Enter your credentials to get logged in to the console.

Next, on the Amazon Marketplace, search for Matillion. You’ll see two options: one for Redshift and one for Snowflake. I’m choosing Matillion ETL for Snowflake:

logging into AWS with Matillion

Once found, click Continue to Subscribe then Accept Terms and Continue to Configuration.

Note: In this write-up, I am using the free trial, but after 14 days, there is a cost associated with Matillion; additionally, there is a small fee for using AWS EC2 instances, starting at $0.0464 per hour for t2.medium instance type we’re using here:

launching a new instance of Matillion

In the configuration, I’m setting this up on an EC2 instance, so I select the Amazon Machine Image as my Fulfillment Option and then Continue to Subscribe. What exactly is an AMI? It’s all the information needed to launch an instance, which is a virtual server in the cloud. You can dig in more on the AWS docs where they detail more specifically what’s included:

  • A template for the root volume for the instance (for example, an operating system, an application server and applications)
  • Launch permissions that control which AWS accounts can use the AMI to launch instances
  • A block device mapping that specifies the volumes to attach to the instance when it’s launched

Other fulfillment options include CloudFormation (which you’d want to use if you were launching your instance using templates) and a Private Amazon Machine Image (this option would be ideal for needing to work with very sensitive data or strict security protocols, such as HIIPA).

Launching Your Matillion Instance

On the final screen, I set my Choose Action to Launch from EC2 console, and I adjust the EC2 instance type to t2.medium since I’m working with very small datasets. Feel free to keep the large size if that suits your use case. Next, launch!

launching a new instance of Matillion

Woo hoo! Successfully deployed!

Next time you log into the AWS console, or if your company already has AWS set up, you’ll want to go back to that AWS console. Look under Compute and select EC2. This will take you to your EC2 console:

launching a new instance of Matillion

When I click on EC2, I can see I have one EC2 instance running. If I click on 1 Running Instance, I’ll see the Matillion instance I just set up:

launching a new instance of Matillion

launching a new instance of Matillion

Now, under the Public DNS, I can grab that URL, and once I put it in my browser, I can now log into Matillion:

launching a new instance of Matillion

More About the Author

Kathryn Bridges

Analytics Consultant
Matillion 101: Completing My First Job in Matillion If you’ve been following along in this Matillion series, you’ve already set up and logged into Matillion. From here, I’m going to talk ...
Time Travel with Snowflake A couple years ago, as I was trying to teach myself programming, I managed to wipe out my computer. It was a silly mistake—all I was ...

See more from this author →

Subscribe to our newsletter

  • I understand that InterWorks will use the data provided for the purpose of communication and the administration my request. InterWorks will never disclose or sell any personal data except where required to do so by law. Finally, I understand that future communications related topics and events may be sent from InterWorks, but I can opt-out at any time.
  • This field is for validation purposes and should be left unchanged.

InterWorks uses cookies to allow us to better understand how the site is used. By continuing to use this site, you consent to this policy. Review Policy OK

×

Interworks GmbH
Ratinger Straße 9
40213 Düsseldorf
Germany
Geschäftsführer: Mel Stephenson

Kontaktaufnahme: markus@interworks.eu
Telefon: +49 (0)211 5408 5301

Amtsgericht Düsseldorf HRB 79752
UstldNr: DE 313 353 072