Often when working with Matillion Support or investigating other Matillion issues, you will need to retrieve the logs from the Matillion virtual machine. This could be for several reasons, including:

• The logs available through the UI may be trimmed whilst the virtual machine itself stores the unabridged logs and archives.
• You may be unable to access the UI and need to download the logs directly from the virtual machine to identify the issue.

How to Download Logs from the UI

Prerequisites

• Administrator credentials to your Linux virtual machine. This could be either:
  • A username and password
  • A username and private SSH key
• The OpenSSH Client optional service must enabled on your machine, and OpenSSH must be added to your PATH environment variable. You can read how to do that here.

Copying the Matillion Logs to a Temp Directory on the Virtual Machine

Before we download the log file, we first move it to a temporary directory and modify the security of the file, so it can be downloaded. For our example, we will download the main log file (catalina.out); however, you could follow this process for any of the archived files, too (such as catalina.out-2021-08-22.gz).

First, log into your Matillion virtual machine via SSH as demonstrated in this guide. Leverage the following command to make a new temporary directory that will store our copied log file:

sudo mkdir /tmp_<date>

For our example, we use the following command:

sudo mkdir /tmp_2021-09-01

We can then navigate to the folder to verify its existence with the cd command, and use the pwd command to verify which directly we are currently in:

Navigate to the folder on the Matillion VM that stores the logs, using the following command:

cd /var/log/tomcat8

Use the list command ls to verify that the file is within the desired folder. The log file is called catalina.out:

Copy the catalina.out log file into the temporary directory that was created earlier using the following command:

sudo cp catalina.out /tmp_<date>

For our example, this command is:

sudo cp catalina.out /tmp_2021-09-01

Navigate to the temporary directory using the cd command. For our example, this command is:

cd /tmp_2021-09-01

To be safe, use the ls command again to verify that the catalina.out file exists in your temporary directory:

Prepare the file for download by using the chmod command to change the privileges on the file to 755:

sudo chmod 755 catalina.out

Downloading the Log File Using SCP

With a single command, we can leverage SCP (Secure Copy Protocol) to copy a file from a Linux virtual machine to our local machine. SCP is downloaded as part of the standard OpenSSH Client optional service discussed in the prerequisites.

Downloading Files with SCP with SSH Password Authentication

If authenticating via an SSH with a password, this can be achieved by entering your password after using a command such as:

scp user@host:path/to/remote/file path/to/local/destination/filename

For example, my command could be:

scp MatillionAdmin@xx.xx.xx.xx:/tmp_2021-09-01/catalina.out "C:\Users\chris.hastie\Downloads\catalina.out"

The file will now be sitting safely in your local directory, where you can either read it yourself or zip it and send it through to Matillion support.

Downloading Files with SCP with SSH Key Authentication

If authenticating via an SSH private key, this can be achieved with a command such as:

scp -i path/to/ssh/key user@host:path/to/remote/file path/to/local/destination/filename

For example, my command could be:

scp -i "C:\...\matillion-ssh-key.pem" centos@xx.xx.xx.xx:/tmp_2021-09-01/catalina.out "C:\Users\chastie\Downloads\catalina.out"

The file will now be sitting safely in your local directory, where you can either read it yourself or zip it and send it through to Matillion support.