Comparing Performance on Tableau 8.3 Server vs Tableau 9.0 Server

Data

Comparing Performance on Tableau 8.3 Server vs Tableau 9.0 Server

Here in InterWorks we are getting a little bit over-excited about Tableau 9.

Especially, when this blog post came out from the Tableau mothership.

Individual Query time improved by 10x
Dashboard Query times improved by 9x
Query Fusion improving times by 2x
And Query Caching improving times by 50x

Those are some pretty bold figures to put out there..

Not being one to believe everything that I’m told, I thought that maybe it would be great for someone to do their own independent testing to check on those numbers.

So…… I waited, and as no one did, it was left to me to do my own testing. So here we go….

Testing

Step 1: Build out 2 identical servers in Amazon Web Services EC2 (http://aws.amazon.com/ec2/) I went for an m3.2xlarge server which has 8 vCPU (Intel Xeon E5-2670 v2) and 30 GB of RAM. (A pretty standard server setup.)

Step 2: Install Tableau 8.3 64bit Server on the first server, and Tableau 9 Beta 4 64-bit Server on the other.

Step 3: Upload my 150 million row Data extract to both servers. Step 4: Create a dashboard to query the Data Source. (if you click on the image it will expand)

Tableau Server Performance Testing Dashboard

Tableau Server Performance Testing Dashboard[/caption] Step 5: Test dashboard, record the timings and compare.

Results

First let’s look at the Tableau 8.3 performance recording.

Tableau 8.3 8 vCPU Performance Recording

Tableau 8.3 8vCPU Performance Recording 

So what we have here is a pretty standard Tableau 8 dashboard performance recording, with each query happening sequentially, one after the other. The duration of the longest query came in at 10.78 seconds, with the overall duration from start to finish of around 22 seconds.

Now, let’s take a look at the same query on Tableau 9 Server

Tableau 9 8vCPU Performance Recording

Tableau 9 8vCPU Performance Recording

Whoa!!! So what’s happening here then?

To start with, we have much shorter query durations, with the longest query length down from 10.78 to 5.4 seconds, and we also have parallel query executions to bring the overall time down from 22 seconds to 7.5 seconds

That’s not bad at all…

Now, let’s take a look at the underlying server performance, to see what’s happening

First the Tableau 8.3 server

Tableau 8.3 8vCPU CPU Performance

Tableau 8.3 8vCPU CPU Performance 

There are 2 points to notice here.

Firstly, the total processor usage never gets above 27% [blue] of what the server is capable of for the duration of the query. 
Secondly, both the VizQL (vizqlserver [purple]) and Data Engine (tdeserver64 [green]) processes max out at one vCPU each.

Now let’s take a look at the Tableau 9 server..

Tableau 9 8vCPU CPU performance

Tableau 9 8vCPU CPU performance

We see that again the VizQL process continues to use 1 vCPU, but now the Data Engine is multi-core, with the result that the total processor usage peaked at around 96%, and a much much shorter overall query time.

Now, I know what you’re thinking; what effect does increasing the number of cores in a server have on the overall query times?

As you’ve asked so nicely, let’s find out…

I changed the underlying image on AWS from a m3.2xlarge to a c4.4xlarge for both the Tableau 8.3 and Tableau 9 Servers.

c4.4xlarge server has 16 vCPU (Intel Xeon E5-2666 v3) with 30GB of RAM (effectively doubling the core count for this test)

The same query run on Tableau 8.3 Server.

Tableau 8.3 16vCPU Performance Recording

Tableau 8.3 16vCPU Performance Recording

Tableau 8.3 16vCPU Processor Performance

Tableau 8.3 16vCPU Processor Performance

We do see a shorter overall query time (down from 22 seconds to around 16 seconds).

But, we see the same process & processor usage profile as Tableau 8.3 running on 8 vCPU, with only one core being used for each of the VizQL and Data Engine processes.

(Note that the improved query times moving from 8 to 16 vCPU may well be due to the changes in the underlying hardware, including the use of a faster Processor.

Now, on to Tableau 9 running on 16 vCPU

Tableau 9 16vCPU Performance Recording

Tableau 9 16vCPU Performance Recording

Tableau 9 16vCPU Processor Performance

Tableau 9 16vCPU Processor Performance

Again, we see the parallel querying and also, and an improvement in query times, with the longest query now taking 3.34 seconds, and the overall query time down to a sparkling 4.5 seconds.

This is reflected in the performance chart, where again we see the multi-core in action. (This time peaking at just under 60% utilisation)

Summary and Conclusions

 

Server Type and Tableau Version Time Taken for Longest Query (secs) Overall Dashboard Duration (secs)
Tableau 8.3 (8vCPU) 10.78 secs 22 secs
Tableau 8.3 (16vCPU) 8.19 secs 16 secs
Tableau 9 Beta 4 (8vCPU) 5.4 secs 7.5 secs
Tableau 9 Beta 4 (16vCPU) 3.34 secs 4.5 secs

  So what to make of all those figures…

  1. In my test, moving from 8 vCPU to 16 vCPU give me a speed increase of around 30% for Tableau 8.3 and around 60% for Tableau 9. As I said earlier, this may well be due to the underlying hardware being better on the c4.4xlarge than on the m3.2xlarge, and nothing to do with Tableau itself. (Investigating this thoroughly is beyond the remit of this piece.)
  2. However, upgrading from Tableau 8.3 to 9.0 on 8 vCPU server boosted performance by just under 3 times, and for the 16 vCPU server, by just over 3.5 times.

Those are some pretty thrilling figures. They’re not quite in the same league as the Tableau Blog figures, but still interesting enough..

I guess it’s going to depend on your own dashboards, your datasets and the overall load on your Servers, as to what performance improvements you will see. Also, in this testing I don’t take into account the other features offered in Tableau 9 such as Query Caching or Query Fusion (which I haven’t gone into here, and if no-one else writes about these features, I may have to do so myself)

So overall, you are going to see great improvements in the times taken to render dashboards. This is going to make your customers happier which is going to make you happier. However, what I’m not so sure about is what effect this upgrade is going to have on your servers. There are a few unknowns out there such as;

  • Are you going to see more user load on your Tableau server, as they now are more readily gain access to their reports?
  • Are you going to be publishing dashboards which you would never have done on Tableau 8 because their performance wasn’t acceptable, but is now (additional load)?
  • Is having a more ‘peaky’ CPU performance profile going to cause you issues, which may necessitate additional Hardware purchases?
  • Or will having Query Caching and Query Fusion save the day, and actually result in less load on the server?

All these are questions and more are what makes this server upgrade extremely exciting and takes Tableau 9.0 to a #WholeNewLevel. If you’re yet to download Tableau Server, or would like more information on it, please click here or feel free to shoot me a message.

More About the Author

Glen Robinson

Platforms Architect
Adding Images to Projects on Tableau Server Here’s a feature that’s been unofficially around for a while on Tableau Server, but is now supported on Tableau Server 10. ...
What’s New in Tableau Server 10 With the imminent arrival of Tableau 10, I thought that it’s time to take a quick look at all the new features and functionality ...

See more from this author →

Subscribe to our newsletter

  • I understand that InterWorks will use the data provided for the purpose of communication and the administration my request. InterWorks will never disclose or sell any personal data except where required to do so by law. Finally, I understand that future communications related topics and events may be sent from InterWorks, but I can opt-out at any time.
  • This field is for validation purposes and should be left unchanged.

InterWorks uses cookies to allow us to better understand how the site is used. By continuing to use this site, you consent to this policy. Review Policy OK

×

Interworks GmbH
Ratinger Straße 9
40213 Düsseldorf
Germany
Geschäftsführer: Mel Stephenson

Kontaktaufnahme: markus@interworks.eu
Telefon: +49 (0)211 5408 5301

Amtsgericht Düsseldorf HRB 79752
UstldNr: DE 313 353 072