AlertFox is part of the Ipswitch family of tools and services. Learn More »

Synthetic vs Real User Monitoring

by / Tuesday, 08 April 2014 / Published in Of Interest

Let me start out by being honest. It seems every single post about this topic on the web is biased. Synthetic monitoring vendors highlight the advantages of synthetic monitoring, real user monitoring (RUM) vendors point out the advantages of RUM. And vendors that offer both, recommend to use both. At AlertFox we focus on having the best synthetic monitoring solution on the market. And here is why we think that a synthetic monitoring solution, when combined with Google Analytics, is a great overall solution for website monitoring:

(1) Use Google Analytics (GA) as a reliable, free real-user monitoring tool, kind of a “RUM lite” solution. Most likely you are already using Google Analytics, so you already have the performance data that RUM provides. This is not surprising. From a technical perspective, RUM services do nothing other than GA does: They insert a clever piece of Javascript into our website and use it to siphon client side browser performance data info back to the mothership.

ga site speed real user monitoring

With GA you will see how your site performs, how the user experience is for your site, and what is slowing your site down. This is in addition to the usual things you will use GA for, like traffic sources and how users navigate around your site.

There is one caveat:  Google Analytic’s performance data is sampled data, which means it uses only a subset of the clients. But as this is a random sample, that is usually not an issue at all.

Image left: Real user monitoring with Google Analytics Site Speed. And in case you wonder, the city with the slowest response time in the top right corner (304s) is Norilsk in Russia.

Note that RUM is by design not suitable for triggering alerts. If you break a website during an early morning update on Sunday, and there are no users on your website making purchases at that time, no alerts can be triggered. Also, RUM is not suitable for monitoring websites that have only low traffic numbers (new websites, niche websites, specialized B2B apps, and especially internal applications).

(2) Use Synthetic Monitoring for alerting and performance monitoring

Synthetic performance monitoring involves having external “monitoring agents”, ideally real web browsers, run scripted transactions against a web application.  These scripts are meant to follow the steps a typical user might–search, view product, log in, download, upload, check out–in order to assess the experience of a user.  Some services use simple low-level agents for this, but increasingly, it’s necessary to run full web browsers to process JavaScript, CSS, and AJAX calls that occur on a website.

Only running real web browsers (IE, Firefox, Chrome) against your website gives you the confidence that if the synthetic transactions are completing, the site is up and running – and actually working as intended.

Measuring synthetic web transaction performance before/after database performance tweaks.Some of our customers actually use synthetic performance monitoring/testing in every part of the lifecycle: Development, QA, staging service and production services. They use iMacros macros as part of their integration tests and can re-use these same scripts on AlertFox for monitoring their production website.

 Last but not least: We use synthetic monitoring ourselves to measure the uptime and performance of the AlertFox system. We use the same macro scripts for QA and on our staging service. For example, the multi-step plot above shows the performance of our public status page before/after a database performance tweaks. And we use Google Analytics data to make sure our website and especially the control panel perform well for every user world wide.


Leave a Reply