Network Latency vs. Throughput vs. Response Time

The most common complaint from a cloud customer might be, ‘It works much faster inside my data center than on the cloud hosted version’.  There are a few ways to investigate this claim and determine where the actual bottleneck might be.

In network and application latency issues, there are three metrics that are helpful.

Unfortunately, these are used interchangeably. This post will try and clarify what each one means and when to use which.

Response Times

Response Time gives a clear picture of the overall performance of the cloud.

For instance, when both, the end user and the data center are in the same region, the average overall response time may be  50 milliseconds. When the user and data centers are located in different regions, the average response time average may be 300 milliseconds (6 times slower). What gives – and how do your justify this difference?

Network Latency

This is the time interval between submitting a packet and arrival at its destination. If you have ruled out application layer (see load testing your application) as your culprit, this is the first metric you need to examine.

Throughput (aka bandwidth aka data rate)

This refers to the performance of tasks by a computing service or device over a specific period.

1. For transaction processing systems, it is normally measured as transactions-per-second.
2. For systems processing bulk data, such as audio or video servers, it is measured as a data rate (e.g., Megabytes per second).
3. Web server throughput is often expressed as the number of supported users — though clearly this depends on the level of user activity, which is difficult to measure consistently.

Summary

This post serves to differentiate between network latency, throughput and response times.  You would need to start by establishing a baseline (for network latency and response times) by load testing your app (inside your data center). Fiddler, StresStimulus, New Relic and other APM tools can help you establish such a baseline. The same tools can then be run against your cloud hosted application and the new metrics compared with the baseline.

Need help with your application perforrmance? Need an APM ( Application Performance ) certified consultant ? Setup a time with Anuj Varma – a certified New Relic professional and an experienced Fiddler, StresStimulus and web performance specialist.