Extracting Scalability and Performance Metrics from TCP Traffic

A session at Surge 2011

Friday 30th September, 2011

10:00am to 11:00am (EST)

Many systems that speak over TCP have simple call-and-response protocols. These lend themselves well to extracting reasonably accurate request and response start and end times. This is a fabulously rich source of data that can be mined for many types of purposes. All that is needed is the first 384 bytes of each packet, containing IP and TCP headers but no payload. This is simple and non-intrusive to capture.

What can we do with this data? Here are some examples:

  • Apply Little's Law to determine metrics such as utilization and concurrency.
  • Extract the data needed for queueing theory analysis.
  • Apply the Universal Scalability Law to determine the system's scalability limits.
  • Compute aggregate metrics such as response-time quantiles for SLA management.
  • Apply algorithms to detect spikes and stalls at various levels of granularity.
  • Produce plots and graphs such as time-series and scatter-plot to reveal important performance and scalability characteristics of the system.
  • Collect historical data for later analysis, trending, and capacity planning.

This analysis technique works for any system, at any layer in the stack, as long as the TCP-based protocol follows call-and-response semantics. Examples of systems that conform to this are most databases, including relational and NoSQL databases; key-value caches such as memcached; and many types of HTTP interactions.

This talk will introduce tools and techniques for capturing and analyzing the network traffic to give deeper insight into system behavior. This knowledge can reveal patterns that can be predictive of emerging problems, such as impending saturation or lock contention in an architectural layer. This overall system performance analysis is a valuable technique for finding problems without needing to analyze each layer independently.

About the speaker

This person is speaking at this event.
Baron Schwartz

Chief Performance Architect, Percona

Next session in Corinthian Room

11am Building a cloud service on a cloud infrastructure. Also, cloud. by Mike Panchenko

Coverage of this session

Sign in to add slides, notes or videos to this session

Tell your friends!


Time 10:00am11:00am EST

Date Fri 30th September 2011


Corinthian Room, Tremont Plaza Hotel

Short URL


View the schedule



Books by speaker

  • High Performance MySQL

See something wrong?

Report an issue with this session