Home
Cloud
MTR and Iperf – Network Diagnostics and Performance Issue Resolution with AWS Direct Connect

MTR and Iperf – Network Diagnostics and Performance Issue Resolution with AWS Direct Connect

Kerry Cordero
CloudRouting & Switching
5 min read

A fast and efficient network is crucial to ensuring optimum organizational performance. In this blog post, we provide a detailed guide on isolating and diagnosing network and application performance issues using various tools such as iPerf3 and MTR. Follow these guidelines meticulously to streamline your network operations.

IMPORTANT: This post focuses on AWS Direct Connect, but these tools should be used throughout your network.

Preparation

As a best practice, set up an on-premises dedicated test machine and an Amazon Virtual Private Cloud (Amazon VPC). Utilize an Elastic Compute Cloud (Amazon EC2) instance of type size C5 or larger.

Diagnosing Network or Application Issues

To diagnose network issues, installing and using the iPerf3 tool can be a great start. Benchmark your network bandwidth and cross-check the results using other applications.

Step 1: Install iPerf3
For Linux/REHEL:

sudo yum install iperf3 -y

For Ubuntu:

$ sudo apt install iperf3 -y

Step 2: Run iPerf3

Proceed to measure the throughput bidirectionally on both the client and server. Replace “ with your EC2 instance’s private IP address in the following commands:

On your Amazon EC2 instance (server):

$ iperf3 -s -V

On the on-premises localhost (client):

$ iperf3 -c 203.0.113.0 -P 15 -t 15
$ iperf3 -c 203.0.113.0 -P 15 -t 15 -R
$ iperf3 -c 203.0.113.0 -w 256K
$ iperf3 -c 203.0.113.0 -w 256K -R
$ iperf3 -c 203.0.113.0 -u -b 1G -t 15
$ iperf3 -c 203.0.113.0 -u -b 1G -t 15 -R

iPerf3 Options

iPerf3 has many options but int this example:

  • -c: This option allows you to specify the host to connect to; it essentially sets the machine running the command to client mode. You follow this option with the IP address or hostname of the server you want to connect to.
  • -s: This option puts iPerf3 in server mode, allowing it to accept connections from iPerf3 clients.
  • -P, --parallel n: The -P option allows you to set the number of parallel client threads to run during the test. For instance, -P 15 would run 15 parallel threads.
  • -t, --time n: The -t option lets you specify the time, in seconds, for which the test should run. For instance, -t 15 will run the test for 15 seconds.
  • -w, --window n[KM]: The -w option allows you to set the TCP window size or socket buffer size, which can affect the bandwidth measurement. For example, -w 256K sets the window size to 256 KBytes.
  • -u, --udp: This option specifies to use UDP rather than TCP for the test. When using this option, you can observe the potential packet loss along the path being tested.
  • -b, --bandwidth n[KM]: When using -u to perform a UDP test, the -b option allows you to set the target bandwidth for the test. For example, -b 1G would set a target bandwidth of 1 gigabit per second.
  • -R, --reverse: This option reverses the direction of the test, allowing you to measure the bandwidth from the server to the client, instead of from the client to the server.
  • -V, --verbose: This option gives more detailed output during the test, helping to provide more insight into what is happening during the test.
  • -i, --interval n: Although not used in your initial commands, this option sets the interval, in seconds, for reporting bandwidth and other statistics. This can be helpful for seeing how performance varies over time.

Each of these options allows you to tailor your iPerf3 tests to your specific needs and environment, providing a robust tool for diagnosing network performance issues. Combining different options can help you identify problems more effectively by varying the conditions of the test.

Analyzing Metrics and Interface Counters

Inspect the Amazon CloudWatch Logs for crucial metrics and ensure the optical signal readings remain within the acceptable range. Additionally, verify the bitrate does not max out, and consult with the Direct Connect owner if utilizing a Hosted Virtual Interface.

Also, check the health of your router and firewall at the Direct Connect location and maintain the fiber patch lead and SFP module regularly.

Utilizing MTR for Network Path Analysis

MTR is a powerful tool for network performance analysis. It is accessible on Linux and can be installed on Windows through WSL 2 or by downloading WinMTR. It’s best practice to use MTR via WSL on a Windows device.

Step 1: Install MTR
For Linux/REHEL:

$ sudo yum install mtr -y

For Ubuntu:

$ sudo apt install mtr -y

Step 2: Run MTR

Run MTR bidirectionally to analyze the network path. Replace “ and “ with your EC2 instance details and “ and “ with your local host details.

On-premises to AWS:

$ mtr -n -c 100 203.0.113.0 --report
$ mtr -n -T -P 8080 -c 100 203.0.113.0 --report

AWS to on-premises:

$ mtr -n -c 100 192.0.2.0 --report
$ mtr -n -T -P 8080 -c 100 192.0.2.0 --report

MTR Options

MTR has many options but int this example:

  • -n, --no-dns: This option instructs mtr to display network addresses as IP addresses instead of trying to resolve them to hostnames, which can make the output more concise and eliminate delays due to DNS resolution failures.
  • -c, --report-cycles COUNT: This option specifies the number of pings sent to determine both the machines on the network and the network quality. For instance, -c 10 would perform 10 cycles of pings.
  • --report: This option tells mtr to run in report mode, where it runs for the number of cycles specified by the -c option (or the default if -c is not specified) and then prints a report and exits. This is useful for batch jobs and other automated environments where you just want a single report rather than the default interactive mode.
  • -T, --tcp: By default, mtr uses ICMP ECHO requests (like ping) to probe the network. The -T option tells mtr to use TCP SYN packets instead, which can sometimes provide more accurate results, particularly when ICMP is being rate-limited or blocked by a firewall.
  • -r, --raw: This option causes mtr to output raw, unprocessed data, which can be useful when you want to process the output with another tool or script. It is a machine-friendly output format, which can be used for scripting solutions to parse the mtr results easily.
  • -w, --wide: While not mentioned in your initial list, this option is useful as it allows for wide output, displaying more information and avoiding the truncation of hostnames.
  • -g, --gtk: Again not mentioned in your initial list but useful to know, this option enables the GTK+ based graphical user interface for mtr. This might not be available in all installations, depending upon whether mtr was compiled with GTK+ support.

Analyzing MTR Test Results

Review the ICMP and TCP-based MTR test results to identify potential issues at various network hops. A deeper analysis could reveal packet losses that indicate issues with Direct Connect connections or remote routers.

Packet Capture and Analysis

Lastly, capture packets on both localhost and the EC2 instance for a detailed network traffic analysis using utilities like tcpdump or Wireshark. Utilize the TCP Throughput Calculator on the Switch website to further understand network limitations and optimize performance.

For instance, use the following command to obtain a packet capture with tcpdump, replacing “ and “ with your specific details:

$ sudo tcpdump -i {network interface} port {port} -w network_capture.pcap

Conclusion

By leveraging tools like iPerf3 and MTR, you can effectively diagnose and analyze network performance issues. Understanding the results these tools provide to implement network optimizations is essential. Periodic checks and maintenance are crucial to ensure your organization’s healthy and robust network infrastructure.

© 2023 cordero.me • All Rights Reserved