Using iperf3 to Help Troubleshoot Speed and Throughput Issues

When someone reports that a network connection is slow, the first question is usually whether the problem is bandwidth, latency, packet loss, TCP behavior, an endpoint limitation, or the application itself. iperf3 is one of the most useful tools for separating those possibilities because it generates controlled traffic between two endpoints and measures the throughput achieved across the path.

The key is not simply running one command and treating the result as the speed of the network. A useful iperf3 test should compare a single TCP stream, multiple parallel streams, both traffic directions, and when appropriate, controlled UDP traffic. The differences between those tests often tell you more than the final throughput number.

What iperf3 actually measures

iperf3 measures the throughput that its test process can achieve between a client and server. It supports TCP, UDP, and SCTP, although TCP and UDP are the protocols most commonly used during network troubleshooting.

A standard TCP test measures the interaction of the entire path:

• The client and server operating systems
• TCP congestion control and receive-window behavior
• Endpoint CPU and memory performance
• NIC drivers and hardware offload features
• Firewalls, routers, load balancers, and security inspection devices
• Packet loss, latency, congestion, and queueing across the network

This is important because a low iperf3 result does not automatically mean the network link is defective. The endpoint, TCP stack, virtual machine, NIC, firewall, or test methodology may be the limiting factor.

The central rule: iperf3 measures end-to-end achievable throughput. It does not independently identify which component is responsible for limiting that throughput.

The basic client and server relationship

One endpoint runs in server mode and waits for a test connection:

iperf3 -s

By default, the iperf3 server listens on TCP port 5201. The client connects to the server by specifying the server’s hostname or IP address:

iperf3 -c 172.16.10.100

During a normal test, the client sends the test traffic and the server receives it.

Know the important defaults

The default TCP buffer length is not the same thing as a fixed TCP receive window. Modern operating systems normally use TCP autotuning, which adjusts the usable TCP window based on conditions and system limits.

Start with a single TCP stream

A single stream is not a meaningless test. It establishes the performance of one TCP connection, which is often exactly how an application or file transfer behaves.

Start with a 30-second forward test and omit the first three seconds from the reported statistics. This reduces the influence of TCP slow start on the result:

iperf3 -c 172.16.10.100 -t 30 -O 3 -i 1 -T "TCP-1Stream-Forward"

The options used are:

A slow single-stream result can expose a limitation involving latency, packet loss, TCP congestion control, receive-window growth, endpoint processing, or a per-flow policer. Those problems can be hidden when the test immediately begins with dozens of parallel streams.

Always test the reverse direction

Next, run the same test in reverse:

iperf3 -c 172.16.10.100 -t 30 -O 3 -i 1 -T "TCP-1Stream-Reverse" -R

The client still establishes and controls the test, but the server becomes the sender and the client becomes the receiver.

This comparison matters because the two directions may encounter different conditions:

• Asymmetric routing
• Different firewall processing paths
• Different QoS or policing policies
• Different send and receive host limitations
• Different congestion levels
• Different cloud or provider return paths

If the forward direction performs well but the reverse direction performs poorly, that is already useful evidence. It narrows the investigation to something directional rather than a general lack of bandwidth.

Then add parallel streams

After establishing the single-stream baseline, increase the number of streams incrementally. The -P option creates multiple parallel connections:

# Four parallel streams
iperf3 -c 172.16.10.100 -P 4 -t 30 -O 3 -i 1 -T "TCP-4Streams"

# Eight parallel streams
iperf3 -c 172.16.10.100 -P 8 -t 30 -O 3 -i 1 -T "TCP-8Streams"

# Eight streams in the reverse direction
iperf3 -c 172.16.10.100 -P 8 -t 30 -O 3 -i 1 -T "TCP-8Streams-Reverse" -R

When reading parallel-stream output, use the SUM line for aggregate throughput rather than any individual stream line.

Parallel streams can achieve higher aggregate throughput because each stream maintains its own TCP congestion state. On a multipath network, the connections may also hash across different ECMP paths because each connection uses a different transport-layer flow.

Why not start with 20 or 40 streams?

Large stream counts can be useful when testing very high-speed links, but they should not be the first test. Starting with 20, 30, or 40 streams changes too many things at once and may conceal the original problem.

A large number of streams can:

• Spread traffic across multiple ECMP paths
• Distribute work across multiple CPU cores and NIC queues
• Overcome a per-flow TCP limitation
• Bypass the traffic pattern of the application being investigated
• Create unnecessary load on production infrastructure

A better progression is 1, 4, 8, and then 16 streams if needed. Increase beyond that only when the endpoints and network still cannot drive the expected capacity.

Important: A test reaching line rate with 20 parallel connections does not prove that a single application connection will achieve the same result.

How to interpret single-stream and parallel-stream results

Watch retransmissions, not just throughput

During a TCP test, iperf3 normally displays a retransmission count on the sender. Retransmissions mean TCP had to send data again because the original segments were not acknowledged as expected.

A high retransmission count may be caused by:

• Congestion and queue drops
• Physical-interface errors
• Packet loss across a carrier or WAN service
• Firewall or security-device resource exhaustion
• MTU or TCP offload problems
• Wireless interference
• An overloaded receiving host

Retransmissions do not identify the location of the loss. They tell you that TCP experienced loss or acknowledgement disruption somewhere between the sending process and receiving process.

That is where other tools become important. Use interface counters, firewall statistics, packet captures, MTR, path monitoring, telemetry, and endpoint CPU data to determine where the loss is occurring.

Understanding the TCP window and the -w option

The -w option sets the socket send and receive buffer sizes on both endpoints. It indirectly affects the maximum TCP window available to the connection.

Do not automatically set a window size on every test. Begin with the operating system’s normal TCP autotuning behavior. Add -w only when investigating whether socket-buffer limits are preventing the connection from filling a high-bandwidth, higher-latency path.

The approximate bandwidth-delay product is:

Required window in bytes =
Bandwidth in bits per second × RTT in seconds ÷ 8

For example, a 10 Gbps path with 20 milliseconds of round-trip latency has a bandwidth-delay product of approximately 25 MB:

10,000,000,000 × 0.020 ÷ 8 = 25,000,000 bytes

A test using a manually configured buffer could look like this:

iperf3 -c 172.16.10.100 -P 4 -w 32M -t 30 -O 3 -i 1 -T "TCP-4Streams-32M"

Operating-system limits may prevent iperf3 from receiving the exact buffer size requested. On Linux, the effective maximum window may also be approximately twice the value supplied because of how the kernel accounts for socket-buffer overhead.

Testing discipline: Change one major variable at a time. If you change both the stream count and window size between tests, you cannot confidently determine which change affected the result.

Testing 40 Gbps, 100 Gbps, and faster links

High-speed testing introduces endpoint constraints that may not appear on lower-speed links. The network may be capable of line rate while the iperf3 host cannot generate or receive traffic quickly enough.

Beginning with iperf3 version 3.16, each parallel test stream uses a separate thread. This allows a modern iperf3 test to use multiple CPU cores rather than forcing all parallel streams through one CPU core.

For that reason, verify the installed version before testing a high-speed link:

iperf3 --version

For a 40 Gbps or 100 Gbps test, begin with version 3.16 or later and increase the stream count gradually:

iperf3 -c 172.16.10.100 -P 4 -t 60 -O 5 -i 1 -T "TCP-40G-Test-P4"

iperf3 -c 172.16.10.100 -P 8 -t 60 -O 5 -i 1 -T "TCP-40G-Test-P8"

iperf3 -c 172.16.10.100 -P 16 -t 60 -O 5 -i 1 -T "TCP-40G-Test-P16"

Also monitor the test hosts for:

• CPU utilization and individual core saturation
• NIC queue and interrupt distribution
• NUMA placement
• Virtual-machine CPU scheduling
• Memory pressure
• TCP segmentation and receive offload behavior
• Interface errors and dropped packets

If the host CPU reaches its limit before the link does, the test result describes the host’s maximum performance, not the network’s maximum performance.

When multiple iperf3 processes are still useful

Older iperf3 releases were effectively limited to one CPU core per process. This made multiple independent processes a common technique when attempting to drive 40 Gbps or 100 Gbps links.

Version 3.16 and later reduces that requirement because parallel streams use separate threads. Multiple processes are still useful when you need to:

• Run several independent tests concurrently
• Test separate NICs or source addresses
• Bind processes to different CPUs or NUMA nodes
• Compare separate network paths
• Work around limitations in an older iperf3 release
• Determine whether one process is still the bottleneck

Start separate server processes on different ports:

iperf3 -s -p 5101 &
iperf3 -s -p 5102 &
iperf3 -s -p 5103 &

Then start separate client processes using the corresponding ports:

iperf3 -c 172.16.10.100 -p 5101 -P 4 -t 60 -T "Test-1" &
iperf3 -c 172.16.10.100 -p 5102 -P 4 -t 60 -T "Test-2" &
iperf3 -c 172.16.10.100 -p 5103 -P 4 -t 60 -T "Test-3" &

These processes will compete for the same CPU, memory, NIC, PCIe, and network resources unless they are deliberately separated. More processes do not guarantee a more accurate result.

Using UDP to test loss and jitter

TCP automatically adjusts its sending rate in response to congestion and loss. UDP does not. An iperf3 UDP test sends traffic at a configured target rate and reports whether the path delivered it successfully.

A basic 500 Mbps UDP test is:

iperf3 -c 172.16.10.100 -u -b 500M -t 30 -i 1 -T "UDP-500M"

The important options are:

• -u selects UDP for the test traffic.
• -b 500M sets the target rate to 500 Mbps.
• -t 30 runs the test for 30 seconds.

UDP testing is useful for measuring:

• Packet loss at a specific offered rate
• Jitter
• QoS behavior
• Policer thresholds
• Real-time traffic characteristics

Increase UDP bandwidth gradually rather than immediately transmitting at the full circuit rate:

iperf3 -c 172.16.10.100 -u -b 500M -t 30 -T "UDP-500M"
iperf3 -c 172.16.10.100 -u -b 1G -t 30 -T "UDP-1G"
iperf3 -c 172.16.10.100 -u -b 2G -t 30 -T "UDP-2G"

Be careful when combining UDP with parallel streams. The configured -b rate is applied to each stream. For example, -P 4 -b 500M can offer approximately 2 Gbps in total.

Production warning: UDP does not reduce its rate in response to congestion. An aggressive UDP test can create packet loss for other applications. Coordinate the test and begin below the expected available bandwidth.

Bidirectional testing

Current iperf3 versions also support --bidir, which sends traffic in both directions simultaneously:

iperf3 -c 172.16.10.100 --bidir -P 4 -t 30 -O 3 -T "TCP-Bidirectional"

This can be useful for testing full-duplex utilization, but it should not replace separate forward and reverse tests during initial troubleshooting. Separate tests make directional differences easier to isolate.

A repeatable troubleshooting sequence

The following sequence provides a much clearer picture than immediately running a large number of streams:

# 1. Confirm the installed version
iperf3 --version

# 2. Start the server
iperf3 -s

# 3. Single-stream forward baseline
iperf3 -c 172.16.10.100 -t 30 -O 3 -i 1 -T "TCP-1Stream-Forward"

# 4. Single-stream reverse baseline
iperf3 -c 172.16.10.100 -t 30 -O 3 -i 1 -T "TCP-1Stream-Reverse" -R

# 5. Four parallel streams
iperf3 -c 172.16.10.100 -P 4 -t 30 -O 3 -i 1 -T "TCP-4Streams"

# 6. Eight parallel streams
iperf3 -c 172.16.10.100 -P 8 -t 30 -O 3 -i 1 -T "TCP-8Streams"

# 7. Reverse the parallel test
iperf3 -c 172.16.10.100 -P 8 -t 30 -O 3 -i 1 -T "TCP-8Streams-Reverse" -R

# 8. Optional controlled UDP test
iperf3 -c 172.16.10.100 -u -b 500M -t 30 -i 1 -T "UDP-500M"

What to document with every test

An iperf3 result without context is difficult to compare or reproduce. Record the following information:

• Client and server hostnames and IP addresses
• iperf3 version on both endpoints
• Operating system and whether the endpoint is physical or virtual
• Test date and time
• Traffic direction
• TCP or UDP
• Number of parallel streams
• Test duration
• Configured socket-buffer size, if one was used
• Expected circuit or interface capacity
• Average round-trip latency
• TCP retransmissions or UDP loss
• Endpoint CPU utilization
• Known QoS policies, policers, firewalls, or ECMP paths

For automated testing or long-term comparisons, use JSON output:

iperf3 -c 172.16.10.100 -P 4 -t 30 -O 3 -J > iperf3-test.json

Firewall and security considerations

An iperf3 server should be reachable only from authorized test endpoints. Do not leave an unrestricted throughput-test server exposed to the public internet or an untrusted network.

For the default configuration:

• Allow TCP port 5201 for the control connection and TCP testing.
• Allow UDP port 5201 when performing UDP tests.
• Restrict access by source and destination IP address.
• Disable or stop the server when testing is complete.

Also coordinate high-bandwidth tests with network operations teams. A successful iperf3 test can still create congestion, trigger monitoring alerts, affect production traffic, or activate usage-based cloud charges.

Final takeaway

iperf3 is not simply a command for proving that a connection is fast or slow. Its real value comes from comparing controlled tests and looking at how the result changes.

Begin with one TCP stream. This establishes per-flow performance and gives you a realistic baseline.

Test both directions. Forward and reverse results can reveal asymmetric routing, QoS, firewall, endpoint, or congestion problems.

Add streams gradually. Parallel streams measure aggregate capacity and may use additional CPU cores, NIC queues, or ECMP paths.

Watch loss and retransmissions. Throughput alone does not explain why TCP is performing the way it is.

Tune only when the evidence supports it. Do not change the stream count, TCP buffers, test direction, and duration all at once.

Validate the test hosts. At 40 Gbps, 100 Gbps, and beyond, the endpoint frequently becomes the limiting factor before the network does.

// one stream shows the flow. parallel streams show the path’s aggregate potential. the difference between them is where the troubleshooting begins.