TCP DupACKs and TCP Fast Retransmits
Hello all, my name is Mehul and I am with Platforms Networking support. We receive many calls dealing with slow file transfer when copying files using TCP (such as SMB file copying or FTP file transfer). The recommended approach to troubleshoot these scenarios is to take Network Monitor traces at the source and the destination. Analysis of these traces often reveals the following:
- Packets are getting dropped -we will see Retransmits, Duplicate Acknowledgements (DupACKs), and Fast Retransmits. Also:
- ‘The receiver (such as a Windows file server where a client is copying a file or files to the server)’ has a TCP Receive Window Size of Zero.
- Small data packets transferred over a high bandwidth network.
In this post we will be concentrating on DupACKs and Fast Retransmits.
When a Sender sends a segment, it also sends information about the sequence number used. The receiver in return sends an acknowledgment (ACK) – with the ACK flag set, to update sender that it received that segment. For each TCP segment sent there is a retransmission timer bound to it. The value of the retransmission timer is initially defined by TcpInitialRtt and then recalculated to a dynamic value based on connection experience. If the sender does not receive an ACK from the receiver for the TCP segment it sent before the timer expired, the sender retransmits the same TCP segment. If the sender was sending a SYN packet, then TcpMaxConnectRetransmissions would be used to define how many retransmissions are done. For any other type of packet, TcpMaxDataRetransmissions is used to define how many retransmissions are done.
At times, it may so happen that a receiver receives a TCP segment with a sequence number higher than the expected one (out of order segments). The receiver then sends an immediate ACK with the Acknowledgement field set to the Sequence number the receiver was expecting. This ACK is a duplicate of an ACK (DupACK) which was sent previously. This is basically done to update the sender with regards to the dropped/missing TCP segments. After receiving 2 DUPACKs, TCP performs a retransmission of that segment without waiting for the retransmission timer to expire .This is called a Fast Retransmit. The number of ACKs to receive before resending can be set with the TcpMaxDupACKs registry value under the HKEY_LOCAL_MACHINE\System\CurrentControlSetServices\Tcpip\Parameters key.
When a sender has to retransmit, he assumes that network congestion is occurring and goes into a recovery mode on subsequent packets. With Fast Retransmit the recovery is faster. This is known as the Fast Recovery Process.
Let’s take an example where a Windows XP client machine-(Receiver) is copying a 200MB file from a Windows Server 2003 machine (Sender) and some data packets are dropped. When the Receiver receives a packet with a TCP sequence number higher than the expected one, it understands, that some packets were dropped. Receiver then updates the sender about the packet getting dropped as quickly as possible by sending a DupACK, with the ACK number set to the sequence number that is missing. Below is a network trace snippet , taken on the Receiver (192.168.2.4) as it received a stream of data from Sender.
2615 192.168.1.22 192.168.2.4 TCP: Flags=...A...., SrcPort=Microsoft-DS(445), DstPort=1271, PayloadLen=1460, Seq=40444 - 41904, Ack=73803, Win=65535
Server sends data packets with Sequence no’s: 40444 - 41904
2616 192.168.1.22 192.168.2.4 TCP: Flags=...A...., SrcPort=Microsoft-DS(445), DstPort=1271, PayloadLen=1460, Seq=41904 - 43364, Ack=73803, Win=65535
2617 192.168.2.4 192.168.1.22 TCP:Flags=...A...., SrcPort=1271, DstPort=Microsoft-DS(445), PayloadLen=0, Seq=73803, Ack=43364, Win=65535
Receiver acknowledges the data by sending a TCP ACK frame with Ack set to 43364 in frame 2617
The next packet that Receiver expects is with the TCP Sequence starting from 43364.
2618 192.168.1.22 192.168.2.4 TCP: Flags=...A...., SrcPort=Microsoft-DS(445), DstPort=1271, PayloadLen=1460, Seq=44824 - 46284, Ack=73803, Win=65535
However in frame 2618, Receiver receives a packet with sequence starting from 44824. In the next frame(2619,below) Receiver informs Sender that it didn’t receive packets [43364 –44824] by sending a DupACK frame. Note that the AckValue set to 43364.
2619 192.168.2.4 192.168.1.22 TCP: [Dup Ack #2617] Flags=...A...., SrcPort=1271, DstPort=Microsoft-DS(445), PayloadLen=0, Seq=73803, Ack=43364, Win=65535
2620 192.168.1.22 192.168.2.4 TCP:[Continuation to #2609]Flags=...A...., SrcPort=Microsoft-DS(445), DstPort=1271, PayloadLen=1460, Seq=46284 - 47744, Ack=73803, Win=65535
In Frame 2620, above, Receiver gets 46284 –47744, even after sending a DupACK.
Receiver sends back another DupACK frame with ACK set to 43364. After sending 2 DupACK s with the same ACK number, Receiver still receives packets higher than the sequence it was expecting. At this stage Receiver sends Request Fast-Retransmit frame for each frame it receives from the sender here after.
2621 192.168.2.4 192.168.1.22 TCP: [Dup Ack #2617] Flags=...A...., SrcPort=1271, DstPort=Microsoft-DS(445), PayloadLen=0, Seq=73803, Ack=43364, Win=65535
2622 192.168.1.22 192.168.2.4 TCP: Flags=...A...., SrcPort=Microsoft-DS(445), DstPort=1271, PayloadLen=1460, Seq=47744 - 47855, Ack=73803, Win=65535
2623 192.168.2.4 192.168.1.22 TCP: [Request Fast-Retransmit] Flags=...A...., SrcPort=1271, DstPort=Microsoft-DS(445), PayloadLen=0, Seq=73803, Ack=43364, Win=65535
2624 192.168.1.22 192.168.2.4 TCP: Flags=...A...., SrcPort=Microsoft-DS(445), DstPort=1271, PayloadLen=1460, Seq=47855 - 48444, Ack=73803, Win=65535
2625 192.168.2.4 192.168.1.22 TCP: [Request Fast-Retransmit] Flags=...A...., SrcPort=1271, DstPort=Microsoft-DS(445), PayloadLen=0, Seq=73803, Ack=43364, Win=65535
With Selective ACK (SACK) options enabled, the Sender specifies the left edge (starting sequence number of the data stream) and the right edge (ending sequence number of the data stream) in the DupACK frame. This helps Sender in sending only the packets which didn’t make it to Receiver. In the above trace sample, we would see the left edge set to 40444 and right edge set to 43364. Sender then sends the data starting from the sequence 43364 .The receiver continues to use the ACK number to acknowledge the left edge of the receive window, but it also acknowledges other blocks of data individually .SACK options are negotiated during TCP session setup. With SACK options disabled, Sender resends the whole data stream again.
The whole process is treated as a fast recovery mechanism and it has the following advantages:
- With fast retransmit, the sender retransmits the missing TCP segments before their retransmission timers expires
- Only missing/dropped packets are being sent again; NOT the whole data stream.
A point to note, though, is that it’s quite possible that packets may arrive to the receiver out of order and you would see receiver sending DUPACKs.
Note: This behavior is applicable to Windows 2000, Windows Server 2003 and Windows XP. In Windows Vista, Windows 7 and Windows Server 2008 R2, there is a slight change in behavior for fast retransmit which I plan to discuss in my next blog post.
References:
http://msdn.microsoft.com/en-us/library/ms819737.aspx
