How RED works with TCP with or without ECN.
Test Date: 98/01/30
Test System Configuration
The following measurements were done with three PentiumPro machines
(all 200MHz with 440FX chipset) running FreeBSD-2.2.5/altq-0.4.3.
Host A is a source, host B is a router, and host C is a sink.
RED is enabled only on the interface of host B connected to host C.
The link used is 155Mbps ATM (134Mbps over AAL5), but the link speed
between host A and host B is set to 45Mbps, and the link speed between
host B and host C is set to 6Mbps by the hardware shaper to create a
The tests use the TCP test mode of Netperf benchmark program.
Four 20-second-long TCP flows are sent from host A to host C; the flows
start 3-second-apart from each other. (host B is also a source in
trace-5 to trace-8).
Figure 1 Test System Configuration
The following 8 conditions are measured.
- Trace-1: No RED
- Trace-2: simple RED (Drop Only)
- Trace-3: RED with ECN (Packet Marking Only)
- Trace-4: RED with ECN, 2 ECN-capable TCP, 2 non-ECN TCP (Mixed)
- Trace-5: 2 Sources: No RED
- Trace-6: 2 Sources: simple RED (Drop Only)
- Trace-7: 2 Sources: RED with ECN (Packet Marking Only)
- Trace-8: 2 Sources: RED with ECN, 2 ECN-capable TCP, 2 non-ECN TCP (Mixed)
RED statistics are taken at host B.
Table 1. RED statistics
| Trace No. ||xmit pkts ||drop pkts |
|trace-1: No RED ||2853 ||49 (-, -) || -
|trace-2: Drop Only ||2833 ||21 (0, 21) || 0
|trace-3: ECN Only ||2844 ||0 (0, 0) || 24
|trace-4: Mixed ||2849 ||11 (0, 11) || 11
|trace-5: 2 SRC (No RED) ||3084 ||54 (-, -) || -
|trace-6: 2 SRC (Drop) ||2854 ||15 (0, 15) || 0
|trace-7: 2 SRC (ECN) ||2859 ||0 (0, 0) || 29
|trace-8: 2 SRC (Mixed) ||2879 ||9 (0, 9) || 12
The traffic traces are taken at host C using
Tele Traffic Tapper (that uses BPF). The granularity of the traces is
4 TCP flows use src-port 6790 - 6793. The uppermost red line shows the total
The raw tcpdump
traces for those tests are also available.
Trace-1: No RED
When there is no active queue management, packet drop occurs without
any control. Although the link is kept full, traffic behavior is
unpredictable in this situation.
When a flow loses packets, it also loses its share of the link bandwidth.
Figure 2 Trace-1: No RED
Trace-2: simple RED (Drop Only)
When RED is used, the bandwidth allocation becomes fair.
The flows sometimes lose packets but, in many cases,
the flows keep their share by the fast recovery mechanism.
However, flow-6790 experienced a retransmit timeout at time 29.
The fast recovery mechanism fails when the sender's window is not
large enough, or multiple packets get dropped in a window (in a
Reno derived TCP).
Figure 3 Trace-2: Drop Only
Trace-3: RED with ECN (Packet Marking Only)
When ECN is used, link-sharing becomes more stable.
Figure 4 Trace-3: ECN Only
Trace-4: RED with ECN, 2 ECN-capable TCP, 2 non-ECN TCP (Mixed)
This test illustrates what will happen when ECN is introduced to the
2 flows (6791 and 6793) are ECN-capable, but the other 2 (6790 and 6792)
are not ECN-capable. RED drops non-ECN-capable packets instead
of marking. Both of non-ECN-capable flows (6790 and 6792) experienced
retransmit timeouts at time 28, 30 and 35.
Figure 5 Trace-4: Mixed case
Comments by Sally Floyd:
My guess would be that under medium congestion,
ECN-capable TCP might get higher bandwidth then non-ECN-capable TCP,
because it is less likely to have to wait for a retransmit timeout.
However, my conjecture would be that the differences in throughput are
neglibable in mild congestion. The non-ECN-capable TCP has a packet
**dropped** and the ECN-capable TCP has a packet **marked**, but both
TCP's reduce their congestion window by half.
Effects of different RTT
I couldn't produce large RTT difference, but tried to use host B (router)
as another source of flows. RTT between host A and host C is 0.750 ms and
RTT between host B and host C is 0.539 ms (measured by ping). So, the
difference is small. (note that when the link is full, RTT can be more
than 100ms, partly because the queues are full and partly because the
process is not scheduled.)
I tested the same settings used in trace-1 to trace-4
but 2 flows (6791 and 6793) are sent from host B.
The other 2 flows (6790 and 6792) were sent from host A.
Trace-5: 2 Sources: No RED
The flows oscillate more than trace-1.
Figure 6 Trace-5: No RED from 2 Sources
Trace-6: 2 Sources: simple RED (Drop Only)
More retransmit timeouts are observed than trace-2.
Figure 7 Trace-6: Drop Only from 2 Sources
Trace-7: 2 Sources: RED with ECN (Packet Marking Only)
ECN works much better than dropping (trace-6).
Figure 8 Trace-7: ECN Only from 2 Sources
Trace-8: 2 Sources: RED with ECN, 2 ECN-capable TCP, 2 non-ECN TCP (Mixed)
One of non-ECN-capable flows (6792) experienced a retransmit timeout.
Figure 9 Trace-8: Mixed case from 2 Sources
Although these tests are limited in many ways, it is clear that RED
works pretty well with TCP. Even simple RED (without ECN) can provide
fairness to some extent and can provide triggers for fast recovery.
ECN makes these features more stable, and will reduce end-to-end delay
by preventing TCP from retransmission timeouts.
Back to ALTQ/RED page
Back to my home page