Tuesday, May 3, 2016

[Linux Bonding] 802.3ad bond interface has shown RX dropped packets

If someone uses Linux Bonding and finds some or a lot of RX dropped packets in bond interface, please ignore these dropped packet message because of the following informations:

1. Linux Bonding and Single Nic with 1GE switch are no difference in packet loss
I use iperf tool with UDP to test packet drop and jitter and the result shows that there are no difference of packet loss between linux bonding and single nic.

2. Bond0 RX packet dropped is not a bug.
Please check out this: https://bugs.launchpad.net/ubuntu/+source/bridge-utils/+bug/1041070
This is related to the bonding mode and _not_ a bug. The bonding module will drop duplicate frames received on inactive ports, which is normal behavior. [0] Overall the packets should be getting into the machine without problems since they are received on the active slave. To confirm this do the following

1) Check dropped packets from all interfaces. So if eth0/eth1 are connected to bond0, we may see dropped packets for bond0 and eth0, but not for eth1. This depends on which interface is the active interface. This can be checked using the following:
cat /sys/class/net/bond0/bonding/active_slave

So if the active_slave isn't dropping packets, and the inactive slave is dropping packets this is normal in 'active-backup' mode (or any mode where there is an inactive slave).

2) If we want both interfaces to not drop packets we can use 'all_slaves_active' bonding module parameter [0].
cat /sys/class/net/bond0/bonding/all_slaves_active, it should default to 0 which means drop frames on the inactive slave.

If we set this to 1, we will no longer drop frames:
echo 1 | sudo tee /sys/class/net/bond0/bonding/all_slaves_active
3. This article suggests to turn off rp_filter ( could reduce RX dropped )
echo net.ipv4.conf.all.rp_filter=0 >> /etc/sysctl.conf
echo net.ipv4.conf.default.rp_filter=0 >> /etc/sysctl.conf
echo net.bridge.bridge-nf-call-iptables=1 >> /etc/sysctl.conf
echo net.ipv4.ip_forward=1 >> /etc/sysctl.conf
sysctl -p
What is reverse path filtering?

Reverse path filtering is a mechanism adopted by the Linux kernel, as well as most of the networking devices out there to check whether a receiving packet source address is routable.

So in other words, when a machine with reverse path filtering enabled recieves a packet, the machine will first check whether the source of the recived packet is reachable through the interface it came in.

If it is routable through the interface which it came, then the machine will accept the packet
If it is not routable through the interface, which it came, then the machine will drop that packet.

Other References:

No comments: