I attended NTT ( Ryu/Lagopus ) seminar in Aug. at NCTU, Taiwan and noticed that Lagopus(SDN/OpenFlow Software Switch) is amazing.
Its L2 switch performance with 10GbE X 2 (RFC2889 test ) for most of packet size are near to 10Gbps and the test platform is
Intel xeon E5-2660 (8 cores, 16 threads), Intel X520-DA2 DDR3-1600 64GB.
For more information please see the attachment pictures that I took from the seminar.
The following is the features:
- Best OpenFlow 1.3 compliant software-based switch
- Multi tables, Group tables support
- MPLS, PBB, QinQ, support
- ONF standard specification support
- OpenFlow Switch Specification 1.3.3
- OF-CONFIG 1.1
- Multiple data-plane configuration
- High performance software data-plane on Intel x86 bare-metal server
- Intel DPDK, Raw socket
- Bare metal switch
- Various management/configuration interfaces
- OF-CONFIG, OVSDB, CLI
- SNMP, Ethernet-OAM functionality
For installing Lagopus switch, you can refer to the following URL. It can give us a common installation guide for Lagopus switch.
https://github.com/lagopus/lagopus/blob/master/QUICKSTART.mdAbout my lagopus environment:sudo vi /usr/local/etc/lagopus/lagopus.conf
interface {
ethernet {
eth0;
eth1;
eth2;
}
}
bridge-domains {
br0 {
port {
eth0;
eth1;
eth2;
}
controller {
127.0.0.1;
}
}
}
But, I put 2 shell scripts for quickly installing and setting up the DPDK. I use DPDK-1.6.0 so that all the scripts are based on this version.
#!/bin/sh
export RTE_SDK=/home/myname/git/DPDK-1.6.0
export RTE_TARGET="x86_64-default-linuxapp-gcc"
make config T=${RTE_TARGET}
make install T=${RTE_TARGET}
install_dpdk.sh
#!/bin/sh
export RTE_SDK=/home/myname/git/DPDK-1.6.0
export RTE_TARGET="x86_64-default-linuxapp-gcc"
DPDK_NIC_PCIS="0000:00:08.0 0000:00:09.0 0000:00:0a.0"
HUGEPAGE_NOPAGES="1024"
set_numa_pages()
{
for d in /sys/devices/system/node/node? ; do
sudo sh -c "echo ${HUGEPAGE_NOPAGES} > $d/hugepages/hugepages-2048kB/nr_hugepages"
done
}
set_no_numa_pages()
{
sudo sh -c "echo ${HUGEPAGE_NOPAGES} > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages"
}
# install module
sudo modprobe uio
sudo insmod ${RTE_SDK}/${RTE_TARGET}/kmod/igb_uio.ko
sudo insmod ${RTE_SDK}/${RTE_TARGET}/kmod/rte_kni.ko
# unbind e1000 NICs from igb and bind igb_uio for DPDK
sudo ${RTE_SDK}/tools/pci_unbind.py --bind=igb_uio ${DPDK_NIC_PCIS}
sudo ${RTE_SDK}/tools/pci_unbind.py --status
# mount fugepagefs
echo "Set hugepagesize=${HUGEPAGE_NOPAGES} of 2MB page"
NCPUS=$(find /sys/devices/system/node/node? -maxdepth 0 -type d | wc -l)
if [ ${NCPUS} -gt 1 ] ; then
set_numa_pages
else
set_no_numa_pages
fi
echo "Creating /mnt/huge and mounting as hugetlbfs"
sudo mkdir -p /mnt/huge
grep -s '/mnt/huge' /proc/mounts > /dev/null
if [ $? -ne 0 ] ; then
sudo mount -t hugetlbfs nodev /mnt/huge
fi
unset RTE_SDK
unset RTE_TARGET
Here is one thing needs to be notice.
The variable DPDK_NIC_PCIS is my Linux eth1, eth2, and eth3's bus info as follows:
DPDK_NIC_PCIS="0000:00:08.0 0000:00:09.0 0000:00:0a.0"
You have to change them by running ethtool to see your eth bus info.So, we need to use the comand "ethtool" to find out the NIC's bus-info as follows:
# ethtool -i eth4
driver: igb
version: 5.0.5-k
firmware-version: 3.11, 0x8000046e
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no
After executing the 2 shell scripts, then we can start the lagopus switch by this:
sudo lagopus -d -- -c3 -n1 -- -p3
4 comments:
I executed the 2 scripts (modified the env variables and NICs accordingly), but when I try to run lagopus it is throwing error that "Cannot init TX queue 0 for port 0 (-22)".
I updated the lagopus.conf in /usr/local/etc/lagopus/lagopus.conf as follows:
interface {
ethernet {
eth0;
eth1;
eth2;
eth3;
}
}
bridge-domains {
br0 {
port {
eth0;
eth1;
eth2;
eth3;
}
controller {
192.168.2.51;
}
}
}
sudo ${RTE_SDK}/tools/pci_unbind.py --status gives the following output.
Network devices using IGB_UIO driver
====================================
0000:04:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=vmxnet3
0000:0b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=vmxnet3
0000:13:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=vmxnet3
0000:1b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=vmxnet3
Network devices using kernel driver
===================================
0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio *Active*
Other network devices
=====================
When I try to run lagopus using "sudo lagopus -d -- -c3 -n1 -- -p15", it is giving the following error:
.........
Initializing NIC port 0 ...
Initializing NIC port 0 RX queue 0 ...
Initializing NIC port 0 TX queue 0 ...
PANIC in app_init_nics():
Cannot init TX queue 0 for port 0 (-22)
.........
Can you help me to find out what am I doing wrong?
The value of the variable DPDK_NIC_PCIS should be modified to your interface bus info. For my example, I reserve the Linux eth0 and put Linux eth1, eth2 and eth3's bus info as 0000:00:08.0, 0000:00:09.0, and 0000:00:0a.0
You can use ethtool to check your interface's bus info.
Could you plz tell me how to use Ryu controller with lagopus via Openflow Protocol!!! I have read ryubook and I am trying to implement SDN in DPDK using RYU controller and Lagopus datapath in Ubuntu 14.04/12. Is it necessary to start/run RYU using mininet. I am new in DPDK so if you know where am i wrong then plz correct me. I am following this paper http://lagopus.github.io/handson/handson.pdf
Post a Comment