Danny's tech notebook | 丹尼技術手札

Wednesday, July 25, 2012

[Shell Script][Example] Files Copying Operation with Warning When the Same File Exists

Because someone asks me to do him a favor for a sample of of file copying operation, I just use bash shell to do so. Here is an example to use shell script to copy files from source path to destination path, which will give a warning message and skip copying if there is a file that exists.

#!/bin/bash

SRC_DIR="/home/liudanny/scripts"                        # put your source directory
DST_DIR="/home/liudanny/Downloads"                      # put your destination directory

for src_full_name in "$SRC_DIR"/*                       # loop all the files in source directory 
do
    fname=$(echo ${src_full_name}|sed 's#^.*/##')       # get the file name
    echo "name:$fname"
    dst_full_name=`printf "%s/%s" $DST_DIR $fname`      # generate the destination file name
    echo "dst_full_name:$dst_full_name"
    if [ -e $dst_full_name ];                           # check if destination file exists
    then
        echo "File: $fname exists..."
    else
        echo "Do copying $fname ..."
        cp $src_full_name $dst_full_name                # if not exists, then copy file
    fi
done

Monday, July 9, 2012

[ARP] How to look up arp table (mac/ip mapping) in Linux

First, we can ping hostname(ip_addr) to add mac/ip mapping into arp table
or use arp command to do so:
> arp -s hostname(ip_addr) MAC_addr

Second, look up arp table:
> arp -n   (I recommend to use "-n" for avoiding DNS lookup. It is faster.)
        Address                  HWtype    HWaddress               Flags Mask            Iface
        10.3.207.245             ether      20:aa:4b:a3:4c:b7            C                     eth4

or there is another way:
> ip ne
        10.3.207.245 dev eth4 lladdr 20:aa:4b:a3:4c:b7 REACHABLE

Thursday, June 28, 2012

[OpenFlow] Update and Setup TP-LINK TL-WR1043ND for OpenFlow

Currently the version of TP-LINK TL-WR1043ND which I can get in Taiwan is "(TW) ver:1.0" instead of ver:1.8 I want. But, fortunately, after asking the vendor about this issue, he responds that there is no difference in hardware between "(TW) ver:1.0"and ver:1.8 or ver:1.4. That is a good news to me though. The following items are my steps to transform original TP-LINK TL-WR1043ND to OpenFlow-enabled switch. Most of information is from http://www.openflow.org/wk/index.php/Pantou_:_OpenFlow_1.0_for_OpenWRT

1. Update firmware on your device
Installing OpenWrt this web site has 4 methods for us to do updating. Because my TL-WR1043ND is brand new one, I choose the simplest of method 1: via orginal firmware.

Download image (v1.8)
Change your PC ip address to 192.168.1.2
Connect to the switch with LAN port
Browse http://192.168.1.1
Choose function "update firmware" with the image bin file

2. Modify Configuration

Login to 192.168.1.1 using telnet
Setup your controller ip address (my controller ip is 192.168.1.244)

vi /etc/config/openflow

        config 'ofswitch'
                option 'dp' 'dp0'

                option 'dpid' '000000000011'

                option 'ofports' 'eth0.0 eth0.1 eth0.2 eth0.3 '
                option 'ofctl' 'tcp:192.168.1.244:6633'
                option 'mode'  'outofband'

Setup your network configuration and setup this switch ip address (my switch ip is 192.168.1.11)

vi /etc/config/network and paste the following lines

        config 'switch'
                option 'name' 'rtl8366rb'
                option 'reset' '1'
                option 'enable_vlan' '1'
                option 'enable_learning' '0'
                
        config 'switch_vlan'
                option 'device' 'rtl8366rb'
                option 'vlan' '1'
                option 'ports' '1 5t'
        
        config 'switch_vlan'
                option 'device' 'rtl8366rb'
                option 'vlan' '2'
                option 'ports' '2 5t'
        
        config 'switch_vlan'
                option 'device' 'rtl8366rb'
                option 'vlan' '3'
                option 'ports' '3 5t'
        
        config 'switch_vlan'
                option 'device' 'rtl8366rb'
                option 'vlan' '4'
                option 'ports' '4 5t'
        
        config 'switch_vlan'
                option 'device' 'rtl8366rb'
                option 'vlan' '5'
                option 'ports' '0 5t'
        
        config 'interface' 'loopback'
                option 'ifname' 'lo'
                option 'proto'  'static'
                option 'ipaddr' '127.0.0.1'
                option 'netmask' '255.0.0.0'
                
        config 'interface'
                option 'ifname' 'eth0.1'
                option 'proto' 'static' 
        
        config 'interface'
                option 'ifname' 'eth0.2'
                option 'proto' 'static' 
        
        config 'interface'
                option 'ifname' 'eth0.3'
                option 'proto' 'static' 
        
        config 'interface'
                option 'ifname' 'eth0.4'
                option 'proto' 'static' 
        
        config 'interface'
                option 'ifname' 'eth0.5'
                option 'proto' 'static'
                option 'ipaddr' '192.168.1.11'
                option 'netmask' '255.255.255.0'  

3. Restart networking to enable your changes

```
/etc/init.d/network restart
```

4. Verify (My case is for Trema Controller)

Connect your Ethernet cable from LAN port to WAN port
Start any kind of Trema app

for instance : ./trema run ./objects/examples/learning_switch/learning_switch

Show switch description

TREMA_HOME=`pwd` ../apps/show_description/show_description
And then we can get the information as follows:

        Manufacturer description: Stanford University
        Hardware description: Reference Userspace Switch
        Software description: 1.0.0
        Serial number: None
        Human readable description of datapath: OpenWrt pid=1933
        Datapath ID: 0x11
        Port no: 1(0x1)(Port up)
          Hardware address: 90:f6:52:89:c9:d0
          Port name: eth0.1
        Port no: 2(0x2)(Port up)
          Hardware address: 90:f6:52:89:c9:d0
          Port name: eth0.2
        Port no: 3(0x3)(Port up)
          Hardware address: 90:f6:52:89:c9:d0
          Port name: eth0.3
        Port no: 4(0x4)(Port up)
          Hardware address: 90:f6:52:89:c9:d0
          Port name: eth0.4

Wednesday, June 27, 2012

[OpenFlow] Summary of some current OpenFlow Related Articles

1. http://blog.ioshints.info/2011/11/openflow-enterprise-use-cases.html
This article discusses enterprise use cases in OpenFlow

There are four functions you can easily implement with OpenFlow (Tony Bourke wrote about them in more details)

packet filters – flow classifier followed by a drop or normal action
policy based routing – flow classifier followed by outgoing interface and/or VLAN tag push
static routes – flow classifiers using only destination IP prefix
NAT – some OpenFlow switches might support source/destination IP address/port rewrites.

It suggests that security could be the killer app as the following URL:

http://www.rationalsurvivability.com/blog/2011/10/the-killer-app-for-openflow-and-sdn-security/

2. http://routerjockey.com/2011/11/02/nec-and-programmableflow-switching/
This article writer give some information and comments about NEC programmableflow because he joined NEC presenting at Networking Tech Field Day 2

3. http://blog.ioshints.info/2011/11/openflow-deployment-models.html#of_native
This article provides four different models for OpenFlow deployment have already emerged:

Native OpenFlow

The controller performs all control-plane functions, including running control-plane protocols with the outside world.
This model has at least two serious drawbacks even if we ignore the load placed on the controller by periodic control-plane protocols:

The switches need IP connectivity to the controller for the OpenFlow control session.
Fast control loops like BFD are hard to implement with a central controller, more so if you want to have very fast response time.

NEC seems to be using this model quite successfully (although they probably have a few extensions), but already encountered inherent limitations: a single controller can control up to ~50 switches, ~600000 end points (VMs or Physical Nodes), and rerouting around failed links takes around 200 msec (depending on the network size). For more details, watch their Networking Tech Field Day presentation.

Native OpenFlow with extensions

A switch controlled entirely by the OpenFlow controller could perform some of the low-level control-plane functions independently.
Using OpenFlow extensions or functionality implemented locally on the switch, you destroy the mirage of the “OpenFlow networking nirvana”-- smart open-source programmable controllers control dumb low-cost switches, busting the “networking = mainframes” model and bringing the Linux-like golden age to every network.

Ships in the night

Switches have traditional control plane; OpenFlow controller manages only certain ports or VLANs on trunked links. The local control plane (or linecards) can perform the tedious periodic tasks like running LACP, LLDP and BFD, passing only the link status to the OpenFlow controller.

Integrated OpenFlow

OpenFlow classifiers and forwarding entries are integrated with the traditional control plane. For example, Juniper’s OpenFlow implementation inserts compatible flow entries (those that contain only destination IP address matching) as ephemeral static routes into RIB (Routing Information Base)
From my perspective, this approach makes most sense: don’t rip-and-replace the existing network with a totally new control plane, but augment the existing well-known mechanisms with functionality that’s currently hard (or impossible) to implement.

Monday, June 18, 2012

[OpenFlow] Wildcard Explanation

This article is about flow wildcard for match field. Basically, we can get understood most of them at a glance. But, for NW_SRC_MASK and NW_DST_MASK they need to do more a little bit math. I only give an example with NW_SRC_MASK because NW_DST_MASK is similar case. Please refer to the following picture:

The position of NW_SRC_MASK is from 8 to 13. If we want to setup a IP subnet mask as 192.168.1.0/24, we should give the value: 001000 (8 bits are wirdcarded). Another example, for instance, 192.168.0.0/16 (16 bits are wirdcarded), the value should be 010000.

/* Flow wildcards. */   
enum ofp_flow_wildcards {   
    OFPFW_IN_PORT = 1 << 0, /* Switch input port. */
    OFPFW_DL_VLAN = 1 << 1, /* VLAN id. */
    OFPFW_DL_SRC = 1 << 2, /* Ethernet source address. */
    OFPFW_DL_DST = 1 << 3, /* Ethernet destination address. */
    OFPFW_DL_TYPE = 1 << 4, /* Ethernet frame type. */
    OFPFW_NW_PROTO = 1 << 5, /* IP protocol. */
    OFPFW_TP_SRC = 1 << 6, /* TCP/UDP source port. */
    OFPFW_TP_DST = 1 << 7, /* TCP/UDP destination port. */
    /* IP source address wildcard bit count. 0 is exact match, 1 ignores the
    * LSB, 2 ignores the 2 least-significant bits, ..., 32 and higher wildcard
    * the entire field. This is the *opposite* of the usual convention where
    * e.g. /24 indicates that 8 bits (not 24 bits) are wildcarded. */
    OFPFW_NW_SRC_SHIFT = 8,
    OFPFW_NW_SRC_BITS = 6,
    OFPFW_NW_SRC_MASK = ((1 << OFPFW_NW_SRC_BITS) - 1) << OFPFW_NW_SRC_SHIFT,
    OFPFW_NW_SRC_ALL = 32 << OFPFW_NW_SRC_SHIFT,
    /* IP destination address wildcard bit count. Same format as source. */
    OFPFW_NW_DST_SHIFT = 14,
    OFPFW_NW_DST_BITS = 6,
    OFPFW_NW_DST_MASK = ((1 << OFPFW_NW_DST_BITS) - 1) << OFPFW_NW_DST_SHIFT,
    OFPFW_NW_DST_ALL = 32 << OFPFW_NW_DST_SHIFT,
    OFPFW_DL_VLAN_PCP = 1 << 20, /* VLAN priority. */
    OFPFW_NW_TOS = 1 << 21, /* IP ToS (DSCP field, 6 bits). */
    /* Wildcard all fields. */
    OFPFW_ALL = ((1 << 22) - 1)
};  

Friday, June 15, 2012

[Summary] Data Center Network Issues

http://muratbuffalo.blogspot.tw/2010/11/vl2-scalable-and-flexible-data-center.html
The key points from my point of view in this article for Data Center :

Agility:

Without agility, each service must pre-allocate enough servers to meet difficult to predict demand spikes, or risk failure at the brink of success. With agility, the data center operator can meet the fluctuating demands of individual services from a large shared server pool, resulting in higher server utilization and lower costs. In order to achieve agility, assigning servers to a service should be independent of network topology.

Downtime Issue:

Still, downtimes can be significant, and with no obvious way to eliminate all failures from the top of the hierarchy, this paper's approach is to broaden (fatten) the topmost levels of the network so that the impact of failures is muted and performance degrades gracefully.

Data center traffic:

The paper proposes to use valiant load balancing (vlb) to randomize end-to-end communication paths to cope with volatility and achieve load balancing. In this scheme, the ToR switch randomly chooses an intermediate switch (among many available options) on a per flow basis.

This paper provide an approach of Clos topology.

Wednesday, June 13, 2012

[Demo] OpenFlow GUI demo

Here is a video about OpenFlow GUI demo on a simple topology.
I modified the flow animation so that the flow animation is different from the original one.
Check it out.
http://youtu.be/8r93qgt7VVE

Monday, June 11, 2012

[Tutorial] An example of using juju to deploy cloud services

Before you get started with juju, please refer to the official documens
https://juju.ubuntu.com/docs/getting-started.html
https://juju.ubuntu.com/docs/user-tutorial.html

P.S: When you lunch an instance, be careful about the instance type. Except t1.micro, others will charge you money~~
I strongly suggest to add parameter after juju bootstrap and deploy command as follows:

--constraints "instance-type=m1.micro"

or execute this command:
> juju set-constraints instance-type=t1.micro
For more info about this, please check out this document:
https://juju.ubuntu.com/docs/constraints.html

My environment is using EC2.
> vi .juju/environment.yaml

environments:
  sample:
    type: ec2
    access-key: << your access key >>
    secret-key: << your secret key >>
    control-bucket: juju-0f3b4bce2d944893a74967016c98b903
    admin-secret: 0d748130374946babe1f2531d77620d0
    default-series: precise
    ssl-hostname-verification: true

When you prepare your .juju/environment.yaml ready, you are able to do the following steps to try juju:
> juju bootstrap
> juju deploy wordpress
> juju deploy mysql
> juju add-relation mysql wordpress
> juju expose wordpress

After executing above commands, we can use "juju status" to see what we have now on Amazon EC2:
> juju status
2012-06-11 09:00:26,558 INFO Connecting to environment...
2012-06-11 09:00:43,657 INFO Connected to environment.
machines:
0:
    agent-state: running
    dns-name: ec2-23-22-111-234.compute-1.amazonaws.com
    instance-id: i-30fa4a49
    instance-state: running
1:
    agent-state: running
    dns-name: ec2-50-17-117-72.compute-1.amazonaws.com
    instance-id: i-b055eac9
    instance-state: running
2:
    agent-state: running
    dns-name: ec2-23-22-205-88.compute-1.amazonaws.com
    instance-id: i-de52eda7
    instance-state: running
services:
mysql:
    charm: cs:precise/mysql-2
    relations:
      db:
      - wordpress
    units:
      mysql/0:
        agent-state: started
        machine: 2
        public-address: ec2-23-22-205-88.compute-1.amazonaws.com
wordpress:
    charm: cs:precise/wordpress-1
    exposed: true
    relations:
      db:
      - mysql
    units:
      wordpress/0:
        agent-state: started
        machine: 1
        open-ports:
        - 80/tcp
        public-address: ec2-50-17-117-72.compute-1.amazonaws.com
2012-06-11 09:01:18,809 INFO 'status' command finished successfully

From now on, we can check EC2 dashboard and see what instances are there ( it will be the same as the result of "juju status")

Because we do "expose" for wordpress, we have a public address ( ec2-50-17-117-72.compute-1.amazonaws.com) and it looks like as below:

Once you are done with an juju deployment, you need to terminate all running instances in order to stop paying for them.
> juju destroy-environment
WARNING: this command will destroy the 'sample' environment (type: ec2).
This includes all machines, services, data, and other resources. Continue [y/N]y
2012-06-11 09:53:03,181 INFO Destroying environment 'sample' (type: ec2)...
2012-06-11 09:53:10,018 INFO Waiting on 3 EC2 instances to transition to terminated state, this may take a while
2012-06-11 09:53:53,257 INFO 'destroy_environment' command finished successfully

Tuesday, June 5, 2012

[How to] do trouble shooting with LLDP setting on Switch

Let me assume if there is a simple topology here

                              +------------------+
                      Switch  |  172.17.255.254  |
                              +-----+------------+
              +-----------------+   |
      Switch  |   172.17.4.1    |<--+
              +----------+------+
 +--------------+        |
 | Your Server: |        |
 | 172.17.2.200 |<-------+
 +--------------+

A. Make sure Switch and Switch could ping each other

telnet 172.17.255.254
ping 172.17.4.1

B. Make sure that every port should have "management address" checked.

C. Use snmpwalk to check 172.17.255.254 has remote ip address of 172.17.4.1

exp: snmpwalk -c public -v2c 172.17.255.254 1.0.8802.1.1.2.1.4.2.1.3
if we cannot see the result as follows, it means that the LLDP configuration setting on Switch 172.17.255.254 is wrong.

Result: iso.0.8802.1.1.2.1.4.2.1.3.0.4.55.1.4.172.17.4.1 = INTEGER: 2

D. Use snmpwalk to check 172.17.4.1if it has remote ip address of 172.17.255.254

exp: snmpwalk -c public -v2c 172.17.4.1 1.0.8802.1.1.2.1.4.2.1.3
if we cannot see the result as follows, it means that the LLDP configuration setting on Switch 172.17.4.1 is wrong.

Saturday, May 26, 2012

[Qt] memory management and implicit sharing

There are some points that we have to know when we program using Qt:

The ownership of all child QObjects is transferred to the parent.
- Automatic deletion by the parent
- Allocated from the heap (using new)
- manual deletion is not necessary but it won't cause any problems.
All QObjects without a parent must be deleted manually.
Pay attention to ownership and responsibilities. Qt does not provide a garbage collection.

Only if you have Classes with QObjects, then child items will be deleted if the parent is deleted:

QObject *parent = new QObject; 

QWidget *child1 = new QWidget(parent);
QPushButton *child2 = new QPushButton(parent);
delete parent; // child1 and child2 will be deleted automatically!

Implicit sharing (IS)
The following example is about how implicit sharing works when test() returns QList object (on stack). This memory address of "a" is shared to "result" and "a" is not destroyed when method:test() is out of scope.

#include "mainwindow.h" #include "ui_mainwindow.h" #include <QList> #include <QtGui> QList<QString> MainWindow::test() { QList<QString> a; // on stack QList<QString> b = QList<QString>(); // on stack QList<QString> *c = new QList<QString>(); // on heap for (int i = 0; i < 10; i++) { /* if we append a reference of QString, it will consume the memory of the QList Object based on the size of string */ a.append("a_helo:"); a.append(QString::number(i)); b.append("b_helo:"); b.append(QString::number(i)); c->append("c_helo:"); c->append(QString::number(i)); } qDebug() << a << " addres:" << &a; qDebug() << b << " addres:" << &b; qDebug() << *c << " addres:" << c; //delete &b; // we cannot manually free the memory of b or c on stack delete c; // OK return a; // Implicit Sharing } MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent), ui(new Ui::MainWindow) { ui->setupUi(this); /* The memory address of result is same with a object in test() because of implicit sharing */ QList<QString> result = test(); QList<QString> result2; qDebug() << result << " addres:" << &result; qDebug() << result2 << " addres:" << &result2; } MainWindow::~MainWindow() { delete ui; }

Monday, May 14, 2012

[Explanation][Trema] The message mechanism in flow_manager app

I give 2 pictures to illustrate the message mechanism in flow_manager app.

1. Request/Reply message using send_request_message() and send_reply_message()

2. (Received) Message using send_message()

Thursday, May 10, 2012

[Open vSwitch] Simply test ovs-ofctl command on GNS3 simulation network

This article is to test ovs-ofctl command on GNS3 simulation network. PC1 and PC2 are virtual machine on Qemu. U_OVS is emulated with Open vSwitch and an OpenFlow Controller is also on it. Please see the following picture:

1. For how to build the following GNS3 environment, please refer to the URL:
http://brezular.wordpress.com/2011/06/25/part2-openvswich-vlans-trunks-l3-vlan-interface-intervlan-routing-configuration-and-testing/
2. Run > sudo ovs-ofctl show br0

OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:000008002725cd53
n_tables:1, n_buffers:256
features: capabilities:0x87, actions:0xfff
1(eth1): addr:08:00:27:95:ef:0b
     config:     0
     state:      LINK_DOWN
     current:    COPPER AUTO_NEG
     advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
     supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
2(eth3): addr:08:00:27:25:cd:53
     config:     0
     state:      0
     current:    1GB-FD COPPER AUTO_NEG
     advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
     supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
3(eth2): addr:08:00:27:c3:5f:90
     config:     0
     state:      0
     current:    1GB-FD COPPER AUTO_NEG
     advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
     supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
LOCAL(br0): addr:08:00:27:25:cd:53
     config:     PORT_DOWN
     state:      LINK_DOWN
OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0

3. After PC1 and PC2 ping each other, run > sudo ovs-ofctl dump-flows br0
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=17.125s, table=0, n_packets=14, n_bytes=1372, idle_timeout=5,priority=65535,icmp,in_port=2,vlan_tci=0x0000,dl_src=00:aa:00:3c:24:00,dl_dst=00:aa:00:4d:1d:00,nw_src=192.168.1.2,nw_dst=192.168.1.1,nw_tos=0,icmp_type=8,icmp_code=0 actions=output:3
cookie=0x0, duration=18.13s, table=0, n_packets=15, n_bytes=1470, idle_timeout=5,priority=65535,icmp,in_port=3,vlan_tci=0x0000,dl_src=00:aa:00:4d:1d:00,dl_dst=00:aa:00:3c:24:00,nw_src=192.168.1.1,nw_dst=192.168.1.2,nw_tos=0,icmp_type=0,icmp_code=0 actions=output:2

p.s:
the script of PC1 configuration
Login is tc without password set.
Assign IP address 192.168.1.1/24 to eth0 and make it persistent after next reboot of Microcore
sudo hostname PC1
sudo ifconfig eth0 192.168.1.1 netmask 255.255.255.0
echo "hostname PC1" >> /opt/bootlocal.sh
echo "ifconfig eth0 192.168.1.1 netmask 255.255.255.0" >> /opt/bootlocal.sh
/usr/bin/filetool.sh -b

Monday, May 7, 2012

[Explanation][Trema] The Register Event in Routing Switch App

If we want to understand how Routing Switch App works in Trema, we have to know the event register first. Here is a simple explanation about the register event in Routing Switch App, which is related Topology Ap.

Wednesday, May 2, 2012

[How to] Find a key word or Replace it with other string on Linux

Please refer to these examples:

Find the key word "ProcessType=6" in text files under current path (included sub folder)
> find ./ -name "*.txt" -exec grep -n -H "ProcessType=6" {} \;

Replace the key word "python2.4" with "python2.6" in python files under /opt/stack (included sub folder)
> find /opt/stack -name "*.py" -exec sed -i 's/python2.4/python2.6/' '{}' \;

Based on the script above, you will be able to find or replace a key word quickly and efficiently.

Sunday, April 29, 2012

[How to] Install Qt SDK on Ubuntu

1. Download Qt SDK
wget http://www.developer.nokia.com/dp?uri=http%3A%2F%2Fsw.nokia.com%2Fid%2F14b2039c-0e1f-4774-a4f2-9aa60b6d5313%2FQt_SDK_Lin64_offlinehttp://www.developer.nokia.com/dp?uri=http%3A%2F%2Fsw.nokia.com%2Fid%2F14b2039c-0e1f-4774-a4f2-9aa60b6d5313%2FQt_SDK_Lin64_offline

2. Change the file's mode for execution and run it.
> chmod u+x Qt_SDK_Lin64_offline_v1_2_en.run ( e.g )
> ./Qt_SDK_Lin64_offline_v1_2_en.run ( e.g. )

3. Install g++ compiler
> sudo apt-get install g++

4. If the error message ( Gtk-WARNING **: Unable to locate theme engine in module_path: "pixmap" ) shows when running Qt Creator, you need to run:
> sudo apt-get install gtk2-engines-pixbuf

[How to] Install Boost library on Ubuntu

> sudo apt-get install libboost-all-dev
That's it.

And also, there are plenty of examples as follows:
http://www.boost.org/doc/libs/1_38_0/doc/html/boost_asio/examples.html

Tuesday, April 24, 2012

Look for a Qt library for network topology

Qanava could be my solution for network topology based on Qt4. But, it had been not maintained anymore...

main page:
http://gna.org/projects/qanava

git repository
http://repo.or.cz/w/qanava.git

Document
Qanava Manual v0.1.0

Monday, April 23, 2012

[How to] Use tcpdump to look at LLDP packet

Here is an example about how to use tcpdump tool to look at LLDP information

> sudo tcpdump -c 1 -lv -v -i eth0 -a -e -s 1514 ether proto 0x88cc

The result:
tcpdump: listening on eth4, link-type EN10MB (Ethernet), capture size 1514 bytes
13:30:46.426056 08:00:27:09:61:e9 (oui Unknown) > 01:80:c2:00:00:0e (oui Unknown), ethertype LLDP (0x88cc), length 156: LLDP, length 142
    Chassis ID TLV (1), length 7
    Subtype MAC address (4): 08:00:27:09:61:e9 (oui Unknown)
    0x0000: 0408 0027 0961 e9
    Port ID TLV (2), length 7
    Subtype MAC address (3): 08:00:27:09:61:e9 (oui Unknown)
    0x0000: 0308 0027 0961 e9
    Time to Live TLV (3), length 2: TTL 120s
    0x0000: 0078
    System Name TLV (5), length 15: Ubuntu-Devstack
    0x0000: 5562 756e 7475 2d44 6576 7374 6163 6b
    System Description TLV (6), length 43
    Ubuntu 11.10\0x0a Linux 3.0.0-14-generic x86_64
    0x0000: 5562 756e 7475 2031 312e 3130 0a20 4c69
    0x0010: 6e75 7820 332e 302e 302d 3134 2d67 656e
    0x0020: 6572 6963 2078 3836 5f36 34
    System Capabilities TLV (7), length 4
    System Capabilities [Bridge, WLAN AP, Router] (0x001c)
    Enabled Capabilities [Bridge, Router] (0x0014)
    0x0000: 001c 0014
    Management Address TLV (8), length 12
    Management Address length 5, AFI IPv4 (1): Ubuntu-Devstack.local
    Interface Index Interface Numbering (2): 5
    0x0000: 0501 c0a8 7a01 0200 0000 0500
    Port Description TLV (4), length 4: eth4
    0x0000: 6574 6834
    Organization specific TLV (127), length 9: OUI IEEE 802.3 Private (0x00120f)
    Link aggregation Subtype (3)
        aggregation status [supported], aggregation port ID 0
    0x0000: 0012 0f03 0100 0000 00
    Organization specific TLV (127), length 9: OUI IEEE 802.3 Private (0x00120f)
    MAC/PHY configuration/status Subtype (1)
        autonegotiation [supported, enabled] (0x03)
        PMD autoneg capability [10BASE-T hdx, 10BASE-T fdx, 100BASE-TX hdx, 100BASE-TX fdx, 1000BASE-T fdx] (0x6c01)
        MAU type 1000BASET fdx (0x001e)
    0x0000: 0012 0f01 036c 0100 1e
    Organization specific TLV (127), length 6: OUI IEEE 802.3 Private (0x00120f)
    Max frame size Subtype (4)
        MTU size 0
    0x0000: 0012 0f04 0000
    End TLV (0), length 0

The following is the argument list for reference:
-a 　　　將網絡地址和廣播地址轉變成名字
-d 　　　將匹配信息包的代碼以人們能夠理解的彙編格式給出
-dd 　　將匹配信息包的代碼以c語言程序段的格式給出
-ddd 　　將匹配信息包的代碼以十進制的形式給出
-e 　　　在輸出行打印出數據鏈路層的頭部信息
-f 　　　將外部的Internet地址以數字的形式打印出來
-l 　　　使標準輸出變為緩衝行形式( line buffered mode )
-n 　　　不把網絡地址轉換成名字
-t 　　　在輸出的每一行不打印時間戳
-v 　　　輸出一個稍微詳細的信息，例如在ip包中可以包括ttl和服務類型的信息
-vv 　　輸出詳細的報文信息
-c 　　　在收到指定的包的數目後，tcpdump就會停止
-F 　　　從指定的文件中讀取表達式,忽略其它的表達式
-i 　　　指定監聽的網絡接口
-r 　　　從指定的文件中讀取包(這些包一般通過-w選項產生)
-w 　　　直接將包寫入文件中，並不分析和打印出來
-T 　　　將監聽到的包直接解釋為指定的類型的報文，常見的類型有rpc （遠程過程調用）和snmp（簡單網絡管理協議；）

Wednesday, April 18, 2012

[ZeroMQ] The new solution for building up distributed system

I have used TIBCO Rendezvous (RV Message), ActiveMQ (JMS), and RabbitMQ(AMQP) before. They all have message broker, which means all the messages will be send to broker (centralized control) first, and then the client or node will receives them later. But, ZeroMQ has totally different story. I personally believe that it could be a very good solution for building up distributed system.

http://www.zeromq.org/

And also, this article gives the short introduction and summarizes the important items including "communication transport", "End Point Implementation", "The Socket Object", and so on about ZeroMQ. This content is very uesful for beginner to get to know and read first.
http://www.coastrd.com/zeromq-messaging

[Python] Make pyc file for your Python source code

It could be a situation when you have to give your Python program to customers, but you don't want to give them Python source code. Here is a solution. Give them Python byte code!

For example, on command line:

Compile one file

> python -c "import compileall; compileall.compile_file('YourPythonFile.py')"

Compile one folder

         > python -c "import compileall; compileall.compile_dir('YourFolder/', force=True)"

Or, you can put the compiling action in source code

      import compileall
      compileall.compile_dir('YourFolder/', force=True)

Tuesday, April 17, 2012

[Quantum] An example of OpenStack Quantum's table schema and data

Monday, April 16, 2012

[sFlow] sFlow Agent and sFlow Collector

sFlow is a technology for monitoring network, wireless and host devices. Based on the following topology, there is a test about using sFlow Agent and sFlow Collector to observe the sFlow data from Open vSwitch.

sFlow agent is from Open vSwitch

How to setup sFlow on Open vSwitch:
> sudo ovs-vsctl -- --id=@s create sFlow agent=eth0 target=\"colletor ip:6343\" header=128 sampling=64 polling=10 -- set Bridge br0 sflow=@s

sFlow collector is sFlowTrend

Now, we can see the results from sFlowTrend:

When trying to ping PC1 and PC2, the result is below:

[Comparsion] Topology graph in NOX GUI and OpenFlow GUI

Using GNS3 for constructing an emulation network environment, I give a simple topology as follows to try running NOX GUI and OpenFlow GUI.
P.S: for more information in GNS3, please refer to this:
http://www.gns3.net/gns3-virtualbox-edition/

In NOX GUI, the topology graph is created like this:

In OpenFlow GUI, the topology graph is created like this:

Saturday, April 14, 2012

[Trema] Try Trema App: flow_dumper

1. Choose "routing_switch" as my testing app. (any other simple switch app is all available...)
> ./trema run -c ../apps/routing_switch/routing_switch.conf

2. Send packets between these 2 hosts
> ./trema send_packets -s host2 -d host1
> ./trema send_packets -s host1 -d host2

3. Try to run Trema App: "flow_dumper" as follows
> TREMA_HOME=. ../apps/flow_dumper/flow_dumper
[0x000000000000e0] priority = 65535, match = [wildcards = 0, in_port = 1, dl_src = 00:00:00:01:00:02, dl_dst = 00:00:00:01:00:01, dl_vlan = 65535, dl_vlan_pcp = 0, dl_type = 0x800, nw_tos = 0, nw_proto = 17, nw_src = 192.168.0.2/32, nw_dst = 192.168.0.1/32, tp_src = 1, tp_dst = 1], actions = [output: port=2 max_len=65535]
[0x000000000000e0] priority = 65535, match = [wildcards = 0, in_port = 2, dl_src = 00:00:00:01:00:01, dl_dst = 00:00:00:01:00:02, dl_vlan = 65535, dl_vlan_pcp = 0, dl_type = 0x800, nw_tos = 0, nw_proto = 17, nw_src = 192.168.0.1/32, nw_dst = 192.168.0.2/32, tp_src = 1, tp_dst = 1], actions = [output: port=1 max_len=65535]

4. So, we can see all the flows here.

Wednesday, April 11, 2012

[Tutorial] How to setup QoS on Open vSwitch

There are two ways to do that:

1. Interface Rate Limiting ( on Interface )

For instance:

> sudo ovs-vsctl set Interface eth1 ingress_policing_rate=10000
> sudo ovs-vsctl set Interface eth1 ingress_policing_burst=1000

2. Port QoS Policy ( on Port )

For instance:

         > sudo ovs-vsctl set port eth1 qos=@newqos \
             -- --id=@newqos create qos type=linux-htb \
             other-config:max-rate=200000000 queues=0=@q0,1=@q1 \
             -- --id=@q0 create queue \
             other-config:min-rate=100000000 \
             other-config:max-rate=100000000 \
              -- --id=@q1 create queue \
             other-config:min-rate=50000000 \
             other-config:max-rate=50000000

Qos can have more than 1 queue

Tuesday, April 10, 2012

[Tutorial] Trema Tutorial

http://www.fp7-ofelia.eu/assets/Uploads/201203xx-TremaTutorial.pdf

[Open vSwitch] How to get port statistics from interface in OVS

1. Show the bridge info
> sudo ovs-vsctl show
result:
2909bfce-536e-4184-a5bb-507f0553abee
    Bridge "br0"
        Controller "tcp:10.6.186.244"
        Port "br0"
            Interface "br0"
                type: internal
        Port "eth3"
            Interface "eth3"
        Port "eth2"
            Interface "eth2"
        Port "eth1"

2. Get port statistics from interface
> sudo ovs-vsctl get Interface br0 statistics
result:

{collisions=0,

rx_bytes=0,

rx_crc_err=0,

rx_dropped=0,

rx_errors=0,

rx_frame_err=0,

rx_over_err=0,

rx_packets=0,

tx_bytes=0,

tx_dropped=0,

tx_errors=0,

tx_packets=0}

Monday, April 9, 2012

[Python] How to use Decorators for Functions and Methods

When list_ports() is executed, Class ApiCall will be created for becoming a decorator to examine the arguments as follows:

import sys

class ApiCall(object):
    """A Decorator to add support for format and tenant overriding"""
    def __init__(self, function):
        self.function = function

    def __get__(self, instance, owner):
        def with_params(*args, **kwargs):
            """
            Temporarily sets the format and tenant for this request
            """
            (format, tenant) = (instance.format, instance.tenant)

            if 'format' in kwargs:
                instance.format = kwargs['format']
            if 'tenant' in kwargs:
                instance.tenant = kwargs['tenant']

            ret = self.function(instance, *args)
            (instance.format, instance.tenant) = (format, tenant)
            return ret
        return with_params


class Client(object):
    def __init__(self, tenant=None, format="xml"):
        self.tenant = tenant
        self.format = format

    @ApiCall
    def list_ports(self, network):
        """
        Fetches a list of ports on a given network
        """
        return network
    
def main():
    client = Client(tenant="AAA",format="xml")
    client.list_ports('my network')
    sys.exit(0)
    
if __name__ == "__main__":
    main()

[Tutorial][Trema] Show Topology

Trema有一個很不錯的App, 可以利用toplogy與 topology_discovery App, 去產生topology graph…如下:

下列為Network emulation的設定檔內容

# virtual switches

vswitch("switch1") { datapath_id "0x1" }

vswitch("switch2") { datapath_id "0x2" }

vswitch("switch3") { datapath_id "0x3" }

vswitch("switch4") { datapath_id "0x4" }

vswitch("switch4") { datapath_id "0x5" }

# virtual hosts

vhost("host1")

vhost("host2")

vhost("host3")

vhost("host4")

# virtual links

link "switch1", "switch2"

link "switch1", "switch3"

link "switch2", "switch3"

link "switch2", "switch4"

link "switch2", "switch5"

link "switch3", "switch4"
link "switch3", "switch5"

link "switch4", "switch5"

link "switch4", "host1"

link "switch4", "host2"

link "switch5", "host3"

link "switch5", "host4"

Wednesday, April 4, 2012

[Tutorial] ovsdbmonitor GUI 設定步驟 (setup procedure)

The following steps are the setup procedure for running ovsdbmonitor GUI

(You have to make sure that your "PYTHONPATH" contains the path of openvswitch source path/python. )

1. 增加PYTHONPATH到 .bashrc

for example:
PYTHONPATH="${PYTHONPATH}:/home/liudanny/Source/openvswitch-1.2.2/python/"
Export PYTHONPATH

2. 增加一個symbolic link for ovsdb-client
sudo ln -s /usr/local/bin/ovsdb-client /usr/bin/ovsdb-client

3. 執行ovsdbmonitor
“your openswitch path”/ovsdb/ovsdbmonitor/ovsdbmonitor

4. 設定Host Properties:
Host name or IP: your openvswitch host
SSH Password: ***
Connect target: unix:/usr/local/var/run/openvswitch/db.sock
＊這個connect target 會與啟動ovsdb-server daemon 的參數有相關＊

sudo ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \

--remote=db:Open_vSwitch,manager_options \

--private-key=db:SSL,private_key \

--certificate=db:SSL,certificate \

--bootstrap-ca-cert=db:SSL,ca_cert \

--pidfile --detach

5. 相關畫面 (可以看到ovs database 內的table content, 每個頁簽都代表一個table)