Tuesday, December 3, 2013

[NSX] Network Visualization Gets Physical

In VMware NSX, how to let network visualization gets physical?
The answer is "Layer 2 Gateway Services". The physical switches have to implement the OVSDB and related APIs that communicate with NSX Controller so that they can provide the mapping of the segment in virtual network and VLAN in physical network. For more information in details, you can check out the following URLs.

http://networkheresy.com/2013/08/15/network-virtualization-gets-physical/


https://www.youtube.com/watch?v=QDOlggwyrVA&feature=c4-overview-vl&list=PLdYldEmmLm2lz5Bd0bzGCDKSULL52ytJT

Ivan's NSX Architecture Introduction, it is very good.
http://demo.ipspace.net/get/NSXArch

Tuesday, November 19, 2013

[OpenVNet] The introduction of OpenVNet

Why I pay attention to OpenVNet is because it uses Trema-Edge as OpenFlow 1.3 Controller to communicate with Open vSwitch 1.10. It gives us a great example to use Trema-Edge in Virtual Network Environment and learn more about some use cases. Please check out the following lists:

What is OpenVNet?
http://www.slideshare.net/akirayokokawa/openvnet

Offical Web Site:
http://openvnet.com/

Github Source:
https://github.com/axsh/openvnet

[Quagga] How to compile and install Quagga on Ubuntu 12.04

The following steps are the compilation and installation for Quagga on Ubuntu 12.04. Hopefully it is helpful for you.
  • Install all dependency(package) 
    • sudo apt-get build-dep quagga
  • Give the path for all configurations files and libraries used by quagga for starting. I use this one /opt/quagga .... and this is done by this comand: 
    • ./configure --enable-vtysh --prefix=/opt/quagga --localstatedir=/opt/quagga sysconfdir=/opt/quagga 
  • Create new folder: 
    • sudo mkdir /opt/quagga
  • Add new user : 
    • sudo adduser quagga 
  • Give the priviledge for the user quagga over the folder: /opt/quagga and for the folders and subfolders you give the privilegde for all rights : reading, changing and executing
    • sudo chown quagga:quagga /opt/quagga
    • sudo chmod 777 /opt/quagga 
  • Then you will be able to comiple Quagga: 
    • make
    • sudo make install 
Quagga configuration
--------------------
quagga version          : 0.99.22
host operating system   : linux-gnu
source code location    : .
compiler                : gcc
compiler flags          : -Os -fno-omit-frame-pointer -g -std=gnu99 -Wall -Wsign-compare -Wpointer-arith -Wbad-function-cast -Wwrite-strings -Wmissing-prototypes -Wmissing-declarations -Wchar-subscripts -Wcast-qual
make                    : make
includes                : 
linker flags            :  -lcrypt   -lrt -lcap  -lm
state file directory    : /opt/quagga
config file directory   : /opt/quagga
example directory       : /opt/quagga
user to run as  : quagga
group to run as  : quagga
group for vty sockets : 
config file mask        : 0600
log file mask           : 0600

Now, let us try Quagga for some examples
  • Zebra
    • Copy sample to conf file:
      • cd /opt/quagga
      • cp zebra.conf.sample zebra.conf
    • Start zebra daemon, you use this command 
      • sudo /opt/quagga/sbin/zebra &
    • Check the daemon zebra:
      • netstat -a | grep zebra 
    • Get into the zebra, you use 
      • telnet localhost zebra
  • BGP
    • Copy sample to conf file:
      • cd /opt/quagga
      • cp bgpd.conf.sample bgpd.conf
    • Start bgpd: 
      • sudo /opt/quagga/sbin/bgpd &
    • Check the daemon zebra:
      • netstat -a | grep bgpd
    • Get into the bgpd, you use 
      • telnet localhost bgpd
  • vtysh
    • cd /opt/quagga
    • vi /opt/quagga/vtysh.conf
    • sudo /opt/quagga/bin/vtysh
! Sample configuration file for vtysh. ! !service integrated-vtysh-config hostname quagga-router username root nopassword !
Stop Quagga Daemon ( for example: bgpd ):
  • sudo kill `cat /opt/quagga/bgpd.pid`
Port Number:

  • zebra: 2601
  • ripd:  2602
  • ripng: 2603
  • ospfd: 2604
  • bgpd:  2605
  • ospf6d: 2606



Friday, November 8, 2013

[Puppet] The studying track of Puppet

After taking some time to study Puppet a while, I become to believe more and more that it is a great auto configuration tool. The following URLs are my studying track of Puppet.

For the beginner, it is recommended to walk through for learning the basic concept
Learning Puppet
http://docs.puppetlabs.com/learning/index.html

Once you finish the above document, you probably want to do your own type and provider. Here you go:
Customer Type
http://docs.puppetlabs.com/guides/custom_types.html
Provider Development
http://docs.puppetlabs.com/guides/provider_development.html

Juniper provides its open source project for using Puppet to manage and configure the switches that support Puppet Agent. It is a very good example for you to understand how to define customer type and to implement provider.

To define customer type:
https://github.com/NetdevOps/puppet-netdev-stdlib
To implement provider:
https://github.com/Juniper/puppet-netdev-stdlib-junos

Monday, November 4, 2013

[OpenFlow 1.X] The Flow Table Usage

When OpenFlow 1.0 guys try to reach the version 1.1 or more, the first question coming up with would most likely be "how to use multi-flow tables ?" Well, we could see an example in RYU OpenFlow Controller as follow: http://www.slideshare.net/yamahata/ryu-sdnframeworkupload
In page 33, there are 3 flow tables which contains match conditons and actions. It can give an initial idea for that question.






P.S: Pica8 works with Broadcom to double flow table size in its OF1.3 Switch
http://searchsdn.techtarget.com/news/2240214709/Pica8-doubles-flow-rule-capacity-in-its-new-OpenFlow-13-switch

Wednesday, October 30, 2013

[Trema] A good example with Trema to build OpenFlow Controller ( use case )

Previously I just wondered how to use Trema to build a useful and piratical solution with a good architecture for users to leverage its functionality. Well, it does exist. The following URL is a good example of a good use case containing a good architecture.
http://www.slideshare.net/chibayasunobu/developing-production-open-flow-controller-with-trema-201304160

Source Code:
https://github.com/trema/virtual-network-platform


Design:


Implementation:

Friday, October 25, 2013

[Network] EtherTypes Reference

802 EtherTypes Reference:

0x0000
-
0x05DC
IEEE 802.3 length.
0x0600XEROX NS IDP.
0x0660
0x0661
DLOG.
0x0800IP, Internet Protocol.
0x0801X.75 Internet.
0x0802NBS Internet.
0x0803ECMA Internet.
0x0804Chaosnet.
0x0805X.25 Level 3.
0x0806ARP, Address Resolution Protocol.
0x0807XNS compatability.
0x0808Frame Relay ARP.
0x8035DRARP, Dynamic RARP.
RARP, Reverse Address Resolution Protocol.
0x80F3AARP, AppleTalk Address Resolution Protocol.
0x8100EAPS, Ethernet Automatic Protection Switching.
0x8137IPX, Internet Packet Exchange.
0x814CSNMP, Simple Network Management Protocol.
0x86DDIPv6, Internet Protocol version 6.
0x8808MPCP, Multi-Point Control Protocol.
0x880BPPP, Point-to-Point Protocol.
0x880CGSMP, General Switch Management Protocol.
0x8847MPLS, Multi-Protocol Label Switching (unicast).
0x8848MPLS, Multi-Protocol Label Switching (multicast).
0x8863PPPoE, PPP Over Ethernet (Discovery Stage).
0x8864PPPoE, PPP Over Ethernet (PPP Session Stage).
0x886FNetwork Load Balancing.
0x888EEAPOL, EAP over LAN.
0x88A2AoE, ATA over Ethernet.
0x88A4EtherCAT.
0x88CATIPC, Transparent Inter Process Communication Protocol.
0x88BBLWAPP, Light Weight Access Point Protocol.
0x88CCLLDP, Link Layer Discovery Protocol.
0x88DCWSMP, WAVE S

Monday, October 21, 2013

[Static Analysis] Tools for static analysis ( C/C++ )

Here are the list of some software tools for static analysis ( C/C++ ). FYI.
  • Klocwork Insight ( Commercial )
  • Cppcheck
  • LintProject Pro
  • scan-build

Thursday, October 17, 2013

[dbus] The notes for DBus APIs

The previous post: [dbus] The useful information and example of D-Bus has introduced the concept and an example. This post is focused on its API.

DBusConnection:
http://dbus.freedesktop.org/doc/api/html/group__DBusConnection.html#details
The keypoint that is worth mentioning is about dealing with asynchronous message. 
The function dbus_connection_read_write_dispatch() for example does all three of these things, offering a simple alternative to a main loop. If you don't need to be asynchronous, you can ignore DBusWatchDBusTimeout, and dbus_connection_dispatch(). Instead, dbus_connection_read_write_dispatch() can be used.

Here is an example of how to use dbus watch and dbus timeout:
http://stackoverflow.com/questions/9378593/dbuswatch-and-dbustimeout-examples
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <signal.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/socket.h>
#include <arpa/inet.h>

#include <dbus/dbus.h>

struct dbus_ctx {
    DBusConnection *conn;
    struct event_base *evbase;
    struct event dispatch_ev;
    void *extra;
};

static void dispatch(int fd, short ev, void *x)
{
    struct dbus_ctx *ctx = x;
    DBusConnection *c = ctx->conn;

    logger(LOG_DEBUG "dispatching\n");

    while (dbus_connection_get_dispatch_status(c) == DBUS_DISPATCH_DATA_REMAINS)
        dbus_connection_dispatch(c);
}

static void handle_dispatch_status(DBusConnection *c,
                                   DBusDispatchStatus status, void *data)
{
    struct dbus_ctx *ctx = data;

    logger(LOG_DEBUG "new dbus dispatch status: %d\n", status);

    if (status == DBUS_DISPATCH_DATA_REMAINS) {
        struct timeval tv = {
            .tv_sec = 0,
            .tv_usec = 0,
        };
        event_add(&ctx->dispatch_ev, &tv);
    }
}

static void handle_watch(int fd, short events, void *x)
{
    struct dbus_ctx *ctx = x;
    struct DBusWatch *watch = ctx->extra;

    unsigned int flags = 0;
    if (events & EV_READ)
        flags |= DBUS_WATCH_READABLE;
    if (events & EV_WRITE)
        flags |= DBUS_WATCH_WRITABLE;
    /*if (events & HUP)
        flags |= DBUS_WATCH_HANGUP;
    if (events & ERR)
        flags |= DBUS_WATCH_ERROR;*/

    logger(LOG_DEBUG "got dbus watch event fd=%d watch=%p ev=%d\n",
           fd, watch, events);
    if (dbus_watch_handle(watch, flags) == FALSE)
        logger(LOG_ERROR "dbus_watch_handle() failed\n");

    handle_dispatch_status(ctx->conn, DBUS_DISPATCH_DATA_REMAINS, ctx);
}

static dbus_bool_t add_watch(DBusWatch *w, void *data)
{
    if (!dbus_watch_get_enabled(w))
        return TRUE;

    struct dbus_ctx *ctx = data;
    ctx->extra = w;

    int fd = dbus_watch_get_unix_fd(w);
    unsigned int flags = dbus_watch_get_flags(w);
    short cond = EV_PERSIST;
    if (flags & DBUS_WATCH_READABLE)
        cond |= EV_READ;
    if (flags & DBUS_WATCH_WRITABLE)
        cond |= EV_WRITE;

    struct event *event = event_new(ctx->evbase, fd, cond, handle_watch, ctx);
    if (!event)
        return FALSE;

    event_add(event, NULL);

    dbus_watch_set_data(w, event, NULL);

    logger(LOG_DEBUG "added dbus watch fd=%d watch=%p cond=%d\n", fd, w, cond);
    return TRUE;
}

static void remove_watch(DBusWatch *w, void *data)
{
    struct event *event = dbus_watch_get_data(w);

    if (event)
        event_free(event);

    dbus_watch_set_data(w, NULL, NULL);

    logger(LOG_DEBUG "removed dbus watch watch=%p\n", w);
}

static void toggle_watch(DBusWatch *w, void *data)
{
    logger(LOG_DEBUG "toggling dbus watch watch=%p\n", w);

    if (dbus_watch_get_enabled(w))
        add_watch(w, data);
    else
        remove_watch(w, data);
}

static void handle_timeout(int fd, short ev, void *x)
{
    struct dbus_ctx *ctx = x;
    DBusTimeout *t = ctx->extra;

    logger(LOG_DEBUG "got dbus handle timeout event %p\n", t);

    dbus_timeout_handle(t);
}

static dbus_bool_t add_timeout(DBusTimeout *t, void *data)
{
    struct dbus_ctx *ctx = data;

    if (!dbus_timeout_get_enabled(t))
        return TRUE;

    logger(LOG_DEBUG "adding timeout %p\n", t);

    struct event *event = event_new(ctx->evbase, -1, EV_TIMEOUT|EV_PERSIST,
                                    handle_timeout, t);
    if (!event) {
        logger(LOG_ERROR "failed to allocate new event for timeout\n");
        return FALSE;
    }

    int ms = dbus_timeout_get_interval(t);
    struct timeval tv = {
        .tv_sec = ms / 1000,
        .tv_usec = (ms % 1000) * 1000,
    };
    event_add(event, &tv);

    dbus_timeout_set_data(t, event, NULL);

    return TRUE;
}

static void remove_timeout(DBusTimeout *t, void *data)
{
    struct event *event = dbus_timeout_get_data(t);

    logger(LOG_DEBUG "removing timeout %p\n", t);

    event_free(event);

    dbus_timeout_set_data(t, NULL, NULL);
}

static void toggle_timeout(DBusTimeout *t, void *data)
{
    logger(LOG_DEBUG "toggling timeout %p\n", t);

    if (dbus_timeout_get_enabled(t))
        add_timeout(t, data);
    else
        remove_timeout(t, data);
}

static DBusHandlerResult handle_nameownerchanged(DBusMessage *message,
                                                 void *data)
{
    struct dbus_ctx *ctx = data;
    char *name, *old, *new;
    if (dbus_message_get_args(message, NULL,
                              DBUS_TYPE_STRING, &name,
                              DBUS_TYPE_STRING, &old,
                              DBUS_TYPE_STRING, &new,
                              DBUS_TYPE_INVALID) == FALSE) {
        logger(LOG_ERROR "spurious NameOwnerChanged signal\n");
        return DBUS_HANDLER_RESULT_NOT_YET_HANDLED;
    }
    logger(LOG_DEBUG "dbus NameOwnerChanged %s -> %s\n", old, new);

    if (new[0] != '\0')
        return DBUS_HANDLER_RESULT_NOT_YET_HANDLED;

    /* XXX handle disconnecting clients */

    return DBUS_HANDLER_RESULT_NOT_YET_HANDLED;
}

static DBusHandlerResult msg_filter(DBusConnection *connection,
                                    DBusMessage *message, void *data)
{
    if (dbus_message_is_signal(message, DBUS_INTERFACE_DBUS,
                               "NameOwnerChanged"))
        return handle_nameownerchanged(message, data);

    logger(LOG_DEBUG "got dbus message %d %s -> %s %s/%s/%s %s\n",
           dbus_message_get_type(message),
           dbus_message_get_sender(message),
           dbus_message_get_destination(message),
           dbus_message_get_path(message),
           dbus_message_get_interface(message),
           dbus_message_get_member(message),
           dbus_message_get_type(message) == DBUS_MESSAGE_TYPE_ERROR ?
           dbus_message_get_error_name(message) : "");

    return DBUS_HANDLER_RESULT_NOT_YET_HANDLED;
}

static void unregister_func(DBusConnection *connection, void *data)
{
}

static DBusHandlerResult message_func(DBusConnection *connection,
                                      DBusMessage *message, void *data)
{
    struct dbus_ctx *ctx = data;

    logger(LOG_DEBUG "got dbus message sent to %s %s %s\n",
           dbus_message_get_destination(message),
           dbus_message_get_interface(message),
           dbus_message_get_path(message));

    /* XXX handle DBus message */

    return DBUS_HANDLER_RESULT_HANDLED;
}

static DBusObjectPathVTable dbus_vtable = {
    .unregister_function = unregister_func,
    .message_function = message_func,
};

struct dbus_ctx *dbus_init(struct event_base *eb)
{
    DBusConnection *conn = NULL;
    struct dbus_ctx *ctx = calloc(1, sizeof(struct dbus_ctx));
    if (!ctx) {
        logger_perror("can't allocate dbus_ctx\n");
        goto out;
    }

    conn = dbus_bus_get_private(DBUS_BUS_SESSION, NULL);
    if (conn == NULL) {
        logger(LOG_ERROR "failed to get bus\n");
        goto out;
    }

    dbus_connection_set_exit_on_disconnect(conn, FALSE);

    ctx->conn = conn;
    ctx->evbase = eb;
    event_assign(&ctx->dispatch_ev, eb, -1, EV_TIMEOUT, dispatch, ctx);

    if (!dbus_connection_set_watch_functions(conn, add_watch, remove_watch,
                                             toggle_watch, ctx, NULL)) {
        logger(LOG_ERROR "dbus_connection_set_watch_functions() failed\n");
        goto out;
    }

    if (!dbus_connection_set_timeout_functions(conn, add_timeout,
                                               remove_timeout, toggle_timeout,
                                               ctx, NULL)) {
        logger(LOG_ERROR "dbus_connection_set_timeout_functions() failed\n");
        goto out;
    }

    if (dbus_connection_add_filter(conn, msg_filter, ctx, NULL) == FALSE) {
        logger(LOG_ERROR "dbus_connection_add_filter() failed\n");
        goto out;
    }

    dbus_connection_set_dispatch_status_function(conn, handle_dispatch_status,
                                                 ctx, NULL);

    char match[256];
    snprintf(match,
             sizeof(match),
             "type='signal',interface='%s',member='NameOwnerChanged'",
             DBUS_INTERFACE_DBUS);
    DBusError error;
    dbus_error_init(&error);
    dbus_bus_add_match(conn, match, &error);
    if (dbus_error_is_set(&error)) {
        logger(LOG_ERROR "dbus_bus_add_match() %s failed: %s\n",
               "NameOwnerChanged", error.message);
        dbus_error_free(&error);
        goto out;
    }

    snprintf(match,
             sizeof(match),
             "type='signal',interface='%s',member='%s'",
             GNP_IPC_INTERFACE, GNP_IPC_SIGNAL_DELIVER_SA);
    dbus_error_init(&error);
    dbus_bus_add_match(conn, match, &error);
    if (dbus_error_is_set(&error)) {
        logger(LOG_ERROR "dbus_bus_add_match() %s failed: %s\n",
               GNP_IPC_SIGNAL_DELIVER_SA, error.message);
        dbus_error_free(&error);
        goto out;
    }

    if (dbus_connection_register_object_path(conn, GNP_IPC_PATH, &dbus_vtable,
                                             ctx) != TRUE) {
        logger(LOG_ERROR "failed to register object path\n");
        goto out;
    }

    return ctx;

out:
    if (conn) {
        dbus_connection_close(conn);
        dbus_connection_unref(conn);
    }
    if (ctx)
        free(ctx);
    return NULL;
}

void dbus_close(struct dbus_ctx *ctx)
{
    if (ctx && ctx->conn) {
        dbus_connection_flush(ctx->conn);
        dbus_connection_close(ctx->conn);
        dbus_connection_unref(ctx->conn);
        event_del(&ctx->dispatch_ev);
    }
    if (ctx)
        free(ctx);
}

DBusMessage:
http://dbus.freedesktop.org/doc/api/html/group__DBusMessage.html
This module is to deal with messages.

DBus Message Bus API:
http://dbus.freedesktop.org/doc/api/html/group__DBusBus.html
This module is the only one in libdbus that's specific to communicating with the message bus daemon. The rest of the API can also be used for connecting to another application directly.

DBusPendingCall API:
http://dbus.freedesktop.org/doc/api/html/group__DBusPendingCall.html


An example to look over names on the message (bus driver) by using proxy object:
http://stackoverflow.com/questions/14263390/how-to-compile-a-basic-d-bus-glib-example
#include <stdlib.h>          // for exit()   
#include <dbus/dbus.h>       // for dbus_*   
#include <dbus/dbus-glib.h>  // for dbus_g_*

int
main (int argc, char **argv)
{
  DBusGConnection *connection;
  GError *error;
  DBusGProxy *proxy;
  char **name_list;
  char **name_list_ptr;

  g_type_init ();

  error = NULL;
  connection = dbus_g_bus_get (DBUS_BUS_SESSION,
                           &error);
  if (connection == NULL)
    {
      g_printerr ("Failed to open connection to bus: %s\n",
              error->message);
      g_error_free (error);
      exit (1);
    }

  /* Create a proxy object for the "bus driver" (name "org.freedesktop.DBus") */

  proxy = dbus_g_proxy_new_for_name (connection,
                                 DBUS_SERVICE_DBUS,
                                 DBUS_PATH_DBUS,
                                 DBUS_INTERFACE_DBUS);

  /* Call ListNames method, wait for reply */
  error = NULL;
  if (!dbus_g_proxy_call (proxy, "ListNames", &error, G_TYPE_INVALID,
                      G_TYPE_STRV, &name_list, G_TYPE_INVALID))
    {
      /* Just do demonstrate remote exceptions versus regular GError */
      if (error->domain == DBUS_GERROR && error->code == DBUS_GERROR_REMOTE_EXCEPTION)
    g_printerr ("Caught remote method exception %s: %s",
            dbus_g_error_get_name (error),
            error->message);
      else
    g_printerr ("Error: %s\n", error->message);
      g_error_free (error);
      exit (1);
    }

  /* Print the results */

  g_print ("Names on the message bus:\n");

  for (name_list_ptr = name_list; *name_list_ptr; name_list_ptr++)
    {
      g_print ("  %s\n", *name_list_ptr);
    }
  g_strfreev (name_list);

  g_object_unref (proxy);

  return 0;
}

The example of dbus_connection_add_filter() and handler method:
http://lists.freedesktop.org/archives/dbus/2003-September/000468.html