Posts

Showing posts from September, 2012

[JSON] How to use jansson lib to generate JSON data in C

This exmple method will generate the result string below:

{ "command":"link_status",
  "result":["AAABBBCCC"] }


char*generate_json_data(){char*ret_strings=NULL;char*ret_string="AAABBBCCC";json_t*root=json_object();json_t*result_json_arr=json_array();json_object_set_new(root,"command",json_string("link_status"));json_object_set_new(root,"result",result_json_arr);...json_array_append(result_json_arr,json_string(ret_string));free(ret_string);ret_strings=json_dumps(root,0);json_decref(root);returnret_strings;}

For more example in details:
#include <stdio.h> #include <jansson.h> voidadd_2array_to_json(json_t*obj,constchar*name,constint*marr,size_tdim1,size_tdim2){size_ti,j;json_t*jarr1=json_array();for(i=0;i<dim1;++i){json_t*jarr2=json_array();for(j=0;j<dim2;++j){intval=marr[i*dim2+j];json_t*jval=json_integer(val);json_array_append_new(jarr2,jval);}json_array_append_new(jarr1,jarr2);}

[epoll] How to use epoll in socket program?

When we do socket program, the most popular way is of PPC/TPC mode ( Process Per Connection / Thread Per Connection ). But the mode has some drawbacks, for instance, it will consum too much memory and CPU for context switch when creating more than hundreds of connections. epoll is good solution to deal with that.

The following URL is a good explaintion about epoll.
How to use epoll? A complete example in C

And here is a complete examle of using epoll in socket program.
#include <stdio.h>#include <stdlib.h>#include <errno.h>#include <string.h>#include <unistd.h>#include <fcntl.h>#include <sys/types.h>#include <arpa/inet.h>#include <netinet/in.h>#include <sys/socket.h>#include <sys/epoll.h>#include <sys/wait.h>#define SERVPORT 9527 /* Server Port Number */#define BACKLOG 10 /* The Max of Client Number */voidsetnonblocking(int sock) { int opts; opts = fcntl(sock,F_GETFL); if(opts<0) { perror…

[Socket] How to use he function fcntl()

When serveris runningtoaccept()andnoclient connectionrequest comes, what it happens? Whentheserverstopswaitingforthearrivaloftheconnectionrequestonaccept(). Similarly, whentheprogramisrunningtoreceive(), ifnodatacan be read, the programwill alsostopreceivingstatements. This situationiscalledblocking.
If you wanttoonlypay attention to checkwhether there arecustomerswaiting for a connectionservertoacceptconnection, otherwise the program continuestodo other things, throughtheSocketsetfornon-blockingway: non-blockingsocketwhennocustomerswaiting foracceptcallsreturn immediately.

# include <unistd.h>
# include <fcntl.h>
....
sockfd = socket (AF_INET, SOCK_STREAM, 0);
fcntl (sockfd, F_SETFL, O_NONBLOCK);
....

Set socketto non-blockingmode, "polling" the Socket. Whennodatais waiting to be processedfrom anon-blockingSocket( for readdata ), thefunctionwillreturn immediately, andthe return valueis set to -1anderrnosetof EWOULDBLOCK. 
But this"polling" causes theCPUis busyw…

[Storage] Some articles of NoSQL and Object Storage ( S3 and Swift )

Severl months ago, I saw the first article "LINE Starage" and didn't pay too much attention on it. Recently Someone discussed storage with me and let me remember it. So I found it again ( took me a bunch of time to look for ) and also took some pretty nice related articles here as well.
LINE Storage: Storing billions of rows in Sharded-Redis and HBase per Monthhttp://tech.naver.jp/blog/?p=1420 NoSQL Data Modeling Techniqueshttp://highlyscalable.wordpress.com/2012/03/01/nosql-data-modeling-techniques/

Cassandra vs MongoDB vs CouchDB vs Redis vs Riak vs HBase vs Membase vs Neo4j comparisonhttp://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

Hadoop + S3http://wiki.apache.org/hadoop/AmazonS3

Apache Hadoop over OpenStack Swifthttp://bigdatacraft.com/archives/349


[Sqlite] A simple example of Sqlite in C

Here is a simple example of how to deal with Sqlite Database in C language.


#include <stdio.h> #include <sqlite3.h> staticchar*createsql="CREATE TABLE Employee(""ID INTEGER PRIMARY KEY,""Name VARCHAR(10),""BadgeID VARCHAR(10));";staticchar*insertsql="INSERT INTO Employee VALUES(NULL, 'Danny', '12345');";staticchar*querysql="SELECT * FROM Employee;";voidmain(void){introws,cols;sqlite3*db;char*errMsg=NULL;char**result;/* Open database file */if(sqlite3_open("my_example.db3",&db,SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE,NULL)){return;}/* Build Table */sqlite3_exec(db,createsql,0,0,&errMsg);/* Add a new record */sqlite3_exec(db,insertsql,0,0,&errMsg);/* Get the last insert record's ID */printf("%d\n",sqlite3_last_insert_rowid(db));/* Get all records in database */sqlite3_get_table(db,querysql,&result,&rows,&cols,&errMsg);/* List all the data */for(i=0;…

[tun/tap] An introduction of TUN/TAP

http://www.kernel.org/pub/linux/kernel/people/marcelo/linux-2.4/Documentation/networking/tuntap.txt

TUN/TAP provides packet reception and transmission for user space programs. It can be viewed as a simple Point-to-Point or Ethernet device, which instead of receiving packets from a physical media, receives them from user space program and instead of sending packets via physical media writes them to the user space program. When a program opens /dev/net/tun, driver creates and registers corresponding net device tunX or tapX. After a program closed above devices, driver will automatically delete tunX or tapX device and all routes corresponding to it.And also, here is a web site about Universal TUN/TAP device driver Frequently Asked Question.http://vtun.sourceforge.net/tun/faq.htmlThe description of Universal TUN/TAP device driverhttps://www.kernel.org/doc/Documentation/networking/tuntap.txt

[SLB] Server Load Balancing with DSR

IPVS ( based on LVS) provides 3 kind of ways to do server load balancing:

Virtual Server via Network Address Translation(VS/NAT)Virtual Server via IP Tunneling(VS/TUN)Virtual Server via Direct Routing(VS/DR) Recently I find another way to do load balancing, and it is Direct Server Return
http://techiess.wordpress.com/2010/09/09/load-balancing-dsr-direct-server-return/
Using DSR, packet is returning directly from the server to client without going through L4 switch.

[Virtual Network] Midokura’s MidoNet

Recently I saw the article about Midokura MidoNet as follows:
http://blog.ioshints.info/2012/08/midokuras-midonet-layer-2-4-virtual.html
It is a layer 2-4 virtual network solution and catches my attention and lets me want to know more about.

Meanwhile, Midokura also provide Quantum Plugin for MidoNet to integrate into OpenStack.
https://github.com/midokura/midonet-openstack

On its web site, it provides a lot of features in virtual network solution and is pretty amazing. Due to MidoNet is not an open source project, so we are not able to see how it exactly achieves all of these things.
http://www.midokura.com/midonet/openstack/
MidoNet Key FeaturesFully virtualized Layer 2 through 7 networkingVLAN-less VLANs – Virtual L2 Distributed Level Isolation and Switching with virtually no limitationsFully Distributed Architecture with no single points of failureVirtual L3 Distributed RoutingDistributed Load Balancing and Firewall ServicesNATVPNAccess Control Lists (ACLs)Restful APIWeb Based Manag…

[OpenStack] Existing High Availability Options for Networking

Image
Existing High Availability Options for Networking

    OpenStack offical web site has document to explain the existing HA options for networking. I think most of important item is the option 1: multi-host networking mode. This mode avoids from single point of failure when Nova-Network component is down and also eliminate the potential network bottleneck in single-host networking mode.

For more in details, please see the this:
http://docs.openstack.org/trunk/openstack-compute/admin/content/existing-ha-networking-options.html