[Ceilometer] To collect the bandwidth of Neutron L3 router

Ceilometer component in OpenStack is a great project for me to study further in depth I think. I am not going to introduce it because it's not the scope of this post. Instead, I only want to list some resources about Ceilometer to collect the bandwith/traffic accounting of Neutron L3 router.

From this offical document: https://wiki.openstack.org/wiki/Neutron/Metering/Bandwidth
we can know that the actual implementation to collect the bandwidth of Neutron L3 router is based on iptables.

We can use the neutron command to list the related metering rules and labels
$ neutron meter-label-list
$ neutron meter-label-rule-list

The "neutron-meter-agent" will collect the traffic accounting in the iptables chain and push to oslo-messaging. So the following command can show the traffic accounting in Neutron L3 router


root@sn1:~# nova --os-project-name mickey list


root@sn1:~# nova show bb1879bd-1202-4007-9376-41be30e07ae9


root@sn1:~# neutron --os-tenant-name mickey router-list


root@node-5:~# neutron meter-label-rule-list


root@node-5:~# neutron meter-label-rule-show d31f0b9e-8824-4857-a2dc-19f237723f0c


root@node-5:~# neutron meter-label-rule-show 1ae289ef-687e-4303-8036-9a7566dd5365


So, we need to find the metering labels: "neutron-meter-l-78675a84" and "neutron-meter-l-b88c5977"

root@cn3:~#  ip netns exec qrouter-b1741371-ee12-46a1-831b-d3b35429d7c8 iptables -nL -v -x



root@cn3:~# ip netns exec qrouter-b1741371-ee12-46a1-831b-d3b35429d7c8 iptables -t nat -S




Here is the example to query the metering data in Ceilometer
$ root@node-5:~# ceilometer statistics -m bandwidth -q "resource=b88c5977-4445-4f19-9c8f-3d92809f844e;timestamp>=2016-03-01T00:00:00" --period 86400



P.S:
The following article is to introduce "Traffic Accounting with Linux IPTables" which can make us more understand it.
http://www.catonmat.net/blog/traffic-accounting-with-iptables/
https://wiki.openstack.org/wiki/Neutron/Metering/Bandwidth

[iptables] some common examples of iptables rule

• Read all tables without DNS lookup
• > iptables -L -n
• Obtain the line number of the lines: 
•  > iptables -L -nv --line-numbers

• Read NAT table in list without DNS lookup
• > iptables -t nat -L -n

• Do NAT ( SNAT )
• > echo "1" > /proc/sys/net/ipv4/ip_forward
• > iptables-t nat -A POSTROUTING -s ${INSIDE_NETWORK}/${INSIDE_NETMASK} -o ${OUTSIDE_DEVICE} -j MASQUERADE
• or > iptables-t nat -A POSTROUTING -s ${INSIDE_NETWORK}/${INSIDE_NETMASK} -o ${OUTSIDE_DEVICE} -j SNAT --to ${TARGET_IP}

• Do DNAT 
• > iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to-destination 192.168.100.10:80

• Drop the packet which is from 192.168.2.20 to 192.168.1.100 with TCP port 80
• > iptables -A POSTROUTING -t nat -s 192.168.2.20 -d 192.168.1.100 -p TCP --dport 80 -j DROP

• Accept the packet which is from 192.168.100.0/24 and interface eth1
• > iptables -A INPUT -i eth1 -s 192.168.100.0/24 -j ACCEPT
•  Insert a logging rule between the last one which drops packet with iptables something like this would do the trick
• > iptables -I INPUT (next-to-the-last rule number) -j LOG --log-prefix "blocked packets : "

iptables [-AI 鏈名] [-io 網路介面] [-p 協定] \

> [-s 來源IP/網域] [-d 目標IP/網域] -j [ACCEPT|DROP|REJECT|LOG]

選項與參數：

-S：規則列表

-t：指定表格 ( nat / filter ) 不用t 則預設為 filter

-AI 鏈名：針對某的鏈進行規則的 "插入" 或 "累加"

    -A ：新增加一條規則，該規則增加在原本規則的最後面。例如原本已經有四條規則，

         使用 -A 就可以加上第五條規則！

    -I ：插入一條規則。如果沒有指定此規則的順序，預設是插入變成第一條規則。

         例如原本有四條規則，使用 -I 則該規則變成第一條，而原本四條變成 2~5 號

    鏈 ：有 INPUT, OUTPUT, FORWARD 等，此鏈名稱又與 -io 有關，請看底下。

-io 網路介面：設定封包進出的介面規範

    -i ：封包所進入的那個網路介面，例如 eth0, lo 等介面。需與 INPUT 鏈配合；

    -o ：封包所傳出的那個網路介面，需與 OUTPUT 鏈配合；

-p 協定：設定此規則適用於哪種封包格式

   主要的封包格式有： tcp, udp, icmp 及 all 。

-s 來源 IP/網域：設定此規則之封包的來源項目，可指定單純的 IP 或包括網域，例如：

   IP  ：192.168.0.100

   網域：192.168.0.0/24, 192.168.0.0/255.255.255.0 均可。

   若規範為『不許』時，則加上 ! 即可，例如：

   -s ! 192.168.100.0/24 表示不許 192.168.100.0/24 之封包來源；

-d 目標 IP/網域：同 -s ，只不過這裡指的是目標的 IP 或網域。

-j ：後面接動作，主要的動作有接受(ACCEPT)、丟棄(DROP)、拒絕(REJECT)及記錄(LOG)

iptables -L -n -v -x

iptables -N TRAFFIC_ACCT

iptables -I FORWARD -j TRAFFIC_ACCT

iptables -D FORWARD -j TRAFFIC_ACCT

iptables -X TRAFFIC_ACCT

iptables -A TRAFFIC_ACCT -p tcp

iptables -A TRAFFIC_ACCT -p udp

iptables -A TRAFFIC_ACCT -p icmp