[IOMMU] The error of "VFIO group is not viable"

As my previous article mentioned, DPDK is a library to accelerate packet transmission between user space application and physical network device. Recently there is an article "Using Open vSwitch 2.4 with DPDK-2.2.0 for Inter-VM NFV Applications" that aspires me to do the similiar test environment to witness how powerful DPDK is.
But, I enounter the error of "VFIO group is not viable" when I start openvswitch daemon. After searching the information on Google, it is related with IOMMU Group issue as described below:

http://vfio.blogspot.tw/2014/08/vfiovga-faq.html
Question 1:

I get the following error when attempting to start the guest:

vfio: error, group $GROUP is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.

Answer:

There are more devices in the IOMMU group than you're assigning, they all need to be bound to the vfio bus driver (vfio-pci) or pci-stub for the group to be viable.  See my previous post about IOMMU groups for more information.  To reduce the size of the IOMMU group, install the device into a different slot, try a platform that has better isolation support, or (at your own risk) bypass ACS using the ACS override patch.

So, the solutions are these two:
1. Install the device into a different slot
2. Bypass ACS using the ACS overrides patch

For the more explanation in details, you could see this article:
http://vfio.blogspot.tw/2014/08/iommu-groups-inside-and-out.html


The following Scripts are the summary from the article:

# Update Grub
GRUB_CMDLINE_LINUX_DEFAULT="... default_hugepagesz=1G hugepagesz=1G hugepages=16 hugepagesz=2M hugepages=2048 iommu=pt intel_iommu=on isolcpus=1-13,15-27"
update-grub

# Export Variables
export OVS_DIR=/root/soucecode/ovs
export DPDK_DIR=/root/soucecode/dpdk-2.1.0
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
locale-gen "en_US.UTF-8"
dpkg-reconfigure locales

# build DPDK
curl -O http://dpdk.org/browse/dpdk/snapshot/dpdk-2.1.0.tar.gz
tar -xvzf dpdk-2.1.0.tar.gz
cd $DPDK_DIR
sed 's/CONFIG_RTE_BUILD_COMBINE_LIBS=n/CONFIG_RTE_BUILD_COMBINE_LIBS=y/' -i config/common_linuxapp
make install T=x86_64-ivshmem-linuxapp-gcc
cd $DPDK_DIR/x86_64-ivshmem-linuxapp-gcc
EXTRA_CFLAGS="-g -Ofast" make -j10

# build OVS
git clone https://github.com/openvswitch/ovs.git
cd $OVS_DIR
./boot.sh
./configure --with-dpdk="$DPDK_DIR/x86_64-ivshmem-linuxapp-gcc/" CFLAGS="-g -Ofast"
make 'CFLAGS=-g -Ofast -march=native' -j10

# Setup OVS
pkill -9 ovs
rm -rf /usr/local/var/run/openvswitch
rm -rf /usr/local/etc/openvswitch/
rm -f /usr/local/etc/openvswitch/conf.db
mkdir -p /usr/local/etc/openvswitch
mkdir -p /usr/local/var/run/openvswitch
cd $OVS_DIR
./ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db ./vswitchd/vswitch.ovsschema
./ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach
./utilities/ovs-vsctl --no-wait init

# Check IOMMU & Huge info
cat /proc/cmdline
grep Huge /proc/meminfo

# Setup Hugetable
mkdir -p /mnt/huge
mkdir -p /mnt/huge_2mb
mount -t hugetlbfs hugetlbfs /mnt/huge
mount -t hugetlbfs none /mnt/huge_2mb -o pagesize=2MB
 
# Bind Ether card
modprobe vfio-pci
cd $DPDK_DIR/tools
./dpdk_nic_bind.py --status
./dpdk_nic_bind.py --bind=vfio-pci 08:00.2

# Start OVS
modprobe openvswitch
$OVS_DIR/vswitchd/ovs-vswitchd --dpdk -c 0x2 -n 4 --socket-mem 2048 -- unix:/usr/local/var/run/openvswitch/db.sock --pidfile --detach

# Setup dpdk vhost user port
$OVS_DIR/utilities/ovs-vsctl show
$OVS_DIR/utilities/ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
$OVS_DIR/utilities/ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
$OVS_DIR/utilities/ovs-vsctl add-port br0 vhost-user1 -- set Interface vhost-user1 type=dpdkvhostuser
$OVS_DIR/utilities/ovs-vsctl add-port br0 vhost-user2 -- set Interface vhost-user2 type=dpdkvhostuser

find /sys/kernel/iommu_groups/ -type l

# Create VMs
qemu-system-x86_64 -m 1024 -smp 4 -cpu host -hda /tmp/vm1.qcow2 -boot c -enable-kvm -no-reboot -nographic -net none \
-chardev socket,id=char1,path=/usr/local/var/run/openvswitch/vhost-user1 \
-netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce \
-device virtio-net-pci,mac=00:00:00:00:dd:01,netdev=mynet1 \
-object memory-backend-file,id=mem,size=1024M,mem-path=/dev/hugepages,share=on \
-numa node,memdev=mem -mem-prealloc
 
qemu-system-x86_64 -m 1024 -smp 4 -cpu host -hda /tmp/vm2.qcow2 -boot c -enable-kvm -no-reboot -nographic -net none \
-chardev socket,id=char1,path=/usr/local/var/run/openvswitch/vhost-user2 \
-netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce \
-device virtio-net-pci,mac=00:00:00:00:dd:02,netdev=mynet1 \
-object memory-backend-file,id=mem,size=1024M,mem-path=/dev/hugepages,share=on \
-numa node,memdev=mem -mem-prealloc

[Django] How to upload a file to via Django REST Framework?

Here is an example to use Parser: MultiPartParser to parse the media type: multipart/form-data. You also can take a look at this for more in detail: http://www.django-rest-framework.org/api-guide/parsers

Client Side via curl: ( Basically you can use either PUT or POST)
Command format:
curl -X PUT -H 'Content-Type:multipart/form-data' -F 'data=@/{your_file}' -u {account}:{password} http://{your server ip address}/

for instance:
curl -X PUT -H 'Content-Type:multipart/form-data' -F 'data=@/Users/administrator/Desktop/2015.12.01.yaml' -u danny:password http://192.168.1.100:8000/api/upload_func/file/



Server side ( Django REST Framework )

in your settings.py, add the following lines
==>
    'DEFAULT_PARSER_CLASSES': (
        'rest_framework.parsers.MultiPartParser',
    )


For your_api.py, you can refer to this example as below:

class UploadAction(APIView):
    permission_classes = (IsAuthenticated, IsAdminOrReadOnly)
    parser_classes = (MultiPartParser,)

    def put(self, request, format=None):
        try:
            # Read uploaded files
            my_file = request.FILES['data']
            filename = "/tmp/" + str(my_file)
            with open(filename, 'wb+') as temp_file:
                for chunk in my_file.chunks():
                    temp_file.write(chunk)
                temp_file.close()
        except Exception as e:
            return Response(status=status.HTTP_500_INTERNAL_SERVER_ERROR)
        return Response(status=status.HTTP_201_CREATED)


P.S: Please change 'data' to others if you put the different variable name in curl command!!

Reference:
http://stackoverflow.com/questions/21012538/retrieve-json-from-request-files-in-django-without-writing-to-file

If you want to get the content of uploaded file, you can directly use the. read() api.

Something like:

if request.FILES.has_key('data'):
    file = request.Files['data']
    data = file.read()
    #you have file contents in data




[Virtual Switching] What are macvlan and macvtap?

I excerpt the content of the following URL about what macvlan and macvtap are for reference quickly.
https://www.netdev01.org/docs/netdev_tutorial_bridge_makita_150213.pdf

macvlan
VLAN using not 802.1Q tag but mac address
    • 4 types of mode
        • private
        • vepa
        • bridge
        • passthru
Using unicast filtering if supported, instead of promiscuous mode (except for passthru)
    • Unicast filtering allows

NIC to receive multiple mac addresses
    • Light weight bridge
    • No source learning
    • No STP
Only one uplink
Allow traffic between macvlans (via macvlan stack)


macvtap
with KVM has three mode
    Private
    vepa
    bridge
tap-like macvlan variant
    • packet reception
        -> file read
    • file write
        -> packet transmission



Further reading:
http://140.120.15.179/Presentation/20150203/
Virtual networking: TUN/TAP, MacVLAN, and MacVTap

[GitHub] How to rebase your branch from upstream branch?

If you fork a copy (branch) from a repository and you want to rebase your branch with this upstream repository, you can refer to the following URLs or directly see the content below.

https://help.github.com/articles/configuring-a-remote-for-a-fork/
https://help.github.com/articles/syncing-a-fork/

1. List the current configured remote repository for your fork.

   git remote -v
   # origin  https://github.com/YOUR_USERNAME/YOUR_FORK.git (fetch)
   # origin  https://github.com/YOUR_USERNAME/YOUR_FORK.git (push)
   

2. Specify a new remote upstream repository that will be synced with the fork.

   git remote add upstream https://github.com/ORIGINAL_OWNER/ORIGINAL_REPOSITORY.git
   

3. Verify the new upstream repository you've specified for your fork.

   git remote -v
   # origin    https://github.com/YOUR_USERNAME/YOUR_FORK.git (fetch)
   # origin    https://github.com/YOUR_USERNAME/YOUR_FORK.git (push)
   # upstream  https://github.com/ORIGINAL_OWNER/ORIGINAL_REPOSITORY.git (fetch)
   # upstream  https://github.com/ORIGINAL_OWNER/ORIGINAL_REPOSITORY.git (push)
   

1. Fetch the branches and their respective commits from the upstream repository. Commits tomaster will be stored in a local branch, upstream/master.

   git fetch upstream
   # remote: Counting objects: 75, done.
   # remote: Compressing objects: 100% (53/53), done.
   # remote: Total 62 (delta 27), reused 44 (delta 9)
   # Unpacking objects: 100% (62/62), done.
   # From https://github.com/ORIGINAL_OWNER/ORIGINAL_REPOSITORY
   #  * [new branch]      master     -> upstream/master
   

2. Check out your fork's local master branch.

   git checkout master
   # Switched to branch 'master'
   

3. Merge the changes from upstream/master into your local master branch. This brings your fork's master branch into sync with the upstream repository, without losing your local changes.

   git merge upstream/master
   # Updating a422352..5fdff0f
   # Fast-forward
   #  README                    |    9 -------
   #  README.md                 |    7 ++++++
   #  2 files changed, 7 insertions(+), 9 deletions(-)
   #  delete mode 100644 README
   #  create mode 100644 README.md
   

   If your local branch didn't have any unique commits, Git will instead perform a "fast-forward":

   git merge upstream/master
   # Updating 34e91da..16c56ad
   # Fast-forward
   #  README.md                 |    5 +++--
   #  1 file changed, 3 insertions(+), 2 deletions(-)

[Django] How to Find the Performance Bottlenecks in Your Django Views?

Do you want to find the performance bottlenecks in your Django Views? I recently found this article that introduces nice tools for us to check the performance issues in Django. How to do it? Please follow this URL:
http://djangotricks.blogspot.tw/2015/01/performance-bottlenecks-in-django-views.html

To setup line profiling, install line_profiler and django-devserver to you virtual environment:

(myproject_env)$ pip install line_profiler
(myproject_env)$ pip install django-devserver

Then make sure that you have the following settings in your settings.py or local_settings.py:

# settings.py
INSTALLED_APPS = (
    # ...
    'devserver',
)

MIDDLEWARE_CLASSES = (
    # ...
    'devserver.middleware.DevServerMiddleware',
)

DEVSERVER_MODULES = (
    'devserver.modules.sql.SQLRealTimeModule',
    'devserver.modules.sql.SQLSummaryModule',
    'devserver.modules.profile.ProfileSummaryModule',

    # Modules not enabled by default
    'devserver.modules.profile.LineProfilerModule',
)

DEVSERVER_AUTO_PROFILE = True  # profiles all views without the need of function decorator

I try these tools and get a bunch of analysis report from them, and I think they are useful and convenient. The following picture is one part of my result:

[Selenium] How to use xpath to define the locator in Selenium Library?

When I first time touch Selenium library in Robot Framework, the first question mark coming up in my mind is "How to use xpath to define the locator?"

 Here is an example. If I want to select the listbox that is near by my email td tag, and the HTML looks like the follows:

There could be several ways to do so. But, I like to use this way to achieve the action that I described below. "Select From List" is the standard keyword to do the action for selecting a list, but you have to give the locator. Based on my HTML code, the answer is this:

Another example, if I want to select the checkbox that is near by my email td tag, I can use the following way to do so:

Or, if I want to get some information in a table, what can I do for this?
OK, if I have the table's HTML code like this:

I can use "Get Table Cell" keyword with xpath //table[@class='your table's class name'] and the information in which row and column, then you can finish the task to get the data from the cell in the table. Here you go:


Reference:
http://stackoverflow.com/questions/247135/using-xpath-to-search-text-containing

[Python] How can I patch a Python decorator before it wraps a function?

As title, Python decorator could be a problem if you want to write some unit cases that test these functions which contains decorators because you cannot use Mock to patch your decorators. Why? Please refer to the following link to check out:

http://stackoverflow.com/questions/7667567/can-i-patch-a-python-decorator-before-it-wraps-a-function

Decorators are applied at function definition time. For most functions, this is when the module is loaded. (Functions that are defined in other functions have the decorator applied each time the enclosing function is called.)

So if you want to monkey-patch a decorator, what you need to do is:

1. Import the module that contains it
2. Define the mock decorator function
3. Set e.g. module.decorator = mymockdecorator
4. Import the module(s) that use the decorator, or use it in your own module

Let me give an exmple:

Here is a decorator function(user_passes_test) in the following file path

/usr/local/lib/python2.7/dist-packages/django/contrib/auth/decorators.py

If you have a function using this decorator and need to write the test case for it as follows:

@user_passes_test(filter_root,
                  login_url=settings.LOGIN_PAGE)
def my_dashboard_view(request):


Then, you can put your patch code in the beginning of the tests python file like this:


def mock_function(test_func, login_url=None, redirect_field_name=None):
    def wrapped_f(view_func):
        @wraps(view_func, assigned=available_attrs(view_func))
        def decorator_view(request, *args, **kwargs):
            return view_func(request, *args, **kwargs)
        return decorator_function
    return wrapped_f

from django.contrib.auth.decorators import user_passes_test
user_passes_test = mock_function


After doing so, the decorator:  user_passes_test() won't infect your test case any more~

[SSH] To examine the Linux PAM for login mechanism

Based on Linux PAM module, we can easily configure the related pam file and use the functionality that it provides. I just examine login authentication mechanism from PAM a little bit. The scenario is like this:
  I change the login authentication mechanism from key pair to password and to see what happens in /var/log/auth.log if I give the wrong password.

/etc/ssh/sshd_config

The value to change:

PubkeyAuthentication yes ==> no

PasswordAuthentication no ==> yes

$> service ssh restart

After restarting ssh daemon, I give the wrong password and then I can see the error message in the log file:

The following image is for reference if everything is correct.