Friday, February 21, 2020

[Tracing] Use BCC tools to do dynamic tracing in Linux user space

Here is a very simple example of using BCC to trace a program and get the data from the function's argument.

A Simple Test Code in C: "test.c"
#include <stdio.h>

// student structure
struct student {
  char id[15];
  char firstname[64];
  char lastname[64];
  float points;
};
// function declaration
void getDetail(struct student *);
void displayDetail(struct student *);

int main(void) {
  // student structure variable
  struct student std[1];
  // get student detail
  getDetail(std);
  // display student detail
  displayDetail(std);
  return 0;
}

// function definition
void getDetail(struct student *ptr) {
  int i;
  for (i = 0; i < 1; i++) {
    printf("Enter detail of student #%d\n", (i + 1));
    printf("Enter ID: ");
    scanf("%s", ptr->id);
    printf("Enter first name: ");
    scanf("%s", ptr->firstname);
    printf("Enter last name: ");
    scanf("%s", ptr->lastname);
    printf("Enter Points: ");
    scanf("%f", &ptr->points);
    // update pointer to point at next element
    // of the array std
    ptr++;
  }
}

void displayDetail(struct student *ptr) {
  int i;
  for (i = 0; i < 1; i++) {
    printf("\nDetail of student #%d\n", (i + 1));
    // display result via ptr variable
    printf("\nResult via ptr\n");
    printf("ID: %s\n", ptr->id);
    printf("First Name: %s\n", ptr->firstname);
    printf("Last Name: %s\n", ptr->lastname);
    printf("Points: %f\n", ptr->points);
    // update pointer to point at next element
    // of the array std
    ptr++;
  }
}

The simple BCC example code: "mytrace.py"
It will get and print out the string of firstname when it runs
from __future__ import print_function
from bcc import BPF
from time import strftime
import argparse


# load BPF program
bpf_text = """
#include <uapi/linux/ptrace.h>
#include <linux/sched.h>
struct str_t {
    u64 pid;
    char str[80];
};

// student structure
struct student {
  char id[15];
  char firstname[64];
  char lastname[64];
  float points;
};

BPF_PERF_OUTPUT(events);
int printret(struct pt_regs *ctx) {
    struct str_t data  = {};
    char comm[TASK_COMM_LEN] = {};
    u32 pid;
    if (!PT_REGS_RC(ctx))
        return 0;
    pid = bpf_get_current_pid_tgid();
    data.pid = pid;
    bpf_probe_read(&data.str, sizeof(data.str),
        ((struct student *)PT_REGS_RC(ctx))->firstname);
    bpf_get_current_comm(&comm, sizeof(comm));
    events.perf_submit(ctx,&data,sizeof(data));
    return 0;
};
"""

b = BPF(text=bpf_text)
b.attach_uprobe(name="<...your test binary file location...>/test",
        sym="displayDetail", fn_name="printret")

# header
print("%-9s %-6s %s" % ("TIME", "PID", "ARGUMENT"))

def print_event(cpu, data, size):
    event = b["events"].event(data)
    print("%-9s %-6d %s" % (strftime("%H:%M:%S"), event.pid,
                            event.str.decode('utf-8', 'replace')))

b["events"].open_perf_buffer(print_event)
while 1:
    try:
        b.perf_buffer_poll()
    except KeyboardInterrupt:
        exit()
Here is the steps:
$ sudo python mytrace.py

# open another terminal
$ gcc -o test test.c
$ ./test
Enter detail of student #1
Enter ID: 1234
Enter first name: Danny
Enter last name: Liu
Enter Points: 100

Detail of student #1

Result via ptr
ID: 1234
First Name: Danny
Last Name: Liu
Points: 100.000000
Then you will see the tracing message in your previous terminal as follows:
(print out the first name from function's argument)
$ sudo python mytrace.py
TIME      PID    ARGUMENT
17:01:32  7265   Danny

Thursday, February 20, 2020

Thursday, February 13, 2020

[Darknet][Python] Support using Numpy array as image input instead of image path in Darknet Python API

When I dealt with Darknet's detect function in Python, I found that there is only one way for accepting an image file path as the argument, which is convenient for inferencing images from files.  But, it is not very well for images from video files or cameras.
Due to this reason, I dig out some solutions already on the Internet, and modify as follows:

Thursday, October 24, 2019

[Spring Boot] My first Spring Boot Project for demo

Recently, the term "Microservices" is very hot in the cloud service/architecture and it gets my attention. So, I want to learn more about what it is. Furthermore, I also notice Spring Cloud is a very popular microservice framework based on Spring Boot and it provides tools for developers to quickly build some of the common patterns in distributed systems (e.g. configuration management, service discovery, circuit breakers, intelligent routing, micro-proxy, control bus, one-time tokens, global locks, leadership election, distributed sessions, cluster state)

Monday, October 7, 2019

[Dockerfile] Some of the little skills used in Dockerfile

I collect some of the little skills used in my Dockerfiles and also keep in the record for my reference.

Thursday, September 19, 2019

[IJava] A Jupyter kernel for executing Java code

As we know that Jupyter’s is a Python-based solution, and it provides a lot of convenience for interactivity with Python code. But, I just wonder if Jupyter can use Java as its kernel, and it can. I find that it allows to integrate additional "kernels" with Java capabilities and someone has done it -IJava: https://github.com/SpencerPark/IJava

[Kubernetes] The simple introduction of VPA

VPA stands for Vertical Pod Autoscaling, which frees you from having to think about what values to specify for a container's CPU and memory requests. The autoscaler can recommend values for CPU and memory requests, or it can automatically update values for CPU and memory requests.

Before using VPA, we need to install Metrics Server first as follows:

Wednesday, August 21, 2019

[Kubernetes] Call Kubernetes APIs

If you want to use Bash Shell  to call Kubernetes APIs, Here is the way to get the token and use it as follows:

Prepare variables
#APISERVER="https://10.10.0.100:6443"
$ APISERVER=$(kubectl config view | grep server | cut -f 2- -d ":" | tr -d " ")
$ TOKEN=$(kubectl describe secret $(kubectl get secrets | grep default | cut -f1 -d ' ') | \
    grep -E '^token' | cut -f2 -d':' | tr -d '\t')
Call APIs
$ curl $APISERVER/api --header "Authorization: Bearer ${TOKEN//[[:space:]]/}" --insecure
{
  "kind": "APIVersions",
  "versions": [
    "v1"
  ],
  "serverAddressByClientCIDRs": [
    {
      "clientCIDR": "0.0.0.0/0",
      "serverAddress": "192.168.0.100:6443"
    }
  ]
}

[Docker] Troubleshooting to docker private registry

I create a private docker registry as follows and sometimes it cannot reply the http request.
$ sudo docker run -p 7443:7443 --restart=always \
  -v /raid/registry2:/var/lib/registry \
  -e REGISTRY_HTTP_ADDR=0.0.0.0:7443 \
  --name registry registry:2

$ curl -v http://192.168.10.10:7443/v2/_catalog
*   Trying 192.168.10.10...
* TCP_NODELAY set
* Connected to 192.168.10.10 (192.168.10.10) port 7443 (#0)
> GET /v2/_catalog HTTP/1.1
> Host: 192.168.0.109:7443
> User-Agent: curl/7.58.0
> Accept: */*
...(hang)...

Tuesday, August 20, 2019

[Kubernetes] Add new DNS server in CoreDNS Configuration

After finished the K8S installation, I had encountered an issue that the pod cannot resolve the domain name. I use these commands to check the DNS issue.
$ systemd-resolve --status
nameservers:
                  addresses:
                  - 210.240.232.1
                  search: []

# use busybox pod to run nslookup
$ kubectl apply -f https://k8s.io/examples/admin/dns/busybox.yaml
$ kubectl exec -it busybox -- nslookup kubernetes.default
$ kubectl exec -it busybox -- nslookup google.com

# check local DNS configuration
$ kubectl exec busybox cat /etc/resolv.conf

Monday, August 5, 2019

[Qt] How to install Qt for WebAssembly

In the previous post: [Qt] The Qt features of WebAssembly and Qt quick WebGLStreaming, I mentioned about Qt for WebAssembly but didn't try it yet. Here, this post is to introduce how to install Qt for WebAssembly and give an example to try by the following steps:

Tuesday, July 30, 2019

[Kubernetes] How to install Kubernetes Dashboard and without invalid certification

When I follow the instructions from the official site: https://github.com/kubernetes/dashboard to install Kubernetes Dashboard, I encounter the problem that I cannot access the dashboard via my browser because the certificate is invalid. After figuring it out, Here is my approach to resolving it.

Sunday, July 21, 2019

[DDS] Install OpenSplice DCPS Python API on Raspberry Pi 3

Before starting to introduce how to install OpenSplice DCPS Python API on Raspberry Pi 3, we can take a look at the guide of OpenSplice DCPS Python API as list:

Thursday, July 18, 2019

[Kubernetes] The example of commands for commonly checking kubernetes status and troubleshooting

This post is about the example of commands for checking kubernetes status and troubleshooting. The purpose is for the reference by myself.

Wednesday, July 17, 2019

[Python] The issues of converting Python2 to Python3

If you are working on converting Python2 source code to Python3, there are some issues you will encounter sooner or later. I arrange my part here and will continue to update it as follows:

Monday, July 15, 2019

[Qt] The Qt features of WebAssembly and Qt quick WebGLStreaming

I recently noticed that there are 2 major Qt features: Qt WebAssembly and Qt quick WebGL which are very powerful and useful. For the Qt WebAccembly, any your C++ Qt applications could be executed on a browser. For Qt quick WebGL, it is optimized for Qt Quick and allows you to run remote Qt Quick applications in a browser.

Qt WebAssembly

Qt for WebAssembly

Get started with Qt for WebAssembly
Here is a presentation about Qt for WebAccembly.
QtWS18 – Qt for WebAssembly by Morten Sørvig, The Qt Company

Qt WebAssembly Examples
https://github.com/msorvig/qt-webassembly-examples

Qt For WebAssembly Examples
(source code: https://github.com/msorvig/qt-webassembly-examples)
SensorTagDemo
colordebugger
gui_lifecycle
gui_localfiles
gui_opengl
gui_raster
mqtt_simpleclient
quick_clocks
quick_controls2_gallery
quick_controls2_testbench
quick_hellosquare
slate
widget_wiggly
widgets_wiggly

https://www.jianshu.com/p/bed6c2963435

Qt quick WebGL 

Here is a quick introduction to Qt quick WebGL

The introduction of Qt Quick WebGL Streaming

WebGL streaming in a Raspberry PI Zero W

Related Demo:

[Shared Library] How to check the used shared libraries in your program

This post is only to show the command about checking the used shared libraries in your program.

Wednesday, June 19, 2019

[DDS] Install OpenSplice Community Version

What is DDS? The Data Distribution Service (DDS™) is a middleware protocol and API standard for data-centric connectivity from the Object Management Group® (OMG®). It integrates the components of a system together, providing low-latency data connectivity, extreme reliability, and a scalable architecture that business and mission-critical Internet of Things (IoT) applications need. For more information, check it out: https://www.dds-foundation.org/what-is-dds-3/
https://zhuanlan.zhihu.com/p/32278571

In this post, I install OpenSplice as my DDS runtime environment and library. You can download it from here: https://www.adlinktech.com/en/dds-community-software-evaluation.aspx

Wednesday, June 12, 2019

[TensorFlow] Build TensorFlow from source with Intel MKL enabled

Based on this document "Intel® Math Kernel Library for Deep Learning Networks: Part 1–Overview and Installation",  I give a quick summary to do it.
Here are the steps for building TensorFlow from source with Intel MKL enabled.

For MKL DNN:
# Install Intel MKL & MKL-DNN
$ git clone https://github.com/01org/mkl-dnn.git
$ cd mkl-dnn
$ cd scripts && ./prepare_mkl.sh && cd ..
$ mkdir -p build && cd build && cmake .. && make -j$(nproc)
$ make test
$ sudo make install
For building TensorFlow:

Monday, June 10, 2019

[Squid] Setup Linux Proxy Server and Client

The following steps are about installing and setup Linux proxy server (Squid) 

#install squid proxy
$ sudo apt-get install squid

#modify squid.conf
$ sudo vi /etc/squid/squid.conf
  ==> #add your local network segment
  acl mynetwork src 192.168.0.0/24
  ==> #allow "mynetwork" be accessing via http
  http_access allow mynetwork

#restart squid...OK
$ sudo service squid restart

Tuesday, May 28, 2019

[Docker] Using GUI with Docker

Recently I need to run my GUI application with Docker and it can show up either directly on my desktop operating system or in my ssh terminal client via X11.

Basically, there are some people who already provide the solution for the cases. I just list the reference and quickly give my examples.

Monday, April 22, 2019

[TVM] Deploy TVM Module using C++ API

When the first time to deal with deploying TVM module using C++ API, I found this official web site: Deploy TVM Module using C++ API which only gives a fine example for deploying, but it doesn't explain how to generate the related files and modules.

So, after several times of trial and error, I figure it out to generate the required files for deploying.
Basically you can refer to the TVM tutorials about compiling models:
https://docs.tvm.ai/tutorials/index.html

Monday, April 15, 2019

[Experiment] Compare the inference performance of TensorFlow Lite and TVM

I compare the inference performance of both TensorFlow Lite and TVM on my laptop with the same MobileNet model and the same input size of 224*224.
They both assign two threads to do the inference task and see the average inferencing time it spent.
(P.S: Giving the same of 10 threads in these 2 cases )