Wednesday, May 25, 2022

[Golang] Golang應用程式使用TensorFlow/TensorFlow Lite(TFLite)/Tensorflow Serving 的方法整理

Golang應用程式使用TensorFlow 或是 Tensorflow Lite(TFLite) 的前提是必須要有Tensorflow C 函式庫 或是 TensorFlow Lite C 函式庫，一般來說，比較少情況會是用Golang應用程式使用TensorFlow透過TensorFlow C 函式庫，因為未經過最佳化而效能不佳。Golang應用程式使用TensorFlow Lite(TFLite) 透過TensorFlow Lite C 函式庫會是效能較好的其中一種方式。以下將整理相關資訊來說明:

 

TensorFlow C 函式庫

Documentation for installing the Go bindings for TensorFlow.

這篇文件算是官方整理出如何安裝 Go bindings for TensorFlow，Go's application 可以透過TensorFlow C API 來進行推論(非訓練). 其中，他使用已經編譯好的TensorFlow C 函式庫，下載後安裝。個人覺得此法是最方便的。


使用Go語言來理解Tensorflow

這篇文章說明了使用Go綁定的用途: 

Go綁定可用於導入和定義常量圖，Go API缺少對Variable的支持：該API旨在使用已經訓練過的模型，而不是從頭開始訓練模型。安裝TensorFlow for Go的時候已經明確說明了：

TensorFlow提供了可用於Go程序的API。這些API特別適合於加載用Python創建並需要在Go程序中執行的模型。換句話說，在訓練階段使用Python定義並訓練模型；之後使用Go來加載並使用訓練過的模型！


TensorFlow in Go (Construct and execute TensorFlow graphs in Go.)

這篇比較不同的地方是從源代碼來構建 TensorFlow C 庫 (Build the C library for Tensorflow)

bazelisk build -c opt --config monolithic //tensorflow:libtensorflow.so

但編譯好的 TensorFlow C 庫需要設置環境變量，對應的路徑要換為Linux 上存儲TensorFlow C庫的位置。例如 :

export LIBRARY_PATH=$LIBRARY_PATH:~/go/src/github.com/tensorflow/tensorflow/bazel-bin/tensorflow
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/go/src/github.com/tensorflow/tensorflow/bazel-bin/tensorflow


Golang部署TensorFlow機器學習模型

這篇提供了簡單的範例，用Python做模型訓練，用Golang做預測服務。

首先需要注意Python模型保存時需要保存為指定的格式：

from tensorflow.python.saved_model.builder_impl import SavedModelBuilder
with tf.Session() as session:  
    # 训练模型操作。。。
    
    # 保存模型
    builder = SavedModelBuilder("存储路径")
    # 保存时需要定义tag
    builder.add_meta_graph_and_variables(session, ["tag"])
    builder.save()

Golang運行模型，具體代碼：

package main

import (
	"fmt"
	tf "github.com/tensorflow/tensorflow/tensorflow/go"
)

func main() {
	m, err := tf.LoadSavedModel("modelPath", []string{"modelTag"}, nil) // 载入模型
	if err != nil {
		// 模型加载失败
		fmt.Printf("err: %v", err)
	}
	// 打印出所有的Operator
	for _, op := range m.Graph.Operations() {
		fmt.Printf("Op name: %v", op.Name())
	}
	// 构造输入Tensor。根据你的模型入参格式来定义
	x := [1][8]int32{
		{0,1,2,3,4,5,6,7},
	}
	tensor_x, err := tf.NewTensor(x)
	if err != nil {
		fmt.Printf("err: %s", err.Error())
		return
	}
	kb, err := tf.NewTensor(float32(1))
	if err != nil {
		fmt.Printf("err: %s", err.Error())
		return
	}
	s := m.Session
	feeds := map[tf.Output]*tf.Tensor{
		// operation name 需要根据你的模型入参来写
		m.Graph.Operation("input_x").Output(0): tensor_x,
		m.Graph.Operation("keep_prob").Output(0): kb,
	}
	fetches := []tf.Output{
		// 输出层的name 也要根据你的模型写
		m.Graph.Operation("score/ArgMax").Output(0),
	}
	result, err:= s.Run(feeds, fetches,nil)
	if err != nil {
		// 模型预测失败
		fmt.Printf("err: %s  ", err.Error())
	}
	fmt.Printf("%#v", result)
}


TensorFlow Lite C 函式庫

Build the C library for TFLite

這篇包含 TensorFlow Lite 的 C API的建置與使用

bazelisk build -c opt --config monolithic //tensorflow/lite/c:libtensorflowlite_c.so

P.S: 編譯完成後，shared library: libtensorflowlite_c.so 在bazel-bin/tensorflow/lite/c

Go-tflite: Go binding for TensorFlow Lite

https://github.com/mattn/go-tflite

這篇文章提供了很不錯的使用TFLite的範例程式。例如: label_image 可以快速驗證編譯好的TensorFlow Lite C 函式庫。


採用的編譯TensorFlow方式:

使用 Bazelisk 安裝 Bazel: https://bazel.build/install/bazelisk，此方式是使用 Go 從來源編譯：go install github.com/bazelbuild/bazelisk@latest (需要 Go 1.17 以上版本) 或是用 npm install的方式(建議)。

下列式實驗過後成功的做法:

$ sudo npm install -g @bazel/bazelisk
$ go env GOPATH
$ git clone --branch v2.9.0 https://github.com/tensorflow/tensorflow.git ${GOPATH}/src/github.com/tensorflow/tensorflow
$ sudo apt install libprotobuf-dev protobuf-compiler
$ sudo apt install python3 swig python3-numpy
$ cd ${GOPATH}/src/github.com/tensorflow/tensorflow

# 設定版本
$ ./configure
You have bazel 4.2.1 installed.
Please specify the location of python. [Default is /usr/bin/python3]:

Found possible Python library paths:
  /usr/lib/python3.9/dist-packages
  /usr/lib/python3/dist-packages
  /usr/local/lib/python3.9/dist-packages
Please input the desired Python library path to use.  Default is [/usr/lib/python3.9/dist-packages]

Do you wish to build TensorFlow with ROCm support? [y/N]: N
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: N
No CUDA support will be enabled for TensorFlow.

Do you wish to download a fresh release of clang? (Experimental) [y/N]: N
Clang will not be downloaded.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -Wno-sign-compare]:

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: N
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
        --config=mkl            # Build with MKL support.
        --config=mkl_aarch64    # Build with oneDNN and Compute Library for the Arm Architecture (ACL).
        --config=monolithic     # Config for mostly static monolithic build.
        --config=numa           # Build with NUMA support.
        --config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
        --config=v1             # Build with TensorFlow 1 API instead of TF 2 API.
Preconfigured Bazel build configs to DISABLE default on features:
        --config=nogcp          # Disable GCP support.
        --config=nonccl         # Disable NVIDIA NCCL support.
Configuration finished



# Build TensorFlow C 函式庫
$ bazel build -c opt //tensorflow:libtensorflow.so

# Build TensorFlow Lite C 函式庫
$ bazel build -c opt //tensorflow/lite/c:libtensorflowlite_c.so


TensorFlow Lite for Go

看到還不錯的TensorFlow Lite for Golang的專案. Github: https://github.com/nbortolotti/tflitego

tflitego 為在 Golang 中使用 TensorFlow Lite 提供了一個簡單明了的解決方案。其目標是提供一個有凝聚力的 API，與 TensorFlow Lite C API 連接並且容易維護。

Examples for tflitego: https://github.com/nbortolotti/tflitego_examples



Tensorflow Serving

對於Tensorflow Serving，目前只能先放上別人的連結，日後再研究:
https://sineyuan.github.io/post/tensorflow-mnist-pratice/



其他資源:

TensorFlow Lite淺度解析

golang-tf (contains Dockerfiles for using Tensorflow in Go)

此提供了包含用於在 Go 中使用 Tensorflow 的 Dockerfile。您可以自行構建或從Docker Hub 上的wamuir/golang-tf拉取鏡像


安裝Golang

Download Go binary

In current development, Golang version in the environment is v1.14

mkdir tmp
cd /tmp
wget https://dl.google.com/go/go1.18.3.linux-amd64.tar.gz
sudo tar -xvf go1.18.3.linux-amd64.tar.gz
sudo mv go /usr/local

Environment setup

We can add the following content in ~/.profile or ~/.bashrc

#Golang path 
export PATH=$PATH:/usr/local/go/bin 
export GOPATH=$HOME/go 
export PATH=$PATH:$GOPATH/bin

Then, renewing the shell sessions

source ~/.profile

Golang should have been now installed successfully on the machine and to check if it is run below command

go version





-
Labels: ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)