Tensorflow Serving安装+部署CycleGan生成器模型(马和斑马)

章节:

  1. 本地Ubuntu虚拟机部署效果
  2. 云服务器centos:Docker安装Tensorflow Serving。
  3. Tensorflow Serving部署CycleGan已训练模型(马转换为斑马)。
  4. 上传你的图片试试效果
  5. tensorflow2-Cycgan源码

1.本地Ubuntu虚拟机部署效果

部署的模型为CycleGan的已训练好的生成模型,如图1。(模型保存方法为model.save())

图1 CycleGan 生成器已训练模型

已经在本地的Ubuntu虚拟机部署成功了,图2。

图2 虚拟机Ubuntu部署

使用Tensorflow Serving客户端的REST API发送请求即可调用部署的模型进行predict。图3左侧为输入图片数据,右侧为模型predict后的数据。

图3 左:输入图片 右:预测图片(转换后的图片)

请求代码如下:

import json
import numpy
import requests
#定义instances,传入需要转换的图片
data = json.dumps({"instances":img.tolist()})
#定义headers
headers = {"content-type": "application/json"}
# 定义 url
url = 'http://192.168.5.118:8501/v1/models/model:predict'
# 传输数据进行预测,得到返回结果
json_response = requests.post(url, data=data, headers=headers)
# 对结果进行解析,然后变成 array
img_pre = numpy.array(json.loads(json_response.text)["predictions"])

代码中img为图3左侧图片,发送至tensorflow serving后返回的img_pre为图3右侧转换后的图片。虚拟机Ubuntu(ip:192.168.5.118)为桥接模式,与本地电脑(运行上述代码的电脑)在同一个网段。

下面继续在云服务器上安装,系统是centos。

2.Docker安装Tensorflow Serving

Tensorflow Serving的安装过程官方文档github都讲的很清楚。

官方示例的Docker安装Tensorflow Serving,以及部署一个简单的模型,该模型用于计算\(y = 0.5 \times \mathrm{x}+2\),即输入为x,输出为y。

示例命令如下:

# Download the TensorFlow Serving Docker image and repo
docker pull tensorflow/serving

git clone https://github.com/tensorflow/serving
# Location of demo models
TESTDATA="$(pwd)/serving/tensorflow_serving/servables/tensorflow/testdata"

# Start TensorFlow Serving container and open the REST API port
docker run -t --rm -p 8501:8501 \
    -v "$TESTDATA/saved_model_half_plus_two_cpu:/models/half_plus_two" \
    -e MODEL_NAME=half_plus_two \
    tensorflow/serving &

# Query the model using the predict API
curl -d '{"instances": [1.0, 2.0, 5.0]}' \
    -X POST http://localhost:8501/v1/models/half_plus_two:predict

# Returns => { "predictions": [2.5, 3.0, 4.5] }

docker命令没啥好讲的。用docker命令pull一下tensorflow/serving的镜像,然后用git clone一下模型(\(y = 0.5 \times \mathrm{x}+2\))。docker run的时候相关参数的指定是最重要的,其中包括本地模型的路径、模型挂载到docker容器中的路径,指定模型名等等。具体怎么指定自己的模型下一节介绍。

上面命令敲下来,最后得到返回结果,输入的三个数1.0、2.0和5.0经过模型(\(y = 0.5 \times \mathrm{x}+2\))计算后返回的结果就是2.5、3.0和4.5。(反手我立马心算检查一下这个结果,我相信我不是一个人(还能是一条狗?)。最后成功得到结果就说明,说明模型(\(y = 0.5 \times \mathrm{x}+2\))已经被成功部署了,也就说明环境没问题了,可以弄自己的模型上去了。

 连接服务器,一行行命令开(fu)敲(zhi)。

ubuntu@VM-0-7-ubuntu:~$ docker pull tensorflow/serving
Using default tag: latest
latest: Pulling from tensorflow/serving
01bf7da0a88c: Pull complete 
f3b4a5f15c7a: Pull complete 
57ffbe87baa1: Pull complete 
e72e6208e893: Pull complete 
6ea3f464ef73: Extracting [===============>                                   ]  25.62MB/80.77MB
01e9bf86544b: Download complete 
68f6bba3dc50: Download complete 
latest: Pulling from tensorflow/serving
01bf7da0a88c: Pull complete 
f3b4a5f15c7a: Pull complete 
57ffbe87baa1: Pull complete 
e72e6208e893: Pull complete 
6ea3f464ef73: Pull complete 
01e9bf86544b: Pull complete 
68f6bba3dc50: Pull complete 
Digest: sha256:6651f4839e1124dbde75ee531825112af0a6b8ef082c88ab14ca53eb69a2e4bb
Status: Downloaded newer image for tensorflow/serving:latest
docker.io/tensorflow/serving:latest
ubuntu@VM-0-7-ubuntu:~$ sudo git clone https://github.com/tensorflow/serving
Cloning into 'serving'...
remote: Enumerating objects: 26248, done.
remote: Counting objects: 100% (918/918), done.
remote: Compressing objects: 100% (219/219), done.
Receiving objects:  30% (7974/26248), 6.73 MiB | 7.00 KiB/s       
remote: Total 26248 (delta 754), reused 847 (delta 698), pack-reused 25330
Receiving objects: 100% (26248/26248), 11.81 MiB | 10.00 KiB/s, done.
Resolving deltas: 100% (20427/20427), done.
ubuntu@VM-0-7-ubuntu:~$ TESTDATA="$(pwd)/serving/tensorflow_serving/servables/tensorflow/testdata"
ubuntu@VM-0-7-ubuntu:~$ docker run -t --rm -p 8501:8501 \
>     -v "$TESTDATA/saved_model_half_plus_two_cpu:/models/half_plus_two" \
>     -e MODEL_NAME=half_plus_two \
>     tensorflow/serving &
[1] 20914

Command 'amp' not found, did you mean:

  command 'apm' from snap atom (1.38.2)
  command 'amd' from deb am-utils
  command 'ams' from deb ams
  command 'amap' from deb amap-align
  command 'pamp' from deb paml
  command 'bmp' from deb ztex-bmp
  command 'xmp' from deb xmp
  command 'amph' from deb amphetamine
  command 'cmp' from deb diffutils
  command 'arp' from deb net-tools
  command 'apm' from deb apmd
  command 'asp' from deb asp
  command 'amq' from deb am-utils
  command 'amr' from deb acr
  command 'omp' from deb openvas-cli

See 'snap info <snapname>' for additional versions.

ubuntu@VM-0-7-ubuntu:~$ 2021-05-15 06:00:32.695784: I tensorflow_serving/model_servers/server.cc:89] Building single TensorFlow model file config:  model_name: half_plus_two model_base_path: /models/half_plus_two
2021-05-15 06:00:32.700690: I tensorflow_serving/model_servers/server_core.cc:465] Adding/updating models.
2021-05-15 06:00:32.701081: I tensorflow_serving/model_servers/server_core.cc:591]  (Re-)adding model: half_plus_two
2021-05-15 06:00:32.702569: I tensorflow_serving/core/basic_manager.cc:740] Successfully reserved resources to load servable {name: half_plus_two version: 123}
2021-05-15 06:00:32.702865: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: half_plus_two version: 123}
2021-05-15 06:00:32.703115: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: half_plus_two version: 123}
2021-05-15 06:00:32.703347: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/half_plus_two/00000123
2021-05-15 06:00:32.705372: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:90] Reading meta graph with tags { serve }
2021-05-15 06:00:32.705677: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:132] Reading SavedModel debug info (if present) from: /models/half_plus_two/00000123
2021-05-15 06:00:32.705974: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-05-15 06:00:32.734629: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:206] Restoring SavedModel bundle.
2021-05-15 06:00:32.736725: I external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 2394445000 Hz
2021-05-15 06:00:32.777765: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:190] Running initialization op on SavedModel bundle at path: /models/half_plus_two/00000123
2021-05-15 06:00:32.786657: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: success: OK. Took 83305 microseconds.
2021-05-15 06:00:32.787739: I tensorflow_serving/servables/tensorflow/saved_model_warmup_util.cc:59] No warmup data file found at /models/half_plus_two/00000123/assets.extra/tf_serving_warmup_requests
2021-05-15 06:00:32.788278: I tensorflow_serving/core/loader_harness.cc:87] Successfully loaded servable version {name: half_plus_two version: 123}
2021-05-15 06:00:32.789742: I tensorflow_serving/model_servers/server_core.cc:486] Finished adding/updating models
2021-05-15 06:00:32.790125: I tensorflow_serving/model_servers/server.cc:367] Profiler service is enabled
2021-05-15 06:00:32.791048: I tensorflow_serving/model_servers/server.cc:393] Running gRPC ModelServer at 0.0.0.0:8500 ...
[warn] getaddrinfo: address family for nodename not supported
2021-05-15 06:00:32.793470: I tensorflow_serving/model_servers/server.cc:414] Exporting HTTP/REST API at:localhost:8501 ...
[evhttp_server.cc : 245] NET_LOG: Entering the event loop ...

没有出错,再开一个终端。

ubuntu@VM-0-7-ubuntu:~$ curl -d '{"instances": [1.0, 2.0, 5.0]}' \
>     -X POST http://localhost:8501/v1/models/half_plus_two:predict
{
    "predictions": [2.5, 3.0, 4.5
    ]
}ubuntu@VM-0-7-ubuntu:~$ 

计算结果正确,成功返回,说明环境没问题。接下来开始加载自己的训练模型。

3.Tensorflow Serving加载自己的训练模型

首先把自己的模型上传,保存的时候代码是“model.save(‘my_model/1’)”,所以模型保存目录结构如图4。

图4 保存模型的目录结构

把my_model整个目录打个包直接上传到服务器。

图5 打包上传服务器

然后解压。解压后看一下目录结构。这里的目录“1”的意思是当前模型的版本号。

ubuntu@VM-0-7-ubuntu:~$ tree my_model
my_model
└── 1
    ├── assets
    ├── saved_model.pb
    └── variables
        ├── variables.data-00000-of-00002
        ├── variables.data-00001-of-00002
        └── variables.index

3 directories, 4 files

接下来就是创建容器了,先上命令,再解释。

docker run -p 8500:8500 -p 8501:8501 --mount type=bind,source=$(pwd)/my_model,target=/models/generator_horse -e MODEL_NAME=generator_horse_model -t tensorflow/serving --model_base_path=/models/generator_horse

docker相关命令就不说了。

  • source=$(pwd)/my_model:服务器本地的模型目录,也就是之前tree命令演示了的那个目录。
  • target=/models/generator_horse:指定docker容器里面的目录,将source挂载到该目录下。也就是把本地的my_model中的所有目录和文件挂载到容器的models/generator_horse目录中。
  • -e MODEL_NAME=generator_horse_model:指定模型的名字,在发送请求的时候会用到。
  • –model_base_path=/models/generator_horse:容器中模型路径。(MODEL_NAME与model_base_path绑定,请求MODEL_NAME时调用的模型将在model_base_path下寻找,有时间再翻翻文档,这里是自己光看命令的理解_(:зゝ∠)_ )
  • -t tensorflow/serving:英雄联盟启,啊不,tensorflow serving启动!!!!

使用docker exec命令进容器看看。可以看到容器中的models/genetator_horse/和本地my_model/下的目录和文件是一样的了。

root@VM-0-7-ubuntu:~# docker exec -it 64d bash
root@64d4ba25c19d:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  models  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@64d4ba25c19d:/# cd models
root@64d4ba25c19d:/models# ls
generator_horse
root@64d4ba25c19d:/models# cd generator_horse/
root@64d4ba25c19d:/models/generator_horse# ls
1
root@64d4ba25c19d:/models/generator_horse# cd 1
root@64d4ba25c19d:/models/generator_horse/1# ls
assets  saved_model.pb  variables
root@64d4ba25c19d:/models/generator_horse/1# 

贴一下成功的样子(这里注意,之前的官方示例命令会占用8501端口,所以要么把之前的容器删了,要么这里换个映射端口):

ubuntu@VM-0-7-ubuntu:~$ docker run -p 8500:8500 -p 8501:8501 --mount type=bind,source=$(pwd)/my_model,target=/models/generator_horse -e MODEL_NAME=generator_horse_model -t tensorflow/serving --model_base_path=/models/generator_horse
2021-05-15 06:32:47.161468: I tensorflow_serving/model_servers/server.cc:89] Building single TensorFlow model file config:  model_name: generator_horse_model model_base_path: /models/generator_horse
2021-05-15 06:32:47.162199: I tensorflow_serving/model_servers/server_core.cc:465] Adding/updating models.
2021-05-15 06:32:47.162462: I tensorflow_serving/model_servers/server_core.cc:591]  (Re-)adding model: generator_horse_model
2021-05-15 06:32:47.164113: I tensorflow_serving/core/basic_manager.cc:740] Successfully reserved resources to load servable {name: generator_horse_model version: 1}
2021-05-15 06:32:47.164379: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: generator_horse_model version: 1}
2021-05-15 06:32:47.164590: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: generator_horse_model version: 1}
2021-05-15 06:32:47.164850: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/generator_horse/1
2021-05-15 06:32:47.210478: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:90] Reading meta graph with tags { serve }
2021-05-15 06:32:47.211028: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:132] Reading SavedModel debug info (if present) from: /models/generator_horse/1
2021-05-15 06:32:47.211401: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-05-15 06:32:47.362239: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:206] Restoring SavedModel bundle.
2021-05-15 06:32:47.379354: I external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 2394445000 Hz
2021-05-15 06:32:47.687629: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:190] Running initialization op on SavedModel bundle at path: /models/generator_horse/1
2021-05-15 06:32:47.795639: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: success: OK. Took 630784 microseconds.
2021-05-15 06:32:47.815054: I tensorflow_serving/servables/tensorflow/saved_model_warmup_util.cc:59] No warmup data file found at /models/generator_horse/1/assets.extra/tf_serving_warmup_requests
2021-05-15 06:32:47.817052: I tensorflow_serving/core/loader_harness.cc:87] Successfully loaded servable version {name: generator_horse_model version: 1}
2021-05-15 06:32:47.818385: I tensorflow_serving/model_servers/server_core.cc:486] Finished adding/updating models
2021-05-15 06:32:47.818691: I tensorflow_serving/model_servers/server.cc:367] Profiler service is enabled
2021-05-15 06:32:47.819321: I tensorflow_serving/model_servers/server.cc:393] Running gRPC ModelServer at 0.0.0.0:8500 ...
[warn] getaddrinfo: address family for nodename not supported
2021-05-15 06:32:47.820160: I tensorflow_serving/model_servers/server.cc:414] Exporting HTTP/REST API at:localhost:8501 ...
[evhttp_server.cc : 245] NET_LOG: Entering the event loop ...

部署成功了,就可以写代码试试效果了。

首先加载一张图片,预处理成生成器模型的输入数据。

# 需要导入的库
import cv2
import numpy as np
import matplotlib.pyplot as plt
# 读取指定路径的图片
def imread(path):
    img = cv2.imread(path)
    # 转换颜色模式
    img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
    return img
#加载单张图片用于测试
def load_img(path):
    img = imread(path)
    img = img = cv2.resize(img, (256, 256))
    img = img/127.5 - 1.
    return img[np.newaxis, :, :, :]
#加载测试图片
img = load_img("test.jpg")
plt.imshow(((img[0]+1)*127.5).astype(np.int))

打印图片看看:

图6 加载图片

之后就是给服务器发请求了。代码如下

import json
import numpy
import requests
#定义instances,传入需要转换的图片
data = json.dumps({"instances":img.tolist()})
#定义headers
headers = {"content-type": "application/json"}
# 定义 url
url = 'http://111.222.333.444:8501/v1/models/generator_horse_model:predict'
# 传输数据进行预测,得到返回结果
json_response = requests.post(url, data=data, headers=headers)
# 对结果进行解析,然后变成 array
img_pre = numpy.array(json.loads(json_response.text)["predictions"])

输入数据,也就是发送的数据,记得转成list。还有就是url构造的时候需要注意如下几点:

  • IP:服务器IP(111.222.333.444)请换成自己的IP。
  • v1:指模型的版本,应该是对应前面的“1”文件夹,如果有包含模型的“2”文件夹,如果想调用“2”文件夹下的模型则这里要指定为v2。(还没试,万一错了请喷我_(:зゝ∠)_ )
  • models:这里是存放模型的路径,也就是挂载目录的父目录。
  • generator_horse_model:MODEL_NAME指定的模型名字。

对“http://111.222.333.444:8501/v1/models/generator_horse_model:predict”,课代表总结一下就是,Tensorflow Serving会在models目录下找和“MODEL_NAME=generator_horse_model”名字绑定了的目录,也就是找“model_base_path=/models/generator_horse”;并在这个目录下找对应v1版本的目录“1”;目录“1”下就是模型了;使用该模型进行predict。

运行代码,发送请求并接受返回数据img_pre。输出img_pre的维度看看如图7,是否符合自己的模型输出,没问题是一张生成图的维度。

图7 输出返回数据的维度

最后把原图和生成图对比一下,代码如下,对比图如图8。

# 把-1~1的输出数据转换成0~255
fake = (img_pre*0.5 + 0.5)*255
#绘制原图
plt.figure(figsize=(12,10))
plt.subplot(1,2,1) 
plt.imshow(((img[0]+1)*127.5).astype(np.int))
#绘制风格转换后的图
plt.subplot(1,2,2) 
plt.imshow(fake[0].astype(np.int))
图8 原图和生成图对比

接下来准备学一下Flask搞个网页上传图片自动转换然后展示或下载啥的。


4.上传一张有马(骑兵?)的图片试试效果

输入图片尺寸在处理前会被强制转换成256×256的大小,生成图的大小也同样是256×256,所以上传图片尽量是长宽比为1:1的图片,否则生成图会扭曲。

小水管服务器,有点慢。点击提交按钮后(点一下就行了),请等待一分钟(慢的怀疑人生)。。。

4.tensorflow-Cycgan源码


实验20 CycleGAN-风格迁移

发表评论

您的电子邮箱地址不会被公开。 必填项已用*标注