Platform For AI：TensorFlow服務要求構造

EAS在華東2（上海）的VPC環境中部署了一個Public的測試案例，其服務名稱為mnist_saved_model_example，訪問Token為空白。您可以通過URLhttp://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/mnist_saved_model_example訪問該服務。具體方式如下：

1. 擷取模型資訊。

   通過GET請求可以擷取模型的相關資訊，包括signature_name、name、type及shape，樣本如下。

   $curl http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/mnist_saved_model_example | python -mjson.tool
   {
       "inputs": [
           {
               "name": "images",
               "shape": [
                   -1,
                   784
               ],
               "type": "DT_FLOAT"
           }
       ],
       "outputs": [
           {
               "name": "scores",
               "shape": [
                   -1,
                   10
               ],
               "type": "DT_FLOAT"
           }
       ],
       "signature_name": "predict_images"
   }

   該模型是一個MNIST資料集（下載MNIST資料集）分類模型。輸入資料為DT_FLOAT類型，例如shape為[-1,784]，其中第一維表示batch_size（如果單個請求只包含一張圖片，則batch_size為1），第二維表示784維的向量。因為訓練該測試模型時，將其輸入展開成了一維，所以單張圖片輸入也需要變換為28*28=784的一維向量。構建輸入時，無論shape取值如何，都必須將輸入展開成一維向量。該樣本中，如果輸入單張圖片，則輸入為1*784的一維向量。如果訓練模型時輸入的shape為[-1, 28, 28]，則構建輸入時就需要將輸入構建為1*28*28的一維向量。如果服務要求中指定的shape與模型的shape不一致，則預測請求報錯。

2. 安裝ProtoBuf並調用服務（以Python 2為例，介紹如何對TensorFlow服務進行調用）。

   EAS為Python預先產生了ProtoBuf包，您可以使用如下命令直接安裝。

   $ pip install http://eas-data.oss-cn-shanghai.aliyuncs.com/sdk/pai_tf_predict_proto-1.0-py2.py3-none-any.whl

   調用服務進行預測的Python 2範例程式碼如下。

   #!/usr/bin/env python
   # -*- coding: UTF-8 -*-
   import json
   from urlparse import urlparse
   from com.aliyun.api.gateway.sdk import client
   from com.aliyun.api.gateway.sdk.http import request
   from com.aliyun.api.gateway.sdk.common import constant
   from pai_tf_predict_proto import tf_predict_pb2
   import cv2
   import numpy as np
   with open('2.jpg', 'rb') as infile:
       buf = infile.read()
       # 使用numpy將位元組流轉換成array。
       x = np.fromstring(buf, dtype='uint8')
       # 將讀取到的array進行圖片解碼獲得28 × 28的矩陣。
       img = cv2.imdecode(x, cv2.IMREAD_UNCHANGED)
       # 因為預測服務API需要長度為784的一維向量，所以將矩陣reshape成784。
       img = np.reshape(img, 784)
   def predict(url, app_key, app_secret, request_data):
       cli = client.DefaultClient(app_key=app_key, app_secret=app_secret)
       body = request_data
       url_ele = urlparse(url)
       host = 'http://' + url_ele.hostname
       path = url_ele.path
       req_post = request.Request(host=host, protocol=constant.HTTP, url=path, method="POST", time_out=6000)
       req_post.set_body(body)
       req_post.set_content_type(constant.CONTENT_TYPE_STREAM)
       stat,header, content = cli.execute(req_post)
       return stat, dict(header) if header is not None else {}, content
   def demo():
       # 輸入模型資訊，單擊模型名稱即可擷取。
       app_key = 'YOUR_APP_KEY'
       app_secret = 'YOUR_APP_SECRET'
       url = 'YOUR_APP_URL'
       # 構造服務。
       request = tf_predict_pb2.PredictRequest()
       request.signature_name = 'predict_images'
       request.inputs['images'].dtype = tf_predict_pb2.DT_FLOAT  # images參數類型。
       request.inputs['images'].array_shape.dim.extend([1, 784])  # images參數的形狀。
       request.inputs['images'].float_val.extend(img)  # 資料。
       request.inputs['keep_prob'].dtype = tf_predict_pb2.DT_FLOAT  # keep_prob參數的類型。
       request.inputs['keep_prob'].float_val.extend([0.75])  # 預設填寫一個。
       # 將ProtoBuf序列化成string進行傳輸。
       request_data = request.SerializeToString()
       stat, header, content = predict(url, app_key, app_secret, request_data)
       if stat != 200:
           print 'Http status code: ', stat
           print 'Error msg in header: ', header['x-ca-error-message'] if 'x-ca-error-message' in header else ''
           print 'Error msg in body: ', content
       else:
           response = tf_predict_pb2.PredictResponse()
           response.ParseFromString(content)
           print(response)
   if __name__ == '__main__':
       demo()

   該樣本的輸出如下。

   outputs {
     key: "scores"
     value {
       dtype: DT_FLOAT
       array_shape {
         dim: 1
         dim: 10
       }
       float_val: 0.0
       float_val: 0.0
       float_val: 1.0
       float_val: 0.0
       float_val: 0.0
       float_val: 0.0
       float_val: 0.0
       float_val: 0.0
       float_val: 0.0
       float_val: 0.0
     }
   }

   其中outputs為10個類別對應的得分，即輸入圖片為2.jpg時，除value[2]外，其他均為0。因此最終預測結果為2，預測結果正確。

除Python外，使用其它語言用戶端調用服務都需要根據.proto檔案手動產生預測的請求代碼檔案。調用樣本如下：

1. 編寫請求代碼檔案（例如建立tf.proto檔案），內容如下。

   syntax = "proto3";
   option cc_enable_arenas = true;
   option java_package = "com.aliyun.openservices.eas.predict.proto";
   option java_outer_classname = "PredictProtos";
   enum ArrayDataType {
     // Not a legal value for DataType. Used to indicate a DataType field
     // has not been set.
     DT_INVALID = 0;
     // Data types that all computation devices are expected to be
     // capable to support.
     DT_FLOAT = 1;
     DT_DOUBLE = 2;
     DT_INT32 = 3;
     DT_UINT8 = 4;
     DT_INT16 = 5;
     DT_INT8 = 6;
     DT_STRING = 7;
     DT_COMPLEX64 = 8;  // Single-precision complex.
     DT_INT64 = 9;
     DT_BOOL = 10;
     DT_QINT8 = 11;     // Quantized int8.
     DT_QUINT8 = 12;    // Quantized uint8.
     DT_QINT32 = 13;    // Quantized int32.
     DT_BFLOAT16 = 14;  // Float32 truncated to 16 bits.  Only for cast ops.
     DT_QINT16 = 15;    // Quantized int16.
     DT_QUINT16 = 16;   // Quantized uint16.
     DT_UINT16 = 17;
     DT_COMPLEX128 = 18;  // Double-precision complex.
     DT_HALF = 19;
     DT_RESOURCE = 20;
     DT_VARIANT = 21;  // Arbitrary C++ data types.
   }
   // Dimensions of an array.
   message ArrayShape {
     repeated int64 dim = 1 [packed = true];
   }
   // Protocol buffer representing an array.
   message ArrayProto {
     // Data Type.
     ArrayDataType dtype = 1;
     // Shape of the array.
     ArrayShape array_shape = 2;
     // DT_FLOAT.
     repeated float float_val = 3 [packed = true];
     // DT_DOUBLE.
     repeated double double_val = 4 [packed = true];
     // DT_INT32, DT_INT16, DT_INT8, DT_UINT8.
     repeated int32 int_val = 5 [packed = true];
     // DT_STRING.
     repeated bytes string_val = 6;
     // DT_INT64.
     repeated int64 int64_val = 7 [packed = true];
     // DT_BOOL.
     repeated bool bool_val = 8 [packed = true];
   }
   // PredictRequest specifies which TensorFlow model to run, as well as
   // how inputs are mapped to tensors and how outputs are filtered before
   // returning to user.
   message PredictRequest {
     // A named signature to evaluate. If unspecified, the default signature
     // will be used.
     string signature_name = 1;
     // Input tensors.
     // Names of input tensor are alias names. The mapping from aliases to real
     // input tensor names is expected to be stored as named generic signature
     // under the key "inputs" in the model export.
     // Each alias listed in a generic signature named "inputs" should be provided
     // exactly once in order to run the prediction.
     map<string, ArrayProto> inputs = 2;
     // Output filter.
     // Names specified are alias names. The mapping from aliases to real output
     // tensor names is expected to be stored as named generic signature under
     // the key "outputs" in the model export.
     // Only tensors specified here will be run/fetched and returned, with the
     // exception that when none is specified, all tensors specified in the
     // named signature will be run/fetched and returned.
     repeated string output_filter = 3;
   }
   // Response for PredictRequest on successful run.
   message PredictResponse {
     // Output tensors.
     map<string, ArrayProto> outputs = 1;
   }

   其中PredictRequest定義TensorFlow服務的輸入格式，PredictResponse定義服務的輸出格式。關於ProtoBuf的詳細用法請參見ProtoBuf介紹。

2. 安裝protoc。

   #/bin/bash
   PROTOC_ZIP=protoc-3.3.0-linux-x86_64.zip
   curl -OL https://github.com/google/protobuf/releases/download/v3.3.0/$PROTOC_ZIP
   unzip -o $PROTOC_ZIP -d ./ bin/protoc
   rm -f $PROTOC_ZIP

3. 產生請求代碼檔案：

   • Java版本

     $ bin/protoc --java_out=./ tf.proto

     命令執行完成後，系統會在目前的目錄產生com/aliyun/openservices/eas/predict/proto/PredictProtos.java，在專案中匯入該檔案即可。

   • Python版本

     $ bin/protoc --python_out=./ tf.proto

     命令執行完成後，系統會在目前的目錄產生tf_pb2.py，通過import命令匯入該檔案即可。

   • C++版本

     $ bin/protoc --cpp_out=./ tf.proto

     命令執行完成後，系統在目前的目錄產生tf.pb.cc和tf.pb.h。在代碼中使用include tf.pb.h命令，並將tf.pb.cc添加至compile列表即可。