Elasticsearch:Use Elastic Eland to upload third-party NLP models

Background information

Elastic Eland is a powerful tool that combines the big data processing capability and the Python data science ecosystem. You can use Elastic Eland to perform operations such as analyzing Elasticsearch data and running machine learning tasks. For example, you can use Elastic Eland to transform a pre-training model in the Transformer library of Hugging Face into a TorchScript model, split the TorchScript model into chunks, and then import the chunks into Elasticsearch. A TorchScript model can run in an environment that does not have a Python interpreter.

For more information about Elastic Eland, see Elastic Eland.

Version compatibility

• Alibaba Cloud Elasticsearch clusters of V7.11 or later support Elastic Eland.

  Elasticsearch clusters later than V7.11 support more abundant features of Elastic Eland. We recommend that you use Elastic Eland in Elasticsearch clusters of V8.3 or later.

• When you use Elastic Eland, you must make sure that the major version of your Elasticsearch cluster is the same as the major version of Elastic Eland. For example, if you use an Elasticsearch V8.X cluster, you must use Elastic Eland 8.X. If you use an Elasticsearch V7.X cluster, you must use Elastic Eland 7.X. For more information, see Elastic Eland.

• Python 3.8, Python 3.9, and Python 3.10 are supported.

• Pandas 1.5.3 is supported.

Prerequisites

1. An Elastic Compute Service (ECS) instance is created, and a Python environment is configured for the ECS instance. In this example, Python 3.10.12 is used. For information about how to create an ECS instance, see Get started with Linux instances.

   • When you create an ECS instance, you can select a Ubuntu 22 image, which provides Python 3.10. If you select another image, you must manually download and configure a Python environment. For more information, see Python official documentation.

   • You must make sure that the ECS instance can access Hugging Face.

     You can use Elastic Eland to upload Hugging Face models. Models in other libraries may not be able to be uploaded by using Elastic Eland.

2. An Alibaba Cloud Elasticsearch cluster is created. In this example, an Alibaba Cloud Elasticsearch V8.9 cluster is used. For information about how to create an Alibaba Cloud Elasticsearch cluster, see Create an Alibaba Cloud Elasticsearch cluster.

   • You must add the public or private IP address of the ECS instance to the IP address whitelist of the Elasticsearch cluster to allow the ECS instance to access the Elasticsearch cluster. For more information, see Configure a public or private IP address whitelist for an Elasticsearch cluster.

   • The feature of using Elastic Eland to upload models is provided by the open source Elasticsearch Platinum edition and the open source Elasticsearch Enterprise edition. Alibaba Cloud subscribes to Elasticsearch of the Platinum edition. You can directly upload models to Alibaba Cloud Elasticsearch.

   • The usage of Elasticsearch Machine Learning may differ between Elasticsearch clusters of different major versions. For information about the usage of Elasticsearch Machine Learning in Elasticsearch clusters of versions other than V8.9, see Machine Learning.

Preparations

1. Run the following commands in the ECS instance to download and install Elastic Eland and PyTorch whose versions are compatible with the version of the Alibaba Cloud Elasticsearch cluster. In this example, Elastic Eland 8.7.0 is downloaded and installed.

   pip3 install eland==8.7.0
   pip3 install torch==1.11.0 torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu
   pip install eland[pytorch]

   Note

   If you want to use PyTorch 1.13.1 or earlier to import a natural language processing (NLP) model, you can run the pip install torch==1.13.1 command to install PyTorch whose version meets your business requirements.

2. Check the version of Pandas.

   Run the following command to check whether the version of Pandas is 1.5.3:

   pip show pandas

   If the version of Pandas is not 1.5.3, run the following command to install Pandas 1.5.3:

   pip install pandas==1.5.3

Step 1: Upload a model to Alibaba Cloud Elasticsearch

Method 1: Upload a model in online mode

Run the following command to upload a model in a Hugging Face library to Alibaba Cloud Elasticsearch:

eland_import_hub_model \
      --url 'http://es-cn-w*****.ES.aliyuncs.com:9200' \
      --hub-model-id 'madhurjindal/autonlp-Gibberish-Detector-492513457' \
      --task-type text_classification \
      --es-username yourusername \
      --es-password  yourpassword \
      --es-model-id your-es-model-id-name \

Method 2: Upload a model in offline mode

Download a model in the huggingface_hub library to an on-premises machine and then upload the model to Alibaba Cloud Elasticsearch.

1. Run the following command in the ECS instance to download the huggingface_hub library:

   # Download the huggingface_hub library. huggingface_hub is a library provided by Hugging Face. This library is used to interact with model libraries of Hugging Face, and download, upload, and list models and resources in other libraries. 
   pip install huggingface_hub
   python3
   
   # Import the snapshot_download function to the Python interpreter.
   from huggingface_hub import snapshot_download
   # Use the snapshot_download function to download the snapshot of a storage library named FlagAlpha/Llama2-Chinese-13b-Chat.
   snapshot_download(repo_id="madhurjindal/autonlp-Gibberish-Detector-492513457")

2. Run the following command in the ECS instance to upload the model file to Alibaba Cloud Elasticsearch:

   eland_import_hub_model \
         --url 'http://es-cn-w*****.ES.aliyuncs.com:9200' \
         --hub-model-id '/model/.cache/huggingface/hub/models--madhurjindal--autonlp-Gibberish-Detector-492513457/snapshots/c068f552cdee957e45d8773db9f7158d43902244' \
         --task-type text_classification \
         --es-username yourusername \
         --es-password  yourpassword \
         --es-model-id madhurjindal-autonlp-gibberish-detector-492513457-offline \

The following table describes some parameters in the preceding commands.

Step 2: Deploy the model

1. Log on to the Kibana console of the Elasticsearch cluster. For more information, see Log on to the Kibana console.

2. Click the icon in the upper-left corner of the Kibana console. In the left-side navigation pane, choose Analytics > Machine Learning.

3. In the left-side navigation pane of the page that appears, choose Model Management > Trained Models.

4. Optional. In the upper part of the Trained Models page, click Synchronize your jobs and trained models. In the panel that appears, click Synchronize.

5. On the Trained Models page, find the uploaded model and click the icon in the Actions column to start the model.

   

6. In the dialog box that appears, configure the model and click Start.

   If a message indicating that the model is started is displayed in the lower-right corner of the page, the model is deployed.

   Note

   If the model cannot be started, the memory of the Elasticsearch cluster may be insufficient. You can start the model again after you upgrade the configuration of the Elasticsearch cluster. In the dialog box that prompts you about the failure, you can click View complete error message to view the failure cause.

Step 3: Test the model

1. On the Trained Models page, find the deployed model, click the icon in the Actions column, and then click Test model.

   

2. In the Test trained model panel, test the model and check whether the output result meets your expectations.