Platform For AI:Best practices for using FeatureStore

Method 1: Use the PAI console

1. Create a project in FeatureStore.

   1. Log on to the PAI console. In the left-side navigation pane, choose Data Preparation > FeatureStore.

   2. On the FeatureStore page, select a workspace from the drop-down list and click Enter FeatureStore.

   3. Click Create Project. On the Create Project page, configure the project parameters that are described in the following table.

      Parameter	Description
      Name	Specify a custom name. In this example, fs_demo is used.
      Description	Enter a custom description.
      Offline Store (Offline Store)	Select the MaxCompute data store that you configured.
      Online Store (Online Store)	Select the Hologres data store that you configured.

   4. Click Submit.

2. Create feature entities.

   1. On the FeatureStore page, find the created project and click the project name to go to the Project Details page.

   2. On the Feature Entity tab, click Create Feature Entity. In the Create Feature Entity panel, configure the parameters that are described in the following table for the user feature entity.

      Parameter	Description
      Feature Entity Name	Specify a custom name. In this example, user is used.
      Join Id	Set this parameter to user_id.

   3. Click Submit.

   4. Click Create Feature Entity. In the Create Feature Entity panel, configure the parameters that are described in the following table for the item feature entity.

      Parameter	Description
      Feature Entity Name	Specify a custom name. In this example, item is used.
      Join Id	Set this parameter to item_id.

   5. Click Submit.

3. Create feature views.

   1. On the Feature View tab of the Project Details page, click Create Feature View. In the Create Feature View panel, configure the parameters that are described in the following table for the user feature view.

      Parameter	Description
      View Name	Specify a custom name. In this example, user_table_preprocess_all_feature_v1 is used.
      Type	Select Offline.
      Write Mode	Select Use Offline Table.
      Store	Select the MaxCompute data store that you configured.
      Feature Table	Select the prepared user table rec_sln_demo_user_table_preprocess_all_feature_v1.
      Feature Field	Select the user_id primary key field.
      Synchronize Online Feature Table	Select Yes.
      Feature Entity	Select user.
      Feature Lifecycle	Use the default value.

   2. Click Submit.

   3. Click Create Feature View. In the Create Feature View panel, configure the parameters that are described in the following table for the item feature view.

      Parameter	Description
      View Name	Specify a custom name. In this example, item_table_preprocess_all_feature_v1 is used.
      Type	Select Offline.
      Write Mode	Select Use Offline Table.
      Store	Select the MaxCompute data store that you configured.
      Feature Table	Select the prepared item table rec_sln_demo_item_table_preprocess_all_feature_v1.
      Feature Field	Select the item_id primary key field.
      Synchronize Online Feature Table	Select Yes.
      Feature Entity	Select item.
      Feature Lifecycle	Use the default value.

   4. Click Submit.

4. Create a label table.

   1. On the Label Table tab of the Project Details page, click Create Label Table. In the Create Label Table panel, configure the parameters that are described in the following table for the label table.

      Parameter	Description
      Store	Select the MaxCompute data store that you configured.
      Table Name	Select the prepared label table rec_sln_demo_label_table.

   2. Click Submit.

5. Create a model feature.

   1. On the Model Features tab of the Project Details page, click Create Model Feature. In the Create Model Feature panel, configure the parameters that are described in the following table for the model feature.

      Parameter	Description
      Model Feature Name	Specify a custom name. In this example, fs_rank_v1 is used.
      Select Feature	Select the user feature view and item feature view that you created.
      Label Table Name	Select the label table rec_sln_demo_label_table that you created.

   2. Click Submit.

   3. On the Model Features tab, find the model feature that you created and click the name of the model feature.

   4. On the Basic Information tab of the Model Feature Details panel, view the value of the Export Table Name parameter. In this example, the value of the Export Table Name parameter is fs_demo_fs_rank_v1_trainning_set. You can use this table to generate features and train a model.

6. Install FeatureStore SDK for Python. For more information, see the Use FeatureStore to manage features in a recommendation system section of this topic.

Method 2: Use FeatureStore SDK for Python

1. Log on to the DataWorks console.

2. In the left-side navigation pane, click Resource Groups.

3. On the Exclusive Resource Groups tab, find the resource group that you want to manage. Move the pointer over the icon in the Actions column and select O&M Assistant.

4. Click Create Command. In the Create Command panel, configure the command parameters that are described in the following table.

   Parameter	Description
   Command Name	Specify a custom name. In this example, install is used.
   Command Type	Select Manual Installation(You cannot run pip commands to install the third-party packages.).
   Command Content	/home/tops/bin/pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple https://feature-store-py.oss-cn-beijing.aliyuncs.com/package/feature_store_py-1.3.1-py3-none-any.whl
   Timeout	Specify a timeout period.

5. Click Create.

6. Click Run Command. In the message that appears, click Run.

7. Click Refresh to view the latest status of the command. If the state of the command changes to Successful, FeatureStore SDK is installed.

For more information about how to use FeatureStore SDK, see DSW Gallery.

FeatureStore SDK for Go

FeatureStore SDK for Go is open source. For more information, see aliyun-pai-featurestore-go-sdk.

1. Install FeatureStore SDK for Go

   Run the following code to install FeatureStore SDK for Go:

   go get github.com/aliyun/aliyun-pai-featurestore-go-sdk/v2

2. Use FeatureStore SDK for Go

   1. Run the following commands to initialize the client:

      accessId := os.Getenv("AccessId") 
      accessKey := os.Getenv("AccessKey") 
      regionId := "cn-hangzhou" projectName := "fs_test_ots" 
      
      client, err := NewFeatureStoreClient(regionId, accessId, accessKey, projectName)

      Note

      The FeatureStore client must run in a virtual private cloud (VPC) to allow FeatureStore SDK to directly connect to online data stores. For example, FeatureStore SDK can access a Hologres instance or a GraphCompute instance only over a specific VPC.

   2. Retrieve features from a feature view.

      // get project by name project, err := client.GetProject("fs_test_ots") 
      if err != nil {     
      	// t.Fatal(err) 
      } 
      // get featureview by name 
      user_feature_view := project.GetFeatureView("user_fea") 
      if user_feature_view == nil {     
      	// t.Fatal("feature view not exist") 
      } 
      
      // get online features 
      features, err := user_feature_view.GetOnlineFeatures([]interface{}{"100043186", "100060369"}, []string{"*"}, nil)

      In the preceding code, []string{"*"} indicates that all features in the feature view are retrieved. You can also specify the features that you want to retrieve.

      Sample response:

      [     
      {         
      "city":"Hefei",    "follow_cnt":1,         "gender":"male",         "user_id":"100043186"    
       },     {                     
       "city":"",         "follow_cnt":5,         "gender":"male",         "user_id":"100060369"     
      } 
      ]

   3. Retrieve feature data from a model feature.

      Each model feature can be associated with multiple feature entities. You can specify multiple join IDs to retrieve the corresponding features at a time.

      The following sample code specifies two join IDs: user_id and item_id. For each join ID, the same number of values must be specified.

      // get project by name project, err := client.GetProject("fs_test_ots") 
      if err != nil {     
      	// t.Fatal(err) 
      }
      
      // get ModelFeature by name 
      model_feature := project.GetModelFeature("rank") 
      if model_feature == nil {     
      	// t.Fatal("model feature not exist") 
      } 
      
      // get online features 
      features, err := model_feature.GetOnlineFeatures(map[string][]interface{}{"user_id": {"100000676", "100004208"}, "item_id":{"238038872", "264025480"}} )

      Sample response:

      [     
      {         
      "age":26,         "author":100015828,         "category":"14",         "city":"Shenyang",         "duration":63,         "gender":"male",         "item_id":"238038872",         "user_id":"100000676"    
       },     {         
      "age":23,         "author":100015828,         "category":"15",         "city":"Xi'an",         "duration":22,         "gender":"male",         "item_id":"264025480",         "user_id":"100004208"    
       }
       ]
      
      

      You can also specify a feature entity to retrieve its features.

      Sample response:

      [    
       {         
      "age":26,         "city":"Shenyang",         "gender":"male",         "user_id":"100000676"    
       },     {        
       "age":23,         "city":"Xi'an",         "gender":"male",         "user_id":"100004208"    
       } 
      ]

FeatureStore SDK for Java

FeatureStore SDK for Java is open source. For more information, see aliyun-pai-featurestore-java-sdk.

In this example, a Hologres data store is used.

1. Run the following code to load environment variables and initialize the service.

   public static String accessId = "";
   public static String accessKey = "";
   # Configure the host based on the region in which the service resides.
   public static String host = "";
   # Obtain the AccessKey ID and AccessKey secret from the configured environment variables.
   static {
       accessId = System.getenv("ACCESS_KEY_ID");
       accessKey = System.getenv("ACCESS_KEY_SECRET");
   }

2. Initialize the Configuration class, which includes the region ID, AccessKey ID, AccessKey secret, and project name.

   Configuration cf = new Configuration("cn-hangzhou",Constants.accessId,Constants.accessKey,"ele28");
   cf.setDomain(Constants.host);// By default, the VPC environment is used.

3. Initialize the client.

   ApiClient apiClient = new ApiClient(cf);
   # Initialize the FeatureStore client.
   FeatureStoreClient featureStoreClient = new FeatureStoreClient(apiClient);

4. Obtain the project name. In this example, the project is named ele28.

   Project project=featureStoreClient.getProject("ele28");
   if(project==null){
       throw new RuntimeException("Project not found");
   }

5. Retrieve the feature view of the project. In this example, the feature view is named mc_test.

   FeatureView featureView=project.getFeatureView("mc_test");
   if (featureView == null) {
       throw  new RuntimeException("FeatureView not found");
   }

6. Retrieve feature data from a real time feature view.

   Map<String,String> m1=new HashMap<>();
   m1.put("gender","gender1"); // Configure an alias.                                                                                                                                                                                                 user_id='100027781'(FS_INT64) age='28'(FS_INT64) city='null'(FS_STRING) item_cnt='0'(FS_INT64) follow_cnt='0'(FS_INT64) follower_cnt='2'(FS_INT64) register_time='1697641608'(FS_INT64) tags='0'(FS_STRING) gender1='female'(FS_STRING) ---------------

   You can use String[]{"*"} to retrieve all features from the feature view. You can also specify the features that you want to retrieve.

   FeatureResult featureResult1=featureView.getOnlineFeatures(new String[]{"100017768","100027781","100072534"},new String[]{"*"},m1);

   Sample response:

   while(featureResult1.next()){
       System.out.println("---------------");
       # Specify the feature name.
       for(String m:featureResult1.getFeatureFields()){
           System.out.print(String.format("%s=%s(%s) ",m,featureResult1.getObject(m),featureResult1.getType(m)));
       }
       System.out.println("---------------");
   }   

   Sample response:

   ---------------
   user_id='100017768'(FS_INT64)  age='28'(FS_INT64)  city='Dongguan'(FS_STRING) item_cnt='1'(FS_INT64)  follow_cnt='1'(FS_INT64)  follower_cnt='0'(FS_INT64) register_time='1697202320'(FS_INT64)  tags='1,2'(FS_STRING)  gender1='female'(FS_STRING) 
   ---------------

7. Retrieve a model.

   Model model=project.getModelFeature("model_t1");
   if(model==null){
       throw new RuntimeException("Model not found");
   }

   1. Retrieve data from a model feature.

      The following sample code specifies two join IDs: user_id and item_id. The number of values that are specified for user_id must be the same as that for item_id. In this example, only one value is specified for user_id and item_id.

      Map<String, List<String>> m2=new HashMap<>(); 
      m2.put("user_id",Arrays.asList("101683057"));
      m2.put("item_id",Arrays.asList("203665415"));

   2. Retrieve all the feature data of the user feature entity that is associated with a model feature.

      FeatureResult featureResult2 = model.getOnlineFeaturesWithEntity(m2,"user");

      Sample response:

      ---------------
      user_id='101683057' age='28' city='Shenzhen' follower_cnt='234' follow_cnt='0' gender='male' item_cnt='0' register_time='1696407642' tags='2' item_id='203665415' author='132920407' category='14' click_count='0' duration='18.0' praise_count='10' pub_time='1698218997' title='#Idiom story' 
      ---------------

FeatureStore SDK for C++

FeatureStore SDK for C++ is integrated with the EasyRec processor that is deployed as a scoring service. FeatureStore SDK for C++ is optimized for feature extraction, cache management, and read operations to provide high-performance and low-latency solutions for large-scale recommendation scenarios. FeatureStore SDK for C++ provides the following capabilities:

• Reduced memory usage: The memory usage is greatly reduced when you process a large amount of complex feature data by using FeatureStore SDK for C++, especially in the case of high feature load.

• Accelerated feature extraction: FeatureStore SDK for C++ extracts feature data from a MaxCompute data store to the EAS cache instead of extracting feature data from an online data store such as a Hologres instance or a GraphCompute instance. This reduces the amount of time that is required to load feature data. In addition, MaxCompute provides higher stability and better extensibility, which reduces the impacts that are caused by scaling up online storage.

• Improved model scoring: FeatureStore SDK for C++ further improves the tp100 scoring metric, supports more stable response time, and reduces the number of timeout requests. This improves the reliability of recommendation services and user experience.