All Products
Search

This Product

    Platform For AI:Use custom components

    Document Center

    Platform For AI:Use custom components

    Last Updated:May 22, 2024

    You can create a custom algorithm component for specific scenarios. After you create a custom component, you can use the component in Machine Learning Designer of Platform for AI (PAI) to train models. This topic describes how to create and use a PyTorch-based component in Machine Learning Designer. The code directory of the PyTorch-based component is in an Object Storage Service (OSS) bucket.

    Limits

    You can run custom components only by using the public resource group of general computing resources in Machine Learning Designer.

    Prerequisites

    The execution code is uploaded to an OSS bucket. For more information, see Upload objects. In this example, the following sample code is used: main.py and requirements.txt.

    Procedure

    1. Create a custom PyTorch-based component.

      On the Create Component page, create a PyTorch-based component. For more information, see Create a custom component.

      • Execution Configuration image..png

        Parameter

        Description

        Job Type

        Select PyTorch.

        Image

        Select Community Image and then select the pytorch-training:1.12-cpu-py39-ubuntu20.04 image from the drop-down list.

        Code

        Select Mount OSS Path and enter the path of the OSS bucket in which the main.py and requirements.txt files are stored.

        Command

        Set the command to python main.py $PAI_USER_ARGS --train $PAI_INPUT_TRAIN --test $PAI_INPUT_TEST --model $PAI_OUTPUT_MODEL --checkpoints $PAI_OUTPUT_CHECKPOINTS && echo "job finished".

      • Pipeline and Parameter image..png

        Parameter

        Description

        Input

        Click the image..png icon to configure the following input pipelines:

        • Name: train and test

        • Source: OSS

        • Required: Yes and No

        • Description: training data and test data.

        Output

        Click the image..png icon to configure the following output pipelines:

        • Name: model and checkpoints

        • Storage: OSS

        • Required: Yes

        • Description: model data and algorithm checkpoints

        Parameter

        Click the image..png icon to configure the following parameters:

        • Parameter Name: param1, param2, param3, param4, and param5

        • Type: Int, Float, String, Bool, and Int

        • Default Value: 6, 0.3, test1, true, and 2

        • Constraint: Click Constraint on the right of the Default Value parameter to configure constraints based on on-screen instructions for each parameter.

        • Required: Yes, Yes, Yes, No, and No

      • Constraints image..png

        Parameter

        Description

        Enable Constraints

        Turn on Enable Constraints.

        Instance Type

        Select CPU and GPU.

        Multiple Instances

        Select Supported.

        Multiple GPUs

        Select Not Supported.

    2. Create a blank pipeline and go to the pipeline page. For more information, see Create a blank pipeline.

    3. In this example, the component name component is used. You can drag the component from the left-side component list to the canvas and use the component in the same manner as you use the built-in components in Machine Learning Designer. 7706a160936ede732315086bf3745c83..png

      After you create a custom component, the component is displayed in the algorithm tree in Machine Learning Designer of the current workspace.

      • The input and output pipelines that you configured for the custom component are used as the input and output ports of the component. 40ea48eb12349496759fb55cb857a050..png

      • The parameters that you configured for the custom component are used as the parameters of the component and are displayed in the right-side configuration pane. image..png

      • You can select an OSS path as the output path for each output port. image..png

      • The constraints that you configured for the custom component are used as parameters on the Tuning tab of the configuration pane. image..png

    If the custom component does not meet your business requirements, you can update the configuration of the custom component or add a component version on the Custom Components page. For more information, see Manage custom components.