Platform For AI:Segment and pre-label images

Background information

Grounded SAM is one of the best zero-shot vision models. The model can detect, segment, and generate images based on input text. Grounded SAM is developed by a Chinese professional team led by IDEA, based on the SAM model of Meta and three other powerful zero-shot models.

Grounded SAM consists of two models: Grounding DINO and SAM. SAM is a zero-shot segmentation model that can generate masks for arbitrary objects in an image or a video, including objects and images that do not appear during the training process. Grounding DINO is an advanced zero-shot detector that can find arbitrary objects in an image based on text description.

Go to the details page of the model

1. Go to the QuickStart page.

   1. Log on to the Machine Learning Platform for AI (PAI) console.

   2. In the left-side navigation pane, click Workspaces. On the Workspaces page, find the workspace that you want to manage and click the name of the workspace. The Workspace Details page appears.

   3. In the left-side navigation pane of the Workspace Details page, click QuickStart.

2. On the QuickStart page, click image-segmentation in the CV section. In the model list that is displayed on the right, find and click the modelscope_segment-anything model to go to the details page of the model.

   

Deploy the model as a model service

QuickStart integrates the features of Elastic Algorithm Service (EAS). You can directly deploy Grounded SAM as an online service and generate a web UI for the model. You can use the web UI to easily experience the image segmentation features of Grounded SAM. To deploy the model as an online service, perform the following steps:

1. On the details page of the model, click Deploy in the upper-right corner.

   

2. Verify the parameter settings in the Model Service Information and Resource Deployment Information sections.

   QuickStart specifies a service name and presets the computing resources that are required to deploy a model based on characteristics of the model. In this example, the default parameter settings are used. You can also modify the parameter settings based on your business requirements. For more information about how to configure the parameters, see the "Deploy and debug a model" section of the Deploy and train models topic.

   

3. Click Deploy. In the Billing Notification message, click OK.

   The details page of the service appears. On the Service details tab, you can view the deployment status of the model service in the Basic information section. You can also view the deployed service on the Inference Service tab of the Elastic Algorithm Service (EAS) page. For more information about how to go to the EAS-Online Model Services page, see Model service deployment by using the PAI console. The deployment process takes about 10 minutes due to the large size of the model. If the value of the Status parameter changes to In operation, the model service is deployed.

Then, you can start the web UI of the model to experience the inference features of Grounded SAM. For more information, see the Use Grounded SAM section of this topic.

Use Grounded SAM

1. In the Web Application section of the Service details tab, click View Web App to start the web UI.

2. On the web UI, perform model inference.

   Grounded SAM provides multiple modes. In the Image section on the left side of the page, upload the image that you want to segment. In the lower part of the page, select a mode from the task_type drop-down list, set the Text Prompt parameter, and then click Run to segment the image.

   In this example, the following modes are used: automatic and scribble. The two modes allow you to experience the features of Grounded SAM in image segmentation.

   • automatic

     Select automatic from the task_type drop-down list. Leave the Text Prompt parameter empty. Upload an image and click Run to automatically segment the image without interactions.

   • scribble

     Select scribble from the task_type drop-down list. Click the part of the image that you want to segment and leave the Text Prompt parameter empty. Then, click Run. The image is segmented.