All Products
Search
Document Center

E-MapReduce:Use a StarRocks shared-nothing instance

Last Updated:Dec 13, 2024

StarRocks shared-nothing instances are suitable for business scenarios in which high query performance is required, such as online analytical processing (OLAP) multidimensional analysis, high-concurrency queries, and real-time data analysis. StarRocks shared-nothing instances store data in cloud disks or local disks to ensure high data read and write efficiency. This topic describes how to create and use a StarRocks shared-nothing instance.

Prerequisites

  • An Alibaba Cloud account is created, and real-name verification is performed for the account.

  • If you use a RAM user, make sure that the AliyunEMRStarRocksFullAccess policy is attached to the RAM user.

    Note

    To create and manage StarRocks instances, you must attach the AliyunEMRStarRocksFullAccess policy to your account.

Precautions

The runtime environment of the code is managed and configured by the owner of the environment.

Step 1: Create a StarRocks shared-nothing instance

  1. Go to the Instances tab of the E-MapReduce (EMR) console.

    1. Log on to the EMR console.

    2. In the left-side navigation pane, choose EMR Serverless > StarRocks.

    3. In the top navigation bar, select a region based on your business requirements.

  2. On the Instances tab, click Create Instance.

  3. In the dialog box that appears, select Shared-nothing.

  4. On the E-MapReduce Serverless StarRocks page, configure the parameters. The following table describes the parameters.

    Parameter

    Example

    Description

    Billing Method

    Pay-as-you-go

    The billing method. Valid values: Subscription and Pay-as-you-go.

    Region

    China (Hangzhou)

    The geographic location where the instance resides.

    Important

    You cannot change the region after the instance is created. Select a region based on your business requirements.

    Zone

    Zone I

    The zone where the instance resides.

    Important

    You cannot change the zone after the instance is created. Select a zone based on your business requirements.

    VPC

    vpc_Hangzhou/vpc-bp1f4epmkvncimpgs****

    The virtual private cloud (VPC) in which the instance is deployed. Select a VPC in the current region. If no VPC is available, click Create VPC to create a VPC. After you create a VPC, click the 刷新图标 icon and select the created VPC.

    vSwitch

    vsw_i/vsw-bp1e2f5fhaplp0g6p****

    The vSwitch of the instance. Select a vSwitch in the specified zone. If no vSwitch is available, click Create vSwitch to create a vSwitch. After you create a vSwitch, click the 刷新图标 icon and select the created vSwitch.

    Instance Edition

    Standard Edition

    The edition of the instance. Valid values: Basic Edition and Standard Edition. For more information, see the Instance editions section of the "Create an instance" topic.

    Version

    3.1

    The community version of StarRocks.

    BE Specification Type

    Standard Specifications

    The specification type of BEs in an instance.

    • Basic Edition: Only Standard Specifications is supported.

    • Standard Edition: Standard Specifications, High-performance Storage, and High-specification Storage are supported.

    BE Specifications

    8 CU

    • Basic Edition: The value of the BE Specifications parameter is fixed to 8. Unit: CUs. Default configurations of the BE Storage Configurations parameter:

      • Data Disk Type: Only ESSD PL1 (Recommended) can be selected.

      • Data Disk Size: The value is fixed to 50. Unit: GB.

      • Data Disk Quantity: The value is fixed to 1.

    • Standard Edition:

      • Standard Specifications: The value of the BE Specifications parameter can be 8, 16, 32, and 64. Unit: CUs. You must also configure the BE Storage Configurations parameter. Involved parameters:

        • Data Disk Type: Valid values: ESSD PL0, ESSD PL1 (Recommended), ESSD PL2, and ESSD PL3. For more information, see ESSDs.

        • Data Disk Size: Valid values: 100 to 65000. Unit: GB. Step size: 100.

        • Data Disk Quantity Valid values: 1 to 14. step size: 1.

      • High-performance Storage or High-specification Storage: Select a value from the BE Specifications drop-down list.

    Number of BEs

    3

    The number of backend nodes (BEs) on an instance.

    • If you set the Instance Edition parameter to Basic Edition, the number of BEs is 1.

    • If you set the Instance Edition parameter to Standard Edition, the number of BEs ranges from 3 to 100. Step size: 1.

    HA

    On

    By default, this switch is turned on. You can turn on HA only if you set the Instance Edition parameter to Standard Edition. If you turn on HA, the system increases the number of frontend nodes (FEs) from 1 to 3 to reduce the risk of failures.

    Important

    We strongly recommend that you turn on HA in the production environment.

    FE Resource Configuration

    • Specifications: 8 CU

    • Storage Size: 100 GB

    • Number of Nodes: 3

    • If you set the Instance Edition parameter to Basic Edition, the FE resource configurations are fixed. The number of CUs is 4, the storage size is 50 GB, and the number of FEs is 1.

    • If you set the Instance Edition parameter to Standard Edition, the number of FEs varies based on whether you turn on HA. If you do not turn on HA, the number of FEs is 1. If you turn on HA, the number of FEs is 3.

    Total Resource Configuration

    FE Computing Resources: 24 CU

    Storage Resources: ESSD PL1 (Recommended) 100 GB

    BE Computing Resources: 24 CU Storage Resources: ESSD PL1 (Recommended) 300 GB

    The details of the FE and BE resources, and the data storage.

    Instance Name

    Custom instance name

    The name of the instance. The name must be 1 to 64 characters in length and can contain only letters, digits, hyphens (-), and underscores (_).

    Administrator

    admin

    The administrator that is used to manage the StarRocks instance. The value is fixed to admin.

    Password and Confirm Password

    Custom password

    The password of the administrator. You must record the password. The password is required when you manage and use the instance.

    RAM Role

    AliyunEMRStarRocksAccessingOSSRole

    The RAM role that allows the instance to access OSS.

    (Optional) Advanced Settings > Resource Group

    An existing resource group

    You can use the default resource group, select an existing resource group, or click Create Resource Group to create a resource group in the Resource Management console. For information about how to create a resource group, see Create a resource group.

    Note

    Resource Group allows you to sort resources owned by your Alibaba Cloud account into groups. This simplifies resource and permission management within your Alibaba Cloud account. For more information, see What is Resource Group?

    (Optional) Advanced Settings > Tags

    Custom tag

    You can add a tag when you create an instance or add a tag on the Instance Details tab after an instance is created. Tags help you identify and manage instance resources.

  5. Read the terms of service, select the check box for Terms of Service, and then click Create Instance. Then, complete the payment.

    After the payment is complete, you can view the creation progress on the Instances tab. The instance is successfully created if the instance is in the Running state.

Step 2: Connect to the StarRocks instance

  1. On the Instances tab, find the instance that you want to connect and click Connect in the Actions column.

    You can use other methods to connect to a StarRocks instance. For more information, see the topics in the Instance connection directory.

  2. Connect to the StarRocks instance.

    1. On the New Connection tab of the page that appears, configure parameters. The following table describes the parameters.

      image.png

      Parameter

      Example

      Description

      Region

      China (Hangzhou)

      Select the region in which the instance resides.

      Instance

      StarRocks_Serverlesss

      Select the name of the instance.

      Connection Name

      Connection_Serverlesss

      The name of the connection. Enter a custom name.

      The name must be 1 to 64 characters in length and can contain only letters, digits, hyphens (-), and underscores (_).

      Username

      Default username or custom username

      The default initial username is admin. You can use the default username or create a username to connect to the instance.

      Password

      Custom password

      The password of the username that is created for the instance.

    2. Click Test Network Connectivity.

    3. After the network connectivity test is successful, click OK.

      In the left-side navigation pane, click SQL Editor. You can create a query on the Queries tab and perform the query. For more information, see Use EMR StarRocks Manager to connect to an EMR Serverless StarRocks instance.

Step 3: Perform SQL queries

On the Queries tab of the SQL Editor page, click the image.png icon to create a query, enter the following code in the SQL editor, and then click Run.

/**Create a database**/
CREATE DATABASE IF NOT EXISTS load_test;

/**Use the database**/
USE load_test;

/**Create a table**/
CREATE TABLE insert_wiki_edit
(
    event_time DATETIME,
    channel VARCHAR(32) DEFAULT '',
    user VARCHAR(128) DEFAULT '',
    is_anonymous TINYINT DEFAULT '0',
    is_minor TINYINT DEFAULT '0',
    is_new TINYINT DEFAULT '0',
    is_robot TINYINT DEFAULT '0',
    is_unpatrolled TINYINT DEFAULT '0',
    delta INT SUM DEFAULT '0',
    added INT SUM DEFAULT '0',
    deleted INT SUM DEFAULT '0'
)
AGGREGATE KEY(event_time, channel, user, is_anonymous, is_minor, is_new, is_robot, is_unpatrolled)
PARTITION BY RANGE(event_time)
(
    PARTITION p06 VALUES LESS THAN ('2015-09-12 06:00:00'),
    PARTITION p12 VALUES LESS THAN ('2015-09-12 12:00:00'),
    PARTITION p18 VALUES LESS THAN ('2015-09-12 18:00:00'),
    PARTITION p24 VALUES LESS THAN ('2015-09-13 00:00:00')
)
DISTRIBUTED BY HASH(user) BUCKETS 10
PROPERTIES("replication_num" = "1");

/**Insert data**/
INSERT INTO insert_wiki_edit VALUES("2015-09-12 00:00:00","#en.wikipedia","GELongstreet",0,0,0,0,0,36,36,0),("2015-09-12 00:00:00","#ca.wikipedia","PereBot",0,1,0,1,0,17,17,0);

/**Query data**/
select * from insert_wiki_edit;

The following figure shows the output.

image.png

Step 4: Perform performance tests

For more information, see Test description.

(Optional) Step 5: Release the instance

Important

After you release an instance, the instance and its resources are deleted. This operation is irreversible. Proceed with caution.

If you no longer require an instance, you can release the instance to prevent unnecessary costs.

  1. On the Instances tab, find the instance that you want to release and click Release in the Actions column.

  2. In the message that appears, click OK.

Contact us

If you have any questions when you use a StarRocks instance, you can join the DingTalk group whose ID is 24010016636 for technical support.