DataWorks:Create an EMR Trino node

Limits

• You can run EMR Trino tasks only on a serverless resource group.

• If you want to manage metadata for a DataLake or custom cluster in DataWorks, you must configure EMR-HOOK in your cluster first. If you do not configure EMR-HOOK in your cluster, metadata cannot be displayed in real time, audit logs cannot be generated, and data lineages cannot be displayed in DataWorks. EMR governance tasks also cannot be run. For information about how to configure EMR-HOOK, see Use the Hive extension feature to record data lineage and historical access information.

• If you enable Lightweight Directory Access Protocol (LDAP) authentication for Trino, you must log on to the master node of the EMR cluster and download the keystore file in the /etc/taihao-apps/trino-conf directory. Then, perform the following steps to upload the keystore file: Log on to the DataWorks console. In the left-side navigation pane, click Management Center. On the page that appears, select the desired workspace and click Go to Management Center. In the left-side navigation pane of the page that appears, click Cluster Management. Then, find the desired cluster and click the Account Mappings tab. In the upper-right corner of the tab, click Edit Account Mappings. On the page that appears, click Upload Keystore File to upload the keystore file.

(Optional) Select an EMR cluster

If multiple EMR clusters are registered to your workspace, you must select one from the Engine Instance EMR drop-down list in the upper part of the configuration tab of the node based on your business requirements. If only one EMR cluster is registered to your workspace, the EMR cluster is used to develop tasks.

Configure connectors

• Before you query data from MySQL tables, you must configure the built-in MySQL connector. For more information, see MySQL connector.

• Before you query data from Hive tables, you must configure the built-in Hive connector. For more information, see Hive connector.

• Before you query data from tables of other data sources, you must configure the related connector of the data source. For more information, see Configure a connector.

Develop SQL code

In the SQL editor, enter the code for the node. Example:

-- Use method
-- SELECT * FROM <catalog>.<schema>.<table>;
-- Parameter description
-- <catalog> specifies the name of the desired data source. 
-- <schema> specifies the name of the database that you want to use. 
-- <table> specifies the name of the table that you want to query. 
-- For example, you can execute the following statement to query data from the hive_table table in the default database of the Hive data source.
-- Query data from a Hive table.
SELECT * FROM hive.default.hive_table;

-- For example, you can execute the following statement to query data from the rt_user table in the rt_data database of the MySQL data source. 
-- Query data from a MySQL table.  
SELECT * FROM mysql.rt_data.rt_user;

-- Perform a join query on the Hive and MySQL tables.
SELECT DISTINCT a.id, a.name,b.rt_name FROM hive.default.hive_table a INNER JOIN mysql.rt_data.rt_user b ON a.id = b.id;

-- Use scheduling parameters to query data from a specific Hive table.
SELECT * FROM hive.default.${table_name};

Run the Trino task

1. In the toolbar, click the icon. In the Parameters dialog box, select the desired resource group from the Resource Group Name drop-down list and click Run.

   Note

   • If you want to access a data source over the Internet or a virtual private cloud (VPC), you must use the resource group for scheduling that is connected to the data source. For more information, see Network connectivity solutions.

   • If you want to change the resource group in subsequent operations, you can click the icon to change the resource group in the Parameters dialog box.

   • If you use an EMR Trino node to query data, up to 10,000 data records can be returned, and the total size of the returned data records cannot exceed 10 MB.

2. Click the icon to save the SQL statements.

(Optional) Configure advanced parameters

If you want to modify the execution mode of SQL statements, click Advanced Settings in the right-side navigation pane and configure the parameters described in the following table.