Gateways are used to deploy the Apache Livy service, so that tools, such as Airflow and Jupyter, can be used to submit jobs to E-MapReduce (EMR) Serverless Spark over the Apache Livy interface. This topic describes how to create a gateway.
Create a gateway
After a gateway is created, you can select it as a compute when you create an SQL job.
Go to the Compute page.
Log on to the EMR console.
In the left-side navigation pane, choose .
On the Spark page, click the name of the workspace that you want to manage.
In the left-side navigation pane of the EMR Serverless Spark page, click Compute.
On the Compute page, click Gateway.
On the Gateway tab, click Create Gateway.
On the Create Gateway page, configure the parameters described in the following table and click Create.
Parameter
Description
Name
The gateway name. The name can contain lowercase letters, digits, and hyphens (-). It must start and end with a letter or digit.
Gateway Resources
The resource configurations. Default value:
1 CPU, 4 GB. You cannot change the value of this parameter.API Type
The interface type. You can select only Livy.
Livy Version
The Livy version. Default value: 0.8.0.
Engine Version
The engine version of the gateway. For more information about engine versions, see Engine versions.
Associated Queue
The queue in which the gateway is deployed. To submit a Spark job by using a gateway, the job submitter must use the identity of the gateway creator.
Automatic Stop
By default, this feature is disabled.
If you enable this feature for a gateway, the system automatically stops the gateway if no activity is detected in the gateway in the last 45 minutes.
Authentication Method
The authentication mode. You can select only Token.
After you create a gateway, you must generate a unique authentication token for the gateway. This way, you can use the token for access control and authentication when you submit requests over the gateway. For more information about how to generate a token, see the Manage tokens section of this topic.
Manage tokens
To use the token of a gateway, add --header 'x-acs-spark-livy-token: token' to the headers of the requests that are submitted over the gateway.
On the Gateways page, find the gateway whose token you want to manage and click Tokens in the Actions column.
On the Tokens tab, click Create Token.
In the Create Token dialog box, configure the parameters described in the following table and click OK.
Parameter
Description
Name
The token name.
Token Expired
The validity period of the token. The validity period must be greater than or equal to 1 day. By default, this parameter is enabled and set to 365 days.
Copy the token.
ImportantAfter you create a token, you must immediately copy the token. You cannot view the token later. If your token expires or is lost, reset the token or create a new token.
References
For information about how to submit jobs to EMR Serverless Spark by using the Apache Livy interface, see Use the LivyOperator of Apache Airflow to submit a job.