You can access the web UIs of open source components by using the links provided in the E-MapReduce (EMR) console. You can also create an SSH tunnel to access the web UIs of open source components. You can use an SSH tunnel and a SOCKS proxy server to access the web UIs of open source components without exposing the internal network structure of a cluster.
For information about the methods that you can use to access the web UIs of open source components, see Access the web UIs of open source components.
Prerequisites
An EMR cluster is created. For more information, see Create a cluster.
The client that you want to use is connected to the master node of the desired EMR cluster. For more information, see Associate an EIP with an ECS instance.
Create an SSH tunnel to enable local port forwarding
If you use this method to access the web UI of an open source component, you cannot go to the job details page.
You can use the local port forwarding method to forward data on a port of the master node to the local port and access the web application interface running on the master node.
1. Create an SSH tunnel
Open the CLI on your on-premises machine and connect to the master node of the desired EMR cluster in SSH mode. You can connect to the master node by using the password or the key that you configured when you create the EMR cluster.
Connect to the master node by using a key
ssh -i <Key file path> -N -L <Local port>:<Name of the master node>:<Component port> root@<Public IP address of the master node>
Parameter description:
-i
: the path of the private key file used for identity authentication.-N
: used only to enable port forwarding and create an SSH tunnel without the need to run commands.-L
: Local port forwarding is enabled. You can specify a local port to forward data to the remote port hosted on your on-premises web server of the master node.<Key file path>
: the path in which the key file is stored. For more information, see Manage SSH key pairs.<Local port>
: You can use an unoccupied port on your on-premises server. Example: 8156.<Name of the master node>
: the name of the master node. You can view the name on the Nodes tab of the cluster. Example: master-1-1. For more information, see Obtain the public IP address and the name of a node.<Component port>
: the port of an open source component. For more information about component ports, see Common ports of open source components. Example: 8088.<Public IP address of the master node>
: the public IP address of the master node. You can view the address on the Nodes tab of the cluster. For more information, see Obtain the public IP address and the name of a node.
Connect to the master node by using a username and password
ssh -N -L <Local port>:<Name of the master node>:<Component port> root@<Public IP address of the master node>
Parameter description:
-N
: used only to enable port forwarding and create an SSH tunnel without the need to run commands.-L
: Local port forwarding is enabled. You can specify a local port to forward data to the remote port hosted on your on-premises web server of the master node.<Local port>
: You can use an unoccupied port on your on-premises server. Example: 8156.<Name of the master node>
: the name of the master node. You can view the name on the Nodes tab of the cluster. Example: master-1-1. For more information, see Obtain the public IP address and the name of a node.<Component port>
: the port of an open source component. For more information about component ports, see Common ports of open source components. Example: 8088.<Public IP address of the master node>
: the public IP address of the master node. You can view the address on the Nodes tab of the cluster. For more information, see Obtain the public IP address and the name of a node.
Keep your on-premises server running after you create the SSH tunnel. No responses are returned.
2. Access the web UI of an open source component
Enter http://localhost:<Local port>
in the address bar of a browser and press Enter to access a specific web UI. <Local port>
is the local port that you configured in Step 1. For example, to access the web UI of YARN, enter http://localhost:8156
.
If you want to access the web UIs of multiple open source components at the same time, you must repeat the preceding steps to enable another local port for forwarding.
Create an SSH tunnel to enable dynamic port forwarding
After you create an SSH tunnel to allow communication between a port of your on-premises server and the master node of an EMR cluster, you can use a SOCKS proxy server that listens on the port to forward port data to the master node of the EMR cluster. Then, you can access the web UI of an open source component on the master node in the EMR cluster from your on-premises server.
1. Create an SSH tunnel
Open the CLI on your on-premises machine and connect to the master node of the desired EMR cluster in SSH mode. You can connect to the master node by using the password or the key that you configured when you create the EMR cluster.
Connect to the master node by using a key
ssh -i <Key file path> -N -D <Port> root@<Public IP address of the master node>
Parameter description:
-i
: the path of the private key file used for identity authentication.-N
: used only to enable port forwarding and create an SSH tunnel without the need to run commands.-D
: Dynamic port forwarding is enabled. Start the SOCKS proxy process to listen on the port.<Key file path>
: the path in which the key file is stored. For more information, see Manage SSH key pairs.<Local port>
: You can use an unoccupied port on your on-premises server. Example: 8157.<Public IP address of the master node>
: the public IP address of the master node. You can view the address on the Nodes tab of the cluster. For more information, see Obtain the public IP address and the name of a node.
Connect to the master node by using a username and password
ssh -N -D <Port> root@<Public IP address of the master node>
Parameter description:
-N
: used only to enable port forwarding and create an SSH tunnel without the need to run commands.-D
: Dynamic port forwarding is enabled. Start the SOCKS proxy process to listen on the port.<Local port>
: You can use an unoccupied port on your on-premises server. Example: 8157.<Public IP address of the master node>
: the public IP address of the master node. You can view the address on the Nodes tab of the cluster. For more information, see Obtain the public IP address and the name of a node.
Keep your on-premises server running after you create the SSH tunnel. No responses are returned.
2. Configure the Google Chrome browser
After you enable dynamic port forwarding, you can use one of the following methods to configure a browser proxy:
Use the CLI
Open the CLI on your on-premises machine and go to the installation directory of the Google Chrome browser client.
Operating system
Default installation directory of Google Chrome
macOS X
/Applications/Google\ Chrome.app/Contents/macOS
Linux
/usr/bin/google-chrome
Windows
Default installation directories of Google Chrome for 64-bit and 32-bit systems:
C:\Program Files\Google\Chrome\Application\
C:\Program Files (x86)\Google\Chrome\Application\
NoteThe default installation directory of Google Chrome varies based on the operating system.
Configure Google Chrome to use a local SOCKS proxy and start the Google Chrome browser. Use the port that you enabled when you create an SSH tunnel in Step 1.
NoteIn this example, Port 8157 is used.
macOS X
./Google\ Chrome --proxy-server="socks5://localhost:8157" --host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE localhost" --user-data-dir=/tmp/
Linux
chrome --proxy-server="socks5://localhost:8157" --host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE localhost" --user-data-dir=/tmp/
Windows
chrome --proxy-server="socks5://localhost:8157" --host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE localhost" --user-data-dir=c:\\tmppath\\
Use a Google Chrome extension
Extensions allow you to easily manage and use proxies in your web browser. You can use an extension to browse web pages and access web UIs at the same time.
Add the Google Chrome extension SwitchyOmega.
On the SwitchyOmega page, click New profile in the left-side navigation pane. In the New Profile dialog box, enter a profile name, such as SSH tunnel, in the Profile name field, select PAC Profile, and then click Create.
Enter the following content in the PAC Script editor:
NoteIn this example, Port 8157 is used.
function regExpMatch(url, pattern) { try { return new RegExp(pattern).test(url); } catch (ex) { return false; } } function FindProxyForURL(url, host) { if (shExpMatch(url, "*localhost*")) return "SOCKS5 localhost:8157"; if (shExpMatch(url, "*emr-header*")) return "SOCKS5 localhost:8157"; if (shExpMatch(url, "*emr-worker*")) return "SOCKS5 localhost:8157"; if (shExpMatch(url, "*master*")) return "SOCKS5 localhost:8157"; if (shExpMatch(url, "*core*")) return "SOCKS5 localhost:8157"; return 'DIRECT'; }
In the left-side navigation pane, click Apply changes to complete the configuration.
In the upper-right corner of the browser, select the SSH tunnel that you created.
3. Access the web UI of an open source component
Enter http://<Name of the master node>:<Port>
in the address bar of the browser and press Enter to access a specific web UI. For example, to access the web UI of YARN, enter http://master-1-1:8088
.
Parameter description:
<Name of the master node>
: the name of the master node. You can view the name on the Nodes tab of the cluster. Example: master-1-1. For more information, see Obtain the public IP address and the name of a node.<Port>
: the local port that you configured in Step 1.
If you want to access the web UIs of multiple open source components at the same time, change the port number.
References
For information about how to access the web UIs of open source components in the EMR console, see Access the web UIs of open source components in the EMR console.
For information about service-related issues, see FAQ.