A system exception occurs on Linux instances after SSH logon due to Ulimit - Elastic Compute Service

Disclaimer: This article may contain information about third-party products. Such information is for reference only. Alibaba Cloud does not make any guarantee, express or implied, with respect to the performance and reliability of third-party products, as well as potential impacts of operations on the products.

Problem description

After you log on to a Linux instance through SSH, the following error message is displayed when you use the command line interface (CLI).

-bash: fork: retry: Resource temporarily unavailable.
pam_limits(sshd:session):could not sent limit for 'nofile':operaton not permitted.
Permission denied.

Cause

The maximum number of files that a Shell process can contain is higher than the quota contained in the Ulimit system environment of the server.

Solution

Alibaba Cloud reminds you that:

Before you perform operations that may cause risks, such as modifying instance configurations or data, we recommend that you check the disaster recovery and fault tolerance capabilities of the instances to ensure data security.

If you modify the configurations and data of instances including but not limited to ECS and RDS instances, we recommend that you create snapshots or enable RDS log backup.

If you have authorized or submitted security information such as the logon account and password in the Alibaba Cloud Management console, we recommend that you modify such information in a timely manner.

Note: the configurations and descriptions in this article are tested in CentOS 7.6 64-bit and CentOS 5 64-bit operating systems. The operating system configurations of other types and versions may be different. For more information, see the official documentation of the operating system.

Based on the preceding causes, you must modify the limits.conf file in the CentOS system version to permanently modify the Ulimit system environment limit. In CentOS 6 and later versions, the X-nproc.conf file manages Ulimit system environment limits. Therefore, the operation instructions for different versions are as follows.

Note: the X-nproc.conf file has different prefixes in different system versions, such as 90-nproc.conf in CentOS 6 and 20-nproc.conf in CentOS 7.

Versions earlier than CentOS 6

Take a CentOS 5 64-bit system version as an example. The procedure is as follows:

Passmanagement terminallog on to the instance and run the following command to view the current Ulimit system resource limit and check whether to enable the system resource limit because the value of the open system resource limit is too small.
```
cat /etc/security/limits.conf
```
The command output is as follows: No Limits of system resources are set by default.
Note:
- System users <domain> need to limit, you can use * Replace all users.
- <type> have soft, hard and - Three parameters.
  - soft: the <value> value that is active in the current system.
  - hard refers to the maximum <value> value set in the system.
  - soft the limit cannot be higher than hard limit, with - The values of soft and hard are set at the same time.
- <item> the types of resources that need to be restricted. The following table describes the resource type parameters.
  - core indicates limiting the size of the kernel file.
  - rss refers to the maximum persistent setting size.
  - nofile indicates the maximum number of open files.
  - noproc indicates the maximum number of processes.
If the system enables and configures the system resource limit, edit the limits.conf file and select the resource type code operations to comment out and modify or delete the noproc or nofile parameter limit based on the on-site feedback results.

Note: take the note noproc as an example.
1. Run the following command to back up the limits.conf file:
```
cp -af /etc/security/limits.conf /root/limits.conf_bak
```
2. Run the following command to enter the edit mode:
```
vim /etc/security/limits.conf
```
3. The resource type code that the nofile parameter limits. The command output is as follows:
After saving and exiting the editing mode, restart the Linux instance.

CentOS 6 and later versions

Take a CentOS 7.6 64-bit version as an example. The procedure is as follows:

Log on to the instance through the management terminal and run the following command to view the current Ulimit system resource limit.
```
cat /etc/security/limits.d/20-nproc.conf
```
A similar output is displayed. The following figure shows that system resource throttling is enabled and the maximum number of connection processes is 4096 for all users except the root user. Verify with the on-site environment.
Follow steps in earlier versions than CentOS 6 to modify the CentOS file.

Note: back up these files before you perform the preceding operations.
After the preceding operations are completed and the configuration is saved, restart the corresponding Linux instance.

Documentation

For more information about SSH remote connection issues, see guidelines for troubleshooting remote logon failures of Linux instances through SSH and SSH principles.

Elastic Compute Service:A system exception occurs on Linux instances after SSH logon due to Ulimit