Elastic Compute Service:The system prompts "fatal: mm_request_send: write: Broken pipe" error when the SSH service runs abnormally caused by viruses

Disclaimer: This article may contain information about third-party products. Such information is for reference only. Alibaba Cloud does not make any guarantee, express or implied, with respect to the performance and reliability of third-party products, as well as potential impacts of operations on the products.

Problem description

This article describes how to handle the "fatal: mm_request_send: write: Broken pipe" error when the SSH service runs abnormally caused by viruses.

Possible cause

This problem may be caused by viruses such as udev-fall that affect the normal operation of the SSH service.

Solution

Alibaba Cloud reminds you that:

• Before you perform operations that may cause risks, such as modifying instance configurations or data, we recommend that you check the disaster recovery and fault tolerance capabilities of the instances to ensure data security.
• If you modify the configurations and data of instances including but not limited to ECS and RDS instances, we recommend that you create snapshots or enable RDS log backup.
• If you have authorized or submitted security information such as the logon account and password in the Alibaba Cloud Management console, we recommend that you modify such information in a timely manner.

Select an appropriate solution based on the on-site environment. In this document, as well as the system configuration and its description in CentOS 7.6 64 bit operating system test. The operating system configurations of other types and versions may be different. For more information, see the official documentation of the operating system.

Method 1: temporary solution

Taking udev-fall virus as an example, you can temporarily restore the normal operation of the SSH service by using the following method.

1. Log on to the management terminal, and run the following command to view udev-fall virus-infected process and record the process ID.

   ps aux | grep udev-fall

2. Run the following command to end the udev-fall virus process based on the udev-fall virus process ID obtained in the previous step.

   kill -9 [$PID]

   Note:[$PID] indicates the udev-fall virus process ID obtained in the previous step.

3. Run the following command to obtain the automatic running settings of the udev-fall virus elimination program:

   chkconfig udev-fall off

4. Run the following command to delete all commands and startup configurations udev-fall the virus program:

   for i in ` find / -name "udev-fall"`;
   do echo '' > $i && rm -rf $i;
   done

5. Run the following command to restart the SSH service:
   

   Note: you can run the service sshd restart command to restart the SSH service before CentOS 7.

   systemctl restart sshd.service

Method 2: reliable troubleshooting

It is not clear whether viruses or malicious intruders have tampered with the system or hidden other virus files. To ensure the server runs stably for a long period of time, we recommend that you roll back the historical snapshots of the system disk to normal. For more information about how to roll back a disk, see the following documents.

• Restore a disk

Note:

• The rollback snapshot causes data loss after the rollback point. Make sure that the data in the current environment has no risk before proceeding.
• If the SSH service fails to run after the rollback, the system has an exception at the specified time point. We recommend that you roll back the snapshots one by one from near to far until the SSH service runs properly.
• For more information about SSH service connection, see the following documents.
  • Guidelines for troubleshooting failed remote logon to Linux instances and overview of SSH principles

Application scope

• ECS