Use the HDFS shell to connect to and use LindormDFS - Lindorm

The HDFS shell is the command-line tool provided by Hadoop Distributed File System (HDFS). LindormDFS is fully compatible with HDFS. You can use the HDFS shell to manage the files stored in LindormDFS. For example, you can query files, delete files, manage file permissions, and modify file names. This topic describes the common commands of the HDFS shell and provides examples.

Environment

For more information about how to configure the environment, see Download the client and Configure Apache Hadoop.

Common commands

Upload data

The following table lists the commands that are commonly used to upload data from local paths to LindormDFS.

Command	Description
put	This command copies files from one or more source paths in the local file system to LindormDFS. You can also run this command to read data from a standard input (stdin) and write the input data to LindormDFS.
copyFromLocal	This command is similar to the put command. However, the source path specified in this command must be a local path.
moveFromLocal	This command moves files from the local file system to LindormDFS.
appendToFile	This command appends one or more source files from the local file system to LindormDFS.

Process data

The following table lists the commands that are commonly used to process data stored in LindormDFS.

Command	Description
cp	This command copies files from the source path to the destination path. You can run this command to copy files from multiple source paths to the destination path. In this case, the destination path must be a directory.
mv	This command moves files from the source path to the destination path. You can run this command to copy files from multiple source paths to the destination path. In this case, the destination path must be a directory. Files cannot be moved between different file systems.
chown	This command changes the owner of one or more files. The -R option recursively makes the change through the directory structure. The user who runs the command must have superuser permissions.
chgrp	This command changes the group to which a file belongs. The -R option recursively makes the change through the directory structure. The user who runs the command must be the owner of the file or a superuser.
chmod	This command changes the permissions on a file. The -R option recursively makes the change through the directory structure. The user who runs the command must be the owner of the file or a superuser.
mkdir	This command accepts a path URL as an argument and creates a directory. The command is similar to the `mkdir -p` command in UNIX, which creates parent directories along the path.
du	This command displays the size of each file in the directory or displays the size of a specified file.
df	This command displays the free storage capacity.
cat	This command copies the content of a specified file in a path to a standard output (stdout).
rm	This command deletes a specified file. You can delete only non-empty directories and files.
ls	If you specify a file for this command, this command returns the details of the file in the following format: `File name <Number of replicas> File size Modification date Modification time Permission User ID Group ID` If you specify a directory for this command, this command returns a list of the direct children of the directory. This command is the same as that in UNIX. The information in the list is returned in the following format: `Directory name <dir> Modification date Modification time Permission User ID Group ID`.

Download data

The following table lists the commands that are commonly used to download data from LindormDFS to the local storage.

Command	Description
get	This command copies files from LindormDFS to the local file system. You can specify the -ignorecrc option to copy files that failed the cyclic redundancy check (CRC). You can specify the -crc option to copy files with their CRC information.
getmerge	This command accepts a source directory and a destination file as inputs and concatenates files in the source directory into a local destination file.
copyToLocal	This command is similar to the get command. However, the destination path specified in this command must be a local path.

You can run the help command to view all commands supported by the HDFS shell. For more information, see Introduction to the Apache HDFS shell.

Examples

Replace ${Instance ID} with the ID of your Lindorm instance when you run commands in actual business.

Query all files in the specified directory.

${HADOOP_HOME}/bin/hadoop fs -ls hdfs://${Instance ID}/

Upload a local file to LindormDFS.

${HADOOP_HOME}/bin/hadoop fs -put test.txt hdfs://${Instance ID}/

Download a file to the specified local path.
```
${HADOOP_HOME}/bin/hadoop fs -get hdfs://${Instance ID}/${filename}
```
In the command, ${filename} specifies the name of the file that you want to download.
Delete a file.
```
${HADOOP_HOME}/bin/hadoop fs -rm hdfs://${Instance ID}/${filename}
```
In the command, ${filename} specifies the name of the file that you want to delete.
Note
If you use the preceding command to delete a file, the file is not completely deleted but is moved to the Trash directory. To completely delete the file, run the hadoop fs -expunge command to clear the recycle bin.