E-MapReduce:Jindo CLI user guide

Command

Description

Example

Scenario

archive

Sets the storage class of Object Storage Service (OSS) objects to Infrequent Access (IA), Archive, or Cold Archive. Optional parameters:

• -i: the IA storage class.

• -c: the Cold Archive storage class.

• jindo fs -archive oss://<bucket>/<dir>

• jindo fs -archive -i oss://<bucket>/<dir>

• jindo fs -archive -c oss://<bucket>/<dir>

OSS

cat

Displays text content.

jindo fs -cat oss://<bucket>/<dir>

OSS and OSS-Hadoop Distributed File System (HDFS)

count

Displays the file size and the number of files. Optional parameter: -h: displays the unit of the file size.

jindo fs -count -h oss://<bucket>/<dir>

OSS and OSS-HDFS

cp

Copies files. Optional parameter: -f: specifies whether to perform forceful overwrite.

jindo fs -cp -f oss://<bucket>/<sourcedir> oss://<bucket>/<targetdir>

OSS and OSS-HDFS

checksum

Calculates the CRC32 checksum of files.

jindo fs -checksum oss://<bucket>/file

OSS-HDFS

chgrp

Changes the group to which a file belongs.

jindo fs -chgrp <groupname> oss://<bucket>/<dir>

OSS-HDFS

chmod

Changes the access permissions on files.

jindo fs -chmod <mode> oss://<bucket>/<dir>

OSS-HDFS

chown

Changes the owner of a file.

jindo fs -chown <username> oss://<bucket>/<dir>

OSS-HDFS

copyFromLocal

Copies files from your on-premises machine to an OSS bucket. Optional parameter: -f: specifies whether to perform forceful overwrite.

jindo fs -copyFromLocal -f <localdir> oss://<bucket>/<targetdir>

OSS and OSS-HDFS

copyToLocal

Copies files from an OSS bucket to your on-premises machine. Optional parameter: -f: specifies whether to perform forceful overwrite.

jindo fs -copyToLocal -f oss://<bucket>/<targetdir> <localdir>

OSS and OSS-HDFS

createSnapshot

Creates snapshots. You must be authorized to use snapshots before you can run this command.

jindo fs -createSnapShot <snapshotDir> [<snapshotName>]

OSS and OSS-HDFS

checkStoragePolicy

Queries the file storage policy.

jindo fs -checkStoragePolicy -path oss://<bucket>/<dir>

OSS-HDFS

deleteSnapshot

Deletes snapshots.

jindo fs -deleteSnapshot oss://<bucket>/<dir> <snapshotName>

OSS-HDFS

du

Queries the size of all files in a directory. Optional parameters:

• -s: specifies whether to calculate the total size of destination folders.

• -h: specifies whether to use the standard unit.

jindo fs -du oss://<bucket>/<dir>

OSS and OSS-HDFS

get

Downloads files to your on-premises machine. Optional parameter: -f: specifies whether to perform forceful overwrite.

jindo fs -get -f oss://<bucket>/<dir> <localdst>

OSS and OSS-HDFS

getfacl

Queries the access control list (ACL) of a file or directory. Optional parameter: -R: specifies whether to use recursive display.

jindo fs -getfacl -R oss://<bucket>/<dir>

OSS-HDFS

getfattr

Queries the extended property names and values of files or directories. Optional parameters:

• -R: specifies whether to use recursive display.

• -n: specifies whether to display results by name.

• -d: specifies whether to display all results.

jindo fs -getfattr [-R] {-n user.myAttr | -d} oss://<bucket>/<dir>

OSS-HDFS

getStoragePolicy

Queries the storage policy.

jindo fs -getStoragePolicy -path oss://<bucket>/<dir>

OSS-HDFS

ls

Lists all files in a directory. Optional parameter: -R: specifies whether to use recursive display.

jindo fs -ls [-R] oss://<bucket>/<dir>

OSS and OSS-HDFS

listPolicies

Lists supported storage policies.

jindo fs -listPolicies

OSS-HDFS

load

Caches data stored in a specific path to the local disks of a cluster. Then, you can read the data from local disks without the need to read data from backend storage systems such as OSS. Optional parameters:

• -s: specifies whether the data caching process is executed in synchronous mode. If this parameter is configured, the progress and execution information about the data caching process are displayed. We recommend that you configure this parameter.

• -replica: the number of replicas to be cached. By default, only one replica is cached.

• -R: specifies whether to recursively list all files in a folder. If the value of the path parameter is a folder, you need to configure this parameter.

jindo fs -load [-meta] [-data] [-s] [-m] [-R] [-replica value] oss://<bucket>/<dir>

None

mkdir

Creates a folder.

jindo fs -mkdir oss://<bucket>/<dir>

OSS and OSS-HDFS

mv

Moves a file.

jindo fs -mv oss://<bucket>/<srcdir> oss://<bucket>/<dstdir>

OSS and OSS-HDFS

moveFromLocal

Moves a file from your on-premises machine to a remote path. After the file is successfully moved, the local file is deleted.

jindo fs -moveFromLocal <localsrc> oss://<bucket>/<dstdir>

OSS and OSS-HDFS

moveToLocal

Moves a file from a remote path to your on-premises machine. After the file is successfully moved, the file in the OSS bucket is deleted.

jindo fs -moveToLocal oss://<bucket>/<srcdir> <localdst>

OSS and OSS-HDFS

metaDiff

Queries the synchronization status between the local metadata and remote metadata after the cache-based acceleration system is used.

jindo fs -metaDiff oss://<bucket>/<srcdir>

OSS and OSS-HDFS

put

Copies a file from your on-premises machine to a remote directory. Optional parameter: -f: specifies whether to perform forceful overwrite.

jindo fs -put -f <localsrc> oss://<bucket>/<targetdir>

OSS and OSS-HDFS

rm

Deletes a file from a remote path. Optional parameters:

• -f: specifies whether to perform forceful deletion.

• -r/-rmr/-R: specifies whether to perform recursive deletion.

• jindo fs -rm -f oss://<bucket>/<dir>

• jindo fs -rm -R oss://<bucket>/<dir>

OSS and OSS-HDFS

rmdir

Deletes a folder.

jindo fs -rmdir oss://<bucket>/<dir>

OSS and OSS-HDFS

test

Tests basic features. Optional parameters:

• -d: specifies whether a folder is used.

• -e: specifies whether a path exists.

• -f: specifies whether a file is used.

• -s: specifies whether a folder is empty.

• -z: specifies whether the file length is 0.

jindo fs -test -d oss://<bucket>/<dir>/file

OSS and OSS-HDFS

renameSnapshot

Renames a snapshot.

jindo fs -renameSnapshot oss://<bucket>/<dir> <oldName> <newName>

OSS-HDFS

restore

Restores files from the Archive or Cold Archive state. You must restore files in the Cold Archive or Archive state before you unarchive the files. Optional parameter: -days: the number of days during which files are restored.

jindo fs -restore -days value oss://<bucket>/<dir>

OSS

stat

Queries the file status.

jindo fs -stat oss://<bucket>/<dir>

OSS and OSS-HDFS

setStoragePolicy

Specifies a storage policy for data in a path.

• -path: specifies the path where a file or directory is located.

• -policy: specifies a storage policy. Valid values:

  • CLOUD_STD: the Standard storage class.

  • CLOUD_IA: the IA storage class.

  • CLOUD_AR: the Archive storage class.

  • CLOUD_COLD_AR: the Cold Archive storage class.

jindo fs -setStoragePolicy -path oss://<bucket>/<dir> -policy <policy>

OSS-HDFS

setfacl

Configures file access policies. Optional parameters:

• -b: specifies whether to delete all access policies.

• -k: specifies whether to delete the default access policy.

• -m: specifies whether to modify access policies by name.

• -x: specifies whether to delete access policies by name.

• -set: specifies whether to configure access policies.

• jindo fs -setfacl -b oss://<bucket>/<dir>

• jindo fs -setfacl -k oss://<bucket>/<dir>

• jindo fs -setfacl -m user:<username>:<acl> oss://<bucket>/<dir> jindo fs -setfacl -m user:<username>:<acl> oss://<bucke

• jindo fs -setfacl -x user:<username> oss://<bucket>/<dir>

• jindo fs -setfacl -set user::<acl> oss://<bucket>/<dir>

OSS-HDFS

setfattr

Configures the extended property names and values of files or directories. Optional parameters:

• -n: specifies whether to configure properties by name.

• -v: specifies whether to configure property values.

• -x: specifies whether to delete properties by name.

jindo fs -setfattr {-n name [-v value] | -x name} oss://<bucket>/<dir>

OSS-HDFS

sync

Uploads files from your on-premises machine to a remote path. Optional parameters:

• -update: specifies whether to use resumable upload.

• -thread: specifies whether to use multiple threads.

jindo fs -sync [-update] [-thread thread_num] <localsrc> oss://<bucket>/<dir>

OSS and OSS-HDFS

touchz

Generates a file whose size is 0 bytes.

jindo fs -touchz oss://<bucket>/<dir>

OSS and OSS-HDFS

truncate

Truncates a file to the specified size.

jindo fs -truncate <length> oss://<bucket>/<dir>

OSS-HDFS

unarchive

Restores files to standard files. This operation is dedicated for archived files and code archived files.

jindo fs -unarchive oss://<bucket>/<dir>

OSS

uncache

Deletes cached files from the cache system.

jindo fs -uncache oss://<bucket>/<dir>

OSS and OSS-HDFS

unsetStoragePolicy

Disables a storage policy.

jindo fs -unsetStoragePolicy -path oss://<bucket>/<dir>

OSS-HDFS

Command

Description

Example

Scenario

mount

Mounts an OSS or OSS-HDFS path.

jindo admin -mount <localdir> oss://<Bucket>/<dir>

OSS and OSS-HDFS

report

Queries information about the JindoFSx storage acceleration system, such as the cache size and cache capacity.

jindo admin -report

OSS and OSS-HDFS

reportMetrics

Queries metrics of the JindoFSx storage acceleration system.

jindo admin -reportMetrics

OSS and OSS-HDFS

umount

Unmounts an OSS or OSS-HDFS path.

jindo fsxadmin -unmount <localdir>

OSS and OSS-HDFS

addProxyUser

Adds proxy users.

jindo dlsadmin -addProxyUser -dlsUri oss://<bucket> -proxyUser <value> -users|-groups <value> -hosts <value>]

OSS-HDFS

allowSnapshot

Allows the use of snapshots.

jindo admin -allowSnapshot -dlsUri oss://<bucket>/<dir>

OSS-HDFS

addUserGroupsMapping

Adds mappings between users and user groups.

jindo admin -addUserGroupsMapping -dlsUri oss://<bucket> <path> -user <value> -groups <value>

OSS-HDFS

disallowSnapshot

Disallows the use of snapshots.

jindo admin -disallowSnapshot -dlsUri oss://<bucket>/<dir>

OSS-HDFS

deleteProxyUser

Deletes proxy users.

jindo admin -deleteProxyUser -dlsUri oss://<bucket>/<dir> -proxyUser <value> -users|-groups <value> -hosts <value>]

OSS-HDFS

deleteUserGroupsMapping

Deletes mappings between users and user groups.

jindo admin -deleteUserGroupsMapping -dlsUri oss://<bucket> -user <value>

OSS-HDFS

dumpFile

Queries the detailed storage information about files.

jindo dlsadmin -dumpFile -dlsUri oss://<bucket>/<dir>

OSS-HDFS

dumpInventory

Exports file metadata.

jindo dlsadmin -dumpInventory -dlsUri oss://<bucket>/<dir>

OSS-HDFS

dumpBlockFiles

Downloads all files in the specified folder to a local directory.

jindo dlsadmin -dumpBlockFiles -in oss://<bucket>/<dir> -out <localdir>

OSS-HDFS

listProxyUsers

Lists proxy users.

jindo admin -listProxyUsers -dlsUri oss://<bucket> [-maxKeys <value>] [-marker <value>]

OSS-HDFS

listUserGroupsMappings

Lists mappings between all users and all user groups.

jindo admin -listUserGroupsMappings -dlsUri oss://<bucket> [-maxKeys <value>] [-marker <value>]

OSS-HDFS

snapshotDiff

Queries differences between snapshots.

jindo admin -snapshotDiff -dlsUri oss://<bucket>/<dir> -fromSnapshot <value> -toSnapshot <value>

OSS-HDFS

setRootPolicy

Configures an access path with a custom prefix for a bucket.

jindo admin -setRootPolicy <dlsRootPath> <accessRootPath>

OSS-HDFS

unsetRootPolicy

Deletes an access path with a custom prefix for a bucket.

jindo admin -unsetRootPolicy <dlsRootPath> <accessRootPath>

OSS-HDFS

listAccessPolicies

Queries all access paths that are supported by a bucket.

jindo admin -listAccessPolicies <dlsRootPath>

OSS-HDFS