All Products
Search

This Product

    Object Storage Service:probe

    Document Center

    Object Storage Service:probe

    Last Updated:Aug 19, 2024

    This topic describes how to use the probe command to detect exceptions of network connectivity between the local client and Object Storage Service (OSS), check upload and download bandwidths, and obtain the status of local symbolic links.

    Usage notes

    • To check upload bandwidth, you must have the oss:GetObject, oss:PutObject, and oss:DeleteObject permissions. To check download bandwidth, you must have the oss:GetObject permission. For more information, see Attach a custom policy to a RAM user.

    • For ossutil 1.6.16 and later, you can directly use ossutil as the binary name in the command line. You do not need to update the binary name based on the operating system. For ossutil earlier than 1.6.16, you need to update the binary name based on the operating system. For more information, see ossutil command reference.

    Usage notes

    You can run the probe command to detect exceptions of network connectivity between the local client and OSS and check the upload and download bandwidths. You can also run this command to check the status of a large number of symbolic links before you upload them.

    • Detect network exceptions

      ossutil checks the status of the network connection between the local client and OSS by uploading or downloading objects. To upload or download objects to the specified location when you detect network exceptions, we recommend that you specify the names of the actual objects. If you need only to detect network exceptions, run the command without specifying object names. ossutil generates temporary objects for connectivity detection and deletes the objects after connectivity detection is complete.

    • Check upload and download bandwidths

      When you run this command to check upload and download bandwidths, OSS provides a recommended number of concurrent tasks in uploads and downloads based on the CPU of your local devices and your network bandwidth. You can use the recommended number of concurrent tasks to maximize the usage of your network bandwidth.

    • Check the status of local symbolic links

      When you upload a large number of symbolic links, the upload is interrupted when one of the links is abnormal. We recommend that you check the status of local symbolic links before you upload them. If one of the symbolic links is abnormal, fix the issue before you upload the links.

    After you run the probe command, you can view the procedure and result of each upload or download task.

    • Detect network exceptions by uploading or downloading objects

      • A check mark (√) indicates that the check result is positive. A cross sign (×) indicates that the check result is negative.

      • If the detection succeeds, ossutil returns the information about the upload or download task, including the size of the uploaded or downloaded object, the time used to perform the upload or download task, and the location to which the object is uploaded or downloaded.

      • If the detection fails, ossutil returns the error causes or error codes, which can be used to troubleshoot the errors.

        For more information about error codes, see Overview.

      • After the detection is complete, ossutil creates a log file whose name is in the logOssProbe+DetectionCompletionTime.log format in the installation directory. The log file contains detailed information about the probe command.

    • Check a specific item

      When you run the probe command to check the upload and download bandwidths or the status of local symbolic links, ossutil provides the check results and suggestions.

    Check the network connection status by uploading an object and generate a report

    ossutil checks the network connection status between the local client and a bucket by uploading an object to the bucket.

    • Command syntax

      ossutil probe {--upload [file_name]} {--bucketname bucket_name} [--object object_name] [--addr domain_name] [--upmode]

      The following table describes the parameters in the preceding command.

      Parameter

      Required

      Description

      --upload

      Yes

      Specifies that the connectivity detection is performed by uploading an object.

      file_name

      No

      The full path of the local file that you want to upload to the bucket. If you leave this parameter unspecified, ossutil generates a temporary object and detects connectivity by uploading the object.

      --bucketname

      No

      The name of the bucket whose connection you want to check.

      --object

      No

      The name of the object that you specify for the uploaded file. If you specify this parameter in the command, ossutil stores the uploaded file in the specified bucket by using the specified name. If you do not specify this parameter in the command, ossutil deletes the uploaded file after the connectivity detection is complete.

      --addr

      No

      Specifies the network address. ossutil uses a ping operation to check the network connectivity between your local computer and the specified address.

      Default value: www.aliyun.com.

      --upmode

      No

      The method used to upload the object.

      Valid values:

      • normal (default): The object is uploaded by using simple upload.

      • append: The object is uploaded by using append upload.

      • multipart: The object is uploaded by using multipart upload.

    • Examples

      • Check the network connectivity by uploading a random object and specifying the network address

        In this example, the bucket to which the object is uploaded is named examplebucket and the network address to check is aliyun.com. Sample command:

        ossutil probe --upload --bucketname examplebucket --addr aliyun.com

        Sample output:

        begin parse parameters and prepare file...[✓]
begin network detection...[✓]
begin upload file(normal)...[✓]

*************************  upload result  *************************
upload file:success
upload file size:122880(byte)
upload time consuming:245(ms)
(only the time consumed by probe command)


************************* report log info*************************
report log file:/root/logOssProbe20201201173031.log

      • Check network connectivity by uploading a specific object in default simple upload mode and delete the uploaded object after the detection is complete

        In this example, a file named example.txt in the local root directory is uploaded to the bucket named examplebucket. Sample command:

        ossutil probe --upload example.txt --bucketname examplebucket

        Sample output:

        begin parse parameters and prepare file...[✓]
begin network detection...[✓]
begin upload file(normal)...[✓]

*************************  upload result  *************************
upload file:success
upload file size:7(byte)
upload time consuming:224(ms)
(only the time consumed by probe command)
upload object is example.txt

************************* report log info*************************
report log file:/root/logOssProbe20201201173841.log

      • Check network connectivity by uploading a specified object in append upload mode and store the uploaded object in the bucket after detection is complete

        In this example, a file named example.txt in the local root directory is uploaded to the bucket named examplebucket. Sample command:

        ossutil probe --upload example.txt --bucketname examplebucket --object example.txt --upmode append

        Sample output:

        begin parse parameters and prepare file...[✓]
begin network detection...[✓]
begin upload file(append)...[✓]

*************************  upload result  *************************
upload file:success
upload file size:7(byte)
upload time consuming:171(ms)
(only the time consumed by probe command)

upload object is example.txt

************************* report log info*************************
report log file:/root/logOssProbe20201201174126.log

    Check network connectivity by downloading an object using its URL and generate a report

    ossutil checks network connectivity between the local client and a bucket by downloading an object using its URL from the bucket.

    • Command syntax

      ossutil probe {--download} {--url http_url} [--addr=domain_name] [file_name]

      The following table describes the parameters in the preceding command.

      Parameter

      Required

      Description

      --download

      Yes

      Specifies that the connectivity check is performed by downloading an object.

      --url

      Yes

      The URL of the object that you want to download. ossutil uses this URL to download the object to a local directory.

      • If the ACL of the object that you want to download is public-read, set this parameter to the URL of the object. Example: https://examplebucket.oss-cn-beijing.aliyuncs.com/example.jpg.

      • If the ACL of the object that you want to download is private, set this parameter to the signed URL of the object. The URL must be enclosed in double quotation marks (""). Example: "https://examplebucket.oss-cn-beijing.aliyuncs.com/example.jpg?Expires=1552015472&OSSAccessKeyId=TMP.******5r9f1FV12y8_Qis6LUVmvoSCUSs7aboCCHtydQ0axN32Sn-UvyY3AAAwLAIUarYNLcO87AKMEcE5O3A******oCFAQuRdZYyVFyqOW8QkGAN-bamUiQ&Signature=bIa4llbMbldrl7rwckr%2FXXvTtxw%3D".

      --addr

      No

      Specifies the network address. ossutil uses a ping operation to check the network connectivity between your local computer and the specified address.

      Default value: www.aliyun.com.

      file_name

      No

      The local path to which the object is downloaded.

      • If you specify only the object name but do not specify a directory, the object is stored by using the specified name in the installation directory of ossutil.

      • If you specify only a directory but do not specify the object name, the object is stored by using its original name in the specified directory.

      • If you leave this parameter unspecified, the object is stored by using its original name in the installation directory of ossutil.

    • Examples

      • Check network connectivity by downloading a specific object by using the object URL and rename the object

        In this example, the example.txt object whose ACL is public-read is downloaded from https://examplebucket.oss-cn-beijing.aliyuncs.com/example.txt to a local directory and renamed as /localfile/test.txt. Sample command:

        ossutil probe --download --url https://examplebucket.oss-cn-beijing.aliyuncs.com/example.txt  /localfile/test.txt

        Sample output:

        begin parse parameters and prepare object...[✓]
begin network detection...[✓]
begin download file...[✓]

*************************  download result  *************************
download file:success
download file size:57374182(byte)
download time consuming:1246(ms)
(only the time consumed by probe command)

download file is /localfile/test.txt

************************* report log info*************************
report log file:/root/logOssProbe20201202171639.log

      • Check network connectivity by downloading an object by using its URL and specifying the network address for a ping test

        In this example, the example.txt object whose ACL is public-read is downloaded from https://examplebucket.oss-cn-beijing.aliyuncs.com/example.txt to a local directory and the network address for a ping test is aliyun.com. Sample command:

        ossutil probe --download --url https://examplebucket.oss-cn-beijing.aliyuncs.com/example.txt --addr aliyun.com

        Sample output:

        begin parse parameters and prepare object...[✓]
begin network detection...[✓]
begin download file...[✓]

*************************  download result  *************************
download file:success
download file size:57374182(byte)
download time consuming:1344(ms)
(only the time consumed by probe command)

download file is /root/example.txt

************************* report log info*************************
report log file:/root/logOssProbe20201202172232.log

    Check network connectivity by downloading a specified object and generate a report

    ossutil checks network connectivity between the local client and a bucket by downloading an object from the bucket.

    • Command syntax

      ossutil probe {--download} {--bucketname bucket_name} [--object object_name] [--addr domain_name] [file_name]

      The following table describes the parameters in the preceding command.

      Parameter

      Required

      Description

      --download

      Yes

      Specifies that the connectivity check is performed by downloading an object.

      --bucketname

      Yes

      The name of the bucket whose connection you want to check.

      --object

      No

      The name of the object that you want to download. If you specify this parameter in the command, ossutil downloads the specified object to a local directory. If you do not specify this parameter in the command, ossutil generates a temporary object, uploads the object to the specified bucket, and then downloads the object. After the connectivity check is complete, ossutil deletes the temporary object from the bucket.

      --addr

      No

      Specifies the network address. ossutil uses a ping operation to check the network connectivity between your local computer and the specified address.

      Default value: www.aliyun.com.

      file_name

      No

      The local path to which the object is downloaded.

      • If you specify only the object name but do not specify a directory, the object is stored by using the specified name in the installation directory of ossutil.

      • If you specify only a directory but do not specify the object name, the object is stored by using its original name in the specified directory.

      • If you leave this parameter unspecified, the object is stored by using its original name in the installation directory of ossutil.

    • Examples

      • Check network connectivity by downloading an object and rename the object

        In this example, the /ossfolder/example.txt object is downloaded from the examplebucket bucket and renamed as /localfolder/text.txt. Sample command: 

        ossutil probe --download --bucketname examplebucket --object /ossfolder/example.txt /localfolder/text.txt

        Sample output:

        begin parse parameters and prepare object...[✓]
begin network detection...[✓]
begin download file...[✓]

*************************  download result  *************************
download file:success
download file size:57374182(byte)
download time consuming:1108(ms)
(only the time consumed by probe command)

download file is /localfolder/text.txt

************************* report log info*************************
report log file:/root/logOssProbe20201202173032.log

      • Check the network connectivity status by downloading a temporary object and specifying the address of the network for a ping test

        In this example, a temporary object is downloaded from the examplebucket bucket. The network address to check is aliyun.com. Sample command:

        ossutil probe --download --bucketname examplebucket --addr aliyun.com

        Sample output:

        begin parse parameters and prepare object...[✓]
begin network detection...[✓]
begin download file...[✓]

*************************  download result  *************************
download file:success
download file size:122880(byte)
download time consuming:516(ms)
(only the time consumed by probe command)

download file is /root/oss-test-probe-1606902911701892000-3ifwj9t0ln

************************* report log info*************************
report log file:/root/logOssProbe20201202173512.log

    Check a specific item

    You can run the probe command to check a specific item, including the status of local symbolic links and the upload and download bandwidths. ossutil also returns the names of abnormal symbolic links and a recommended number of concurrent tasks in uploads and downloads based on the check results.

    • Command syntax

      ossutil probe {--probe-item item_value} {--bucketname bucket-name} [--object object_name][--parallel <value>][--part-size <value>][--runtime <value>]

      The following table describes the parameters in the preceding command.

      Parameter

      Required

      Description

      --probe-item

      Yes

      The item that you want to check.

      Valid values:

      • cycle-symlink: ossutil checks whether abnormal symbolic links exist in the local path.

      • upload-speed: ossutil checks the upload bandwidth.

      • download-speed: ossutil checks the download bandwidth.

      • download-time: ossutil checks the time required to download an object.

      --bucketname

      Yes when the value of --probe-item is not cycle-symlink

      The name of the bucket whose connection you want to check.

      --object

      Yes when the value of --probe-item is download-speed

      The path of the object. The object must exist. We recommend that you specify an object that is larger than 5 MB in size. Example: ossfolder/example.txt.

      --parallel

      No

      The number of concurrent tasks for an operation on a single object. Default value: 1.

      Note

      This parameter takes effect only when the value of --probe-item is download-time.

      --part-size

      No

      The part size. Unit: bytes. By default, ossutil determines the part size based on the object size.

      Valid values: 1 to 9223372036854775807.

      Note

      This parameter takes effect only when the value of --probe-item is download-time.

      --runtime

      No

      The maximum execution time. If the maximum execution time is reached, the command execution is interrupted.

      Note

      This parameter takes effect only when the value of --probe-item is upload-speed or download-speed.

    • Examples

      • Check whether abnormal symbolic links exist in the localfolder directory of the local root directory

        Sample command:

        ossutil probe --probe-item cycle-symlink /root/localfolder

        Sample output:

        Error: stat /root/localfolder/example.jpg: no such file or directory

        The output shows that the example.jpg symbolic link is abnormal.

      • Check the upload bandwidth

        In this example, ossutil uploads a temporary object to a bucket named examplebucket and returns a recommended number of concurrent upload tasks based on the hardware specifications of the current device and the upload bandwidth.

        Sample command:

        ossutil probe --probe-item upload-speed --bucketname examplebucket

        Sample output:

        cpu core count:2
parallel:2,average speed:679.72(KB/s),current speed:1344.00(KB/s),max speed:1440.00(KB/s))
parallel:3,average speed:643.31(KB/s),current speed:704.00(KB/s),max speed:1632.00(KB/s))
parallel:4,average speed:646.62(KB/s),current speed:512.00(KB/s),max speed:1600.00(KB/s))

suggest parallel is 2, max average speed is 679.72(KB/s)

        The output shows that the CPU of the device has two cores, the maximum average upload bandwidth is 679.72 KB/s, and the recommended number of concurrent upload tasks is 2.

      • Check the download bandwidth

        In this example, ossutil downloads the example.txt object from the examplebucket bucket and recommends the number of concurrent downloads based on the hardware performance and download bandwidth.

        Sample command:

        ossutil probe --probe-item download-speed --bucketname examplebucket --object example.txt

        Sample output:

        cpu core count:2
parallel:2,average speed:12524.93(KB/s),current speed:12288.63(KB/s),max speed:14302.25(KB/s)
parallel:3,average speed:12564.45(KB/s),current speed:12144.39(KB/s),max speed:14484.24(KB/s)
parallel:4,average speed:12545.21(KB/s),current speed:12766.58(KB/s),max speed:13534.42(KB/s)

suggest parallel is 3, max average speed is 12564.45(KB/s)

        The output shows that the CPU of the device has two cores, the maximum average download bandwidth is 12,564.45 KB/s, and the recommended number of concurrent download tasks is 3.

      • Check the download bandwidth and specify the maximum execution time

        In this example, ossutil downloads the example.zip object from the examplebucket bucket and returns a recommended number of concurrent download tasks based on the hardware performance and download bandwidth, with a maximum execution time specified. 

        ossutil probe --probe-item download-speed --bucketname examplebucket --object  example.zip --runtime 2

        Sample output:

        cpu core count:8
parallel:8,average speed:10610.66(KB/s),current speed:11245.14(KB/s),max speed:11470.41(KB/s)
parallel:9,average speed:9915.97(KB/s),current speed:10763.22(KB/s),max speed:11401.97(KB/s))
parallel:10,average speed:9969.66(KB/s),current speed:11213.43(KB/s),max speed:11384.52(KB/s))
parallel:11,average speed:10298.34(KB/s),current speed:10522.27(KB/s),max speed:11282.21(KB/s)
parallel:12,average speed:10454.31(KB/s),current speed:9725.25(KB/s),max speed:11426.21(KB/s))
parallel:13,average speed:9531.86(KB/s),current speed:8634.53(KB/s),max speed:11220.90(KB/s)))
parallel:14,average speed:9329.45(KB/s),current speed:8542.94(KB/s),max speed:10505.16(KB/s))
parallel:15,average speed:10193.00(KB/s),current speed:10770.35(KB/s),max speed:10971.39(KB/s)
parallel:16,average speed:10637.59(KB/s),current speed:10483.77(KB/s),max speed:11179.55(KB/s)

suggest parallel is 16, max average speed is 10637.59(KB/s)

run download-speed  2 seconds with parallel 16
parallel:16,average speed:9197.00(KB/s),current speed:9849.30(KB/s),max speed:9849.30(KB/s)
280.345640(s) elapsed

      • Check the amount of time required to complete a download task

        In this example, ossutil downloads the example.txt object from the examplebucket bucket and returns the amount of time required to complete a download task. Sample command: 

        ossutil probe --probe-item download-time --bucketname examplebucket --object example.txt

        Sample output:

        downloading average speed:6178.75(KB/s),current speed:14446.02(KB/s),max speed:14446.02(KB/s)
download-speed part-size:-1, parallel:1 total bytes:31707929, cost:4.918 s, avg speed:7741.00(kB/s)

5.206202(s) elapsed

      • Check the amount of time required to complete a download task and specify the part size

        In this example, ossutil downloads the example.zip object from the examplebucket bucket based on the specified part size and returns the amount of time required to complete a download task. 

        ossutil probe --probe-item download-time --bucketname examplebucket --object example.zip --part-size 10000000

        Sample output:

        downloading average speed:9229.00(KB/s),current speed:18458.57(KB/s),max speed:18458.57(KB/s)
download-speed part-size:10000000, parallel:1 total bytes:31707929, cost:3.230 s, avg speed:10321.33(kB/s)

3.569254(s) elapsed

      • Check the amount of time required to complete a download task and configure the concurrency level

        In this example, ossutil downloads the example.zip object from the examplebucket bucket based on the specified part size and returns the amount of time required to complete a download task. 

        ossutil probe --probe-item download-time --bucketname examplebucket --object example.zip --parallel 3

        Sample output:

        downloading average speed:9155.00(KB/s),current speed:18310.68(KB/s),max speed:18310.68(KB/s)
download-speed part-size:-1, parallel:3 total bytes:31707929, cost:3.121 s, avg speed:10321.33(kB/s)

3.424191(s) elapsed