When you download a large object that is larger than 5 GB in size from Object Storage Service (OSS) to a local computer, you may fail to download the object due to network interruptions or program crashes. If the object fails to be downloaded after multiple retries, you can download the large object by performing resumable download. You can split the object into multiple parts and download the parts in parallel to speed up the download. During resumable download, the download progress is recorded in a checkpoint file. If a part fails to be downloaded, the next download starts from the position that is recorded in the checkpoint file. After all parts are downloaded, all parts are combined into a complete object.
Usage notes
In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions, endpoints and open ports.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Initialization.
To use resumable download, you must have the
oss:GetObjectpermission. For more information, see Attach a custom policy to a RAM user.During resumable download, the download progress is recorded in a checkpoint file. If a part fails to be downloaded, the next download starts from the position that is recorded in the checkpoint file. The checkpoint file is deleted after resumable download is complete.
OSS SDK for Go records the download progress in the checkpoint file. Make sure that you have write permissions on the checkpoint file.
You cannot modify the checksum that is contained in the checkpoint file. If the checkpoint file is damaged, you need to download all parts again.
If the ETag of the object is not consistent or parts are lost or modified during the download, you need to download the object again.
Examples
package main
import (
"fmt"
"os"
"github.com/aliyun/aliyun-oss-go-sdk/oss"
)
func main() {
/// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
provider, err := oss.NewEnvironmentVariableCredentialsProvider()
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Create an OSSClient instance.
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint.
// Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. Specify the actual region.
clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
clientOptions = append(clientOptions, oss.Region("yourRegion"))
// Specify the version of the signature algorithm.
clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
client, err := oss.New("yourEndpoint", "", "", clientOptions...)
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Specify the name of the bucket. Example: examplebucket.
bucket, err := client.Bucket("examplebucket")
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Specify the full path of the object. Do not include the bucket name in the full path. Example: exampledir/exampleobject.txt.
// Specify the full path of the local file. Example: D:\\localpath\\examplefile.txt. If you do not specify the path of the local file, the downloaded object is saved to the path of the project to which the sample program belongs.
// Set the size of each part that you want to download to 100 KB (100 x 1024) and set the number of concurrent download parts to 3.
// oss.Checkpoint(true, "") Specify that resumable download is enabled. By default, the checkpoint file and the downloaded object are stored in the same directory and share the same name. However, the extension of the checkpoint file is .temp. You can also use oss.Checkpoint(true, "your-cp-file.temp") to specify the name of the checkpoint file.
err = bucket.DownloadFile("file.zip", "D:\\file.zip", 100*1024, oss.Routines(3), oss.Checkpoint(true, ""))
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
}