When working with large objects or processing data incrementally, you can use streaming download to read object content from OSS in chunks to avoid loading the entire object into memory at once, thereby improving the efficiency and performance. This method is suitable for downloading objects that exceed memory limits, processing data in real-time to reduce memory usage, and scenarios where data is retrieved step by step over the network.
Usage notes
In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions, endpoints and open ports.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Create an OSSClient instance.
To use streaming download, you must have the
oss:GetObjectpermission. For more information, see Attach a custom policy to a RAM user.
Sample code
The following code provides an example on how to read object content from OSS in chunks and store it in a byte array.
import com.aliyun.oss.ClientBuilderConfiguration;
import com.aliyun.oss.OSS;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.common.comm.SignVersion;
import com.aliyun.oss.model.*;
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
public class Stream {
public static void main(String[] args) throws Exception {
// In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. For more information about the endpoints of other regions, see Regions and endpoints.
String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
// Specify the name of the bucket. Example: examplebucket.
String bucketName = "examplebucket";
// Specify the full path of the object. Example: exampledir/exampleobject.txt. Do not include the bucket name in the full path.
String objectName = "exampledir/exampleobject.txt";
// Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou.
String region = "cn-hangzhou";
// Create an OSSClient instance.
ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);
OSS ossClient = OSSClientBuilder.create()
.endpoint(endpoint)
.credentialsProvider(credentialsProvider)
.clientConfiguration(clientBuilderConfiguration)
.region(region)
.build();
try {
// ossObject includes the bucket name, object name, object metadata, and an input stream.
OSSObject ossObject = ossClient.getObject(bucketName, objectName);
InputStream inputStream = ossObject.getObjectContent();
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
// Read the object content and store it in a byte array.
byte[] readBuffer = new byte[1024];
int bytesRead;
while ((bytesRead = inputStream.read(readBuffer)) != -1) {
byteArrayOutputStream.write(readBuffer, 0, bytesRead);
}
// Obtain the final result.
byte[] fileBytes = byteArrayOutputStream.toByteArray();
// Display the length of the byte array.
System.out.println("Downloaded file size: " + fileBytes.length + " bytes");
// You must close the obtained stream after the object is read. Otherwise, connection leaks may occur. Consequently, no connections are available and an exception occurs.
inputStream.close();
byteArrayOutputStream.close();
// If you do not close the ossObject object after it is used, connection leaks may occur. Consequently, no connections are available and an exception occurs.
ossObject.close();
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} catch (Throwable ce) {
System.out.println("Caught an ClientException, which means the client encountered "
+ "a serious internal problem while trying to communicate with OSS, "
+ "such as not being able to access the network.");
System.out.println("Error Message:" + ce.getMessage());
} finally {
if (ossClient != null) {
ossClient.shutdown();
}
}
}
}