All Products
Search
Document Center

MaxCompute:Optimize the costs of data uploads and downloads

Last Updated:Oct 30, 2024

This topic describes how to optimize the synchronization costs incurred by data uploads and downloads.

  • Use the cloud product interconnection network or a Virtual Private Cloud (VPC)

    You can use an internal network, such as the cloud product interconnection network or VPC, to upload or download data at no cost. For more information about how to configure networks, see Endpoints.

  • Use Elastic Compute Service (ECS) to download resources

    If you create a subscription ECS instance, you can use a data synchronization tool such as Tunnel to synchronize data from MaxCompute to the ECS instance. Then, download the data to your local directory. For more information, see Examples of exporting SQL execution results.

  • Optimize Tunnel-based file uploads

    Separate uploads of small files consume too many computing resources. We recommend that you upload a large number of small files at a time. For example, if you call Tunnel SDK, we recommend that you upload files when the cache of the files reaches 64 MB.

  • Estimate the VPC bandwidth

    If you want to synchronize data from your on-premises data center to MaxCompute by using a physical connection, you must estimate the bandwidth and costs of data synchronization. For example, if you migrate 50 TB of data to MaxCompute, the estimated bandwidth for one day is 5 Gbit/s. The estimated bandwidth is calculated by using the following formula:

    50 × 1024 × 8/(24 × 3600) = 4.7 Gbit/s