Locally encrypt an object before you upload the object to OSS by using client-side encryption - Object Storage Service

If client-side encryption is enabled, objects are locally encrypted before they are uploaded to Object Storage Service (OSS). Only the owner of the customer master key (CMK) can decrypt the objects. This way, data security during data transmission and storage is enhanced.

Disclaimer

When you use client-side encryption, you must ensure the integrity and validity of the CMK. If the CMK is incorrectly used or lost due to improper maintenance, you are responsible for all losses and consequences caused by decryption failures.
When you copy or migrate encrypted data, you are responsible for the integrity and validity of object metadata. If the encrypted metadata is incorrect or lost due to improper maintenance, you are responsible for all losses and consequences caused by data decryption failures.

Scenarios

Highly sensitive data: For data that contains extremely sensitive information, such as personally identifiable information (PII), financial transaction records, medical and health data, users may want to encrypt the data before the data leaves the local environment to make sure that the original data is effectively protected even if it is intercepted during transmission.
Compliance requirements: Certain industries and regulations, such as HIPAA and GDPR, require strict encryption management for data stored on third-party platforms. Client-side encryption meets these compliance requirements because CMKs are managed by users and not passed through the network or owned by cloud service providers.
Strong permission control: An enterprise or developer may want to have full control over the encryption process, including selecting encryption algorithms and managing and rotating CMKs. Client-side encryption ensures that only legally authorized users can decrypt and access data.
Security of cross-region data migration: Client-side encryption helps maintain data in the encrypted state before and after cross-region data migration. This amplifies the security of data transmission over the Internet.

Background information

In client-side encryption, a random data key is generated for each object to perform symmetric encryption on the object. The client uses a CMK to encrypt the random data key. The encrypted data key is uploaded as a part of the object metadata and stored in the OSS server. When an encrypted object is downloaded, the client uses the CMK to decrypt the random data key and then uses the data key to decrypt the object. To ensure data security, the CMK is used only on the client and is not transmitted over the network or stored in the server.

Important

Client-side encryption supports multipart upload for objects larger than 5 GB in size. When you use multipart upload to upload an object, you must specify the total object size and the part size. The size of each part except the last part must be the same and must be an integer multiple of 16.
If you upload an object encrypted on a local client, the encrypted metadata of the object is protected and cannot be modified by calling CopyObject.

You can use two types of CMKs for client-side encryption.

KMS-managed CMKs
User-managed CMKs

For the complete sample code for client-side encryption, visit GitHub.

Use KMS-managed CMKs

If you use a KMS-managed CMK for client-side encryption, you need to only specify a CMK ID when you upload an object. You do not need to provide the encryption client with a data key. The following figure shows the encryption process.

Encrypt and upload an object
1. Obtain a data key.
  The client uses the specified CMK ID to request a data key from KMS to encrypt the object. KMS generates a random key and returns the plaintext and ciphertext of the data key.
2. Encrypt the object and upload the encrypted object to OSS.
  The client receives the plaintext and ciphertext of the data key from KMS and uses the plaintext of the data key to encrypt the object on the client side.
Download and decrypt an object
1. Download an object.
  The client downloads an encrypted object from OSS. The ciphertext of the data key is included in the metadata of the object.
2. Decrypt the object.
  The client sends the ciphertext of the data key and the corresponding CMK ID to KMS. KMS uses the CMK sent by the client to decrypt the encrypted data and returns the plaintext of the data key to the client.

Note

The client uses a unique data key to encrypt each object to upload.
To ensure data security, we recommend that you periodically rotate or update the CMK.
You need to maintain the mapping relationship between the CMK IDs and the encrypted objects.

Use customer-managed CMKs

To use user-managed CMKs for client-side encryption, you must manually generate and manage CMKs. You must upload a symmetric CMK or an asymmetric CMK to the encryption client for object encryption. The following figure shows the encryption process.

Encrypt and upload an object
1. You provide the client with a symmetric CMK or an asymmetric CMK.
2. The client uses the CMK to generate a one-time symmetric data key that is used only to encrypt the object to upload. The client generates a random and unique data key for each object that you want to upload.
3. The client uses the data key to encrypt the object that you want to upload and uses the CMK to encrypt the data key.
4. The client uploads the encrypted object to OSS along with the encrypted data key included in the metadata of the object.
Download and decrypt an object
1. The client downloads an encrypted object and its metadata from OSS.
2. The client uses the corresponding CMK to decrypt the encrypted data key based on the key material indicated by the metadata, and then uses the decrypted data key to decrypt the object.

Important

The client does not send CMKs and unencrypted data to OSS. Therefore, keep your CMKs secure. If a CMK is lost, the objects that are encrypted by using the data keys generated by using this CMK cannot be decrypted.
Data keys are randomly generated by the client.

Use OSS SDKs

The following sample code provides examples on how to configure client-side encryption by using OSS SDKs for common programming languages. For more information about how to configure client-side encryption by using OSS SDKs for other programming languages, see Overview.

Java

import com.aliyun.oss.*;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.crypto.SimpleRSAEncryptionMaterials;
import com.aliyun.oss.model.OSSObject;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.InputStreamReader;
import java.security.KeyPair;
import java.security.interfaces.RSAPrivateKey;
import java.security.interfaces.RSAPublicKey;
import java.util.HashMap;
import java.util.Map;

public class Demo {
    public static void main(String[] args) throws Throwable {
        // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Specify the name of the bucket. Example: examplebucket. 
        String bucketName = "examplebucket";
        // Specify the full path of the object. Example: exampleobject.txt. Do not include the bucket name in the full path. 
        String objectName = "exampleobject.txt";
        String content = "Hello OSS!";

        // Specify your RSA private key string. You can generate the string by using OpenSSL. The following sample code shows a sample RSA private key string. 
        final String PRIVATE_PKCS1_PEM =
                "-----BEGIN RSA PRIVATE KEY-----\n" +
                "MIICWwIBAAKBgQCokfiAVXXf5ImFzKDw+XO/UByW6mse2QsIgz3ZwBtMNu59fR5z\n" +
                "ttSx+8fB7vR4CN3bTztrP9A6bjoN0FFnhlQ3vNJC5MFO1PByrE/MNd5AAfSVba93\n" +
                "I6sx8NSk5MzUCA4NJzAUqYOEWGtGBcom6kEF6MmR1EKib1Id8hpooY5xaQIDAQAB\n" +
                "AoGAOPUZgkNeEMinrw31U3b2JS5sepG6oDG2CKpPu8OtdZMaAkzEfVTJiVoJpP2Y\n" +
                "nPZiADhFW3e0ZAnak9BPsSsySRaSNmR465cG9tbqpXFKh9Rp/sCPo4Jq2n65yood\n" +
                "JBrnGr6/xhYvNa14sQ6xjjfSgRNBSXD1XXNF4kALwgZyCAECQQDV7t4bTx9FbEs5\n" +
                "36nAxPsPM6aACXaOkv6d9LXI7A0J8Zf42FeBV6RK0q7QG5iNNd1WJHSXIITUizVF\n" +
                "6aX5NnvFAkEAybeXNOwUvYtkgxF4s28s6gn11c5HZw4/a8vZm2tXXK/QfTQrJVXp\n" +
                "VwxmSr0FAajWAlcYN/fGkX1pWA041CKFVQJAG08ozzekeEpAuByTIOaEXgZr5MBQ\n" +
                "gBbHpgZNBl8Lsw9CJSQI15wGfv6yDiLXsH8FyC9TKs+d5Tv4Cvquk0efOQJAd9OC\n" +
                "lCKFs48hdyaiz9yEDsc57PdrvRFepVdj/gpGzD14mVerJbOiOF6aSV19ot27u4on\n" +
                "Td/3aifYs0CveHzFPQJAWb4LCDwqLctfzziG7/S7Z74gyq5qZF4FUElOAZkz123E\n" +
                "yZvADwuz/4aK0od0lX9c4Jp7Mo5vQ4TvdoBnPuGo****\n" +
                "-----END RSA PRIVATE KEY-----";
        // Specify your RSA public key string. You can generate the string by using OpenSSL. The following sample code shows a sample RSA public key string. 
        final String PUBLIC_X509_PEM =
                "-----BEGIN PUBLIC KEY-----\n" +
                "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCokfiAVXXf5ImFzKDw+XO/UByW\n" +
                "6mse2QsIgz3ZwBtMNu59fR5zttSx+8fB7vR4CN3bTztrP9A6bjoN0FFnhlQ3vNJC\n" +
                "5MFO1PByrE/MNd5AAfSVba93I6sx8NSk5MzUCA4NJzAUqYOEWGtGBcom6kEF6MnR\n" +
                "1EKib1Id8hpooY5xaQID****\n" +
                "-----END PUBLIC KEY-----";

        // Create an RSA key pair. 
        RSAPrivateKey privateKey = SimpleRSAEncryptionMaterials.getPrivateKeyFromPemPKCS1(PRIVATE_PKCS1_PEM);
        RSAPublicKey publicKey = SimpleRSAEncryptionMaterials.getPublicKeyFromPemX509(PUBLIC_X509_PEM);
        KeyPair keyPair = new KeyPair(publicKey, privateKey);

        // Specify the description of the CMK. The description cannot be modified after it is specified. You can specify only one description for a CMK. 
        // If all objects share a CMK, you can leave the description of the CMK empty. In this case, you cannot replace the CMK. 
        // If you leave the description of the CMK empty, the client cannot determine which CMK to use for decryption. 
        // We recommend that you specify a description for each CMK and store the mapping relationship between the CMK and the description on the client. The server does not store the mapping relationship. 
        Map<String, String> matDesc = new HashMap<String, String>();
        matDesc.put("desc-key", "desc-value");

        // Create RSA encryption materials. 
        SimpleRSAEncryptionMaterials encryptionMaterials = new SimpleRSAEncryptionMaterials(keyPair, matDesc);
        // To download and decrypt objects encrypted by using other CMKs, add these CMKs and their descriptions to the encryption materials. 
        // encryptionMaterials.addKeyPairDescMaterial(<otherKeyPair>, <otherKeyPairMatDesc>);

        // Create a client for client-side encryption. 
        OSSEncryptionClient ossEncryptionClient = new OSSEncryptionClientBuilder().
                build(endpoint, credentialsProvider, encryptionMaterials);

        try {
            // Encrypt the object that you want to upload. 
            ossEncryptionClient.putObject(bucketName, objectName, new ByteArrayInputStream(content.getBytes()));

            // Download the object. The object is automatically decrypted. 
            OSSObject ossObject = ossEncryptionClient.getObject(bucketName, objectName);
            BufferedReader reader = new BufferedReader(new InputStreamReader(ossObject.getObjectContent()));
            StringBuffer buffer = new StringBuffer();
            String line;
            while ((line = reader.readLine()) != null) {
                buffer.append(line);
            }
            reader.close();

            // Check whether the decrypted content is the same as the uploaded object in plaintext. 
            System.out.println("Put plain text: " + content);
            System.out.println("Get and decrypted text: " + buffer.toString());
        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossEncryptionClient != null) {
                ossEncryptionClient.shutdown();
            }
        }
    }
}

Python

# -*- coding: utf-8 -*-
import os
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from  oss2.crypto import RsaProvider
from oss2.cryptoimportAliKMSProvider

# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuth(EnvironmentVariableCredentialsProvider())

kms_provider=AliKMSProvider(auth, 'yourRegion', 'yourCMKID')
bucket = oss2.CryptoBucket(auth, 'yourEndpoint', 'yourBucketName', crypto_provider = kms_provider)

key = 'motto.txt'
content = b'a' * 1024 * 1024
filename = 'download.txt'


# Upload the object. 
bucket.put_object(key, content, headers={'content-length': str(1024 * 1024)})

# Download the object from OSS to the local memory. 
result = bucket.get_object(key)

# Check the data consistency between the content of the downloaded object and that of the object before upload. 
content_got = b''
for chunk in result:
    content_got += chunk
assert content_got == content

# Download the object from OSS to the local memory. 
result = bucket.get_object_to_file(key, filename)

# Check the data consistency between the content of the downloaded object and that of the object before upload. 
with open(filename, 'rb') as fileobj:
    assert fileobj.read() == content

package main

import (
	"bytes"
	"fmt"
	"io/ioutil"
	"os"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
	"github.com/aliyun/aliyun-oss-go-sdk/oss/crypto"
)

func main() {
	// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	// Create an OSSClient instance. 
	// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
	client, err := oss.New("yourEndpoint", "", "", oss.SetCredentialsProvider(&provider))
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Specify a description for the CMK. The description cannot be modified after being specified. You can specify only one description for a CMK. 
	// If all objects share a CMK, you can leave the description of the CMK empty. In this case, the CMK cannot be replaced. 
	// If the description of the CMK is left empty, the client cannot determine which CMK to use for decryption. 
	// We recommend that you configure a description in the form of a JSON string for each CMK and store the mapping relationship between the CMK and the description on the client. The server does not store the mapping relationship. 

	// Obtain the mapping relationship based on the CMK description that is a JSON string. 
	materialDesc := make(map[string]string)
	materialDesc["desc"] = "your master encrypt key material describe information"

	// Create a CMK object based on the description of the CMK. 
	// Set yourRsaPublicKey to the public key of the CMK that you manage, and set yourRsaPrivateKey to the private key of the CMK that you manage. 
	masterRsaCipher, err := osscrypto.CreateMasterRsa(materialDesc, "yourRsaPublicKey", "yourRsaPrivateKey")
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Create an encryption operation based on the CMK object and perform encryption in AES CTR mode. 
	contentProvider := osscrypto.CreateAesCtrCipher(masterRsaCipher)

	// Obtain an existing bucket to be used for client-side encryption. 
	// The bucket for which client-side encryption is configured is used in the same manner in which a common bucket is used. 
	cryptoBucket, err := osscrypto.GetCryptoBucket(client, "yourBucketName", contentProvider)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Encryption is automatically performed when you call the PutObject operation. 
	err = cryptoBucket.PutObject("yourObjectName", bytes.NewReader([]byte("yourObjectValueByteArrary")))
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Decryption is automatically performed when you call the GetObject operation. 
	body, err := cryptoBucket.GetObject("yourObjectName")
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	defer body.Close()

	data, err := ioutil.ReadAll(body)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	fmt.Println("data:", string(data))
}

C++

#include <alibabacloud/oss/OssEncryptionClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize information about the account that is used to access OSS. */
    
    /* Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Specify the name of the bucket. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. Do not include the bucket name in the full path. Example: exampledir/exampleobject.txt. */
    std::string ObjectName = "exampledir/exampleobject.txt";

    /* Specify the CMK and the description. */
    std::string RSAPublicKey = "your rsa public key";
    std::string RSAPrivateKey = "your rsa private key";
    std::map<std::string, std::string> desc;
    desc["comment"] = "your comment";

    /* Initialize resources, such as network resources. */
    InitializeSdk();
    
    ClientConfiguration conf;
    /* Obtain access credential from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
  
    CryptoConfiguration cryptoConf;
    auto materials = std::make_shared<SimpleRSAEncryptionMaterials>(RSAPublicKey, RSAPrivateKey, desc);
    OssEncryptionClient client(Endpoint, credentialsProvider, conf, materials, cryptoConf);
    /* Upload the object. */
    auto outcome = client.PutObject(BucketName, ObjectName, "yourLocalFilename");
    if (!outcome.isSuccess()) {
        /* Handle exceptions. */
        std::cout << "PutObject fail" <<
        ",code:" << outcome.error().Code() <<
        ",message:" << outcome.error().Message() <<
        ",requestId:" << outcome.error().RequestId() << std::endl;
        return -1;
    }
    /* Release resources, such as network resources. */
    ShutdownSdk();
    return 0;
}