Object Storage Service:Can I use ETag values as OSS MD5 hashes to check data consistency?

Objects in Object Storage Service (OSS) have ETag values that are used to identify whether changes are made to data on the server. However, these ETag values are not necessarily equal to the MD5 hashes of the objects. We recommend that you do not use ETag values to verify data consistency.

To check whether an uploaded object in OSS is consistent with the local file, you can include the Content-MD5 header value in the upload request. When OSS receives the object, OSS compares the MD5 hash with the Content-MD5 header value. The object can be uploaded only when the MD5 hash is consistent with the Content-MD5 header value. This way, data consistency is ensured.

In the following examples, the string "0123456789" is used to show how to calculate the Content-MD5 value of the request content.

• Correct calculation
  1. Calculate the MD5 hash of the string, which is a 128-bit binary array.
  2. Encode the binary array (instead of the 32-bit string) in Base64.
  The following Python code provides an example on how to calculate the Content-MD5 value:

  >>> import base64,hashlib
  >>> hash = hashlib.md5()
  >>> hash.update("0123456789")   // If you use Python 3, change this part to hash.update(b"0123456789"). 
  >>> base64.b64encode(hash.digest())
  'eB5eJF1ptWaXm4bijSPyxw=='

  Call hash.digest() to calculate the 128-bit binary array.

  >>> hash.digest()
  'x\x1e^$]i\xb5f\x97\x9b\x86\xe2\x8d#\xf2\xc7'

• Incorrect calculation
  Note A common incorrect operation is to encode the calculated 32-bit string in Base64 to obtain the Content-MD5 value.

  # Call hash.hexdigest() to obtain a 32-bit plaintext string. 
  >>> hash.hexdigest()
  '781e5e245d69b566979b86e28d23f2c7'
  # The following code provides an example on the result of encoding the incorrect MD5 hash in Base64: 
  >>> base64.b64encode(hash.hexdigest())
  'NzgxZTVlMjQ1ZDY5YjU2Njk3OWI4NmUyOGQyM2YyYzc='