Build a TDX confidential computing environment and verify remote attestation - Elastic Compute Service

This topic describes how to build a Trust Domain Extensions (TDX) confidential computing environment on an Intel^® TDX-enabled Elastic Compute Service (ECS) instance and verify the TDX feature on the instance.

Background information

Intel^® TDX is a CPU hardware-based technology that provides hardware-assisted isolation and encryption for ECS instances to protect runtime data, such as CPU registers, memory data, and interrupt injections. Intel® TDX helps achieve higher levels of data privacy and mitigate risks that are associated with unauthorized access to running processes or sensitive data that is being processed. For more information about Intel^® TDX, see Intel® Trust Domain Extensions (Intel® TDX).

Intel^® TDX provides default out-of-the-box protection for your ECS instances and applications. You can migrate applications to a TDX-enabled instance without the need to modify the application code.

Create a TDX-enabled instance

Create a TDX-enabled instance in the ECS console

The steps for creating a TDX-enabled instance in the ECS console are similar to the steps for creating a regular instance. Take note of specific parameters. The following section describes the parameters specific to TDX-enabled instances. For information about other parameters that are common to all instances, see Create an instance on the Custom Launch tab.

Log on to the ECS console.
In the left-side navigation pane, choose Instances & Images > Instances.
In the top navigation bar, select the region and resource group to which the resource belongs.

Click Create Instance and configure the parameters. The following table describes the parameters.

Parameter	Description
Network and Zone	You can only select Beijing Zone I.
Instance Type	You can only select ecs.g8i.xlarge or a larger instance type.
Image	Select Alibaba Cloud Linux 3.2104 LTS 64-bit (UEFI) as the image version, and then select Confidential VM.

Create the instance as prompted.

Create a TDX-enabled instance by calling an API operation or by using Alibaba Cloud CLI

You can call the RunInstances operation or use Alibaba Cloud CLI to create a TDX-enabled instance. The following table describes the related parameters.

Parameter	Description	Example value
RegionId	Select the China (Beijing) region.	cn-beijing
ZoneId	Select Beijing Zone I.	cn-beijing-i
InstanceType	Select ecs.g8i.xlarge or a larger instance type.	ecs.g8i.2xlarge
ImageId	Specify the ID of an image that supports TDX. Only Alibaba Cloud Linux 3.2104 LTS 64-bit Unified Extensible Firmware Interface (UEFI) images whose kernel version is 5.10.134-16.al8.x86_64 or later support TDX.	aliyun_3_9_x64_20G_uefi_alibase_20231219.vhd
SecurityOptions.ConfidentialComputingMode	Specify the confidential computing mode.	TDX

Example for creating a TDX-enabled instance by using Alibaba Cloud CLI

aliyun ecs RunInstances \
  --SecurityOptions.ConfidentialComputingMode TDX \
  --Region cn-beijing \
  --ZoneId cn-beijing-i \
  --SystemDisk.Category cloud_essd \
  --ImageId 'aliyun_3_9_x64_20G_uefi_alibase_20231219.vhd' \
  --InstanceType 'ecs.g8i.2xlarge' \
  --SecurityGroupId 'sg-[SecurityGroupId]' \
  --VSwitchId 'vsw-[VSwitchID]' \
  --KeyPairName [KEY_PAIR_NAME] \

Procedure

Step 1: Check whether TDX is enabled

Before you build a TDX confidential computing environment on an ECS instance, make sure that TDX is enabled on the instance.

Check whether TDX is enabled.
```
lscpu |grep -i tdx_guest
```
The following command output indicates that TDX is enabled.
Check whether the TDX driver is installed.
```
ls -l /dev/tdx_guest
```
The following command output indicates that the TDX driver is installed.

Step 2: Build a TDX confidential computing environment

Note

In most cases, you can migrate applications from a non-TDX environment to a TDX environment without the need to modify the application code. You can use TDX-enabled instances in the same manner that you use regular instances. We recommend that you use the features that are described in the following sections to maximize TDX protection for your ECS instances.

TDX Report is a data structure that is generated by CPU hardware to represent the identities of TDX-enabled instances. TDX Report includes critical information about TDX-enabled instances, such as attributes (ATTRIBUTES), runtime-extendable measurement registers (RTMR), and trusted-computing base (TCB) security-version number (SVN). TDX Report uses cryptographic methods to ensure information integrity. For more information, see Intel TDX Module.

Import the YUM software repository for Alibaba Cloud confidential computing.
- The public URLs of the YUM software repository are in the following format: https://enclave-[Region-ID].oss-[Region-ID].aliyuncs.com/repo/alinux/enclave-expr.repo.
- The internal URLs of the YUM software repository are in the following format: https://enclave-[Region-ID].oss-[Region-ID]-internal.aliyuncs.com/repo/alinux/enclave-expr.repo.
Replace [Region-ID] in the preceding URLs with the actual region ID of the TDX-enabled instance. You can only create a TDX-enabled instance in the China (Beijing) region. To import the YUM software repository from an internal URL that is specific to the China (Beijing) region, run the following commands:
```
region="cn-beijing"

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://enclave-${region}.oss-${region}-internal.aliyuncs.com/repo/alinux/enclave-expr.repo
```

Install a compilation tool and sample code.

sudo yum groupinstall -y "Development Tools"
sudo yum install -y sgxsdk libtdx-attest-devel

Compile the sample TDXReportParse code.
1. Access the TDXReportParse directory.
```
cd /opt/alibaba/teesdk/intel/sgxsdk/SampleCode/TDXReportParse
```
2. Compile TDXReportParse.
```
sudo make
```
Run the compiled executable file.
```
sudo ./tdx_report_parse
```
The following command output is returned. If the TDX-enabled instance is protected, the instance is in a non-debugging state and NO_DEBUG is displayed on the attributes line.

Verify the TDX feature by using the remote attestation service

In an Alibaba Cloud TDX encryption environment, the remote attestation service can be used to verify the trustworthiness of the platform and the integrity and confidentiality of the code running on the platform. For more information, see Remote attestation service.

Platform is the hardware platform and virtualization software stacks used by Alibaba Cloud.
Code running on the platform is an operating system, such as Alibaba Cloud Linux, and applications, such as NGINX and Java, running in a TDX-enabled instance.

Install dependency packages required for TDX remote attestation.

sudo yum install -y gcc gcc-c++ make openssl-devel git jq
sudo yum install -y tdx-quote-generation-sample tee-appraisal-tool libsgx-dcap-ql-devel libsgx-dcap-quote-verify-devel libsgx-dcap-default-qpl-devel tdx-quote-verification-sample

Configure the Alibaba Cloud TDX remote attestation service.
Specify the PCCS_URL parameter in the /etc/sgx_default_qcnl.conf file. You can only specify the PCCS_URL parameter to point to the Distributed Cooperative Access Points (DCAP) service in the China (Beijing) region.
```
sudo sed -i.$(date "+%m%d%y") 's|PCCS_URL=.*|PCCS_URL=https://sgx-dcap-server.cn-beijing.aliyuncs.com/sgx/certification/v4/|' /etc/sgx_default_qcnl.conf
```
Access the tdx-quote-generation-sample directory and compile the tdx-quote-generation-sample file.
```
cd /opt/alibaba/tdx-quote-generation-sample
sudo make
```
Perform one of the following operations to enable the system to generate a remote attestation report, which is also referred to as a quote:
- Enable the system to randomly generate report_data and generate a quote.
```
sudo ./app
```
- Specify report_data and generate a quote.
```
sudo ./app -d <report_data_in_hex>
```
If the remote attestation report (quote) is generated, the command outputs similar to the command outputs shown in the following figure are returned.

Verify the remote attestation report (quote).

Configure and sign an appraisal policy.
Important
We recommend that you use a known and secure environment. Do not use the production environment to generate the appraisal policy.
You can define the appraisal policy in the JSON format.
- For example, you can configure the following appraisal policy to verify whether your TDX-enabled ECS instance runs in an encrypted and non-debugging state, which indicates that the instance is protected.
- You can also include additional parameters in the appraisal policy to verify the integrity of the operating system and applications. For more information about the appraisal policy, see Intel DCAP Appraisal Engine Developer Guide.
```
{
    "policy_array":[
        {
            "environment":{
                "class_id":"45b734fc-aa4e-4c3d-ad28-e43d08880e68",
                "description":"Application TD TCB 1.5"
            },
            "reference":{
                "tdx_attributes":"0000000010000000",
                "#NOTE": "0000000010000000 means for NO_DEBUG and SPTE_VE_DISABLE"
            }
        }
    ]
}
```
You can use the default policy in the /opt/alibaba/tdx-quote-verification-sample/Policies/tenant_td_policy.json file or compile a custom policy to use a policy token for subsequent verification. The policy token can be transferred to any environment that generates a remote attestation report for subsequent verification.
```
cd /opt/alibaba/tdx-quote-verification-sample
sudo make Policies/tenant_td_policy.jwt
```
If a policy token is generated, command outputs similar to the command outputs shown in the following figure are returned.
Compile other components on which TDX remote attestation depends.
```
cd /opt/alibaba/tdx-quote-verification-sample
sudo make all
```
If the dependent components are compiled, command outputs similar to the command outputs shown in the following figure are returned.

Verify the remote attestation report (quote).

Important

Replace the value of the <path_to_quote> parameter in the following command with the actual path of the remote attestation report (quote). Example: /opt/alibaba/tdx-quote-generation-sample/quote.dat.

Based on the policy in the Policies/tenant_td_policy.json file, the quote generated for remote attestation is verified, and the corresponding verification results are converted from the JSON Web Token (JWT) format into the standard format.
```
./verifier -quote <path_to_quote>
```
Command outputs similar to the command outputs shown in the following figure are returned.

Use RelyingParty to verify the identity of the JWT signature.

./verifier -quote <path_to_quote> |./relying_party -v |grep "json payload" |awk -F 'payload:' '{print $2}'|jq

Command outputs similar to the command outputs shown in the following figure are returned. In the command outputs, only specific fields are displayed.

Note

For simplicity, you can check the overall_appraisal_result field in appraisal_result to check whether the evidence complies with the appraisal policy.

./verifier -quote <path_to_quote> |./relying_party -v |grep "json payload" |awk -F 'payload:' '{print $2}'|jq '.[0].result.overall_appraisal_result'

In addition, the report_data parameter specified when the quote is generated is also included in the verification result of the remote attestation service. You can use the report_data value to implement business logic, such as key exchange.

./verifier -quote <path_to_quote> |./relying_party -v -a|grep "json payload" |awk -F 'payload:' '{print $2}'|jq '.. | .tdx_reportdata?  | select(. != null)'

[
  {
    "result": {
      "appraisal_check_date": 1710400829000000000,
      "nonce": 502551065253582,
      "certification_data": [
        {
          "certification_data": {
            "qe_identity_issuer_chain": "LS0t...",
            "root_ca_crl": "MzA4...",
            "pck_crl": "LS0t...",
            "pck_crl_issuer_chain": "LS0t...",
            "tcb_info": "eyJ0...",
            "qe_identity": "eyJl...",
            "tcb_info_issuer_chain": "LS0t..."
          }
        }
      ],
      "overall_appraisal_result": 1,
      "appraised_reports": [
        {
          "appraisal_result": 1,
          "detailed_result": [
            {
              "td_mrownerconfig_check": true,
              "td_xfam_check": true,
              "td_mrservicetd_check": true,
              "td_attributes_check": true,
              "td_rtmr3_check": true,
              "td_mrtd_check": true,
              "td_mrowner_check": true,
              "td_rtmr0_check": true,
              "td_mrconfigid_check": true,
              "td_rtmr1_check": true,
              "td_rtmr2_check": true
            }
          ],
          "policy": {
            "environment": {
              "description": "Application TD TCB 1.5",
              "class_id": "45b734fc-aa4e-4c3d-ad28-e43d08880e68"
            },
            "signature": "-6C0-...",
            "reference": {
              ...
            },
            "signing_key": {
              "kty": "EC",
              "crv": "P-384",
              "alg": "ES384",
              "y": "CeW8...",
              "x": "NmSa..."
            }
          },
          "report": {
            "environment": {
              "Description": "Application TD TCB",
              "class_id": "45b734fc-aa4e-4c3d-ad28-e43d08880e68"
            },
            "measurement": {
              "tdx_mrownerconfig": "0000...",
              "tdx_mrservicetd": "3D03...",
              "tdx_xfam": "00000000000642E7",
              "tdx_mrtd": "0A40...",
              "tdx_mrowner": "0000...",
              "tdx_attributes": "0000000010000000",
              "tdx_mrconfigid": "0000...",
              "tdx_reportdata": "D98B...",
              "tdx_rtmr3": "0000...",
              "tdx_rtmr2": "0000...",
              "tdx_rtmr1": "6368...",
              "tdx_rtmr0": "D0FD..."
            }
          }
        },
        {
          "appraisal_result": 1,
          "detailed_result": [
            {
              "platform_provider_id_check": true,
              "sgx_types_check": true,
              "cached_keys_check": true,
              "smt_enabled_check": true,
              "accepted_tcb_level_date_tag_check": true,
              "advisory_ids_check": true,
              "expiration_date_check": true,
              "tcb_eval_num_check": true,
              "earliest_accepted_tcb_level_date_tag_check": true,
              "tcb_status_check": true,
              "dynamic_platform_check": true
            }
          ],
          "policy": {
            "environment": {
              "description": "Alibaba Cloud Evaluation Num Policy for TDX Platform",
              "class_id": "f708b97f-0fb2-4e6b-8b03-8a5bcd1221d3"
            },
            "signature": "l00p...",
            "reference": {
              "accepted_tcb_status": [
                "UpToDate"
              ],
              "allow_dynamic_plaform": true,
              "min_eval_num": 16
            },
            "signing_key": {
              "kty": "EC",
              "crv": "P-384",
              "alg": "ES384",
              "y": "7hlr...",
              "x": "OSbD..."
            }
          },
          "report": {
            "environment": {
              "description": "TDX Platform TCB",
              "class_id": "f708b97f-0fb2-4e6b-8b03-8a5bcd1221d3"
            },
            "measurement": {
              "earliest_issue_date": "2018-05-21T10:45:10Z",
              "is_cached_keys_policy": true,
              "fmspc": "90C06F000000",
              "is_smt_enabled": true,
              "earliest_expiration_date": "2024-04-02T10:22:51Z",
              "root_ca_crl_num": 1,
              "root_key_id": "9309...",
              "pck_crl_num": 1,
              "tcb_eval_num": 16,
              "latest_issue_date": "2024-03-13T15:45:02Z",
              "tcb_status": [
                "UpToDate"
              ],
              "tcb_level_date_tag": "2023-08-09T00:00:00Z",
              "is_dynamic_platform": true,
              "sgx_types": 1
            }
          }
        },
        {
          "appraisal_result": 1,
          "detailed_result": [
            {
              "td_qe_expiration_date_check": true,
              "td_qe_tcb_eval_num_check": true,
              "td_qe_earliest_accepted_tcb_level_date_tag_check": true,
              "td_qe_tcb_status_check": true,
              "td_qe_accepted_tcb_level_date_tag_check": true
            }
          ],
          "policy": {
            "environment": {
              "description": "Alibaba Cloud Num Policy for Verified TDQE",
              "class_id": "3769258c-75e6-4bc7-8d72-d2b0e224cad2"
            },
            "signature": "l00p...",
            "reference": {
              "accepted_tcb_status": [
                "UpToDate"
              ],
              "min_eval_num": 16
            },
            "signing_key": {
              "kty": "EC",
              "crv": "P-384",
              "alg": "ES384",
              "y": "7hlr...",
              "x": "OSbD..."
            }
          },
          "report": {
            "environment": {
              "Description": "RAW TDX QE Report",
              "class_id": "3769258c-75e6-4bc7-8d72-d2b0e224cad2"
            },
            "measurement": {
              "earliest_expiration_date": "2025-05-21T10:50:10Z",
              "earliest_issue_date": "2018-05-21T10:45:10Z",
              "root_key_id": "9309...",
              "tcb_eval_num": 16,
              "latest_issue_date": "2018-05-21T10:50:10Z",
              "tcb_status": [
                "UpToDate"
              ],
              "tcb_level_date_tag": "2023-08-09T00:00:00Z"
            }
          }
        }
      ]
    }
  }
]

Known issues and functionality limits

TDX-enabled instances provide CPU hardware-based encryption to protect CPU registers and memory status. As a result, the overall performance of the TDX-enabled instances is affected.
You cannot restart TDX-enabled instances from within the instances due to the design limitations of RTMR, interrupt protection, and register state recovery. You cannot run the reboot command in the operating system of a TDX-enabled instance to restart the instance.
You cannot log on to TDX-enabled instances by using Virtual Network Computing (VNC). You can log on to TDX-enabled instances over SSH by using tools, such as Workbench and third-party clients. You can obtain the logs that are generated during the runtime of a TDX-enabled instance. For more information, see View system logs and screenshots or GetInstanceConsoleOutput.
The following limits on images apply:
- Only UEFI images are supported.
- Windows images are not supported.
- Images whose kernel version is 5.10.134-16.al8.x86_64 or later are supported.
Elastic Remote Direct Memory Access (eRDMA) and accelerators are not supported.
The following known issues are related to the Software Input Output Translation Lookaside Buffer (SWIOTLB) in the Linux kernel:
- The amount of available memory on TDX-enabled instances is smaller than the amount of available memory on regular instances. TDX-enabled instances use a specific area, called SWIOTLB, in unencrypted memory for peripheral communication. By default, the SWIOTLB consumes 6% of the total available memory of a TDX-enabled instance and does not exceed 1 GiB in size.
- If you bind multiple elastic network interfaces (ENIs) to a TDX-enabled instance of a large instance type, the instance may freeze when memory allocation fails due to insufficient SWIOTLB in network interface controller (NIC) multi-queue scenarios. You can stop the TDX-enabled instance and unbind ENIs from the instance in the ECS console to recover the instance.
- If your instance workload involves a large amount of I/O operations, the performance of the TDX-enabled instance may degrade due to insufficient SWIOTLB. You can run the following command to check whether the preceding issue exists:
```
dmesg| grep 'swiotlb buffer is full'
```
  Important
  Incorrect SWIOTLB-related parameter settings may cause your instance to fail to start. To facilitate data rollback when an error occurs, we recommend that you create snapshots for disks on the instance before you proceed. For more information, see Create a snapshot.
  If the preceding issue exists, you can perform the following steps to improve the performance of the instance by increasing the size of the SWIOTLB:
  1. Open the /etc/default/grub file.
```
vim /etc/default/grub
```
  2. Append the swiotlb parameter to the GRUB_CMDLINE_LINUX field.
    For example, change GRUB_CMDLINE_LINUX to GRUB_CMDLINE_LINUX="... swiotlb=524288" to increase the size of the SWIOTLB to 1 GiB.
    Note
    You can calculate the value of the swiotlb parameter by using the following formula: Size of the SWIOTLB that you require in MiB × 512. For more information, see The kernel's command-line parameters.
  3. Update the grub.cfg file to make the preceding modifications take effect.
```
grub2-mkconfig -o /boot/efi/EFI/alinux/grub.cfg
```