Elastic GPU Service:GPU-accelerated compute-optimized instance families

Instance types

Features:

• You can select instance types that provide different numbers of GPUs and CPUs to meet your business requirements for AI use cases.

• This instance family uses the third-generation SHENLONG architecture and doubles the average bandwidths of virtual private clouds (VPCs), networks, and disks compared with instance families of the previous generation.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports only ESSDs and ESSD AutoPL disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • Small- and medium-scale AI training workloads

  • High-performance computing (HPC) business accelerated by using Compute Unified Device Architecture (CUDA)

  • AI inference tasks that require high GPU processing capabilities or large amounts of GPU memory

  • Deep learning applications such as training applications of AI algorithms used in image classification, autonomous vehicles, and speech recognition.

  • Scientific computing applications that require robust GPU computing capabilities such as computational fluid dynamics, computational finance, molecular dynamics, and environmental analytics.

  Important

  When you use AI training services that feature a high communication load, such as transformer models, you must enable NVLink for GPU-to-GPU communication. Otherwise, data may be damaged due to unpredictable failures that are caused by large-scale data transmission over Peripheral Component Interconnect Express (PCIe) links. If you do not understand the topology of the communication links that are used for AI training services, submit a ticket to obtain technical support.

Instance types

Features:

• This instance family uses the third-generation SHENLONG architecture to provide predictable and consistent ultra-high performance. This instance family utilizes fast path acceleration on chips to improve storage performance, network performance, and computing stability by an order of magnitude.

• Compute:

  • Uses NVIDIA A10 GPUs that have the following features:

    • Innovative Ampere architecture

    • Support for acceleration features such as RTX and TensorRT

  • Uses 2.9 GHz Intel^® Xeon^® Scalable (Ice Lake) processors that deliver an all-core turbo frequency of 3.5 GHz.

  • Provides up to 752 GiB of memory, which is much larger than the memory sizes of the gn6i instance family.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports only ESSDs and ESSD AutoPL disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • Concurrent AI inference tasks that require high-performance CPUs, memory, and GPUs, such as image recognition, speech recognition, and behavior identification

  • Compute-intensive graphics processing tasks that require high-performance 3D graphics virtualization capabilities, such as remote graphic design and cloud gaming

Instance types

Features:

• This instance family uses the latest Intel Ice Lake processors and NVIDIA A30 GPUs that are based on NVIDIA Ampere architecture. You can select instance types that comprise appropriate mixes of GPUs and vCPUs to meet your business requirements in AI scenarios.

• This instance family uses the third-generation SHENLONG architecture and doubles the average bandwidths of VPCs, networks, and disks compared with instance families of the previous generation.

• Compute:

  • Uses NVIDIA A30 GPUs that have the following features:

    • Innovative NVIDIA Ampere architecture

    • Support for the multi-instance GPU (MIG) feature and acceleration features (based on second-generation Tensor cores) to provide diversified business support

  • Uses 2.9 GHz Intel^® Xeon^® Scalable (Ice Lake) processors that deliver an all-core turbo frequency of 3.5 GHz.

  • Improves memory sizes significantly from instance families of the previous generation.

• Storage: Supports only enhanced SSDs (ESSDs) and ESSD AutoPL disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios: concurrent AI inference tasks that require high-performance CPUs, memory, and GPUs, such as image recognition, speech recognition, and behavior identification.

Instance types

Features:

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports only ESSDs and ESSD AutoPL disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • Deep learning applications such as training applications of AI algorithms used in image classification, autonomous vehicles, and speech recognition

  • Scientific computing applications that require robust GPU computing capabilities such as computational fluid dynamics, computational finance, molecular dynamics, and environmental analytics

Instance types

Features:

• Compute:

  • Uses NVIDIA T4 GPUs that have the following features:

    • Innovative NVIDIA Turing architecture

    • 16 GB memory (320 GB/s bandwidth) per GPU

    • 2,560 CUDA cores per GPU

    • Up to 320 Turing Tensor cores per GPU

    • Mixed-precision Tensor cores that support 65 FP16 TFLOPS, 130 INT8 TOPS, and 260 INT4 TOPS

  • Offers a CPU-to-memory ratio of 1:4.

  • Uses 2.5 GHz Intel^® Xeon^® Platinum 8163 (Skylake) processors.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports ESSDs, ESSD AutoPL disks, standard SSDs, and ultra disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • AI (deep learning and machine learning) inference for computer vision, speech recognition, speech synthesis, natural language processing (NLP), machine translation, and recommendation systems

  • Real-time rendering for cloud gaming

  • Real-time rendering for AR and VR applications

  • Graphics workstations or graphics-heavy computing

  • GPU-accelerated databases

  • High-performance computing

Instance types

Features:

• Compute:

  • Uses NVIDIA V100 GPUs, each of which has 32 GB of GPU memory and supports NVLink.

  • Uses NVIDIA V100 GPUs (SXM2-based) that have the following features:

    • Innovative NVIDIA Volta architecture

    • 32 GB HBM2 memory (900 GB/s bandwidth) per GPU

    • 5,120 CUDA cores per GPU

    • 640 Tensor cores per GPU

    • Support for up to six NVLink bidirectional connections, each of which provides a bandwidth of 25 GB/s in each direction for a total bandwidth of 300 GB/s (6 × 25 × 2 = 300)

  • Offers a CPU-to-memory ratio of 1:8.

  • Uses 2.5 GHz Intel^® Xeon^® Platinum 8163 (Skylake) processors.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports ESSDs, ESSD AutoPL disks, standard SSDs, and ultra disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • Deep learning applications such as the training and inference applications of AI algorithms used in image classification, autonomous driving, and speech recognition

  • Scientific computing applications, such as computational fluid dynamics, computational finance, molecular dynamics, and environmental analytics

Instance types

Features:

• Compute:

  • Uses NVIDIA V100 GPUs.

  • Uses NVIDIA V100 GPUs (SXM2-based) that have the following features:

    • Innovative NVIDIA Volta architecture

    • 16 GB HBM2 memory (900 GB/s bandwidth) per GPU

    • 5,120 CUDA cores per GPU

    • 640 Tensor cores per GPU

    • Support for up to six NVLink bidirectional connections, each of which provides a bandwidth of 25 GB/s in each direction for a total bandwidth of 300 GB/s (6 × 25 × 2 = 300)

  • Offers a CPU-to-memory ratio of 1:4.

  • Uses 2.5 GHz Intel^® Xeon^® Platinum 8163 (Skylake) processors.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports ESSDs, ESSD AutoPL disks, standard SSDs, and ultra disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • Deep learning applications such as the training and inference applications of AI algorithms used in image classification, autonomous driving, and speech recognition

  • Scientific computing applications, such as computational fluid dynamics, computational finance, molecular dynamics, and environmental analytics

Instance types

Instance types

Features:

• This instance family uses the SHENLONG architecture to provide flexible and powerful software-defined compute.

• Compute:

  • Uses 2.9 GHz Intel^® Xeon^® Scalable (Ice Lake) processors that deliver an all-core turbo frequency of 3.5 GHz and supports PCIe 4.0 interfaces.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports only ESSDs and ESSD AutoPL disks.

• Network:

  • Supports IPv6.

  • Provides ultra-high network performance with a packet forwarding rate of 24,000,000 pps.

• Supported scenarios:

  • Deep learning training and development

  • High-performance computing (HPC) and simulations

  Important

  When you use AI training services that feature a high communication load, such as transformer models, you must enable NVLink for GPU-to-GPU communication. Otherwise, data may be damaged due to unpredictable failures that are caused by large-scale data transmission over Peripheral Component Interconnect Express (PCIe) links. If you do not understand the topology of the communication links that are used for AI training services, submit a ticket to obtain technical support.

Instance types

Features:

• This instance family uses the SHENLONG architecture to provide flexible and powerful software-defined compute.

• Compute:

  • Uses NVIDIA A10 GPUs that have the following features:

    • Innovative NVIDIA Ampere architecture

    • Support for acceleration features such as vGPU, RTX technology, and TensorRT inference engine

  • Uses 2.9 GHz Intel^® Xeon^® Scalable (Ice Lake) processors that deliver an all-core turbo frequency of 3.5 GHz.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports only ESSDs and ESSD AutoPL disks.

• Network:

  • Supports IPv6.

  • Provides ultra-high network performance with a packet forwarding rate of 24,000,000 pps.

• Supported scenarios:

  • Concurrent AI inference tasks that require high-performance CPUs, memory, and GPUs, such as image recognition, speech recognition, and behavior identification

  • Compute-intensive graphics processing tasks that require high-performance 3D graphics virtualization capabilities, such as remote graphic design and cloud gaming

  • Scenarios that require high network bandwidth and disk bandwidth, such as the creation of high-performance render farms

  • Small-scale deep learning and training applications that require high network bandwidth

Instance types

Features:

• This instance family uses the SHENLONG architecture to provide flexible and powerful software-defined compute.

• Compute:

  • Uses 2.5 GHz Intel^® Xeon^® Platinum 8269CY (Cascade Lake) processors.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports only ESSDs and ESSD AutoPL disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • Deep learning applications, such as training applications of AI algorithms used in image classification, autonomous vehicles, and speech recognition

  • Scientific computing applications that require robust GPU computing capabilities such as computational fluid dynamics, computational finance, molecular dynamics, and environmental analytics

Instance types

Features:

• This instance family uses the third-generation SHENLONG architecture and fast path acceleration on chips to provide predictable and consistent ultra-high computing, storage, and network performance.

• This instance family uses NVIDIA T4 GPUs to offer GPU acceleration capabilities for graphics and AI applications and adopts container technology to start up to 60 virtual Android devices and provide hardware-accelerated video transcoding.

• Compute:

  • Offers a CPU-to-memory ratio of 1:3.

  • Uses 2.8 GHz Ampere^® Altra^® Arm-based processors that deliver a turbo frequency of 3.0 GHz and provides high performance and high compatibility with applications for Android servers.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports only ESSDs and ESSD AutoPL disks.

• Network:

  • Supports IPv6.

• Supported scenarios:

  • Remote application services based on Android, such as always-on cloud-based services, cloud-based mobile games, cloud-based mobile phones, and Android service crawlers.

Instance types

Features:

• This instance family uses the SHENLONG architecture to provide flexible and powerful software-defined compute.

• This instance family uses NVIDIA V100 GPUs that each has 32 GB of GPU memory and support NVLink.

• This instance family uses NVIDIA V100 GPUs (SXM2-based) that have the following features:

  • Innovative NVIDIA Volta architecture.

  • 32 GB of HBM2 memory (900 GB/s bandwidth) per GPU.

  • 5,120 CUDA cores per GPU.

  • 640 Tensor cores per GPU.

  • Support for up to six NVLink connections per GPU. Each NVLink connection provides a bandwidth of 25 GB/s in each direction for a total bandwidth of 300 GB/s (6 × 25 × 2 = 300).

• Compute:

  • Offers a CPU-to-memory ratio of 1:8.

  • Uses 2.5 GHz Intel^® Xeon^® Platinum 8163 (Skylake) processors.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports ESSDs, ESSD AutoPL disks, standard SSDs, and ultra disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • Deep learning applications, such as training and inference applications of AI algorithms used in image classification, autonomous vehicles, and speech recognition

  • Scientific computing applications, such as computational fluid dynamics, computational finance, molecular dynamics, and environmental analytics

Instance types

Features:

• This instance family uses the SHENLONG architecture to provide flexible and powerful software-defined compute.

• This instance family uses NVIDIA V100 GPUs.

• This instance family uses NVIDIA V100 GPUs (SXM2-based) that have the following features:

  • Innovative NVIDIA Volta architecture.

  • 16 GB of HBM2 memory (900 GB/s bandwidth) per GPU

  • 5,120 CUDA cores per GPU.

  • 640 Tensor cores per GPU.

  • Support for up to six NVLink connections per GPU. Each NVLink connection provides a bandwidth of 25 GB/s in each direction for a total bandwidth of 300 GB/s (6 × 25 × 2 = 300).

• Compute:

  • Offers a CPU-to-memory ratio of 1:4.

  • Uses 2.5 GHz Intel^® Xeon^® Platinum 8163 (Skylake) processors.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports ESSDs, ESSD AutoPL disks, standard SSDs, and ultra disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • Deep learning applications, such as training and inference applications of AI algorithms used in image classification, autonomous vehicles, and speech recognition

  • Scientific computing applications, such as computational fluid dynamics, computational finance, molecular dynamics, and environmental analytics

Instance types

Features:

• This instance family uses the SHENLONG architecture to provide flexible and powerful software-defined compute.

• This instance family uses NVIDIA T4 GPUs that have the following features:

  • Innovative NVIDIA Turing architecture

  • 16 GB of memory (320 GB/s bandwidth) per GPU

  • 2,560 CUDA cores per GPU

  • Up to 320 Turing Tensor cores per GPU

  • Mixed-precision Tensor cores that support 65 FP16 TFLOPS, 130 INT8 TOPS, and 260 INT4 TOPS

• Compute:

  • Offers a CPU-to-memory ratio of 1:4.

  • Uses 2.5 GHz Intel^® Xeon^® Platinum 8163 (Skylake) processors.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports ESSDs, ESSD AutoPL disks, standard SSDs, and ultra disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • AI (deep learning and machine learning) inference for computer vision, voice recognition, speech synthesis, natural language processing (NLP), machine translation, and reference systems

  • Real-time rendering for cloud games

  • Real-time rendering for AR and VR applications

  • Graphics workstations or graphics-heavy computing

  • GPU-accelerated databases

  • High-performance computing

Instance types

Features:

• Compute:

  • Uses NVIDIA P4 GPUs.

  • Offers a CPU-to-memory ratio of 1:4.

  • Uses 2.5 GHz Intel^® Xeon^® E5-2682 v4 (Broadwell) processors.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • Supports only standard SSDs and ultra disks.

• Network:

  • Supports IPv6.

  • Provides high network performance based on large computing capacity.

• Supported scenarios:

  • Deep learning inference

  • Server-side GPU compute workloads such as multi-media encoding and decoding

Instance types