#amdinstinct

The AMD Pensando Pollara 400 NIC, AMD Pensando Salina DPU, and AMD Instinct MI325X accelerators are the newest networking and accelerator solutions that AMD unveiled today. These solutions will enable the next generation of AI infrastructure at scale. Performance benchmarks for Gen AI models and data centers are raised using AMD Instinct MI325X accelerators.

Based on the AMD CDNA 3 architecture, AMD Instinct MI325X accelerators are engineered to deliver outstanding performance and efficiency for challenging AI activities that include inferencing, fine-tuning, and foundation model training. With the help of these components, AMD partners and customers may develop AI systems that are highly efficient and optimized at the system, rack, and data center levels.

AMD Instinct MI325X Continues to Provide Superior AI Performance

Industry-leading memory capacity and bandwidth are provided by AMD Instinct MI325X accelerators; 256GB of HBM3E supporting 6.0TB/s offers 1.8X more capacity and 1.3X more bandwidth than the H200. In addition, the AMD Instinct MI325X has 1.3X higher peak theoretical compute performance for FP16 and FP8 than the H200.

Up to 1.3X, 1.2X, and 1.4X the inference performance on Mistral 7B at FP16, Llama 3.1 70B at FP8, and Mixtral 8x7B at FP16 of the H200, respectively, may be obtained with this leadership memory and compute.

Currently scheduled for production shipments in Q4 2024, AMD Instinct MI325X accelerators are anticipated to be widely available for use in systems from a variety of platform providers beginning in Q1 2025, including Dell Technologies, Eviden, Gigabyte, Hewlett Packard Enterprise, Lenovo, Supermicro, and others.

AMD showcased its upcoming AMD Instinct MI350 series accelerators, continuing its dedication to an annual roadmap cadence. In comparison to AMD CDNA 3-based accelerators, AMD Instinct MI350 series accelerators are built on the AMD CDNA 4 architecture and are intended to provide a 35x increase in inference performance.

The AMD Instinct MI350 series, which offers up to 288GB of HBM3E memory per accelerator, will continue to lead the market in memory capacity. The second part of 2025 is when the AMD Instinct MI350 series accelerators are expected to be ready.

AMD Next-Gen AI Networking

The most popular programmable DPU for hyperscalers is being used by AMD to support next-generation AI networking. AI networking, which is divided into two components the front-end, which provides data and information to an AI cluster, and the backend, which controls data transmission between accelerators and clusters is essential to making sure CPUs and accelerators are used effectively in AI infrastructure.

The AMD Pensando Pollara 400, the first AI NIC in the industry ready for the Ultra Ethernet Consortium (UEC), and the AMD Pensando Salina DPU were introduced by AMD to efficiently manage these two networks and promote high performance, scalability, and efficiency throughout the system.

The third iteration of the most powerful and programmable DPU in the world, the AMD Pensando Salina, offers up to two times the speed, bandwidth, and scalability of its predecessor. Optimizing performance, efficiency, security, and scalability for data-driven AI applications, the AMD Pensando Salina DPU is a crucial part of AI front-end network clusters, supporting 400G throughput for fast data transfer rates.

The AMD Pensando Salina DPU and AMD Pensando Pollara 400 are scheduled to be available in the first half of 2025, and they are now sampling with consumers in Q4 of 2024.

New Generative AI Capabilities Offered by AMD AI Software

In order to provide the AMD ROCm open software stack with powerful new features and capabilities, AMD keeps investing in expanding software capabilities and the open ecosystem.

Among the most popular AI frameworks, libraries, and models, such as PyTorch, Triton, Hugging Face, and many more, AMD is promoting support for AMD compute engines within the open software community. For well-known generative AI models like Stable Diffusion 3, Meta Llama 3, 3.1, and 3.2, as well as the more than one million models at Hugging Face, this work translates to unconventional performance and support with AMD Instinct accelerators.

With the addition of the newest features to support cutting-edge training and inference on generative AI workloads, AMD is further developing its ROCm open software stack outside of the community. Flash Attention 3, Kernel Fusion, FP8 datatype, and other important AI capabilities are now supported by ROCm 6.2. For a range of LLMs, ROCm 6.2 offers up to a 2.4X performance boost on inference and 1.8X on training when compared to ROCm 6.0.

Engineering Insights: Introducing AMD Instinct MI300X Accelerators’ MLPerf Results. The full-stack AMD inference platform demonstrated its prowess with the remarkable results AMD Instinct MI300X GPUs, powered by one of the most recent iterations of open-source ROCm, obtained in the MLPerf Inference v4.1 round.

LLaMA2-70B

The first submission concentrated on the well-known LLaMA2-70B type, which is renowned for its excellent performance and adaptability. By outperforming the NVIDIA H100 in Gen  AI inference, it established a high standard for what AMD Instinct MI300X accelerators are capable of.

MLPerf Inference

Comprehending MLPerf and Its Relevance to the Industry

Efficient and economical performance is becoming more and more important for inference and training as large language models (LLMs) continue to grow in size and complexity. Robust parallel processing and an optimal software stack are necessary to achieve high-performance LLMs.

This is where the best benchmarking package in the business, MLPerf, comes into play. The open-source AI benchmarks known as MLPerf Inference, which were created by the cross-industry cooperation MLCommons, of which AMD is a founding member, include Gen AI, LLMs, and other models that give exacting, peer-reviewed criteria. Businesses are able to assess the efficacy of AI technology and software by using these benchmarks.

A major accomplishment for AMD, excelling in MLPerf Inference v4.1 demonstrates their dedication to openness and providing standardized data that enables businesses to make wise choices.

An Extensive Analysis of the LLaMA2-70B Benchmark

The AMD LLaMA2-70B model was utilized in their first MLPerf Inference. A major development in LLMs, the LLaMA2-70B model is essential for practical uses such as large-scale inference and natural language processing. A Q&A scenario using 24,576 samples from the OpenORCA dataset, each with up to 1,024 input and output tokens, was included in the MLPerf benchmarking test. Two situations were analyzed by the benchmark to assess inference performance:

In an offline scenario, queries are processed in batches to increase throughput in tokens per second.

 Server Scenario: This model tests the hardware’s capacity to provide quick, responsive performance for low-latency workloads by simulating real-time queries with stringent latency limitations (TTFT* < 2s, TPOT* ≤ 200ms).

Performance of AMD Instinct MI300X in MLPerf

With four important entries for the LLaMA2-70B model, the AMD Instinct MI300X demonstrated remarkable performance in its first MLPerf Inference utilizing the Supermicro AS-8125GS-TNMR2 machine. These findings are especially noteworthy since they provide an apples-to-apples comparison with rival AI accelerators, are repeatable, vetted by peer review, and grounded in use cases that are relevant to the industry.

Combination Performance of CPU and GPU

Submission ID 4.1-0002: Two AMD EPYC 9374F (Genoa) CPUs paired with eight AMD Instinct MI300X accelerators in the Available category.

This setup demonstrated the potent synergy between 4th Gen EPYC CPUs (previously codenamed “Genoa”) and AMD Instinct MI300X GPU accelerators for AI workloads, providing performance within 2-3% of NVIDIA DGX H100 with 4th Gen Intel Xeon CPUs in both server and offline environments at FP8 precision.

Previewing Next-Generation CPU Performance

Submission ID 4.1-0070: Two AMD EPYC “Turin” CPUs and eight AMD Instinct MI300X CPUs in the Preview category.

It showcased the performance increases from the next AMD EPYC “Turin” 5th generation CPU when paired with AMD Instinct MI300X GPU accelerators. In the server scenario, it outperformed the NVIDIA DGX H100 with Intel Xeon by a small margin, and it maintained a similar level of performance even offline at FP8 precision.

LLaMA2-70B GPU

Efficiency of a Single GPU

Submission ID 4.1-0001: In the Available category, AMD Instinct MI300X accelerator with AMD EPYC 9374F 4th Gen CPUs (Genoa).

This submission emphasized the AMD Instinct MI300X’s enormous 192 GB memory, which allowed a single GPU to effectively execute the whole LLaMA2-70B model without requiring the network cost that comes with dividing the model over many GPUs at FP8 precision.

The AMD Instinct MI300X has 192 GB of HBM3 memory and a peak memory bandwidth of 5.3 TB/s thanks to its AMD CDNA 3 architecture. The AMD Instinct MI300X can execute and host a whole 70 billion parameter model, such as LLaMA2-70B, on a single GPU with ease because to its large capacity.

The findings in Figure 2 show that the scaling efficiency with the ROCm software stack is almost linear from 1x AMD Instinct MI300X (TP1) to 8x AMD Instinct MI300X (8x TP1), indicating that AMD Instinct MI300X can handle the biggest MLPerf inference model to date.

Outstanding Dell Server Architecture Outcomes Using AMD Instinct MI300X Processors

Submission ID 4.1-0022: Two Intel Xeon Platinum 8460Y+ processors and eight AMD Instinct MI300X accelerators in the Available category.

Along with AMD submissions, Dell used their PowerEdge XE9680 server and LLaMA2-70B to submit their findings, validating the platform-level performance of AMD Instinct accelerators on an 8x AMD Instinct MI300X arrangement. This proposal demonstrates their collaboration and emphasizes how strong it ecosystem is, making them a great option for deployments including both data centers and edge inference. Further information on such outcomes is available here.

Performance Of Engineering Insights

The AMD Instinct MI300X accelerators exhibit great competitive performance due to their high computational power, huge memory capacity with rapid bandwidth, and optimized ROCm software stack. The latter enables effective processing of large  AI models such as LLaMA2-70B. A few important elements were pivotal:

Big GPU Memory Capacity

The AMD Instinct MI300X has the most GPU memory that is currently on the market, which enables the whole LLaMA2-70B model to fit into memory while still supporting KV cache. By avoiding model splitting among GPUs, this maximizes inference speed while avoiding network cost.

Batch Sizes: They set the max_num_seqs parameter to 2048 in the offline scenario to optimize throughput, and to 768 in the server scenario to achieve latency requirements. These values are much greater than the 256 default value used in vLLM.

Effective KV cache management is made possible by the vLLM’s paged attention support, which helps prevent memory fragmentation brought on by huge memory AMD Instinct MI300X accelerators.

FP8 Precision

AMD expanded support for the FP8 numerical format throughout the whole inference software stack, using the AMD Instinct MI300X accelerator hardware. They quantized the LLaMA2-70B model weights to FP8 using Quark while maintaining the 99.9% accuracy needed by MLPerf. To further improve speed, it improved the hipBLASLt library, introduced FP8 support to vLLM, and implemented FP8 KV caching.

Software Enhancements

Kernel Optimization: AMD Composable Kernels (CK) based prefill attention, FP8 decode paged attention, and fused kernels such residual add RMS Norm, SwiGLU with FP8 output scaling were among the many profiles and optimizations to carried out.

vLLM Enhancements: The scheduler was improved to optimize both offline and server use cases, allowing for quicker decoding scheduling and better prefill batching.

CPU Enhancement

While GPUs handle the majority of the AI task processing, CPU speed is still quite important. CPUs with fewer cores and higher peak frequencies such as the 32-core EPYC 9374F offer the best performance, particularly in server applications. Performance improvements over the 4th generation EPYC CPUs which were submitted as a preview were seen during testing with the forthcoming “Turin” generation of EPYC CPUs.

LLaMa 3.1 405B

Establishing a Standard for the Biggest Model

The AMD Instinct MI300X GPU accelerators have shown their performance in MLPerf Inference with LLaMA2-70B, and the positive outcomes set a solid precedent for their future efficacy with even bigger models, such as Llama 3.1. They are pleased to provide Day 0 support for AMD Instinct MI300X accelerators with Meta’s new LLaMa 3.1 405B parameter model.

Only a server driven by eight AMD Instinct MI300X GPU accelerators can fit the whole LLaMa 3.1 model, with 405 billion parameters, on a single server utilizing FP16 datatype MI300-7A, owing to the industry-leading memory capacities of the AMD Instinct MI300X platform MI300-25. This lowers expenses and lowers server use. The most ideal way to power the biggest open models on the market right now is with AMD Instinct MI300X accelerators.

GigaIO

GigaIO & AMD: Facilitating increased computational effectiveness, scalability, and quicker AI workload deployments.

It always find it interesting to pick up knowledge from those that recognise the value of teamwork in invention. GigaIO CEO Alan Benjamin is one of those individuals. GigaIO is a workload-defined infrastructure provider for technical computing and  artificial intelligence.

GigaIO SuperNODE

They made headlines last year when they configured 32 AMD Instinct MI210 accelerators to a single-node server known as the SuperNODE. Previously, in order to access 32 GPUs, four servers with eight GPUs each were needed, along with the additional costs and latency involved in connecting all of that additional hardware. Alan and myself had a recent conversation for the AMD EPYC TechTalk audio series, which you can listen to here. In the blog article below, They’ve shared a few highlights from the interview.

Higher-performance computing (HPC) is in greater demand because to the emergence of generative AI at a time when businesses are routinely gathering, storing, and analysing enormous volumes of data. Data centres are therefore under more pressure to implement new infrastructures that meet these rising demands for performance and storage.

However, setting up larger HPC systems is more complicated, takes longer, and can be more expensive. There’s a chance that connecting or integrating these systems will result in choke points that impede response times and solution utilisation.

A solution for scaling accelerator technology is offered by Carlsbad, California-based GigaIO, which does away with the increased expenses, power consumption, and latency associated with multi-CPU systems. GigaIO provides FabreX, the dynamic memory fabric that assembles rack-scale resources, in addition to SuperNode. Data centres can free up compute and storage resources using GigaIO and share them around a cluster by using a disaggregated composable infrastructure (DCI).

GigaIO has put a lot of effort into offering something that may be even more beneficial than superior performance, in addition to assisting businesses in getting the most out of their computer resources.

GigaIO Networks Inc

“Easy setup and administration of rapid systems may be more significant than performance “Alan said. “Many augmented-training and inferencing companies have approached us for an easy way to enhance their capabilities. But assure them that their ideas will function seamlessly. You can take use of more GPUs by simply dropping your current container onto a SuperNODE.”

In order to deliver on the “it just works” claim, GigaIO and AMD collaborated to design the TensorFlow and PyTorch libraries into the SuperNODE’s hardware and software stack. SuperNODE will function with applications that have not been modified.

“Those optimised containers that are optimised literally for servers that have four or eight GPUs, you can drop them onto a SuperNODE with 32 GPUs and they will just run,” Alan stated. “In most cases you will get either 4x or close to 4x, the performance advantage.”

The necessity for HPC in the scientific and engineering communities gave rise to GigaIO. These industries’ compute needs were initially driven by CPUs and were just now beginning to depend increasingly on GPUs. That started the race to connect bigger clusters of GPUs, which has resulted in an insatiable appetite for more GPUs.

Alan stated that there has been significant increase in the HPC business due to the use of  AI and huge language models. However, GigaIO has recently witnessed growth in the augmentation space, where businesses are using  AI to improve human performance.

GigaIO Networks

In order to accomplish this, businesses require foundational models in the first place, but they also want to “retrain and fine-tune” such models by adding their own data to them.

Alan looks back on his company’s accomplishment of breaking the 8-GPU server restriction, which many were doubtful could be accomplished. He believes GigaIO’s partnership with AMD proved to be a crucial component.

Alan used the example of Dr. Moritz Lehmann’s testing SuperNODE on a computational fluid dynamic program meant to replicate airflow over the Concord’s wings at landing speed last year to highlight his points. Lehmann created his model in 32 hours without changing a single line of code after gaining access to SuperNODE. Alan calculated that the task would have taken more than a year if he had relied on eight GPUs and conventional technology.

“A great example of AMD GPUs and CPUs working together “Alan said. This kind of cooperation has involved several iterations. [Both firms have] performed admirably in their efforts to recognise and address technological issues at the engineering level.”

AMD Instinct MI210 accelerator

The Exascale-Class Technologies for the Data Centre: AMD INSTINCT MI210 ACCELERATOR

With the AMD Instinct MI210 accelerator, AMD continues to lead the industry in accelerated computing for double precision (FP64) on PCIe form factors for workloads related to mainstream HPC and artificial intelligence.

The 2nd Gen AMD CDNA architecture of the AMD Instinct MI210, which is based on AMD Exascale-class technology, empowers scientists and researchers to address today’s most critical issues, such as vaccine research and climate change. By utilising the AMD ROCm software ecosystem in conjunction with MI210 accelerators, innovators can leverage the capabilities of  AI and HPC data centre PCIe GPUs to expedite their scientific and discovery endeavours.

Specialised Accelerators for  AI & HPC Tasks

With up to a 2.3x advantage over Nvidia Ampere A100 GPUs in FP64 performance, the AMD Instinct MI210 accelerator, powered by the 2nd Gen AMD CDNA architecture, delivers HPC performance leadership over current competitive PCIe data centre GPUs today, delivering exceptional performance for a broad range of HPC & AI applications.

With an impressive 181 teraflops peak theoretical FP16 and BF16 performance, the MI210 accelerator is designed to speed up deep learning training. It offers an extended range of mixed-precision capabilities based on the AMD Matrix Core Technology and gives users a strong platform to drive the convergence of  AI and HPC.

New Ideas Bringing Performance Leadership

Through the unification of the CPU, GPU accelerator, and most significant processors in the data centre, AMD’s advances in architecture, packaging, and integration are pushing the boundaries of computing. Using AMD EPYC CPUs and AMD Instinct MI210 accelerators, AMD is delivering performance, efficiency, and overall system throughput for HPC and  AI thanks to its cutting-edge double-precision Matrix Core capabilities and the 3rd Gen AMD Infinity Architecture.

2nd Gen AMD CDNA Architecture

The computing engine chosen for the first U.S. Exascale supercomputer is now available to commercial HPC & AI customers with the AMD Instinct MI210 accelerator. The 2nd Generation AMD CDNA architecture powers the MI210 accelerator, which offers exceptional performance for  AI and HPC. With up to 22.6 TFLOPS peak FP64|FP32 performance, the MI210 PCIe GPU outperforms the Nvidia Ampere A100 GPU in double and single precision performance for HPC workloads.

This allows scientists and researchers worldwide to process HPC parallel codes more efficiently across several industries. For any mix of  AI and machine learning tasks you need to implement, AMD’s Matrix Core technology offers a wide range of mixed precision operations that let you work with huge models and improve memory-bound operation performance.

With its optimised BF16, INT4, INT8, FP16, FP32, and FP32 Matrix capabilities, the MI210 can handle all of your AI system requirements with supercharged compute performance. For deep learning training, the AMD Instinct MI210 accelerator provides 181 teraflops of peak FP16 and bfloat16 floating-point performance, while also handling massive amounts of data with efficiency.

AMD Fabric Link Technology

AMD Instinct MI210 GPUs, with their AMD Infinity Fabric technology and PCIe Gen4 support, offer superior I/O capabilities in conventional off-the-shelf servers. Without the need of PCIe switches, the MI210 GPU provides 64 GB/s of CPU to GPU bandwidth in addition to 300 GB/s of Peer-to-Peer (P2P) bandwidth performance over three Infinity Fabric links.

The AMD Infinity Architecture provides up to 1.2 TB/s of total theoretical GPU capacity within a server design and allows platform designs with two and quad direct-connect GPU hives with high-speed P2P connectivity. By providing a quick and easy onramp for CPU codes to accelerated platforms, Infinity Fabric contributes to realising the potential of accelerated computing.

Extremely Quick HBM2e Memory

Up to 64GB of high-bandwidth HBM2e memory with ECC support can be found in AMD Instinct MI210 accelerators, which operate at 1.6 GHz. and provide an exceptionally high memory bandwidth of 1.6 TB/s to accommodate your biggest data collections and do rid of any snags when transferring data in and out of memory. Workload can be optimised when you combine this performance with the MI210’s cutting-edge Infinity Fabric I/O capabilities.

AMD Instinct MI210 Price

AMD Instinct MI210 GPU prices vary by retailer and area. It costs around $16,500 in Japan.. In the United States, Dell offers it for about $8,864.28, and CDW lists it for $9,849.99. These prices reflect its high-end specifications, including 64GB of HBM2e memory and a PCIe interface, designed for HPC and  AI server applications.

AMD ROCm 6.1

With the AMD ROCm 6 open-source software platform, AMD hopes to maintain its commitment to open-source and device-independent solutions while creating an environment that maximises the performance and potential of AMD Instinct accelerators. Consider ROCm 6 as the link that will allow your most ambitious AI concepts to be implemented successfully. In the current market, it gives developers the opportunity to create at their own speed, testing and deploying applications across a wide range of GPU architectures, and it delivers outstanding interoperability with key industry frameworks.

The most recent platform upgrade from AMD, ROCm 6.1, adds a host of new features for both academics and developers. In order to stay up with the quick developments in AI frameworks, AMD will examine how ROCm 6.1 builds on the fundamental advantages of ROCm 6 by supporting the most recent AMD Instinct and Radeon GPUs, boosting optimisations across a wide range of computational domains, and extending ecosystem support. The goal of ROCm 6.1’s new features and updates is to enhance application performance and reliability so that AI and HPC developers can push the boundaries of what is feasible.

Presenting rocDecode, a video processing tool Thanks to the new ROCm library, AMD GPUs now have high-performance video decoding capabilities directly on the GPU thanks to the Video Core Next (VCN) specialised media engines. These hardware-based decoders are effective at handling video streams.

By enabling direct decoding of compressed video into visual memory, rocDecode reduces the amount of data transferred via the PCIe bus and gets rid of typical bottlenecks in video processing. With real-time applications like video scaling, colour conversion, and augmentation which are critical for advanced analytics, inferencing, and machine learning training this feature enables rapid post-processing with the ROCm HIP framework.

The efficiency and scalability of video decoding activities are maximised with rocDecode. The API fully utilises all of the VCNs on a GPU device by permitting the development of numerous decoder instances that can run concurrently. The capacity to process in parallel ensures that even large-volume video streams can be simultaneously decoded and processed. To put it succinctly, rocDecode strengthens the video processing pipeline, providing power efficiency and performance increases that are necessary for contemporary AI and HPC applications.

MIGraphX adds Flash Attention and PyTorch backend The AMD graph inference engine is called MIGraphX. MIGraphX is a command-line programme called migraphx-driver and is available through C++ and Python APIs. Its purpose is to speed up deep learning neural networks. Because of this flexibility, developers may incorporate sophisticated model inference features into their applications with ease.

With support for Flash Attention, which increases the memory efficiency of well-known models like BERT, GPT, and Stable Diffusion, ROCm 6.1 enhances performance for transformer-based models and contributes to the faster, more power-efficient processing of complicated neural networks.

A new Torch-MIGraphX library is also included in ROCm 6.1, allowing the PyTorch workflows to directly incorporate MIGraphX capabilities. It defines an immediate-use “migraphx” backend for the torch.compile API. A variety of data types, such as FP32, FP16, and INT8, are supported by the Torch-MIGraphX library to meet various computing requirements.

Better MIOpen Library performance AMD’s open-source MIOpen deep learning primitives library is made especially to improve GPU performance. It has a full suite of tools to maximise GPU launch overheads and memory bandwidth using cutting-edge methods like fusion and auto-tuning infrastructure. This infrastructure adapts algorithms to optimise convolutions for different filter and input sizes, and it handles a wide range of issue setups efficiently.

The goal of MIOpen’s most recent upgrades is to improve performance, especially for convolutions and inference. ROCm 6.1 features Find 2.0 fusion plans, which are intended to maximise system resource utilisation and enhance the library’s capacity to carry out inference jobs more effectively. The convolution kernels for the Number of samples, Height, Width, and Channels (NHWC) format have been enhanced by AMD. The new heuristics especially optimise efficiency for this format, allowing better handling and processing of convolution operations across multiple applications. NHWC prioritises the height and width dimensions, followed by channels.

New Composable Kernel Library Architecture Support The Composable Kernel (CK) library now has expanded architecture support thanks to ROCm 6.1, providing extremely effective capabilities on a larger variety of AMD GPUs. The addition of stochastic rounding to the FP8 rounding mechanism is a major update in this version. By simulating more realistic data behaviour, this rounding technique improves model convergence and provides a more accurate and dependable means of handling data in machine learning models.

Enlarged hipSparse Computations using SPARSELt To speed up deep learning tasks, ROCm 6.1 adds extensions to hipSPARSELt that allow structured sparsity matrices. Support for configurations in which ‘B’ denotes the sparse matrix and ‘A’ the dense matrix in Sparse Matrix-Matrix Multiplication (SPMM) is noteworthy in this release. The library’s capabilities were previously restricted to multiplications where the sparse matrix was represented by the letter “A” and the dense matrix by the letter “B.” This addition expands the library’s capabilities. The performance and versatility of SPMM operations are improved by support for various matrix configurations, which further optimises deep learning computations.

Higher-Level Tensor Functions using hipTensor The AMD-specific C++ library hipTensor uses the Composable Kernel Library’s primitives to speed up tensor operations. hipTensor was created by AMD to take advantage of general-purpose kernel languages like HIP C++. In cases where complicated tensor computations are needed, hipTensor optimises the way tensor primitives are executed.

HipTensor’s most recent version adds support for 4D tensor contraction and permutation. A critical operation in many tensor-based computations, permutations on 4D tensors can now be efficiently carried out by users with ROCm 6.1. 4D contractions for F16, BF16, and Complex F32/F64 data formats are now supported by the library. With this additional functionality, hipTensor can now optimise a wider range of operations, enabling more complicated and varied manipulations of tensor data many of which are necessary for sophisticated computing activities like training neural networks and running complex simulations.

AMD wants to provide you with the newest in high-performance computing through the ROCm platform. Every upgrade in ROCm 6.1 has been created to increase productivity, optimise processes, and assist you in reaching your objectives more quickly by offering useful, strong tools that unleash your creative potential.