Opening Up New Paths in AI and HPC with AMD’s Release ROCm 6.3. With the introduction of cutting-edge tools and optimizations to improve AI, ML, and HPC workloads on AMD Instinct GPU accelerators, ROCm 6.3 represents a major milestone for the AMD open-source platform. By increasing developer productivity, ROCm 6.3 is designed to enable a diverse spectrum of clients, from cutting-edge AI startups to HPC-driven businesses.

This blog explores the release’s key features, which include a redesigned FlashAttention-2 for better AI training and inference, the introduction of multi-node Fast Fourier Transform (FFT) to transform HPC workflows, a smooth integration of SGLang for faster AI inferencing, and more. Discover these fascinating developments and more as ROCm 6.3 propels industry innovation.

Super-Fast Inferencing of Generative AI (GenAI) Models with SGLang in ROCm 6.3

Industries are being revolutionized by GenAI, yet implementing huge models frequently involves overcoming latency, throughput, and resource usage issues. Presenting SGLang, a new runtime optimized for inferring state-of-the-art generative models like LLMs and VLMs on AMD Instinct GPUs and supported by ROCm 6.3.

Why It Is Important to You

6X Higher Throughput: According to research, you can outperform current systems on LLM inferencing by up to 6X, allowing your company to support AI applications on a large scale.

Usability: With Python integrated and pre-configured in the ROCm Docker containers, developers can quickly construct scalable cloud backends, multimodal processes, and interactive AI helpers with less setup time.

SGLang provides the performance and usability required to satisfy corporate objectives, whether you’re developing AI products that interact with customers or expanding AI workloads in the cloud.

Next-Level Transformer Optimization: Re-Engineered FlashAttention-2 on AMD Instinct

The foundation of contemporary AI is transformer models, although scalability has always been constrained by their large memory and processing requirements. AMD resolves these issues with FlashAttention-2 designed for ROCm 6.3, allowing for quicker, more effective training and inference.

Why It Will Be Favorite by Developers

3X Speedups: In comparison to FlashAttention-1, achieve up to 3X speedups on backward passes and a highly efficient forward pass. This will speed up model training and inference, lowering the time-to-market for corporate AI applications.

Extended Sequence Lengths: AMD Instinct GPUs handle longer sequences with ease with to their effective memory use and low I/O overhead.

With ROCm’s PyTorch container and Composable Kernel (CK) as the backend, you can easily add FlashAttention-2 on AMD Instinct GPU accelerators into your current workflows and optimize your AI pipelines.

AMD Fortran Compiler: Bridging Legacy Code to GPU Acceleration

With the release of the new AMD Fortran compiler in ROCm 6.3, businesses using AMD Instinct accelerators to run historical Fortran-based HPC applications may now fully utilize the potential of contemporary GPU acceleration.

Principal Advantages

Direct GPU Offloading: Use OpenMP offloading to take advantage of AMD Instinct GPUs and speed up important scientific applications.

Backward Compatibility: Utilize AMD’s next-generation GPU capabilities while building upon pre-existing Fortran code.

Streamlined Integrations: Connect to ROCm Libraries and HIP Kernels with ease, removing the need for intricate code rewrites.

Businesses in sectors like weather modeling, pharmaceuticals, and aerospace may now leverage the potential of GPU acceleration without requiring the kind of substantial code overhauls that were previously necessary to future-proof their older HPC systems. This comprehensive tutorial will help you get started with the AMD Fortran Compiler on AMD Instinct GPUs.

New Multi-Node FFT in rocFFT: Game changer for HPC Workflows

Distributed computing systems that scale well are necessary for industries that depend on HPC workloads, such as oil and gas and climate modeling. High-performance distributed FFT calculations are made possible by ROCm 6.3, which adds multi-node FFT functionality to rocFFT.

The Significance of It for HPC

The integration of the built-in Message Passing Interface (MPI) streamlines multi-node scalability, lowering developer complexity and hastening the deployment of distributed applications.

Scalability of Leadership: Optimize performance for crucial activities like climate modeling and seismic imaging by scaling fluidly over large datasets.

Larger datasets may now be processed more efficiently by organizations in sectors like scientific research and oil and gas, resulting in quicker and more accurate decision-making.

Enhanced Computer Vision Libraries: AV1, rocJPEG, and Beyond 

AI developers need effective preprocessing and augmentation tools when dealing with contemporary media and datasets. With improvements to its computer vision libraries, rocDecode, rocJPEG, and rocAL, ROCm 6.3 enables businesses to take on a variety of tasks, from dataset augmentation to video analytics.

Why It Is Important to You

Support for the AV1 Codec: rocDecode and rocPyDecode provide affordable, royalty-free decoding for contemporary media processing.

GPU-Accelerated JPEG Decoding: Use the rocJPEG library’s built-in fallback methods to perform image preparation at scale with ease.

Better Audio Augmentation: Using the rocAL package, preprocessing has been enhanced for reliable model training in noisy situations.

From entertainment and media to self-governing systems, these characteristics allow engineers to produce more complex AI solutions for practical uses.

It’s important to note that, in addition to these noteworthy improvements, Omnitrace and Omniperf which were first released in ROCm 6.2 have been renamed as ROCm System Profiler and ROCm Compute Profiler. Improved usability, reliability, and smooth integration into the existing ROCm profiling environment are all benefits of this rebranding.

Why ROCm 6.3?

AMD With each release, ROCm has advanced, and version 6.3 is no different. It offers state-of-the-art tools to streamline development and improve speed and scalability for workloads including AI and HPC. ROCm enables companies to innovate more quickly, grow more intelligently, and maintain an advantage in cutthroat markets by adopting the open-source philosophy and constantly changing to satisfy developer demands.

Are You Prepared to Jump? Examine ROCm 6.3‘s full potential and discover how AMD Instinct accelerators may support the next significant innovation in your company.

AMD ROCm 6.2.3

AMD recently published AMD ROCm 6.2.3, the most recent version of their open compute software that supports Radeon GPUs on native Ubuntu Linux systems. Most significantly, this latest edition enables developers to use Stable Diffusion (SD) 2.1 text-to-image capabilities in their AI programming and offers amazing inference performance with Llama 3 70BQ4.

AMD focused on particular features to speed up the development of generative AI after its last release with AMD ROCm 6.1. Using vLLM and Flash Attention 2, AMD ROCm 6.2 provides pro-level performance for Large Language Model inference. This release also includes beta support for the Triton framework, enabling more users to develop AI functionality on AMD hardware.

The following are AMD ROCm 6.2.3 for Radeon GPUs’ four main feature highlights:

The most recent version of Llama is officially supported by vLLM. AMD ROCm on Radeon with Llama 3 70BQ4 offers amazing inference performance.

Flash Attention 2 “Forward Enablement” is officially supported. Its purpose is to speed up inference performance and lower memory requirements.

Formally endorsing stable diffusion (SD) The SD text-to-image model can be integrated into your own AI development.

Triton Beta Support: Use the Triton framework to quickly and simply develop high-performance AI programs with little experience

Since its first 5.7 release barely a year ago, AMD ROCm support for Radeon GPUs has advanced significantly.

With version 6.0, it formally qualified the usage of additional Radeon GPUs, such as the Radeon PRO W7800 with 32GB, and greatly increased AMD ROCm’s capabilities by adding support for the widely used ONNX runtime.

Another significant milestone was reached with the release of AMD ROCm 6.1, where it declared official support for the TensorFlow framework and multi-GPU systems. It also granted beta access to Windows Subsystem for Linux (WSL 2), which is now officially eligible for use with 6.1.

The AMD ROCm 6.2.3 solution stack for Radeon GPUs is as follows:

Although Linux was the primary focus of AMD ROCm 6.2.3, WSL 2 support will be released shortly.

ROCm on Radeon for AI and Machine Learning development has had a fantastic year, and it is eager to keep collaborating closely with the community to improve its product stack and support its system builders in developing attractive on-premises, client-based solutions.

Evolution of AMD ROCm from version 5.7 to 6.2.3

From version 5.7 to 6.2.3, AMD ROCm (Radeon Open Compute) has made substantial improvements to performance, hardware support, developer tools, and deep learning frameworks. Each release’s main improvements are listed below:

AMD ROCm 5.7

Support for New Architectures: ROCm 5.7 included support for AMD’s RDNA 3 family. This release expanded the GPUs that can utilize ROCm for deep learning and HPC.

HIP Improvements: AMD’s HIP framework for running CUDA code on AMD GPUs was optimized to facilitate interoperability between ROCm-supported systems and CUDA-based workflows.

Deep Learning Framework Updates: TensorFlow and PyTorch were made more compatible and performant. These upgrades optimized AI workloads in multi-GPU setups.

Performance Optimizations: This version improved HPC task performance, including memory management and multi-GPU scaling.

AMD ROCm 6.0

Unified Memory Support: ROCm 6.0 fully supported unified memory, making CPU-GPU data transfers smoother. This feature improved memory management, especially for applications that often access these processors’ memory.

New Compiler Infrastructure: AMD enhanced the ROCm Compiler (LLVM-based) for greater performance and larger workload support. We wanted to boost deep learning, HPC, and AI efficiency.

ROCm 6.0 might target more GPUs, especially in HPC, due to improved scalability and RDNA and CDNA architecture compatibility.

New CUDA compatibility features were added to the HIP API in this edition. These changes let developers convert CUDA apps to ROCm.

AMD ROCm 6.1

Optimized AI/ML Framework Compatibility: ROCm 6.1 improved PyTorch and TensorFlow performance. This improved mixed precision training, which maximizes GPU utilization in deep learning.

Experimental HIP Tensor Cores support allowed AI models to use hardware-accelerated matrix operations. This improvement greatly accelerated matrix multiplication, which is essential for deep learning.

Expanded Container Support: AMD included pre-built Docker containers that were easier to connect with Kubernetes in ROCm 6.1, simplifying cloud and cluster deployment.

More efficient data transfer in multi-GPU systems was achieved by improving memory and I/O operations.

AMD ROCm 6.1.3

Support for several GPUs makes it possible to create scalable AI desktops for multi-user, multi-serving applications.

These solutions can be used with ROCm on a Windows OS-based system thanks to beta-level support for Windows Subsystem for Linux.

More options for AI development are provided via the TensorFlow Framework.

AMD ROCm 6.2

New Kernel and Driver Features: ROCm 6.2 improved low-level driver and kernel support for advanced computing workload stability and performance. This significantly strengthened ROCm in enterprise environments.

AMD Infinity Architecture Integration: ROCm 6.2 enhances AMD’s Infinity Architecture, which connects GPUs at fast speeds. Multi-GPU configurations performed better, especially for large-scale HPC and AI applications.

HIP API expansion: ROCm 6.2 improved the HIP API, making CUDA-based application conversion easier. Asynchronous data transmission and other advanced features were implemented in this release to boost computational performance.

AMD ROCm 6.2.3

The most recent version of Llama is officially supported by vLLM. AMD ROCm on Radeon with Llama 3 70BQ4 offers amazing inference performance.

Flash Attention 2 “Forward Enablement” is officially supported. Its purpose is to speed up inference performance and lower memory requirements.

Formally endorsing stable diffusion (SD) The SD text-to-image model can be integrated into your own AI development.

Triton Beta Support: Use the Triton framework to quickly and simply develop high-performance AI programs with little experience

Key Milestone Summary

These versions supported new AMD architectures (RDNA, CDNA) as ROCm expanded its hardware support.

CUDA application porting became easier with HIP updates, while data scientists and academics found AI/ML framework support more useful.

Multi-GPU Optimizations: Unified memory support, RDMA, and AMD Infinity Architecture improved multi-GPU deployments, which are essential for HPC and large-scale AI training.

Each release improved ROCm’s stability and scalability by fixing bugs, optimizing memory management, and improving speed.

AMD now prioritizes an open, high-performance computing platform for AI, machine learning, and HPC applications.

AMD Instinct Accelerators

AMD Instinct MI300X Accelerators with ROCm Software: Boost Your LLMs

Although large language models (LLMs) appear to be widely available and unrestricted, fierce rivalry exists behind the scenes for the AMD Instinct GPUs resources required to run them. For those wishing to develop and implement LLMs and their visual counterparts, there are substantial obstacles due to cost, availability, and performance limits.

These models place heavy demands on memory and processing power due to their reliance on processing billions of parameters at once. Their impressive powers are made possible by their huge scale, but this also makes it difficult to deploy them economically. TCO issues can also arise from AI inferencing, which uses trained models to produce and provide predictions or outputs. But the AMD Instinct MI300X accelerator aids in removing these obstacles and maximising LLM potential.

AMD MI300X accelerator Vs Nvidia H200

Huge memory bandwidth and the ability to accommodate larger models

Large datasets and computations required by LLMs require high bandwidth, which is essential for enabling quicker processing, lower latency, and improved overall performance. With a peak memory bandwidth of up to 5.3 TB/s, the AMD MI300X accelerator outperforms the Nvidia H200 by a wide margin.

The MI300X does not require splitting models of this magnitude across many GPUs because it can support models with up to 80 billion parameters on a single GPU because to its 192 GB of HBM3 memory. On the other hand, the Nvidia H200, which has 141 GB of HBM2e memory, could need to split models, which would complicate matters and reduce data transfer efficiency.

More of the model can be stored closer to the computing units thanks to the AMD Instinct GPUs huge memory capacity, which lowers latency and boosts performance. Furthermore, the MI300X’s enormous memory capacity allows it to manage numerous large models on a single GPU, which solves the problem of dividing these models between AMD Instinct GPUs and the related execution complexity that accompany this operation.

The MI300X is a great option for handling the rigorous requirements of LLMs since it minimises potential inefficiencies in data transfer, which simplifies implementation and improves performance.

Due of its massive memory capacity and high bandwidth, the MI300X GPU can complete tasks on a single GPU that the H200 would require numerous AMD Instinct GPUs to complete. This can reduce expenses and ease deployment. These features can increase performance while simplifying the management of many GPUs. It might require less GPUs to run a model like ChatGPT on the MI300X than it would on the H200. making it a fantastic choice for businesses looking to implement cutting-edge  AI models.

Using Flash Attention to Improve LLM inference

Flash Attention, a significant advancement in optimising LLM inference on GPUs, is supported by AMD Instinct GPUs such the MI300X. Conventional attention techniques cause bottlenecks because they require numerous reads and writes to high-bandwidth memory (HBM). To combat this, Flash Attention reduces data transmission and boosts speed by consolidating processes like activation and dropout into a single step. LLMs especially benefit from this optimisation since it enables quicker and more effective processing.

AMD Instinct MI300X

Performance of floating point operations

One key indicator of LLM performance is floating point operation performance. Up to 1.3 PFLOPS of half-precision floating point (FP16) and 163.4 TFLOPS of single-precision floating point (FP32) performance are provided by the MI300X. These performance thresholds contribute to the precise and efficient operation of the intricate calculations involved in LLMs. Deep-learning models rely on complex numerical computations for tasks like matrix multiplications and tensor operations, for which this performance is also important.

The MI300X can handle several operations at once because of its advanced parallelism-supporting architecture. The MI300X can easily manage the large amount of parameters in LLMs thanks to its 304 compute units, which allows it to carry out complicated tasks.

AMD ROCm

An ideal open software stack for developing and transferring LLMs

For  AI and HPC applications, the AMD ROCm software platform offers a solid and open basis. ROCm makes AI-specific libraries, tools, and frameworks available so that  AI developers may easily take advantage of the MI300X GPU’s capabilities. Code created on CUDA can be easily ported to ROCm with little modifications from developers, ensuring efficiency and compatibility.

Leading AI frameworks like PyTorch and TensorFlow are supported by upstream ROCm software, making millions of Hugging Face and other LLMs functional right out of the box. Additionally, it makes it easier to integrate libraries like Hugging Face and frameworks like PyTorch with AMD GPUs, making the integration of LLMs on the MI300X simple. With AMD Instinct GPUs, this integration guarantees developers to optimise application performance and give optimal performance for LLM inference.

AMD ROCm GPUs

Making a tangible difference

To improve LLM inference models and address real-world issues, AMD works in an open ecosystem with industry partners like Microsoft, Hugging Face, and the OpenAI Triton team. AMD Instinct GPUs, such as the MI300X, are used by the Microsoft Azure cloud platform to improve enterprise AI services. Another noteworthy MI300X implementation by Microsoft and OpenAI is ChatGPT-4, which demonstrates how well AMD GPUs can manage demanding  AI workloads.

Hugging Face collaborates with the OpenAI Triton team to integrate cutting-edge tools and frameworks, while utilising AMD technology to optimise models and accelerate inference times.

In conclusion, because the AMD Instinct MI300X accelerator can handle issues with availability, speed, and cost, it’s a great option for implementing big language models. By offering a dependable, effective substitute and a robust ROCm ecosystem, AMD supports companies in maintaining stable  AI operations and achieving peak performance.

ROCm: What is it?

ROCm is an open-source stack for graphics processing unit (GPU) compute that is mainly made up of open-source software. GPU programming is made possible by ROCm, which is a set of drivers, development tools, and APIs that range from low-level kernel to end-user programs.

Heterogeneous-computing Interface for Portability (HIP) powers ROCm, which comes with all the required libraries, debuggers, and compilers for open source applications. It also supports programming models like OpenMP and OpenCL. It is completely integrated with PyTorch and TensorFlow, two machine learning (ML) frameworks.

Open-source software Development: An Exclusive Look inside the New AMD ROCm Software Blog Platform

Following on top of some of the most recent advances in the arenas of high-performance computing (HPC) and accelerated computing might be problematic since both these forms of technology undergo continual change. There exists an updated tool that has been developed to help bridge the gap for builders who are collaborating alongside AMD ROCm, which is an open-source software platform for AMD Radeon GPUs. This resource is the AMD ROCm Software Book Platform. This platform is not simply another blog, rather, it is a carefully selected venue within the ROCm community that is meant to encourage open-source collaboration and the exchange of information.

Is AMD ROCm open source?

Developers are able to obtain insights directly from AMD engineers and researchers, delving deeply into technical details, optimizations, and best practices for ROCm development. This is considered to be “direct from the source.” Through this openness, trust is fostered, and developers are given the ability to successfully participate. By simplifying complex ideas and transforming them into material that is easy to understand, the platform helps to demystify the complex. Regardless of whether you are an experienced professional or are just beginning your adventure with ROCm, you will discover knowledge that is very relevant to your level. Above and beyond the code: The blog does not shy away from covering more general subjects such as ecosystem relationships, forthcoming features, and the overall vision for ROCm. This comprehensive approach ensures that developers are kept up to date and involved in the development of the interface. ROCm is, at its heart, an open-source project, and the blog platform represents this principle. At the same time, ROCm encourages collaboration. In order to stimulate cooperation and ensure that the platform continues to be current and active, contributions from the community are welcomed. Accessible and Searchable: The platform has an intuitive user interface that is easy to use and has robust search tools made available to users. Because of this, it is simple to locate the particular information that you want, regardless of whether you are seeking for a particular feature or a more comprehensive grasp of ROCm’s characteristics.

A few of the unique ROCm advantages that the platform provides are as follows:

Obtain early insights on new ROCm features and improvements to ensure that you remain ahead of the curve. Insights from the knowledgeable: Learn about AMD engineers’ best practices and access their technical deep dives. Look for answers to the problems you’re face: Obtain the answers to the queries you have about ROCm development. Get in touch with the neighborhood: Discuss and work together with other ROCm developers. Participate in conversations. Be a part in shaping the future: Your contribution to the direction of ROCm would be greatly appreciated.

Anyone who is participating in the development of ROCm or who is investigating its possibilities will find the new AMD ROCm Software Blog Platform to be an extremely helpful resource. Without the company’s commitment to free software cooperation as well as educational material, it strives to speed up discovery and provide players the ability to challenge the limits of just what is possible with the AMD Radeon graphics processing systems (GLIs).

What is AMD ROCm Software?

The blog Interface for the New AMD ROCm Software The importance of becoming current know the most modern developments cannot be overestimated in the continually evolving scientific world, particularly with regard to the field of technologically advanced computing and GPU-accelerated processing.

Whether chip has to do with leveraging the promise of GPUs power computation-intensive AI and HPC software, AMD seems to be constantly challenging beyond boundaries by developing the AMD ROCm Applications stack. Furthermore, AMD has been catering to the different demands of developers and academics.

A new web page that is devoted to hosting blogs that are focused on the most recent ROCm features has been developed by AMD in order to further increase community involvement and information exchange. The current website acts as an administrative center for communicating the most current data on the capabilities of the AMD ROCm platform and the various manners in which these capabilities may be deployed to develop leading-edge intelligent machines and outstanding performance. computing systems.

AMD Blog Platform Revealed towards the Public It also highlights the most recent and compelling story for the AMD ROCm Software and what AMD has accomplished in terms of accelerated computing advancements.

What does AMD ROCm do?

At this time, they are highlighting the following facts:

Mathematical algorithms, including linear algebra, discretizations of partial differential equations, and solvers, are implemented in this context.

Applications of artificial intelligence and high-performance computing, as well as machine learning models, are targeted for optimization.

To make the most of the AMD ROCm tools, the AMD ROCm software stack, and hardware-level improvements, here are some points to consider.

Partner relationships and the ecosystem If you are interested in investigating performance enhancements, gaining a comprehension of programming models, or digging into particular use cases, the AMD ROCm blog platform is designed to accommodate audiences with varying degrees of expertise. In spite of the quick rate at which technology is advancing, the AMD ROCm Blog Platform is unwavering in its dedication to maintaining its position at the front of progress. Those who are enthusiastic in exploring the immense possibilities of GPU computing with AMD ROCm Software will find this site to be the destination of choice because it has frequent updates, smart conversations, and a devoted community that drives participation.