#amdinstinctmi300x

AMD GPUs Powering AI-Driven Biology: Reprogramming Discovery

Contribution from IPA Therapeutics

Part 2: Protein Embeddings Improve Biology Analysis

The second chapter of the benchmarking series compares AMD Instinct MI300X and NVIDIA H100 GPUs for drug development AI activities.

Part 1: NLP-based biomedical research knowledge extraction. It now examines the protein layer to determine how well both GPUs handle large-scale protein language models (pLLMs), which help understand structure, function, and mutational impacts.

These benchmarks were conducted by ImmunoPrecise Antibodies (IPA) and its AI subsidiary BioStrand, makers of LENSai, an AI-native platform that uses sequencing, structure, and functional reasoning to advance biologics discovery. Every test employed Vultr's high-performance cloud architecture for reproducible, side-by-side comparisons in a production environment.

HYFT biological fingerprinting by LENSai combines conserved sequence, structure, and function into one index. HYFTs were created to solve AI's incapacity to understand biological processes. HYFTs integrate biological reasoning into computational fabric to let AI models reason biologically rather than only calculate.

AMD currently studies protein embeddings to understand binding interactions, mutation consequences, and molecular function. These embeddings underpin structural models and therapeutic target prioritisation. It compares different ESM-2 models and examines 'anchored embeddings' in LENSai and HYFT to show AMD GPUs' performance in a sector where memory capacity and biological accuracy are critical.

Using functional and evolutionary information, Protein Language Models (pLLMs) decode amino acid sequences into manageable vectors, improving biological data processing.

They let researchers ask: How similar is this sequence to druggable targets? Which structure will this unnamed protein adopt? How might a mutation affect binding or function? Embeddings improve immunogenicity screening, antibody identification, and multi-omics interpretation by reducing data for machine learning algorithms.

Benchmarks for ESM-2 Protein Language Model

AMD's throughput and scaling advantages were examined using ESM-2 benchmarks with various model sizes:

AMD GPUs handled larger batches smoothly, reducing costs and improving throughput.

Combined Drug Discovery With HYFT/LENSai

LENSai uses "HYFT anchored embeddings," which pick residues inside conserved HYFT patterns to reduce noise and improve biological signal clarity.

Biologically significant HYFTs are conserved structure or function motifs. LENSai embeds HYFTs to decrease noise and focus on the sequence's most functionally informative parts.

HYFT embedding support:

Estimating structural mutation consequences precisely.

Finding conserved motifs.

Effective semantic search across treatment libraries.

Easy Transition to AMD Graphics Processors

AMD GPU protein embeddings use a single-line Dockerfile update to minimise disruption:

FROM ROCM6.3.1_ubuntu22.04_py3.10_pytorch rocm/pytorch

No major code modifications are needed.

Protein embeddings connect unprocessed sequences to function. The experiment demonstrates that AMD MI300X has the memory headroom and performance for the most advanced protein models.

AMD compares RFdiffusion, a generative model that can imagine and produce new proteins, in the final chapter of this series to push AI-driven design.

Conclusion

NVIDIA H100 vs AMD Instinct MI300X

AMD Instinct GPUs Drive AI-Driven Biology's Future

Contribution from IPA Therapeutics

Part 1: Life Sciences NLP Embeds

This is the first of three articles comparing AMD Instinct MI300X GPUs versus NVIDIA H100 GPUs in drug discovery AI workloads. The benchmarks were performed by ImmunoPrecise Antibodies (IPA) and its AI subsidiary BioStrand, which collaborated with Vultr on the LENSai platform for AI-powered biologics discovery. Fast deployment and hardware design reproducibility were possible with Vultr's high-performance cloud infrastructure. It evaluated these GPUs for real-world medical development tasks like generative protein design and NLP-driven target discovery.

LENSai's HYFT technology biological fingerprinting method encodes conserved sequence, structure, and function into a single index. To overcome AI's inability to understand biological systems, HYFTs were created. HYFTs integrate biological reasoning into computational fabric to let AI models reason biologically rather than only calculate.

In three papers, AMD will test the MI300X GPUs in the LENSai tech stack: generative design using RFdiffusion, protein embedding generation for structure-function inference, and NLP-driven literature mining.

AMD used real-world NLP, protein embedding, and de novo protein design benchmarks to evaluate modern bioinformatics pipelines' raw performance, cost, and deployment practicality.

This first episode focusses on Natural Language Processing (NLP) and how Retrieval-Augmented Generation (RAG) and huge language models speed up early-stage therapy discovery by gleaning practical insights from scientific literature. Most significant lesson? AMD GPUs are cost-effective and fast, which is vital for life science organisations building AI-driven systems.

Natural Language Processing (NLP) efficiently mines enormous volumes of textual data, boosting therapeutic breakthroughs. NLP extracts hidden insights from genomic datasets, clinical reports, and scientific literature. Computational models, which emphasise safety, efficacy, and cost-effectiveness, are similar to NLP-driven large language models (LLMs), which ease drug research analysis and prediction.

Knowledge-aware models can shed light on RAG (Retrieval-Augmented Generation) vector embeddings using semantics rather than language. Since they support biological sequences and structures as well as text, these embeddings help NLP connect life science silos.

LENSai improves vector search by adding a powerful semantic layer that detects sub-sentence units and extracts subject-predicate-object triples to identify biological connections. By documenting molecular interactions between targets, pathways, and compounds, LENSai helps researchers map disease pathways, identify therapeutic targets, and better anticipate drug behaviour. This depth of understanding, often concealed in unstructured biological data, can be discovered and leveraged to hasten discovery and reduce risk and cost before wet lab research begin.

Infrastructure context

It used the AMD Instinct MI300X and NVIDIA H100 GPUs in a flexible, cloud-native environment to provide repeatable benchmarking and fair hardware generation comparisons.

Benchmark NLP Results

It retrieval-augmented generation (RAG) techniques use literature vector embeddings for contextual insights. AMD outperformed in cost and throughput:

MI300X also improved stability under high-concurrency workloads.

Technical Execution: Smooth AMD GPU Switch

Use ROCm PyTorch Docker images to migrate NLP jobs on AMD GPUs easily:

FROM ROCM6.3.1_ubuntu22.04_py3.10_pytorch rocm/pytorch

No Python code changes are needed. The PyTorch device abstraction (torch.device(“cuda”)) ensures compatibility.

AMD applauds Meta for their most recent Llama 3.2 release. Llama 3.2 is intended to increase developer productivity by assisting them in creating the experiences of the future and reducing development time, while placing a stronger emphasis on data protection and ethical AI innovation. The focus on flexibility and openness has resulted in a tenfold increase in Llama model downloads this year over last, positioning it as a top option for developers looking for effective, user-friendly AI solutions.

Llama 3.2 and AMD Instinct MI300X GPU Accelerators

The world of multimodal AI models is changing with AMD Instinct MI300X accelerators. One example is Llama 3.2, which has 11B and 90B parameter models. To analyze text and visual data, they need a tremendous amount of processing power and memory capa

AMD and Meta have a long-standing cooperative relationship. Its is still working to improve AI performance for Meta models on all of AMD platforms, including Llama 3.2. AMD partnership with Meta allows Llama 3.2 developers to create novel, highly performant, and power-efficient agentic apps and tailored AI experiences on AI PCs and from the cloud to the edge.

AMD Instinct accelerators offer unrivaled memory capability, as demonstrated by the launch of Llama 3.1 in previous demonstrations. This allows a single server with 8 MI300X GPUs to fit the largest open-source project currently available with 405B parameters in FP16 datatype something that no other 8x GPU platform can accomplish. AMD Instinct MI300X GPUs are now capable of supporting both the latest and next iterations of these multimodal models with exceptional memory economy with the release of Llama 3.2.

By lowering the complexity of distributing memory across multiple devices, this industry-leading memory capacity makes infrastructure management easier. It also allows for quick training, real-time inference, and the smooth handling of large datasets across modalities, such as text and images, without compromising performance or adding network overhead from distributing across multiple servers.

With the powerful memory capabilities of the AMD Instinct MI300X platform, this may result in considerable cost savings, improved performance efficiency, and simpler operations for enterprises.

Throughout crucial phases of the development of Llama 3.2, Meta has also made use of AMD ROCm software and AMD Instinct MI300X accelerators, enhancing their long-standing partnership with AMD and their dedication to an open software approach to AI. AMD’s scalable infrastructure offers open-model flexibility and performance to match closed models, allowing developers to create powerful visual reasoning and understanding applications.

Developers now have Day-0 support for the newest frontier models from Meta on the most recent generation of AMD Instinct MI300X GPUs, with the release of the Llama 3.2 generation of models. This gives developers access to a wider selection of GPU hardware and an open software stack ROCm for future application development.

CPUs from AMD EPYC and Llama 3.2

Nowadays, a lot of AI tasks are executed on CPUs, either alone or in conjunction with GPUs. AMD EPYC processors provide the power and economy needed to power the cutting-edge models created by Meta, such as the recently released Llama 3.2. The rise of SLMs (small language models) is noteworthy, even if the majority of recent attention has been on LLM (long language model) breakthroughs with massive data sets.

These smaller models need far less processing resources, assist reduce risks related to the security and privacy of sensitive data, and may be customized and tailored to particular company datasets. These models are appropriate and well-sized for a variety of corporate and sector-specific applications since they are made to be nimble, efficient, and performant.

The Llama 3.2 version includes new capabilities that are representative of many mass market corporate deployment situations, particularly for clients investigating CPU-based AI solutions. These features include multimodal models and smaller model alternatives.

When consolidating their data center infrastructure, businesses can use the Llama 3.2 models’ leading AMD EPYC processors to achieve compelling performance and efficiency. These processors can also be used to support GPU- or CPU-based deployments for larger AI models, as needed, by utilizing AMD EPYC CPUs and AMD Instinct GPUs.

AMD AI PCs with Radeon and Ryzen powered by Llama 3.2

AMD and Meta have collaborated extensively to optimize the most recent versions of Llama 3.2 for AMD Ryzen AI PCs and AMD Radeon graphics cards, for customers who choose to use it locally on their own PCs. Llama 3.2 may also be run locally on devices accelerated by DirectML AI frameworks built for AMD on AMD AI PCs with AMD GPUs that support DirectML. Through AMD partner LM Studio, Windows users will soon be able to enjoy multimodal Llama 3.2 in an approachable package.

Up to 192 AI accelerators are included in the newest AMD Radeon, graphics cards, the AMD Radeon PRO W7900 Series with up to 48GB and the AMD Radeon RX 7900 Series with up to 24GB. These accelerators can run state-of-the-art models such Llama 3.2-11B Vision. Utilizing the same AMD ROCm 6.2 optimized architecture from the joint venture between AMD and Meta, customers may test the newest models on PCs that have these cards installed right now3.

AMD and Meta: Progress via Partnership

To sum up, AMD is working with Meta to advance generative AI research and make sure developers have everything they need to handle every new release smoothly, including Day-0 support for entire AI portfolio. Llama 3.2’s integration with AMD Ryzen AI, AMD Radeon GPUs, AMD EPYC CPUs, AMD Instinct MI300X GPUs, and AMD ROCm software offers customers a wide range of solution options to power their innovations across cloud, edge, and AI PCs.

Engineering Insights: Introducing AMD Instinct MI300X Accelerators’ MLPerf Results. The full-stack AMD inference platform demonstrated its prowess with the remarkable results AMD Instinct MI300X GPUs, powered by one of the most recent iterations of open-source ROCm, obtained in the MLPerf Inference v4.1 round.

LLaMA2-70B

The first submission concentrated on the well-known LLaMA2-70B type, which is renowned for its excellent performance and adaptability. By outperforming the NVIDIA H100 in Gen  AI inference, it established a high standard for what AMD Instinct MI300X accelerators are capable of.

MLPerf Inference

Comprehending MLPerf and Its Relevance to the Industry

Efficient and economical performance is becoming more and more important for inference and training as large language models (LLMs) continue to grow in size and complexity. Robust parallel processing and an optimal software stack are necessary to achieve high-performance LLMs.

This is where the best benchmarking package in the business, MLPerf, comes into play. The open-source AI benchmarks known as MLPerf Inference, which were created by the cross-industry cooperation MLCommons, of which AMD is a founding member, include Gen AI, LLMs, and other models that give exacting, peer-reviewed criteria. Businesses are able to assess the efficacy of AI technology and software by using these benchmarks.

A major accomplishment for AMD, excelling in MLPerf Inference v4.1 demonstrates their dedication to openness and providing standardized data that enables businesses to make wise choices.

An Extensive Analysis of the LLaMA2-70B Benchmark

The AMD LLaMA2-70B model was utilized in their first MLPerf Inference. A major development in LLMs, the LLaMA2-70B model is essential for practical uses such as large-scale inference and natural language processing. A Q&A scenario using 24,576 samples from the OpenORCA dataset, each with up to 1,024 input and output tokens, was included in the MLPerf benchmarking test. Two situations were analyzed by the benchmark to assess inference performance:

In an offline scenario, queries are processed in batches to increase throughput in tokens per second.

 Server Scenario: This model tests the hardware’s capacity to provide quick, responsive performance for low-latency workloads by simulating real-time queries with stringent latency limitations (TTFT* < 2s, TPOT* ≤ 200ms).

Performance of AMD Instinct MI300X in MLPerf

With four important entries for the LLaMA2-70B model, the AMD Instinct MI300X demonstrated remarkable performance in its first MLPerf Inference utilizing the Supermicro AS-8125GS-TNMR2 machine. These findings are especially noteworthy since they provide an apples-to-apples comparison with rival AI accelerators, are repeatable, vetted by peer review, and grounded in use cases that are relevant to the industry.

Combination Performance of CPU and GPU

Submission ID 4.1-0002: Two AMD EPYC 9374F (Genoa) CPUs paired with eight AMD Instinct MI300X accelerators in the Available category.

This setup demonstrated the potent synergy between 4th Gen EPYC CPUs (previously codenamed “Genoa”) and AMD Instinct MI300X GPU accelerators for AI workloads, providing performance within 2-3% of NVIDIA DGX H100 with 4th Gen Intel Xeon CPUs in both server and offline environments at FP8 precision.

Previewing Next-Generation CPU Performance

Submission ID 4.1-0070: Two AMD EPYC “Turin” CPUs and eight AMD Instinct MI300X CPUs in the Preview category.

It showcased the performance increases from the next AMD EPYC “Turin” 5th generation CPU when paired with AMD Instinct MI300X GPU accelerators. In the server scenario, it outperformed the NVIDIA DGX H100 with Intel Xeon by a small margin, and it maintained a similar level of performance even offline at FP8 precision.

LLaMA2-70B GPU

Efficiency of a Single GPU

Submission ID 4.1-0001: In the Available category, AMD Instinct MI300X accelerator with AMD EPYC 9374F 4th Gen CPUs (Genoa).

This submission emphasized the AMD Instinct MI300X’s enormous 192 GB memory, which allowed a single GPU to effectively execute the whole LLaMA2-70B model without requiring the network cost that comes with dividing the model over many GPUs at FP8 precision.

The AMD Instinct MI300X has 192 GB of HBM3 memory and a peak memory bandwidth of 5.3 TB/s thanks to its AMD CDNA 3 architecture. The AMD Instinct MI300X can execute and host a whole 70 billion parameter model, such as LLaMA2-70B, on a single GPU with ease because to its large capacity.

The findings in Figure 2 show that the scaling efficiency with the ROCm software stack is almost linear from 1x AMD Instinct MI300X (TP1) to 8x AMD Instinct MI300X (8x TP1), indicating that AMD Instinct MI300X can handle the biggest MLPerf inference model to date.

Outstanding Dell Server Architecture Outcomes Using AMD Instinct MI300X Processors

Submission ID 4.1-0022: Two Intel Xeon Platinum 8460Y+ processors and eight AMD Instinct MI300X accelerators in the Available category.

Along with AMD submissions, Dell used their PowerEdge XE9680 server and LLaMA2-70B to submit their findings, validating the platform-level performance of AMD Instinct accelerators on an 8x AMD Instinct MI300X arrangement. This proposal demonstrates their collaboration and emphasizes how strong it ecosystem is, making them a great option for deployments including both data centers and edge inference. Further information on such outcomes is available here.

Performance Of Engineering Insights

The AMD Instinct MI300X accelerators exhibit great competitive performance due to their high computational power, huge memory capacity with rapid bandwidth, and optimized ROCm software stack. The latter enables effective processing of large  AI models such as LLaMA2-70B. A few important elements were pivotal:

Big GPU Memory Capacity

The AMD Instinct MI300X has the most GPU memory that is currently on the market, which enables the whole LLaMA2-70B model to fit into memory while still supporting KV cache. By avoiding model splitting among GPUs, this maximizes inference speed while avoiding network cost.

Batch Sizes: They set the max_num_seqs parameter to 2048 in the offline scenario to optimize throughput, and to 768 in the server scenario to achieve latency requirements. These values are much greater than the 256 default value used in vLLM.

Effective KV cache management is made possible by the vLLM’s paged attention support, which helps prevent memory fragmentation brought on by huge memory AMD Instinct MI300X accelerators.

FP8 Precision

AMD expanded support for the FP8 numerical format throughout the whole inference software stack, using the AMD Instinct MI300X accelerator hardware. They quantized the LLaMA2-70B model weights to FP8 using Quark while maintaining the 99.9% accuracy needed by MLPerf. To further improve speed, it improved the hipBLASLt library, introduced FP8 support to vLLM, and implemented FP8 KV caching.

Software Enhancements

Kernel Optimization: AMD Composable Kernels (CK) based prefill attention, FP8 decode paged attention, and fused kernels such residual add RMS Norm, SwiGLU with FP8 output scaling were among the many profiles and optimizations to carried out.

vLLM Enhancements: The scheduler was improved to optimize both offline and server use cases, allowing for quicker decoding scheduling and better prefill batching.

CPU Enhancement

While GPUs handle the majority of the AI task processing, CPU speed is still quite important. CPUs with fewer cores and higher peak frequencies such as the 32-core EPYC 9374F offer the best performance, particularly in server applications. Performance improvements over the 4th generation EPYC CPUs which were submitted as a preview were seen during testing with the forthcoming “Turin” generation of EPYC CPUs.

LLaMa 3.1 405B

Establishing a Standard for the Biggest Model

The AMD Instinct MI300X GPU accelerators have shown their performance in MLPerf Inference with LLaMA2-70B, and the positive outcomes set a solid precedent for their future efficacy with even bigger models, such as Llama 3.1. They are pleased to provide Day 0 support for AMD Instinct MI300X accelerators with Meta’s new LLaMa 3.1 405B parameter model.

Only a server driven by eight AMD Instinct MI300X GPU accelerators can fit the whole LLaMa 3.1 model, with 405 billion parameters, on a single server utilizing FP16 datatype MI300-7A, owing to the industry-leading memory capacities of the AMD Instinct MI300X platform MI300-25. This lowers expenses and lowers server use. The most ideal way to power the biggest open models on the market right now is with AMD Instinct MI300X accelerators.