#nvidiah100gpu

EvolutionaryScale

A startup with support from NVIDIA and others presents the NVIDIA H100-enabled model for studying new proteins. Prompt-based code creation by generative AI has transformed software development; protein design is the next.

The third-generation ESM model, called ESM3, was released today by EvolutionaryScale. It provides protein discovery engineers with a programmable platform by reasoning over the sequence, structure, and functions of proteins simultaneously.

The business, which sprang out of the Meta Fair (Fundamental AI Research) team, has received money from NVIDIA and Amazon in addition to Lux Capital, Nat Friedman, and Daniel Gross.

EvolutionaryScale, at the vanguard of programmable biology, can help scientists build proteins that target cancer cells, discover safer plastic substitutes, promote environmental mitigation, and more.

NVIDIA H100 Tensor GPU

With the development of the scale-out model of ESM3, EvolutionaryScale is leading the way in programmable biology. This model leverages NVIDIA H100 Tensor Core GPUs to provide the highest computational capacity ever included in a biological foundation model. Compared to the ESM2 model, which included 98 billion parameters, the ESM3 model requires about 25 times more flops and 60 times more data.

The company offers technology that can give drug discovery researchers hints about how diseases can be cured, drugs developed, and, as its name suggests, how humans have evolved at scale as a species. The company created a database of over 2 billion protein sequences to train its AI model.

Using ESM3 to Quicken In Silico Biological Research

EvolutionaryScale intends to accelerate protein discovery with ESM3 by leveraging large improvements in training data.

With the help of around 2.8 billion protein sequences taken from various organisms and biomes, scientists were able to train the model to recognize and certify novel proteins with ever-increasing accuracy.

ESM3 provides a lot of improvements over earlier iterations. Because the model is “all to all” and naturally generative, structure and function annotations can be supplied as input in addition to output.

Scientists can refine this base model to create custom models based on their own proprietary data once it is made publicly available. A time-traveling device for in silico biological research is made possible by the increase in protein engineering capabilities brought forth by ESM3’s large-scale generative training across massive volumes of data.

NVIDIA BioNeMo

Creating the Next Major Advancements The generative AI boost that NVIDIA BioNeMo ESM-3 offers to biologists and protein designers enhances their engineering and comprehension of proteins. It can create new proteins using a framework that is provided, self-improve its protein design based on input, and design proteins depending on the functionality that the user specifies with only a few basic prompts.

Users can iterate back and forth using these capabilities in tandem or in any combination to provide chain-of-thought protein design. It is as if the user were messaging a researcher who had learned the language fluently and had memorized the complex three-dimensional meaning of every protein sequence known to humans.

According to Tom Sercu, vice president of engineering at EvolutionaryScale and co-founder, “They’ve been impressed by the ability of ESM3 to creatively respond to a variety of complex prompts in its internal testing.” A new green fluorescent protein was created by tackling a difficult protein design problem. They anticipate that ESM3 will help scientists work more quickly and create new opportunities; they’re interested to see how it will impact life sciences research in the future.

NVIDIA H100

Today, EvolutionaryScale will launch a closed beta for its API, and code and weights for a limited open version of ESM3 are freely accessible for non-commercial purposes. NVIDIA BioNeMo, a generative AI drug discovery platform, will shortly get this version. Select customers will soon have access to the whole ESM3 family of models as an NVIDIA NIM microservice, which has been run-time optimized in partnership with NVIDIA and is backed by an NVIDIA AI Enterprise software licence that can be tested at ai.nvidia.com.

These models require significantly more processing power to train. The Andromeda cluster, which makes use of NVIDIA H100 GPUs and NVIDIA Quantum-2 InfiniBand networking, was used to train ESM3.

The ESM3 model will be accessible on a few partner platforms, such as NVIDIA BioNeMo, Amazon Bedrock, Amazon Sagemaker, and AWS HealthOMICs.

ESM3 Futures

A large-scale language model created specifically for protein sequences is called ESM3 (Evolutionary Scale Modelling version 3). Some of its highlights:

High Predictability of Protein Properties: ESM3 accurately predicts protein structure, function, and evolutionary relationships.

Large-Scale Training: Thanks to training on an enormous protein sequence dataset, it is able to comprehend and produce extremely accurate protein-related data.

Transfer Learning: ESM3 has excellent versatility and adaptability to various protein analysis tasks, as it can be tailored for particular protein prediction tasks.

Efficient Model Architecture: To provide efficient processing and prediction, the model architecture is tailored to handle the length and complexity of protein sequences.

Drug Discovery Applications: ESM3 is a useful tool in drug development because of its precision in predicting protein structures and activities.

Integration with Bioinformatics Tools: Its usefulness in a range of scientific and medical applications can be increased by integrating it with current bioinformatics pipelines and tools.

Interpretable Predictions: By offering outcomes that can be easily understood, the model enables researchers to make well-informed judgements in their research by comprehending the foundation of its predictions.

Evolutionary links between proteins can be analyzed using ESM3, which provides support for the research of protein evolution and the discovery of conserved areas. Because of these characteristics, ESM3 is an effective instrument for promoting protein studies and their uses in biotechnology and medicine.

Workloads including artificial intelligence and machine learning are growing in popularity, thus it’s critical to protect them with specific data security procedures. In order to preserve the privacy of user prompts and AI/ML models during inference and to facilitate safe collaboration during model building, confidential computing can assist in protecting sensitive data used in ML training.

Google cloud unveiled two new Confidential Computing solutions at Google Cloud Next that are intended to safeguard the privacy and confidentiality of AI/ML workloads: Confidential VMs with support for Intel AMX and NVIDIA H100 Tensor Core GPUs with HGX protected PCIe.

A3 Secure Virtual Machines: Safeguarding Private AI/ML Tasks NVIDIA and Google Cloud have partnered to enable Confidential Computing on GPUs. With a specific Confidential Computing mode, the H100 is NVIDIA’s first GPU to support Confidential Computing. Enabling strong hardware-based security in this mode guarantees that data and code within the Trusted Execution Environment (TEE) can only be executed by authorised users.

You will be able to unlock use cases involving highly restricted datasets, sensitive models that require extra protection, and the ability to collaborate with multiple untrusted parties and collaborators while mitigating infrastructure risks and strengthening isolation through confidential computing hardware with Confidential VMs with NVIDIA H100 Tensor Core GPUs with HGX protected PCIe.

Adjusting and instruction You might want to make sure that your code and data are always protected when using AI models for training. Training involving input data with personally identifiable information (PII), proprietary data labelling, and trade secrets can be completed in a TEE with A3 Confidential VMs equipped with NVIDIA H100 GPUs. This ensures that any fine-tuning or additional AI/ML training is hidden from view outside the TEE.

To enhance customer service, a merchant would wish to develop a customised suggestion engine; nevertheless, this would necessitate training on consumer attributes and purchase history. The retailer can contribute to ensuring that customer data is safeguarded throughout by conducting training in a TEE.

Getting in touch AI models and weights are frequently delicate intellectual property that requires robust security. The models run the risk of being manipulated, disclosing private client information, or even being reverse-engineered if they are not safeguarded while being used. By enabling data and models to be processed in a hardened state, A3 Confidential VMs with NVIDIA H100 GPUs can help safeguard models and inferencing requests and responses even from the model developers, if desired. This prevents unauthorized access or leaking of the sensitive model and requests.

This is especially important for people who use chatbots that are AI/ML based. When interacting with a chatbot that uses a natural language processing (NLP) model, users frequently provide private information. Because of data privacy laws, these user inquiries may need to be secured. The author of the model-based chatbot may give consumers further reassurances that their inputs are private if the chatbot is hosted on A3 Confidential Virtual Machines.

Working together It is necessary for several organisations to train and make inferences on models while keeping their own models and constrained data private from one another. Organisations can collaborate in confidence by using NVIDIA H100 Tensor Core GPUs in Confidential VMs to enforce verifiable policies on data processing and outcome sharing.

In the healthcare sector, this use case is frequently seen when hospitals and medical organisations need to combine extremely private medical data sets or records in order to train models without disclosing the raw data from each party. Healthcare organisations can work together in a data clean room, such as Confidential Space, to guarantee security and performance, using A3 Confidential VMs with NVIDIA H100 GPUs.

A3 Confidential virtual machines (VMs) provide complete data protection by extending the Trusted Execution Environment (TEE) to include the integrated NVIDIA H100 GPUs. This is accomplished by using a privacy-centric strategy: a robust security border may be maintained by the CPU hardware preventing direct GPU access to the memory of the Confidential VM.

An encrypted “bounce buffer” in shared system memory is used by the NVIDIA driver, which runs within the CPU TEE, to enable safe data transport. By acting as a middleman, this buffer mitigates potential in-band attacks by guaranteeing that all communication, including that between command buffers and CUDA kernels, is encrypted between the CPU and GPU. This architecture essentially establishes a secure data pipeline that protects integrity and secrecy even when sensitive data is processed on the potent NVIDIA H100 GPUs.

The finest aspect? Performance and usability are maintained by transparent encryption handled by the GPU firmware and CUDA driver.

Secret Virtual Machines Using Intel TDX and Intel AMX According to Anand Pashupathy, vice president and general manager of Intel’s Security Software and Solutions Group, “AI will transform almost every industry, but it comes with critical security, privacy, and regulatory requirements.” “AI practitioners can safeguard their data and models and improve their compliance posture with Google Cloud’s new confidential AI offerings protected with Intel TDX, regardless of whether they are utilising CPU-based AI enhanced by Intel AMX instructions or AI accelerated by an external GPU.”

Intel TDX on C3 machine series The security features that are available for private virtual machines are determined by the CPU of a machine series. For instance, the 4th generation Intel Xeon scalable processors (code called Sapphire Rapids) power the general purpose C3 machine series. Intel Trust Domain Extensions (Intel TDX), a Confidential Computing technology, is supported by these processors.

To prevent unwanted access to critical data and programmes, Intel TDX establishes a hardware-based trusted execution environment that places each guest virtual machine (VM) in a cryptographically segregated “trust domain.” Since March, the C3 machine series with Intel TDX has offered confidential virtual machines (VMs) in preview.

Nevertheless, the C3 CPU has an additional crucial function known as Intel Advanced Matrix Extensions (Intel AMX). The purpose of this new instruction set architecture (ISA) extension is to speed up AI/ML workloads. It adds additional instructions that can be utilised for two of the most popular AI/ML operations: matrix multiplication and convolution. Compared to instances from earlier generations, AI inference performance is improved by C3 instances equipped with Intel AMX.

All Confidential VMs on the C3 machine series feature Intel AMX instruction sets by default in order to provide a higher level of protection for AI/ML workloads. As a result, your AI/ML workloads can operate in secret and remain shielded from unauthorised access by cloud operators and privileged administrators. The Intel AMX is an integrated accelerator designed to enhance the efficiency of CPU-based training and inference. It is a cost-effective solution for tasks like image recognition, recommendation systems, and natural language processing. It is possible to lessen the chance of unapproved parties discovering AI/ML code or data by using Intel AMX on Confidential VMs.

Using Confidential Computing to further safeguard their sensitive workloads, Thales, a global leader in innovative technologies across three business domains defense and security, aerospace and space, and cybersecurity and digital identity has benefited from this arrangement.

“As more businesses migrate their data, there is an increasing need to protect data integrity and privacy, particularly with regard to intellectual property, AI models, sensitive workloads, and valuable information. and workloads to the cloud. Through this partnership, businesses can safeguard and manage their data while it’s in use, in transit, and at rest with completely verifiable attestation. Our strong partnership with Google Cloud and Intel boosts our clients’ confidence in their cloud migration,” stated Todd Moore, vice president of Thales’ data security products.

Give it a shot now Intel AMX is turned on by default in Confidential VMs with Intel TDX on C3 machine series. To test out Intel AMX, just establish a Confidential virtual machine (VM) on the C3 machine series and execute your AI/ML workloads there.

Google Cloud and Altair EDEM

An essential skill for manufacturing, industrial, and life science companies is modelling the interactions between bulk and granular materials and containers, machinery, and each other. The more realistic these simulations get, the less time and money businesses need to spend refining their designs and prototypes. Recently, Altair and Google Cloud worked together to see how big of a simulation EDEM Altair could run on a single Google Cloud virtual machine. The outcomes were revolutionary.

Software for the Discrete Element Method (DEM) High-performance software for simulating both bulk and granular materials is called EDEM. Equipped with DEM, EDEM simulates and analyses a wide range of materials’ behaviours, including coal, mined ores, soils, fibres, grains, tablets, powders, and more, with speed and accuracy.

Engineers may get vital insight into how those materials will behave with their equipment under various operating and process settings by using EDEM modelling. It may be used in conjunction with other CAE tools or on its own.

Using EDEM, top businesses in the heavy equipment, off-road, mining, steel, and process manufacturing sectors can assess equipment performance, optimise workflows, and comprehend and forecast granular material behaviours.

GPUs drive simulations for EDEM A high-performance software programme for simulating bulk and granular materials is called EDEM Altair. Discrete element method (DEM) simulations and analysis of the behaviour of mined ores, soils, fibres, grains, tablets, powders, and more are performed fast and precisely thanks to EDEM calculations.

Several advancements in model size have been made as a result of industry demands for EDEM to operate at even larger scale. The maximum two decades ago was 200,000 particles, requiring up to ten days to generate in a simulation. That increased swiftly to one million particles, ten million, and finally twenty million. Today, completing a simulation with one billion particles in a few days is the holy grail.

Since these simulations necessitate a significant amount of processing power, as one might anticipate, the use of graphics processing units (GPUs) has significantly increased simulation speed and efficiency. GPUs are extremely effective at handling the massive volumes of data and intricate computations required for EDEM simulations because they are specifically made to handle parallel processing tasks.

The EDEM Altair experiment running on Google Cloud EDEM was initially intended to be a desktop application and has only ever worked with shared-memory architectures that are hosted on a single host. Unlike distributed memory, which can scale across multiple hosts, this restricts access to processors. EDEM is unable to benefit from the scalability and flexibility of distributed memory without a significant rebuild. On the other hand, multi-GPU systems provide a way to boost processing capacity without switching to distributed memory programming models.

The partnership between Altair and Google Cloud had two objectives: to simulate the largest system that could be created, with one billion particles, and to collect data in order to create estimates for mapping a specific type of hardware to a potential simulation scale. Google Cloud made the announcement that its A3 virtual machines (VMs) with NVIDIA H100 GPUs were now available in May 2023. This level of simulation is made feasible by the A3 VMs, which combine modern CPUs with NVIDIA H100 Tensor Core GPUs and also provide significant network upgrades and enhanced host memory.

Two simulation scenarios were conducted by Altair and Google Cloud on a single A3 virtual machine (VM) equipped with eight NVIDIA H100 GPUs, each having 80 GB of GPU memory and 3.6 TB/s of bisectional bandwidth. A 4th Generation Intel Xeon Scalable processor and 2 TB of host memory were also included in the system.

The filling test was selected as the test scenario. This useful simulation examines the effects of particle behaviour in an actual industrial environment, where particles are dropped into a container from a moving plate. In this simulation, single- and multi-sphere particles were employed.

As previously shown, the combined filing test simulation with single- and multi-sphere EDEM Altair simulations scales well across the eight NVIDIA H100 GPUs that are available, reaching a total particle count of one billion.

The manufacturing industries can now assess equipment performance, optimise processes on a never-before-seen scale, and better understand and predict granular material behaviours thanks to this new breakthrough in simulation technology billion particles which will enable future simulations with greater fidelity and larger scale.

Key points are summarised below: Importance of Bulk Material Simulation The behaviour of these materials must be simulated in bulk because it affects many different industries, including manufacturing, life sciences, and industrial processes.

Software called Altair EDEM Software called EDEM Altair was created especially for high-performance simulation of granular and bulk materials.

Need for Greater Scale The industry has been requesting that EDEM be able to manage even bigger and more intricate simulations.

Cooperation Goal Two primary objectives drove the collaboration between Altair EDEM and Google Cloud

Run the biggest simulation you can, aiming for a billion particles as a benchmark. Collect information to determine the possible simulation scale that can be reached with various hardware setups. Making Use of Google Cloud Hardware The secret to the success was to leverage the A3 virtual machines (VMs) on Google Cloud, which were outfitted with potent NVIDIA H100 GPUs.

A3 virtual machines offer the base for general-purpose computing. The processing power for high-performance computations is provided by H100 GPUs. The article discusses utilising eight H100 GPUs, but it makes no mention of the precise A3 VM configuration.