#dellaifactory

There are several obstacles to overcome while preparing data center networking architecture for AI workloads. Waiting for network connectivity can take up to 33% of the time spent on AI/ML operations, leaving expensive GPU resources unused. In addition, cluster sizes are quadrupling and AI application traffic is growing exponentially, doubling every two years, placing enormous strain on network infrastructure.

Its developed a thorough process for building AI networks based on specific use cases using Dell Design Services for AI Networking. With this addition to Dell AI Factory services, you can build your AI networking to ensure maximum network performance.

Needs: Bandwidth Boosts, Minimized Latency & Lossless Transmission

A combination of AI inferencing and training tasks are included in enterprise use cases. Inferencing is the process by which a trained AI model converts the input data into actionable information by applying its learned parameters, weights, or rules. When using larger models, a network carrying inferencing traffic needs high bandwidth and low latency for real-time responsiveness.

Extreme bandwidth and parallel processing are necessary for complex AI training workloads in order to synchronize computations across the numerous GPUs in a cluster. The “elephant flows” produced by GPU synchronization are revolutionizing data center networking by necessitating previously unheard-of increases in bandwidth, reduced latency, and lossless data transfer.

Attributes of AI Network Fabrics

To overcome the difficulties presented by AI model training, AI back-end fabrics must be designed. Low latency and large capacity are necessary for these fabrics. Tail latency, which happens when processing is slowed down by a few odd requests, is something network designers must take into account.

AI fabrics use 800 Gb/s switching backplanes with optional 400Gb/s breakouts and non-blocking topologies to meet these requirements. Remote Direct Memory Access (RDMA) Over Converged Ethernet (RoCEv2) is one of the advanced features used. InfiniBand, a high-speed, low-latency networking solution, likewise relies heavily on RDMA. Two important possibilities for AI training fabrics are InfiniBand and 400/800 Gb Ethernet.

In AI networks, managing network congestion is essential. Priority-based Flow Control (PFC) allows network software to halt transmissions until the network can “catch up,” while Explicit Congestion Notification (ECN) provides early warning of a network congestion problem. Adaptive routing, dynamic load balancing, increased hashing modes, and packet/cell spraying are other sophisticated strategies that might be used.

Zero-touch provisioning and automated deployment, which allow for smooth scalability, are the foundation for efficient management and orchestration of these networks. Under heavy AI workloads, the network stays stable and dependable with advanced network monitoring technologies that offer early insight into possible problems or anomalies.

Strategic Planning for Future-Ready AI Networks

As is always the case with significant changes in technology, careful, in-depth planning and analysis are necessary for success.

A comprehensive examination of your existing network architecture is the first step. Capabilities, constraints, AI use cases, workload types, growth paths, and geographic reach are all assessed during this process. During this evaluation, locating integration sites for new AI networks components is essential.

Creating a vision of your ideal future network is the next stage. This necessitates a thorough examination of workload kinds, performance factors, and AI adoption trends. For smooth network growth as demand increases, a thorough GPU network design and integration advice are necessary.

Lastly, create a solid AI networks plan that covers connectivity options, network architecture, and technology selections. In order to ensure a robust and flexible network architecture that can satisfy future demands, this strategy should handle scaling requirements and growth management.

Access Extensive AI Network Experience and Expertise with Dell Services

Working with knowledgeable consultants can give you the technical know-how and specialized knowledge you need to integrate cutting-edge technologies, optimize AI networks performance, and uphold strong security measures so you can provide the infrastructure performance and dependability your AI use cases demand.

Building an AI Factory that consistently delivers AI-powered use cases, creates more effective workflows, and improves business outcomes requires optimizing AI network infrastructure. From strategy to technology architectures, data management, use case deployments, acceptance and change management, and more, Dell Technologies‘ AI specialists can help you move more quickly toward AI results. Its make use of Dell’s extensive network of partners to guarantee the comprehensiveness of your AI solutions.

When It Comes to Cooling and Powering Your  AI Factory, Think Dell. As part of the Dell AI Factory initiative, the company is thrilled to introduce a variety of new server power and cooling capabilities.

Dell PowerEdge XE9680L Server

As part of the Dell AI Factory, they’re showcasing new server capabilities after a fantastic Dell Technologies World event. These developments, which offer a thorough, scalable, and integrated method of imaplementing AI solutions, have the potential to completely transform the way businesses use artificial intelligence.

These new capabilities, which begin with the PowerEdge XE9680L with support for NVIDIA B200 HGX 8-way NVLink GPUs (graphics processing units), promise unmatched AI performance, power management, and cooling. This offer doubles I/O throughput and supports up to 72 GPUs per rack 107 kW, pushing the envelope of what’s feasible for AI-driven operations.

Integrating AI with Your Data

In order to fully utilise AI, customers must integrate it with their data. However, how can they do this in a more sustainable way? Putting in place state-of-the-art infrastructure that is tailored to meet the demands of AI workloads as effectively as feasible is the solution. Dell PowerEdge servers and software are built with Smart Power and Cooling to assist IT operations make the most of their power and thermal budgets.

Astute Cooling

Effective power management is but one aspect of the problem. Recall that cooling ability is also essential. At the highest workloads, Dell’s rack-scale system, which consists of eight XE9680 H100 servers in a rack with an integrated rear door heat exchanged, runs at 70 kW or less, as we disclosed at Dell Technologies World 2024. In addition to ensuring that component thermal and reliability standards are satisfied, Dell innovates to reduce the amount of power required to maintain cool systems.

Together, these significant hardware advancements including taller server chassis, rack-level integrated cooling, and the growth of liquid cooling, which includes liquid-assisted air cooling, or LAAC improve heat dissipation, maximise airflow, and enable larger compute densities. An effective fan power management technology is one example of how to maximise airflow. It uses an AI-based fuzzy logic controller for closed-loop thermal management, which immediately lowers operating costs.

Constructed to Be Reliable

Dependability and the data centre are clearly at the forefront of Dell’s solution development. All thorough testing and validation procedures, which guarantee that their systems can endure the most demanding situations, are clear examples of this.

A recent study brought attention to problems with data centre overheating, highlighting how crucial reliability is to data centre operations. A Supermicro SYS‑621C-TN12R server failed in high-temperature test situations, however a Dell PowerEdge HS5620 server continued to perform an intense workload without any component warnings or failures.

Announcing AI Factory Rack-Scale Architecture on the Dell PowerEdge XE9680L

Dell announced a factory integrated rack-scale design as well as the liquid-cooled replacement for the Dell PowerEdge XE9680.

The GPU-powered Since the launch of the PowerEdge product line thirty years ago, one of Dell’s fastest-growing products is the PowerEdge XE9680. immediately following the Dell PowerEdge. Dell announced an intriguing new addition to the PowerEdge XE product family as part of their next announcement for cloud service providers and near-edge deployments.

 AI computing has advanced significantly with the Direct Liquid Cooled (DLC) Dell PowerEdge XE9680L with NVIDIA Blackwell Tensor Core GPUs. This server, shown at Dell Technologies World 2024 as part of the Dell AI Factory with NVIDIA, pushes the limits of performance, GPU density per rack, and scalability for AI workloads.

The XE9680L’s clever cooling system and cutting-edge rack-scale architecture are its key components. Why it matters is as follows:

GPU Density per Rack, Low Power Consumption, and Outstanding Efficiency

The most rigorous large language model (LLM) training and large-scale AI inferencing environments where GPU density per rack is crucial are intended for the XE9680L. It provides one of the greatest density x86 server solutions available in the industry for the next-generation NVIDIA HGX B200 with a small 4U form factor.

Efficient DLC smart cooling is utilised by the XE9680L for both CPUs and GPUs. This innovative technique maximises compute power while retaining thermal efficiency, enabling a more rack-dense 4U architecture. The XE9680L offers remarkable performance for training large language models (LLMs) and other AI tasks because it is tailored for the upcoming NVIDIA HGX B200.

More Capability for PCIe 5 Expansion

With its standard 12 x PCIe 5.0 full-height, half-length slots, the XE9680L offers 20% more FHHL PCIe 5.0 density to its clients. This translates to two times the capability for high-speed input/output for the North/South AI fabric, direct storage connectivity for GPUs from Dell PowerScale, and smooth accelerator integration.

The XE9680L’s PCIe capacity enables smooth data flow whether you’re managing data-intensive jobs, implementing deep learning models, or running simulations.

Rack-scale factory integration and a turn-key solution

Dell is dedicated to quality over the XE9680L’s whole lifecycle. Partner components are seamlessly linked with rack-scale factory integration, guaranteeing a dependable and effective deployment procedure.

Bid farewell to deployment difficulties and welcome to faster time-to-value for accelerated AI workloads. From PDU sizing to rack, stack, and cabling, the XE9680L offers a turn-key solution.

With the Dell PowerEdge XE9680L, you can scale up to 72 Blackwell GPUs per 52 RU rack or 64 GPUs per 48 RU rack.

With pre-validated rack infrastructure solutions, increasing power, cooling, and  AI fabric can be done without guesswork.

AI factory solutions on a rack size, factory integrated, and provided with “one call” support and professional deployment services for your data centre or colocation facility floor.

Dell PowerEdge XE9680L

The PowerEdge XE9680L epitomises high-performance computing innovation and efficiency. This server delivers unmatched performance, scalability, and dependability for modern data centres and companies. Let’s explore the PowerEdge XE9680L’s many advantages for computing.

Superior performance and scalability

Enhanced Processing: Advanced processing powers the PowerEdge XE9680L. This server performs well for many applications thanks to the latest Intel Xeon Scalable CPUs. The XE9680L can handle complicated simulations, big databases, and high-volume transactional applications.

Flexibility in Memory and Storage: Flexible memory and storage options make the PowerEdge XE9680L stand out. This server may be customised for your organisation with up to 6TB of DDR4 memory and NVMe,  SSD, and HDD storage. This versatility lets you optimise your server’s performance for any demand, from fast data access to enormous storage.

Strong Security and Management

Complete Security: Today’s digital world demands security. The PowerEdge XE9680L protects data and system integrity with extensive security features. Secure Boot, BIOS Recovery, and TPM 2.0 prevent cyberattacks. Our server’s built-in encryption safeguards your data at rest and in transit, following industry standards.

Advanced Management Tools

Maintaining performance and minimising downtime requires efficient IT infrastructure management. Advanced management features ease administration and boost operating efficiency on the PowerEdge XE9680L. Dell EMC OpenManage offers simple server monitoring, management, and optimisation solutions. With iDRAC9 and Quick Sync 2, you can install, update, and troubleshoot servers remotely, decreasing on-site intervention and speeding response times.

Excellent Reliability and Support

More efficient cooling and power

For optimal performance, high-performance servers need cooling and power control. The PowerEdge XE9680L’s improved cooling solutions dissipate heat efficiently even under intense loads. Airflow is directed precisely to prevent hotspots and maintain stable temperatures with multi-vector cooling. Redundant power supply and sophisticated power management optimise the server’s power efficiency, minimising energy consumption and running expenses.

A proactive support service

The PowerEdge XE9680L has proactive support from Dell to maximise uptime and assure continued operation. Expert technicians, automatic issue identification, and predictive analytics are available 24/7 in ProSupport Plus to prevent and resolve issues before they affect your operations. This proactive assistance reduces disruptions and improves IT infrastructure stability, letting you focus on your core business.

Innovation in Modern Data Centre Design Scalable Architecture

The PowerEdge XE9680L’s scalable architecture meets modern data centre needs. You can extend your infrastructure as your business grows with its modular architecture and easy extension and customisation. Whether you need more storage, processing power, or new technologies, the XE9680L can adapt easily.

Ideal for virtualisation and clouds

Cloud computing and virtualisation are essential to modern IT strategies. Virtualisation support and cloud platform integration make the PowerEdge XE9680L ideal for these environments. VMware, Microsoft Hyper-V, and OpenStack interoperability lets you maximise resource utilisation and operational efficiency with your visualised infrastructure.

Conclusion

Finally, the PowerEdge XE9680L is a powerful server with flexible memory and storage, strong security, and easy management. Modern data centres and organisations looking to improve their IT infrastructure will love its innovative design, high reliability, and proactive support. The PowerEdge XE9680L gives your company the tools to develop, innovate, and succeed in a digital environment.

Dell AI Factory

AI is revolutionizing organisations by delivering unprecedented efficiency and competitive advantage. Data sovereignty, skills shortages, and the cost of deploying and growing AI in a corporation make navigating the AI opportunity tough.

This week at Dell Technologies World, They’re talking about how the Dell AI Factory tackles these most important issues by providing a range of AI technologies, an accessible partner ecosystem, integrated, validated solutions, and knowledgeable services to help you get AI results more quickly. Dell is pleased to announce a number of new infrastructure, ecosystem partnerships, and services that will further enhance the capabilities of the Dell AI Factory and contribute to the world’s largest range of AI solutions. Their objective is to assist you in reducing risk associated with your priceless data and intellectual property, increasing control over your budget and resources, and speeding up the time to business outcomes.

AI Infrastructure Provides an Adaptable IT Base

Businesses are realising that the data centres of today were not designed to fully utilise artificial intelligence. The new paradigm is the AI factory, and the infrastructure that supports AI is what makes it feasible. In order to provide you with even more alternatives for the computing power, scalability, data accessibility, and dependability you need for AI success, Dell are expanding dell’s range of AI infrastructure across data centre components and client devices.

Dell’s array of AI infrastructure additions gives you better options for data storage, data security and speedy access. Dell’s largest capacity all-flash NAS system, the Dell PowerScale F910, is the newest model in dell’s next-generation file storage lineup and was designed with artificial intelligence in mind. With this most recent system, it offers up to 127% better performance to deliver AI insights faster, speeding up every stage of the AI pipeline. In addition to PowerScale, they’re excited to introduce Project Lightning, which will enable you to meet the demands of AI factories by providing a scale-out NAS and parallel file system. Dell PowerScale performance will soar up to 20 times faster than competitors using only flash for scale-out files.

The Dell Solution for AI Data Protection provides a structure for enterprises to follow when incorporating strong data security measures into their AI frameworks. Along with new features for Dell’s Enterprise SONiC distribution, Dell is also expanding dell’s networking offerings with the introduction of Dell SmartFabric Manager for SONiC, a new management tool that streamlines deployment, orchestration, and lifecycle management with a single view of SONiC fabrics. These tools combined will enable secure, feature-rich, scalable fabrics with optimised traffic flows for demanding workloads like GenAI. Additionally, AI applications benefit from the new Dell PowerSwitch Z9864F-ON, which can handle large volumes of data and provides two times the switching capacity and bandwidth with state-of-the-art Ethernet technology.

Leveraging AI’s potential goes beyond the data centre; it might also mean arming your team with strong instruments and apparatus to spur innovation and expediency. They’re excited to present Dell’s inaugural fleet of Copilot+ PCs, which are powered by the Qualcomm Snapdragon X Elite and X Plus. These devices support critical AI use cases, such as running Windows 11 AI experiences locally. The performance and efficiency of new on-device AI capabilities are revolutionary, enabling previously unheard-of levels of worker productivity. With lightning-fast speeds and all-day battery life, these PCs come equipped with a specialised AI engine that can run generative AI (GenAI) large language models (LLMs) with over 13 billion parameters on-device.

Partnerships in the AI Ecosystem Facilitate the Adoption of Reliable, Integrated Solutions

Advanced technologies from numerous technology sources are advantageous to AI factories. Dell’s extensive AI ecosystem, which has a track record of fostering industry collaborations, is intended to provide you with increased access to innovation and choice. Dell’s industry-leading AI ecosystem partners work with them to offer seamless compatibility and integration throughout the stack of AI infrastructure. For instance, our pre-validated and pre-tested AI solutions eliminate uncertainty and dramatically reduce deployment time with reliable, full-stack configurations tailored for certain GenAI applications and use cases.

Dell AI Factory with Nvidia

Expanding upon dell’s March launch of the Dell AI Factory in partnership with NVIDIA, a number of new developments leverage the most recent NVIDIA and Dell products and services to boost AI innovation and adoption. Personalised retail experiences, industrial automation, digital assistants, and other use cases are all supported by portfolio expansions. View the significant changes to Dell AI Factory with NVIDIA that are the outcome of dell’s close partnership with NVIDIA.

Data scientists and the developer community benefit from dell’s collaboration with Hugging Face, and dell understand the value of fostering an open ecosystem of AI technology partners. Dell becomes the first infrastructure provider to collaborate with Hugging Face to offer optimised on-premises deployment of generative AI models with the launch of the Dell Enterprise Hub on the Hugging Face platform. In the end, this new portal provides a simplified method for popular LLMs to be deployed and customised on Dell infrastructure for Dell clients.

Furthermore, the open-source AI foundation model space has advanced significantly with the publication of Meta Llama 3 models. Together, Meta and Dell will continue to present benchmarking results, performance information, and deployment recipes that show off how simple it is to set up Meta Llama 3 models on Dell’s reliable infrastructure. With the use of specially made containers and scripts, these LLMs are also accessible on the Dell Enterprise Hub.

To ensure seamless integration and compatibility across various components of the AI infrastructure stack, Dell is expanding dell’s pre-tested and proven Dell Generative AI Solutions in collaboration with other ecosystem partners. For instance, to streamline the delivery of AI services like voice transcription and translation, a new Dell AI Solution for Microsoft Azure AI Services combines Microsoft Azure AI with Dell APEX Cloud Platform for Microsoft Azure. One potential use case where the system can facilitate easier and more effective communication between doctors and patients who speak different languages is the healthcare industry.

AI Services Provide Professional Advice and Direct Assistance to Quicken IT Results

While GenAI is predicted to be the most popular sector for technology investment in 2024, CEOs see talent and AI capabilities as the biggest obstacles to enterprise adoption. Here, Dell’s entire package of professional services for AI fills the gap by helping establish strategies and accelerating the implementation of GenAI.

Dell have added a number of additional services to dell’s portfolio, such as Implementation Services for Microsoft Copilot Solutions, which include GitHub Copilot, Microsoft Copilot for Security, Microsoft Copilot for Sales, and Microsoft Copilot in Windows. To optimise the return on your investments and boost productivity and efficiency throughout the company, these services assist you in planning, implementing, and promoting the adoption of new GenAI copilots. Dell’s extensive service offering now includes new Accelerator Services for RAG on Precision AI workstations and Dell Enterprise Hub on Hugging Face, enhancing the developer experience for quick GenAI development.

Find Out More About The AI Factory at Dell

As a result of dell’s ongoing innovation, Dell is prepared to assist you in deploying AI strategically and swiftly with a wide range of market-leading solutions and alliances, giving you the ability to obtain favourable results and a competitive edge as you apply various AI models to your data across PCs, on-premises IT, multiple clouds, and edge locations.