AI chip shortage: It's time for GPUs
Recent advances in artificial intelligence (AI) technology and its expanding applications have created a shortage of graphics processing units (GPUs). As AI takes over more of our lives, especially as generative AI becomes more mainstream, the computing power it requires becomes a critical factor, and GPUs are at the center of this.
While large GPU manufacturers like Nvidia are responding by announcing that they are ramping up production, the current demand for GPUs is far outpacing supply. This situation has forced cloud service providers like Amazon Web Services (AWS) and Microsoft Azure to limit the amount of computing power they offer, even to the point where Oracle and Microsoft are discussing an unusual deal that would allow them to lend each other servers when they run out of computing power for their cloud customers.
Why does computing power matter in generative AI? Because the performance of generative AI models is highly dependent on the input of computing power, especially GPUs. UBS analysts estimate that the initial version of the chatbot GPT required around 10,000 GPUs, with updated versions requiring three to five times that number.
As a solution to the GPU shortage, Nvidia announced at its recent Computex event in Taiwan that it is ramping up production of its H100 chips, but many AI founders expect the shortage to continue for at least the next year.
As a result, AI startups and venture capitalists (VCs) are scrambling to secure GPUs. Some VCs are ordering servers and processors in bulk to share among the AI startups they invest in. Some AI startups are buying their own servers with GPUs, while others are using minor cloud services until they can scale their AI models or the shortage is resolved.
The GPU shortage is spawning a new market, the GPU secondary market. This includes cryptocurrency miners. Companies that have accumulated large amounts of GPUs for mining but no longer need them due to the market downturn are finding themselves in the secondary market.
As a result, the GPU shortage is becoming a major issue for AI-based companies, such as chatbot GPT developer OpenAI, which has received numerous complaints from customers about slow service speeds. Resolving the GPU shortage is a challenge that must be overcome in order to improve the quality of AI technology and its services.














