One of the key challenges in Large Language Model (LLM) training is reducing the memory requirements needed for training without sacrificing
Support for memory efficient optimizations enables effective training of large models on the Habana Gaudi platform. DeepSpeed ZeRO-2 is available with the 1.6 release of Habana Synapse AI Software toolkit.Â















