Enhanced computing efficiency expected to activate a wider range of users and application scenarios
On March 10, Chinese internet company ByteDance's DouBao large model team open-sourced a key optimization technology called COMET on GitHub, an internationally renowned open-source community platform. It is reported that this technology can improve the training efficiency of large models by 1.7 times and save 40% in costs. Currently, COMET has been actually applied to ByteDance's ten-thousand-GPU-cluster training, cumulatively helping to save millions of GPU hours of training computing power.
On February 12, the DouBao large model team announced that the team has proposed a brand-new sparse model architecture called UltraMem, which can effectively solve the high memory access problem during MoE inference. The inference speed is 2-6 times faster than that of the MoE architecture, and the cost can be reduced by up to 83%.
Industry insiders analyzed that the open-sourced COMET can be used in combination with the previously proposed UltraMem, further cutting down the training costs of large models.
Currently, with the continuous advancement and iteration of technology and the gradual participation of leading manufacturers in the open-source trend, the training costs of large models are continuously decreasing. Zhou Zhifeng, a managing partner of Qiming Venture Partners, once said that the invocation cost per million tokens of large models has dropped from USD 120 (CNY 800) in 2023 to less than CNY 1 in 2024, a decrease of 99.9%. It is estimated that the cost is very likely to drop another 99.9% in the future.