Google DeepMind, Google’s AI research lab, has introduced a groundbreaking AI training method called JEST, which significantly improves both training speed and energy efficiency. DeepMind claims that JEST yields 13 times more performance and 10 times higher power efficiency compared to traditional methods. This advancement is especially timely considering the growing environmental concerns associated with AI data centers.
The JEST method, or joint example selection, differs from conventional AI training techniques by focusing on entire batches of data instead of individual data points. Initially, a smaller AI model is created to evaluate data quality from high-quality sources and rank batches accordingly. This smaller model’s findings are then used to train a larger model, ensuring that only the most suitable data is utilized.
DeepMind’s researchers emphasize that well-curated data is crucial to the JEST method’s success. They claim their approach reduces the number of training iterations by up to 13% and computational resources by 10%. However, this method’s effectiveness depends heavily on the initial human-curated dataset. This requirement makes it harder for hobbyists and amateur AI developers to replicate the process due to the expertise needed in data curation.
The announcement comes as the tech industry and governments are discussing the high power demands of AI. In 2023, AI workloads consumed about 4.3 GW of power, nearly matching the annual consumption of Cyprus. With AI’s power consumption projected to increase, innovations like JEST could potentially reduce energy demands and help manage costs. However, it remains to be seen if major AI firms will adopt JEST to lower power usage or to enhance training speeds.
The impact of JEST on the AI industry is yet to be determined, but the balance between cost savings and output efficiency will likely influence its adoption. While some hope JEST will make AI training more sustainable and cost-effective, others speculate that companies may prioritize maximizing output.
