The Environmental Impact of Large Language Models

Like it or not, large language models have quickly become an integral part of our daily lives. However, due to their significant energy and water requirements, they may be contributing to the acceleration of climate change. A recent study has found that some large language models (LLMs) may be releasing more planet-warming pollution than others.

Carbon Emissions from LLMs

According to a new study published in Frontiers in Communication, queries made to some models can generate up to 50 times more carbon emissions than others. Unfortunately, models that are more accurate tend to have the largest energy costs. It’s challenging to estimate the exact environmental impact of LLMs, but some studies suggest that training models like ChatGPT can use up to 30 times more energy than the average American uses in a year. What’s unknown is whether some models have steeper energy costs than their peers when answering questions.

Research Methodology

Researchers from the Hochschule München University of Applied Sciences in Germany evaluated 14 LLMs with parameters ranging from 7 to 72 billion on 1,000 benchmark questions across various subjects. LLMs convert each word or parts of words into a string of numbers called a token, which uses energy and releases CO2. Some LLMs, particularly reasoning models, also insert special "thinking tokens" into the input sequence to allow for additional internal computation and reasoning before generating output.

Key Findings

The study found that reasoning models, on average, created 543.5 thinking tokens per question, whereas concise models required just 37.7 tokens per question. This conversion and subsequent computations drive up energy needs. "The environmental impact of questioning trained LLMs is strongly determined by their reasoning approach," said study author Maximilian Dauner. "We found that reasoning-enabled models produced up to 50 times more CO2 emissions than concise response models."

Accuracy-Sustainability Trade-off

The study found that the more accurate the models were, the more carbon emissions they produced. The reasoning model Cogito, with 70 billion parameters, reached up to 84.9% accuracy but produced three times more CO2 emissions than similarly sized models that generate more concise answers. "Currently, we see a clear accuracy-sustainability trade-off inherent in LLM technologies," said Dauner. "None of the models that kept emissions below 500 grams of CO2 equivalent achieved higher than 80% accuracy on answering the 1,000 questions correctly."

Subject Matter and Emissions

Another factor that affected emissions was the subject matter. Questions that required detailed or complex reasoning, such as abstract algebra or philosophy, led to up to six times higher emissions than more straightforward subjects.

Limitations and Future Directions

There are some caveats to the study’s findings, as emissions are dependent on local energy grids and the models being examined. Nevertheless, the study authors hope that their work will encourage people to be "selective and thoughtful" about their LLM use. "Users can significantly reduce emissions by prompting AI to generate concise answers or limiting the use of high-capacity models to tasks that genuinely require that power," said Dauner. By being more mindful of LLM use, we can mitigate the environmental impact of these models and work towards a more sustainable future.