While doing some research on the environmental impact of LLMs, especially their energy consumption, I stumbled upon this interesting study that attempts to estimate the power consumption of various LLMs. There’s a Power BI Dashboard showing off the study’s findings in a more digestible manner.

However, looking at the linked dashboard, I quickly noticed that something felt off about the data. DeepSeek R1 and V3 were ranked way higher than I would’ve expected, so I set out to understand how this data was created in the first place. This post also serves as a cautionary tale about data found online, even from reputable sources.

The problems

Below is the equation used by the study to calculate the estimated power consumption of LLMs per query of fixed length (water consumption and carbon footprint are derived from this): But there’s a flawed assumption here. The first part of the equation calculates the inference time of a query, then multiplies it with the estimated power draw of the infrastructure running the LLM (which is estimated based on the LLMs size, which is estimated as well for closed models). PUE stands for Power Usage Effectiveness, which in this case accounts for additional datacenter overheads.

In this equation, as TPS (Tokens Per Second) increases and latency (Time To First Token) decreases, the energy consumption per query goes down. It’s important to note that they didn’t run the models in some kind of controlled lab setup, but just measured against the public APIs. Also, note that the infrastructure to run the models is assumed to be the same for all models in a model class (which is based on the actual or estimated size of the model, with all models that can be considered somewhat state-of-the-art (SOTA) in the “Large” class).

To put it into perspective, this approach is a little like putting all cars with 200+ horsepower into the same category, assuming a static fuel consumption per hour and then letting them drive 200km to see which one gets there the fastest, which must mean it’s the most efficient.

Even if we assume the hardware estimates to be correct, there is still a problem with regard to capacity. DeepSeek (the company) simply doesn’t have access to anywhere close the amount of GPUs OpenAI or Anthropic are using, but still have to serve a large amount of users. Let’s assume the worst case: at the time a new query arrives, there is simply no capacity for it. What happens now is that the query simply has to wait for others to finish before it gets scheduled for completion. The cost of keeping it in memory while waiting is completely negligible, but the above equation would still assume it to draw full power.

At the other end, techniques like speculative decoding can increase compute utilization to increase TPS, which means more power consumption for more speed. Then there’s also the option of simply using more than the assumed 8 GPUs per model instance, which is almost certainly something that’s happening and would result in faster responses while drawing more power.

Could they have done better?

However, all of these flaws are simply caused by the fact that due to the complexity of on-scale LLM-inference and almost all providers not being transparent about how they actually do it, there is a lot of guesswork in trying to figure out the actual environmental costs of running these models.

This paper should to be seen as an *attempt *to put *some *data into our hands, even if it is based on many potentially flawed assumptions and rough estimates. I think the authors could have done a better job of acknowledging these flaws, but they’re not at fault for their existence. They were simply working with what they had.

Unless model providers start releasing their own data, I think this paper might be the best we can do for now.

Can we still learn from this?

I think there are still some things we can take away from this data. I think it’s safe to say that we can disregard the absolute values, but we can still, albeit carefully, use it as a relative benchmark between models. The DeepSeek models ranked so far up are the ones hosted by DeepSeek themselves on weaker GPUs due to US export controls, but the authors also included the DeepSeek models hosted by Azure in their data, which perform a lot better on infrastructure that’s likely to be more comparable to OpenAI and others.

Therefore, looking at the top models of each company, we can come up with this ranking as of today (from most efficient to least efficient):

1. Google (running on their own TPUs, so careful with this one)
2. DeepSeek
3. Anthropic
4. Mistral
5. OpenAI
6. xAI

I often see findings from AI papers with questionable methodology or data representation taken at face value, without examining how those claims were actually produced. One well-known example is the “Model Collapse” paper. I hope this analysis encourages readers to approach such claims more critically in the future.

Anyone who witnessed the outcries after the releases of DeepSeek V3 and R1 will be very much aware that there is a technological race happening between the United States and China in the field of AI, which of course is only one battleground in an ongoing war for economic hard power between two rivaling nations.

We’re currently witnessing a profound change in the geopolitical landscape. After the fall of the Soviet Union, the global system was basically a unipolar one, with the US as a mostly unchallenged global power. Over the last decades, however, China has grown to become a serious contender, challenging the US’s position in the global order and perhaps even creating a new, multipolar world order. One of the most important resources one needs to influence the global order is economic leverage.

Globalization has caused great economic interdependencies between countries all over the world, which in turn means that control over those supply chains is directly tied to geopolitical influence. One example of this control would be TSMC, the Taiwan-based chip manufacturer powering the current AI boom with their high-performance chips used in Nvidia’s GPUs. Access to those chips is to a large degree controlled by the US, which is blocking China from freely accessing TSMC’s chips. However, that doesn’t stop China from becoming a serious contender in the AI race, and with that a threat to the US’s economic hard power.

It seems like both China and the US have recognized that being a technological leader in the field of Artificial Intelligence will be paramount to being a global economic power. At the moment, it’s probably fair to say that the technological advantage lies with the US, through companies like OpenAI, Anthropic, Google and Meta. However, it seems like China might be catching up, causing industry leaders and politicians in the US to become increasingly nervous. Many Chinese companies have joined the AI race and produced notable models that challenge those of OpenAI and Anthropic: DeepSeek (financed by Chinese hedge fund High-Flyer), Alibaba (with the Qwen and Wan model families), Tencent (with the Hunyuan model family) and z.ai (with the GLM model family), among others.

While most of these models aren’t on the same level as the US ones in terms of capabilities, they’re close, and getting closer with each new release. They’re also often a lot cheaper to run and released as open-weight models under permissive open-source licenses, meaning they can be run by anyone. DeepSeek is also notable for publishing very detailed papers about their models and surrounding infrastructure compared to US companies like OpenAI and Anthropic, who aren’t very transparent at all about what’s behind their models.

Source: Artificial Analysis

In a recent article called “The Cost of the AGI Delusion”, Michael C. Horowitz and Lauren A. Kahn argue that the US’s focus on creating AGI (a goal we’re not even sure is even reachable) is causing it to fall behind in the AI race. While American companies focus on more and more expensive innovation, their Chinese counterparts focus on efficiency and cost-effectiveness, which are major factors in driving adoption. In “The Innovation Fallacy”, Jeffrey Ding argues that the long-term winners aren’t the leaders in innovation, but in widespread adoption of new technologies. The argument could be made that China is going to become a leader in AI adoption and thereby win the race for economic hard power through AI.

Now, why do I write about this? I’m trying to navigate the jungle of AI information and figure out the actual truths between all the hype and bad-faith actors in the field. After the release of DeepSeek R1, many attempts have been made to denounce the claims of innovations in efficiency made by this model and I’ve come to the conclusion that most of them were plain wrong (DeepSeek publications that show the degree of optimization they use for their models: 1,2,3,4,5). I’ve also seen some notable figures in the AI space, who usually know what they are talking about, starting to spread misleading narratives about Chinese AI models, for example Groq CEO Jonathan Ross suggesting that US models are optimized for cheaper inference while Chinese models are optimized for cheaper training (Link to the post). This implies that US models are ultimately more practical or cost-effective for end-users. In reality, as the market clearly shows, Chinese AI APIs are often significantly cheaper than their US counterparts.

Information on AI has been dominated by the companies that profit from AI for a while now and I’ve gotten used to their often misleading and overhyping narratives being spread, but it seems like the nervousness about China is starting to intensify this behavior even more, making it even harder to find truthful information on AI out there. It is important to be aware of these political interests when trying to stay informed about AI.

Of course there are examples of objectively good prompts and objectively bad prompts, but just like with good and bad communication, it’s incredibly context-dependent and can often only be judged in hindsight (usually by asking “Did I *actually *achieve what I was trying to achieve?”).

The philosophy

So we will treat good prompting like good communication, which can tell us a lot about how to get good at it:

Step 1: Say the thing you want to say. Step 2: Observe. Step 3: Learn from what you’ve seen.

And here is the nice part about LLMs: As users, we have full control over their memories! It doesn’t matter if our first attempt at communicating something fails, we can just erase that from the LLM’s memory and start from scratch. Another benefit we get from this is that we don’t have to be sure what exactly we want and can just find out on the fly by observing results until we get one we like (this is something you should *not *be doing with humans unless you’re in something like a brainstorming setting).

The practice

So how can we apply this?

1. Start off by telling the LLM what you want it to do, in the clearest way you can.
2. Let the LLM generate its answer. Is it what you wanted? Great, we’re done! Good job! It’s not what you wanted? Let’s observe and refine.
3. Look at the LLM’s answer in relation to your prompt. What exactly did it get wrong? Did it misinterpret a part of your prompt? Did it lack some information? Is the answer alright, but just not to your taste?
4. Edit your prompt to provide additional clarity, instructions (some step-by-step guidance can go a long way!) and information. Yes, edit. Try to avoid continuing the conversation to fix errors, as there is a solid body of research (1,2,3,4) suggesting that LLMs are steered to make more mistakes after an initial error.
5. Repeat steps 3 and 4 as necessary.

I know this might not sound as sexy as “Here’s the ultimate prompt that will fix all your problems!”, but this approach will help you develop prompting as a skill. You will get better over time and your intuition will allow you to get more prompts right on the first try.

Since all of this was pretty abstract, here are a couple of more concrete tips:

• Be careful with role assignments (“You are an industry expert in XYZ”), this could lead to an increased chance of hallucinations since the LLM might simply start cosplaying (imagine you said that to another human, replace the word “hallucination” with “bullshit” and you will see where I’m coming from with this). However, role-assignments can be useful for setting the tone/style of a reply.
• When switching topics or jumping to a new task, unless you need the context of the current chat, open a new one. LLMs are steered by the total context given to them and you don’t want unrelated information to interfere with what you’re trying to do.
• LLMs are probabilistic, so regenerating an answer with the same inputs will, in most cases, lead to different outputs (this is especially useful in that “I don’t actually know what I’m looking for”-case, so just let the LLM give you options!)
• Keep in mind that you’ll have (possibly hidden) system prompts in place on most chat platforms, so there might be additional information and instructions that are fed into the LLM and are not written by you.
• Every LLM is different in both the way it interprets your inputs and the style of its outputs. This is where observation comes into play again!
• Structure is useful. Use line breaks and even markdown where appropriate.
• Write properly. Fix typos in your own prompt and make sure it wouldn’t cause someone teaching the respective language to break down in tears.

The ability of LLMs to produce “fluent nonsense”—plausible but logically flawed reasoning chains—can be more deceptive and damaging than an outright incorrect answer, as it projects a false aura of dependability.

This idea is something that has been on my mind for a while now, but reading it phrased like this made something click.

When I hear people talk about hallucinations, the focus still seems to be on a simple, binary true/false in terms of the information that is returned by an LLM, but that’s not the real issue and as long as this is not collectively understood, the use of LLMs can be more harmful than we might expect.

We train LLMs to be helpful and to produce outputs we deem correct, but since these models don’t actually have a proper concept of true and false information, we actually train them to produce outputs that look correct. This isn’t just limited to answering questions, but can also mean that the model will deceive its users about what it can and can’t actually do.

When dealing with an LLM’s outputs, we have to think of it not as reading something that was written by someone who might not know what they’re talking about, but rather like it might be a sophisticated piece of fake news. It might make sense, it might sound plausible, but there could be an important detail that doesn’t actually reflect the truth.

This is true for code produced by LLMs as well. “LLMs are like junior developers” is a popular saying, but it doesn’t quite capture that you shouldn’t review that code as if it was written by a junior, as the mistakes made by it might be a lot more subtle than that. AI-written code should be reviewed as if it was written by a senior, someone who knows what they’re doing and can produce coherent code, but still might miss an important detail that doesn’t become apparent at first glance. Though even that analogy might not quite capture reality, because one of the big differences between a human developer and an LLM is the fact that LLMs can produce syntactically perfect solutions without any understanding of their real-world implications.

These insights should shape the way we educate people on LLMs and their effective usage to ensure that they actually benefit us, not just deceive us into thinking they do, which could have devastating consequences. Education on LLMs should not just be about prompting techniques, but about developing a critical mindset specific to verifying their outputs.

We should put more effort into explaining the fundamental concepts of LLMs and generative AI models in general to a non-technical audience, because the insights gained from that are invaluable when it comes to analyzing outputs generated by these models.