The Real Carbon Cost of an AI Token

Every time you prompt an AI and it generates a word, there's an invisible cost paid in carbon.

We usually think in dollars per token, but what about CO₂ per token? A single ChatGPT request emits roughly 4.32 g CO₂ on average, an order of magnitude more than a Google search at ≈ 0.2 g CO₂. A few grams seem trivial until millions of users ask millions of questions: the grams quickly add up to tonnes.

Below, we dissect the carbon footprint of today's large-language models (LLMs), covering both one-time training and day-to-day inference, and finish with a checklist you can apply right now to slash emissions without sacrificing quality.

Training

For scale, 550 t CO₂ ≈ the emissions of 30 round-trip flights NYC to London. Training matters-but, as we'll see, inference dominates long-term impact.

Inference

Analyses by Meta AI, AWS SageMaker and Google show that 60 to 90% of an LLM's life-cycle emissions come from inference, not training (Google Green-AI audit). Power-hungry GPUs (or TPUs) sit on-call 24 × 7, generating answers token-by-token.

How Much CO₂ per Token?

Figures synthesise measurements from "From Words to Watts: Benchmarking the Energy Costs of LLM Inference" with vendor-reported perf/W data. Note that specific CO₂ figures depend heavily on the electricity grid's carbon intensity where the computation occurs.

Generating ~350 tokens (≈ 500 words) on an H100 draws only 0.008 kWh, yet at global ChatGPT scale that's tens of tonnes per day.

A separate empirical study put a 1,000-token, image-enhanced ChatGPT request at 8.3 g CO₂, roughly the footprint of charging a smartphone ten times. For perspective, a single task on the ARC-AGI benchmark for GPT-o3 consumes approximately 1,785 kWh of energy, equivalent to two months of an average U.S. household's electricity use.

Choosing a Low-Carbon Model (Without Wrecking Quality)

Rule #1: Use the smallest model that meets the task's quality bar.

Below are common tasks and evidence that "smaller" often suffices:

Fine-tuning, prompt engineering and RAG let smaller models "punch above their weight," delivering orders-of-magnitude greener inference (Luccioni et al.). The newest generation of efficient models like Llama 4 Scout (17B parameters) and Gemma 3 demonstrate that smaller doesn't necessarily mean less capable.

Checklist: Ten Immediate Wins for Greener LLM Apps

1. Right-size the model. Benchmark a 7B or 13B alternative before defaulting to GPT-4 or GPT-o3.
2. Fine-tune or distil. A domain-specific 7B often beats a generic 70B.
3. Quantise aggressively. INT8 / FP8 cuts energy 2 to 4× with negligible quality loss (TensorRT-LLM case study).
4. Pick efficient accelerators. H100 or TPU v5e deliver > 2× tokens/W versus A100.
5. Consider mixture-of-experts models. Models like Gemini 2.5 Pro and Llama 4 Maverick activate only relevant parameters, improving efficiency.
6. Batch and stream smartly. Full GPU utilisation slashes joules per token.
7. Trim prompts & max-tokens. Don't encode or generate text you'll discard.
8. Cache recurring answers. Stop paying (in dollars and CO₂) for repeat queries.
9. Choose green regions. Oregon's hydro-heavy grid beats Virginia's coal-heavy grid.
10. Schedule maintenance jobs for clean-grid hours. Solar-rich midday or windy nights.

Real-world teams have achieved > 10× CO₂ reduction by combining just a few of the above.

A Call for Transparency

Meta publishes full emissions for LLaMA-2 and has continued this practice with Llama 4, reporting 1,999 tons CO₂eq for training Scout and Maverick models. Hugging Face now displays per-model estimates on the Open LLM Leaderboard.

By contrast, OpenAI provides limited environmental data for GPT-o3, with third-party analysis suggesting extremely high per-task energy consumption. Google has not published specific carbon footprint data for Gemini 2.5 Pro or Gemma 3, though they emphasise Gemma's efficiency focus. Anthropic still provides no model-specific carbon data. If a carmaker hid its MPG you'd balk; why tolerate opacity from AI vendors whose flagship training runs may exceed 15,000 t CO₂?

We need:

• Training-emission disclosure (compute hours, PUE, grid mix, offsets).
• Standardised inference-efficiency reporting (Wh or g CO₂ per 100 tokens).
• Clear offset accounting (quality and permanence of removals).

Competition on efficiency is healthy; secrecy delays progress.

What's Next?

The age of foundation models has arrived, and with it a surge in computing's climate cost. Yet the data show that smarter choices (smaller models, quantisation, efficient hardware and clever system design) can cut emissions > 10× with little or no quality loss.

The emergence of mixture-of-experts architectures in models like Gemini 2.5 Pro and Llama 4 Maverick represents a promising direction, allowing selective activation of parameters rather than running entire trillion-parameter models for every query. Similarly, efficiency-focused models like Gemma 3 demonstrate that high-quality results don't always require massive computation.

The next time you marvel at an AI-generated poem or bug-fix, ask yourself: "How many milligrams of CO₂ did that cost?" By making that question normal, we drive the entire ecosystem toward models that are not only powerful, but planet-friendly.

Let's build AI we can be proud of, technically and environmentally.

‍