AI Sustainability First Steps.

AI-sustainability slide deck

As the Innovation Studio team at Mozilla has been ramping up its investigation into responsible AI, it’s become clear that we need to consider the environmental impact of the tools we build. I was asked to put together a quick primer on initial considerations.

Why prioritize sustainability in AI

  • AI Model Training utilizes cloud GPUs that require 10–15 times the energy a traditional CPU (source)
  • Training an LLM such as GPT-4 is estimated to take up to 300 tons of CO2e (source)
  • NVIDIA estimates that 80-90% of the energy cost of neural networks lies in ongoing inference processing after a model has been trained (source).
  • A single request in ChatGPT can consume 100 times more energy than one Google search (source)
  • 70% of the energy used by a typical laptop computer is incurred during its manufacture. Desktops even higher (source)
  • The difference in CO2e powering the infrastructure behind AI systems can vary wildly (as much as 30x)

Where to Optimize

  • Model Training
  • Inference (prediction using new input data)
  • Hardware Life Cycle Assessment and cloud data center energy usage

How to Optimize

  • Use existing large generative models, don’t generate your own.
  • Fine-tune train existing models or use embeddings.
  • Employ the minimum sized model necessary for your AI task
    • pruning
    • compression
    • distillation
    • quantization
  • Use web search & non-ML APIs / algorithms
  • Optimize infrastructure (source)
    • On cloud vs local (1.4-2x reduction cloud vs. local)
    • in regions running on clean energy (5-10x reduction)
    • utilizing optimized hardware (2-5X reduction)
    • maximizing utilization
  • Cache results of repeat tasks used in dev / testing,
  • (e.g. generating a stock translation or LLM response)
  • Include AI in your carbon monitoring, and adjust as necessary
  • Make carbon impacts a core consideration alongside functional and business requirements

Tools

  • Experiment Impact Tracker | Code Carbon code-based package to track energy usage of code (backend)
  • CO2.js code based packages to estimate energy use (frontend)
  • Carbon Aware SDK - utilize to optimize timing of energy intensive tasks
  • ML CO2 Impact Estimator - Quick web tool for calculating training CO2 based on GPU / hours / provider / region
  • EcoPing - Utilizes lighthouse and other metrics to measure changes in energy consumption of your site over time

Roadmap

  • Checklist: Establish a checklist for AI sustainability that is utilized in all product codebases.
  • Conduct: audit against checklist before launch with a pass / fail criteria and recommendations for improvement
  • Code: Create an code energy monitoring guide and relevant infrastructure.
  • Cloud: Ensure cloud infrastructure for ML ops is being measured and optimization recommendations are being surfaced to relevant stakeholders / projects
  • Catalog: Up-to-date AI sustainability best-practices in a company-wide repository.
  • Coach: new members / teams on best-practices as they evolve.

Research