Most data teams track Accuracy, Latency, and maybe GPU Utilization if someone is watching the dashboard. Almost no one tracks:
- How many GPU-hours a model run consumed
- How many kWh of electricity that implies
- How much CO₂ and cloud spend are associated with each experiment
Once I started paying attention to these metrics, it completely changed how I design and run experiments.