If you're running a GPU-equipped VPS for AI workloads, monitoring GPU utilization, memory, and temperature is essential for understanding performance and catching problems — this guide covers the standard tools.
The Core Tool: nvidia-smi
nvidia-smiShows current GPU utilization, memory usage, temperature, and running processes using the GPU — the first place to check for any GPU-related question.
Continuous Monitoring (Live Updates)
watch -n 1 nvidia-smiRefreshes the display every second, useful for watching utilization change in real time during a training run or inference workload.
Understanding the Key Fields
| Field | Meaning |
|---|---|
| GPU-Util | Percentage of GPU compute capacity currently in use |
| Memory-Usage | How much GPU VRAM is allocated vs total available |
| Temp | GPU temperature — sustained high values may indicate thermal throttling |
| Processes | Which processes are currently using the GPU and how much memory each consumes |
Logging GPU Metrics Over Time
nvidia-smi --query-gpu=timestamp,utilization.gpu,memory.used,temperature.gpu --format=csv -l 5 >> gpu-log.csvLogs key metrics every 5 seconds to a CSV file, useful for reviewing performance patterns after a long-running job completes.
Checking Which Process Is Using GPU Memory
nvidia-smi --query-compute-apps=pid,used_memory --format=csvIntegrating GPU Metrics into Prometheus/Grafana
The NVIDIA GPU Exporter (a community-maintained tool) exposes GPU metrics in Prometheus format, letting you build proper dashboards and alerts alongside your existing server monitoring — see How to Set Up Prometheus and Grafana for VPS Monitoring for the base monitoring stack this integrates with.
Common Issues to Watch For
- GPU memory fully allocated but utilization low — often indicates a memory leak or an application holding memory without doing useful compute work
- Consistently high temperature — may indicate inadequate cooling for the workload; sustained thermal issues can cause automatic throttling, reducing performance
- Multiple processes competing for the same GPU — can cause unpredictable performance for each individual workload; consider whether workloads should be scheduled sequentially instead
Checking Driver and CUDA Version Compatibility
nvidia-smi | head -n 5Shows the installed driver version and maximum supported CUDA version — verify your AI framework's CUDA requirements are compatible before troubleshooting unrelated issues.
Common Errors
"nvidia-smi: command not found" — NVIDIA drivers aren't installed, or this VPS doesn't actually have GPU access; verify your plan includes GPU hardware and drivers are properly installed.
GPU shows 0% utilization despite a running AI workload — the application may be running on CPU instead of GPU due to a configuration issue; verify the framework detects the GPU correctly (e.g. torch.cuda.is_available() for PyTorch).
Continue Reading
- GPU vs CPU VPS: What You Actually Need for AI Workloads
- How to Set Up Prometheus and Grafana for VPS Monitoring
- How to Install PyTorch and TensorFlow on a VPS
Browse more articles in AI & Machine Learning on a VPS.
