jerome
05eccd5b53
- Moved scattered scripts, templates, and documentation into organized directories (applications/, scripts/, assets/). - Updated .gitignore to strictly exclude secrets, state files, and IDE configs. - Added comprehensive README.md outlining repository structure and best practices. - Preserved all existing documentation and technical architecture files. - Prepared infrastructure/ for AWS Org and Proxmox Terraform management.
51 lines
1.3 KiB
Markdown
51 lines
1.3 KiB
Markdown
1|# 02 - GPU Monitoring — Home
|
|
2|**Purpose:** Grafana dashboards and monitoring setup for GPU telemetry.
|
|
3|
|
|
4|## Contents
|
|
5|
|
|
6|This directory contains:
|
|
7|- Grafana dashboard configuration
|
|
8|- ROCm monitoring setup
|
|
9|- Real-time performance tracking
|
|
10|
|
|
11|### Monitoring Features
|
|
12|
|
|
13|- **Real-time GPU utilization:** Track VRAM, compute, and memory usage
|
|
14|- **Temperature monitoring:** Alert on overheating conditions
|
|
15|- **Performance metrics:** Core clock, power consumption, fan speeds
|
|
16|- **Historical data:** Grafana time-series graphs for trend analysis
|
|
17|
|
|
18|## Quick Start
|
|
19|
|
|
20|### Grafana Dashboard Setup
|
|
21|
|
|
22|```bash
|
|
23|# Install Grafana
|
|
24|sudo apt install grafana
|
|
25|
|
|
26|# Start Grafana service
|
|
27|systemctl start grafana-server
|
|
28|
|
|
29|# Access dashboard
|
|
30|# http://localhost:3000 (admin/admin)
|
|
31|```
|
|
32|
|
|
33|### ROCm Monitoring
|
|
34|
|
|
35|```bash
|
|
36|# Install ROCm health monitoring tools
|
|
37|sudo apt install rocminfo rocm-smi
|
|
38|
|
|
39|# Real-time monitoring
|
|
40|watch -n 1 rocm-smi
|
|
41|```
|
|
42|
|
|
43|## Related Infrastructure
|
|
44|
|
|
45|- **[AMD GPU Passthrough](../03\ -\ AMD\ GPU\ Passthrough/)** — Hardware configuration
|
|
46|- **[Qwen3.5 Model Setup](../04\ -\ Qwen3.5\ Model\ Setup/)** — Model deployment
|
|
47|
|
|
48|---
|
|
49|
|
|
50|*Monitor your GPUs 24/7 to ensure optimal performance for LLM inference workloads.*
|
|
51| |