jerome
05eccd5b53
- Moved scattered scripts, templates, and documentation into organized directories (applications/, scripts/, assets/). - Updated .gitignore to strictly exclude secrets, state files, and IDE configs. - Added comprehensive README.md outlining repository structure and best practices. - Preserved all existing documentation and technical architecture files. - Prepared infrastructure/ for AWS Org and Proxmox Terraform management.
90 lines
2.3 KiB
Markdown
90 lines
2.3 KiB
Markdown
# 02 - TECHNICAL INFRASTRUCTURE — Overview
|
|
**Purpose:** Complete technical guide for Syslog infrastructure, GPU passthrough, and model deployment.
|
|
|
|
## Infrastructure Components
|
|
|
|
### **01 - Proxmox & VM Passthrough**
|
|
- GPU passthrough configuration (AMD/Intel)
|
|
- VM networking setup
|
|
- Resource allocation best practices
|
|
|
|
### **02 - GPU Monitoring**
|
|
- Grafana dashboards for GPU telemetry
|
|
- ROCm monitoring setup
|
|
- Real-time performance tracking
|
|
|
|
### **03 - AMD GPU Passthrough (Strix Halo)**
|
|
- Detailed AMD APU passthrough for host LLM access
|
|
- Vulkan driver configuration
|
|
- HIP SDK integration
|
|
|
|
### **04 - Qwen3.5 Model Setup**
|
|
- Qwen3.5-MoE (Mixture of Experts) installation
|
|
- Vulkan backend optimization
|
|
- Production inference configuration
|
|
|
|
### **05 - AWS Cloud Infrastructure**
|
|
|
|
### **06 - Troubleshooting**
|
|
- Common issues and resolutions
|
|
- Debugging methodology
|
|
- Emergency procedures
|
|
- VPC and networking architecture
|
|
- EC2 instance configuration for AI workloads
|
|
- S3 data pipeline setup
|
|
- Cost optimization strategies
|
|
|
|
## Deployment Standards
|
|
|
|
- **GPU Access:** AMD Strix Halo for host inference, NVIDIA for training
|
|
- **Model Hosting:** Ollama local deployment @ 192.168.68.8:8080
|
|
- **Cloud:** AWS for scalable cloud workloads
|
|
- **Storage:** S3 for model weights and datasets
|
|
- **Monitoring:** Grafana + Prometheus for observability
|
|
|
|
## Common Operations
|
|
|
|
### Start Qwen3.5-MoE
|
|
```bash
|
|
ollama pull qwen3.5:250b-moe
|
|
ollama serve --model qwen3.5:250b-moe
|
|
```
|
|
|
|
### Check GPU Status
|
|
```bash
|
|
# AMD
|
|
rocm-smi
|
|
watch -n 5 rocm-smi
|
|
|
|
# NVIDIA
|
|
nvidia-smi
|
|
watch -n 5 nvidia-smi
|
|
```
|
|
|
|
### Deploy New Agent
|
|
```bash
|
|
# Clone agent repository
|
|
git clone https://github.com/sysloggh/agent-template.git
|
|
cd agent-template
|
|
|
|
# Configure environment
|
|
cp .env.example .env
|
|
# Edit .env with API keys and settings
|
|
|
|
# Install dependencies
|
|
pip install -r requirements.txt
|
|
|
|
# Deploy
|
|
source venv/bin/activate
|
|
python main.py
|
|
```
|
|
|
|
## Related Documentation
|
|
|
|
- [Business Strategy](../01\ -\ BUSINESS\ \&\ STRATEGY/00\ -\ Business Strategy Home.md)
|
|
- [AI Agents Framework](../03\ -\ AI\ AGENTS\ \&\ LEARNING/00\ -\ AI Agents Learning Home.md)
|
|
- [Operational Excellence](../05\ -\ OPERATIONAL\ EXCELLENCE/00\ -\ Operational Excellence Home.md)
|
|
|
|
---
|
|
|
|
*This section pairs implementation guides with their respective scripts for rapid deployment.* |