jerome
05eccd5b53
- Moved scattered scripts, templates, and documentation into organized directories (applications/, scripts/, assets/). - Updated .gitignore to strictly exclude secrets, state files, and IDE configs. - Added comprehensive README.md outlining repository structure and best practices. - Preserved all existing documentation and technical architecture files. - Prepared infrastructure/ for AWS Org and Proxmox Terraform management.
1.5 KiB
1|# 04 - Qwen3.5 Model Setup — Home
2|Purpose: Installation and configuration guides for Qwen3.5 model variants.
3|
4|## Available Models
5|
6|### Qwen3.5-MoE (Mixture of Experts) — Optimized for production inference with dynamic expert routing.
7|
8|### Qwen3.5-Base — Standard dense model variant.
9|
10|### Qwen3.5-FineTuned — Domain-specific fine-tuned variants.
11|
12|## GPU Compatibility
13|
14|- AMD Strix Halo — See [AMD GPU Passthrough Guide](../03\ -\ AMD\ GPU\ Passthrough/00\ -\ AMD\ GPU\ Passthrough.md)
15|- NVIDIA RTX 4090 — Standard CUDA setup
16|- Multi-GPU Clusters — Distributed inference setup
17|
18|## Quick Start
19|
20|### Install Qwen3.5-MoE via Ollama
21|bash 22|ollama pull qwen3.5:250b-moe 23|ollama serve 24| 25|# Run locally at 192.168.68.8:8080 26|ollama run qwen3.5:250b-moe "What are the top AI trends for 2026?" 27|
28|
29|### Advanced Configuration
30|
31|For production deployments with dynamic expert selection:
32|- See /60\ -\ Syslog/02\ -\ TECHNICAL\ INFRASTRUCTURE/04\ -\ Qwen3.5\ Model\ Setup/01\ -\ Qwen3.5-MoE\ Configuration.md
33|- GPU passthrough: /60\ -\ Syslog/02\ -\ TECHNICAL\ INFRASTRUCTURE/03\ -\ AMD\ GPU\ Passthrough/00\ -\ AMD\ GPU\ Passthrough.md
34|
35|## Scripts
36|
37|- vulkan-qwen35-moe-setup.sh — Complete setup script for Qwen3.5-MoE with Vulkan backend
38|- qwen35-moe-inference-config.sh — Production inference configuration script
39|
40|---
41|
42|For AMD GPU-specific setup, refer to the AMD GPU Passthrough guide.
43|