Testing Qwen3.8-27B KV cache dtypes
Qwen3.8-27B FP16: Is there a quality difference between FP8 and FP16 for the KV cache?
Local LLMvLLMDeveloper toolsQwen3.8
Programming
Hardware
Physics
Field notes from computers, local AI, scientific software and the physics problems that send me back to the terminal.
Latest writing
Serving Qwen3.8-27B with vLLM on four RTX 3090s
The model, memory and gateway configuration behind the long-context language-model service running on mlrig.
Local AIQwen3.8vLLMLong contextInference
mlrig: six RTX 3090s under Proxmox
A technical overview of the GPU compute host used for local language models, numerical experiments and isolated rental workloads.
Local AIHomelabProxmoxGPU computingvLLM
Running Roo Code against local vLLM models
Notes from configuring Roo Code with locally hosted DeepSeek and GPT-OSS models through vLLM.
Local AIvLLMDeveloper tools
Active projects