llm-serve-dashboard: single-file live dashboard for llama.cpp/vLLM
A developer released llm-serve-dashboard, a single-file, dependency-free live dashboard for local LLM serving with llama.cpp and vLLM. It displays GPU utilization, per-model throughput, KV/context fill, and system stats via a green terminal-styled HTML page and a stdlib Python backend reading nvidia-smi and Prometheus metrics.
Entities
Related
Reame: CPU-first LLM inference server built on llama.cpp released
Reame is a new LLM inference server designed to run efficiently on cheap CPU hardware, including shared vCPUs, free tiers, and 2-core ARM boxes. Built on llama.cpp, it features disk KV cache, self-regulating speculation, generation archive, and interleaved multi-user support. The project emphasizes treating CPU hardware as a first-class citizen rather than a fallback.
LLM comparison dashboard for quality, latency, and cost
A dashboard that lets users test LLMs on their own data, comparing quality, latency, and cost side by side. It runs on Nebius Serverless and helps developers choose the best model for their specific use case rather than relying on leaderboards.
LLM hardware recipe database with filters and community usage tracking
A community-driven database that lists which LLM models run on which hardware, with performance details. Users can filter by hardware, submit new recipes, and mark which recipes they actively use to show popularity. It solves the problem of finding compatible model-hardware combinations for LLM deployment.
Community shares llama-server configs for 24GB GPUs
A Reddit thread collects proven llama-server startup configurations for 24GB VRAM GPUs (RTX 3090, 7900XTX, RTX 4090). Users are asked to share commands that maximize VRAM usage and provide at least 200,000 tokens KV cache, along with system RAM, OS, and CPU details.
llm-eyes tool lets any blind LLM see via tiny local VLM
A GitHub project called llm-eyes bolts a tiny vision-language model (Qwen3.5-0.8B) onto any machine as a local vision service, enabling text-only models like DeepSeek V4 Flash, Qwen, and Llama to caption images, watch webcams, or read video. It runs on Mac (MLX), PC+NVIDIA (llama.cpp), and DGX Spark, and exposes an OpenAI-compatible API.