Run Gemma 4 LLM inside Godot using GDScript and Vulkan compute shaders
A proof-of-concept project that runs the Gemma 4 LLM directly inside the Godot game engine without external dependencies like llama.cpp or Python. It uses Vulkan compute shaders for model inference and GDScript for loading, tokenization, and UI. This enables local LLM inference within a Godot game or application, albeit with slower performance than dedicated solutions.
Visit project ↗Entities
Related
Gemma 4 inference in Godot via GDScript and Vulkan compute shaders
An experimental Godot 4.7 project that runs Gemma 4 inference entirely in GDScript and Vulkan compute shaders. It enables LLM inference within the Godot game engine, potentially for in-game AI or interactive experiences.
Autonomous NPCs powered by Gemma 4 E2B in the browser
An experiment that runs small language models (Gemma 4 via E2B) locally in the browser to control autonomous NPCs. The NPCs are intentionally 'dumb' to create humorous interactions, targeting developers and hobbyists exploring local AI gaming.
Rust+CUDA LLM inference engine for RTX 5090, optimized with NVFP4, MoE, and speculative decoding
A from-scratch LLM inference engine written in Rust and CUDA, specifically tuned for the RTX 5090 Laptop GPU (Blackwell sm_120a). It supports NVFP4 quantization, mixture-of-experts (MoE), and multi-token prediction (MTP) speculative decoding, achieving up to 1.6x speedup over llama.cpp on target models. Designed for developers who need maximum inference performance on a single high-end laptop GPU.
llm-serve-dashboard: single-file live dashboard for llama.cpp/vLLM
A developer released llm-serve-dashboard, a single-file, dependency-free live dashboard for local LLM serving with llama.cpp and vLLM. It displays GPU utilization, per-model throughput, KV/context fill, and system stats via a green terminal-styled HTML page and a stdlib Python backend reading nvidia-smi and Prometheus metrics.
GGUFun: play Snake and maze on Ollama with hand-crafted GGUF models
GGUFun is a project that creates hand-crafted GGUF model files to run deterministic games like Snake and a simple maze on Ollama. The models are built manually without training, using custom weights to compute game logic and respond with fixed outputs. It demonstrates that LLM inference engines can be repurposed for non-language tasks.