Gemma 4 inference in Godot via GDScript and Vulkan compute shaders
An experimental Godot 4.7 project that runs Gemma 4 inference entirely in GDScript and Vulkan compute shaders. It enables LLM inference within the Godot game engine, potentially for in-game AI or interactive experiences.
Visit project ↗Entities
Related
Rust+CUDA LLM inference engine for RTX 5090, optimized with NVFP4, MoE, and speculative decoding
A from-scratch LLM inference engine written in Rust and CUDA, specifically tuned for the RTX 5090 Laptop GPU (Blackwell sm_120a). It supports NVFP4 quantization, mixture-of-experts (MoE), and multi-token prediction (MTP) speculative decoding, achieving up to 1.6x speedup over llama.cpp on target models. Designed for developers who need maximum inference performance on a single high-end laptop GPU.
Community compares local LLMs for agentic workflows using tool-eval-bench
A GitHub user published an interactive comparison report evaluating local LLMs for agentic workflows, using the tool-eval-bench benchmark (84 scenarios, 16 categories, 8 trials). The report targets single DGX Spark or other 96-128GB rigs and covers multi-turn tool orchestration, function calling, and autonomous planning as exercised by Hermes Agent.
Community member extends Gemma 4 to 40.5B parameters via layer insertion
A Reddit user, TOTORONG, successfully extended Google's Gemma 4 31B model to 40.5B parameters by inserting additional layers, releasing the result as extGemma4-40_5B on Hugging Face. The project builds on previous attempts documented in earlier posts, overcoming challenges where new inserted layers initially degraded performance. This demonstrates community-driven model customization and fine-tuning techniques.
Ollama merges CUDA toolkit fix, JetPack fallback, and agent harness core
Ollama merged several pull requests on July 6, 2026, including fixes for CUDA toolkit lookup and parallelism, a fallback to standard CUDA when JetPack runner is absent, and the initial core of an agent harness. These changes improve GPU compatibility and lay groundwork for agent functionality.
↑ Updated Mon, Jul 6, 2026, 10:28 PM — Agent harness core merged alongside CUDA and JetPack fixes.
Reame: CPU-first LLM inference server built on llama.cpp released
Reame is a new LLM inference server designed to run efficiently on cheap CPU hardware, including shared vCPUs, free tiers, and 2-core ARM boxes. Built on llama.cpp, it features disk KV cache, self-regulating speculation, generation archive, and interleaved multi-user support. The project emphasizes treating CPU hardware as a first-class citizen rather than a fallback.