RightNow AI releases 'Auto' compiler that records LLM agent behavior into verified WASM binaries
RightNow AI introduced Auto, a compiler that records LLM agent behavior, identifies repeatable patterns, and compiles them into verified, sandboxed WebAssembly binaries that run for microdollars. The system includes a tiered runtime that falls back to frontier models for novelty and recompiles results, aiming to eliminate re-solving the same tasks. A paper detailing the approach is available on arXiv.
Entities
Related
AgentMaker: a new Python framework for building LLM agents and multi-agent systems
AgentMaker is a general-purpose Python framework for building LLM agents and multi-agent systems, featuring tools, memory, RAG, context engineering, guardrails, human-in-the-loop, and observability. It is released under MIT license on GitHub and PyPI.
Reame: CPU-first LLM inference server built on llama.cpp released
Reame is a new LLM inference server designed to run efficiently on cheap CPU hardware, including shared vCPUs, free tiers, and 2-core ARM boxes. Built on llama.cpp, it features disk KV cache, self-regulating speculation, generation archive, and interleaved multi-user support. The project emphasizes treating CPU hardware as a first-class citizen rather than a fallback.
Rust+CUDA LLM inference engine for RTX 5090, optimized with NVFP4, MoE, and speculative decoding
A from-scratch LLM inference engine written in Rust and CUDA, specifically tuned for the RTX 5090 Laptop GPU (Blackwell sm_120a). It supports NVFP4 quantization, mixture-of-experts (MoE), and multi-token prediction (MTP) speculative decoding, achieving up to 1.6x speedup over llama.cpp on target models. Designed for developers who need maximum inference performance on a single high-end laptop GPU.
LLM hardware recipe database with filters and community usage tracking
A community-driven database that lists which LLM models run on which hardware, with performance details. Users can filter by hardware, submit new recipes, and mark which recipes they actively use to show popularity. It solves the problem of finding compatible model-hardware combinations for LLM deployment.
Software engineer publishes final part of LLM-from-scratch series covering inference and decoding
A software engineer published the fourth and final part of a blog series explaining LLMs from the ground up, focusing on token-by-token generation, KV cache, and decoding strategies (temperature, top-k, top-p). The series aims to help other software engineers understand the internals of LLMs.