Mpsify: runtime patcher to run CUDA scripts on Apple MPS
Mpsify is a Python package that patches PyTorch at import time to transparently redirect CUDA calls to Apple's Metal Performance Shaders (MPS). It solves the problem of editing CUDA scripts to run on Apple Silicon Macs, automatically remapping .cuda(), device='cuda', and map_location='cuda' to MPS equivalents. The tool is for ML practitioners who want to run training scripts or HuggingFace repos on M2 Macs without manual code changes.
Entities
Related
Local image-to-3D model for Apple Silicon and iPhone
A port of Hunyuan3D-Paint and Hunyuan3D-Shape to Swift MLX and Python MLX, enabling image-to-3D generation on Apple Silicon Macs and iPhones with low memory usage. It solves the problem of running 3D generation locally without cloud dependencies or high-end GPUs.
Turboquant v0.3.0 fixes silent FP16 precision bug on Tesla P100 in llama.cpp
A three-line fix in turboquant v0.3.0 corrects a long-standing bug where llama.cpp's CUDA code forced FP16 math on Tesla P100 GPUs, despite the P100 having fast FP16 hardware. The fix restores correct precision and performance for P100 users running llama.cpp.
↑ Updated Sun, Jul 12, 2026, 05:41 AM — Turboquant v0.3.0 released with fix for P100 FP16 bug.
Ollama merges CUDA toolkit fix, JetPack fallback, and agent harness core
Ollama merged several pull requests on July 6, 2026, including fixes for CUDA toolkit lookup and parallelism, a fallback to standard CUDA when JetPack runner is absent, and the initial core of an agent harness. These changes improve GPU compatibility and lay groundwork for agent functionality.
↑ Updated Mon, Jul 6, 2026, 10:28 PM — Agent harness core merged alongside CUDA and JetPack fixes.
Rust+CUDA LLM inference engine for RTX 5090, optimized with NVFP4, MoE, and speculative decoding
A from-scratch LLM inference engine written in Rust and CUDA, specifically tuned for the RTX 5090 Laptop GPU (Blackwell sm_120a). It supports NVFP4 quantization, mixture-of-experts (MoE), and multi-token prediction (MTP) speculative decoding, achieving up to 1.6x speedup over llama.cpp on target models. Designed for developers who need maximum inference performance on a single high-end laptop GPU.
User reports Qwen3.6 35B-A3B model improves with Q8_0 CPU quantization
A user on Reddit reported that switching from Q4_K_M on GPU to Q8_0 on CPU significantly improved the performance of the Qwen3.6 35B-A3B model for a complex coding task. The user noted the model 'punches far above its weight' and found the quality gain worth the slowdown.
