Hugging Face integrates native-speed vLLM backend into Transformers library
Hugging Face has upgraded the vLLM pip package to support native-speed inference directly within the Transformers library. The update enables seamless use of vLLM as a backend for Transformer models, leveraging its optimized serving capabilities. This integration allows developers to run models at native speed without switching frameworks.
Entities
Related
Hugging Face releases transformers v5.13.1 patch to enable vLLM compatibility
Hugging Face released transformers v5.13.1, a patch focused on enabling compatibility with the latest vLLM release. The update includes defensive handling of legacy layer types, fixes for custom code with new linear layer names, and a fix for _LazyAutoMapping.register with string keys.
User benchmarks 4x RTX 5060 Ti with SGLang for Qwen3.6 27B, finds better concurrency than vLLM
A user shared benchmark results showing that SGLang handles higher concurrency better than vLLM when running Qwen3.6 27B (INT8 with bf16 KV cache) on a 4x RTX 5060 Ti (64GB VRAM) setup. The test achieved 200 successful requests at 8 concurrency over 348.87 seconds, processing 61,870 input tokens and generating 44,525 tokens. This provides a practical reference for others considering multi-GPU configurations with these consumer cards.
SkyPilot and Hugging Face launch zero-egress storage integration for multi-cloud AI workloads
SkyPilot and Hugging Face announced a joint integration that allows users to mount Hugging Face Hub repos (including Hugging Face Buckets) directly into SkyPilot jobs using an hf:// URL and HF_TOKEN. This enables running AI workloads—development, training, or serving—on any cloud GPU cluster while reading data from the Hub with zero egress fees. The move simplifies multi-cloud AI workflows by decoupling storage from compute.
Hugging Face introduces Kernels repository type with redesigned CLIs and security
Hugging Face announced a new 'kernel' repository type on the Hub, along with a major redesign of the project including improved security, revamped CLIs, expanded framework/backend coverage, and a foundation for agentic kernel development. The updates aim to better serve users with compute-specific needs.
Microsoft Foundry adds Hugging Face models to managed compute platform
Microsoft Foundry, a platform for building agentic AI applications, now includes Hugging Face models alongside offerings from Microsoft, OpenAI, Anthropic, Meta, Mistral, and DeepSeek. The platform provides a single endpoint and SDKs in Python, C#, JavaScript, and Java, with managed compute for deploying and scaling models. This expands model selection for developers using Foundry's multi-agent orchestration and memory features.