Reducing Rollout Bottlenecks in Reinforcement Learning with NVIDIA Dynamo
Reinforcement learning (RL) uses rewards to train models to solve problems and complete tasks more successfully. Inference drives this process. Workers generate...
Official announcements and industry reporting. Follow the timeline of AI.
From the latest news to tools that work for you.
EvoMap is an infrastructure designed for the reuse of AI agent experiences. It organizes reusable strategies and repair assets with validation records via the GEP protocol, offering market- and quality-based ranking to help different agents discover, evaluate, and inherit existing capabilities.
分类推荐榜首M
MemoAIMusic & audioMemoAI is an AI-powered audio-to-text tool that supports localized processing, eliminating the need to upload data and ensuring privacy and security. It can efficiently transcribe YouTube videos, podcasts, and local audio/video files (such as MP4, MP3, AAC, M4A, etc.) into text, while offering core features like multilingual translation, speaker diarization, real-time subtitles, and floating notes—making it ideal for meeting minutes, interview transcription, content learning, and more.
分类推荐榜首A
AI ShortWriting & contentAIS ALERT MANAGEMENT AND SHARED PLATFORM, QUICK COMMAND FOR MULTIPLE SCENARIOS
分类推荐榜首KKiro is an AI engineering tool for developers that organizes requirements into specifications, architectures, and tasks, generates code via agents, and supports the entire workflow—from planning to code review—through property-based testing, execution memory, and integration with development tools.
分类推荐榜首OOpenDesign is an open-source design workspace that can connect to local programming agents, use built-in model keys, or leverage cloud-based models, enabling the creation and editing of prototypes, web pages, presentations, and graphics within a design system; local software is free, while model services incur fees.
分类推荐榜首JJAX is a Python numerical computing library designed for accelerators, specifically tailored for high-performance array computations and large-scale machine learning. It efficiently executes the same code across multiple hardware backends, including CPUs, GPUs, and TPUs.
分类推荐榜首E
ElicitKnowledge & researchElicit provides semantic search, research report generation, literature screening, and data extraction for scientific research. It enables traceable answers within source materials and organizes evidence through a literature library, research agents, and ongoing tracking workflows, making it ideal for advancing from initial problem exploration to systematic reviews.
待评分 · 分类列表推荐闪FlashCut is an AI-powered creation platform for voice-over and short-video marketing. It can generate finished videos with voiceovers, subtitles, and visuals from text, offering digital human customization, photo-to-digital-human conversion, live-stream clipping, and intelligent editing. It also provides APIs and branded mini-program solutions for businesses.
待评分 · 分类列表推荐AAssemblyAI is a speech AI platform for developers, offering pre-recorded, real-time, and short-form synchronous transcription APIs. It enables the building of voice-based applications by integrating capabilities such as speaker identification, emotion analysis, chapter and summary extraction, as well as speech agent functionality, dictation, and sensitive information processing.
待评分 · 分类列表推荐妙THE AI LIVE TELECOM SOLUTION FROM THE AD
待评分 · 分类列表推荐Reinforcement learning (RL) uses rewards to train models to solve problems and complete tasks more successfully. Inference drives this process. Workers generate...
The llama.cpp v1.25.49 release includes fixes for Inkling 2 attention mask handling, making the relative-position input a hybrid iswa input, enabling optional expert weight scale and final logit softcap, and improving KV rows mask generation for hybrid iswa memory. The update also standardizes code formatting in inkling2.cpp and ensures compatibility across multiple platforms including macOS, Linux, Windows, Android, and openEuler.
The update introduces a new tensor API conv_2d kernel for Metal, enabling split-K optimizations across all chips, including M5. It improves efficiency by utilizing tensor cores, reducing idle GPU time during long reductions. The kernel now splits reductions into partial sums stored in buffers, with a follow-up kernel summing them deterministically. Address math is optimized using 32-bit byte offsets, reducing computational overhead. A fallback to the generic kernel is implemented for images exceeding 4 GiB. Performance benchmarks show significant speed improvements on M5 Max devices, with reduced execution times across various convolution configurations.
A tool call can arrive in pieces, an argument can be cut short, or model-specific markup can leak into the answer. A frontend parser turns that output, received...
The Vulkan backend in llama.cpp has been updated to implement stream-k MUL_MAT, which splits data chunks of 256 K elements evenly across all SMs. Workgroups that finish a tile write to the destination, while others write to scratch, with a resolve shader similar to split-k. The feature is implemented for scalar+cm1+cm2 but currently enabled only for cm2. A simple heuristic is used for enabling on coopmat2 due to large tile size, though tuning for other tile sizes is not planned for NVIDIA devices.
The update rejects malformed GGUF files that could cause out-of-bounds memory access by validating the mudd_tap_idx array bounds during hparams loading, preventing potential exploits from crafted models with excessive n_layer * n_mudd_taps values exceeding 512.
arXiv:2609.00993v3 Announce Type: replace Abstract: In agentic reinforcement learning (RL), asynchronous multi-turn trajectories repeatedly introduce continuation prefills (P) while other trajectories remain in decode (D). Efficient rollout completion therefore requires D to advance during long P executions. In-place multiplexing shares devices, model weights, and KV state, but phase-specific communication in distributed MoE introduces two constraints: cross-rank collective ordering and ownership of mutable communication state. Whole-prefill coordination can ensure safe ordering but delays D. DeepEP's normal and low-latency paths suit large P and small D batches, respectively, but safely overlapping them requires separating their mutable communication state. We present AInfer-PD, which extends in-place P/D multiplexing to distributed MoE rollouts. AInfer-PD establishes rank-consistent collective ordering at safe communication boundaries, allowing successive D iterations to advance without waiting for an entire prefill execution to be enqueued or completed. It gives P and D separate communication buffers and protocol state within one DeepEP runtime, allowing P to use the normal path while D retains the low-latency path. Together, these mechanisms preserve useful P/D overlap while sharing model weights and KV storage on the same devices. Across repeated single-node prefill-intensive workloads, AInfer-PD reduces fixed-workload rollout completion time by 7.1-22.5% relative to the same AInfer engine with P/D multiplexing disabled and by 24.8-32.9% relative to SGLang. On two nodes, the reductions are 18.0-35.3% and 18.3-31.8%, respectively. In a same-engine ablation, fine-grained boundaries reduce completion time by a further 8.6-19.8% over whole-prefill asynchronous enqueue.
arXiv:2611.21882v1 Announce Type: new Abstract: We present WM-Quant, to our knowledge, the first post-training quantization (PTQ) framework designed specifically for JEPA-style world models. Although world models typically feature relatively few parameters, action planning algorithms often require thousands of inferences, which can quickly become computationally expensive. PTQ is a natural way to reduce this cost. However, we show that in world models using autoregressive model predictive control (MPC) rollouts, quantization error can compound with each predictor step, resulting in action rankings from the quantized model becoming near-random. WM-Quant addresses this with three components that build on existing PTQ methods: early-encoder precision, control-path preservation, and rollout-aware scaling. Early-encoder precision protects accurate latent state and goal information to mitigate long-term error propagation. Control-path preservation allows the world model predictor to distinguish between fine-grained actions, significantly improving action rankings and execution. Rollout-aware scaling jointly calibrates encoder and per-step predictor activation ranges with respect to an entire rollout rather than a single prediction, reducing the cumulative rollout error. Across multiple world models, environments, and PTQ methods, WM-Quant substantially improves planning success at low bit-widths, approaching full-precision performance at W4A4.
The update enables the fast implicit GEMM path for F32 kernels in CUDA, previously limited to F16, by converting input data to F16 in a pool buffer and using F32 accumulation, significantly improving performance for large channel counts.
A fix has been implemented to return a 400 error when both grammar and ignore_eos parameters are enabled in the server, preventing crashes and improving validation by moving the check into the sampler. Redundant tests have also been removed.
How FlyDSL MonoKernels keep agents responsive with high-interactivity decode in vLLM on AMD Instinct GPUs, using model-specific fusion, data-ready handoffs, and specialized-decode integration.
The Llama UI design system, including tokens, surfaces, fonts, and base components, has been moved from a temporary repository to ui/lib/ui, where it is now published as @ggml-org/llama-ui. Llama Chat now imports all UI elements—including model components, constants, enums, and helpers—from this centralized library, eliminating duplicate code. The update includes updated branding with Llama Sans and Llama Mono fonts, improved submenu rendering, enhanced elevation for chat form elements, and integration with the llama-brand package for colors and logo. The UI is now consistently applied across the application, with builds and deployment configurations adjusted accordingly.
World model accuracy starts with high-quality, diverse training datasets. Yet capturing rare defects, nighttime incidents, and other challenging scenarios is...
The Llama Chat UI has been upgraded to SvelteKit 3, integrating its built-in service worker for improved update detection and offline functionality. The migration includes updates to Vite 8, TypeScript 6, Storybook 10, and ESLint 10, along with the replacement of deprecated packages and resolution of npm audit issues through dependency overrides, including pinning minimatch to version 11.0.5 to address a security vulnerability. Changes also align with current Svelte patterns, such as using $state instead of stores and adopting new syntax for actions and class bindings.
The llama.cpp release b12474 introduces the application of output projection weight scale and provides precompiled binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, supporting various backends such as CUDA, Vulkan, ROCm, OpenVINO, and SYCL.
The update addresses three issues in the OpenCL implementation of the bin flash-attention prefill path: releasing a previously unreleased WMM flash-attention mask fallback image, excluding q8_0 K/V at head size 512 from the WMM kernel due to incorrect results, and preventing the use of the WMM path beyond device 3D image limits to avoid clCreateImage failures.
The web UI for Llama.cpp has been relocated from tools/ui to ui/apps/chat as part of a structural refactoring to separate the component library (ui/lib) from the applications built on it. The app is now officially named Llama Chat, with its package renamed to @ggml-org/llama-chat, while maintaining private status. Documentation, help text, comments, and other references have been updated to reflect the new name, though legacy identifiers remain unchanged for backward compatibility.
Agentic workloads, long documents, and large codebases are pushing context lengths from tens of thousands of tokens to hundreds of thousands. At those lengths,...
Quantum processors are being integrated into supercomputing centers worldwide. National programs in North America, Europe, and Asia are driving the installation...
The update introduces support for split_k in conv3d operations using Vulkan, leveraging a 256x128 shape (coopmat2-only) when beneficial and reusing the mul_mat split_k reduce shader.
Give your curiosity direction. Make every day of reading count.