THE NEXT THING, IN PERSPECTIVEIndependent AI news

The latest AI news, in focus.

Official announcements and industry reporting. Follow the timeline of AI.

Today's news

News feed
AI NEWS × FOXAIFind your next step

From the latest news to tools that work for you.

Explore AI tools

Discover AI tools

Swipe to explore →
KiroCoding & development

Kiro is an AI engineering tool for developers that organizes requirements into specifications, architectures, and tasks, generates code via agents, and supports the entire workflow—from planning to code review—through property-based testing, execution memory, and integration with development tools.

AI programming tool分类推荐榜首Free entry availableView details
MemoAIMusic & audio

MemoAI is an AI-powered audio-to-text tool that supports localized processing, eliminating the need to upload data and ensuring privacy and security. It can efficiently transcribe YouTube videos, podcasts, and local audio/video files (such as MP4, MP3, AAC, M4A, etc.) into text, while offering core features like multilingual translation, speaker diarization, real-time subtitles, and floating notes—making it ideal for meeting minutes, interview transcription, content learning, and more.

AI AUDIO TOOL分类推荐榜首Pricing unverifiedView details
JAXModels & infrastructure

JAX is a Python numerical computing library designed for accelerators, specifically tailored for high-performance array computations and large-scale machine learning. It efficiently executes the same code across multiple hardware backends, including CPUs, GPUs, and TPUs.

AI DEVELOPMENT PLATFORM分类推荐榜首Pricing unverifiedView details
EvoMapAgents & automation

EvoMap is an infrastructure designed for the reuse of AI agent experiences. It organizes reusable strategies and repair assets with validation records via the GEP protocol, offering market- and quality-based ranking to help different agents discover, evaluate, and inherit existing capabilities.

分类推荐榜首Pricing unverifiedView details
Open DesignDesign & 3D

OpenDesign is an open-source design workspace that can connect to local programming agents, use built-in model keys, or leverage cloud-based models, enabling the creation and editing of prototypes, web pages, presentations, and graphics within a design system; local software is free, while model services incur fees.

AI design tools分类推荐榜首Free entry availableView details
AI ShortWriting & content

AIS ALERT MANAGEMENT AND SHARED PLATFORM, QUICK COMMAND FOR MULTIPLE SCENARIOS

AI HINT COMMAND分类推荐榜首Pricing unverifiedView details
IamIPIndustry applications

IamIP is a patent platform that provides AI-driven search, analysis, and monitoring tools to help users conduct patent research and management more efficiently.

待评分 · 分类列表推荐Pricing unverifiedView details
My Glow Up AILife & personal

My Glow Up AI provides free analysis based on selfies, without requiring registration, and can generate hairstyle recommendations and beauty plans that suit you.

待评分 · 分类列表推荐Pricing unverifiedView details
AssemblyAIMusic & audio

AssemblyAI is a speech AI platform for developers, offering pre-recorded, real-time, and short-form synchronous transcription APIs. It enables the building of voice-based applications by integrating capabilities such as speaker identification, emotion analysis, chapter and summary extraction, as well as speech agent functionality, dictation, and sensitive information processing.

AI AUDIO TOOL待评分 · 分类列表推荐Pricing unverifiedView details
ClipDropImages & editing

Clipdrop is a suite of AI-powered image creation and editing tools, covering background removal, local cleanup, image enlargement, upscaling, relighting, text removal, and background replacement. It allows users to process images directly in the browser and also provides developers with an API for integrating these capabilities into their applications.

常用AI图像工具待评分 · 分类列表推荐View details

AI, as it happens

2409 published stories
Inference systems & deployment · 213 storiesClear filters ×
2026-11-20
Follow the timeline
19:35

b12563

The update addresses a precision issue in CUDA's per-expert fallback path where the MUL_MAT_ID operation was not properly preserving precision due to an unset operation type. The fix sets the operation to GGML_OP_MUL_MAT and reverts an earlier change that forced fp32 output, ensuring consistent precision handling across quantized matrix multiplication paths. This resolves issues with overflow to infinity in fp16 when results exceed 65504, particularly affecting models using IQ1_M weights with large batches.

Inference systems & deploymentRead original
18:46

b12561

The update introduces a chunked GATED_DELTA_NET implementation for Vulkan, including shader additions, optimizations for RDNA4 and Ampere architectures, fixes for shared memory bank conflicts and NVIDIA shader compiler issues, and support for various subgroup sizes and backend configurations. It also enables the feature by default and adds diagnostic markers for device information.

Inference systems & deploymentRead original
14:38

b12558

A fix has been implemented to disable subgroup arithmetic in the Vulkan driver for Qualcomm's Adreno GPUs, addressing an issue where subgroupAdd() after divergent code returns incorrect sums. This correction resolves problems with the decode-step norm, preventing models from generating stray tokens after long prompts. Testing on Adreno 850 and X2-90 showed improved results for ADD_RMS_NORM and RMS_NORM_MUL_ADD operations, while performance on llama 3B Q4_0 remained unchanged.

Inference systems & deploymentRead original
07:12

b12556

A vulnerability in llama.cpp's HRM layer count validation allows integer overflow due to improper handling of uint32_t multiplication, leading to potential buffer overflows during tensor loading. The issue arises when the product of n_hrm_layers_per_stack, n_hrm_h_cycles, and (n_hrm_l_cycles + 1) is computed as a uint32_t and compared to block_count, enabling maliciously crafted inputs to pass validation despite exceeding the actual block count.

Inference systems & deploymentRead original
05:51

b12552

The update includes fixes for OpenVINO CI, optimizations for model recompiles in the prefill stage, improvements to output handling during model reuse, and support for f16 SCALE and f16/bf16 TRI operations. It also addresses issues with static-shape views in MoE layers, enhances SSM_SCAN build checks, and adds Falcon-H1 to the list of validated OpenVINO models with noted limitations.

Inference systems & deploymentRead original
01:46

b12549

The llama.cpp v1.25.49 release includes fixes for Inkling 2 attention mask handling, making the relative-position input a hybrid iswa input, enabling optional expert weight scale and final logit softcap, and improving KV rows mask generation for hybrid iswa memory. The update also standardizes code formatting in inkling2.cpp and ensures compatibility across multiple platforms including macOS, Linux, Windows, Android, and openEuler.

Inference systems & deploymentRead original
00:33

b12543

The update introduces a new tensor API conv_2d kernel for Metal, enabling split-K optimizations across all chips, including M5. It improves efficiency by utilizing tensor cores, reducing idle GPU time during long reductions. The kernel now splits reductions into partial sums stored in buffers, with a follow-up kernel summing them deterministically. Address math is optimized using 32-bit byte offsets, reducing computational overhead. A fallback to the generic kernel is implemented for images exceeding 4 GiB. Performance benchmarks show significant speed improvements on M5 Max devices, with reduced execution times across various convolution configurations.

Inference systems & deploymentRead original
2026-11-19
Follow the timeline
19:40

b12535

The Vulkan backend in llama.cpp has been updated to implement stream-k MUL_MAT, which splits data chunks of 256 K elements evenly across all SMs. Workgroups that finish a tile write to the destination, while others write to scratch, with a resolve shader similar to split-k. The feature is implemented for scalar+cm1+cm2 but currently enabled only for cm2. A simple heuristic is used for enabling on coopmat2 due to large tile size, though tuning for other tile sizes is not planned for NVIDIA devices.

Inference systems & deploymentRead original
06:56

b12527

The update rejects malformed GGUF files that could cause out-of-bounds memory access by validating the mudd_tap_idx array bounds during hparams loading, preventing potential exploits from crafted models with excessive n_layer * n_mudd_taps values exceeding 512.

Inference systems & deploymentRead original
2026-11-18
Follow the timeline
13:00

AInfer-PD: Communication-Safe In-Place Prefill-Decode Multiplexing for Distributed MoE Rollouts

arXiv:2609.00993v3 Announce Type: replace Abstract: In agentic reinforcement learning (RL), asynchronous multi-turn trajectories repeatedly introduce continuation prefills (P) while other trajectories remain in decode (D). Efficient rollout completion therefore requires D to advance during long P executions. In-place multiplexing shares devices, model weights, and KV state, but phase-specific communication in distributed MoE introduces two constraints: cross-rank collective ordering and ownership of mutable communication state. Whole-prefill coordination can ensure safe ordering but delays D. DeepEP's normal and low-latency paths suit large P and small D batches, respectively, but safely overlapping them requires separating their mutable communication state. We present AInfer-PD, which extends in-place P/D multiplexing to distributed MoE rollouts. AInfer-PD establishes rank-consistent collective ordering at safe communication boundaries, allowing successive D iterations to advance without waiting for an entire prefill execution to be enqueued or completed. It gives P and D separate communication buffers and protocol state within one DeepEP runtime, allowing P to use the normal path while D retains the low-latency path. Together, these mechanisms preserve useful P/D overlap while sharing model weights and KV storage on the same devices. Across repeated single-node prefill-intensive workloads, AInfer-PD reduces fixed-workload rollout completion time by 7.1-22.5% relative to the same AInfer engine with P/D multiplexing disabled and by 24.8-32.9% relative to SGLang. On two nodes, the reductions are 18.0-35.3% and 18.3-31.8%, respectively. In a same-engine ablation, fine-grained boundaries reduce completion time by a further 8.6-19.8% over whole-prefill asynchronous enqueue.

Inference systems & deploymentRead original
13:00

WM-Quant: Rollout-Aware Post-Training Quantization for JEPA-Style World Models

arXiv:2611.21882v1 Announce Type: new Abstract: We present WM-Quant, to our knowledge, the first post-training quantization (PTQ) framework designed specifically for JEPA-style world models. Although world models typically feature relatively few parameters, action planning algorithms often require thousands of inferences, which can quickly become computationally expensive. PTQ is a natural way to reduce this cost. However, we show that in world models using autoregressive model predictive control (MPC) rollouts, quantization error can compound with each predictor step, resulting in action rankings from the quantized model becoming near-random. WM-Quant addresses this with three components that build on existing PTQ methods: early-encoder precision, control-path preservation, and rollout-aware scaling. Early-encoder precision protects accurate latent state and goal information to mitigate long-term error propagation. Control-path preservation allows the world model predictor to distinguish between fine-grained actions, significantly improving action rankings and execution. Rollout-aware scaling jointly calibrates encoder and per-step predictor activation ranges with respect to an entire rollout rather than a single prediction, reducing the cumulative rollout error. Across multiple world models, environments, and PTQ methods, WM-Quant substantially improves planning success at low bit-widths, approaching full-precision performance at W4A4.

Inference systems & deploymentRead original
10:21

b12493

The update enables the fast implicit GEMM path for F32 kernels in CUDA, previously limited to F16, by converting input data to F16 in a pool buffer and using F32 accumulation, significantly improving performance for large channel counts.

Inference systems & deploymentRead original
08:49

b12489

A fix has been implemented to return a 400 error when both grammar and ignore_eos parameters are enabled in the server, preventing crashes and improving validation by moving the check into the sampler. Redundant tests have also been removed.

Inference systems & deploymentRead original
03:11

b12480

The Llama UI design system, including tokens, surfaces, fonts, and base components, has been moved from a temporary repository to ui/lib/ui, where it is now published as @ggml-org/llama-ui. Llama Chat now imports all UI elements—including model components, constants, enums, and helpers—from this centralized library, eliminating duplicate code. The update includes updated branding with Llama Sans and Llama Mono fonts, improved submenu rendering, enhanced elevation for chat form elements, and integration with the llama-brand package for colors and logo. The UI is now consistently applied across the application, with builds and deployment configurations adjusted accordingly.

Inference systems & deploymentRead original
2026-11-17
Follow the timeline
04:44

b12445

The update introduces support for split_k in conv3d operations using Vulkan, leveraging a 256x128 shape (coopmat2-only) when beneficial and reusing the mul_mat split_k reduce shader.

Inference systems & deploymentRead original
Every update links back to its source.
STAY CURIOUS. STAY INFORMED.

Stay informed about the changes that matter.

Give your curiosity direction. Make every day of reading count.

Subscribe via RSS