THE NEXT THING, IN PERSPECTIVEIndependent AI news

The latest AI news, in focus.

Official announcements and industry reporting. Follow the timeline of AI.

Today's news

News feed
AI NEWS × FOXAIFind your next step

From the latest news to tools that work for you.

Explore AI tools

Discover AI tools

Swipe to explore →
KiroCoding & development

Kiro is an AI engineering tool for developers that organizes requirements into specifications, architectures, and tasks, generates code via agents, and supports the entire workflow—from planning to code review—through property-based testing, execution memory, and integration with development tools.

AI programming tool分类推荐榜首Free entry availableView details
JAXModels & infrastructure

JAX is a Python numerical computing library designed for accelerators, specifically tailored for high-performance array computations and large-scale machine learning. It efficiently executes the same code across multiple hardware backends, including CPUs, GPUs, and TPUs.

AI DEVELOPMENT PLATFORM分类推荐榜首Pricing unverifiedView details
Open DesignDesign & 3D

OpenDesign is an open-source design workspace that can connect to local programming agents, use built-in model keys, or leverage cloud-based models, enabling the creation and editing of prototypes, web pages, presentations, and graphics within a design system; local software is free, while model services incur fees.

AI design tools分类推荐榜首Free entry availableView details
MemoAIMusic & audio

MemoAI is an AI-powered audio-to-text tool that supports localized processing, eliminating the need to upload data and ensuring privacy and security. It can efficiently transcribe YouTube videos, podcasts, and local audio/video files (such as MP4, MP3, AAC, M4A, etc.) into text, while offering core features like multilingual translation, speaker diarization, real-time subtitles, and floating notes—making it ideal for meeting minutes, interview transcription, content learning, and more.

AI AUDIO TOOL分类推荐榜首Pricing unverifiedView details
EvoMapAgents & automation

EvoMap is an infrastructure designed for the reuse of AI agent experiences. It organizes reusable strategies and repair assets with validation records via the GEP protocol, offering market- and quality-based ranking to help different agents discover, evaluate, and inherit existing capabilities.

分类推荐榜首Pricing unverifiedView details
AI ShortWriting & content

AIS ALERT MANAGEMENT AND SHARED PLATFORM, QUICK COMMAND FOR MULTIPLE SCENARIOS

AI HINT COMMAND分类推荐榜首Pricing unverifiedView details
IamIPIndustry applications

IamIP is a patent platform that provides AI-driven search, analysis, and monitoring tools to help users conduct patent research and management more efficiently.

待评分 · 分类列表推荐Pricing unverifiedView details
闪剪Video & animation

FlashCut is an AI-powered creation platform for voice-over and short-video marketing. It can generate finished videos with voiceovers, subtitles, and visuals from text, offering digital human customization, photo-to-digital-human conversion, live-stream clipping, and intelligent editing. It also provides APIs and branded mini-program solutions for businesses.

待评分 · 分类列表推荐Pricing unverifiedView details
妙播Marketing & commerce

THE AI LIVE TELECOM SOLUTION FROM THE AD

待评分 · 分类列表推荐Pricing unverifiedView details
RowsData & analytics

Rows is a spreadsheet tool that combines AI analytics, data integration, and interactive reporting, enabling you to organize documents and business data using natural language, merge datasets, generate reports, and build computational models. The official website currently indicates that it has been integrated with Superhuman and provides a corresponding link to continue using the service.

DATA PROCESSING FOR AI FORMS待评分 · 分类列表推荐Pricing unverifiedView details

AI, as it happens

2122 published stories
Compute & infrastructure · 294 storiesClear filters ×
2026-11-17
Follow the timeline
21:43

b12475

The Llama Chat UI has been upgraded to SvelteKit 3, integrating its built-in service worker for improved update detection and offline functionality. The migration includes updates to Vite 8, TypeScript 6, Storybook 10, and ESLint 10, along with the replacement of deprecated packages and resolution of npm audit issues through dependency overrides, including pinning minimatch to version 11.0.5 to address a security vulnerability. Changes also align with current Svelte patterns, such as using $state instead of stores and adopting new syntax for actions and class bindings.

Inference systems & deploymentRead original
21:25

b12474

The llama.cpp release b12474 introduces the application of output projection weight scale and provides precompiled binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, supporting various backends such as CUDA, Vulkan, ROCm, OpenVINO, and SYCL.

Inference systems & deploymentRead original
13:05

b12456

The update addresses three issues in the OpenCL implementation of the bin flash-attention prefill path: releasing a previously unreleased WMM flash-attention mask fallback image, excluding q8_0 K/V at head size 512 from the WMM kernel due to incorrect results, and preventing the use of the WMM path beyond device 3D image limits to avoid clCreateImage failures.

Inference systems & deploymentRead original
13:00

Benchmarking LLM Inference on Desktop-Scale NVIDIA Grace Blackwell Clusters

arXiv:2611.20289v1 Announce Type: new Abstract: Two NVIDIA DGX Spark GB10 systems connect directly over 200 GbE to serve larger language models and increase inference capacity, but the benefits depend on how they are deployed. We benchmark 17 open-weight checkpoints spanning dense and mixture-of-experts architectures, 8B to 235B parameters and four weight precisions on one and two RDMA-connected systems. Using vLLM, we compare tensor parallelism (TP=2), pipeline parallelism (PP=2) and a two-replica baseline at concurrency 1 to 64, with three independent launches per configuration. TP=2 delivers a median 1.59x peak-throughput gain over one Spark, versus 1.39x for PP=2 and has higher throughput in 93 of 96 matched comparisons with separated 95% Welch intervals. It reduces unloaded inter-token latency to 0.52x to 0.73x of one Spark, whereas PP=2 provides little reduction. TP=2 also achieves the highest goodput under a strict 100 ms inter-token latency objective wherever both strategies are supported and the objective is met. For models fitting on one Spark, TP=2 generally favors low-concurrency serving, while replication offers higher throughput at high concurrency. One Spark uses less GPU energy per output token at matched concurrency. Across three precision ladders, NVFP4 reduces unloaded latency more than adding a second Spark at BF16. NCCL microbenchmarks and comparisons with an earlier TCP campaign suggest that collective communication overhead contributes to the advantage of TP=2. These results provide deployment guidance for desktop-scale inference.

Chips, GPUs & hardwareRead original
13:00

Coordinating Distributed AI Servers with Industrial Loads to Reduce Grid-Interconnection Needs

arXiv:2611.18951v1 Announce Type: new Abstract: We show that co-locating distributed AI servers with industrial demand and coordinating AI workloads with production loads can substantially reduce the additional grid interconnection that AI computing requires. Grid interconnection is increasingly constraining the expansion of large AI data centers. Factories already have grid connections sized for their peak production loads, and we propose hosting AI servers during production valleys. We formulate a joint optimization that sizes CPU and GPU servers, schedules deadline-constrained tasks, and routes tasks among factories, and use it to compare three factory-site architectures with new large data centers serving the same demand. Applying this framework to 31 Chinese manufacturing industries, we find that AI demand and production loads often do not peak together. As a result, AI servers placed at factories and scheduled according to production peaks need about 60% less additional interconnection capacity than new large data centers, and 98% less when tasks are routed among a group's factories.

Data centres & cloud computeRead original
13:00

Making Preemption Productive in Long-Output LLM Serving

arXiv:2611.20106v1 Announce Type: new Abstract: Reasoning LLMs generate long outputs whose key--value (KV) caches occupy GPU memory for minutes. When memory fills, the server preempts requests, and a preempted request can repeatedly lose and recover its state while producing little new output. We call this behavior KV thrashing. In certain cases, existing systems leave more than a fifth of requests stalled for over a minute. Because preemption cannot be avoided without delaying admission, we ask how to make it productive. We present Stint, a serving system that preserves a preempted request's work, bounds how long it waits, and gives it fresh output after it returns. Stint stores fixed-length KV snapshots in a contiguity-first CPU pool with an exact-extent allocator, and moves them with fused K/V gather/scatter kernels and double buffering. Age-based readmission brings stalled requests back, and protection gives them a turn of fresh decoding before another preemption. Remaining-length estimates from decoding hidden states guide victim selection. On olympiad-math replays with Qwen3-32B on H200 GPUs, no request pauses for more than 50 seconds under Stint and p99 time per output token falls by 50.7\% relative to recomputation in the same engine. The replay as a whole finishes 10.7\% sooner (12.0\% higher throughput), because the requests that baselines starve no longer finish last.

Chips, GPUs & hardwareRead original
13:00

Supercharging Olmo-core for Efficient and Scalable MoE Training

arXiv:2611.18956v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models add parameters without proportionally increasing per-token compute. However, training cost also depends on bytes moved, memory held, and kernels launched, and many of those costs follow total parameter size rather than active compute. On infrastructure built for dense models, this mismatch is costly: in early baselines, throughput fell as stored expert parameter size grew, even though the useful work per token stayed nearly fixed. To close this gap, we present a redesigned training stack in Olmo-core, built for efficient, scalable MoE training. We replace the Fully Sharded Data Parallel (FSDP) of earlier Olmo-core versions with an architecture built on Distributed Data Parallel (DDP), and add Expert Parallelism (EP), Pipeline Parallelism (PP), a distributed optimizer, and activation recompute. We propose NVSHMEM-based rowwise EP, which writes each token directly into its expert's buffer, avoiding dispatch and combine permutation overhead. We make EP fully synchronization-free by keeping routing metadata on the GPU and using device-scheduled grouped GEMMs. Closely integrated MXFP8 cuts compute, activation-memory, and transport costs, and topology-agnostic checkpoints let the parallel layout change between runs. On NVL8 B300 nodes, the stack supports training configurations from 12.9 billion to 1.2 trillion total parameters on up to 512 GPUs, reaching 858 useful-model TFLOP/s/GPU at 1.2 trillion with MXFP8 and per-layer recompute; an experimental capacity test with the optional DeepEP v2 backend reaches 2.38 trillion. We also explain each design choice and share findings for other training systems; for example, we identify a failure pattern we call Token Gerrymandering, in which the MoE router can learn to hack the load-balancing loss, and find that overlapping communication with computation can slow the overlapped kernels by more than it hides.

Token pricing, cost & energyRead original
11:34

b12451

The web UI for Llama.cpp has been relocated from tools/ui to ui/apps/chat as part of a structural refactoring to separate the component library (ui/lib) from the applications built on it. The app is now officially named Llama Chat, with its package renamed to @ggml-org/llama-chat, while maintaining private status. Documentation, help text, comments, and other references have been updated to reflect the new name, though legacy identifiers remain unchanged for backward compatibility.

Inference systems & deploymentRead original
04:44

b12445

The update introduces support for split_k in conv3d operations using Vulkan, leveraging a 256x128 shape (coopmat2-only) when beneficial and reusing the mul_mat split_k reduce shader.

Inference systems & deploymentRead original
04:00

How the World’s Most Advanced AI Infrastructure Supply Chain Invents the Future

Deb Shoquist leads operations at NVIDIA, overseeing the supply chain and manufacturing behind its AI platforms. That description sums up the job, but it doesn’t begin to encompass what Shoquist has made possible over decades. Shoquist, her team and their global network of manufacturing partners enable NVIDIA to introduce world-leading technologies year-round. Much of the […]

Chips, GPUs & hardwareRead original
00:00

NVIDIA Sweeps MLPerf Training and Endpoints as Vera Rubin Debuts With Leading Performance

In its MLPerf benchmarks debut, with results released today, the NVIDIA Vera Rubin platform delivers leading performance across training, post-training and agentic inference. NVIDIA swept every MLPerf Training v6.1 benchmark and, in its first MLPerf Endpoints submission, delivered more than 10x generational gains in throughput per watt for agentic AI — measuring how much agentic […]

Chips, GPUs & hardwareRead original
2026-11-16
Follow the timeline
19:16

b12433

A fix was implemented to prevent the MUL_MAT_ID operation from aborting on AMD devices when using F32, F16, or BF16 data types with an odd number of columns in src0, particularly affecting rank 1 LoRA on MoE expert weights. The issue arose because MMVF read src0 in pairs and required an even ne[0], so the fix restricts MMVF usage to cases with even ne[0], defaulting to fallback for other shapes. This resolves issue #30576.

Inference systems & deploymentRead original
17:20

b12431

The update addresses issues with reasoning effort levels by ensuring rejected levels are properly handled during parsing, preventing 500 errors on API endpoints. It maintains support for template-ignored levels and aliases while trimming the output_config.effort test to a single level.

Inference systems & deploymentRead original
08:12

b12428

The update includes the addition of four missing features in jinja, along with applied suggestions and a fix for an error message. This release is part of the llama.cpp project, offering binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, supporting various backends like CUDA, Vulkan, ROCm, OpenVINO, and SYCL.

Inference systems & deploymentRead original
08:00

AI Day Seoul Brings Together Korea’s Industry Leaders, Startups and Developer Communities

Over two days at COEX, more than 2,000 attendees converged for AI Day Seoul, bringing together Korea’s industry leaders, startups and developer communities advancing physical AI and AI agents. At the plenary sessions, Deepu Talla, vice president of robotics and edge AI at NVIDIA, said Korea is “punching above its weight,” with Doosan Robotics, Hyundai […]

Chips, GPUs & hardwareRead original
02:14

b12421

The llama.cpp release b12421 includes updated binaries for multiple platforms including macOS, Linux, Windows, Android, and iOS, with support for various backends such as CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon. The documentation has been reorganized, moving per-model TTS content to /docs and linking it from other pages. Some builds, like macOS Apple Silicon with KleidiAI and openEuler variants, are currently disabled.

Inference systems & deploymentRead original
Every update links back to its source.
STAY CURIOUS. STAY INFORMED.

Stay informed about the changes that matter.

Give your curiosity direction. Make every day of reading count.

Subscribe via RSS