Official announcements and industry reporting. Follow the timeline of AI.
From the latest news to tools that work for you.
EvoMapAgents & automationEvoMap is an infrastructure designed for the reuse of AI agent experiences. It organizes reusable strategies and repair assets with validation records via the GEP protocol, offering market- and quality-based ranking to help different agents discover, evaluate, and inherit existing capabilities.
分类推荐榜首M
MemoAIMusic & audioMemoAI is an AI-powered audio-to-text tool that supports localized processing, eliminating the need to upload data and ensuring privacy and security. It can efficiently transcribe YouTube videos, podcasts, and local audio/video files (such as MP4, MP3, AAC, M4A, etc.) into text, while offering core features like multilingual translation, speaker diarization, real-time subtitles, and floating notes—making it ideal for meeting minutes, interview transcription, content learning, and more.
分类推荐榜首A
AI ShortWriting & contentAIS ALERT MANAGEMENT AND SHARED PLATFORM, QUICK COMMAND FOR MULTIPLE SCENARIOS
分类推荐榜首J
JAXModels & infrastructureJAX is a Python numerical computing library designed for accelerators, specifically tailored for high-performance array computations and large-scale machine learning. It efficiently executes the same code across multiple hardware backends, including CPUs, GPUs, and TPUs.
分类推荐榜首O
Open DesignDesign & 3DOpenDesign is an open-source design workspace that can connect to local programming agents, use built-in model keys, or leverage cloud-based models, enabling the creation and editing of prototypes, web pages, presentations, and graphics within a design system; local software is free, while model services incur fees.
分类推荐榜首K
KiroCoding & developmentKiro is an AI engineering tool for developers that organizes requirements into specifications, architectures, and tasks, generates code via agents, and supports the entire workflow—from planning to code review—through property-based testing, execution memory, and integration with development tools.
分类推荐榜首妙
妙播Marketing & commerceTHE AI LIVE TELECOM SOLUTION FROM THE AD
待评分 · 分类列表推荐I
IamIPIndustry applicationsIamIP is a patent platform that provides AI-driven search, analysis, and monitoring tools to help users conduct patent research and management more efficiently.
待评分 · 分类列表推荐M
My Glow Up AILife & personalMy Glow Up AI provides free analysis based on selfies, without requiring registration, and can generate hairstyle recommendations and beauty plans that suit you.
待评分 · 分类列表推荐E
ElicitKnowledge & researchElicit provides semantic search, research report generation, literature screening, and data extraction for scientific research. It enables traceable answers within source materials and organizes evidence through a literature library, research agents, and ongoing tracking workflows, making it ideal for advancing from initial problem exploration to systematic reviews.
待评分 · 分类列表推荐MMVF is recommended for BF16 matrices with few rows and small batch sizes due to its superior performance when K is large, while MMF remains efficient for larger row counts. The decision depends on the condition rows × batch × 256 ≤ nsm × min(K, max_k) and K ≥ 1024, with GPU-specific limits on columns and max_k. Evaluation cases include 128×2560, 512×2560, and 320×10240 at batch sizes 2–8.
arXiv:2611.13442v1 Announce Type: new Abstract: Parallel drafters accelerate LLM decoding by proposing a whole block of draft tokens in a single forward pass, making them attractive for LLM serving. Recent systems optimize goodput (i.e., accepted tokens per second) because rejected drafts steal compute from other requests. However, these systems control only how many tokens are verified, so every request still drafts a full block. At high serving load, we find that drafting consumes up to 40% of step time, of which a majority is discarded because it is never verified. Yet, draft-length control from autoregressive drafters does not carry over, because a parallel drafter commits to its length before its single pass. Moreover, draft and verification lengths are coupled across the batch, so they must be selected jointly for every request while keeping the decision computationally cheap at every step. In this paper, we propose work-conserving speculation (WCS), a design that jointly allocates draft and verification budgets per request in LLM serving, and present ForeSpark as its prototype. ForeSpark realizes WCS with (i) a context-aware pre-draft estimator that predicts acceptance for every candidate length from the target model's context features, (ii) a lightweight joint draft-verify scheduler that selects both lengths through a global sort, and (iii) a post-draft budget distribution that spends verification on the most promising tokens. Implemented on vLLM, ForeSpark improves throughput by up to 1.77x over the state-of-the-art DSpark and by 2.49x over autoregressive decoding at a matched per-request generation rate, without changing the output distribution. It also tolerates higher load, sustaining up to 1.51x the request rate of DSpark under latency service level objectives (SLOs).
The update addresses CUDA out of memory errors by reducing the VMM pool reservation size to 8 GB on GPUs with combined memory, halving the reservation size and retrying on failure only when encountering CUDA_ERROR_OUT_OF_MEMORY, while also logging a warning when the reservation is less than the maximum size.
Recent advances in vLLM encoder disaggregation: metadata-only multimodal requests, dedicated encoder execution, remote embedding transfer, cross-encoder caching, and lower-overhead EPD routing.
The latest release of llama.cpp (b12259) adds support for Kimi K3.1 Vision, includes fixes for preprocessing and projector MLP in both K2.5 and K3.1 models, and features minor renaming changes. The release provides precompiled binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, with support for various backends such as CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon-specific hardware.
The update introduces graceful failure handling for unreachable RPC endpoints and failed handshakes in ggml-rpc, replacing fatal aborts with reportable errors. Functions now return appropriate values (e.g., false, nullptr) on connection failures, allowing host applications to handle issues without crashing. A regression test was added for unreachable endpoints, and low-value comments were removed. Mid-stream failures remain unchanged to prevent silent corruption.
TL;DR We serve Meta’s Muse Spark Open, an 816B-parameter mixture-of-experts model, on a single TPU v7 host of four chips, with SGLang running on the TorchTPU PyTorch backend. At the... The post Optimizing Muse Spark Open on TPU v7 with SGLang-TorchTPU appeared first on PyTorch.
On Qualcomm Adreno GPUs, scalar flash attention was limited to 8 rows per workgroup when head size exceeds 128 to prevent register spilling and performance degradation. This change improved performance significantly, reducing execution time from several seconds to milliseconds and enabling completion of the test-backend-ops case.
Autonomous vehicle (AV) developers need reasoning models that can operate within the compute and latency constraints of an onboard system. Larger models provide...
NVIDIA Snapshot, a new open-source component of the NVIDIA DSX AI Factory Platform, makes initialized GPU workloads reusable across inference...
AI agents spend most of their time idle, waiting on a human prompt, a tool result or a model response. Kubernetes wasn’t built for that. Running one Pod per...
Picture guiding a robot arm by hand while its motors supply just enough torque to support its weight and keep the robot upright. This simple interaction, a...
A fix was implemented to ensure the OpenAI-compatible chat completions endpoint correctly processes JSON schema when provided at the top level in response_format.schema, rather than requiring it to be nested within response_format.json_schema.schema. The update also improves error messages, documents the correct usage in the README, and adds test cases to verify behavior, resolving issue #10732.
The Metal kernel for ggml_tri has been updated to use a select operation instead of scaling by comparison, aligning its behavior with the CPU kernel. This change ensures that values outside the triangle are set to zero rather than becoming NaN when encountering ±inf. A new test, test_tri_special, validates this fix across all four triangle types, confirming it passes with the update but failed previously.
arXiv:2611.08770v1 Announce Type: new Abstract: Prefill--decode disaggregation moves a prompt's key--value (KV) cache between workers before the first token can be generated. Reducing this handoff to either cache compression or recomputation misses a useful choice: which layers to transmit, at what precision, and which layers to execute again on the decode worker. ReKV transfers a quantized prefix KV cache and one BF16 boundary state, then reconstructs a contiguous suffix while the prefix is in flight. In a model-shaped synthetic benchmark on cross-node H100s, the Q8 configuration meeting separately evaluated quality criteria reduces 8K handoff latency by $2.36\times$, $2.84\times$, and $3.65\times$ over BF16 transfer for Qwen2.5-7B, Qwen2.5-3B, and Qwen3-8B. It is also faster than the best measured transfer-only or recompute-only action at those points. A separate real-weight BF16 recomputation experiment reduces prefill-to-token-return time by $1.16$--$1.63\times$. The preferred action changes with prompt length and link bandwidth.
The release includes updates for SYCL backend with exact rounding support in CPY operations for Q4_1, Q5_1, and Q8_0 quantization formats.
The llama.cpp release b12195 introduces a change to refuse device buffers that would exceed VRAM, improving memory management. It includes precompiled binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, with support for various backends such as CUDA, ROCm, Vulkan, OpenVINO, SYCL, and others. Some builds are marked as disabled or experimental.
The llama.cpp release b12193 includes a change to move log-mel computation to the cgraph, removes stray files, updates mtmd_audio_track to include source, and adds tests. It provides precompiled binaries for macOS, Linux, Windows, Android, and openEuler across various architectures and backends including CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon-specific platforms.
The latest release of llama.cpp adds support for the MageVL-Cowork model and adjusts how rope frequencies are handled, now stored at the model level within clip_model instead of being copied into each clip_layer. The loader verifies that rope_freqs is a 32-bit floating-point array with dimensions matching d_head, ensuring compatibility with expected 4:6:6 splits in the computational graph.
The latest update to llama.cpp introduces enhanced MoE (Mixture of Experts) cache functionality, enabling computation of cache misses on the CPU with a configurable CPU share. A new tuning script, moe-cache-tune.py, automatically determines optimal cache size and CPU share settings based on system performance. The update also improves scheduling efficiency, removes redundant sync operations, and refactors code for better maintainability, while maintaining compatibility across multiple platforms including macOS, Linux, Windows, and Android.
Give your curiosity direction. Make every day of reading count.
