05:56
Chinese frontier AI models carry the fingerprint of export-controlled GPUs
Article URL: https://unemploymentlabs.com/checkpoint-forensics Comments URL: https://news.ycombinator.com/item?id=50474883 Points: 42 # Comments: 7
Today's news
Products & applications / 013 ways AI is changing the healthcare revenue cycleSiliconANGLE AI
01 / IN FOCUS
3 ways AI is changing the healthcare revenue cycle
SiliconANGLE AIModels / 024 insights from Block Builderlab Launch Event 2026: AI agents join the team as working colleaguesSiliconANGLE AI
02 / IN FOCUS
4 insights from Block Builderlab Launch Event 2026: AI agents join the team as working colleagues
SiliconANGLE AISafety & governance / 03AI Models Know What Women, Peace and Security Is. They Just Don't Bring It UpHacker News · AI热门原文
03 / IN FOCUS
AI Models Know What Women, Peace and Security Is. They Just Don't Bring It Up
Hacker News · AI热门原文AI NEWS × FOXAIFind your next step
Explore AI tools From the latest news to tools that work for you.
Discover AI tools
Swipe to explore →K
KiroCoding & development
EvoMapAgents & automation
AI ShortWriting & content
JAXModels & infrastructure
MemoAIMusic & audio
Open DesignDesign & 3D
Meta AI助手Assistants & search
ElicitKnowledge & research
ClipDropImages & editing
AssemblyAIMusic & audio
KiroCoding & developmentKiro is an AI engineering tool for developers that organizes requirements into specifications, architectures, and tasks, generates code via agents, and supports the entire workflow—from planning to code review—through property-based testing, execution memory, and integration with development tools.
分类推荐榜首EEvoMapAgents & automationEvoMap is an infrastructure designed for the reuse of AI agent experiences. It organizes reusable strategies and repair assets with validation records via the GEP protocol, offering market- and quality-based ranking to help different agents discover, evaluate, and inherit existing capabilities.
分类推荐榜首AAI ShortWriting & contentAIS ALERT MANAGEMENT AND SHARED PLATFORM, QUICK COMMAND FOR MULTIPLE SCENARIOS
分类推荐榜首JJAXModels & infrastructureJAX is a Python numerical computing library designed for accelerators, specifically tailored for high-performance array computations and large-scale machine learning. It efficiently executes the same code across multiple hardware backends, including CPUs, GPUs, and TPUs.
分类推荐榜首MMemoAIMusic & audioMemoAI is an AI-powered audio-to-text tool that supports localized processing, eliminating the need to upload data and ensuring privacy and security. It can efficiently transcribe YouTube videos, podcasts, and local audio/video files (such as MP4, MP3, AAC, M4A, etc.) into text, while offering core features like multilingual translation, speaker diarization, real-time subtitles, and floating notes—making it ideal for meeting minutes, interview transcription, content learning, and more.
分类推荐榜首OOpen DesignDesign & 3DOpenDesign is an open-source design workspace that can connect to local programming agents, use built-in model keys, or leverage cloud-based models, enabling the creation and editing of prototypes, web pages, presentations, and graphics within a design system; local software is free, while model services incur fees.
分类推荐榜首MMeta AI助手Assistants & searchMeta's free AI chat assistant
待评分 · 分类列表推荐EElicitKnowledge & researchElicit provides semantic search, research report generation, literature screening, and data extraction for scientific research. It enables traceable answers within source materials and organizes evidence through a literature library, research agents, and ongoing tracking workflows, making it ideal for advancing from initial problem exploration to systematic reviews.
待评分 · 分类列表推荐CClipDropImages & editingClipdrop is a suite of AI-powered image creation and editing tools, covering background removal, local cleanup, image enlargement, upscaling, relighting, text removal, and background replacement. It allows users to process images directly in the browser and also provides developers with an API for integrating these capabilities into their applications.
待评分 · 分类列表推荐AAssemblyAI is a speech AI platform for developers, offering pre-recorded, real-time, and short-form synchronous transcription APIs. It enables the building of voice-based applications by integrating capabilities such as speaker identification, emotion analysis, chapter and summary extraction, as well as speech agent functionality, dictation, and sensitive information processing.
待评分 · 分类列表推荐AI, as it happens
1819 published storiesCompute & infrastructure · 251 storiesClear filters ×
2026-11-13
Follow the timeline00:45
Fast by Default: Compile-Time Heuristics for Helion Kernels
TL;DR Helion now uses compile-time heuristics that analyze a kernel’s IR and dataflow to pick fast settings without slow autotuning. It classifies the kernel as matmul, reduction, or pointwise and... The post Fast by Default: Compile-Time Heuristics for Helion Kernels appeared first on PyTorch.
2026-11-12
Follow the timeline22:45
Integrating SGLang into Spyre in Under a Week with AI
TL;DR With help from AI coding agents, our two-person team integrated SGLang with IBM Spyre in less than a week, building a plugin of roughly 1,500 lines. Existing Spyre support... The post Integrating SGLang into Spyre in Under a Week with AI appeared first on PyTorch.
20:02
The Sequence Opinion - Issue #970: The Neocloud Financing Squeeze
Why booming AI demand can coexist with fragile balance sheets
08:00
DeepSeek-V4.1-Flash on Vera Rubin NVL72: 58x Throughput vs. GB300 NVL72 at 387 tokens/s
On one NVIDIA Vera Rubin NVL72 rack, vLLM serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 1.7M tokens/s per GPU at a P90 of 387 tokens/s per user, 58x GB300 NVL72 at the same interactivity.
08:00
H3-Image Day-0 Support: 4K in 0.87s with Vera Rubin
H3-Image on Vera Rubin: 0.87-second 4K generation or 6,115 images/hour on four GPUs, with projected API revenue and support across consumer and data-center hardware.
08:00
Inkling 2 Small on vLLM: Day-0 Support
vLLM supports Thinking Machines' Inkling 2 Small from day 0: attention residuals, a new decode kernel for relative-bias attention, all-reduce fused with the ops around it, and DSpark speculative decoding.
02:15
b12287
MMVF is recommended for BF16 matrices with few rows and small batch sizes due to its superior performance when K is large, while MMF remains efficient for larger row counts. The decision depends on the condition rows × batch × 256 ≤ nsm × min(K, max_k) and K ≥ 1024, with GPU-specific limits on columns and max_k. Evaluation cases include 128×2560, 512×2560, and 320×10240 at batch sizes 2–8.
2026-11-11
Follow the timeline13:00
ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding
arXiv:2611.13442v1 Announce Type: new Abstract: Parallel drafters accelerate LLM decoding by proposing a whole block of draft tokens in a single forward pass, making them attractive for LLM serving. Recent systems optimize goodput (i.e., accepted tokens per second) because rejected drafts steal compute from other requests. However, these systems control only how many tokens are verified, so every request still drafts a full block. At high serving load, we find that drafting consumes up to 40% of step time, of which a majority is discarded because it is never verified. Yet, draft-length control from autoregressive drafters does not carry over, because a parallel drafter commits to its length before its single pass. Moreover, draft and verification lengths are coupled across the batch, so they must be selected jointly for every request while keeping the decision computationally cheap at every step. In this paper, we propose work-conserving speculation (WCS), a design that jointly allocates draft and verification budgets per request in LLM serving, and present ForeSpark as its prototype. ForeSpark realizes WCS with (i) a context-aware pre-draft estimator that predicts acceptance for every candidate length from the target model's context features, (ii) a lightweight joint draft-verify scheduler that selects both lengths through a global sort, and (iii) a post-draft budget distribution that spends verification on the most promising tokens. Implemented on vLLM, ForeSpark improves throughput by up to 1.77x over the state-of-the-art DSpark and by 2.49x over autoregressive decoding at a matched per-request generation rate, without changing the output distribution. It also tolerates higher load, sustaining up to 1.51x the request rate of DSpark under latency service level objectives (SLOs).
09:26
b12263
The update addresses CUDA out of memory errors by reducing the VMM pool reservation size to 8 GB on GPUs with combined memory, halving the reservation size and retrying on failure only when encountering CUDA_ERROR_OUT_OF_MEMORY, while also logging a warning when the reservation is less than the maximum size.
08:00
DeepSeek-V4.1-Flash on Vera Rubin NVL72: 58x Throughput vs. GB300 NVL72 at 387 tokens/s
On one NVIDIA Vera Rubin NVL72 rack, vLLM serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 1.7M tokens/s per GPU at a P90 of 387 tokens/s per user, 58x GB300 NVL72 at the same interactivity.
08:00
Encoder Disaggregation in vLLM: Scaling Multimodal Serving with EPD
Recent advances in vLLM encoder disaggregation: metadata-only multimodal requests, dedicated encoder execution, remote embedding transfer, cross-encoder caching, and lower-overhead EPD routing.
06:15
ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser
TL;DR PyTorch models now run in the web browser on the GPU through ExecuTorch, with no conversion to another framework’s format. You export with torch.export to an ExecuTorch .pte file,... The post ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser appeared first on PyTorch.
06:11
b12259
The latest release of llama.cpp (b12259) adds support for Kimi K3.1 Vision, includes fixes for preprocessing and projector MLP in both K2.5 and K3.1 models, and features minor renaming changes. The release provides precompiled binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, with support for various backends such as CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon-specific hardware.
00:57
b12249
The update introduces graceful failure handling for unreachable RPC endpoints and failed handshakes in ggml-rpc, replacing fatal aborts with reportable errors. Functions now return appropriate values (e.g., false, nullptr) on connection failures, allowing host applications to handle issues without crashing. A regression test was added for unreachable endpoints, and low-value comments were removed. Mid-stream failures remain unchanged to prevent silent corruption.
00:45
Optimizing Muse Spark Open on TPU v7 with SGLang-TorchTPU
TL;DR We serve Meta’s Muse Spark Open, an 816B-parameter mixture-of-experts model, on a single TPU v7 host of four chips, with SGLang running on the TorchTPU PyTorch backend. At the... The post Optimizing Muse Spark Open on TPU v7 with SGLang-TorchTPU appeared first on PyTorch.
2026-11-10
Follow the timeline21:09
b12246
On Qualcomm Adreno GPUs, scalar flash attention was limited to 8 rows per workgroup when head size exceeds 128 to prevent register spilling and performance degradation. This change improved performance significantly, reducing execution time from several seconds to milliseconds and enabling completion of the test-backend-ops case.
13:00
Firepool: Brokering Warm GPU Capacity for Agentic and RL Workloads at LinkedIn
arXiv:2611.10832v1 Announce Type: new Abstract: AI agents and reinforcement-learning (RL) trainers run thousands of short, isolated GPU jobs: tool calls, verifiers, and rollouts. A job often finishes in seconds, but starting a GPU pod for it takes 20-35 s even when the container image is already on the node, and minutes when it is not. A warm pod removes the wait only if it can serve the next job, and keeping pods warm must neither waste GPUs nor leak state between callers. Firepool, the sandbox and execution broker behind agentic and RL workloads at LinkedIn, meets both requirements. It makes pods interchangeable by building them from a shared base image per hardware profile and binding each job's code, extra dependencies, and cluster topology when the job starts. It keeps them warm cheaply and safely by reusing a pod only for the caller that used it last, staging zero-VRAM standby pods in fractional GPU shares alongside active pods, and capping the GPUs that warm pools may hold via queue-driven budgets. Over a sampled peak period in production, Firepool serves 94.7% of 3,206 pod requests from warm or standby capacity. In paired benchmarks with fixed-duration jobs on production H200 nodes, agent and RL loops finish 13x and 10.8x faster than with a fresh pod per job; against a fresh reuse tag per job, warm reuse cuts p90 start latency from 59.1 s to 3.5 s. When a job cannot reuse the previous pod, standby promotion shortens the handover wait from 51 s to 0.94 s. In local and simulated runs, a keep-alive matched to the loop's pauses keeps the latency benefit while holding about half the capacity of an always-on pool.
06:01
Agent Substrate: Advancing Kubernetes for Agents at Scale
AI agents spend most of their time idle, waiting on a human prompt, a tool result or a model response. Kubernetes wasn’t built for that. Running one Pod per...
00:41
b12229
A fix was implemented to ensure the OpenAI-compatible chat completions endpoint correctly processes JSON schema when provided at the top level in response_format.schema, rather than requiring it to be nested within response_format.json_schema.schema. The update also improves error messages, documents the correct usage in the README, and adds test cases to verify behavior, resolving issue #10732.
Every update links back to its source.