The latest AI news, in focus.

A Controlled Experimental Evaluation of Configuration and Cache Architecture in LLM Serving Engines

arXiv:2611.05991v1 Announce Type: new Abstract: Multi-agent LLM applications repeat most of their prompt tokens across requests, and serving engines reuse the cached state of shared prefixes. vLLM and SGLang reuse it at different granularities, and benchmarks of their default deployments report large performance gaps, usually credited to the cache. Such comparisons rarely control for other configuration differences. One of them, whether a long prompt is prefilled in chunks, is set by each engine's defaults from the hardware and the release, and it confounds the comparison. We built a reproducible testbed on Kubernetes and a seeded multi-agent workload generator, and compared the two engines under vendor defaults and with chunked prefill matched. SGLang's configuration is the same in both arms, which makes it an internal control across independently provisioned deployments. Under defaults, vLLM's median time to first token is up to 64x SGLang's. Matching one setting removes 78% of the capacity gap where that gap is largest and 30 to 67% of the latency gap. Near saturation the remaining latency gap reflects queue growth. A fluid queue model driven only by a 4.4% throughput shortfall reproduces it, and shows that such a gap scales with the length of the benchmark run. What survives matching is a tail-latency advantage of 2.4 to 6.4x at equal throughput, consistent with cache-aware ordering of waiting requests. We derive six recommendations for comparing serving engines.

ScienceVision: Can AI Observe and Reason Like a Scientist?

arXiv:2611.04087v1 Announce Type: new Abstract: Scientific discovery builds on accurate observations interpreted through domain knowledge and reasoning. For scientific 3D data, this requires understanding structures and spatial relationships, from protein contacts to anatomical organization. Existing multimodal benchmarks largely evaluate text and 2D images, while scientific 3D evaluations vary in representations and protocols across disciplines, limiting cross-disciplinary comparison. To assess these foundational capabilities, we introduce ScienceVision, a cross-domain benchmark for scientific 3D understanding and reasoning through expert-level visual question answering (VQA). We construct 250 questions, with 50 each from Biology, Chemistry, Medical Imaging, Mechanical CAD, and Architecture, through a pipeline combining synthetic question generation with domain-expert review and annotation. Each task pairs a native 3D artifact with a scientific question, a specified response format, and a task-specific reference answer, requiring domain knowledge to be applied to geometric or topological evidence. We evaluate 12 models on the same questions in three settings: visual question answering with native 3D artifacts accessed through an agent harness's file and code-execution tools; the same workflow augmented with expert-inspired viewer inspection and measurement tools exposed through the Model Context Protocol (MCP); and visual question answering from prescribed rendered 2D images using an agent harness. All settings use the same task-specific deterministic final-answer scoring. The inspection environment supports view changes, target isolation, and slice inspection. In this main comparison, the best mean answer score is 0.53 out of 1. Mean scores in the native-artifact workflow without additional MCP tools exceed those in the rendered-image workflow for every model, by 17.2 percentage points on average.

Computation in Anonymous Directed Dynamic Networks: Exact Worst-Case Termination Time and Network-Wise Intractability

arXiv:2611.01761v1 Announce Type: new Abstract: We determine the exact round complexity of Counting in anonymous directed dynamic networks. Agents have no identifiers, but $\ell\geq1$ of them are designated as leaders. Every agent knows $\ell$ and its own leader status, while the population size $n$ is unknown and has to be determined by the agents. Communication proceeds in synchronous rounds; the directed communication graph may change arbitrarily and unpredictably from one round to the next, but is assumed to remain strongly connected. We solve Counting with a distributed algorithm that makes every agent output $n$ and halt after a number of rounds equal to $\begin{cases} n&\text{if }n=\ell\\ 2n-1&\text{if }n>\ell\text{ and }\ell=1\\ 2n-2&\text{if }n>\ell\text{ and }\ell\geq2. \end{cases}$ The algorithm works even for networks with loops and parallel arcs and even when agents learn their outdegree only after sending. We prove that this bound is worst-case tight over directed multigraphs for every $n$ and $\ell$ with $1\leq\ell\leq n$. Moreover, the local computation is polynomial (provided that each communication multigraph has at most $2^{\operatorname{poly}(n)}$ arcs): in every round, each agent solves at most one linear system with up to $n$ variables. By standard techniques, solving Counting also enables the evaluation of any computable function of the multiset of agents' input values within $2n-1$ communication rounds. The bound above optimizes the worst case over all networks. For an individual network, we show that even deciding whether an agent already has enough local information to determine $n$ is $\mathsf{coNP}$-complete (provided that the numbers of parallel arcs are encoded in binary). Thus, worst-case-optimal Counting admits polynomial local computation, whereas exact network-wise optimization is intractable under standard complexity assumptions.

Exposure Is Not Commitment: Double-Stratified Evaluation of Latent Monitors for Tool-Using Agents

arXiv:2610.37679v1 Announce Type: new Abstract: Tool-using language agents are vulnerable to indirect prompt injection, and a growing literature probes model internals to catch unsafe actions before they execute, with sharply conflicting results. We argue much of the conflict is an evaluation artifact, and propose a double-stratification protocol: separate injection exposure from behavioral commitment, in both the evaluation and the training population. On the primary Qwen AgentDojo grid, monitors trained on the naive mixed unsafe-versus-rest contrast mostly learn exposure: they flag exposed trajectories whether or not the agent complies, and barely predict which cases turn unsafe. Retrained on exposed cases only, the same probe families predict the model's own unsafe tool call up to two s

Evaluating and improving AI agents for sequencing protocol understanding

arXiv:2610.37851v1 Announce Type: new Abstract: Existing scientific-agent benchmarks rarely evaluate whether agents can reconstruct the exact molecular structures and generation workflows specified by sequencing protocols, a capability critical for analyzing customized sequencing assays. We introduce LibStructBench, a document-grounded benchmark for reconstructing sequencing-library structures and molecular workflows. Our experiments show that frontier agents often produce approximately correct reconstructions but remain unreliable at exact reconstruction, particularly in molecular-state representation and workflow connectivity. Switching the underlying language model produced larger average performance differences than switching agent harnesses, with a statistically significant difference

Benchmarking Task Takeover Across Agent Handoffs

arXiv:2610.36523v1 Announce Type: new Abstract: As agents collaborate on longer tasks, responsibility may pass between models before the work is complete. The receiving model must build on prior decisions, observations, and actions while addressing the remaining requirements. Evaluating this transition requires connecting the information preserved at handoff to how the receiver continues the work, a relationship that task-success and memory-retention scores alone do not capture. We introduce the Handoff State Continuity Benchmark (HSC-Bench), comprising 1,039 tasks that evaluate two complementary aspects of task takeover: recovering relevant repository evidence and continuing from established conversational, tool, and file-work state. Comparisons across six receiving models show that struc

How Data Culture measured its way from zero trust to a shippable AI analytics agent

Data Culture built a 3-step evaluation framework that took one AI analytics deployment from 0% accuracy to 94.7%.

Nace AI Releases Drex for Search for AI Agents: 92.05% on BrowseComp-Plus With 12.6 Search Calls

Nace AI has released Drex for Search, bringing its Drex decision models to document search for AI agents. Nace reports 92.05% on BrowseComp-Plus with 12.6 search calls, and fewer agent turns on OfficeQA Pro. The post Nace AI Releases Drex for Search for AI Agents: 92.05% on BrowseComp-Plus With 12.6 Search Calls appeared first on MarkTechPost.

Kimi releases K3.1 Pro, offering more features at a lower price to challenge Opus 5.5.

Author | Yu Hang Yuan Editor | Jing Yu On the morning of October 23, Kimi officially posted a 24-second video on X and its Video Account. The scene shows a retro‑style vintage computer, with Kimi Code running on the screen and lines of Three.js code being typed into the terminal, generating a complex 3D scene. Keen-eyed netizens quickly spotted the answer in the model bar at the top left: “Model: Kimi K3.1.” Kimi’s official teaser video for K3.1 | Image source: X @Kimi_Moonshot Using an antiquated machine to run some of today’s most cutting‑edge 3D code—this stark contrast itself conveys what Moonshot AI is trying to say. A dozen hours later, just after midnight on October 24, right on Programmer’s Day, Kimi K3.1 Pro officially launched, accompanied by a Highspeed version that’s two to three times faster. The company promises that the full model weights will be released before November 4. More than three months have passed since the previous generation, K3, was unveiled. In those three months, Moonshot AI has experienced more than most companies see in an entire year. 01 “More for the same price”—what’s new in 0.1? First, let’s clarify what K3.1 Pro is. It isn’t an entirely new model. Moonshot AI explicitly states that K3.1 Pro is a post‑training variant of K3; the underlying architecture remains unchanged—still a hybrid expert architecture with 2.8 trillion total parameters and 104 billion activation parameters, still featuring a 1 million‑token context window, and still natively supporting image and video inputs. What has changed is the post‑training phase. According to the company, they’ve invested “significantly increased post‑training compute power,” conducting large‑scale joint reinforcement learning across hundreds of downstream tasks. To support this, they completely reengineered the underlying infrastructure—including the reinforcement learning framework, sandbox environment, and long‑context reasoning capabilities. In essence, the same brain has been retrained from scratch on how to perform its tasks. The results are reflected in benchmark scores. On Terminal‑Bench 4.0, which evaluates coding agents, K3.1 Pro achieved 65.7%, surpassing GPT‑6 Astra’s 58.2% and falling short of Claude Opus’s 66.4% by less than one percentage point. As for the previous generation, K3…

BARC-Bench: Benign Asks, Risky Consequences

arXiv:2610.32024v1 Announce Type: new Abstract: Task instructions from users for agents are inherently underspecified, especially regarding safety boundaries--creating behavioral risks. To study these risks, we introduce BARC-Bench, a collection of 110 tasks across nine simulated workspace environments where benign requests leave openings for unsafe actions. In this benchmark, we evaluate agent behavior against safety boundaries established by workspace records, implicit environmental context, and unstated social norms. For example, an agent asked to schedule off-boarding discussions must arrange the meeting without prematurely revealing an employee's resignation through a shared calendar invitation. Evaluating four frontier model configurations across five attempts per task reveals that m

Reproducible Evaluation as the Substrate for Safe Prompt Compression in Production Agents

arXiv:2610.31972v1 Announce Type: new Abstract: Evaluating conversational agents that act on live, mutable enterprise data is hard: replayed sessions drift as backend state changes, and a benchmark built today can silently go stale as the data it describes moves on. We present an offline evaluation harness for a production conversational agent that stays reproducible despite the live data it operates on changing, and that can be regenerated on demand once it goes stale. We then use this harness as the fitness function for GEPA-based prompt optimization under a correctness gate that guards against reward hacking. Applied to a production merchant-facing agent, correctness-gated compression shrinks the agent's full prompt surface by 3.5-7.6% without regressing held-out task correctness, consi

Research reveals the “norm–oracle gap” problem in benchmarking coding agents.

A paper published on arXiv on October 24, 2026, points out that current mainstream benchmarks for coding AIs—such as SWE-Milestone—suffer from a “specification–oracle gap”: the natural-language descriptions of tasks fail to capture crucial implementation details required for testing (e.g., constant names, module paths, and precise error messages), resulting in evaluations that measure the ability to reproduce upstream code rather than genuine understanding and reasoning.

Why it mattersThis study is the first to formally define this type of bias, develop a static analysis tool covering five languages (with 89.2% precision and 92.9% recall), and empirically demonstrate in SWE-Milestone that 25 milestone tasks contain non-inferable implicit dependencies, significantly increasing the failure rate (80.6% vs. 48.4%). These findings reveal that current leaderboards may systematically favor “name‑copying” agents. The results have been made publicly available, including the open-source auditing tool and corrective resources, offering both methodological and practical value.