Signal SentrySearchUpdated Nov 20, 05:30 UTCAM Drop
Agents · industry

Agents in AI infrastructure

Agent deployments and launches in AI infrastructure, from the Drops.

From the Drops

AgentsAI infrastructureTestingHuman in loop: unclear

Preprint: agent-aware serving cut P95 time to first token up to 14x versus Nvidia Dynamo

Researchers report in a preprint that Evolving Infra moves nodes between prefill and decode and keeps KV caches agents will reuse. On up to 256 GB300 GPUs, it cut mean latency up to 4.1x and P95 TTFT up to 14x versus Dynamo.

Why it matters: If you serve your own models for agents, check whether your stack evicts context the agent will need again; the authors argue generic serving signals miss how agents reuse it.

From Drop #087 · Sun, Nov 15, 2026 · Morning
AgentsAI infrastructureTestingHuman in loop: unclear

Preprint: model rankings shift when agents get 24 hours to tune AI infrastructure

Researchers report in a preprint that across 150 Infrathon runs with 24-hour budgets, six models' rankings changed with the horizon and 135 runs kept improving in the second half. Claude Opus 5.5 beat a human-designed codec.

Why it matters: If you pick agent models from short evals, the authors say you can miss later gains and rank changes. They name task violations, unreliable tests and early stopping as the blockers.

On the Frontier watch →From Drop #083 · Fri, Nov 13, 2026 · Morning
AgentsAI infrastructureTestingHuman in loop: unclear

Preprint: 24-hour agent runs reorder model rankings seen in shorter tests

Infrathon ran six models over 150 runs with 24-hour budgets. The authors report 135 of 150 runs still improved in the second half, and Claude Opus 5.5 raised a KV-cache compression ratio from 6.82% to 19.86%.

Why it matters: Short evals can misrank agents meant for long jobs. The authors say agents stall on task violations, flaky testing and stopping too early.

On the Frontier watch →From Drop #078 · Tue, Nov 10, 2026 · Evening
AgentsAI infrastructureTestingHuman in loop: unclear

Preprint: agents that drop their own KV cache cut cost 27.7% vs summarizing

Researchers propose cache engineering: agents drop or reposition cached context instead of summarizing history. A rolling-drop policy raised average benchmark scores 1.7 points and cut cost 27.7%, the abstract says.

Why it matters: Long agent runs get expensive as context grows. The authors pair the method with an AKV serving engine, suggesting the serving layer, not just prompts, can help.

On the Frontier watch →From Drop #039 · Thu, Oct 22, 2026 · Morning