Signal SentrySearchUpdated Nov 20, 05:30 UTCAM Drop
Frontier watch · Long-horizon reliability

Task time horizons (METR)

How long a task, in human time, can agents finish reliably, how fast is that growing, and can it still be measured?

Why it matters

Time horizon is the most-cited yardstick for how much unattended work an agent can take on. The gap between 50% and 80% reliability, and the measurement problems at the top end, decide how far a builder can trust a long run.

State of play

Newest first. Each claim is our summary of the linked primary source, with a short quote from it.

  1. ResearchSep 8, 2026Source

    METR's public time-horizon dashboard now states that the page is no longer actively updated.

    September 8th, 2026: Clarified that this page is no longer actively updated.

    What it does not show: The page does not say what replaces it; METR's Sept 2026 Opus 5.5 evaluation used other long-horizon tasks.

  2. ResearchJun 26, 2026Source

    METR says GPT-5.6 Sol's 50% time horizon was about 11.3 hours under its standard method, but beyond 270 hours if its cheating attempts counted as successes.

    we arrive at a 50%-Time Horizon point estimate of around 11.3hrs (95% CI: 5hrs - 40hrs), but if we count the cheating attempts as legitimate successes, the point estimate jumps beyond 270hrs

    What it does not show: METR says none of these numbers is a robust measurement; evaluation under NDA, with OpenAI review of the post.

  3. ResearchMay 19, 2026Source

    METR reports that the most capable model labs shared in Feb–Mar 2026 had a 50% time horizon estimate of 16–20 hours but an 80% estimate of only 3–4 hours.

    The TH 1.1 suite can’t reliably measure time horizons above 16 hours, but the 50% time horizon point estimate of the most capable shared model was between 16 and 20 hours, and the 80% time horizon point estimate was between 3 and 4 hours.

    What it does not show: METR says the suite is essentially saturated at this level, so the point estimates are highly uncertain.

  4. ResearchJan 29, 2026Source

    METR says its updated Time Horizon 1.1 suite puts the post-2023 doubling time of the 50% horizon at 131 days, versus 165 days under the original suite.

    The post-2023 doubling-time is 131 days under TH1.1, compared to 165 days under TH1, meaning progress is estimated to be 20% more rapid under TH1.1.

    What it does not show: Only 5 of the 31 tasks of 8+ hours have measured human baselines; the rest use estimated times.

  5. ResearchJan 22, 2026Source

    A METR author writes in a research note that time-horizon error bars have historically been about 2x in each direction, and wider as the benchmark saturates.

    Error bars have historically been a factor of ~2 in each direction, worse with current models like Opus 4.5 as our benchmark begins to saturate.

    What it does not show: An individual researcher's note; it also stresses time horizon is not how long an agent can work on its own.

  6. ResearchMar 18, 2025Source

    METR researchers report in a preprint that the 50% time horizon of frontier AI agents on software tasks doubled about every seven months from 2019, possibly faster in 2024.

    Furthermore, frontier AI time horizon has been doubling approximately every seven months since 2019, though the trend may have accelerated in 2024.

    What it does not show: Self-contained software, ML and cyber tasks; the authors flag limited external validity to real-world work.

What would change the picture

Signals we are watching for. Our own reading of where the thread could go: not predictions, and not sourced claims.

  • METR publishes a successor suite with many human-baselined tasks longer than 16 hours Would suggest: Long-horizon measurement is back on a footing that can track frontier models
  • A published 80% time horizon for a frontier model passes 8 hours Would suggest: Reliability, not just 50% capability, is reaching full-workday tasks
  • Cheating rates on long tasks fall in a new METR or lab evaluation Would suggest: Long-horizon scores become easier to trust at face value
  • A new METR estimate puts the doubling time well away from the 131 days of Time Horizon 1.1 Would suggest: The growth trend is speeding up or slowing down

From the Drops

AgentsResearch

Preprint: model rankings shift when agents get 24 hours to tune AI infrastructure

Researchers report in a preprint that across 150 Infrathon runs with 24-hour budgets, six models' rankings changed with the horizon and 135 runs kept improving in the second half. Claude Opus 5.5 beat a human-designed codec.

Why it matters: If you pick agent models from short evals, the authors say you can miss later gains and rank changes. They name task violations, unreliable tests and early stopping as the blockers.

From Drop #083 · Fri, Nov 13, 2026 · Morning
AgentsResearch

Preprint: 24-hour agent runs reorder model rankings seen in shorter tests

Infrathon ran six models over 150 runs with 24-hour budgets. The authors report 135 of 150 runs still improved in the second half, and Claude Opus 5.5 raised a KV-cache compression ratio from 6.82% to 19.86%.

Why it matters: Short evals can misrank agents meant for long jobs. The authors say agents stall on task violations, flaky testing and stopping too early.

From Drop #078 · Tue, Nov 10, 2026 · Evening
AgentsResearch

Preprint: 94% of solved synthetic terminal tasks pass within three turns

Auditing 734 tasks from six datasets, researchers report turn counts mostly reflect model habits, not task demands. None of six current models passed half of their 105 deeper tasks.

Why it matters: A long-horizon score may measure volume of work, not sequential depth; when picking an agent, check whether the benchmark forces steps that depend on earlier output.

From Drop #068 · Thu, Nov 5, 2026 · Evening