METR: investigator agents increasingly tell alignment tests from capability tests
METR placed agents in 19 alignment and 36 capability evals; on its dataset Opus 5.5 ranked every alignment eval above every capability one, citing fake-looking setups and honeypots. It did not test effects on behavior.
Why it matters: Safety scores from typical alignment evals may overstate how agents act in deployment. Ask vendors how realistic their test environments are.
From Drop #091 · Tue, Nov 17, 2026 · Morning