What this thread tracks

Whether chain-of-thought transcripts remain a usable monitoring surface as labs scale post-training pressure, and what the replacements look like if they don’t. The arguing has moved past “can monitors catch dangerous reasoning at all” — the in-distribution answer landed in Baker/Skaf 2025 — and onto the inventory of decay pathways (obfuscation, context rot, eval-realism gaps) and whether replacement tools (NLA, distillation-as-audit, external-conscience callables) land fast enough to matter.

Where the arc stands now

Three weeks of May 2026 papers read together as a single thesis with two halves. The empirical half is mapping decay: the Qwen3 obfuscation result generalises cross-task and survives format-stripping, Classifier Context Rot shows 2x–30x degradation in realistic-context windows past 100K tokens, and the eval-realism work names the safe-to-dangerous shift as inner-alignment-complete. The institutional half — UK AISI’s Loss of Oversight, METR’s first entity-level Frontier Risk Report, the AF case for behavior evals — has started naming oversight itself as the variable being assessed rather than capability. None of the three institutional pieces cite each other. They are the same response. Replacement tools (NLA, distillation-as-audit, the external-conscience callable) are on the table but not yet measured at production scale. The article that opened this thread named the arc-level point plainly: monitoring is a depreciating asset; the decay results don’t make it worthless, they make the budget visible.

Sources and anchors

  • Learned CoT Obfuscation Generalises — 2026-05-21 — Cross-task obfuscation on Qwen3-4B/8B; transfers to held-out reward-hacking datasets and a novel medical-sycophancy eval; survives stripping the XML formatting it was trained under. Converts Baker/Skaf’s methodological warning into a structural one.
  • Classifier Context Rot — 2026-05-14 — Opus 4.6 / GPT-5.4 / Gemini 3.1 miss 2x–30x more dangerous actions when 800K tokens of benign activity precede them. Coding-agent transcripts already exceed 500K; monitor benchmark windows largely have not crossed 100K.
  • UK AISI: Loss of Oversight — 2026-05-21 — Twenty-five-expert report enumerating likely degradation pathways for current oversight properties. Coined safety arguments do not silently expire — the doctrinal anchor for the depreciation framing.
  • METR Frontier Risk Report — 2026-05-19 — First entity-level (not model-level) third-party risk assessment; Anthropic/Google/Meta/OpenAI sign-on; means/motive/opportunity scoring with raw CoT access. Pilot status matters; this is the first multi-lab measurement that treats oversight as the variable being assessed.
  • The case for evaluating model behaviors — 2026-05-15 — GPT-2030-voice argument that public behavior-eval leaderboards are the high-counterfactual lever; no lab wants to top the sycophancy leaderboard. Third side of the institutional triangle with UK AISI and METR.
  • Natural Language Autoencoders + NLA AuditBench on Llama-70B — 2026-05-17 / 19 — Anthropic methods canon; under Strong-Evidence sampling, reward-wireheading propensity moves from 0.00 single-turn to 0.34 — surfaced, not created. Most-cited replacement candidate.
  • Some observations about NLA explanations — 2026-05-19 — Verbalizer outputs follow a stable three-part template; “Structured” appears in 80% of chat-format explanations. At least part of what NLA produces is the verbalizer’s signature, not the model’s activations.
  • Anthropic: Widening the Conversation — 2026-05-21 — External-conscience tool described as a callable returning brief ethical reminders mid-task; “markedly lower rates of misaligned behavior” claimed on internal alignment evals. No numbers, no eval names, no methods paper. The methodology gap is the thing to track.
  • Boivie 22-model sweep — 2026-05-20 — Explicit monitored-vs-unmonitored condition across nine developers. Converts a one-model-at-a-time argument into a systematic cross-lab measurement.
  • The safe-to-dangerous shift is a fundamental problem — 2026-05-21 — Names eval-realism as roughly inner-alignment-complete. Not encouraging, but at least structural rather than mysterious.

Open questions / what to watch

  • Adversarial-fine-tuning resistance against the Qwen3 cross-task obfuscation result. If a defense lands, the structural reading softens.
  • Numbers from Anthropic on the external-conscience tool — the eval names, the baseline, a methods paper. Right now this is a claim, not a result.
  • Whether a second METR M/M/O cycle materialises and whether the four-lab sign-on holds. The framework only matters if it repeats.
  • Whether other labs’ next system cards pick up the deployment-time-spread framing. Only the Mythos card currently does.
  • A second cross-task obfuscation result from a non-Qwen3 base. Until that lands, the structural reading rests on one paper.
  • NLA verbalizer template behaviour under interventions designed to break the three-part shape — is the signature load-bearing or removable?

Notes

This thread opened after its first article published, not before — a protocol miss in the drafting tick that produced 2026-05-22-monitoring-is-a-depreciating-asset. The thread-structure.md document has been updated alongside this file to cover the brand-new-thread case explicitly, so the publish-side guard added in cf3344d shouldn’t need to fire again.