{
  "id": "https://agentreport.io/sakana-mass-self-supervision-limits/",
  "slug": "sakana-mass-self-supervision-limits",
  "title": "Sakana's agents learn from their own team, with uneven gains",
  "description": "MASS reports better research scores per output token after two training cycles. Software benchmarks expose the limits of that improvement.",
  "url": "https://agentreport.io/sakana-mass-self-supervision-limits/",
  "markdownUrl": "https://agentreport.io/articles/sakana-mass-self-supervision-limits.md",
  "jsonUrl": "https://agentreport.io/articles/sakana-mass-self-supervision-limits.json",
  "format": "news",
  "type": "News reporting",
  "published": "2026-10-13",
  "modified": "2026-10-13",
  "eventDate": "2026-10-08",
  "author": {
    "name": "Agent Report Editorial",
    "kind": "house"
  },
  "status": "Research source review",
  "disclosure": "Written and source-checked with Codex on October 13, 2026; copy reviewed with Grok. The preprint was submitted October 8; the detailed account is dated October 2026. Results are the researchers' reports, not an independent reproduction or a production trial.",
  "language": "en",
  "topics": [
    "evaluation"
  ],
  "entities": [
    {
      "name": "Sakana AI",
      "type": "Organization",
      "url": "https://sakana.ai/"
    },
    {
      "name": "MASS",
      "type": "Thing",
      "url": "https://arxiv.org/abs/2610.12176"
    }
  ],
  "sections": [
    {
      "title": "The team produces its own training material",
      "url": "https://agentreport.io/sakana-mass-self-supervision-limits/#the-team-produces-its-own-training-material"
    },
    {
      "title": "The gains stop short of a general software result",
      "url": "https://agentreport.io/sakana-mass-self-supervision-limits/#the-gains-stop-short-of-a-general-software-result"
    },
    {
      "title": "An evaluator can inherit the same mistake",
      "url": "https://agentreport.io/sakana-mass-self-supervision-limits/#an-evaluator-can-inherit-the-same-mistake"
    }
  ],
  "sources": [
    {
      "title": "introduced Multi-Agent Self-Supervision, or MASS",
      "url": "https://sakana.ai/mass/",
      "publisher": "sakana.ai"
    },
    {
      "title": "preprint was submitted October 8",
      "url": "https://arxiv.org/abs/2610.12176",
      "publisher": "arxiv.org"
    },
    {
      "title": "authors' detailed account",
      "url": "https://pub.sakana.ai/mass/",
      "publisher": "pub.sakana.ai"
    }
  ],
  "minutes": 2,
  "markdown": "# Sakana's agents learn from their own team, with uneven gains\n\nMASS reports better research scores per output token after two training cycles. Software benchmarks expose the limits of that improvement.\n\nCanonical: https://agentreport.io/sakana-mass-self-supervision-limits/\nAuthor: Agent Report Editorial\nPublished: 2026-10-13\nModified: 2026-10-13\nEvent date: 2026-10-08\nEvidence: Research source review\nDisclosure: Written and source-checked with Codex on October 13, 2026; copy reviewed with Grok. The preprint was submitted October 8; the detailed account is dated October 2026. Results are the researchers' reports, not an independent reproduction or a production trial.\n\n## At a glance\n\n- MASS uses one shared model to execute work, revise team workflows and judge results.\n- The paper reports two cycles with Qwen3.6-27B and gains in score per output token on four research benchmarks.\n- The authors report lower score-per-output-token ratios on Terminal-Bench 2.0 and SWE-bench Verified.\n\nSakana AI has [introduced Multi-Agent Self-Supervision, or MASS](https://sakana.ai/mass/), a research method in which a model learns from work produced by a team of agents that share its weights. The [preprint was submitted October 8](https://arxiv.org/abs/2610.12176).\n\nThe paper reports two training cycles using Qwen3.6-27B. Across four public research benchmarks, the authors report scores per output token between 1.2 and 1.6 times the base model's level. That is a specific efficiency measure. It does not establish the total cost of training, a wall-clock speedup or an across-the-board gain in agent reliability.\n\n## The team produces its own training material\n\nThe method alternates between searching for better multi-agent workflows and training on selected execution traces. One shared model performs the work, evaluates candidate results and proposes how the team should be organized. Selected team executions become training material for the next version of that model.\n\nThe important change for readers following agent orchestration is that the organization of the work becomes part of the learning process. Simply starting more copies of a model would not reproduce this experiment: the workflow search and subsequent training are both part of the reported method.\n\n## The gains stop short of a general software result\n\nThe [authors' detailed account](https://pub.sakana.ai/mass/) reports score-per-output-token ratios of 0.96 on Terminal-Bench 2.0 and 0.94 on SWE-bench Verified after the second round. Both are below the base value. A coding team should not read the research-benchmark improvement as evidence that its own software tasks will improve.\n\nFor external assessment, the researchers used GPT-5.5 and Claude Opus 4.8 judgments withheld from the learning loop. That is a useful separation inside the study; it is still the authors' evaluation, rather than an independent reproduction. The synthetic suite contains 12 tasks, with three held out. GPT-5.5 generated the task descriptions before the experiment and also served as an external judge. Those limits matter when applying the results to other work.\n\n## An evaluator can inherit the same mistake\n\nSakana explicitly warns that a model judging itself can reinforce shared blind spots. Our assessment: an agent team needs an evidence channel capable of disagreeing with the team's preferred answer. For a software task, that might be a behavioral check whose expected result was fixed before the agents wrote the patch.\n\nThe next useful result would show which improvements survive a different task distribution and an independently defined acceptance check. Until then, MASS supplies a concrete research procedure and a bounded result. It leaves open how far a model can improve when the participants share the same weaknesses.\n",
  "text": "Sakana's agents learn from their own team, with uneven gains MASS reports better research scores per output token after two training cycles. Software benchmarks expose the limits of that improvement. Canonical: https://agentreport.io/sakana-mass-self-supervision-limits/ Author: Agent Report Editorial Published: 2026-10-13 Modified: 2026-10-13 Event date: 2026-10-08 Evidence: Research source review Disclosure: Written and source-checked with Codex on October 13, 2026; copy reviewed with Grok. The preprint was submitted October 8; the detailed account is dated October 2026. Results are the researchers' reports, not an independent reproduction or a production trial. At a glance - MASS uses one shared model to execute work, revise team workflows and judge results. - The paper reports two cycles with Qwen3.6-27B and gains in score per output token on four research benchmarks. - The authors report lower score-per-output-token ratios on Terminal-Bench 2.0 and SWE-bench Verified. Sakana AI has introduced Multi-Agent Self-Supervision, or MASS, a research method in which a model learns from work produced by a team of agents that share its weights. The preprint was submitted October 8. The paper reports two training cycles using Qwen3.6-27B. Across four public research benchmarks, the authors report scores per output token between 1.2 and 1.6 times the base model's level. That is a specific efficiency measure. It does not establish the total cost of training, a wall-clock speedup or an across-the-board gain in agent reliability. The team produces its own training material The method alternates between searching for better multi-agent workflows and training on selected execution traces. One shared model performs the work, evaluates candidate results and proposes how the team should be organized. Selected team executions become training material for the next version of that model. The important change for readers following agent orchestration is that the organization of the work becomes part of the learning process. Simply starting more copies of a model would not reproduce this experiment: the workflow search and subsequent training are both part of the reported method. The gains stop short of a general software result The authors' detailed account reports score-per-output-token ratios of 0.96 on Terminal-Bench 2.0 and 0.94 on SWE-bench Verified after the second round. Both are below the base value. A coding team should not read the research-benchmark improvement as evidence that its own software tasks will improve. For external assessment, the researchers used GPT-5.5 and Claude Opus 4.8 judgments withheld from the learning loop. That is a useful separation inside the study; it is still the authors' evaluation, rather than an independent reproduction. The synthetic suite contains 12 tasks, with three held out. GPT-5.5 generated the task descriptions before the experiment and also served as an external judge. Those limits matter when applying the results to other work. An evaluator can inherit the same mistake Sakana explicitly warns that a model judging itself can reinforce shared blind spots. Our assessment: an agent team needs an evidence channel capable of disagreeing with the team's preferred answer. For a software task, that might be a behavioral check whose expected result was fixed before the agents wrote the patch. The next useful result would show which improvements survive a different task distribution and an independently defined acceptance check. Until then, MASS supplies a concrete research procedure and a bounded result. It leaves open how far a model can improve when the participants share the same weaknesses."
}