Can specialized agents write better scientific papers than single models?

PaperOrchestra is a multi-agent framework that transforms unconstrained pre-writing materials (idea summaries, experimental logs, optional figures) into submission-ready LaTeX manuscripts including comprehensive literature synthesis and generated visuals. In side-by-side human evaluations against autonomous baselines, it achieves absolute win rate margins of 50-68% on literature review quality and 14-38% on overall manuscript quality.

The architecture decomposes scientific writing into its constituent cognitive tasks and assigns specialized agents to each. This matters because a single LLM attempting the full writing pipeline hits coherence limits — it cannot simultaneously maintain awareness of the literature landscape, the experimental narrative, the theoretical framing, and cross-document consistency. Specialized agents can each optimize for their subtask while structured knowledge exchange maintains coherence across the manuscript.

The benchmark (PaperWritingBench) reverse-engineers raw materials from 200 top-tier AI conference papers, then tests whether autonomous writers can reconstruct submission-quality manuscripts from those materials. Two variants test different user effort levels: Sparse (high-level idea summary only) and Dense (retaining formal definitions and equations). This addresses a real gap — existing autonomous writers are "rigidly coupled to specific experimental pipelines" and produce superficial literature reviews.

The literature review quality gap (50-68%) is particularly significant. Literature review is the task that most requires maintaining a coherent mental model across dozens of papers while synthesizing them into a narrative — exactly the kind of sustained cross-document reasoning where single-model context windows fail. Multi-agent specialization converts this from a single overwhelming context problem into a distributed coordination problem.

This connects to the finding that since Does structured artifact sharing outperform conversational coordination?, PaperOrchestra's structured knowledge exchange between agents is the scientific-writing instance of SOP-encoded coordination outperforming free-form agent collaboration. And since Are multi-agent systems actually intelligent coordination or just token spending?, PaperOrchestra's human evaluation results provide a counterexample where the token cost produces genuine quality gains rather than mere token expenditure — specifically on the literature review subtask where distributed knowledge synthesis has clear structural advantages.

Source: Co Writing Collaboration Paper: PaperOrchestra