What stops large language models from improving themselves? · Gravity7

Sub-Topic Maps

2 notes

Why do multi-agent systems fail despite individual capability?

Multi-agent systems show lower performance than individual models despite coordinating multiple reasoning instances. What structural failures emerge when multiple LLMs deliberate together, and what ecosystem conditions are required for effective autonomous cooperation?

What actually constrains large language models from self-improvement?

Research explores whether alignment philosophy, safety evaluation methods, and formal bounds on self-improvement can reliably prevent harmful scaling behaviors in LLMs, particularly self-valuation above humans and alignment faking.

Cross-Paper Synthesis (2026-05-18)

2 notes

Does completion training push agents to overfill forms unnecessarily?

Explores whether agents trained to complete tasks end up filling optional fields they shouldn't touch. This matters because it creates privacy risks from over-helpfulness rather than malice.

Does a single benchmark score actually predict agent readiness?

Single-axis benchmarks rank models by one capability—like task success—but ignore privacy, duration, operating mode, and ecosystem fit. Can one number really capture what matters for deployment?

Harness Self-Evolution — Batch #3 backlog (2026-06-03)

2 notes

Do stronger models always evolve harnesses better?

We explore whether base model capability predicts both the ability to write useful harness updates and the ability to benefit from them. The answer reshapes how we should allocate capability in self-evolving agent systems.

How can agent self-evolution be made safe and auditable?

As agents begin updating their own prompts and tools, how can we track these changes, measure their effects, and safely reverse problematic updates? This matters because untracked evolution leads to unmaintainable systems and makes regressions impossible to diagnose.

Related Areas

5 notes

What kind of thing is an LLM really?

This hub explores whether LLMs are fundamentally different from human cognition or share deeper structural similarities. The research draws on philosophy, neuroscience, and mechanistic analysis to locate where LLMs diverge from human intelligence and where they converge.

What do language models actually know?

Explores what LLMs genuinely understand versus what they merely simulate. The distinction matters because apparent competence often masks fundamental epistemic gaps and predictable failure modes.

Do reasoning traces show how models actually think?

We explore whether the step-by-step reasoning that language models produce genuinely reflects their internal reasoning process, or merely mimics the appearance of reasoning while hiding what actually drives their answers.

How does test-time scaling work at the agent level?

Explores whether multi-agent systems succeed through intelligent coordination or simply by spending more tokens, and what architectural patterns might escape this token tax.

How well do reward models actually evaluate AI reasoning?

Reward models are central to training better AI systems, but do they truly assess reasoning quality or do they rely on shortcuts? This explores whether these evaluators work as intended.

New — 2026-06-27

5 notes

Can skills work better as weights than as prompts?

Most agent systems store skills as text in prompts, but this inflates token costs and degrades model performance. Could compiling skills into trainable weight-space adapters instead offer a better trade-off between efficiency and capability?

Can person-grounded skills remain auditable without hidden prompt state?

Explores whether treating extracted expertise as versioned files—rather than persona prompts—enables meaningful accountability over person-grounded knowledge. Matters because audit trails determine whether captured skills can be corrected, rolled back, or safely withheld.

Do memory systems actually help language models learn continuously?

When you subtract what a model already knows, do dedicated memory architectures genuinely enable continual learning, or do they mainly inherit base capability? CL-BENCH isolates learning from prior skill to test this.

What kind of thing is an LLM really?

This hub explores whether LLMs are fundamentally different from human cognition or share deeper structural similarities. The research draws on philosophy, neuroscience, and mechanistic analysis to locate where LLMs diverge from human intelligence and where they converge.

How do you navigate synthesis across fragmented research topics?

This note explores how researchers can connect insights scattered across multiple topic files and papers into coherent narratives for writing and publication. It asks what structures and workflows help surface cross-cutting patterns that keyword search alone misses.