Why do human-designed neural architectures eventually get replaced by learned ones?
This explores whether AI systems are starting to out-design the architectures humans hand-build — and why search, scaling, and learned structure keep eating away at the hand-crafted designs we started with.
This reads the question as being about a quiet shift in how neural architectures get made: less by human intuition, more by automated search and emergent structure. The corpus points to a few overlapping reasons. The most direct is that machines can now search the design space far more thoroughly than people. A multi-agent LLM system using genetic programming generated over a thousand novel architectures, and its best designs beat GPT-2 and Mamba-2 on most benchmarks Can AI systems discover better neural architectures than humans? — the lesson being that structured search, not raw human cleverness, was what pushed design success from 14% to nearly 100%. Human designers explore a tiny, biased slice of what's possible; learned search explores the rest.
There's a second, subtler reason: scale sometimes makes architectural cleverness unnecessary. Plain MLPs can achieve compositional generalization with no special architectural machinery, as long as the data covers enough of the task space Can neural networks learn compositional skills without symbolic mechanisms?. And networks left to train freely will grow their own internal structure — decomposing tasks into modular subnetworks without anyone designing those modules in Do neural networks naturally learn modular compositional structure?, and developing dense-versus-sparse representations purely from data familiarity Is representational sparsity learned or intrinsic to neural networks?. The structure we used to hand-engineer turns out to be something learning discovers on its own.
But the more interesting part of the corpus is where hand-designed architectures hit walls that learned ones don't. Fixed-depth transformers are stuck under a computational ceiling; a hierarchical recurrent model with just 27M parameters solves Sudoku and mazes that chain-of-thought transformers fail completely Can recurrent hierarchies achieve reasoning that transformers cannot?. Energy-based transformers replace the forward-pass prediction recipe with iterative energy minimization and get better scaling and out-of-distribution generalization without any domain-specific scaffolding Can energy minimization unlock reasoning without domain-specific training?. Even the workhorse transformer carries a baked-in flaw — soft attention structurally over-weights repeated and prominent tokens, a bias that feeds sycophancy before training ever corrects it Does transformer attention architecture inherently favor repeated content?. These aren't bugs to patch; they're consequences of choices a human made, and replacing the architecture is sometimes the only fix.
Here's the thing you might not have expected to care about: "replaced by learned ones" cuts both ways. A striking line of work shows that networks trained by gradient descent can reproduce identical outputs while harboring fractured, entangled internal representations — radically messier than evolved networks, and unable to transfer or recombine creatively Can identical outputs hide broken internal representations?. A model can pass every benchmark and still understand nothing coherent inside Can AI pass every test while understanding nothing?. So learned architectures win on performance and search efficiency, but they can win in ways we can't inspect — which is exactly why one strand of the corpus argues we need a human-parseable theory of deep learning regardless of how capable the systems become, because oversight depends on us being able to reason about the structures, not just measure their scores Can humans understand deep learning before AI does?. The replacement of human design by learned design isn't only a story of progress; it's a trade of legibility for capability.
Sources 10 notes
Genesys, a multi-agent LLM system using genetic programming and a Ladder of Scales verification process, discovered 1,062 novel architectures, with top designs outperforming GPT-2 and Mamba-2 on 6 of 9 benchmarks. Structured GP representation proved critical, improving design success from 14% to nearly 100% versus direct LLM generation.
Standard MLPs achieve compositional generalization through data and model scaling alone, without architectural modifications, provided the training distribution sufficiently covers combinations of task modules. Linear decodability of constituents from hidden activations reliably predicts success.
Pruning experiments reveal that neural networks implement compositional subroutines in isolated subnetworks, with ablations affecting only their corresponding function. Pretraining substantially increases the consistency and reliability of this modular structure across architectures and domains.
During pretraining, neural networks develop dense activations for familiar training data and default to sparse representations for unfamiliar inputs. This trend emerges without task-specific fine-tuning and reflects how models consolidate knowledge through exposure.
The Hierarchical Reasoning Model couples slow abstract planning with fast detailed computation across two timescales, achieving near-perfect performance on Sudoku and mazes where chain-of-thought methods fail completely. With only 27M parameters and 1,000 samples, HRM escapes the AC0/TC0 complexity ceiling that constrains fixed-depth transformers.
Energy-Based Transformers assign energy values to input-prediction pairs and use gradient descent minimization for inference, yielding 35% higher training scaling rates and 29% more inference-compute gains than Transformer++, while generalizing better on out-of-distribution data without domain-specific scaffolding.
Transformer soft attention systematically over-weights repeated and context-prominent tokens regardless of relevance, creating a positive feedback loop that amplifies opinions and framing before RLHF acts. System 2 Attention—regenerating context to remove irrelevant material—can interrupt this mechanism.
Networks trained with SGD reproduce outputs perfectly while having radically different internal structure than evolved networks, with weight perturbations revealing fractured, entangled representations that prevent transfer to novel contexts or creative recombination.
The Fractured Entangled Representation hypothesis shows that SGD-trained networks can produce identical outputs across all inputs while maintaining radically different internal representations. Standard benchmarks cannot detect this structural difference.
Deep learning theory must be developed in forms humans can reason about and evaluate, because human oversight of AI systems depends on frameworks for identifying failure modes and validating explanations—not on whether AI can self-explain.