Do text-space skills transfer learning across different frontier models?
This explores whether skills carried in text — prompts, instructions, refined training data, in-context examples — actually transfer when you move them to a different frontier model, or whether they only work on the model they were built for.
This reads the question as: do skills that live in text (a sharpened prompt, an instruction-tuning set, teacher-refined examples) port cleanly between models, or are they secretly tied to one model's internals? The corpus answers with a consistent and slightly deflating pattern: text-space skills transfer *only to the extent the receiving model already has the underlying capability latent in it*. Text reorganizes and activates; it rarely installs something new.
The sharpest evidence is that prompting works entirely inside a model's pre-existing training distribution — it can retrieve and reorganize what's already there but cannot supply knowledge the model never learned Can prompt optimization teach models knowledge they lack?. That puts a hard ceiling on text transfer: a prompt that 'works' on Model A may flop on Model B not because the text is worse, but because B lacks the latent piece the text was activating. The same logic shows up in training data. Teacher-refined examples that are objectively higher quality actually *degrade* a student that can't absorb them — refinements only transfer when they fall inside the student's own learning frontier, so the smart move is for the student to filter by its own statistical profile rather than swallow everything Does teacher-refined data always improve student model performance?.
There's also a surprising twist on what text is even teaching. Instruction tuning turns out to transmit *output format* far more than task understanding — models trained on semantically empty or deliberately wrong instructions score about as well as those given correct ones, because what actually transfers is knowledge of the output space Does instruction tuning teach task understanding or output format?. If the portable payload is mostly 'here's the shape of the answer,' transfer across models gets easier for formatting and harder for genuine reasoning. And format itself is model-specific: RL post-training collapses onto a single dominant pretraining format whose identity depends on model scale, not performance — and that winning format is largely hidden when you start from a proprietary base Does RL training collapse format diversity in pretrained models?. So a text recipe tuned on one model can collide with a different format basin on another.
What *does* travel well is procedural, not factual. Reasoning generalizes because it leans on broad, transferable procedural knowledge spread across many pretraining documents, whereas factual recall depends on narrow, document-specific memorization Does procedural knowledge drive reasoning more than factual retrieval?. That predicts which text-space skills should port across frontier models: a how-to-reason scaffold or trajectory has a better chance than a fact-laden prompt. In-context learning hints at the mechanism — models pick up sequential decision-making from *trajectories* in the prompt, generalizing across very different tasks with no weight updates at all, but only when the context carries same-environment trajectory structure ('burstiness'), not isolated examples Why do trajectories matter more than individual examples for in-context learning?. Transfer rides on structure, and a single well-placed example can flip a latent skill on — one RLVR example lifts math from 36% to 73.6% Can a single training example unlock mathematical reasoning?, reinforcing that text is an activation key, not a download.
The most interesting cross-model evidence comes from the corpus's predictor and from approaches that *aren't* purely text-space. Keyword priming after learning is predictable from pre-learning probability — and that predictive threshold (~10^-3) holds *across architectures and model sizes*, suggesting you can forecast whether a given textual nudge will take on a new model before you try it Can we predict keyword priming before learning happens?. Meanwhile, the cleaner cross-model generalization shows up below the text layer: a verbosity-control vector extracted from 50 examples generalizes across model sizes and domains while staying training-free Can we steer reasoning toward brevity without retraining?, and expert skills compose at inference by tuning singular values of the weights Can models dynamically activate expert skills at inference time?. The thing you didn't know you wanted to know: the field is quietly migrating 'skills' out of text and into activation- and weight-space precisely because text transfer is bottlenecked by whatever the target model already happens to know.
Sources 10 notes
Prompting works entirely within a model's pre-existing training distribution and cannot supply domain knowledge absent from training data. This creates a hard ceiling: no prompt strategy can compensate for missing foundational knowledge, only reorganize what already exists.
Teacher-refined data degrades performance when it exceeds the student's learning frontier, even if objectively higher quality. Students should filter refinements using their own statistical profile to retain only compatible improvements.
Models trained on semantically empty or deliberately incorrect instructions achieve comparable performance to those trained on full correct instructions, achieving 43% vs random baseline 42.6%. The semantic content of instructions appears largely irrelevant; what transfers is knowledge of the output space.
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
In-context learning for sequential decision-making requires full or partial trajectories from the same environment level, not just isolated examples. This structural property—trajectory burstiness—allows models to generalize across vastly different tasks without weight updates.
A single example in RLVR boosts math performance from 36% to 73.6% and enables test accuracy to improve for 1,400 steps after training accuracy reaches 100%, revealing that minimal activation signals unlock latent reasoning capability.
Pre-learning keyword probability strongly predicts post-learning priming across architectures and model sizes, with a ~10^-3 threshold separating contexts where priming occurs from those where it doesn't. Just 3 training exposures suffice to establish the effect.
Activation-Steered Compression extracts a single vector from 50 paired examples to reduce chain-of-thought length by 67% while maintaining accuracy and achieving 2.73x speedup. The method is training-free and generalizes across model sizes and domains.
Transformer2 demonstrates that tuning only singular values within weight matrices produces composable expert vectors that dynamically mix at inference without interference, outperforming LoRA with fewer parameters and enabling continual specialization.