Grounded Skill Synthesis from Code at Scale for Agentic Intelligence
Reusable skills endow agentic systems with transferable procedural knowledge, making scalable skill acquisition essential for extending agents beyond their accumulated experience. Existing skill synthesis methods have two main limitations: trajectory-based skill synthesis inherently coupled to the environments, while document-sourced productions may lack evidence support and verifications. Source code offers a different pathway—it requires no previous experience while retaining executable evidence for grounding later abstractions. Hence, we present Code2Skill, a fully automated pipeline that lifts selected code units into implementation-anchored atomic-operation, composite-workflow, and recurring-pattern skill records, and verifies them through source-body-blind reconstruction followed by source-aware comparison. Applying Code2Skill to 19,769 popular and actively maintained GitHub repositories yields our CodeSkillBank, a grounded skill bank containing 1,006,822 accepted records with workflow, boundary, provenance, and source-evidence metadata.
Introduction. Large foundation models have endowed modern artificial intelligence (AI) systems with sophisticated reasoning capabilities. However, agentic AI must tackle complex, long-horizon tasks that extend beyond what is encoded in model parameters. Beyond generating content following user instructions and reasoning over internal knowledge, agents must interact with external environments, invoke tools appropriately [Yao et al., 2023b, Schick et al., 2023, Qin et al., 2024], learn from failures and feedback [Shinn et al., 2023, Madaan et al., 2023], plan over extended horizons [Yao et al., 2023a, Zhou et al., 2024], coordinate specialized agents [Bansal et al., 2024, Hong et al., 2024], access external memory [Park et al., 2023], and more.
Discussion / Conclusion. We present Code2Skill, a fully automated framework for synthesizing reusable procedural skills from source code. Code2Skill distills selected source units into atomicoperation, composite-workflow, and recurring-pattern skills, and grounds them through source-body-blind reconstruction followed by source-aware comparison. Applying Code2Skill to 19,769 actively developed GitHub repositories yields CodeSkillBank, a large-scale skill bank containing 1,006,822 accepted records. Across nine model settings and eight benchmarks, skills retrieved from CodeSkillBank improve the macro-average score from 42.90 to 47.90 and yield gains in 57 of 72 protocol-matched evaluations, especially outperforms in coding tasks. Under a unified downstream interface, repository-derived skills also outperform the compared trajectory-derived skill banks across all seven shared benchmarks. These results position source code as a scalable and auditable foundation for constructing reusable skill repositories and establish code-based skill synthesis as a promising pathway toward continually expanding the procedural capabilities of agentic systems.
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
When should work require human-AI partnership versus full automation? How should agent systems validate and persist generated code artifacts?- How do agents retrieve and compose skills from hierarchical multimodal wikis?
- How do skill libraries from human resources compare to hand-written skill libraries?
- What makes a distilled skill verifiable and ready for agent execution?
- How does source-blind reconstruction verify that extracted skills are specific enough to be reusable?
- What metadata properties make code-derived skills auditable and comparable to their original source?
- Can curator modules trained on one executor transfer to entirely different agent backbones?
- Why do generic skill descriptions evolve into execution-oriented ones?
- Why do trajectory-based skills fail to transfer across different environments and use cases?
- Can tool adaptation work without freezing the agent in the loop?
- What role does environment diversity play in preventing agents from overfitting to curator imagination?
- Can agentic reasoning outperform rigid rule-based systems for skill refinement?
- How much does agent performance depend on demonstration quantity versus curation quality?
- What infrastructure decouples generation from training in asynchronous agent loops?
- Can agentic AI tools deliver productivity gains on learning tasks differently?
- How do task stream groupings provide long-horizon learning signals for curation decisions?
- Which AI interaction patterns preserve learning while which ones degrade skill formation?
- Does outsourcing tasks to AI reduce opportunities for skill development?