AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

AuthorsYutao Yang, Junsong Li, Qianjun Pan et al.

arXiv 20262026

TL;DR

AutoSkill turns recurring interaction experience into explicit versioned skills via query-only extraction and hybrid retrieval, enabling training-free lifelong personalization with a growing SkillBank of 1,858 skills from WildChat-1M.

SharePost on XLinkedIn

Read our summary here, or open the publisher PDF on the next tab.

THE PROBLEM

LLM agents fail to turn repeated preferences into reusable skills

AutoSkill targets the failure that interaction experience is seldom consolidated into reusable knowledge, so user habits must be reestablished from scratch in each session.

This breaks long term personalization for writing assistance, planning, and coding, because agents never crystallize stable preferences, constraints, or workflows into durable capabilities.

HOW IT WORKS

AutoSkill — experience-driven explicit skill self-evolution

AutoSkill’s core mechanism chains Query Rewriting, Hybrid Skill Retrieval, Skill Extraction, Skill Management Decision, and Versioned Skill Merging around a persistent SkillBank of SKILL.md artifacts.

You can think of AutoSkill like a personal library card catalog layered beside an LLM: each recurring behavior becomes a labeled, versioned card instead of a forgotten conversation.

This explicit skill memory lets AutoSkill preserve stable styles, policies, and workflows across sessions, something a plain context window or raw retrieval log cannot reliably maintain.

DIAGRAM

Skill-enhanced response generation flow

This diagram shows how AutoSkill rewrites queries, retrieves skills with hybrid dense plus BM25 scoring, and injects them into skill-conditioned response generation.

DIAGRAM

Skill lifecycle and maintenance pipeline

This diagram shows how AutoSkill ingests experience, extracts skill candidates, applies add or merge decisions, and updates the SkillBank with versioned skills.

PROCESS

How AutoSkill Handles a Skill Lifecycle

  1. 01

    Experience ingestion

    AutoSkill ingests dialogue history X_u and user query sequence Q_tu so Skill Extraction can see recent user-only evidence for durable preferences and workflows.

  2. 02

    Skill Extraction from Interaction

    Using Skill Extraction, AutoSkill proposes candidates z_t with fields like name, description, prompt, triggers, tags, examples, and confidence from recent user queries.

  3. 03

    Retrieval-Assisted Skill Management

    AutoSkill applies Skill Management Decision by retrieving nearest neighbors from the SkillBank and choosing add, merge, or discard for each candidate.

  4. 04

    Versioned Skill Merging

    Through Versioned Skill Merging, AutoSkill semantically merges candidates into existing skills, bumps version numbers, and updates the SkillBank without duplicating artifacts.

KEY CONTRIBUTIONS

Key Contributions

  • 01

    Experience-driven lifelong learning framework

    AutoSkill formulates transforming interaction experience into explicit reusable skills and operationalizes this with Skill Extraction and Skill Management Decision over user-specific SkillBanks built from WildChat-1M.

  • 02

    Skill lifecycle with self-evolution

    AutoSkill introduces a full lifecycle covering Query Rewriting, Hybrid Skill Retrieval, Skill Extraction, and Versioned Skill Merging, enabling continual evolution without any parameter updates.

  • 03

    Editable SKILL.md artifacts and deployable system

    AutoSkill designs skills as editable SKILL.md artifacts with version fields like 0.1.34 and ships an open source system with SDK, Web UI, and OpenAI-compatible proxy for practical deployment.

RESULTS

By the Numbers

Conversations count

10243 conversations

+4331 over Chinese GPT-3.5 subset

Total messages

267681 messages

vs English GPT-4 subset

Extracted skills

631 skills

SkillBank size for English GPT-3.5 subset

Avg messages per conversation

32.17 messages

highest average in Chinese GPT-4 subset

AutoSkill is evaluated by building four SkillBank subsets from WildChat-1M, measuring conversations, messages, and extracted skills to show that AutoSkill can induce 1,858 explicit skills from long real-world dialogues.

BENCHMARK

By the Numbers

AutoSkill is evaluated by building four SkillBank subsets from WildChat-1M, measuring conversations, messages, and extracted skills to show that AutoSkill can induce 1,858 explicit skills from long real-world dialogues.

BENCHMARK

SkillBank scale across WildChat-1M subsets

Extracted Skills per subset in AutoSkill’s SkillBank.

KEY INSIGHT

The Counterintuitive Finding

AutoSkill shows that a single English skill, professional_text_rewrite, can reach version 0.1.34 through incremental merges without any model fine-tuning.

This is surprising because many assume stable personalization requires parameter updates, yet AutoSkill achieves rich evolution entirely via external SKILL.md artifacts.

WHY IT MATTERS

What this unlocks for the field

AutoSkill unlocks training-free lifelong personalization where agents accumulate explicit, inspectable skills instead of opaque latent adaptations.

Builders can now ship agents that remember styles, workflows, and policies across sessions via SKILL.md artifacts, while keeping base models frozen and easily swappable.

~12 min read← Back to papers

Related papers

BenchmarkAgent Memory

Active Context Compression: Autonomous Memory Management in LLM Agents

Nikhil Verma

· 2026

Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.

Questions about this paper?

Paper: AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Answers use this explainer on Memory Papers.

Checking…