What is In-Context Learning (ICL)?
TL;DR
The ability of AI to instantly learn new tasks from examples and context provided in a prompt. A revolutionary LLM capability.
In-Context Learning (ICL): Definition & Explanation
In-Context Learning (ICL) is the ability of LLMs to instantly 'learn' how to perform new tasks from examples and context provided within the prompt, without any additional training (fine-tuning). Few-shot prompting is one form of ICL, where task adaptation occurs entirely within the input text without updating model parameters. First highlighted significantly with GPT-3, this capability is a key explanation for why LLMs are so versatile. ICL accuracy tends to improve with larger model sizes, closely relating to scaling laws.