Concept Beginner to Intermediate · 3 min read

What is in-context learning in LLMs

Q: What is in-context learning in LLMs

In-context learning in large language models (LLMs) is the ability to perform tasks by conditioning on examples or instructions provided directly in the input prompt, without any model fine-tuning or weight updates. The model uses the prompt context to generate relevant outputs dynamically based on the examples it sees.

Quick answer

In-context learning in large language models (LLMs) is the ability to perform tasks by conditioning on examples or instructions provided directly in the input prompt, without any model fine-tuning or weight updates. The model uses the prompt context to generate relevant outputs dynamically based on the examples it sees.

In-context learning is a capability of large language models (LLMs) that allows them to learn and perform new tasks by conditioning on examples or instructions provided within the input prompt, without changing their underlying parameters.

How it works

In-context learning works by providing a language model with a prompt that includes task instructions and examples. The model treats the prompt as context and generates outputs conditioned on it. Think of it like a student who reads a few solved problems and then solves a new problem using the patterns observed, without needing to attend a class or update their knowledge permanently.

Technically, the model uses its attention mechanism to weigh the prompt tokens and generate predictions based on the patterns it infers from the examples. This is different from traditional training where model weights are updated; here, the model's parameters remain fixed.

Concrete example

Here is a simple example using the gpt-4o model from OpenAI's API to perform in-context learning for a sentiment classification task by providing labeled examples in the prompt:

python

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

prompt = (
    "Classify the sentiment of the following sentences.\n"
    "Example 1: 'I love this product!' -> Positive\n"
    "Example 2: 'This is the worst experience ever.' -> Negative\n"
    "Example 3: 'The movie was okay, not great.' -> Neutral\n"
    "Sentence: 'I am very happy with the service.' ->"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}]
)

print(response.choices[0].message.content)

output

Positive

When to use it

Use in-context learning when you want to quickly adapt a large language model to a new task without retraining or fine-tuning. It is ideal for few-shot learning scenarios, prototyping, or when you have limited labeled data.

Do not use it when you need consistent, large-scale deployment with strict accuracy guarantees or when you require the model to retain knowledge permanently across sessions. In those cases, fine-tuning or training specialized models is better.

Key terms

Term	Definition
In-context learning	Learning from examples or instructions provided in the input prompt without updating model weights.
Prompt	The input text given to the model that includes instructions and examples.
Few-shot learning	Learning a task from a small number of examples.
Attention mechanism	A neural network component that allows the model to focus on relevant parts of the input.
Fine-tuning	Updating model weights by training on task-specific data.

✅

Key Takeaways

In-context learning enables LLMs to perform new tasks by conditioning on prompt examples without retraining.
It is best for quick adaptation and few-shot learning but not for permanent knowledge updates.
Providing clear, relevant examples in the prompt is critical for effective in-context learning.
The model’s parameters remain fixed; learning happens dynamically during inference.
Use in-context learning to prototype or handle tasks with limited labeled data.

Verified 2026-04 · gpt-4o

Verify ↗