Few-Shot Learning

Few-shot learning is a machine learning paradigm where a model learns to make predictions from a very small number of labeled examples per class.

Few-shot learning addresses the challenge of training models when labeled data is scarce. Traditional supervised learning requires large datasets, but few-shot learning aims to generalize from only a handful of examples, often one to five per class. This is achieved through techniques such as meta-learning, where the model is trained on a variety of tasks to learn a general learning strategy that can be quickly adapted to new tasks with minimal data.

Common approaches include metric-based methods, which learn a similarity function to compare new examples with stored prototypes; model-based methods, which use architectures like memory-augmented networks to store and retrieve information; and optimization-based methods, which learn an initialization for model parameters that can be efficiently fine-tuned with a few gradient steps. Few-shot learning is often evaluated on benchmarks like Omniglot or miniImageNet, which contain many classes with few examples each.

The paradigm is closely related to zero-shot learning, where no labeled examples are provided, and one-shot learning, which uses exactly one example per class. Few-shot learning has practical applications in domains where data collection is expensive or impractical, such as medical imaging, rare species identification, and personalized user interfaces.

Why it matters

Few-shot learning matters because it reduces the dependency on large labeled datasets, which are costly and time-consuming to obtain. It enables machine learning models to be deployed in specialized or niche domains where data is inherently limited, such as diagnosing rare diseases or recognizing endangered species. By mimicking human ability to learn from few examples, it also advances research toward more sample-efficient and adaptable AI systems.

Related terms

• Zero-Shot Learning
• In-Context Learning

FAQ

How does it work?

Few-shot learning typically uses meta-learning, where a model is trained on many related tasks to learn a general learning procedure. During meta-training, the model sees episodes each containing a small support set and a query set, learning to adapt quickly. At test time, it applies this learned strategy to a new task with only a few labeled examples.

What is the difference between few-shot and zero-shot learning?

Zero-shot learning requires no labeled examples for the target classes; instead, it uses auxiliary information like semantic attributes or descriptions to generalize. Few-shot learning provides a small number of labeled examples per class, allowing the model to learn from direct supervision, albeit limited. Both aim to handle rare or unseen classes, but few-shot learning offers more concrete guidance.

When should few-shot learning be used instead of traditional supervised learning?

Few-shot learning is appropriate when labeled data is scarce for each class, such as in medical diagnosis with rare conditions or in personalized applications with few user examples. It is also useful when new classes appear frequently and retraining from scratch is impractical. However, if abundant labeled data is available, traditional supervised learning often achieves higher accuracy.