Jadwal Sholat

Memuat jadwal sholatโ€ฆ

Pendidikan editorial

Open AccessOA2022

Learning Curves for Decision Making in Supervised Machine Learning: A Survey

This survey categorizes learning curve models in supervised machine learning by decision-making situation, intrinsic question, and resource type. It highlights applications in data acquisition, early stopping, and model selection, and proposes a framework to classify existing approaches.
F. Mohr; J. V. van Rijnยท Machine-mediated learningยท 2022ยท DOI 10.1007/s10994-024-06619-7

The core problem

Learning curves, originally from social sciences, have been adopted in machine learning to assess algorithm performance relative to resources such as training examples or iterations. They are crucial for decision-making in data acquisition, early stopping, and model selection. For instance, learning curves can model the performance of an algorithm-hyperparameter combination, providing early insights into suitability and expediting algorithm selection. Various learning curve models have been proposed: some answer binary questions like whether an algorithm at a certain budget will outperform a reference performance, while others predict the entire learning curve. This survey contributes a framework that categorizes learning curve approaches using three criteria: the decision-making situation they address, the intrinsic learning curve question they answer, and the type of resources they use. The authors survey literature and classify papers into this framework.

Innovation

The authors conduct a systematic literature survey on learning curves in supervised machine learning. They propose a framework to categorize existing approaches based on three dimensions:

1. **Decision-making situation**: The context in which the learning curve is used, such as data acquisition, early stopping, or model selection.
2. **Intrinsic learning curve question**: The specific question the model answers, e.g., binary performance comparison or full curve prediction.
3. **Type of resources**: The resource considered, such as number of training examples or training iterations.

Papers from the literature are surveyed and classified into this framework. The framework helps in understanding the landscape of learning curve models and their applications.

Introduction
Learning curves, originally from social sciences, have been adopted in machine learning to assess algorithm performance relative to resources such as training examples or iterations. They are crucial for decision-making in data acquisition, early stopping, and model selection. For instance, learning curves can model the performance of an algorithm-hyperparameter combination, providing early insights into suitability and expediting algorithm selection. Various learning curve models have been proposed: some answer binary questions like whether an algorithm at a certain budget will outperform a reference performance, while others predict the entire learning curve. This survey contributes a framework that categorizes learning curve approaches using three criteria: the decision-making situation they address, the intrinsic learning curve question they answer, and the type of resources they use. The authors survey literature and classify papers into this framework.

Why it matters

The authors discuss the implications of their framework for both practitioners and researchers. The categorization helps in selecting appropriate learning curve models for specific decision-making situations. It also identifies areas where further research is needed, such as developing more accurate models for full curve prediction and extending learning curves to other resource types. The survey emphasizes the importance of learning curves in efficient machine learning workflows, enabling early decisions that save computational resources and time. The framework provides a structured way to compare and contrast different approaches, facilitating the choice of suitable methods for given applications.

Who should read this

Pembaca Research Digest dan praktisi rumpun ini.

Opening member contentโ€ฆ