What Is Supervised Learning?

Supervised learning is a machine learning (ML) training method that requires labeled data. The labels indicate the desired output for each input, so the supervised learning model learns to map inputs to specific outputs. The goal is to teach the model to generalize so that it can predict outputs for unknown inputs. For example, a supervised learning task for predicting credit approval uses a dataset with features like credit score, income, employment history, and labels indicating whether the credit was approved or denied. The model learns from this data to predict credit approval for new applications based on the observed patterns.

Machine learning models or algorithms must first be trained with datasets relevant to the task. You use datasets of known inputs and outputs so that the algorithm learns the mathematical relationship between them. Once the model has been trained, tested, and validated, it will produce accurate output based on new, unknown inputs. To produce accurate results with a strong degree of confidence, the supervised learning dataset should be clean and large enough to capture the diversity of real-world data.

Supervised learning data

The original supervised learning dataset is a set of data points, each with one or more inputs and one or more corresponding outputs. This original dataset is split into three separate sets:

• Training data
• Testing data
• Validation data

In the training and validation data sets, each data point has the inputs with the original outputs labeled alongside. The testing dataset has the outputs stripped and set to the side.

Labeled data sets are available for free or purchase on many websites. Organizations can also hire humans to label the data.

Supervised learning process

Data scientists first choose and configure the supervised machine learning model. Then, they feed the model with the training dataset. The model processes the data set and mathematically computes a relationship between the given inputs and labeled outputs.

Next, data scientists feed the testing dataset as input to see if the trained model outputs match the original labeled outputs. They use the validation dataset to create another trained model and compare the results of both supervised machine learning algorithms with the test data.

There are some types of machine learning models or algorithms that you must train with labeled data if you want to use them. They are called supervised learning algorithms or supervised machine learning models. They can be broadly grouped into two categories: classification and regression.

Classification

Supervised machine learning techniques related to classification predict categorical labels, such as whether an email is spam or not spam, by assigning input data into specific categories. We give some examples below.

Binary classification

Binary classification assigns an individual to one of two predefined and mutually exclusive classes based on the individual's attributes. A medical diagnosis of whether an individual has a disease or not based on the results of diagnostic tests is an example of binary classification.

Multiclass classification

Multiclass classification assigns an individual to one of several classes based on the individual's attributes. An example is predicting the topic most relevant to a text document. A document may be classified as being about religion, politics, or finance or as about one of several other predefined topic classes.

Logistic regression

Logistic regression determines the likelihood of the input data belonging to a certain classification, whether binary (e.g., yes, no) or multiclass (e.g., apple, orange, pear).

Decision trees

Decision trees are based on if-then-else structures. For example, if a person has a history of paying on time and multiple references, they will also be likely to vacate with notice.

Random Forest

A random forest uses multiple decision trees to arrive at more accurate results than single decision tree-based supervised learning models.

Support vector machines

A support vector machine is a more accurate way to fit classification data that may not conform to typical logistic regression data spread. It works by transforming the dataset into higher dimensions before processing it.

Regression

Regression models or algorithms produce continuous numeric outputs rather than categories, such as temperatures, lengths, and costs. There are supervised learning models for both single-output regression and multi-output regression.

Linear regression

Linear regression uses a straight line for data fit and is considered one of the most simple and fundamental machine learning algorithms.

Polynomial regression

As the name suggests, polynomial regression fits data to a polynomial curve.

Decision trees

While decision trees can be used for classification tasks, they are also applicable to regression tasks.

Neural networks

Neural networks are machine learning algorithms often used for multi-output regression tasks. Their hidden layer nodes fit these more complex relationship modeling tasks well.

Supervised learning requires input data with corresponding labeled output data. In contrast, unsupervised learning models train on input data without specific outputs. Instead, the model finds patterns and relationships in and between the data. The types of models used in unsupervised learning include clustering, association rule learning, and probability density.

Semi-supervised learning combines both supervised and unsupervised learning techniques. Here, there is a smaller amount of already labeled data plus a larger, unlabeled dataset without defined outputs. The model first trains on the unlabelled dataset and then uses the labeled dataset to further refine output modeling.

For an in-depth explainer, visit What’s the Difference Between Supervised and Unsupervised Learning?

Depending on the task, it may be challenging to find large, representative datasets or create them. The larger and more representative a dataset is, the more accurate the supervised learning results will be and the lower the bias in the results. For instance, facial recognition tasks trained on one race will not be able to recognize faces from other populations accurately.

Unlabeled data

When datasets are not yet labeled or have a lot of unlabeled data, this task can take a significant amount of time, depending on the complexity of labeling and the size of the dataset. Human labeling tasks may require expert domain knowledge in a particular area, which can be difficult to source.

Model selection

Choosing the right supervised learning model for the task at hand is also important. Depending on your use case, some supervised machine learning algorithms are more accurate and cost-efficient. While more basic models can be created, trained, and run with little time and investment, other more complex models require more configuration and cost more at each stage.

Inconsistent results

Output quality in supervised learning may be inconsistent due to overfitting and underfitting. Overfitting occurs when models follow the patterns in training data too accurately, producing inconsistent results with new data. Underfitting occurs when a model that is too simple for the data is used, and thus, it also produces inaccuracies. For example, most real-world data will not fit a linear regression model.

Model adaptability

Sometimes, labels may no longer be accurate due to real-world changes. For example, in a model designed to predict creditworthiness, the economic indicators and personal employment status used to train the model might shift significantly due to economic recessions or booms. If the data shifts slightly, the original dataset and supervised learning training may no longer apply. It requires model-finetuning or retraining to ensure the accuracy of results.

Amazon SageMaker is a fully managed service that prepares data and builds, trains and deploys supervised machine learning (ML) models for any use case with fully managed infrastructure, tools, and workflows. It provides several built-in supervised learning algorithms that can be used for either classification or regression problems. For example:

• AutoGluon-Tabular is an open-source AutoML framework that succeeds by ensembling models and stacking them in multiple layers.
• Factorization Machines Algorithm is an extension of a linear model designed to capture interactions between features within high-dimensional sparse datasets economically.
• LightGBM implements the gradient-boosted trees algorithm, adding two novel techniques for improved efficiency and scalability for supervised learning.
• DeepAR Forecasting Algorithm is a supervised learning algorithm for forecasting scalar (one-dimensional) time series using recurrent neural networks (RNN).
• Object2Vec Algorithm is a new, highly customizable multi-purpose algorithm for feature engineering. It can learn low-dimensional dense embeddings of high-dimensional objects to produce features that improve training efficiencies for downstream models. While this is a supervised algorithm, as it requires labeled data for training, there are many scenarios in which the relationship labels can be obtained purely from natural clusterings in data without any explicit human annotation.

Amazon SageMaker Ground Truth allows you to set up labeling jobs, choose human labelers, and create your own data sets for supervised machine-learning tasks.

Get started with supervised learning on AWS by creating a free account today.