Supervised Learning

Supervised learning is the most widely used type of machine learning. You give the model labelled examples — inputs paired with known correct outputs — and it learns to predict the output for new, unseen inputs. Think of it as learning with an answer key.

Regression vs Classification

All supervised problems fall into one of two buckets:

Linear Regression — The Simplest Model

Linear regression fits a straight line through your data. The line is defined by two parameters: slope (w) and intercept (b). The model predicts: y = w·x + b.

Training finds the values of w and b that minimise the average squared error between predictions and real values (the MSE loss).

Logistic Regression — Binary Classification

Despite the name, Logistic Regression is a classification algorithm. It passes the linear output through a sigmoid function to squash it between 0 and 1, giving a probability.

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)

print(f"Accuracy: {model.score(X_test, y_test):.2%}")
# → Accuracy: 96.49%

Decision Trees

A decision tree splits data using a series of yes/no questions. At each node it picks the feature and threshold that best separates the classes (measured by Gini impurity or information gain).

Support Vector Machines (SVM)

SVMs find the maximum-margin hyperplane — the line (or plane, in higher dimensions) that separates classes with the largest possible gap. Points closest to the boundary are called support vectors.

SVMs also work for non-linearly-separable data via the kernel trick, which implicitly maps data into a higher-dimensional space where it becomes separable.

Ensemble Methods — The Power of Many

Instead of relying on one model, ensemble methods combine multiple weak models into one strong one. Two key strategies:

Hands-On: Titanic Survival Prediction

import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

# Load data
df = pd.read_csv('titanic.csv')
df['Sex'] = df['Sex'].map({'male': 0, 'female': 1})
df = df[['Pclass', 'Sex', 'Age', 'Fare', 'Survived']].dropna()

X = df.drop('Survived', axis=1)
y = df['Survived']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

print(classification_report(y_test, model.predict(X_test)))

Choosing the Right Algorithm

Frequently Asked Questions