Linear Regression in Supervised Learning
What is Linear Regression?
Linear regression is a supervised learning algorithm used for predicting numerical values. It finds a relationship between an input variable (X) and an output variable (Y) by fitting a straight line through the data points. Linear regression is one of the simplest yet most powerful algorithms in machine learning. It is used to predict continuous values (like prices, temperatures, or scores). This line is called the regression line.
What is the Mathematical Formula for Linear Regression?
The equation of a Linear Regression is:
Y= mX + b
Where:
- Y= Predicted value
- m = Slope of the line (Rate of Change-how much Y changes when X increases)
- X = Input variable
- b= Intercept (where the line crosses the Y-axis)
Difference Between Dependent and Independent Variables
In Linear Regression, we work with two types of variables:
- Independent Variable (X) – The input or cause. It influences the dependent variable
- Dependent Variable (Y) – The output or effect. It changes based on the independent variable
Key Differences:
| Feature | Independent Variable (X) | Dependent Variable (Y) |
|---|---|---|
| Meaning | The variable that influences the outcome | The variable that we predict |
| Control | We change/control it | It depends on X |
| Example | Temperature | Ice Cream Sales |
| Notation | Usually represented as X | Usually represented as Y |
What is the Intercept (b) in the Formula?
The intercept (b) represents the predicted value of Y when X = 0. If the regression line crosses below the Y-axis, then the intercept value will be negative.
- Intercept (b) is the point where the regression line meets the Y-axis (when X=0).
- The slope (m) determines how much Y changes for every 1 unit increase in X.
Here is the Linear Regression graph, where the regression line starts from the intercept (b) at X=0 and b=-200
Example to Understand Linear Regression
Imagine you are a shopkeeper who wants to predict the sales of ice cream based on temperature. You observe that as the temperature increases, the sales also increase. By using linear regression, you can create a model that predicts sales based on temperature.
For example:
| Temperature (°C) | Ice Cream Sales (Rs) |
|---|---|
| 20 | 200 |
| 25 | 250 |
| 30 | 400 |
| 35 | 450 |
| 40 | 600 |
| 45 | 650 |
A linear regression model will find the best straight line that fits this data and allows you to predict sales for any temperature.
Examples of Linear Regression
1. House Price Prediction
Linear regression helps estimate house prices based on features like size, number of bedrooms, and age. The model learns patterns from past sales to predict future prices.
2. Temperature vs Ice Cream Sales
Higher temperatures generally lead to more ice cream sales. A linear regression model can predict expected sales based on temperature trends.
3. Salary Prediction Based on Experience
Companies use regression to determine how work experience affects salary. The model can predict an employee’s expected salary based on years of experience.
4. Advertising Spend vs Sales Revenue
Businesses analyze the relationship between advertising budget and sales. Linear regression helps predict how increasing ad spend can boost revenue.
5. Student Performance Prediction
Regression models estimate student scores based on factors like study hours and attendance. This helps educators identify students who need extra support.
Types of Linear Regression
1. Simple Linear Regression – One independent variable (X) is used to predict the dependent variable (Y).
2. Multiple Linear Regression – Two or more independent variables (X1, X2, …) are used to predict Y.
Where Can We Use Linear Regression?
Linear regression is widely used in different fields, including:
- Finance – Predicting stock prices based on past data.
- Healthcare – Estimating patient recovery time based on age and health conditions.
- Marketing – Forecasting sales based on advertising spend.
- Real Estate – Predicting house prices based on size, location, and number of rooms.
Python Example of Linear Regression
We will train a model to predict ice cream sales based on temperature and visualize it using a regression chart.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
# Step 1: Prepare the Data (Temperature in °C and Ice Cream Sales in Rs)
temperature = np.array([20, 25, 30, 35, 40, 45]).reshape(-1, 1) # Temperature in °C
sales = np.array([200, 250, 400, 450, 600, 650]) # Ice Cream Sales in Rs
# Step 2: Train the Linear Regression Model
model = LinearRegression()
model.fit(temperature, sales)
# Step 3: Make Predictions
predicted_sales = model.predict(temperature)
# Step 4: Visualize the Regression Line
plt.scatter(temperature, sales, color='blue', label="Actual Sales") # Data points
plt.plot(temperature, predicted_sales, color='red', linestyle='--', label="Regression Line") # Best fit line
# Labels and Title
plt.xlabel("Temperature (°C)")
plt.ylabel("Ice Cream Sales (Rs)")
plt.title("Temperature vs Ice Cream Sales Prediction")
plt.legend()
# Show Plot
plt.show()