ADVERTISEMENT
ADVERTISEMENT

Polynomial Regression in Machine Learning with Examples

What is Polynomial Regression?

Polynomial Regression is an extension of Linear Regression, where the relationship between the independent variable (X) and dependent variable (Y) is not a straight line but a curve.

In simple words:
 If Linear Regression draws a straight line, then Polynomial Regression draws a curved line to fit the data better.

When Should We Use Polynomial Regression?

We use Polynomial Regression when:

  • The data does not fit a straight line but follows a curved trend.
  • A linear model (y = mx + b) is too simple and does not give accurate results.
  • We need to capture complex relationships between variables.

Importance of Polynomial Regression

  • Helps in predicting trends when data is non-linear.
  • Provides a better fit when the relationship is curved.
  • Used in real-world applications like stock market prediction, house price prediction, and physics experiments.

What is the Mathematical Formula of Polynomial Regression?

Polynomial regression is an extension of linear regression where the relationship between the input (X) and output (Y) is not a straight line but a curve.

The general form of a quadratic (degree 2) polynomial regression is:

where:

  • Y = Output (dependent variable)
  • X = Input (independent variable)
  • b0​ = Intercept (constant)
  • b1​ = Coefficient of X (linear term)
  • b2​ = Coefficient of X2 (quadratic term)

Understanding Polynomial Degree

The degree of a polynomial determines how curved the line is:

1. Degree 1 (Linear Regression)

Y=b0 + b1X

  • The line is straight.
  • Example: Salary vs. Experience (linear growth).

2. Degree 2 (Quadratic Regression)

Y=b0 + b1X + b2X2

  • The line is a parabola (U-shaped or inverted U).
  • Example: Ball thrown in the air (height vs. time).

3. Degree 3 (Cubic Regression)

Y=b0 + b1X + b2X2 +b3X3

  • The curve has one or more bends.
  • Example: Stock price fluctuations.

4. Higher Degrees (More Complex Curves)

Y=b0 + b1X + b2X2 +b3X3...... + bnXn

  • More degrees increase flexibility but can cause overfitting.
  • Example: Complicated real-world trends.

How to Calculate Coefficients in Polynomial Regression?

To determine the coefficients b0,b1 and b2 in the polynomial equation:

Y=b0 + b1X + b2X2

Step-by-Step Calculation

Step 1: Define the System of Equations

We are given the dataset:

X Y
1 3
2 6
3 11
4 18
5 27

 

For the equation Y=b0 + b1X + b2X2, we substitute values of X and get:

  1.  3=b0+b1(1)+b2(1)2
  2.  6=b0+b1(2)+b2(2)2
  3. 11=b0+b1(3)+b2(3)2
  4. 18=b0+b1(4)+b2(4)2
  5. 27=b0+b1(5)+b2(5)2

Step 2: Convert to Matrix Form

We rewrite the system as:

or simply:

AX=B

where:

  • A is the matrix of input values,
  • X is the vector of coefficients [b0,b1,b2]
  • B is the output values.

Step 3: Solve for b0,b1,b2

We solve the equation AX=B using matrix algebra:

X=A−1B

Let me compute these values for you.

Solving the system, we get:

b0≈2, b1≈0, b2≈1

Thus, the polynomial equation is:

Y=2+X2

This equation perfectly fits the given data

 


ADVERTISEMENT
ADVERTISEMENT