Machine Learning

Gradient Descent – Machine Learning


Gradient Descent is the most used algorithm in Machine Learning. In this article, you will learn how to implement the Gradient Descent algorithm in python. Gradient Descent is a method of minimizing the cost function by an iterative method. In this method, we assume initial weights(theta) and go on minimizing these weights by learning rate. Here the learning rate defines how the weights have to be changed so that the cost function reach the minimum. We go on changing weights until we get minimum value.

What is a Gradient?

 The gradient represents the slope of the tangent of the graph of the function. More precisely, the gradient points in the direction of the greatest rate of increase of the function, and its magnitude is the slope of the graph in that direction. 

Gradient descent

The formula of Gradient Descent Algorithm is :



  1. Start
  2. Assume initial weights theta1 and theta2 and learning rate.
  3. Find new weights ntheta1 and ntheta2.
  4. Update theta1 and theta2 with computed new theta’s.
  5. Repeat Step 3 and Step 4 until you get minimum value.
  6. Stop.




Useful Resources: