## Introduction to Linear Regression

### What is Linear Regression?

Linear regression is a supervised machine learning approach for modeling a linear relationship between a dependent variable and one or more independent variables. In very simple words, it is an approach to create a straight line or a model from discrete values and generate an output which is continuous. It is used to solve classification problems. let's take an example sample training data set to predict the house prices.
Size in Feet(x) Price(\$ in 1000s) (Y)
1000450
1500520
850340
...... Linear Regression line is a straight line going through all points(may or may not overlap with points).The equation of a straight line is y=mx + c where m is the slope and c are the constant.In the linear regression model, slope will become the Weight and the constant will act as bias. The basic model will be ho=Weight*x + bias. where ,
W is the weight
b is the bias
x is "input" variable or features
y is "output" Variable or "target" variable
let's build a program in tensorflow.
 import numpy as npimport tensorflow as tf# declaring and initializing Weights and biasWeight=tf.Variable([.3], dtype=tf.float32)bias=tf.Variable([-.3], dtype=tf.float32)# defining x and y parameters and making it placeholder since at first we don't know it's valuex=tf.placeholder(tf.float32)linear_model=Weight * x + biasy=tf.placeholder(tf.float32) Now, we need to predict Weight and bias. To predict the Weight and bias, we will use the training data.

## Cost Function

A cost function helps us to fit the best straight line to our data.It is also called Squared error function.It is denoted by J, J(Weight,bias)

J(Weight,bias)=(1/(2m))\sum_{i=1}^m (h_o(x^i) -y^i)^2

We can measure the accuracy of our hypothesis function by using a cost function. This takes average difference of all results of the hypothesis with inputs from x's and the actual output y's
h_o(x^i) is the predicted value and
y^i is the actual value.
therefor this function is called "Squared error function" or "mean squared error". The mean is halved (1/2) as a convenience for the computation of the gradient descent, as the derivative term of the squared function will cancel out (1/2) term.
Now the objective is to minimize this to find accurate value of Weight and bias. In order to achieve all this in tensorflow, we can use tensorflow library functions
 # In order to calulate lossloss=tf.reduce_sum(tf.square(linear_model - y)) # calculates sum of the squares# optimizeroptimizer=tf.train.GradientDescentOptimizer(0.01)train=optimizer.minimize(loss)
tf.reduce_sum(input_tensor) : This function adds the elements across dimensions of a tensor.So, it acts as summation \sum_{}
tf.train.GradientDescentOptimizer() : This function is an optimizer which uses the gradient descent algorithm.Gradient descent is a first-order iterative optimization algorithm for finding the minimum of a function.

## Training data and Training loop

 # training valuesx_trainingData=[1,2,3,4]y_trainingData=[0,-1,-2,-3]# traininginit=tf.global_variables_initializer() # To initialize all Variablesess=tf.Session()sess.run(init) # reset values to wrong# loopfor i in range(1000):    sess.run(train,{x:x_trainingData, y:y_trainingData})
while training we have to feed all training data repeat this task

## Calculate training accuracy

 curr_Weight, curr_bias, curr_loss=sess.run([Weight, bias, loss],{x:x_trainingData, y:y_trainingData})print("W: %s b: %s loss: %s"%(curr_Weight, curr_bias, curr_loss)) We need top get weight, bias and loss, and print accordinly.

## Final Program

 import numpy as npimport tensorflow as tf# declaring and initializing Weights and biasWeight=tf.Variable([.3], dtype=tf.float32)bias=tf.Variable([-.3], dtype=tf.float32)# defining x and y parameters and making it placeholder since at first we don't know it's valuex=tf.placeholder(tf.float32)linear_model=Weight * x + biasy=tf.placeholder(tf.float32)# In order to calulate lossloss=tf.reduce_sum(tf.square(linear_model - y)) # calculates sum of the squares# optimizeroptimizer=tf.train.GradientDescentOptimizer(0.01)train=optimizer.minimize(loss)# training valuesx_trainingData=[1,2,3,4]y_trainingData=[0,-1,-2,-3]# traininginit=tf.global_variables_initializer() # To initialize all Variablesess=tf.Session()sess.run(init) # reset values to wrong# loopfor i in range(1000):sess.run(train,{x:x_trainingData, y:y_trainingData})curr_Weight, curr_bias, curr_loss=sess.run([Weight, bias, loss],{x:x_trainingData, y:y_trainingData})print("W: %s b: %s loss: %s"%(curr_Weight, curr_bias, curr_loss))

## Output

 W: [-0.9999969] b: [ 0.99999082] loss: 5.69997e-11