# Neural Networks

## Is random weight assignment better than assigning same weights to the units in the hidden layer?
Because of the symmetry problem, all the units will get the same values during the forward propagation. This also will bias you to a specific local minima. 
https://stackoverflow.com/questions/20027598/why-should-weights-of-neural-networks-be-initialized-to-random-numbers
https://medium.com/usf-msds/deep-learning-best-practices-1-weight-initialization-14e5c0295b94
 
## Why is gradient checking important? 
Gradient checking can help to find bugs in a backpropagation implementation, it is done by comparing the analytical gradient and the numerical gradient computed with calculus.
https://stackoverflow.com/questions/47506521/what-exactly-is-gradient-checking
http://cs231n.github.io/optimization-1/

## What is the loss function in a NN?
The loss function depends on the type of problem:
Regression:  Mean squared error
Binary classification: Binary cross entropy
Multiclass: Cross entropy
Ranking: Hinge loss

## There is a neuron in the hidden layer that always has a large error found in backpropagation. What can be the reason?
It can be either the weight transfer from the input layer to the hidden layer for that neuron is to be blamed or the activation function for the neuron should be changed.
https://medium.com/@karpathy/yes-you-should-understand-backprop-e2f06eab496b