转载：DNN VS DBN

https://www.researchgate.net/post/What_is_difference_between_DNN_and_DBN

http://stackoverflow.com/questions/24545725/deep-belief-networks-vs-convolutional-neural-networks

DNN VS DBN

The difference mostly lies in how they are trained. DBN are pretrained to reconstruct the input and then fine-tuned with back-prop, while DNN has a pure supervised training with back-prop. However, I think one could say that DBNs are a subset of DNNs. DNN is a general concept which refers to a neural network with a lot of layers. The pretraining of DBN has shown to be beneficial if the training set is small. 

 

To add further, DBN is formed of stacked Restricted Boltzmann Machines (RBM). Each RBM is a two-level model, a layer with visible units and a layer with hidden units. 

DNN is a totally different concept which refers to neural network with multiple number of hidden layers.

The difference also lies in the fact that the RBMs are unsupervised models while neural networks are not.

 

Another aspect to consider is the use of Convolutional Neural Networks for deep learning. They contain convolutional layers connected to a fully connected MLP, these convolutional layers form the 'deep' aspect. This can be further seen in work of Mnih (2013) which uses a CNN for deep reinforcement learning.

 

DBN VS CNN

Generally speaking, DBNs are generative neural networks that stack Restricted Boltzmann Machines (RBMs) . You can think of RBMs as being generative autoencoders; if you want a deep belief net you should be stacking RBMs and not plain autoencoders as Hinton and his student Yeh proved that stacking RBMs results in sigmoid belief nets.

Convolutional neural networks have performed better than DBNs by themselves in current literature on benchmark computer vision datasets such as MNIST. If the dataset is not a computer vision one, then DBNs can most definitely perform better. In theory, DBNs should be the best models but it is very hard to estimate joint probabilities accurately at the moment. You may be interested in Lee et. al's (2009) work on Convolutional Deep Belief Networks which looks to combine the two.