deep learning:DA降噪自编码源码加注释 ----- C++
琢磨了两天DA算法，还有有3个地方不是很明白，贴出代码，在最后写出我的疑惑，望解答。最初源代码由yusugomori编写。
第一个代码、迭代多少次便重新产生多少次噪声，代码如下：
DA.CPP
#include <iostream> #include <math.h> #include <stdlib.h> #include "DA.h" using namespace std; void start_da(); double sigmoid(double); int binomial(int n,double p); double uniform(double min,double max); int main() { start_da(); return 0; } //开始 void start_da() { srand(0); double learning_rate = 0.1; //学习率 double error_rate = 0.3; //噪声率 int train_N = 10; int test_N = 2; int n_visible = 20; int n_hidden = 5; int train_data[10][20] = { {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0} }; DA da(train_N,n_visible,n_hidden,NULL,NULL,NULL); //第一步、构造DA，并进行赋值初始化W和双向的偏移量 for(int i=0;i<100;i++) { for(int j=0;j<train_N;j++) { da.train(train_data[j],learning_rate,error_rate); //第二步、开始训练数据 } } int test_data[2][20] = { {1,0,1,0,1,0,1,0,1,1,0,0,0,0,0,0,0,0,0,0}, {0,0,0,0,0,0,0,0,0,0,0,1,1,0,1,1,0,1,1,0} }; double reconstruct[2][20]; for(int i=0;i<test_N;i++) { da.reconstruct(test_data[i],reconstruct[i]); //第三步、获得测试数据的重构Z for(int j=0;j<n_visible;j++) { cout << reconstruct[i][j] << " "; } cout << endl; } cout << endl; } void DA::reconstruct(int* x,double* z) { double *y = new double[n_hidden]; get_hidden_values(x,y); get_reconstructed_input(y,z); delete[] y; } //第一步 DA::DA(int n,int n_v,int n_h,double** w,double* hb,double* vb) { train_N = n; n_visible = n_v; n_hidden = n_h; if(w == NULL) { W = new double*[n_hidden]; for(int i=0;i<n_hidden;i++) { W[i] = new double[n_visible]; } double a = 1.0 / n_visible; for(int i=0;i<n_hidden;i++) { for(int j=0;j<n_visible;j++) { W[i][j] = uniform(-a,a); } } } else { W = w; } if(hb == NULL) { hbias = new double[n_hidden]; for(int i=0;i<n_hidden;i++) { hbias[i] = 0; } } else { hbias = hb; } if(vb == NULL) { vbias = new double[n_visible]; for(int i=0;i<n_visible;i++) { vbias[i] = 0; } } else { vbias = vb; } } DA::~DA() { for(int i=0;i<n_hidden;i++) delete[] W[i]; delete[] W; delete[] hbias; delete[] vbias; } //第一步1.1 double uniform(double min,double max) { return rand() / (RAND_MAX + 1.0) * (max - min) + min; } //第二步 void DA::train(int* train_data,double learning_rate,double error_rate) { int *noise_x = new int[n_visible]; //随机加入噪声 double *y = new double[n_hidden]; double *z = new double[n_visible]; double *Learning_vbias = new double[n_visible]; double *Learning_hbias = new double[n_hidden]; double p = 1 - error_rate; get_noise_input(train_data,noise_x,p); //获得噪声后的训练集 get_hidden_values(noise_x,y); //获得隐藏层数据输出 get_reconstructed_input(y,z); //获得重构输入Z for(int i=0;i<n_visible;i++) { Learning_vbias[i] = noise_x[i] - z[i]; vbias[i] += learning_rate * Learning_vbias[i] / train_N; //反向的偏移量更新 } for(int i=0;i<n_hidden;i++) { Learning_hbias[i] = 0; for(int j=0;j<n_visible;j++) { Learning_hbias[i] += W[i][j] + Learning_vbias[j]; } Learning_hbias[i] *= y[i] * (1 - y[i]); hbias[i] += learning_rate * Learning_hbias[i] / train_N; //前向利用BP进行更新偏移量 } for(int i=0;i<n_hidden;i++) { for(int j=0;j<n_visible;j++) { //W[i][j] += learning_rate * (Learning_hbias[i] * y[i]) / train_N; W[i][j] += learning_rate * (Learning_hbias[i] * noise_x[i] + Learning_vbias[i] * y[i]) / train_N; //更新权值 } } delete[] Learning_hbias; delete[] Learning_vbias; delete[] z; delete[] y; delete[] noise_x; } void DA::get_reconstructed_input(double* y,double* z) { for(int i=0;i<n_visible;i++) { z[i] = 0; for(int j=0;j<n_hidden;j++) { z[i] += W[j][i] * y[j]; } z[i] += vbias[i]; z[i] += sigmoid(z[i]); } } void DA::get_hidden_values(int* noise_x,double* y) { for(int i=0;i<n_hidden;i++) { y[i] = 0; for(int j=0;j<n_visible;j++) { y[i] += W[i][j] * noise_x[j]; } y[i] += hbias[i]; y[i] = sigmoid(y[i]); } } double sigmoid(double x) { return exp(x)/(1+exp(x)); } void DA::get_noise_input(int* train_data,int* noise_x,double p) { for(int i=0;i<n_visible;i++) { if(train_data[i] == 0) { noise_x[i] = 0; } else { noise_x[i] = binomial(1,p); } } } int binomial(int n,double p) { if(p > 1 || p < 0) { return 0; } int c = 0; double r; for(int i=0;i<n;i++) { r = rand() / (RAND_MAX + 1.0); if(r < p) { c++; } } return c; }
DA.h
class DA{ public: int train_N; int n_visible; int n_hidden; double **W; double *hbias; double *vbias; DA(int ,int ,int ,double** W,double* hbias,double* vbias); ~DA(); void train(int* ,double ,double); void get_noise_input(int*,int*,double); void get_hidden_values(int*,double*); void get_reconstructed_input(double* ,double*); void reconstruct(int* ,double*); private: };
第一个代码、只产生1次噪声，并把这个噪声数据当作原始数据训练，代码如下：
DA.CPP
#include <iostream> #include <math.h> #include <stdlib.h> #include "DA.h" using namespace std; void start_da(); double sigmoid(double); int binomial(int n,double p); double uniform(double min,double max); int main() { start_da(); return 0; } //开始 void start_da() { srand(0); double learning_rate = 0.1; //学习率 double error_rate = 0.3; //噪声率 int train_N = 10; int test_N = 2; int n_visible = 20; int n_hidden = 5; int train_data[10][20] = { {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0} }; DA da(train_N,n_visible,n_hidden,NULL,NULL,NULL); //第一步、构造DA，并进行赋值初始化W和双向的偏移量 int noise_x[10][20]; double p = 1 - error_rate; for(int i=0;i<train_N;i++) { for(int j=0;j<n_visible;j++) { if(train_data[i][j] == 0) { noise_x[i][j] = 0; } else { noise_x[i][j] = binomial(1,p); } } } for(int i=0;i<train_N;i++) { for(int j=0;j<n_visible;j++) { cout << noise_x[i][j] << "--"; } cout << endl; } for(int i=0;i<100;i++) { for(int j=0;j<train_N;j++) { da.train(noise_x[j],learning_rate,error_rate); //第二步、开始训练数据 } } int test_data[2][20] = { {1,0,1,0,1,0,1,0,1,1,0,0,0,0,0,0,0,0,0,0}, {0,0,0,0,0,0,0,0,0,0,0,1,1,0,1,1,0,1,1,0} }; double reconstruct[2][20]; for(int i=0;i<test_N;i++) { da.reconstruct(test_data[i],reconstruct[i]); //第三步、获得测试数据的重构Z for(int j=0;j<n_visible;j++) { cout << reconstruct[i][j] << " "; } cout << endl; } cout << endl; } void DA::reconstruct(int* x,double* z) { double *y = new double[n_hidden]; get_hidden_values(x,y); get_reconstructed_input(y,z); delete[] y; } //第一步 DA::DA(int n,int n_v,int n_h,double** w,double* hb,double* vb) { train_N = n; n_visible = n_v; n_hidden = n_h; if(w == NULL) { W = new double*[n_hidden]; for(int i=0;i<n_hidden;i++) { W[i] = new double[n_visible]; } double a = 1.0 / n_visible; for(int i=0;i<n_hidden;i++) { for(int j=0;j<n_visible;j++) { W[i][j] = uniform(-a,a); } } } else { W = w; } if(hb == NULL) { hbias = new double[n_hidden]; for(int i=0;i<n_hidden;i++) { hbias[i] = 0; } } else { hbias = hb; } if(vb == NULL) { vbias = new double[n_visible]; for(int i=0;i<n_visible;i++) { vbias[i] = 0; } } else { vbias = vb; } } DA::~DA() { for(int i=0;i<n_hidden;i++) delete[] W[i]; delete[] W; delete[] hbias; delete[] vbias; } //第一步1.1 double uniform(double min,double max) { return rand() / (RAND_MAX + 1.0) * (max - min) + min; } //第二步 void DA::train(int* noise_x,double learning_rate,double error_rate) { double *y = new double[n_hidden]; double *z = new double[n_visible]; double *Learning_vbias = new double[n_visible]; double *Learning_hbias = new double[n_hidden]; get_hidden_values(noise_x,y); //获得隐藏层数据输出 get_reconstructed_input(y,z); //获得重构输入Z for(int i=0;i<n_visible;i++) { Learning_vbias[i] = noise_x[i] - z[i]; vbias[i] += learning_rate * Learning_vbias[i] / train_N; //反向的偏移量更新 } for(int i=0;i<n_hidden;i++) { Learning_hbias[i] = 0; for(int j=0;j<n_visible;j++) { Learning_hbias[i] += W[i][j] + Learning_vbias[j]; } Learning_hbias[i] *= y[i] * (1 - y[i]); hbias[i] += learning_rate * Learning_hbias[i] / train_N; //前向利用BP进行更新偏移量 } for(int i=0;i<n_hidden;i++) { for(int j=0;j<n_visible;j++) { //W[i][j] += learning_rate * (Learning_hbias[i] * y[i]) / train_N; W[i][j] += learning_rate * (Learning_hbias[i] * noise_x[i] + Learning_vbias[i] * y[i]) / train_N; //更新权值 } } delete[] Learning_hbias; delete[] Learning_vbias; delete[] z; delete[] y; } void DA::get_reconstructed_input(double* y,double* z) { for(int i=0;i<n_visible;i++) { z[i] = 0; for(int j=0;j<n_hidden;j++) { z[i] += W[j][i] * y[j]; } z[i] += vbias[i]; z[i] += sigmoid(z[i]); } } void DA::get_hidden_values(int* noise_x,double* y) { for(int i=0;i<n_hidden;i++) { y[i] = 0; for(int j=0;j<n_visible;j++) { y[i] += W[i][j] * noise_x[j]; } y[i] += hbias[i]; y[i] = sigmoid(y[i]); } } double sigmoid(double x) { return exp(x)/(1+exp(x)); } int binomial(int n,double p) { if(p > 1 || p < 0) { return 0; } int c = 0; double r; for(int i=0;i<n;i++) { r = rand() / (RAND_MAX + 1.0); if(r < p) { c++; } } return c; }
DA.h
class DA{ public: int train_N; int n_visible; int n_hidden; double **W; double *hbias; double *vbias; DA(int ,int ,int ,double** W,double* hbias,double* vbias); ~DA(); void train(int* ,double ,double); void get_noise_input(int*,int*,double); void get_hidden_values(int*,double*); void get_reconstructed_input(double* ,double*); void reconstruct(int* ,double*); private: };
我的疑惑：1、噪声需要每次随机更换还是只一次？ 2、权值W的更新公式怎么推到得出，我只推到出注释掉的那个公式？ 3、这个输出Z的误差也太大了点吧。百分之30的误差率？ 求解答。。。。
