Logistic回归,梯度上升算法理论详解和实现
经过对Logistic回归理论的学习,推导出取对数后的似然函数为
现在我们的目的是求一个向量,使得
最大。其中
对这个似然函数求偏导后得到
根据梯度上升算法有
进一步得到
我们可以初始化向量为0,或者随机值,然后进行迭代达到指定的精度为止。
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
1 def sigmoid(inX): 2 return 1.0/(1+exp(-inX))
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
1 def gradAscent(dataMatIn, classLabels): 2 dataMatrix = mat(dataMatIn) #convert to NumPy matrix 3 labelMat = mat(classLabels).transpose() #convert to NumPy matrix 4 m,n = shape(dataMatrix) 5 alpha = 0.001 6 maxCycles = 500 7 weights = ones((n,1)) 8 for k in range(maxCycles): #heavy on matrix operations 9 h = sigmoid(dataMatrix*weights) #matrix mult 10 error = (labelMat - h) #vector subtraction 11 weights = weights + alpha * dataMatrix.transpose()* error #matrix mult 12 return weights