史上最直白的logistic regression教程整理稿。讲4篇博文整理成一个完整的pdf文档。且修改成学术语境。OLoss0;=02(01-(2x+…-20kx2,k)(m2-(01+1xn,1+…+1kxn,k)2/0tl4[(91-(04+1x1,1+…+k1,k)x1,y+(y2-(01+:21+…+k2k)2y+(13)(mh2-(01+1n,1+…+k2xn,k)xn∑(-(1+m1+i2, k))2=1∑(z)+∑W要把式(13)改写成矩阵形式。令y1,y2,…,yn](14)1,k2,k(15)1n,1&n k显然,X∈Rmn×(k式(13)的最后一个等式中的∑就可以写成J2 m2(16)同理,该式中的可以写成WX27,所以,式(13)最终可以表示成OLe(Y-WX8根据式(18),对W而言,矩阵化的计算方式就是:∧W=(Y-WX1)X19所以,以梯庋下降法计算最优W的更新公式是:W+△W=W+(Y-WX2)3线性回归求解线性回归求解就是以代码实现(20)。本文用 Python2.7实现,需要安装 matplotlib和nmpy代码如下4#!/usr/bin/env pythonding: utf-8import matplotlib. pyplot as pltfr。 m numpy import*#load datadef load dataset on=100X =[[1,0.005+xi] for xi in range(1: 100)]X1for xi inX, Y#grad descent solove linear regressiondef grad_descent(X, Y)row, col shape(x)alpha0.001maxiter=5000w= ones((1, col))for k in range(maxIter):W=W+alpha * (Y -w*x transpose())*Xdef mainoⅩ,Y=1oad_ dataset()w grad_descent(X, Y)Pfdraw linear regression result[xi[1] for xi in X]plt. plotarkerxM= mat(X)y2 =W*xM transpose()22=[y2[0n range(y2 shape [1])1plt plot(x, y22, markeIt. showO)it namemain imain o在梯度下降求解函数里多了两个变量, alpha是学习速率,一般取0.001~0.01,太大可能会导致震荡。 maxIter是最大迭代次数,它决定结果的精确度,通常是越大越好,但越大越耗时。不同 maxIter的拟合效果如以下各图Figure 1: maxIter=5oEFigure 2: maxIter=5(Igure0Figure 4: maxIter=10001.0Figure 5: maxIter=50004 Logistic Regession在线性拟合的基础上, Logistic regressionLogistic regression的拟合形式如下zi= wa(22)其中,f(2)是 Logistic函数根据该两式则有Wi=f(wii)Logistic Regression对应的损失函数为1∑(m-f(wa:)如前,以梯度下降法求解。OLoss∑((v-f(Wx;)(of(w. idu(25∑(y-fWz)(-1)af(ei a0x;0O(2等价于f(2),囚为只有一个自变量x对式(1)求导,可得:f(2)(26)根据式(1),可得根据上两式可知,可以用f(x)表示∫(z)能减少计算量,故:(28SS∑(-f(Wx)(dans∑(v-(wx)(-1)/(z)(1-f(2)∑(-f(Wm)(-1)f(WFr;)(1-f(1a(w0(29)∑(v-W;)(-1)/(Wx;)(1-/(W0(Wr)∑(-f(x)(-1)f(Wr:)(1-f2(i-f(W xi))f(Wai)(f(Wxi)-1)a.i j对式(29)进行矩阵化,令:f(Wx1)(f(Wx1)-1)f(Wx2)(∫(W2)-1)0f(W.cn)(f(w n)-1L=[f(W31),f(W c2),f(Wan)可得(Y-L)V I320那么,对W而言,更新公式就是W=W-Y-LVX至此 logistic Regression理论推导结束5 Logistic Regression求解依然用 python实现,代码如下9#!/usr/bin/env python# -* coding: utf-8import matplotlib. pyplot as pltfrom numpy import#卫 oad datadef load dataset o100[[1,0.005*xi]fXIin range(1: 100)JY =[2*xi[11 for xi in XIreturndef sigit= exp(zeturn t/(1+t)sigmoid-vec vectorize(sigmoid)#solve logistic regressiondef grad-descent(X, Y):Y mat(Y)rowshape(x)a1pha=0.05maxiter 5000W= ones((1, col))v= zeros((row, row), float32)for k in range(maxIter)oid-vec(W+X transpose())for i in range(row):V[土,1][0,i]*(L[0,i]W=w- alpha *(y-L)*v*Xreturn wdef main oX, Y= loaddataset oraddescent(x, y)rint w#draw im[xi[1] foyplt plot(x, y, marker="*")M= mat(X)y2= sigmoid_vec(W*xM transpose())22= [y2[O, i] for i in range(y2 shape [1])1lt plot(x, y22marker=plt. showif namemainiTmaln10