回归问题及正则化
1.线性回归模型及求解方法
什么是回归?
X的行表示每个样本,列表示每个特征。
研究X和Y之间关系的统计分析方法称之为回归。其中X是自变量,Y是因变量。
利用训练数据,使用回归模型(如线性模型)去拟合变量之间的关系。因此训练任务就是利用数据,来学习模型中的参数 parameter(如线性模型中的斜率和截距)。
回归和分类的区别和联系
-
区别:
分类:使用训练集推断输入x所对应的离散类别(如:+1,-1)。
回归:使用训练集推断输入x所对应的输出值,为连续实数。
-
联系:
利用回归模型进行分类:可将回归模型的输出离散化以进行分类,即y= sign(f(x))。
利用分类模型进行回归:也可利用分类模型的特点,输出其连续化的数值。
线性模型
非线性模型
线性回归
线性回归模型中,假设自变量和因变量满足如下形式:![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAMgAAAAoCAYAAAC7HLUcAAAHzUlEQVR4nO2cO0wVTRiGB0JBAREjDUQjjTExBmgMNATjJbEgwdJo4qWhsIFQGSgghIqQoDHRwkIt1EQTLqGwUKOGglh4i/ZKSQgBQ2HHcd/5ec8/O+zs7uzZ3XNhnmSznN09OzPfzDffZeZQV/AQDocjkPpyV8DhqGScgjgcITgFcThCcAricITQUO4KOBwm/vz5E+u55uZmUV+fzVxf57JYjkol7qD/+PGj6Ovry6YOmbzV4SiR1dVVeR4bGxO7u7tFa9LQ0CA/47h06ZK8lpVyAKcgOfDt27eKeEc18fbtW3HkyBExPT0tP79580aeL168WHzm9OnT4tixY5nWo6YV5MOHD74jT7a3t8XIyIjo6OiQ51JA3W/evCm6u7vF4uJiSjWsbK5fvy4+f/5c/Pzu3Tt5vnDhQvHauXPnxMOHD7OtSKGGmZiYKBw6dAgxVqGrqyu3cr9+/Vo4fvy4LPPXr1+pvXdubk625caNG6m9M2/ev39f6O/vl+1A36AtW1tbkd87ceJEoa6urvDjx48cavk/Na0gYHh4WHYGznkA5UDHQznidLwtjx8/rlolwYSl1h3KwokkCigHjrypeQXhbLWwsJBreWlaDh0MMJQBi1ItsM5QcBUqjX5d5cuXL1I5vJgj20oGUPMKAuHjyGI216ELNDg4mGk5aAusFI482lUqVAJMHqZ7OJuYmZmRCjI6OppdJQ3UfJAOPBMuWlpaMi9vcnJSnksNyqNAWy5fvixTn5UetCNZcffuXfl3mFzCsnTIaAE1QM8L40o6GoXGIXuCTIzK79+/5QHQWciuVCJUEAwmwnadPXtWHmmWhQELZYzzXgxsDIqgeuA67kPukH8QaNPTp0/lc6ZnKoEnT55IuXjWztcPOmETWFCKN4ow+apKy0nNiG5SYLIRNIk91wRBlA78eVznM1n626UAVwf1QzCIOqrtEin78HFcBUJ/PEh+DMJFRGIB/SRydB+Twj4wtYWyMMnNm4itA3Rdvuh/wiSKiJnZ9CkIlQONUTtALYCE+ZXGwpRK2x5xBp4OBcG0K9vBTrOpexSmIFQHSonyIV9+h4MHisLUJ54JkrsKJ6mo51TwbF79gPaEjSHASSvo/uvXr6U8qCBx+gvyxXOQr57BpHwhW1yLM0H6XCyYOZhEukxeRxXNuG6m6L5EmigFT7ixn9WxdYfgAsK0e4NIuiAwqXwHPi8tLSWui6k8oLujOigbB2SNM+QLVwBmHy4I6og+iAPKWltbk30RVz74Tl79oLbDNE6+f/9erJcOtpJ4A92idv/JF3KkfO/du1eMb3Ad5djEbftiEDWewN/sQBV0JjaIeZpqJTAbZSoVKjAGEIVG1PgpCnSgN7uJzc1N0dPTIwYGBgKf4zujUMvk35AvA9i4ygHQP+gHGzBA8uoHygSTVNA4wX3UH/GJaWLxLIdVmZAp5Qr54N2qfK13VISZF7gmYs9EqtBPtjHteUPzGuRnxo0XhoaGpGk/f/584erVq/Jv03dEhCthgm4SzraxhE3cUw64JmSqXxI3PWkdkqbEQ9O81EAAq0Ewy9laj7wJymDp98Lqf+3aNfHo0SPx/PlzmWZ89uyZ3Bw3NTWVaj1ZB8zqtqnouFarXNDzMMmZrk6W44jvpttlS+QPpmjGmTLD4MJnL+BJXNkk0HePA5SZvm1QmXRLTPV58OCBePHihVSSK1euFK8fPXpU/Pz5s5i2TAMOoiS7dZMoiOpuJMGmH7hFPWgZAHVnH2WZpqYSJt0NHblQyMaxAMx0nvsSGYyWE1UYpuQCLKCJ8fFxeb5//77v+sbGhjyvr6/v+w7WP9T3x0EdqEl2G1NBKnUdigTN3IyDkAjKaixxNzUmMyqjNVE+GLdPwF+Ef10N2xvCfFs9tarz6tUrGWt0dnbuu3f48GF57+/fv/vuMeaJu4kQcRxkqaZCbdaTqmEdRATEr4BtznIsYa2O8mUcgpjalkgLwhkYpgp+HOKPPLZtlAJnY5NpB4xN0B41c7S8vCzPMPvz8/O+g3FYY2PjvvdSTmFuD8rBfZxv3bol64kZjtaH7gDuc6U3qo15baNJwuDgoDzrMqFLlfZYgkzgPeBgah/y5TigzHCOkm+ROFok9maCrDfhpQUXCIN28HI2wRmzPp5VZxZ1YSroOHnyZGS5plmR94WW2WFWEJks/oYlararhh29+tZ8zOasdxbb9bkArMsX48BWviSWgiBVWg2uFaBJ5WqqjrrVAALV3RrTtoY7d+7I6zibCEu7MmWOsvX76jYYnKNW4wHTw5W6zYeoW2fQdvRLVj89YL8Guc9UHsgtjnxJpIJUw5pHWuzs7EglaG9v33evra1N3sNvE0xAIfMYuIwL8/oR2EEm1kJhpS5EpQ0GNZTA81l91z99+iSv9/b2Rr6D5jzrXxRm9X6HH5+CYIDQ/HA2PGgdAUU4c+aM71pPT4+8HteKMkOV1W/STe6jI318CgJLAaVg8HrQlAOcOnXKF4Pcvn1bfp6dnbV6DxQDfi9355YC4yq8y8Z/dpSOT0HU30vE/W8TtcbKykqhsbHRl7l6+fJl4vfBTU0j04R3HMT+KDe+fz3KPHEl/0owL5C7b2pqEq2treWuiqOMuP/N63CEUNP/tMHhKBWnIA5HCE5BHI4QnII4HCE4BXE4QvgHW3HmJuT+9zEAAAAASUVORK5CYII=)
问题:已知一些数据,如何求里面的未知参数,给出一个最优解。因此通常将参数求解问题转化为求最小误差问题。
一般采用模型预测结果与真实结果的差的平方和作为损失函数:
概率解释(为什么采用差的平方和作为误差函数):
求解参数(就是求解使得上式最小的参数θ)
-
矩阵解法:scikit-learn中的 Linear Regression类使用的是矩阵解法(有时也称为最小二乘法)。可以解出线性回归系数θ
-
梯度下降法:梯度下降( Gradient descent)是利用一阶的梯度信息找到函数局部最优解的一种方法。
其基本思想是,要找代价函数最小值,只需要每一步都往下走,也就是每一步都可以让误差损失函数小一点。
对于线性回归,参数的更新方法一般为(其中L为超参数,学习速率,即每一步走多远)![]()
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAiEAAAA6CAYAAABmmYn2AAAfTklEQVR4nO2dB7gTxdeH17+iogKKvfeCXUGsiKAC9oYNBQELgmAFrIAiWFEE7IodRaxcG4iCoIK9oBQLRQELKggqCqj58s7nuc9m7u5mk8xmN8m8z5NnITc3mZtMZs6c8zvnrJBK41gsFovFYrEUmf/FPQCLxWKxWCyViTVCLBaLxWKxxII1QixG+O2335wXXnhBXfPhjTfecD788EPDo7JYLBZLkrFGSIUxZcoU57bbbnN69uzpPPLII86CBQsKfs7HH3/cadOmTSgDhNf7+OOPncWLF2fcv+GGG6rn6dSpk/Pdd98VPCaLxWKxJJ8VrDC1MsBAuPrqq50JEyY4++67r7Paaqs5r7/+ulO7dm3nrrvucnbaaaeMx1dVVTmDBw/OuG/48OHOOuusU+M5v/zyS2VA1KlTx/f1v/nmG6d///7KANliiy2c2bNnO0cffbTTq1evjMfhTbnlllvU8x500EGF/+EWXwYNGuQ89thjzvvvvx/3UIrGP//846y44oqBj1m2bJmz8sorF2lEFktls1LcA7AUBwyK77//3nnttdecevXqqfvGjh3rXHrppWojuv766zMej4GAt4KNCsaMGeOsueaaGY/BUBg/frwzbNiwQAME70v79u2dLbfc0nnxxRed9ddf33nqqaecm266yWnUqJFz2GGHVT/2qKOOUmGZ7t27O/fcc4/TsGFDU2+B5T9eeuklZ8CAAc7vv/8e91CKyqmnnup89dVXah7zPejWrZszadIk9TM8g8y1jh07On/88Ye6D+NshRVWiHPIFkvZY8MxFcIll1yivBVigEDz5s3VlQU5iL322quGAYKhgAFyzjnnONtvv33g71944YXqikGDAQJ77723uo4bN85zrGussYYycizm+OKLL9TnhfHXoEGDuIdTVI455hgVMsQQWbRokXofmjZtquY+hsbIkSOd0047zRk6dKhz0UUXqftGjx4d97AtlrLHGiEVwqqrrup5f926ddX1zz//rPEzPBiAt0Ln3nvvVdcjjzwy8HWHDBni/Prrr87xxx+vdB/C//73/1PPS/+BV4VQDJ4bBKsWM+CF2mWXXZTx2Lt377iHU1QwMvCyzZs3T/1/v/32c1q3bl1tXBNSfOWVV5xtttmm+jux0UYbxTZei6VSsOGYCuOjjz5yvv76a+WROPDAA9WCS9jll19+cTbZZJOMx6LfgN133z3jfrQgeEJwX2dbqBG/wllnnZVxP4ZJEGwYbJq8jtWGmAEPUxJAG1QMMHQvv/zyjPsmTpyoroQH4ccff1TX008/3VlrrbXUv99991113XXXXYsyToulkrFGSIVAfPu6665z5s6dq06B6JGJg4sRoXtKiItjmMAee+yR8TNOjZBNr8HJErbaaitn3XXXzfjZjBkz1NUtdHUjzy2vZSkfZs2a5Xz66aeRv85xxx1X4z6EqeidBDFK3I9166YsFku0WCOkAuBk17VrV2VoPP30087mm2+u7h84cKDSiYBuDIgXZNNNN60hzpN6HtmMkPfee09dDz30UCUIdDN9+vTq5w/C1g4pP2644Qbn8MMPd9yJeXfffXfBImQM1ltvvdV3zojBsc8++1Tf9+abb6rrZpttpq4YKdyaNWtW0FgsFks4rBFSAdx8883qeuONN1YbILDjjjuqq4hF3YgRontBckGMELJcuHnh3hB0EKdWWgZHJYDBiyFCZpZA2ATPWbb02SC22247Zczg6Vu+fHmNn4vBQYq6+7769etX/188b6SsP/fcc0o/Q10di8USDVaYWuZQC4QaHRgfLM5ufvrpJ3X1in376UHCQgbC/Pnz1b9Z6PWbIFkyXmTLurGULmRmkY0iLFy4sIZ+I19OOOEEz/tl3mHcwr///quujRs3rn7MDjvsoK7oVp555hlrgFgqik8++aQ6RV0gq44bPProo8Zf0xohZc6oUaPUtUmTJjV+NnnyZHX1MkI+++wzdd1zzz3zel20J4DmhDCQ+0Z9Eth6662dlVbyd8bZyqnlDanb7rlHuvYTTzxR8PPuv//+nvcjdHYXZkO4yv+vvfba6vsIPXIfdXGon2OxVAIcAkgGIETPvwX2CETbEsr866+/VMkGCeObwBohZc7MmTPV1cuYkEJNhxxySMb9IhrFNb7xxhvn9boianWHfwSpS0LabhCk6FrKG8IybtB0TJ06taDnFG9GIeh1cSyWcqZFixaqls7DDz+ckSWJ8cH+cMABB6j/n3322UpjiLHy0EMPGXlta4SUOYRFYO211864n/LtYtXqolTSeMEvFCP1PsRF54W4uvUsA4wi3OKrrLKKc9JJJ2Udv7u2iKX8IGtKN0SuuOKKgp4TAwJvRqHPY7FUAl26dFG1oPQyCnDuueeqatrbbrtt9X14EO+8807njjvuMKLZs0ZImUOfFhCjQBAr9rzzzqvxO9n0IAgAISh9VoyHv//+O+N+KXJGVcogJBQjr2UxCxkgSeHggw/O0IdQUMxWy40Ovlv0Zwo6ROQDrRjcUI9IXoN6LFL80Asymlgb8u3CnWQKeV+ihvUZg51KwjpkiHFIJZNNh/sJXdJ3rFCsEVLmSEVT0WEAYjs0H507d67RuA7ECPHTgyAYxcgIqmbKYziRynMBhg9CWcIwfuJBQZ7bFiqLBjQPQhKq0qIPoZqrQH+b559/PsYRlSds9NRJYbM3VRGWppNsSlKNVqBfFXoC4MTcr18/58QTT/SsziyHFnQJclApdUy8L1HzwAMPqCuFK3WkzpOfxgpP9ogRIwoeQ+QpuiLuOuKII6orElqKx7HHHqs61qJqxgCQCpF9+/bNaBwnLFmyxPn555/Vv4MyY+i9cc0116gvGguHF1deeaXKeGCC48LD6sbzItUqg0CgSBaDNULMQZYUJx49fbVHjx7qSojsrbfeimNoCsIyrBMCGSqkiHvpiiy5g3cJca7JxpCkMVMEkQ1LD+uuvvrq1XozROh8p9mAWQ/0zs0YRKwpGCOsK3z36fVTqph6X6KGvcEPKSgoehCdbDWeQpOKkDlz5qQaNWqkbumFL8qXsmRh2bJlqc8//zw1Y8aMwMeNGjVKfV7pzSrrc/KYpk2bphYvXhz4OF5z5syZocc6bNiwVHqRVFeLWdInr8Bb3IwZM6Z6zQg7Dy3ZGTdunPpOpQ0Qo8/LZzR69Oicfqddu3aptOHr+/MBAwaosU6fPr3Q4cVGFO9LFKQNvVTLli09f8bnELR3jx8/Xv18wYIFBY2hoHBM+sWVu53eI15IXj6il6BUzDghPprenGtoF8qNWrVqqdALJdSDkJTebI3poE+fPuravXv3wMfxmltuuWWocRIrxR3L6yf9JFSKc4dTWNAtbjghuuPTVNql62+5wJp53333VRfyg+HDhyuRn9yicMujAcHLgLfBFOLKJ7PCDV4AwhDcnn322Rq/d8opp3h2zxboccRYyZQqRUy/L3iwWGeiwKuon0DyAmTbu9MH3ILGkJcRgluXyZy2oJTLCWGZO9deENcuCtukgUaClCTCFbijqaKYPnnHPSxfunXrpia13LJpKnJBRIoIqPjMNthgg1AGALoPXLtsxp06dSpY6EZoB4Ei4Z0kCxNLbe5EiXtOcpPqvIVw8cUXOzvvvHP1/5mXbg1LqXL//ferNZODm7sVAkY6CzmaKcIltWvXNvq6iD5Jd/cSHxYC1WlZ+3Xow3PGGWeof3vpCSQMHFQThrEybtPi2WJg8n0hg5H50KFDB/W9WLp0qdGxEuqUcgo6rOsYTH5IaN+r4nYu5GyEoOSlBTZdUPnCsEDQCK2qqqpayCJg7a+22mrObrvtVtAgTfPkk0+qktEILxkj40YJjGBTqogmjSFDhqjTAYVkiMVRzdEUeDKYbJw4MSw4jYWFx1O4hhgzJ5d8C4zh/UAgiVGTlG6vXpTi3ImSV199Vc1JbgiORV9SKF5pu1KBtxRhjWRu83eQ3uhe3KnYKrVNgioI5wvfLTQWftqtfEmlUjU6bwtSCDFog+Iw64eMlT2m1DD5vlDckfUGASgGGZkq2TqQ54Kf3kqMHbIT33nnHc9yCj/88IORMeRshKBih0GDBlW/kfLFcbuSxEWdNC8Ik2DAgAFqI5dwAhAyAndJ8aSR+q/hV5hQSS5ccMEFztChQ5UnAiGxNPMKS506dZRnjEU2X8U9v4/L2JRgLgpKee4UA5ObHGsLXlY3pVr3Aw8IayOGiFdnX5ACbaaNEA4FeBQ4LPA9NY1fHR+8PUGh35VXXjlQFMlYs2XgJRnT7wvhbNZopA8mD2liXHzwwQcZ9yNSBzy8eKW9MpZohqoXusyHnIwQTuNYYZx43G8ymQ/gPgXLgmzaBVgo0gtCN47EPZrkUuHTpk1TV9n0TEEtEcpnE4ax+FPKcycqpDsyJ7Z8q+v6Qfdl4uQCav3bb7/d6GtEDZlmGOdk+ZCG6Yeksrv72JhAavmYrrcjByK9wzZIASu/rAr5vWxaAg40hJFKKSQT5fvC+owBwmEIA8AE6623njKKvJ6PTJ2XX35Z6QT1CsJ4a8ikNOFkyMkIeeSRR9RVr6zm5R5C8NKqVasChmYeFjHqY5AypYuGTLq4okBcdExSY6lRltCU8tyJEhPdloNg0ZVuz0BZ6TjTiHNFRIp+HhCBjZaqxtxmzZqljC2M3qCQRRjECAnjBSG8SFq03vEaXQI6B8Kt+ibr5ZKXdhB6w0w3uPulkKIf4hXFkxMnbMSkDeshcLQU6DfkBlG/L6IFlHmlQ00SPMp0TNcFzmiO+Hwpu+4GHRflG6jNo0NFYy/atm2rPJ8m0udDp6yI3gOrSR+Y9Bpx50OLUC1XUBDnCj1OKMCTDVEke7mQ+Bv4guk53Umh0K62lYCdO8WnGPOSBdUd6iEsw8Yg3XCTjGgavGryCO5QDPoq+nLUrVtXud7Z+AoJ80kF0myeEKlrgYaPEy5zmQ2P0/vJJ5+s9DjUeUIc6cZrsxXPBa/JpkdozaskeLYNTD7foMrMUcP4+QyRF3BlzOKJ5j7eX7Kd+P4jVBdRcVTvC1mOzBMMCfSZ7mKTrENEHtif+bw4HFFyHdBqSWiLJAI3hN/RAVLDicNWUNiTjBl+jpeyd+/evo/LhdBGiKSU8eLighWmT5+uriZO6HocOAxYZWEI+huYIFixueohigUtliHfrraVQLHnDvOFxSfpcydKimGE4IZm0ab4HXDC49/o0pLMt99+q8aabV1092ritCoFrpo3b64METbhfMMpYTwhvAbFsjCA2LjoJcLJGCMEvRihd67c5wbjBKE2hQ/dsEbhsWL8ZJHpG61shkHhKUDHAnE1ssQzwKaMBwPRvmRpiRGCESGlB/D6iwES9ftC5hhGCIJRtxFChioiVj43hM80CsUIwaPGY/FwofnzOiwRBiT8gkeHrBc/4SyhGSIi2Uo95ELORgh/iO6uE/bZZ5+CB0Rlz1zxir/p8OWSSqBnnnmm7+O8YrJiwcqXwo98NAGMPUyTtlwXe8ZsorlQksgmWi323MEAcf++PnfCzhswOXdMfvZsfn4bGK8h6X1RG8d4VQmHUVOjVBAN1zbbbBP4ODFCMKLZ5CT+jkgR/N5/jGE2F1Ls0enhPdERT0jQGsPv6fUpeK8xTJj76AWYa7pnGz0Amy0HJPe6RLiBDRUvI1ohHZ4PDRpeF5OYXn+p3ktKNTRt2lQZIXIYFHifwK1divp9kWZyes8Zd1dbWiAwNj5XxKX8DIMl2/c0m4Yzn4NeNkIZIXRilfQ4L9dgkyZN1FVXdueyCAsm2nB7IUYU8WXdiMICxM2IdacXZuFnnAy6du2a9e/Aus0VTuLnn39+4GNw+8kXLKwRQvw27liqaXQFt06x5w4LGD/zmju5zBswOXdMfvZBJb5lQS6WB4j1RYwQk0W3ooK0Zcimx5ADxlVXXZUhAMTAQ/TvtUnKyZy6OggZqUvB6ViSBATxhOSStcZmxQaHUU+Yxs9QJ1zSrl071d5dLzfuV/iOMTNOvZyDCaJYf+U7LesuXcDdkKLO4dtd1ybq90Xmk8wvLzgQYYRcdtllyiPj1SMsKYQyQubOnauuTGTdgiM2C9TAL2QRjpo5c+aoK2PR/wZxM3p5chDBoQMIUjQL/K25glWcDTkpoWT2sqC98PNWWXInn7mTy7wBk3OnWJ99MXVKeF3oQwSUCUjyoiqEMULwsInXyr2JynfeyzOLKJFNC9e+iHb5P9oEPd6PAZNrOINTNEYIsX+/ehcCRRRJQsD9TzggKBmBVE9CHKSaRqGfimr9BVJWGTOfl4QrOJBjKJK2rxPl+yLzieq7fkgzSGoYBemRkkAoI0Rcrl6CGeJOgDvQTa6LsJDr44Fc52zeBHGneymP5W/wsqSztZx3I9XwTBOUgYAYiVs+IuByI0lzJ5d5A1HNnSgJMkIIKxCuMjUv2RDJ0mBR5fRfCshmERQak/dQr7ApGQy6cYtXlPbpvA/urCFAg6Ijaa65IJonUkGzZfUAnkB0EGyifpstaxTegigrIUf9HWJjJ7yBdwkj5MEHH1R/r9sL4iaq90XmU5BxK/NKiqMFkU/EwiShjJB///1XXevVq5dxP64prEGsRL2iWq6LsOD3gRaKpJbp+c7E4UiLovS2LrZBQCQtr0eOHBnJuMIgbm8vI4TTEGl81ghJztxhg2ChiHveRI3Ew73izGR4EFM3MS9ZxBEH8v5HEZOOCukaLroML8TjoVeVlhCgHuKWsvi6xwO8XP0SymGjCbPJ8PzSxkG6qIaBGjFB2RJ4yrnlgmyOYTRzxYD1BSOEYmIcTPAWZasuHcX7IvOpfv36nj8npEMojTWJPRpxtF8rgCRELEIZITIJ9EZdUkVNNzgKWYTJR48Cv79BXNdSCdYNaZlkQwTldRcDie/rJ05OLEy4jh07xjGsxFHsuSOvp88d4uhJmDdR8vXXX6sr7my9SBlppZSwxzgrFDZpeZ9R+ie1EaYXopUJqvUhRgiCRDfSsMwtQiQEJWnozEXZpKUJmZfuQ+4LI1QmM4Ly4KSi4lUgDEm6LkJJPFvM6WIiYzZdaC1f5JDDxo4WBO+c6T4/YZCKql6eWTwfGD0Y62+//bYaK+Jl9mRCdnrmTb4RC5OE+kZjIXEKERcPoLYlHk4YRm+mlsRF+MADD1QFgLD8pC4E6Uyov/nQvFKORAhpIusnXxiDnMRZULBqOakwuaRpYJzjqwSC5g7l2/W5k4R5EzUShsIwlqJIf/zxh/JYSN+XQjNmmOeiA+Hkb7pScNRIXF7qKHkhQke3xkU8EUD5A4TGGHpk2eCVRih6+umn13gur5LvsoFzkPETGGMs0gri+eefV5sXoQbe69GjR6u1nNckNbPYRogcvpLSykHCX3iI8FzlI4Q1gRio7vUFRwCCWD4nMnX4rAhfojWhHQeHBcTMzBG3kDzfiIVJQh8ryMtnQWBBRoGNJU5xk/bt29d4bBIXYfK5cTvRPIrx4U7jw8Bt7GcFSoW7uP4OvBxUvxN4772IqlolljOLICchvoBhBU5kUiFmbtCgQSwnBdP4zR3y773mTtzzJmpESAdsFF7zUjbgQmC9QXzHwulV1KkUoFsqJ1LmhO4ZknCHfqJlw8C4IwyLsUGmBbolCX9Rs8NthLAZkj3H901HjBC/0uf8Hms5BghiSjE0eE6MEOY4gni/Cp1RQtgBL2QSEhsAUTohLzxEtDCJC6kY7DY6cRBwCMCrIb1lMCwJFyGZ4MZe4jZAEhM2TuVI2qpPpa33wMf07ds3lbakU2lLLNenj5zly5en0l/mVPrLl/WxrVq1Un9HXKRPmKn0JMl6M01VVVUqPcFTxx13XCptPafatm2r3of+/fsH/t5HH32UatOmjXrs0Ucfra7pL6vx8cVF2LnTsmXLWOdN1ISZk4XOy2HDhqn3ML2JpxYtWmRo5MUnbWiov6Nnz56ePw96r/g+/fTTT9X/f/XVV1MNGzZMpQ3hjMfx/JdddpnvGNKbkfq9xYsXe/48fbLOeB0hbWynpk2b5vu8UcJYGXOfPn1ieX0v0hu9eq+feOKJ2MaQNgzVGNLGYcb97Mtp46jG49mDGffSpUtr/GzhwoXquW644YbIxhuGnI2QMJTLIszf0KFDh7iHUVSWLFmi/u4RI0Zk3N+6dWt1/+zZsz1/b+zYsernXbp0qTY+Bw4cqO5Ln+giH3eSYPGstHljkilTpqh5w238+PFxD6dgevToof6WSZMmFfQ8EyZMUHPrq6++qr6vX79+6rm9jAiBQwW/x7VUkDGPGzcu7qEo2LBbtGiR6t27d6zjOPTQQ1NNmjTxNShzYcyYMWruvPHGGwZGlj85NbALC+5qE+7YOJE4brm61P0gfILrThcwUTkwPV+qtQBuECHSphw3MqW0pQ201Dgo1Xbc+VCp88YkogOhkJRfCLKUIBZPvQ2KkRXSB0VEqpKKS5iHMAqhmqD6EvTdQS8gfWxKAcZKKOaggw6KbQx8bvfff78SglIcj/L7NLKLC8Is1J4hrBKmIWE2khI2Nm6EsAgjnIr7DyuUiRMnqqsJhX+p4VUQjf4dfK4U6tFBVwOUK3ZnL0ilxbh6P8QB84a/uxLnjQkQ+qJT4BCTrX5LKUHKJJoNapy4y2vnAt9BkgCkPwhZWRj/YfoftWnTRul3SuFAgJCSsYq2IS7QWZA9yYEMLYhkgxabsWPHqnmDRgiNBwXQTIARC3JojAvj+W7lsnkjyoJS9+jkC8JSRKmk6HECkL4UUrhLwAtC+h7oIuVKbHGftHnDhkeKKJuWpBjizWKRp228W/BJJgSpoJzavQTnUcPrSwVm6f5ZLuCJQMxI9gIpzGS75JNtgpeIG9/DXKqOcpLHAEFUzMaaFLGnDgJaTvqIKuP0ggAHKkT/7GUdOnSIbRy8H3x/MURMGgzMIT01PA6MGyFJW4TzgZM7XwZaM1cipOmxIVBEh34sVGiUgjp6cyXcwSD9g9yQfpiqoBb3SZo3LDCdO3dWfUXwLOB6ZQPk5Ew7bkkDnTVrlsp8I9ONujMUXwOU9txXLBiHFCLr16+fbxfPUofKmUElvMOSz3cK46N79+4q3ZmNLSmprwKeBm40UYvbCwL5NMSMAslEM0mSwsZGwzFJWoTzgbGjfaAWBLFoaR1eSdDwCAMEy59mYZycpLkS6JuD3uLefcOTUgkt7pM4b6ghQ8VSUu84PZGGSXEl0kEpXkSsG0j3o1oiNX34t9Sekc+1WIgOpHXr1tWdSwtB2hl4wefll7JazqAjwBBhg2ezT9J7gJeG8cj4LNGSpIiFEU8Ik4f4LTn9SVmE8wEXJU2gyn3T9AOLmwJ01AShLoaAKxnXPT2E9E61UlI+qDSxie7KSSaJ8waBsIB3Q8Ymn6uUfCbcRqEsOn4Cn7X7WgxuvPFGZTDgbUPvYALCOXjiiKfrUGODeWzqpBu2JHpSQKjKLUkQeok7/FJJJCliYcQISeIinC/l8Dfki5yOvaoxShNDdwwR/QCgF9FdhoQAaN7EKdzd+DAJvQqiIKnzRqor4pniOyoQ/gDc+u6Tp5Rj13uZRAUhInrwgEkdyNSpUz0F1rwPFHTyKnmdKxg4GN8IRuVvsFiSjkQswjQnLAbGNCFJXYQt4SANDTEq6IK5efPmqStCLXc4Rh4fZYt7S2FI6Wt0CO4GZ+LB0qvgYiSCKQV+ED/88IPTq1cv9W9CRVSmNQGGBv1UvIwQ0lwLNXYwcBg3uhvR0FgsSSepEYvS6QZliRRKEQOZMDoSP9Rj9ZIp4/Z0CFJPRNcHJaFXQSUhRohuWErjNP1+0YJE1QrADYYHHH744UZPZWiZvECzI+niYmzlAx4QSqezkBfDWLNYTJDUiIU1QiyKRYsWqauX6p6UQqBZoZvUf431pGW5IN0bcXm7PR7NmzdX/SBi71VQQfh1YJZQmrvfiBTSkkZdUYJmhV4ozAXGJt1hCwHxLeEnSfPVqaqqUkZDvXr1Mu7HUBZvnx+kLEu2EKFEi6UUSZoBAtYIsSgkRu7u4AmkcyIaPPjgg2vklEubcL3FvRT10b0ezz77bOK6K5czbKyk6NIBW+q8gAiD9fCHdMmWzxlvFqEz04wfP9557LHH1L/RDklqbrGgqZybMN45qgFbLBbzWCPEoqCQFUbF5MmTlQiVbBhOy6RskikhqZtuxJiQ0zZQf4CYObUIdGMjid2VyxkJueihFblfNyopCQ0YlVSt7Nu3r0rr9GoRny943CQdt9iIkaXrkZo2bRrHcCwWi2ONEIsLSmZTKAoRIwYJp1Q8INQKqVWrVo3HI/wjrs9JlhMz6Z5sWmQMeKUAJqVXQaXgF4rxM0LYnIcOHaoyPUjhJVvKdJYMVXQJ47nL+xcLvHqge0Lo6ZIN+pgUs3ibxVIprJCSwL7F8h9kLSA6pXw3rvww4OKnmqqXsFUgE4PnLUQUaAkPVU/BnRUTdD/QGwhDoZzSpwVSxtEq6fMPTQjZNEEgnvUyQtCYIMy2KboWS35YT4ilBtQ94JYLYTatcuiuXEp4GRlB9wMp2OVaMh0DhHmNOBqDmBolhBql9UC+4AG0WCz5YbyLrsXiBeJWnG42FGOJC+YeKbrNmjVz+vfvn3dVWLK/WrRo4TRq1EjN6fnz5zuNGzdW91ksltyw4RhLUaBE9uDBg50HHnjAekMssUE9nKCQocViKS7WE2IpCknqVWCpXKwBYrEkC2uEWCKn1LsrWywWiyUa/g/KsE6+OapHbgAAAABJRU5ErkJggg==)
如何求梯度?
先初始化一组θ,在这个θ值之上,用梯度下降法去求出下一组θ的值。当迭代到一定程度,J(θ)的值趋于稳定,此时的θ即为要求得的值。
2.多元回归与多项式回归
sklearn的一元线性回归
在sklearn中,所有的估计器都带有fit0和 predict0方法。
fit()用来拟合模型(将输入数据输入给模型,训练得到参数), predict()利用拟合出来的模型对样本进行预测。
实例:用sklearn来构建一元线性回归预测披萨价格
1 from sklearn.linear_model import LinearRegression 2 X=[[6],[8],[10],[14],[18]] 3 y=[[7],[9],[13],[17.5],[18]] 4 model=LinearRegression() 5 model.fit(X, y) 6 print('预测12英寸披萨价格:$%.2f'%model.predict([12])[0])
输出结果:预测12英寸披萨价格:$13.68
线性回归的参数
对于刚才的例子,Linear Regression类的fit()方法学习线性回归模型y=w0+w1x
线性回归模型学习到的参数是截距和权重系数。下图中的直线就是匹萨直与价格的线性关系。
残差(residual)
估计值(拟合值)与实际观察值之间的差
多元线性回归
from sklearn.linear_model import LinearRegression X=[[6,2],[8,1],[10,0],[14,2],[18,0]] y=[[7],[9],[13],[17.5],[18]] model=LinearRegression() model.fit(X, y)
X_test=[[8,2],[9,0],[11,2],[16,2],[12,0]] y_test=[[11],[8.5],[15],[18],[11]] predictions=model.predict(X_test) for i, prediction in enumerate(predictions): print('Predicted:%s, Target: %s'%(prediction,y_test[i])) print('R-squared: %.2f'%model.score(X_test,y_test))
R方
可以用于评估回归模型对现实数据拟合的程度。
计算步骤:
多项式回归
用适当幂次的多项式来近似反应因变量与自变量之间的关系。
多项式模型:![]()
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAjQAAAA1CAYAAABWSHGPAAAgAElEQVR4nO3dBZh1VdUH8KsYGBigWCAqKYqFYICJioqFgd2tqKgoFgao2N2KggoGAnYHdmChoIBiAgp2i3W/89vPt+bZs+fknTvvvPNy/s9znpl77rkn9tl7rf+KvfZ5phUmI0aMGDFixIgRaxjnXe0bGDFixIgRI0aMWC5GQjNixIgRI0aMWPMYCc2IESNGjBgxYs1jJDQjRowYMWLEiDWP3oRm//33X8n7mAs+9rGPTb73ve/N7Xy/+MUvJo973OMmRx555MK+P//5z4PO8eUvf3nyqle9qva+XvKSl0yOPvroQef75S9/OfnKV74y6Dd98aEPfWjyl7/8ZW7n++lPfzp505veNLfzrUV8+9vfXufX3HfffSdvfetbZ/rtne9859Qv13fc6173mrz97W9f0Wuccsopy/r9H/7wh8nvf//7mX5LLnz/+99f1vVzfOADH5j88Ic/nNv5mvCtb31r8q9//WvFr9OFr33ta5PXve51td+9973vnTzmMY+Z/P3vfx90zp///OfzuLXBMG/n3e9+9+TUU09dp9d97Wtf21t+/fvf/5584xvfmPzjH/9oPW6o/hyK8/SZ5fSzn/1sst12200ufvGLJyV10YtedPCFvvnNbyZBuemmm07Oe956HvW///1v8qc//WlywAEHTK5xjWsMvsbb3va2yYMf/ODJWWedNbn0pS89+PePfexj04BEQgChudKVrjT5wQ9+MNlhhx0mxx133OTmN7/55G53u9vkPe95T+u5Hv3oR6cO8aUvfWly4xvfeHLaaadNrnzlK0+22mqryec+97nJVa5ylXQe3//qV7/qfY/f+c53JjvvvHMib3vuuefgZ2yD+3Q/j3rUoyavec1rln2+Zz3rWZODDz548pCHPGRmYvOGN7xhssUWWzT2mcA///nPJHAe//jHz3SdlcK2226bxs5HPvKR9Plvf/vb5CIXuUjtsRTBBS5wgdrvfv3rX6d3c9e73rXzmhtvvPHkOc95ThpHQ6GdH/CABwwmRMccc8zkUpe6VLo2GMu/+c1vJttss00a82RHCcred/GbviDYjUd942EPe9ig3z73uc9NCqIUez7/9a9/TYTeeP/ud7+bPhvDj3zkIxvPd+ihhyaZU4fjjz9+cr3rXW/ylre8ZfKgBz1o0H0++9nPnhx00EFJUWy00UaDfluHrbfeenKFK1xh8oUvfGHZ56LkrnOd69R+d+CBB06e97znTX784x8nGdcH+vXuu+++7PvKod1PPvnk1C/32GOPRd9ph8tc5jKDjQ2ye6+99kpE6aEPfejkspe9bO27QWS33377JEdLMG7Js/Of//y116C7dtppp3T/OYzLWfrRcnCjG90oGc/Gg+dpA7m2ySabTN7//vdP7nCHOzQed4tb3GLyxS9+cXLUUUe1HjcrztfnIEpdQ1/1qldd+H/oILvwhS+cmPFTnvKU1uMIKUpwFlziEpdIf2chM2Ag7rLLLol45NYR4QmE5x3veMdOMsOL8vrXvz69sFBezknBePE6PLAiHvGIRwy6x89+9rPp77zJDBAs++233+RlL3vZXM5Hqd761ree7Lbbbom9UxJDgRj2UeKYP/KwPoHARGSf+MQnLuzTN5GvOhB0SHQdnvzkJ0/e+c53dgoM0OcI7Sbogxe60IWWkMSw4Lu8sYhKOcaQ1hx//OMfk2C++93vPnnGM57ReC5tdK1rXav2u/e97321755hAUPJDFzzmtdMY7gOlNXFLnaxJAP22WefNGbbyNbzn//89GwnnHBCUlQloo1mUUKf//znkzybB5kJXPe61132OW53u9tNPvrRjyYi98AHPnDJ95ttttnkile8Ym8yc+9733vyrne9KxmS1772tdM+fXfzzTdvVPqI8OGHH57kSh14t5BJz/vUpz51YT+yyuBhQJ7nPOdJ4y2AgF/talebfOpTn6o9J089uc6gBc/v3puuT9fVQZ+h+C94wQumz+SWPpc/m/aoww1veMPa/SsFhj1ifb/73S/15fOdbzFdOPvssxeiBaHnmsZy4Mwzz0yG/EqQGVhCaG5605sm9lo3kOJl3ve+913yHSHnJevMdYgX6IW24QUveMGShltp6MQGECArd7nLXZIiChBYPAz/+c9/0gv2GVjNdcRiyy23TJYKoRiWOYLz4he/ePKjH/0oWUssrzPOOGNyn/vcZ9C9hkuPou9zrE734Q9/uPb7d7zjHamjEtpf/epX0z4dLRRGDgPvt7/9baNwvtzlLpf6DUFRghKBEFgBVjHFxwPYBJZ9H6KCLBEU6xMQA5ZnTlp5OLVTjvA4vPrVr248l/CKMXmnO90pKRREcVYgRm1eh6tf/eqtv9dHEKscPC2EPlc+zwYh/aIXvSgpjyaihlDx6tSBwqIwkRZGTg5WN+txFtz+9rdPyqvuXo444ohFyqULT3va09LxPLv6X+n9WI4cQzqhzzg3NhEIBsRQUDDCjCH/gKfQews5l4Ms4ZGy+f4Vr3jFou89c5OXsQ7a/He/+13yOgvxGetkDLnUdB7HN3lsyTsKk2FG3tz//vdPstu5Tj/99DQWfeY51D8Zl4ccckiS7cZmHZ7whCekscngC1KBbLnPm93sZkuOR2hC35WgMwK3utWtkhFhPPKO59DHkXl9at7eqz745Cc/mf7qfz/5yU/S+zAOhbMDyGuJLgLOa/akJz1p0b7DDjss6cVLXvKSi/br1+TcECK3ZMRRMpe//OWTgBoCMT4uuC50hQ76gPDQmHVubCjjdJ4p3N91QEpYrDmiEe3PGzT/H2EohSMF/fKXvzy9HAJGqA3OOeec5PngaiMs3L9NR8nv1wDj2myD75sGTA731mTlAG8ZwRkkRHvuvffetccKBebWTgnfUdT5+33pS1+aOiTLB3lD6rj8A94L4dSFoX1xfcCxxx6biKHnBiRESLTO+uLFoKCbPAcBv3c+fz/96U8nS7ZJaepHhDBvEI8DQyUg/+ThD3/4kt94d94PRd0GHto2CJVQTMYVYjILKCJKhmuft8JG6RpfIWybZAn3eJkvon8ivOHFrQPFUgfjUz9lkJQg4IXHtS+FfIMb3GCJ9yp31zsXg5Ay7YI2qDMQSjQpftcpyXOEISnWV77ylalNeU/f/OY3LxxDgX3iE59ovB7DlfeNV4oB+8IXvnBRuzp3/szaTg5Pk0fl4x//eOqjfqNvzeqhpyx5CG5zm9sseJkpWEajcYPE6/vhTfTOeBboEwSoBNlIfvFUIBalYi1DWUOAGBrDwjnukYxjJASxCbKwGmQGeJWFyt0HwuEdkud0Rq53jUH9rM84N3bhtre97eS///1vMny8F7wBOSy9Nh/84AfTGBiCJdKQoqRoeBj6gptMvkQfZk5Itd5QD6uGoCbUm9AktLgMm1zxhBFLUqO3CT2KAlkhVOusZG0nuTYsT50WKDRg/SAkBAhlU3ctyWpdeQXzCAu5x5vc5CaLkp6boOOWDDoHKxV4ngj5N77xjWkAAIspLDn/U9yUZhknrgMi1SZcA9rSsesLDHLviOcKeC54uIQZcyFFkLJ0wyLvQihzbShcUzdeEFl9EIxN1m+OOk8Wbwjl2UVmoFSSJeTK5V5B90ABDgUvAAVF2fnLmxAe3jovC+inPEUlGB88sW1jm1Vch65wJiXkftyfZ88JjfeUJ1k7V4Sc+2C54zxkMsNG21AkwDubKyFemhjfT3/60zvzJPVbG8+h54t2lceG6OTPTBlKOWiDEBGZPmu6AJCxZDivuPQBeWCem2LOPSmRQgDkbO51yOGdIjxyHo1Z5+VpDO/mZz7zmSQ/S/TJMaW8eTziXnheESweJnJ5Hob/rDBOTjrppNRuAV4n3sg6J0JXSkD5LCETQb+LvMfS6+t3Q/N1l0hD7DouyMMgHIDV3vKWt2w8id/osNh1l7Bb7swBwGaFv4IsBFjFwkWlsOMd4Wlpcm9DuMa7vAEGAMKh49WBUIhkM20m3EJhuVceCpaxl4jY5ffp5RnUpfJZSfSx/oZCQqmwXVPSIOikLFmWGbdrCYpBH3R/3mmfMID29DtoS7ztC8lrwnDc0xErZ+X7jJzVhV2BQCesKGHjIbxa3NkloRaSIvwpRGNIzkZfsHibwOoW9+4bziSoKTftXecyjgTaJhKRg3t+xx13TFZYwJjitSlhrHbBeEAEGQJf//rXk6IUlgZWpJw+AlV/Ca9HXV5aGRoBY845eF78j4RpV7/ndRDu4nm+xz3useS32otRIjk/CCLPdh6SFeIAYa7VANISYGWzhPW5Wb1mPBsIUj57SD/LEZMJZnlmpBqZIqe7QheMF2Qlz8nU92xgDMiZaTN84z6bjGjvlYwK8BYwnBj9DDeyoGmyA51TB2SWR5ZONUakNngWhEYUgdxoCn2tK3jP5F8eJian6D59PBL+gxyX0Y0SQnXSLLwb41iUApC5eeR15VjyJvPpxXJBkJqmToG5Yr4EGPdRH7DKmzprH4EJs7LXttlVAcqWx0LyWwkCIhIcKYs2hcG6xOC5KiksbJyglERlILJsuEi91HDF9ZnZFUSKcmwDpY6k1cV4c7h23bPOAspbSCXyhJr6jbg54svDxRooXbcsoPAwDAVCBW3Jpl3wHlj53pW/CA3BacaLdjXYm8D7JqGcotWvWDQUpSTBIDPCIYTaiSeemPoUBaG96maXEZwEK4MhSHA+1poSZ/uCW5cwBe7vOhjndS75OjBYCH3CLh/nTaHMPggjg5yh7CRIA6EqbML7BUIn8taaINyi7XIrsySmDJJQlq7Fu0ToluFqigdJi/ebE7h5IMJ6XeOcgkT4Ir+xCTxDbR7WPmCg3fOe90xjllzLLe1ZoK9HmJpcIx8ZQ8hIl4z3juqIZkDfQ2jkPM4DvDj6gvb2zhmflHudLjPOm5L7EbYwxOkZJMF7YbQhVgwiyeldk2dWCjzo+pzcmZjp6F2TDYxLIN+HzKCMUB3jII8GyB3cdddd53j3HbOcNHBd4o8BxE1EMUm8kqFeh1yoEdh9CctqIqxR0w7r0PUMCKHcAQJJ3D/A2mMhGcTCBiwDglDcPmb/tOW7BFiQOlxXng0QuqzaNnAV5y7YJkTIrAmsF8pZvNm7BkInLJX8f+BKJ3CQqbJ2QXgfKHCWAcHRBYqfh2aWqcolKExkFSniVfPs7oPrvMsagbyPyB9imSAdYdEF2XK/+gTCZD+BjgSyYEJB6hPIkOMQrHw2jfwsCjy8DEOhLwoVImiErGerS+pHaPo8N/D2sPAQgNzSJMjr0DTjqw4UCtIS4NXKPQHIXReQoPyapUKKUK+QqbC7d1Pn2UWuWNc8Mu7Bu2SJ1oVMymRh1ycfmrx8YIwja7YuyHvqIjS8ynm4bRbvLBKO/DJKtQvDQ3vVhSHKZ0aoGE+5EYgkIy/0DG+W8+VyV8haAmmfvK46MASe+cxnth7TlbcGwleMDvfrWRkB3nubvCbrjGMEIYdnjERX/cq41mdCB5iazevVJbdXAqZTI2JlIjrjHMmKtuLl64rElECC6NYwOCJvVCgPOZwXBqXhe5ESXilq4Rlu2brkwgAFt9x58x58lro3s4IF3jalrIt06JxenHCFgc9lHtN2nTfOzd0esUNu676ut77FnQy8fOZCG7oGPXRZOjwIYqAsqxBKEVLwWbtRdDo2YnD9618/uezbcgQIOoOnrPMR9UtKSFacB6EJoYtUIDRIF6LaV6kH3DfrK/IJ9A3PzvtEQORAeCVNGuDlzLnoM+HdC+gzCLH7ZT3Xhe/qgHQioJI/KSi5I95bW62JIbViCGXnowQi9BfJ0a4ZU3q1K1nSlKyfw3slb3KLTtL1kBCt3weMz/AgRqhS3wzIdYr75alGXuqINSsWKaSseQzKMceLwauRA4GnyNoITR/jj8GoP/TNO8k9KrPOwBJuCeUrxMeDX5eXVD4zgyby7AKRTxm1tUogMby+QWbok7Y2K8HwbtNPfcAzox/nCc2ME+1e14aMNESKJ9AYKwmNXLI8nwwxkMQcKMsfrEuYmBBEKsK3ZIu+zwgPbzB5PrROHBlo7AaR9i4ZvkNmxPVBr15NMWNRGCWGJXzSVWgHuJnlizTlQAgPiKk1JaUaBE1kRj2WJrfkchKqSkHiXEMKeBGAYdHKPzJYhS38BUKIItIxkD37EUUx1XmCFdh3ultXdcc+oPihru3zqfrIE6ITcdQ+U61LRRLVV8v98x4cEa5jmXVNZS6BAMpb4DkShuFZoCQJZJ6VXHkG9HfW0ZDaI/qNMJ/8gyANXdB++imhldecabKUeB7aZrjVAcHyvBFO1BYxPT2sd33GLCYKsS2fhodLDgxyFOTbOKIQCWBjqCwJUAfep8hrCSVsnIRC8TcSgyVoGhfGsPBhm5eQt43nx7vOgZwiodph3q71uPe4Thu8a8jlQR9vcBv0CZ7MCP8FjH9efe2xHPBYIoe8IyDkgSAgDHUF6+pAXzWVrAi0GZIMGjk0IhF53lBbJWT9hJHWFA4TshJWC68fmUVXIAv6M69j3/DuuoAxyhsp1YKcce8M8KHVxBmv+Uw6hGbeOg86CQ0GKUaNmHRlqpcoXZEECoHd14pumpYNsyQFt4HniWVZp2BZYXnlVIoH865LdCSsS/d65HUAoUIYIGoscYqPwmsq0gQ6Eouuy0qOZGXWEeHax6oW17TNC9H2wl0xGywgXGKwBvnpiyZSu9Keu3i/dXV52oBcROJlbsUhsLx3CG/0MyGa8NbwRjZNbW0Dz1RfbxwIkRCc5W8YHnUhJ/11KBCGkmCZNcGbYCzJI4qQQumtysFidXwk1AfUDgFeHqRMvloXuPs9o9CJvmkMO38ofPvIOblSkatGBpWehRJCU8ZoHi7hBUNm9H9khifDTNA2mElinPfJH3F/0W5dJRwQvjLEtNxZNEgmkp/LxRgv4fGSJ1gmDfcBwi38GrWxgFLVV3hMjKMu2cZDgjwMnfabQ6hZ+FuoJQgN8ujaTcYTUuK5beSxvpbXfEKG9N3IZ+KxRbpsUZuMjBy6LMNKgZwKDxmvuOeBoXmXxhnjxRghfxh4fWavDkUvD83Q4m91wOpYR6U7XYfJZ6QQ+CyPdZ1vw6qoW3vIgCyn9bES63KLgCVrIyS50rnUufI8D+WWJ3vJe+C6YyW0VdYUmtLBuwQXixLpDHdvH+8HS6LvtO0hoGAoNcKH0CPcDVRT7ttmm9WhKeu/3D/PNWSEcigpwqxtRlEdKF7vSwgIgVVhuw5RgA6hXM6srCFkpu03bTPT+oIiotAI6LqkcN/zdlIIiGJZUCxABvDuUGr6UpncjiA4DzIhJNRHeZZeJt4XYQnkzlhBQkzDlbPBMIlcsC7wIoQnIcCTzXCTn8Q75X6FGhCmJiBYlEZXfgvi61mjYGXXOEeuS2/tcmY4yrXgZcy9FmQ74wwJ4P2JnDrKa0gyMjkpxMrw1RbImHdEEfqO95JM6ZJZsVQAo0e7hkc4wDvgHIihsdpkrJc5NmZnCt01ebh4ksLjh3iWcpODIE8bMOZcozSC5VgNqQZPvpJXcvaGVp/vC7Jbe8mzyWdkxqSWNtCrxjDDXcjdO2xzWMyKuZTkNUg9bFu59bDKeSPy2SdeZu725koOhr8uER20JG8EB6s5f4EITVcM2oAmwKIyJC+NwZAz2yi6568kQIK5Dnl1yT6IxOamqs3rAhS1Z2JZUQ4IHgXUp/JpgLVjim45myLqFeRufMfmRftmhVADQuK9m4XGikBoEFMK2Lvo8jDJD6pbHqMuSXBeECqRnE+AUgBD830gqlqXkD9UhhaagBSwND2/Zy2nCDuPhFvjo4nMUJRhROQl8QOEITnhfSPHiLLxxWubl7NvApIkJGZMl7OIKFAhdnVnKOiu/DKueMQ9J0ty5oQewovk3qLyK6LWZN0PNeJi1klXDg1yWRKAktDEvfYB74x3m4dryHHPHbW5vHfPTLlHPlIfeC+Us42njDeEQeBdCf3xpPOYIbKlocCTyItTZ/iVVbj10dgXY5VB2JWMG8n3yDAj1PvPPfW8jhT1LJ6pHH7f9xy8HaIo2lx+UyT5zxs8iGRY7rEmi5uM+xIMbeNLuIq8LstrzFpOIEcnoYm55k0QK/VQbrZJWMsXCIvNcRhxvg7IPCzD9Q2ENitfYmi46fKBbeAQzAYgZSm3wDTrIaXXmxAdA4HqA4p7JYDgev6oAWKwccH3TV4lwLjgSwGlTyIW5TTjmEk1Swn4APJlM3iFSAhSOVCsC9a7cFFfGBtRIdg5Kd9ZCY2+5L0aS7wBpUXHqEASCAnEsU+NlxKIYx0JHrKCfdsK0e7frB35AsgM4lc3nVuOBCWj35RkJnKThNnC0ydnQX6Oe8/L09dB37DVhawDxibi5VrIK4vXVs7s4CGUH5bPrtJvKN1ylW15csgmj8W8XO08e13emUjuzHM6hPvKnKC+3k3eViQzHwfGBzJWzrRjiWtnRknTNGS6IOSVd+5cwhPGWlPekusY+yWh4b1BGvMkZaSKkVKG9JHaMsF4SN4P2cQjQp/xWOiTQ5ETWAnr+gyyYBz3zRMCZFUbCgHaumrvzALPqg2RydwQGbImUyw7If+V/EL88rp0eY7NzJi24OMf//i0YvLTSuFOK6tvWrGqafUSppXQnFaDeVqx6GklSKeVxdF2munVrna1aaUQ0v+VQkvnrATVtHpx06pDTP/whz8s2s4888zpj3/842n14NMjjjhiyfkqYZPOMXSrBm3jPVYDNR1Twr6f/OQn6f9KKEyrAZP2VS+49ZnB7/bYY490fGV5pL8HHHDA9L3vfW/637MHKsWZ9lWDuvO8bfjYxz6WzlMJ985jr3nNa04rIdBrc85KkPS6h4pYTCulkX5TkZr0Hr/5zW9Ob3vb2y68i8pym1aDePqOd7xj+pnPfKb2PFUHn1ZCMfWZfNPnKqG3ZH9lPUwrRTeovUqcdtpp08qKXrSvsuamlUXW6/eVQp9e61rXWtTvKus13VulbKcVIZn+8pe/XNh8n3+uBvi0Ij+LzlkJkHTcq171qmmlPKfXuMY10udKIKdjzz777DQmKwtz+rvf/S6d54QTTphWinP69re/fVpZRJ337Xw///nPa7+ryPm0UhKtv996663T/ZV43eteN91yyy2ne+65Z7pGZd2n/U996lPT50rhdt4b6CNkht9UCrL2mOte97rp+0oZpPYIkFn77rvvwvtwTyXs129z6Huey3eVoljym5122mlaKdWFz5V1no595CMf2bj5vhLqvZ65CeRwRaQWrtUE8s4xD33oQ9Nn/bIigWnfQQcdlPZpG58rA2Hhd9rKsSW0j2P322+/hd9WiqnXM5PnJbwH3+22227Tgw8+eLr55ptPL33pS08rQjitSFetvPZeK/I6rYzo1AZt8IzOXxHURfuPPfbYTp1VYvfdd58+/vGPX7KfbKsI08LnrbbaalqRi9pz/PWvf23UTdtss820MvSmldKfVobtkt86RpvUoSIw04p4L3y+yEUuMujZ6kA2b7/99un/yvBO1w8dXofoR2edddaS78i0kGEVEV7YjxcYq8Ez6s7nXQ9BK6GBykptJQkGNcJTh1y4Vyx20XeEbGVddJIQRKqEAacTaai+m3NVDLzxOSnryppYst/vTjzxxPT//vvvn4Tqs571rNY20zEpAL+lxENBUjRBcB7wgAcs+g2lSWj67oEPfGDr+UsQAuXgqCyfzt9d5SpXSeSgD5z3wAMPbPyewia88v5iUJQgOCkzhCuOKwcwUmE/AVe3ed6m7y55yUtON954417PtFK47GUvm/oAhZhj55137kW89ZkcFNJHPvKRRfsq63yB2HRtT3ziE1vvV99znHGS49GPfvRC33KtNlTWdC3RuM51rpN+v8suuyxRas997nPTd95XGA11iHu40IUu1En4EV3HvvrVr160X5+73vWul8hfCUTQb84444zacyKRJZ797Gen33zqU59a2Ies2Wdc3frWt06KgMKneJGd17zmNdO73e1u6Zg//vGPrc9Rwrgp3yuS2ASkhzFx/vOff2Hf0Ucfna7PiAh4Zorlu9/97sI+92wrQeaWigehs4+cr6z1pJC1jb6AmCAtm2222XTHHXdccj7HPvjBD16074Mf/GAiOH369V3ucpcl50S+GZ10y2Uuc5lkRJRgjDLShuD6179+0jt1+OhHPzq9053uNN1nn33SfVHQTfBs2pZsYLi34Qc/+MGi56Xkm+Da2lhblwbRLEAGr3SlK6VzIUjeaR0YJPpZGOQl6DL73VdJyOJ5GF0lgdbfjeUvf/nLg+67k9AsFx7IC2+DwX3SSSfVbnUwKHfYYYfe90BYEih9rewAUqADYuFD8K1vfSt1sDpGTUnwPDThec973qBrgQHC26VzlUqpDTqh5+sDXgZsugk8AzouwvaBD3yg1zn/8pe/TA877LAl+7W7jsxjMHTjATnuuON6XX9DAuJPOdW1yTnnnNP6W7+jHEtPGcv2hje8YSKqSGYbEPgnPelJS/bzRjzjGc9o/B2l973vfa/13Keffno6ri/qvLpd56cMuu6jxN577z3o+ECdku0DVq4xyJtcZ+jlMLaQCd6IeYHhQcnkaFOyga997WvpfmYBWcBr73nLjcejBOOT96XNK6kvIShDwFBDlJqg/yB34YGcF+5973unc5ZeppUGLxxS2AVGusiFflkacIG+Xth5IFGq5QeuRowYMWLEiBEjVg+rt6TniBFrGJK+V6M8+YgRI0aMqMdIaEaMGACVQ03Zz4tljdiw0Dbl1UwMdVhWE2bDqLGyPsLsQ7OoYlmDdQ2F6kwLrpsqbqatekUjNlysU0Jj2m7UEOkDBXtMnTTn/9wItRhMpY21jFRaNIV5SM2Iczvm2YamHPLMEJojNkyYDqysQNMaY2oUdZXTD5hKr9+pqdO0+b6tSngd1HYxrdyq18uBaeR9trbV5Usom6C4my2+CO8AABFgSURBVKnxShbkNcbqYNquir6KBbZtSFzTgsEQVZhlUGgb5QtAuQUyAJAcNYGG1vUasXawTnJoEBOL4Jm7rwhWXliv63fm5fsNYpMvIbChQ50HxZIU9lJDQp0aNS8oVHUs1sLK5auNlWpDNTsU/IoFSEdsWFArR3XuukrGqtcq+EZxWzU9X/28hHooXfVE1KBSgNN6cX0QdXQQ8zrwjCismFe3VsNIZei8aq2aK+SwOk+xdphV3dUwyiu/WjtJ1dq+tcKiIjrygUAoVGoMNq0aPqQCuXo2TQUeYzFUdWg8q/uI1eSj0jRippaQtZlGbJiYS6XgLihsZPAMLT5EICBCfo8METCzFDBaizAATzzxxIViVQQe4aqY1pDFCwOUsArM2vHcgr5tiPgoFtYExfVmKVQ3ohnhdV0f+qPCkhRdVC0VThRWsh6ZcUP5RxX0qPLrL4VJ0TcZWo5RxRc5aoIq5H1XvramDqXe5vUQbikLISpiGoQG4XE9z6QgXL4sgUJ4KnvnxT3931YBvkQQFFV3LS+AEGlPxKlugdMuo4IxizRaRqHNk6Xgn0KlCA1YageZET6MZTMUdGuqhj1iA8FKT6N63OMeN734xS+ept4tBwryuN28MM+5AaYXqqcQUFdCzYOhMHXetOrlvoe1iK42VPtCGYCm7S1vecui85kGrS1NRx0xG25yk5ssKkq3mlDPpKvmiUKioObGkUcemf5XH8h3TdO9y/pQTZtSDm1QO0cRRceqP6SuTTmNPgpZNk2dhUrBp/o4OZRHcC4lLfxePS7/29SmqYjdwrGm9X/1q19NU6PbNudR4FPZCpvaMGqSKX7ZF8acIoDbbrtt53T6iuilshymY9tcP/5XC0tBPO/MfsUB88204xEbDlaU0BACSEhXIbq+UGtFPZRzEwxCgij//M53vnPwec7NhGZebRjoQ2gUUZtXv++C6tNl4b31HesToWmrIF4iJzRQV4AugNC87GUvm/73v/9t3NT5aSM06pBEUcGoX6PqOGUfUHHcMTn5qEMdoekiW8hIIKoTz7qpotuEvOipWlKOj2rGgGApwFYHFc/Vv9Iuilr6rf9tqtvuuuuurUUo86KCI9Y2VjQpWAKw9Rq4DesgxsqdX65Aan0IC16Vq3gKPYkNr1aSsPUsLMhmBVfrxVifpWthuDr0fe7p/6c3xXo3FvYC6wGt1CKHTRC333777dPzciVbGNBaPKuF1WzDWPembZ0za1bFOjorDdc5/vjjW4/hfhfesT6OvA6frRkzBEPH60rB+LcIomta0d46ONb+yddUGgIL01qXybl23333yY1udKOFTYKrnBkJvpG3InThf5swrvWZ2iDxtWnrgvwXC9pa80h4CvQt4XihFLLI4oSnnXZauvdZYHFcoZ9yswJ1HpISMqo7rtzIBXkz5f5Sbgt5geRoIbEIDcdK8LH+k5w3/c7il3V9zGQToSWL+8ptAp+1h3el3awxJrk47kU4UNv6P1YtH7EBIGc3XJsq3Kr6aR0YUPJYCWSWRFfF0RLCTSyxOqj0iB0rK54vBWDNjFizpUSljJLHp0/1y2qg997y8uVNePjDH57uKUqvs8B9Vtp9CIY8d6z5FFAqO6yKQw45ZNB1l+OhYd2wrqLyMS+Zc60rD0SJ1WpDuMENbjA973nPu3AOn5UoL6EMPKt6XcB9NJUmV03VchCs1FivJ5bYyD0NXZhlvLZhVg+Nsepawj2WFGHZx7uoqxzbF87lHLwA+abtLBlg/POEGQf+xqY6Mi9KHXho9AHytGlT1bYr5FSC10Qo1P0Kt/RFk4dGZXOwRAYPB68SKHlv/AyFJQ0qMth5nFBwLIlgPaKQpypXxxIClpHwf6xVJFxct85UrA0WywUIvflrCRLVjVVg//a3v73ouYdWhh6x/mOB0FBW0XFinYy99torrUeijLR9fdf9CRBaTYQGPvzhD6e/W2yxRRpIhMMtbnGL9hvuGcLKhU7X1rQ4YkApdc+fr4FiscWhAiUwy3MvF7MSmliLI1+L65nPfGbaZ3HS1cJqtOEQrC+EJhaFyxFrzliuYgjm2eazEhr3XV4TKcjXLJoFwj91pAyhkWMV8E6t6Wb5gTpYF2s5KBfIBGPPwp/Wmgvydrvb3S79rVt/rgldhAbBz9ugidCc73znm2666aaJsOUbgmaBUDqEodgFa5blazlZHsF6W/66D6FAy9xYlyogFy43RJB0Iacg6EFwLadDRsvBAcsHuDcIwjRiw8MiD42EL/Cyn/KUp6TF4wL26chDIBm4jdAEwvshPt15w3PMyekLA9Wz5EBu3PNyFgIb8txDcPzxxw+KbTchBMvd7373RftDIa4PWKk2nAVd7dy1/k5fhJeiabPYIFBUPpdrKYUCmBVD2zwWZOyztRGkSHwt10Ky7/a3v/3MzwNWcXYe3ph8Q2jyBTFDNjY9+5BxV7fl76rMWeGZsOimtel8RgbIn3Lj2ahb0buL0EjE7UNoEAd90Jpp3i2jx//2yU1j6PTpX95ZuVq4NYEkD/s9Q7K8Pg/ZK17xikX7eAi9KyCrYnXxHJKf454858UudrHO+xux9rBovqDYZ8Ve0/+m8JnyGlCMLOKT84bcGEXLTM2bFyLPoQ9iKmYd5ApUVtOSqZlRIFCcfVasxHODWhmmMZYwrbFufxMOP/zw9LciMIv2e/Z1Nd1W/LwSWJNdd9219vuVasNZkE/PVYdD4bO8orCcj3lAPZ38Wi9/+csXfY7px1EVVQ2eHKb17rnnnrXnPvbYY1Mdk6Y6JzC0zStLPG05lF84++yzG+uK1CGeJ3JJoFJU6W/5PKboyzW54x3v2CtXSLG1S13qUpOKUEwqxZfaQK6QHBX3CdP/z8fyHpumMqvZ4jzOUUIFXTk5TVOV//3vfy9cC+Ss/POf/0w5NKaQA1lUKfV0fbWUbCXIcPfRBPlC6sK89KUvXbTfVO4+ffSe97znwv/yi0AOUwn31tTPSqjHc9/73jcVxFO7BuQYHXzwwanMQkVYan936qmnJjm1ySabpBwdtXTk90WunLbeeeedU84jHHXUUSkPU97XiA0QJcM59NBDE5MtmbF9W2655SC21BVyCoilnqdnTsakp4dmiFXUlkNj9ohjKgGw5Pxc78vBkOdeLmYJOYUlnq9SLrfKvtKymjdM+9xoo43StdpWml2XbTgE60PIydgrLWUrMNtXWrnyCy5/+cun7+qs+xzzaPNZQk51XiHT7+2vCOTCPnkYVkbWbzfeeOPeYbHwxJjqK88Ictkg5OZapjbLSerC/e53v5RrxPsDkd8xK0IWeZ7Ih8phZXXf18284rnhoTE7yjHy4oR0cg+N9ALeu0CfHJrw0JSQQ2MV5jaEh0YelvuIlbzzkBNv1Pbbb58+m7JeemiOOeaYVtm++eabLxwrV0nOjv2RIzpiw8KSWU5RTjufARKeGgx6SCn5PhWBH/GIR0y222679L91ctqgsF7f87Lc+m4qdTZBWW4IqxfM8oE99thj0bFD2mbIc68W6p49rN3lPHsX+i4xsBbacDVRzgjhAYiibHk5+y996Utp5lOUiG/Dard53hd5YMxega233jr9JatY5yxxngxeHZViPXsXeLIqpbrw2awbyyD86Ec/Sp8rwrPE29UGHiyzkcwWKp8h3xSja1sfipeYd5WnRnE592islWtO8Zwp9MdDkcP5eVB43w444IBU7O6EE05Is7sg1l2yVEHuUTGzKpYQGApeFu3VB4pdKnhnnSyIfqv4pdlNCh+SueGlymH2YsxcAh7o+KyAYMyUAkUHecm8E++1CcaBWVp9qzePWI9QMhzJXpLsckQtBMlx4vMsGZ9ZWGYcicvWWR4KG7lEFKXKwUKTVBheDjUD4hy+M5ui6XzrspaKWiPuS2ErEOM2e8u+Aw88MLXJFa94xelnP/vZXm0zy3PPA7N4aCQ8+424uFkgLEOx6vCasNBZaH2ffSii3kvpoVmtNhyCNg9NzJIKhGckZulEjsupp566cExYnHVo8tBErpP8hrPOOitZunIw7JN8yiJXdKw8V52HZt5tPouHRt6Da+oXEvnltpnFwgvzj3/8I51Praq8nWL2UledHjlxm2yySfo/99B4Tu9HeznP+9///l4eGh5Gx+sHgfDQ8Ijkm5yRU045pfY8lWJNv6mMg4U8ETCTyH6eNbNTzURt8hgrOmeGaB1CrsesIHmUjgfjXVu0oc5Do4heeH+akqehzKEJGWWrSGDtb+pyaIA3TD+IIphkc15ME8ipOL+cmjqQNTx8nkGfUN9mxNrBEkJTCs4IMUjIC7zoRS9K+z7/+c+nzxKx6hKGERnJtKVr2uD1ex3QYAQKMK4t6a4OhOBqFNYLxWwjSCDCIbLof//73y8c29Y2sz73PDALoTHzI692SjhVlt3CZwozR1e/oCgpg6btpJNOWnS+OkJDya9WG84L5RgzRnzmagd9yudcyflMgdfBe6pD9DcbEmUs522VFxvMr1MSmpVo81kIzWtf+9qFa5q2bCaLMELeP4U5SuLns1lCTaC8grhDTmiAYjfj0wwcKAkNElVWnPVsyIwwvcJ3MGvIqamgXBTUs+22224L/WcoIonXc+oT/kdU+iAIjXCQJGUkPow/s8Hy8W/GXR4qq0sKZjBKBv7f//5Xe70mQiO8ePDBB6eQZLRJXpnYTDH79I9IUlfVuARjVX8IDJlFNmL1sYjQxOyJy13uctOdd955ob4ERZVjSDl+XpW6pQ9K5QWEZV4xMoc6Bc7TZGmsNH71q18tUhwGXF0cu6tt6mqWtD33vIDQELyzeLei/kzA/dYp0a5nJ3Dalhj4xje+seh8TR6aoX1nLSJv3yCj8l9mgcqrOdrKvTd5aObd5vqibRaU4y6/t9JDAz635YK8613vmm6zzTYLn0tCs99++6VzxHVKQuO7N7/5zQufY/pzzGpzrLEQHk9jMN94zyhksxMRgy78+c9/TjOgwtgwrmYBr5McH+fYZZddFvYjUJGvdPrppy/6DWOFJ9b1TR3n0Q8CoSYTcpvnHYX3kHHr/3xW6J577pnIz5CtidBEuzCigpD5q/yIWXFB+gJRzkA9sRz0TNS8AXlEI9YOFhEaLuWwFAmqulARlNZdbt3UgQu9T3JwE6Kgns62vmNo22xImPezNxGacxt22mmnlDi5LtBEaNYKlMvPCU0kmPYpb4+UMObC+8iq1/Y+514SCp3Bh/QIBeXXCw+HQm45Pv3pT6dE2VD+TVvu7eKRUvNJwTvKPMJ8Np6QWQpDglAjb1+cCzkpEWsy2UwDD0QNKpvEa2QQkWjC4YcfvnB8nqAL2pDOQcj6bsJqeTkR4OlhSLmGWjNRM0sYMghOOV0dhCt9h5jl8O4lD2+22WbLKssxYt1jEaHpU58iClDFIpFiuBHX1sHrwArhynP+IYtL+p1wFc9Mn+rAqw1em7q2EU9uapsNBbP0iy6EMlrNAn6rjZNPPjlZk/OqYdOF5XiC1gdErkusayTUNCTMY4ZQng/k94cddtiS44QzhEYo+7xC8VFHHTW9173u1XkdXk8en3JT9C0Hz6X3z9MgPMPTVFd8bwiiWnZTMcaAwosInplGAd6+fEZZH5AJ7r0pT2gI5MookpeD8Y2cRB21HC94wQsWrQlVYt999132PY1Yf7CI0Ojk2223XesPIvEzMKSUPI/P0JCRwbBWFlRcTtusdazWEgMj5gPtq4hctLcwSZ/wx/oIijqscn/zkvcjRozYcJE0kNlOihhd9apXTTOfbnnLW9YWbBoxYsSItQJlB/LFFUeMGLFhY4HQjBgxYsSIESNGrFUsKaw3YsSIESNGjBix1jASmhEjRowYMWLEmsdIaEaMGDFixIgRax4joRkxYsSIESNGrHmMhGbEiBEjRowYseYxEpoRI0aMGDFixJrHSGhGjBgxYsSIEWse/weTt/PBnwJ6AgAAAABJRU5ErkJggg==)
多项式回归的步骤:
二次回归指多项式的最高次为2次。
3.损失函数的正则化(可以有效解决过拟合问题)
向量范数
矩阵范数
线性回归的正则化
应对过拟合 (Overfitting)。因为在某些情况下,学习得到的模型在训练集上也许误差较小,但是对于测试集中之前未见样本的预测却未必有效。为此可以在损失函数中加入正则化项。
以线性回归为例:
其中α是正则化参数( regularization parameter),用于控制两个不同的目标的平衡。
- 第一个目标是使假设更好地拟合训练数据。
- 第二个目标是要正则化处理,使得模型不要太复杂。
线性回归正则化后的梯度更新方法:
为什么回归要正则化?
实例:使用一个2次函数加上随机的扰动来生成500个点,然后尝试用1,2,100次方的多项式对该数据进行拟合。
拟合的性能:一次方:R2=0.82
二次方:R2=0.88
100次方:R2=0.89
下面从500个样本中去掉2个,而在测试时使用所有的500个样本。
多项式拟合结果:一次方:R2=0.85
二次方:R2=0.90
100次方:R2=0.57
结果表明,高次多项式过度拟合了训练数据。
通过查看,会发现100次多项式拟合出的系数数值通常很大。因此在拟合过程中可以限制这些系数数值的大小来避免生成畸形的拟合函数。
具体做法就是在损失函数中加入正则化项,以抑制这些系数的数值。
- 如: Lasso回归(使用L1正则化)
- 岭( Ridge)回归(使用L2正则化)
- 弹性网( Elastic net,使用L1+12正则化)等回归
通过岭回归,拟合结果为:一次方:R2=0.78
二次方:R2=0.87
100次方:R2=0.90
岭回归中正则权重的作用
alpha值越大(即越偏左),weights(即回归权重参数)变得较小,即对权重的约束越大。
alpha值越小(即越偏右),权重系数 weights的取值表现出较大的震荡。特别是当接近最右侧时,相当于没有对权重参数进行约束。
通过交叉验证找到最佳超参数alpha
sklearn提供了 RidgeCV,可以允许输入一系列的 alphas,然后 sklearn通过交叉验证得到适合的alpha。
4.逻辑回归
线性回归的输出值的范围通常是无法限定的。
逻辑回归通过使用 logistic函数(或称为 sigmoid函数)将其转化为(0,1)区间的数值。
逻辑回归可以被理解为是一个被 logistic函数归一化后的线性回归,也可以被视为一种广义线性模型。
逻辑回归应用举例:垃圾短信分类
可以用TF-IDF来抽取短信的特征向量,然后用逻辑回归分类:
(逻辑回归的输出值是连续的,而垃圾短信分类则是经典的两类分类问题,如何解决?)
办法:如果逻辑回归响应变量等于或超过指定的临界值(如0.5),预测结果就是正例,否则为反例。
逻辑回归中的损失函数优化方法
即如何找到能使损失函数取值尽可能小的模型参数。
class sklearn、linear_model.LogisticRegression(solver='liblinear')。其中的 solver参数决定了逻辑回归损失函数的优化方法。
Solver的常见取值包括:
-
sag:即随机平均梯度下降,是梯度下降法的变种,和普通梯度下降法的区别是每次迭代仅仅用一部分的样本来计算梯度,适合于样本数据多的情况。
-
liblinear:使用了开源的 liblinear库实现,内部使用了坐标轴下降法CD来迭代优化损失函数。
-
其他取值,如 bfgs、 newton-cg等
坐标下降法
Sklearn中逻辑回归的优化解决方案“ liblinear”使用基于 Liblinear的 coordinate descent CD(坐标下降)算法。
CD是一种非梯度优化算法。在每次迭代中,在当前点处沿一个坐标方向进行一维搜索以求得一个函数的局部极小值。在整个过程中循环使用不同的坐标方向。
如果在某次迭代中,函数得不到优化,说明一个驻点已经达到。
Tip:对于非平滑函数,CD法可能会遇到问题。即函数等高线非平滑时,算法可能在非驻点中断执行。