沉淀再出发：使用python进行机器学习
沉淀再出发：使用python进行机器学习
一、前言
使用python进行学习运算和机器学习是非常方便的，因为其中有很多的库函数可以使用，同样的python自身语言的特点也非常利于程序的编写和使用。
二、几个简单的例子
2.1、使用python实现KNN算法
1 ######################################### 2 # kNN: k Nearest Neighbors 3 4 # Input: newInput: vector to compare to existing dataset (1xN) 5 # dataSet: size m data set of known vectors (NxM) 6 # labels: data set labels (1xM vector) 7 # k: number of neighbors to use for comparison 8 9 # Output: the most popular class label 10 ######################################### 11 12 from numpy import * 13 import operator 14 15 # create a dataset which contains 4 samples with 2 classes 16 def createDataSet(): 17 # create a matrix: each row as a sample 18 group = array([[1.0, 0.9], [1.0, 1.0], [0.1, 0.2], [0.0, 0.1]]) 19 labels = ['A', 'A', 'B', 'B'] # four samples and two classes 20 return group, labels 21 22 # classify using kNN 23 def kNNClassify(newInput, dataSet, labels, k): 24 numSamples = dataSet.shape[0] # shape[0] stands for the num of row 25 26 ## step 1: calculate Euclidean distance 27 # tile(A, reps): Construct an array by repeating A reps times 28 # the following copy numSamples rows for dataSet 29 diff = tile(newInput, (numSamples, 1)) - dataSet # Subtract element-wise 30 squaredDiff = diff ** 2 # squared for the subtract 31 squaredDist = sum(squaredDiff, axis = 1) # sum is performed by row 32 distance = squaredDist ** 0.5 33 34 ## step 2: sort the distance 35 # argsort() returns the indices that would sort an array in a ascending order 36 sortedDistIndices = argsort(distance) 37 38 classCount = {} # define a dictionary (can be append element) 39 for i in range(k): 40 ## step 3: choose the min k distance 41 voteLabel = labels[sortedDistIndices[i]] 42 43 ## step 4: count the times labels occur 44 # when the key voteLabel is not in dictionary classCount, get() 45 # will return 0 46 classCount[voteLabel] = classCount.get(voteLabel, 0) + 1 47 48 ## step 5: the max voted class will return 49 maxCount = 0 50 for key, value in classCount.items(): 51 if value > maxCount: 52 maxCount = value 53 maxIndex = key 54 55 return maxIndex
测试文件：
1 import knn 2 from numpy import * 3 4 dataSet, labels = knn.createDataSet() 5 6 testX = array([1.2, 1.0]) 7 k = 3 8 k = 3 9 outputLabel = knn.kNNClassify(testX, dataSet, labels, 3) 10 print ("Your input is:", testX, "and classified to class: ", outputLabel) 11 12 testX = array([0.1, 0.3]) 13 outputLabel = knn.kNNClassify(testX, dataSet, labels, 3) 14 print ("Your input is:", testX, "and classified to class: ", outputLabel)
其中knn的思路非常简单，就是在某一个范围内观察距离该需要归类的样本的所有样本之中，那一个种类的样本数目最多，通过对附近的K个样本的寻找来进行分类。
2.2、使用scikit-learn库进行机器学习
1 #!usr/bin/env python 2 # coding:utf-8 3 4 import sys 5 import os 6 import time 7 from sklearn import metrics 8 import numpy as np 9 import pickle 10 # import imp 11 12 # imp.reload(sys) 13 14 15 # Multinomial Naive Bayes Classifier 16 def naive_bayes_classifier(train_x, train_y): 17 from sklearn.naive_bayes import MultinomialNB 18 model = MultinomialNB(alpha=0.01) 19 model.fit(train_x, train_y) 20 return model 21 22 23 # KNN Classifier 24 def knn_classifier(train_x, train_y): 25 from sklearn.neighbors import KNeighborsClassifier 26 model = KNeighborsClassifier() 27 model.fit(train_x, train_y) 28 return model 29 30 31 # Logistic Regression Classifier 32 def logistic_regression_classifier(train_x, train_y): 33 from sklearn.linear_model import LogisticRegression 34 model = LogisticRegression(penalty='l2') 35 model.fit(train_x, train_y) 36 return model 37 38 39 # Random Forest Classifier 40 def random_forest_classifier(train_x, train_y): 41 from sklearn.ensemble import RandomForestClassifier 42 model = RandomForestClassifier(n_estimators=8) 43 model.fit(train_x, train_y) 44 return model 45 46 47 # Decision Tree Classifier 48 def decision_tree_classifier(train_x, train_y): 49 from sklearn import tree 50 model = tree.DecisionTreeClassifier() 51 model.fit(train_x, train_y) 52 return model 53 54 55 # GBDT(Gradient Boosting Decision Tree) Classifier 56 def gradient_boosting_classifier(train_x, train_y): 57 from sklearn.ensemble import GradientBoostingClassifier 58 model = GradientBoostingClassifier(n_estimators=200) 59 model.fit(train_x, train_y) 60 return model 61 62 63 # SVM Classifier 64 def svm_classifier(train_x, train_y): 65 from sklearn.svm import SVC 66 model = SVC(kernel='rbf', probability=True) 67 model.fit(train_x, train_y) 68 return model 69 70 # SVM Classifier using cross validation 71 def svm_cross_validation(train_x, train_y): 72 from sklearn.grid_search import GridSearchCV 73 from sklearn.svm import SVC 74 model = SVC(kernel='rbf', probability=True) 75 param_grid = {'C': [1e-3, 1e-2, 1e-1, 1, 10, 100, 1000], 'gamma': [0.001, 0.0001]} 76 grid_search = GridSearchCV(model, param_grid, n_jobs = 1, verbose=1) 77 grid_search.fit(train_x, train_y) 78 best_parameters = grid_search.best_estimator_.get_params() 79 for para, val in best_parameters.items(): 80 print (para, val) 81 model = SVC(kernel='rbf', C=best_parameters['C'], gamma=best_parameters['gamma'], probability=True) 82 model.fit(train_x, train_y) 83 return model 84 85 def read_data(data_file): 86 import gzip 87 f = gzip.open(data_file, "rb") 88 train, val, test = pickle.load(f,encoding='bytes') 89 f.close() 90 train_x = train[0] 91 train_y = train[1] 92 test_x = test[0] 93 test_y = test[1] 94 return train_x, train_y, test_x, test_y 95 96 if __name__ == '__main__': 97 data_file = "mnist.pkl.gz" 98 thresh = 0.5 99 model_save_file = None 100 model_save = {} 101 102 test_classifiers = ['NB', 'KNN', 'LR', 'RF', 'DT', 'SVM', 'GBDT'] 103 classifiers = {'NB':naive_bayes_classifier, 104 'KNN':knn_classifier, 105 'LR':logistic_regression_classifier, 106 'RF':random_forest_classifier, 107 'DT':decision_tree_classifier, 108 'SVM':svm_classifier, 109 'SVMCV':svm_cross_validation, 110 'GBDT':gradient_boosting_classifier 111 } 112 113 print ('reading training and testing data...') 114 train_x, train_y, test_x, test_y = read_data(data_file) 115 num_train, num_feat = train_x.shape 116 num_test, num_feat = test_x.shape 117 is_binary_class = (len(np.unique(train_y)) == 2) 118 print ('******************** Data Info *********************') 119 print ('#training data: %d, #testing_data: %d, dimension: %d' % (num_train, num_test, num_feat)) 120 121 for classifier in test_classifiers: 122 print ('******************* %s ********************' % classifier) 123 start_time = time.time() 124 model = classifiers[classifier](train_x, train_y) 125 print ('training took %fs!' % (time.time() - start_time)) 126 predict = model.predict(test_x) 127 if model_save_file != None: 128 model_save[classifier] = model 129 if is_binary_class: 130 precision = metrics.precision_score(test_y, predict) 131 recall = metrics.recall_score(test_y, predict) 132 print ('precision: %.2f%%, recall: %.2f%%' % (100 * precision, 100 * recall)) 133 accuracy = metrics.accuracy_score(test_y, predict) 134 print ('accuracy: %.2f%%' % (100 * accuracy) ) 135 136 if model_save_file != None: 137 pickle.dump(model_save, open(model_save_file, 'wb'))
其中数据集使用的minst数据集。
2.3、sklearn的使用
1 Supervised Learning 2 Classification 3 Regression 4 Measuring performance 5 Unsupervised Learning 6 Clustering 7 Dimensionality Reduction 8 Density Estimation 9 Evaluation of Learning Models 10 Choosing the right algorithm for your dataset
2.3.1、分类任务（随机梯度下降(SGD)算法）
1 >>> import matplotlib.pyplot as plt 2 >>> plt.style.use('seaborn') 3 >>> from fig_code import plot_sgd_separator 4 >>> plot_sgd_separator() 5 >>> plt.show()
其中sgd_separator.py：
1 import numpy as np 2 import matplotlib.pyplot as plt 3 from sklearn.linear_model import SGDClassifier 4 from sklearn.datasets.samples_generator import make_blobs 5 6 def plot_sgd_separator(): 7 # we create 50 separable points 8 X, Y = make_blobs(n_samples=50, centers=2, 9 random_state=0, cluster_std=0.60) 10 11 # fit the model 12 clf = SGDClassifier(loss="hinge", alpha=0.01, 13 max_iter=200, fit_intercept=True) 14 clf.fit(X, Y) 15 16 # plot the line, the points, and the nearest vectors to the plane 17 xx = np.linspace(-1, 5, 10) 18 yy = np.linspace(-1, 5, 10) 19 20 X1, X2 = np.meshgrid(xx, yy) 21 Z = np.empty(X1.shape) 22 for (i, j), val in np.ndenumerate(X1): 23 x1 = val 24 x2 = X2[i, j] 25 p = clf.decision_function(np.array([x1, x2]).reshape(1, -1)) 26 Z[i, j] = p[0] 27 levels = [-1.0, 0.0, 1.0] 28 linestyles = ['dashed', 'solid', 'dashed'] 29 colors = 'k' 30 31 ax = plt.axes() 32 ax.contour(X1, X2, Z, levels, colors=colors, linestyles=linestyles) 33 ax.scatter(X[:, 0], X[:, 1], c=Y, cmap=plt.cm.Paired) 34 35 ax.axis('tight') 36 37 38 if __name__ == '__main__': 39 plot_sgd_separator() 40 plt.show()
2.3.2、线性回归任务
@1、linear_regression.py:
1 import numpy as np 2 import matplotlib.pyplot as plt 3 from sklearn.linear_model import LinearRegression 4 5 6 def plot_linear_regression(): 7 a = 0.5 8 b = 1.0 9 10 # x from 0 to 10 11 x = 30 * np.random.random(20) 12 13 # y = a*x + b with noise 14 y = a * x + b + np.random.normal(size=x.shape) 15 16 # create a linear regression classifier 17 clf = LinearRegression() 18 clf.fit(x[:, None], y) 19 20 # predict y from the data 21 x_new = np.linspace(0, 30, 100) 22 y_new = clf.predict(x_new[:, None]) 23 24 # plot the results 25 ax = plt.axes() 26 ax.scatter(x, y) 27 ax.plot(x_new, y_new) 28 29 ax.set_xlabel('x') 30 ax.set_ylabel('y') 31 32 ax.axis('tight') 33 34 35 if __name__ == '__main__': 36 plot_linear_regression() 37 plt.show()
导入上面的程序并使用：
1 >>> from fig_code import plot_linear_regression 2 >>> plot_linear_regression() 3 >>> plt.show()
@2、接下来我们自己生成数据并进行拟合：
生成模型：
1 >>> from sklearn.linear_model import LinearRegression 2 >>> model = LinearRegression(normalize=True) 3 >>> print(model.normalize) 4 True 5 >>> print(model) 6 LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=True)
生成数据：
1 >>> x = np.arange(10) 2 >>> y = 2 * x + 1 3 >>> print(x) 4 [0 1 2 3 4 5 6 7 8 9] 5 >>> print(y) 6 [ 1 3 5 7 9 11 13 15 17 19] 7 >>> plt.plot(x, y, 'o'); 8 [<matplotlib.lines.Line2D object at 0x0000020B141025F8>] 9 >>> plt.show() 10 >>>
进行拟合：
1 >>> X = x[:, np.newaxis] 2 >>> print(X) 3 [[0] 4 [1] 5 [2] 6 [3] 7 [4] 8 [5] 9 [6] 10 [7] 11 [8] 12 [9]] 13 >>> print(y) 14 [ 1 3 5 7 9 11 13 15 17 19] 15 >>> model.fit(X, y) 16 LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=True) 17 >>> print(model.coef_) 18 [2.] 19 >>> print(model.intercept_) 20 0.9999999999999982 21 >>>
可以看到斜率和截距确实是我们期望的。
2.3.3、Iris Dataset的简单样例
1 >>> from sklearn.datasets import load_iris 2 >>> iris = load_iris() 3 >>> iris.keys() 4 dict_keys(['data', 'target', 'target_names', 'DESCR', 'feature_names', 'filename']) 5 >>> n_samples, n_features = iris.data.shape 6 >>> print((n_samples, n_features)) 7 (150, 4) 8 >>> print(iris.data[0]) 9 [5.1 3.5 1.4 0.2] 10 >>> print(iris.data.shape) 11 (150, 4) 12 >>> print(iris.target.shape) 13 (150,) 14 >>> print(iris.target) 15 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 17 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 18 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 19 2 2] 20 >>> print(iris.target_names) 21 ['setosa' 'versicolor' 'virginica'] 22 >>> import numpy as np 23 >>> import matplotlib.pyplot as plt 24 >>> x_index = 0 25 >>> y_index = 1 26 >>> formatter = plt.FuncFormatter(lambda i, *args: iris.target_names[int(i)]) 27 >>> plt.scatter(iris.data[:, x_index], iris.data[:, y_index], 28 ... c=iris.target, cmap=plt.cm.get_cmap('RdYlBu', 3)) 29 <matplotlib.collections.PathCollection object at 0x0000020B0BDE89B0> 30 >>> plt.colorbar(ticks=[0, 1, 2], format=formatter) 31 <matplotlib.colorbar.Colorbar object at 0x0000020B0AFC8A58> 32 >>> plt.clim(-0.5, 2.5) 33 >>> plt.xlabel(iris.feature_names[x_index]) 34 Text(0.5, 0, 'sepal length (cm)') 35 >>> plt.ylabel(iris.feature_names[y_index]); 36 Text(0, 0.5, 'sepal width (cm)') 37 >>> plt.show() 38 >>>
2.3.4、knn分类器
1 """ 2 Small helpers for code that is not shown in the notebooks 3 """ 4 5 from sklearn import neighbors, datasets, linear_model 6 import pylab as pl 7 import numpy as np 8 from matplotlib.colors import ListedColormap 9 10 # Create color maps for 3-class classification problem, as with iris 11 cmap_light = ListedColormap(['#FFAAAA', '#AAFFAA', '#AAAAFF']) 12 cmap_bold = ListedColormap(['#FF0000', '#00FF00', '#0000FF']) 13 14 def plot_iris_knn(): 15 iris = datasets.load_iris() 16 X = iris.data[:, :2] # we only take the first two features. We could 17 # avoid this ugly slicing by using a two-dim dataset 18 y = iris.target 19 20 knn = neighbors.KNeighborsClassifier(n_neighbors=3) 21 knn.fit(X, y) 22 23 x_min, x_max = X[:, 0].min() - .1, X[:, 0].max() + .1 24 y_min, y_max = X[:, 1].min() - .1, X[:, 1].max() + .1 25 xx, yy = np.meshgrid(np.linspace(x_min, x_max, 100), 26 np.linspace(y_min, y_max, 100)) 27 Z = knn.predict(np.c_[xx.ravel(), yy.ravel()]) 28 29 # Put the result into a color plot 30 Z = Z.reshape(xx.shape) 31 pl.figure() 32 pl.pcolormesh(xx, yy, Z, cmap=cmap_light) 33 34 # Plot also the training points 35 pl.scatter(X[:, 0], X[:, 1], c=y, cmap=cmap_bold) 36 pl.xlabel('sepal length (cm)') 37 pl.ylabel('sepal width (cm)') 38 pl.axis('tight') 39 40 41 def plot_polynomial_regression(): 42 rng = np.random.RandomState(0) 43 x = 2*rng.rand(100) - 1 44 45 f = lambda t: 1.2 * t**2 + .1 * t**3 - .4 * t **5 - .5 * t ** 9 46 y = f(x) + .4 * rng.normal(size=100) 47 48 x_test = np.linspace(-1, 1, 100) 49 50 pl.figure() 51 pl.scatter(x, y, s=4) 52 53 X = np.array([x**i for i in range(5)]).T 54 X_test = np.array([x_test**i for i in range(5)]).T 55 regr = linear_model.LinearRegression() 56 regr.fit(X, y) 57 pl.plot(x_test, regr.predict(X_test), label='4th order') 58 59 X = np.array([x**i for i in range(10)]).T 60 X_test = np.array([x_test**i for i in range(10)]).T 61 regr = linear_model.LinearRegression() 62 regr.fit(X, y) 63 pl.plot(x_test, regr.predict(X_test), label='9th order') 64 65 pl.legend(loc='best') 66 pl.axis('tight') 67 pl.title('Fitting a 4th and a 9th order polynomial') 68 69 pl.figure() 70 pl.scatter(x, y, s=4) 71 pl.plot(x_test, f(x_test), label="truth") 72 pl.axis('tight') 73 pl.title('Ground truth (9th order polynomial)')
引用上面的包：
1 >>> from sklearn import neighbors, datasets 2 >>> iris = datasets.load_iris() 3 >>> X, y = iris.data, iris.target 4 >>> knn = neighbors.KNeighborsClassifier(n_neighbors=5) 5 >>> knn.fit(X, y) 6 KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski', 7 metric_params=None, n_jobs=None, n_neighbors=5, p=2, 8 weights='uniform') 9 >>> result = knn.predict([[3, 5, 4, 2],]) 10 >>> print(iris.target_names[result]) 11 ['versicolor'] 12 >>> knn.predict_proba([[3, 5, 4, 2],]) 13 array([[0. , 0.8, 0.2]]) 14 >>> from fig_code import plot_iris_knn 15 >>> plot_iris_knn() 16 >>> plt.show() 17 >>>
2.3.5、Regression Example
1 >>> import numpy as np 2 >>> np.random.seed(0) 3 >>> X = np.random.random(size=(20, 1)) 4 >>> y = 3 * X.squeeze() + 2 + np.random.randn(20) 5 >>> plt.plot(X.squeeze(), y, 'o'); 6 [<matplotlib.lines.Line2D object at 0x0000020B0C045828>] 7 >>> plt.show()
1 >>> 2 >>> from sklearn.linear_model import LinearRegression 3 >>> model = LinearRegression() 4 >>> model.fit(X, y) 5 LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, 6 normalize=False) 7 >>> X_fit = np.linspace(0, 1, 100)[:, np.newaxis] 8 >>> y_fit = model.predict(X_fit) 9 >>> plt.plot(X.squeeze(), y, 'o') 10 [<matplotlib.lines.Line2D object at 0x0000029B6B6B7BE0>] 11 >>> plt.plot(X_fit.squeeze(), y_fit); 12 [<matplotlib.lines.Line2D object at 0x0000029B5BA68BA8>] 13 >>> plt.show()
Scikit-learn also has some more sophisticated models, which can respond to finer features in the data:：
1 >>> from sklearn.ensemble import RandomForestRegressor 2 >>> model = RandomForestRegressor() 3 >>> model.fit(X, y) 4 FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22. 5 "10 in version 0.20 to 100 in 0.22.", FutureWarning) 6 RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None, 7 max_features='auto', max_leaf_nodes=None, 8 min_impurity_decrease=0.0, min_impurity_split=None, 9 min_samples_leaf=1, min_samples_split=2, 10 min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=None, 11 oob_score=False, random_state=None, verbose=0, warm_start=False) 12 >>> X_fit = np.linspace(0, 1, 100)[:, np.newaxis] 13 >>> y_fit = model.predict(X_fit) 14 >>> plt.plot(X.squeeze(), y, 'o') 15 [<matplotlib.lines.Line2D object at 0x0000029B6BDEB198>] 16 >>> plt.plot(X_fit.squeeze(), y_fit); 17 [<matplotlib.lines.Line2D object at 0x0000029B5BA683C8>] 18 >>> plt.show() 19 >>>
2.3.6、Dimensionality Reduction: PCA
1 >>> from sklearn import datasets 2 >>> iris = datasets.load_iris() 3 >>> X, y = iris.data, iris.target 4 >>> from sklearn.decomposition import PCA 5 >>> pca = PCA(n_components=0.95) 6 >>> pca.fit(X) 7 PCA(copy=True, iterated_power='auto', n_components=0.95, random_state=None, 8 svd_solver='auto', tol=0.0, whiten=False) 9 >>> X_reduced = pca.transform(X) 10 >>> print("Reduced dataset shape:", X_reduced.shape) 11 Reduced dataset shape: (150, 2) 12 >>> import pylab as plt 13 >>> plt.scatter(X_reduced[:, 0], X_reduced[:, 1], c=y, 14 ... cmap='RdYlBu') 15 <matplotlib.collections.PathCollection object at 0x0000029B5C7E77F0> 16 >>> print("Meaning of the 2 components:") 17 Meaning of the 2 components: 18 >>> for component in pca.components_: 19 ... print(" + ".join("%.3f x %s" % (value, name) 20 ... for value, name in zip(component, 21 ... iris.feature_names))) 22 ... 23 0.361 x sepal length (cm) + -0.085 x sepal width (cm) + 0.857 x petal length (cm) + 0.358 x petal width (cm) 24 0.657 x sepal length (cm) + 0.730 x sepal width (cm) + -0.173 x petal length (cm) + -0.075 x petal width (cm) 25 >>> plt.show() 26 >>>
2.3.7、Clustering: K-means
1 >>> from sklearn.cluster import KMeans 2 >>> k_means = KMeans(n_clusters=3, random_state=0) # Fixing the RNG in kmeans 3 >>> k_means.fit(X) 4 KMeans(algorithm='auto', copy_x=True, init='k-means++', max_iter=300, 5 n_clusters=3, n_init=10, n_jobs=None, precompute_distances='auto', 6 random_state=0, tol=0.0001, verbose=0) 7 >>> y_pred = k_means.predict(X) 8 >>> plt.scatter(X_reduced[:, 0], X_reduced[:, 1], c=y_pred, 9 ... cmap='RdYlBu'); 10 <matplotlib.collections.PathCollection object at 0x0000029B5C7456A0> 11 >>> plt.show() 12 >>>
2.4、sklearn的模型验证
An important piece of machine learning is model validation: that is, determining how well your model will generalize from the training data
to future unlabeled data. Let's look at an example using the nearest neighbor classifier. This is a very simple classifier:
it simply stores all training data, and for any unknown quantity, simply returns the label of the closest training point.With the iris data,
it very easily returns the correct prediction for each of the input points:
样例如下：
1 >>> from sklearn.neighbors import KNeighborsClassifier 2 >>> X, y = iris.data, iris.target 3 >>> clf = KNeighborsClassifier(n_neighbors=1) 4 >>> clf.fit(X, y) 5 KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski', 6 metric_params=None, n_jobs=None, n_neighbors=1, p=2, 7 weights='uniform') 8 >>> y_pred = clf.predict(X) 9 >>> print(np.all(y == y_pred)) 10 True 11 >>>
A more useful way to look at the results is to view the confusion matrix, or the matrix showing the frequency of inputs and outputs:
>>> from sklearn.metrics import confusion_matrix >>> print(confusion_matrix(y, y_pred)) [[50 0 0] [ 0 50 0] [ 0 0 50]]
For each class, all 50 training samples are correctly identified. But this does not mean that our model is perfect! In particular, such a model generalizes extremely poorly to new data. We can simulate this by splitting our data into a training set and a testing set. Scikit-learn contains some convenient routines to do this:
>>> from sklearn.model_selection import train_test_split >>> Xtrain, Xtest, ytrain, ytest = train_test_split(X, y) >>> clf.fit(Xtrain, ytrain) KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski', metric_params=None, n_jobs=None, n_neighbors=1, p=2, weights='uniform') >>> ypred = clf.predict(Xtest) >>> print(confusion_matrix(ytest, ypred)) [[12 0 0] [ 0 6 0] [ 0 2 18]]
This paints a better picture of the true performance of our classifier: apparently there is some confusion between the second and third species, which we might anticipate given what we've seen of the data above.This is why it's extremely important to use a train/test split when evaluating your models.
