数据挖掘实战（二）—— 类不平衡问题_信用卡欺诈检测
写在jupyter里面比较漂亮：
In [34]:
import pandas as pd
import matplotlib.pylab as plt
import numpy as np
%matplotlib inline
In [35]:
data = pd.read_csv('creditcard.csv')
data.head()
# 0 - 正常的样本，1 - 有问题的数据
Out[35]:
In [36]:
count_classes = pd.value_counts(data['Class'], sort=True).sort_index()
count_classes.plot(kind = 'bar', alpha=0.5)
plt.title('Fraud class histogram')
plt.xlabel('Class')
plt.ylabel('Frequency')
# 此时无缺陷的样本数非常多，有缺陷的样本非常的少，可以明显看到 样本分布非常不平衡
Out[36]:
<matplotlib.text.Text at 0xfc64470>
样本不均衡解决方案
-
下采样 (两者一样小)
- 从0的样本中选取 跟 1的样本一样小
-
过采样 (两者一样多)
- 1的样本中采取生成策略，生成的数据和 0号样本一样多
标准化
- 比如Amount，数据量区间很大，会对模型有一个“数据大就重要的”误区; 所以要进行归一化，或者标准化
- 把他们区间放在 [-1,1] 或者[0, 1]区间上
In [37]:
# 这个sklearn库: 预处理操作 => 标准化的模块
from sklearn.preprocessing import StandardScaler
# fit_transform(): 对数据进行一个变化了; 变化好的列成为一个新属性添加到 data
data['normAmount'] = StandardScaler().fit_transform(data['Amount'].reshape(-1, 1))
data = data.drop(['Time', 'Amount'], axis=1)
data.head()
Out[37]:
下采样实现
In [38]:
X = data.ix[:, data.columns != 'Class']
y = data.ix[:, data.columns == 'Class']
# Number of data points in the minority class
# 样本的个数
number_records_fraud = len(data[data.Class == 1])
# 样本的索引, 所有值为1的 索引
fraud_indices = np.array(data[data.Class == 1].index)
# Picking the indices of the normal classes
# 值为0 的 数据的索引
normal_indices = data[data.Class == 0].index
# 使两个样本一样少
# Out of the indices we picked, randomly select 'x' number (number_records_fraud)
# 随机选取样本中的数据, 随机选择: np.random.choice(Class为0的数据索引=>样本, 选择多少数据量, 是否选择代替=false)
random_normal_indices = np.random.choice(normal_indices, number_records_fraud, replace=False)
random_normal_indices = np.array(random_normal_indices)
# Appending the 2 indices (连接缺陷数据索引 和 随机选取的Class=0的数据索引)
under_sample_indices = np.concatenate([fraud_indices, random_normal_indices])
# Under sample dataset 下采样 (选取该索引下的数据)
under_sample_data = data.iloc[under_sample_indices, :]
X_undersample = under_sample_data.ix[:, under_sample_data.columns != 'Class']
y_undersample = under_sample_data.ix[:, under_sample_data.columns == 'Class']
# Showing Ratio
print('Percentage oif normal transactions: ', len(under_sample_data[under_sample_data.Class == 0]) / len(under_sample_data))
print('Percentage oif fraud transactions: ', len(under_sample_data[under_sample_data.Class == 1]) / len(under_sample_data))
print('Total number of transactions in resampled data: ', len(under_sample_data))
Percentage oif normal transactions: 0.5
Percentage oif fraud transactions: 0.5
Total number of transactions in resampled data: 984
In [39]:
# 交叉验证， train_test_split: 切分数据
from sklearn.cross_validation import train_test_split
# Whole dataset, 将数据切割成训练集0.7 和测试集 0.3
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 0)
print('Number transactions train dataset: ', len(X_train))
print('Number transactions test dataset: ', len(X_test))
print('Total number of transactions: ', len(X_train) + len(X_test))
# Undersampled dataset(对下采样数据集相同操作)
# 只用下采样模型进行训练
# 最终用原始数据集进行测试
X_train_undersample, X_test_undersample, y_train_undersample, y_test_undersample = train_test_split(X_undersample,
y_undersample,
test_size = 0.3,
random_state = 0)
print('')
print("Number transactions train dataset: ", len(X_train_undersample))
print("Number transactions test dataset: ", len(X_test_undersample))
print("Total number of transactions: ", len(X_train_undersample) + len(X_test_undersample))
Number transactions train dataset: 199364
Number transactions test dataset: 85443
Total number of transactions: 284807
Number transactions train dataset: 688
Number transactions test dataset: 296
Total number of transactions: 984
模型评估标准
-
在类不平衡问题中，用精度来衡量指标是骗人的！没有意义(1000个人，全部预测为正常， 0个癌症) 精度 = TPTotalTPTotal
-
这里使用Recall(召回率, 查全率) = TPTP+FNTPTP+FN, 1000个人(990个正常,10个癌症),如果检测出0个病人 010=0,检测出2个病人210=0.2010=0,检测出2个病人210=0.2
-
检测任务上，通常用Recall作为模型的评估标准
In [40]:
# Recall = TP / (TP + FN)
from sklearn.linear_model import LogisticRegression
# KFold->做几倍的交叉验证
from sklearn.cross_validation import KFold, cross_val_score
# confusion_matrix: 混淆矩阵
from sklearn.metrics import confusion_matrix, recall_score, classification_report
正则化惩罚
- 设置正则化惩罚项
- 希望当前模型的泛化能力(更稳定一些)， 不仅满足训练数据，还要在测试数据上尽可能满足
- 浮动的差异小 ===> 过拟合的风险小
- 惩罚 θθ (一组分布大A，一组分布小B) => 大力度惩罚A模型，小力度惩罚B模型
- L2正则化L2正则化: loss函数(越低越好) + 12W2(惩罚项)12W2(惩罚项) => 计算哪个模型loss值小, 分析哪个模型更好
- L1正则化L1正则化: loss+|W|loss+|W|
- λL2:设置惩罚力度λλL2:设置惩罚力度λ
In [41]:
def printing_Kfold_score(x_train_data, y_train_data):
# 切分成5部分，把原始训练集进行切分
fold = KFold(len(y_train_data), 5, shuffle=False)
# Different C parameters (正则化惩罚项)
# 希望当前模型的泛化能力(更稳定一些) ， 不仅满足训练数据，还要在测试数据上尽可能满足
# 浮动的差异小 ===> 过拟合的风险小
c_param_range = [0.01, 0.1, 1, 10, 100]
# C_parameter => lambda
results_table = pd.DataFrame(index = range(len(c_param_range), 2), columns = ['C_parameter', 'Mean recall score'])
results_table['C_parameter'] = c_param_range
# the k-fold will give 2 lists: train_indics = indices[0], test_indices = indices[1]
j = 0
# 查看哪一个C值比较好
for c_param in c_param_range:
print('-----------------------------------------')
print('C parameter: ', c_param)
print('-----------------------------------------')
print('')
recall_accs = []
# 每次交叉验证的结果
for iteration, indices in enumerate(fold, start=1):
# Call the logistic regresstion model with a certain C parameter, "L1惩罚 or L2惩罚"
lr = LogisticRegression(C = c_param, penalty='l1')
# Use the training data to fit the model. In the case, we use the portion of the fold to train
# with indices[0]. We then predict on the portion assigned as the test cross validation with indices.
# 进行训练, 交叉验证里的数据(训练集)-->建立一个模型
lr.fit(x_train_data.iloc[indices[0], :], y_train_data.iloc[indices[0], :].values.ravel())
# Predict values using the test indices in the training data
# 进行预测, 用交叉验证中的验证集 再进行一个验证(测试)的操作
y_pred_undersample = lr.predict(x_train_data.iloc[indices[1], :].values)
# Calculate the recall score and append it to a list for recall scores representing the current xx
# 计算当前模型的recall (indices[1]: 测试集)
recall_acc = recall_score(y_train_data.iloc[indices[1], :].values, y_pred_undersample)
recall_accs.append(recall_acc)
print("Iterator ", iteration, ': recall score = ', recall_acc)
# The mean value of those recall scores is the metric we want to save and get hold of.
results_table.ix[j, 'Mean recall score'] = np.mean(recall_accs)
j += 1
print('')
print('Mean recall score ', np.mean(recall_accs))
print('')
best_c = results_table.loc[results_table['Mean recall score'].idxmax()]['C_parameter']
# Finally, we can check which C parameter is the best amongst the chosen.
print('*********************************************************************************')
print('Best model to choose from cross validation is with C parameter = ', best_c)
print('*********************************************************************************')
return best_c
In [42]:
best_c = printing_Kfold_score(X_train_undersample, y_train_undersample)
[out]:
混淆矩阵
- 如何绘制混淆矩阵
- 混淆矩阵用途
In [45]:
lr = LogisticRegression(C = best_c, penalty='l1')
lr.fit(X_train_undersample, y_train_undersample.values.ravel())
y_pred = lr.predict(X_test.values)
# COmputer confusion matrix
cnf_matrix = confusion_matrix(y_test, y_pred)
np.set_printoptions(precision=2)
print('Recall metric in the testing dataset: ', cnf_matrix[1, 1] / (cnf_matrix[1, 0] + cnf_matrix[1, 1]))
# Plot non-normalized confusion matrix
class_names = [0, 1]
plt.figure()
plot_confusion_matrix(cnf_matrix,
classes=class_names,
title='Confusion matrix')
plt.show()
# Recall = TP / (TP + FN)
# 7101 ==> 误杀了那么多数据
注: 下采样伴随着误杀的问题，精度不高
In [46]:
# 什么都不做的话
# best_c = printing_Kfold_score(X_train, y_train)
Logical 回归
- 阈值对结果的影响
In [47]:
lr = LogisticRegression(C = 0.01, penalty='l1')
lr.fit(X_train_undersample, y_train_undersample.values.ravel())
y_pred_undersample_proba = lr.predict_proba(X_test_undersample.values)
# 设置不同的阈值
thresholds = [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
plt.figure(figsize=(10, 10))
j = 1
for i in thresholds:
y_test_undersample_high_recall = y_pred_undersample_proba[:, 1] > i
plt.subplot(3, 3, j)
j += 1
# Compute confusion matrix
cnf_matrix = confusion_matrix(y_test_undersample, y_test_undersample_high_recall)
np.set_printoptions(precision=2)
print('Recal metric in the testing dataset: ', cnf_matrix[1, 1] / (cnf_matrix[1, 0] + cnf_matrix[1, 1]))
# Plot non-normalized confusion matrix
class_names = [0, i]
plot_confusion_matrix(cnf_matrix,
classes=class_names,
title='Threshold >= %s' %i)
[out]:
过采样 — SMOTE样本生成策略
- 对于少数类中每一个样本x, 以欧氏距离为标准计算它到少数类样本集中所有样本的距离，得到其k近邻。
- 根据样本不平衡比例设置一个采样比例以确定采样倍率N，对于每一个少数类样本x, 从其k近邻中随机选择若干样本，假设选择的近邻为 xn。
- 对于每一个随机选出的近邻 xn, 分别与原样本按照如下的公式构建新的样本。
- Xnew=X+rand(0,1)×欧式距离Xnew=X+rand(0,1)×欧式距离
In [48]:
import pandas as pd
from imblearn.over_sampling import SMOTE
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import train_test_split
In [49]:
credit_cards = pd.read_csv('creditcard.csv')
columns = credit_cards.columns
# The labels are in the last column ('Class'), Simply remove it to obtain features cloumns
features_columns = columns.delete(len(columns) - 1)
features = credit_cards[features_columns]
labels = credit_cards['Class']
In [50]:
features_train, features_test, labels_train, labels_test = train_test_split(features,
labels,
test_size=0.2,
random_state=0)
In [51]:
oversampler = SMOTE(random_state=0)
os_features, os_labels = oversampler.fit_sample(features_train, labels_train)
In [ ]:
len(os_labels[os_labels == 1])
Out[ ]:
In [ ]:
os_features = pd.DataFrame(os_features)
os_labels = pd.DataFrame(os_labels)
best_c = printing_Kfold_score(os_features, os_labels)
Out[]:
In [56]:
lr = LogisticRegression(C = best_c, penalty='l1')
lr.fit(os_features, os_labels.values.ravel())
y_pred = lr.predict(features_test.values)
# Compute confusion matrix
cnf_matrix = confusion_matrix(labels_test, y_pred)
np.set_printoptions(precision=2)
print("Recall metric in the testing dataset: ", cnf_matrix[1, 1] / (cnf_matrix[1, 0] + cnf_matrix[1, 1]))
# Plot non-nonmalized confusion matrix
class_names = [0, 1]
plt.figure()
plot_confusion_matrix(cnf_matrix, classes = class_names,
title = 'Confusion matrix')
plt.show()
过采样--> Recal 低一些，模型整体的精度更高一些
- 精度 = (TP + TN) / Total
In [ ]:
In [ ]:
