python MLP 神经网络使用 MinMaxScaler 没有 StandardScaler效果好
MLP 64,2 preprocessing.MinMaxScaler().fit(X)
test confusion_matrix:
[[129293 2734]
[ 958 23375]]
precision recall f1-score support
0 0.99 0.98 0.99 132027
1 0.90 0.96 0.93 24333
avg / total 0.98 0.98 0.98 156360
all confusion_matrix:
[[646945 13384]
[ 4455 117015]]
precision recall f1-score support
0 0.99 0.98 0.99 660329
1 0.90 0.96 0.93 121470
avg / total 0.98 0.98 0.98 781799
black verify confusion_matrix:
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0
0 0 0 0 0]
/root/anaconda2/lib/python2.7/site-packages/sklearn/metrics/classification.py:1137: UndefinedMetricWarning: Recall and F-score are ill-defined and being set to 0.0 in labels with no true samples.
'recall', 'true', average, warn_for)
precision recall f1-score support
0 0.00 0.00 0.00 0
1 1.00 0.07 0.13 42
avg / total 1.00 0.07 0.13 42
white verify confusion_matrix:
[1 1 1 1 1 1 0]
precision recall f1-score support
0 1.00 0.14 0.25 7
1 0.00 0.00 0.00 0
avg / total 1.00 0.14 0.25 7
unknown_verify:
[1 0 0 1 1 0 0 0 1 1 0 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 0 0 1 1 1 1
0 1 1 1 1 0 1 0 0 1 0 1 0 1 0 0 1 0 0 1 1 0 0 1 0 0 0 1 0 1 1 0 0 1 0 0 0]
MLP 64,2 使用preprocessing.StandardScaler().fit(X)
[[131850 180]
[ 230 24100]]
precision recall f1-score support
0 1.00 1.00 1.00 132030
1 0.99 0.99 0.99 24330
avg / total 1.00 1.00 1.00 156360
all confusion_matrix:
[[659500 829]
[ 1195 120275]]
precision recall f1-score support
0 1.00 1.00 1.00 660329
1 0.99 0.99 0.99 121470
avg / total 1.00 1.00 1.00 781799
black verify confusion_matrix:
[0 1 1 0 0 0 0 1 1 1 0 1 1 1 1 1 1 0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1 1 1 1 1
0 0 0 1 1]
/root/anaconda2/lib/python2.7/site-packages/sklearn/metrics/classification.py:1137: UndefinedMetricWarning: Recall and F-score are ill-defined and being set to 0.0 in labels with no true samples.
'recall', 'true', average, warn_for)
precision recall f1-score support
0 0.00 0.00 0.00 0
1 1.00 0.62 0.76 42
avg / total 1.00 0.62 0.76 42
white verify confusion_matrix:
[0 0 1 0 1 1 0]
precision recall f1-score support
0 1.00 0.57 0.73 7
1 0.00 0.00 0.00 0
avg / total 1.00 0.57 0.73 7
unknown_verify:
[1 0 0 0 1 0 1 1 0 0 1 0 1 1 0 1 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 1 0 0
0 1 1 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0]
代码:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | from sklearn import preprocessing scaler = preprocessing.StandardScaler().fit(X) #scaler = preprocessing.MinMaxScaler().fit(X) X = scaler.transform(X) print ( "standard X sample:" , X[: 3 ]) black_verify = scaler.transform(black_verify) print (black_verify) white_verify = scaler.transform(white_verify) print (white_verify) unknown_verify = scaler.transform(unknown_verify) print (unknown_verify) # ValueError: operands could not be broadcast together with shapes (756140,75) (42,75) (756140,75) for i in range ( 20 ): X = np.concatenate((X, black_verify)) y + = black_verify_labels labels = [ 'white' , 'CC' ] if True : # pdb.set_trace() ratio_of_train = 0.8 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = ( 1 - ratio_of_train)) # X_train=preprocessing.normalize(X_train) # X_test=preprocessing.normalize(X_test) clf = MLPClassifier(solver = 'sgd' , batch_size = 128 , learning_rate = 'adaptive' , max_iter = 256 , hidden_layer_sizes = ( 64 , 2 ), random_state = 1 ) """ clf = sklearn.ensemble.RandomForestClassifier(n_estimators=n_estimators, verbose=verbose, n_jobs=n_jobs, random_state=random_state, oob_score=True) """ clf.fit(X_train, y_train) print "test confusion_matrix:" # print clf.feature_importances_ y_pred = clf.predict(X_test) print (sklearn.metrics.confusion_matrix(y_test, y_pred)) print (classification_report(y_test, y_pred)) else : #clf = pickle.loads(open("mpl-acc97-recall98.pkl", 'rb').read()) clf = pickle.loads( open ( "mlp-add-topx10.model" , 'rb' ).read()) y_pred = clf.predict(X) print (sklearn.metrics.confusion_matrix(y, y_pred)) print (classification_report(y, y_pred)) import sys #sys.exit(0) print "all confusion_matrix:" y_pred = clf.predict(X) print (sklearn.metrics.confusion_matrix(y, y_pred)) print (classification_report(y, y_pred)) |
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· 记一次.NET内存居高不下排查解决与启示
· 探究高空视频全景AR技术的实现原理
· 理解Rust引用及其生命周期标识(上)
· 浏览器原生「磁吸」效果!Anchor Positioning 锚点定位神器解析
· 没有源码,如何修改代码逻辑?
· 全程不用写代码,我用AI程序员写了一个飞机大战
· MongoDB 8.0这个新功能碉堡了,比商业数据库还牛
· 记一次.NET内存居高不下排查解决与启示
· 白话解读 Dapr 1.15:你的「微服务管家」又秀新绝活了
· DeepSeek 开源周回顾「GitHub 热点速览」
2017-05-24 LMDB中的mmap、Copy On Write、MVCC深入理解——讲得非常好,常来看看!
2017-05-24 golang LMDB入门例子——key range查询
2017-05-24 golang LMDB入门例子——尼玛,LMDB的文档真的是太少了
2017-05-24 golang import all 类似python import * 效果
2017-05-24 python lmdb demo 这接口和BDB一样恶心啊!
2017-05-24 ledisDB底层实现——本质上就是用leveldb这样的底层存储,和ssdb一样,meta里存的是hash、list等的元数据
2017-05-24 ssdb底层实现——ssdb底层是leveldb,leveldb根本上是skiplist(例如为存储多个list items,必然有多个item key,而非暴力string cat),用它来做redis的list和set等,势必在数据结构和算法层面上有诸多不适