【心跳信号分类预测】Datawhale打卡- Task4 建模与调参 (贝叶斯搜索调参调包实践)

自学打卡,请勿推荐,谢谢!

比赛地址
https://tianchi.aliyun.com/competition/entrance/531883/information

4.1 学习目标

  • 学习机器学习模型的建模过程与调参流程
  • 完成相应学习打卡任务

4.2 内容介绍 (以下内容已更新为自己博客的链接)

4.4 模型对比与性能评估

4.4.1 逻辑回归 (核心就是sigmod函数)

  • 优点

    • 训练速度较快，分类的时候，计算量仅仅只和特征的数目相关；
    • 简单易理解，模型的可解释性非常好，从特征的权重可以看到不同的特征对最后结果的影响；
    • 适合二分类问题，不需要缩放输入特征；
    • 内存资源占用小，只需要存储各个维度的特征值；

  • 缺点

    • 逻辑回归需要预先处理缺失值和异常值【可参考task3特征工程】；

    • 不能用Logistic回归去解决非线性问题，因为Logistic的决策面是线性的；

    • 对多重共线性数据较为敏感，且很难处理数据不平衡的问题；

    • 准确率并不是很高，因为形式非常简单，很难去拟合数据的真实分布；

4.4.2 决策树模型 (二叉搜索树模型)

  • 优点
    • 简单直观，生成的决策树可以可视化展示
    • 数据不需要预处理，不需要归一化，不需要处理缺失数据
    • 既可以处理离散值，也可以处理连续值
  • 缺点
    • 决策树算法非常容易过拟合，导致泛化能力不强（可进行适当的剪枝）
    • 采用的是贪心算法，容易得到局部最优解

4.4.3 集成模型集成方法（ensemble method）

通过组合多个学习器来完成学习任务，通过集成方法，可以将多个弱学习器组合成一个强分类器，因此集成学习的泛化能力一般比单一分类器要好。

集成方法主要包括Bagging和Boosting，Bagging和Boosting都是将已有的分类或回归算法通过一定方式组合起来，形成一个更加强大的分类。两种方法都是把若干个分类器整合为一个分类器的方法，只是整合的方式不一样，最终得到不一样的效果。常见的基于Baggin思想的集成模型有：随机森林、基于Boosting思想的集成模型有：Adaboost、GBDT、XgBoost、LightGBM等。

Baggin和Boosting的区别总结如下：

  • 样本选择上： Bagging方法的训练集是从原始集中有放回的选取，所以从原始集中选出的各轮训练集之间是独立的；而Boosting方法需要每一轮的训练集不变，只是训练集中每个样本在分类器中的权重发生变化。而权值是根据上一轮的分类结果进行调整
  • 样例权重上： Bagging方法使用均匀取样，所以每个样本的权重相等；而Boosting方法根据错误率不断调整样本的权值，错误率越大则权重越大
  • 预测函数上： Bagging方法中所有预测函数的权重相等；而Boosting方法中每个弱分类器都有相应的权重，对于分类误差小的分类器会有更大的权重
  • 并行计算上： Bagging方法中各个预测函数可以并行生成；而Boosting方法各个预测函数只能顺序生成，因为后一个模型参数需要前一轮模型的结果。

4.4.4 模型评估方法

对于模型来说，其在训练集上面的误差我们称之为训练误差或者经验误差，而在测试集上的误差称之为测试误差。

对于我们来说，我们更关心的是模型对于新样本的学习能力，即我们希望通过对已有样本的学习，尽可能的将所有潜在样本的普遍规律学到手，而如果模型对训练样本学的太好，则有可能把训练样本自身所具有的一些特点当做所有潜在样本的普遍特点，这时候我们就会出现过拟合的问题。

因此我们通常将已有的数据集划分为训练集和测试集两部分，其中训练集用来训练模型，而测试集则是用来评估模型对于新样本的判别能力。

对于数据集的划分，我们通常要保证满足以下两个条件：

  • 训练集和测试集的分布要与样本真实分布一致，即训练集和测试集都要保证是从样本真实分布中独立同分布采样而得；
  • 训练集和测试集要互斥

对于数据集的划分有三种方法：留出法，交叉验证法和自助法，下面挨个介绍：

  • ①留出法

    留出法是直接将数据集D划分为两个互斥的集合，其中一个集合作为训练集S，另一个作为测试集T。需要注意的是在划分的时候要尽可能保证数据分布的一致性，即避免因数据划分过程引入额外的偏差而对最终结果产生影响。为了保证数据分布的一致性，通常我们采用分层采样的方式来对数据进行采样。

    Tips： 通常，会将数据集D中大约2/3~4/5的样本作为训练集，其余的作为测试集。

  • ②交叉验证法

    k折交叉验证通常将数据集D分为k份，其中k-1份作为训练集，剩余的一份作为测试集，这样就可以获得k组训练/测试集，可以进行k次训练与测试，最终返回的是k个测试结果的均值。交叉验证中数据集的划分依然是依据分层采样的方式来进行。

    对于交叉验证法，其k值的选取往往决定了评估结果的稳定性和保真性，通常k值选取10。

    当k=1的时候，我们称之为留一法

  • ③自助法

    我们每次从数据集D中取一个样本作为训练集中的元素，然后把该样本放回，重复该行为m次，这样我们就可以得到大小为m的训练集，在这里面有的样本重复出现，有的样本则没有出现过，我们把那些没有出现过的样本作为测试集。

    进行这样采样的原因是因为在D中约有36.8%的数据没有在训练集中出现过。留出法与交叉验证法都是使用分层采样的方式进行数据采样与划分，而自助法则是使用有放回重复采样的方式进行数据采样

数据集划分总结

  • 对于数据量充足的时候，通常采用留出法或者k折交叉验证法来进行训练/测试集的划分；
  • 对于数据集小且难以有效划分训练/测试集时使用自助法；
  • 对于数据集小且可有效划分的时候最好使用留一法来进行划分，因为这种方法最为准确

4.4.5 模型评价标准

对于本次比赛，我们选用自定义的abs-sum作为模型评价标准。

1. 贝叶斯调参

在使用之前需要先安装包bayesian-optimization，运行如下命令即可：

pip install bayesian-optimization

贝叶斯调参的主要思想是：给定优化的目标函数(广义的函数，只需指定输入和输出即可，无需知道内部结构以及数学性质)，通过不断地添加样本点来更新目标函数的后验分布(高斯过程,直到后验分布基本贴合于真实分布）。简单的说，就是考虑了上一次参数的信息，从而更好的调整当前的参数。

贝叶斯调参的步骤如下：

  • 定义优化函数(rf_cv）
  • 建立模型
  • 定义待优化的参数
  • 得到优化结果，并返回要优化的分数指标
#############################################

### baseline

#### 1.导入第三方包

import time
import warnings
import lightgbm as lgb
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.model_selection import KFold
from sklearn.preprocessing import OneHotEncoder
warnings.filterwarnings('ignore')
"""
sns 相关设置
@return:
"""
import datetime
import seaborn as sns
sns.set()
sns.set_style("whitegrid")
sns.set_context('talk')
plt.rcParams['font.sans-serif'] = ['SimHei']
plt.rcParams['axes.unicode_minus'] = False
sns.set(font='SimHei')

t0 = time.time()
print('import done, sns & plt preset done, clock() start', datetime.datetime.now())

import done, sns & plt preset done, clock() start 2021-03-22 10:51:00.725114

#### 1.1 公共变量
win_file_path = 'E:\\competition-data\\016_heartbeat_signals\\'

now = datetime.datetime.now().strftime('%Y-%m-%d_%H_%M_%S')

out_path = 'E:\\PycharmProjects\\TianChiProject\\00_hardworker\\competitions\\016_heartbeat_signals\\predict_result\\'+ 'dw_baseline_merror_predict_{}.csv'.format(now)
print('out_path: ', out_path)

out_path:  E:\PycharmProjects\TianChiProject\00_hardworker\competitions\016_heartbeat_signals\predict_result\dw_baseline_merror_predict_2021-03-22_10_51_25.csv

# mode = 'baidu'
mode = 'win'
print('\n\n\nmode:{}\n\n'.format(mode))

if mode == 'baidu':
    win_file_path = 'data/data74975/'
    out_path = 'work/heartbeats_data/' + 'dw_baseline_merror_subsample_{}.csv'.format(now)


mode:win

2.读取数据

train = pd.read_csv(win_file_path+'train.csv')
# test=pd.read_csv(win_file_path+'testA.csv')
train.head()
id heartbeat_signals label
0 0 0.9912297987616655,0.9435330436439665,0.764677... 0.0
1 1 0.9714822034884503,0.9289687459588268,0.572932... 0.0
2 2 1.0,0.9591487564065292,0.7013782792997189,0.23... 2.0
3 3 0.9757952826275774,0.9340884687738161,0.659636... 0.0
4 4 0.0,0.055816398940721094,0.26129357194994196,0... 2.0 
# test.head()

3.数据预处理

def reduce_mem_usage(df):
    start_mem = df.memory_usage().sum() / 1024**2
    print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))

    for col in df.columns:
        col_type = df[col].dtype

        if col_type != object:
            c_min = df[col].min()
            c_max = df[col].max()
            if str(col_type)[:3] == 'int':
                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:
                    df[col] = df[col].astype(np.int8)
                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:
                    df[col] = df[col].astype(np.int16)
                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:
                    df[col] = df[col].astype(np.int32)
                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:
                    df[col] = df[col].astype(np.int64)
            else:
                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:
                    df[col] = df[col].astype(np.float16)
                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:
                    df[col] = df[col].astype(np.float32)
                else:
                    df[col] = df[col].astype(np.float64)
        else:
            df[col] = df[col].astype('category')

    end_mem = df.memory_usage().sum() / 1024**2
    print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))
    print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))

    return df


# 简单预处理
train_list = []

for items in train.values:
    train_list.append([items[0]] + [float(i) for i in items[1].split(',')] + [items[2]])

train = pd.DataFrame(np.array(train_list))
train.columns = ['id'] + ['s_' + str(i) for i in range(len(train_list[0])-2)] + ['label']
train = reduce_mem_usage(train)



Memory usage of dataframe is 157.93 MB
Memory usage after optimization is: 39.67 MB
Decreased by 74.9%

train.head()
id s_0 s_1 s_2 s_3 s_4 s_5 s_6 s_7 s_8 ... s_196 s_197 s_198 s_199 s_200 s_201 s_202 s_203 s_204 label
0 0.0 0.991211 0.943359 0.764648 0.618652 0.379639 0.190796 0.040222 0.026001 0.031708 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
1 1.0 0.971680 0.929199 0.572754 0.178467 0.122986 0.132324 0.094421 0.089600 0.030487 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
2 2.0 1.000000 0.958984 0.701172 0.231812 0.000000 0.080688 0.128418 0.187500 0.280762 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0
3 3.0 0.975586 0.934082 0.659668 0.249878 0.237061 0.281494 0.249878 0.249878 0.241455 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
4 4.0 0.000000 0.055817 0.261230 0.359863 0.433105 0.453613 0.499023 0.542969 0.616699 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0

5 rows × 207 columns

train['label'].value_counts()


0.0    64327
3.0    17912
2.0    14199
1.0     3562
Name: label, dtype: int64

4.训练数据/测试数据准备

x_train = train.drop(['id','label'], axis=1)
y_train = train['label']


from sklearn.model_selection import cross_val_score
from sklearn.metrics import make_scorer,f1_score

"""定义优化函数"""

def rf_cv_lgb(num_leaves, max_depth, bagging_fraction, feature_fraction, bagging_freq, min_data_in_leaf,
            min_child_weight, min_split_gain, reg_lambda, reg_alpha):
  # 建立模型
  model_lgb = \
      lgb.LGBMClassifier(
          boosting_type='gbdt',
          objective='multiclass',
          metric='multi_error', # 多分类的错误率
          num_class=4,
          learning_rate=0.02,
          n_estimators=5000,
          num_leaves=int(num_leaves),
          max_depth=int(max_depth),
          bagging_fraction=round(bagging_fraction, 2),
          feature_fraction=round(feature_fraction, 2),

          bagging_freq=int(bagging_freq),
          min_data_in_leaf=int(min_data_in_leaf),
          min_child_weight=min_child_weight,
          min_split_gain=min_split_gain,

          reg_lambda=reg_lambda,
          reg_alpha=reg_alpha,
          n_jobs= 15
           )

  f1 = make_scorer(f1_score, average='micro')
  val = cross_val_score(model_lgb, x_train, y_train, cv=5, scoring=f1).mean()

  return val

from bayes_opt import BayesianOptimization
"""定义优化参数"""
bayes_lgb = BayesianOptimization(
  rf_cv_lgb,
  {
      'num_leaves':(10, 200),
      'max_depth':(3, 20),
      'bagging_fraction':(0.5, 1.0),
      'feature_fraction':(0.5, 1.0),
      'bagging_freq':(0, 100),
      'min_data_in_leaf':(10,100),
      'min_child_weight':(0, 10),
      'min_split_gain':(0.0, 1.0),
      'reg_alpha':(0.0, 10),
      'reg_lambda':(0.0, 10),
  }
)

"""开始优化"""
bayes_lgb.maximize(n_iter=10)

|   iter    |  target   | baggin... | baggin... | featur... | max_depth | min_ch... | min_da... | min_sp... | num_le... | reg_alpha | reg_la... |
-------------------------------------------------------------------------------------------------------------------------------------------------
[LightGBM] [Warning] feature_fraction is set=0.96, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.96
[LightGBM] [Warning] min_data_in_leaf is set=56, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=56
[LightGBM] [Warning] bagging_fraction is set=0.89, subsample=1.0 will be ignored. Current value: bagging_fraction=0.89
[LightGBM] [Warning] bagging_freq is set=71, subsample_freq=0 will be ignored. Current value: bagging_freq=71
[LightGBM] [Warning] feature_fraction is set=0.96, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.96
[LightGBM] [Warning] min_data_in_leaf is set=56, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=56
[LightGBM] [Warning] bagging_fraction is set=0.89, subsample=1.0 will be ignored. Current value: bagging_fraction=0.89
[LightGBM] [Warning] bagging_freq is set=71, subsample_freq=0 will be ignored. Current value: bagging_freq=71
[LightGBM] [Warning] feature_fraction is set=0.96, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.96
[LightGBM] [Warning] min_data_in_leaf is set=56, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=56
[LightGBM] [Warning] bagging_fraction is set=0.89, subsample=1.0 will be ignored. Current value: bagging_fraction=0.89
[LightGBM] [Warning] bagging_freq is set=71, subsample_freq=0 will be ignored. Current value: bagging_freq=71
[LightGBM] [Warning] feature_fraction is set=0.96, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.96
[LightGBM] [Warning] min_data_in_leaf is set=56, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=56
[LightGBM] [Warning] bagging_fraction is set=0.89, subsample=1.0 will be ignored. Current value: bagging_fraction=0.89
[LightGBM] [Warning] bagging_freq is set=71, subsample_freq=0 will be ignored. Current value: bagging_freq=71
[LightGBM] [Warning] feature_fraction is set=0.96, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.96
[LightGBM] [Warning] min_data_in_leaf is set=56, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=56
[LightGBM] [Warning] bagging_fraction is set=0.89, subsample=1.0 will be ignored. Current value: bagging_fraction=0.89
[LightGBM] [Warning] bagging_freq is set=71, subsample_freq=0 will be ignored. Current value: bagging_freq=71
|  1        |  0.9765   |  0.8884   |  71.15    |  0.9562   |  4.636    |  6.249    |  56.51    |  0.5129   |  97.85    |  7.626    |  1.75     |
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=15, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=15
[LightGBM] [Warning] bagging_fraction is set=0.99, subsample=1.0 will be ignored. Current value: bagging_fraction=0.99
[LightGBM] [Warning] bagging_freq is set=55, subsample_freq=0 will be ignored. Current value: bagging_freq=55
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=15, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=15
[LightGBM] [Warning] bagging_fraction is set=0.99, subsample=1.0 will be ignored. Current value: bagging_fraction=0.99
[LightGBM] [Warning] bagging_freq is set=55, subsample_freq=0 will be ignored. Current value: bagging_freq=55
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=15, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=15
[LightGBM] [Warning] bagging_fraction is set=0.99, subsample=1.0 will be ignored. Current value: bagging_fraction=0.99
[LightGBM] [Warning] bagging_freq is set=55, subsample_freq=0 will be ignored. Current value: bagging_freq=55
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=15, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=15
[LightGBM] [Warning] bagging_fraction is set=0.99, subsample=1.0 will be ignored. Current value: bagging_fraction=0.99
[LightGBM] [Warning] bagging_freq is set=55, subsample_freq=0 will be ignored. Current value: bagging_freq=55
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=15, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=15
[LightGBM] [Warning] bagging_fraction is set=0.99, subsample=1.0 will be ignored. Current value: bagging_fraction=0.99
[LightGBM] [Warning] bagging_freq is set=55, subsample_freq=0 will be ignored. Current value: bagging_freq=55
|  2        |  0.9858   |  0.9862   |  55.81    |  0.5786   |  4.557    |  6.03     |  15.8     |  0.1627   |  165.2    |  0.01718  |  4.919    |
[LightGBM] [Warning] feature_fraction is set=0.66, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.66
[LightGBM] [Warning] min_data_in_leaf is set=12, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=12
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=39, subsample_freq=0 will be ignored. Current value: bagging_freq=39
[LightGBM] [Warning] feature_fraction is set=0.66, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.66
[LightGBM] [Warning] min_data_in_leaf is set=12, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=12
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=39, subsample_freq=0 will be ignored. Current value: bagging_freq=39
[LightGBM] [Warning] feature_fraction is set=0.66, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.66
[LightGBM] [Warning] min_data_in_leaf is set=12, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=12
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=39, subsample_freq=0 will be ignored. Current value: bagging_freq=39
[LightGBM] [Warning] feature_fraction is set=0.66, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.66
[LightGBM] [Warning] min_data_in_leaf is set=12, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=12
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=39, subsample_freq=0 will be ignored. Current value: bagging_freq=39
[LightGBM] [Warning] feature_fraction is set=0.66, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.66
[LightGBM] [Warning] min_data_in_leaf is set=12, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=12
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=39, subsample_freq=0 will be ignored. Current value: bagging_freq=39
|  3        |  0.9832   |  0.708    |  39.78    |  0.6577   |  7.057    |  9.023    |  12.94    |  0.585    |  120.1    |  1.478    |  9.095    |
[LightGBM] [Warning] feature_fraction is set=0.55, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.55
[LightGBM] [Warning] min_data_in_leaf is set=46, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=46
[LightGBM] [Warning] bagging_fraction is set=0.76, subsample=1.0 will be ignored. Current value: bagging_fraction=0.76
[LightGBM] [Warning] bagging_freq is set=25, subsample_freq=0 will be ignored. Current value: bagging_freq=25
[LightGBM] [Warning] feature_fraction is set=0.55, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.55
[LightGBM] [Warning] min_data_in_leaf is set=46, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=46
[LightGBM] [Warning] bagging_fraction is set=0.76, subsample=1.0 will be ignored. Current value: bagging_fraction=0.76
[LightGBM] [Warning] bagging_freq is set=25, subsample_freq=0 will be ignored. Current value: bagging_freq=25
[LightGBM] [Warning] feature_fraction is set=0.55, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.55
[LightGBM] [Warning] min_data_in_leaf is set=46, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=46
[LightGBM] [Warning] bagging_fraction is set=0.76, subsample=1.0 will be ignored. Current value: bagging_fraction=0.76
[LightGBM] [Warning] bagging_freq is set=25, subsample_freq=0 will be ignored. Current value: bagging_freq=25
[LightGBM] [Warning] feature_fraction is set=0.55, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.55
[LightGBM] [Warning] min_data_in_leaf is set=46, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=46
[LightGBM] [Warning] bagging_fraction is set=0.76, subsample=1.0 will be ignored. Current value: bagging_fraction=0.76
[LightGBM] [Warning] bagging_freq is set=25, subsample_freq=0 will be ignored. Current value: bagging_freq=25
[LightGBM] [Warning] feature_fraction is set=0.55, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.55
[LightGBM] [Warning] min_data_in_leaf is set=46, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=46
[LightGBM] [Warning] bagging_fraction is set=0.76, subsample=1.0 will be ignored. Current value: bagging_fraction=0.76
[LightGBM] [Warning] bagging_freq is set=25, subsample_freq=0 will be ignored. Current value: bagging_freq=25
|  4        |  0.9867   |  0.7604   |  25.11    |  0.5506   |  8.891    |  6.293    |  46.42    |  0.1518   |  165.5    |  1.392    |  8.024    |
[LightGBM] [Warning] feature_fraction is set=0.9, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.9
[LightGBM] [Warning] min_data_in_leaf is set=80, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=80
[LightGBM] [Warning] bagging_fraction is set=0.92, subsample=1.0 will be ignored. Current value: bagging_fraction=0.92
[LightGBM] [Warning] bagging_freq is set=84, subsample_freq=0 will be ignored. Current value: bagging_freq=84
[LightGBM] [Warning] feature_fraction is set=0.9, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.9
[LightGBM] [Warning] min_data_in_leaf is set=80, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=80
[LightGBM] [Warning] bagging_fraction is set=0.92, subsample=1.0 will be ignored. Current value: bagging_fraction=0.92
[LightGBM] [Warning] bagging_freq is set=84, subsample_freq=0 will be ignored. Current value: bagging_freq=84
[LightGBM] [Warning] feature_fraction is set=0.9, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.9
[LightGBM] [Warning] min_data_in_leaf is set=80, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=80
[LightGBM] [Warning] bagging_fraction is set=0.92, subsample=1.0 will be ignored. Current value: bagging_fraction=0.92
[LightGBM] [Warning] bagging_freq is set=84, subsample_freq=0 will be ignored. Current value: bagging_freq=84

显示优化结果

bayes_lgb.max

{'target': 0.9867100000000001,
 'params': {'bagging_fraction': 0.7603792587322773,
  'bagging_freq': 25.11487816145165,
  'feature_fraction': 0.5506325820307314,
  'max_depth': 8.890642924214916,
  'min_child_weight': 6.293486431144558,
  'min_data_in_leaf': 46.420027874232126,
  'min_split_gain': 0.1517965938219028,
  'num_leaves': 165.51283932289553,
  'reg_alpha': 1.3916293713985783,
  'reg_lambda': 8.023989846968007}}

使用后, 偏差更高,方差也增加, 所以暂不使用.

其他

暂时还没有遇到瓶颈,暂时不需要进行模型调参!

posted @ 2021-04-21 23:47  山枫叶纷飞  阅读(281)  评论(0)  编辑  收藏  举报