【天池】心跳信号分类预测 baseline Part1

2024-03-29 20:08

本文主要是介绍【天池】心跳信号分类预测 baseline Part1,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

1 导包

import os 
import gc 
import mathimport pandas as pd
import numpy as npimport lightgbm as lgb
import xgboost as xgb
from catboost import CatBoostRegressor
from sklearn.linear_model import SGDRegressor, LinearRegression, Ridge
from sklearn.preprocessing import MinMaxScalerfrom sklearn.model_selection import StratifiedKFold,KFold
from sklearn.metrics import log_loss
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import OneHotEncoderfrom tqdm import tqdm
import matplotlib.pyplot as plt
import time
import warnings
warnings.filterwarnings('ignore')

2 导入数据

train = pd.read_csv('./train.csv')
test = pd.read_csv('./testA.csv')

3 数据探索

train.head()
idheartbeat_signalslabel
000.9912297987616655,0.9435330436439665,0.764677...0.0
110.9714822034884503,0.9289687459588268,0.572932...0.0
221.0,0.9591487564065292,0.7013782792997189,0.23...2.0
330.9757952826275774,0.9340884687738161,0.659636...0.0
440.0,0.055816398940721094,0.26129357194994196,0...2.0
train['label'].value_counts()
0.0    64327
3.0    17912
2.0    14199
1.0     3562
Name: label, dtype: int64

4 数据预处理

# 通过转换数据格式减小数据内存消耗
def reduce_mem_usage(df):start_mem = df.memory_usage().sum() / 1024**2 print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))for col in df.columns:col_type = df[col].dtypeif col_type != object:c_min = df[col].min()c_max = df[col].max()if str(col_type)[:3] == 'int':if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:df[col] = df[col].astype(np.int8)elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:df[col] = df[col].astype(np.int16)elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:df[col] = df[col].astype(np.int32)elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:df[col] = df[col].astype(np.int64)  else:if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:df[col] = df[col].astype(np.float16)elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:df[col] = df[col].astype(np.float32)else:df[col] = df[col].astype(np.float64)else:df[col] = df[col].astype('category')end_mem = df.memory_usage().sum() / 1024**2 print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))return df
# 将数据简单处理:数据切割以及打标签
train_list = []for items in train.values:train_list.append([items[0]] + [float(i) for i in items[1].split(',')] + [items[2]])train = pd.DataFrame(np.array(train_list))
train.columns = ['id'] + ['s_'+str(i) for i in range(len(train_list[0])-2)] + ['label']
train = reduce_mem_usage(train)test_list=[]
for items in test.values:test_list.append([items[0]] + [float(i) for i in items[1].split(',')])test = pd.DataFrame(np.array(test_list))
test.columns = ['id'] + ['s_'+str(i) for i in range(len(test_list[0])-1)]
test = reduce_mem_usage(test)
Memory usage of dataframe is 157.93 MB
Memory usage after optimization is: 39.67 MB
Decreased by 74.9%
Memory usage of dataframe is 31.43 MB
Memory usage after optimization is: 7.90 MB
Decreased by 74.9%
train.head()
ids_0s_1s_2s_3s_4s_5s_6s_7s_8...s_196s_197s_198s_199s_200s_201s_202s_203s_204label
00.00.9912110.9433590.7646480.6186520.3796390.1907960.0402220.0260010.031708...0.00.00.00.00.00.00.00.00.00.0
11.00.9716800.9291990.5727540.1784670.1229860.1323240.0944210.0896000.030487...0.00.00.00.00.00.00.00.00.00.0
22.01.0000000.9589840.7011720.2318120.0000000.0806880.1284180.1875000.280762...0.00.00.00.00.00.00.00.00.02.0
33.00.9755860.9340820.6596680.2498780.2370610.2814940.2498780.2498780.241455...0.00.00.00.00.00.00.00.00.00.0
44.00.0000000.0558170.2612300.3598630.4331050.4536130.4990230.5429690.616699...0.00.00.00.00.00.00.00.00.02.0

5 rows × 207 columns

5 模型训练

x_train = train.drop(['id','label'],axis=1)
y_train = train ['label']
x_test = test.drop(['id'],axis=1)
# 定义结果评价函数
def abs_sum(y_pre,y_tru):y_pre=np.array(y_pre)y_tru=np.array(y_tru)loss=sum(sum(abs(y_pre-y_tru)))return loss
# 训练模型
def cv_model(clf, train_x, train_y, test_x, clf_name):folds = 5seed = 2021kf = KFold(n_splits=folds, shuffle=True, random_state=seed) # 5折交叉验证test = np.zeros((test_x.shape[0],4))cv_scores = []onehot_encoder = OneHotEncoder(sparse=False) # 数据转换for i, (train_index, valid_index) in enumerate(kf.split(train_x, train_y)):print('************************************ {} ************************************'.format(str(i+1)))trn_x, trn_y, val_x, val_y = train_x.iloc[train_index], train_y[train_index], train_x.iloc[valid_index], train_y[valid_index]if clf_name == "lgb":train_matrix = clf.Dataset(trn_x, label=trn_y)valid_matrix = clf.Dataset(val_x, label=val_y)params = {'boosting_type': 'gbdt','objective': 'multiclass','num_class': 4,'num_leaves': 2 ** 5,'feature_fraction': 0.8,'bagging_fraction': 0.8,'bagging_freq': 4,'learning_rate': 0.1,'seed': seed,'nthread': 28,'n_jobs':24,'verbose': -1,}model = clf.train(params, train_set=train_matrix, valid_sets=valid_matrix, num_boost_round=2000, verbose_eval=100, early_stopping_rounds=200)val_pred = model.predict(val_x, num_iteration=model.best_iteration)test_pred = model.predict(test_x, num_iteration=model.best_iteration) val_y=np.array(val_y).reshape(-1, 1)val_y = onehot_encoder.fit_transform(val_y)print('预测的概率矩阵为:')print(test_pred)test += test_predscore=abs_sum(val_y, val_pred)cv_scores.append(score)print(cv_scores)print("%s_scotrainre_list:" % clf_name, cv_scores)print("%s_score_mean:" % clf_name, np.mean(cv_scores))print("%s_score_std:" % clf_name, np.std(cv_scores))test=test/kf.n_splitsreturn test
# 采用基于GBDT算法的LightGBM框架建模,速度更快
def lgb_model(x_train, y_train, x_test):lgb_test = cv_model(lgb, x_train, y_train, x_test, "lgb")return lgb_test
lgb_test = lgb_model(x_train, y_train, x_test)
************************************ 1 ************************************
[LightGBM] [Warning] num_threads is set with nthread=28, will be overridden by n_jobs=24. Current value: num_threads=24
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0525735
[200]	valid_0's multi_logloss: 0.0422444
[300]	valid_0's multi_logloss: 0.0407076
[400]	valid_0's multi_logloss: 0.0420398
Early stopping, best iteration is:
[289]	valid_0's multi_logloss: 0.0405457
预测的概率矩阵为:
[[9.99969791e-01 2.85197261e-05 1.00341946e-06 6.85357631e-07][7.93287264e-05 7.69060914e-04 9.99151590e-01 2.00810971e-08][5.75356884e-07 5.04051497e-08 3.15322414e-07 9.99999059e-01]...[6.79267940e-02 4.30206297e-04 9.31640185e-01 2.81516302e-06][9.99960477e-01 3.94098074e-05 8.34030725e-08 2.94638661e-08][9.88705846e-01 2.14081630e-03 6.67418381e-03 2.47915423e-03]]
[607.0736049372185]
************************************ 2 ************************************
[LightGBM] [Warning] num_threads is set with nthread=28, will be overridden by n_jobs=24. Current value: num_threads=24
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0566626
[200]	valid_0's multi_logloss: 0.0450852
[300]	valid_0's multi_logloss: 0.044078
[400]	valid_0's multi_logloss: 0.0455546
Early stopping, best iteration is:
[275]	valid_0's multi_logloss: 0.0437793
预测的概率矩阵为:
[[9.99991401e-01 7.69109547e-06 6.65504756e-07 2.42084688e-07][5.72380482e-05 1.32812809e-03 9.98614607e-01 2.66534396e-08][2.82123411e-06 4.13195205e-07 1.34026965e-06 9.99995425e-01]...[6.96398024e-02 6.52459907e-04 9.29685742e-01 2.19960932e-05][9.99972366e-01 2.75069005e-05 7.68142933e-08 5.07415018e-08][9.67263676e-01 7.26154408e-03 2.41533542e-02 1.32142531e-03]]
[607.0736049372185, 623.4313863731124]
************************************ 3 ************************************
[LightGBM] [Warning] num_threads is set with nthread=28, will be overridden by n_jobs=24. Current value: num_threads=24
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0498722
[200]	valid_0's multi_logloss: 0.038028
[300]	valid_0's multi_logloss: 0.0358066
[400]	valid_0's multi_logloss: 0.0361478
[500]	valid_0's multi_logloss: 0.0379597
Early stopping, best iteration is:
[340]	valid_0's multi_logloss: 0.0354344
预测的概率矩阵为:
[[9.99972032e-01 2.62406774e-05 1.17282152e-06 5.54230651e-07][1.05242811e-05 6.50215805e-05 9.99924453e-01 6.93812546e-10][1.93240868e-06 1.10384984e-07 3.76773426e-07 9.99997580e-01]...[1.34894410e-02 3.84569683e-05 9.86471555e-01 5.46564350e-07][9.99987431e-01 1.25532882e-05 1.03902298e-08 5.46727770e-09][9.78722948e-01 1.06329839e-02 6.94192038e-03 3.70214810e-03]]
[607.0736049372185, 623.4313863731124, 508.02381607269535]
************************************ 4 ************************************
[LightGBM] [Warning] num_threads is set with nthread=28, will be overridden by n_jobs=24. Current value: num_threads=24
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0564768
[200]	valid_0's multi_logloss: 0.0448698
[300]	valid_0's multi_logloss: 0.0446719
[400]	valid_0's multi_logloss: 0.0470399
Early stopping, best iteration is:
[250]	valid_0's multi_logloss: 0.0438853
预测的概率矩阵为:
[[9.99979692e-01 1.70821979e-05 1.27048476e-06 1.95571841e-06][5.66207785e-05 4.02275314e-04 9.99541086e-01 1.82828519e-08][2.62267451e-06 3.58613522e-07 4.78645006e-06 9.99992232e-01]...[4.56636552e-02 5.69497433e-04 9.53758468e-01 8.37980573e-06][9.99896785e-01 1.02796802e-04 2.46636563e-07 1.72061021e-07][8.70911669e-01 1.73790185e-02 1.04478175e-01 7.23113697e-03]]
[607.0736049372185, 623.4313863731124, 508.02381607269535, 660.4867407547266]
************************************ 5 ************************************
[LightGBM] [Warning] num_threads is set with nthread=28, will be overridden by n_jobs=24. Current value: num_threads=24
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0506398
[200]	valid_0's multi_logloss: 0.0396422
[300]	valid_0's multi_logloss: 0.0381065
[400]	valid_0's multi_logloss: 0.0390162
[500]	valid_0's multi_logloss: 0.0414986
Early stopping, best iteration is:
[324]	valid_0's multi_logloss: 0.0379497
预测的概率矩阵为:
[[9.99993352e-01 6.02902202e-06 1.13002685e-07 5.06277302e-07][1.03959552e-05 5.03778956e-04 9.99485820e-01 5.07638601e-09][1.92568065e-07 5.07155306e-08 4.94690856e-08 9.99999707e-01]...[8.83103121e-03 2.51969353e-05 9.91142776e-01 9.96143937e-07][9.99984791e-01 1.51997858e-05 5.62426491e-09 3.80450197e-09][9.86084001e-01 8.75968498e-04 1.09742304e-02 2.06580027e-03]]
[607.0736049372185, 623.4313863731124, 508.02381607269535, 660.4867407547266, 539.2160054696064]
lgb_scotrainre_list: [607.0736049372185, 623.4313863731124, 508.02381607269535, 660.4867407547266, 539.2160054696064]
lgb_score_mean: 587.6463107214719
lgb_score_std: 55.944536405714565

5 模型预测

temp=pd.DataFrame(lgb_test)
result=pd.read_csv('sample_submit.csv')
result['label_0']=temp[0]
result['label_1']=temp[1]
result['label_2']=temp[2]
result['label_3']=temp[3]
result.to_csv('./submit.csv',index=False)

6 提交结果

7 总结

这只是一个baseline,基本上没有对数据有其它的处理

这篇关于【天池】心跳信号分类预测 baseline Part1的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/859518

相关文章

基于人工智能的图像分类系统

目录 引言项目背景环境准备 硬件要求软件安装与配置系统设计 系统架构关键技术代码示例 数据预处理模型训练模型预测应用场景结论 1. 引言 图像分类是计算机视觉中的一个重要任务,目标是自动识别图像中的对象类别。通过卷积神经网络(CNN)等深度学习技术,我们可以构建高效的图像分类系统,广泛应用于自动驾驶、医疗影像诊断、监控分析等领域。本文将介绍如何构建一个基于人工智能的图像分类系统,包括环境

认识、理解、分类——acm之搜索

普通搜索方法有两种:1、广度优先搜索;2、深度优先搜索; 更多搜索方法: 3、双向广度优先搜索; 4、启发式搜索(包括A*算法等); 搜索通常会用到的知识点:状态压缩(位压缩,利用hash思想压缩)。

列举你能想到的UNIX信号,并说明信号用途

信号是一种软中断,是一种处理异步事件的方法。一般来说,操作系统都支持许多信号。尤其是UNIX,比较重要应用程序一般都会处理信号。 UNIX定义了许多信号,比如SIGINT表示中断字符信号,也就是Ctrl+C的信号,SIGBUS表示硬件故障的信号;SIGCHLD表示子进程状态改变信号;SIGKILL表示终止程序运行的信号,等等。信号量编程是UNIX下非常重要的一种技术。 Unix信号量也可以

用Pytho解决分类问题_DBSCAN聚类算法模板

一:DBSCAN聚类算法的介绍 DBSCAN(Density-Based Spatial Clustering of Applications with Noise)是一种基于密度的聚类算法,DBSCAN算法的核心思想是将具有足够高密度的区域划分为簇,并能够在具有噪声的空间数据库中发现任意形状的簇。 DBSCAN算法的主要特点包括: 1. 基于密度的聚类:DBSCAN算法通过识别被低密

PMP–一、二、三模–分类–14.敏捷–技巧–看板面板与燃尽图燃起图

文章目录 技巧一模14.敏捷--方法--看板(类似卡片)1、 [单选] 根据项目的特点,项目经理建议选择一种敏捷方法,该方法限制团队成员在任何给定时间执行的任务数。此方法还允许团队提高工作过程中问题和瓶颈的可见性。项目经理建议采用以下哪种方法? 易错14.敏捷--精益、敏捷、看板(类似卡片)--敏捷、精益和看板方法共同的重点在于交付价值、尊重人、减少浪费、透明化、适应变更以及持续改善等方面。

【python计算机视觉编程——8.图像内容分类】

python计算机视觉编程——8.图像内容分类 8.图像内容分类8.1 K邻近分类法(KNN)8.1.1 一个简单的二维示例8.1.2 用稠密SIFT作为图像特征8.1.3 图像分类:手势识别 8.2贝叶斯分类器用PCA降维 8.3 支持向量机8.3.2 再论手势识别 8.4 光学字符识别8.4.2 选取特征8.4.3 多类支持向量机8.4.4 提取单元格并识别字符8.4.5 图像校正

PMP–一、二、三模–分类–14.敏捷–技巧–原型MVP

文章目录 技巧一模14.敏捷--原型法--项目生命周期--迭代型生命周期,通过连续的原型或概念验证来改进产品或成果。每个新的原型都能带来新的干系人新的反馈和团队见解。题目中明确提到需要反馈,因此原型法比较好用。23、 [单选] 一个敏捷团队的任务是开发一款机器人。项目经理希望确保在机器人被实际建造之前,团队能够收到关于需求的早期反馈并相应地调整设计。项目经理应该使用以下哪一项来实现这个目标?

基于深度学习 卷积神经网络resnext50的中医舌苔分类系统

项目概述 本项目旨在通过深度学习技术,特别是利用卷积神经网络(Convolutional Neural Networks, CNNs)中的ResNeXt50架构,实现对中医舌象图像的自动分类。该系统不仅能够识别不同的舌苔类型,还能够在PyQt5框架下提供一个直观的图形用户界面(GUI),使得医生或患者能够方便地上传舌象照片并获取分析结果。 技术栈 深度学习框架:采用PyTorch或其他

Linux中如何屏蔽信号

本篇文章主要学习Linux的信号处理机制,着重学习屏蔽信号部分。屏蔽信号处理的两种方式类似于信号的捕获,一种方式是直接对其设置,另一种方式是先获得描述符的掩码,然后对其设置操作。 本文主要参考自《嵌入式linux系统使用开发》,作者何永琪,Thanks. 在linux系统中,如何处理某个进程发送的一个特定信号呢?一般来说有三种方式: 1) 忽略信号 2) 屏蔽信号 3) 为该信号添

电脑驱动分类

电脑驱动程序(驱动程序)是操作系统与硬件设备之间的桥梁,用于使操作系统能够识别并与硬件设备进行通信。以下是常见的驱动分类: 1. 设备驱动程序 显示驱动程序:控制显卡和显示器的显示功能,负责图形渲染和屏幕显示。 示例:NVIDIA、AMD 显示驱动程序。打印机驱动程序:允许操作系统与打印机通信,控制打印任务。 示例:HP、Canon 打印机驱动程序。声卡驱动程序:管理音频输入和输出,与声卡硬件