本文主要是介绍Keras(十七)关于feature_column的使用、keras模型转tf.estimator，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！
本文将介绍：
- 加载Titanic数据集
- 使用feature_column做数据处理,并转化为tf.data.dataset类型数据
- keras_to_estimator
一，加载Titanic数据集
1，下载Titanic数据集,使用pandas读取并解析数据集
# 在如下的两个网址下载数据
# https://storage.googleapis.com/tf-datasets/titanic/train.csv
# https://storage.googleapis.com/tf-datasets/titanic/eval.csvimport matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
import sklearn
import pandas as pd
import os
import sys
import time
import tensorflow as tf
from tensorflow import keras# 打印使用的python库的版本信息
print(tf.__version__)
print(sys.version_info)
for module in mpl, np, pd, sklearn, tf, keras:print(module.__name__, module.__version__)train_file = "./data/titanic/train.csv"
eval_file = "./data/titanic/eval.csv"train_df = pd.read_csv(train_file)
eval_df = pd.read_csv(eval_file)print(train_df.head()) # 默认取出前5条数据
print(eval_df.head())
2，分离出特征值和目标值
y_train = train_df.pop('survived')
y_eval = eval_df.pop('survived')print(train_df.head())
print(eval_df.head())
print(y_train.head())
print(y_eval.head())
3，使用panda对数值型数据的字段进行统计
print(train_df.describe())# ---output------age n_siblings_spouses parch fare
count 627.000000 627.000000 627.000000 627.000000
mean 29.631308 0.545455 0.379585 34.385399
std 12.511818 1.151090 0.792999 54.597730
min 0.750000 0.000000 0.000000 0.000000
25% 23.000000 0.000000 0.000000 7.895800
50% 28.000000 0.000000 0.000000 15.045800
75% 35.000000 1.000000 0.000000 31.387500
max 80.000000 8.000000 5.000000 512.329200
4,查看数据集中的测试集,验证集的数据维度
print(train_df.shape, eval_df.shape)# ---output------
(627, 9) (264, 9)
5,使用pands中的matplotlib绘制图表,更直观的了解数据
1）统计-年龄直观图
train_df.age.hist(bins = 50)# bins是将所有数据分为多少份
2）统计-性别直观图
# value_counts() --> 将value归类并按类计数
train_df.sex.value_counts().plot(kind = 'barh') # 横向的柱状图是"barh";纵向的柱状图"bar"
3）统计-不同仓位的乘客各有多少
train_df['class'].value_counts().plot(kind = 'barh')
4）统计-在Titanic中,男性有多少人获救了,女性有多少人获救了
pd.concat([train_df, y_train], axis = 1).groupby('sex').survived.mean()
pd.concat([train_df, y_train], axis = 1).groupby('sex').survived.mean().plot(kind='barh')
二，使用feature_column做数据处理,并转化为tf.data.dataset类型数据
1，将"离散特征"和"连续特征"整合为one-hot编码
1）将特征分为"离散特征"和"连续特征"两个列表
categorical_columns = ['sex', 'n_siblings_spouses', 'parch', 'class','deck', 'embark_town', 'alone']
numeric_columns = ['age', 'fare']feature_columns = []
2）使用tf.feature_column对"离散特征"做处理
for categorical_column in categorical_columns:vocab = train_df[categorical_column].unique()print(categorical_column, vocab)feature_columns.append(tf.feature_column.indicator_column(tf.feature_column.categorical_column_with_vocabulary_list(categorical_column, vocab)))# ---output------
sex ['male' 'female']
n_siblings_spouses [1 0 3 4 2 5 8]
parch [0 1 2 5 3 4]
class ['Third' 'First' 'Second']
deck ['unknown' 'C' 'G' 'A' 'B' 'D' 'F' 'E']
embark_town ['Southampton' 'Cherbourg' 'Queenstown' 'unknown']
alone ['n' 'y']
3）使用tf.feature_column对"连续特征"做处理
for categorical_column in numeric_columns:feature_columns.append(tf.feature_column.numeric_column(categorical_column, dtype=tf.float32))
2，将ndarray数据转化为tf.data.dataset中的BatchDataset类型数据
def make_dataset(data_df, label_df, epochs = 10, shuffle = True,batch_size = 32):dataset = tf.data.Dataset.from_tensor_slices((dict(data_df), label_df))if shuffle:dataset = dataset.shuffle(10000)dataset = dataset.repeat(epochs).batch(batch_size)return datasettrain_dataset = make_dataset(train_df, y_train, batch_size = 5)# 查看转化后的tf.data.dataset中的一条数据的信息
for x, y in train_dataset.take(1):print(x, y)# ---output---------
{'sex': <tf.Tensor: shape=(5,), dtype=string, numpy=array([b'female', b'male', b'male', b'male', b'male'], dtype=object)>, 'age': <tf.Tensor: shape=(5,), dtype=float64, numpy=array([32., 28., 44., 28., 28.])>, 'n_siblings_spouses': <tf.Tensor: shape=(5,), dtype=int32, numpy=array([1, 0, 1, 0, 0], dtype=int32)>, 'parch': <tf.Tensor: shape=(5,), dtype=int32, numpy=array([1, 0, 0, 0, 0], dtype=int32)>, 'fare': <tf.Tensor: shape=(5,), dtype=float64, numpy=array([15.5 , 7.2292, 26. , 8.05 , 7.8958])>, 'class': <tf.Tensor: shape=(5,), dtype=string, numpy=array([b'Third', b'Third', b'Second', b'Third', b'Third'], dtype=object)>, 'deck': <tf.Tensor: shape=(5,), dtype=string, numpy=
array([b'unknown', b'unknown', b'unknown', b'unknown', b'unknown'],dtype=object)>, 'embark_town': <tf.Tensor: shape=(5,), dtype=string, numpy=
array([b'Queenstown', b'Cherbourg', b'Southampton', b'Southampton',b'Southampton'], dtype=object)>, 'alone': <tf.Tensor: shape=(5,), dtype=string, numpy=array([b'n', b'y', b'n', b'y', b'y'], dtype=object)>} tf.Tensor([0 1 0 0 0], shape=(5,), dtype=int32)
3，使用keras.layers.DenseFeature将一条数据其中两个字段转化为one-hot处理后的数据
# keras.layers.DenseFeature
for x, y in train_dataset.take(1):age_column = feature_columns[7]gender_column = feature_columns[0]print(keras.layers.DenseFeatures(age_column)(x).numpy())print(keras.layers.DenseFeatures(gender_column)(x).numpy())# ---output----------
[[28.][50.][27.][28.][32.]][[1. 0.][0. 1.][0. 1.][0. 1.][1. 0.]]
4，使用keras.layers.DenseFeature将一条数据中所有字段转化为one-hot处理后的数据
# keras.layers.DenseFeature
for x, y in train_dataset.take(1):print(keras.layers.DenseFeatures(feature_columns)(x).numpy())
三，keras_to_estimator
1，定义keras模型,输入层输入为转化为one-hot处理后的数据
model = keras.models.Sequential([keras.layers.DenseFeatures(feature_columns),keras.layers.Dense(100, activation='relu'),keras.layers.Dense(100, activation='relu'),keras.layers.Dense(2, activation='softmax'),
])
model.compile(loss='sparse_categorical_crossentropy',optimizer = keras.optimizers.SGD(lr=0.01),metrics = ['accuracy'])
2，训练模型
训练模型可以使用如下两种方法：
1）使用普通的model模型训练
train_dataset = make_dataset(train_df, y_train, epochs = 100)
eval_dataset = make_dataset(eval_df, y_eval, epochs = 1, shuffle = False)
history = model.fit(train_dataset,validation_data = eval_dataset,steps_per_epoch = 19,validation_steps = 8,epochs = 100)
2）使用转化为estimator后的model模型训练
注：在tensorflow2中该方法还存在bug，待解决。
estimator = keras.estimator.model_to_estimator(model)
# 1. function
# 2. return a. (features, labels) b. dataset -> (feature, label)
estimator.train(input_fn = lambda : make_dataset(train_df, y_train, epochs=100))
这篇关于Keras(十七)关于feature_column的使用、keras模型转tf.estimator的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！
