scikit-learn linearRegression 1.1.10 逻辑回归

本文主要是介绍scikit-learn linearRegression 1.1.10 逻辑回归，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

逻辑回归形如其名，是一个线性分类模型而不是回归模型。逻辑回归在文献中也称为logit回归、最大熵分类(MaxEnt) 或者 log-linear classifier。在这个模型中，描述单次可能结果输出概率使用 logistic function 来建模。

scikit-learn中逻辑回归的实现为 LogisticRegression 类。它可以拟合含L2或者L1正则化项的多类逻辑回归问题。

作为一个优化问题，二分类L2 通过下方的代价函数来惩罚逻辑回归:

$\underset{w, c}{min\,} \frac{1}{2}w^T w + C \sum_{i=1}^n \log(\exp(- y_i (X_i^T w + c)) + 1) .$

类似的，L1 正则化逻辑回归解决下述的优化问题:

$\underset{w, c}{min\,} \|w\|_1 + C \sum_{i=1}^n \log(\exp(- y_i (X_i^T w + c)) + 1) .$

LogisticRegression 中的实现是solver “liblinear” (一个扩展的C++ library,LIBLINEAR), “newton-cg”, “lbfgs” and “sag”。

“lbfgs” 和 “newton-cg” 只支持L2罚项，并且对于一些高维数据收敛非常快。L1罚项产生稀疏预测的权重。

“liblinear” 使用了基于Liblinear的坐标下降法(CD)。对于F1罚项， sklearn.svm.l1_min_c 允许计算C的下界以获得一个非”null” 的模型（所有特征权重为0）。这依赖于非常棒的一个库 LIBLINEAR library ,用在scikit-learn中。然而，CD算法在liblinear中的实现无法学习一个真正的多维（多类）的模型。反而，最优问题被分解为 “one-vs-rest” 多个二分类问题来解决多分类。由于底层是这样实现的，所以使用了该库的 LogisticRegression 类就可以作为多类分类器了。

LogisticRegression 使用 “lbfgs” 或者 “newton-cg” 程序来设置 multi_class 为 “multinomial”，则该类学习了一个真正的多类逻辑回归模型，也就是说这种概率估计应该比默认 “one-vs-rest” 设置要更加准确。但是 “lbfgs”, “newton-cg” 和 “sag” 程序无法优化含L1罚项的模型，所以”multinomial” 的设置无法学习稀疏模型。

“sag” 程序使用了随机平均梯度下降（ Stochastic Average Gradient descent [3]）。它无法解决多分类问题，而且对于含L2罚项的模型有局限性。然而在超大数据集下计算要比其他程序快很多，当样本数量和特征数量都非常大的时候。

简单概括下，可以按照以下规则来选择solver:

Case	Solver
Small dataset or L1 penalty	“liblinear”
Multinomial loss	“lbfgs” or newton-cg”
Large dataset	“sag”

对于超大数据集，你同样可以考虑使用带log损失的 SGDClassifier

Examples:

L1 Penalty and Sparsity in Logistic Regression
Path with L1- Logistic Regression

Differences from liblinear:

There might be a difference in the scores obtained between LogisticRegression with solver=liblinear or LinearSVC and the external liblinear library directly, when fit_intercept=False and the fit coef_ (or) the data to be predicted are zeroes. This is because for the sample(s) with decision_functionzero, LogisticRegression and LinearSVC predict the negative class, while liblinear predicts the positive class. Note that a model with fit_intercept=False and having many samples with decision_function zero, is likely to be a underfit, bad model and you are advised to set fit_intercept=True and increase the intercept_scaling.

Note

Feature selection with sparse logistic regression

A logistic regression with L1 penalty yields sparse models, and can thus be used to perform feature selection, as detailed in 基于L1的特征选择(L1-based feature selection).

LogisticRegressionCV 实现了一个内建的交叉验证来寻找最优的参数C的逻辑回归模型。”newton-cg”,”sag” 和　”lbfgs” 程序在高维稠密数据上计算更快,原因在于warm-starting.对于多类问题,如果 multi_class 选项设置为 “ovr” ,那么最优的C从每个类别中获得，如果 multi_class 选项设置为　”multinomial” ,那么最优的Ｃ通过最小化交叉熵损失得到。

Examples:

logistic回归中的L1罚和稀疏性

对不同的C值使用L1和L2惩罚时的稀疏性（零系数百分比）的比较，我们可以看出C的大值给模型带来了更多的自由度。相反，较小的C值限制了模型的更多。在L1处罚的情况下，这会导致稀疏的解决方案。
我们把数字8x8图像可分为两类：0-4对5-9。可视化显示了不同C模型的系数。

../../_images/plot_logistic_l1_l2_sparsity_001.png

Script output:

 C=100.00
Sparsity with L1 penalty: 6.25%
score with L1 penalty: 0.9104
Sparsity with L2 penalty: 4.69%
score with L2 penalty: 0.9098
C=1.00
Sparsity with L1 penalty: 10.94%
score with L1 penalty: 0.9098
Sparsity with L2 penalty: 4.69%
score with L2 penalty: 0.9093
C=0.01
Sparsity with L1 penalty: 85.94%
score with L1 penalty: 0.8614
Sparsity with L2 penalty: 4.69%
score with L2 penalty: 0.8915
 

Python source code: plot_logistic_l1_l2_sparsity.py

 print(__doc__)# Authors: Alexandre Gramfort <alexandre.gramfort@inria.fr>
#          Mathieu Blondel <mathieu@mblondel.org>
#          Andreas Mueller <amueller@ais.uni-bonn.de>
# License: BSD 3 clauseimport numpy as np
import matplotlib.pyplot as pltfrom sklearn.linear_model import LogisticRegression
from sklearn import datasets
from sklearn.preprocessing import StandardScalerdigits = datasets.load_digits()X, y = digits.data, digits.target
X = StandardScaler().fit_transform(X)# classify small against large digits
y = (y > 4).astype(np.int)# Set regularization parameter
for i, C in enumerate((100, 1, 0.01)):# turn down tolerance for short training timeclf_l1_LR = LogisticRegression(C=C, penalty='l1', tol=0.01)clf_l2_LR = LogisticRegression(C=C, penalty='l2', tol=0.01)clf_l1_LR.fit(X, y)clf_l2_LR.fit(X, y)coef_l1_LR = clf_l1_LR.coef_.ravel()coef_l2_LR = clf_l2_LR.coef_.ravel()# coef_l1_LR contains zeros due to the# L1 sparsity inducing normsparsity_l1_LR = np.mean(coef_l1_LR == 0) * 100sparsity_l2_LR = np.mean(coef_l2_LR == 0) * 100print("C=%.2f" % C)print("Sparsity with L1 penalty: %.2f%%" % sparsity_l1_LR)print("score with L1 penalty: %.4f" % clf_l1_LR.score(X, y))print("Sparsity with L2 penalty: %.2f%%" % sparsity_l2_LR)print("score with L2 penalty: %.4f" % clf_l2_LR.score(X, y))l1_plot = plt.subplot(3, 2, 2 * i + 1)l2_plot = plt.subplot(3, 2, 2 * (i + 1))if i == 0:l1_plot.set_title("L1 penalty")l2_plot.set_title("L2 penalty")l1_plot.imshow(np.abs(coef_l1_LR.reshape(8, 8)), interpolation='nearest',cmap='binary', vmax=1, vmin=0)l2_plot.imshow(np.abs(coef_l2_LR.reshape(8, 8)), interpolation='nearest',cmap='binary', vmax=1, vmin=0)plt.text(-8, 3, "C = %.2f" % C)l1_plot.set_xticks(())l1_plot.set_yticks(())l2_plot.set_xticks(())l2_plot.set_yticks(())plt.show() 

L1 logistic回归路径

Computes path on IRIS dataset.

../../_images/plot_logistic_path_001.png

Script output:

 Computing regularization path ...
This took  0:00:00.147946
 

Python source code: plot_logistic_path.py

 print(__doc__)# Author: Alexandre Gramfort <alexandre.gramfort@inria.fr>
# License: BSD 3 clausefrom datetime import datetime
import numpy as np
import matplotlib.pyplot as pltfrom sklearn import linear_model
from sklearn import datasets
from sklearn.svm import l1_min_ciris = datasets.load_iris()
X = iris.data
y = iris.targetX = X[y != 2]
y = y[y != 2]X -= np.mean(X, 0)###############################################################################
# Demo path functionscs = l1_min_c(X, y, loss='log') * np.logspace(0, 3)print("Computing regularization path ...")
start = datetime.now()
clf = linear_model.LogisticRegression(C=1.0, penalty='l1', tol=1e-6)
coefs_ = []
for c in cs:clf.set_params(C=c)clf.fit(X, y)coefs_.append(clf.coef_.ravel().copy())
print("This took ", datetime.now() - start)coefs_ = np.array(coefs_)
plt.plot(np.log10(cs), coefs_)
ymin, ymax = plt.ylim()
plt.xlabel('log(C)')
plt.ylabel('Coefficients')
plt.title('Logistic Regression Path')
plt.axis('tight')
plt.show() 

这篇关于scikit-learn linearRegression 1.1.10 逻辑回归的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！

scikit-learn linearRegression 1.1.10 逻辑回归

相关文章

usaco 1.1 Broken Necklace（DP）

【WebGPU Unleashed】1.1 绘制三角形

✨机器学习笔记（二）—— 线性回归、代价函数、梯度下降

逻辑表达式，最小项

UMI复现代码运行逻辑全流程（一）——eval_real.py（尚在更新）

用Python实现时间序列模型实战——Day 14: 向量自回归模型 (VAR) 与向量误差修正模型 (VECM)

Learn ComputeShader 09 Night version lenses

深度学习与大模型第3课：线性回归模型的构建与训练

【python因果推断库11】工具变量回归与使用 pymc 验证工具变量4

什么是GPT-3的自回归架构？为什么GPT-3无需梯度更新和微调