python 相关性分析切点寻找,Python自然平滑样条线

2023-10-24 15:30

本文主要是介绍python 相关性分析切点寻找,Python自然平滑样条线,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

I am trying to find a python package that would give an option to fit natural smoothing splines with user selectable smoothing factor. Is there an implementation for that? If not, how would you use what is available to implement it yourself?

By natural spline I mean that there should be a condition that the second derivative of the fitted function at the endpoints is zero (linear).

By smoothing spline I mean that the spline should not be 'interpolating' (passing through all the datapoints). I would like to decide the correct smoothing factor lambda (see the Wikipedia page for smoothing splines) myself.

What I have found

scipy.interpolate.CubicSpline [link]: Does natural (cubic) spline fitting. Does interpolation, and there is no way to smooth the data.

scipy.interpolate.UnivariateSpline [link]: Does spline fitting with user selectable smoothing factor. However, there is no option to make the splines natural.

解决方案

After hours of investigation, I did not find any pip installable packages which could fit a natural cubic spline with user-controllable smoothness. However, after deciding to write one myself, while reading about the topic I stumbled upon a blog post by github user madrury. He has written python code capable of producing natural cubic spline models.

The model code is available here (NaturalCubicSpline) with a BSD-licence. He has also written some examples in an IPython notebook.

But since this is the Internet and links tend to die, I will copy the relevant parts of the source code here + a helper function (get_natural_cubic_spline_model) written by me, and show an example of how to use it. The smoothness of the fit can be controlled by using different number of knots. The position of the knots can be also specified by the user.

Example

from matplotlib import pyplot as plt

import numpy as np

def func(x):

return 1/(1+25*x**2)

# make example data

x = np.linspace(-1,1,300)

y = func(x) + np.random.normal(0, 0.2, len(x))

# The number of knots can be used to control the amount of smoothness

model_6 = get_natural_cubic_spline_model(x, y, minval=min(x), maxval=max(x), n_knots=6)

model_15 = get_natural_cubic_spline_model(x, y, minval=min(x), maxval=max(x), n_knots=15)

y_est_6 = model_6.predict(x)

y_est_15 = model_15.predict(x)

plt.plot(x, y, ls='', marker='.', label='originals')

plt.plot(x, y_est_6, marker='.', label='n_knots = 6')

plt.plot(x, y_est_15, marker='.', label='n_knots = 15')

plt.legend(); plt.show()

3f3a5b82f015ba8333b227668f2b60af.png

The source code for get_natural_cubic_spline_model

import numpy as np

import pandas as pd

from sklearn.base import BaseEstimator, TransformerMixin

from sklearn.linear_model import LinearRegression

from sklearn.pipeline import Pipeline

def get_natural_cubic_spline_model(x, y, minval=None, maxval=None, n_knots=None, knots=None):

"""

Get a natural cubic spline model for the data.

For the knots, give (a) `knots` (as an array) or (b) minval, maxval and n_knots.

If the knots are not directly specified, the resulting knots are equally

space within the *interior* of (max, min). That is, the endpoints are

*not* included as knots.

Parameters

----------

x: np.array of float

The input data

y: np.array of float

The outpur data

minval: float

Minimum of interval containing the knots.

maxval: float

Maximum of the interval containing the knots.

n_knots: positive integer

The number of knots to create.

knots: array or list of floats

The knots.

Returns

--------

model: a model object

The returned model will have following method:

- predict(x):

x is a numpy array. This will return the predicted y-values.

"""

if knots:

spline = NaturalCubicSpline(knots=knots)

else:

spline = NaturalCubicSpline(max=maxval, min=minval, n_knots=n_knots)

p = Pipeline([

('nat_cubic', spline),

('regression', LinearRegression(fit_intercept=True))

])

p.fit(x, y)

return p

class AbstractSpline(BaseEstimator, TransformerMixin):

"""Base class for all spline basis expansions."""

def __init__(self, max=None, min=None, n_knots=None, n_params=None, knots=None):

if knots is None:

if not n_knots:

n_knots = self._compute_n_knots(n_params)

knots = np.linspace(min, max, num=(n_knots + 2))[1:-1]

max, min = np.max(knots), np.min(knots)

self.knots = np.asarray(knots)

@property

def n_knots(self):

return len(self.knots)

def fit(self, *args, **kwargs):

return self

class NaturalCubicSpline(AbstractSpline):

"""Apply a natural cubic basis expansion to an array.

The features created with this basis expansion can be used to fit a

piecewise cubic function under the constraint that the fitted curve is

linear *outside* the range of the knots.. The fitted curve is continuously

differentiable to the second order at all of the knots.

This transformer can be created in two ways:

- By specifying the maximum, minimum, and number of knots.

- By specifying the cutpoints directly.

If the knots are not directly specified, the resulting knots are equally

space within the *interior* of (max, min). That is, the endpoints are

*not* included as knots.

Parameters

----------

min: float

Minimum of interval containing the knots.

max: float

Maximum of the interval containing the knots.

n_knots: positive integer

The number of knots to create.

knots: array or list of floats

The knots.

"""

def _compute_n_knots(self, n_params):

return n_params

@property

def n_params(self):

return self.n_knots - 1

def transform(self, X, **transform_params):

X_spl = self._transform_array(X)

if isinstance(X, pd.Series):

col_names = self._make_names(X)

X_spl = pd.DataFrame(X_spl, columns=col_names, index=X.index)

return X_spl

def _make_names(self, X):

first_name = "{}_spline_linear".format(X.name)

rest_names = ["{}_spline_{}".format(X.name, idx)

for idx in range(self.n_knots - 2)]

return [first_name] + rest_names

def _transform_array(self, X, **transform_params):

X = X.squeeze()

try:

X_spl = np.zeros((X.shape[0], self.n_knots - 1))

except IndexError: # For arrays with only one element

X_spl = np.zeros((1, self.n_knots - 1))

X_spl[:, 0] = X.squeeze()

def d(knot_idx, x):

def ppart(t): return np.maximum(0, t)

def cube(t): return t*t*t

numerator = (cube(ppart(x - self.knots[knot_idx]))

- cube(ppart(x - self.knots[self.n_knots - 1])))

denominator = self.knots[self.n_knots - 1] - self.knots[knot_idx]

return numerator / denominator

for i in range(0, self.n_knots - 2):

X_spl[:, i+1] = (d(i, X) - d(self.n_knots - 2, X)).squeeze()

return X_spl

这篇关于python 相关性分析切点寻找,Python自然平滑样条线的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/276122

相关文章

Python判断for循环最后一次的6种方法

《Python判断for循环最后一次的6种方法》在Python中,通常我们不会直接判断for循环是否正在执行最后一次迭代,因为Python的for循环是基于可迭代对象的,它不知道也不关心迭代的内部状态... 目录1.使用enuhttp://www.chinasem.cnmerate()和len()来判断for

使用Python实现高效的端口扫描器

《使用Python实现高效的端口扫描器》在网络安全领域,端口扫描是一项基本而重要的技能,通过端口扫描,可以发现目标主机上开放的服务和端口,这对于安全评估、渗透测试等有着不可忽视的作用,本文将介绍如何使... 目录1. 端口扫描的基本原理2. 使用python实现端口扫描2.1 安装必要的库2.2 编写端口扫

使用Python实现操作mongodb详解

《使用Python实现操作mongodb详解》这篇文章主要为大家详细介绍了使用Python实现操作mongodb的相关知识,文中的示例代码讲解详细,感兴趣的小伙伴可以跟随小编一起学习一下... 目录一、示例二、常用指令三、遇到的问题一、示例from pymongo import MongoClientf

使用Python合并 Excel单元格指定行列或单元格范围

《使用Python合并Excel单元格指定行列或单元格范围》合并Excel单元格是Excel数据处理和表格设计中的一项常用操作,本文将介绍如何通过Python合并Excel中的指定行列或单... 目录python Excel库安装Python合并Excel 中的指定行Python合并Excel 中的指定列P

一文详解Python中数据清洗与处理的常用方法

《一文详解Python中数据清洗与处理的常用方法》在数据处理与分析过程中,缺失值、重复值、异常值等问题是常见的挑战,本文总结了多种数据清洗与处理方法,文中的示例代码简洁易懂,有需要的小伙伴可以参考下... 目录缺失值处理重复值处理异常值处理数据类型转换文本清洗数据分组统计数据分箱数据标准化在数据处理与分析过

Python调用另一个py文件并传递参数常见的方法及其应用场景

《Python调用另一个py文件并传递参数常见的方法及其应用场景》:本文主要介绍在Python中调用另一个py文件并传递参数的几种常见方法,包括使用import语句、exec函数、subproce... 目录前言1. 使用import语句1.1 基本用法1.2 导入特定函数1.3 处理文件路径2. 使用ex

Python脚本实现自动删除C盘临时文件夹

《Python脚本实现自动删除C盘临时文件夹》在日常使用电脑的过程中,临时文件夹往往会积累大量的无用数据,占用宝贵的磁盘空间,下面我们就来看看Python如何通过脚本实现自动删除C盘临时文件夹吧... 目录一、准备工作二、python脚本编写三、脚本解析四、运行脚本五、案例演示六、注意事项七、总结在日常使用

Python将大量遥感数据的值缩放指定倍数的方法(推荐)

《Python将大量遥感数据的值缩放指定倍数的方法(推荐)》本文介绍基于Python中的gdal模块,批量读取大量多波段遥感影像文件,分别对各波段数据加以数值处理,并将所得处理后数据保存为新的遥感影像... 本文介绍基于python中的gdal模块,批量读取大量多波段遥感影像文件,分别对各波段数据加以数值处

python管理工具之conda安装部署及使用详解

《python管理工具之conda安装部署及使用详解》这篇文章详细介绍了如何安装和使用conda来管理Python环境,它涵盖了从安装部署、镜像源配置到具体的conda使用方法,包括创建、激活、安装包... 目录pytpshheraerUhon管理工具:conda部署+使用一、安装部署1、 下载2、 安装3

Python进阶之Excel基本操作介绍

《Python进阶之Excel基本操作介绍》在现实中,很多工作都需要与数据打交道,Excel作为常用的数据处理工具,一直备受人们的青睐,本文主要为大家介绍了一些Python中Excel的基本操作,希望... 目录概述写入使用 xlwt使用 XlsxWriter读取修改概述在现实中,很多工作都需要与数据打交