本文主要是介绍梯度下降算法实现,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!
分治方法:先分成n×n个点进行搜索,选择最低的点,对最低的点再分成n×n份再次进行搜索,选择最小的值,反复如此,找到最小值。但是这对于凸函数是较为有用的,对于不规则的函数,可能搜索到的是极小值点(局部最优点),而不是最小值点(全局最优点)。
Gradient Descent Algorithm : 梯度下降算法(贪心思想,局部最优)
Gradient : 梯度,梯度大于0上升,梯度小于0下降,所以参数向梯度的反方向更新。
w = w − x g ′ ( w ) w=w-xg'(w)w=w−xg
′
(w)
x:学习率
损失函数的局部最优点比较少,但是我们有可能遇到鞍点,鞍点的导数等于0(梯度等于0, g ′ ( w ) = 0 g'(w)=0g
′
(w)=0),这时候参数无法更新。
对损失函数求导,求参数更新公式:
import numpy as np
import matplotlib.pyplot as plt
x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]w = 1.0 # y = x * w, w的初始值def forward(x):return x * wdef cost(xs, xy):cost = 0for x, y in zip(xs, xy):y_pred = forward(x)cost += (y_pred - y) ** 2return cost / len(xs)def gradient(xs, ys):grad = 0for x, y in zip(xs, ys):grad += 2 * x * (x * w - y)return grad / len(xs)def gradient(xs, ys):grad = 0for x, y in zip(xs, ys):grad += 2 * x * (x * w - y)return grad / len(xs)loss_list = []
trainnum_list = []
print("Predict (before training)", 4, forward(4)) # 训练前x = 4,对应预测的y值
for epoch in range(100):cost_val = cost(x_data, y_data)grad_val = gradient(x_data, y_data)w -= 0.01 * grad_valprint("Epoch: ", epoch, "w = ", w, "loss = ", cost_val)loss_list.append(cost_val)trainnum_list.append(epoch)
print("Predict(after training)", 4, forward(4)) # 训练后x = 4,对应预测的y值
画出loss的变化趋势:
plt.plot(trainnum_list, loss_list)
plt.ylabel("Cost")
plt.xlabel("Epoch")
plt.show()
import numpy as np
import matplotlib.pyplot as plt
x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]w = 1.0 # y = x * w, w的初始值def forward(x):return x * wdef loss(x, y):y_pred = forward(x)return (y_pred - y) ** 2def gradient(x, y):return 2 * x * (x * w - y)loss_list = []
trainnum_list = []
print("Predict (before training)", 4, forward(4)) # 训练前x = 4,对应预测的y值
for epoch in range(100):for x, y in zip(x_data, y_data):grad = gradient(x, y)w -= 0.01 * gradprint("\tgrad", x, y, grad)l = loss(x, y)print("Epoch: ", epoch, "w = ", w, "loss = ", l)loss_list.append(l)trainnum_list.append(epoch)
print("Predict(after training)", 4, forward(4)) # 训练后x = 4,对应预测的y值plt.plot(trainnum_list, loss_list)
plt.ylabel("Loss")
plt.xlabel("Epoch")
plt.show()
这篇关于梯度下降算法实现的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!