本文主要是介绍[deeplearning-015]一文理解rnn，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！
1.参考文档 https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/
前提，熟悉bp算法推导，熟悉bptt算法推导
2.py源码
#!/usr/bin/env python3#参考文献 https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/import copy, numpy as npnp.random.seed(0)# compute sigmoid nonlinearity
def sigmoid(x):output = 1 / (1 + np.exp(-x))return output# convert output of sigmoid function to its derivative
def sigmoid_output_to_derivative(output):return output * (1 - output)# training dataset generation
int2binary = {}#空字典
binary_dim = 8#=256
largest_number = pow(2, binary_dim)binary = np.unpackbits(np.array([range(largest_number)], dtype=np.uint8).T, axis = 1)
for i in range(largest_number):int2binary[i] = binary[i]# input variables
alpha = 0.1
input_dim = 2
hidden_dim = 16
output_dim = 1# initialize neural network weights
synapse_0 = 2 * np.random.random((input_dim, hidden_dim)) - 1 #把值调整到(-1,1)
synapse_1 = 2 * np.random.random((hidden_dim, output_dim)) - 1
synapse_h = 2 * np.random.random((hidden_dim, hidden_dim)) - 1
synapse_0_update = np.zeros_like(synapse_0)
synapse_1_update = np.zeros_like(synapse_1)
synapse_h_update = np.zeros_like(synapse_h)# 随机生成ab两个整数，c=a+b。将abc转成二进制，a和b的二进制是rnn的输入，c的二进制是目标值，训练rnn。# training logic
for j in range(50000):# generate a simple addition problem (a + b = c)a_int = np.random.randint(largest_number / 2) # int versiona = int2binary[a_int] # binary encodingb_int = np.random.randint(largest_number / 2) # int versionb = int2binary[b_int] # binary encoding# true answerc_int = a_int + b_intc = int2binary[c_int]# print(a_int, a, b_int, b,c_int,c)# exit(1)# where we'll store our best guess (binary encoded)d = np.zeros_like(c)overallError = 0#记录第二层的deta值layer_2_deltas = list()#记录第一层的输出值layer_1_values = list()layer_1_values.append(np.zeros(hidden_dim))# moving along the positions in the binary encodingfor position in range(binary_dim):# generate input and output# 从abc三个数的二进制的最低位开始，每次取一位，生成训练样本# print("\nposition=", position)# print("a=",a)# print("b=",b)# print("c=", c)#比如，取第一位的时候，X=[[1,0]],y=[[1]]，是训练样本和目标值X = np.array([[a[binary_dim - position - 1], b[binary_dim - position - 1]]])y = np.array([[c[binary_dim - position - 1]]]).T# print('X=',X)# print('y=',y)# hidden layer (input ~+ prev_hidden)# 计算第一层的输出值：sigmoid里面有两个部分，分别是input*synapse_0和前次第一层输出值*synapse_h隐层layer_1 = sigmoid(np.dot(X, synapse_0) + np.dot(layer_1_values[-1], synapse_h))# output layer (new binary representation)#第二层输出就很直接了layer_2 = sigmoid(np.dot(layer_1, synapse_1))# did we miss?... if so, by how much?# 计算输出误差layer_2_error = y - layer_2#根据误差，计算第二层权重系数的deta，这个公式是对第二层微分求导得到的，熟悉BP算法的都清楚，不在叙述layer_2_deltas.append((layer_2_error) * sigmoid_output_to_derivative(layer_2))#计算绝对误差之和，以衡量8个位的总误差大小overallError += np.abs(layer_2_error[0])# decode estimate so we can print it out# 记录下 rnn对当前位的预测值d[binary_dim - position - 1] = np.round(layer_2[0][0])# store hidden layer so we can use it in the next timestep# 把本轮第一层的输出记录到layer_1_values，以备bptt使用。layer_1_values.append(copy.deepcopy(layer_1))#计算第一个隐层的deltafuture_layer_1_delta = np.zeros(hidden_dim)for position in range(binary_dim):#注意喽，X这一轮循环是从ab的二进制的高位开始往低位走的X = np.array([[a[position], b[position]]])#高位二进制是后计算的，因此其值在layer_1的尾部layer_1 = layer_1_values[-position - 1]prev_layer_1 = layer_1_values[-position - 2]# error at output layerlayer_2_delta = layer_2_deltas[-position - 1]# error at hidden layerlayer_1_delta = (future_layer_1_delta.dot(synapse_h.T) + layer_2_delta.dot(synapse_1.T)) * sigmoid_output_to_derivative(layer_1)# let's update all our weights so we can try againsynapse_1_update += np.atleast_2d(layer_1).T.dot(layer_2_delta)synapse_h_update += np.atleast_2d(prev_layer_1).T.dot(layer_1_delta)synapse_0_update += X.T.dot(layer_1_delta)future_layer_1_delta = layer_1_deltasynapse_0 += synapse_0_update * alphasynapse_1 += synapse_1_update * alphasynapse_h += synapse_h_update * alphasynapse_0_update *= 0synapse_1_update *= 0synapse_h_update *= 0# print out progressif (j % 1000 == 0):print("Error:" + str(overallError))print("Pred:" + str(d))print("True:" + str(c))out = 0for index, x in enumerate(reversed(d)):out += x * pow(2, index)print(str(a_int) + " + " + str(b_int) + " = " + str(out))print("------------")
这篇关于[deeplearning-015]一文理解rnn的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！
