theano中对adadelta的实现

本文主要是介绍theano中对adadelta的实现，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

最开始不太会写，后来发现自己给写错了。先给出adadelta的伪代码

egs(t+1) = rho*egs(t) + (1-rho)*grad(t+1)**2

delta(t+1) =sqrt(exs(t)**2 + epsilon)/sqrt(egs(t+1)**2 + epsilon)*(-grad(t+1))

exs(t+1) = rho*esx(t) + (1-rho)*delta(t+1)**2

w(t+1)=w(t)+delta(t+1)

下面给出theano的代码，假定参数全部都是params中，导数是param_grads

  rho = 0.95epsilon = 1e-6egs = [(theano.shared(numpy.asarray(numpy.zeros(param.get_value().shape), theano.config.floatX))) for param in params]exs = [(theano.shared(numpy.asarray(numpy.zeros(param.get_value().shape), theano.config.floatX))) for param in params]new_egs = [(rho*eg + (1-rho)*g**2) for eg, g in zip(egs, param_grads)]delta_x = [(T.sqrt(ex+epsilon)/T.sqrt(new_eg+epsilon)*(-g)) for ex, new_eg, g in zip(exs, new_egs, param_grads)]new_exs = [(rho*ex + (1-rho)*dx**2) for ex, dx in zip(exs, delta_x) ]new_params = [(p + dx) for p, dx in zip(mlp.params, updates = zip(egs, new_egs) +  zip(exs, new_exs) + zip(params, new_params)

这篇关于theano中对adadelta的实现的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！

---