Thepre-dominantapproachtolanguagemodel-ingtodateisbasedonrecurrentneuralnetworks.Inthispaperwepresentaconvolutionalapproachtolanguagemodeling.Weintroduceanovelgatingmechanismthateasesgradientpropaga-tionandwhichperformsbetterthantheLSTM-stylegatingofOordetal.(2016b)despitebeingsim