This work aims at simulating the time response of a structure damped by viscoelastic materials. The structure is discretised by finite elements and a 4-parameter fractional derivative model is used to describe the frequency-dependency of the mechanical properties of the viscoelastic material. The proposed approach combines a classical Newmark time-integration scheme to solve the semi-discretised equation of motion with a diffusive representation of fractional derivatives. This approach is applied to a finite element model, and validated on a single degree-of-freedom system for which an analytical solution can be derived.