Direct numerical simulations are carried out to study the effect of finite Weissenberg number up to Wi = 16 on laminar and turbulent channel flows of an elastoviscoplastic (EVP) fluid, at a fixed bulk Reynolds number of 2800. The incompressible flow equations are coupled with the evolution equation for the EVP stress tensor by a modified Saramito model that extends both the Bingham viscoplastic and the finite extensible nonlinear elastic-Peterlin (FENE-P) viscoelastic models. In turbulent flow, we find that drag decreases with both the Bingham and Weissenberg numbers, until the flow laminarises at high enough elastic and yield stresses. Hence, a higher drag reduction is achieved than in the viscoelastic flow at the same Weissenberg number. The drag reduction persists at Bingham numbers up to 20, in contrast to viscoplastic flow, where the drag increases in the laminar regime compared with a Newtonian flow. Moreover, elasticity affects the laminarisation of an EVP flow in a non-monotonic fashion, delaying it at lower and promoting it at higher Weissenberg numbers. A hibernation phenomenon is observed in the EVP flow, leading to large changes in the unyielded regions. Finally, plasticity is observed to affect both low- and high-speed streaks equally, attenuating the turbulent dissipation and the fragmentation of turbulent structures.