Households can provide demand-side flexibility by changing their consumption behaviour and shifting energy-intensive activities - such as vacuuming, cooking or charging electric cars - to off-peak hours. Such behaviour-based demand response management could balance out consumption peaks without the need for costly smart home devices or automation technologies. However, existing research has struggled to motivate consumers to adapt their behaviour or maintain behaviour change over extended periods of time. This field study explored a scalable and cost-effective behavioural demand response tool and investigated its short-and long-term impacts on average and peak hour electricity consumption under realistic conditions: A smartphone app provided users with social comparison feedback on their electricity consumption and invited them to participate in "pause hours" by avoiding energy-intensive activities during peak hours. To appeal even to hard-to-reach energy users and elicit longer-term engagement, the app also contained a neighbourhood newsfeed and was framed as a local social network. In a 15-month trial with 550 student apartments in Sweden, more than half of the residents voluntarily installed the app, many app users stayed engaged over an extended period of time and pause hour participants achieved substantial peak-load consumption reductions of 46% on average. These results indicate that smartphone apps can achieve peak load consumption reduction and long-term engagement, although they may be particularly attractive to already energy efficient households. Avenues for future research are discussed.