A search for non-resonant Higgs boson pair (HH) production is presented, in which one of the Higgs bosons decays to a b-quark pair (bb¯) and the other decays to WW*, ZZ*, or τ+τ−, with in each case a final state with ℓ+ℓ−+ neutrinos (ℓ = e, μ). The analysis targets separately the gluon-gluon fusion and vector boson fusion production modes. Data recorded by the ATLAS detector in proton-proton collisions at a centre-of-mass energy of 13 TeV at the Large Hadron Collider, corresponding to an integrated luminosity of 140 fb−1, are used in this analysis. Events are selected to have exactly two b-tagged jets and two leptons with opposite electric charge and missing transverse momentum in the final state. These events are classified using multivariate analysis algorithms to separate the HH events from other Standard Model processes. No evidence of the signal is found. The observed (expected) upper limit on the cross-section for non-resonant Higgs boson pair production is determined to be 9.7 (16.2) times the Standard Model prediction at 95% confidence level. The Higgs boson self-interaction coupling parameter κλ and the quadrilinear coupling parameter κ2V are each separately constrained by this analysis to be within the ranges [−6.2, 13.3] and [−0.17, 2.4], respectively, at 95% confidence level, when all other parameters are fixed.