A bstract The first search for nonresonant production of Higgs boson pairs (HH) with one H decaying into four leptons and the other into a pair of b quarks is presented, using proton-proton collisions recorded at a center-of-mass energy of $$ \sqrt{s} $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msqrt> <mml:mi>s</mml:mi> </mml:msqrt> </mml:math> = 13 TeV by the CMS experiment. The analyzed data correspond to an integrated luminosity of 138 fb − 1 . A 95% confidence level upper limit of 32.4 is set on the signal strength modifier μ , defined as the ratio of the observed HH production rate in the $$ \textrm{HH}\to {\textrm{ZZ}}^{\ast}\textrm{b}\overline{\textrm{b}}\to 4\ell \textrm{b}\overline{\textrm{b}} $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>HH</mml:mi> <mml:mo>→</mml:mo> <mml:msup> <mml:mi>ZZ</mml:mi> <mml:mo>∗</mml:mo> </mml:msup> <mml:mi>b</mml:mi> <mml:mover> <mml:mi>b</mml:mi> <mml:mo>¯</mml:mo> </mml:mover> <mml:mo>→</mml:mo> <mml:mn>4</mml:mn> <mml:mi>ℓ</mml:mi> <mml:mi>b</mml:mi> <mml:mover> <mml:mi>b</mml:mi> <mml:mo>¯</mml:mo> </mml:mover> </mml:math> decay channel to the standard model (SM) expectation. Possible modifications of the H trilinear coupling λ HHH with respect to the SM value are investigated. The coupling modifier κ λ , defined as λ HHH divided by its SM prediction, is constrained to be within the observed (expected) range − 8 . 8 ( − 9 . 8) &lt; κ λ &lt; 13 . 4 (15 . 0) at 95% confidence level.