Abstract A search for new physics in top quark production is performed in proton-proton collisions at $$13\,\text {TeV} $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mn>13</mml:mn><mml:mspace/><mml:mtext>TeV</mml:mtext></mml:mrow></mml:math> . The data set corresponds to an integrated luminosity of $$35.9{\,\text {fb}^{-1}} $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mn>35.9</mml:mn><mml:mrow><mml:mspace/><mml:msup><mml:mtext>fb</mml:mtext><mml:mrow><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mrow></mml:math> collected in 2016 with the CMS detector. Events with two opposite-sign isolated leptons (electrons or muons), and $$\mathrm{b}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>b</mml:mi></mml:math> quark jets in the final state are selected. The search is sensitive to new physics in top quark pair production and in single top quark production in association with a $$\mathrm{W}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>W</mml:mi></mml:math> boson. No significant deviation from the standard model expectation is observed. Results are interpreted in the framework of effective field theory and constraints on the relevant effective couplings are set, one at a time, using a dedicated multivariate analysis. This analysis differs from previous searches for new physics in the top quark sector by explicitly separating $$\mathrm{t}\mathrm{W}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>t</mml:mi><mml:mi>W</mml:mi></mml:mrow></mml:math> from $$\mathrm{t}{\bar{\mathrm{t}}}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>t</mml:mi><mml:mover><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mo>¯</mml:mo></mml:mrow></mml:mover></mml:mrow></mml:math> events and exploiting the specific sensitivity of the $$\mathrm{t}\mathrm{W}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>t</mml:mi><mml:mi>W</mml:mi></mml:mrow></mml:math> process to new physics.