We present measurements of cross sections for production of a leptonically decaying $Z$ boson in association with a large-radius jet in 13 TeV proton-proton collisions at the LHC, using $36\text{ }\text{ }{\mathrm{fb}}^{\ensuremath{-}1}$ of data from the ATLAS detector. Integrated and differential cross sections are measured at particle level in both a flavor inclusive and a doubly $b$-tagged fiducial phase space. The large-radius jet mass and transverse momentum, its kinematic relationship to the $Z$ boson, and the angular separation of $b$-tagged small-radius track jets within the large-radius jet are measured. This measurement constitutes an important test of perturbative quantum chromodynamics in kinematic and flavor configurations relevant to several Higgs boson and beyond-Standard-Model physics analyses. The results highlight issues with modeling of additional hadronic activity in the flavor-inclusive selection, and a distinction between flavor-number schemes in the $b$-tagged phase space.