The size of the proton is of lasting and high interest in the subatomic physics community. The most well-known example is the electric radius which has been the subject of the proton radius puzzle for more than a decade. While tremendous progress in $ep$-scattering, atomic spectroscopy, and lattice QCD has brought this puzzle closer to its resolution, one also finds discrepant results for the magnetic radius. In light of the upcoming high-precision measurements of the hyperfine splitting (HFS) in muonic hydrogen, other definitions of radii gain relevance as well. On the one hand, to infer the electric radius from the observed Lamb shift in muonic hydrogen, higher-order nuclear structure corrections need to be subtracted, which depend on the Friar radius of the proton. The magnetic properties of the proton, on the other hand, only enter the HFS via the proton's Zemach radius. Based on our previous calculation of the electromagnetic form factors of the proton and neutron, which includes both quark-connected and -disconnected contributions and assesses all sources of systematic uncertainties, we now present results for the Zemach and Friar radii. For the proton, we obtain $r_Z^p = ( 1.013 \pm 0.010\ (\mathrm{stat}) \pm 0.012\ (\mathrm{syst}) )~\mathrm{fm}$ and $r_F^p = ( 1.301 \pm 0.012\ (\mathrm{stat}) \pm 0.014\ (\mathrm{syst}) )~\mathrm{fm}$. These numbers suggest small values of the Zemach and Friar radii of the proton, and have a precision which is sufficient to make a meaningful comparison to data-driven evaluations.