$\mu\to e$ conversion in nuclei gives one of the leading limits on lepton-flavor-violating (LFV) processes, with upcoming measurements calling for a more consistent theoretical description. This can be done model independently using an effective-field-theory framework in terms of effective beyond-Standard-Model operators, which, however, crucially depends on hadronic and nuclear matrix elements. In particular, the uncertainties inherent in these non-perturbative inputs limit the discriminating power that can be achieved. In order to quantify the associated uncertainties, we revisit nuclear charge densities and propagate uncertainties from elastic electron scattering experiments. These charge densities, parameterized in terms of Fourier--Bessel series, can be correlated with results from modern ab-initio methods and thus allow for the evaluation of general $\mu\to e$ conversion rates with quantified uncertainties. The resulting description of $\mu\to e$ conversion enables improved studies of the appearing effective operators, which can also be related to LFV pseudoscalar decays.