We determine the nucleon axial, scalar and tensor charges at the continuum limit by analyzing three $N_f=2+1+1$
twisted mass fermion ensembles with all quark masses tuned to approximately their physical values. We include all contributions from valence and sea quarks. We use the Akaike Information Criterion to evaluate systematic errors due to excited states and the continuum extrapolation. For the nucleon isovector axial charge we find $g_A^{u-d}=1.250(24)$, in agreement with the experimental value. We compute the axial, tensor and scalar charges for each quark flavor. The axial charge provides crucial information on the intrinsic spin carried by quark in the nucleon and the the latter two provide input for experimental searches of physics beyond the standard model. Moreover, we extract the nucleon $\sigma$-terms and find $\sigma_{\pi N}=41.9(8.1)$ MeV, for the strange $\sigma_{s}=30(17)$ MeV and for the charm $\sigma_{c}=82(29)$ MeV. We also present preliminary results on the isovector quantities using a fourth ensemble at smaller lattice spacing.