The low-energy $J/\psi N$ scattering is of significant importance for various reasons. It is deeply interconnected with the hidden-charm $P_c$ pentaquark states, provides insights into the role of gluons in nucleon structures, and is pertinent to the properties of $J/\psi$ in nuclear medium. The scattering can occur through two distinct mechanisms: the coupled-channel mechanism involving open-charm meson-baryon intermediate states $\Lambda_c \bar D^{(*)}$ and $ \Sigma_c^{(*)}\bar D^{(*)}$, and the soft-gluon exchange mechanism. In this study, we investigate the $S$-wave $J/\psi N$ scattering length arising from both mechanisms. Our findings indicate that both mechanisms lead to attractive interactions, yielding scattering lengths of $[-10, -0.1] \times 10^{-3}$ fm for the coupled-channel mechanism and $<-0.16$ fm for the soft-gluon exchange mechanism, respectively. Notably, the soft-gluon exchange mechanism produces a scattering length that is at least one order of magnitude larger than that from the coupled-channel mechanism, indicating its predominance. These findings can be corroborated through lattice calculations and will enhance our understanding of scattering processes that violate the Okubo-Zweig-Iizuka rule.