The elliptic flow of jet particles at midrapidity ($|\eta|$ < 0.8) is measured for the first time in p--Pb collisions at $\sqrt{s_{\rm NN}}$ = 5.02 TeV and extended down to lower $\it{p}_{\rm T}$ in Pb--Pb collisions at $\sqrt{s_{\rm NN}}$ = 5.02 TeV. The jet particles are extracted with the two-particle correlation method using a two-dimensional fit, and their $v_{2}$ is calculated with the novel three-particle correlation technique. The $v_{2}$ of jet particles is found positive in high-multiplicity p--Pb collisions and exhibits no dependence on $\it{p}_{\rm T}$ within uncertainties. Comparisons with the jet-particle $v_{2}$ in Pb--Pb collisions and the inclusive charged-particle $v_{2}$ in both collision systems are also discussed, aiming to bring new insights into the understanding of the origin of the high-$\it{p}_{\rm T}$ azimuthal anisotropy observed in small collision systems.