Charged lepton flavor and number violating $\tau$ lepton decays ($\tau$LFV and $\tau$LNV) are strongly suppressed in the Standard Model (SM) while New Physics scenarios beyond the SM predict branching fractions in the range of $(O(10^{-7}) -O(10^{-9}))$, reachable in the current or near-future experiments. The LFV $\tau$ lepton modes are studied using almost all data samples from the Belle experiment at the KEKB collider to set 90\% C.L. upper limits of $O(10^{-8})$ on the branching fractions. The Belle II experiment started physics data taking in 2019 to achieve 50 times higher statistics of the Belle data in the coming ten years. The physics prospect on the $\tau$ LFV decay search in the Belle II experiment, as extrapolations of the Belle results, are described.