We report the result of a search for $\tau^\pm\rightarrow\ell^\pm\gamma$ ($\ell=\mu,e$) using the full data sample at Belle. Since the observation of neutrino oscillations has unambiguously shown that the lepton flavor is no longer conserved, we can expect lepton flavor violation (LFV) in the charged lepton sector. Though the standard model (SM) does not predict charged LFV decays at an observable rate, $\tau^\pm\rightarrow\ell^\pm\gamma$ is predicted by many new physics scenarios and is thus one of the most promising LFV modes. Consequently, we have obtained the most stringent limit on the branching fraction of $\tau^\pm\rightarrow\mu^\pm\gamma$. In addition, we report the result of a search for tau electric dipole moment (EDM) evaluating $u$ vertex coupling using the full data sample at Belle. At present, the observed CP violation is insufficient to explain the prevalent matter-antimatter asymmetry in our universe. On the other hand, the EDM of leptons is predicted to be negligibly small in the SM and is expected as a source of CPV in the lepton sector induced by some new physics. We have obtained one order more sensitive result both for the real and imaginary parts of the $\tau$ EDM.