For a given Markov chain Monte Carlo (MCMC) algorithm, we define the distance between configurations that quantifies the difficulty of transitions. This distance enables us to investigate MCMC algorithms in a geometrical way, and we investigate the geometry of the simulated tempering algorithm implemented for an extremely multimodal system with highly degenerate vacua. We show that the large scale geometry of the extended configuration space is given by an asymptotically anti-de Sitter metric, and argue in a simple, geometrical way that the tempering parameter should be best placed exponentially to acquire high acceptance rates for transitions in the extra dimension. We also discuss the geometrical optimization of the tempered Lefschetz thimble method, which is an algorithm towards solving the numerical sign problem.