It is widely recognized that cosmic rays are accelerated in supernova remnants by the diffusive shock acceleration (DSA). Although the standard DSA predicts the energy spectrum of $E^{-2}$, several observations suggest a deviation from $E^{-2}$. To explain such spectral deviations, we introduce a density fluctuation of interstellar medium (ISM) to the DSA. Particle acceleration is examined by Monte Carlo simulations. The simulations show that the density fluctuation of ISM does change the energy spectrum of accelerated particles from $E^{-2}$ and the effect is remarkable when the length scale of the fluctuation is comparable to the mean free path of the accelerated particle.