Cosmic rays leave their sources mainly along the local magnetic field present in the region around the source and in doing so they excite both resonant and non-resonant modes through streaming instabilities. The excitation of these modes leads to enhanced particle scattering and in turn to a large pressure gradient that causes the formation of expanding bubbles of gas and self-generated magnetic fields. By means of hybrid particle-in-cell simulations, we demonstrate that, by exciting this instability, cosmic rays excavate a cavity around their source where the diffusivity is strongly suppressed. This phenomenon is general and is expected to occur around any sufficiently powerful cosmic ray source in the Galaxy. Our results are consistent with recent $\gamma$-ray observations where emission from the region around supernova remnants and stellar clusters have been used to infer that the diffusion coefficient around these sources is $\sim 10-100$ times smaller than the typical Galactic one.