In order to reach (sub-)per cent level precision in lattice calculations of the hadronic vacuum polarisation, isospin breaking corrections must be included. This requires introducing QED on the lattice, and the associated finite-size effects are potentially large due to the absence of a mass gap. This means that the finite-size effects scale as an inverse polynomial in $L$ rather than being exponentially suppressed. Considering the $\mathcal{O}(\alpha)$ corrected hadronic vacuum polarisation in QED$_{\mathrm{L}}$ with scalar QED as an effective theory, we show that the first possible term, which is of order $1/L^{2}$, vanishes identically so that the finite-size effects start at order $1/L^{3}$. This cancellation is understood from the neutrality of the currents involved, and we show that this cancellation is universal by also including form factors for the pions. We find good numerical agreement with lattice perturbation theory calculations, as well as, up to exponentially suppressed terms, scalar QED lattice simulations.