Can we detect dark matter using light from pulsars?