Replacing your double glazed windows won't offer any noticeable thermal benefit, that's a cosmetic thing.
Loft insulation can be added to, no need to remove the existing.
Cavity wall insulation will certainly make a difference, in some cases it can cause damp issues when it can bridge the cavity...