Exactly JS, England did not vote to leave the EU any more than NI or Scotland voted to remain, England, Wales, NI & Scotland voted as constituent parts of the U.K., in a full UK wide referendum, the result of which was for the UK to leave.
There was no separate referendum for any of the four countries, just the same as there was no separate referendum for ‘Manchester’, or ‘Abercwmboi’!
By voting in the referendum, Each country ‘legitimised’ both the referendum & the result, and no political party can cherry pick’ to suit it’s own political ends.
Clearly, no political party represents all views, but if they continue to argue that they represent the majority and that the majority view must prevail in their particular country, then by the very same token they must accept that the majority decision in the EU referendum must prevail – they cannot have it both ways.