Put a note on the door saying let yourself in. Tell them it makes you live in fear as it’s so far to the front door.
The way way I understand you can refuse twice but they can still turn up for a home visit if they want.
Remember pip pip is based on what you can and can’t do, diagnosis isn’t the only issue.
But the main thing is sticking buy what you put on your application, with medical evidence as proof. ??
good luck to you.