Well I personally would be more inclined to believe the system (but that is largely down to a lack of faith in dealers) Presumably you have your copy of the original order form, have you dug that out and checked it states grey upholstery on that? If so then if it does arrive with black then it’s clearly not what was ordered and so rejection would be perfectly acceptable.
If, however, the form also says black then the area of rejection becomes a little greyer (no pun intended) as it might be argued you didn’t check the form etc. But others with more knowledge of rejection criteria will doubtless be along to clarify further.
Ultimately, you can’t be forced to put your pin in.
Of course, it might be black and you might decide you like it anyway 🙂