I think it's like other things...not a deal breaker, but probably an obstacle. The reason being showing is a subjective activity. This isn't a contest where strictly impartial data is tallied and scored, and winners are the ones with the most points or the fastest times. It's somebody opinion regarding which animal or animal/showman team most resembles what that judge pictures as ideal in the industry. Opinions can vary. A lot!...
I was a tall kid, and I have a daughter showing now. She's 5'11" and she just turned 16. And I admit a taller person looks somewhat awkward presenting a smaller animal. When it's the other way around and you have a young exhibitor and a really big animal, as long as the exhibitor can handle that calf, it's just cute and endearing. That doesn't look as much like a mismatch. But while it may be more challenging or a little awkward, that doesn't mean it can be well done or deserve to be placed high in a class.
Particularly if this is not a showmanship class, a judge should just be evaluating the animal on its own merit. I think where things that don't really matter come into play is when placings get tight, and the judge is forced outside the logic he or she uses in most cases to place a class. It's human nature to have some gut feeling...to "go with your heart" in a situation like that, and in those cases, an awkward pairing of animal and exhibitor can be the deciding factor. I don't know that many judges would admit to that...it's just a hunch I have. I don't even think a lot of judges would be totally conscious of the reasoning behind what they did IF that was a part of the reason. And like I said, it shouldn't come into play unless placings are tight. We can criticize anybody who might admit to that being part of the criteria they used, but I think a lot of us might fall victim to the same instances of human nature if we were in the same shoes.