Might not be able to put into words MY thoughts on this, but will try.
I don't think they are tied together. I have a heifer that is VERY feminine, yet she is not frail boned at all. In fact, I am surprised she isn't pink!!! She moves very fluidly, with very smooth joints (compare any mans hands to a female hands). Her head has fine features, with a soft eye. She just LOOKS female.
I have another heifer that is a bit fine boned, but is also very feminine. Just not as broody looking.
On the other hand, I have seen some heifers win shows (and in the pasture) that you have to look behind (use the one hole, two hole theory) to tell what sex it is. Course shoulders, big joints (not just big boned) and a head that would make a bull proud. Thick neck, etc. Most have very small vulvas and I dare say, have a hard time breeding. I have seen it in every breed, every age group.
So I guess I would say I judge general head/neck look with joint size and shape.