I agree with the OP, although maybe for a more fundamental reason.
I agree that the idea of inter-school competition is (or was) "our best student-athletes against your best student-athletes". That at least makes some kind of sense. It's easier to root for a group of people who have some kind of shared roots, shared background, shared goals, etc, as you.
What I don't understand is how you are still supposed to root for your school when the "student-athletes" are mostly just guns-for-hire brought in from who-knows-where. How can you feel any shared sense of accomplishment? This idea would seem to reach its most mystifying peak in the big football and basketball programs: you are supposed to be rooting for a bunch of clowns who are going to your school because they are being paid to. They are not from the area, they have no interest in the area or school, they look down on the rest of the student body, and they are only using their time at your school as a kind of training camp.
But obviously I'm missing something here, because the glaringly obvious fact it, students DO root for these meatheads with whom they have nothing in common. In fact, they don't just root for them, they look up to them and admire them, and brag about it to their friends when they are so fortunate as to meet one of these gods on campus.
Now obviously XC/Track athletes are a little different, because nobody really cares about them, but still the same principle is in operation.