Well said, Paul. That's where I come down on this.
But there will be massive pushback, because this is seen by many on the other side as an important inflection point, one in which the freedom to be who you are, or feel yourself most profoundly to be, in gender/sexuality terms, trumps all other considerations.
I don't think that men like you and I, regardless of how calm, reasonable, and powerful our arguments might be, can make a dent in how the other side, and how the general public, thinks. I don't think that moral suasion from us carries any weight at all. The ONLY thing, in my view, that will make a difference is if the female athletes themselves mobilize and resist and, if necessary, engage in nonviolent direct action, such as sitting out a race, as another poster has suggested.
The other thing that could conceivably make a difference is if trans athletes themselves came to the table and said, "Look, we want and need to be who we are, but we feel uncomfortable, to be honest, using our unfair advantage to take trophies away from our female peers." I see the chances of that happening as miniscule.