Have you ever met a narcissist or psychopath? I have. Postmodern gender theories are a dream-come-true for them. Many women have encountered nasty, spiteful males in women's clothing who use gender identity ideology as a cover for their misogyny. It's the worst kind of gaslighting.
The fact that you know transpeople who are just trying to live their lives (spoiler alert: many of us know people like this) does not preclude bad faith actors from wielding a subjectivist ideology to abuse people and consolidate power.
Aside from all of that, trans people with good intentions should not be given special privileges in sport. Fairness matters.