Heyy, so I've seen someone make this arguement and I honesty don't have an answer I can defend.
I do see saying "I'd never date a black person" as racist, but I don't see saying "I'd never date a man" as sexist. They're inherently different, but I can't word why.
I'd like to hear any arguements you have
Edit: I think what makes it hard for me to understand is that I don't know what makes a person attracted to any gender. Because what makes a woman a woman? Trans women are women and not all women look even similar to each other so it's not physical characteristics. But women also have very different personalities so it's not truley that either. So I think it's hard for me to point out the difference in different attractions when I don't understand how they even work