Say it with me now: Sex is a social construct.
A 'convenient' fitting into boxes usually two, but accurately not always.
The problem with such a system is that it's limiting and not at all useful in trying to help people.
We should be specific in what we are trying to say, because the miasma of a sex binary isn't useful, even in medical fields as it conditions doctors to think in very limiting ways and not actually help accurately.
It also has many roots in patriarchal violence in determining what a person (though to them a body) is for.
It's also problematic in a racist sense because of the colonialist white 'western' ideas of what makes a certain sex or gender often don't fit those who aren't in those categories (except colonised).
Which is why the 'science' is very problematic in this regard because that is the bias/lens with which it looks at this specific 'field' through.