(I'm not a doctor and barely skimmed through the relevant wikipedia articles, but here goes)

The "soy panic" is the result of a similar cognitive deficiency to why flat earthers, anti-vaxxers, sovereign citizens, climate deniers, and anti-nuclear activists exist: they hear something that sounds like it makes sense in their head, promptly shit their pants, and base their entire worldview on that misconception, which also lets them rage against the status quo and gives them a false sense of intellectual superiority (the "everybody else but me is wrong, wake up sheeple" mentality). That misconception then gets spread around in a massive game of Chinese telephone (I hate social media) and everybody else that tries to explain the science behind it gets labelled as a cuck, beta, soyboy, chud, every new-age insult they can think of.

In this case, the base misunderstanding comes from the fact that soy contains something called phytoestrogen. Sounds scary, right? People don't know what phytoestrogen is. I didn't. But they saw that it had "estrogen" in it and immediately latched onto the idea that consuming it would somehow suppress testosterone*, ruin their sperm, and feminize them, and the solution is to eat only a Manly Macho Man's Diet with lots of meat and dairy (which contains actual mammalian estrogen).

The problems with this hypothesis (I refuse to call it a "theory") are innumerable, not smallest of which is that it's complete hogwash with zero clinical evidence linking sperm count and testosterone levels to consuming soy products. Phytoestrogen is also a plant-based estrogen, which, unlike animal estrogen, can't even do whatever they claim it can do to the human body.

* On the topic of suppressing testosterone: it doesn't work like that. It works the opposite way in fact. Biological males have estrogen, but the testosterone suppresses its effects. That's why trans women have to take testosterone blockers.