With the increased interest in vegetarian and vegan diets a number of young men are turning to alternative foods such as soy, but if you are planning a family it might not be such a good idea. According to a new report in the Journal of Human Reproduction, consumption of even small amounts of soy products can lower a man’s sperm concentration.

Harvard School of Public Health conducted a study and found that the men who ate the most soy had sperm counts far lower than those who reported eating no soy at all. On average, the soy eaters had 41 million fewer sperm per milliliter. The study found that the soy eaters consumed a relatively low amount of 15 soy-based foods including tofu, soy burgers, soy ice cream, soy energy bars, and miso soup — with an average of just one serving of every other day. Now, this may not be significant enough to cause fertility problems, but if you are planning a family then it seems sensible to take every precaution. The study’s lead researcher Jorge Chavarro feels that the evidence is not strong enough one way or the other to deter men from eating soy, but it is known that the isoflavones in soy have long been tied to infertility in animal studies and it seems reasonable to infer they could also affect humans.