There is some evidence to suggest that high sugar intake can have a negative impact on testosterone levels in men. When a person consumes a high amount of sugar, their body produces insulin to help process the sugar. High levels of insulin can interfere with testosterone production and lead to decreased levels.