Prolonged exposure to stress hormones can negatively impact the structure and function of the brain, particularly in areas related to memory.