Thinking and memory skills were most improved when people exercised the heart and muscles on a regular basis, a review of 39 studies found.
This remained true in those who already showed signs of cognitive decline.
Taking up exercise at any age was worthwhile for the mind and body, the Australian researchers said.
Exercises such as T'ai Chi were recommended for people over the age of 50 who couldn't manage other more challenging forms of exercise, the study in the British Journal of Sports Medicine said.
Physical activity has long been known to reduce the risk of a number of diseases, including type-2 diabetes and some cancers, and it is thought to play a role in warding off the brain's natural decline as we enter middle age.
The theory is that through exercise the brain receives a greater supply of blood, oxygen and nutrients that boost its health as well as a growth hormone that helps the formation of new neurons and connections.
In this analysis of previous studies, researchers from the University of Canberra looked at the effects of at least four weeks of structured physical exercise on the brain function of adults.
In a variety of brain tests, they found evidence of aerobic exercise improving cognitive abilities, such as thinking, reading, learning and reasoning, while muscle training - for example, using weights - had a significant effect on memory and the brain's ability to plan and organise, the so-called executive functions.
Joe Northey, study author and researcher from the Research Institute for Sport and Exercise at Canberra, said the findings were convincing enough to enable both types of exercise to be prescribed to improve brain health in the over-50s.
"Even if you are doing moderate exercise only once or twice a week there are still improvements in cognitive function, but the improvements were better the more exercise was done," he said.
He said people should be able to hold a conversation while doing moderate exercise.
NHS guidelines recommend that adults do at least 150 minutes of moderate aerobic activity every week and exercise the major muscles on two or more days a week.
Heavy shopping bags
Dr Justin Varney, lead for adult health and wellbeing at Public Health England, said any physical activity was good for brain and body.
"Whilst every 10 minutes of exercise provides some benefit, doing 150 minutes a week cuts the chances of depression and dementia by a third, and boosts mental health at any age.
"Doing both aerobic and strengthening exercises leads to a greater variety of health benefits."
Inspired to try some exercise?
He said cycling or walking to work could count as aerobic activity and carrying heavy shopping bags was one example of a good strength exercise.
Dr Dean Burnett, lecturer in neuroscience and psychiatry at Cardiff University, said the study gave a clearer picture of how exercise affected the brain - but there were still issues.
"It could lead to increased pressure for the 50-plus age group to exercise more in order to stay mentally healthy, which is good advice but also overlooks the fact that as we age it's increasingly difficult to engage in physical activity, as our bodies are simply less capable of it," he said.
"Physical exercise is one element of improved brain functioning, but not the whole story."
As well as staying physically active, Dr David Reynolds, from Alzheimer's Research UK, said it was equally important to look after our brains by staying mentally active, eating a balanced diet, drinking only in moderation and not smoking.