The group, all from Birmingham, were arrested on suspicion of possessing articles and documents with intent to use them for terrorist purposes overseas.
Officers from West Midlands Counter Terrorism Unit travelled to Oman last week to liaise with the authorities there before deciding to arrest the men as they touched down in the UK.
The pre-planned operation was intelligence led and the men were arrested under Section 57 of the Terrorism Act 2000.
It is not clear why the men had travelled to Oman, but West Midlands Police said the arrests had not been made in response to any immediate threat to public safety.
Two of the men, aged 33 and 39, are from the Small Heath area of Birmingham, while the third, aged 33, is from the Alum Rock district.
There were being questioned by anti-terror officers at Heathrow Airport.