The US administration has not approved the transfer but is considering it as an incentive for the militants to enter negotiations in Afghanistan.
None of the five inmates is accused of directly killing Americans.
They would be re-united with their families in Qatar, which is playing an increasing role in negotiations.
They reportedly agreed to the transfer when they met Afghan government officials who visited the US prison this week on a mission from President Hamid Karzai.
Correspondents point out that the visit to the US prison on Cuba would not have been possible without US approval.
Aside from the aim of ending the war in Afghanistan, the prospect of transferring Taliban detainees proves once again that, more than three years after he promised to close it, Guantanamo Bay remains a thorn in President Obama's side, the BBC's Jonathan Blake reports from Washington.
If the president pursues this strategy, though, he will need support from wary politicians in Congress, our correspondent says.
Many there see a transfer of what they call the most dangerous inmates at Guantanamo as a step too far, he adds.
Ibrahim Spinzada, a senior President Karzai aide, visited Guantanamo on Monday, according to Reuters.
Both he and Shahida Abdali, a senior Afghan security official, also visited the US this week, the White House said without giving details.
Qatar's growing global clout
"We are hopeful this will be a positive step towards peace efforts," Mr Karzai's spokesman, Aimal Faizi, told Reuters news agency.
Asked about the transfer plan, White House spokeswoman Caitlin Hayden said: "The United States has not decided to transfer any Taliban officials from Guantanamo Bay.
"We are not in a position to discuss ongoing deliberations or individual detainees, but our goal of closing Guantanamo is well established and widely understood."
The spokeswoman pointed out that any decision on transfers would be undertaken in accordance with US law and in consultation with Congress.
US officials are hoping that President Obama can announce the establishment of fully fledged political talks between the Karzai government and the Taliban at a Nato summit in May.
The international peacekeeping mission in Afghanistan is due to finish at the end of 2014.
A total of 171 detainees were still being held at Guantanamo as of this month, Reuters reports.
In January, the Taliban announced it was opening a political office in Qatar.
Afghan Foreign Minister Zalmay Rasool is expected to visit the Gulf state this month for talks with government officials on reconciliation with the Taliban.