Speaking on the sidelines of a post-election victory parade on Saturday, Mr Lee said he has yet to accept their decision, although he is set to give his answer by Monday.
Although PM Lee said the move was initiated by the two senior leaders themselves, he said he had already known about their intention.
He added that his decision will also affect his new Cabinet line-up.
"It is a major event for Singapore for two major figures like MM and SM to leave the government," he was reported as saying on Channel NewsAsia.
"It shows that they are thinking of Singapore and for the future of Singapore. The basic consideration is what is the best approach for forming a team which would be most effective in solving problems and establishing rapport and trust with the people going forward for the next five years," he added.
The term Minister Mentor was a term coined specially for the senior Lee in 2004 , as a reflection of his role in office, as well as his wealth of experience.
It will be the first time the 87-year-old founding father of Singapore will have no part to play in the Cabinet.
PAP MP-elect for Ang Mo Kio GRC Inderjit Singh called his decision to step down a major turning point for Singapore.
"Politics, or a Cabinet without MM, is something that we all never thought of. With MM stepping down and SM, this actually gives PM a clean slate to redraw his new Cabinet," he said.
"And I think this must have also been because of the messages that we got during this GE, of people wanting to see a different approach."
Another MP-elect for Ang Mo Kio GRC, Dr Intan Azura, said: "MM and SM have done a lot for Singapore. I think we have to really appreciate what they have done for the country, what they have done for the government; they have set a really strong foundation.
"But I guess they decided that it is time for us to move ahead with the times, with the new generation of leaders."