It comes three months after the Black Lives Matter protests prompted the removal of the statues of the slave traders Edward Colston in Bristol and Robert Milligan in London.
Mr Dowden wrote: "The government does not support the removal of statues or other similar objects. History is ridden with moral complexity. Statues and other historical objects were created by generations with different perspectives and understandings of right and wrong. Some represent figures who have said or done things which we may find deeply offensive and would not defend today.
"But though we may now disagree with those who created them or who they represent, they play an important role in teaching us about our past, with all its faults."
Mr Dowden said that museums "should seek to contextualise or reinterpret them in a way that enables the public to learn about them in their entirety" rather than "erasing" the objects.
He writes that he expects their “approach to issues of contested heritage to be consistent with the government’s position” and adds: "Further, as publicly funded bodies, you should not be taking actions motivated by activism or politics. The significant support that you receive from the taxpayer is an acknowledgement of the important cultural role you play for the entire country. It is imperative that you continue to act impartially, in line with your publicly funded status, and not in a way that brings this into question."
The British Museum, which last month announced it was moving the bust of its founder, the slave owner Hans Sloane, to a less-prominent position, confirmed it had received the letter.
A spokesperson responded: "The British Museum has no intention of removing controversial objects from public display. Instead, it will seek where appropriate to contextualise or reinterpret them in a way that enables the public to learn about them in their entirety."
The Museum said that the Sloane bust remains on display in the Enlightenment Gallery "juxtaposed with objects that reflect that Sloane's collection was created in the context of the British Empire and the slave economy". It added: "We continue to acknowledge Sloane's radical vision of universal free public access to a national museum collection and the public benefit that is generated through the British Museum."
Similar letters were also sent to the National Portrait Gallery, National Museums Liverpool, the Royal Armouries, the Science Museum and the Victoria and Albert Museum.
The British Library has also "reinterpreted" the busts of the historical figures Sir Hans Sloane, Sir Robert Cotton, Sir Joseph Banks, and Thomas Grenville, which have been on display in the front hall since it opened in 1997.
"As part of our wider programme of work on inclusivity and anti-racism we have reviewed the labelling and presentation of the busts, in order to add information to help visitors understand them within their wider historical context," it said.
The Department for Digital, Culture, Media and Sport confirmed the letter had been sent out earlier this week.