Academics consider legal action to force Foreign Office to release public records

Mau Mau suspects being led away by police in Kenya's Rift Valley in 1952. The huge cache of unreleased papers only came to light after Kenyans tortured during the 1950s sued the British government to win compensation in 2011. Photo: Popperfoto/Getty

Leading historians are calling on the UK's Foreign Office to "come clean" over its plans for a massive archive of public documents, which it has unlawfully kept hidden for decades, prompting accusations that it has been attempting to manipulate impressions of Britain's past.

The Foreign and Commonwealth Office (FCO) has hoarded 1.2m files – some of them dating back to the 1840s – in breach of the 30-year rule of the Public Records Act, which should have seen them transferred to the National Archive.

Such is the level of concern among some historians that a number of leading figures from Oxford, Cambridge and London universities are known to have discussed whether legal action may be necessary to secure the archive and to bring it into the public domain.

Some are concerned that major works about contemporary British and imperial history may need to be rewritten, while others believe that what they describe as a scandalous act of concealment underlines the need for a major overhaul of the system for declassification of government papers as public records.

After the existence of the hidden archive was reported in the Guardian in October, FCO minister David Lidington told MPs that those documents considered by his department to be of the greatest public interest would be transferred to the National Archives at Kew, south-west London, over a six-year period.

One of the historians considering whether legal action may be needed to secure access is Richard Drayton, Rhodes professor of imperial history at King's College London, who described the pledge to hand over a proportion of the documents within six years as "weak and evasive".

Currently the archive – which the FCO calls its "special collections" – is hidden from public view inside a high-security facility at Hanslope Park, 60 miles north of London, which the department shares with MI5 and MI6.

Since Lidington's announcement, the FCO has refused to answer questions about its plans for the transfer of the archive. As a result, it is not yet known what proportion of the 1.2m documents will be released within six years; whether any of the remaining documents will be made public, and, if so, the timescale over which this will take place; and how much public money the operation will cost.

Some details of the FCO's plans for the archive were outlined at the National Archives in November, at a meeting from which the public was excluded. After studying the plan, the archives' advisory council, which more usually examines government departments' applications to retain files beyond 30 years, told the FCO that it needed to see greater detail. Neither the FCO nor the archives will make the plan public.

Meanwhile, the FCO is failing to meet its new public records obligations under the Constitutional Reform and Governance Act 2010, which is gradually reducing the 30-year rule to 20 years from this month. Other government departments are now releasing papers from both 1983 and 1984, and in 2015 will be releasing papers from 1985 and 1986.

The FCO, however, maintains that it is too busy dealing with the backlog created by its hoarding of the 1.2m files to be in any position to meet its new legal target.

Drayton said: "The FCO has so far released an inventory of such opacity that the public has only the slightest chance of identifying what is of 'greatest interest'. What we can glean from it, however, is that it contains material that runs back into at least the 1840s. That these public records are being withheld from the public is a scandal."

Drayton said he was concerned that the FCO would declassify some of its secret archive by quietly slipping papers into declassified files at Kew, as has happened in the past with both FCO and other government department papers. "The burying of secrets by the state at the department level must not be followed, as it has been in the past, by their burying in existing records at the National Archives." Drayton also believes that it is time for the Public Records Act to be overhauled.

Professor Margaret MacMillan, warden of St Antony's College, Oxford, said: "I am one of many historians who has benefitted from using the British archives and who had confidence that the documents had not been weeded to suit particular interests.

"Now I am wondering whether I will have to go back and rethink my work on such matters as the outbreak of the first world war or the peace conference at the end. But when are we going to get the complete records? So far the pace of transferring them is stately, to put it politely."

Professor Philip Murphy, director of the Institute of Commonwealth Studies at London University, warned against further secrecy in the handling of the special collections. "It's extremely important that the British government establishes a mechanism for the review of these files that is both transparent and credible," he said.

"It may well be that a significant proportion of this material is either of little historical interest or substantially duplicates material that is already in the public domain. But the very fact that the British government has – apparently – failed to act in accordance with the 1958 Public Records Act in respect to these files has already undermined its credibility.

"Unless a proper system of oversight is established, that credibility will suffer further. It's a mistake to imagine that the sense of resentment felt in those countries affected by British imperial policy will diminish as the years pass. If anything, we see signs of that collective feeling of resentment heightening in the early years of the 21st century. It is vital for the UK's international reputation that it can prove it is not seeking to whitewash the historical record."

The Foreign Office admitted to the existence of 8,800 files in 2011, during litigation brought on behalf of thousands of Kenyans whosuccessfully sued the UK government to win compensation for the tortures they suffered while prisoners of the British colonial authorities during the 1950s Mau Mau insurgency.

It did not admit at that time that these colonial-era files were just a tiny part of the secret archive of 1.2m papers. The department quietly arranged for the archive to be put on a legal footing in November 2012, by asking Chris Grayling, the justice secretary, to sign a 12-month authorisation. That was extended for a further 12 months last November, ostensibly to give the FCO more time to draw up plans for declassification of its secret archive.

An FCO spokesman said that the department would have further information early this year about both the special collections and releasing the 20-year-old papers.

The FCO is not the only government department that has been unlawfully hoarding public records. Earlier this year the Guardian disclosed that the Ministry of Defence was holding 66,000 files at an archive in the Midlands, in breach of the Public Records Act.

The disclosure caused concern in Northern Ireland, where police investigations into killings by soldiers and police are continuing, and where many inquests have yet to be concluded. However, the FCO's secret archive dwarves those of other departments.