A legal challenge to the UK’s controversial mass surveillance regime has revealed shocking failures by the main state intelligence agency, which has broad powers to hack computers and phones and intercept digital communications, in handling people’s information.

The challenge, by rights group Liberty, led last month to an initial finding that MI5 had systematically breached safeguards in the UK’s Investigatory Powers Act (IPA) — breaches the Home Secretary, Sajid Javid, euphemistically couched as “compliance risks” in a carefully worded written statement that was quietly released to parliament.

Today Liberty has put more meat on the bones of the finding of serious legal breaches in how MI5 handles personal data, culled from newly released (but redacted) documents that it says describe the “undoubtedly unlawful” conduct of the UK’s main security service which has been retaining innocent people’s data for years.

The series of 10 documents and letters from MI5 and the Investigatory Powers Commissioner’s Office (IPCO), the body charged with overseeing the intelligence agencies’ use of surveillance powers, show that the spy agency has failed to meet its legal duties for as long as the IPA has been law, according to Liberty.

The controversial surveillance legislation passed into UK law in November 2016 — enshrining a system of mass surveillance of digital communications which includes a provision that logs of all Internet users’ browsing activity be retained for a full year, accessible to a wide range of government agencies (not just law enforcement and/or spy agencies).

The law also allows the intelligence agencies to maintain large databases of personal information on UK citizens, even if they are not under suspicion of any crime. And sanctions state hacking of devices, networks and services, including bulk hacking on foreign soil. It also gives U.K. authorities the power to require a company to remove encryption, or limit the rollout of end-to-end encryption on a future service.

The IPA has faced a series of legal challenges since making it onto the statute books, and the government has been forced to amend certain aspects of it on court order — including beefing up restrictions on access to web activity data. Other challenges to the controversial surveillance regime, including Liberty’s, remain ongoing.

The newly released court documents include damning comments on MI5’s handling of data by the IPCO — which writes that: “Without seeking to be emotive, I consider that MI5’s use of warranted data… is currently, in effect, in ‘special measures’ and the historical lack of compliance… is of such gravity that IPCO will need to be satisfied to a greater degree than usual that it is ‘fit for purpose'”.”

Liberty also says MI5 knew for three years of failures to maintain key safeguards — such as the timely destruction of material, and the protection of legally privileged material — before informing the IPCO.

Yet a key government sales pitch for passing the legislation was the claim of a ‘world class’ double-lock authorization and oversight regime to ensure the claimed safeguards on intelligence agencies powers to intercept and retain data.

So the latest revelations stemming from Liberty’s legal challenge represent a major embarrassment for the government.

“It is of course paramount that UK intelligence agencies demonstrate full compliance with the law,” the home secretary wrote in the statement last month, before adding his own political spin: “In that context, the interchange between the Commissioner and MI5 on this issue demonstrates that the world leading system of oversight established by the Act is working as it should.”

Liberty comes to the opposite conclusion on that point — emphasizing that warrants for bulk surveillance were issued by senior judges “on the understanding that MI5’s data handling obligations under the IPA were being met — when they were not”.

“The Commissioner has pointed out that warrants would not have been issued if breaches were known,” it goes on. “The Commissioner states that “it is impossible to sensibly reconcile the explanation of the handling of arrangements the Judicial Commissioners [senior judges] were given in briefings…with what MI5 knew over a protracted period of time was happening.”

So, basically, it’s saying that MI5 — having at best misled judges, whose sole job it is to oversee its legal access to data, about its systematic failures to lawfully handle data — has rather made a sham of the entire ‘world class’ oversight regime.

Liberty also flags what it calls “a remarkable admission to the Commissioner” — made by MI5’s deputy director general — who it says acknowledges that personal data collected by MI5 is being stored in “ungoverned spaces”. It adds that the MI5 legal team claims there is “a high likelihood [of material] being discovered when it should have been deleted, in a disclosure exercise leading to substantial legal or oversight failure”.

“Ungoverned spaces” is not a phrase that made it into Javid’s statement last month on MI5’s “compliance risks”.

But the home secretary did acknowledge: “A report of the Investigatory Powers Commissioner’s Office suggests that MI5 may not have had sufficient assurance of compliance with these safeguards within one of its technology environments.”

Javid also said he had set up “an independent review to consider and report back to me on what lessons can be learned for the future”. Though it’s unclear whether that report will be made public. 

We reached out to the Home Office for comment on the latest revelations from Liberty’s litigation. But a spokesman just pointed us to Javid’s prior statement. 

In a statement, Liberty’s lawyer, Megan Goulding, said: “These shocking revelations expose how MI5 has been illegally mishandling our data for years, storing it when they have no legal basis to do so. This could include our most deeply sensitive information – our calls and messages, our location data, our web browsing history.

“It is unacceptable that the public is only learning now about these serious breaches after the Government has been forced into revealing them in the course of Liberty’s legal challenge. In addition to showing a flagrant disregard for our rights, MI5 has attempted to hide its mistakes by providing misinformation to the Investigatory Powers Commissioner, who oversees the Government’s surveillance regime.

“And, despite a light being shone on this deplorable violation of our rights, the Government is still trying to keep us in the dark over further examples of MI5 seriously breaching the law.”