While data classification is an information security concept, it's an intuitive idea to everyone. Certain information is sensitive, and access must be limited. That relates to another intuitive concept, the need to know. Unless there's a compelling reason for a person to know about something, access to that information should be restricted.
The US government and military have long considered data classification as part of their core security programs. They have formalized the program and embedded it into all aspects of government and military life. But over the last half-century, that best practice of data classification has become a monstrosity.
In The Declassification Engine: What History Reveals About America's Top Secrets (Pantheon Books), author Matthew Connelly, professor of international and global history at Columbia University, has written a fascinating account of the topic. He details how data classification has betrayed its intent, and can in fact, be a danger to democracy.
Connelly makes a compelling argument that the US government's current policies and practices regarding data classification are outdated and ineffective. China, for example, has exfiltrated tens of millions of records revealing personal information about US government employees, and Russia has infiltrated hundreds of government and corporate networks. And all of that data was classified.
For decades, government committees and commissions have identified the same fundamental problem: officials have not clearly and consistently specified what information actually requires safekeeping, making it impossible to prioritize what is truly sensitive. And for decades, there has been nothing significant done to fix that, as the book states.
He writes that those in the military will painstakingly study forty-year-old military records page by page because of the infinitesimal risk that a nuclear bomb design might slip by. Meanwhile, sniper manuals and recipe books for high explosive documents that could easily kill people are accidentally released and left open on shelves of the National Archives.
Unless someone can systematically analyze what kinds of information are classified and why, we can't begin to develop practical techniques for a more rational, risk-management approach to releasing non-sensitive records.
Connelly and his team at History Lab developed a sophisticated software tool to do that systematic analysis. After presenting their findings and approach to government and military officials, Connelly was surprised to learn that they were not interested in moving forward..
They developed this declassification engine, a platform that combined big data, high-performance computing, and sophisticated algorithms to reveal what the government did not want us to know and why they did not want us to know it.
A large part of the government and military's reticence to change a clearly broken system is that data is a valuable and powerful asset for them. Reclassifying data, even if it is in the best interests of everyone else, would diminish their power.
Connelly does a fantastic job of detailing the history of data classification over the last 100 years. He writes that as far back as 1940, Franklin Delano Roosevelt became the first president to issue an executive order that began to define a hierarchy of classified information for the federal government.
While the idea of creating a system of data classification had legitimate merits, it soon collapsed under its own weight. The problem was so bad that a 1956 Defense Department study found that data overclassification had reached serious proportions.
Numerous reviews about data classification all came to the same conclusion: that all of the data classification systems made it harder to prioritize the protection of truly sensitive information.
In addition, a bigger problem with data overclassification is that it prevents Americans from knowing what their government and military are doing. Connelly writes that they have a legitimate right to know this information, but it's often unlawfully withheld from them.
The culture of secrecy within the military and government that data classification endears not only makes it harder for people to access information they have a legitimate right to, but it also weakens democracy as a whole.
A key point from the book is that the situation is only getting worse as the digital footprint of US top-secret data grows at a massive scale—so much so that the National Archives is clueless about its actual size. The government as a whole lacks the capabilities to track all of the classified data.
At a little over 400 pages, this is an extremely dense and broad book, and an important information security read. Connelly takes the reader through the history of classification, various wars, surveillance and espionage, military conflicts, and more. And shows how data classification affected these activities and history.