Algocracy – ever hear of it? Me neither. I was shocked to find out that the concept pre dates the smartphone, the cloud and social media. Worse yet, all this time it has been seeping into our world with a tyrannical force.
In a small town the government is quietly experimenting with having the children monitored very closely. It is searching for patterns that might indicate patterns of fraud or abuse. Vast amounts of data from almost every source imaginable – health records, parent’s employment records, even electricity bills – are regularly scanned and correlated to their national identity number. Potential offenders are put on “Puzzlement Lists” for potential investigation. These lists are then acted upon by local “Control Units”.
Scary isn’t it? Where do you think our town is located – China? Russia? North Korea? How about Denmark? Yes, this is a well-intentioned (remember what road that paves) attempt to extend the embrace of its vaunted welfare state to protect children from abuse and/or prevent parents from defrauding on benefits payments.
Public awareness and resentment has recently slowed the program but with all this vast amount of data available to be crunched by government programs the potential for a dangerous, even tyrannical mission creep is clear. And, guess what? The Danish government – like all EU governments – is exempt from all those highly praised new European rules designed to insure the privacy of personal data.
The key to efficiently generating those lists from the ocean of data collected is to use computers running algorithms to flag up the problematic behaviors. Of course, it is people with their conscious and unconscious biases that write these algorithms. Back in 2006, A. Aneesh – a respected scholar – wrote the book “Virtual Migration” where he coined the term “algocracy” to convey the growing importance and threat of algorithms in social life. Unaccountable computer algorithms replace the formerly accountable (sort of) human bureaucracy.
Aneesh developed these insights before the advent of the incredible amounts of personal data that we now generate from our smartphones and social media accounts. And, it was also before the arrival of the cloud, which could process it cheaply with gobs of computing power. Recent advances in artificial intelligence just add to the dystopian potential. All of which, as in Denmark’s case, are being implemented for the “public good”.
Sure the Chinese are the most Orwellian with their Social Credit Scores but it is a mistake to assume that our western societies are not succumbing to the same dynamic. The West is just being more decentralized about it – some driven by the profit motive and some by those best intentions for the good of the people. Profit drives the private hands. Think about it: who determines if you get a mortgage or how much your insurance costs – banks, credit agencies and insurance companies.
Public good means public agencies. Some of it is done by any number of the 18,000 police forces we have in the US as they use algorithms to predict crime hot spots and which offender’s presence alone is a red flag. If you are unlucky enough to get involved in the criminal justice system an algorithm may recommend your sentence to the presiding judge.
About a year ago I wrote a post about how AI was super charging the power of algorithms and making them into “black boxes” where we could not even begin to fathom how they arrived at their conclusions. In it I recommended a book by the author Cathy O’Neil where she exposes algorithms harmful effects – “Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy”. (Here is also her TED Talk.) She provides a decadal update to Aneesh’s work.
Since then the EU implemented the GDPR – General Data Protection Regulation – a good start against the power of corporations. (Except, as we saw, it doesn’t apply to the state – sigh). 2018 was also the year that Tech got knocked off its pedestal with the scandals of Facebook and Google. Looks like they may be in for some US regulations in 2019. So, there is hope we can hold back the forces of algocracy but it takes constant awareness and effort. People knew this many, many years ago and now it is our turn.
“Eternal vigilance is the price of liberty—power is ever stealing from the many to the few…. The hand entrusted with power becomes … the necessary enemy of the people. Only by continual oversight can the democrat in office be prevented from hardening into a despot: only by unintermitted Agitation can a people be kept sufficiently awake to principle not to let liberty be smothered in material prosperity.”
WENDELL PHILLIPS, speech in Boston, Massachusetts, January 28, 1852
Speeches Before the Massachusetts Anti-Slavery Society, p. 13 (1853)