In Dan Davies - On "The Unaccountability Machine" author and former investment banker Dan Davies explains how modern organizations act as artificial intelligences that diffuse responsibility.
Society has shifted away from responsibility ("the buck stops here") towards procedural compliance.
"A corporation, or a government department isn't a conscious being, but it is an artificial intelligence. It has the capability to take decisions which are completely distinct from the intentions of any of the people who compose it."
"Under stressful conditions, it can go stark raving mad."
"If people are worried about out-of-control AI taking over the world and destroying everything, shouldn't we have been trying to do something about them at least seventy years ago – and probably more like two hundred? "
"Businesses ought to be like artists, not paperclip maximizers."
The world is now dystopian although I am a great believer in the human spirit which will will rise up and conquer this absurdity.
The European Union is a very good example of an organisation as artificial intelligence. The problem with the European Union is not just that power is taken from democratically accountable bodies such as national parliaments but that the power then disappears into an opaque bureaucratic and sterile matrix of almost infinite complexity, like EU law.
"We have created an entire economy predicated on 'defensive living' and preemptive documentation. The big winners of this era aren't the innovators or the public, but the specialized legal and corporate compliance class, who have successfully monetized complexity and turned checking a box into an extraordinarily lucrative form of institutional power."
— David Graeber, Anthropologist and Author, Bullshit Jobs: A Theory (2018)
"The great triumph of contemporary compliance is the total elimination of human discretion. By reducing every ethical choice to a codified rule, the compliance state ensures that you no longer need to look your neighbor in the eye to decide what is just. The law doesn't live within us anymore; it sits outside us as a cold text that demands absolute obedience, regardless of the human cost." Adrian Vermeule, Constitutional Law Professor at Harvard Law School.
The Tyranny of the Abstract Over the Real
When words are taken away from human beings and weaponized as a "compliance layer," language ceases to be a tool for mutual understanding and becomes a mechanism of automated control.
"Compliance law treats society like software. It operates on the dystopian premise that if you can just write enough rules, you can automate virtue. But virtue cannot be codified; it must be lived. When we worship the compliance manual, we are worshiping an idol made of sterile syntax, while the living, breathing human element is left to starve." Douglas Rushkoff, the author of Team Human
"The danger of the legalistic society is that it mistakes compliance for conscience. When the Word is made flesh, it invites relationship, judgment, and love. When the flesh is made compliance, it requires only an automated surrender to the text. It is a bleak, bureaucratic inversion of the sacred." Archbishop Rowan Williams, former Archbishop of Canterbury