[...] Facebook is a carefully managed top-down system, not a robust public square. It mimics some of the patterns of conversation, but that's a surface trait. In reality, Facebook is a tangle of rules and procedures for sorting information, rules devised by the corporation for the ultimate benefit of the corporation. Facebook is always surveilling users, always auditing them, using them as lab rats in its behavioral experiments. While it creates the impression that it offers choice, Facebook paternalistically nudges users in the direction it deems best for them, which also happens to be the direction that thoroughly addicts them. [...]

[...] A system is a human artifact, not a mathematical truism. The origins of the algorithm are unmistakably human, but human fallibility isn't a quality that we associate with it. When algorithms reject a loan application or set the price for an airline flight, they seem impersonal and unbending. The algorithm is supposed to be devoid of bias, intuition, emotion, or forgiveness. [...]

[...] even as an algorithm mindlessly implements its procedures--and even as it learns to see new patterns in the data--it reflects the minds of its creators, the motives of its trainers. Both Amazon and Netflix use algorithms to make recommendations about books and films. [...] the algorithms make fundamentally different recommendations. Amazon steers you to the sorts of books that you've seen before. Netflix directs users to the unfamiliar. There's a business reason for this difference. Blockbuster movies cost Netflix more to stream. Greater profit arrives when you decide to watch more obscure fare. [...]

[...] Google has explicitly built its search engine to reflect values that it holds dear. It believes that the popularity of a Web site gives a good sense of its utility; it chooses to suppress pornography in its search results and not, say, anti-Semitic conspiracists; it believes that users will benefit from finding recent articles more than golden oldies. These are legitimate choices--and perhaps wise business decisions--but they are choices, not science.

Perhaps Facebook no longer fully understands its own tangle of algorithms--the code, all sixty million lines of it, is a palimpsest, where engineers and layer upon layer of new commands.

Facebook would never put it this way, but algorithms are meant to erode free will, to relieve humans of the burden of choosing, to nudge them in the right direction. Algorithms fuel a sense of omnipotence, the condescending belief that our behavior can be altered, without our even being aware of the hand guiding us, in a superior direction. [...]