View all notes
Computer code itself functions as a form of law. It is written by humans and it regulates their behavior, like other systems of power distribution. It is not an objective process or force of nature. It expresses a power relation between coder and user, and it will reflect the system in which coders work. “Code is never found,” Lawrence Lessig reminds us. “It is only ever made, and only ever made by us.” Letting the free market determine these matters means that digital technology risks reproducing discrimination under the cover of an inscrutable process. [...]
The implication was that these modifications to the standard testing process were done in response to public outrage, generated by Dowie’s article and the lawsuits. Ford was held to standards that other companies were not—standards it had no way of knowing that it was required to meet. Ford has never admitted that it did anything wrong.
Yet the lesson is this: as we learn more about an industry and what is possible technologically, we need to update our expectations in terms of safety and accountability. We need to organize activists, lawyers and journalists to highlight the human consequences of badly designed technology, and force the industry to adapt to design culture that values safety and works to mitigate bias. We need to demand that governments intervene in this industry to establish publicly determined standards and methods for holding companies accountable when they are breached. The standards must be constantly updated and responsive to changing circumstances as we learn more about the problems and experiment with solutions. Just like we would not let a car onto the road without crash tests, the parameters of which are subject to public scrutiny and influence, algorithms and products should not be inflicted on the public if they have not met certain standards or been tested for certain biases before shipping. We need to create a feedback loop for good design that allows lessons learned in the field to inform improvements to a product.
Digital technology, at least as much as any innovation over the last two centuries, offers us the opportunity to create a society that can meet the needs of every human being and allow them to explore their potential. But at present, too much power over the development of technology rests in the hands of technology capitalists and political elites who do their bidding. These people are good at what they do and also at convincing us that they are the best people to do it. They talk about egalitarianism and social connection in their public relations pitches and marketing campaigns, but what they hope to gain from digital technology is different: they aspire to wealth and power.
what if some of them actually believe in it tho. that's the scariest thing to me
These are serious and frightening design problems, but they also result in an immense waste of human potential. It is not just that companies do not invite feedback on their software or input from users on their design. Their objective, the purpose of their software, is not to service the user. Their primary goal is to retain control of that software. They want to control who uses it (that is, only paying customers). Proprietary software design makes a fetish of creativity—turning it into something abstract and commodified, geared to the purpose of money-making, rather than a collective or public good.
Paine’s writings mark the beginning in Western political thought of a very long tradition, namely that people are worth more than their productive capacity. People are not simply economic units whose moral worth is measured by what they produce for the market. Every person is entitled to a share in the wealth that is created collectively in the world, as part of recognizing his or her existence. The capacity to feed, clothe and shelter oneself ought not be dependent on the capacity to do some kind of productive work, especially when being productive is defined within the narrow confines of material value determined by market capitalism. The moral hierarchy created around people’s economic success as a productive unit is false and dehumanizing. People have a claim upon society by virtue of their humanity.
The aristocracy represented the ossified capriciousness of wealth distribution that Paine confronted in pre-revolutionary America; Engels denounced the horrors imposed on workers by industrialists in nineteenth-century England. The leaders of technology capitalism fall into a similar category today. Their much-lauded intellect and innovations are directed toward inflating and jealously guarding the wealth they accumulate and helping their fellow captains of industry to do the same. They are indifferent to the miseries they inflict on countless people; they see themselves as noble individuals pursuing their divine right to make money. Yet this is only possible through the exploitation of labor. Which also means that organized labor still holds the power to transform society.
The survival of capitalism is also dependent on using technology to extract more from workers wherever possible. Companies are investing in technology that allows them to reduce their labor requirements through data-driven scheduling, and the costs of such arrangements fall to workers to bear. The increasing adoption of this kind of technology is responsible in part for the rise in casualization of labor over the last several decades. Nearly 60 percent of American workers (around 80 million people) are paid by the hour, and nearly half of these are subject to just-in-time scheduling, with no certainty about hours or start times. When we are at work, we are watched more closely than ever. We carry devices connected to the cloud, for example, which monitor every task and the time it takes to perform. The much-derided consumer version of Google Glass has been reincarnated as a workplace management tool, to better track every task assigned to a worker. Amazon is patenting a wristband to squeeze more out of its workers by guiding their movements so they can pack items more quickly. Employers are making the most of technology for the purposes of optimizing how they use and monitor labor.
Automation under our current system—where retraining is expensive and the welfare net is minimal—wastes the potential of both workers and technology. It creates the paradox of both increased productivity and impoverishment. Outsourcing work to machines, particularly if the task is boring or dangerous, is a highly worthwhile goal, but it is not one that we are moving toward under the current system. And those who will suffer under successive waves of automation will often be the most vulnerable layers of society—the worst-paid, with the fewest skills and the fewest resources to fall back on in times of crisis. [...]
We are already moving into a world where work can be be flexible, self-directed, prioritize the sustainable use of resources, and be based on trust and community-building. It is possible to imagine how gig enterprises could be transformed into worker cooperatives to ensure that the benefits of working in this way accrue not just to company owners but also to the people who are actually doing the work. To do so might require certain interim policies, such as tax breaks for cooperatives or finding ways to facilitate the transition to worker ownership when a company becomes insolvent. It could also take the form of unions demanding a form of ownership, even control of members’ workplaces. When workers have a greater opportunity to participate in decisions about how things are done, it is easy to see how the worst kinds of work will gradually be prioritized for automation. “Bullshit jobs,” like flunkies for management or fixers of nonexistent problems, would be deprioritized, even rendered unnecessary once there was greater feedback between people making strategic decisions and people doing the work. The necessary skills for adapting to new technologies would be openly shared. This model takes the advantages of digital technology—its capacity to scale up a sense of trust and community—and distributes them far more broadly than is the case at present.
hell yeah
A better way to understand what we mean when we talk about privacy, then, is to see it as a right to self-determination. Self- determination is about self-governance, or determining one ’s own destiny. Its origins as a legal concept stretch back to the American Declaration of Independence, which states that governments derive “their just powers from the consent of the governed.” It has always featured as a right of some description in international law, usually in the framework of nationhood and governance of territory. But with the explosion of postcolonial struggles in the latter half of the twentieth century, it gained new meaning—not least in the struggle for Algerian independence that Fanon was involved in. In places like South Africa, Zimbabwe (then Rhodesia), the Democratic Republic of Congo and others, mass social movements struggled for recognition outside the confines of colonial settler states. Later these places often found themselves burdened with postcolonial systems that reproduced familiar hierarchies. The right to self-determination took on a renewed and deeper urgency, raising questions about how to empower people culturally, socially and politically, outside of the European ideals that offered lofty language but had also legitimized colonialism.
huh