The implication was that these modifications to the standard testing process were done in response to public outrage, generated by Dowie’s article and the lawsuits. Ford was held to standards that other companies were not—standards it had no way of knowing that it was required to meet. Ford has never admitted that it did anything wrong.

Yet the lesson is this: as we learn more about an industry and what is possible technologically, we need to update our expectations in terms of safety and accountability. We need to organize activists, lawyers and journalists to highlight the human consequences of badly designed technology, and force the industry to adapt to design culture that values safety and works to mitigate bias. We need to demand that governments intervene in this industry to establish publicly determined standards and methods for holding companies accountable when they are breached. The standards must be constantly updated and responsive to changing circumstances as we learn more about the problems and experiment with solutions. Just like we would not let a car onto the road without crash tests, the parameters of which are subject to public scrutiny and influence, algorithms and products should not be inflicted on the public if they have not met certain standards or been tested for certain biases before shipping. We need to create a feedback loop for good design that allows lessons learned in the field to inform improvements to a product.