The potential to collect brain data more directly, with higher resolution, and in greater amounts has heightened worries about mental and brain privacy. In order to manage the risks to individuals posed by these privacy challenges, some have suggested codifying new privacy rights, including a right to “mental privacy.” In this paper, we consider these arguments and conclude that while neurotechnologies do raise significant privacy concerns, such concerns are—at least for now—no different from those raised by other well-understood data collection (...) technologies, such as gene sequencing tools and online surveillance. To better understand the privacy stakes of brain data, we suggest the use of a conceptual framework from information ethics, Helen Nissenbaum’s “contextual integrity” theory. To illustrate the importance of context, we examine neurotechnologies and the information flows they produce in three familiar contexts—healthcare and medical research, criminal justice, and consumer marketing. We argue that by emphasizing what is distinct about brain privacy issues, rather than what they share with other data privacy concerns, risks weakening broader efforts to enact more robust privacy law and policy. (shrink)

Privacy and surveillance scholars increasingly worry that data collectors can use the information they gather about our behaviors, preferences, interests, incomes, and so on to manipulate us. Yet what it means, exactly, to manipulate someone, and how we might systematically distinguish cases of manipulation from other forms of influence—such as persuasion and coercion—has not been thoroughly enough explored in light of the unprecedented capacities that information technologies and digital media enable. In this paper, we develop a definition of manipulation that (...) addresses these enhanced capacities, investigate how information technologies facilitate manipulative practices, and describe the harms—to individuals and to social institutions—that flow from such practices. -/- We use the term “online manipulation” to highlight the particular class of manipulative practices enabled by a broad range of information technologies. We argue that at its core, manipulation is hidden influence—the covert subversion of another person’s decision-making power. We argue that information technology, for a number of reasons, makes engaging in manipulative practices significantly easier, and it makes the effects of such practices potentially more deeply debilitating. And we argue that by subverting another person’s decision-making power, manipulation undermines his or her autonomy. Given that respect for individual autonomy is a bedrock principle of liberal democracy, the threat of online manipulation is a cause for grave concern. (shrink)

Since 2016, when the Facebook/Cambridge Analytica scandal began to emerge, public concern has grown around the threat of “online manipulation”. While these worries are familiar to privacy researchers, this paper aims to make them more salient to policymakers — first, by defining “online manipulation”, thus enabling identification of manipulative practices; and second, by drawing attention to the specific harms online manipulation threatens. We argue that online manipulation is the use of information technology to covertly influence another person’s decision-making, by targeting (...) and exploiting their decision-making vulnerabilities. Engaging in such practices can harm individuals by diminishing their economic interests, but its deeper, more insidious harm is its challenge to individual autonomy. We explore this autonomy harm, emphasising its implications for both individuals and society, and we briefly outline some strategies for combating online manipulation and strengthening autonomy in an increasingly digital world. (shrink)

The American justice system, from police departments to the courts, is increasingly turning to information technology for help identifying potential offenders, determining where, geographically, to allocate enforcement resources, assessing flight risk and the potential for recidivism amongst arrestees, and making other judgments about when, where, and how to manage crime. In particular, there is a focus on machine learning and other data analytics tools, which promise to accurately predict where crime will occur and who will perpetrate it. Activists and academics (...) have begun to raise critical questions about the use of these tools in policing contexts. In this chapter, I review the emerging critical literature on predictive policing and contribute to it by raising ethical questions about the use of predictive analytics tools to identify potential offenders. Drawing from work on the ethics of profiling, I argue that the much-lauded move from reactive to preemptive policing can mean wrongfully generalizing about individuals, making harmful assumptions about them, instrumentalizing them, and failing to respect them as full ethical persons. I suggest that these problems stem both from the nature of predictive policing tools and from the sociotechnical contexts in which they are implemented... (shrink)

For several years, scholars have (for good reason) been largely preoccupied with worries about the use of artificial intelligence and machine learning (AI/ML) tools to make decisions about us. Only recently has significant attention turned to a potentially more alarming problem: the use of AI/ML to influence our decision-making. The contexts in which we make decisions—what behavioral economists call our choice architectures—are increasingly technologically-laden. Which is to say: algorithms increasingly determine, in a wide variety of contexts, both the sets of (...) options we choose from and the way those options are framed. Moreover, artificial intelligence and machine learning (AI/ML) makes it possible for those options and their framings—the choice architectures—to be tailored to the individual chooser. They are constructed based on information collected about our individual preferences, interests, aspirations, and vulnerabilities, with the goal of influencing our decisions. At the same time, because we are habituated to these technologies we pay them little notice. They are, as philosophers of technology put it, transparent to us—effectively invisible. I argue that this invisible layer of technological mediation, which structures and influences our decision-making, renders us deeply susceptible to manipulation. Absent a guarantee that these technologies are not being used to manipulate and exploit, individuals will have little reason to trust them. (shrink)

The dominant approach in privacy theory defines information privacy as some form of control over personal information. In this essay, I argue that the control approach is mistaken, but for different reasons than those offered by its other critics. I claim that information privacy involves the drawing of epistemic boundaries—boundaries between what others should and shouldn’t know about us. While controlling what information others have about us is one strategy we use to draw such boundaries, it is not the only (...) one. We conceal information about ourselves and we reveal it. And since the meaning of information is not self-evident, we also work to shape how others contextualize and interpret the information about us that they have. Information privacy is thus about more than controlling information; it involves the constant work of producing and managing public identities, what I call “social self- authorship.” In the second part of the essay, I argue that thinking about information privacy in terms of social self- authorship helps us see ways that information technology threatens privacy, which the control approach misses. Namely, information technology makes social self- authorship invisible and unnecessary, by making it difficult for us to know when others are forming impressions about us, and by providing them with tools for making assumptions about who we are which obviate the need for our involvement in the process. (shrink)

The dominant legal and regulatory approach to protecting information privacy is a form of mandated disclosure commonly known as “notice-and-consent.” Many have criticized this approach, arguing that privacy decisions are too complicated, and privacy disclosures too convoluted, for individuals to make meaningful consent decisions about privacy choices—decisions that often require us to waive important rights. While I agree with these criticisms, I argue that they only meaningfully call into question the “consent” part of notice-and-consent, and that they say little about (...) the value of notice. We ought to decouple notice from consent, and imagine notice serving other normative ends besides readying people to make informed consent decisions. (shrink)

Automated influence, delivered by digital targeting technologies such as targeted advertising, digital nudges, and recommender systems, has attracted significant interest from both empirical researchers, on one hand, and critical scholars and policymakers on the other. In this paper, we argue for closer integration of these efforts. Critical scholars and policymakers, who focus primarily on the social, ethical, and political effects of these technologies, need empirical evidence to substantiate and motivate their concerns. However, existing empirical research investigating the effectiveness of these (...) technologies (or lack thereof), neglects other morally relevant effects—which can be felt regardless of whether or not the technologies "work" in the sense of fulfilling the promises of their designers. Drawing from the ethics and policy literature, we enumerate a range of questions begging for empirical analysis—the outline of a research agenda bridging these fields—and issue a call to action for more empirical research that takes these urgent ethics and policy questions as their starting point. (shrink)

The 2010s were a golden age of information privacy research, but its policy accomplishments tell a mixed story. Despite significant progress on the development of privacy theory and compelling demonstrations of the need for privacy in practice, real achievements in privacy law and policy have been, at best, uneven. In this chapter, I outline three broad shifts in the way scholars (and, to some degree, advocates and policy makers) are approaching privacy and social media. First, a change in emphasis from (...) individual to structural approaches. Second, increasing attention to the political economy of privacy—especially the business models of information companies, such as social media platforms. And third, a deeper understanding of privacy’s role in a healthy digital public sphere. (shrink)

Differential privacy (DP) aims to confer data processing systems with inherent privacy guarantees, offering strong protections for personal data. But DP’s approach to privacy carries with it certain assumptions about how mathematical abstractions will be translated into real-world systems, which—if left unexamined and unrealized in practice—could function to shield data collectors from liability and criticism, rather than substantively protect data subjects from privacy harms. This article investigates these assumptions and discusses their implications for using DP to govern data-driven systems. In (...) Parts 1 and 2, we introduce DP as, on one hand, a mathematical framework and, on the other hand, a kind of real-world sociotechnical system, using a hypothetical case study to illustrate how the two can diverge. In Parts 3 and 4, we discuss the way DP frames privacy loss, data processing interventions, and data subject participation, arguing it could exacerbate existing problems in privacy regulation. In part 5, we conclude with a discussion of DP’s potential interactions with the endogeneity of privacy law, and we propose principles for best governing DP systems. In making such assumptions and their consequences explicit, we hope to help DP succeed at realizing its promise for better substantive privacy protections. (shrink)

Our lives are guided by habits. Most of the activities we engage in throughout the day are initiated and carried out not by rational thought and deliberation, but through an ingrained set of dispositions or patterns of action—what Aristotle calls a hexis. We develop these dispositions over time, by acting and gauging how the world responds. I tilt the steering wheel too far and the car’s lurch teaches me how much force is needed to steady it. I come too close (...) to a hot stove and the burn I get inclines me not to get too close again. This feedback and the habits it produces are bodily. They are possible because the medium through which these actions take place is a physical, sensible one. The world around us is, in the language of postphenomenology, an opaque one. We notice its texture and contours as we move through it, and crucially, we bump up against it from time to time. The digital world, by contrast, is largely transparent. Digital media are designed to recede from view. As a result, we experience little friction as we carry out activities online; the consequences of our actions are often not apparent to us. This distinction between the opacity of the natural world and the transparency of the digital one raises important questions. In this chapter, I ask: how does the transparency of digital media affect our ability to develop healthy habits online? If the digital world is constructed precisely not to push back against us, how are we supposed to gauge whether our actions are good or bad, for us and for others? The answer to this question has important ramifications for a number of ethical, political, and policy debates around issues in online life. For in order to advance cherished norms like privacy, civility, and fairness online, we need more than good laws and good policies—we need good habits, which dispose us to act in ways conducive to our and others’ flourishing. (shrink)

Throughout the 20th century, market capitalism was defended on parallel grounds. First, it promotes freedom by enabling individuals to exploit their own property and labor-power; second, it facilitates an efficient allocation and use of resources. Recently, however, both defenses have begun to unravel—as capitalism has moved into its “platform” phase. Today, the pursuit of allocative efficiency, bolstered by pervasive data surveillance, often undermines individual freedom rather than promoting it. And more fundamentally, the very idea that markets are necessary to achieve (...) allocative efficiency has come under strain. Even supposing, for argument’s sake, that the claim was true in the early 20th century when von Mises and Hayek pioneered it, advances in computing have rekindled the old “socialist calculation” debate. And this time around, markets—as information technology—are unlikely to have the upper hand. -/- All of this, we argue, raises an important set of governance questions regarding the political economy of the future. We focus on two: How much should our economic system prioritize freedom, and to what extent should it rely on markets? The arc of platform capitalism bends, increasingly, toward a system that neither prioritizes freedom nor relies on markets. And the dominant critical response, exemplified by Shoshana Zuboff’s work, has been to call for a restoration of market capitalism. Longer term, however, we believe it would be more productive to think about how “postmarket” economic arrangements might promote freedom—or better yet, autonomy—even more effectively than markets, and to determine the practical steps necessary to realize that possibility. (shrink)

Advances in AI are powering increasingly precise and widespread computational propaganda, posing serious threats to national security. The military and intelligence communities are starting to discuss ways to engage in this space, but the path forward is still unclear. These developments raise pressing ethical questions, about which existing ethics frameworks are silent. Understanding these challenges through the lens of “cognitive security,” we argue, offers a promising approach.

Existing discussions about automated decision-making focus primarily on its inputs and outputs, raising questions about data collection and privacy on one hand and accuracy and fairness on the other. Less attention has been devoted to critically examining the temporality of decision-making processes—the speed at which automated decisions are reached. In this paper, I identify four dimensions of algorithmic speed that merit closer analysis. Duration (how much time it takes to reach a judgment), timing (when automated systems intervene in the activity (...) being evaluated), frequency (how often evaluations are performed), and lived time (the human experience of algorithmic speed) are interrelated, but distinct, features of automated decision-making. Choices about the temporal structure of automated decision-making systems have normative implications, which I describe in terms of "disruption," "displacement," "re-calibration," and "temporal fairness," with values such as accuracy, fairness, accountability, and legitimacy hanging in the balance. As computational tools are increasingly tasked with making judgments about human activities and practices, the designers of decision-making systems will have to reckon, I argue, with when—and how fast—judgments ought to be rendered. Though computers are capable of reaching decisions at incredible speeds, failing to account for the temporality of automated decision-making risks misapprehending the costs and benefits automation promises. (shrink)

In Husserl’s Missing Technologies, Don Ihde urges us to think deeply and critically about the ways in which the technologies utilized in contemporary science structure the way we perceive and understand the natural world. In this paper, I argue that we ought to extend Ihde’s analysis to consider how such technologies are changing the way we perceive and understand ourselves too. For it is not only the natural or “hard” sciences which are turning to advanced technologies for help in carrying (...) out their work, but also the social and “human” sciences. One set of tools in particular is rapidly being adopted—the family of information technologies that fall under the umbrella of “big data.” As in the natural sciences, big data is giving researchers in the human sciences access to phenomena which they would otherwise be unable to experience and investigate. And like the former, the latter thereby shape the ways those scientists perceive and understand who and what we are. Looking at two case studies of big data-driven research in the human sciences, I begin in this paper to suggest how we might understand these phenomenological and hermeneutic changes. (shrink)

This chapter provides a high-level overview of key ethical issues raised by the use of surveillance technologies, such as digital contact tracing, disease surveillance, and vaccine passports, to combat the COVID-19 pandemic. To some extent, these issues are entirely familiar. I argue that they raise old questions in new form and with new urgency, at the intersection of information ethics, research ethics, and public health. Whenever we deal with data-driven technologies, we have to ask how they fare in relation to (...) important values like privacy, fairness, transparency, and accountability—values emphasized by information ethics scholars. Likewise, when such technologies put individuals at risk in order to drive scientific research and knowledge construction, we have to ask how they implicate values such as autonomy, beneficence and non-maleficence, and justice—values central to research ethics. And as researchers focusing on health information have long argued, when the data collected by these technologies pertain to individuals and public health, these ethical issues take on a special cast. -/- It is also true, however, that the pandemic has placed these questions in a new and revealing light. I highlight three insights from information ethics and research ethics that can help us navigate this difficult terrain. First, the value of privacy is instrumental, not absolute—there is nothing wrong with asking how to balance privacy against other important values. Second, privacy has both individual and social importance. Weighing privacy, on one hand, and public health, on the other, is not, therefore, a contest between individual and collective interests. Rather, it is an attempt to balance disparate public goods. Third, we ought to put these kinds of ethical decisions in the hands of third parties, rather than leaving them up to those who directly stand to benefit from them. In the case of pandemic surveillance technologies, this should mean more public, democratic oversight. (shrink)

Surveillance studies scholars and privacy scholars have each developed sophisticated, important critiques of the existing data-driven order. But too few scholars in either tradition have put forward alternative substantive conceptions of a good digital society. This, I argue, is a crucial omission. Unless we construct new “sociotechnical imaginaries,” new understandings of the goals and aspirations digital technologies should aim to achieve, the most surveillance studies and privacy scholars can hope to accomplish is a less unjust version of the technology industry’s (...) own vision for the future. (shrink)

Public health scholars and public health officials increasingly worry about health-related misinformation online, and they are searching for ways to mitigate it. Some have suggested that the tools of digital influence are themselves a possible answer: we can use targeted, automated digital messaging to counter health-related misinformation and promote accurate information. In this commentary, I raise a number of ethical questions prompted by such proposals—and familiar from the ethics of influence and ethics of AI—highlighting hidden costs of targeted digital messaging (...) that ought to be weighed against the health benefits they promise. (shrink)

For those who find Dreyfus’s critique of AI compelling, the prospects for producing true artificial human intelligence are bleak. An important question thus becomes, what are the prospects for producing artificial non-human intelligence? Applying Dreyfus’s work to this question is difficult, however, because his work is so thoroughly human-centered. Granting Dreyfus that the body is fundamental to intelligence, how are we to conceive of non-human bodies? In this paper, I argue that bringing Dreyfus’s work into conversation with the work of (...) Mark Bickhard offers a way of answering this question, and I try to suggest what doing so means for AI research. (shrink)

The era of AI-based decision-making fast approaches, and anxiety is mounting about when, and why, we should keep “humans in the loop” (“HITL”). Thus far, commentary has focused primarily on two questions: whether, and when, keeping humans involved will improve the results of decision-making (making them safer or more accurate), and whether, and when, non-accuracy-related values—legitimacy, dignity, and so forth—are vindicated by the inclusion of humans in decision-making. Here, we take up a related but distinct question, which has eluded the (...) scholarship thus far: does it matter if humans appear to be in the loop of decision-making, independent from whether they actually are? In other words, what is stake in the disjunction between whether humans in fact have ultimate authority over decision-making versus whether humans merely seem, from the outside, to have such authority? Our argument proceeds in four parts. First, we build our formal model, enriching the HITL question to include not only whether humans are actually in the loop of decision-making, but also whether they appear to be so. Second, we describe situations in which the actuality and appearance of HITL align: those that seem to involve human judgment and actually do, and those that seem automated and actually are. Third, we explore instances of misalignment: situations in which systems that seem to involve human judgment actually do not, and situations in which systems that hold themselves out as automated actually rely on humans operating “behind the curtain.” Fourth, we examine the normative issues that result from HITL misalignment, arguing that it challenges individual decision-making about automated systems and complicates collective governance of automation. (shrink)