
==== Front
Sci Rep
Sci Rep
Scientific Reports
2045-2322
Nature Publishing Group UK London

39237635
71761
10.1038/s41598-024-71761-0
Article
Trust, trustworthiness and AI governance
Lahusen Christian 1
Maggetti Martino 2
Slavkovik Marija marija.slavkovik@uib.no

3
1 https://ror.org/02azyry73 grid.5836.8 0000 0001 2242 8751 Department of Social Sciences, Universität Siegen, 57068 Siegen, Germany
2 https://ror.org/019whta54 grid.9851.5 0000 0001 2165 4204 Université de Lausanne, Institute of Political Studies, CH-1015 Lausanne, Switzerland
3 https://ror.org/03zga2b32 grid.7914.b 0000 0004 1936 7443 Information Science and Media Studies, Universitetet I Bergen, 5007 Bergen, Norway
5 9 2024
5 9 2024
2024
14 2075218 3 2024
30 8 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by/4.0/ Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
An emerging issue in AI alignment is the use of artificial intelligence (AI) by public authorities, and specifically the integration of algorithmic decision-making (ADM) into core state functions. In this context, the alignment of AI with the values related to the notions of trust and trustworthiness constitutes a particularly sensitive problem from a theoretical, empirical, and normative perspective. In this paper, we offer an interdisciplinary overview of the scholarship on trust in sociology, political science, and computer science anchored in artificial intelligence. On this basis, we argue that only a coherent and comprehensive interdisciplinary approach making sense of the different properties attributed to trust and trustworthiness can convey a proper understanding of complex watchful trust dynamics in a socio-technical context. Ensuring the trustworthiness of AI-Governance ultimately requires an understanding of how to combine trust-related values while addressing machines, humans and institutions at the same time. We offer a road-map of the steps that could be taken to address the challenges identified.

Subject terms

Computer science
Computational science
University of Bergen (incl Haukeland University Hospital)Open access funding provided by University of Bergen.

issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

Artificial intelligence (AI) is now widely recognised as a socio-technical phenomenon that creates unprecedented opportunities while also possibly posing existential risks1,2. Overly positive or negative outlooks on the impact of AI tend to overestimate both the potential as well as the threat of artificial intelligence, but they are correct in highlighting the progressive incorporation of AI-based systems into all aspects of societal life3. This observation also applies to public authorities, which notably adopted AI because they see the merits of investing into technological innovation in order to increase the efficiency, effectiveness, and quality of services. Automation of decisions with the help of AI has thus been increasingly, and sometimes invisibly, introduced into core state activities such as public policy decision-making, internal management, and public service delivery in a wide range of areas4,5.

The relationships between public authorities and citizens are particularly affected by the growing reliance on AI automation, namely algorithmic decision-making (ADM)—software tools that use AI to aid and to make automated decisions. ADM is increasingly used to manage citizens’ applications and requests, identify personalized services, predict risks in regard to service provision, specify potential sanctions, and communicate decisions. ADM systems are thus having concrete effects in several domains, especially in—but not limited to—high-income countries. For instance, automation is being applied by public authorities to identify the eligibility for children’s allowances6,7, to select available employment offers for jobseekers8, to calculate social benefits9, to detect tax fraud10,11, and to forecast and monitor criminal behaviour12,13. The implications of these developments are considerable and need to be examined closely. In fact, public authorities are sole providers of service, particularly in sensitive areas of citizens’ lives, they have privileged access to personal data, take binding decisions, and might thus inescapably and directly affect beneficiaries and society as a whole.

The magnitude of the problem is determined by the fact that the growing insertion of AI into public service provision has ushered forms of AI-Governance, i.e., highly complex and dynamic socio-technical configurations that comprise technological, institutional, and regulatory elements. Specifically, they consist of:– ADM applications developed for public authorities;

– Institutional practices that define the purposes and forms of usages; and

– A regulatory framework that both enables and constrains ADM deployment.

Algorithms introduce a strong transformative element into these configurations, because they constitute an inscrutable, opaque system whose operations, choices, and consequences are still poorly understood. They might encourage or facilitate improper or malicious use by individuals and governments, e.g., for criminal or censoring purposes. Furthermore, they may yield unpredictably inaccurate or biased results, as shown by the case of Dutch tax authorities who used a self-learning algorithm for spotting child care benefits fraud, which proved to be biased against lower incomes and ethnic minorities, and violated privacy laws1.

The question then is how aligned is AI-governance with the values of the society that it serves and that sustains it—particularly core democratic values? In this regard, trust and trustworthiness plays a particularly important role, because trust is at the same time a precondition, a product, and a foundational ethical value for functioning human societies. Our claim is that work on AI-governance “trust-alignment” is lacking a profound multi-disciplinary understanding of how trust and trustworthiness should be conceived and they operate in a socio-technical context where the main agent of trust and the main bearer of trustworthiness is not human but an algorithm.

In this paper, we propose to focus on AI-Governance as the main object of inquiry. We argue that the question of trust and trustworthiness can only be properly addressed, when considering the interplay between its three components: AI applications, administrative practices, and regulatory systems. Such a research object calls for an interdisciplinary research agenda that transcends the limited insights of previous studies. We argue that wide gaps exist in-between the different research fields when it comes to trustworthy AI. Only an interdisciplinary collaboration, involving sociology, political science, and artificial intelligence, is thus able to generate the necessary synergies to address the challenges associated with the development of value-aligned AI-Governance. The aim of this paper is thus to provide a selective overview of the core literature in research fields involved in the analysis of AI-Governance. It wishes to engage computational science, sociology and political science into an interdisciplinary dialogue aimed at merging available evidence and synchronising research agendas. For this purpose, the paper pursues the following goals: to (1) justify the relevance of a study of trust and trustworthiness of AI-Governance; (2) provide an overview of the state of the art in the three research fields; and (3) discuss implications for future research.

Our literature review is a systematic effort to merge interrelated disciplinary research fields. We started by identifying and defining the trustworthiness of AI-governance as the core research question, and then we engaged in a comprehensive bibliographic search, article screening and synthesis of findings for the three components of the socio-technical system14. The literature review is wide-ranging, but not exhaustive, because we are dealing with early accumulation of knowledge in highly dynamic research field, where trust-related issues are increasingly updated. Moreover, relevant evidence is unequally developed, with more consistent efforts in the fields of the social sciences, when compared to computational science. The aim of our overview is to provide a pluralistic account of the main issues at stake. Our contribution thus is to build a “point of departure” for scientists that will engage with the trust and trustworthiness of AI-government.

Trust and trustworthiness

Finding the adequate definition of trust is an arduous undertaking, when considering the highly diverse interdisciplinary field relevant for the study of AI governance. A starting point for such a conceptualisation could be the proposal to understand trust as “the willingness of a party to be vulnerable to the actions of another party”15, be this other party another human, an institution or a machine. However, the agreement would only subsist under the condition that trust is related to human trustors, that is, when conceiving trust as a human property. Conceptual junctures emerge when considering that trust in humans, institutions and machines are categorically distinct. Additionally, it is necessary to remember that conceptual disagreements also run across disciplines, thus contributing to interdisciplinary discrepancies and disagreements. In fact, scholars diverge conceptually when relating trust to either behaviors, attitudes or relations16–19. Anchoring trust in behavioral practices and norms, in perceptions and preferences and/or in functional or formal properties of interactions between people, institution and/or machines has considerable implications for the way how trust is conceptualized, operationalized, and analyzed.

An interdisciplinary approach to the study of trust in AI governance should thus abstain from an overhasty synthesis of available evidence. This is also true for the focus of this collection, namely AI alignment to human and societal values. Trust is in itself a value, when defining a social value as an ethical conception of what is a desirable good within a given society. Scholars in various research fields would converge in this assessment, as trust is widely considered to be an important precondition and ingredient of societal cohesion, political stability, economic development, and/or technological innovation20–24.

However, societal values are intrinsically plural, dynamic and context-dependent, which means that even trust is a conditional posture. “We learn that, tentatively and conditionally, we can trust trust and distrust distrust, that it can be rewarding to behave as if we trusted even in unpromising situations”25. Unconditional trust can transcend what we conceive to be socially desirable, while conditional distrust might be a value in itself—when voiced through appropriate channels—because it may serve to prevent misbehaviour and keep social relations or societal systems in check26–28. Consequently, finding a careful balance between trust and distrust is crucial to ensure that trust is placed in those who deserve trust, whilst avoiding to place blind trust in the untrustworthy.

The alignment of AI to societal values has thus to consider the pluralism, dynamism and situatedness of values. This is particularly true with respect to the use of AI systems by public authorities, which are still opaque and largely unaccountable but potentially deploy effects erga omnes. In this regard, the number of relevant values increases, when considering the specificities of humans, institutions and machines as ‘trustworthy’ trustees. Scholars in the various research fields have engaged identifying those values that determine the attribution of trustworthiness, and the list of values that are considered to be relevant exhibit both, similarities and dissimilarities. In regard to interpersonal trust, for instance, it has been highlighted that competence, predictability, benevolence and integrity are crucial values that help to ascertain a trustee’s trustworthiness29,30. With respect to public institutions the list includes values pertinent for the qualities (e.g., competence, reliability, democratic participation), procedures (e.g., transparency, accountability) and results (e.g., effectiveness, general welfare, justice) of political-administrative work31. In regard to AI systems, a number of properties have been identified as trust-relevant, such as reliability, robustness, safety, interpretability, explainability, fairness, transparency, and accountability. The existence of partially overlapping but also varying lists and typologies of values demonstrates that AI governance as a socio-technical system has a series of distinct, yet interlocked problems of trustworthiness: there are doubts about the trustworthiness of processes and outputs of artificial intelligence, about the trustworthiness of humans and public authorities to properly handle this technology, and about the trustworthiness of governments to properly regulate this area of innovation. These problems of trustworthiness are, however, not necessarily identical, possibly complementary but maybe even incompatible. And this means that value alignment cannot rely on a simple list of shared values, but requires a careful analysis that helps to ascertain the specifics and interfaces between the various components of AI governance and their problems of trustworthiness.

Within computational science, the trustworthiness of AI is taken as a long-term goal and under that umbrella, in recognition of the complexity of the problem, several distinguished networks have been established. This, however, does not mean that computer scientists have explored the concept of trust and use this to guide the qualification and quantification of computational and interaction properties associated with trust and trustworthiness of AI. Topics studied with respect to trust and trustworthiness in the AI field pertain to issues as diverse as reliability, robustness, safety, security, bias, interpretability, privacy, explainability, fairness, transparency, resilience, and accountability32,33. For computer scientists it can be very hard to find a good point from which to learn about and engage with the conceptualisation of trust as studied in the social sciences, as this research is typically not part of Trustworthy AI agendas in computer science.

Algorithmic decision-making (ADM), why trust matters

“Algorithmic decision-making” (ADM) is defined by the European parliamentary research service34 as computational systems that rely on the analysis of large amounts of personal data, usually using deep learning to infer correlations or, more generally, to derive information deemed useful to making decisions.

Deep learning re-energised neural network research in the early 2000s due to some fundamental research breakthroughs. Since then, Neural Networks (NN) have proved to be a useful tool for modelling complex phenomena. In the past decade, these networks have been at the core of widely diffused AI used for image recognition, natural language processing and, most recently, the generation of content35.

The ADM as it typically happens is illustrated in Fig. 1, where the decision instance is exemplified by two human figures, emphasizing that decisions are made about the people directly affected by them. The ADM system may be used to aid a human in making decisions, as shown by arrow b, by offering a comprehensive analysis of a large volume of data. Alternatively, the human level of intervention in the decision-making process may vary, as shown by arrow a, not being made by institution representatives but directly supplied to beneficiaries by the ADM system. Figure 1 uses a pink brain to illustrate human intervention in- or on-the-loop36,37 of the decision-making.Fig. 1 An algorithmic decision-making/-aiding system.

The European union general data protection regulation (GDPR)38 uses the term “automated decision-making” to refer to decisions being made without human involvement. Sometimes in the literature, both algorithmic and automated decision making are used as synonyms. For clarity, we should also state that we define an algorithm as “an unambiguous procedure to solve a problem or a class of problems. It is typically composed of a set of instructions or rules that take some input data and return outputs”34.

What separates NN artifacts from previous AI applications is their inbuilt incorrectness of output. Automated decisions that are obtained by utilising NN models can be assessed for accuracy, but unlike other non-AI computation, we do not have a mathematical guarantee of correctness. This is one of the main reasons why the intrinsic (or endogenous) trustworthiness of AI becomes such a dominant concern with the rising use of AI. Notwithstanding these limitations, the automation of tasks in the public sector using algorithmic decision-making is relatively widespread in public sector organisations39–42.

The use of ADM increasingly shapes core government functions, namely public service delivery, internal management, and policy decision making. In this regard, a number of challenges are often mentioned, including the variance in acceptance, privacy concerns, sample biases, discrimination, surveillance or the digital divide43–47. What is more, ADM implementation is complex and largely dependent on institutional practices and contextual factors5. These topics are considered to be urgent, because the automation of routines and decisions by public authorities deeply affect individuals, for instance, when they apply for social benefit, submit tax declarations, seek for health services or have violated the law. When a decision is made with the use of AI, public authorities insert the AI in the middle of the relationship between citizens and state institutions, making its (endogenous and exogenous) trustworthiness crucial. Due to accelerating technological developments, the trust relationship that holds between the institution and the individual needs to be re-built when including the AI system and its usage.

An overview of individual disciplines

The insertion of ADM into the relationship between citizens and public authorities has aroused considerable research efforts48–50. Elements of these are devoted to trustworthiness issues and aim to settle a number of questions: Can AI become trustworthy and how? Can and should citizens and authorities trust a non-human agent and its usage by humans? Which forms of use by citizens and authorities are adequate? Which regulatory policies are in place, and which regulatory approach could be established by public authorities to fruitfully mediate adequate trust relationships unfolding between developers, adopters in the public sectors, and citizens as beneficiaries, with AI/ADM systems in the middle? The three disciplines, computer science, sociology, political science, have contributed considerably to these debates by providing evidence about the three pillars on which the topic resides (AI, society, political governance).

In the following sections we give an overview of the trustworthy AI literature in these three fields.

Overview: citizens

The nexus between AI and society has become an increasingly important topic of analysis in the social sciences. Generally speaking, research has abandoned assumptions of binary relationships between distinct spheres (technology and society), acknowledging that AI is inherently embedded in social reality51,52. Scholars thus tend to speak of complex socio-technological systems that mirror established structures of social reality: machine-learning systems use informational and behavioural data generated by individual, corporate and state actors, who very often use decision-making models to replicate standard probabilistic reasoning widely used by humans, and the impact of machine-learning systems on society depends on the way ADM is used by humans or organisations53. Social sciences have thus painted a nuanced picture of the relation of AI and society, arguing that ADM is transforming but also reproducing societal reality47.

The topics of investigation are as diverse as the implications of artificial intelligence for social reality (e.g., work and leisure, news coverage and media usage, welfare services and public safety). But in regard to AI-governance, three topics are of paramount importance.

First, research has been interested in the perceptions and attitudes of citizens in regard to AI48,54,55. In spite of the categorical difference in trusting humans and machines, findings show an increasing readiness within society to assess AI in regard to trustworthiness, thus mirroring the growing relevance of AI-based applications in individual’s lives. Not only does trust vary depending on the respondents’ characteristics (e.g., age, gender, competence, education, personality traits, political orientations), but also regarding institutional and contextual factors (e.g., the countries’ socio-economic and cultural background)55–59. AI systems and the AI’s perceived characteristics (e.g., performance, reliability, anthropomorphism, rule-based operation) are crucial, as well48,54. Research thus suggests that citizens are moderately and conditionally trustful, which is considered positive, because blind trust in untrustworthy socio-technological systems is, in itself, highly problematic60.

A second strand of research centres more explicitly on the effects AI systems may have on the readiness of citizens to trust political institutions. Earlier studies of public trust in e-Government have already shown that the use of ICT and digitalisation can increase a citizen’s perception of institutional trustworthiness. However, trust remains highly conditional and depends on the general propensity to trust, the perception of the quality and usefulness of services, and risk sensitivity, particularly in regard to data privacy and security61,62. Digitalisation may also increase public perception of institutional efficiency and transparency, but not necessarily the recognition of institutional fairness, procedural or distributional justice63. The implementation of AI systems seems to exacerbate these conditionalities, because ADM systems are being adopted primarily for efficiency and effectivity reasons. At the same time, however, trust in the use of ADM by public authorities greatly depends on the specific areas of application64and the way they are implemented and used65.

A third research debate corroborates that conditionality of trust is an adequate public response to the application of ADM systems. ADM systems and their usage have been shown to generate problematic societal consequences, primarily by reproducing, even enlarging, existing social inequalities and power asymmetries47,53,66. This is particularly the case when ADM is used in contexts of public services, as has been shown in regard to algorithmic profiling in the context of public employment services67. Against this backdrop, issues of fairness and justice have received particular attention by social scientists when assessing ADM68. Before this backdrop, they (social scientists) were asked whether public acceptability of ADM systems might be influenced by fairness perceptions. Findings paint an inconclusive picture, as automated decision-making is evaluated on a par with human decision-making, sometimes even better43,67,69,70. Problems are associated with ADM design, but also with the institutional context of its applications, thus furthering discomfort, and in part, also contestation71.

Overview: governance

In recent years, concerns about regulation and governance by and of algorithms have emerged as a major challenge for both scholars and policy makers. Public authorities—especially in, but not limited to, high-income countries-increasingly rely on AI and specifically on ADM systems to inform, assist, and make decisions; in that regard, for instance, OECD reports have documented the remarkable extent to which countries are integrating AI into public administration72. At the same time, governments also face urgent calls to regulate these systems73–76.

Increasing worries are indeed propagated by many social and political actors about a wide range of unwanted impacts of AI, and particularly of ADM, that threaten to erode essential collective goods, including democracy, solidarity, justice, privacy, and individual freedom77,78. At the same time, great expectations are formulated about the beneficial “disruptive” innovation potential of trustworthy ADM in countless areas, ranging from health to transportation and finance45 (see79 for a critique). In this sense, a trade-off may emerge between the efficiency gains associated with the deployment of AI and room for manoeuvre of public authorities to create and implement rules to protect the public good80. In that respect, key issues identified in the literature involve the need to ensure AI accountability by choosing transparent, interpretable models over black-box alternatives81,82 and the focus on governing the use, instead of the technology itself by discouraging and punishing the abuse of AI83. It has also been noted that unless specifically regulated, de facto rules by the private sector84 will govern the development and use of AI. Public regulation is nonetheless essential to ensure control over AI and, specifically, ADM systems and secure their trustworthiness82,85–87.

Different regulatory approaches are being developed, adopted, and tentatively implemented, all of which center on public trust88,89. The EU, in the wake of the AI Act, relies on comprehensive framework inspired by the precautionary principle, which imposes strict requirements on high-risk applications. The GDPR also has significant implications for AI, especially concerning data privacy and protection, aiming at ensuring that AI systems comply with stringent data handling and user consent requirements. For instance, in digital health, this does not only safeguards patient privacy but also builds public confidence in the use of AI for health purposes. The US approach, following the Blueprint for an AI Bill of Rights, is more fragmented and sector-specific, with different agencies overseeing AI use in various areas, while maintaining a conducive environment for innovation and technological advancement. The UK is developing a pro-innovation regulatory framework, whose goal is to position the country as a global AI leader, by relying on adaptive and principles-based regulation.

However, it is crucial to recognise that the current political and societal discourses on building trust in AI often fail to recognise that trust in AI is not unconditionally beneficial, and that focusing exclusively on how to build public trust in AI systems may obscure the questions about what might be required to actually build trustworthy systems60, namely because, as we have seen above, AI systems intrinsically operate in a state of limited trustworthiness. Rather than blind trust, a certain measure of functional distrust may indeed help to secure the long-term trustworthiness of AI and ADM systems, which bears the highest potential for transformative change, but also the highest risk of intrusiveness into people’s private life, especially when it is deployed by public authorities themselves. In this case, the regulation of algorithmic decision-making systems is particularly challenging, as public authorities take both the role of rule-maker and regulatory target90.

Given that trust is a relational property, the trustworthiness of AI depends not only on AI itself, but on its complex interplay with the main actors involved in the operation of AI systems, namely: developers, public authorities that adopt ADM systems, and citizens/users as beneficiaries. These complex interactions involve further dilemmas. Developers in the IT sector program, maintain and implement AI systems, and at the same time, their work can be both facilitated and challenged by AI. Public authorities (including regulators and intermediaries) face calls to regulate AI while already heavily relying on algorithmic systems to assist decision-making, and being confronted with a regulatory issue that is global in scale91,92. Citizens can be empowered by AI, and benefit from countless new services and opportunities, but biases, errors, new cleavages, social exclusion, surveillance and privacy protection concerns also loom large.

Overview: computer science

The question that computer science is concerned with when it comes to trust and trustworthiness of AI in ADM specifically, is which technical tools and properties are relevant or should be developed in this context to make systems “endogenously” (or intrinsically) trustworthy.

What trusting a non-human agent means has been explored within the field of human-computer and human-robot interaction, see for e.g.93,94. People appear to have different expectations from machines compared to their expectations directed to other people, regardless of how intelligent those machines appear to be93. The trustworthiness that is attached to a non-human agent has a multidimensional structure that involves being capable, ethical, sincere, and reliable32,95.

Trust and trustworthiness have been extensively used in relation to Artificial Intelligence to signal the need for adequately embedding AI in a socio-technical society96. Chatila et al.32 ground trustworthiness of AI in general into delivery of service that can be justifiably trusted, and define it to be one that entails the following properties: availability, reliability, safety, confidentiality, integrity, maintainability, and security. Within each of these sub-topics, thousands of research papers with technical contributions are produced each year and published at computer science venues. However, although there is much research work under the keywords of trustworthy AI, not all of it discusses the subject33, making it difficult to ascertain where researchers are making progress.

Transparency is seen as quintessential to trust97,98, but at the same time, it is also recognised that transparency is not a “silver bullet”2,99 nor is it clear how it can be operationalised into more trustworthy AI. In the context of algorithmic decision-making, explainability, in particular, is a very powerful tool in facilitating trust in the process that involves AI100. The field of explainable artificial intelligence (XAI) is sometimes thought to subsume the work on transparency59,101 and sometimes to be a tool for transparency98. The field of explainable AI, specifically the moniker XAI, originates in a 2017 DARPA initiative102, with some very influential papers like73 beginning to appear in 2016. The field is thus very recent, but also very active. There are numerous systematic analyses already59,103 and even a systematic analysis of systematic analyses104 Work in XAI can range from developing technical tools like105 to meta discussions on what explainability should focus on, for example106.

Algorithmic decision-making is of particular concern to the field of algorithmic fairness, which specifically focuses on fairness in decisions made using machine learning. Algorithmic fairness is as new and as fruitful as XAI4,107–109. It studies problems of assessing the societal and individual impact of decisions made or aided by machine learning algorithms with the aim of ensuring that individuals (individual fairness) or protected groups (group fairness) are not discriminated against. Beyond assessment, algorithmic fairness is also concerned with developing methods to remove representational bias from data sets used to train prediction models, and methods that correct for allocational bias directly in the decision-making algorithms.

Necessarily, algorithmic fairness also studies possible trade-offs between a more efficient, more accurate and more “fair” decision algorithm. GDPR article 538 requires that ‘personal data shall be: 1. processed lawfully, fairly and in a transparent manner in relation to the data subject (‘lawfulness, fairness and transparency’)”. Understandably, it would be practical to have a computational tool that automatically assesses the fairness of an ADM and the data used for it and by it. There are, at present, numerous proposed mathematical specifications of fairness108,109 that are not always grounded in the corresponding ethical desiderata101,110. Article 21 of the GDPR38 stipulates the right to object to processing of personal data, and the right to appeal a decision. The possibility for objection should be afforded by the computational design of the ADM111–113, and it is our task to create that affordance. Contestable AI is a very new field that studies how to make AI systems open and responsive to human intervention throughout their lifecycle, not only after an automated decision has been made114.

Discussion

Dealing with the complexities of trust alignment for AI-governance requires joint efforts. On the one hand, the issue of trust is of paramount relevance with respect to the quick development of AI systems. AI-Governance is affected by substantial changes located at the level of technological advances, administrative usages and regulatory policies, whose proper functioning requires the presence of trust among the various actors involved, specifically including developers, administrative users, or regulators. Public administrators and regulators need to place trust in the ability of developers to generate trustworthy AI systems that are appropriate for the foreseen administrative usages and comply with regulatory goals and instruments. Similarly, developers need to trustfully rely on public authorities and regulators to properly employ, monitor and steer the technology in order to capitalize on its potentials while limiting downsides. For this purpose, it is important to develop sound scientific evidence to identify the necessary properties of AI systems, administrative usages, and regulations that sustain and enhance mutual trust, and to demonstrate how the features of trustworthy AI, usages and regulators are deeply intertwined. At the same time, interdisciplinary efforts are needed to determine the limits of trust and the conditions under which a watchful attitude could emerge and be maintained, as well as the mechanisms allowing for this vigilance to deploy. It is indeed only through the combination between trust and a watchful attitude (watchful trust), which implies a certain level of functional distrust, rather than blind trust, that developers, administrative users, and regulators can anticipate and tame potential risks and harms emanating from technological developments, administrative practices and regulatory measures. In particular, we identify four main challenges that require future interdisciplinary research efforts.

First, the mushrooming research efforts have contributed to a significant fragmentation of insights. There is an inflationary use of the concept of trust and trustworthiness33, and little theoretical clarity in regard to trust in AI, institutions and humans. This is complemented by a plethora of empirical measurements of trust in AI, institutions and humans, which makes almost impossible to systematically compare the evidence and produce cumulative knowledge.

Second, studies have provided insights into the acceptability of AI in general, and about the usage of ADM in institutional contexts. However, research is dominated by single case studies and lab experiments that say little about long-term dynamics and real-life settings, and are blind with regard to situational and contextual conditions. We thus know very little about whether and how ADM systems and practices contribute to the formation of trust and distrust. Additionally, the dominance of single case or country-specific analyses have led to inconclusive findings.

Thirdly, notwithstanding early attempts, which are fruitful but mainly set out to explore and scope the field83,92, research has yet to systematically develop an explicit research agenda addressing the regulatory challenges and responses to the technological transformations spurred by ADM systems and their use.

Fourth, research has not yet provided a consistent answer to the question of whether ADM systems and their utilisation can and should be trustworthy. There is an implicit agreement that blind trust is, generally speaking, inadequate posture in complex technological, social and political contexts, inasmuch as disconsolate distrust will be harmful.

In conclusion, to make substantial progress on the study AI value alignment, we need to first understand values, not only from the perspective of humans and human society, but also—and above all—from the perspective of machines, and how such values are intertwined and possibly interact.

Conclusion

Considerable progress has been made in recent years in deepening our knowledge about trustworthy AI governance. This is due to the increasing societal and scientific relevance of AI, the growing need to provide robust knowledge, and intensifying research efforts in a number of research fields. However, we still see limits in our understanding of trust and the trustworthiness of AI governance.

We identify four challenges that require future interdisciplinary research efforts. “Interdisciplinary” is always desirable but not trivial to achieve. To overcome the fragmentation of insights (first challenge) we need to lower the barrier to understanding across researchers who study citizens, governance and AI on one end and those who “create” AI, on the other end. This may be accomplished by providing a simplified “interface” to a specific discipline: identifying the critical ideas, research programs, and accomplishments in each of the concerned research areas. This article is intended to be a “prototype” of such “interface” work.

A successfully discipline interfacing will enable empirical and theoretical framework of analysis that helps to identify adequate forms of trustworthy ADM systems, societal usages, and political regulations grounded on the concept of watchful trust. This work needs to be aligned with a systematic, comparative research agenda that empirically examines how the relevance of the trust and trustworthiness properties may vary, and how cross-sectional, longitudinal, and, respectively, contextual factors shape and condition different levels and forms of trust. Lastly, regulatory regimes are currently being developed from the perspective of regulation and governance of AI. This development needs to be aligned with and investigation of how they address the problem of AI trustworthiness (or fail to do so), and, in turn, how these embryonic regimes reshape trust and distrust in AI and in ADM.

In conclusion, to make substantial progress on the study of AI value alignment, we need to first understand values, not only from the perspective of humans and human society, but also—and above all—from the perspective of machines, and how such values are intertwined and possibly interact. Values that unequivocally require understanding from the perspective of machines are trust and trustworthiness. Progress in AI Governance depends on this understanding.

Author contributions

All authors contributed equally to this work. State of the art computer science was written by Slavkovik. State of the art governance was written by Maggetti. State of the art citzens was written by Lahusen.

Funding

Open access funding provided by University of Bergen.

Data availability

This study does not use datasets. The articles datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Competing interests

The authors declare no competing interests.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

These authors contributed equally: Christian Lahusen, Martino Maggetti and Marija Slavkovik.
==== Refs
References

1. Heikkilä, M. & Heikkilä, M. Dutch scandal serves as a warning for Europe over risks of using algorithms. POLITICO (2022). https://www.politico.eu/article/dutch-scandal-serves-as-a-warning-for-europe-over-risks-of-using-algorithms/.
2. Knowles, B., Fledderjohann, J., Richards, J. T. & Varshney, K. R. Trustworthy ai and the logics of intersectional resistance. In Proc. 2023 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’23, 172–182, (Association for Computing Machinery, USA, 2023).
3. Jiang Y Li X Luo H Quo vadis artificial intelligence? Discov. Artif. Intell. 2022 10.1007/s44163-022-00022-8
Jiang, Y. et al. Quo vadis artificial intelligence?. Discov. Artif. Intell.10.1007/s44163-022-00022-8 (2022).10.1007/s44163-022-00022-8
4. Kearns M Roth A The Ethical Algorithm: The Science of Socially Aware Algorithm Design 2019 Oxford University Press
Kearns, M. & Roth, A. The Ethical Algorithm: The Science of Socially Aware Algorithm Design (Oxford University Press, 2019).
5. Neumann O Guirguis K Steiner R Exploring artificial intelligence adoption in public organizations: A comparative case study Pub. Manag. Rev. 2023 26 114 141 10.1080/14719037.2022.2048685
Neumann, O., Guirguis, K. & Steiner, R. Exploring artificial intelligence adoption in public organizations: A comparative case study. Pub. Manag. Rev. 26, 114–141 (2023).10.1080/14719037.2022.2048685
6. Brown, A., Chouldechova, A., Putnam-Hornstein, E., Tobin, A. & Vaithianathan, R. Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services. In Proc. 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, 1–12, (Association for Computing Machinery, New York, NY, USA, 2019).
7. Chouldechova, A., Benavides-Prado, D., Fialko, O. & Vaithianathan, R. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Proc. 1st Conference on Fairness, Accountability and Transparency, vol. 81, 134–148 (Proceedings of Machine Learning Research, 2018).
8. Flügge, A. A. Perspectives from practice: Algorithmic decision-making in public employment services. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing, CSCW ’21, 253–255, 10.1145/3462204.3481787 (Association for Computing Machinery, New York, NY, USA, 2021).
9. Sarlin R Suksi M Automationin administrative decision-makingconcerningsocialbenefits: A government agency perspective The Rule of Law and Automated Decision-Making 2023 Springer
Sarlin, R. Automationin administrative decision-makingconcerningsocialbenefits: A government agency perspective. In The Rule of Law and Automated Decision-Making (ed. Suksi, M.) (Springer, 2023).
10. Asquith, R. Tax authorities adopt AI for tax fraud and efficiencies-vatcalc.com. Section: Artificial Intelligence (2023).
11. de la Feria R Grau Ruiz MA Grau A The Robotisation of tax administration Interactive Robotics: Legal, Ethical, Social and Economic Aspects 2022 Springer Nature
de la Feria, R. & Grau Ruiz, M. A. The Robotisation of tax administration. In Interactive Robotics: Legal, Ethical, Social and Economic Aspects (ed. Grau, A.) (Springer Nature, 2022).
12. Mugari I Obioha EE Predictive policing and crime control in the United States of America and Europe: Trends in a decade of research and the future of predictive policing Soc. Sci. 2021 10 234 10.3390/socsci10060234
Mugari, I. & Obioha, E. E. Predictive policing and crime control in the United States of America and Europe: Trends in a decade of research and the future of predictive policing. Soc. Sci. 10, 234. 10.3390/socsci10060234 (2021).10.3390/socsci10060234
13. Van Brakel R How to watch the watchers? Democratic oversight of algorithmic police surveillance in Belgium Surveill. Soc. 2021 19 228 240 10.24908/ss.v19i2.14325
Van Brakel, R. How to watch the watchers? Democratic oversight of algorithmic police surveillance in Belgium. Surveill. Soc. 19, 228–240. 10.24908/ss.v19i2.14325 (2021).10.24908/ss.v19i2.14325
14. Nowell L Interdisciplinary mixed methods systematic reviews: Reflections on methodological best practices, theoretical considerations, and practical implications across disciplines Soc. Sci. Humanit. Open 2022 6 100295 10.1016/j.ssaho.2022.100295
Nowell, L. et al. Interdisciplinary mixed methods systematic reviews: Reflections on methodological best practices, theoretical considerations, and practical implications across disciplines. Soc. Sci. Humanit. Open 6, 100295. 10.1016/j.ssaho.2022.100295 (2022).10.1016/j.ssaho.2022.100295
15. Mayer RC Davis JH Schoorman FD An integrative model of organizational trust Acad. Manag. Rev. 1995 20 709 734 10.2307/258792
Mayer, R. C., Davis, J. H. & Schoorman, F. D. An integrative model of organizational trust. Acad. Manag. Rev. 20, 709–734 (1995).10.2307/258792
16. Schilke O Reimann M Cook KS Trust in social relations Annu. Rev. Sociol. 2021 47 239 259 10.1146/annurev-soc-082120-082850
Schilke, O., Reimann, M. & Cook, K. S. Trust in social relations. Annu. Rev. Sociol. 47, 239–259. 10.1146/annurev-soc-082120-082850 (2021).10.1146/annurev-soc-082120-082850
17. Zmerli S Maggino F Political trust Encyclopedia of Quality of Life and Well-Being Research 2020 Springer International Publishing
Zmerli, S. Political trust. In Encyclopedia of Quality of Life and Well-Being Research (ed. Maggino, F.) (Springer International Publishing, 2020).
18. Nguyen CT Nguyen CT Trust as an unquestioning attitude Oxford Studies in Epistemology 2022 Oxford University Press
Nguyen, C. T. Trust as an unquestioning attitude. In Oxford Studies in Epistemology (ed. Nguyen, C. T.) (Oxford University Press, 2022).
19. Luhmann N Trust and Power 1979 Wiley
Luhmann, N. Trust and Power (Wiley, 1979).
20. Rodriguez-Soto M Serramia M Lopez-Sanchez M Rodriguez-Aguilar JA Instilling moral value alignment by means of multi-objective reinforcement learning Eth. Inf. Technol. 2022 24 9 10.1007/s10676-022-09635-0
Rodriguez-Soto, M., Serramia, M., Lopez-Sanchez, M. & Rodriguez-Aguilar, J. A. Instilling moral value alignment by means of multi-objective reinforcement learning. Eth. Inf. Technol. 24, 9. 10.1007/s10676-022-09635-0 (2022).10.1007/s10676-022-09635-0
21. Arnold, T., Kasenberg, D. & Scheutz, M. Value alignment or misalignment - what will keep systems accountable? In The Workshops of the The Thirty-First AAAI Conference on Artificial Intelligence, Saturday, February 4–9, 2017, San Francisco, California, USA, vol. WS-17 of AAAI Technical Report (AAAI Press, 2017).
22. Gabriel I Artificial intelligence, values, and alignment Minds Mach. 2020 30 411 437 10.1007/s11023-020-09539-2
Gabriel, I. Artificial intelligence, values, and alignment. Minds Mach. 30, 411–437. 10.1007/s11023-020-09539-2 (2020).10.1007/s11023-020-09539-2
23. Sutrop M Challenges of aligning artificial intelligence with human values Acta Balt. Hist. Et Philos. Sci. 2020 8 54 72 10.11590/abhps.2020.2.04
Sutrop, M. Challenges of aligning artificial intelligence with human values. Acta Balt. Hist. Et Philos. Sci. 8, 54–72. 10.11590/abhps.2020.2.04 (2020).10.11590/abhps.2020.2.04
24. Hirschman AO Against parsimony: Three easy ways of complicating some categories of economic Am. Econ. Rev. 1984 74 89 96 10.1017/S0266267100001863
Hirschman, A. O. Against parsimony: Three easy ways of complicating some categories of economic. Am. Econ. Rev. 74, 89–96. 10.1017/S0266267100001863 (1984).10.1017/S0266267100001863
25. Gambetta D Gambetta D Can we trust trust? Making and Breaking Cooperative Relations 1988 Basil Blackwell
Gambetta, D. Can we trust trust? In Making and Breaking Cooperative Relations (ed. Gambetta, D.) (Basil Blackwell, 1988).
26. Lewicki RJ McAllister D Bies RJ Trust and distrust: New relationships and realities Acad. Manag. Rev. 1998 23 438 458 10.2307/259288
Lewicki, R. J., McAllister, D. & Bies, R. J. Trust and distrust: New relationships and realities. Acad. Manag. Rev. 23, 438–458 (1998).10.2307/259288
27. Sztompka P Trust distrust and two paradoxes of democracy Eur. J. Soc. Theory 1998 1 19 32 10.1177/136843198001001003
Sztompka, P. Trust distrust and two paradoxes of democracy. Eur. J. Soc. Theory 1, 19–32. 10.1177/136843198001001003 (1998).10.1177/136843198001001003
28. Warren M Uslaner E Trust and democracy The Oxford Handbook on Social and Political Trust 2018 Oxford University Press
Warren, M. Trust and democracy. In The Oxford Handbook on Social and Political Trust (ed. Uslaner, E.) (Oxford University Press, 2018).
29. Bacharach M Gambetta D Cook KS Trust in signs Trust in Society 2001 Russell Sage Foundation
Bacharach, M. & Gambetta, D. Trust in signs. In Trust in Society (ed. Cook, K. S.) (Russell Sage Foundation, 2001).
30. Lewicka D Zakrzewska-Bielawska AF Trust and distrust in interorganisational relations—Scale development PLoS ONE 2022 10.1371/journal.pone.0279231 36525450
Lewicka, D. & Zakrzewska-Bielawska, A. F. Trust and distrust in interorganisational relations—Scale development. PLoS ONE10.1371/journal.pone.0279231 (2022).36525450 10.1371/journal.pone.0279231
31. Levi M Stoker L Political trust and trustworthiness Annu. Rev. Polit. Sci. 2000 3 475 507 10.1146/annurev.polisci.3.1.475
Levi, M. & Stoker, L. Political trust and trustworthiness. Annu. Rev. Polit. Sci. 3, 475–507. 10.1146/annurev.polisci.3.1.475 (2000).10.1146/annurev.polisci.3.1.475
32. Chatila R Braunschweig B Ghallab M Trustworthy ai Reflections on Artificial Intelligence for Humanity 2021 Springer International Publishing
Chatila, R. et al. Trustworthy ai. In Reflections on Artificial Intelligence for Humanity (eds Braunschweig, B. & Ghallab, M.) (Springer International Publishing, 2021).
33. Probasco, E. S., Toney, A. S. & Curlee, K. T. The Inigo Montoya problem for trustworthy AI. The use of keywords in policy and research. Technical Report Center Security and Emerging Technologies. (2023). https://cset.georgetown.edu/publication/the-inigo-montoya-problem-for-trustworthy-ai/.
34. Castelluccia, C. & Le Métayer, D. Understanding algorithmic decision-making: Opportunities and challenges https://www.europarl.europa.eu/thinktank/en/document/EPRS_STU(2019)624261 (European Union, Brussels, 2019).
35. Bengio Y Lecun Y Hinton G Deep learning for AI Commun. ACM 2021 64 58 65 10.1145/3448250
Bengio, Y., Lecun, Y. & Hinton, G. Deep learning for AI. Commun. ACM 64, 58–65. 10.1145/3448250 (2021).10.1145/3448250
36. Christopher S Calhoun JJG Philip B Lyons JB Linking precursors of interpersonal trust to human-automation trust: An expanded typology and exploratory experiment J. Trust Res. 2019 9 28 46 10.1080/21515581.2019.1579730
Christopher, S., Calhoun, J. J. G., Philip, B. & Lyons, J. B. Linking precursors of interpersonal trust to human-automation trust: An expanded typology and exploratory experiment. J. Trust Res. 9, 28–46. 10.1080/21515581.2019.1579730 (2019).10.1080/21515581.2019.1579730
37. Fischer JE In-the-loop or on-the-loop? Interactional arrangements to support team coordination with a planning agent Concurr. Comput. Pract. Exp. 2021 33 e4082 10.1002/cpe.4082
Fischer, J. E. et al. In-the-loop or on-the-loop? Interactional arrangements to support team coordination with a planning agent. Concurr. Comput. Pract. Exp. 33, e4082. 10.1002/cpe.4082 (2021).10.1002/cpe.4082
38. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) http://data.europa.eu/eli/reg/2016/679/oj (2016).
39. Binder, N. B. et al. Einsatz Künstlicher Intelligenz in der Verwaltung: rechtliche und ethische Fragen. https://www.zh.ch/content/dam/zhweb/bilder-dokumente/themen/politik-staat/kanton/digitale-verwaltung-und-e-government/projekte_digitale_transformation/ki_einsatz_in_der_verwaltung_2021.pdf (2021).
40. Loi, M., Mätzener, A., Müller, A. & Spielkamp, M. Automated Decision-Making Systems in the Public Sector: An Impact Assessment Tool for Public Authorities. Tech. Rep., algorithmwatch.org. https://algorithmwatch.org/en/wp-content/uploads/2021/06/ADMS-in-the-Public-Sector-Impact-Assessment-Tool-AlgorithmWatch-June-2021.pdf. (2021).
41. de Sousa WG de Melo ERP Bermejo PHDS Farias RAS Gomes AO How and where is artificial intelligence in the public sector going? A literature review and research agenda Gov. Inf. Q. 2019 36 101392 10.1016/j.giq.2019.07.004
de Sousa, W. G., de Melo, E. R. P., Bermejo, P. H. D. S., Farias, R. A. S. & Gomes, A. O. How and where is artificial intelligence in the public sector going? A literature review and research agenda. Gov. Inf. Q. 36, 101392. 10.1016/j.giq.2019.07.004 (2019).10.1016/j.giq.2019.07.004
42. Spielkamp, M. Automating Society: Taking Stock of Automated Decision-Making in the EU. https://algorithmwatch.org/en/wp-content/uploads/2019/02/Automating_Society_Report_2019.pdf. (2019).
43. Araujo T Helberger N Kruikemeier S In AI we trust? Perceptions about automated decision-making by artificial intelligence AI Soc. 2020 35 611 623 10.1007/s00146-019-00931-w
Araujo, T. et al. In AI we trust? Perceptions about automated decision-making by artificial intelligence. AI Soc. 35, 611–623. 10.1007/s00146-019-00931-w (2020).10.1007/s00146-019-00931-w
44. Fazelpour S Danks D Algorithmic bias: Senses, sources, solutions Philos. Compass. 2021 10.1111/phc3.12760
Fazelpour, S. & Danks, D. Algorithmic bias: Senses, sources, solutions. Philos. Compass.10.1111/phc3.12760 (2021).10.1111/phc3.12760
45. van Noordt C Misuraca G Artificial intelligence for the public sector: results of landscaping the use of AI in government across the European Union Gov. Inf. Q. 2022 39 101714 10.1016/j.giq.2022.101714
van Noordt, C. & Misuraca, G. Artificial intelligence for the public sector: results of landscaping the use of AI in government across the European Union. Gov. Inf. Q. 39, 101714 (2022).10.1016/j.giq.2022.101714
46. Wirtz BW Weyerer JC Sturm BJ The dark sides of artificial intelligence: An integrated AI governance framework for public administration Int. J. Pub. Adm. 2020 43 818 829 10.1080/01900692.2020.1749851
Wirtz, B. W., Weyerer, J. C. & Sturm, B. J. The dark sides of artificial intelligence: An integrated AI governance framework for public administration. Int. J. Pub. Adm. 43, 818–829 (2020).10.1080/01900692.2020.1749851
47. Zajko M Artificial intelligence, algorithms, and social inequality: Sociological contributions to contemporary debates Soc. Compass. 2022 10.1111/soc4.12962
Zajko, M. Artificial intelligence, algorithms, and social inequality: Sociological contributions to contemporary debates. Soc. Compass.10.1111/soc4.12962 (2022).10.1111/soc4.12962
48. Glikson E Woolley AW Human trust in artificial intelligence: review of empirical research Acad. Manag. Ann. 2020 14 627 660 10.5465/annals.2018.0057
Glikson, E. & Woolley, A. W. Human trust in artificial intelligence: review of empirical research. Acad. Manag. Ann. 14, 627–660. 10.5465/annals.2018.0057 (2020).10.5465/annals.2018.0057
49. Marcus G Davis E Rebooting AI: Building artificial intelligence we can trust 2019 Vintage
Marcus, G. & Davis, E. Rebooting AI: Building artificial intelligence we can trust (Vintage, 2019).
50. Rossi F Building trust in artificial intelligence J. int. Aff. 2018 72 127 134
Rossi, F. Building trust in artificial intelligence. J. int. Aff. 72, 127–134 (2018).
51. Lange AC Lenglet M Seyfert R On studying algorithms ethnographically: Making sense of objects of ignorance Organization 2019 26 598 617 10.1177/1350508418808230
Lange, A. C., Lenglet, M. & Seyfert, R. On studying algorithms ethnographically: Making sense of objects of ignorance. Organization 26, 598–617. 10.1177/1350508418808230 (2019).10.1177/1350508418808230
52. Seyfert R Algorithms as regulatory objects Inform. Commun. Soc. 2022 25 1542 1558 10.1080/1369118X.2021.1874035
Seyfert, R. Algorithms as regulatory objects. Inform. Commun. Soc. 25, 1542–1558 (2022).10.1080/1369118X.2021.1874035
53. Gerdon F Bach RL Kern C Kreuter F Social impacts of algorithmic decision-making: A research agenda for the social sciences Big Data Soc, 2022 10.1177/20539517221089305
Gerdon, F., Bach, R. L., Kern, C. & Kreuter, F. Social impacts of algorithmic decision-making: A research agenda for the social sciences. Big Data Soc,10.1177/20539517221089305 (2022).10.1177/20539517221089305
54. Kaplan AD Kessler TT Brill JC Hancock PA Trust in artificial intelligence: Meta-analytic findings Hum. Fact. 2023 65 337 359 10.1177/00187208211013988
Kaplan, A. D., Kessler, T. T., Brill, J. C. & Hancock, P. A. Trust in artificial intelligence: Meta-analytic findings. Hum. Fact. 65, 337–359. 10.1177/00187208211013988 (2023).10.1177/00187208211013988
55. Lockey, S., Gillespie, N., Holm, D. & Someh, I. A. A review of trust in artificial intelligence: Challenges, vulnerabilities and future directions. Proc. 54th Hawaii Int. Conf. on Syst. Sci.10.24251/hicss.2021.664 (2021).
56. Chen YNK Wen CHR Impacts of attitudes toward government and corporations on public trust in artificial intelligence Commun. Stud. 2021 72 115 131 10.1080/10510974.2020.1807380
Chen, Y. N. K. & Wen, C. H. R. Impacts of attitudes toward government and corporations on public trust in artificial intelligence. Commun. Stud. 72, 115–131 (2021).10.1080/10510974.2020.1807380
57. Choung H David P Ross A Trust and ethics in AI AI Soc. 2023 38 733 745 10.1007/s00146-022-01473-4
Choung, H., David, P. & Ross, A. Trust and ethics in AI. AI Soc. 38, 733–745 (2023).10.1007/s00146-022-01473-4
58. Molina MD Sundar SS Does distrust in humans predict greater trust in AI? Role of individual differences in user responses to content moderation New Media Soc. 2022 10.1177/14614448221103534
Molina, M. D. & Sundar, S. S. Does distrust in humans predict greater trust in AI? Role of individual differences in user responses to content moderation. New Media Soc.10.1177/14614448221103534 (2022).10.1177/14614448221103534
59. Schepman A Rodway P The general attitudes towards artificial intelligence scale (gaais): Confirmatory validation and associations with personality, corporate distrust, and general trust Int. J. Hum. Comput. Interact. 2023 39 2724 2741 10.1080/10447318.2022.2085400
Schepman, A. & Rodway, P. The general attitudes towards artificial intelligence scale (gaais): Confirmatory validation and associations with personality, corporate distrust, and general trust. Int. J. Hum. Comput. Interact. 39, 2724–2741. 10.1080/10447318.2022.2085400 (2023).10.1080/10447318.2022.2085400
60. Krüger S Wilson C The problem with trust: On the discursive commodification of trust in ai AI Soc. 2023 38 1753 1761 10.1007/s00146-022-01401-6
Krüger, S. & Wilson, C. The problem with trust: On the discursive commodification of trust in ai. AI Soc. 38, 1753–1761 (2023).10.1007/s00146-022-01401-6
61. Colesca SE Understanding trust in e-government Econ. Eng. Decis. 2009 3 7 15
Colesca, S. E. Understanding trust in e-government. Econ. Eng. Decis. 3, 7–15 (2009).
62. Ejdys J Ginevicius R Rozsa Z Janoskova K The role of perceived risk and security level in building trust in e-government solutions E+XXII 2019 10.15240/tul/001/2019-3-014
Ejdys, J., Ginevicius, R., Rozsa, Z. & Janoskova, K. The role of perceived risk and security level in building trust in e-government solutions. E+XXII10.15240/tul/001/2019-3-014 (2019).10.15240/tul/001/2019-3-014
63. Smith ML Limitations to building institutional trustworthiness through e-government: A comparative study of two e-services in Chile J. Inf. Technol. 2011 26 78 93 10.1057/jit.2010.17
Smith, M. L. Limitations to building institutional trustworthiness through e-government: A comparative study of two e-services in Chile. J. Inf. Technol. 26, 78–93. 10.1057/jit.2010.17 (2011).10.1057/jit.2010.17
64. Aoki N An experimental study of public trust in AI chatbots in the public sector Gov. Inf. Q. 2020 37 101490 10.1016/j.giq.2020.101490
Aoki, N. An experimental study of public trust in AI chatbots in the public sector. Gov. Inf. Q. 37, 101490 (2020).10.1016/j.giq.2020.101490
65. Kreps S Jakesch M Can AI communication tools increase legislative responsiveness and trust in democratic institutions? Gov. Inf. Q. 2023 40 101829 10.1016/j.giq.2023.101829
Kreps, S. & Jakesch, M. Can AI communication tools increase legislative responsiveness and trust in democratic institutions?. Gov. Inf. Q. 40, 101829 (2023).10.1016/j.giq.2023.101829
66. Maas J Machine learning and power relations AI Soc. 2023 38 1493 1500 10.1007/s00146-022-01400-7
Maas, J. Machine learning and power relations. AI Soc. 38, 1493–1500. 10.1007/s00146-022-01400-7 (2023).10.1007/s00146-022-01400-7
67. Kern, C., Bach, R. L., Mautner, H. & Kreuter, F. Fairness in Algorithmic Profiling: A German Case Study. CoRR abs/2108.04134 (2021).
68. Kuppler M Kern C Bach RL Kreuter F From fair predictions to just decisions? Conceptualizing algorithmic fairness and distributive justice in the context of data-driven decision-making Front. Sociol. 2022 7 883999 10.3389/fsoc.2022.883999 36299413
Kuppler, M., Kern, C., Bach, R. L. & Kreuter, F. From fair predictions to just decisions? Conceptualizing algorithmic fairness and distributive justice in the context of data-driven decision-making. Front. Sociol. 7, 883999. 10.3389/fsoc.2022.883999 (2022).36299413 10.3389/fsoc.2022.883999
69. Helberger N Araujo T de Vreese CH Who is the fairest of them all? Public attitudes and expectations regarding automated decision-making Comput. Law Secur. Rev. 2020 39 105456 10.1016/j.clsr.2020.105456
Helberger, N., Araujo, T. & de Vreese, C. H. Who is the fairest of them all? Public attitudes and expectations regarding automated decision-making. Comput. Law Secur. Rev. 39, 105456. 10.1016/j.clsr.2020.105456 (2020).10.1016/j.clsr.2020.105456
70. Miller SM Keiser LR Representative bureaucracy and attitudes toward automated decision making J. Pub. Adm. Res. Theory 2021 31 150 165 10.1093/jopart/muaa019
Miller, S. M. & Keiser, L. R. Representative bureaucracy and attitudes toward automated decision making. J. Pub. Adm. Res. Theory 31, 150–165. 10.1093/jopart/muaa019 (2021).10.1093/jopart/muaa019
71. Kaun A Suing the algorithm: The mundanization of automated decision-making in public services through litigation Inform. Commun. Soc. 2022 25 2046 2062 10.1080/1369118X.2021.1924827
Kaun, A. Suing the algorithm: The mundanization of automated decision-making in public services through litigation. Inform. Commun. Soc. 25, 2046–2062 (2022).10.1080/1369118X.2021.1924827
72. Berryhill J Heang KK Clogher R McBride K Hello World: Artificial Intelligence and its Use in the Public Sector 2019 OECD
Berryhill, J., Heang, K. K., Clogher, R. & McBride, K. Hello World: Artificial Intelligence and its Use in the Public Sector (OECD, 2019).
73. Buiten MC Towards intelligent regulation of artificial intelligence Eur. J. Risk Regul. 2019 10 41 59 10.1017/err.2019.8
Buiten, M. C. Towards intelligent regulation of artificial intelligence. Eur. J. Risk Regul. 10, 41–59 (2019).10.1017/err.2019.8
74. Burrell J Fourcade M The society of algorithms Annu. Rev. Sociol. 2021 47 213 237 10.1146/annurev-soc-090820-020800
Burrell, J. & Fourcade, M. The society of algorithms. Annu. Rev. Sociol. 47, 213–237. 10.1146/annurev-soc-090820-020800 (2021).10.1146/annurev-soc-090820-020800
75. Justo-Hanani R The politics of artificial Intelligence regulation and governance reform in the European union Policy Sci. 2022 55 137 159 10.1007/s11077-022-09452-8
Justo-Hanani, R. The politics of artificial Intelligence regulation and governance reform in the European union. Policy Sci. 55, 137–159 (2022).10.1007/s11077-022-09452-8
76. Yeung K Lodge M Algorithmic Regulation 2019 Oxford University Press
Yeung, K. & Lodge, M. Algorithmic Regulation (Oxford University Press, 2019).
77. Ulbricht L Yeung K Algorithmic regulation: A maturing concept for investigating regulation of and through algorithms Regul. Gov. 2022 16 3 22 10.1111/rego.12437
Ulbricht, L. & Yeung, K. Algorithmic regulation: A maturing concept for investigating regulation of and through algorithms. Regul. Gov. 16, 3–22 (2022).10.1111/rego.12437
78. Zuboff S The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power 2019 Public Affairs
Zuboff, S. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power (Public Affairs, 2019).
79. Bourne C AI cheerleaders: Public relations, neoliberalism and artificial intelligence Pub. Relat. Inq. 2019 8 109 125 10.1177/2046147X19835250
Bourne, C. AI cheerleaders: Public relations, neoliberalism and artificial intelligence. Pub. Relat. Inq. 8, 109–125. 10.1177/2046147X19835250 (2019).10.1177/2046147X19835250
80. Gritsenko D Wood M Algorithmic governance: A modes of governance approach Regul. Gov. 2022 16 45 62 10.1111/rego.12367
Gritsenko, D. & Wood, M. Algorithmic governance: A modes of governance approach. Regul. Gov. 16, 45–62 (2022).10.1111/rego.12367
81. Busuioc M Accountable artificial intelligence: Holding algorithms to account Pub. Adm. Rev. 2021 81 825 836 10.1111/puar.13293 34690372
Busuioc, M. Accountable artificial intelligence: Holding algorithms to account. Pub. Adm. Rev. 81, 825–836. 10.1111/puar.13293 (2021).34690372 10.1111/puar.13293
82. Grimmelikhuijsen, S. Introduction to the Digital Government and Artificial Intelligence Minitrack. In Proceedings of the 55th Hawaii International Conference on System Sciences (2022).
83. Büthe T Djeffal C Lütge C Maasen S Ingersleben-Seip NV Governing AI—Attempting to herd cats? Introduction to the special issue on the governance of artificial intelligence J. Eur. Pub. Policy 2022 29 1721 1752 10.1080/13501763.2022.2126515
Büthe, T., Djeffal, C., Lütge, C., Maasen, S. & Ingersleben-Seip, N. V. Governing AI—Attempting to herd cats? Introduction to the special issue on the governance of artificial intelligence. J. Eur. Pub. Policy 29, 1721–1752. 10.1080/13501763.2022.2126515 (2022).10.1080/13501763.2022.2126515
84. Nitzberg M Zysman J Algorithms, data, and platforms: The diverse challenges of governing AI J. Eur. Pub. Policy 2022 29 1753 1778 10.1080/13501763.2022.2096668
Nitzberg, M. & Zysman, J. Algorithms, data, and platforms: The diverse challenges of governing AI. J. Eur. Pub. Policy 29, 1753–1778 (2022).10.1080/13501763.2022.2096668
85. Busuioc M AI Algorithmic Oversight: New Frontiers in Regulation 2022 Edward Elgar Publishing
Busuioc, M. AI Algorithmic Oversight: New Frontiers in Regulation (Edward Elgar Publishing, 2022).
86. Russell S Hannes W Erich P Edward AL Carlo G Artificial intelligence and the problem of control Perspectives on Digital Humanism 2022 Springer
Russell, S. Artificial intelligence and the problem of control. In Perspectives on Digital Humanism (eds Hannes, W. et al.) (Springer, 2022).
87. Six F Verhoest K Trust in Regulatory Regimes 2017 Edward Elgar Publishing
Six, F. & Verhoest, K. Trust in Regulatory Regimes (Edward Elgar Publishing, 2017).
88. Buiten MC Towards intelligent regulation of artificial intelligence Eur. J. Risk Regul. 2019 10 41 59 10.1017/err.2019.8
Buiten, M. C. Towards intelligent regulation of artificial intelligence. Eur. J. Risk Regul. 10, 41–59. 10.1017/err.2019.8 (2019).10.1017/err.2019.8
89. Justo-Hanani R The politics of artificial intelligence regulation and governance reform in the European union Policy Sci. 2022 55 137 159 10.1007/s11077-022-09452-8
Justo-Hanani, R. The politics of artificial intelligence regulation and governance reform in the European union. Policy Sci. 55, 137–159. 10.1007/s11077-022-09452-8 (2022).10.1007/s11077-022-09452-8
90. Di Mascio F Maggetti M Natalini A Exploring the dynamics of delegation over time: Insights from Italian anti-corruption agencies (2003–2016) Policy Stud. J. 2020 48 367 400 10.1111/psj.12253
Di Mascio, F., Maggetti, M. & Natalini, A. Exploring the dynamics of delegation over time: Insights from Italian anti-corruption agencies (2003–2016). Policy Stud. J. 48, 367–400. 10.1111/psj.12253 (2020).10.1111/psj.12253
91. Abbott KW Levi-faur D Snidal D Theorizing regulatory intermediaries: The RIT model Ann. Am. Acad. Polit. Soc. Sci. 2017 670 14 35 10.1177/0002716216688272
Abbott, K. W., Levi-faur, D. & Snidal, D. Theorizing regulatory intermediaries: The RIT model. Ann. Am. Acad. Polit. Soc. Sci. 670, 14–35. 10.1177/0002716216688272 (2017).10.1177/0002716216688272
92. Tallberg, J. et al. The Global Governance of Artificial Intelligence: Next Steps for Empirical and Normative Research. ArXiv:2305.11528 (2023).
93. Hidalgo CA Orghian D Albo Canals J de Almeida F Martin N How Humans Judge Machines 2021 The MIT Press
Hidalgo, C. A., Orghian, D., Albo Canals, J., de Almeida, F. & Martin, N. How Humans Judge Machines (The MIT Press, 2021).
94. Ingram, M. Calibrating trust between humans and artificial intelligence systems. PhD Thesis, University of Glasgow (2023).
95. Ullman, D. & Malle, B. F. What Does it Mean to Trust a Robot? Steps Toward a Multidimensional Measure of Trust. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, 263–264, 10.1145/3173386.3176991 (Association for Computing Machinery, New York, NY, USA, 2018).
96. Ethics guidelines for trustworthy AI|Shaping Europe’s digital future (2019).
97. vonEschenbach W Transparency and the black box problem: Why we do not trust AI Philos. Technol. 2021 34 1607 1622 10.1007/s13347-021-00477-0
vonEschenbach, W. Transparency and the black box problem: Why we do not trust AI. Philos. Technol. 34, 1607–1622. 10.1007/s13347-021-00477-0 (2021).10.1007/s13347-021-00477-0
98. Winfield AFT P7001: A proposed standard on transparency Front. Robot. AI 2021 10.3389/frobt 34381820
Winfield, A. F. T. et al. P7001: A proposed standard on transparency. Front. Robot. AI10.3389/frobt (2021).34381820 10.3389/frobt
99. Wang H Why should we care about the manipulative power of algorithmic transparency? Philos. Technol. 2023 36 9 10.1007/s13347-023-00610-1
Wang, H. Why should we care about the manipulative power of algorithmic transparency?. Philos. Technol. 36, 9 (2023).10.1007/s13347-023-00610-1
100. Grimmelikhuijsen S Explaining why the computer says no: Algorithmic transparency affects the perceived trustworthiness of automated decision-making Pub. Adm. Rev. 2023 83 241 262 10.1111/puar.13483
Grimmelikhuijsen, S. Explaining why the computer says no: Algorithmic transparency affects the perceived trustworthiness of automated decision-making. Pub. Adm. Rev. 83, 241–262. 10.1111/puar.13483 (2023).10.1111/puar.13483
101. Floridi L Cowls J A unified framework of five principles for ai in society Harv. Data Sci. Rev. 2019 10.1162/99608f92.8cd550d1
Floridi, L. & Cowls, J. A unified framework of five principles for ai in society. Harv. Data Sci. Rev.10.1162/99608f92.8cd550d1 (2019).10.1162/99608f92.8cd550d1
102. Turek, M. Explainable Artificial Intelligence (XAI) (2017).
103. Speith, T. A review of taxonomies of explainable artificial intelligence (XAI) methods. In Proc. 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22), 2239–2250, 10.1145/3531146.3534639 (Association for Computing Machinery, New York, NY, USA, 2022).
104. Schwalbe G Finzel B A comprehensive taxonomy for explainable artificial intelligence: A systematic survey of surveys on methods and concepts Data Min. Knowl. Disc. 2023 10.1007/s10618-022-00867-8
Schwalbe, G. & Finzel, B. A comprehensive taxonomy for explainable artificial intelligence: A systematic survey of surveys on methods and concepts. Data Min. Knowl. Disc.10.1007/s10618-022-00867-8 (2023).10.1007/s10618-022-00867-8
105. Ribeiro, M. T., Singh, S. & Guestrin, C. “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In Proc. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, 1135–1144, 10.1145/2939672.2939778 (Association for Computing Machinery, New York, NY, USA, 2016).
106. Miller, T. Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven Decision Support using Evaluative AI. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 333–342, 10.1145/3593013.3594001 (2023).
107. Chouldechova A Roth A A snapshot of the frontiers of fairness in machine learning Commun. ACM 2020 63 82 89 10.1145/3376898
Chouldechova, A. & Roth, A. A snapshot of the frontiers of fairness in machine learning. Commun. ACM 63, 82–89. 10.1145/3376898 (2020).10.1145/3376898
108. Mehrabi N Morstatter F Saxena N Lerman K Galstyan A A survey on bias and fairness in machine learning ACM Comput. Surv. 2021 10.1145/3457607
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K. & Galstyan, A. A survey on bias and fairness in machine learning. ACM Comput. Surv.10.1145/3457607 (2021).10.1145/3457607
109. Pessach D Shmueli E A review on fairness in machine learning ACM Comput. Surv. 2022 55 1 44 10.1145/3494672
Pessach, D. & Shmueli, E. A review on fairness in machine learning. ACM Comput. Surv. 55, 1–44 (2022).10.1145/3494672
110. Kasirzadeh, A. Algorithmic Fairness and Structural Injustice: Insights from Feminist Political Philosophy. In Proc. 2022 AAAI/ACM Conference on AI, Ethics, and Society, 349–356, 10.1145/3514094.3534188 (Association for Computing Machinery, 2022).
111. Almada, M. Human intervention in automated decision-making: Toward the construction of contestable systems. In Proc. 17th International Conference on Artificial Intelligence and Law, 2–11 (2019).
112. Henin, C. & Le Métayer, D. Beyond explainability: Justifiability and contestability of algorithmic decision systems. AI & Soc. (2021).
113. Lyons H Velloso E Miller T Conceptualising contestability: Perspectives on contesting algorithmic decisions Proc. ACM Hum. Comput. Interact. 2021 5 1 25 10.1145/3449180 36644216
Lyons, H., Velloso, E. & Miller, T. Conceptualising contestability: Perspectives on contesting algorithmic decisions. Proc. ACM Hum. Comput. Interact. 5, 1–25 (2021).36644216 10.1145/3449180
114. Alfrink K Keller I Kortuem G Doorn N Contestable AI by design: Towards a framework Minds Mach. 2022 33 613 639 10.1007/s11023-022-09611-z
Alfrink, K., Keller, I., Kortuem, G. & Doorn, N. Contestable AI by design: Towards a framework. Minds Mach. 33, 613–639 (2022).10.1007/s11023-022-09611-z
