Paper deep dive

Co-constructing sociotechnical AI governance: participatory system mapping using algorithm registers

Íñigo de Troya, Maurus Enbergs, Neelke Doorn, Roel Dobbe

Year: 2026Venue: arXiv preprintArea: cs.CYType: PreprintEmbeddings: 78

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Algorithm registers have been championed as a means of providing transparency on the use of algorithms in public services. Yet potential publics differ in their expectations of what should be made transparent and how, as well as in their interest in and ability to parse the information currently published in the registers. Moreover, it remains unclear how these instruments can represent the sociotechnical systems in which these algorithms are embedded, and how system-level transparency can facilitate accountability. In this paper, we ask, what do algorithm registers reveal (and occlude) about the sociotechnical systems governing algorithmic systems, and how can diverse stakeholder perspectives inform a more pluralistic system-theoretic safety analysis? To do this, we probe the municipal algorithm register of a Dutch city through a case study of a decision-support tool for caseworkers' assessment of citizens' welfare benefits eligibility based on legal automation through a business rule engine. Through interviews, surveys, and participatory system mapping workshops (with municipal staff, civil society organisations, and ombudsmen, N=8), we seek to understand to what extent the register allows stakeholders to map the algorithmic system in question. These maps inform a System-Theoretic Process Analysis (STPA) that situates the register within a wider sociotechnical governance structure. Participants' contributions allow us to identify potential safety hazards which would not have been possible to see using the algorithm register alone, including benefits eligibility denial, system performance deterioration, and inability to contest wrongful decisions. By engaging both direct and indirect stakeholders, we reflect on the normative dimensions of algorithm governance efforts and how politics shape the practice of system safety analysis.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

76,547 characters extracted from source content.

Expand or collapse full text

Co-constructing sociotechnical AI governance: participatory system mapping using algorithm registers ́ I ̃ nigo de Troya 1 , Maurus Enbergs 1 , Neelke Doorn 1 , Roel Dobbe 1 1 TU Delft Abstract Algorithm registers have been championed as a means of pro- viding transparency on the use of algorithms in public ser- vices. Yet potential publics differ in their expectations of what should be made transparent and how, as well as in their inter- est in and ability to parse the information currently published in the registers. Moreover, it remains unclear how these in- struments can represent the sociotechnical systems in which these algorithms are embedded, and how system-level trans- parency can facilitate accountability. In this paper, we ask, what do algorithm registers reveal (and occlude) about the sociotechnical systems governing algorithmic systems, and how can diverse stakeholder perspectives inform a more plu- ralistic system-theoretic safety analysis? To do this, we probe the municipal algorithm register of a Dutch city through a case study of a decision-support tool for caseworkers’ assess- ment of citizens’ welfare benefits eligibility based on legal automation through a business rule engine. Through inter- views, surveys, and participatory system mapping workshops (with staff from a municipalility, civil society organisations, and municipal Ombudsmen, N=8), we seek to understand to what extent the register allows stakeholders to map the al- gorithmic system in question. These maps inform a System- Theoretic Process Analysis (STPA) that situates the regis- ter within a wider sociotechnical governance structure. Par- ticipants’ contributions allow us to identify potential safety hazards which would not have been possible to see using the algorithm register alone, including benefits eligibility de- nial, system performance deterioration, and inability to con- test wrongful decisions. By engaging both direct and indirect stakeholders, we reflect on the normative dimensions of algo- rithm governance efforts and how politics shape the practice of system safety analysis. Introduction The opacity of algorithmic systems can prevent affected in- dividuals and communities from understanding when they are subjected to erroneous or harmful decisions (Janssen and Kuk 2016), and frustrate efforts by concerned publics to scrutinise these systems for potential risks (Burrell 2016; Ellen Dingemans and van Dalen 2021; Wieringa 2023). In the context of algorithmic systems in public services, transparency is a prerequisite for accountability (Kroll et al. 2017; Binns 2018; Nieuwenhuizen 2025). High-profile cases like the Dutch childcare benefits scandal (the Toesla- genaffaire) have shown that there is a need for more trans- parency into how these systems function, and for account- ability mechanisms that can help restore a sense of jus- tice that is increasingly lost to the digitalisation of the state (Frederik 2021; Peeters and Widlak 2023; Buszydlik et al. 2025). However, what constitutes meaningful transparency remains elusive, particularly concerning efforts to inform the general public (Ananny and Crawford 2018). In recent years, municipal algorithm registers have be- come a popular instrument for providing transparency into the use of algorithmic systems in local government (van Vliet et al. 2024; Rotterdam Court of Auditors 2024; Nieuwenhuizen 2025). Early adopters have embraced reg- isters as a platform for informing citizens about how algo- rithms are used to provide services such as welfare bene- fits eligibility (Meeri Haataja and Rautio 2020; Popa 2025). However, recent empirical studies have shown that there are still open challenges to be resolved before these instruments can provide transparency that is meaningful and actionable (IA Ciudadana 2025). Both government auditors and aca- demic researchers have found that these registers either dis- close too little information to be useful, or are too techni- cal for citizens to understand (Rotterdam Court of Auditors 2024; Nieuwenhuizen 2025). In Rotterdam, the Court of Au- ditors concluded that it is simply not clear who the city’s reg- ister is meant to inform (Rotterdam Court of Auditors 2024). The challenge remains of how algorithm registers can rec- oncile the varying information needs of different stakehold- ers, while respecting organisations’ hesitation to provide full disclosure. There is a risk that the register may present a fur- ther administrative burden on citizens who are ill prepared to parse the information disclosures required to understand how algorithmic systems are used in public services (Mad- sen, Lindgren, and Melin 2022). It is clear that more work is needed before algorithm registers provide enough trans- parency to facilitate accountability (Williams et al. 2022). Despite these shortcomings, algorithm registers can have a “disciplinary effect” on organisations, serving as a “mean- ingful box-ticking exercise” (Nieuwenhuizen 2025, p. 429) that motivates them to take stock of their algorithmic sys- tems, identify their risks, and ensure that appropriate mitiga- tion strategies are in place. This secondary use suggests that registers may help to improve internal awareness about al- gorithms and their governance, contributing to managing the risks of AI in the public sector (Zuiderwijk, Chen, and Salem arXiv:2608.12166v1 [cs.CY] 12 Aug 2026 2021). To realise this ambition, we require a conceptuali- sation of algorithms in sociotechnical systems which com- bines the currently disconnected operational practices and governance instruments that populate the register. Further- more, assessing whether strict sociotechnical requirements like safety or fairness are met requires methods to analyze algorithmic tools as embedded in wider sociotechnical sys- tems, to characterize and potentially shape the systemic dy- namics across technical and non-technical factors and com- ponents (Dobbe 2022; de Troya et al. 2025). Without under- standing how algorithm registers are both a part of but also inform algorithm governance, the resulting governance prac- tices risk becoming a set of of disparate efforts that cannot meaningfully anticipate or address the actual safety hazards that may emerge in the system. In this study, we ask two complementary questions. What do algorithm registers reveal (and occlude) about the algo- rithmic systems and their governance? (RQ.1.) Then, how can diverse stakeholder perspectives inform a more plu- ralistic system-theoretic safety analysis? (RQ.2.) We an- swer these questions through a series of workshops, inter- views, and surveys with both direct and indirect stakehold- ers, including civil society organisations (CSOs), Ombuds- men (public mediators), and municipal staff. In so doing, we seek to understand their expectations, experiences, and de- sires of algorithmic transparency, and how their unique po- sitionality can help inform a critical safety assessment of a specific decision-support tool listed in the register. The tool selected for this study is called Avola, a decision-support tool for caseworkers’ assessment of citizens’ welfare ben- efits eligibility that is based on legal automation through a business rule engine. We draw on system-theoretic princi- ples and methods from safety science (Leveson 2011; Leve- son and Thomas 2018) to reconstruct the operational and governance architectures in which the decision-support tool and the register are embedded. By answering these questions we make three core con- tributions. First, we provide an empirical assessment of a municipal algorithm register to understand the extent to which it serves the transparency needs of different relevant stakeholder groups. Second, we develop and validate a par- ticipatory schema for mapping an algorithmic system and its governance structure by integrating the perspectives, in- sights and needs of different direct and indirect stakeholders. Lastly, we reflect on the normative choices and political di- mensions behind the abstractions developed while mapping an algorithm governance structure through system safety methods. Background & Related Work The blindspot of algorithm registers: transparency is an essentially contested concept While algorithm transparency in the public sector is thought to foster citizens’ trust in government (Grimmelikhuijsen 2023), it remains an essentially contested concept. Its re- alisation is highly variable depending on who is disclosing information, and to whom. In a review of transparency in public administration, Meijer notes that “transparency may contribute to accountability [...] when there are actors capa- ble of processing the information” (Meijer 2014, p.1) Trans- parency is often treated as a performative act – transparency as something that is done (Cellard 2020). However, it is ul- timately a relational act concerning, socially situated actors with differentiated levels of access to information, and abil- ity to understand that information (Murray-Rust, Alfrink, and Zaga 2025). Such efforts should thus be evaluated in the context and form in which they are provided – trans- parency as something that is given by someone to some- one else (Felzmann et al. 2019). Cellard describes algorith- mic transparency as a kind of ‘theatre’ play which requires the staging of identities, issues, and algorithms, “disclosed through the mise en sc ` ene of citizens’ motivations, the plac- ing of controversial requests on public bodies, and a regula- tory framework redefining administrative procedures as ‘al- gorithms’.” (Cellard 2020, p.1) As such, for transparency to meaningfully serve its intended audience, algorithmic trans- parency efforts must grapple with what is to be made trans- parent, to whom, in what way, and to what ends, acknowl- edging the heterogeneity of the intended publics (Axelsson, Melin, and Lindgren 2013). The transparency provided by registers is intended to le- gitimate the use of algorithmic systems in public services. However, for controversial systems, such as those profil- ing vulnerable individuals or groups for the distribution of welfare services, such legitimization efforts may nonethe- less still be challenged (Haug, Haug, and Hansen 2026). In discourses surrounding algorithm governance, transparency is an instrumental means towards achieving accountability, such that violations of subjects’ rights or freedoms may be identified and addressed in a legitimate public forum (Wieringa 2020). There is a consensus within the literature that “transparency may contribute to accountability [...] when there are actors capable of processing the informa- tion” (Meijer 2014, p. 1) Norval et al. note that if trans- parency is too highly technical, it may “undermin[e] the discloure’s effectiveness, can disempower subjects, and ul- timately hinder broader transparency aims” (Norval et al. 2022, p. 679). They suggest thinking of disclosures as ‘inter- faces’, “designed for the needs, expectations, and require- ments of the recipients they serve to inform.” (Norval et al. 2022, p. 679) However, there is a tension between how much information organisations are willing to disclose and what information different publics need in order to be adequately informed. The heterogenity of potential publics remains a challenge for public services aiming to provide a one-size- fits-all solution to algorithm transparency (Axelsson, Melin, and Lindgren 2013). Without a clear notion of who those actors are, their information needs, and what that trans- parency should achieve, registers will remain unable to pro- vide meaningful accountability (Murray-Rust, Alfrink, and Zaga 2025). Critics in both government and academia have proposed shifting the focus away from providing information to citi- zens and instead towards providing transparency to interme- diaries, such as oversight authorities and societal watchdogs (Rotterdam Court of Auditors 2024; Nieuwenhuizen 2025) - actors more capable of understanding meaningful disclosure and thus of using the register to hold the system providers to account. Furthermore, these indirect stakeholders may be well positioned to support directly affected parties, by virtue of their proximity, experience, and tact for intermedi- ation (Chalke 2023; Dahlvik 2022; Madise and Pilvik 2024). Third parties acting on the interest of affected individuals, can serve as a bridge between the lived experience of algo- rithmic harm and the technical expertise required to unpack it. However, the challenge remains of what level of informa- tion the register should provide in order to serve the infor- mation needs of these indirect stakeholders. Total access is not feasible due to the sensitivity of the data used to make decisions, the fact that source code of underlying algorithms does not explain decisions without said data, and the sheer burden on the deployer organisation to provide all documen- tation upfront (Fenster 2005). As such, some choices need to be made, and these inevitably lead to the selective inclusion and abstraction of information (Paudyal and William Wong 2018). What information is considered relevant to reveal is thus a political question (de Troya et al. 2025). Modeling sociotechnical AI governance as a hierarchical control structure The algorithm register serves as a way of “figuring” the com- plexity of the Avola system in a way that affected individuals and external stakeholders can understand how the algorith- mic system and its governance are structured (Andersson, Hallin, and Ivory 2022). The information presented in al- gorithm registers points toward a whole-system perspective, revealing elements of the technology, its operational proce- dures, organisational arrangements, and the institutions that govern it. However, many stakeholders still struggle to make sense of this information, meaning that the systems view it suggests remains only partial in practice (Rotterdam Court of Auditors 2024). To understand how the various interven- tions and components relate to one another, a more system- atic and integrated approach is needed — one that can recon- struct how these disparate elements fit together. To support this effort, we now turn to system-theoretic methods from the field of system safety. While initially developed in the context of safety science in industrial applications, such as aviation and energy (Leveson 2011), the theories and meth- ods of system safety have recently gained traction in the field of algorithm governance and AI safety (Raji and Dobbe 2020; Dobbe 2022; Rismani et al. 2023; Delfos et al. 2024). The System-Theoretic Accident Model and Processes (STAMP) framework models safety and accidents in dy- namic complex systems as a problem of inadequate control (Leveson 2004). Unlike earlier paradigms in safety science, which focused on human error, component failure, or safety culture, STAMP models the emergence of safety hazards and the ensuing accidents as failures to adequately control a sys- tem’s state in the presence of worst-case environmental con- ditions (Rismani et al. 2023). To do so, STAMP proposes an ontology for modeling safety in sociotechnical systems as a hierarchical control structure composed of control ac- tuators which respond to feedback signals relayed from un- derlying controlled processes (Leveson 2011). These con- trol and feedback signals span the technical, operational, or- ganisational, and institutional layers of a sociotechnical sys- tem, and may include components such as technical risk as- sessments, human operators with discretionary power, for- mal recourse channels, and governance instruments such as service-level agreements and regulatory frameworks. The virtue of STAMP lies in its ability to analyse and orchestrate diverse safety mechanisms across multiple levels through the application of system-theoretic principles (Leveson and Thomas 2018). STAMP provides the conceptual basis for the System-Theoretic Process Analysis (STPA) method we used in this study, described further in the Methods section. Participatory design and mapping While the system-theoretic basis of STPA is predicated on the consideration of various stakeholder groups, safety sci- ence remains a largely expert-led discipline in which the stakeholders who are brought into the fold are those already on the inside, capable of contributing their expert knowl- edge to the safety engineers conducting the assessment (Bj ̈ ornsd ́ ottir et al. 2023). However, a growing body of evi- dence on the harmful consequences of algorithmic systems has shown that affected and indirect stakeholders experi- ence and understand these systems in ways that those spared of their outputs simply do not (Eubanks 2018; Costanza- Chock 2020; Katell et al. 2020). The “participatory turn” in AI (Delgado et al. 2023) has acknowledged that affected stakeholders can bring new forms of situated knowledge that designers may be unaware of (Greenbaum and Kyng 1991; Haraway 1988), enhance public oversight (Kallina, Bohn ́ e, and Singh 2025), anticipate risks (Kallina, Bohn ́ e, and Singh 2025), reflect on design objectives (Katell et al. 2020), or help to ensure the systems are useful for those who ulti- mately have to use them (Guti ́ errez et al. 2019; Zejnilovi ́ c et al. 2020). Participation is also often seen as a moral im- perative to give these systems social legitimacy (D’ignazio and Klein 2023; Costanza-Chock 2020). As such, participatory approaches to the design and gov- ernance of algorithmic systems can complement safety engi- neers’ own expertise. Despite this, the role of critical actors outside academia, such as civil society organisations and Ombudsmen (public mediators), is underexplored in both the literature on participatory approaches to AI, and in safety science. While affected individuals and communities may not need technical or legal knowledge to realise that they are experiencing harm or injustice (Kuo et al. 2023), they often do need such forms of expertise in order to make their case and seek redress. As such, rather than seek to involve di- rectly affected parties in the design and governance of these systems, we may also consider how to engage other actors which already act as critical governance and safety mecha- nisms on their behalf (Himmelreich 2023). Case study: the Avola welfare eligibility algorithm De Gemeente (a ficitious name) is one of the first municipal- ities in the Netherlands to adopt algorithm registers. Their municipal algorithm register adheres to the goals set for the national algorithm register by the Ministry of Internal Af- fairs. The national algorithm register’s website states that “The Algorithm Register contains information about algo- rithms used by the government. This makes this informa- tion findable and available to citizens, their advocates, the media and supervisors.” (Ministry of Internal Affairs 2025) Among the stated goals of the register are “Increasing trust in government” and ”Increasing the controllability of the government”, proposing that “If the government shows what it does, citizens and organisations can better control it. The Algorithm Register supports this control by society.” At the time of writing, De Gemeente’s register documents 18 al- gorithmic systems that are in use, and 2 that have been de- commissioned. The systems are labeled as either high risk or low risk, and range from monitoring improper waste dis- posal (low risk, decommissioned) to determining eligibility for welfare benefits (high risk, in use). We probe the algorithm register through a case study of an algorithmic decision support tool that municipal case- workers use to determine whether or not citizens are eligi- ble for welfare benefits. The system in question, Avola, is a “no-code” business rule engine which implements the writ- ten laws that govern welfare subsidy eligibility (Kaeseberg 2019; Escher et al. 2024). The resulting law-codification im- plementation is validated by legal experts. Frontline case- workers use Avola as a decision-support tool when they make their own determinations about applicants’ welfare el- igibility status. Avola was developed by an independent con- tractor, Bizzomate (since acquired by CIPHIX; see (Black- trace Mergers & Acquisitions 2024)). The register provides information regarding technical, op- erational, organisational, and institutional components, de- scribing, among others, the datasets used by Avola to de- termine benefits eligibility, the role of caseworkers in ex- ercising discretion over final eligibility decisions, informa- tion regarding complaints procedures and Freedom of Infor- mation Requests (Open Government Law, Wet Open Over- heid [WOO] in Dutch), and the legal articles implemented through Avola. Additionally, it indicates that data protec- tion (DPIA) and fundamental rights (FRAIA) impact assess- ments have been conducted. It is worth noting that while the AI Act is set to make the FRAIA mandatory (Iwanska et al. 2024; European Centre for Not-for-profit Law and Danish Institute for Human Rights 2025), it remains a tool to facil- itate reflection within the organisation, rather than a strict compliance instrument for external auditors (Gerards et al. 2022). Methods The study was composed of five stages (Stage I-V), sum- marised in Table 1. In this paper we focus on reporting the findings from Stages IV-V. The contributions made in these later stages were informed by the earlier Stages I-I, which served to familiarise the participants with the different meth- ods and resources used in this study, including the algorithm register and the Avola system, in order to be able to perform the participatory system mapping in Stage IV. The study was approved by a Human Research Ethics Council review. Stakeholder identification In order to capture a plurality of views on Avola and the reg- ister, we identified three relevant stakeholder groups: munic- ipal staff, municipal ombudsman, and civil society organisa- tions. These groups were chosen by virtue of their expertise on the governance of algorithmic systems in public services, and understanding of their impacts on citizens. As our pri- mary focus was the algorithm register, not Avola, we did not engage with system developers. Direct stakeholders (Group M) As internal stakeholders, municipal staff were those most familiar with both the algo- rithm register and the Avola system itself. These included staff in charge of algorithm governance (such as maintain- ing the algorithm register, conducting risk assessments, etc.) (M1, M2, M4), as well as the product owner (M3), who is in charge of commissioning and managing the development and maintenance of the Avola system, which they see as a means for operationalising municipal policy. The product owner also provides information and documentation to staff in charge of algorithm governance who publish and maintain the relevant information in the algorithm register. Indirect stakeholders (Group OC) Municipal Ombuds- men and CSOs were chosen as external stakeholders who play a vital role in protecting citizens from administrative harms in public services. Municipal ombudsmen mediate disputes between citizens and the municipality, when citi- zens feel like their concerns are not being appropriately ad- dressed, or struggle to resolve their issues directly with mu- nicipal staff (e.g. caseworkers or the complaints department) (Chalke 2023; Dahlvik 2022; Madise and Pilvik 2024). In the Netherlands, Municipal Ombudsmen are installed by and report back to City Council, who grants them a supervisory mandate over the municipality, which is thus compelled to be cooperative during independent investigations. Civil society organisations represent citizens’ interests through various forms of policy advocacy work and rais- ing public awareness. While CSOs have no formal author- ity over the municipality, they can conduct research to ex- ert influence externally by influencing other actors who do, such as policymakers or ombudsmen. They may achieve this through policy briefs, public awareness campaign that gen- erate political pressure, and other strategies. Participatory mapping workshops (Stage IV) Two separate mapping workshops were held with each of the groups, Group M and Group OC. Each workshop lasted 3 hours, and produced multiple sources of data: au- dio recordings, annotated system maps, and worksheets. An introductory 20-minute presentation illustrated an example STPA analysis of a separate welfare benefits eligibility sys- tem. This served to familiarise participants with the STPA method and potential outcomes of the analysis. Following this, participants were presented with a pre- liminary system map (M 0 ) and a worksheet (W ). MapM 0 , drafted by the authors, presented the Avola system compo- nents revealed in the algorithm register. Additionally, some components known to the authors through publicly available Table 1: Study design. See Appendix A for the interview protocols; Appendix B for the worksheet questions. Stage DescriptionPurposeDatasets IInterviewsUnderstanding the Algorithm Register & Avola sys- tem Interviews (audio, transcriptions) IIPreliminary map Drafting an initial system map for later Workshop 2 (Stage IV) Algorithm register entry on Avola, internal documen- tation (incl. FRAIA), interviews with municipal staff (Stage I), preliminary mapM 0 IIIWorkshop1: Brainstorming Understanding indirect stakeholders’ expectations of, experiences with, and suggestions for the register Algorithm register entry on Avola, Surveys, Workshop recordings (audio, transcripts), paper annotated with suggestions IVWorkshop2: Participatory mapping Gathering stakeholders’ input for later STPA system safety analysis (Stage V) Workshop recordings (audio, transcripts), worksheetW , preliminary mapM 0 (Stage I), annotated mapsM 0 * (A1 paper) VSystem safety analysis Situating the Algorithm Register & Avola system within the broader sociotechnical system architecture Hierarchical control structure, STPA analysis table, an- notated maps (Stage IV) Table 2: Participants’ background and involvement in interviews and workshops (see Table 1). The dotted line separates direct and indirect stakeholders (Groups M and OC), who were grouped for the two iterations of Workshop 2. IDOrganisationExpertiseTenure in role (years) Interviewed (Stage I) Workshop1 (Stage I) Workshop2 (Stage IV) M1MunicipalityAlgorithm governance1-5DXD M2MunicipalityAlgorithm governance0-1DXX M3MunicipalityProduct owner5-10DXX M4MunicipalityAlgorithm governance1-5DXD O1Municipal OmbudsmanApplied Research0-1XDD O2Municipal OmbudsmanApplied Research1-5XDD C1European Center for Not-for-Profit Law Legal advisor1-5XDD C2Open State FoundationPolicy advisor0-1XDD documentation were added to the map in order to prompt participants to think about what other components may have been left out of the register. The map was printed on a large A1 size sheet of paper to allow participants to annotate it freely. WorksheetW posed 11 guiding questions (see Ap- pendix B) to help participants to assess the map and con- tribute to it. Participants were first given 40 minutes to work individually to complete the worksheet and annotate Map M 0 . Then, participants exchanged their insights with others in the group and continued to annotate the map for another 40 minutes. Group OC had an additional plenary session to share insights among both Ombudsman and CSO staff. System-Theoretic Process Analysis (STPA) STPA consists of 4 steps, which are performed iteratively, starting at higher levels of abstraction and working towards more nuanced representations of components and their inter- relations. STPA is typically conducted in consultation with direct stakeholders such as system designers, managers, pol- icymakers, engineers, and system operators, all of whom have specialised knowledge about different aspects of the system design, deployment, use, and governance (Leveson and Thomas 2018). In this study, we propose involving in- direct stakeholders due to their commitment to the public interest and their differentiated expert knowledge. Given the complexity of the STPA, we asked participants to focus on mapping the system (Step 2), and later used their input to inform a full STPA. The 4 steps are as follows: Step 1: Identify Losses First, the analysts must identify losses which are to be prevented. This requires engaging with direct stakeholders to determine what is at stake in the event of a loss. For example, a municipality using an algo- rithmic system to determine welfare benefits eligibility may identify wrongful denial of benefits as the loss to be pre- vented through the proactive design of safety measures. Step 2: Model the Hierarchical Control Structure Next, the safety engineers model the hierarchical control structure (HCS) which describes the sociotechnical governance archi- tecture, which includes both the process within which the al- gorithmic tool is used and the broader set of processes that contribute to ensuring the system operates within a margin of safety. In our running example, the municipality may have a series of checks and balances in place to prevent and mit- igate wrongful benefits denial, including risk assessments and channels for recourse. The safety engineers may draw on interviews, technical documentation, or other available means in order to understand and reconstruct the feedback and control mechanisms which keep the system safe during operation. The HCS is divided into development and operations. Each block represents an entity which may be a process, in- strument, or organisation. Horizontal arrows represent feed- back channels (upwards) and control actions (downwards). Lateral lines represent information flows. (For an example, see the final system map in Fig. 2.) Step 3: Identify Unsafe Control Actions With the hier- archical control structure in place, the engineers then set out to identify Unsafe Control Actions (UCAs) which are ac- tions that, combined with worst-case environmental condi- tions, in the presence of systemic hazards, can lead to a loss. 1 For example, if a human-in-the-loop is given discretionary power to overturn erroneous recommendations provided by a decision-support tool, but fails to do so (perhaps due to au- tomation bias), this would constitute a UCA of Type 1 (“ac- tion not provided when needed”). This failure to act, com- bined with opaque decision-making logic and overtrust in automation (systemic hazards) and a model error (a worst- case environmental condition), would lead to a loss of citi- zen’s benefits eligiblity. Step 4: Identify Loss Scenarios Finally, the safety engi- neers can use the HSC and UCAs to reason through poten- tial scenarios that may play out in which system hazards result in a loss. For example, a loss scenario may recon- struct how a human-in-the-loop safeguard may break down due to operational pressures. This requires identifying sys- tem hazards and system constraints to prevent those haz- ards from emerging. System hazards are caused by systemic conditions which can contribute to specific hazardous situa- tions. These can be refined into sub-hazards which are lo- calised instances of the broader system hazards (Leveson and Thomas 2018, p. 21). For example, the opacity of a decision-making algorithm poses a system hazard because it affects multiple stakeholders. This manifests as sub-hazards when a caseworker using the algorithm as a decision-support tool, a citizen subject to those decisions, or an ombudsman wants to understand why a decision was made. Participatory System Mapping The participatory mapping workshops resulted in six system maps, one by each participant. These were combined into one by the authors, as shown in Fig. 2. In so doing, the final map combined the plurality of participants’ subjective and socially situated understandings of the Avola system and its governance architecture into a single multifaceted represen- tation that illustrated the diversity of their concerns and ex- pertise. Participants’ contributions ranged from adding miss- ing actors, organisations, institutions, and links among these, to points of interrogation on parts of the map which were not explained by the algorithm register alone. The contri- butions made by each stakeholder group were informed by their positionality – their role in relation to the algorithmic 1 Different types of UCAs (see (Leveson and Thomas 2018, p. 35)): 1. Not provided when needed; 2. Provided when not needed; 3. Provided at the wrong time (too early or too late); 4. Applied for too long or not long enough; 5. Provided in the wrong order; 6. Provided under the wrong conditions or context; 7. Provided to the wrong component or actuator; 8. Provided with the wrong magni- tude or intensity; 9. Provided with incorrect parameters or settings. system, its governance structure, and the citizen. This influ- enced what aspects of the map they thought were missing, and what areas of the map they raised further questions on. The colour-coding in Fig. 2 shows clearly how distinctive these contributions were among the groups. Below, we sum- marise these distinctions. Civil Society Organisations Drawing on their experience working on civil advocacy regarding social impacts of al- gorithmic systems and digital rights, CSO staff’s contri- butions and questions focused on technical aspects of the system, algorithm governance instruments, as well as com- plaints channels. These included expanding the range of complaints channels beyond the municipal complaint chan- nel mentioned in the register to also include those available through the Dutch Data Protection Authority (Autoriteiten Persoonsgegevens), the AI Act, Civil Court, and the Om- budsman. They pointed out that the Freedom of Information Request (WOO) would likely prove a high barrier to access for citizens, and was more likely to be used by expert users such as themselves, Ombudsmen, journalists, or academics. This remark showed how the register already provides some information which is intended for more expert publics rather than for citizens subject to algorithms like Avola. Regarding the risk assessments (both DPIA and FRAIA), CSO staff questioned who had been involved in conduct- ing them, or consulted during the process, and whether these assessments informed each other. They also asked whether these had been conducted once or multiple times, and noted that it was not clear when such assessments had taken place, or whether they were still relevant for the given software version of Avola currently in deployment. Regarding tech- nical aspects, CSO staff asked about what the Avola inci- dent reports contained (whether they were merely error logs or caseworkers’ assessments of individual citizens’ cases), and what was done in response to these reports. They also pointed out that the little information published in the regis- ter about what Avola did said very little about how it was ac- tually implemented in practice (i.e. what type of algorithm it was, what features it used, etc.) Lastly, CSO staff questioned what accountability body Bizzomate was subject to. Municipal staff The contributions by municipal staff fo- cused on the role of various political bodies in governing the system, as well as introducing nuance that was not avail- able through the algorithm register. For example, M1 and M4 explained that the Aldermen (members of the municipal legislative body) were in charge of commissioning both the Avola system itself, as well as the algorithm register. The algorithm register team and Avola’s product owner report back to the Aldermen such things as the results of the risk assessments, which the Aldermen must then rely on to deter- mine whether to approve or reject these systems prior to de- ployment. The Aldermen in turn report to the City Council, above which sit the Ministry of Interior, the House of Repre- sentatives, and finally the Senate. For matters concerning the municipal algorithm register, CSOs and Ombudsmen may engage with the Aldermen or City Council directly. How- ever, should they identify deeper issues stemming from the national register, CSOs and Ombudsmen may address the Avola (welfare benefit eligibility legal automation) Bizzomate file WOO request WOO response file complaint complaint response citizen- caseworker interaction citizen- caseworker interaction Ombudsman Civil Society Organisations public consultation file complaint via Ombudsman oversight mandate comply with oversight requests incident report incident response Audiotrs audit Avola system update Service Level Agreement, contractual obligations, etc. Municipality Citizen National and Municipal Political Bodies policy recommendations (reports, lobbying, etc.) public pressure policy recommendations (reports, lobbying, etc.) commission system to implement welfare eligibility policy Figure 1: A high-level representation of the hierarchical control structure of Avola, showing the control (downward arrows) and feedback (upward arrows) relationships between different stakeholders. (Please see the Supplementary Material for a larger version of the map, and a detail of the hierarchical control structure within the municipality.) political bodies higher up in the hierarchy. Additionally, any public actor may choose to seek redress through Civil Court against either the municipality or the third-party developers of the Avola system itself. Ombudsman staff Ombudsman staff added the least to the maps, likely due to their reservations about how well mapping served their own operational needs and their per- spective of how they seek to understand the systems they examine. O2 noted that “our work is more about the hu- man interest and this is more the systems [point of view]. A completely different way of thinking.” O1 illustrated this further, noting that the citizen-caseworker interaction was “the only human connection” on the map. Further, O2 stated that while the algorithm register gives “the illusion of ex- planatory power”, Ombudsmen are more interested in un- derstanding how the citizen is affected than how the system works. The components they did contribute consisted of support organisations (focused on youth), as well as counsellors who citizens could seek help from. Beyond these few additions, O1 and O2 mostly used map as a prompt to ask questions, such as, is it clear for the citizen which data are being used? (O1) Additionally, they questioned the sufficiency of some of the register’s disclosures, noting that it was not clear what department handled the Freedom of Information Request, thus making them doubt how effective such a channel could be (O1). This enabled us to consider what frustrations citi- zens may feel when using the register themselves. System safety analysis Building on the contributions from the participatory map- ping workshops, we next conducted the full STPA system safety analysis. Here, we share an overview of the STPA in a narrative form that illustrates how successively impli- cating different system actors and components at increasing levels of the hierarchy allows us to gradually broaden the sociotechnical frame – across the technical, operational, or- ganisational, and institutional frames. We do so through a series of Loss Scenarios which span (A) the human-in-the- loop, (B) the complaints procedure, (C) the Ombudsman’s oversight mandate, and (D) political pressure and the public sphere. We also note where participants’ contributions in the precending stages (I-IV) informed these scenarios. In Table 3, we provide an excerpt of the full STPA which breaks down the Loss Scenarios into the corresponding Losses, System Hazards, Subhazards, and UCAs. These are indexed for later reference in the text (L-1, etc.). Loss Scenario A: the human-in-the-loop The algorithm register insists that Avola does not make any decisions automatically, but rather serves as a decision- support tool for caseworkers that act as a human-in-the- loop who has the final say. However, as much literature has shown (e.g. (Green and Chen 2019; Ruschemeier and Hon- Table 3: Excerpt of the STPA analysis. LossSystem HazardSubhazardUnsafe Control Action (UCA)Loss Scenario L-1: Denial of benefits eligibil- ity SH-1.1 Opaque decision logic SH-1.1.b. Automation bias leads to overreliance on Avola by caseworker U-1.1.b.i. Caseworker approves erroneous Avola recommendation A.Human-in-the- loop L-2: System per- formance deteri- oration SH-2.1: Failure to identify errors SH-2.1.a. Failure to learn from ag- gregated complaints U-2.1.a.i. Change request not trig- gered U-2.1.a.i. Designer fails to correct model B. Complaints pro- cedure SH-9.1.a. FRAIA does not report er- ror metrics and performance U-9.1.a.i. CSOs unable to contest Avola on technical grounds D. Public sphere L-3: Inability to contest decision SH-3.2. Lack of sufficient infor- mation (e.g. in Register) SH-3.2.b. Register lacks a well- defined audience U-3.2.b.i. Citizen unable to con- test Avola via complaints proce- dure B. Complaints pro- cedure SH-3.2.c. Register lacks support- ing documentation (e.g. FRAIA and other risk assessments, technical info including error rates, etc.) U-3.2.c. Ombudsman unable to respond to citizen complaint C.Ombudsman oversight mandate drich 2024)), this reliance on the human agent to steer the algorithm rests on strong assumptions (e.g. that the case- worker will be able to catch the algorithm’s errors, for ex- ample, by understanding what factors influenced its output) and are prone to error in many ways (e.g. caseworkers may feel compelled to accept algorithmic outputs perceived as value-neutral or objective). CSO staff C1 noted that while the register stressed the role of the caseworker as the ulti- mate decision-maker, it could not portray the conditions un- der which caseworkers made those decisions (perhaps pres- sured by time constraints to not disagree with Avola and having to write a justification report under vigilance from the Quality Assurance staff who check whether deviations from Avola’s recommendation are justified). Despite this limitation, which may prove beyond the scope of the reg- ister’s capacity, we may interpret the assertion of the case- workers’ discretion as a comforting but shaky reassurance, hiding from citizens such issues as algorithmic bias, opac- ity, and organisational pressures, which may betray the fact that Avola’s innocuous function as a decision-support tool may be superseded by the perceived objectivity of its logic, whether in the eyes of caseworkers themselves or of supe- rior managerial staff from whom they receive this narrative of automated administrative neutrality. We may consider a hypothetical situation in which citizen data which is out-of-date (see map: “[BRP/Suwinet check fails due to out-of-date data]”, raised as a potential issue by Ombudsman staff O1), which may occur for various rea- sons, including human error (e.g. citizens failing to update their own data) or technical issues (e.g. internal server er- rors pertaining to database maintenance). Automation bias (SH-1.1.b.) may lead caseworkers to over-rely on Avola, for example, because they see it as a legal automation model that simply “implements the law” and is thus not subject to spurious data biases or other issues that beset data-driven al- gorithms. Whatever the case may be, it falls upon the case- worker to recognise and identify these errors as such. Should they fail to do so (U-1.1.b.i.), the human-in-the-loop safe- guard falls apart, and the citizen may be wrongfully denied benefits for which they are, in fact, eligible (L-1). In this scenario, a technical error (BRP/Suwinet check fails) leads to an operational error (casework misses a technical error), which is a result of an organisational safety culture of undue trust in legal automation as error-free and objective. Loss Scenario B: the complaints procedure As a result of Loss Scenario A, a citizen may seek recourse for loss L-1 by filing a complaint through the municipality. Complaints are triaged among the relevant departments and then entrusted to the caseworker responsible for the case (as per our interview with G1). However, CSO staff C1 noted that it wasn’t clear from the register how complaints would be handled in practice, leaving them in doubt as to whether filing complaints would be an efficient form of recourse. In the FRAIA documentation, we found that Avola shares incident reports with Bizzomate. However, CSO staff C2 noted that it was not clear what these error reports entailed (e.g., whether they relate to the individual case or are sim- ply technical logs). Furthermore, the FRAIA documentation noted that error metrics were not tracked. As such, it is pos- sible that complaints are treated on a case-by-case basis, and not aggregated and analysed in a way that would reveal sys- tematic error patterns. Failing to identify these systematic errors would mean that the appropriate change requests are not filed (U-2.1.a.i), and thus developers would fail to cor- rect for the errors and deploy a model update (U-2.1.a.i). Loss Scenario C: the Ombudsman’s oversight mandate If the citizen is not satisfied with how their complaint was handled, they may choose to take their issue to the Om- budsman, who can act as a mediator between the citizen and the municipality. One clarification which Ombudsmen may wish to pursue is how the implementation of legal rule au- tomation has been implemented in Avola. In response, mu- nicipal staff may provide the FRAIA documentation which mentions that “legal tests” are performed prior to deploy- ment. However, the FRAIA does not provide any detail as to what those tests entail, preventing the Ombudsman from us- ing existing documentation to scrutinise the implementation details of Avola. As a result, the Ombudsman may be unable to respond appropriately to citizens’ complaints (U-3.2.b.i.). Loss Scenario D: political pressure and the public sphere Eventually, CSOs and Ombudsmen may also choose to pur- sue legal action through Civil Court (suggested as a potential course of action by G4). However, in order to mount a legal case (e.g. against the developer of Avola), they would need to supply evidence of wrongdoing. Both CSO and Ombuds- man staff pointed out that the register’s explication of the FRAIA and DPIA assessments was altogether insufficient. The register simply stated that these had been conducted (”FRAIA: Yes; DPIA: Yes”) but withheld any further elab- oration as to who was involved and what they found While one may assume the risk assessments had not yielded any problematic outcomes, even this is left to the imagination. Thanks to the cooperation of municipal staff, we had the privilege of consulting the FRAIA documentation ourselves. While we did not find any evidence for fundamental rights violations, some of the answers provided to questions on the validation and evaluation of Avola’s performance left much to be desired. For example, when asked about what error metrics were used to evaluate Avola (FRAIA ques- tion 2B.3.6.), it was simply stated that, “it happens, but it is so marginal that no number can be attached to it.” This lack of detail is likely due to insufficient expertise or re- sources to conduct the FRAIA (as per our interview with M4). (While Bizzomate may have a rigorous evaluation pro- cedure in place, this is simply not properly reported in the FRAIA where prompted to do so.) This lack of adequate technical documentation regarding the performance of Avola means that, even after filing a Freedom of Information Request, external independent ob- servers, such as Ombudsmen or CSOs, may be unable to provide evidence of wrongdoing in order to bring forth a credible legal case in Civil Court (U-9.1.a.i.). Discussion On the role for public actors in realising the ambitions of the algorithm register Amid increasing debate about enabling public oversight of algorithmic systems (Sloane et al. 2020; Himmelreich 2023; Williams et al. 2022), we find that the algorithm register falls short of these ambitions in practice. For both actors focused on the impacts for citizens, like Ombudsmen, and those concerned with the innerworkings and governance of the algorithms, like CSOs, the algorithm register raised more questions than it answered about Avola and its governance. While the stated goal of the algorithm register is to inform the public about how algorithmic systems are used and gov- erned (Ministry of Internal Affairs 2025), we found that the information they reveal is too abstract to be meaningful for external actors. The declaration of risk assessments such as FRAIA or DPIA is underwhelming to actors who are in- terested in understanding what those assessments actually found. Furthermore, while the FRAIA instrument is meant to catalyse reflective discussions among a diverse range of stakeholders, including civil society organisations and citi- zens, we found that, in this case, the municipality did not have the capacity to facilitate such forms of participation. Once we gained access to the FRAIA documentation, we found it to be limited for certain critical points, like evalua- tion. As such, the mere declaration of risk assessments may give the public a false sense of security. While these doc- uments can be obtained through a Freedom of Information Request (WOO), having to do so provides an obstacle to ac- cessibility, in an instrument for which the intended purpose is to inform the public. Nevertheless, these instruments are an important step in making algorithms and their governance more accountable to the public. As we have shown in this study, scrutiny from indirect stakeholders, such as Ombudsmen, civil society or- ganisations, and academic researchers, can help to identify the affordances and limitations of these efforts. The open- ness of the municipality to participate in this study bears testament to their willingness to reflect and improve on their algorithm governance practices. We recognise that the duty of documentation (e.g. through the algorithm register or the FRAIA) can be a burden on already under-resourced pub- lic organisations. As a result, it is understandable that these efforts may fall short of the expectations of more critical ac- tors. While most public engagement efforts concerning algo- rithm registers’ design requirements have sought to involve the general public (Ellen Dingemans and van Dalen 2021; Ministry of Internal Affairs 2023), we encourage greater in- clusion of expert communities such as civil society organisa- tions, Ombudsmen, independent researchers, journalists and other actors with the capacity to make use of the informa- tion and to grapple with the heterogeneity of affected publics (Axelsson, Melin, and Lindgren 2013). At the same time, we caution external actors to be mindful of public organisations’ resource challenges, and avoid diminishing such efforts as disingenuous (Ananny and Crawford 2018; Cath and Jansen 2021). System mapping as a participatory practice In our attempts to reconstruct the governance architecture of the Avola system, we found that the different stakeholder groups’ input significantly enriched the information avail- able in the algorithm register. While the register is intended to inform the public about what safeguards are in place to mitigate potential risks (e.g. the FRAIA and DPIA as- sure the public that potential infringements of data protec- tion and fundamental rights have been assessed and safe- guarded against), the potential Loss Scenarios that we found would not have been evident relying on the algorithm reg- ister alone. The practice of sharing our process with partic- ipants enabled us to perform a more comprehensive safety analysis of the system, and to appreciate the forms of infor- mation that the register would need to divulge in order to truly function as an instrument that can support public ac- countability and control. While the initial goal of the mapping exercise was to in- form the STPA safety analysis, we found that the practice of mapping itself can also be seen as a valuable interven- tion. Mapping allowed participants to ask questions to each other, and to the experts in the room. These questions in turn helped the safety experts to review their assumptions, for ex- ample, about the value of mapping for different participants’ existing practices. As such, we encourage safety experts to view mapping to as a means to an ends, but as an ends in itself – to see mapping as an opportunity to invite critical reflection that can inform their own expert safety analysis, and to empower public actors to understand and scrutinise algorithmic systems and their governance. System-theoretic methods from safety science, such as STPA, can help indirect stakeholders to make a compelling case for why the inclusion of certain contextual factors is significant for informing the public about how these risks are mitigated against. At the same time, while these meth- ods provide an ontological basis through which to describe the sociotechnical context in which such algorithms are de- ployed, these methods also impose a certain understanding of these systems onto the world. As both Ombudsman staff noted, system maps lack the ability to tell the human story, and as such, they are unlikely to speak to the concerns of affected citizens. Beyond algorithm registers: mapping sociotechnical systems and their politics Beyond documenting algorithmic systems as technical arti- facts, algorithm registers also detail the operational, organ- isational and institutional processes surrounding their use and governance. In so doing, they bear witness to the com- plex sociotechnical systems in which algorithms are embed- ded, and which are required to make them function (Chen and Metcalf 2024). As shown in our system safety analy- sis, safety is not something that can be achieved by techni- cal means alone (Nouws et al. 2023; Dobbe 2025). Rather, safety requires the orchestration and cooperation of multi- ple actors, procedures, and organisations (de Troya et al. 2025). As such, algorithm registers play an important role in illustrating how algorithmic systems can be better under- stood as sociotechnical systems, requiring technical and so- cial components to be jointly designed (Baxter and Som- merville 2011; Chen and Metcalf 2024). The documentation of measures such as channels for filing complaints, discre- tionary human oversight, and an adequate legal basis serve to contextualise the development and operation of the al- gorithmic system within the wider scope of governance in- struments and resources already existing at the municipality (Dourish 2004; de Troya et al. 2025). At the same time, what components are accounted for, and how, is subject to political forces among socially situ- ated actors (Selbst et al. 2019; de Troya et al. 2025; Mahroof et al. 2025). Every actor has knowledge about different as- pects of the system: municipal staff were able to elucidate the political structures governing the municipality and its efforts, Ombudsmen were able to speak to the concerns of citizens, and CSOs’ critical eye revealed many shortcom- ings in how the register represented the governance struc- ture around Avola. However, the municipality is ultimately subject to political actors, such as Aldermen and the City Council, who depend on expert counsel to inform their de- cisions (including through policy advocacy by Ombudsmen and CSOs). As such, the register is bound to reflect the pub- lic concerns prioritised by these political actors. External or indirect stakeholders who wish to expand the scope or detail covered in the register must then seek to advocate for these changes through appropriate channels such as by policy ad- vocacy efforts. Conclusion Algorithm registers are an important step in facilitating transparency on the use of algorithms in public services. As public organisations continue improving these instru- ments, there is an opportunity for diverse stakeholders to en- gage with the institutional design of algorithmic governance. While algorithm registers have been criticised for lacking a clear audience, and not having teeth, it is important to recog- nise the unresolved challenges that need to be overcome in order to make these instruments more useful. In order to be effective tools for accountability, algorithm registers need to be better aligned with the information needs of different publics. System-theoretic methods like STPA can help to show how algorithm registers integrate with ex- isting governance practices, and can be used to guide a par- ticipatory critical reflection that meaningfully engages vi- tal public actors, like CSOs and Ombudsmen. Our analysis highlights that we still lack sufficiently developed concep- tual frameworks for thinking about, designing, and govern- ing sociotechnical systems in an integrated manner. We have also demonstrated that mapping and documenta- tion practices are inherently political. What becomes visible in a register reflects negotiations among actors with differ- ent roles, forms of expertise, and institutional power. This raises a final, critical question: who has a say in what gets documented? At the same time, our study of the algorithm register insists that algorithmic systems must be understood across the technical, operational, organisational, and institu- tional contexts that make up the sociotechnical systems in which they are embedded. References Ananny, M.; and Crawford, K. 2018. Seeing without know- ing: Limitations of the transparency ideal and its application to algorithmic accountability. new media & society, 20(3): 973–989. Andersson, C.; Hallin, A.; and Ivory, C. 2022. Unpacking the digitalisation of public services: Configuring work dur- ing automation in local government. Government Informa- tion Quarterly, 39(1): 101662. Axelsson, K.; Melin, U.; and Lindgren, I. 2013. Public e- services for agency efficiency and citizen benefit—Findings from a stakeholder centered analysis. Government informa- tion quarterly, 30(1): 10–22. Baxter, G.; and Sommerville, I. 2011. Socio-technical sys- tems: From design methods to systems engineering. Inter- acting with computers, 23(1): 4–17. Binns, R. 2018. Algorithmic accountability and public rea- son. Philosophy & technology, 31(4): 543–556. Bj ̈ ornsd ́ ottir, S. H.; Jensson, P.; Thorsteinsson, S. E.; Dokas, I. M.; and Ingason, H. T. 2023. Aligning Stakeholders and Actors: A New Safety and Security-Based Design Approach for Major National Infrastructures. Sustainability, 16(1): 328. Blacktrace Mergers & Acquisitions. 2024.Overname MendixpartnerBizzomateversterkthyperautoma- tion propositie Ciphix.https://w.blacktrace.nl/nl/ transacties/transactie/overname-van-mendix-platinum- partner-bizzomate-versterkt-hyperautomation-propositie- van-ciphix.html. Burrell, J. 2016. How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big data & society, 3(1): 2053951715622512. Buszydlik, A.; Altmeyer, P.; Dobbe, R.; and Liem, C. C. 2025. Understanding the Affordances and Constraints of Explainable AI in Safety-Critical Contexts: A Case Study in Dutch Social Welfare. In International Conference on Elec- tronic Participation, 118–136. Springer. Cath, C.; and Jansen, F. 2021. Dutch Comfort: The limits of AI governance through municipal registers. arXiv preprint arXiv:2109.02944. Cellard, L. 2020. Theatres of algorithmic transparency: a post-digital ethnography. Ph.D. thesis, University of War- wick. Chalke, J. 2023. Implications and Challenges of AI for Par- liamentary Ombuds Work in Canada. Canadian Parliamen- tary Review, 46(2). Chen, B. J.; and Metcalf, J. 2024. Explainer: A sociotechni- cal approach to AI policy. Data & Society. Costanza-Chock, S. 2020. Design justice: Community-led practices to build the worlds we need. The MIT Press. Dahlvik, J. 2022. Access to administrative justice in the dig- ital era: Contact possibilities and the personal encounter in public ombuds institutions worldwide. Recht der Werke- lijkheid. Journal of Empirical Research on Law in Action, 43(2): 48–67. de Troya, ́ I.; Kernahan, J.; Doorn, N.; Dignum, V.; and Dobbe, R. 2025. Misabstraction in Sociotechnical Systems. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, 1829–1842. Delfos, J.; Zuiderwijk, A. M.; van Cranenburgh, S.; Chorus, C. G.; and Dobbe, R. I. 2024. Integral system safety for machine learning in the public sector: An empirical account. Government Information Quarterly, 41(3): 101963. Delgado, F.; Yang, S.; Madaio, M.; and Yang, Q. 2023. The participatory turn in AI design: Theoretical foundations and the current state of practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mecha- nisms, and Optimization, 1–23. D’ignazio, C.; and Klein, L. F. 2023. Data feminism. MIT press. Dobbe, R. 2022. System Safety and Artificial Intelligence. In 2022 ACM Conference on Fairness, Accountability, and Transparency, 1584–1584. Dobbe, R. 2025. AI Safety is Stuck in Technical Terms– A System Safety Response to the International AI Safety Report. arXiv preprint arXiv:2503.04743. Dourish, P. 2004. What we talk about when we talk about context. Personal and ubiquitous computing, 8: 19–30. Ellen Dingemans, M. S., Fenna Bijster; and van Dalen, B. 2021.Informatiebehoeften van burg- ers over de inzet van algoritmes door overheden. https://openresearch.amsterdam/en/page/82112/citizens-of- amsterdam-informing-about-algorithms-is-a-must. Escher, N.; Bilik, J.; Banovic, N.; and Green, B. 2024. Code- ifying the Law: How Disciplinary Divides Afflict the De- velopment of Legal Software. Proceedings of the ACM on Human-Computer Interaction, 8(CSCW2): 1–37. Eubanks, V. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press. European Centre for Not-for-profit Law and Danish Institute for Human Rights. 2025. A Guide to Fundamental Rights Impact Assessments (FRIA). Felzmann, H.; Villaronga, E. F.; Lutz, C.; and Tam ` o- Larrieux, A. 2019.Transparency you can trust: Trans- parency requirements for artificial intelligence between legal norms and contextual concerns. Big Data & Society, 6(1): 2053951719860542. Fenster, M. 2005. The opacity of transparency. Iowa L. Rev., 91: 885. Frederik, J. 2021. De tragedie achter de toeslagenaffaire. De Correspondent. Gerards, J.; Sch ̈ afer, M. T.; Muis, I.; Vankan, A.; et al. 2022. Fundamental rights and algorithms impact assessment (FRAIA). Rijksoverheid. Green, B.; and Chen, Y. 2019. The principles and limits of algorithm-in-the-loop decision making. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW): 1–24. Greenbaum, J.; and Kyng, M. 1991. Introduction: Situated Design. In Design at Work: Cooperative Design of Com- puter Systems, 1–24. CRC Press. Grimmelikhuijsen, S. 2023. Explaining why the computer says no: Algorithmic transparency affects the perceived trustworthiness of automated decision-making. Public Ad- ministration Review, 83(2): 241–262. Guti ́ errez, F.; Charleer, S.; De Croon, R.; Htun, N. N.; Goetschalckx, G.; and Verbert, K. 2019. Explaining and ex- ploring job recommendations: a user-driven approach for in- teracting with knowledge-based job recommender systems. In Proceedings of the 13th ACM Conference on Recom- mender Systems, 60–68. Haraway, D. 1988. Situated knowledges: The science ques- tion in feminism and the privilege of partial perspective. Feminist Studies. Haug, K. B.; Haug, A.; and Hansen, J. U. 2026. Algorithmic profiling of the unemployed: A case study and a framework for understanding legitimization processes. Government In- formation Quarterly, 43(1): 102103. Himmelreich, J. 2023. Against “democratizing AI”. AI & SOCIETY, 38(4): 1333–1346. IA Ciudadana. 2025. Making Algorithm Registers Work for Meaningful Transparency. Iwanska, K.; Skoric, V.; Fanucci, F.; Keskindemir, B.; and Kokkula, S. 2024. Towards an AI Act that serves people and society: strategic actions or civil society and funders on the enforcement of the EU AI Act. Janssen, M.; and Kuk, G. 2016. The challenges and limits of big data algorithms in technocratic governance. Kaeseberg, T. 2019. The Code-ification of law and its po- tential effects. Computer Law Review International, 20(4): 107–110. Kallina, E.; Bohn ́ e, T.; and Singh, J. 2025. Stakeholder Participation for Responsible AI Development: Disconnects Between Guidance and Current Practice. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, 1060–1079. Katell, M.; Young, M.; Dailey, D.; Herman, B.; Guetler, V.; Tam, A.; Bintz, C.; Raz, D.; and Krafft, P. 2020. Toward situated interventions for algorithmic equity: lessons from the field. In Proceedings of the 2020 conference on fairness, accountability, and transparency, 45–55. Kroll, J. A.; Huey, J.; Barocas, S.; Felten, E. W.; Reidenberg, J. R.; Robinson, D. G.; and Yu, H. 2017. Accountable Al- gorithms. University of Pennsylvania Law Review, 165(3): 633. Kuo, T.-S.; Shen, H.; Geum, J.; Jones, N.; Hong, J. I.; Zhu, H.; and Holstein, K. 2023. Understanding Frontline Work- ers’ and Unhoused Individuals’ Perspectives on AI Used in Homeless Services. In Proceedings of the 2023 CHI Con- ference on Human Factors in Computing Systems, CHI ’23. New York, NY, USA: Association for Computing Machin- ery. ISBN 9781450394215. Leveson, N. 2004. A new accident model for engineering safer systems. Safety Science, 42(4): 237–270. Leveson, N. G. 2011. Engineering a safer world: Systems thinking applied to safety. The MIT Press. Leveson, N. G.; and Thomas, J. P. 2018. STPA Handbook. MIT Partnership for Systems Approaches to Safety and Se- curity (PSASS). Madise, ̈ U.; and Pilvik, K. 2024. The role of Ombuds institu- tions in a rush of digitalization. In La Ley del Ararteko. Con- struyendo el futuro: una reflexi ́ on sobre las defensor ́ ıas del pueblo: 2023ko ekainaren 12an eta 13an Vitoria-Gasteizen egindako mintegia= Seminario celebrado en Vitoria-Gasteiz los d ́ ıas 12 y 13 de junio de 2023, 239–264. Eusko Lege- biltzarra= Parlamento Vasco. Madsen, C. Ø.; Lindgren, I.; and Melin, U. 2022. The ac- cidental caseworker–How digital self-service influences citi- zens’ administrative burden. Government Information Quar- terly, 39(1): 101653. Mahroof, K.; Weerakkody, V.; Hussain, Z.; and Sivarajah, U. 2025. Navigating power dynamics in the public sector through AI-driven algorithmic decision-making. Govern- ment Information Quarterly, 42(3): 102053. Meeri Haataja, L. v. d. F.; and Rautio, P. 2020. Public AI Registers: Realising AI transparency and civic participation in government use of AI. https://openresearch.amsterdam/ en/page/73074/public-ai-registers. Meijer, A. 2014. Transparency. In Bovens, M.; Goodin, R.; and Schillemans, T., eds., The Oxford Handbook of Public Accountability. Oxford University Press.ISBN 9780199641253. Ministry of Internal Affairs, M. v. B. Z. e. K. 2025. About the Algorithm Register. https://algoritmes.overheid.nl/en/ footer/over. Accessed on 30-12-2025. Ministry of Internal Affairs, U. I., (Ministerie van Binnen- landse Zaken en Koninkrijksrelaties). 2023. Doelgroepen- analyse Algoritmeregister: Kwalitatief onderzoek naar hoe (potenti ̈ ele) gebruikers het lgoritmeregister ervaren.Ac- cessed on 30-12-2025. Murray-Rust, D.; Alfrink, K.; and Zaga, C. 2025.To- wards Meaningful Transparency in Civic AI Systems. arXiv preprint arXiv:2510.07889. Nieuwenhuizen, E. 2025.Algorithm Registers: A Box- Ticking Exercise or Meaningful Tool for Transparency? In- formation Polity, 0(0). Norval, C.; Cornelius, K.; Cobbe, J.; and Singh, J. 2022. Disclosure by Design: Designing information disclosures to support meaningful transparency and accountability. In Proceedings of the 2022 ACM Conference on Fairness, Ac- countability, and Transparency, 679–690. Nouws, S.; Martinez De Rituerto De Troya, ́ I.; Dobbe, R.; and Janssen, M. 2023. Diagnosing and Addressing Emer- gent Harms in the Design Process of Public AI and Algo- rithmic Systems. In Proceedings of the 24th Annual Inter- national Conference on Digital Government Research, 679– 681. Paudyal, P.; and William Wong, B. 2018. Algorithmic opac- ity: making algorithmic processes transparent through ab- straction hierarchy. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 62(1): 192–196. Peeters, R.; and Widlak, A. C. 2023. Administrative exclu- sion in the infrastructure-level bureaucracy: The case of the Dutch daycare benefit scandal. Public Administration Re- view. Popa, D. M. 2025. Frontrunner model for responsible AI governance in the public sector: The Dutch perspective. AI and Ethics, 5(3): 2789–2799. Raji, I. D.; and Dobbe, R. I. J. 2020. Concrete problems in AI safety, revisited. In Workshop on Machine Learning In Real Life at the International Conference on Learning Representations. Addis Abeba. Rismani, S.; Shelby, R.; Smart, A.; Jatho, E.; Kroll, J.; Moon, A.; and Rostamzadeh, N. 2023. From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–18. Rotterdam Court of Auditors. 2024. Kleur bekennen: vervol- gonderzoek naar algoritmes. https://rekenkamer.rotterdam. nl/onderzoeken/kleur-bekennen/. Published: 2024-05-13. Ruschemeier, H.; and Hondrich, L. J. 2024. Automation bias in public administration–an interdisciplinary perspec- tive from law and psychology. Government Information Quarterly, 41(3): 101953. Selbst, A. D.; Boyd, D.; Friedler, S. A.; Venkatasubrama- nian, S.; and Vertesi, J. 2019. Fairness and abstraction in sociotechnical systems. In Proceedings of the conference on fairness, accountability, and transparency, 59–68. Sloane, M.; Moss, E.; Awomolo, O.; and Forlano, L. 2020. Participation is not a design fix for machine learning. arXiv preprint arXiv:2007.02423. van Vliet, M.; Schuitemaker, N.; Espana, S.; van de Weerd, I.; and Brinkkemper, S. 2024. Defining and Implement- ing Algorithm Registers: An Organizational Perspective. In ECIS 2024 Proceedings. 5. Wieringa, M. 2020. What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability. In Proceedings of the 2020 conference on fairness, accountability, and transparency, 1–18. Wieringa, M. 2023. “Hey SyRI, tell me about algorithmic accountability”: Lessons from a landmark case. Data & Pol- icy, 5: e2. Williams, R.; Cloete, R.; Cobbe, J.; Cottrill, C.; Edwards, P.; Markovic, M.; Naja, I.; Ryan, F.; Singh, J.; and Pang, W. 2022. From transparency to accountability of intelligent systems: Moving beyond aspirations. Data & Policy, 4: e7. Zejnilovi ́ c, L.; Lavado, S.; Mart ́ ınez de Rituerto de Troya, ́ I.; Sim, S.; and Bell, A. 2020. Algorithmic long-term un- employment risk assessment in use: counselors’ perceptions and use practices. Global Perspectives, 1(1). Zuiderwijk, A.; Chen, Y.-C.; and Salem, F. 2021. Impli- cations of the use of artificial intelligence in public gover- nance: A systematic literature review and a research agenda. Government information quarterly, 38(3): 101577. Appendix A: Interview protocols Interview protocol regarding the algorithm register Purpose of the register Q1. What is your role at the municipality? Q2. What is the intended purpose of the municipal algorithm register? Q3. Are the objectives of the municipal register the same as those of the national register? Reflecting on the current register Q4. How and to what extent are the objectives stated in the national register being met by the municipal register? Q5. What do you think the register currently does well? Desired future state of the register and challenges to get there Q6. What do you think can be improved? Q7. What are you currently doing to improve the register? Q8. What challenges have you faced in implementing those improvements? Q9. Do you have any further comments that you believe I may have overlooked in my questions? Interview protocol regarding the Fundamental Rights Impact Assessment (FRAIA) Q1. Can you please introduce yourself and say a little bit about your role at the municipality? Q2. Who else was involved in the FRAIA of the Avola system, besides yourself? Q3. What insights did the FRAIA process give you about the system? (e.g. lack of change management procedure; ”instruments for monitoring, steering and accountability are missing or still under development”; etc.) Q4. Were any actions taken after doing the FRAIA? Q5. What do you do if you are unable to answer a question? Q6. Once the FRAIA is first completed, is it reviewed by someone else before it is declared finished? Q7. How can citizens ”request their records that include advice from Avola”? (3.6.2) How are they made aware that they can do this? Q8. Did you raise any issues found through FRAIA with Bizzomate? How did they respond? Q9. Why was it decided to deploy the system despite there not being ”proper tools provided for evaluation, auditing, and assurance of the algorithm”? (3.7.1.) Appendix B: Worksheet Q1. Where would you place yourself on the map? Q2. What components of the map are you connected to, and in what way? Q3. What is missing from the map? (e.g. components, connections) Q4. What other elements of the map come into view through your presence on the map? (e.g. other organisations, mechanisms, procedures, etc. that you are ore directly connected to) Q5. What part of the map would you like more detail on? If so, why? Q6. What abstractions or assumptions did you have to make? Q7. In your expereince of working with complaints procedures, is there any information missing from its representa- tion on the map? Q8. What other experiences did you draw on when mapping? (e.g. working with a citizen on a similar case in the past...) Q9. What additional resources would you use to map the system (besides the register)? Q10.i. Are there any limitations to the mapping approach we used which make it hard to represent certain aspects of the system? Q10.i. If so, do you have any suggestions of what can be improved and how? Q11. What value do you see in mapping of algorithmic systems for understanding their governance structure and po- tential negative impacts? In other words, how could a map inform your work? (e.g. identifying points of intervention, lack of detail about specific areas; helping me to see the bigger picture so that I know where to focus) Appendix C: Hierarchical Control Structure Algorithm Register Algorithm Register team maintain Avola Product Owner request info provide info DPO FRAIA team conduct Service Level Agreement Processing Agreement (GDPR compliance) FRIA Steering Committee (incl. Director W&I, Alderman) FRIA FRAIA R Socrates (determines subsidy amount) RK Data Protection Impact Assessment (DPIA) R Freedom of Information request (WOO) R Incident report (technical) Incident report (operational) Change management Change requests (by Gemeente) System updates (by Bizzomate) Thomas (Web Portal) back- end if eligible, calculate amount provide subsidy amount R FRIA Benefits Eligibility Process Thomas (Web Portal) front- end Complaints procedure R Complaints team handle complaint Kwaliteits- medewerker assign complaint to relevant staff Caseworker Check that deviation from Avola recommendation was justified incident reports Avola decision considered by caseworker caseworker controls eligibility decision Avola report update decision Avola (Business Rule Engine) determines elgibility ensure GDPR compliance assess update based on inc.rep., ch.,req., sys.up. incident reports process application eligibility recommendation R Municipality Alderman submit incident reports, change requests responses/updates based on incident reports, change requests, system updates Bizzomate Complaints Form (Web Portal) complaint submitted WOO response file WOO request seek info. provide information about Avola citizen consultation to inform the "incidents" [C1Q4] submit application and data on income and benefits citizen ↔ caseworker interaction response to complaint Citizen file a complaint Figure 2: A detail of the hierarchical control structure of the municipality, part of the design and governance structure around the Avola system.)

Instant research discovery