Introduction
“Across the federal government, we are beginning to observe the dawn of a new chapter—perhaps even a digital revolution—in how government does its work.”1 This “new chapter,” as described in a new report commissioned by the Administrative Conference of the United States (ACUS Report),2 is the use of artificial intelligence (AI)3 in the federal government—what the report terms “algorithmic governance.”4 The ACUS Report documents 157 instances of algorithmic governance in several federal departments, agencies, and sub-agencies.5
The Social Security Administration (SSA), for example, has implemented a system called Insight, which administrative law judges (ALJ) and the SSA Appeals Council use for quality assurance.6 At the hearing level, an ALJ uses Insight to identify problems in draft decisions, such as citing incorrect authority or certain internal discrepancies.7 The SSA Appeals Council can use the program to find inconsistencies in the ALJ decision from which the claimant appealed.8
The Securities and Exchange Commission (SEC) uses several algorithmic systems in its enforcement efforts.9 One such system, the Corporate Issuer Risk Assessment, uses around 200 metrics to identify irregular patterns in corporate financial statements.10 The system uses information from past enforcement actions and past filings.11 SEC employees analyze and use the results to allocate its enforcement resources.12
Besides enforcement, federal agencies are using algorithmic governance to aid in adjudication. The U.S. Patent and Trademark Office uses AI tools to adjudicate patent and trademark applications.13 One prototype tool uses deep learning to identify existing marks from a database that are like the applied-for mark.14 Other tools used by the agency aim to automate mark and patent classification.15
The ACUS Report highlights several legal, policy, and institutional concerns about algorithmic governance, including: (1) the possibility for bias in the algorithms16 or the violation of due process rights;17 (2) the lack of transparency and public accountability;18 and (3) the danger that regulatory targets might unfairly game or exploit the algorithmic models to their advantage.19 Scholars and commentators have raised these issues before, especially in the context of algorithmic governance at the state and local level.20
What scholarly discourse lacks is any significant discussion into how algorithmic governance may upset and undermine the “administrative separation of powers.”21 As proposed by Professor Jon Michaels, the theory of the administrative separation of powers contends that the administrative state remains legitimate and consistent with our constitutional precepts so long as it retains the “dispositional characteristics” of the traditional branches of government.22 Michaels argues that the three classic branches of government—the executive, judicial, and legislative branches—have been reconstructed in the administrative state.23 Playing the role of the executive is the politically accountable agency leadership.24 The judiciary is found in the disinterested, independent, legally-protected, and professionalized civil service.25 And standing in as the legislature is a vibrant and diverse civil society—the members of which use administrative processes to debate, educate, and keep the other two subconstitutional branches accountable.26 Maintaining this administrative separation of powers promotes our constitutional commitments to a constrained and rivalrous government, and for that reason, qualifies the administrative state as a worthy heir to the Framers’ tripartite regime.27
Michaels has sounded the alarm on how this subconstitutional framework is being undone by the rise of privatization.28 Privatization—whether in the form of contracting out important federal tasks29 or “marketizing” the civil service30—usurps power from the bureaucracy and the civil society and amasses it within agency leadership through various means.31
Like privatization’s constitutional threat, this Essay contends that algorithmic governance threatens to dismantle the subconstitutional checks and balances within administrative agencies by creating concentrated, unchecked power at the agency leadership level.
I. A New Threat to the Administrative Separation of Powers
Algorithmic governance could undermine the administrative separation of powers because its use consolidates agency leader control at the expense of the civil service and civil society. Algorithmic governance does this by allowing leaders to: (1) unilaterally advance their agenda; (2) marginalize and erode the civil service; and (3) restrain public participation.
A. Unilateral Advancement of Agency Leaders’ Agenda
The first way algorithmic governance creates concentrated agency leader power is by allowing leaders to unilaterally advance their policy agenda. Algorithmic governance, or a “robo-bureaucrat,” is programmed to engage in a specified function and abide by certain protocol—all of which are likely to be directly informed by agency leaders.32 In other words, a robo-bureaucrat is more or less programmed to do what the agency leaders want and it must abide by its programming. If a robo-bureaucrat, for some reason, goes rogue33 and pushes back against the agency head, agency leadership can likely replace or reprogram it with little resistance.34 This compliant robot stands in sharp contrast to the independent, professionalized, legally protected, and often rivalrous civil service.35
B. Marginalization & Erosion of the Civil Service
The second way algorithmic governance consolidates agency leader control is by marginalizing the civil service. It does this in at least three ways. First, government agencies are likely to use algorithmic governance to replace important civil service discretionary duties and work altogether.36 There is little indication that governments will end their foray into algorithmic governance: it is hard to imagine cash-strapped agencies declining automation when automation’s advocates promise increased efficiency, productivity, cost-savings, and quality of public services.37 Indeed, one report suggests that AI and machine learning may replace or change 130,000 federal jobs in the next two decades.38 As the ACUS Report observed, “[t]he advent of algorithmic enforcement may . . . supplant expertise within the federal bureaucracy, exacerbating a perceived trend toward politicized federal administration and the hollowing out of the administrative state.”39
Although algorithmic governance has yet to displace the federal civil service in any significant way, as the technology advances in sophistication, government agencies may use AI to shape and replace “discretion at all levels of bureaucracy.”40 Indeed, it is not hard to imagine the federal government replacing workers with automated systems that would provide licenses to pilots, close factories violating environmental protection regulations, and find that a proposed merger violates antitrust law.41
Labor unions representing federal employees appear to lack any power to fight back against AI displacing civil service jobs and duties. To begin with, the Federal Service Labor-Management Relations Act (FSLMR)42—the act governing labor relations in most of the federal public sector—provides that management has the right “to assign work, to make determinations with respect to contracting out, and to determine the personnel by which agency operations shall be conducted.”43 So federal agencies have the unilateral right to control what duties to assign and to whom or what positions job responsibilities will be assigned.44 And in a similar vein, agencies also may unilaterally contract out certain functions.45 It follows that management rights could be construed to cover an agency’s decision to replace an employee’s job responsibilities with a robo-bureaucrat. As a result, a labor union likely cannot, for example, demand to bargain over the agency’s decision to supplant a civil service employee’s duties with a machine.
And although the FSLMR provides that agencies and unions may negotiate the technology, methods, and means of performing work, this negotiation may only occur “at the election of the agency.”46 So even when an agency deploys AI to merely complement a civil service worker’s decision making, the agency’s decision is likely only negotiable when the agency decides it is negotiable. Of course, even though a union likely cannot negotiate over the decision to replace duties or federal workers with a robo-bureaucrat, the negotiable effects of the decision may be within the duty to bargain.47
Several government agencies at the state level have already begun replacing important state worker decisions with those of machines. For example, in 2013, the Michigan Unemployment Insurance Agency implemented MiDAS, an algorithmic decision-making system used to detect unemployment insurance fraud.48 By implementing this system, Michigan sought to save the state money and replace around 400 state workers.49 MiDAS ended up falsely accusing over 34,000 unemployed people of fraud.50 Across other state and local agencies, algorithmic governance is replacing human decision making in areas such as social services and criminal justice.51
Second, algorithmic governance creates a vulnerable civil service. As discussed, when AI becomes more advanced and widespread in government, it is probable that certain swaths of the civil service will be in danger of being replaced.52 This threat will contribute to public workers’ feelings of job insecurity—feelings already exacerbated by nationwide efforts to remove for-cause protection from the civil service.53
And the advent of what may be characterized as “robo-boss”—an algorithmic system used to manage employees—may leave workers in the civil service even more defenseless. For example, a Texas school district implemented a teacher appraisal algorithmic system to make termination decisions allegedly using data from classroom statistics throughout Texas, performance of the teacher’s students, and the teacher’s performance record.54 A lawsuit over the school district’s use of this system argued, in part, that there was no way teachers could verify the accuracy of the algorithm’s scores.55 The court agreed, observing that uncontradicted testimony from the plaintiffs’ expert confirmed that the algorithm’s teacher scores would “remain a mysterious ‘black box,’ impervious to challenge.”56 So “teachers have no meaningful way to ensure correct calculation of their . . . scores, and as a result are unfairly subject to mistaken deprivation of constitutionally protected property interests in their jobs.”57 And thus a robo-boss’s inscrutable decision-making processes—like the system in Texas—could prevent public workers from challenging, in any meaningful way, a robo-boss’s employment decisions.
Third, algorithmic governance creates a more compliant civil service. With the threat of intelligence automation looming, the bureaucracy is less likely to challenge agency leaders’ decisions and policies for fear of being replaced by an automaton.58 For that reason, even though the civil service might still perform certain governmental functions, the robo-bureaucrat threat would create a more compliant and subservient public workforce.59 This result would further an agency leader’s pursuit of securing unitary control over the administrative state.
In a similar vein, even when a civil servant uses or oversees an algorithmic governance system, that worker may be subject to “automation bias”: “the over-reliance of decision-makers on automated predictions, even when such deference is unreasonable and mistaken.”60 For example, “[u]nder the influence of automation bias, workers will likely adopt a computer’s suggested eligibility determinations and benefit calculations.”61 Or an ALJ may review AI-created content in a mechanical manner, without second-guessing the automated results.62 All of this results in a civil service more likely to comply with—and less likely to question—a robo-bureaucrat’s decision or decision-making process.
C. Restraint on Public Participation
The final way algorithmic governance consolidates agency leader power is by restraining public participation. An agency’s use of a robo-bureaucrat will likely make it harder for civil society to hold the agency accountable.63 One reason is that “many of the more advanced AI tools are not, by their structure, fully explainable.”64 It may be impossible to determine how a robo-bureaucrat made a decision, and therefore, it will be harder for the public to peer inside its decision-making process.65 And public accountability is even more constrained when a third-party AI developer refrains from divulging information about the programs to protect its trade secrets or other confidential information.66 Given this opaqueness, when automated systems effectively constitute changes to agency rules, for example, “[c]itizens cannot see or debate these new rules.”67
This AI “black box” also has profound consequences on legal accountability, in part, because much legal doctrine is built on notions of human behavior, such as the doctrines of intent and causation.68 That is why AI may disrupt doctrines such as Article III standing (which requires causation)69 or the rational basis test (which requires a justification for government action).70 The opacity and complexity of automated systems also works to prevent an agency from providing sufficient individual due process before depriving a person of liberty and property using that system.71 Specifically, these systems often undermine the due process requirements of notice and an opportunity to be heard.72
In sum, because algorithmic governance appears likely to promote the leadership’s agenda, marginalize the civil service, and restrain participation by civil society, algorithmic governance threatens to consolidate agency leader control within the agency. This concentrated power jeopardizes the crucial balancing role the civil service and civil society play in the administrative separation of powers. As a result, the rise in algorithmic governance increases the risk that agency heads will be standing alone among a coterie of obedient robo-bureaucrats to advance their agenda—a result inconsistent with our core constitutional commitments to a constrained and rivalrous government, and in violation of the administrative separation of powers.
II. Countering the Threat
Although I leave to later scholarship more detailed recommendations to alleviate this Essay’s concerns with algorithmic governance, to begin taking the first steps in this direction, this Essay proposes that Congress amend federal public sector labor law to require collective bargaining over an agency’s decision to use algorithmic governance.
To put this proposal in context, suppose that the U.S. Department of Labor’s (DOL) Wage and Hour Division (WHD) is considering whether to implement a system that will use AI to automate its investigative and enforcement functions. The WHD reasons that this system will save it thousands of dollars. The automaton identifies businesses most likely to have committed a wage and hour violation based on certain characteristics of its industry, its past violations, and records of complaints. It then automatically requests the business payroll records, time records, government contracts, annual financial statements, and other pertinent information. Based on this information, it creates investigatory lines of questions and specific topics that a human investigator is to ask employees. The WHD investigator then arrives on the employer’s worksite, interviews employees, and inputs their answers directly into the system. Based on this information, the AI system determines whether the employer violated the law, appropriate corrective actions the employer must take, the amount of back wages due (if applicable), and the overall strength of the WHD’s legal position.
Under this Essay’s proposal, before deciding whether to adopt this AI system, the DOL must notify the union representing the WHD investigators and allow the union to negotiate over this system. The union could, for example, request records and evidence related to the new system. It could then propose that the system only analyze complaints and identify employers to investigate; all the other tasks would be assigned to WHD investigators. The union might also propose specific criteria, which the system would have to consider before determining whether a business violated the law. Or the union could propose that rather than implement the system, the union would address the underlying concern: cutting costs. The union might propose cuts to WHD investigators’ salaries or bonuses, or other cost-cutting measures. This proposal might alleviate some of the WHD’s concerns. The agency, of course, must negotiate in good faith and consider the union’s proposal.73 When appropriate, the DOL would offer counterproposals.
If the parties cannot reach an agreement and declare impasse, they would then follow the established procedures in federal public sector labor relations. First, the parties are encouraged to find a third-party neutral to mediate the dispute.74 Second, if mediation fails, either party may seek help from the Federal Service Impasses Panel75—an entity comprised of presidential appointees76—that has the power to impose contract terms through a final action.77 The merits of the Federal Service Impasses Panel’s decision may not be appealed.78 This whole process may take more than a year,79 during which the WHD would generally be prohibited from implementing its algorithmic enforcement system.80
Requiring bargaining over algorithmic governance alleviates several concerns raised by this Essay. The economic and political costs that this long and arduous process impose reduces the risk that agency leadership will use AI to circumvent the civil service. The proposal gives the civil service—through those employees’ elected representative—a voice into how algorithmic governance is developed and implemented. As a result, agency leaders will be less likely to create and implement the system to replace workers or civil service discretionary decision making. And with the threat of losing their jobs to a machine somewhat subsided, the civil service should find themselves instilled with a bit more security in their jobs.
Indeed, assuming the agency adopts the algorithmic system, workers should have a better idea how the algorithmic system works (and how it is not supposed to work). This is because the union will be more encouraged to request information about the system during bargaining and propose to the agency what the system should and should not do. This information should presumably be available to all members of the union likely affected by the algorithmic system. For that reason, if the algorithmic governance program makes a decision contrary to a bureaucrat’s moral or professional judgment, the bureaucrat overseeing it may be more likely to challenge the program’s decisions. And with a fuller understanding of the system’s purpose and functioning, bureaucrats should be less likely to harbor automation bias.
Conclusion
As the ACUS Report suggests, the prevalence of algorithmic governance at the federal level is likely to rise.81 Indeed, the Trump administration issued an executive order to promote federal agencies’ AI research and development.82 And in response to the recent COVID-19 pandemic, commentators are already proposing algorithmic governance tools to help stop further infections.83 To be sure, algorithmic governance has the potential to benefit the public by increasing efficiency, providing additional programs and initiatives, and augmenting enforcement efforts. But along with its promise, comes its potential peril. Policymakers, government officials, and others concerned about government accountability should ensure that the use of algorithmic governance does not lead to unchecked, concentrated power. This Essay provides a useful starting point to that end. Considering algorithmic governance through the lens of the administrative separation of powers, as this Essay seeks to do, helps to further our core constitutional commitments. The public will benefit from this consideration.