This blog is post 1 of the SLSA blog series ‘Empirical Approaches to the Rule of Law’, which takes a socio-legal and citizen-focussed approach to the rule of law, exploring its social foundations, innovative methods, and perspectives from Hungary, Poland, Germany, the Netherlands and Serbia. The series is guest edited by Dr Erin Jackson, postdoctoral researcher at the University of Groningen on the comparative research project CITIZENS-LAW, which aims to strengthen the rule of law in the EU through mixed method research on societal conceptions of law.

Empirical Approaches to the Rule of Law

Marc Hertogh, Professor of Law & Society, University of Groningen, The Netherlands

This blog post is a shortened version of: Marc Hertogh, ‘Empirical Approaches to the Rule of Law: Contours and Challenges of a Social Science that Does Not Quite Yet Exist,’ Annual Review of Law and Social Science 20 (2024).

Introduction

In theory, the rule of law guarantees important values like equality, government transparency, and access to justice. In practice, however, these guarantees are increasingly under threat. This is not only true for well-known examples like Hungary or Poland [1], but also for established democracies like the United Kingdom, the United States and the Netherlands [2]. According to a recent study, we are in the middle of a ‘global rule of law recession’ and it is estimated that more than 6 billion people worldwide live in a country where the rule of law has weakened between 2022 and 2023 [3]. This development not only has far-reaching political and social consequences, but it also raises questions about the way in which scholars study the rule of law. Over the years, a small library of books and papers has been written about the rule of law. However, none of these studies really help us to understand the current trend of rule of law erosion. In this blog I will therefore explore the contours of a new – empirical – approach to the rule of law.

For a long time, the rule of law used to be the exclusive domain of lawyers and legal theorists. Their research focuses on the legal foundations of the rule of law. For example, what are important principles of the rule of law? What is the role of the judiciary and other legal institutions? And what is the legal background of formal rules and procedures? These are, of course, relevant questions. But from this perspective, another dimension of the rule of law is often overlooked. In addition to these legal foundations, we should also focus on the social foundations of the rule of law. How, for example, do the institutions of the rule of law function in practice? What role does the law play in the everyday lives of ordinary citizens? And how much do people support the rule of law? These are not legal or normative questions, but these questions can only be answered with empirical research. Against this background, the past three decades have seen an increasing interest in empirical studies of the rule of law. I will review this emerging field and I will discuss some challenges for future research.

Mapping, Ranking and Understanding the Rule of Law

Looking over the development of the field so far, I would argue that we can distinguish three generations of empirical rule of law research. I will call these: the ‘rule of law in action’, the ‘rule of law index’ and the ‘living rule of law’. All these studies have a different aim or rationale to analyze the rule of law (Why). To achieve this goal, they also focus on a different empirical puzzle (What). Moreover, they take different actors as their primary unit of analysis (Who). Finally, all these studies also employ different methods of empirical research (How) (Table 1).

Table 1. Three Empirical Approaches to the Rule of the Law

The first generation of empirical studies mostly focuses on the effectiveness of the rule of law. The primary focus of these studies is: How much do you comply with the rule of law? In answering this question, the primary unit of analysis are public officials (like civil servants, police officers or judges), but sometimes also ordinary citizens. These studies use both quantitative and qualitative research methods. Echoing Pound’s [4] classic view on law and society, the rationale of these studies is to map the impact of the rule of law and to identify the main gaps between the ‘rule of law on the books’ and the ‘rule of law in action’ [5].

The second generation of empirical research builds on these early studies, but it uses a more specific methodological approach. The primary focus of these studies is to assess the rule of law performance of one country in comparison with other countries: How do you rank the rule of law? The most important unit of analysis of these studies are national states. These studies use quantitative research methods and the results are often translated into various indexes. The main rationale of these studies is to rank different countries, based on their rule of law performance [6].

Following Ehrlich’s [7] example, the third generation of empirical research looks at the ‘living rule of law’. These studies are not interested in measuring the effectiveness of the legal version of the rule of law or the rule of law performance of an individual country. The main focus of these studies is to analyze the rule of law in people’s everyday life: How do you experience the rule of law? Their primary units of analysis are not states or officials but ordinary citizens. These studies mostly use qualitative research methods. The main goal of this type of research is to understand the societal (in)significance of the rule of law [8].

Challenges for Future Research

These three generations of research demonstrate how empirical studies may contribute to our understanding of the global rule of law recession. However, these studies also illustrate several important challenges for future research.

The first challenge is to decide if the study of the rule of law should be an empirical or a normative project. According to some scholars, the empirical study of the rule of law is an important tool to implement a certain normative vision of the rule of law. However, this approach is vulnerable to strong methodological criticism. In Black’s words: ‘When legal reality is compared to an ideal with no identifiable empirical referent, such as the ‘rule of law’…, the investigator may inadvertently implant his personal ideals as the society’s legal ideals. At this point social science ceases and advocacy begins. …’ [9].

This criticism is also reflected in recent debates about ‘scholactivism’ (a combination of ‘scholarship’ and ‘activism’). According to Khaitan, ‘truth-seeking and knowledge dissemination’ are ‘constitutive of the role of the scholar’ [10]. By contrast, scholactivism is characterised by ‘the existence of a motivation to directly pursue special material outcomes[…] through one’s scholarship’ [11]. This debate is also highly relevant for rule of law scholars [12]. A better approach may therefore be what Cheesman describes as ‘passionate humility’ [13]. In this approach, empirical rule of law scholars should take a stance on the idea of the rule of law, but at the same time they must also be constantly willing to adjust their position in the light of the results of empirical research.

A second challenge is to find the right approach to define the concept of the rule of law in empirical research. Thus far, most discussions have focused on what defines the rule of law. However, a major challenge for future empirical research is also who defines the rule of law. On the one hand, empirical research is most useful if it closely follows the official legal definition of the rule law. On the other hand, most legal definitions of the rule of law also have a strong ideological bias. Therefore, empirical research should not only start from the legal definition of the rule of law, but social scientists should also develop their own working definition of the rule of law. In other words, instead of treating the rule of law as an independent variable (separate from the empirical research), we should also look at the rule of law as a dependent variable (which is included in empirical research) [14].

Finally, if we want to understand the erosion of the rule of law it is not enough to collect data, but we also need to understand the meaning of these data. However, most empirical studies have produced a wealth of empirical findings, but these are usually not embedded in a (middle-range) sociological theory. As a result, most of these studies focus only on what, but (due to a lack of theory) not on how people think about the rule of law [15]. That is why it is important that future research is not only aimed at collecting as much (quantitative) data as possible, but that researchers also focus on theorizing about the social significance of the rule of law.

Conclusion

The rule of law is under pressure. Thus far, most research has focused on the legal foundations of the rule of law. However, a growing number of empirical studies have demonstrated that the legal foundations strongly rest on the social foundations of the rule of law. This suggests that the ongoing debate about the future of the rule of law should also include a conversation about the future of rule of law research. To understand (and ultimately reverse) the growing rule of law recession, it is important to recognize that the rule of law is not only defined by formal institutions and legal documents, but also by the place of law in people’s everyday life.

References

[1] See, e.g, Szente Z. 2017. Challenging the Basic Values – the Problems with the Rule of Law in Hungary and the EU’s Failure to Tackle Them. In The Enforcement of EU Law and Values, eds. A Jakab and D Kochenov, pp. 456-475, Oxford: Oxford University Press; Sadurski W. 2019. Poland’s Constitutional Breakdown. Oxford: Oxford University Press.

[2] See, e.g., JUSTICE. 2023. The State We’re In: Addressing Threats & Challenges to the Rule of Law. London: JUSTICE; Gowder P. 2020. The Dangers to the American Rule of Law Will Outlast the Next Election. Cardozo L. Rev. De-Novo: 126-164; Graaf N. (2023). Dutch Rule of Law Alert. Verfassungsblog. https://verfassungsblog.de/dutch-rule-of-law-alert/

[3] https://worldjusticeproject.org/news/wjp-rule-law-index-2023-global-press-release

[4] Pound R. 1910. Law in Books and Law in Action. Am L Rev 44:12–36

[5] See, e.g., Galligan D and Kurkchiyan M eds. 2003. Law and Informal Practices. Oxford: Oxford University Press; Jensen EG and Heller TC, eds. 2003. Beyond Common Knowledge: Empirical Approaches to the Rule of Law. Stanford University Press; Carothers T. 2006. Promoting the Rule of Law Abroad: In Search of Knowledge. Washington DC: Carnegie Endowment for International Peace.

[6] See, e.g., Versteeg M and Ginsburg T. 2017. Measuring the Rule of Law: A Comparison of Indicators. Law & Social Inquiry 42: 100-137.

[7] Ehrlich E. 1936. The Fundamental Principles of the Sociology of Law. Harvard University Press, Cambridge, Mass.

[8] See, e.g., Massoud M. 2013. Law’s Fragile State: Colonial, Authoritarian, and Humanitarian Legacies in Sudan. Cambridge: Cambridge University Press; McConnachie K. 2014. Governing Refugees: Justice, Order and Legal Pluralism. Abingdon: Routledge; Dunn H. 2021. Emergent Hybrid Legality: A Study of Legal Encounters in the DRC. Journal of Intervention and Statebuilding 15: 201-220.

[9] Black D. 1973. The Boundaries of Legal Sociology. In: The Social Organization of Law, eds. D Black and M Mileski, pp. 41-56, New York: Seminar Press.

[10] Khaitan T. 2022. On Scholactivism in Constitutional Studies: Skeptical Thoughts. International Journal of Constitutional Law 20: 547-556.

[11] ibid.

[12] See Verfassungsblog. 2022. Debate #Scholactivism (blog symposium in response to Tarunabh Khaitan), https://verfassungsblog.de/category/debates/scholactivism-debates/

[13] Cheesman N. 2018. Rule-of-Law Ethnography. Annual Review of Law and Social Science 14: 167-184.

[14] Hertogh M. 2016. Your Rule of Law is Not Mine: Rethinking Empirical Approaches to EU Rule of Law Promotion. Asia Europe Journal 14: 43-59.

[15] See Sarat A. 1977. Studying American Legal Culture: An Assessment of Survey Evidence. Law and Society Review 11: 427-488.