Evaluating the Predictive Power of WGI Against WJP Rule of Law Scores

Abstract: This analysis builds on previous research comparing the World Governance Indicator (WGI) and the World Justice Project (WJP) Rule of Law indices. It addresses whether the WGI can reliably predict the WJP’s Rule of Law levels, with findings suggesting strong predictive capability and minimal forecasting errors.

This post is the second in a series examining the intricate links between rule of law and socio-economic factors.

Context and Motivation

In my last examination, I aimed to evaluate whether the WGI's Rule of Law (RoL) score could serve as a suitable substitute for the WJP’s score. The WGI offers a broader geographical coverage, which is appealing for analysts who require data from a diverse range of countries. Feedback from experts indicated that while my previous analysis confirmed a general agreement in rankings, there was a clear need for a deeper inquiry into the WGI’s predictive power regarding RoL levels. It’s vital to understand not only if one index correlates with another but also how well it can predict specific outcomes in diverse contexts.

Data Preparation

For this study, data was gathered from the WGI and WJP, both updated as of July 2025. This continual update is essential, as both indices evolve based on changing governance dynamics and legal standards. The necessary data can be accessed through the following links: WGI Data and WJP Data.

The data cleaning process paralleled the previous analysis, ensuring that country names and regions were uniformly represented across both indices. Standardization is crucial in data analysis, as discrepancies can lead to erroneous interpretations. This phase involved renaming columns for clarity and merging datasets effectively, which not only streamlines analysis but also reduces potential errors that may arise from compounded variables.

Modeling the Relationship

A linear regression model was constructed to determine if the WGI RoL measure could estimate WJP levels. Initial findings were promising, showcasing a fit of 97 percent accuracy in cross-country predictions. This high level of accuracy implies that the WGI is not just a rough approximation; it may indeed provide valuable insights into governmental performance. However, this doesn't erase the necessity for rigorous validation. The data filtered to include only countries with valid RoL values further strengthened these results, narrowing down the focus to those nations where legal frameworks are actively assessed.

Cross-Validation Approach

Leveraging Leave-One-Out Cross-Validation (LOO RMSE), we evaluated the model's predictive accuracy while neglecting the data of individual countries. By removing one country at a time and predicting its score based on the remaining data, it helped identify any anomalies or inconsistencies. The RMSE results indicated that the typical forecast error when using the WGI to estimate the WJP levels stood at below three percent, representing about 19% of the overall variability in WJP scores. Given the stakes involved in policy-making and governance, a prediction error around this range is relatively low and suggests a sound foundational relationship between these indices.

Examining Systematic Bias

Despite the model's robust predictive capability, the RMSE could obscure systematic biases that might exist across the WGI score spectrum. A residual analysis comparing predicted and actual WJP scores revealed a slight under-prediction at both the high and low ends, with a bias magnitude around 0.05 units. This indicates that while the WGI performs well on average, it tends to underestimate outcomes in countries with extreme scores. Addressing these biases is vital for real-world applications, where even minor inaccuracies can have significant implications.

To address this, I incorporated a non-linear term into the model. This adjustment helped flatten the error distribution across the WGI RoL spectrum, leading to a more balanced and accurate model. Introducing non-linearity often reflects real-world complexities better than simple linear relationships. While employing this non-linear estimation did improve prediction accuracy, it still allowed for reasonable uses of the WGI as a proxy for levels of law adherence. This is particularly relevant for policymakers and researchers who rely on these data points for analysis and decision-making.

Temporal Stability of Predictions

Another significant aspect addressed was whether the WGI's predictive accuracy remained consistent over time. This is where stability in governance metrics comes into play, as fluctuations can mislead longitudinal analyses. Using an analysis of variance (ANOVA), I compared a fixed-effects model with one allowing for changes over time in the WGI's influence. The resulting p-value of 0.09 suggested that the simpler time-invariant model was adequate for prediction. This stability in the WGI’s predictive ability concerning the WJP index adds an additional layer of confidence for future analyses. You want to know if your data will still be relevant tomorrow, after all.

Conclusion and Implications

This analysis asserts that the WGI can effectively serve as a proxy for the WJP’s Rule of Law Index. It demonstrates its utility for both rankings and levels, offering researchers and policymakers alike a reliable resource. However, attention must be paid to the potential biases—especially at the extremes of the RoL range. Going forward, a deeper understanding of these dynamics may enhance the model's accuracy and its utility in real-world applications. The implications here aren’t trivial; they impact governance, international relations, and the effectiveness of legal reforms. What this means for you, is that as these metrics evolve, so too must our understanding of their potential—because trends in governance can influence everything from economic development to societal stability.

Note on Methodology: The coding components of this analysis were supported by AI-driven suggestions, notably during the draft preparation phase, ensuring rigor in analytical methods.

The ongoing dialogue within this research series aims to deepen understanding of how governance metrics relate closely to real-world outcomes and impacts across different nations.