ÀŠ°óSMÉçÇű

Detail of a high rise in Montreal. By Phil Deforges at https://unsplash.com/photos/ow1mML1sOi0

The Numbers Don’t Lie, Or Do They? Measuring Justice with the World Bank’s “Doing Business” Indicators

Over the past 17 years, the World Bank’s Doing Business (DB) index has evolved to be the most cited benchmark for evaluating the regulatory environment of small businesses around the world. Recently, the World Bank acknowledged “data irregularities” in audit reports and suspended its 2021 publication. The external panel specifically identified irregularities in the indicators regarding the judiciary’s role in contract enforcement and those which suggest the need for law reform. This author argues that rankings of favourable business climates must be independently scrutinized, especially where the indices extend to the role of the local judiciary.

The Legacy of “Doing Business” and recent data manipulation revelations 

In September 2021, the World Bank Group the immense scope of “data irregularities” in the Doing Business (DB) reports of 2018 and 2020, and decided to and . Presumably, this decision was triggered by the the Bank received not only from those countries that received lower rankings than in previous years, but also from others who criticized the rankings’ methodologies, including lumping together countries, some of which had not even been included in the data collection process.ÌęÌę

The DB project, which has been met with both and , produces annual reports calculating aggregate scores of each jurisdiction based on the “ease of doing business” criteria. , “every year, DB produces eleven numerical indicators, offering a ‘snapshot’ of business regulations in 190 countries, on themes ranging from access to electricity to conditions of employment.” A notable illustration of “,” DB became a much-used measurement instrument which, through its deceptively simple and eye-catching rankings, offered investors and policymakers what was touted as a ‘clear picture# of a country’s business regulatory infrastructure. Since its launch in 2003, the DB project has expanded from ranking 133 economies to 189 in 2020. This ranking is created based on an array of indicators, and has become one of the most widely watched indices issued by the World Bank, along with , , and .ÌęÌę

For decades, the DB index has remained heavily scrutinized with regard to its Given the wide interest in and broad impact of the DB project, the allegations of its ranking manipulation have periodically made thes. Recently, both the World Bank internal inquiry and the external review carried out by the law firm concluded that occurred in the 2018 and 2020 reports. According to the WilmerHale report, China and Saudi Arabia were able to successfully pressure DB staff into manipulating the rankings in their favour. 

Since this report by WilmerHale was commissioned by the current World Bank management, some readers interpret the investigation as a sign of within the World Bank leadership. Others like Joseph Stiglitz, the Nobel-winning American economist, display their support for the DB project and claim the WilmerHale review as an attempted coup against Ms. Kristalina , the previous Chief Executive of the World Bank and current Managing Director of the IMF. 

What, exactly, does DB measure? The Answer is “289” 

By producing “quantitative indicators” based on “hard data,” the DB index once pioneered analyzing institutional structure with a synthetic score. The DB team first with around 10,000 questionnaires covering eleven specific “data points” (initially it covered five but has since grown). The central part of this data collection process, according to , is asking law firms and businesses to count the number of proceedings, the cost, time, or pieces of laws and regulations involved in the life of a business. Based on a series of quantifiable goals and empirical evidence, DB experts create a score for each set of indicators, then convert those into a simple unweighted average of the ranks, and finally, provide an aggregate score of any given jurisdiction, leading to a global ranking.ÌęÌę

However, there are defects in those seemingly objective DB indicators. First, for those areas that are difficult to quantify—courts, labour, or other aspects in business regulation—the  DB simplifies them into routinized, one-size-fits-all indicators. This standardization is deceptive, however. There is a cost when measuring a complicated court system based on simple statistics. In extreme cases, this use of indicators some countries to game the index with self-reported data, making it susceptible to manipulation, as the 2021 °ù±đ±č±đČč±ô±đ»ć.Ìę

Moreover, the inherent problem with the DB measurement may stem from their choice of those eleven specific indicator sets, which, as suggests, reflect the decades-old “.” At the end of the cold war, Washington, D.C.-based think tanks prescribed a set of neoliberal economic principles for the development of emerging economies, enabling a free and efficient transition to the market economy. To calculate efficiency, the DB founders developed a number of, purportedly, easy-to-use toolkits. When set out to open a small garment company in Peru, he recorded how long it took to obtain the government permits: 289 days. Inspired by such a simple method to measure efficiency, the DB experts designed a handful of questionnaire spreadsheets listing variables such as days, costs, or procedures in opening and managing a small- or medium-sized business. This approach revealed is limitations when used to assess the role of courts.ÌęÌę

How Toronto’s courts fell out of favor with the DB Index 

The DB indicators are further problematic when it comes to the ranking of local courts with regard to their contribution to creating an attractive economic environment. The “Enforcing Contracts” index comes down to : “Doing Business measures the time and cost for resolving a commercial dispute through a local first-instance court and the quality of judicial processes index, evaluating whether each economy has adopted a series of good practices that promote quality and efficiency in the court system.” ). Through questionnaires, the DB working groups collect raw data such as items of reform, numbers of cases, quality and number of proceedings, cost estimates (% of claim) and time estimates (days), which are then transformed into a quantitative value between 0 and 1.ÌęÌę

Once exceptional weights are assigned to the time variable, fortune always favours the nimble ones. With speedy proceedings, China could easily beat many counterparts, including Canada. In the 2017 DB rankings, Canada witnessed significant downward shifts, with the Enforcing Contracts index falling from 49th to 112th. The days required to enforce contracts through courts increased from an average of 570 days to 910 days Canada’s benchmark area. In China, with a party-run, concentrated court system, and a , contract disputes can allegedly be resolved within a comparatively speedy .ÌęÌę

Out of Sync? Numerical Rankings and the Struggle for Law and Justice 

Yet, there is a much deeper problem when it comes to tying the duration of court proceedings to a country’s ability to create a welcoming business environment for local and foreign investors. The DB’s counting game treats the judiciary along with the norms that govern its operation and the claims that judges preside over as a single-purpose, input-output mechanism. But, by assessing a country’s judicial system through an exclusive business climate lens, the judiciary is reduced to not much more than an obstacle course to get through. What the indicators don’t grasp is the actual place of the judiciary in a country’s evolving legal, political, cultural and socio-economic fabric. The indicators take numerical probes of what is, fundamentally, a continuing struggle not just for business-friendly legal norms, but justice. And it is this productive tension in a country’s legal culture between law and justice that animates the ongoing efforts to create as well as to contest, to reform and to improve the law so that it may get closer to the ideals of justice. At the heart of this effort lies a democratic practice that must be protected and supported also by the courts. How courts can fulfil that function surely cannot be reduced to the label ‘business enabling.’  

The same bias has governed the DB’s ranking of other regulatory-related measurements such as registering property, resolving insolvency, or, prior to 2020, employing workers, all of which share common factors such as time, cost, outcome, and the formality of regulation or institutional structure. While those numbers are meant to convey some degree of cost-efficiency, they don’t account for the actual function of a judiciary to be a forum in which to engage not only the validity but also the meaning of a country’s law.ÌęÌę

It is no surprise, then, that the DB’s judiciary rankings have since day one. The Indicators’ “embedded policy preferences” tacitly endorse a legal system bending to the will of businesses, and investigate businesses’ perspectives only. The DB portfolio embraces a court system prioritizing business interests and undermining the courts’ true goals: justice and fairness. 

Against this background, the DB indicators get it all wrong. One last example, where the Enforcing Contracts benchmark calculates how many pieces of law and regulation are published in any given year, they pay little attention to evaluating the substantive effects of those documents. This is exacerbated when there is a flooding of the index with “favourable” data relating to allegedly pro-business laws and regulations. As identifying a greater number of “laws and regulations” is rewarded with a higher place on the index, this strategy has led a Chinese working group to instruct law firms to t issued by regulatory agencies.ÌęÌę

What’s Next for “Doing Business”, not just for China? 

Interestingly, the WilmerHale report revealed favouritism towards China in the 2018 rankings. With the help of “data irregularities,” China’s overall ranking rose from 85 to 78. However, even where there was no suspicion of “manipulation”, China still showed a great leap forward, to 46 in 2019, and . Therefore, with or without “data irregularities,” countries may find other “effective” ways to game the rankings.ÌęÌę

Back in 2017, when the Chinese officials complained to officials of the World Bank, then-World Bank President Kim urged them to “focus on enacting economic reforms that would boost the country’s ranking ,” and asserted that “this was the same response he gave to every country”. Indeed, , DB has always applauded “” reform agendas in developing countries, awarding them with an exciting raise in rankings, persuading more to join the club of globalization.ÌęÌę

The core aim of reforms is to increase “efficiency in governance” and to regulate informal business activities. To facilitate the transformation of informal practices into formal institutions, de Soto and other pro-market protagonists  suggest governments should write, pass and revise efficiently. Under the baton of reform, the top awards go to the countries that performed poorly previously and are exhibiting in writing. That may explain why, although with similar data of time and costs in the Enforcing Contracts index, China’s ranking jumped to 5 during 2015-2020.  But, by writing and publishing many laws and regulations, or , the of China could also take of reform—one which generates piles of legislation but doesn’t address much in substantive justice—at a much lower cost. After all, if the Chinese characteristics fit with the DB methodology so well, is that perhaps inevitable? 

The long-term goal – ‘reform’? 

All this only underscores the contested and competing meanings of “reform.” At this point in time, the discussion around a reform of the DB rankings risks leaving the law in action untouched. Worse, the many challenges of doing business in some jurisdictions may be eclipsed by a better score of de jure reform. But, sooner or later, the ongoing data scandal could cast a shadow over what the World Bank has always encouraged developing countries to engage in, and what the DB has proudly taken the credit for: reform. This reform would, ideally, be closer to the afore-mentioned idea of a country’s citizens, through democratic practice, law making and its judiciary, striving for law and justice. 

Meanwhile, the measuring of real life through numbers and standardized indicators doesn’t seem to go away any time soon. For decades, the World Bank has been a key driver in quietly building norms of relying on their own knowledge products. The amount of resources dedicated to the DB rankings, including the political and judicial resources spent at the national level, indicates how powerful international institutions and global governance are in shaping domestic judiciaries and influencing state actors’ behaviour. Further, they show how realpolitik, equipped with , could distort soft governance, or in more extreme cases, facilitate a corrupt court system.ÌęÌę

The World Bank Group is currently formulating climate, and even considering abandoning the notorious system altogether. As long as the DB project continues its mission as a transnational legal indicator, the prevailing approach to measuring judicial processes needs to be revisited. That methodology reflects biased perspectives in the role of law, narrow-sighted indicators of the court system, and an oversimplified ranking that allows for falsification of data that is misleading and undermines its credibility. Regardless, the current rumours of reform may suggest that a search may be under way for a clearer vision beyond the obsession with efficiency, and a deeper insight into a regulatory framework to advance justice.ÌęÌę

Back to top