India's rank in global perception indices declined due to shallow, opaque methodologies: PM advisory panel working paper

New Delhi [India], November 22 : With a decline in India's rankings in recent years on opinion-based global indices on issues like democracy and freedom, a working paper that investigated three well-known Western think-tanks found shallow and opaque methodologies having been used with a common thread that they are derived from perceptions of a few experts.
The working paper 'Why India does poorly on global perception indices: Case study of three opinion-based indices' written by economist Sanjeev Sanyal and Aakanksha Arora, looked at three indices - Freedom in the World Index, Economist Intelligence Unit (EIU) Democracy Index and Variety of Democracy indices. It said the government should request the World Bank to demand transparency and accountability from think tanks that provide inputs for the World Governance Indicators (WGI).
The study has been brought out in the EAC-PM Working Paper series.
The paper said all three indices "are almost entirely perception based" and noted that these perception-based indices should not be ignored as mere opinions as they find their way into concrete things such as sovereign ratings via WGI, which is based on a combination of a lot of these indices.
"These will become even more important in the future as Environmental, Social and Governance (ESG) indices are introduced into global business/investment decisions. Hence, as a first step, the Indian government should request the World Bank to demand transparency and accountability from think-tanks that provide inputs for the WGI. In the longer term, independent think-tanks in India should be encouraged to research in these areas and come up with their own indices so that comparative indices are available," the paper said.
Sanyal, who served as the Principal Economic Advisor in the Finance Ministry and is now a member of the Prime Minister's Economic Advisory Council (PM-EAC), said the growing calls for ESG scores in investment/trade decisions will give this class of opinion-based indices even greater currency.
He said there are serious problems with the methodology used in these perception-based indices, these indices are primarily based on the opinions of a tiny group of unknown "experts" and certain questions used by these indices are not an appropriate measure of democracy across all countries.
"My working paper on why opinion-based global indices on issues like democracy & freedom have been lowering India's rankings/scores since 2014. The paper specifically investigated three well-known Western think-tanks and found laughably shallow, opaque methodologies. The problem is that these opinions find their way into concrete things into sovereign ratings (through the World Bank's WGI index). The growing calls for ESG scores in investment/trade decisions will give this class of opinion-based indices even greater currency. Therefore, it is time to demand more transparency from both the think-tanks and World Bank," he said in a tweet.
The authors said that Freedom in the World Index has been published since 1973 by Freedom House.
India's score on Civil Liberties was flat at 42 till 2018 but dropped sharply to 33 by 2022; that for Political Rights dropped from 35 to 33. Thus, India's total score dropped to 66 which places India in the "partially free" category - the same status it had during the Emergency.
"Since the publication of the index, the only two previous instances where India was considered as Partially Free was during the time of emergency and then during 1991-96 which were years of economic liberalisation. Clearly, this seems very arbitrary as what did the period of emergency which was a period of obvious curtailment of various activities had similar to the period of economic liberalisation or of the current times," the paper said.
It noted that the Freedom in the World report has given the territory of Northern Cyprus a score of 77 which makes it a free democracy. "This is a territory only recognized by Turkey, not even by the United Nations. Meanwhile, the think-tank continues to treat Jammu and Kashmir as a separate territory since the early 1990s and now places it in the category of "not free"' the paper said.
Referring to Economist Intelligence Unit (EIU) Democracy Index, published by EIU, the paper said India is placed in the category of "Flawed Democracy" and its rank deteriorated sharply from 27 in 2014 to 53 in 2020 and then improved a bit to 46 in 2021.
The decline in rank has been on account of a decline in scores primarily in the categories- Civil Liberties and Political Culture. The maximum decline has been in the category Civil Liberties, for which the score declined from 9.41 in 2014 to 5.59 in 2020.
The paper noted that the comparison of scores of other countries on this EIU-Democracy Index has some surprising results.
"India's latest score for Civil Liberties lags that of Hong Kong (8.53). Similarly, India's score for Political Culture is much lower than that of Hong Kong (7.5) and Sri Lanka (6.25). Clearly, this seems very arbitrary."
The study said EIU in their methodology mention that they do not only rely on opinion of experts but also take responses for some questions from opinion polls, if available (primarily from World Value Survey (WVS)).
"If the responses from opinion polls are not available, then the responses for those questions also are answered by experts. In case of India, the latest available WVS report is of 2012. The latest round of Wave 7 (2017-2020) has not been conducted for India yet. This implies that score of EIU Democracy index for India is based only on expert opinion since 2012," the paper said.
Referring to the Varieties of Democracy (V-DEM) indices produced by the Varieties of Democracy Institute at the University of Gothenburg in Sweden, the paper said it comes up with six indices covering various aspects of democracy and the indices are comprised of various sub-indices.
Some of the variables that feed into these indices are objective whereas some other variables are subjective and the final indices are created combining both kind of variables, the report said.
The authors said that a time series analysis of the V-DEM scores show that India does well on objective parameters such as share of population with suffrage, but scores on various subjective sub-indices have declined sharply since 2014.
The report expressed surprise that India's rank on Liberal Democracy Index is far lower than Lesotho, which started having a democratically elected government only in mid-1990, or Kosovo, which declared independence from Serbia only in 2008.
"In fact, India has been termed as an "electoral autocracy" in the 2021 report, same as it was during the period of Emergency....the cross-country comparison of this index also throws up some interesting results. India's rank on Liberal Democracy Index is 93. In contrast, Kingdom of Lesotho which started having democratically elected government only in mid 1990s, and has since faced various four disruptions including a military coup and emergencies has a rank of 60. Or Kosovo, which declared independence from Serbia only in 2008 has a rank of 79.
"A common thread in all these indices is that they are derived from the perceptions or opinions of few experts. These institutions do not provide any transparency on how the experts were chosen or even their expertise or nationality (expect in case of V-DEM where they clarify that they chose some experts from each country from different fields)," the working paper said.
It noted that the Freedom House report mentions that report is produced by a team of in-house staff, analysts, consultants and external analysts and expert advisers from the academic, think tank, and human rights communities.
"The nationality and expertise of the experts are not clear in the report. "
The authors said that another common feature of these indices is all these are based on a set of questions.
"A reading of the questionnaire shows that most of the questions are subjective in nature, hence simply providing the same questions for all countries does not mean getting comparable scores for different countries as the generic questions can be answered very differently by experts," the paper said.