This paper, and the analysis presented here, is intended as a complement to the more general conceptual framework on poorly performing countries (PPCs) being developed as part of the ODI study. Countries are classified and ‘rated’ according to performance on an increasingly wide range of criteria. The quantitative nature of most economic and socio-economic variables has given rise to a long tradition of ranking countries, by income level, growth rates, etc. In economic terms, a simple indicator of poor (economic) performance is having a value (some distance) below the average value of the selected indicator for the population (sample) of countries. There are various ways in which such ‘deviations from the average’ can be measured (discussed in section 3 below). Most importantly, one may wish to control for certain characteristics that are known to affect performance, especially characteristics that are (largely) beyond the influence of policy. For example, landlocked countries will tend to grow slower than non-landlocked countries with otherwise similar characteristics. It is important to distinguish between ‘natural’ characteristics that affect growth and other variables that affect growth but are amenable to policy influence. For example, high inequality is associated with lower growth (ceteris paribus). In this sense, inequality is an indicator of poor (policy) performance rather than a determinant. These issues are discussed in section 2 below.
An important distinction should be drawn between measures (to be used as ranking criteria) that are ‘objectively quantifiable’ from those that are ‘subjectively quantifiable’ (even if in practice the distinction is somewhat blurred). Most economic variables are objectively quantified (even if they may be inaccurate): of a group of countries it is evident which have higher incomes, lower growth, higher investment etc. Most natural characteristics are objective (distance, being landlocked), and most socio-economic measures are objectively quantified (infant mortality, school enrolment). Furthermore, the measurement units are cardinal – 20 is twice the value of 10, etc. With such measures, to the extent that the data are reasonably accurate (and available), one can make comparisons across and between countries. Specifically, one can make ‘objective’ judgements on relative performance according to the measures using statistical analysis. Our analysis concentrates on variables and measures that are of this objectively quantifiable form.
Recent decades have seen a proliferation of ‘league tables’ to rank countries according to measures that are only subjectively quantifiable. Examples include the many indices of governance, corruption, freedom or human rights. Two features of such data deserve mention. First, the data are ordinal and provide only a ranking – 4 is above 2 (or below, depending on the order of the index), but is not twice the value of 2. For example, a country with a corruption score of 6 (where 1 = least corruption and 10 = most corruption) can be claimed to have higher corruption as a country with a score of 3, but cannot be claimed to be twice as corrupt. Second, and related, the measure in its construction embodies subjective judgements. For example, indices such as the CPIA or Freedom House are based on collating subjective responses (rankings) to a set of questions. Measures of this form pose problems for statistical analysis. Consequently, such measures are not used in our core analysis.
The statistical analysis presented in this paper has an intentionally narrow objective to address a specific question. Using (objectively quantifiable) data available for as many developing countries as possible, is it possible to identify a set of countries that would be classified as poor performers using a number of criteria, and is it then possible to identify country characteristics that determine poor performance? At this stage we restrict attention to two broad performance measures, economic growth and infant mortality (a measure of health status that is highly correlated with poverty, used here as a measure of human development performance). In simple terms, we want to see if we can identify a group of countries that could be widely agreed to have performed poorly on either or both of these measures.
Having completed the basic analysis and identified (if we do) poor performers, we then conduct a number of further exercises. First, we assess if the poor performers share certain natural or structural characteristics, i.e. can we identify factors not readily amenable to policy influence that determine poor performance. Second, we can assess if the countries we identify as poor performers are ranked low on ordinal measures (we use governance indicators here). Furthermore, we attempt to assess the direction of causality between poor performance and governance. It is often asserted that poor governance causes poor performance, but the reverse may be the case. Finally (in a future extension) we can relate our list of poor performers to classifications that have been produced by others (e.g. LICUS, MCA).
The statistical analysis cannot identify all poor performers. Cross-country statistical analysis can only cover the countries for which data are available. However, it would be useful to know if, from the set of countries for which data are available, our statistical approach does (or does not) identify as poor performers countries that others, using whatever criteria, have classified as such.
It may be the case that the absence of data is itself a sign of poor performance, in which case the approach will ‘miss’ many poor performers. We do report the countries for which data are not available, but some other judgement is required to decide if these are indeed poor performers (i.e. the data alone provides insufficient information). We may well conclude that the statistical approach is not appropriate for identifying the group of countries that all could agree are poor performers. This in itself would be an important conclusion, with the critical implication that judgement (or information beyond that contained in available objectively quantified data) is required. It would be a matter of concern if poorly performing countries are none other than those that donors or commentators, using their own criteria, classify as such. For these reasons, our approach is purposefully transparent and verifiable.
Section 2 presents a brief review of the literature on economic performance, with the principal aim of distinguishing between performance measures, indicators (or correlates) of performance, and (structural or natural) determinants. The crucial distinction is that the latter are not amenable to policy influence (at least in the relatively short term). If countries are poor performers because of structural characteristics, this has implications for the interventions and policies required to improve performance. The section includes discussion of the role of aid in influencing performance. Section 3 presents the statistical criteria and methods used in the analysis. Section 4 discusses the results, in particular whether poor performers are identified, and Section 5 presents a summary and preliminary conclusions.