EquityTool release date: October 7, 2021

Previous version release date: December 9, 2015

Source data: DHS 2016-17

# of survey questions in original wealth index: 36

# of variables in original index: 114

# of survey questions in EquityTool: 12

# of variables in EquityTool: 14

Questions:

Technical notes:

The standard simplification process was applied to achieve high agreement with the original wealth index. Kappa was greater than 0.75 for the national and urban indices. For this EquityTool, the minimum required kappa score of 0.75 was achieved for both national and urban indices with 10 variables. However, an additional four variables were added in order to improve the indices’ ability to discriminate between the first and second quintiles. Details on the standard process can be found in this article. The data used to identify important variables comes from the factor weights released by ICF.

Level of agreement:

Respondents in the original dataset were divided into 3 groups – those in the 1^st and 2^nd quintiles (poorest 40%), those in the 3^rd quintile, and those in the 4^th and 5^th quintiles (richest 40%). After calculating their wealth using the simplified index, they were again divided into 3 groups. Agreement between the original data and our simplified index is presented below.

What does this mean?

When shortening and simplifying the index to make it easier for programs to use to assess equity, it no longer matches the original index with 100% accuracy. At an aggregate level, this error is minimal, and this methodology was deemed acceptable for programmatic use by an expert panel. However, for any given individual, especially those already at a boundary between two quintiles, the quintile the EquityTool assigns them to may differ to their quintile according to the original DHS wealth index.

The graph below illustrates the difference between the EquityTool generated index and the full DHS wealth index. Among all of those people (20% of the population) originally identified as being in the poorest quintile, approximately 83.5% are still identified as being in the poorest quintile when we use the simplified index.  However, approximately 16.5% of people are now classified as being in Quintile 2.  From a practical standpoint, all of these people are relatively poor. Yet, it is worthwhile to understand that the simplified index of 12 questions produces results that are not identical to using all 36 questions in the original survey.

The following table provides the same information on the movement between national quintiles when using the EquityTool versus the original DHS wealth index:

The following graph provides information on the movement between urban quintiles when using the EquityTool versus the original DHS wealth index:

The following table provides the same information on the movement between urban quintiles when using the EquityTool versus the original DHS wealth index:

Data interpretation considerations:

1. This tool provides information on relative wealth – ‘ranking’ respondents within the national or urban population. The most recent available data from the World Bank indicates that 24.5% of people in Haiti live below $1.90/day [1]. This information can be used to put relative wealth into context.
2. People who live in urban areas are more likely to be wealthy. In Haiti, 43% of people living in urban areas are in the richest national quintile, compared to only 5% of those living in rural areas [2].
   1. If your population of interest is predominantly urban, we recommend you look at the urban results to understand how relatively wealthy or poor they are, in comparison to other urban dwellers.
   2. If the people you interviewed using the EquityTool live in rural areas, or a mix of urban and rural areas, we recommend using the national results to understand how relatively wealthy or poor they are, in comparison to the whole country.
3. Some departments in Haiti are wealthier than others. It is important to understand the country context when interpreting your results.
4. In most cases, your population of interest is not expected to be equally distributed across the five wealth quintiles. For example, if your survey interviewed people exiting a shopping mall, you would probably expect most of them to be relatively wealthy.

Changes from the previous EquityTool

We released an EquityTool on December 9, 2015 which compared user data to a benchmark of 2012.  A new source survey, the 2016-17 Haiti DHS survey was recently released, and allows us to benchmark results to a more recent population.  This is important, because wealth generally increases over time, and comparing your respondents to an old benchmark population will lead to over-estimating the relatively wealthy in your survey.  The new EquityTool was generated using the exact same methodology as the previous version, and in generating the new EquityTool, no attempt was made to account for the fact that a previous version existed. In other words, we did not explicitly try to keep the same questions or response options as the previous tool.

For those who have not previously conducted an EquityTool based study in Haiti, the remainder of this section is not particularly relevant.  For those who have used the previous EquityTool, you may be interested to know how the two versions compare.

Practical considerations for users of the previous EquityTool

Comparing the results of surveys that used the previous EquityTool against those that use the current EquityTool is difficult. It will not always be clear whether any difference is because of actual differences in the wealth level of the respondents or because the EquityTool has changed.

The technical comparison section below, particularly the 3^rd comparison, illustrates how quintile results compare when using the previous EquityTool and the current one. Generally, there is a partial shift down in quintiles when using a more recent EquityTool. In other words, the current EquityTool will usually put some respondents into a lower quintile than the previous one would.

It is generally best to use the current version of the EquityTool, since it will give a more accurate quintile estimate. If you are currently collecting data, it is best to continue to use the previous tool.  Note that if you have created a survey in the EquityTool web application using the previous EquityTool, that survey will continue to use the previous EquityTool.

If conducting a follow-up survey to a baseline that used the previous EquityTool, and the most important result is change from the baseline, it may be preferable to continue to use the previous EquityTool for comparability. If you need to do this, please contact us at support@equitytool.org.

Technical comparison between the current and previous EquityTool

All the questions and response options for the previous EquityTool are found in the new source data (DHS 2016-17), however factor weights are not available for shared or non-shared toilet in the 2016-17 data. This makes comparison between the two versions of the EquityTool, and two different data sources, slightly more difficult.

The comparison will be assessed in 3 different ways, described below.

1. Using the same 13 questions and response options, and scoring system as in the previous EquityTool, with two different benchmark populations.

This analysis simulates results if the only thing which changes is the benchmark against which respondents are compared. In the 4 years between the two source data studies, more people have acquired assets that are indicative of wealth. In the graph below, the previous EquityTool, derived from the 2012 DHS, is applied to the 2012 DHS data and the newer 2016-17 DHS data. In both 2012 and 2016-2017, the proportion of households in each of the 5 quintiles is very close to 20%. Using the older tool, there are fewer people in the lower 3 quintiles, and more in the upper 2 quintiles, in 2016. Nevertheless, differences are not large.

We do not use the previous questions and weights, because over time, the population becomes wealthier. As years pass and the population continues to gain wealth there is an increasing risk that the 2012 DHS EquityTool will assign households to wealthier quintiles who would otherwise be considered relatively poor according to more up-to-date indices of wealth.

2. Keeping the same 13 questions and response options as the previous EquityTool, but calculating scores based upon the 2016-17 data.

As an alternative, one might wish to use the same questions as the previous tool but update the weighting. This seems reasonable, as the relative contribution of each asset towards overall wealth may have changed over time. One variable used to the construct the 2012 DHS Haiti wealth index and 2012 Haiti EquityTool was not used in the 2016-17 DHS Haiti survey. In order to update all weights for the 13 questions of the 2012 Haiti EquityTool it was first necessary to recreate the 2016-17 DHS Haiti wealth index after adding back in this missing variable. Using these newly calculated weights allows us to simulate how households’ wealth in the 2016-17 DHS Haiti survey would have been categorized within the five quintiles using the full 13-question 2012 EquityTool with updated variable weights.

The table below presents the agreement between the quintiles created from the full wealth index in the 2016-2107 DHS dataset and the quintiles created by the 2012 EquityTool, the 2012 EquityTool variables with updated weighting, and the current EquityTool. As with the agreement statistics above, these figures are for the bottom 2 quintiles, middle quintile and top 2 quintiles.

The current EquityTool has greater agreement with the full wealth index quintiles, though all three EquityTools have similar Kappa scores and exceed our minimum threshold of 0.75. Though all three EquityTool versions have similar levels of agreement, the 13 questions from the 2012 tool are no longer the conclusively best predictors of the overall wealth distribution. As previously mentioned, as time passes population will likely continue to gain wealth. As this happens, the 2012 DHS EquityTool questions will grow increasingly outdated and will falsely assign households to wealthier quintiles who would otherwise be considered relatively poor according to more current indices of wealth.

3. Comparing the previous 13 questions and scores, and the new EquityTool (12 questions)

The previous EquityTool’s 13 questions may no longer most accurately predict wealth. Because more people may own the assets predictive of wealth in the most recent survey years, we need to adjust the mix of questions necessary to differentiate people and households more accurately.

The table below shows how the previous and current EquityTool compare when applied to the 2016-17 DHS dataset. This is analogous to a comparison of the two versions of the EquityTool on the population you surveyed using our previous EquityTool.

The rightmost column indicates that the current EquityTool does in fact very closely divide the population into 5 groups. The bottom row shows that using the older EquityTool also evenly divided the population into equal quintiles. The cells within the table indicate how respondents are categorized, if measured using the two different tools. When analyzing the 2016-17 DHS dataset the 2012 and 2016 Haiti EquityTools had high agreement in their assignment of households into wealth quintiles. These findings suggest that your organization can adopt the 2016 EquityTool and reliably compare its wealth quintile assignments to those that you may have collected using the 2012 EquityTool.

Metrics for Management provides technical assistance services to those using the EquityTool, or wanting to collect data on the wealth of their program beneficiaries. Please contact equitytool@m4mgmt.org and we will assist you.

[1] From povertydata.worldbank.org, reporting Poverty headcount ratio at $1.90/day at 2011 international prices.

[2] From the Haiti DHS 2012 dataset household recode, available at http://dhsprogram.com/