Analysis shows Unlock Literacy is changing children's lives—that's real impact

Unlock Literacy is one of World Vision’s core program models in the education sector, working to improve reading fluency and comprehension among early primary school-age children through partnerships with schools and communities.

This happens through three channels: improving the quality of instruction through teacher training, providing materials such as books in local languages, and engaging the community to support children’s literacy. Ongoing assessment of children’s literacy skills is conducted throughout the Unlock Literacy program.

Between 2019 and 2021, we implemented 72 unique projects in 29 countries using Unlock Literacy as a core program model. During that period, conservative estimates show that we reached over 221,000 girls and boys with valuable educational support, thanks to support from donors in Canada.¹ This widespread implementation gave us the data we needed to comprehensively analyze the model’s social and financial costs and benefits—with excellent results.

We can now say with confidence that Unlock Literacy is making a real impact.

As a model, it is effective and delivering long-term benefits.

Independent analysis has revealed quantifiable improvements in children’s learning—the literacy gains from our Unlock Literacy programs were comparable to adding two years of schooling, on average, for the over 221,000 children that participated over the past three years. That means, for example, that if the average grade level a child previously reached was grade 6, they are now more likely to graduate from grade 8 and transition to secondary school.

For every dollar invested in Unlock Literacy programs, an average of $20 in economic benefits are being generated back to society.

In line with our strategic approach, World Vision is developing evidence-based frameworks to provide economic analyses of our core program models, with Unlock Literacy among the first to be studied. This cost-benefit analysis (CBA) was conducted in partnership with an external analytics firm, using historical data from 10 countries where Unlock Literacy has been implemented: Bangladesh, Burundi, Eswatini, Ethiopia, Ghana, India, Malawi, Nepal, Rwanda and Senegal.

COSTS

While the CBA accounts for the financial costs associated with Unlock Literacy—training and mentoring teachers, operating reading camps and community awareness workshops, monitoring and evaluating projects, and management overhead—it also measures its social costs: the opportunity costs of time that children, volunteers, teachers, principals and Ministry of Education staff spend participating in the program. By accounting for these social costs, the CBA offers a more comprehensive estimate of the costs borne by communities and governments alongside World Vision offices and staff.

BENEFITS

The CBA accounts for one main benefit generated by Unlock Literacy. This is:

1. Increased literacy for participating children. Across the 10 countries, the average increases in children’s literacy test scores² were converted to equivalent years of schooling (EYOS) gained.³ This gain in EYOS was then monetized by estimating the change in lifetime earning that children would experience because of their increased reading skills acquired through Unlock Literacy.

While it’s reasonable to expect additional benefits through Unlock Literacy, there is not enough evidence to merit their inclusion as individual benefit streams within the CBA. These include professional development benefits for classroom teachers and local youth volunteering at reading camps, as well as cost savings for ministries of education, from both the indirect effect of reducing grade repetition as well as anecdotal reductions in teacher turn-over.

To account for some of these unknowns, a sensitivity analysis will be conducted, asking questions such as, “What if, in addition to the observed learning gains, we also see cost savings from reduced grade repetition?” This will provide insight into the potential importance such benefit streams could provide and inform future research to reduce gaps.

Results from the 10-country CBA show that Unlock Literacy is a highly successful program model according to multiple outcomes.⁴ Based on the historical analysis, the average student participating in Unlock Literacy can expect to gain two additional EYOS, compared to their non-participating peers.

In addition, across the 10 implementing countries analyzed, the present value of the total number of additional EYOS gained through Unlock Literacy over the period ranged from 1,601 in Eswatini to 26,079 in Ethiopia.

Two distinct approaches were used to assess the cost-effectiveness of the program in different contexts.

The first approach compares the EYOS gained per US$100 invested.⁵ Using this metric, Ethiopia’s 12-month program is estimated to have had the most cost-effective result, gaining over 10 equivalent years of schooling per US$100 invested. Eswatini is the least cost-effective, gaining just over 0.5 EYOS per US$100 spent.⁶ However, even in lower-performing countries, for every US$100 spent on Unlock Literacy, there are sizable increases seen in equivalent years of schooling.

This analysis shows us where investments are bringing the greatest gains in education, and where further investigation is needed to understand challenges and opportunities for improvement.

Countries that see the largest gains in education do not necessarily experience the largest gains in lifetime income—because the economic rewards associated with higher education vary from one country to another.⁷ To account for these factors, the second approach—the benefit-cost ratio—is a useful metric: It compares the financial and economic gains that result from educational improvements against the costs of implementing the program. This helps us assess the program model’s financial feasibility.

Analysis found that the average⁸ benefit-cost ratio across all 10 implementations of Unlock Literacy is 20.39, with a range between 1.36 and 56.46. Since 1.0 is the benchmark at which costs and benefits are exactly equal, this result shows that across all 10 countries, Unlock Literacy’s benefits outweighed the costs of implementation. It also tells us that for every dollar invested, an average of $20 dollars were generated back to society in benefits such as lifetime earnings for the children who participated and cost savings for governments.

The results discussed up to this point give us confidence about Unlock Literacy’s effectiveness and efficiency. To contextualize those results and understand the benefits of our programs supported by Canadian funds over the past three years, an analysis was made using the results of the 10-country CBA together with monitoring data from the 29 countries where Unlock Literacy was implemented.

This analysis revealed that between 2019 and 2021, World Vision’s investments in Unlock Literacy generated over 435,000 additional equivalent years of schooling for the over 221,000 participating girls and boys around the globe, according to lower-bound estimates.⁹ This translates to an average of over two additional years of schooling per child.

The graph below presents this estimate of total EYOS gained, along with the estimate of the total EYOS gained based on upper-bound estimates of the number of children reached through World Vision investments in Unlock Literacy, by country.¹⁰ These results highlight the strong impact World Vision has had on literacy skills across various geographical contexts.

The results of this cost-benefit analysis are very encouraging. Based on this work, we have the evidence to show that Unlock Literacy—a core programming model that has been implemented in 33 countries across the world and reached 1,700,000 girls and boys since 2012—is making a real, positive impact for the children who participate and the societies where it’s implemented.

Moving forward, we’ll continue to test and refine our CBA model to better understand the factors affecting Unlock Literacy’s financial and economic outcomes and identify why some countries present better results than others. This information will help us improve the already positive impacts of Unlock Literacy programming.

¹ This lower-bound estimate reports zero children reached in Ecuador, the Philippines and Zimbabwe due to inadequate indicator data for projects implemented in these countries. This figure is built on the assumption that each year of implementation includes a new cohort of children participating. We report the lower-bound in the absence of more complete data at this point in our commitment to not overclaim results where data can’t be audited.

² As reported by Zook et al. (2021)

³ The conversion method follows Evans and Yuan (2019).

⁴ It is important to note that the values included within this report are subject to change as we continue to improve the precision of the estimates for each country’s benefits and costs.

⁵ 2015 United States Dollars.

⁶ This value may overstate the estimated EYOS per US$100 spent for Eswatini, as data for teacher training or teacher training materials were not reported by the program, as mentioned in Ryall et al. (2020). 

⁷ It is important to recognize the complementarity of financial results and educational impacts in determining the overall success of Unlock Literacy. For example, Rwanda’s financial success is not primarily driven by strong per-student educational gains; instead, the high number of students reached and high estimates of the financial returns to an additional year of primary schooling drive the overall success of the program model.

⁸ “Average” values included in this report are defined as the weighted average across all 10 pilot countries. Weighting is based on the total number of children participating in the program.

⁹ This figure includes four projects for which data on the country of implementation is currently unavailable, coded under UNKNOWN.

¹⁰ Figure 8 excludes Zimbabwe, as it is not possible to estimate the number of additional EYOS gained based on the data available for the Unlock Literacy implementation in that country at the time of this report’s publication.