Evaluation

The Energy Sector Management Assistance Program (ESMAP) is one of the World Bank’s flagship global energy programs, providing grant funding and technical assistance to support clean energy transitions, energy access, and energy efficiency across dozens of countries. As ESMAP approached the end of its 2021–24 Business Plan, the World Bank commissioned an independent external evaluation to assess the program’s performance, identify opportunities for improvement, and inform the design of the next business plan. The evaluation needed to assess a highly diverse portfolio of technical assistance activities across multiple sectors and regions while meeting the methodological rigor expected of World Bank evaluations and producing findings that program managers and donors could use in practice.

The World Bank hired DHInfrastructure to conduct the evaluation using a mixed-methods approach that combined quantitative analysis of program outputs and results with qualitative assessment of program processes, stakeholder perspectives, and implementation realities. We selected Rwanda, Nigeria, and India as country case studies for detailed evaluation of all ESMAP grants in those countries, and developed a stratified sampling methodology to select an additional representative sample of grants from all other countries—ensuring that our findings reflected the full diversity of ESMAP's portfolio rather than just its most visible projects. We reviewed documents related to all sampled grants, developed structured interview guides, and conducted interviews with key informants globally and in each case study country. We assessed the program against the Organization for Economic Co-operation and Development's Development Assistance Committee (OECD-DAC) evaluation criteria of relevance, effectiveness, efficiency, impact, coherence, and sustainability—applying each criterion rigorously while remaining alert to the specific characteristics of a technical assistance program operating across diverse country contexts. We synthesized our findings into a set of prioritized recommendations and presented them to the ESMAP team and its donor community. The evaluation produced findings that translated directly into program improvements, with most of DHInfrastructure's recommendations adopted as part of ESMAP's next business plan.

As the Millennium Challenge Corporation (MCC) prepared a series of major electricity sector investments in Tanzania, it wanted to understand whether those investments would translate into measurable improvements in household welfare, business performance, health, and education outcomes. To generate credible evidence, the evaluation framework needed to be designed before implementation began, with survey instruments, sampling strategies, and analytical methods capable of isolating the effects of expanded electricity access from broader economic and social change.

DHInfrastructure advised on the overall evaluation framework, bringing to bear our understanding of infrastructure economics and the practical realities of electricity sector investment to help MCC design an approach that was both methodologically sound and feasible to implement. We advised on the design of household and business surveys, including the sampling strategies and questionnaire approaches needed to support robust impact measurement. We also advised on the design of an output-based subsidy mechanism for new electricity connections — a feature of the program with its own evaluation implications. As a complement to the broader household-level evaluation, DHInfrastructure designed and administered a survey of businesses on Zanzibar's Unguja Island to study the effect of electricity supply interruptions and voltage and frequency fluctuations on business operations — a more focused exercise that demonstrated how well-designed survey instruments can generate actionable evidence on the economic costs of poor power quality even in the absence of a full counterfactual evaluation.

Ukraine’s municipal heating sector faced persistent challenges related to utility finances, operational efficiency, and service quality, leading USAID to launch the Municipal Heating Reform Project to support sector modernization and reform. As the project reached its midpoint, USAID sought an independent evaluation of whether the program’s regulatory reform activities, energy audits, and municipal planning efforts were effectively designed and positioned to deliver lasting improvements. DHInfrastructure was hired to assess program performance and provide actionable recommendations for strengthening implementation during the remainder of the project.

DHInfrastructure assessed the relevance and effectiveness of the program's activities across three dimensions that were central to its theory of change. We evaluated the effectiveness of regulatory reform activities, examining whether the approaches being used were well-suited to the institutional context and likely to produce durable changes in how the sector was governed. We assessed the approach, quality, and efficacy of energy audits—a core program activity—examining both how they were being conducted and whether they were generating the kind of actionable findings that could drive operational improvements in heating utilities. And we reviewed the thoroughness of municipal energy plans and the extent to which participating municipalities were actually implementing them, identifying gaps between planning and execution that the program could address in its remaining period. Our findings and recommendations were presented to USAID in a final evaluation report that combined an honest assessment of what was and was not working with practical guidance for improving program performance going forward.

The European Bank for Reconstruction and Development (EBRD) and Switzerland's State Secretariat for Economic Affairs (SECO) have worked together since 2003 to improve water services in Tajikistan, combining physical infrastructure investment with capacity building support intended to strengthen utility management and create more sustainable service providers. After two decades of engagement, the physical results were tangible—but whether the capacity building had produced lasting institutional improvements was less clear. EBRD and SECO wanted an independent evaluation of their water sector projects in the country to understand what had genuinely been achieved, what had worked and what had not, and how future engagements could be made more effective. They hired DHInfrastructure to conduct that evaluation.

DHInfrastructure evaluated both projects against the Organization for Economic Cooperation and Development's Development Assistance Committee (OECD-DAC) criteria of relevance, effectiveness, efficiency, impact, coherence, and sustainability. We reviewed project documentation to establish a foundation for the assessment, then conducted an in-country mission to Tajikistan to interview key stakeholders—including project staff, government counterparts, and beneficiary communities — gathering firsthand evidence that could not be obtained from documents alone. Drawing on both the documentary review and the mission findings, we identified lessons learned from EBRD and SECO's activities and developed recommendations for future engagements in Tajikistan's water sector, grounded in a realistic assessment of what had worked, what had not, and why.