Dec 5, 2017

Delivery Challenges: The Devil Is in the Details

Daniel Ortega Nieto , Operations Officer
Vivek Agarwal , Data Analytics Specialist

Focus on delivery is the new mantra in international development, and at the heart of it are delivery challenges in project implementation. Repeated failure of well-designed projects because of political polarization during election campaigns, excess staff turnover, or limited citizen uptake because of insufficient stakeholder consultations are examples that we have seen come up repeatedly. However, not all delivery challenges are as straightforward or easily described. Complex linkages between multiple causes-and-effects has led to the use of all-encompassing jargon, such as institutional weakness or lack of political will, to report problems. While these stock phrases broadly describe the problems in implementation, they have limited analytical or practical value since they fail to provide actionable knowledge to practitioners on ground. This issue has further been exacerbated as these stock phrases have become embedded our day-to-day business processes. Very simply, if what went wrong is not identified with greater specificity, the current knowledge will not help us get delivery right the next time. There is, therefore, a clear need to capture delivery challenges not just in a consistent but also a detailed manner. The devil truly lies in the detail.

 Truth be told, the choice of any approach to identify and structure information on delivery challenges is bound to be somewhat arbitrary. Ultimately, the appropriateness of any approach can only be measured by its usefulness to practitioners. Guided by this principle we worked towards mapping the landscape of delivery challenges in international development as a part of the Global Delivery Initiative’s effort to organize the burgeoning knowledge on delivery know-how. We used a three-pronged strategy to build the analytical underpinnings of a delivery challenge taxonomy: literature review, text analytics, and stakeholder consultation.

The first strategy – an extensive review of over 160 articles – produced two key insights. First, the same or similar delivery issues are referred to in multiple ways by development practitioners. This not only prevents seamless communication between practitioners, it greatly hinders the use of knowledge codified in myriad documents. Second, as discussed earlier, challenges are addressed in very broad terms making them hard to translate into concrete issues that affect practitioners. This exercise helped build a broader framework of delivery challenges, but this framework fell short of being useful to practitioners. To address this, we adopted a bottom-up strategy with a deep dive into over 2,000 World Bank project completion reports (ICRs). Initially through a time-consuming reading of relevant sections of ICRs and later using advanced text-segmentation techniques, we were able to identify recurring challenges. We deployed unsupervised machine-learning techniques to unearth these patterns to label and categorize delivery challenges in a two-level taxonomy. Finally, our third strategy focused on consulting practitioners and experts in different fields across multiple organizations to verify these results. This allowed us to ensure the issues are referred to in an actionable manner, that the challenges are structured with how practitioners currently think about delivery issues and, lastly, that the language that we use corresponds to the language that practitioners use to refer to delivery challenges.

This three-pronged strategy has resulted in a delivery challenges taxonomy (see here) with 15 broad categories and 54 sub-categories. These fifteen categories can further be organized into three clusters: stakeholder issues, contextual realities, and project design flaws. As users of the taxonomy will notice, two working principles underlie this taxonomy. First, the causal mechanisms leading to these challenges overlap. Second, the challenges are at a level that is not too specific such that they fail to render any cross-project analysis, yet are not too general to prevent drawing actionable insights. These working principles were adopted partly because limitations of the data available and partly because of the immense complexity of unearthing root-causes behind how delivery challenges express themselves. A clear next step that remains is to identify the third level of delivery challenges by sector.

Unarguably, delivery challenges can shift in form and substance over time. Reflective of this fact, this taxonomy is a living document that feeds off new information as projects report from the ground. At the same time, it is encouraging to see that practitioners from organizations and governments such as the World Bank, GIZ and the Government of Cambodia have already begun using the taxonomy, and that it is helping them to better organize their work and guide discussions to improve operations.