Evaluation list

Executive Summary

The operation is characterised by its long duration and scale. It was amongst WFP’s top five emergency operations in 2003. It is also one of WFP’s most complex operations due to the combination of war and drought/chronic food insecurity, the frequent emergence of new crises and the necessity of developing a sophisticated air operation. The unique characteristics of and serious constraints faced by the operation must be acknowledged when assessing accomplishments and weaknesses, in particular the impact on staff time of securing and maintaining humanitarian access.

WFP’s emergency operation has been central to humanitarian efforts in Sudan and WFP has saved lives. This is widely recognised by donors, other humanitarian agencies and beneficiaries. WFP staff and implementing partners have distributed food in difficult and often hazardous living and working conditions, over sustained periods of time.

Unfortunately, it is not possible to quantify how many lives have been saved, and, as is recognized by field staff, it would be difficult in any event to attribute lives saved to food aid alone. It will always be difficult to measure results at this level, although WFP would be in a better position to demonstrate results if the successes highlighted by staff and partners had been better documented.

Under the current (third) phase of EMOP 10048, the prime objective of contributing to saving lives is to be effected by ‘improving and/or maintaining the nutritional status of target persons with specific emphasis on women.’ The specific targets set are the reduction of malnutrition rates to below the (2002) national average of 18 percent Global Acute Malnutrition (GAM), or maintenance below the present rate where it is less than 18 percent.

The 2003/2004 Annual Needs Assessment reported an overall GAM rate of 21 percent for both sectors, which is higher than the target and well above the ‘critical threshold’ of 15 percent. The reality is that unacceptably high malnutrition rates have persisted for many years in Sudan and unless a concerted effort is made by WFP and operational partners to develop a holistic and comprehensive approach to tackling malnutrition, there is little reason to believe that the situation will improve.

The second indicator specified in the current phase of the EMOP relates to outputs – timeliness of food distributions in accordance with EMOP plans. There was considerable under-delivery against what was planned over the three calendar years 2001-2003. In terms of commodities, 66% of what was planned for distribution was actually delivered (or air dropped) for final distribution. Under-delivery was most pronounced in the northern sector, and most pronounced for both sectors in 2003, when the targets were higher, however. This was compounded by an incomplete food basket – pulses, oil, salt and/or CSB were often missing. Although every effort was made to carry out distributions during the ‘hunger season’, food supplies were often available later than planned.

There have been annual increases in commodity requirements. Overall, the EMOP has been resourced to an average of only 70.4 percent in dollar terms over the three phases and the percentage of requirements met by donors has recently declined. The timing of commitments has been a problem, with the bulk of funds only registered during the second and third quarters resulting in shipments reaching Port Sudan or Mombasa during the second part of the year.

It has been possible to mitigate these difficulties by utilising carry-over stocks (in the first phase of the EMOP) and making extensive borrowings. The additional factor at play has been pipeline flow. The relatively large stocks on hand in the northern sector in December 2002, and again in December 2003 through February 2004, indicate some congestion over and above what can be explained by a normal build-up at this time of the year.

The major constraints are limited road and air capacity from the major WFP hubs and restricted access. As mentioned above, 66 percent of what was planned for distribution in the period 2001-2003 was actually delivered for final distribution. What is not clear is the exact proportion of commodities that has reached intended beneficiaries.

Although targeting has been a major preoccupation of the operation and numerous exercises have been undertaken to improve it, redistribution in the form of sharing on the basis of kinship is widespread. Although it may be hoped that redistribution favours the most vulnerable, or at least does not discriminate against them, we do not know enough to be confident. In addition to redistribution, there has been some diversion of commodities, further reducing the quantities available to intended beneficiaries. Taxation has occurred (and may still be occurring) in the southern sector where it has been organised by local authorities through traditional structures.

Non-civilians have also had access to WFP relief supplies in the southern sector, putting WFP staff and intended beneficiaries at risk. Distance decay (the relationship between distance and the breakdown in the effectiveness of targeting) is also a significant factor, particularly in the southern sector where it has been most difficult to establish distribution points close to beneficiaries. It should also be noted that planned ration sizes and duration have generally been modest and finely tuned.

This is positive because it maximizes the use of scarce resources, but it leaves little room for error in the event of undersupply, late distribution, redistribution, etc. Given the reductions that had to be made to the commodities distributed due to under-delivery, timing problems, and the various effects of redistribution, diversion, taxation and distance decay, senior managers need to consider the possibility that the hungry poor WFP seeks to assist in Sudan sometimes do not receive enough food to make a real difference nutritionally, and work through the implications.

More generally, the evaluation team believes that improving the effectiveness of final distribution has to be accorded a higher corporate priority by WFP – it goes directly to WFP’s core mandate. In order to be able to guide programming in Sudan and assess the progress and impact of its food relief operations, WFP needs to be more involved in the collection and analysis of nutrition data and information, including the analysis of the causality of malnutrition in areas with chronic food insecurity that appear unresponsive to ongoing food relief. WFP has a global

MoU with UNICEF through which UNICEF is expected to take the lead, but its terms related to nutrition are not being implemented by UNICEF for a variety of reasons principally related to capacity.

WFP has a strong logistics operation, particularly in air transport, with national reach and extensive coverage, putting WFP in a league of its own in relation to other agencies and organisations.

WFP logistics have long provided the backbone to Operation Lifeline Sudan. WFP recognises the need to substantially improve overland transport to move commodities, and substantial funds have finally been received to do so. But it will be only possible to switch from airlift / airdrop operations to road deliveries once the road repair and rehabilitation operations are well under way and peace becomes a reality. The funds requested for the rehabilitation of the road system and runways in the southern sector by way of a budget revision are significant (US$26.7 million) and the responsibility to be shouldered by WFP is considerable.

It is recommended that HQ assist the Logistics Unit in the southern sector conduct a risk management exercise in relation to the successful completion of these works. As is well known, the costs of conducting emergency operations in Sudan are high – the final cost per ton under EMOP 10048.2 Budget Revision 3 has risen to around US$1,000 due principally to the inclusion of the aforementioned works. But under the circumstances, with a transport system stretched to the limit and persistent insecurity in some areas, WFP has little leverage to contain costs. Together with the Crop and Food Supply Assessment Missions, the Annual Needs Assessment (ANA) exercise has been the most regular and extensive source of information on food security covering Sudan. But despite the time and energy put into conducting and strengthening the ANA by WFP, and wide participation, donors are not necessarily convinced by ANA recommendations. This was most notable in 2003.

The inclusion of recovery/peace related objectives in the EMOP, with modest targets for Food-For-Work, Food-For-Training and Emergency School Feeding Programmes (7 percent of commodities in the third phase), was appropriate, although the EMOP documents did not adequately connect planned activities to the achievement of the recovery/peace related objectives. This reflects inadequate corporate guidance about how to realize ‘recovery’ and ‘transition’ objectives. Food-for-Work and Food-for-Training activities have been difficult to get in motion and there have similarly been constraints on scaling up Emergency School Feeding Programmes. While concrete outputs have been achieved, targets have not quite been met. There has been no assessment of recovery outcomes for these activities, making it difficult to evaluate effectiveness. The setting of global targets, while necessary as a guide, has tended to ignore the local situation, resulting in some areas moving ahead with these activities when staff capacity, local IP capacity, and/or technical expertise are inadequate. Expectations among the donor and humanitarian communities of making the transition to more developmental activities have risen steadily in recent years. This has been positive but to some extent transition has become an imperative to move out of relief rather than a fair assessment of relief needs moving forward and the challenges of effectively utilising food assistance to support recovery and rehabilitation (the difficulties relate to geographic coverage, targeting, timing and capacity).

Without dampening enthusiasm for a shift from relief to recovery, senior WFP managers need to provide more guidance and counsel caution and realism. It also needs to be recognised that peace not only offers opportunities for recovery – it offers opportunities to improve the effectiveness of food relief (better coverage, better targeting, improved timing, more attention to micronutrients, etc) and thereby make a more effective contribution to finally reducing persistently high malnutrition rates. WFP has been a leader and an advocate for women within the Sudan emergency operation, a role clearly recognized by its partners, and has made progress towards institutionalizing Enhanced Commitments to Women (ECW).

The responsible staff members in both Khartoum and Lokichoggio agree that management must now shift the focus toward mainstreaming and better monitoring the implementation of the ECW. Overall, monitoring at both the output and outcome levels remains an institutional weakness, a reflection of practical difficulties and the lower priority accorded to performance information. Commodity tracking remains problematic. A major problem is that the Logistics and Programme Units monitor commodity throughput in a different way. A means has to be found of reconciling this information and presenting a readily intelligible overview to senior managers. The management structure for the EMOP, which is divided between two country offices, involves some logistical and programming inefficiency, and to this extent hinders the effective implementation of the operation.

However, WFP has no choice but to operate two systems in tandem and it is unlikely that a peace agreement would change this. It would be appropriate nonetheless for the CO Khartoum and CO Nairobi to jointly review the management structure of WFP’s operations in the Sudan (led by the ODK Regional Director or HQ). The evaluation team agrees that the EMOP should be converted to a PRRO for 2005 – it should bring more rigour to strategic planning and provide a longer timeframe. Relief and recovery strategies should now be developed, starting with locally determined strategies and targets based on realistic assessments of how best to meet local food needs and strengthen livelihoods.