Monitoring, Evaluation & Learning

Monitoring, Evaluation & Learning (MEL)

Altai Consulting has developed a strong expertise in Monitoring, Evaluation and Learning programmes in fast changing environments and created innovative approaches that support agile programming.

In the last 10 years, our MEL team has become increasingly involved in the monitoring of large regional programme portfolios, requiring the development of real time result tracking tools to support decision makers in the adjustment of their strategies. In response to this growing need, Altai designed dynamic TPML (third-party monitoring & learning) solutions to:

Track progress made at a multi-country / multi-sector level in real time
Aggregate, visualise and cross-analyse tangible outputs delivered by large portfolios
Monitor activities in real time and analyse the relevance and initial effects before making formal mid-term and end-line evaluations
Integrate results, lessons learned and best practices into programming (feedback loop)
Integrate contextual changes and key socio-political trends
Bring these analyses to the level of project managers and decision makers

Monitoring & Learning System of the EU Trust Fund (Migration, Displacement & Stabilisation) – EUTF Horn of Africa and Sahel - Lake Chad regions

The European Union Emergency Trust Fund for stabilising and addressing the root causes of irregular migration and displaced persons in Africa (the EU “Trust Fund”, or EUTF) was initiated at the end of 2015, following the Valetta Summit.

Altai Consulting has been tasked by the Horn of Africa and Sahel Lake Chad divisons of the EU Trust Fund to develop a real-time Monitoring and Learning System (MLS) at the regional level to help the EUTF management team and member states in Brussels better understand and measure the progress made by migration and stabilisation projects, track their outputs and outcomes and connect these projects to macro-level changes in the Horn of Africa.

The MLS was designed to facilitate results management and cross-country analysis, strengthen strategic planning, and promote more effective communication. The MLS covers a €4bn portfolio, comprising over 400 projects spread across twenty-one fragile and conflict-affected countries, and includes:

Monitoring: Developing a regional results management system. Altai is working with EUTF and its implementing partners to develop a set of common output- and outcome-level indicators covering all EUTF projects. A data pipeline has been developed and connected to all implementing partners, allowing for indicators to be updated on a quarterly basis and for the production of customised reports and data visualisations analysing outputs and outcomes, to track progress made and analyse regional trends based on relevant data.
Learning: Collating qualitative evidence of effects and facilitating uptake of lessons learned for future programme design and implementation. Altai is developing qualitative case studies on a sample of projects to better analyse the reasons behind successes and challenges met by individual projects and to capture best practice and lessons learned, to be shared and directly incorporated into project design (real time learning).
Technical assistance and capacity building: Altai is conducting a review of the portfolio’s existing logical frameworks and M&E systems, working closely with EUTF and implementing partners to strengthen projects' results frameworks, Theories of Change and improve their alignment with EUTF’s overall strategy and objectives. This component also includes strengthening the capacity of project M&E teams based on needs identified at the project level.

MLS quarterly and bi-yearly reports are available here

Learning Lessons from the EUTF - February 2021 Executive Summary is available here

Monitoring and Evaluation of the UK Conflict, Stability & Security Fund East Africa and Somalia Programme – FCDO

Altai Consulting, together with Itad (the prime on this contract), were contracted to establish a Monitoring, Evaluation and Learning (MEL) partnership for the UK Conflict, Stability & Security Fund (CSSF) programmes in Eastern Africa. The CSSF is a funding mechanism which supports the UK Government’s work in addressing conflict and fragility globally, under the direction of the National Security Council (NSC).

The MEL Partner format was designed to support the FCDO and MoD teams who contribute to the CSSF in strengthening their M&E capacity to design systems and collect evidence that allow them to document the impact trajectory of their local interventions and regional programs. At the same time, the MEL partner has supported CSSF to deliver on its monitoring, evaluation and learning requirements by providing formative research and thematic evaluations, third party monitoring, outcome assessments and lessons learned with both an individual program and cross-program focus.

Altai has notably led the following assessments in Somalia and Eastern Africa:

Substantive Review of the Building Resilience in Civil Society (BRICS), a CVE programme in Kenya
Research into CVE best practices in Eastern Africa and globally
Evaluation of BBC Media Action strategic communication campaigns in Somalia and East Africa
Research into militias and regional forces in Somalia
Baseline mapping of stabilisation activities in Somalia
Evaluation of Adam Smith’s International Mogadishu Security Programme
Evaluation of Stabilisation programmes and Early Recovery Initiatives in Jubaland and SWA
Baseline mapping of Stabilisation programmes in Somalia
Thematic research into regional non-state security forces

Learning and Monitoring Programme in the Horn of Africa – FCDO

Altai Consulting, together with Integrity, is delivering the FCDO-funded Learning Monitoring Program in Somalia (LAMPS).

The Learning and Monitoring Programme Somalia (LAMPS) provides third-party monitoring and technical assistance services to FCDO implementing partners throughout Somalia. In addition to developing a digital platform and a learning framework for FCDO partners, LAMPS acts as the vehicle through which FCDO-Somalia can monitor and improve its programming, providing credible data for effective decision-making and the articulation of best practices, lessons learned, and recommendations from across FCDO Somalia’s portfolio.

The LAMPS TPM contributes towards a robust evidence base, increased transparency, and greater accountability of FCDO programming across Somalia. It does so by selecting a representative sample of IP activities to be verified at the activity and output levels. Over the course of the four-year programme, LAMPS will conduct 800 verifications (200 per year). The LAMPS TPM objectives:

Increase FCDO’s visibility and insight into funded activities through data collection with key stakeholders and beneficiaries.
Inform programme decisions by triangulating information on what activities are being delivered, how they are being delivered and whether these activities appear to have a positive effect.

Integrated Monitoring & Learning Platform for AFD’s Minka Middle East Initiative

Altai supports the AFD (French Agency for Development) by providing integrated monitoring, learning and capacity building services to the Middle East part of its Minka Peace and Resilience Fund. Altai has developed an encompassing Theory of Change for this initiative, by interacting with the various AFD agencies. Various indicators of outputs and outcomes were derived from this analytical framework, and they are now tracked regularly in an online platform. The platform is developed and maintained by Altai (in partnership with its sister company Masae Analytics) and allows for a systematic exploration of the project portfolio and the quantified achievements (by country, type of beneficiary, etc.).

Altai also provides continuous capacity building services to AFD and its implementing partners in the region, on evaluation methodologies, logframes, definitions of indicators, data management, and conflict sensitivity.

Additionally, Altai conducts regular learning modules that gather all the stakeholders involved in interactive workshops around a key question, such as Social Cohesion, Mental Health and Psychosocial Support or aid localisation. The final outputs of these modules are available online.

Community Safety Programme and External Evaluation – DDG

DDG contracted Altai Consulting to conduct the external evaluation of its three-year CSP programme across the Somali territories (South-central, Puntland, and Somaliland). Altai sent teams in Gedo, Galgaduud, Banadir, Puntland and Somaliland (including Sanaag) to survey the CSP’s outcome in light of the goals stated in the Wadajir Framework for development in Somalia. The research was qualitative in nature (223 key informant interviews and paired interviews, as well as structured observation exercises) to measure the outputs of the programme focused on mine risk education, firearm safety education, conflict management education, community-police dialogues, security infrastructure (police stations and court houses) and explosive ordnance advocacy and disposal. The outcomes and impacts of the programme were measured through the same types of qualitative research. Altai evaluated the extent to which local ownership was achieved for the activities and newly established state structures – village and district level committees – and what this meant for the sustainability of achievements. Recommendations to enhance future programming and to improve M&E systems were made.

Media Programme Evaluation in the Great Lakes Region, La Benevolencija/University of Pennsylvania, Rwanda, Burundi – DRC

Radio La Benevolencija started broadcasting in the Great Lakes region in 2002, with a mix of informational and educational programs promoting conflict resolution, a sense of shared history and transitional justice based on analytical frameworks developed over the years by experts from John Hopkins University. Altai’s mission was to conduct a field-based cross-analysis of the programme in the Great Lakes as part of a wider evaluation led by University of Pennsylvania. Altai conducted field research in each country, which comprised of a series of qualitative interviews (between one and two hours long) with staff from La Benevolencija, staff from partner media, listeners and potential listeners in urban and rural areas, prominent civil society actors and government officials. The research was conducted in more central areas (Kigali, Bujumbura, Bukavu, Goma) and in rural communities that were selected depending on their geography, their conflict history and exposure to the programme. It led to the identification of key challenges, lessons learned and best practices as well as strategic and operational recommendations to help La Benevolencija improve its programming.

Monitoring & Evaluation of Action Against Conflict and for the Tolerance Project

Altai Consulting has conducted the final evaluation of the Action Against Conflict and for the Tolerance Project in Somalia (ACT) implemented by the Danish Refugee Council (DRC) in Eastern Somaliland (Sool and Sanaag regions) and in Puntland (Bari and Mudug regions). The research focused on conflict resolution mechanisms in some of the most unstable areas of Somaliland and Puntland. The report provides a comprehensive assessment of the programmes processes and impact, as well as best practices, lessons learned, and operational and strategic recommendations.

Evaluation of the Facilitating Partners of the National Solidarity Program (NSP) – The World Bank

Altai Consulting was mandated to conduct an evaluation of the facilitating partners of NSP, a large-scale World Bank-funded program (>$500m) implemented in 25 Provinces of Afghanistan. The facilitating partners included Care International, Oxfam, the IRC, AKDN, BRAC, ACTED and other major Afghan and international NGOs. The evaluation covered contractual fulfillments, adherence to the objectives and processes of NSP, management, cost-effectiveness and implementation, as well as a study of the social impact of the programme in more than 250 Afghan villages.