Using Technology for Development Project Monitoring & Evaluation: An Interview with Linda Raftree

Linda Raftree is Senior Advisor for Innovation, Transparency and Strategic Change at Plan International USA; Special Advisor on ICT and Monitoring & Evaluation at the Rockefeller Foundation; and Co-Founder of the independent consulting firm, Kurante. She also manages the New York City Technology Salons, is on the board of the kiwanja Foundation, and is part of the Regarding Humanity project. An anthropologist by training, she has worked at the intersection of community development, participatory media, rights-based approaches, and new information and communication technologies for nearly 20 years.

Why is monitoring and evaluation, or M&E, important to development work and when do those designing development projects need to start thinking about building M&E into their work?

If you’re talking about monitoring and evaluation for an ICT project, it’s helpful to think about it in two ways. First, how will you measure the performance of your program? And, second, how might ICT be used to support that M&E plan? These are two separate pieces, and it’s important to address them individually.

The systematic monitoring and evaluating of ICT-enabled development projects is still relatively new. Monitoring is important because it can go a long way to helping you figure out what’s working and what’s not working. Monitoring can help you establish a framework through which you ask: What is my program trying to achieve, and what are the small steps I’ll measure along the way to ensure I’m on the road to achieving my project goal?

A lot of what we know about ICT4D projects is anecdotal. We don’t have a lot of good research about what makes projects work and what makes projects fail. So evaluation can be useful both to the project manager, and to the wider field so that other people can learn from you and avoid your mistakes.

Finally, it’s really important to think about M&E at the beginning of the project. If you don’t know where you’re starting from, it’s hard to measure where you’ve gotten to. If your project has already been underway without an M&E component, be reflective on what you’re already done and where you are: all that learning is really important. It becomes more difficult to do M&E midstream, but there are techniques that professional evaluators can use. It’s never too late to start.

New technologies are opening up all kinds of possibilities for improving monitoring and evaluation. Could you talk a bit about ways technology is being used to support M&E?

What ICT is doing for M&E is really broadening it out and allowing more people to participate. It also allows project managers to analyze data better and to make decisions more quickly. New ICT tools can make monitoring faster and more accurate.  Using mobile phones can also help you reach out to a wider group, and get feedback from the communities you serve. Some of the new ICT-enabled visualizations tools, including maps, graphs and charts, make it easier to analyze the data you collect and share it back to the community.

One way we’re seeing ICTs being used frequently in M&E is in mobile data collection. A lot of people are transferring paper surveys to a mobile format. There are cultural and technology challenges associated with making this transition, but it can help get data collected more quickly and, some say, more accurately.

Another way that new tech can help with M&E is on the qualitative side. There are interesting ways that video can be used. For example, a group called Insight Share uses participatory video to work with communities to understand the most significant changes that have happened at the personal or community level due to the project. If the process is done correctly, it can inform future project design and bring learning to bear on how future programs are developed and carried out.

Finally, while SMS is not a new technology, now almost everyone has access to a mobile phone. In addition to using text messages to survey communities, you can also use the cameras built into mobile phones to take before and after pictures, to have people to share their ideas about a project, and to draw out insights from a project.

How is M&E changing as a result of this technology?

Ideally, people would be able to collect and share information more quickly using new ICT tools. Data visualization tools are also helping people make better use of data, and make better-informed decisions about framing and refining the kind of data to collect in the first place. But we’re still at the beginning of using ICT for M&E and we have several big challenges before us.

One of these challenges is that we’re collecting a lot more data than we’ve ever collected before. We’re not always collecting the most useful data or headquarters may be asking for data that is unnecessary. This can be burden to people on the ground if the data is not actually used.

Another stumbling block comes around the sharing of M&E data. We haven’t completely cracked this one yet, but we do know there would be value in trying to collect information that could be comparable across different organizations. For example, data collected as part of a large mosquito bed net distribution campaign could be useful to the organization conducting the campaign and to the Ministry of Health. A lot of organizations are not thinking about this and checking which other organizations might have similar data needs before starting.

This gets to a larger conversation about standards in data collection, and the health field is a lot further ahead of other development sectors in this area. It also opens up concerns about how to protect respondents’ privacy, how to keep data secure, and how to ensure that respondents are fully informed about how data collected about them might be used.

What organizational changes, such as new policies and processes, are needed for organizations who are growing their monitoring & evaluation processes?

It’s important to keep in mind that using ICT for M&E is not necessarily cheaper when you start out. It requires a thoughtful process to change from paper to mobile.  It’s also important to be mindful that this is a behavior change process and needs to be managed properly. People may feel threatened or think that their jobs are going to be eliminated. Let people know what’s going on, and make sure people feel confident and well trained.

For organizations that are new to M&E it’s also important to be clear from the beginning about how the data and reports collected are going to be used. You don’t want to generate a lot of reports and then put them in a drawer. So think from the outset about how an M&E program is going to influence future decision-making and program design. This can help you identify what data to collect, and ensure that your organization, project or initiative can change according to what you learn.

—

On her blog at Wait…What?, Linda has several excellent posts about the use of ICT for monitoring and evaluation including: Twelve Tips on Using ICTs for Social Monitoring and Accountability, and The Benefits and Barriers of ICT-Enabled M&E.