A real life workshop with a virtual facilitator

A few weeks ago I ended up as the virtual facilitator in a workshop that everybody else attended in ‚real life‘, at a pleasant venue in the countryside – and it worked out nicely! Here is how we went about it. Spoiler: Sunshine and plenty of greenery have played an important part.

The planned workshop was supposed to happen at a lovely place in the countryside, on a sunny late summer day. I was looking forward to enjoying being there, and working with a group of people who had hired me as an external facilitator. Then, two days before the workshop, COVID-19 arrived at my household. I was fine, testing negative, but my client felt it was safer I’d stay away from workshop venue. We had to regroup and reorganise, both on the human and the technical front:

On the human side, I needed a pair of eyes and ears in the room. We appointed a participant who would be my connection to „the room“ (that is how facilitators sometimes call the group they work with). That turned out to be essential, not only because ‚the room‘ was outdoors and all over the place. We agreed that the co-facilitator would devote most of her attention to her co-faciliating role, which involved not only eyes and ears, but also hands-on management of the participants‘ verbal contributions.

At the physical venue, there was something they called a ‚tower‘ – basically, a webcam and a multidirectional microphone on top of a set of speakers. When people took turns speaking, it worked well enough, but I could not see more than a fifth or a quarter of the actual participants. There was also a projector that initally beamed my face onto a videoconference screen – I quickly added an online whiteboard where I summarised key points on virtual post-its (instead of posters in the room).

Most importantly, there was the wonderful countryside outside. It had been my plan to organise plenty of small group work, anyway – so, for most of the day, I invited the participants to wander off in random or purposefully composed duos and trios and quartets to work in the vast outdoor space. During small group work, the co-facilitator would walk around, listen in here and there, and ring me up with information as to how the groups were doing and what subsequent steps would make sense. After each small group session (varying from 15 minutes to an hour or so), the participants came back to the conference room to share key conclusions, which I recorded on the virtual whiteboard, before sending them off again with new small group assignments (in varying groups).

Near the end of the day, there was a strong feeling that one issue needed plenary discussion – again, I decided to relinquish control and make use of the outdoor space. I provided only simple rules for the discussion that would allow every participant to speak up in a calm atmosphere, and asked the co-facilitator to remind participants of the rules if needed. (Hint: I use rules inspired by Nancy Kline’s Time to Think.) After an hour, everybody came back to the webcam, seemingly refreshed – which is extremely unusual for a long workshop day! – and equipped with important insights.

Would I do it again? With a co-facilitator, OK equipment and a pleasant space for the participants, absolutely!

Everyday evaluation template

The evaluation budget is too small to give serious attention to the 45 evaluation questions you are supposed to answer within four weeks? Hanneke de Bode has the solution! She has shared a long rant about the contentious power of evaluations on a popular evaluation mail server.

Hanneke contributed to a discussion about the lack of published evaluations commissioned by non-governmental organisations (NGOs). Arguably, one reason is the limited quality one can achieve with often very limited resources for smaller evaluations. She has made such a beautiful point that I am not the only one sharing this on my blog – our much-esteemed colleague Jindra Cekan is also going to spread it across her networks, with Hanneke’s kind permission. And here comes Hanneke’s 101 for small evaluations! Does that ring a bell?

Most important elements of a standard evaluation report for NGOs and their donors, about twenty days of work about 20.000 € (VAT included)

In reality, the work takes at least twice as much time as calculated and will still be incomplete/ quick and dirty because it cannot decently be done within the proposed framework of conditions and answering all 87 questions or so that normally figure in the ToR.


The main issues in the project/ programme, the main findings, the main conclusions and the main recommendations, presented in a positive and stimulating way (the standard request from the Comms and Fundraising departments) and pointing the way to the sunny uplands. This summary is written after a management response to the draft report has been ‘shared with you’. The management response normally says:

  • this is too superficial (even if you explain that it could not be done better, given the constraints);
  • this is incomplete (even if you didn’t receive the information you needed)
  • this is not what we asked (even if you had agreement about the deliverables)
  • you have not understood us (even if your informants do not agree among themselves and contradict each other)
  • you have not used the right documents (even if this is what they gave you)
  • you have got the numbers wrong; the situation has changed in the meantime  (even if they were in your docs)
  • your reasoning is wrong (meaning we don’t like it)
  • the respondents to the survey(s)/ the interviews were the wrong ones (even if the evaluand suggested them)
  • we have already detected these issues ourselves, so there is no need to put them in the report (meaning don’t be so negative)

Who the commissioning organisation is, what they do, who the evaluand is, what the main questions for the evaluators were, who got selected to do this work and how they understood the questions and the work in general.


In the Terms of Reference for the evaluation, many commissioners already state how they want an evaluation done. This list is almost invariably forced on the evaluators, thereby reducing them from having independent status to being the ‘hired help’ from a Temp Agency:

  • briefings by director and SMT members for scoping and better understanding
  • desk research leading to notes about facts/ salient issues/ questions for clarification
  • survey(s) among a wider stakeholder population
  • 20-40 interviews with internal/ external stakeholders
  • analysis of data/ information
  • recommendations
  • processing feedback on the draft report

In the Terms of Reference, many commissioners already state which deliverable they want and in what form:

  • survey(s)
  • interviews
  • round table/ discussion of findings and conclusions
  • draft report
  • final report
  • presentation to/ discussion with selected stakeholders

Many commissioners send evaluators enormous folders with countless documents, often amounting to over 3000 pages of uncurated text with often unclear status (re. authors, purpose, date, audience) and more or less touching upon the facts the evaluators are on a mission to find. This happens even when the evaluators give them a short list with the most relevant docs (such as grant proposal/ project plan with budget, time and staff calculations, work plans, intermediate reports, intermediate assessments and contact lists). Processing them leads to the following result:

According to one/ some of the many documents that were provided:

  • the organisation’s vision is that everybody should have everything freely and without effort
  • the organisation’s mission is to work towards having part of everything to not everybody, in selected areas
  • the project’s/ programme’s ToC indicates that if wishes were horses, poor men would ride
  • the project’s/ programme’s duration was four/ five years
  • the project’s/ programme’s goal/ aim/ objective was to provide selected parts of not everything to selected parts of not everybody, to make sure the competent authorities would support the cause and enshrine the provisions in law, the beneficiaries would enjoy the intended benefits, understand how to maintain them and teach others to get, enjoy and amplify them, that the media would report favourably on the efforts, in all countries/ regions/ cities/ villages concerned and that the project/ programme would be able to sustain itself and have a long afterlife
  • the project’s/ programme’s instruments were fundraising and/ or service provision and/ or advocacy
  • the project/ programme  had some kind of work/ implementation plan


This is where practice meets theory. It normally ends up in the report like this:

Due to a variety of causes:

  • unexpectedly slow administrative procedures
  • funds being late in arriving
  • bigger than expected pushback and/ or less cooperation than hoped for from authorities- competitors- other NGOs- local stakeholders
  • sudden changes in project/ programme governance and/ or management
  • incomplete and/ or incoherent project/ programme design
  • incomplete planning of project/ programme activities
  • social unrest and/ or armed conflicts
  • Covid

The project/ programme had a late/ slow/ rocky start. Furthermore, the project/ programme was hampered by:

  • partial implementation because of a misunderstanding of the Theory of Change which few employees know about/ have seen/ understand, design and/ or planning flaws and/ or financing flaws and/ or moved goalposts and/ or mission drift and/ or personal preferences and/ or opportunism
  • a limited mandate and insufficient authority for the project’s/ programme’s management
  • high attrition among and/ or unavailability of key staff
  • a lack of complementary advocacy and lobbying work
  • patchy financial reporting and/ or divergent formats for reporting to different donors taking time and concentration away
  • absent/ insufficient monitoring and documenting of progress
  • little or no adjusting because of absent or ignored monitoring results/ rigid donor requirements
  • limited possibilities of stakeholder engagement with birds/ rivers/ forests/ children/ rape survivors/ people in occupied territories/ murdered people/ people dependent on NGO jobs & cash etc.
  • internal tensions and conflicting interests
  • neglected internal/ external communications
  • un/ pleasant working culture/ lack of trust/ intimidation/ coercion/ culture of being nice and uncritical/ favouritism
  • the inaccessibility of conflict areas
  • Covid

Although these issues had already been flagged in:

  • the evaluation of the project’s/ programme’s first phase
  • the midterm review
  • the project’s/ programme’s Steering Committee meetings
  • the project’s/ programme’s Advisory Board meetings
  • the project’s/ programme’s Management Team meetings

very little change seems to have been introduced by the project managers/ has been detected by the evaluators.

In terms of the OECD/ DAC criteria, the evaluators have found the following:

  • relevance – the idea is nice, but does it cut the mustard?/ others do this too/ better
  • coherence – so so, see above
  • efficiency – so so, see above
  • effectiveness – so so, see above
  • impact – we see a bit here and there, sometimes unexpected positive/ negative results too, but will the positives last? It is too soon to tell, but see above
  • sustainability – unclear/ limited/ no plans so far

If an organisation is (almost) the only one in its field, or if the cause is still a worthy cause, as evaluators you don’t want the painful parts of your assessments to reach adversaries. This also explains the vague language in many reports and why overall conclusions are often phrased as:

However, the obstacles mentioned above were cleverly navigated by the knowledgeable and committed project/ programme staff in such a way that in the end, the project/ programme can be said to have achieved its goal/ aim/ objective to a considerable extent.


Most NGO commissioners make drawing up a list of recommendations compulsory. Although there is a discussion within the evaluation community about evaluators’ competence to do precisely that, many issues found in this type of evaluation have organisational; not content; origins. The corresponding recommendations are rarely rocket science and could be formulated by most people with basic organisational insights or a bit of public service or governance experience. Where content is concerned, many evaluators are selected because of their thematic experience and expertise, so it is not necessarily wrong to make suggestions.

They often look like this:

Project/ programme governance
  • limit the number of different bodies and make remit/ decision making power explicit
  • have real progress reports
  • have real meetings with a real agenda, real documents, real minutes, real decisions and real follow-up
  • adjust
  • communicate
Organisational management
  • consult staff on recommendations/ have learning sessions
  • draft implementation plan for recommendations
  • carry them out
  • communicate
Processes and procedures
  • get staff agreement on them
  • commit them to paper
  • stick to them – but not rigidly
  • communicate

Obviously, if we don’t get organisational structure and functioning, programme or project design, implementation, monitoring, evaluation and learning right, there is scant hope for the longer-term sustainability of the results that we should all be aiming for.

Less is more in evaluation questions

I am republishing this 2019 post because of a recent, heated discussion on a popular evaluation list server. It is about the harmful impact of excessive evaluation questions on evaluation quality.

Writing evaluation terms of reference (TOR) – that is, the document that tells the evaluators what they are supposed to find out – is not a simple exercise. Arguably, the hardest part are the evaluation questions. That section of evaluation TOR tends to grow longer and longer. This is a problem because: Abundant, detailed evaluations questions may lock the evaluator into the perspective of those who have drawn up the TOR, turning the evaluation into an exercise with quite predictable outcomes. That limits learning opportunities for everyone involved.

Imagine you are an evaluator who is developing an offer for an evaluation, or who is working on an inception report. You sit at your table, alone, or with your team mates, and you gaze at the TOR page (or pages) with the evaluation questions. Lists of 30-40 items totalling 60-100 questions are not uncommon. Some questions may be broad – of the type, „how relevant is the intervention in its context“, some extremely specific, for instance, „do the training materials match the trainers‘ skills“. (I am making these up but they are pretty close to real life.) While you are reading, sorting and restructuring the questions, important questions come to your mind that are not on the TOR list. You would really like to look into them. But there are already 70 evaluation questions your client wants to see answered and the client has made it clear they won’t shed a single one. There is only so much one can do within a limited budget and time frame. What will most evaluation teams do? You bury your own ideas and you focus on the client’s questions. You end up carrying out the evaluation within your client’s mental space. That mental space may be rich in knowledge and experience – but still, it represents the client’s perspective. That is an inefficient use of evaluation consultants – especially in the case of external evaluations, which are supposed to shed an independent, objective or at least different light on a project.

Why do organisations come up with those long lists of very specific questions? As an evaluator and an author of meta-evaluations based on hundreds of evaluation reports, I have two hypotheses:

  • Some evaluations are shoddy. Understandably, people in organisations that have experienced sloppy evaluations wish to take some control of the process and they don’t realise that tight control means losing learning opportunities. But! It takes substantial evaluation experience to provide meaningful guidance to evaluators – where evaluation managers have limited experience in the type of evaluation they are commissioning, their efforts to take control can be counter-productive.
  • Many organisations adhere to the very commendable practice of involving many people in TOR preparation – but their evaluation department is shy about filtering and tightening the questions, losing an opportunity to shape them into a coherent, manageable package.

What can we do about it? Those who develop TOR should focus on a small set of central questions they would like to have answered – try to remain within five broad questions and leave the detail to be sorted during the inception phase. Build in time for an inception report, where the evaluators present how they will answer the questions, and what indicators or what guiding questions they’ll use in their research. Read that report carefully to see whether it addresses the important details you are looking for – if it doesn’t and if you still feel certain details are important, then discuss them with the evaluators.

My advice to evaluators is not to surrender too early – some clients will be delighted to be presented with a restructured, clearer set of evaluation questions. If they can’t be convinced to reduce their questions, then try to find an agreement as to which questions should be prioritised, and explain which cannot be answered with a reasonable degree of validity. This may seem banal to some among you – but to tell from many evaluation reports in the international cooperation sector, it doesn’t always happen.

Five tips for remote facilitation

This is a rerun of a blog post I wrote a year before I started running training workshops on online facilitation (with the PME Campus, for example). All of what I wrote then is still valid. Since I have promised I would move some posts from my old blog to this new one, here is the post:

Despite the risks and uncertainties associated with independent consulting, I have never felt as privileged as I do now, living in a country with a highly developed, accessible health system, working from my customary home office, and equipped with a decent internet connection and the hardware needed to stay in touch with friends and colleagues. The crisis has been an opportunity to develop my remote facilitation skills. Before, I facilitated the occasional „real-life“ workshop in a conference room with video equipment, with participants in other locations joining us via Skype or the like. I have shared that type of hybrid experience on the Gender and Evaluation community pages. Now I have gone one step further, facilitating fully remote workshops from my home office. I mean interactive workshops with some 5-20 people producing a plan, a strategic review or other joint piece of work together – not webinars or explanatory videos with hundreds of people huddling around a lecturer who dominates the session. To my delight, virtual facilitation has worked out beautifully in the workshops I have run so far. Good preparation is a key element – as in any workshop. I have distilled a few tips from my recent experience and from the participants‘ feedback.

  • Plan thoroughly and modestly. Three to four hours per workshop day is enough – and there is only so much you can do in half a day. Factor in breaks (at least one per hour), time for people to get into and out of virtual breakout rooms, and at least five minutes per workshop hour for any technical glitches.
  • Try to make sure all participants can see each other’s faces. Some videoconferencing platforms allow you to see dozens of participants on the same screen. If you use a platform that shows only a handful of speakers, try to rotate speakers so that everyone can catch a glimpse of every participant. Apparently, recent research shows that remote meetings are more effective if people see each other. Smile! Keep interacting with your webcam and watch participants‘ faces as carefully as you would if you were in a room with them.
  • Pick facilitation tools that match your participants‘ digital skills. I love software that allows everyone to post „virtual“ sticky notes and move them around on a shared whiteboard. But that’ll work only if all (or a critical mass of) participants like experimenting with web-based tools. If many participants are uncomfortable with collaborative web-based visualisation, then you can record key points on the virtual whiteboard (life or between sessions), or ask participants to send their text contributions to you or your co-facilitator to post them on their behalf. The best way to gauge participants‘ readiness is a technical rehearsal well before the workshop (ideally, at least a week earlier).
  • Share a written technical briefing before the workshop. That should include (i) the links and passwords to the conference and the tools, (ii) guidance as to how to maximise data transmission speed – for instance, by using a LAN cable or by switching off WIFI on all non-necessary devices, by temporarily disabling Windows updates, closing all other computer windows etc., (iii) guidance on troubleshooting in case of major technical problems (e.g. alternative dial-in numbers, persons to contact if a participant fails to get back on-line), and possibly (iv) links to a couple of very short (1-2 minute-) tutorials for any software you may use for web-based joint visualisation or other forms of co-creation.
  • Do your homework. And give homework. If the digital tools you’ll use are new to you, try them out with colleagues and friends before the actual workshop. There is a growing body of video tutorials on the sprawling world of virtual collaboration; check out these resources. I also like quick primer for running online events on Better Evaluation which contains plenty of useful links. Before and in-between workshops, invite participants to try out any tools that are new to them, and/or to continue working on the collaborative virtual whiteboard.

It is generally recommended to work as a tandem, with one facilitator running the workshop and the other one looking after the technical aspects. But if you facilitate only one to two three- to four-hour sessions a week and you type really fast, then you can manage on your own. Be prepared, though, to feel totally exhausted after each session!

Take time when preparing (for) evaluations

In 2012, I published a post with the title above on my former blog. And I still see major evaluations with budgets running into hundreds of thousands of euros that come with a four-week inception phase, or that are supposed to start basically the day after the evaluation firm or evaluator has been selected. That is wasteful, because an evaluation that is not tailored to its users‘ needs risks being… useless.

Ideally, one should start planning evaluations right when the project/programme that is supposed to be evaluated starts. Back in 2012, I recommended to start recruiting evaluation teams at least six months ahead of the field work – at that time, the evaluations I had on my mind were evaluations of individual projects run by civil society organisations (CSOs). With anything bigger or more complicated, I’d plea for much, much more time for finding the evaluation team, briefing it and developing a robust evaluation design with instruments that fit their purpose. But the gist of my 2012 blog is still valid – and I had promised to re-publish a few of my earlier posts. Here it is:

There has been an extraordinary flurry of calls for proposals for external evaluations. This is good news; it suggests that people find it important to evaluate their work. But, upon closer examination, you’ll notice that many calls envisage the evaluations to begin just a couple of weeks after the deadline for offers, and to end within a month or so. That is frustrating for experienced consultants, who tend to be fully booked several months ahead. Narrow time-frames may also make it difficult for those who commission the evaluation to identify sufficiently skilled and experienced candidates. If you take evaluation seriously, then surely you want it to be done in the best possible way with the available resources?

Over the years, I have come to appreciate time as a major element of evaluation quality. Most development organisations (not only CSOs) cannot and do not want to afford full-fledged scientific-quality research, which typically involves plenty of people with advanced academic degrees and several years of research. That is perfectly reasonable: if you need to make programme decisions on the basis of evaluations, you can’t afford to wait for years. (The programme would be over, the context would have changed, your organisation would have changed their priorities, to quote but a few likely problems.) But what one can afford – even on a shoestring budget – is to allow plenty of time for thinking and discussing during the preparatory phase of an evaluation. In that way, you can make sure (among other things):

  • the terms of reference (TOR) express exactly what you need
  • the participating organisations are well-prepared and welcoming (which they are more likely to be if the TOR have been worked out with them and take their wishes into account)
  • the evaluation team understands what they are supposed to evaluate
  • the evaluators can reflect on different options, discuss these with key evaluation stakeholders, and let their thoughts mature over a few weeks before deciding on the final design
  • there is enough time to sample sites & projects/components so as to achieve a maximum of representativeness or a good choice of cases – to avoid visiting only what a Chinese expression calls „fields by the road“
  • data collection tools can be pre-tested, adjusted, those collecting the data trained and so forth

Extra time for these activities does not necessarily mean more consulting-days – just spreading out the days budgeted for, and identifying and finding ways of making better use of existing data in the project, can make a big difference.

Written surveys without writing

Back in 2013 my colleague Wolfgang Stuppert and I carried out a written survey that did not involve any writing! A useful instrument when you work with people with limited literacy. This is a ‚reprint‘ of the post I wrote on my former blog (developblog – now just an archive of posts from 2008 to mid-2021).

The survey was part of an evaluation of services for survivors of violence against women and girls in Mozambique. We felt it was important to gather basic data and feedback from as many women and girls who used the services as possible. But we had only little travel time in Mozambique and no resources to recruit and train dedicated enumerators who would adminster a survey on our behalf. Therefore, we decided to organise a written survey that the clients would fill in themselves. Some users, we were told, could not read and write well enough to fill in a form. Still, virtually anyone could hold a pen and tick off images. That is why we went for the following process:

We wrote up a set of short, simple questions, to be read out by the receptionist or other staff of the service centre to the client, just before the client would leave the centre. The questions were preceded by a straightforward explanation as to how the client would use the answer sheet (pictured below).

Of course we briefed the centre staff as to how to read out the instructions and questions, without paraphrasing or using their own examples, so as to reduce the potential for bias induced by those reading out the questions.

And this is how the clients recorded their answers on the exit poll: Each client received the answer sheet/card with rows of symbols, each row representing the possible answers to one of the questions. Each time the centre staff read out a question, the client would tick off the relevant symbol on the card. Sitting at a distance from the staff, she could hide her response.

At the end, the client would fold up the response card, staple it and insert it into a sealed box.

That process was organised during a couple of months preceding our ‚own‘ field work time in Mozambique. Upon arrival, the boxes were collected; we broke the seals and coded the responses. We did not come across anything that would have suggested ballot-rigging or other tampering by centre staff. And we were very impressed by the large numbers of answers, which generated quite interesting statistics – including some data that helpfully challenged our assumptions about the service users and their experience.


Unclutter your video appearance!

Videoconferences can be too revealing. Not just for people who forget to switch off their camera/microphone during breaks, only to blow their nose trumpet-style or even poke it (believe it or not, I still witness such moments)! Interior decoration or disorder can also be an issue.

I do not like to blur my background (most videoconference programs have an option to do that) because I tend to move and gesture a great deal, especially when facilitating. A blurred background can make limbs disappear when I move, or any objects that I hold. But I still want to have an uncluttered image on video, which is hard to achieve when there are shelves in the office or the desk is a bit messy.

My office holds some artworks that I do not want to have at home – and that I am not too eager to share with everyone I talk to on-screen. The picture to the left shows what the office looks like when I walk into it on a sunny winter day.

Now, the middle picture displays my most uncluttered background for videoconferences, and the right hand one an intermediate option which includes my cheerful yellow door. The latter is a selfie and my arms are quite short; in real videoconferences I tend to be more remote from the screen so that there is more empty space around me – I feel that introduces a nice sense of calm. It also provides a good background for my wild gestures!

This simple trick is to place the videocamera right on top of the (external) computer screen and rotate the screen to the only empty spot on my office walls. As you see on the picture to the left, the position of my desk is already slightly oblique, so that I can look out of the window without getting blinded by the (occasional) sunshine. If you have an external video camera (as opposed to a camera built into your screen), you can experiment with its place on the screen, shifting it more to the left or more to the right. Notebook/laptop users can use (or build, for instance with a cardboard box) a camera stand/tripod, to place the webcam at a comfortable height (eye level or above, unless you want people to peek into your nostrils) and turn it to the place where you have the best background.

Remote evaluation: a new norm

The German evaluation society DeGEval has published a discussion paper (in German) that provides guidance for remote evaluation. It is based on experienced evaluators‘ lessons from more than a year of remote evaluation.

The term „remote evaluation“ refers to an evaluation that is carried out by a person/team based outside of the country/region/place where the project to be evaluated has taken place. Due to travel restrictions linked to the COVID-19 pandemic, many organisations – especially those active in international cooperation – have commissioned remote or semi-remote/ hybrid evaluations. In most of the examples known to me, the team leader or sole evaluator has been working from Europe, conducting interviews, surveys and workshops online and by telephone, with people all over the world. I have carried out a bunch of remote and hybrid evaluations, too, assessing multicountry initiatives in global policy advocacy and local economic empowerment, country programmes of international development players, and regional learning initiatives. It has been an enlightening experience. I can fully subscribe to the conclusion the DeGEval paper reaches: European evaluators don’t need to travel that much.

The working paper balances the challenges of remote evaluation – you don’t get to meet people in person, you don’t visit the places where the project has happened… – against an impressive array of advantages. For example, a remote approach allows you to spread primary data collection over a longer stretch of time, because the evaluators do not need to squeeze all interviews into their two-week field trip. The money you save on travel-related costs can go into enlarging the data collection team. For instance, a colleague has recruited research assistants to carry out phone interviews. That has proven an excellent means to reach many more people, in many more places, than the number of persons the average evaluator can interview during a field trip.

The best thing about the paper is its conclusion. German readers, go to page 35 of the paper and read the last sentence! It encourages those who commission evaluations to carefully examine whether „international evaluators“ really have to travel. In many cases, evaluations can be carried out locally, supporting local consulting firms and research institutions. Where it is considered important to have someone from abroad on the team, a hybrid model may still be a good – and environmentally sound – solution. As much as I enjoy interacting with people in other countries and places: Often, we can make better use of our resources if we skip international travel.


Semi-audio interviewing

Your interviewee speaks via a mobile phone and their signal is too bad for a proper, two-way (remote) interview. What can you do? Reschedule the conversation? Opt for an ‚e-mail interview‘ instead? A new, hybrid method emerged in an interview I carried out a few days ago, after both videoconferencing via a popular online platform and audio conferencing via the interviewee’s favourite smartphone messenger had failed. I’d call it the ‚written question, spoken answer method‘, or, say, semi-audio interviewing. It is easy and astonishingly effective – as a method of last resort.

Basically, after two-way speaking had failed in that interview, I simply typed my first interview question and asked the interlocutor to respond directly with brief voice messages. Many (or all?) smartphone messengers (Signal, WhatsApp etc.) come with an option to send voice messages. That is less burdensome than typing on the phone screen. Also, the interlocutor’s voice adds nuances that written text can’t capture. The interviewee has your questions in writing, allowing them to focus. You can adjust or develop subsequent questions as the interview progresses. As an added bonus, you can listen to your interlocutor’s messages again after the interview, just like with a classical audio-recorded conversation, but visually structured by your written questions (i.e. easily searchable).

Of course, it is not a real-life real-time conversation. It doesn’t beat a regular two-way video conference, either. The interview progresses slowly, as it takes time for the voice messages to upload (remember, the signal is poor). On the other hand, that allows the interview partners to gather their thoughts – well, unless they are distracted by other incoming messages. But that is always a risk in remote interviewing.

Evaluator’s Dilemma

The expectations evaluators are facing have changed. The resources we get for evaluation have, too – but they still fall short of what it takes to meet new expectations. This week Friday, Ines Freier, Berward Causemann, and I will discuss this dilemma at the annual conference of the German Evaluation Society (DeGEval). The event is in German (see my announcement on my homepage). But I have summarised the dilemma in English as well – see the picture above. Does this ring a bell with my fellow evaluators?