… two years ago, and I haven’t even managed to write a diary entry about it. Has it really been that long? I’ve been meaning to post a summary for the longest time, but somehow life got in the way. Fortunately, this week David Garcia and me will host a workshop at the HOT Summit and present a talk at State of the Map in Heidelberg. I’m super excited about both sessions – you should join us if you’re coming! And it became the perfect excuse to dust off my draft for this post, and make sure all the research outcomes are finally assembled in one place.
But let me start in the beginning.
Hallo, my name is Martin Dittus, and between 2014 and 2017 I’ve accompanied HOT and Missing Maps for my PhD on community engagement in humanitarian mapping. We’ve had an amazing time together! During the PhD, we were trying to understand how best to build HOT volunteer capacity online and offline. How can we best train our volunteers so that they are available when needed? What kinds of support can we give them to ensure they don’t drop out early? And the age-old question: are the most highly engaged
contributors “born or made”?
HOT’s diversity of settings provided excellent opportunities to observe the outcomes of different coordination practices. The research was largely quantitative and observational, using data in the full OSM contribution history and the HOT tasking manager, combined with a lot of prep work to understand the nuances. This was very much a collaborative effort and a real partnership with quite a few groups and individuals in the OSM and HOT community, and the evaluations were always informed by real concerns in the community. I was blessed to find an incredibly active community in London, and made some great friends along the way. It made a massive difference that early on I met Pete Masters and Andrew Braye who then connected me to many others in the vast global network of enthusiasts and experts that is HOT. In addition to looking for answers to pressing organiser issues, we revisited some existing theories in social and behavioural science in the context of HOT, using statistical methods to analyse contribution activity across multiple years of HOT’s edit history.
We could demonstrate that coordination practices can have a marked impact on volunteer activity and retention, and the work has already achieved quite some impact within and beyond HOT. I’ve been using this OSM diary to document my research progress since 2015. Over time this has lead to significant debate and reflection within the HOT community, and on more than one occasion it has informed specific changes in organiser practice. In addition, four studies have been published in major academic venues, and two of them have received awards. We couldn’t have done any of this without a lot of community support!
Publications
If you’re curious you can read my full dissertation here: Analysing Volunteer Engagement in Humanitarian Crowdmapping (PDF). However I don’t expect many would want to read it in full, so for the impatient it may suffice to read the abstract on page 5, and the concluding summary chapter from page 151, in particular the summary of findings from page 153. (This by the way is often the best way to read academic papers: read the abstract to see if you like the paper, then read the conclusion to get the summary. You could then read the full paper if you’re still curious about the details.) The very impatient can get a 1-minute summary below.
The bulk of the thesis simply reproduces four papers that were published during the PhD, you can also get those separately. They’ve already been discussed in this diary in various forms, typically while the work was still in progress. It’s interesting to revisit the work now with some distance – you can really see a progression in the quality of execution. The papers are, in order of publication:
Analysing volunteer engagement in humanitarian mapping: building contributor communities at large scale (PDF). This is maybe quite a rough paper in terms of the quality of execution, but to this day it’s somehow the most widely cited one. Likely because it was one of the few quantitative studies of HOT at the time. I now think of it as an early exploratory study that establishes some basic concerns. A key finding: maybe unsurprisingly, it appears that complex task designs can be discouraging to newcomers.
Social Contribution Settings and Newcomer Retention in Humanitarian Crowd Mapping (PDF), where we investigate the role of mapathons as attractors for new and existing volunteers. This one was quite hard to complete, took longer than expected, and the findings were a bit underwhelming… quantitative methods are useful for certain things, but I think for a real evaluation of mapathon settings a qualitative research approach is likely more powerful. Broadly we found that attendees at one-off events at “corporate” mapathons were often quite committed during the duration of an event, but unlikely to keep mapping afterwards. By comparison, the monthly rhythm of the public Missing Maps mapathon appeared to foster more of a longer-term engagement.
Mass participation during emergency response: Event-centric crowdsourcing in humanitarian mapping (PDF). This paper is maybe my personal favourite. It only required some very basic statistics, but yielded some novel observations about the nature of community engagement in emergency response. We found that disaster response campaigns such as after the Nepal earthquake can be significant recruiting events, however that these newcomers might not stick around for very long… on the other hand, for many long-term contributors HOT engagement is very much characterised as a dormancy-reactivation cycle, responding to events as they happen rather than always mapping. Is this an opportunity to optimise HOT workflows, for example through the introduction of targeted notification channels when a need arises? Our quantitative methods were perfectly suited for this kind of analysis. Paper reviewers must have thought so too: it was awarded Honorable Mention at CSCW 2017.
Private Peer Feedback as Engagement Driver in Humanitarian Mapping (PDF). Here we looked at the impact of private validator feedback on the tasking manager on newcomer retention. This is a classic behavioural study based on loads of prior work, but with HOT as a novel setting – it’s a bit unusual because on the tasking manager, the peer feedback relationship with validators is more akin to private mentoring than the public rating you often find it on other platforms. Maybe as a result, we found that critical/corrective feedback in HOT did not actually appear to discourage people, in contrast to what you can commonly find on platforms with public reviews. One the other hand, feedback that included social affirmation and appreciation was significantly associated with increased newcomer retention. Maybe this is due to the online nature of the practice, the fact that HOT remote participation can happen in a kind of “depersonalised” space for many? In the absence of other prominent social cues, small phrases of support can likely have a powerful effect. However, note that like the other papers this is an observational study rather than a controlled experiment, so I would love if if people tried to reproduce the findings with other methods. In any case, this paper is another favourite of mine – and it was awarded Honorable Mention at CSCW 2018.
A very rough summary
So, coming back to the opening questions – are the most highly engaged contributors “born or made”? As you may expect, the answer is “yes” :)
To paraphrase the conclusion section of the dissertation, sustained community engagement is only partially a matter of optimising the contribution process. Overall it appears that a pursuit of indiscriminate community growth would likely be an inefficient use of organiser resources, in part because it seems unlikely that prolific mappers can be “created” by a particular process, and instead many will drop out early. Instead, other factors beyond the contribution process are likely just as important. This includes factors relating to the individual person, such as their interests and prior experience, and factors relating to their participation context, such as the presence of a supportive community that organises regular activities. However, the evidence also suggests that newcomers who are likely to become highly engaged can be discovered, for example through public events like mapathons and the promotion of major mapping actions, and these newcomers may then need some support in order to get started.
Advice to prospective PhD students
Early on during the PhD, my amazing supervisor Licia Capra recommended to structure the research as individual papers rather than a large monograph, and to try to publish each project as we go. This was a transformative decision that significantly improved my PhD experience! It gave me regular deadlines, I learned a lot from reviewer feedback, and it meant that the main work was already written up by the time I needed to produce the final dissertation. It meant I was able to finish the first full dissertation draft in only 8 days, plus a couple of weeks of refinements; and it meant that I was not stressed about my viva (the final verbal examination for PhD students in the U.K.), because the work had already been reviewed by experts, sometimes multiple times (I had to resubmit two of the papers after they were initially rejected for publication.) Over time this gave me a good sense of the strengths and limitations of the work, and I learned some key methods from the peer feedback that I relied on in the later work.
Granted, a paper-based approach may not be suitable for everyone; our particular research approach lent itself to project-based work. But even if you decide against this approach, I would recommend not to leave the writing until the very end. It is quite well-established that PhD life is a mental health hazard. In my personal experience, and from talking to others, this is in part because you will find yourself in a perpetually drifting state where it’s at times hard to tell if you’re doing well, and it is hard to retain a sense of certainty that you’re on track. Instead your work resembles a seemingly infinite and never-shrinking list of tasks with indeterminate outcome. So in addition to maintaining an active life outside the PhD, I would also recommend to structure your work in ways that help you manage these uncertainties. Anything that makes your work and your progress more tangible will help you in the long run.
And then…
Anything that comes after such a glorious journey with HOT and Missing Maps would have a lot to live up to, so I feel blessed that since 2017 I’ve been a postdoc at the Oxford Internet Institute, and I couldn’t have asked for a better place to land. I’m not sure yet if I’ll stay an academic for life, but for now it’s perfect. You can follow my work on Twitter.
OpenStreetMap Blogs
OpenStreetMap Blog
die Sammlung von Daten über wilde Deponien in der Gemeinde Ratoma in Conakry an, die auf einer 
(automatic 
