Guidelines for carrying out a citation analysis: following evidence from production to use

  • William Hankey
  • Gabriel Pictet
Keywords: International Federation of Red Cross and Red Crescent Societies; citation analysis; evidence; humanitarian crises


These guidelines describe the steps taken in Hankey and Pictet (2019) to carry out a citation analysis of a sample of the International Federation of Red Cross and Red Crescent Societies (IFRC) document base. The analysis followed evidence from production to use in order to assess what kind of evidence is produced and the degree to which it is taken up by other materials. These guidelines outline the steps taken to carry out the analysis and discusses theory in relevant parts. The first part covers document gathering and inclusion in the study, how to code the data, and briefly introduces the network analysis software used. The second part presents the approaches used, namely statistical analysis and network analysis. The former is used to analyse the dataset while the latter was used for metrics, visual analysis and analysis of the degree distribution. Finally, we discuss how the tools were used together to answer the research question. We hope these guidelines can help initiate a deeper discussion about how evidence is produced and used in the humanitarian aid and development sector.


Bastian M., Heymann S., and M. Jacomy (2009) Gephi: an open source software for exploring and manipulating networks, presented at the International AAAI Conference on Weblogs and Social Media, 17-20 May 2009. Home page:

Clarke, P. K. and Darcy, J. (2014) Insufficient evidence? The quality and use of evidence in humanitarian action, ALNAP/ODI: London, 87pp

Christoplos, I., Knox-Clarke, P. Cosgrave, J. Bonino, F. and J. Alexander (2017) Strengthening the quality of evidence in humanitarian evaluations. ALNAP Method Note 2017, ALNAP/ODI: London, 40pp

Davey, E., Borton, J. and Foley, M. (2013) A history of the humanitarian system: Western origins and foundations, HPG Working Paper, Overseas Development Institute: London, 60pp

Ham, K. (2013), OpenRefine (version 2.5). httep:// Free, open-source tool for cleaning and transforming data, Journal of the Medical library Association 101(3) 233-234

Hankey, W. and Pictet, G. (2019) Following evidence from production to use at the International Federation of Red Cross and Red Crescent Societies: where does it all go?, Knowledge Management for Development Journal 14(1), 38-66

Kalamaras, D. (2015) Social Network Visualizer (SocNetV). Social network analysis and visualization software. Home page:

Latour, B. and Woolgar, S. (1986) Laboratory life: the construction of scientific facts, Princeton University Press: Princeton, NJ, 294pp

Latour, B. (1993) We have never been modern, Harvard University Press: Cambridge, MA, 168pp

Levallois, C. (2017), Using filters. Accessed 20/08/2019 at:

Long, A. (2005) Evaluative tool for mixed method studies. Accessed 15/01/2017 at:

Newman, M.E.J. (2003) The structure and function of complex networks, SIAM Review 45(2), 167-256

Newman, M.E.J. (2010) Networks: an introduction, Oxford University Press: Oxford, 772pp

Ognyanova, K. (2012) COMM 645 Handout – Gephi basics. Accessed 20/08/2019 at:

R Core Team (2017) R: a language and environment for statistical computing, R Foundation for Statistical Computing: Vienna, 2673pp

University of Groningen (2019) Scale-free networks. Accessed 09/09/2019 at: