Measuring the intangible: lessons from USAID partners on how to measure the impact of organizational learning and adaptive management
Keywords:international development, organizational development, USAID
This paper explores lessons learned from a USAID-funded learning network of implementing partners, known as CLAIM, which developed innovative methods to measure the seemingly intangible contributions of collaborating, learning, and adapting (CLA) to organizational effectiveness and development outcomes. CLA is a USAID Framework and a set of practices that can be used to strengthen organizational learning and the conditions that enable it. CLA can be applied to USAID’s programming to improve organizational effectiveness and development outcomes. Learning network members developed and tested a range of methods and tools to measure the extent to which organizations and projects integrated CLA and whether that integration contributed to organizational or development outcomes. Approaches analyzed in this paper include: developing clear theories of change to determine what researchers would “expect to see” were CLA to contribute to outcomes; CLA self-assessment processes to measure the extent of CLA integration; and the use of pivot or change logs to document both CLA integration and its contributions. Learning network members also found the learning network model to be a useful approach for sharing and pooling learning across members, despite some structural challenges identified in this paper during the life of the network. The approach and early findings may be useful for a broad audience, including knowledge management professionals who regularly facilitate peer learning, and researchers who study difficult-to-measure outcomes in various technical sectors. Specifically, it may useful for those interested in measuring the contribution of organizational learning, knowledge management, and adaptive management to organizational change and development results. Challenges and limitations around time, samples, and resources impacted the ability of the learning network members to produce categorical evidence about the impact of CLA on organizational effectiveness and development outcomes. However, the initiative produced interesting findings, such as how pivot logs proved most helpful for capturing development outcomes but are subject to biases.
Algoso, D. and Hemberger, A., 2018, ‘Measuring Adaptive Management’, USAID LEARN Internal Working Paper. Unpublished.
Askin, R. and Credle, V., 2018, ‘CLAIM: Collaborating, Learning, and Adapting Initiative for Measurement’, USAID LEARN Internal Working Paper. Unpublished.
Bourque, R., Derks, E., and Gross, L., 2018, ‘Evidence Base for Collaboration, Learning and Adaptation (EB4CLA) Final Report’, USAID LEARN Internal Working Paper. Unpublished.
Campbell, R., Undated, ‘A Framework for Inclusive Market Systems Development’, USAID Leveraging Economic Opportunities Brief. https://www.marketlinks.org/sites/marketlinks.org/files/resource/files/Market_Systems_Framework.pdf
Fowler, B. and Vasudevan, R., 2018, ‘EB4CLA Final Report for the CLAIM Network’, USAID LEARN Internal Working Paper. Unpublished.
Girard, John P. and Girard, JoAnn L., 2015, ‘Defining knowledge management: Toward an applied compendium’, Online Journal of Applied Knowledge Management 3(1), 14.
Haugh, K., Nadeem, A., and Salib, M., 2017, ‘What Difference Does Collaborating, Learning, and Adapting Make to Development? Key Findings from our Literature Review’, Washington DC: USAID. https://usaidlearninglab.org/library/what-difference-does-collaborating%2C-learning%2C-and-adapting-make-development-key-findings-our
Ives, K. and Mehta, M., 2018, ‘Feed the Future: Knowledge-Driven Agricultural Development Evidence Base for Collaborating, Learning, and Adapting’, USAID LEARN Internal Working Paper. Unpublished.
Mercy Corps and International Rescue Committee, 2016, ‘ADAPTing Aid: Lessons From Six Case Studies’, https://www.mercycorps.org/research/adaptive-management-case-studies
Senge, P., 1993, ‘The art and practice of the learning organization’ in Ray, M. and Rinzler, A., The new paradigm in business: emerging strategies for leadership and organizational change, New York, NY: J.P. Tarcher/Perigee, 126-138.
Shapiro, I., 2017, ‘Plausible Contribution: CLAIM Memo’, USAID LEARN Internal Working Paper. Unpublished.
White, N., Cardone, R., and de Moor, A., 2004, ‘Learning 3.0: Collaborating for impact in large development organizations’, Knowledge Management for Development Journal 10(3), 21-37.
USAID LEARN, 2017, ‘What Can the Federal Employee Viewpoint Survey Tell Us About Collaborating, Learning and Adapting in USAID Missions?’, USAID Learning Lab, https://usaidlearninglab.org/sites/default/files/resource/files/fevs_report_final_20170919.pdf.
Young, S., 2012, ‘The Value of Learning: Understanding and Measuring the Value and Impact of KM in International Development’, Knowledge Management for Development Journal (8)1, 2-12.
Young, S. (2019) ‘How USAID is building the evidence base for knowledge management and organizational learning,’ Knowledge Management for Development Journal (14)2:
Copyright (c) 2020 Ben Fowler, Katherine Haugh, Manmeet Mehta, Monalisa Salib
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.The copyright of the articles published in this journal remains the property of the authors. For liability reasons, the title belongs to the Foundation for the Support of the Knowledge Management for Development Journal. The journal is published under a Creative Commons Attribution Non-Commercial Share Alike License. This journal is currently an open access journal as it has a funding model that does not charge readers or their institutions for access. From the BOAI definition  of "open access", we support the rights of users to "read, download, copy, distribute, print, search, or link to the full texts of these articles." However, some of the content (2009-2012) is only available on the Taylor and Francis website. Within the next few months, this issue too will become available on the OJS.  http://www.earlham.edu/~peters/fos/boaifaq.htm#openaccess