Measuring the intangible: lessons from USAID partners on how to measure the impact of organizational learning and adaptive management

Authors

  • Ben Fowler Market Share Associates, USA
  • Katherine Haugh Dexis Consulting Group, USA
  • Manmeet Mehta Global Knowledge Initiative, USA
  • Monalisa Salib USAID, USA

Keywords:

international development, organizational development, USAID

Abstract

This paper explores lessons learned from a USAID-funded learning network of implementing partners, known as CLAIM, which developed innovative methods to measure the seemingly intangible contributions of collaborating, learning, and adapting (CLA) to organizational effectiveness and development outcomes. CLA is a USAID Framework and a set of practices that can be used to strengthen organizational learning and the conditions that enable it. CLA can be applied to USAID’s programming to improve organizational effectiveness and development outcomes. Learning network members developed and tested a range of methods and tools to measure the extent to which organizations and projects integrated CLA and whether that integration contributed to organizational or development outcomes. Approaches analyzed in this paper include: developing clear theories of change to determine what researchers would “expect to see” were CLA to contribute to outcomes; CLA self-assessment processes to measure the extent of CLA integration; and the use of pivot or change logs to document both CLA integration and its contributions. Learning network members also found the learning network model to be a useful approach for sharing and pooling learning across members, despite some structural challenges identified in this paper during the life of the network. The approach and early findings may be useful for a broad audience, including knowledge management professionals who regularly facilitate peer learning, and researchers who study difficult-to-measure outcomes in various technical sectors. Specifically, it may useful for those interested in measuring the contribution of organizational learning, knowledge management, and adaptive management to organizational change and development results. Challenges and limitations around time, samples, and resources impacted the ability of the learning network members to produce categorical evidence about the impact of CLA on organizational effectiveness and development outcomes. However, the initiative produced interesting findings, such as how pivot logs proved most helpful for capturing development outcomes but are subject to biases.

Author Biographies

Ben Fowler, Market Share Associates, USA

Ben specializes in identifying, implementing and measuring business solutions for positive social impact. He has 17 years of experience across more than 30 countries. Ben is one of a select group of auditors for a global social impact measurement standard. 

Katherine Haugh, Dexis Consulting Group, USA

Katherine Haugh is the Monitoring, Evaluation, Research, and Learning (MERL) Specialist for Dexis Consulting Group, providing technical assistance to the USAID LEARN project, funded by the United States Agency for International Development (USAID). Ms. Haugh specializes in monitoring and evaluation, data visualization, and organizational learning. Ms. Haugh was a co-facilitator for the CLAIM learning network.

Manmeet Mehta, Global Knowledge Initiative, USA

Manmeet Mehta is the Senior Program Officer at Global Knowledge Initiative, leading design and delivery of projects that facilitate the application of systems thinking and collaborative innovation strategies, tools, and methods for improved social impact. She has 14 years of experience in developing programs using systems thinking and user-centered design to strengthen the innovation and entrepreneurship ecosystem. Her experience includes creating a global mechanism for creating global access to the philanthropic capital market in the US and she has worked closely with nonprofits from over 110 countries on a range of projects aimed at deepening organizational capacity for innovation and effectiveness.

Monalisa Salib, USAID, USA

Monalisa Salib, MS OD & KM, is the Organization Development and Learning Advisor for Dexis Consulting Group, providing technical assistance to the US Agency for International Development (USAID) in organizational learning, development, and change. She has 13 years of experience in international development, was a co-creator of USAID’s Collaborating, Learning, and Adapting (CLA) framework and maturity tool and managed USAID’s approach to building the evidence base for CLA. Ms. Salib was a technical expert and co-facilitator for the CLAIM learning network.

References

Algoso, D. and Hemberger, A., 2018, ‘Measuring Adaptive Management’, USAID LEARN Internal Working Paper. Unpublished.

Askin, R. and Credle, V., 2018, ‘CLAIM: Collaborating, Learning, and Adapting Initiative for Measurement’, USAID LEARN Internal Working Paper. Unpublished.

Bourque, R., Derks, E., and Gross, L., 2018, ‘Evidence Base for Collaboration, Learning and Adaptation (EB4CLA) Final Report’, USAID LEARN Internal Working Paper. Unpublished.

Campbell, R., Undated, ‘A Framework for Inclusive Market Systems Development’, USAID Leveraging Economic Opportunities Brief. https://www.marketlinks.org/sites/marketlinks.org/files/resource/files/Market_Systems_Framework.pdf

Fowler, B. and Vasudevan, R., 2018, ‘EB4CLA Final Report for the CLAIM Network’, USAID LEARN Internal Working Paper. Unpublished.

Girard, John P. and Girard, JoAnn L., 2015, ‘Defining knowledge management: Toward an applied compendium’, Online Journal of Applied Knowledge Management 3(1), 14.

Haugh, K., Nadeem, A., and Salib, M., 2017, ‘What Difference Does Collaborating, Learning, and Adapting Make to Development? Key Findings from our Literature Review’, Washington DC: USAID. https://usaidlearninglab.org/library/what-difference-does-collaborating%2C-learning%2C-and-adapting-make-development-key-findings-our

Ives, K. and Mehta, M., 2018, ‘Feed the Future: Knowledge-Driven Agricultural Development Evidence Base for Collaborating, Learning, and Adapting’, USAID LEARN Internal Working Paper. Unpublished.

Mercy Corps and International Rescue Committee, 2016, ‘ADAPTing Aid: Lessons From Six Case Studies’, https://www.mercycorps.org/research/adaptive-management-case-studies

Senge, P., 1993, ‘The art and practice of the learning organization’ in Ray, M. and Rinzler, A., The new paradigm in business: emerging strategies for leadership and organizational change, New York, NY: J.P. Tarcher/Perigee, 126-138.

Shapiro, I., 2017, ‘Plausible Contribution: CLAIM Memo’, USAID LEARN Internal Working Paper. Unpublished.

White, N., Cardone, R., and de Moor, A., 2004, ‘Learning 3.0: Collaborating for impact in large development organizations’, Knowledge Management for Development Journal 10(3), 21-37.

USAID LEARN, 2017, ‘What Can the Federal Employee Viewpoint Survey Tell Us About Collaborating, Learning and Adapting in USAID Missions?’, USAID Learning Lab, https://usaidlearninglab.org/sites/default/files/resource/files/fevs_report_final_20170919.pdf.

Young, S., 2012, ‘The Value of Learning: Understanding and Measuring the Value and Impact of KM in International Development’, Knowledge Management for Development Journal (8)1, 2-12.

Young, S. (2019) ‘How USAID is building the evidence base for knowledge management and organizational learning,’ Knowledge Management for Development Journal (14)2:

Published

2020-09-11