Evaluation and Impact Studies

• Evaluation and Impact Studies

IIE is at the forefront of measuring the long-term impact of international scholarship and capacity-building programs focused on education and development. Our evaluations and impact studies approach impact in a holistic way, understanding the outcomes of key interventions on beneficiaries, communities, and policy and programming.

Methodological Approaches 

Since its founding in 1919, IIE has played a central role in conducting research on international student flows and policy issues in international education. IIE’s evaluation services are provided to facilitate the collection of more comprehensive and policy-relevant data on international education, and to assess the impact of international fellowship and scholarship programs. IIE innovative approaches include the use of mixed-methods approaches, longitudinal designs and alumni tracking. IIE has a deep understanding of the various program elements needed to bring about change at the individual, community, and societal level. IIE’s approach to evaluation and impact is distinctive in four ways: a unique approach to the analysis of individual and communal change, leadership, social networks, and the use of participatory approaches. IIE’s methodological approaches have been highlighted in various publications and conferences.

Linking Individual Transformations to Social Impact

A majority of evaluations focus on individual recipient outputs and outcomes. A useful methodology that maps individual to communal change comes from a revised model of Donald Kirkpatrick’s (1979; 1994) Four Levels of Evaluation. Kirkpatrick’s model describes the levels of impact that measure change resulting from an academic experience, ranging from a short-term training to a full degree program. The model outlines levels of change starting from the individual and proceeds to measure change at the institutional level.

 

The Kirkpatrick Model, as adapted by IIE

Harnessing the Power of Leaders

IIE has designed an IIE Leadership Matrix, to identify key competencies related to leadership development, including strategic thinking, effective communication, and the abilities to understand greater context, to motivate others and to drive results. This Matrix allows IIE to understand better the particular impact participants of leadership programs are having. IIE recently used the Leadership Matrix to evaluate the impact of two leadership development programs:

• For the Cargill Global Scholars Program (CGSP), where the Matrix was adapted to conduct a pre and post analysis of the leadership potential of 52 students from five countries: Brazil, China, Russia, India and the United States.
• For the Higher Education Readiness (HER) program in Ethiopia, where the Matrix was translated into Amharic and adapted to high school level self-assessment.

Measuring the Impact of Networks

Recognizing that relationships can play an important role in program implementation and impact, IIE uses Social Network Analysis (SNA), to look into how program participants use their networks, new and old, to apply their learned skills and bring about change. SNA is a modern sociological tool used to identify the links between individuals in various social systems.

A sociogram depicts the social networks among scholars before and after a scholarship program. Circle size is larger for scholars with more connections. Each line is a connection between two scholars.

Ensuring Participants have a Voice

IIE uses innovative qualitative techniques to expand on outcomes and impacts beyond the individual scholarship experience. Participatory action research advocates for alternatives to traditional question-answer methods, using modes such as drawing to allow participants to express their transformation. This powerful tool can guide scholarship recipients through the transformative process of documenting change from the individual to the communal.

Importance of Learning

IIE links evaluations and impact studies to key programmatic learning points. The outcomes of an evaluation are not only important to the accountability and transparency of a program. They are also valid tools for learning, both for the program itself and other practitioners in the field.

• After the evaluation of the Higher Education Readiness (HER) program identified parents/guardians’ education, leadership development training and mentoring as program components that needed improvements, the program staff added parental/guardian workshop to the activities, increased leadership training, and expanded mentoring to include inspirational local role models. 
• Following the midterm evaluation of the Carnegie African Diaspora Fellowship Program (CADFP) which revealed low participation, lack of post-fellowship structure and guidance, and lack of alumni engagement of best practices, the program staff increased its outreach to higher education institutions, established a post-fellowship alumni awards, and created a formal social media platform for alumni communication.

Evaluating the impact of your international education programs? Work with us.