Evaluation Q & A #2 with Wright & Wallis
In October 2019 SAGE MethodSpace focused on evaluation and other types of applied research.
Evaluation is often seen as an activity within the purview of non-profit organizations, NGOs, or agencies. But as we've seen in this month's focus, the skills and practices associated with evaluation are valuable for researchers, academics, educators, and students. We all need to know how to measure and understand the extent to which we are meeting our goals-- and how to do better next time.
This broad view is exemplified by the work of Bernadette Wright and Steve Wallis, co-authors of the book Practical Mapping for Applied Research and Program Evaluation. Mentors-in-residence for SAGE MentorSpace this month, they've agreed to answer some questions. This is the second of four Q & A posts.
You suggest the idea of creating maps from text as a class exercise in this handout, or from research findings in another handout. Can you share some tips for instructors who might like to try these exercises in the classroom?
An important part of the program evaluation process is looking at what others have found, and what that means for your evaluation. Practical mapping provides a systematic way to do that.
Instructors helping students learn about mapping and synthesizing literature on a topic may want to use Class Activity 3.1 (on page 87) in their classrooms. You can download the activity handout on the book website at practicalmapping.com.
In this activity, students work in small groups of 2 to 3. They start with a few sample excerpts of text from an actual publication. Their task is to read the text, identify concepts and causal connections, and diagram them to create a practical map. Then they discuss their results with each other and with the full class. The entire activity will take a little under an hour.
Similar to how you can make a practical map from the knowledge in the literature, you can also create a map from your own evaluation or other research that you conduct. We have found that clients often find a visual practical map to be a useful way to communicate and share evaluation results.
In Class Activity 4.3, students practice mapping from research using a short sample of text from an actual program evaluation. Similar to Class Activity 3.1, in small groups, students read the text, identify and diagram concepts and causal relationships stated in the text to create a practical map, and discuss their results with each other and with the full class.
Here are a few questions the students might consider and discuss:
What did you learn about the topic?
How clear (or confusing) was the text?
What different interpretations emerged during the activity and how did students resolve them?
What logical gaps exist in the maps (spaces where additional circles and/or arrows might be added – and to add them, what research is needed)?
Whatis the quality or usefulness of the knowledge according to the Knowledge Appraisal Matrix (Chapter One?
Teachers may modify both of these activities to fit their specific courses. One alternative is for teachers to replace the sample text in the activities with a different sample text that the teacher chooses, such as an excerpt from a report that relates to the focus of the class. For more advanced students, another option is for students to find an existing study through the university library or other source such as Google Scholar. Also, rather than all students analyzing the same text, you may want to have different student teams analyze different texts. For example, in classes where students are working on group projects, each group of students may practice developing a map from a study that relates to their project.
To learn more, see Practical Mapping for Applied Research and Program Evaluation.