Assessment for Experiential Learning by Cecilia Ka Yuk Chan
By Michelle Skowbo and Kara Mac Donald
True-to-life learning experiences can be deeply practical and motivating, so why aren’t they a greater part of students’ education? In Assessment for Experiential Learning, Cecilia Ka Yuk Chan acknowledges the numerous pitfalls that can sour students, teachers, and administrators against its adoption. Chan affirms that research supports the inclusion of experiential learning and gives steps for educators to make it meaningful and beneficial. She explores experiential learning issues, offers solutions, and tackles the often nebulous challenge of assessing experiential learning.
Chapter 1: What IS and IS Not Experiential Learning
Chan begins the first chapter by describing the two categories of experiential learning: field-based learning and classroom based learning. Field based learning can be “internships, clinical experiences, research and practical fellowships, apprenticeships, student exchanges, undergraduate research, practicums, cooperative education, service learning, and community-based learning. Classroom-based learning can be experiential, too: Chan lists “role-playing, games, case studies, simulations, presentations, debates, discussions, hands-on technology, and…group work” (p. 6). She notes the benefits and primary purposes for institutions, teachers, and students. Experiential learning provides relevancy and “holistic competency development,” (p. 9) i.e. developing multiple skills needed for a task. It also creates potential opportunities for reflection and feedback that prompts students to examine preconceptions and learn from mistakes. Chan concludes by acknowledging how experiential learning has numerous challenges. Quoting Fenton, Gallant, Pran, Seow, and Koh, she notes, among other considerations, the time, effort, safety measures, and challenged expectations that can frustrate those involved in this type of learning (p. 11).
Chapter 2: Experiential Learning Theories and Frameworks
In chapter 2, Chan provides an overview of two established frameworks for analyzing and designing experiential learning and then offers her own. The first framework, Edgar Dale’s “Cone of Experience,” categorizes different activities into “learning by doing,” “learning through observation,” and “learning through abstraction” (pp. 18-19). Another important theorist in experiential learning, David Kolb, offers principles of experiential learning and a framework which, like Dale’s, includes learning by doing and watching; Kolb’s also includes thinking and feeling as components (pp. 20-22). Chan expands and adapts these frameworks in her own “Chain of Mirrors,” which builds student metacognition into the effective development of experiential learning curricula. Her guide is made up of several components: motives (rationale), self-awareness of prior knowledge, expectations, experience, and learning; additionally, appraisal (or feedback) and reflection should be interwoven throughout a curriculum.
Chapter 3: Assessing Academic Knowledge and Experiential Learning
Experiential learning can be an opportunity for students to gain and solidify academic knowledge in conjunction with developing practical skills and holistic competencies. However, Chan paraphrases Qualters, noting that “the influence of confounding variables may affect the learning gains… and are varied and unpredictable, hence, difficult to establish and measure.” She notes a number of challenges that beleaguer all types of assessments, including treating assessments as “an end to the learning process (p. 48, quoted from Winstone & Boud),” assessments that are poorly designed (often due to logistical constraints), plagiarism, focusing on scores and rankings rather than learning, and discomfort with “innovative or unfamiliar” assessments. She offers a helpful list of questions teachers and curriculum designers can ask themselves to make sure that they know the what, when, how, and why of an assessment, giving ample consideration to whether it is valid, appropriate, practical, and humane. Experiential learning assessments have their own challenges, and Chan describes 15. Several of especial consideration are:
A practical experience may allow students to improve competencies in important ways, but if these competencies haven’t been identified, they won’t be included in the assessment.
Experiential learning takes time and effort to assess.
Institutions may be less willing to support curricula and assessments that don’t provide “traditional evidence of student learning.”
In the latter part of the chapter, Chan notes that, in addition to designing an effective assessment for experiential learning, students must be involved as partners in their assessments “to be able to critique, to develop, to judge, and to self-assess [feedback from summative and formative assessments]” (p. 70). This is key because students should see assessments as an opportunity for learning and further growth.
Chapter 4: Designing Experiential Learning Assessment
In chapter 4, Chan examines the common assessment approaches, and summarizes their advantages, disadvantages, and design tips. These approaches are, “blogs, direct observation, learning contracts, portfolios, posters, presentations, reflective journal[s], short answer questions, [and] written reports” (p. 84).
She also includes a helpful chart summarizing key assessment characteristics:
- Take time to set
- Take time to answer
- Take time to correct
- Take time to provide feedback
- Suitable for a large class
- Can use technology
- Passive approach
- Active approach
- Process Oriented Method
- Product Oriented Method
- (p. 85).
Even experienced teachers who are already familiar with blogs, posters, presentations, short answer questions, and written reports will find the acknowledgment of issues validating; the design tips and model rubrics for each approach are helpful reminders of how to make these assignments robust learning tools.
Direct observation is a common classroom tool, but its application to experiential learning may be less familiar to teachers. Activities that assessors might observe range from simulations to field studies. Chan affirms that “[direct observation] is a great way to assess practice skills and holistic competencies,” but “[g]rading criteria for observation assessment can be difficult to design, develop, and follow” (p. 89). Chan emphasizes that it is key that students know in advance when and how the assessment will be conducted. If community partners are doing the assessing, they will need guidance on giving feedback as well.
Like direct observation, portfolios and reflective journals are included in teaching method courses and are fairly common in practice; however, the precise way that they are implemented varies greatly. Chan teases out the details of these approaches, noting that a portfolio might include only evidence of meeting standards (a documentation portfolio), finished products along with drafts (a process portfolio), or a collection of “a student’s best work” (a product portfolio). Similarly, a reflective journal might be a distilled response to a predetermined prompt or a freer writing assignment. Both portfolios and reflective journals can include a variety of media. Chan emphasizes that students will need clear guidance on what they will be expected to produce.
Learning contracts involve teachers and students creating a “contract” of what students will achieve, what students will need, what “evidence [is needed] for indicating the learning objectives,” a timeline, and a rubric (p. 93). This pushes students to take charge of their learning and articulate their needs and interests. On the negative side, learning contracts are time-intensive and unfamiliar to many educators and students.
The latter half of the chapter includes suggestions for bundling these approaches together to create effective capstone projects. Chan summarizes case studies from Australia, China (with a focus on Hong Kong), Canada, Malaysia, Singapore, New Zealand, the United States, and the United Kingdom. The types of learning experiences include design projects, community service, field trips, student organizations, residential education, and exchange programs.
Chapter 5: Reflection as Assessment in Experiential Learning
In earlier chapters, Chan frequently highlights why reflections need to be a key part of experiential learning but notes in chapter 3 that although students who complete internships for academic credit often write reflections on their experiences, these are frequently treated as an afterthought and a box to check by both the writer and the assessor. Teachers and administrators may not take steps to set aside time to reflect on an experience. In contrast, Chan advocates for students to consider expectations (before), reflect on what is happening (during), analyze what happened and how the student feels (after), and plan for the future (after) in order to use reflection to fully benefit from experiential learning events. Chan lists educators who have teased out what differentiates between a surface-level reflection and an in-depth reflection. One of the more detailed models described is by Sparks-Langer, Simmons, Pasch, Colton, and Starko. This framework includes seven levels and the higher levels are as follows:
5: “Explanation supported by theory as rationale,” 6: Explanation with broader contexts,” and 7: “Explanation of ethical, moral and political issues” (p. 176). A handful of researchers have found that instructing students about how to write a higher level reflection does improve the quality of reflections overall. Chan closes the chapter with 13 tips. Some of these were expected: teachers and then students reflection guidelines, teachers should model reflective behavior, reflection assignments at various levels of depth should become a routine part of coursework, and give question prompts that students can pull from. She also addresses a “stickier” issue in reflective assignments: Is it ethical to assess students on such personal writing? One solution is to have “pre” reflections that are more personal and then a different assignment that draws on these informal reflections. Chan also suggests giving students flexibility in how they create what questions they answer in their reflections. She provides a helpful chart that students can draw from as they reflect on their experiences. She concludes by acknowledging, “Whether reflection should be assessed or not, I think [...] is something the teacher needs to design with caution” (p 184).
Chapter 6: Feedback in Experiential Learning
In chapter 6, Chan begins by noting that a number of researchers have delved into how to give effective feedback. She breaks the study of “the feedback mechanism…[into] four components” (p. 193).
The first component, quality of feedback input, includes timeliness and specificity. Quoting Nicol and Macfarlane-Dick (2006), she emphasizes that effective feedback should also be motivational, encourage further learning, and “help shape…teaching” (p. 194).
The second component, qualified sources of feedback, includes a discussion of how students perceive feedback from different participants. Ideally, feedback will come from multiple sources (peers, teachers, and external partners). External partners, for example, an internship supervisor, may need training on giving feedback. Students may not value peer feedback, and peers may not know what type of feedback they are qualified to give.
Chan provides a list of different types of feedback practices, the third component:
i. Stamps/Digital badges
iii. Simple Corrections
v. Comments to the whole class
vi. Generic exemplars
vii. Personal comments (198).
She encourages teachers to be open to technology that speeds up giving and distributing feedback to students, noting that audio and video feedback is generally better at “communicating…tone…and nuances” than written comments (p. 198). She also raises the thorny issue of determining the “boundary between assessment and feedback.” (p. 197). Some researchers, including Chan, argue against feedback being treated “as something that is used for grade justification” (p. 197).
Both teachers and students will need feedback literacy, the fourth component. Chan advocates “allocating in-class time for reflection after feedback is given,” and, quoting Carles and Boud, emphasizes the importance of students “appreciating feedback, making judgments, managing affect, and taking action” (p. 199).
Chan concludes with several thought provoking questions: “How do you build relationships with your students in order for them to feel comfortable with your feedback?” and “How do you help or train other stakeholders in the programme to provide constructive feedback in experiential learning?” (p. 202). She notes that while there is ample research on feedback in general, there are few studies on feedback in experiential learning settings, and that these questions, among others, should be more deeply explored in the future.
Chapter 7: Ethics in Assessing Experiential Learning
The author begins by stating that ethical issues around experiential learning are not always transparent for teachers and course/workshop designers, as ethics is intrinsically situated in individuals’ culture, social practices, religion, and values and beliefs. This can make the identification of ethical issues difficult, and therefore the chapter offers case studies to assist teachers and designers in recognizing ethical dilemmas in experiential learning, matched with suggestions to assist them in reconciling such issues.
A definition of ethics is provided, reiterating that they are personal values and beliefs that are based on one’s culture, social practices and religion that set a standard for what is right and wrong. The standard may be based on an individual’s worldview, but they are derived from an individual’s family, socio-cultural, religious, and ethnic background.
The author raises the issue that many teachers are not fully aware of a teacher code of conduct and little explicit instruction is provided on the topic, and less so for experiential learning. She presents dilemmas often present in experiential learning contexts, which are: i) forced or optional participation, ii) a teacher’s biased experiences, perception and expectation, iii) honesty in assessment, iv) student evaluation and teacher’s ethical responsibilities, v) emotion associated with experiential learning assessment, vi) student self-disclosure, vi) accessibility of reflections made in public spaces, and vii) complex relationship.
In the next section, the author provides three case studies from a variety of instructional contexts and fields (i.e., internship at an international consulting company, NGO community service project, university legal clinics providing pro-bono legal advice) to assist readers to notice and consider how professional standards and values may differ among teachers with different backgrounds and hence, worldviews of what is appropriate, inappropriate, right, and wrong. The final section provided recommendations for overcoming ethical dilemmas in experiential learning, which range from creating a trusted environment, fostering a code of ethics in teacher and student professional development, and understanding assessment in experiential learning, with guidelines for experiential instructional practice.
Chapter 8: Assessment Cases around the World
This chapter provides twelve case studies of experiential learning courses, activities offered, and types of assessment used from a variety of countries (i.e., Hong Kong, Singapore, Australia, New Zealand, Canada, the United Kingdom and the United States). Each case study description provides i) background on the course, ii) learning outcomes, iii) learning activities, iv) forms of assessment, v) teachers’ accounts, and vi) student reflections, along with video links for activities related to the course. The cases studies provided are:
“Transformative Business Immersion in Developing Economics”, Community Services, Business school, University of Hong Kong, Hong Kong.
· “Build Your Own Airship Experiential Learning”, School of Engineering, Hong Kong University of Science and Technology, Hong Kong.
· “Hands-on Work Experience – Assessing Practicum Health Sciences”, Simon Fraser University, Canada.
· “Understanding British Legal System – Assessing Pre-law Track Student Performance in the Law Course of London Internship Performance”, Boston University, The United States of America.
· “Building Personal and Workplace Skills – Assessing Humanities Internships”, University of Otago, New Zealand.
· “Practicing Geographical Fieldwork Skills – Assessing a Field-Intensive Geography Class”, University of Western Ontario.
· “Being ELITE – Assessing Engineering Project-Based Assignments”, Chinese University of Hong Kong, Hong Kong Canada.
· “Experiential Learning in Social Ventures – Assessing Social Entrepreneurial Internship”, Business School, University of Hong Kong, Hong Kong.
· “Camping for Ornithology- Assessing Student Learning Services in a Field-Intensive Biology Class”, Dalhousie University, Canada.
· “Understanding Interconnections between Water Resources and Water Engineering – Assessing Overseas Engineering Field Trip”, University of Washington, the United States of America.
· “Ready for the Chemical Industry? – Assessing Chemistry Internship Experience”, National University of Singapore, Singapore.
· “Understanding Contexts of Law and Legal Practice – The Undergraduate Legal Internship Programme”, University of Wollongong, Australia.
Each of these case studies offers distinct course design, instructional activities, and forms of assessment across various disciplines. The objective is that teachers and workshop/course designers can draw from several as they best inform their teaching context regarding experiential learning and assessment. The author also provides a link and QR code to access online assessment resources from the University of Hong Kong.
Chapter 9: Assessing Experiential Learning with Technology
Chan acknowledges the increased role of technology in education, even citing how the COVID-19 pandemic was a catalyst in altering the role of technology in learning but argues that pedagogy should always be at the core foundation for instruction and not technology. In experiential learning, like in most instruction, technology can be used as a principal tool for learning or a supplementary tool.
In the next two sections, the author describes examples of how technology can be integrated into experiential learning in both cases: as a core or supplementary tool. As a core tool, technology is often present in simulation-based learning, and many examples across diverse fields are described. For example, the author provides examples from the medical field, where technology offers students the opportunity to engage with diagnosis and treatment of an avatar patient, offering students a real-like experience with no risk to human patients. Game-based learning has elements that intersect with simulation-based learning and are often used in tandem to provide motivation for user engagement to practice skills learned. As a supplementary learning tool, technology serves a place to share course content material, archive student learning and experiences, and collaborate synchronously and/or asynchronously. Examples from the legal field in a tort law course are shared, where students view recorded videos in the classroom and identify examples of tort in their community, as well as other examples.
In the next sections, the author addresses the positive and negative effects of technology’s role in experiential learning, framed by the literature. Some examples discussed are the pleasant appearance of graphics and colors on the screen and the heightened engagement of learners from video, audio, and gaming. An example of the negative affects mentioned are how mobile device use in outdoor experiential learning can inhibit the learner from fully being immersed in the experience. The author follows with a case study description of a construction safety course, using a technology platform (i.e., YOCLE) that is student-centered and specifically designed for out-of-class experiential learning.
The chapter closes with a reflection that often the biggest obstacle is keeping up with ever-increasing technology tools on the part of both teachers and students. Additionally, students need to be trained to use the selected tools, and students with different internet and devices can pose challenges. Some recommendations to address these are offered.
Chapter 10: Quality Assurance and Evaluation in Experiential Learning
Experiential learning programmes need to be continually evaluated to ensure they are meeting the students’ needs. There are four types of experiential learning evaluation questions: i) needs evaluation, ii) process evaluation, iii) impact and outcome evaluation, and iv) economic evaluation. The author provides sample questions for each category.
In the next section, the author addresses the methods for evaluation programmes, stating that such assessments employ quantitative, qualitative, and mixed-methods approaches, as the purpose of the evaluation drives the questions asked and the form/s of data collected. The approach to programme evaluation can be done by comparing experiential learning with other instructional methods or by assessing changes in student learning before and after instruction. Common factors for measuring experiential learning are: i) cognitive learning, ii) attitude changes, and iii) behavioral/skill acquisition changes. Common data collection methods (i.e., surveys and questionnaires) are discussed, as well as grade performance, peer/supervisor observation and a multi-methods approach.
The last section of the chapter explores the value of holistic competencies of students participating in experiential learning which has become a standard to develop students’ broad skill set for better employability, but it can be challenging to measure objectively. The author shares her work with others in developing the International Holistic Competency Foundation (IHCF), which fosters the development of holistic competency development in experiential programmes, outlining the criteria for course certification under the scheme.
The book is very comprehensive and addresses many issues in identifying, describing, and documenting learning among students in experiential learning objectively. The value and promotion of experiential learning is well established in the TESOL field, especially in pre-service teacher training programs. Although the book’s focus is not on experiential learning in ELT, its discussions provide valuable information for ELT educators.