October Provisions Session – The Culture of Assessment

For the podcast from the session, visit the mediasite or our Soundcloud page –Mediasite / Soundcloud 

The 2nd Provisions session of the year was on the Culture of Assessment. 27 attendees were present to hear talks from John Dion, of the Marking Department in the school of business, Dr. Claudia Lingertat-Putnam, the deputy chair of the Counselling and CSSA programs, and Dr. Stephanie Bennett, from the Sociology Department.

John Dion kicked things off by sharing his experience of assessment from the school of business. He asserted that the main desire of accreditors was to see that professors had a good understanding of what students should leave the program knowing. More specifically, what components of the curriculum students are expected to learn target material through, how to assess progress toward learning targets, and how to make changes if student are failing to make sufficient progress. Then, using the School of Business as an example, Dion outlined the process of developing learning outcomes, and curriculum mapping through which those desired outcomes may be realized. To see Dion’s PowerPoint presentation, click on the link – Assessment for Provisions

Next up was Dr. Claudia Lingertat-Putnam. She demonstrated how her program uses data to inform what they were doing. As of now, the school of education is operating under conceptual framework where there are eight standards to meet – Provisions Flowchart Program Assessment (The presentation).
The counselling students are evaluated in three phases during their master’s program to ensure they are meeting standards. Dr. Lingertat-Putnam described how well-designed rubrics, including those on Chalk and Wire may be used to facilitate evaluation of student progress across learning domains. Chalk and Wire is a particularly useful tool for professors in that, following the input of a rubric, it provides detailed student evaluation reports. Dr. Lingertat-Putnam’s student learning outcome assessment data, collected via a Chalk and Wire rubric, showed that her students were struggling with both their writing, and coming to grips with the APA style. With these problems highlighted, changes were able to be made. Due to the nature of these two problems, a trip to the writing center was seen as the remedial action to be taken.

Last but not least, Dr. Stephanie Bennett focused in on the use of rubrics and the important role that they play. Within the Sociology Department, herself and her colleague created pre and post-test rubrics. Based on their experiences, Dr. Bennett recommends that professors create rubrics that are divided by specific learning outcomes, and in which instructions are detailed and expectations are explicitly stated. Having implemented such rubrics, and thus improved communication between her students and herself, Dr. Bennett discovered that her students were more knowledgeable than she initially realized. Bennett explained that, while her students possessed the intelligence and understanding of the target material necessary to complete assignments well, they depended on clearly stated expectations to demonstrate that intelligence and subject comprehension. For example, some of Dr. Bennett’s students realized only when writing style was included on Dr. Bennett’s grading rubrics, that their manner of writing was as important to their grade as the content of their paper.

After the three presenters had their say, the floor was opened up for discussion. These were a few of the highlights:

  • Chalk & Wire can track progress over time, and inform professors on what works and what doesn’t.
  • Adjunct professors need to be more integrated into the process in order to maintain consistency throughout the faculty.
  • Faculty development in the form of mentoring can help to inform and reach out to adjuncts.
  • Writing skills need to be emphasized and reinforced throughout each year of the program.
  • Clearer guidelines and better communication with the students has generated richer discussions within the classroom.
  • A lot of data is collected, but it does not always make the transition to practical information.
  • Students are honest about their own performance, especially when working with well developed and clear rubrics.

For the podcast from the session, visit the mediasite or our Soundcloud page –Mediasite / Soundcloud 

Advertisements

The Culture of Assessment

In their 2013 article,  “Assessment culture: From ideal to real – A process of tinkering,” California State University Monterey Bay [CSUMB] professors Pat Tinsley, Marylou Shockley, Patricia Whang, Paoze Thao, Becky Rosenberg and Brian Simmons introduce a set of curricular assessment guidelines recently adopted by departments across CSUMB. According to Tinsley et al., implementation of these guidelines will promote “meaningful, sustained, and systematic assessment of student learning,” thus fostering a “culture of assessment.”  Their goals for operating within a culture of assessment include “increased curricular coherence” (i.e., across departments and between undergraduate and graduate curricula), helping students to identify milestones in the learning process and assess their own learning, and improvement of the curriculum to facilitate improved learning outcomes through ongoing assessment.

However, the interest in growing a culture of assessment has not been unanimous, and has even been controversial in the world of higher education. In their 2013 “Promoting a “Culture of Engagement,” Not a “Culture of Assessment,”  the Trustees of Princeton University cautioned that, by striving for a culture of assessment (i.e., with externally benchmarked measures), faculty risk promoting: a) standardization of curricula across departments and institutions, at the expense of diverse, individualized missions, b) the inappropriate evaluation of programs using generic/vague surveys and standardized assessments, c) undervaluing non-benchmarked evidence of learning, d) overvaluing standardized test results while undervaluing real-world outcomes like employment and fulfillment post-graduation, e) teaching towards the tests, f), self-validation of assessment policies with no external evidence to support their efficacy in improving real-world learning outcomes. In order to avoid these pitfalls, the authors recommend that institutions of higher education foster a culture of engagement rather than assessment.

References:

Tinsley, P., Shockley, M., Whang, P., Thao, P., Rosenberg, B., and Simmons, B. (2010). Assessment culture: From ideal to real – A process of tinkering. Peer Review, 12(1).

Retrieved from http://www.aacu.org/publications-research/periodicals/assessment-culture-ideal-real%E2%80%94-process-tinkering

&

The Trustees of Princeton University. (2013, September 12). Promoting a “culture of engagement,” not a “culture of assessment” [Remarks to presidents of the American Association of Universities (AAU), prepared for delivery at the AAU meeting on Oct. 23, 2012].

Retrieved from http://www.princeton.edu/president/eisgruber/speeches-writings/archive/?id=10613

Teaching in a Culture of Assessment

Megan Overby, Assistant Professor of Communication Sciences & Disorders, spoke of her responsibities to academic standards (those of St. Rose and of other organizations) as she teaches diagnostics and clinical writing to her students. Since Overby is able to find academic freedom within the educational standards she must uphold, she focuses on finding common themes to address within her discipline. One example of how Overby incorporates her culture of assessment into teaching is through her graduate students’ Cultural Observation Project. For this project, students must attend a cultural event, research the phonology of the culture they choose, interview someone at the event using a rubric, and write a reaction paper. Overby explained that the interview rubric not only focuses the student on necessary language and communication characteristics, it (and the project itself) is embedded with many other standards set by St. Rose and other important professional entities. Overby recognizes the positive experience her students continually have in this project while they are also meeting standards. Some of the standards students meet in this project are achieving knowledge of the discipline, observing cultural influences on communication style and parts of speech, learning about the diagnostic process by understanding something about a particular culture, and achieving an awareness of their community.

J. Daniel Beaudry, Adjunct Instructor for the English Department, presented on his experience using contract learning. Exercising his academic freedom, Beaudry experimented with this grading system first by addressing concerns like: Where do the standards go in this system? Might this system lead to grade inflation? Will this system just be more work for the professor? Because a teacher’s purpose it to teach and evaluate, Beaudry had to deal with the confusion for both professor and student when evaluation is redirected into instilling particular habits and processes while giving constructive, timely feedback. In order to construct a learning contract Beaudry suggested beginning by assessing what is in one’s subject that a good practitioner does all of the time. As an example of implementing contract learning and following this line of thought, Beaudy assembled a contract for a writing class that required good readers who could comment, criticize, and connect readings in order to write effectively. Though the contract Beaudry gave his students was long, he found that it was not confusing and overwhelming; rather it served as a helpful tool with a lot of information and prevented students from getting lost in assignments. Some benefits Beaudry has found with a contract grading system are a reduction of cheating (students do not fear grade damage), that there is no longer a need to justify grading, and when assessing the final work of students in a course, he is able to really discern exceptional work.

Jennifer Childress, Associate Professor in Art Education, presented on the assessment processes used by and on her art education students. Implementing a layering of goals, Childress uses textbooks that have longevity, addresses problems with solving and predicting in lessons, incorporates critical thinking skills, and works on reading and writing skills with her students. Childress finds academic freedom in her teaching and for her student teachers in being able to move around within rubrics always holding that the teacher is at least doing what she must. As her student teachers evaluate themselves and their lessons, they are asked to revise and reflect on what they did well and what they need to improve on—they score themselves (1, 2, 3, or 4). In this process Childress finds that her students become objective observers of data and they can utilizes readings that are pertinent to what they need to improve on. Also teaching professional standards, Childress has her student teachers write a letter to the next batch of student teachers. Not only does this exercise promote teachers helping teachers, it also engages her students with research and thinking through classroom exercises that help/hinder their students. Childress has found that her rubric evaluation system helps her students think about and perform quality work, rather than worrying about grades.