Iowa State University

College of Agriculture

Department of Agricultural Education & Studies

Got a question?

Graduate Programs: Greg Miller 515-294-2583 / Undergraduate Programs: Mike Retallick 515-294-4810

Learner Outcomes Assessment Portfolio for the Undergraduate Curriculum

Communications Outcomes Progress and Assessment

The greatest needs for curriculum reform was identified by stakeholders (employers and agricultural organizations) and an external review team as the area of communications and ability to reflect on experience. Three courses were selected for improvements in this area. Two of the courses (311 and 315) were already considered communications intensive courses. The third (450) is a capstone course for the Agricultural Studies major. Changes to courses from 2004 to 2006 are discussed below.

AGEDS 311 Presentation and Sales Strategies for Agricultural Audiences (four sections)
AGEDS 315 Personal and Professional Leadership in Agriculture (two sections)
AGEDS 450 Farm Management and Operation (one section)

Changes to AGEDS 311: Presentation and Sales Strategies for Agricultural Audiences

This course has provided excellent training in oral communications. The goal for this course was to increase reflection on experience. Reflection is considered a means for enhancing learning of complex skills with affective dimensions (like public speaking), leading to greater ability to apply new skills to situations beyond the classroom. Two sections of the course now require, and provide feedback on, student written reflections on their own presentations.

Independence. “Independence” in an outcome demonstrated by the student in a situation different from the classroom environment in which students are provided less structure for performing the skill. When students use new skills in new places, instructors have evidence that transfer of learning has occurred. Consequently, independence measures are highly valued. In the AGEDS 311 course, independence is tracked in several ways. First, it is expected that students use demonstration and oral communications skills, including Powerpoint™ software, to give presentations in the AGEDS 315 course and the AGEDS 450 course. Oral presentations are required and scored by rubric in 315, but direct instruction is not provided in the 315 course or in the 450 course. Even less structured, but highly valued, students frequently give presentations in clubs and organizations.

Changes to AgEds 315: Leadership in Agriculture
The AGEDS 315 Leadership in Agriculture course was changed in fall 2004 to emphasize oral communications in group settings (i.e., use of flip charts and markers, brainstorming, fishbowl technique, and consensus building process based on (Kaner, 1996). These changes were based on (a) evaluation of course and instructor through end of the term anonymous Scantron survey, (b) feedback from stakeholders regarding the need for higher levels of communication abilities among graduates, and (c) AGEDS 315 outcomes assessment survey conducted by Grudens-Schuck and Cramer in spring 2004.

Peer facilitation. Students were required in Year 1 (fall 2004 and spring 2005) to complete one satisfactory Peer Facilitation in section 02 of the course. In Year 2 (fall 2005 and spring 2006), students were required to complete two satisfactory Peer Facilitations in section 02 of the course, and one in section 01 of the course. The main techniques are Small Group Facilitation and Small Group Recorder. Plans are to have both sections of the course require two facilitations per student. Students' skills are assessed by observational rubric. Rubric A was used to assess Year 1. This rubric was modified based on instructor and student feedback. Rubric version B is currently used to assess Peer Facilitation. The rubric is based on observation (criterion referenced assessment) and is considered a direct measure. The measure is also embedded into the curriculum because students use the scores as a basis for their Peer Facilitation grade (200 out of 1000 points).

Method for analysis of rubric scores as outcomes measurement. Rubric scores were analyzed for section 01. The data produced a portrait of current student functioning with respect to communications in group settings. Analysis of the data are provided in a chart titled, Analysis of Rubric Scores. Year 1 data were different from Year 2 data but could be analyzed similarly. Please note: all rubrics scored for students were analyzed, not just scores submitted for a grade (students were permitted to “drop lowest grade/s” by repeating a Peer Facilitation—in fact this in encouraged because practice leads to mastery.

Findings and discussion. Current functioning of students, obtained through direct measures of observed student performance, average a “B+” in grade terms, between 84 and 89 percent over three semesters. Please note, however, that scores are based on standardized criteria and are not norm referenced or curved, so comparison between years is legitimate for use in outcomes assessment. Number of rubrics at the “very high” level of performance is commendable, with 47% and 34% in Year 1 climbing to 52% percent in Year 2 under the “double performance” requirement. Number of rubrics below a threshold level of performance was steady at 2 or 3 per semester, highest percentage of 6%, which would seem laudable for a skill that was identified as challenging for our graduates. Most notable is the drop in number of “do-overs” when students are required to perform two Peer Facilitations. This could be explained by the “practice effect”—both their own and others’—which would also include more debriefing and observing of Peer Facilitation practice overall for all students.

Satisfaction and motivation. Changes in the syllabus also led to an increase by an estimated average of 2 points (of 9 total) in course evaluation ratings for Fall 2004 compared to earlier course ratings, suggesting greater student satisfaction and higher motivation related to the new subject matter emphasis.

Independence. “Independence” in an outcome demonstrated by the student in a situation different from the classroom environment in which students are provided less structure for performing the skill. When students use new skills in new places, instructors have evidence that transfer of learning has occurred. Consequently, independence measures are highly valued. In the case of Peer Facilitation, independence has been demonstrated in two ways. First, students are now expected and encouraged (but not required) to use small group facilitation, recorder techniques, and brainstorming in the AGEDS 450 course. The same flip chart paper and markers are provided out at the farm to ease transfer of learning. The instructor of AGEDS 450 has reported that prior to curriculum changes, zero students used flip chart and markers, brainstorming, or other group facilitation techniques. Students in Year 1 used techniques 2 times per semester. Year 2, students have already used the techniques 6 times. We are gaining! Another instance of independence is illustrated by four undergraduates who had completed AGEDS 315 who volunteered to assist facilitation of a regional farm policy event in November 2005. The experience was fast and furious, and required much more stamina than classroom practice, but the students did well. Moreover, the event increased their confidence and interest in group communication techniques. Students have also independently reported using brainstorming and other techniques in clubs and at conferences.

Changes to AGEDS 450 Farm Management and Operations
This capstone course changed its expectations to include more whole-class farm business meetings and written reflections. Moreover, as discussed above, materials with which students in the AGEDS 315 course trained (flip chart, markers) are stocked in the 450 farm classroom to stimulate transfer of learning. Additionally, the instructor from the AGEDS 450 course speaks every semester to both sections of the AGEDS 315 course to underscore the importance of group communication skills in co-managing the student-led AG450 farm during the capstone experience. Students are able to make the link between skill building and utilization in a real farming business, leading to higher levels of motivation for learning among Agricultural Studies majors.