The Office of Institutional Assessment and Studies coordinated the 2008-2009 assessment of undergraduate competency in written communication. A faculty committee composed of representatives of the undergraduate schools provided oversight for the process, from establishment of the learning outcomes and standards for the assessment to the determination of findings and recommendations. Recognizing the decentralized nature of curriculum and instruction at the University, individual schools and some programs were invited to design assessments of student writing that would address their own purposes and goals. Five separate assessments constituted the overall assessment of undergraduate student writing:
Assessment of writing by fourth-year students from the College (with some students from the Schools of Architecture and Nursing), although the assessment also incorporates findings from the assessment of first-year students participating in the First Writing Requirement (FWR). The assessments were designed to address three sets of questions:
- How well do fourth-year students write? Where is their writing strongest? Weakest?
- How well do fourth-year students write as compared to first-years?
- How much do students who took FWR courses improve their writing, and how do they compare to students who were exempted from FWR?
- Assessment of students’ writing within the ENWR program to investigate student performance both at the beginning and end of the ENWR writing course—for most students, a one-semester course but for some a two-semester sequence. Student papers were assessed on learning outcomes developed by ENWR faculty.
- The McIntire School of Commerce (SCC) conducted an assessment of students’ memorandum writing to investigate strengths and weaknesses in student writing both at the beginning and end of the fall semester in the students’ third year.
- The School of Engineering and Applied Sciences (SEAS) completed an assessment of student writing on Science, Technology and Society papers to evaluate student writing both at the beginning and the end of the program.
- The Bachelor of Interdisciplinary Studies (BIS) program assessed student writing on liberal studies and capstone papers to investigate performance in student writing both at the beginning (liberal studies) and the end of the program (capstone).
Student Learning Outcomes
Upon graduating from the University, students in the College of Arts and Sciences and Schools of Architecture, Commerce, Education, Nursing, and Continuing and Professional Studies who complete (or place out of) the First and Second Writing Requirements (FWR and SWR) should be able to:
- Create introductions that explain and propose to solve a problematic attitude, idea, or practice in their field and motivate reader interest.
- Compose balanced arguments that make specific, debatable claims, supporting those claims with precise, authoritative, and varied evidence.
- Acknowledge and respond to opposed arguments through dialogue rather than verbal combat.
- Produce a cohesive, clear, and coherent document with correct grammar, diction, and spelling.
- Choose a style appropriate to the goals, readers, situation, purpose, and structure of the argument.
- Draw and communicate conclusions that are logical, clear, and consistent with those proposed in the introduction.
In addition to these outcomes associated with FWR and SWR, students from SEAS develop additional skills from writing in their major:
Upon graduating from the University, students from SEAS should be able to:
- Produce prose that is correct with regard to grammar, diction, spelling, and sentence structure.
- Design documents that exhibit an understanding of audience, occasion, purpose, and structure.
- Frame introductions that quickly and reliably establish context and signal a document’s purpose to its readers.
- Delineate methods, present results, and reach conclusions that are logical and clear.
- Produce coherent and cohesive document subsections and paragraphs.
- Integrate appropriate graphics into the text and document sources in a correct and consistent style.
The following standards were established for graduating fourth-years:
- 40% of undergraduates are expected to be highly competent;
- 85% competent or above;
- 100% minimally competent or above.
Assessments were designed to provide information about student learning, that is, change in student performance (“value-added”). Cross-sectional designs compared first-year student performance with fourth-year student performance (CLAS/Arch/Nursing), or performance of students at the beginning of a program/course versus at the end of the program/course (SEAS, BIS, SCC). A pre-post design compared performance by the same students at the beginning of a course and at the end (FWR). All five assessments employed descriptive, skills-based scoring rubrics that reflected the pertinent learning outcomes.
When samples were required, IAS employed stratified random sampling technique to identify students. Students’ papers were de-identified to protect student confidentiality and to reduce the possibility of rater bias. To ensure reliability of the scoring, each student paper was scored by two raters separately and by a third rater if the scores from first two differed by more than one point. Each scoring session began with a “norming” to ensure that raters were in accord in their application of the criteria.
The methods for each assessment were as follows:
CLAS/Arch/Nursing: Value added and comparison to expectations for competency: A cross-sectional comparison of first-year and fourth-year student writing and a comparison of fourth-year students’ actual performance to expected performance. Faculty applied a scoring rubric to 179 first-year papers (from randomly-selected FWR courses) and 355 fourth-year papers (from a random sample of students).
FWR: Student performance was assessed “pre and post” the First Writing Requirement (FWR) course using a scoring rubric that reflects the FWR writing curriculum. From a randomly-selected set of FWR courses, 50 papers submitted to fulfill the initial writing assignment were randomly selected to serve as the “pre-FWR” papers. Final FWR papers from the same 50 students served as “post-FWR” papers. Faculty applied a 16-item scoring rubric to the 100 papers. Rater bias was controlled by randomly assigning papers to raters and de-identifying all student papers. Raters were also blind to whether they were rating an end-of-course or beginning-of-course paper.
SCC: A cross-sectional assessment of third-year student performance at the beginning and end of a writing course employed a scoring rubric reflective of the Integrated Core Experience writing curriculum. Eighty randomly-selected papers (40 from the beginning of the course and 40 from the end of the course) were assessed according to the scoring rubric.
SEAS: A cross-sectional comparison of first-year and fourth-year student writing in Science, Technology, and Society (STS) courses using a scoring rubric reflective of the STS curriculum (six learning outcomes plus four ABET outcomes, which were applied to the fourth-year student work only). Faculty used the scoring rubric to evaluate a random sample of 101 STS101 papers and a random sample of 97 STS402 papers.
BIS: A cross-sectional comparison of student writing in introductory (liberal studies) and end-of-program (capstone) courses, employing a scoring rubric adopted from the College of Arts and Sciences Writing Program. The faculty applied the scoring rubric to 46 papers randomly selected from introductory liberal studies courses and to all 27 capstone papers.
List of 2008-09 Committee Members
Writing Assessment Committee:
- Liz Arkush, Anthropology
- Dorothe Bach, Teaching Resource Center
- Claire Chantell, English
- Greg Colomb, English
- Ryan Cordell, English
- Jon D’Errico, English
- Scott DeVeaux, Music
- Deandra Little, Teaching Resource Center
- Josipa Roksa, Sociology
- Carl Trindle, Chemistry