The Office of Institutional Assessment and Studies coordinated the 2009-2010 scientific reasoning competency assessment. A faculty committee composed of representatives of the undergraduate schools wrote the definition, learning outcomes, standards, and instrument for the assessment. Please direct all questions about the assessment to Sarah Schultz Robinson (982-2321).
Scientific reasoning is a mode of thought that:
- draws on systematic observation and description of phenomena;
- employs established facts, theories, and methods to analyze such phenomena;
- draws inferences and frames hypotheses consistent with that body of public knowledge and understanding;
- subjects explanations to empirical tests, including scrutiny of their declared and latent assumptions;
- and allows the possibility of changes in explanations as new evidence emerges.
Student Learning Outcomes
UVa expects all undergraduates to be able to employ scientific reasoning for their own purposes but especially for the purpose of evaluating the quality of scientific information, argument, and conclusions. Graduating fourth-year students at the University of Virginia are expected to:
- understand that, while scientific statements are in principle tentative, criteria exist by which they can be judged, including consistency with the body of scientific theory, method, and knowledge;
- display a grasp of experimental and non-experimental research design, including the notion of control, the idea of statistical significance, and the difference between causation and association;
- interpret quantitative data presented in graphical form;
- acknowledge the possibility of alternative accounts of events and properties and judge their relative plausibility by standard criteria;
- identify sources of error in scientific investigation, including errors of measurement and ambiguity of judgment;
- recognize unsound conclusions.
The following standards were established for graduating fourth-years:
- 25% of undergraduates are expected to be highly competent;
- 75% competent or above;
- 90% minimally competent or above.
The criterion used to indicate competence for fourth-year students was the total score on the scientific reasoning test. A score of 46-55 indicated minimal competence, a score of 56-70 indicated competence, and a score of 71-100 indicated high competence.
After a review of available instruments, the faculty committee developed an instrument to assess scientific reasoning. The test contained 18 questions for scoring: 11 multiple-choice questions, 5 short-answer questions, and 2 experimental design essay questions. Each question mapped to at least one of the learning outcomes. The test was administered at scheduled one and one-half hour test sessions. Results were reported and evaluated for the major disciplines in Arts and Sciences (Math/Science, Social Sciences, Humanities/Fine Arts) as well as aggregated for the University as a whole.
Approximately 500 4th-year students were sampled from five undergraduate schools at the University (Commerce, Engineering, Nursing, Architecture, and the College of Arts and Sciences) using a disproportionate stratified sampling method. Approximately 749 1st-year students also were randomly sampled to add a value-added perspective. The first year cohort results were compared with the fourth years, and at a future point in time the first-year cohort will be tested again to provide further perspective on value-added. All-school results for fourth-year students were aggregated to form an overall result for the University; first-year student results were used exclusively as a point of comparison.
Confidentiality and Compensation
Only students who consented to participate voluntarily were assessed. Confidentiality was ensured. Students who consented to participate were given a $20 gift certificate to Barnes and Noble to complete the test.
List of 2009-10 Committee Members
- Carl Trindle, Chair – Chemistry
- Tony Baglioni – Commerce
- John Bonvillian – Psychology
- Robert Bryant – Chemistry
- Michael Gorman – Engineering
- Theodore Homyk – Biology
- Paul Humphreys – Philosophy
- Victor Luftig – English
- Kathryn Neeley – Engineering
- Michael Palmer – Chemistry
- Donna Plasket – Continuing and Professional Studies
- John Portmann – Religious Studies
- Karen Schmidt – Psychology
- Thomas Smith – Environmental Studies
- Richard Steeves – Nursing
- Chip Tucker - English