Appendix 10: General Education Capstone Exam
April 2005

Creation of the examination

          This exam was constructed by the General Education Task Force upon consultation with faculty members from a variety of departments (as appropriate depending on the question involved) and with collaboration from both the Assessment Committee and members from the WASC group working on educational effectiveness indicators. After the task force had a draft of the exam, it was critiqued by the two other committees and then redrafted. Two or three cycles of that process were completed before the exam was ready to be administered to students.

Description of the examination

The exam is constructed of three questions and is meant to take an hour so that it can be given during a classroom period. Much of the exam was constructed to test knowledge that has been integrated into a student’s thinking process rather than specific content knowledge.

The first questions consisted of a selection from a political speech. Students were asked to identify 3 specific passages where the author implicitly or explicitly casts America or Americans in the role of Jesus Christ. They were also asked to identify the aspect of Christ’s character or work that was implicitly referenced in each passage.

The second question was a real-life, complex math problem. The students were not asked to solve the problem. They were asked to come up with one or more ways of solving the problem. This question did indeed replicate the kinds of problems dealt with in the general education mathematics class called Problem Solving.

The final question involved asking the student to think of the kinds of concerns that various people, coming from different perspectives, would have on the same given issue. This question was designed to test the student’s ability  to think from more than one perspective. It also helps us see whether students have learned the different ways in which people can think about things.

Administration of the examination

Given the current absence of a specific day or time set aside for senior-year assessment, it was decided that the easiest thing to do would be to administer the exam during a session of various departmental capstone courses. Since the exam was not ready until fairly late in the semester, only about 6 departments were able to volunteer their class. We understand that the data is particularly influenced by the unrepresentative segment of seniors tested, but felt that this method was currently the best way of capturing an appropriate number of students. Whereas in 2004, using volunteers only, we had only 43 participants (about 8% of the graduating class), in 2005 we had 96 students take the exam (about 20% of the class).

Arrangements for grading of the examination

Before grading the first question, a small group of faculty (a literature professor, a language professor, and a religion professor) met and created a grading rubric for the exam (see below). A session was then held where five professors (literature, sociology, biology, psychology and language) graded the first question using the two grader agreement method that is standard in composition departments. Each exam was read by two readers. A score was obtained by consensus with exams on which the readers did not agree being given to a third reader as a “tie breaker”.

One thing that emerged from this grading session was the wisdom of using a sample of actual answers while creating the rubric. We discovered that we had not accurately guessed what would be the issue that divided good work from superior work and that our rubric therefore did not do an accurate job of distinguishing among the students and their level of understanding.

When the second question was graded by the Math department, they followed the protocol suggested above. They looked at a fair sample of the kinds of answers they got and then created a grading rubric (see below). Exams were then graded, with each exam being read by two readers. A score was obtained by consensus.  Any exams on which the readers did not agree were given to a third reader as a “tie breaker”.

The third question on this exam is being graded in the same manner. Currently, a scoring rubric for each disciplinary perspective is being generated. Then professors from that discipline will be asked to grade those answers that pertain to their actual discipline since we wish to know whether students can think from different perspectives.

Future Adjustments to the Examination

We learned several things through administering this exam. First, we need to give better directions in the first question to get the kind of answer we want from students. Second, because the exam tests integrated knowledge, we need to help students understand what is being tested. Some students were frustrated when they could not see specific facts being called upon.

Rubric for Grading the Religion Question

Task	4	3	2	1
Identify an appropriate passage in the speech	Passage contains a clear and complete idea	Passage has a theme, though may have more than one idea	Passage has varied ideas and themes.	No passage has been identified.
Identify an aspect of  Jesus’ character or ministry	States a specifically labeled attribute.	Attribute  is described but not labeled.	Confused combination of multiple characteristics.	No attribute stated.
Connect the right passage and element of Jesus Christ	Logical, precise link of attribute and passage.	Loose, general connection but not clear on detail.	Poor link or only vague one.	Incorrect connection or none at all.
Answer the right quantity of questions.	Selects three separate attributes. Does all correctly.	Answers 2 of 3 correctly. Or repeats once.	Gets only one correct or repeats same aspect of Jesus Christ 3 times.	Has none correct.

 

Rubric for Grading the Mathematics Question

Content	Score
Student could not identify the basic ingredients in a solution to the problem.	1
Student could carry out one solution or had legitimate partial ideas for two solutions.	2
Student could carry out one solution and had legitimate partial ideas for a second solution.	3
Student could articulate and work through two possible solutions with few or no mistakes.	4