The World of Testing

I recently taught a seminar on assessment for the Graduate Program of Teton Science Schools. To engage the students, I began with a series of slides with comical solutions to standardized exam questions. For example, on a traditional multiple choice test, a question showed a triangle with one side labeled with the variable x and the other two sides identified with numbers. The question asked, “Find x.” The student responded by circling the x and writing, “here it is.” A correct answer, but not the “right” answer. Standardized tests: the word evokes joy in some (those that excel at these tests and policy makers who find an easy data point to use for measurement) and pain in others (those that are taking the tests or schools that are penalized by the results). The week before break, all Journeys School students in grades 3-8 took an exam called the CTP-4 created by the Educational Records Bureau. Students spend four mornings filling in bubbles (either on paper for the younger students or on the computer for the older students). Due to our independence as a school, we do not have to follow the guidelines of No Child Left Behind, the federal law that mandates testing for all public school students. This freedom allows us to use these assessments as they have been designed – to help watch students grow over time and to help inform our teaching. Teachers will use the data from the test, along with hundreds of other data points, and paint a picture of a student over the course of the year. As we distribute results to parents, I know that they will be looked at with more scrutiny than the in-class essay, performance, project, or unit test by some parents. Every year, I preach caution about the interpretation of these scores. Most research suggests that results of large scale norm referenced tests are to be interpreted carefully and with limitations. Scores are heavily dependent on the timing of the taught curriculum, the comfort level of the test-taker, and the linkages between the content on the test and the content taught in the school. As more weight is put into standardized testing in the United States (contrary to the rest of the world), the public curriculum moves further from a liberal arts education to a narrow curriculum teaching only a small fraction of the knowledge and skills necessary for an adult to succeed in a thriving democracy. This is no way to create an innovative, intelligent, and thoughtful society. Organizations like fairtest.org promote changes in how the tests are being used and how the tests are constructed. At the university level, more and more highly selective schools are test optional for admissions which acknowledges that the success of a student is not predicted by test scores alone. Even the ERB organization itself cautions against any interpretation of changes in norms. Increases in scores are normal and expected but norm scores have larger errors (greater than one stanine (all norms are broken into nine stanines)). So where does this leave Journeys School? We analyze the results of the tests as we do any other assessment. We look for deviations from expected results in individual students and we investigate patterns in the larger content areas. These investigations allow us to determine how well our students performed on the content areas assessed by the exams and how well they take standardized tests. As most high school students have been told, there is a skill to taking a standardized test that has nothing to do with the content on the test. The large scale industry around test prep is an indicator of this (luckily Journeys School offers free test prep to all of our high school students – outside of class hours). We also are beginning a project to closely analyze the content on the CTP-4 and compare it to the content in our classes. It is not our intent to teach to the test, but to better understand how this test can help inform how to better teach our students. As I reviewed the scores this week, I was reminded of another test response from my presentation. The question asked, “How far will a ball roll down a hill before stopping?” The student responded by drawing a dragon in the middle of the hill and stating “the ball will stop when it hits the dragon.” This question would have been marked wrong, but certainly is a measure of some level of creativity. (For a good review of the literature on standardized testing, please see http://jalt.org/test/edw_1.htm. )