Skip to main content

A former test scorer tells all

The other day I promised to bring my insights to the topic of new standards for nonfiction in the English classroom as a tool to improve test scores. But, forgive me, I'm going to try to get wider...though not too wide, hopefully.

I'm going to use this article on vocabulary being linked to academic (and life) success as my jumping off point, but you don't need to read it to appreciate the bigger picture. The topic is a big one, but let me just give my take via what I know best. Testing.

Over the years, I've been a Team Leader and test scorer for two different test scoring companies. Both with very different approaches to proficiency scoring. One very industrial in their approach to measurement. One more relaxed. But, of course, it comes down to the states and what they want/require.

For those not familiar with how the process works, there are companies that specialize in educational measurement and contract with individual states to essentially grade the written portions of your student's standardized test. Whether they are measuring proficiency or required to graduate or continue to the next grade level, the test gets a control group of rubrics to which all 10,000 tests (or however many) will be compared.

My job then was to train test scorers--real people with college degrees, teachers, people with a background in the topic usually--to be able to recognize where any given student falls on the rubric. Often a score of 0 to 5. With 5 being "amazing, this student fully comprehends and has done a great job." To zero being, well, nothing. Pencil marks on the page. No attempt to answer. Off topic. And scores in between depending on grade level, the leniency of the state standards, and--to be honest--a lot of attempts on the part of the test scorers to give students the benefit of the doubt so they'd pass.

The tests are scored multiple times...usually by 2 lower level graders then possibly by a third or even fourth supervisor if scores are in conflict. There were rules...if the student gets a 3 and 4, fine. If the student gets a 2 and a 4, it gets sent to me, the Team Leader, to evaluate who is correct. I may need my boss to get clarification from the state.

I generally scored history and social science. Usually essay questions on the Industrial Revolution, basic economics, the Underground Railroad, the impact of the cotton gin on slavery, topics like that. Over my years in the business, I saw 3rd graders up to seniors trying to graduate. Sometimes, it was ugly. And would come down to buzzwords teachers had taught students to use--without really knowing them--to gain favor with the test scorer. Few and far between were the students who wrote you a coherent essay. More likely, it was gibberish you were trying to decide whether it elevated them from terrible to mediocre.

With that in mind, I have a word or two to say about proficiency testing. It stinks. And is pointless. And measures nothing. And we're obsessed with it. That said, we do need some way to measure student achievement and evaluate learning.

Which brings us back to the article and, especially, to the increased use of nonfiction in the English classroom as a tool to improve language skills. I have mixed feelings. I'm not opposed to including nonfiction. I'm a nonfiction reader myself. But I also recognize the role of literature not only to history and culture, but the need to learn good storytelling, literary techniques, language devices that only come up in fiction, etc..

The problem, really, is about the use of testing and state curriculum standards to narrow the scope of student knowledge. The real issue is with our getting our students excited about not only depth but width of subject. Our children are taught not just too few concepts, but they're not given the classroom time to pursue certain topics in greater focus. So we come away, for instance, knowing less science and having had less contact with the broad category of science. Our kids might get biology and chemistry, but how many have been exposed pre-college to engineering or archaeology or robotics? Some of that is changing in the very best districts--mine being one of the lucky ones where we have a competitive robotics program in middle school--but generally across the nation this isn't the case.

Obviously, here in Chicago, the Mayor made classroom time an issue...if you don't know the history there you can search for it. But with it came a fight over teacher pay, how the time would be used, and the like.

Kelly and I have this discussion quite frequently about potentially homeschooling, the limits of our academic system, the lack of topic diversity in our educational institutions. The problem with bringing in more subject matter is that teachers simply do not have time--or sometimes the knowledge themselves--to teach in depth about highly specific yet integral subject matter. But, Kelly and I argue, that comes from two things: not identifying student strengths and weaknesses enough to tailor academic programs to the individual and the loss of technical programs or career tracks for some students. If you know you want to be an architect, that certainly helps form your schooling experience. (What to do about students who don't know or change their minds is another post altogether!) Kelly frequently mentions how much better other nations are at identifying winners and losers earlier. Not to create castes of success and to doom others. But, rather, to better serve both those populations. We need to be better about the way we funnel children towards specific areas of academics, in my opinion.

Not that I'm against a general, broad sense of academics. All students should be required to have some academic diversity outside their field of interest. But if a child is interested in science or math or Spanish or computers or history we should be doing all we can to give them outlets for their interest along with a very, very intense environment to get a feel for the range of themes in that subject. If a child is into computers, it's not enough to teach them programming. They need to know security, graphics design, techniques for organizing massive amounts of data and information, etc..

And this is a far reach from our current hope that a student at Grade 12 can briefly mention the inflation rate or the migration of people to cities for factory work. Which makes the issue of, "hey let's tie test scores to teacher pay" a bit of a red herring. Ok, a lot. It's just a jumble of loose concepts we hope we're drilling into them...without putting them in any sort of larger context.

When I ask myself if I want my kids in the system that rewards them and their teachers for knowing the answers that will be on the test of state standards for basic function, I hang my head. As a victim success story of the public school system, I can honestly say that I feel like my duty to them is to take a more active role in their overall personal education than they will currently get. Yes, two parent household and involved dad linked to academic success. Blah blah. I actually turn the idea on its head...people always say the parents have to be involved. Well, the school systems need to be involved as well.

The low bar of students showing up, showing rudimentary knowledge, then being passed off to someone else to repeat the process is shooting American students in the foot. Because employers expect more. Society needs more. And if we're going to raise fully-formed, educated citizens who can make informed political, economic, and personal choices it will take a lot more than what any of our schools are currently offering.