Affect Students, Schools
Educators: Flaws In Test Scoring

By MEREDITH KOLODNER

Some Principals and Teachers are concerned that a tiny scoring distinction between students designated Level 3 and Level 4 on state English exams is shortchanging them and may harm their schools' ability to show progress on their annual report cards.

RANDI WEINGARTEN: Creates wrong focus.

Third- and fifth-grade students who miss more than one multiple-choice question out of 31 are graded as a Level 3, meaning that they are performing at grade level. The scoring process means that a child who gets about 97 percent of the questions correct is considered performing above grade level but a child who gets about 94 percent is not.

Slippage Questionable

The state considers all Level 3 and 4 children to be achieving satisfactorily, but the city's new report card school scoring system looks at the progress made each year by children when it rates schools. That means a child who last year answered 97 percent of the questions right and this year answered 93 percent correct is not considered to be making progress.

"I'm not someone who's anti-testing - I think they have some value," said Elizabeth Phillips, the Principal of P.S. 321 in Brooklyn. "But these scores are being used as if it really means something if a child got a 3 instead of a 4 on a test."

Ms. Phillips sees the issue as one primarily for the state to correct since it sets the scoring process. She says that she has many children who are reading at two or three levels above their grade who are scoring as Level 3s. She believes that the letters that parents receive from the state should include how many questions their children got right and wrong. But State Education Department spokesman Thomas Dunn argues that a graph on the parent report letters should make it clear whether a child has scored relatively higher or lower within each level. "For state accountability purposes," said Mr. Dunn, "what matters for a school are Levels 3 and 4 combined. We're not pointing the finger at anybody if they have [fewer] 4s. There might be other measurements that do that, but that is not our purpose."

City Heavily Reliant

The city will rely heavily on the tests for its new report card system in which every school will be given a letter grade based on its progress. About 85 percent of elementary and middle schools' city DOE report cards are based on their state English (ELA) and math exam scores.

"The implication for the report card is that the test scores are a very imperfect system," Ms. Philips said, "and to have so much at stake on such an imperfect system can be a problem."

Under the No Child Left Behind Law, school districts must show yearly progress. The city report cards will grade schools based on their test progress compared to other schools with similar demographics. Principals and Assistant Principals whose schools are assessed as top performers will receive bonuses, with the highest rated Principals earning $25,000.

Schools graded poorly and showing insufficient progress risk being reorganized or closed down.

A DOE spokesman noted that the report card scoring system will take into account the gradations within levels, so that student progress moving from a low Level 3 to a high Level 3 will be incorporated into the overall assessment.

UFT: Time Misspent

Council of School Supervisors and Administrators officials declined to comment on the issue, citing the complications involved in scoring and assessing tests.

United Federation of Teachers Randi Weingarten asserted that the concern about the test scoring practices reflected the DOE's overemphasis on high-stakes testing. "When testing is so central, everybody's going to look at the nuance of the tests," she said. "People are going to be really proficient about how the tests are marked, and they're going to spend a whole bunch of time focused on that as opposed to focusing on teaching and learning." Testing experts said the scoring system was bound to cause some confusion. "This is one of the unintended consequences of moving to a performance-based scoring system, relative to standards," said Robert Tobias, who was the head of testing under former Mayor Giuliani and is currently the director of the Center on Teaching and Learning at New York University. "The relationship is not quite straightforward. It's not like if you get 90 percent correct, it converts into an above-grade-level score."

Hazy Distinctions

He added that the distinctions were often unclear even to educators. "It's not really transparent to the audience," he explained, "and I include in that audience not just parents but Principals and Teachers."

Other education advocates took a stronger stand against the process, noting that the younger the child, the more likely she is to miss a question, not due to lack of knowledge, but due to lack of concentration.

"The notion that you have to be just about perfect to be above grade level suggests poor test design," said Robert Schaeffer, the public education director at the National Center for Fair and Open Testing. "Grading schools on flawed tests inevitably produces bad education policy."