DOE Sidestepping UFT
Scores May Decide Teachers' Tenure

By MEREDITH KOLODNER

Department of Education officials may seek to use data tracking thousands of students' tests scores for Teacher tenure decisions in the coming years without negotiating the shift with the United Federation of Teachers.

'FUNDAMENTALLY FLAWED': United Federation of Teachers officials are incensed over the city's announcement that it may use student test data to evaluate Teachers and make tenure decisions next year. UFT Vice President Leo Casey argued that the pilot program at 240 city schools would not lead to improved education. 'The way they are analyzing the data could not possibly produce a meaningful review of Teachers' contributions,' he said.

There are currently 240 elementary and middle schools in the privately-funded pilot program, which measures the improvement in students' standardized test scores and rates their Teachers compared to others citywide and those with similar classroom demographics. Most Teachers were unaware that they were being monitored.

Union Will Fight It

United Federation of Teachers officials were told about the data collection last summer and UFT President Randi Weingarten said the union would fight any attempt to use the information in formal evaluations.

"It's educationally unsound, it's legally impermissible and it's statistically flawed," she said.

The pilot program, first reported in The New York Times Jan. 21, will not affect tenure decisions and ratings this year, according to DOE officials. But the department will decide this summer if it will use the data alongside other indicators to evaluate Teachers next year.

Deputy Chancellor Christopher Cerf said that the impact of the pilot program had been blown out of proportion. "The whole idea of this program is to explore in a disciplined and scientific way whether it is possible to research and determine how Teachers affect student learning," he said.

He argued that if used at all to evaluate Teachers in the future, it would not be done in a "formulaic" way. "This is one thing to possibly put in the mix," Mr. Cerf said. "It is a work in progress."

RANDI WEINGARTEN: Will fight tenure link.

But a DOE spokeswoman confirmed that officials believe they have the authority to integrate the data into Teacher evaluations.

"We don't think we would have to negotiate a new contract to do so," said DOE spokeswoman Melody Meyer.

Principals were permitted to tell their Teachers about the program, which some did, but many more did not. UFT officials said they tried to find out which Teachers were being affected but were denied the information because participating Principals were given a guarantee of confidentiality.

'We Were Shocked'

Some Teachers were frustrated to learn about the program in the newspaper and were left wondering whether they were being evaluated. "Teachers at my school were shocked," said Lisa North, a Brooklyn elementary school Teacher. "We expected the DOE to do something like this and not tell us, but we expected that at least our union would tell us it was happening."

UFT officials say that when Ms. Weingarten was informed last summer, she objected to the program and was promised that the data would not be used for evaluations. The UFT director of research was allowed to sit on the DOE panel that was charged with figuring out how to craft the new system.

The pilot program was slated to move forward regardless of UFT opposition, and so the union made the decision to participate.

"I thought it was a good idea to have somebody sit on the task force," said Ms. Weingarten. "I wanted to know what they were doing."

The final incarnation of the data-collection system convinced UFT officials that, in addition to the inappropriateness of using the information for evaluations, the program itself was faulty.

"By sending Principals this two-page template on Teachers without giving the union any notice," said Ms. Weingarten, "it went from a data-collection exercise to a spying exercise."

A Split Evaluation

The UFT objects to the system due to both its high-stakes use of the standardized test results and the way the data is being collected. Because the tests take place in the middle of the school year, improvement in a student's test scores must include the work of two Teachers; for example, the fourth and fifth grade Teachers of a particular student.

The DOE's statisticians said they solved this problem by splitting the gains made between the two Teachers. Since the math test takes place in March, 60 percent of the gains would be credited to the 5th grade Teacher and 40 percent to the previous Teacher when the student was in fourth grade to reflect the amount of time the student spent with each Teacher. Each student has a "predicted gain," based on a series of factors. The student's actual gain is measured against this forecast, and the Teacher is rated accordingly.

Too Many Factors

Only 120 of the participating Principals will receive the individual Teacher reports that are generated. The other 120 Principals will be used as a control group and will be asked to make subjective evaluations so officials can see if the two systems produce significantly different results.

UFT officials say the system has too many factors at play and point out that differences such as a student's use of a tutor or access to after-school programs could have a significant impact on the gains, making comparisons between students' improvements unreliable.

'Defeats Its Purpose'

"There is a fundamental flaw at the heart of this," said UFT Vice President Leo Casey. "They designed the system to identify individual Teacher contributions, but there's no way of distinguishing between two Teachers with so many other major contributions ... it defeats the whole purpose of what they said they were doing."

DOE officials say they have taken into account as many factors as possible that are outside the control of a Teacher, weighting differences such as summer school, special education, Teacher experience, class size, English Language Learners and poverty.

They also say they have specifically instructed the Principals involved not to use the data for Teacher ratings and tenure decisions this year, but UFT officials are not convinced that the direction from the DOE will be effective once the Principals have the data. "It's kind of like when a jury hears testimony and then the judge says don't consider it," said Mr. Casey.

Cerf's Up At Times

UFT representatives met with DOE officials in late November as the individual Teacher reports were being prepared to ask for the names of the Teachers who were included in the program. The DOE objected because it had promised confidentiality to the Principals who volunteered to participate. UFT officials said that they won agreement to keep the results private, not to be shared with parents or colleagues, and were still in discussions with the DOE when Mr. Cerf spoke publicly about the program and the story appeared in the Times.

UFT officials say they also object to the program because they believe it will only increase the pressure to focus on test prep in English and math, to the exclusion of other subjects. They note that only a quarter of Teachers could be evaluated using the system, since it excludes pre-k to third grade and high school Teachers and all science and social studies Teachers. They argue that a school-wide evaluation system has none of those drawbacks and could more easily include the contributions of all Teachers, instead of parceling it up to create individual scores.

DOE: A Positive Tool

DOE officials argued that the data could be used to help Principals improve their schools by, for example, noting which Teachers were having success with English Language Learners and then having those Teachers assist others.

The UFT, however, was not convinced of the educational use of the data. Mr. Casey said that if there was a joint study that both the UFT and the DOE agreed was producing genuine data, there might be a way to use the results to improve instruction. "This stuff is so seriously flawed that we can't conceive of a productive use that it can be put to," he said.