Skip to main content

Join the Community

or Close

Search

Policymakers, do your homework on evaluation reform

Last week, Stephen Sawchuk of Education Week reported on the “race” to implement the new teacher evaluation system in New York City. The system has been marked by “years of bitter disagreement” among political and union leaders over the purpose of the new policy as well as the criteria for determining the quality of teaching.

The debate over using test scores (and a value-added or student-growth statistical model) to determine teacher effectiveness continues to rage, and implementation problems abound.

You should check out the differing critiques of Diane Ravitch and Rick Hess on using test scores alone as a basis for teacher evaluation, as well as the case made by education economist Dan Goldhaber, insisting that flawed systems of measuring teachers on the basis of student results are better than the status quo.

Ravitch, Hess, and Goldhaber make important points for policymakers, the public, and practitioners to consider. But the real story—and the potential policy disaster of teacher evaluation reform in New York City—is not in the compromise over test scores but in the implementation timeline.

The city has three months to get ready for a uniform evaluation system for 75,000 teachers.

That’s right: NYC principals and teachers have three months to implement the complex system of teacher evaluation, using multiple measures of student learning. Large numbers of administrators will need to be trained to use more detailed classroom observation tools, as well as to interpret a diverse set of data points in valid and reliable ways.

Even a district official, David Weiner (deputy chancellor for talent, innovation, and development), admitted that the challenge was enormous: "It's a large cultural change for a lot of schools, and they'll be asked to do things they've never been asked to do before.”

Looking back to move forward
I wonder if school reformers of today have studied the rich history of failed school reforms of the past. Looking back at research I drew on when teaching graduate students—more than two decades ago—I came up with a few lessons worth remembering:

  • Policy success is most directly impacted by two broad factors—local capacity and will (McLaughlin, 1987);
  • Schools should not adopt a single design, but co-construct their models to fit their environments (Olson, 1999);
  • People’s participation in various communities and relationships is essential to implementation (McLaughlin and Talbert, 2001); and
  • It takes three to eight years to successfully implement comprehensive school reform (Rand, 2004).

In particular, I would encourage readers to return to the Rand Change Agent Studies of the late 1970s, in which Paul Berman and colleagues shared what did not work for successful implementation:

  • Reliance on outside consultants;
  • Prepackaged management approaches; and
  • One-shot, pre-implementation training.

Berman and colleagues also shared what did work:

  • Concrete, teacher-specific, and extended training;
  • Teacher participation in project decisions; and
  • Local development of project materials.

If you only read one study
But perhaps more than anything else, I encourage today’s reformers and union leaders to read Arthur E. Wise’s 1984 study of effective teacher evaluation practices, which alerted policymakers and practitioners to four areas of focus:

  • Organizational commitment: Investing in oversight to ensure fidelity;
  • Evaluator competence: Including teachers as peer reviewers;
  • Teacher-administrator collaboration: Focusing on joint ownership; and
  • Strategic compatibility: Ensuring personalization over standardization of the reviews.

Dr. Wise’s study was released 30 years ago. He found that most school districts, like those today, are uncertain about which instrumentation to use, how frequently teachers should be assessed, and how data informs other elements of the system. Successful teacher evaluation systems begin with district leadership allowing time for implementation:

“With regard to commitment, all four case study districts recognize that the key obstacle to successful evaluation is time—or, more precisely the lack of it—for observing, conferring with, and especially assisting teachers who most need intensive help. (Successful) districts create time for evaluation.”

Recently, the NYC union fought for and won “teacher participation” in the evaluation process, and the district has doubled the amount of professional-development funds they send to schools that could be used to support local implementation. But the timeline is ungracious at best. There appears to be a gross neglect by the district of the need to take the time to scale their system of teacher evaluation.

Charlotte Danielson, whose teaching-observation framework was bought by the district, was reported as saying, "There aren't enough consultants in the world, or enough money in the world to pay the consultants, to do that training face to face in three months.”

But as the change agent studies of the 1970s and Wise’s evaluation study of the 1980s remind us, the implementation of complex school reforms requires less traditional training and more capacity-building at the school level so that those whose work is to be judged can use the evidence to improve their practice.

It seems that this has been a very hard lesson for today’s school reformers to learn. I wonder why?

2 Comments

Richard Capozzi commented on June 19, 2013 at 2:36pm:

NYC Teacher Evaluation Model

Excellent! Thank you.  Teacher voice is notably absent from the teacher evaluation model being hastily implemented.  (I wonder.  Do Duncan's Tuesday 6/19/13 waivers affect the timeline in NYC?)  First, peer review is glaringly absent; second, teachers have no input into the evaluation of the evaluators, namely principals and other administrators.  This makes for poor dialogue partners.

Brad Clark commented on June 20, 2013 at 11:32pm:

"he is very thorough" -maude

Barnett,

I love the longitudinal perspective you bring to the table.  Teacher evaluations are not new, so it is terribly valid to reassess the previous research so that former mistakes are not repeated.  In light of previous attempts to evaluate educators, don't we have the opportunty to truly institute authentic teacher eval systems that are well suited to the current state of education, with an eye clearly focused towards the necessary reforms that our consumers/students demand and require?

KY is adopting the Danielson Framework:  

http://education.ky.gov/teachers/hieffteach/pages/designing-pges.aspx

Our Dept of Ed is now one school year away from full implementation (The pilot was a three year process).  Here is an idea of what the field test attempted to do:

http://education.ky.gov/teachers/HiEffTeach/Pages/PGES-Field-Test-Districts-.aspx

I am interested in 1) Your impressions of the KY Teacher Eval System (called PGES, the Professional Growth and Effectiveness System) in light of your understanding of the previous research; and 2) Does the thoroughness of the Danielson Framework address some of the fears/issues present in NY and other states in similar situations?

Thanks.  Enjoyed your post.

Join the Conversation!

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Subscribe to Blogs by Barnett Berry

Stay Informed

Sign up to receive the latest news and events through email!

Sign Up