Take Me Out [of the VAM] Game

Growing up in Canarsie, Brooklyn as a die-hard Mets fan, I collected baseball cards.  Piles and boxes of wonderful statistics, much to my mother’s chagrin, littered my bedroom.  These cards served as my baseball library as each one painted a picture and told a story of a player’s season.  I would consistently pore over the numbers, studying the effective from the ineffective players; gleaning who had a “career” year and others that produced a down year.  And, in an era before satellite and cable television, I was informed about players I rarely saw hit or pitch, just by studying these easy to understand statistics.  Admittedly, a complete player picture is not derived from these statistics.  Players could argue that their use of strategies, their ability to work on deficiencies, and the many nuances of the game are not represented in these numbers.  Overall, I would agree since watching players perform gives me a more complete picture of each players’ abilities, but the statistics on his card present a portrait or snapshot of his career.

Now, I imagine myself on a baseball card, or in this case, I would call it my teacher card.  On the front, it would show me in action, teaching a lesson, conferencing with a student on their writing, or even sharing a laugh with an eighth grader.  The back of my card would have my statistic, my Value-Added Model number, and I wonder what story it would tell?  Well, last season, my VAM score was a 27.10.  This number is not easy to understand, I can’t calculate it on my own, nor does it illuminate my strengths or weaknesses.  Additionally, it takes approximately four months for computers, mathematicians, calculators, abacuses, and quite possibly the counting of fingers and toes to deliver this Value-Added Model score to educators.

The Value-Added Model number, according to my school district, is a statistical measure indicating the contribution a teacher has toward a student’s learning.  Therefore, I contributed 27.10 to my students’ learning.  This number makes up 40% of my overall evaluation score with the remaining 60% from my written observation evaluations.  The only thing I truly know about my VAM score is that the higher it is, the better my students performed on standardized assessments.  If baseball cards only showed one statistic titled “player effect on the team” giving players a number based on their performances that was otherwise virtually impossible to calculate, my mother would have had fewer piles of statistics to gripe about.

I often wonder if we are attempting to mathematically quantify effective teaching when that is impossible.  There are just too many variables, facets, nuances, and conditions of teaching to accurately paint a picture of a teacher’s school year using a number.  Educators need to be measured by their implementation of best classroom practices, through observations, as evidence of working in their students’ best interests.  As a classroom teacher, when I undergo an evaluation observation, I get to dialogue with my evaluator and reflect on what I would improve while discussing the strengths and weaknesses of the lesson itself.  The process is a learning experience.  My Value-Added Model score is not.

Rob Kriete, Teacher Leader & Baseball Star

I’m confident the sharing of my Value-Added Model score does not help anyone understand the effectiveness of my classroom teaching; yet, if given the opportunity to observe my classroom, that young adults are thinking, learning and collaborating would be evident.  Transparent classrooms will improve teaching, not a “magical” number.

When (or if) the day arrives that my beloved Metropolitans, in their orange and blue, win the World Series, I will be able to reference their baseball cards and immediately understand how they effectively hit and pitched their way there.  When my students achieve success beyond my classroom, my VAM score offers no clue or insight into how I helped them.

Related categories: ,
  • Jarauxwashington

    Great post

    Hey Rob I love this post and the use of the baseball card example. I wonder if we did have stats on our teacher card what would they be? Perhaps ability to differentiate , classroom management reports. I too am an advocate of transparent classroom . The challenge is how can we encourage transparency and at the same time protect our teachers from the abuse of their non teacher partners in education?

    Perhaps if our VAM scores were calculated more straight forward or connected to VALID tests and used to support growth our “teacher cards” could be more useful in created winning teams.

     

    jaraux

  • Rob Kriete

    Transparency is key.  If we

    Transparency is key.  If we create transparent classrooms, observations can serve as formative evaluations, wherein teachers are continuously improving.  Currently, most evaluations are summatives that serve as a type of “final score.”

  • JulieHiltz

    My VAM is not my work.

    Like many teachers that are not part of the core curriculum subjects my VAM is a school-wide score that is intended to reflect on my contribution to all students. For my position as a Media Speciaslist my VAM is reading.

    Sounds good on paper, but I bristle at the fact that I have LITTLE TO NO direct impact on reading scores: I can’t make students come to the library, I don’t hold guided reading groups, I can’t implement/monitor RTI for every grade level. I am at the mercy of my colleagues. 

    What about the other subjects? My evaluation says I am required to be knowledgeable of and support most curriculum areas. Why aren’t they reflected on my VAM, too? I love to joke with colleagues (that understand my sense of humor) “I’d love to pull that video for you, but I don’t support science.” 

    VAM- no feedback, no transparency in the formulation, not representative of what I teach, NO GOOD.

  • simonmeshbesher

    Great post!!  The analogy is

    Great post!!  The analogy is accurate and sports can really help us to understand how ridculous things like VAM are…Absolutely right along with the fact that I have yet spoken to any teacher that I work with that can be sure what is exactly in the calculation and better yet if it is correctly being applied.  When you hit a Home Run that is easily visible in your stats….I agree with you the transparency of the process is necessary with 40% riding on your evaluation.  After the fcat changes writing assessment measurements and the test changes from previous versions how do we know or trust our VAM number anyway.  Students wouldn’t email you years later to tell you that you had a great VAM year that impacted their future decisions, but they would if you impacted them with your style and influence….which counts more? 

  • DavidMiddleton

    Reliable score?

    After our discussion the other day, I realized that you and I get pretty good VAM scores, but our EET evaluation scores are just a little better than OK. How reliable is the EET if teachers who have the better VAM scores are getting only OK evaluation scores? I would think the two would go hand in hand, but that doesn’t seem to be the case.

    Two years ago (the most recent data that compares my scores to others), I ranked in the middle of the pack (about 1,200 teachers I think) on my evaluation scores, but my VAM score was beaten by only 120 teachers. If the VAM measures student growth, shouldn’t that be observable during my lessons? Or maybe I’m not good at the dog and pony show of the observation. Ironically, when my observation scores are lower, my VAM is much higher and vice versa. My lowest observation score year was the year I received my highest VAM.

    While the VAM is mysterious in its calculations, I place more emphasis on that score than the observations. If it really measures how much my students improve during the school year, that is the score I want to be higher. The truth is, I got into teaching to help students become smarter and better readers and writers, not to impress my administrators and random peers.

    It will also be interesting to see if my VAM improves or decreases this year. I switched from a Title I school to a higher SES school. I will be the guineae pig to see if student population has anything to do with the VAM. Same teacher + different population =?.

    • Rob Kriete

      aha! Something VAM can tell us?

      David-

      Very interesting insight.  If our evaluation does not score high on a rubric of “effective teaching,” but our students do well on their assessments, clearly the disconnect is found in the observation analysis.  And, one could assume that there are teachers out there that have inverted scores…high classroom evaluation scores and low(er) achieving students on standardized tests.  How can the two align?

  • JillHorner

    Poor Mets!

    …and poor reading coaches, media specialists, etc. After leaving the classroom for a reading coach position, I quickly went from “highly effective” (VAM=26.76) to “needs improvement” (VAM=19.76). As far as the illustrious VAM score goes, becoming a reading coach in a Title 1 school was not such a good idea. However, since I was in charge of 800 students instead of my own 130+, was all my work for naught? Did I help make any teachers stronger? Should my VAM score be based on increases of teacher VAM scores since I am to be COACHING them to become more effective?  I do not know, but seeing that Needs Improvement sure made me think I may be better off going back to my own, unimportant classroom to get back to the VAM game!

  • Rob Kriete

    Vital work

    Jill-

    Clearly, those that designed the VAM usages haven’t considered every possible situation, and unfortunately, your becoming a reading coach is one of these.  It seems, ultimately, that there has to e a more effective way to measure your work in a school year.  Once again, trying to measure any coaches’ effectiveness has to entail more than measuring every students standardized test scores. 

    What do you think is the best way to measure academic coaches effectiveness?

  • DavidCohen

    Batter up!

    Hi Rob – 

    I found baseball to be an interesting way to discuss VAM – had a lot of fun writing this one:

    http://accomplishedcaliforniateachers.wordpress.com/2010/08/21/batter-up/

    As for your interest in what happens to your test scores this year, in general, I think research models suggest you’re likely to see lower scores. But research also suggests that your scores are really not reliably based on your work anyways – so whether they go up or down or don’t change at all, the only guarantee I can offer is that trying to intrepret the change (or lack of change) will be an empty exercise in speculation.