So, it’s gotten to the point where I wake up the morning after writing a column with an epiphany about the perfect way to explain a concept.
The bad news for me is that I can’t sleep in. The good news – for you – is that I have a blog where I can share these thoughts.
In today’s column, I take the side of a couple of researchers who say that Gov. Andrew Cuomo’s performance grant won’t actually identify and reward truly successful districts. I explain that is in part because his team chose to measure success using growth in indexes, as opposed to using raw scores.
Why do we care whether this grant is any good? Well, because this is $75 million that districts desperately need in a bad economy.
So, here’s a couple more explanations on why using the governor’s formulas muddies some successes and could arbitrarily reward lesser achievement.
The grant uses indexes to calculate success. Indexes are based on how many students fall within a certain group – proficient or advanced, etc. But for simplicity’s sake let’s call those groups “A, B, C and D” students.
The governor’s grant gives you points if you have more students in 2010 who got As than the 2009 students did.
So, let’s drill down into how that plays out with raw scores.
Say your fourth graders in 2009 were all getting 81s on the test. In 2010, your fourth graders all got 86s. Hurray! Progress … success, right?
Well, actually with the index, you had all B students in 2009, and all B students in 2011 so … No points for you.
In another school district, the fourth grade also has a five-point gain. Luckily, the five points brought them from having 88s to 93s … from Bs to As! In this scenario, they earn points toward the grant.
So five points on one part of the scale gets you credit, while five points elsewhere, does not.
To compound the randomness, the grant weights some progress more than
others. That means, if you started out with a whole lot of C students in 2009, if your next class all earned Bs, you would only get more credit than districts that shifted from all B students to all A students.
The other failing of this grant – and of a lot of education policy these days – is that we’re not looking at student performance over time. We’re comparing last year’s fourth graders to this year’s fourth graders. You get points if the 2010 fourth-graders got more As than the 2009 fourth graders.
What would truly measure success is if 2009 fourth graders, who were B students, got As in 2010 – as fifth graders. Or even better, if your school moved a D-average student up to an A-average student. Researchers call this a student growth model for good reasons.
And, in case it’s not obvious already, the state could do away with all of the complicated formulas by just comparing the raw student scores.
Governor’s office spokesman Matt Wing offered me this quote for my column (it got cut for space)
“Districts that don’t make the cut this year will now have clear benchmarks for improvement so they can apply in future years as we continue to dispense the full $250 million created by the awards program.”
The only thing clear about this measure – or the benchmarks – is the governor’s spin on it.
In case you needed further evidence, I submit to you the extraordinarily “clear” formula for calculating gain. My favorite part about this formula is that – at the end of all this calculating – you slap on an extra 10 points (for no apparent reason).
D. Calculation of All Students Gain/Loss Point(s)
In 2010-11, the district gain/loss point(s) for all students are calculated by:
1) For each measure (ELA, math, science, and graduation rate), subtracting the 2009-10 PI (or 2005 cohort grad rate) from the 2010-11 PI (or 2006 cohort grad rate) to get the PI gain or loss (e.g., 185 (2011 PI) – 180 (2010 PI) = 5);
2) Multiplying the result by the appropriate weighting percentage (e.g., 5 Í 30% = 1.5 for ELA);
3) Summing the resulting weighted gains and/or losses for each measure (e.g., 1.5 (ELA) + 0.6 (Math) + 1.0 (Science) + 0.9 (Grad Rate) = 4.0);
4) Multiplying the resulting sum by 8 (e.g., 4.0 Í 8 = 32.0); and
5) Adding 10 points to this result for a maximum of 50 points (e.g., 32.0 + 10 = 42 point district gain).
Districts with resulting calculated points for all students that are less than 0 are awarded 0 points for the all students part of the Gain Index calculation.
ELA = 185 (2011PI) – 180 (2010PI) = 5; 5 x 30% (weight) = 1.5 (gain/loss)
Math = 175 (2011PI) – 173 (2010PI) = 2; 2 x 30% (weight) = 0.6 (gain/loss)
Science = 193 (2011PI) – 183 (2010PI) = 10; 10 x 10% (weight) = 1.0 (gain/loss)
Grad Rate = 75 (2011PI) – 72 (2010PI) = 3; 3 x 30% (weight) = 0.9 (gain/loss)
Total District Gain/Loss = 4.0
Sample Performance Points for All Students = (4.0 x 8 ) + 10 = 42