Swift & Change Able

Education Sector has just published a paper that I authored on the Tennessee Growth Model being used to measure school and district AYP.

Great pains were made to make the paper as fair as possible. All growth models have strengths relative to the status and safe harbor models laid out in the ESEA statute, and Tennessee’s model is no exception. But, as the paper points out, the Tennessee model (variations of which are also being employed in Ohio and Pennsylvania and are under debate in Texas) also has some serious shortcomings.

The Tennessee model is highly technical, using multivariate, inferential statistics. And therein lies a good deal of the problem. More statistical sophistication means less transparency. And it’s not clear to me that all that sophistication actually buys you much relative to other, simpler methods of measuring growth.

Some of the methodology and most of the data are proprietary, meaning they are privately owned, i.e., no public access. This all makes it very difficult for even Ph.D. and J.D.-level policy experts to get a handle on what is going on (which I found as a peer-reviewer last year), let alone teachers, parents, and the general public. 

Tennessee officials communicating with me privately acknowledge the points raised in the paper. They also point out, in all fairness, that they are in the midst of raising Tennessee standards, in response to the second key point in the paper: slow growth toward a low standard, especially for grade school children who risk falling too far behind and never catching up, may be a risky and inefficient accountability strategy.

The third key point is that the Tennessee model has an inherent paradox: while the Tennessee model implies a 3-year trajectory to proficiency, even if a student meets his or her interim goals each year, they likely will not reach proficiency in three years. This is a well-known statistical phenomenon called "Zeno’s Paradox."

What drives the paradox is that Tennessee recalculates a a child’s "projected" 3-year path to proficiency each year.

Think of a child’s academic progress as covering the "distance" from where they are academically this year to "proficiency" three years from now. Here’s a simple example of the problem:

Let’s say a frog very much wants to get to a lily pad that is 100 meters away. He knows he can’t make it all the way, but judges he can make it a third of the way (he’s a championship frog). So on his first try, he jumps 33 1/3rd meters. He decides to jump a third of the way to his goal from where he stands each time, until he gets there.

When does he get there? On first glance, most of us would say in three jumps. In fact, in strict mathematical terms, he never gets there. After three tries, he is actually only 19/27ths (71 percent) of the way there. The Table below shows the calculation.