Occasionally, the New York Times’ headline writers are shocked — usually when something good happens where they don’t expect it. Hence we see, “Surprise: Florida and Texas Excel in Math and Reading Scores.”
The subject of the Times’ surprise is a new study by Matthew Chingos of the Urban Institute which looks at the fourth and eighth grade math and reading results from the National Assessment of Educational Progress (NAEP), a national standardized test, then adjusts the scores for demographic factors. Considering demographic factors allows for a comparison of public education effectiveness between the states.
Placing the role of demographics in testing results into context, the study notes that, “Existing research indicates that levels of education service delivery that are further removed from students tend to have weaker associations with student achievement,” with “63 percent of the variance in achievement (in math scores in North Carolina) at the student level, 5 percent at the teacher level, 3 percent at the school level, and 2 percent at the district level.” In other words, 63 percent of test scores are due to the children themselves, their intelligence, and their family background.
In making demographic adjustments, the Urban Institute report considered 11 factors at the test-taking student level: gender; race and ethnicity; eligibility for free or reduced-price lunch (poverty); limited English proficiency; special education; age; whether the student was given an accommodation on the NAEP exam (e.g., extra time or a separate room); whether the student has various amenities in their home (e.g., computer, Internet, own room, dishwasher, and clothes dryer); the number of books in the home; the language spoken at home; the family structure (e.g., two-parent, single-parent, foster).
Once differences in the population of test takers are taken into account, then it is possible to measure educational effectiveness. As Chingos noted, accounting for demographics “means that states are judged by how well their students do relative to students with similar characteristics across the country.” Even small differences in teacher, school, district or state effectiveness can translate into significant results overall, according to Chingos, as the system subtly affects larger numbers of students moving out from the classroom to the school, to the district and then the state. The author cited a separate study which found that good school districts — ones that are one standard deviation better than the rest in the state — can add the equivalent of two to three months of schooling to their students.
So, comparing the most populous five states, California, Texas, Florida, New York and Illinois, where 38 percent of America’s 50 million public school students call home, shows Texas students leading the way with about 5.6 extra months of learning in math and reading vs. their average counterparts across the nation. At the bottom, California students lag 6.3 months behind their peers in learning.
| Learning (Months) 2013 NAEP | Avg. Pay of Public School Teachers 2012-13 | Public School Teacher Pay Adjusted for State Cost of Living | Students in ADA per Teacher in Public K-12, 2013-14 | Avg. SAT Scores | SAT Partici-pation Rate | Public high school 4-year adjusted cohort graduation rate 2012-13 | |
| Texas | 5.6 | $48,819 | $53,179 | 14.3 | 1432 | 62% | 88% |
| Florida | 3.6 | $46,598 | $47,452 | 15.6 | 1448 | 72% | 76% |
| New York | 0.9 | $75,279 | $55,189 | 16.0 | 1468 | 76% | 77% |
| Illinois | -0.4 | $59,113 | $61,833 | 15.7 | 1802 | 5% | 83% |
| California | -6.3 | $69,435 | $54,203 | 21.2 | 1504 | 60% | 80% |
The real puzzle is, why the wide discrepancy in results between Texas and Florida at the high end and California at the low end? (Texas and Florida and third and fourth overall among the 50 states, California is 46th.)
After adjusting for California’s high cost of living, California teachers’ pay falls in the middle of five big states, but, student-to-teacher ratios are among the nation’s highest in California. Texas and Florida, on the other hand, have the lowest adjusted pay for teachers — though not as low as critics contend when considering the cost of living.
SAT scores, another avenue of criticism levied against Texas’ educational achievement are a poor proxy for success, as SAT scores are heavily influenced by both demographics and by the participation rate for the test. As more students take the test, the scores fall towards the mean. In states such as Illinois, where only 5 percent of students take the SAT, the ACT is the favored college entry test. This is why the Urban Institute’s study of NAEP test scores is a valuable tool to compare states while the SAT isn’t. The figure below further illustrates a challenge with using the SAT as an indication of state public education success.
With the Urban Institute’s study providing an opportunity to compare states’ educational effectiveness, report author Chingos observes that “Politicians and advocates from across the political spectrum,” have cited NAEP scores to make points about education policies, with former, “Florida governor Jeb Bush … trumpet(ing) his state’s NAEP scores, especially the average reading score of fourth-grade Hispanic students in Florida—ranked highest in 2013,” and “the American Legislative Exchange Council … point(ing) to increases in NAEP scores in states that have adopted school choice policies.”
That school choice increases test scores is an especially important lesson for the five biggest state as their school-age children are now minority-majority and becoming increasingly so, a factor often accompanied by the twin challenges of English language proficiency and poverty.
Further complicating the effort to improve educational outcomes in the nation’s most-populous state, teachers unions in California have enormous political clout, so much so, that one of the only paths of significant reform has been in the courts with lawsuits seeking to address the negative effect of teacher tenure on student performance and poor educational quality for low income students.
Texas has also seen frequent resort to the legal system to affect public education, though, unlike in California, the seven suits in 31 years in Texas focused only on how much money to spend in public education and where to spend it.
One thing that should be clear from the Urban Institute study is that money itself in public education is little guarantee of success. If America’s large, diverse states are to continue to improve educational outcomes for their increasingly diverse population, they must encourage the kind of innovation that is only possible with a thriving school choice market — one in which the parents and students pick the best schools for their specific needs.