Opinion and News Analysis
Opinion: The accountability plateau
E. Finn, Jr. and Michael
“Consequential accountability,” à la No Child Left Behind
and the high-stakes state testing systems that preceded it, corresponded with a
significant one-time boost in student achievement, particularly in primary and
middle school math. Like the meteor that led to the decline of the dinosaurs
and the rise of the mammals, results-based accountability appears to have shocked
the education system. But its effect seems to be fading now, as earlier gains
are maintained but not built upon. If we are to get another big jump in
academic achievement, we’re going to need another shock to the system—another
meteor from somewhere beyond our familiar solar system.
So argues Mark Schneider, a scholar, analyst, and friend
whom we once affectionately (and appropriately) named “Stat stud.” Schneider, a
political scientist, served as commissioner of the National Center for
Education Statistics from 2005 to 2008, and is now affiliated with the American
Institutes for Research and the American Enterprise Institute. In a
Fordham-commissioned analysis released
today, he digs into twenty years of trends on the National Assessment of
Educational Progress (NAEP), aka the “Nation’s Report Card.”
We originally asked Schneider to investigate the achievement
record of the great state of Texas. At the time—it feels like just yesterday—Rick
Perry was riding high in the polls, making an issue of education, and taking
flak from Secretary Arne Duncan for running an inadequate school system. We
wondered: Was Duncan right to feel “very,
very badly” for the children of Texas? Had the state’s schools—once
darlings of the standards movement and prototypes for NCLB—really slipped into
decline since Perry took office? What do the NAEP data really show?
Schneider agreed to take on the project but quickly
concluded that there’s a larger and more interesting story to tell than simply
the saga of Texas. It was true, he noted, that Texas’s achievement slowed during
the Perry years, particularly as compared to the rest of the country. But
rather than pin that development on the governor, Schneider saw a more likely
explanation: As an early adopter of standards, testing, and accountability,
Texas got a head start on the big achievement gains that these initiatives
brought in many place—and realized most of these gains in the 1990s when George
W. Bush was governor.
Take fourth-grade math, for example:
In 1992, fourth graders in Texas were performing
on par with their peers nationwide on NAEP’s math assessment. In the 1993-94
school year, however, Texas introduced its strict “consequential accountability”
system, and its effects can be seen in NAEP’s next iteration in 1996: Texas
fourth graders scored about five points above students nationwide. That gap
persisted through the 2000 NAEP assessment as well, and not just for Texas
students as a whole. Rather, Texas’s black, Hispanic, and lowest-performing
fourth graders—those groups that served as particular focal points of NCLB and
accountability systems more generally—all scored at least thirteen points above
their national peers in 2000. That translates to roughly a year’s worth of
schooling, or more.
Indeed, the Lone Star State made Texas-sized gains from the
early- to mid-1990s, as its accountability system got traction. But as other
states followed suit in the late 90s, and as all remaining states introduced
NCLB-style accountability systems after 2001, they too jumped onto the
achievement fast-track, leading to sizable national gains. Between 2000 and
2003 alone, fourth-grade performance in math nationwide jumped a full nine
points on NAEP’s scale.
But much as Texas led the nation in making positive strides
throughout the 90s, so did it lead the nation in petering out thereafter.
Texas’s fourth graders peaked on the NAEP math exam in 2005; their average
score has wavered between scale scores of 240 and 242 in the last four
iterations of NAEP. Students nationwide caught up to Texas by 2009. In that year,
and in the 2011 assessment that followed, there was no significant difference
in the scores posted by students in Texas and those nationwide. That’s true of
sub-groups, too. While black, Hispanic, and low-performing students in Texas
still led their peers nationwide in math by eight, six, and five points in
2011, respectively, their progress and the progress of those groups nationally
has stalled over the last few assessment cycles. The same general patterns can
also be traced for eighth-grade math students, though their progress built upon
fourth-grade gains both in Texas and nationally and thus occurred in later
years, without clear signs of stalling as yet. (Overall reading scores,
however, have changed little over the last two decades, both nationally and in
Texas, though some subgroups have shown significant gains.)
So while Texas’s progress has cooled in recent years, the same
pattern can be observed in the country as a whole—only offset by about half a
decade. It’s not that Perry was a worse “education governor” than Bush (or, for
that matter, Ann Richards) before him, but that he presided over an
accountability strategy that was running out of steam. Like the meteor that set
in place a new state of equilibrium for our Earth, consequential accountability
shocked our education system and improved math scores, but that system has now
begun to settle into a new stasis. So sayeth the stats stud.
It’s an intriguing argument, and one that deserves serious
consideration, even more so as the U.S. marks the tenth anniversary of the
enactment of NCLB and tries to figure out what the next version of that law
should entail. If school-level accountability, as currently practiced, is no
longer an effective lever for raising student achievement, then what is? If we
need another “meteor” to disrupt the system, where should we look? Mark
suggests that the Common Core and rigorous teacher evaluations have potential.
We also see promise in the digital-learning revolution. But other shocks to the
system might work even better. What are they?
Interested in learning more—or just want to keep the
conversation going? Attend our event, “Has the Accountability Movement Run its
Course,” at Fordham from 8:30 to 10:00AM on January 5, 2012. More
information can be found here.
|Click to listen to commentary on the "accountability plateau" from the Education Gadfly Show podcast.
Opinion: In praise of performance pay—for online-learning companies
you consider this week’s New York Times article on K12.com a “hit
piece” (Tom Vander Ark) or a “blockbuster”
(Dana Goldstein), there’s little doubt that it (and the recent
crop of other pieces like it) will have a long-term impact on the debate
around digital learning. Polls show
that the public in general and parents in particular are leery of cyber
schools, and this kind of media attention (sure to be mimicked in local papers)
will only make them more so.
just as these criticisms aren’t going away, neither is online learning. That
genie is out of the bottle. So how can we go about drafting policies that will
push digital learning in the direction of quality?
something we at Fordham are busy pondering. To that end, we’ve published
three papers (so far) in our series Creating Sound Policy for Digital Learning: Rick Hess on quality
control; Paul Hill on funding;
and Bryan and Emily Hassel on teachers.
And in January, we’ll publish an analysis by the Parthenon Group of what
high-quality fulltime online learning really costs.
aside for a moment the Times’s extremely selective use of data, expert
opinion, and evidence. Where the article landed a punch, in my view, was around
the perverse incentives at play today. Clearly K12, and its well-paid CEO, Ron
Packard, face strong incentives to boost enrollment at their schools.
Unfortunately, states haven’t figured out a way to create similar incentives
around quality. And that needs to change.
a short digression. I worked at K12 many moons ago, just after its creation. (I
believe I was employee number ten.) I needed a job, and I convinced Bill
Bennett to create a role for me in which I would figure out how to take K12’s
rich resources and make them available for poor kids. (After-school tutoring
programs seemed to be particularly promising venues.) Our basic assumption was
that K12’s model—which relied on parents or other caretakers doing most of the
instruction—wouldn’t be feasible for kids living in poverty, most of whom would
need the custodial care offered by traditional public schools.
honest, I didn’t make much progress. The learning materials weren’t even
created yet, and so I had few “partnerships” to offer to communities; I left
after nine months.
what a difference a decade makes. One of the real surprises of the online-learning
movement is that lots of poor families are choosing to give it a try, and that
explains (to a large degree) why K12’s test scores are lagging. (Yes, poverty
and achievement are linked, at least for now.)
Schools of all kinds should only get paid for the days of instruction that kids actually show up (or log in) for.
impression painted by the Times article is that online education
companies like K12 have every reason to sign up as many parents as
possible—poor, rich, whatever—regardless of how prepared they are to tackle the
challenge of home-based instruction. Because of some states’ sloppy finance
systems, the schools can keep the money if the families change their minds and
head back to traditional schools. And, as has been true for all public schools
since the beginning of time, the online schools get paid whether their students
are learning or not.
the payment problem is the easier challenge. Schools of all kinds should only
get paid for the days of instruction that kids actually show up (or log in) for.
But is it time to consider performance-based funding, too? To pay companies
like K12 more or less depending on how their students perform on state tests or
depending on their graduation rates?
In his paper
for Fordham, Paul Hill dismisses the idea, arguing that:
for performance would create a harsh environment for all education providers.
Conventional, virtual, and hybrid schools might spend money on a student’s
instruction for a whole course or semester yet receive nothing in return.
Online vendors of all kinds, who have little control over their students’
effort or persistence, could be even more at risk. In general, this approach
would limit the unproductive use of public funds and quickly destroy any vendor
that could not demonstrate good results. It would favor providers with deep
pockets, e.g., district-run schools and online vendors supported by large
foundations. Performance-based payment as defined here could create a lethal
environment for smaller-scale innovators.
probably right about smaller-scale innovators, but I still think it’s worth a
try, at least for full-time online schools. (It might be harder in the “blended
learning” setting, where a child might be taking just one or two subjects
K12 only got paid for every student that made at least a year’s worth of
progress on the state test? Some argue that this would create its own perverse
incentives, encouraging the company to cherry pick students who are most likely
to succeed. But if the measure is student growth, and if the test being used is
a good one (the latter being a big if, admittedly), then all kids but those
with severe cognitive disabilities should be seen as contenders.
of signing up students willy-nilly, K12 would then have a reason to vet each
family’s situation to make sure they are ready
for the rigors of online learning. They would invest, up-front, in
assessing whether the child’s parents or other caretakers are up to the task of
instructing the student, and whether they have a home situation conducive to
success. And then K12 would work like the dickens to make sure every student makes
strong progress over the course of the year.
I’d like to see performance-based pay for all schools. That may not fly anytime
soon, but performance pay for online learning (at the least the full-time,
virtual-charter-school version) could. Which state is ready to give it a try?
This piece originally
appeared (in a slightly different form) on Fordham’s Flypaper blog. To subscribe to Flypaper, click here.
|Click to listen to commentary on digital learning from the Education Gadfly Show podcast.
News Analysis: It's not voodoo
2005’s hurricane catalyzed one of the largest
governance experiments in American education to date, as Louisiana implemented
its Recovery School District law under which it took responsibility for the
worst schools in the Big Easy (and a few others throughout the Bayou State).
While other state-takeover initiatives have seen mixed results, Louisiana’s push has yielded big upticks in student-test scores. Two reasons why
Louisiana’s initiative has fared well: It doesn’t get bogged down in the
schools’ day-to-day operations. (It offloads that responsibility on school
leaders—where it belongs.) And it scraps the current edu-governance system (no
more school boards, locally elected or otherwise), giving site management over to
charter networks and other external providers. And the idea has some converts:
its Education Achievement System) and Tennessee both recently announced the
creation of their own “recovery school districts” (though both remain in the
pilot stage). This slowly widening movement holds much promise: States can
offer management know-how and dedicated resources and can skirt district contracts that stymie creative school
models—without getting bogged down in local politics or bureaucracy. Successful
state takeovers of failing districts are elusive—often written off (including by us) as a lost cause. But this 2.0 model sure
|Click to listen to commentary on the state-led districts from the Education Gadfly Show podcast.
News Analysis: The monolith shifts
Seven days ago, the National Education Association
(NEA)—long dormant in matters of education reform—began to stir. The nation’s
largest teacher union unveiled “fourteen points” to promote teacher effectiveness last
Thursday. Some of them we’ve heard before (the NEA has long endorsed
teacher-residency and “peer assistance and review” programs, for example). But
many are worthy new ideas—new, at least, to the NEA. For prioritizing these,
the union should be commended. (Gadfly readers might find the appeal for a
career ladder for teachers, with differentiated pay and responsibility, to be a
reasonably mainstream idea, but remember who’s doing the talking here.) To be
sure, old-school NEA thought does seep into the reform plan in places: While
it’s a good notion to disallow inexperienced teachers from leading the
classrooms of our neediest students, the back-handed knock at Teach For America
inherent in this recommendation is unnecessary. Further, the union’s avoidance
of linking teacher evaluations to additional teacher compensation is
short-sighted. Overall, though, nice start, NEA. The question now looms: How
will you turn these recommendations into reality? Might we offer a suggestion?
Find a partner district—we think Columbus might do the trick—and get to work
piloting these initiatives, lest the worthwhile ideas set forth by the
committee remain just so (as they did with last summer’s promising NCATE
announcement calling for revamped education-school models.)
Review: All Over the Map: Comparing States’ Expectations for Student Performance in Science
Parents, be aware: The “proficient” designation
that your child received on her state’s science test may not signify much. This
new report from Change the Equation (a STEM-advocacy outfit) and the American
Institutes for Research evaluates the proficiency cut scores of thirty-seven
states’ eighth-grade science assessments, comparing their rigor to that of the
2009 National Assessment of Educational Progress. The upshot? Fifteen states
set their bars for proficient below NAEP’s basic
designation. Virginia is the worst of the lot—setting its cut scores far below
the rest of the pack—and repaying itself with a 91 percent proficiency rate on
its state exam. Only four states (New Hampshire, Rhode Island, Massachusetts,
and Louisiana) expect their students to be at or above NAEP’s proficient
threshold. (Feeling a bit of déjà vu? This report is a lot like Fordham’s own Proficiency
Illusion blockbuster from 2007, in which we drew similar conclusions
about reading and math.) A word on the forthcoming common science standards, this
work is necessary—and hugely important. But, as we are reminded time and again,
it is not sufficient. A failure to link quality standards to rigorous assessments
with balanced cut scores is akin to swiping the legs of any common science-standards
initiative, just as it’s learning to walk.
|Click to listen to commentary on this science "proficiency illusion" from the Education Gadfly Show podcast.
Review: "Multiplication is for White People": Raising Expectations for Other People’s Children
In this book, MacArthur “genius” Lisa Delpit
offers an interesting follow-up to her acclaimed Other People’s Children, tackling the continuing challenge of
boosting minority student achievement. Using innumerous anecdotes and the
occasional data point, Delpit weaves through the complexities of race, class,
and culture in America’s schools—and society. In the end, she finds a racial “expectation
gap” that pervades our present system. To counter it, educators must develop a “no
excuses” attitude (though not necessarily the KIPP-like model of how to implement
it), and fight the “responses to oppression” that foster chronic
underachievement. The read is quick and enjoyable, and she covers a number of
issues, from malnutrition myths to stereotyping to the squishy meaning of
“basic skills.” While we don’t always agree on the means of reaching the end,
we can definitely get behind Delpit when she says “There is no simple recipe,
and the only real solution is for humans who care…to confer, collaborate,
argue, ponder, and act to fashion a space for real dialogue and understanding.”
Educators and reformers alike would be wise to give this book a look (it’s now
available on pre-order)—Delpit adds grounding, and some color, to a discussion
that is often arid and unproductive.
Review: Creating a New Public Pension System
Few issues as serious as the pension crunch are equally
as dull. Addressing unfunded liabilities and implementing defined-contribution
plans simply aren’t compelling calls to arms, despite the widening consensus
that the balance sheets of public-sector retirement-benefit systems pose grave threats to state budgets. That’s why the
clarity and concision found in this recent “solution paper,” penned by Josh
McGee for the Laura and John Arnold Foundation, are so valuable. The piece may be
light on detail, but that’s part of the point: It doesn’t aspire to wonky
analysis. Instead, it aims right at policymakers and the public in explaining
why the set payouts of the traditional defined-benefit (DB) retirement-benefit
structure are unsustainable. McGee efficiently makes the case that
irresponsible pols inevitably underfund DBs, explains the challenges in
projecting their costs, and lays out how they incentivize expensive (and
counterproductive) employee behaviors. He then outlines the major cost-saving
alternatives on the table—including defined-contributions, cash balance plans,
and “stacked hybrid.” (OK, it’s just a little wonky.) There’s far more to this
complex topic than McGee includes in this brief paper (case in point: Fordham’s
recent study of successful pension reforms), but as an accessible
introduction to a vital issue, it’s hard to beat.
From The Web
The Education Gadfly Show Podcast: Wanna make a bet?
Mike and Daniela go edu-meta, asking whether the
accountability era has run its course, what the role of for-profits are in
digital education, and how state-run districts and schools may reshape
governance. Amber investigates the science “proficiency illusion” and Chris
channels the Grinch.
Click to listen to the podcast on our website. You can also download the podcast here or subscribe on iTunes here.
Flypaper's Finest: Schools of choice need to be schools of quality
E. Finn, Jr.
An updated and expanded version of this piece, written by Checker and Fordham's Terry Ryan, appeared in today's Akron Beacon Journal.
news that White Hat Management, the big, Ohio-based, profit-seeking
charter-school operator, faces financial problems surely was received as an
early Christmas present by many longtime charter opponents, particularly within
the Buckeye State.
The company’s founder and leader, Akron industrialist
David Brennan, has been a larger-than-life target for school-choice foes since
Gov. George V. Voinovich appointed him in 1992 to head a commission intended to
advance choice in Ohio kindergarten-through-12th grade education.…
Click to read the rest on Flypaper.
Flypaper's Finest: Building a test worth teaching to
“Believing we can improve schooling with more
tests,” Robert Schaeffer of FairTest once argued, “is like believing you can
make yourself grow taller by measuring your height.”
It’s a great line.
Such statements are the seductive battle cries of the anti-standards and
anti-assessment crowd. But is there any reason behind this kind of rhetoric?
Parents rarely complain that their young babies are
being weighed and measured too much—even though it can create an extra burden
in an often stressful time in their lives.…
Click to read the rest on Flypaper.
Gadfly Studios: Education governance in the twenty-first century
Time and again, we see promising
education-reform programs and policies crash upon the shoals of failure, broken
apart by the jagged rocks of our current governance system. But what is to be
done about it? Watch the panels from our recent conference (Rethinking
Education Governance for the Twenty-First Century) for an outline of our
system’s key challenges, promising innovations, and bold paths forward. Watch
Briefly Noted: Eighty-two percent, you say?
a few months back when Arne Duncan dropped jaws (and elicited some guffaws)
with his proclamation that 82 percent of students would be listed as failing
under AYP this year? Turns out, the
real number is 48 percent, according to the Center on Education Policy. Way
to make a nearly 50 percent AYP failure rate seem not so bad, Duncan.
schools in the Buckeye State are reaching a “district-like” racial profile as more white students
enroll in them. Rick Kahlenberg would agree—there’s nothing wrong with that.
you remain unconvinced about the need to think about America’s “excellence gap”
in K-12 schooling—even after reading our High
Flyers report—then check out Sol
Stern’s latest from City Journal.
Jay Greene, you found us out: We’re not actually in the business of reforming
schools—this is all just a big ploy to support Checker’s “plan for a world government.” Bwahahaha.
We’re not so different, after all. Leaders in the
developing world are now wrestling with the question: “Should we switch focus from improving schools to improving
parents?” It’s worth keeping an eye on what they decide.
Announcement: Oh man-ager, does Fordham have the opportunity for you!
Christmas came early this year; control your
excitement. Fordham is on the hunt for a new research manager to keep our
projects pipeline flowing and our final reports engaging and rigorous. If you
have both qualitative and quantitative research skills and are smart, dedicated—and
just a little punchy—you may just be the right fit. Learn more about the
position, including how to apply, here.
Announcement: Start the New Year off right
For the Gadfly readers now resolving to be better
ed-policy wonks in 2012, here’s your first opportunity. Join us on January 5,
2012 from 8:30 to 10:00AM for our inaugural Year of the Dragon event. An
impressive set of panelists (Charles Barone, Eric Hanushek, Sandy Kress, and
Marc Schneider) will answer the question “has the accountability movement run
its course?”—spring-boarded by Schneider’s paper, The Accountability Plateau, released
here to learn more about the event.
Announcement: It's a win-win
Public Impact—the stellar research shop out of
Chapel Hill—is looking to turn theory into action. Through the group’s new
Opportunity Culture initiative, it’s looking to pilot twenty models that
schools can use to extend the reach of great teachers—and it’s in need of
districts ready to partner with it on this work. Interested in becoming an
implementation site? Find
out more here.
Announcement: A ‘Bright new opportunity
2012 will mark the first cohort of the Fulbright
Public Policy Fellowship (which focuses on education in other countries)—will
you rank among its members? The program places fellows in foreign ministries or
institutions in the developing world, allowing them to gain hands-on
experience, while simultaneously carrying out their own research. Sound cool? Learn more here.
Fordham's featured publication: The Proficiency Illusion
A groundbreaking study published in 2007, The Proficiency Illusion revealed that
the tests that states use to measure academic progress under the No Child Left
Behind Act have created a false impression of success, as states set their
“proficiency” cut offs low, in order to meet NCLB mandates. The report, a
collaboration of the Thomas B. Fordham Institute and the Northwest Evaluation
Association, contains several major findings. Among them: Although there has
been no “race to the bottom,” the report did find a “walk to the middle,” as
some states with high standards saw their expectations drop toward the middle
of the pack. States are aiming particularly low at the elementary level and in
on to learn more.