Measuring Up? The Trouble with Debt to Degree

The following is a guest blog post by Robert Kelchen, graduate student in Educational Policy Studies at UW-Madison, and a frequent co-author of mine. --Sara

I was pleased to see the release of Education Sector’s report, “Debt to Degree: A New Way of Measuring College Success,” by Kevin Carey and Erin Dillon. They created a new measure, a “borrowing to credential ratio,” which divides the total amount of borrowing by the number of degrees or credentials awarded. Their focus on institutional productivity and dedication to methodological transparency (their data are made easily accessible on the Education Sector’s website) are certainly commendable.

That said, I have several concerns with their report. I will focus on two key points, both of which pertain to how this approach would affect the measurement of performance for 2-year and 4-year not-for-profit (public and private) colleges and universities. My comments are based on an analysis in which I merged IPEDS data with the Education Sector data to analyze additional measures; my final sample consists of 2,654 institutions.

Point 1: Use of the suggested "borrowing to credential" ratio has the potential to reduce college access for low-income students.

The authors rightly mention that flagship public and elite private institutions appear successful on this metric because they have a lower percentage of financially needy students and more institutional resources (thus reducing the incidence of borrowing). The high-performing institutions also enroll students who are easier to graduate (e.g. those with higher entering test scores, better academic preparation, etc) which increases the denominator in the borrowing to credential ratio.

Specifically, the correlations between the percentage of Pell Grant recipients (average of 2007-08 and 2008-09 academic years from IPEDS) and the borrowing to credential ratio is 0.455 for public 4-year and 0.479 for private 4-year institutions, compared to 0.158 for 2-year institutions. This means that the more Pell recipients an institution enrolls, the worse it performs on this ratio.

While even though Carey and Dillon focus on comparing similar institutions in their report (for example, Iowa State and Florida State), it is very likely that in real life (e.g. the policy world) the data will be used to compare dissimilar institutions. The expected unintended consequence is “cream skimming,” in which institutions have incentives to enroll either high-income students or low-income students with a very high likelihood of graduation. (Sara and I have previously raised concerns about “cream skimming” with Pell Grant recipients in other work.)

The graphs below further illustrate the relationship between the percentage of Pell recipients and the borrowing to credential ratio for each of the three sectors.

There is also a fairly strong relationship between a university’s endowment (per full-time equivalent student) and the average borrowing to credential ratio. Among public 4-year universities, the correlation between per-student endowment and the borrowing to credential ratio is -.134, suggesting that institutions with higher endowments tend to have lower borrowing to credential ratios. The relationship at private four-year universities is even stronger, with a correlation of -.346. For example, Princeton, Cooper Union, Caltech, Ponoma, and Harvard are all in the top 15 for lowest borrowing to credential ratios.

The relationship between borrowing to credential ratios and standardized test scores is even stronger. The correlations for four-year public and private universities are -.488 and -.589, respectively. This suggests that low borrowing to credential ratios are in part a function of student inputs, not just factors within an institution’s control. In other words, the metric does not solely measure college performance.

It is critical to note that the average borrowing to credential ratio should be lower at institutions with more financial resources and who enroll more students who can afford to attend college without borrowing. However, institutions who enroll a large percentage of Pell recipients should not be let off the hook for their borrowing to credential ratios. These two examples highlight the importance of input-adjusted comparisons, in which statistical adjustments are used so institutions can be compared based more than their value-added than their initial level of resources. The authors should be vigilant to make sure their work gets used in input-adjusted comparisons rather than unadjusted comparisons. Otherwise, institutions with fewer resources will be much more likely to be punished for their actions even if they are successfully graduating students with relatively low levels of debt.


Point 2: The IPEDS classification of two-year versus four-year institutions does not necessarily reflect a college’s primary mission.

IPEDS classifies a college as a 4-year institution if it offers at least one bachelor’s degree program, even if the vast majority of students are enrolled in 2-year programs. Think of Miami Dade College, where more than 97% of students are in 2-year programs but the institution is classified as a 4-year institution.

For the purposes of calculating a borrowing to credential ratio, the Carnegie basic classification system is more appropriate. Under that system an institution is classified as an associate’s college if bachelor’s degrees make up less than ten percent of all undergraduate credentials. The Education Sector report classifies 60 institutions as four-year colleges that are Carnegie associate’s institutions.

This classification decision has important ramifications for the borrowing to credential comparisons. The average borrowing to credential ratio by sector is as follows:

Two-year colleges, Carnegie associate’s: $6,579 (n=942)
Four-year colleges, Carnegie associate’s: $13,563 (n=60)
Four-year colleges, Carnegie bachelor’s or above: $23,166 (n=1,421)

Ten of the twelve and 20 of the top 40 four-year colleges with the lowest borrowing to credential ratios are classified as Carnegie associate’s institutions. For example, Madison Area Technical College is 54th on the Education Sector’s list of four-year colleges, but is 564th of 1,002 associate’s-granting institutions. These two-year institutions with a small number of bachelor’s degree offerings should either be placed with the other two-year institutions or in a separate category. Otherwise, anyone who wishes to rank institutions based on their classification would be comparing apples to oranges.

In conclusion: the effort in this report to measure institutional performance is a laudable one. But the development and use of metrics is challenging precisely because of their potential for misuse and unintended consequences. Refining the proposed metrics as described above may make them more useful.

It Rhymes With 'Tool'

UPDATED, 8/11/2011, 1:10 pm



Thursday morning in Washington DC -- the only city that could host such a vacuous, inane event -- the Thomas B. Fordham Institute is hosting (the hopefully one-off) "Education Reform Idol." The event has nothing to do with recognizing states that get the best results for children or those that have achieved demonstrated results from education policies over time -- but simply those that have passed pet reforms over the past year.



It purports to determine which state is the "reformiest" (I kid you not) with the only contenders being Florida, Illinois, Indiana, Ohio and Wisconsin and the only judges being: (1) a representative of the pro-privatization Walton (WalMart) Family Foundation; (2) the Walton-funded, public education hater Jeanne Allen; and (3) the "Fox News honorary Juan Williams chair" provided to the out-voted Richard Lee Colvin from Education Sector.



With the deck stacked like that, Illinois is out of the running immediately because its reforms were passed in partnership with teachers' unions. Plus it has a Democratic governor. Tssk, tssk. That's too bad, because Illinois represents the most balanced approach to education and teaching policy of the five states over the past year. And the absence of a state like Massachusetts from the running is insane. It has the best NAEP scores of any state and has a long track record of education results from raising standards and expectations, not by attacking teachers or privatizing our schools. But that's not the point here, of course. This is ALL politics. [UPDATE 8/11/2011: Yes, all politics. Mike Petrilli of Fordham says that "the lesson of Education Reform Idol" is --- ba-ba-ba-baaah ... ELECT REPUBLICANS. "When Republicans take power, reforms take flight."]



So I digress.... The coup de grace of ridiculousness for me is the inclusion of Wisconsin among the list of "contenders." What exactly has Scott Walker and his league of zombies actually accomplished for education over the last seven-and-a-half months OTHER THAN eliminating collective bargaining rights, a historic slashing of state school aids, and a purely political expansion of the inefficacious school voucher program?



What's even worse than the inclusion of Wisconsin among the nominees is the case made by Scott Walker's office for the 'reformiest' award. As a policy advisor to the former Wisconsin governor, I am amazed by the brazenness and spin from Walker's office. I would expect nothing less from a political campaign. But someone's gotta tell these folks that while they theoretically represent the public trust, the content of their arguments suggests we can't trust them as far as we can throw them. And here in cheese curd land, that ain't very far.



A quick look at Walker's argument reveals an upfront invocation of Tommy Thompson (Wisconsin's version of Ronald Reagan) to pluck at Badgers' heart strings and make them long for the good old days of the 1990s (when the rich paid their fair share in taxes). It is soon followed by the refuted and refuted claims that Walker's deep education cuts "protect students in the short term" and give districts "tools" to manage the fiscal slaughter. Just read the well-respected Milwaukee school superintendent's opinion of such "tools." Then there's this gem: "Districts immediately began to set aside more time for teacher collaboration as well as money for merit pay." I'd LOVE to see the data behind this claim because as I am aware there is no state survey that measures collaborative time for teachers for starters. Walker's staff probably lifted it from a single school district's claims detailed in this Milwaukee Journal-Sentinel story -- claims trumpeted by dozens upon dozens of right-wing bloggers such as Wisconsin's own Ann Althouse -- claims which since have been exposed as "literally unbelievable".



The irony is that this event is taking place in DC just two days after the recall elections of six seemingly vulnerable, incumbent Republican state senators. The repudiation of Walker's slash-and-burn policies will be testament enough to the destructiveness of his leadership both for public education and for the Badger State as a whole. In Wisconsin, recall would appear to be a far more effective 'tool' than the tools tentatively running the show under the Golden Dome in Madison.



[UPDATE 8/11/2011: For anyone who cares ... Indiana apparently is the "reformiest" state. By reformers' preferred metrics, I believe this means that Indiana will have the top NAEP scores in the nation next time 'round. Right?]





Image courtesy of Democurmudgeon

A View from the Right in a Left-Leaning Tower

What follows is a GUEST POST by University of Wisconsin-Madison graduate student Robert Kelchen. I have had the privilege of working with Robert since 2008; we have co-authored two articles, including this one on the effects of financial aid. Upon reading John Tierney's take on the dominance of liberals in academe, I asked Robert for his thoughts-- and here they are. SGR

************

My name is Robert Kelchen, but many students and faculty who know me at the University of Wisconsin-Madison often introduce me as "the conservative guy" or "my Republican friend." I am used to this sort of introduction after being in Madison for four years; after all, I can count the number of conservative or libertarian doctoral students who I know on two hands. I have been told several times in the past by fellow students that I am the first right-leaning person with whom they have ever interacted on a regular basis. Prior to the passage of Act 10 (the law that restricted collective bargaining), I was one of the few students at the university to request a refund of the portion of the Teaching Assistants' Association dues that went toward political or ideological activities. This also meant that I had to give up my right to vote on issues germane to collective bargaining (the primary purpose of the union), but it was a sacrifice that I was willing to make. During the protests at the Capitol throughout the spring semester, I did my best to stay out of the fray and keep very quiet about my personal opinions.

Sara asked me for my thoughts on the recent New York Times article about why there are so few conservative students in graduate school. I had to consider the offer for a while, as making this post would make my political leanings more publicly known and could potentially affect my chances of getting a job in two years. However, I just could not pass up the opportunity to comment on this article in the newspaper of record for American liberals--and the same paper that ran a front-page article about Sara being one of a new generation of less politically-oriented professors.

My initial reaction to the article was to try to think of a conservative or libertarian professor in the School of Education at UW-Madison. To the best of my knowledge, there are no professors in the entire school, let alone my home department (Educational Policy Studies) who publicly identify as being right of center. However, this does not mean that there are no conservative faculty. A likely explanation is that faculty (and students) who do not identify with the liberal majority stay quiet about their political beliefs. The reaction of the majority of the faculty and graduate students during recent political events makes speaking out as a conservative a lonely proposition. It also means that there must exist other "elite" institutions that have a higher proportion of conservative faculty.

I do not put any stock in the Gross et al experiment mentioned in the article, which sent out letters asking for information about top graduate schools and included whether a fictional student worked for the Obama or McCain campaigns. Working on a presidential campaign does tell something about a student's political beliefs, but a student's GRE score and college performance (in addition to ability to pay) matter much more than that information. Additionally, the study only used male "prospective" applicants, a potentially serious limitation. (Not to mention that John McCain is a fairly liberal Republican who partnered with ex-Senator--and Madison hero--Russ Feingold on campaign finance reform. He is much more palatable to the left than someone like Michelle Bachmann.)

This leaves several possible explanations for why conservative students are less likely to go to graduate school and stay in academia later in life than liberals. A potential explanation mentioned in the article (and is echoed by several of the comments on the article) is that conservatives do not have the mental abilities to go to graduate school. That is entirely bogus, as noted in the article. I do not put much stock into the hypothesis that conservatives are less likely to be in academia due to discrimination on the acceptance (graduate students) or hiring (faculty) side, although this very well may be true in isolated institutions and departments.

The argument of self-selection, in which conservatives choose not to pursue a career in higher education, is the likely culprit for why I know only one other conservative graduate student in the entire School of Education. Much self-selection occurs because of how attending graduate school delays one's ability to make a reasonable salary. In "red" states, adults are more likely to get married at a younger age than those in "blue" states; the need to support a family can detract both women and men from spending an additional six or more years in school. The claim made by Peter Wood from the conservative National Association of Scholars, that conservatives choose not to pursue a graduate degree because of the perception of liberal bias, is likely responsible for part of the attendance gap. I would say that, holding all other factors constant, it is easier to be a majority liberal than a minority conservative. However, the common perception that conservatives know all other Republicans in the area or that we're always expected to engage in political discussions at the drop of a hat (or that we agree with everything that Sarah Palin says) probably do not cause many students to shun away from graduate school. The perception of liberal bias likely drives away many more students than the actual amount of liberal bias.

In closing, I would like to thank Sara again for the opportunity to post my thoughts. Next time you talk with a conservative, please realize that we are not bad people because we have different political viewpoints. Most of us, regardless of ideology or partisan affiliation, believe in the importance of public education even though we disagree on the best ways to improve the current system.

Robert

Caring for the Me Generation

During the past semester, a time where I constantly felt split between my academic life and my civic life, I became acutely aware of an attitude among undergraduates that perplexed me. I tried writing about it , describing what readers pointed out (in a far more articulate manner than I'd managed) was a notable lack of empathy among some students.

Since I've spent the last 10 years trying to make convince higher education institutions to prioritize their students' needs and desires, these realizations about who some of the students seemed to be and especially what they seemed to believe, made me pretty depressed. Don't get me wrong: it's not that I expect students to speak and act in one voice--far from it, given how much I value the democratic process. I don't want them to share my opinions or perspectives, but rather simply want them to formulate opinions and perspectives after asking good questions and gathering and evaluating information. But what I hope for, most of all, is their recognition that they are part of a worldwide community of students, and their strength lies in that community. I hope that such a larger sense of the world will guide them to think of more than themselves, and to act for the greater good.

Of course, what I learned from social media engagement this spring is what the Me Generation is really all about. "Me."

As it turns out, this is not at all a Wisconsin phenomenon. There's rigorous research from the University of Michigan demonstrating a sharp decline in empathy among undergraduates, based on data from 14,000 students over 30 years. Compared to students who attended college 20-30 years ago, undergraduates in the first decade of the new millenium scored 40% lower in empathy. Said one of the study's authors, this group is among the "most self-centered, narcissistic, competitive, confident and individualistic in recent history."

The authors speculate as to the root causes, citing among other things the influence of media and social media in particular. But those are worldwide phenomena, and this is a U.S. study (I strongly suspect such trends aren't felt in many other countries). Instead, I'm betting that we have Ronald Reagan to thank. The undergraduates of the 1970s and 1980s were raised by parents who came of age under the New Deal, during times when social justice and civil rights for all were demanded and (to some degree) received. They were more often raised to appreciate the luck and good fortune that gave opportunities to them, and worked to share those opportunities with others. Not so for the undergraduates of the 2000s, whose parents came into adulthood under the Gipper, a period in which inequality blossomed, and consumption was conspicuous. They've never known a time when college wasn't insanely expensive, always assumed that the American Dream was only about individual effort, and they were listening as even the Democrats placed all of the blame for poverty at the feet of the poor (yes, I'm looking at you Bill Clinton).

Reversing this trend is absolutely necessary for ensuring the well-being of people everywhere. As the Michigan researchers noted, what accompanies an exclusive emphasis on oneself is a "corresponding devaluation of others." Such a condition tears at the web of our social life and creates conditions of anomie that increase the spread of poverty and perpetuate hatred and fear like that evidenced in recent events in Norway.

So, start now. Take this quiz created by the Michigan researchers and see how YOU compare to those undergrads they surveyed. Then decide what to do with your results.


ps. In case you are curious, I scored a 59/70, meaning more empathetic than 80% of the study's participants. Thanks Poppa.

Anger Management

I am appalled by this malicious attack on teachers and teachers' unions by Jay Greene. He claims that teachers are engaging in mob-like behavior, are seething anger and are intimidating politicians. The irony is that I've met few teachers who are nearly as angry as Jay himself comes across.

But when the public face of the teacher unions is the Army of Angry Teachers, they no longer seem like Mary Poppins and begin to look a lot more like longshoremen beating their opponents with metal pipes.

Giant mobs of yelling protesters and blogs filled with tirades may increase the intimidation politicians feel, but it seriously undermines the image of teachers as an extension of our family.

Jay's "mob" is my "democratic gathering". Here in Wisconsin (the featured photo on Jay's blog post) there was an organic outpouring of disgust and determination as a result of Governor Scott Walker's attacks on collective bargaining and public employee and teachers unions -- and his decisions to balance the state budget on the backs of public workers and by gutting public education while steering tax breaks to corporations and providing massive funding increases to voucher schools.

Jay is mad that teachers are mad, but they have every right to be, especially in a state like Wisconsin. Have you visited Wisconsin in the past six months, Mr. Greene? Have you actually talked to teachers here? Have you seen and heard the thousands and thousands of protesters that have no vested or financial interest that nonetheless turned out en masse to speak out on behalf of others? (Clearly, these are rhetorical questions.)

This *is* what democracy looks like. The allowance of such an outpouring of opposition is why our nation was founded. Apparently, Jay's preferred answer to the Palin-esque question of "How's that redress of grievances thing workin' out for ya?" would be "It should not be allowed."

Wisconsin teachers have not and should not lie down and take the beating they've received here. Their right to bargain has been stripped. They've seen massive cuts to their pay and benefits. They're now working in public school systems that have had resources sucked out of them. They're standing up for their rights and for a far different state of Wisconsin than has emerged under the leadership of Governor Walker and his legislative Rubber Stamps.

Have teachers and their unions always advocated for and prioritized the best educational policies? Sure they haven't. Has any one education group or interest? (Greene's free market approach to education certainly doesn't represent sound policy.) Reforms can only succeed when teachers are full partners in their creation and implementation. And I will fight for the right of their voices to be heard in policy debates, in schools, and, yes, at the bargaining table.

It seems that Mr. Greene would prefer that teachers simply shut up.

The New Normal

Too many Americans appear willing to accept the hand they're dealt. Most shockingly, many of our political, educational and civic leaders seem to have fallen into the same trap. "The New Normal," they call it. Like Death and Taxes. Etched in stone. Undefined, yet not re-definable. Inevitable.

Fortunately, there are those among us willing to demand a new deck of cards -- and a new dealer!

We've seen the rise of the Forces For Fairness in states like Wisconsin where there is no disguising the unsubtle, in-your-face, anti-democratic, vitriolic, bought-and-paid-for policies of Governor Scott Walker, the Brothers Fitzgerald, ALEC, the Koch Brothers and their yes men and women (even the few remaining Republican moderates - if they still can be called such - who should know better). In the Badger State, tens of thousands took to the streets of Madison and are now actively participating in recall efforts to change the equation and prevent Wisconsin from being turned into a place totally unrecognizable.

Nationally, I see a rising consciousness and an emerging consensus that congressional Republicans have one-upped their Gingrichian colleagues from the 1990s in overreaching on fiscal matters. Voters do not like the draconian cuts being pushed through by House Republicans, the intransigence and obstructionism practiced as a religion by Senate Republicans, the GOP's willingness to hold America's bond rating and our economic recovery hostage by refusing to raise the debt ceiling, and an adherence to a baseless and extremist anti-tax philosophy. In a recent CBS News poll, 71 percent of Americans are opposed to the way the Republicans are approaching the debt limit debate. As well they should be.

Americans are NOT opposed to raising taxes on the wealthy to address our national debt. A recent Reuters poll found that 52 percent of Americans believed that "a combination of spending cuts and tax increases was the best strategy to reduce deficits." Republicans are so constrained by anti-tax pledges that they even believe a repeal of ANY tax cut or the closing of ANY tax loophole (even those for corporate jet owners!) would result in Grover Norquist gagging them with a mouthful of tea bags and ordering them to a permanent political purgatory.

If more Democrats had shown the courage to stand up sooner and establish the terms of the debate, this emerging consensus could have been precipitated. The likes of Vermont's Bernie Sanders have had it right for some time in the call for "shared sacrifice." Others, including President Obama, appear to be catching up to the reality that was evident to Sanders and other Progressives: Congressional Republicans are economic extremists willing to drive the American economy into the ground in order to assuage the Anti-Tax God (don't let it go to your head, Mr. Norquist).

"The Rock and the Hard Place on the Deficit", an op-ed in last Sunday's New York Times, written by Christina Romer, is one of the best articles I've read that puts the substance of this issue into context. For the benefit of you non-Times subscribers, here are some key highlights:

The economic evidence doesn’t support the anti-tax view. Both tax increases and spending cuts will tend to slow the recovery in the near term, but spending cuts will likely slow it more. Over the longer term, sensible tax increases will probably do less damage to economic growth and productivity than cuts in government investment.
...

There is a basic reason why government spending changes probably have a larger short-term impact than tax changes. When a household’s tax bill rises by, say, $100, that household typically pays for part of that increase by reducing its savings. Its spending tends to fall by less than $100. But when the government cuts spending by $100, overall demand goes down by that full amount.

Wealthier households typically pay for more of a tax increase out of savings, and so they reduce their spending less than ordinary households. This implies that tax increases on wealthy households probably have less effect on the economy than those on the poor or the middle class.

All of this argues against any form of fiscal austerity just now. Even some deficit hawks warn that immediate tax increases or spending cuts could push the economy back into recession. Far better to pass a plan that phases in spending cuts or tax increases over time.

But if federal policy makers do decide to reduce the deficit immediately, reducing spending alone would probably be the most damaging to the recovery. Raising taxes for the wealthy would be least likely to reduce overall demand and raise unemployment.

The politics behind this issue is another matter. But it has huge implications for issues like education. Too many educational advocates, policy types, and yes, even elected leaders seem all too willing to accept "The New Normal" -- and even pontificate about it -- as opposed to fight for a new deal and attempt to redefine the debate. President Obama too often appears to allow congressional Republicans to define the terms of the conversation, such as tying long-term deficit reduction to the debt ceiling, as Robert Reich noted over Twitter yesterday.

There's a time and a place for acknowledging political realities and accepting half a loaf. The problem is we've entered the second coming of the Robber Barons where the rich are hoarding their loaves of bread and too many Americans aren't getting a chance to get their hands in the dough at all. Until we address the historic economic inequality in this country and put spending power back in the pockets of working families, there is a tremendous likelihood that the economy will never fully recover. Never. That requires us -- and our elected leaders -- to speak out and act.

The time is now. Reality is what we make it. More of us have got to be willing to step up and say, "Enough!" I've witnessed Democrats and independents get energized in Wisconsin. We need a similar dynamic to take hold nationally. My guess is that it will build in time. But will it be enough to change the equation?

The forces of fiscal lunacy had better listen to the American people now or my guess is that they'll be hearing from the silent majority of sensible Americans at the ballot boxes in 2012 -- and even sooner in states like Wisconsin. If the "Republican Revolution" in the 1990s is any signal, past is prologue.

Affordability and Attainment in Wisconsin Public Higher Education

As I noted in a prior post, last week the Wisconsin Scholars Longitudinal Study (which I co-direct) hosted a statewide conference on the issues of affordability and attainment in Wisconsin Public Higher Education.

The research released became part of this morning's UW Regents discussion (start around 1:03).

In case you missed the event, which was attended by more than 150 leaders from all over Wisconsin, you can watch most of it on Wisconsin Eye. The main presentation of findings is here (see part1).

We will also be posting conference materials on the WSLS website soon.